research
∙
06/12/2023
Benchmarking Neural Network Training Algorithms
Training algorithms, broadly construed, are an essential part of every d...
share
research
∙
03/09/2023
Kernel Regression with Infinite-Width Neural Networks on Millions of Examples
Neural kernels have drastically increased performance on diverse and non...
research
∙
11/23/2022
Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Tasks
Bayesian deep learning seeks to equip deep neural networks with the abil...
research
∙
07/29/2022
Adaptive Gradient Methods at the Edge of Stability
Very little is known about the training dynamics of adaptive gradient me...
research
∙
07/07/2022
Pre-training helps Bayesian optimization too
Bayesian optimization (BO) has become a popular strategy for global opti...
research
∙
12/15/2021
Predicting the utility of search spaces for black-box optimization: a simple, budget-aware approach
Black box optimization requires specifying a search space to explore for...
research
∙
10/08/2021
A Loss Curvature Perspective on Training Instability in Deep Learning
In this work, we study the evolution of the loss Hessian across many cla...
research
∙
06/07/2021
Uncertainty Baselines: Benchmarks for Uncertainty Robustness in Deep Learning
High-quality estimates of uncertainty and robustness are crucial for num...
research
∙
02/12/2021
A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizes
Recently the LARS and LAMB optimizers have been proposed for training ne...
research
∙
11/06/2020
Underspecification Presents Challenges for Credibility in Modern Machine Learning
ML models often exhibit unexpectedly poor behavior when they are deploye...
research
∙
07/10/2020
Revisiting One-vs-All Classifiers for Predictive Uncertainty and Out-of-Distribution Detection in Neural Networks
Accurate estimation of predictive uncertainty in modern neural networks ...
research
∙
06/19/2020
Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift
Covariate shift has been shown to sharply degrade both predictive accura...
research
∙
10/11/2019
On Empirical Comparisons of Optimizers for Deep Learning
Selecting an optimizer is a central step in the contemporary deep learni...
research
∙
07/09/2019
Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Model
Increasing the batch size is a popular way to speed up neural network tr...
research
∙
06/06/2019
Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
Modern machine learning methods including deep learning have achieved gr...
research
∙
10/16/2018