research
∙
02/12/2021
A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizes
Recently the LARS and LAMB optimizers have been proposed for training ne...
share
research
∙
10/30/2020
Identifying Exoplanets with Deep Learning. IV. Removing Stellar Activity Signals from Radial Velocity Measurements Using Neural Networks
Exoplanet detection with precise radial velocity (RV) observations is cu...
research
∙
10/11/2019
On Empirical Comparisons of Optimizers for Deep Learning
Selecting an optimizer is a central step in the contemporary deep learni...
research
∙
07/12/2019
Faster Neural Network Training with Data Echoing
In the twilight of Moore's law, GPUs and other specialized hardware acce...
research
∙
07/09/2019
Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Model
Increasing the batch size is a popular way to speed up neural network tr...
research
∙
11/08/2018
Measuring the Effects of Data Parallelism on Neural Network Training
Recent hardware developments have made unprecedented amounts of data par...
research
∙
06/12/2018