research
∙
07/31/2023
Lookbehind Optimizer: k steps back, 1 step forward
The Lookahead optimizer improves the training stability of deep neural n...
share
research
∙
07/18/2023
Promoting Exploration in Memory-Augmented Adam using Critical Momenta
Adaptive gradient-based optimizers, particularly Adam, have left their m...
research
∙
12/03/2022
CrossSplit: Mitigating Label Noise Memorization through Data Splitting
We approach the problem of improving robustness of deep learning algorit...
research
∙
09/19/2022
Lazy vs hasty: linearization in deep networks impacts learning schedule based on example difficulty
Among attempts at giving a theoretical account of the success of deep ne...
research
∙
06/02/2022
Expressiveness and Learnability: A Unifying View for Evaluating Self-Supervised Learning
We propose a unifying view to analyze the representation quality of self...
research
∙
02/10/2021
On the Regularity of Attention
Attention is a powerful component of modern neural networks across a wid...
research
∙
08/03/2020
Implicit Regularization in Deep Learning: A View from Function Space
We approach the problem of implicit regularization in deep learning from...
research
∙
07/06/2020
A Mathematical Theory of Attention
Attention is a powerful component of modern neural networks across a wid...
research
∙
10/19/2018
A Modern Take on the Bias-Variance Tradeoff in Neural Networks
We revisit the bias-variance tradeoff for neural networks in light of mo...
research
∙
06/22/2018
On the Spectral Bias of Deep Neural Networks
It is well known that over-parametrized deep neural networks (DNNs) are ...
research
∙
01/12/2018
MINE: Mutual Information Neural Estimation
We argue that the estimation of the mutual information between high dime...
research
∙
01/12/2018