research
∙
05/27/2023
Fine-Tuning Language Models with Just Forward Passes
Fine-tuning language models (LMs) has yielded success on diverse downstr...
share
research
∙
05/18/2023
Smoothing the Landscape Boosts the Signal for SGD: Optimal Sample Complexity for Learning Single Index Models
We focus on the task of learning a single index model σ(w^⋆· x) with res...
research
∙
05/11/2023
Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks
One of the central questions in the theory of deep learning is to unders...
research
∙
09/30/2022
Self-Stabilization: The Implicit Bias of Gradient Descent at the Edge of Stability
Traditional analyses of gradient descent show that when the largest eige...
research
∙
06/30/2022
Neural Networks can Learn Representations with Gradient Descent
Significant theoretical work has established that in specific regimes, n...
research
∙
06/11/2021
Label Noise SGD Provably Prefers Flat Global Minimizers
In overparametrized models, the noise in stochastic gradient descent (SG...
research
∙
05/09/2018