research
∙
08/06/2023
The Effect of SGD Batch Size on Autoencoder Learning: Sparsity, Sharpness, and Feature Learning
In this work, we investigate the dynamics of stochastic gradient descent...
share
research
∙
01/31/2023
The Power of External Memory in Increasing Predictive Model Capacity
One way of introducing sparsity into deep networks is by attaching an ex...
research
∙
01/30/2023
Alternating Updates for Efficient Transformers
It is well established that increasing scale in deep transformer network...
research
∙
07/23/2022
A Universal Trade-off Between the Model Size, Test Loss, and Training Loss of Linear Predictors
In this work we establish an algorithm and distribution independent non-...
research
∙
02/20/2022
Deconstructing Distributions: A Pointwise Framework of Learning
In machine learning, we traditionally evaluate the performance of a sing...
research
∙
11/13/2021
The Three Stages of Learning Dynamics in High-Dimensional Kernel Methods
To understand how deep learning works, it is crucial to understand the t...
research
∙
10/27/2019