research
∙
05/17/2023
Epsilon Sampling Rocks: Investigating Sampling Strategies for Minimum Bayes Risk Decoding for Machine Translation
Recent advances in machine translation (MT) have shown that Minimum Baye...
share
research
∙
02/19/2023
Scaling Laws for Multilingual Neural Machine Translation
In this work, we provide a large-scale empirical study of the scaling pr...
research
∙
02/09/2023
Binarized Neural Machine Translation
The rapid scaling of language models is motivating research using low-bi...
research
∙
09/23/2022
Do Current Multi-Task Optimization Methods in Deep Learning Even Help?
Recent research has proposed a series of specialized optimization algori...
research
∙
07/29/2022
Adaptive Gradient Methods at the Edge of Stability
Very little is known about the training dynamics of adaptive gradient me...
research
∙
02/04/2022
Data Scaling Laws in NMT: The Effect of Noise and Architecture
In this work, we study the effect of varying the architecture and traini...
research
∙
02/01/2022
Examining Scaling and Transfer of Language Model Architectures for Machine Translation
Natural language understanding and generation models follow one of the t...
research
∙
10/08/2021
A Loss Curvature Perspective on Training Instability in Deep Learning
In this work, we study the evolution of the loss Hessian across many cla...
research
∙
09/16/2021
Scaling Laws for Neural Machine Translation
We present an empirical study of scaling properties of encoder-decoder T...
research
∙
06/24/2020
When Do Neural Networks Outperform Kernel Methods?
For a certain scaling of the initialization of stochastic gradient desce...
research
∙
06/21/2019
Limitations of Lazy Training of Two-layers Neural Networks
We study the supervised learning problem under either of the following t...
research
∙
04/27/2019
Linearized two-layers neural networks in high dimension
We consider the problem of learning an unknown function f_ on the d-dime...
research
∙
01/29/2019
An Investigation into Neural Net Optimization via Hessian Eigenvalue Density
To understand the dynamics of optimization in deep neural networks, we d...
research
∙
10/17/2018
Optimal Covariance Estimation for Condition Number Loss in the Spiked Model
We study estimation of the covariance matrix under relative condition nu...
research
∙
02/02/2018