research
∙
03/07/2022
Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer
Hyperparameter (HP) tuning in deep learning is an expensive process, pro...
share
research
∙
09/28/2021
Bit Complexity of Jordan Normal Form and Spectral Factorization
We study the bit complexity of two related fundamental computational pro...
research
∙
07/07/2021
Evaluating Large Language Models Trained on Code
We introduce Codex, a GPT language model fine-tuned on publicly availabl...
research
∙
10/28/2020
Scaling Laws for Autoregressive Generative Modeling
We identify empirical scaling laws for the cross-entropy loss in four do...
research
∙
05/28/2020
Language Models are Few-Shot Learners
Recent work has demonstrated substantial gains on many NLP tasks and ben...
research
∙
06/22/2019
Asymmetric Random Projections
Random projections (RP) are a popular tool for reducing dimensionality w...
research
∙
01/02/2018
The geometry of rank decompositions of matrix multiplication II: 3× 3 matrices
This is the second in a series of papers on rank decompositions of the m...
research
∙
11/30/2017