research
∙
05/17/2023
DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining
The mixture proportions of pretraining data domains (e.g., Wikipedia, bo...
share
research
∙
12/13/2021
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts
Scaling language models with more data, compute and parameters has drive...
research
∙
11/19/2021
Combined Scaling for Zero-shot Transfer Learning
We present a combined scaling method called BASIC that achieves 85.7 zer...
research
∙
09/19/2021
Towards Zero-Label Language Learning
This paper explores zero-label learning in Natural Language Processing (...
research
∙
09/03/2021
Finetuned Language Models Are Zero-Shot Learners
This paper explores a simple method for improving the zero-shot learning...
research
∙
08/24/2021
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision
With recent progress in joint modeling of visual and textual representat...
research
∙
08/15/2020
Compositional Generalization via Neural-Symbolic Stack Machines
Despite achieving tremendous success, existing deep learning models have...
research
∙
06/05/2020
AutoHAS: Differentiable Hyper-parameter and Architecture Search
Neural Architecture Search (NAS) has achieved significant progress in pu...
research
∙
04/23/2018
QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
Current end-to-end machine reading and question answering (Q&A) models a...
research
∙
03/11/2018
Detecting Nonlinear Causality in Multivariate Time Series with Sparse Additive Models
We propose a nonparametric method for detecting nonlinear causal relatio...
research
∙
10/13/2017
DSCOVR: Randomized Primal-Dual Block Coordinate Algorithms for Asynchronous Distributed Optimization
Machine learning with big data often involves large optimization models....
research
∙
07/16/2017
Normalized Gradient with Adaptive Stepsize Method for Deep Neural Network Training
In this paper, we propose a generic and simple algorithmic framework for...
research
∙
04/23/2017
Learning to Skim Text
Recurrent Neural Networks are showing much promise in many sub-areas of ...
research
∙
02/23/2016
An Improved Gap-Dependency Analysis of the Noisy Power Method
We consider the noisy power method algorithm, which has wide application...
research
∙
01/09/2016
On Computationally Tractable Selection of Experiments in Measurement-Constrained Regression Models
We derive computationally tractable methods to select a small subset of ...
research
∙
08/20/2015
AdaDelay: Delay Adaptive Distributed Stochastic Convex Optimization
We study distributed stochastic convex optimization under the delayed gr...
research
∙
08/14/2015