research
∙
09/07/2023
Trash to Treasure: Low-Light Object Detection via Decomposition-and-Aggregation
Object detection in low-light scenarios has attracted much attention in ...
share
research
∙
07/20/2023
Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization
Despite extensive studies, the underlying reason as to why overparameter...
research
∙
07/07/2023
One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attention
Recent works have empirically analyzed in-context learning and shown tha...
research
∙
06/28/2023
Beyond NTK with Vanilla Gradient Descent: A Mean-Field Analysis of Neural Networks with Polynomial Width, Samples, and Time
Despite recent theoretical progress on the non-convex optimization of tw...
research
∙
06/22/2023
The Inductive Bias of Flatness Regularization for Deep Matrix Factorization
Recent works on over-parameterized neural networks have shown that the s...
research
∙
05/26/2023
Large Language Models as Tool Makers
Recent research shows the potential of enhancing the problem-solving abi...
research
∙
05/23/2023
Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training
Given the massive cost of language model pre-training, a non-trivial imp...
research
∙
05/17/2023
DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining
The mixture proportions of pretraining data domains (e.g., Wikipedia, bo...
research
∙
05/15/2023
Symbol tuning improves in-context learning in language models
We present symbol tuning - finetuning language models on in-context inpu...
research
∙
04/29/2023
Toward L_∞-recovery of Nonlinear Functions: A Polynomial Sample Complexity Bound for Gaussian Random Fields
Many machine learning applications require learning a function with a sm...
research
∙
03/07/2023
Larger language models do in-context learning differently
We study how in-context learning (ICL) in language models is affected by...
research
∙
02/06/2023
Data Selection for Language Models via Importance Resampling
Selecting a suitable training dataset is crucial for both general-domain...
research
∙
11/28/2022
What learning algorithm is in-context learning? Investigations with linear models
Neural sequence models, especially transformers, exhibit a remarkable ca...
research
∙
11/27/2022
A Theoretical Study of Inductive Biases in Contrastive Learning
Understanding self-supervised learning is important but challenging. Pre...
research
∙
11/21/2022
First Steps Toward Understanding the Extrapolation of Nonlinear Models to Unseen Domains
Real-world machine learning applications often involve deploying neural ...
research
∙
11/10/2022
How Does Sharpness-Aware Minimization Minimize Sharpness?
Sharpness-Aware Minimization (SAM) is a highly effective regularization ...
research
∙
10/25/2022
Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models
Language modeling on large-scale datasets leads to impressive performanc...
research
∙
07/18/2022
Calibrated ensembles can mitigate accuracy tradeoffs under distribution shift
We often see undesirable tradeoffs in robust machine learning where out-...
research
∙
06/16/2022
Max-Margin Works while Large Margin Fails: Generalization without Uniform Convergence
A major challenge in modern machine learning is theoretically understand...
research
∙
06/06/2022
Asymptotic Instance-Optimal Algorithms for Interactive Decision Making
Past research on interactive decision making problems (bandits, reinforc...
research
∙
05/22/2022
Near-Optimal Algorithms for Autonomous Exploration and Multi-Goal Stochastic Shortest Path
We revisit the incremental autonomous exploration problem proposed by Li...
research
∙
04/21/2022
Toward Fast, Flexible, and Robust Low-Light Image Enhancement
Existing low-light image enhancement techniques are mostly not only diff...
research
∙
04/06/2022
Beyond Separability: Analyzing the Linear Transferability of Contrastive Representations to Related Subpopulations
Contrastive learning is a highly effective method which uses unlabeled d...
research
∙
04/01/2022
Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptation
We consider unsupervised domain adaptation (UDA), where labeled data fro...
research
∙
02/21/2022
Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution
When transferring a pretrained model to a downstream task, two popular m...
research
∙
02/15/2022
Safe Reinforcement Learning by Imagining the Near Future
Safe reinforcement learning is a promising path toward applying reinforc...
research
∙
12/09/2021
Learning with Nested Scene Modeling and Cooperative Architecture Search for Low-Light Vision
Images captured from low-light scenes often suffer from severe degradati...
research
∙
12/09/2021
DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization
Despite overparameterization, deep networks trained via supervised learn...
research
∙
11/22/2021
Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectification
The idea of conservatism has led to significant progress in offline rein...
research
∙
11/03/2021
An Explanation of In-context Learning as Implicit Bayesian Inference
Large pretrained language models such as GPT-3 have the surprising abili...
research
∙
10/11/2021
Self-supervised Learning is More Robust to Dataset Imbalance
Self-supervised learning (SSL) is a scalable way to learn general visual...
research
∙
08/04/2021
Learning Barrier Certificates: Towards Safe Reinforcement Learning with Zero Training-time Violations
Training-time safety violations have been a major concern when we deploy...
research
∙
07/28/2021
Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers
A common lens to theoretically study neural net architectures is to anal...
research
∙
07/12/2021
Calibrating Predictions to Decisions: A Novel Approach to Multi-Class Calibration
When facing uncertainty, decision-makers want predictions they can trust...
research
∙
06/18/2021
Iterative Feature Matching: Toward Provable Domain Generalization with Logarithmic Environments
Domain generalization aims at performing well on unseen test environment...
research
∙
06/17/2021
Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning
Pretrained language models have achieved state-of-the-art performance wh...
research
∙
06/11/2021
Label Noise SGD Provably Prefers Flat Global Minimizers
In overparametrized models, the noise in stochastic gradient descent (SG...
research
∙
06/09/2021
Joint System-Wise Optimization for Pipeline Goal-Oriented Dialog System
Recent work (Takanobu et al., 2020) proposed the system-wise evaluation ...
research
∙
06/08/2021
Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss
Recent works in self-supervised learning have advanced the state-of-the-...
research
∙
03/24/2021
Why Do Local Methods Solve Nonconvex Problems?
Non-convex optimization is ubiquitous in modern machine learning. Resear...
research
∙
02/09/2021
Fine-Grained Gap-Dependent Bounds for Tabular MDPs via Adaptive Multi-Step Bootstrap
This paper presents a new model-free algorithm for episodic finite-horiz...
research
∙
02/08/2021
Provable Model-based Nonlinear Bandit and Reinforcement Learning: Shelve Optimism, Embrace Virtual Curvature
This paper studies model-based bandit and reinforcement learning (RL) wi...
research
∙
12/08/2020
In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustness
Consider a prediction setting where a few inputs (e.g., satellite images...
research
∙
11/03/2020
Meta-learning Transferable Representations with a Single Target Domain
Recent works found that fine-tuning and joint training—two popular appro...
research
∙
10/22/2020
Beyond Lazy Training for Over-parameterized Tensor Decomposition
Over-parametrization is an important technique in training neural networ...
research
∙
10/21/2020
Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling
Document-level relation extraction (RE) poses new challenges compared to...
research
∙
10/07/2020
Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data
Self-training algorithms, which train a model to fit pseudolabels predic...
research
∙
08/27/2020
Entity and Evidence Guided Relation Extraction for DocRED
Document-level relation extraction is a challenging task which requires ...
research
∙
07/09/2020
Learning Over-Parametrized Two-Layer ReLU Neural Networks beyond NTK
We consider the dynamic of gradient descent for learning a two-layer neu...
research
∙
06/29/2020