research
∙
08/16/2023
Two Phases of Scaling Laws for Nearest Neighbor Classifiers
A scaling law refers to the observation that the test performance of a m...
share
research
∙
06/26/2023
Near-Optimal Fully First-Order Algorithms for Finding Stationary Points in Bilevel Optimization
Bilevel optimization has various applications such as hyper-parameter op...
research
∙
03/19/2023
Lower Generalization Bounds for GD and SGD in Smooth Stochastic Convex Optimization
Recent progress was made in characterizing the generalization error of g...
research
∙
01/02/2023
On Bilevel Optimization without Lower-level Strong Convexity
Theoretical properties of bilevel problems are well studied when the low...
research
∙
09/28/2022
Online Policy Optimization for Robust MDP
Reinforcement learning (RL) has exceeded human performance in many synth...
research
∙
06/01/2022
Realistic Deep Learning May Not Fit Benignly
Studies on benign overfitting provide insights for the success of overpa...
research
∙
04/03/2022
Understanding the unstable convergence of gradient descent
Most existing analyses of (stochastic) gradient descent rely on the cond...
research
∙
02/13/2022
Minimax in Geodesic Metric Spaces: Sion's Theorem and Algorithms
Determining whether saddle points exist or are approximable for nonconve...
research
∙
10/12/2021
On Convergence of Training Loss Without Reaching Stationary Points
It is a well-known fact that nonconvex optimization is computationally i...
research
∙
06/08/2021
Fast Federated Learning in the Presence of Arbitrary Device Unavailability
Federated Learning (FL) coordinates with numerous heterogeneous devices ...
research
∙
04/18/2021
Complexity Lower Bounds for Nonconvex-Strongly-Concave Min-Max Optimization
We provide a first-order oracle complexity lower bound for finding stati...
research
∙
02/05/2021
Provably Efficient Algorithms for Multi-Objective Competitive RL
We study multi-objective reinforcement learning (RL) where an agent's re...
research
∙
10/23/2020
Coping with Label Shift via Distributionally Robust Optimisation
The label shift problem refers to the supervised learning setting where ...
research
∙
06/08/2020
Stochastic Optimization with Non-stationary Noise
We investigate stochastic optimization problems under relaxed assumption...
research
∙
02/10/2020
On Complexity of Finding Stationary Points of Nonsmooth Nonconvex Functions
We provide the first non-asymptotic analysis for finding stationary poin...
research
∙
12/06/2019
Why ADAM Beats SGD for Attention Models
While stochastic gradient descent (SGD) is still the de facto algorithm ...
research
∙
05/28/2019
Analysis of Gradient Clipping and Adaptive Scaling with a Relaxed Smoothness Condition
We provide a theoretical explanation for the fast convergence of gradien...
research
∙
05/25/2019
Quantifying Exposure Bias for Neural Language Generation
The exposure bias problem refers to the training-inference discrepancy c...
research
∙
12/13/2018
A Probe into Understanding GAN and VAE models
Both generative adversarial network models and variational autoencoders ...
research
∙
12/13/2018
A Probe Towards Understanding GAN and VAE Models
This project report compares some known GAN and VAE models proposed prio...
research
∙
11/10/2018
R-SPIDER: A Fast Riemannian Stochastic Optimization Algorithm with Curvature Independent Rate
We study smooth stochastic optimization problems on Riemannian manifolds...
research
∙
05/01/2018