research
∙
08/13/2023
Law of Balance and Stationary Distribution of Stochastic Gradient Descent
The stochastic gradient descent (SGD) algorithm is the algorithm we use ...
share
research
∙
03/23/2023
The Probabilistic Stability of Stochastic Gradient Descent
A fundamental open problem in deep learning theory is how to define and ...
research
∙
10/02/2022
What shapes the loss landscape of self-supervised learning?
Prevention of complete and dimensional collapse of representations has r...
research
∙
09/02/2022
Three Learning Stages and Accuracy-Efficiency Tradeoff of Restricted Boltzmann Machines
Restricted Boltzmann Machines (RBMs) offer a versatile architecture for ...
research
∙
05/25/2022
Exact Phase Transitions in Deep Learning
This work reports deep-learning-unique first-order and second-order phas...
research
∙
01/30/2022
Stochastic Neural Networks with Infinite Width are Deterministic
This work theoretically studies stochastic neural networks, a main type ...
research
∙
01/28/2022
Interplay between depth of neural networks and locality of target functions
It has been recognized that heavily overparameterized deep neural networ...
research
∙
07/25/2021
SGD May Never Escape Saddle Points
Stochastic gradient descent (SGD) has been deployed to solve highly non-...
research
∙
06/29/2021
A Convergent and Efficient Deep Q Network Algorithm
Despite the empirical success of the deep Q network (DQN) reinforcement ...
research
∙
05/20/2021
Logarithmic landscape and power-law escape rate of SGD
Stochastic gradient descent (SGD) undergoes complicated multiplicative n...
research
∙
02/10/2021
On Minibatch Noise: Discrete-Time SGD, Overparametrization, and Bayes
The noise in stochastic gradient descent (SGD), caused by minibatch samp...
research
∙
12/07/2020
Stochastic Gradient Descent with Large Learning Rate
As a simple and efficient optimization method in deep learning, stochast...
research
∙
09/28/2020
Improved generalization by noise enhancement
Recent studies have demonstrated that noise in stochastic gradient desce...
research
∙
06/15/2020
Neural Networks Fail to Learn Periodic Functions and How to Fix It
Previous literature offers limited clues on how to learn a periodic func...
research
∙
05/26/2020
Is deeper better? It depends on locality of relevant features
It has been recognized that a heavily overparameterized artificial neura...
research
∙
03/25/2020
Volumization as a Natural Generalization of Weight Decay
We propose a novel regularization method, called volumization, for neura...
research
∙
02/16/2020
Learning Not to Learn in the Presence of Noisy Labels
Learning in the presence of label noise is a challenging yet important t...
research
∙
02/12/2020
LaProp: a Better Way to Combine Momentum with Adaptive Gradient
Identifying a divergence problem in Adam, we propose a new optimizer, La...
research
∙
10/21/2019
Deep Reinforcement Learning Control of Quantum Cartpoles
We generalize a standard benchmark of reinforcement learning, the classi...
research
∙
06/29/2019