research
∙
08/14/2023
S3IM: Stochastic Structural SIMilarity and Its Unreasonable Effectiveness for Neural Fields
Recently, Neural Radiance Field (NeRF) has shown great success in render...
share
research
∙
12/05/2022
Rethinking the Structure of Stochastic Gradients: Empirical and Statistical Evidence
Stochastic gradients closely relate to both optimization and generalizat...
research
∙
06/17/2022
Sparse Double Descent: Where Network Pruning Aggravates Overfitting
People usually believe that network pruning not only reduces the computa...
research
∙
05/19/2022
Dataset Pruning: Reducing Training Data by Examining Generalization Influence
The great success of deep learning heavily relies on increasingly larger...
research
∙
01/31/2022
On the Power-Law Spectrum in Deep Learning: A Bridge to Protein Science
It is well-known that the Hessian matters to optimization, generalizatio...
research
∙
03/31/2021
Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization
It is well-known that stochastic gradient noise (SGN) acts as implicit r...
research
∙
11/23/2020
Stable Weight Decay Regularization
Weight decay is a popular regularization technique for training of deep ...
research
∙
11/12/2020
Artificial Neural Variability for Deep Learning: On Overfitting, Noise Memorization, and Catastrophic Forgetting
Deep learning is often criticized by two serious issues which rarely exi...
research
∙
06/29/2020
Adai: Separating the Effects of Adaptive Learning Rate and Momentum Inertia
Adaptive Momentum Estimation (Adam), which combines Adaptive Learning Ra...
research
∙
02/10/2020