research
∙
06/23/2023
Predicting Grokking Long Before it Happens: A look into the loss landscape of models which grok
This paper focuses on predicting the occurrence of grokking in neural ne...
share
research
∙
12/06/2021
Multi-scale Feature Learning Dynamics: Insights for Double Descent
A key challenge in building theoretical foundations for deep learning is...
research
∙
10/27/2021
Simple data balancing achieves competitive worst-group-accuracy
We study the problem of learning classifiers that perform well across (k...
research
∙
11/18/2020
Gradient Starvation: A Learning Proclivity in Neural Networks
We identify and formalize a fundamental gradient descent phenomenon resu...
research
∙
09/18/2018
On the Learning Dynamics of Deep Neural Networks
While a lot of progress has been made in recent years, the dynamics of l...
research
∙
07/12/2018
Negative Momentum for Improved Game Dynamics
Games generalize the optimization paradigm by introducing different obje...
research
∙
01/10/2017
Towards End-to-End Speech Recognition with Deep Convolutional Neural Networks
Convolutional Neural Networks (CNNs) are effective models for reducing s...
research
∙
06/03/2016
Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations
We propose zoneout, a novel method for regularizing RNNs. At each timest...
research
∙
05/09/2016
Theano: A Python framework for fast computation of mathematical expressions
Theano is a Python library that allows to define, optimize, and evaluate...
research
∙
01/01/2015
Sequence Modeling using Gated Recurrent Neural Networks
In this paper, we have used Recurrent Neural Networks to capture and mod...
research
∙
12/20/2013
Deep Belief Networks for Image Denoising
Deep Belief Networks which are hierarchical generative models are effect...
research
∙
12/20/2013