research
∙
06/21/2023
Quantifying lottery tickets under label noise: accuracy, calibration, and complexity
Pruning deep neural networks is a widely used strategy to alleviate the ...
share
research
∙
06/17/2023
The RL Perceptron: Generalisation Dynamics of Policy Learning in High Dimensions
Reinforcement learning (RL) algorithms have proven transformative in a r...
research
∙
05/18/2023
Attacks on Online Learners: a Teacher-Student Analysis
Machine learning models are famously vulnerable to adversarial attacks: ...
research
∙
04/14/2023
Optimal inference of a generalised Potts model by single-layer transformers with factored attention
Transformers are the type of neural networks that has revolutionised nat...
research
∙
11/21/2022
Neural networks trained with SGD learn distributions of increasing complexity
The ability of deep neural networks to generalise well even when they in...
research
∙
05/29/2022
The impact of memory on learning sequence-to-sequence tasks
The recent success of neural networks in machine translation and other f...
research
∙
05/18/2022
Maslow's Hammer for Catastrophic Forgetting: Node Re-Use vs Node Activation
Continual learning - learning new tasks in sequence while maintaining pe...
research
∙
02/01/2022
Data-driven emergence of convolutional structure in neural networks
Exploiting data invariances is crucial for efficient learning in both ar...
research
∙
01/06/2022
The dynamics of representation learning in shallow, non-linear autoencoders
Autoencoders are the simplest neural network for unsupervised learning, ...
research
∙
07/09/2021
Continual Learning in the Teacher-Student Setup: Impact of Task Similarity
Continual learning-the ability to learn many tasks in sequence-is critic...
research
∙
06/07/2021
Representation mitosis in wide neural networks
Deep neural networks (DNNs) defy the classical bias-variance trade-off: ...
research
∙
05/16/2021
Bayesian reconstruction of memories stored in neural networks from their connectivity
The advent of comprehensive synaptic wiring diagrams of large neural cir...
research
∙
02/23/2021
Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeed
A recent series of theoretical works showed that the dynamics of neural ...
research
∙
02/16/2021
Capturing the learning curves of generic features maps for realistic data sets with a teacher-student model
Teacher-student models provide a powerful framework in which the typical...
research
∙
11/24/2020
The dynamics of learning with feedback alignment
Direct Feedback Alignment (DFA) is emerging as an efficient and biologic...
research
∙
06/25/2020
The Gaussian equivalence of generative models for learning with two-layer neural networks
Understanding the impact of data structure on learning in neural network...
research
∙
09/25/2019
Modelling the influence of data structure on learning in neural networks
The lack of crisp mathematical models that capture the structure of real...
research
∙
06/18/2019
Dynamics of stochastic gradient descent for two-layer neural networks in the teacher-student setup
Deep neural networks achieve stellar generalisation even when they have ...
research
∙
01/25/2019