research
∙
09/15/2023
Replacing softmax with ReLU in Vision Transformers
Previous research observed accuracy degradation when replacing the atten...
share
research
∙
08/02/2023
OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Models
We introduce OpenFlamingo, a family of autoregressive vision-language mo...
research
∙
07/31/2023
Guiding Image Captioning Models Toward More Specific Captions
Image captioning is conventionally formulated as the task of generating ...
research
∙
06/07/2023
Improving neural network representations using human similarity judgments
Deep neural networks have reached human-level performance on many comput...
research
∙
01/11/2023
Does progress on ImageNet transfer to real-world datasets?
Does progress on ImageNet transfer to real-world datasets? We investigat...
research
∙
12/15/2022
FlexiViT: One Model for All Patch Sizes
Vision Transformers convert images to sequences by slicing them into pat...
research
∙
12/13/2022
On the Relationship Between Explanation and Prediction: A Causal View
Explainability has become a central requirement for the development, dep...
research
∙
11/02/2022
Human alignment of neural network representations
Today's computer vision models achieve human or near-human level perform...
research
∙
10/19/2022
Gaussian-Bernoulli RBMs Without Tears
We revisit the challenging problem of training Gaussian-Bernoulli restri...
research
∙
10/11/2022
Improving Dense Contrastive Learning with Dense Negative Pairs
Many contrastive representation learning methods learn a single global r...
research
∙
10/07/2022
Scaling Forward Gradient With Local Losses
Forward gradient learning computes a noisy directional gradient and is a...
research
∙
08/10/2022
Patching open-vocabulary models by interpolating weights
Open-vocabulary models like CLIP achieve high accuracy across many image...
research
∙
07/09/2022
A Study on Self-Supervised Object Detection Pretraining
In this work, we study different approaches to self-supervised pretraini...
research
∙
05/23/2022
Decoder Denoising Pretraining for Semantic Segmentation
Semantic segmentation labels are expensive and time consuming to acquire...
research
∙
05/19/2022
Robust and Efficient Medical Imaging with Self-Supervision
Recent progress in Medical Artificial Intelligence (AI) has delivered sy...
research
∙
03/10/2022
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
The conventional recipe for maximizing model accuracy is to (1) train mu...
research
∙
02/15/2022
On the Origins of the Block Structure Phenomenon in Neural Network Representations
Recent work has uncovered a striking phenomenon in large-capacity neural...
research
∙
11/02/2021
Meta-Learning to Improve Pre-Training
Pre-training (PT) followed by fine-tuning (FT) is an effective method fo...
research
∙
10/27/2021
Generalized Shape Metrics on Neural Representations
Understanding the operation of biological and artificial networks remain...
research
∙
08/19/2021
Do Vision Transformers See Like Convolutional Neural Networks?
Convolutional neural networks (CNNs) have so far been the de-facto model...
research
∙
01/13/2021
Big Self-Supervised Models Advance Medical Image Classification
Self-supervised pretraining followed by supervised fine-tuning has seen ...
research
∙
11/23/2020
Boosting Contrastive Self-Supervised Learning with False Negative Cancellation
Self-supervised representation learning has witnessed significant leaps ...
research
∙
11/05/2020
Teaching with Commentaries
Effective training of deep neural networks can be challenging, and there...
research
∙
10/30/2020
What's in a Loss Function for Image Classification?
It is common to use the softmax cross-entropy loss to train neural netwo...
research
∙
10/29/2020
Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth
A key factor in the success of deep neural networks is the ability to sc...
research
∙
06/17/2020
Big Self-Supervised Models are Strong Semi-Supervised Learners
One paradigm for learning from few labeled examples while making best us...
research
∙
02/13/2020
A Simple Framework for Contrastive Learning of Visual Representations
This paper presents SimCLR: a simple framework for contrastive learning ...
research
∙
02/11/2020
Generalised Lipschitz Regularisation Equals Distributional Robustness
The problem of adversarial examples has highlighted the need for a theor...
research
∙
02/10/2020
Subclass Distillation
After a large "teacher" neural network has been trained on labeled data,...
research
∙
02/07/2020
Revisiting Spatial Invariance with Low-Rank Local Connectivity
Convolutional neural networks are among the most successful architecture...
research
∙
11/20/2019
Exploring the Origins and Prevalence of Texture Bias in Convolutional Neural Networks
Recent work has indicated that, unlike humans, ImageNet-trained CNNs ten...
research
∙
08/20/2019
Saccader: Improving Accuracy of Hard Attention Models for Vision
Although deep convolutional neural networks achieve state-of-the-art per...
research
∙
06/06/2019
When Does Label Smoothing Help?
The generalization and learning speed of a multi-class neural network ca...
research
∙
05/28/2019
Cerberus: A Multi-headed Derenderer
To generalize to novel visual scenes with new viewpoints and new object ...
research
∙
05/01/2019
Similarity of Neural Network Representations Revisited
Recent work has sought to understand the behavior of neural networks by ...
research
∙
11/16/2018
Domain Adaptive Transfer Learning with Specialist Models
Transfer learning is a widely used method to build high performing compu...
research
∙
09/04/2018
Lipschitz Networks and Distributional Robustness
Robust risk minimisation has several advantages: it has been studied wit...
research
∙
05/23/2018