research
∙
08/31/2023
Socratis: Are large multimodal models emotionally aware?
Existing emotion prediction benchmarks contain coarse emotion labels whi...
share
research
∙
08/08/2023
From Fake to Real (FFR): A two-stage training pipeline for mitigating spurious correlations with synthetic data
Visual recognition models are prone to learning spurious correlations in...
research
∙
07/24/2023
Multiscale Video Pretraining for Long-Term Activity Forecasting
Long-term activity forecasting is an especially challenging research pro...
research
∙
06/20/2023
LNL+K: Learning with Noisy Labels and Noise Source Distribution Knowledge
Learning with noisy labels (LNL) is challenging as the model tends to me...
research
∙
05/27/2023
Text-to-image Editing by Image Information Removal
Diffusion models have demonstrated impressive performance in text-guided...
research
∙
05/09/2023
WikiWeb2M: A Page-Level Multimodal Wikipedia Dataset
Webpages have been a rich resource for language and vision-language task...
research
∙
05/05/2023
COLA: How to adapt vision-language models to Compose Objects Localized with Attributes?
Compositional reasoning is a hallmark of human visual intelligence; yet ...
research
∙
05/05/2023
A Suite of Generative Tasks for Multi-Level Multimodal Webpage Understanding
Webpages have been a rich, scalable resource for vision-language and lan...
research
∙
04/04/2023
ERM++: An Improved Baseline for Domain Generalization
Multi-source Domain Generalization (DG) measures a classifier's ability ...
research
∙
03/28/2023
Language-Guided Audio-Visual Source Separation via Trimodal Consistency
We propose a self-supervised approach for learning to perform audio sour...
research
∙
11/22/2022
Human Evaluation of Text-to-Image Models on a Multi-Task Benchmark
We provide a new multi-task benchmark for evaluating text-to-image model...
research
∙
10/04/2022
Collecting The Puzzle Pieces: Disentangled Self-Driven Human Pose Transfer by Permuting Textures
Human pose transfer aims to synthesize a new view of a person under a gi...
research
∙
09/30/2022
Bias Mimicking: A Simple Sampling Approach for Bias Mitigation
Prior work has shown that Visual Recognition datasets frequently under-r...
research
∙
07/26/2022
NewsStories: Illustrating articles with visual summaries
Recent self-supervised approaches have used large-scale image-text datas...
research
∙
07/13/2022
Supervised Attribute Information Removal and Reconstruction for Image Manipulation
The goal of attribute manipulation is to control specified attribute(s) ...
research
∙
03/24/2022
Effectively leveraging Multi-modal Features for Movie Genre Classification
Movie genre classification has been widely studied in recent years due t...
research
∙
03/24/2022
Semantic Image Manipulation with Background-guided Internal Learning
Image manipulation has attracted a lot of interest due to its wide range...
research
∙
02/04/2022
Interactive Mobile App Navigation with Uncertain or Under-specified Natural Language Commands
We introduce Mobile app Tasks with Iterative Feedback (MoTIF), a new dat...
research
∙
12/11/2021
Show and Write: Entity-aware News Generation with Image Information
Automatically writing long articles is a complex and challenging languag...
research
∙
12/06/2021
Learning to Reason from General Concepts to Fine-grained Tokens for Discriminative Phrase Detection
Phrase detection requires methods to identify if a phrase is relevant to...
research
∙
12/06/2021
Anchoring to Exemplars for Training Mixture-of-Expert Cell Embeddings
Analyzing the morphology of cells in microscopy images can provide insig...
research
∙
10/20/2021
Look at What I'm Doing: Self-Supervised Spatial Grounding of Narrations in Instructional Videos
We introduce the task of spatially localizing narrated interactions in v...
research
∙
04/17/2021
Mobile App Tasks with Iterative Feedback (MoTIF): Addressing Task Feasibility in Interactive Visual Environments
In recent years, vision-language research has shifted to study tasks whi...
research
∙
09/16/2020
Detecting Cross-Modal Inconsistency to Defend Against Neural Fake News
Large-scale dissemination of disinformation online intended to mislead o...
research
∙
08/01/2020
Self-supervised Visual Attribute Learning for Fashion Compatibility
Many self-supervised learning (SSL) methods have been successful in lear...
research
∙
06/18/2020
Shapeshifter Networks: Cross-layer Parameter Sharing for Scalable and Effective Deep Learning
We present Shapeshifter Networks (SSNs), a flexible neural network frame...
research
∙
04/09/2020
Learning to Scale Multilingual Representations for Vision-Language Tasks
Current multilingual vision-language models either require a large numbe...
research
∙
03/18/2020
Cross-domain Self-supervised Learning for Domain Adaptation with Few Source Labels
Existing unsupervised domain adaptation methods aim to transfer knowledg...
research
∙
02/18/2020
Multi-Task Learning from Videos via Efficient Inter-Frame Attention
Prior work in multi-task learning has mainly focused on predictions on a...
research
∙
09/27/2019
wMAN: Weakly-supervised Moment Alignment Network for Text-based Video Segment Retrieval
Given a video and a sentence, the goal of weakly-supervised video moment...
research
∙
09/08/2019
MULE: Multimodal Universal Language Embedding
Existing vision-language methods typically support two languages at a ti...
research
∙
08/22/2019
Learning Similarity Conditions Without Explicit Supervision
Many real-world tasks require models to compare images along multiple si...
research
∙
08/17/2019
Language Features Matter: Effective Language Representations for Vision-Language Tasks
Shouldn't language and vision features be treated equally in vision-lang...
research
∙
05/26/2019
Why do These Match? Explaining the Behavior of Image Similarity Models
Explaining a deep learning model can help users understand its behavior ...
research
∙
11/17/2018
Open-vocabulary Phrase Detection
Most existing work that grounds natural language phrases in images start...
research
∙
09/24/2018
Give me a hint! Navigating Image Databases using Human-in-the-loop Feedback
In this paper, we introduce an attribute-based interactive image search ...
research
∙
03/25/2018
Learning Type-Aware Embeddings for Fashion Compatibility
Outfits in online fashion data are composed of items of many different t...
research
∙
11/22/2017
Conditional Image-Text Embedding Networks
This paper presents an approach for grounding phrases in images which jo...
research
∙
11/21/2016