research
∙
06/28/2023
Towards Language Models That Can See: Computer Vision Through the LENS of Natural Language
We propose LENS, a modular approach for tackling computer vision problem...
share
research
∙
06/21/2023
OBELISC: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documents
Large multimodal models trained on natural documents, which interleave i...
research
∙
01/24/2023
The Semantic Scholar Open Data Platform
The volume of scientific output is creating an urgent need for automated...
research
∙
11/23/2022
SciRepEval: A Multi-Format Benchmark for Scientific Document Representations
Learned representations of scientific documents can serve as valuable in...
research
∙
08/16/2022
FeedLens: Polymorphic Lenses for Personalizing Exploratory Search over Knowledge Graphs
The vast scale and open-ended nature of knowledge graphs (KGs) make expl...
research
∙
07/11/2022
Embedding Recycling for Language Models
Training and inference with large neural models is expensive. However, f...
research
∙
04/07/2022
Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality
We present a novel task and dataset for evaluating the ability of vision...
research
∙
03/01/2022
Unsupervised Vision-and-Language Pre-training via Retrieval-based Multi-Granular Alignment
Vision-and-Language (V+L) pre-training models have achieved tremendous s...
research
∙
12/08/2021
FLAVA: A Foundational Language And Vision Alignment Model
State-of-the-art vision and vision-and-language models rely on large-sca...
research
∙
06/04/2021
Human-Adversarial Visual Question Answering
Performance on the most commonly used Visual Question Answering dataset ...
research
∙
05/12/2021
TextOCR: Towards large-scale end-to-end reasoning for arbitrary-shaped scene text
A crucial component for the scene text based reasoning required for Text...
research
∙
04/02/2021
Physics Informed Convex Artificial Neural Networks (PICANNs) for Optimal Transport based Density Estimation
Optimal Mass Transport (OMT) is a well studied problem with a variety of...
research
∙
02/22/2021
Transformer is All You Need: Multimodal Multitask Learning with a Unified Transformer
We propose UniT, a Unified Transformer model to simultaneously learn the...
research
∙
11/15/2020
Open4Business(O4B): An Open Access Dataset for Summarizing Business Documents
A major challenge in fine-tuning deep learning models for automatic summ...
research
∙
07/20/2020
Seeing the Un-Scene: Learning Amodal Semantic Maps for Room Navigation
We introduce a learning-based approach for room navigation using semanti...
research
∙
05/10/2020
The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes
This work proposes a new challenge set for multimodal classification, fo...
research
∙
04/19/2020
Are we pretraining it right? Digging deeper into visio-linguistic pretraining
Numerous recent works have proposed pretraining generic visio-linguistic...
research
∙
03/24/2020
TextCaps: a Dataset for Image Captioning with Reading Comprehension
Image descriptions can help visually impaired people to quickly understa...
research
∙
11/14/2019
Iterative Answer Prediction with Pointer-Augmented Multimodal Transformers for TextVQA
Many visual scenes contain text that carries crucial information, and it...
research
∙
05/02/2019
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
In the last year, new models and methods for pretraining and transfer le...
research
∙
04/18/2019
Towards VQA Models that can Read
Studies have shown that a dominant class of questions asked by visually ...
research
∙
03/05/2019
Learning Dynamics Model in Reinforcement Learning by Incorporating the Long Term Future
In model-based reinforcement learning, the agent interleaves between mod...
research
∙
12/23/2018
Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks
Learning when to communicate and doing that effectively is essential in ...
research
∙
10/05/2018
CanvasGAN: A simple baseline for text to image generation by incrementally patching a canvas
We propose a new recurrent generative model for generating images from t...
research
∙
05/31/2018
Neural Network Acceptability Judgments
In this work, we explore the ability of artificial neural networks to ju...
research
∙
10/02/2017