research
∙
09/18/2023
Multimodal Foundation Models: From Specialists to General-Purpose Assistants
This paper presents a comprehensive survey of the taxonomy and evolution...
share
research
∙
07/27/2023
Spatial-Frequency U-Net for Denoising Diffusion Probabilistic Models
In this paper, we study the denoising diffusion probabilistic model (DDP...
research
∙
06/30/2023
DisCo: Disentangled Control for Referring Human Dance Generation in Real World
Generative AI has made significant strides in computer vision, particula...
research
∙
06/07/2023
MultiSum: A Dataset for Multimodal Summarization and Thumbnail Generation of Videos
Multimodal summarization with multimodal output (MSMO) has emerged as a ...
research
∙
04/13/2023
Diagnostic Benchmark and Iterative Inpainting for Layout-Guided Image Generation
Spatial control is a core capability in controllable image generation. A...
research
∙
03/25/2023
Equivariant Similarity for Vision-Language Foundation Models
This study explores the concept of equivariance in vision-language found...
research
∙
03/20/2023
MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action
We propose MM-REACT, a system paradigm that integrates ChatGPT with a po...
research
∙
03/20/2023
Revisiting Transformer for Point Cloud-based 3D Scene Graph Generation
In this paper, we propose the semantic graph Transformer (SGT) for the 3...
research
∙
02/21/2023
Learning 3D Photography Videos via Self-supervised Diffusion on Single Images
3D photography renders a static image into a video with appealing 3D vis...
research
∙
12/01/2022
GRiT: A Generative Region-to-text Transformer for Object Understanding
This paper presents a Generative RegIon-to-Text transformer, GRiT, for o...
research
∙
11/15/2022
PromptCap: Prompt-Guided Task-Aware Image Captioning
Image captioning aims to describe an image with a natural language sente...
research
∙
10/17/2022
Prompting GPT-3 To Be Reliable
Large language models (LLMs) show impressive abilities via few-shot prom...
research
∙
06/14/2022
TransVG++: End-to-End Visual Grounding with Language Conditioned Vision Transformer
In this work, we explore neat yet effective Transformer-based frameworks...
research
∙
05/27/2022
GIT: A Generative Image-to-text Transformer for Vision and Language
In this paper, we design and train a Generative Image-to-text Transforme...
research
∙
01/18/2022
Cross-modal Contrastive Distillation for Instructional Activity Anticipation
In this study, we aim to predict the plausible future action steps given...
research
∙
11/24/2021
Scaling Up Vision-Language Pre-training for Image Captioning
In recent years, we have witnessed significant performance boost in the ...
research
∙
11/23/2021
Crossing the Format Boundary of Text and Boxes: Towards Unified Vision-Language Modeling
In this paper, we propose UNICORN, a vision-language (VL) model that uni...
research
∙
11/19/2021
UFO: A UniFied TransfOrmer for Vision-Language Representation Learning
In this paper, we propose a single UniFied transfOrmer (UFO), which is c...
research
∙
09/10/2021
An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA
Knowledge-based visual question answering (VQA) involves answering quest...
research
∙
04/17/2021
TransVG: End-to-End Visual Grounding with Transformers
In this paper, we present a neat yet effective transformer-based framewo...
research
∙
12/08/2020
TAP: Text-Aware Pre-training for Text-VQA and Text-Caption
In this paper, we propose Text-Aware Pre-training (TAP) for Text-VQA and...
research
∙
10/30/2020
Pose-based Body Language Recognition for Emotion and Psychiatric Symptom Interpretation
Inspired by the human ability to infer emotions from body language, we p...
research
∙
09/04/2020
Dynamic Context-guided Capsule Network for Multimodal Machine Translation
Multimodal machine translation (MMT), which mainly focuses on enhancing ...
research
∙
08/03/2020
Improving One-stage Visual Grounding by Recursive Sub-query Construction
We improve one-stage visual grounding by addressing current limitations ...
research
∙
07/17/2020
A Novel Graph-based Multi-modal Fusion Encoder for Neural Machine Translation
Multi-modal neural machine translation (NMT) aims to translate source se...
research
∙
07/03/2020
Improving Weakly Supervised Visual Grounding by Contrastive Knowledge Distillation
Weakly supervised phrase grounding aims at learning region-phrase corres...
research
∙
12/13/2019
Grounding-Tracking-Integration
In this paper, we study tracking by language that localizes the target b...
research
∙
08/18/2019
A Fast and Accurate One-Stage Approach to Visual Grounding
We propose a simple, fast, and accurate one-stage approach to visual gro...
research
∙
07/30/2019
Weakly Supervised Body Part Parsing with Pose based Part Priors
Human body part parsing refers to the task of predicting the semantic se...
research
∙
04/27/2019
Human-Centered Emotion Recognition in Animated GIFs
As an intuitive way of expression emotion, the animated Graphical Interc...
research
∙
11/26/2018
Attentive Relational Networks for Mapping Images to Scene Graphs
Scene graph generation refers to the task of automatically mapping an im...
research
∙
01/31/2018
Action Recognition with Visual Attention on Skeleton Images
Action recognition with 3D skeleton sequences is becoming popular due to...
research
∙
01/20/2018