research
∙
07/28/2023
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
We study how vision-language models trained on Internet-scale data can b...
share
research
∙
07/26/2023
Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisition
We present a framework for robot skill acquisition, which 1) efficiently...
research
∙
07/10/2023
Large Language Models as General Pattern Machines
We observe that pre-trained large language models (LLMs) are capable of ...
research
∙
03/06/2023
PaLM-E: An Embodied Multimodal Language Model
Large language models excel at a wide range of complex tasks. However, e...
research
∙
03/01/2023
Grounded Decoding: Guiding Text Generation with Grounded Models for Robot Control
Recent progress in large language models (LLMs) has demonstrated the abi...
research
∙
01/18/2023
NeRF in the Palm of Your Hand: Corrective Augmentation for Robotics via Novel-View Synthesis
Expert demonstrations are a rich source of supervision for training visu...
research
∙
12/13/2022
Single-Level Differentiable Contact Simulation
We present a differentiable formulation of rigid-body contact dynamics f...
research
∙
12/12/2022
MIRA: Mental Imagery for Robotic Affordances
Humans form mental images of 3D scenes to support counterfactual imagina...
research
∙
10/12/2022
Interactive Language: Talking to Robots in Real Time
We present a framework for building interactive, real-time, natural lang...
research
∙
10/07/2022
VIRDO++: Real-World, Visuo-tactile Dynamics and Perception of Deformable Objects
Deformable objects manipulation can benefit from representations that se...
research
∙
09/16/2022
Code as Policies: Language Model Programs for Embodied Control
Large language models (LLMs) trained on code completion have been shown ...
research
∙
07/12/2022
Inner Monologue: Embodied Reasoning through Planning with Language Models
Recent works have shown how the reasoning capabilities of Large Language...
research
∙
06/03/2022
Reinforcement Learning with Neural Radiance Fields
It is a long-standing problem to find effective representations for trai...
research
∙
05/12/2022
Visuomotor Control in Multi-Object Scenes Using Object-Aware Representations
Perceptual understanding of the scene and the relationship between its d...
research
∙
04/01/2022
Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language
Large foundation models can exhibit unique capabilities depending on the...
research
∙
03/03/2022
Implicit Kinematic Policies: Unifying Joint and Cartesian Action Spaces in End-to-End Robot Learning
Action representation is an important yet often overlooked aspect in end...
research
∙
03/03/2022
NeRF-Supervision: Learning Dense Object Descriptors from Neural Radiance Fields
Thin, reflective objects such as forks and whisks are common in our dail...
research
∙
02/02/2022
VIRDO: Visio-tactile Implicit Representations of Deformable Objects
Deformable object manipulation requires computationally efficient repres...
research
∙
09/10/2021
Trajectory Optimization with Optimization-Based Dynamics
We present a framework for bi-level trajectory optimization in which a s...
research
∙
09/01/2021
Implicit Behavioral Cloning
We find that across a wide range of robot policy learning scenarios, tre...
research
∙
06/07/2021
XIRL: Cross-embodiment Inverse Reinforcement Learning
We investigate the visual cross-embodiment imitation setting, in which a...
research
∙
12/10/2020
iNeRF: Inverting Neural Radiance Fields for Pose Estimation
We present iNeRF, a framework that performs pose estimation by "invertin...
research
∙
12/06/2020
Learning to Rearrange Deformable Cables, Fabrics, and Bags with Goal-Conditioned Transporter Networks
Rearranging and manipulating deformable objects such as cables, fabrics,...
research
∙
10/27/2020
Transporter Networks: Rearranging the Visual World for Robotic Manipulation
Robotic manipulation can be formulated as inducing a sequence of spatial...
research
∙
09/10/2020