Introduction

The intersection of 3D vision and language models (VLMs) in robotics presents a new frontier, blending spatial understanding with contextual reasoning. This workshop, seeks to explore the opportunities and challenges posed by in- tegrating these technologies to enhance robot perception, decision-making, and interaction with the real world. As robots evolve to operate in increasingly com- plex environments, bridging the gap between 3D spatial reasoning and language understanding becomes critical. Key questions at the heart of this workshop include:

• High-level vs. Low-level Representations: Is 3D vision crucial, or can 2D representations suffice for robotic tasks? How should robots inter- pret the world—through point clouds, 3D bounding boxes, or other output formats? What input modalities offer the most efficiency and generaliza- tion?
• Pretraining for Policy Learning: Do low-level policies require exten- sive pretraining, or could projections from vision-language models serve as adequate features? Is it possible that distilled 2D features are sufficient for policy learning, or is a deeper, 3D-centric approach needed?
• 3D Vision-Language Action Models: What are the specific challenges in using 3D VLMs for robotic actions, particularly regarding sensor cali- bration and real-time performance?

We are bringing together a diverse group of leading experts in the field to present their latest research findings and future perspectives, with an emphasis on scalable, generalizable, and adaptable 3D VLM frameworks for Robotics. Our goal is for this workshop to inspire and drive future innovations in 3D foundation models specifically realting to the robotics manipulation domain.

Call for Papers

Coming soon!

Paper topics

A non-exhaustive list of relevant topics:

• 3D Vision-Language Policy Learning
• Pretraining for 3D Vision-Language Models
• 3D Representations for Policy Learning (i.e. NeRF, Gaussian Splatting, SDF)
• 3D Benchmarks and Simulatotion frameworks
• 3D Vision-Language Action Models
• 3D Vision-Language or Large-Language Models for Robotics
• 3D Instruction-tuning datasets for Robotics
• 3D pretraining datasets for Robotics
• Other topics about 3D Vision-Language Models for Robotics Manipulations