LATIDIA · Robótica

3D-MoE: Hacia la inteligencia espacial con mezcla de expertos para el razonamiento 3D y la generación de acciones

arXiv:2501.16698v2 Announce Type: replace-cross Resumen: La inteligencia espacial, que abarca la percepción y el razonamiento 3D, es la próxima frontera esencial de la IA. Escalar los modelos de lenguaje de visión 3D (VLM) actuales que dependen de

La noticia

arXiv:2501.16698v2 Announce Type: replace-cross Abstract: Spatial intelligence, encompassing 3D perception and reasoning, is the essential next frontier of AI. Scaling current 3D vision-language models (VLMs) that rely on dense Transformers for spatial tasks incurs prohibitive computational costs. In this paper, we introduce 3D-MoE, a 3D VLM leveraging an efficient mixture-of-experts architecture with a modality- and spatial-context-aware probabilistic routing scheme, stably cultivated by a novel routing curriculum. To seamlessly extend 3D-MoE to embodied AI, we integrate a diffusion-based action head, Pose-DiT, transforming 3D-MoE into a 3D vision-language-action (VLA) model. By employing a rectified flow framework, Pose-DiT generates precise 6D pose actions in a single sampling step. Extensive experiments demonstrate that 3D-MoE