LATIDIA · Robótica
Levantar, ver, actuar: preentrenamiento de políticas robóticas jerárquicas con modelos base 3D
arXiv:2611.20481v1 Tipo de anuncio: nuevo Resumen: El preentrenamiento de la política de los robots basado en vídeos de humanos es crucial para mejorar la capacidad de generalización de la política. Uno de los principales retos de esta tarea es la ausencia de acciones explícitas.
WhatsApp ↗Telegram ↗
La noticia
arXiv:2611.20481v1 Announce Type: new Abstract: Robot policy pretraining based on human videos is crucial for improving the policy's generalization ability. One main challenge for this task is the lack of explicit action-relevant representations in such unlabeled data. Recent works tend to estimate 3D hand motion trajectories from videos using 3D foundation models (3D FMs). However, they keep solely the sparse trajectories as supervision, ignoring the rich, fine-grained information produced during the trajectory extraction process. To address this issue, we propose LSA, a hierarchical robot policy pretraining framework that includes three phases: Lift, See, and Act. Specifically, the Lift phase elevates 2D video observations into dense 3D representations by adaptively aligning