人手數據如何重塑機器人基礎模型

#foundation-models#human-data#robotics-learninglast-hdpeking universityzhijian dynamicslast-hdvla

💡最新研究揭示如何利用人手數據解決機器人基礎模型的數據匱乏難題。

⚡ 30 秒速覽

有什麼變化

LaST-HD專注於對齊物理世界的變化，而非僅僅是對齊動作軌跡。

為什麼重要

該研究為昂貴的遙操作數據提供了可擴展的替代方案，有望解決訓練通用機器人基礎模型的數據瓶頸。

下一步行動

將人手交互數據集納入您的VLA訓練流程，以提升物理推理能力。

誰應關注：Researchers & Academics

本篇為 AI 生成分析，非原文內容。

•LaST-HD utilizes a 'Latent State Transition' framework that decouples physical interaction dynamics from specific robot embodiments, enabling cross-platform transferability.
•The research addresses the 'sim-to-real' gap by training models on egocentric video data, allowing robots to infer object affordances without explicit 3D mesh annotations.
•The data collection pipeline employs a proprietary multi-view camera array system to reconstruct 3D hand-object interaction states with sub-millimeter precision.
•The model architecture incorporates a transformer-based temporal consistency module that predicts future state transitions based on partial observation sequences.
•Zhijian Dynamics is integrating these models into their proprietary 'Z-Hand' dexterous manipulator hardware to validate real-world grasping performance in unstructured environments.

📊 競品分析▸ Show

Feature	LaST-HD (Peking/Zhijian)	Google RT-2	Stanford Mobile ALOHA
Primary Focus	Physical Law Alignment	Vision-Language-Action	Teleoperation Mimicry
Data Source	Egocentric Human Hands	Web-scale VLA Data	Human Teleoperation
Generalization	High (Physics-based)	Medium (Semantic-based)	Low (Task-specific)
Hardware Agnostic	Yes	Yes	No

Architecture: Employs a latent diffusion model conditioned on egocentric video embeddings to predict state transitions.
Input Modality: Processes synchronized RGB-D video streams and proprioceptive robot state data.
Training Objective: Minimizes the divergence between predicted latent state transitions and observed physical outcomes in the real world.
Inference: Uses a model-predictive control (MPC) loop to map latent transitions to joint-level torque commands.
Data Processing: Utilizes a custom hand-pose estimation algorithm to filter and label 2,000 hours of raw video into actionable interaction sequences.

LaST-HD will reduce robot training time by 40% compared to traditional teleoperation-based imitation learning.

By learning underlying physical laws from human data, the model requires fewer demonstrations to generalize to novel objects.

The project will achieve a 90% success rate in zero-shot manipulation of unseen household objects by Q4 2026.

The focus on physical state transitions rather than motion trajectories allows the model to adapt to varying object geometries.

2025-03

Zhijian Dynamics initiates the LaST-HD research partnership with Peking University.

2025-11

Completion of the first 500 hours of high-fidelity human hand interaction data collection.

2026-05

Successful deployment of the LaST-HD model on Z-Hand hardware for complex assembly tasks.

AI 週報

閱讀本週精選 AI 大事摘要 →

同主題

探索 #foundation-models

同產品

更多關於 last-hd

同來源