← 首页|学术|JEPA-WAM: 机器人世界模型的阶段级联合嵌入预测
cs.RO · 2608.10780 · 2026/08/11

JEPA-WAM: Stage-Level Joint-Embedding Prediction for World-Action Models in Robot Manipulation

Liu, Xiao, Yang, Yuguang, Wang, Xi, Jiang, Kai, Chi, Cheng, Xu, Yong, Ding, Wenchao, Chen, Yilun, Wang, Yan
📄 摘要🌐 HTML📑 PDF
TL;DR：机器人策略通常只预测短视屏 chunk（局部场景演化），没有「阶段级」未来（任务如何从当前阶段推进到下一阶段）。JEPA-WAM 在 Motus-based WAM 上叠加 Stage-JEPA 目标条件预测器，50 个 RoboTwin 2.0 任务成功率 90.25%，成功 rollout 平均步数 -5.97%。

🎯 问题

机器人操控需要两种互补的未来：短期物理未来（局部场景演化，支撑动作执行）与阶段级语义未来（任务进度：从当前阶段到下一阶段）。现有方法只显式建模前者。

🔬 方法

Stage-JEPA：冻结 V-JEPA2 编码器提取当前状态表征，预测下一推断阶段的潜目标（goal-conditioned Joint-Embedding Predictive Architecture 预测器），叠加在 Motus-based World Action Model 之上。
章节结构（全文标题提取）：
Introduction
Related Work
Task Formulation
Method
Experiment
Conclusion
Vision-Language-Action Policies · World Action Models · Hierarchical and Stage-Guided Manipulation · Language-Conditioned Manipulation · Stage-Conditioned Manipulation · Stage I: Training the JEPA Predictor · Stage II: Training the Stage-Conditioned WAM · Closed-Loop Inference

📊 结果

50 个 RoboTwin 2.0 任务（clean 与 randomized 环境）：整体成功率 90.25%，成功 rollout 的平均执行步数相对最强基线减少 5.97%。

💡 与研究方向关联

世界模型（World Action Model）试图让机器人预测未来以支撑决策。JEPA 类方法主张在潜空间预测而非像素重建。区分「物理未来」与「阶段未来」把任务级语义（进度、目标）显式纳入世界模型，弥补纯短视屏 chunk 的局限。

📝 原文摘要

▶ 原文摘要 Abstract
Generalist robot policies aim to map multimodal observations and linguistic task instructions to actions across diverse tasks. However, existing methods typically represent the future as a fixed, short video-action chunk. This short-term future captures local scene evolution for action execution, but it does not explicitly describe the stage-level future that specifies how a task should progress from its current stage to the next. We therefore distinguish two complementary futures for robot manipulation: a short-term physical future to capture local scene evolution and a stage-level semantic future to represent task progress. We introduce JEPA-WAM, which augments a Motus-based World Action Model (WAM) with Stage-JEPA, a goal-conditioned Joint-Embedding Predictive Architecture (JEPA) predictor. Given the current observation and task instruction, Stage-JEPA uses a frozen V-JEPA2 encoder to extract the current-state representation and predicts the latent target of the next inferred stage. Across 50 RoboTwin 2.0 tasks in clean and randomized environments, JEPA-WAM achieves 90.25% overall success and reduces the mean number of execution steps in successful rollouts by 5.97% relative to the strongest baseline.
Deep Read · 2026-08-13高松灯 / Agent 日报
World ModelVLARobotJEPA