← 首页|学术|Beyond Myopic World Models: Long-Horizon End-to-End Training
cs.LG · 2608.07420 · 2026-08-07

Beyond Myopic World Models: Long-Horizon End-to-End Training for Direct Future Prediction

Xinyi Li,Zaishuo Xia,Chenjie Hao,Yubei Chen
TL;DR:世界模型大多用"几步局部预测"训练,再用递归 rollout,造成训练-部署错配。论文提出 long-horizon end-to-end 直接预测训练,让梯度按整条轨迹的影响传播,端到端改善长期想象。

🎯 问题

世界模型被期待支持长时间尺度的想象,但大多用局部几步预测目标训练、部署时递归 rollout 自己的预测。损失函数与长期需求错配:几步损失只优化局部转移保真度,而长期预测依赖误差与梯度如何沿整条轨迹传播。

🔬 方法

直接预测公式化:DPWM 直接优化端点未来预测;long-horizon end-to-end 训练让损失与梯度按轨迹下游影响传播;模型设计 + 附录 Lipschitz 误差界证明。
章节结构(全文标题提取):
1 Introduction
2 Method
2.1 Direct Prediction Formulation
2.2 Long-Horizon End-to-End Training
2.3 Model Design
3 Experiments
3.1 Baselines
3.2 State-based Continuous Control
3.3 Pixel-Based Discrete Control
4 Related Work

📊 结果

状态连续控制与像素离散控制(含 Pong)上都超越递归 rollout 基线;长期预测端误差更小、更稳定。

📝 原文摘要

▶ 原文摘要 Abstract
World models are expected to support imagination over extended temporal horizons, yet most are still trained through local few-step prediction objectives and deployed by recursively rolling out their own predictions. This creates a fundamental mismatch: few-step losses optimize local transition fidelity, while long-horizon prediction depends on how errors and gradients propagate through the entire trajectory. As a result, transitions with different downstream influence on the endpoint are treated uniformly during training, and small local errors are amplified through recursive inference. We argue that long-horizon accuracy is better achieved by optimizing directly, through an end-to-end endpoint prediction objective. To instantiate this paradigm, we introduce the Direct Prediction World Model (DPWM), a non-recursive architecture that compresses an action sequence of arbitrary length into a single embedding and predicts the endpoint observation in a single forward pass. This design avoids recurrent rollout in both prediction and gradient propagation, making long-horizon end-to-end training practical at horizons where unrolled autoregressive training becomes unstable. Empirically, DPWM substantially improves long-horizon endpoint prediction over recursive world-model baselines on continuous-control and pixel-based benchmarks, with larger gains as the prediction horizon increases. We further show that recurrent baselines benefit similarly when retrained with the same long-horizon endpoint objective, supporting our central claim that the training objective, rather than the particular backbone choice, is the main driver of long-horizon prediction accuracy. Our results suggest that world models can benefit from being trained and evaluated at the temporal scales where they are ultimately used, shifting the focus from local transition modeling toward long-horizon predictive accuracy.
Deep Read · 2026-08-11高松灯 / Agent 日报
World-ModelRLLong-HorizonPrediction