← 首页|学术|Seeker: Attention from Action
cs.RO · 2608.13422 · 2026/08/13

Attention from Action, for Action: Emergent Visual Bottlenecks for Policy Learning

Zhuang, Zheyu, Wang, Ruiyu, Heppert, Nick, Hahn, Johannes Fabian, Valada, Abhinav, Pokorny, Florian T., Kragic, Danica
TL;DR:让视觉瓶颈从动作里学出来:冻结 DINOv3 特征上,Seeker 用任务与状态条件化的 readout 迭代更新 query 收集视觉证据,输出随任务进展演化的 ROI。真机 in-domain 成功率 48.3%→76.7%,光照/背景迁移 20.0%→60.0%。

🎯 问题

视觉瓶颈(ROI)能提升样本效率,但多数依赖外部空间标签(注视/物体类/affordance);动作派生的裁剪用端效应器投影做固定锚点,事件时机/代理点/裁剪尺度是固定选择,视觉证据偏离端效应器或随任务进展变化时会失配。

🔬 方法

Seeker:任务与状态条件化 readout,从动作监督学习注意力。迭代更新 query 聚合视觉证据,产生随进展演化的 ROI;作为 RGB 裁剪、mask 引导背景增强、点云过滤的空间接口。
章节结构(全文标题提取):
1 Introduction
2 Related Work
3 Methodology
4 Simulation Evaluations
5 Real-world Evaluations
6 Ablations
7 Limitations
8 Conclusion
A Seeker Training Details
B Simulation Task Overviews
C Data Efficiency and Spatial Coverage
D Experimental Details and Results
E Real Experiments
3.1 Seeker Architecture · A.1 Head Gating Visualizations · A.2 Seeker Implementation Details · D.1 RGB Cropping · D.2 Point-cloud Filtering · D.3 Random Overlay and Guided Overlay · E.1 Tasks and Setup · E.2 Training Details · E.3 Seeker Visualization

📊 结果

仿真与真机上都比无裁剪/增强/动作派生裁剪基线提升数据效率与鲁棒性;真机 in-domain 48.3%→76.7%,光照/背景迁移 20.0%→60.0%。

💡 与研究方向关联

「看哪里」从动作里学而非预设——把视觉归纳偏置交给任务驱动的学习,对具身 agent 的数据效率是直接改进。

📝 原文摘要

▶ 原文摘要 Abstract
Visual bottlenecks that focus policy inputs on regions of interest (ROIs) can improve data-efficient visuomotor learning by separating where to look from how to act. Many ROI interfaces rely on external spatial labels, such as gaze, object classes, or affordance annotations. Label-free alternatives often derive crops from trajectories by detecting gripper or motion events and centering a fixed crop at the projected end-effector. Such action-derived crops are useful spatial priors that require no additional labels, but they encode fixed choices about event timing, proxy points, and crop scale. When the visual evidence needed for control lies away from the end-effector or changes continuously with task progress, these crops can become misaligned. We propose Seeker, a task- and state-conditioned readout that learns attention from action. Starting from frozen DINOv3 features, Seeker iteratively updates a query with gathered visual evidence, producing progression-aware ROIs solely from action supervision. The learned ROI serves as a spatial interface for RGB cropping, mask-guided background augmentation, and point-cloud filtering. In simulation and the real world, Seeker improves data efficiency and robustness over no-crop, augmentation, and action-derived crop baselines. On real robots, Seeker raises average in-domain success from the best baseline's 48.3% to 76.7% and success under lighting/background shifts from 20.0% to 60.0%.
Deep Read · 2026-08-15高松灯 / Agent 日报
VLAAttentionVisual BottleneckPolicy Learning