← 首页|学术|SkillGate: Training In-Policy Skill Selection in Long-Horizon Agents
Artificial Intelligence (cs.AI) · 2608.18852 · 2026/08/19

SkillGate: Training In-Policy Skill Selection in Long-Horizon Agents

Li, Qingyao, Jiao, Wenxiang, Shao, Shuai, Zhang, Kangning, Lu, Yuan, Guo, Yi, Liu, Weiwen, Zhang, Weinan
TL;DR:SkillGate:技能选择是策略在 episode 中段自己做的决定,却无信号训练——selector credit starvation 让结果奖励 RL 教不会,双信用通道从构造上修好。

🎯 问题

agent 框架把程序知识打包成技能文件按需读取,选哪个技能是策略在 episode 中段做的决定,但没有任何信号训练它;结果奖励 RL 教不会,因为 broadcast 序列级优势下命名技能的少数 token 分摊到近零损失,且随轨迹变长信用越来越错号。

🔬 方法

识别并命名 selector credit starvation;SkillGate 把 token 支持分成两个不相交信用通道:结果信用只达执行 token,独立 action-local 优势精确达命名技能的 token,仅当轨迹那次读取正确时为正。
章节结构(全文标题提取):
1 Introduction
2 Problem Setup
3 SkillGate
4 Experiments
5 Related Work
6 Conclusion

📊 结果

五个 agentic 基准 16 候选书面上,9B 策略从 40.8% 提升到 53.2% trial success,远超同算力基线。

💡 与研究方向关联

「决策发生的 token 必须收到自己的信用」——细粒度信用分配是 agent 训练的核心,也与 duplex 中每个时间片的决策质量归因呼应。

📝 原文摘要

▶ 原文摘要 Abstract
Agent frameworks increasingly package procedural knowledge as skills: instruction files an agent reads on demand, while public libraries now hold thousands of them. Which skill to read has thus become a decision the policy itself makes in the middle of an episode, yet no existing signal trains it. We show that the default remedy, outcome-rewarded RL over the candidate slate, cannot teach it, for a structural reason we identify and name selector credit starvation: under a broadcast, sequence-level advantage, the few tokens that name the chosen skill carry a vanishing share of the loss, and the credit they inherit is increasingly wrong-signed as trajectories lengthen. A correct choice is punished whenever the execution after it fails, even though the choice itself is among the most valuable decisions in the trajectory. Auditing a completed run's own training artifacts confirms all three properties, each worsening monotonically with horizon. SkillGate removes the failure by construction: it partitions the token support into two disjoint credit channels, outcome credit reaching only execution tokens, and a separate action-local advantage reaching exactly the skill-naming tokens, positive only when a trajectory's single read is the correct one. On five agentic benchmarks under a 16-candidate slate, SkillGate lifts a 9B policy from 40.8% to 53.2% trial success, well ahead of the identical budget spent on outcome reward alone, while cutting exposure to misleading candidates by two thirds and reading fewer skills.
Deep Read · 2026-08-21高松灯 / Agent 日报
Skill SelectionLong-HorizonCredit AssignmentRL Training