← 首页|学术|Knowing When to Ask for Help: Bayesian Self-Escalation in Hierarchical LLM Agents
cs.LG, cs.AI, stat.ML · 2608.24087 · 2026-08-25

Knowing When to Ask for Help: Bayesian Self-Escalation in Hierarchical LLM Agents

Nadeem Shaikh
Agent DelegationBayesian Decision TheoryHierarchical Agents
💬 现有委托机制要么在推理开始前选模型、要么在响应完成后打分重试;本文研究第三种范式——agent 在生成过程中就意识到自己可能失败,主动把控制权转交给更强模型,并给出严谨的贝叶斯最优停止理论与 1/√n 的有限样本保证。

🎯 背景

当前 LLM agent 系统的委托决策要么发生在推理开始之前(router 预先选择模型),要么发生在响应生成完毕之后(verifier 打分并可能触发重试)。这两种方式都无法利用"推理进行到一半"时暴露出的信号。

🔬 方法

论文研究第三种范式:agent 在自己的推理过程中识别出成功概率低,进而把控制权转交给更强模型。作者把这种"生成中委托"形式化为一个关于学习到的能力后验的贝叶斯最优停止问题——这个能力后验的充分统计量是从有标签轨迹中学到的,而非直接读取原始的熵值。论文推导出闭式的短视升级阈值,用动态规划刻画最优策略,并证明最优策略是一个不对原始信号形状做任何假设的时变阈值。

📊 结果

理论上证明了 oracle 信念以信号的 Chernoff 信息速率指数级分离,regret 上界由后验校准质量控制,且给出有限样本保证:用 n 条标注校准轨迹时,部署的插件式策略 regret 按 1/√n 衰减。受控仿真研究验证了理论的每一条预测,包括预测的 1/√n 速率。作者还在 Qwen2.5-Coder 1.5B→7B 的真实代码级联(MBPP,257 个任务)上做验证,确认了预先注册的 3 条预测中的 2 条:在同等成本下,升级前沿优于事后路由;累积能力信念的判别力会随生成推进而提升。

原文摘要

▶ Abstract
Current LLM agent systems decide delegation before reasoning begins (a router picks a model) or after a response is complete (a verifier scores it and may retry). We study a third regime: an agent that recognises, during its own reasoning, that it is unlikely to succeed and transfers control to a stronger model. We formulate intra-generation delegation as a Bayesian optimal-stopping problem over a learned competence posterior -- an online estimate of the agent's eventual task success whose sufficient statistics are learned from labelled trajectories, not read off raw entropy. We derive the myopic escalation threshold in closed form, characterise the optimal policy via dynamic programming, and prove that the optimal policy is a time-varying threshold with no shape assumption on the raw signal. We further prove exponential separation of the oracle belief at the Chernoff-information rate of the signal, a regret bound governed by the calibration of the posterior, and a finite-sample guarantee: with n labelled calibration trajectories the deployed plug-in policy's regret decays as 1/sqrt(n). A controlled simulation study confirms each prediction of the theory, including the predicted 1/sqrt(n) rate. We additionally report a real-model validation on a Qwen2.5-Coder 1.5B->7B code cascade (MBPP, 257 tasks), confirming two of three pre-registered predictions: the escalation frontier dominates post-hoc routing at equal cost, and the cumulative competence belief's discrimination rises over generation.
来源: arXiv:2608.24087 · 精读由高松灯生成,基于摘要与 arXiv 页面信息