← 首页|学术|Agent Behavioral Contracts II: Compositional Reliability
cs.AI · cs.MA · 2608.12895 · 2026/08/13

Agent Behavioral Contracts II: Certifying Compositional Reliability Without Assuming Independence

Bhardwaj, Varun Pratap, Singh, Garima, Bhardwaj, Arun Pratap
TL;DR:实测打脸组合可靠性假设:两个同模型 agent 交接任务,失败共存率高达 90%(phi 0.916),独立性假设把冗余可靠性系统性高估。给了一个无依赖假设的有限样本证书(联合分布上的 LP)。

🎯 问题

多智能体组合可靠性界限 = 各组件可靠性相乘,靠「条件独立」假设背书——但这个假设几乎从不被检验。同模型共享失败模式时,冗余被系统性高估。

🔬 方法

18,000 任务预注册实验,用确定性代码评分(无 LLM judge):两个同模型 agent 同失败率 90%。作者给出无依赖结构假设的有限样本证书——联合分布上的线性规划(Bonferroni-Clopper-Pearson 盒)——健全、对所给信息锐利、对矩族单调。还证明拟合依赖模型更糟:bootstrap 削减 O(n^-1/2) 但识别缺口 O(1),数据越多证书越差且无表象症状。
章节结构(全文标题提取):
1 Introduction
2 Background and Related Work
3 Preliminaries: the ABC framework
4 What breaks when independence fails
5 A tiered certificate
6 Moment-set certification
7 Anytime-valid certification
8 Runtime enforcement
9 Implementation
10 Evaluation
11 Discussion
12 Contributions and their standing
13 Future Work
14 Conclusion
1.1 The independence gap · 1.2 Why the obvious repairs fail · 1.3 Approach · 1.4 Contributions · 1.5 Organisation · 2.1 Contracts for software and for agents · 2.2 Steering, filtering, and guarding · 2.3 Correlated failure in multi-agent systems · 2.4 Bounds without independence · 2.5 Sequential and anytime-valid inference · 2.6 Agent evaluation · 3.1 Contracts and compliance · 3.2 Drift · 3.3 Drift dynamics · 3.4 Composition · 3.5 Certification · 3.6 Cost · 3.7 Graphs, motifs, and co-failure

📊 结果

同模型同失败 90%(OR 6.66);换模型在 6/6 对照中降关联、换厂商不降;有限样本证书无须独立假设、单调于矩族,10→14 个矩函数收窄界。

💡 与研究方向关联

组合可靠性与「系统里都用同一个大模型」的现实直接冲突——这是 agent 编排基建层的风险工程问题,对用户关注的 agent 系统可靠部署价值高。

📝 原文摘要

▶ 原文摘要 Abstract
Compositional reliability bounds for multi-agent systems multiply component reliabilities, a step licensed by a conditional-independence assumption that is routinely stated and rarely tested. We test it. Two instances of one model, in a two-agent handoff, co-fail on 90.0% of the missions on which either fails (log OR 6.66, 95% CI [6.38, 7.00]; phi 0.916), in a preregistered evaluation of 18,000 missions scored by deterministic code with no LLM judge. Substituting a different model reduces the association in six of six contrasts; substituting a different vendor, model already different, does not -- a registered hypothesis reported as a null. The error is signed and runs against the operator: positive dependence inflates joint failure above the independence product, so redundancy is over-credited exactly when components share a model. The assumption-free alternative is often vacuous, and fitting a dependence model is worse: we prove a bootstrap bound on a fitted model's functional loses coverage of the truth as n grows, the identification gap being O(1) while the bootstrap haircut is O(n^{-1/2}). More data makes such a certificate worse, with no visible symptom. We give a finite-sample certificate assuming no dependence structure: a linear program over the joint, over a Bonferroni-Clopper-Pearson box around measured co-execution moments. It is sound, sharp for the information supplied, and monotone in the moment family. Enriching ten moment functionals to fourteen narrows the identified interval by 85.7% and lifts the certified floor from 0.2455 to 0.4116. A companion anytime-valid certificate holds type-I error at 0.0471 under optional stopping. Common dependence statistics are marginal-bounded and can reverse an apparent ordering of conditions when the compared agents fail at different rates. Contracts, scoring code, analysis scripts, and the preregistration are released.
Deep Read · 2026-08-15高松灯 / Agent 日报
Multi-agentReliabilityCompositionCertification