← 首页|学术|Split the Labor: Separating Evidence Interpretation from Decision Aggregation
cs.AI, cs.CL, cs.LG · 2608.14509 · 2026/08/14

Split the Labor: Separating Evidence Interpretation from Decision Aggregation

Wu, Zhelun
TL;DR:把「从多源得出结论」拆成证据解释(要容量与上下文)与决策聚合(要固定算术与可比性);四字段证据元组固定了这两半;揭示 count-scale drift 失效模式——未归一化权重求和的阈值随源数量滑动;pooled calibrated log-likelihood ratios 同时修复两个问题。

🎯 问题

把多源信息拼接进一个 prompt 让 LLM 下结论,混淆了两个需求不同的操作:解释证据(要能力与上下文)与组合解释(要固定算术、跨实例可比、可返回空)。

🔬 方法

分离两半后设计接口:四字段证据元组 (hypothesis, reliability bucket, rationale, provenance);分析 count-scale drift 并证明用 calibrated log-likelihood ratios 做 pooling 可同时修复投票规则与后验排序冲突。
章节结构(全文标题提取):
1 Introduction
2 Related work
3 The evidence interface
4 Combining evidence: a defect and its repair
5 Experimental setup
6 Instantiation I: the resolved regime
7 Instantiation II: the unresolved regime
8 Transfer
9 Discussion
10 Falsifiable predictions
11 Limitations
12 Broader impacts
13 Conclusion

📊 结果

在同一纵向语料上两次实例化(结果解析后/前)都有效,且证明该修复适用于分数求和式 triage、诊断面板、加法多信号检测器一类更广的规则类。

💡 与研究方向关联

证据聚合的数学拆解——对多智能体结论合成、评估面板的可靠性分析有直接借鉴。

📝 原文摘要

▶ 原文摘要 Abstract
Systems that ask a language model to reach a conclusion from many sources usually concatenate them into one prompt. This conflates two operations with different requirements. Interpreting a source rewards capacity and context. Combining interpretations rewards fixed arithmetic, comparability across instances, and the option to return nothing. Once separated, the design problem becomes the interface between them. We propose a four-field evidence tuple (hypothesis, reliability bucket, rationale, provenance) and show that fixing it determines both halves. The separation also reveals a failure mode in how such systems combine, which we call count-scale drift. Thresholding a sum of unnormalized weights is exactly posterior thresholding, but at an operating point that slides with the number of sources consulted. The slide grows with reader reliability. When source reliabilities differ, the vote rule and the posterior order instances differently, and no threshold reconciles them. Pooling calibrated log-likelihood ratios addresses both problems. The fix is arithmetic rather than architectural, and applies to a class of rules beyond language models: score-summing triage engines, diagnostic panels scored by counting positives, and additive multi-signal detectors. We then instantiate the principle twice on one longitudinal corpus, once after outcomes resolve and once before. The same partition helps in both, at different granularities: over reading in the first, over learning capacity in the second. There, a small sequence encoder on an easy auxiliary objective plus a tree ensemble carrying the censored survival loss reaches 0.921 AUPRC against 0.805 for a hand-crafted baseline. We separate what transfers from what must be re-estimated per domain, and state five predictions that would falsify the framework, three negative results, and which comparisons remain confounded.
Deep Read · 2026-08-18高松灯 / Agent 日报
Evidence AggregationDeliberationDecompositionCalibration