← 首页|学术|Auto-Research as Greybox Fuzzing: Feedback-Driven Search
cs.AI cs.CL · 2608.09855 · 2026-08-10

Agentic Auto-Research is Fuzz Testing

Yifeng He, Jicheng Wang, Yinzhe Zhao, Jiachen Liu, Hao Chen
TL;DR:自治研究 agent 生成实验比研究者验证还快,generate-and-rank 范式错过稀疏反馈问题。论文把 auto-research 类比 greybox fuzzer 控制循环:每个实验应暴露廉价密集的认知进步信号,且该信号决定下一步干预(搜索而非重复采样)。

🎯 问题

自治研究 agent 生成实验速度快于验证。研究者靠扩大 proposer、用 learned judge 或人工评审排序更多样本,但 generate-and-rank 范式错过稀疏反馈问题——反馈稀缺时排序无济于事。

🔬 方法

把 auto-research 建模为 greybox fuzzer 控制循环:提出候选、执行、观察反馈、选择下一步。fuzzer 很少找到 bug 但覆盖率让每次执行的局部进展可观测。提议 (1) 每个实验暴露廉价密集的认知进步信号;(2) 该信号决定下一次干预使 agent 搜索而非重复采样。最终验证仍须用受保护证据判定"发现",防止自适应复用污染。

📊 结果

提出三个受控测试:候选信号是否预测验证进度、反馈定向搜索是否比重复采样每单位成本产出更多已验证发现、受保护验证是否减少假发现。结论:反馈架构(不只是生成)是 auto-research 的核心瓶颈。

📝 原文摘要

▶ 原文摘要 Abstract
Autonomous research agents can generate experiments faster than researchers can validate them. Researchers have responded by scaling the proposer and ranking more samples with a learned judge or human reviewers. We argue that this *generate-and-rank* paradigm misses the problem of sparse feedback. Within a declared research problem, an agent follows the control loop of a greybox fuzzer: it proposes a candidate, executes it, observes feedback, and chooses what to try next. A fuzzer rarely finds a bug, but coverage makes partial progress observable on every execution. Fuzzers then use that signal to mutate inputs and allocate effort, rather than only to rank completed runs. Auto-research needs the same two capabilities. First, each experiment should expose a cheap, dense signal of epistemic progress before final scientific validation is available. Second, that signal should determine the next intervention so that the agent searches rather than repeatedly samples. Because the optimized progress signal is guidance rather than a verdict, final validation must still decide what counts as a discovery using evidence protected from adaptive reuse. We propose controlled tests of whether candidate signals predict validated progress, whether feedback-directed search yields more validated discoveries per unit cost than repeated sampling, and whether protected validation reduces false discoveries. Feedback architecture, not only generation, is a central bottleneck in auto-research.
Deep Read · 2026-08-12高松灯 / Agent 日报
AgentResearchFuzzingFeedback