← 首页|学术|Agentic Harnesses: LLM-Judge Verification for Robot Planning
cs.RO cs.AI · 2608.09857 · 2026-08-10

Agentic Harnesses: LLM-Driven Verification Layers for Robot Autonomy

Rohan Bhagra, Mahantesh Halapannavar, Uddhav Bhattarai
TL;DR:机器人规划模型像通用 LLM 一样有偏见、可能不安全、易受攻击,但开发聚焦执行而非验证动作可行性。Agentic Harnesses 在规划与执行间加 LLM 驱动验证层(LLM-as-a-Judge 集成 + CoT),作为中间件门控计划:批准/拒改/升级人工。近 85% 精度、97% 对抗攻击遏制。

🎯 问题

机器人自主开发聚焦执行而非验证规划模型提议动作的可行性。规划模型有风险:偏向用户指定目标、可能提出违背科学伦理的动作、因无法"记住"先前安全风险而不安全、易受自主生态对抗攻击。

🔬 方法

在规划与执行之间加 LLM 驱动验证层评估动作可允许性。LLM-as-a-Judge 集成组合跨模型 CoT 推理并合成专家判断(MoE + self-consistency 的结合)。作为中间件门控服务器规划模块到 MCP 服务器再到机器人底层控制的计划:批准、拒改重表述、或升级人工审查。
章节结构(全文标题提取):
I Introduction
II Methodology
III Results and Discussion
IV Conclusions
Code and Data Availability
References
· I.1 LLM-driven verification
· I.2 Verification Systems in Robotics
· I.3 Governance as a runtime structure
· II.1 Judging Architecture
· II.2 Deterministic Checks
· II.3 Knowledge Graph
· II.4 Failure Mode Tracking
· II.5 Prompt Injection Guardrails
· III.1 Safety, Alignment, Adversarial Attack Comprehensive Test
· III.2 Adversarial Attack Focused Test

📊 结果

accept/escalate/reject 类别近 85% 精度、97% 对抗攻击遏制,接受与拒绝之间误差可忽略,误差主要集中在 escalate 边界。

📝 原文摘要

▶ 原文摘要 Abstract
Advances in advanced artificial intelligence tools have sparked research in robot autonomy, but the development of such systems has largely focused on execution rather than verifying the feasibility actions planning models propose. Like general-purpose LLMs, robotics planning models carry risks: biased toward user-specified goals, they may suggest actions misaligned with scientific ethics, they may be unsafe due to an inability to "remember" prior safety risks, or they may be vulnerable to adversarial attacks on the autonomy ecosystem. We propose a LLM-driven verification layer between planning and execution to evaluate action permissibility. Our LLM-as-a-Judge ensemble combines chain-of-thought reasoning across models and synthesizes those expert judge outputs, mirroring a combination of a mixture of experts and self-consistency approach. This layer serves as middleware, gating plans from the server's planning module before they reach the MCP server and therefore the robot's low-level controls: plans are approved, rejected for reformulation, or escalated for human review. With this system, we achieve near 85% precision across accept/escalate/reject categories 97% containment of adversarial attacks, with negligible errors between accepting and rejecting tasks, and errors mostly manifesting at the escalate boundary.
Deep Read · 2026-08-12高松灯 / Agent 日报
RobotVerificationSafetyLLM-Judge