← 首页|学术|Reflection Steering: Disentangling Reflection from Reasoning in Activation Space for Token-Efficient Inference
cs.LG, cs.CL · 2608.25542 · 2026-08-26

Reflection Steering: Disentangling Reflection from Reasoning in Activation Space for Token-Efficient Inference

Jiarui Hu, Zhiyuan Wen, Xiaoyun Liu, Jiaxing Shen et al.
推理效率激活空间编辑延迟优化
💬 大推理模型常做无意义的"自我复查"浪费token,本文通过对比反思/非反思隐藏状态并与通用推理方向正交化,分离出纯粹的"反思方向",平均减少16.9%推理token,对延迟敏感的实时交互系统是即插即用的手段。

🎯 背景

大推理模型的推理轨迹常包含验证、修正与回溯。当反思只是重新核查已确定的结果时,就是在浪费推理token并增加延迟。现有反思调控方法多在预设层上叠加标签导出的均值差方向,但其与推理和长度信号的纠缠会破坏精度-效率权衡的稳定性。

🔬 方法

Reflection Steering是一个训练自由的框架,通过将反思相关激活与通用推理解耦来控制LLM内部与反思关联的计算:在每一层对比反思与非反思隐藏状态,用PCA对得到的反思方向去噪,并使其与通用推理方向正交化;为限制早层干预向下游放大,在小样本集上对每层用多个干预强度校准、只保留稳定层,并对其残差流激活做有界投影消除。

📊 结果

在两个公开benchmark和三个开源权重LLM上、对比多个SOTA激活引导基线,Reflection Steering在六个匹配设置下平均减少16.9%的推理token;方法引入一个有界的反思干预强度参数α,支持部署时在token节省、准确率与生成稳定性之间调节。

原文摘要

▶ Abstract
Large reasoning models often produce reasoning traces with verification, revision, and backtracking. When reflection merely re-checks established results, it wastes reasoning tokens and increases latency. Most existing reflection steering methods add a label-derived mean-difference direction across preset layers, but its entanglement with reasoning and length signals destabilizes the accuracy-efficiency trade-off. In this paper, we propose Reflection Steering, a training-free framework for controlling reflection-associated computation within LLMs by disentangling reflection-related activations from general reasoning. Specifically, we contrast reflective and non-reflective hidden states at each LLM layer, denoise the resulting reflection directions with PCA, and orthogonalize them against general-reasoning directions. To limit downstream amplification from early-layer interventions, we calibrate each layer across multiple intervention strengths on a small set, retain only stable layers, and apply bounded projection removal to their residual-stream activations. We conduct extensive experiments across two public benchmarks and three open-weight LLMs against state-of-the-art activation-steering baselines. Results show that Reflection Steering reduces reasoning tokens by 16.9% on average across six matched settings. Besides, our method further introduces a bounded reflection intervention-strength parameter $\alpha$, enabling deployment-time adjustment to balance token savings, accuracy, and generation stability.
来源: arXiv:2608.25542 · 精读由高松灯生成,基于摘要与 arXiv 页面信息