← 首页|学术|GRPO Beyond English: A Large-Scale Study of GRPO in Non-English and Multilingual Settings
cs.CL, cs.LG · 2608.13698 · 2026/08/13

GRPO Beyond English: A Large-Scale Study of GRPO in Non-English and Multilingual Settings

Dobler, Konstantin, Scozzafava, Federico, Janke, Jonathan, Ali, Mohamed, Lehnerer, Simon
TL;DR:大规模多语言 GRPO 实证:母语推理训练与英语推理训练差距很小;跨语言迁移强(一种语言训练常提升多语言);但特定模型-语言组合会引发严重回归——RLVR 超越英语可广泛获益,但必须配广泛评测抓语言级退化。

🎯 问题

RLVR/GRPO 研究严重英语中心化——非英语与多语言设定的行为几乎无人系统测量。

🔬 方法

大规模跨基座模型、训练语言、推理语言奖励的实证研究,覆盖广泛 base model 与多语/非英语组合。
章节结构(全文标题提取):
1 Introduction
2 Related Work
3 Experimental Setup
4 Results
5 Conclusion
Limitations
Acknowledgments

📊 结果

母语推理训练 vs 英语训练差距小;强跨语言迁移;但存在模型-语言依赖的严重域外回归——结论:RLVR beyond English 收益广,但需要广泛评测防特定语言退化。

💡 与研究方向关联

RL 训练的一个被忽视维度——多语言设定下的 GRPO 行为,对多语 agent 的训练配方有直接参考。

📝 原文摘要

▶ 原文摘要 Abstract
Reinforcement Learning with Verifiable Rewards (RLVR), often optimized with Group Relative Policy Optimization (GRPO), has become a central recipe for improving the reasoning capabilities of pretrained language models but current studies remain heavily English-centric. We conduct a large-scale empirical study of multilingual and non-English GRPO across a wide range of base models, training languages, and different reasoning language rewards. We find that training to reason in the native language often leaves only a small gap to training for English reasoning. We further observe strong crosslingual transfer: training in one language often improves performance in many others. However, specific trends are highly model- and language-dependent. In some cases, training in a particular language induces severe regressions on out-of-domain capabilities in other languages. Our analysis shows that RLVR beyond English can provide broad crosslingual gains, but also requires broad evaluation to detect language-specific regressions.
Deep Read · 2026-08-18高松灯 / Agent 日报
GRPOMultilingual RLRLVRCrosslingual Transfer