arXiv cs.CLPaper
Same Trajectory, Contradictory Rewards (ROBORMBENCH): Paraphrase Fragility in Vision Language Reward Models
This is a real failure mode for anyone training robots with VLM reward models: the same trajectory gets marked success or failure depending on how you phrase the instruction. That's not just a quirk, it's dangerous if you're fine-tuning a policy. The paper shows dedicated trajectory-grounded reward models are more stable. If you're building robot learning systems, you need to know whether your reward function has this problem. This should change how you instrument training.