arXiv cs.CLPaper
Direct Diversity Optimization for Diverse Successful Trajectories in Preference Post-Training
The practical problem is real: agents trained on outcome labels often converge to one policy path even when multiple valid strategies exist, limiting robustness and generalization. DDO addresses this by explicitly training for strategy diversity during post-training. If you're building multi-step agent systems and noticing policy brittleness or poor performance on held-out task variants, this signals that diversity-aware training might matter more than you think. The method is evaluated on relatively simple environments, so test it on your actual domain before investing engineering effort.