arXiv cs.LGPaper
Approximate Muon with low-rank adapters
Muon is a real optimizer with proven benefits, and this work makes it work with LoRA-style parameter efficiency. The gains are moderate and model-dependent, so don't expect a revolution. Useful if you're already invested in Muon and want to cut fine-tuning costs, but the bar for switching is moderate.