arXiv cs.AIPaper
LOCUS: Task-Aware Low-Rank Post-Training for Token-Efficient Language Generation
Token reduction at inference time translates directly to serving cost, and this paper shows you can achieve significant cuts in verbosity without sacrificing preference quality by constraining updates to low-rank subspaces. The mechanism is elegant: different tasks need different amounts of verbosity, and low-rank adapters can capture that without full fine-tuning. If you run inference at scale, this is worth testing on your most verbose use cases.