arXiv cs.AIPaper
Stick to What You Know: A Study of Knowledge-Aligned Supervised Fine-Tuning
The insight is sound: fine-tuning on facts outside the model's grasp teaches it to hallucinate confidently. Recall Rewrite (retrain only what the model can consistently recall) is a simple idea with measurable gains. Worth testing if you're fine-tuning models and seeing factual regression, but it's incremental defense rather than structural innovation.