arXiv cs.LGPaper
Learned, Then Lost: A Measured Single-Example Counterfactual in Pre-training
This is a rare empirical measurement rather than an estimate of memorization dynamics during pretraining, and the finding that a single exposure's effect decays measurably within 50 steps matters for anyone thinking about data poisoning, unlearning, or provenance claims. Worth reading if you care about how much any one document actually shapes a trained model, since the answer here is: less and more briefly than most assume.