This week
Suggest a nameObligation Marketing for AI Builders
The 7-Rung LadderThe 8-Type TaxonomyWrapper or WinnerThe Sample ColumnThe Signal
Updated through the day. Every headline links straight to the source. The two lines underneath are ours.
This matters if you're training your own models or selecting checkpoints from others. The insight is that robustness under weight perturbations predicts downstream task performance better than raw benchmark scores. Use solution density as a checkpoint selection signal, especially for production fine-tuning.
Optimization method papers are incremental by nature, and this one solves a real problem (slow progress along flat directions) but only matters if you're running your own pretraining. Most builders aren't. The lab-specific papers like this cluster in the 40-50 range: solid technical work with narrow applicability.
This is a synthetic data technique aimed at improving how models learn to write structured long-form documents, by exposing the plan-then-write process rather than just the final text. Teams doing continued pre-training on scientific or technical corpora should watch whether this measurably improves long-document generation quality, since the corpus-doubling trick is easy to replicate if it works.
This is a rare empirical measurement rather than an estimate of memorization dynamics during pretraining, and the finding that a single exposure's effect decays measurably within 50 steps matters for anyone thinking about data poisoning, unlearning, or provenance claims. Worth reading if you care about how much any one document actually shapes a trained model, since the answer here is: less and more briefly than most assume.
The pitch is that alignment baked in from token zero is more robust than a post-training veneer, which is a real hypothesis worth testing but unproven at scale here. If it holds up, it changes how labs think about pretraining data mixes for safety, not just RLHF pipelines. Watch for follow-up work showing this survives fine-tuning attacks before treating it as more than a promising idea.