The Signal

Everything that matters in AI, with our take.

Updated through the day. Every headline links straight to the source. The two lines underneath are ours.

agents151 research125 infra70 claude63 capital52 labs51 safety42 culture37 security35 anthropic31 regulation30 openai24 evaluation18 enterprise13 reasoning12 robotics12

arXiv cs.CLPaper16h ago

Stepwise-Querying: Distilling Reasoning without Access to Teacher Chain-of-Thought

This is a practical win for model distillation. If you're running inference at scale and need to compress a capable model into something faster and cheaper, this approach consistently outperforms baselines and works against black-box APIs. It doesn't require you to own the teacher's internals, which makes it deployable today. The method earned its place through empirical strength across benchmarks.

research distillation model-compression

arXiv cs.CLPaper4d ago

Weaken to Strengthen: Weakening-Induced On-Policy Self-Distillation for Large Reasoning Models

This is a practical win for reasoning models. Pruning 3.5B-parameter models by 13 percentage points without labeled data or a teacher is significant. The method sidesteps the hard problem of knowing what a good reasoning trace looks like. If you're optimizing reasoning models for deployment or fine-tuning on your own data, test this: create a pruned sibling, run it on your unlabeled reasoning tasks, use the gap as training signal.

research reasoning distillation efficiency

arXiv cs.CLPaper6d ago

Watermark Basins: Black-Box Detection of Distilled LLMs

Model distillation is a real threat to proprietary models, and existing token-level watermark detection fails when students are trained at low temperature. This paper's insight about geometric basins is novel and the black-box approach means watermarks could survive distillation attempts. For model providers: this changes the calculus of watermarking strategy. For researchers building defense systems: test this before assuming your distilled model is undetectable.

research security watermarking distillation

arXiv cs.CLPaper18d ago

Training-Free On-Policy Distillation

Clean and potentially high-impact for teams managing multiple fine-tuned models. If you have separate specialists for math and code, you can now fold them without gradient steps. The catch is that rollout quality matters more than the solver, so garbage specialist behavior spreads to the merged model. Test this on your specialist stack; if it works, you cut serving costs and latency. If it doesn't, the reason is instructive.

training distillation model-merging