arXiv cs.CLPaper
Stepwise-Querying: Distilling Reasoning without Access to Teacher Chain-of-Thought
This is a practical win for model distillation. If you're running inference at scale and need to compress a capable model into something faster and cheaper, this approach consistently outperforms baselines and works against black-box APIs. It doesn't require you to own the teacher's internals, which makes it deployable today. The method earned its place through empirical strength across benchmarks.