arXiv cs.CLPaper
ProVe: Proposer-Verifier Decoding
Speculative decoding usually runs small drafts through large verification. ProVe flips it: large model proposes a block, small model verifies. 2.5x throughput without retraining is production-grade. If you're running inference at scale and margin matters, this is worth implementing. The lossless variant is especially compelling.