arXiv cs.CLPaper
BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference
Reasoning models generate massive KV caches that run out of GPU memory. This method identifies that certain pivotal reasoning steps re-attend to early context, and maintains representative beacon queries instead of full cache history. If you're running inference on long-chain reasoning models, this is worth testing against your memory constraints. Training-free means low friction to adopt.