arXiv cs.AIPaper
PACE: Perceived-Latency-Aware Cascading Service Routing and Filler Control for QoE-Efficient Retrieval-Augmented Dialogue Serving
The innovation is treating perceived latency as the quality metric, not just raw latency, and using filler to manage user experience while waiting for better answers. Deployed on a robot sales chatbot and showing real improvements. If you're building production RAG systems, the cascading routing and cache strategies are usable. The paper is dense on mechanisms but the practical payoff is there.