arXiv cs.CLPaper
Incremental Pooled LLM Evaluation for Cost-Effective Retrieval Model Selection
Practical tooling for building production RAG systems. Instead of getting fresh relevance judgments for every retrieval candidate you want to test, you pool documents and judge only the new ones each time, reusing labels. They hit 97% consistency with gold-standard rankings and cut judgment work by 65-80%. If you're doing retrieval model selection at scale, this method cuts a real cost in your evaluation loop.