arXiv cs.CLPaper
Difference-in-Differences on a Censored Rating Scale Can Manufacture an Effect: Evidence from a Pre-Registered LLM-Judge Audit
This is a methods paper that quietly undermines a lot of published LLM-judge bias audits: if your rating scale is bounded, a common severity shift can look like a targeted bias just because the two conditions sit at different distances from the bounds. Anyone running LLM-as-judge evaluations for fairness or bias claims needs to check whether their stats are actually identified on the scale they're using. Treat existing bias claims from bounded-scale DiD designs with more skepticism until re-audited.