The Signal

Everything that matters in AI, with our take.

Updated through the day. Every headline links straight to the source. The two lines underneath are ours.

research1156 agents716 infra561 labs317 culture268 capital253 regulation233 claude175 openai149 safety103 security89 evaluation77 reasoning61 alignment52 benchmarks50 interpretability48

arXiv cs.CLPaper21d ago

Anchoring Bias in LLM-as-a-Judge Systems: Prior Scores Compromise Evaluation Independence

This is a direct hit on a widely trusted production pattern: if your pipeline gates content or triggers revisions using LLM judges that see prior scores, those judgments are not independent, they're anchored, with an effect size (d=0.71) too large to ignore. Anyone running iterative refinement loops with LLM judges should strip prior-score metadata from the prompt and re-test before trusting the gate.

research agents llm-as-judge

arXiv cs.CLPaper1mo ago

A Four-Axis Trustworthiness Benchmark for LLM-as-Judge in Principle-Based Regulation

Regulators are pushing LLMs into judgment roles for principle-based rules, and no existing method handles all four evaluation axes well. This benchmark matters because it's the first to test adversarial robustness and calibration together in a regulatory context. If you're building compliance automation for financial services or other regulated sectors, this defines what to measure. The Ceca method is a practical step toward auditable decisions.

regulation research compliance llm-as-judge