The Signal

Everything that matters in AI, with our take.

Updated through the day. Every headline links straight to the source. The two lines underneath are ours.

research1156 agents716 infra561 labs317 culture268 capital253 regulation233 claude175 openai149 safety103 security89 evaluation77 reasoning61 alignment52 benchmarks50 interpretability48

arXiv cs.AIPaper12h ago

Coding Agents Have Converged: Why the SWE-bench Leaderboard Can No Longer Order Its Top Entries, and What to Measure Instead

This is essential reading if you care about coding-agent benchmarks or are building one. The finding that the top thirty systems are statistically indistinguishable on Verified split demolishes the leaderboard's ranking function. The implication: published leaderboards are theater until they redesign. Builders should focus on specific failure modes, not ordinal score chasing.

research agents benchmarks swe-bench

Anthropic EngineeringArticleClaude Watch1mo agooriginally Jan 2025

Raising the bar on SWE-bench Verified with Claude 3.5 Sonnet

SWE-bench Verified is the benchmark serious coding-agent builders actually trust, so a documented jump here matters more than most leaderboard news. The value is in the engineering detail: how they structured the agent scaffold and tool use to get the score, which is directly reusable for anyone building a coding agent on Claude. If you shelved a code-agent project over reliability concerns, this is worth revisiting against the current model.

claude agents research swe-bench