The Signal

Everything that matters in AI, with our take.

Updated through the day. Every headline links straight to the source. The two lines underneath are ours.

agents151 research125 infra70 claude63 capital52 labs51 safety42 culture37 security35 anthropic31 regulation30 openai24 evaluation18 enterprise13 reasoning12 robotics12

arXiv cs.AIPaperClaude Watch4d ago

How Deterministic SAST and AI Security Agents Divide the Work: Coverage, Harness Policy and Cost in Snyk VulnBench 2.0

Claude Code's security review feature is competitive with industry-standard SAST tools, which is significant for an AI-native approach. The lack of overlap—they catch different vulnerabilities—suggests they should be used together, not as alternatives. For security teams evaluating tools, this is real data showing AI doesn't replace deterministic analysis yet, but augments it. For builders of security agents, this sets a benchmark you need to beat.

research security agents claude benchmark

arXiv cs.CLPaper18d ago

FDE-Gym: The Last Mile of AI Delivery

This closes a real gap. Existing agent benchmarks test clean task specifications; real field deployment demands ambiguity handling, live system manipulation, and stakeholder coordination. The grading on customer success metric (not task completion) changes what winning means. If you're building agents for customer-facing work, this benchmark matters now.

agents benchmark research

arXiv cs.AIPaper19d ago

LabOPBench: A Multimodal Benchmark for Evidence-Grounded Laboratory Decision Making

This is a real test of agent reasoning in wet-lab domains. The models perform above undergraduates but below PhDs, which is useful ground truth. The error analysis matters more than the headline scores: agents misread evidence or fail to connect observations to decisions. If you're building for scientific workflows, this tells you not to deploy agents unsupervised on high-stakes decisions yet. The benchmark itself is worth bookmarking.

agents benchmark research