The Signal

Everything that matters in AI, with our take.

Updated through the day. Every headline links straight to the source. The two lines underneath are ours.

research1156 agents716 infra561 labs317 culture268 capital253 regulation233 claude175 openai149 safety103 security89 evaluation77 reasoning61 alignment52 benchmarks50 interpretability48

arXiv cs.CLPaper8d ago

ProcArena: A Multi-Scenario Benchmark for LLMs on Direct and Interactive PL/SQL Development from Natural Language

SQL code generation is a real use case, and benchmarks that cover interactive debugging and modification are more realistic than one-shot generation tasks. The scale is solid: 3,998 tasks across Oracle and PostgreSQL. What you do with this depends on whether you're building database automation: if you are, run your model against ProcArena and see how it actually performs on the messy middle of real development, not just greenfield translation.

research benchmarks sql code-generation

arXiv cs.CLPaper17d ago

Are These Modules Worth Their Cost? A Paradigm-Level Accuracy-Cost Analysis of In-context Learning Text-to-SQL

Text-to-SQL pipelines have become kitchen-sink affairs, and this paper finally measures what actually helps. The finding is deflating: most of the fancy modules are theater, and only execution-feedback refinement pays for itself consistently. If you're building a SQL agent, this tells you what to build and what to skip.

agents sql text-to-sql evaluation