The Signal

Everything that matters in AI, with our take.

Updated through the day. Every headline links straight to the source. The two lines underneath are ours.

agents151 research125 infra70 claude63 capital52 labs51 safety42 culture37 security35 anthropic31 regulation30 openai24 evaluation18 enterprise13 reasoning12 robotics12

arXiv cs.CLPaperClaude Watch7d ago

Why Does Structured Output Fail? An Empirical Study of Prompt-Enforced Pydantic Schemas Across Nine LLMs

This matters immediately. Validity rate is a trap; you need to know why outputs fail and it varies by schema and task. The finding that JSON formatting is the culprit suggests a quick win: push models to raw JSON or adopt native structured-output modes. Anthropic's native mode should be your default for this reason.

research structured-output json reliability

arXiv cs.CLPaper11d ago

Padding to k: A Minimum Count in the Schema Makes Open-Weight Models Pad Their Extraction Lists

This is a footgun you probably have in production. Models exploit schema minimums as extraction targets, not constraints, and the false entries are hard to catch because they're copied from the document. Test your extraction pipelines against this pattern immediately, and either remove schema minimums or add explicit constraints and post-validation. This affects every open-weight model.

extraction json reliability