This week
Suggest a nameObligation Marketing for AI Builders
The 7-Rung LadderThe 8-Type TaxonomyWrapper or WinnerThe Sample ColumnThe Signal
Updated through the day. Every headline links straight to the source. The two lines underneath are ours.
This taxonomy clarifies what we've been lumping together as 'hallucinations' and gives you a diagnostic tool. Confabulations need external knowledge or retraining; semantic illusions need better retrieval or routing. The re-elicitation test is simple and reproducible. For teams debugging reliability issues, this gives you a clearer map of what broke and what to fix.
This matters immediately. Validity rate is a trap; you need to know why outputs fail and it varies by schema and task. The finding that JSON formatting is the culprit suggests a quick win: push models to raw JSON or adopt native structured-output modes. Anthropic's native mode should be your default for this reason.
LLMs conducting security audits using static-analysis output show asymmetric credulity: they're highly vulnerable to false negatives if the tool falsely claims something is safe, but resistant to false positives. This is the wrong asymmetry for security. If you're building LLM-plus-static-analyzer pipelines for vulnerability scanning, you need explicit confirmation logic and out-of-band verification of any safety claim, not just model-level confidence.
This matters if you're building agent systems. The key finding is brutal and actionable: model scale does not buy fault tolerance, larger models fail just as hard when tools fail. The fault distribution is real (calibrated to 1,774 open-source repos), and the result should force you to build explicit error handling, retry logic, and graceful degradation instead of assuming scale solves it.
This is the underrated pattern: use wet lab data to close the loop on model reliability. Leash caught that their models were confabulating chemistry by checking against real experiments. For biotech builders using agents, this is your playbook: pair inference with measurement. The data that teaches chemists also teaches AI to stop lying.
This is a footgun you probably have in production. Models exploit schema minimums as extraction targets, not constraints, and the false entries are hard to catch because they're copied from the document. Test your extraction pipelines against this pattern immediately, and either remove schema minimums or add explicit constraints and post-validation. This affects every open-weight model.