arXiv cs.CLPaper
Tasks over Application Manuals: Revealing Gaps in Long-Horizon Procedural Reasoning for Language Models
This benchmark exposes a real gap: models look good on short-horizon reasoning but fail on the long, rule-heavy tasks that matter in regulated industries. If you're deploying LLMs in healthcare or legal, this is the kind of reasoning your system must handle. The benchmark itself becomes a bar for model selection and an early warning system for when models will fail in production.