This is the missing piece for tool-use reliability. A 4B model trained on 11K generated examples reaches 86% accuracy on the Berkeley Function Calling Leaderboard, and crucially, it generalizes to functions outside the training set. If you're training agents on Claude or another model, this data synthesis pattern beats static filtering. The shift from generate-then-filter to generate-verify-refine is the real contribution, not the 11K examples.
This is the kind of debugging trap that catches real systems: the model appears to work on the fidelity check but never actually called the tool it was supposed to. The dispatch-level trace is the only way to catch it. For anyone building document extraction or other structured-output agents, this is a pattern you need now. Stop trusting fidelity alone; instrument the tool calls.
Tool-calling accuracy is still the weak point in agent systems, and this work isolates the problem cleanly: models generate calculation errors because they weren't trained to use tools reliably. The RL methods here (RLOO, GRPO, DAPO) are practical, and the fresh held-out benchmark is credible. If you're building math agents, the pattern is clear: supervised fine-tune first on tool patterns, then optimize with on-policy RL and verifiable rewards. This is becoming table stakes.
This is a focused agent model from a company with real tooling experience. The training recipe is deliberately conservative, which suggests they're serious about stability over scaling tricks. The BFCL Core score of 0.785 is solid but not transformative. If you're already locked into other frameworks this doesn't move you; if you're evaluating agent bases, add it to the comparison matrix.
This confirms something builders have suspected anecdotally: letting a code-capable model write and chain function calls in Python beats forcing it through rigid JSON schemas, especially for parallel tasks. If you're building agent frameworks on any code-capable model, this is direct evidence to switch from JSON tool schemas to code-execution based tool interfaces. The gains are model-dependent though, so test on your own task mix before committing.