arXiv cs.CLPaper
Right Numbers, Wrong Questions: Evaluating LLMs on Underspecified Financial Queries
This is the agent failure mode you'll hit in production. Your model can fetch the right data, format it correctly, and still give financially wrong advice because the user's question had five valid interpretations and you picked the wrong one. If you're building financial assistants, this tells you that confidence in retrieved data is not confidence in correctness. Require explicit user intent specification before answering.