Function Calling Works Great at Five Tools. It Falls Apart at Fifty
With five tools an agent picks correctly almost every time. With fifty it starts firing the wrong tool, inventing arguments, and doing it all with the exact same serene confidence it uses when it is right. A wrong tool call is not a harmless mistake either. It sends the email. It writes the file. It runs the query.
toolrouter is the missing piece: a selector that knows when it does not know. It scores confidence as the margin between the best tool and the runner-up, and abstains when that margin is thin instead of guessing.
The Margin, Not the Score
This is the one idea that matters. Confidence is not how well the top tool scored. It is how much it beat the second-best tool by.
• A 0.9 to 0.1 win: is a confident pick. Route it.
• A 0.42 to 0.40 win: is a coin flip wearing a confident face. This is exactly how agents fire the wrong tool on ambiguous input, and gating on the margin is what catches it.
Real Ties, Not Softballs
The benchmark's ambiguous queries are genuine ties I found empirically, not hand-waved examples. "Convert this" scores identically for unit conversion and currency conversion. "Read this" scores identically for a plain file and a PDF. There is no right pick for either, only a right abstention.
The Number
Always-picking-the-top-tool fires a wrong tool on all 5 genuinely ambiguous queries in the 20-tool benchmark. Confidence-gated routing abstains on all 5, and it does that at zero cost: clear-query accuracy stays at 100% either way. Caution that costs nothing on the easy cases is the whole point.
9 tests pass in CI across Python 3.9, 3.11, and 3.13.