The Coworker Who Actually Asks "Wait, Which Jaguar?"
Every team has the person who answers your question at length, with total confidence, about something you never asked. Most RAG agents are that person. Ask for the price of "apple" and you get a rock-solid answer about the fruit or the company, chosen by a coin flip you never saw happen.
The fix is not a smarter model. It is one instant of humility. Knowing when to ask instead of barreling ahead.
clarifyrag asks only when it is genuinely torn. If the retrieved evidence splits cleanly across two meanings, it asks one sharp question. If one meaning clearly wins, it just answers. No committee, no interrogating you about things that were never ambiguous.
A Gate You Can Read
Zero dependencies, a plain TF-IDF retriever, and a gate that shows its work.
• Ambiguity score: the mass of the runner-up meaning over the top one. Zero is crystal clear, one is a coin flip.
• Lexical backstop: a known trap word (apple, jaguar, python, mercury) with nothing nearby to disambiguate it raises the flag even when the scores alone look borderline.
• Multi-turn: ask, take your hint, then answer, all tracked so you can see how many turns it took and whether it landed.
• Bring your own retriever: swap TF-IDF for a real vector DB behind the same interface and the ask-or-answer logic never changes.
Measuring the Right Thing
The trap was the metric. Grade on accuracy alone and you reward an agent that asks about everything. Grade on speed alone and you reward one that never asks and stays confidently wrong. So each policy is scored as accuracy minus a penalty per question, which finally makes "asks when needed" and "asks about everything" comparable instead of a shouting match about which number matters.
And the gate is never a black box. Every decision comes back with the actual sense ratio and a plain-English reason, which is worth more in a real deployment than a slightly prettier F1.
The Number
Answer everything and you get 77% accuracy at zero questions. Ask about everything and you hit 100% but nag on every obvious query. Clarify only when it counts and you get the same 100% while asking 38% fewer questions. Best of every policy tested.
The Benchmark Was Grading Its Own Homework, and the Retriever Had a Real Bug
I checked where the bundled benchmark's "clear" queries came from. They are close paraphrases of the corpus documents themselves: the query "apple stock company share price" shares four of five content words with the document text it is supposed to retrieve. That measures whether TF-IDF can match a query to a document built out of the query's own words. It says nothing about whether the agent understands an actual question.
So I wrote seven questions the way a person actually asks them, unambiguous, no shared vocabulary with the docs. Top-1 retrieval accuracy: 3 of 7. The retriever had no stopword list at all. "how", "an", "in", and "are" were scored by IDF exactly like "apple" was, and because a common word can appear in only one or two documents by coincidence, it can end up rarer, by the math, than a genuine content-word match. "How many calories are in an apple" retrieved the entry about pythons, the snake, because the single word "how" happens to appear once in its text ("how fast it moves").
The clarification gate inherited the same brittleness from its own hardcoded disambiguator list. "Calories" isn't one of the ~25 words on it, so a question that is not remotely ambiguous to a human still triggered a clarifying question. Across eight naturally-phrased clear queries containing a known trap word, it asked on 7 of 8.
The fix strips stopwords from both indexing and querying, enriches each document's terse one-sentence description with the vocabulary a real question about that topic actually uses (written from general knowledge, not reverse-engineered from the test set), and widens the disambiguator list to match. Held out and evaluated exactly once, after the fix was frozen: top-1 accuracy 0.29 to 1.00, gate F1 0.46 to 0.75, unnecessary asks 7 to 2. The original retriever and gate are untouched in parallel _v2 modules, so the published 100%-accuracy, 38%-fewer-questions benchmark above still reproduces exactly.
20 tests green across Python 3.9 through 3.13, with the example and both benchmarks on every push.