The Opposite of the Intern Who's Never Once Said "I'm Not Sure"
You've worked with the intern who's never once said "I'm not sure." Every answer arrives with total conviction, whether it's right or dead wrong. An auto-tagger that always guesses has the same problem: it is wrong exactly as often as it's overconfident, and it never tells you which one it's being right now.
taggate scores its own confidence as the margin between its top and runner-up category, not just whether it found a match at all, and routes anything below threshold to a human instead of silently mislabeling something genuinely ambiguous. A "smart watch band" is legitimately both electronics and apparel. No keyword scorer should be forcing a confident guess there, and neither should a person rushing through a queue.
Technical Architecture
Built in TypeScript with zero runtime dependencies:
• Margin-based confidence: a category that wins 1-to-0 with no real competition scores differently than one that wins 5-to-4, so a genuinely contested item can't sneak past the gate on a single lucky keyword hit.
• Confidence-gated pipeline: auto-applies the tag when confidence clears the threshold, otherwise escalates to a pluggable HumanReviewer interface.
• Bring your own tagger: the deterministic keyword-overlap tagger swaps out for a real LLM behind the same Tagger interface, with the confidence-gating and escalation logic unchanged.
• Bring your own reviewer: OracleReviewer is a test stand-in; a real deployment wires in a Slack approval flow or a ticketing system instead.
Challenges & Solutions
The design decision that mattered most was rejecting a naive "did I find any keyword match" confidence signal in favor of the margin between the top two categories. Early versions treated any match as confident, which meant a product with one incidental keyword hit for the wrong category would get auto-tagged with total certainty. Scoring the gap between the top and runner-up category instead of the raw match strength is what actually catches the ambiguous cases.
I also picked a genuinely mixed catalog for the benchmark (12 products, 4 deliberately ambiguous) instead of an easy set where every item has one obvious category, since a benchmark that never tests the gate's judgment doesn't prove anything about the gate.
Impact & Results
Always-auto-tagging the 12-product benchmark catalog scores 75% accuracy, with the ambiguous quarter guessed wrong at full confidence every time. Confidence-gating escalates the ambiguous 42% to a human reviewer and reaches 100% accuracy.
8 tests pass across the full CI matrix (Node 18, 20, and 22), plus a typecheck and build step so `noUnusedLocals`/`noUnusedParameters` catch dead code before it ships.