One Model, One Shot at Whatever Blind Spot It Has Today
Ask one model a hard question and you get one shot at the truth, plus whatever bias that particular model happens to carry. The multiagent debate idea (Du et al., 2023) is the fix humans already figured out: a jury usually beats a single juror, not because any one juror is smarter, but because bad independent guesses rarely agree with each other while good ones tend to converge.
debatekit simulates that dynamic without calling a real model. Each agent is a noisy classifier with its own accuracy. Debate is rounds of exposure to the group's current answers, followed by revision weighted toward a clear plurality, then a final vote.
The Persuasion Knob Is the Interesting Part
Turn persuasion to 1.0 and the panel converges instantly to whatever the first-round plurality happened to be, which can lock in a wrong answer just as fast as a right one. Turn it to 0.0 and debate does nothing, it is just an expensive way to run a vote. The default of 0.6 models agents that take the group seriously without being pushovers, closer to how a real debate protocol actually behaves.
Stable Numbers, Not One Lucky Run
The benchmark averages 20 independent trials per question with plausible distractors, not random noise, so voting has to do real work to converge on the truth. One lucky draw does not get to write the headline.
The Number
At a deliberately mediocre 55% individual accuracy, a lone agent gets there 57% of the time. A panel of five with zero communication, just independent votes, already jumps to 79%: the classic wisdom-of-crowds effect. Letting the panel see each other's answers and revise for two rounds buys one more point on top, landing at 80%.
10 tests pass in CI across Python 3.9, 3.11, and 3.13.