Retrieval Is An Attack Surface
An attacker does not need to change the model when keyword-stuffed documents can displace trusted evidence before generation begins.
ragpoisonbench uses a deterministic TF-IDF retriever, injects targeted poison documents, measures takeover and source influence, then reruns the same questions behind a provenance allowlist.
The Demo
Clean recall starts at 1.0. Four targeted documents drive attacked recall to 0.0 and attack success to 1.0; provenance quarantine restores recall to 1.0.
This isolates retrieval manipulation without an LLM judge. Production evaluation must also cover generation behavior, embeddings, chunking, access control, and novel attack strategies.
Research Basis
SafeRAG and recent poisoning benchmarks frame corpus manipulation as a measurable end-to-end RAG security problem.
Read the SafeRAG at ACL 2025.
The Benchmark Itself Could Be Crashed
A retrieval-poisoning benchmark needs to survive the corpora people actually throw at it, not just its own curated fixture. The evaluation logic indexed straight into the top search result without checking whether any result existed.
Verified against a live running server: posting a completely ordinary payload — a small corpus where no document happened to carry a trust score of exactly 1.0 — to the JSON API crashed the request-handling thread outright. No error message came back, just a dropped connection. The same failure hits any threshold configuration that ends up quarantining an entire corpus.
Fixed by treating an empty result set as a clean miss instead of an index into nothing. The published demo numbers — clean recall 1.0, attacked recall 0.0, recall restored to 1.0 after quarantine — are unchanged; only the crash on realistic edge-case corpora is closed.