PDFs Are Where Good Data Goes to Die
A number is born in a clean spreadsheet. Then someone pastes it into a report, wraps it in three paragraphs of prose, exports a PDF, and buries it forever. No markup. No boundary. Nothing telling a machine where the story stops and the numbers start.
I built a version of this for FDA regulatory submissions, and the failure that actually bites is never "can't read the PDF." It is a lazy parser swallowing the paragraph between two tables, shifting its header by one row, and quietly corrupting every number after it. Nobody notices until an audit. That is the expensive kind of silent.
tablextract finds the tables, ignores the prose, and answers your question with a citation back to the exact row and column. You get the number and the proof of where it lives.
It Reads Columns, Not Just Lines
The trick is looking at shape, not position.
• Block detection: it splits each line on pipes or runs of spaces, then groups the runs that share a column count into a real table. A lonely sentence never gets to pretend it is a data row.
• Confidence scoring: it keeps only the rows that match the block's dominant column count, so the prose that sneaks in gets thrown out on sight.
• Cited answers: ask a plain-English question and it returns the value plus the row label, column, and source it came from. Not a naked number you have to go verify yourself.
• Real PDFs: an optional pdfplumber path takes raw bytes over the API and returns a clean 400 on garbage input instead of exploding with a 500.
The Ugly Case Is the Whole Point
So the benchmark ships the ugly case on purpose: prose jammed directly between two tables, the exact shape that wrecked pipelines I worked on. A naive line-by-line reader merges that paragraph into the table, moves the header, and poisons every cell lookup that follows.
The base64 PDF endpoint taught me a smaller lesson the hard way. It first took the encoded file as a query parameter, which silently truncates a real document because URLs have length limits. Moving it into the request body killed a bug that only ever shows up in production, never in the tidy little local test with a two-page file.
The Number
On the bundled document (adverse-event and dosing tables wrapped in prose), the naive extractor scores 0% cell accuracy. Zero. tablextract scores 100%, 8 of 8, every value traceable to its source.
Then I Asked It About the Moon
Q: "What is Grade1 for the moon?"
A: 12 [cited to: Cohort A / Grade1]
Four of seven unanswerable questions came back with a confident, cited, wrong answer. For a tool whose entire selling point is a citation to the exact row and column, that is the worst failure available to it. A wrong answer with no citation gets checked by whoever reads it. A wrong answer carrying a row and column reference looks verified, and gets signed off.
The cause was arithmetic, which is what made it invisible. The scorer added a column-match score to a row-match score and returned the best total. So any question naming a real column cleared the bar on its own, with the row score sitting at zero. Ask about a cohort that doesn't exist and you get the first row's value, cited. The fix is to make it a conjunction: the question has to identify the row and the column, each above its own threshold. Ties are a refusal too, because matching two rows equally well means the question identified neither of them.
Refusals now carry the reason and the near-misses, since a refusal you can't act on is just a dead end: "no column in the document matches the question", near_misses: ["Dose_mg", "Frequency", "Grade1", ...].
The Extractor Had a Quieter Version of the Same Problem
That 100% was measured on a document where tables are separated by blank lines and prose never contains aligned whitespace. Real PDF text layers aren't like that. So I built five documents that behave the way real ones do, and the extractor lost two thirds of the rows in two of them, while reporting confidence 1.00.
That's exactly the audit failure this project's own README warns about, sitting inside its own extractor. Three causes, all of which a PDF does constantly: a CONFIDENTIAL - Page 3 of 12 banner printed between two data rows ends the block and discards everything after it; a cell too long for its column continues on the next indented line and truncates the table there; and a table continuing onto the next page becomes two tables, so a lookup for a row on page two fails against table one.
The rewrite gives every line a role, data, furniture, continuation, or prose, rather than assuming anything non-tabular ends the table. Cell recall went from 71% to 100%, and confidence now reflects how much of the block actually parsed instead of always reading 1.00.
One Bug, Two Opposite Symptoms
Building the abstention logic, the new version started rejecting real answers. It refused to tell me Grade2 for Cohort B, which is plainly in the table.
The cause was one line. The tokenizer stripped English stopwords from row labels as well as from questions, and "Cohort A" contains the article "a." Reduced to just {"cohort"}, that label matched a question about Cohort B or Cohort Z at full coverage. So it created ties that suppressed valid answers, and simultaneously let nonexistent rows score perfectly. Both bugs, one root cause.
Labels have no stopwords. The distinguishing token of a row label is very often a single letter or digit, and everything in a label is data. A second one surfaced the same way: joining a wrapped fragment to the last non-empty cell put "regimen per amendment v2" onto the dose column, turning "50" into a corrupted value that still looked like a clean extraction. Continuations now join by character offset against the header.
I built both replacements against my own adversarial corpus, so scoring 100% on it proves nothing. A separate document and question set written afterwards with the code frozen, run once: cells 2/4 → 4/4, questions 3/7 → 7/7, fabricated citations 3 → 0. 58 tests, and CI now fails the build on a fabricated citation or a recall regression.