PDFs Are Where Structured Data Goes to Die
I spent real time on this exact problem building a table-extraction pipeline for FDA regulatory submissions. The failure mode that actually costs you is never "can't read a PDF." It's a naive parser choking on the paragraph between two tables and misaligning every row downstream of it, silently, so nobody notices until an audit. Somewhere between the spreadsheet a table gets built in and the document it gets pasted into, every table ends up buried inside three paragraphs of narrative text with zero markup telling you where the prose stops and the numbers start.
tablextract is that lesson, rebuilt as an open, runnable service instead of proprietary pipeline code. It finds the table blocks, skips the prose, and answers questions against the extracted data with a citation back to the exact row and column it came from.
Technical Architecture
The extraction logic looks at column consistency, not just line position:
• Block detection: splits each line on pipe delimiters or runs of two-plus spaces, and groups consecutive lines with a consistent column count into a table block.
• Confidence scoring: keeps only rows matching the block's dominant column count, so a stray prose sentence never gets treated as a data row.
• Cited querying: answers a natural-language question against the extracted tables with the exact row label, column, and source document it came from, not just a bare value.
• Real PDFs: an optional pdfplumber-backed path takes raw PDF bytes over the API and returns a clean 400 for malformed input instead of a bare 500.
Challenges & Solutions
The honest failure case is the one worth showing, so the bundled benchmark uses a document with prose sandwiched directly between two data tables, the exact adversarial shape I dealt with professionally. A naive line-by-line extractor merges the prose paragraph into the table it's parsing, which shifts its header assumption and corrupts every single ground-truth cell lookup after that point.
Getting the base64 PDF endpoint right was its own small lesson: it originally took the encoded PDF as a query parameter, which silently truncates on a real-sized document because query strings have length limits. Moving it into the JSON request body fixed a bug that would only ever show up in production, never in a quick local test with a tiny file.
Impact & Results
On the labeled synthetic document bundled with the repo (adverse-event and dosing-schedule tables surrounded by prose), the naive line-splitting extractor scores 0% cell accuracy. tablextract scores 100%, 8 out of 8 cells, with every value traceable to its source row and column.
17 tests pass across the full CI matrix, plus a live Docker smoke test that boots the service and calls the real extract endpoint end to end.