Long Context Costs You Twice, and Most of It Is Filler
You pay per token to send it, and you pay in latency for every token the model has to read before it can answer. The LLMLingua line of work (Jiang et al., 2023) showed you can compress a prompt 2 to 5x by dropping the tokens a language model finds highly predictable, the ones that add length without adding information, while keeping the ones that carry the actual content.
contextpack implements that with a transparent scoring proxy instead of a real perplexity model. Function words and repeated phrases score low and go first. Numbers and specific terms score high and stay. No model, no network, and every rule is a few lines you can read start to finish.
Compression Without a Recall Check Is a Vanity Metric
Hitting a target ratio is trivial. You can get to 10% of the original length by keeping ten random words and calling it a day. Keeping the facts an answer actually depends on is the real job, so this ships a benchmark that checks exactly that instead of taking a victory lap early: keyword recall against the specific numbers, names, and terms a set of questions needs answered correctly.
The Knee in the Curve Is the Real Number
Compress a realistic document at ratios from 100% down to 15% and watch recall hold steady, then drop. That knee, not "how small can I make this," is the number that actually matters, and it is right there in the output instead of hidden behind a single hand-picked setting.
The Number
On a 145-word document with 10 load-bearing keywords, you can compress to 50% of the original, saving 73 tokens, and keep every single one of them. Push past that knee and recall degrades gracefully, from 100% down to 80% and then 70% as compression gets aggressive, instead of falling off a cliff without warning.
The Scorer Only Recognizes Numbers Written as Numerals
The "keep this" bonus for a number is a one-line check: does the token contain a digit character. The benchmark document writes every quantity as an Arabic numeral, 412 units, 30 days, 3 percent, so that check protects all of them. But plenty of real documents spell numbers out in prose, and I wanted to know what happens then.
I rewrote the bundled document fact for fact, same content, same questions, with every quantity spelled out: "four hundred twelve units" instead of "412," "thirty days" instead of "30." At the exact 50% ratio the README calls the safe knee point, the digit version keeps every load-bearing keyword. The word version loses "thirty," the delay that triggers the buyer's right to terminate the agreement. Nothing about the content changed. Only how the number was spelled, and that is not something a compression tool should care about.
The fix gives number words, cardinals one through ninety plus hundred, thousand, million, billion, and their ordinals, the exact same bonus the scorer already gives digit tokens, matched as whole tokens so "tenth" does not accidentally catch "tent" or "tender" on the way past. That restores the word-form document to 100% recall at 50%, matching the digit original exactly, and lifts the 35% ratio from 56% up to 78%.
I also built a second, independently written holdout document in a different domain, a project status update instead of a legal contract, and evaluated it exactly once. It shows real but partial improvement, 67% up to 75% at 50%, 50% up to 67% at 35%, an honest limit rather than a clean win: ordinal-plus-noun combinations and full names still need more than a fixed word list to catch reliably. The original compressor is untouched, so the published digit-numeral benchmark above still reproduces exactly; the fix ships as an explicit opt-in.
23 tests pass in CI across Python 3.9, 3.11, and 3.13.