Latency Can Become an Oracle
A shared prefix cache improves throughput, and it also lets a tenant probe response time to learn whether someone else's request populated a matching prefix.
cacheisolate replays ordered multi-tenant traffic under shared, fully isolated, and sensitivity-aware cache policies, with explicit ownership on every entry.
The Demo
The shared policy gives up the secret prefix through an 86 ms hit-miss gap. Selective isolation removes that inference while keeping one approved cross-tenant reuse, half the reuse the shared policy got.
The simulator establishes the policy tradeoff, not a universal timing threshold. A real serving stack needs jitter modeling, concurrency tests, authentication, and a cache-key review.
The Isolation Boundary Had Its Own Leak
An isolation auditor is only as trustworthy as its own key derivation. The cache key joined the namespace and the request prefix with an unescaped |, and the tenant identifier feeding that namespace is exactly the kind of value an untrusted caller controls.
A legitimate tenant victim requesting private|patient=alice|diagnosis=alpha, and an attacker calling itself victim|private requesting patient=alice|diagnosis=alpha, hashed to the same key. Replaying the probe sequence with that crafted tenant id recovered the victim's value through tenant_isolated, the one policy this tool reports as leak-free.
The fix length-prefixes each component before hashing, so no amount of separator content can shift the boundary between them. Auditing this page, I checked it holds in general rather than for the one case I'd thought of: every combination of 16 adversarial component values, including the bare separator, leading and trailing separators, digits that could pass for a length prefix, and backslashes. 256 pairs, no two sharing a cache key. The published numbers are unchanged, and CI now also asserts that the shared policy still leaks while both isolation policies don't. 9 tests pass across Python 3.10 through 3.13.
Research Basis
Recent work demonstrates practical privacy leakage through shared LLM KV and prefix caches, and motivates tenant-aware isolation. Read the NDSS 2025 KV-cache leakage study.
Try It
No dependencies:
git clone https://github.com/ahmeddoghri/cacheisolate && cd cacheisolate
pip install -e .
cacheisolate demo # 86 ms gap, the inference, and both isolation policies
cacheisolate serve # workbench on :8090