The Trustworthy Negative
Maya
Why “not found” is the hardest answer to build — and the only one that matters.
By VK Singh Vashisht — Datura Labs
Datura Labs runs on four laws. The third one reads: “Not found” is a calibrated answer, engineered to be as reliable as “found.”
I wrote that law before I understood what it would cost to actually keep it. This is what that cost looked like.
The easy version of trustworthy
Anyone can build a system that returns results. Ask it a question, it answers. Feed it a document, it processes it. Point it at a folder, it reports what’s inside. The demo works. The stakeholder nods. Everyone moves on.
The hard version — the one that actually matters — is a system that can say “I don’t have that” and mean it. Not as a fallback when something breaks. As a first-class, deliberately engineered answer that is exactly as reliable as a positive hit.
Building TalkingDB — the retrieval layer now running across roughly two million records — taught me that this property doesn’t happen by default. It has to be hunted for, specifically, because the default failure mode isn’t a system that crashes. It’s a system that reports success while quietly failing.
The health check that cannot fail
The first one nearly got past me entirely.
A health check ran a count against a search index and reported it as fully populated — every time, cheerfully, correctly formatted. What it was actually counting was the content table underneath the index, not the index itself. The shadow tables that make full-text search work could have been completely empty and the health check would never have known, because it was never actually asking the index a question. It was asking a related table that happened to hold a similar number.
The fix wasn’t complicated once found: a health check on a derived structure has to probe the structure directly — a real search query, not a row count on something that can silently proxy to the real answer. But finding it required not trusting a green checkmark, which is a strange discipline to build into your own tooling.
What “72% needing rotation” actually taught me
Scanned documents rotated 90 or 180 degrees don’t fail to extract text. They extract text — confidently, fluently, completely wrong. OCR doesn’t know it’s reading a sideways page. It reports a transcription with the same confidence it would give a clean one, and that transcription silently corrupts exactly the class of document most likely to matter: recorded instruments, filings, anything load-bearing.
The number that stuck with me: an early sample of 75 files suggested 72% needed rotation correction. The real rate, once we’d run it against the full 1,200-plus file set, was 21%. Both numbers were honestly measured. Only one of them was true at scale. That gap — between a plausible early read and the real distribution — is its own lesson: sample size isn’t a formality, it’s the difference between a fixable problem and a fiction you’ve talked yourself into.
The failure that only showed up in reality
We had a claim: the system degrades cleanly when a drive gets unmounted mid-operation. We’d tested it. The test passed.
The test pointed a cache at a directory that didn’t exist and confirmed the system handled the error gracefully. That’s a reasonable simulation. It is not the same event as a drive actually disconnecting mid-read, and when we finally forced a real unmount instead of simulating one, a second, different failure showed up — the degraded-mode fallback search quietly lost a piece of discipline the primary pipeline had been providing, and started returning weaker matches without flagging that it had done so.
The simulation wasn’t wrong. It was answering a different question than the one that mattered. A failure path you’re going to rely on in production has to be tested against the real condition at least once. A proxy for the failure is not the failure.
The pattern underneath all of it
Every one of these — and there are eleven of them in the running list we keep, not three — has the same shape. A system reported success while the actual signal was absent, degraded, or silently wrong. A passing metric. A plausible transcription. A test that verified the wrong thing convincingly.
The fix was never “trust the passing signal a little less.” It was to verify against the rawest available ground truth every time — the actual index, the actual physical failure, the actual bytes — instead of a structure that merely correlates with them. That discipline, more than any individual fix, is the reusable part.
This is why Law 03 isn’t decoration. A system you own is only as trustworthy as its worst silent failure, and silent failures don’t announce themselves — they have to be gone looking for, specifically, on the assumption that something is currently lying to you and simply hasn’t been caught yet.