Same question, different numbers, wrong answer
"Allocate 400 units" and "allocate 1,000 units" embed almost identically. A cache that matches on similarity alone will serve one manager another manager's numbers.
Same question, different numbers, wrong answer. That’s what happened when a supply-chain cache matched two queries on similarity alone.
Why it matters
Caching is how fast systems stay fast: store the answer, serve it again when someone asks the same question. The standard method converts each question into a numerical fingerprint and checks how closely a new one matches a stored one.
Here is where it breaks. “Allocate 400 units from Supplier A” and “Allocate 1,000 units from Supplier A” produce nearly identical fingerprints. The meaning is the same. The numbers are different. The correct answer changes completely.
When cached answers feed real procurement decisions, a wrong cache hit means a procurement manager approves a supplier allocation based on someone else’s numbers, and the variance shows up in next quarter’s inventory audit.
For the manager signing off on that allocation, a cache that is wrong fast is worse than a system that is slow and correct.
How it works
The default in production caches is cosine similarity alone, with the threshold set high enough to feel safe. Entity-aware guards are how you close the gap that threshold can’t. In a domain where “400 units” and “1,000 units” embed almost identically, raising the threshold alone doesn’t fix it.
I built this fix into a LangGraph supply-chain ERP copilot. It runs two layers of matching.
First, an exact text match. Hash the question, look it up. Same question, stored answer.
Second, a meaning match with a guard. Compare fingerprints at 0.95 similarity. Before accepting, verify both questions contain the same numbers and entity codes. “400 units” against “1,000 units” fails the guard. “Supplier A” against “Supplier B” fails. Same numbers, same codes, different wording? That’s a genuine match.
Some answers never get cached at all: those awaiting manager approval, because they carry a live decision ID, and those produced during a model outage.
What surprised me: the guard that extracts numbers and identifiers took longer to build than the similarity search. The original cache was 24 lines. The production version is 201. In a domain where a wrong answer moves money, I tuned this cache for correctness over hit rate, and tested the guard for exactly that.