Why do people say AI codegen has improved?

Since at least 2023, there’s been a persistent trend of people saying that “AI code generation is much better now than it was six months ago.” There are multiple possible reasons why someone might say this, and a neutral observer should consider all of them.

I don’t believe the phenomenon can be fully explained by any one reason. Rather, I believe all of these reasons have, to varying extents, played a role.

1. Face value: AI has actually improved the rate at which it generates correct, high-quality code.
2. Null hypothesis: The user is simply mistaken. Due to human error, they believe AI has improved even though it hasn’t.
3. Skill development: The user has become more skilled at using AI, and is misattributing the improvement to AI itself.
4. Familiarity bias: The user has become more accustomed to AI-generated code, which makes them think it’s better when it’s actually just more familiar.
5. Illusion: AI has not improved at generating high-quality code, but has improved at persuading the user that it does, e.g. by mimicking common markers of high-quality code without reproducing the quality of thought normally attributed to them.
6. Anecdotal: Due to the inherently chaotic nature of AI output, the user sees a trend in their personal experience (a linear regression over random data) which isn’t representative of any fundamental change.
7. Survivorship bias: During a hype cycle, those with positive experiences are incentivized to speak out. If the user believed AI had gotten worse over time, they would have quietly moved on.
8. Sunk cost fallacy: The user has invested significant time and money into AI tools, predisposing them to give a positive assessment of it.
9. Personal sensemaking: The user previously had a negative opinion of AI code generation but no longer does; they feel compelled to make their change in perspective seem rational, regardless of the actual basis for it.
10. Rationalization: The user has an unhealthy AI habit (or, alternately, recognizes the technology’s harmful effects) and is trying to rationalize their use by focusing the conversation on its usefulness or growth as a technology, rather than its externalities.
11. Low specificity: The user is only considering the speed at which they complete tasks, not the quality or coherence of the results.
12. Ignorance of long-term effects: The user is only assessing new code, and hasn’t had time to observe the long-term resilience and viability of AI-generated code.
13. Halo effect: The user enjoys using AI or has a positive impression of it in general, and is overapplying that impression to all aspects of the technology.
14. Reasoning by ego: AI has improved at flattering the user, which gradually influences their emotions and compromises their ability to be objective about its output.
15. Loss of discernment: The user’s programming skills have atrophied from lack of use, such that they’ve lost the ability to judge between low-quality and high-quality outputs.
16. Detachment: The user has lost interest in discerning between low-quality and high-quality outputs, or they no longer meaningfully inspect AI output at all.
17. Group psychology: Public admiration of AI is a shibboleth in the tech industry, so the statement is only meant to affirm one’s in-group status; it has no literal referent.
18. Identity: The user is projecting a socially constructed identity—whether personal, political, or professional—that they understand to be AI-inflected. For example, they want to be seen as an “AI thought leader” or “power user.”
19. Duress: The user is under pressure from shareholders, managers, or peers to show their alignment with AI.
20. Reasoning by effect: The user wishes for AI use to become more widespread (thus advancing their financial, professional, political, or other goals), so they are speaking to persuade others, not to communicate their genuine experiences.