comprehension debt

comprehension-debt.md

background

the (known) origins of the term "comprehension debt"

Jeremy Twei coined the perfect term for this: comprehension debt. It’s certainly tempting to just move on when the LLM one-shotted something that seems to work. This is the insidious part. The agent doesn’t get tired. It will sprint through implementation after implementation with unwavering confidence. The code looks plausible. The tests pass (or seem to). You’re under pressure to ship. You move on.

symptoms

• volume & verbosity inflation; PRs grow in size, making diffs harder to review & reason about.

• shift from “author knows” to “author curated”; folks merge code they steered but didn’t fully internalize - particularly when delivery pressure is high.

• PR reviews trend towards “syntax checking” over “model checking”; reviewers fail to grasp deep system invariants/regressions while spending time on surface-level issues.

• context rot; long agentic runs accumulate contradictory approaches which confuses author & reviewers.

• weak ownership; if “who understands this end-to-end” isn’t explicit, everyone assumes someone else does.

approach to mitigate

• robust PR reviews; reviewers follow conventional commits. Authors structure changes into atomic commits & stacked PRs. Complex PRs include analysis docs summarizing the nature of the effort. We expect that reviews may take time.

• AI-augmented review; we use LLMs to outline module hierarchies, suggest refactors, and verify assumptions when reading other contributors' code.

• quick feedback loops; we invest in ensuring that we are able to have automated validation of business logic & outcomes.

• ownership; we expect team members to be able to articulate their work and ship in reviewable chunks.

• metrics; as necessary, track review iteration rate, follow-up PRs per PR, Revert rate / hot fixes per PR/epic.

---

related reading

 We find that AI use impairs conceptual understanding, code reading, and debugging abilities, without delivering significant efficiency gains on average.

• "How AI impacts skill formation"

Over time, overwork can impair judgment, increase the likelihood of errors, and make it harder for organizations to distinguish genuine productivity gains from unsustainable intensity.

• AI Doesn’t Reduce Work—It Intensifies It

The fundamental challenge is that organizations cannot optimize for what they cannot measure. Velocity is measurable. Comprehension is not, or at least not through any mechanism that currently feeds into performance evaluation, promotion decisions, or headcount planning.

• Cognitive Debt: When Velocity Exceeds Comprehension | rockoder

https://aphyr.com/posts/418-the-future-of-everything-is-lies-i-guess-work

https://www.404media.co/software-developers-say-ai-is-rotting-their-brains/

"Bosses Horrified as “AI Native” College Graduates Hit the Workplace
"Massive numbers of students are going to emerge from university with degrees, and into the workforce, who are essentially illiterate.""
https://futurism.com/future-society/college-critical-thinking-ai

https://hachyderm.io/@mitchellh/116580433508108130

I strongly believe there are entire companies right now under heavy AI psychosis and its impossible to have rational conversations about it with them. I can't name any specific people because they include personal friends I deeply respect, but I worry about how this plays out.

I lived through the great MTBF vs MTTR (mean-time-between-failure vs. mean-time-to-recovery) reckoning of infrastructure during the transition to cloud and cloud automation. All those arguments are rearing their ugly heads again but now its... the whole software development industry (maybe the whole world, really).

It's frightening, because the psychosis folks operate under an almost absolute "MTTR is all you need" mentality: "its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!" We learned in infrastructure that MTTR is great but you can't yeet resilient systems entirely.

The main issue is I don't even know how to bring this up to people I know personally, because bringing this topic up leads to immediately dismissals like "no no, it has full test coverage" or "bug reports are going down" or something, which just don't paint the whole picture.

We already learned this lesson once in infrastructure: you can automate yourself into a very resilient catastrophe machine. Systems can appear healthy by local metrics while globally becoming incomprehensible. Bug reports can go down while latent risk explodes. Test coverage can rise while semantic understanding falls. Changes happens so fast that nobody notices the underlying architecture decaying.

I worry.

https://mastodon.social/@pojntfx/116580516167414646

One of the best descriptions I've heard lately was that it feels like "losing coworkers to dementia" as people adopt it, where everyone feels like they know everything, but when you talk with them in person or there is a problem that needs to be fixed now it becomes very clear that the capability to do that has atrophied basically completely

https://discuss.systems/@tobinbaker/116580627094675024

that's an interesting analogy, feels like both vibecoding and "resilience engineering" tend to mask systemic risk by superficially and temporarily mitigating the symptoms

https://mastodon.social/@slacy/116580749844092232

story I've heard is the "well, we just rewrite the entire thing every six months so there's no point in fixing/improving because the next iteration/generation will be that much better as the agents improve."

I can actually sort of see this, and it's somewhat along the lines of "spec-driven-development" but ... ?

https://berlin.social/@jnfrd/116581079155866158

yep. Because specs that covered everything (aka waterfall) failed so perfectly before. Why not try it again, this time with an algorithm that has no comprehension to tell you that your spec is trash.

"Anti-distillation for employee Skills."
https://github.com/leilei926524-tech/anti-distill

"The AI Great Leap Forward
Apr 5, 2026 • Han Lee | 12 min read (2193 words)  

In 1958, Mao ordered every village in China to produce steel. Farmers melted down their cooking pots in backyard furnaces and reported spectacular numbers. The steel was useless. The crops rotted. Thirty million people starved.

In 2026, every other company is having top down mandate on AI transformation.

Same energy."

https://leehanchung.github.io/blogs/2026/04/05/the-ai-great-leap-forward/

"A non-anthropomorphized view of LLMs"
http://addxorrol.blogspot.com/2025/07/a-non-anthropomorphized-view-of-llms.html#:~:text=A%20non%2Danthropomorphized%20view%20of%20LLMs

Instead of saying "we cannot ensure that no harmful sequences will be generated by our function, partially because we don't know how to specify and enumerate harmful sequences", we talk about "behaviors", "ethical constraints", and "harmful actions in pursuit of their goals". All of these are anthropocentric concepts that - in my mind - do not apply to functions or other mathematical objects. And using them muddles the discussion, and our thinking about what we're doing when we create, analyze, deploy and monitor LLMs.

https://zirk.us/@LJ/116609499914539590

You learn by doing. With intentional practice.

That's it. The whole deal with creativity is that the process helps you grow. No process? No growth.

Using these tools shortcuts the process. They creat an outcome but don't create an artist. We can argue about quality from now to the heat death of the universe, but unless you engage in the process, you will never develop as an artist.

Whatever "voice" this LLM is leading the writer to, it sure as shit isn't their voice.

https://www.reddit.com/r/cscareerquestions/s/XBfsaiMpN0

"The search engine no longer says “here, go read what this person wrote.” It now says “here, I’ve already read it for you.” The contract is broken.

What's the deal when everyone can just access anything without caring about where it's coming from?

I, for one, enjoy a human touch. I enjoy craft and care. I enjoy the tiny details. I like the idea of a human putting in the work. Regardless of whether it's writing, speaking, recording... I'm online for seeing "the good stuff". And I agree with Mat's take.

why should anyone care about reading something nobody cared enough to write?"
https://webweekly.email/archive/web-weekly-193/

"But that output will be wrong in subtler and more nuanced ways across potentially fatter piles of code with more unrelated output and unwanted dependencies. It will require a more specialized set of skills to identify errors. And those errors will be different every time the code is generated."

https://adrianroselli.com/2026/05/maybe-dont-rely-on-googles-modern-web-guidance.html

#accessibilty

"My Students Can’t Read"

https://www.chronicle.com/article/my-students-cant-read

"The generational collapse in literacy is measurable, persistent, and likely to get worse."

Rhetoric and writing students that lose track reading a 20-page article. 😳

"When I assign analysis, I am not trying to extract a polished product; I am trying to put the student’s mind through resistance in order to make it stronger. Offloading the struggle to a chatbot does not “free students up for higher-order work.” It deprives them of building the strength to do any substantial cognitive work at all."

"The ability to read deeply and reason at length is fragmenting along class lines as ultra-processed digital media replaces text in everyday life, much as ultra-processed food has replaced cooking."

"My students from districts that protected sustained reading through small class sizes, strict phone policies, and faculty who refused to teach to the test all arrive with their attention relatively intact. My students from districts that surrendered to devices and standardized testing arrive cognitively winded. A democracy that requires a literate electorate is now training one fraction of that electorate out of literacy while marketing to the other a “deep work” lifestyle as a luxury good. The students who cannot read a 20-page article today are the voters who will not be able to read a bill, or the jurors who cannot follow a closing argument, tomorrow."

"Benjamin Piel, Chefredakteur vom "Weser Kurier", schreibt in einem Kommentar für "Kress" über die "Dumm-Nutzung" und "Schlau-Nutzung" von KI-Systemen.

Er schreibt:

"Die Dumm-Nutzung der KI ist, (…), sie die Hausaufgaben erledigen zu lassen. Das bringt zwar verlässlichen Kurzfrist-Erfolg, führt aber zur Verblödung."

Aber er schreibt auch:

"Wer am Ende des Schreibprozesses eine intensive Redigatur verlangt, bekommt sie mit dem richtigen Prompt von kaum jemandem so gnadenlos und sortiert wie von der KI.""

https://www.mdr.de/altpapier/das-altpapier-ki-journalismus-einsatz-vertrauen-glaubwuerdigkeit-100.html

"CTOs Agree: Cognitive Debt Is the New Technical Debt"

https://shiftmag.dev/ctos-agree-cognitive-debt-is-the-new-technical-debt-10229/

"The room had seen the same pattern: teams adding features at a pace that would have been impossible two years ago, now dealing with the maintenance overhead that comes with it. Legacy code that was already hard to understand is now harder, because the people who wrote it aren’t being careful."

"write whatever you want, but writing it doesn’t mean you’re shipping it. Code is cheap, but launching it isn’t."

"if anyone can ship, whose job is it to hold everything together? Several people in the room said the only realistic response is radical team autonomy: small groups of three to five people who own their own decisions, with management’s job shifting to alignment rather than gatekeeping."

"The deeper point was about abstraction layers. We don’t read assembly, we trust compilers. The question is whether you can build enough validation infrastructure, feature flags, observability, acceptance tests, mutation testing, to make a similar trust relationship work with AI-generated code."

"The cost of building AI features isn’t the hard part. The cost of verification is. If you build a solid eval suite today, you can swap providers, survive model deprecations, or move to open source without starting from scratch"

"To a senior developer, AI is a competent, fast-working junior or mid-level developer that, when properly directed, produces mostly solid work but lacks the institutional knowledge and the deep and broad systems-level knowledge that you have developed over the past twenty years."

https://adiamond.me/2026/06/software-engineering-in-the-age-of-ai/

"The novelist, like the software engineer, is no longer deeply engaged in her work. Editing is not creating. You do not give yourself over to your imagination. You do not immerse your mind and feelings in the process of invention. Instead, you’re rooting out problems, trying to clean up clumsy wording and redundant descriptions instead. The flow state is gone. You are now a cog in a larger process that doesn’t really value your creativity or your need to exercise it.

Worse still–and I have felt this personally after months of reviewing AI-generated code–your skills drop off sharply."

https://news.ycombinator.com/item?id=48710554

I write code myself and use the LLM to find mistakes then fix them manually. I recommend inverting the conventional wisdom on LLMs:

1. Don't use it to write code. It's a terrible
   programmer. But it's an intelligent rubber duck
   and a solid analysis tool.

2. Write the code yourself. It'll go faster than
   figuring it out as you go along. It's just typing
   if you have a plan.

3. Use an LLM to sanity check what you wrote.
   It can find potential design issues or future
   problems or breaks from convention. Decide based on
   your experience how much you want to address these
   issues before moving on.
   Is this as fast as AI writing the code? Not at first but possibly over time. Maybe faster. Certainly better quality.
   Plus you don't lose any of your skills in the process.

You're still doing the gruntwork necessary to keep your skillset.

And the token cost is a small fraction of what it takes to get LLMs to write good code.

"The alternative he proposed is a ratio: customer value delivered against the total cost of producing it, headcount, tooling, and token spend included. DORA metrics and PR throughput are not useless, he argued, but they measure the inside of the system, not its output. Treating them as primary goals disconnects engineering effort from the outcomes the business actually cares about."

"analysis found that when controlling for pull request size, AI has a negligible independent effect on defect rates. The actual driver is that AI generates larger pull requests, and larger pull requests correlate with more bugs. That is, in principle, actionable: you don’t need to focus on the AI generated code, but to the PR scope discipline."

"Code review is absorbing the strain more visibly. Poletto cited 30 percent more rework time on AI-generated code compared to human-generated code, along with more review round trips."

https://shiftmag.dev/ai-generates-larger-pull-requests-larger-pull-requests-bring-more-bugs-9932/

“The craftsmen are tired,” he wrote. “Day after day, their workload grows. Bugs seep into production. No one seems to care. Another round of AI is thrown at it. Their animosity to their colleagues rises.” 

“Eventually, they give up,” he added. “The craft they loved is dead.”

https://futurism.com/artificial-intelligence/software-engineers-crisis-drown-ai-code

"Anthropic is actively campaigning to end software engineering. They need you to believe they can do that. Well, maybe it’s not you that they need to convince. Maybe it’s your C-Suite, various world leaders, or the manager of your retirement fund. They’ve raised $132 billion in investment, and are approaching an IPO valued over $1 trillion. Since they cannot show profitability, this depends on selling their hypothetical future impact."

https://raymyers.org/post/zig-creator-calls-spade-a-spade/

"Anthropic needs people to believe that all problems can be solved by adding more agents, even the problems that agents introduce. Instead of better tools we buy tokens. Instead of learning skills we paste a SKILL.md. Instead of reading about The Psychology of Software Teams we call some parallel agent sessions a Team."

AI Coding will Prevent Expertise

The need for ongoing friction in long-term skill formation.

https://larsfaye.com/articles/ai-coding-will-prevent-expertise

"Coding/programming/software is a unique intersection of logic, math, problem-solving, critical thinking, planning, communication, and creativity."

"My AI-assistance checklist:

• If I did not have access to an AI tool, could I still accomplish this task?
• Am I using the model to deepen my understanding, or expedite the answer?
• If I had to audit and verify the generated output, could I adequately explain what was happening?
• If I'm learning a new concept, have I done proper research to know the right questions to ask?
• Have I cross-referenced and verified the approach through other methods (reading documentation, standard search tools, StackOverflow, Reddit)?
• Is this a truly rote task that's been done 100 times before, or a task that requires executive decision-making somewhere in the process?"

"The key is to detect the difference between cognitive debt and cognitive offloading: Cognitive debt is abdicating your judgment and decisions, whereas cognitive offloading is delegating the mechanical or tedious."

"The realignment I hope to see over the years is the understanding that skills don't develop without active participation. You must engage directly and continously to experience the essential friction that culminates in expertise (even if it means moving more slowly)."

"The problem is that code produced by an AI agent looks reasonable, but can contain ‘hard-to-spot bugs’ that end up causing major problems. As a result, you should carefully review your agent’s output, but this is difficult. As the engineer told me, it’s “famously hard” to understand code you didn’t write yourself, so this extra step becomes “easy to just blow it off (especially when we are all trying to ‘10x’ our velocity).” Soon, systems start to break.

“The coding harnesses are useful and make life as a developer easier,” he summarized, “but they also encourage laziness.”"

https://calnewport.com/on-ai-coding-and-its-discontents/

"The skills that didn't go anywhere"

https://ryangjchandler.co.uk/posts/the-skills-that-didnt-go-anywhere

"The codebase is a fixed input, but everything around it changes weekly. Priorities shift, a customer churns, someone makes a decision in Slack that quietly invalidates the plan from a fortnight ago. You're carrying a model of the system that updates continuously, and the agent starts every session with none of it.

So you're not just handing over a problem. You're handing over the conditions that make one solution correct and another one technically fine but wrong for you."

"“Writing is a technology for thinking,” cognitive psychologist Ronald T. Kellogg told the New York Times in a new essay mulling the consequences of sidestepping the hard work of writing.

Writing, the piece notes, helps develop all sorts of cognitive faculties, including building our working memory, executive planning skills, and our metacognition, or the ability to be aware of your own thinking."

https://futurism.com/future-society/students-lose-ability-think-ai

https://correctiv.org/aktuelles/bildung/2026/08/28/wie-verhindert-werden-soll-dass-schueler-durch-ki-das-lernen-verlernen/

"This Fence Has No Farmer
Every developer inherits code nobody can explain. The old assumption, that someone had a reason for it, is getting harder to trust. Pondering Chesterton's Fence and AI code."
https://adamgreenough.net/blog/this-fence-has-no-farmer/#:~:text=This%20Fence%20Has,and%20AI%20code.

"We're creating more unexplained code at the same time as we're creating better tools for explaining code. The same thing that's making the archaeological layer thicker is giving us a pretty good shovel.

Which ties back nicely to something I wrote a while back, Our Clients Are Using AI Now: users were never going to wait around for developers to bring AI into the workflow, and plenty of them are already in there themselves. This is just what that looks like a few months on, showing up as actual code in actual production sites, WordPress or otherwise.

The bit of the job that still matters most was never writing the code in the first place. It's knowing what you're looking at, working out what it does, deciding whether that fence still needs to be there and, increasingly, accepting that there may not have been a farmer at all."

"I stopped using the AI chat because it became very clear it wasn't helpful or healthy, and it was in fact dangerous. But I kept using AI for coding at work, using it to take on more and more of the heavy-lifting to the point where I was just a code reviewer and a quality assurance tester for the features the coding harness would spit out. Getting a lot of tasks done, it can feel exhilarating. That is, until I realized, well, I'm basically just a hamster on a wheel, checking boxes, getting tickets done with no real purpose. Sure, I catch an issue here or there. But it's impossible to review the quantity of code the AI writes, and we've got a different AI to review it anyway! So what did this do? Well, it made me lazy. It made me stop caring. It made me a worse programmer. It made me depressed. Because I stopped doing the hard work, I stopped learning, I stopped growing, I stopped being the one making the software."

https://brettcodes.com/im-done-using-ai/#:~:text=I%20stopped%20using,making%20the%20software.

So in summary: using AI for coding led to depression, apathy, existential dread, and worse software. Not good. Add on top of that the negative environmental, community, and economic impacts of AI, and it became extremely apparent to me that something's gotta change.

But there is now a scenario in which an autonomous AI harness, backed by an enormous amount of computational resources, performs this entire iteration internally, and ends up producing the final ansatz, and thence the solution to the Navier-Stokes regularity problem, while the AI company running the harness keeps the process to arrive at that ansatz almost completely out of public view. Technically, one of the most prominent open problems in mathematics would now be solved; but there would be almost no value added to mathematics as a consequence. It is theoretically possible that with some herculean (and heavily AI-assisted) additional effort by a third party, some portion of the process could be reverse-engineered to recover some actual insight and understanding from the solution; but this would be a far less efficient process than if the solution had been obtained via a diverse combination of both human mathematicians and machine assistance as mentioned above. (5/6)

https://mathstodon.xyz/@tao/117207855800042681