Beyond Vibe Coding: Engineering Discipline for Clean AI Output

When Andrej Karpathy coined “vibe coding” in a February 2025 tweet, he was precise about what he meant. You “fully give in to the vibes, embrace exponentials, and forget that the code even exists.” You accept every diff without reading it. You paste error messages back without understanding them. The code grows beyond your comprehension. And — this is the part most people skip when they quote him — he was explicit that this works “for throwaway weekend projects.”

That last clause did not survive contact with the industry. Within a year, “vibe coding” had detached from its weekend-project caveat and become shorthand for any prompt-driven development, including the kind that ships to production and handles other people’s money, data, and trust. Simon Willison drew the line cleanly: vibe coding is “building software with an LLM without reviewing the code it writes.” The moment you review it, test it, and can explain it to a colleague, “that’s not vibe coding, it’s software development.”

The thesis of this post is simple and slightly unfashionable: the discipline did not disappear when the typing got easier. It moved. The craft of software engineering used to live mostly in writing code. It now lives mostly in specifying what you want and verifying what you got. Those are harder skills, not softer ones, and the teams that treat AI as a license to stop doing them are quietly accumulating a bill they will pay later, with interest.

What the data already says about unreviewed output

This is not a vibes-versus-rigor culture-war argument. There is evidence on the table, and it points the same direction.

GitClear analyzed 211 million lines of code across five years and found that the structural signatures of carelessness are climbing in lockstep with AI adoption. The share of copy-pasted lines rose from 8.3% in 2020 to 12.3% in 2024, and for the first time copy-pasted lines outnumbered “moved” (refactored) lines. Code churn — the proportion of new code revised or reverted within two weeks of being committed — grew from 3.1% to 5.7% over the same window. Duplicated code blocks of five or more lines increased roughly eightfold during 2024. None of this proves AI causes the decline on its own, but it is exactly the fingerprint you would expect when generation gets cheap and review gets skipped: more code, more duplication, more rework, less consolidation.

The delivery-level data agrees. The 2024 DORA report found that a 25% increase in AI adoption was “accompanied by an estimated decrease in delivery throughput by 1.5%, and an estimated reduction in delivery stability by 7.2%.” DORA’s interpretation is the load-bearing sentence: “improving the development process does not automatically improve software delivery — at least not without proper adherence to the basics of successful software delivery, like small batch sizes and robust testing mechanisms.” AI is not a panacea. It is an amplifier, and an amplifier with no discipline upstream just makes the noise louder.

And then there is the productivity illusion. In METR’s 2025 randomized controlled trial, 16 experienced open-source developers working in repositories they knew well were actually 19% slower when allowed to use early-2025 AI tools — even though they believed the tools had sped them up by about 20%. The gap between felt speed and measured speed is the whole problem. Vibe coding feels fast. It feels fast precisely because you have removed the steps — reading, testing, verifying — that make it slow and make it correct. The feeling is real. The speed is not always there.

It is worth being careful about what METR’s number does and does not mean. The study was deliberately scoped to expert developers in large, mature repositories they already understood deeply — exactly the setting where a human’s existing mental model is most valuable and an AI’s context is most likely to be incomplete. It is not a universal claim that AI slows everyone down everywhere; results almost certainly differ for unfamiliar codebases, greenfield work, or boilerplate-heavy tasks. The durable lesson is not “AI is slow.” It is that your perception of speedup is an unreliable instrument. If the people most likely to benefit can be wrong about it by nearly forty percentage points, then “it feels faster” is not evidence of anything. You need to actually measure, and you need to measure the right thing — not keystrokes saved, but work delivered and defects avoided.

The craft moved; it did not vanish

Fred Brooks gave us the vocabulary for understanding why AI does not rescue us in his 1986 essay “No Silver Bullet.” Brooks split software difficulty into two kinds. Accidental complexity is the friction of the tools and representations — fighting the compiler, wrestling boilerplate, remembering syntax. Essential complexity is the difficulty inherent in the problem itself: specifying behavior precisely, getting the data model right, handling the thirty things the program genuinely has to do. His argument was that we had already squeezed most of the accidental complexity out, so no single tool could deliver an order-of-magnitude gain, because what remained was essence.

AI is the most powerful accidental-complexity reducer we have ever built. It makes the typing nearly free. But Brooks’s point lands harder than ever: the typing was never where the hard part lived. Deciding what the system should do, how the pieces fit, and whether the result is correct — that is essential complexity, and an LLM does not remove it. It relocates your job to the two endpoints that Brooks would recognize as the real work: specification and verification. Vibe coding is the fantasy that the middle (generation) is the whole job. It never was.

So the discipline reasserts itself in a recognizable shape. Here are the practices that hold up.

Spec-first, intent-first prompting

The single highest-leverage move is to write the specification before you write the prompt. Not a novel — a tight statement of intent: what the function does, what the inputs and outputs are, what the edge cases are, what it must not do. A vague prompt produces plausible code that solves a problem adjacent to yours. A precise prompt produces code you can actually check against a stated intent.

This is also where the skill genuinely shifts. The bottleneck used to be expressing intent in a programming language. Now the bottleneck is expressing intent clearly enough in any language that both you and the model agree on what “done” means. If you cannot specify it, the model will happily guess, and you will spend your “saved” time debugging the guess. We have written more on this in the AI prompting skills every developer now needs.

A practical heuristic: if you could not hand the same prompt to a competent contractor and expect roughly the right result, the model has even less context than the contractor would, and it will fail in quieter, harder-to-spot ways. Constraints are not bureaucracy here; they are the spec. Naming the libraries you want used, the patterns the codebase already follows, the error-handling convention, and the things that are explicitly out of scope is what converts a generation into a verifiable deliverable. The clearer the intent, the smaller the gap you have to close at review time — which is the whole game.

Small, reversible diffs

DORA’s research has held for years that small batch sizes contribute to both higher throughput and higher stability — the two are not a tradeoff. This finding predates AI and survives it. The danger of a capable model is that it will cheerfully generate four hundred lines across nine files in one shot, and the social pressure to just accept it is enormous because it looks done.

Resist. The discipline is to keep changes small enough to review in one sitting and revert in one command. If the AI proposes a sprawling change, that is a signal to decompose the task, not to relax your review. A small diff is a reviewable diff. A reviewable diff is the only kind you can honestly approve.

Test-driven development, with the AI inside the loop

Kent Beck’s red-green-refactor cycle — write a failing test, write the minimal code to pass it, then clean up — turns out to be almost purpose-built for AI collaboration. The failing test is an executable specification. It is an unambiguous, machine-checkable statement of intent that does not care about the model’s confidence or your own optimism.

When you let the AI generate the implementation and an independent test suite asserts the behavior, you have converted “trust the vibes” into “trust the green bar.” The ordering matters: write or at least review the tests yourself, because a model asked to write both the code and its tests will tend to write tests that pass rather than tests that matter. The test is your leverage point. Hand it away and you have automated the wrong half.

There is a subtler payoff, too. Writing the test first forces you to confront the essential complexity Brooks described before a single line of implementation exists. You cannot write a meaningful assertion without first deciding what the correct behavior actually is — what the boundaries are, what should happen on bad input, what the contract with the rest of the system looks like. That act of deciding is the real engineering, and it is exactly the step vibe coding skips. The model is extraordinary at producing an implementation once the target is unambiguous. It is far less reliable at inventing the target for you. Tests are how you keep that decision in human hands while still letting the machine do the typing. For the broader version of this argument, see our AI-generated code testing strategy.

Read every diff

This is the non-negotiable one, and it is the exact line Willison drew between software development and vibe coding. Reading the diff is not a formality. It is the moment where essential complexity gets verified by a human who is accountable for the result. The METR slowdown and the GitClear churn both trace back to the same root: code that entered the codebase faster than anyone understood it.

Reading every diff has a second, underrated benefit — it keeps your mental model of the system intact. The most expensive failure mode of AI-assisted work is not a bug; it is a team that no longer understands its own codebase because no human ever read the code that built it. Once comprehension drifts away, every subsequent change gets slower and riskier, which is precisely the doom-loop pattern worth designing against.

Treat the AI as a fast, tireless junior

The most useful mental model is the one most senior engineers already have muscle memory for. A capable junior produces a lot of plausible code quickly, is occasionally brilliant, frequently confident when wrong, and has no stake in the long-term health of the system. You would never merge a junior’s pull request unread. You would never let a junior’s output define your test cases. You would never assume a junior understood an ambiguous ticket without confirming. Apply exactly that posture to the model and most of the failure modes evaporate.

This framing also resolves the seniority question. AI does not make junior engineers obsolete; it makes the reviewing skill — the senior skill — more valuable, because there is suddenly far more output that needs a discerning reviewer. The people best equipped to lead this transition are the ones who already have that judgment, not the ones who can type the fastest.

Where vibe coding actually belongs

None of this is an argument against vibe coding. It is an argument for putting it where Karpathy originally put it. Throwaway prototypes, one-off scripts, spikes to learn an unfamiliar API, internal tools with a blast radius of one person, weekend experiments where the cost of being wrong is your own Saturday afternoon — vibe coding is genuinely excellent there. The whole point of a prototype is to discover what you want by building something disposable, and stopping to write tests for code you are about to throw away is its own kind of waste.

The mistake is not vibe coding. The mistake is the silent promotion: code that was vibed into existence as an experiment, never re-engineered, then quietly shipped because it “worked on my machine.” The transition from prototype to production is exactly the moment to switch modes — to add the spec, the tests, the small diffs, and the review that the prototype legitimately skipped. Teams that name this transition explicitly avoid the trap. Teams that pretend the prototype was production-grade all along inherit the GitClear churn numbers. If your organization runs on AI-assisted work at scale, it is worth codifying this boundary the way you would any other engineering standard — making the mode-switch a shared norm rather than a personal habit, so the move from draft to production is a deliberate decision rather than an accident nobody noticed.

Discipline you can actually see

There is a reason “just be more disciplined” usually fails as advice: discipline that is invisible does not survive deadline pressure. If nobody can see whether diffs are getting reviewed, whether tests are getting written, whether change sizes are creeping up, then the disciplined path and the vibe path look identical from the outside — right up until an incident makes the difference obvious.

This is where measurement earns its place, and it is worth being honest about the trap. The goal is not to turn “lines reviewed” or “tests written” into a target, because the moment a proxy becomes a target it stops measuring what you cared about — the familiar shape of Goodhart’s Law. The goal is to make the shape of the work visible: are change sizes staying small, is test coverage moving with the code, is acceptance happening after review or instead of it. Those are diagnostic signals, not scoreboards. We have written about which signals are worth watching in the right AI metrics that actually matter, and the throughline is the same one DORA keeps confirming: stability and speed come from the fundamentals, and the fundamentals are observable if you choose to observe them.

The point of looking at this data is not to police engineers. It is to catch drift early — to notice the week that diffs started getting bigger, the sprint where churn ticked up, the moment the felt-speed-versus-real-speed gap started to widen — before it compounds into a codebase nobody understands. Visibility is what lets a team keep the upside of AI without sleepwalking into the downside the research keeps documenting.

The Takeaway

Vibe coding is not the enemy. Treating vibe coding as a destination instead of a draft is the enemy. The evidence is consistent across very different methodologies: GitClear sees the structural decay of unreviewed output, DORA sees the delivery-level cost of skipping the fundamentals, and METR sees engineers who feel faster while measurably going slower. They are all describing the same gap — the one between code that exists and code that is understood.

Brooks told us forty years ago that no tool removes essential complexity, and he was right again. AI removed the typing. It did not remove the specifying or the verifying, which were always the hard parts and are now, if anything, more central to the job. The craft did not disappear. It moved to both ends of the workflow, and the engineers who thrive in this era are the ones who lean into the spec and the review rather than treating them as overhead the model made optional.

Write the spec. Keep the diffs small. Let the tests carry the intent. Read every line. Treat the model like the fast, brilliant, unaccountable junior it is. Do that, and AI is the genuine accelerator it promises to be. Skip it, and you have not eliminated the work — you have only deferred it, and made it someone’s incident.