Sequencing an AI Rollout: What to Adopt First

Most failed AI rollouts do not fail because the tools are bad. They fail because the organization adopted the wrong capability first. A team turns on an agentic coding assistant, points it at the billing service or the auth layer, and asks it to ship features autonomously on week one. The demo was magic. The production incident three weeks later is not.

The thing nobody tells you in a vendor pitch is that “adopt AI coding” is not a single decision. It is a dozen decisions wearing a trench coat. Code explanation, test generation, review assistance, documentation, refactoring, and autonomous feature work are wildly different in their risk profile, their feedback speed, and the prerequisites they demand. Treating them as one switch is the original sin.

This post is about order of operations. Not whether to adopt AI-assisted development, and not which vendor to pick, but the sequence in which a mature organization should climb the capability ladder. Start where risk is low and the feedback loop is tight. Earn the right to climb. The teams that get the sequence right compound their gains. The teams that start at the top spend their first quarter cleaning up.

Why sequence beats speed

There is a strong, well-documented temptation to skip straight to the headline capability: autonomous code generation on real systems. It is the most impressive demo and the most quotable promise. It is also where the evidence for caution is strongest.

The 2024 Accelerate State of DevOps Report from DORA found something counterintuitive. AI adoption was associated with gains in individual productivity and developer flow, but a 25% increase in AI adoption was associated with an estimated 1.5% decrease in delivery throughput and a 7.2% decrease in delivery stability. The mechanism the DORA team points to is not “AI writes bad code.” It is that AI tends to increase batch size, and larger changesets have been linked to instability across more than a decade of DORA research. More code, faster, with the same review and testing discipline as before, produces a worse delivery outcome.

The slowdown is not only an organizational phenomenon. In a 2025 randomized controlled trial, METR measured 16 experienced open-source developers across 246 real tasks on repositories they knew intimately. Developers expected AI to speed them up by 24%. After the study, they still believed it had sped them up by about 20%. In reality, allowing AI tools made them 19% slower. The perception gap is the dangerous part: a team can feel faster while shipping more slowly, which means subjective enthusiasm is a terrible signal for where to deploy AI next.

Code-level data tells a parallel story. GitClear’s analysis of 211 million changed lines of code authored between 2020 and 2024 found that copy-pasted (cloned) lines rose from 8.3% of changes in 2021 to 12.3% in 2024, while the share of changes associated with refactoring fell from around 25% in 2021 to under 10% in 2024. Code churn — lines reverted or substantially revised within two weeks of being committed, a proxy for “we shipped it before it was ready” — is on track to roughly double against its pre-AI 2021 baseline. The pattern is consistent: AI assistance, applied without discipline, biases toward adding code rather than improving the code that exists.

None of this is an argument against AI-assisted development. It is an argument against starting at the top of the ladder. The capabilities that produce these failure modes — high-volume, low-supervision generation on consequential code — are exactly the ones that should come last, after the guardrails that catch them are already in place.

Borrowing from capability maturity, lightly

The software industry already has a mental model for this, even if it has fallen out of fashion. The Capability Maturity Model Integration, developed at Carnegie Mellon, organized organizational process into staged levels — from initial and ad hoc, through managed and defined, to quantitatively managed and optimizing. You did not jump to level five. You earned each level by demonstrating the practices of the one below it.

We are not proposing you adopt CMMI. The heavyweight appraisal machinery is overkill for most teams, and the framework predates anything resembling a coding agent. But the core intuition is exactly right for AI rollout: capabilities have prerequisites, and the value of a higher rung depends on having genuinely mastered the lower one. CMMI’s staged representation deliberately defined a standard sequence of improvements precisely because organizations that skipped ahead kept regressing — the higher practices had nothing underneath them to stand on. Autonomy on critical code is a level-five capability. Most organizations are trying to deploy it from a level-one starting point — no measurement, no guardrails, no shared norms — and then are surprised when it does not stick.

The other reason maturity framing helps is that it reframes the question leadership keeps asking. “Are we behind on AI?” is the wrong question, because it invites the answer “yes, so skip ahead.” The right question is “which rung are we genuinely operating at, and what is the next one’s prerequisite?” That question has a concrete answer, it points to specific work, and it cannot be satisfied by buying a more aggressive tool. It is satisfied only by building the capability below.

The ladder below is a sequence, not a calendar. A disciplined team might climb two rungs in a quarter; a sprawling org with regulated code might spend a year on the lower ones. What matters is that you do not skip.

The capability ladder

Rung 1: Comprehension — explanation and onboarding

Start where the AI cannot break anything: helping people understand code that already exists. Explaining an unfamiliar function, summarizing a legacy module, answering “why does this service do that,” accelerating onboarding for a new hire navigating a million-line codebase.

The risk here is close to zero. The model produces words, not commits. A wrong explanation is caught the moment someone reads the actual code. And the feedback loop is immediate: the developer either understands faster or they do not. This is the rung where a skeptical senior engineer discovers the tool is genuinely useful without having to trust it with anything that ships.

There is a quieter benefit to starting here, too. Comprehension is the rung where trust gets built. A senior engineer who has watched the assistant correctly explain a gnarly piece of their own legacy code is far more willing to entertain the next rung than one whose first exposure was a hallucinated refactor on a Friday afternoon. Adoption is a social process before it is a technical one, and the cheapest place to earn credibility is the place where the AI is purely advisory. This is also where resistance tends to soften on its own — a dynamic we explore in helping traditional developers embrace AI.

Prerequisite to climb: essentially none. This is the entry point. The only thing to establish is that people are actually using it — which means you need baseline telemetry before you move on, so you can tell adoption from theater.

Rung 2: Generation under a human gate — tests and documentation

Next, let AI generate artifacts where a human reviews everything before it lands, and where the artifact’s correctness is checkable. Test generation and documentation are the canonical examples.

Test generation is the highest-leverage early rung, and it is not an accident that DORA keeps returning to “robust testing” as the fundamental that protects delivery stability. AI-generated tests have a useful property: a test that is wrong tends to fail loudly or get rejected in review, rather than silently corrupting production. You are using AI to strengthen the safety net before you ask it to do anything risky. That is the whole strategy in one move.

The caution: do not measure test volume. A model will happily generate a thousand assertions that all pass and prove nothing. This is the first rung where Goodhart’s Law bites — the moment a count becomes a target, it stops being a measure. Watch mutation-test survival, coverage of genuinely uncovered branches, and whether the tests catch real regressions, not the raw number. We dug into the trap of vanity metrics in the metrics that actually matter.

Prerequisite to climb: a code review culture that actually reviews, and a CI pipeline that runs the generated tests. If review is a rubber stamp today, AI generation will simply flood it.

Rung 3: Review assistance — a second set of eyes

Now point AI at the review process itself: flagging likely bugs, surfacing security smells, checking for missing error handling, summarizing large diffs so a human reviewer can focus. Crucially, the AI is advising the reviewer, not approving the merge.

This rung matters because it directly counters the batch-size problem DORA identified. If AI is going to make changesets bigger, AI-assisted review helps a human stay on top of bigger diffs. It is one of the few places where AI helps fix a problem that AI created. Done well, it tightens the loop. Done badly — where the bot’s approval is treated as sufficient — it removes the human judgment that was the entire point.

There is a subtle trap on this rung worth flagging. As AI review assistance gets good, reviewers start to lean on it, and the temptation is to treat a clean bot report as a green light. That is how you quietly convert a second set of eyes into a single set of eyes that happens to be a model. The discipline is to keep the human reviewer accountable for the judgment calls — security boundaries, architectural fit, whether the change should exist at all — and let the AI handle the mechanical sweep. The two are complementary only as long as the human stays in the loop rather than deferring to the machine.

Prerequisite to climb: clear ownership of merge decisions. Someone human is always accountable. If you cannot point to the person who owns the merge, you are not ready for the next rung.

Rung 4: Bounded generation — scoped feature work with a safety net

Here is where AI starts writing code that ships, but inside firm boundaries: well-specified tasks, on non-critical paths, with full test coverage, behind a human review gate, ideally behind a feature flag. A new internal tool. A peripheral endpoint. A self-contained utility. Not the payment processor.

This is the first genuinely risky rung, and the prerequisites are heavier. You need the guardrails from the rungs below already operating: tests that the AI cannot route around, review that is real, and the measurement to know whether the bounded experiment is working. The METR and GitClear findings are the warning label here — without those guardrails, you get more code, more churn, and a team that feels faster while delivery slows. With them, you get a controlled way to learn where AI generation actually pays off in your codebase.

The discipline that makes this rung safe is the same discipline that prevents the spiral we describe in how to prevent AI coding doom loops: small batches, human ownership, and a hard stop when the change touches something consequential.

Prerequisite to climb: measured evidence that rungs 1 through 3 are working. Not vibes. Acceptance rates, churn trends, review throughput, and delivery stability holding steady — the kind of multidimensional read we will get to in a moment.

Rung 5: Autonomy on core systems — earned, not assumed

The top rung is what most failed rollouts attempted first: agentic, semi-autonomous work on the systems the business depends on. It is genuinely valuable when you reach it correctly, and genuinely dangerous when you do not.

By the time you climb here, everything below should be load-bearing: a testing safety net you trust, review that catches what tests miss, telemetry that tells you when stability dips, and a team with the prompting and oversight skills to direct an agent rather than be surprised by it. Autonomy is not the absence of control. It is control that has been built up so thoroughly on the lower rungs that you can extend the leash without losing the dog.

If you reach this rung and your change failure rate climbs or your churn spikes, that is not a reason to push through. It is the ladder telling you a lower rung was never solid. Climb back down.

The two things you build before climbing anything

Two prerequisites run underneath every rung, and both must be in place before you take the first step — not bolted on at rung four when things get scary.

Guardrails that an AI cannot route around

Automated tests, CI gates, review requirements, and clear merge ownership are the structural safety net. DORA’s research is blunt about this: the fundamentals — small batch sizes and robust testing — are what protect delivery stability when AI enters the workflow. The capability ladder is, in a sense, a sequence for building those guardrails before you need them. You generate tests (rung 2) before you generate features (rung 4) precisely so the net exists before you start working without one. If you want the structural detail, our AI coding governance framework covers the gates rung by rung.

Measurement that is multidimensional from day one

You cannot sequence a rollout you cannot see. And the METR perception gap proves the obvious single metric — “do developers feel faster” — is actively misleading. You need a constellation of signals.

This is what the SPACE framework, published in ACM Queue by Nicole Forsgren and colleagues, was built for. SPACE argues that productivity is multidimensional and cannot be reduced to one number: it spans Satisfaction and well-being, Performance, Activity, Communication and collaboration, and Efficiency and flow. The framework’s central warning is that any single metric, optimized in isolation, will be gamed or will mislead.

For an AI rollout, read each rung through several SPACE dimensions at once. At rung 2, Activity (tests generated) means nothing without Performance (do they catch regressions). At rung 4, individual Efficiency can rise while team Performance — delivery stability — falls, which is precisely the DORA tradeoff. The point of measuring across dimensions is to catch that divergence before it ships, well before a quarterly review surfaces it.

The failure mode of starting at the top

It is worth naming the anti-pattern precisely, because it is so common it looks normal. An organization buys an agentic tool, skips rungs one through four, and points it at autonomous work on a critical system. There are no AI-generated tests because they skipped rung two. Review is a rubber stamp because they never hardened rung three. There is no telemetry because measurement felt like overhead.

Then the predictable happens. Batch sizes balloon, exactly as DORA observed. Churn rises, exactly as GitClear measured. Stability drops, and because nobody instrumented the rollout, the team cannot tell whether AI helped or hurt — they only have the feeling that they were faster, which METR demonstrates is the one signal they should trust least.

The diagnosis that follows is almost always wrong: “the AI tool is bad, rip it out.” The actual problem was sequence. The same tool, introduced at rung one and climbed deliberately, would likely have compounded value. Order of operations is not bureaucracy. It is the difference between a tool that gets blamed and a tool that gets trusted.

The Takeaway

Adopting AI-assisted development is not one decision, and it is not a race to the top of the ladder. It is a sequence of capabilities with real prerequisites, and the order you climb them determines whether the gains compound or the failures pile up.

Start at comprehension, where nothing can break. Move to generation under a human gate, building the testing safety net before you need it. Add review assistance to counter the batch-size problem. Only then attempt bounded generation, and only with measured evidence — not enthusiasm — that the lower rungs hold. Autonomy on core systems is the reward for getting everything below it right, not the opening move.

The evidence is consistent across DORA, METR, and GitClear: AI applied without discipline makes teams feel faster while shipping slower, with more code and more churn. The same AI, applied in sequence with guardrails and multidimensional measurement underneath, is one of the most valuable shifts in how software gets built. The tool is rarely the variable that decides which outcome you get. The sequence is.