¡Hola, soy DORA. Why hasn’t AI improved my metrics?

Originally published at show things improving and moving quickly but the quantitative measures like DORA, the entrenched way to measure how things are moving...haven't moved. In some cases they've reportedly got worse.

DORA Metric	Change
Deployment Frequency	➖
Lead Time For Change	➖
Time to Restore Service	➖
Change Failure	🔻

On the surface this is an uncomfortable place to be. AI is the second coming of Jesus if you believe the hype, so why doesn't it make numbers go up? Engineers reckon they’re faster and stuff is going out faster for the most part - and performed your favourite ritual of computation; slaughter a couple of goats, scroll through   where 150 or so professional developers were asked to write new features for a Java app, or expand its existing feature set with and without AI. They found task completion time improved by 30-50% with no overall change in code maintainability. Though there does seem to be a new study like this every five-minutes and I'm never sure who actually funded them. Scepticism aside, that does seem interesting that the previous generation of similar studies finding worse code and worse results. Have models got better? Are we more accepting of slop? Are we getting better at steering AI for our goals?

History doesn't repeat but it does have four legs and barks

Manufacturing ran into similar problems like this decades ago; I've never worked in Manufacturing but Gene Kim et.al promised me in

Throughput: how much the system delivers or can be processed by the system.

The AI development workflows compress parts of cycle time though throughput continues with a slightly different set of constraints or bottlenecks imposed by the front-loading and backloading of effort to refinement and review phases.

DORA metrics are of course not the only way you can measure things in the traditional SDLC and general software development spaces. My favourites or at the very least the four I have some actual experience and exposure to are;

• Cycle time variants, basically measuring where time is spent, not just the total.




• focuses on feedback loops, cognitive load, and flow.It comes from the same people who wrote SPACE, and is more focused, as the name suggests, on the Developer Experience.




• DX Core 4 tries to reconcile measures into speed, effectiveness, quality, and business impact. This is the only one of the list which at least acknowledges AI as a factor in modern development

From a certain height with a bit of a squint they are all trying to surface out similar useful information, I think it can all boil down to asking "How long does it take to do stuff?" and "How hard is it to do it?", essentially identifying where lifecycle friction exists. AI seems to just be adding a different type of friction which needs a highlighting without listening to the AI-token-hocking vendors who are profit incentivised to sell more tokens rather than useful implementations.

Which looks better ; 👓.....or 🕶️. 👓.......or 🕶️

In most projects, DORA is already being derived from the existing toolchain. Jira tracks state transitions, commits reference tickets, and CI/CD systems mark deployments. Lead time is often calculated as something like “In Progress → Deployed” or “In Progress → Closed” and that's cool, we should continue to do so but we'll need make sure our measures account for the changes in how work is fed into the system for AI based working patterns.

Instead of reducing everything to a single lead time number, we'll need to keep more of the intermediate signals and derive some additional ones from the same data. By that I means, if you already have:

• Jira issue transitions (e.g. In Progress, In Review, Done)




• Git commits linked via ticket IDs




• Pull request timestamps




• CI/CD deployment events

Given the existing data we can pull out these three extra measures which I think should give the extra information to make what we showing previously under the same DORA sauce.

1. Review latency (from Git)

You’re likely already collecting PR data implicitly for current measures, just instead of counting merges, extract:

• first_review_at - opened_at




• merged_at - opened_at

This should give us the surface for how long work waits once it leaves “In Progress”, so what would previously have been 'developer effort' now being 'reviewer effort' and treating the review phase as the time or engineering friction point. This should feed in to the lead time for change metrics we're reporting.

2. Change footprint (from Git + Jira linkage)

You’re already linking commits to tickets so measure aggregates per ticket for:

• Total files changed




• Total additions/deletions




• Number of PRs / commits

Now a “3-point story” has a measurable implementation footprint rather than just a planning estimate, I like to call this one the 'Slop-osity' measure. I'm going to add a 'Slop-o-meter' to my dashboard when nobody is watching. These should feed heavily into the Deployment Rework rate reporting.

3. Flow distribution (from Jira)

If lead time today is measured as:

In Progress → Closed

Then keep the breakdown instead of collapsing it into a single A-B keep the

• Time in “In Progress”




• Time in “In Review”




• Time in “Ready for Release” / waiting states

You’re already storing this in the issue history but previously you probably only needed the collapsed result. Depending on how your releases go this could feed into Deployment Frequency or Lead Time metrics.

4. Spec/rework loop count (from Jira + Git)

This is the one I think maybe the most important one for AI driven delivery. If a task goes:

CODE Kopieren

flowchart LR

  A["Spec"] --> B["Generated Implementation"]

  B --> C["Review"]

  C --> D["Spec Correction"]

  D --> E["Regenerated Implementation"]

The loop is really useful information as it tells us whether the issue was poor requirements, poor model output, too much ambiguity, too large a change, or a workflow that encourages people to keep rolling the dice and burning ~rainforests~ tokens rather than narrowing the target.

This should be trackable by measuring:

• Number of PR update bursts after review




• Number of times a ticket moves backwards from review to active work - which I've never really thought to include, partially because Jira doesn't make it easy.




• Number of linked spec or prompt changes after implementation starts

🌟'We did it!'

Given how AI code generation seem to be materialising within enterprises with an AI strategy, where the strategy isn't just "Here's Claude, do more, k thx bye" and feeding all the money into the AI money hole. There's still need to accurately know how long something's going to take to deliver and how much it will cost. It needs to work, and it needs to be understood, documented, supported, and stable to various levels and degrees depending on what it is.

As we move away from each line of code being lovingly crafted like the Code Artisans of old to something where expertise and engineering is used to steer, guide, and assess AI code output. We still need to get our hands dirty with an increasingly smaller and smaller pile of business logic and niche bits of logic where a human is still better. I still see AI as being a major contributor to the 80/20 rule in this regard, where we offload large portions of the tasks and focus on the difficult bits, the review and oversight areas.

Measuring stuff is obviously important and until someone smarter than me comes up with a better solution, I'm sticking with DORA's concepts with some changes in how I'm measuring rather than what I'm measuring for.

Vollständiger Original-Bericht

Ausführliche Details, Code-Beispiele & Hersteller-Stellungnahme auf dev.to.

↗ Original-Artikel auf dev.to lesen