A CIO’s framework for measuring engineering productivity

In boardrooms across the tech industry, CTOs and CIOs face a common challenge: effectively communicating and demonstrating IT productivity to leadership. Especially as organizations aim to deploy generative AI and other transformative technologies at a rapid pace, it’s critical to employ a reliable framework to measure engineering productivity.

Established frameworks including DORA, SPACE, and DevEx exist for this purpose, but each takes a different approach to defining and measuring productivity, and none of them alone provide a truly comprehensive analysis.

“In every board meeting, there’s the inevitable slide where the CTO is talking about productivity, and it always feels unsatisfying,” said Abi Noda, co-founder and CEO of DX, the company that created DX Core 4 . “We heard from CEOs and CIOs asking how to consolidate these different perspectives on productivity, so our goal was to create a single, benchmarkable approach that encapsulates all three major frameworks.”

The researchers behind DORA, SPACE, and DevEx — which include Abi Noda, Nicole Forsgren, and Margaret-Anne Storey — collaborated to develop the DX Core 4 framework. This new framework provides a multi-dimensional approach that combines the three established methodologies into a single framework. DX Core 4 measures four key dimensions of engineering productivity: speed, effectiveness, quality, and impact (see Figure 1).

Figure 1

DX

The framework takes a balanced approach to measurement, combining both quantitative and qualitative metrics. Qualitative metrics are captured by surveying IT team members, which is important due to the limitations of quantitative data alone. Survey responses reveal the “why” behind the metrics and provide context.

“Let’s say you were feeling sick and went to the doctor,” Noda said. “The nurse takes your temperature, and says, ‘Your temperature is normal, so it looks like you’re fine.’ You would protest, saying, ‘But I still feel sick!’” The issue is quantitative data may not detect a problem that people in the midst of the process can clearly see. Survey responses can also add critical explanatory context to the quantitative data, enabling leadership to clearly see the root cause of an issue, he said.

The framework also addresses a common pitfall in productivity measurement: the tendency to optimize for a single metric at the expense of others. “Everyone in tech knows that if you just index on lines of code or a measure of speed, you’ll get more of that at the expense of other things,” Noda cautions. “The multi-dimensional approach of DX Core 4 helps organizations maintain a balanced perspective on productivity.”

Perhaps most importantly, the framework is designed to be useful across all organizational levels, from business managers to C-suite executives. This alignment helps ensure productivity goals remain relevant and consistent throughout the organization, preventing the common disconnect between executive objectives and team-level metrics.

To learn more about the DX Core 4 and view industry benchmarks, visit getdx.com/core4.