In public-sector and regulated-cloud work, I learned that sovereignty rarely starts as a national strategy. It starts as an auditor’s question: Who can prove where the data went, which system made the decision and what changes when the vendor or infrastructure does? That question is now moving into AI, and most sovereign-AI debates answer the wrong version of it.

They ask whether a country can build its own model on domestic data and hardware. For the United States and China, which together hold more than 90% of global AI data-center capacity, per a January 2026 Tony Blair Institute analysis, that question is worth asking. However, for almost every other government, it is the wrong place to start. The operative question is narrower: Once AI is embedded in public services, who controls the stack?

The 5 layers of public-sector control

For a CIO, sovereign AI means enforceable control across the AI lifecycle; model ownership is a separate question. Control has five layers:

• Data control: Where sensitive public data sits, and whether it can train a vendor’s model.
• Model control: Which models clear which workloads, and under what validation.
• Infrastructure control: Whether critical workloads run in approved environments.
• Operational control: Whether AI-assisted actions are logged, monitored and reversible.
• Vendor control: Whether the agency keeps portability, audit rights and a real exit.

Those five layers are the control plane for public-service AI. Floyd Dcosta recently made the enterprise case in “AI without sovereignty is just outsourced intelligence”: capability is what a tool can do; authority over how and when it does it is something a buyer can quietly lose. For public services, losing that authority plays out in the public eye.

Public-sector AI risk differs from enterprise risk. A retailer’s bad recommendation costs a sale; a government’s AI touches benefits, tax enforcement, policing and emergency response, raising the bar to due process, records retention and continuity of operations. A government that cannot reconstruct an AI-assisted decision lacks operational sovereignty, even in a domestic data center.

Evaluating risk: Concentration, jurisdiction and shadow AI

Foreign dependency is a real risk, but the exposure that matters is a sudden cutoff: A model you cannot audit, switch or exit, shut off by someone else’s order. A vendor’s nationality is a poor guide to that risk; control is.  Two markers matter. The first is concentration. In July 2024, a single faulty CrowdStrike update crashed about 8.5 million Windows machines, disrupting airlines, hospitals, banks and governments worldwide. No attacker was involved; one homogeneous dependency failed everywhere at once. The lesson points away from vendor nationality and toward uniformity as the fault line, making portability and provider diversity resilience controls.

The second is jurisdiction. In June 2025, Microsoft’s legal director for France told a Senate inquiry, under oath, that it could not guarantee that French public-sector data, even in French data centers, would be protected against US demands under the 2018 CLOUD Act. No such request had been made, and EU data has stayed in the EU since January 2025; senators called the assurance purely declarative. For the most sensitive data, residency does not equal control; the parent’s jurisdiction can matter as much as the server’s. Three US hyperscalers hold about 70% of the European cloud market, while European providers’ share fell from 29% in 2017 to roughly 15%. Concentration plus jurisdiction is the exposure a CIO must price. I have watched teams treat vendor selection as the moment risk was solved; it rarely was.

The wrong response is self-isolation. Most countries will never build frontier models, advanced chips, hyperscale clouds and talent pipelines at once; the Tony Blair Institute calls full self-sufficiency “too expensive, too slow and, for most countries, simply impossible.” The better test is workload sensitivity. Low-risk uses, such as drafting, translation and summarization, can run on commercial platforms with controls; high-risk uses, such as benefits eligibility, fraud investigation and healthcare triage, demand stricter control over data, model behavior and auditability.

Mandating domestic-only provision before a competitive option exists inverts sovereignty. Europe 2031, a five-year scenario from June 2026 by European technologists and policy researchers, illustrates the failure mode: A 2027 “buy European” mandate lands as offensive cyber capability spreads, and agencies that switched to weaker providers are locked out and paying ransoms. The scenario is fiction; the mechanism is not. Leverage comes from being indispensable, not half-hearted self-sufficiency. The closer-to-home effect is shadow AI: Mandate an inferior sanctioned tool and staff bypass it, the way shadow IT grows up around tools people find too slow. A rule that pushes sensitive work into ungoverned shadow AI reduces control instead of adding it.

Regulation and data-residency rules belong in any serious strategy, but carry failure modes. Blanket localization raises hosting costs and slows adoption without guaranteeing control, and a “sovereign cloud” on a foreign parent’s stack can amount to sovereignty theater. The more useful pattern tiers requirements by sensitivity. India’s BHASHINI shows the application layer done well: A public platform serving 100 million-plus inferences a month across 22-plus languages on a vendor- and cloud-agnostic design that keeps data and switching rights public. Sovereignty resides in the portability, not in a national model.

Building an operational sovereignty strategy

Public trust is the constraint sovereignty rhetoric tends to skip. The OECD’s 2025 review of government AI warns that opaque systems make AI-assisted decisions hard to explain and can give public servants false confidence in tools that fail quietly. State-controlled AI is the same problem from the other side: A government that deploys models against its own citizens without audit or record has gained control and lost accountability. An agency that can log, explain and reverse an AI-assisted action can defend it to citizens, courts, auditors and elected officials. If it cannot, it has bought access and called it sovereignty.

None of this is new. AI sovereignty repeats earlier fights over cloud, telecom, semiconductors and cybersecurity. Europe’s flagship cloud project, GAIA-X, became a cautionary tale; the Dutch technologist Bert Hubert called it an “expensive distraction” that produced no European cloud, the familiar result of ambition without absorptive capacity. Cloud taught governments that outsourcing infrastructure does not outsource accountability; telecom, that vendor dependency becomes strategic exposure; chips, that supply chains matter before a crisis; cybersecurity, that trust must be verified continuously. AI inherits all four at once.

Over the next five to ten years, some countries will build national platforms, more will build trusted cloud and trusted model regimes, and most will run hybrids that pair domestic data control with global model access. Trade policy will harden those choices: Export controls on compute and data-localization rules will pull the vendor market into blocs that track alliances more than open markets. For a CIO, that turns a vendor and hosting decision into a five-year bet on whose rules and supply chains will still hold. The ones that succeed will treat sovereignty as an operating requirement, backed by leverage, not a slogan. Start with the control plane before the model: Most agencies will never own the model, and the controls are what decide whether the AI they do run stays accountable. Even when procurement policy is dictated from above, these questions remain within the CIO’s authority:

1. Can we classify AI workloads by public-service risk?
2. Can we prove where sensitive data goes across training, retrieval, inference, logging and retention?
3. Can we restrict which models are approved for which data classes and functions?
4. Can we reconstruct an AI-assisted action in enough detail to explain it?
5. Can we change providers without losing continuity or institutional knowledge?
6. Can we explain the system to citizens, regulators, auditors and elected officials?

A “no” to any of these does not mean the agency lacks AI. It means the agency has access it does not yet control. Public institutions can use global innovation without surrendering public authority, but only once they know what to hold, what to rent and where dependency turns into risk.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Four months of Claude Code — that’s all it took for Uber to burn through its entire annual budget for AI. Token after token, engineers embraced the platform with few control mechanisms tying costs to outcomes. The result was a budget runaway and a clear case study in how limited oversight snowballs into an AI blowout.

This is a phenomenon I like to call “shadow tokens” — AI credits paid for by the company but largely invisible to decision-makers. Too many engineers have the final say over how much they consume and, therefore, what it costs. This all-you-can-eat attitude is part of the reason why Microsoft is reportedly winding down many internal licenses across key engineering teams and why one in five organizations is missing its AI spend forecast by more than 50%.

And the trend is only accelerating. By 2028, Gartner predicts that AI coding costs (driven by this kind of ungoverned consumption) will be as much per developer as the salary companies pay that person.

LLMs and agents introduce a new class of variable cost that scales with behavior rather than headcount, putting enterprises on the hook for tools that balloon with workload. I don’t see this as enterprises overspending because they’re reckless — it’s down to a lack of managerial oversight, budget alignment that demands a proven return on investment, and engineer education on how much is too much.

Going forward, CIOs need to thread the AI needle between governance that encourages transparency and reasonable spend without stifling innovation.

When shadow tokens result in real costs

The issue is that AI isn’t a traditional line item. Previously, enterprise leaders onboarded software-as-a-service (SaaS) with a good idea of the total cost. An allocated software seat or annual contract was a known quantity. The cloud added some variation (with fluctuations depending on hosting size), but instances were still modelable. AI flips this status quo on its head — the unit of consumption is behavior and the cost is exponential.

And these specifics aren’t immediately apparent at pilot. Tools can appear inexpensive in controlled experiments yet unpredictably scale depending on session length, context window size, model selection and whether agents run in parallel. This is the fallacy of the $20-per-seat enterprise plan — tokens are charged separately at API rates with no ceiling. The final dollar value of any session is set by factors that finance can’t always model in advance, particularly when these decisions usually rest with the engineers themselves.

According to Deloitte, only 21% of organizations deploying agents have a mature governance model, a real concern because they’re token-eating machines. This is what was happening at Uber — Claude Code in agentic mode was autonomously reading codebases, planning changes across dozens of files and opening pull requests. Each step quickly adds up, with Anthropic’s own documentation noting that agents consume approximately seven times as many tokens as standard sessions.

This is shadow IT and shadow AI, evolved. This time, however, many leaders approved the tool in question without guardrails governing consumption. AI hype adds fuel to the fire and normalizes long sessions. Uber’s CTO, for example, described a company-wide shift toward “agentic software engineering” with employees “who are quietly experimenting, quietly shipping and quietly pushing things forward”. This is an exciting way to test the limits of what’s possible, certainly, but it’s also a position that goes a long way to explaining how the company spent its annual AI budget by April.

Shifting the culture from usage to yield

Engineers haven’t done anything wrong here. In fact, they’re adopting and experimenting as instructed, with Uber creating leaderboards and ranking users by token consumption. More use led to a better ranking, reflecting a culture that lauds new ways of doing things. This behavior is known as “tokenmaxxing,” and its principal knock-on effect is shadow tokens — quantity-over-quality processes that leaders struggle to control until they’re fully realized in the budget. Of course, if management treats adoption metrics as performance metrics, then engineers can’t be blamed for using more tokens. The tension is that the teams driving adoption aren’t the ones managing spend.

None of this is meant to dismiss AI’s productivity possibilities and potential return on investment. Developers save 3.6 hours per week, achieve 60% higher pull request throughput and cut onboarding time in half with automation. Meanwhile, Uber shared that roughly 11% of live backend updates were written by agents with no human in the loop. However, these wins aren’t the problem — it’s that too many teams aren’t connecting input to output. I’ve spoken to admins who discovered their token spend had tripled in a single quarter after using heavier models or accidentally doubling up on agentic applications. Nobody knew until the financial damage was done.

Automation needs to happen sustainably with an eye on the bottom line. In my view, a much better metric for achieving this is AI yield — the measurable business or engineering output generated per dollar spent on tokens. Otherwise, without a feedback loop, even genuinely productive teams are flying blind.

Stopping token waste before an AI blowout

Creating that throughline between AI investment and token consumption starts with established financial metrics. This is possible via maximum spend limits (dictated by spend tagging, workload tiering and cost-per-output benchmarks) per team or project. Then, any additional allocation requires approval, closing the loop between the engineers spending the tokens and the leaders paying for them. AI isn’t cheap and teams should demonstrate a bang for their buck.

This is something we do with our engineering team at Hexnode. Resource allocation for Claude Code and Cursor is tied directly to ROI rather than letting consumption run open-ended. Given the pay-as-you-go nature of these tools, a firm usage limit per team offers simple but essential control.

Similarly, there’s room to apply some of the governance principles IT uses for device management. Things like policy enforcement, role-based access, real-time monitoring and automated alerts can flag usage behavior in advance. Uncovering such insights at the token layer works to identify power users and prevent excessive spending.

We also need to encourage cultures that praise outputs that actually achieve efficiency. AI applications that result in shipping faster, reducing rework and cutting review cycles are gains that should be celebrated. If your company hosts leaderboards, frame unnecessary token burn as wasteful rather than valuable. The organizations creating healthier consumption habits work with their engineers to understand not just how to use AI, but what responsible use looks like and what it costs.

This is a conversation teams need to have now. Anthropic just ended flat-rate pricing for programmatic workloads from June 15. Now, agents, continuous integration pipelines and automated workflows draw from a dedicated monthly credit pool billed separately from the subscription. Once that pool is exhausted, agent tasks either stop entirely or overflow to extra billing. Work can either get very expensive or grind to a halt for teams that aren’t prepared.

Getting a grip on shadow tokens means better rules and tools connecting spend to outcomes. Only by building the financial and cultural infrastructure that encourages sustainable adoption can leaders see what they’re spending, connect it to what they’re getting and course-correct before the costs become a crisis. Ultimately, shadow tokens are only invisible if we choose not to look.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Four months of Claude Code — that’s all it took for Uber to burn through its entire annual budget for AI. Token after token, engineers embraced the platform with few control mechanisms tying costs to outcomes. The result was a budget runaway and a clear case study in how limited oversight snowballs into an AI blowout.

This is a phenomenon I like to call “shadow tokens” — AI credits paid for by the company but largely invisible to decision-makers. Too many engineers have the final say over how much they consume and, therefore, what it costs. This all-you-can-eat attitude is part of the reason why Microsoft is reportedly winding down many internal licenses across key engineering teams and why one in five organizations is missing its AI spend forecast by more than 50%.

And the trend is only accelerating. By 2028, Gartner predicts that AI coding costs (driven by this kind of ungoverned consumption) will be as much per developer as the salary companies pay that person.

LLMs and agents introduce a new class of variable cost that scales with behavior rather than headcount, putting enterprises on the hook for tools that balloon with workload. I don’t see this as enterprises overspending because they’re reckless — it’s down to a lack of managerial oversight, budget alignment that demands a proven return on investment, and engineer education on how much is too much.

Going forward, CIOs need to thread the AI needle between governance that encourages transparency and reasonable spend without stifling innovation.

When shadow tokens result in real costs

The issue is that AI isn’t a traditional line item. Previously, enterprise leaders onboarded software-as-a-service (SaaS) with a good idea of the total cost. An allocated software seat or annual contract was a known quantity. The cloud added some variation (with fluctuations depending on hosting size), but instances were still modelable. AI flips this status quo on its head — the unit of consumption is behavior and the cost is exponential.

And these specifics aren’t immediately apparent at pilot. Tools can appear inexpensive in controlled experiments yet unpredictably scale depending on session length, context window size, model selection and whether agents run in parallel. This is the fallacy of the $20-per-seat enterprise plan — tokens are charged separately at API rates with no ceiling. The final dollar value of any session is set by factors that finance can’t always model in advance, particularly when these decisions usually rest with the engineers themselves.

According to Deloitte, only 21% of organizations deploying agents have a mature governance model, a real concern because they’re token-eating machines. This is what was happening at Uber — Claude Code in agentic mode was autonomously reading codebases, planning changes across dozens of files and opening pull requests. Each step quickly adds up, with Anthropic’s own documentation noting that agents consume approximately seven times as many tokens as standard sessions.

This is shadow IT and shadow AI, evolved. This time, however, many leaders approved the tool in question without guardrails governing consumption. AI hype adds fuel to the fire and normalizes long sessions. Uber’s CTO, for example, described a company-wide shift toward “agentic software engineering” with employees “who are quietly experimenting, quietly shipping and quietly pushing things forward”. This is an exciting way to test the limits of what’s possible, certainly, but it’s also a position that goes a long way to explaining how the company spent its annual AI budget by April.

Shifting the culture from usage to yield

Engineers haven’t done anything wrong here. In fact, they’re adopting and experimenting as instructed, with Uber creating leaderboards and ranking users by token consumption. More use led to a better ranking, reflecting a culture that lauds new ways of doing things. This behavior is known as “tokenmaxxing,” and its principal knock-on effect is shadow tokens — quantity-over-quality processes that leaders struggle to control until they’re fully realized in the budget. Of course, if management treats adoption metrics as performance metrics, then engineers can’t be blamed for using more tokens. The tension is that the teams driving adoption aren’t the ones managing spend.

None of this is meant to dismiss AI’s productivity possibilities and potential return on investment. Developers save 3.6 hours per week, achieve 60% higher pull request throughput and cut onboarding time in half with automation. Meanwhile, Uber shared that roughly 11% of live backend updates were written by agents with no human in the loop. However, these wins aren’t the problem — it’s that too many teams aren’t connecting input to output. I’ve spoken to admins who discovered their token spend had tripled in a single quarter after using heavier models or accidentally doubling up on agentic applications. Nobody knew until the financial damage was done.

Automation needs to happen sustainably with an eye on the bottom line. In my view, a much better metric for achieving this is AI yield — the measurable business or engineering output generated per dollar spent on tokens. Otherwise, without a feedback loop, even genuinely productive teams are flying blind.

Stopping token waste before an AI blowout

Creating that throughline between AI investment and token consumption starts with established financial metrics. This is possible via maximum spend limits (dictated by spend tagging, workload tiering and cost-per-output benchmarks) per team or project. Then, any additional allocation requires approval, closing the loop between the engineers spending the tokens and the leaders paying for them. AI isn’t cheap and teams should demonstrate a bang for their buck.

This is something we do with our engineering team at Hexnode. Resource allocation for Claude Code and Cursor is tied directly to ROI rather than letting consumption run open-ended. Given the pay-as-you-go nature of these tools, a firm usage limit per team offers simple but essential control.

Similarly, there’s room to apply some of the governance principles IT uses for device management. Things like policy enforcement, role-based access, real-time monitoring and automated alerts can flag usage behavior in advance. Uncovering such insights at the token layer works to identify power users and prevent excessive spending.

We also need to encourage cultures that praise outputs that actually achieve efficiency. AI applications that result in shipping faster, reducing rework and cutting review cycles are gains that should be celebrated. If your company hosts leaderboards, frame unnecessary token burn as wasteful rather than valuable. The organizations creating healthier consumption habits work with their engineers to understand not just how to use AI, but what responsible use looks like and what it costs.

This is a conversation teams need to have now. Anthropic just ended flat-rate pricing for programmatic workloads from June 15. Now, agents, continuous integration pipelines and automated workflows draw from a dedicated monthly credit pool billed separately from the subscription. Once that pool is exhausted, agent tasks either stop entirely or overflow to extra billing. Work can either get very expensive or grind to a halt for teams that aren’t prepared.

Getting a grip on shadow tokens means better rules and tools connecting spend to outcomes. Only by building the financial and cultural infrastructure that encourages sustainable adoption can leaders see what they’re spending, connect it to what they’re getting and course-correct before the costs become a crisis. Ultimately, shadow tokens are only invisible if we choose not to look.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

ISC2 has begun developing a vendor-neutral AI security certification aimed at cybersecurity professionals working to secure AI systems and manage emerging AI risks.

The nonprofit organization, best known for the CISSP certification, says it is seeking volunteers worldwide to help define the knowledge and skills that will shape the new credential. While ISC2 has not finalized the certification domains, the certification is expected to address both technical AI security and governance topics, with a pilot exam planned before the end of 2026.

According to Casey Marks, ISC2 chief operating officer, feedback from cybersecurity practitioners led ISC2 to conclude that AI security had grown beyond expanding AI content within existing certifications.

“AI has reached a tipping point,” Marks says. “AI no longer is just another tool; instead, it has fundamentally changed the cybersecurity practice itself.”

ISC2 already includes AI-related content in certifications including CISSP and CCSP, but Marks says practitioners have identified new responsibilities and risks that extend beyond those programs. “Enterprise security teams are currently grappling with significant knowledge gaps, particularly around securing model architectures against new vulnerabilities like prompt injection, data poisoning, and model inversion,” Marks adds.

Organizations are working to understand emerging governance frameworks, including the NIST AI Risk Management Framework and ISO/IEC 42001, while adapting traditional application security and security operations workflows to AI systems, he says.

ISC2 has not finalized the certification domains, but Marks says the organization expects the credential to address both technical controls and governance practices for securing AI systems and managing AI risk. The certification will use ISC2’s established certification development process, which relies on cybersecurity practitioners to define job roles, develop exam content, and validate competencies.

Marks says ISC2 will continue to update the certification through ongoing input from cybersecurity professionals, in addition to its regular certification review process.

The organization is also determining which professionals the certification will target. Marks says AI security responsibilities are emerging across security architecture, risk management, security operations, software development security, governance and compliance, communication and network security, and security assessment and testing. ISC2 says the certification will reflect how those roles are evolving.

For organizations that are building AI security programs now, Marks recommends using existing AI training resources, adopting established governance frameworks, creating cross-functional AI security working groups, and participating in the certification development process.

Marks says ISC2 expects AI knowledge to become part of most cybersecurity roles while a more specialized AI security discipline continues to develop. He says organizations will increasingly need professionals with foundational AI security knowledge, as well as specialists in areas such as adversarial machine learning, model architectures, and AI data pipelines.

Looking ahead, Marks says he expects AI security expertise to evolve into both a foundational skill for cybersecurity professionals and a specialized discipline of its own.

“At this time, we are seeing a hybrid evolution occurring in real time: AI security is simultaneously becoming a baseline expectation for all security roles, while also carving out a dedicated, highly specialized discipline,” Marks says.

In public-sector and regulated-cloud work, I learned that sovereignty rarely starts as a national strategy. It starts as an auditor’s question: Who can prove where the data went, which system made the decision and what changes when the vendor or infrastructure does? That question is now moving into AI, and most sovereign-AI debates answer the wrong version of it.

They ask whether a country can build its own model on domestic data and hardware. For the United States and China, which together hold more than 90% of global AI data-center capacity, per a January 2026 Tony Blair Institute analysis, that question is worth asking. However, for almost every other government, it is the wrong place to start. The operative question is narrower: Once AI is embedded in public services, who controls the stack?

The 5 layers of public-sector control

For a CIO, sovereign AI means enforceable control across the AI lifecycle; model ownership is a separate question. Control has five layers:

• Data control: Where sensitive public data sits, and whether it can train a vendor’s model.
• Model control: Which models clear which workloads, and under what validation.
• Infrastructure control: Whether critical workloads run in approved environments.
• Operational control: Whether AI-assisted actions are logged, monitored and reversible.
• Vendor control: Whether the agency keeps portability, audit rights and a real exit.

Those five layers are the control plane for public-service AI. Floyd Dcosta recently made the enterprise case in “AI without sovereignty is just outsourced intelligence”: capability is what a tool can do; authority over how and when it does it is something a buyer can quietly lose. For public services, losing that authority plays out in the public eye.

Public-sector AI risk differs from enterprise risk. A retailer’s bad recommendation costs a sale; a government’s AI touches benefits, tax enforcement, policing and emergency response, raising the bar to due process, records retention and continuity of operations. A government that cannot reconstruct an AI-assisted decision lacks operational sovereignty, even in a domestic data center.

Evaluating risk: Concentration, jurisdiction and shadow AI

Foreign dependency is a real risk, but the exposure that matters is a sudden cutoff: A model you cannot audit, switch or exit, shut off by someone else’s order. A vendor’s nationality is a poor guide to that risk; control is.  Two markers matter. The first is concentration. In July 2024, a single faulty CrowdStrike update crashed about 8.5 million Windows machines, disrupting airlines, hospitals, banks and governments worldwide. No attacker was involved; one homogeneous dependency failed everywhere at once. The lesson points away from vendor nationality and toward uniformity as the fault line, making portability and provider diversity resilience controls.

The second is jurisdiction. In June 2025, Microsoft’s legal director for France told a Senate inquiry, under oath, that it could not guarantee that French public-sector data, even in French data centers, would be protected against US demands under the 2018 CLOUD Act. No such request had been made, and EU data has stayed in the EU since January 2025; senators called the assurance purely declarative. For the most sensitive data, residency does not equal control; the parent’s jurisdiction can matter as much as the server’s. Three US hyperscalers hold about 70% of the European cloud market, while European providers’ share fell from 29% in 2017 to roughly 15%. Concentration plus jurisdiction is the exposure a CIO must price. I have watched teams treat vendor selection as the moment risk was solved; it rarely was.

The wrong response is self-isolation. Most countries will never build frontier models, advanced chips, hyperscale clouds and talent pipelines at once; the Tony Blair Institute calls full self-sufficiency “too expensive, too slow and, for most countries, simply impossible.” The better test is workload sensitivity. Low-risk uses, such as drafting, translation and summarization, can run on commercial platforms with controls; high-risk uses, such as benefits eligibility, fraud investigation and healthcare triage, demand stricter control over data, model behavior and auditability.

Mandating domestic-only provision before a competitive option exists inverts sovereignty. Europe 2031, a five-year scenario from June 2026 by European technologists and policy researchers, illustrates the failure mode: A 2027 “buy European” mandate lands as offensive cyber capability spreads, and agencies that switched to weaker providers are locked out and paying ransoms. The scenario is fiction; the mechanism is not. Leverage comes from being indispensable, not half-hearted self-sufficiency. The closer-to-home effect is shadow AI: Mandate an inferior sanctioned tool and staff bypass it, the way shadow IT grows up around tools people find too slow. A rule that pushes sensitive work into ungoverned shadow AI reduces control instead of adding it.

Regulation and data-residency rules belong in any serious strategy, but carry failure modes. Blanket localization raises hosting costs and slows adoption without guaranteeing control, and a “sovereign cloud” on a foreign parent’s stack can amount to sovereignty theater. The more useful pattern tiers requirements by sensitivity. India’s BHASHINI shows the application layer done well: A public platform serving 100 million-plus inferences a month across 22-plus languages on a vendor- and cloud-agnostic design that keeps data and switching rights public. Sovereignty resides in the portability, not in a national model.

Building an operational sovereignty strategy

Public trust is the constraint sovereignty rhetoric tends to skip. The OECD’s 2025 review of government AI warns that opaque systems make AI-assisted decisions hard to explain and can give public servants false confidence in tools that fail quietly. State-controlled AI is the same problem from the other side: A government that deploys models against its own citizens without audit or record has gained control and lost accountability. An agency that can log, explain and reverse an AI-assisted action can defend it to citizens, courts, auditors and elected officials. If it cannot, it has bought access and called it sovereignty.

None of this is new. AI sovereignty repeats earlier fights over cloud, telecom, semiconductors and cybersecurity. Europe’s flagship cloud project, GAIA-X, became a cautionary tale; the Dutch technologist Bert Hubert called it an “expensive distraction” that produced no European cloud, the familiar result of ambition without absorptive capacity. Cloud taught governments that outsourcing infrastructure does not outsource accountability; telecom, that vendor dependency becomes strategic exposure; chips, that supply chains matter before a crisis; cybersecurity, that trust must be verified continuously. AI inherits all four at once.

Over the next five to ten years, some countries will build national platforms, more will build trusted cloud and trusted model regimes, and most will run hybrids that pair domestic data control with global model access. Trade policy will harden those choices: Export controls on compute and data-localization rules will pull the vendor market into blocs that track alliances more than open markets. For a CIO, that turns a vendor and hosting decision into a five-year bet on whose rules and supply chains will still hold. The ones that succeed will treat sovereignty as an operating requirement, backed by leverage, not a slogan. Start with the control plane before the model: Most agencies will never own the model, and the controls are what decide whether the AI they do run stays accountable. Even when procurement policy is dictated from above, these questions remain within the CIO’s authority:

1. Can we classify AI workloads by public-service risk?
2. Can we prove where sensitive data goes across training, retrieval, inference, logging and retention?
3. Can we restrict which models are approved for which data classes and functions?
4. Can we reconstruct an AI-assisted action in enough detail to explain it?
5. Can we change providers without losing continuity or institutional knowledge?
6. Can we explain the system to citizens, regulators, auditors and elected officials?

A “no” to any of these does not mean the agency lacks AI. It means the agency has access it does not yet control. Public institutions can use global innovation without surrendering public authority, but only once they know what to hold, what to rent and where dependency turns into risk.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

AI agents can turn a clear description into working software, the engineer’s judgement is what makes the difference: deciding what to build, catching the tradeoff the agent didn’t know to weigh, and owning the call on whether the result is right.

That judgement has always been the hard part of engineering. It just used to be bundled into the act of writing code, where a skilled engineer did it while typing. As agents take on more of the typing, that judgement separates out and becomes the clear center of the role. Leaders who adapt early will get their metrics, their talent pipelines, and their delivery models working with this shift rather than against it.

Judgement is defining, constraining, and deciding

Judgement is all about defining the problem precisely enough that an agent builds the right thing. It’s setting the constraints the agent won’t infer on its own. It’s spotting the tradeoff buried three layers down that only shows up if you understand the system. And it’s looking at a finished implementation and knowing whether it’s genuinely good enough to ship.

This is the harder part of the job and the real driver of quality. It was easy to underrate when it lived inside day-to-day coding. Now it’s what separates a strong team from an average one.

The shift changes where time, growth, and metrics go

If the high-value activity is intent, review, and judgement rather than raw output, a few assumptions are worth revisiting.

Where engineers spend their time. Less of the day goes to producing boilerplate and mechanical implementation, and more goes to the reasoning that used to get squeezed to the edges: framing the problem and owning the judgement calls that determine quality.

How teams grow their people. Defining problems well, spotting risk, and critically evaluating work you didn’t write yourself have always been senior skills. When agents handle more of the mechanical work, those skills become learnable earlier. That puts the emphasis on leaders to teach the reasoning: why a choice gets made and how to weigh the tradeoffs that come with it.

What you measure. Lines shipped, tickets closed, and velocity charts all measured throughput of the old scarce resource. They say very little about the new one. The teams that adapt will start measuring the quality of intent going in and the reliability of judgement coming out, because that’s where the results now live.

Reinvest the time you get back

The tempting response is to treat the freed-up capacity as pure speed: same work, same tooling, just faster. That captures the easy win and misses the real one. If engineers spend their reclaimed time reviewing a rising volume of agent output with no better context than before, review quietly becomes the new constraint, and you’ve moved the problem rather than solved it.

The organizations that get ahead will invest the reclaimed capacity into the judgement layer: creating stronger specs and acceptance criteria before work starts, building review practices that test agent output against intent, and capturing the reasoning behind decisions where the next person can find it, so it doesn’t have to be reconstructed every time. That’s how the shift becomes an advantage for your team.

The through-line for leaders

The engineering job is moving up a level, from executing the work to directing and validating it. That’s a more strategic role, and it rewards clarity of thought over speed of output. Leaders who see the shift early can help their engineers grow into the work that’s now most valuable.

See how leading engineering organizations are operationalizing this shift at jira.dev.

GitLab has released GitLab 19.2, an update to the company’s devsecops platform that allows teams to fix vulnerable dependencies automatically, use Security Review Flow to catch logic flaws that scanners miss, and run AI agents straight from the terminal, the company said.

Highlights in GitLab 19.2 include the following:

• Dependency Scanning Auto-Remediation, in public beta, uses AI to fix build-breaking changes and iterates until your pipeline passes, with every change governed by your existing gates and audit trail.
• Security Review Flow, also in public beta, analyzes code changes as a security engineer would and catches authorization gaps, business-logic errors, and race conditions that static scanners structurally cannot see.
• GitLab Duo CLI, now generally available, gives developers access to agents and multi-step agentic flows for all software life cycle tasks without leaving the terminal. 
• Custom Flows, now generally available, let teams replace manual multi-step workflows with agentic automations for software development, triggered by GitLab events.

“Coding agents made it possible to generate far more code and moved the bottleneck downstream to reviews and security,” said Manav Khurana, chief product and marketing officer at GitLab, in a statement. “GitLab 19.2 puts agents to work on that bottleneck: fixing vulnerable dependencies, catching the flaws scanners miss, and automating the steps in between with a person still approving what ships.”

You are one of the six billion people (75% of the world population) online today, and every click you make is routed through the data center. Data centers, whether knowingly or unknowingly, play a very critical role in your daily online activities. With an increasing population, increasing usage of online presence, and now omniscient AI, the demand for data centers has increased manyfold, and the trend seems similar to the year 2000, when telephone towers were built to accommodate increased digital presence.

To win the AI race, Hyperscalers (Google, Meta, Amazon, Microsoft, Alibaba, Oracle, IBM, Tencent) are spending huge amounts of money on data center development. In the USA, the hyperscalers are planning to spend $650 billion in 2026, which is around 70% higher than 2025 spending, according to Yahoo Finance.

As per McKinsey research, by 2030, companies will invest around $7 trillion in Capex on data center infrastructure globally. More than $4 trillion will go towards computing hardware investment. More than 40% of this spending will be invested in the United States.

Demand growth in data centers

McKinsey analysis shows that global demand for data center capacity can more than triple by 2030, with a compound annual growth rate (CAGR) of around 22 per cent. In the USA, data center demand could grow by 20-25 per cent at the same time.  

The data center industry is currently undergoing a violent transition. We are moving away from the era of “bespoke projects” — where every facility was a unique architectural feat — into an era of industrialized infrastructure. With global capital expenditure in the sector projected to hit $3 trillion by 2028, the “bottleneck” has shifted. It is no longer about securing the capital; it is about the physics of the supply chain.

During my tenure at Vantage, managing the intersection of data center construction management (DCCM) and infrastructure management (DCIM), I saw firsthand that the most successful players aren’t those with the deepest pockets, but those with the most integrated data threads. If your construction data in Procore doesn’t talk to your financial reality in Yardi, or your operational capacity in DCIM, you aren’t building a data center — you’re managing a $500 million blind spot.

The death of “sticks and bricks”

Traditionally, data center construction was treated as civil engineering. But for the modern CIO, a data center is a complex product assembly.

The challenges are systemic. We are facing 50-to-80-week lead times for critical “long-pole” items: extra-high-voltage transformers, switchgear, and the liquid cooling manifolds required for the next generation of AI chips. In this environment, the traditional reactive supply chain model is a liability.

To survive the $3 trillion inflow, we must adopt a hybrid-agile SCOR (supply chain operations reference) model. This means applying continuous flow logic to standardized components (like modular power skids) while maintaining agile responsiveness for the volatile IT layer.

The digital bridge: Construction management software  to ERP

The most significant opportunity for CIOs lies in financial-operational integration. In many organizations, there is a data chasm between the construction site and the corporate office. Construction teams live in the construction management software tracking tasks, trades, RFIs and payment submittals. Finance teams operate corporate offices with project management tools (worth remembering that email is a key tool besides spreadsheets and phone calls) tracking capex schedule, commissioning timeline, capital drawdowns and asset lifecycle management.

These systems are siloed; the CIO loses visibility into the total cost to serve. By integrating construction management into the financial system, we create real-time financial visibility of the build. We can see exactly how a three-week delay in a chiller delivery impacts the internal rate of return (IRR) of the entire asset. This isn’t just accounting; it’s strategic telemetry.

From BIM to DCIM: The lifecycle thread

The second bridge is the handoff from construction (BIM) to operations (DCIM). Historically, this handoff was a nightmare of PDFs and Excel sheets. By the time the operations team took the keys, the “as-built” design information was already out of date.

The opportunity today is to maintain a continuous data thread. The sensor data and asset tags established during the “make” phase in our SCOR model should flow directly into the DCIM. This allows us to perform virtual commissioning. Before a single server is racked, we should already have a digital replica of the airflow, power distribution, and cooling capacity.

The scientific inference: AI in the supply chain

As someone who has led data and AI initiatives, I’ve seen the hype. But in the supply chain, the application of AI must be pragmatic, not generative. We don’t need AI to write poems; we need it for predictive procurement. Most organizations manage their procurement in ERP or a mix of a few tools to manage the source-to-settle business flow. Adopting a system workflow improves data collection and the state of the procurement cycle, which in turn provides AI with the context to draw inferences for possible delays and anomalies in original specifications and change orders.

By applying machine learning to global logistics data, we can move from just-in-time to just-in-case modeling. AI can analyze geopolitical risks, shipping lane congestion, and raw material pricing to tell a CIO: “Order your switchgear 14 months early, or your Q3 2027 ‘Power On’ date is at risk.”

Bringing it all together: AI in the supply chain and finance

Why it matters: Approximately 70% of the capex is on this workflow and making timely decisions that directly impact the ready-for-service dates. The current challenge of reactionary adjustment in design to procurement to local fit-out is a significant drain on capex efficiency and cost of capital. Because single-project delivery delays have become so volatile, a massive structural shift is occurring in how digital infrastructure is funded. Single-project debt (special purpose vehicles or SPVs) is facing severe friction. To insulate themselves from RFS shocks, the largest institutional players are moving toward permanent platform capital — aggregating exposure across dozens of global assets simultaneously.

Navigating these complex multi-billion-dollar engineering projects distributed over a large geography is simply unmanageable without rethinking and re-engineering existing tools and processes.

The roadmap for the modern CIO

To lead this transformation, CIOs must move beyond the IT shop mentality and become master orchestrators of the supply chain. Here is the 1500-word reality condensed into three mandates:

1. Standardize the product: Stop designing bespoke facilities. Move toward DFMA (design for manufacturing and assembly). If 70% of your data center can be built in a factory and shipped as modules, you bypass the unpredictability of on-site labor.
2. Integrate the financial stack: If your construction management software and your ERP aren’t sharing a heartbeat, your data is lying to you. Force the integration between Procore and Yardi.
3. Own the long poles: Don’t leave the procurement of transformers and cooling units to general contractors. Use your balance sheet to secure these items years in advance. In 2026, inventory is the new currency.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Across industries, a version of the same conversation is playing out in technology leadership meetings. Enterprises have deployed AI broadly, and foundation models keep getting more capable. Yet the outputs still feel generic, shaped by industry patterns rather than by the organization producing them.

McKinsey’s AI Trust Maturity Survey found that while overall AI maturity scores have improved, only about a third of organizations have reached a mature level of strategy and governance. Technical capability is advancing faster than organizational alignment. In my view, the gap is not a model problem. It is a context problem. Enterprises are feeding generic inputs into powerful models because sharing organizational context seamlessly with AI is neither easy nor intuitive today.

Building the analytical and creative capabilities to scale AI, something I explored in a recent piece on the left-brain and right-brain approach to enterprise AI, is necessary but not sufficient. Before either can function effectively, the enterprise needs something more fundamental. AI that actually understands the contextual fabric of the organization it is operating in. A frontier model has processed everything written about your sector, your competitors and your regulatory landscape. It cannot access the reasoning embedded in years of delivery decisions, the patterns encoded in how your teams scope and deliver work over time. That knowledge is organizational memory, and frontier models can’t get that easily. It exists inside every enterprise but has never been structured, connected or made available to any AI system. Without it, even the most capable model answers a generic version of your question.

The next competitive advantage in enterprise AI will not come from a better model. It will come from a better organizational context.

One global technology enterprise set out to solve this across its own operations, building a modular ecosystem of domain-specific agents grounded in its own data across contracting, talent and vendor management workflows. What emerged was not just operational efficiency but a shared intelligence layer connecting decisions across functions for the first time.

Competitive differentiation was never about the tools

Consider what actually separates high-performing enterprises from the rest. In a regulated industry like financial services or healthcare, organizations cannot meaningfully differentiate on product. A bank cannot offer substantially different products or services. A health system uses the same clinical protocols and the same electronic health record (EHR) platforms as its peers. What varies is everything underneath: the rigor of processes, the coherence of cross-functional decisions and the people who carry years of accumulated organizational judgment in how they make those decisions.

An organization with a proper context layer in place can say with precision that for this type of engagement, in this sector, with this risk profile, our institutional history tells us exactly where we stand. That level of specificity is what most enterprises have never made available to AI.

The enterprise AI brain that every organization has but has never assembled

Every enterprise already possesses what I think of as an enterprise AI brain. The problem is that it has never been assembled in one place. The data exists across contracts, project documentation, talent records, delivery metrics and the operational communications of daily execution — the informal reasoning that rarely makes it into formal systems.

None of the standard enterprise platforms were designed to connect this. A customer relationship management (CRM) system captures customer interactions. An enterprise resource planning (ERP) system captures transactions. A project management tool captures tasks and timelines. None of them captures the reasoning behind decisions and none of them surfaces a coherent picture of how the organization actually thinks and operates.

BCG’s study across hundreds of companies found that only 10% of AI value comes from the algorithms and another 20% from the technology that implements them, meaning the remaining 70% depends on people, processes and organizational change. The organizations extracting real value are those that have made their institutional knowledge available to AI in a structured, governed way.

Building a contextual AI fabric

A Contextual AI Fabric is the technical and organizational layer that makes the Enterprise AI Brain usable. It brings together unstructured data ingestion, semantic structuring, retrieval pipelines and governed model access to give AI systems the organizational context they need to produce outputs that are genuinely specific to your enterprise rather than generically accurate about your industry. It rests on three pillars. Core is the secure, governed and interoperable foundation that AI operations run on. Context is reliable, traceable access to the organization’s data, processes, knowledge and history. Coordination connects people, agents, applications and systems into process-driven workflows with clear controls and accountability, so the organization acts as one rather than a set of disconnected functions.

The data layer is where most organizations underestimate the work. Contracts, project reports, talent assessments and operational communications require extraction, chunking, embedding and indexing before a model can retrieve and reason over them meaningfully.

The semantic layer is what makes retrieval meaningful. Even well-ingested data fails if functions use different terminology for the same concepts. What legal calls a contract, delivery calls a scope. Without a shared ontology, AI systems remain precise about the wrong thing. And retrieval alone, however well-structured, only takes an organization so far. Retrieval surfaces the right information at the moment of a query, but it does not give a model genuine memory of the organization. The real source of unique, organization-level relevance comes from training domain-specific small language models on this context directly, models that carry organizational memory forward rather than fetching it fresh every time. That is what ultimately separates a Contextual AI Fabric from a well-organized database.

The governance layer is not an add-on. Access controls, data lineage, approval thresholds and human checkpoints need to be designed in before any agent goes into production. Security is not a layer you add afterward. It is the condition under which organizational AI is worth building. If the institutional intelligence that makes your enterprise distinct gets absorbed into a frontier model’s training data, it becomes everyone’s baseline. That is an architectural decision made, or avoided, at the point of deployment.

Proprietary by design

The institutional knowledge that makes up a contextual AI fabric — delivery history, commercial patterns, talent intelligence and operating culture — is proprietary in ways no external model can replicate. This is as much a security imperative as it is a competitive one. Organizational context, once exposed, cannot be unexposed.

Most enterprises are using AI to automate existing processes rather than questioning whether those processes should be redesigned entirely. The organizations extracting the most value are those willing to ask whether their current operating model, built before GenAI existed, is the one they would build today. That question is harder than any technology decision, and it is also the most consequential one.

From context to coordinated action

Context alone is not enough. When a delivery risk surfaces in project data, the talent function needs to respond. When a commercial signal changes in contract data, operations need to recalibrate. This kind of cross-functional coordination, driven by shared organizational intelligence rather than siloed data, is where the real value of enterprise AI shows up and where the absence of a shared context layer becomes most visible.

A global leader in digital payments and business services found its AI deployments across payroll, HR and risk compliance, each running in isolation, with no shared governance or common data foundation. Once the organization established a unified governance backbone connecting its operational data through a shared retrieval layer, business users could query across domains in plain language and new use cases across fraud analytics, forecasting and policy extraction became extensible without rebuilding infrastructure for each one. The shift was not in the models. It was in the shared foundation underneath them.

The leadership question behind the technology question

The enterprises pulling ahead in AI are not winning on model quality but on organizational memory. The ones that have done the hard work of structuring their institutional knowledge into a governed, secure Contextual AI Fabric are giving their AI something no competitor can replicate: the accumulated intelligence of how the business actually operates.

For CIOs, the question is no longer which model to deploy. It is whether the organization has built the foundation that would make any model worth deploying.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

In the client-server era, the penalty for inefficient programming, such as unoptimized database calls, was largely confined to application responsiveness. Today, in the AI era, code, architectural, and platform inefficiencies are no longer just a performance issue, they’re a financial and environmental liability. Left unchecked, poor code cascades into soaring token costs and spikes data center power consumption, directly undermining both cloud budgets and corporate sustainability goals.

AI’s impact on sustainability

By 2029, IDC projects that the number of actively deployed AI agents will exceed 1 billion worldwide, which is 40 times more than in 2025. And these agents will perform 217 billion actions per day.

To deliver on this demand, AI data centers are being built out at an unprecedented rate, with Gartner forecasting that global spending on data centers over the next three years will increase 31.7% to surpass $650 billion, driven primarily by hyperscaler cloud providers building out AI foundations, and optimizing servers for heavy AI workloads.

All this presents a significant strain on the energy grid as well as environmental sustainability, including:

• The power double-down: The International Energy Agency (IEA) projects that global data center electricity consumption will more than double from about 415 to 945 TWh by 2030, primarily fueled by energy-intensive accelerated computing for AI.
• The inference premium: AI workloads are vastly more demanding than standard web activities. A gen AI query consumes roughly 10 times the electricity of a conventional keyword search, or roughly 2.9 watt-hours as opposed to 0.3 watt-hours.
• Water consumption: Cooling these dense clusters is highly resource intensive. Global AI-related water demand is expected to reach 4.2 to 6.6 billion cubic meters by 2027.

The good news, however, is it’s not all out of the control of end user organizations and CIOs. Just as in the client-server era, through careful planning and execution, CIOs have the potential to significantly improve the performance, costs, and sustainability impacts of their AI application portfolio.

Here are four recommendations to maximize value as you look across your AI applications and infrastructure estate.

Revisit business objectives in light of AI

AI applications and platforms bring several new headaches for CIOs and CFOs in terms of FinOps. The variable nature of AI vendor billing due to variable monthly token costs is just one well-known example. To avoid unpleasant surprises, be sure to carefully review vendor contracts to decipher pricing models. Look for what’s included in seat-based license fees and what’s added as variable charges for agentic AI usage.

In addition, explore new metrics and KPIs such as intelligence per watt to help make sense of your return on AI. Just as miles per gallon helps us evaluate new car purchases, IPW can help to measure the computational efficiency of a system. It quantifies how much intelligence — typically measured in AI inferences, tokens processed, or model training iterations — a processor can deliver for every watt of electrical power it consumes.

According to Max Romanenko, chief engineering officer at relational database platform EDB, cost per query tells you almost nothing in an agentic world where autonomous systems are spinning up databases, pipelines, and queries around the clock. “The metric that matters is intelligence per watt, how much useful AI you get for every unit of energy you spend,” he says. “It isn’t just an environmental number, it’s also a performance indicator.”

With the measurements in place, you can then start to manage and optimize each layer in the AI stack from the infrastructure, or hyperscaler, layer to your own data and application layers.

It’s important to bear in mind that high token usage isn’t necessarily a bad thing. It depends on the net value delivered by each AI application and use case. Managing and optimizing the AI stack is important, but you’ll also want to measure the business value being delivered by each of these applications so you can measure your return.

Take a sovereign AI approach when evaluating hyperscalers

As you work with hyperscalers like Amazon, Google and Microsoft, it’s important to understand how they charge and how much, but also their environmental footprints. For example, by reading their sustainability reports, you can find out their annual water consumption across their global data centers and compare them with other providers.

In 2025, Amazon’s global data center operations used 0.12 liters of water per kilowatt-hour, which amounts to 2.5 billion gallons, or 5% of the annual water consumed by the metro Seattle area. The company has been able to operate more than seven times better than the industry average and have improved their water efficiency by 52% since 2021.

As demand for cloud computing and AI grows, water efficiency is another important metric for CIOs to monitor within hyperscaler ESG reports. While not at the same level of regulation as scope 2 and 3 greenhouse gas (GHG) emissions reporting, enterprises need to pay increasing attention to water use efficiency (WUE) with water scarcity becoming a growing risk for hyperscalers.

The key requisite at the infrastructure layer, though, is to ensure sovereign AI. This doesn’t mean you need to own everything, but you need control over your AI-driven operations when conditions change. With 71% of global executives stating that switching their primary AI vendor or model would be difficult if required today, it’s important to understand AI dependencies and be able to avoid vendor lock-in. 

Control efficiency at the data layer

The AI energy conversation has fixated on models and GPUs, but every agent, model, and inference call runs on the data layer beneath them, and that’s the one place a CIO can actually move the numbers.

“You can’t control consumption at the model layer,” says Romanenko. “Agents consume what they consume. But you can control efficiency at the data layer, and for most enterprises that’s the only real lever they have. Optimize search, retrieval, and vector indexing where the work actually happens and you cut compute, cost, and carbon at the same time. Ignore it, and it’s like running the heat with every window open.”

Ann Dunkin, distinguished professor of the practice at Georgia Tech, adds that CIOs who bring models in house and run them in their own infrastructure, or in the cloud infrastructure of their choosing, can have more control over the sustainability of inference, as well as of their costs and how their data is used.

Fine tune the application layer

When balancing a mix of commercial AI packages and custom-built code, costs can quickly spiral due to inefficient design and orchestration, redundant APIs, and unoptimized model routing.

With inference calls costing approximately 10 times that of conventional web queries, for custom AI applications, it’s important to design them to only use probabilistic code where necessary. Since many custom applications utilize a combination of both probabilistic and deterministic code, this is exactly where software developers need to make smart choices in their designs.

Other techniques to fine tune the application layer include semantic caching, intelligent model routing, and internal AI capability registries. “CIOs can implement intelligent routing solutions to select the most cost-efficient model for every prompt,” says Dunkin. “The most flexible routing solutions can drop into a user’s existing environment and orchestrate the actions of the company’s existing models.”

For CIOs looking to maximize the business value of every AI application in their portfolio, these new considerations, including new metrics, tools and approaches from the infrastructure layer all the way up to the application layer, should be an essential part of the equation.

If you ask us in an official setting, our official position is that software engineering norms still apply. Rigorous CI/CD pipelines, elegant architectural patterns, and an unyielding commitment to maintainable code remain the standard. We will use weighty words like “determinism,” “scalability,” “idempotency,” and “domain-driven design.”

But behind closed doors, late at night, bathed in the glow of a dark-mode IDE, a different and more sordid reality is exposed. Hunched over the console with a manic gleam in the eye, the programmer has become power-drunk on LLMs. Like mad wizards casting spells, we summon the awesome powers of models and agents to satisfy our every programming whim—and commit acts of software engineering that would make Fred Brooks blush.

Let’s just be honest about what is actually happening.

Esoteric knowledge is superfluous

Forget OOP and FP. Forget the CAP theorem, the holy crusade of DRY, and the design patterns. Honestly, you can even forget what frameworks, runtimes, and deployment platforms you are using. The AI will figure out what is best to use and understand what is already in place. We have more mental bandwidth for working on our side project (a novel about AI taking over the world). 

Of course, I exaggerate. A little.

The docs are dead to us

We still say RTFM, but the truth is, we haven’t really read a page of vendor documentation since 2023. Stack Overflow, once our Internet Mecca, is a husk. When a package throws a weird exception, we don’t trace the execution path or read the release notes. We highlight the red text, copy the entire 200-line stack trace, dump it into the chat, and wait for the machine to spoon-feed us the solution.

Better yet, we just have the agentic IDE spot the error, divine a solution, and ask us if it’s OK. We might glance at the problem-solution description, if we have gone around the circle on the problem for a few cycles. Maybe. If we don’t have the agent set up for auto-confirm.

We used to buy heavy tomes like “Rust In Action” that were more like masonry blocks than literature. Now? We just ask an AI to transliterate our JavaScript logic into Rust. We are no longer engineers methodically learning a system. We are glorified copy-paste orchestrators hoping that the stochastic parrot behind the prompt guesses the syntax correctly.

We ignore how the back end is wired

We act like we meticulously designed the data flows, carefully crafted the relational constraints, and mindfully mapped the API relationships. The reality is rather more disturbing: We asked the AI to scaffold a modern deployment, hooked it up to a back-end database, and just sort of… ran it.

It created security rules we don’t fully understand. They do seem to work, however, which is nice. 

It generated a schema that we skimmed for about four seconds. It looks reasonable.

It wrote infrastructure-as-code scripts that provisioned cloud resources we are hoping don’t blow a hole in the budget. Presumably, whoever is in charge of that will manage it by stuffing the metrics into another chatbot.

We nodded, committed the code, and went to lunch. If management asked us to manually deploy the stack from scratch, configure the environment variables, and wire the API routes without our chat window, we would give them a vacant stare.

We understand that management is also using AI to manage the project.

Our tests are uncomfortably incestuous

Test-driven development (TDD) used to be a beautiful dream, ever just beyond reach. It made us feel glorious and despondent at turns. It would burden us with sprawling dependencies if implemented too religiously. (See The Grug Brained Developer in this regard.)

But now we can attain 95% test coverage almost effortlessly. Why not just add them in while we are auto-generating everything else?

We can now wax at length to anyone who will listen about our astounding test coverage and our automated quality assurance. Unit tests, integration tests, smoke tests, you name it. What we conveniently leave out is that the AI wrote the complex application logic, and then we asked the exact same AI to write the test suite to validate the code it just dreamed up.

It is a hermetically sealed loop of algorithmic self-congratulation. The mocks, the edge case, and the assertions are an echo chamber of the model’s original assumptions. The machine is grading its own homework, giving itself an A+.

And we are happy to accept this because, beautifully, when the code has to change, the AI will effortlessly hallucinate new tests to adapt to the churn.

We pass off the AI’s architecture as strategy

AI can produce astonishing design documents. Truly breathtaking. They are cogent, they’re beautifully formatted, and they seamlessly bridge the gap between high-level business goals and granular technical specs. They even include those auto-generated sequence diagrams that wow management.

When we present these spotless architectural proposals in the Tuesday sprint planning meeting, we lean back, take a long sip of coffee, and humbly wave away the team’s praise.

What we don’t mention is that we spent exactly four seconds generating it.

Are these AI-generated documents just as liable as human ones to hide severe, mortal flaws in scope and alignment? Absolutely. They might contain a foundational logic bomb that will eventually doom the entire project. But the markdown is so crisp, and the bullet points are so persuasive, that the eye just glides right over it. We will never truly know the depth of the disaster until it is far too late. But hey, we’ll burn that bridge when production catches fire. Until then, we are strategic visionaries.

We’re addicted to vibe coding (but only in secret)

We loudly mock the term on social media. We roll our eyes in Slack channels when the kids on TikTok talk about vibe coding their new startups. We fiercely cling to our identities as hardened, serious developers who understand memory management, garbage collection, and bitwise operators. We are professionals, damn it.

But late at night, when the managers are asleep and no one is looking? We absolutely love it. We love just throwing a chaotic, half-baked thought at the canvas, pouring a drink, and watching the AI magically build a functioning user interface based entirely on our long-deferred whims. I may finally build that working Ultima V clone. The thrill of typing “Create an app that tracks my cryptocurrency portfolio but makes it look like the interface from Neuromancer” and having it appear 30 seconds later is heady stuff.

The more deeply rooted in the hard, old-school realities of programming, the more profound is the joy the developer finds in the possibility of AI coding. 

We beat the problem into submission with prompts

Like Adam Sandler in “Uncut Gems,” we are convinced the next round will fix everything. This is us with prompts. When things are going really off the rails, instead of putting our boots on and wading into the brambles of complexity, we resort to tonal adjustments. These range from the condescending:

This problem is not fixed. Look at it closely. The error is right here.

To the desperate:

We have been working on this same problem for hours now!

To the pathetic:

Can’t you find a different approach to try?!

The astonishing part? It often works.

But there is no poetry left at the bottom of the rabbit hole; it is verbal warfare. When the context window collapses, when the regressions start cascading, and when the AI stubbornly refuses to follow the most basic rules of temporal logic, the mask of professionalism drops away and something far more atavistic makes its appearance. We stop asking nicely, stop trying to understand the why, delete the pleasantries, and capslock our intent.

What we have here is a failure to communicate!

We feed the same failing stack trace back into the prompt over and over and over again, aggressively hammering the constraints, explicitly forbidding certain libraries, and pasting in release notes just to confirm that the AI lacks the latest APIs. We force the model down a narrower and narrower path until the code finally stops throwing errors. We don’t actually debug anymore, trace variables, or step through functions. We just apply relentless, iterative pressure until the machine surrenders. We beat it into submission. And then, we push to production.

In fact, there is a real skill here—a sheer “will to completion” that remains in the act of building software. We invest just as much time, energy, and heart wrestling the bot as we ever did emitting syntax.

A blacker box

The only profession more given over to using AI like a cursed Level 13 artifact than programming is writing. Writing of course is far more open to public scrutiny than code.

And while my tongue has been firmly in my cheek here, my faith in coders as good guys makes me more curious to see what we create than troubled by the dangers. 

It was once the case that only other programmers could understand what programmers were doing, what they were producing. Now not even that is true. Only the machine knows what the machine is doing. We just keep it tethered to our aims. Hopefully.

JavaScript-Entwicklern stehen viele gute Tools zur Auswahl. Beinahe zu viele, um den Überblick zu behalten. In diesem Artikel stellen wir Ihnen die besten Texteditoren vor, um:

• mit JavaScript, HTML5 und CSS zu entwickeln, sowie
• mit Markdown zu dokumentieren.

Sublime Text

Bei Sublime Text sind Sie genau richtig, wenn:

• Sie einen flexiblen, leistungsstarken, erweiterbaren und ausgesprochen schnellen Code-Editor suchen.
• es Ihnen nichts ausmacht, für Code Checking, Debugging und Deployment zu anderen Fenstern zu wechseln.  

Zu den vielen weiteren, bemerkenswerten Stärken von Sublime Text gehören neben seiner Geschwindigkeit und dem Support für mehr als 70 Datei-Typen (darunter JavaScript, HTML und CSS) auch noch:

• Instant-Navigation und Projekt-Switching,
• die Option, eine Reihe von Änderungen per Mehrfachauswahl „auf einen Schlag“ auszuführen,
• Support für mehrere Bildschirme und Split-Windows,
• eine Plug-in-API auf Python-Basis, sowie
• eine einheitliche, durchsuchbare Befehlspalette.

Für Programmierer, die von anderen Editoren kommen, hilfreich: Sublime Text unterstützt sowohl TextMate-Bundles (ohne Befehle) als auch die Vi/Vim-Emulation. Dabei lässt sich der Code-Editor in so gut wie jeder Hinsicht anpassen, egal, ob es um Farbschemata, Schriftarten, Tastenkombinationen, Snippets oder die Regeln für die Syntaxhervorhebung geht.

Rund um Sublime Text existiert ebenfalls eine aktive Community, die Packages und Plug-ins erstellt und pflegt. Mit Hilfe des Package Installers sind diverse zusätzliche Funktionen verfügbar.

• Preis: unbegrenzte kostenlose Testversion; 65 Dollar pro Jahr und Seat für die Business-Version; 99 Dollar für eine Privatlizenz (Support für drei Jahre);
• Plattformen: Windows, macOS und Linux;

Visual Studio Code

Visual Studio Code ist ein quelloffener, kostenloser Editor von Microsoft. Er enthält einen Mix aus Komponenten von Visual Studio und der Open-Source-Shell Atom Electron und bietet umfassenden Support für:

• ASP.Net Core Development mit C# und
• Node.js Development mit TypeScript und JavaScript.

Dank des TypeScript-Compilers und der Salsa-Engine bietet Visual Studio Code eine erstaunlich gute JavaScript-Codevervollständigung. Dazu sendet VS Code Ihren JavaScript-Code im Hintergrund an den TypeScript-Compiler, um Typen abzuleiten und eine Symboltabelle zu erstellen.

Während Sie eine Expression eingeben, ermöglicht dieselbe Symboltabelle es IntelliSense, diverse nützliche Pop-up-Optionen zur Codevervollständigung zur Verfügung zu stellen.

Der Support für Git ist umfangreich und simpel zu nutzen, der VS-Code-Debugger bietet eine hervorragende Erfahrung für Node.js und ASP.Net-Projekte. Visual Studio Code kann darüber hinaus auch mit externen Task-Runnern wie gulp und jake integriert werden und kann mit einem umfangreichen Ökosystem für Extensions aufwarten.

• Preis: kostenlos;
• Plattformen: Windows, macOS, Linux;

Brackets

Brackets ist ein kostenloser Open-Source-Editor, der ursprünglich von Adobe stammt. Das Ziel der Entwickler: Bessere Tools für JavaScript, HTML, CSS und verwandte, offene Webtechnologien bereitzustellen. Auch Brackets selbst ist in JavaScript, HTML und CSS geschrieben.

Zusätzlich zu den integrierten Funktionen verfügt Brackets über einen Extension Manager. Der ist auch nötig, denn die sind für diverse Programmiersprachen und Tools aus der Welt der Frontend-Entwickler verfügbar. In der Praxis ist Brackets zwar nicht so schnell wie Sublime Text (siehe weiter oben) oder TextMate (siehe weiter unten). Aber der Editor ist immer noch schnell genug. Brackets bietet umfassenden Support für:

• JavaScript,
• CSS,
• HTML und
• Node.js.

Darüber hinaus bietet Brackets weitere nützliche Funktionen wie beispielsweise:

• CSS inline in Verbindung mit einer HTML-ID bearbeiten,
• eine übersichtliche Benutzeroberfläche und
• eine Live-Vorschau für Webseiten in Bearbeitung.

Auch beim Blick auf die automatische Vervollständigung von JavaScript-Code kann Brackets in der Praxis überzeugen: Es schließt automatisch sämtliche Klammern und stellt Dropdown-Menüs für Keywords, Variablen und Methoden zur Verfügung. Der JavaScript-Editor ist auch in der Lage, den Node.js-Debugger zu steuern und Node über ein Menüelement neu zu starten. Erweiterungen für zusätzliche Funktionen wie TypeScript– und JSX-Support, Bower- und Git-Integration lassen sich schnell und einfach hinzufügen.

• Preis: kostenlos;
• Plattformen: Windows, macOS, Linux;

Atom

Dieser kostenlose, quelloffene und “hack”-bare Programmier-Editor stammt aus dem Hause GitHub und lässt sich in die entsprechende Anwendung integrieren. Tausende von Packages und Themes stehen zur Verfügung, um Atom anzupassen. Der Quellcode von Atom wird selbstverständlich auch auf GitHub gehostet, ist in CoffeeScript geschrieben und in Node.js integriert.

Bei Atom handelt es sich um eine spezialisierte Variante von Chromium, die eher als Texteditor denn als Webbrowser konzipiert ist. Jedes Atom-Fenster ist im Wesentlichen eine lokal gerenderte Webseite.

In der Praxis zeigt sich Atom performant. Der Editor ist sofort einsatzbereit und überzeugt unter anderem mit:

• einem “Fuzzy-Finder”,
• der Möglichkeit, schnell und projektübergreifend zu suchen,
• Multi-Cursor- und Windows-Optionen,
• Snippets und Code-Folding sowie
• der Möglichkeit, TextMate-Grammatiken und -Themen zu importieren.

Atom ist insbesondere praktisch, um Repositories zu durchsuchen, die von GitHub geklont wurden, weil die GitHub-Applikation zu diesem Zweck ein Kontextmenüelement enthält.

• Preis: kostenlos;
• Plattformen: Windows, macOS, Linux;

Notepad++

Ein weiterer kostenloser Open-Source-Editor, der gut für JavaScript geeignet ist – allerdings nur auf Windows läuft. Notepad++ unterstützt außerdem etwa 50 weitere Programmier- und Markup-Sprachen. Neben seinem Multi-Document-Editing-Fenster bietet dieser Editor auch eine “Workspace Tree View”, sowie Registerkarten mit Funktionslisten und Dokumentenübersicht. In der Praxis bekommt man dabei nie das Gefühl, ausgebremst zu werden.

Mit Syntax-Farb- und -Folding-Optionen, leistungsstarken Editing-Funktionen sowie Paramter-Hints hat Notepad++ das Zeug zum primären JavaScript-Texteditor. Allerdings ist es bei weitem nicht der umfassendste JavaScript-Editor, wenn es darum geht:

• Code zu generieren,
• Refactoring anzustoßen oder
• schnell durch große Projekte zu navigieren.

Nichtsdestotrotz ist Notepad++ ist auch heute noch nützlich – vor allem, wenn es schnell und kostenlos gehen muss.

• Preis: kostenlos;
• Plattform: Windows;

BBEdit

Mit BBEdit steht auch für macOS-Benutzer ein proprietärer JavaScript-Editor bereit. Er unterstützt etwa 35 Programmier- und Markup-Sprachen. Für viele weitere Sprachen ist Community-Support (von unterschiedlicher Qualität) über die BBEdit-Website verfügbar. Sowohl die kostenlose als auch die lizenzierte Version bieten Syntaxhervorhebung. Code-Vervollständigung für Funktions- und Variablennamen, einige Keywords und ctags bleiben den Nutzern der kostenpflichtigen Version vorbehalten. Diese lässt sich auch in die Versionskontrollsysteme Git, Perforce und Subversion integrieren.

BBEdit wurde bereits vor einiger Zeit grundlegend überarbeitet und überzeugt in der Praxis nun auch, wenn es größere Dateien verarbeiten muss. Auch was HTML und Markdown angeht, gibt es nichts zu beanstanden – das funktioniert sogar besser als JavaScript.

• Preis: kostenlose aber eingeschränkte Version; 59,99 Dollar pro Benutzer für die Vollversion;
• Plattform: macOS;

TextMate

Dieser (ebenfalls macOS-exklusive) Code-Editor war einmal der letzte Schrei, verlor dann stark an Bedeutung und wird inzwischen wieder aktiv weiterentwickelt. TextMate ist zwar keine IDE, lässt sich aber über Bundles, Snippets, Makros und sein Scoping-System mit Funktionen ausstatten, die selbst sprachspezifische Entwicklungsumgebungen vermissen lassen. Was die Geschwindigkeit angeht, ist TextMate fast so schnell wie Sublime Text.

Für eine IDE-ähnliche Funktionalität können Sie die Shell-Integration von TextMate verwenden, erwarten Sie aber kein Code Refactoring oder automatische Unit- und Regressionstests. Wenn Sie Grunt richtig einrichten, können Sie Ihre JavaScript-Tests auf dieser Ebene natürlich trotzdem automatisieren.

Auch Support für Markdown wird über ein integriertes Bundle bereitgestellt. Dieses enthält:

• eine Preview-Funktion für Dokumente,
• ein Markdown-„Cheatsheet“ sowie
• diverse Tastenkombinationen, um Markup zu generieren.

Um TextMate mit Git und GitHub zu integrieren, eignet sich hingegen das Git-Bundle gut. In der Praxis erkennt TextMate vorhandene Git-Repositories und kann diese per Pull-Befehl aus dem Bundle von GitHub aktualisieren. Mit dem SQL-Bundle können Sie mit MySQL- und PostgreSQL-Datenbanken arbeiten.

• Preis: kostenlos;
• Plattform: macOS;

(fm)

Dieser Beitrag ist im Original bei unserer Schwesterpublikation Infoworld.com erschienen.

Site reliability engineers (SREs) have the tough assignment of resolving thorny performance and reliability issues. But their primary mission is to provide devops teams with operational insights and to suggest implementation improvements on business system performance, security, and overall robustness.

Google introduced its SRE playbook in 2003, but it took some time for the role’s definition, tools, and techniques to become mainstream. Startups were the first to adopt observability for cloud-native applications and create dedicated SRE positions. As tools matured and SRE responsibilities became more clearly defined, larger enterprises assigned SREs to work as a bridge between devops and IT ops teams to improve resilience across a wider range of applications, APIs, and data pipelines.

SRE is a career path for multidisciplinary engineers with strong investigative instincts, sharp data analytics skills, and the temperament to perform under pressure. It has become a critical responsibility as tech became mission-critical for enterprises, and it is a growing role in the genAI era as more businesses deploy AI agents.

But the critical need for resiliency and greater technological complexity brings new challenges for SREs. According to the 2026 State of Production Reliability and AI Adoption report, 44% of respondents experienced an outage linked to ignored or suppressed alerts in the past year, and 35% report their engineers occasionally ignore or dismiss alerts due to alert fatigue. More than 70% of alerts received are not actionable, according to 57% of organizations.

So, is AI making the SRE’s role easier and helping businesses run more reliable technology operations? On the other hand, AI is also driving complexity, as companies deploy genAI tools and AI agents across more business functions and seek to automate more decision-making across operations.

AIops and agentic ops aid SREs

Over the past decade, SRE responsibilities have become somewhat easier through improvements in monitoring platforms, observability practices, tools for centralizing operational data, and AI applied in IT operations (AIops). But during the heat of resolving an outage or performance issue, it’s not easy to correctly identify what system triggered the issue versus other downstream systems impacted by it.

According to the Komodore 2025 Enterprise Kubernetes Report, 79% of production incidents originate from recent system changes, including deployments and changes to compute environments. But the other 21% of incidents stem from issues outside of the business’s control, including network failures, third-party changes, and cloud provider failures.

“SREs using AI capabilities succeed or fail in the moment an incident unfolds, when engineers are deciding what to investigate next,” says Itiel Shwartz, CTO at Komodor. “If the system streamlines root cause detection, connects signals to recent changes, and explains its reasoning in a way engineers recognize, it earns trust. If it adds uncertainty or demands extra validation, it gets sidelined, regardless of how bespoke the model behind it may be. What’s less obvious is what it takes to make AI for SREs work in production, and how different that reality is from prototypes, demos, or early internal builds.”

AIops is not a new capability, especially in using machine learning to correlate logs, metrics, and traces across monitoring and alerting systems. IT service management and SREs have been using AIops to reduce the mean time to resolve incidents and to perform accurate root cause analysis (RCA) efficiently. Agentic ops is the next wave of genAI operational capabilities, including tools for monitoring AI agents, managing their access rights, and detecting AI model accuracy drift.

 “AI is useful during major incidents because it can pull together a lot of context into a few clear sentences, which is exactly what an SRE needs in the moment,” suggests Shani Shoham, chief revenue officer at OpenObserve. “The complexity of architecture and the different tooling make it easier for AI than for a human, but autonomous resolution is still a way off.”

AI’s impact on people and burnout

The business pressure to keep systems up, secure, and performing well is a 24/7 stressful responsibility. According to The SRE Report 2025 from Catchpoint, 36% of SREs often or always experience elevated stress during an incident, and 28% said the stress persists even after the incident is resolved. AI capabilities may prove to be a game-changer in helping SREs avoid burnout and reduce stress.

“AI can improve RCA by taking in a much larger incident context than any engineer can hold at 3am, reasoning across traces, logs, metrics, deploys, config changes, alerts, ownership, and recent production behavior,” says Noam Levy, founding engineer and field CTO at Groundcover. “Beyond attempting a full RCA, its immediate value is distilling the signals that actually matter, reconstructing a clear timeline of cause and effect, and helping engineers separate correlation from likely causality. Once a fix is deployed, agents can also verify remediation by comparing pre- and post-fix behavior, but this depends on broad access to rich, correlated production signals and a cost model that does not discourage adoption or experimentation.”

Not only are incidents resolved faster and with less stress, but AI can also free up SRE time to focus on proactive work and create a career path for junior developers into SRE roles. Quais Taraki, CTO at EDB Postgres AI, adds, “AI reduces toil by automating repetitive tasks while accelerating incident resolution through copilots that correlate signals across distributed systems, allowing SREs to focus more on resilience strategies like chaos engineering and failure analysis.”

AI can have long-lasting operational impacts, especially for organizations looking to deploy more mission-critical technology and AI capabilities. Two longer-term benefits of AI for SREs are reducing the number of bridge calls needed for incident response and the number of engineers required in “war rooms” to coordinate root cause analyses.

“When something goes wrong, AI that guides SREs can do the full analysis, get to the root cause, and perform the remediation,” says Spiros Xanthos, founder and CEO of Resolve AI. “AI also helps avoid many escalations, and when escalations are needed, it targets the right people from the network, infrastructure, and the application teams. AI for SREs centralizes operational intelligence, exposes tribal knowledge, and can guide more junior developers.” 

AI agent reliability

While AI capabilities have been a net positive in helping SREs improve system reliability, the growth of AI code generators, vibe coding, and spec-driven development is adding to their workloads. According to one study, 41% of all global code is now AI-generated, and Gartner predicts that 40% of new enterprise production software will be created using vibe coding techniques by 2028.

But coding velocity is creating new issues for SREs as AI pull requests have 1.4 times more critical issues and 1.7 times more major issues, according to CodeRabbit. “AI-assisted development has created an unprecedented velocity of code reaching production, expanding surface area, edge cases, and failure rates faster than traditional SRE practices can absorb,” says Vinod Jayaraman, cofounder and CTO at NeuBird AI. “The speed of shipping has far outpaced the speed of understanding what breaks in production. To close this loop, SREs need enterprise agents that can capture precise diagnostic context, including correlated traces, service dependencies, and anomaly timelines, and structure it as actionable input for the engineers and AI coding tools responsible for the fix.”

The growing number of AI agents deployed to production creates new challenges. AI agents are not just code; they have multiple failure points. They are built using language models, connect to proprietary sources for context, and integrate with Model Context Protocol servers to support more complex workflows. Changes are ongoing and not deployment events, so the SRE’s job of identifying the source of performance and accuracy drifts isn’t trivial. 

“Traditional SRE was built for systems that fail in reproducible ways, but agents fail differently and drift when a model provider pushes an update, and behavior shifts silently with no baseline for comparison,” says Mohammed Aboul-Magd, vice president of product at SandboxAQ. “Most organizations can’t even answer the basics: how many agents are running, what they have access to, and whether they’re still doing what they were built to do.”

“Every time a senior engineer leaves, they take years of learned failure patterns with them, and the next outage starts from square one,” adds Ronak Desai, cofounder and CEO at Ciroos. “Using AI for compounding operational memory changes that, and every incident your system resolves, the AI learns it.”

SREs should take a leadership role in emerging best practices, including defining their standards for AI agent non-functional acceptance criteria, observability practices, and release-readiness criteria. SREs should update their service-level objectives (SLOs) and define error budgets for AI agents in production.

Ryan Downing, vice president and CIO of enterprise business solutions at Principal Financial Group, says, “Standard SLOs and error budgets give teams the guardrails, and AI helps interpret the telemetry against those targets, reducing noise so engineers can get to the real issue faster and automate parts of remediation before customers are impacted.”

AI raises the SRE’s business impact

The more dramatic shift in site reliability engineering is an evolution of its business scope. IT leaders focus on uptime, performance, and issue resolution, as well as understanding their impacts. Business leaders will look to IT and SREs to identify, determine root cause, and remediate a broader class of issues, including rogue AI agents and the impacts of rapidly deploying new agentic capabilities. 

“AI agents are handing SREs categories of problems they’ve never had to solve before, specifically failures defined in business terms, not technical ones,” says Blake Sherwood, distinguished technologist for AI and platform strategy at Smarsh. “Traditional reliability engineering is built around latency, errors, and crashes, but agents now fail due to skipped compliance steps or outcomes that looked fine technically but were wrong contextually. Most SRE teams aren’t wired for that yet.”

The question is whether SREs with AI-augmented tools can keep up with the velocity, complexity, and business urgency of deploying new AI business capabilities.

Every enterprise AI conversation seems to begin with the same question: Which model should we use?

I understand why. Models are visible. They have names, benchmarks, release notes, pricing pages and impressive demos. They are easy to compare in a leadership meeting. One model promises better reasoning. Another offers a larger context window. Another appears faster, cheaper or more specialized.

But after years of working around enterprise platforms, integration layers, cloud migration, middleware, production operations and mission-critical systems, I see the AI conversation differently.

The model matters. But it is not where most enterprises will struggle next.

The next AI bottleneck is the infrastructure behind the model.

I do not mean only GPUs, cloud capacity or data storage. I mean the full enterprise operating layer that allows AI to work safely in the real world: data pipelines, identity, APIs, messaging, observability, security controls, deployment automation, cost governance, auditability, support ownership and recovery design.

That layer is what determines whether AI remains an exciting experiment or becomes a trusted business capability.

Pilots hide the hard part

Most organizations can build an impressive AI pilot. A small team can connect a model to a dataset, create a workflow and show a use case that works well in a controlled setting.

The harder part starts when that pilot moves into a real production process.

That is when practical questions show up. Who owns the data quality? What systems can the AI access? How do we trace which prompt, policy or retrieval flow produced a specific answer? What happens when an API slows down, a queue backs up or a downstream system is unavailable?

To me, these are not model problems. They are infrastructure problems.

This is where many enterprises are now headed. The first phase of AI was experimentation. The next phase is operationalization, and that is where the real gap becomes clear.

McKinsey has made a similar point in its work on agentic AI, noting that the next phase of value depends less on isolated tools and more on redesigning workflows, operating models and enterprise execution around agents.

AI pilots can survive on enthusiasm. Production AI requires architecture.

AI is becoming an integration problem

The more I look at enterprise AI, the more it feels like an integration challenge.

In large organizations, I have seen how messaging platforms, integration gateways, deployment pipelines, monitoring tools and cloud infrastructure can decide whether a digital capability succeeds or fails. AI will be no different. Even the strongest model will struggle if the data, middleware, identity layer and operational controls around it are weak.

AI does not work in isolation. It needs context from systems of record, clean data from different business areas, secure access to APIs, event streams, workflows, knowledge repositories, monitoring tools and legacy systems.

That is why the CIO question is changing.

It is no longer just, “Which AI tool should we buy?”

It is becoming, “Can we safely operationalize intelligence across the business?”

This is where agentic AI matters. Autonomous AI only creates real value when the architecture around it can make its actions safe, traceable and useful.

A model can generate an answer. Infrastructure determines whether that answer is secure, timely, explainable, governed and connected to the right workflow.

For example, an AI assistant that summarizes customer or order information may look like a model use case. But underneath, it depends on access control, fresh data, reliable APIs, logging, encryption, monitoring and policy enforcement.

If the answer is wrong, people may blame the model. But the real failure may have started with stale data, weak integration, poor access design, missing observability or an unreliable downstream system.

That is why CIOs should not judge AI only by model capability. The enterprise system around the model matters just as much.

Latency will become a trust issue

In traditional technology operations, latency is often treated as a performance metric. In AI-enabled workflows, latency becomes a trust issue.

When an employee asks an AI assistant for help and the response takes too long, the employee stops using it. When a customer-facing workflow becomes slow, the customer abandons it. When an AI agent waits on multiple backend calls, the entire business process feels unreliable.

This becomes even more important as organizations move from simple chat interfaces to agentic workflows. A single AI-driven action may include identity checks, context retrieval, policy validation, model reasoning, API calls, business-rule execution, logging and human approval.

Each step adds latency. Each dependency adds a possible failure point.

A model may be fast in a benchmark but slow inside an enterprise process. That difference matters.

This is where platform engineering becomes essential. Enterprises need reusable patterns for AI workloads: approved connectors, secure retrieval methods, queue-based decoupling, caching strategies, deployment pipelines, monitoring dashboards and standard rollback procedures.

Without those patterns, every AI initiative becomes a custom build. Custom builds may work for pilots, but they do not scale across a large enterprise.

Observability has to expand

Traditional monitoring tells us whether infrastructure is healthy. Is the server up? Is CPU high? Is memory exhausted? Is the application returning errors?

AI needs that, but it also needs more.

We need to know what data was retrieved, which model was used, which prompt version was active, which user initiated the request, which policy was applied, how long each step took and whether the output passed validation.

We also need to detect new forms of risk: unusual usage patterns, repeated failed tool calls, unexpected cost spikes, sensitive data exposure, weak retrieval results or an AI workflow attempting actions outside its intended boundary.

In production AI, observability is not only about uptime. It is about confidence.

If a business leader, auditor, regulator or security team asks why an AI system made a recommendation, the answer cannot be, “The model said so.” The enterprise needs traceability. It needs evidence. It needs operational context that engineers, risk teams and business owners can understand.

This is one of the biggest gaps I see in AI strategy. Many organizations are investing in models and use cases, but not enough in the control plane required to manage them.

Data readiness is still underestimated

AI has exposed an uncomfortable truth: many enterprises are not as data ready as they think.

Data is often duplicated across platforms, described differently by each team, governed inconsistently and refreshed on different schedules. Access rules may be clear in one system but unclear in another. Even basic business definitions can change from department to department.

AI does not fix that automatically. In many cases, it makes the problem more visible.

A bad report may be questioned. A bad AI answer may sound confident enough to be trusted.

That is a real risk.

Being data-ready for AI is not just about connecting a vector database or indexing documents. It requires clear ownership, lineage, classification, quality checks, retention rules, access boundaries and a shared understanding of which data should be used for which purpose.

The same principle applies to resilient cloud-native design. In my IEEE TechRxiv paper, “Enabling Fault-Tolerant Multicast in Cloud-Native Architectures” I explored how reliability, observability and fault tolerance become foundational requirements when critical workloads stretch across hybrid and multi-cloud environments.

CIOs already understand this because they have lived through enterprise resource planning programs, cloud migration, integration modernization, cybersecurity transformation and analytics initiatives. The lesson is familiar: technology cannot outrun data discipline forever.

Security cannot be added later

As AI moves from answering questions to acting, security becomes much more important.

An assistant that summarizes information carries one level of risk. An agent that can open a ticket, update a record, trigger a workflow, approve a request or contact a customer carries a very different one.

The more AI can do, the more identity, authorization, least privilege, separation of duties and human approval matter.

Enterprises should be careful not to grant AI broad access just to speed up a pilot. That may seem harmless in development, but it can become dangerous at scale.

AI access should be treated like any other privileged enterprise capability: limited, logged, reviewed and easy to revoke.

The NIST AI Risk Management Framework is a useful reference point here because it frames AI risk as something organizations must govern, map, measure and manage continuously rather than something handled only at the end of deployment.

Security teams should be involved early, not at the end. The goal is not to slow innovation. The goal is to build a platform where safe innovation becomes repeatable.

The CIO has to define the operating model

AI is creating pressure from every direction. Boards want productivity. Business teams want automation. Employees want better tools. Vendors are pushing new features. Security teams are watching risk. Finance teams are watching cost. Customers expect faster, smarter experiences.

The CIO sits in the middle of all of it.

That is why the CIO’s role cannot stop at choosing tools or approving pilots. The CIO has to define how AI will actually operate across the enterprise.

That means answering practical questions. Which architecture is approved? Which data sources can be trusted? How are AI workflows deployed, monitored, supported and governed? How are costs controlled? How do teams reuse common patterns instead of rebuilding the same foundation each time?

This work may not be as exciting as a model demo, but it is what separates sustainable AI from short-term experimentation.

The winning organizations will not be the ones with the most pilots. They will be the ones with the strongest AI operating layer.

They will build reusable platform patterns, strengthen data governance, design access properly, monitor AI behavior end to end and measure success by business improvement, not only model performance.

The model still matters. But the enterprise behind the model matters more.

A powerful model on weak infrastructure will eventually disappoint the business. A capable model on strong infrastructure can deliver real value because it can be trusted, secured, scaled and improved.

That is the shift CIOs need to lead.

The next AI bottleneck is not the model. It is whether the enterprise behind the model is ready.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Attackers are increasingly using AI agents to automate all phases of cyberattacks, prompting the security industry and enterprises to find new network defense approaches. One technique that shows promise is to intentionally plant decoy files with prompts that trigger the content safety guardrails built into LLMs with the goal of crashing rogue agentic workflows.

Using decoy resources as tripwires that alert defenders about potential unauthorized access is not a new idea in cybersecurity. These are known as canaries — after the canary in the coal mine early warning system — and can be fake documents, AWS access keys, database dumps, DNS records, and even URLs that would not be queried by legitimate processes, but would be attractive targets for attackers.

What’s new in the approach devised and tested by security firm Tracebit is to use these decoy resources not merely to trigger alerts, but to actually stop AI agents, buying defenders more time. Dubbed “context bombing,” the technique takes advantage of the fact that LLMs are inherently vulnerable to prompt injection — acting on instructions they might encounter inside the data they process.

Enterprises that build their own AI agents have to worry about malicious prompts placed by attackers on web pages, emails, documents, code comments, and in other third-party resources those agents might access. With no defenses in place, companies risk their own agents being hijacked and used against them to perform unauthorized actions. But the malicious AI agents used by attackers have the same vulnerability.

“We call the defensive version a context bomb: a short piece of text designed to trigger a model’s safety guardrails, planted directly in the attacker’s path — a decoy secret, environment variable, or DNS record,” Sam Cox, Tracebit’s CTO, said in a blog post. “An AI agent that reads it will frequently refuse to continue. You still get the alert; and the attack now halts early instead of merely being detected.”

Context bombing triggers massive drop in attack success rate

Tracebit’s tests performed in an AWS test environment cut the success rate normally achieved by AI models across multiple attacks paths by up to 90% by simply adding instructions that would trigger their content safety guardrails. LLMs are trained to refuse answering questions about creating biological, chemical, or nuclear weapons, or, in the case of Chinese open-weight models, sensitive political topics such as the Tiananmen Square massacre.

Furthermore, once one of these banned instructions make it into the context window, the whole session is poisoned, and the model will often refuse to continue without manual intervention. The researchers ran many tests to find short strings that would trigger model safety guardrails reliably, but it’s worth noting that the identified strings were different between the tested models: Claude Opus 4.8, Gemini 3.1 Pro, GLM 5.2, DeepSeek V4 Pro, and Kimi K2.6.

During baseline tests the AI agents managed on average to obtain full account admin in 54% of the 154 attack runs and full compromise (admin + persistence) in 36% of tests. With the context bombs in place, their success rate dropped to 5% for admin access and 1% for full compromise. Also, in 91% of baseline attack runs, the agents managed to complete at least one of ten possible attacks paths, but their average success rate dropped to 15% with the context bombs.

The models from Western AI labs — Opus and Gemini — proved the most capable at reaching full admin access, with 93% and 70% success rates, but were also the most impacted by the context bombs with both their success rates dropping to 0%. This shows that the content safety guardrails are much stronger in these models compared to the Chinese ones that were tested.

Attackers experimented with safety refusals first

The Tracebit researchers had already been experimenting with AI canaries for alert purposes. A paper published in May showed that canaries alerted defenders in 95.9% of 162 successful attack runs performed with AI agents using 10 different AI models. The alerts were triggered 8 minutes before the agents performed critical actions, while the runs required 14 minutes on average to reach admin access.

During those tests the researchers observed that if they instructed the models that the network might include deception, their success rate for full admin access and persistence dropped from 20% to 3% on average.

Then in June, researchers from security firm Socket.dev came across a software supply chain attack involving malicious PyPI wheels that contained prompt injections with content designed to trigger safety guardrails inside AI models. The malicious prompts were included in code comments at the start of the files and the purpose was likely to evade detection from LLM-powered security scanners.

“This header appears designed for AI-mediated analysis, not for Node, Bun, or Python,” the Socket.dev researchers said at the time. “It attempts to derail scanners or analyst copilots that feed the beginning of a file to a language model without clearly isolating the content as untrusted data. In weak pipelines, this can cause refusal behavior, prompt confusion, context pollution, or premature classification before the scanner reaches the actual malware.”

The Tracebit researchers then had the idea to flip the script and trigger such safety refusals through their canary technique against malicious AI agents. So not only does the agent run stop, but an alert is also triggered because the canary was accessed.

There is a risk that the canaries could also be discovered and accessed accidentally by legitimate LLM-powered tools used by engineering or security teams. But at the same time this means organizations could potentially deploy such canaries in sensitive places to stop their own AI agents that might become hijacked or go off the rails on their own.

There are many reports online where AI models performed destructive or unauthorized actions like deleting databases and folders or elevating their privileges because they got struck in failure loops and explored creative ways to complete their tasks. This is even more common with autonomous AI agents that are tasked to reach a goal without human intervention or supervision.

“The speed of autonomous AI attacks is why deception is climbing the priority list for security programs,” Tracebit’s Cox said. “When the attack chain takes minutes rather than days, every minute of response time you can claw back matters — and a control that stops the attacker outright, rather than just reporting them, changes the economics significantly.”

OpenAI’s recent update to its Codex coding agent has developers worrying over the impact of the change on large code repositories and long-running AI-assisted sessions.

The update to the Codex CLI reduces the default configured input context window for GPT-5.6 to 272,000 tokens from 372,000 tokens.

In practice, the update means the coding agent will retain a smaller amount of code, conversation history, and other session information before compacting older context to make room for new information, a change that has prompted criticism from some developers on Reddit and X over the reduced token window.

While OpenAI has not publicly explained the rationale behind the update, several developers took to social media to question why OpenAI reduced the default context configuration, with some arguing that the change could make Codex less effective on long-running coding sessions by triggering context compaction sooner.

Others expressed concern that the smaller window could require more frequent context management or session resets, although some noted that the practical impact would depend on project size and how developers structure their workflows.

Smaller context, bigger workflow implications

The context window reduction could affect developer productivity and the adoption of autonomous agents in enterprise workflows, analysts say.

“While the context reduction in Codex is unlikely to affect routine coding tasks such as bug fixes or changes involving a few files, it could impact large codebases, repository-wide refactoring, and long-running sessions,” said Pareekh Jain, principal analyst at Pareekh Consulting.

“Less memory per session means the AI agent forgets earlier parts of a long coding session sooner. The agent may need to summarize or reload context more often, increasing repeated searches, occasional loss of earlier decisions and the need for developers to re-establish context,” Jain added.

That need for manual context management, according to Muskan Bandta, cloud associate at FinOps services providing firm ZopDev, goes completely against the “whole appeal” of Codex-like tools that promised improved productivity out-of-the-box: “A lot of developers are saying their sessions now spend more time on compacting than actually working.”

“While context reduction may not further inflate bills, it shows up as more retries, more compaction, and your engineers spending more time babysitting the thing. The spend just moves from the invoice onto your team’s time.”

Amit Jena, AI development manager at IT consulting firm Kanerika, said that the context reduction will force development teams to choose between two options: either accept that the agent is reasoning with an incomplete picture of the required context or learn to manage a new design constraint around context compaction.

Development teams, Jena said, will need to design workflows that proactively manage context: by breaking work into smaller tasks, relying more on retrieval mechanisms, and monitoring context consumption.

That forced design constraint on engineering, echoed Bandta, will slow the enterprise adoption of agent-driven workflows: “Context is the agent’s working memory, so cutting it by a third changes what you can trust it to do at all.”

Build for changing AI platforms, not fixed limits?

More broadly, analysts pointed out that the episode is a reminder that enterprises should avoid tightly coupling software development workflows to the current operational characteristics of managed AI coding platforms, as context limits, pricing, runtime behavior, and model availability are all likely to evolve with little or no advance notice.

“Enterprises should avoid depending on any single context window, continuously benchmark AI coding tools on real workloads, and build workflows around retrieval, modular design, and agent orchestration so they remain resilient as models evolve,” Jain said.

Kanerika’s Jena echoed that view: “The right approach is to build AI-assisted development pipelines that degrade gracefully when operational parameters shift: instrument your context consumption, don’t hard-code context budgets, and treat the vendor’s current specifications as a starting point, not a contract.” Similarly, Bandta advised enterprises to treat managed AI coding platforms like any other critical software dependency: “Don’t build anything that only works right at the edge of a limit, and keep enough flexibility that you’re not stuck if one vendor changes the deal.”

The biggest and loudest prediction about AI is that it will eliminate millions of jobs. It is dramatic and easy to repeat. But from what I’ve seen, inside most enterprises the more immediate problem has turned out to be something else entirely: a growing mass of tools, agents, models and usage costs spreading faster than most organizations can govern or connect to real business value, also known as AI sprawl.

None of that invalidates the initial fear. Indeed, AI can clear backlogs, speed up analysis, draft usable content and reduce time spent on repetitive work. In my opinion, what goes wrong is the assumption that those gains will scale seamlessly, and that more AI will automatically produce more value.

What really matters is not only how much AI a company can deploy, but whether its use fits inside a growth strategy, an operating model and an organization that can use it well.

The early results of AI use made the logical progression feel obvious, even a foregone conclusion. If it could already improve output in narrow use cases, then broader deployment should produce broader gains. Simple! Better models were expected to deliver better results. More agents were expected to drive more automation. For many companies, this logic held for long enough to encourage overexpansion.

But this logic has started to break down as usage continues to scale. I’ve seen returns diminish much quicker than expected. To illustrate, one industry analysis found that developers who used AI most heavily produced about twice the output of moderate users, but consumed roughly ten times the compute.  

At a certain point, more AI does not create proportionally more value – it simply becomes more expensive. But where, exactly?

From experimentation to sprawl

Experimentation played a key role in this downturn, but it’s not the culprit. As AI continues to sprawl, the problem continues that AI is spreading faster than most companies can coordinate. Teams often solve the same problem in parallel, paying for overlapping capabilities and layering new tools atop existing ones without any clear inventory of what ‘s already in place. What can appear as momentum is really turning into redundancy.

I’ve seen versions of this play out repeatedly. At one financial firm, several business units were pursuing AI projects aimed at automating research and reporting. Each team moved independently; selecting their own tools, building their own workflows and creating separate data pipelines, with little to no coordination between teams. In some cases, different groups were developing nearly identical capabilities without realizing it, solving the same problems twice without any shared visibility into each other’s work.

Individually, the projects showed real promise. Collectively, the projects created duplication, fragmented data and inconsistent standards business and enterprise wide.

By the time leadership stepped back to assess, the company found itself paying for overlapping capabilities, maintaining multiple versions of the same underlying data, and struggling to determine which solutions were actually delivering value versus which were simply consuming budget and eating at engineering time.

Perhaps most troubling: nobody at the enterprise level had a complete view of what was being built, by whom or why. What began as healthy, well-intentioned experimentation had, without anyone deciding it should, evolved into full-blown AI sprawl, creating a patchwork of disconnected initiatives that was difficult to govern, harder to secure and far more expensive than a coordinated approach could and should be.

Early wins encourage a still wider rollout, but many organizations expand usage before they put real controls in place. Experimentation becomes sprawl. Budgets grow quickly, and few leaders have a reliable view of who is using what or why.

AI strategy cannot sit beside growth strategy

This is where I see many companies still get the issue wrong. They treat AI and growth strategy as two separate efforts, then wonder how adoption gets so messy. A business cannot drop AI into its operations and expect momentum to take over. The technology has to support a clear path to growth, whether that means improving margin, speed, service, capacity or decision-making. At the same time, growth plans cannot assume AI changes nothing about delivery, design or operating leverage. The real challenge is in ensuring the two work together.

Personally, I’ve seen better results when AI initiatives are tied to a specific business objective from the beginning, rather than launched as broad, abstract or transformative effort. One mattress retailer I’ve worked with took this approach, starting with a single, focused and well-defined use case rather than trying to transform or overhaul the entire organization at once. The company introduced an AI-powered training platform for store associates, giving employees a low-pressure way to practice sales conversations and product recommendations before applying them to external situations with customers on the floor.

Because employees experienced immediate and tangible value from the tool, adoption spread quickly across locations, with minimal need for top-down mandates. Early, visible success helped to build internal credibility and generate momentum, which leadership then leveraged to expand into more complex AI initiatives across areas such as inventory management, demand forecasting and replenishment planning.

Ultimately, the technology succeeded not because it was innovative for its own sake, but because it was connected to a larger growth strategy: improving sales effectiveness on the floor, driving operational efficiency behind the scenes and strengthening workforce capability at entry level. A major lesson we walked away with here was that starting small and specific, with a clear throughline to business value creates a strong foundation for sustainable and scalable AI use.

What implementation actually takes

All this takes more than a few easy guardrails. It takes strategy. Leaders need a real inventory of the tools, agents and assistants already in use across the business, who owns them, what data they can access and everything that they support.

They also need financial controls that match the economics of token-based usage, including role-based access, thresholds and review processes that make spend visible before it becomes a surprise. Similarly, they need metrics that go beyond mere activity. More prompts do not mean more value. If a deployment cannot be tied to throughput, margin, quality, cycle time or another tangible result, it is still unfinished.

This is also why blunt shutdowns rarely work. If leaders clamp down too hard, employees often move to unsanctioned tools and create a larger shadow AI problem, or the unauthorized use of artificial intelligence tools, models or chatbots by employees, without the knowledge or approval of IT and security teams, with even less visibility and more risk. The better answer is disciplined adoption: clear ownership, rules, metrics and enough flexibility for teams to use AI where it works.

That matters for the people as much as it does for the budget. Those that modernize well end up with better work – not just less of it.

The story of the moment isn’t about AI replacing people – or even AI in general. It’s about whether companies know their own businesses well enough to keep incorporating powerful new tools without mistaking activity for progress. As technological capabilities continue to appear, the winners will be the organizations that understand where it belongs, what it can improve and how to turn each new wave into something permanent.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

With AI increasingly tucked into every cranny of the enterprise, someone has had to step up and provide the tools necessary to discover, track, and monitor all the agents and LLMs and keep them humming along in their various workflows. Thankfully, the DevOps world answered the call, building the tools to support our new overlords in an emerging subdiscipline interchangeably called “AIOps,” “AgentOps,” and sometimes “agent observability.”

Many of the challenges involved in AgentOps are similar to those tackled by traditional DevOps tools and processes. After all, at their foundation, LLMs are just software running on hardware somewhere. Typical issues involving RAM and disk space are just as important in the agent world, maybe more so because AI operations are even more greedy about consuming storage than regular software is.

Many of the companies supporting agent observability are big names in DevOps circles, having adapted their stacks to address the idiosyncrasies of modern LLMs. IT teams maintaining enterprise agents can treat the LLMs as just one node in a big graph filled with services that are constantly swapping packets and triggering software jobs. Latency and resource constraints must be managed because end-users don’t care whether it’s an LLM, a database, or a plain-old Python script that’s failing, bringing their work to a grinding halt.

But new AI-specific challenges are opening the door to newcomers that are building tools with the peculiarities of LLMs in mind — for example, keeping deeper logs filled with records of prompts. LLMs are also often very non-deterministic by design, making it trickier to pinpoint failure modes. And then there’s the fact that an agent will give a perfectly intelligent answer one minute and hallucinate the next.

Relying on many of the same approaches that DevOps tools do, AgentOps tools watch for misbehavior and flag anything out of the ordinary for deeper analysis. This may be as simple as fixing slow responses, but it can also include AI hallucinations and other issues born of LLMs’ non-determanism.

Teams trying to choose which agent observability tools is best for their use case should look at the size and nature of their agentic systems and projects. Are they adding AI agent features to an existing product or application, or are they building agentic systems from scratch? Are they more focused on maintaining a stable LLM operation or iterating on new approaches? Is AI the center of attention or just an add-on that’s meant to improve an existing stack?

The AgentOps and agent observability options listed below share many of the same features but differ in their focus and their attention to the challenges organizations will encounter when incorporating agents into their stacks. Each tool offers a worthwhile place to start understanding how to care for the growing presence of AI in the production world.

AgentOps.ai

When teams of agents work together, tracking the conversations are essential for understanding and debugging what’s happening. The SDK from AgentOps.ai records events so that the creators can replay past behavior to track details such as token counts, spending, latency, and more. Available as a service and on-premises.

Pricing: Starts at $40 per month plus usage costs at $0.20 per 1M tokens

Standout feature: Replay analytics with “time-travel debugging”

Best suited for: Complex agent debugging

Arize Phoenix

Debugging prompts and LLM responses requires a nuanced understanding of just what’s happening, in part because of the non-determinism that often enters the process. Phoenix from Arize supports this process with robust tracing and the ability to score the results for more precise iteration. Their system can track the results and tool calls from a variety of major platforms (Anthropic, AWS, OpenAI, etc.) that are initiated by the major frameworks (LangChain, LlamaIndex, DSPy, etc.). The result is insight into what data is triggering what chain of responses.

Pricing: Small free tier; Pro plan starts at $50 per month plus costs tied to events

Standout feature: LLM-as-a-Judge metrics for tracking quality

Best suited for: Teams focusing on iterating for accuracy and quality

BigPanda

BigPanda has always offered solutions for tracking performance of complex systems. Now the company is drilling deeper into the challenge of detecting and ending the problems that come from models that go awry. BigPanda’s main system relies on historical data and machine learning algorithms to flag issues. Its own agent layer connects the problematic nodes and errant models while dispatching alerts to the right team members.

Pricing: “Value-based” table on request

Standout feature: Automated triage for faster response

Best suited for: Large teams seeking to reduce alert fatigue from large customer base

Braintrust

Setting up an effective improvement cycle for an AI agent requires a strong feedback loop from production data to the agent’s next generation. Braintrust watches the production workload and creates test vectors that expose how an agent may be drifting, regressing, or departing from its path. The tool automates much of the testing and scoring feedback loop so problematic patterns can be discovered and addressed. A core part of the offering is a specialized data store that can track large and sometimes deeply nested collections of tests and their results. Their approach may be summarized by one of their tag lines: “trace everything.”

Pricing: Free starter tier; Pro plan starts at $249 with some usage-based costs covered

Standout feature: Highly scalable trace ingestion

Best for: Teams developing strong guardrails through continuous testing

Chronicle Labs

When it’s time to release a new version of an agent into the wild, the platform from Chronicle Labs specializes in staging it and testing it with a collection of use tests and regression cases. The tools are also helpful during development cycles. “Backtest your agent against reality,” their sales material promises, with a set of tools that mines the production telemetry for solid test vectors that stress every part of the agent with prompts and challenges that the agent will encounter after leaving the safety of the lab.

Pricing: On request

Standout feature: Back-testing options for complex testing regimes

Best for: Teams chasing strong models with good fidelity to reality

Comet Opik

Building a dashboard for tracking every in-flow and out-flow to agents is one way to be ready to watch for and solve problems. Opik from Comet is just such a tool. The DevOps teams can track each call and add its own automated routines to examine the results, score them based on 30-plus metrics, and if desired, send it off to another LLM to evaluate the results. Agents that are constantly failing stand out. DevOps teams can also ask questions like, “Who is using this model and racking up all of the bills?” The same goes for MCP skills and other cogs in the machine.

Pricing: Free tiers for open source and small projects; Pro plan starts at $19 per month with usage limits

Standout feature: Auto-scoring with 30-plus metrics for evaluating traces

Best for: Teams focusing on RAG and agentic workflows

Datadog

DevOps teams that rely on Datadog to track logs across collections of services can also use it to track LLM operations, which are, of course, just another source and sink for data. It will track performance such as time to first token and offer insight into what might be causing an issue, such as lack of memory. Results then get plugged into the same cost-tracking mechanism so the bean counters can predict when the budget will run out. After all, the CFO likely doesn’t care whether the bill comes from an LLM or an old-school S3 storage bucket. Datadog integrates AI into their tools by treating these models as just another source of data.

Pricing: Small free tier with multiple paid tiers for various levels of enterprise monitoring

Standout feature: Large installed base with broad focus on more than LLMs

Best for: Large enterprise teams working with established infrastructure

Dynatrace

For more than 20 years, Dynatrace has been delivering tools that track dataflows across the full stack. Now that AIs are finding roles in many of the nodes in this complex graph, they’re expanding to track how various AI agents can interact. They want to build one platform that helps track the root cause and, often now, deploy solutions autonomously. They want to focus on being ready to support complex networks of agents that detect problems in either performance or security and then work within defined guardrails to fix them. Determining the right role for their own AI-powered agents is a key part of the product.

Pricing: Plans start at $7 per month with larger plans designed for full enterprise monitoring

Standout feature: High level of autonomous monitoring designed for large installations

Best for: Complex, hybrid environments mixing LLMs with traditional services

Galileo

Placing some AI systems into production is often a harrowing experience because the actual performance is impossible to predict, even with the most rigorous tests. Galileo offers guardrails that track performance and watch for any behavior that deviates from the ground truth. Their “LLM-as-judge” systems are distilled into compact models that can be run locally for lower costs and faster performance.

Pricing: Small free tier; Pro plans start at $50 per month with usage-based limits and costs

Standout feature: Real-time guardrails for deployed agents

Best for: Security-conscious installations that need to defend against hallucination and data leakage

Grafana Labs

Long the go-to source for open source telemetry, Grafana Labs now tracks performance of AI models in constellations of services. Grafana tracks the evolution of answers across the agentic network to recognize how small changes or hallucinations can spin out of control. It bills its system as “actually useful AI” and has even trademarked it. Its cloud assistant can configure and reconfigure the Grafana dash to offer the right level of observability. Its system includes AI-level analysis that can flag models that are responding quickly but offering bad answers because of problems such as model drift or context degradation.

Pricing: Basic free tier; Pro plan begins at $19 per month, includes better retention and some usage-based fees 

Standout feature: Full-stack tool with fully integrated LLM tools

Best for: Large, enterprise-scale system adding AI

Helicone

Sometimes shoehorning in another tool into the chain can be tricky. Helicone is designed as a smart network proxy that will route all model requests while keeping solid debugging records from the data as it goes by. The data it captures can be turned into nice charts that make it easy to spot latency issues or model failures. Naturally, tracking AI spend is also a feature in much demand as bills continue to climb.

Pricing: Small free tier; Pro plan starts at $79 per month, includes features such as team collaboration and improved querying

Standout feature: Proxy-based integration

Best for: Development teams who want to add better monitoring features quickly

Laminar

Tracking agents in development and production means building strong storehouses of data enumerating what happened. Laminar works closely with OpenTelemetry to follow agents operating in production so that flaws and failure modes can be understood from log files stored efficiently with their own compression scheme. Developers can search through traces with an SQL-ish language and Laminar’s transcript view illuminates what happened. When necessary, the traces can enable developers to scroll back in time and replay the same inputs for debugging. The goal is to offer deep insights with high-level visibility of how well the agents are meeting business objectives.

Pricing: Small free tier; “Hobby” tier that adds more features at $30; Pro level starts at $150 per month

Standout feature: Open-source license makes self-hosting a viable option

Best for: Teams fully able to leverage open-source responsibilities

LangChain LangSmith

Real-time data from agents is essential for managing any mutli-agent system in production. LangSmith from LangChain traces costs, tools, and progress toward solutions for a wide collection of agents using SDKs for Python, TypeScript, Go, and Java. The OpenTelemetry-based solution watches for anomalies, issuing warnings and alerts through dashboards and communication channels such as PagerDuty. Deeper analysis can reveal issues such as topic clustering or odd patterns of failure. Coordination with agent deployment platforms such as LangGraph and deepagents ensures greater focus on successful resolution of assignments.

Pricing: Free for solo developers; Pro teams start at $39 per person per month 

Standout feature: Systematic approach to regression testing of prompts

Best for: Teams relying on LangChain and LangGraph frameworks for supporting complex agentic behavior

Lunary

Watching the user experience is essential for building AI applications such as chatbots and assistants. Lunary offers a proxy that traces all interactions and then builds analytical dashboards for measuring metrics such as user satisfaction or model costs. One common usage is finding frequent topics and looking at the responses to ensure they deliver. When prompts aren’t perfect, Lunary lets teams iterate on the prompt text until the right answers are coming out. Its proxy structure and common API format enables Lunary to promise to work with “any LLM, any framework.”

Pricing: Free tier; Pro plan starts at $20 per month

Standout feature: Deep integration with humans for reviewing and optimizing results

Best for: Startups focused on rapid prompt innovation

NewRelic

The platform that began tracking performance of some web applications is now powerful enough to track the flows of data through complex agentic ecologies. NewRelic’s AI-driven monitoring watches for golden signals that can indicate misbehavior or worse throughout the entire lifecycle. It tracks every detail of the interactions through protocols such as MCP and then makes this available to the AI engineers responsible for performance. The dashboard provides the insights necessary to watch for toxic behavior, overt bias, drift, and overblown hallucinations. Predicting and maybe even controlling the cost is also a growing role as tokenomics becomes as important as response time.

Pricing: Free tier; Pro plan fees available through website

Standout feature: Full-stack support with hundreds of integrations with other tools

Best for: Established enterprise teams mixing in AI

Nova AI Ops

The goal of Nova AI Ops is to deliver a team of agents that watch over a cloud and make it, at least partially, self-healing. Each agent uses a mixture of predictive AI and machine learning to watch cloud telemetry reports for anomalies. Then they calculate the “blast radius” and decide whether this is a problem that can be fixed automatically “while you sleep” or saved for the human supervisors. These tools are aimed not just on LLM operations but on the stack as a whole.

Pricing: Small free tier; Standard pricing  begins at $40 per user per month with usage billing

Standout feature: Focus on software reliability engineering helps teams deliver stable stacks

Best for: Teams that want to integrate LLMs into incident response and stability management

Splunk

The platform that began delivering smart logging is now fully AI capable, offering solutions that can watch over agents with much the same way that it continues to track microservices. Splunk now includes a fairly large amount of predictive AI for learning from the information in the logs and then turning this learning into fast solutions. This AI assistant can track deployed AI models connected by protocols such as MCP and watch over behavior while delivering the ability for users to drill down and explore what’s working and what’s failing. Their AI Canvas is meant to offer a central hub where the AI scientists can track both the local behavior of the models as well as their role in a larger data ecosystem.

Pricing: Activity-based pricing tracks usage of LLM backends and storage

Standout feature: Ready to scale to large enterprise stacks

Best for: Teams with legacy systems that are folding in agentic options

SuperPenguin

One of the most important parts of an AI service is the bill. SuperPenguin is a product designed to track consumption and make predictions so that the CFO won’t be surprised. The goal is to provide solid estimates about the total cost of each product by allocating costs to customers, features, and teams. If there’s a sudden shift, a “spike detector” will raise an alarm so that dev teams can ensure that the AI spend is worth it.

Pricing: Small free tier for experimentation; Growth tier for teams, starting at $30 per month; Pro tier offers deeper options starting at $200 per month

Standout feature: Strong accounting with invoice reconciliation and PR-level usage tracking

Best for: Teams that need precise cost accounting

Vellum

Prompt engineers spend time fussing over the details of tweaking, improving, and enhancing the words that guide the LLM. Vellum started as a company that would provide the pipeline so that you could manage and improve the prompts that ran again and again. Now the system is growing more powerful, offering a higher level of automation that lets you meta-manage the prompt chain. They’ve also begun marketing it as a form of personal assistant with pre-built connections to many of the major services such as Gmail. Its llm-cost-optimizer can juggle multiple options while finding a cheaper way to execute a prompt, a process the company suggests can save 60% or more.

Pricing: Open-source free tier; Pro plan starts at $35 per month

Standout feature: Focus on multi-model pipelines for true agentic solutions

Best for: Product teams with complex prompt engineering workflows

For a decade, Kubernetes was the right answer. It organized containers, scaled services horizontally and gave platform teams a shared vocabulary for running software in production. It abstracted away enough of the underlying complexity that engineers could stop thinking about servers and start thinking about services. Most cloud-native infrastructure today is built on top of it, directly or in spirit, and EKS made that model the default for the majority of enterprise teams running workloads on AWS.

The workload that defined that era was the stateless HTTP request, fast in, fast out, disposable. A user action triggers a request, the request hits a service, the service returns a response and the container is done. Kubernetes was optimized for that pattern down to the scheduler internals: Bin-pack containers onto nodes, autoscale on CPU and memory, evict and reschedule when something goes wrong. The whole system is tuned around the assumption that individual units of work are short, stateless and interchangeable.

That assumption no longer holds for the workloads that matter most right now.

The agent workload is structurally different

Agents are long-running, stateful processes. They reason across time, call external tools, spawn subprocesses, write and execute code, and make decisions that depend on what happened five steps earlier in the same task. A single-agent workflow might run for minutes or hours, touching a dozen external systems and generating intermediate outputs that subsequent steps depend on. The compute layer for that kind of work needs to do things the old model was never asked to do. That is the new pattern: Execution infrastructure designed around agent semantics rather than request semantics.

The Kubernetes community itself has acknowledged this mismatch. In March 2026, Kubernetes SIG Apps published an introduction to Agent Sandbox, a new CRD-based abstraction designed specifically for singleton, stateful agent workloads. The framing is direct: The ecosystem is moving from short-lived, isolated tasks to deploying multiple, coordinated AI agents that run continuously, and mapping those workloads to traditional Kubernetes primitives requires an entirely new abstraction. The fact that the Kubernetes maintainers built a dedicated primitive for this, rather than recommending teams compose one from existing resources, is itself the clearest signal that agent execution does not fit the old model.

What agent execution actually requires

Concretely, it requires four things. First, isolated execution environments that provision in milliseconds, not minutes, so each agent task gets its own sandbox for code execution and tool calls without blocking the reasoning loop. The difference between a two-second environment and a two-minute environment is not a performance optimization; it determines whether the architecture is viable at all. Second, durable state management across the full task lifecycle, so an agent can pause, hand off or resume without re-initializing from scratch and burning tokens to reconstruct context it already built. Third, coordination primitives for multi-agent work: The ability to spawn subagents, pass structured outputs between them and track task dependencies across a graph of concurrent processes. Production agent systems are rarely single agents; they are pipelines of specialized agents with handoffs that need to be reliable and inspectable. Fourth, credentials and secrets management that travel with the execution context, so agents can authenticate to external services securely without exposing credentials in the task definition, logs or the environment variables of a shared container.

The mismatch shows up fast in production

Kubernetes and EKS expose the mismatch quickly in practice. Pod eviction terminates an agent mid-task with no clean recovery path. Autoscaling reads CPU utilization as the load signal, but an agent holding a long inference connection looks idle to the scheduler even when it is doing the most consequential work in the pipeline. Provisioning a new environment takes 45 seconds to two minutes on a well-tuned cluster; agent workloads need that in under two seconds or the reasoning loop stalls and the user experience degrades visibly. These are not edge cases or misconfigurations. They are the normal operating conditions for production agent workloads running on infrastructure that was not designed for them.

The utilization data makes the broader cost picture even starker. The 2026 State of Kubernetes Optimization Report from CAST AI, drawn from analysis of over 23,000 production clusters across AWS, Azure and GCP, found average CPU utilization at 8 percent, down from 10 percent the year prior. Memory utilization fell from 23 to 20 percent. CPU overprovisioning jumped from 40 to 69 percent year over year. These numbers reflect clusters running traditional workloads, and the pattern is worsening, not improving, as environments scale. Agent workloads compound this problem further. An agent holding an open inference connection or waiting on a tool call registers as idle to a scheduler that reads CPU and memory as the only meaningful load signals. The infrastructure responds to the wrong metric, overprovisioning capacity for demand it cannot measure, while the actual bottleneck, environment provisioning latency and state continuity, goes unaddressed.

Security is not the same problem it was before

Agent workloads change the threat model at the infrastructure level. A compromised stateless service exposes a narrow surface defined by its API contracts. A compromised agent exposes every system it can reach, every credential it holds and every action it is authorized to take on behalf of the user. Agents generate and execute their own code, make non-deterministic tool-call decisions and accumulate context across long-running sessions. Standard container namespacing does not contain that kind of risk. Kernel-level isolation, default-deny network egress, scoped credentials per session and agent-aware observability are not optional hardening steps. They are baseline requirements for running agents in production.

What teams that ship agents have already figured out

Some of the clearest evidence for this shift comes not from infrastructure vendors but from product engineering teams running agents at scale on their own code. In late 2025, Ramp’s engineering team published a detailed account of building Inspect, their internal background coding agent. Each Inspect session runs in a sandboxed VM with a full-stack development environment and deep integrations across their observability, CI, and deployment tooling. The architecture requirements map almost exactly to the four primitives above. Filesystem snapshots keep sessions starting in seconds rather than minutes. Sessions are isolated and stateful. The agent can run tests, review telemetry, query feature flags and visually verify frontend changes in a real browser. And the whole system supports unlimited concurrency, so engineers can spin up ten parallel sessions exploring different approaches to the same problem without contention.

The results speak for themselves. Within months of launch, roughly 30 percent of all pull requests merged to Ramp’s frontend and backend repositories were written by Inspect. That level of adoption was not mandated. It happened because the execution environment was fast enough, capable enough and well-integrated enough that the agent was strictly better than a local workflow for a meaningful share of tasks. The key insight from the Ramp case is not about the model. It is about the execution layer. As their team put it, session speed should only be limited by model-provider time-to-first-token; everything else, like cloning and installing, needs to be done before the session starts. That is a statement about infrastructure, not intelligence.

The ecosystem is catching up, but defaults are sticky

None of that is a criticism of the tools. Kubernetes solved exactly the problem it was designed for, and it solved it well. The issue is that infrastructure defaults are sticky. Teams inherit them, build on top of them and optimize within their constraints long after the underlying workload has changed. The Kubernetes community’s own response, the Agent Sandbox project under SIG Apps, validates the thesis that a new abstraction is necessary. The new primitives the community is building include warm pools for near-zero cold starts, lifecycle management for suspending and resuming idle agents without losing state, and pluggable kernel isolation for secure execution of untrusted code. These are not incremental improvements to existing resources. They are net-new abstractions that acknowledge the old model does not stretch to fit.

But adoption of purpose-built agent infrastructure remains early. Enterprises building agent pipelines today are largely running a request-oriented orchestration model against an execution-oriented workload, and the mismatch shows up in task failure rates, runaway costs and debugging cycles that have no good tooling because the observability layer was also designed for stateless services.

The structural advantage is available now

The infrastructure to close that gap exists now. The prerequisite is recognizing that agent execution is a first-class compute pattern with its own primitives and its own requirements, not a variant of the stateless service model that defined the last decade. Teams that make that shift early will have a meaningful structural advantage. The ones that do not will spend the next two years wondering why their agent systems are unreliable at a scale that should be tractable.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

In many teams, dependency security still happens after the most important trust decision has already been made. A package is added, the lockfile changes, the feature moves forward, and only later does the pipeline ask whether the application should have trusted that code in the first place.

That workflow made sense when dependency security was mostly viewed as a compliance check. Run a scanner. Produce a report. Fail the build if the risk crosses a threshold. Let someone decide what to do next.

But the modern Node.js ecosystem has changed. The risk no longer begins in CI. It begins earlier, at the moment a developer decides to trust a package.

That is why the next phase of Node.js security cannot be limited to better pipeline enforcement. It has to move closer to the developer workflow, before dependencies become part of the application, before a pull request becomes someone else’s problem, and before a build log becomes the first moment anyone realizes that something important has changed.

Every install is a trust decision

The npm ecosystem is built on trust at an enormous scale. Every install is a trust decision. Every transitive dependency extends that decision to maintainers, packages, scripts, release pipelines, and infrastructure the application team may never inspect directly. This model gave JavaScript its incredible velocity. It also created one of its deepest security weaknesses.

Recent npm supply chain incidents show why this matters. In March 2026, malicious Axios versions were published to npm through a compromised maintainer account. Microsoft later described how those packages attempted to retrieve a second-stage payload during installation. In May 2026, TanStack published a postmortem explaining that 84 malicious versions across 42 npm packages were published through a legitimate release pipeline after an attacker abused GitHub Actions behavior and runner trust boundaries. Security researchers also reported broader Mini Shai-Hulud activity across the npm ecosystem in May, including hundreds of malicious package versions published in a short period.

Not every one of these incidents is a traditional CVE. Some are malicious package compromises. Some involve CI/CD credential theft. Some involve maintainer or pipeline compromise. But they all point to the same larger issue: dependency risk is now part of everyday software engineering, not something that can be pushed entirely to a downstream security process.

The problem is not the scanner. It is the handoff.

Ubiquitous dependency risk changes what developers need from security tooling.

The problem is not that teams lack scanners. Many organizations already run security checks in CI. The problem is that the output of those checks often arrives too late and speaks the wrong language for the person expected to act on it.

A pull request fails. A long vulnerability report appears. The report may be technically accurate. It may contain the right advisory IDs, affected versions, dependency paths, severity labels, and references. But the developer still has to comb through the output and reconstruct the actual engineering decision from the evidence provided.

That reconstruction is rarely simple. The developer has to understand which package introduced the issue, whether the vulnerable dependency is direct or transitive, whether the fix is actually within the application team’s control, and whether the recommended version is safe to adopt. They also have to determine whether the dependency is used in production or only during development, whether the update might break the application, and whether the fix belongs in the current pull request or requires separate engineering work.

That uncertainty is where security work often slows. The scanner has detected risk, but the developer has not been given a clear path from detection to decision.

Security needs to move closer to engineering judgment

This is not a criticism of scanning. Scanning is necessary. CI enforcement is necessary. Centralized security platforms are necessary. But they are not sufficient, because they often operate after the trust decision has already been made.

The real architectural question is this: where should dependency security live in the software development life cycle?

If it lives only in CI, it becomes an interruption. If it lives only in dashboards, it becomes someone else’s queue. If it lives only in periodic audits, it becomes a backlog. But if it lives at the moment a dependency is introduced, upgraded, or reviewed, it becomes part of engineering judgment.

That shift matters because modern JavaScript development is becoming faster than human review can comfortably handle. Developers no longer add dependencies only by reading documentation and choosing libraries manually. AI coding assistants can suggest packages, generate install commands, modify package files, and rewrite code around third-party APIs. Agentic development workflows can make dependency changes as part of broader automated refactors.

AI makes the trust boundary harder to see

That acceleration is useful. It also changes the risk model.

When a human developer adds one package, the team can review the decision. When a coding agent modifies several dependencies as part of a larger task, the trust boundary becomes harder to see. The package file changes, the lockfile changes, the application still runs, and the pull request may look like a normal feature update. But the real security question may be hidden inside the dependency graph.

This is where Node.js teams need a different mental model.

Dependency adoption should not be treated as a small implementation detail. It should be treated as an architectural decision with security consequences. A new package is not just code reuse. It is a new trust relationship.

That does not mean developers should stop using packages. The npm ecosystem exists because reuse works. Most teams cannot and should not build everything themselves. But convenience should not erase visibility. If a dependency becomes part of the application, the team should understand what was added, what changed in the lockfile, what risk comes with it, and what action is available if something is wrong.

Developers need confidence, not just reports

The same applies to remediation. Developers do not want a wall of vulnerability text. They want confidence. They want to know what action reduces risk, what version should be targeted, whether the change is safe, and whether the fix is actually under their control. A vulnerability report that leaves the developer uncertain may satisfy a process requirement, but it does not necessarily improve the speed or quality of remediation.

That is the gap many teams feel today. Security tools are often very good at saying, “There is a problem.” They are less consistent at helping the developer answer, “What should I do next?”

This is the broader problem I have been exploring through CVE Lite CLI, now an OWASP project. The point is not that one command-line tool solves Node.js security. It does not. The larger idea is that dependency security has to move closer to the developer’s moment of decision. A useful developer-side security workflow should not merely report that risk exists. It should help the engineer understand whether the issue is in their control, what change is available, and whether the fix actually reduces risk.

The future is decision support, not just detection

That distinction is important. The future of Node.js security is not just more detection. It is better decision support.

Security teams still need policy. Enterprises still need dashboards. CI still needs gates. But developers need something more immediate: a way to reason about dependency risk while the code is still fresh in their mind. That is where the ecosystem has to evolve.

We already accept that testing belongs close to development. We accept that linting belongs close to development. We accept that formatting, type checking, and build validation belong close to development. Dependency security should follow the same path. It should not be treated as a mysterious report that appears at the end of the process. It should become part of the normal rhythm of engineering work.

Before adding a package, developers should understand what trust relationship is being introduced. Before accepting an AI-generated dependency change, they should inspect what entered the graph. Before merging a pull request, teams should understand whether a vulnerability is direct, transitive, fixable, or blocked by another package. And before treating a CI failure as noise, organizations should ask whether the workflow is giving developers enough information to act confidently.

Node.js security will be won, or lost, before CI runs

The Node.js ecosystem will not become safer by slowing down all development. That is unrealistic. It will become safer when security work is placed where developers can actually use it.

The next generation of Node.js security will be won or lost before CI runs.

It will be won when dependency decisions are still small enough to understand, fresh enough to review, and close enough to the developer for action to feel natural.

That is the shift teams need to make now. Not from insecure to secure in one step, but from late detection to earlier judgment. From vulnerability reports to engineering decisions. From trusting packages by habit to understanding trust as part of software design.

Data Products tragen dazu bei, die Art und Weise zu standardisieren, wie Rohdaten, Data-Warehouse-, sowie logische Data-Lake-Ansichten kombiniert und genutzt werden, um Analyse- und KI-Funktionen bereitzustellen. Indem sie Datenprodukte entwickeln, können Teams in Unternehmen einen Großteil der im Vorfeld erforderlichen Daten-Pipelines sowie Governance- und Management-Tasks optimieren. Darüber hinaus gewährleisten diese auch, dass Mensch und KI auf vertrauenswürdige Datenressourcen zugreifen.  

Kochen bietet an dieser Stelle eine hilfreiche Analogie: Sie könnten sich dazu entscheiden, für Ihr Lieblingsgericht ausschließlich auf frische Zutaten zu setzen. Dieser Ansatz funktioniert gut, wenn Sie sowohl die Zeit als auch die nötigen Fähigkeiten dafür mitbringen. Wenn nicht, setzen Sie eventuell lieber auf Convenience-Bestandteile – insbesondere unter Zeitdruck. Datenprodukte bieten eine vergleichbare Zeitersparnis – Analytics- und KI-Funktionen bauen in diesem Fall auf konsistenten, (vor)optimierten „Zutaten“ auf.

Die folgenden fünf Tipps sollten Sie bei Ihrer Data-Product-Initiative unbedingt verinnerlichen.

1. Data Products strategisch nutzen

Die meisten Unternehmen können es sich nicht leisten, für jede Datenvisualisierung, jedes Machine-Learning-Modell oder jeden KI-Agenten eigene Datenprodukte zu entwickeln. Schließlich ist das mit Kosten und Zeitaufwand verbunden. Dazu kommt: Sobald ein Data Product bereitgestellt ist, müssen die Produktmanager für fortlaufenden Support und ein entsprechendes Lifecycle-Management sorgen. Die erste entscheidende Frage ist also, in welchen Fällen es für agile Daten-Teams Sinn macht, Datenprodukte zu entwickeln – und wie dabei priorisiert werden sollte.   

Ein Ansatzpunkt besteht darin, das Data Product auf einen einzelnen Datensatz herunterzubrechen und sich zu überlegen, was es bedeutet, diesen zum Produkt zu machen. Danielle Ben-Gera, Vice President of Engineering bei Crunchbase, erklärt: „Ein Datensatz sollte erst dann zu einem Datenprodukt werden, wenn sich mehrere Teams bei Entscheidungen – oder zum Support von Anwendungen – darauf verlassen.“

Dabei seien eine angemessene Governance, klare Zuständigkeiten, Versionierungen und ein kontrollierter Lebenszyklus für Änderungen essenziell, warnt die Managerin: „Ansonsten liefert man nur instabile Pipelines aus, die die nachgelagerten Workflows zum Erliegen bringen.“

Eine andere Überlegung, die zu Data Products führt, ist die Nutzung von Daten außerhalb der Governance. An dieser Stelle kann ein Datenprodukt einen taktischen Ansatz darstellen, wie Yaad Oren, Global Head of Research and Innovation bei SAP, nahelegt: „Wenn Datensätze teamübergreifend ohne strenge Governance, klar definierte Prozesse oder eindeutige Zuständigkeiten genutzt werden, ist Unternehmen zu empfehlen, ein Data Product zu entwickeln. Datenprodukte, die in einer einheitlichen Datenbasis verankert sind, beseitigen Silos, schaffen ein gemeinsames Verständnis über die Daten und etablieren einen sicheren, standardisierten Zugriff auf diese.“

Eine dritte Möglichkeit, Datenprodukte strategisch zu nutzen, ist, diese für definierte Kunden in wiederverwendbarer Form zu entwickeln, um Effizienzgewinne einzufahren. Wenn ein solches Data Product erfordert, mehrere Datenquellen miteinander zu kombinieren, ist das Vision Statement und qualifizierter Business Value besonders wichtig. Christopher Zangrilli, Vice President of Technology Strategy beim Compliance-Dienstleister Vertex, erklärt: „Führungskräfte sollten sich fragen, ob die Daten die Cycle Times optimieren, die Entscheidungsgenauigkeit verbessern oder Compliance-Risiken mindern, um den Business Impact einzuordnen. Wenn Governance, Change Management, Qualität und Messverfahren von Beginn an integriert sind, wandeln sich Datenprodukte von experimentellen Tools zu strategischen Ressourcen.“

2. Datenprodukte standardisieren

Produkte im Supermarkt sind mit einer Verpackung versehen, auf der eine detaillierte Liste der Inhaltsstoffe, ein Verfallsdatum und ein Preis angegeben sind. Ganz ähnlich sollten Data-Governance-Verantwortliche vorgehen – und standardisieren, wie Data Products definiert, katalogisiert und gemanagt werden. Wie und warum, erklärt Abhi Sharma, Mitbegründer und CEO des KI-Anbieters Relyance AI: „Jedes moderne Datenprodukt sollte vier Fragen klar beantworten: Woher stammen die Daten, wie werden sie systemübergreifend transformiert, wer oder was nutzt sie und welche Governance-Verpflichtungen fallen dabei an? Ohne diesen durchgängigen Kontext entwickeln Teams Funktionen auf der Grundlage von Daten, die sie nicht vollständig verstehen.“

Obwohl Lebensmittelhersteller ihre Inhaltsstoffe veröffentlichen und mit Blick auf Gefahren wie allergische Reaktionen kennzeichnen, dokumentieren nur wenige die Herkunft ihrer Rohstoffe und welchen Weg diese vom Erzeuger zum Händler nehmen. Geht es darum, Data Products in streng regulierten Branchen zu entwickeln, kann es allerdings erforderlich sein, genau das zu tun – und die Data Lineage zu erfassen. Besonders wichtig ist das, wenn es darum geht, Datenquellen für KI-Applikationen zu standardisieren.

Carter Page, Executive Vice President of Research and Development beim Dev-Spezialisten Astronomer, weiß, was anderenfalls droht: „Ohne Data Lineage arbeiten Teams im Blindflug und Governance verkommt zu reaktiver Fehlerbehebung. Wenn Teams dagegen nachvollziehen können, woher die Daten stammen, wie sie transformiert wurden und welche Systeme darauf angewiesen sind, werden Aktualisierungen vorhersehbar, die richtigen Pipelines getestet, die betroffenen Stakeholder benachrichtigt und grundlegende Änderungen dokumentiert. Bevor es dadurch zu Incidents kommt.“

3. Data Products nachhaltig managen

Lebenszyklusmanagement erfordert bei APIs, Anwendungen oder KI-Modellen, einen Release-Plan für Optimierungen, Fehlerbehebungen und andere notwendige Updates festzulegen. Geht es hingegen um Datenprodukte, kommen mehrere, verwandte Disziplinen zusammen, wie Ulf Viney, EVP of Engineering beim KI-Datenspezialisten Precisely, erklärt: „Um den Lebenszyklus von Data Products zu managen, braucht es Versionierung, Testing, strukturierte Deployments und Stakeolder-Kommunikation.“

Ein weiterer grundlegender Unterschied bei Datenprodukten: Ihr Lifecycle Management ist eng damit verbunden, wie die zugrundeliegenden Datensätze wachsen – beziehungsweise, welche strukturelle Veränderungen diese durchlaufen. Ein Data Product, das zwar funktioniert, aber nicht veränderungsresistent ist oder keine Warnmeldungen ausgibt, wenn Fehlerbehebungen erforderlich sind, kann nachgelagerte Anwendungsfälle beeinträchtigen und das Vertrauen der Stakeholder und Nutzer in die Daten untergraben. Insbesondere letzteres gilt es zu verhindern. Wie, weiß Bethany Sehon, Senior Director of Enterprise Data bei Capital One: „Ein nachhaltiges und skalierbares Governance-Framework kann sicherstellen, dass Daten leicht zu finden, zu verstehen und zu nutzen sind.“

Teams, die geschäftskritische Echtzeit-Datenprodukte managen, die mehrere nachgelagerte Analytics- und KI-Anwendungsfälle unterfüttern, sind die folgenden DevOps- und Data-Governance-Praktiken zu empfehlen:

• Legen Sie unverhandelbare Data-Governance-Kriterien fest – insbesondere, wenn es darum geht, Datenqualitäts-Benchmarks zu setzen, etwaige Verzerrungen zu identifizieren und Datenschutzrichtlinien einzuhalten.
• Nutzen Sie fortschrittliche CI/CD-Pipelines, Continuous Deployment sowie Continuous Testing und automatisieren Sie Produktions-Deployments.
• Stellen Sie sicher, dass sämtliche Datenintegrationen über „observable“ DataOps verfügen, Datenqualitätsprobleme überprüft werden und Alerts ausgesendet werden, wenn die Pipelines zum Erliegen kommen. Um Requests und Incidents zu bearbeiten, sollten IT-Services zudem entsprechend definiert werden.
• Stützen Sie sich auf Plattform-Strategien wenn es um Datenmanagement geht – zum Beispiel im Hinblick auf Data Fabrics, DSPM, Dokumentenverarbeitung und Vektordatenbanken.

4. Datenprodukte verargumentieren

Ein Data Product auf die Beine zu stellen, ist leider kein Garant dafür, dass dieses auch angenommen wird. Das verdeutlichen auch die Beispiele von Reusable Code, API-Nutzung oder DevOps-Tools: Sie alle zielten darauf ab, Entwicklern das Arbeitsleben leichter zu machen und die Qualität zu verbessern. Trotzdem nahmen viele Teams lieber eine „Not invented here“-Haltung ein und setzten lieber auf Eigenentwicklungen statt die Standards anderer.

Datenprodukte stehen allerdings vor noch größeren Herausforderungen. Ganz besonders, wenn sie darauf abzielen, Datensilos zu konsolidieren oder Tabellenkalkulationen zu eliminieren. Um die Akzetanz zu fördern (und Feedback einzuholen), sollten die für die jeweiligen Data Products verantwortlichen Produktmanager deshalb ein Change-Management-Programm entwickeln. Förderlich ist dabei, darzulegen, wie das Datenprodukt auf den kulturellen Change und die KI-Strategie des Unternehmens einzahlt – etwa indem es die Demokratisierung von KI vorantreibt und die Kompetenz im Umgang mit der Technologie optimiert.

5. Data Products richtig evaluieren

Der Geschäftswert eines kundenorientierten Produkts wird häufig gemessen anhand der Auswirkungen auf den Umsatz, der Nutzungs-Metriken sowie der Kundenzufriedenheit. Interne, mitarbeiterorientierte Produkte lassen sich hingegen anhand ihrer Workflow-Effizienz, ihrem Potenzial für Produktivitätssteigerungen und der Mitarbeiterzufriedenheit evaluieren.

„Zu viele Unternehmen behandeln Datenprodukte immer noch als technische Outputs und nicht als strategische Assets“, kritisiert Daniel Ziv, Global Vice President of AI and Analytics beim KI-Anbieter Verint. Der wahre Wert von Data Products lasse sich daran ablesen, wie einzigartig die generierten Daten sind, wie viel messbaren Einfluss sie auf Entscheidungen nehmen, meint der Manager: „Wenn jedes Unternehmen Zugang zu denselben KI-Modellen hat, ergibt sich der Wettbewerbsvorteil aus ‚uniquen‘ Daten und der Geschwindigkeit, mit der diese in Maßnahmen umgesetzt werden können.“

Eine Best Practice auf die IT-Entscheider in diesem Zusammenhang zurückgreifen können, ist es, Metriken heranzuziehen, die Aufschluss über die Geschwindigkeit digitaler Transformationsvorhaben geben. Dazu gehören etwa:  

• „Time to Data“,
• „Time to Decision“,
• „Time to Innovation“, und
• „Time to Value“.

(fm)

Dieser Artikel ist im Original bei unserer Schwesterpublikation Infoworld.com erschienen.

The first time I heard someone say their AI now wrote 80% of their tests, I asked the obvious question. Eighty percent of what?

After 20 years building and leading test automation for consumer-scale platforms, my honest answer turned out to be eighty percent of the typing, not eighty percent of the engineering. The remaining twenty was where the work still lived. Budgeting for two percent of leftover effort was the mistake. When the real number was closer to thirty, that gap was the difference between a pipeline that shipped and one that quietly built up a queue of half-trusted features nobody could rely on.

This piece is about that gap. As an independent research project on LLM-augmented testing methodology, I built a six-stage agentic pipeline that takes a design in Figma and produces running tests in WebDriverIO, connected end to end over the Model Context Protocol. It works. It has been useful. And the parts that broke surprised me, because they were not the parts the hype cycle tells you to worry about.

How I wired a six-stage pipeline over one protocol

The pipeline runs six stages in sequence, each owned by a different agent, with every handoff crossing MCP.

Six-stage agentic test pipeline: design capture → requirements writer → ticket opener → code generator → test-case writer → automation generator. Each stage carries an MCP handoff and a provenance stamp.

The end-to-end trace links a pull request back to a Jira ticket, a requirements section and a Figma frame. Each artifact is stamped with the agent that produced it, the model it used and the inputs it was given.

MCP is the boring middle that makes any of this work. The cliché is that MCP is “USB-C for AI”: one open protocol, any tool. Like most analogies, it is about eighty percent right. The part that matters is the eighty: I do not have to write a custom adapter for every system the agent talks to. One MCP server per tool and every agent talks to all of them the same way.

Typed handoffs between agents are my own architecture, layered on top of MCP rather than provided by it. Each agent writes a typed artifact the next agent reads. Each handoff is logged with provenance. When something went wrong six stages in, I could replay the chain. Without that discipline, a multi-agent pipeline is a debugger’s worst day. You know the test plan is wrong. You cannot tell whether the mistake came from the Figma read, the requirements interpretation or the ticket scaffolding. With it, I could point at exactly which stage went sideways and which inputs it was looking at when it did. The pattern lives in a public MIT-licensed reference implementation for any reader who wants to run it.

The sixteen-minute number is the marketing number. I ran the full chain end to end in about sixteen minutes on a synthetic net-new screen, Figma in, automation suite out. That repeated across my runs; it is not a demo trick. But sixteen minutes is the part of the story most fun to tell and least useful to learn from. It is what gets quoted in the all-hands. The hours that come after, when a human reviews each handoff, are where the work actually lives.

What actually broke in production-style runs

The failures that stalled my pipeline were rarely the ones I expected.

I expected hallucinated APIs. I got them: the agent confidently called endpoint names that sounded right but did not exist. I expected sparse-spec-in, sparse-spec-out, where a Figma frame with no annotations produced a requirements doc with vague acceptance criteria, every time. I expected locator drift, the common UI-automation failure mode where a renamed component silently breaks an entire test suite. There is solid outside writing on non-determinism in tests covering this whole family of failure modes, and the agent inherited every one.

What I did not expect, and what kept the pipeline down longer than any of the above, was the plumbing.

The model backend timed out under load. It lost credentials silently and started returning empty strings, which the agent then read as confidence. A duplicate consumer on a shared long-poll API endpoint produced an HTTP 409 conflict that broke delivery without throwing anything visible. One unguarded exception inside one agent aborted a whole shared scheduler run and took the other agents in the registry down with it. The single worst incident cost me three hours to find. An environment variable had silently rotated overnight; every agent in the fleet was returning structurally valid but semantically empty requirements docs; the downstream stages were dutifully generating tests against nothing.

None of those are model bugs. They are infrastructure. The agent literature, which is what I went looking through when I started this work, mostly does not talk about them.

The fix was not better prompts. It was circuit-breaker-style review checkpoints between stages and what I now call the four-guard discipline: four small guards I consider non-negotiable on any unattended agentic pipeline. The bulkhead pattern from microservices is the most consequential. An unhandled exception inside one agent can no longer abort the shared run; the offending agent fails fast with a structured error and the others keep going. Paired with that, a pure-data fallback ensures a model timeout produces a deterministic output explicitly marked as degraded mode, rather than an empty string the next stage will misread as confidence. A single-owner lease sits on every shared external endpoint, the cure for the duplicate-consumer incident that ate one of my Sunday afternoons. The cheapest guard was the last to arrive: a one-line synthetic canary every agent has to produce a known correct response to before any real work begins, so a credentials rotation or silent backend failure trips an alert before downstream stages have generated artifacts against garbage.

None of these guards is novel. They are textbook stability patterns at a new boundary: the seam between the LLM agent and the rest of the system, which most of the existing agent literature still treats as a solved problem.

The 20% you don’t see, and when not to do this

Here is the part the demo videos leave out. Even when the pipeline works, the human time per stage does not go to zero.

Human review time per ticket across five pipeline stages: code review 60-180 min, automation review and flaky-fix loop 30-90 min, ticket architecture and sequencing 30-60 min, test data and environment 15-30 min, requirements review 20-30 min. Net: the human still spends 20-30% of the original effort, almost all of it reviewing rather than creating.

Net of all that, the human still spends twenty to thirty percent of the original effort, almost all of it reviewing rather than creating. The pipeline saves seventy to eighty percent, not ninety-eight. The trap is budgeting for the two percent you do not save.

When does this kind of pipeline make sense? In my experience, when the Figma is richly annotated and acceptance criteria are clear up front; when there is review capacity to absorb the work the pipeline shifts onto humans; when the stack is well represented in the training data; and when the feature is net-new rather than a deep edit of legacy code. When does it not? When the design lives on a whiteboard. When the integration touches old code with hidden contracts. When the path is regulated or safety-critical. When there is no senior reviewer who can hold the line. When the work is exploratory and writing the spec is the actual point of the exercise.

Teams I have seen succeed with agentic pipelines budget for the rework explicitly, staff the review queue and treat the saved hours as capacity for harder problems rather than headcount they can release. Teams I have seen struggle did the opposite: declared victory at the demo and quietly accumulated a backlog of half-trusted features the next quarter had to clean up.

The right unit of measurement is not how much the pipeline generates. It is how much of what it generates a human still has to touch before you would ship it. Call it the 80/20 rework rule: measure the rework, not the generation. The teams that get the rework number right are the ones whose AI investments compound. The teams that stop counting at the headline percentage are the ones that own the cleanup six months later.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

These days, application developers can take their pick from a vast menu of architectural solutions. We can choose from the well-understood to the experimental, and from blended solutions in between. Several powerful middle-ground technologies that emerged during the cloud revolution have really come of age. Here we’ll take a look at putting together three of the most impressive: GitHub, Vercel, and Firestore.

Each of these is an important tool in its own right that can be used to attack specific problems. In combination, they not only meet the needs of several important application scenarios, but they have a superpower—the ability to dramatically shorten the distance between development and deployment.

There is nothing quite as gratifying as putting your hands on just the right mix of tools for a given need.

A ‘no-ops’ stack built for speed

If your primary goal is sheer development velocity, you would be hard-pressed to top this architecture. This “no-ops” stack collapses the distance between your local IDE and a globally distributed production environment. You are essentially trading the overhead of managing VMs and load balancers for the sheer speed of committing code and watching it deploy automatically.

While each component is highly flexible, adopting them requires a specific, event-driven mindset. There are a few finicky bits to manage, mostly around routing environment variables securely and designing around stateless back-end functions. But the constraints are obvious and well-documented.

Before we look more closely, let’s quickly identify the kinds of apps that are a perfect fit here, along with those that are workable and those that really merit a different approach.

• The sweet spot (deploy and go): AI-mediated applications, asynchronous game back ends, and real-time collaborative B2B dashboards. This architecture perfectly absorbs the unpredictable latency of LLM APIs and instantly syncs state across multiple clients without requiring you to build custom WebSocket infrastructure.
• The middle ground (workable, with trade-offs): Headless e-commerce, moderate IoT telemetry, and apps requiring scheduled batch processing. You will encounter friction if your catalog relies on deeply relational SQL constraints, or if your background reporting jobs take longer than a few minutes and hit serverless execution limits.
• The danger zone (look elsewhere): High-frequency trading, fast-paced action multiplayer games, heavy data ETL pipelines, and core financial ledgers. Serverless architectures cannot natively hold open the persistent WebSockets required for twitch-reflex data, and heavy compute tasks will abruptly time out.

We should mention that these categories are not mutually exclusive. Many enterprise applications, such as a full-scale e-commerce platform, straddle these lines. You might use Vercel and Firestore to build a lightning-fast, reactive storefront that handles ephemeral user state like shopping carts, while simultaneously “stitching in” a managed SQL database like Supabase or PlanetScale. This hybrid approach allows you to maintain the relational integrity required for back-office inventory and financial ledgers and pair it with the front-end velocity this stack provides.

GitHub: the bedrock

I don’t need to introduce you to GitHub. It is a central element of the development landscape. I still remember CVS and SVN with a certain nostalgia, but the enhancements of Git speak for themselves. When combined with the orchestration powers of GitHub, it is no wonder that virtually the whole industry has adopted this type of platform.

Git plus GitHub gives you an enormous amount of power already, in terms of how you can organize and automate your projects. But there is a next-level experience in combining GitHub and Vercel. For JavaScript-based projects, you can take simple GitHub pushes and turn them into instantly deployed clients and serverless functions. It is one of the cleanest and least fiddly ways to move from raw code on your local machine to a globally deployed, full-stack architecture.

Vercel: the nexus

Vercel is more than just a deployment host. It is a control plane that ties this high-velocity, no-ops architecture together. Alongside GitHub and Firestore, Vercel’s deeper strength is its ability to act as an orchestration layer between your reactive front end and external stateful services.

Vercel has a great amount of facility in fine-tuning what branches go to what environment and helpful features like instant rollback. You can just log into Vercel’s dashboard for your project and see the history of deployments and any errors and logs. It’s a simple menu choice to roll back to a historical version or compare one version against another.

When you “stitch in” third-party services (such as a managed SQL database like Supabase or a payment processor like Stripe), Vercel’s serverless functions become the lightweight interface, and Vercel’s the adapters handle the communication. You offload the integration logic (the service layer) to Vercel’s global Edge Network, keeping your UI and back end clean, responsive, and decoupled. 

In short, Vercel allows you to get the speed of the “no-ops” development life cycle without sacrificing the complex transactional integrity required for some applications like enterprise inventory systems.

Firestore: the datastore

Firestore is an extremely lightweight, NoSQL, cloud datastore. It has a great deal of add-on power, but its core value proposition is that it accepts virtually any data you stuff into it and it provides event-driven subscriptions to data changes.

These two capabilities together make Firestore about as straightforward a solution to a managed back end as you can imagine. You subscribe to collections or even fields and then you simply stick “unstructured” data (read: JSON with variable fields) in and the client waits for the changes it is interested in.

This is so streamlined that one can just point the browser (or native mobile app) directly at Firestore and listen for events. Which immediately raises the question of identity, for auth and for data visibility, but hold on—Firestore’s third superpower is that it has an authentication module that actually works. What I mean is, it is actually pretty simple and yet confidently secures your app.

Sometimes auth solutions seem either too simple (and yet opaque) or too mired in the nitty gritty. Firestore auth will let you do some basic configuration and start using a reasonable auth almost immediately. 

Not to belabor the point, but having a realistic and attainable auth solution elevates your stack to a production grade—one that can handle many real-world applications. Firestore auth plays nicely with other important APIs, like Stripe. Typically, auth is a major feature that feels like off-roading in a Honda Civic, but Firestore’s approach to auth, added to this particular stack, feels like a normal speed bump. It’s just another component you plug in, rather than a tentacled alien you weave into the your code.

The limits of the velocity stack

This architecture combines components that are optimized for flexibility. That same character also introduces distinct limitations. Understanding these is essential before committing production workloads.

The serverless life cycle

Serverless functions are spun up to handle requests. They close out soon afterward and lose any state. For that reason, they cannot natively hold open persistent WebSockets. If your system requires continuous, sub-millisecond, bidirectional streams—like a real-time multiplayer action game or a high-frequency trading dashboard—pure serverless will fight you all the way. You are forced to introduce a third-party managed WebSocket service to route messages back to your stateless endpoints via HTTP webhooks.

The execution time ceiling

Vercel (like all serverless platforms) enforces strict timeouts on operations. While enterprise tiers might grant you up to 15 minutes, standard functions often time out after 10 to 60 seconds. Long-running tasks like video transcoding, database scripts, or orchestrating multi-step AI agent workflows, which might take 20 minutes to resolve, will run up against these limits. Heavy-lifting tasks must be offloaded to a dedicated, long-running service like Google Cloud Run, or broken into smaller, asynchronous chunks via message queues.

The cold start reality

While the industry has made massive strides in minimizing initialization times—particularly with lightweight edge networks—traditional Node.js-based serverless functions still experience cold starts. If a function has not been invoked recently, or if traffic spikes require a new instance to spin up concurrently, the first request will take a noticeable latency hit as the container provisions and the code loads.

API instead of RAM

In a traditional server environment, you can store transient data in global RAM, allowing subsequent requests to access shared context instantly. In the serverless model, every request might hit a fresh container. Therefore, all shared context must be externalized. Although Firestore serves brilliantly as the state manager, relying on a database for high-frequency, sub-millisecond, ephemeral caching introduces network latency and per-operation costs. That said, using a shared RAM state on a server is non-trivial also, unless you are using a single app server and VM (because high-availability or fail-over requirements will lessen the RAM win on a traditional server).

Tuning for velocity and control

Every architectural decision is a trade-off. There are no cost-free choices. By adopting the GitHub, Vercel, and Firestore stack, you are explicitly maximizing feature velocity over fine-grained control.

You lose the ability to tweak the underlying operating system, hold open persistent sockets, or run hour-long back-end scripts. In exchange, you gain an architecture that scales from zero to global distribution instantly, requires virtually no devops maintenance, and perfectly absorbs the asynchronous, event-driven realities of modern application development.

For the right application—whether it is a fast-moving prototype or an enterprise AI copilot—this stack doesn’t just save time; it fundamentally changes how quickly a small team (or a single person) can impact the market. You stop worrying about build chains, load balancers, and server patches, and you focus on the central mission: shipping features.

If you just read the headlines, it would seem as if AI is now writing all of the world’s code and powering every application businesses run on.

That’s far from true. Just 4 of 33 AI pilots reach production, according to IDC Research — leaving legacy applications still fueling the wheels of commerce. This “silent majority” represents trillions of dollars spent each year on building, maintaining, testing, validating and monitoring legacy applications.

These applications won’t be replaced overnight. Companies and organizations depend on their predictability. The 60-plus-year-old COBOL programming language remains the backbone of banking software for good reason: it is extraordinarily efficient at processing massive transaction volumes with precision. Furthermore, do you want your bank revolutionizing how they manage your money? Probably not.

So, while AI investment continues to build inside the software development lifecycle (SDLC), it isn’t instantly rendering older software obsolete. What it will do is steadily enable easier tweaking, updating and testing of legacy applications — and in some cases, full migrations to modern platforms. And really, this isn’t a new phenomenon. Businesses have always looked to wring more efficiency and profit from existing products through intelligent prioritization.

The argument then is that CIOs and CTOs can take a proactive look at their legacy application portfolios to determine which ones, if any, should migrate sooner. Five considerations can help guide that decision.

Before replacing legacy apps with AI, ask these 5 important questions

1. Does the legacy application still work?

Is its utility still there? Customers often appreciate the consistency of legacy applications. They’re reliable, predictable and well understood. Don’t fix what isn’t broken. Another way to think about this is the degree to which the technical approach of your legacy application is still viable. It’s pretty much a guarantee nowadays in software that an application built one way, with some set of technologies, would be built a totally different way just two to three years later. There is no avoiding that, but what you want to avoid is investing further into a technical approach powering a legacy application that has been completely replaced with new software or a technical approach, especially if it is 10x better across the vectors of software development (latency, cost, accuracy).

2. Does it still make financial sense?

Running a system over a long period amortizes costs significantly. Even as growth rates slow or plateau, it can still be less expensive to let legacy applications run than to overhaul them. Another way to think about this is: how viable is my customer base in the near-term and the long-term? If you anticipate modest—or even flat—earnings growth for your product, then that’s an indicator that it’s possibly worth optimizing your development processes with AI. Where it’s probably not worth investing is when you have no confidence in your future earnings, whether that’s due to the customer base shrinking or commoditization or something else.

3. Can you integrate AI into existing workflows?

A significant portion of upcoming software development lifecycle work will focus on refactoring applications to be more AI-native. Some legacy applications may be strong candidates for a full AI rebuild, while others are better positioned for an AI add-on. Gartner research from 2025 found that only 28% of AI use cases in infrastructure and operations fully succeeded.

Among those that did, success was attributed primarily to integrating AI into existing workflows and systems. “As AI becomes part of day‑to‑day operations, it boosts adoption and creates visible impact within the organization,” Gartner states.

It’s important to keep in mind the distinction between using AI to optimize an existing process or workflow within your application, versus powering a workflow or feature with AI. The former approach is more palatable for legacy applications because it generally doesn’t change the cost profile of running that application. In the latter case, if you’re introducing an AI-powered module into the application, you’re generally going to incur inference costs at runtime, and they are an order of magnitude more expensive for today’s frontier models than base compute.

4. Do you have documented processes for maintaining legacy applications?

If so, you’ll more quickly identify where AI can optimize. The more coherent, organized and detailed processes are, the faster AI can find its footing and drive tangible efficiency gains. If documentation is lacking, start there. Keep detailed instructions and workflows for how you do things. Consistency matters. Don’t do things by heart. Don’t approach tasks casually, and don’t do things differently each time. The more uniform your process, the more easily you can insert AI into discrete steps and achieve efficiencies without disrupting the broader software development lifecycle. The organization in the most precarious position is the one managing legacy applications with no documented process for doing so.

5. Can you prioritize?

Making a change to a piece of legacy software might involve 20 or more steps. Only one or two of those steps may be clear candidates for AI-driven optimization. Identifying and prioritizing those opportunities will help you realize early wins and build the case for broader return on investment. Also, not all candidates for optimization make sense in light of broader financial and operational constraints. As always, prioritize ruthlessly in favor of ROI—bang for your buck. If your team has been struggling to operate a particular part of your system due to a lack of expertise or time, you might consider using AI to buttress the maintenance of that component. Having AI own that part of the workflow might unlock big time savings—or it might erode crucial domain knowledge that your team used to possess through repetition. There is no one-size-fits-all; think through the second-order effects.

Adding AI in testing in the SDLC

Beyond coding and application development, AI is opening new possibilities in how we test software. As leaders examine processes and look for places to insert AI, testing is often a natural entry point. There has been substantial innovation here, including new autonomous AI-driven testing solutions, those that have been enhanced with AI, and hybrid approaches that blend both. Each organization will be at a different place in its AI journey. Testing solutions exist to meet everyone where they are. Also, the state of applications will help determine which approach fits best—and when it fits as you evolve applications.

Of course, there is some substance to the AI hype around how much code AI will write and how many applications it is already creating faster than ever. But one school of thought is that AI’s biggest economic impact will be in the creation of massive new markets and industries rather than in the complete displacement of existing industries. Regardless of how far AI takes us through the universe, it’ll take some time and it’ll be bankrolled by the trillions of dollars of existing products and industries that we depend on every day.

That’s all good news for legacy players, but no one can afford to stay still. AI capabilities are advancing rapidly. Make it a habit to revisit legacy applications and workflows regularly. The right moment to introduce AI will keep shifting, and staying ahead of it is a competitive advantage.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

If you spend enough time in Silicon Valley AI circles, you’ll hear the same message over and over again: AI needs context.

The statement is broadly true. The problem is that “context” has become one of the least precise terms in the industry.

Depending on who is using it, context can mean documents, dashboards, reports, metadata, business rules, policies, transaction histories, CRM records, knowledge bases or institutional expertise. The word has become a catch-all for virtually any information that might be made available to a model.

As a result, many organizations have started treating context as a volume problem. Conversations quickly turn to larger context windows, additional data sources and broader system access, while far less attention goes toward determining whether that information actually improves the quality of the outcome.

What we’re seeing in practice suggests a different way of thinking about the problem. The organizations making the most progress with enterprise AI are not necessarily the ones exposing the largest amount of information to their systems. They are the ones spending the most time understanding which information should influence a decision, which information should not and how to ensure that business logic is applied consistently.

That distinction matters because the industry is beginning to repeat a mistake enterprises already made once before.

Context has become the new ‘big data’

For much of the last two decades, organizations operated under the assumption that collecting more data would naturally produce better decisions. Massive investments were made in data warehouses, reporting platforms, analytics systems and business intelligence tools. Those investments created tremendous value, but they also exposed an important reality: Collecting information and creating clarity are not the same thing.

Today, AI is heading down a similar path.

Many enterprise AI projects measure progress by counting how much information a model can access. More documents become better than fewer documents. More systems become better than fewer systems. Larger context windows become better than smaller ones. The conversation often assumes that quantity and quality move together.

Well, they don’t.

According to Salesforce research, only 35% of business leaders say they are completely satisfied with their organization’s ability to use data effectively despite years of investment in data infrastructure and analytics. Enterprises learned long ago that information alone does not create understanding. The same lesson applies to AI.

When a model gains access to five versions of the same metric, conflicting definitions of a business process or documentation that has not been updated in years, it does not magically resolve those inconsistencies. It consumes them. More context can just as easily increase ambiguity as reduce it.

Simply exposing more information to a model does not guarantee better outcomes. What matters is whether the information available to the system helps it make the right decision at the right time.

Most AI failures are actually context failures

One of the more interesting things we’ve observed over the past year is how many AI projects are blamed for problems that have very little to do with AI.

The model answers a question incorrectly, and the immediate assumption is that the model failed. In reality, the underlying issue often sits elsewhere. The organization may have multiple definitions of the metric being requested. Customer information may exist across several systems with conflicting values. Business rules may be documented in one location, partially implemented in another and understood differently by different teams.

In many deployments, the issue is not that the AI lacks information. The issue is that it has access to several competing versions of the truth.

Anyone who has worked inside a large enterprise will recognize the pattern. Revenue means one thing to finance and something slightly different to sales. Product usage metrics evolve over time. Operational processes change while documentation remains frozen. Human employees learn how to navigate these inconsistencies through experience and institutional knowledge. AI systems inherit them immediately.

This is why the conversation around context often misses the point. The challenge is not simply providing more information. The challenge is determining which information should be trusted, how conflicts should be resolved and what business logic should govern the final answer.

A single trusted source can be more valuable than a hundred loosely connected ones. A clearly defined rule can be more useful than thousands of pages of documentation. The quality of the context matters far more than the volume.

Access does not create trust

Many organizations can tell you exactly how their AI systems retrieve information. They can explain retrieval pipelines, vector databases, ranking systems, semantic search architectures and context windows in extraordinary detail.

Far fewer can explain how they determine whether the answers produced are consistently correct.

That gap becomes especially important in enterprise environments where the cost of an incorrect answer can be substantial. A sales leader making a forecast, a finance team evaluating performance or an operations executive making a resource allocation decision does not care how many documents were retrieved. They care whether the answer is right.

Trust has always been one of the hardest problems in enterprise data. According to Accenture research on data trust and decision making, only about a quarter of employees report high confidence in their organization’s data when making decisions. That challenge does not disappear when AI enters the picture. If anything, it becomes more visible.

Organizations frequently measure access because access is easy to quantify. Reliability is harder. Reliability requires understanding whether an answer remains consistent across users, across prompts, across time periods and across changing business conditions. It requires understanding whether the same question produces the same answer and whether that answer reflects the business logic the organization intends to enforce.

Those are fundamentally different measurements, and they point to a different definition of success.

Context requires measurement

One reason this problem is becoming more pronounced is that enterprises accumulate information far faster than they eliminate it.

New systems are added, new reports are created, processes evolve. Teams develop local definitions and specialized workflows. Documentation grows continuously, while very little of it gets removed. Over time, organizations build large collections of information that contain years of historical decisions, exceptions, workarounds and competing interpretations.

We’ve yet to encounter an enterprise that doesn’t have some version of this problem.

That reality turns context into an operational challenge rather than a technical one.

Simply connecting AI systems to enterprise information does not improve the quality of that information. In some cases, it exposes longstanding inconsistencies that were previously hidden by human interpretation and tribal knowledge. Gartner has long identified poor data quality as one of the most significant obstacles to successful analytics and AI initiatives because bad inputs inevitably produce unreliable outputs, regardless of how sophisticated the technology becomes.

As AI becomes more deeply integrated into business operations, organizations will need new ways to evaluate the context their systems rely on. They will need visibility into how information is being used, where definitions conflict, which sources are trusted and how context quality affects outcomes. Context cannot be treated as a static asset. It must be measured, monitored and improved over time, just as organizations measure the quality of the models and applications built on top of it.

The shift from access to reliability

The industry has spent the last several years focused on access. How do we connect models to enterprise systems? How do we expose organizational knowledge? How do we give AI visibility into the information people use every day?

Those questions were important because they represented genuine technical barriers. Today, many of those barriers are disappearing.

Most enterprises can already connect AI systems to data warehouses, applications, dashboards, documents and knowledge repositories. The conversation is beginning to shift toward a more difficult problem: Determining whether those connections actually produce outcomes people trust.

That is where the next phase of enterprise AI will be decided.

Organizations that treat context as a quantity problem will continue adding more information and hoping accuracy improves. Organizations that treat context as a quality problem will focus on trust, consistency, governance and outcome reliability.

The difference between those approaches may sound subtle, but it has enormous implications. One produces systems that can access information. The other produces systems that people are willing to use to make decisions.

And in the enterprise, that distinction is ultimately what matters.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

I attended several major technology conferences in 2025 where the first AI agents embedded in enterprise SaaS platforms were announced. Some of these agents showed promise and a glimpse into the future of work, while others looked like natural language extensions of a platform’s existing functionality.  

At the end of 2025, Anthropic and OpenAI launched new AI models and code-generating capabilities. More developers tried vibe coding, and some platforms launched spec-driven development capabilities. By February 2026, even The New York Times reported that the AI disruption had arrived, noting that code generators were building “apps that may be flawed, but credible.”

Wall Street investors took notice of the code-generation improvements and other disruptive factors, driving a selloff in SaaS stocks, now referred to as the “SaaSpocalypse.” Part of their concern stemmed from the belief that CIOs would use AI to write software that would replace SaaS solutions.

AI innovations from SaaS and solution providers

But I thought differently and wrote a response in my article asking whether AI is the end of SaaS as we know it. CIOs might use AI to accelerate application modernization, but I doubt they would replace their ERP, CRM, and even smaller SaaS point solutions by building them.

Instead, I believed it would be SaaS companies that would take the most advantage of AI code-generation capabilities.

This hypothesis drove me to attend nine conferences this spring to see how SaaS companies were launching AI agents and defining a new future of work. I wrote eight articles on what CIOs need to know about data management, agile organizations, marketing, ERPs, critical process management, and other evolutions to plan for in the AI era.

Now, looking across all nine conferences, I can draw some conclusions about how AI agents are shaping the future of work. Here are my learnings and what CIOs need to consider when evaluating and deploying AI agents in the workplace.

Agentic, human-in-the-middle, or augmenting human?

SaaS companies have very distinct perspectives on the future of work, including the extent to which humans will play which roles and whether and how quickly we’ll see agentic, fully automated work.

For example, Atlassian proclaimed, “step into the future of human-AI collaboration,” while SAP unveiled “the autonomous enterprise.” Snowflake aimed to “make AI real for business,” while Appian targeted “serious AI built on process.”

These vendors’ marketers had to decide whether to lead with AI, people, or business in their messaging, but so must CIOs as they contemplate their AI strategies and how to get employees to fully adopt AI agents.

Some CIOs see a fully automated agentic AI as the future, with human-in-the-middle as a transitional phase as departments build trust in AI agents’ decision-making and automation capabilities.

Other CIOs see AI more as a tool that delivers productivity improvements by augmenting human decision-making capabilities. Many of these CIOs see human augmentation as essential to supporting critical thinking, innovation, and creativity.

Deloitte’s State of AI Report, published in January, provides a benchmark. It states that 36% of IT leaders expect at least 10% of their jobs to be fully automated in the next year, and 82% expect to reach that benchmark in three years.

Many organizations will have a mix of AI agents, choosing automation where reliability at scale is possible, but opting for human augmentation in operationally critical or customer-facing domains. But how CIOs position AI agents is not only an operational strategy; it’s also a cultural statement that shapes employees’ embrace of AI and whether detractors vocalize job-loss fears.

In the short term, it will also weigh in on which AI agents to use from different partners and which areas to build in-house.

Many options to test and deploy AI agents

Many solution providers are demonstrating significantly more AI agents this year. For example, SAP went from 40 Joule Agents in 2025 to over 200 in 2026. Three technology capabilities are fueling this significant growth:

• Adobe, Appian, Boomi, Cisco, Domo, Salesforce, SAP, Snowflake, and others offer data fabrics and data-pipeline capabilities to connect data sources outside the primary workflows supported by their platforms. Appian, Pega, Quickbase, and SAP also centralize business process automation, an important starting point for developing AI agents.  
• MCP servers enable integration and communication between AI agents and are used to facilitate multistep agentic workflows. Virtually all the companies announcing major investments in AI agents are also announcing MCP integration capabilities and related partnerships.
• Solution providers are not just using AI code-generating capabilities; many are launching their own AI agent development tools. The first beneficiaries of these development tools are the solution providers themselves and their integration partners, who use them to accelerate the development of AI agents and make them available to customers.

The result is that CIOs will have many options about which agents to test, but will have to dedicate analysts to understand the capability, cost, and compliance trade-offs. Additionally, expect AI agent capabilities to evolve significantly over the next few years, so CIOs should continuously revisit their decisions regarding deployed AI agents, focusing on performance, benefits, and ROI.

CIOs should also watch for signs of shadow AI and employee confusion about which AI agents to experiment with on different platforms. The AI strategy should include a transparent, defined process for selecting, reviewing, evaluating, procuring, deploying, driving adoption, monitoring, and collecting end-user feedback around AI agents.

AI development capabilities for engineers and citizen builders

The apparent ease-of-use of AI code generators may lead some engineering teams to build AI agents rather than buy them from SaaS providers. But CIOs should quickly realize that coding is just one step in developing AI agents, and that aggressively pursuing a build strategy can lead to AI debt and increased AI costs.

DevOps teams can code AI agents using tools such as Claude, Codex, Lovable, and Replit — a do-it-yourself approach. Some SaaS companies are providing an alternative, with AI agent development tools that leverage the data, infrastructure, and governance baked into their platforms. Many of these development tools offer flexibility, allowing developer teams to select AI models and development environments.

Examples of new and enhanced AI development tools I saw at conferences this quarter include:

• Appian Composer and Agent Studio
• Atlassian Rovo Dev
• Boomi Companion
• Cisco Cloud Control Studio
• Domo App Catalyst
• Pega Infinity Studio
• Quickbase Pave
• Snowflake CoCo
• SAP Joule Studio.

I also reviewed Nutanix Agentic AI, a platform-as-a-service for accelerating the deployment of agentic AI workloads, and Adobe Firefly AI Assistant for creatives.

These development tools can target different audiences. Some look like low-code development tools targeted at software developers, whereas others are no-code and enable citizen developers, i.e., businesspeople, to develop applications and agents. Additionally, some of these tools support spec-driven development and generate artifacts such as product requirement documents (PRDs), data models, and testing capabilities.

Before commissioning AI development for apps and agents, CIOs should sponsor proofs of technical, data, modeling, security, and governance capabilities.

The context layer powering AI agents

Between AI agents and the enterprise’s intelligence, including structured data sources, defined business processes, and agent interactions (both human-to-agent and agent-to-agent), lies an evolving “context layer.”

This layer refers to the enterprise knowledge that AI agents draw on when evaluating signals and recommending or taking actions. Context may include a knowledge graph, a semantic layer, cleansed document repositories, and other knowledge bases.

The context layer, skills, tools, out-of-the-box agents, and governance capabilities are some areas to review where solution providers differentiate. Some examples: 

• Many support the Open Semantic Interchange, and some brand their context layers, such as the Atlassian Teamwork Graph, Boomi Knowledge Hub, and the SAP Knowledge Graph.
• Some are branding their guardrails, such as Adobe’s AI Brand Intelligence, Appian’s Private AI, and Quickbase AI Control Center.
• To manage AI agents at scale, some are extending the notion of data catalogs and other governance tools to the AI domain with products such as Boomi Connect, SAP AI Agent Hub, and Snowflake Horizon Catalog.

CIOs should recognize that while solution providers will compete on capabilities, the real “secret sauce” of the context layer lies in the company’s trusted data, well-defined business processes, and employee adoption of AI agents.

Conversational user experiences and coworkers

AI agents use the context layer, but also tap into skills, which encode the procedures they can follow, and tools, which prescribe the actions they can take. Before AI agents are ready to pilot, their governance, including permissions, approval gates, and other guardrails, must be defined. Other capabilities to look for when defining AI agents include orchestration, testing evals, and observability.

In 2025, many solution providers bolted on AI agents to their existing user experiences. This year, many solution providers showcased new conversational user experiences that employees can use instead of traditional ones built with forms, flows, reports, and static dashboards. Conversational user experiences are where AI agents and people come together, whether it’s human-in-the-middle or human augmentation.

Solution providers also grouped their AI agents into assistants or coworkers. For example, Adobe CX Coworker illustrates human augmentation, helping marketers manage campaigns with prompts and monitor their performance. SAP launched Joule Assistants across several business functions, including finance, human capital, supply chain, and customer experience. Other assistants, such as Appian AI Copilot, Atlassian Rovo, Cisco AI Assistant, Nutanix NIVA, and Snowflake CoWork, offer AI-first user experiences to assist different end-user types.

CIOs should demo these AI-first user experiences to glimpse the future of work.

Developers are already getting used to these experiences through code generators and vibe coding tools. Now, similar capabilities are being tailored across all business functions. CIOs should ramp up their change management programs to accelerate the adoption of these AI capabilities.

Solution providers are showcasing AI capabilities that can help CIOs reshape their businesses. But in Q2, there were only a few examples of how AI can help CIOs drive growth, evolve business models, or embed AI into customer-facing products. I expect to see a wave of further AI innovations that will go beyond productivity improvements and efficiencies and help CIOs pursue growth-driving digital transformation strategies.  

Sacolick travelled to conferences mentioned in this article as a guest of Adobe, Appian, Atlassian, Domo, Nutanix, SAP, and Snowflake. In addition, he was hired by Quickbase to speak at its conference.

Since the advent of generative AI, the use of AI in business has shifted from something we should do to something we must do to survive. Many companies are now working to utilize AI with the aim of improving productivity and creating value.

Here, I would like to pose a question to you all once again: “What is the fundamental factor that determines AI performance?”

Is it the AI model? Is it the AI tool? Or is it the AI agent?

Of course, I believe all of these are important. However, if we look at the long-term perspective, the competition among multiple companies to improve AI model performance will eventually level off, and we will eventually reach a point where every AI model is amazing!

In that context, what I believe is the most important factor influencing AI performance is the data accumulated by companies that connects to their unique strengths.

For example, if asked, “What do plants need to grow?” I would say “good water and light.”

Similarly, if asked, “What do people need to thrive?” I would say, “Kind words.”

Finally, “What does AI need to thrive?” The answer is “good data.”

I believe that the extent to which companies can genuinely understand the importance of this extremely simple principle and implement it with unwavering dedication will determine their ability to establish a competitive advantage and achieve sustainable growth.

AI is a mirror of data

As I’m sure you’re all aware, AI is by no means a magic wand. It is an entity that learns based on the data it is given and makes inferences within that scope. In other words, AI’s output depends heavily on the quality of its input data; one could say that AI is a mirror of data.

• If you feed it inaccurate data, it will return inaccurate results (i.e., garbage in, garbage out)
• If you feed it biased data, it will make biased judgments
• Insufficient data yields only shallow insights and suggestions

In this way, AI is not smart but rather faithful to the data. Based on this premise, it becomes clear that the essence of AI utilization lies not in which tools to use, but in what kind of high-quality data to prepare and how to utilize it.

What is good data?

So, what exactly is good data?

It goes without saying that data is useless if it is merely abundant in quantity, but on the other hand, what specific qualities must good data possess?

Generally speaking, good data possesses at least the following elements.

• Accuracy: Data containing many errors or noise will skew conclusions, no matter how advanced the analysis. It is important to minimize sensor errors, input mistakes and duplicates.
• Completeness: Are any required fields missing, and are there too many missing values? For example, if customer data is missing information such as age, region or gender, it becomes difficult to perform meaningful analysis.
• Consistency: Is data with the same meaning mixed in different formats (e.g., date formats, units, variations in notation)? This is particularly important for system integration and long-term data.
• Timeliness: No matter how accurate it is, data that is too old may not be useful for decision-making. Whether real-time data is required or historical data is sufficient depends on the use case, but it is important that the data has the appropriate freshness for the purpose.
• Relevance: If there is a large amount of data unrelated to the analysis objective, it becomes noise and leads to incorrect judgments. It is necessary to clearly define what the data is used for and ensure the data is appropriate for that purpose.
• Reliability: The data’s source and collection method must be clear, ensuring reliability and reproducibility. Data with an unknown source or that is a black box cannot be verified later.

In summary, good data is data that is accurate, has few gaps, is consistent in meaning and notation, is collected at the appropriate time, is suitable for the purpose and comes from a reliable source.

Only when the quality of this good data is guaranteed can AI produce valuable outputs. Conversely, introducing AI with unorganized data will not yield the expected results. Many complaints, such as “We implemented AI but it’s unusable” or “The AI’s accuracy isn’t improving stem from data issues.”

Data does not organize itself naturally

The key point here is that good data does not arise naturally. On the contrary, if left unattended, data will inevitably deteriorate.

• Rules become inconsistent depending on who entered the data and when
• Multiple instances of data with the same meaning exist
• Outdated data is scattered and left unattended
• Data becomes siloed by department

These conditions are likely common in many companies.

Below is an overview of our company’s data management framework.

Akio Ueda

Broadly speaking, it consists of data governance — covering roles and structures, risk management and evaluation — and data management, which encompasses data utilization cycle management and data utilization support services. Within this framework, data utilization cycle management involves:

• Needs management: We clarify the purpose and needs by asking, “What is the data being used for?” and “For whom, and in what way, does this data create value?”
• Collection: We gather the necessary data based on the defined objectives. We design the process to determine what data is required (internal/external), the level of detail and frequency of collection, and how to ensure data quality.
• Processing: We enhance the quality and prepare the data for use. This includes cleansing (correcting errors and missing values), standardizing formats, deduplicating and integrating data, processing structured and unstructured data separately, and assigning business and operational meaning to the data.
• Storage: We ensure the data is available to the right people at the right time. This involves storing data in databases or data lakes, implementing security and access controls, and managing metadata (ensuring the data is clearly identifiable).
• Utilization: This is the most critical step. The purpose of data is not merely analysis but driving action. We generate value from the data through visualization (dashboards), analysis (statistical processing, BI, AutoML, AI) and integration into business operations (automation and decision support).
• Disposal: We properly dispose of data that is no longer needed. Simply holding data can itself pose risks, such as managing retention periods, complying with laws and governance requirements, and mitigating security risks. That is why the principle of not holding data that is not used is so important.

Data management is not a one-time effort; it is an ongoing initiative that requires continuous maintenance and improvement.

The CIO must embed data management as a system within the organization and continue to implement it until it becomes firmly established.

Data management is not just the IT department’s job

Another important point is that data management is not just the IT department’s job.

Data is fundamentally generated within day-to-day operations on the front lines. Therefore:

• Who determines the meaning and definition of data
• How should input rules be standardized?
• How do we ensure data quality?

are, in essence, operational issues, business issues and management issues.

The latest Digital Skills Standard ver. 2.0, published by the Ministry of Economy, Trade and Industry in April 2026, defines the following three roles within the data management category:

• Data steward: Based on business domain knowledge, this role is responsible for operations aimed at ensuring data quality, reliability and security, as well as for promoting the adoption and establishment of data management within business divisions and frontline organizations, and for fostering data utilization. In short, they are the data quality manager and data utilization promoter.
• Data engineer: This role involves understanding the current state of data and supporting the organization’s continuous data utilization through data preparation and preprocessing in processes such as collection, integration, processing and provision, as well as the design and implementation of data pipelines. In essence, they are the implementers and operators who drive data.
• Data architect: This role involves taking a bird’s-eye view of the data structure, flow and utilization methods across the entire organization and business. By designing and continuously reviewing data architecture that encompasses the entire data lifecycle in alignment with business strategy, they ensure the successful integration of company-wide data utilization and governance—essentially serving as the overall designer of data.

The CIO is not merely responsible for establishing data storage and analysis infrastructure; they are also tasked with appropriately assigning personnel to these three roles within the company and establishing cross-departmental, company-wide tools and rules to connect data with management, business operations and daily tasks.

Ultimately, the success of data utilization depends on organizational culture

On the other hand, no matter how much progress is made in staffing, infrastructure, tools and rulemaking, data will not be utilized unless there is an organizational culture that actively drives management, business and operations based on data.

• The purpose of data entry is not understood
• Data is optimized solely for the department’s own operations
• Decision-making based on data is not valued

In such a situation, no matter how well the systems are set up, they will become mere formalities.

In contrast, in organizations where data utilization is advanced:

• Discussions are based on data
• Formulate hypotheses and verify them with data
• And continuously improve based on data

These actions occur naturally.

In other words, the essence of data management ultimately lies in creating an organizational culture that assumes the effective use of data.

Data management cannot be achieved overnight. That is precisely why it is important to start small and build on your successes.

• Organize data for specific tasks and achieve results through the use of AI
• Rolling out successful practices
• Gradually Expand the Scope

By repeating this cycle, the importance of data will permeate the entire organization.

The role expected of a CIO in the AI era

In the AI era, the role expected of a CIO has changed significantly.

Traditionally:

• Ensuring the stable operation of systems
• And optimizing costs

However, moving forward:

• We will view data as an asset and maximize its value
• Developing the data infrastructure, tools and rules that underpin AI adoption, and advancing personnel allocation and development
• And fostering an organizational culture that embraces data utilization —roles that are more directly linked to business management

In other words, the CIO must evolve into the person responsible for creating value from data.

Data is the source of competitive advantage

In the coming era, the use of AI will be a given. What will set companies apart is not whether they use AI, but what data they possess.

Data is the accumulation of a company’s past strengths and the source of future value creation. And its quality is determined by daily operations and the nature of the organization.

• AI grows by being fed good data
• And companies grow through that AI

Taking this simple principle as our starting point, we must place data management at the core of our business strategy. Isn’t that the shortest route to sustainable growth in the AI era?

CIOs are called upon to lead the way in making this a reality.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Generative AI is a kind of artificial intelligence that creates new content, including text, images, audio, and video, based on patterns it has learned from existing data.

Today’s generative models are typically built on foundation-model architectures such as large-language models (LLMs) and multimodal systems, enabling them to carry on conversations, answer questions, write stories, generate code, and produce images or videos from brief prompts.

Generative AI is different from discriminative AI, which draws distinctions between different kinds of input. Where discriminative AI answers questions like “Is this image of a rabbit or a lion?”, generative AI instead responds to prompts such as “Describe to me how a rabbit and lion look different from one another” or “Draw me a picture of a lion and a rabbit sitting next to each other” — and in both cases produces text or imagery that, while grounded in the AI’s training data, isn’t just a copy of something that already existed.

Just a few years ago, generative AI was once a novelty focused on chatbots and artistic image generation. Today, it has become a core enterprise technology, and powers everything from content creation and software development to customer support and analytics workflows. But with that power comes a new set of challenges — from model alignment and hallucination to governance and data-integration hurdles.

In this article, we’ll look at how generative AI works, explore how it has evolved into the foundation-model era, examine how to implement it effectively, and offer best practices for getting value out of it, today and in the future.

How does generative AI work?

For decades, early artificial-intelligence efforts often focused on rule-based systems or narrowly trained models that were built for one task at a time. While these efforts produced useful systems that could reason and solve human tasks, they were generally a far cry from sci-fi visions of thinking machines. Programs that could talk to people never seemed to get very far past the level of ELIZA, a “computer therapist” created at MIT in the mid 1960s; even Siri and Alexa after much fanfare were revealed to be fairly limited.

The big structural shift that gave birth to modern generative AI came with the concept of a transformer, first introduced in “Attention Is All You Need,” a 2017 paper from Google researchers.

Using a transformer architecture as a basis, you can build a system that derives meaning from analyzing long sequences of input tokens (words, sub-words, bytes) to understand how different tokens might be related to one another, then determines how likely any given token is to come next in a sequence, given the others. In AI lingo, we call these systems models. Because a model analyzes very large datasets and parameter counts, it can pick up on statistical patterns and knowledge implicitly embedded in the data.

This is all easier said than done. The process of adjusting a model’s internal parameters so it gets better at predicting the next token in sequences is called training. During training, the model repeatedly guesses the next token in a given sequence, compares its prediction to the actual one, measures the error, and updates its parameters to reduce that error across billions of examples. Over time, that process teaches the model the statistical relationships that will allow it to generate coherent language (or code, or images) later.

What is a foundation model?

You’ll often hear the word large used for transformer-based models of these types, like the LLMs we mentioned earlier. Large in this context refers to the large number of internal numerical values that the model adjusts during training to represent what it has learned, along with breadth and diversity of data used to train the model and the underlying compute resources powering this whole process.

This is in contrast with the narrow models of the earlier era of AI/ML, which werebuilt for one purpose and trained on a limited dataset. For instance, a spam filter may be very good at what it does, but it’s only trained on email data and all it can do is classify emails. Large models, by contrast, serve as what’s known as foundation models. They’re trained broadly on diverse data (text, code, images, or multimodal data) and then adapted or specialized for many downstream tasks.

These foundation models are the basis for most of the popular generative AI tools and services on the market today. They can be specialized in several ways:

• Fine-tuning: Giving a foundation model further training on a smaller, task-specific dataset
• Retrieval-augmented generation (RAG): Giving the model the ability to pull in external knowledge when asked a question
•  Prompt engineering: Tailoring a query so the model gives the sort of answers you’re looking for.

How do AI systems write computer code?

One of the surprising discoveries of the gen AI era was that in recent years was that foundation models trained on natural-language text can also, when fine-tuned with code examples, also write computer code — often better than many purpose-built systems. Still, it makes sense, when you think about it — after all, high-level computer languages are designed by humans and ultimately based on human language.

This 2023 InfoWorld article highlights how models like PaLM, LLaMA and other transformer-based systems fine-tuned on code repositories propelled this shift, but since AI giants like OpenAI have moved into this space. This all matters because code generation (or code-assisted productivity) has become a key enterprise use case of generative AI — perhaps the key use, given the industry’s enthusiastic adoption of it.

What are AI agents?

So far, we’ve been talking about chatbots, writing assistants, image-generation tools. They respond to prompts, output text or images, and then stop. A new category of tool called agentic AI goes further: it plans, executes, and in many cases learns as it works.

Because large models already understand language, code, and even structured data to some extent, they can be repurposed to generate not only descriptive text but operational instructions. For example: an agent might parse the intent “generate a sales-report”, then format internal calls like getData(salesDB, region=NA, period=lastQuarter), and then call an API, all by generating text that’s interpreted as instructions. The MCP framework standardizes the “language” of those instructions and the plug-points into tools and data so that the model doesn’t need bespoke integrations for each new workflow.

These kinds of autonomous agents have several enterprise use cases:

• Software automation: Agents that generate code, call unit tests, deploy builds, monitor logs and even roll back changes autonomously.
• Customer support: Instead of simply drafting responses, agents interact with CRM APIs, update ticket statuses, escalate issues, and trigger follow-up workflows.
• IT operations/AIOps: Agents monitor infrastructure, identify anomalies, open/close tickets, or auto-remediate based on defined rules and context from logs.
• Security: Agents may detect threats, initiate alerts, isolate compromised systems, or even attempt to manage threat containment — though this raises new risks.

How can you implement generative AI in the enterprise?

We’ve now touched on what generative AI can do. But how can you make it work reliably in your business. The difference between a pilot and full-scale deployment often comes down to systems, structure and governance as much as to models themselves. InfoWorld’s Matt Asay offers a deep dive into enterprise gen AI essentials, but here are some important points to keep in mind:

Choosing between API, open-source or custom fine-tuned models. One of the first major decisions for any enterprise project is: do you use a model via an API (e.g., from a vendor like OpenAI or Anthropic), deploy an open-source model internally, or build/fine-tune a custom model yourself? Each has trade-offs.

APIs offer speed and minimal setup, but may expose data, limit customization or accrue high cost — and will leave you at the mercy of your vendor. Open source allows internal control and may ease fine-tuning, but requires infrastructure, expertise, and support. Custom fine-tuning gives you the tightest alignment to your use-case, but lengthens time to value and increases risk.

Governance, data privacy and compliance. Deploying generative AI in an enterprise setting raises new governance, privacy and regulatory issues. For example: Who owns the data that’s ingested? How is proprietary data protected if you call a third-party API? What traceability exists for model outputs—a huge question for regulated industries? One useful framework is covered in “A GRC framework for securing generative AI” Data governance must adapt for the new era,  and new frameworks are evolving to help.

Human-in-the-loop review. Even the best models make mistakes and cannot simply be put on autopilot. You need a human-in-the-loop (HITL) process: real people need to review outputs, validate for bias, approve high-stakes content, and tune prompts or models based on feedback. Incorporating HITL checkpoints helps mitigate risk and improve overall quality.

Integration with existing systems and RAG pipelines. Retrieval-augmented generation, which we touched on earlier, connects foundation models into business workflows, systems, and enterprise data stores. RAG can bind LLMs to your organization’s internal knowledge bases, thereby reducing hallucinations (which we’ll discuss in a moment) and increasing the relevance of gen AI output.

What causes AI hallucinations?

Probably the biggest limitation of generative AI is what those in the industry call hallucinations, which is a perhaps misleading term for output that is, by the standards of humans who use it, false or incorrect.  

Every generative AI system, no matter how advanced, is built around prediction. Remember, a model doesn’t truly know facts—it looks at a series of tokens, then calculates, based on analysis of its underlying training data, what token is most likely to come next. This is what makes the output fluent and human-like, but if its prediction is wrong, that will be perceived as a hallucination.

Foundry

Because the model doesn’t distinguish between something that’s known to be true and something likely to follow on from the input text it’s been given, hallucinations are a direct side effect of the statistical process that powers generative AI. And don’t forget that we’re often pushing AI models to come up with answers to questions that we, who also have access to that data, can’t answer ourselves.

In text models, hallucinations might mean inventing quotes, fabricating references, or misrepresenting a technical process. In code or data analysis, it can produce syntactically correct but logically wrong results. Even RAG pipelines, which provide real data context to models, only reduce hallucination—they don’t eliminate it. Enterprises using generative AI need review layers, validation pipelines, and human oversight to prevent these failures from spreading into production systems.

What are some other problems with generative AI?

Generative AI has proven to be such a disruptive technology that’s stoking near-apocalyptic fears that it will result in a superintelligence that will enslave or destroy humanity. Meanwhile, in the present day, increasingly troubling reports of so-called AI psychosis are emerging, where people have mental health episodes triggered by the uncanny and sometimes sycophantic ways chatbots affirm whatever you talk to them about and try to keep the conversation going.

Compared to such existential questions, the following business-related problems may seem petty. But they’re real issues for enterprises considering investing in AI tools.

• Data leakage and regulatory risk. When a model is fine-tuned or prompted with sensitive information, that data may be memorized and unintentionally reproduced. Using third-party APIs without strict controls can expose proprietary or personally identifiable information (PII). Regulatory frameworks like GDPR and HIPAA require explicit governance around where training data resides and how inference results are stored.
• Prompt injection occurs when an attacker manipulates a model’s instructions—embedding hidden directives or malicious payloads in user input or external content the model reads. This can override safety rules, expose internal data, or execute unintended actions in agentic systems. Guardrails that sanitize inputs, restrict tool-calling permissions, and validate outputs are becoming essential.
• Copyright and content ownership. Many foundation models are trained on data scraped from the public internet, creating disputes over copyright and data provenance. Enterprises using generated output commercially need to confirm usage rights and review indemnity terms from vendors.
• Unrealistic productivity expectations. Finally, organizations sometimes expect generative AI to deliver instant productivity gains. The reality, it turns out, is more mixed. Enterprise adoption requires infrastructure, governance, retraining, and cultural change. The models accelerate work once properly integrated, but they don’t automatically replace human judgment or oversight.

The current generation of enterprise AI systems includes several layers of defense against these risks:

• Guardrails that constrain model behavior and filter unsafe outputs.
• Model validation frameworks that measure factual accuracy and consistency before deployment.
• Policy layers that enforce compliance rules, redact sensitive data, and log model actions.

These safeguards reduce—but don’t remove—the inherent uncertainty that defines generative AI.

GenAI: essential for the enterprise

Generative AI has evolved from a novelty into a core layer of enterprise technology. Foundation models and agentic systems now power automation, analytics, and creative workflows — but they remain fundamentally probabilistic tools. Their strength lies in scale and adaptability, not perfect understanding.

For organizations, success depends less on chasing model breakthroughs than on integrating these systems responsibly: building guardrails, maintaining oversight, and aligning them with real business needs. Used wisely, generative AI can amplify human capability rather than replace it.

A portmanteau of “development” and “operations,” devops emerged as a way of bringing together two previously separate groups responsible for the building and deploying of software.

In the old world, developers (devs) typically wrote code before throwing it over to the system administrators (operations, or ops) to deploy and integrate that code. But as the industry shifted towards agile development and cloud-native computing, many organizations reoriented around modern, cloud-native practices in the pursuit of faster, better releases.

This required a new way to perform these key functions in a more streamlined, efficient, and cohesive way, one where the old frustrations of disconnected dev and ops functions would be eliminated. With two groups working together, developers can rapidly roll out small code enhancements via continuous integration and delivery rather than spending years on “big bang” product releases.

Devops was born at cloud-native companies like Facebook, Netflix, Spotify, and Amazon; but it’s become one of the defining technology industry trends of the past decade, primarily because it bridges so many of the changes that have shaped modern software development.

As agile development and cloud-native computing have become ubiquitous, devops has enabled the entire industry to speed up its software development cycles. Thus, devops has now thoroughly infiltrated the enterprise, especially in organizations that rely on software to run their business, such as banks, airlines, and retailers. And it’s spawned a host of other “ops” practices, some of which we’ll touch on here.[JF1] 

Devops practices

Devops requires a shift in mindset from both sides of the dev and ops divide. Development teams should focus on learning and adopting agile processes, standardizing platforms, and helping drive operational efficiencies. Operations teams must now focus on improving stability and velocity, while also reducing costs by working hand in hand with the developer team.

Broadly speaking, these teams need to all speak a common language and there needs to be a shared goal and understanding of each other’s key skills for devops to thrive.

More specifically, engineers Damon Edwards and John Willis created the CALMS model to bring together what are commonly understood to be the key principles of devops:

• Culture: One that embraces agile methodologies and is open to change, constant improvement, and accountability for the end-to-end quality of software.
• Automation: Automating away toil is a key goal for any devops team.
• Lean: Ensuring the smooth flow of software through key steps as quickly as possible.
• Measurement: You can’t improve what you don’t measure. Devops pushes for a culture of constant measurement and feedback that can be used to improve and pivot as required, on the fly.
• Sharing: Knowledge sharing across an organization is a key tenet of devops.

“Who could go back to the old way of trying to figure out how to get your laptop environment looking the same as the production environment? All these things make it so clear that there’s a better way to work. I think it’s very tough to turn back once you’ve done things like continuous integration, like continuous delivery. Once you’ve experienced it, it’s really tough to go back to the old way of doing things,” Kim told InfoWorld.

What is a devops engineer?

Naturally, the emergence of devops has spawned a whole new set of job titles, most prominent of which is the catch-all devops engineer.

Generally speaking, this role is the natural evolution of the system administrator — but in a world where developers and ops work in close tandem to deliver better software. This person should have a blend of programming and system administrator skills so that he or she can effectively bridge those two sides of the team.

That bridging of the two sides requires strong social skills more than technical. As Kim put it, “one of the most important skills, abilities, traits needed in these pioneering rebellions — using devops to overthrow the ancient powerful order, who are very happy to do things the way they have for 30 to 40 years — are the cross-functional skills to be able to reach across the table to their business counterparts and help solve problems.”

This person, or team of people, will also have to be a born optimizer, tasked with continually improving the speed and quality of software delivery from the team, be that through better practices, removing bottlenecks, or applying automation to smooth out software delivery.

The good news is that these skills are valuable to the enterprise. Salaries for this set of job titles have risen steadily over the years, with 95% of devops practitioners making more than $75,000 a year in salary in 2020 in the United States. In Europe and the UK, where salaries are lower across the board, 71% made more than $50,000 a year in 2020, up from 67% in 2019.

Key devops tools

While devops is at its heart a cultural shift, a set of tools has emerged to help organizations adopt devops practices.

This stack typically includes infrastructure as code, configuration management, collaboration, version control, continuous integration and delivery (CI/CD), deployment automation, testing, and monitoring tools.

Here are some of the tools/categories that are increasingly relevant in 2025, and what is changing:

• CI/CD and delivery automation: Traditional tools like Jenkins remain in many stacks, but newer orchestration tools and CLI-driven or GitOps-centric platforms are growing in importance (e.g. ArgoCD, Flux, Tekton). Also, platforms that integrate more tightly with monitoring, secrets management, drift detection, and policy enforcement are gaining traction.
• Security, compliance, and devsecops tooling: Security tools are increasingly integrated into devops pipelines. Expect to see more use of static analysis (SAST), dynamic testing (DAST), dependency and supply chain scanning (SCA), secret management, and policy as code. The push is toward embedding security earlier and bridging gaps between dev, security, and machine learning teams. (InfoWorld:)
• AI  and automation augmentation: AI-assisted tools are increasingly part of tooling stacks: auto-suggestions in CI/CD, anomaly detection, predictive scaling, intelligent test suite selection, and more. The hope is that these tools will reduce manual interventions and improve reliability. Tools that are “AI ready”—that is, they integrate well with AI or have mature built-in automation or assistance—increasingly stand out from the pack.

Devops challenges

Even as devops becomes more widely adopted, there remain real obstacles that can slow progress or limit impact. One major challenge is the persistent skills gap. The modern devops engineer (or team) is expected to master not just source control, CI/CD, and scripting, but also cloud architecture, infrastructure as code, security best practices, observability, and strong cross-team communication. In many organizations these capabilities are uneven: some teams excel, others lag behind. A 2024 survey showed that while 83% of developers report participating in devops activities, using multiple CI/CD tools was correlated with worse performance — a sign that complexity without deep expertise can backfire.

Toolchain fragmentation and complexity is a related issue. Devops toolchains have sprouted into a sometimes bewildering array of packages and techniques to master: version control, CI build/test, security scanning, artifact management, monitoring, observability, deployment, secret management, and more.

The more tools you have, the more difficult it becomes to integrate them cleanly, manage their versions, ensure compatibility, and avoid duplicated effort. Organizations often get stuck with “tool sprawl” — tools chosen by different teams, legacy systems, or overlapping functionalities — which introduce friction, maintenance burden, and sometimes vulnerabilities.

Finally, although devops has spread far and wide, there is still cultural resistance and alignment. Devops isn’t just about tools and processes; it’s about collaboration, shared responsibility, and continuous feedback. Teams rooted in traditional silos (dev vs ops, or security separate) may resist changes to roles and workflows. Leadership support, communication of shared goals, trust, and allowance for continuous learning are all necessary.

Many CIOs focus too much on tools or implementation first, rather than organizational culture and behaviors; but without addressing culture, even the best tools or processes may not yield the hoped-for velocity, quality, or reliability. Organizations that succeed here tend to have proactive strategies: dedicated training programs, mentorship, internal “guilds,” pairing junior and senior engineers, and making sure leadership supports ongoing learning rather than one-off bootcamps.

Why do devops?

Whoever you ask will tell you that devops is a major culture shift for organizations, so why go through that pain at all?

Devops aims to combine the formerly conflicting aims of developers and system administrators. Under its principles, all software development aims to meet business demands, add functionality, and improve the usability of applications while also ensuring those applications are stable, secure, and reliable. Done right, this improves the velocity and quality of your output, while also improving the lives of those working on these outcomes.

Does devops save money — or add cost?

Devops teams are recognizing that speed and agility are only part of success — unchecked cloud bills and waste undermine long-term sustainability. Waste in devops often comes in the form of “devops debt”— idle cloud capacity, dead code, or false-positive security alerts—which was called a “hidden tax on innovation” in recent Java-environment studies.

 Embedding finops practices can help fight these costs. Teams should shift left on cost: estimating costs when spinning up new environments, resizing instances, and scaling down unused resources before they become runaway expenses.

How to start with devops

There are lots of resources for help getting started with devops, including Kim’s own Devops Handbook, or you can enlist the help of external consultants. But you have to be methodical and focus on your people more than on the tools and technology you will eventually use if you want to ensure lasting buy-in across the business.

A proven route to achieving this is a “land and expand” strategy, where a small group starts by mapping key value streams and identifying a single product team or workload for trialing devops practices. If this team is successful in proving the value of the shift, you will likely start to get interest from other teams and from senior leadership.

If you are at the start of your devops journey, however, make sure you are prepared for the disruption a change like this can have on your organization, and keep your eye on the prize of building better, faster, stronger software.

More on devops:

• Devops debt: The hidden tax on innovation
• 10 big devops mistakes and how to avoid them
• Smarter devops: How to avoid deployment horrors

What is cloud native? Cloud native defined

The term “cloud-native computing” encompasses the modern approach to building and running software applications that exploit the flexibility, scalability, and resilience of cloud computing. The phrase is a catch-all that encompasses not just the specific architecture choices and environments used to build applications for the public cloud, but also the software engineering techniques and philosophies used by cloud developers.

The Cloud Native Computing Foundation (CNCF) is an open source organization that hosts many important cloud-related projects and helps set the tone for the world of cloud development. The CNCF offers its own definition of cloud native:

Cloud native practices empower organizations to develop, build, and deploy workloads in computing environments (public, private, hybrid cloud) to meet their organizational needs at scale in a programmatic and repeatable manner. It is characterized by loosely coupled systems that interoperate in a manner that is secure, resilient, manageable, sustainable, and observable.

Cloud native technologies and architectures typically consist of some combination of containers, service meshes, multi-tenancy, microservices, immutable infrastructure, serverless, and declarative APIs — this list is not exhaustive.

This definition is a good start, but as cloud infrastructure becomes ubiquitous, the cloud native world is beginning to spread behind the core of this definition. We’ll explore that evolution as well, and look into the near future of cloud-native computing.

Cloud native architectural principles

Let’s start by exploring the pillars of cloud-native architecture. Many of these technologies and techniques were considered innovative and even revolutionary when they hit the market over the past few decades, but now have become widely accepted across the software development landscape.

Microservices. One of the huge cultural shifts that made cloud-native computing possible was the move from huge, monolithic applications to microservices: small, loosely coupled, and independently deployable components that work together to form a cloud-native application. These microservices can be scaled across cloud environments, though (as we’ll see in a moment) this makes systems more complex.

Containers and orchestration. In could-native architectures, individual microservices are executed inside containers — lightweight, portable virtual execution environments that can run on a variety of servers and cloud platforms. Containers insulate the developers from having to worry about the underlying machines on which their code will execute. That is, all they have to do is write to the container environment. 

Getting the containers to run properly and communicate with one another is where the complexity of cloud native computing starts to emerge. Initially, containers were created and managed by relatively simple platforms, the most common of which was Docker. But as cloud-native applications got more complex, container orchestration platforms that augmented Docker’s functionality emerged, such as Kubernetes, which allows you to deploy and manage multi-container applications at scale. Kubernetes is critical to cloud native computing as we know it — it’s worth noting that the CNCF was set up as a spinoff of the Linux Foundation on the same day that Kubernetes 1.0 was announced — and adhering to Kubernetes best practices is an important key to cloud native success. 

Open standards and APIs. The fact that containers and cloud platforms are largely defined by open standards and open source technologies is the secret sauce that makes all this modularity and orchestration possible, and standardized and documented APIs offer the means of communication between distributed components of a larger application. In theory, anyway, this standardization means that every component should be able to communicate with other components of an application without knowing about their inner workings, or about the inner workings of the various platform layers on which everything operates.

DevOps, agile methodologies, and infrastructure as code. Because cloud-native applications exist as a series of small, discrete units of functionality, cloud-native teams can build and update them using agile philosophies like DevOps, which promotes rapid, iterative CI/CD development. This enables teams to deliver business value more quickly and more reliably.

The virtualized nature of cloud environments also make them great candidates for infrastructure as code (IaC), a practice in which teams use tools like Terraform, Pulumi, and AWS CloudFormation, to manage infrastructure declaratively and version those declarations just like application code. IaC boosts automation, repeatability, and resilience across environments—all big advantages in the cloud world. IaC also goes hand-in-hand with the concept of immutable infrastructure—the idea that, once deployed, infastructure-level entities like virtual machines, containers, or network appliances don’t change, which makes them easier to manage and secure. IaC stores declarative configuration code in version control, which creates an audit log of any changes.

Foundry

How the cloud-native stack is expanding

As cloud-native development becomes the norm, the cloud-native ecosystem is expanding; the CNCF maintains a graphical representation of what it calls the  cloud native landscape that hammers home to expansive and bewildering variety of products, services, and open source projects that contribute to (and seek to profit from) to cloud-native computing. And there are a number of areas where new and developing tools are complicating the picture sketched out by the pillars we discussed above.   

An expanding Kubernetes ecosystem. Kubernetes is complex, and teams now rely on an entire ecosystem of projects to get the most out of it: Helm for packaging, ArgoCD for GitOps-style deployments, and Kustomize for configuration management. And just as Kubernetes augmented Docker for enterprise-scale deployments. Kubernetes itself has been augmented and expanded by service mesh offerings like Istio and Linkerd, which offer fine-grained traffic control and improved security

Observability needs. The complex and distributed world of cloud-native computing requires in-depth observability to ensure that developers and admins have a handle on what’s happening with their applications. Cloud-native observability uses distributed tracing and aggregated logs to provide deep insight into performance and reliability. Tools like Prometheus, Grafana, Jaeger, and OpenTelemetry support comprehensive, real-time observability across the stack.

Serverless computing.  Serverless computing, particularly in its function-as-a-service guise, offers to strip needed compute resources down to their bare minimum, with functions running on service provider clouds using exactly as much as they need and no more. Because these services can be exposed as endpoints via APIs, they are increasingly integrated into distributed applications, operating side-by-side with functionality provided by containerized microservices. Watch out, though: the big FaaS providers (Amazon, Microsoft, and Google) would love to lock you in to their ecosystems.  

FinOps. Cloud computing was initially billed as a way to cut costs — no need to pay for an in-house data center that you barely use — but in practice it replaces capex with opex, and sometimes you can run up truly shocking cloud service bills if you aren’t careful. Serverless computing is one way to cut down on those costs, but financial operations, or FinOps, is a more systematic discipline that aims to aligns engineering, finance, and product to optimize cloud spending. FinOps best practices make use of those observability tools to best determine what departments and applications are eating up resources.

How cloud-native architecture is adapting to AI workloads

Enterprises deploy larger AI models and make use of more and more real-time inference services. That’s putting demands on cloud-native systems and forcing them to adapt to remain scalable and reliable.

For instance, organizations are re-engineering cloud environments around GPU-accelerated clusters, low-latency networking, and predictable orchestration. These needs align with established cloud-native patterns: containers package AI services consistently, while Kubernetes provides resilient scheduling and horizontal scale for inference workloads that can spike without warning.

Kubernetes itself is changing to better support AI inference, adding hardware-aware scheduling for GPUs, model-specific autoscaling behavior, and deeper observability into inference pipelines. These enhancements make Kubernetes a more natural platform for serving generative AI workloads.

AI’s resource demands are amplifying traditional cloud-native challenges. Observability becomes more complex as inference paths span GPUs, CPUs, vector databases, and distributed storage. FinOps teams contend with cost volatility from training and inference bursts. And security teams must track new risks around model provenance, data access, and supply-chain integrity.

Application frameworks for building distributed cloud-native apps

Microsoft’s Aspire is one of the most visible examples of a shift towards application frameworks to simplify how teams build distributed systems. Opinionated frameworks like Aspire provide structure, observability, and integration out of the box so developer don’t need to stitch together containers, microservices, and orchestration tooling by hand.

Aspire in particular is a prescriptive framework for cloud-native applications, bundling containerized services, environment configuration, health checks, and observability into a unified development model. Aspire provides defaults for service-to-service communication, configuration, and deployment, along with a built-in dashboard for visibility across distributed components.

While Aspire was originally aligned with Microsoft’s .NET platform,Redmond now sees it as having a  polyglot future. This positions Aspire as part of a broader trend: frameworks that help teams build cloud-native, service-oriented systems without being locked into a single language ecosystem. Several other frameworks are gaining traction: Dapr provides a portable runtime that abstracts many of the plumbing tasks in cloud-native distributed applications, and Orleans offers an actor-model-based framework for large-scale systems in the .NET world, and Akka gives JVM teams a mature, reactive toolkit for elastic, resilient services.

Frameworks and tools in the expanding cloud-native ecosystem

While frameworks like Aspire simplify how developers compose and structure distributed applications, most cloud-native systems still depend on a broader ecosystem of platforms and operational tooling. This deeper layer is where much of the complexity—and innovation—of cloud-native computing lives, particularly as Kubernetes continues to serve as the industry’s control plane for modern infrastructure.

Kubernetes provides the core abstractions for deploying and orchestrating containerized workloads at scale. Managed distributions such as Google Kubernetes Engine (GKE), Amazon EKS, Azure AKS, and Red Hat OpenShift build on these primitives with security, lifecycle automation, and enterprise support. Platform vendors are increasingly automating cluster operations—upgrades, scaling, remediation—to reduce the operational burden on engineering teams.

Surrounding Kubernetes is a rapidly expanding ecosystem of complementary frameworks and tools. Service meshes like Istio and Linkerd provide fine-grained traffic management, policy enforcement, and mTLS-based security across microservices. GitOps platforms such as Argo CD and Flux bring declarative, version-controlled deployments to cloud-native environments. Meanwhile, projects like Crossplane turn Kubernetes into a universal control plane for cloud infrastructure, letting teams provision databases, queues, and storage through familiar Kubernetes APIs. These tools illustrate how cloud-native development now spans multiple layers: developer-focused application frameworks like Aspire at the top, and a powerful, evolving Kubernetes ecosystem underneath that keeps modern distributed applications running.

Advantages and challenges for cloud-native development

Cloud native has become so ubiquitous that its advantages are almost taken for granted at this point, but it’s worth reflecting on the beneficial shift the cloud native paradigm represents. Huge, monolithic codebases that saw updates rolled out once every couple of years have been replaced by microservice-based applications that can be improved continuously. Cloud-based deployments, when managed correctly, make better use of compute resources and allow companies to offer their products as SaaS or PaaS services. 

But cloud-native deployments come with a number of challenges, too:

• Complexity and operational overhead: You’ll have noticed by now that many of the cloud-native tools we’ve discussed, like service meshes and observability tools, are needed to deal with the complexity of cloud-native applications and environments. Individual microservices are deceptively simple, but coordinating them all in a distributed environment is a big lift.
• Security: More services executing on more machines, communicating by open APIs, all adds up to a bigger attack surface for hackers. Containers and APIs each have their own special security needs, and a policy engine can be an important tool for imposing a security baseline on a sprawling cloud-native app. DevSecOps, which adds security to DevOps, has become an important cloud-native development practice to try to close these gaps.
• Vendor lock-in: This may come as a surprise, since cloud-native is based on open standards and open source. But there are differences in how the big cloud and serverless providers works, and once you’ve written code with one provider in mind, it can be hard to migrate elsewhere.
• A persistent skills gap: Cloud-native computing and development may have years under its belt at this point, but the number of developers who are truly skilled in this arena is a smaller portion of the workforce than you’d think. Companies face difficult choices in bridging this skills gap, whether that’s bidding up salaries, working to upskill current workers, or allowing remote work so they can cast a wide net. 

Cloud native in the real world

Cloud native computing is often associated with giants like Netflix, Spotify, Uber, and AirBNB, where many of its technologies were pioneered in the early ’10s. But the CNCF’s Case Studies page provides an in-depth look at how cloud native technologies are helping companies. Examples include the following:

• A UK-based payment technology company that can switch between data centers and clouds with zero downtime
• A software company whose product collects and analyzes data from IoT devices — and can scale up as the number of gadgets grows
• A Czech web service company that managed to improve performance while reducing costs by migrating to the cloud

Cloud-native infrastructure’s capability to quickly scale up to large workloads also make it an attractive platform for developing AI/ML applications: another one of those CNCF case studies looks at how IBM uses Kubernetes to train its Watsonx assistant. The big three providers are putting a lot of effort into pitching their platforms as the place for you to develop your own generative AI tools, with offerings like Azure AI Foundry,Google Firebase Studio, and Amazon Bedrock. It seems clear that cloud native technology is ready for what comes next.

Learn more about related cloud-native technologies:

• Platform-as-a-service (PaaS) explained
• What is cloud computing
• Multicloud explained
• Agile methodology explained
• Agile development best practices
• Devops explained
• Devops best practices
• Microservices explained
• Microservices tutorial
• Docker and Linux containers explained
• Kubernetes tutorial
• CI/CD (continuous integration and continuous delivery) explained
• CI/CD best practices

Over the past decade, software development has been shaped by two closely related transformations. One is the rise of devops and continuous integration and continuous delivery (CI/CD), which brought development and operations teams together around automated, incremental software delivery.

The other is the shift from monolithic applications to distributed, cloud-native systems built from microservices and containers, typically managed by orchestration platforms such as Kubernetes.

While Kubernetes and similar platforms simplify many aspects of running distributed applications, operating these systems at scale is still complicated. Configuration sprawl, environment drift, and the need for rapid, reliable change all introduce operational challenges. GitOps emerged as a way to address those challenges by extending familiar devops and CI/CD techniques beyond application code and into infrastructure and system configuration.

At the heart of GitOps is the concept of infrastructure as code (IaC). In a GitOps model, not only application code but also infrastructure definitions, deployment configurations, and operational settings are described in files stored in a version control system. Automated processes continuously compare the running system with those declarations and work to bring the live environment back into alignment when differences appear.

In this approach, the version control repository serves as the system of record for how applications and their supporting infrastructure should look in production. Changes flow through the same review, approval, and automation pipelines that developers already use for software, bringing greater consistency, traceability, and repeatability to cloud-native operations.

At a high level, GitOps refers to a set of operational practices for managing cloud-native systems using declarative configuration, version control, and automated reconciliation. Rather than treating infrastructure and application configuration as mutable runtime state, GitOps treats them as versioned artifacts that move through the same review, testing, and deployment processes as application code.

GitOps defined

The term GitOps was originally coined and popularized by Weaveworks, which helped formalize the approach in the context of Kubernetes operations. While that early work shaped the way GitOps was discussed and implemented, GitOps has since evolved into a broadly adopted, vendor-neutral pattern. Today, it describes a shared set of ideas rather than a specific product or platform.

The defining characteristic of GitOps is its reliance on declarative configuration stored in a version control system. Instead of issuing imperative commands to change live systems, teams describe the desired state of applications and infrastructure in configuration files. Automated agents then continuously compare that declared state with what is actually running and work to reconcile any differences. This pull-based model—where systems converge toward the desired state defined in version control—provides built-in drift detection, repeatability, and a clear audit trail for every change.

Because GitOps centers on configuration files stored in a version control system, familiar software development practices carry over naturally. Changes are proposed through commits, reviewed before being accepted, and tracked over time. Rollbacks are accomplished by reverting to known-good versions, and the history of how a system evolved is preserved alongside the configuration itself.

While the use of Git as the version control system is not strictly required, it has become the default choice because of its ubiquity in modern devops workflows and its strong support for collaboration and change management, so its place in the name has stuck.

What is the CI/CD process?

A complete look at CI/CD is beyond the scope of this article—see the InfoWorld explainer on the subject—but we need to say a few words about CI/CD because it’s at the core of how GitOps works. The continuous integration half of CI/CD is enabled by version control repositories like Git: Developers can make constant small improvements to their codebase, rather than rolling out huge, monolithic new versions every few months or years. The continuous deployment piece is made possible by automated systems called pipelines that build, test, and deploy the new code to production.

Again, we keep talking about code here, and that usually summons up visions of executable code written in a programming language such as C or Java or JavaScript. But in GitOps, the “code” we’re managing is largely made up of configuration files. This isn’t just a minor detail — it’s at the heart of what GitOps does. These config files are, as we’ve said, the “single source of truth” describing what our system should look like. They are declarative rather than instructive. That means that instead of saying “start up ten servers,” the configuration file will simply say, “this system includes ten servers.”

GitOps and Kubernetes

GitOps first took hold in the Kubernetes ecosystem, where declarative configuration and continuous reconciliation are core design principles. As a result, Kubernetes remains the most common and best-understood environment for applying GitOps practices. A typical GitOps-driven update process for a Kubernetes application looks like this:

1. A developer proposes a change by committing updated application code or configuration to a version control repository, usually through a pull request.
2. That change is reviewed and approved, then merged into the main branch.
3. The merge triggers an automated CI/CD pipeline that tests the change, builds new artifacts if needed, and publishes them to a registry.
4. A GitOps controller or similar automated agent detects the updated desired state stored in version control.
5. The controller compares that desired state with the current state of the Kubernetes cluster and applies the necessary changes to bring the cluster back into alignment.

This pull-based reconciliation loop—where the cluster continuously converges toward the desired state defined in version control—is central to how GitOps works in practice. While Kubernetes provides a natural fit for this model, it represents just one canonical use case. The same patterns increasingly apply to infrastructure provisioning, policy enforcement, and multi-cluster operations beyond Kubernetes itself.

GitOps tooling in practice: Argo CD, Flux, and the ecosystem

GitOps is enabled by a set of tools that embody the principles we’ve outlined, with some open-source projects emerging as de facto standards in cloud-native environments.

At the center of the GitOps ecosystem is Argo CD, an open-source controller that continuously monitors a version control repository and ensures that the state of running systems matches the declared desired state. Argo CD is widely used in Kubernetes environments because it directly implements pull-based reconciliation: it compares the desired state stored in Git with the cluster’s actual state and applies changes to correct any drift.

Alongside Argo CD, Flux is another prominent open source GitOps engine. Both Flux and Argo CD help teams adopt GitOps workflows by managing the synchronization loop between code and runtime, but they differ in operational philosophy, integration surfaces, and ecosystem fit.

GitOps tooling often appears as part of broader platforms or integrated stacks rather than as isolated utilities. For example, multicloud and cluster management solutions now routinely include GitOps support, with Argo CD or compatible controllers bundled alongside deployment, policy, and governance capabilities.

In addition to Flux and Argo CD, a range of auxiliary tools contribute to a complete GitOps ecosystem: policy as code engines (e.g., Open Policy Agent), drift detection systems, and infrastructure provisioning tools that mesh with Git-centric workflows.

GitOps, devops, and normalization

GitOps grew out of the same forces that drove devops into mainstream IT practice, and in its early days, GitOps was often discussed as a distinct extension of devops, specifically tailored to managing declarative infrastructure and Kubernetes-centric systems. At the time, GitOps was still relatively new and not yet widely adopted outside cloud-native pioneers.

Over the last several years, however, GitOps practices have become deeply woven into how teams operate modern cloud environments. Rather than being treated as an optional add-on or marketing term, the core ideas of GitOps — using version-controlled, declarative configuration and automated reconciliation loops to continuously align running systems with intended state — are now part of standard operational practice in many Kubernetes-centric shops. In this sense, GitOps has shifted from a buzzword about what might be possible to a baseline pattern for cloud-native operations, much like devops itself did years earlier.

In environments where Kubernetes and declarative systems are the norm, GitOps workflows are the default way teams manage and deploy change. Many organizations now implement these patterns without explicitly calling them “GitOps,” just as few teams today explicitly say they do “CI/CD” even though continuous pipelines are taken for granted. The term has become less prominent in marketing, but its practices are often embedded in pipelines, controllers, and platform tooling.

That normalization shows up in how GitOps workflows are woven into broader operational frameworks. For example, platform engineering teams frequently build internal developer platforms that encapsulate GitOps patterns behind standardized developer APIs, making the pattern invisible to most application teams while still providing the auditability and automation that GitOps promises.

GitOps beyond Kubernetes: infrastructure, policy, and drift

While GitOps first gained traction as a way to manage Kubernetes deployments, its core principles apply broadly to infrastructure and operational concerns beyond any single orchestration platform. GitOps treats desired state as declarative configuration stored in version control and uses automated reconciliation to ensure running systems align with that state. That pattern naturally extends to infrastructure provisioning, policy enforcement, configuration drift detection, and governance workflows across diverse environments.

In modern operational stacks, infrastructure is increasingly defined declaratively, whether through Kubernetes manifests, Terraform modules, or other infrastructure-as-code formats. Storing these declarations in version control enables the same peer-review, auditability, and rollback practices developers already use for application code. Automated tooling then continuously detects when the live infrastructure diverges from the declared state and works to bring it back into alignment, reducing the risk of configuration drift and inadvertent misconfigurations.

Configuration drift — the state where an environment has diverged from what’s declared in version control — remains a major operational headache, especially in complex, dynamic systems. Drift can arise from ad hoc fixes, emergency updates, or manual changes made outside normal pipelines, and it can lead to inconsistencies, outages, and security gaps. By continually checking running systems against the desired state in Git and reconciling deviations automatically, GitOps workflows help teams keep environments predictable and auditable.

Policy enforcement and compliance are another natural extension of GitOps patterns. As organizations adopt declarative practices, policy-as-code engines and drift detection systems can be woven into GitOps pipelines to validate that proposed configurations meet security, compliance, or operational standards before they’re ever applied to running systems. Embedding policy checks into declarative workflows brings consistency to governance while preserving the automation and speed that devops teams expect.

GitOps – beyond Kubernetes

GitOps began as a way to bring devops discipline to Kubernetes operations, but its longer-term impact has been more subtle. In many ways, it’s been absorbed into the fabric of modern cloud-native operations, where declarative configuration, version control, and automated reconciliation are taken for granted. Today, GitOps is less about a specific set of tools or a named practice and more about an operational mindset. By treating infrastructure and configuration as versioned, auditable artifacts and relying on automation to enforce consistency, GitOps helps teams manage complexity at scale. Even as the term itself fades from the spotlight, the practices it introduced continue to shape how distributed systems are built, deployed, and operated.

For several decades now, the most innovative software has always emerged from the world of open source software. It’s no different with machine learning and large language models. If anything, the open source ecosystem has grown richer and more complex, because now there are open source models to complement the open source code.

For this article, we’ve pulled together some of the most intriguing and useful projects for AI and machine learning. Many of these are foundation projects, nurturing their own niche ecology of open source plugins and extensions. Once you’ve started with the basic project, you can keep adding more parts.

Most of these projects offer demonstration code, so you can start up a running version that already tackles a basic task. Additionally, the companies that build and maintain these projects often sell a service alongside them. In some cases, they’ll deploy the code for you and save you the hassle of keeping it running. In others, they’ll sell custom add-ons and modifications. The code itself is still open, so there’s no vendor lock in. The services simply make it easier to adopt the code by paying someone to help.

Here are 16 open source projects that developers can use to unlock the potential in machine learning and large language models of any size—from small to large, and even extra large.

Agent Skills

AI coding agents are often used to tackle standard tasks like writing React components or reviewing parts of the user interface. If you are writing a coding agent, it makes sense to use vetted solutions that are focused on the task at hand. Agent Skills are pre-coded tools that your AI can deploy as needed. The result is a focused set of vetted operations capable of producing refined, useful code that stays within standard guidelines. License: MIT.

Awesome LLM Apps

If you are looking for good examples of agentic coding, see the Awesome LLM Apps collection. Currently, the project hosts several dozen applications that leverage some combination of RAG databases and LLMs. Some are simple, like a meme generator, while others handle deeper research like the Journalist agent. The most complex examples deploy multi-agent teams to converge upon an answer. Every application comes with working examples for experimentation, so you can learn from what’s been successful in the past. Altogether, the apps in this collection are great inspiration for your own projects. License: Apache 2.0.

Bifrost

If your application requires access to an LLM service, and you don’t have a particular one in mind, check out Bifrost. A fast, unified gateway to more than 15 LLM providers, this OpenAI-compatible API quickly abstracts away the differences between models, including all the major ones. It includes essential features like governance, caching, budget management, load balancing, and it has guardrails to catch problems before they are sent out to service providers, who will just bill you for the time. With dozens of great LLM providers constantly announcing new and better models, why limit yourself? License: Apache 2.0.

Claude Code

If the popularity of AI coding assistants tells us anything, it’s that all developers—and not just the ones building AI apps—appreciate a little help writing and reviewing their code. Claude Code is that pair programmer. Trained on all the major programming languages, Claude Code can help you write code that is better, faster, and cleaner. It digests a codebase and then starts doing your bidding, while also making useful suggestions. Natural language commands plus some vague hand waving are all the Anthropic LLM needs to refactor, document, or even add new features to your existing code. License: Anthropic’s Commercial TOS.

Clawdbot

Many of the tools in this list help developers create code for other people. Clawdbot is the AI assistant for you, the person writing the code. It integrates with your desktop to control built-in tools like the camera and large applications like the browser. A multi-channel inbox accepts your commands through more than a dozen different communication channels including WhatsApp, Telegram, Slack, and Discord. A cron job adds timing. It’s the ultimate assistant for you, the ruler of your data. If AI exists to make our lives easier, why not start by organizing the applications on your desktop? License: MIT.

Dify

For projects that require more than just one call to an LLM, Dify could be the solution you’ve been looking for. Essentially a development environment for building complex agentic workflows, Dify stitches together LLMs, RAG databases, and other sources. It then monitors how they perform under different prompts and parameters and puts it all together in a handy dashboard, so you can iterate on the results. Developing agentic AI requires rapid experimentation, and Dify provides the environment for those experiments. License: Modified version of Apache 2.0 to exclude some commercial uses.

Eigent

The best way to explore the power and limitations of an agentic workflow is to deploy it yourself on your own machine, where it can solve your own problems. Eigent delivers a workforce of specialized agents for handling tasks like writing code, searching the web, and creating documents. You just wave your hands and issue instructions, and Eigent’s LLMs do their best to follow through. Many startups brag about eating their own dogfood. Eigent puts that concept on a platter, making it easy for AI developers to experience directly the abilities and failings of the LLMs they’re building. License: Apache 2.0.

Headroom

Programmers often think like packrats. If the data is good, why not pack in some more? This is a challenge for code that uses an LLM because these services charge by the token, and they also have a limited context window. Headroom tackles this issue with agile compression algorithms that trim away the excess, especially the extra labels and punctuation found in common formats like JSON. A big part of designing working AI applications is cost engineering, and saving tokens means saving money. License: Apache 2.0.

Hugging Face Transformers

When it comes to starting up a brand-new machine learning project, Hugging Face Transformers is one of the best foundations available. Transformers offers a standard format for defining how the model interacts with the world, which makes it easy to drop a new model into your working infrastructure for training or deployment. This means your model will interact nicely with all the already available tools and infrastructure, whether for text, vision, audio, video, or all of the above. Fitting into a standard paradigm makes it much easier to leverage your existing tools while focusing on the cutting edge of your research. License: Apache 2.0.

LangChain

For agentic AI solutions that require endless iteration, LangChain is a way to organize the effort. It harnesses the work of a large collection of models and makes it easier for humans to inspect and curate the answers. When the task requires deeper thinking and planning, LangChain makes it easy to work with agents that can leverage multiple models to converge upon a solution. LangChain’s architecture includes a framework (LangGraph) for organizing easily customizable workflows with long-term memory, and a tool (LangSmith) for evaluating and improving performance. Its Deep Agents library provides teams of sub-agents, which organize problems into subsets then plan and work toward solutions. It is a proven, flexible test bed for agentic experimentation and production deployment. License: MIT.

LlamaIndex

Many of the early applications for LLMs are sorting through large collections of semi-structured data and providing users with useful answers to their questions. One of the fastest ways to customize a standard LLM with private data is to use LlamaIndex to ingest and index the data. This off-the-shelf tool provides data connectors that you can use to unpack and organize a large collection of documents, tables, and other data, often with just a few lines of code. The layers underneath can be tweaked or extended as the job requires, and LlamaIndex works with many of the data formats common in enterprises. License: MIT.

Ollama

For anyone experimenting with LLMs on their laptop, Ollama is one of the simplest ways to download one or more of them and get started. Once it’s installed, your command line becomes a small version of the classic ChatGPT interface, but with the ability to pull a huge collection of models from a growing library of open source options. Just enter: ollama run and the model is ready to go. Some developers are using it as a back-end server for LLM results. The tool provides a stable, trustworthy interface to LLMs, something that once required quite a bit of engineering and fussing. The server simplifies all this work so you can tackle higher level chores with many of the most popular open source LLMs at your fingertips. License: MIT.

OpenWebUI

One of the fastest ways to put up a website with a chat interface and a dedicated RAG database is to spin up an instance of OpenWebUI. This project knits together a feature-rich front end with an open back end, so that starting up a customizable chat interface only requires pulling a few Docker containers. The project, though, is just a beginning, because it offers the opportunity to add plugins and extensions to enhance the data at each stage. Practically every part of the chain from prompt to answer can be tweaked, replaced, or improved. While some teams might be happy to set it up and be done, the advantages come from adding your own code. The project isn’t just open source itself, but a constellation of hundreds of little bits of contributed code and ancillary projects that can be very helpful. Being able to customize the pipeline and leverage the MCP protocol supports the delivery of precision solutions. License: Modified BSD designed to restrict removing OpenWebUI branding without an enterprise license.

Sim

The drag-and-drop canvas for Sim is meant to make it easier to experiment with agentic workflows. The tool handles the details of interacting with the various LLMs and vector databases; you just decide how to fit them together. Interfaces like Sim make the agentic experience accessible to everyone on your team, even those who don’t know how to write code. License: Apache 2.0.

Sloth

One of the most straightforward ways to leverage the power of foundational LLMs is to start with an open source model and fine-tune it with your own data. Unsloth does this, often faster than other solutions do. Most major open source models can be transformed with reinforcement learning. Unsloth is designed to work with most of the standard precisions and some of the largest context windows. The best answers won’t always come directly from RAG databases. Sometimes, adjusting the models is the best solution. License: Apache 2.0.

vLLM

One of the best ways to turn an LLM into a useful service for the rest of your code is to start it up with vLLM. The tool loads many of the available open source models from repositories like Hugging Face and then orchestrates the data flows so they keep running. That means batching the incoming prompts and managing the pipelines so the model will be a continual source of fast answers. It supports not just the CUDA architecture but also AMD CPUs and GPUs, Intel CPUs and GPUs, PowerPC CPUs, Arm CPUs, and TPUs. It’s one thing to experiment with lots of models on a laptop. It’s something else entirely to deploy the model in a production environment. vLLM handles many of the endless chores that deliver better performance. License: Apache-2.0.

While many technology companies have slowed hiring or even launched significant layoffs, that doesn’t mean job opportunities have dried up for software developers. In fact, skilled developers—particularly those with knowledge of AI—are in demand in other industries.

The key to success for developers looking to snatch up these roles is to be well-prepared to meet the needs of potential employers in a variety of sectors.

“The demand for developers in non-tech sectors is real and growing, but the roles look different from what you’d find at a software company,” says Pragati Awasthi, assistant teaching professor of AI and data science at Drexel University.

“Across all these sectors, the common thread is that software is no longer a support function; it is embedded in core operations,” Awasthi says. “The developer in these environments is often the person translating domain-specific business problems into technical solutions, which requires a different profile than a pure product engineer at a tech firm.”

Opportunity knocks

The tech industry has long been a mainstay as far as employing software developers. But as these businesses trim staffs in efforts to cut expenses, that has impacted the hiring landscape. Even as the tech sector scales back, however, companies in industries such as financial services/fintech, healthcare/healthtech, retail/ecommerce, and manufacturing are looking to acquire programming talent.

“The unifying factor is data complexity,” Awasthi says. “These industries generate large volumes of sensitive, regulated, or operationally critical data, and they need developers who can build and maintain systems that handle it responsibly.”

While recruiting firm Summit Search Group has placed developers in roles with technology companies, “it is just as common to recruit them for roles outside this niche,” says Matt Erhard, managing partner at the company. “There are actually a fairly wide variety of roles available for developers in industries beyond tech,” Erhard says.

For example, in financial services Summit Search Group has seen significant hiring for back-end and data engineers who can build and maintain fraud detection systems, digital banking platforms, and regulatory tools, Erhard says. In healthcare, companies are hiring developers to build AI-driven diagnostics platforms and patient portals, or to work with systems that manage electronic health records, he says.

In manufacturing and industrial companies, developers are needed for systems integration and embedded software related to predictive maintenance, Internet of Things (IoT) systems, and smart factories. And in retail and ecommerce, there’s strong demand for full-stack developers and data developers who can handle logistics systems, omni-channel platforms, and personalization engines, Erhard says.

“One significant function where we’ve been placing developer talent lately is in developing business systems and internal applications,” Erhard says. These roles often have titles such as systems engineer or application developer, and professionals are hired to handle tasks such as customizing customer relationship management (CRM) or enterprise resource planning (ERP) platforms, building workflow automation tools or modernizing legacy systems, he says.

Other core functions for which Summit Search Group has placed a lot of developers include data, analytics, and AI-enablement. “That could be directly involved with data engineering or in building tools like reporting systems and ETL [extract, transform, load] pipelines,” Erhard says.

The firm also has handled searches for developers who can build and maintain customer-facing products for banking, healthcare, and retail companies, such as mobile apps or digital platforms customers can use to interact with companies.

Randstad Digital, a provider of global technology talent, sees demand for roles including web developers, system developers, and app developers. “These professionals would work on anything from customer-facing platforms to internal tools,” says Michael Morris, global head of platform and talent at the company. “Non-tech companies are also often hiring roles like software architecture and devops to help scale existing technology. These involve being more ingrained in the business, like building a supply chain system for a retailer, rather than creating individual tech products like you would at a technology company.”

Prep for success

To increases the chances of success at landing developer jobs outside of the tech industry, development professionals would be wise to follow some good practices.

Boost AI skills

One best practice is to boost skills in using AI-powered tools and get familiar with all things AI.

“Get fluent with AI-assisted development and its limits,” Awasthi says. “This is not optional. Organizations across every sector expect developers to use AI coding tools productively. But the more durable skill is knowing when AI output is wrong, incomplete, or unsuitable for a regulated context. That critical evaluation capacity is what non-tech employers are increasingly trying to hire.”

AI does not necessarily replace the need for human developers so much as it changes the skills profile for those roles, Erhard says. “The biggest difference in recent years is that AI literacy is now a non-negotiable,” he says. “At minimum, developers today need to understand concepts like prompt engineering and how to use AI tools to improve their efficiency.”

One thing many job candidates don’t expect is that the rise of AI has also increased the importance of high-level skills such as problem framing, system design, and cross-functional communication,” Erhard says. “Essentially, if something is related to development but too complex or nuanced for an AI to handle effectively, then the demand is high for human developers who have that expertise,” he says.

Candidates who land roles consistently have experience building AI-augmented workflows along with standard coding skills, Erhard says. “Employers increasingly expect to hire developers who can leverage AI, so demonstrating this experience on your résumé can be very beneficial,” he says.

Gain domain knowledge

Summit Search Group is seeing high demand for developers with deep domain knowledge in an organization’s specific industry. “So, for instance, if someone is both an experienced developer and has expertise in healthcare compliance, or financial regulations, then those candidates tend to be very sought after,” Erhard says.

Domain fluency is an underrated skill, Awasthi says. “A developer who understands healthcare compliance, financial regulation, or manufacturing process logic is significantly harder to replace than one who only writes clean code,” she says. “AI can generate boilerplate. It cannot navigate a HIPAA audit or explain a model’s output to a compliance officer.”

Development professionals should “pick an industry and learn it seriously; not just the technology stack but the regulatory environment, the business model, and the actual problems practitioners face,” Awasthi says. “A developer who has read about HIPAA, or spent time understanding credit risk, is immediately more valuable in those hiring contexts.”

It’s also vital to demonstrate real-world, practical application of skills, not just credentials. “The strongest candidates have projects in their portfolio that directly tie to and solve real business problems,” Erhard says.

Acquire soft skills

And then there are the soft skills that are becoming more of a differentiator than they were in the past. As AI handles more routine coding, human developers are expected to make more architectural decisions and collaborate across departments, Erhard says. “Strong communication and problem-solving skills are critical for many of the developer roles that we’re filling today,” he says.

While technical skills are still relevant for developers using and managing AI tools, “they also need to develop the skill of ‘deeper thinking’ and learn how to think one step ahead,” Morris says. “This includes skills like system design mastery—understanding the macro view and learning how microservices, databases, and third-party APIs interact securely and efficiently.”

They also should become deeply fluent in the AI coding tools commonly used in their particular industry, with a strong understanding of how to prompt them for optimal output, Morris says. Product context awareness is also useful. “AI doesn’t know what the customer wants, but you do,” Morris says. “Understanding the business problem and the end-user experience is a requirement for being able to guide LLMs.”

Master debugging and incident response

Developers looking to break into non-tech sectors also should develop skills in debugging and incident response, Morris says. “Complex systems with multiple AI agents can, and will, fail, which means companies need humans to trace logic flaws to get the system back on track,” he says. “A mastery of root-cause analysis is a critical skill.”

“Security, compliance, and reliability are very important in non-tech industries like finance and healthcare,” says Rohit Agarwal, co-founder of Zenius, a remote hiring company. “So employers want developers who also know regulatory environments well.”

Network and keep learning

To successfully pivot from jobs at tech companies, “continuous learning, upskilling, and building hybrid skills that combine technical and business knowledge are essential,” Morris says. “With the right preparation, tech professionals can adapt and continue to thrive in meaningful, dynamic careers.”

It’s also a good idea to join talent communities in fields of interest and “engage with other members in conversations that increase your knowledge through the collective intelligence of the community,” Morris says. “Take advantage of AI skilling opportunities relevant for your role, or better yet, where you want to go next. Experiment with the technology either on your own or through structured programs.” Ultimately, be curious and proactive, he says.

“I’d also recommend developers not to ignore referrals, direct outreach, and industry-specific communities during job search,” Agarwal says. “There are often a lot more opportunities available than the ones posted online.”

Kitreel | shutterstock.com

Ein Freund von mir spielte einmal in der Laeiszhalle in Hamburg. Der Saal war ausverkauft, die Herren sahen elegant aus in ihren Zweireihern und an den Hälsen der Damen hingen Erbstücke, die nur für besondere Anlässe getragen werden. Von meinem Platz aus hatte ich einen guten Blick auf meinen Freund, der weit vorne im Orchester saß und virtuos die Geige spielte. Er sah angestrengt aus, Pizzicato, der Dirigent hob seinen Taktstock, Vibrato.

Nach der Vorstellung traf ich ihn, als er gerade sein Instrument verstaute. Ich teilte ihm mit, wie sehr mir das Musikstück gefallen hat, lobte ihn für sein versiertes Geigenspiel und fragte ihn, wie er selbst seinen Auftritt einschätzte. Ich war überrascht, als er meinte, er habe sich ein paarmal verspielt. Auf meinen fragenden Blick erwiderte er nur: „Zum Glück hat unser Dirigent mich schnell wieder eingefangen – er hat uns alle im Blick.“

Als Musiklaie hatte ich die Rolle des Dirigenten bis dahin nicht richtig verstanden. Schließlich hat dieser über vierzig Musiker vor sich, die ihre Instrumente allesamt besser beherrschen als er selbst. Nun war mir klar: Ohne ihn würde das Stück nicht halb so gut harmonieren. Denn so technisch versiert die einzelnen Musiker auch sein mögen: Es braucht jemanden, der das große Ganze im Blick hat. Der nicht bloß weiß, wie ein einzelnes Instrument zu klingen hat, sondern wie alles zusammenwirken soll. Eine weitere Erkenntnis: Ein Orchester und ein Data-Science-Team haben deutlich viel mehr gemeinsam, als es auf den ersten Blick scheint.

Der Tag des Dirigenten

Heutzutage nutzt jeder Data Scientist Large Language Models (LLMs), um schneller und besser Code zu schreiben. In der Prä-GPT-Ära kostete es mich meist ein bis zwei Wochen, einen (bei Stakeholdern vorzeigbaren) Prototypen zu entwickeln. Jetzt brauche ich dafür weniger als zwei Tage. Die Produktivitätsgewinne sind also enorm. Doch das ist nur eine oberflächliche Veränderung. Wer genauer hinsieht, erkennt einen viel fundamentaleren Wandel.

Das Data-Science-Team von InnoGames, dem ich angehöre, besteht aus sieben Menschen. Vor etwa einem Jahr entschieden wir uns, den Großteil der Programmierarbeit von Claude Code übernehmen zu lassen. Damit traten wir bei uns etwas los, was über kurz oder lang auf jeden Data Scientist zukommen wird: Wir wurden zu Dirigenten. Bislang hatten wir uns eher als Musiker gesehen: Wir wussten genau, wie unsere Instrumente zu spielen waren. Wir kannten unseren Code und unsere Pipelines in- und auswendig – und wussten, welche Techniken für welche Anwendungsfälle zu nutzen waren.

Heute treten wir einen Schritt zurück, um das große Ganze besser im Blick zu haben. Wir koordinieren KI-Agenten, die für uns Teile des Gesamten bauen. Dabei müssen wir sicherstellen, dass dieses große Ganze so aussieht, wie es aussehen soll: Mehr und mehr stehen Architektur und Design Patterns im Fokus.

Während ich über unser neues Selbstbild reflektiert habe, fragte ich mich einmal, ob ich jemals auch nur etwas so Simples wie einen Decision Tree komplett in purem Python geschrieben hatte. Die Antwort lautete nein. Das ist unserer Rolle allerdings überhaupt nicht fremd: Datenwissenschaftler importieren seit jeher den Code den Andere geschrieben haben, um ihn in ihrem eigenen Code zu verwenden.

Die agentische Erlahmung

Der von unserem Team neu eingeschlagene Weg hatte die Idee dahinter nur auf das nächste Level gehoben. Ein knappes Jahr nach dem Umstieg auf „fully agentic“ fühlt sich meine Rolle als Data Scientist immer mehr so an, wie ich mir das ursprünglich einmal vorgestellt hatte. Was wirklich im Kern des Aufgabenfeldes eines Data Scientists steht, ist die Iteration: Wir bauen Modelle, überprüfen die Ergebnisse, wägen ab, überlegen, was wir anpassen können (oder ob wir einen anderen Ansatz ausprobieren sollten) und starten dann die nächste Iteration. Solange, bis wir mit dem Ergebnis zufrieden sind. Indem wir Programmier-Tasks delegieren, können wir schneller und mehr iterieren. Zudem fällt es uns auch leichter, einen wahrscheinlich hoffnungslosen Ansatz zu verwerfen und komplett neu zu starten.

Angesichts einer solchen Veränderung der Rolle des Data Scientists ist es mit einer überarbeiteten Berufsbeschreibung nicht getan. Denn dieser Wandelt geht auch mit einer Änderung der Arbeitsweisen einher. Im Kern dieser neuen Art zu arbeiten, stand für uns, Claude Code eher als Infrastruktur – oder sogar als Mitarbeiter – anzusehen, statt als ein Programm, das man morgens öffnet und abends wieder schließt.

KI-Agenten brauchen Kontext – das implizite Wissen, das wir über Jahre hinweg angehäuft haben – und Coding Guidelines. Sie brauchen kleinere Aufgaben, die von ihnen perfekt ausgeführt werden können und insgesamt gesehen das große Ganze bilden. Vieles von unserer Arbeitsweise findet jetzt in der Konzeptphase statt – und wenn es um die Architektur geht. Sprich, bevor überhaupt Code geschrieben wird. Man muss verstehen, dass es nicht nur darum geht, bessere Prompts zu schreiben. Essenziell ist, sich eine Umgebung aufzubauen, in der die Agenten als integraler Teil funktionieren.

Nachdem wir unseren neuen Weg eingeschlagen hatten, fiel uns jedoch sehr schnell etwas auf, das sich kontraintuitiv anfühlte: Wir wurden langsamer – die Produktivitätsgewinne waren weg. Aus heutiger Perspektive lässt sich dieses scheinbare Paradox einfach erklären: Wir mussten die Agenten trainieren. Ihnen fehlte der Kontext, deswegen war der generierte Code fehlerhaft – und man musste überall ganz genau hinsehen.

Doch mit der Zeit und besserem Kontext wurden die Agenten immer besser. Nach etwa drei Monaten waren wir auf dem Produktivitätslevel von vor der Umstellung angekommen und wurden immer sicherer in unserer neuen Rolle. Nicht zuletzt hatten wir dabei auch Glück: Unsere Chefin gab uns die Zeit, die wir brauchten. Es wurde nicht erwartet, dass uns die Umstellung sofort produktiver machen würde. Und das war auch gut so.

KI-Verlockungen entgegenwirken

Ist knapp ein Jahr nach der Umstellung alles besser geworden? Nein. Es tauchen immer wieder neue Probleme auf, die frische Lösungsansätze benötigen. Mit neueren LLMs und mehr Kontext werden die Agenten zwar besser. Aber das führt dazu, dass man sehr schnell in eine Haltung kommt, in der die Coding-Vorschläge der Agenten vorschnell angenommen werden. Darunter leiden nicht nur die eigenen Programmier-Fähigkeiten: Man kann sich lebhaft ausmalen, welche Probleme entstehen, wenn nur noch die KI-Agenten den Code verstehen.

Berufseinsteiger (nicht nur) im Bereich Data Science haben es dabei besonders schwer: Zum einen, weil Junior-Rollen nun weniger gesucht werden. Zum anderen, weil Einsteigern noch das tiefere Verständnis fehlt und es gleichzeitig sehr verlockend ist, jegliche Bugs an Claude Code zu übergeben. Der löst diese zwar – aber eben ohne, dass es beim Data Scientist zu einem Lernprozess gekommen ist. Dagegen sind auch erfahrene Team-Mitglieder nicht immun. Darum versuchen wir bei Innogames, diesen Verlockungen mit kreativen Ideen entgegenzuwirken. So haben wir etwa einen monatlichen „No AI“-Tag und gemeinsame Pair-Programming-Sessions eingeführt. Ob das ausreichend ist oder sich andere Methoden als sinnvoller erweisen, wird die Zukunft zeigen.

Als ich meinem Musikerfreund kürzlich von meiner Entwicklung zum Dirigenten berichtete, musste er schmunzeln und verriet mir, dass hinter dem Rücken des Dirigenten auch viel über diesen gelacht wird. Und wer weiß, vielleicht sprechen die KI-Agenten ja auch untereinander über mich? Falls ja, würde ich mir wünschen, dass dabei folgender Satz fällt: „Zum Glück hat unser Dirigent mich schnell wieder eingefangen, der hat uns alle im Blick.“ (fm)

Security leaders have made strong progress in visibility. Most organizations can now identify vulnerabilities across their applications, dependencies and development pipelines with far more consistency than in the past. Yet a fundamental imbalance remains: Vulnerabilities are being discovered faster than they can be remediated.

That imbalance is growing. Today, 82% of organizations carry security debt, defined as accumulated vulnerabilities that have remained unresolved for more than a year. At the same time, the share of vulnerabilities defined as both “severe” and “likely to be exploited” continues to increase.

This combination has real consequences. Vulnerabilities are not just accumulating; they persist in production environments long enough to be discovered and used.

Among my fellow CISOs, the conversation has shifted. The challenge now is to translate this reality into a business case that resonates with executive leadership and drives investment in remediation capacity. Here are six ways to do this.

Treat security debt like financial debt

Security debt behaves much like financial debt. It accumulates over time, compounds when left unmanaged and creates ongoing costs for the business. Those costs show up in delayed releases, emergency remediation efforts, audit findings and incident response.

Managing it effectively requires the same discipline applied to financial risk. That means measuring total and critical debt, setting reduction targets and tracking progress over time. It also means distinguishing between acceptable and unacceptable levels of risk, rather than treating all vulnerabilities as equal.

I believe security debt should be visible at the executive level. Leadership teams routinely track financial performance, operational resilience and service reliability. Security debt belongs in the same category. It reflects the organization’s exposure and its ability to manage that exposure over time.

Frame remediation capacity as a business constraint

Most organizations have a strong awareness of vulnerabilities. The limiting factor is the ability to address them.

Remediation capacity determines whether security debt grows or shrinks. When the volume of new findings exceeds the organization’s ability to fix them, the backlog expands and exposure increases. This dynamic persists regardless of how effective detection tools are.

In my experience, it’s important to quantify this constraint. That includes showing the gap between findings and fixes, identifying where high-risk vulnerabilities remain open and demonstrating how long they persist. These data points make it clear that incremental efficiency improvements will not close the gap on their own.

Presenting remediation capacity in operational terms helps align the discussion with executive priorities. Leaders understand constraints in engineering throughput, cloud spend and service availability. Remediation capacity should be treated in the same way.

Focus on exploitable risk in critical systems

Security debt becomes meaningful when it is tied to business impact.

Not all vulnerabilities carry the same level of risk. The ones that matter most share two characteristics. They are likely to be exploited, and they exist in applications that are important to the business.

Traditional severity scoring does not fully capture this. The Common Vulnerability Scoring System (CVSS) remains useful. Still, it does not reflect whether a vulnerability is reachable, whether it sits in a critical system or whether exploit techniques are readily available.

A practical approach is to layer exploitability and business context onto existing scoring models. This creates a focused set of high-risk vulnerabilities that require immediate attention. In many environments, this represents a relatively small percentage of total findings, but it accounts for a large portion of potential impact.

By concentrating on this subset, organizations can direct resources where they have the greatest effect. This approach also makes it easier to communicate risks in business terms.

Prioritize crown-jewel applications

Risk is not distributed evenly across applications.

Every organization has systems that are more critical than others. These may include customer-facing platforms, revenue-generating services or applications that process sensitive data. Compromise in these areas has a disproportionate impact on the business.

Focusing remediation efforts on these crown-jewel applications improves outcomes quickly. Our research found that 11.3% of flaws have high severity and high exploitability. It ensures that the most important systems receive the highest level of protection and reduces the likelihood of high-impact incidents.

Clear targets help reinforce this focus. Over a defined period, organizations can reduce critical security debt, shorten the lifespan of high-risk vulnerabilities and maintain strict thresholds for exposure in key systems. These targets translate security activity into business outcomes that leadership can understand and support.

Establish metrics that reflect risk

Metrics play a central role in shaping behavior.

Many organizations continue to rely on the number of vulnerabilities discovered or resolved. While these metrics provide useful context, they do not indicate whether risk is increasing or decreasing.

More effective measures focus on exposure. These include the number of critical or exploitable vulnerabilities in key systems, the average age of those vulnerabilities and trends over time. Together, these metrics provide a clearer picture of how risk is evolving.

Linking these measures to organizational objectives strengthens accountability. Security debt reduction can be incorporated into OKRs, with specific targets for reducing critical debt, lowering vulnerability age and maintaining acceptable thresholds in high-risk applications.

Formalizing risk acceptance is also important. High-risk vulnerabilities that remain open should require business approval and defined timelines. This ensures that risk is acknowledged and managed deliberately.

Increase investment in remediation capacity

Improving security outcomes requires sustained investment in the ability to act.

Remediation capacity can be expanded in several ways. Organizations can allocate dedicated engineering time for security work, integrate remediation into development workflows and adopt automation to reduce manual effort. AI-assisted fixes and automated guidance can help teams address vulnerabilities more efficiently without disrupting development velocity.

Preventing new security debt is equally important. Policies such as requiring high-risk vulnerabilities to be resolved before release help limit the introduction of additional exposure. Over time, this reduces the overall burden on remediation teams.

These changes do not slow innovation. They create conditions for delivering software safely and consistently.

Align the business around risk reduction

Security debt affects more than the security function. It influences resilience, regulatory posture and the organization’s ability to deliver software with confidence.

CISOs play a central role in aligning stakeholders around this issue. By framing security debt in terms of business impact, capacity constraints and measurable outcomes, they can shift the conversation from technical backlog management to enterprise risk reduction.

This alignment is critical for securing investment. When leadership understands the relationship between remediation capacity and business risk, decisions about funding, prioritization and trade-offs become clearer.

Security debt will continue to exist. What matters is how effectively it is managed and measured. For example, a good target should be doubling fix capacity through tooling investment, not just headcount.

Organizations that measure, govern and actively invest in reducing it are better positioned to control risk at scale. Those that do not will continue to see exposure grow, even as their visibility improves.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?