Over the past year, I had the same conversation with almost every team starting an AI initiative. Should we build on Semantic Kernel, AutoGen or Foundry?

At first it felt like the most important architectural decision we’d make. Each framework had its own philosophy, each promised to be the foundation for enterprise AI, and picking the wrong one felt like an expensive mistake. I spent a lot of time helping teams weigh the trade-offs.

Looking back, I think we were asking the wrong question. I certainly was.

I watched teams spend months debating SDKs while the decisions that actually decided whether their applications survived production went unexamined. Some built elaborate orchestration layers for workflows that a few deterministic functions would have handled. Others avoided agent frameworks entirely and later found they’d designed themselves into a corner.

Then Microsoft settled it for us. It introduced the unified Agent Framework, quietly moved Semantic Kernel and AutoGen into maintenance mode, and the debate I’d spent months refereeing was suddenly over. Turns out the answer to “which of the three” was “none of the three, here’s a fourth.” The framework hit version 1.0 and general availability in April 2026, stable across .NET and Python.

What surprised me wasn’t the decision. It was how fast a debate that had eaten so much of our attention stopped mattering. Microsoft changed the menu.

It didn’t change the meal.

The framework was never the hard part

Framework selection dominated almost every early conversation I had about enterprise agents. Which SDK do we standardize on? Which orchestration model gives us the most flexibility? Which one is Microsoft actually betting on?

Fair questions. But after a year of watching these projects play out, I’ve slowly come around to a different view. Those weren’t the questions that decided anything.

The first question I ask now is much smaller. Does this thing actually need an agent?

It sounds obvious, and I still get it wrong sometimes. But it’s the mistake I see most. On one project, a team spent weeks designing a multi-agent workflow for a process that ran the same four steps every time: read a document, validate it, call an API, send a notification. The diagrams looked great. The system in production didn’t. A few well-tested functions would have been easier to build, easier to maintain and a lot easier to trust.

Part of this is just that “agent” has become the word everyone reaches for. Sometimes it’s the right call. Sometimes it’s a workflow we already knew how to build, wearing a newer label. An agent earns its complexity when it genuinely has to decide things you can’t predetermine, choosing between tools, adapting to what it finds, working out its own next step. If you already know every step, you have a workflow, and a workflow is usually the better engineering choice. The consolidation didn’t change that. It just made it easier to see.

What building production agents actually taught me

Once I stopped fixating on frameworks, the same three problems kept showing up. None of them had anything to do with the SDK.

Context beats model choice

Early on I spent a lot of time comparing models, the way you’d agonize over a restaurant menu and then order what you always order. Now I spend most of it thinking about context, which is far less fun and far more useful.

I’ve watched good models fail because they were handed too much, not too little. One team I worked with gave the model access to nearly every internal document they had on the theory that more information meant better answers. It went the other way. Responses got slower, less consistent and sometimes skipped right past the thing that actually mattered. When we cut the context down to only what the task needed, the quality jumped almost immediately. I didn’t predict that. It taught me to be suspicious of “just give it everything.”

The best agent systems I’ve worked on weren’t the ones with the biggest context windows. They were the ones careful about what reached the model, and when. That’s not something the framework hands you.

Failure is where the real work is

Most agent demos look great because they’re built around the happy path. Production doesn’t extend that courtesy.

I remember a project where everything held up in testing. Then a downstream API timed out after the agent had already completed several earlier steps. We couldn’t just restart, because part of the business process had already gone through. We ended up spending far more time on recovery logic than we ever spent on prompts. That project changed how I think about this work. The hard part was never getting the model to make a decision. It was making sure the system didn’t fall apart when reality refused to follow the script.

Tool calls fail partway through. APIs return inconsistent data. Models call the same tool over and over because the last answer wasn’t what they wanted. That’s not the exception; that’s a normal Tuesday. Whether you retry, roll back, pause for a human or push on with partial results is a judgment call, and no framework is going to make it for you.

Identity is the real security boundary

This one surprised me most. The moment an agent stops being a chatbot and starts touching real business systems, identity matters more than orchestration.

Every project gets to the same question eventually. Who is this agent actually acting as? The developer’s credentials? A service account? The user who asked? Get it wrong and you’ve built something autonomous running with more access than any single person should have, which is exactly the kind of thing that looks fine until an audit. The Agent Framework, like most modern tooling, makes it easier to wire agents to tools through standards like the Model Context Protocol. That helps. But where human approval belongs, what needs extra authorization, how much rope to give the thing, those are still yours to decide.

The surprises weren’t technical

Here’s what I didn’t see coming. The hardest part of last year wasn’t technical at all. It was organizational. The moment a team heard “agent,” expectations shifted under everyone’s feet. Business stakeholders started expecting full autonomy. Developers assumed the thing could reason its way through anything. People started designing for flexibility before we’d even agreed on what problem we were solving. The word did damage before any code did. I found myself spending as much time resetting expectations as I did discussing architecture.

Build for change, not for today’s winner

I don’t think the teams that struggled last year picked the wrong framework. Semantic Kernel was reasonable. AutoGen was reasonable. Foundry made sense for plenty of cases. I’d have signed off on any of them.

The ones that got hurt put all their eggs in one framework, treating it as the foundation of the whole system instead of as one more dependency. Microsoft provided a migration path. But teams that had tightly coupled their applications to framework-specific abstractions discovered that migrating and rewriting are not the same thing. That wasn’t Microsoft’s doing. It was their own architecture’s. The teams that moved easily had kept their business logic, prompts and orchestration loose enough to evolve independently of any one SDK. For them, the change was a manageable project, not a teardown.

For what it’s worth, nobody I work with is treating this as an emergency. Most are moving the smaller workloads first, watching how they behave and leaving the production-critical systems alone until they actually understand the new abstractions. That’s the right instinct. And I doubt this is the last consolidation we’ll see, the ecosystem is still young, frameworks will keep absorbing each other and over time the differences between them will be operational more than architectural.

I don’t regret the framework debates, honestly. They were reasonable at the time. What changed wasn’t Microsoft’s roadmap.

It was mine. Watching these systems run in production taught me that the framework is the easiest piece to swap out. Recovery logic, context management, security boundaries, the business workflow itself, those stay with you long after today’s SDK gets replaced by tomorrow’s.

So, Microsoft made one decision easier by turning three frameworks into one. Good. Five years from now we’ll be on different tools, and we’ll still be asking the same handful of questions.

Does this actually need an agent? Does it have the right context? Can it recover when something breaks, because something will? Is it acting as the right person?

Those questions outlast every rewrite. That’s where I’ve learned to put my effort.

Frameworks come and go. Good architecture has to survive all of them.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

In the client-server era, the penalty for inefficient programming, such as unoptimized database calls, was largely confined to application responsiveness. Today, in the AI era, code, architectural, and platform inefficiencies are no longer just a performance issue, they’re a financial and environmental liability. Left unchecked, poor code cascades into soaring token costs and spikes data center power consumption, directly undermining both cloud budgets and corporate sustainability goals.

AI’s impact on sustainability

By 2029, IDC projects that the number of actively deployed AI agents will exceed 1 billion worldwide, which is 40 times more than in 2025. And these agents will perform 217 billion actions per day.

To deliver on this demand, AI data centers are being built out at an unprecedented rate, with Gartner forecasting that global spending on data centers over the next three years will increase 31.7% to surpass $650 billion, driven primarily by hyperscaler cloud providers building out AI foundations, and optimizing servers for heavy AI workloads.

All this presents a significant strain on the energy grid as well as environmental sustainability, including:

• The power double-down: The International Energy Agency (IEA) projects that global data center electricity consumption will more than double from about 415 to 945 TWh by 2030, primarily fueled by energy-intensive accelerated computing for AI.
• The inference premium: AI workloads are vastly more demanding than standard web activities. A gen AI query consumes roughly 10 times the electricity of a conventional keyword search, or roughly 2.9 watt-hours as opposed to 0.3 watt-hours.
• Water consumption: Cooling these dense clusters is highly resource intensive. Global AI-related water demand is expected to reach 4.2 to 6.6 billion cubic meters by 2027.

The good news, however, is it’s not all out of the control of end user organizations and CIOs. Just as in the client-server era, through careful planning and execution, CIOs have the potential to significantly improve the performance, costs, and sustainability impacts of their AI application portfolio.

Here are four recommendations to maximize value as you look across your AI applications and infrastructure estate.

Revisit business objectives in light of AI

AI applications and platforms bring several new headaches for CIOs and CFOs in terms of FinOps. The variable nature of AI vendor billing due to variable monthly token costs is just one well-known example. To avoid unpleasant surprises, be sure to carefully review vendor contracts to decipher pricing models. Look for what’s included in seat-based license fees and what’s added as variable charges for agentic AI usage.

In addition, explore new metrics and KPIs such as intelligence per watt to help make sense of your return on AI. Just as miles per gallon helps us evaluate new car purchases, IPW can help to measure the computational efficiency of a system. It quantifies how much intelligence — typically measured in AI inferences, tokens processed, or model training iterations — a processor can deliver for every watt of electrical power it consumes.

According to Max Romanenko, chief engineering officer at relational database platform EDB, cost per query tells you almost nothing in an agentic world where autonomous systems are spinning up databases, pipelines, and queries around the clock. “The metric that matters is intelligence per watt, how much useful AI you get for every unit of energy you spend,” he says. “It isn’t just an environmental number, it’s also a performance indicator.”

With the measurements in place, you can then start to manage and optimize each layer in the AI stack from the infrastructure, or hyperscaler, layer to your own data and application layers.

It’s important to bear in mind that high token usage isn’t necessarily a bad thing. It depends on the net value delivered by each AI application and use case. Managing and optimizing the AI stack is important, but you’ll also want to measure the business value being delivered by each of these applications so you can measure your return.

Take a sovereign AI approach when evaluating hyperscalers

As you work with hyperscalers like Amazon, Google and Microsoft, it’s important to understand how they charge and how much, but also their environmental footprints. For example, by reading their sustainability reports, you can find out their annual water consumption across their global data centers and compare them with other providers.

In 2025, Amazon’s global data center operations used 0.12 liters of water per kilowatt-hour, which amounts to 2.5 billion gallons, or 5% of the annual water consumed by the metro Seattle area. The company has been able to operate more than seven times better than the industry average and have improved their water efficiency by 52% since 2021.

As demand for cloud computing and AI grows, water efficiency is another important metric for CIOs to monitor within hyperscaler ESG reports. While not at the same level of regulation as scope 2 and 3 greenhouse gas (GHG) emissions reporting, enterprises need to pay increasing attention to water use efficiency (WUE) with water scarcity becoming a growing risk for hyperscalers.

The key requisite at the infrastructure layer, though, is to ensure sovereign AI. This doesn’t mean you need to own everything, but you need control over your AI-driven operations when conditions change. With 71% of global executives stating that switching their primary AI vendor or model would be difficult if required today, it’s important to understand AI dependencies and be able to avoid vendor lock-in. 

Control efficiency at the data layer

The AI energy conversation has fixated on models and GPUs, but every agent, model, and inference call runs on the data layer beneath them, and that’s the one place a CIO can actually move the numbers.

“You can’t control consumption at the model layer,” says Romanenko. “Agents consume what they consume. But you can control efficiency at the data layer, and for most enterprises that’s the only real lever they have. Optimize search, retrieval, and vector indexing where the work actually happens and you cut compute, cost, and carbon at the same time. Ignore it, and it’s like running the heat with every window open.”

Ann Dunkin, distinguished professor of the practice at Georgia Tech, adds that CIOs who bring models in house and run them in their own infrastructure, or in the cloud infrastructure of their choosing, can have more control over the sustainability of inference, as well as of their costs and how their data is used.

Fine tune the application layer

When balancing a mix of commercial AI packages and custom-built code, costs can quickly spiral due to inefficient design and orchestration, redundant APIs, and unoptimized model routing.

With inference calls costing approximately 10 times that of conventional web queries, for custom AI applications, it’s important to design them to only use probabilistic code where necessary. Since many custom applications utilize a combination of both probabilistic and deterministic code, this is exactly where software developers need to make smart choices in their designs.

Other techniques to fine tune the application layer include semantic caching, intelligent model routing, and internal AI capability registries. “CIOs can implement intelligent routing solutions to select the most cost-efficient model for every prompt,” says Dunkin. “The most flexible routing solutions can drop into a user’s existing environment and orchestrate the actions of the company’s existing models.”

For CIOs looking to maximize the business value of every AI application in their portfolio, these new considerations, including new metrics, tools and approaches from the infrastructure layer all the way up to the application layer, should be an essential part of the equation.

I use a very cool and relatively new web framework called Astro. The keen insight that the Astro team had was that most websites are made up of static content, so they made it really easy to add content to a website. To add a blog post to my personal website, all I have to do is create a Markdown file with some front matter, deploy it, and the blog post automatically appears. If I need to reach deeper for more dynamic functionality, I can easily do that with TypeScript, React, or almost any other framework. It’s really cool.

And these days, I really don’t write any code. Claude Code does most (all?) of the work. Since Astro is an open-source project and has excellent documentation, Claude knows all about how Astro works. It has no trouble at all managing my site and making the improvements I ask for.  

And that got me thinking, how does Astro get built? Is the Astro team building with agentic coding? Astro itself has many dependencies, including big projects like Vite and Node. And of course, Vite and Node have dependencies, too. Are those dependencies being developed by hand, or are those development teams also using AI agents to code?

My curiosity got the best of me, and I asked Claude to dig deeper. It turns out that the Astro repository has an AGENTS.md file, and some of the commits even have commit message trailers indicating that they were at least co-authored by Claude and GitHub Copilot. Further down, there is a .agents/skills directory with skills covering development, merging, triage, and more. I poked around for a look, and someone has done a great job building agentic support.

Now my interest is really piqued, and further investigation reveals quite a bit of interesting stuff. About a year ago, documentation started appearing about how to build Astro sites with coding agents.  Around that same time, the docs team released an MCP server that gives developers coding agents deeper, easier access to the Astro documentation.  

And there are small steps in the Astro codebase that indicate it is “agentic ready.” For instance, the command-line development server can tell when it is being started by an agent, and the application itself can tell if it is being driven by an agent. Small things, but steps in the direction of embracing Astro developers who use coding agents. 

Okay, that was a fun spelunking trip. But so what?

The “so what” is that code is going to be commoditized. As an Astro developer I am using AI agents pretty much all of the time. The Astro development team is starting to use AI agents more and more. The folks building the Astro dependencies are using AI agents. Shoot, the people building Claude Code and the agents themselves are “eating their own dogfood” and using their own tools to build the next frontier model. Before we know it, it will be turtles all the way down. 

No one says “who generated that electricity?” or “who wove the fabric in that shirt?” any more. And it won’t be long before no one says “Who wrote the code for that app?” because it won’t matter. Just as we don’t look at the assembly code written by our compilers, we’ll stop looking at the “regular” code written by our agents. I’m not even sure anyone is writing assembly code anymore. Soon we’ll be saying that about TypeScript, Python, and C++.

Every enterprise AI conversation seems to begin with the same question: Which model should we use?

I understand why. Models are visible. They have names, benchmarks, release notes, pricing pages and impressive demos. They are easy to compare in a leadership meeting. One model promises better reasoning. Another offers a larger context window. Another appears faster, cheaper or more specialized.

But after years of working around enterprise platforms, integration layers, cloud migration, middleware, production operations and mission-critical systems, I see the AI conversation differently.

The model matters. But it is not where most enterprises will struggle next.

The next AI bottleneck is the infrastructure behind the model.

I do not mean only GPUs, cloud capacity or data storage. I mean the full enterprise operating layer that allows AI to work safely in the real world: data pipelines, identity, APIs, messaging, observability, security controls, deployment automation, cost governance, auditability, support ownership and recovery design.

That layer is what determines whether AI remains an exciting experiment or becomes a trusted business capability.

Pilots hide the hard part

Most organizations can build an impressive AI pilot. A small team can connect a model to a dataset, create a workflow and show a use case that works well in a controlled setting.

The harder part starts when that pilot moves into a real production process.

That is when practical questions show up. Who owns the data quality? What systems can the AI access? How do we trace which prompt, policy or retrieval flow produced a specific answer? What happens when an API slows down, a queue backs up or a downstream system is unavailable?

To me, these are not model problems. They are infrastructure problems.

This is where many enterprises are now headed. The first phase of AI was experimentation. The next phase is operationalization, and that is where the real gap becomes clear.

McKinsey has made a similar point in its work on agentic AI, noting that the next phase of value depends less on isolated tools and more on redesigning workflows, operating models and enterprise execution around agents.

AI pilots can survive on enthusiasm. Production AI requires architecture.

AI is becoming an integration problem

The more I look at enterprise AI, the more it feels like an integration challenge.

In large organizations, I have seen how messaging platforms, integration gateways, deployment pipelines, monitoring tools and cloud infrastructure can decide whether a digital capability succeeds or fails. AI will be no different. Even the strongest model will struggle if the data, middleware, identity layer and operational controls around it are weak.

AI does not work in isolation. It needs context from systems of record, clean data from different business areas, secure access to APIs, event streams, workflows, knowledge repositories, monitoring tools and legacy systems.

That is why the CIO question is changing.

It is no longer just, “Which AI tool should we buy?”

It is becoming, “Can we safely operationalize intelligence across the business?”

This is where agentic AI matters. Autonomous AI only creates real value when the architecture around it can make its actions safe, traceable and useful.

A model can generate an answer. Infrastructure determines whether that answer is secure, timely, explainable, governed and connected to the right workflow.

For example, an AI assistant that summarizes customer or order information may look like a model use case. But underneath, it depends on access control, fresh data, reliable APIs, logging, encryption, monitoring and policy enforcement.

If the answer is wrong, people may blame the model. But the real failure may have started with stale data, weak integration, poor access design, missing observability or an unreliable downstream system.

That is why CIOs should not judge AI only by model capability. The enterprise system around the model matters just as much.

Latency will become a trust issue

In traditional technology operations, latency is often treated as a performance metric. In AI-enabled workflows, latency becomes a trust issue.

When an employee asks an AI assistant for help and the response takes too long, the employee stops using it. When a customer-facing workflow becomes slow, the customer abandons it. When an AI agent waits on multiple backend calls, the entire business process feels unreliable.

This becomes even more important as organizations move from simple chat interfaces to agentic workflows. A single AI-driven action may include identity checks, context retrieval, policy validation, model reasoning, API calls, business-rule execution, logging and human approval.

Each step adds latency. Each dependency adds a possible failure point.

A model may be fast in a benchmark but slow inside an enterprise process. That difference matters.

This is where platform engineering becomes essential. Enterprises need reusable patterns for AI workloads: approved connectors, secure retrieval methods, queue-based decoupling, caching strategies, deployment pipelines, monitoring dashboards and standard rollback procedures.

Without those patterns, every AI initiative becomes a custom build. Custom builds may work for pilots, but they do not scale across a large enterprise.

Observability has to expand

Traditional monitoring tells us whether infrastructure is healthy. Is the server up? Is CPU high? Is memory exhausted? Is the application returning errors?

AI needs that, but it also needs more.

We need to know what data was retrieved, which model was used, which prompt version was active, which user initiated the request, which policy was applied, how long each step took and whether the output passed validation.

We also need to detect new forms of risk: unusual usage patterns, repeated failed tool calls, unexpected cost spikes, sensitive data exposure, weak retrieval results or an AI workflow attempting actions outside its intended boundary.

In production AI, observability is not only about uptime. It is about confidence.

If a business leader, auditor, regulator or security team asks why an AI system made a recommendation, the answer cannot be, “The model said so.” The enterprise needs traceability. It needs evidence. It needs operational context that engineers, risk teams and business owners can understand.

This is one of the biggest gaps I see in AI strategy. Many organizations are investing in models and use cases, but not enough in the control plane required to manage them.

Data readiness is still underestimated

AI has exposed an uncomfortable truth: many enterprises are not as data ready as they think.

Data is often duplicated across platforms, described differently by each team, governed inconsistently and refreshed on different schedules. Access rules may be clear in one system but unclear in another. Even basic business definitions can change from department to department.

AI does not fix that automatically. In many cases, it makes the problem more visible.

A bad report may be questioned. A bad AI answer may sound confident enough to be trusted.

That is a real risk.

Being data-ready for AI is not just about connecting a vector database or indexing documents. It requires clear ownership, lineage, classification, quality checks, retention rules, access boundaries and a shared understanding of which data should be used for which purpose.

The same principle applies to resilient cloud-native design. In my IEEE TechRxiv paper, “Enabling Fault-Tolerant Multicast in Cloud-Native Architectures” I explored how reliability, observability and fault tolerance become foundational requirements when critical workloads stretch across hybrid and multi-cloud environments.

CIOs already understand this because they have lived through enterprise resource planning programs, cloud migration, integration modernization, cybersecurity transformation and analytics initiatives. The lesson is familiar: technology cannot outrun data discipline forever.

Security cannot be added later

As AI moves from answering questions to acting, security becomes much more important.

An assistant that summarizes information carries one level of risk. An agent that can open a ticket, update a record, trigger a workflow, approve a request or contact a customer carries a very different one.

The more AI can do, the more identity, authorization, least privilege, separation of duties and human approval matter.

Enterprises should be careful not to grant AI broad access just to speed up a pilot. That may seem harmless in development, but it can become dangerous at scale.

AI access should be treated like any other privileged enterprise capability: limited, logged, reviewed and easy to revoke.

The NIST AI Risk Management Framework is a useful reference point here because it frames AI risk as something organizations must govern, map, measure and manage continuously rather than something handled only at the end of deployment.

Security teams should be involved early, not at the end. The goal is not to slow innovation. The goal is to build a platform where safe innovation becomes repeatable.

The CIO has to define the operating model

AI is creating pressure from every direction. Boards want productivity. Business teams want automation. Employees want better tools. Vendors are pushing new features. Security teams are watching risk. Finance teams are watching cost. Customers expect faster, smarter experiences.

The CIO sits in the middle of all of it.

That is why the CIO’s role cannot stop at choosing tools or approving pilots. The CIO has to define how AI will actually operate across the enterprise.

That means answering practical questions. Which architecture is approved? Which data sources can be trusted? How are AI workflows deployed, monitored, supported and governed? How are costs controlled? How do teams reuse common patterns instead of rebuilding the same foundation each time?

This work may not be as exciting as a model demo, but it is what separates sustainable AI from short-term experimentation.

The winning organizations will not be the ones with the most pilots. They will be the ones with the strongest AI operating layer.

They will build reusable platform patterns, strengthen data governance, design access properly, monitor AI behavior end to end and measure success by business improvement, not only model performance.

The model still matters. But the enterprise behind the model matters more.

A powerful model on weak infrastructure will eventually disappoint the business. A capable model on strong infrastructure can deliver real value because it can be trusted, secured, scaled and improved.

That is the shift CIOs need to lead.

The next AI bottleneck is not the model. It is whether the enterprise behind the model is ready.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Somewhere in the last eighteen months, “AI agent” stopped being a useful term. CIOs may even be afraid to ask what “agent” truly means, as it now seems to describe everything from a chatbot that answers HR questions to an autonomous research system that plans its own week of work. When a single phrase carries that much weight, well, it stops carrying any.

I’ve spent the last three years inside hundreds of enterprise AI deployments, and the factor that separates the programs scaling elegantly from the ones still shuffling is often the CIO’s architectural fluency: The ability to look at business problems across the organization and recognize, on sight, what kind of AI architecture is the right fit. In my experience there are six archetypes, each with their own nuances, that CIOs should internalize to make well-informed decisions going forward.

1. The conversational assistant

The first, and the one most enterprises meet first, is the conversational assistant: The chat-based partner that an employee or customer opens when they want to think out loud. Deloitte found that 38% of organizations report AI is already strengthening their client or customer relationships. This is the architecture people fall in love with: A well-designed assistant with constantly updated information, persistent user-level memory, tools that can act on behalf of users, and citations on every factual claim becomes a useful problem-solver that’s available at any hour of the day.

A global law firm I work with deployed an internal assistant that gives every attorney instant access to the firm’s accumulated precedent, memos and prior matter work. Associates who used to spend the first hour of a research task hunting through document management systems now start with a grounded, citation-backed answer and refine from there. This helped the firm’s institutional knowledge, previously locked in the heads of senior partners, become queryable by anyone with a deadline at 11 p.m., or later.

A second example: A mid-market wealth management firm built a client-facing assistant that handles portfolio questions, statement explanations and routine servicing requests. The assistant draws from each client’s actual holdings, recent activity and the firm’s published market commentary, with citations linking back to source documents. Advisors stopped being interrupted for the questions that didn’t require an advisor, and clients got answers on a Sunday.

2. The triggered workflow

Another pattern producing the value across the enterprises I work with is something that runs silently: An email arrives, a ticket is created, a file lands in a folder and the agent executes a process utilizing both reasoning and determinism. These agents don’t even require user adoption, because they’re invisible to the end user. They produce measurable outcomes, but fit cleanly into the audit and change-control processes IT teams have run for decades.

A commercial insurer I advise built a triggered workflow for inbound submissions. Every broker email that arrives at the underwriting inbox is classified by line of business, the attachments are parsed, key risk fields are extracted into the policy administration system, and a draft acknowledgment is queued for the underwriter’s review. Seemingly overnight, the inbox began arriving pre-sorted, and submission throughput rose meaningfully without any change to headcount.

Another example, this time from a private equity firm: Every inbound confidential information memorandum (CIM) that hits the deal team’s shared inbox triggers a workflow that extracts the financial summary, screens it against the firm’s investment criteria, drafts a preliminary memo and posts the result into the deal-tracking system. Associates still make the call on what to pursue, but the first three hours of manual work on each opportunity now happen before anyone even opens the file.

3. The autonomous agent — with sub-agents

Here we have the architecture that gets the most conference attention: The autonomous agent, given a task and left to plan its own steps by utilizing its own sub-agents. Autonomous agents are not one-size-fits-all, but they do meet a specific need: Multi-source research, complex cross-system lookups, deep-dive investigations. All of these are processes where the path isn’t usually specified in advance, but the tools are. With the right design discipline, an autonomous agent feels like having a self-sufficient teammate who can call in the right resources and specialists if needed.

A global consulting firm I work with uses an autonomous research agent for early-stage engagement scoping. Given a target company and a strategic question, the agent decides for itself which sub-agents to consult (choosing from internal proprietary databases, prior engagement archives, licensed market data, public filings) and produces a structured briefing with its reasoning chain attached.

Another large technology company I know of deployed an autonomous agent for cross-system incident investigation. When a production alert fires, the agent forms a hypothesis, queries the necessary sub-agents with relevant monitoring tools, log stores and deployment systems, and follows the trail until it reaches a defensible root-cause summary to surface to an engineer.

4. The multi-agent team

The fourth pattern is where the next wave of enterprise quality gains is going to come from. According to Databricks, usage of multi-agent systems grew 327% in just four months as enterprises moved beyond single chatbots. Several specialized agents, each with its own role and toolset, coordinate through a shared protocol: A researcher and a writer, a planner and a set of executors, a proposer and a critic. The proposer-critic feedback loop is one of the smartest techniques in agent design today. One model produces an answer; a second, with a different prompt and often a different provider, evaluates it against explicit criteria. For compliance review, contract analysis, high-stakes classification and any output that will be audited, this second pass is extremely helpful and mirrors how human teams work.

A global bank I work with uses a multi-agent system for marketing and communications review. One agent drafts client-facing copy, a second checks it against the firm’s regulatory and brand guidelines and a third checks it against jurisdiction-specific disclosure rules. Disagreements among the agents are surfaced to a human reviewer with the specific clauses flagged. The compliance team stopped being the bottleneck on every routine piece of copy and started focusing on the high-judgment cases instead.

The next example: A pharmaceutical company built a multi-agent workflow for medical literature summarization. A retriever agent gathers candidate studies, a reader agent extracts study design and findings, a critic agent challenges the reader’s claims against the source text, and a synthesizer agent composes the final brief. The proposer-critic loop in the middle is the reason the medical affairs team trusts the output enough to act on it.

5. The human-in-the-loop (HITL) agent

The fifth pattern is the one I think we’ll see increasingly more of in the future. While many see “full automation” as the goal, the right target is actually to let the agent handle the 80% of a task that is mechanical, while preserving human judgment at the most critical moments. This is achievable via human-in-the-loop (HITL) agents. According to Moody’s, 42% of compliance professionals believe that human oversight is mandatory, and I agree: AI should run right, by getting approval and review before any sensitive business action is taken. HITL is the architecture that can help turn a skeptical team into an enthusiastic one.

A regional health system I worked with uses a HITL agent for prior-authorization letters. The agent assembles the clinical evidence, drafts the letter against the relevant payer’s criteria, and routes it to a nurse case manager for review inside the existing workflow tool. The nurse approves, edits or rejects in seconds rather than minutes, and every edit helps make the next draft better.

A property management company uses a HITL agent to run its maintenance work orders. When a tenant emails about a problem (an HVAC unit that died overnight, say), the agent pulls the structured details (tenant, unit, issue type, urgency), matches the job to the right vendor from the directory, and drafts the work order. A team member approves it in Slack before anything goes out. From there the agent emails the vendor with the full order, confirms with the tenant that someone is on the way and updates Airtable, closing the loop completely.

6. The scheduled agent

On a set schedule or against a batch of inputs, this agent runs the same defined task: Produce a report, refresh a dataset, monitor a set of sources or summarize a period of activity. Under this archetype, unsexy work gets done consistently, integrated into existing operational rhythms like the Monday morning meeting, the daily standup and the monthly board deck, without asking anyone to change their behavior. This is the architecture that shifts AI from feeling like even more work, to a seamless teammate that just works.

A private equity firm I work with runs a scheduled agent every Monday at 6 a.m. that monitors news, filings and earnings activity across every portfolio company and produces a single PDF that lands in the deal partners’ inboxes before the weekly investment meeting. No one logs into a dashboard. The agent shows up, on time, with the same format every week, and the meeting now starts from a shared baseline rather than from whatever each partner happened to read over the weekend.

A second example: A global manufacturer runs a nightly batch agent that ingests the day’s quality-control reports across plants, summarizes anomalies against a rolling baseline, and produces an end-of-shift handoff document for each site lead’s morning. The agent doesn’t flag emergencies, but it ensures that the slow-moving patterns no human would catch reading one shift’s data in isolation get surfaced.

Bringing it together

None of these six archetypes is more advanced than the others or inherently better. But CIOs can have an edge by choosing the one that the operational problem actually calls for.

Before you scope a single deployment, you should be able to look at a business problem and name its shape: Is this a question someone needs answered in the moment, or a process that should run the instant a trigger fires? Does the path need to be discovered, or is it known in advance and just waiting to be executed? Where, exactly, does human judgment have to stay in the loop, and where is it just friction?

Going forward, CIOs should start treating the architecture decision as the first design choice. Everything downstream — adoption, governance, trust — only gets easier if the architecture is the right fit.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Shadow IT has long been a major problem for CISOs, but the biggest problem may be coming from the executive suite’s hunger for unsanctioned AI.

Nearly two-thirds of senior decision-makers admit to using unapproved AI tools, compared to just 31% of lower-level employees, according to a survey by Microsoft solutions partner TrustedTech.

The use of shadow AI is prevalent among senior executives even though three in four employees acknowledge security or data privacy risks related to the practice.

“Most shadow AI users are not ignorant of the risk,” TrustedTech says in a white paper. “They are deliberately choosing to use these tools anyway. This is not a training issue. It is a culture, incentives, and alternatives issue.”

In many cases, the problem is driven by a lack of approved tools, the report adds.

“People use shadow AI because what their employer hands them is worse than mainstream AI tools, or because nothing has been approved in the first place,” the report says. “That doesn’t change until the sanctioned tools are genuinely worth using.”

A question of authority

The use of shadow AI by CEOs and other C-suite executives can create major problems for CISOs, CIOs, and other IT executives because they may not have the authority to put the kibosh on it.

It also presents a challenge for IT leaders to provide the AI tools that employees and executives want to use.

When executives use shadow AI, CISOs are in a difficult position, because governance only works when it’s modeled from the top, says Andy Nolan, VP of technology at TrustedTech.

“If senior leaders bypass approved AI tools or policies, it sends an implied message that speed matters more than security and compliance,” he adds. “Employees notice that behavior, and it becomes much harder to ask the rest of the organization to follow standards that leadership isn’t following themselves, first.”

Another major problem is that executives often work with highly sensitive information, including financial data, strategic plans, intellectual property, and customer information, he notes.

But CISOs and CIOs also can’t solve the problem by becoming the AI police in every situation, Nolan says, because their role is to help the business innovate safely.

“That requires executive alignment, clear governance, and providing secure AI tools that people actually want to use,” he adds. “When leadership embraces those solutions, the rest of the organization is almost sure to follow.”

All risk, no reward

The use of shadow AI by senior executives puts CISOs and CIOs in an impossible position, agrees Amit Maloo, CISO at AI procurement provider Ivalua. CISOs and CIOs are held accountable for the risk exposure but have no visibility into the problem, he says.

“When senior leaders use ungoverned AI tools for business decisions, those decisions still have consequences, such as financial commitments, contract reviews, and data sharing,” he adds. “But there is no audit trail, no permissions model, or no way to reconstruct what happened or why.”

Part of the problem is that approved AI options often don’t meet the needs of users, Maloo says.

“AI policies alone aren’t enough; organizations need to pair governance with usability,” he adds. “If approved AI tools don’t meet the pace of business, employees at every level, including leadership, will find their own solutions. Successful organizations will be those that make the secure path the easiest path.”

IT leaders can’t solve the problem with more governance, he notes. “Policies and restrictions slow shadow AI down, but they don’t stop it, especially when the people using it are senior enough to absorb the disciplinary risk,” Maloo adds. “What CIOs can do is focus on providing tools that grant users full access to the necessary systems and data, eliminating the need to choose between a capable but ungoverned tool and a safe but limited one.”

Speed over security

The TrustedTech data echoes a June report from employee monitoring software vendor Teramind, which found that more than two-thirds of C-level executives prioritize speed over security when using AI tools, notes Nik Kale, a principal engineer and product architect at Cisco, and member of the Coalition for Secure AI.

In addition, the Teramind report found that two-thirds of enterprise AI activity runs through personal accounts on platforms for which the company already owns licenses, he notes.

“People are paying for the governed version and using the ungoverned version of the same product, so the problem isn’t the tools,” he says. “The approved path is slower, buried in procurement, or disconnected from where the work actually happens, and speed wins every time under a deadline.”

The problem then isn’t with the AI tools, but with the friction involved, he says. “People aren’t going around the front door because the room is locked,” Kale adds. “They’re going around it because the front door is slower.”

In many cases, the use of shadow AI exposes a couple of shortcomings in enterprise processes, adds Matthew Scavetta, chief technology innovation officer at IT solutions provider Future Tech Enterprise.

Many organizations don’t do a good job of making employees aware of the AI tools available to them, he says, and many organizations don’t offer training on the sanctioned applications, which drives users to pick products they are familiar with.

“If you don’t solve problems for people quickly or make people aware of which tools they can use safely, they will find a workaround,” he adds. “AI tools are no different than anything else.”

Shadow AI use by executives puts IT leaders in an incredibly difficult position, he says.

“CIOs, in particular, are under more and more pressure each year to keep up with what’s possible as tech influencers keep preaching about the potential of these tools,” Scavetta says. “CEOs and board members are constantly getting swept up in the hype; meanwhile, there are more and more case studies coming out showing how little ROI some organizations have realized. It’s a never-ending game of balancing possible with practical.”

In many teams, dependency security still happens after the most important trust decision has already been made. A package is added, the lockfile changes, the feature moves forward, and only later does the pipeline ask whether the application should have trusted that code in the first place.

That workflow made sense when dependency security was mostly viewed as a compliance check. Run a scanner. Produce a report. Fail the build if the risk crosses a threshold. Let someone decide what to do next.

But the modern Node.js ecosystem has changed. The risk no longer begins in CI. It begins earlier, at the moment a developer decides to trust a package.

That is why the next phase of Node.js security cannot be limited to better pipeline enforcement. It has to move closer to the developer workflow, before dependencies become part of the application, before a pull request becomes someone else’s problem, and before a build log becomes the first moment anyone realizes that something important has changed.

Every install is a trust decision

The npm ecosystem is built on trust at an enormous scale. Every install is a trust decision. Every transitive dependency extends that decision to maintainers, packages, scripts, release pipelines, and infrastructure the application team may never inspect directly. This model gave JavaScript its incredible velocity. It also created one of its deepest security weaknesses.

Recent npm supply chain incidents show why this matters. In March 2026, malicious Axios versions were published to npm through a compromised maintainer account. Microsoft later described how those packages attempted to retrieve a second-stage payload during installation. In May 2026, TanStack published a postmortem explaining that 84 malicious versions across 42 npm packages were published through a legitimate release pipeline after an attacker abused GitHub Actions behavior and runner trust boundaries. Security researchers also reported broader Mini Shai-Hulud activity across the npm ecosystem in May, including hundreds of malicious package versions published in a short period.

Not every one of these incidents is a traditional CVE. Some are malicious package compromises. Some involve CI/CD credential theft. Some involve maintainer or pipeline compromise. But they all point to the same larger issue: dependency risk is now part of everyday software engineering, not something that can be pushed entirely to a downstream security process.

The problem is not the scanner. It is the handoff.

Ubiquitous dependency risk changes what developers need from security tooling.

The problem is not that teams lack scanners. Many organizations already run security checks in CI. The problem is that the output of those checks often arrives too late and speaks the wrong language for the person expected to act on it.

A pull request fails. A long vulnerability report appears. The report may be technically accurate. It may contain the right advisory IDs, affected versions, dependency paths, severity labels, and references. But the developer still has to comb through the output and reconstruct the actual engineering decision from the evidence provided.

That reconstruction is rarely simple. The developer has to understand which package introduced the issue, whether the vulnerable dependency is direct or transitive, whether the fix is actually within the application team’s control, and whether the recommended version is safe to adopt. They also have to determine whether the dependency is used in production or only during development, whether the update might break the application, and whether the fix belongs in the current pull request or requires separate engineering work.

That uncertainty is where security work often slows. The scanner has detected risk, but the developer has not been given a clear path from detection to decision.

Security needs to move closer to engineering judgment

This is not a criticism of scanning. Scanning is necessary. CI enforcement is necessary. Centralized security platforms are necessary. But they are not sufficient, because they often operate after the trust decision has already been made.

The real architectural question is this: where should dependency security live in the software development life cycle?

If it lives only in CI, it becomes an interruption. If it lives only in dashboards, it becomes someone else’s queue. If it lives only in periodic audits, it becomes a backlog. But if it lives at the moment a dependency is introduced, upgraded, or reviewed, it becomes part of engineering judgment.

That shift matters because modern JavaScript development is becoming faster than human review can comfortably handle. Developers no longer add dependencies only by reading documentation and choosing libraries manually. AI coding assistants can suggest packages, generate install commands, modify package files, and rewrite code around third-party APIs. Agentic development workflows can make dependency changes as part of broader automated refactors.

AI makes the trust boundary harder to see

That acceleration is useful. It also changes the risk model.

When a human developer adds one package, the team can review the decision. When a coding agent modifies several dependencies as part of a larger task, the trust boundary becomes harder to see. The package file changes, the lockfile changes, the application still runs, and the pull request may look like a normal feature update. But the real security question may be hidden inside the dependency graph.

This is where Node.js teams need a different mental model.

Dependency adoption should not be treated as a small implementation detail. It should be treated as an architectural decision with security consequences. A new package is not just code reuse. It is a new trust relationship.

That does not mean developers should stop using packages. The npm ecosystem exists because reuse works. Most teams cannot and should not build everything themselves. But convenience should not erase visibility. If a dependency becomes part of the application, the team should understand what was added, what changed in the lockfile, what risk comes with it, and what action is available if something is wrong.

Developers need confidence, not just reports

The same applies to remediation. Developers do not want a wall of vulnerability text. They want confidence. They want to know what action reduces risk, what version should be targeted, whether the change is safe, and whether the fix is actually under their control. A vulnerability report that leaves the developer uncertain may satisfy a process requirement, but it does not necessarily improve the speed or quality of remediation.

That is the gap many teams feel today. Security tools are often very good at saying, “There is a problem.” They are less consistent at helping the developer answer, “What should I do next?”

This is the broader problem I have been exploring through CVE Lite CLI, now an OWASP project. The point is not that one command-line tool solves Node.js security. It does not. The larger idea is that dependency security has to move closer to the developer’s moment of decision. A useful developer-side security workflow should not merely report that risk exists. It should help the engineer understand whether the issue is in their control, what change is available, and whether the fix actually reduces risk.

The future is decision support, not just detection

That distinction is important. The future of Node.js security is not just more detection. It is better decision support.

Security teams still need policy. Enterprises still need dashboards. CI still needs gates. But developers need something more immediate: a way to reason about dependency risk while the code is still fresh in their mind. That is where the ecosystem has to evolve.

We already accept that testing belongs close to development. We accept that linting belongs close to development. We accept that formatting, type checking, and build validation belong close to development. Dependency security should follow the same path. It should not be treated as a mysterious report that appears at the end of the process. It should become part of the normal rhythm of engineering work.

Before adding a package, developers should understand what trust relationship is being introduced. Before accepting an AI-generated dependency change, they should inspect what entered the graph. Before merging a pull request, teams should understand whether a vulnerability is direct, transitive, fixable, or blocked by another package. And before treating a CI failure as noise, organizations should ask whether the workflow is giving developers enough information to act confidently.

Node.js security will be won, or lost, before CI runs

The Node.js ecosystem will not become safer by slowing down all development. That is unrealistic. It will become safer when security work is placed where developers can actually use it.

The next generation of Node.js security will be won or lost before CI runs.

It will be won when dependency decisions are still small enough to understand, fresh enough to review, and close enough to the developer for action to feel natural.

That is the shift teams need to make now. Not from insecure to secure in one step, but from late detection to earlier judgment. From vulnerability reports to engineering decisions. From trusting packages by habit to understanding trust as part of software design.

Every day this past week I did something I suspect millions of other people also did: I stared at an LLM model picker and wondered which one I was supposed to want.

OpenAI just released ⁠GPT-5.6 Sol, Terra, and Luna. Sol is the flagship. Terra offers much of its intelligence for less money. Luna is cheaper still. Anthropic released ⁠Claude Sonnet 5 at the end of June and Opus 4.8 the month prior, with a little Fable 5 emerging in between. Meanwhile, Google, which seemed to be winning the model wars a few months ago, is now getting shade from Gergely Orosz, who ⁠argues that Gemini has slipped outside the top tier for software development and has been out of the major model release game for eons (May 19).

Perhaps Orosz is right. Perhaps he’ll be wrong again in six weeks. Honestly, it’s exhausting.

I use ChatGPT and Claude constantly and still have no principled idea which model to choose most of the time. I tend to click whatever looks like the biggest, most expensive option because I don’t know what I’m giving up by choosing something smaller. “Instant” sounds dangerously unserious. “Thinking” sounds expensive but powerful.

A quick survey of my LinkedIn crowd suggests others also feel my “WHICH MODEL???” pain. More importantly, I suspect most enterprises do, too.

A model doesn’t rot

Before getting carried away, however, it’s worth considering whether any of this model churn actually matters. After all, a model doesn’t rot. The model an enterprise put into production in March performs just as well in July as it did when the company selected it. “Obsolete” generally means that something better now exists, not that the deployed model suddenly stopped summarizing insurance claims or classifying support tickets. (In other words, once you have something working, the idea that “but maybe Opus 200.2 is better!” is really a FOMO problem, not a performance issue.)

Most enterprise workloads don’t live at the frontier anyway. Extraction, summarization, classification, document comparison, and customer-service assistance often work perfectly well with smaller, cheaper models. OpenAI’s own pitch for the trio of GPT-5.6 models isn’t simply that Sol is better. It’s that ⁠Terra and Luna deliver different combinations of intelligence, latency, and cost. Luna, the cheapest tier, nearly matches the previous generation’s peak performance at less than half the estimated cost, according to OpenAI.

The practical question, of course, is where to start. An enterprise can’t test every model, every reasoning setting, and every price tier before doing any work. So here’s my advice (which I don’t follow in my own work, but I’m not defining enterprise strategy and can be a little price-insensitive). Start with the cheapest credible model that appears capable of the task. Give it a representative set of real examples and, before you start testing, define what counts as good enough. If it passes, stop. If it fails, move up a tier or try a model with strengths better suited to the work.

That sounds almost offensively simple, but it reverses the way many people, including me, use these products. We start with the biggest model because we’re afraid of what we might lose. Enterprises should start lower and require evidence before paying for more intelligence.

There are exceptions, of course. For genuinely difficult work, such as autonomous coding, complex research, or high-stakes reasoning, beginning with a frontier model may save time. But even then, the goal should be to establish a quality ceiling, then test whether a cheaper model can meet it. It’s changing the question from “which model is best?” to “what is the least expensive model that reliably clears the bar for this job?”

For many workloads, that price improvement matters more than a few extra benchmark points. ⁠As I argued back in 2023, following AI hype doesn’t help anyone. If your model strategy depends on whichever benchmark screenshot is circulating on X this week, you don’t have a strategy. Not a viable one, anyway. Pick a model and ignore the noise.

Except, of course, when that noise suggests a serious signal.

Sometimes better really is better

Frontier improvements aren’t always incremental, making it advantageous to consider an upgrade. Coding is the obvious example. There’s a significant difference between a model that suggests the next few lines of code and one that can inspect a repository, plan a change, use tools, run tests, discover its own mistakes, and keep working for an extended period. That isn’t merely a nicer autocomplete experience. It can reorganize a development workflow.

This is why enterprises can’t simply standardize on an 18-month-old model and declare victory. In some areas, particularly software development and other agentic work, better models can unlock compounding productivity. A model that reliably completes 80% of a bounded task rather than 50% may justify an entirely different division of labor between humans and machines.

Still, that upgrade isn’t free.

Models differ in how they interpret instructions, call tools, manage context, refuse requests, and fail. Prompts and scaffolding tuned for one model can regress when moved to another. Or costs can explode. As one of my Oracle colleagues discovered just this week, running the same tasks in GPT 5.6 was orders of magnitude more expensive than 5.5. The API change may be trivial, but the revalidation and implications are not.

This leaves enterprises caught between two bad options. They can freeze and potentially miss out on meaningful improvements or chase every release and repeatedly test production systems on faith. What to do?

Stop making model bets

The answer is to stop making LLM bets and start making job-to-be-done bets. Stop asking which model is fastest. Instead, figure out what work you are trying to improve. What does a good result look like? How much latency and cost can the workflow tolerate? How wrong can it be before a human must intervene? Once those questions have answers, model selection becomes less opaque.

A difficult code migration may justify GPT-5.6 Sol or Claude Sonnet 5. A repetitive classification task may work just as well with Luna or another smaller model. A regulated workflow may require a model or deployment option that offers particular data controls. Sometimes the correct model is no LLM at all, like when I’m writing this post. Sorry, AI vendors! (At least you won’t get blamed for my mistakes.)

This is where evaluations become the center of enterprise AI strategy. ⁠As I’ve said before, most companies don’t have an AI quality problem so much as an AI measurement problem. Hence, a private evaluation suite built from real company work is the only leaderboard that matters. Does the new model materially improve quality? If so, use it! Does it reduce cost or latency? Again, that’s your free pass to adoption. Does the improvement justify the expense and effort of revalidation? If yes, continue.

Make model releases boring

As important as the model is, keep in mind that AI success always comes back to your company’s data, your company’s workflows, your company’s integrations, etc. That’s the ⁠dull reality behind sexy AI. Retrieval, memory, governance, data quality, observability, and feedback loops aren’t as exciting as a new model launch, but they’re what ultimately make AI truly work.

Again, when it’s time to consider something new, the principle should be to default to the least expensive model that reliably passes your evaluations. Only escalate harder tasks to more capable models when measurement shows that the premium pays. Tip: Make this invisible to employees so that the system routes to the best model for a particular prompt. As dbt Labs’ Jon Lewis expresses it, “The best model is ‘Auto’ and I won’t hear anyone say otherwise.” OpenAI’s own ⁠migration guidance recommends testing models on representative tasks, including trying a lower reasoning level rather than automatically cranking everything to the maximum.

As for me, I’ll probably keep clicking the shiniest option. I don’t have a formal evaluation suite for InfoWorld columns, and the marginal cost is a subscription I already pay. Enterprises don’t get that excuse.

Gen AI spending is moving beyond the familiar software model of seats, licenses, and pilots. As AI shifts from copilots to embedded workflows and autonomous agents, one user request can trigger multiple model calls, retrieval steps, retries, orchestration layers, and infrastructure events. A tool that looks affordable in pilot may behave very differently once connected to production systems or allowed to act with less human supervision.

According to Michael Corrigan, CIO of World Insurance Associates, AI introduces a fundamentally different cost model — one that’s usage driven, non-linear, and tightly coupled to business activity. “Success requires shifting from traditional IT budgeting to FinOps-style discipline where consumption, value, and governance are actively managed in real time,” he says.

Here are five ways CIOs can build that discipline before AI costs spiral.

Forecast AI by workflow, not by user

At World, a top 25 insurance broker with about 3,000 employees across roughly 300 locations, AI use falls into three broad categories, Corrigan says. One is broad tools, such as copilots. Another is embedded AI inside SaaS platforms. And the third is bespoke AI built around specific workflows and manual processes.

“The bespoke is the area that’s growing the most right now,” he says. “And that’s where the model, from a cost perspective, has really been shifting from a license seat cost to a token consumption or token burn cost, or even a hybrid.”

Seat-based pricing is relatively easy to forecast whereas consumption-based AI isn’t. Costs may depend on prompt complexity, output length, model choice, workflow design, and whether the system calls a model once or many times in the background.

World tries to manage that uncertainty by defining the business problem, success criteria, and expected operational improvement upfront. Pilots help estimate consumption before scaling, but Corrigan says they don’t remove the ambiguity.

“We’ll try our best in the pilot to understand what the consumption rate is, what the token burn rate is,” he says. But once a consumption-based workflow goes into production, he adds, an estimate is put into place. That estimate is informed, but still rough.

Elmer Morales, founder and CEO of koder.com, an agentic AI coding startup, says CIOs should think less about headcount and more about workflow mechanics. Agentic AI costs are driven by the number of decisions an agent makes, how often it retrieves external data, how much context it carries, and how many systems it touches.

“CIOs should start by mapping workflows, not necessarily users,” he says. “The relevant variable isn’t going to be the headcount but how many decisions an agent makes per task.”

Model the failure path, not just the happy path

Pilots can mislead because they often test the cleanest version of an AI workflow. Morales says many enterprises model agentic AI costs around the happy path: the user gives a clear prompt, the system understands the request, the agent completes the task, and the process ends. Production is messier.

“They generally don’t model for situations where the agent is going to need to go back and check its work and redo things,” Morales says. “A lot of times, agents are wrong, either because they hallucinate or they understood the problem incorrectly.”

In an agentic workflow, the system may check its work, call another tool, retrieve more data, or redo a step. While that may improve quality, it also adds cost.

The difference between copilots and agents is central. A copilot interaction is often one prompt and one response. An agentic workflow may involve agents moving through a decision tree, executing tasks in sequence or in parallel, and calling sub-agents or external systems along the way. “By the time it’s achieved the original goal, the agent might have made 50 or 100 model calls, compared with a single call for a traditional copilot prompt,” Morales says.

That’s why CIOs should require teams to model the failure path before production, like how many retries are allowed, how much context is resent, which tools can be called, when a human should intervene, and what happens when the agent can’t complete the task.

Build cost controls into the architecture

Traditional FinOps practices still matter, but AI requires more than retrospective dashboards and chargebacks.

According to Pavan Madduri, senior cloud platform engineer at industrial supply company Graigner, looking backward at usage data, as traditional FinOps often does, can be too late. Costs are shaped by prompt design, model selection, agent behavior, orchestration choices, and runtime loops.

“Dashboards or chargebacks, those are historical accounting,” he says. “The money’s already gone.” For AI, he argues, cost controls need to be embedded into the architecture. That includes hard token caps, retry-depth limits, maximum runtime limits, workload prioritization, background-job throttling, and cluster-level controls that prevent runaway consumption.

“The real FinOps means you need to have the cost constraints embedded into your architecture framework,” Madduri says.

Those controls also extend to infrastructure. Expensive GPUs may sit warm between jobs because systems need capacity available when inference demand arrives. Teams may pass huge schemas, databases, or thousands of lines of code into frontier models when a smaller or more focused prompt would do.

Enterprises should also adopt event-driven autoscaling, Madduri says. “Use tools like KEDA to scale GPU nodes down to zero the moment inference demand drops, so teams only pay for the windows when the silicon is actively crunching tokens.”

Corrigan says World uses rate limits, spend limits, alerts, and approval gateways for consumption-based tools. When users approach token consumption limits, automated alerts allow IT and the business to review whether the continued spend is justified.

“If it’s not meeting the success criteria we expected, you have to have the control in place to say we’re going to move on or kill that process,” Corrigan says.

Route work to the right model

CIOs can also reduce AI bill shock by avoiding a default assumption that every task requires the most powerful model available. While some tasks need advanced reasoning, many others don’t. A simple support ticket, log-parsing task, or structured database transaction may be handled by a smaller or cheaper model. A complex architecture decision, legal analysis, or multi-step reasoning task may justify a more powerful one.

“Choosing the right model for the right prompt and right question — that’s where you leverage the maximum from that model, and you can decrease the costing,” Madduri says. “If you default every single call to a frontier model, that’s architectural laziness.”

Morales makes a similar point. Not every step in an agentic workflow requires a top-of-the-line model. Model routing, he says, is the discipline of determining the best model for the task, and providing the relevant context when the model needs it.

According to Jim Olsen, CTO of enterprise software company ModelOp, CIOs should use the least expensive model that can accomplish the business goal. Using the biggest model for everything is easier, but expensive. “It’s like hiring the most expensive engineer to change a few colors in a website’s CSS, or visual styling,” he says. “You wouldn’t do that. You use the appropriate tools for the task.”

Tie consumption to business value

For Olsen, the deeper enterprise problem is AI value shock, not just bill shock. Spending $200,000 in a quarter on AI is justified if it produces $2 million in business value. The problem is spending heavily on use cases that don’t generate a meaningful return.

“Are you actually getting that return on investment, or are you just blowing tokens for something that’s not delivering the value to your business?” Olsen asks. Tracking token usage by user or department may show who consumed AI, but not whether the consumption mattered.

For most enterprise AI systems, Olsen says costs should be tied back to business use cases. A model may be used for HR document search, customer support, code review, problem resolution, or other functions. Each use case may draw on the same underlying models or agents, but the business value can be very different.

That’s why he argues that companies need an AI inventory, a record of which business workflows use which models, agents, providers, workflows, and systems. Without that inventory, enterprises can’t connect consumption to value.

Corrigan takes a similar approach from a governance perspective. At World, new AI ideas go through an intake process. Business users propose improvements, and IT, finance, operations, sales, and business stakeholders evaluate, prioritize, and monitor them from pilot through production.

That may be where the next stage of AI FinOps is heading, toward a clearer understanding of which AI consumption deserves to scale, not just to lower bills. So the question, as Olsen puts it, isn’t whether someone used a million tokens. It’s what are they using them for.

Shadow IT has long been a major problem for IT leaders, but the biggest problem may be coming from the executive suite’s hunger for unsanctioned AI.

Nearly two-thirds of senior decision-makers admit to using unapproved AI tools, compared to just 31% of lower-level employees, according to a survey by Microsoft solutions partner TrustedTech.

The use of shadow AI is prevalent among senior executives even though three in four employees acknowledge security or data privacy risks related to the practice.

“Most shadow AI users are not ignorant of the risk,” TrustedTech says in a white paper. “They are deliberately choosing to use these tools anyway. This is not a training issue. It is a culture, incentives, and alternatives issue.”

In many cases, the problem is driven by a lack of approved tools, the report adds.

“People use shadow AI because what their employer hands them is worse than mainstream AI tools, or because nothing has been approved in the first place,” the report says. “That doesn’t change until the sanctioned tools are genuinely worth using.”

A question of authority

The use of shadow AI by CEOs and other C-suite executives can create major problems for CIOs, CISOs, and other IT executives because they may not have the authority to put the kibosh on it.

It also presents a challenge for IT leaders to provide the AI tools that employees and executives want to use.

When executives use shadow AI, CIOs are in a difficult position, because governance only works when it’s modeled from the top, says Andy Nolan, VP of technology at TrustedTech.

“If senior leaders bypass approved AI tools or policies, it sends an implied message that speed matters more than security and compliance,” he adds. “Employees notice that behavior, and it becomes much harder to ask the rest of the organization to follow standards that leadership isn’t following themselves, first.”

Another major problem is that executives often work with highly sensitive information, including financial data, strategic plans, intellectual property, and customer information, he notes.

But CIOs and CISOs also can’t solve the problem by becoming the AI police in every situation, Nolan says, because their role is to help the business innovate safely.

“That requires executive alignment, clear governance, and providing secure AI tools that people actually want to use,” he adds. “When leadership embraces those solutions, the rest of the organization is almost sure to follow.”

All risk, no reward

The use of shadow AI by senior executives puts CIOs and CISOs in an impossible position, agrees Amit Maloo, CISO at AI procurement provider Ivalua. CIOs and CISOs are held accountable for the risk exposure but have no visibility into the problem, he says.

“When senior leaders use ungoverned AI tools for business decisions, those decisions still have consequences, such as financial commitments, contract reviews, and data sharing,” he adds. “But there is no audit trail, no permissions model, or no way to reconstruct what happened or why.”

Part of the problem is that approved AI options often don’t meet the needs of users, Maloo says.

“AI policies alone aren’t enough; organizations need to pair governance with usability,” he adds. “If approved AI tools don’t meet the pace of business, employees at every level, including leadership, will find their own solutions. Successful organizations will be those that make the secure path the easiest path.”

IT leaders can’t solve the problem with more governance, he notes. “Policies and restrictions slow shadow AI down, but they don’t stop it, especially when the people using it are senior enough to absorb the disciplinary risk,” Maloo adds. “What CIOs can do is focus on providing tools that grant users full access to the necessary systems and data, eliminating the need to choose between a capable but ungoverned tool and a safe but limited one.”

Speed over security

The TrustedTech data echoes a June report from employee monitoring software vendor Teramind, which found that more than two-thirds of C-level executives prioritize speed over security when using AI tools, notes Nik Kale, a principal engineer and product architect at Cisco, and member of the Coalition for Secure AI.

In addition, the Teramind report found that two-thirds of enterprise AI activity runs through personal accounts on platforms for which the company already owns licenses, he notes.

“People are paying for the governed version and using the ungoverned version of the same product, so the problem isn’t the tools,” he says. “The approved path is slower, buried in procurement, or disconnected from where the work actually happens, and speed wins every time under a deadline.”

The problem then isn’t with the AI tools, but with the friction involved, he says. “People aren’t going around the front door because the room is locked,” Kale adds. “They’re going around it because the front door is slower.”

In many cases, the use of shadow AI exposes a couple of shortcomings in enterprise processes, adds Matthew Scavetta, chief technology innovation officer at IT solutions provider Future Tech Enterprise.

Many organizations don’t do a good job of making employees aware of the AI tools available to them, he says, and many organizations don’t offer training on the sanctioned applications, which drives users to pick products they are familiar with.

“If you don’t solve problems for people quickly or make people aware of which tools they can use safely, they will find a workaround,” he adds. “AI tools are no different than anything else.”

Shadow AI use by executives puts IT leaders in an incredibly difficult position, he says.

“CIOs, in particular, are under more and more pressure each year to keep up with what’s possible as tech influencers keep preaching about the potential of these tools,” Scavetta says. “CEOs and board members are constantly getting swept up in the hype; meanwhile, there are more and more case studies coming out showing how little ROI some organizations have realized. It’s a never-ending game of balancing possible with practical.”

I attended several major technology conferences in 2025 where the first AI agents embedded in enterprise SaaS platforms were announced. Some of these agents showed promise and a glimpse into the future of work, while others looked like natural language extensions of a platform’s existing functionality.  

At the end of 2025, Anthropic and OpenAI launched new AI models and code-generating capabilities. More developers tried vibe coding, and some platforms launched spec-driven development capabilities. By February 2026, even The New York Times reported that the AI disruption had arrived, noting that code generators were building “apps that may be flawed, but credible.”

Wall Street investors took notice of the code-generation improvements and other disruptive factors, driving a selloff in SaaS stocks, now referred to as the “SaaSpocalypse.” Part of their concern stemmed from the belief that CIOs would use AI to write software that would replace SaaS solutions.

AI innovations from SaaS and solution providers

But I thought differently and wrote a response in my article asking whether AI is the end of SaaS as we know it. CIOs might use AI to accelerate application modernization, but I doubt they would replace their ERP, CRM, and even smaller SaaS point solutions by building them.

Instead, I believed it would be SaaS companies that would take the most advantage of AI code-generation capabilities.

This hypothesis drove me to attend nine conferences this spring to see how SaaS companies were launching AI agents and defining a new future of work. I wrote eight articles on what CIOs need to know about data management, agile organizations, marketing, ERPs, critical process management, and other evolutions to plan for in the AI era.

Now, looking across all nine conferences, I can draw some conclusions about how AI agents are shaping the future of work. Here are my learnings and what CIOs need to consider when evaluating and deploying AI agents in the workplace.

Agentic, human-in-the-middle, or augmenting human?

SaaS companies have very distinct perspectives on the future of work, including the extent to which humans will play which roles and whether and how quickly we’ll see agentic, fully automated work.

For example, Atlassian proclaimed, “step into the future of human-AI collaboration,” while SAP unveiled “the autonomous enterprise.” Snowflake aimed to “make AI real for business,” while Appian targeted “serious AI built on process.”

These vendors’ marketers had to decide whether to lead with AI, people, or business in their messaging, but so must CIOs as they contemplate their AI strategies and how to get employees to fully adopt AI agents.

Some CIOs see a fully automated agentic AI as the future, with human-in-the-middle as a transitional phase as departments build trust in AI agents’ decision-making and automation capabilities.

Other CIOs see AI more as a tool that delivers productivity improvements by augmenting human decision-making capabilities. Many of these CIOs see human augmentation as essential to supporting critical thinking, innovation, and creativity.

Deloitte’s State of AI Report, published in January, provides a benchmark. It states that 36% of IT leaders expect at least 10% of their jobs to be fully automated in the next year, and 82% expect to reach that benchmark in three years.

Many organizations will have a mix of AI agents, choosing automation where reliability at scale is possible, but opting for human augmentation in operationally critical or customer-facing domains. But how CIOs position AI agents is not only an operational strategy; it’s also a cultural statement that shapes employees’ embrace of AI and whether detractors vocalize job-loss fears.

In the short term, it will also weigh in on which AI agents to use from different partners and which areas to build in-house.

Many options to test and deploy AI agents

Many solution providers are demonstrating significantly more AI agents this year. For example, SAP went from 40 Joule Agents in 2025 to over 200 in 2026. Three technology capabilities are fueling this significant growth:

• Adobe, Appian, Boomi, Cisco, Domo, Salesforce, SAP, Snowflake, and others offer data fabrics and data-pipeline capabilities to connect data sources outside the primary workflows supported by their platforms. Appian, Pega, Quickbase, and SAP also centralize business process automation, an important starting point for developing AI agents.  
• MCP servers enable integration and communication between AI agents and are used to facilitate multistep agentic workflows. Virtually all the companies announcing major investments in AI agents are also announcing MCP integration capabilities and related partnerships.
• Solution providers are not just using AI code-generating capabilities; many are launching their own AI agent development tools. The first beneficiaries of these development tools are the solution providers themselves and their integration partners, who use them to accelerate the development of AI agents and make them available to customers.

The result is that CIOs will have many options about which agents to test, but will have to dedicate analysts to understand the capability, cost, and compliance trade-offs. Additionally, expect AI agent capabilities to evolve significantly over the next few years, so CIOs should continuously revisit their decisions regarding deployed AI agents, focusing on performance, benefits, and ROI.

CIOs should also watch for signs of shadow AI and employee confusion about which AI agents to experiment with on different platforms. The AI strategy should include a transparent, defined process for selecting, reviewing, evaluating, procuring, deploying, driving adoption, monitoring, and collecting end-user feedback around AI agents.

AI development capabilities for engineers and citizen builders

The apparent ease-of-use of AI code generators may lead some engineering teams to build AI agents rather than buy them from SaaS providers. But CIOs should quickly realize that coding is just one step in developing AI agents, and that aggressively pursuing a build strategy can lead to AI debt and increased AI costs.

DevOps teams can code AI agents using tools such as Claude, Codex, Lovable, and Replit — a do-it-yourself approach. Some SaaS companies are providing an alternative, with AI agent development tools that leverage the data, infrastructure, and governance baked into their platforms. Many of these development tools offer flexibility, allowing developer teams to select AI models and development environments.

Examples of new and enhanced AI development tools I saw at conferences this quarter include:

• Appian Composer and Agent Studio
• Atlassian Rovo Dev
• Boomi Companion
• Cisco Cloud Control Studio
• Domo App Catalyst
• Pega Infinity Studio
• Quickbase Pave
• Snowflake CoCo
• SAP Joule Studio.

I also reviewed Nutanix Agentic AI, a platform-as-a-service for accelerating the deployment of agentic AI workloads, and Adobe Firefly AI Assistant for creatives.

These development tools can target different audiences. Some look like low-code development tools targeted at software developers, whereas others are no-code and enable citizen developers, i.e., businesspeople, to develop applications and agents. Additionally, some of these tools support spec-driven development and generate artifacts such as product requirement documents (PRDs), data models, and testing capabilities.

Before commissioning AI development for apps and agents, CIOs should sponsor proofs of technical, data, modeling, security, and governance capabilities.

The context layer powering AI agents

Between AI agents and the enterprise’s intelligence, including structured data sources, defined business processes, and agent interactions (both human-to-agent and agent-to-agent), lies an evolving “context layer.”

This layer refers to the enterprise knowledge that AI agents draw on when evaluating signals and recommending or taking actions. Context may include a knowledge graph, a semantic layer, cleansed document repositories, and other knowledge bases.

The context layer, skills, tools, out-of-the-box agents, and governance capabilities are some areas to review where solution providers differentiate. Some examples: 

• Many support the Open Semantic Interchange, and some brand their context layers, such as the Atlassian Teamwork Graph, Boomi Knowledge Hub, and the SAP Knowledge Graph.
• Some are branding their guardrails, such as Adobe’s AI Brand Intelligence, Appian’s Private AI, and Quickbase AI Control Center.
• To manage AI agents at scale, some are extending the notion of data catalogs and other governance tools to the AI domain with products such as Boomi Connect, SAP AI Agent Hub, and Snowflake Horizon Catalog.

CIOs should recognize that while solution providers will compete on capabilities, the real “secret sauce” of the context layer lies in the company’s trusted data, well-defined business processes, and employee adoption of AI agents.

Conversational user experiences and coworkers

AI agents use the context layer, but also tap into skills, which encode the procedures they can follow, and tools, which prescribe the actions they can take. Before AI agents are ready to pilot, their governance, including permissions, approval gates, and other guardrails, must be defined. Other capabilities to look for when defining AI agents include orchestration, testing evals, and observability.

In 2025, many solution providers bolted on AI agents to their existing user experiences. This year, many solution providers showcased new conversational user experiences that employees can use instead of traditional ones built with forms, flows, reports, and static dashboards. Conversational user experiences are where AI agents and people come together, whether it’s human-in-the-middle or human augmentation.

Solution providers also grouped their AI agents into assistants or coworkers. For example, Adobe CX Coworker illustrates human augmentation, helping marketers manage campaigns with prompts and monitor their performance. SAP launched Joule Assistants across several business functions, including finance, human capital, supply chain, and customer experience. Other assistants, such as Appian AI Copilot, Atlassian Rovo, Cisco AI Assistant, Nutanix NIVA, and Snowflake CoWork, offer AI-first user experiences to assist different end-user types.

CIOs should demo these AI-first user experiences to glimpse the future of work.

Developers are already getting used to these experiences through code generators and vibe coding tools. Now, similar capabilities are being tailored across all business functions. CIOs should ramp up their change management programs to accelerate the adoption of these AI capabilities.

Solution providers are showcasing AI capabilities that can help CIOs reshape their businesses. But in Q2, there were only a few examples of how AI can help CIOs drive growth, evolve business models, or embed AI into customer-facing products. I expect to see a wave of further AI innovations that will go beyond productivity improvements and efficiencies and help CIOs pursue growth-driving digital transformation strategies.  

Sacolick travelled to conferences mentioned in this article as a guest of Adobe, Appian, Atlassian, Domo, Nutanix, SAP, and Snowflake. In addition, he was hired by Quickbase to speak at its conference.

Deluxe may have prevailed against the odds when it successfully migrated from a 50-plus-year-old mainframe recently.

The company was able to move from it to a cloud environment in about 12 months without a major hitch, and while AI did some of the heavy lifting. The save will amount to about $4.9 million a year by retiring old hardware and software, cutting labor costs, and consolidating IT resources.

Deluxe, based in Minneapolis and traditionally known as a check printer, has transformed itself into a payments IT provider in recent years. But it desperately needed to end its reliance on its ancient mainframe, says Yogaraj Jayaprakasam, the company’s chief technology and digital officer, pictured.

So AI played a huge role in the IT modernization project, he says, using it to rewrite the old mainframe code, generate documentation, and regenerate code test cases. The company also used an AI-powered test automation suite, as well as AI tools to help move assets to the new cloud environment.

While AI can’t do everything during mainframe migration, it can make the move easier, Jayaprakasam says. “The biggest lessons learned is AI is one of the missing tools in your transformation tool set,” he adds.

Failing projects

Deluxe’s mainframe migration earned it a 2026 CIO 100 Award for IT innovation and leadership, but it also seems to have bucked a recent trend. Many mainframe migration projects haven’t gone as planned, and Gartner recently predicted that more than 70% of mainframe exit projects that started in 2026 will fail to produce intended benefits because of an overreliance on gen AI.

Mainframe transformation projects tend to work better when they’re part of a larger business transformation rather than a one-off project using gen AI to do most of the work, says Gartner analyst Alessandro Galimberti.

“Generative AI and agentic AI are extremely powerful, but they also have their own limits,” he says. “With all these kinds of tools trying to convert code or somehow fit into a non-mainframe workload, we don’t really see a track record of success.”

Gartner also sees a declining interest in mainframe migration projects, Galimberti says. With mainframes getting support from several IT vendors, and with a general lack of migration success, many companies are choosing to keep many workloads on their existing big iron.

But mainframes also have a proven track record of very high uptime and backward capability, Galimberti says.

“If I’m a bank, a financial institution, or a transportation company, I need to run applications that are the core of my business,” he adds. “I need reliability, transactional integrity, and security, and these applications have a low change rate over the years because they map very stable business processes.”

The Gartner prediction makes sense to John McKenny, senior VP and GM of Intelligent Z optimization and transformation for mainframe support vendor BMC Software.

“With organizations thinking about mainframe exits, the expected benefits they’re looking for are usually pretty straightforward,” he says. “They think, ‘It’s going to be lower cost, I’m going to get equal or better capabilities, I should be more agile.’  The reality is those outcomes rarely show up at scale.”

Mainframe migration is possible, but the successful projects tend to be small scale, McKenny adds. He was on a recent call about a failed migration project in Europe, with a large bank cancelling the project at the end of 2025 and recommitting to the mainframe as a strategic platform.

“I’ve never seen a large-scale mainframe migration project finish under budget, ever,” he says. “Most of the projects I hear about fail outright.”

Trying again

Like some organizations that Gartner has observed, Deluxe tried to move away from its mainframe several years ago, but the project failed, Jayaprakasam says. Yet the company took the steps it needed this time to ensure the new migration succeeded.

While a mainframe migration isn’t for every organization, the latest move made sense for Deluxe, he says.

The mainframe, after all, was the backbone for a large portion of the company’s annual revenue, and interfaces with several top banks across North America. The modernization effort rebuilt core business processes and data, moving them from the mainframe to a modern cloud-native technology stack, including Salesforce, Mulesoft, and SAP S4/HANA.

While AI played a big part, there’s danger in overestimating the power of AI during a migration project, Jayaprakasam says, and organizations need to follow best practices for IT migration.

“If you minimize the importance of communication, risk planning, and business alignment because you have AI, you tend to fail,” he says. “But as long as you play all those cards and recognize AI was the missing piece to the puzzle, then you have a much better chance of winning.”

Deluxe also used a cross-functional tiger team to look at the various options available to accelerate reverse engineering, including AI tools from OpenAI, Anthropic, as well as GitHub Copilot throughout the project.

In addition, AI was useful to dig through the mainframe code and understand what needed to be updated, Jayaprakasam says. Organizations sitting on decades-old code often no longer have people who understand it.

“I always tell people that the code remembers what the organization forgot, because with people going and changing, people don’t remember what we wrote in the code, but the code remembers,” he adds. “The amazing tool that was missing before is we didn’t have an interpreter who understood what the code remembered. Now with AI, you have the interpreter.”

Generative AI is a kind of artificial intelligence that creates new content, including text, images, audio, and video, based on patterns it has learned from existing data.

Today’s generative models are typically built on foundation-model architectures such as large-language models (LLMs) and multimodal systems, enabling them to carry on conversations, answer questions, write stories, generate code, and produce images or videos from brief prompts.

Generative AI is different from discriminative AI, which draws distinctions between different kinds of input. Where discriminative AI answers questions like “Is this image of a rabbit or a lion?”, generative AI instead responds to prompts such as “Describe to me how a rabbit and lion look different from one another” or “Draw me a picture of a lion and a rabbit sitting next to each other” — and in both cases produces text or imagery that, while grounded in the AI’s training data, isn’t just a copy of something that already existed.

Just a few years ago, generative AI was once a novelty focused on chatbots and artistic image generation. Today, it has become a core enterprise technology, and powers everything from content creation and software development to customer support and analytics workflows. But with that power comes a new set of challenges — from model alignment and hallucination to governance and data-integration hurdles.

In this article, we’ll look at how generative AI works, explore how it has evolved into the foundation-model era, examine how to implement it effectively, and offer best practices for getting value out of it, today and in the future.

How does generative AI work?

For decades, early artificial-intelligence efforts often focused on rule-based systems or narrowly trained models that were built for one task at a time. While these efforts produced useful systems that could reason and solve human tasks, they were generally a far cry from sci-fi visions of thinking machines. Programs that could talk to people never seemed to get very far past the level of ELIZA, a “computer therapist” created at MIT in the mid 1960s; even Siri and Alexa after much fanfare were revealed to be fairly limited.

The big structural shift that gave birth to modern generative AI came with the concept of a transformer, first introduced in “Attention Is All You Need,” a 2017 paper from Google researchers.

Using a transformer architecture as a basis, you can build a system that derives meaning from analyzing long sequences of input tokens (words, sub-words, bytes) to understand how different tokens might be related to one another, then determines how likely any given token is to come next in a sequence, given the others. In AI lingo, we call these systems models. Because a model analyzes very large datasets and parameter counts, it can pick up on statistical patterns and knowledge implicitly embedded in the data.

This is all easier said than done. The process of adjusting a model’s internal parameters so it gets better at predicting the next token in sequences is called training. During training, the model repeatedly guesses the next token in a given sequence, compares its prediction to the actual one, measures the error, and updates its parameters to reduce that error across billions of examples. Over time, that process teaches the model the statistical relationships that will allow it to generate coherent language (or code, or images) later.

What is a foundation model?

You’ll often hear the word large used for transformer-based models of these types, like the LLMs we mentioned earlier. Large in this context refers to the large number of internal numerical values that the model adjusts during training to represent what it has learned, along with breadth and diversity of data used to train the model and the underlying compute resources powering this whole process.

This is in contrast with the narrow models of the earlier era of AI/ML, which werebuilt for one purpose and trained on a limited dataset. For instance, a spam filter may be very good at what it does, but it’s only trained on email data and all it can do is classify emails. Large models, by contrast, serve as what’s known as foundation models. They’re trained broadly on diverse data (text, code, images, or multimodal data) and then adapted or specialized for many downstream tasks.

These foundation models are the basis for most of the popular generative AI tools and services on the market today. They can be specialized in several ways:

• Fine-tuning: Giving a foundation model further training on a smaller, task-specific dataset
• Retrieval-augmented generation (RAG): Giving the model the ability to pull in external knowledge when asked a question
•  Prompt engineering: Tailoring a query so the model gives the sort of answers you’re looking for.

How do AI systems write computer code?

One of the surprising discoveries of the gen AI era was that in recent years was that foundation models trained on natural-language text can also, when fine-tuned with code examples, also write computer code — often better than many purpose-built systems. Still, it makes sense, when you think about it — after all, high-level computer languages are designed by humans and ultimately based on human language.

This 2023 InfoWorld article highlights how models like PaLM, LLaMA and other transformer-based systems fine-tuned on code repositories propelled this shift, but since AI giants like OpenAI have moved into this space. This all matters because code generation (or code-assisted productivity) has become a key enterprise use case of generative AI — perhaps the key use, given the industry’s enthusiastic adoption of it.

What are AI agents?

So far, we’ve been talking about chatbots, writing assistants, image-generation tools. They respond to prompts, output text or images, and then stop. A new category of tool called agentic AI goes further: it plans, executes, and in many cases learns as it works.

Because large models already understand language, code, and even structured data to some extent, they can be repurposed to generate not only descriptive text but operational instructions. For example: an agent might parse the intent “generate a sales-report”, then format internal calls like getData(salesDB, region=NA, period=lastQuarter), and then call an API, all by generating text that’s interpreted as instructions. The MCP framework standardizes the “language” of those instructions and the plug-points into tools and data so that the model doesn’t need bespoke integrations for each new workflow.

These kinds of autonomous agents have several enterprise use cases:

• Software automation: Agents that generate code, call unit tests, deploy builds, monitor logs and even roll back changes autonomously.
• Customer support: Instead of simply drafting responses, agents interact with CRM APIs, update ticket statuses, escalate issues, and trigger follow-up workflows.
• IT operations/AIOps: Agents monitor infrastructure, identify anomalies, open/close tickets, or auto-remediate based on defined rules and context from logs.
• Security: Agents may detect threats, initiate alerts, isolate compromised systems, or even attempt to manage threat containment — though this raises new risks.

How can you implement generative AI in the enterprise?

We’ve now touched on what generative AI can do. But how can you make it work reliably in your business. The difference between a pilot and full-scale deployment often comes down to systems, structure and governance as much as to models themselves. InfoWorld’s Matt Asay offers a deep dive into enterprise gen AI essentials, but here are some important points to keep in mind:

Choosing between API, open-source or custom fine-tuned models. One of the first major decisions for any enterprise project is: do you use a model via an API (e.g., from a vendor like OpenAI or Anthropic), deploy an open-source model internally, or build/fine-tune a custom model yourself? Each has trade-offs.

APIs offer speed and minimal setup, but may expose data, limit customization or accrue high cost — and will leave you at the mercy of your vendor. Open source allows internal control and may ease fine-tuning, but requires infrastructure, expertise, and support. Custom fine-tuning gives you the tightest alignment to your use-case, but lengthens time to value and increases risk.

Governance, data privacy and compliance. Deploying generative AI in an enterprise setting raises new governance, privacy and regulatory issues. For example: Who owns the data that’s ingested? How is proprietary data protected if you call a third-party API? What traceability exists for model outputs—a huge question for regulated industries? One useful framework is covered in “A GRC framework for securing generative AI” Data governance must adapt for the new era,  and new frameworks are evolving to help.

Human-in-the-loop review. Even the best models make mistakes and cannot simply be put on autopilot. You need a human-in-the-loop (HITL) process: real people need to review outputs, validate for bias, approve high-stakes content, and tune prompts or models based on feedback. Incorporating HITL checkpoints helps mitigate risk and improve overall quality.

Integration with existing systems and RAG pipelines. Retrieval-augmented generation, which we touched on earlier, connects foundation models into business workflows, systems, and enterprise data stores. RAG can bind LLMs to your organization’s internal knowledge bases, thereby reducing hallucinations (which we’ll discuss in a moment) and increasing the relevance of gen AI output.

What causes AI hallucinations?

Probably the biggest limitation of generative AI is what those in the industry call hallucinations, which is a perhaps misleading term for output that is, by the standards of humans who use it, false or incorrect.  

Every generative AI system, no matter how advanced, is built around prediction. Remember, a model doesn’t truly know facts—it looks at a series of tokens, then calculates, based on analysis of its underlying training data, what token is most likely to come next. This is what makes the output fluent and human-like, but if its prediction is wrong, that will be perceived as a hallucination.

Foundry

Because the model doesn’t distinguish between something that’s known to be true and something likely to follow on from the input text it’s been given, hallucinations are a direct side effect of the statistical process that powers generative AI. And don’t forget that we’re often pushing AI models to come up with answers to questions that we, who also have access to that data, can’t answer ourselves.

In text models, hallucinations might mean inventing quotes, fabricating references, or misrepresenting a technical process. In code or data analysis, it can produce syntactically correct but logically wrong results. Even RAG pipelines, which provide real data context to models, only reduce hallucination—they don’t eliminate it. Enterprises using generative AI need review layers, validation pipelines, and human oversight to prevent these failures from spreading into production systems.

What are some other problems with generative AI?

Generative AI has proven to be such a disruptive technology that’s stoking near-apocalyptic fears that it will result in a superintelligence that will enslave or destroy humanity. Meanwhile, in the present day, increasingly troubling reports of so-called AI psychosis are emerging, where people have mental health episodes triggered by the uncanny and sometimes sycophantic ways chatbots affirm whatever you talk to them about and try to keep the conversation going.

Compared to such existential questions, the following business-related problems may seem petty. But they’re real issues for enterprises considering investing in AI tools.

• Data leakage and regulatory risk. When a model is fine-tuned or prompted with sensitive information, that data may be memorized and unintentionally reproduced. Using third-party APIs without strict controls can expose proprietary or personally identifiable information (PII). Regulatory frameworks like GDPR and HIPAA require explicit governance around where training data resides and how inference results are stored.
• Prompt injection occurs when an attacker manipulates a model’s instructions—embedding hidden directives or malicious payloads in user input or external content the model reads. This can override safety rules, expose internal data, or execute unintended actions in agentic systems. Guardrails that sanitize inputs, restrict tool-calling permissions, and validate outputs are becoming essential.
• Copyright and content ownership. Many foundation models are trained on data scraped from the public internet, creating disputes over copyright and data provenance. Enterprises using generated output commercially need to confirm usage rights and review indemnity terms from vendors.
• Unrealistic productivity expectations. Finally, organizations sometimes expect generative AI to deliver instant productivity gains. The reality, it turns out, is more mixed. Enterprise adoption requires infrastructure, governance, retraining, and cultural change. The models accelerate work once properly integrated, but they don’t automatically replace human judgment or oversight.

The current generation of enterprise AI systems includes several layers of defense against these risks:

• Guardrails that constrain model behavior and filter unsafe outputs.
• Model validation frameworks that measure factual accuracy and consistency before deployment.
• Policy layers that enforce compliance rules, redact sensitive data, and log model actions.

These safeguards reduce—but don’t remove—the inherent uncertainty that defines generative AI.

GenAI: essential for the enterprise

Generative AI has evolved from a novelty into a core layer of enterprise technology. Foundation models and agentic systems now power automation, analytics, and creative workflows — but they remain fundamentally probabilistic tools. Their strength lies in scale and adaptability, not perfect understanding.

For organizations, success depends less on chasing model breakthroughs than on integrating these systems responsibly: building guardrails, maintaining oversight, and aligning them with real business needs. Used wisely, generative AI can amplify human capability rather than replace it.

Angular is a cohesive, all-in-one reactive framework for web development. It is one of the larger reactive frameworks, focused on being a single architectural system that handles all your web development needs under one idiom. While Angular was long criticized for being heavyweight as compared to React, many of those issues were addressed in Angular 19. Modern Angular is built around the Signals API and minimal formality, while still delivering a one-stop-shop that includes dependency injection and integrated routing.

Angular is popular with the enterprise because of its stable, curated nature, but it is becoming more attractive to the wider developer community thanks to its more community engaged development philosophy. That, along with its recent technical evolution, make Angular one of the most interesting projects to watch right now.

Why choose Angular?

Choosing a JavaScript development framework sometimes feels like a philosophical debate, but it should be a practical decision. Angular is unique because it is strongly opinionated. It doesn’t just give you a view layer; it provides a complete toolkit for building web applications.

Like other reactive frameworks, Angular is built around its reactive engine, which lets you bind state (variables) to the view. But if that’s all you needed, one of the smaller, more focused frameworks would be more than enough. What Angular has that some of these other frameworks don’t is its ability to use data binding to automatically synchronize data from your user interface (UI) with your JavaScript objects. Angular also leverages dependency injection and inversion of control to help structure your application and make it easier to test. And it contains more advanced features like server-side rendering (SSR) and static-site generation (SSG) within itself, rather than requiring you to engage a meta-framework for either style of development.

While Angular might not be your top choice for every occasion, it’s an excellent option for larger projects that require features you won’t get with a more lightweight framework.

Also see: Catching up with Angular 19.

Getting started with Angular

With those concepts in mind, let’s set up Angular in your development environment. After that, we can run through developing a web application with Angular. To start, make sure you have Node and NPM installed. From the command line, enter:

$ node -v
$ npm -v

Next, you can use the Angular CLI to launch a new app:

$ ng new iw-ng

You can use the defaults in your responses to the interactive prompts shown here:

Matthew Tyson

We now have a basic project layout in the new directory, which you can import into an IDE (such as VS Code) or edit directly.

Looking at the project layout, you might notice it is fairly lean, a break from Angular projects of the past. The most important parts are:

• src/main.ts: This is the main entry point. In older versions of Angular, this file had to bootstrap a module, which then bootstrapped a component. Now, it avoids any verbose syntax, calling bootstrapApplication with your root component directly.
• src/index.html: The main HTML page that hosts your application. This is the standard index.html that serves all root requests in a web page and contains the tag where your Angular component will render. It is the “body” that the “spirit” of your code animates.
• src/app/app.ts: The root component of your application. This single file defines the view logic and the component metadata. In the new “standalone” world, it manages its own imports, meaning you can see exactly what dependencies it uses right at the top of the file. (This is the root element that appears in src/index.html.)
• src/app/app.config.ts: This file is new in modern Angular and replaces the old AppModule providers array. It is where you configure global services, like the router or HTTP client.
• angular.json: The configuration file for the CLI itself. It tells the build tools how to process your code, though you will rarely need to touch this file manually anymore.

Here is the basic flow of how the engine renders these components:

1. The arrival (HTML): The browser receives index.html. The tag is there, but it’s empty.
2. The unpacking (JavaScript): The browser sees the tags at the bottom of the HTML and downloads the JavaScript bundles (your compiled code) from src/app/app.ts.
3. The assembly (Bootstrap): The browser runs that JavaScript. The code “wakes up,” finds the tag in the DOM, and dynamically inserts your title, buttons, and lists.

This flow will be different if you are using server-side rendering (SSR), but we’ll leave that option aside for now. Now that you’ve seen the basic architecture, let’s get into the code.

Developing your first web app in Angular

If you open src/app/app.ts (more info here) the component definition looks like this:

import { Component, signal } from '@angular/core';
import { RouterOutlet } from '@angular/router';

@Component({
  selector: 'app-root',
  imports: [RouterOutlet],
  templateUrl: './app.html',
  styleUrl: './app.css'
})
export class App {
  protected readonly title = signal('iw-ng');
}

Before we dissect the code, let’s run the app and see what it produces:

$ ng serve

You should see a page like this one at localhost:4200:

Matthew Tyson

Returning to the src/app.ts component, notice that there are three main parts of the definition: the class, the metadata, and the view. Let’s unpack these separately.

The class (export class App)

Export class App is vanilla TypeScript that holds your component’s data and logic. In our example, title = signal(‘iw-ng’) defines a piece of reactive state. Unlike older versions of Angular where data was just a plain property, here we use a signal. Signals are wrappers around values that notify the template precisely when they change, enabling fine-grained performance.

The metadata (@Component)

The @Component decorator tells Angular it is dealing with a component, not just a generic class. There are several elements involved in the decorator’s communication with the engine:

• selector: 'app-root': Defines the custom HTML tag associated with any given component. Angular finds in your index.html and renders the component there.
• imports: In the new Angular era, dependencies are explicit. You list exactly what a component needs (like RouterOutlet or other components) here, rather than hiding them in a separate module file.
• templateUrl: Points to the external HTML file that defines the view.

The view (the template)

This is the visual part of the component, defined in app.html. It combines standard HTML with Angular’s template syntax. (JSX handles this part for React-based apps.)

We can modify src/app/app.html to see how these three elements work together. To start, delete the default content and add the following:

Hello, {{ title() }}

The double curly braces {{ }} are called interpolation. Notice the parentheses in title(). We are reading the “title” signal value by calling its function. If you were to update that signal programmatically (e.g., this.title.set('New Value')), the text on the screen would update instantly.

Angular’s built-in control flow

Old-school Angular required “structural directives” like *ngIf and *ngFor logic control. These were powerful but required importing CommonModule and learning a specific micro-syntax. Modern Angular uses a built-in control flow that looks like standard JavaScript (similar to other Reactive platforms).

To see the new control flow in action, let’s add a list to our component. Update src/app/app.ts as follows, leaving the rest of the file the same:

export class App {
  protected readonly title = signal('iw-ng');
  protected readonly frameworks = signal(['Angular', 'React', 'Vue', 'Svelte']);
  protected showList = signal(true);

  toggleList() {
    this.showList.update(v => !v);
  }
}

While we’re at it, let’s also update src/app/app.html to render this new list (don’t worry about for now; it just tells Angular where to render the framing template):

Toggle List

@if (showList()) {
  

    @for (tech of frameworks(); track tech) {
      
{{ tech }}
    }
  
} @else {
  
List is hidden
}

The app will now display a list that can be toggled for visibility:

Matthew Tyson

This syntax is cleaner and easier to read than the old *ngFor loops:

• @if conditionally renders the block if the signal’s value is true.
• @for iterates over the array. The track keyword is required for performance (it tells Angular how to identify unique items in the list).
• (click) is an event binding. It lets us run code (the toggleList method) when the user interacts with the button.

Services: Managing business logic in Angular

Components focus on the view (i.e., what you see). For the business logic that backs the application functionality, we use services.

A service is just a class that can be “injected” into a component that needs it. This is Angular’s famous dependency injection system. It allows you to write logic once and reuse it anywhere. It’s a slightly different way of thinking about how an application is wired together, but it gives you real organizational benefits over time.

To generate a service, you can use the CLI:

$ ng generate service frameworks

This command creates a src/app/hero.ts file. In modern Angular, we define services using the @Injectable decorator. Currently, the src/app/hero.ts file just has this:

import { Injectable } from '@angular/core';

@Injectable({
  providedIn: 'root',
})
export class Frameworks {
  
}

Open the file and add a simple method to return our data:

import { Injectable } from '@angular/core';

@Injectable({
  providedIn: 'root', // Available everywhere in the app
})
export class Frameworks {
  getList() {
    return ['Angular', 'React', 'Vue', 'Svelte'];
  }
}

The providedIn: 'root' metadata is important, it tells Angular to create a single, shared instance of this service for the entire application (you might recognize this as an instance of the singleton pattern).

Using the service

In the past, we had to list dependencies in the constructor. Modern Angular offers a cleaner way: the inject() function. Subsequently, we can refactor our src/app/app.ts to get its data from the service instead of hardcoding it:

import { Component, inject, signal } from '@angular/core';
import { RouterOutlet } from '@angular/router';
import { Frameworks } from './frameworks'; // Import the service

@Component({
  selector: 'app-root',
  imports: [RouterOutlet],
  templateUrl: './app.html',
  styleUrl: './app.css'
})
export class App {
  private frameworksService = inject(Frameworks); // Dependency Injection
  
  protected readonly title = signal('iw-ng');
  
  // Initialize signal with data directly from the service
  protected readonly frameworks = signal(this.frameworksService.getList());
  protected showList = signal(true);

  toggleList() {
    this.showList.update(v => !v);
  }
}

Dependency injection is a powerful pattern. The component doesn’t need to know where the list came from (it could be coming from an API, a database, or a hard-coded array); it just asks the service for what it needs. This pattern adds a bit of extra work up front, but it delivers a more flexible, organized codebase as the app grows in size and complexity.

Routers and routes

Once your application grows beyond a single view, you need a way to navigate between different screens. In Angular, we use the built-in router for this purpose. In our example project, src/app/app.routes.ts is the dedicated home for the router config. Let’s follow the steps for creating a new route.

First, we define the route. When you open src/app/app.routes.ts, you will see an exported routes array. This array contains the available routes for your app. Each string name resolves to a component that handles rendering that route. In effect, this is the map of your application’s landscape.

In a real application, you’d often have “framing template” material in the root of the app (like the navbar) and then the routes fill in the body content. (Remember that by default, Angular is designed for single-page apps, where navigation does reload the screen, but swaps content.)

For now, let’s just get a sense of how the router works. First, create a new component so we have a destination to travel to. In your terminal, run:

$ ng generate component details

This will generate a simple details component in the src/app/details directory.

Now we can update src/app/app.routes.ts to include this new path. We will also add a “default” path that redirects empty requests to the home view, ensuring the user always lands somewhere:

import { Routes } from '@angular/router';
import { App } from './app'; // Matches src/app/app.ts
import { Details } from './details/details'; // Matches src/app/details/details.ts

export const routes: Routes = [
  { path: '', redirectTo: '/home', pathMatch: 'full' },
  { path: 'home', component: App },
  { path: 'details', component: Details },
];

Now if you visit localhost:4200/home, you’ll get the message from the details component: “Details works!”

Next, we’ll use the routerLink directive to move between views without refreshing the page. In src/app/app.html, we create a navigation bar that sits permanently at the top of the page (the “stationary” element), while the router swaps the content below it (the “impermanent” element):

And with that, the application has a navigation flow. The user clicks, the URL updates, and the content transforms, all without the jarring flicker of a browser reload.

Parametrized routes

The last thing we’ll look at is handling route parameters, where the route accepts variables in the path. To manage this kind of dynamic data, you define a route with a variable, marked by a colon. Open src/app/app.routes.ts and add a dynamic path:

export const routes: Routes = [
  // ... existing routes
  { path: 'details/:id', component: Details }, 
];

The :id is a placeholder. Whether the URL is /details/42 or /details/108, this router will receive it because it matches the path. Inside the details component, we have access to this parameter (using the ActivatedRoute service or the new withComponentInputBinding). We can use that value to retrieve the data we need (like using it to recover a detail item from a database).

Conclusion

We have seen the core elements of modern Angular: Setting up the environment, building reactive components with signals, organizing logic with services, and tying it all together with interactive routing.

Deploying these pieces together is the basic work in Angular. Once you get comfortable with it, you have an extremely powerful platform at your fingertips. And, when you are ready to go deeper, there is a whole lot more to explore in Angular, including:

• State management: Beyond signals, Angular has support for managing complex, application-wide state.
• Forms: Angular has a robust system for handling user input.
• Signals: We only scratched the surface of signals here. Signals offer a powerful, fine-grained way to manage state changes.
• Build: You can learn more about producing production builds.
• RxJS: Takes reactive programming to the next level.

Every day this past week I did something I suspect millions of other people also did: I stared at an LLM model picker and wondered which one I was supposed to want.

OpenAI just released ⁠GPT-5.6 Sol, Terra, and Luna. Sol is the flagship. Terra offers much of its intelligence for less money. Luna is cheaper still. Anthropic released ⁠Claude Sonnet 5 at the end of June and Opus 4.8 the month prior, with a little Fable 5 emerging in between. Meanwhile, Google, which seemed to be winning the model wars a few months ago, is now getting shade from Gergely Orosz, who ⁠argues that Gemini has slipped outside the top tier for software development and has been out of the major model release game for eons (May 19).

Perhaps Orosz is right. Perhaps he’ll be wrong again in six weeks. Honestly, it’s exhausting.

I use ChatGPT and Claude constantly and still have no principled idea which model to choose most of the time. I tend to click whatever looks like the biggest, most expensive option because I don’t know what I’m giving up by choosing something smaller. “Instant” sounds dangerously unserious. “Thinking” sounds expensive but powerful.

A quick survey of my LinkedIn crowd suggests others also feel my “WHICH MODEL???” pain. More importantly, I suspect most enterprises do, too.

A model doesn’t rot

Before getting carried away, however, it’s worth considering whether any of this model churn actually matters. After all, a model doesn’t rot. The model an enterprise put into production in March performs just as well in July as it did when the company selected it. “Obsolete” generally means that something better now exists, not that the deployed model suddenly stopped summarizing insurance claims or classifying support tickets. (In other words, once you have something working, the idea that “but maybe Opus 200.2 is better!” is really a FOMO problem, not a performance issue.)

Most enterprise workloads don’t live at the frontier anyway. Extraction, summarization, classification, document comparison, and customer-service assistance often work perfectly well with smaller, cheaper models. OpenAI’s own pitch for the trio of GPT-5.6 models isn’t simply that Sol is better. It’s that ⁠Terra and Luna deliver different combinations of intelligence, latency, and cost. Luna, the cheapest tier, nearly matches the previous generation’s peak performance at less than half the estimated cost, according to OpenAI.

The practical question, of course, is where to start. An enterprise can’t test every model, every reasoning setting, and every price tier before doing any work. So here’s my advice (which I don’t follow in my own work, but I’m not defining enterprise strategy and can be a little price-insensitive). Start with the cheapest credible model that appears capable of the task. Give it a representative set of real examples and, before you start testing, define what counts as good enough. If it passes, stop. If it fails, move up a tier or try a model with strengths better suited to the work.

That sounds almost offensively simple, but it reverses the way many people, including me, use these products. We start with the biggest model because we’re afraid of what we might lose. Enterprises should start lower and require evidence before paying for more intelligence.

There are exceptions, of course. For genuinely difficult work, such as autonomous coding, complex research, or high-stakes reasoning, beginning with a frontier model may save time. But even then, the goal should be to establish a quality ceiling, then test whether a cheaper model can meet it. It’s changing the question from “which model is best?” to “what is the least expensive model that reliably clears the bar for this job?”

For many workloads, that price improvement matters more than a few extra benchmark points. ⁠As I argued back in 2023, following AI hype doesn’t help anyone. If your model strategy depends on whichever benchmark screenshot is circulating on X this week, you don’t have a strategy. Not a viable one, anyway. Pick a model and ignore the noise.

Except, of course, when that noise suggests a serious signal.

Sometimes better really is better

Frontier improvements aren’t always incremental, making it advantageous to consider an upgrade. Coding is the obvious example. There’s a significant difference between a model that suggests the next few lines of code and one that can inspect a repository, plan a change, use tools, run tests, discover its own mistakes, and keep working for an extended period. That isn’t merely a nicer autocomplete experience. It can reorganize a development workflow.

This is why enterprises can’t simply standardize on an 18-month-old model and declare victory. In some areas, particularly software development and other agentic work, better models can unlock compounding productivity. A model that reliably completes 80% of a bounded task rather than 50% may justify an entirely different division of labor between humans and machines.

Still, that upgrade isn’t free.

Models differ in how they interpret instructions, call tools, manage context, refuse requests, and fail. Prompts and scaffolding tuned for one model can regress when moved to another. Or costs can explode. As one of my Oracle colleagues discovered just this week, running the same tasks in GPT 5.6 was orders of magnitude more expensive than 5.5. The API change may be trivial, but the revalidation and implications are not.

This leaves enterprises caught between two bad options. They can freeze and potentially miss out on meaningful improvements or chase every release and repeatedly test production systems on faith. What to do?

Stop making model bets

The answer is to stop making LLM bets and start making job-to-be-done bets. Stop asking which model is fastest. Instead, figure out what work you are trying to improve. What does a good result look like? How much latency and cost can the workflow tolerate? How wrong can it be before a human must intervene? Once those questions have answers, model selection becomes less opaque.

A difficult code migration may justify GPT-5.6 Sol or Claude Sonnet 5. A repetitive classification task may work just as well with Luna or another smaller model. A regulated workflow may require a model or deployment option that offers particular data controls. Sometimes the correct model is no LLM at all, like when I’m writing this post. Sorry, AI vendors! (At least you won’t get blamed for my mistakes.)

This is where evaluations become the center of enterprise AI strategy. ⁠As I’ve said before, most companies don’t have an AI quality problem so much as an AI measurement problem. Hence, a private evaluation suite built from real company work is the only leaderboard that matters. Does the new model materially improve quality? If so, use it! Does it reduce cost or latency? Again, that’s your free pass to adoption. Does the improvement justify the expense and effort of revalidation? If yes, continue.

Make model releases boring

As important as the model is, keep in mind that AI success always comes back to your company’s data, your company’s workflows, your company’s integrations, etc. That’s the ⁠dull reality behind sexy AI. Retrieval, memory, governance, data quality, observability, and feedback loops aren’t as exciting as a new model launch, but they’re what ultimately make AI truly work.

Again, when it’s time to consider something new, the principle should be to default to the least expensive model that reliably passes your evaluations. Only escalate harder tasks to more capable models when measurement shows that the premium pays. Tip: Make this invisible to employees so that the system routes to the best model for a particular prompt. As dbt Labs’ Jon Lewis expresses it, “The best model is ‘Auto’ and I won’t hear anyone say otherwise.” OpenAI’s own ⁠migration guidance recommends testing models on representative tasks, including trying a lower reasoning level rather than automatically cranking everything to the maximum.

As for me, I’ll probably keep clicking the shiniest option. I don’t have a formal evaluation suite for InfoWorld columns, and the marginal cost is a subscription I already pay. Enterprises don’t get that excuse.

Customer expectations don’t end when business hours do, which is why delivering a fast, always-on customer experience (CX) has traditionally required large call centres and significant resources. This often placed small businesses at a disadvantage, as many lacked the manpower and budget to provide 24/7 support at scale. Today, AI has completely levelled the playing field. Even small businesses now have access to powerful tools that can answer queries, resolve routine issues, and deliver highly personalised interactions around the clock.

But adopting AI in customer engagement is not just a question of efficiency. For smaller businesses especially, where loyalty is often built on familiarity, trust, and personal service, the real challenge is using AI in ways that strengthen rather than dilute the human connection that customers value most.

Human empathy combined with AI efficiency is a delicate blend. Done right, it ensures that every customer interaction feels personal, thoughtful, and seamless, whether the customer is engaging with a bot at 2 a.m. or a live agent during office hours.

So, how can small businesses embrace always-on virtual agents without losing the human connection that defines their identity? Here’s a practical playbook to guide the transition.

1. Understand what customers want: Speed, simplicity, and empathy

Before diving into AI adoption, it’s critical to understand what customers expect. Twilio’s Digital Patience study suggests that while speed matters, it is not the only thing that customers value. Twilio found that 46% of respondents in the Asia-Pacific and Japan region say quick service and resolution are most important, but 51% say delays are acceptable if they lead to better customer support. The study also notes that customers are open to AI, but still value human touchpoints more highly.

The takeaway: AI should enhance CX, not replace it. Businesses can let natural-sounding AI voice agents handle inbound calls, regardless of peak hours or time zones. These virtual agents act as an intelligent frontline – answering common questions and qualifying leads – before seamlessly routing the conversation to a live human representative. The result? Callers get immediate answers, and the business captures every opportunity without losing the human touch.

2. Map the handover points between AI and humans

One of the most common pitfalls in implementing AI is failing to clearly define when and how customers transition from bots to human agents. To avoid customer frustration, organisations must thoughtfully map out these “handover points” by designing for two key principles: choice and continuity.

Designing for Choice

Give customers the option to reach a human when needed. While AI is perfectly suited for routine inquiries like FAQs or order tracking, customers should never feel trapped in a bot loop. Always provide a clear, accessible option for them to choose to escalate the issue. Additionally, configure your system to proactively step in and offer a human handoff the moment it detects emotion, ambiguity, or complex steps.

Designing for Continuity

Effective handovers rely on technology that recognises when an issue exceeds AI’s scope. By leveraging natural language processing and intelligent routing, organisations can ensure the transition from machine to human is frictionless. Crucially, this means automatically carrying the full history and context of the interaction forward so the customer never needs to repeat themselves.

Achieving this level of continuity requires a new approach to managing interaction data during handovers. Instead of passing along a raw transcript, organisations need a managed memory service that provides agents with persistent context across every conversation, channel, and session. By transforming customer preferences, unresolved issues, and intent into a structured semantic profile—one that continuously evolves and reconciles new interactions as they occur—agents can quickly understand the relationship and continue the interaction without disruption.

To support truly omnichannel experiences, the system must also resolve identity automatically across touchpoints, linking interactions from phone, email, messaging apps, and other channels to a single customer profile. Equally important is the ability to surface only the information that is relevant to the task at hand. By presenting agents with a concise summary of the active issue and customer preferences, grounded in verified business knowledge such as product policies and FAQs, organisations can reduce resolution times while ensuring customers experience a seamless continuation of the conversation.

3. Don’t automate for automation’s sake

AI adoption should never feel like a “set it and forget it” strategy. Instead, it should be approached as a way to solve real business problems. It starts with asking questions like: What are the most time-consuming tasks for the team? What frustrates customers the most?

For instance, a restaurant might automate table reservations and menu queries, while a small online retailer could deploy AI to handle order status updates or product recommendations. These targeted use cases ensure that AI adds tangible value without overwhelming operations.

Take the example of Driva, a fast-growing online finance broker that deployed AI-powered customer service tools to answer routine enquiries and provide immediate assistance while customers wait in the call queue. By automating common interactions, Driva reduced the volume of requests requiring human intervention and achieved a 5% uplift in conversion rates at key points in the customer journey.

4. Invest in AI that connects

While consumers embrace automation, research shows they still draw comfort from the warmth of a human voice. To make your virtual agents feel less robotic and more like an extension of your team, look for tools that:

• Deliver human-like voice AI experiences at scale through natural turn-taking and barge-in capabilities.
• Connect interactions across voice, messaging, and digital channels into a single thread so every exchange builds on the last.
• Leverage Natural Language Processing (NLP) that enables conversational systems to interpret context, mimic human tone, and even recognise sentiment.
• Place orchestration at the heart of the experience. An effective orchestration engine acts as the “conductor,” actively coordinating workflows and routing interactions so the right resource—whether an AI bot or a human—handles the right moment.

When AI bots, automated workflows, and human teams are seamlessly coordinated behind the scenes, the customer simply experiences one unbroken, dynamic dialogue. For small enterprises, this means delivering sophisticated experiences that effortlessly bridge the gap between automation and live support, even at scale.

5. Empower teams with real-time context

AI is not about replacing human workers; it’s here to make jobs easier. However, for teams to fully embrace this new dynamic, organisations must shift their focus from retrospective performance reviews to real-time agent assistance. By feeding agents context as the conversation happens, businesses ensure that every interaction never starts from scratch.

• Leveraging Conversational Intelligence: Use a real-time intelligence layer that turns live conversations into signals and actions. By analysing voice and messaging with generative AI Language Operators, businesses can understand intent, sentiment, and churn risk instantly, allowing human and AI agents to act in the moment with the right response or escalation.
• In-the-Moment Guidance: Give agents instant context and in-the-moment guidance during every interaction. Surfacing relevant customer history, next-best action suggestions, and summaries in real time allows agents to resolve issues faster without switching tools.
• Resolving Complex Customer Needs: AI can handle routine enquiries with low latency, but human agents still excel at nuanced problem-solving. With AI feeding them persistent customer memory and sentiment analysis in real time, human agents can skip the repetitive questions and immediately focus on resolving complex issues, rescuing deals, or preventing churn.

When employees are equipped with real-time customer data and voice-driven insights, SMEs empower their teams to stop reacting to problems and start responding to customers proactively.

Consider global AI platform Genspark, which leverages a Programmable Voice API for its “Call for Me” agent to handle complex outbound tasks like checking supplier pricing or booking international hotels. The AI can conduct real-time, natural conversations across different languages on the user’s behalf, seamlessly navigating the live interactions before delivering a structured summary. Because these natural voice experiences depend entirely on speed and consistency, the underlying infrastructure provides the critical sub-second latency necessary to keep every automated call clear and uninterrupted.

6. Maintain transparency with customers

Finally, a successful AI implementation requires transparency. Customers should always know when they’re communicating with a bot and when they’ve been handed over to a human. AI-powered interactions must offer clarity by providing transparency about when and how AI is used and explaining next steps in plain language.

Transparency builds trust. Small businesses can go a step further by soliciting customer feedback on their AI interactions and using this input to fine-tune their systems.

For small enterprises, the AI-to-human handover isn’t about choosing between humans and machines; it’s about combining the strengths of both to create exceptional customer experiences. AI can provide the speed and efficiency customers expect, while humans deliver the empathy and creativity they value.

By strategically defining handover points, investing in human-like AI, and empowering agents to work alongside technology, organisations can build a CX strategy that’s as scalable as it is personal.

This blended approach ensures that every interaction – whether managed by a bot or a human – is thoughtful, natural, and distinctly on-brand.  

To learn more about Twilio, visit here.

Choosing the right eSign solution is less about picking the tool with the most bells and whistles and more about confirming that the features support your company’s requirements for security and compliance, workflow automation, cost-effectiveness, and operational efficiency.

The best eSign solutions let teams securely collect legally binding electronic signatures while integrating signing workflows with business systems, compliance controls, and document lifecycle processes — evaluated across four factors: security and compliance, workflow integration, total cost of ownership, and measurable business ROI.

When evaluating eSignature solutions, look beyond signing functionality and consider these four factors:

• Security and compliance
• Workflow integration
• Total cost of ownership
• Measurable business ROI

Security and compliance are the foundation of eSignatures

Yes, you want eSigning to be convenient, but it’s arguably even more important that your eSignature solution provides the security, auditability, and legal validity required to support critical business transactions.

Look for solutions that offer:

• Comprehensive audit trails
• Strong authentication controls
• Encryption in transit and at rest
• Support for established legal frameworks (e.g., the ESIGN Act, UETA, eIDAS)

Independent certifications, including SOC 2 Type II and ISO 27001, provide additional assurance that an eSign vendor follows recognized security and information management practices.

The signature is only one step in the document lifecycle

During the digital signing process, documents typically move through multiple workflows:

During the digital signing process, documents typically move through multiple stages: creation, review, approval, signature collection, storage, reporting, and retention.

Consider a typical sales contract: it might originate in a CRM, require review and approval from finance, get routed for signature, then need to be stored in a repository, reported on for compliance, and retained per policy. If each of these steps happens in a separate, disconnected tool, the signature may be digital, but the workflow is still manual.

When one or more of these steps rely on email attachments, manual routing, or moving files across disconnected applications, delays and inefficiencies quickly snowball.

An eSignature platform that supports document workflow automation can help you avoid this by connecting approval workflows, document routing, signature collection, and archival processes into a low-friction experience.

Evaluating the true cost of ownership of an eSignature solution

When you’re evaluating the cost of eSignature solutions, subscription pricing only tells part of the story. The solution with a lower upfront cost may require additional integrations, administrative effort, training, or support resources that increase long-term expenditure.

When calculating total cost of ownership, be sure to consider:

• Licensing and transaction costs
• Implementation and integration requirements
• Administrative overhead
• User adoption and training
• Compliance and audit support
• Scalability as your business needs evolve

How to measure eSignature ROI

Traditionally, the value proposition for eSignature software was that it reduced paper, printing, and shipping costs. Today, the value is firmly centered on operational outcomes, including:

• Contract turnaround times
• Employee onboarding speed
• Approval cycle duration
• Manual labor reduction
• Error elimination
• Compliance risk mitigation
• Customer and employee experience improvements

For example, reducing contract processing from days to hours can have a greater business impact than eliminating printing costs. Similarly, automated approval workflows can take over repetitive administrative tasks, freeing up employees to work on higher-value initiatives.

What to look for in an eSignature solution

As eSignature technology matures, the evaluation criteria have expanded beyond ease of signing. Today, organizations need solutions that can support compliance requirements, integrate with existing business systems, automate document workflows, and scale alongside broader digital transformation initiatives.

When comparing eSignature solutions, don’t just look at signing capabilities. Assess how well each eSign solution supports the entire document lifecycle through:

• Strong security and compliance controls
• Support for ESIGN, UETA, and eIDAS requirements
• Workflow automation capabilities
• Integration with existing business systems
• API accessibility for future automation initiatives
• Comprehensive audit trails and reporting
• Predictable, scalable pricing

In 2026, the best eSignature solution isn’t the one with the most features. It’s the one that connects signing to the rest of the document lifecycle while keeping security, cost, and ROI measurable. Nitro Sign is built around that principle: it goes beyond electronic signatures to support secure, compliant, connected document workflows that integrate with the systems teams already use, so governance improves, operations accelerate, and the solution scales with long-term business goals.

Discover why Nitro Sign has been recognized by IDC as a global leader in electronic signature software solutions.

Speak with an eSign Expert

US companies are increasingly opting for Chinese AI models from firms such as Deepseek and Z.ai, CNBC reports. A key reason is that they offer performance comparable to that of leading US models but at a significantly lower cost.

According to the development platform Openrouter, Chinese models have accounted for over 30% of the AI tokens used by US companies each week since February. In some weeks, the share has been as high as 46%, compared to an average of 11% over the previous 12 months.

Companies are reportedly increasingly choosing open AI models for tasks that do not require the highest performance. According to Openrouter, Chinese models can be between 60% and 90% cheaper than comparable alternatives from OpenAI and Anthropic.

Private browsing. Incognito. Privacy mode.

Web browser functions like those trace their roots back more than a decade, and the feature — first found in a top browser in 2005 — spread quickly as one copied another, made tweaks and minor improvements.

But privacy-promising labels can be treacherous. Simply put, going “incognito” is as effective in guarding online privacy as witchcraft is in warding off a common cold.

That’s because private browsing is intended to wipe local traces of where you’ve been, what you’ve searched for, the contents of forms you’ve filled. It’s meant to hide, and not always conclusively at that, your tracks from others with access to the personal computer. That’s it.

How to keep web browsing private

We’ve spelled out how to go into incognito or private browsing mode for the four major browsers in this order:

• Google Chrome’s Incognito mode
• Microsoft Edge’s InPrivate browsing
• Mozilla Firefox’s New Private Window mode
• Apple Safari’s New Private window mode

At their most basic, these features promise that they won’t record visited sites to the browsing history, save cookies that show you’ve been to and logged into sites, or remember credentials like passwords used during sessions. But your traipses through the web are still traceable by Internet providers – and the authorities who serve subpoenas to those entities – employers who control the company network and advertisers who follow your every footstep.

To end that cognitive dissonance, most browsers have added more advanced privacy tools, generically known as “anti-trackers,” which block various kinds of bite-sized chunks of code that advertisers and websites use to trace where people go in attempts to compile digital dossiers or serve targeted advertisements.

Although it might seem reasonable that a browser’s end game would be to craft a system that blends incognito modes with anti-tracking, it’s highly unlikely. Using either private browsing or anti-tracking carries a cost: site passwords aren’t saved for the next visit or sites break under the tracker scrubbing. Nor are those costs equal. It’s much easier to turn on some level of anti-tracking by default than it would be to do the same for private sessions, as evidenced by the number of browsers that do the former without complaint while none do the latter.

Private browsing will, by necessity, always be a niche, as long as sites rely on cookies for mundane things like log-ins and cart contents.

But the mode remains a useful tool whenever the browser — and the computer it’s on — are shared. To prove that, we’ve assembled instructions and insights on using the incognito features — and anti-tracking tools — offered by the top four browsers: Google Chrome, Microsoft’s Chromium-based Edge, Mozilla’s Firefox and Apple’s Safari.

How to go incognito with Google Chrome

Although incognito may be a synonym to some users for any browser’s private mode, Google gets credit for grabbing the word as the feature’s snappiest name when it launched the tool in late 2008, just months after Chrome debuted.

The easiest way to open an Incognito window is with the keyboard shortcut combination Ctrl-Shift-N (Windows) or Command-Shift-N (macOS).

Another way is to click on the menu on the upper right — it’s the three vertical dots — and select New Incognito Window from the list.

The new Incognito window can be recognized by the dark background and the stylized “spy” icon just to the left of the three-dots menu. Chrome also reminds users of just what Incognito does and doesn’t do each time a new window is opened. The message may get tiresome for regular Incognito users, but it may also save a job or reputation; it’s important that users remember Incognito doesn’t prevent ISPs, businesses, schools and organizations from knowing where customers, workers, students, and others went on the web or what they searched for.

Incognito’s introductory screen also displays a toggle — it’s on by default — along with text that states third-party cookies will be blocked while in the privacy mode. Although cookies are never saved locally as long as the user stays in Incognito, websites have been able to track user movements from site to site while within Incognito. Such tracking might be used, for example, to display ads to a user visiting multiple sites in Incognito. This third-party cookie blocking, which halts such behavior, debuted in May 2020.

Google has been experimenting with new language on Chrome’s Incognito introductory page, but it’s yet to make it to the desktop browser. In the Canary build of Chrome on Android, however, the intro now outlines “What Incognito does” and “What Incognito doesn’t do,” to make the mode’s capabilities somewhat clearer to the user. (Some have speculated that the changes were made in reaction to a still-ongoing class-action lawsuit file in 2020 that alleged Google continued to track users’ online behavior and movements in Incognito.)

Once a tab in Incognito has been filled with a website, Chrome continues to remind users that they’re in Incognito by the dark background of the address bar and window title.

A link on an existing page can be opened directly into Incognito by right-clicking the link, then choosing Open Link in Incognito Window from the resulting menu.

Pro tip: To close an Incognito window, shutter it like any other Chrome window by clicking the X in the upper right corner (Windows) or the red dot in the upper left (macOS).

How to privately browse with Microsoft Edge

Microsoft borrowed the name of its private browsing mode, InPrivate, from Internet Explorer (IE), the finally-being-retired legacy browser. InPrivate appeared in IE in March 2009, about three months after Chrome’s Incognito and three months before Firefox’s privacy mode. When Edge was first released in 2015 and then relaunched as a clone of Chrome in January 2020, InPrivate was part of the package, too.

At the keyboard, the combination of Ctrl-Shift-N (Windows) or Command-Shift-N (macOS) opens an InPrivate window.

A slower way to get there is to click on the menu at the upper right — it’s three dots arranged horizontally — and choose New InPrivate Window from the menu.

Edge does a more thorough job of explaining what its private browsing mode does and doesn’t do than any of its rivals, with on-screen paragraphs dedicated to describing what data the browser collects in InPrivate and how the strictest additional anti-tracking setting can be called on from within the mode. In addition, Edge uses the more informal “What Incognito does” and “What Incognito doesn’t do” language on its InPrivate introductory screen.

Microsoft’s browser also well marks InPrivate when the mode is operating: an oval marked “In Private” to the right of the address bar combines with a full-black screen to make sure users know where they’re at.

It’s also possible to launch an InPrivate session by right-clicking a link within Edge and selecting Open in InPrivate Window. That option is grayed out when already in a private browsing session but using Open Link in New Tab does just that within the current InPrivate frame.

To end InPrivate browsing, simply shut the window by clicking the X in the upper right corner (Windows) or click the red dot at the upper left (macOS).

Although Edge is based on Chromium, the same open-source project that comes up with the code to power Chrome, the Redmond, WA company integrated anti-tracking into its browser. Dubbed “Tracking Prevention,” it works both in Edge’s standard and InPrivate modes.

To set Tracking Prevention, choose Settings from the three-ellipses menu at the right, then at the next page, pick Privacy, Search and Services. Choose one of the three options — Basic, Balanced or Strict — and make sure the toggle for Tracking prevention is in the “on” position. If you want InPrivate to always default to the harshest anti-tracking — not a bad idea — toggle Always use “Strict” tracking prevention when browsing InPrivate to “on.”

Pro tip: To open Edge with InPrivate — rather than first opening Edge in standard mode, then launching InPrivate — right-click the Edge icon in the Windows taskbar and select New InPrivate Window from the list. There is no similar one-step way to do this in macOS.

How to privately browse with Mozilla Firefox

After Chrome trumpeted Incognito, browsers without something similar hustled to catch up. Mozilla added its take — dubbed Private Browsing — about six months after Google, in June 2009.

From the keyboard, a private browsing session can be called up using the combination Ctrl-Shift-P (Windows) or Command-Shift-P (macOS).

Alternately, a private window will open from the menu at the upper right of Firefox — three short horizontal lines — after selecting New private window.

A private session window is marked by the purple “mask” icon in the title bar of the Firefox frame. In Windows, the icon is to the left of the minimize/maximize/close buttons; on a Mac, the mask squats at the far right of the title bar. Unlike Chrome and Edge, Firefox does not color-code the top components of the browser window to signify the user is in privacy mode.

Like other browsers, Firefox warns users that private browsing is no cure-all for privacy ills but is limited in what it blocks from being saved during a session. “Private window: Firefox clears your search and browsing history when you close all private windows. This doesn’t make you anonymous,” the caution reads.

A link can be opened into a Firefox Private Window by right-clicking the link, then choosing Open Link in New Private Window from the menu.

To close a Private Window, shut it down just as one would any Firefox window by clicking the X in the upper right corner (Windows) or the red dot in the upper left (macOS).

Notable is that Firefox’s private browsing mode is accompanied by the browser’s superb “Enhanced Tracking Protection,” a suite of tracker blocking tools that stymie all sorts of ad-and-site methods for identifying users, then watching and recording their online behavior. While the earliest version of this was offered only inside Private Windows, the expanded technologies also work within standard mode.

Because Enhanced Tracking Protection is enabled by default within Firefox, it doesn’t matter which of its settings — Standard, Strict or Custom — is selected as far as private browsing goes; everything that can be blocked will be blocked.

Pro tip: Private Browsing sessions take place over the more secure HTTPS, not the once-standard HTTP protocol. Users don’t need to do anything: The new HTTPS-only policy is on by default. (If the destination site doesn’t support HTTPS, Firefox will go into fallback mode, connecting via HTTP instead.)

How to browse privately with Apple’s Safari

Chrome may get far more attention for its Incognito than any other browser — no surprise, since it’s by far the most popular browser on the planet — but Apple’s Safari was actually the first to introduce private browsing. The term private browsing was first bandied in 2005 to describe early Safari features that limited what was saved by the browser.

Side note: Early in private browsing, the label porn mode was often used as a synonym to describe what many writers and reporters assumed was the primary application of the feature. The term has fallen out of favor.

To open what Safari calls a Private Window on a Mac, users can do a three-key combination of Command-Shift-N, the same shortcut Chrome adopted. Otherwise, a window can be called up by selecting the File menu and clicking on New Private Window.

Safari tags each Private Window by darkening the address bar. It also issues a reminder of what it does — or more accurately — what it doesn’t do. “Safari will keep your browsing history private for all tabs of this window. After you close this window, Safari won’t remember the pages you visited, your search history or your AutoFill information,” the top-of-the-page note reads. The warning is more terse than those of other browsers and omits cautions about still-visible online activity.

Like Firefox, Safari automatically engages additional privacy technologies, whether the user browses in standard or private mode. Safari’s Intelligent Tracking Protection (ITP), which has been around for nearly a decade and repeatedly upgraded, now blocks all third-party cookies, among other components advertisers and services use to track people as they bounce from one site to another. ITP is controlled by a single on-off switch — on is the default — found in Preferences under the Privacy icon. If the Website tracking: box is checked to mark Prevent cross-site tracking, ITP is on.

A link can be opened directly to a Private Window by right-clicking, then selecting Open Link in New Private Window. Close a Private Window just as any Safari window, by clicking the red dot in the upper left corner of the browser frame.

Pro tip: Once in a Safari Private Window, opening a new tab — either by clicking the + icon at the upper right or by using the Command-T key combo — omits the Private Browsing Enabled notice. (The darkened address bar remains as the sole indicator of a private browsing session.) Other browsers, such as Firefox, repeat their cautionary messages each time a tab is opened in an incognito session.

Related reading:

• Chromium explained: How the open-source engine drives today’s browsers
• AI web browsers are cool, helpful, and utterly untrustworthy
• How AI will transform your Windows web browser

You can be forgiven if you think the most important thing AWS ever sold developers was EC2. It’s not. No, AWS’s big gift to developers was permission to stop fretting about servers. That sounds obvious now, but it was close to magical at the time. Before the cloud, getting infrastructure meant waiting on procurement, hardware, and the somewhat arcane process that stood between a developer and a running machine. AWS turned that into a credit card and an API.

It was awesome.

AWS still (over)uses a great phrase for what it removed: “undifferentiated heavy lifting.” That is, all the mess associated with racking servers, patching operating systems, managing storage, planning capacity, etc. Important work, sure, but not the work that makes your application special. Let AWS do that, the company intoned, and developers could focus on the thing their customers actually cared about.

It was a brilliant abstraction. It helped build one of the most important technology companies of the past two decades. It’s also the same logic that increasingly makes the cloud seem superfluous. Not because the cloud is dying. It isn’t. The cloud is bigger and more essential than ever. But developers don’t begin by asking, “Which cloud should I use?” They begin with, “How quickly can I get this thing working?”

That is a different question, and it leads to different tools.

Where to begin?

Let’s get the obvious thing out of the way: AWS remains the biggest developer cloud, and its revenue growth has accelerated in the wake of AI. In the 2024 Stack Overflow Developer Survey⁠, AWS was the top cloud platform, used by 48% of respondents. Microsoft Azure and Google Cloud followed at 27.8% and 25.1%, respectively.

The 2025 Stack Overflow Developer survey⁠ is more interesting, however, because the “cloud development” category no longer reads like a clear cloud-vendor horse race. Docker jumped 17 points to 71% usage, followed by npm and then AWS at 43%. Kubernetes, Azure, Google Cloud, Cloudflare, Terraform, Firebase, Vercel, Netlify, and Supabase all show up in the same mental map.

This is my point: Developers aren’t simply choosing a different cloud first. Often, they’re not choosing a cloud first at all. They’re choosing a workflow, and a workflow is an increasingly separate decision from the underlying cloud.

The first mile moved

For a long time, AWS owned the first mile. If you wanted to build something, you opened an AWS account. Need compute? Launch EC2. Storage? S3. Whatever a developer needed—database, queue, function, CDN, identity layer—AWS, the “everything store,” almost certainly had an answer.

It was magical! Then it became the norm. And, over time, it became a lot. The “everything store” arguably had too much, making it hard for developers to know how to use it effectively. The developer’s question became, “How much AWS do I have to understand before I can ship?” Developers increasingly don’t want to answer that question at the start.

The first mile now often begins in GitHub⁠, which says more than 36 million developers joined in a single year. Or it begins in VS Code, which keeps winning because it’s familiar, extensible, and already open. Or it begins in Cursor, GitHub Copilot, Claude, Codex, and the broader AI coding layer. GitHub’s 2025 Octoverse⁠ says nearly 80% of new developers on GitHub use Copilot within their first week. Stack Overflow’s 2025 AI survey results⁠ point the same way, with 84% of respondents using or planning to use AI tools in their development process, up from 76% in 2024.

The starting point is changing. The default interface to building software is becoming an AI-assisted workflow, not a cloud console. This doesn’t make the cloud less important, but it definitely makes it less visible.

The joy of not caring

The developer platforms with momentum right now are winning because they expose less cloud infrastructure. Call it serverless if you like, but it’s deeper than AWS Lambda ever was. AWS Lambda still made you think in AWS. These new platforms make you think in your app.

Take Vercel⁠. It didn’t win developer mindshare by offering more than 200 services and praying developers would navigate them all. Instead, it attached itself to the way many front-end and full-stack developers already work: GitHub, Next.js, previews, deployments, performance, and a short path from code to live application. It seems to be working: Reuters reported last year that Vercel raised $300 million at a $9.3 billion valuation⁠, after doubling its user base and growing revenue 82%.

Cloudflare offers a similar promise from a different angle, as I’ve written: deploy globally, run close to users, scale automatically, and don’t make infrastructure the developer’s first problem. Supabase⁠ does the same for data, wrapping Postgres with authentication, instant APIs, edge functions, real-time subscriptions, storage, and vectors so the database feels less like a separate system and more like an app platform.

None of this is anti-cloud. But it’s very definitely stripping away the need to even think about cloud. And it’s just as definitely where developers are focused today.

AI accelerated all of this because it changed the first question developers ask. For an AI application, the opening move often isn’t where do you host it, but rather which model, which agent framework, etc., will get you to a live application fastest? For this reason, OpenAI’s developer platform⁠ isn’t framed as renting infrastructure. It’s framed as building with models, APIs, tools, docs, and examples. Similarly, Anthropic’s Claude Code⁠ isn’t positioned as configuring a cloud environment; it’s an agentic coding system that works in your code base and helps you build, test, and ship.

The second mile matters more

I’m not saying the cloud vendors should abandon hope. Indeed, though the first mile has moved away from the hyperscalers, the second mile is still very much theirs to win.

Vercel, Cloudflare, Supabase, OpenAI, Anthropic, and GitHub are brilliant at helping developers make something work, fast. But in the enterprise, “fast” isn’t the key priority: Enterprises eventually need the boring stuff like identity, network controls, observability, cost management, compliance, and the rest of the unglamorous apparatus that keeps customer data from leaking onto Reddit.

Boring is good when boring means the thing works.

This is where AWS, Azure, Google Cloud, Oracle, and other infrastructure companies should have an advantage. But it won’t be enough to say, “We have the grown-up features.” That’s true, but it’s dull, and in this case, “dull” isn’t a winning proposition. No, the strategic challenge for the hyperscalers is to make the jump from prototype to production feel like an upgrade instead of a punishment.

AWS’s new $1 billion investment in forward-deployed AI engineers⁠ is interesting for exactly this reason. AWS says the new organization will embed experts with customers to co-develop and deploy agentic AI solutions in days. That’s not classic bottom-up developer love, but it’s a smart recognition that the bottleneck has moved closer to the application. The hard part is turning capability into a working system, not merely getting access to infrastructure or a model.

Yep. Exactly.

Hiding the cloud

The mistake for cloud infrastructure companies would be to respond to this by trying to become a Vercel, Supabase, GitHub, OpenAI, or Anthropic all at once. That’s the classic incumbent move, and it rarely works. The better move is narrower: become the most natural next step when the prototype starts to matter.

That means meeting developers where they work: GitHub, VS Code, Cursor-style environments, CI/CD systems, and AI agents. It means outcome-native starts: deploy this app, connect this data, expose this API, add auth, evaluate this agent, secure this workflow, move this prototype into production. It means fewer scavenger hunts across product pages and establishing more golden paths (yes, even if that infuriates a service team that gets left out).

The old question was, “Can you run my workload?” For AWS, Azure, Google Cloud, Oracle, and others, the answer is almost always yes. But no one is impressed by this anymore. Therefore, the better question is, “Can you become part of how I build?”

Developers came to the cloud because it was the easiest way to get infrastructure. They’re shifting to developer experience platforms because those are the easiest way to get outcomes. If the cloud providers can become part of the first mile before the Vercels of the world become part of the second mile, they’ll win, and big. If they don’t, well….

As an old Delphi guy, I remember well the “language wars” we had with the Visual Basic guys. An early codename for Delphi was “VBK” — VB Killer — and the VB community took exception. They’d come to our Delphi forums and pick fights. Naturally, we brash Delphi guys would fight back, engaging in big flame wars and getting all worked up over what wasn’t much more than a personal preference. Good times.

These days, we’ve moved the discussion up a layer — what is the better model for coding? Things aren’t quite as intense as the VB/Delphi dustups, but people have their opinions. Companies are taking a look at different models before choosing one for their teams. Most teams have arrived at a family of models that they use. 

At some point, chatting with Claude or Codex started to seem a bit raw. It wasn’t long before scaffolding tools like GStack and Superpowers were adding underpinnings for interacting with LLMs — baseline instructions for handling prompts before they get to the model itself. They help establish useful context and act as a layer above “raw prompting”. Context engineering is the first and most common layer to add on top of the chat interface.

And then once the choice of models and harnesses was made, everyone went crazy with tokenmaxxing. If you have a model, of course you want to get the most out of it. But when the bill came in, managers were not pleased. As costs skyrocketed, leadership worried that the money wasn’t being well spent. 

Model routing – the next layer

Just as assembly language and hand-tuning registers gave way to compilers and structured languages, which led to frameworks and libraries, and most recently to LLMs and prompting, it is starting to occur to developers and managers that there is a better way to manage LLM spending. 

But naturally, the minute you figure out how things work, another layer appears, making all your hard-earned knowledge outdated. Apparently being able to code in English isn’t enough to stop the next abstraction from appearing.

So as is always the case, another layer of abstraction has come along. (Sic semper fuit.) Thus model routing is the latest way to maximize the value for each dollar spent on tokens. 

The idea is that not all prompts are created equal. Not everything that you ask Claude is going to require the deep thinking of a frontier model. A model router can take a look at the prompt and decide what model is best suited to answer that prompt and direct the query to that model. Maybe simpler requests are better suited for an older model. Maybe code reviews are better done with a model specifically designed for that purpose. 

Model routing leads to more efficient token spending. When you run Claude Code today, you have to choose a model for the whole session, and if you want to use the top-tier model, you have to pay for it no matter what you end up doing. A model router lets you vary the model — and thus the cost. Organizations like Coinbase are seeing their AI spend cut in half while their token usage increases. 

From tokenmaxxing to tokenmatching

LLMs are constantly evolving, becoming both more powerful and more specialized. Being able to route a prompt to the model that is both well-suited for the task and cost-effective is the way to maximize token effectiveness. Teams are doing this manually now, but AI itself will become the best way to make such decisions. 

For example, Claude Code Router can route prompts to any number of popular models, depending on the type of work each prompt requires. And it’s open source.

The next layer that is coming is the preprocessing of prompts. We can work to write good prompts, but AI itself can improve upon what we ask. One of the best techniques in prompting is to tell the LLM to “ask the questions that I’m not asking but should be asking”. I can easily imagine a world in which you write a prompt, AI helps you clarify it, improves it, and then routes it to the best, most cost-effective model for an answer.

You won’t be choosing a given LLM provider anymore. Instead, you can focus on specifying exactly what you want. So stop hand-crafting your prompts for a specific model. Let the coming model routers and prompt preprocessors do the hard work for you.

As an old Delphi guy, I remember well the “language wars” we had with the Visual Basic guys. An early codename for Delphi was “VBK” — VB Killer — and the VB community took exception. They’d come to our Delphi forums and pick fights. Naturally, we brash Delphi guys would fight back, engaging in big flame wars and getting all worked up over what wasn’t much more than a personal preference. Good times.

These days, we’ve moved the discussion up a layer — what is the better model for coding? Things aren’t quite as intense as the VB/Delphi dustups, but people have their opinions. Companies are taking a look at different models before choosing one for their teams. Most teams have arrived at a family of models that they use. 

At some point, chatting with Claude or Codex started to seem a bit raw. It wasn’t long before scaffolding tools like GStack and Superpowers were adding underpinnings for interacting with LLMs — baseline instructions for handling prompts before they get to the model itself. They help establish useful context and act as a layer above “raw prompting”. Context engineering is the first and most common layer to add on top of the chat interface.

And then once the choice of models and harnesses was made, everyone went crazy with tokenmaxxing. If you have a model, of course you want to get the most out of it. But when the bill came in, managers were not pleased. As costs skyrocketed, leadership worried that the money wasn’t being well spent. 

Model routing – the next layer

Just as assembly language and hand-tuning registers gave way to compilers and structured languages, which led to frameworks and libraries, and most recently to LLMs and prompting, it is starting to occur to developers and managers that there is a better way to manage LLM spending. 

But naturally, the minute you figure out how things work, another layer appears, making all your hard-earned knowledge outdated. Apparently being able to code in English isn’t enough to stop the next abstraction from appearing.

So as is always the case, another layer of abstraction has come along. (Sic semper fuit.) Thus model routing is the latest way to maximize the value for each dollar spent on tokens. 

The idea is that not all prompts are created equal. Not everything that you ask Claude is going to require the deep thinking of a frontier model. A model router can take a look at the prompt and decide what model is best suited to answer that prompt and direct the query to that model. Maybe simpler requests are better suited for an older model. Maybe code reviews are better done with a model specifically designed for that purpose. 

Model routing leads to more efficient token spending. When you run Claude Code today, you have to choose a model for the whole session, and if you want to use the top-tier model, you have to pay for it no matter what you end up doing. A model router lets you vary the model — and thus the cost. Organizations like Coinbase are seeing their AI spend cut in half while their token usage increases. 

From tokenmaxxing to tokenmatching

LLMs are constantly evolving, becoming both more powerful and more specialized. Being able to route a prompt to the model that is both well-suited for the task and cost-effective is the way to maximize token effectiveness. Teams are doing this manually now, but AI itself will become the best way to make such decisions. 

For example, Claude Code Router can route prompts to any number of popular models, depending on the type of work each prompt requires. And it’s open source.

The next layer that is coming is the preprocessing of prompts. We can work to write good prompts, but AI itself can improve upon what we ask. One of the best techniques in prompting is to tell the LLM to “ask the questions that I’m not asking but should be asking”. I can easily imagine a world in which you write a prompt, AI helps you clarify it, improves it, and then routes it to the best, most cost-effective model for an answer.

You won’t be choosing a given LLM provider anymore. Instead, you can focus on specifying exactly what you want. So stop hand-crafting your prompts for a specific model. Let the coming model routers and prompt preprocessors do the hard work for you.

As an old Delphi guy, I remember well the “language wars” we had with the Visual Basic guys. An early codename for Delphi was “VBK” — VB Killer — and the VB community took exception. They’d come to our Delphi forums and pick fights. Naturally, we brash Delphi guys would fight back, engaging in big flame wars and getting all worked up over what wasn’t much more than a personal preference. Good times.

These days, we’ve moved the discussion up a layer — what is the better model for coding? Things aren’t quite as intense as the VB/Delphi dustups, but people have their opinions. Companies are taking a look at different models before choosing one for their teams. Most teams have arrived at a family of models that they use. 

At some point, chatting with Claude or Codex started to seem a bit raw. It wasn’t long before scaffolding tools like GStack and Superpowers were adding underpinnings for interacting with LLMs — baseline instructions for handling prompts before they get to the model itself. They help establish useful context and act as a layer above “raw prompting”. Context engineering is the first and most common layer to add on top of the chat interface.

And then once the choice of models and harnesses was made, everyone went crazy with tokenmaxxing. If you have a model, of course you want to get the most out of it. But when the bill came in, managers were not pleased. As costs skyrocketed, leadership worried that the money wasn’t being well spent. 

Model routing – the next layer

Just as assembly language and hand-tuning registers gave way to compilers and structured languages, which led to frameworks and libraries, and most recently to LLMs and prompting, it is starting to occur to developers and managers that there is a better way to manage LLM spending. 

But naturally, the minute you figure out how things work, another layer appears, making all your hard-earned knowledge outdated. Apparently being able to code in English isn’t enough to stop the next abstraction from appearing.

So as is always the case, another layer of abstraction has come along. (Sic semper fuit.) Thus model routing is the latest way to maximize the value for each dollar spent on tokens. 

The idea is that not all prompts are created equal. Not everything that you ask Claude is going to require the deep thinking of a frontier model. A model router can take a look at the prompt and decide what model is best suited to answer that prompt and direct the query to that model. Maybe simpler requests are better suited for an older model. Maybe code reviews are better done with a model specifically designed for that purpose. 

Model routing leads to more efficient token spending. When you run Claude Code today, you have to choose a model for the whole session, and if you want to use the top-tier model, you have to pay for it no matter what you end up doing. A model router lets you vary the model — and thus the cost. Organizations like Coinbase are seeing their AI spend cut in half while their token usage increases. 

From tokenmaxxing to tokenmatching

LLMs are constantly evolving, becoming both more powerful and more specialized. Being able to route a prompt to the model that is both well-suited for the task and cost-effective is the way to maximize token effectiveness. Teams are doing this manually now, but AI itself will become the best way to make such decisions. 

For example, Claude Code Router can route prompts to any number of popular models, depending on the type of work each prompt requires. And it’s open source.

The next layer that is coming is the preprocessing of prompts. We can work to write good prompts, but AI itself can improve upon what we ask. One of the best techniques in prompting is to tell the LLM to “ask the questions that I’m not asking but should be asking”. I can easily imagine a world in which you write a prompt, AI helps you clarify it, improves it, and then routes it to the best, most cost-effective model for an answer.

You won’t be choosing a given LLM provider anymore. Instead, you can focus on specifying exactly what you want. So stop hand-crafting your prompts for a specific model. Let the coming model routers and prompt preprocessors do the hard work for you.

Over the past two years, working inside the enterprise AI infrastructure world, tracking where the industry is heading, I have noticed the same question surface repeatedly: should we build our own large language model? I understand the instinct. The model feels like the thing, the engine, the brain, the asset worth owning. But after significant years as a product manager in the AI world in both customer experience and grounding infrastructure I concluded that it tends to unsettle the room: the model is the least durable part of your AI strategy.

I say this not to be provocative, but because over the last few years we have seen organizations pour their scarcest resources, executive attention, engineering talent, capital, into the one layer of the stack that is commoditizing fastest. Meanwhile, the layer that determines whether their AI is trustworthy, accurate and defensible gets treated as plumbing. That inversion is, in my experience, the single most expensive mistake enterprises are making with AI right now.

The model is becoming a commodity

Let us consider economics. Gartner projects that by 2030, performing inference on a trillion-parameter model will cost providers more than 90% less than it did in 2025, with models becoming up to 100 times more cost-efficient than the earliest versions of comparable size. When the cost of the underlying capability collapses by that magnitude, it stops being a differentiator. Anything that gets that cheap, that fast, is not where competitive advantage lives.

Models that feel innovative are routinely surpassed by something cheaper and better within months. If your advantage is tied to a specific model, it will evaporate the moment the frontier moves, which it always does. But if an enterprise instead invests in how reliably it can feed any model its proprietary context, that investment holds. That part travels from one model generation to the next. When a better model arrives, the organization can simply connect it and immediately capture the upside, because the hard and durable work was already done one layer down.

I wish more leaders could observe this pattern before they commit. The model layer is improving so quickly that any advantage you build into it has a short half-life. The grounding layer behaves in the opposite way: every improvement you make to your data quality, your retrieval logic and your governance compounds, and it carries forward regardless of which model sits on top.

This is why the build-your-own LLM debate so often misses the mark. Training or even meaningfully fine-tuning a foundation model is enormously expensive, and the moment you finish, the open and commercial frontier has usually moved past you. So, technically you spent a fortune to own a depreciating asset. The capability that you should focus on is an AI that knows your business, was never going to come from the weights of the model anyway. It comes from what you put in front of it.

Why grounding is the real moat

Grounding is the discipline of connecting a general-purpose model to your enterprises’ current and authoritative information, most commonly through retrieval-augmented generation, or RAG. Rather than hoping the model memorized something useful during training, you retrieve the relevant facts from your own systems in real time of the query and give the model the context it needs to answer correctly.

Here is the part that matters for anyone thinking about competitive advantage: your competitors can rent the exact same model you use. What they cannot rent is your data, your institutional knowledge, your processes and the quality of the pipeline that surfaces all of it accurately at the right moment. That pipeline is genuinely proprietary, genuinely hard to replicate and it compounds in value over time. That is the textbook definition of a moat, and it has almost nothing to do with which model you chose.

The industry is starting to recognize this. Gartner predicts that by 2027, organizations will use small, task-specific models at least three times more than general-purpose LLMs, precisely because accuracy in real business workflows depends on domain context rather than raw model scale. But a smaller model holds less in its parameters by design, which means it leans even harder on retrieval to supply current, authoritative context in real time. The model gets smaller and more swappable. The grounding becomes the part that carries the weight. In that same analysis, Gartner makes the same point from the data side: what sets enterprises apart is how well they prepare, check, version and manage their own data. Read that again: the differentiator is the data discipline, not the model.

This matches what I have observed directly. Getting hold of an excellent model was never the hard part, and it was rarely where things broke. The failures I have seen came from not connecting the model efficiently to the right data sources or orchestrating retrieval well. The patterns repeat: missing data produces incomplete summaries, truncated documents leave answers without key details, and noisy context yields irrelevant or confusing responses.

When grounding is absent, answers become inconsistent from one client to the next; when retrieval comes back empty, the model fills the gap with something hallucinated or useless. Stale data produces confidently outdated answers, retrieval gaps surface as generic non-answers, and poor-quality data drags down both speed and output. None of these are model problems. They are grounding problems. And when a system hands an executive an answer that is wrong, no one in the boardroom cares how sophisticated the model was. They care that it was wrong, and the fix always lives in the grounding layer.

One example has stayed with me. In a real enterprise scenario, an AI assistant returned inconsistent answers to the same query across different environments whenever grounding was unavailable, and some of those answers contradicted each other outright. The cause was straightforward in hindsight. With no grounding, the system fell back on its own internal knowledge instead of a shared, grounded source of truth, so its responses drifted with each configuration and context. The damage was not just technical. Users stopped trusting an assistant that could not give them the same answer to the same question twice. That is the actual cost of weak grounding, and it is why consistency and reliability in production depend far more on the data layer than on the model sitting above it. No model upgrade would have fixed that.

Where leaders should focus their investment

If you accept that grounding is where advantage accrues, a few priorities shift in ways that should change how you allocate budget and attention.

First, treat your organization’s data foundation as a first-class AI investment, not a prerequisite you rush through. The unglamorous work, cleaning, structuring, governing and versioning your knowledge, is the work that determines AI quality. I would rather inherit a mediocre model with an excellent retrieval pipeline than the reverse, every single time.

Second, build for model portability from day one. Assume the model you use today will be replaced within a year because it certainly will. If swapping it out is painful, you have coupled your architecture to the wrong layer. Your grounding infrastructure, your evaluation framework and your data contracts should be the stable core; the model should be a component you can swap with minimal disruption.

Third, invest in observability and evaluation for retrieval, not just for the model. The emerging discipline here matters: Gartner expects LLM observability investments to reach 50% of GenAI deployments by 2028, up from 15% today, as trust requirements outpace the technology itself. Knowing why your system retrieved a particular piece of context, and whether that context was correct, is what makes an AI output defensible and auditable. For any organization operating under real regulatory or reputational scrutiny, that is not optional.

None of this means the model is irrelevant. You still need a capable one and choosing well matters. But choosing a model is now a procurement decision with several excellent options, not a source of lasting differentiation. The lasting differentiation is everything you wrap around it.

I think the organizations that internalize this will look, in a few years, meaningfully ahead of the ones still debating whether to train their own model. Not because they made a bolder bet, but because they made a more durable one. They understood that in a world where everyone has access to the same extraordinary models, the advantage belongs to whoever grounds those models best in the reality of their own business. The model is rented. The grounding is owned. Build accordingly.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

With headquarters in Johannesburg, South Africa, Absa also operates in many other African countries, with international offices in Europe and the US. Running an organization across several markets has its unique complexities, especially in the integration layer, because each region has its own systems, business processes, regulatory requirements and data standards.

For Absa, replacing an integration layer that had reached breaking point was a fundamental shift in its banking philosophy. It wasn’t just a technical project. Duplication was rampant, complexity was baked in, and reusability was non-existent. Every change had far-reaching ripple effects, and each new channel had to be built from scratch. As it stood, making the improvements the business demanded at the speed required to remain competitive was impossible.

According to Tamu Dutuma, Absa’s head of technology strategy for Africa Regions, this integration layer had been in place for close to a decade. While it played an important role in enabling business in the past, it was too difficult to maintain and no longer aligned to current standards and ways of working.

Integration standardization

Absa evaluated a range of available solutions in the market, but given the complexity of integrating with legacy systems across a multi-country financial environment, the team decided a more tailored approach was required.

“It was critical to establish the right architecture from the outset, which is why we worked with a strategic partner to build a solution that could better meet our specific integration needs, while also creating a stronger foundation for future scalability,” says Dutuma.

Balancing the long-term benefits of standardization against the immediate complexity of making the shift meant taking time to understand the upstream and downstream impact. The team had to be realistic about how they would standardize banking services, systems, and integrations while keeping disruption to a minimum.

As part of this process, Absa aligned with globally recognized standards, including BIAN, which provides a common framework for designing and integrating banking systems. The goal is to give banks a blueprint to successfully modernize complicated legacy architectures by defining standardized business capabilities, service domains, APIs, and data models.

The new integration layer provided three critical things for the business: decoupling and abstraction, standardization, and strategic orchestration. This meant separating customer-facing channels from core banking and backend services, using BIAN frameworks to enforce strict governance, and orchestrating only where necessary to keep the architecture lean.

Choosing the right implementation strategy

With this plan in mind, the bank needed to decide how to execute it. “We took a phased approach to the rollout, starting with a specific use case, our chatbot Chat Banking in our Africa Regions business,” says Dutuma. “This allowed us to build and test the new integration layer in a controlled, practical way. From there, we introduced an architecture principle that all new initiatives would integrate through this platform, while only time-critical projects continued to rely on the legacy environment.” The goal was to set a North Star project, which allowed them to quickly demonstrate value.

But this wasn’t a copy-paste exercise, and everything didn’t fit perfectly from the start. The bank admits that managing legacy outliers remains one of the biggest challenges on this modernization journey. Data mapping was another challenge. To ensure data moved correctly and quickly from one system to another, Absa had to build a data mapping framework to automate parts of the process.

As the project progressed and the team ironed out these kinks, they gradually migrated existing services to the new layer. “This wasn’t a like-for-like replacement,” he says. “We were also simplifying and standardizing the architecture, which required careful mapping, redesign, and end-to-end testing across both channels and core systems.”

Banking on the future

For Dutuma, this multi-year journey has allowed Absa to incrementally modernize the environment while continuing to support ongoing business delivery. And the project has delivered several strategic wins, from a drastic reduction in time-to-market to an equally dramatic reduction in costs. Standardization also opened additional opportunities for innovation across the business. For example, using a standardized API catalog enables plug-and-play integration capabilities, which means developers aren’t reinventing the wheel for every project. Where there used to be 20 disparate payment services, for instance, because everything is standardized, there are now four, which markedly reduces maintenance costs.

 “This also provides a stronger foundation for Absa Group’s open banking initiatives, enabling selected services to be securely exposed for integration with FinTech partners and other ecosystem players,” he says. Plus, integrating new channels has become more straightforward, as teams can now leverage consistent, reusable integration patterns. This makes it easier to scale digital capabilities and accelerate delivering new customer-facing solutions.

This project, according to Dutuma, wasn’t just about fixing the old tech, but enabling cloud readiness and creating a leaner, modular application stack that can be used across other markets. Now, Absa doesn’t need to build a unique integration for a wallet in Botswana or for internet banking in Tanzania. There’s a common middleware layer across all regions, allowing countries to independently replace or upgrade core applications without affecting the broader regional footprint. In this way, Absa has essentially dissociated geography from technology to reduce complexity, improve interoperability, and ensure that different systems all speak the same language.

Rosemarie Mosteller | shutterstock.com

Vergleicht man den Bau von Brücken mit der Softwareentwicklung, zeigen sich bedeutende Unterschiede: Denn auch wenn keine Brücke – ähnlich wie ein Softwareprojekt – der anderen bis aufs „Haar“ gleicht, werden sie aus bekannten Materialien mit bekannten Eigenschaften geschaffen.  

Im Gegensatz dazu beinhaltet der Softwareentwicklungsprozess wesentlich mehr „unknown Unknowns“. Was dazu führt, dass er jede Menge Paradoxa beinhaltet, mit denen Developer teilweise nur schwer umgehen können. Wichtig ist aber vor allem, sich ihre Existenz bewusst zu machen – nur so lassen sich die daraus entstehenden Fallstricke umgehen. Insbesondere, wenn es dabei um die folgenden vier Paradoxa geht.

1. Ohne Plan, aber mit Deadline

Zielführend einzuschätzen, wie lange ein Softwareprojekt dauern wird, ist wahrscheinlich die größte Herausforderung für Softwareentwickler überhaupt. Denn darüber lässt sich keine abschließende, verbindliche Aussage treffen. Sicher, man kann den ungefähren Aufwand schätzen – das geht im Regelfall allerdings daneben. Meistens wird der zeitliche Aufwand drastisch unterschätzt.

Wird die gesetzte Deadline dann verpasst, ärgern sich vor allem die Kunden. Sie stecken nicht in der Haut der Devs und durchblicken die Abläufe und möglichen Hindernisse (im Regelfall) nicht. Also sind sie frustriert, weil ihre Software nicht zum vereinbarten Zeitpunkt ausgeliefert wird.  

Auch sämtliche Versuche, mit schicken, agilen Methoden wie Story Points oder Planing Poker zielführender vorhersagen zu wollen, wann ein Softwareprojekt abgeschlossen wird, bringen nichts wenig. Wir scheinen einfach nicht in der Lage, Hofstadters Gesetz (der Verzögerung) zu überwinden.

2. Mehr Mannstärke, mehr Verzug

Stellt ein Manager einer Fabrik fest, dass die monatliche Quote für abgefüllte Zahnpastatuben in Gefahr ist, setzt er mehr Arbeiter ein, um die Vorgabe zu erfüllen. Ähnlich verhält es sich beim Hausbau: Wenn Sie doppelt so viele Häuser wie im Vorjahr bauen wollen, hilft es in der Regel, die Vorleistungen – Arbeit und Material – zu verdoppeln.

Im Fall der Softwareentwicklung verhält sich das völlig anders, wie Frederick Brooks bereits 1975 in seinem Buch „Vom Mythos des Mann-Monats“ herausgearbeitet hat. Demnach hilft es wenig, verzögerte Softwareprojekte mit zusätzlicher Mannstärke retten zu wollen. Im Gegenteil: Gemäß dem Brooks’schen Gesetz verzögert das das Projekt nur noch zusätzlich. Schließlich können neu hinzukommende Teammitglieder nicht sofort zum Projekt beitragen. Sie benötigen Zeit, um sich in den Kontext komplexer Systeme einzuarbeiten, was oft auch zusätzliche Kommunikationsmaßnahmen nach sich zieht. Am Ende verzögert sich dann nicht nur alles noch weiter – es kostet auch mehr.

3. Mehr Skills, weniger Programmier-Tasks

Als Softwareentwickler umfassende Expertise aufzubauen und sämtliche erforderlichen Regeln und Feinheiten zu verinnerlichen, um wartbaren, sauberen Code zu schreiben, nimmt etliche Jahre in Anspruch. Dabei erscheint es auch relativ paradox, dass die Programmieraufgaben mit steigender Erfahrung eher weniger werden: Statt zu programmieren, sitzen leitende Entwickler vor allem in Design-Meetings, überprüfen den Code anderer und übernehmen weitere Führungsaufgaben.  

Das heißt zwar nicht, dass Senior Developer einen kleineren Beitrag leisten. Schließlich sorgen sie in Führungspositionen dafür, dass zeitgemäß und zielführend gearbeitet wird und tragen so wesentlich zum Team- und Unternehmenserfolg bei. Aber am Ende schreiben sie dennoch weniger Code.

4. Bessere Tools, keine Zeitvorteile

Vergleicht man die Webentwicklung von heute mit performanten Tools wie React, Astro und Next.js mit dem Gebaren von vor 30 Jahren (Stichwort Common Gateway Interface), wird klar, dass wir uns seitdem um Lichtjahre weiterentwickelt haben. Doch obwohl unsere Tools immer besser und die Prozessoren immer schneller werden, scheinen sich Softwareprojekte insgesamt nicht zu beschleunigen. Das wirft Fragen auf:

• Unsere Websites sehen zwar immer besser aus, aber sind wir wirklich produktiver?
• Laufen unsere Websites schneller und verarbeiten sie Daten besser?

Natürlich abstrahieren die Frameworks und Bibliotheken von heute viele Komplexitäten. Sie führen aber auch zu neuen Problemen. Zum Beispiel langen Build-Pipelines, Konfigurationsalbträumen oder Abhängigkeitsproblemen. (fm)

Sie wollen weitere interessante Beiträge zu diversen Themen aus der IT-Welt lesen? Unsere kostenlosen Newsletter liefern Ihnen alles, was IT-Profis wissen sollten – direkt in Ihre Inbox!

AI is entering a phase of sustained enterprise adoption. As the technology rapidly advances, organizations are moving beyond isolated use cases and short-term efficiency gains and rethinking how they use AI to create value, meet changing customer expectations and evolve their operating models over the next several years.

That requires a clear end goal, an honest assessment of current capabilities and a practical roadmap for moving from today’s reality to that end goal.

Today, we are seeing five accelerating trends shaping how that transition is unfolding.

LLMs are evolving into AgenticOS platforms

Horizontal LLM providers like Anthropic and vertical AI companies like Harvey are moving beyond standalone AI models and building broader enterprise platforms. These platforms combine AI models with workflows, playbooks, integrations and governance tools inside a single environment, which are beginning to be described as an “AgenticOS.” As a result, the market is beginning to consolidate around a smaller number of platform providers that can simplify procurement, integration, spend management and data privacy compliance.

Context windows have expanded by orders of magnitude

Leading AI models can now process dramatically more information at once than they could just a few years ago, with the amount of information they can analyze in a single interaction expanding roughly 125× since 2023. That shift is making more complex, enterprise-scale work, like large-scale contract review, codebase-wide analysis and multi-document research synthesis, possible. Such capabilities, which once felt cutting-edge, are becoming standard expectations.

John Wei

Token pricing has stabilized at the production tier

After dropping rapidly between 2023 and 2025, the cost of using mainstream AI models has started to stabilize. Today, many enterprise-grade models fall within a relatively predictable range of roughly $2–$3 per million input tokens and about $15 per million output tokens, making costs easier to anticipate and manage.

At the same time, cost-saving features like prompt caching (which can reduce costs by up to 90%) and batch APIs (which can cut costs by roughly 50%) are making AI significantly cheaper to operate at scale. Together, those shifts are making AI spending easier for enterprises to budget, forecast and manage like other core technology investments.

John Wei

AI is functioning as a productivity assistant, not a human replacement.

I had the chance to speak with senior leaders at this year’s WSJ Future of Everything conference, and one theme consistently emerged: despite the hype around AI agents, many companies are still using AI to support human decision-making rather than replace it.

Data shared by the senior leadership team of a prominent AI company at the WSJ conference shows that AI agents consume less than 5% of tokens today, and 84% of enterprise use cases are growth-focused rather than productivity-focused. That is largely because AI workflows still depend heavily on the quality and consistency of inputs. In complex enterprise environments with variable scenarios and edge cases, human judgment, prompt refinement and iterative review remain essential.

As a result, workflows can rarely be fully automated, and many automation gains translate into incremental productivity improvements rather than meaningful headcount reduction without broader operating model changes.

Instead, many organizations are using AI to drive growth, support new business models and enable new ways of operating.

Software development is the leading edge of human-AI collaboration.

In our experience at Integreon, vibe coding has produced a few notable success stories. At enterprise scale, though, it can introduce architectural limitations and sometimes even hardcoding or semi-hardcoded shortcuts that undermine the long-term sustainability of the code. As a result, we primarily use AI coding tools as developer assistants for targeted tasks rather than end-to-end software development. That approach reflects a broader industry trend: despite high-profile tech layoffs, overall demand for developers has remained steady, and demand for developers with AI skills is rising.

Across all five trends, the focus has shifted from automating legacy workflows to assisting human workflows. This is a fundamental change in how work will be organized across the enterprise.

As AI technologies mature and become widely accessible across industries, competitive advantage will increasingly come from strategy rather than the technology itself. Many businesses will have access to the same AI platforms, models and tools. What will differentiate organizations is how they apply those technologies to shape customer experience, operating models and market positioning.

The airline industry offers a useful parallel. Most airlines operate similar aircraft under the same regulatory and labor constraints, yet they differ dramatically in market positioning, customer experience and operational performance. What separates airlines is not the plane itself, but how the business is built around it.

For CIOs and CTOs, choosing an AI platform is no longer the main challenge. The more important conversations now center on where the business is headed and how AI supports that strategy. Leaders must ask themselves questions like:

• Who do we want to become? Most enterprises have mission statements, but far fewer know exactly where they want the business to go over the next three to five years as AI reshapes customer expectations, competition and economics. That answer needs to be concrete enough to guide real decisions.

• What are we choosing not to do? Strategic restraint matters just as much as strategic ambition. AI lowers many costs, making it tempting for organizations to spread themselves across too many initiatives. But without clear boundaries, organizations risk stretching resources too thin.

• Where are we today? That means taking a real look at which parts of the business AI may shrink or disrupt over the next three to five years. Many companies struggle to assess this honestly because those areas still generate revenue today. Sometimes it takes an outside perspective to spot risks internal teams are too close to see.

• What capabilities do we need to succeed three to five years from now? Companies often plan by projecting today’s business forward instead of starting with where they want to end up. Usually, the answer comes down to a few key differentiators, like proprietary data, customer trust or distribution, along with a broader set of capabilities that simply need to be strong and reliable.

• How will we organize work? Enterprises must rethink how work gets done. Most operating models today were built around human labor. Going forward, many workflows will likely be shared between AI systems and human oversight.

• What kind of talent do we need? This can be especially difficult for companies with long histories and established teams. Employees who drove success in the past may not align perfectly with where the business is headed next. Companies will need to think carefully about how experienced employees can help build and support future capabilities.

• Where can we simplify workflows? In many cases, workflows can be reduced to three core steps. First is building context, including defining the goals, data, constraints and decision-making framework. Then comes AI execution, where AI is applied to workflows and tasks. Finally, humans review outputs and make judgment calls.

• Which AI platforms do we actually need? Most enterprises do not have the capacity to effectively manage dozens of AI vendors and tools at once. Every additional platform adds more integration work, governance, vendor oversight and security review requirements. In most cases, organizations are better off making a small number of focused platform bets than constantly chasing the latest AI tool.

Finally, a few thoughts on what to avoid

The best mentors I’ve had taught me to think in three-to-five-year terms. A good strategy should remain relatively stable over that period. Without that consistency, organizations end up resetting direction too often and losing credibility in the process.

Today, I see two common mistakes. The first is staying too anchored to the past, defaulting to reasons something cannot happen because of security, compliance or organizational resistance. The second is the opposite: chasing every new technology simply because it is new. Most enterprises will need to find a middle ground over the next several years.

AI is the aircraft. Strategy is the route.

The companies that pull ahead will not necessarily be the ones spending the most on AI or launching the most pilots. They will be the ones whose leaders answered the hard questions, stayed committed to a direction and learned from mistakes along the way.

Technology will continue to change. Strategy is what will determine who uses it well.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

For the past several years, the default assumption in enterprise IT was that AI would follow the same path as many other workloads and settle into the public cloud. That assumption seemed reasonable on the surface. The hyperscalers had the infrastructure, GPU capacity, managed services, and developer ecosystems. If you wanted to move fast, public cloud AI looked like the obvious answer.

That logic is now being challenged by reality. As enterprises move from AI experiments to AI in production, they increasingly find that the public cloud is a convenient place to start but not the most practical place to stay. Enterprises are wondering if they can afford to base their long-term AI strategies on cost models they do not control, risks they cannot fully contain, and architectures that are optimized for provider scale rather than enterprise economics.

This is why private cloud AI is becoming more popular. Enterprises are not moving on-premises because it’s a fashionable choice. They are moving because, in many cases, it is the financially rational choice.

The expense of token-based AI

The market still treats token-based AI pricing as a stable, mature economic model. It is not. Much of what enterprises pay today reflects a highly competitive environment in which providers are still subsidizing adoption, offering aggressive discounts, and prioritizing market share over normalized margins. That may be good news in the short term, but it is dangerous to assume those conditions will persist.

As enterprises scale their usage, token consumption shifts from an interesting line item to serious financial exposure. A chatbot pilot is one thing. Enterprisewide inference across business operations, customer engagement, knowledge systems, automation, analytics, and embedded software is something else entirely. When AI becomes part of the daily operating fabric of the business, token charges stop being experimental expenses and become recurring utility bills. At that point, even modest changes in pricing can have major budget consequences.

Many tech leaders are now rethinking their assumptions about AI costs, realizing that current pricing may not reflect long-term expenses. As subsidies fade and usage increases, token costs are likely to rise sharply, potentially making large-scale public AI deployments less economically viable. That is the trap enterprises want to avoid. No CIO wants to explain that the company successfully operationalized AI only to discover that a growing bill from a public provider offsets every business gain. Enterprises have seen this before with cloud cost overruns, and they do not want to repeat it with AI.

Hybrid AI is the natural end state

It is becoming clear that the future of enterprise AI is neither all public cloud nor all on-premises. It is a hybrid. The market is maturing beyond ideology and moving toward workload placement based on economics, governance, latency, and control.

That shift matters because not every AI problem requires a giant hosted model. In fact, many enterprise use cases do not. A growing number of organizations are finding that smaller, domain-specific models can perform as well as, and often better than, larger ones for targeted business tasks. Some use tuned models. Some rely on classic machine learning and predictive systems. Some combine retrieval techniques with smaller language models. Others build tightly constrained models tailored to specific operational domains.

These systems are often better suited to private infrastructure. They run closer to enterprise data, can be optimized for predictable workloads, and avoid the open-ended cost profile of external tokenized services. This is especially true when the model is used repeatedly within internal business processes rather than occasionally by a limited set of users. In other words, enterprises are not just choosing private AI because they dislike public cloud pricing. They are choosing it because they are learning to build AI systems that meet enterprise requirements rather than defaulting to whatever is easiest to consume from the outside.

Security and governance

Cost may be the loudest concern, but it is not the only one. Security and governance are becoming equally powerful drivers. Enterprises are increasingly uncomfortable with the idea of sensitive information flowing through public AI tools, public APIs, and user workflows that are difficult to monitor and control. The concern is not abstract. Employees routinely paste confidential information into public AI interfaces to boost productivity. Development teams sometimes move faster than policy can keep pace. Business units adopt tools before governance can catch up. The result is a growing risk of data leakage, unauthorized exposure, compliance failures, and security incidents directly tied to the use of AI.

This changes the conversation. Once AI touches customer records, financial models, regulated data, or other proprietary information, the focus shifts from deployment speed to the risk you introduce to the core of the business. While public clouds can provide strong security, many enterprises prefer tighter internal controls for sensitive AI workloads to ensure better observability, access, data locality, and policy enforcement.

There’s no question that private AI reduces the number of unknowns. It gives enterprises more direct control over where data resides, how models are used, who can access them, and how systems are audited. That does not eliminate risk, but it makes risk easier to manage.

Private AI is harder but worth it

Private AI is not effortless. Building AI on premises or in a private cloud requires investment, planning, specialized skills, operational discipline, and a willingness to own more of the stack. Enterprises must think about infrastructure design, GPU utilization, life-cycle management, model operations, integration, and resilience in ways that public services often abstract away.

That extra work introduces real risk. Some organizations will underestimate the operational burden, some will overspend on infrastructure, and some will struggle to attract the right talent. Even with those challenges, many enterprises are concluding that the cost savings are too compelling to ignore.

Enterprises are not moving toward private AI because it is easier. They are moving because it’s smarter in the long term. They would rather take on more responsibility now than remain exposed to a pricing model that could become unsustainable later. They would rather invest in owned capability than rent critical intelligence from an outside platform with uncertain future economics.

The public cloud will remain important, especially for experimentation, bursting, and select services. But for many production workloads, the balance is shifting. As token costs rise, governance pressures intensify, and organizations become better at building focused models rather than defaulting to giant LLMs, more enterprises will conclude that their most valuable AI belongs closer to home.

As AI adoption grows, I see fellow enterprise leaders realizing that just implementing AI is not enough. We need to develop and adopt the best, fastest and most efficient AI models. It’s not just a matter of pride about who has the shiniest toy; optimizing models for efficiency can be the difference between a failed pilot and an effective business strategy.

At the most extreme end of the spectrum, inefficient use of AI can cost billions of dollars. Sam Altman, CEO of OpenAI, made headlines when he admitted on X that his company loses tens of millions of dollars every time people say “please” and “thank you” to his AI models, even though he added that he feels it’s money well spent.

Model efficiency also matters for those of us not operating at OpenAI’s scale. A more efficient model helps reduce overall costs because it doesn’t require as powerful or expensive hardware, uses less electricity, delivers output faster and can operate with a smaller cloud footprint.

Models that are optimized for efficiency deliver lower latency, improved scalability, increased flexibility and are less likely to drift. In my experience, all of this adds up to higher profit margins, a sharper competitive edge and a faster time to market, which are crucial whether you’re planning to use your model internally or sell it to others.

The new CIO investment dilemma

For a long time, it was believed that hardware must continually increase in power to enable models to grow in size. Then DeepSeek v2 came along and demolished all those theories. It showed that more efficient hardware can deliver equivalent results with less compute power by running smaller, smarter models.

Now, those of us in the CIO seat face a new dilemma: should we increase investment in computing power, focus on hardware or concentrate on software?

In my view, the correct answer is: all the above. AI efficiency is a full-stack problem. Hardware, compilers, runtime and model architecture must be co-designed to work in harmony; otherwise, we’re wasting money and failing to achieve the results we need. Today, choosing GPUs vs. custom accelerators vs. CPUs affects which model optimizations are viable.

Hardware power constraints model capabilities

It remains true that even the most powerful model in the world can’t function without access to the necessary hardware. Hardware performance is ultimately bounded by memory bandwidth, interconnect speed and compute units, no matter how optimized our models are.

This means that scalability depends on interconnects. Multi-node training and large inference clusters hinge on the performance of NVLink, InfiniBand or Ethernet fabric, not just model quality, so decisions about hardware investments or cloud providers can be critical to overall functionality.

“The pace of innovation is directly tied to advances in GPUs, tensor processing units (TPUs) and custom accelerators. The real question isn’t just what models we can build, but whether we have the compute infrastructure to support them,” says Gaurav Dewan, a research director at Avasant. “Models can only grow as powerful as the chips, memory systems and data center networks sustaining them.”

Compute power isn’t everything

That said, in my experience, you can’t just throw computing power at every problem. Choices about hardware and cloud architecture determine how effectively users can tap into the potential of compute resources. Modern AI workloads are often memory-bound rather than compute-bound, so faster HBM, cache hierarchies and interconnects directly lower latency.

What’s more, the energy for computing power is limited. Companies can’t always afford the compute power they want, with 58% saying their AI cloud costs are too high. Cost per inference is hardware-driven and compute is usually the biggest line item in AI TCO. It’s not even easy to find space for enough GPUs, creating board-level power and cooling constraints in enterprise AI. More efficient silicon reduces data center strain, sustainability risk and cost per token/inference.

Additionally, reliability and utilization affect ROI. Features like MIG partitioning, hardware scheduling and fault tolerance determine how fully we can monetize expensive accelerators. Performance per watt is now the bottom line, with CIOs like me striving to get more out of every existing GPU per watt, dollar and square meter. We need to make our hardware more efficient by fine-tuning models and software to maximize capability.

“DeepSeek’s breakthrough suggests that AI models no longer need to scale indefinitely in size and complexity to achieve superior performance. Instead, they can be algorithmically optimized to deliver the same, if not better, results while consuming significantly fewer resources,” explains Matthew Taylor in his post on LinkedIn.

Rethinking cloud strategy in the age of AI

That cost pressure has forced many of us to revisit assumptions we held for the better part of a decade. Cloud computing has reached an uncertain crossroads. The hyperscaler-by-default posture that defined the last era of enterprise IT no longer survives a serious look at AI economics.

When inference costs scale linearly with usage and training runs can consume an annual infrastructure budget in weeks, the question I hear in every CIO conversation is the same: does our cloud strategy still match the workload we are actually running?

In my experience, the answer is increasingly no, at least not without significant rebalancing. Private clouds, written off as legacy not long ago, are quietly making a comeback. The combination of predictable cost structures, tighter control over data residency and the sensitivity of the proprietary data feeding our AI systems is making on-premise and colocation options compelling again, particularly for regulated industries.

At the same time, purpose-built neoclouds for GPU workloads, along with sovereign clouds responding to jurisdictional and data-protection mandates, are steadily chipping away at the dominance of AWS, Azure and GCP. None of these alternatives replace the hyperscalers outright, but they are forcing every CIO I know to think about cloud as a portfolio rather than a single vendor relationship.

What I have found is that navigating this shift takes more than a procurement decision. It takes a clear-eyed view of where each workload genuinely belongs. Training, inference, retrieval, fine-tuning and experimentation each carry different cost curves, latency profiles and data-gravity considerations. As organizations move towards the agentic AI era, the underlying data platform becomes equally important, requiring architectures that can support multimodal data, real-time processing and governance at scale.

The enterprises I have seen handle this best treat cloud strategy as an ongoing exercise in workload placement, not a one-time platform commitment.

That is also where the conversation tends to outgrow internal teams.

As AI moves from pilots to production, the questions get harder: how to architect data foundations that survive model churn, how to govern AI without strangling it, how to translate technical efficiency into measurable business value. I have seen organizations lean on specialist partners to think through these problems alongside them. Among the consultancies working at this intersection is Artefact, founded in Paris and operating across data strategy, AI engineering and enterprise transformation. Its work includes governance, platform development, operating models and workforce enablement—areas that have become increasingly important as organizations move from AI pilots to large-scale deployment.

What I find useful about these consultancies is not the technology recommendations themselves; it is the pattern recognition they bring from seeing similar cloud and AI transitions play out across geographies and sectors. In a moment when every CIO is rewriting the playbook simultaneously, that outside vantage point matters more than it used to.

Hardware is often underused and misused

A lot of hardware goes unused or underutilized. Often, GPUs sit idle due to deployment complexity and data infrastructure bottlenecks, so enterprises don’t see the value of the compute power they’re paying for. When data and computing are on two separate chips, compute is wasted moving data between the two locations.

Likewise, models that exceed accelerator memory or require excessive HBM traffic suffer steep latency and cost penalties. Optimizing models to align with hardware means that all the compute power is being put to good use.

Techniques like operator fusion, activation management, fine-tuning smaller models, pruning unnecessary parameters and memory-aware architectures keep more of the model resident on the accelerator, reduce unnecessary read/write cycles and combine steps so data is touched fewer times.

Kfir Aberman, founding member at Decart AI, explains this approach. “Our solution to this was to optimize our kernels for how [Nvidia GPU] Hopper works. Essentially, we created a single ‘mega kernel’ that enables the chip to process all of a model’s computations in a single, continuous pass. By doing this, we eliminate all of the stopping, starting and data movement, allowing more of the GPU to be utilized more of the time, speeding up processing by an order of magnitude.”

When models match accelerator characteristics such as tensor core shapes, SIMD widths and kernel libraries, this keeps expensive silicon working effectively and translates theoretical FLOPs into real throughput.

More hardware can’t overcome model mismatch

Another way that organizations undermine ROI on their own AI investments is by ignoring coordination efficiency.

They’ll buy large GPU clusters but pay little attention to what seem like minor issues with batching and alignment. Unfortunately, when batch sizes are wrong, work is split inefficiently and network links become bottlenecks, you see expensive but underutilized clusters.

Ultimately, more GPUs don’t guarantee more performance. Parallelism and batching must match the system topology. Effective scaling depends on aligning data, tensor and pipeline parallelism and batch sizing with the actual interconnect bandwidth and node configuration.

The magic happens when model and hardware come together

The lesson that those of us in CIO roles are learning is that symbiosis between model and hardware is critical. Code determines what our AI can do, hardware determines how efficiently we can afford to do it and co-design determines whether our AI program scales economically and successfully.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

I still remember the first time we tried to simulate a large-scale water sequence nearly two decades ago. It was a simple brief — “make it look real.” What followed was anything but simple. Machines struggled, artists waited and we often had to compromise between realism and deadlines. Back then, simulation in VFX felt like a powerful but unpredictable beast — something you respected, but never fully controlled.

Fast forward to today, and that beast has grown bigger, faster and far more demanding. As someone who has spent over 25 years in animation and VFX technology, I’ve seen simulation evolve from a niche capability into the backbone of modern visual effects. Whether it’s oceans, explosions, cloth, smoke, or destruction — simulation now defines realism. But with that realism comes a level of complexity that is reshaping how studios think, build and operate their pipelines.

This is the story of that shift — and how we’re learning to tame it.

When realism became data

In the early days, simulations were relatively lightweight. A smoke sim might take hours, maybe a day. Today, a high-resolution fluid simulation can generate terabytes of data for a single sequence.

That’s the first big change: Simulation is no longer just computation — it’s data generation at scale.

Every frame we simulate produces layers of information — velocity fields, density grids, particle caches, mesh outputs. Multiply that across hundreds of shots, and suddenly your pipeline isn’t just about rendering images — it’s about managing massive datasets.

I’ve seen studios hit a point where storage, not compute, became the bottleneck. Artists weren’t waiting for simulations to finish — they were waiting for data to move.

This shift forces a fundamental rethink:
We are no longer just running simulations. We are managing simulation ecosystems.

Lessons from other worlds

What’s interesting is — VFX is not alone in this journey. Other industries faced similar challenges earlier, and there’s a lot we can quietly borrow from them.

In weather forecasting, global climate models run on massive HPC systems, producing petabytes of data daily. But meteorologists don’t store everything forever. They prioritize derived insights over raw data — keeping summaries, patterns and key states instead of full datasets.

In genomics, sequencing a single human genome produces hundreds of gigabytes of raw data. Labs long ago realized that recomputing certain stages is cheaper than storing everything indefinitely. So they intentionally discard intermediate data — but keep the pipeline reproducible.

In autonomous driving, simulation environments generate enormous synthetic datasets. Companies don’t just store scenarios — they index them semantically: “Pedestrian crossing at night in rain,” for example. That makes retrieval intelligent, not just archival.

The pattern across all these domains is clear: They don’t fight data growth — they design around it.

The rise of HPC in VFX

To handle this scale, High Performance Computing (HPC) has become essential.

Years ago, a render farm was enough. Today, simulations demand tightly coupled compute — clusters with high-speed interconnects, parallel file systems and optimized schedulers. In many ways, VFX studios now resemble scientific research labs.

But here’s the catch:
More compute doesn’t automatically mean better outcomes.

Throwing thousands of cores at a problem can speed things up, but it also increases cost, complexity and coordination challenges.

Here’s a practice I’ve seen work well, but is rarely talked about:
treat compute like a budget, not a resource pool.

Instead of unlimited access, assign “compute envelopes” per sequence or department. This forces smarter iteration — teams think before re-running simulations blindly.

Another overlooked idea: Simulate at multiple fidelities intentionally, not progressively.

Most pipelines go low → mid → high resolution. But some studios now run parallel exploratory sims at different fidelities and let ML or heuristics decide which path to invest in further. It reduces dead-end iterations dramatically.

Complexity is no longer in the solver

Traditionally, we focused on improving solvers. Today, the hardest problems are about context — understanding what was done, why it worked and whether it can be reproduced.

Questions like which version was used, what parameters changed, or how upstream assets influenced the result are now central to the pipeline.

A practical way to address this is to treat each simulation as a uniquely identifiable event. By capturing not just inputs but also solver versions, environments and dependencies, teams can create what I often call a “simulation fingerprint.” If anything changes, the fingerprint changes — making reproducibility far more reliable.

The power of structured data

Metadata is no longer optional — it’s foundational.

However, the real value lies not in storing metadata, but in using it actively. When structured correctly, metadata can guide decisions — helping systems route jobs, anticipate failures and recommend better configurations.

At that point, the pipeline begins to evolve from a passive system into something more adaptive — one that supports teams rather than slowing them down.

Learning from the past: Machine learning as a guide

Machine learning in VFX is often misunderstood as a replacement for physics. In reality, its strength lies in learning from experience.

Every simulation leaves behind valuable data. When used correctly, this data can help teams avoid repeating work. For example, before launching a new simulation, systems can check whether something similar has already been done and suggest reuse or adaptation. Similarly, early signals in a simulation can indicate whether it is likely to fail, allowing teams to stop it before wasting hours of compute.

In this sense, machine learning becomes an intelligence layer — quietly improving efficiency without replacing the underlying physics.

Rethinking storage: Not everything needs to live forever

One of the hardest mindset shifts is accepting that not all data needs to be preserved.

Instead of treating storage as infinite, a more sustainable approach is to prioritize what truly matters. High-resolution outputs are retained for final shots, while lighter representations can support iteration history. In many cases, recomputing data is more efficient than storing it indefinitely.

This is a model that other industries have adopted successfully — and one that VFX is gradually moving toward.

Hybrid HPC: The new normal

Most studios today operate in a hybrid model, combining on-premise infrastructure with cloud resources.

The challenge, however, is not where the compute exists — it’s how decisions are made. Choosing where to run a simulation depends on factors like data location, system load and cost efficiency.

One principle that consistently proves effective is simple: Move compute closer to data whenever possible. Transferring large datasets is often far more expensive than relocating compute.

A simple way to think about it

A modern simulation pipeline is less like a factory and more like an airport — constantly managing traffic, prioritizing tasks and adapting to change.

At its core, it follows a simple loop: Data leads to compute, which produces more data, which informs decisions — and the cycle repeats.

The studios that succeed are the ones that optimize this loop as a whole, rather than focusing on individual steps.

What breaks next?

Looking ahead, the pressure will only increase.

As real-time expectations grow through virtual production, and AI-generated environments increase the demand for simulations, pipelines will be pushed further. Storage costs will become more significant, and energy consumption will no longer be ignored.

The next bottleneck may not be obvious — but it will arrive.

Looking back, the challenges we faced 25 years ago seem simple compared to today. But the goal remains unchanged — to create believable worlds that captivate audiences.

Simulation has grown from a tool into an ecosystem — of compute, data and decisions.

We may never fully tame the complexity — but we can learn to guide it.

Because in modern VFX, the challenge is no longer creating complexity — it’s choosing when not to.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

AI has emerged as a top priority for businesses and a vehicle for transformation, as evidenced by Accenture research: 97% of executives believe AI will transform their company and industry. But as companies move from AI pilots to scaling AI across the enterprise, we have had repeated conversations with CIOs and technology leaders who are arriving at the same uncomfortable realization: AI stack decisions are not easily reversible.

Unlike earlier eras of enterprise IT, where abstraction layers insulated applications from hardware choices, today’s AI stack—the infrastructure, technologies and frameworks that powers AI systems – tends  to be tightly co-engineered, with stronger dependencies in the underlying compute layers. Choices made about models, runtimes and compute platforms now shape cost structures, performance ceilings and strategic flexibility. AI-ready infrastructure has re-emerged as a new source of differentiation, and with it, a new kind of vendor lock-in.

At the center of this shift is the move from training – building AI models – to inference, where those models are used in production to generate outputs from new data. While early attention focused on the cost of training large models, enterprises are now scaling AI across the organization, running models continuously across workflows. This shift significantly changes the economics of AI.

For instance, agentic AI is reshaping infrastructure architecture and platforms because inference is becoming persistent, stateful and increasingly data intensive. As AI Factories scale, the focus is shifting from peak model performance toward sustainable token economics, where the key differentiators are lowest cost per generated token, power efficiency and infrastructure utilization at scale. In this environment, achieving those outcomes requires full-stack optimization across compute, networking, memory, storage and data fabrics, curated and integrated across ecosystem partners. Secure multitenancy and confidential computing are becoming core design principles, and enterprise AI is now ready to be industrialized at scale.

Modern AI infrastructure is a strategic bet

What makes AI infrastructure different is not just scale, but integration. Modern AI systems are built on tightly co-engineered stacks where GPU accelerators, high-bandwidth interconnects, compilers and runtimes are designed in tandem to maximize throughput and efficiency for AI workloads.

To get the massive computing power required for AI, providers design their hardware and software to work exclusively with one another. This has shifted enterprise decision-making from choosing hardware one piece at a time to committing to ecosystems. And that commitment carries consequences.

In traditional IT environments, applications could also generally move across environments with a manageable amount of effort. In AI systems, that assumption breaks down. What appears portable at the model or application layer often depends on deeply optimized components underneath that layer, such as memory handling and compiler frameworks like CUDA or ROCm that are fine-tuned to specific hardware.

We find it useful to think about AI systems as a layered structure:

Accenture

While upper layers retain some flexibility, dependencies increase as you move downward. Changing your foundational AI provider often means having to rebuild and re-optimize large portions of your technology from scratch.

This is why infrastructure decisions in AI feel less like procurement choices and more like strategic, high-stakes bets.

Why switching AI platforms is harder than it looks

In theory, switching platforms should be straightforward. Models can be retrained, applications rewritten, and infrastructure replaced. In reality, the cost of switching extends far beyond hardware or licensing.

• The first challenge is engineering effort. Migrating to different platforms requires engineers to revalidate model behavior, re-tune inference pipelines, and rebuild performance baselines. During this period, teams spend most of their time stabilizing and not innovating.
• The second challenge is hidden dependency. Over time, system optimization becomes tied to a specific stack. This might include latency expectations, batching strategies, orchestration logic and even human workflows. These ties are not always obvious, but they shape how systems behave in production.
• The third challenge is timing. There is never a convenient time to migrate, especially factoring in rising AI infrastructure and inference costs, competitive pressure or scaling demands. Organizations are often forced to switch platforms precisely when disruption is hardest to absorb.

Rethinking performance vs control

Despite these barriers, organizations do switch. In our experience, this typically happens under three conditions.

One common trigger is when the opportunity cost of staying begins to outweigh the cost of leaving. As performance gaps widen across competing ecosystems, inefficiencies accumulate to the point that remaining on the current platform is no longer viable. Another driver comes from shifts in vendor dynamics. Pricing volatility, supply constraints, or misalignment in product roadmaps can introduce risks that force a re-evaluation. Finally, regulatory requirements, data sovereignty constraints or geopolitical shifts can force platform changes regardless of technical preference.

Across all three strategies, one principle stands out. Lock-in is not inherently negative, and openness is not inherently superior. Timing matters more than ideology.

Given these dynamics, the central question for CIOs is not how to avoid lock-in, but how to manage it deliberately. This represents a significant shift in strategies that previously considered vendor lock-in as a detriment. In practice, we see three broad approaches emerge, each reflecting a different balance between performance and control.

Some organizations take a performance-first approach. They optimize deeply within a specific ecosystem because performance directly drives business outcomes. Eli Lilly’s AI Factory is a strong example. The company has invested heavily in a tightly integrated NVIDIA-based stack to maximize throughput and utilization. In this case, infrastructure is a competitive lever and not merely a support function. Higher switching costs are accepted because near-term performance advantages are decisive.

Others lean toward a portability-first model. These organizations prioritize flexibility, governance, and long-term independence over absolute performance. BNP Paribas illustrates this well through its internal LLM platform built on open-source models and controlled infrastructure. By retaining ownership of the stack, the bank ensures data sovereignty, regulatory alignment and predictable cost.

A growing number are adopting a hybrid approach. Rather than applying a single strategy across the enterprise, they segment workloads based on sensitivity to performance, cost and governance. For example, in late 2024, JPMorganChase outlined its approach at a leading cloud and technology conference. It described combining a firm-wide internal AI platform with cloud-based services to move generative AI into production at scale. This reflects a broader enterprise pattern of pairing internally controlled environments with external ecosystems to balance control, scalability and cost.

A performance advantage is only valuable if it lasts long enough to justify the lock-in it creates. Similarly, portability only matters if the ecosystem evolves in ways that make switching worthwhile. This is where many organizations struggle. They evaluate platforms based on current benchmarks rather than the direction of the ecosystem.

In practice, we encourage leaders to track a set of evolving signals. These range from the maturity of open compiler ecosystems and improvements in cross-platform runtimes, to shifts in performance per watt and increasing regulatory focus on sovereign AI. Together, these indicators help determine whether the industry is moving toward convergence or further fragmentation.

Conclusion

AI is forcing a reset in how technology leaders think about IT architecture. The goal for CIOs is no longer to eliminate dependency, but to choose it consciously and manage and revisit that choice over time.

In our experience, the most effective organizations treat this as a dynamic problem. They evaluate where performance truly differentiates them, where flexibility protects them, and how quickly those boundaries are shifting. They also recognize that some degree of re-platforming is inevitable and plan for it, rather than treating it as a failure.

Ultimately, AI infrastructure strategy is not about optimizing for today’s conditions. It is about getting ready for where the ecosystem is going next. The leaders who navigate this well are not those who avoid lock-in entirely, but those who understand when to embrace it when to limit it and when to move beyond it before the market forces that decision on them.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Microsoft has been pushing hard to make Visual Studio Code a major way to consume its AI services, mostly in the form of GitHub Copilot. GitHub Copilot’s deep integration with VS Code brings many conveniences — inline autocomplete, for instance — but it’s frustrating for those, like me, who would rather use another model provider, or even a locally hosted LLM, for those functions.

Visual Studio Code 1.122 introduced a new feature, “Use BYOK [Bring Your Own Key] without a GitHub sign-in,” that allows you to “use chat, tools, and MCP servers in air-gapped or restricted environments where GitHub sign-in isn’t possible.” More importantly, it “enables fully offline workflows with local models like Ollama.”

In other words, you can now use locally hosted LLMs for chat, tools, and Model Context Protocol servers inside Visual Studio Code. The one thing you still can’t do is use a local LLM for inline and next-edit suggestions — at least, not without additional tooling.

Choosing a model for BYOK mode

If you want to use a local LLM with VS Code’s bring-your-own-model system, the first thing you need is a way to host the model. VS Code lacks a model-hosting mechanism of its own, although it’s conceivable that a VS Code extension may offer something like that in the future. That said, hosting models is complicated enough that a dedicated app is really needed for the job.

One easy way to host models is via a product like LM Studio, a convenient GUI for standing up, serving, and managing LLMs on one’s own hardware. The model host does not have to be the same system you run VS Code on, either. It can be on a server box you control, or on a cloud instance.

The choice of model is also important. Many models are powerful but won’t run well on commodity hardware because they’re simply too big. A good rule of thumb is to choose a model that fits into existing VRAM, along with the memory needed for a sizable token context (the more, the better). Also, the model should be suited to coding and development work. Some models in this vein that fit comfortably into 8GB VRAM include:

• Gemma4 (effective 2 billion parameters version)
• Qwen3.5 9B
• Codestral 22B v.0.1 (proprietary license)

Setting up BYOK mode in VS Code

Once you have a model up and running, you can integrate it with Visual Studio Code. If you’ve disabled VS Code’s AI features, you will need to turn them on. Make sure the setting chat.disableAIFeatures is turned off. You can find it in Settings | Chat | Miscellaneous.

Third-party language models are managed through Visual Studio Code’s language model list. Press Ctrl-Shift-P and type Manage Language Models to open the list of existing language models.

Foundry

First you will see a list of the built-in models, which are all externally hosted. To add a new model, select Add Models at the top right and select Custom Endpoint.

You’ll then get a series of prompts:

• Group Name: This is “Custom Endpoint” by default, but you can choose any name you want. The name is strictly for organizing the model list and doesn’t affect things like model recognition or connectivity.
• API Key: If you’ve configured LM Studio to use an API key for serving models, provide it here. If you’re hosting the model locally and you haven’t explicitly set up API keys, you can leave this blank.
• API Type: The options here are Chat Completions, Responses, and Messages. Most of the time you’ll want to use Responses, as it’s the most general-purpose option of the three.

Once you finish providing those answers, you’ll be dropped into a modal editor for a JSON file that holds the details about the endpoint you’re configuring.

Foundry

You’ll need to provide a few more details by typing them into the labeled fields:

• id: A text field that uniquely identifies this particular entry. The choice of ID is pretty much arbitrary; if you’re using only a single model, the ID could be the model name.
• name: The name of the model that is used to identify it on the model server. In LM Studio, you can get this name by clicking on My Models in the main interface, then selecting the three-dot icon for the model in question and clicking Copy Default Identifier. For Qwen 2.5, for instance, name might be something like qwen2.5-coder-7b-instruct.
• url: The URL to the server’s endpoint. On LM Studio, this defaults to something like http://127.0.0.1:1234/v1. The /v1 at the end is important because that endpoint is used for autodiscovery of models and their capabilities.

The other fields generally don’t need editing. Most models have tool calling functionality. If you know for a fact that the model you’re using doesn’t have vision support, then set vision to false.

Once you have these fields filled in, you can close the modal editor to save the changes. If you reload the Manage Language Models page, you’ll now see your new endpoint:

Foundry

You should now be able to launch the chat window and use the defined model for conversation and utilities:

Foundry

One current, and major, limitation of Visual Studio Code’s BYOK functionality is that it only works for chat and utility tasks. It doesn’t allow you to use a local model for inline suggestions or code completions. The only way to take advantage of local models for expanded functionality with VS Code is to use a third-party tool like Continue.

It isn’t clear if Microsoft will eventually lift this restriction. GitHub Copilot integration in VS Code is a large part of how Copilot as a service reaches its target audience. For the time being, you can certainly use third-party and local models for a significant part of your AI-assisted development work in VS Code, and you can close the functionality gap with additional tooling.

Widespread enterprise adoption of AI has created a pressing need for security solutions — a tall order given that AI’s reach into organizational infrastructure and data is enormous and continues to grow.

Moreover, where an organization sits on the AI maturity curve impacts its security needs. Trail of Bits CEO Dan Guide describes the AI journey as a migration from AI-assisted, where AI tools are used on existing workflows; through AI-augmented, which uses new workflows based on AI; to the AI-native organization, where AI “becomes a core participant in the delivery and operations of a business.”

Those three stages require very different approaches to securing AI. They also present challenges for AI security vendors, whose platforms must fit in multiple places in a corporate network and interact with a broad spectrum of applications — especially as agentic AI expands. As analyst David Linthicum recently posted, “the conversation now has to shift from model fascination to operational discipline. The question is how those agents should be governed once they begin touching workflows that affect customers, employees, suppliers, compliance, and revenue.” 

Making matters worse is that the average enterprise manages 37 agents, with more than half running without security oversight or logging, according to Microsoft’s 2026 Cyber Pulse report, which also found that, while 80% of Fortune 500 companies use active AI agents, only 10% have a clear strategy for managing them.

That lack of strategy also opens the door for attackers to abuse corporate AI systems for malicious purposes, as the recent exploit of Meta’s account recovery using chatbots demonstrated.

The trick to securing AI systems is in understanding how much protection is needed and where it should be applied in the expanding AI universe. While one could rent a well-meaning AI agent called Sentry for $7,400 per month to automate the daily work of a SOC analyst, many organizations rolling out AI across their business would be best served by considering AI security posture management (AI-SPM) tools.

Over the past two years, this emerging field has matured, with many security vendors incorporating or acquiring SPM features as part of their general security product portfolio.

Some vendors, such as SentinelOne and Concentric, don’t specifically sell AI-SPM per se, but offer an SPM tool that is part of a larger package of AI security services. Others offer AI-SPM in conjunction with their other SPM tools or CNAPP security offerings. Some vendors, such as Cyera and Palo Alto, offer multiple AI-SPM packaging alternatives with differing feature sets.

Choosing the right product requires careful examination of the roster of features and integrations each product offers to ensure that it doesn’t duplicate existing security tooling or worse, leave important coverage gaps.

Here we take a deeper look at the AI-SPM product category, with a breakdown of offerings from 14 of the leading vendors in this increasingly important security ecosystem.

AI security posture management explained

AI security posture management is an evolving cybersecurity discipline focused on ensuring the integrity and security of AI and machine learning systems. AI-SPM encompasses strategies, tools, and techniques for monitoring, assessing, and enhancing the security of AI models, data, pipelines, applications, and services, even as threats to those entities continually evolve.

In the past, security posture management tools were designed for two situations: to protect general cloud operations against misconfigurations and abuse, which is the province of cloud security posture management tools; and to protect against data leakage or malware infections, which is the province of data security posture management tools. With the rise of AI and large language models (LLMs), a third SPM product category is needed to check AI cloud services and their SDKs (like Hugging Face Transformers or Azure Open AI SDK) to prevent model abuses. This is because numerous studies have documented how AI training data can be the subject of an attack or how bad data can be injected into models to manipulate results, including creating malicious backdoors for attackers to use to enter your enterprise.

The latest reports about attacks on AI and AI abuse can help you better understand the scope of security challenges rapidly evolving today. MITRE continues to enhance its comprehensive database of adversary tactics — Adversarial Threat Landscape for Artificial-Intelligence Systems (ATLAS) — based on real-world attack observations. ATLAS currently spans 170 techniques and 57 case studies. MIT researchers also maintain a growing database of more than 1,700 AI-related risks that they have observed from various AI sources. Another great source of AI-related attack methods is from the Open Worldwide Application Security Project (OWASP), which maintains a Top 10 list of LLM exploits. Security managers should examine them before choosing any AI-SPM product. They should also consult Richard Stiennon’s Guardians of the Machine Age, the most comprehensive collection of general security vendors, listing more than 100 AI security vendors. The printed book offers a deeper dive into the specifics of these tools.

The AI-SPM vendor landscape is quickly evolving, as incumbent security vendors have made numerous acquisitions. Palo Alto Networks bought Protect.ai last year; Cato Networks acquired Aim.security; Orca acquired Opus for AI agentic security; SentinelOne acquired Prompt.Security; Varonis acquired a variety of companies, including Cyral, SlashNext, and AllTrue.ai; and Google acquired Wiz.

Why enterprises need AI-SPM

AI-SPMs have been designed to protect enterprise networks and applications from a range of threats to AI systems. Just like no modern business would assemble a network without an appropriate firewall, AI-SPMs “ensure that AI models stay explainable, fair, accountable, transparent and equitable,” Forrester analyst Andras Cser tells CSO. “Further good security hygiene dictates that AI infrastructure should not be allowed to be used as a steppingstone for hackers for lateral movement and data exfiltration, and should include policies to prevent and fix configuration drift.”

AI-SPM can also help organizations standardize on a series of AI policies, procedures, tools, and workflows that can boost their security. Guido’s talk — linked above — is chock full of suggestions on how Trail of Bits accomplished this.

Major AI-SPM trends and product features

All AI-SPM vendors make use of agentless configurations, accessing cloud-based models and leaving data on their existing platforms. This is both a security measure and to avoid moving the massive data repositories involved across the internet.

AI-SPM vendors also make use of AI-related mechanisms to classify and track these vast data collections and to protect them against potential abuse and attack. Many have integrated their AI-SPM solutions in one of three directions:

• Bolting AI-SPM onto their existing cloud or data SPM platforms with rules, compliance checking, best practices, and protection policies that bridge all three types of security postures.
• Stitching AI-SPM into their general AI security product that can be used to formulate AI-specific policies and perform AI-based red team and penetration testing in an effort to protect AI pipelines and workloads and uncover ways that shared AI services and platforms could be compromised.
• Incorporating AI-SPM to help identify sensitive data referenced by an AI model and to examine training data exposed to a third-party or external application.

Some vendors, especially established security vendors such as CrowdStrike, Proofpoint, Palo Alto, Varonis, and Wiz, have hundreds of third-party integrations that cover the AI waterfront (such as AI assistants and model suppliers) and general IT security arena (such as development pipelines, data feeds, and tools such as SOAR and SIEM). All three types of integrations can provide better guiderails and limit an AI’s blast radius.

But AI-SPM is still evolving. Some vendors’ tools just perform a top-level inspection of one or two services from each of the big three cloud platforms’ AI services (Amazon, for example, has dozens of AI-related service offerings), whereas others (such as Palo Alto Networks, Cato, Cyera, Varonis, and Wiz) take a deeper dive, performing a more comprehensive examination of AI data from the AI vendors themselves and other model sources.

There are two open source efforts as well: Orca’s GOAT is a free learning platform that is based on the OWASP top 10 risks. Palo Alto’s Protect.ai has its collection of open-source tools on GitHub for scanning models and discovering AI interactions and automated red teaming called ProtectAI OSS. However, neither of these projects has been recently updated.

How to choose an AI-SPM tool

Here are several considerations when deciding on the best AI-SPM tool for your enterprise: 

1. Does the vendor work with your existing security tool collection? This has two dimensions: integrating with other SPM products (such as data or cloud protection), and integrating with third-party tools such as SOARs, SIEMs, or DLP products. We have included some vendors that don’t have a specific AI-related SPM (such as Concentric and CrowdStrike) but have deeply embedded AI protection into their platforms.
2. How deep is the coverage across the cloud platform providers? The big three (AWS, Azure, and GCP) have many services that touch various aspects of AI, and some products only work with a few of them, or only connect with PaaS security “hubs.”
3. Does the vendor continuously scan your infrastructure looking for vulnerabilities? AI can be quickly adopted and is very dynamic, so discrete scans are less useful.
4. How important is having a tool that can help with AI red teaming? Understanding the dynamic nature of how AI operates means having a different approach to penetration testing, and this can be a very useful feature. Only a few vendors offer this feature (such as Concentric, Palo Alto Networks, and Varonis).

Leading AI-SPM vendors and products

We reached out to a range of leading AI-SPM security vendors to demonstrate their AI-related tools. Below are more details about each of the 14 we had the opportunity to preview. We have also summarized each vendor’s offerings in the features table, which also provides links, when available, to pricing and third-party integration details. Several vendors didn’t respond to our inquiries, including Baffle.io, Invicti, SecurityCompass, Tonic Security, and Zscaler.

Arthur.ai

Arthur.ai’s platform is a single product that offers deep PaaS coverage with both AWS and Google Cloud Platform, although unlike other AI-SPMs it doesn’t offer a wide range of third-party integrations. It includes application runtime security protection. It also scans network traffic continuously and watches for agent activity, along with policy guardrails to protect against prompt injection and sensitive data leakage. It includes behavioral analytics and governance that catch abusive agentic activities. There are free and paid versions starting at $10,000 annual plans for smaller networks.

Cato Networks AI Security for End Users

Cato Networks AI Security for End Users is one of three separate AI security packages that work together with Cato’s SASE platform, the other two being protection for applications (both runtime and across the software development lifecycle) and for real-time agentic operations. The three AI packages are meant to be purchased together to provide audit trails showing what users are doing with their AI tools and to help understand and illustrate the risks. Cato’s tools can also prevent prompt injection and data leaks and find compliance blind spots. Its platform has a wide collection of third-party integrations, including CrowdStrike, Microsoft, and Splunk SIEMs, and various data sources such as Google’s Chronicle and Rapid7. Cato Networks did not reveal pricing.

Concentric AI and Data Security Governance

Concentric sells a DSPM platform labelled “AI and Data Security Governance.” There is no specific AI tool, although AI pervades its product in a variety of places, including scanning various models for prompt injection, automated remediation, and the discovery and classification of data flows. It offers a wide collection of third-party integrations. On the AWS Marketplace, it sells an entry-level version for $50,000 per year that covers up to 25TB of data, with higher fees for larger data collections.

CrowdStrike Falcon AI-SPM

CrowdStrike Falcon AI-SPM is not a separate product, but part of the overall Falcon Cloud security platform. It can correlate risk findings with other security services monitored by the full Falcon platform. It includes discovery of AI services and models across a variety of cloud platforms, including containers and virtual images, and can detect misconfigurations and dependencies with other software. It scans OpenAI, Amazon Bedrock, Amazon SageMaker, and Vertex AI models. Falcon has more than 250 integrations available to a wide collection of third-party security tools. You can request a free 15-day trial, but no further pricing information was disclosed.

Cyera AI Guardian

Cyera.io specializes in data file level classification. It packages its AI-SPM product in two separate bundles: either with its flagship DSPM product that has added what you might think of as AI-enriched data link protection as part of the default product’s features, or with a more complete set of security features called AI Guardian. Cyera also offers a specialized add-on module used for Microsoft Copilot data scanning that can detect data used by insiders, for example. Cyera’s AWS Marketplace pricing can be found here and starts at $50,000 per year. 

Guardrail Technologies Traffic Light for Code and AI

Guardrail Technologies Traffic Light for Code and AI is designed to be a simple way to flag potential AI abuse by scanning AI-generated code and returning a red/yellow/green result to indicate potential for compromise. There is no remediation, but the tool integrates across the major AI vendors, including Anthropic, Azure Open AI, Hugging Face, and AWS Bedrock, and general security tools such as Wiz and Snyk. Guardrail has a custom AI security consulting business as well called AI Guardian. Very transparent pricing page and a 60-day free trial is available.

Microsoft Purview

Microsoft has bundled its various security posture tools into its Purview offering, which includes a series of AI-based Copilot apps, data SPM and classification tools, and data loss prevention extensions tuned to its various SaaS platforms such as 365, Azure, and Windows endpoints. This extends the AI security features that were originally part of its Defender for Cloud offerings. It has a limited number of third-party integrations. One-month free trials are available, and the entire suite is available for $12.60 per month per user. Microsoft has stepped up its involvement with AI with its Scout, a collection of autonomous AI agents built on top of OpenClaw. It is designed to work with its applications, using built-in security and privacy controls.

OneTrust AI Governance

OneTrust offers AI Governance, a platform that automates compliance and provides continuous monitoring of the AI landscape, across the software lifecycle starting with any AI usage at the beginning of any build. It can detect policy violations, and which AI agents are running. It offers a series of third-party integrations such as Amazon’s Bedrock and Sagemaker; Azure Foundry, ML Studio, and OpenAI; Databricks Unity Catalog and ML flow; and Google Vertex. Its subscription price is based on the number of admin users and number of AI inventory records, although no specifics were provided.

Orca AI-SPM

Orca Security’s AI-SPM is tightly integrated into the company’s security platform. It continues to expand its features, offering detections of more than 50 AI models, including training data and runtime threats, remediation, and support for Model Context Protocol to connect to other Orca-based telemetry. It continues to expand its nearly 100 integrations across SIEM and SOAR systems and various cloud providers’ services. For example, it works with AWS S3, SQS, SNS, CodeBuild, CloudTrail, and Security Hub. It comes with dozens of best-practice security rules that initially focused on compliance. It also alerts when sensitive data is detected inside models and when secrets are exposed. Orca’s overall security platform shows an AWS Marketplace annual pricing that ranges from $84,000 to $360,000, depending on the number of workloads scanned.

Palo Alto Networks AIRS AI Security

Palo Alto Networks has been busy acquiring point security vendors (Dig, ProtectAI, and an offer on Portkey) and incorporating their code into its two major product lines, Prisma and Cortex. You can purchase AI-SPM functionality in either Palo Alto product line, but they cover different aspects of the AI ecosystem. Cortex offers AI-SPM alongside the data and cloud SPMs integrated into the CNAPP suite. Prisma offers AI-SPM as part of a total AI security package called AIRS AI Security, which includes runtime protection, model scanning, and a more comprehensive platform. We focus on AIRS AI, which supports top-level scans of Amazon, Google Cloud, and Azure AI services to discover AI content and can classify and examine model data and secrets and comes with many built-in AI-related policies. Prisma has a long list of third-party integrations, including significant depth in AWS security services. That link will also take you to detailed instructions on how to set up these integrations. To complicate matters further, Palo Alto also sells a separate Prisma secure browser extension that works with these products to protect your endpoints, and that originated from technology it purchased from Talon Cyber Security in 2023. While pricing was not disclosed, our estimate is that AIRS will cost in the low six figures annually.

Proofpoint People Protection Platform

Proofpoint includes a general AI security product as part of its People Protection Platform that covers a wide range of protective services integrated across its other non-AI security tools. It provides runtime inspection of potential AI misconfigurations, as well as policies that include detection of agent, tools, and MCP connections, and it can generate forensic audits of AI interactions. Proofpoint’s general security platform starts at $96,000 annually on AWS Marketplace. It has several integrations with third-party services across the major cloud platform providers.

SentinelOne Singularity Platform

SentinelOne’s Singularity platform offers several AI protective features, including misconfiguration detection, attack path analysis, automated AI inventory and remediation, and integration with a variety of AI PaaS platforms such as Azure OpenAI, Google’s Vertex AI, and various AWS services. It is bundled within the company’s Cloud Native Security tool. Some of these features originated with Singularity’s purchase of Prompt.Security. Access to all the features requires purchasing the enterprise edition, which is offered with custom pricing, but lower feature tiers are available for $80 per year on this public pricing page. There are also numerous integrations with its Marketplace.

Varonis Atlas AI Security

Varonis Atlas AI Security is a multipurpose security platform that offers a variety of modules, including red team/penetration testing, compliance, and third-party risk management. Its AI-SPM module is combined with an AI inventory scanner and can be used to help development teams classify data used in the AI ecosystem, such as scanning for bad AI behavior, leveraging identities improperly, and examining data flows. Automated remediation processes are built into the tool as well. There are several hundred third-party integrations available for a wide collection of security tools, such as JFrog, Jira, Okta, and Salesforce. Varonis has two pricing components; one based on per user and per protected application and an additional price for resource consumption. Atlas is sold on the AWS Marketplace starting at $108,000 per year and free risk assessments are available to qualified customers.

Wiz/Google AI Application Protection Platform

Google has acquired Wiz but kept its operation independent. It has a multipurpose security platform that comes from a strong posture management (cloud and data) background. Its advanced version has been augmented with a comprehensive AI-related series of policies, detection algorithms, and pipeline, model, and data scanners. These are assembled into a separate AI dashboard page. It can also detect AI pipeline abuses, protect AI runtimes, identify and classify tools and agents, map dependencies graphically and suggest remediation steps. It also contains core AI-SPM features such as discovery, attack path analysis, and supply chains. Pricing for the Wiz Advanced bundle on AWS Marketplace is $38,000 annually.

What about AI-SPM pricing?

Pricing and packaging of AI-SPM tools vary widely. Many vendors offer free trials limited to differing periods (an option that is also available on the AWS Marketplace). We pointed out the open-source alternatives earlier, which is also a good way to see how the products work, but we wouldn’t recommend relying on these tools given their lack of recent updates. The only vendors that have (mostly) transparent pricing are Guardrail Technologies (with both free and monthly plans) and SentinelOne (with various annual plans starting at $80 per endpoint). Most of the vendors didn’t want to provide pricing directly but have published pricing on the AWS Marketplace, which can give you a rough indication that most start in the low six figures for annual contracts. For a typical situation with 1,000 users the total could be in the low six-figure range annually.

The question every board, CEO and CIO must answer in 2026 isn’t whether to use AI. It’s whether to use AI to improve what you have, or to start again. Most organizations are getting this choice wrong, defaulting to whichever option matches their risk appetite, rather than applying clear strategic criteria.

Rewiring treats existing processes, teams and systems as the frame, using AI as the wiring that makes them faster and smarter. The enterprise stays recognisable. Org charts shift modestly. Underneath, AI accelerates throughput and cuts manual effort e.g. AI co-pilots in legal review, ML-driven demand forecasting, generative AI auto-resolving Tier 1 support tickets.

Rebuilding treats the current operating model as legacy and uses AI as the architectural foundation for something structurally different. Entire functions may disappear or be reborn. Processes are redesigned from first principles with AI at the core, not bolted on. Think: a digital-only insurance carrier built around AI underwriting by default, not as an add-on.

Neither is universally correct. The organizations winning this decade are applying disciplined criteria and increasingly, sequencing both.

The decision framework

Five questions determine the right path, and they need to be asked together, not in isolation.

Is the operating model the constraint, or is execution? If processes are sound but slow and error-prone: rewire, AI removes friction without touching the underlying logic. If the architecture itself is fragmented and siloed by design, rebuild. AI plugged into a broken process just produces faster, better-documented brokenness.

How much runway do you have? Rewiring delivers ROI in 3–12 months. Rebuilding takes 18–48 months before material value shows up. If competitive pressure demands proof of AI value within a year, rewire first. If an AI-native competitor has already entered your market with a structurally lower cost base, incremental improvement won’t close that gap, only rebuilding will.

Can your people absorb the change? A workforce that’s risk-averse or change-fatigued can adopt AI-in-place without existential threat to most roles. A rebuild without genuine leadership mandate and a credible workforce transition plan isn’t transformation, it’s poorly managed redundancy with better PR.

How bad is the technology debt, really? Most AI use cases can be delivered via APIs and abstraction layers without core system replacement. Rebuild only when the estate is so fragmented that a unified data layer or real-time decisioning is structurally impossible otherwise.

Does the prize justify the disruption? Bounded efficiency gains of 10–25% rarely justify a rebuild’s cost and risk. Step-changes in unit economics or customer proposition do.

Who should decide

This is a capital allocation and talent strategy decision with technology implications, not a technology decision. The most common governance failure is letting the CIO or a transformation consultancy own it unilaterally.

The decision table needs the CEO, who owns the risk-return trade-off and the mandate to change; the CFO, who must model the economics of both paths honestly, including the productivity dip during transition, not just peak-state ROI; the CHRO, who needs a credible transition strategy in place before the decision is taken, not after; the CIO, who assesses technical feasibility but shouldn’t be making the strategic call alone; and business unit leaders, whose operational insight and buy-in are non-negotiable. An AI-literate independent board voice helps prevent both excessive caution and hype-driven overreach.

Costs, benefits and where maximum value sits

Rewiring’s ceiling is real, gains are bounded by the existing model, and it risks “AI-washing”: surface deployment without structural impact. But it’s fast, lower risk, preserves institutional knowledge and compounds across multiple waves over several years.

Rebuilding can deliver 30–60% structural cost reduction and capabilities simply unavailable to a rewired legacy model, but it carries a real failure rate (high for large transformations), heavy upfront investment and a multi-year J-curve before returns appear.

Maximum value rarely comes from choosing one exclusively. It comes from sequencing: rewire to generate cash, capability and credibility, then rebuild the two or three domains where AI-native architecture creates a genuine moat, while continuing to rewire everything else.

Case in point: An Australian tourism and cruise operator

Consider one of Australia’s largest integrated tourism and cruise businesses, simultaneously a B2C retailer, a B2B distributor to thousands of agency and wholesale clients globally, an aggregator marketplace for 1,800-plus independent tourism operators, and a cruise operator with offshore shared services spanning finance, customer contact and content management.

By 2024, the pressures had converged: AI-native travel platforms eroding acquisition economics, independent operators demanding dynamic pricing the platform couldn’t offer, and offshore cost structures under threat from automation. Leadership’s assessment found a split picture. The B2C and shared-services functions were sound but manual, a rewiring opportunity. The aggregator marketplace’s static catalogue and rules-based search were the actual constraint, no amount of AI on top would fix that. It needed rebuilding.

Rather than choose one path, the executive team sequenced three horizons. Horizon 1 rewired customer contact (AI triage cut Tier 1 escalations by 34%), content management (AI drafting cut operator listing time by 70%, eliminating a 23-day onboarding backlog), finance operations, B2C personalization (higher email revenue) and cruise crew scheduling (15% lower overtime). Within 18 months this delivered a million in annualised savings, funding and validating the next move.

Horizon 2 rebuilt the marketplace itself: AI-native semantic search lifted booking conversion by 24%; opt-in dynamic pricing lifted operator revenue per booking 16% for the first cohort; automated onboarding cut new-operator time-to-live from 23 days to three.

Critically, the offshore teams whose roles were most exposed to automation weren’t reduced, they were redeployed into quality assurance and operator onboarding, work that leveraged the institutional knowledge AI couldn’t replicate. Zero redundancies came out of Horizon 1. That decision wasn’t only ethical; the content quality gains from experienced specialists focusing on QA rather than production were measurable.

The lesson generalises well beyond travel: rewiring generated the cash, capability and credibility that made rebuilding possible. Neither path alone would have delivered the same outcome, and the sequencing mattered as much as the technology choices themselves.

What this means for CIOs

Start with rewiring, generate tangible ROI within 12 months and use it to build capability and board trust. Watch for your structural ceiling: the point where further rewiring yields diminishing returns because the model itself is the constraint. That’s your signal to rebuild selectively. Don’t rebuild everything; identify the two or three domains where AI-native architecture creates real competitive advantage and rewire the rest. And treat workforce transition as a strategic priority from day one, not an HR afterthought bolted on after the technology decisions are made.

The rewire-or-rebuild question isn’t a technology question. It’s a question about what kind of enterprise you’re choosing to become. The CIOs who get this right won’t be the ones who pick a side, they’ll be the ones who know exactly when to switch.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

AI coding agents are making it easier than ever to produce software. Ensuring that software is secure before deployment is another matter — one that AWS thinks AI should help with too.

As enterprises adopt agentic development workflows, the volume of first-party code being created and modified is rising rapidly. Yet the process of validating vulnerabilities, determining whether they are exploitable, and fixing them often still depends on developers and security teams working through findings manually.

AWS is aiming to address that imbalance with Continuum, a new service designed to continuously discover, investigate, and remediate vulnerabilities in enterprise environments, whether the code is their own or from third parties.

Rather than simply generating alerts, the service is intended to help enterprises move findings through the entire remediation lifecycle, AWS VP of Security and Observability Chet Kapoor wrote in a blog post.

For first-party applications, Continuum can analyze code, validate whether vulnerabilities are exploitable, generate remediation recommendations, and propose fixes that can be reviewed through existing software development workflows, helping developers address security issues without requiring security teams to manually investigate every finding, Kapoor said.

Once users think Continuum has learned enough about their environment and understands their guardrails, they can put it in what AWS calls “enforce mode” to autonomously fix any code lapses, Kapoor said.

Continuum borrows some of its capabilities, penetration testing and code scanning features, from an existing service, Security Agent.

Other capabilities are all-new, including threat modeling, which is designed to automatically generate threat models from source code or design documents and output them in STRIDE format.

Keeping pace with AI-driven software development

Analysts see Continuum helping enterprise developer teams ship more secure code while keeping pace with AI coding tools.

“The harder problem is no longer just finding issues, it is knowing which ones are real, which ones matter in their environment, and which ones need to be fixed first,” said Akshat Tyagi, associate practice leader at HFS Research. “Traditional workflows built around dashboards and manual triage struggle with that volume. A dashboard can show the backlog, but it does not validate the finding, assess business impact, or help remediate it.”

Continuum’s value, according to Tyagi, “is not just more detection, but using AI to prioritize risk findings, suggest mitigations, and support faster action while keeping humans in control of high-risk decisions.”

Taking faster action is becoming increasingly important as attackers are gaining access to many of the same AI capabilities that enterprises are using to accelerate software development and security testing, according to Amit Chandak, chief analytics officer at IT consulting firm Kanerika. “The gap between a flaw being disclosed and a working exploit is shrinking rapidly from months to hours,” he said.

While Continuum may reduce repetitive work for developers and SREs, it could also create new responsibilities for CISOs around governance, oversight, testing, and maintaining guardrails for automated actions.

“Continuum changes the CISO’s role from managing findings to governing how findings are handled. The focus moves to setting rules: what can be automated, what needs human approval, and what level of risk is acceptable in production,” Tyagi said. “Staffing will shift too. There may be less manual triage, but more need for people who can review AI-generated fixes, set guardrails, and know when not to trust the system.”

Even so, Chandak does not expect the offering to lead to immediate headcount reductions, particularly given that Continuum is only available as a gated preview.

Continuum could change how CISOs measure work, Tyagi said: “Ticket count matters less. Better measures are how quickly real risks are validated and fixed, how many false positives are removed, and whether automation is reducing risk without causing new problems.”

Those same metrics could also become a yardstick for CISOs determining how much autonomy to give tools like Continuum, said Chandak. Most enterprises’ data and governance practices are not yet ready for fully autonomous remediation, said Chandak, adding that, “AWS’ graduated trust design, under which enterprises have the option of choosing the degree of autonomy, from human in the loop to fully automatic remediation, is an admission of that fact.”

Beyond first-party code

Continuum could also help CISOs with third-party code vulnerability analysis, where enterprises often have less visibility and control.

“Most third party vulnerability alerts are noise. A tool may flag a vulnerable library, but the real question is whether that vulnerable code is actually used in production. If Continuum can answer that, it helps teams focus on the few issues that matter,” Tyagi said. “This is especially useful for open-source and software supply chain risk, where enterprises depend on packages and hidden transitive dependencies they may not fully track. It also helps when no patch is available yet.”

However, he warned, Continuum might not offer a direct fix to third-party code: “You usually cannot patch third-party code yourself as you don’t own it, so remediation there means version pinning or compensating controls.”

This article first appeared on InfoWorld.

AI coding agents are making it easier than ever to produce software. Ensuring that software is secure before deployment is another matter — one that AWS thinks AI should help with too.

As enterprises adopt agentic development workflows, the volume of first-party code being created and modified is rising rapidly. Yet the process of validating vulnerabilities, determining whether they are exploitable, and fixing them often still depends on developers and security teams working through findings manually.

AWS is aiming to address that imbalance with Continuum, a new service designed to continuously discover, investigate, and remediate vulnerabilities in enterprise environments, whether the code is their own or from third parties.

Rather than simply generating alerts, the service is intended to help enterprises move findings through the entire remediation lifecycle, AWS VP of Security and Observability Chet Kapoor wrote in a blog post.

For first-party applications, Continuum can analyze code, validate whether vulnerabilities are exploitable, generate remediation recommendations, and propose fixes that can be reviewed through existing software development workflows, helping developers address security issues without requiring security teams to manually investigate every finding, Kapoor said.

Once users think Continuum has learned enough about their environment and understands their guardrails, they can put it in what AWS calls “enforce mode” to autonomously fix any code lapses, Kapoor said.

Continuum borrows some of its capabilities, penetration testing and code scanning features, from an existing service, Security Agent.

Other capabilities are all-new, including threat modeling, which is designed to automatically generate threat models from source code or design documents and output them in STRIDE format.

Keeping pace with AI-driven software development

Analysts see Continuum helping enterprise developer teams ship more secure code while keeping pace with AI coding tools.

“The harder problem is no longer just finding issues, it is knowing which ones are real, which ones matter in their environment, and which ones need to be fixed first,” said Akshat Tyagi, associate practice leader at HFS Research. “Traditional workflows built around dashboards and manual triage struggle with that volume. A dashboard can show the backlog, but it does not validate the finding, assess business impact, or help remediate it.”

Continuum’s value, according to Tyagi, “is not just more detection, but using AI to prioritize risk findings, suggest mitigations, and support faster action while keeping humans in control of high-risk decisions.”

Taking faster action is becoming increasingly important as attackers are gaining access to many of the same AI capabilities that enterprises are using to accelerate software development and security testing, according to Amit Chandak, chief analytics officer at IT consulting firm Kanerika. “The gap between a flaw being disclosed and a working exploit is shrinking rapidly from months to hours,” he said.

While Continuum may reduce repetitive work for developers and SREs, it could also create new responsibilities for CISOs around governance, oversight, testing, and maintaining guardrails for automated actions.

“Continuum changes the CISO’s role from managing findings to governing how findings are handled. The focus moves to setting rules: what can be automated, what needs human approval, and what level of risk is acceptable in production,” Tyagi said. “Staffing will shift too. There may be less manual triage, but more need for people who can review AI-generated fixes, set guardrails, and know when not to trust the system.”

Even so, Chandak does not expect the offering to lead to immediate headcount reductions, particularly given that Continuum is only available as a gated preview.

Continuum could change how CISOs measure work, Tyagi said: “Ticket count matters less. Better measures are how quickly real risks are validated and fixed, how many false positives are removed, and whether automation is reducing risk without causing new problems.”

Those same metrics could also become a yardstick for CISOs determining how much autonomy to give tools like Continuum, said Chandak. Most enterprises’ data and governance practices are not yet ready for fully autonomous remediation, said Chandak, adding that, “AWS’ graduated trust design, under which enterprises have the option of choosing the degree of autonomy, from human in the loop to fully automatic remediation, is an admission of that fact.”

Beyond first-party code

Continuum could also help CISOs with third-party code vulnerability analysis, where enterprises often have less visibility and control.

“Most third party vulnerability alerts are noise. A tool may flag a vulnerable library, but the real question is whether that vulnerable code is actually used in production. If Continuum can answer that, it helps teams focus on the few issues that matter,” Tyagi said. “This is especially useful for open-source and software supply chain risk, where enterprises depend on packages and hidden transitive dependencies they may not fully track. It also helps when no patch is available yet.”

However, he warned, Continuum might not offer a direct fix to third-party code: “You usually cannot patch third-party code yourself as you don’t own it, so remediation there means version pinning or compensating controls.”

A simple framework has always governed security operations that I call the SOC Triangle. It is a balance between quality, consistency and cost efficiency.

Every SOC operates within it. Push for higher-quality investigations, deeper analysis, richer context, fewer missed signals and you pay for it in time and expertise. Standardize workflows to ensure consistency across every alert, and you often lose the flexibility needed to handle real-world complexity and nuance. Optimize for cost efficiency, and the pressure shows up quickly in both quality and consistency.

For years, the SOC Triangle has shaped how security teams are built and how they perform. This is why organizations add headcount to improve outcomes, rely on rigid playbooks to reduce variability and improve scale, and still struggle to operate at their theoretical best and optimize security and quality of service outcomes.

The constraint is not a failure of strategy. It is structural. And until recently, it was largely unavoidable.

Why the SOC was built this way

Most security operations centers are designed as human-routing systems. Alerts are ingested, triaged, escalated and resolved by analysts at multiple levels. Every meaningful step, including collecting evidence, correlating signals and making decisions, depends on human capacity.

That dependency introduces variability. Two analysts can approach the same alert differently, influenced by experience, fatigue and time pressure. To improve consistency, organizations introduce playbooks and workflows. But those controls often reduce flexibility, especially in complex cases, and fail to provide coverage where decision making relies in part on unstructured context, and where workflows may not be fully deterministic and require real-time reasoning to determine the best course of action.

At the same time, scaling either quality or consistency typically requires more people, reducing cost efficiency.

This is the SOC Triangle in practice: a system where improving one dimension creates friction in another.

The same constraint is also why the managed detection and response market exists. When organizations could not solve the triangle in-house, they outsourced it. But the service model does not eliminate the trade-offs. It reconstitutes them at the provider layer, where the same human-routing architecture, the same playbooks and the same staffing economics drive the same limits. Customers pay for consistency and predictability, and they get it. What they often do not get is the investigation depth and environmental customization tailored to their business context and to optimizing against their security program maturity goals that they would want if resources were not the binding constraint.

Where the model starts to break

The challenge is not just the existence of trade-offs, but their growing intensity.

Modern SOCs must process higher volumes of alerts across more tools and environments. The work itself, gathering and correlating evidence across identity systems, endpoints, cloud platforms and threat intelligence, is both repetitive and cognitively demanding.

Under this pressure, the triangle tightens.

Quality degrades because analysts do not have time to fully investigate every signal and rigid automation playbooks often fail to capture the depth and nuance that security leaders expect which results in increased friction for end users. Consistency suffers because decisions are made under time constraints. Cost rises because the only way to compensate is to add more people or accept increased risk.

This hits hardest for organizations that have outsourced SOC operations. Service economics lock the trade-offs in place. Per-alert pricing constrains how much investigation each signal receives. Standardized playbooks limit how much the service can tailor to a specific environment. Tier structures exist because the math of humans investigating alerts demands they exist. Every one of those mechanisms is a rational response to the triangle. None of them changes its shape and its fundamental constraints.

For years, this has been accepted as the cost of doing business, whether that business is run in-house or outsourced.

How AI changes the constraint

AI is often framed as a tool for efficiency. The more meaningful shift is that it changes how certain SOC workflows are executed.

Much of SOC work follows a pattern: gather data, correlate signals, ask follow-up questions and form a conclusion. These workflows are complex but repeatable. They require consistency and scale as much as expertise.

When those workflows are no longer constrained by human bandwidth, the SOC Triangle begins to change shape.

Quality improves because investigations can incorporate more meaningful data, apply investigative reasoning in real time and take into account unstructured information and business-specific context without shortcuts. Consistency improves because the same logic is applied across every alert. Cost efficiency improves because scaling no longer depends on linear increases in headcount.

I am watching this play out in production environments today. Investigations that used to consume the majority of Tier 1 and 2 analysts’ shifts now resolve in minutes, with deeper context than the human path could produce within these time frames. The same rigor is applied to every alert, not only the anecdotal ones that earn attention. What used to be a choice between going deep on a few cases or going shallow on many is no longer a compromise security leaders need to make.

For the first time, these dimensions are not strictly in opposition.

From trade-offs to expansion

This does not eliminate the SOC Triangle. It expands it.

Not every workflow can be automated, and not every decision can be reduced to a repeatable process. Strategic judgment, incident leadership and risk appetite remain human responsibilities and business decisions.

But the boundary within which SOC teams operate is no longer tied to legacy constraints.

Instead of choosing between quality, consistency and cost, organizations can begin to improve all three for the types of work best suited to machine execution. That is a meaningful shift, whether it occurs within a company’s SOC or in the service relationship with a partner that operates it.

Where it matters most

The impact is most visible in the high-volume workflows where performance gaps have been largest: alert triage and enrichment, initial investigation and evidence gathering, correlation across systems and routine response recommendations. These are the areas where human-led processes introduce the most variability, where time pressure degrades quality and where scaling costs are most visible. They are also the areas where trade-offs have historically been unavoidable.

The human role evolves

AI does not remove the need for human expertise. It changes where that expertise is applied.

As machines take on repeatable work, human effort shifts toward higher-value activities: interpreting ambiguous signals, managing complex incidents, setting policy and making risk-based decisions. The operating model moves from human-executed workflows to human-governed systems.

That changes what organizations should expect from security operations, whether in-house or outsourced. The conversation moves from “how many alerts did you close last week” to “what patterns are you seeing in my environment, and what should I do about them.” The output is judgment, not throughput. That is a different product than most security teams have been buying, and it is a different service than most managed detection and response service providers have been selling.

The shift that matters

For years, SOC leaders have accepted the triangle as a fixed constraint. What is changing now is not just the tooling. It is the economics of how security work is performed.

The triangle still exists. But it no longer defines a rigid set of trade-offs. In parts of the SOC and the services that support it, those trade-offs are beginning to loosen.

In a field where constraints have long dictated outcomes, that shift matters.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Meta’s long-awaited Pyrefly linter is out in a 1.0 version, and the forthcoming Python 3.15 has a super-efficient sampling profiler. Plus we have a comprehensive rundown of Python’s indispensable virtual environments — and a warning about a novel breed of malware that exploits Python’s package ecosystem.

Top picks for Python readers on InfoWorld

How to use virtual environments in Python
Isolate and protect your Python projects from each other, and empower them to do more, with virtual environments and their native-to-Python tooling.

Pyrefly 1.0: A fast, forward-looking Python linter
The first full release of Meta’s long-awaited linting and type checking tool for Python delivers speed and offers advanced features for type-checking PyTorch and Django projects.

Hands-on with the new sampling profiler in Python 3.15
Among Python 3.15’s best new features is a sampling profiler, for instrumenting your code and finding its bottlenecks with a minimum of performance impact or fuss. See up-close how it works.

All about Hades, the supply-chain malware that hides in Python packages
It hides in Python packages. It replicates itself across systems. It fools LLM-based code analysis tools into ignoring it. And there may be a lot more like it to come.

More good reads and Python updates elsewhere

Python Steering Council calls for temporary pause on JIT project
The requested pause stays in place until a proper Standards Track PEP lands for the experimental JIT (just-in-time) compiler, the better to describe how the JIT will be a formal and supported part of Python.

Pyodide 314.0: Pyodide packages on PyPI
Thanks to PEP 783, Python packages built with Pyodide (Python ported to WebAssembly) can be installed straight from PyPI instead of through Pyodide — another step closer to Py-on-Wasm becoming an everyday thing.

All about that Python 3.14 garbage collection rigmarole
A new garbage collector introduced in Python 3.14 was yanked at the last minute due to reports of higher memory usage. Here’s a deep dive into what changed for the worse and why.

Are you really expected to run five type checkers now?
No, but you should keep your options open. This blog post from a Pyrefly contributor recommends choosing one of the major offerings (Mypy, Pyrefly, Pyright, ty, Zuban, etc.), but also getting to know the others too.