The head of the US government’s AI testing institute, Chris Fall, has resigned about three months after taking charge of the Center for AI Standards and Innovation (CAISI), the federal organization responsible for evaluating advanced artificial intelligence models for safety and security.

Current National Institute of Standards and Technology NIST Director Arvind Raman will serve as acting CAISI Director following Fall’s departure while continuing to oversee the Commerce Department office responsible for the institute, the Daily Signal reported, citing two people familiar with the matter.

A Commerce Department spokesperson who spoke to the publication did not disclose a reason for the resignation.

Fall assumed leadership of CAISI in April after the Trump administration reorganized the former US AI Safety Institute under NIST. The institute develops methodologies for evaluating frontier AI models and works with AI developers on voluntary technical assessments covering areas such as cybersecurity, model misuse, reliability and other risks associated with increasingly capable AI systems.

The leadership change comes as governments and AI companies continue developing technical approaches for evaluating frontier AI models while enterprises expand deployments of generative AI and agentic AI across business operations.

In recent months, the Commerce Department has taken a more active role in AI policy involving advanced models, placing greater attention on how the federal government evaluates technologies with potential national security implications.

Continuity matters more than personalities

CAISI works with AI developers such as Anthropic, Google’s DeepMind and OpenAI on voluntary evaluations of frontier AI models and develops methodologies for testing model capabilities and risks. The institute does not regulate AI developers or certify commercial AI systems.

For enterprises, those evaluations are one source of technical information alongside vendors’ own testing, third-party security assessments and internal AI governance programs.

Sanchit Vir Gogia, chief analyst at Greyhound Research, said enterprises should focus less on the individual leading the institute and more on whether its technical work continues with the same level of consistency and transparency.

“Leadership churn at CAISI weakens the signal long before it weakens the science,” Gogia said. “The testing has not stopped. Its authority simply does not travel as cleanly once the leadership does not.”

According to Gogia, the more important question for enterprises is not whether the institute’s evaluation work will continue but whether the processes supporting those evaluations remain stable.

“The instinct is to ask whether the pipeline is breaking,” he said. “The more useful question is where the pipeline now sits.”

Enterprises still carry the burden of AI governance

Gogia said organizations should continue treating government-led AI evaluations as one input into their governance processes rather than as evidence that a model is inherently safe for enterprise deployment.

“A government evaluation was always a signal, never a certificate,” he said. “A signal loses value the moment its issuer becomes unpredictable.”

He said enterprises should instead monitor whether CAISI maintains consistent evaluation methodologies, continues publishing technical findings and preserves continuity within its research teams under interim leadership.

“The name on the door is not the signal. The behaviour underneath it is,” Gogia said.

Gogia also cautioned against linking Fall’s resignation to recent Commerce Department actions involving AI policy or export controls, noting that there is no public evidence connecting the two.

“CAISI evaluates; it does not enforce export controls, because it holds no such power,” he said. “This is not a testing body reaching for enforcement. It is enforcement reaching past the testing body.”

With Raman assuming the role on an interim basis, the next significant milestone for enterprises will be the appointment of a permanent director, and whether the institute’s evaluation programs continue without disruption, the analyst said.

Gogia said the successor’s mandate may prove more important than the individual selected.

“A CAISI result is not a safe harbour,” he said. “It informs an obligation; it does not discharge one.” NIST did not immediately respond to a request for comment.

Building a presentation can take lots of time. There are design choices to figure out: the slide layouts, fonts, theme colors, and so on. You can use a template to skip this hassle, but you still have to paste your text and other content into the slides and edit it all so that the results are visually appealing.

In PowerPoint, Microsoft’s Copilot AI assistant can now automate the heavy lifting of presentation creation. It can generate a first-draft presentation in minutes, then help you edit it. You can also prompt Copilot to help you quickly understand the contents of a presentation and glean insights from it. Use the tips in this guide to save oodles of time as you create and work with presentations.

Who can use Copilot in PowerPoint

Individuals with a Microsoft 365 Personal, Family, or Premium subscription have access to Copilot from within PowerPoint and other Microsoft 365 apps. Users with a Premium plan have higher Copilot usage allowances and access to advanced AI features.

For business users, it’s more complicated. Organizations with more than 2,000 users must pay for Microsoft 365 Copilot licenses for their users in addition to their regular Microsoft 365 licenses. Users at organizations with fewer than 2,000 users can use Copilot within M365 apps even without the M365 Copilot add-on licenses, but there are limitations in usage, speed, and feature availability.

To see what kind of access you have, log in to Microsoft’s Copilot Chat web hub and look for your name in the lower left corner. If you see “M365 Copilot (Premium)” under your name, you can use Copilot in M365 apps with priority access and advanced features. “M365 Copilot (Basic)” means you can use Copilot in M365 apps with lower-priority access and limited features. If you see “Copilot Chat (Basic)” or nothing below your name, you can’t use Copilot in M365 apps.

(Copilot Chat Basic users do get some Copilot functionality, including the ability to generate presentations, via the Copilot Chat hub. See our Copilot Chat tutorial for details.)

In this article:

• Working with Copilot in PowerPoint
• Create a presentation template
• Create a presentation from a document
• Add content from a document to a slide
• Refine your slide text
• Find or create an image
• Expand your presentation with relevant slides
• Summarize a presentation
• Answer questions about a presentation
• Help you navigate a large presentation
• Generate speaker notes and/or an FAQ

Working with Copilot in PowerPoint

First, let’s quickly go over the notable settings of the Copilot sidebar.

When you have a presentation open in PowerPoint, click the Copilot icon; it may be floating at the lower-right corner of your PowerPoint window or parked at the right end of the Ribbon toolbar. The Copilot sidebar will open along the right of the page. You’ll type your prompts to Copilot inside the chat window in this pane.

Howard Wen / Foundry

Agent mode: By default, Copilot can build a new presentation or make changes to an existing one in the main PowerPoint window. This is known as “agent mode.” To change this so that Copilot can’t take direct action on a presentation (all its responses appear in the sidebar), click the Allow editing button above the chat window and change it to Chat only.

The tips in this guide require that Copilot be in agent mode, so make sure you see Allow editing above the chat window.

Choice of AI model: Behind the scenes, Copilot has access to various genAI models, including different versions of Anthropic Claude and OpenAI GPT.  By default, it decides which model to use based on your prompt. You can set it to use a particular model: click Auto at the upper right of the Copilot pane and select a model from the dropdown that opens.

Howard Wen / Foundry

The tips in this guide should work fine on the default Auto setting. But feel free to experiment switching to specific models to see which give you the best results for particular tasks.

Important: Remember that generative AI output often includes errors, so always check Copilot’s output for accuracy. (Also see our tips for reducing hallucinations in Copilot.) You’ll likely want to rewrite it in your own voice as you’re reviewing it.

1. Create a presentation template

For many people, the hardest part of creating a presentation is getting started. What types of information should be included on the slides, and in what order? Copilot can give you a leg up by creating the type of presentation you need, with placeholder data that you can later replace with your own.

Start a new presentation, open the Copilot sidebar, and type your prompt into the chat window. It’s best to provide very specific details in your prompt. The more context or details you provide, the more likely Copilot will generate a presentation template that suits your needs.

A good prompt should contain the slide count, subject, audience, and tone. Example:

• Create a 6-slide presentation for a sales meeting focusing on Q1 revenue. The audience is the sales team, so keep the tone professional and focused on the sales data.

Copilot may ask a series of follow-up questions, such as your preferred visual style and desired level of detail. Then it will generate a presentation template.

Howard Wen / Foundry

You can optionally prompt Copilot for revisions, and when you’re happy with the template, swap in your own data.

2. Create a presentation from a document

You can attach a document (such as a Word document, Excel spreadsheet, or PDF) and prompt Copilot to generate a presentation based on its contents. This works best with a structured-format document (such as a business plan, project proposal, or summary report) that contains sections with headings.

Copilot can extract the document’s text and structure to generate the slide content for the new presentation. This can especially be useful for quickly turning a long report into a visually appealing presentation.

In the Copilot pane, click the + icon at the bottom of the chat window. A list of documents that you’ve recently accessed appears. Select the one that you want Copilot to use. Alternatively, click the magnifying glass icon and inside its search box, type a few letters of the filename for the document you want. (Business users with an M365 Copilot license can select up to five files for Copilot to pull from when creating a presentation.)

Howard Wen / Foundry

Then in the chat window, you can enter a prompt that’s as simple as “Create a presentation,” although as always, providing more details and context is better. This is especially important for corporate users who reference multiple source files. It’s useful to tell Copilot what data to pull from each document.

Answer any follow-up questions that Copilot asks, and it will then generate the presentation.

Howard Wen / Foundry

Note: Your marketing department may have created one or more branded company templates for Copilot to work from. If that’s the case at your organization, simply open the appropriate company template as your first step. Then you can upload docs and type a prompt as described above. Copilot will create a presentation using the branded template.

3. Add content from a document to a slide

Manually copying text or other content from a document and pasting it into a new slide is a chore. Instead, you can prompt Copilot to extract information directly from a Word document, Excel spreadsheet, or PDF to create new slides.

In the Copilot pane, attach the document using the same steps described in tip 2, then tell Copilot to create a slide from the document. As always, it helps to provide details such as the new slide’s focus or what data to include:

• Add a slide based on the attached document.
• Use the attached file to add a slide about the project budget that focuses on Q1 projections.
• Summarize only the financial section of the attached document as a slide.

Howard Wen / Foundry

4. Refine your slide text

A presentation should be visual and display only the core message. Conciseness and proper writing tone are essential for your slides, so that they don’t lose the attention of your audience.

You can prompt Copilot to refine text on an individual slide in various ways, such as rewriting it in a more professional tone or making it more concise. Highlight the text inside a text box on the slide. On the toolbar that appears over the highlighted text, click Edit with Copilot.

On the menu that opens, you can select a preset prompt to refine the text, such as Condense or Make professional. Or, at the top of this menu, you can type a prompt to rewrite the highlighted text.

Howard Wen / Foundry

Note that this feature affects all the text inside the text box. To rewrite only a portion of text inside a text box, you must split that portion out into a separate text box.

Alternatively, you can prompt Copilot to analyze your entire presentation and tighten up the wording throughout all of its slides. For example:

• Make these slides more visual and use less text.

5. Find or create an image

If you have Copilot generate a presentation from an existing Word document that contains images, it will incorporate those images into the presentation. If there are no images in the source document, you can ask Copilot to find or create one and add it to a slide.

To add a stock image or an image from your organization’s brand library, tell Copilot what you’re looking for:

• Add a stock photo of young adults in a cafe drinking boba tea.
• Add a photo from our asset library of young adults in a cafe drinking boba tea.

To have Copilot create an image using Microsoft’s Designer image generation tool, describe your desired image. As always, specificity is helpful:

• Create a photorealistic image of a diverse group of 5 or 6 fashionable young adults sitting in a cafe drinking boba tea. They’re smiling or laughing, and some are looking at their phones.

Howard Wen / Foundry

Just as you need to review any text output from Copilot, take a close look at generated images to be sure nothing looks off.

Also note that Copilot image generation isn’t always reliable in PowerPoint. For some time during our testing for this story, Copilot said it couldn’t create an image because “the image generation service is returning a server error on every attempt.” After about a day and a half, the service began working again.

6. Expand your presentation with relevant slides

As you’re building your presentation, you may find that it’s become text heavy. Or perhaps it could use more visually oriented slides to break things up and make its progression flow better. Copilot can generate and insert new slides that are based on the content of the slides already in the presentation.

In the Copilot pane, specify exactly where you want the new slide to go. This helps Copilot to analyze the content of the slides before and after where you want the new slide. Then it can generate a slide to bridge between the two slides. Examples:

• Add a slide after slide 3 about our competitive advantages.
• Add a slide after slide 11 that transitions to slide 12.

Howard Wen / Foundry

7. Summarize a presentation

Maybe you need a quick refresh of your presentation before an important meeting. Or maybe a co-worker has sent you a presentation that’s packed with lots of slides. You can prompt Copilot to generate a summary of the presentation’s overall messaging.

In the Copilot pane, just type “summarize this presentation.” You can also have Copilot flag key slides that contain important information: “show me key slides.”

Howard Wen / Foundry

8. Answer questions about a presentation

As you’re reviewing a presentation, especially one that you didn’t create and are not familiar with, you can get Copilot to pull key data points from its slides.

In the Copilot pane, type specific informational questions. Examples:

• What are the action items in this deck?
• What is the proposed budget mentioned here?

If Copilot can’t find the exact answer to the question you ask, it will provide related information from the presentation.

Howard Wen / Foundry

This method can also help you validate that your presentation includes everything you want it to. If you ask Copilot about the action items in a presentation and it can’t find any, you know you need to add them. (Copilot will likely offer to generate them for you based on the rest of the slides.)

You can even take this tactic a step further and ask Copilot if the presentation is missing any important data, if any slides are weak or confusing, if there are any awkward transitions, if there are key points that should be better emphasized, and so on.

9. Help you navigate a large presentation

In the business world, presentations with dozens of slides are not uncommon, such as for financial reports or project documentation. Trying to find a specific slide or multiple slides can be tough. Copilot can help you navigate such a presentation.

In the Copilot pane, prompt Copilot to find slides based on specific topics. Example:

• Show me the slides about the project timeline.

Copilot will analyze the presentation and reply with a list of links to the relevant slides. Click one of these to jump directly to that slide.

Howard Wen / Foundry

10. Generate speaker notes and/or an FAQ

Here’s a great timesaver when you’re preparing to show your presentation to an audience: Copilot can automatically generate suggested speaker notes for you, based on the content of your slides. Example prompt:

• Write speaker notes for every slide with one talking point per slide.

Howard Wen / Foundry

In a related feature, Copilot can create a frequently asked questions list (FAQ) for you to consult in your speaker notes or to present as a slide:

• Write an FAQ for these slides.

Copilot will ask where you want the questions and answers added — as a new slide at the end, integrated into the speaker notes of relevant slides, or somewhere else that you designate. Make a selection, and Copilot will generate the FAQ based on the content of your presentation.

Howard Wen / Foundry

Related reading:

• 11 cool things Copilot can do in Excel
• 9 ways Copilot can turbocharge OneNote
• PowerPoint for Microsoft 365 cheat sheet
• Copilot Chat: Your hub for document creation and analysis
• How to curb hallucinations in Copilot (and other genAI tools)
• Microsoft Copilot can boost your writing in Word, Outlook, and OneNote — here’s how
• More Microsoft tips and tutorials

Healthy software companies typically don’t suddenly eliminate one-fifth of their workforce, but monday.com is doing just that as it bets on flatter teams, AI agents, and customer implementation expertise as the winning combination in the AI era.

Monday.com co-founder and co-CEO Eran Zinman today announced the “very difficult decision” to reduce the AI work platform company’s global workforce by about 20%, or 620 people.

The move has nothing to do with increasing margins or replacing humans with AI, he insisted in his post on LinkedIn; rather, it’s a calculated decision to trim down and hone the company’s focus as AI becomes integral to day-to-day workflows.

“This is not a distress signal; it is a deliberate reset, disclosed with its price attached,” said Sanchit Vir Gogia, chief analyst at Greyhound Research. “The industry has quietly swapped the meaning of productivity, and this filing is the clearest exhibit yet.”

A ‘significant opportunity’ in technology

In a SEC filing this week, monday.com said its restructuring plan reflects the “ongoing transformation of its product, marketing, and go-to-market strategy.” The move is intended to support a “leaner, more focused operating model” as the company continues to invest in its AI-driven strategy.

Zinman noted in his post that the company has shifted to “doing the work with AI and not just managing it,” and is focused on building environments where “people and AI agents [work] together in one workspace.”

In recent months, monday.com has evolved its products, strategy, and the way it serves its customers, and Zinman contended that “the organization we built for our previous chapter is not the organization that fits the new AI era.” Monday.com needs to “execute more decisively,” take on new challenges, and quickly respond to market changes, he said.

“We have never seen such a significant opportunity in software, driven by such exciting technology,” Zinman noted. He emphasized that the reduction is not to replace people with AI, nor to improve margins; the “vast majority” of savings will be reinvested into talent, products, and AI.

The restructuring will result in a “flatter organization” with fewer management layers and smaller, more autonomous teams, and monday.com also has a new go-to-market model, Zinman explained. Customers expect “deeper implementation support” as they deploy AI, and the company will work more closely with customers, increase its on-site presence, create new roles, and “adapt many existing ones.” In its SEC filing, the company said it expects to continue hiring in “key strategic areas” throughout 2026.

Workers will be expected to work better, “not harder,” Zinman noted. He pointed to several past examples where work could have been done in a few days, but instead took many months with “multiple meetings and endless friction.”

“This wasn’t people’s fault and everyone was frustrated by this,” he said. “Our new org changes ownership to allow people to make decisions and move fast.”

A spokesperson for monday.com declined to comment further on the staff reductions.

Monday.com’s key market advantages

Monday.com certainly isn’t struggling; the company expects 19% to 20% year-over-year growth in 2026.

“Companies in that position do not restructure because they must,” Greyhound’s Gogia noted. “They restructure because they have decided to become something else.”

Melody Brue, VP and principal analyst at Moor Insights & Strategy, pointed out that organizational redesign is important for real AI transformation, but while it can signal confidence to the market, it can still be “devastating” to humans.

While the company looks as though it’s trying to do right, that ultimately remains to be seen, she said. “There are often hidden internal bruises that can surface long after layoffs.”

Monday.com’s advantage is in its “structured substrate,” Gogia noted; its boards, permissions and typed workflows give agents something firmer to act on than just documents and chat history. The company highlights its natively built agents that can be configured by any team member, as well as connectors with Claude, Microsoft Copilot, and ChatGPT, and dedicated routes for external agents to authenticate and operate.

“For some time, the sharper enterprise question has been shifting from who has an agent to who owns the governed runtime in which an agent can safely act,” he said. “Structured work is a serious claim on that runtime.”

But parts of monday.com’s agent estate remain in staged release, and its product is ultimately “mid-transition,” Gogia pointed out; its agent builder carried a beta label as recently as March,. Also, the company’s pricing model changed in May to a hybrid model charging for seats as well as mandatory AI credits. And, while its AI-powered no-code builder monday vibe passed $1 million in annual recurring revenue within two and a half months, monday.com has not released subsequent outcomes, usage volumes, or attach rates.

Further, there’s an element of “gravity” with its competitors, he observed. Asana is reorganizing teams around agents, Atlassian is wiring agents into the developer estate, and others are simply bundling them into their offerings: Microsoft is doing so across the productivity stack, and ServiceNow across enterprise operations, each with identity and procurement built in.

“Their pull is strongest exactly where monday.com wants to grow, in the largest accounts, where control-plane depth and administrative reach decide the deal,” said Gogia.

Actions for the near-term

Going forward, buyers should focus on operating risk, not headline risk, Moor’s Brue noted. In practice, that’s continuity of service, roadmap consistency, and strength of enterprise support. Productivity should be valued as better outcomes per unit of organizational effort, not mere activity.

“It should be a measure of how much smoother, faster, and more effective the operating model becomes when AI is built into the work,” said Brue.

Gogia noted that strain surfaces first in customer service, and monday.com’s attention is being redistributed. The company’s annual report disclosed that its focus is now concentrated on the largest accounts, with support for medium-sized clients moved to an AI-first and human-supported model.

During the first month of the transition, buyers should track named account continuity and escalation times, he advised. By the first quarter, keep an eye on whether credit governance and admin controls mature on schedule, and if the roadmap beyond the AI estate keeps pace. By the half-year mark, determine whether promised implementation depth is producing outcomes or “simply more billable engagement.”

Support tiers should be enumerated in writing before renewal, and buyers should contract for “side exits,” Gogia emphasized, with overage pricing fixed in advance, the right to pause consumption, and portability for workflows and agent configuration “if the relationship sours.” Finance should also insist on monthly consumption reporting by capability. Further, integration efforts, partner dependency, and change management should be considered first-class costs of the agent era, “not as afterthoughts to a license.”

“A license was a known cost,” said Gogia. “A meter is a behavior, and behavior is harder to forecast than headcount.”

This article originally appeared on CIO.com.

The traditional methods of measuring success no longer tell the full story. Here’s what should replace them — and why.

In brief:

• AI is transforming software as a service (SaaS), and the old ways of keeping score no longer apply.
• Smart companies are evolving new metrics that provide deeper insight into how AI-native software is performing in a new marketplace.
• These changes impact everything from pricing to valuations.

The transformation of the software-as-a-service (SaaS) industry toward AI-native operating companies is rapidly changing the unit of value across the industry.

The traditional metric of seats — which measured access — is rapidly giving way to credits designed to measure work performed. This evolution is upending the industry in multiple ways, impacting everything from pricing to enterprise valuations.

While many companies still cling to seat-based metrics to measure growth, efficiency and durability, the future is likely to be one in which companies utilize a credit-centric metrics framework, with seats and outcomes as the bookends of a spectrum.

Why do software companies need new metrics?

Why the rethink, and why now? There are five major forces that are driving this shift:

1. The unit of value is changing. Seats measured who could access software, and credits measure what the software actually does. But in an AI-native world, agents don’t have seats; they have workloads. Over the past 18 months, every major SaaS platform has moved to some forms of credit or consumption unit.
2. The cost of goods sold (COGS) is exploding. AI inference adds real per-unit costs that scale with usage. In an AI-native world, software companies can’t scale to infinite users at near‑zero marginal cost as before.
3. Buying is moving up the org chart. AI-native applications shift purchasing to higher-level operators — such as line-of-business leaders or chief operating officers — which expands the market from software budgets to labor budgets. And because AI agents replace services as well as software, the total market opportunity is 3x to 10x larger than traditional SaaS.
4. Time to value (TTV) is collapsing. With AI-native tools, customers start seeing meaningful results in weeks rather than quarters. Onboarding and setup are fast, workflows are pre-built, and there’s no need for extensive customer success or professional services — dramatically reducing implementation time and costs.
5. Retention is bifurcating. AI forces clarity in a way that traditional SaaS couldn’t. Products that can provide value become even “stickier” and retain customers. Those that don’t churn faster. In an AI-native marketplace, the middle disappears.

How this shift is impacting pricing

Given how AI-native software is transforming the market, the shift to more variable pricing options is inevitable.

Seats won’t go away completely. Subscription pricing based on the number of users is stable and predictable and will continue to work for some customers. Tokens — the use of pass-through pricing for underlying compute — will fit those customers where the AI feature is commoditized or the buyer wants transparency into costs.

Credits will likely become the dominant architecture because they provide a simple metric for both customers and providers. The vendor sets the conversation ratio between credits and underlying compute, shielding the customer from inference cost details. Credits are easy to understand and can be packaged into annual contracts for multiple features and products.

Finally, the industry will likely see some move toward outcome-based pricing for results such as resolved tickets, recovered revenue or qualified leads. This strategy will mostly be limited to verticals where it is easy to prove AI impacted the result.

Where a software vendor sits on this spectrum is a signal of differentiation and pricing power. Credits are where most defensible AI-native businesses are landing because they balance customer predictability with vendor margin control.

How AI upends classic SaaS metrics

When SaaS was in its infancy, companies settled on key metrics designed to answer a small set of core questions. Are we growing? Are customers using the product? Are we retaining and expanding accounts?

But as AI upends software itself, it is also requiring companies to adopt new metrics to track success. These new metrics fall into three primary buckets, rebuilt around the pricing spectrum described earlier and the trend toward credits as the primary frame:

Revenue composition

• Committed credit annual recurring revenue (ARR) vs. burndown ARR: Measuring the credits sold on annual commitment vs. those consumed and replenished. This is the single most important split for valuation. Committed credits behave like subscription and burndown behaves like usage.
• Credit utilization rate: The percentage of purchased credits consumed per period. This is a leading indicator of renewal sizing.
• Credit burn velocity: How fast is a customer consuming their credits, and is that consumption increasing or decreasing quarter over quarter? This metric predicts expansion or contraction before it shows up in ARR.
• Effective price per credit: The real revenue per credit after discounts, overage and rollover, which can detect revenue leakage and help companies set smarter guide rails.

Margin reality

• Credit margin: The gross profit the company earns per credit after subtracting inference costs. This is the core economic unit for AI-native, usage-based businesses — the replacement for gross margin per seat used in SaaS.
• Inference-adjusted gross margin: By carving out AI inference costs separately in the P&L statement, you can see true AI margins, avoid hiding deterioration inside blended SaaS margins, and clearly distinguish AI economics from legacy SaaS economics.
• Compute leverage ratio: This metric measures how efficiently the business converts compute spend into revenue. It shows whether your AI margins are improving as you scale.
• AI-adjusted “Rule of 40”: This updated metric recalibrates the traditional growth and profitability benchmark to account for AI’s lower gross margins and variable inference costs, giving a more accurate picture of business health for AI-native companies.

Behavioral and value signals

• Time-to-first outcome: Replaces traditional onboarding metrics. Tracks how fast a customer reaches their first measurable result.
• Adoption: AI-native adoption is measured by workflow penetration and active agent density, not seat count. As AI replaces human-driven usage, the unit of adoption shifts from people to automated workflows and agents.
• Net credit retention (NCR): Credit-volume retention across the customer base, tracked separately from net recurring revenue to avoid price-change impact.

Along with these new metrics, the industry’s transformation is prompting companies to retire or recalibrate old SaaS measures, including per-seat ARR as a primary key performance indicator (KPI), traditional magic number calibrated to subscription dynamics, unadjusted Rule of 40, customer success metrics tied to human touchpoints, and blended gross margin without AI COGS carve-outs.

What does this mean for enterprise value calculations?

As the internal metrics of success change, so do the ways the investment community measures growth and long-term viability.

Increasingly, a company’s valuation multiple depends on whether its revenue behaves like committed subscription ARR or volatile usage ARR, and the commit‑to‑burndown ratio is the metric investors use to decide where the company fits.

For example, a business with 80% committed credit ARR could trade closer to subscription comps and one with 80% burndown could trade closer to usage comps even though both have the same types of customers. Being able to proactively explain the commit‑to‑burndown mix can help companies avoid undervaluation.

In addition, utilization is expected to replace net promoter scores and seat usage as the primary predictor of churn or expansion. Low utilization guarantees downsizing at renewal, so companies must track utilization cohorts the same way SaaS tracks logo retention cohorts today.

We’re also seeing an inversion of the operating model, with R&D and COGS moving up the P&L and sales and marketing (S&M) and customer success (CS) moving down or sideways. The net operating leverage profile is structurally different from classical SaaS, and the cost-to-scale curve looks different too.

Finally, credit margin engineering is a hidden value-creation lever. The gap between price per credit and cost per credit is set by the software vendor and can be optimized. Most operators have barely started managing this rigorously, and the ones who do will pull away on margin.

What this means for leaders, boards and investors

The shift from classic SaaS metrics to new AI‑native measures isn’t cosmetic. It represents the seismic change the industry is experiencing as AI matures and transforms products and organizations.

While these metrics — and perhaps others yet to be determined — may evolve over time, there is no doubt they are already changing how AI companies allocate capital, price products, incent sales teams, evaluate performance and communicate with investors.

It’s important to remember that SaaS metrics were practical tools for a specific era of software. As that era draws to a close, winning companies will choose new metrics that shape behavior and drive smart decision-making.

The views reflected in this article are the views of the author and do not necessarily reflect the views of Ernst & Young LLP or other members of the global EY organization.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Late in a large enterprise SAP transformation, the strategic governance conversations began to drift. Instead of executive decisions, we found ourselves debating whether the program needed dedicated testing, whether cutover required a full weekend, whether twenty Agile teams really needed coordination support and whether offshore resources were adding value at all.

The questions were not coming from the delivery teams. They were coming from the executive sponsor.

The sponsor had recently been elevated into a senior leadership role and had never sponsored a technology transformation at this scale. The challenge was not authority. The sponsor had every right to ask any question they wanted. The challenge was that strategic governance had quietly turned into a debate about delivery practices, because the sponsor did not yet have the transformation context to focus the conversation anywhere else.

This is not a story about a bad sponsor. The executive in this case was a capable senior leader with strong judgment and authentic intent. They had been placed into a role they had not yet been prepared for, and the pattern that followed was structural, not personal. It is one of the more common patterns I have observed across enterprise transformation programs, and one of the most consistently misdiagnosed.

Most program failures are not execution failures. They are sponsor mismatches.

When governance becomes a debate about delivery practices

When the sponsor does not understand what an enterprise transformation actually requires, governance forums stop functioning as decision bodies and start functioning as practice debates.

You see it in the questions that get asked. Why do we need a dedicated testing phase? Can the Build timeline be compressed? Why does cutover need a full weekend? Why do we need more Scrum Masters across 20 product teams? Can the US team simply work longer hours instead of using offshore resources? On one program, the sponsor suggested shifting the entire project’s working hours to India time, despite roughly 85 percent of the delivery organization being based in the United States.

None of these questions are unreasonable in isolation. Each one targets a real cost or timeline pressure. The problem is what is missing underneath them: an understanding of the operational risks the original choices were designed to mitigate.

When sponsors ask delivery-practice questions without that context, the program leadership team ends up defending the work instead of advancing it. Decision velocity drops. Trust between the program and its sponsor erodes. Senior delivery talent disengages from governance forums where the conversation never reaches the decisions they need made. What looks from the outside like an active sponsor producing engagement is, from inside the program, an active drain on the cycles needed to deliver.

The compounding cost is not unique to any single program. PMI’s research on executive sponsorship consistently identifies sponsor engagement quality, rather than sponsor presence alone, as one of the strongest predictors of project success. The visible symptom is debate. The actual cost is unmade decisions.

Authority is rarely the issue. Literacy is

When transformations stall under a mismatched sponsor, the diagnostic instinct is to question the sponsor’s authority. Are they senior enough? Do they have the cross-functional reach? Can they unblock?

In most of the programs I have led or advised, authority was not the limiting factor. The sponsor in the SAP program above had ample authority. They could unblock any decision the program needed. What had not been developed was the transformation literacy to know which decisions mattered, which were technical noise and which were execution risks that should not be optimized away.

This is what I have come to think of as the literacy problem. Sponsors elevated into transformation roles often have deep functional expertise (finance, operations, business unit leadership) but limited exposure to the distinct functions of PMO, organizational change management, agile delivery, testing and cutover, and how each one reduces a specific category of implementation risk. They are not expected to be SAP configuration experts. But they need enough transformation literacy to recognize which questions actually belong in a steering committee.

Harvard Business Review’s research on effective executive sponsorship has emphasized that sponsorship effectiveness depends as much on judgment as on authority. Judgment is where literacy becomes operational. A sponsor with authority but limited transformation literacy will optimize for speed and cost in ways that consistently underestimate risk. A sponsor with both will make the tradeoffs the program actually needs.

Prosci’s longstanding benchmark studies on change management have ranked active and visible executive sponsorship as the single greatest contributor to change success for two decades. The word that matters in that finding is active. Active sponsorship without transformation literacy can introduce real cost. Not because the sponsor is acting against the program, but because the optimization choices they make are based on incomplete information about what the program is built to protect against.

Shift the conversation from delivery practices to business risk

When the sponsor relationship is already in place and cannot be changed, the program leadership team has one move that consistently works: shift the conversation.

On the SAP program above, we stopped explaining why the testing phase existed. We started explaining the business risk of reducing it. We stopped debating the number of Scrum Masters. We started connecting delivery capacity to coordination across more than twenty Agile teams and the business cost of losing that coordination. We reframed offshore support as a way to maintain delivery momentum around the clock rather than asking the U.S. team to sustain fifteen-hour days.

The shift is from defending delivery practice to explaining business risk. The sponsor does not need to understand why testing takes the time it does. They need to understand what the program is exposed to if testing is compressed. They do not need to know how many Scrum Masters are statistically optimal for twenty Agile teams. They need to know what coordination breaks when the number is wrong.

This reframing accomplishes two things. First, it brings the conversation back to the level at which sponsors actually make decisions: tradeoffs between business outcomes and business risks. Second, it builds transformation literacy in the sponsor over time, almost as a byproduct. By the third or fourth iteration of business-risk-framed conversations, the sponsor begins to ask the right questions on their own.

In practice, this happens through small but deliberate moves. When the sponsor asks why a phase needs the time it takes, the program lead names two or three things that could go wrong if the time is cut and what each would cost the business. When the sponsor asks why a role is needed, the program lead names the work that would not get done without it. Every delivery-practice question gets converted into a business-risk answer.

The program leadership team’s job is not to make the sponsor an expert in SAP delivery. It is to provide enough transformation context so that executive decisions reflect both business priorities and implementation realities.

There are a few phrases I have used with executive sponsors over the years that capture the underlying issue. The sharpest one:

If the decision has to go above the sponsor, they are not the sponsor.

Sponsorship is defined by what the sponsor can decide without asking someone else. That is the test. Anything else is the appearance of sponsorship, not the substance.

For CIOs supporting enterprise transformation, the implication is direct. Sponsor selection, or sponsor preparation when selection is not an option, is not a hierarchy question. It is a transformation capability question. The same execution discipline that goes into defining decision rights, structuring governance and protecting delivery momentum should apply, with equal rigor, to assessing sponsor fit and building sponsor literacy before the program begins.

A sponsor does not need to be the technical expert. They do need to know when to trust the people who are.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Four months of Claude Code — that’s all it took for Uber to burn through its entire annual budget for AI. Token after token, engineers embraced the platform with few control mechanisms tying costs to outcomes. The result was a budget runaway and a clear case study in how limited oversight snowballs into an AI blowout.

This is a phenomenon I like to call “shadow tokens” — AI credits paid for by the company but largely invisible to decision-makers. Too many engineers have the final say over how much they consume and, therefore, what it costs. This all-you-can-eat attitude is part of the reason why Microsoft is reportedly winding down many internal licenses across key engineering teams and why one in five organizations is missing its AI spend forecast by more than 50%.

And the trend is only accelerating. By 2028, Gartner predicts that AI coding costs (driven by this kind of ungoverned consumption) will be as much per developer as the salary companies pay that person.

LLMs and agents introduce a new class of variable cost that scales with behavior rather than headcount, putting enterprises on the hook for tools that balloon with workload. I don’t see this as enterprises overspending because they’re reckless — it’s down to a lack of managerial oversight, budget alignment that demands a proven return on investment, and engineer education on how much is too much.

Going forward, CIOs need to thread the AI needle between governance that encourages transparency and reasonable spend without stifling innovation.

When shadow tokens result in real costs

The issue is that AI isn’t a traditional line item. Previously, enterprise leaders onboarded software-as-a-service (SaaS) with a good idea of the total cost. An allocated software seat or annual contract was a known quantity. The cloud added some variation (with fluctuations depending on hosting size), but instances were still modelable. AI flips this status quo on its head — the unit of consumption is behavior and the cost is exponential.

And these specifics aren’t immediately apparent at pilot. Tools can appear inexpensive in controlled experiments yet unpredictably scale depending on session length, context window size, model selection and whether agents run in parallel. This is the fallacy of the $20-per-seat enterprise plan — tokens are charged separately at API rates with no ceiling. The final dollar value of any session is set by factors that finance can’t always model in advance, particularly when these decisions usually rest with the engineers themselves.

According to Deloitte, only 21% of organizations deploying agents have a mature governance model, a real concern because they’re token-eating machines. This is what was happening at Uber — Claude Code in agentic mode was autonomously reading codebases, planning changes across dozens of files and opening pull requests. Each step quickly adds up, with Anthropic’s own documentation noting that agents consume approximately seven times as many tokens as standard sessions.

This is shadow IT and shadow AI, evolved. This time, however, many leaders approved the tool in question without guardrails governing consumption. AI hype adds fuel to the fire and normalizes long sessions. Uber’s CTO, for example, described a company-wide shift toward “agentic software engineering” with employees “who are quietly experimenting, quietly shipping and quietly pushing things forward”. This is an exciting way to test the limits of what’s possible, certainly, but it’s also a position that goes a long way to explaining how the company spent its annual AI budget by April.

Shifting the culture from usage to yield

Engineers haven’t done anything wrong here. In fact, they’re adopting and experimenting as instructed, with Uber creating leaderboards and ranking users by token consumption. More use led to a better ranking, reflecting a culture that lauds new ways of doing things. This behavior is known as “tokenmaxxing,” and its principal knock-on effect is shadow tokens — quantity-over-quality processes that leaders struggle to control until they’re fully realized in the budget. Of course, if management treats adoption metrics as performance metrics, then engineers can’t be blamed for using more tokens. The tension is that the teams driving adoption aren’t the ones managing spend.

None of this is meant to dismiss AI’s productivity possibilities and potential return on investment. Developers save 3.6 hours per week, achieve 60% higher pull request throughput and cut onboarding time in half with automation. Meanwhile, Uber shared that roughly 11% of live backend updates were written by agents with no human in the loop. However, these wins aren’t the problem — it’s that too many teams aren’t connecting input to output. I’ve spoken to admins who discovered their token spend had tripled in a single quarter after using heavier models or accidentally doubling up on agentic applications. Nobody knew until the financial damage was done.

Automation needs to happen sustainably with an eye on the bottom line. In my view, a much better metric for achieving this is AI yield — the measurable business or engineering output generated per dollar spent on tokens. Otherwise, without a feedback loop, even genuinely productive teams are flying blind.

Stopping token waste before an AI blowout

Creating that throughline between AI investment and token consumption starts with established financial metrics. This is possible via maximum spend limits (dictated by spend tagging, workload tiering and cost-per-output benchmarks) per team or project. Then, any additional allocation requires approval, closing the loop between the engineers spending the tokens and the leaders paying for them. AI isn’t cheap and teams should demonstrate a bang for their buck.

This is something we do with our engineering team at Hexnode. Resource allocation for Claude Code and Cursor is tied directly to ROI rather than letting consumption run open-ended. Given the pay-as-you-go nature of these tools, a firm usage limit per team offers simple but essential control.

Similarly, there’s room to apply some of the governance principles IT uses for device management. Things like policy enforcement, role-based access, real-time monitoring and automated alerts can flag usage behavior in advance. Uncovering such insights at the token layer works to identify power users and prevent excessive spending.

We also need to encourage cultures that praise outputs that actually achieve efficiency. AI applications that result in shipping faster, reducing rework and cutting review cycles are gains that should be celebrated. If your company hosts leaderboards, frame unnecessary token burn as wasteful rather than valuable. The organizations creating healthier consumption habits work with their engineers to understand not just how to use AI, but what responsible use looks like and what it costs.

This is a conversation teams need to have now. Anthropic just ended flat-rate pricing for programmatic workloads from June 15. Now, agents, continuous integration pipelines and automated workflows draw from a dedicated monthly credit pool billed separately from the subscription. Once that pool is exhausted, agent tasks either stop entirely or overflow to extra billing. Work can either get very expensive or grind to a halt for teams that aren’t prepared.

Getting a grip on shadow tokens means better rules and tools connecting spend to outcomes. Only by building the financial and cultural infrastructure that encourages sustainable adoption can leaders see what they’re spending, connect it to what they’re getting and course-correct before the costs become a crisis. Ultimately, shadow tokens are only invisible if we choose not to look.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Enterprises typically operate separate systems for internal and external DNS because the two serve different audiences. Public DNS resolves names for services meant to be reached from the internet. Private DNS resolves internal resources, such as databases and internal applications, that should never be visible outside the corporate network. 

While public DNS is usually a single system, private DNS is often scattered across on-premises appliances, cloud-native resolvers, and split-horizon setups, where the same hostname resolves to a different answer depending on whether the query comes from inside or outside the network. Coordinating those deployments across headquarters, branch offices, and multiple clouds means ongoing manual synchronization work for network teams. 

Private DNS itself is not a new concept. It is already available from hyperscalers and established enterprise DNS vendors, but it typically runs apart from public DNS, with its own console, control plane and policy engine.

Cloudflare’s answer is a product it calls Internal DNS.

“Many organizations already use Cloudflare for their public DNS,” Enrique Somoza, product, performance and infrastructure at Cloudflare, told Network World. “Internal DNS extends that same platform to private DNS, so public and private are managed from the same global network and control plane.” 

How it works

Query handling starts at the resolver, not at the zone. That consolidation extends to daily operations as well.

“Instead of operating two separate DNS systems, customers use one API, one audit trail, one dashboard, and one policy engine for every DNS query—whether it is for a public website or an internal application,” Somoza said.

Policy first. The resolver sits ahead of every lookup, not behind it. “Architecturally, Cloudflare Gateway becomes the resolver that customers connect to, and can use WARP, DNS over HTTPS, DNS over TLS, or traditional DNS,” Somoza said. “Gateway evaluates zero -trust policies first, then routes the query to the appropriate DNS view based on context, such as source IP, device posture, or network location.”

No public path in. Internal zones sit outside the public DNS hierarchy entirely. “Internal zones are never assigned public nameservers—they are only reachable through Gateway, so every query is evaluated before it is resolved,” Somoza said.

One hostname, multiple answers. Branch offices, data centers and cloud environments no longer each need their own resolver stack. “Operationally, this simplifies environments that span branch offices, data centers, and multiple clouds,” Somoza said. “The same internal hostname can return different answers depending on where the request originated without maintaining separate resolver infrastructure, conditional forwarders, or duplicate zone files.”

Somoza described the underlying objective in direct terms: “The goal is to make internal DNS behave like a single service instead of a collection of independent deployments,” he said.

View selection. The same hostname can resolve to different IP addresses depending on where the request comes from. Gateway makes that call using several client signals. 

“View selection is policy driven,” Somoza said. “Gateway resolver policies evaluate the context of each DNS query, including attributes like source IP, device identity, or network location and determine which DNS view should answer the request.”

A view is a container, not a separate infrastructure stack. Somoza explained that a view is simply a logical grouping of internal zones. For example, a company could have separate views for Europe and North America, or for corporate users and operational technology networks.

Latency and resilience. Internal DNS inherits its performance characteristics from Cloudflare’s existing public network. “Internal DNS runs on Cloudflare’s global network, so queries are answered by the nearest available Gateway location, helping keep latency low for connected users,” Somoza said. “Because Internal DNS runs on the same global infrastructure as Cloudflare’s public DNS, it benefits from the same anycast architecture, geographic distribution, and resilient network design.”

How this differs from split-horizon DNS

Internal DNS replaces the duplicate-zone model traditional split-horizon setups depend on.

“Before migrating, many organizations maintain multiple versions of the same internal DNS zones across headquarters, branch offices, and cloud environments,” Somoza explained. “Conditional forwarders determine which resolver answers each query, and keeping those environments synchronized becomes an ongoing operational task.”

Internal DNS collapses those duplicate zones into a single authoritative copy split across views instead. “With Internal DNS, that configuration becomes much simpler,” Somoza said. “A customer might create a single corp.internal zone in Cloudflare and define multiple DNS views.”

For example, users in headquarters could receive one internal IP address for wiki.corp.internal, while branch offices receive a different address. Somoza emphasized that the zone itself only exists once. “Instead of maintaining multiple copies of the same configuration, administrators manage a single source of truth,” he said.

Early use cases and migration challenges

Not surprisingly, Somoza noted that the first use case Cloudflare sees for Internal DNS is for split-horizon DNS consolidation. There is also interest from organizations that operate across multiple cloud providers that want one consistent internal DNS service instead of managing separate DNS platforms in each environment.

Another common use case is extending zero-trust policies to internal name resolution. “Customers already use Gateway to control access to internet traffic, and Internal DNS lets them apply similar policy decisions before internal names are resolved,” Somoza said.

When it comes to migration, the friction customers report during migration is procedural rather than architectural.

“Customers need to think through API permissions, connectivity, and how existing local DNS forwarding rules interact with Gateway,” Somoza said. “Those are all well understood migration steps and customers often run both environments in parallel before completing the transition.”

Four months of Claude Code — that’s all it took for Uber to burn through its entire annual budget for AI. Token after token, engineers embraced the platform with few control mechanisms tying costs to outcomes. The result was a budget runaway and a clear case study in how limited oversight snowballs into an AI blowout.

This is a phenomenon I like to call “shadow tokens” — AI credits paid for by the company but largely invisible to decision-makers. Too many engineers have the final say over how much they consume and, therefore, what it costs. This all-you-can-eat attitude is part of the reason why Microsoft is reportedly winding down many internal licenses across key engineering teams and why one in five organizations is missing its AI spend forecast by more than 50%.

And the trend is only accelerating. By 2028, Gartner predicts that AI coding costs (driven by this kind of ungoverned consumption) will be as much per developer as the salary companies pay that person.

LLMs and agents introduce a new class of variable cost that scales with behavior rather than headcount, putting enterprises on the hook for tools that balloon with workload. I don’t see this as enterprises overspending because they’re reckless — it’s down to a lack of managerial oversight, budget alignment that demands a proven return on investment, and engineer education on how much is too much.

Going forward, CIOs need to thread the AI needle between governance that encourages transparency and reasonable spend without stifling innovation.

When shadow tokens result in real costs

The issue is that AI isn’t a traditional line item. Previously, enterprise leaders onboarded software-as-a-service (SaaS) with a good idea of the total cost. An allocated software seat or annual contract was a known quantity. The cloud added some variation (with fluctuations depending on hosting size), but instances were still modelable. AI flips this status quo on its head — the unit of consumption is behavior and the cost is exponential.

And these specifics aren’t immediately apparent at pilot. Tools can appear inexpensive in controlled experiments yet unpredictably scale depending on session length, context window size, model selection and whether agents run in parallel. This is the fallacy of the $20-per-seat enterprise plan — tokens are charged separately at API rates with no ceiling. The final dollar value of any session is set by factors that finance can’t always model in advance, particularly when these decisions usually rest with the engineers themselves.

According to Deloitte, only 21% of organizations deploying agents have a mature governance model, a real concern because they’re token-eating machines. This is what was happening at Uber — Claude Code in agentic mode was autonomously reading codebases, planning changes across dozens of files and opening pull requests. Each step quickly adds up, with Anthropic’s own documentation noting that agents consume approximately seven times as many tokens as standard sessions.

This is shadow IT and shadow AI, evolved. This time, however, many leaders approved the tool in question without guardrails governing consumption. AI hype adds fuel to the fire and normalizes long sessions. Uber’s CTO, for example, described a company-wide shift toward “agentic software engineering” with employees “who are quietly experimenting, quietly shipping and quietly pushing things forward”. This is an exciting way to test the limits of what’s possible, certainly, but it’s also a position that goes a long way to explaining how the company spent its annual AI budget by April.

Shifting the culture from usage to yield

Engineers haven’t done anything wrong here. In fact, they’re adopting and experimenting as instructed, with Uber creating leaderboards and ranking users by token consumption. More use led to a better ranking, reflecting a culture that lauds new ways of doing things. This behavior is known as “tokenmaxxing,” and its principal knock-on effect is shadow tokens — quantity-over-quality processes that leaders struggle to control until they’re fully realized in the budget. Of course, if management treats adoption metrics as performance metrics, then engineers can’t be blamed for using more tokens. The tension is that the teams driving adoption aren’t the ones managing spend.

None of this is meant to dismiss AI’s productivity possibilities and potential return on investment. Developers save 3.6 hours per week, achieve 60% higher pull request throughput and cut onboarding time in half with automation. Meanwhile, Uber shared that roughly 11% of live backend updates were written by agents with no human in the loop. However, these wins aren’t the problem — it’s that too many teams aren’t connecting input to output. I’ve spoken to admins who discovered their token spend had tripled in a single quarter after using heavier models or accidentally doubling up on agentic applications. Nobody knew until the financial damage was done.

Automation needs to happen sustainably with an eye on the bottom line. In my view, a much better metric for achieving this is AI yield — the measurable business or engineering output generated per dollar spent on tokens. Otherwise, without a feedback loop, even genuinely productive teams are flying blind.

Stopping token waste before an AI blowout

Creating that throughline between AI investment and token consumption starts with established financial metrics. This is possible via maximum spend limits (dictated by spend tagging, workload tiering and cost-per-output benchmarks) per team or project. Then, any additional allocation requires approval, closing the loop between the engineers spending the tokens and the leaders paying for them. AI isn’t cheap and teams should demonstrate a bang for their buck.

This is something we do with our engineering team at Hexnode. Resource allocation for Claude Code and Cursor is tied directly to ROI rather than letting consumption run open-ended. Given the pay-as-you-go nature of these tools, a firm usage limit per team offers simple but essential control.

Similarly, there’s room to apply some of the governance principles IT uses for device management. Things like policy enforcement, role-based access, real-time monitoring and automated alerts can flag usage behavior in advance. Uncovering such insights at the token layer works to identify power users and prevent excessive spending.

We also need to encourage cultures that praise outputs that actually achieve efficiency. AI applications that result in shipping faster, reducing rework and cutting review cycles are gains that should be celebrated. If your company hosts leaderboards, frame unnecessary token burn as wasteful rather than valuable. The organizations creating healthier consumption habits work with their engineers to understand not just how to use AI, but what responsible use looks like and what it costs.

This is a conversation teams need to have now. Anthropic just ended flat-rate pricing for programmatic workloads from June 15. Now, agents, continuous integration pipelines and automated workflows draw from a dedicated monthly credit pool billed separately from the subscription. Once that pool is exhausted, agent tasks either stop entirely or overflow to extra billing. Work can either get very expensive or grind to a halt for teams that aren’t prepared.

Getting a grip on shadow tokens means better rules and tools connecting spend to outcomes. Only by building the financial and cultural infrastructure that encourages sustainable adoption can leaders see what they’re spending, connect it to what they’re getting and course-correct before the costs become a crisis. Ultimately, shadow tokens are only invisible if we choose not to look.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Over the past year, I had the same conversation with almost every team starting an AI initiative. Should we build on Semantic Kernel, AutoGen or Foundry?

At first it felt like the most important architectural decision we’d make. Each framework had its own philosophy, each promised to be the foundation for enterprise AI, and picking the wrong one felt like an expensive mistake. I spent a lot of time helping teams weigh the trade-offs.

Looking back, I think we were asking the wrong question. I certainly was.

I watched teams spend months debating SDKs while the decisions that actually decided whether their applications survived production went unexamined. Some built elaborate orchestration layers for workflows that a few deterministic functions would have handled. Others avoided agent frameworks entirely and later found they’d designed themselves into a corner.

Then Microsoft settled it for us. It introduced the unified Agent Framework, quietly moved Semantic Kernel and AutoGen into maintenance mode, and the debate I’d spent months refereeing was suddenly over. Turns out the answer to “which of the three” was “none of the three, here’s a fourth.” The framework hit version 1.0 and general availability in April 2026, stable across .NET and Python.

What surprised me wasn’t the decision. It was how fast a debate that had eaten so much of our attention stopped mattering. Microsoft changed the menu.

It didn’t change the meal.

The framework was never the hard part

Framework selection dominated almost every early conversation I had about enterprise agents. Which SDK do we standardize on? Which orchestration model gives us the most flexibility? Which one is Microsoft actually betting on?

Fair questions. But after a year of watching these projects play out, I’ve slowly come around to a different view. Those weren’t the questions that decided anything.

The first question I ask now is much smaller. Does this thing actually need an agent?

It sounds obvious, and I still get it wrong sometimes. But it’s the mistake I see most. On one project, a team spent weeks designing a multi-agent workflow for a process that ran the same four steps every time: read a document, validate it, call an API, send a notification. The diagrams looked great. The system in production didn’t. A few well-tested functions would have been easier to build, easier to maintain and a lot easier to trust.

Part of this is just that “agent” has become the word everyone reaches for. Sometimes it’s the right call. Sometimes it’s a workflow we already knew how to build, wearing a newer label. An agent earns its complexity when it genuinely has to decide things you can’t predetermine, choosing between tools, adapting to what it finds, working out its own next step. If you already know every step, you have a workflow, and a workflow is usually the better engineering choice. The consolidation didn’t change that. It just made it easier to see.

What building production agents actually taught me

Once I stopped fixating on frameworks, the same three problems kept showing up. None of them had anything to do with the SDK.

Context beats model choice

Early on I spent a lot of time comparing models, the way you’d agonize over a restaurant menu and then order what you always order. Now I spend most of it thinking about context, which is far less fun and far more useful.

I’ve watched good models fail because they were handed too much, not too little. One team I worked with gave the model access to nearly every internal document they had on the theory that more information meant better answers. It went the other way. Responses got slower, less consistent and sometimes skipped right past the thing that actually mattered. When we cut the context down to only what the task needed, the quality jumped almost immediately. I didn’t predict that. It taught me to be suspicious of “just give it everything.”

The best agent systems I’ve worked on weren’t the ones with the biggest context windows. They were the ones careful about what reached the model, and when. That’s not something the framework hands you.

Failure is where the real work is

Most agent demos look great because they’re built around the happy path. Production doesn’t extend that courtesy.

I remember a project where everything held up in testing. Then a downstream API timed out after the agent had already completed several earlier steps. We couldn’t just restart, because part of the business process had already gone through. We ended up spending far more time on recovery logic than we ever spent on prompts. That project changed how I think about this work. The hard part was never getting the model to make a decision. It was making sure the system didn’t fall apart when reality refused to follow the script.

Tool calls fail partway through. APIs return inconsistent data. Models call the same tool over and over because the last answer wasn’t what they wanted. That’s not the exception; that’s a normal Tuesday. Whether you retry, roll back, pause for a human or push on with partial results is a judgment call, and no framework is going to make it for you.

Identity is the real security boundary

This one surprised me most. The moment an agent stops being a chatbot and starts touching real business systems, identity matters more than orchestration.

Every project gets to the same question eventually. Who is this agent actually acting as? The developer’s credentials? A service account? The user who asked? Get it wrong and you’ve built something autonomous running with more access than any single person should have, which is exactly the kind of thing that looks fine until an audit. The Agent Framework, like most modern tooling, makes it easier to wire agents to tools through standards like the Model Context Protocol. That helps. But where human approval belongs, what needs extra authorization, how much rope to give the thing, those are still yours to decide.

The surprises weren’t technical

Here’s what I didn’t see coming. The hardest part of last year wasn’t technical at all. It was organizational. The moment a team heard “agent,” expectations shifted under everyone’s feet. Business stakeholders started expecting full autonomy. Developers assumed the thing could reason its way through anything. People started designing for flexibility before we’d even agreed on what problem we were solving. The word did damage before any code did. I found myself spending as much time resetting expectations as I did discussing architecture.

Build for change, not for today’s winner

I don’t think the teams that struggled last year picked the wrong framework. Semantic Kernel was reasonable. AutoGen was reasonable. Foundry made sense for plenty of cases. I’d have signed off on any of them.

The ones that got hurt put all their eggs in one framework, treating it as the foundation of the whole system instead of as one more dependency. Microsoft provided a migration path. But teams that had tightly coupled their applications to framework-specific abstractions discovered that migrating and rewriting are not the same thing. That wasn’t Microsoft’s doing. It was their own architecture’s. The teams that moved easily had kept their business logic, prompts and orchestration loose enough to evolve independently of any one SDK. For them, the change was a manageable project, not a teardown.

For what it’s worth, nobody I work with is treating this as an emergency. Most are moving the smaller workloads first, watching how they behave and leaving the production-critical systems alone until they actually understand the new abstractions. That’s the right instinct. And I doubt this is the last consolidation we’ll see, the ecosystem is still young, frameworks will keep absorbing each other and over time the differences between them will be operational more than architectural.

I don’t regret the framework debates, honestly. They were reasonable at the time. What changed wasn’t Microsoft’s roadmap.

It was mine. Watching these systems run in production taught me that the framework is the easiest piece to swap out. Recovery logic, context management, security boundaries, the business workflow itself, those stay with you long after today’s SDK gets replaced by tomorrow’s.

So, Microsoft made one decision easier by turning three frameworks into one. Good. Five years from now we’ll be on different tools, and we’ll still be asking the same handful of questions.

Does this actually need an agent? Does it have the right context? Can it recover when something breaks, because something will? Is it acting as the right person?

Those questions outlast every rewrite. That’s where I’ve learned to put my effort.

Frameworks come and go. Good architecture has to survive all of them.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Late in a large enterprise SAP transformation, the strategic governance conversations began to drift. Instead of executive decisions, we found ourselves debating whether the program needed dedicated testing, whether cutover required a full weekend, whether twenty Agile teams really needed coordination support and whether offshore resources were adding value at all.

The questions were not coming from the delivery teams. They were coming from the executive sponsor.

The sponsor had recently been elevated into a senior leadership role and had never sponsored a technology transformation at this scale. The challenge was not authority. The sponsor had every right to ask any question they wanted. The challenge was that strategic governance had quietly turned into a debate about delivery practices, because the sponsor did not yet have the transformation context to focus the conversation anywhere else.

This is not a story about a bad sponsor. The executive in this case was a capable senior leader with strong judgment and authentic intent. They had been placed into a role they had not yet been prepared for, and the pattern that followed was structural, not personal. It is one of the more common patterns I have observed across enterprise transformation programs, and one of the most consistently misdiagnosed.

Most program failures are not execution failures. They are sponsor mismatches.

When governance becomes a debate about delivery practices

When the sponsor does not understand what an enterprise transformation actually requires, governance forums stop functioning as decision bodies and start functioning as practice debates.

You see it in the questions that get asked. Why do we need a dedicated testing phase? Can the Build timeline be compressed? Why does cutover need a full weekend? Why do we need more Scrum Masters across 20 product teams? Can the US team simply work longer hours instead of using offshore resources? On one program, the sponsor suggested shifting the entire project’s working hours to India time, despite roughly 85 percent of the delivery organization being based in the United States.

None of these questions are unreasonable in isolation. Each one targets a real cost or timeline pressure. The problem is what is missing underneath them: an understanding of the operational risks the original choices were designed to mitigate.

When sponsors ask delivery-practice questions without that context, the program leadership team ends up defending the work instead of advancing it. Decision velocity drops. Trust between the program and its sponsor erodes. Senior delivery talent disengages from governance forums where the conversation never reaches the decisions they need made. What looks from the outside like an active sponsor producing engagement is, from inside the program, an active drain on the cycles needed to deliver.

The compounding cost is not unique to any single program. PMI’s research on executive sponsorship consistently identifies sponsor engagement quality, rather than sponsor presence alone, as one of the strongest predictors of project success. The visible symptom is debate. The actual cost is unmade decisions.

Authority is rarely the issue. Literacy is

When transformations stall under a mismatched sponsor, the diagnostic instinct is to question the sponsor’s authority. Are they senior enough? Do they have the cross-functional reach? Can they unblock?

In most of the programs I have led or advised, authority was not the limiting factor. The sponsor in the SAP program above had ample authority. They could unblock any decision the program needed. What had not been developed was the transformation literacy to know which decisions mattered, which were technical noise and which were execution risks that should not be optimized away.

This is what I have come to think of as the literacy problem. Sponsors elevated into transformation roles often have deep functional expertise (finance, operations, business unit leadership) but limited exposure to the distinct functions of PMO, organizational change management, agile delivery, testing and cutover, and how each one reduces a specific category of implementation risk. They are not expected to be SAP configuration experts. But they need enough transformation literacy to recognize which questions actually belong in a steering committee.

Harvard Business Review’s research on effective executive sponsorship has emphasized that sponsorship effectiveness depends as much on judgment as on authority. Judgment is where literacy becomes operational. A sponsor with authority but limited transformation literacy will optimize for speed and cost in ways that consistently underestimate risk. A sponsor with both will make the tradeoffs the program actually needs.

Prosci’s longstanding benchmark studies on change management have ranked active and visible executive sponsorship as the single greatest contributor to change success for two decades. The word that matters in that finding is active. Active sponsorship without transformation literacy can introduce real cost. Not because the sponsor is acting against the program, but because the optimization choices they make are based on incomplete information about what the program is built to protect against.

Shift the conversation from delivery practices to business risk

When the sponsor relationship is already in place and cannot be changed, the program leadership team has one move that consistently works: shift the conversation.

On the SAP program above, we stopped explaining why the testing phase existed. We started explaining the business risk of reducing it. We stopped debating the number of Scrum Masters. We started connecting delivery capacity to coordination across more than twenty Agile teams and the business cost of losing that coordination. We reframed offshore support as a way to maintain delivery momentum around the clock rather than asking the U.S. team to sustain fifteen-hour days.

The shift is from defending delivery practice to explaining business risk. The sponsor does not need to understand why testing takes the time it does. They need to understand what the program is exposed to if testing is compressed. They do not need to know how many Scrum Masters are statistically optimal for twenty Agile teams. They need to know what coordination breaks when the number is wrong.

This reframing accomplishes two things. First, it brings the conversation back to the level at which sponsors actually make decisions: tradeoffs between business outcomes and business risks. Second, it builds transformation literacy in the sponsor over time, almost as a byproduct. By the third or fourth iteration of business-risk-framed conversations, the sponsor begins to ask the right questions on their own.

In practice, this happens through small but deliberate moves. When the sponsor asks why a phase needs the time it takes, the program lead names two or three things that could go wrong if the time is cut and what each would cost the business. When the sponsor asks why a role is needed, the program lead names the work that would not get done without it. Every delivery-practice question gets converted into a business-risk answer.

The program leadership team’s job is not to make the sponsor an expert in SAP delivery. It is to provide enough transformation context so that executive decisions reflect both business priorities and implementation realities.

There are a few phrases I have used with executive sponsors over the years that capture the underlying issue. The sharpest one:

If the decision has to go above the sponsor, they are not the sponsor.

Sponsorship is defined by what the sponsor can decide without asking someone else. That is the test. Anything else is the appearance of sponsorship, not the substance.

For CIOs supporting enterprise transformation, the implication is direct. Sponsor selection, or sponsor preparation when selection is not an option, is not a hierarchy question. It is a transformation capability question. The same execution discipline that goes into defining decision rights, structuring governance and protecting delivery momentum should apply, with equal rigor, to assessing sponsor fit and building sponsor literacy before the program begins.

A sponsor does not need to be the technical expert. They do need to know when to trust the people who are.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

The traditional methods of measuring success no longer tell the full story. Here’s what should replace them — and why.

In brief:

• AI is transforming software as a service (SaaS), and the old ways of keeping score no longer apply.
• Smart companies are evolving new metrics that provide deeper insight into how AI-native software is performing in a new marketplace.
• These changes impact everything from pricing to valuations.

The transformation of the software-as-a-service (SaaS) industry toward AI-native operating companies is rapidly changing the unit of value across the industry.

The traditional metric of seats — which measured access — is rapidly giving way to credits designed to measure work performed. This evolution is upending the industry in multiple ways, impacting everything from pricing to enterprise valuations.

While many companies still cling to seat-based metrics to measure growth, efficiency and durability, the future is likely to be one in which companies utilize a credit-centric metrics framework, with seats and outcomes as the bookends of a spectrum.

Why do software companies need new metrics?

Why the rethink, and why now? There are five major forces that are driving this shift:

1. The unit of value is changing. Seats measured who could access software, and credits measure what the software actually does. But in an AI-native world, agents don’t have seats; they have workloads. Over the past 18 months, every major SaaS platform has moved to some forms of credit or consumption unit.
2. The cost of goods sold (COGS) is exploding. AI inference adds real per-unit costs that scale with usage. In an AI-native world, software companies can’t scale to infinite users at near‑zero marginal cost as before.
3. Buying is moving up the org chart. AI-native applications shift purchasing to higher-level operators — such as line-of-business leaders or chief operating officers — which expands the market from software budgets to labor budgets. And because AI agents replace services as well as software, the total market opportunity is 3x to 10x larger than traditional SaaS.
4. Time to value (TTV) is collapsing. With AI-native tools, customers start seeing meaningful results in weeks rather than quarters. Onboarding and setup are fast, workflows are pre-built, and there’s no need for extensive customer success or professional services — dramatically reducing implementation time and costs.
5. Retention is bifurcating. AI forces clarity in a way that traditional SaaS couldn’t. Products that can provide value become even “stickier” and retain customers. Those that don’t churn faster. In an AI-native marketplace, the middle disappears.

How this shift is impacting pricing

Given how AI-native software is transforming the market, the shift to more variable pricing options is inevitable.

Seats won’t go away completely. Subscription pricing based on the number of users is stable and predictable and will continue to work for some customers. Tokens — the use of pass-through pricing for underlying compute — will fit those customers where the AI feature is commoditized or the buyer wants transparency into costs.

Credits will likely become the dominant architecture because they provide a simple metric for both customers and providers. The vendor sets the conversation ratio between credits and underlying compute, shielding the customer from inference cost details. Credits are easy to understand and can be packaged into annual contracts for multiple features and products.

Finally, the industry will likely see some move toward outcome-based pricing for results such as resolved tickets, recovered revenue or qualified leads. This strategy will mostly be limited to verticals where it is easy to prove AI impacted the result.

Where a software vendor sits on this spectrum is a signal of differentiation and pricing power. Credits are where most defensible AI-native businesses are landing because they balance customer predictability with vendor margin control.

How AI upends classic SaaS metrics

When SaaS was in its infancy, companies settled on key metrics designed to answer a small set of core questions. Are we growing? Are customers using the product? Are we retaining and expanding accounts?

But as AI upends software itself, it is also requiring companies to adopt new metrics to track success. These new metrics fall into three primary buckets, rebuilt around the pricing spectrum described earlier and the trend toward credits as the primary frame:

Revenue composition

• Committed credit annual recurring revenue (ARR) vs. burndown ARR: Measuring the credits sold on annual commitment vs. those consumed and replenished. This is the single most important split for valuation. Committed credits behave like subscription and burndown behaves like usage.
• Credit utilization rate: The percentage of purchased credits consumed per period. This is a leading indicator of renewal sizing.
• Credit burn velocity: How fast is a customer consuming their credits, and is that consumption increasing or decreasing quarter over quarter? This metric predicts expansion or contraction before it shows up in ARR.
• Effective price per credit: The real revenue per credit after discounts, overage and rollover, which can detect revenue leakage and help companies set smarter guide rails.

Margin reality

• Credit margin: The gross profit the company earns per credit after subtracting inference costs. This is the core economic unit for AI-native, usage-based businesses — the replacement for gross margin per seat used in SaaS.
• Inference-adjusted gross margin: By carving out AI inference costs separately in the P&L statement, you can see true AI margins, avoid hiding deterioration inside blended SaaS margins, and clearly distinguish AI economics from legacy SaaS economics.
• Compute leverage ratio: This metric measures how efficiently the business converts compute spend into revenue. It shows whether your AI margins are improving as you scale.
• AI-adjusted “Rule of 40”: This updated metric recalibrates the traditional growth and profitability benchmark to account for AI’s lower gross margins and variable inference costs, giving a more accurate picture of business health for AI-native companies.

Behavioral and value signals

• Time-to-first outcome: Replaces traditional onboarding metrics. Tracks how fast a customer reaches their first measurable result.
• Adoption: AI-native adoption is measured by workflow penetration and active agent density, not seat count. As AI replaces human-driven usage, the unit of adoption shifts from people to automated workflows and agents.
• Net credit retention (NCR): Credit-volume retention across the customer base, tracked separately from net recurring revenue to avoid price-change impact.

Along with these new metrics, the industry’s transformation is prompting companies to retire or recalibrate old SaaS measures, including per-seat ARR as a primary key performance indicator (KPI), traditional magic number calibrated to subscription dynamics, unadjusted Rule of 40, customer success metrics tied to human touchpoints, and blended gross margin without AI COGS carve-outs.

What does this mean for enterprise value calculations?

As the internal metrics of success change, so do the ways the investment community measures growth and long-term viability.

Increasingly, a company’s valuation multiple depends on whether its revenue behaves like committed subscription ARR or volatile usage ARR, and the commit‑to‑burndown ratio is the metric investors use to decide where the company fits.

For example, a business with 80% committed credit ARR could trade closer to subscription comps and one with 80% burndown could trade closer to usage comps even though both have the same types of customers. Being able to proactively explain the commit‑to‑burndown mix can help companies avoid undervaluation.

In addition, utilization is expected to replace net promoter scores and seat usage as the primary predictor of churn or expansion. Low utilization guarantees downsizing at renewal, so companies must track utilization cohorts the same way SaaS tracks logo retention cohorts today.

We’re also seeing an inversion of the operating model, with R&D and COGS moving up the P&L and sales and marketing (S&M) and customer success (CS) moving down or sideways. The net operating leverage profile is structurally different from classical SaaS, and the cost-to-scale curve looks different too.

Finally, credit margin engineering is a hidden value-creation lever. The gap between price per credit and cost per credit is set by the software vendor and can be optimized. Most operators have barely started managing this rigorously, and the ones who do will pull away on margin.

What this means for leaders, boards and investors

The shift from classic SaaS metrics to new AI‑native measures isn’t cosmetic. It represents the seismic change the industry is experiencing as AI matures and transforms products and organizations.

While these metrics — and perhaps others yet to be determined — may evolve over time, there is no doubt they are already changing how AI companies allocate capital, price products, incent sales teams, evaluate performance and communicate with investors.

It’s important to remember that SaaS metrics were practical tools for a specific era of software. As that era draws to a close, winning companies will choose new metrics that shape behavior and drive smart decision-making.

The views reflected in this article are the views of the author and do not necessarily reflect the views of Ernst & Young LLP or other members of the global EY organization.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Healthy software companies typically don’t suddenly eliminate one-fifth of their workforce, but monday.com is doing just that as it bets on flatter teams, AI agents, and customer implementation expertise as the winning combination in the AI era.

Monday.com co-founder and co-CEO Eran Zinman today announced the “very difficult decision” to reduce the AI work platform company’s global workforce by about 20%, or 620 people.

The move has nothing to do with increasing margins or replacing humans with AI, he insisted in his post on LinkedIn; rather, it’s a calculated decision to trim down and hone the company’s focus as AI becomes integral to day-to-day workflows.

“This is not a distress signal; it is a deliberate reset, disclosed with its price attached,” said Sanchit Vir Gogia, chief analyst at Greyhound Research. “The industry has quietly swapped the meaning of productivity, and this filing is the clearest exhibit yet.”

A ‘significant opportunity’ in technology

In a SEC filing this week, monday.com said its restructuring plan reflects the “ongoing transformation of its product, marketing, and go-to-market strategy.” The move is intended to support a “leaner, more focused operating model” as the company continues to invest in its AI-driven strategy.

Zinman noted in his post that the company has shifted to “doing the work with AI and not just managing it,” and is focused on building environments where “people and AI agents [work] together in one workspace.”

In recent months, monday.com has evolved its products, strategy, and the way it serves its customers, and Zinman contended that “the organization we built for our previous chapter is not the organization that fits the new AI era.” Monday.com needs to “execute more decisively,” take on new challenges, and quickly respond to market changes, he said.

“We have never seen such a significant opportunity in software, driven by such exciting technology,” Zinman noted. He emphasized that the reduction is not to replace people with AI, nor to improve margins; the “vast majority” of savings will be reinvested into talent, products, and AI.

The restructuring will result in a “flatter organization” with fewer management layers and smaller, more autonomous teams, and monday.com also has a new go-to-market model, Zinman explained. Customers expect “deeper implementation support” as they deploy AI, and the company will work more closely with customers, increase its on-site presence, create new roles, and “adapt many existing ones.” In its SEC filing, the company said it expects to continue hiring in “key strategic areas” throughout 2026.

Workers will be expected to work better, “not harder,” Zinman noted. He pointed to several past examples where work could have been done in a few days, but instead took many months with “multiple meetings and endless friction.”

“This wasn’t people’s fault and everyone was frustrated by this,” he said. “Our new org changes ownership to allow people to make decisions and move fast.”

A spokesperson for monday.com declined to comment further on the staff reductions.

Monday.com’s key market advantages

Monday.com certainly isn’t struggling; the company expects 19% to 20% year-over-year growth in 2026.

“Companies in that position do not restructure because they must,” Greyhound’s Gogia noted. “They restructure because they have decided to become something else.”

Melody Brue, VP and principal analyst at Moor Insights & Strategy, pointed out that organizational redesign is important for real AI transformation, but while it can signal confidence to the market, it can still be “devastating” to humans.

While the company looks as though it’s trying to do right, that ultimately remains to be seen, she said. “There are often hidden internal bruises that can surface long after layoffs.”

Monday.com’s advantage is in its “structured substrate,” Gogia noted; its boards, permissions and typed workflows give agents something firmer to act on than just documents and chat history. The company highlights its natively built agents that can be configured by any team member, as well as connectors with Claude, Microsoft Copilot, and ChatGPT, and dedicated routes for external agents to authenticate and operate.

“For some time, the sharper enterprise question has been shifting from who has an agent to who owns the governed runtime in which an agent can safely act,” he said. “Structured work is a serious claim on that runtime.”

But parts of monday.com’s agent estate remain in staged release, and its product is ultimately “mid-transition,” Gogia pointed out; its agent builder carried a beta label as recently as March,. Also, the company’s pricing model changed in May to a hybrid model charging for seats as well as mandatory AI credits. And, while its AI-powered no-code builder monday vibe passed $1 million in annual recurring revenue within two and a half months, monday.com has not released subsequent outcomes, usage volumes, or attach rates.

Further, there’s an element of “gravity” with its competitors, he observed. Asana is reorganizing teams around agents, Atlassian is wiring agents into the developer estate, and others are simply bundling them into their offerings: Microsoft is doing so across the productivity stack, and ServiceNow across enterprise operations, each with identity and procurement built in.

“Their pull is strongest exactly where monday.com wants to grow, in the largest accounts, where control-plane depth and administrative reach decide the deal,” said Gogia.

Actions for the near-term

Going forward, buyers should focus on operating risk, not headline risk, Moor’s Brue noted. In practice, that’s continuity of service, roadmap consistency, and strength of enterprise support. Productivity should be valued as better outcomes per unit of organizational effort, not mere activity.

“It should be a measure of how much smoother, faster, and more effective the operating model becomes when AI is built into the work,” said Brue.

Gogia noted that strain surfaces first in customer service, and monday.com’s attention is being redistributed. The company’s annual report disclosed that its focus is now concentrated on the largest accounts, with support for medium-sized clients moved to an AI-first and human-supported model.

During the first month of the transition, buyers should track named account continuity and escalation times, he advised. By the first quarter, keep an eye on whether credit governance and admin controls mature on schedule, and if the roadmap beyond the AI estate keeps pace. By the half-year mark, determine whether promised implementation depth is producing outcomes or “simply more billable engagement.”

Support tiers should be enumerated in writing before renewal, and buyers should contract for “side exits,” Gogia emphasized, with overage pricing fixed in advance, the right to pause consumption, and portability for workflows and agent configuration “if the relationship sours.” Finance should also insist on monthly consumption reporting by capability. Further, integration efforts, partner dependency, and change management should be considered first-class costs of the agent era, “not as afterthoughts to a license.”

“A license was a known cost,” said Gogia. “A meter is a behavior, and behavior is harder to forecast than headcount.”

This article originally appeared on CIO.com.

Healthy software companies typically don’t suddenly eliminate one-fifth of their workforce, but monday.com is doing just that as it bets on flatter teams, AI agents, and customer implementation expertise as the winning combination in the AI era.

Monday.com co-founder and co-CEO Eran Zinman today announced the “very difficult decision” to reduce the AI work platform company’s global workforce by about 20%, or 620 people.

The move has nothing to do with increasing margins or replacing humans with AI, he insisted in his post on LinkedIn; rather, it’s a calculated decision to trim down and hone the company’s focus as AI becomes integral to day-to-day workflows.

“This is not a distress signal; it is a deliberate reset, disclosed with its price attached,” said Sanchit Vir Gogia, chief analyst at Greyhound Research. “The industry has quietly swapped the meaning of productivity, and this filing is the clearest exhibit yet.”

A ‘significant opportunity’ in technology

In a SEC filing this week, monday.com said its restructuring plan reflects the “ongoing transformation of its product, marketing, and go-to-market strategy.” The move is intended to support a “leaner, more focused operating model” as the company continues to invest in its AI-driven strategy.

Zinman noted in his post that the company has shifted to “doing the work with AI and not just managing it,” and is focused on building environments where “people and AI agents [work] together in one workspace.”

In recent months, monday.com has evolved its products, strategy, and the way it serves its customers, and Zinman contended that “the organization we built for our previous chapter is not the organization that fits the new AI era.” Monday.com needs to “execute more decisively,” take on new challenges, and quickly respond to market changes, he said.

“We have never seen such a significant opportunity in software, driven by such exciting technology,” Zinman noted. He emphasized that the reduction is not to replace people with AI, nor to improve margins; the “vast majority” of savings will be reinvested into talent, products, and AI.

The restructuring will result in a “flatter organization” with fewer management layers and smaller, more autonomous teams, and monday.com also has a new go-to-market model, Zinman explained. Customers expect “deeper implementation support” as they deploy AI, and the company will work more closely with customers, increase its on-site presence, create new roles, and “adapt many existing ones.” In its SEC filing, the company said it expects to continue hiring in “key strategic areas” throughout 2026.

Workers will be expected to work better, “not harder,” Zinman noted. He pointed to several past examples where work could have been done in a few days, but instead took many months with “multiple meetings and endless friction.”

“This wasn’t people’s fault and everyone was frustrated by this,” he said. “Our new org changes ownership to allow people to make decisions and move fast.”

A spokesperson for monday.com declined to comment further on the staff reductions.

Monday’s key market advantages

Monday.com certainly isn’t struggling; the company expects 19% to 20% year-over-year growth in 2026.

“Companies in that position do not restructure because they must,” Greyhound’s Gogia noted. “They restructure because they have decided to become something else.”

Melody Brue, VP and principal analyst at Moor Insights & Strategy, pointed out that organizational redesign is important for real AI transformation, but while it can signal confidence to the market, it can still be “devastating” to humans.

While the company looks as though it’s trying to do right, that ultimately remains to be seen, she said. “There are often hidden internal bruises that can surface long after layoffs.”

Monday.com’s advantage is in its “structured substrate,” Gogia noted; its boards, permissions and typed workflows give agents something firmer to act on than just documents and chat history. The company highlights its natively built agents that can be configured by any team member, as well as connectors with Claude, Microsoft Copilot, and ChatGPT, and dedicated routes for external agents to authenticate and operate.

“For some time, the sharper enterprise question has been shifting from who has an agent to who owns the governed runtime in which an agent can safely act,” he said. “Structured work is a serious claim on that runtime.”

But parts of monday.com’s agent estate remain in staged release, and its product is ultimately “mid-transition,” Gogia pointed out; its agent builder carried a beta label as recently as March,. Also, the company’s pricing model changed in May to a hybrid model charging for seats as well as mandatory AI credits. And, while its AI-powered no-code builder monday vibe passed $1 million in annual recurring revenue within two and a half months, monday.com has not released subsequent outcomes, usage volumes, or attach rates.

Further, there’s an element of “gravity” with its competitors, he observed. Asana is reorganizing teams around agents, Atlassian is wiring agents into the developer estate, and others are simply bundling them into their offerings: Microsoft is doing so across the productivity stack, and ServiceNow across enterprise operations, each with identity and procurement built in.

“Their pull is strongest exactly where monday.com wants to grow, in the largest accounts, where control-plane depth and administrative reach decide the deal,” said Gogia.

Actions for the near-term

Going forward, buyers should focus on operating risk, not headline risk, Moor’s Brue noted. In practice, that’s continuity of service, roadmap consistency, and strength of enterprise support. Productivity should be valued as better outcomes per unit of organizational effort, not mere activity.

“It should be a measure of how much smoother, faster, and more effective the operating model becomes when AI is built into the work,” said Brue.

Gogia noted that strain surfaces first in customer service, and monday.com’s attention is being redistributed. The company’s annual report disclosed that its focus is now concentrated on the largest accounts, with support for medium-sized clients moved to an AI-first and human-supported model.

During the first month of the transition, buyers should track named account continuity and escalation times, he advised. By the first quarter, keep an eye on whether credit governance and admin controls mature on schedule, and if the roadmap beyond the AI estate keeps pace. By the half-year mark, determine whether promised implementation depth is producing outcomes or “simply more billable engagement.”

Support tiers should be enumerated in writing before renewal, and buyers should contract for “side exits,” Gogia emphasized, with overage pricing fixed in advance, the right to pause consumption, and portability for workflows and agent configuration “if the relationship sours.” Finance should also insist on monthly consumption reporting by capability. Further, integration efforts, partner dependency, and change management should be considered first-class costs of the agent era, “not as afterthoughts to a license.”

“A license was a known cost,” said Gogia. “A meter is a behavior, and behavior is harder to forecast than headcount.”

Building a presentation can take lots of time. There are design choices to figure out: the slide layouts, fonts, theme colors, and so on. You can use a template to skip this hassle, but you still have to paste your text and other content into the slides and edit it all so that the results are visually appealing.

In PowerPoint, Microsoft’s Copilot AI assistant can now automate the heavy lifting of presentation creation. It can generate a first-draft presentation in minutes, then help you edit it. You can also prompt Copilot to help you quickly understand the contents of a presentation and glean insights from it. Use the tips in this guide to save oodles of time as you create and work with presentations.

Who can use Copilot in PowerPoint

Individuals with a Microsoft 365 Personal, Family, or Premium subscription have access to Copilot from within PowerPoint and other Microsoft 365 apps. Users with a Premium plan have higher Copilot usage allowances and access to advanced AI features.

For business users, it’s more complicated. Organizations with more than 2,000 users must pay for Microsoft 365 Copilot licenses for their users in addition to their regular Microsoft 365 licenses. Users at organizations with fewer than 2,000 users can use Copilot within M365 apps even without the M365 Copilot add-on licenses, but there are limitations in usage, speed, and feature availability.

To see what kind of access you have, log in to Microsoft’s Copilot Chat web hub and look for your name in the lower left corner. If you see “M365 Copilot (Premium)” under your name, you can use Copilot in M365 apps with priority access and advanced features. “M365 Copilot (Basic)” means you can use Copilot in M365 apps with lower-priority access and limited features. If you see “Copilot Chat (Basic)” or nothing below your name, you can’t use Copilot in M365 apps.

(Copilot Chat Basic users do get some Copilot functionality, including the ability to generate presentations, via the Copilot Chat hub. See our Copilot Chat tutorial for details.)

In this article:

• Working with Copilot in PowerPoint
• Create a presentation template
• Create a presentation from a document
• Add content from a document to a slide
• Refine your slide text
• Find or create an image
• Expand your presentation with relevant slides
• Summarize a presentation
• Answer questions about a presentation
• Help you navigate a large presentation
• Generate speaker notes and/or an FAQ

Working with Copilot in PowerPoint

First, let’s quickly go over the notable settings of the Copilot sidebar.

When you have a presentation open in PowerPoint, click the Copilot icon; it may be floating at the lower-right corner of your PowerPoint window or parked at the right end of the Ribbon toolbar. The Copilot sidebar will open along the right of the page. You’ll type your prompts to Copilot inside the chat window in this pane.

Howard Wen / Foundry

Agent mode: By default, Copilot can build a new presentation or make changes to an existing one in the main PowerPoint window. This is known as “agent mode.” To change this so that Copilot can’t take direct action on a presentation (all its responses appear in the sidebar), click the Allow editing button above the chat window and change it to Chat only.

The tips in this guide require that Copilot be in agent mode, so make sure you see Allow editing above the chat window.

Choice of AI model: Behind the scenes, Copilot has access to various genAI models, including different versions of Anthropic Claude and OpenAI GPT.  By default, it decides which model to use based on your prompt. You can set it to use a particular model: click Auto at the upper right of the Copilot pane and select a model from the dropdown that opens.

Howard Wen / Foundry

The tips in this guide should work fine on the default Auto setting. But feel free to experiment switching to specific models to see which give you the best results for particular tasks.

Important: Remember that generative AI output often includes errors, so always check Copilot’s output for accuracy. (Also see our tips for reducing hallucinations in Copilot.) You’ll likely want to rewrite it in your own voice as you’re reviewing it.

1. Create a presentation template

For many people, the hardest part of creating a presentation is getting started. What types of information should be included on the slides, and in what order? Copilot can give you a leg up by creating the type of presentation you need, with placeholder data that you can later replace with your own.

Start a new presentation, open the Copilot sidebar, and type your prompt into the chat window. It’s best to provide very specific details in your prompt. The more context or details you provide, the more likely Copilot will generate a presentation template that suits your needs.

A good prompt should contain the slide count, subject, audience, and tone. Example:

• Create a 6-slide presentation for a sales meeting focusing on Q1 revenue. The audience is the sales team, so keep the tone professional and focused on the sales data.

Copilot may ask a series of follow-up questions, such as your preferred visual style and desired level of detail. Then it will generate a presentation template.

Howard Wen / Foundry

You can optionally prompt Copilot for revisions, and when you’re happy with the template, swap in your own data.

2. Create a presentation from a document

You can attach a document (such as a Word document, Excel spreadsheet, or PDF) and prompt Copilot to generate a presentation based on its contents. This works best with a structured-format document (such as a business plan, project proposal, or summary report) that contains sections with headings.

Copilot can extract the document’s text and structure to generate the slide content for the new presentation. This can especially be useful for quickly turning a long report into a visually appealing presentation.

In the Copilot pane, click the + icon at the bottom of the chat window. A list of documents that you’ve recently accessed appears. Select the one that you want Copilot to use. Alternatively, click the magnifying glass icon and inside its search box, type a few letters of the filename for the document you want. (Business users with an M365 Copilot license can select up to five files for Copilot to pull from when creating a presentation.)

Howard Wen / Foundry

Then in the chat window, you can enter a prompt that’s as simple as “Create a presentation,” although as always, providing more details and context is better. This is especially important for corporate users who reference multiple source files. It’s useful to tell Copilot what data to pull from each document.

Answer any follow-up questions that Copilot asks, and it will then generate the presentation.

Howard Wen / Foundry

Note: Your marketing department may have created one or more branded company templates for Copilot to work from. If that’s the case at your organization, simply open the appropriate company template as your first step. Then you can upload docs and type a prompt as described above. Copilot will create a presentation using the branded template.

3. Add content from a document to a slide

Manually copying text or other content from a document and pasting it into a new slide is a chore. Instead, you can prompt Copilot to extract information directly from a Word document, Excel spreadsheet, or PDF to create new slides.

In the Copilot pane, attach the document using the same steps described in tip 2, then tell Copilot to create a slide from the document. As always, it helps to provide details such as the new slide’s focus or what data to include:

• Add a slide based on the attached document.
• Use the attached file to add a slide about the project budget that focuses on Q1 projections.
• Summarize only the financial section of the attached document as a slide.

Howard Wen / Foundry

4. Refine your slide text

A presentation should be visual and display only the core message. Conciseness and proper writing tone are essential for your slides, so that they don’t lose the attention of your audience.

You can prompt Copilot to refine text on an individual slide in various ways, such as rewriting it in a more professional tone or making it more concise. Highlight the text inside a text box on the slide. On the toolbar that appears over the highlighted text, click Edit with Copilot.

On the menu that opens, you can select a preset prompt to refine the text, such as Condense or Make professional. Or, at the top of this menu, you can type a prompt to rewrite the highlighted text.

Howard Wen / Foundry

Note that this feature affects all the text inside the text box. To rewrite only a portion of text inside a text box, you must split that portion out into a separate text box.

Alternatively, you can prompt Copilot to analyze your entire presentation and tighten up the wording throughout all of its slides. For example:

• Make these slides more visual and use less text.

5. Find or create an image

If you have Copilot generate a presentation from an existing Word document that contains images, it will incorporate those images into the presentation. If there are no images in the source document, you can ask Copilot to find or create one and add it to a slide.

To add a stock image or an image from your organization’s brand library, tell Copilot what you’re looking for:

• Add a stock photo of young adults in a cafe drinking boba tea.
• Add a photo from our asset library of young adults in a cafe drinking boba tea.

To have Copilot create an image using Microsoft’s Designer image generation tool, describe your desired image. As always, specificity is helpful:

• Create a photorealistic image of a diverse group of 5 or 6 fashionable young adults sitting in a cafe drinking boba tea. They’re smiling or laughing, and some are looking at their phones.

Howard Wen / Foundry

Just as you need to review any text output from Copilot, take a close look at generated images to be sure nothing looks off.

Also note that Copilot image generation isn’t always reliable in PowerPoint. For some time during our testing for this story, Copilot said it couldn’t create an image because “the image generation service is returning a server error on every attempt.” After about a day and a half, the service began working again.

6. Expand your presentation with relevant slides

As you’re building your presentation, you may find that it’s become text heavy. Or perhaps it could use more visually oriented slides to break things up and make its progression flow better. Copilot can generate and insert new slides that are based on the content of the slides already in the presentation.

In the Copilot pane, specify exactly where you want the new slide to go. This helps Copilot to analyze the content of the slides before and after where you want the new slide. Then it can generate a slide to bridge between the two slides. Examples:

• Add a slide after slide 3 about our competitive advantages.
• Add a slide after slide 11 that transitions to slide 12.

Howard Wen / Foundry

7. Summarize a presentation

Maybe you need a quick refresh of your presentation before an important meeting. Or maybe a co-worker has sent you a presentation that’s packed with lots of slides. You can prompt Copilot to generate a summary of the presentation’s overall messaging.

In the Copilot pane, just type “summarize this presentation.” You can also have Copilot flag key slides that contain important information: “show me key slides.”

Howard Wen / Foundry

8. Answer questions about a presentation

As you’re reviewing a presentation, especially one that you didn’t create and are not familiar with, you can get Copilot to pull key data points from its slides.

In the Copilot pane, type specific informational questions. Examples:

• What are the action items in this deck?
• What is the proposed budget mentioned here?

If Copilot can’t find the exact answer to the question you ask, it will provide related information from the presentation.

Howard Wen / Foundry

This method can also help you validate that your presentation includes everything you want it to. If you ask Copilot about the action items in a presentation and it can’t find any, you know you need to add them. (Copilot will likely offer to generate them for you based on the rest of the slides.)

You can even take this tactic a step further and ask Copilot if the presentation is missing any important data, if any slides are weak or confusing, if there are any awkward transitions, if there are key points that should be better emphasized, and so on.

9. Help you navigate a large presentation

In the business world, presentations with dozens of slides are not uncommon, such as for financial reports or project documentation. Trying to find a specific slide or multiple slides can be tough. Copilot can help you navigate such a presentation.

In the Copilot pane, prompt Copilot to find slides based on specific topics. Example:

• Show me the slides about the project timeline.

Copilot will analyze the presentation and reply with a list of links to the relevant slides. Click one of these to jump directly to that slide.

Howard Wen / Foundry

10. Generate speaker notes and/or an FAQ

Here’s a great timesaver when you’re preparing to show your presentation to an audience: Copilot can automatically generate suggested speaker notes for you, based on the content of your slides. Example prompt:

• Write speaker notes for every slide with one talking point per slide.

Howard Wen / Foundry

In a related feature, Copilot can create a frequently asked questions list (FAQ) for you to consult in your speaker notes or to present as a slide:

• Write an FAQ for these slides.

Copilot will ask where you want the questions and answers added — as a new slide at the end, integrated into the speaker notes of relevant slides, or somewhere else that you designate. Make a selection, and Copilot will generate the FAQ based on the content of your presentation.

Howard Wen / Foundry

Related reading:

• 11 cool things Copilot can do in Excel
• 9 ways Copilot can turbocharge OneNote
• PowerPoint for Microsoft 365 cheat sheet
• Copilot Chat: Your hub for document creation and analysis
• How to curb hallucinations in Copilot (and other genAI tools)
• Microsoft Copilot can boost your writing in Word, Outlook, and OneNote — here’s how
• More Microsoft tips and tutorials

A client called me last week with a problem I have been hearing about more often lately. They had made significant reserved instance commitments with a major cloud provider, overbuying for what they thought would be heavy AI training workloads. Now they were sitting on thousands of dollars in idle capacity every month. Their plan was simple: resell it to someone else. Except they couldn’t.

I have been doing cloud consulting for a long time, and this situation once had a straightforward solution. You went to the marketplace, listed your unused reservations, and found a buyer. The process was a bit clunky, but it worked. These days, the answer is far more complicated, and my client learned this the hard way.

AI has made this problem increasingly common. Companies initially committed to compute capacity based on ambitious training plans. Prototype projects were expected to scale, and inference workloads were projected to grow substantially. Then reality hit. Some projects did not materialize. Some models trained faster than expected. Some inference patterns were lighter than anticipated.

Many organizations now hold reserved capacity they can’t use, discard, or share without a complex, increasingly restricted process. This reality is something every company with significant cloud spend needs to clearly understand.

The history of cloud resale

There was once a functioning resale market for cloud reserved instances. AWS, for example, maintained a Reserved Instances Marketplace where companies that had purchased reserved capacity could sell those reservations to other AWS customers. This was a legitimate, AWS-sanctioned process. Companies would register as sellers, list their unused reservations with pricing and terms, and if a buyer appeared, the marketplace would facilitate the transaction.

The resale market was useful for companies that had overestimated their needs or whose business changes reduced their cloud consumption. Instead of simply absorbing the cost of unused commitments, they could recoup some of that investment by selling to other organizations with unmet demand. It created a secondary market that added liquidity to what was otherwise a rigid financial arrangement.

My client had some experience with this resale market a few years ago and assumed they could use it again. They were unpleasantly surprised to learn that the rules had changed.

AWS changes the rules

In January 2024, AWS implemented a significant policy change that effectively shut down the resale of EC2 Reserved Instances on its platform. AWS stopped allowing companies to resell their unused reserved capacity through the Reserved Instance Marketplace or any other official channel. If you have a reserved instance commitment with AWS, you are essentially stuck with it unless you can use it yourself or modify your reservation.

This change had a real impact on companies that had relied on resale as part of their cloud financial management strategy. It reduced flexibility and increased the risk of long-term reserved commitments. When I explained this AWS policy change to my client’s representatives, I could hear the frustration in their voices. They had made their commitment in good faith, carefully modeled their expected AI workloads, and now faced the reality that there was no easy exit.

The reasoning behind this change is not entirely clear, but AWS likely viewed capacity resales as something that complicated their billing and commitment models without providing enough benefit to the overall ecosystem. Regardless of the company’s reasons, the primary resale path for the largest cloud provider has been effectively closed.

What options still exist?

What can companies do now when they find themselves with reserved capacity they no longer need? The first possibility is to work directly with the cloud provider to modify or exchange the reservation if it is convertible. Some reservation types allow modifications, such as changing the instance type, region, or tenancy. This will not eliminate the commitment, but it may help companies better align their reservations with actual workload needs.

The second option is to use third-party brokers and marketplaces that operate independently of the cloud providers. Although AWS has shut down its official resale channel, brokers and marketplaces still facilitate resale arrangements for other cloud providers and for some AWS scenarios. These arrangements can be more complex and carry more risk, but they remain a possibility for companies determined to move unused capacity.

The third alternative is to optimize usage. Companies can invest in better utilization monitoring, workload placement, and automation to ensure that reserved capacity is used as efficiently as possible. This does not recover the money already spent, but it reduces future waste.

My client explored all three alternatives and found that each had significant limitations. Modifications were possible, but only within a narrow range. Third-party brokers were interested, but the process was opaque and uncertain. Optimization helped, but it could not eliminate the fundamental overcommitment they had already made.

The broader implications

Cloud commitments are more rigid than many enterprises initially realize because they lack a liquid market and because providers control modifications, transfers, or cancellations. Right now, I see this pattern most often in the AI space. Companies commit to massive amounts of compute for training and inference based on projections that rarely reflect the actual workloads. Then they are surprised to find themselves locked into payments. The AI boom has led to significant overcommitment because enterprises remain unaware that the resale mechanisms that once existed have been largely shut down.

This is why cloud financial management has become such an important discipline. Companies need to be far more thoughtful about how they commit to cloud resources, how they model their future consumption, and how they build flexibility into their cloud strategies. The days of assuming you can always resell your way out of an overcommitment are effectively over, at least with AWS.

For Azure and Google Cloud, the resale landscape is slightly different, but the same general principles apply. These providers have their own capacity transfer policies and, like AWS, those policies can change at any time. Companies should understand their options before making large, committed purchases and build contingency plans in case their actual usage diverges from their projections—or if resale policies change.

The bottom line is that reselling unused reserved cloud instances is far more complicated than it sounds. The market is not as open as it once was, the options are limited, and the providers themselves hold most of the cards. My client got burned, and I doubt they will be the only one. Companies that want to optimize their cloud spending should focus on accurate forecasting, thoughtful commitment sizing, and ongoing optimization rather than relying on resale as a safety valve. That approach worked at one point, but those days are largely gone.

Microsoft and Mistral are betting that the future of enterprise AI is in sovereign infrastructure and model choice, rather than with one locked-in system. 

The companies have announced a “significant expansion” of their strategic partnership, which includes a multibillion dollar commitment from Microsoft. Mistral will add to its GPU infrastructure in Europe and extend access to its frontier multilingual models, while Microsoft will expand its sovereign cloud capabilities. The companies will also align on a joint go-to-market plan and will pursue enterprise opportunities together across Europe and globally, as well as funding proofs of concept (PoCs), offering Azure credits, and leading workshops to drive AI innovation with customers.

The partnership between the tech giant and the three-year-old French startup might seem an odd combination at first glance, analysts note, as both develop enterprise AI models and offer access as-a-service. But it reflects changing AI market dynamics.

“It’s possible to be both a competitor and a partner at the same time,” noted technology analyst Carmi Levy. Large cloud providers are becoming AI marketplaces in their own right, he pointed out, and are drifting away from exclusively promoting their own models. Building Mistral support into their infrastructure avoids platform lock-in and removes a “key objection for customers looking for options.”

“As much as Microsoft would want everybody standardizing on Copilot and Phi, it recognizes the simple fact that customers increasingly want to choose their own models,” said Levy.

Expands model access, sovereign cloud capabilities

As part of the agreement, Mistral will expand its Europe-based capacity with thousands of Nvidia Vera Rubin GPUs.

Mistral CEO and co-founder Arthur Mensch described a “slight gap” in compute capacity in Europe, noting that this expansion will provide more compute capability and support Microsoft’s cloud and AI services, providing a “shared platform for training, inference and large-scale deployment.” The companies call it a critical step to allow Microsoft customers to benefit from Mistral’s “scientific and compute innovations.”

In addition, Mistral Medium 3.5 and OCR 4 models are now available in Microsoft Foundry, and Mistral Medium 3.5 can be used in Microsoft Copilot Studio.

The partnership also extends Microsoft’s sovereign cloud infrastructure as well as combining Mistral’s frontier models with Microsoft’s security, compliance, and cloud-to-edge platform. This gives enterprises, particularly those in regulated markets, the ability to deploy AI where they see fit, while maintaining control over their data and workloads, according to the companies.

Further, customers will be able to build AI using the same models, tools, APIs, and workflows they’re used to, across Microsoft Foundry, Foundry Local, and Azure Local, and opt for fully Azure-hosted cloud environments; cloud-connected, controlled Azure Local environments that only use cloud-based Azure when necessary; and fully-disconnected environments that can operate independently for more sensitive scenarios.

“Europe should have access to the world’s most capable AI without compromising control over their data, operations or digital future,” said Brad Smith, vice chair and president, Microsoft, noting that with this partnership, the company is honoring its European digital commitments and giving customers a foundation for AI so they can “operate on their own terms.” Customers with “heightened sovereignty needs” will be able to exercise more control with “resilience and assurance” and continued access to Mistral’s open-weight models.

Enterprise credibility

Gartner distinguished VP analyst Arun Chandrasekaran noted that there’s no doubt that this agreement strengthens Microsoft’s sovereignty messaging and its position in regulated industries, and the tech giant benefits by expanding its AI portfolio with a “credible European frontier model provider”

He pointed to key differences from the initial partnership struck by the two companies in 2024; whereas originally Microsoft was hosting Mistral’s models, it is now consuming capacity built by Mistral in Europe.

Ultimately, the deal emphasizes European data centers, customer-controlled deployments, Azure Local, and fully-disconnected environments, addressing many of the concerns that surrounded the original Azure cloud only relationship, Chandrasekaran explained.

For Mistral, the partnership provides “enterprise credibility, and repeatable infrastructure revenue” that can fund continued AI platform development, he said. The combination of Microsoft’s enterprise AI platform with Mistral’s models and European AI infrastructure will give joint customers more deployment flexibility and expand options around data residency, sovereign AI deployments, and disconnected/on-premises environments.

“It also gives customers more model choice, reducing dependence on a single AI provider,” said Chandrasekaran.

A complementary partnership

Mistral continues to innovate with its frontier AI models and its chat and coding agent, Vibe (formerly Le Chat), yet it doesn’t attract as much attention as Claude or ChatGPT.

One of the company’s key differentiators is its targeted business model. Levy pointed out that not every workload requires “full-flight GPT.” For customers trying to rein in costs and limit exposure with on-premises deployments, Mistral’s “more focused capabilities can represent a cost-effective alternative.”

Chandrasekaran pointed to Mistral’s combination of high-performance open-weight models, strong multilingual capabilities, and a “focus on efficient inference that lowers deployment costs.”

Unlike many frontier AI companies, it offers customers greater flexibility to self-host and customize models; this makes it particularly attractive for enterprises and governments with sovereignty or regulatory requirements, he said. Its European roots also position it as the leading alternative for organizations seeking cutting-edge AI outside the US and Chinese ecosystems.

Bill Wong, research fellow at Info-Tech Research Group, also pointed to Mistral’s high-quality models and “adeptness as a sovereign AI leader.” There is growing demand for AI companies that comply with regional laws and data residency, and Mistral is established as “one of the most prominent European players.”

“Such a strategic position makes it a great partner for Microsoft to further expand its AI offerings beyond just being a single-model provider,” he said. Customers get freedom of choice while complying with data sovereignty and regulatory limitations without having to execute a separate AI deployment, while Mistral, for its part, can go beyond Europe and gain more visibility with international businesses.

Mistral brings both “technological and political advantages,” Levy noted. The startup’s European roots give Microsoft more credibility “at a fraught time for geopolitical relationships.” Customers in Europe and beyond are concerned about US exposure, and Mistral can provide a safer choice.

Meanwhile, Microsoft can deploy European-developed AI models running on European infrastructure, thus maximizing regulatory compliance while offering next-level enterprise marketing scale that Mistral “simply couldn’t achieve on its own,” said Levy. Mistral-based workloads deployed on Azure will also benefit from Microsoft’s “comprehensive security certifications, governance frameworks, and monitoring.”

Bottom line: Both companies can maximize their unique roadmaps through the partnership, he said. “As the rules of the AI economy continue to evolve, expect more eyebrow-raising deals like this to be signed.”

The head of the US government’s AI testing institute, Chris Fall, has resigned about three months after taking charge of the Center for AI Standards and Innovation (CAISI), the federal organization responsible for evaluating advanced artificial intelligence models for safety and security.

Current National Institute of Standards and Technology NIST Director Arvind Raman will serve as acting CAISI Director following Fall’s departure while continuing to oversee the Commerce Department office responsible for the institute, the Daily Signal reported, citing two people familiar with the matter.

A Commerce Department spokesperson who spoke to the publication did not disclose a reason for the resignation.

Fall assumed leadership of CAISI in April after the Trump administration reorganized the former US AI Safety Institute under NIST. The institute develops methodologies for evaluating frontier AI models and works with AI developers on voluntary technical assessments covering areas such as cybersecurity, model misuse, reliability and other risks associated with increasingly capable AI systems.

The leadership change comes as governments and AI companies continue developing technical approaches for evaluating frontier AI models while enterprises expand deployments of generative AI and agentic AI across business operations.

In recent months, the Commerce Department has taken a more active role in AI policy involving advanced models, placing greater attention on how the federal government evaluates technologies with potential national security implications.

Continuity matters more than personalities

CAISI works with AI developers such as Anthropic, Google’s DeepMind and OpenAI on voluntary evaluations of frontier AI models and develops methodologies for testing model capabilities and risks. The institute does not regulate AI developers or certify commercial AI systems.

For enterprises, those evaluations are one source of technical information alongside vendors’ own testing, third-party security assessments and internal AI governance programs.

Sanchit Vir Gogia, chief analyst at Greyhound Research, said enterprises should focus less on the individual leading the institute and more on whether its technical work continues with the same level of consistency and transparency.

“Leadership churn at CAISI weakens the signal long before it weakens the science,” Gogia said. “The testing has not stopped. Its authority simply does not travel as cleanly once the leadership does not.”

According to Gogia, the more important question for enterprises is not whether the institute’s evaluation work will continue but whether the processes supporting those evaluations remain stable.

“The instinct is to ask whether the pipeline is breaking,” he said. “The more useful question is where the pipeline now sits.”

Enterprises still carry the burden of AI governance

Gogia said organizations should continue treating government-led AI evaluations as one input into their governance processes rather than as evidence that a model is inherently safe for enterprise deployment.

“A government evaluation was always a signal, never a certificate,” he said. “A signal loses value the moment its issuer becomes unpredictable.”

He said enterprises should instead monitor whether CAISI maintains consistent evaluation methodologies, continues publishing technical findings and preserves continuity within its research teams under interim leadership.

“The name on the door is not the signal. The behaviour underneath it is,” Gogia said.

Gogia also cautioned against linking Fall’s resignation to recent Commerce Department actions involving AI policy or export controls, noting that there is no public evidence connecting the two.

“CAISI evaluates; it does not enforce export controls, because it holds no such power,” he said. “This is not a testing body reaching for enforcement. It is enforcement reaching past the testing body.”

With Raman assuming the role on an interim basis, the next significant milestone for enterprises will be the appointment of a permanent director, and whether the institute’s evaluation programs continue without disruption, the analyst said.

Gogia said the successor’s mandate may prove more important than the individual selected.

“A CAISI result is not a safe harbour,” he said. “It informs an obligation; it does not discharge one.” NIST did not immediately respond to a request for comment.

What if the next breakthrough in Enterprise AI doesn’t come from computer science alone?

What if it comes from applying principles that physicists have understood for more than a century?

According to Gartner, rising token-driven AI spend is straining budgets and challenging cost justification. As organizations race to deploy generative AI and agentic systems, token consumption dominates nearly every executive discussion: How many tokens did we use? How much did inference cost? Can we reduce our AI bill?

These are important operational questions. But they are not the strategic questions.

I believe the economics of enterprise AI can be viewed through the lens of three well-established principles from thermodynamics: the conservation of energy, entropy, and exergy.

While these principles describe physical systems — not AI —they offer a useful way to think about how organizations should measure AI success.

Principle 1: Value is created through transformation

The 1^st Law of Thermodynamics tells us that energy cannot be created or destroyed. It can only be transformed.

Enterprise AI presents a similar management lesson: Tokens are not valuable because they are consumed; they become valuable only when they are transformed into business outcomes: A faster loan application decision. A better customer experience. Faster and more accurate software. Reduced fraud. Higher employee productivity. A new product. A strategic insight.

The executive question therefore is not, “How many tokens did we consume?” It is: “How much business value did those tokens create?”

This leads to a new executive metric: return on tokens (ROT).

Just as organizations measure return on investment, they should begin measuring the business value generated for every million AI tokens consumed.

The organizations that win will not necessarily consume fewer tokens. They will generate more value from every token they use.

Principle 2: Every transformation creates waste

The 2nd Law of Thermodynamics teaches us that every energy transformation introduces inefficiencies.

Some energy inevitably becomes less useful for doing work.

The same pattern appears in enterprise AI: Not every token contributes equally to business outcomes.

Some are spent on:

• Repeated prompts
• Oversized context windows
• Redundant reasoning
• Hallucinations requiring correction
• Multiple agents performing the same work
• Expensive models solving simple problems

Those tokens are not “lost.” They simply produce very little business value.

I think of this as token entropy. Every enterprise deploying AI will experience it. The goal is not to eliminate token entropy completely — that would be unrealistic. The goal is to continuously identify it, measure it and reduce it. Because every unnecessary token represents an opportunity to improve both cost and business performance.

Principle 3: Useful work matters more than energy consumed

Thermodynamics introduces another important idea: Exergy.

Unlike energy, exergy measures how much energy can actually be converted into useful work. Two systems may consume the same amount of energy while producing dramatically different results.

The same is true for enterprise AI.

Imagine two companies each consuming one billion tokens. One produces meeting summaries. The other transforms claims operations, accelerates software delivery, detects fraud, improves customer retention, and creates new revenue opportunities. Both consumed the same number of tokens. Only one extracted significantly more business value.

Borrowing this concept as a management analogy, I call this token exergy.

Token exergy represents an organization’s ability to convert AI intelligence into meaningful business outcomes:

• High token exergy means AI is solving important business problems.
• Low token exergy means AI is generating activity without creating proportional enterprise value.

The distinction matters, because activity is not the same as impact.

A new responsibility for CIOs

For years, CIOs have monitored infrastructure: Cloud costs, storage, network utilization, GPU consumption.

These metrics remain important, but they tell only part of the story.

Token usage needs to be measured, planned, optimized and governed with the same discipline as any other cloud resource. This means that the next generation of CIO dashboards should answer different questions:

• What is our return on tokens?
• Where is token entropy reducing our effectiveness?
• How much token exergy are we generating?
• Which AI initiatives produce the greatest business value?
• Which use cases create the strongest competitive advantage?

These are no longer technology metrics. They are business metrics.

The next generation of CIOs will not simply deploy AI. They will manage an economy of intelligence.

Their role will resemble that of a portfolio manager — allocating AI capacity where it creates the greatest enterprise value, reducing waste and continuously improving the productivity of every autonomous workflow.

That responsibility cannot be fulfilled by dashboards alone.

It requires an intelligent layer capable of observing, learning and optimizing the entire AI  ecosystem. Three-layer enterprise agentic architecture Will enable this.

The next competitive advantage

Every major technology revolution eventually shifts from measuring inputs to measuring outcomes:

• Factories stopped measuring coal consumption and began measuring productivity.
• Cloud computing evolved beyond server utilization to business agility.
• Digital businesses measured customer acquisition costs and lifetime value.

Enterprise AI is approaching the same inflection point. Organizations that focus only on token costs will optimize for efficiency. Organizations that measure return on tokens, minimize token entropy and maximize token exergy will optimize for business transformation.

That is a fundamentally different objective. And I believe it will separate AI leaders from AI followers.

Because in the end, the future of enterprise AI will not be determined by how many tokens an organization consumes. It will be determined by how effectively those tokens are transformed into lasting business value. The AI adoption spending spree is over. Time to focus on value.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Technology executives from across the New York metropolitan area gathered July 16 at Convene, One Liberty Plaza, for CIO 100 Leadership Live New York, a full day of roundtables and panel discussions on enterprise AI investment, governance, and organizational change.

Several key areas of consensus emerged throughout this highly interactive event. Infrastructure fragmentation continues to block the path to securing returns on AI investments prompting leaders to understand rising cloud spend attributed to large language model utilization. This has caused a growing number of organizations to refocus on on-premises and hybrid options in C-suite and board-level capital planning conversations. Speakers, along with comments from the audience, described a shift from project thinking to product thinking, with smaller multidisciplinary teams moving faster than legacy structures.

Several participants repeatedly warned that automating broken processes just amplifies dysfunction. Governance and measurement remain unresolved, with usage metrics still getting mistaken for business value. One of the panels explored how CIOs may benefit from applying venture capital-style scrutiny to enterprise bets, weighing team execution as heavily as the technology itself. The throughline was a redefinition of the CIO role, from technology executor to business strategist fluent in revenue, board engagement, and transformation ownership.

Morning roundtable tackles AI infrastructure

The day opened with an invitation-only executive breakfast roundtable, “Beyond the Pilot, Building the Infrastructure for Real AI Returns,” co-hosted by Unisys and Dell Technologies. Over a dozen executives representing major public and private sector organizations across the New York metropolitan area joined Steve Hollander, senior director of Americas global alliances at Dell Technologies, and Matt Marshall, CIO at Unisys for a workshop-style discussion.

The session explored the strategic, operational, financial, and technological issues that must be mastered to optimize infrastructure decisions and separate organizations that are experimenting with AI from those competing on it. Discussion questions probed how CIOs measure whether AI investment is translating into business results, how they can break the cycle of fragmented and siloed AI deployments, how boards are beginning to scrutinize seven-figure token spend and whether on-premises or hybrid infrastructure can rein in costs.

The take-home point: the organizations pulling ahead are the ones that stopped treating AI as four separate problems, strategic, operational, financial, technological, owned by four separate functions, and started running it as one coordinated decision. Fragmentation is the actual cost center here, not the token spend itself. A CIO who solves the infrastructure question in isolation from the governance question, or the cost question in isolation from the talent question, ends up optimizing one silo while the other three keep bleeding value. Competing on AI, instead of just experimenting with it, means the finance, operations, technology and business sides are reasoning from the same picture of what’s being built and why, so the tradeoffs get made once, together, instead of getting re-litigated at every handoff.

Forum sessions open with a mandate for growth

Following breakfast, the main forum program began with “The New CIO Mandate, Delivering Growth, Not Just Technology.” In a moderated conversation, Laksh Nathan, chief information officer at Paramount Skydance, drew on his experience with mergers, enterprise transformation and AI-enabled development to describe a shift from project and application management toward a product-centric operating model. Nathan addressed how smaller, multidisciplinary teams are changing expectations on both the business and technology sides of the enterprise, and what mindset changes CIOs must lead to turn AI into an engine of growth rather than a cost center.

PwC followed with a session on “Designing the Intelligent Enterprise, From AI Investment to Evolving Operations.” Darren O’Meara, principal and chief technology officer for managed services, and Meghna Shah, principal for engineering and AI, examined why fragmented outcomes persist even after heavy investment in technology and transformation.

The intelligent enterprise, they posited, is less about working toward achieving specific technology outcomes and more about creating operating models that integrate strategy, technology, operations, and governance into one system. This, they explained, requires linking AI, data, and decisions across the business and will leave an indelible mark on how decision rights are redesigned, funding models are developed, and accountability is enforced to accommodate the speed of the agentic economy.

Talent, tradeoffs, and the cost of getting it wrong

The session “Return on Transformation: Time, Talent, and Tradeoffs” — with Prashant Hinge, chief information and transformation officer at MSIG USA; Joseph Gimigliano, chief technology officer at Northwell Health; and Eduard de Vries Sands, AI executive advisor at PatientPoint — examined why transformation initiatives so often lose their way.

The main culprit, even today in 2026, continues to revolve around a persistent instinct for technology implementations to become the objective rather than the means to a measurable business outcome. The panelists made the case for doing the incredibly difficult work of re-engineering (if not entirely re-imagining) existing processes before automating them and then placing smaller bets inside that bigger vision.

Ricky Thakrar, head of sales and account management at Zoho, took the stage to present “Smaller, Smarter, Safer, The Enterprise AI Architecture Most Leaders Get Backwards,” arguing that constrained, context-rich architectures consistently outperform expensive models bolted onto fragmented systems.

A round of Hot Topic Discussion Groups and a networking lunch followed, including the Next CIO Luncheon featuring Robert Half Regional Director Jason Deneu.

Afternoon sessions turn to security, scale, and investment signals

CSO and CIO Contributor Joan Goodchild moderated “Securing Trust in the Agentic Economy,” a discussion with Marlowe Cochran, CISO at the New York State Education Department, and Gee Rittenhouse, vice president of security services at AWS, on how organizations are balancing speed, innovation and security as AI agents move from experimentation into productization at scale.

Rittenhouse framed agentic risk as closer to human risk than traditional software risk, describing how an independent agent acting in a non-deterministic way really does look like a potential insider threat, pushing CISOs toward behavioral monitoring over static workload protection. He tied this to a structural shift in defense, noting it’s hard to do agentic security if you’re not observing it, putting observability at the center of agentic risk management.

Cochran concurred, adding that many of the key tools that are needed to move into the agentic economy already exist, but must be implemented more aggressively, comprehensively and even more creatively. CISOs don’t need to invent an entirely new security discipline for the agentic era so much as extend identity management, access control and monitoring frameworks they already run to cover a new class of non-human actor — agents.

A session on “AI, From Experimentation to Enterprise Impact” brought together Meagan Gentry, national AI practice manager and distinguished technologist at Insight and Yuri Gubin, chief technology officer at DataArt, for a candid look at why pilots stall before reaching scaled production and what operating capabilities, governance, cost visibility, continuous education, must be in place to sustain AI once a proof of concept works.

During the session’s Q&A segment, a discussion emerged around how proof-of-concept success can result in a false signal, raising questions about whether pilots should be considered successful before the intended outcomes have had time to materialize, and drawing a distinction between measuring usage and adoption versus measuring business value.

The panelists explored how CIOs can identify the small number of transformational AI opportunities worth pursuing rather than managing hundreds of incremental use cases, and even challenged whether prioritization is the CIO’s job at all. The discussion closed on a sequencing question with real strategic weight, whether AI-first strategies are putting the technology ahead of the business problem CIOs are trying to solve, and what role CIOs should play with boards in defining the outcomes AI is expected to support.

A shift in perspectives

The “Think Like a VC, Investment Shifts Towards Focused AI Applications” session featured three venture investors, Aaron Darr, partner at Lead Edge; Isabelle Phelps, partner at Lerer Hippeau; and Marshall Porter, general partner at AlleyCorp. The panel explored how investors evaluate risk and talent in a market where products and competitive positions can shift within months, and what separates a focused AI application with durable enterprise value from an AI wrapper built to chase a trend.

The panel challenged the enterprise instinct to seek certainty in a market moving this fast, questioning whether CIOs should stop looking for technologies that will future-proof the enterprise and instead grow more comfortable continuously reassessing their bets. Investors framed this as a deliberate departure from the traditional low-tolerance-for-failure posture that has long governed enterprise technology purchasing, arguing that the search for certainty has itself become a risk in a market where products and business models can shift within months. The discussion pressed CIOs to weigh how they can adopt a more dynamic investment mindset without compromising the enterprise security, governance and accountability their organizations still depend on.

A Lightning Insights followed, featuring five-minute briefings from Insight, Platform9 and Console, followed by Keystone Senior Principal Ellora Sarkar’s talk on why most enterprise AI investment fails to produce measurable value and what separates the small share of firms capturing real return on investment from the majority still stuck in pilots.

Closing the day

The forum closed with “What’s Next for the CIO, Preparing for the Next 12 to 24 Months,” a fireside conversation with Leif Maiorini, CIO for corporate services at Omnicom. Maiorini discussed why business processes need to be redesigned for agentic speed rather than automated around existing human workflows, how organizational structures may shift as autonomous agents reshape visibility and decision support, and where sustainable differentiation will come from once AI capability itself becomes widely accessible.

Maiorini encouraged the industry to clearly distinguish between nondifferentiated services that should be made as efficient as possible and the differentiated capabilities that actually influence why customers choose to do business with an organization, once the major efficiency gains from optimization and AI have been captured.

He was candid about the governance gap agentic systems open up, noting that agents lack the professional reputation, personal accountability and inherent constraints that shape human behavior, which creates new risk when autonomous decisions occur at machine speed. That combination, reinvesting efficiency gains into genuine differentiation while building governance models suited to non-human decision-makers, framed his closing case for why human creativity and judgment remain the enterprise’s most durable asset even as the underlying technology becomes commoditized.

Join the CIO 100 Awards & Conference Aug 17–19, 2026 at Omni PGA Frisco Resort & Spa, Frisco, TX — where top IT leaders celebrate innovation and connect.  Learn more to attend or partner.

Last year Anthropic gave away one of the most successful things it has ever built. And, no, I’m not talking about Claude. I’m referring to MCP, the now ubiquitous Model Context Protocol, which Anthropic donated to the Linux Foundation’s new Agentic AI Foundation⁠. At the time, MCP was pulling nearly 100 million monthly SDK downloads across more than 10,000 active servers⁠, prompting the question as to why any company would give up such a popular piece of technology.

Google did much the same months earlier, handing its Agent2Agent (A2A) protocol to the Linux Foundation⁠ with AWS, Cisco, Microsoft, Salesforce, SAP, and ServiceNow signing on as founding members. OpenAI, not to be outdone, supports remote MCP servers in its Responses API⁠, sits on the MCP steering committee, and contributed AGENTS.md to that same foundation alongside its fiercest rival’s protocol.

It’s like Game of Thrones, except the principal AI powers seek regime change through seeming acts of beneficence rather than violence. For those who have been around for a while, it’s also entirely predictable, following a similar script we’ve seen in the cloud, on-premises servers, and more. Platform companies don’t give away technologies they’ve stopped caring about. They give away technologies they no longer need to own because competitive advantage has shifted to new ground.

What does this mean for AI?

Gravity has shifted before

Google has long been an exceptionally active contributor to open source. As I wrote in 2017, Google wasn’t open sourcing TensorFlow and Kubernetes out of generosity but rather turning these open source assets into on-ramps for Google Cloud. Google was playing catch-up to AWS and Microsoft. As then Google product manager Martin Buhr said, the company hoped to “create a gravity well in the market for container-based apps [so] that a significant percentage of them will end up with us.”

In other words, platform companies routinely commoditize one layer of the stack so they can compete somewhere where they hold a stronger hand.

GitHub may be an even better example. Git is open. Anyone can host a Git repository and, once upon a time, different companies did just that. Yet GitHub became the default place software development happens for millions of developers. Nobody pays for Git, but lots of people pay for GitHub. We’re seeing this same phenomenon play out in AI.

Trading contributions for control

Anthropic and OpenAI have both pretended at being all for humanity’s good, but that’s not a good explanation for why they’re racing to give away things like MCP. The deeper reason is that the model itself has turned out to be a poor place to build a lasting moat, and they’re trying to figure out what’s next. As I pointed out recently, the frontier model leaderboards change almost weekly. As such, enterprises shouldn’t build their AI strategy around the assumption that any one vendor will remain permanently ahead on model quality. Instead, as I suggested, AI may be sexy, but the “dull reality” is connecting those models to enterprise data, workflows, etc.

The AI companies understand this better than anyone. Sure, they’ll continue spending billions training ever more capable models because frontier models attract developers, generate headlines, and open enterprise doors. But they’re also quietly acknowledging that benchmark leadership alone doesn’t create a durable platform.

Developers return to the places where their tools, workflows, teammates, and accumulated work already live. Enterprises double down on the systems where their data, permissions, governance, and business processes are already connected. Every new integration makes that destination a little harder to leave, and every new workflow increases its pull. That’s what MCP, A2A, etc., are all about: increasing gravity around the models.

Every major AI company wants to become the place where AI-assisted work naturally happens, and they’re now amassing armies of forward deployed engineers and trying other means to get legacy infrastructure to tie back to their frontier models. The enterprise incumbents want the same thing, but from the opposite direction. They don’t need to own the frontier; instead they need to connect the frontier to the systems that already safely run the business.

That’s why I’m skeptical whenever someone confidently predicts that AI will sweep away enterprise software. I’ve seen this movie before. Developers absolutely live on the frontier, but enterprises don’t. Enterprises create value by connecting new capabilities to decades of accumulated applications, data, policies, and business processes. The newest model matters, and so does the newest agent framework. But neither creates much business value until it’s connected to customer records, financial systems, supply chains, HR data, and everything else enterprises already depend on.

That’s where incumbents still possess enormous gravitational pull. My employer, Oracle, certainly believes so, just as Microsoft, SAP, Salesforce, and ServiceNow do. (Disclosure: I run developer relations at Oracle, which participates in the Agentic AI Foundation.) Ironically, open protocols strengthen that position rather than weaken it. If every model can speak MCP and every agent can interoperate through common standards, enterprises gain the freedom to adopt whichever frontier technology looks best without rebuilding every integration. The protocol becomes interchangeable.

Open standards don’t stop gravity

None of this diminishes the importance of open standards. MCP succeeded because it solves a genuine problem. Developers shouldn’t have to build a custom connector every time an AI application needs access to a database or other business system. Neutral governance also matters because nobody wants foundational infrastructure controlled by a direct competitor. But we shouldn’t confuse open interfaces with open markets.

An enterprise may find it easy to swap one MCP-compatible model for another while still remaining deeply dependent on the place where its prompts, evaluations, security policies, and employee habits have accumulated. Again, we’ve seen this before. Kubernetes made workloads dramatically more portable without making AWS, Microsoft Azure, and Google Cloud interchangeable. SQL has been standardized for decades, yet databases remain fiercely differentiated businesses. Standards reduce friction, but they rarely eliminate competitive advantage. They simply move it.

In like manner, Anthropic, Google, OpenAI, and others are happily standardizing how models, agents, tools, and enterprise systems communicate because they don’t expect the connection itself to determine the winner. Instead they expect to win by becoming the place where AI-assisted work naturally accumulates. Along the way, we’re going to see copious quantities of code given away, increasing developer productivity for all and outsized financial bonanzas for a few. Game on.

Somewhere in the last eighteen months, “AI agent” stopped being a useful term. CIOs may even be afraid to ask what “agent” truly means, as it now seems to describe everything from a chatbot that answers HR questions to an autonomous research system that plans its own week of work. When a single phrase carries that much weight, well, it stops carrying any.

I’ve spent the last three years inside hundreds of enterprise AI deployments, and the factor that separates the programs scaling elegantly from the ones still shuffling is often the CIO’s architectural fluency: The ability to look at business problems across the organization and recognize, on sight, what kind of AI architecture is the right fit. In my experience there are six archetypes, each with their own nuances, that CIOs should internalize to make well-informed decisions going forward.

1. The conversational assistant

The first, and the one most enterprises meet first, is the conversational assistant: The chat-based partner that an employee or customer opens when they want to think out loud. Deloitte found that 38% of organizations report AI is already strengthening their client or customer relationships. This is the architecture people fall in love with: A well-designed assistant with constantly updated information, persistent user-level memory, tools that can act on behalf of users, and citations on every factual claim becomes a useful problem-solver that’s available at any hour of the day.

A global law firm I work with deployed an internal assistant that gives every attorney instant access to the firm’s accumulated precedent, memos and prior matter work. Associates who used to spend the first hour of a research task hunting through document management systems now start with a grounded, citation-backed answer and refine from there. This helped the firm’s institutional knowledge, previously locked in the heads of senior partners, become queryable by anyone with a deadline at 11 p.m., or later.

A second example: A mid-market wealth management firm built a client-facing assistant that handles portfolio questions, statement explanations and routine servicing requests. The assistant draws from each client’s actual holdings, recent activity and the firm’s published market commentary, with citations linking back to source documents. Advisors stopped being interrupted for the questions that didn’t require an advisor, and clients got answers on a Sunday.

2. The triggered workflow

Another pattern producing the value across the enterprises I work with is something that runs silently: An email arrives, a ticket is created, a file lands in a folder and the agent executes a process utilizing both reasoning and determinism. These agents don’t even require user adoption, because they’re invisible to the end user. They produce measurable outcomes, but fit cleanly into the audit and change-control processes IT teams have run for decades.

A commercial insurer I advise built a triggered workflow for inbound submissions. Every broker email that arrives at the underwriting inbox is classified by line of business, the attachments are parsed, key risk fields are extracted into the policy administration system, and a draft acknowledgment is queued for the underwriter’s review. Seemingly overnight, the inbox began arriving pre-sorted, and submission throughput rose meaningfully without any change to headcount.

Another example, this time from a private equity firm: Every inbound confidential information memorandum (CIM) that hits the deal team’s shared inbox triggers a workflow that extracts the financial summary, screens it against the firm’s investment criteria, drafts a preliminary memo and posts the result into the deal-tracking system. Associates still make the call on what to pursue, but the first three hours of manual work on each opportunity now happen before anyone even opens the file.

3. The autonomous agent — with sub-agents

Here we have the architecture that gets the most conference attention: The autonomous agent, given a task and left to plan its own steps by utilizing its own sub-agents. Autonomous agents are not one-size-fits-all, but they do meet a specific need: Multi-source research, complex cross-system lookups, deep-dive investigations. All of these are processes where the path isn’t usually specified in advance, but the tools are. With the right design discipline, an autonomous agent feels like having a self-sufficient teammate who can call in the right resources and specialists if needed.

A global consulting firm I work with uses an autonomous research agent for early-stage engagement scoping. Given a target company and a strategic question, the agent decides for itself which sub-agents to consult (choosing from internal proprietary databases, prior engagement archives, licensed market data, public filings) and produces a structured briefing with its reasoning chain attached.

Another large technology company I know of deployed an autonomous agent for cross-system incident investigation. When a production alert fires, the agent forms a hypothesis, queries the necessary sub-agents with relevant monitoring tools, log stores and deployment systems, and follows the trail until it reaches a defensible root-cause summary to surface to an engineer.

4. The multi-agent team

The fourth pattern is where the next wave of enterprise quality gains is going to come from. According to Databricks, usage of multi-agent systems grew 327% in just four months as enterprises moved beyond single chatbots. Several specialized agents, each with its own role and toolset, coordinate through a shared protocol: A researcher and a writer, a planner and a set of executors, a proposer and a critic. The proposer-critic feedback loop is one of the smartest techniques in agent design today. One model produces an answer; a second, with a different prompt and often a different provider, evaluates it against explicit criteria. For compliance review, contract analysis, high-stakes classification and any output that will be audited, this second pass is extremely helpful and mirrors how human teams work.

A global bank I work with uses a multi-agent system for marketing and communications review. One agent drafts client-facing copy, a second checks it against the firm’s regulatory and brand guidelines and a third checks it against jurisdiction-specific disclosure rules. Disagreements among the agents are surfaced to a human reviewer with the specific clauses flagged. The compliance team stopped being the bottleneck on every routine piece of copy and started focusing on the high-judgment cases instead.

The next example: A pharmaceutical company built a multi-agent workflow for medical literature summarization. A retriever agent gathers candidate studies, a reader agent extracts study design and findings, a critic agent challenges the reader’s claims against the source text, and a synthesizer agent composes the final brief. The proposer-critic loop in the middle is the reason the medical affairs team trusts the output enough to act on it.

5. The human-in-the-loop (HITL) agent

The fifth pattern is the one I think we’ll see increasingly more of in the future. While many see “full automation” as the goal, the right target is actually to let the agent handle the 80% of a task that is mechanical, while preserving human judgment at the most critical moments. This is achievable via human-in-the-loop (HITL) agents. According to Moody’s, 42% of compliance professionals believe that human oversight is mandatory, and I agree: AI should run right, by getting approval and review before any sensitive business action is taken. HITL is the architecture that can help turn a skeptical team into an enthusiastic one.

A regional health system I worked with uses a HITL agent for prior-authorization letters. The agent assembles the clinical evidence, drafts the letter against the relevant payer’s criteria, and routes it to a nurse case manager for review inside the existing workflow tool. The nurse approves, edits or rejects in seconds rather than minutes, and every edit helps make the next draft better.

A property management company uses a HITL agent to run its maintenance work orders. When a tenant emails about a problem (an HVAC unit that died overnight, say), the agent pulls the structured details (tenant, unit, issue type, urgency), matches the job to the right vendor from the directory, and drafts the work order. A team member approves it in Slack before anything goes out. From there the agent emails the vendor with the full order, confirms with the tenant that someone is on the way and updates Airtable, closing the loop completely.

6. The scheduled agent

On a set schedule or against a batch of inputs, this agent runs the same defined task: Produce a report, refresh a dataset, monitor a set of sources or summarize a period of activity. Under this archetype, unsexy work gets done consistently, integrated into existing operational rhythms like the Monday morning meeting, the daily standup and the monthly board deck, without asking anyone to change their behavior. This is the architecture that shifts AI from feeling like even more work, to a seamless teammate that just works.

A private equity firm I work with runs a scheduled agent every Monday at 6 a.m. that monitors news, filings and earnings activity across every portfolio company and produces a single PDF that lands in the deal partners’ inboxes before the weekly investment meeting. No one logs into a dashboard. The agent shows up, on time, with the same format every week, and the meeting now starts from a shared baseline rather than from whatever each partner happened to read over the weekend.

A second example: A global manufacturer runs a nightly batch agent that ingests the day’s quality-control reports across plants, summarizes anomalies against a rolling baseline, and produces an end-of-shift handoff document for each site lead’s morning. The agent doesn’t flag emergencies, but it ensures that the slow-moving patterns no human would catch reading one shift’s data in isolation get surfaced.

Bringing it together

None of these six archetypes is more advanced than the others or inherently better. But CIOs can have an edge by choosing the one that the operational problem actually calls for.

Before you scope a single deployment, you should be able to look at a business problem and name its shape: Is this a question someone needs answered in the moment, or a process that should run the instant a trigger fires? Does the path need to be discovered, or is it known in advance and just waiting to be executed? Where, exactly, does human judgment have to stay in the loop, and where is it just friction?

Going forward, CIOs should start treating the architecture decision as the first design choice. Everything downstream — adoption, governance, trust — only gets easier if the architecture is the right fit.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Dear Microsoft,

I don’t want to have a love-hate relationship with Visual Studio Code, but you’re making it really hard.

Once upon a time, Visual Studio Code was just an editor. You configured it with add-ons to make it do whatever job you needed.

And that was great! I gravitated towards using VS Code for multiple jobs in various languages. I could set up a Python workflow with extensions to add database-wrangling functionality. I could put together an environment for digital book publishing. I could even set up a project for screenwriting! (Yes, I have a screenplay in progress. A man can dream.)

Now, it’s not like all this went away yesterday. I can still do all of the above. But literally every new feature in VS Code these days is all about agents, AI, and LLMs. It makes me wonder if there’s anyone left in the building who is working on the actual editor anymore.

In the release notes for VS Code 1.127, literally every single new feature save one was about agents and AI. Ditto the version before that. The version before that was a little better, but the trend has been clear for some time now: Visual Studio Code is fast becoming an agent front end first, and everything else is a distant second.

One argument I’ve heard is that VS Code is now a mature project. Because there just isn’t much left to be done to improve the core, you have directed your attention towards other features that will attract new users. And because agent-and-AI mania is still burning through the enterprise (and everyone’s budgets), agent-and-AI features are the order of the day.

But do these features really need to be IDE-level features? Instead of drastically retooling VS Code to support agents, and making AI features native elements of the IDE, shouldn’t the real mission be to support VS Code’s native extensibility? After all, VS Code was built to be extended, and thousands of AI add-ons — including GitHub Copilot and dozens of your own extensions — already take advantage of that.

Is all this really about what AI features VS Code can support natively, or is it more about making VS Code a default first point of contact for agents and AI — specifically GitHub Copilot? Is this less about developer convenience, and more about guaranteeing — however obnoxiously — that agents and AI tools always have a central place in VS Code?

Whatever the goal is, if you think every VS Code user wants an “agentic development environment” instead of an IDE, I think you’re dead wrong.

No doubt AI tools are here to stay. But I do think they will experience a reckoning. Frontier models that do everything and six more things on top of that are becoming of real use only to the big vendors who stick API tollbooths in front of them and construct wetland-devouring data centers the size of the Vegas strip to power them. The future looks more like smaller, locally-hosted models that handle highly specialized tasks and require far fewer resources to train, deploy, and serve. (Even you seem to be moving in that direction, with the Aion on-device models.)

The deeper we go into that reckoning, the less it makes sense to have cart-before-the-horse arrangements like VS Code’s native AI features. How long before these features get hoisted back out of VS Code and moved to an add-on (albeit one we get nagged to install every time we fire up our IDE)? Sooner rather than later, I’d bet.

Still, until that fine day comes, I’m resigning myself to the idea that my current favorite editor will be stuffed with more and more AI features, whether or not they even make sense in my use case, for a long, wearying time to come.

At least I can turn all that stuff off. For now.

As I was writing this (in VS Code, no less), I got an update to VS Code 1.128. There’s one nice feature in there I could see myself using quite widely: the ability to register OS-level keyboard shortcuts. But the rest? All AI, all agents, all the time.

Last year Anthropic gave away one of the most successful things it has ever built. And, no, I’m not talking about Claude. I’m referring to MCP, the now ubiquitous Model Context Protocol, which Anthropic donated to the Linux Foundation’s new Agentic AI Foundation⁠. At the time, MCP was pulling nearly 100 million monthly SDK downloads across more than 10,000 active servers⁠, prompting the question as to why any company would give up such a popular piece of technology.

Google did much the same months earlier, handing its Agent2Agent (A2A) protocol to the Linux Foundation⁠ with AWS, Cisco, Microsoft, Salesforce, SAP, and ServiceNow signing on as founding members. OpenAI, not to be outdone, supports remote MCP servers in its Responses API⁠, sits on the MCP steering committee, and contributed AGENTS.md to that same foundation alongside its fiercest rival’s protocol.

It’s like Game of Thrones, except the principal AI powers seek regime change through seeming acts of beneficence rather than violence. For those who have been around for a while, it’s also entirely predictable, following a similar script we’ve seen in the cloud, on-premises servers, and more. Platform companies don’t give away technologies they’ve stopped caring about. They give away technologies they no longer need to own because competitive advantage has shifted to new ground.

What does this mean for AI?

Gravity has shifted before

Google has long been an exceptionally active contributor to open source. As I wrote in 2017, Google wasn’t open sourcing TensorFlow and Kubernetes out of generosity but rather turning these open source assets into on-ramps for Google Cloud. Google was playing catch-up to AWS and Microsoft. As then Google product manager Martin Buhr said, the company hoped to “create a gravity well in the market for container-based apps [so] that a significant percentage of them will end up with us.”

In other words, platform companies routinely commoditize one layer of the stack so they can compete somewhere they hold a stronger hand.

GitHub may be an even better example. Git is open. Anyone can host a Git repository and, once upon a time, different companies did just that. Yet GitHub became the default place software development happens for millions of developers. Nobody pays for Git, but lots of people pay for GitHub. We’re seeing this same phenomenon play out in AI.

Trading contributions for control

Anthrophic and OpenAI have both pretended at being all for humanity’s good, but that’s not a good explanation for why they’re racing to give away things like MCP. The deeper reason is that the model itself has turned out to be a poor place to build a lasting moat, and they’re trying to figure out what’s next. As I pointed out recently, the frontier model leaderboards change almost weekly. As such, enterprises shouldn’t build their AI strategy around the assumption that any one vendor will remain permanently ahead on model quality. Instead, as I suggested, AI may be sexy, but the “dull reality” is connecting those models to enterprise data, workflows, etc.

The AI companies understand this better than anyone. Sure, they’ll continue spending billions training ever more capable models because frontier models attract developers, generate headlines, and open enterprise doors. But they’re also quietly acknowledging that benchmark leadership alone doesn’t create a durable platform.

Developers return to the places where their tools, workflows, teammates, and accumulated work already live. Enterprises double down on the systems where their data, permissions, governance, and business processes are already connected. Every new integration makes that destination a little harder to leave, and every new workflow increases its pull. That’s what MCP, A2A, etc., are all about: increasing gravity around the models.

Every major AI company wants to become the place where AI-assisted work naturally happens, and they’re now amassing armies of forward deployed engineers and trying other means to get legacy infrastructure to tie back to their frontier models. The enterprise incumbents want the same thing, but from the opposite direction. They don’t need to own the frontier; instead they need to connect the frontier to the systems that already safely run the business.

That’s why I’m skeptical whenever someone confidently predicts that AI will sweep away enterprise software. I’ve seen this movie before. Developers absolutely live on the frontier, but enterprises don’t. Enterprises create value by connecting new capabilities to decades of accumulated applications, data, policies, and business processes. The newest model matters, and so does the newest agent framework. But neither creates much business value until it’s connected to customer records, financial systems, supply chains, HR data, and everything else enterprises already depend on.

That’s where incumbents still possess enormous gravitational pull. My employer, Oracle, certainly believes so, just as Microsoft, SAP, Salesforce, and ServiceNow do. (Disclosure: I run developer relations at Oracle, which participates in the Agentic AI Foundation.) Ironically, open protocols strengthen that position rather than weaken it. If every model can speak MCP and every agent can interoperate through common standards, enterprises gain the freedom to adopt whichever frontier technology looks best without rebuilding every integration. The protocol becomes interchangeable.

Open standards don’t stop gravity

None of this diminishes the importance of open standards. MCP succeeded because it solves a genuine problem. Developers shouldn’t have to build a custom connector every time an AI application needs access to a database or other business system. Neutral governance also matters because nobody wants foundational infrastructure controlled by a direct competitor. But we shouldn’t confuse open interfaces with open markets.

An enterprise may find it easy to swap one MCP-compatible model for another while still remaining deeply dependent on the place where its prompts, evaluations, security policies, and employee habits have accumulated. Again, we’ve seen this before. Kubernetes made workloads dramatically more portable without making AWS, Azure, and Google Cloud interchangeable. SQL has been standardized for decades, yet databases remain fiercely differentiated businesses. Standards reduce friction, but they rarely eliminate competitive advantage. They simply move it.

In like manner, Anthropic, Google, OpenAI, and others are happily standardizing how models, agents, tools, and enterprise systems communicate because they don’t expect the connection itself to determine the winner. Instead they expect to win by becoming the place where AI-assisted work naturally accumulates. Along the way, we’re going to see copious quantities of code given away, increasing developer productivity for all and outsized financial bonanzas for a few. Game on.

We are inundated with promises that artificial intelligence will save us and that the next governance framework will protect us. Buy this platform, adopt that model and the hard part finally gets easier. After 15 years in this field, I have wanted that shortcut as much as anyone.

But both promises are downstream of something neither one can produce. You cannot automate trust between two people. You cannot govern your way to a relationship. As AI moves into the core of how organizations operate, and accountability stops mapping cleanly to the org chart, what holds when the stakes are highest is not the platform or the policy. It is two human leaders who know each other well enough to carry the weight together.

I think about this often now, a year after publishing a book about the pressures bearing down on security leaders, “The CISO On The Razor’s Edge: Leading Cybersecurity When The System Is Designed To Break.” The partnership between the CIO and the CISO is the last human relationship in cybersecurity. AI raises the stakes. Governance sets the floor. The relationship is what holds.

I saw it work once, up close. When I worked in Washington State, the CIO, Bill Kehoe, talked to his CISO, Ralph Johnson, every day. Weekends included. Not because a policy required it, but because the mission did. That partnership is a large part of why the role stayed sustainable for them when it broke so many others.

The promise we keep believing

Walk any conference floor and you will hear the same pitch in a hundred variations. The next AI layer will close the gap. The next framework will lock down the risk. The technology is usually ready. The organization is not. I have watched too many well-funded programs stall to still believe the tool is the answer, and almost every time, the breakdown traced back to leaders who were not aligned before the work began. A framework run by misaligned leaders inherits the misalignment. You can buy the best controls on the market and still watch them fail when two leaders work from different assumptions about who owns what.

Bill and Ralph understood this. Security decisions were not handed to Ralph after the fact to bless or block. They were made with him, inside the technology decisions, because the two had already agreed on what mattered. That is not governance. That is leadership creating the conditions in which governance can work.

It is the real lesson I came to in the book. Technical knowledge matters, but it is not enough. As I wrote then, “Influence, trust and internal relationships are non-negotiable.” Without influence, CISOs cannot lead. Without technical substance, they cannot prioritize what matters. And without partnership, especially with their CIO, “they’re operating without a safety net.”

AI does not change that truth. It raises the cost of ignoring it.

When decisions move at machine speed

The ground under both roles is shifting. Work no longer flows through people alone. It moves across people, platforms, partners and agents at the same time, and it moves fast. Decisions that once waited for a meeting now form in seconds. The org chart, built for an era when humans did the work and reporting lines explained accountability, struggles to keep up.

This is where the partnership stops being a nicety and becomes infrastructure. When decisions form at machine speed, the human escalation path has to be instant. There is no time to negotiate a relationship in the middle of an incident. Either the trust is already there, built in the quiet stretches before anything goes wrong, or it is not there when it counts.

I asked Bill what he would lose if his daily calls with Ralph dropped to once a week. His answer cut straight to it.

“Cyber does not rest,” he told me. “It is active and dynamic and requires 24/7/365 attention.” Drop to a weekly check-in, he explained, and “I am treating the CISO like any other executive position.” For Bill, AI only raises the stakes on that daily contact. “Relationships and partnerships between the CIO and CISO will never die due to AI,” he said. “I can’t even imagine a scenario where I don’t talk to my CISO on a daily basis including weekends to discuss the latest risks and vulnerabilities or news on potential AI attacks.”

That is the point most of the market misses. A platform can flag the anomaly. It cannot decide what the organization is willing to risk, who carries that decision or how two leaders stand behind it together. The faster the machines move, the more the partnership has to already be in place.

The loneliest seat in the building

There is a reason some now call the CISO job the least desirable role in business. The seat carries enormous accountability and rarely the authority to match. As one security leader put it, the pressure has never been higher and the control has never felt lower. People are burning out and walking away from a role that has never mattered more.

Here is the hard part. There is no log file for burnout. No alert fires when the weight finally exceeds the leader. That drain is invisible right up until it is not, and it raises organizational risk as surely as any unpatched system. The structural fixes the industry debates are all real and all slow.

The fastest source of relief available to a CISO is not a framework. It is a CIO who treats the relationship as a daily partnership rather than a line on a chart. An isolated CISO is a vulnerability. A partnered one is an asset.

You see what that partnership is worth in the worst moment. I asked Bill what it looks like when an incident hits and public trust is on the line. He did not reach for a tool.

“I am accountable as CIO to everything that occurs in the state from a technology lens including cyber,” he said. When a severe incident hits, the call comes to him from agency leadership or the Governor’s Office. Then he follows the plan, but never alone: “I will be in constant contact with the CISO on the details of the incident.”

That is the safety net made real. The CISO is not carrying the mission alone at the moment it matters most. On the razor’s edge, leadership keeps you upright. Partnership keeps you in the fight.

The work no tool will do for you

In my advisory work, I sit with C-suite leaders who share values and still cannot find alignment. The barrier is rarely disagreement. It is that they are not communicating clearly or often enough to build the trust that alignment requires. I have watched negotiations that could only happen by proxy, over email, because two capable leaders had stopped talking directly.

I recently sat in an hour-long discussion where alignment and shared values were present the whole time. It did not become clear until the final fifteen minutes. That is what real alignment costs: patience, persistence and a stubborn commitment to clarity. If leaders cannot do that work themselves, no AI model or governance tool will do it for them.

This is why I stand up an AI review board for the organizations I work with and host the leadership conversations that decide whether a company’s AI ambitions thrive or stall. The board itself matters less than what it provides: neutral ground, a regular cadence and an agenda that forces the hard issues into the open before a crisis forces them. If your organization has no venue like that, that absence is its own form of dysfunction. The cadence is what makes communication effective. Not easy. Effective.

Build the bond on purpose

You cannot framework your way to trust. But you can build it deliberately, and that is a leadership act, not a governance one. The partnership and stakeholdering skills that once looked like soft extras are now the core executive work. A few moves matter most:

• Set a standing contact rhythm with your counterpart before you need one, daily or near-daily, not quarterly
• Make decision rights and accountability explicit while it is calm, so no one improvises them mid-incident
• Translate security into business outcomes together, so the board hears one aligned voice

Build the relationship as deliberately as you would build any critical control, because that is what it is. If you cannot connect the partnership to outcomes the business actually feels, you have a friendship, not a performance lever.

A year after writing “The CISO On the Razor’s Edge,” I am even more convinced that strong leadership precedes effective governance and partnership precedes them both. This is the good news, not the hard news. The CIO and CISO who build real trust do not just reduce risk. They move faster than their competitors, because they spend no energy fighting each other. They earn the board’s confidence, because the board hears one clear voice. And they unlock the AI strategy everyone else is still struggling to govern, because they have already done the human work that makes governance hold.

That is the upside waiting on the other side of this relationship. AI will keep advancing. Governance will keep maturing. But the organizations that win the next decade will be the ones where two leaders decided the partnership was worth building before they needed it. Bill and Ralph knew it every day, weekends included. The edge is there for anyone willing to do the same.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

I keep seeing a version of the same scene. A CEO has a question about AI. It is a real question, the kind that will shape where the company spends the next two years. The CEO does not bring it to the CIO. They bring it to a data leader two levels down, or to a vendor who presented at a conference, or to an AI specialist a board member recommended. The CIO finds out the strategy is forming when a slide shows up that they did not build. By then, the direction is already half-set, and the CIO is being asked to react to it rather than shape it.

I want to be precise about what is happening, because it is easy to misread. The CIO has not been removed from anything. Title intact, budget intact, seat at the table intact. What has changed is quieter. On one of the most consequential technology conversations the company will have this decade, the CIO is being routed around. The work still flows through them eventually. The thinking no longer starts with them.

I have watched this happen to capable people who would have given the CEO a better answer than the person who was asked. That is what makes it worth naming. This is not a competence gap. It is a positioning gap, and positioning gaps close in the wrong direction if you ignore them long enough.

How the routing actually starts

The routing does not begin with a decision to exclude anyone. It begins with a CEO who is anxious about AI and looking for someone who sounds certain. AI is moving fast enough that executives feel the pressure to have a point of view before they have earned one. That pressure usually arrives secondhand, from a board member or a peer on the golf course describing what is working at their company. So the CEO goes looking for someone who will confirm the answer they already want to hear, and they keep going back to whoever gives it to them.

Here is where many technology leaders lose the thread. For years, the safe posture in the CIO seat was measured caution. You raised the risks, you flagged the integration cost, you asked who owns the data and what the compliance exposure looks like. That posture built credibility in an era when the failure mode was moving too fast on technology nobody understood. With AI, the same posture reads as drag. A CEO who is being told by three vendors that “the future is already here” does not want to hear why they should slow down and be cautious. They hear caution as losing the race, and they go find a point of view somewhere else.

The data leaders, vendors and specialists who get the call are not necessarily more capable. They are more available with a confident answer. A vendor’s whole job is to arrive with conviction. A data scientist who has shipped one impressive model carries more apparent authority on AI, in that moment, than a CIO who runs the entire estate but talks about AI the way they talk about every other risk. The CEO is not weighing depth against depth. They are weighing the person who said yes against the person who said it depends.

Once that pattern sets, it compounds. The CEO who got a satisfying answer from the data leader goes back to the data leader. The vendor who shaped the first conversation gets invited into the second. Each loop the CIO is not in makes the next one easier to run without them. The org chart still says the CIO owns technology strategy. The actual conversation has relocated.

What it costs before anyone notices

The cost shows up late, which is exactly why it is dangerous. For a while nothing looks broken. The CIO is still delivering. The AI initiatives are still landing on their plate to execute. The damage is happening upstream, in the room where the bets get made, and the CIO is not in that room.

I have seen what arrives downstream when the strategy was set without the person who has to run it. A model gets championed that the data cannot actually support. A vendor commitment gets made that locks the company into an architecture the CIO would have flagged in the first meeting. An agent gets deployed inside a business unit, with executive blessing, and the CIO inherits accountability for it months later without ever having shaped how it was governed. The recent IBM finding that CIOs are increasingly held accountable for AI they do not fully control is the visible end of this. The invisible front end is the conversation the CIO was routed around, the one where the accountability got created in the first place.

What I find most corrosive is what it does to the CIO’s standing over time. Every initiative the CIO executes but did not shape reinforces a story about what the CIO is for. They become the person who runs the technology other people decided on. That is a fine description of an order taker and a poor description of a strategic leader, and CEOs do not promote, fund, or defend order takers when budgets tighten. The routing-around does not just cost the company a worse AI strategy. It quietly recasts the CIO as the implementer of everyone else’s thinking, and that recasting is hard to reverse once the executive team has internalized it.

What the leaders who stayed in the conversation did

The technology leaders I have watched hold their position on AI did one thing first. They stopped leading with caution and started leading with a point of view. Not a reckless one. A real, defensible position on where AI creates value in their specific business and where it does not, delivered with the same conviction the vendors bring, before the CEO went looking elsewhere for it. They made themselves the person with the clearest answer, which is the role the routing-around was filling with someone else.

That requires giving up a posture that felt safe for a long time. The CIOs who made the shift accepted that on AI, being right and cautious is worth less than being early and directional. They formed a view ahead of being asked. They walked into the CEO’s office with where we should place our AI bets and why, rather than waiting to be handed someone else’s bets to pressure-test. The difference is whether you are the author of the strategy or its editor, and CEOs route around editors.

They also changed how they talk about risk. Instead of presenting risk as the reason to slow down, they folded it into the recommendation. The data is not ready for that use case, so here is the use case where it is ready, and here is what we do in parallel to unlock the first one. That framing keeps the CIO inside the conversation as the person making AI happen responsibly, rather than the person standing outside it explaining why it is hard. Same expertise, opposite effect on whether the CEO keeps coming back.

None of this is about pushing the data leaders and specialists out. The strongest CIOs I know pulled those people closer and brought them into the room under their own framing, so that when the CEO wanted the specialist’s input, it arrived through the CIO rather than around them. They made themselves the orchestrator of the AI conversation instead of one of its casualties.

If you are a technology leader right now, the question worth sitting with is not whether you are good at AI. You probably are. The question is whether the most important AI conversations in your company are still starting with you, or whether you have quietly become the person they get handed to after the thinking is done. That answer is set in rooms you may not be in, and the only way to find out is to ask who your CEO called the last three times AI came up. If the answer is not you, the role is still yours. The conversation has already started leaving.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Organizations usually face the same questions concerning XLAs: What should we measure, who owns the data, how should incentives work, and how will this change provider behavior after signature.

There are no easy answers either, but after advising clients in MSP relationships with major providers, I’ve seen what works and what doesn’t. Successful XLA programs rarely start with massive transformation, nor rely on perfection before adding experience accountability to the contract.

Start with the right metrics

The first concern I hear is what to measure. MSPs often steer that discussion toward metrics already in their reporting stack. That’s a trap.

Unlike SLAs, which measure operational outputs, XLAs should focus on employee experience and business outcomes. The strongest programs start with three to five high-signal metrics tied to the employee journeys creating the most friction. More than that and the program loses focus before it gains traction.

I typically recommend starting with employee satisfaction scores, perceived lost productivity time, repeat incident rates, task completion success, and ease of getting support. Then focus early measurement on common employee experiences like service desk interactions, employee onboarding, application reliability, and device performance.

Trying to measure everything is understandable, but it’s also one of the fastest ways to stall an XLA program.

Precisely define roles and responsibilities

This is the part of XLA contract design where I spend the most time with clients, and it’s the part that major MSPs are most likely to leave vague if you let them. Accenture and TCS both have mature commercial teams skilled at agreeing to things in principle while avoiding specific accountability in writing. Don’t let that happen here.

Employee experience isn’t solely the vendor’s responsibility. It’s genuinely shared, which is a more productive framing than pure vendor accountability, but only if the split is clearly spelled out. This is what I’ve found works in practice.

Customer responsibilities

• Selecting tools and platforms
• Managing data infrastructure
• Sharing experience data openly with the provider
• Supporting internal improvement initiatives that the provider flags

Vendor responsibilities

• Running the measurement cadence
• Delivering monthly experience reporting
• Identifying and surfacing improvement opportunities from the data
• Executing operational improvements within agreed timelines

Without this level of specificity, XLA programs almost always become reporting exercises. The data gets collected, the scorecard gets presented, and nothing actually changes.

Build flexible targets

One of the biggest mistakes in XLA design is treating experience targets like traditional SLAs,  setting once at contract signing and left unchanged for years. Employee expectations, workforce patterns, and technology environments, after all, evolve constantly. A target that feels ambitious in year one may become meaningless by year three.

The strongest XLA contracts include formal reviews every three to six months to recalibrate targets, align with business priorities, and raise expectations as experience improves. This prevents providers from locking in easy wins and coasting. When providers resist review cycles, it’s often a sign they believe the targets can be met on autopilot, a red flag in any XLA program.

Use the right scoring method

One overlooked XLA best practice is how experience scores are calculated. Point-in-time scores can be distorted by outages, isolated incidents, or low survey participation, and providers sometimes exploit that volatility.

I advise clients to calculate official XLA scores using rolling two-month averages instead of snapshots. It creates a more stable and accurate view of experience trends, and makes operational timing games much harder. Most importantly, define the scoring methodology explicitly in the contract. Don’t leave it to be worked out operationally after signing.

Structure incentives carefully

Relying on penalty-only incentives is one of the most expensive XLA mistakes. On paper, the model is simple: miss the target, pay the penalty. In practice, it drives the wrong behavior. Providers focus on protecting themselves instead of improving employee experience, optimizing survey timing, and managing averages rather than solving problems collaboratively.

I’ve seen this repeatedly in Infosys, HCL, and TCS relationships. The strongest XLA structures combine risk and reward where providers earn meaningful upside for exceeding targets, innovating, and improving outcomes. Penalties still matter, especially in mature programs, but they can’t be the only lever otherwise the contract becomes another SLA model with better branding.

Define escalation processes

When experience scores fall below threshold, the contract needs to specify what happens next. This sounds obvious, but I’ve reviewed many service delivery measurement frameworks in clients’ incumbent MPS contracts that specify financial consequences without defining any collaborative process to address the underlying problem.

The escalation language I push clients to include specifies:

• a joint review process triggered when scores fall below threshold.
• root cause analysis expectations and timelines.
• remediation planning requirements with named owners on both sides.
• timelines for corrective action and progress reporting.

The framing matters as much as the mechanics. Escalation should be positioned as collaborative problem-solving, not blame assignment. Contracts that turn every missed score into a commercial dispute damage the relationship when provider engagement matters most. The best MSPs treat escalation as a shared diagnostic exercise, not a contractual confrontation.

Establish an operating rhythm

Signing the contract is the beginning, not the end. In my experience, the organizations that get the most out of XLA programs are those that build a disciplined operating cadence and stick to it. The ones that treat XLAs as a reporting exercise almost never see meaningful improvement.

This is the cadence I recommend:

Daily: Both parties maintain live dashboards showing experience trends, application performance, regional issues, and persona-specific insights to catch emerging issues.

Weekly: Customer and vendor teams hold focused working sessions to determine what improved experience this week, what hurt it, which remediation actions were completed, and what’s the priority for next week.

Monthly: Formal governance meetings to review experience scores, improvement actions, root cause discussions, and cross-functional issues that need escalation.

Biannually: Leadership steering meetings to assess overall experience performance, recalibrate targets, and align the XLA program with evolving business priorities to honestly evaluate whether or not the program is driving the outcomes the organization actually cares about.

Common mistakes organizations make

After working through XLA design and implementation with clients across their MSP relationships, the failure modes are predictable. Here’s what to watch for.

Setting targets before establishing a baseline
Rushing into targets before understanding your current state is one of the fastest ways to create disputes. Spend the first three to six months gathering baseline data, then negotiate targets based on evidence rather than guesswork.

Measuring too much
More metrics don’t create more insight. Frameworks with 20 data points rarely survive operational reality. Start focused and expand gradually.

Hiding the data
Transparency is foundational to XLAs. Providers who obscure poor scores, especially when controlling the measurement platform, undermine the entire model. Clients who weaponize the data create the same problem. Build mutual transparency obligations into the contract.

Over-relying on penalties
Penalty-only structures recreate legacy SLA behaviors. Balanced incentives drive better long-term outcomes.

Treating XLAs as static
Employee expectations, technology, and business priorities evolve constantly. Without formal review cycles, XLA programs quickly become irrelevant.

Start smaller than you think you need to

The organizations that get XLAs right are rarely the ones with the most sophisticated tooling. They’re the ones that stopped waiting for a perfect program and introduced real accountability into the contract with what they had.

The most effective starting points are often simple: agree on a focused set of experience metrics, establish a six-month review cycle, commit to shared visibility and data transparency, and create joint accountability for continuous improvement.

From there, maturity develops over time. Governance builds trust, data becomes more actionable, and targets evolve alongside business priorities. The relationship shifts from compliance management to outcome-driven partnership.

In my experience, the organizations that succeed are the ones that stopped accepting green scorecards at face value and demanded something more meaningful.

The Linux Foundation has launched the x402 Foundation, a new industry body that will oversee the x402 payment protocol, an open standard designed to let AI agents, applications, and APIs pay for digital services over HTTP.

The x402 protocol, originally developed by Coinbase, embeds payment capabilities directly into web interactions, allowing AI agents, APIs, and applications to send and receive payments as part of standard HTTP requests rather than through separate checkout or billing systems, according to the Linux Foundation. The protocol supports multiple payment types, from traditional cards to stablecoins.

“Under the neutral governance of the Linux Foundation, the x402 Foundation will allow developers, financial institutions, cloud providers, and other community members to collaboratively shape the protocol’s development,” the Linux Foundation said in a statement. “This open structure ensures that payments remain highly secure and adaptable, supporting multiple payment types, from traditional cards to stablecoins, without vendor lock-in.”

Forty organizations have joined the x402 Foundation since the Linux Foundation announced plans for the project in April, the statement added.

Members include Amazon Web Services (AWS), Google, Visa, Mastercard, Stripe, American Express, Cloudflare, Coinbase, Fiserv, Ripple, and Shopify, representing cloud providers, payment companies, and financial services firms.

Foundation targets a gap in agent-to-agent commerce

The announcement comes as software vendors add AI agents to business applications and developer platforms. Many of these agents are designed to call APIs, access third-party services, and complete tasks on behalf of users, creating demand for ways to pay for digital services without relying on separate payment systems.

Jim Zemlin, CEO of the Linux Foundation, said AI agents and automated systems are becoming active participants in the global economy but have lacked a native, secure way to transact.

“By bringing together leading companies across finance, technology and more, we’re ensuring that the payment layer of the internet remains neutral, highly interoperable and ready to support digital commerce,” he said in the statement.

The protocol addresses what several founding members described as a structural gap in how the web handles machine-initiated transactions.

How the protocol works

The x402 protocol is based on the HTTP 402 “Payment Required” status code, which was originally defined for internet payments but has seen limited use.

The protocol is intended for transactions involving paid APIs, AI services, cloud computing resources, digital content, and other online services that require payment. The Linux Foundation said it also supports machine-to-machine payments between software applications and can work with multiple payment methods, including traditional payment cards and stablecoins.

Today, developers typically monetize APIs and online services through subscriptions, prepaid credits, API keys, or account-based billing systems. The Linux Foundation said x402 is designed to standardize payment directly within HTTP interactions, allowing applications and AI agents to complete transactions without separate payment flows or custom billing integrations.

Governance structure spans payments, cloud and blockchain sectors

Under the Linux Foundation’s neutral governance model, the x402 Foundation will let developers, financial institutions, cloud providers, and other members collaboratively shape the protocol’s development.

“This open structure ensures that payments remain highly secure and adaptable, supporting multiple payment types, from traditional cards to stablecoins, without vendor lock-in,” the statement added.

Technology vendors have introduced AI agents that can search for information, generate code, analyze documents, and interact with external applications. Many of those systems also rely on APIs and cloud-based services to complete tasks.

According to the Linux Foundation, x402 is designed to provide a standard way for those applications and agents to pay for services during a transaction rather than relying on separate purchasing or billing processes. The foundation said developers can integrate payment capabilities into applications using open web standards across different payment providers and software platforms.

Growing ecosystem

The launch comes as technology vendors begin adding payment capabilities to AI agent platforms.

In May, AWS introduced Amazon Bedrock AgentCore Payments in preview, enabling AI agents to autonomously pay for APIs, Model Context Protocol (MCP) servers, web content, and other agents. AWS had then said the service uses the x402 protocol to negotiate HTTP 402 payment requests while handling wallet authentication, spending controls, and transaction logging.

The Linux Foundation said the x402 Foundation will serve as the neutral home for the protocol as organizations contribute technical specifications, implementation guidance, and future extensions. The Linux Foundation and Coinbase did not respond to requests for additional comment by publication time.

The article originally appeared on InfoWorld.

The Linux Foundation has launched the x402 Foundation, a new industry body that will oversee the x402 payment protocol, an open standard designed to let AI agents, applications, and APIs pay for digital services over HTTP.

The x402 protocol, originally developed by Coinbase, embeds payment capabilities directly into web interactions, allowing AI agents, APIs, and applications to send and receive payments as part of standard HTTP requests rather than through separate checkout or billing systems, according to the Linux Foundation. The protocol supports multiple payment types, from traditional cards to stablecoins.

“Under the neutral governance of the Linux Foundation, the x402 Foundation will allow developers, financial institutions, cloud providers, and other community members to collaboratively shape the protocol’s development,” the Linux Foundation said in a statement. “This open structure ensures that payments remain highly secure and adaptable, supporting multiple payment types, from traditional cards to stablecoins, without vendor lock-in.”

Forty organizations have joined the x402 Foundation since the Linux Foundation announced plans for the project in April, the statement added.

Members include Amazon Web Services (AWS), Google, Visa, Mastercard, Stripe, American Express, Cloudflare, Coinbase, Fiserv, Ripple, and Shopify, representing cloud providers, payment companies, and financial services firms.

Foundation targets a gap in agent-to-agent commerce

The announcement comes as software vendors add AI agents to business applications and developer platforms. Many of these agents are designed to call APIs, access third-party services, and complete tasks on behalf of users, creating demand for ways to pay for digital services without relying on separate payment systems.

Jim Zemlin, CEO of the Linux Foundation, said AI agents and automated systems are becoming active participants in the global economy but have lacked a native, secure way to transact.

“By bringing together leading companies across finance, technology and more, we’re ensuring that the payment layer of the internet remains neutral, highly interoperable and ready to support digital commerce,” he said in the statement.

The protocol addresses what several founding members described as a structural gap in how the web handles machine-initiated transactions.

How the protocol works

The x402 protocol is based on the HTTP 402 “Payment Required” status code, which was originally defined for internet payments but has seen limited use.

The protocol is intended for transactions involving paid APIs, AI services, cloud computing resources, digital content, and other online services that require payment. The Linux Foundation said it also supports machine-to-machine payments between software applications and can work with multiple payment methods, including traditional payment cards and stablecoins.

Today, developers typically monetize APIs and online services through subscriptions, prepaid credits, API keys, or account-based billing systems. The Linux Foundation said x402 is designed to standardize payment directly within HTTP interactions, allowing applications and AI agents to complete transactions without separate payment flows or custom billing integrations.

Governance structure spans payments, cloud and blockchain sectors

Under the Linux Foundation’s neutral governance model, the x402 Foundation will let developers, financial institutions, cloud providers, and other members collaboratively shape the protocol’s development.

“This open structure ensures that payments remain highly secure and adaptable, supporting multiple payment types, from traditional cards to stablecoins, without vendor lock-in,” the statement added.

Technology vendors have introduced AI agents that can search for information, generate code, analyze documents, and interact with external applications. Many of those systems also rely on APIs and cloud-based services to complete tasks.

According to the Linux Foundation, x402 is designed to provide a standard way for those applications and agents to pay for services during a transaction rather than relying on separate purchasing or billing processes. The foundation said developers can integrate payment capabilities into applications using open web standards across different payment providers and software platforms.

Growing ecosystem

The launch comes as technology vendors begin adding payment capabilities to AI agent platforms.

In May, AWS introduced Amazon Bedrock AgentCore Payments in preview, enabling AI agents to autonomously pay for APIs, Model Context Protocol (MCP) servers, web content, and other agents. AWS had then said the service uses the x402 protocol to negotiate HTTP 402 payment requests while handling wallet authentication, spending controls, and transaction logging.

The Linux Foundation said the x402 Foundation will serve as the neutral home for the protocol as organizations contribute technical specifications, implementation guidance, and future extensions. The Linux Foundation and Coinbase did not respond to requests for additional comment by publication time.

For a decade, Kubernetes was the right answer. It organized containers, scaled services horizontally and gave platform teams a shared vocabulary for running software in production. It abstracted away enough of the underlying complexity that engineers could stop thinking about servers and start thinking about services. Most cloud-native infrastructure today is built on top of it, directly or in spirit, and EKS made that model the default for the majority of enterprise teams running workloads on AWS.

The workload that defined that era was the stateless HTTP request, fast in, fast out, disposable. A user action triggers a request, the request hits a service, the service returns a response and the container is done. Kubernetes was optimized for that pattern down to the scheduler internals: Bin-pack containers onto nodes, autoscale on CPU and memory, evict and reschedule when something goes wrong. The whole system is tuned around the assumption that individual units of work are short, stateless and interchangeable.

That assumption no longer holds for the workloads that matter most right now.

The agent workload is structurally different

Agents are long-running, stateful processes. They reason across time, call external tools, spawn subprocesses, write and execute code, and make decisions that depend on what happened five steps earlier in the same task. A single-agent workflow might run for minutes or hours, touching a dozen external systems and generating intermediate outputs that subsequent steps depend on. The compute layer for that kind of work needs to do things the old model was never asked to do. That is the new pattern: Execution infrastructure designed around agent semantics rather than request semantics.

The Kubernetes community itself has acknowledged this mismatch. In March 2026, Kubernetes SIG Apps published an introduction to Agent Sandbox, a new CRD-based abstraction designed specifically for singleton, stateful agent workloads. The framing is direct: The ecosystem is moving from short-lived, isolated tasks to deploying multiple, coordinated AI agents that run continuously, and mapping those workloads to traditional Kubernetes primitives requires an entirely new abstraction. The fact that the Kubernetes maintainers built a dedicated primitive for this, rather than recommending teams compose one from existing resources, is itself the clearest signal that agent execution does not fit the old model.

What agent execution actually requires

Concretely, it requires four things. First, isolated execution environments that provision in milliseconds, not minutes, so each agent task gets its own sandbox for code execution and tool calls without blocking the reasoning loop. The difference between a two-second environment and a two-minute environment is not a performance optimization; it determines whether the architecture is viable at all. Second, durable state management across the full task lifecycle, so an agent can pause, hand off or resume without re-initializing from scratch and burning tokens to reconstruct context it already built. Third, coordination primitives for multi-agent work: The ability to spawn subagents, pass structured outputs between them and track task dependencies across a graph of concurrent processes. Production agent systems are rarely single agents; they are pipelines of specialized agents with handoffs that need to be reliable and inspectable. Fourth, credentials and secrets management that travel with the execution context, so agents can authenticate to external services securely without exposing credentials in the task definition, logs or the environment variables of a shared container.

The mismatch shows up fast in production

Kubernetes and EKS expose the mismatch quickly in practice. Pod eviction terminates an agent mid-task with no clean recovery path. Autoscaling reads CPU utilization as the load signal, but an agent holding a long inference connection looks idle to the scheduler even when it is doing the most consequential work in the pipeline. Provisioning a new environment takes 45 seconds to two minutes on a well-tuned cluster; agent workloads need that in under two seconds or the reasoning loop stalls and the user experience degrades visibly. These are not edge cases or misconfigurations. They are the normal operating conditions for production agent workloads running on infrastructure that was not designed for them.

The utilization data makes the broader cost picture even starker. The 2026 State of Kubernetes Optimization Report from CAST AI, drawn from analysis of over 23,000 production clusters across AWS, Azure and GCP, found average CPU utilization at 8 percent, down from 10 percent the year prior. Memory utilization fell from 23 to 20 percent. CPU overprovisioning jumped from 40 to 69 percent year over year. These numbers reflect clusters running traditional workloads, and the pattern is worsening, not improving, as environments scale. Agent workloads compound this problem further. An agent holding an open inference connection or waiting on a tool call registers as idle to a scheduler that reads CPU and memory as the only meaningful load signals. The infrastructure responds to the wrong metric, overprovisioning capacity for demand it cannot measure, while the actual bottleneck, environment provisioning latency and state continuity, goes unaddressed.

Security is not the same problem it was before

Agent workloads change the threat model at the infrastructure level. A compromised stateless service exposes a narrow surface defined by its API contracts. A compromised agent exposes every system it can reach, every credential it holds and every action it is authorized to take on behalf of the user. Agents generate and execute their own code, make non-deterministic tool-call decisions and accumulate context across long-running sessions. Standard container namespacing does not contain that kind of risk. Kernel-level isolation, default-deny network egress, scoped credentials per session and agent-aware observability are not optional hardening steps. They are baseline requirements for running agents in production.

What teams that ship agents have already figured out

Some of the clearest evidence for this shift comes not from infrastructure vendors but from product engineering teams running agents at scale on their own code. In late 2025, Ramp’s engineering team published a detailed account of building Inspect, their internal background coding agent. Each Inspect session runs in a sandboxed VM with a full-stack development environment and deep integrations across their observability, CI, and deployment tooling. The architecture requirements map almost exactly to the four primitives above. Filesystem snapshots keep sessions starting in seconds rather than minutes. Sessions are isolated and stateful. The agent can run tests, review telemetry, query feature flags and visually verify frontend changes in a real browser. And the whole system supports unlimited concurrency, so engineers can spin up ten parallel sessions exploring different approaches to the same problem without contention.

The results speak for themselves. Within months of launch, roughly 30 percent of all pull requests merged to Ramp’s frontend and backend repositories were written by Inspect. That level of adoption was not mandated. It happened because the execution environment was fast enough, capable enough and well-integrated enough that the agent was strictly better than a local workflow for a meaningful share of tasks. The key insight from the Ramp case is not about the model. It is about the execution layer. As their team put it, session speed should only be limited by model-provider time-to-first-token; everything else, like cloning and installing, needs to be done before the session starts. That is a statement about infrastructure, not intelligence.

The ecosystem is catching up, but defaults are sticky

None of that is a criticism of the tools. Kubernetes solved exactly the problem it was designed for, and it solved it well. The issue is that infrastructure defaults are sticky. Teams inherit them, build on top of them and optimize within their constraints long after the underlying workload has changed. The Kubernetes community’s own response, the Agent Sandbox project under SIG Apps, validates the thesis that a new abstraction is necessary. The new primitives the community is building include warm pools for near-zero cold starts, lifecycle management for suspending and resuming idle agents without losing state, and pluggable kernel isolation for secure execution of untrusted code. These are not incremental improvements to existing resources. They are net-new abstractions that acknowledge the old model does not stretch to fit.

But adoption of purpose-built agent infrastructure remains early. Enterprises building agent pipelines today are largely running a request-oriented orchestration model against an execution-oriented workload, and the mismatch shows up in task failure rates, runaway costs and debugging cycles that have no good tooling because the observability layer was also designed for stateless services.

The structural advantage is available now

The infrastructure to close that gap exists now. The prerequisite is recognizing that agent execution is a first-class compute pattern with its own primitives and its own requirements, not a variant of the stateless service model that defined the last decade. Teams that make that shift early will have a meaningful structural advantage. The ones that do not will spend the next two years wondering why their agent systems are unreliable at a scale that should be tractable.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Every day this past week I did something I suspect millions of other people also did: I stared at an LLM model picker and wondered which one I was supposed to want.

OpenAI just released ⁠GPT-5.6 Sol, Terra, and Luna. Sol is the flagship. Terra offers much of its intelligence for less money. Luna is cheaper still. Anthropic released ⁠Claude Sonnet 5 at the end of June and Opus 4.8 the month prior, with a little Fable 5 emerging in between. Meanwhile, Google, which seemed to be winning the model wars a few months ago, is now getting shade from Gergely Orosz, who ⁠argues that Gemini has slipped outside the top tier for software development and has been out of the major model release game for eons (May 19).

Perhaps Orosz is right. Perhaps he’ll be wrong again in six weeks. Honestly, it’s exhausting.

I use ChatGPT and Claude constantly and still have no principled idea which model to choose most of the time. I tend to click whatever looks like the biggest, most expensive option because I don’t know what I’m giving up by choosing something smaller. “Instant” sounds dangerously unserious. “Thinking” sounds expensive but powerful.

A quick survey of my LinkedIn crowd suggests others also feel my “WHICH MODEL???” pain. More importantly, I suspect most enterprises do, too.

A model doesn’t rot

Before getting carried away, however, it’s worth considering whether any of this model churn actually matters. After all, a model doesn’t rot. The model an enterprise put into production in March performs just as well in July as it did when the company selected it. “Obsolete” generally means that something better now exists, not that the deployed model suddenly stopped summarizing insurance claims or classifying support tickets. (In other words, once you have something working, the idea that “but maybe Opus 200.2 is better!” is really a FOMO problem, not a performance issue.)

Most enterprise workloads don’t live at the frontier anyway. Extraction, summarization, classification, document comparison, and customer-service assistance often work perfectly well with smaller, cheaper models. OpenAI’s own pitch for the trio of GPT-5.6 models isn’t simply that Sol is better. It’s that ⁠Terra and Luna deliver different combinations of intelligence, latency, and cost. Luna, the cheapest tier, nearly matches the previous generation’s peak performance at less than half the estimated cost, according to OpenAI.

The practical question, of course, is where to start. An enterprise can’t test every model, every reasoning setting, and every price tier before doing any work. So here’s my advice (which I don’t follow in my own work, but I’m not defining enterprise strategy and can be a little price-insensitive). Start with the cheapest credible model that appears capable of the task. Give it a representative set of real examples and, before you start testing, define what counts as good enough. If it passes, stop. If it fails, move up a tier or try a model with strengths better suited to the work.

That sounds almost offensively simple, but it reverses the way many people, including me, use these products. We start with the biggest model because we’re afraid of what we might lose. Enterprises should start lower and require evidence before paying for more intelligence.

There are exceptions, of course. For genuinely difficult work, such as autonomous coding, complex research, or high-stakes reasoning, beginning with a frontier model may save time. But even then, the goal should be to establish a quality ceiling, then test whether a cheaper model can meet it. It’s changing the question from “which model is best?” to “what is the least expensive model that reliably clears the bar for this job?”

For many workloads, that price improvement matters more than a few extra benchmark points. ⁠As I argued back in 2023, following AI hype doesn’t help anyone. If your model strategy depends on whichever benchmark screenshot is circulating on X this week, you don’t have a strategy. Not a viable one, anyway. Pick a model and ignore the noise.

Except, of course, when that noise suggests a serious signal.

Sometimes better really is better

Frontier improvements aren’t always incremental, making it advantageous to consider an upgrade. Coding is the obvious example. There’s a significant difference between a model that suggests the next few lines of code and one that can inspect a repository, plan a change, use tools, run tests, discover its own mistakes, and keep working for an extended period. That isn’t merely a nicer autocomplete experience. It can reorganize a development workflow.

This is why enterprises can’t simply standardize on an 18-month-old model and declare victory. In some areas, particularly software development and other agentic work, better models can unlock compounding productivity. A model that reliably completes 80% of a bounded task rather than 50% may justify an entirely different division of labor between humans and machines.

Still, that upgrade isn’t free.

Models differ in how they interpret instructions, call tools, manage context, refuse requests, and fail. Prompts and scaffolding tuned for one model can regress when moved to another. Or costs can explode. As one of my Oracle colleagues discovered just this week, running the same tasks in GPT 5.6 was orders of magnitude more expensive than 5.5. The API change may be trivial, but the revalidation and implications are not.

This leaves enterprises caught between two bad options. They can freeze and potentially miss out on meaningful improvements or chase every release and repeatedly test production systems on faith. What to do?

Stop making model bets

The answer is to stop making LLM bets and start making job-to-be-done bets. Stop asking which model is fastest. Instead, figure out what work you are trying to improve. What does a good result look like? How much latency and cost can the workflow tolerate? How wrong can it be before a human must intervene? Once those questions have answers, model selection becomes less opaque.

A difficult code migration may justify GPT-5.6 Sol or Claude Sonnet 5. A repetitive classification task may work just as well with Luna or another smaller model. A regulated workflow may require a model or deployment option that offers particular data controls. Sometimes the correct model is no LLM at all, like when I’m writing this post. Sorry, AI vendors! (At least you won’t get blamed for my mistakes.)

This is where evaluations become the center of enterprise AI strategy. ⁠As I’ve said before, most companies don’t have an AI quality problem so much as an AI measurement problem. Hence, a private evaluation suite built from real company work is the only leaderboard that matters. Does the new model materially improve quality? If so, use it! Does it reduce cost or latency? Again, that’s your free pass to adoption. Does the improvement justify the expense and effort of revalidation? If yes, continue.

Make model releases boring

As important as the model is, keep in mind that AI success always comes back to your company’s data, your company’s workflows, your company’s integrations, etc. That’s the ⁠dull reality behind sexy AI. Retrieval, memory, governance, data quality, observability, and feedback loops aren’t as exciting as a new model launch, but they’re what ultimately make AI truly work.

Again, when it’s time to consider something new, the principle should be to default to the least expensive model that reliably passes your evaluations. Only escalate harder tasks to more capable models when measurement shows that the premium pays. Tip: Make this invisible to employees so that the system routes to the best model for a particular prompt. As dbt Labs’ Jon Lewis expresses it, “The best model is ‘Auto’ and I won’t hear anyone say otherwise.” OpenAI’s own ⁠migration guidance recommends testing models on representative tasks, including trying a lower reasoning level rather than automatically cranking everything to the maximum.

As for me, I’ll probably keep clicking the shiniest option. I don’t have a formal evaluation suite for InfoWorld columns, and the marginal cost is a subscription I already pay. Enterprises don’t get that excuse.

For decades, software engineering has been plagued by “productivity theater.” Every few years, the industry aligns around a new vanity metric — usually one that latches onto whatever technology happens to be in vogue at the time. For a discipline rooted in creativity and problem-solving, this is a poor way to demonstrate progress. Yet, we find ourselves in this position once again. The pattern is often the same: reach for something we can easily count, and in doing so, lose sight of what we are actually trying to achieve.

Quantity over quality: the wrong measurement, every time

I recall when I was coming up as a software engineer in the 1990s, a small number of companies took up the practice of paying their engineers by each line of code. This may have been productivity theater at its worst, leading to negative incentives, inefficient processes, and just generally bad engineering. Developers were rewarded for writing far more code than the problems they were facing required — classic “quantity over quality” — and the result was bloated, brittle codebases that were all but impossible to maintain. The goal — to create reliable software that solved real user problems — got buried under the incentive to produce.

Then in the 2000s, the rise of Agile brought us story points, an abstract way to estimate task complexity, effort, and risk relative to other work. Rather than answering “How long will this take?,” story points were meant to answer, “How big is this compared to what we’ve done before?” This approach sounds good in theory, but in practice, some development teams learned to game the system by inflating estimates, over-engineering solutions to look productive, and losing sight of whether the work they produced actually created value. Once again, the metric became the goal, and the actual goal — delivering outcomes that mattered to the business — became secondary.

Every one of these metrics failed for the same reason: they measured effort instead of value.

Quantity in the age of AI

Today, “tokenmaxxing,” a trend in which developers and teams optimize for consuming as many AI model tokens as possible, treats raw consumption as an equivalent for output. As I see it, this is the latest flawed productivity metric to make its way into the world of software engineering. Tokenmaxxing is nothing more than another vanity metric, and is just as useless as using “lines of code” or inflated “story points” as a benchmark.

Tokenmaxxing is the result of a few different behaviors, including:

• Prompt flooding: stuffing massive codebases, documentation, and context into every prompt, burning tokens on context the model doesn’t actually need.
• Agent swarms: running multiple AI agents in parallel to maximize code output, regardless of whether the work is coordinated or coherent.
• Background loops: keeping AI sessions or agents running continuously in the background, racking up token spend without clear ownership of what is being produced — or why.

Now, it is no secret that AI is reshaping how software is developed, and these behaviors are the result of that reshaping. Providing AI with codebases, running multiple agents at once, and even relying on coding assistants for help all have their uses. But when we lose control of the changes we are making and why we are making them, we find ourselves facing a new version of the same old problem: measuring engineering productivity with the wrong metrics.

A more useful question to ask isn’t, “How many tokens did we spend?” but rather, “What problem did we actually solve, and for whom?”

Spending resources without goals

Yes, AI is giving software engineers the ability to do more with less, to move quickly, and to experiment in ways that were previously out of reach. But leaning on AI to perform productivity, rather than deliver it, is a trap that will cost us in code quality, team capability, and business credibility.

As a CTO, I am all for experimenting with AI. I want to use it to make our programs better, stronger, and future-proof. What I don’t want is for it to drive us toward excess while leaving us with little to show for it.

The test I keep coming back to is simple: does this AI-generated output help us ship something that matters? Does it reduce friction for a user, close a gap in a workflow, or improve reliability for a customer? If the answer isn’t clear, then we are spending resources — both human and computational — without a defined goal. And that is not engineering. That is activity.

Spec-driven development: where value gets defined

It is time to adopt newer approaches like spec-driven development, a method where engineers write detailed specifications first and AI generates code against them. Rather than relying on prompt flooding and agent swarms and hoping AI produces the best result, we need to shift toward defining requirements, reviewing AI-generated output, and orchestrating systems with intent.

But spec-driven development is more than a methodology. It is the place where engineering intent and business value get defined together. The spec is where you answer, “Why does this matter, and what problem are we solving?” before a single token gets spent.

Software engineers have long taken pride in writing elegant code, and I would hate to see AI cheapen that pride rather than elevate it. In an AI-first world, the craft shouldn’t disappear; it should simply move upstream. The spec is where elegance lives now, and it deserves the same attention to detail we once reserved for the code itself.

At its core, software engineering is about defining, analyzing, and resolving technical challenges. If we are willingly giving all of that up to AI, we will lose the integrity of our discipline and the ability to prove our value. Using the maximum number of tokens to produce code isn’t impressive. Using a well-crafted, intentional prompt to solve a specific problem? That’s the work worth celebrating.

Stop performing productivity and start delivering it

We are at an inflection point. Many organizations are defaulting to activity-based metrics, measuring how much AI is being used rather than whether it is improving delivery, product quality, or business outcomes.

The question worth asking is not, “How much AI did we use this sprint?” It is “What value did we deliver for our users, our team, or our business?” Was it the ability to resolve a critical bug more quickly? Reduced cycle time on a high-value feature? A customer workflow that now takes minutes instead of hours? Those are outcomes. Those are the things worth measuring.

AI can help us deliver meaningful outcomes faster, but only if we use it with the same rigor and intent we expect from every other engineering or business decision. Don’t let it become another form of productivity theater. The most successful engineering organizations in the age of AI won’t be the ones that consumed the most tokens, they’ll be the organizations that never lost sight of why they were building in the first place.

—

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.

The first time I heard someone say their AI now wrote 80% of their tests, I asked the obvious question. Eighty percent of what?

After 20 years building and leading test automation for consumer-scale platforms, my honest answer turned out to be eighty percent of the typing, not eighty percent of the engineering. The remaining twenty was where the work still lived. Budgeting for two percent of leftover effort was the mistake. When the real number was closer to thirty, that gap was the difference between a pipeline that shipped and one that quietly built up a queue of half-trusted features nobody could rely on.

This piece is about that gap. As an independent research project on LLM-augmented testing methodology, I built a six-stage agentic pipeline that takes a design in Figma and produces running tests in WebDriverIO, connected end to end over the Model Context Protocol. It works. It has been useful. And the parts that broke surprised me, because they were not the parts the hype cycle tells you to worry about.

How I wired a six-stage pipeline over one protocol

The pipeline runs six stages in sequence, each owned by a different agent, with every handoff crossing MCP.

Six-stage agentic test pipeline: design capture → requirements writer → ticket opener → code generator → test-case writer → automation generator. Each stage carries an MCP handoff and a provenance stamp.

The end-to-end trace links a pull request back to a Jira ticket, a requirements section and a Figma frame. Each artifact is stamped with the agent that produced it, the model it used and the inputs it was given.

MCP is the boring middle that makes any of this work. The cliché is that MCP is “USB-C for AI”: one open protocol, any tool. Like most analogies, it is about eighty percent right. The part that matters is the eighty: I do not have to write a custom adapter for every system the agent talks to. One MCP server per tool and every agent talks to all of them the same way.

Typed handoffs between agents are my own architecture, layered on top of MCP rather than provided by it. Each agent writes a typed artifact the next agent reads. Each handoff is logged with provenance. When something went wrong six stages in, I could replay the chain. Without that discipline, a multi-agent pipeline is a debugger’s worst day. You know the test plan is wrong. You cannot tell whether the mistake came from the Figma read, the requirements interpretation or the ticket scaffolding. With it, I could point at exactly which stage went sideways and which inputs it was looking at when it did. The pattern lives in a public MIT-licensed reference implementation for any reader who wants to run it.

The sixteen-minute number is the marketing number. I ran the full chain end to end in about sixteen minutes on a synthetic net-new screen, Figma in, automation suite out. That repeated across my runs; it is not a demo trick. But sixteen minutes is the part of the story most fun to tell and least useful to learn from. It is what gets quoted in the all-hands. The hours that come after, when a human reviews each handoff, are where the work actually lives.

What actually broke in production-style runs

The failures that stalled my pipeline were rarely the ones I expected.

I expected hallucinated APIs. I got them: the agent confidently called endpoint names that sounded right but did not exist. I expected sparse-spec-in, sparse-spec-out, where a Figma frame with no annotations produced a requirements doc with vague acceptance criteria, every time. I expected locator drift, the common UI-automation failure mode where a renamed component silently breaks an entire test suite. There is solid outside writing on non-determinism in tests covering this whole family of failure modes, and the agent inherited every one.

What I did not expect, and what kept the pipeline down longer than any of the above, was the plumbing.

The model backend timed out under load. It lost credentials silently and started returning empty strings, which the agent then read as confidence. A duplicate consumer on a shared long-poll API endpoint produced an HTTP 409 conflict that broke delivery without throwing anything visible. One unguarded exception inside one agent aborted a whole shared scheduler run and took the other agents in the registry down with it. The single worst incident cost me three hours to find. An environment variable had silently rotated overnight; every agent in the fleet was returning structurally valid but semantically empty requirements docs; the downstream stages were dutifully generating tests against nothing.

None of those are model bugs. They are infrastructure. The agent literature, which is what I went looking through when I started this work, mostly does not talk about them.

The fix was not better prompts. It was circuit-breaker-style review checkpoints between stages and what I now call the four-guard discipline: four small guards I consider non-negotiable on any unattended agentic pipeline. The bulkhead pattern from microservices is the most consequential. An unhandled exception inside one agent can no longer abort the shared run; the offending agent fails fast with a structured error and the others keep going. Paired with that, a pure-data fallback ensures a model timeout produces a deterministic output explicitly marked as degraded mode, rather than an empty string the next stage will misread as confidence. A single-owner lease sits on every shared external endpoint, the cure for the duplicate-consumer incident that ate one of my Sunday afternoons. The cheapest guard was the last to arrive: a one-line synthetic canary every agent has to produce a known correct response to before any real work begins, so a credentials rotation or silent backend failure trips an alert before downstream stages have generated artifacts against garbage.

None of these guards is novel. They are textbook stability patterns at a new boundary: the seam between the LLM agent and the rest of the system, which most of the existing agent literature still treats as a solved problem.

The 20% you don’t see, and when not to do this

Here is the part the demo videos leave out. Even when the pipeline works, the human time per stage does not go to zero.

Human review time per ticket across five pipeline stages: code review 60-180 min, automation review and flaky-fix loop 30-90 min, ticket architecture and sequencing 30-60 min, test data and environment 15-30 min, requirements review 20-30 min. Net: the human still spends 20-30% of the original effort, almost all of it reviewing rather than creating.

Net of all that, the human still spends twenty to thirty percent of the original effort, almost all of it reviewing rather than creating. The pipeline saves seventy to eighty percent, not ninety-eight. The trap is budgeting for the two percent you do not save.

When does this kind of pipeline make sense? In my experience, when the Figma is richly annotated and acceptance criteria are clear up front; when there is review capacity to absorb the work the pipeline shifts onto humans; when the stack is well represented in the training data; and when the feature is net-new rather than a deep edit of legacy code. When does it not? When the design lives on a whiteboard. When the integration touches old code with hidden contracts. When the path is regulated or safety-critical. When there is no senior reviewer who can hold the line. When the work is exploratory and writing the spec is the actual point of the exercise.

Teams I have seen succeed with agentic pipelines budget for the rework explicitly, staff the review queue and treat the saved hours as capacity for harder problems rather than headcount they can release. Teams I have seen struggle did the opposite: declared victory at the demo and quietly accumulated a backlog of half-trusted features the next quarter had to clean up.

The right unit of measurement is not how much the pipeline generates. It is how much of what it generates a human still has to touch before you would ship it. Call it the 80/20 rework rule: measure the rework, not the generation. The teams that get the rework number right are the ones whose AI investments compound. The teams that stop counting at the headline percentage are the ones that own the cleanup six months later.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

The White House is expanding the use of AI beyond cyber threat detection into vulnerability management, launching a new program that aims to help government agencies and critical infrastructure operators identify, prioritize, and remediate software vulnerabilities faster.

Called Gold Eagle, the initiative will act as a centralized clearinghouse for cybersecurity vulnerabilities, coordinating vulnerability reporting, verification, and remediation across federal agencies, open-source software communities, and operators of critical infrastructure, the White House said in a statement.

“This new model will leverage frontier AI capabilities to continue advancing faster than adversaries, reduce duplicative scanning efforts, and deliver prioritized and actionable threat and remediation information to defenders across the Federal government and the private sector,” the statement added.

The initiative stems from President Donald Trump’s June 2 executive order on advanced AI innovation and security, which directed federal agencies to expand the use of frontier AI to strengthen cybersecurity while working more closely with the private sector.

The administration said the program has already begun receiving vulnerability reports from multiple industries and coordinating validation and remediation efforts.

For enterprise security leaders, the announcement signals a government effort to move beyond traditional vulnerability disclosure toward coordinated vulnerability response.

A move toward coordinated vulnerability response

Prabhjyot Kaur, senior analyst at Everest Group, said Gold Eagle should be viewed as “a significant evolution” of existing vulnerability disclosure and government-industry coordination mechanisms rather than a replacement for them.

“Its potential significance lies in creating a more operational clearinghouse that can consolidate vulnerability findings, reduce duplicative scanning, validate exposure across sectors, and coordinate remediation with critical infrastructure operators and open-source software communities,” Kaur said.

The more meaningful shift, she said, is from largely distributed vulnerability disclosure processes toward centralized prioritization and coordinated action. Whether the initiative changes enterprise vulnerability management, however, will depend on execution, including industry participation, information-sharing protocols, and whether it can shorten the time between vulnerability discovery, validation, and remediation.

The White House said Gold Eagle has already begun receiving and prioritizing vulnerability reports from multiple industries, coordinating scanning verification, and supporting remediation efforts using existing federal authorities and resources.

AI can accelerate prioritization, not replace judgment

The administration said the initiative is designed to help government and industry reduce duplicative vulnerability scanning and accelerate remediation by using AI to prioritize findings.

Treasury Secretary Scott Bessent said the program reflects closer collaboration between the government and the private sector to protect financial institutions and other critical infrastructure.

“Treasury, along with our partner agencies, will continue to harness frontier AI capabilities to stay ahead of our adversaries and defend the American people from emerging threats,” Bessent said in the statement.

Kaur said AI is likely to deliver the greatest value in vulnerability triage and prioritization.

“It can correlate findings from multiple scanners, remove duplicate alerts, link vulnerabilities to known exploitation activity, assess internet exposure, and combine technical severity with asset criticality and potential business impact,” she said.

However, she cautioned that AI-generated prioritization is only as reliable as the underlying asset inventories, vulnerability data, and threat intelligence.

“AI should therefore support, rather than replace, human validation, compensating-control analysis, and enterprise-specific risk decisions,” she said.

Apeksha Kaushik, senior principal analyst at Gartner, said the initiative reflects a broader shift toward measuring cybersecurity performance by reducing actual risk exposure rather than simply increasing patch counts.

By helping unify and accelerate vulnerability coordination between government and industry, the initiative could address long-standing challenges around fragmented reporting and inconsistent disclosure practices, enabling enterprises to respond more quickly and efficiently to vulnerabilities, she said.

Execution will determine enterprise impact

The announcement outlines Gold Eagle’s objectives but provides few operational details about how organizations will participate, how AI will validate or prioritize vulnerabilities, or how the initiative will work alongside existing coordinated vulnerability disclosure and vulnerability management programs.

Kaur said CISOs should view the initiative as an additional source of vulnerability intelligence rather than a replacement for enterprise risk management.

“The biggest takeaway is that vulnerability response is moving toward faster, more intelligence-led, and more coordinated prioritization across government and industry,” she said.

Even if government coordination improves the quality and timeliness of vulnerability intelligence, enterprises will continue to own remediation decisions, Kaur added. “Government coordination may improve the quality and timeliness of intelligence, but enterprise context must continue to determine the final remediation priority.”

Security engineers play a pivotal role in enterprise cybersecurity, because they are the professionals who design, build, and deploy security systems to protect an organization’s data, applications, systems, networks, and other IT components against a variety of cyber threats.

Finding not just qualified security engineers, but the best and brightest available, needs to be a priority for CISOs and others overseeing security at their organizations. That’s especially true with the rapid rise of AI and the threats that brings to the enterprise.

Here are some of the key skills and traits of elite security engineers to look for when hiring — or to acquire in order to uplevel your cybersecurity career.

Acumen with AI-powered tools

These days, AI-related skills are in demand regardless of domain, and this certainly applies to security engineers. There’s a wealth of solutions leveraging AI in the market, tools that engineers can add to their defense arsenal.

“AI is transforming security engineering from reactive alerting to predictive threat detection,” says Praveen Margabandhu, digital engineering anchor at financial services firm Navy Federal Credit Union. “AI-driven anomaly detection now identifies behavioral patterns that indicate fraud or compromise before traditional threshold-based systems would fire. This shifts the security engineer’s role from incident responder to threat model designer.”

AI-powered tools have taken over a large portion of the detection and triage work that used to be the core of a security engineer’s day, says Maruf Ahmed, cofounder and CEO of global tech staffing firm Dexian. “Vulnerability scanning runs on its own now,” he says. “Threat flagging that used to require a team pulling through logs for hours happens in minutes.”

This has freed up capacity on most security teams and changed what the day-to-day work looks like, Ahmed says. “With detection increasingly automated, the engineer’s value sits more in interpreting what gets flagged and deciding what to do about it,” he says.

Keen understanding of emerging and established AI threats

Engineers must also have a thorough understanding of the risks AI presents, including AI-enhanced cyberattacks using large language models (LLMs) to automate and scale highly personalized social engineering attacks, craft sophisticated malware, and generate deepfakes.

Other AI threats they need to be aware of include prompt injections, data and model poisoning, disclosure of sensitive information, model theft, supply chain compromises, and excessive agency.

“The same generative tools that help security teams work faster are available to adversaries, and it shows,” Ahmed says. “Phishing campaigns read better and land more precisely than they did a year ago. Social engineering is harder to catch when the language is polished and tailored to the target, and security engineers are now defending against threats built with the same class of technology they use on the defensive side.”

That has raised the bar for what reliable detection looks like, Ahmed says. “The objective shift I hear most from clients is about trust in their own systems,” he says. “Two years ago, the priority was visibility — making sure you could see across your environment. Most organizations have that now. The harder problem is knowing whether what those tools are telling you holds up under scrutiny and having people on the team who can stand behind those findings in front of a regulator or a board.”

Appreciation of performance and business goals

The best security engineers understand how performance and security intersect, says Margabandhu, who leads performance engineering across Navy Federal Credit Union’s digital banking infrastructure, including real-time fraud detection, identity and access management, and cybersecurity infrastructure resilience.

“A fraud detection system that is secure but too slow to catch transactions in real-time is not secure at all,” Margabandhu says. “Elite engineers optimize for both simultaneously.”

Engineers must be able to put things in business context, Ahmed says. “An engineer who can work across domains, validate AI outputs, and learn new tools fast is valuable. But that value compounds when the person also understands what the organization is trying to protect and why,” he says.

Security engineers who understand the business make better risk decisions, write more effective policies, and generate less friction with the teams around them, Ahmed says. “That is the profile employers are hiring toward right now, and it is where the talent shortage is most pronounced,” he says.

Systems mindset

“One of the biggest misconceptions in cybersecurity hiring is that elite security engineers are defined purely by technical certifications or tool familiarity,” says Juan Mathews Rebello Santos, an independent cybersecurity researcher and ethical hacker.

“Technical skill absolutely matters, but the strongest engineers I’ve worked with consistently share a combination of analytical thinking, operational adaptability, communication ability, and deep systems understanding,” Santos says.

Elite security engineers understand how infrastructure, cloud services, identity systems, applications, APIs, networks, users, and business operations connect, Santos says.

“Modern attacks rarely target a single isolated component anymore,” he says. “Threat actors chain together weaknesses across environments. Engineers who can understand those relationships holistically are significantly more effective at both prevention and incident response.”

Cross-disciplinary fluency and broad stack know-how

Being an elite software engineer today means having a range of technology experience and knowledge. “Organizations want engineers who can work across more of the stack than they used to,” Ahmed says. “A role that might have asked for deep specialization in one area now expects someone who can move between cloud infrastructure, application security, and compliance without needing a handoff at every boundary.”

The attack surface has continued to get wider, and the job descriptions for security engineers has followed suit. “That cross-domain fluency matters because security incidents rarely stay contained in one layer,” Ahmed says. “The engineer who can follow a problem from the network through the application to the data governance framework resolves it faster, with fewer people involved.”

The strongest security engineers bridge infrastructure, application, and business domains, Margabandhu says. “They can speak to a CISO, a developer, and a cloud architect in the same conversation,” he says. “An engineer who can explain what an authentication problem means for fraud exposure moves faster in a room full of executives than one who can only describe it in infrastructure terms. I’ve watched technically brilliant people lose that race repeatedly.”

Having the ability to communicate technical risk clearly to non-technical leadership can mean the difference between success and failure of attacks.

“Many security failures today are not caused by lack of tooling, but by misalignment between technical teams and business decision-makers,” Santos says. “Elite engineers can explain operational risk, prioritization, and security tradeoffs in language executives understand.”

Deep understanding of third-party risk and non-human threats

Threats can come from anywhere, including supply chains and non-human combatants. Third-party cybersecurity risks are on the rise. The 2026 Global CISO Leadership Report by executive search firm Hitch Partners, based on a survey of more than 625 information security executives across the US and Canada, says 43% put third-party risks as the No. 1 priority.

“Most teams are still better at securing what they own than securing what they depend on,” Margabandhu says. “The mental shift from perimeter thinking to dependency thinking is real and not everyone has made it. The engineers who treat every API call, every credentialed vendor, every third-party model as part of their attack surface approach design differently.”

Another growing source of potential threats are not human. Machine identities now outnumber human identities by ratios exceeding 100 to 1 in most enterprise environments, with some sectors closer to 500 to 1, according to the ManageEngine Identity Security Outlook 2026 report.

This includes service accounts, API keys, automation tokens, and AI agents, any one of which can present data governance and security risks.

Many organizations are still managing machine identities through manual processes that weren’t designed for scale, Margabandhu says. “Engineers who understand non-human identity governance are rare and increasingly important. This is not a future problem.”

Willingness to keep learning

Security engineers need to have a desire to never stopped learning.

“That sounds obvious until you work with people who’ve been doing this for 15 years and are still operating from the same threat models they built in 2012,” Margabandhu says. “Security changes fast enough that standing still is the same as going backwards.”

The security engineers who keep up aren’t reading one report a year. “They’re genuinely curious about what attackers are doing right now, this month, and they adjust how they think accordingly,” Margabandhu says. “That quality is harder to hire for than most technical skills, because it’s not on a resume.”

With AI presenting new and more sophisticated threats, keeping up with the latest developments is perhaps more important than ever. “Strong engineers are naturally investigative,” Santos says. “They actively study attack techniques, test assumptions, reverse engineer failures, and continuously adapt their understanding of risk.”

The best security engineers are often the people who remain intellectually uncomfortable because they know the landscape is always evolving, Santos says.

Employers have started paying closer attention to how fast someone can learn, Ahmed says. “The threat landscape and the defensive toolkit are both moving faster than any certification program can track, so hiring managers are probing for adaptability in interviews: how candidates have responded to recent shifts, whether they have picked up unfamiliar platforms on their own, how they work through problems they have not seen before,” he says.

Venture-backed firms are buying up the support, finance-ops and managed-services providers enterprises rely on and re-platforming them around AI agents — and the renewal that follows arrives priced per outcome, sold as your advantage. The acquisition-built structure and unproven stability create governance and continuity risks your vendor process isn’t sized for. Here’s how to keep the leverage on your side of the table.

The first time an AI-native services pitch crossed my desk, I nearly signed it. The savings were real, the agents demoed cleanly and the pricing was the kind procurement loves — per resolved case, not per seat. What I almost missed was who got to define the word “resolved.” On an earlier outsourced-support arrangement, back in my public-sector days, the vendor’s reported resolution rate looked excellent right up until we pulled the reopen numbers ourselves. Auto-closed tickets had been counted as wins. Users had quietly stopped logging issues at all. The dashboard was green; the service was not.

That gap — between the number on the contract and what your users actually live with — is the whole game now, and it is about to scale across your portfolio. Gartner’s 2026 CIO and Technology Executive Survey found only 17% of organizations have deployed AI agents, but more than 60% expect to within two years — the steepest adoption curve of any emerging technology it tracks. The providers running your services are moving first, and the contracts are changing faster than most of us can govern them.

The agents underneath these pitches are also nowhere near as reliable in production as they look in the room. Gartner expects more than 40% of agentic AI projects to be canceled by the end of 2027 on cost, unclear value and weak risk controls, and reckons only about 130 of the thousands of self-described agentic vendors are the real thing — the rest are “agent washing” old chatbots and RPA. Treat any headline resolution rate the way you treat a vendor’s own uptime stats: Marketing, until you have seen it run on accounts like yours.

What’s behind the pitch

The challenger’s economics aren’t magic. They come from a reshaping of the services market: Venture-backed firms buying up fragmented, labor-heavy providers — support desks, contact centers, AP and finance ops, slices of managed IT — and re-platforming them around agents. Many of the companies pitching you are not one company at all but several acquired shops stitched onto a shared AI layer. That barely comes up in the sales meeting. It matters enormously once you are the customer.

The direction is independently corroborated, not vendor hype. Everest Group reports outcome-based pricing in business-process services moving from pilots to scaled adoption as AI makes outcomes measurable enough to contract on, with the binding constraint now governance and verified baselines. HFS Research tracks the same “services-to-software” shift across consulting, IT, and operations providers. Here is the part worth holding onto: Most enterprises are early, so you have a little time. But the first vendors to reprice you this way will be the small, single-source ones in the long tail of your portfolio — which, if your stack looks anything like mine, is most of it.

Don’t assume the incumbent is the safe choice

And don’t kid yourself that renewing with the familiar name keeps you clear of this. The big integrators are pulling labor out of their own delivery just as fast — Accenture cut tens of thousands of roles and rehired against an AI-skills filter — and rewriting deals around a share of savings instead of time and materials. Outcome pricing is becoming the default everywhere. There is no version of this where you sit it out.

Two risks your vendor process won’t catch

The first is governance, and the roll-up structure makes it worse than the usual AI-vendor worry. The company you are contracting with isn’t one system. It is several acquired firms with different data practices and security postures, with an AI layer dropped on top at speed. Your customer records, invoices and support transcripts flow into agents whose decisions you often can’t trace, across entities that were never built to one standard. The numbers here aren’t comforting: IBM’s 2025 Cost of a Data Breach Report found 63% of breached organizations had no AI governance policy at all, and 97% of those that suffered an AI-related breach lacked basic AI access controls — AI adoption, IBM concluded, is outpacing both security and governance. When an agent botches a dispute or misroutes regulated data, the regulator and the customer come looking for you, not the platform. And here is the organizational trap: The savings line is what your CFO signs; the provenance question is the one your audit committee won’t ask until after the incident. Nobody raises it for you.

The second is whether the provider will still be standing in three years. These platforms are new, built by acquisition, venture-funded and not one has run through a full contract term or a real downturn. With Gartner expecting more than 40% of agentic AI projects to be canceled by 2027, putting core operations on a young, agent-dependent vendor is a single point of failure dressed up as innovation. Write the exit and data-portability terms before you sign — while you still have leverage to.

Six questions for the renewal

When the challenger shows up — or your incumbent reprices to match — these are the questions I would put on the table. They come down to one thing: Making sure you, not the vendor, own the number.

1. Definition, baseline, guardrails. Make “resolved” mean what your users experience, measured against a baseline you have captured yourself, and tie it to metrics you own — first-contact resolution, reopen rate, time-to-resolution. Expect procurement to resist because pinning this down slows the deal. Hold the line; it is the whole ballgame.
2. Real agent, or agent washing. Gartner reckons only a sliver of self-described agentic vendors are the genuine article. Make them prove it: Production resolution on accounts like yours, and the human-escalation rate sitting behind that number. Not a demo.
3. Auditability, as a gate. SOC 2 at minimum, increasingly ISO 42001 or NIST AI RMF alignment, plus model cards, decision logs and an incident-response plan they have actually tested. If they can’t show how data is walled off between their acquired entities, or how an agent’s decision gets traced, they aren’t ready for anything regulated. This belongs in the shortlist criteria, not the post-mortem.
4. Where autonomy stops. Decide which actions an agent can take alone and which need a human, how it hands off with context and who is accountable when it acts on its own. Put names against it before go-live.
5. The exit. An embedded agent platform gets stickier than the staffed incumbent it replaced, faster than you would think. Lock down data portability, knowledge-base ownership and a way out while you are still the one with leverage.
6. Capacity, not just cost. The best outcome here often isn’t a smaller bill. It is the demand that your old service levels were quietly turning away. Nobody answered the tickets. The cases that aged out. Ask what fixing that is worth before you optimize purely for headcount.

The move

Outcome pricing is where this lands, and on balance, that is progress. But in the near term, it hands the advantage to whoever can measure the outcome — and in most shops, that isn’t the buyer. The edge isn’t picking the cleverest challenger or the safest incumbent. It is being able to hold any of them to a result, on your numbers. Look again at why Gartner thinks so many of these projects die: Not the technology — cost, fuzzy value, weak controls. Our side of the table. So, start there. Take one high-volume, measurable workflow, pilot it against a baseline you own, instrument it with your own metrics, and treat the muscle you build doing that as the real deliverable. Get it right and the pricing model stops mattering. Skip it, and you have just agreed to pay for someone else’s definition of done.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

A few weeks ago, a new hire at Aviator, fresh out of college, asked me a question I didn’t have a clean answer to. How do I become a senior engineer, or even a staff engineer? What should I learn, and how?

It’s a fair question and a harder one to answer than it was just a year ago.

The path used to be well-known. As a newly hired junior software engineer, you were given an experienced mentor who would assign you simple tasks to learn the ropes. You’d write some code, ask plenty of questions, open a pull request, get feedback in code review, think about it and fix your code. Rinse and repeat that a few hundred times. The tasks became more complex; the feedback got shorter and along the way you’ve been building judgment, the thing that separates a senior engineer from a junior one.

Now AI writes most of the code. The loop looks different and the easy assumption is that it’s broken: fewer tasks for juniors to cut their teeth on, an agent fixing the code that another agent wrote, thinner path to judgment.

Mentoring got easier, not harder

I knew just the person to ask: how do we grow senior and staff engineers in the AI era? Adam Berry is a staff engineer at Netflix, a member of The Hangar, our community of engineering leaders, and someone I have been discussing the evolving role of code review and knowledge sharing for a while now. He spends his time on getting AI adoption right, rather than just fast, in their engineering organization and has been working out the question of growing new engineers in practice. Mentoring juniors in an agentic world, Adam says, isn’t harder; it’s embarrassingly easy.

“You grow juniors and help them become better engineers the same way you always did. AI isn’t changing the methodology. It’s changing the details,” he told me. His point is that the agentic world gives a lot more options for giving juniors bounded tasks to work on and more feedback. The scattered remarks and comments seniors used to give in person now can be put into instructions and guardrails.

Adam breaks working with agents into three foundational skills:

• If you don’t know how to do something with the agent, ask the agent.
• If the agent does something you don’t like, figure out how to correct it and then codify it so it doesn’t happen again.
• Your sense of when the agent has gone off the rails.
  
  

“Most juniors can pick up the first two on their own. On the third one, they need guidance, he says.

His process for building it is staged. “The stages are about growing scope. First, you give a junior engineer a well-specified task—and these are now bigger than what you’d have given a junior before. You can give them task definitions that are like a prompt and instructions to drive that prompt, make sure they got to a good plan, make sure they understood the plan, and that they thought through the test cases, etc.

Then gradually you peel off some of that specificity so they have to build the muscle themselves. Once they’ve gotten good at that level of scope, they’re ready to work on larger scoped problems.”

Starting from more specific problems and going towards ambiguous problems is the definition of growing as an engineer.

Pair programming with the agent in the room

Seniors can still do pairing sessions with juniors, now with the agent in the room.

“In the pairing session, the earlier-career engineer should be the one driving. The agent can be set up to interrogate the junior rather than just answer them. None of you is manually writing code, but you’re still doing pair programming and mentoring. Even if it’s just a trivial bug fix, if you guide a junior through it, it forces them to do just that little bit of thinking.”

Adam says mentoring juniors today does not have to mean forcing them to write code manually. Seniors should teach them the process of agentic engineering, and that’s exactly what they should focus on during the pairing sessions. The habit he wants to be installed early is asking for options instead of answers.

“I aim to teach juniors to ask for options and think through them, even on small tasks. I ask them to explain what their input to the AI tool was that led to the code they got. But I’d also show them how I would have done the same thing.”

The pairing produces artifacts as it goes. “That’s where you get into conversations of, ‘This is why that wasn’t quite it for me,’ and if I see that that’s not baked into the repo, I’m going to add this into the ADR, into the design, into the instruction set. I’ll codify that so the junior gets it too, and they know why it exists, because they watched me go through it with the tool myself.”

That reshapes the code review instead of removing it. Making that work puts more on senior engineers, not less. “Senior engineers need to ensure that things like ADRs, or whatever system you use, are properly encapsulated in the repo for both the agents and the humans to consume.” His team also attaches the prompts to the pull request and has the agent summarize what it did against what the prompt asked.

Adam also teaches junior engineers how to bring in expert sources from outside into AI tools. He’ll point an agent at a book like Michael Feathers’ Working with Legacy Code as an example of what quality code looks like and have it work from the concepts directly.

Don’t outsource the thinking

His arguments make sense, but I also recently came across research examining the influence of AI tools on how and why members of software engineering teams interact. Their finding was not surprising: of the 131 surveyed developers, 51% said they now ask GenAI for technical help they once would have asked a person, and 62% said it was easier to ask GenAI without fear of embarrassment.

When a junior gets stuck now, their first move usually isn’t to message a senior on Slack. It’s to ask the agent. This is also the case in code reviews. The purpose of code reviews was always knowledge sharing as much as it was a quality gate. What I see junior engineers do now is take the feedback and, without reading it closely, hand it straight to the agent to resolve. The part where they would have to understand how their work differed from what the senior expected disappears. It got passed from the reviewer to the agent, and the junior skipped the understanding.

This isn’t really the junior’s fault. They need the motivation and the space to do it differently, and the default pattern under delivery pressure is to push the thing out and worry about it later.

The same research showed that developers turned to colleagues with questions about context (65% to clarify business logic or requirements, 50% for how something had been done before).

Respondents were also aware of the trap that Berry’s approach was created to avoid: AI tends to hand back a single answer, compared to multiple perspectives a colleague surfaces and they kept seeking teammates for context-specific expertise, mentorship and plain social connection.

The fix is almost mechanical: if you have to make a choice, you have to think about it. I do this in my own product thinking. Most of it happens by bouncing ideas off Claude, but whenever something is complex, I make myself lay out a few options, trade them against each other and decide which direction to take. That’s the same move Berry wants juniors making, and it’s what builds judgment, whether you’re twenty-two or forty.

Why we should still hire juniors

There’s also a hiring question underneath all of this. We recently hosted Kent Beck, an industry legend, at the Hangar in a session we called “Juniors FTW,” and his reasoning was that the industry is being remade fast enough that being new is an advantage. Juniors are too new to have absorbed what everyone “knows” is impossible, which leaves them less biased, more creative and carrying fewer preconceived mental barriers.

Beck also wrote about how important it is to hire juniors without the calculation of how many tasks they can perform.

“If all we cared about was today’s productivity, we wouldn’t have hired you at all. Instead, we (the seniors) are focused on the future: we know there’s going to be far more work here than we could possibly accomplish. We are paying your salary now as the option premium on the engineer you will become. If we play this game right, we’ll have a kick-ass next generation of engineers. If not, we’ll have to be doing the same engineering jobs ten years from now, and we really don’t want to be doing that.”

The pipeline is thinning

The trend is running the other way. Entry-level hiring at the 15 biggest tech firms fell 25 percent from 2023 to 2024, according to a report from SignalFire. In a recent survey of engineering leaders, a majority said they plan to hire fewer juniors, on the logic that AI lets seniors cover more ground.

That logic is short-sighted in a specific way. Senior engineers don’t appear from nowhere. They’re the juniors someone hired five or ten years ago and invested in mentoring them. Stop hiring and growing juniors now, and the gap doesn’t show up this year. It shows up later, when the industry needs people with the judgment that only comes from years of making mistakes and recovering from them and finds it stopped producing them.

So, here’s the answer to the question that the new hire asked: the path to senior and to staff is the same path it always was. You grow the range of ambiguity you can handle, and you stay honest about the part you can’t handle yet. What changed is the interface. The agent writes the code. Your job is to keep asking questions to your colleagues and the agents and keep doing the thinking until the thinking is good.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Seventy-one percent of organizations say AI has access to core business systems. Only 16% govern that access effectively, according to the 2026 CISO AI Risk Report. Ask your IR team three questions: Where is your AI system inventory? What happens if a production model starts generating harmful outputs? Who has the authority to take it offline?

I’ve spent 14 years in security — energy, banking, telecom, manufacturing. Red team work, detection programs and the last several years focused on AI risk and ShadowAI. What I see consistently: Organizations have AI in production, they have an IR playbook and they think those two things are connected. They’re not.

The CISO who thinks their IR playbook covers AI incidents probably hasn’t tested it. The ones who have tested it know it doesn’t.

Two kinds of AI incident — and why that split matters more than the list

AI incidents surged 56.4% from 2023 to 2024, reaching 233 documented cases. Most IR frameworks — including NIST SP 800-61, MITRE ATLAS and the GLACIS AI Incident Response Playbook — provide you with a taxonomy of six incident types and stop there. While useful, it misses the more important split: Failures the model causes on its own, versus failures caused by a human. Your detection approach, your containment logic and your legal exposure are very different between those two groups.

Model-originated failures — degradation, bias, hallucinations — happen when the system does exactly what it was built to do, just badly. The Epic Sepsis Model, deployed across hundreds of US hospitals, had a sensitivity of only 33% at external validation. It missed two-thirds of actual sepsis cases and flooded physicians with false alerts, as a 2021 JAMA Internal Medicine study found. No one attacked it. It just quietly stopped working while every dashboard stayed green.

Externally induced failures — adversarial attacks, data poisoning, privacy breaches — happen when someone corrupts the inputs or the training environment. Tesla’s Autopilot phantom braking cases, investigated by NHTSA across hundreds of thousands of vehicles, show what adversarial input failures look like in a safety-critical system. These two groups need different primary defenses and their own playbooks.

Then there is the hybrid case, which carries the most legal exposure right now. Hallucinations are model-originated but they land in court like human errors. When Air Canada’s chatbot invented a bereavement fare policy, the airline was held liable. When a US federal court let Mobley v. Workday proceed, it accepted that an AI hiring platform could be directly liable as an ‘agent’ of the employers using it. Neither failure looked like a security incident. Both ended up as legal ones. If your legal team is not on your IR call tree, your playbook is already incomplete.

The CIA triad doesn’t cover a hallucination

The CIA triad — confidentiality, integrity, availability — does not apply to most AI incidents. When Air Canada’s chatbot made up a policy, nothing was unavailable, nothing was changed without authorization, nothing was disclosed. The framework simply doesn’t reach it. When the Epic Sepsis Model missed two-thirds of cases, there was no breach, no intrusion, no indicator of compromise. By every traditional IR metric, the system looked fine.

This is not an edge case. Classical IR frameworks assume deterministic failures with static indicators of compromise — an assumption that breaks down against probabilistic systems. Microsoft’s Security Blog said it well in April 2026: A model may produce harmful output today and something completely different from the same prompt tomorrow. The root cause is not a line of code. It is a probability distribution, and as Microsoft’s Security Blog put it, you cannot patch a probability distribution.

The numbers confirm the gap. Average AI incident detection time is 4.5 days. Sixty-seven percent of AI incidents come from model errors, not adversarial attacks — yet security budgets keep funding perimeter tools built for the latter. We are looking for the wrong signals, with the wrong tools, for the wrong failure modes.

What a mature AI IR capability looks like

I get asked this at every conference I speak at. Here is the short answer: Three things that mature teams have in place before any incident occurs.

First, an AI Bill of Materials (AIBOM) for every production system. Think of it like a software SBOM, but for AI: It documents the base model, training datasets, third-party dependencies and the full component stack. Without it, you don’t know what your AI is made of — and you can’t investigate a data poisoning incident or a supply chain compromise without that baseline. The OWASP GenAI Security Project released an open-source AIBOM generator in December 2025 that produces output in CycloneDX format aligned with SPDX standards. It is practical to implement now.

Second, a model card for every production AI system — not a document in a shared drive nobody opens, but something your IR team can pull up in the first ten minutes of a response. Training data provenance. Model version. Known performance limits, including which subpopulations showed weaker accuracy in testing. Access controls. Blast radius if it fails. Most organizations I work with have model documentation written for data scientists that no one in security can use at 2am. That is not documentation. That is liability.

Third, a named data scientist on the IR call tree. Not someone to brief after the incident — someone with authority to interrogate model behavior in real time. Traditional IR has a network engineer on call. AI IR needs the same logic applied to the people who understand how the failing system works.

A fourth thing that very few teams have: A documented rollback threshold for each deployed model. A pre-agreed definition of what anomaly rate, drift metric or fairness deviation triggers containment or a fallback switch. Teams without this spend the first hours of an AI incident debating whether what they are seeing is actually a problem. Teams with a threshold spend those hours responding.

Four things to do before the next incident

Rewrite your detection triggers. Output anomaly scoring, data distribution monitoring for drift and behavioral tracking of model API usage need to be in your detection layer. They will not come from your SIEM. This is instrumentation work at the AI system level.

Redefine containment. For most AI incidents, ‘isolate the system’ is the wrong first move. Switching to a rule-based fallback while keeping the service running may cause less harm than taking the system offline and triggering a business escalation. Each deployed model needs pre-defined rollback criteria and a named fallback. Write those down now.

Get legal in the room before the incident. Mobley v. Workday means both the AI vendor and the deploying organization can carry liability for bias incidents. Air Canada means you cannot disclaim what your AI says to a customer. If your legal team is learning about an AI incident from a press inquiry, something has already gone wrong.

Build your AI inventory and treat it like your asset register. Start with the AIBOM for your highest-risk systems — those with access to customer data, financial decisions or clinical workflows. The GenAI-IRF framework gives you a structured taxonomy for this work and the GLACIS AI Incident Response Playbook maps it to NIST SP 800-61 and MITRE ATLAS procedures your team can adapt without starting from scratch.

Forty-two percent of organizations have already had a suspicious or confirmed AI incident, and more than half say their security posture is catching up, inconsistent or reactive. Updating your playbook isn’t optional. Fix it before you need it.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Jurassic Park wasn’t really about dinosaurs.

It was about arrogant people building systems they believed were controllable.

“Life finds a way” is probably the most famous line from the entire franchise. Ian Malcolm’s warning that no matter how sophisticated the technology becomes, no matter how expensive the fences are, and no matter how confident the operators feel, nature eventually escapes containment.

And in every movie, it does.

The dinosaurs always get out. The systems fail. Eventually, the humans lose control.

What makes Jurassic Park fascinating is that despite advanced monitoring, complex containment systems and sophisticated operational controls, the outcome never really changes. At its core, the story is about people mistaking visibility for control.

Cybersecurity has the same problem.

For years, security teams have operated under the assumption that with enough tooling, governance, process, maturity and spend, we can build environments that are effectively secure. Maybe not perfect, but secure enough that compromise becomes rare and manageable.

But attackers find a way.

Given enough time, skill or motivation, they eventually identify the weakness nobody considered. The overlooked privilege. The dependency nobody mapped. The misconfiguration hiding behind layers of dashboards, process, and compliance reporting.

We are already seeing this play out. Nation-state attacks are becoming increasingly sophisticated, while AI-driven exploit discovery is beginning to compress vulnerability research from weeks into minutes.

The raptors are learning faster now.

Mistaking visibility for control

That does not mean prevention no longer matters. The fences in Jurassic Park still slowed the dinosaurs down. They created friction. They reduced exposure. Modern security controls do the same thing.

But the failure in Jurassic Park was never simply that the fences broke.

It was that the entire system assumed the fences represented certainty.

Cybersecurity often makes the same mistake.

The industry has become incredibly good at demonstrating preparedness in controlled environments. Dashboards. Compliance reports. Tabletop exercises. RTO metrics. Recovery attestations.

Jurassic Park had dashboards too.

The problem is that visibility is often mistaken for survivability. Organizations can prove they monitored the environment, documented the process, and ran the exercise, while still having very little confidence that the business could continue operating during a genuine systemic failure.

Most organizations still operate with an implicit belief that compromise is exceptional rather than inevitable. Disaster recovery plans, business continuity workshops, and annual tabletop exercises are treated as evidence of resilience. In reality, many of them are carefully controlled simulations of a world that no longer exists.

Traditional disaster recovery was designed for an era where infrastructure changed slowly, applications were relatively static, and dependencies were limited enough that recovery assumptions could remain valid for months or even years.

That world is gone. AI killed it.

Environments now evolve constantly. Cloud infrastructure changes daily. AI-assisted development accelerates release cycles. Applications rely on sprawling third-party ecosystems. APIs connect systems in ways many organizations do not fully understand. Entire workloads appear and disappear dynamically.

The environment you tested last quarter may no longer exist today.

And yet many resilience programs still operate as if annual or quarterly testing provides meaningful confidence.

Most companies do not really test resilience.

They test optimism.

The backup fallacy

And nowhere is this overconfidence more obvious than backups.

Somewhere along the way, organizations confused “having backups” with “being resilient.” Those are not remotely the same thing.

A backup simply proves you stored a copy of something at a specific point in time. It does not prove you can survive.

Most recovery models were designed in the late 90s and early 2000s for relatively static systems and predictable infrastructure. The core philosophy has barely evolved since then, even as environments have become increasingly distributed, ephemeral, and interconnected.

Restoring data is not the same as restoring operations.

Restoring infrastructure is not the same as restoring business functionality. Modern application are complex and rely on ephemeral elements, third party components and applications as well as complex data flows not just data sets.

Very few organizations continuously validate whether they can recover full feature-function applications, maintain operational workflows, preserve data integrity, reconnect dependencies, restore permissions correctly, or continue operating under active attack conditions.

We built incredibly sophisticated telemetry for understanding how we die.

We built almost none for proving we can survive.

That gap is becoming impossible to ignore.

The recent rise of continuous resilience testing and recovery validation is not accidental. It reflects a growing realization that recovery assumptions themselves may no longer be trustworthy.

Static resilience models are struggling to survive dynamic infrastructure.

This is where resilience starts becoming an engineering problem rather than a compliance exercise.

When restoration assumptions fail

Because the real question is no longer, “How quickly can we restore the application?”

The real question is, “What happens if we cannot restore it?”

Jurassic Park repeatedly explored exactly this scenario. The real panic never started when the fences failed. It started when the operators realized they could not regain control quickly enough.

Businesses now face the same risk.

What happens if AWS experiences a prolonged outage? What happens if Azure Identity Services fail globally? What happens if Stripe, Salesforce, Slack, or Microsoft 365 disappear for days rather than hours?

Many organizations do not actually have business continuity strategies for those situations.

They have restoration assumptions.

Twenty years ago, most organizations directly owned large portions of their operational stack. Today, companies increasingly rent critical business capability from a relatively small number of providers.

Identity. Infrastructure. Communications. Payments. Collaboration. Customer operations.

The efficiency gains are enormous.

So is the concentration risk.

Resilience as an engineering discipline

Historically, business continuity planning assumed localized disruption. A building burned down. A regional data center failed. A storm impacted an office. The internet itself was not the dependency.

Today, entire businesses are built on tightly interconnected SaaS and cloud ecosystems where operational survivability depends on third parties remaining continuously available.

We optimized organizations for efficiency, automation, integration, and scale.

Not necessarily survivability.

That is why resilience needs to evolve beyond annual tabletop exercises and static recovery plans.

True resilience is not a binder sitting on a shelf. It is not a workshop performed once a year. It is not a recovery document written against an environment that changed six months ago.

It is a continuous understanding of the environment itself.

It requires live telemetry, operational visibility, dependency awareness, continuous validation, and the ability to adapt under changing conditions.

Adapting to chaos

The survivors in Jurassic Park only succeeded once they stopped pretending the environment was fully controllable and instead adapted to the reality in front of them.

Cybersecurity needs to make the same shift.

Attackers will keep adapting.

AI will accelerate faster than most governance models can handle.

Complexity will continue to outpace our assumptions about control.

The organizations that survive will not necessarily be the ones with the tallest fences. They will be the ones who understand their environments deeply enough to continue operating when control is lost.

The goal was never to eliminate chaos.

It was to survive long enough to adapt to it.

Because resilience is not about preventing chaos.

It is about operating through it.

Because eventually, one way or another, life finds a way.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Every day this past week I did something I suspect millions of other people also did: I stared at an LLM model picker and wondered which one I was supposed to want.

OpenAI just released ⁠GPT-5.6 Sol, Terra, and Luna. Sol is the flagship. Terra offers much of its intelligence for less money. Luna is cheaper still. Anthropic released ⁠Claude Sonnet 5 at the end of June and Opus 4.8 the month prior, with a little Fable 5 emerging in between. Meanwhile, Google, which seemed to be winning the model wars a few months ago, is now getting shade from Gergely Orosz, who ⁠argues that Gemini has slipped outside the top tier for software development and has been out of the major model release game for eons (May 19).

Perhaps Orosz is right. Perhaps he’ll be wrong again in six weeks. Honestly, it’s exhausting.

I use ChatGPT and Claude constantly and still have no principled idea which model to choose most of the time. I tend to click whatever looks like the biggest, most expensive option because I don’t know what I’m giving up by choosing something smaller. “Instant” sounds dangerously unserious. “Thinking” sounds expensive but powerful.

A quick survey of my LinkedIn crowd suggests others also feel my “WHICH MODEL???” pain. More importantly, I suspect most enterprises do, too.

A model doesn’t rot

Before getting carried away, however, it’s worth considering whether any of this model churn actually matters. After all, a model doesn’t rot. The model an enterprise put into production in March performs just as well in July as it did when the company selected it. “Obsolete” generally means that something better now exists, not that the deployed model suddenly stopped summarizing insurance claims or classifying support tickets. (In other words, once you have something working, the idea that “but maybe Opus 200.2 is better!” is really a FOMO problem, not a performance issue.)

Most enterprise workloads don’t live at the frontier anyway. Extraction, summarization, classification, document comparison, and customer-service assistance often work perfectly well with smaller, cheaper models. OpenAI’s own pitch for the trio of GPT-5.6 models isn’t simply that Sol is better. It’s that ⁠Terra and Luna deliver different combinations of intelligence, latency, and cost. Luna, the cheapest tier, nearly matches the previous generation’s peak performance at less than half the estimated cost, according to OpenAI.

The practical question, of course, is where to start. An enterprise can’t test every model, every reasoning setting, and every price tier before doing any work. So here’s my advice (which I don’t follow in my own work, but I’m not defining enterprise strategy and can be a little price-insensitive). Start with the cheapest credible model that appears capable of the task. Give it a representative set of real examples and, before you start testing, define what counts as good enough. If it passes, stop. If it fails, move up a tier or try a model with strengths better suited to the work.

That sounds almost offensively simple, but it reverses the way many people, including me, use these products. We start with the biggest model because we’re afraid of what we might lose. Enterprises should start lower and require evidence before paying for more intelligence.

There are exceptions, of course. For genuinely difficult work, such as autonomous coding, complex research, or high-stakes reasoning, beginning with a frontier model may save time. But even then, the goal should be to establish a quality ceiling, then test whether a cheaper model can meet it. It’s changing the question from “which model is best?” to “what is the least expensive model that reliably clears the bar for this job?”

For many workloads, that price improvement matters more than a few extra benchmark points. ⁠As I argued back in 2023, following AI hype doesn’t help anyone. If your model strategy depends on whichever benchmark screenshot is circulating on X this week, you don’t have a strategy. Not a viable one, anyway. Pick a model and ignore the noise.

Except, of course, when that noise suggests a serious signal.

Sometimes better really is better

Frontier improvements aren’t always incremental, making it advantageous to consider an upgrade. Coding is the obvious example. There’s a significant difference between a model that suggests the next few lines of code and one that can inspect a repository, plan a change, use tools, run tests, discover its own mistakes, and keep working for an extended period. That isn’t merely a nicer autocomplete experience. It can reorganize a development workflow.

This is why enterprises can’t simply standardize on an 18-month-old model and declare victory. In some areas, particularly software development and other agentic work, better models can unlock compounding productivity. A model that reliably completes 80% of a bounded task rather than 50% may justify an entirely different division of labor between humans and machines.

Still, that upgrade isn’t free.

Models differ in how they interpret instructions, call tools, manage context, refuse requests, and fail. Prompts and scaffolding tuned for one model can regress when moved to another. Or costs can explode. As one of my Oracle colleagues discovered just this week, running the same tasks in GPT 5.6 was orders of magnitude more expensive than 5.5. The API change may be trivial, but the revalidation and implications are not.

This leaves enterprises caught between two bad options. They can freeze and potentially miss out on meaningful improvements or chase every release and repeatedly test production systems on faith. What to do?

Stop making model bets

The answer is to stop making LLM bets and start making job-to-be-done bets. Stop asking which model is fastest. Instead, figure out what work you are trying to improve. What does a good result look like? How much latency and cost can the workflow tolerate? How wrong can it be before a human must intervene? Once those questions have answers, model selection becomes less opaque.

A difficult code migration may justify GPT-5.6 Sol or Claude Sonnet 5. A repetitive classification task may work just as well with Luna or another smaller model. A regulated workflow may require a model or deployment option that offers particular data controls. Sometimes the correct model is no LLM at all, like when I’m writing this post. Sorry, AI vendors! (At least you won’t get blamed for my mistakes.)

This is where evaluations become the center of enterprise AI strategy. ⁠As I’ve said before, most companies don’t have an AI quality problem so much as an AI measurement problem. Hence, a private evaluation suite built from real company work is the only leaderboard that matters. Does the new model materially improve quality? If so, use it! Does it reduce cost or latency? Again, that’s your free pass to adoption. Does the improvement justify the expense and effort of revalidation? If yes, continue.

Make model releases boring

As important as the model is, keep in mind that AI success always comes back to your company’s data, your company’s workflows, your company’s integrations, etc. That’s the ⁠dull reality behind sexy AI. Retrieval, memory, governance, data quality, observability, and feedback loops aren’t as exciting as a new model launch, but they’re what ultimately make AI truly work.

Again, when it’s time to consider something new, the principle should be to default to the least expensive model that reliably passes your evaluations. Only escalate harder tasks to more capable models when measurement shows that the premium pays. Tip: Make this invisible to employees so that the system routes to the best model for a particular prompt. As dbt Labs’ Jon Lewis expresses it, “The best model is ‘Auto’ and I won’t hear anyone say otherwise.” OpenAI’s own ⁠migration guidance recommends testing models on representative tasks, including trying a lower reasoning level rather than automatically cranking everything to the maximum.

As for me, I’ll probably keep clicking the shiniest option. I don’t have a formal evaluation suite for InfoWorld columns, and the marginal cost is a subscription I already pay. Enterprises don’t get that excuse.

A product manager on my team recently asked me where we were seeing the most issues across the engineering team. Instead of guessing, I had an engineering lead point Claude at our Jira via an MCP connector and look at the bug patterns himself.

One team had a wildly disproportionate share of tickets — about 50% of their sprint time was spent on “bugs,” versus roughly 25% for everyone else. The headline number suggested a quality problem.

It wasn’t. When we layered in the context around those tickets, almost none of them were bugs. They were manual workarounds for a missing product capability: customers asking us, one request at a time, to restore items they had accidentally deleted. Not shipping an item restore feature was burning roughly 1.5 engineers’ worth of capacity. I went back to our product team and said, “Build this, and you reclaim a person and a half.”

The analysis took 45 minutes. It was only possible because our data was already organized, tagged by team, connected to contributors, accessible through MCP and protected by role-based access. None of that is “AI.” All of it is the layer underneath AI that almost nobody invests in first. That’s probably because the investment is unglamorous: updating data dictionaries, access controls, team taxonomies, system-to-system mappings. Most of the work has been the same for twenty years. AI just raised the cost of skipping it.

The intelligence underneath the models

I keep coming back to the value of context data layers as a CTO in the middle of an AI rollout. I have started calling that value proposition contextual intelligence because I haven’t found a better name. Anthropic’s engineering team has been calling this kind of work “context engineering” since late 2025, and CIO ran its own feature on the term shortly after. Whether you describe it as contextual intelligence or context engineering, it’s the part of the stack where the actual programming work still lives.

If business logic is your company’s official org chart, then contextual intelligence is knowing who actually gets things done, how decisions are actually made and what the unwritten rules are. One is theory. The other is reality.

Most enterprise systems capture the theory. The systems that capture how work actually happens — what people do, how teams operate, where decisions get stuck — are rarer and harder to build. And modern LLMs, it turns out, are useless without both.

I learned this the hard way at a recent company hackathon. Nine engineering teams, one prompt: make our operational dataset more usable through AI. My team built persona-based chatbots (CFO, CIO, sales manager) on top of an MCP server backed by Postgres and our enrichment data. Other teams built dashboard generators, Looker conversational analytics and workflow agents.

The initial demos all had the same problem. Claude could talk to our data, but the answers were either generic or confidently wrong. The CFO persona would happily report a “spend trend” that quietly conflated two distinct cost categories across two different tables. The CIO persona would answer questions about team productivity, but the averages across roles should never have been aggregated. The sales manager persona returned answers that were technically correct against the schema and completely wrong against the business. The raw data was rich. The context layer around it didn’t exist yet. Chatting with raw data is not an AI product. It’s a demo.

One of my senior engineers spent the second day ripping out the agent’s direct database connection. He stopped trying to prompt-engineer the LLM to understand our business and instead codified that logic into the data pipeline. Working backward from the failed CFO answers, he mapped out the implicit knowledge an experienced controller relies on: Explicitly defining which legacy tables actually represent ‘spend,’ writing the rules for currency normalization and hardcoding our fiscal time windows. He built a series of semantic SQL views to enforce these rules and restricted the MCP server to exposing only this curated layer. When we pointed the same model at those same questions, it returned completely different answers. They were specific, evidence-based and grounded in our actual business reality. The model didn’t get smarter. The engineering beneath it did.

The same pattern shows up everywhere I look right now

McKinsey keeps publishing that software development tops enterprise AI use cases, with companies reporting 30–50% productivity gains in pilots. The pilot numbers are real. They rarely translate to top- or bottom-line impact in production. Our own company data tells the same story: Between Q1 2025 and Q1 2026, our total AI tool usage grew by 328% (over 4x). Over that same period, PR throughput grew by just 49%.

That gap — adoption way up, outcomes inching along — is the context gap. Plug a generic agent into raw, uninterpreted data, and it will act inefficiently at best, harmfully at worst. An agent optimizing sales without your customer segmentation or product hierarchy will confidently recommend the wrong thing. Anthropic framed the shift directly: building with language models is becoming “less about finding the right words and phrases for your prompts, and more about answering the broader question of what context configuration is most likely to generate our model’s desired behavior.” That second question — what context configuration  — is the entire game. Most organizations are still answering the first one.

Where the work actually lives

A growing number of CTOs I talk to are shifting their AI investments accordingly. Less attention on the model. More on the layer between the model and the data.

When peers ask me what that actually looks like day-to-day, I tell them I give every engineering role the same mandate: the LLM should never see raw, uncontextualized data.

In practice, that breaks down to three pieces of work, none of them glamorous.

The first is semantic middleware. We need code that transforms raw data into business-meaningful signals before it ever reaches the model. Our feature stores hold things like “employee code velocity on critical-path features,” not “X logged 50 Git commits.” The work of figuring out what “critical-path” means in our product, in our org, on this team is the work. It does not get cheaper because the model has gotten better.

The second is multi-agent design. Instead of one omniscient orchestrator, we run smaller agents scoped to specific domains, each with rules that catch the failure modes the main model is known for. We pair them with RAG that retrieves precomputed insights, with their rules attached, rather than raw documents. Validation checkpoints sit between steps and flag suggestions that violate known constraints, such as averaging productivity across completely different job functions. The guardrails are not there to be clever. They are there because we already watched the model make those exact mistakes.

The third is evaluation that takes business logic seriously. When I look at a model, general benchmark accuracy is the least interesting number. I want to know whether it respects our constraints and integrates cleanly with our existing architecture. That sometimes means fine-tuning our patterns, sometimes constitutional approaches to embed principles, sometimes hybrid systems where deterministic rules sit alongside the probabilistic ones. The throughline is the same: validate against reality, not against the benchmark.

Why this matters now

The reason this matters more now than it did six months ago is that adoption is moving faster than measurement, let alone integration. Model Evaluation & Threat Research’s (METR) developer productivity work tells the story in a way they didn’t intend. In early 2025, they ran a controlled study and found AI tools slowed experienced open-source developers by 19%. When they tried to repeat the study in late 2025, the experiment broke. Thirty to fifty percent of developers refused to submit tasks under the no-AI condition. They wouldn’t accept working without their tools. METR is now redesigning the study because the original methodology no longer holds up against how developers actually work. That’s how fast adoption moved. But I’d be willing to bet the organizational scaffolding required to convert that adoption into outcomes — context layers, workflow redesign, retraining around new tools — moved nowhere near as fast.

Get ahead with context

The teams I’ve seen succeed with AI built the context layer first. The teams I’ve seen struggle eventually built in context anyway, just at higher cost and with more scar tissue. Raw data is the new currency. But raw data without a context layer is cash sitting in a vault. It cannot act on anything. The difference between insight and noise is a layer of code that understands what your data means.

That layer is the work. It is where the next decade of competitive advantage will sit. And in my experience, the organizations that build it first are the ones that will actually get the productivity gains the rest of the market keeps promising.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

A new era of customer engagement is taking shape. AI agents are quickly becoming integral to the way businesses serve, support, and sell to customers — able to respond, reason, and take action in ways that go far beyond scripted automation.

Many customer journeys, however, are still built on systems that don’t talk to each other. Customer data lives in one place, channel history in another, and AI agents often operate with only part of the picture. Customers feel the pain when they switch between channels like voice and messaging, get transferred, and have to repeat themselves yet again. It doesn’t matter that they’ve been loyal to a brand for years, every interaction feels like a cold start. That is the conversation gap.

It’s clear that AI isn’t the problem, infrastructure is. Closing the gap requires new building blocks that focus on continuity, so context can carry forward across systems, channels, human agents, and AI agents.

To bridge the gap, at SIGNAL 2026, we are introducing a new conversation layer for the Twilio Platform.

Twilio Conversation Orchestrator, Twilio Conversation Memory, and Twilio Conversation Intelligence are now generally available. Together, they help businesses coordinate interactions, preserve context, and connect human and AI agents so every conversation is more continuous and useful.

In addition to the new Conversations layer, we’re also announcing platform updates that make it easier to build, manage, and scale customer engagement on Twilio — from a reimagined Twilio Console to expanded channels and new voice AI capabilities.

New building blocks for connected conversations

The conversation gap does more than create inconsistent customer experiences. It hurts conversion and retention, increases operational costs, adds integration complexity, and makes agents less productive. The new platform capabilities we’re introducing are designed to fix that by coordinating interactions, maintaining context, and surfacing signals as conversations happen.

Conversation Orchestrator

Conversation Orchestrator helps businesses coordinate interactions across Twilio channels without complex custom logic. Teams can configure it in Console or configure their implementation with the API. It connects interactions into a single thread and manages handoffs between human agents and automated systems.

Conversation Memory

Conversation Memory creates a living, identity-resolved profile by connecting customer data with conversation history and customer traits. That means each interaction starts with the right context. It’s built specifically for LLMs to reduce latency and token usage by surfacing the most relevant details when they matter.

A new Enterprise Knowledge API (now generally available) also allows teams to deliver more relevant experiences and ground interactions in trusted business knowledge such as FAQs, policies, and product documentation.

Conversation Intelligence

Conversation Intelligence provides real-time understanding of live interactions. Using prebuilt and custom LLM-based operators, it can detect changes in sentiment, flag potential escalations, and trigger action during a conversation, not only after it ends.

That gives teams the ability to respond sooner, support agents more effectively, and improve customer outcomes while the conversation is still in progress.

Together, these products help businesses create customer experiences that feel more connected across channels.

Open by design

Twilio remains neutral by design. We start with the premise that you know your business. We aren’t here to prescribe a model, framework, or data strategy. We provide the infrastructure that helps you build customer engagement in the way that works best for your business. You pick the model and agent runtime. You own the data.

That doesn’t mean you need to start from scratch, either. We partnered with Microsoft, AWS, and others to create blueprints that support faster development. We are also introducing an open-source developer toolkit, Twilio Agent Connect (now generally available), that lets your teams connect agents built on any LLM or framework directly to Twilio’s infrastructure.

For developers, this means more flexibility. For businesses, it means less lock-in and the ability to get value from existing investments. For partners, it means more ways to build with Twilio.

A new front door

We are also introducing a reimagined Twilio Console, because as customer engagement grows more complex, managing the infrastructure behind it should feel effortless.

The new Console is a single mission control center that brings your communications, identity, and data into one experience: one login, consistent logs across every surface, an intelligent Console Assistant, transparent billing insights, and streamlined compliance workflows that no longer slow you down.