Providers and deployers of AI systems: You only have a couple of weeks left until you must explicitly inform users when they are interacting with AI content.

To assist in the effort, the European Commission (Commission) has published guidelines to help AI deployers get in line with the AI Act’s transparency obligations, which will begin to go into effect on August 2.

After that, companies providing AI systems must alert users when they are interacting with AI. They must also tell users when they have been exposed to deepfakes, “emotion recognition,” or biometric categorization systems, or when they are given AI-manipulated content in matters of “public interests without human review or editorial control.”

Henna Virkkunen, the Commission’s executive VP for tech sovereignty, security and democracy, said in a statement, “with today’s guidelines, the Commission supports the smooth and effective application of the AI Act to make AI systems interacting with people such as chatbots and AI agents and AI content more transparent and trustworthy. These guidelines support providers and deployers in meeting their obligations under the AI Act, while helping citizens know when they are interacting with AI.”

Systems must include machine-readable markers to reveal such content, to reduce “the risk of deception and manipulation” and build public trust in AI.

“Generative systems have collapsed the cost of producing convincing content while the cost of judging it stands where it always stood,” said Sanchit Vir Gogia, chief analyst at Greyhound Research. This requirement is “an attempt to restore friction to that imbalance.”

A company’s non-compliance could result in fines anywhere from €750K (about $856K) to €15M (about $17 million), or even up to 3% of its total worldwide annual revenue.

Transparency requirements

The EU AI Act’s transparency requirements apply to “natural or legal persons,” public authorities, agencies, or other bodies that develop AI systems, or have them developed, and place them on the EU market or into use under their name or trademark. This means all companies, regardless of whether or not they are EU-based.

“Systems placed on the European market, put into service there, or producing outputs used there are inside the field, wherever the developer sits,” Gogia noted.

Applicable systems must be intended to interact directly with “natural persons”; these systems include AI-enabled chatbots or conversational agents, AI companions, or coding agents. However, AI-enabled tools like recommender systems, spam filters, authentication, search and retrieval, transcription, text and code auto-completion, or predictive maintenance do not fall under the rule.

Specific outputs such as AI-generated text, images, video, and audio must contain a machine-readable mark. Deepfakes and public interest-related text created by AI without human review or control must be clearly labeled, however, deepfake content that is “artistic, creative, satirical, or fictional” is largely exempt.

AI content must be marked with one of three labels: “AI,” “Fully AI-generated,” or “Partially AI-modified.” For instance, “Fully AI-generated” applies when news summaries, music, art, or videos have been created without any human oversight (apart from prompting), while “partially AI-modified” could mean a person’s face is swapped into an authentic photograph to create a deepfake.

The three icons are publicly available for free use; enterprises can download zip files in PNG and SVG formats.

Most of the Act’s transparency rules begin to go into effect on August 2. But AI systems placed on the market before then will have some leeway; they must be in compliance by December 2.

However, a four-month allowance “on one obligation, for one population of systems, contingent on one procedural step, is not a strategy,” Gogia emphasized. Enterprises should plan to comply by August 2 and “treat any relief that arrives as margin.”

A consistent code of practice

Along with the transparency guidelines, the Commission has introduced a code of practice that essentially serves as a gesture of good faith. When signed, it can provide “legal certainty” and a “simple and practical” way to demonstrate compliance with the AI Act, according to the Commission. Signatories can also collaborate through the ‘Signatory Taskforce,’ which will share practices and advance technologies around marking and labeling practices.

Providers that choose not to sign must comply through other methods and demonstrate that those methods are “adequate” through assessment by surveillance authorities, according to the Commission.

Non-signatories “keep their flexibility, and will face more case-by-case scrutiny for it,” said Gogia.

Criteria for compliance

Shashi Bellamkonda, principal research director at Info-Tech Research Group, pointed out that the transparency requirements apply to content only when three criteria are met: It has been published, is informative to the public, or is on matters of public interest.

B2B business content or blogs may not need an AI disclosure if they do not meet these criteria, he noted. Also, published text that has undergone human review or is under editorial control does not need to be labeled. Editorial control means that a person must hold the ultimate legal responsibility for the publication of the content.

Many companies like Google, Adobe, and LinkedIn have already established ways to identify images marked as AI-generated. Meta has made it a requirement, but the creator has to add the AI-generated label, Bellamkonda said.

“This is a good move for guardrails around public information, and companies with good compliance and ethical oversight may not have to worry about this,” he noted. But as a general practice, companies should disclose AI-generated content and state whether it has been human reviewed.

Creating a transparency pipeline

Establishing full transparency means identifying who carries the responsibility for the content, whether the marking survives real use, not just testing, and what evidence will defend the decision, Gogia said.

Concerns cluster around responsibility, durability and evidence. Several organizations usually touch one piece of content, and none controls the whole chain, which is why contracts become the “pressure point,” he said. Most current agreements were written to deliver software and say “almost nothing” about provenance persistence, verification access, or evidence retention.

The durability concern is the most difficult, Gogia noted, because marking performs well in controlled settings but “badly in ordinary life.” Meta, for one, said its invisible watermark was designed to survive cropping; a published test, however, found the company’s preview detector missed 55% of cropped images.

“CIOs should ask which platform can actually provide evidence before believing its dashboard,” said Gogia.

Disclosure of AI use must be “clear, distinguishable and accessible,” he emphasized. “A notice buried in lengthy terms, or reachable only through determined clicking, satisfies nobody, least of all a market surveillance authority.”

Sustained compliance is a “living control” requiring a central record of systems, duties and evidence; testing taking place where the user meets the control rather than where the developer built it; and continuous supplier assurance. Enforcement will vary by country, so keep one common baseline with local overlays, Gogia said.

His advice: Inventory every system that talks to people, generates content, or gauges sentiment; classify provider and deployer roles; place disclosures at first interaction; define substantive human review; keep the evidence.

Marks and provenance signals should be tested after content undergoes cropping, compression, translation, transcription, and other editing, Gogia said. A useful audit starts from a real output and follows its “pulse” through generation, editing and publication, identifying at “each beat” the responsible party, the surviving mark, and evidence for exceptions. Missed labels should also be traced for root cause and recurrence.

To ensure compliance, before August 2, enterprises need a prioritized inventory, live disclosures on the highest-risk use cases, and a “named owner for every control,” he noted. In the first 30 days, they should stabilize and test; in the first 90 days, push requirements into procurement processes as a standing discipline. Procurement must secure commitments on marking methods, known failure modes, and evidence access, with explicit notice if/when any of them change.

“The sensible architecture is a common transparency baseline carrying traceability, responsibility, and evidence, with jurisdictional overlays for language, sector rules, and local practice,” Gogia said.

This article originally appeared on CIO.com.

The head of the US government’s AI testing institute, Chris Fall, has resigned about three months after taking charge of the Center for AI Standards and Innovation (CAISI), the federal organization responsible for evaluating advanced artificial intelligence models for safety and security.

Current National Institute of Standards and Technology NIST Director Arvind Raman will serve as acting CAISI Director following Fall’s departure while continuing to oversee the Commerce Department office responsible for the institute, the Daily Signal reported, citing two people familiar with the matter.

A Commerce Department spokesperson who spoke to the publication did not disclose a reason for the resignation.

Fall assumed leadership of CAISI in April after the Trump administration reorganized the former US AI Safety Institute under NIST. The institute develops methodologies for evaluating frontier AI models and works with AI developers on voluntary technical assessments covering areas such as cybersecurity, model misuse, reliability and other risks associated with increasingly capable AI systems.

The leadership change comes as governments and AI companies continue developing technical approaches for evaluating frontier AI models while enterprises expand deployments of generative AI and agentic AI across business operations.

In recent months, the Commerce Department has taken a more active role in AI policy involving advanced models, placing greater attention on how the federal government evaluates technologies with potential national security implications.

Continuity matters more than personalities

CAISI works with AI developers such as Anthropic, Google’s DeepMind and OpenAI on voluntary evaluations of frontier AI models and develops methodologies for testing model capabilities and risks. The institute does not regulate AI developers or certify commercial AI systems.

For enterprises, those evaluations are one source of technical information alongside vendors’ own testing, third-party security assessments and internal AI governance programs.

Sanchit Vir Gogia, chief analyst at Greyhound Research, said enterprises should focus less on the individual leading the institute and more on whether its technical work continues with the same level of consistency and transparency.

“Leadership churn at CAISI weakens the signal long before it weakens the science,” Gogia said. “The testing has not stopped. Its authority simply does not travel as cleanly once the leadership does not.”

According to Gogia, the more important question for enterprises is not whether the institute’s evaluation work will continue but whether the processes supporting those evaluations remain stable.

“The instinct is to ask whether the pipeline is breaking,” he said. “The more useful question is where the pipeline now sits.”

Enterprises still carry the burden of AI governance

Gogia said organizations should continue treating government-led AI evaluations as one input into their governance processes rather than as evidence that a model is inherently safe for enterprise deployment.

“A government evaluation was always a signal, never a certificate,” he said. “A signal loses value the moment its issuer becomes unpredictable.”

He said enterprises should instead monitor whether CAISI maintains consistent evaluation methodologies, continues publishing technical findings and preserves continuity within its research teams under interim leadership.

“The name on the door is not the signal. The behaviour underneath it is,” Gogia said.

Gogia also cautioned against linking Fall’s resignation to recent Commerce Department actions involving AI policy or export controls, noting that there is no public evidence connecting the two.

“CAISI evaluates; it does not enforce export controls, because it holds no such power,” he said. “This is not a testing body reaching for enforcement. It is enforcement reaching past the testing body.”

With Raman assuming the role on an interim basis, the next significant milestone for enterprises will be the appointment of a permanent director, and whether the institute’s evaluation programs continue without disruption, the analyst said.

Gogia said the successor’s mandate may prove more important than the individual selected.

“A CAISI result is not a safe harbour,” he said. “It informs an obligation; it does not discharge one.” NIST did not immediately respond to a request for comment.

The traditional methods of measuring success no longer tell the full story. Here’s what should replace them — and why.

In brief:

• AI is transforming software as a service (SaaS), and the old ways of keeping score no longer apply.
• Smart companies are evolving new metrics that provide deeper insight into how AI-native software is performing in a new marketplace.
• These changes impact everything from pricing to valuations.

The transformation of the software-as-a-service (SaaS) industry toward AI-native operating companies is rapidly changing the unit of value across the industry.

The traditional metric of seats — which measured access — is rapidly giving way to credits designed to measure work performed. This evolution is upending the industry in multiple ways, impacting everything from pricing to enterprise valuations.

While many companies still cling to seat-based metrics to measure growth, efficiency and durability, the future is likely to be one in which companies utilize a credit-centric metrics framework, with seats and outcomes as the bookends of a spectrum.

Why do software companies need new metrics?

Why the rethink, and why now? There are five major forces that are driving this shift:

1. The unit of value is changing. Seats measured who could access software, and credits measure what the software actually does. But in an AI-native world, agents don’t have seats; they have workloads. Over the past 18 months, every major SaaS platform has moved to some forms of credit or consumption unit.
2. The cost of goods sold (COGS) is exploding. AI inference adds real per-unit costs that scale with usage. In an AI-native world, software companies can’t scale to infinite users at near‑zero marginal cost as before.
3. Buying is moving up the org chart. AI-native applications shift purchasing to higher-level operators — such as line-of-business leaders or chief operating officers — which expands the market from software budgets to labor budgets. And because AI agents replace services as well as software, the total market opportunity is 3x to 10x larger than traditional SaaS.
4. Time to value (TTV) is collapsing. With AI-native tools, customers start seeing meaningful results in weeks rather than quarters. Onboarding and setup are fast, workflows are pre-built, and there’s no need for extensive customer success or professional services — dramatically reducing implementation time and costs.
5. Retention is bifurcating. AI forces clarity in a way that traditional SaaS couldn’t. Products that can provide value become even “stickier” and retain customers. Those that don’t churn faster. In an AI-native marketplace, the middle disappears.

How this shift is impacting pricing

Given how AI-native software is transforming the market, the shift to more variable pricing options is inevitable.

Seats won’t go away completely. Subscription pricing based on the number of users is stable and predictable and will continue to work for some customers. Tokens — the use of pass-through pricing for underlying compute — will fit those customers where the AI feature is commoditized or the buyer wants transparency into costs.

Credits will likely become the dominant architecture because they provide a simple metric for both customers and providers. The vendor sets the conversation ratio between credits and underlying compute, shielding the customer from inference cost details. Credits are easy to understand and can be packaged into annual contracts for multiple features and products.

Finally, the industry will likely see some move toward outcome-based pricing for results such as resolved tickets, recovered revenue or qualified leads. This strategy will mostly be limited to verticals where it is easy to prove AI impacted the result.

Where a software vendor sits on this spectrum is a signal of differentiation and pricing power. Credits are where most defensible AI-native businesses are landing because they balance customer predictability with vendor margin control.

How AI upends classic SaaS metrics

When SaaS was in its infancy, companies settled on key metrics designed to answer a small set of core questions. Are we growing? Are customers using the product? Are we retaining and expanding accounts?

But as AI upends software itself, it is also requiring companies to adopt new metrics to track success. These new metrics fall into three primary buckets, rebuilt around the pricing spectrum described earlier and the trend toward credits as the primary frame:

Revenue composition

• Committed credit annual recurring revenue (ARR) vs. burndown ARR: Measuring the credits sold on annual commitment vs. those consumed and replenished. This is the single most important split for valuation. Committed credits behave like subscription and burndown behaves like usage.
• Credit utilization rate: The percentage of purchased credits consumed per period. This is a leading indicator of renewal sizing.
• Credit burn velocity: How fast is a customer consuming their credits, and is that consumption increasing or decreasing quarter over quarter? This metric predicts expansion or contraction before it shows up in ARR.
• Effective price per credit: The real revenue per credit after discounts, overage and rollover, which can detect revenue leakage and help companies set smarter guide rails.

Margin reality

• Credit margin: The gross profit the company earns per credit after subtracting inference costs. This is the core economic unit for AI-native, usage-based businesses — the replacement for gross margin per seat used in SaaS.
• Inference-adjusted gross margin: By carving out AI inference costs separately in the P&L statement, you can see true AI margins, avoid hiding deterioration inside blended SaaS margins, and clearly distinguish AI economics from legacy SaaS economics.
• Compute leverage ratio: This metric measures how efficiently the business converts compute spend into revenue. It shows whether your AI margins are improving as you scale.
• AI-adjusted “Rule of 40”: This updated metric recalibrates the traditional growth and profitability benchmark to account for AI’s lower gross margins and variable inference costs, giving a more accurate picture of business health for AI-native companies.

Behavioral and value signals

• Time-to-first outcome: Replaces traditional onboarding metrics. Tracks how fast a customer reaches their first measurable result.
• Adoption: AI-native adoption is measured by workflow penetration and active agent density, not seat count. As AI replaces human-driven usage, the unit of adoption shifts from people to automated workflows and agents.
• Net credit retention (NCR): Credit-volume retention across the customer base, tracked separately from net recurring revenue to avoid price-change impact.

Along with these new metrics, the industry’s transformation is prompting companies to retire or recalibrate old SaaS measures, including per-seat ARR as a primary key performance indicator (KPI), traditional magic number calibrated to subscription dynamics, unadjusted Rule of 40, customer success metrics tied to human touchpoints, and blended gross margin without AI COGS carve-outs.

What does this mean for enterprise value calculations?

As the internal metrics of success change, so do the ways the investment community measures growth and long-term viability.

Increasingly, a company’s valuation multiple depends on whether its revenue behaves like committed subscription ARR or volatile usage ARR, and the commit‑to‑burndown ratio is the metric investors use to decide where the company fits.

For example, a business with 80% committed credit ARR could trade closer to subscription comps and one with 80% burndown could trade closer to usage comps even though both have the same types of customers. Being able to proactively explain the commit‑to‑burndown mix can help companies avoid undervaluation.

In addition, utilization is expected to replace net promoter scores and seat usage as the primary predictor of churn or expansion. Low utilization guarantees downsizing at renewal, so companies must track utilization cohorts the same way SaaS tracks logo retention cohorts today.

We’re also seeing an inversion of the operating model, with R&D and COGS moving up the P&L and sales and marketing (S&M) and customer success (CS) moving down or sideways. The net operating leverage profile is structurally different from classical SaaS, and the cost-to-scale curve looks different too.

Finally, credit margin engineering is a hidden value-creation lever. The gap between price per credit and cost per credit is set by the software vendor and can be optimized. Most operators have barely started managing this rigorously, and the ones who do will pull away on margin.

What this means for leaders, boards and investors

The shift from classic SaaS metrics to new AI‑native measures isn’t cosmetic. It represents the seismic change the industry is experiencing as AI matures and transforms products and organizations.

While these metrics — and perhaps others yet to be determined — may evolve over time, there is no doubt they are already changing how AI companies allocate capital, price products, incent sales teams, evaluate performance and communicate with investors.

It’s important to remember that SaaS metrics were practical tools for a specific era of software. As that era draws to a close, winning companies will choose new metrics that shape behavior and drive smart decision-making.

The views reflected in this article are the views of the author and do not necessarily reflect the views of Ernst & Young LLP or other members of the global EY organization.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

I have worked alongside enterprise technology for more than 30 years and watched AI evolve from a lab experiment into the modern boardroom’s core focus. However, the last few years of implementing AI alongside our customers have delivered our most profound reality checks.

The initial hype has faded, leaving CIOs to drive real enterprise value. Based on my experience implementing Google, OpenAI and Anthropic technologies, here are the fundamental, technology-agnostic lessons every leader must anchor their strategy around.

AI as a leadership multiplier

The most common tactical error we see is treating AI as an isolated technology project. What I have observed among our customers is that true success does not come from organizations that define a standalone “AI strategy,” but rather from those leaders that integrate AI into their business strategy.

When our customers isolate AI and define an AI strategy, it inevitably treats it like a “technological toy” to experiment with. This approach yields fragmented, orphaned initiatives that fail to scale because they are fundamentally disconnected from their core corporate objectives. What I learned is that AI is not the ultimate destination; it is a powerful catalyst. We have replaced “What can AI do for our customers?” with a more strategic question, “How does AI accelerate their existing business goals?”

Think of AI like electricity. No modern corporation designs a standalone “electricity strategy.” Instead, all companies route it invisibly across the entire organization to illuminate offices, power production lines and drive communication. AI must be woven into the enterprise fabric in the exact same way, acting as an underlying utility that supercharges your existing operational model.

Integrating AI into the broader business strategy also dictates how we measure success. It forces a shift away from short-term tech vanity metrics and anchors the technology into a long-term roadmap.

When AI remains trapped within the IT department of our customers, we notice that it is relegated to a mere “software experiment.” To become a true competitive advantage, we observed that AI requires intense cross-functional orchestration. This perspective does not diminish the merit of the technical team; their expertise is fundamental for establishing the architecture, data governance and tools your enterprise requires. However, while IT builds the foundational infrastructure, it lacks the organizational authority to decide what should be built on top of it. Only the CEO or the owner of the company can step in to ensure AI leaves the “toy project” phase and integrates into the DNA of the organization.

The requirement for top-down, executive ownership stems from three critical realities observed in the field:

• Silo-smashing and data collaboration: True enterprise AI is data-hungry and that data lives across disparate business lines, finance, operations, marketing and customer service. Only the CEO possesses the cross-functional authority to demand that data silos be dismantled.
• Cultural transformation and fear mitigation: AI triggers widespread anxiety over job displacement across all industries and hierarchies. When relegated to an “IT project,” resistance spikes as teams view it as a threat to their livelihoods. When I saw the CEO lead this cultural shift directly is when I noticed the best results.
• C-Suite education and strategic alignment: The mandate for AI capability cannot just be delegated downward; the transformation must begin at the very top. I have conducted more than 70 presentations for the Board of Directors and C-Level teams. These people need to be actively educated not on technical code, but on specific business use cases, return on investment (ROI) frameworks and how AI resolves core organizational bottlenecks.

PwC’s data found that only 12% of enterprises have achieved both cost and revenue benefits from AI. Those elite 12% succeeded precisely because their CEOs embedded AI extensively across strategic decision-making and cross-functional workflows. AI is simply too disruptive and too critical to be left exclusively in the hands of technical experts. If AI is not on the CEO’s weekly agenda, it is fundamentally missing from the company’s true strategy.

AI as a new operational framework

Traditional IT systems have operated on strict algorithmic certainty: if you input a specific set of data, the system executes an immutable line of code and guarantees the same, predictable output every single time.

AI completely breaks this paradigm. Because modern AI is built on probabilistic models, it does not execute static formulas; instead, it predicts the most likely correct response based on mathematical probabilities. This means that AI solutions carry an inherent, small percentage of uncertainty and variability. A prompt entered today might yield a slightly different, though contextually valid, output tomorrow.

Executive leadership and organizational cultures must be actively educated to accept and navigate this fundamental shift. Traditional quality assurance frameworks for software are designed for a 100% success rate. Applying this rigid standard to AI will paralyze your initiatives, keeping 80% of your projects trapped eternally in the pilot phase. This happened to us in a food and beverage company in Latin America a couple of years ago. After this experience, we started to include conditions in our contracts that tolerate statistical margins of error and still define the project as a success.

In terms of cost calculation, we had to teach CIOs and business managers to forget the monthly subscription model for AI and learn to manage the primary unit of exchange in modern AI: the token.

To understand AI costs, executives must understand how large language models process data. AI models do not read full words; instead, they break text, images or code down into “pieces” called tokens. As a baseline, every 100 words process as approximately 130 to 140 tokens. Because the major AI providers use the token as their currency, your business is billed dynamically based on the exact volume of tokens consumed by every query submitted (input) and every response generated (output).

Many leaders believe AI costs are fixed due to flat-rate enterprise tiers ($25–$30/user). This is a temporary illusion. These venture-capital-subsidized rates mask true operational costs and come with dynamic usage limits. Modeling long-term ROI on them guarantees a severe budget shock when true consumption pricing takes over.

The solution is not to halt AI adoption; doing so means losing your competitive edge. Instead, the cost per token must cease to be treated as a technical footnote relegated to the IT department. It must be elevated to a core business variable.

Risks in the AI adoption model

Since the beginning of the AI boom, I have seen all our customers making a critical tactical error that could cost them heavily in the medium term: they are focusing only on operational efficiency (reducing costs with AI).

I have observed that an alarmingly high percentage of companies remain trapped in pilot phases focused exclusively on short-term cost reduction. Bain & Company’s global Automation and AI Pathfinder Survey found that the largest share of companies measuring their AI initiatives (exactly 40%) realized cost reductions of 10% or less, heavily missing their internal targets. Our customers are putting too many resources and effort into marginal financial gains and in doing so, they are jeopardizing their most valuable assets: service quality, resilience and customer trust.

Utilizing AI solely to slash headcount or cut operational corners is a dangerous trap that introduces severe field liabilities. A financial service organization in Latin America announced that they saved $1 million in customer support by replacing humans with AI chatbots. However, the mid-term reality revealed a different story: a damaged brand reputation due to AI errors and an influx of frustrated clients fleeing because the automated system cannot handle special cases.

Putting a company on an extreme AI diet might make it look leaner on next quarter’s financial statement, but over-indexing on cost-cutting will ultimately leave the business too weak to compete when market dynamics shift. We are now inviting our customers to change the question from “How much money will AI save us?” to “How will we leverage AI to exponentially increase the long-term value of our enterprise?”

Deploying enterprise AI is a marathon, not a sprint, and the terrain changes with every mile. The organizations that thrive in this next era will be those that transition from fascination to discipline, treating AI not as a magic bullet for immediate savings, but as a core capability that demands rigorous governance, architectural foresight and cultural maturity. Navigating this shift requires moving past the theoretical hype and anchoring decisions in raw, field-tested reality.

As we continue to deploy these technologies across industries, the blueprint for success is being rewritten in real time. Let’s keep this conversation going as we map out the future of business intelligence together.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Four months of Claude Code — that’s all it took for Uber to burn through its entire annual budget for AI. Token after token, engineers embraced the platform with few control mechanisms tying costs to outcomes. The result was a budget runaway and a clear case study in how limited oversight snowballs into an AI blowout.

This is a phenomenon I like to call “shadow tokens” — AI credits paid for by the company but largely invisible to decision-makers. Too many engineers have the final say over how much they consume and, therefore, what it costs. This all-you-can-eat attitude is part of the reason why Microsoft is reportedly winding down many internal licenses across key engineering teams and why one in five organizations is missing its AI spend forecast by more than 50%.

And the trend is only accelerating. By 2028, Gartner predicts that AI coding costs (driven by this kind of ungoverned consumption) will be as much per developer as the salary companies pay that person.

LLMs and agents introduce a new class of variable cost that scales with behavior rather than headcount, putting enterprises on the hook for tools that balloon with workload. I don’t see this as enterprises overspending because they’re reckless — it’s down to a lack of managerial oversight, budget alignment that demands a proven return on investment, and engineer education on how much is too much.

Going forward, CIOs need to thread the AI needle between governance that encourages transparency and reasonable spend without stifling innovation.

When shadow tokens result in real costs

The issue is that AI isn’t a traditional line item. Previously, enterprise leaders onboarded software-as-a-service (SaaS) with a good idea of the total cost. An allocated software seat or annual contract was a known quantity. The cloud added some variation (with fluctuations depending on hosting size), but instances were still modelable. AI flips this status quo on its head — the unit of consumption is behavior and the cost is exponential.

And these specifics aren’t immediately apparent at pilot. Tools can appear inexpensive in controlled experiments yet unpredictably scale depending on session length, context window size, model selection and whether agents run in parallel. This is the fallacy of the $20-per-seat enterprise plan — tokens are charged separately at API rates with no ceiling. The final dollar value of any session is set by factors that finance can’t always model in advance, particularly when these decisions usually rest with the engineers themselves.

According to Deloitte, only 21% of organizations deploying agents have a mature governance model, a real concern because they’re token-eating machines. This is what was happening at Uber — Claude Code in agentic mode was autonomously reading codebases, planning changes across dozens of files and opening pull requests. Each step quickly adds up, with Anthropic’s own documentation noting that agents consume approximately seven times as many tokens as standard sessions.

This is shadow IT and shadow AI, evolved. This time, however, many leaders approved the tool in question without guardrails governing consumption. AI hype adds fuel to the fire and normalizes long sessions. Uber’s CTO, for example, described a company-wide shift toward “agentic software engineering” with employees “who are quietly experimenting, quietly shipping and quietly pushing things forward”. This is an exciting way to test the limits of what’s possible, certainly, but it’s also a position that goes a long way to explaining how the company spent its annual AI budget by April.

Shifting the culture from usage to yield

Engineers haven’t done anything wrong here. In fact, they’re adopting and experimenting as instructed, with Uber creating leaderboards and ranking users by token consumption. More use led to a better ranking, reflecting a culture that lauds new ways of doing things. This behavior is known as “tokenmaxxing,” and its principal knock-on effect is shadow tokens — quantity-over-quality processes that leaders struggle to control until they’re fully realized in the budget. Of course, if management treats adoption metrics as performance metrics, then engineers can’t be blamed for using more tokens. The tension is that the teams driving adoption aren’t the ones managing spend.

None of this is meant to dismiss AI’s productivity possibilities and potential return on investment. Developers save 3.6 hours per week, achieve 60% higher pull request throughput and cut onboarding time in half with automation. Meanwhile, Uber shared that roughly 11% of live backend updates were written by agents with no human in the loop. However, these wins aren’t the problem — it’s that too many teams aren’t connecting input to output. I’ve spoken to admins who discovered their token spend had tripled in a single quarter after using heavier models or accidentally doubling up on agentic applications. Nobody knew until the financial damage was done.

Automation needs to happen sustainably with an eye on the bottom line. In my view, a much better metric for achieving this is AI yield — the measurable business or engineering output generated per dollar spent on tokens. Otherwise, without a feedback loop, even genuinely productive teams are flying blind.

Stopping token waste before an AI blowout

Creating that throughline between AI investment and token consumption starts with established financial metrics. This is possible via maximum spend limits (dictated by spend tagging, workload tiering and cost-per-output benchmarks) per team or project. Then, any additional allocation requires approval, closing the loop between the engineers spending the tokens and the leaders paying for them. AI isn’t cheap and teams should demonstrate a bang for their buck.

This is something we do with our engineering team at Hexnode. Resource allocation for Claude Code and Cursor is tied directly to ROI rather than letting consumption run open-ended. Given the pay-as-you-go nature of these tools, a firm usage limit per team offers simple but essential control.

Similarly, there’s room to apply some of the governance principles IT uses for device management. Things like policy enforcement, role-based access, real-time monitoring and automated alerts can flag usage behavior in advance. Uncovering such insights at the token layer works to identify power users and prevent excessive spending.

We also need to encourage cultures that praise outputs that actually achieve efficiency. AI applications that result in shipping faster, reducing rework and cutting review cycles are gains that should be celebrated. If your company hosts leaderboards, frame unnecessary token burn as wasteful rather than valuable. The organizations creating healthier consumption habits work with their engineers to understand not just how to use AI, but what responsible use looks like and what it costs.

This is a conversation teams need to have now. Anthropic just ended flat-rate pricing for programmatic workloads from June 15. Now, agents, continuous integration pipelines and automated workflows draw from a dedicated monthly credit pool billed separately from the subscription. Once that pool is exhausted, agent tasks either stop entirely or overflow to extra billing. Work can either get very expensive or grind to a halt for teams that aren’t prepared.

Getting a grip on shadow tokens means better rules and tools connecting spend to outcomes. Only by building the financial and cultural infrastructure that encourages sustainable adoption can leaders see what they’re spending, connect it to what they’re getting and course-correct before the costs become a crisis. Ultimately, shadow tokens are only invisible if we choose not to look.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Four months of Claude Code — that’s all it took for Uber to burn through its entire annual budget for AI. Token after token, engineers embraced the platform with few control mechanisms tying costs to outcomes. The result was a budget runaway and a clear case study in how limited oversight snowballs into an AI blowout.

This is a phenomenon I like to call “shadow tokens” — AI credits paid for by the company but largely invisible to decision-makers. Too many engineers have the final say over how much they consume and, therefore, what it costs. This all-you-can-eat attitude is part of the reason why Microsoft is reportedly winding down many internal licenses across key engineering teams and why one in five organizations is missing its AI spend forecast by more than 50%.

And the trend is only accelerating. By 2028, Gartner predicts that AI coding costs (driven by this kind of ungoverned consumption) will be as much per developer as the salary companies pay that person.

LLMs and agents introduce a new class of variable cost that scales with behavior rather than headcount, putting enterprises on the hook for tools that balloon with workload. I don’t see this as enterprises overspending because they’re reckless — it’s down to a lack of managerial oversight, budget alignment that demands a proven return on investment, and engineer education on how much is too much.

Going forward, CIOs need to thread the AI needle between governance that encourages transparency and reasonable spend without stifling innovation.

When shadow tokens result in real costs

The issue is that AI isn’t a traditional line item. Previously, enterprise leaders onboarded software-as-a-service (SaaS) with a good idea of the total cost. An allocated software seat or annual contract was a known quantity. The cloud added some variation (with fluctuations depending on hosting size), but instances were still modelable. AI flips this status quo on its head — the unit of consumption is behavior and the cost is exponential.

And these specifics aren’t immediately apparent at pilot. Tools can appear inexpensive in controlled experiments yet unpredictably scale depending on session length, context window size, model selection and whether agents run in parallel. This is the fallacy of the $20-per-seat enterprise plan — tokens are charged separately at API rates with no ceiling. The final dollar value of any session is set by factors that finance can’t always model in advance, particularly when these decisions usually rest with the engineers themselves.

According to Deloitte, only 21% of organizations deploying agents have a mature governance model, a real concern because they’re token-eating machines. This is what was happening at Uber — Claude Code in agentic mode was autonomously reading codebases, planning changes across dozens of files and opening pull requests. Each step quickly adds up, with Anthropic’s own documentation noting that agents consume approximately seven times as many tokens as standard sessions.

This is shadow IT and shadow AI, evolved. This time, however, many leaders approved the tool in question without guardrails governing consumption. AI hype adds fuel to the fire and normalizes long sessions. Uber’s CTO, for example, described a company-wide shift toward “agentic software engineering” with employees “who are quietly experimenting, quietly shipping and quietly pushing things forward”. This is an exciting way to test the limits of what’s possible, certainly, but it’s also a position that goes a long way to explaining how the company spent its annual AI budget by April.

Shifting the culture from usage to yield

Engineers haven’t done anything wrong here. In fact, they’re adopting and experimenting as instructed, with Uber creating leaderboards and ranking users by token consumption. More use led to a better ranking, reflecting a culture that lauds new ways of doing things. This behavior is known as “tokenmaxxing,” and its principal knock-on effect is shadow tokens — quantity-over-quality processes that leaders struggle to control until they’re fully realized in the budget. Of course, if management treats adoption metrics as performance metrics, then engineers can’t be blamed for using more tokens. The tension is that the teams driving adoption aren’t the ones managing spend.

None of this is meant to dismiss AI’s productivity possibilities and potential return on investment. Developers save 3.6 hours per week, achieve 60% higher pull request throughput and cut onboarding time in half with automation. Meanwhile, Uber shared that roughly 11% of live backend updates were written by agents with no human in the loop. However, these wins aren’t the problem — it’s that too many teams aren’t connecting input to output. I’ve spoken to admins who discovered their token spend had tripled in a single quarter after using heavier models or accidentally doubling up on agentic applications. Nobody knew until the financial damage was done.

Automation needs to happen sustainably with an eye on the bottom line. In my view, a much better metric for achieving this is AI yield — the measurable business or engineering output generated per dollar spent on tokens. Otherwise, without a feedback loop, even genuinely productive teams are flying blind.

Stopping token waste before an AI blowout

Creating that throughline between AI investment and token consumption starts with established financial metrics. This is possible via maximum spend limits (dictated by spend tagging, workload tiering and cost-per-output benchmarks) per team or project. Then, any additional allocation requires approval, closing the loop between the engineers spending the tokens and the leaders paying for them. AI isn’t cheap and teams should demonstrate a bang for their buck.

This is something we do with our engineering team at Hexnode. Resource allocation for Claude Code and Cursor is tied directly to ROI rather than letting consumption run open-ended. Given the pay-as-you-go nature of these tools, a firm usage limit per team offers simple but essential control.

Similarly, there’s room to apply some of the governance principles IT uses for device management. Things like policy enforcement, role-based access, real-time monitoring and automated alerts can flag usage behavior in advance. Uncovering such insights at the token layer works to identify power users and prevent excessive spending.

We also need to encourage cultures that praise outputs that actually achieve efficiency. AI applications that result in shipping faster, reducing rework and cutting review cycles are gains that should be celebrated. If your company hosts leaderboards, frame unnecessary token burn as wasteful rather than valuable. The organizations creating healthier consumption habits work with their engineers to understand not just how to use AI, but what responsible use looks like and what it costs.

This is a conversation teams need to have now. Anthropic just ended flat-rate pricing for programmatic workloads from June 15. Now, agents, continuous integration pipelines and automated workflows draw from a dedicated monthly credit pool billed separately from the subscription. Once that pool is exhausted, agent tasks either stop entirely or overflow to extra billing. Work can either get very expensive or grind to a halt for teams that aren’t prepared.

Getting a grip on shadow tokens means better rules and tools connecting spend to outcomes. Only by building the financial and cultural infrastructure that encourages sustainable adoption can leaders see what they’re spending, connect it to what they’re getting and course-correct before the costs become a crisis. Ultimately, shadow tokens are only invisible if we choose not to look.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

I have worked alongside enterprise technology for more than 30 years and watched AI evolve from a lab experiment into the modern boardroom’s core focus. However, the last few years of implementing AI alongside our customers have delivered our most profound reality checks.

The initial hype has faded, leaving CIOs to drive real enterprise value. Based on my experience implementing Google, OpenAI and Anthropic technologies, here are the fundamental, technology-agnostic lessons every leader must anchor their strategy around.

AI as a leadership multiplier

The most common tactical error we see is treating AI as an isolated technology project. What I have observed among our customers is that true success does not come from organizations that define a standalone “AI strategy,” but rather from those leaders that integrate AI into their business strategy.

When our customers isolate AI and define an AI strategy, it inevitably treats it like a “technological toy” to experiment with. This approach yields fragmented, orphaned initiatives that fail to scale because they are fundamentally disconnected from their core corporate objectives. What I learned is that AI is not the ultimate destination; it is a powerful catalyst. We have replaced “What can AI do for our customers?” with a more strategic question, “How does AI accelerate their existing business goals?”

Think of AI like electricity. No modern corporation designs a standalone “electricity strategy.” Instead, all companies route it invisibly across the entire organization to illuminate offices, power production lines and drive communication. AI must be woven into the enterprise fabric in the exact same way, acting as an underlying utility that supercharges your existing operational model.

Integrating AI into the broader business strategy also dictates how we measure success. It forces a shift away from short-term tech vanity metrics and anchors the technology into a long-term roadmap.

When AI remains trapped within the IT department of our customers, we notice that it is relegated to a mere “software experiment.” To become a true competitive advantage, we observed that AI requires intense cross-functional orchestration. This perspective does not diminish the merit of the technical team; their expertise is fundamental for establishing the architecture, data governance and tools your enterprise requires. However, while IT builds the foundational infrastructure, it lacks the organizational authority to decide what should be built on top of it. Only the CEO or the owner of the company can step in to ensure AI leaves the “toy project” phase and integrates into the DNA of the organization.

The requirement for top-down, executive ownership stems from three critical realities observed in the field:

• Silo-smashing and data collaboration: True enterprise AI is data-hungry and that data lives across disparate business lines, finance, operations, marketing and customer service. Only the CEO possesses the cross-functional authority to demand that data silos be dismantled.
• Cultural transformation and fear mitigation: AI triggers widespread anxiety over job displacement across all industries and hierarchies. When relegated to an “IT project,” resistance spikes as teams view it as a threat to their livelihoods. When I saw the CEO lead this cultural shift directly is when I noticed the best results.
• C-Suite education and strategic alignment: The mandate for AI capability cannot just be delegated downward; the transformation must begin at the very top. I have conducted more than 70 presentations for the Board of Directors and C-Level teams. These people need to be actively educated not on technical code, but on specific business use cases, return on investment (ROI) frameworks and how AI resolves core organizational bottlenecks.

PwC’s data found that only 12% of enterprises have achieved both cost and revenue benefits from AI. Those elite 12% succeeded precisely because their CEOs embedded AI extensively across strategic decision-making and cross-functional workflows. AI is simply too disruptive and too critical to be left exclusively in the hands of technical experts. If AI is not on the CEO’s weekly agenda, it is fundamentally missing from the company’s true strategy.

AI as a new operational framework

Traditional IT systems have operated on strict algorithmic certainty: if you input a specific set of data, the system executes an immutable line of code and guarantees the same, predictable output every single time.

AI completely breaks this paradigm. Because modern AI is built on probabilistic models, it does not execute static formulas; instead, it predicts the most likely correct response based on mathematical probabilities. This means that AI solutions carry an inherent, small percentage of uncertainty and variability. A prompt entered today might yield a slightly different, though contextually valid, output tomorrow.

Executive leadership and organizational cultures must be actively educated to accept and navigate this fundamental shift. Traditional quality assurance frameworks for software are designed for a 100% success rate. Applying this rigid standard to AI will paralyze your initiatives, keeping 80% of your projects trapped eternally in the pilot phase. This happened to us in a food and beverage company in Latin America a couple of years ago. After this experience, we started to include conditions in our contracts that tolerate statistical margins of error and still define the project as a success.

In terms of cost calculation, we had to teach CIOs and business managers to forget the monthly subscription model for AI and learn to manage the primary unit of exchange in modern AI: the token.

To understand AI costs, executives must understand how large language models process data. AI models do not read full words; instead, they break text, images or code down into “pieces” called tokens. As a baseline, every 100 words process as approximately 130 to 140 tokens. Because the major AI providers use the token as their currency, your business is billed dynamically based on the exact volume of tokens consumed by every query submitted (input) and every response generated (output).

Many leaders believe AI costs are fixed due to flat-rate enterprise tiers ($25–$30/user). This is a temporary illusion. These venture-capital-subsidized rates mask true operational costs and come with dynamic usage limits. Modeling long-term ROI on them guarantees a severe budget shock when true consumption pricing takes over.

The solution is not to halt AI adoption; doing so means losing your competitive edge. Instead, the cost per token must cease to be treated as a technical footnote relegated to the IT department. It must be elevated to a core business variable.

Risks in the AI adoption model

Since the beginning of the AI boom, I have seen all our customers making a critical tactical error that could cost them heavily in the medium term: they are focusing only on operational efficiency (reducing costs with AI).

I have observed that an alarmingly high percentage of companies remain trapped in pilot phases focused exclusively on short-term cost reduction. Bain & Company’s global Automation and AI Pathfinder Survey found that the largest share of companies measuring their AI initiatives (exactly 40%) realized cost reductions of 10% or less, heavily missing their internal targets. Our customers are putting too many resources and effort into marginal financial gains and in doing so, they are jeopardizing their most valuable assets: service quality, resilience and customer trust.

Utilizing AI solely to slash headcount or cut operational corners is a dangerous trap that introduces severe field liabilities. A financial service organization in Latin America announced that they saved $1 million in customer support by replacing humans with AI chatbots. However, the mid-term reality revealed a different story: a damaged brand reputation due to AI errors and an influx of frustrated clients fleeing because the automated system cannot handle special cases.

Putting a company on an extreme AI diet might make it look leaner on next quarter’s financial statement, but over-indexing on cost-cutting will ultimately leave the business too weak to compete when market dynamics shift. We are now inviting our customers to change the question from “How much money will AI save us?” to “How will we leverage AI to exponentially increase the long-term value of our enterprise?”

Deploying enterprise AI is a marathon, not a sprint, and the terrain changes with every mile. The organizations that thrive in this next era will be those that transition from fascination to discipline, treating AI not as a magic bullet for immediate savings, but as a core capability that demands rigorous governance, architectural foresight and cultural maturity. Navigating this shift requires moving past the theoretical hype and anchoring decisions in raw, field-tested reality.

As we continue to deploy these technologies across industries, the blueprint for success is being rewritten in real time. Let’s keep this conversation going as we map out the future of business intelligence together.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Cybercriminals are actively exploiting a recently discovered vulnerability in Palo Alto Networks firewall and VPN appliances to deploy the Qilin ransomware strain.

A critical authentication bypass flaw (CVE-2026-0257) in Palo Alto GlobalProtect portal and gateway was the common link in a series of intrusions in June, Arctic Wolf Labs warns. Exploitation of the vulnerability came within days of disclosure.

“Post-exploitation tradecraft varied across intrusions, from rapid encryption-only operations to full double-extortion, possibly suggesting multiple affiliates operating under the Qilin ransomware-as-a-service (RaaS) umbrella,” Arctic Wolf’s researchers wrote in a post on the threat.

The campaign against Palo Alto’s VPN client is part of a rising trend that sees ransomware groups increasingly targeting vulnerabilities in network edge tools and devices.

Ransomware takes aim at the edge

Beyond GlobalProtect, Qilin — the most active threat group in Q2 2026, responsible for 14% of attacks, according to NCC Group’s latest Quarterly Cyber Threat Intelligence Report —  has also targeted flaws in Fortinet’s FortiGate, Citrix NetScaler, and Check Point Remote Access VPN.

Check Point warned in June of ransomware attacks against VPNs that still use the deprecated Internet Key Exchange version 1 (IKEv1) protocol. Citrix issued patches in early July for a CitrixBleed-like flaw in its NetScalar devices that had come under attack.

Meanwhile, Fortibleed, a massive credential-compromise campaign, exposed 75,000 FortiGate firewalls in June.  

Qilin is by no means alone in increasing its operations against VPNs and other network security tools.

The Gentlemen, No. 2 on NCC Group’s list with 238 victims in Q2 2026, is noted for breaking into organizations through firewalls, VPNs, and other internet-exposed systems — FortiGate and Cisco products in particular.

Akira, No. 4 on NCC Group’s list (127 victims), is also known for exploiting VPN vulnerabilities and abusing legitimate credentials, primarily versus products from Ivanti, Cisco, and Fortinet.

In the line of fire

Network edge security devices are becoming security liabilities for enterprise security professionals, with an alarming rise in zero-day exploits arising from what experts describe as basic and readily preventable vulnerabilities.

A range of attackers spanning opportunistic hackers to ransomware-as-a-service operators and nation-state sponsored APT (advanced persistent threat) groups are actively exploiting software vulnerabilities in edge devices to hack into corporate networks.

“Although there has not been a material rise in ransomware volume in the last quarter, the trajectory of attacks continues upwards, and VPNs remain an increasingly attractive target,” said Matt Hull, VP and head of cyber intelligence and response at NCC Group.

Unpatched vulnerabilities in edge devices are far from the only software bugs fueling ransomware attacks. For example, last year the Clop ransomware gang hacked hundreds of companies by exploiting zero-day vulnerabilities in Oracle’s E-Business Suite software.

Edge of darkness

VPNs and other internet-facing edge devices remain prime targets for ransomware operators because they provide a direct route into an organization’s network.

“Attackers may exploit an unpatched vulnerability, use stolen credentials, or target weak authentication controls,” said Alexander Leslie, a senior advisor at cyber threat intelligence firm Recorded Future. “In some cases, exploitation begins before organizations have had sufficient time to apply vendor guidance, leaving security teams with a very narrow window to respond.”

VPN exploitation sits alongside other initial access methods, such as phishing, compromised credentials, or software supply chain attacks. The preferred attacker infiltration method varies by campaign and sector but locating security in edge devices carry particular advantages from the perspective of attackers.

“Vulnerabilities in perimeter devices are particularly valuable to attackers because those systems are continuously exposed to the internet and can provide privileged access while bypassing some endpoint controls,” said Leslie.

Dray Agha, senior manager of security operations at managed detection and response firm Huntress, backed up this assessment that exploiting internet-facing VPNs and edge devices remains the “dominant, volume-driven tactic” for ransomware operators because these appliances offer a “direct, publicly accessible gateway straight into the heart of corporate networks.”

Rather than exploiting vulnerabilities in edge devices, attackers more commonly use internet-facing gateways as a means to abuse stolen credentials to break into corporate networks, according to Huntress.

“What we see at Huntress is that the VPN is the site of initial access some 70% of the time, for advanced threat actors,” said Agha. “Overwhelmingly, however, they are not exploiting for access; rather they are using stolen credentials to authenticate to non-MFA’d [multi-factor authentication] user accounts.”

Hardened perimeter

CSOs should treat their network perimeter as hostile territory by enforcing aggressive patch management, applying critical edge device updates within 24 to 48 hours, and mandating strict MFA for all access.

Implementing zero-trust network segmentation to trap attackers and prevent lateral movement if the initial gateway is compromised also helps in making enterprise networks more resilient against attacks, Huntress’ Agha advised.

Phishing-resistant multi-factor authentication, removal of unsupported systems, and close monitoring for unusual authentication or administrative activity also form key components in attack impact mitigation.

Internet-facing assets that are known to be actively exploited should be prioritized as a patching priority.

“Threat intelligence and evidence of active exploitation should help determine which vulnerabilities demand immediate action,” Recorded Future’s Leslie said.

The traditional methods of measuring success no longer tell the full story. Here’s what should replace them — and why.

In brief:

• AI is transforming software as a service (SaaS), and the old ways of keeping score no longer apply.
• Smart companies are evolving new metrics that provide deeper insight into how AI-native software is performing in a new marketplace.
• These changes impact everything from pricing to valuations.

The transformation of the software-as-a-service (SaaS) industry toward AI-native operating companies is rapidly changing the unit of value across the industry.

The traditional metric of seats — which measured access — is rapidly giving way to credits designed to measure work performed. This evolution is upending the industry in multiple ways, impacting everything from pricing to enterprise valuations.

While many companies still cling to seat-based metrics to measure growth, efficiency and durability, the future is likely to be one in which companies utilize a credit-centric metrics framework, with seats and outcomes as the bookends of a spectrum.

Why do software companies need new metrics?

Why the rethink, and why now? There are five major forces that are driving this shift:

1. The unit of value is changing. Seats measured who could access software, and credits measure what the software actually does. But in an AI-native world, agents don’t have seats; they have workloads. Over the past 18 months, every major SaaS platform has moved to some forms of credit or consumption unit.
2. The cost of goods sold (COGS) is exploding. AI inference adds real per-unit costs that scale with usage. In an AI-native world, software companies can’t scale to infinite users at near‑zero marginal cost as before.
3. Buying is moving up the org chart. AI-native applications shift purchasing to higher-level operators — such as line-of-business leaders or chief operating officers — which expands the market from software budgets to labor budgets. And because AI agents replace services as well as software, the total market opportunity is 3x to 10x larger than traditional SaaS.
4. Time to value (TTV) is collapsing. With AI-native tools, customers start seeing meaningful results in weeks rather than quarters. Onboarding and setup are fast, workflows are pre-built, and there’s no need for extensive customer success or professional services — dramatically reducing implementation time and costs.
5. Retention is bifurcating. AI forces clarity in a way that traditional SaaS couldn’t. Products that can provide value become even “stickier” and retain customers. Those that don’t churn faster. In an AI-native marketplace, the middle disappears.

How this shift is impacting pricing

Given how AI-native software is transforming the market, the shift to more variable pricing options is inevitable.

Seats won’t go away completely. Subscription pricing based on the number of users is stable and predictable and will continue to work for some customers. Tokens — the use of pass-through pricing for underlying compute — will fit those customers where the AI feature is commoditized or the buyer wants transparency into costs.

Credits will likely become the dominant architecture because they provide a simple metric for both customers and providers. The vendor sets the conversation ratio between credits and underlying compute, shielding the customer from inference cost details. Credits are easy to understand and can be packaged into annual contracts for multiple features and products.

Finally, the industry will likely see some move toward outcome-based pricing for results such as resolved tickets, recovered revenue or qualified leads. This strategy will mostly be limited to verticals where it is easy to prove AI impacted the result.

Where a software vendor sits on this spectrum is a signal of differentiation and pricing power. Credits are where most defensible AI-native businesses are landing because they balance customer predictability with vendor margin control.

How AI upends classic SaaS metrics

When SaaS was in its infancy, companies settled on key metrics designed to answer a small set of core questions. Are we growing? Are customers using the product? Are we retaining and expanding accounts?

But as AI upends software itself, it is also requiring companies to adopt new metrics to track success. These new metrics fall into three primary buckets, rebuilt around the pricing spectrum described earlier and the trend toward credits as the primary frame:

Revenue composition

• Committed credit annual recurring revenue (ARR) vs. burndown ARR: Measuring the credits sold on annual commitment vs. those consumed and replenished. This is the single most important split for valuation. Committed credits behave like subscription and burndown behaves like usage.
• Credit utilization rate: The percentage of purchased credits consumed per period. This is a leading indicator of renewal sizing.
• Credit burn velocity: How fast is a customer consuming their credits, and is that consumption increasing or decreasing quarter over quarter? This metric predicts expansion or contraction before it shows up in ARR.
• Effective price per credit: The real revenue per credit after discounts, overage and rollover, which can detect revenue leakage and help companies set smarter guide rails.

Margin reality

• Credit margin: The gross profit the company earns per credit after subtracting inference costs. This is the core economic unit for AI-native, usage-based businesses — the replacement for gross margin per seat used in SaaS.
• Inference-adjusted gross margin: By carving out AI inference costs separately in the P&L statement, you can see true AI margins, avoid hiding deterioration inside blended SaaS margins, and clearly distinguish AI economics from legacy SaaS economics.
• Compute leverage ratio: This metric measures how efficiently the business converts compute spend into revenue. It shows whether your AI margins are improving as you scale.
• AI-adjusted “Rule of 40”: This updated metric recalibrates the traditional growth and profitability benchmark to account for AI’s lower gross margins and variable inference costs, giving a more accurate picture of business health for AI-native companies.

Behavioral and value signals

• Time-to-first outcome: Replaces traditional onboarding metrics. Tracks how fast a customer reaches their first measurable result.
• Adoption: AI-native adoption is measured by workflow penetration and active agent density, not seat count. As AI replaces human-driven usage, the unit of adoption shifts from people to automated workflows and agents.
• Net credit retention (NCR): Credit-volume retention across the customer base, tracked separately from net recurring revenue to avoid price-change impact.

Along with these new metrics, the industry’s transformation is prompting companies to retire or recalibrate old SaaS measures, including per-seat ARR as a primary key performance indicator (KPI), traditional magic number calibrated to subscription dynamics, unadjusted Rule of 40, customer success metrics tied to human touchpoints, and blended gross margin without AI COGS carve-outs.

What does this mean for enterprise value calculations?

As the internal metrics of success change, so do the ways the investment community measures growth and long-term viability.

Increasingly, a company’s valuation multiple depends on whether its revenue behaves like committed subscription ARR or volatile usage ARR, and the commit‑to‑burndown ratio is the metric investors use to decide where the company fits.

For example, a business with 80% committed credit ARR could trade closer to subscription comps and one with 80% burndown could trade closer to usage comps even though both have the same types of customers. Being able to proactively explain the commit‑to‑burndown mix can help companies avoid undervaluation.

In addition, utilization is expected to replace net promoter scores and seat usage as the primary predictor of churn or expansion. Low utilization guarantees downsizing at renewal, so companies must track utilization cohorts the same way SaaS tracks logo retention cohorts today.

We’re also seeing an inversion of the operating model, with R&D and COGS moving up the P&L and sales and marketing (S&M) and customer success (CS) moving down or sideways. The net operating leverage profile is structurally different from classical SaaS, and the cost-to-scale curve looks different too.

Finally, credit margin engineering is a hidden value-creation lever. The gap between price per credit and cost per credit is set by the software vendor and can be optimized. Most operators have barely started managing this rigorously, and the ones who do will pull away on margin.

What this means for leaders, boards and investors

The shift from classic SaaS metrics to new AI‑native measures isn’t cosmetic. It represents the seismic change the industry is experiencing as AI matures and transforms products and organizations.

While these metrics — and perhaps others yet to be determined — may evolve over time, there is no doubt they are already changing how AI companies allocate capital, price products, incent sales teams, evaluate performance and communicate with investors.

It’s important to remember that SaaS metrics were practical tools for a specific era of software. As that era draws to a close, winning companies will choose new metrics that shape behavior and drive smart decision-making.

The views reflected in this article are the views of the author and do not necessarily reflect the views of Ernst & Young LLP or other members of the global EY organization.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Leadership wants to scale AI. Budgets are tripling. Adoption is up.

Then the CFO asks the question every board now asks: which of these initiatives is actually profitable?

Most organizations cannot answer that question, not because they lack visibility into cost, but because the cost data they have was never designed to produce that answer.

Applying lessons learned from managing cloud spend won’t be a fix for the AI and ROI quandary. True, cloud taught a generation of CFOs that billing without business context is noise. So to get cloud ROI, they stitched two data sources together: cost data plus business data. AWS reveals which account, which region, which tag, which resource. Merge in customer and product mappings on top and the ROI of the cloud spend comes into focus.

But AI is harder. It requires three data sources: cost, business, and telemetry—the automatic collection of data from disparate sources that helps to clarify the whole picture of what happened and why. An executive or engineering lead can have AI invoices and customer revenue. But they have no way to connect them to business value. The token count on the OpenAI invoice does not specify which customer triggered which call, which feature it served, or whether the prompt produced a business outcome. That data does not exist in the provider’s billing.

AI providers won’t fix this problem

The situation is not likely to change anytime soon because AI providers are not in the business of attributing an enterprise’s costs to that enterprise’s customers. Instead, AI providers are in the business of selling tokens. The granularity they expose is the granularity their billing systems require, not the granularity a CFO requires.

Not convinced? Compare what AWS gives you to what an AI provider gives you.

AWS billing exposes resource IDs, account hierarchies, region, SKU, tag metadata, usage by the minute. Every dollar can be attributed to a workload, a team, a customer segment if it was tagged correctly. The data is rich enough that mature FinOps teams built unit economics on top of it years ago.

An AI provider invoice gives you tokens consumed by model, with optional grouping by API key. That is the resolution. No request-level attribution. No customer ID. No feature mapping. No prompt outcome. No retry identification. Multi-step agent workflows collapse into a token count. Imagine a large bank receives a multi-million dollar AI invoice each month. But it has no visibility into what parts of the business were responsible for what parts of the cost so cannot allocate them.

If an enterprise wants to know what AI cost drove which customer or feature, it has to capture that data itself, inside an application, before the call leaves it. 

Three required sources

Building AI ROI measurement requires three data sources, stitched together in a single model.

1. Cost data, normalized across providers. Every AI provider delivers cost differently. OpenAI invoices in one taxonomy, Anthropic in another, fine-tuning vendors and inference platforms each in their own. Cloud GPU costs sit in AWS or Azure billing. Vector database costs land in Pinecone or Snowflake invoices. None interoperate by default. Normalization is necessary but not sufficient. It will put all your AI costs in one schema. It does not tell you what they produced.
2. Application-layer telemetry. This is the source most organizations are missing, and the one that makes AI ROI structurally different from cloud ROI. It requires instrumenting AI calls inside your application across six categories: request-level tracing tied to a customer or session ID; feature attribution tied to the product surface that triggered the call; agent-step capture for multi-step workflows; retry and fallback identification so recovery costs don’t get attributed to primary calls; model selection logging that records which model was chosen and why; and outcome capture that ties each call to whether it produced business value. None of this data exists in the provider’s billing. All of it has to be captured at the moment the call is made and stored in a system that can be stitched to the cost data.
3. Business data. Revenue, customer segments, product hierarchies, and feature usage. The same business data already feeding your CRM and analytics stack, mapped to the customers and features the telemetry layer attributes calls to.

Stitched together, the three sources produce the unit economics every AI investment decision now requires: cost per customer interaction, margin per feature, profitability per agent workflow, ROI per model choice. None of these can be calculated from billing data alone. None can be calculated from telemetry alone. They require all three sources, modeled together in a way that maps cost to outcome.

Why agentic AI makes this urgent

Single-call inference is the easy case. One request, one cost, one customer, one outcome.

Agentic workflows are different. An agent decomposes a task into multiple steps. Each step calls a model. Some steps fall back to a different model when the first fails. Some steps retry on a poor result. Some steps invoke external tools that themselves cost money. A single user request can produce dozens of inference calls across multiple providers, with the cost compounding in ways the provider invoice cannot disaggregate.

If telemetry does not capture agent-step granularity, no one will know which steps are profitable. Aggregate costs will show up three weeks later in the invoice. By then, the workflow has been running at scale, customers are onboarded, and unprofitable paths have been retried thousands of times.

When agents make the calls, the volume of cost-generating events without business context attached grows by an order of magnitude. The window for instrumenting this before it becomes unmanageable is closing.

What changes when the three sources come together

Once the three sources are stitched together, the AI investment conversation changes.

Five different ways to build the same AI capability stop looking equivalent. They converge on adoption metrics and diverge by 10x on cost. The team picks the approach that delivers a similar business outcome at one-fifth the cost, because the team can finally see the difference. Product teams design features with margin awareness from the architecture phase, not from the post-launch budget review. Engineering teams choose model architectures with cost-per-outcome data alongside latency and quality. Leadership evaluates AI initiatives the way they evaluate any other capital allocation: on unit economics, not on the engagement chart. Aggregated invoices track the cost per customer interaction. Engagement metrics reveal margin per feature. Gut-instinct model selection is checked against real cost-per-outcome model selection results. 

Within seconds, everyone can see which AI features are profitable, which should scale, and which should be killed. This is the insight everyone is looking for and companies that achieve it will optimize the benefits of AI.

The build trap

AI costs are compounding now. The board is not waiting 18 months for an internal project to reach production.

The temptation to build it anyway has never been sharper. AI coding tools have changed what a small engineering team can ship in a quarter. The instrumentation layer looks tractable. The cost normalization looks like a weekend project. The semantic model feels like something a senior engineer could draft over a sprint.

It is a trap. Three reasons.

Volume is the first. A production AI footprint generates millions of telemetry events per hour, and that volume scales with agentic adoption. Real-time ingestion, correlation, and attribution at that scale is not the same problem as vibe coding a prototype in an afternoon. It is a permanent operational system that has to be right every minute of every day.

The vendor landscape is the second. Cost data arrives in delayed billing windows from providers with non-interoperable schemas. Schemas change without notice. New AI providers enter the landscape monthly, each with its own taxonomy and metering. The system is not built once. It is maintained against a moving target that moves faster than most internal release cycles.

The third is what the first two add up to: this is business-critical infrastructure. The CFO and the board are going to make capital allocation decisions on the data this system produces. When schema drift goes unnoticed for two weeks, when an agent telemetry stream stops correlating to a vendor that quietly changed its billing API, the cost of being wrong is not a sprint of cleanup. It is a quarter of misallocated capital.

The build-vs.-buy question for engineering leaders has changed. It’s not “can we build this?” The honest answer is yes. The real question is whether the marginal hour of your strongest engineers is best spent stitching cost data to telemetry to business outcomes, or building the AI products that produce the revenue the cost data is measuring.

The capability is reproducible in weeks. The choice is whether to spend the next 18 months building it, or the next 18 months acting on it.

—

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.

As the CSO grows in prominence, security leaders are increasingly earning a seat at the executive table, reporting directly to the CEO with the expectation to help drive business strategy and ensure organizational success.

Reporting to the CEO unlocks greater access and influence for security leaders, and while CSOs who report to their organization’s CIO still have clout, it’s a very different experience picking up the phone to speak directly with the CEO as a strategic partner.

Regardless of reporting structure, CSOs must clearly understand what they are being tasked to solve. That might sound simple, but making the leap to being a CEO’s direct report requires a new perspective, a different set of skills, and a business-level focus on metrics to do so.

We asked several current CSOs, CEOs, and IT staffing experts for advice on how security executives can best navigate a direct reporting relationship with their CEO. Offering insights below are George Gerchow, CSO at Bedrock Data and member of the IANS faculty; Matt Chiodi, CSO of Cerby; Chris Schueler, CEO at Cyderes; and Greg Fuller, vice president of the Technology Skills Suite at Skillsoft.

1. Understand how the CEO views your role

Most CEOs expect that, when you report directly to them, you fully own your functional area. Whether it’s cybersecurity, operations, or finance, they look to you as the expert in that domain. The CEO may have opinions, but ultimately, you are expected to lead and provide direction.

CEOs expect their CSO to be a true strategic partner, not just a risk reporter — connecting cybersecurity to revenue protection, regulatory compliance, customer trust, and operational resilience. In turn, CSOs should expect CEOs to treat governance as a strategic enabler, not a bureaucratic necessity.

2. Power up on skills vital to your organization at an executive level

On the technology side, AI and machine learning, cloud security, incident response, zero trust architecture, and governance, risk, and compliance (GRC) are the areas where threats evolve fastest and strategic leadership has the greatest impact. 

Equally important are “power skills”: communication, critical thinking, adaptability, and emotional intelligence. The ability to translate complex risk into business terms is what separates a strong CSO from a purely technical one. Skills, not titles, define effectiveness in the eyes of a CEO.

3. Take advantage of your direct access

Direct access to the CEO will enable you to influence strategy, shape resilience planning, and ensure cybersecurity is treated as a business imperative rather than a cost center. That authority is strongest when the CEO understands cybersecurity as a strategic lever, not just a technical function. 

While a direct reporting relationship gives you access to the CEO, it also comes with the responsibility to operate at that level. You need to provide clear, executive-level visibility into your cybersecurity program.

4. Brush up on business translation

A CSO who leads with business alignment will always carry more influence when they can translate risk into business language rather than technical jargon. Building programs that must survive an IPO, a FedRAMP audit, and real customer scrutiny forces you to tie security to revenue and trust.

The most valuable skill is translation — defining technical risk in terms of executive action and business impact that a CEO and a board can act on. You must build trust through transparency. These are the human skills that complement technology, creating a collaborative human-AI dynamic where leaders make faster, better-informed decisions. 

5. Treat conversations as risk assessment opportunities

Highly effective security leaders treat every business conversation as a risk conversation in disguise. That mindset is what largely separates a great CSO from a great technologist. Earn the CEO’s trust by speaking business first, security second. Translate every risk into revenue, reputation, or regulatory exposure.

Remember, a good CEO wants a translator, not an alarm system. They expect no surprises, a clear read on the risks that matter, and a security leader who helps the business move faster rather than slowing it down.

6. Define what a successful relationship should look like and put it in writing

Regardless of the reporting relationship, start by defining the end goal and putting it in writing. It will evolve over time, but having that initial clarity is critical. This is especially important when you’re new in a role and aiming to make your first 60, 90, or 120 days, and your first year, successful. In such cases, it’s essential to align early.

Do that collaboratively, and document it.

7. Prioritize trust and candor

The CEO needs to trust that the CSO isn’t sandbagging, and the CSO needs enough psychological safety to deliver bad news fast. When those conditions exist, security becomes a strategic asset — not a cost center.

To that end, focus on clear communication above all, and present yourself as part of a team, not a solo player. Stay calm under pressure during incidents, and treat people as peers rather than policing them. The leaders who last build trust before they need it.

8. Treat governance as a strategic competitive advantage

The strongest partnerships also share a commitment to governance as a competitive advantage.

Governance is the brakes that let you drive fast safely. When a CSO and CEO are aligned on that principle, the organization can innovate with AI while maintaining oversight and protecting against unnecessary risk. The result is an organization that does not just react to threats but builds resilience into how it operates.

9. Set clear goals and measure progress

Setting clear goals and measuring progress against those goals is essential. When expectations are clear, the areas you need to focus on become much clearer. It doesn’t solve every problem, but aligning early with your leadership, whether that’s a CEO or a CIO, can significantly reduce the pressure you may feel.

Also, never let your boss be surprised. This is where being clear on goals and consistently tracking both leading and lagging metrics becomes especially important, particularly in a direct reporting relationship with the CEO.

10. Be willing to endure challenge and discomfort

Finally, persistence and a willingness to endure discomfort for something that matters more than the pain itself are critical to surviving in this relationship. The role of a cybersecurity leader is often thankless. If you’re doing your job well, no one really notices.

In a single month, one enterprise reportedly spent half a billion dollars on AI. A consultant told Axios that the client had handed its workforce AI licenses, set no usage limits and let the meter run until finance noticed. The figure is spectacular, and it is the wrong thing to fear. That half-billion-dollar accident is only the visible part of a quieter, far larger failure. Worldwide AI spending is forecast to reach $2.52 trillion in 2026, more than any technology category in a generation, and by the most cited measure, roughly 95 percent of it returns nothing. Boards read that as proof that the technology does not work. The evidence points somewhere less comfortable, and it is not a technology problem at all. Most boards cannot see it because they are reading the wrong number: They track failure when the number that matters is allocation. The discipline that separates the winners is not technical. It is how they allocate capital across time, and how willing they are to stop. The hardest discipline in the AI era is not adopting faster. It is allocating honestly and refusing to judge a three-year bet on a six-month cycle.

The number everyone quotes, and no one acts on

The headline statistic is now familiar. MIT’s Project NANDA, in its 2025 study The GenAI Divide, found that about 95 percent of enterprise generative AI pilots produced no measurable impact on the P&L, while roughly 5 percent captured nearly all the value. S&P Global Market Intelligence found that the share of companies abandoning most of their AI initiatives jumped from 17 percent to 42 percent in a single year, with the average organization scrapping 46 percent of its proofs-of-concept before production. Gartner expects more than 40 percent of agentic AI projects to be canceled by the end of 2027, citing escalating costs, unclear business value and inadequate risk controls. And the pattern predates generative AI: RAND found that more than 80 percent of AI projects fail, roughly twice the rate of comparable work that does not involve AI.

Read as a technology story, these numbers say AI does not work. Read correctly, they say something more useful. MIT’s own authors located the cause not in model quality but in a learning and integration gap. The winners were not running better models. They picked one problem, executed and worked well together. Purchased solutions reached production about 67 percent of the time, while internal builds succeeded roughly a third as often. Gartner’s own spending forecast notes the same pivot, with CIOs scaling back ambitious internal builds in favor of commercial solutions that promise more predictable value. None of that is a verdict on the technology. It is a verdict on allocation: What gets funded, for how long and against which yardstick. The popular prescription, heard in every boardroom this year, is to measure harder and prove value sooner. That advice quietly repeats the mistake, because forcing a three-year bet to prove itself sooner is precisely how you kill it. The fix is not more measurement. It is measuring each bet against the right clock and subtracting the ones that miss.

The six-month cycle problem

Return to that 95 percent, because the way it is measured is the whole argument. Much of the reported failure is judged on a short clock, with a pilot counted as a failure if it has not shown a measurable financial return within roughly six months. The single most quoted number in enterprise AI is therefore a six-month yardstick applied to every initiative, including the bets designed to pay back in three years. The headline failure rate is not only a measure of AI. It is a measure of impatience.

The most expensive mistake in enterprise AI is a timing error. Enterprises have been spending heavily on AI for more than two years, and 2026 is the year boards are demanding returns. The multi-year bets funded during the 2024 and 2025 scale-up are only now far enough along to be judged. When a board reviews an initiative, it applies the yardstick it knows, which is quarterly return. That yardstick is correct for an efficiency project and ruinous for a capability bet. A workflow automation that should pay back in two quarters and a foundational data and agent capability that pays back in three years are not the same instrument, yet they are reviewed in the same meeting against the same metric.

This is the heart of the divide. The 5 percent did not simply pick better projects. They judged each project against its own horizon. McKinsey’s enduring Three Horizons model made this discipline standard in corporate strategy a generation ago: near-term, emerging and long-term bets are funded and measured differently. AI erased that discipline because the hype compressed every timeline into the current quarter. The result is two failure modes that appear opposite yet share a common root. Organizations kill three-year bets at month six because they miss a metric the bet was never designed to hit. And they keep funding six-month theater for years because it is visible, safe and never asked to prove a return. Both are allocation failures. Neither is a technology failure.

Subtraction is a strategy

There is a second discipline, the 5 percent share, and it is the one boards find hardest. They subtract. Every credible study of the failure rate describes the same chaotic pattern underneath it: Initiatives are abandoned late, without criteria, after the money is spent and the credibility is gone. Disciplined organizations do the opposite. They decide the conditions for stopping before they start, and they stop on schedule. Subtraction is not the absence of strategy. It is the strategy. Capital removed from a failing bet is capital available for a surviving one, and the survivors are where the entire return lives.

This reframes the 42 percent abandonment figure. Abandonment is not the problem. Undisciplined abandonment is. An organization that liquidates a position the moment it breaches a pre-agreed kill line is practicing portfolio hygiene. An organization that lets a doomed pilot run until someone loses patience is paying full price for a lesson it could have bought at a discount. The 5 percent who won were not smarter. They were patient in the right places and ruthless in the wrong ones.

The HALT framework: Horizon, Allocation, Liquidation, Tracking

Treating AI as a portfolio rather than a pile of pilots requires four disciplines, and the organizations that execute well put all four in place before the next funding cycle, not after the next failure. The name is deliberate. The discipline most enterprises lack is the willingness to halt the wrong bets in time to fund the right ones.

Component 1: Horizon. Classify every AI initiative by its true payoff horizon before it is funded. Horizon 1 covers efficiency plays that should return value within two quarters. Horizon 2 covers capability bets, data foundations, agent platforms and integration work that pays back in roughly 6 to 18 months. Horizon 3 covers transformation bets that take eighteen months to three years or longer. Each horizon carries its own success metric, set at funding time. A Horizon 1 yardstick never judges a Horizon 3 bet. This single rule prevents the most common and most expensive error in the portfolio.

Component 2: Allocation. Decide the split across horizons deliberately, as a board-level capital decision, not as the accidental sum of whatever pilots happened to win approval. A practical reference point, borrowed from decades of innovation-portfolio practice, is roughly 70% to near-term value, 20% to capability, and 10% to transformation. The exact ratio is yours; the discipline is to choose and defend it. The failure mode is an unmanaged portfolio: 90 percent scattered across disconnected Horizon 1 experiments, with nothing compounding into the Horizon 2 capability that the buy-and-integrate winners actually built.

Component 3: Liquidation. Attach a kill line to every initiative at the moment it is funded: A named milestone, a date and an owner empowered to stop it. If a bet misses its horizon-appropriate milestone, it is liquidated, and capital is reallocated on schedule without debate over sunk costs. The absence of a pre-agreed kill line is not patience. It is an unpriced liability that the board has almost certainly not been shown.

Component 4: Tracking. Report the portfolio to the board on a fixed cadence using a single instrument: The AI Portfolio Scorecard. Not a deck of project updates, but a single view of allocation by horizon, burn against milestone, liquidation decisions taken and capital reallocated to survivors. The cadence is the control. A portfolio reviewed once a year is a portfolio managed by hope.

THE AI PORTFOLIO SCORECARD: SCORE EVERY INITIATIVE BEFORE IT IS FUNDED

Score each criterion: 0 = not present, 1 = partially documented, 2 = fully verified. Total out of 12. Bands: 0 to 4 = DO NOT FUND  |  5 to 8 = CONDITIONAL  |  9 to 12 = FUND.

THE LIQUIDATION GATE: RUN AT EVERY BOARD REVIEW BEFORE CONTINUING FUNDING

Any unresolved FAIL = stop funding, liquidate the position, reallocate the capital to a survivor and record the decision on the scorecard.

The cost of the timing error

The financial case follows the pattern and is consistent. Consider two organizations that funded the same class of Horizon 3 bet: A domain-specific agent platform meant to compound over three years. The first review was conducted at month six against a quarterly return test, found no payback and killed it, booking the write-off as a lesson about AI being overhyped. Its competitor classified the same work as Horizon 3, set an 18-month capability milestone, protected funding through two review cycles and shipped to production within the window the work actually required. One organization spent its money to learn that it lacks allocation discipline. The other spent comparable money and now owns a capability its rival has abandoned and cannot quickly rebuild. The dollars on the two income statements are similar. The competitive positions are not.

The governance return the board has been waiting for

Allocation discipline does two things at once. It stops the bleed by liquidating failures on a schedule rather than at the point of exhaustion. And it concentrates capital where the entire return lives, in the small number of bets that survive their horizon. The 5 percent figure is not a ceiling imposed by the technology. It is the current yield of an industry allocated by hype. An organization that classifies by horizon, allocates on purpose, liquidates on a line and tracks on a cadence is not trying to beat the technology. It is trying to beat its own indiscipline, and that is a far more winnable contest.

The board conversation about AI returns is coming for every organization, and it arrives the moment the spending outpaces the story. When it does, the CIO will be asked a simple question: Where did the money go? The leaders who can answer will not show a pile of pilots. They will show a portfolio: What was funded, against which horizon, what was liquidated and when, and what the survivors are now worth. Subtraction is a strategy. The only question is whether you are practicing it on purpose or about to learn it by accident.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

The head of the US government’s AI testing institute, Chris Fall, has resigned about three months after taking charge of the Center for AI Standards and Innovation (CAISI), the federal organization responsible for evaluating advanced artificial intelligence models for safety and security.

Current National Institute of Standards and Technology NIST Director Arvind Raman will serve as acting CAISI Director following Fall’s departure while continuing to oversee the Commerce Department office responsible for the institute, the Daily Signal reported, citing two people familiar with the matter.

A Commerce Department spokesperson who spoke to the publication did not disclose a reason for the resignation.

Fall assumed leadership of CAISI in April after the Trump administration reorganized the former US AI Safety Institute under NIST. The institute develops methodologies for evaluating frontier AI models and works with AI developers on voluntary technical assessments covering areas such as cybersecurity, model misuse, reliability and other risks associated with increasingly capable AI systems.

The leadership change comes as governments and AI companies continue developing technical approaches for evaluating frontier AI models while enterprises expand deployments of generative AI and agentic AI across business operations.

In recent months, the Commerce Department has taken a more active role in AI policy involving advanced models, placing greater attention on how the federal government evaluates technologies with potential national security implications.

Continuity matters more than personalities

CAISI works with AI developers such as Anthropic, Google’s DeepMind and OpenAI on voluntary evaluations of frontier AI models and develops methodologies for testing model capabilities and risks. The institute does not regulate AI developers or certify commercial AI systems.

For enterprises, those evaluations are one source of technical information alongside vendors’ own testing, third-party security assessments and internal AI governance programs.

Sanchit Vir Gogia, chief analyst at Greyhound Research, said enterprises should focus less on the individual leading the institute and more on whether its technical work continues with the same level of consistency and transparency.

“Leadership churn at CAISI weakens the signal long before it weakens the science,” Gogia said. “The testing has not stopped. Its authority simply does not travel as cleanly once the leadership does not.”

According to Gogia, the more important question for enterprises is not whether the institute’s evaluation work will continue but whether the processes supporting those evaluations remain stable.

“The instinct is to ask whether the pipeline is breaking,” he said. “The more useful question is where the pipeline now sits.”

Enterprises still carry the burden of AI governance

Gogia said organizations should continue treating government-led AI evaluations as one input into their governance processes rather than as evidence that a model is inherently safe for enterprise deployment.

“A government evaluation was always a signal, never a certificate,” he said. “A signal loses value the moment its issuer becomes unpredictable.”

He said enterprises should instead monitor whether CAISI maintains consistent evaluation methodologies, continues publishing technical findings and preserves continuity within its research teams under interim leadership.

“The name on the door is not the signal. The behaviour underneath it is,” Gogia said.

Gogia also cautioned against linking Fall’s resignation to recent Commerce Department actions involving AI policy or export controls, noting that there is no public evidence connecting the two.

“CAISI evaluates; it does not enforce export controls, because it holds no such power,” he said. “This is not a testing body reaching for enforcement. It is enforcement reaching past the testing body.”

With Raman assuming the role on an interim basis, the next significant milestone for enterprises will be the appointment of a permanent director, and whether the institute’s evaluation programs continue without disruption, the analyst said.

Gogia said the successor’s mandate may prove more important than the individual selected.

“A CAISI result is not a safe harbour,” he said. “It informs an obligation; it does not discharge one.” NIST did not immediately respond to a request for comment.

Something interesting happened during an internal evaluation of AI meeting transcription tools at Tungsten Automation. The products worked. They weren’t bad. But sitting across from the pricing, we kept asking the same question: what exactly are we paying for?

We already had a secure enterprise AI environment. Building a meeting summary workflow took days, not months. We customized the outputs, injected our own internal context, and controlled security our way instead of working around someone else’s roadmap. We built it. It works better. We own it.

That’s not a knock on those vendors. It’s a signal of something more fundamental happening across enterprise software.

The moat was never the product

For two decades, SaaS rode a favorable asymmetry: building internal tools was hard, integrations were messy, and even modest automation required developers and long timelines. Buying was faster and cheaper than building. That asymmetry fueled the explosion of SaaS into every corner of the enterprise stack.

AI is collapsing that asymmetry. Large language models and agentic workflows can orchestrate APIs, move data between systems, generate interfaces, and automate business logic with a fraction of the engineering effort required even two years ago. The integration friction that once protected entire product categories is evaporating.

The vendors most exposed are not the deeply embedded enterprise platforms. They’re the lightweight workflow layers, the products that essentially put a polished interface on top of accessible data and relatively straightforward processes. Reporting dashboards. Meeting tools. Narrow productivity applications. These products created value by simplifying implementation. That rationale is getting harder to sustain when implementation is no longer the real barrier.

Here’s the part most analyses miss: it’s not just that AI makes development faster. It’s that agents change the integration model entirely. For 30 years, enterprise software was built for humans navigating UIs. Agentic systems don’t use UIs. They call APIs, read from multiple sources simultaneously, and move data freely across systems. The switching costs that once made incumbent software sticky are collapsing, because an agent doesn’t care which UI it used last quarter.

The SaaS that survives

The question isn’t whether SaaS survives. It’s which SaaS survives.

The companies with durable positions are not the ones with the cleanest interface. They’re the ones that transfer operational risk customers genuinely cannot absorb themselves. Compliance. Regulatory certification. Accumulated domain expertise. Liability.

Think about compliant invoicing across 140 countries. That’s not a workflow someone builds in a sprint. The certifications alone take years. A single regulatory change in one jurisdiction can break an AP process for a global enterprise overnight. Customers don’t pay for that capability because it’s technically complex. They pay because they cannot afford to own the risk of getting it wrong.

That’s the distinction that matters: AI lowers the cost of building software. It does not lower the cost of absorbing risk. The vendors who understand this are building durable businesses. The ones who don’t are quietly subsidizing their customers’ internal build programs.

Software sells features. Platforms sell accountability.

The prototype trap

The danger for enterprise buyers right now is overcorrection. Every successful prototype looks like a cost-saving opportunity. Very few survive the jump to production.

Building a workflow with generative AI is becoming straightforward. Maintaining it is not. Models evolve. Outputs drift. Governance requirements tighten. What worked cleanly in a controlled environment behaves differently at scale, and the failure mode is worse than traditional software. Rule-based automation, when it fails, fails obviously. Agents fail silently, confidently, at scale, often with a completely reasonable-sounding explanation.

Engineering teams that take on AI-powered systems need to solve for observability, model drift, access controls, audit trails, and long-term maintenance ownership. In regulated industries, they need to demonstrate exactly how the system reached every decision. That’s not a weekend project. That’s an ongoing operational commitment that compounds over time as models change and regulatory requirements evolve.

Before a team decides to replace an external platform with internal AI tooling, the honest question isn’t, “Can we build this?” The real question is, “Are we prepared to own this in production, for years, as the underlying models change beneath us?” Sometimes the answer is yes. Often the answer is no, and the true cost only becomes visible after the vendor contract is canceled.

Build vs. partner: a sharper frame

The build vs. buy framing has always been too binary. The right question is build vs. partner.

Partner for the capabilities where risk transfer, regulatory complexity, and domain expertise create genuine value your team cannot replicate. Build for the capabilities that actually differentiate your business from your competitors. Don’t burn your best engineers rebuilding compliant invoice processing or production-grade document extraction. Those aren’t competitive advantages. They’re table stakes, and someone else has already paid the cost, across decades, to make them reliable.

The organizations getting this right are honest about where they create unique value. They focus development there, and partner for everything else. The ones getting it wrong are vibe-coding solutions to non-differentiating problems while their actual competitive moat goes unattended.

The true value of software

We’re not watching the death of SaaS. We’re watching the end of the friction-based value proposition: the idea that software is worth renewing because integration used to be painful. That rationale is largely gone.

What survives is software that does something customers cannot reasonably replicate internally: absorb risk, maintain regulatory compliance, deliver operational reliability at scale, and bring genuine domain expertise into a production-grade system that someone else already stress-tested for years.

The vendors who recognize this are already repositioning around accountability, governance, and outcomes. The ones who haven’t will find the next renewal conversation noticeably harder.

Software sells features. Platforms sell accountability. That distinction is about to separate a lot of winners from a lot of cautionary tales.

—

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.

Every enterprise AI conversation seems to begin with the same question: Which model should we use?

I understand why. Models are visible. They have names, benchmarks, release notes, pricing pages and impressive demos. They are easy to compare in a leadership meeting. One model promises better reasoning. Another offers a larger context window. Another appears faster, cheaper or more specialized.

But after years of working around enterprise platforms, integration layers, cloud migration, middleware, production operations and mission-critical systems, I see the AI conversation differently.

The model matters. But it is not where most enterprises will struggle next.

The next AI bottleneck is the infrastructure behind the model.

I do not mean only GPUs, cloud capacity or data storage. I mean the full enterprise operating layer that allows AI to work safely in the real world: data pipelines, identity, APIs, messaging, observability, security controls, deployment automation, cost governance, auditability, support ownership and recovery design.

That layer is what determines whether AI remains an exciting experiment or becomes a trusted business capability.

Pilots hide the hard part

Most organizations can build an impressive AI pilot. A small team can connect a model to a dataset, create a workflow and show a use case that works well in a controlled setting.

The harder part starts when that pilot moves into a real production process.

That is when practical questions show up. Who owns the data quality? What systems can the AI access? How do we trace which prompt, policy or retrieval flow produced a specific answer? What happens when an API slows down, a queue backs up or a downstream system is unavailable?

To me, these are not model problems. They are infrastructure problems.

This is where many enterprises are now headed. The first phase of AI was experimentation. The next phase is operationalization, and that is where the real gap becomes clear.

McKinsey has made a similar point in its work on agentic AI, noting that the next phase of value depends less on isolated tools and more on redesigning workflows, operating models and enterprise execution around agents.

AI pilots can survive on enthusiasm. Production AI requires architecture.

AI is becoming an integration problem

The more I look at enterprise AI, the more it feels like an integration challenge.

In large organizations, I have seen how messaging platforms, integration gateways, deployment pipelines, monitoring tools and cloud infrastructure can decide whether a digital capability succeeds or fails. AI will be no different. Even the strongest model will struggle if the data, middleware, identity layer and operational controls around it are weak.

AI does not work in isolation. It needs context from systems of record, clean data from different business areas, secure access to APIs, event streams, workflows, knowledge repositories, monitoring tools and legacy systems.

That is why the CIO question is changing.

It is no longer just, “Which AI tool should we buy?”

It is becoming, “Can we safely operationalize intelligence across the business?”

This is where agentic AI matters. Autonomous AI only creates real value when the architecture around it can make its actions safe, traceable and useful.

A model can generate an answer. Infrastructure determines whether that answer is secure, timely, explainable, governed and connected to the right workflow.

For example, an AI assistant that summarizes customer or order information may look like a model use case. But underneath, it depends on access control, fresh data, reliable APIs, logging, encryption, monitoring and policy enforcement.

If the answer is wrong, people may blame the model. But the real failure may have started with stale data, weak integration, poor access design, missing observability or an unreliable downstream system.

That is why CIOs should not judge AI only by model capability. The enterprise system around the model matters just as much.

Latency will become a trust issue

In traditional technology operations, latency is often treated as a performance metric. In AI-enabled workflows, latency becomes a trust issue.

When an employee asks an AI assistant for help and the response takes too long, the employee stops using it. When a customer-facing workflow becomes slow, the customer abandons it. When an AI agent waits on multiple backend calls, the entire business process feels unreliable.

This becomes even more important as organizations move from simple chat interfaces to agentic workflows. A single AI-driven action may include identity checks, context retrieval, policy validation, model reasoning, API calls, business-rule execution, logging and human approval.

Each step adds latency. Each dependency adds a possible failure point.

A model may be fast in a benchmark but slow inside an enterprise process. That difference matters.

This is where platform engineering becomes essential. Enterprises need reusable patterns for AI workloads: approved connectors, secure retrieval methods, queue-based decoupling, caching strategies, deployment pipelines, monitoring dashboards and standard rollback procedures.

Without those patterns, every AI initiative becomes a custom build. Custom builds may work for pilots, but they do not scale across a large enterprise.

Observability has to expand

Traditional monitoring tells us whether infrastructure is healthy. Is the server up? Is CPU high? Is memory exhausted? Is the application returning errors?

AI needs that, but it also needs more.

We need to know what data was retrieved, which model was used, which prompt version was active, which user initiated the request, which policy was applied, how long each step took and whether the output passed validation.

We also need to detect new forms of risk: unusual usage patterns, repeated failed tool calls, unexpected cost spikes, sensitive data exposure, weak retrieval results or an AI workflow attempting actions outside its intended boundary.

In production AI, observability is not only about uptime. It is about confidence.

If a business leader, auditor, regulator or security team asks why an AI system made a recommendation, the answer cannot be, “The model said so.” The enterprise needs traceability. It needs evidence. It needs operational context that engineers, risk teams and business owners can understand.

This is one of the biggest gaps I see in AI strategy. Many organizations are investing in models and use cases, but not enough in the control plane required to manage them.

Data readiness is still underestimated

AI has exposed an uncomfortable truth: many enterprises are not as data ready as they think.

Data is often duplicated across platforms, described differently by each team, governed inconsistently and refreshed on different schedules. Access rules may be clear in one system but unclear in another. Even basic business definitions can change from department to department.

AI does not fix that automatically. In many cases, it makes the problem more visible.

A bad report may be questioned. A bad AI answer may sound confident enough to be trusted.

That is a real risk.

Being data-ready for AI is not just about connecting a vector database or indexing documents. It requires clear ownership, lineage, classification, quality checks, retention rules, access boundaries and a shared understanding of which data should be used for which purpose.

The same principle applies to resilient cloud-native design. In my IEEE TechRxiv paper, “Enabling Fault-Tolerant Multicast in Cloud-Native Architectures” I explored how reliability, observability and fault tolerance become foundational requirements when critical workloads stretch across hybrid and multi-cloud environments.

CIOs already understand this because they have lived through enterprise resource planning programs, cloud migration, integration modernization, cybersecurity transformation and analytics initiatives. The lesson is familiar: technology cannot outrun data discipline forever.

Security cannot be added later

As AI moves from answering questions to acting, security becomes much more important.

An assistant that summarizes information carries one level of risk. An agent that can open a ticket, update a record, trigger a workflow, approve a request or contact a customer carries a very different one.

The more AI can do, the more identity, authorization, least privilege, separation of duties and human approval matter.

Enterprises should be careful not to grant AI broad access just to speed up a pilot. That may seem harmless in development, but it can become dangerous at scale.

AI access should be treated like any other privileged enterprise capability: limited, logged, reviewed and easy to revoke.

The NIST AI Risk Management Framework is a useful reference point here because it frames AI risk as something organizations must govern, map, measure and manage continuously rather than something handled only at the end of deployment.

Security teams should be involved early, not at the end. The goal is not to slow innovation. The goal is to build a platform where safe innovation becomes repeatable.

The CIO has to define the operating model

AI is creating pressure from every direction. Boards want productivity. Business teams want automation. Employees want better tools. Vendors are pushing new features. Security teams are watching risk. Finance teams are watching cost. Customers expect faster, smarter experiences.

The CIO sits in the middle of all of it.

That is why the CIO’s role cannot stop at choosing tools or approving pilots. The CIO has to define how AI will actually operate across the enterprise.

That means answering practical questions. Which architecture is approved? Which data sources can be trusted? How are AI workflows deployed, monitored, supported and governed? How are costs controlled? How do teams reuse common patterns instead of rebuilding the same foundation each time?

This work may not be as exciting as a model demo, but it is what separates sustainable AI from short-term experimentation.

The winning organizations will not be the ones with the most pilots. They will be the ones with the strongest AI operating layer.

They will build reusable platform patterns, strengthen data governance, design access properly, monitor AI behavior end to end and measure success by business improvement, not only model performance.

The model still matters. But the enterprise behind the model matters more.

A powerful model on weak infrastructure will eventually disappoint the business. A capable model on strong infrastructure can deliver real value because it can be trusted, secured, scaled and improved.

That is the shift CIOs need to lead.

The next AI bottleneck is not the model. It is whether the enterprise behind the model is ready.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Providers and deployers of AI systems: You only have a couple of weeks left until you must explicitly inform users when they are interacting with AI content.

To assist in the effort, the European Commission (Commission) has published guidelines to help AI deployers get in line with the AI Act’s transparency obligations, which will begin to go into effect on August 2.

After that, companies providing AI systems must alert users when they are interacting with AI. They must also tell users when they have been exposed to deepfakes, “emotion recognition,” or biometric categorization systems, or when they are given AI-manipulated content in matters of “public interests without human review or editorial control.”

Henna Virkkunen, the Commission’s executive VP for tech sovereignty, security and democracy, said in a statement, “with today’s guidelines, the Commission supports the smooth and effective application of the AI Act to make AI systems interacting with people such as chatbots and AI agents and AI content more transparent and trustworthy. These guidelines support providers and deployers in meeting their obligations under the AI Act, while helping citizens know when they are interacting with AI.”

Systems must include machine-readable markers to reveal such content, to reduce “the risk of deception and manipulation” and build public trust in AI.

“Generative systems have collapsed the cost of producing convincing content while the cost of judging it stands where it always stood,” said Sanchit Vir Gogia, chief analyst at Greyhound Research. This requirement is “an attempt to restore friction to that imbalance.”

A company’s non-compliance could result in fines anywhere from €750K (about $856K) to €15M (about $17 million), or even up to 3% of its total worldwide annual revenue.

Transparency requirements

The EU AI Act’s transparency requirements apply to “natural or legal persons,” public authorities, agencies, or other bodies that develop AI systems, or have them developed, and place them on the EU market or into use under their name or trademark. This means all companies, regardless of whether or not they are EU-based.

“Systems placed on the European market, put into service there, or producing outputs used there are inside the field, wherever the developer sits,” Gogia noted.

Applicable systems must be intended to interact directly with “natural persons”; these systems include AI-enabled chatbots or conversational agents, AI companions, or coding agents. However, AI-enabled tools like recommender systems, spam filters, authentication, search and retrieval, transcription, text and code auto-completion, or predictive maintenance do not fall under the rule.

Specific outputs such as AI-generated text, images, video, and audio must contain a machine-readable mark. Deepfakes and public interest-related text created by AI without human review or control must be clearly labeled, however, deepfake content that is “artistic, creative, satirical, or fictional” is largely exempt.

AI content must be marked with one of three labels: “AI,” “Fully AI-generated,” or “Partially AI-modified.” For instance, “Fully AI-generated” applies when news summaries, music, art, or videos have been created without any human oversight (apart from prompting), while “partially AI-modified” could mean a person’s face is swapped into an authentic photograph to create a deepfake.

The three icons are publicly available for free use; enterprises can download zip files in PNG and SVG formats.

Most of the Act’s transparency rules begin to go into effect on August 2. But AI systems placed on the market before then will have some leeway; they must be in compliance by December 2.

However, a four-month allowance “on one obligation, for one population of systems, contingent on one procedural step, is not a strategy,” Gogia emphasized. Enterprises should plan to comply by August 2 and “treat any relief that arrives as margin.”

A consistent code of practice

Along with the transparency guidelines, the Commission has introduced a code of practice that essentially serves as a gesture of good faith. When signed, it can provide “legal certainty” and a “simple and practical” way to demonstrate compliance with the AI Act, according to the Commission. Signatories can also collaborate through the ‘Signatory Taskforce,’ which will share practices and advance technologies around marking and labeling practices.

Providers that choose not to sign must comply through other methods and demonstrate that those methods are “adequate” through assessment by surveillance authorities, according to the Commission.

Non-signatories “keep their flexibility, and will face more case-by-case scrutiny for it,” said Gogia.

Criteria for compliance

Shashi Bellamkonda, principal research director at Info-Tech Research Group, pointed out that the transparency requirements apply to content only when three criteria are met: It has been published, is informative to the public, or is on matters of public interest.

B2B business content or blogs may not need an AI disclosure if they do not meet these criteria, he noted. Also, published text that has undergone human review or is under editorial control does not need to be labeled. Editorial control means that a person must hold the ultimate legal responsibility for the publication of the content.

Many companies like Google, Adobe, and LinkedIn have already established ways to identify images marked as AI-generated. Meta has made it a requirement, but the creator has to add the AI-generated label, Bellamkonda said.

“This is a good move for guardrails around public information, and companies with good compliance and ethical oversight may not have to worry about this,” he noted. But as a general practice, companies should disclose AI-generated content and state whether it has been human reviewed.

Creating a transparency pipeline

Establishing full transparency means identifying who carries the responsibility for the content, whether the marking survives real use, not just testing, and what evidence will defend the decision, Gogia said.

Concerns cluster around responsibility, durability and evidence. Several organizations usually touch one piece of content, and none controls the whole chain, which is why contracts become the “pressure point,” he said. Most current agreements were written to deliver software and say “almost nothing” about provenance persistence, verification access, or evidence retention.

The durability concern is the most difficult, Gogia noted, because marking performs well in controlled settings but “badly in ordinary life.” Meta, for one, said its invisible watermark was designed to survive cropping; a published test, however, found the company’s preview detector missed 55% of cropped images.

“CIOs should ask which platform can actually provide evidence before believing its dashboard,” said Gogia.

Disclosure of AI use must be “clear, distinguishable and accessible,” he emphasized. “A notice buried in lengthy terms, or reachable only through determined clicking, satisfies nobody, least of all a market surveillance authority.”

Sustained compliance is a “living control” requiring a central record of systems, duties and evidence; testing taking place where the user meets the control rather than where the developer built it; and continuous supplier assurance. Enforcement will vary by country, so keep one common baseline with local overlays, Gogia said.

His advice: Inventory every system that talks to people, generates content, or gauges sentiment; classify provider and deployer roles; place disclosures at first interaction; define substantive human review; keep the evidence.

Marks and provenance signals should be tested after content undergoes cropping, compression, translation, transcription, and other editing, Gogia said. A useful audit starts from a real output and follows its “pulse” through generation, editing and publication, identifying at “each beat” the responsible party, the surviving mark, and evidence for exceptions. Missed labels should also be traced for root cause and recurrence.

To ensure compliance, before August 2, enterprises need a prioritized inventory, live disclosures on the highest-risk use cases, and a “named owner for every control,” he noted. In the first 30 days, they should stabilize and test; in the first 90 days, push requirements into procurement processes as a standing discipline. Procurement must secure commitments on marking methods, known failure modes, and evidence access, with explicit notice if/when any of them change.

“The sensible architecture is a common transparency baseline carrying traceability, responsibility, and evidence, with jurisdictional overlays for language, sector rules, and local practice,” Gogia said.

This article originally appeared on CIO.com.

Providers and deployers of AI systems: You only have a couple of weeks left until you must explicitly inform users when they are interacting with AI content.

To assist in the effort, the European Commission (Commission) has published guidelines to help AI deployers get in line with the AI Act’s transparency obligations, which will begin to go into effect on August 2.

After that, companies providing AI systems must alert users when they are interacting with AI. They must also tell users when they have been exposed to deepfakes, “emotion recognition,” or biometric categorization systems, or when they are given AI-manipulated content in matters of “public interests without human review or editorial control.”

Henna Virkkunen, the Commission’s executive VP for tech sovereignty, security and democracy, said in a statement, “with today’s guidelines, the Commission supports the smooth and effective application of the AI Act to make AI systems interacting with people such as chatbots and AI agents and AI content more transparent and trustworthy. These guidelines support providers and deployers in meeting their obligations under the AI Act, while helping citizens know when they are interacting with AI.”

Systems must include machine-readable markers to reveal such content, to reduce “the risk of deception and manipulation” and build public trust in AI.

“Generative systems have collapsed the cost of producing convincing content while the cost of judging it stands where it always stood,” said Sanchit Vir Gogia, chief analyst at Greyhound Research. This requirement is “an attempt to restore friction to that imbalance.”

A company’s non-compliance could result in fines anywhere from €750K (about $856K) to €15M (about $17 million), or even up to 3% of its total worldwide annual revenue.

Transparency requirements

The EU AI Act’s transparency requirements apply to “natural or legal persons,” public authorities, agencies, or other bodies that develop AI systems, or have them developed, and place them on the EU market or into use under their name or trademark. This means all companies, regardless of whether or not they are EU-based.

“Systems placed on the European market, put into service there, or producing outputs used there are inside the field, wherever the developer sits,” Gogia noted.

Applicable systems must be intended to interact directly with “natural persons”; these systems include AI-enabled chatbots or conversational agents, AI companions, or coding agents. However, AI-enabled tools like recommender systems, spam filters, authentication, search and retrieval, transcription, text and code auto-completion, or predictive maintenance do not fall under the rule.

Specific outputs such as AI-generated text, images, video, and audio must contain a machine-readable mark. Deepfakes and public interest-related text created by AI without human review or control must be clearly labeled, however, deepfake content that is “artistic, creative, satirical, or fictional” is largely exempt.

AI content must be marked with one of three labels: “AI,” “Fully AI-generated,” or “Partially AI-modified.” For instance, “Fully AI-generated” applies when news summaries, music, art, or videos have been created without any human oversight (apart from prompting), while “partially AI-modified” could mean a person’s face is swapped into an authentic photograph to create a deepfake.

The three icons are publicly available for free use; enterprises can download zip files in PNG and SVG formats.

Most of the Act’s transparency rules begin to go into effect on August 2. But AI systems placed on the market before then will have some leeway; they must be in compliance by December 2.

However, a four-month allowance “on one obligation, for one population of systems, contingent on one procedural step, is not a strategy,” Gogia emphasized. Enterprises should plan to comply by August 2 and “treat any relief that arrives as margin.”

A consistent code of practice

Along with the transparency guidelines, the Commission has introduced a code of practice that essentially serves as a gesture of good faith. When signed, it can provide “legal certainty” and a “simple and practical” way to demonstrate compliance with the AI Act, according to the Commission. Signatories can also collaborate through the ‘Signatory Taskforce,’ which will share practices and advance technologies around marking and labeling practices.

Providers that choose not to sign must comply through other methods and demonstrate that those methods are “adequate” through assessment by surveillance authorities, according to the Commission.

Non-signatories “keep their flexibility, and will face more case-by-case scrutiny for it,” said Gogia.

Criteria for compliance

Shashi Bellamkonda, principal research director at Info-Tech Research Group, pointed out that the transparency requirements apply to content only when three criteria are met: It has been published, is informative to the public, or is on matters of public interest.

B2B business content or blogs may not need an AI disclosure if they do not meet these criteria, he noted. Also, published text that has undergone human review or is under editorial control does not need to be labeled. Editorial control means that a person must hold the ultimate legal responsibility for the publication of the content.

Many companies like Google, Adobe, and LinkedIn have already established ways to identify images marked as AI-generated. Meta has made it a requirement, but the creator has to add the AI-generated label, Bellamkonda said.

“This is a good move for guardrails around public information, and companies with good compliance and ethical oversight may not have to worry about this,” he noted. But as a general practice, companies should disclose AI-generated content and state whether it has been human reviewed.

Creating a transparency pipeline

Establishing full transparency means identifying who carries the responsibility for the content, whether the marking survives real use, not just testing, and what evidence will defend the decision, Gogia said.

Concerns cluster around responsibility, durability and evidence. Several organizations usually touch one piece of content, and none controls the whole chain, which is why contracts become the “pressure point,” he said. Most current agreements were written to deliver software and say “almost nothing” about provenance persistence, verification access, or evidence retention.

The durability concern is the most difficult, Gogia noted, because marking performs well in controlled settings but “badly in ordinary life.” Meta, for one, said its invisible watermark was designed to survive cropping; a published test, however, found the company’s preview detector missed 55% of cropped images.

“CIOs should ask which platform can actually provide evidence before believing its dashboard,” said Gogia.

Disclosure of AI use must be “clear, distinguishable and accessible,” he emphasized. “A notice buried in lengthy terms, or reachable only through determined clicking, satisfies nobody, least of all a market surveillance authority.”

Sustained compliance is a “living control” requiring a central record of systems, duties and evidence; testing taking place where the user meets the control rather than where the developer built it; and continuous supplier assurance. Enforcement will vary by country, so keep one common baseline with local overlays, Gogia said.

His advice: Inventory every system that talks to people, generates content, or gauges sentiment; classify provider and deployer roles; place disclosures at first interaction; define substantive human review; keep the evidence.

Marks and provenance signals should be tested after content undergoes cropping, compression, translation, transcription, and other editing, Gogia said. A useful audit starts from a real output and follows its “pulse” through generation, editing and publication, identifying at “each beat” the responsible party, the surviving mark, and evidence for exceptions. Missed labels should also be traced for root cause and recurrence.

To ensure compliance, before August 2, enterprises need a prioritized inventory, live disclosures on the highest-risk use cases, and a “named owner for every control,” he noted. In the first 30 days, they should stabilize and test; in the first 90 days, push requirements into procurement processes as a standing discipline. Procurement must secure commitments on marking methods, known failure modes, and evidence access, with explicit notice if/when any of them change.

“The sensible architecture is a common transparency baseline carrying traceability, responsibility, and evidence, with jurisdictional overlays for language, sector rules, and local practice,” Gogia said.

For diversified enterprises, few operating model questions are as persistent or polarizing as centralization versus decentralization. Decentralization promises speed, ownership, and local responsiveness. Centralization promises efficiency, standardization, and leverage. Both can be right. Both can be wrong. The challenge is that many organizations end up with both models operating at once, without enough clarity about why.

The result of fragmented systems, duplicated capabilities, inconsistent data, rising IT spend, and a complexity tax that compounds over time is familiar to many CIOs. What starts as autonomy can become architectural sprawl. What starts as enterprise leverage can become bureaucracy. And as companies modernize core platforms, integrate data, and scale capabilities like AI, the tension becomes harder to ignore.

Paul Krebs has lived that tension from multiple vantage points. Most recently as CIO and chief transformation officer at Koch Industries, and previously a technology and transformation leader at The Coca-Cola Company, he’s worked in environments where business units value autonomy, enterprise scale matters, and the wrong operating model can slow progress just as easily as the wrong technology architecture.

His conclusion isn’t that CIOs should pick a side, but they need a more intentional form of centralization, one that starts with business architecture, clarifies decision rights, and continually revisits where capabilities should sit as the organization matures.

Centralization: a design choice, not a doctrine

In diversified organizations, decentralization often starts as the default because it aligns with how the business creates value. Local businesses understand their customers, markets, regulatory environments, and operating realities, and giving them decision rights can increase speed and accountability.

In Krebs’ experience, the default model often leaned toward decentralization, he says, with the belief that optimizing for customers and markets would allow different businesses to be as responsive as possible to the specific customers and markets they served. But that logic isn’t complete. Leaders also need to ask whether there’s a compelling case where a more centralized approach can generate additional value, accelerate progress, or optimize investments.

Digital transformation created one of those moments. Krebs recalls around 2016 when Koch challenged its businesses to build multi-year digital transformation roadmaps. The ambition was there, but the capabilities to execute at the necessary pace weren’t evenly distributed. In response, the organization invested more aggressively from the center, building shared services and centers of expertise in areas such as business transformation, enterprise applications, and data and analytics.

The purpose was acceleration, not control. Centralizing those capabilities helped accelerate learnings, capability building, and their ability to deploy new solutions at scale. But the move wasn’t treated as permanent. “There was always a belief that the centralization push should be re-looked at on a regular basis, not thought of as a forever decision,” he says.

Know what belongs at the center

Over time, Krebs learned that  the capabilities most likely to remain centralized were those where scale, consistency, and risk management mattered more than local differentiation. Infrastructure, collaboration platforms, cybersecurity, cloud management, FinOps, and the help desk were natural candidates to remain shared services.

Other areas were more nuanced. Some application capabilities moved back into the businesses as local maturity increased. Many data and insights capabilities also moved closer to the business once teams had built enough muscle to own them. Meanwhile, certain emerging capabilities such as spatial technologies like AR/VR remained centralized because it didn’t yet make sense for each business to build them independently. Many companies have lived this journey as well, for example, with gen AI, which often started with a center of excellence, and then evolved into a more decentralized approach, enabling teams across the business to innovate quickly.

That distinction avoids the trap of treating the enterprise as one uniform operating model. “Both models can be successful, and both have advantages,” he says. “That’s what makes the balance so difficult.”

Centralization provides a clearer path to execution at scale and cleaner decision rights, but it requires change management and careful attention to bureaucracy. Decentralization provides ownership and speed, but it can also over index toward preference versus real differentiation, he adds, while making architecture harder to scale later.

Don’t confuse standardization with centralization

One of the most important distinctions Krebs makes is between centralization and standardization. Many organizations treat them as interchangeable, but they’re not.

“You can have a centralized team that can manage the nuances of different requirements,” Krebs says. “You can also have a centralized standard platform that can be used in a decentralized manner.”

That distinction opens up more operating model choices. A company may centralize a platform but decentralize how business teams configure or use it. It may standardize process patterns while keeping execution close to the region or business unit. It may also centralize architectural governance while allowing local teams to move quickly within defined guardrails.

This is especially important in global organizations, where regional needs are real but not always unique. Krebs advises leaders to examine whether local requirements can be made more generic and reusable. The risk is solving each local requirement as a one-off, so the better path is to understand the underlying requirement, build it in a way that can scale, and still allow local teams to execute within the standard model.

Let business architecture lead technology architecture

Few topics expose the centralization tension more clearly than ERP consolidation. Many diversified companies, particularly those shaped by acquisition, end up with dozens or hundreds of ERP instances. Some leaders push for massive consolidation. Others prefer to build integration layers on top of the existing environment.

Krebs’s starting point is neither technology nor cost. It’s business architecture. “The easiest and most effective path is when the IT or systems architecture follows and aligns to the business architecture,” he says.

If the business is truly going to operate processes separately, separate systems may be appropriate. But if the organization has numerous teams, processes, and tools, leaders need to ask whether there’s enough differentiation and value to justify that complexity.

The same logic applies to M&A. Companies can get into trouble when integration synergies are held hostage by ERP migration timelines. Instead, Krebs advises starting with the business integration strategy. Understand where the synergies are, how the business architecture should come together, and then decide whether the IT architecture needs to be fully integrated, or whether a data layer, reporting platform, or other integration approach can deliver value faster.

Make the cost of complexity visible

CIOs in decentralized companies often face a frustrating dynamic. The business wants autonomy and speed, but the same leadership team still questions why IT spend is high relative to benchmarks. Krebs says the answer starts with cost alignment and visibility.

In environments with a mix of centralized and decentralized services, Krebs saw centralized capabilities like infrastructure, help desk, and security perform well on benchmarks. More decentralized areas, such as BI, reporting, and commercial applications, often had more redundancy and higher cost.

The point isn’t to blame the business but make the economics of complexity visible. CIOs need to show how flexibility in one area may require multiple systems, data stores, or teams elsewhere. “I understand we want flexibility here,” Krebs says. “But leaders must see when that flexibility may cost the company money, and be clear on whether the value justifies it.”

That shifts the conversation from IT cost to business service economics. A single aggregate IT spend number is rarely useful in a decentralized environment. More helpful is a capability-based view that shows which areas are scaled efficiently, which are fragmented, and where the business architecture is driving the technology cost structure.

Revisit the model as maturity changes

For a new CIO entering a decentralized environment, Krebs cautions against immediately declaring that too many things need to be centralized. The better starting point is curiosity. “I would begin with just trying to understand why they’ve made the decisions they have,” he says.

From there, CIOs can engage leaders in a conversation about the target operating model, connecting business architecture to technology, data, and organizational capabilities. Once the direction is clear, he advises CIOs to work with the willing. Find the parts of the organization that already see the need for change, prove the model there, and scale from demonstrated success.

Regardless of execution, though, the right model changes over time. A low-maturity capability may benefit from centralization because the organization needs to build talent, avoid reinventing the wheel, and accelerate learning. As maturity grows, decentralization may make more sense because business teams need flexibility to adapt quickly. Once maturity is high and patterns stabilize, the organization may be ready to centralize again to leverage scale.

“Once I’ve decided I’m going to start with centralized or decentralized, you don’t necessarily need to stay in that model,” Krebs says. “You need to be continually revisiting the operating model as your organization matures and evolves.”

That may be the heart of smart centralization. It rejects the false permanence of operating model decisions, and recognizes that autonomy and scale are both valuable, but in different places, at different times, for different reasons.

CISOs at government organizations and universities have an unexpected ally coming to their aid: OnlyFans models.

For some time, hackers have exploited weaknesses in the websites of universities or government departments to host scams or malware, using content stolen from the OnlyFans website as bait to attract victims.

Now, according to security researchers at Upguard, the fightback has begun: creators of adult content on OnlyFans are leveraging Google search results and the protection offered by copyright law to break up the traffic distribution systems created by bad actors.

These distribution systems work in three stages: entry points using adult or other content to attract and capture web traffic, a routing system sends it to destination sites, and those sites monetize the traffic through scams and malware. It has proved to be a lucrative business for the scammers.

Google recognizes the approach and calls such actors SEO parasites as they benefit from the reputations of other organizations — in particular government or academic sites, which Google views as having high authority.

Since the creators of OnlyFans content are also the copyright holders, they are able to issue Digital Millennium Copyright Act (DMCA) take-down notices for the stolen content posted by the bad actors to other sites. Upguard was able to track this through Google’s DMCA Transparency Report, and through the Lumen Database, another tracker of takedown notices, to which it was granted research access.

“This allows us to identify likely compromised sites: government and university domains advertising unlicensed adult content,” Upguard said.

The OnlyFans creators’ action has two benefits for the operators of the affected websites: The adult content associated with their domain disappears from Google search results, no longer affecting their reputation — and if they receive takedown notices for such content they can check their webservers for the vulnerabilities that enabled the bad actors to post it there in the first place.

This article first appeared on CSO.

CISOs at government organizations and universities have an unexpected ally coming to their aid: OnlyFans models.

For some time, hackers have exploited weaknesses in the websites of universities or government departments to host scams or malware, using content stolen from the OnlyFans website as bait to attract victims.

Now, according to security researchers at Upguard, the fightback has begun: creators of adult content on OnlyFans are leveraging Google search results and the protection offered by copyright law to break up the traffic distribution systems created by bad actors.

These distribution systems work in three stages: entry points using adult or other content to attract and capture web traffic, a routing system sends it to destination sites, and those sites monetize the traffic through scams and malware. It has proved to be a lucrative business for the scammers.

Google recognizes the approach and calls such actors SEO parasites as they benefit from the reputations of other organizations — in particular government or academic sites, which Google views as having high authority.

Since the creators of OnlyFans content are also the copyright holders, they are able to issue Digital Millennium Copyright Act (DMCA) take-down notices for the stolen content posted by the bad actors to other sites. Upguard was able to track this through Google’s DMCA Transparency Report, and through the Lumen Database, another tracker of takedown notices, to which it was granted research access.

“This allows us to identify likely compromised sites: government and university domains advertising unlicensed adult content,” Upguard said.

The OnlyFans creators’ action has two benefits for the operators of the affected websites: The adult content associated with their domain disappears from Google search results, no longer affecting their reputation — and if they receive takedown notices for such content they can check their webservers for the vulnerabilities that enabled the bad actors to post it there in the first place.

An automated chatbot working for Anthropic this month shot down a Wiz researcher’s security hole report, saying that it “falls outside of the Claude Code threat model.” That was news to the security researchers at Wiz. 

It also turned out to be news to Anthropic execs, who had a very different view. 

In reality, Anthropic was one of many victims of the hole — including Amazon, Google and Cursor, among others. But what makes the incident so bizarre is that, far from dismissing the threat, Anthropic had detected it before the security researchers and had even patched it before the researchers alerted them. 

As these AI bots are wont to do, the bot didn’t merely reject the request. It confidently explained its rationale, even though its reasoning was wrong. 

“This falls outside our current threat model,” the chatbot said, according to a report by Wiz. “When the user first starts Claude Code in a directory, they must confirm that they trust the directory prior to starting the session. The scenario you describe involves a user explicitly confirming a permission prompt inside of a directory containing a malicious symlink, which falls outside of the Claude Code threat model.”

That researchers said Anthropic management later clarified the situation: “The symlink warning in the Edit/Write permission dialog shipped in v2.1.32 (Feb 5, 2026), nine days before this report was submitted to us. It was added as part of proactive security hardening based on internal review. The decline to comment was an autoreply from our triage system.” 

An autoreply from our triage system? How many other make-believe replies did this system send? And what level of damage is Anthropic exposing itself to? 

This is not just an Anthropic issue. There have been numerous enterprise bot glitches in communications  with customers. Some of my favorites include:

• Bots that chose on their own to cancel customers. (This actually was another Anthropic incident.) In this case, an Anthropic bot cancelled the AI account of a Swiss company that depended on the service. A lawyer got involved and the account was restored within a day — minus 80% of the data. Oops.
• A Cursor bot decided to log customers off when they switched devices, which it shouldn’t have done. The bot then emailed customers and lied that, “The logouts were expected behavior under a new login policy.” A Fortune story detailed how “the news spread rapidly in the developer community, leading to reports of users cancelling their subscriptions, while some complained about the lack of transparency. Cofounder Michael Truell finally posted on Reddit acknowledging the ‘incorrect response from a front-line AI support bot’ and said it was investigating a bug that logged users out. ‘Apologies about the confusion here,’ he wrote.”
• Voters in Scottish elections were tricked by government AI bots that “variously invented fictitious scandals, gave the wrong date for the election, claimed wrongly that voters in Scottish elections needed ID at polling stations and placed candidates in the wrong contests.”
• And let’s not forge the classic story about the Air Canada bot, where “Air Canada was ordered to compensate a customer after its chatbot gave incorrect information about the airline’s bereavement fare policy. The tribunal found that Air Canada was responsible for information provided through its website, including the chatbot.”

Let’s be clear, here: Bots should be limited to relaying only pre-approved scripts. 

Generative AI allows for far greater chatbot sophistication, but that also means the chance of far greater errors. This is untenable in any business function. And when the app is pretending to be a human — and interacting with human customers — it’s even more unacceptable.

Organizations usually face the same questions concerning XLAs: What should we measure, who owns the data, how should incentives work, and how will this change provider behavior after signature.

There are no easy answers either, but after advising clients in MSP relationships with major providers, I’ve seen what works and what doesn’t. Successful XLA programs rarely start with massive transformation, nor rely on perfection before adding experience accountability to the contract.

Start with the right metrics

The first concern I hear is what to measure. MSPs often steer that discussion toward metrics already in their reporting stack. That’s a trap.

Unlike SLAs, which measure operational outputs, XLAs should focus on employee experience and business outcomes. The strongest programs start with three to five high-signal metrics tied to the employee journeys creating the most friction. More than that and the program loses focus before it gains traction.

I typically recommend starting with employee satisfaction scores, perceived lost productivity time, repeat incident rates, task completion success, and ease of getting support. Then focus early measurement on common employee experiences like service desk interactions, employee onboarding, application reliability, and device performance.

Trying to measure everything is understandable, but it’s also one of the fastest ways to stall an XLA program.

Precisely define roles and responsibilities

This is the part of XLA contract design where I spend the most time with clients, and it’s the part that major MSPs are most likely to leave vague if you let them. Accenture and TCS both have mature commercial teams skilled at agreeing to things in principle while avoiding specific accountability in writing. Don’t let that happen here.

Employee experience isn’t solely the vendor’s responsibility. It’s genuinely shared, which is a more productive framing than pure vendor accountability, but only if the split is clearly spelled out. This is what I’ve found works in practice.

Customer responsibilities

• Selecting tools and platforms
• Managing data infrastructure
• Sharing experience data openly with the provider
• Supporting internal improvement initiatives that the provider flags

Vendor responsibilities

• Running the measurement cadence
• Delivering monthly experience reporting
• Identifying and surfacing improvement opportunities from the data
• Executing operational improvements within agreed timelines

Without this level of specificity, XLA programs almost always become reporting exercises. The data gets collected, the scorecard gets presented, and nothing actually changes.

Build flexible targets

One of the biggest mistakes in XLA design is treating experience targets like traditional SLAs,  setting once at contract signing and left unchanged for years. Employee expectations, workforce patterns, and technology environments, after all, evolve constantly. A target that feels ambitious in year one may become meaningless by year three.

The strongest XLA contracts include formal reviews every three to six months to recalibrate targets, align with business priorities, and raise expectations as experience improves. This prevents providers from locking in easy wins and coasting. When providers resist review cycles, it’s often a sign they believe the targets can be met on autopilot, a red flag in any XLA program.

Use the right scoring method

One overlooked XLA best practice is how experience scores are calculated. Point-in-time scores can be distorted by outages, isolated incidents, or low survey participation, and providers sometimes exploit that volatility.

I advise clients to calculate official XLA scores using rolling two-month averages instead of snapshots. It creates a more stable and accurate view of experience trends, and makes operational timing games much harder. Most importantly, define the scoring methodology explicitly in the contract. Don’t leave it to be worked out operationally after signing.

Structure incentives carefully

Relying on penalty-only incentives is one of the most expensive XLA mistakes. On paper, the model is simple: miss the target, pay the penalty. In practice, it drives the wrong behavior. Providers focus on protecting themselves instead of improving employee experience, optimizing survey timing, and managing averages rather than solving problems collaboratively.

I’ve seen this repeatedly in Infosys, HCL, and TCS relationships. The strongest XLA structures combine risk and reward where providers earn meaningful upside for exceeding targets, innovating, and improving outcomes. Penalties still matter, especially in mature programs, but they can’t be the only lever otherwise the contract becomes another SLA model with better branding.

Define escalation processes

When experience scores fall below threshold, the contract needs to specify what happens next. This sounds obvious, but I’ve reviewed many service delivery measurement frameworks in clients’ incumbent MPS contracts that specify financial consequences without defining any collaborative process to address the underlying problem.

The escalation language I push clients to include specifies:

• a joint review process triggered when scores fall below threshold.
• root cause analysis expectations and timelines.
• remediation planning requirements with named owners on both sides.
• timelines for corrective action and progress reporting.

The framing matters as much as the mechanics. Escalation should be positioned as collaborative problem-solving, not blame assignment. Contracts that turn every missed score into a commercial dispute damage the relationship when provider engagement matters most. The best MSPs treat escalation as a shared diagnostic exercise, not a contractual confrontation.

Establish an operating rhythm

Signing the contract is the beginning, not the end. In my experience, the organizations that get the most out of XLA programs are those that build a disciplined operating cadence and stick to it. The ones that treat XLAs as a reporting exercise almost never see meaningful improvement.

This is the cadence I recommend:

Daily: Both parties maintain live dashboards showing experience trends, application performance, regional issues, and persona-specific insights to catch emerging issues.

Weekly: Customer and vendor teams hold focused working sessions to determine what improved experience this week, what hurt it, which remediation actions were completed, and what’s the priority for next week.

Monthly: Formal governance meetings to review experience scores, improvement actions, root cause discussions, and cross-functional issues that need escalation.

Biannually: Leadership steering meetings to assess overall experience performance, recalibrate targets, and align the XLA program with evolving business priorities to honestly evaluate whether or not the program is driving the outcomes the organization actually cares about.

Common mistakes organizations make

After working through XLA design and implementation with clients across their MSP relationships, the failure modes are predictable. Here’s what to watch for.

Setting targets before establishing a baseline
Rushing into targets before understanding your current state is one of the fastest ways to create disputes. Spend the first three to six months gathering baseline data, then negotiate targets based on evidence rather than guesswork.

Measuring too much
More metrics don’t create more insight. Frameworks with 20 data points rarely survive operational reality. Start focused and expand gradually.

Hiding the data
Transparency is foundational to XLAs. Providers who obscure poor scores, especially when controlling the measurement platform, undermine the entire model. Clients who weaponize the data create the same problem. Build mutual transparency obligations into the contract.

Over-relying on penalties
Penalty-only structures recreate legacy SLA behaviors. Balanced incentives drive better long-term outcomes.

Treating XLAs as static
Employee expectations, technology, and business priorities evolve constantly. Without formal review cycles, XLA programs quickly become irrelevant.

Start smaller than you think you need to

The organizations that get XLAs right are rarely the ones with the most sophisticated tooling. They’re the ones that stopped waiting for a perfect program and introduced real accountability into the contract with what they had.

The most effective starting points are often simple: agree on a focused set of experience metrics, establish a six-month review cycle, commit to shared visibility and data transparency, and create joint accountability for continuous improvement.

From there, maturity develops over time. Governance builds trust, data becomes more actionable, and targets evolve alongside business priorities. The relationship shifts from compliance management to outcome-driven partnership.

In my experience, the organizations that succeed are the ones that stopped accepting green scorecards at face value and demanded something more meaningful.

Meta is reportedly planning a cloud business that would sell access to AI computing power and models, extending its internal infrastructure into a commercial service for outside developers and enterprises. Reuters, citing Bloomberg’s reporting, noted that the planned offering would allow customers to access AI models hosted on Meta’s infrastructure and pay based on usage, effectively positioning the company in the infrastructure-as-a-service and AI platform markets. On the surface, this seems like a logical next step. If you are already spending enormous amounts of money to build AI infrastructure, there is a natural temptation to ask whether some of that investment can be monetized beyond your own internal use.

I have seen this pattern before. A company builds sophisticated internal systems, recognizes their value, and then begins to imagine that becoming a cloud provider is simply a matter of exposing those capabilities to external customers. It sounds straightforward, especially given the excitement around AI and the demand for high-performance infrastructure. But cloud computing is not just another distribution model. It is not simply a matter of offering on-demand multitenant services and charging a fee. It is a deeply operational, trust-based business in a market that punishes companies that do not fully understand what enterprise customers require.

A crowded neocloud market

The first problem Meta faces is that this is not an open opportunity. The neocloud space, meaning purpose-built AI infrastructure delivered as a service, is already crowded and increasingly difficult to enter. Amazon, Microsoft, and Google dominate the conversation for obvious reasons. They have years of cloud operating experience, broad service portfolios, global reach, mature ecosystems, and deeply established enterprise relationships. Oracle remains a serious player as well, especially in enterprise applications, data platforms, and performance-sensitive workloads. IBM still matters in hybrid cloud, operations, and industries where governance and regulatory rigor remain central.

That list alone should give Meta pause. These companies are not just infrastructure vendors. They are experienced cloud operators. They have spent years building not only the underlying platforms, but also the native capabilities enterprises now expect by default. Those capabilities include security, governance, identity management, observability, support, compliance, billing controls, resilience planning, and integration with the broader enterprise technology estate. These are not secondary features. They are part of the core value proposition.

This is why late entry into the cloud market is so hard. A new provider is not just competing on price or capacity. It is competing against accumulated trust. Enterprises are not casual buyers. They are selecting long-term operating environments for applications, data, AI models, and business-critical processes. They want confidence that the provider understands how these services will be consumed, governed, and supported over time. Meta is entering a market where the incumbents already have a major head start on all of those fronts.

Harder than it looks

Over the years, I have had many technology companies come to me and say they wanted to reposition their technology in the cloud space, either as software as a service or infrastructure as a service. In the beginning, enthusiasm is always high. The technology is impressive. The market size looks attractive. The revenue models appear compelling. Investors love the story. Then we begin to walk through what it really means to operate as a cloud provider, and the optimism usually fades fast.

The questions become very practical and very uncomfortable. How will tenants be isolated? How will identity and access controls work across different kinds of customers? What governance models will be built in natively? How will workloads be monitored, optimized, and secured? What does support look like 24 hours a day, across regions, across industries, across compliance boundaries? How will outages be handled, communicated, and remediated? How will the platform integrate with existing customer tools for operations, policy management, and security response? How much investment will it take just to become credible before you even begin to differentiate?

Once companies fully understand the complexities, market dynamics, and the capital and execution required to compete even with secondary players, many of them back off. They realize that cloud technology is not a packaging exercise. It is a transformation in how a company designs, operates, supports, sells, and evolves technology. That is why I remain skeptical when any company assumes it can translate internal infrastructure excellence into external cloud success without a very long, disciplined commitment.

Meta’s market readiness

Of course, Meta is not lacking in financial resources. If any company can afford to spend aggressively in this space, it is Meta. The company has the capital to build infrastructure, absorb losses, hire experienced talent, and stay in the market long enough to make a serious attempt. I would never argue that Meta is too small or too poor to try. Quite the opposite. If there is any non-traditional entrant with the financial scale to force itself into the conversation, Meta would be high on the list.

But money does not erase complexity. It only gives you the chance to confront it. The real question is not whether Meta can afford to become a cloud provider. The question is whether Meta has what it takes to become an excellent cloud provider. Those are two very different things. Enterprises are not going to move meaningful workloads to a new platform simply because the company behind it is wealthy or technically famous. They are going to ask whether the provider understands enterprise consumption patterns, enterprise risk, enterprise governance, and enterprise operations.

That is where the challenge becomes much more serious. Meta has extensive experience running infrastructure for itself. That is valuable, but internal operating excellence is not the same thing as external service maturity. Running systems for your own workloads allows a high degree of control over architecture, standards, priorities, and operating assumptions. Running systems for paying customers requires flexibility, consistency, transparency, and support across a wide range of use cases that you do not control. Those are very different disciplines, and companies often underestimate the gap between them.

What exactly is Meta?

Another concern here is strategic clarity. Meta already has a complicated market identity. It is a social media company, an advertising platform company, a hardware company, an AI company, and still, in the minds of many, the company that spent billions pursuing the metaverse. If it now wants to be viewed as a serious cloud infrastructure provider, it will need to explain not only what it is offering, but why customers should believe this is a durable long-term commitment and not just another adjacent experiment.

That uncertainty can be damaging. Customers want stable providers with clear strategic intent. They do not want to architect important systems around a platform if they suspect the provider may lose interest, shift direction, or reframe the business after a few years of uneven results. Cloud computing requires patience, consistency, and deep customer orientation. It is not a market where strategic ambiguity helps.

This could become confusing for Meta internally as well. Building a true cloud business demands focus. It demands years of investment in areas that may not be glamorous but are absolutely necessary, such as governance, operations, controls, support frameworks, partner programs, and enterprise sales alignment. If the company is not willing to make those sacrifices fully and for the long term, the initiative will struggle.

Threat actors are now abusing an ordinary font file to deliver low-detection malware capable of stealing credentials and establishing persistence on compromised Windows systems.

According to a new research from Fortinet’s FortiGuard Labs, a global phishing campaign is actively using heavily obfuscated JavaScript and a Lua-based loader posing as a TrueType Font (TTF) file to evade security and drop RATs and infostealers.

A TTF file is a standard font file used by operating systems and applications to display text.

The campaign has been deploying malware families such as Agent Tesla, Remcos, XWorm, and a Snake Keylogger variant known as Best Private LOGGER, since at least late March 2026. “In these attacks, the threat actor impersonates several well-known companies, using the guise of business cooperation to launch phishing attacks,” FortiGuard researchers said in a blog post.

Talking about how a new attack technique seems to still rely on conventional phishing tricks, Shane Barney, CISO at Keeper Security, said, “The most sophisticated technical evasion in the world still starts the same way: someone opens an email from what looks like a trusted company and acts on it.”

“The obfuscation layers, the Lua loader disguised as a font file, the fileless execution chain – all of it exists to survive detection after that human decision has already been made, and organizations would do well to keep that in their sightline,” he added.

Business and payment-themed phishing lures used

According to the researchers, victims receive phishing emails impersonating well-known companies and using business collaboration or payment-related themes to trick recipients into opening compressed archives. These archives contain the obfuscated JScript that establishes persistence before dropping either a legitimate Autolt executable or a LuaJIT interpreter, along with a malicious script packaged within a .ttf extension.

The fake font file functions as a Lua-based loader that runs multiple de-obfuscation steps before decrypting and executing shellcode directly in memory.

“Security controls cannot treat a file extension as proof of file type or intent,” said Jason Soroko, senior fellow at Sectigo. “Each component (of the campaign) may appear less suspicious when reviewed alone, while the combined sequence leads to in-memory execution of RATs and infostealers.”

Some of the new variants, the researchers pointed out, are getting more sophisticated by introducing segmented shellcode encryption, Vectored Exception Handler (VEH)- based runtime decryption, AMSI and ETW bypasses, API unhooking, and other anti-analysis techniques designed to evade endpoint defenses.

The final malware payload is delivered using Donut shellcode, allowing execution without writing the payload to disk.

Protection requires targeted mitigations and routine security hygiene

Fortinet’s findings confirm the attackers’ endgame to be stealing credentials and maintaining long-term access. The malware families observed, including Agent Tesla, Remcos, XWorm, and Best Private LOGGER, are all focused on credential theft, surveillance, or remote access.

Barney said organizations should resist focusing exclusively on the loader’s technical sophistication and instead strengthen the systems attackers eventually want to compromise.

In his opinion, identity and access controls are what it comes down to, as signature-based detection often fails against the loader sophistication of this grade. “Limiting what any given set of credentials can reach, enforcing least privilege, requiring re-authentication for sensitive systems, and monitoring for anomalous session behavior will not stop every phishing email from landing, but they significantly constrain what an attacker can accomplish after one succeeds,” he explained.

Soroko, on the other hand, recommends focusing controls on the technical indicators. He urged organizations to restrict Windows Script Host, Autolt, and LauJIT wherever they are not operationally required, monitor for behaviors such as process injection, remote memory allocation, and shellcode execution, and use Fortinet’s published indicators for threat hunting.

The indicators of compromise (IOCs) Fortinet shared include the command-and-control (C2) addresses, file hashes, and filenames.

Soroko warned against relying solely on hashes or C2 infrastructure because the loader has changed over time. “The stronger approach is to detect the stable behavior across versions, then test controls against the complete chain,” he said.

It’s not so much generosity that’s behind Anthropic’s decision to extend free access to its most advanced model, Fable, for paid subscribers until July 19, analysts say. Its a last-minute move to grab users, data and model evaluation results.

After the free-access period, Anthropic plans to convert Fable to a pay-per-use model, at $10 per million input tokens and a whopping $50 for 1 million output tokens.

That is double the price of its next most advanced model, Opus 4.8, for input and output tokens. “We’re extending Claude Fable 5 access on all paid plans, as well as keeping Claude Code’s weekly rate limits 50% higher, through July 19,” Anthropic’s team said in a July 12 tweet.

Anthropic keeps extending Fable because it does not yet know what its flagship is worth, said Sanchit Vir Gogia, principal analyst at Greyhound Research. “A vendor confident in its price does not move the same cutoff twice in six days, both times at the wire,” Gogia said.

Anthropic is essentially pushing deadlines to test its products, while users gain by being able to put their toughest tasks to Fable, Gogia said.

Anthropic, which did not immediately reply to a request for comment about the situation, has already seen plenty of action with Fable and its sister model Mythos. Both have been touted as the company’s most advanced models yet.

Fable stumbles, then reappears

Fable was officially launched June 9. Just three days later, on June 12, the US government put export controls on it after Amazon researchers bypassed Fable’s safeguards, prompting the model to identify software vulnerabilities and demonstrate an exploit. 

After Anthropic scrambled to address the issues — and after the export controls were lifted — Fable was relaunched July 1.

Fable’s freebie extension comes after OpenAI’s latest model, ChatGPT 5.6 Sol, became generally available July 9. Sol is cheaper at $5 per one million tokens input, and $30 for 1 million output tokens.

Anthropic and OpenAI are competing aggressively to build market share, said Jack Gold, principal analyst at J. Gold Associates. “Anthropic and OpenAI are looking to go public and the more users they have, the more attractive it is — even if they are not yet producing income,” he said.

In some ways, the two companies are following a well-trodden path to get customers hooked on their products and turned into paying customers. That’s what Meta, Google and Microsoft, for instance, have done over the years with various “free” offers that later morphed into paid products. 

Plus, said Gold, ”The more users you have, the better you can train your models across multiple data sets.”

That’s a potential boon for proprietary large language model (LLM) vendors offering free tokens in a bid to lock enterprises and vendors into their AI environments. But numerous experts have warned enterprises not to fall for that tactic. Instead, they argue enterprises should diversify AI development across multiple AI and cloud vendors, and adopt open-source models.

An LLM space race?

According to LLM benchmarks maintained by Artificial Analysis, Fable is the most intelligent model currently available, with Sol just behind it in second place. One benchmark by LiveBench places Sol as being better in reasoning, with Fable better at math, data analysis, instruction following and language. Both models have advantages in coding.

Meanwhile, Cursor and SpaceXAI on July 8 unveiled Grok 4.5, which the companies said can “handle difficult, long-running tasks that require creatively using tools to solve problems, whether in software engineering, data science, finance, legal work, or anything else you do on a computer,” the company said in a blog entry.

Its pricing is even more aggressive than Fable and ChatGPT 5.6 Sol. Grok 4.5 charges $2 for 1 million input tokens and $6 for 1 million output tokens.

There are growing concerns about tokenmaxxing, where enterprises rack up billions of dollars in token spending, blowing past usage limits before finance controls are implemented.

Enterprises might decide to spend more on models such as Mythos and Fable — if the benefits are tangible, said Max Leaming, head of data science and AI solutions at ManpowerGroup. Fable and Mythos may “actually be less expensive to use in spite of the spiked token cost because it’s far more efficient,” he said.

A company might find that the models use fewer tokens, are faster, and can reduce compute time, he said. “Even though the per-token costs may go up, we may see overall costs go down,” Leaming said.

Most security leaders I know working on AI robotics are being shown the same kind of video. A humanoid folds a shirt, sorts a bin, walks a warehouse aisle and a vendor uses the clip to move an embodied AI system from pitch to purchase order. Someone then has to sign off. Robot demos create procurement momentum before security teams receive the artifacts needed to evaluate the system as cyber-physical infrastructure.

Before the book, I prepared cloud infrastructure operating in China and the United States for cybersecurity compliance audits and for the Multi-Level Protection Scheme, China’s mandatory security-grading regime that determines whether a system is allowed to operate. That work taught me a lesson I carry into every AI conversation now. You cannot secure what you cannot see into, and the buyer rarely sees in. A demo makes it worse. It shows one task, completed once, under conditions the vendor chose. None of what a security team must evaluate is on screen.

This used to be a research-lab problem. It is now a procurement line item. The risk changed when embodied AI moved from a research demo to a purchase order.  Vendors are asking security teams to approve embodied AI before the category has audit evidence, logging norms, supplier transparency or a shared-responsibility model.

Embodied AI puts a model inside a machine that operates in the physical world: a robot, an arm, a humanoid. Once a model gains motors, sensors and a body, it ceases to be a software endpoint and becomes a cyber-physical system. It inherits hardware, firmware, a supply chain, an installer and a set of remote-access paths. Every one of those is an attack surface that the demo video doesn’t show. An embodied system is sold like software and behaves like a fleet of networked machinery on your floor.

Evaluate these systems across five questions: provenance, access, integrity, evidence and accountability. Here is what each means.

Evaluation question #1: Provenance

What is inside, and who controls it? A humanoid is an assembly of actuators, lidar units, battery packs, joint modules and controllers, most from a supply chain the buyer never vetted, each running firmware the buyer cannot read. Software teams already fought this fight, which is why the software bill of materials became standard practice. Lack of transparency creates systemic risk. Embodied systems raise the stakes because the firmware now lives in dozens of parts that move. The risk does not depend on whether the robot is Chinese, American, German or Japanese. It depends on how much of the system the buyer can see: the hardware, firmware, remote-access paths and maintenance relationships behind it.  China installs more industrial robots than any other country and sits near the center of the battery supply chain, as well as parts of the lidar and machine-vision supply base, which these systems draw on. Lidar, short for Light Detection and Ranging, uses pulsed laser beams to map an environment in 3D; machine vision handles optical inspection and guidance. Much of that lineage traces to suppliers your team has no relationship with. This is the hardware and firmware version of the third-party risk NIST’s supply chain guidance was written for, except that the component has motors. Demand a hardware and firmware bill of materials, then use it. Flag unsigned firmware. Map which supplier holds update authority for each part. Require a way to verify integrity, and treat any component you cannot identify as unmanaged.

Evaluation question #2: Access

Who can reach the fleet? Someone installs these machines, someone services them and the vendor pushes software updates.  Where teleoperation is part of the support model, treat it as a privileged remote-access path, not a convenience feature.  Each is a standing path into a machine that moves and lifts. Security teams have seen this story before. Operational Technology (OT) security went mainstream once industrial systems joined IT networks, and the recurring failure is unmanaged remote access that nobody inventoried. According to one industry survey, roughly half of attacks on OT assets originate in an IT network breach. SolarWinds showed why a trusted update channel deserves scrutiny when one delivered a backdoor to thousands of networks. Embodied systems add the harder part. The compromised endpoint can move. A remote operator on that channel can drive a machine and push code to every unit at once. Treat the fleet like high-value OT. Inventory every remote path, segment it from the production network, default to deny, require signed and verified updates, apply privileged-access controls to vendor maintenance, and treat an always-on teleoperation link as a backdoor until it is governed.

Evaluation question #3: Integrity

Whether the machine can be made to misperceive or misbehave. Researchers have shown that lidar spoofing can cause an autonomous system to brake for an obstacle that is not there or miss one that is. The same class of sensor and model manipulation, on a humanoid sharing a floor with people, produces motion, not a wrong answer on a screen. This is where safety engineering and security part ways. Functional safety stops hazardous motion when a component fails. It plans for accidents. Security plans for an adversary. A hardwired safety circuit can stay independent of the control plane, and a good one does. What it does not tell you is how an attacker reached that control plane, altered the model’s inputs or seized the fleet-management path. Ask the vendor to threat-model sensor spoofing and model manipulation as a path to physical motion. Then ask how you will even know it happened. A spoofed sensor does not announce itself. It shows up as a machine acting incorrectly with confidence.

Picture the failure in plain terms. A warehouse robot takes a routine vendor update that changes how it navigates. The buyer cannot verify the firmware, cannot identify the supplier of the sensor module and has no logs to distinguish a spoofed sensor from a model error. The machine keeps moving, and no one can say why.

Evaluation question #4: Evidence

Whether the claims are true. You have not found an independent audit of embodied-AI field performance, so the uptime and reliability numbers come from the vendor. You are buying a claim, not a track record. Require independently verified uptime, intervention rate and incident history from a named deployment you can call. “Cutting-edge” is not a control.

Evaluation question #5: Accountability

Who owns the risk when it fails? Cloud taught security teams shared responsibility the hard way, after years of arguing which side of the line a breach fell on. Embodied AI arrives without that model, and the stakes are physical: the machine can injure someone. In my compliance work, the question that decided everything was always who is accountable when this thing breaks. Put it in the contract. Define the responsibility boundary, an incident-disclosure timeline, a right to audit and liability for physical harm. A vendor who will not commit in writing is showing you who bears the risk.

These five questions share one root. For a decade, the security question was whether you could trust what a model generates. The embodied question is who can reach the machine and what they can make it do. A demo answers neither.

Before any embodied system reaches your floor, make these five demands of the vendor.

• Provenance. A hardware and firmware bill of materials with named suppliers, integrity verification and a vulnerability-disclosure record. No bill of materials, no deal.
• Access. A full map of who installs, who services and every update and teleoperation path, with segmentation, default-deny and signed updates required.
• Integrity. A threat model for sensor spoofing and model manipulation that treats the failure as physical motion, plus logging that a defender can use.
• Evidence. Independently verified uptime, intervention and incident history from a named deployment you can call.
• Accountability. A contract that defines the responsibility boundary, incident-disclosure timelines, audit rights and liability for physical harm.

The robot demo is built to make you feel the future has arrived. My job, and now yours, is the unglamorous question behind it. Ask what the machine’s attack surface looks like once it is bolted to your floor, wired to your network and updated by someone you have never met.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

A year ago, most enterprise AI systems generated recommendations. Today, AI systems are approving transactions, routing shipments, updating records, interacting with customers, and triggering downstream software actions with little or no human involvement.

For CIOs, that shift changes the central governance question. The challenge is no longer simply whether an AI model is accurate. It is whether the organization can explain, audit, and defend the decisions the system makes.

When an AI assistant suggests a meeting time or summarizes a document, mistakes are inconvenient. When an autonomous AI system issues a refund, reprices a product, modifies a customer record, or initiates a financial transaction, mistakes carry operational, legal, and reputational consequences.

When those consequences arrive, “the model decided” is not an acceptable explanation.

This is the accountability gap emerging at the center of enterprise AI adoption. Organizations are deploying increasingly autonomous systems while relying on technology that often provides little visibility into how decisions are made. The result is a growing mismatch between the level of authority organizations grant AI and their ability to understand or justify its actions.

Black-box AI may have been acceptable when AI primarily generated predictions. It becomes far more problematic when AI begins taking actions on behalf of the business.

The lesson software already learned

Fortunately, the technology industry has faced a similar challenge before.

As enterprise software systems became more distributed and complex, troubleshooting failures became increasingly difficult. Engineers could no longer rely on intuition to understand what happened when something broke. The solution was observability: the practice of instrumenting systems so their internal state could be understood through logs, metrics, traces, and monitoring.

The goal was not to predict every possible failure in advance. It was to create enough visibility that teams could reconstruct what happened after the fact and identify the root cause.

Enterprise AI now requires a similar discipline.

But AI observability must go beyond traditional software observability. It is not enough to know what action occurred. Organizations also need visibility into why the system believed that action was appropriate.

An auditable AI system should be able to answer questions such as:

• What information did the system rely on?
• Which tools or data sources did it access?
• What alternatives did it consider?
• What verification steps were performed?
• How confident was it in its conclusion?
• What events led to the final action?

These questions are rapidly becoming essential operational requirements rather than technical nice-to-haves.

Why visibility matters more as AI gains autonomy

As AI systems become more autonomous, failures become harder to detect and diagnose.

A human reviewing a single AI-generated recommendation can often spot obvious mistakes. A network of AI agents coordinating multiple tasks across business processes presents a different challenge. Decisions can build upon one another. A flawed assumption early in a workflow can propagate through subsequent actions, creating confident but incorrect outcomes.

The challenge is rarely identifying that something went wrong. Eventually, an error surfaces through a customer complaint, a failed transaction, an audit finding, or an operational disruption.

The challenge is determining why it happened.

Which information influenced the decision? Which tools were consulted? Which safeguards worked as intended? Which ones failed?

Without visibility into the reasoning process, troubleshooting autonomous AI workflows can become significantly more difficult than debugging traditional software systems.

For CIOs responsible for enterprise reliability, compliance, and governance, that lack of visibility creates unacceptable operational risk.

Moving toward glass-box AI

The answer is not to slow AI adoption. The answer is to make AI systems observable.

Increasingly, organizations are seeking AI systems that behave more like a glass box than a black box. The objective is not to expose every parameter inside a neural network. Rather, it is to provide a clear, auditable record of how decisions were reached and why actions were taken.

The most promising approaches share two common characteristics.

The first is verification. Instead of treating a single model’s output as ground truth, systems incorporate independent validation steps before actions are executed. Multiple agents, external checks, business rules, or verification workflows help identify errors before they become operational incidents.

The second is explainability. Effective systems maintain a decision trail that captures inputs, intermediate reasoning steps, tool usage, verification activities, and outputs in a form that human reviewers can understand.

Together, these capabilities create something that has long been expected of human decision-makers but is often missing from AI systems: the ability to show your work.

The regulatory and business reality

The push toward AI observability is not being driven solely by technologists.

Regulators increasingly expect organizations to demonstrate oversight of automated decision-making systems. Emerging AI governance frameworks place growing emphasis on transparency, traceability, accountability, and human oversight.

Customers are moving in the same direction. Whether the decision involves pricing, service, eligibility, or support, people increasingly want the ability to understand and challenge outcomes that affect them.

The result is a convergence of operational, regulatory, and market pressures around a single requirement: organizations must be able to explain what their AI systems are doing.

Three questions every CIO should ask

Before deploying autonomous AI systems, technology leaders should be able to answer three basic questions:

1. Can we reconstruct the complete decision path that led to an action?
2. Can we verify critical outputs before actions are executed?
3. Can a human auditor understand why the decision occurred?

If the answer to any of those questions is no, the organization may be granting more authority to AI than it can responsibly govern.

Accountability will become a competitive advantage

The organizations that succeed with autonomous AI will not necessarily be those that automate the most processes or deploy the largest models. They will be the organizations that combine automation with accountability.

Black-box systems made sense when AI primarily generated predictions. As AI increasingly acts on behalf of businesses, customers, and employees, visibility becomes essential.

The future of enterprise AI will belong not to systems that merely act, but to systems whose actions can be examined, understood, and trusted.

—

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.

Enterprises are facing two simultaneous challenges with AI: The risks associated with it are evolving faster than governance frameworks, while the business benefits are often difficult to measure.

This is one of the key findings of The Value of AI, a study commissioned by SAP from Oxford Economics. Now in its second year, the study surveyed 2,600 executives from 13 countries worldwide.

High expectations, limited preparation

On average, the enterprises surveyed plan to spend around $28 million on AI (up from $26.7 million last year), and expect a 21% ROI (from 16% last year). Expectations for AI agents are particularly high, with ROI expected to reach 17% this year, up from 10% last year. Furthermore, 83% of respondents worldwide said agentic AI has the potential to fundamentally transform their organization. On the other hand, only 3% of respondents said their enterprises were fully prepared for the deployment of AI agents.

There are gaps, particularly when it comes to governance:

• Only 12% of respondents said their skills or processes were able to govern AI effectively,
• 38% do not have human-in-the-loop processes in place for oversight of AI agents, and
• only 63% have established permissions and access controls for agents.

Other concerns include weaknesses in the organization of AI deployment, poor data quality, insufficient employee training, and the widespread use of shadow AI.

Governance is the bigger challenge

In an interview, Sean Kask, Chief AI Strategy Officer at SAP, commented on the study’s key findings.

Mr. Kask, in the study’s foreword, you write that companies are currently facing two challenges simultaneously: The risks associated with AI are evolving faster than governance, while the business benefits are often difficult to measure. Which of these poses the greater problem for companies?

Sean Kask: Measuring the business value of IT investments has never been easy. The same applies to AI. That’s why I currently consider the governance issue to be the greater challenge. While traditional governance principles and best practices for secure software development remain important even in the age of large language models and agent-based AI, entirely new risks are emerging at the same time.

For example, as soon as companies roll out AI on a broad scale, they suddenly discover hundreds or even thousands of so-called shadow agents that employees are using without central oversight. Or they find that a significant portion of the workforce is copying content into private ChatGPT accounts. Such risks often only become apparent once AI is already being used productively.

According to your study, German companies invest an average of nearly $40 million in AI, more than companies in all other countries surveyed. Why is that?

Kask: I was less surprised by the amount of investment than by the fact that, overall, the level of investment and the return on investment achieved have developed very similarly across the various countries. There’s no clear answer as to why Germany invests more. In part, it’s likely simply because costs here are higher than in India, for example.

However, we’re also seeing a high level of AI adoption among German companies. SAP has a dashboard that allows us to track how our customers are using AI features. Germany is among the countries with particularly high usage. Added to this are the strong industrial base and the political impetus from Europe, which are driving the use of AI. Accordingly, companies there are making targeted investments in building the necessary expertise.

According to the study, 47% of German companies are satisfied with the return on investment from their AI investments. At the same time, 77% say they are still far from realizing AI’s full potential. Isn’t that a contradiction?

Kask: No, we see this pattern worldwide. Companies initially invest in a few AI use cases and realize: This works; we’re creating added value. Accordingly, they’re satisfied with their investment.

But this is precisely what leads them to identify further use cases. They explore AI agents and want to utilize them as well. However, it is exactly at this point that many encounter new challenges in implementation and scaling.

The study therefore primarily highlights a learning curve: The more experience companies gain with AI, the greater their awareness of its previously untapped potential becomes.

According to the study, only 33% of companies surveyed have KPIs at the executive board level that are directly linked to the implementation of AI. In your view, which metrics should supervisory boards and CEOs definitely be tracking?

Kask: For us, a key indicator is employee enablement. How many employees have already successfully completed training or upskilling programs related to AI? Without the appropriate skills, AI adoption will fall short of its potential.

Transparency is equally important. Companies should know which AI agents are actually in use within their landscape. SAP offers the SAP AI Agent Hub for this purpose, which automatically discovers and inventories agents from SAP and third-party environments. Customers have already been able to identify thousands of agents this way, which highlights the need for centralized governance and transparency.

In addition, companies should have a complete overview of all AI use cases. A robust business case should be in place for each use case. We often see two extremes: Either the executive board is under pressure to implement AI as quickly as possible and allocates a lump-sum budget for this purpose. Or management initially takes a wait-and-see approach. This leads to independent pilot projects springing up throughout the company, with individual departments procuring their own tools and entering into their own contracts.

At SAP, we therefore follow a clearly structured selection process. Each idea first undergoes an assessment of its expected business value. We then examine technical feasibility, data availability, and ethical and governance aspects. From management’s perspective, it is crucial to maintain transparency regarding all ongoing AI projects at all times and to consistently prioritize them based on their business value.

Agents, too, need a ‘hire-to-retire’ lifecycle

Even with the introduction of dozens or even hundreds of AI agents, governance becomes increasingly complex. What capabilities do enterprise platforms need to manage AI agents securely and in a controlled manner at scale?

Kask: We make a conscious effort not to anthropomorphize AI too much. Nevertheless, the analogy is helpful: Agents require a complete hire-to-retire lifecycle. This begins with the detection and registration of an agent. It is then integrated into the enterprise environment, granted the necessary permissions, and given access to the data sources it needs to perform its tasks.

Observability is just as important. Companies must be able to track what an agent is actually doing in the system at all times. In addition, they should track key performance indicators: Is the agent achieving the desired results? How efficiently is it working? How many tokens does it consume? How many processing steps does it require for a task?

Ultimately, this involves several key components: a complete inventory of all agents, appropriate governance, risk, and compliance (GRC) mechanisms, transparency regarding agent behavior, and continuous monitoring. This is the only way to ensure that AI agents consistently operate within defined parameters and deliver the desired business value.

In your estimation, which business processes will companies actually delegate entirely to AI agents over the next two to three years?

Kask: Currently, such agents work particularly well in clearly defined use cases. SAP will release more than 50 (currently 34) specialized AI agents.

One example is periodic financial reporting. In this context, journal entries must be made based on numerous rules stored in documents, emails, or previous transactions. The agent analyzes these various sources of information, derives a recommendation from them, and suggests the appropriate journal entry to the user.

Based on what we’ve heard from customer projects, employees at medium-sized companies currently spend about twelve hours per month on these tasks. With the help of an AI agent, this effort can be reduced to two to three hours.

Another area of application is production planning. If delivery dates change or new orders come in at short notice, the entire production plan must be adjusted. It is precisely these kinds of complex optimization tasks that are ideally suited for AI agents.

In principle, there are virtually no limits to the narrowly defined business processes in which agents can be deployed. However, they will not operate completely autonomously at first.

Trust in AI begins with a stable foundation

Many companies still struggle to trust AI agents. After all, large language models operate probabilistically and can produce false information. This is particularly problematic in financial processes. How do you build trust?

Kask: Trust begins with a stable foundation. ERP systems remain the reliable system of record. They operate deterministically, contain the business logic, and hold the relevant company data. AI agents build upon this foundation. They do not replace it.

Equally important is the human-in-the-loop principle. Employees must be able to understand what the agent is doing, verify its results, and intervene if necessary. That’s why employee training also plays a crucial role. They must understand how generative AI works and where its limitations lie.

Of course, language models can hallucinate. At the same time, we must not forget that humans are not infallible either. The key lies in the collaboration between humans and AI. This allows us to improve both the efficiency and the quality of many business processes.

Another important component is transparency. Our global AI ethics policy, for example, stipulates that users must always be able to recognize when AI is involved. In Joule, it’s possible to trace which data sources the agent used and which steps it went through in reaching its decision. This traceability is an essential prerequisite for trust.

What distinguishes an SAP agent from a general AI agent that merely accesses an ERP system?

Kask: The key difference is that Joule and the SAP agents are directly embedded in the ERP system. There, for example, we’ve built a knowledge graph that describes the semantic relationships between all tables, business objects, and data fields.

To put this into perspective: The SAP S/4HANA Knowledge Graph is based on approximately 452,000 ABAP tables, 7.3 million data fields, and thousands of analytical views. The semantic relationships between these artifacts are modeled in the Knowledge Graph and made available for AI applications.

For example, if a user wants to view all open purchase orders, the agent does not first have to laboriously search for the relevant information. It immediately knows which tables and objects are relevant and also understands the relationships between a purchase order, a purchase requisition, the responsible approvers, and other business objects. As a result, the agent not only works much more precisely but also requires significantly fewer tokens because it can greatly narrow down the search space.

If, instead, one attempts to simply overlay AI onto an existing system or extract data from a relational ERP system, many of these relationships are lost. In a sense, this destroys the semantic context that is crucial for precise answers.

That is why we view the ERP system as an enormous strategic advantage. It has been the system of record for decades and contains roughly 50 years of codified business and process knowledge. This knowledge forms the foundation for what we call the autonomous enterprise. The agents build upon this knowledge and continue to develop it.

In the future, SAP agents will also communicate bidirectionally with agents from other providers via standards such as Agent-to-Agent (A2A).

According to your study, AI currently creates the greatest added value in decision-making, customer interaction, and gaining new insights, rather than in traditional productivity gains. Will this change the way companies justify AI investments in the future?

Kask: In our study, productivity was simply rated slightly lower than, for example, gaining new insights. In the long term, however, productivity remains the ultimate goal. Europe, in particular, has been suffering from comparatively weak productivity growth for years.

At SAP, we therefore first evaluate every new AI feature based on its specific business value. For all agents and AI features that we include in our AI Feature Catalog, we first conduct a value analysis. We ask: What benefit does the feature offer the user? Does it contribute to higher revenue? Does it increase productivity? Only then is it developed further.

At the moment, the greatest added value often still lies in consolidating information from structured and unstructured data sources and making it accessible via natural language. The next step, however, is to translate these insights directly into more efficient business processes. That is precisely where the greatest productivity gains will be realized in the future.

If you could give CIOs just one or two pieces of advice for the transition from generative AI to AI agents, what would they be?

Kask: In my view, the biggest mistake would be to try to transform the entire company all at once or to attempt to perfectly prepare all the data right from the start.

Instead, you should consider what kind of agent can create significant added value, and then implement it. Of course, this agent needs access to consistent and context-rich enterprise data. That’s exactly what we’re working on at SAP with technologies like the knowledge graph, which maps the semantic relationships within enterprise data.

In addition, with data products and the SAP Business Data Cloud, we provide tools that make data from various sources usable for AI agents. Thanks to zero-copy and data fabric approaches, information from legacy systems, Snowflake, or ERP systems can be consolidated without first having to extensively replicate the data. For a procurement agent, this makes it possible to provide exactly the relevant data for the specific use case.

The key point is this: Companies do not have to wait until they have fully migrated to the cloud or consolidated their entire data landscape. With the technologies available today, data can already be made usable for specific AI agents, managed in a controlled manner, and used to quickly generate initial business value. On the other hand, those who wait for the perfect starting point run the risk of falling behind.

This article is adapted from one first published by Computerwoche.

OpenClaw’s announcement that it has become a nonprofit foundation is generating IT excitement because of the potential for governance and development consistency that the popular platform has thus far lacked. Still, some worry about the risks created by the move. 

“Our ambition is for OpenClaw to be the Switzerland of AI. Neutral ground where every model and every lab can plug into the technology and collaborate on standards in the era of agents,” OpenClaw said in a post. “That work is already underway in Foundation-convened councils on agent identity, agent profiles, evals, and enterprise deployment.”

The statement, co-authored by OpenClaw creator Peter Steinberger, pointed out, “the great open source projects of our time — Linux, Apache, Mozilla — endure because a neutral steward stands behind them. That is the role we are taking on to keep OpenClaw MIT licensed, open, and independent so that everyone building on it can trust it will be here for the long term.”

But it reassured users that the original OpenClaw leadership is still in charge.

“Peter built this thing and Peter keeps making the calls, especially the technical ones. Since joining OpenAI earlier this year, he has continued to steward OpenClaw as an open and independent project, and OpenAI has made a commitment to keep it that way,” the post said. “The foundation is here to serve: good governance, stable funding, and paying the people who keep the claws alive.”

However, some analysts and consultants were skeptical about how much true independence Steinberger would have, given his salaried role with OpenAI. 

Neutrality claim in question

“The Switzerland of AI neutrality claim collapses under its own announcement,” said Noah Kenney, principal consultant at Digital 520. “OpenAI runs a team [at OpenAI] called Claw Labs that Peter leads and OpenAI is a major donor to OpenClaw. The ‘neutral steward’s’ chief technical decision maker is employed by one of the competing labs it is supposed to be neutral with.” To OpenAI, he said, OpenClaw is closer to a tax-exempt nonprofit subsidiary than it is to a neutral ‘Switzerland of AI.’

He pointed out that, in addition, Microsoft is shipping the enterprise version of OpenClaw, and Nvidia is shipping the hardware bundle. “This is being called the Switzerland of AI, but Switzerland does not have its central bank run by France,” he observed.

Kenney said that what the new OpenClaw has actually built is “a shared dependency that several competitors fund, staff, and steer, wrapped in a nonprofit structure. Enterprise IT should understand that structure, because treating OpenClaw as neutral is a mistake,” adding that CIOs need to look at this development devoid of the emotional component. 

“There is a strategic irony here that CIOs should sit with,” Kenney said. “If OpenClaw succeeds at becoming the universal agent substrate, then every model plugs into the same identity layer, the same profiles, and the same deployment plumbing. The thing every vendor is racing to own becomes a commodity that nobody owns.” He pointed out that, in the short term, that is genuinely good news for buyers because it means less lock-in and more portability.

“But,” he said, “when the connective tissue is free and natural, the only labs that benefit are the ones with the best models and the deepest distribution. Commoditize the layer below you and you compete on the layer where you are already strongest. The foundation is not a charity. It is the biggest players agreeing to stop fighting over the plumbing so they can fight over the water, and the enterprise is the one paying the water bill either way.”

Good news, bad news

Jason Andersen, principal analyst at Moor Insights & Strategy, liked the potential consistency that could emerge from the structural change, given the complexity of agent development today. 

“We are seeing a lot of OpenClaw variants hit the market, such as those from Nvidia as well as competing products from cloud and SaaS vendors. A common base helps solidify the common parts,” Andersen noted. “That said, a common challenge is the sustainability of these open source foundations over time. In addition to releasing code, these foundations need funding to evolve and grow. And that funding needs to come from continued momentum to incentivize existing members to increase investment and recruit new members to join.”

Andersen stressed that IT buyers need to keep an eye on the roadmap for any OpenClaw variant they choose to deploy, “as that will directly impact the foundation, and the momentum of the foundation and common base. If the common base loses momentum, it can lead to forks, or just a loss of innovation. When that happens, members tend to back away, which puts customers in limbo.”

But not everyone sees the promised structure as entirely good for IT.

Ishraq Khan, CEO at coding productivity tool vendor Kodezi, said, “most CIOs do not want to bet their future entirely on a single model vendor. They want Claude for some workloads, GPT for others, open models for sensitive environments, and potentially internally fine-tuned systems for specific use cases. The problem is that every vendor currently brings its own identity system, tool interfaces, permissions model, and operational assumptions. That fragmentation does not scale.”

He said, “the risk if standards fail is straightforward: every vendor builds its own closed ecosystem, enterprises become locked into individual stacks, and security becomes dramatically harder. The opportunity if OpenClaw succeeds is equally significant: enterprises get portable agents, common identity standards, interoperable tooling, and a healthier competitive market around models rather than ecosystems.”

Will it remain a nonprofit?

However, said Justin Greis, CEO of consulting firm Acceligence, one of the key details that IT executives will want to keep in mind is that OpenAI also began as a nonprofit, but it was quickly seen as not adhering to nonprofit objectives. 

“OpenAI’s transition from a nonprofit research organization into a more complex structure highlighted the challenge of maintaining mission alignment while scaling technology, capital, partnerships, and commercial operations,” Greis said. “OpenClaw has the opportunity to address some of those governance questions earlier by establishing clear principles around neutrality, transparency, and decision-making before the ecosystem becomes even larger and more valuable.”

He noted, “we have seen this pattern before with technologies like Linux and Kubernetes. The strongest open ecosystems succeeded because they created trusted foundations that enterprises could build upon. The technology was important, but the governance model that underpinned it was equally critical.”

Risks are ‘squarely in IT’s lap’

Consultant Brian Levine, executive director of FormerGov, echoed Greis’ concerns. 

“CIOs shouldn’t assume that this nonprofit will always be a nonprofit, or confuse being a nonprofit with actually being neutral or unbiased,” he said. “The risks are squarely in IT’s lap: autonomous agents ‘with their own identity’ acting on a user’s behalf blow straight through traditional IAM assumptions. Issues, such as agent identity, auditability, secret handling. Identity boundaries have not yet been reliably solved. Until they are, enterprises should treat OpenClaw agents like privileged service accounts, not like a browser plugin.”

Independent cybersecurity and risk advisor Steven Eric Fisher pointed to another IT exposure that might come from this OpenClaw transition: Cost.

“OpenClaw currently has a very high token burn rate in usage, which presents a significant cost consideration for large-scale enterprise adoption,” he said. “The skills marketplace introduces a new supply chain threat that enterprises will need to manage. Threat management, and specifically handling external marketplace elements, can be highly challenging for open-source operations. Ultimately, at scale, enterprise adoption could become a difficult balancing act between managing high operational costs and securing an expanded security surface.”

This article originally appeared on Computerworld.

For decades, software engineering has been plagued by “productivity theater.” Every few years, the industry aligns around a new vanity metric — usually one that latches onto whatever technology happens to be in vogue at the time. For a discipline rooted in creativity and problem-solving, this is a poor way to demonstrate progress. Yet, we find ourselves in this position once again. The pattern is often the same: reach for something we can easily count, and in doing so, lose sight of what we are actually trying to achieve.

Quantity over quality: the wrong measurement, every time

I recall when I was coming up as a software engineer in the 1990s, a small number of companies took up the practice of paying their engineers by each line of code. This may have been productivity theater at its worst, leading to negative incentives, inefficient processes, and just generally bad engineering. Developers were rewarded for writing far more code than the problems they were facing required — classic “quantity over quality” — and the result was bloated, brittle codebases that were all but impossible to maintain. The goal — to create reliable software that solved real user problems — got buried under the incentive to produce.

Then in the 2000s, the rise of Agile brought us story points, an abstract way to estimate task complexity, effort, and risk relative to other work. Rather than answering “How long will this take?,” story points were meant to answer, “How big is this compared to what we’ve done before?” This approach sounds good in theory, but in practice, some development teams learned to game the system by inflating estimates, over-engineering solutions to look productive, and losing sight of whether the work they produced actually created value. Once again, the metric became the goal, and the actual goal — delivering outcomes that mattered to the business — became secondary.

Every one of these metrics failed for the same reason: they measured effort instead of value.

Quantity in the age of AI

Today, “tokenmaxxing,” a trend in which developers and teams optimize for consuming as many AI model tokens as possible, treats raw consumption as an equivalent for output. As I see it, this is the latest flawed productivity metric to make its way into the world of software engineering. Tokenmaxxing is nothing more than another vanity metric, and is just as useless as using “lines of code” or inflated “story points” as a benchmark.

Tokenmaxxing is the result of a few different behaviors, including:

• Prompt flooding: stuffing massive codebases, documentation, and context into every prompt, burning tokens on context the model doesn’t actually need.
• Agent swarms: running multiple AI agents in parallel to maximize code output, regardless of whether the work is coordinated or coherent.
• Background loops: keeping AI sessions or agents running continuously in the background, racking up token spend without clear ownership of what is being produced — or why.

Now, it is no secret that AI is reshaping how software is developed, and these behaviors are the result of that reshaping. Providing AI with codebases, running multiple agents at once, and even relying on coding assistants for help all have their uses. But when we lose control of the changes we are making and why we are making them, we find ourselves facing a new version of the same old problem: measuring engineering productivity with the wrong metrics.

A more useful question to ask isn’t, “How many tokens did we spend?” but rather, “What problem did we actually solve, and for whom?”

Spending resources without goals

Yes, AI is giving software engineers the ability to do more with less, to move quickly, and to experiment in ways that were previously out of reach. But leaning on AI to perform productivity, rather than deliver it, is a trap that will cost us in code quality, team capability, and business credibility.

As a CTO, I am all for experimenting with AI. I want to use it to make our programs better, stronger, and future-proof. What I don’t want is for it to drive us toward excess while leaving us with little to show for it.

The test I keep coming back to is simple: does this AI-generated output help us ship something that matters? Does it reduce friction for a user, close a gap in a workflow, or improve reliability for a customer? If the answer isn’t clear, then we are spending resources — both human and computational — without a defined goal. And that is not engineering. That is activity.

Spec-driven development: where value gets defined

It is time to adopt newer approaches like spec-driven development, a method where engineers write detailed specifications first and AI generates code against them. Rather than relying on prompt flooding and agent swarms and hoping AI produces the best result, we need to shift toward defining requirements, reviewing AI-generated output, and orchestrating systems with intent.

But spec-driven development is more than a methodology. It is the place where engineering intent and business value get defined together. The spec is where you answer, “Why does this matter, and what problem are we solving?” before a single token gets spent.

Software engineers have long taken pride in writing elegant code, and I would hate to see AI cheapen that pride rather than elevate it. In an AI-first world, the craft shouldn’t disappear; it should simply move upstream. The spec is where elegance lives now, and it deserves the same attention to detail we once reserved for the code itself.

At its core, software engineering is about defining, analyzing, and resolving technical challenges. If we are willingly giving all of that up to AI, we will lose the integrity of our discipline and the ability to prove our value. Using the maximum number of tokens to produce code isn’t impressive. Using a well-crafted, intentional prompt to solve a specific problem? That’s the work worth celebrating.

Stop performing productivity and start delivering it

We are at an inflection point. Many organizations are defaulting to activity-based metrics, measuring how much AI is being used rather than whether it is improving delivery, product quality, or business outcomes.

The question worth asking is not, “How much AI did we use this sprint?” It is “What value did we deliver for our users, our team, or our business?” Was it the ability to resolve a critical bug more quickly? Reduced cycle time on a high-value feature? A customer workflow that now takes minutes instead of hours? Those are outcomes. Those are the things worth measuring.

AI can help us deliver meaningful outcomes faster, but only if we use it with the same rigor and intent we expect from every other engineering or business decision. Don’t let it become another form of productivity theater. The most successful engineering organizations in the age of AI won’t be the ones that consumed the most tokens, they’ll be the organizations that never lost sight of why they were building in the first place.

—

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.

The first time I heard someone say their AI now wrote 80% of their tests, I asked the obvious question. Eighty percent of what?

After 20 years building and leading test automation for consumer-scale platforms, my honest answer turned out to be eighty percent of the typing, not eighty percent of the engineering. The remaining twenty was where the work still lived. Budgeting for two percent of leftover effort was the mistake. When the real number was closer to thirty, that gap was the difference between a pipeline that shipped and one that quietly built up a queue of half-trusted features nobody could rely on.

This piece is about that gap. As an independent research project on LLM-augmented testing methodology, I built a six-stage agentic pipeline that takes a design in Figma and produces running tests in WebDriverIO, connected end to end over the Model Context Protocol. It works. It has been useful. And the parts that broke surprised me, because they were not the parts the hype cycle tells you to worry about.

How I wired a six-stage pipeline over one protocol

The pipeline runs six stages in sequence, each owned by a different agent, with every handoff crossing MCP.

Six-stage agentic test pipeline: design capture → requirements writer → ticket opener → code generator → test-case writer → automation generator. Each stage carries an MCP handoff and a provenance stamp.

The end-to-end trace links a pull request back to a Jira ticket, a requirements section and a Figma frame. Each artifact is stamped with the agent that produced it, the model it used and the inputs it was given.

MCP is the boring middle that makes any of this work. The cliché is that MCP is “USB-C for AI”: one open protocol, any tool. Like most analogies, it is about eighty percent right. The part that matters is the eighty: I do not have to write a custom adapter for every system the agent talks to. One MCP server per tool and every agent talks to all of them the same way.

Typed handoffs between agents are my own architecture, layered on top of MCP rather than provided by it. Each agent writes a typed artifact the next agent reads. Each handoff is logged with provenance. When something went wrong six stages in, I could replay the chain. Without that discipline, a multi-agent pipeline is a debugger’s worst day. You know the test plan is wrong. You cannot tell whether the mistake came from the Figma read, the requirements interpretation or the ticket scaffolding. With it, I could point at exactly which stage went sideways and which inputs it was looking at when it did. The pattern lives in a public MIT-licensed reference implementation for any reader who wants to run it.

The sixteen-minute number is the marketing number. I ran the full chain end to end in about sixteen minutes on a synthetic net-new screen, Figma in, automation suite out. That repeated across my runs; it is not a demo trick. But sixteen minutes is the part of the story most fun to tell and least useful to learn from. It is what gets quoted in the all-hands. The hours that come after, when a human reviews each handoff, are where the work actually lives.

What actually broke in production-style runs

The failures that stalled my pipeline were rarely the ones I expected.

I expected hallucinated APIs. I got them: the agent confidently called endpoint names that sounded right but did not exist. I expected sparse-spec-in, sparse-spec-out, where a Figma frame with no annotations produced a requirements doc with vague acceptance criteria, every time. I expected locator drift, the common UI-automation failure mode where a renamed component silently breaks an entire test suite. There is solid outside writing on non-determinism in tests covering this whole family of failure modes, and the agent inherited every one.

What I did not expect, and what kept the pipeline down longer than any of the above, was the plumbing.

The model backend timed out under load. It lost credentials silently and started returning empty strings, which the agent then read as confidence. A duplicate consumer on a shared long-poll API endpoint produced an HTTP 409 conflict that broke delivery without throwing anything visible. One unguarded exception inside one agent aborted a whole shared scheduler run and took the other agents in the registry down with it. The single worst incident cost me three hours to find. An environment variable had silently rotated overnight; every agent in the fleet was returning structurally valid but semantically empty requirements docs; the downstream stages were dutifully generating tests against nothing.

None of those are model bugs. They are infrastructure. The agent literature, which is what I went looking through when I started this work, mostly does not talk about them.

The fix was not better prompts. It was circuit-breaker-style review checkpoints between stages and what I now call the four-guard discipline: four small guards I consider non-negotiable on any unattended agentic pipeline. The bulkhead pattern from microservices is the most consequential. An unhandled exception inside one agent can no longer abort the shared run; the offending agent fails fast with a structured error and the others keep going. Paired with that, a pure-data fallback ensures a model timeout produces a deterministic output explicitly marked as degraded mode, rather than an empty string the next stage will misread as confidence. A single-owner lease sits on every shared external endpoint, the cure for the duplicate-consumer incident that ate one of my Sunday afternoons. The cheapest guard was the last to arrive: a one-line synthetic canary every agent has to produce a known correct response to before any real work begins, so a credentials rotation or silent backend failure trips an alert before downstream stages have generated artifacts against garbage.

None of these guards is novel. They are textbook stability patterns at a new boundary: the seam between the LLM agent and the rest of the system, which most of the existing agent literature still treats as a solved problem.

The 20% you don’t see, and when not to do this

Here is the part the demo videos leave out. Even when the pipeline works, the human time per stage does not go to zero.

Human review time per ticket across five pipeline stages: code review 60-180 min, automation review and flaky-fix loop 30-90 min, ticket architecture and sequencing 30-60 min, test data and environment 15-30 min, requirements review 20-30 min. Net: the human still spends 20-30% of the original effort, almost all of it reviewing rather than creating.

Net of all that, the human still spends twenty to thirty percent of the original effort, almost all of it reviewing rather than creating. The pipeline saves seventy to eighty percent, not ninety-eight. The trap is budgeting for the two percent you do not save.

When does this kind of pipeline make sense? In my experience, when the Figma is richly annotated and acceptance criteria are clear up front; when there is review capacity to absorb the work the pipeline shifts onto humans; when the stack is well represented in the training data; and when the feature is net-new rather than a deep edit of legacy code. When does it not? When the design lives on a whiteboard. When the integration touches old code with hidden contracts. When the path is regulated or safety-critical. When there is no senior reviewer who can hold the line. When the work is exploratory and writing the spec is the actual point of the exercise.

Teams I have seen succeed with agentic pipelines budget for the rework explicitly, staff the review queue and treat the saved hours as capacity for harder problems rather than headcount they can release. Teams I have seen struggle did the opposite: declared victory at the demo and quietly accumulated a backlog of half-trusted features the next quarter had to clean up.

The right unit of measurement is not how much the pipeline generates. It is how much of what it generates a human still has to touch before you would ship it. Call it the 80/20 rework rule: measure the rework, not the generation. The teams that get the rework number right are the ones whose AI investments compound. The teams that stop counting at the headline percentage are the ones that own the cleanup six months later.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

OpenAI’s recent update to its Codex CLI has introduced a new protocol that appears to shift more orchestration decisions from user-defined configuration to the runtime, prompting developers to request greater visibility into the instructions exchanged between AI agents.

In a detailed GitHub merged request, users stated that the Multi-Agent V2 protocol-infused architecture of the CLI no longer exposes the instructions passed between parent and sub-agents, making it difficult to inspect how work is delegated across the system.

“Multi-agent v2 currently routes agent instructions through normal tool arguments and inter-agent context. That means the parent model can emit plaintext task text, Codex can persist it in history/rollouts, and the recipient can receive it as ordinary assistant-message JSON,” the request read.

“This changes the v2 path so agent instructions stay encrypted between model calls: Responses encrypts the message argument returned by the model, Codex forwards only that ciphertext, and Responses decrypts it internally for the recipient model,” it added.

Other users, commenting on the thread, also said that the lack of visibility into agent instructions can be attributed to the recently introduced Multi-Agent V2 protocol, with one user stating that reverting to the previous version of the CLI restored visibility, but only as a temporary workaround.

Separately, Ignat Remizov, CTO at payment service Zolvat, filed a GitHub feature request to offer what can be described as a permanent fix after stating that OpenAI may have introduced the change in efforts to harden security.

“A possible shape is to keep the encrypted message field for model delivery, but add a separate non-encrypted audit field for the readable task text. The audit field should be persisted in rollout/history/trace metadata so users and maintainers can inspect what was delegated without needing to decrypt model-delivery ciphertext,” Zolvat wrote.

Enterprise governance concerns are likely to emerge

While an OpenAI contributor said the protocol remains under development and declined further changes to the request, analysts warned that the issue would create debugging, governance, and operational challenges for development teams and their enterprises if the issue persists or becomes a long-term characteristic of multi-agent systems.

“Hidden agent instructions reduce observability in multi-agent systems. Developers can no longer see whether failures stemmed from incorrect task delegation, poor orchestration, or model reasoning, making debugging, prompt optimization, and root-cause analysis significantly harder. Agent instruction traces are becoming as essential as application logs in modern software,” said Pareekh Jain, principal analyst at Pareekh Consulting.

For CIOs, Jain pointed out, opaque agent interactions create governance challenges.

“Without visibility into how agents delegated and executed tasks, it becomes harder to audit decisions, investigate incidents, demonstrate compliance, and build trust in AI systems. Enterprises will increasingly expect secure but auditable agent communication rather than completely hidden orchestration,” Jain said.

“Any big enterprise, especially in regulated industries such as banks and hospitals, needs to be able to prove what their AI systems did and why, especially if something goes wrong. If a sub-agent does something bad, like touching private data, the company needs to show here’s exactly what it was told to do. If that record doesn’t exist, it is a serious problem for trust and legal accountability, not just an annoyance,” Jain added.

Further, the analyst pointed out that issues around the visibility of agent operations could even slow production deployments of mission-critical AI.

“Enterprises, just like we are seeing with developers on GitHub, are likely to demand stronger observability, audit trails, and governance before trusting autonomous multi-agent systems. It is nearly as important as model performance,” Jain added.

An email sent to OpenAI enquiring about planned changes to the protocol went unanswered.

Venture-backed firms are buying up the support, finance-ops and managed-services providers enterprises rely on and re-platforming them around AI agents — and the renewal that follows arrives priced per outcome, sold as your advantage. The acquisition-built structure and unproven stability create governance and continuity risks your vendor process isn’t sized for. Here’s how to keep the leverage on your side of the table.

The first time an AI-native services pitch crossed my desk, I nearly signed it. The savings were real, the agents demoed cleanly and the pricing was the kind procurement loves — per resolved case, not per seat. What I almost missed was who got to define the word “resolved.” On an earlier outsourced-support arrangement, back in my public-sector days, the vendor’s reported resolution rate looked excellent right up until we pulled the reopen numbers ourselves. Auto-closed tickets had been counted as wins. Users had quietly stopped logging issues at all. The dashboard was green; the service was not.

That gap — between the number on the contract and what your users actually live with — is the whole game now, and it is about to scale across your portfolio. Gartner’s 2026 CIO and Technology Executive Survey found only 17% of organizations have deployed AI agents, but more than 60% expect to within two years — the steepest adoption curve of any emerging technology it tracks. The providers running your services are moving first, and the contracts are changing faster than most of us can govern them.

The agents underneath these pitches are also nowhere near as reliable in production as they look in the room. Gartner expects more than 40% of agentic AI projects to be canceled by the end of 2027 on cost, unclear value and weak risk controls, and reckons only about 130 of the thousands of self-described agentic vendors are the real thing — the rest are “agent washing” old chatbots and RPA. Treat any headline resolution rate the way you treat a vendor’s own uptime stats: Marketing, until you have seen it run on accounts like yours.

What’s behind the pitch

The challenger’s economics aren’t magic. They come from a reshaping of the services market: Venture-backed firms buying up fragmented, labor-heavy providers — support desks, contact centers, AP and finance ops, slices of managed IT — and re-platforming them around agents. Many of the companies pitching you are not one company at all but several acquired shops stitched onto a shared AI layer. That barely comes up in the sales meeting. It matters enormously once you are the customer.

The direction is independently corroborated, not vendor hype. Everest Group reports outcome-based pricing in business-process services moving from pilots to scaled adoption as AI makes outcomes measurable enough to contract on, with the binding constraint now governance and verified baselines. HFS Research tracks the same “services-to-software” shift across consulting, IT, and operations providers. Here is the part worth holding onto: Most enterprises are early, so you have a little time. But the first vendors to reprice you this way will be the small, single-source ones in the long tail of your portfolio — which, if your stack looks anything like mine, is most of it.

Don’t assume the incumbent is the safe choice

And don’t kid yourself that renewing with the familiar name keeps you clear of this. The big integrators are pulling labor out of their own delivery just as fast — Accenture cut tens of thousands of roles and rehired against an AI-skills filter — and rewriting deals around a share of savings instead of time and materials. Outcome pricing is becoming the default everywhere. There is no version of this where you sit it out.

Two risks your vendor process won’t catch

The first is governance, and the roll-up structure makes it worse than the usual AI-vendor worry. The company you are contracting with isn’t one system. It is several acquired firms with different data practices and security postures, with an AI layer dropped on top at speed. Your customer records, invoices and support transcripts flow into agents whose decisions you often can’t trace, across entities that were never built to one standard. The numbers here aren’t comforting: IBM’s 2025 Cost of a Data Breach Report found 63% of breached organizations had no AI governance policy at all, and 97% of those that suffered an AI-related breach lacked basic AI access controls — AI adoption, IBM concluded, is outpacing both security and governance. When an agent botches a dispute or misroutes regulated data, the regulator and the customer come looking for you, not the platform. And here is the organizational trap: The savings line is what your CFO signs; the provenance question is the one your audit committee won’t ask until after the incident. Nobody raises it for you.

The second is whether the provider will still be standing in three years. These platforms are new, built by acquisition, venture-funded and not one has run through a full contract term or a real downturn. With Gartner expecting more than 40% of agentic AI projects to be canceled by 2027, putting core operations on a young, agent-dependent vendor is a single point of failure dressed up as innovation. Write the exit and data-portability terms before you sign — while you still have leverage to.

Six questions for the renewal

When the challenger shows up — or your incumbent reprices to match — these are the questions I would put on the table. They come down to one thing: Making sure you, not the vendor, own the number.

1. Definition, baseline, guardrails. Make “resolved” mean what your users experience, measured against a baseline you have captured yourself, and tie it to metrics you own — first-contact resolution, reopen rate, time-to-resolution. Expect procurement to resist because pinning this down slows the deal. Hold the line; it is the whole ballgame.
2. Real agent, or agent washing. Gartner reckons only a sliver of self-described agentic vendors are the genuine article. Make them prove it: Production resolution on accounts like yours, and the human-escalation rate sitting behind that number. Not a demo.
3. Auditability, as a gate. SOC 2 at minimum, increasingly ISO 42001 or NIST AI RMF alignment, plus model cards, decision logs and an incident-response plan they have actually tested. If they can’t show how data is walled off between their acquired entities, or how an agent’s decision gets traced, they aren’t ready for anything regulated. This belongs in the shortlist criteria, not the post-mortem.
4. Where autonomy stops. Decide which actions an agent can take alone and which need a human, how it hands off with context and who is accountable when it acts on its own. Put names against it before go-live.
5. The exit. An embedded agent platform gets stickier than the staffed incumbent it replaced, faster than you would think. Lock down data portability, knowledge-base ownership and a way out while you are still the one with leverage.
6. Capacity, not just cost. The best outcome here often isn’t a smaller bill. It is the demand that your old service levels were quietly turning away. Nobody answered the tickets. The cases that aged out. Ask what fixing that is worth before you optimize purely for headcount.

The move

Outcome pricing is where this lands, and on balance, that is progress. But in the near term, it hands the advantage to whoever can measure the outcome — and in most shops, that isn’t the buyer. The edge isn’t picking the cleverest challenger or the safest incumbent. It is being able to hold any of them to a result, on your numbers. Look again at why Gartner thinks so many of these projects die: Not the technology — cost, fuzzy value, weak controls. Our side of the table. So, start there. Take one high-volume, measurable workflow, pilot it against a baseline you own, instrument it with your own metrics, and treat the muscle you build doing that as the real deliverable. Get it right and the pricing model stops mattering. Skip it, and you have just agreed to pay for someone else’s definition of done.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

A few weeks ago, a new hire at Aviator, fresh out of college, asked me a question I didn’t have a clean answer to. How do I become a senior engineer, or even a staff engineer? What should I learn, and how?

It’s a fair question and a harder one to answer than it was just a year ago.

The path used to be well-known. As a newly hired junior software engineer, you were given an experienced mentor who would assign you simple tasks to learn the ropes. You’d write some code, ask plenty of questions, open a pull request, get feedback in code review, think about it and fix your code. Rinse and repeat that a few hundred times. The tasks became more complex; the feedback got shorter and along the way you’ve been building judgment, the thing that separates a senior engineer from a junior one.

Now AI writes most of the code. The loop looks different and the easy assumption is that it’s broken: fewer tasks for juniors to cut their teeth on, an agent fixing the code that another agent wrote, thinner path to judgment.

Mentoring got easier, not harder

I knew just the person to ask: how do we grow senior and staff engineers in the AI era? Adam Berry is a staff engineer at Netflix, a member of The Hangar, our community of engineering leaders, and someone I have been discussing the evolving role of code review and knowledge sharing for a while now. He spends his time on getting AI adoption right, rather than just fast, in their engineering organization and has been working out the question of growing new engineers in practice. Mentoring juniors in an agentic world, Adam says, isn’t harder; it’s embarrassingly easy.

“You grow juniors and help them become better engineers the same way you always did. AI isn’t changing the methodology. It’s changing the details,” he told me. His point is that the agentic world gives a lot more options for giving juniors bounded tasks to work on and more feedback. The scattered remarks and comments seniors used to give in person now can be put into instructions and guardrails.

Adam breaks working with agents into three foundational skills:

• If you don’t know how to do something with the agent, ask the agent.
• If the agent does something you don’t like, figure out how to correct it and then codify it so it doesn’t happen again.
• Your sense of when the agent has gone off the rails.
  
  

“Most juniors can pick up the first two on their own. On the third one, they need guidance, he says.

His process for building it is staged. “The stages are about growing scope. First, you give a junior engineer a well-specified task—and these are now bigger than what you’d have given a junior before. You can give them task definitions that are like a prompt and instructions to drive that prompt, make sure they got to a good plan, make sure they understood the plan, and that they thought through the test cases, etc.

Then gradually you peel off some of that specificity so they have to build the muscle themselves. Once they’ve gotten good at that level of scope, they’re ready to work on larger scoped problems.”

Starting from more specific problems and going towards ambiguous problems is the definition of growing as an engineer.

Pair programming with the agent in the room

Seniors can still do pairing sessions with juniors, now with the agent in the room.

“In the pairing session, the earlier-career engineer should be the one driving. The agent can be set up to interrogate the junior rather than just answer them. None of you is manually writing code, but you’re still doing pair programming and mentoring. Even if it’s just a trivial bug fix, if you guide a junior through it, it forces them to do just that little bit of thinking.”

Adam says mentoring juniors today does not have to mean forcing them to write code manually. Seniors should teach them the process of agentic engineering, and that’s exactly what they should focus on during the pairing sessions. The habit he wants to be installed early is asking for options instead of answers.

“I aim to teach juniors to ask for options and think through them, even on small tasks. I ask them to explain what their input to the AI tool was that led to the code they got. But I’d also show them how I would have done the same thing.”

The pairing produces artifacts as it goes. “That’s where you get into conversations of, ‘This is why that wasn’t quite it for me,’ and if I see that that’s not baked into the repo, I’m going to add this into the ADR, into the design, into the instruction set. I’ll codify that so the junior gets it too, and they know why it exists, because they watched me go through it with the tool myself.”

That reshapes the code review instead of removing it. Making that work puts more on senior engineers, not less. “Senior engineers need to ensure that things like ADRs, or whatever system you use, are properly encapsulated in the repo for both the agents and the humans to consume.” His team also attaches the prompts to the pull request and has the agent summarize what it did against what the prompt asked.

Adam also teaches junior engineers how to bring in expert sources from outside into AI tools. He’ll point an agent at a book like Michael Feathers’ Working with Legacy Code as an example of what quality code looks like and have it work from the concepts directly.

Don’t outsource the thinking

His arguments make sense, but I also recently came across research examining the influence of AI tools on how and why members of software engineering teams interact. Their finding was not surprising: of the 131 surveyed developers, 51% said they now ask GenAI for technical help they once would have asked a person, and 62% said it was easier to ask GenAI without fear of embarrassment.

When a junior gets stuck now, their first move usually isn’t to message a senior on Slack. It’s to ask the agent. This is also the case in code reviews. The purpose of code reviews was always knowledge sharing as much as it was a quality gate. What I see junior engineers do now is take the feedback and, without reading it closely, hand it straight to the agent to resolve. The part where they would have to understand how their work differed from what the senior expected disappears. It got passed from the reviewer to the agent, and the junior skipped the understanding.

This isn’t really the junior’s fault. They need the motivation and the space to do it differently, and the default pattern under delivery pressure is to push the thing out and worry about it later.

The same research showed that developers turned to colleagues with questions about context (65% to clarify business logic or requirements, 50% for how something had been done before).

Respondents were also aware of the trap that Berry’s approach was created to avoid: AI tends to hand back a single answer, compared to multiple perspectives a colleague surfaces and they kept seeking teammates for context-specific expertise, mentorship and plain social connection.

The fix is almost mechanical: if you have to make a choice, you have to think about it. I do this in my own product thinking. Most of it happens by bouncing ideas off Claude, but whenever something is complex, I make myself lay out a few options, trade them against each other and decide which direction to take. That’s the same move Berry wants juniors making, and it’s what builds judgment, whether you’re twenty-two or forty.

Why we should still hire juniors

There’s also a hiring question underneath all of this. We recently hosted Kent Beck, an industry legend, at the Hangar in a session we called “Juniors FTW,” and his reasoning was that the industry is being remade fast enough that being new is an advantage. Juniors are too new to have absorbed what everyone “knows” is impossible, which leaves them less biased, more creative and carrying fewer preconceived mental barriers.

Beck also wrote about how important it is to hire juniors without the calculation of how many tasks they can perform.

“If all we cared about was today’s productivity, we wouldn’t have hired you at all. Instead, we (the seniors) are focused on the future: we know there’s going to be far more work here than we could possibly accomplish. We are paying your salary now as the option premium on the engineer you will become. If we play this game right, we’ll have a kick-ass next generation of engineers. If not, we’ll have to be doing the same engineering jobs ten years from now, and we really don’t want to be doing that.”

The pipeline is thinning

The trend is running the other way. Entry-level hiring at the 15 biggest tech firms fell 25 percent from 2023 to 2024, according to a report from SignalFire. In a recent survey of engineering leaders, a majority said they plan to hire fewer juniors, on the logic that AI lets seniors cover more ground.

That logic is short-sighted in a specific way. Senior engineers don’t appear from nowhere. They’re the juniors someone hired five or ten years ago and invested in mentoring them. Stop hiring and growing juniors now, and the gap doesn’t show up this year. It shows up later, when the industry needs people with the judgment that only comes from years of making mistakes and recovering from them and finds it stopped producing them.

So, here’s the answer to the question that the new hire asked: the path to senior and to staff is the same path it always was. You grow the range of ambiguity you can handle, and you stay honest about the part you can’t handle yet. What changed is the interface. The agent writes the code. Your job is to keep asking questions to your colleagues and the agents and keep doing the thinking until the thinking is good.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

OpenClaw’s announcement that it has become a nonprofit foundation is generating IT excitement because of the potential for governance and development consistency that the popular platform has thus far lacked. Still, some worry about the risks created by the move. 

“Our ambition is for OpenClaw to be the Switzerland of AI. Neutral ground where every model and every lab can plug into the technology and collaborate on standards in the era of agents,” OpenClaw said in a post. “That work is already underway in Foundation-convened councils on agent identity, agent profiles, evals, and enterprise deployment.”

The statement, co-authored by OpenClaw creator Peter Steinberger, pointed out, “the great open source projects of our time — Linux, Apache, Mozilla — endure because a neutral steward stands behind them. That is the role we are taking on to keep OpenClaw MIT licensed, open, and independent so that everyone building on it can trust it will be here for the long term.”

But it reassured users that the original OpenClaw leadership is still in charge.

“Peter built this thing and Peter keeps making the calls, especially the technical ones. Since joining OpenAI earlier this year, he has continued to steward OpenClaw as an open and independent project, and OpenAI has made a commitment to keep it that way,” the post said. “The foundation is here to serve: good governance, stable funding, and paying the people who keep the claws alive.”

However, some analysts and consultants were skeptical about how much true independence Steinberger would have, given his salaried role with OpenAI. 

Neutrality claim in question

“The Switzerland of AI neutrality claim collapses under its own announcement,” said Noah Kenney, principal consultant at Digital 520. “OpenAI runs a team [at OpenAI] called Claw Labs that Peter leads and OpenAI is a major donor to OpenClaw. The ‘neutral steward’s’ chief technical decision maker is employed by one of the competing labs it is supposed to be neutral with.” To OpenAI, he said, OpenClaw is closer to a tax-exempt nonprofit subsidiary than it is to a neutral ‘Switzerland of AI.’

He pointed out that, in addition, Microsoft is shipping the enterprise version of OpenClaw, and Nvidia is shipping the hardware bundle. “This is being called the Switzerland of AI, but Switzerland does not have its central bank run by France,” he observed.

Kenney said that what the new OpenClaw has actually built is “a shared dependency that several competitors fund, staff, and steer, wrapped in a nonprofit structure. Enterprise IT should understand that structure, because treating OpenClaw as neutral is a mistake,” adding that CIOs need to look at this development devoid of the emotional component. 

“There is a strategic irony here that CIOs should sit with,” Kenney said. “If OpenClaw succeeds at becoming the universal agent substrate, then every model plugs into the same identity layer, the same profiles, and the same deployment plumbing. The thing every vendor is racing to own becomes a commodity that nobody owns.” He pointed out that, in the short term, that is genuinely good news for buyers because it means less lock-in and more portability.

“But,” he said, “when the connective tissue is free and natural, the only labs that benefit are the ones with the best models and the deepest distribution. Commoditize the layer below you and you compete on the layer where you are already strongest. The foundation is not a charity. It is the biggest players agreeing to stop fighting over the plumbing so they can fight over the water, and the enterprise is the one paying the water bill either way.”

Good news, bad news

Jason Andersen, principal analyst at Moor Insights & Strategy, liked the potential consistency that could emerge from the structural change, given the complexity of agent development today. 

“We are seeing a lot of OpenClaw variants hit the market, such as those from Nvidia as well as competing products from cloud and SaaS vendors. A common base helps solidify the common parts,” Andersen noted. “That said, a common challenge is the sustainability of these open source foundations over time. In addition to releasing code, these foundations need funding to evolve and grow. And that funding needs to come from continued momentum to incentivize existing members to increase investment and recruit new members to join.”

Andersen stressed that IT buyers need to keep an eye on the roadmap for any OpenClaw variant they choose to deploy, “as that will directly impact the foundation, and the momentum of the foundation and common base. If the common base loses momentum, it can lead to forks, or just a loss of innovation. When that happens, members tend to back away, which puts customers in limbo.”

But not everyone sees the promised structure as entirely good for IT.

Ishraq Khan, CEO at coding productivity tool vendor Kodezi, said, “most CIOs do not want to bet their future entirely on a single model vendor. They want Claude for some workloads, GPT for others, open models for sensitive environments, and potentially internally fine-tuned systems for specific use cases. The problem is that every vendor currently brings its own identity system, tool interfaces, permissions model, and operational assumptions. That fragmentation does not scale.”

He said, “the risk if standards fail is straightforward: every vendor builds its own closed ecosystem, enterprises become locked into individual stacks, and security becomes dramatically harder. The opportunity if OpenClaw succeeds is equally significant: enterprises get portable agents, common identity standards, interoperable tooling, and a healthier competitive market around models rather than ecosystems.”

Will it remain a nonprofit?

However, said Justin Greis, CEO of consulting firm Acceligence, one of the key details that IT executives will want to keep in mind is that OpenAI also began as a nonprofit, but it was quickly seen as not adhering to nonprofit objectives. 

“OpenAI’s transition from a nonprofit research organization into a more complex structure highlighted the challenge of maintaining mission alignment while scaling technology, capital, partnerships, and commercial operations,” Greis said. “OpenClaw has the opportunity to address some of those governance questions earlier by establishing clear principles around neutrality, transparency, and decision-making before the ecosystem becomes even larger and more valuable.”

He noted, “we have seen this pattern before with technologies like Linux and Kubernetes. The strongest open ecosystems succeeded because they created trusted foundations that enterprises could build upon. The technology was important, but the governance model that underpinned it was equally critical.”

Risks are ‘squarely in IT’s lap’

Consultant Brian Levine, executive director of FormerGov, echoed Greis’ concerns. 

“CIOs shouldn’t assume that this nonprofit will always be a nonprofit, or confuse being a nonprofit with actually being neutral or unbiased,” he said. “The risks are squarely in IT’s lap: autonomous agents ‘with their own identity’ acting on a user’s behalf blow straight through traditional IAM assumptions. Issues, such as agent identity, auditability, secret handling. Identity boundaries have not yet been reliably solved. Until they are, enterprises should treat OpenClaw agents like privileged service accounts, not like a browser plugin.”

Independent cybersecurity and risk advisor Steven Eric Fisher pointed to another IT exposure that might come from this OpenClaw transition: Cost.

“OpenClaw currently has a very high token burn rate in usage, which presents a significant cost consideration for large-scale enterprise adoption,” he said. “The skills marketplace introduces a new supply chain threat that enterprises will need to manage. Threat management, and specifically handling external marketplace elements, can be highly challenging for open-source operations. Ultimately, at scale, enterprise adoption could become a difficult balancing act between managing high operational costs and securing an expanded security surface.”

Almost every company has a board or executive AI mandate. Vendors are rolling out agentic AI platforms. The pressure to move is intense.

But the reality on the ground looks different. Eighty-three percent of organizations say data quality is their top AI challenge, and 74% struggle to demonstrate ROI, according to Lopez Research. And only 21% report having a mature governance model for AI agents, per Deloitte’s 2026 State of Enterprise AI report.

“Agentic AI is real, and vendors’ offerings are very real, too,” says Boris Evelson, vice president and principal analyst at Forrester. “However, most enterprises are still not ready to adopt at scale.”

Dave Hilborn, who leads West Monroe’s Organization, People & Change practice, frames it as a race with three arrows moving forward — one representing AI and tech evolution, one representing organizations and people, and one representing data. “The AI arrow is far out ahead,” he says. “That delta is the readiness gap.”

The gap isn’t the technology. It’s the foundational work most organizations haven’t done: data readiness, operating models, governance, skills, and culture. The companies making progress aren’t waiting for vendors to solve these problems. They’re tackling the unglamorous work themselves.

AI doesn’t tolerate ambiguity

AI readiness can be framed across six levels — from data foundation at the base to reinvented business experiences at the top, says Afshean Talasaz, former CIO at Colonial Pipeline and now an executive advisor. One of the key areas that doesn’t always get the attention it needs is the operating model.

“The technology playbooks of the past don’t work in the AI world,” Talasaz says. “Those areas were able to tolerate more ambiguity between business and tech teams. AI doesn’t tolerate the same level of ambiguity. It needs clarity.”

That demands a different kind of partnership between IT and the business. AI systems learn from data — records and measurements of what’s actually happening in the business — and then operate within business processes. Unlike traditional software, which is built based on user requirements, AI is sandwiched between the business that produces the data and the business that consumes the outputs.

“AI is requiring IT and business teams to work more closely together, to be clearer about what AI will and will not do — that really close partnership is crucial,” Talasaz says. “It’s not something that will always naturally evolve. It requires a lot of intentionality about how teams need to work together to deliver outcomes.”

The AI questions CIOs must answer aren’t just technical. Do we have the right operating model? Have we balanced governance and standard operating procedures within the model? Have we organized teams appropriately? All this must be designed within the context of what the business actually needs.

Too many organizations are bolting AI onto existing processes without redefining roles or workflows, Forrester’s Evelson. “Organizations can either incrementally enhance existing workflows by augmenting capabilities with AI or pursue a more transformative approach by redesigning the process end-to-end.”

The companies getting value are doing the latter.

Data debt comes due

Data readiness remains the most common barrier to scaling AI. “We’ve never fixed this data quality problem in most organizations,” says Maribel Lopez, founder and principal analyst at Lopez Research, “and it comes back to haunt a company in spades as they move to AI.”

At Levi Strauss, the foundational work came first. “If you think about the Levi’s business, it’s quite complex — 100 countries, over 3,000 stores, multiple business models,” says Jason Gowans, the company’s chief digital and technology officer. “You can imagine the complexity of gathering all that data to understand how the business is performing. The idea of this single source of truth — that’s been the biggest thing.”

Levi’s now has more than 1,100 standard operating procedures that govern how work gets done on top of SAP. “That’s fertile material to feed to LLMs on how work gets done,” Gowans says.The results are tangible: partner onboarding that once took three to six months to set up EDI exchanges now takes days.

At contract manufacturing company Jabil, Chase Christensen, segment CIO, took a similar path. “We had to get everyone to understand where the source data resides, put tech in place so consumption is easier, and drive ownership around data and decision rights — so 140,000 employees don’t feel empowered to create their own data sources that fall out of line.”

The data challenge goes beyond quality, Evelson notes. Most organizations’ data isn’t AI-ready; it hasn’t been prepared for how AI systems consume and learn from information. “Data is siloed, poorly governed, and hard to discover, integrate, and trust,” he says.

Forrester research shows that 45% of data and analytics decision-makers were adopting vector databases in 2025, and 53% were adopting graph databases — investments that signal recognition of how much data architecture needs to evolve. The firm recommends a balanced approach: roughly 48% of AI spending on foundations such as data management and engineering, and 52% on consumption, including analytics, governance, and applications.

But even as organizations work to prepare existing data, AI is creating new challenges. Users leveraging AI tools are generating new forms of data and information that never make it into corporate databases, West Monroe’s Hilborn notes.

“There are explosions of new data, content, and insights being created on the periphery of these data lakes,” he says. “The challenge is how do you capture that and leverage it.”

Who’s sponsoring this?

Even when data is in order, many AI initiatives stall due to how they’re sponsored and funded.

“Enterprise data, analytics, and AI programs succeed when business CxOs sponsor them because they are accountable for business outcomes, not just technology delivery,” Forrester’s Evelson says. “IT-led initiatives often become siloed or tool-centric, whereas business sponsorship ensures alignment to enterprise strategy, prioritization of end-to-end use cases, and a focus on decisions and actions rather than insights alone.”

Too often, AI is still treated as a series of disconnected use cases rather than a sustained, multi-year investment. Evelson calls this the “use case trap” — organizations overindex on individual projects and miss the enterprise-wide compounding impact. That leads to fragmented priorities, inconsistent adoption, and difficulty demonstrating ROI.

Leadership readiness is a distinct layer of AI preparedness, Talasaz says. “Are leaders prepared to provide a vision of reinvented business experiences that become the north star?” he asks. “Leadership teams, at various levels of the organization, need to articulate what a reinvented business looks like so teams have the direction and support to build differentiating capabilities.”

Levi’s offers a counterexample. AI is a CEO priority there. At the last quarterly offsite, the execs were building agents. “When you’re committed to upskilling the workforce, you’re better served to answer how to rewire processes with AI at the core,” Gowans says. “It starts at the top. It has to be an exec priority.”

Fear, literacy, and two types of AI

Technical talent is only part of the equation. Organizations also need to address change management.

“We saw it with the AI boom — fear about jobs, not knowing what AI did,” says Jabil’s Christensen. “The key is demystifying AI. We doubled down and focused on AI literacy. We want everyone to understand how it was put together, and that removed a lot of that fear. That’s been the biggest hurdle.”

Different types of AI require different skills and governance, Talasaz says. “General use focuses on productivity on the desktop,” he says. “Integrated AI — industrial-capable AI embedded within core business processes — requires different skills, capabilities, and governance.”

For desktop AI, training and guardrails help employees be successful — what Talasaz calls “bumpers,” like in bowling. Organizations need to help employees through reskilling and guidance. “You have tools in a toolbox,” he says. “It’s important to know when to use a power tool versus when you need a screwdriver.”

But for integrated AI embedded in core processes, the stakes are higher. “Business leaders responsible for business outcomes based on AI-driven processes need to be fully aware of both the benefits and risks that come along with using these tools,” Talasaz says.

That distinction matters for governance, too. Lower-, medium-, and high-risk AI use cases may require different ways of working and different risk management approaches. “Deploying AI in potentially high-risk or high-cost areas of the business requires a higher level of rigor,” Talasaz says. “That’s different than building something that helps write my emails.”

From POC to production

Perhaps the biggest readiness gap is the transition from proof of concept to production. “It requires such a different approach,” Talasaz says. “A successful proof of concept can create a lot of excitement, but when teams are unprepared to build and scale, it can create the potential to over-promise and under-deliver.”

The operating model that works for experimentation doesn’t work for production at scale. Proofs of concept are designed to demonstrate the efficacy of ideas and the underlying technology. But building, scaling, and sustaining technology in the business requires operating models, standards, roles, and skills that many organizations haven’t developed. Intentionally designed operating models reduce the cost of learning, improve execution, and increase delivery velocity, says Talasaz.

But there’s no one-size-fits-all answer. “A business that needs to build capabilities in a marketplace moving very fast requires one kind of operating model,” Talasaz says. “A business that can take longer to develop business capabilities and adapt to market changes can choose a different operating model. It’s important to design ways of working tailored to what the business needs and the speed at which the business needs to leverage technology to be successful.”

Jabil is navigating this journey as part of its move to SAP’s cloud ERP through RISE, scaling from $29 billion to $34 billion in revenue while keeping selling, general, and administrative (SG&A) expenses relatively flat — in part by layering generative AI onto predictive analytics capabilities built over years.

“We started years ago with computer vision to drive product quality,” Christensen says. “As gen AI blew up, we took the predictive analytics we had built over the years and imbued them with gen AI. We’ve implemented the basics, and now we’re looking for complex scenarios.”

Governance built in, not bolted on

Governance is often treated as a policy document or committee. It should be embedded in the operating model itself, Talasaz argues.

“The operating model doesn’t always get the attention it needs,” he says. “Policies and committees are useful, but they should handle larger enterprise risks. Most of the governance should be embedded in the operating model to ensure you’re getting outcomes you want.”

That might mean peer review built into the development process, bias checks before deployment, or clear escalation paths for high-risk use cases. When governance is separate from the operating model, it tends to slow things down. When it’s integrated, it becomes how work naturally gets done, says Talasaz.

Governance at the agent level matters, too, Levi’s Gowans says. “Know what agents have been deployed, who authored them, and who’s responsible,” he says, noting that the company has established a registry to understand what agents it has operating within its networks.

The challenges of AI governance are unique, Lopez of Lopez Research says. “Very few people have the governance stack required to say they did the right things with AI,” she says. “Non-human identity and access control is totally different and, frankly, evolving so quickly that no one knows what to do.”

The challenge is ultimately a trade-off, Forrester’s Evelson says. “Push agentic AI capabilities too far, and you risk creating a governance and compliance nightmare,” he says. “Tighten controls too aggressively, and you stifle innovation. Best practices for striking the right balance are still being discovered.”

It takes a team

The AI readiness gap isn’t about technology — it’s about the work organizations have been deferring for years. Data quality. Operating models. Executive sponsorship. Skills and culture. Governance embedded in process.

“Once you progress from everyone using Copilot to putting agents in production, then you realize the need for business context,” Gowans of Levi Strauss says.

It’s a shared journey requiring all teams to understand what’s required, Talasaz says. “It involves helping people understand what it takes from all sides — the technology itself, the operating model, the skills and talents needed — but also working with business leaders on the art of the possible,” he says. “Helping them understand both the benefits and the responsibility of deploying this tech.”

A colleague of his calls AI “the ultimate executive team sport.”

“It requires people to do it well and manage it,” Talasaz says.

Customer expectations don’t end when business hours do, which is why delivering a fast, always-on customer experience (CX) has traditionally required large call centres and significant resources. This often placed small businesses at a disadvantage, as many lacked the manpower and budget to provide 24/7 support at scale. Today, AI has completely levelled the playing field. Even small businesses now have access to powerful tools that can answer queries, resolve routine issues, and deliver highly personalised interactions around the clock.

But adopting AI in customer engagement is not just a question of efficiency. For smaller businesses especially, where loyalty is often built on familiarity, trust, and personal service, the real challenge is using AI in ways that strengthen rather than dilute the human connection that customers value most.

Human empathy combined with AI efficiency is a delicate blend. Done right, it ensures that every customer interaction feels personal, thoughtful, and seamless, whether the customer is engaging with a bot at 2 a.m. or a live agent during office hours.

So, how can small businesses embrace always-on virtual agents without losing the human connection that defines their identity? Here’s a practical playbook to guide the transition.

1. Understand what customers want: Speed, simplicity, and empathy

Before diving into AI adoption, it’s critical to understand what customers expect. Twilio’s Digital Patience study suggests that while speed matters, it is not the only thing that customers value. Twilio found that 46% of respondents in the Asia-Pacific and Japan region say quick service and resolution are most important, but 51% say delays are acceptable if they lead to better customer support. The study also notes that customers are open to AI, but still value human touchpoints more highly.

The takeaway: AI should enhance CX, not replace it. Businesses can let natural-sounding AI voice agents handle inbound calls, regardless of peak hours or time zones. These virtual agents act as an intelligent frontline – answering common questions and qualifying leads – before seamlessly routing the conversation to a live human representative. The result? Callers get immediate answers, and the business captures every opportunity without losing the human touch.

2. Map the handover points between AI and humans

One of the most common pitfalls in implementing AI is failing to clearly define when and how customers transition from bots to human agents. To avoid customer frustration, organisations must thoughtfully map out these “handover points” by designing for two key principles: choice and continuity.

Designing for Choice

Give customers the option to reach a human when needed. While AI is perfectly suited for routine inquiries like FAQs or order tracking, customers should never feel trapped in a bot loop. Always provide a clear, accessible option for them to choose to escalate the issue. Additionally, configure your system to proactively step in and offer a human handoff the moment it detects emotion, ambiguity, or complex steps.

Designing for Continuity

Effective handovers rely on technology that recognises when an issue exceeds AI’s scope. By leveraging natural language processing and intelligent routing, organisations can ensure the transition from machine to human is frictionless. Crucially, this means automatically carrying the full history and context of the interaction forward so the customer never needs to repeat themselves.

Achieving this level of continuity requires a new approach to managing interaction data during handovers. Instead of passing along a raw transcript, organisations need a managed memory service that provides agents with persistent context across every conversation, channel, and session. By transforming customer preferences, unresolved issues, and intent into a structured semantic profile—one that continuously evolves and reconciles new interactions as they occur—agents can quickly understand the relationship and continue the interaction without disruption.

To support truly omnichannel experiences, the system must also resolve identity automatically across touchpoints, linking interactions from phone, email, messaging apps, and other channels to a single customer profile. Equally important is the ability to surface only the information that is relevant to the task at hand. By presenting agents with a concise summary of the active issue and customer preferences, grounded in verified business knowledge such as product policies and FAQs, organisations can reduce resolution times while ensuring customers experience a seamless continuation of the conversation.

3. Don’t automate for automation’s sake

AI adoption should never feel like a “set it and forget it” strategy. Instead, it should be approached as a way to solve real business problems. It starts with asking questions like: What are the most time-consuming tasks for the team? What frustrates customers the most?

For instance, a restaurant might automate table reservations and menu queries, while a small online retailer could deploy AI to handle order status updates or product recommendations. These targeted use cases ensure that AI adds tangible value without overwhelming operations.

Take the example of Driva, a fast-growing online finance broker that deployed AI-powered customer service tools to answer routine enquiries and provide immediate assistance while customers wait in the call queue. By automating common interactions, Driva reduced the volume of requests requiring human intervention and achieved a 5% uplift in conversion rates at key points in the customer journey.

4. Invest in AI that connects

While consumers embrace automation, research shows they still draw comfort from the warmth of a human voice. To make your virtual agents feel less robotic and more like an extension of your team, look for tools that:

• Deliver human-like voice AI experiences at scale through natural turn-taking and barge-in capabilities.
• Connect interactions across voice, messaging, and digital channels into a single thread so every exchange builds on the last.
• Leverage Natural Language Processing (NLP) that enables conversational systems to interpret context, mimic human tone, and even recognise sentiment.
• Place orchestration at the heart of the experience. An effective orchestration engine acts as the “conductor,” actively coordinating workflows and routing interactions so the right resource—whether an AI bot or a human—handles the right moment.

When AI bots, automated workflows, and human teams are seamlessly coordinated behind the scenes, the customer simply experiences one unbroken, dynamic dialogue. For small enterprises, this means delivering sophisticated experiences that effortlessly bridge the gap between automation and live support, even at scale.

5. Empower teams with real-time context

AI is not about replacing human workers; it’s here to make jobs easier. However, for teams to fully embrace this new dynamic, organisations must shift their focus from retrospective performance reviews to real-time agent assistance. By feeding agents context as the conversation happens, businesses ensure that every interaction never starts from scratch.

• Leveraging Conversational Intelligence: Use a real-time intelligence layer that turns live conversations into signals and actions. By analysing voice and messaging with generative AI Language Operators, businesses can understand intent, sentiment, and churn risk instantly, allowing human and AI agents to act in the moment with the right response or escalation.
• In-the-Moment Guidance: Give agents instant context and in-the-moment guidance during every interaction. Surfacing relevant customer history, next-best action suggestions, and summaries in real time allows agents to resolve issues faster without switching tools.
• Resolving Complex Customer Needs: AI can handle routine enquiries with low latency, but human agents still excel at nuanced problem-solving. With AI feeding them persistent customer memory and sentiment analysis in real time, human agents can skip the repetitive questions and immediately focus on resolving complex issues, rescuing deals, or preventing churn.

When employees are equipped with real-time customer data and voice-driven insights, SMEs empower their teams to stop reacting to problems and start responding to customers proactively.

Consider global AI platform Genspark, which leverages a Programmable Voice API for its “Call for Me” agent to handle complex outbound tasks like checking supplier pricing or booking international hotels. The AI can conduct real-time, natural conversations across different languages on the user’s behalf, seamlessly navigating the live interactions before delivering a structured summary. Because these natural voice experiences depend entirely on speed and consistency, the underlying infrastructure provides the critical sub-second latency necessary to keep every automated call clear and uninterrupted.

6. Maintain transparency with customers

Finally, a successful AI implementation requires transparency. Customers should always know when they’re communicating with a bot and when they’ve been handed over to a human. AI-powered interactions must offer clarity by providing transparency about when and how AI is used and explaining next steps in plain language.

Transparency builds trust. Small businesses can go a step further by soliciting customer feedback on their AI interactions and using this input to fine-tune their systems.

For small enterprises, the AI-to-human handover isn’t about choosing between humans and machines; it’s about combining the strengths of both to create exceptional customer experiences. AI can provide the speed and efficiency customers expect, while humans deliver the empathy and creativity they value.

By strategically defining handover points, investing in human-like AI, and empowering agents to work alongside technology, organisations can build a CX strategy that’s as scalable as it is personal.

This blended approach ensures that every interaction – whether managed by a bot or a human – is thoughtful, natural, and distinctly on-brand.  

To learn more about Twilio, visit here.

The European Commission is ending an antitrust investigation into SAP after the company made numerous concessions worldwide regarding maintenance and support services for on-premises versions of its ERP solution. The Commission began its investigation in September 2025, concerned that SAP forces customers to buy its services for longer, and for more licenses, than they need.

In accepting SAP’s commitments, the Commission makes them binding on the company. SAP could still face fines if it fails to make good on its concessions over the next ten years. SAP’s promises include:

• Greater freedom of choice in support: Customers can divide their SAP landscape into sub-areas and choose different support and maintenance providers for each area.
• Easier license terminations: Maintenance and support contracts can be terminated in certain cases — such as when products are phased out, SAP projects fail, the company files for bankruptcy, there are staff reductions, or parts of the business are sold.
• More flexible licensing models: SAP is expanding access to so-called “single-metric” contracts as an alternative basis for licensing and maintenance fees.
• Relaxation of contractual obligations: In the future, the purchase of new licenses will no longer automatically extend the minimum term of existing support contracts.
• Easier Re-entry: Customers who resume SAP support after a hiatus will no longer have to pay reinstatement fees; back payments will also be reduced.

In addition, SAP is establishing an internal clearinghouse that customers can contact if they suspect violations of the agreed-upon obligations.

Even though SAP said it welcomed the EU Commission’s decision, the Walldorf-based company is unlikely to be truly happy with the concessions it had to make.

User organizations, though, are happy: Conor Riordan, chair of UKISUG, the UK & Ireland SAP User Group, said, “We welcome the proposed changes to SAP’s support and maintenance policies for on-premise customers. Our members have long called for greater flexibility, transparency and predictability, and these changes appear to be a positive move. This should give organizations more room to adapt to changing business conditions and evolve their SAP estates at a pace that suits them.”

More flexibility with SAP support

Michael Bloch, executive director of licensing, contracts and support at DSAG, the German-speaking SAP User Group, went into greater detail in an interview with Computerwoche, here translated from the German:

How do you assess the European Commission’s decision in general?

Michael Bloch: In principle, we think the decision is a good one for now. Many SAP customers will benefit from it, especially those who continue to run their ERP systems on-premises. Our investment survey shows that numerous companies have still not made the move to the SAP Cloud. They now have significantly more freedom to decide how they want to organize support for their existing software.

Who stands to benefit on the provider side? Does the decision open up new opportunities for third-party providers like Rimini Street? Can the potential demand even be met?

Bloch: That will be interesting to watch. At the moment, third-party maintenance is a total niche business, at least in German-speaking countries. Acceptance is significantly higher in the US, but here in Germany, many companies remain rather skeptical of the model. The EU decision could now give this market a new boost. Customers who do not wish to follow SAP’s current strategy will be able to remain on their existing infrastructure in the future and obtain support from another provider. This certainly opens up opportunities for new business models.

Does this decision undermine SAP’s cloud strategy?

Bloch: For customers, the decision now truly becomes a matter of principle: Do I follow SAP’s strategic direction, or do I consciously choose a different path? Those who aren’t convinced that the cloud strategy is the right one for their own company can now more easily decide to stay on their existing ERP landscape and, for example, use a different support provider.

However, one must be clear about the consequences. Those who remain on their current system landscape — even with an alternative maintenance provider — are forgoing the innovations that SAP is currently developing around the Autonomous Enterprise. There is no middle ground here. Companies would have to develop many of these functions themselves or recreate them at considerable expense.

So is this inevitably an either/or decision?

Bloch: No, that’s exactly the key point. Many companies have heavily customized their ERP systems over the years or decades to fit their industry-specific processes — some customers even refer to “refined” systems. These investments don’t simply have to be written off now.

Instead, companies can continue to operate their proven core systems while simultaneously adding targeted cloud components, such as SAP Cloud ERP for financial processes. This allows them to preserve existing investments while also leveraging new innovations. The EU decision thus opens up additional options for action, but it also makes the strategic decision more challenging for CIOs. They must now define even more precisely what their target architecture should look like for the next five to ten years.

2027 – A pivotal year for SAP support

Does this mean that IT decision-makers will now have to forgo a peaceful summer break?

Bloch: I don’t think this will fundamentally increase the pressure. Extended Maintenance doesn’t start until the end of 2027, so there’s still some time left. But companies now have an additional strategic question to answer. The actual decision year is likely to be 2027. By then, many companies will have to determine which system landscape they’ll use in the future and what their long-term SAP strategy will look like.

Even companies that want to stick with their existing ERP landscape will only receive official support until 2030. While that does buy some time, it’s by no means a long planning horizon, especially when alternative ERP strategies or successor solutions need to be evaluated.

That’s why I view it as a positive development that the decision has now been made. It provides clarity and gives companies the opportunity to incorporate these new conditions into their strategic considerations at an early stage.

The new regulations also make it easier for companies to dispose of unused licenses and the associated maintenance fees. How significant could the potential savings be?

Bloch: We don’t have specific figures on that. You have to be very careful here, because it depends heavily on the individual case. The key factor is whether a company actually terminates its SAP support entirely, switches to a third-party provider, or simply no longer needs certain products. After all, a system still has to be operated.

Is there at least a rough estimate?

Bloch: No, we don’t have reliable empirical data. If companies have products in use that they no longer use at all, they’ll be able to cancel support for them more easily in the future and, of course, save money as a result. However, we don’t know how large this so-called “shelfware” portion actually is.

For companies that have already migrated to S/4HANA or SAP Cloud ERP, this issue should largely be resolved anyway. It’s particularly relevant for those who still have the transition ahead of them. They can now review which unused licenses and maintenance contracts they can phase out and whether this is possible within the framework of SAP’s concessions, since regulations must be observed here as well. Those who are still paying substantial support fees today for products that are no longer in use can achieve significant savings by doing so.

Take advantage of the options

Does the EU decision improve companies’ negotiating position with SAP?

Bloch: There’s no one-size-fits-all answer to that. What matters is how a company makes use of its options. In my view, the best results are generally achieved by working with SAP to find a path forward.

Why?

Bloch: For example, if you completely cancel SAP support and later sign a new cloud contract, you should expect to forgo certain incentives from SAP. That’s why every decision should be made with all dependencies in mind.

In addition, SAP isn’t the only one offering incentives to move to the cloud. The hyperscalers also have a strong interest in attracting companies to their platforms and, in some cases, offer very attractive incentive programs. This means that companies’ starting points vary greatly.

Does this make decisions easier for CIOs?

Bloch: Quite the opposite, actually. While the new situation expands the range of options, it makes strategic decision-making significantly more complex. CIOs must now ask themselves: Which existing systems continue to provide business value for our company? Added to this are questions such as: Where is it worthwhile to retain the existing landscape? And in which areas will we actually benefit from the innovations that SAP will provide in the cloud in the future? Finding exactly this balance will be the real challenge.

So does this decision primarily mean that companies gain more time, for example, to weather difficult economic periods or to better prepare for a move to the cloud?

Bloch: Yes, especially for companies that haven’t yet found a compelling business case for moving to the SAP Cloud. They can continue to operate their existing landscape for the time being while simultaneously assessing whether there is still potential for cost savings by eliminating unused licenses and maintenance contracts, in other words, “shelfware.” This can certainly help in individual cases and provides more room to maneuver.

Complexity Is Increasing

Has the EU decision resolved the biggest problem in SAP licensing policy, or are there still issues to address?

Bloch: Extended Maintenance remains an important issue for companies that have not yet migrated to S/4HANA. The EU decision also has an impact here. Companies now have significantly more options for organizing their existing system landscape and support in different ways. They no longer have to treat their entire software portfolio uniformly, but can make differentiated decisions depending on the situation.

However, this also increases complexity. Companies now have a whole toolbox of options and must carefully weigh which combination is right for their strategy.

What’s the next crucial step?

Bloch: The key now is the practical implementation of the promised measures. Together with SAP, we need to clarify how the new regulations will be structured in detail and how companies can actually make use of them. The obligations will also be monitored by an independent trustee. I therefore hope that, together with SAP, we can provide more clarity on the operational implementation in the near future.

Does the EU decision now create additional pressure to act?

Bloch: I don’t think that this will immediately increase the pressure. There’s still some time left until Extended Maintenance begins at the end of 2027. That said, companies now face an additional strategic task: they must assess which new opportunities they want to take advantage of and how these fit into their SAP strategy.

When will things get serious?

Bloch: In my view, 2027 will be the decisive year. By then, many companies will need to determine which system landscape they will use to transition to Extended Maintenance and what their long-term SAP strategy should look like. Even companies that decide against moving to the SAP Cloud will gain some time as a result of the EU decision — but official support for their existing systems will end by 2030 at the latest. Especially when alternative ERP solutions are being evaluated in parallel, that’s not a particularly long planning horizon.

Your conclusion?

Bloch: Overall, it’s positive that the decision has now been made. It would have been much more difficult for companies if uncertainty had persisted until early 2027. Now the framework is clear, and companies can incorporate it into their strategic decisions early on.

OpenAI is sharpening its enterprise AI strategy with the launch of ChatGPT Work, a new agentic platform designed to automate workplace tasks, alongside the broader rollout of its GPT-5.6 models, which the company says deliver stronger performance at lower operating costs.

According to the company, ChatGPT Work can operate across applications and files, execute long-running tasks, coordinate multiple tools, and produce business documents, presentations, spreadsheets, and websites, allowing employees to delegate more complex workflows rather than interact through individual prompts.

GPT- 5.6 models, generally available weeks after a limited preview following US government restrictions on their broader rollout due to concerns about advanced cybersecurity and biology capabilities, can deliver stronger performance across coding, enterprise knowledge work, cybersecurity, and scientific research while lowering inference costs and token consumption, OpenAI said.

The launch marks a shift in OpenAI’s enterprise strategy. Rather than emphasizing benchmark leadership alone, the company is pitching GPT-5.6 around performance per dollar, arguing that enterprises deploying AI at scale increasingly care as much about operating costs as raw model capability.

“We trained GPT-5.6 to get more useful work from every token,” OpenAI said in a statement. “The result is stronger performance per dollar: more successful work for the same spend, or comparable results at a lower total cost.”

The models are now generally available through ChatGPT, Codex, and the OpenAI API. OpenAI has priced Sol at $5 per million input tokens and $30 per million output tokens, while Terra and Luna provide progressively lower-cost options for organizations scaling AI deployments, the statement added.

Enterprise AI shifts from experimentation to economics

ChatGPT Work combines GPT-5.6 with enterprise integrations and agentic capabilities that allow users to perform multi-step tasks across connected business applications instead of interacting with AI through isolated prompts. OpenAI said the platform is designed to help organizations automate knowledge work while maintaining enterprise-grade governance and security.

The launch comes as enterprises move beyond AI experimentation and begin deploying models across production workloads, making inference costs a growing concern for CIOs.

“The AI wave has brought productivity gains, but rising token consumption has also created bill shocks for enterprises,” said Neil Shah, vice president for research and partner at Counterpoint Research. “This is forcing organizations to adopt different models for different workloads, making performance per dollar the key metric.”

Faisal Kawoosa, co-founder and chief analyst at Techarc, said enterprises are now evaluating AI investments more pragmatically.

“The exploratory stage of AI is over,” he said. “Organizations can derive value from AI today, but performance per dollar will determine whether it becomes part of everyday business operations or remains an ad hoc tool.”

Tiered models for different workloads

GPT-5.6 Sol is OpenAI’s flagship model for complex reasoning, Terra targets mainstream enterprise applications, and Luna is designed for lower-cost, high-volume deployments.

According to OpenAI, GPT-5.6 Sol scored 53.6 on Agents’ Last Exam, a benchmark for long-running professional workflows, outperforming competing frontier models while requiring significantly lower compute costs.

The company also introduced two new reasoning modes. The max mode allocates additional compute for complex problems, while ultra coordinates four AI agents in parallel to accelerate demanding workflows.

“Ultra goes further by coordinating four agents in parallel by default, trading higher token use for stronger results and faster time-to-result on demanding tasks,” the statement added.

Shah said the architecture reflects how enterprises are increasingly orchestrating multiple AI models.

“GPT-5.6 gives enterprise architects flexibility to route workloads from Luna to Sol depending on whether they require automation, logic, or complex reasoning,” he said.

Kawoosa added that the tiered approach aligns with how enterprise software has traditionally been consumed.

“It gives enterprises of different sizes the flexibility to optimize technology consumption according to their requirements,” he said.

Coding, productivity, and security gains

OpenAI said GPT-5.6 Sol achieved a score of 80 on the Artificial Analysis Coding Agent Index while consuming fewer than half the output tokens of competing models. It also reported state-of-the-art results on Terminal-Bench 2.1 and DeepSWE, benchmarks that measure real-world software engineering tasks.

The company said the models also improve enterprise productivity through stronger document generation capabilities and integrations with Microsoft 365, Google Drive, Slack, and Notion.

On cybersecurity, GPT-5.6 Sol scored 73.5% on ExploitBench, up from 47.9% for GPT-5.5, and nearly doubled its predecessor’s performance on ExploitGym.

“GPT-5.6 supports important defensive tasks such as secure code review, patching, threat modeling, and blue teaming,” OpenAI said.

Security remains an enterprise focus

OpenAI said GPT-5.6 incorporates its “most robust safeguards to date,” combining model-level protections with real-time monitoring and extensive safety testing, including approximately 700,000 GPU hours of automated red-team evaluations.

Shah said layered guardrails and monitoring could become an important differentiator for enterprise deployments.

Kawoosa, however, said CIOs will continue demanding greater transparency before fully trusting frontier AI systems.

“Competition among LLM providers will continue, with vendors constantly testing and challenging each other’s guardrails,” he said.

The article originally appeared on InfoWorld.

You’ll be hard pressed to find an area of corporate America where AI hasn’t found a place, and that includes the tech hiring process. A survey from MyPerfectResume found that 73% of employers say they use AI in hiring decisions, while 52% use it for decisions around restructuring and role planning.

On the other side, candidates are also increasingly relying on AI, with 52% of current job seekers reporting they use AI to help them in their job searches to refine submission materials (85%) and prepare for interviews (73%), according to data from SAP.

“Technology can help employers be more efficient, but hiring decisions still benefit from human judgment, especially when a candidate’s experience requires context that automated screening may not understand,” says Jasmine Escalera, career expert at online career and résumé builder Zety.

It’s clear AI is an integral part of the hiring process, and organizations need to prepare a strategy for what that looks like moving forward in terms of hiring bias, transparency, and striking the right balance of human effort and AI assistance.

Recognizing the warning signs

AI has the promise of bringing efficiency in hiring for both job seekers and employees, but if organizations aren’t careful, an overreliance on AI technology can lead to unintended consequences. Further MyPerfectResume data also reveals 65% of respondents say AI often automatically rejects applicants before a person sees them, and 14% say AI rejects more than half of applicants outright.

Additionally, 47% say they feel AI has filtered out candidates who would’ve otherwise advanced in the process. And 51% say they use AI to flag risky candidates, such as people who might be viewed as job-hoppers or who have employment gaps.

Flagging risky candidates and eliminating them before a human can look at their résumé can filter out candidates with experience that tells a more complex story than an algorithm is designed to interpret, says Escalera. Candidates re-entering the workforce after time off, for example, may have valuable skills that don’t fit neatly into automated screening criteria, she adds.

Similarly, there’s concern AI will reject a professional who wants to change industries, or has qualifications that don’t  perfectly reflect the language in a job description before a human has a chance to look.

Laurie Cure, CEO of consulting firm Innovative Connections, says she’s seen instances where AI has eliminated highly qualified yet nervous candidates who take more time than what the AI allocated to answer a question, or candidates may simply use a different language than the AI is programmed to look for, causing them to not be recommended to progress in the process.

She’s also seen where AI might use historical data to determine patterns of a successful employee, identifying certain schools, work histories, tenure, or other characteristics that, while not inherently bias, perpetuates the bias that accurate correlations exist between these elements, when they often don’t. Organizations need to ensure that humans remain a part of these processes, Cure adds, where they can bring context, intuition, nuance, and an ability to identify potential in a candidate that AI can’t replicate.

“I think we’re allowing AI to become the process instead of allowing it to support the process in ways that makes hiring better,” she says.

An emphasis on accuracy over speed

Cure says a major problem for most companies is that the balance is off, with companies using AI for the majority, if not all, of résumé screening rather than as a complement to human efforts. Organizations that simply implement AI to speed up different parts of the hiring process, without taking time to consider if a process stands to benefit from AI, run the risk of introducing bias.

“While this allows for managing high volumes of applicants, and provides greater degrees of consistency in applying job criteria, it likely misses many good candidates,” she says. “The human element needs to be highly active in developing job requirements so they’re not too narrow. Organizations need to look at how they ask AI to do its work, so be cautious how you frame the screening or other criteria.”

Ultimately, AI isn’t a tool to be implemented and forgotten, or one that should be viewed simply as a path to efficiency since many processes still benefit from and require a human touch. It’s important to conduct audits of AI processes in hiring, and to remember that the use of AI doesn’t eliminate the legal or ethical obligations an organization has for equal employment, says Cure, making the balance between human and AI even more important.

AI transparency and fostering candidate trust

AI has also introduced an element of mistrust into hiring on both sides, where employers can’t be sure candidates haven’t relied on AI the same way candidates aren’t always sure exactly how AI is being used in the hiring process. Candidates are aware that employers are implementing AI, but they’re often unsure of the extent it’s being used and when to expect to interact with humans.

“That lack of clarity can create skepticism and frustration, particularly in a job market that already feels highly competitive,” says Escalera. “The goal shouldn’t be to convince candidates that AI isn’t being used, but to help them understand how technology supports decisions rather than replaces the human judgment behind them.”     

Cure recommends organizations start with a process map that outlines every step of an organization’s hiring process to help visualize where AI is beneficial and which processes still require human intervention. Companies can shift to relying too heavily on AI or they may become too dependent on human effort, when that effort could be put toward more important tasks.

“Humans bring an understanding of a person’s broader history, and the ability to detect when a candidate has potential to grow into the role,” she says. “People can see non-traditional career paths and motivations more distinctly than AI. Yet AI brings consistency, efficiency, criteria standardization, and a level of objectivity the process benefits from. If we effectively blend these two at the right points in the process, hiring is enhanced, not diminished.”

OpenAI is sharpening its enterprise AI strategy with the launch of ChatGPT Work, a new agentic platform designed to automate workplace tasks, alongside the broader rollout of its GPT-5.6 models, which the company says deliver stronger performance at lower operating costs.

According to the company, ChatGPT Work can operate across applications and files, execute long-running tasks, coordinate multiple tools, and produce business documents, presentations, spreadsheets, and websites, allowing employees to delegate more complex workflows rather than interact through individual prompts.

GPT- 5.6 models, generally available weeks after a limited preview following US government restrictions on their broader rollout due to concerns about advanced cybersecurity and biology capabilities, can deliver stronger performance across coding, enterprise knowledge work, cybersecurity, and scientific research while lowering inference costs and token consumption, OpenAI said.

The launch marks a shift in OpenAI’s enterprise strategy. Rather than emphasizing benchmark leadership alone, the company is pitching GPT-5.6 around performance per dollar, arguing that enterprises deploying AI at scale increasingly care as much about operating costs as raw model capability.

“We trained GPT-5.6 to get more useful work from every token,” OpenAI said in a statement. “The result is stronger performance per dollar: more successful work for the same spend, or comparable results at a lower total cost.”

The models are now generally available through ChatGPT, Codex, and the OpenAI API. OpenAI has priced Sol at $5 per million input tokens and $30 per million output tokens, while Terra and Luna provide progressively lower-cost options for organizations scaling AI deployments, the statement added.

Enterprise AI shifts from experimentation to economics

ChatGPT Work combines GPT-5.6 with enterprise integrations and agentic capabilities that allow users to perform multi-step tasks across connected business applications instead of interacting with AI through isolated prompts. OpenAI said the platform is designed to help organizations automate knowledge work while maintaining enterprise-grade governance and security.

The launch comes as enterprises move beyond AI experimentation and begin deploying models across production workloads, making inference costs a growing concern for CIOs.

“The AI wave has brought productivity gains, but rising token consumption has also created bill shocks for enterprises,” said Neil Shah, vice president for research and partner at Counterpoint Research. “This is forcing organizations to adopt different models for different workloads, making performance per dollar the key metric.”

Faisal Kawoosa, co-founder and chief analyst at Techarc, said enterprises are now evaluating AI investments more pragmatically.

“The exploratory stage of AI is over,” he said. “Organizations can derive value from AI today, but performance per dollar will determine whether it becomes part of everyday business operations or remains an ad hoc tool.”

Tiered models for different workloads

GPT-5.6 Sol is OpenAI’s flagship model for complex reasoning, Terra targets mainstream enterprise applications, and Luna is designed for lower-cost, high-volume deployments.

According to OpenAI, GPT-5.6 Sol scored 53.6 on Agents’ Last Exam, a benchmark for long-running professional workflows, outperforming competing frontier models while requiring significantly lower compute costs.

The company also introduced two new reasoning modes. The max mode allocates additional compute for complex problems, while ultra coordinates four AI agents in parallel to accelerate demanding workflows.

“Ultra goes further by coordinating four agents in parallel by default, trading higher token use for stronger results and faster time-to-result on demanding tasks,” the statement added.

Shah said the architecture reflects how enterprises are increasingly orchestrating multiple AI models.

“GPT-5.6 gives enterprise architects flexibility to route workloads from Luna to Sol depending on whether they require automation, logic, or complex reasoning,” he said.

Kawoosa added that the tiered approach aligns with how enterprise software has traditionally been consumed.

“It gives enterprises of different sizes the flexibility to optimize technology consumption according to their requirements,” he said.

Coding, productivity, and security gains

OpenAI said GPT-5.6 Sol achieved a score of 80 on the Artificial Analysis Coding Agent Index while consuming fewer than half the output tokens of competing models. It also reported state-of-the-art results on Terminal-Bench 2.1 and DeepSWE, benchmarks that measure real-world software engineering tasks.

The company said the models also improve enterprise productivity through stronger document generation capabilities and integrations with Microsoft 365, Google Drive, Slack, and Notion.

On cybersecurity, GPT-5.6 Sol scored 73.5% on ExploitBench, up from 47.9% for GPT-5.5, and nearly doubled its predecessor’s performance on ExploitGym.

“GPT-5.6 supports important defensive tasks such as secure code review, patching, threat modeling, and blue teaming,” OpenAI said.

Security remains an enterprise focus

OpenAI said GPT-5.6 incorporates its “most robust safeguards to date,” combining model-level protections with real-time monitoring and extensive safety testing, including approximately 700,000 GPU hours of automated red-team evaluations.

Shah said layered guardrails and monitoring could become an important differentiator for enterprise deployments.

Kawoosa, however, said CIOs will continue demanding greater transparency before fully trusting frontier AI systems.

“Competition among LLM providers will continue, with vendors constantly testing and challenging each other’s guardrails,” he said.

ultramansk | shutterstock.com

KI-Pipelines zu optimieren, erfordert mehr als nur oberflächliche Hardwareanpassungen. Es gilt, die Art und Weise, wie Modelle Daten verarbeiten, grundlegend zu verändern. Zwar implementieren KI-Engineers oft einfache Effizienzmaßnahmen innerhalb des Training-Loops. Aber um die Trainingskosten permanent zu reduzieren, sind architektonische Änderungen nötig – direkt im neuronalen Netz.

Die folgenden zwölf Optimierungsmaßnahmen auf Modellebene verwandeln Ihre KI-Strategie von einem Brute-Force-Hardware-Ansatz in eine elegante, softwaredefinierte Disziplin – und senken die Stückkosten Ihrer KI-Pipeline drastisch.

1. Pretraining einsparen

Ein Foundation-Modell von Grund auf neu zu trainieren ist für Standard Enterprise-Applikationen selten nötig und verbietet sich mit Blick auf die dafür nötige Rechenleistung. Statt dafür Millionen zu verschwenden, sollten Engineering-Teams lieber öffentlich verfügbare Open-Weight-Modelle nutzen.

Dieser grundlegende Transfer-Learning-Ansatz ist der unverzichtbare erste Schritt, wenn es darum geht, interne Chatbots oder domänenspezifische Klassifikatoren zu entwickeln. Indem bestehende neuronale Architekturen zum Einsatz kommen, lassen sich die enormen Kosten (auch für Energie), die mit den initialen Pretraining-Phasen verbunden sind, umgehen.

2. Parametereffizient feinabstimmen

Selbst das standardmäßige Feintuning von umfassenden Sprachmodellen erforderte immense Mengen an VRAM, um die States von Optimizern und Gradienten zu speichern. Um dieses Hardware-Bottleneck aufzulösen, sollten Engineers PEFT-Techniken wie LoRA implementieren.

Die Technik reduziert den Memory Overhead drastisch, indem sie dafür sorgt, dass 99 Prozent der vortrainierten Weights „eingefroren“ und kleine, trainier- und adaptierbare Layer injiziert werden. Dieser mathematische Shortcut ist ideal geeignet, um hochgradig anpassbare GenAI-Funktionen umzusetzen – und erlaubt die Feinabstimmung von Milliarden von Parametern mit einer einzigen (Consumer-)GPU.

python
from peft import LoraConfig, get_peft_model

config = LoraConfig(r=8, lora_alpha=32, target_modules=["q_proj", "v_proj"])
efficient_model = get_peft_model(base_model, config)

3. Warmstart-Layer einziehen

Falls Sie spezifische Netzwerkkomponenten von Grund auf neu trainieren müssen, stellt der Import vortrainierter Embeddings sicher, dass nur die verbleibenden Layer erhöhten Rechenaufwand verursachen.

Dieser „Warmstart“-Ansatz reduziert den Rechenaufwand in Frühphasen der KI-Entwicklung erheblich, weil das Modell so grundlegende, universelle Datenrepräsentationen nicht erst neu erlernen muss. Besonders empfehlenswert ist dieser für Spezialdomänen.

python
# PyTorch warm-start example
model.embedding_layer.weight.data.copy_(pretrained_medical_embeddings)
model.embedding_layer.requires_grad = False

4. Gradient Checkpointing anwenden

Memory-Engpässe sind der wesentliche Grund dafür, dass Entwickler gezwungen sind, teure, VRAM-intensive Cloud-Instanzen zu mieten. Gradient Checkpointing (PDF) spart Speicherplatz ein, indem bestimmte Forward Activations während der Backpropagation neu berechnet werden – anstatt sie alle zu speichern.

Entwicklern ist zu empfehlen, diese Technik einzusetzen, wenn sie mit anhaltenden „Out of Memory“-Fehlern konfrontiert sind. Denn Gradient Checkpointing ermöglicht es, zehnmal größere Netzwerke auf derselben GPU unterzubringen – bei einem Mehr an Rechenaufwand von circa 20 Prozent.

python
# Enable in Hugging Face / PyTorch
model.gradient_checkpointing_enable()

5. Compiler-Fusion aktivieren

Moderne Deep-Learning-Frameworks leiden regelmäßig unter Engpässen mit Blick auf die Speicherbandbreite, da ständig Daten über die Hardware gelesen und geschrieben werden. Durch Compiler, die wie XLA oder PyTorch 2.0 auf Graph-Ebene operieren, lässt sich eine Vielzahl von Prozessen in einem einzelnen GPU-Kernel fusionieren.

Diese architektonische Optimierung führt dazu, dass Durchsatz und Ausführungsgeschwindigkeit massiv gesteigert werden. Parallel sind allerdings keine manuellen Änderungen am Code notwendig. Um die Hardwareauslastung zu maximieren, ist es Entwickler-Teams zu empfehlen, die Compiler-Fusion standardmäßig bei sämtlichen Trainings-Sessions in der Produktion zu aktivieren.

python
import torch

# PyTorch 2.0 compiler fusion
optimized_model = torch.compile(model)

6. Pruning und Quantisierung einsetzen

Ein umfangreiches, vollpräzises 16-Bit-Neural-Network in der Produktion bereitzustellen, erfordert ebenfalls oft teure Cloud-Instanzen, was die Gewinnmarge einer Applikation zunichtemachen kann. Durch algorithmisches Pruning werden mathematisch redundante Weights entfernt.

Eine Quantisierung des Modells sorgt hingegen dafür, dass die verbleibenden Parameter von 16-Bit-Gleitkommazahlen auf 8-Bit- oder 4-Bit-Ganzzahlen komprimiert werden. Das ermöglicht es, das KI-Modell auf deutlich kostengünstigeren GPUs mit geringerem Speicherbedarf auszuführen – ohne dass die Qualität der Konversationen darunter leidet. Diese physikalische Reduktion ist entscheidend dafür, Traffic-intensive Anwendungen kosteneffizient skalieren zu können. Davon abgesehen senkt es jedoch auch die CO²-Kosten, die ein API-Call verursacht, wenn Tausende von Usern parallel bedient werden.

python
import torch
import torch.nn.utils.prune as prune

# 1. Prune 20% of the lowest-magnitude weights in a layer
prune.l1_unstructured(model.fc, name="weight", amount=0.2)

# 2. Dynamic Quantization (Compress Float32 to Int8)
quantized_model = torch.ao.quantization.quantize_dynamic(
    model, {torch.nn.Linear}, dtype=torch.qint8
)

7. Curriculum Learning verwenden

Ein untrainiertes neuronales Netzwerk mit hochkomplexen und gleichzeitig verrauschten Datensätzen zu füttern, zwingt den Optimizer teure Extra-Rechenschleifen zu drehen, um chaotische Gradienten abzubilden. Dieses Problem lässt sich mit Curriculum Learning (auch Lehrplanlernen) lösen: Dabei wird die Daten-Pipeline so strukturiert, dass zunächst klare, leicht klassifizierbare Beispiele eingeführt werden – bevor der schrittweise Übergang auf hochpräzise Anomalien erfolgt.

Geht es etwa darum, ein Vision-Modell für autonomes Fahren zu trainieren, sollten die Entwickler diesem zunächst klare Tageslichtbilder von Autobahnen zuführen, bevor sie Rechenleistung für komplexe Nachtaufnahmen verschneiter Stadtkreuzungen in Städten aufwenden. Dieser schrittweise Ansatz ermöglicht dem Netzwerk, zentrale mathematische Merkmale ressourcenschonend abzubilden. Dadurch wird die Konvergenz deutlich schnell und mit geringerem Hardware-Aufwand erreicht.

8. Wissen destillieren

Ein massives KI-Modells mit 70 Milliarden Parametern für simple, repetitive Tasks zu nutzen, kommt einer gravierenden Fehlallokation von Rechenressourcen gleich. Dieses Problem lässt sich mithilfe von Wissensdestillation lösen: Dabei lernt ein hocheffizientes, schlankes „Student“-Modell, die Reasoning-Ketten eines großen „Teacher“-Modells exakt nachzuahmen.

Stellen Sie sich ein E-Commerce-Unternehmen vor, das Produktempfehlungen in Echtzeit direkt auf dem Smartphone eines Nutzers ausführen muss, wo Akku und Speicher streng limitiert sind. Dank Knowledge Distillation kann dieses winzige Mobile-Modell mit der Genauigkeit einer massiven, Cloud-basierten Architektur arbeiten. Das senkt die Inferenzkosten dauerhaft und kann Ihnen außerdem ersparen, in die „AI Accuracy Trap“ zu tappen.

9. Suchmethoden optimieren

Herkömmliche Grid-Search-Algorithmen fressen regelmäßig große Teil des Cloud-Budgets, weil sie blindlings Netzwerkkonfigurationen testen und ausführen, die von vornherein zum Scheitern verurteilt sind. Intelligentere Hyperparameter-Suchmethoden wie die Bayes’sche Optimierung und Hyperband können an dieser Stelle als finanzielle Wächter fungieren: Sie sagen unzureichende Versuche mathematisch vorher und sortieren diese direkt aus.

Optimiert eine Bank beispielsweise ein KI-Modell zur Betrugserkennung, kann Hyperband Konfigurationen aufspüren, die nicht akkurat sind – und lenkt die gesamte Rechenleistung ausschließlich auf die vielversprechendsten Setups um. Um die Kosten weiter zu reduzieren, lässt sich zudem auch das RES-Cost-Aware-Retraining-Framework integrieren.

10. Parallelstrategien fahren

Nicht sachgemäß konfigurierte Cluster führen ebenfalls zu massiven Netzwerk-Bottlenecks. Wenn Sie ein Modell mittlerer Größe auf zu viele GPUs aufteilen (Modellparallelität), verbringen die Prozessoren mehr Zeit damit, auf die Datenübertragung zu warten, als damit, tatsächlich Berechnungen durchzuführen.

Umgekehrt ist es bei der Verarbeitung großer Datensätze hocheffizient, das gesamte Modell über mehrere Knoten (Datenparallelität) zu replizieren – vorausgesetzt, die Batch-Größen sind korrekt abgestimmt. Ein FinOps-Team in der Praxis muss diese Parallelstrategien dynamisch an die jeweilige Architektur anpassen und dabei sicherstellen, dass die GPUs nicht in den Idle-Status verfallen, während das Netzwerk aufholt.

11. Asynchron evaluieren

Standardmäßige Trainings-Pipelines sorgen ständig dafür, dass das primäre (und teure) GPU-Cluster eine Pause einlegen muss. Einfach nur, um routinemäßige Validierungsprüfungen der Modellfortschritte durchzuführen. Anders ausgedrückt: Es ist eine katastrophale Geldverschwendung.

Indem Engineering-Teams asynchrone Evaluierung implementieren, lassen sich die Validierungsprüfungen auf eine separate, wesentlich kostengünstigere CPU- oder Low-Tier-GPU-Instanz auslagern. Die primären, kostenintensiven GPUs möglichst voll auszulasten, ist eine verpflichtende architektonische Trennung. Diese trägt dazu bei, die versteckten Betriebskosten abzumildern, die mit der KI-Governance einhergehen. 

12. Daten kuratieren

Riesige Datensätze blind zu verarbeiten, sorgt ebenfalls dafür, dass teure Compute-Zeit verschwendet wird – in diesem Fall für redundante Informationen von minderer Qualität.

Wenn ein visuelles KI-Modell bereits zehntausend identische Fotos eines Standard-Stoppschilds erfasst hat, liefert es keinerlei Mehrwert, noch einmal ein paar mehr nachzulegen. Algorithmisches Sampling zu nutzen, um informationsreiche Subsets zu kuratieren, resultiert in identischer Modell-Performance – zu einem Bruchteil der Hardwarekosten. (fm)

Dieser Beitrag wurde im Rahmen des englischsprachigen Expert Contributor Network von Foundry veröffentlicht. Alle Infos zum deutschsprachigen Experten-Netzwerk finden Sie hier.

The first time I used Kubernetes in an enterprise setting, I understood the hype. It gives every team the same way to package, deploy and run their apps. No more custom scripts or unique deployment hacks, just one control plane to rule them all. And really, that’s why it’s so popular with big companies: Kubernetes is an open-source system for automating deployment, scaling and management of containerized applications. It says so right on the box, and that’s what people want. But here’s the truth: Kubernetes doesn’t erase operational headaches. It just moves them around.

When your Kubernetes install is small, it feels like rocket fuel for engineers. At enterprise scale, though, suddenly it’s about governance, not just engineering. The game is no longer “Can we get this container running?” It’s “How do hundreds of engineers roll out their stuff safely, consistently, securely and without breaking the bank or burning out the platform team?”

This is where the fun really starts.

YAML isn’t the enemy

Folks new to Kubernetes obsesses over manifests, Helm charts, namespaces, ingress rules, deployments, all that stuff. But they’re not the hardest part once you start scaling. The real beast is standardization.

Every big company I’ve seen ends up with teams going their own way. One group writes beautiful deployment templates. Someone else copies and pastes from a two-year-old manifest. Some folks set resource requirements properly. Others skip them entirely. One team sticks to a strong naming convention, and someone else throws together random namespaces and service accounts that make sense only to them. Individually, this more or less works. At scale, when the whole platform has to operate like one system, it’s a mess.

That’s why I’ll say it: you don’t just need a Kubernetes cluster. You need a paved road. This would involve ensuring that there are approved templates, good deployment patterns, observability, security controls as defaults, good issue escalation processes and accountability.

There is no need for developers to be Kubernetes experts just to release their services. The best enterprise Kubernetes setups work like real products. They let application teams self-serve but never let anyone veer off road without good reason.

RBAC: necessary, but never enough

Security is paramount. Kubernetes supports role-based access control (RBAC), so on paper you can control who does what. In practice, in a big company, RBAC gets confusing fast.

The issue isn’t that engineers ignore security. It’s that permissions grow over time. You need a quick fix during an incident, so you give a service account more access. Maybe a team needs cluster-wide rights for a migration. That “just for now” permission sticks around because no one cleans it up. Month by month, the gap widens between what a workload should do and what it’s actually allowed to do. The only thing that works long-term: treat RBAC as a living thing, not a one-time checklist. Review it. Test it. Stick to least privilege. Service accounts get only what they need. Cluster-admin rights? Rare. Expiring exceptions. Set permissions as code so changes aren’t invisible.

Same story with workload security. Kubernetes brings you Pod Security Standards. There is baseline, restricted and privileged profiles, so everyone speaks the same language. But simply setting a standard isn’t enough. We’d also need things like admission controls, image scanning, runtime monitoring and audit trails.

Honestly, the NSA/CISA Kubernetes Hardening Guidance is still gold. Scan containers and pods. Run workloads as locked down as possible. Use strong authentication. Separate networks. Set up solid logging. These ideas sound obvious until you see what happens when your organization scales without good ops.

Network policies: where “it should work” meets reality

Kubernetes networking can trip up even the best teams. Engineers often think different namespaces mean automatic isolation between apps. Not true.

Kubernetes network policies decide which pods can talk to which, but the policies only matter if your networking plugin actually enforces them. I’ve seen a lot of teams write network controls that look great in YAML but don’t work, because the underlying network just ignores them. Security validation beats documentation every time. If two namespaces shouldn’t talk, test it. If a workload only needs access to a specific backend, check it. If only specific ingress is allowed, make sure nothing else gets through.

At scale, your Kubernetes security has to prove itself. “We have a policy” means nothing unless the platform can show the policy actually works.

Resource management becomes all about money

One of the biggest challenge is resource allocation. Kubernetes lets you set CPU and memory limits, and sure, there are official docs. But getting these numbers right is tough.

Set them too low, and your workload might get throttled or evicted under load. Set them too high, and you’re paying for unused infrastructure. That barely registers on a small cluster, but when you’re running thousands of pods? That’s cloud bills gone wild.

This is where Kubernetes ops and FinOps meet. Platform teams have to know who’s burning through which resources, what’s over-provisioned or flying blind, and where the real money goes. ResourceQuota helps keep things in check, but quotas alone don’t hold people accountable.

The culture shift is moving from “the cluster has spare capacity” to “every service has an owner, a cost profile and a plan for staying lean.” Teams should understand their infrastructure bill. Platform teams need dashboards that point out waste. Engineering leaders need to care about efficiency, not just hear from finance when things go off the rails.

Autoscaling isn’t a magic trick

The Horizontal Pod Autoscaler is handy. It adjusts your workloads automatically to match demand. But don’t overestimate it. Most real-world services don’t scale simply by CPU or memory. Sometimes a service hits latency limits before CPU usage spikes. Workers chewing through queues? You care more about backlog size. Machine learning? Maybe it’s all about GPU use or loading time. Customer-facing apps? You want to be scaled up before traffic hits, not scramble after users start complaining.

So autoscaling isn’t just a box you check. It’s a feedback loop, and it only works if you use the right signals. Sometimes CPU is enough. Sometimes you need to scale on queue length, request rate, latency or something totally custom.

Then there’s node autoscaling to provision infrastructure in response to demand. On paper, it just works. In real life, it runs into startup delays, availability zones, quotas, cloud provider quirks and pod disruption budgets. Scale pods faster than nodes? Users still see delays.

Test autoscaling like you test your app. Load-test it, break it, see what happens after an incident. Otherwise, you’ll find the limits when it hurts most.

Observability doesn’t matter unless it answers questions

Kubernetes has mountains of data. Things like  logs, metrics, traces, events, audits, deployment history, container restarts, control plane noise, you name it. The real challenge isn’t collecting info, but actually it’s making sense of it. The CNCF and others have best practices for logging and telemetry, like centralizing logs and not leaking secrets. Those matter, but at the end of the day, engineers need answers, not just data. When something breaks, no one’s asking, “Is Kubernetes alive?” They want to know what changed. Did something roll out? Did a pod crash? Did autoscaling fire too late? Was a node unhealthy, a secret rotated, a network policy too tight, a downstream DB choking?

Observability should line up with real operational questions and not just ticking boxes for logs, or metrics. Dashboards need to match service ownership. Alerts need to mean something to end users. Telemetry should connect to deployments and incidents. Measure how quickly engineers spot the root cause, not just that you have the data somewhere.

CNCF talks about newer models of unified telemetry and proactive troubleshooting for a reason. All the dashboards in the world don’t help when your team has to play detective during an outage.

Upgrades: Don’t wing it

Kubernetes upgrades catch people out. The CNCF Maturity Model says: Kubernetes drops three big releases a year, so maintenance is part of life—not a once-in-a-blue-moon project.

Upgrading at enterprise scale can involve everything: workloads, admission controllers, CI/CD, service mesh, ingress, storage drivers, monitoring, security, custom controllers. Version skew policies keep you between the lines, but that’s just the beginning. The real question is: can you test your whole stack?

Good upgrade programs need a repeatable process, staging environments that actually look like production, and clear communication so teams know what to expect. The worst upgrade process is the one that relies on heroes to pull it off at the last second. A strong platform turns upgrades into routine.

Reliability: Kubernetes helps, but it doesn’t guarantee it

Yes, Kubernetes restarts crashed containers, reschedules pods and does rolling deployments. But it doesn’t make a bad app reliable.

A poorly coded app will fail on Kubernetes just like anywhere else. Bad readiness or liveness probes? Your app gets traffic too soon. No graceful shutdown? Requests drop during deploy. Forgot pod disruption budgets? The app goes down during node maintenance. A flaky dependency? It will cascade through your services even if all your pods look healthy.

The mature approach is setting service-level objectives and making reliability a product of both platform and engineering. Cluster health isn’t user experience. That green status page can hide a lot of pain.

The platform team is a product team

Here’s the biggest lesson I’ve picked up is that running Kubernetes at enterprise scale isn’t really about the tech. One cluster? Maybe one expert can handle that. But for a full enterprise platform, you need a product mindset. The platform team serves customers such as engineers, security, compliance, finance and business. Everyone wants something a bit different.

Developers want speed and reliability. Security wants oversight. Finance wants transparency. Compliance wants proof. Ops wants predictability. The business wants all of those.

The platform team has to pull those threads together with APIs, docs, dashboards, paved roads, support and feedback. That also means saying “no” to the unique snowflake patterns that create chaos later. Kubernetes is powerful. But it doesn’t replace organizational discipline. That’s still on the shoulders of engineering leaders. The real challenge at enterprise scale isn’t memorizing every API object. It’s building a system where any team can ship safely without needing to be Kubernetes experts themselves.

When you reach that point, Kubernetes stops being just a cluster. It becomes your platform.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

As enterprises struggle to manage their AI strategies, the US AI regulatory environment is sending a wide range of contradictory signals. OpenAI’s Wednesday announcement that it will now release GPT-5.6 Sol, along with Terra and Luna, on Thursday highlights the confusion.

Initially, the US government said that it was asking OpenAI to limit access to its top models, including the three releasing Thursday, to a short list of companies. OpenAI seemingly agreed and held back their general availability.

But on Wednesday, OpenAI reversed its position, with a statement on X saying simply: “GPT-5.6 Sol, along with Terra and Luna, will launch publicly this Thursday. We’re expanding preview access globally now.” No details were released about the extent of the expansion.

Then the White House issued a statement, a copy of which it emailed to InfoWorld, saying that the US government “did not give OpenAI a ‘green light,’ approval or clearance to release its models. No such permission is required or granted. The Administration does not provide approvals for private companies to release AI models – decisions on timing and scope of releases rest entirely with the companies.”

The statement then quoted from the June 2 White House executive order that said, “nothing in this section shall be construed to authorize the creation of a mandatory governmental licensing, preclearance, or permitting requirement for the development, publication, release, or distribution of new AI models, including frontier models.” It also said, “any testing or meetings with government experts is voluntary. Participation is not required to release a model.”

Yet last month, the US Commerce Department weighed in on how Anthropic’s models can be distributed. 

The ‘worst of both worlds’

Lewis Carhart, CEO of software development firm Comp AI, said the statement is frustrating for IT executives on multiple fronts. 

“Think about what that [White House statement] means. OpenAI stationed engineers in Washington for weeks, submitted to government testing, staggered its launch at the government’s request. And the official position is that none of that was required,” Carhart said. “We now have a de facto licensing regime that legally doesn’t exist. There’s no statute, no appeal process, no published criteria. Just [the US Department of] Commerce deciding model by model what ships and when.”

That’s the worst of both worlds, he noted: “All the friction of regulation with none of the predictability. Compliance people have a name for this – it’s an audit with no framework. And the precedent is now locked in for both frontier labs: if you build at the frontier, your launch calendar runs through Washington whether the law says so or not.”

Carhart argued that this regulatory reality should be of extreme concern to enterprise IT executives, given it indicates that model availability is now “a regulatory variable” not driven by the vendor roadmap.

“Anthropic’s most advanced models disappeared from the market for three weeks in June. It was not because of an outage, not because of a pricing change. It was because of an export control directive,” he pointed out. “If your AI architecture assumes the model you deployed today is available tomorrow, that assumption is now demonstrably false. Multi-model resilience just went from nice to have to a board-level risk item.”

It also offers an opportunity, given that a model that cleared government security testing is a model that auditors and boards sign off on faster. “Government review is quietly becoming a procurement asset,” he observed. “The CIOs who win here are the ones who treat ‘regulatory posture of the model itself’ as a line item in vendor risk assessments – most risk teams are still only looking at the provider’s SOC 2 attestation.”

Jason Andersen, principal analyst at Moor Insights & Strategy, agreed with Carhart and described the overall back-and-forth as “a bit of pageantry. OpenAI needs its model to look as powerful, potentially dangerous, as Anthropic’s so it can be a contender to be at the absolute frontier. It also helps to burnish OpenAI’s PR efforts to look more responsible than it has in the past.”

Furthermore, he added, “tech CEOs are acutely aware that flattery towards this administration could keep regulators or the threat of regulation at bay.”

Criteria not clear

Brian Jackson, a principal research director at Info-Tech Research Group, echoed the political concerns. 

“What’s still not clear is the actual criteria being used to deem the models safe for release. The government has said that it’s concerned about cybersecurity risk as well as the risk of AI being used to develop biological weapons,” he said. “But so long as the actual release criteria lack transparency, there will be some perception that the evaluation could be politically motivated.”

Jackson said that one, presumably unintended, result of the US government’s efforts to control AI rollouts is that it is making companies look far more seriously at using non-US vendors for AI strategies. 

“Organizations are looking for alternatives to the private US-based cloud-delivered frontier models. That’s so they can maintain control over their AI supply chain,” he said. “Chinese open source models are one option that’s available, but there are other options too, from Canada and Europe. Companies can either set up AI access through other APIs not connected to US-based AI providers, or download open-source models to run locally.”

He noted that the added US regulatory risk means that some organizations will avoid becoming entrenched within OpenAI’s and Anthropic’s interfaces, where there’s no option to swap out the LLMs for an alternative.

Impacts enterprise AI strategy

Rock Lambros, director of AI standards and governance at AI agent vendor Zenity, shared the frustration that little to no actionable compliance data is being released. 

“Nobody outside a closed room can tell you what standard [the model] passed because it was never written down. For two weeks, [US government officials] kept a model out of defenders’ hands that’s better at guarding your network than breaking into anyone else’s,” Lambros said. “Call that a security review if it helps you sleep better. But it reads to me like a bouncer working a velvet rope nobody hired him to run, waving people through today because he’s in a better mood than he was a couple of weeks ago.”

This unpredictability is likely to have impacts on AI strategy far beyond traditional compliance concerns, Lambros said.

“We’ve built way too much operational reliance on these models to hang it on a review with no rulebook,” he said, pointing out that hospitals, pipelines, banks and water utilities are relying on frontier AI whose availability “can swing from ‘on’ to ‘off’ to ‘on’ with no notice, no appeal, and no published standard behind any of it.”

“You can’t run critical infrastructure on a tool that runs fine Friday and is offline by Monday because an approval process nobody can see reached a verdict nobody can predict,” Lambros said. “That is a supply chain risk with a government hand on the switch, and almost nobody has priced it into a continuity plan.”

To protect themselves, companies need to adjust their expectations. “Treat model availability like any single point of failure you don’t own by standing up a fallback you’ve tested, getting a continuity clause in your contract, and drilling for the blackout, because ‘the government backed off this time’ is not a plan,” he advised.

As enterprises struggle to manage their AI strategies, the US AI regulatory environment is sending a wide range of contradictory signals. OpenAI’s Wednesday announcement that it will now release GPT-5.6 Sol, along with Terra and Luna, on Thursday highlights the confusion.

Initially, the US government said that it was asking OpenAI to limit access to its top models, including the three releasing Thursday, to a short list of companies. OpenAI seemingly agreed and held back their general availability.

But on Wednesday, OpenAI reversed its position, with a statement on X saying simply: “GPT-5.6 Sol, along with Terra and Luna, will launch publicly this Thursday. We’re expanding preview access globally now.” No details were released about the extent of the expansion.

Then the White House issued a statement, a copy of which it emailed to InfoWorld, saying that the US government “did not give OpenAI a ‘green light,’ approval or clearance to release its models. No such permission is required or granted. The Administration does not provide approvals for private companies to release AI models – decisions on timing and scope of releases rest entirely with the companies.”

The statement then quoted from the June 2 White House executive order that said, “nothing in this section shall be construed to authorize the creation of a mandatory governmental licensing, preclearance, or permitting requirement for the development, publication, release, or distribution of new AI models, including frontier models.” It also said, “any testing or meetings with government experts is voluntary. Participation is not required to release a model.”

Yet last month, the US Commerce Department weighed in on how Anthropic’s models can be distributed. 

The ‘worst of both worlds’

Lewis Carhart, CEO of software development firm Comp AI, said the statement is frustrating for IT executives on multiple fronts. 

“Think about what that [White House statement] means. OpenAI stationed engineers in Washington for weeks, submitted to government testing, staggered its launch at the government’s request. And the official position is that none of that was required,” Carhart said. “We now have a de facto licensing regime that legally doesn’t exist. There’s no statute, no appeal process, no published criteria. Just [the US Department of] Commerce deciding model by model what ships and when.”

That’s the worst of both worlds, he noted: “All the friction of regulation with none of the predictability. Compliance people have a name for this – it’s an audit with no framework. And the precedent is now locked in for both frontier labs: if you build at the frontier, your launch calendar runs through Washington whether the law says so or not.”

Carhart argued that this regulatory reality should be of extreme concern to enterprise IT executives, given it indicates that model availability is now “a regulatory variable” not driven by the vendor roadmap.

“Anthropic’s most advanced models disappeared from the market for three weeks in June. It was not because of an outage, not because of a pricing change. It was because of an export control directive,” he pointed out. “If your AI architecture assumes the model you deployed today is available tomorrow, that assumption is now demonstrably false. Multi-model resilience just went from nice to have to a board-level risk item.”

It also offers an opportunity, given that a model that cleared government security testing is a model that auditors and boards sign off on faster. “Government review is quietly becoming a procurement asset,” he observed. “The CIOs who win here are the ones who treat ‘regulatory posture of the model itself’ as a line item in vendor risk assessments – most risk teams are still only looking at the provider’s SOC 2 attestation.”

Jason Andersen, principal analyst at Moor Insights & Strategy, agreed with Carhart and described the overall back-and-forth as “a bit of pageantry. OpenAI needs its model to look as powerful, potentially dangerous, as Anthropic’s so it can be a contender to be at the absolute frontier. It also helps to burnish OpenAI’s PR efforts to look more responsible than it has in the past.”

Furthermore, he added, “tech CEOs are acutely aware that flattery towards this administration could keep regulators or the threat of regulation at bay.”

Criteria not clear

Brian Jackson, a principal research director at Info-Tech Research Group, echoed the political concerns. 

“What’s still not clear is the actual criteria being used to deem the models safe for release. The government has said that it’s concerned about cybersecurity risk as well as the risk of AI being used to develop biological weapons,” he said. “But so long as the actual release criteria lack transparency, there will be some perception that the evaluation could be politically motivated.”

Jackson said that one, presumably unintended, result of the US government’s efforts to control AI rollouts is that it is making companies look far more seriously at using non-US vendors for AI strategies. 

“Organizations are looking for alternatives to the private US-based cloud-delivered frontier models. That’s so they can maintain control over their AI supply chain,” he said. “Chinese open source models are one option that’s available, but there are other options too, from Canada and Europe. Companies can either set up AI access through other APIs not connected to US-based AI providers, or download open-source models to run locally.”

He noted that the added US regulatory risk means that some organizations will avoid becoming entrenched within OpenAI’s and Anthropic’s interfaces, where there’s no option to swap out the LLMs for an alternative.

Impacts enterprise AI strategy

Rock Lambros, director of AI standards and governance at AI agent vendor Zenity, shared the frustration that little to no actionable compliance data is being released. 

“Nobody outside a closed room can tell you what standard [the model] passed because it was never written down. For two weeks, [US government officials] kept a model out of defenders’ hands that’s better at guarding your network than breaking into anyone else’s,” Lambros said. “Call that a security review if it helps you sleep better. But it reads to me like a bouncer working a velvet rope nobody hired him to run, waving people through today because he’s in a better mood than he was a couple of weeks ago.”

This unpredictability is likely to have impacts on AI strategy far beyond traditional compliance concerns, Lambros said.

“We’ve built way too much operational reliance on these models to hang it on a review with no rulebook,” he said, pointing out that hospitals, pipelines, banks and water utilities are relying on frontier AI whose availability “can swing from ‘on’ to ‘off’ to ‘on’ with no notice, no appeal, and no published standard behind any of it.”

“You can’t run critical infrastructure on a tool that runs fine Friday and is offline by Monday because an approval process nobody can see reached a verdict nobody can predict,” Lambros said. “That is a supply chain risk with a government hand on the switch, and almost nobody has priced it into a continuity plan.”

To protect themselves, companies need to adjust their expectations. “Treat model availability like any single point of failure you don’t own by standing up a fallback you’ve tested, getting a continuity clause in your contract, and drilling for the blackout, because ‘the government backed off this time’ is not a plan,” he advised.

This article originally appeared on InfoWorld.

Anthropic says Claude now writes more than 80% of the code merged at one of the most sophisticated AI companies on the planet. Foundry’s 2026 State of the CIO study says fewer than one in five enterprises can show that their AI initiatives have met or exceeded their ROI goals. Both numbers came out this spring. Both are true. And the distance between them is the most important thing an IT leader can understand about AI right now.

Because that distance isn’t a contradiction, it’s a lesson. And the profession sitting in the middle of it, software engineering, is the canary that explains why so much enterprise AI spend has produced so little measurable return.

The report everyone misread

When Anthropic published its recursive self-improvement piece, plenty of people read it as the starting gun for the job apocalypse. Claude writing its own code, models getting better at building models, humans narrowing toward oversight. If you wanted a headline about the end of the software profession, it was right there.

I read it almost the opposite way. What struck me wasn’t how far AI had come. It was how much had to be true first, even in the one profession built from the ground up to let it succeed.

I made this argument back in my “$570K canary” piece, and the Anthropic data only sharpens it. AI coding agents don’t work because coding models are special. The underlying large language models (LLMs) are the same ones answering support tickets and reviewing contracts. They work because software development already had the infrastructure that makes an agent’s output trustworthy: governance baked into branch protection and code review, observability through version control and CI/CD pipelines, evaluation through automated tests, persistent context through commit history. Developers built all of that for themselves over decades. They didn’t build it for AI. But it turned out to be exactly the scaffolding AI needed.

That’s the part the apocalypse reading skips. Claude’s coding gains are real. They also rode on decades of pre-built substrate. Both things are true at once, and the second one is the one CIOs should be paying attention to when it comes to gains from things like recursive self-improvement.

What the CIO data actually shows

Now hold that next to the State of the CIO numbers. Only 19% of the 662 IT leaders surveyed say their AI initiatives have met or exceeded business goals. Another 18% admit fewer than a third of their use cases are hitting defined expectations.

The easy explanation is that the technology isn’t ready. The data says otherwise. This isn’t for lack of trying, and it isn’t for lack of organizing. Eighty-three percent of respondents have stood up cross-functional steering committees or are about to. Just over half have some form of AI approval process in place, with another quarter building one. Forty-seven percent have formal success metrics, with a third more on the way. The field is pouring effort into the organizational machinery of AI. The ROI still isn’t showing up.

Here’s why I think that is. All of that machinery sits above the work. Steering committees, approval gates and KPI dashboards govern the org chart. But the value, or the leak, happens inside the workflow, at the level of the actual task the AI is doing. You can instrument your governance structure perfectly and still have nothing measuring whether the agent’s output was right at the point where it mattered.

TIAA shows how little the org chart settles. The firm is three years in, runs generative and agentic use cases across fraud detection and call centers, and has 85% of its people on TIAA Gate, its internal platform. It also has the full governance stack most CIOs are still assembling. None of it closed the gap. “You need to understand the full cost of operations,” its chief operating, information and digital officer, Sastry Durvasula told CIO.com, “the efficiencies of running tokens or how you’re handling traffic or RAG.” The structures were never the thing leaking value. The workflow underneath them was.

The barriers respondents named back this up. The top three are lack of in-house expertise (40%), ill-defined ROI metrics (32%) and murky corporate AI strategy (31%). Not one of them is “the model isn’t good enough.” And according to the full Foundry report, the expertise gap is deepest in healthcare (52%), retail (51%) and manufacturing (49%), the sectors whose core work looks least like a software development lifecycle. That’s consistent with substrate being the real variable, though a tighter market for AI talent in those industries is surely part of the story too.

The market is already voting

Look at where the AI is actually being pointed, and you’ll see enterprises sequencing by substrate even though nobody’s calling it that. Three-quarters of both IT leaders and line-of-business respondents say AI is primarily being used to automate internal processes rather than customer-facing applications.

That’s not timidity. It’s instinct pointing at the right thing. Internal processes are the ones with structured, observable workflows and users who tolerate a little friction. Customer-facing work is where the trust gaps are still wide open and the cost of a wrong answer is asymmetric. A bad internal draft gets fixed before anyone sees it. A bad customer answer is the whole ballgame.

I’ll be honest about a wrinkle in the data here, because a careful reader will catch it. The same study reports a near-mirror finding, that 66% to 69% of respondents say the bulk of their current AI work is customer-facing. The two stats sit a paragraph apart in the CIO study and almost certainly reflect how the question was framed rather than a real reversal. But the synthesis holds either way: even where customer-facing work is being attempted, it’s where ROI is least realized. The work that lands is the work with the substrate underneath it. The split only reinforces the point.

Sequence by readiness, not by ambition

So, here’s the prescription, and it cuts against the instinct most AI strategies are built on. Stop sequencing your AI portfolio by where the value looks biggest. Start sequencing it by where the work already has, or can be given, a structured workflow with a usable signal for whether the output was right.

The study itself shows what the alternative looks like. Andrea Ballinger, CIO at Rensselaer Polytechnic Institute, described the trap precisely. No one measures ROI on an ongoing basis, she said, “because we are facing counterpressures from every vice president and line-of-business domain looking to implement AI for their own optimization.” The result: “We are saying yes to everyone without stepping back and focusing on the business cases that show real value.” That’s value-led sequencing under pressure from every budget-holder in the building, and it’s exactly how you end up with a sprawling pipeline of pilots and a 19% success rate.

The counterexample comes from the same study. Thomas Prommer, a longtime CTO, CIO and CAIO, funds outcomes instead of deliverables. “We don’t fund ‘build a model,’ we fund ‘reduce returns by 8% on this category’ with checkpoints at 90, 180 and 270 days,” he explained. He kills any project that misses two checkpoints, “roughly a third of what we start, and that’s healthy.” Read that through the substrate lens and you see what he’s really doing. He’s manufacturing a correctness signal where the work didn’t come with one. He’s building the missing piece of scaffolding by hand.

That gives you a simple lens to run any candidate use case through. Does the work break into discernible stages? Can you observe what happens at each one? Is there a usable signal for whether the result was right? Score high on all three and you have a software-engineering-shaped problem, so go now. Score low and you have a choice: build the substrate first or wait. What you shouldn’t do is fund it at scale and hope the ROI materializes, because that’s the pile the 19% number is built on.

The hard part, and the honest caveat

Run the professions through that lens and they sort themselves. Finance is the closest cousin to software. Reconciliation, close processes, approval chains and audit trails already give you staged work with a clear “it reconciles or it doesn’t” signal, which is part of why financial services sits among the sectors furthest along with AI. Legal and medicine are harder. The workflow shell exists, intake to redline to filing, diagnosis to treatment to follow-up, but the correctness signal at the core is weak, delayed or confounded. You can automate the routine staged parts and you hit a wall at the judgment that defines the profession.

And that’s the caveat that keeps this honest. A structured workflow isn’t always buildable in software’s image. For the judgment core of some professions, the substrate is years out no matter how good the model gets or how mature your governance becomes. Anyone selling you a tighter timeline than that is selling.

But notice what this reframe does. It turns “our AI ROI is elusive” from a mystery you wait out into a sequencing-and-instrumentation problem you can actually test. Your timeline isn’t set by how smart the next model is. It’s set by how fast you build the substrate for your own domain, and that’s within your control.

The two numbers, reconciled

Put the 80% and the 19% back next to each other and they stop looking like a paradox. Software engineering didn’t win because its models were better than everyone else’s. It won because the work was already shaped to let an agent succeed, and the scaffolding that makes agent output trustworthy had been in place for decades before the agent showed up.

The question for the rest of the enterprise was never really whether AI can do the work. It’s whether your work is shaped so AI’s output can be trusted. That’s not something you wait for. It’s something you build.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

AI is accelerating modernization projects that previously required months of analysis. But in highly regulated organizations, an uncomfortable reality quickly emerges: The risk is no longer in converting the code, but in demonstrating that the new version still does exactly what the old one did.

Almost every management committee has made the same decision this year: to apply artificial intelligence to their systems. And almost all discover the same thing when they delve into the details: AI is easy to add to the periphery — a chatbot, a copilot, a dashboard — and very difficult to integrate where it really matters, which is the legacy core. In banking, insurance, and much of the public sector, that core is still COBOL on a mainframe, with decades of patches and documentation that, to put it mildly, is incomplete.

That’s precisely where the regulatory risk lies. And that’s where most projects go off the rails.

I’ve spent three decades in regulated sectors, and the pattern repeats itself. The IT team approaches modernization as a delivery problem — deliver quickly, close tickets, move to production — when in a regulated sector, the problem is compliance. Success isn’t measured by what you deliver, but by what you can defend. Changing that mindset is half the battle.

The mirage of COBOL translated

The promise is enticing. Today, a language model can read thousands of lines of COBOL, document them, explain them, and propose an equivalent in Java or Python in a fraction of the time it would take a human team. It works. I’ve seen it accelerate analyses that previously took weeks.

The problem isn’t the code. The problem is the business rules that no one ever wrote down. In a migration project in a highly regulated banking environment, the biggest risk wasn’t in the routines, but in a calculation exception that had been running for 15 years and wasn’t documented anywhere: It existed only in the code and in the mind of a now-retired analyst. When you ask a model to “translate” that, it doesn’t translate; it fills in the gap with what statistically seems correct. And it does so with impeccable certainty.

On a dashboard, a hallucination is a troublesome error. In a financial institution’s calculation engine, it’s a compliance incident, a customer complaint, and potentially a penalty from the regulator.

The temptation, precisely because the tool is so fast, is to skip the slow part: reconstructing that logic with someone who understands it. That’s the worst possible decision. The speed of AI is seductive precisely at the point where making a mistake is most costly.

What the regulator expects — and what changed in May

DORA has been in effect since January 2025 and is very clear: operational resilience, ICT asset management, business continuity, and third-party risk control. Modernizing the core addresses all four areas simultaneously. NIS2 adds the security and notification layer. And the AI ​​Regulation introduces its own framework when the system you deploy is high-risk.

Although DORA, NIS2, and the EU AI ​​Regulation pursue different objectives, they share a common requirement: the ability to demonstrate control, traceability, and accountability over deployed systems. This is the link between the three frameworks, and it’s what a modernization project must protect from day one.

It’s important to clear up a recent misunderstanding here. With the Digital Omnibus agreement of May 2026, the high-risk obligations of Annex III are postponed until December 2027. Many executives have interpreted the headline — “EU delays AI Law” — as a reprieve. This is a dangerous interpretation. Transparency obligations still apply in August 2026, synthetic content marking comes into effect in December 2026, and, most importantly, the underlying risk remains unchanged. An erroneous automated decision in 2026 still falls under the GDPR, under sector-specific regulations, and under the jurisdiction of the relevant supervisor. The deadline has been moved; the responsibility has not.

How to do it without triggering the risk

I don’t have a magic formula, but I do have five principles that I apply to every project of this type:

1. Inventory before modernizing. You can’t secure or migrate what hasn’t been mapped. Assets, dependencies, data flows: If that map doesn’t exist, the first deliverable of the project is to build it, not write code.
2. The AI ​​proposes, a person validates. The model accelerates the analysis and the first draft of the transformation. The critical business rule is confirmed by an engineer who understands the business, not the model. Where there is no one who understands it, it is reconstructed with the business area before anything is changed.
3. End-to-end traceability. Every AI-generated transformation must be recorded: what went in, what went out, who approved it, and why. That’s not bureaucracy; it’s exactly what the auditor will ask for, and it’s what makes a change defensible.
4. Be careful where you put the code. Dumping kernel source code into an external model is a data transfer and a confidentiality issue more than a technical one. DORA requires control over the third party; GDPR requires control over the data. This decision is made at the beginning of the project, not after it’s finished.
5. Govern shadow AI. If the organization doesn’t offer a safe way to use AI, teams will use it anyway, on their own and without oversight. Governance isn’t about prohibition but about providing an enabled path.

Technological leadership has changed

In a regulated sector, the CIO’s challenge is no longer simply to modernize legacy systems, but to do so in a way that withstands the scrutiny of auditors, regulators, and risk committees.

Leading one of these projects is no longer about coordinating deliveries; it’s about making the transformation defensible. It means saying no to a shortcut that would save two weeks but leave a gap without traceability. It means treating governance as an accelerator — because a well-documented change is approved faster — and not a brake.

AI is an extraordinary tool for organizations to overcome their legacy technical debt. But in banking, insurance, or public administration, uncontrolled speed is not an advantage: It’s a liability that surfaces at first inspection. Modernizing quickly can be a competitive advantage; modernizing with traceability, control, and defense capabilities is what makes it sustainable.

Therefore, I summarize what I’ve learned over the years as the following: A secure solution isn’t the slowest or the most expensive; it’s the one that survives the first audit.

José Enrique Ibarra is Interim CIO and AI Project Manager, with three decades of experience leading IT in regulated sectors—banking, insurance, energy, and public administration. His focus is on governing digital transformation and AI adoption under the AI ​​Regulation, DORA, NIS2, GDPR, ISO 27001, and the Spanish National Security Framework (ENS), ensuring it withstands the scrutiny of auditors and regulators. He leads AI Forge, his applied AI initiative. He resides in Almería.

Most software composition analysis tools read what developers declare. Insignary Clarity’s patented binary-first platform analyzes what is actually built, shipped, and deployed — including the open-source components that never appear in any manifest.

Insignary, Inc., whose patented binary fingerprint technology has been cited in four Gartner research reports, today announced its recognition as a Sample Vendor for Reachability Analysis in the Gartner Hype Cycle for Secure Software Engineering, 2026.

According to Gartner: “Open-source and third-party components may contain a long list of vulnerabilities, but not all of them directly impact your code base. Reachability analysis helps in triaging the vulnerabilities based on their exploitability.”*1

The urgency is clear across independent industry research. A 2024 Venafi survey of 800 security decision-makers across the U.S., U.K., Germany, and France found that 92% are concerned about AI-generated code, and 63% have considered banning it outright over security risk.*2 The U.S. National Vulnerability Database recorded more than 48,000 CVEs in 2025 — roughly 130 every day.

AI coding assistants are accelerating the growth of unmanaged open-source dependencies. As organizations adopt these tools at scale, they face a widening challenge: understanding which open-source components enter production software, whether those components can be trusted, and how the resulting security and compliance risks are managed.

The problem is structural. Most SCA tools read what developers declare — not what actually runs. AI-generated code, vendor libraries, and third-party binaries frequently bypass package managers and never appear in a manifest.

“SBOMs are increasingly becoming a regulatory requirement around the world. However, software transparency is only as reliable as the accuracy of an SBOM itself. You cannot verify an SBOM by reading the manifest that created it. You verify an SBOM by examining the software that was actually built, shipped, and deployed. As software supply-chain regulations increasingly depend on SBOMs, the ability to validate software at the binary level becomes essential for organizations operating in regulated industries, critical infrastructure, and AI-enabled software environments.” — Taek Wan Kim, President & CEO, Insignary

INSIGNARY CLARITY: BINARY-FIRST. AI-AWARE.

Insignary Clarity scans both source and binary to build a complete Software Bill of Materials (SBOM) for the applications teams build, the third-party components they incorporate, and the IT infrastructure that bypasses the traditional secure development lifecycle.

Key capabilities include:

• Binary SCA — identifies open-source components, vulnerabilities, and license obligations directly from compiled binaries, without requiring source code or package manifests
• AIBOM Generation — produces an AI Bill of Materials for software containing AI-generated or AI-assisted code, covering components that bypass traditional dependency declarations
• Reachability Analysis — determines which disclosed vulnerabilities actually reach executable code paths, enabling risk-based prioritization rather than raw CVE-count triage
• Continuous Vulnerability Alerting — monitors stored SBOMs against updated vulnerability databases and delivers automated alerts when newly disclosed CVEs match deployed components, without requiring a rescan

“An SBOM is foundational to managing the complexity and securability of modern software deployments.”*3

RECOGNITION IN FOUR GARTNER REPORTS

Insignary has been cited in four Gartner research reports*, Gartner Hype Cycle for Secure Software Engineering,2026, Gartner Hype Cycle for Application Security,2025, Gartner Scale Application Security With AI-Augmented Vulnerability Remediation,2025, and Gartner 3 Steps for Assessing an Open-Source Software Project, 2025.

SUPPORTING GLOBAL SOFTWARE SUPPLY CHAIN REQUIREMENTS

Insignary Clarity supports organisations meeting software supply-chain security requirements across North America and globally:

• U.S. Executive Order 14028 and OMB Memorandum M-26-05 — federal agencies may now independently verify vendor SBOMs rather than accepting a standard attestation form, raising the bar for all software sold into the U.S. government
• FDA Section 524B — every connected medical device premarket submission must include a binary-verified SBOM covering all compiled software components
• Canada’s Bill C-8 Critical Cyber Systems Protection Act (CCSPA), effective June 2026 — mandatory supply chain risk management for banking, telecommunications, energy, and transportation operators

Additional frameworks: CISA and NSA SBOM guidance, NIST SSDF, Australia’s Information Security Manual (ISM), U.S. Connected Vehicle Rule, EU Cyber Resilience Act.

TRUSTED BY GOVERNMENTS AND GLOBAL ENTERPRISES

Globally, BearingPoint — one of Europe’s leading management and technology consulting firms and a strategic investor in Insignary — serves as the company’s exclusive distributor across Europe. Cybertrust Japan, another strategic investor, and its reselling partner TechMatrix drive adoption across Japanese manufacturing under a joint SBOM initiative. Customers include government organizations and global leaders across the electronics, defense, financial services, automotive, manufacturing, medical, and other technology sectors.

GARTNER and Hype Cycle are trademarks of Gartner, Inc. and/or its affiliates. Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.

ABOUT INSIGNARY

Insignary Inc. is a Toronto-based cybersecurity company specializing in binary composition analysis and software supply chain security. Its patented technology enables organizations to identify open-source software components, vulnerabilities, and software provenance directly from compiled binaries without requiring access to source code.

The company’s flagship platform, Insignary Clarity, provides binary analysis and software composition analysis capabilities that enable organizations to verify the contents of deployed software and strengthen software supply chain governance. Insignary Clarity AIR extends this capability to the AI domain by helping organizations identify, assess, and manage risks associated with AI models, AI-generated software, and AI-driven development environments.

The company serves enterprises, governments, and software vendors worldwide and is supported by strategic investors and partners including BearingPoint in Europe, Cybertrust Japan and TechMatrix in Japan, and TMA Solutions. Through its global partner ecosystem, Insignary supports software supply chain security initiatives across North America, Europe, and Asia.

Website: https://insignary.com/

Full report: Gartner Hype Cycle for Secure Software Engineering, 2026

References:

1. Gartner, Hype Cycle for Secure Software Engineering 2026
2. Venafi, “Machine Identity Management Development Survey,” 2024
3. Gartner, “Emerging Tech: A Software Bill of Materials Is Critical to Software Supply Chain Management.”

Contact

Principal Solutions Architect

Jessica DY Lee

Insignary

jessicalee@insignary.com

Most executives approach AI cautiously, like people trying to stretch one bankroll across an entire night on the casino floor. They spread their chips, hedge every move, celebrate the occasional small win, and tell themselves they played wisely because they didn’t lose much. It feels disciplined and responsible. But in a market shifting this quickly, it can become managed irrelevance.

The real question is how to win so decisively that the house starts treating you differently, says Nimesh Mehta. As EVP and chief information and strategy officer at National Life Group, one of the nation’s largest life insurers, Mehta sees AI less as a controlled technology program and more as a sequence of bets that reveal how leadership thinks about risk, conviction, and competitive advantage. In his view, the organizations that pull ahead are the ones willing to place smarter, more consequential bets that reshape how the business operates, competes, and grows.

Stop insuring every move

Mehta’s framing exposes a trap many leadership teams still don’t recognize. A surprising number of organizations believe they’re being strategic with AI when they’re actually over-insuring every decision. They hedge every initiative, over-govern every experiment, over-analyze every downside, and under-commit to anything that might actually matter.

As a result, they reduce the upside to the point of insignificance. Mehta argues that many companies insure the bet so thoroughly that even when they win, the upside is negligible and moves are designed to eliminate discomfort, not win.

That’s what makes the casino analogy useful. In a casino, sitting on your chips can feel safe. In business, that same instinct creates drift. Markets move, competitors learn, and capabilities compound whether you’re ready or not. “Sometimes not betting can, in itself, become a losing strategy,” says Mehta, which strikes the heart of the moment facing CIOs. Waiting may feel like discipline in the short term, but over time it becomes a tax on relevance. The organizations creating separation are those that understand uncertainty, contain it, and move anyway.

Trade certainty for probability

The deeper leadership shift Mehta is calling for is about changing the mental model from deterministic thinking to probabilistic thinking. Most enterprises still want AI to behave like a traditional technology investment. If we spend X, we should get Y. If the model isn’t fully accurate, it’s not ready, and if meaningful risk remains, deployment should wait. That logic made sense in a world built on structured systems and predictable workflows. But it becomes far less useful in a world where advantage comes from learning faster than rivals and improving in motion.

Most organizations still treat AI decisions like fixed equations rather than probabilistic plays, says Mehta. That distinction matters because it changes the questions leaders ask. A deterministic mindset looks at a customer service use case and asks whether the model is accurate enough to be trusted in every case. A probabilistic mindset asks whether the model can handle enough interactions well enough to free human judgment for the moments that matter most. A deterministic leader wants guaranteed ROI before scaling copilots. A probabilistic leader sees that modest improvements, repeated across marketing, operations, distribution, and service, can produce a compounding advantage long before any single use case looks perfect on paper. In that sense, imperfection may be leverage.

Nobody goes to a casino expecting certainty. The energy comes from reading the table, understanding the odds, and knowing that context changes the right move. AI operates in much the same way. “Leaders don’t need to become reckless gamblers,” says Mehta, “They need to become more fluent in uncertainty, more comfortable making directional decisions, and more disciplined about learning from each hand they play.”

When you’re holding 17

Mehta’s blackjack metaphor captures the exact moment where many enterprises stall. With a 17, the dealer will ask if you want to hit or stand. It’s paralyzing, but it’s the position many CIOs are in with AI today. The use case works as it produces value, but it isn’t perfect. The organization can see the upside, but it can still see the gaps, too. So many leaders choose to stand. They wait for better accuracy, cleaner governance, stronger confidence, or a more complete business case. They tell themselves that holding is the responsible move.

“The real risk, however, isn’t the hit but standing still while the table evolves around you,” says Mehta. That line reframes what prudence means in an AI environment. Hitting means understanding the odds well enough to act, not being careless. It means asking what the downside of moving forward actually is, what guardrails are needed, and what the cost of waiting might be if others are learning faster.

The strongest leaders understand that a good but not perfect hand can still be the right time to lean in. They know that learning in production, within bounds, often reveals more than another quarter spent polishing the slide deck.

Bet where the upside compounds

How should CIOs decide where to place bigger bets? Mehta’s answer is to look for asymmetry, where the upside is meaningfully larger than the downside, where learning compounds even if the first outcome is imperfect, and where speed creates an advantage competitors can’t easily copy. Those are better to address than asking which use cases feel safest. Mehta also emphasizes proximity to the business, arguing that the closer an AI use case sits to revenue, customer experience, or core operations, the more valuable the learning becomes. That’s why some of the most important bets may initially feel uncomfortable. But they matter because they produce insight that isolated pilots never can.

This isn’t a call for indiscriminate betting, though. Discipline still matters, and not every initiative deserves to scale, and not every model belongs in production. Good leaders know when to leave the table, reallocate chips, and avoid the sunk-cost logic that keeps weaker organizations trapped in bad hands. At National Life, that means thinking about AI as a portfolio of bets — some exploratory ones, some scaled with intent, and all assessed not only on immediate return but on how they expand capabilities. Positioning the enterprise for its next move is an important lesson for CIOs. “The goal isn’t to win every hand, but build a system to place better bets over time,” says Mehta.

Lessons for CIOs playing to win

For CIOs, the takeaway is clear. Stop defining success as the absence of loss and define it as the presence of disproportionate upside. Shift your decision model from deterministic certainty to probabilistic advantage. Build a portfolio of bets tied closely to the business with enough governance to manage risk, but not so much that governance becomes an excuse for inaction.

Know when to press on emerging momentum and walk away from bets that don’t compound value. Most of all, recognize that AI leadership is about demonstrating you can read the table, understand the odds, and move with conviction before the rest of the market catches up, not just proving you can keep the lights on while experimenting at the edges.

A cyber risk assessment helps security teams identify, estimate, and prioritize potential threats and vulnerabilities to key enterprise digital and physical assets. Yet, despite its importance, many CISOs fall victim to several types of “gotchas” that prevent them from fully achieving their risk assessment goals.

An assessment should be an essential part of every organization’s overall cybersecurity strategy. The process helps security leaders understand risks to business objectives, evaluate the likelihood and impact of cyberattacks, and develop ways to mitigate the risks they uncover.

Here are the top seven mistakes security leaders should avoid to ensure risk assessment effectiveness.

1. Going through the motions

The biggest “gotcha” is treating cyber risk assessments as a preset checklist or control inventory instead of a decision tool tied to real business impact and threat scenarios, says Shirsendu Mondal, a cybersecurity researcher at the University of North Carolina.

“When assessments become all about checking boxes, they lose the ability to reflect how risk actually shows up in an environment,” he states. “The goal should be to inform decisions about where a business is truly exposed.”

Mondal assers that the best way to avoid the complacency trap is to take a context-driven approach. “Ask where the asset is, who can reach it, what data it touches, how important it is to operations, and what happens if it goes down,” he explains. “Risk should always be tied to business impact, not only technical findings.”

Mondal also recommends adding internal business leaders to security teams, including individuals in areas such as IT and operations, given that risk is more than a technical issue.

2. Sugarcoating results

These are challenging times, so we must be honest with our stakeholders, says Pablo Riboldi, CISO at BairesDev, a nearshore software development firm.

“When results are discouraging, admit that the threat landscape has evolved much faster than the previous evaluation framework anticipated,” he says.

Instead of just handing over lists of vulnerabilities, you need to start presenting actual attack scenarios, Riboldi adds. “For example, by prioritizing the top three most critical business assets and conducting an in-depth assessment on them, you can show immediate value.”

3. Falling short on the scope of your assessments

CISOs often securitize document controls, check compliance boxes, and produce a risk register that claims everything looks absolutely fine, says Denis Calderone, CTO at cybersecurity services firm Suzu Labs. Yet nobody bothered to test whether those controls actually work or stopped to ask whether the scope of the assessment covered what really matters.

We see it all the time, Calderone says. “For instance, the assessment covers the production servers and the corporate network, but skips the old dev box in the corner, the third-party vendor portal nobody owns internally, or the API endpoint that was stood up for a project two years ago and never decommissioned.” Attackers don’t care about your scoping decisions, he says. “They look at the whole environment and find the thing you decided wasn’t worth assessing.”

AI is making the situation worse, Calderone says. Organizations are deploying AI tools, connecting them to internal systems, granting them access to sensitive data, and none of this is landing in the risk assessment. Meanwhile, AI agents are out there making API calls, accessing databases, and operating with credentials that nobody is tracking, he says.

“If your risk assessment was written before your organization started plugging AI into its workflows, it’s already stale,” Calderone warns.

4. Overindexing on the risk register without checking your assumptions

When the goal becomes completing the assessment instead of understanding actual exposure, the output is a document that satisfies auditors but misleads leadership, says Amit Basu, CIO and CISO at International Seaways, a major independent maritime shipping company that transports crude oil and refined petroleum products worldwide.

Such an attitude can create false confidence. Executives and board members see a completed risk register and assume the organization is protected, Basu says. Meanwhile, real threats go unaddressed because they didn’t fit neatly into the assessment framework. “The gotcha does not announce itself,” he explains. “It hides inside a green dashboard.”

A risk assessment is only as good as the assumptions that lie underneath it, Basu observes. “Document those assumptions explicitly and review them whenever your business changes, when the threat landscape shifts, or when an incident exposes a gap,” he advises. “The assessment is not a finished product — it’s a living input to an ongoing conversation between security and the business.”

5. Failing to link risk with business impact

Ignoring or downplaying the connection between risk and business makes it easier to de-prioritize or ignore problems, says Dan Moore, senior director of strategy and identity standards at FusionAuth, a customer identity and access management (CIAM) platform provider.

“As a result, it becomes difficult to communicate the real risks of breaches and other risks,” he states. “Worse yet, it gives security team members an excuse to complain about being misunderstood or not valued, which degrades team effectiveness.”

It’s important to be specific and targeted, Moore advises. “For instance, don’t say, ‘We have 95% patch compliance,’” he suggests. “Instead, talk about the risk unpatched systems pose to the business.” Some systems, such as legacy systems that aren’t connected to the internet or the core business, carry a lower risk than others, even if they have the same patch issues. “Acknowledge that fact and weigh your response.”

6. Confusing compliance with real-world security

Compliance alone doesn’t lead to good security, nor does it satisfy even the baseline requirements for effective protection, says Adriel Desautels, CEO of Netragard, a penetration testing and security advisory company.

Organizations tend to fall into this trap when they hire penetration testing firms that focus on compliance while promising top-tier services, Desautels says. “In truth, they deliver autonomous scanning masquerading as human-driven testing.”

The result is a false sense of security — a paper seatbelt, Desautels warns. “You feel protected, but when you crash, even at low speed, you get injured or worse,” he says. “Remember, every major breach in the past decade involved an organization that was compliant at the time of compromise.”

7. Failing to fully understand risk

Organizations often treat risk assessment as a vulnerability-cataloging exercise that includes finding gaps, counting severities, and passing the audit. Yet passing an audit and understanding risk are not the same thing, states Safi Raza, senior director of cyber security at Fusion Risk Management, a firm offering cloud-based operational resilience, business continuity, and risk management solutions.

Raza says that CISOs should focus on connecting technical risk signals to operational outcomes. “This includes understanding what services are affected, how disruption propagates, and what it means for revenue, customers, or regulatory obligations.”

Start by shifting from static assessments to continuous, context-driven risk visibility, Raza advises. “Risk needs to be understood not just technically, but in terms of business impact and financial exposure,” he states.

The European Union’s drive towards some form of digital sovereignty has just received a boost with the first four companies ready to support the EU cloud sovereignty project.

Four cloud service providers — Etix, Phocea, Thésée Datacenter, and Gigas — have signed up for the CISPE Sovereign and Resilient Cloud Service Certification program. Currently, they are all at the self-certification stage and have submitted their services for an independent audit to assess whether they meet the required criteria.

The EU has been pressing hard for a strong European presence in the cloud market. Public bodies are increasingly fearful about exposure of their data to US providers. The US Cloud Act (Clarifying Lawful Overseas Use of Data), for example, permits the US government to access a range of data held by cloud operators, even if that data is held outside the US. This legislation conflicts with the EU GDPR Act and has prompted the call for more digital sovereignty across Europe.

“Public bodies, hospitals and industrial operators are today seeking concrete guarantees of digital sovereignty. The CISPE Sovereignty Badge provides that guarantee. It is a natural complement to European standards such as Gaia-X Level 3, strengthening transparency, compliance and digital trust. It is this ability to provide concrete proof, beyond rhetoric, that underpins genuine European digital autonomy.” said Antoine Fournier, CEO of Thésée Datacenter

The EU is keen to guard against ‘sovereignty washing’ — claims by foreign-owned cloud providers that they meet local control criteria. Last month, CISPE warned about Broadcom’s claim it complied with EU conditions. It probably won’t be the last to make such claims.

The European Union’s drive towards some form of digital sovereignty has just received a boost with the first four companies ready to support the EU cloud sovereignty project.

Four cloud service providers — Etix, Phocea, Thésée Datacenter, and Gigas — have signed up for the CISPE Sovereign and Resilient Cloud Service Certification program. Currently, they are all at the self-certification stage and have submitted their services for an independent audit to assess whether they meet the required criteria.

The EU has been pressing hard for a strong European presence in the cloud market. Public bodies are increasingly fearful about exposure of their data to US providers. The US Cloud Act (Clarifying Lawful Overseas Use of Data), for example, permits the US government to access a range of data held by cloud operators, even if that data is held outside the US. This legislation conflicts with the EU GDPR Act and has prompted the call for more digital sovereignty across Europe.

“Public bodies, hospitals and industrial operators are today seeking concrete guarantees of digital sovereignty. The CISPE Sovereignty Badge provides that guarantee. It is a natural complement to European standards such as Gaia-X Level 3, strengthening transparency, compliance and digital trust. It is this ability to provide concrete proof, beyond rhetoric, that underpins genuine European digital autonomy.” said Antoine Fournier, CEO of Thésée Datacenter

The EU is keen to guard against ‘sovereignty washing’ — claims by foreign-owned cloud providers that they meet local control criteria. Last month, CISPE warned about Broadcom’s claim it complied with EU conditions. It probably won’t be the last to make such claims.

This article first appeared on Network World.

A newly disclosed vulnerability in Argo CD is drawing attention to the security risks of GitOps platforms, with researchers warning that the flaw could allow attackers who gain a foothold inside a Kubernetes cluster to execute code and manipulate application deployments.

Security firm Synacktiv said in a report that the flaw affects Argo CD’s repo-server component, which fetches content from Git repositories and generates Kubernetes manifests used to deploy resources in a cluster. Argo CD is one of the most popular Kubernetes tools and is based on the GitOps paradigm.

“Argo CD requires significant privileges within the cluster,” Synacktiv said. “Additionally, it has access to private Git repositories, making it an attractive target for attackers.”

The issue centers on the repo-server’s unauthenticated GenerateManifest gRPC endpoint. Synacktiv said an attacker able to reach that endpoint could supply Kustomize options in a manifest generation request and abuse Kustomize’s Helm-related build options to execute attacker-controlled commands.

Exploitation requires access to both the repo-server gRPC port and the Redis database port, which should not be exposed to users. Argo CD provides Kubernetes network policies designed to prevent that scenario, but those protections are not enabled by default in Helm chart deployments, according to Synacktiv.

In such deployments, compromising a single pod inside the cluster could be enough to give an attacker the internal access needed to exploit the vulnerability.

Synacktiv said it was able to use the flaw to obtain the Redis password from the repo-server environment and access Argo CD’s Redis database. The researchers then manipulated cached deployment data, allowing a malicious manifest to be deployed automatically when Argo CD’s Auto Sync feature was enabled.

If Auto Sync is not enabled, exploitation would require a user to manually sync the application.

Synacktiv publicly disclosed the details on July 1 after first reporting the issue to Argo CD maintainers in January 2025. The vulnerability remains unpatched, and the firm recommended strict Kubernetes network policies to block untrusted pods from reaching the repo-server and Redis services until a fix is available.

Assessing internal cluster exposure

For CISOs, the key question is not only whether Argo CD is exposed to the internet, but whether other workloads inside the Kubernetes cluster can reach its internal services.

“Because the repo-server’s gRPC service does not enforce authentication, any pod that can reach it becomes equivalent to an authenticated attacker,” said Devashri Datta, a cybersecurity researcher. “In a typical cluster, that means any compromised application pod, misconfigured service mesh, or adjacent workload with local code execution can directly query the GenerateManifest endpoint or hit the Redis cache, no internet exposure required.”

Organizations should not equate “not internet-facing” with “low risk,” because modern attacks often begin with the compromise of an internal workload, according to Sakshi Grover, senior research manager for cybersecurity services research at IDC Asia/Pacific.

“CISOs should therefore evaluate which workloads can communicate with the Argo CD control plane, whether east-west traffic is appropriately segmented, and whether unnecessary trust relationships exist between application workloads and GitOps infrastructure,” Grover said. “The assessment should focus on attack paths rather than perimeter exposure.”

Treating GitOps as tier-zero

The flaw also underscores the role GitOps platforms play in controlling software deployment across enterprise infrastructure.

“GitOps engines aren’t utility services; they’re tier-0 control-plane components,” Datta said. “By design, Argo CD holds read access to private repositories, sync/write access to target clusters, and custody of deployment secrets. It sits at the precise intersection of source code, configuration management, and live infrastructure.”

That level of access means an Argo CD compromise may extend beyond a single application. An attacker could turn the platform used to deploy applications into a channel for malicious manifests, while also interfering with auto-sync behavior and extracting credentials cached in supporting systems such as Redis.

A compromise of these platforms could influence software delivery at scale, making them strategic assets that should be subject to stricter governance and privileged access controls similar to those applied to identity platforms and other critical management systems.

The article originally appeared on CSO.

A newly disclosed vulnerability in Argo CD is drawing attention to the security risks of GitOps platforms, with researchers warning that the flaw could allow attackers who gain a foothold inside a Kubernetes cluster to execute code and manipulate application deployments.

Security firm Synacktiv said in a report that the flaw affects Argo CD’s repo-server component, which fetches content from Git repositories and generates Kubernetes manifests used to deploy resources in a cluster. Argo CD is one of the most popular Kubernetes tools and is based on the GitOps paradigm.

“Argo CD requires significant privileges within the cluster,” Synacktiv said. “Additionally, it has access to private Git repositories, making it an attractive target for attackers.”

The issue centers on the repo-server’s unauthenticated GenerateManifest gRPC endpoint. Synacktiv said an attacker able to reach that endpoint could supply Kustomize options in a manifest generation request and abuse Kustomize’s Helm-related build options to execute attacker-controlled commands.

Exploitation requires access to both the repo-server gRPC port and the Redis database port, which should not be exposed to users. Argo CD provides Kubernetes network policies designed to prevent that scenario, but those protections are not enabled by default in Helm chart deployments, according to Synacktiv.

In such deployments, compromising a single pod inside the cluster could be enough to give an attacker the internal access needed to exploit the vulnerability.

Synacktiv said it was able to use the flaw to obtain the Redis password from the repo-server environment and access Argo CD’s Redis database. The researchers then manipulated cached deployment data, allowing a malicious manifest to be deployed automatically when Argo CD’s Auto Sync feature was enabled.

If Auto Sync is not enabled, exploitation would require a user to manually sync the application.

Synacktiv publicly disclosed the details on July 1, 2026, after first reporting the issue to Argo CD maintainers in January 2025. The vulnerability remains unpatched, and the firm recommended strict Kubernetes network policies to block untrusted pods from reaching the repo-server and Redis services until a fix is available.

Assessing internal cluster exposure

For CISOs, the key question is not only whether Argo CD is exposed to the internet, but whether other workloads inside the Kubernetes cluster can reach its internal services.

“Because the repo-server’s gRPC service does not enforce authentication, any pod that can reach it becomes equivalent to an authenticated attacker,” said Devashri Datta, a cybersecurity researcher. “In a typical cluster, that means any compromised application pod, misconfigured service mesh, or adjacent workload with local code execution can directly query the GenerateManifest endpoint or hit the Redis cache, no internet exposure required.”

Organizations should not equate “not internet-facing” with “low risk,” because modern attacks often begin with the compromise of an internal workload, according to Sakshi Grover, senior research manager for cybersecurity services research at IDC Asia/Pacific.

“CISOs should therefore evaluate which workloads can communicate with the Argo CD control plane, whether east-west traffic is appropriately segmented, and whether unnecessary trust relationships exist between application workloads and GitOps infrastructure,” Grover said. “The assessment should focus on attack paths rather than perimeter exposure.”

Treating GitOps as tier-zero

The flaw also underscores the role GitOps platforms play in controlling software deployment across enterprise infrastructure.

“GitOps engines aren’t utility services; they’re tier-0 control-plane components,” Datta said. “By design, Argo CD holds read access to private repositories, sync/write access to target clusters, and custody of deployment secrets. It sits at the precise intersection of source code, configuration management, and live infrastructure.”

That level of access means an Argo CD compromise may extend beyond a single application. An attacker could turn the platform used to deploy applications into a channel for malicious manifests, while also interfering with auto-sync behavior and extracting credentials cached in supporting systems such as Redis.

A compromise of these platforms could influence software delivery at scale, making them strategic assets that should be subject to stricter governance and privileged access controls similar to those applied to identity platforms and other critical management systems.

In the wake of AI, the early-talent job market is in decline across all jobs and industries, with a 10% drop since 2021, according to a recent report from SAP. When isolated for the top 10 most common entry level job titles, including software engineer, customer support, and data analyst, job openings declined 35% from 2024 to 2025.

AI is already impacting entry-level and task-based roles, leaving the question of what will happen to the future talent pipeline of IT leadership.

“Our research suggests that organizations create a gap in their future leadership pipeline if they continue reducing entry-level hiring without rethinking how early-career talent develops,” says Autumn Krauss, chief scientist for Future of Work Research Lab at SAP SuccessFactors. “Historically, people built business acumen, technical expertise, and leadership skills through the first few years of their careers. As AI becomes embedded in workflows, organizations need to be intentional about creating new ways for employees to build those skills.”

Companies have traditionally followed hierarchal processes that encourage employees to advance up the career ladder via promotions, with many taking a leadership path to IT director, CIO, or CTO. However, this process has always relied on workers starting with entry-level and routine work to create a natural first step for future leaders. But now, more companies are automating the work that entry-level and mid-level employees used to cut their teeth on, says Maruf Ahmed, CEO of IT staffing and consulting company Dexian.

For example, a junior engineer might start in QA and testing on the technical side, getting hands-on experience with how systems work. As they progress in their career, that knowledge continues to stack, creating future leaders who deeply understand the technology and the business.

“Maintaining a strong succession path starts with being honest about what AI removed from someone’s development, and then being intentional about replacing it,” says Ahmed. “Senior leaders need to spend more time actively teaching, and people need exposure to complex decisions earlier in their careers. Day-to-day work used to build that foundation on its own, and it doesn’t anymore.”

Redesigning jobs for AI and leading in uncertainty

As many have discovered, it’s not always easy to decipher quality AI outputs, especially as the technology has become renowned for hallucinating results. Current and future IT leaders need the foundational knowledge to have confidence when evaluating AI outputs, especially if they’re expecting employees to use AI. The most valuable leaders are the ones comfortable making decisions with incomplete information, and who can make calls on the spot about automated processes, no matter what may arise.

“We often see AI doesn’t stay contained in one function for long,” Ahmed adds. “Once an organization automates in one area, there’s an expectation to extend that more broadly, and leaders who built their careers inside a specific function are suddenly being asked to weigh in on AI use in areas they’ve never directly managed.”

According to recent research from Deloitte, 84% of companies haven’t done the work to redesign jobs around AI despite high expectations for automation, and 36% expect at least 10% of their jobs to be fully automated within a year, and 82% say within three years. Even with those results, fewer than half are making significant adjustments to talent strategies, with 53% saying they’re simply focusing on educating employees to raise AI fluency.

That leaves entry-level workers and those in task-aligned roles in somewhat unknown territory as automation replaces time-consuming tasks, and managers shift to overseeing human-AI teams. Deloitte points to a potential shift toward flatter structures, with more than half of businesses considering pod-based or non-hierarchal models, while 16% have already started to make a shift.

IT leaders will likely find themselves relying more on others in the company to make judgment calls around AI, opening communication and transparency as job roles evolve and structures begin to flatten. With AI adoption happening at a rapid pace, organizations need to evaluate how it’ll impact talent and leadership structures, and what the future of leadership will look like with automation.

Investing in early talent to maintain a leadership pathway

IT leaders need to avoid the trap of treating AI adoption as a technology rollout rather than a workforce development project, says Ahmed. Leaders will continue to invest heavily in new tools, services, hardware, and technology, and then expect employees to become self-taught on these platforms in their downtime.

“Our research found that leaders don’t often feel confident or equipped with the skills to lead that level of transformation. Instead, organizations often fall back on the tactic of giving employees AI tools and expecting them to figure out how best to use them on their own,” says Krauss.

If current IT leaders feel unequipped to lead through AI transformation, it’s imperative companies step back and reevaluate training and future leadership talent pools. Investments in AI should be weighed alongside the necessary investments for employee upskilling and training, and to redefine long-standing organizational structures.

Investment in early talent development programs can also be beneficial for tackling potential future leadership gaps left by AI adoption, with 86% of employees saying that an early talent program helped set them up for career success, according to the SAP report. But despite the beneficial nature of such programs, only 32% of early talent report participating in an early talent program, and 49% say their organization doesn’t offer one. Plus, only 35% of early talent employees say they’ve been given sufficient transparency into which roles in their organization may be automated in the future, while one in three express concerns their job may one day cease to exist due to AI advancements.

“As roles change, organizations need to give employees a clearer picture of where opportunities are emerging and what skills will be most important,” says Krauss. “Many employees are already uncertain about how technology will affect their careers, and that uncertainty can make it harder to stay engaged. People are more likely to invest in their growth when they see a path forward. That visibility is becoming increasingly important.”

The PC was heralded as revolutionary; it was going to save time, revolutionize our work… But it became an opportunity lost. Paper became digital files. Filing cabinets became shared drives. Memos became email. We sometimes worked faster. We did not necessarily work differently. And we certainly did not work more efficiently. The underlying logic: approval chains, reporting cycles, hierarchies and incentives remained intact.

The internet and smartphones followed the same pattern, compressing time and distance. But neither forced enterprise changes. The tools changed. The organizational model did not. This stagnation is referred to as the Solow Productivity Paradox, a historic mismatch between massive technology investments and flat corporate productivity. And while the Internet boom did see a raise in productivity, it was due to investment in hardware, not so much due to a change in how we worked, as explained by Robert Gordon in The New Economy: What Productivity Miracle?

And now there’s Artificial Intelligence, AI. AI presents a different kind of challenge because it intervenes in cognition itself. It reaches much closer to the operating logic of the enterprise than previous technology waves.

Yet, once again, the response is surface adaptation rather than structural reinvention. AI is layered onto inherited workflows, old approval thresholds, unclear accountability structures and sprawling software, then expecting cost savings to follow. And again, it is the investments in AI that garner any growth, not changes in corporate infrastructure.

This is not transformation. It is acceleration without reform. And this “slap on AI” will have as much long-term impact as the PC.

Automation = efficiency? Wrong

Chief information officers are under intense pressure to turn AI into measurable financial outcomes. In boardrooms, expectations are explicit: deploy AI, automate, reduce operating cost and show results within a budget cycle.

A central misunderstanding in AI programs is the assumption that if a process is costly and labour-intensive, automation creates efficiency. Unfortunately, what appears as inefficiencies are normalized fragmentations. With AI, hidden workflow contradictions become both significant and visible. Organizations discover it wasn’t running a slow but clean process. It was running an incoherent process that relied on human buffering to keep it functioning. This is precisely why so many AI efforts disappoint immediately after a dazzling pilot, degenerate into AI strategy theatre and fail to scale.

When one part of the workflow becomes lightning-fast, but the surrounding process remains fractured, escalations multiply and the IT department, despite having done its job perfectly, is asked to fix the operational fallout with more tooling, more integration, more controls and more spend. The problem is rarely technical. But it becomes so very quickly.

I increasingly think the more useful concept here is process debt. CIOs are already comfortable talking about technical debt and its complexities. Process debt is the upstream generator of that complexity. It accumulates when temporary fixes become permanent, when controls are added without removing older ones and when incidents leave behind workflows nobody dares to challenge. Over time, the process stops reflecting deliberate design and starts reflecting institutional memory, risk aversion and unresolved negotiations between functions.

Case study 1: The regulated approval-heavy process

I was brought into a regulated organization that wanted to identify opportunities for automation. The assumption was that technology was the main constraint. Workflows involved multiple reviews, approvals and handovers between departments. From a distance, it looked like an obvious candidate for automation.

It was a familiar situation: delays, duplicated effort and frustration with how long routine work was taking to complete. The process seemed overstaffed and underdesigned. The natural conclusion was that automation could remove unnecessary tasks and improve speed.

But as I began interviewing the stakeholders, a different picture emerged. Every group could explain its role in the workflow. But the more I listened, the clearer it became that nobody could describe the process as a coherent whole.

What appeared to be an inefficient process was a process that had accumulated layers of governance without ever being reassembled into a consistent operating model. One approval had been added after an audit finding. Another had been introduced during a restructuring. A third existed because of a past incident. None of those approvals looked unreasonable in isolation. Together, they produced a workflow that nobody owned and with approval layers nobody could justify.

From the CIO’s perspective, this translated into technology sprawl. Unaligned and multiple IT systems were being used to support adjacent parts of the workflow. Software had been purchased to manage steps that shouldn’t have existed in the first place. This meant the entire nature of the automation discussion shifted. The strategic question was no longer which step should be automated first. It was whether those steps deserved to exist at all.

Only after the CIO and I brought the business units together to confront these structural dependencies did the process align and technology become part of the answer. Without that preliminary work, automation would simply have moved a poorly understood process faster while expanding the expensive software estate needed to govern it.

That engagement reinforced my conviction that many workflows presented as automation candidates are not ready for automation because they are not sufficiently coherent to automate.

Uncomfortable questions

Because these questions are operationally and politically sensitive, businesses will try to avoid them and hand the unmapped mess directly to IT. As a strategic partner, the CIO must guide the C-suite through these uncomfortable but necessary inquiries before deploying AI into enterprise workflows:

• Does this process need to exist at all?
• Where are decisions actually made in day-to-day practice? Not according to policy documentation, but according to the informal networks of people who actually know how to navigate the exceptions.
• Which parts of the workflow exist because of corporate history rather than necessity?
• Where do decision rights shift between teams without anyone acknowledging it?

AI systems do not handle ambiguities gracefully. Answering these questions upfront determines whether implementing AI will mean genuine savings or simply move organizational incoherence through the enterprise at lightning speed.

Three-layer governance

Governance is the ultimate reason why AI cannot be treated as a traditional technology delivery program with a bit of business input tacked on at the end. Because AI fundamentally alters how enterprise decisions are informed and executed, its deployment must be shaped by an integrated operating and governance framework.

CIOs can evaluate an organization’s true AI readiness based on three interdependent governance layers. By mastering these, the technology stack is protected from being forced to compensate for bad business design.

1. Organizational governance. Core questions: Is this workflow genuinely needed, who actually owns it and what risk or quality definitions are binding across separate business functions? This is a cross-departmental leadership question. It must be resolved by the business units first, or IT inevitably inherits the resulting operational complexity.
2. Endpoint or tool governance. Core questions: How are outputs interpreted when cognitive work is partially or fully delegated to machines? This layer defines exactly where a human-in-the-loop remains mandatory, how exceptions are escalated to specialists and how accountability is maintained when an AI agent makes an optimized operational prediction.
3. Platform governance. Core questions: What is the foundational security, privacy and technical guardrails? This includes LLM/model selection, vendor standards, data privacy compliance, integration rules and continuous monitoring. Paradoxically, this is the layer most organizations focus on first—yet, because it exists entirely to support the processes and tools above it, it should actually be the last to be set in stone.

These layers interlinked. A weakness in one undermines the others. This is where CIOs become strategically important. They are the executives who see when process incoherence is converted into architectural complexity, application sprawl, higher license costs and long-term support burdens. AI decisions without that perspective and organizations will once again confuse digitization with transformation.

Case Study 2: A downstream bottleneck and the structural solution

In another engagement, the business pushed for an AI-driven intake solution. Frontline teams were spending massive amounts of time on repetitive customer data coordination and document verification. On paper, it was an outright victory: IT delivered an AI agent that dropped data extraction times by 85% with an exceptional accuracy rate. In every sense of the word – the pilot was a triumph. And it was phased to implementation.

Within six weeks, the illusion shattered. While the intake layer was now running at lightning speed, the downstream validation process still relied on traditional compliance handoffs, manual fraud checks and legacy database updates. The AI didn’t solve the operational problem; it simply shoved massive volumes of data into a rigid pipeline that was never designed for that velocity.

Escalation queues exploded. The operations team, buried under an unprecedented backlog, began making manual bypass decisions just to keep up, creating immense operational risk.

Rather than allowing IT to be blamed for the downstream chaos, the CIO and I suggested a structural solution. First, we halted further automated intake scaling and used visual process-mapping data to show the rest of the C-suite exactly where the digital pipeline was hitting an analogue wall.

Second, we championed a cross-functional “value stream” redesign. After much negotiation, we managed to leverage the AI’s data-validation outputs to eliminate three manual review steps downstream and replace them with exception-only automated alerts. Finally, we renegotiated the risk-threshold parameters with the legal and compliance teams, shifting accountability from a multi-stage sign-off to a centralized, systemic audit log.

The solution wasn’t adding more software; it was picking the process apart, data point by data point, to align the business rules with machine capabilities. The result was a substantially trimmer, automated end-to-end stream that freed up human resources, allowed redundant software licenses to be safely withdrawn and actually realized the promised financial savings.

Conclusion

Every CIO knows that AI matters. The real challenge facing enterprises whether the executive leadership team is willing to confront what AI inevitably reveals about the fragmented processes, legacy habits and siloed systems organizations have been carrying for decades.

This is why the broad promise of immediate AI savings is overstated. Automation can produce staggering enterprise value, but it cannot create structural coherence on its own. If a workflow is fractured, historically layered and dependent on invisible human intervention, adding AI will not turn it into an efficient system. It will simply scale, cement and automate the weaknesses that were already there.

The CIO’s ultimate responsibility is to ensure that technically incoherent processes do not get permanent residency in the business architecture. This is why the CIO must have a leading seat at the strategic table. Incoherent processes invariably turn into application sprawl, redundant tooling, excess licensing costs, integration debt and massive security exposure.

To avoid this trap, enterprise AI deployment requires cross-departmental leadership willing to examine which work should be automated, which must be completely redesigned and which should be eliminated. 

If not, we will simply repeat the costly errors of past technology shifts: preserve the outdated operating logic, throw a shiny new layer of tooling on top and call the expensive result “transformation.” This time, the bill will be significantly larger. Not because AI is mysterious, but because it is brutally efficient at exposing the waste that organizations used to hide inside their people, their processes and their software.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Operational autonomy is quickly becoming one of the defining capabilities of a modern enterprise. As digital estates become more distributed, cloud environments more dynamic and AI consumption more expensive and less predictable, traditional operating models begin to show their limits. Teams can no longer rely only on manual oversight, disconnected monitoring tools or periodic financial reviews to keep enterprise technology healthy and cost efficient. What is needed instead is a coordinated operating framework that brings together CloudOps, FinOps and AIOps, while also addressing the emerging discipline of AI token and model consumption governance. When these disciplines are designed as one connected system rather than as isolated workstreams, organizations move closer to operational excellence: faster decisions, better resilience, improved financial control, stronger compliance and a more measurable connection between technology investments and business outcomes.

What operational autonomy means in enterprise IT

Operational autonomy does not mean removing people from operations. In practice, it means designing enterprise IT so that routine sensing, decision support, remediation, optimization and policy enforcement happen with minimal friction and with the right human oversight at the right moments. A mature autonomous operating model continuously observes infrastructure, applications, data flows, AI services and financial consumption patterns; detects risk or inefficiency early; and triggers guided or automated action based on policy, confidence and business criticality. This approach depends on four connected pillars: CloudOps to maintain reliable and scalable digital infrastructure, FinOps to govern cost and value, AIOps to detect patterns and automate response, and AI consumption governance to manage token usage, model selection, inference workloads and unit economics.

Gartner’s 2024 research on FinOps for data and analytics emphasizes that cloud operations and financial governance are no longer separate concerns, especially as AI workloads reshape cost structures and accountability expectations. Forrester’s 2024 analysis of AIOps and observability similarly notes that modern enterprises need deeper operational visibility and broader insight-driven coordination to handle hybrid complexity. IDC’s 2024 perspective on future operations adds another useful lens by framing data-driven operations around agility, resilience and predictability. Taken together, these viewpoints reinforce the same idea: autonomy is not a tool purchase; it is a management framework.

Design principles for an enterprise operational autonomy framework

A practical framework begins with a few disciplined principles. First, the enterprise must build around a shared operational data layer. Telemetry from cloud infrastructure, applications, service management systems, security controls, business transactions and AI services should be normalized so that operations, finance and governance teams work from the same facts. Second, every automated action should be policy-aware. Cost optimization, scaling, failover, remediation, model routing, data retention and access control should all reflect business guardrails rather than isolated technical rules.

Third, the framework should be value-led rather than purely cost-led. FinOps has matured beyond simply lowering spend; the stronger objective is to align spend with business priorities, performance requirements and acceptable risk. Fourth, autonomy should progress in stages. Enterprises usually start with visibility, then introduce recommendations, then guided automation and finally closed-loop autonomy for low-risk scenarios. Fifth, executive accountability must be explicit. Operational autonomy touches architecture, finance, privacy, security, data stewardship and business strategy. Without a cross-functional ownership model, autonomy becomes fragmented and difficult to govern. Everest Group’s 2024 FinOps Cloud Cost Management assessment highlights the growing demand for role-based access, cost intelligence, governance and automation as core requirements for enterprise cloud cost management products. That is a useful signal that the framework must be built for collaboration, not just analytics.

Integrating CloudOps, FinOps and AIOps into one operating model

CloudOps, FinOps and AIOps are often discussed separately because each emerged from a different operational problem. CloudOps grew out of the need to run cloud estates reliably and at scale. FinOps developed in response to unpredictable consumption-based billing. AIOps emerged because traditional monitoring could not keep pace with the volume and complexity of telemetry generated across modern digital systems. Yet in a mature enterprise, these disciplines converge naturally.

A performance incident in a cloud platform is rarely only an availability problem; it may also drive higher infrastructure consumption, trigger excess logging charges, degrade customer experience or increase token usage in AI-enabled workflows. Similarly, a cost spike may not be a finance issue alone; it may reveal inefficient architecture, poor scheduling, unnecessary data movement or an AI agent behaving outside policy.

An integrated operating model therefore links observability signals, service context, business KPIs, financial metrics and automation rules into one decision fabric. CloudOps provides the runtime discipline, FinOps introduces value and accountability, and AIOps adds pattern recognition and intelligent response. When connected well, the enterprise can answer not only what is happening, but why it is happening, what it is costing, what risk it creates and what the best next action should be.

AI token optimization and AI cost spend governance

AI introduces a new cost curve into enterprise operations. Unlike traditional software costs, token spend can vary sharply based on prompt design, model choice, context length, retrieval patterns, orchestration logic, concurrency, caching strategy and user behavior. This makes AI cost governance an essential part of operational autonomy. A strong framework begins by defining the unit economics of AI consumption: cost per request, cost per conversation, cost per business workflow, cost per user segment and cost per outcome.

Once these baselines are visible, the enterprise can introduce optimization controls such as prompt compression, response-length policies, semantic caching, model tiering, workload routing to lower-cost models where quality tolerance allows, context-window discipline, batch processing for non-real-time use cases and approval thresholds for premium model usage. AI gateways and model brokers can enforce these policies consistently across teams.

Chargeback or showback mechanisms should also extend to AI services so that business units see both value and consumption behavior. Recent analysis in Forbes has drawn attention to the financial risks of unmanaged token growth and argues for governance layers that connect finance and engineering before AI expenditure becomes opaque. FinOps Foundation guidance on FinOps for AI reinforces the same message, noting that token-level metrics, quotas, tagging, GPU allocation practices and real-time monitoring are necessary to keep AI costs aligned to business value. In enterprise settings, the lesson is straightforward: if cloud cost needed FinOps, AI cost needs an even tighter form of FinOps because usage can scale much faster and become far less transparent as mentioned in IDC report.

FinOps for cloud infrastructure cost management

Cloud infrastructure cost management remains one of the foundational layers of operational autonomy because every autonomous workflow eventually rests on compute, storage, networking, platform services and data transfer. An effective FinOps capability does more than flag overspend after the month has ended. It creates near-real-time visibility into consumption, ownership, unit economics, forecast variance, commitments and waste patterns.

The enterprise should define standard practices for tagging, cost allocation, commitment management, rightsizing, idle resource detection, storage tiering, Kubernetes cost visibility, environment lifecycle controls and architecture reviews for high-cost services. More importantly, these practices should be tied to business context. For example, a workload serving a mission-critical customer channel may justify higher spend if it supports revenue protection, whereas a non-production environment should have stricter shutdown and spend caps.

Gartner’s 2024 research on FinOps for data and analytics underscores that AI and data workloads are changing the financial profile of cloud operations and increasing the need for more sophisticated tooling and governance. IDC’s market perspective on intelligent cloud and edge operations with FinOps software also points to the rapid growth of platforms that combine operations intelligence with financial control, suggesting that enterprises increasingly view operational management and cost management as linked disciplines rather than separate layers.

Autonomous operations through AIOps

AIOps gives the framework its intelligence and response speed. In most enterprises, operations data is noisy, fragmented and too voluminous for humans to interpret quickly during incidents or performance degradation. AIOps platforms reduce that burden by correlating events, identifying anomalies, clustering symptoms, surfacing probable root causes and recommending or initiating remediation actions. The best outcomes appear when AIOps is connected not only to infrastructure monitoring but also to service maps, change records, configuration data, incident workflows and business priorities.

That connection allows the enterprise to distinguish between a harmless signal fluctuation and an issue that threatens a critical business service. Forrester’s 2024 research on AIOps and observability explains this well by describing the complementary value of breadth and depth: observability provides richer technical insight, while AIOps helps transform those signals into operational action. In practice, autonomy grows when low-risk responses such as service restarts, resource adjustments, ticket enrichment, dependency checks or rollback decisions are automated under policy. High-risk actions should remain human-approved until confidence improves. Over time, the enterprise can move from reactive incident management to predictive operations, where emerging capacity risk, recurring error patterns or unusual AI workload behavior are addressed before service impact is visible to users.

How the framework leads to operational excellence

Operational excellence is the cumulative result of better decisions made earlier, faster and with clearer accountability. A well-designed autonomy framework improves service reliability because systems are observed continuously and remediation can be triggered before failures spread. It improves cost discipline because consumption anomalies are identified at the same time as performance or usage anomalies, not weeks later in a billing report.

It improves strategic focus because technology leaders can evaluate trade-offs in terms of business value rather than technical activity alone. It also improves employee productivity by removing repetitive operational effort and shifting skilled staff toward engineering improvements, policy tuning and service innovation. The most important outcome, however, is predictability. Enterprises become more confident in how they scale AI services, how they control cloud spend, how they handle operational events and how they meet compliance obligations. That confidence is what separates routine automation from genuine operational autonomy.

Security, governance, process implementation and people upskilling

No autonomy framework survives without strong security and governance. Automated operations amplify both efficiency and risk, which means identity controls, segmentation, least-privilege access, secrets management, encryption and auditability have to be embedded from the start. AI services add further concerns: prompt leakage, data residency, model misuse, training-data exposure, shadow AI adoption and uncontrolled access to external models.

Governance therefore needs to extend across cloud resources, operational workflows, AI services and data assets. Enterprises should establish clear policy domains covering infrastructure provisioning, AI model approval, token limits, vendor usage, observability data handling, retention rules, access reviews and exception management. Process implementation is equally important. The framework should define standard operating patterns for incident triage, automated remediation approval, cost anomaly review, model lifecycle management and post-incident learning. None of this works unless people are prepared for the shift.

Operations teams need skills in cloud economics, observability, automation engineering and policy-driven operations. Finance teams need to understand cloud and AI consumption models. Security and privacy teams need fluency in AI risk scenarios and control design. Business leaders need a clearer grasp of unit economics and value realization. IDC’s 2024 briefing on enterprise AI strategy highlights the tension between rapid AI investment, ROI pressure, staffing constraints, security and compliance. That is exactly why upskilling must be treated as part of the framework itself, not as an optional change-management activity as per FinOps Foundation documentation.

The role of regulatory compliance

Regulatory compliance is not a side topic in operational autonomy; it is one of the main reasons the framework must be formalized. Cloud environments frequently span jurisdictions, AI systems process sensitive information, observability platforms collect detailed operational data and automated decisions may influence customer experience or internal controls. Regulations such as GDPR, DPDP, sector-specific cybersecurity directives, financial reporting obligations, contractual data-handling requirements and internal audit standards all shape what autonomy can and cannot do.

Compliance requirements should therefore be translated into operational policy. Examples include residency-aware workload placement, data minimization in logs and prompts, access segregation for financial and regulated data, explainable automated actions, evidence retention, periodic control attestations and approval workflows for AI usage involving personal or confidential information. Chief privacy and data leaders play a central role here because the compliance question is no longer just where data is stored, but also how data is observed, transformed and consumed by AI-driven services. A mature framework reduces compliance risk by making control enforcement systematic rather than dependent on manual effort.

How to implement the framework in practice

Implementation is usually most successful when handled in phases. The first phase is baseline visibility: consolidate telemetry, cloud billing data, service inventory, AI usage data and business ownership into one operational picture. The second phase is governance design: define policies for tagging, spend thresholds, automation boundaries, access controls, model usage and compliance checkpoints.

The third phase is prioritization: choose a small number of use cases where autonomy can produce measurable value, such as cloud rightsizing, incident correlation, cost anomaly detection, AI token governance or automated remediation for recurring low-risk faults. The fourth phase is automation with guardrails: deploy workflows, approval rules and rollback paths. The fifth phase is optimization and learning: review outcomes, refine policies, update unit economics, expand autonomy coverage and measure business impact.

This staged approach matters because full autonomy is not achieved by switching on one platform. It is built progressively through trusted control, good data and disciplined execution.

Useful tools for building the framework

The tool landscape should be chosen based on architecture, governance maturity and operating model rather than vendor popularity alone. Cloud-native cost and operations tools from hyperscalers provide baseline visibility, but many enterprises supplement them with specialized FinOps platforms for allocation, forecasting, commitment analysis and chargeback. Observability platforms help unify metrics, logs, traces and service maps, while AIOps platforms add anomaly detection, event correlation and automation orchestration.

Service management platforms remain important for change control, incident workflows and audit evidence. AI gateways and model management layers are increasingly useful for token monitoring, policy enforcement, prompt controls, model routing and usage analytics. Security posture management, DSPM, identity governance and compliance automation tools also become part of the architecture because autonomy without trust quickly becomes fragile. The most effective toolchains are the ones that integrate technical telemetry, financial signals, governance policy and workflow automation into a coherent operating system for the enterprise.

Executive roles in developing and managing the framework

Here is a table that summarizes various Executive Roles and their responsibilities in Operational Autonomy governance.

Conclusion

Developing operational autonomy for an enterprise is not about chasing a futuristic ideal. It is about building a disciplined and connected operating model that helps the organization run technology with greater confidence, speed and accountability. CloudOps keeps the estate reliable, FinOps ensures that spending reflects value, AIOps makes complexity manageable and AI cost governance brings much-needed control to token-driven consumption. Security, privacy, compliance, process rigor and people capability are what make the framework sustainable. When all of these parts work together, the enterprise does not just automate tasks; it strengthens resilience, improves financial stewardship and creates a more adaptive path to operational excellence.

This article was made possible by our partnership with the IASA Chief Architect Forum. The CAF’s purpose is to test, challenge and support the art and science of Business Technology Architecture and its evolution over time as well as grow the influence and leadership of chief architects both inside and outside the profession. The CAF is a leadership community of the IASA, the leading non-profit professional association for business technology architects.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Italy’s competition watchdog has opened an investigation into Microsoft over concerns it may not have clearly informed consumers about the integration of Copilot and Designer into Microsoft 365 subscriptions, associated price increases, and automatic upgrades to higher-cost plans.

The Italian Competition Authority (AGCM), in a statement to the press, said it had opened an investigation into Microsoft S.r.l., the vendor’s Italian subsidiary, and Microsoft Ireland Operations Ltd. to assess whether the way the changes to M365 pricing were communicated may have unduly restricted consumers’ freedom of choice.

Although the AGCM’s announcement does not explicitly identify the pricing event under investigation, its description aligns with Microsoft’s January 2025 rollout of Copilot and Designer for Microsoft 365 Personal and Family subscribers.

At the time, Microsoft announced its first price increase for the consumer subscriptions since the launch of Copilot and Designer, stating that existing subscribers would pay the higher price at their next renewal after the AI features were added.

Could scrutiny spread beyond Italy?

The Italian probe is not the first time Microsoft’s communication around AI-related Microsoft 365 pricing has drawn regulatory attention.

The investigation follows earlier scrutiny of Microsoft’s consumer pricing communications in Australia and New Zealand, where Microsoft apologized and revised some of its messaging after regulators raised concerns over how AI-enabled Microsoft 365 subscriptions were presented to customers.

That episode could prove relevant for the current investigation, even though the legal frameworks differ, said Pareekh Jain, principal analyst at Pareekh Consulting.

“Microsoft’s apology and revised communications show similar concerns were raised before,” the analyst said.

“While the legal cases differ, Italian regulators may see it as evidence that clearer customer communication was already known to be necessary,” he added.

Jain also expects regulators elsewhere to watch the outcome of the Italian investigation: “Regulators in the EU, UK, Australia, New Zealand, and Canada are likely to watch closely, especially where AI is bundled into existing subscriptions with higher prices or automatic renewals.”

Enterprise buyers are likely to scrutinize AI pricing more closely

Whether the Italian investigation ultimately results in penalties against Microsoft remains to be seen, but analysts say the probe could have an impact on M365 commercial plans, which are separate and set to take effect on July 1.

The investigation serves as a reminder for enterprises to examine AI-related price changes even if Microsoft’s commercial licensing process seems more transparent than its consumer subscription model, Jain said. “CIOs should still verify what is changing at renewal and whether AI features are optional.”

Enterprise procurement teams should ask tougher questions during negotiations, said Bhupendra Chopra, chief revenue officer at IT consulting firm Kanerika.

“Procurement teams should ask questions, such as what AI am I paying for, can I see it itemized, and can I decline it without losing the rest? Buyers would be right to want AI pricing written into renewal terms with clear opt-outs and price protection,” Chopra said.

These questions, according to Jain, could provide additional leverage during licensing discussions to negotiate flexible terms.

However, Chopra pointed out that the investigation is symptomatic of a broader tension between software vendors’ efforts to integrate AI into existing products and regulators’ expectations around transparency and customer choice.

“Building AI into existing products and pricing it in is becoming standard across software, not unique to one vendor. Regulators are testing an old question against a new feature — whether buyers were given clear information and a real choice,” the analyst said.

For enterprises, the analyst added, the practical takeaway is straightforward: “Expect AI to show up inside the tools you already own, expect it to carry a cost, and review what each renewal actually includes. Treated as routine cost discipline, it stops being a surprise.”

Microsoft did not respond to a request for comment. Computerworld has also reached out to the Italian Competition Authority for additional details about the scope of the investigation.

Before Tobi Lütke ran Shopify, he learned programming through Germany’s apprenticeship system⁠, the way people have learned trades forever: in a shared workshop, watching people who already knew what they were doing. More recently, describing Shopify’s River, he reached for a related word: Lehrwerkstatt⁠, a teaching workshop where “the whole shop floor is the classroom.”

X has been agog by the numbers around River⁠, Shopify’s Slack-native AI agent. In total, 5,938 Shopify employees worked with River across 4,450 different Slack channels, and River now coauthors roughly one in eight merged pull requests across the company. It’s a big deal, but understanding why it works that way is the most important part.

River can read code, run tests, open pull requests, query the data warehouse, inspect production traces, and sometimes push back on a plan it thinks is bad. Great. Lots of companies will have clever coding agents someday soon. Some already do.

The interesting part is that River doesn’t work alone; it works where everyone can see it.

Betting on the workshop

I’ve already argued that agents reward explicit, consistent, well-documented software. They like the “boring” stuff, such as schemas, tests, conventions, clean setup instructions, and codebases that don’t require a deep retrospective with the one engineer who remembers why the build script has to run twice. Dropping an agent into a messy repo is mostly an efficient audit of your engineering discipline. Agents hold up a mirror to our engineering practices.

This is where Shopify comes off looking good. Without all the engineering pre-work, River wouldn’t be a success. In early 2024⁠, the company says it had many repositories, bespoke development environments, and slow feedback loops. It then made two unpopular but critically important choices: moved to a monorepo called World and built dev environments, continuous integration, and production images on Nix as one reproducible substrate.

Shopify recognized that “code is going to be increasingly written with AI, and our infrastructure needs to be the substrate for that.” But the company did more than insist on legible code: It started to create shared memory of that code across the company.

Collective coding

River has one design constraint that every enterprise architect should pay attention to: It only works in public Slack channels. No direct messages. No private groups. You summon River where other people can watch, join, search, and learn. That sounds like a small product choice, but it’s not. It’s the operating model, kind of like open sourcing code development within Slack.

Because of this design constraint, every River session becomes a visible transcript. Shopify can then mine those transcripts, see recurring patterns, and feed them back into River’s skills, prompts, and defaults. One engineer’s hard-won fix at two o’clock becomes the next engineer’s starting point at four o’clock. The model doesn’t need to be retrained for the company to get smarter, and developers don’t need to go out of their way to document things. The work just has to leave a trace.

That’s the Lehrwerkstatt, productized. Everyone gets to watch the agent work.

Now compare that with how most enterprises are deploying AI. One developer works with a private chatbot in a private IDE in a private window that no one else will ever see. Multiply that by a few thousand. Each person discovers a clever way to investigate a flaky test, explain a troublesome service boundary, or avoid a migration trap. Then the session closes, and the discovery dies. Sure, the developer may go faster, but the company is no better off than it was yesterday.

The transcript is the artifact

One mistake enterprises have made with knowledge management is treating documentation as something people write after the work. This rarely works. Few employees (developers or otherwise) want to undertake the tedium of documenting what they already did. Not unless someone is paying them to do it.

River suggests a better pattern: The work itself creates the documentation.

Not every transcript is useful, of course. Most probably aren’t. But the useful ones can become skills, defaults, examples, runbooks, repo instructions, or links that help the next person avoid starting from zero. Shopify says River sessions are searchable and reproducible, and the company feeds patterns from those sessions back into River’s skills, prompts, and defaults. That’s not a chatbot; it’s a learning loop.

This is where the usual “AI will make developers more productive” framing feels too small. The more interesting claim is that AI can make software organizations more teachable. However, this won’t happen by default. The shop floor needs to be institutionalized or the enterprise will remain an atomized collection of productivity silos.

A magic memory file

This is where agents.md⁠ is useful, but only if properly used. agents.md describes itself as a README for agents and says it’s now used by more than 60,000 open source projects. How should a developer use it? GitHub, based on analysis of more than 2,500 repositories⁠, gives some clear guidance: Put commands early, be specific, provide real examples, and set explicit boundaries.

In other words, write down what matters.

But don’t mistake the file for the capability. ETH Zurich researchers recently tested whether repository-level context files actually help coding agents⁠ and found that they often reduce task success while increasing inference cost by more than 20%. InfoQ summarized⁠ their finding this way: LLM-generated context files often hurt, and human-written ones should focus on non-inferable details, such as custom tools, unusual build commands, and highly specific project constraints.

That’s the enterprise opportunity.

Public GitHub projects often don’t have much non-inferable domain knowledge to encode, but enterprise software is filled with it: odd quirks such as why the pricing service can’t be called during checkout in a certain region, or which legacy API looks dead but still supports a major customer, or why the data model says one thing but revenue recognition says another. Etc., etc.

That’s the context worth preserving, rather than directory maps an agent can discover or generic coding preferences. That’s what the shop-floor version of agents.md looks like: Not a static file that someone auto-generates and forgets, but rather the residue of observed work. Agents struggle, humans correct, patterns emerge, and only the durable lessons become instructions.

You’re not Shopify

If all this sounds great (and it should), then it’s worth a word of warning: You probably won’t be able to copy Shopify, any more than you could have (or should have) copied Google. You’re not Shopify. Most companies shouldn’t wake up Monday and announce a monorepo migration, a Nix conversion, and a Slack-only agent because River sounds cool. That approach has worked for Shopify, but it doesn’t mean it will work for you.