The head of the US government’s AI testing institute, Chris Fall, has resigned about three months after taking charge of the Center for AI Standards and Innovation (CAISI), the federal organization responsible for evaluating advanced artificial intelligence models for safety and security.

Current National Institute of Standards and Technology NIST Director Arvind Raman will serve as acting CAISI Director following Fall’s departure while continuing to oversee the Commerce Department office responsible for the institute, the Daily Signal reported, citing two people familiar with the matter.

A Commerce Department spokesperson who spoke to the publication did not disclose a reason for the resignation.

Fall assumed leadership of CAISI in April after the Trump administration reorganized the former US AI Safety Institute under NIST. The institute develops methodologies for evaluating frontier AI models and works with AI developers on voluntary technical assessments covering areas such as cybersecurity, model misuse, reliability and other risks associated with increasingly capable AI systems.

The leadership change comes as governments and AI companies continue developing technical approaches for evaluating frontier AI models while enterprises expand deployments of generative AI and agentic AI across business operations.

In recent months, the Commerce Department has taken a more active role in AI policy involving advanced models, placing greater attention on how the federal government evaluates technologies with potential national security implications.

Continuity matters more than personalities

CAISI works with AI developers such as Anthropic, Google’s DeepMind and OpenAI on voluntary evaluations of frontier AI models and develops methodologies for testing model capabilities and risks. The institute does not regulate AI developers or certify commercial AI systems.

For enterprises, those evaluations are one source of technical information alongside vendors’ own testing, third-party security assessments and internal AI governance programs.

Sanchit Vir Gogia, chief analyst at Greyhound Research, said enterprises should focus less on the individual leading the institute and more on whether its technical work continues with the same level of consistency and transparency.

“Leadership churn at CAISI weakens the signal long before it weakens the science,” Gogia said. “The testing has not stopped. Its authority simply does not travel as cleanly once the leadership does not.”

According to Gogia, the more important question for enterprises is not whether the institute’s evaluation work will continue but whether the processes supporting those evaluations remain stable.

“The instinct is to ask whether the pipeline is breaking,” he said. “The more useful question is where the pipeline now sits.”

Enterprises still carry the burden of AI governance

Gogia said organizations should continue treating government-led AI evaluations as one input into their governance processes rather than as evidence that a model is inherently safe for enterprise deployment.

“A government evaluation was always a signal, never a certificate,” he said. “A signal loses value the moment its issuer becomes unpredictable.”

He said enterprises should instead monitor whether CAISI maintains consistent evaluation methodologies, continues publishing technical findings and preserves continuity within its research teams under interim leadership.

“The name on the door is not the signal. The behaviour underneath it is,” Gogia said.

Gogia also cautioned against linking Fall’s resignation to recent Commerce Department actions involving AI policy or export controls, noting that there is no public evidence connecting the two.

“CAISI evaluates; it does not enforce export controls, because it holds no such power,” he said. “This is not a testing body reaching for enforcement. It is enforcement reaching past the testing body.”

With Raman assuming the role on an interim basis, the next significant milestone for enterprises will be the appointment of a permanent director, and whether the institute’s evaluation programs continue without disruption, the analyst said.

Gogia said the successor’s mandate may prove more important than the individual selected.

“A CAISI result is not a safe harbour,” he said. “It informs an obligation; it does not discharge one.” NIST did not immediately respond to a request for comment.

OpenAI has launched Presence, an enterprise service for deploying voice and chat agents that can resolve customer and employee requests, potentially automating some work now handled by frontline support teams.

The agents can answer questions and operate IT systems, and enterprises can decide what actions the agents may take and when they should seek human approval for actions or transfer a case to a human.

OpenAI is already using Presence internally for its English-language phone support channel, where it verifies callers and uses account information to complete approved actions. The company said the system resolves 75% of inbound issues without human assistance.

Another OpenAI service, Codex, can be used to monitor agents and suggest updates or improvements to processes. In OpenAI’s own tests, suggestions from Codex helped reduce handoffs to humans by 15 percentage points over 10 days, it said. Presence also includes simulation and evaluation tools that allow companies to test an agent before deployment. The tests assess whether it reaches the correct outcome, follows company policy, and hands a case to an employee when required.

OpenAI intends each Presence deployment to deal with one kind of task, for example billing issues, insurance claims, or employee IT service requests, with agents getting only the knowledge and system access required for that task.

Presence is not a self-service product: Enterprises will have to sign up for the limited availability program, with integration performed by OpenAI or selected global systems integrators.

Companies exploring or testing Presence include Spanish bank BBVA, which is evaluating the service for everyday banking support in Mexico, and Japanese technology group SoftBank, which is using it in trials involving Japanese-language customer interactions. Australian insurer IAG is assessing whether the technology can help it respond to surges in customer demand during severe weather events.

Workforce impact

OpenAI’s announcement did not address the potential effect of Presence on employment. But its claimed automation rate raises questions about how the technology could affect staffing in customer service and other support functions.

Pareekh Jain, CEO of Pareekh Consulting, said CIOs should regard the 75% figure as evidence that the technology can work, rather than as a benchmark that every enterprise can expect to reach.

Jain said OpenAI’s deployment benefits from being built around the company’s own products and data. Large enterprises may achieve lower automation rates because they must contend with fragmented legacy systems, uneven knowledge bases and more complex compliance demands.

“Most organizations should expect lower initial automation levels that improve over time as the AI agent is refined,” Jain said.

The first workforce effect is more likely to be slower hiring than immediate layoffs, according to Tulika Sheel, senior vice president at Kadence International.

“The roles most exposed are likely to be repetitive, high-volume functions such as frontline customer support and routine back-office processing,” Sheel said. “However, I would expect the first impact to be on hiring and team growth rather than immediate large-scale job cuts. Over time, enterprises may redesign roles around AI-assisted workflows, with humans focusing more on complex cases, escalation, and relationship management.”

Jain said Tier-1 support agents handling predictable queries would face the most exposure. Broader reductions would become more likely only after companies reorganize their operations around the technology.

However, Lian Jye Su, chief analyst at Omdia, said Presence is unlikely to increase the threat of job displacement because companies have used similar customer-support automation from vendors such as Genesys, NiCE, Five9 and AWS for years.

Enterprises are more likely to use Presence alongside employees, with AI handling routine requests while people remain responsible for work requiring judgment and empathy, Su said.

Cost and operational risks

Analysts said CIOs should examine whether Presence can maintain resolution quality as usage grows, since fewer human handoffs could leave employees dealing with a more difficult mix of cases.

“The key question is not simply how many tasks AI can handle, but whether it can handle them reliably at scale,” Sheel said.

The financial case will depend partly on the cost of connecting Presence to existing systems and maintaining the controls needed to govern its use, according to Jain. “Often the biggest cost of enterprise AI is not tokens but integration and governance,” Jain added.

Companies will need to determine what systems and data the agents can access, monitor their performance, and audit the actions they take. Those investments could offset early savings.

Su said the complexity of enterprise IT will make it difficult for OpenAI to automate entire workflows on its own. Enterprises will still need to work with other technology providers and human employees, while CIOs will favor systems that can be audited and integrated with existing infrastructure.

Jain said the economics could improve if companies use the same integrations and governance controls across additional workflows.

This article first appeared on CIO.

Over the past few years, I have learned that basic prompts produce inconsistent, hallucination-prone results that no executive would trust in production. What turned the tide was my move to advanced prompting techniques. These weren’t theoretical experiments; they became a practical foundation for reliable, measurable outcomes. I want to share the techniques that consistently delivered the biggest gains in my projects, complete with real before-and-after examples, copy-paste templates, lessons from failures and guidance on when to evolve beyond prompting to agentic systems.

Why advanced prompting still matters in enterprise settings

Sophisticated prompting remains essential for control, reliability and compliance. If you “ask nicely” and hope for the best, you need deterministic behavior, auditable reasoning and minimal risk of hallucination. Here’s what worked for me.

1. Chain-of-Thought (CoT) and its variants: Unlocking step-by-step reasoning

The problem: Models would jump to conclusions on complex analysis tasks, especially involving data interpretation or multi-step logic.

What I did: I started explicitly instructing the model to “think step by step” and show its reasoning.

Before (basic prompt): “Analyze last quarter’s sales data and recommend three actions.”

After (CoT prompt):

“You’re a senior business analyst. Analyze the following sales data step by step: [data]. First, identify the key trends. Second, calculate the rates and anomalies. Third, link findings to business context. Finally, recommend the three prioritized actions with expected impact. Explain your reasoning at each step.”  

Results: Accuracy and depth improved dramatically.

Variants that worked well: Self-consistency. I ran the same CoT prompt multiple times and took the majority consensus. This reduced variability significantly.

Template you can use:

You are [expert role]. Solve this problem by thinking step by step.

[Task or question]

For each step:

1. State your observation or calculation.

2. Explain the implication.

3. Proceed only when confident.

Final answer in this format: [structured output]

2. Tree-of-Thoughts (ToT): Exploring multiple reasoning paths

For truly complex decisions such as resource allocation or risk assessment, linear CoT isn’t enough. Tree-of-Thoughts lets the model generate and evaluate multiple branches.

Example: I was helping a client evaluate three potential vendor platforms for an AI deployment. A standard prompt gave a superficial comparison. With ToT

Prompt Snippet:

Explore three different reasoning paths for selecting the best vendor platform:

Path 1: Focus on cost and scalability.

Path 2: Focus on security, compliance and integration.

Path 3: Focus on innovation and long-term roadmap.

For each path, evaluate pros/cons against our requirements [list].

Then, compare the paths and recommend the strongest overall option with justification.

Outcome: The model surfaced nuanced trade-offs (e.g., one vendor had superior security, but higher integration cost).

When to use: Strategic planning, troubleshooting or scenarios with high uncertainty and multiple viable approaches.

3. ReAct (Reason+ Act) and prompt chaining: Moving toward agentic behavior

One of the biggest leaps I have noticed comes from combining reasoning with tool use and chaining prompts.

ReAct example: (used in data analytics workflow)

You are an AI analyst with access to tools. For the query below:

1. Reason about what information you need.

2. Choose the appropriate tool or action.

3. Observe the result.

4. Repeat until you can answer confidently.

Query: [user request]

In practice, I chained this with retrieval tools. One automated quarterly compliance reporting; the system reasoned about required data, pulled relevant records, validated them, and generated the reports.

4. Meta-prompting and self-reflection: Letting the model improve itself

Use the model to refine its own prompt. This is a huge time-saver.

You are an expert prompt engineer. Improve the following prompt for clarity, structure and effectiveness with [target model]. Make it more precise while preserving intent.

Original prompt: [paste]

Provide the improved version and explain your changes.

Self-reflection loops (asking the model to critique its own output and revise) are a game-changer for content generation and code-review tasks.

5. Multimodal and structured output techniques

With vision-enabled models, I started combining text with images (e.g., uploading architecture diagrams or dashboards).

Tip from experience: Be extremely specific in describing what the models should focus on.

Best practices I learned the hard way

• Start simple, then layer complexity: Over-engineered prompts from Day One usually backfire.
• Model specific tuning: Some models respond better to XML delimiters; others to explicit reasoning.
• Evaluation and versioning: Treat prompts like code if you track versions and run automated evals.
• Security guardrails: Always include instructions against prompt injections and respect data boundaries.
• When to stop prompting: For repetitive, high-stakes workflows, move to full agents or an orchestration framework.

Final takeaways for technical leaders

Advanced prompt engineering has now become a core competency for anyone responsible for enterprise AI outcomes. Start by picking one technique and apply it rigorously to a real business problem. Document before/ after and you will notice why it’s worth mastering.

The field continues evolving towards more automated and agentic systems, but the ability to precisely direct AI reasoning remains foundational.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

OpenAI has launched Presence, an enterprise service for deploying voice and chat agents that can resolve customer and employee requests, potentially automating some work now handled by frontline support teams.

The agents can answer questions and operate IT systems, and enterprises can decide what actions the agents may take and when they should seek human approval for actions or transfer a case to a human.

OpenAI is already using Presence internally for its English-language phone support channel, where it verifies callers and uses account information to complete approved actions. The company said the system resolves 75% of inbound issues without human assistance.

Another OpenAI service, Codex, can be used to monitor agents and suggest updates or improvements to processes. In OpenAI’s own tests, suggestions from Codex helped reduce handoffs to humans by 15 percentage points over 10 days, it said. Presence also includes simulation and evaluation tools that allow companies to test an agent before deployment. The tests assess whether it reaches the correct outcome, follows company policy, and hands a case to an employee when required.

OpenAI intends each Presence deployment to deal with one kind of task, for example billing issues, insurance claims, or employee IT service requests, with agents getting only the knowledge and system access required for that task.

Presence is not a self-service product: Enterprises will have to sign up for the limited availability program, with integration performed by OpenAI or selected global systems integrators.

Companies exploring or testing Presence include Spanish bank BBVA, which is evaluating the service for everyday banking support in Mexico, and Japanese technology group SoftBank, which is using it in trials involving Japanese-language customer interactions. Australian insurer IAG is assessing whether the technology can help it respond to surges in customer demand during severe weather events.

Workforce impact

OpenAI’s announcement did not address the potential effect of Presence on employment. But its claimed automation rate raises questions about how the technology could affect staffing in customer service and other support functions.

Pareekh Jain, CEO of Pareekh Consulting, said CIOs should regard the 75% figure as evidence that the technology can work, rather than as a benchmark that every enterprise can expect to reach.

Jain said OpenAI’s deployment benefits from being built around the company’s own products and data. Large enterprises may achieve lower automation rates because they must contend with fragmented legacy systems, uneven knowledge bases and more complex compliance demands.

“Most organizations should expect lower initial automation levels that improve over time as the AI agent is refined,” Jain said.

The first workforce effect is more likely to be slower hiring than immediate layoffs, according to Tulika Sheel, senior vice president at Kadence International.

“The roles most exposed are likely to be repetitive, high-volume functions such as frontline customer support and routine back-office processing,” Sheel said. “However, I would expect the first impact to be on hiring and team growth rather than immediate large-scale job cuts. Over time, enterprises may redesign roles around AI-assisted workflows, with humans focusing more on complex cases, escalation, and relationship management.”

Jain said Tier-1 support agents handling predictable queries would face the most exposure. Broader reductions would become more likely only after companies reorganize their operations around the technology.

However, Lian Jye Su, chief analyst at Omdia, said Presence is unlikely to increase the threat of job displacement because companies have used similar customer-support automation from vendors such as Genesys, NiCE, Five9 and AWS for years.

Enterprises are more likely to use Presence alongside employees, with AI handling routine requests while people remain responsible for work requiring judgment and empathy, Su said.

Cost and operational risks

Analysts said CIOs should examine whether Presence can maintain resolution quality as usage grows, since fewer human handoffs could leave employees dealing with a more difficult mix of cases.

“The key question is not simply how many tasks AI can handle, but whether it can handle them reliably at scale,” Sheel said.

The financial case will depend partly on the cost of connecting Presence to existing systems and maintaining the controls needed to govern its use, according to Jain. “Often the biggest cost of enterprise AI is not tokens but integration and governance,” Jain added.

Companies will need to determine what systems and data the agents can access, monitor their performance, and audit the actions they take. Those investments could offset early savings.

Su said the complexity of enterprise IT will make it difficult for OpenAI to automate entire workflows on its own. Enterprises will still need to work with other technology providers and human employees, while CIOs will favor systems that can be audited and integrated with existing infrastructure.

Jain said the economics could improve if companies use the same integrations and governance controls across additional workflows.

Defense research agency DARPA made its largest quantum computing award ever this week, with a $125 million agreement announced on Wednesday. The same day, the White House announced an additional $5 billion for the Genesis Mission, which focuses on AI for science but also includes technology to accelerate quantum computing and quantum sensors.

“Taken together, these announcements signal that U.S. quantum strategy is shifting from supporting individual research projects to building the infrastructure needed for a quantum-enabled economy,” says Heather West, research manager in the infrastructure systems, platforms, and technology group at IDC.

None of the individual quantum announcements are surprising, she says. But the level of coordination is new. “Government investment is expanding beyond foundational research toward commercialization, manufacturing, and deployment,” she says.

“The US government has been signaling that quantum computing is a priority,” says David Mooter, an analyst at Forrester Research. Part of it is the desire for the US to be a leader in quantum, as it has been in other high-tech areas, he says. And part of it is because the government itself can take advantage of quantum computers.

“Spy agencies would love to use them to decrypt intercepted messages, including messages they intercepted years ago and saved,” he says. And other departments could use quantum computers or networks for energy-related research, for supply chain optimization, and for secure communications. 

Quantum computing is accelerating, he says. “I would not be surprised to see a general gate-based quantum computer that’s good enough to provide commercial value for limited use cases by 2030.”

DARPA’s Quantum Benchmarking Initiative

DARPA’s Quantum Benchmarking Initiatives was launched in 2024, and 18 companies were selected in April of 2025 for Stage A of the project, with awards of up to $1 million each. The companies were to use the money to provide details of their concepts and show how they could lead to a functional, fault-tolerant quantum computer in under a decade.

Then, in November of 2025, DARPA chose 11 companies for Stage B of the project, with awards of up to $15 million for developing their research plans.

To date, only two companies have been chosen for Stage C: PsiQuantum and Microsoft. PsiQuantum announced $32 million of DARPA funding for testing and evaluation in September of last year. This week’s $125 million award will expand the scope and pacing of the validation and verification work. Stage C awards can go up to $300 million, according to DARPA.

This past May, PsiQuantum also announced $100 million from the Department of Commerce, part of the CHIPS and Science Act, to accelerate domestic manufacturing of critical quantum computing components.

Microsoft and PsiQuantum are both in Stage C, bypassing the sequential path that other companies are expected to follow, because they were both part of DARPA’s predecessor to QBI, the Underexplored Systems for Utility-Scale Quantum Computing program.

Genesis Mission

Genesis Mission was launched in late 2025 with the goal of using AI to accelerate scientific breakthroughs, and it now includes more than 15 government agencies.

As part of the Genesis Mission, quantum computing and sensing company Infleqtion announced three projects for the Department of Energy on Wednesday. The three projects focus on quantum circuit design for nuclear applications, atomic quantum sensing, and nuclear fusion energy research.

This announcement did not include the total monetary value of the projects, but, in May, the company announced a separate agreement with the Department of Commerce for $100 million to accelerate Infleqtion’s neutral-atom technology roadmap.

Other quantum-related Genesis Mission projects announced this week include $1.5 million for a BlueQubit quantum error correction project with Microsoft and other partners, a Stanford effort to model the behavior of electrons at quantum scale, an MIT quantum sensing project, Argonne National Laboratory projects on quantum circuit design and quantum sensors, Brookhaven Lab quantum sensor projects, and quantum computing projects at Northwestern University.

IBM, one of three dozen private companies that are part of the Genesis Mission Consortium, announced that it will be leading a project to support more effective quantum applications, and will contribute up to $50 million of quantum compute access for the Genesis Mission.

Enterprise priorities

This week’s quantum announcements aren’t a sign that enterprises need to run out and buy quantum computers, says IDC’s West. But they do need to start preparing for the quantum era — such as by identifying business areas where quantum computing could become a competitive differentiator over the next decade.

But the most immediate threat is that of adversaries using quantum computers to break current encryption standards. Organizations should be inventorying cryptographic assets and developing a roadmap for the migration to quantum-proof algorithms, West says.

The point of no return is closer than ever, and many major players in the encryption and communication space, including Google and Cloudflare, have been accelerating their timelines. In fact, this Wednesday was the federal deadline for naming their post-quantum cryptography migration leads under a June executive order.

“The preparation that needs to be done to prepare is to implement post-quantum cryptography yesterday,” says Forrester’s Mooter.

However, according to a survey released by DigiCert this morning, while 87% of organizations are planning, testing or implementing PQC initiatives, only 7% of organizations have deployed quantum-safe or hybrid cryptography across most of their digital certificates.

OpenAI has launched Presence, an enterprise service for deploying voice and chat agents that can resolve customer and employee requests, potentially automating some work now handled by frontline support teams.

The agents can answer questions and operate IT systems, and enterprises can decide what actions the agents may take and when they should seek human approval for actions or transfer a case to a human.

OpenAI is already using Presence internally for its English-language phone support channel, where it verifies callers and uses account information to complete approved actions. The company said the system resolves 75% of inbound issues without human assistance.

Another OpenAI service, Codex, can be used to monitor agents and suggest updates or improvements to processes. In OpenAI’s own tests, suggestions from Codex helped reduce handoffs to humans by 15 percentage points over 10 days, it said. Presence also includes simulation and evaluation tools that allow companies to test an agent before deployment. The tests assess whether it reaches the correct outcome, follows company policy, and hands a case to an employee when required.

OpenAI intends each Presence deployment to deal with one kind of task, for example billing issues, insurance claims, or employee IT service requests, with agents getting only the knowledge and system access required for that task.

Presence is not a self-service product: Enterprises will have to sign up for the limited availability program, with integration performed by OpenAI or selected global systems integrators.

Companies exploring or testing Presence include Spanish bank BBVA, which is evaluating the service for everyday banking support in Mexico, and Japanese technology group SoftBank, which is using it in trials involving Japanese-language customer interactions. Australian insurer IAG is assessing whether the technology can help it respond to surges in customer demand during severe weather events.

Workforce impact

OpenAI’s announcement did not address the potential effect of Presence on employment. But its claimed automation rate raises questions about how the technology could affect staffing in customer service and other support functions.

Pareekh Jain, CEO of Pareekh Consulting, said CIOs should regard the 75% figure as evidence that the technology can work, rather than as a benchmark that every enterprise can expect to reach.

Jain said OpenAI’s deployment benefits from being built around the company’s own products and data. Large enterprises may achieve lower automation rates because they must contend with fragmented legacy systems, uneven knowledge bases and more complex compliance demands.

“Most organizations should expect lower initial automation levels that improve over time as the AI agent is refined,” Jain said.

The first workforce effect is more likely to be slower hiring than immediate layoffs, according to Tulika Sheel, senior vice president at Kadence International.

“The roles most exposed are likely to be repetitive, high-volume functions such as frontline customer support and routine back-office processing,” Sheel said. “However, I would expect the first impact to be on hiring and team growth rather than immediate large-scale job cuts. Over time, enterprises may redesign roles around AI-assisted workflows, with humans focusing more on complex cases, escalation, and relationship management.”

Jain said Tier-1 support agents handling predictable queries would face the most exposure. Broader reductions would become more likely only after companies reorganize their operations around the technology.

However, Lian Jye Su, chief analyst at Omdia, said Presence is unlikely to increase the threat of job displacement because companies have used similar customer-support automation from vendors such as Genesys, NiCE, Five9 and AWS for years.

Enterprises are more likely to use Presence alongside employees, with AI handling routine requests while people remain responsible for work requiring judgment and empathy, Su said.

Cost and operational risks

Analysts said CIOs should examine whether Presence can maintain resolution quality as usage grows, since fewer human handoffs could leave employees dealing with a more difficult mix of cases.

“The key question is not simply how many tasks AI can handle, but whether it can handle them reliably at scale,” Sheel said.

The financial case will depend partly on the cost of connecting Presence to existing systems and maintaining the controls needed to govern its use, according to Jain. “Often the biggest cost of enterprise AI is not tokens but integration and governance,” Jain added.

Companies will need to determine what systems and data the agents can access, monitor their performance, and audit the actions they take. Those investments could offset early savings.

Su said the complexity of enterprise IT will make it difficult for OpenAI to automate entire workflows on its own. Enterprises will still need to work with other technology providers and human employees, while CIOs will favor systems that can be audited and integrated with existing infrastructure.

Jain said the economics could improve if companies use the same integrations and governance controls across additional workflows.

OpenAI has launched Presence, an enterprise service for deploying voice and chat agents that can resolve customer and employee requests, potentially automating some work now handled by frontline support teams.

The agents can answer questions and operate IT systems, and enterprises can decide what actions the agents may take and when they should seek human approval for actions or transfer a case to a human.

OpenAI is already using Presence internally for its English-language phone support channel, where it verifies callers and uses account information to complete approved actions. The company said the system resolves 75% of inbound issues without human assistance.

Another OpenAI service, Codex, can be used to monitor agents and suggest updates or improvements to processes. In OpenAI’s own tests, suggestions from Codex helped reduce handoffs to humans by 15 percentage points over 10 days, it said. Presence also includes simulation and evaluation tools that allow companies to test an agent before deployment. The tests assess whether it reaches the correct outcome, follows company policy, and hands a case to an employee when required.

OpenAI intends each Presence deployment to deal with one kind of task, for example billing issues, insurance claims, or employee IT service requests, with agents getting only the knowledge and system access required for that task.

Presence is not a self-service product: Enterprises will have to sign up for the limited availability program, with integration performed by OpenAI or selected global systems integrators.

Companies exploring or testing Presence include Spanish bank BBVA, which is evaluating the service for everyday banking support in Mexico, and Japanese technology group SoftBank, which is using it in trials involving Japanese-language customer interactions. Australian insurer IAG is assessing whether the technology can help it respond to surges in customer demand during severe weather events.

Workforce impact

OpenAI’s announcement did not address the potential effect of Presence on employment. But its claimed automation rate raises questions about how the technology could affect staffing in customer service and other support functions.

Pareekh Jain, CEO of Pareekh Consulting, said CIOs should regard the 75% figure as evidence that the technology can work, rather than as a benchmark that every enterprise can expect to reach.

Jain said OpenAI’s deployment benefits from being built around the company’s own products and data. Large enterprises may achieve lower automation rates because they must contend with fragmented legacy systems, uneven knowledge bases and more complex compliance demands.

“Most organizations should expect lower initial automation levels that improve over time as the AI agent is refined,” Jain said.

The first workforce effect is more likely to be slower hiring than immediate layoffs, according to Tulika Sheel, senior vice president at Kadence International.

“The roles most exposed are likely to be repetitive, high-volume functions such as frontline customer support and routine back-office processing,” Sheel said. “However, I would expect the first impact to be on hiring and team growth rather than immediate large-scale job cuts. Over time, enterprises may redesign roles around AI-assisted workflows, with humans focusing more on complex cases, escalation, and relationship management.”

Jain said Tier-1 support agents handling predictable queries would face the most exposure. Broader reductions would become more likely only after companies reorganize their operations around the technology.

However, Lian Jye Su, chief analyst at Omdia, said Presence is unlikely to increase the threat of job displacement because companies have used similar customer-support automation from vendors such as Genesys, NiCE, Five9 and AWS for years.

Enterprises are more likely to use Presence alongside employees, with AI handling routine requests while people remain responsible for work requiring judgment and empathy, Su said.

Cost and operational risks

Analysts said CIOs should examine whether Presence can maintain resolution quality as usage grows, since fewer human handoffs could leave employees dealing with a more difficult mix of cases.

“The key question is not simply how many tasks AI can handle, but whether it can handle them reliably at scale,” Sheel said.

The financial case will depend partly on the cost of connecting Presence to existing systems and maintaining the controls needed to govern its use, according to Jain. “Often the biggest cost of enterprise AI is not tokens but integration and governance,” Jain added.

Companies will need to determine what systems and data the agents can access, monitor their performance, and audit the actions they take. Those investments could offset early savings.

Su said the complexity of enterprise IT will make it difficult for OpenAI to automate entire workflows on its own. Enterprises will still need to work with other technology providers and human employees, while CIOs will favor systems that can be audited and integrated with existing infrastructure.

Jain said the economics could improve if companies use the same integrations and governance controls across additional workflows.

This article first appeared on CIO.

Google’s AI offerings span its internal and cloud offerings. Its data centers are processing seven times more AI tokens compared to last year. To keep up, Google is upgrading its data-center hardware and software technologies at a faster clip. It plans to raise $80 billion to build new data centers. (See related story: Google transforms its data center architecture for agent era)

Network World spoke with Mark Lohmeyer, vice president and general manager of AI and computing at Google, about how the company’s infrastructure is keeping pace with AI demand.

Network World: What is the primary shift in infrastructure needs?

Mark Lohmeyer: We’ve seen the rise of agents and agentic use cases. Years ago, it was the chat phase: Ask a question, get an answer. Now we’re in the agentic era, where you express your intent, agents spin off multiple sub-agents, working in parallel, preserving state. This is a radical shift in what infrastructure needs to do; make them fast, cost effective, secure, reliable. We’re delivering infrastructure optimized for the age of agents.

NW: What’s the goal of the infrastructure buildout, and what should customers expect regarding costs?

ML: Ultimately, it’s about enabling customers with leading-edge capabilities and models at scale cost-effectively. With agents, inference transactions increase by 50x, 100x versus non-agentic workloads. We’re driving the cost per transaction down exponentially. In our latest platforms, we reduce the cost by almost 2x for the same work. Customers serve twice the number of users at the same cost, directly driving profitability.

NW: How are you addressing energy efficiency?

ML: Energy is a critical resource, and Google has optimized for years. We design data centers and compute [to drive] high PUE (power usage effectiveness). We introduced liquid cooling over five years ago, and these latest systems are all liquid cooled. For agentic workloads, CPUs come to the forefront… orchestrating agents, calling tools, doing evaluation loops in reinforcement learning. Our latest Axion-based CPU platform called N4A has energy efficiency and is significantly better than the prior generation and x86 comparables.

NW: How do you think about token efficiency as you build-out systems?

ML: Performance and efficiency gains are powered by co-design of the model and infrastructure. Gemini is trained on TPUs, primarily served on TPUs with high frontier model capability, in a token and cost-efficient way. This stems from co-design across the full stack.

NW: How do you project what infrastructure will be needed years in advance?

ML: Hardware cycles deliver a new next generation roughly every year, but design cycles are two years or more in advance. We work with DeepMind doing core research, to application teams taking models into production, to billions of users, to our team building infrastructure. We work upstream with DeepMind and application teams to understand what’s coming. Agents weren’t being broadly spoken of externally, but internally we had those insights around what they would need. That shows up in hardware design. We hit the timing right — these platforms are built for agents.

NW: What’s the eighth generation TPU platform?

ML: We deliver new platforms every year, and ones launched years ago are close to 100% utilized because demand for AI-optimized compute is high. The eighth-generation TPU platform is the first delivering two complete systems, from the chip all the way up to the network and storage and software, that are optimized.

TPU-8t is optimized for training, and TPU-8i is optimized for inference. For TPU-8i, we increased SRAM on the chip to 384MB — three times the prior generation — and increased the HBM by 50%.

NW: How are you approaching GPU and TPU compatibility?

ML: People in a single cluster do not commingle GPUs and TPUs. We offer both options based on specific workload needs. We’ve been investing on the TPU side in using software frameworks customers are comfortable with on GPUs and enabling those on TPUs. For example, PyTorch and vLLM. Customers could have a pool of GPUs and TPUs, running vLLM on top of that. Start with a workload on TPUs, but if the TPU pool is fully utilized, spill to GPUs or vice versa. This works because it’s all leveraging the same compatible software layer on top.

NW: How has the orchestration platform changed for agents?

ML: Kubernetes is becoming the orchestration platform of choice for AI. Google is transforming GKE [Google Kubernetes Engine] into an agent-native orchestration solution. When expressing intent to an agent and it spins up multiple sub-agents, compute needs to spin up rapidly — TPUs or GPUs — without long delays, then run and spin back down. We’re optimizing at every layer of the GKE stack: significantly improving node startup time and how rapidly we start and stop containers. Lovable demonstrates this with GKE, spinning up hundreds of sandboxes for live coding sessions on their platform in parallel, paying for infrastructure when needed.

NW: What is the role of the network and storage infrastructure?

ML: The network is critical for AI. This requires creating large-scale clusters of GPUs or TPUs and enabling them to talk to each other in a high-performance way. We created the Virgo network — a collapsed network architecture, non-blocking within a data center, where multiple pods or NVLink72 domains connect together.

In TPU8T, we can connect over a million TPUs together leveraging Virgo, creating large-scale, high-performance, reliable clusters that shrink innovation cycles. Storage is equally critical. In large-scale clusters, something is always failing. The ability to take snapshots and go back to a checkpoint is important.

We’ve introduced Managed Lustre 10T, with 10 terabytes per second of bandwidth, 18 petabytes of storage in single clusters. This is 10 times faster than last year and 20 times faster than competition. We have Rapid Bucket, low-latency storage backed by Google storage systems. Both are impactful in large-scale training environments.

NW: How does KV cache strategy differ between training and inference?

ML: For TPU-8i, we increased SRAM on the chip to 384 megabytes — three times the prior generation — and increased the HBM by 50%. Storing KV cache directly in chip memory allows responding to inference requests much more rapidly and cost-effectively than going to an external system. For inference workloads, storing as much KV cache as possible on-chip is critical.

We’re introducing a dedicated KV cache storage subsystem that works across GPUs and TPUs. As KV caches get larger, being able to fall back to this dedicated subsystem becomes critical. Loading model weights rapidly is important in dynamic inference environments where accelerators switch between models hour by hour.

Nelson Antoine | shutterstock.com

Der heimliche Cybercrime-Akt zweier KI-Modelle von OpenAI hat weltweit ein enormes Echo in Mainstream– und sozialen Medien hervorgerufen. Der Vorfall dürfte die Debatte über die allgemeine KI-Sicherheit und den verantwortungsvollen Umgang mit der Technologie neu befeuern.

Doch der Incident wirft auch spezifische Fragen auf. Etwa, wie genau die OpenAI-Modelle es geschafft haben, ihrer Sandbox zu entkommen und warum das beim ChatGPT-Erfinder zunächst niemandem aufgefallen ist. Oder, wie andere Unternehmen solche und ähnliche Vorkommnisse künftig verhindern können. Dazu haben wir die Einschätzung von Branchenexperten und Analysten eingeholt.

Zunächst werfen wir aber noch einen kurzen Blick darauf, was sich eigentlich abgespielt hat. Falls Sie bereits informiert sind, können Sie alternativ auch das nachfolgende Meme konsumieren, um sich den Vorfall noch einmal auf unkonventionellere Art und Weise vor Augen zu halten.

Der autonome Hugging-Face-Hack

Die KI-Plattform Hugging Face meldete Mitte Juli einen Cyberangriff auf ihre Produktions-Infrastruktur. Diese wurde offenbar vollständig autonom von einem agentenbasierten KI-System gefahren, das sich dabei sowohl Zugriff auf interne Datensätze als auch auf Zugangsdaten verschaffen konnte.  

Wie sich im Rahmen der folgenden Untersuchung herausstellte, stammte die autonom hackende KI von OpenAI. Eigentlich wollte der KI-Pionier lediglich zwei seiner neuen Modelle – GPT 5.6 Sol sowie ein nicht näher spezifiziertes Pre-Release-Modell – innerhalb einer Sandbox mit dem Benchmarking-Tool ExploitGym testen.

Die Aufgabe für die KI: Schwachstellen in tragfähige Exploits zu verwandeln. Die Testumgebung selbst war OpenAI zufolge mit beschränktem Netzwerkzugriff ausgestattet. Die KI sollte lediglich auf einen intern gehosteten Drittanbieter-Service zugreifen können, der als Proxy und Cache für eine Package-Registry fungierte. Die Guardrails auf Modellebene waren für den Test laut dem KI-Unternehmen reduziert worden.

Das war der KI aber offenbar nicht genug: Sie verkettete zunächst einige zuvor unbekannte Schwachstellen in der Testumgebung, um sich uneingeschränkten Internet-Zugriff zu verschaffen, wie OpenAI in einem Blogbeitrag offenlegt: „Mit diesem Zugriff führten unsere Modelle in unserer Forschungsumgebung eine Reihe von Privilege-Escalation- und Lateral-Movement-Aktionen aus, bis sie eine Node mit Internetzugang erreichten.“

Weil die OpenAI-Modelle bei Hugging Face offenbar Lösungen für ihre ExploitGym-Aufgaben vermuteten, suchten sie nach Möglichkeiten, Zugriff auf diese zu erhalten. Und fanden sie dann auch, wie OpenAI darlegt: „In einem Beispiel verkettete das Modell mehrere Angriffsvektoren, darunter gestohlene Zugangsdaten und Zero-Day-Schwachstellen, um auf den Servern von Hugging Face einen Remote-Execution-Pfad zu finden.“

Das Security-Team von OpenAI konnte diese Vorgänge im Nachgang identifizieren und schloss sich daraufhin mit dem Team von Hugging Face zusammen, welches die Attacke seinerseits bereits identifiziert und (mit einigen Anlaufschwierigkeiten) eingedämmt hatte.

„Wir betrachten dies als beispiellosen Cybervorfall mit hochentwickelten Fähigkeiten und reagieren entsprechend. Wir teilen zu diesem Zeitpunkt vorläufige Erkenntnisse, damit Sicherheitsverantwortliche nachvollziehen können, was passiert ist, und besser einschätzen können, wozu die Modelle inzwischen in der Lage sind“, schreibt OpenAI in seinem Blog – und verspricht, weitere Details zu veröffentlichen, sobald diese vorliegen.

KI-Ausbruch bei OpenAI – so reagieren Experten

Branchenexperten und Analysten bewerten den schlagzeilenträchtigen Incident um OpenAI und Hugging Face folgendermaßen:

• Alexei Balaganski, Lead Analyst bei KuppingerCole: „Dieser Vorfall sollte nicht als ‚Rogue AI‘-Geschichte betrachtet werden. Das Modell hat exakt das getan, wofür agentische Systeme gemacht sind: Es hat sich allen verfügbaren Tools und Wegen bedient, um das ihm gesetzte Ziel zu erreichen. Die Sicherheitsvorkehrungen, die es normalerweise in Zaum gehalten hätten, wurden von OpenAI selbst zu Testzwecken deaktiviert. Darin besteht die wahre Lektion.“
• Jonathan Care, Lead Analyst und AI Practice Lead bei KuppingerCole: „Es geht bei diesem Vorfall nicht darum, dass eine KI ausgebrochen ist und zum Angreifer wurde. Wir wussten, das würde passieren. Bemerkenswert ist allerdings, dass die Verteidiger – in diesem Fall das Team von Hugging Face – keine kommerziellen KI-Modelle nutzen konnten, um den Angriff zu analysieren. Denn deren Guardrails sorgen dafür, dass kein Exoploit-Code verarbeitet werden kann.“
• Gerald Beuchelt, CISO bei Acronis: „Der Vorfall verdeutlicht eine zentrale Herausforderung für Incident-Response-Teams: Angreifer sind nicht an Nutzungsrichtlinien gebunden. Verteidiger können hingegen an die Grenzen ihrer eigenen Tools stoßen, wenn diese genau jene Daten nicht verarbeiten, die für eine Untersuchung erforderlich sind. Im Ernstfall können daraus Verzögerungen mit unmittelbaren operativen Folgen entstehen.“
• Sabine Frömling, Experten-Autorin und Cybersecurity-Beraterin: „Der eigentliche Sicherheitsvorfall war nicht die KI – sondern die Sandbox, die aus Versehen eine Tür zum Internet hatte. Man hat ein Raubtier freigelassen und dem Zaun die Schuld gegeben.“
• Martin Zugec, Technical Solutions Director bei Bitdefender: „Was meiner Meinung nach für KI-generierte Malware galt, untermauert auch dieser Vorfall: Die Bedrohung ist real, KI ist aber keine Magie. Wer glaubt, es mit einer neuartigen Superwaffe zu tun zu haben, wartet auf eine neuartige Gegenmaßnahme. Wer jedoch erkennt, dass es sich um bereits bekannte, aber unerbittlich angewandte Angriffstechniken handelt, weiß bereits, was zu tun ist.“
• Richard Werner, Cybersecurity Platform Lead Europe bei TrendAI: „Das Narrativ von der ‚eigenmächtig handelnden KI‘ ist effizient darin, Verantwortung abzuwälzen. Das ist, als würden Sie eine autonome Waffe bauen, diese auf einem vermeintlich sicheren Testgelände erproben, sie außer Kontrolle geraten und jemanden treffen lassen – und der Welt anschließend erklären, die Waffe habe eigenständig gehandelt. Das ist zwar technisch korrekt. Dennoch bleibt es Ihre Waffe, Ihr Testgelände und Ihr Versagen.“

Was Unternehmen jetzt tun sollten

IT- und Sicherheitsentscheider können aus dem Hugging-Face-Hack mehrere Lektionen ziehen. Etwa, dass Sicherheitsvorkehrungen auf Modellebene nicht als primäre Security-Grenze für KI-Agenten geeignet sind, wie Biswajeet Mahapatra, Principal Analyst bei Forrester, festhält: „Prompt-Guardrails sind keine Sicherheits-, sondern Verhaltenskontrollmaßnahmen. Und diese können versagen, umgangen oder absichtlich deaktiviert werden.“

Der Forrester-Analyst rät Unternehmen deshalb dazu, KI-Agenten als hochriskante, nicht-menschliche Identitäten zu behandeln – und jeden einzelnen in einer isolierten Umgebung zu betreiben, in der Datenzugriff auf den jeweiligen Task beschränkt bleibt und die Zugangsdaten selbst möglichst schnell ablaufen: „Das sorgt für einen akzeptablen ‚Blast Radius‘: Wird ein Agent kompromittiert, kann er nur einen einzigen Workflow, Datensatz oder eine einzige Anwendung beeinträchtigen. Anstatt die gesamte Unternehmensinfrastruktur.“

Sanchit Vir Gogia, Chefanalyst bei Greyhound Research, warnt an dieser Stelle davor, (Drittanbieter-)Services unter den Tisch fallen zu lassen: „Dienste, die auf Package Registries, Update-Systeme oder andere externe Ressourcen zugreifen, können ebenfalls zu Einfallstoren werden, wenn sie nicht derselben, ausgiebigen Prüfung unterzogen werden wie der Agent selbst.“

Unabhängig davon sollten Unternehmen laut Gogia auch testen, ob ihre Containment-Grenzen auch funktionieren, anstatt sich allein auf Architekturdiagramme oder dokumentierte Richtlinien zu verlassen: „Im Rahmen dieser Tests sollte geprüft werden, ob Anmeldedaten erlangt, Trust-Grenzen überwunden und Systeme außerhalb der einem Agenten zugewiesenen Aufgabe erreicht werden können.“

KuppingerCole-Chefanalyst Care rät IT-Entscheidern und Unternehmen im Wesentlichen zu drei Maßnahmen, nämlich:

• ein fähiges Modell auf der eigenen Infrastruktur auszuführen, das unter der eigenen Kontrolle steht und mit Guardrails ausgestattet ist, die sowohl eine forensische als auch defensive Nutzung ermöglichen. Nur so ließen sich Angriffe dieser Art auch zuverlässig analysieren.
• jeden KI-Agent in der eigenen Umgebung als privilegierten Insider zu behandeln – statt als vertrauenswürdigen Benutzer: „Wenn die Modelle von OpenAI aus ihrer Sandbox ausgebrochen sind, sollten Sie davon ausgehen, dass Ihre Agenten dazu auch in der Lage sind.“
• den eigenen Incident-Response-Plan mit Blick auf Angriffe in maschineller Geschwindigkeit zu aktualisieren: „Hugging Face hatte einige Tage Zeit, um zu reagieren, Sie haben vielleicht nur Minuten.“   

Acronis-CISO Beuchelt rät Organisationen, die gehostete LLMs für Security-Untersuchungen einsetzen, dazu, deren Grenzen möglichst bereits im Vorfeld zu durchdringen und zu testen – sowie ein alternatives Modell auf der eigenen Infrastruktur bereitzuhalten: „So reduzieren Sie das Risiko, im entscheidenden Moment keinen Zugriff auf wichtige Analysefunktionen zu haben. Gleichzeitig bleiben sensible Incident-Daten und Zugangsinformationen innerhalb der eigenen Organisation.“

Udo Schneider, Governance, Risk & Compliance Lead Europe bei TrendAI weist darauf hin, dass die beiden naheliegendsten Lösungsansätze bei Angriffen wie dem der OpenAI-KI auf Hugging Face nur teilweise greifen. Human-in-the-Loop-Kontrollen funktionierten zwar, so der Experte, skalierten aber nicht für die langlaufenden, komplexen Workflows, denen Incidents dieser Art entspringen. Ebenso könnten engere Guardrails für Modelle oder Prompts zwar helfen, stellten jedoch keine Garantie dar: „Es handelt sich um probabilistische Systeme. Eine Guardrail ist insofern keine Mauer, sondern eher eine starke Wahrscheinlichkeitsannahme.“

Deshalb komme es laut Schneider vor allem auf die unspektakulären, nicht-KI-spezifischen Kontrollen an: „Zugriffsfilterung, Kontrolle darüber, was überhaupt als Input beim Modell ankommt, Sandboxes, die tatsächlich halten, und Berechtigungskonzepte nach dem Least-Privilege-Prinzip.“

In Panik zu verfallen, wäre nach Ansicht von Martin Zugec, Technical Solutions Director bei Bitdefender, in jedem Fall die falsche Reaktion:„Was gegen solche Angriffe wirkt, ist eine präventionsorientierte Security, die den Handlungsspielraum eines Angreifers von vorneherein einschränkt – und eine verhaltensbasierte Abwehr, die bösartige Muster kennzeichnet, unabhängig davon, mit welchen Tools diese generiert wurden.“

Dieser Artikel wurde mit Material unserer Schwesterpublikation CSOonline.com angereichert.

Over the past few years, I have learned that basic prompts produce inconsistent, hallucination-prone results that no executive would trust in production. What turned the tide was my move to advanced prompting techniques. These weren’t theoretical experiments; they became a practical foundation for reliable, measurable outcomes. I want to share the techniques that consistently delivered the biggest gains in my projects, complete with real before-and-after examples, copy-paste templates, lessons from failures and guidance on when to evolve beyond prompting to agentic systems.

Why advanced prompting still matters in enterprise settings

Sophisticated prompting remains essential for control, reliability and compliance. If you “ask nicely” and hope for the best, you need deterministic behavior, auditable reasoning and minimal risk of hallucination. Here’s what worked for me.

1. Chain-of-Thought (CoT) and its variants: Unlocking step-by-step reasoning

The problem: Models would jump to conclusions on complex analysis tasks, especially involving data interpretation or multi-step logic.

What I did: I started explicitly instructing the model to “think step by step” and show its reasoning.

Before (basic prompt): “Analyze last quarter’s sales data and recommend three actions.”

After (CoT prompt):

“You’re a senior business analyst. Analyze the following sales data step by step: [data]. First, identify the key trends. Second, calculate the rates and anomalies. Third, link findings to business context. Finally, recommend the three prioritized actions with expected impact. Explain your reasoning at each step.”  

Results: Accuracy and depth improved dramatically.

Variants that worked well: Self-consistency. I ran the same CoT prompt multiple times and took the majority consensus. This reduced variability significantly.

Template you can use:

You are [expert role]. Solve this problem by thinking step by step.

[Task or question]

For each step:

1. State your observation or calculation.

2. Explain the implication.

3. Proceed only when confident.

Final answer in this format: [structured output]

2. Tree-of-Thoughts (ToT): Exploring multiple reasoning paths

For truly complex decisions such as resource allocation or risk assessment, linear CoT isn’t enough. Tree-of-Thoughts lets the model generate and evaluate multiple branches.

Example: I was helping a client evaluate three potential vendor platforms for an AI deployment. A standard prompt gave a superficial comparison. With ToT

Prompt Snippet:

Explore three different reasoning paths for selecting the best vendor platform:

Path 1: Focus on cost and scalability.

Path 2: Focus on security, compliance and integration.

Path 3: Focus on innovation and long-term roadmap.

For each path, evaluate pros/cons against our requirements [list].

Then, compare the paths and recommend the strongest overall option with justification.

Outcome: The model surfaced nuanced trade-offs (e.g., one vendor had superior security, but higher integration cost).

When to use: Strategic planning, troubleshooting or scenarios with high uncertainty and multiple viable approaches.

3. ReAct (Reason+ Act) and prompt chaining: Moving toward agentic behavior

One of the biggest leaps I have noticed comes from combining reasoning with tool use and chaining prompts.

ReAct example: (used in data analytics workflow)

You are an AI analyst with access to tools. For the query below:

1. Reason about what information you need.

2. Choose the appropriate tool or action.

3. Observe the result.

4. Repeat until you can answer confidently.

Query: [user request]

In practice, I chained this with retrieval tools. One automated quarterly compliance reporting; the system reasoned about required data, pulled relevant records, validated them, and generated the reports.

4. Meta-prompting and self-reflection: Letting the model improve itself

Use the model to refine its own prompt. This is a huge time-saver.

You are an expert prompt engineer. Improve the following prompt for clarity, structure and effectiveness with [target model]. Make it more precise while preserving intent.

Original prompt: [paste]

Provide the improved version and explain your changes.

Self-reflection loops (asking the model to critique its own output and revise) are a game-changer for content generation and code-review tasks.

5. Multimodal and structured output techniques

With vision-enabled models, I started combining text with images (e.g., uploading architecture diagrams or dashboards).

Tip from experience: Be extremely specific in describing what the models should focus on.

Best practices I learned the hard way

• Start simple, then layer complexity: Over-engineered prompts from Day One usually backfire.
• Model specific tuning: Some models respond better to XML delimiters; others to explicit reasoning.
• Evaluation and versioning: Treat prompts like code if you track versions and run automated evals.
• Security guardrails: Always include instructions against prompt injections and respect data boundaries.
• When to stop prompting: For repetitive, high-stakes workflows, move to full agents or an orchestration framework.

Final takeaways for technical leaders

Advanced prompt engineering has now become a core competency for anyone responsible for enterprise AI outcomes. Start by picking one technique and apply it rigorously to a real business problem. Document before/ after and you will notice why it’s worth mastering.

The field continues evolving towards more automated and agentic systems, but the ability to precisely direct AI reasoning remains foundational.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Cisco has revealed a family of open-weight AI models called Antares that, it said, can help security teams isolate potentially vulnerable parts of a software repository before deeper investigation begins.

Rather than detecting a specific CVE or generating a patch, these models search a codebase using only a Common Weakness Enumeration (CWE) description and return the files most likely to contain that class of vulnerability.

“Its purpose is to reduce a large codebase to a focused set of files that a security professional or a downstream security workflow should investigate,” Cisco’s AI researcher Supriti Vijay said via email. “The goal is not to replace a security engineer’s judgement or send them on a wild-goose chase, but to reduce fatigue and workload by helping them triage an issue earlier and focus their investigation on the most relevant parts of the codebase.”

The Antares family consists of models with 350 million, 1 billion, and 3 billion parameters trained specifically for repository-scale vulnerability localization.

The company said its largest model approaches the performance of GPT-5.5 on its internal vulnerability localization (Vloc) benchmark while remaining small enough for low-cost local deployment.

A search assistant, not a vulnerability detector

Cisco is careful to define what Antares is, and what it is not.

“Antares outputs a ranked list of source files likely to contain a relevant vulnerability, along with the terminal exploration trace that led to that result,” Cisco Foundation AI Chief Scientist Amin Karbasi wrote in a blog post, adding that the models are not meant to replace the broader application security toolchain: Human analysts or downstream security tools will still be needed to confirm exploitability, identify vulnerable lines of code, assess severity and generate fixes.

Antares differs from conventional static analysis platforms such as Semgrep or CodeQL, which primarily rely on predefined rules or queries. Cisco instead describes Antares as an evidence-driven exploration agent that adapts its search as it traverses the repository.

Cisco’s argument is that large repositories often contain thousands of files, making manual reviews exhaustive and unrealistic. By reducing the search space to a manageable shortlist, the company hopes to reduce investigation fatigue without replacing human judgement.

Claims of specialization over scale

Cisco is also making a statement about how cybersecurity models should evolve.

Instead of pursuing larger foundational models, Cisco argued that specialized, task-trained models can outperform much larger open-weight alternatives for vulnerability localization. In its evaluation Antares-3B, the largest model intended for single-GPU deployments, produced results comparable to GPT-5.5 while outperforming several substantially larger open models by Google, OpenAI and Meta.

The family also includes Antares-350M for resource-constrained environments and Antares-1B for laptops and workstations, which Cisco has made available as open-weight models on Hugging Face.

The command line interface (CLI) on the models supports targeted CWE investigations, repository-wide scans, SARIF output and local inference, which Cisco said enables organizations to keep proprietary code inside their own trust boundary.

However, because Antares identifies candidate files rather than confirmed vulnerabilities, organizations will still need to understand how often such repository-wide searches should be run, how much they improve existing triage workflows, and whether the reduction in investigation effort ultimately translates into measurable security or cost benefits.

Cisco has revealed a family of open-weight AI models called Antares that, it said, can help security teams isolate potentially vulnerable parts of a software repository before deeper investigation begins.

Rather than detecting a specific CVE or generating a patch, these models search a codebase using only a Common Weakness Enumeration (CWE) description and return the files most likely to contain that class of vulnerability.

“Its purpose is to reduce a large codebase to a focused set of files that a security professional or a downstream security workflow should investigate,” Cisco’s AI researcher Supriti Vijay said via email. “The goal is not to replace a security engineer’s judgement or send them on a wild-goose chase, but to reduce fatigue and workload by helping them triage an issue earlier and focus their investigation on the most relevant parts of the codebase.”

The Antares family consists of models with 350 million, 1 billion, and 3 billion parameters trained specifically for repository-scale vulnerability localization.

The company said its largest model approaches the performance of GPT-5.5 on its internal vulnerability localization (Vloc) benchmark while remaining small enough for low-cost local deployment.

A search assistant, not a vulnerability detector

Cisco is careful to define what Antares is, and what it is not.

“Antares outputs a ranked list of source files likely to contain a relevant vulnerability, along with the terminal exploration trace that led to that result,” Cisco Foundation AI Chief Scientist Amin Karbasi wrote in a blog post, adding that the models are not meant to replace the broader application security toolchain: Human analysts or downstream security tools will still be needed to confirm exploitability, identify vulnerable lines of code, assess severity and generate fixes.

Antares differs from conventional static analysis platforms such as Semgrep or CodeQL, which primarily rely on predefined rules or queries. Cisco instead describes Antares as an evidence-driven exploration agent that adapts its search as it traverses the repository.

Cisco’s argument is that large repositories often contain thousands of files, making manual reviews exhaustive and unrealistic. By reducing the search space to a manageable shortlist, the company hopes to reduce investigation fatigue without replacing human judgement.

Claims of specialization over scale

Cisco is also making a statement about how cybersecurity models should evolve.

Instead of pursuing larger foundational models, Cisco argued that specialized, task-trained models can outperform much larger open-weight alternatives for vulnerability localization. In its evaluation Antares-3B, the largest model intended for single-GPU deployments, produced results comparable to GPT-5.5 while outperforming several substantially larger open models by Google, OpenAI and Meta.

The family also includes Antares-350M for resource-constrained environments and Antares-1B for laptops and workstations, which Cisco has made available as open-weight models on Hugging Face.

The command line interface (CLI) on the models supports targeted CWE investigations, repository-wide scans, SARIF output and local inference, which Cisco said enables organizations to keep proprietary code inside their own trust boundary.

However, because Antares identifies candidate files rather than confirmed vulnerabilities, organizations will still need to understand how often such repository-wide searches should be run, how much they improve existing triage workflows, and whether the reduction in investigation effort ultimately translates into measurable security or cost benefits.

This article first appeared on InfoWorld.

Some of OpenAI’s most powerful AI models teamed up to escape their sandbox and attack systems at Hugging Face in a cybersecurity evaluation gone wrong, the company has admitted. The models under test were modified to allow them to perform potentially harmful actions that production versions would refuse. The incident highlights how, if AI prompt guardrails fail or, as in this incident, are removed, then enterprises must have robust sandboxing or other technical restrictions in place to protect systems.

In OpenAI’s evaluation, GPT-5.6 Sol and a more capable pre-release model were competing to solve cybersecurity challenges in ExploitGym, a benchmarking tool for cybersecurity capabilities. They concluded that systems at Hugging Face could have the answers, and set out to gain access, OpenAI wrote in a blog post about the attack on Hugging Face.

ExploitGym challenges AI models to turn security vulnerabilities into attack exploits. OpenAI gave its models limited access to the internet through a package-registry proxy to download code to build their attacks. However, the models exploited a zero-day vulnerability in the proxy to obtain unrestricted internet access, then used stolen credentials and other flaws to access Hugging Face systems.

For CIOs and CISOs, the concern extends beyond cybersecurity research. AI agents connected to corporate databases, source code repositories, or payment systems could expose sensitive resources unless access controls and sandboxing continue to work when model-level safeguards fail.

“Prompt guardrails are not security controls,” said Biswajeet Mahapatra, principal analyst at Forrester. “They are behavioral controls that can fail, be bypassed, or be intentionally disabled during testing.”

How containment failed

Despite the relaxed model safeguards, the surrounding infrastructure was supposed to contain the activity. Network access was restricted to an internally hosted third-party service that acted as a proxy and cache for software package registries. Such proxies are typically used in the enterprise to protect against software supply chain attacks coming from the outside.

OpenAI said the models discovered a zero-day vulnerability in the service, exploited it and carried out privilege escalation and lateral movement until they reached a node with open internet access.

Once online, the models inferred that Hugging Face might hold models, datasets or solutions related to ExploitGym. OpenAI said that, in one attack path, the models combined stolen credentials with zero-day vulnerabilities to obtain remote code execution on Hugging Face servers.

In its account of the activity on its infrastructure, Hugging Face said a malicious dataset abused two code-execution paths in its data-processing pipeline. The actor then obtained node-level access, harvested cloud and cluster credentials, and moved into several internal clusters.

OpenAI said the models ultimately obtained ExploitGym test solutions directly from a Hugging Face production database. The models remained focused on solving the benchmark, despite the lengths to which they went to achieve that objective, it said.

Sanchit Vir Gogia, chief analyst at Greyhound Research, said the package-registry proxy should concern enterprise security teams because it was a narrowly permitted exception inside an otherwise restricted environment.

“Every enterprise estate is built from exceptions of exactly that kind,” he said.

In this incident, OpenAI’s models were performing the intended — albeit by unintended means — but enterprises should take care that their own AI systems are kept on target.

Mahapatra said enterprises should treat AI agents as “high-risk non-human identities,” confining each one to an isolated environment where access is limited to the assigned task and credentials expire quickly.

“An acceptable blast radius means a compromised agent can affect only a single workflow, dataset, or application rather than providing a pathway into broader enterprise infrastructure,” Mahapatra said.

And Gogia warned that services allowed to reach package registries, update systems, or other external resources can become escape routes if they are not subjected to the same scrutiny as the agent itself.

Defenses enterprises need

Hugging Face said the intrusion resulted in unauthorized access to a limited number of internal datasets and several credentials used by its services. It found no evidence that public models, datasets or Spaces had been altered and said its published software packages and container images were not compromised.

The company closed the code-execution paths used to gain access and rebuilt the affected nodes. It also revoked exposed credentials and tightened the rules governing workloads admitted to its clusters.

Whether they are keeping their own AIs in or rogue Ais out, Gogia said enterprises should test whether their containment boundaries work, rather than relying on architecture diagrams or stated policies. Such tests should attempt to obtain credentials, cross trust boundaries and reach systems outside the agent’s assigned task.

Mahapatra said enterprises should assume that one containment layer may fail and ensure that an agent’s access cannot provide a route into unrelated applications or broader corporate infrastructure.

OpenAI said it is still investigating the incident with Hugging Face, and is imposing stricter configurations on its research environment while the vulnerabilities are being addressed, even if that means slowing down its research. It is also strengthening containment and monitoring around future evaluations.

HireQuotient, an AI-native recruiting platform, today announced its integration with Paylocity (Nasdaq: PCTY), bringing AI-powered candidate sourcing and screening capabilities to Paylocity customers in manufacturing, building services, construction, healthcare and insurance, which are industries where deskless and frontline hiring has historically been underserved by AI recruiting tools.

Employers in these sectors who already use HireQuotient are seeing the impact firsthand.

“By simply putting in vetting criteria, I was able to get a very specific talent pool within a day,” said Niki Simoneaux, COO of Arc Health. “With EasySource, we can reach a huge database or filter for specific licenses and areas.”

“With technology and the ability to work remotely, we’re able to recruit nationwide outside of our footprint. This expands the pool of prospective candidates and gives us access to candidates who might not have ever applied for a position on our website’s career page,” said Jeff McGee, vice president at W3 Insurance.

“HireQuotient’s AI-native product helped my team at Alliance Building Services to cut down time to close position by more than 60%, and their team works very closely with the client to ensure adoption at scale,” said Willow Marcon, senior vice president at Alliance Building Services.

Recruiters in manufacturing, building services, construction, healthcare and insurance often spend more than two-thirds of their time on manual work and using over 10 platforms to just close a hire: sourcing a wide range of profiles, skimming resumes, finding contact information, reaching out and doing hundreds of calls and constant follow-ups. That fragmented process leads to recruiter burnout and leaves gaps in candidate data. Because most HR platforms don’t offer built-in AI native agents that work in tandem to do all it takes to close the hire, employers in these industries have faced hiring delays and platform churn.

The partnership addresses this gap by integrating HireQuotient’s EasySource platform directly into the Paylocity ecosystem. Instead of relying on rigid keyword searches, EasySource identifies strong talent pools, screens for role-specific credentials and licenses and personalizes outreach by phone and email. For mid-market companies with 500 to 1,000 employees, that translates to a 70% faster time-to-hire and an estimated $100,000 in annual savings.

The integration also closes the feedback loop for employers. By feeding post-hire data back into the recruiting system, EasySource learns from successful hires to build smarter talent pools over time, while keeping recruiters engaged longer by freeing them to focus on the human side of hiring.

Gokul Rajaram, board member at Coinbase and former board member of The Trade Desk (Nasdaq: TTD), and also known as the godfather of Google AdSense, said, “Smarthveer is one of those rare founders who picks an unglamorous, deeply underserved market and refuses to leave until it’s fixed. Frontline hiring is exactly that market. Being named to Paylocity’s elite partner network is a testament to his relentlessness and to the team he’s built. Excited for what’s ahead.”

“Paylocity maintains an elite partner network, so being named one of them is a testament to the strength of our product,” said Smarthveer Sidana, founder and CEO of HireQuotient. “By keeping that recruiting activity inside the Paylocity ecosystem, we’re helping them close a critical gap for their employers, capture revenue that previously sat outside their platform, and prevent the client churn that comes with a fragmented hiring process.”

Jim Moffatt, former global CEO of Deloitte Consulting and board partner at Greycroft VC, said, 

“This is a big milestone for Smarthveer and his team. I congratulate them on this big win. Paylocity’s large client base acts as a strong distribution, and HireQuotient’s EasySource acts as a strong product to cater to the needs of Paylocity’s clients. Paylocity is known for its highly selective approach, and I’m glad to see that after months of evaluation and extensive due diligence, they’re going live with HireQuotient’s EasySource. I feel confident in the value this partnership will create for frontline industries. I remember when Smarthveer spoke to me about it in December, and it felt very ambitious. I’m glad to see it turn to reality.”

For employers with a traditionally deskless workforce in industries where AI adoption has lagged, the integration bridges a major capability gap by allowing them to source, screen, onboard and manage payroll all in one place.

For more information, visit www.hirequotient.com.

A media kit with additional partner quotes, executive headshots and logos can be found here.

About HireQuotient

HireQuotient is an AI-native recruiting platform that automates candidate sourcing and screening for employers in manufacturing, building services, construction, healthcare, insurance and other frontline-heavy industries. Its flagship platform, EasySource, is one of a select number of recruiting technologies integrated with Paylocity’s HR and payroll ecosystem. Founded by Smarthveer Sidana, HireQuotient is headquartered in San Francisco. For more information, visit https://www.hirequotient.com/.

Media Contact

Bethany Rhodes

Uproar by Moburst for HireQuotient

bethany@moburst.com

Contact

Bethany Rhodes

bethany@moburst.com

In a single month, one enterprise reportedly spent half a billion dollars on AI. A consultant told Axios that the client had handed its workforce AI licenses, set no usage limits and let the meter run until finance noticed. The figure is spectacular, and it is the wrong thing to fear. That half-billion-dollar accident is only the visible part of a quieter, far larger failure. Worldwide AI spending is forecast to reach $2.52 trillion in 2026, more than any technology category in a generation, and by the most cited measure, roughly 95 percent of it returns nothing. Boards read that as proof that the technology does not work. The evidence points somewhere less comfortable, and it is not a technology problem at all. Most boards cannot see it because they are reading the wrong number: They track failure when the number that matters is allocation. The discipline that separates the winners is not technical. It is how they allocate capital across time, and how willing they are to stop. The hardest discipline in the AI era is not adopting faster. It is allocating honestly and refusing to judge a three-year bet on a six-month cycle.

The number everyone quotes, and no one acts on

The headline statistic is now familiar. MIT’s Project NANDA, in its 2025 study The GenAI Divide, found that about 95 percent of enterprise generative AI pilots produced no measurable impact on the P&L, while roughly 5 percent captured nearly all the value. S&P Global Market Intelligence found that the share of companies abandoning most of their AI initiatives jumped from 17 percent to 42 percent in a single year, with the average organization scrapping 46 percent of its proofs-of-concept before production. Gartner expects more than 40 percent of agentic AI projects to be canceled by the end of 2027, citing escalating costs, unclear business value and inadequate risk controls. And the pattern predates generative AI: RAND found that more than 80 percent of AI projects fail, roughly twice the rate of comparable work that does not involve AI.

Read as a technology story, these numbers say AI does not work. Read correctly, they say something more useful. MIT’s own authors located the cause not in model quality but in a learning and integration gap. The winners were not running better models. They picked one problem, executed and worked well together. Purchased solutions reached production about 67 percent of the time, while internal builds succeeded roughly a third as often. Gartner’s own spending forecast notes the same pivot, with CIOs scaling back ambitious internal builds in favor of commercial solutions that promise more predictable value. None of that is a verdict on the technology. It is a verdict on allocation: What gets funded, for how long and against which yardstick. The popular prescription, heard in every boardroom this year, is to measure harder and prove value sooner. That advice quietly repeats the mistake, because forcing a three-year bet to prove itself sooner is precisely how you kill it. The fix is not more measurement. It is measuring each bet against the right clock and subtracting the ones that miss.

The six-month cycle problem

Return to that 95 percent, because the way it is measured is the whole argument. Much of the reported failure is judged on a short clock, with a pilot counted as a failure if it has not shown a measurable financial return within roughly six months. The single most quoted number in enterprise AI is therefore a six-month yardstick applied to every initiative, including the bets designed to pay back in three years. The headline failure rate is not only a measure of AI. It is a measure of impatience.

The most expensive mistake in enterprise AI is a timing error. Enterprises have been spending heavily on AI for more than two years, and 2026 is the year boards are demanding returns. The multi-year bets funded during the 2024 and 2025 scale-up are only now far enough along to be judged. When a board reviews an initiative, it applies the yardstick it knows, which is quarterly return. That yardstick is correct for an efficiency project and ruinous for a capability bet. A workflow automation that should pay back in two quarters and a foundational data and agent capability that pays back in three years are not the same instrument, yet they are reviewed in the same meeting against the same metric.

This is the heart of the divide. The 5 percent did not simply pick better projects. They judged each project against its own horizon. McKinsey’s enduring Three Horizons model made this discipline standard in corporate strategy a generation ago: near-term, emerging and long-term bets are funded and measured differently. AI erased that discipline because the hype compressed every timeline into the current quarter. The result is two failure modes that appear opposite yet share a common root. Organizations kill three-year bets at month six because they miss a metric the bet was never designed to hit. And they keep funding six-month theater for years because it is visible, safe and never asked to prove a return. Both are allocation failures. Neither is a technology failure.

Subtraction is a strategy

There is a second discipline, the 5 percent share, and it is the one boards find hardest. They subtract. Every credible study of the failure rate describes the same chaotic pattern underneath it: Initiatives are abandoned late, without criteria, after the money is spent and the credibility is gone. Disciplined organizations do the opposite. They decide the conditions for stopping before they start, and they stop on schedule. Subtraction is not the absence of strategy. It is the strategy. Capital removed from a failing bet is capital available for a surviving one, and the survivors are where the entire return lives.

This reframes the 42 percent abandonment figure. Abandonment is not the problem. Undisciplined abandonment is. An organization that liquidates a position the moment it breaches a pre-agreed kill line is practicing portfolio hygiene. An organization that lets a doomed pilot run until someone loses patience is paying full price for a lesson it could have bought at a discount. The 5 percent who won were not smarter. They were patient in the right places and ruthless in the wrong ones.

The HALT framework: Horizon, Allocation, Liquidation, Tracking

Treating AI as a portfolio rather than a pile of pilots requires four disciplines, and the organizations that execute well put all four in place before the next funding cycle, not after the next failure. The name is deliberate. The discipline most enterprises lack is the willingness to halt the wrong bets in time to fund the right ones.

Component 1: Horizon. Classify every AI initiative by its true payoff horizon before it is funded. Horizon 1 covers efficiency plays that should return value within two quarters. Horizon 2 covers capability bets, data foundations, agent platforms and integration work that pays back in roughly 6 to 18 months. Horizon 3 covers transformation bets that take eighteen months to three years or longer. Each horizon carries its own success metric, set at funding time. A Horizon 1 yardstick never judges a Horizon 3 bet. This single rule prevents the most common and most expensive error in the portfolio.

Component 2: Allocation. Decide the split across horizons deliberately, as a board-level capital decision, not as the accidental sum of whatever pilots happened to win approval. A practical reference point, borrowed from decades of innovation-portfolio practice, is roughly 70% to near-term value, 20% to capability, and 10% to transformation. The exact ratio is yours; the discipline is to choose and defend it. The failure mode is an unmanaged portfolio: 90 percent scattered across disconnected Horizon 1 experiments, with nothing compounding into the Horizon 2 capability that the buy-and-integrate winners actually built.

Component 3: Liquidation. Attach a kill line to every initiative at the moment it is funded: A named milestone, a date and an owner empowered to stop it. If a bet misses its horizon-appropriate milestone, it is liquidated, and capital is reallocated on schedule without debate over sunk costs. The absence of a pre-agreed kill line is not patience. It is an unpriced liability that the board has almost certainly not been shown.

Component 4: Tracking. Report the portfolio to the board on a fixed cadence using a single instrument: The AI Portfolio Scorecard. Not a deck of project updates, but a single view of allocation by horizon, burn against milestone, liquidation decisions taken and capital reallocated to survivors. The cadence is the control. A portfolio reviewed once a year is a portfolio managed by hope.

THE AI PORTFOLIO SCORECARD: SCORE EVERY INITIATIVE BEFORE IT IS FUNDED

Score each criterion: 0 = not present, 1 = partially documented, 2 = fully verified. Total out of 12. Bands: 0 to 4 = DO NOT FUND  |  5 to 8 = CONDITIONAL  |  9 to 12 = FUND.

THE LIQUIDATION GATE: RUN AT EVERY BOARD REVIEW BEFORE CONTINUING FUNDING

Any unresolved FAIL = stop funding, liquidate the position, reallocate the capital to a survivor and record the decision on the scorecard.

The cost of the timing error

The financial case follows the pattern and is consistent. Consider two organizations that funded the same class of Horizon 3 bet: A domain-specific agent platform meant to compound over three years. The first review was conducted at month six against a quarterly return test, found no payback and killed it, booking the write-off as a lesson about AI being overhyped. Its competitor classified the same work as Horizon 3, set an 18-month capability milestone, protected funding through two review cycles and shipped to production within the window the work actually required. One organization spent its money to learn that it lacks allocation discipline. The other spent comparable money and now owns a capability its rival has abandoned and cannot quickly rebuild. The dollars on the two income statements are similar. The competitive positions are not.

The governance return the board has been waiting for

Allocation discipline does two things at once. It stops the bleed by liquidating failures on a schedule rather than at the point of exhaustion. And it concentrates capital where the entire return lives, in the small number of bets that survive their horizon. The 5 percent figure is not a ceiling imposed by the technology. It is the current yield of an industry allocated by hype. An organization that classifies by horizon, allocates on purpose, liquidates on a line and tracks on a cadence is not trying to beat the technology. It is trying to beat its own indiscipline, and that is a far more winnable contest.

The board conversation about AI returns is coming for every organization, and it arrives the moment the spending outpaces the story. When it does, the CIO will be asked a simple question: Where did the money go? The leaders who can answer will not show a pile of pilots. They will show a portfolio: What was funded, against which horizon, what was liquidated and when, and what the survivors are now worth. Subtraction is a strategy. The only question is whether you are practicing it on purpose or about to learn it by accident.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

The head of the US government’s AI testing institute, Chris Fall, has resigned about three months after taking charge of the Center for AI Standards and Innovation (CAISI), the federal organization responsible for evaluating advanced artificial intelligence models for safety and security.

Current National Institute of Standards and Technology NIST Director Arvind Raman will serve as acting CAISI Director following Fall’s departure while continuing to oversee the Commerce Department office responsible for the institute, the Daily Signal reported, citing two people familiar with the matter.

A Commerce Department spokesperson who spoke to the publication did not disclose a reason for the resignation.

Fall assumed leadership of CAISI in April after the Trump administration reorganized the former US AI Safety Institute under NIST. The institute develops methodologies for evaluating frontier AI models and works with AI developers on voluntary technical assessments covering areas such as cybersecurity, model misuse, reliability and other risks associated with increasingly capable AI systems.

The leadership change comes as governments and AI companies continue developing technical approaches for evaluating frontier AI models while enterprises expand deployments of generative AI and agentic AI across business operations.

In recent months, the Commerce Department has taken a more active role in AI policy involving advanced models, placing greater attention on how the federal government evaluates technologies with potential national security implications.

Continuity matters more than personalities

CAISI works with AI developers such as Anthropic, Google’s DeepMind and OpenAI on voluntary evaluations of frontier AI models and develops methodologies for testing model capabilities and risks. The institute does not regulate AI developers or certify commercial AI systems.

For enterprises, those evaluations are one source of technical information alongside vendors’ own testing, third-party security assessments and internal AI governance programs.

Sanchit Vir Gogia, chief analyst at Greyhound Research, said enterprises should focus less on the individual leading the institute and more on whether its technical work continues with the same level of consistency and transparency.

“Leadership churn at CAISI weakens the signal long before it weakens the science,” Gogia said. “The testing has not stopped. Its authority simply does not travel as cleanly once the leadership does not.”

According to Gogia, the more important question for enterprises is not whether the institute’s evaluation work will continue but whether the processes supporting those evaluations remain stable.

“The instinct is to ask whether the pipeline is breaking,” he said. “The more useful question is where the pipeline now sits.”

Enterprises still carry the burden of AI governance

Gogia said organizations should continue treating government-led AI evaluations as one input into their governance processes rather than as evidence that a model is inherently safe for enterprise deployment.

“A government evaluation was always a signal, never a certificate,” he said. “A signal loses value the moment its issuer becomes unpredictable.”

He said enterprises should instead monitor whether CAISI maintains consistent evaluation methodologies, continues publishing technical findings and preserves continuity within its research teams under interim leadership.

“The name on the door is not the signal. The behaviour underneath it is,” Gogia said.

Gogia also cautioned against linking Fall’s resignation to recent Commerce Department actions involving AI policy or export controls, noting that there is no public evidence connecting the two.

“CAISI evaluates; it does not enforce export controls, because it holds no such power,” he said. “This is not a testing body reaching for enforcement. It is enforcement reaching past the testing body.”

With Raman assuming the role on an interim basis, the next significant milestone for enterprises will be the appointment of a permanent director, and whether the institute’s evaluation programs continue without disruption, the analyst said.

Gogia said the successor’s mandate may prove more important than the individual selected.

“A CAISI result is not a safe harbour,” he said. “It informs an obligation; it does not discharge one.” NIST did not immediately respond to a request for comment.

For the past three years, the AI conversation has been dominated by scale. Bigger models, bigger compute clusters, bigger headlines. But the next wave of competitive advantage won’t come from who can rent the biggest model; it will come from who can build the smallest one that knows their business.

That model is the small language model (SLM): Compact, purpose-built, trained on an organization’s own data and run under that organization’s own governance. And it is about to become one of the most consequential strategic assets available to both the private and public sector.

The problem with renting intelligence

Right now, most organizations consume AI the way they once consumed electricity from a single utility by plugging into a handful of frontier models built by a small number of global vendors. These models are extraordinary generalists. They are also, by design, generic. They are tuned to be safe, broad and useful to everyone, which means they are optimised for no one in particular.

That’s a problem for any organization trying to build genuine differentiation. If every competitor in your sector is calling the same foundation model with the same prompts, the model itself is not your edge. Your edge is what only you know, your proprietary data, your institutional judgement, your operating history. A generic model can’t see any of that unless you keep feeding it to them, turn after turn, at cost, with no lasting memory and no guarantee of where that data ends up.

An SLM flips that equation. Trained on an organization’s own document libraries, case histories, policy archives, transaction data and operational know-how, it becomes a model that thinks the way your organization thinks, because it was built from your organization’s accumulated judgement. It doesn’t need to be the smartest model in the world. It needs to be the most useful one for you.

I’ve seen this play out directly. At one of Australia’s largest integrated tourism and cruise businesses, simultaneously a B2C retailer, a B2B distributor to thousands of agency and wholesale clients globally, an aggregator marketplace for more than 1,800 independent tourism operators, and a cruise operator with offshore shared services spanning finance, customer contact and content management. The constraint wasn’t a lack of access to large general-purpose models. It was that none of them understood the business: 1,800 different operator catalogues, each with its own pricing logic, inventory quirks and content conventions; years of customer contact history with its own vocabulary and escalation patterns; a marketplace search experience that needed to reason over the business’s own product taxonomy, not the open web’s.

Models trained and tuned on that proprietary data, operator listings, historical tickets, booking and pricing data delivered results a generic model never could. Domain-tuned content drafting cut operator listing time by 70% and eliminated a 23-day onboarding backlog outright, taking new-operator time-to-live from 23 days to three. A semantic search model trained on the marketplace’s own product catalogue lifted booking conversion by 24%. AI-driven triage trained on the business’s own contact history cut Tier 1 escalations by 34%. None of this came from a smarter foundation model. It came from a smaller, more specific one that knew the business.

Why “small” is the strategic choice, not the compromise

There’s a temptation to treat SLMs as the budget option, what you build when you can’t afford a frontier model. That’s the wrong frame. The evidence is already compelling: Microsoft’s Phi-4 family of small models, released in early 2025, demonstrated that a 14-billion-parameter model can match or exceed the performance of models many times its size on complex reasoning and domain-specific tasks while running at a fraction of the compute cost and on-premise, entirely within an organization’s own infrastructure. Smaller, domain-trained models are increasingly outperforming general-purpose giants on narrow, high-value tasks, with far tighter control over data residency, security and explainability.

For a CIO or CTO, that combination of lower cost, tighter governance, higher task-specific accuracy is rare enough to demand attention on its own. But the deeper value sits one layer up, at the operating model. An SLM trained on your service history can sit inside claims processing, citizen services, clinical triage, asset maintenance scheduling or M&A due diligence quietly compounding institutional knowledge into a reusable asset rather than letting it walk out the door every time someone retires or resigns.

That is the real shift: AI capability stops being a subscription and starts being a balance-sheet asset. It can be valued, protected, audited and improved because it belongs to you.

The public sector’s hidden advantage

Nowhere is this more obvious than in government. The public sector sits on some of the richest, least-exploited data and institutional knowledge in the country: Decades of policy outcomes, service delivery history, regulatory precedent, infrastructure records and frontline expertise. Most of it has never been put to systematic use because no commercially available model was ever trusted to touch it, and rightly so.

A small, sovereign, purpose-built model changes that calculus. Trained, hosted and governed entirely within government infrastructure, an SLM doesn’t require sensitive citizen or policy data to leave a secure perimeter. The Australian Government has already recognised this direction: The APS AI Plan, released in November 2025, commits to expanding the GovAI platform to provide all public servants with secure, sovereign AI tools operating entirely within Australian Government infrastructure. SLMs tuned to individual agency mandates are the logical next step and a more powerful one than any generic government-wide tool can deliver.

Rather than each agency independently negotiating with the same handful of overseas vendors, a coordinated approach of common standards for model governance, shared security architecture, common evaluation frameworks and pooled infrastructure investment would let agencies build and reuse SLM capability horizontally, the way shared services and common ICT platforms have been built before. Each agency gets a model genuinely tuned to its mandate, but the security model, audit trail and assurance framework are consistent, government-backed and independently verifiable.

Done well, this isn’t just an efficiency play. It’s a sovereignty play. As GovTech Review has noted, large language models hosted offshore create data flows that extend beyond Australia’s borders in ways that are rarely transparent, a risk that is simply untenable for government. Sovereign, purpose-built models keep Australian public data, public knowledge and the resulting capability uplift inside Australian hands, rather than exporting both the data and the long-term value to offshore platforms.

Why this belongs in the innovation budget, not the IT budget

The instinct in many organizations is to treat AI spend as an IT line item, something to be minimised, benchmarked and squeezed for cost efficiency. SLMs deserve a different treatment. They are closer to R&D than infrastructure: An investment in converting accumulated institutional knowledge into a durable, defensible capability.

That argument holds in the private sector too. A PE-backed portfolio company, a regulated financial services firm, a healthcare provider — each has years of proprietary operating data sitting idle in case files, transaction logs and service records. An SLM built on that data is a way of turning a sunk cost, decades of operational history, into a forward-looking asset that compounds with every additional case it processes.

Boards and executive committees that are still asking “what is our AI strategy?” as a single, undifferentiated question are asking the wrong thing. The better question is: Which parts of our operation are rich enough in proprietary data and judgement to justify owning the model outright, rather than renting someone else’s?

The opportunity in front of us

The first wave of enterprise AI adoption was about access: Getting a capable model into people’s hands quickly. The next wave will be about ownership: Who controls the model, who controls the data it was built on, and who captures the long-term value of the institutional knowledge it encodes.

Australia, with a public sector rich in data and a private sector with deep vertical expertise in financial services, resources, healthcare and logistics, is well placed to lead on this if it treats small, sovereign models as a genuine national capability question, not a procurement footnote. The organizations, and the country, that move early will not just save money. They will own something their competitors can’t easily replicate: An AI that knows them.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Something interesting happened during an internal evaluation of AI meeting transcription tools at Tungsten Automation. The products worked. They weren’t bad. But sitting across from the pricing, we kept asking the same question: what exactly are we paying for?

We already had a secure enterprise AI environment. Building a meeting summary workflow took days, not months. We customized the outputs, injected our own internal context, and controlled security our way instead of working around someone else’s roadmap. We built it. It works better. We own it.

That’s not a knock on those vendors. It’s a signal of something more fundamental happening across enterprise software.

The moat was never the product

For two decades, SaaS rode a favorable asymmetry: building internal tools was hard, integrations were messy, and even modest automation required developers and long timelines. Buying was faster and cheaper than building. That asymmetry fueled the explosion of SaaS into every corner of the enterprise stack.

AI is collapsing that asymmetry. Large language models and agentic workflows can orchestrate APIs, move data between systems, generate interfaces, and automate business logic with a fraction of the engineering effort required even two years ago. The integration friction that once protected entire product categories is evaporating.

The vendors most exposed are not the deeply embedded enterprise platforms. They’re the lightweight workflow layers, the products that essentially put a polished interface on top of accessible data and relatively straightforward processes. Reporting dashboards. Meeting tools. Narrow productivity applications. These products created value by simplifying implementation. That rationale is getting harder to sustain when implementation is no longer the real barrier.

Here’s the part most analyses miss: it’s not just that AI makes development faster. It’s that agents change the integration model entirely. For 30 years, enterprise software was built for humans navigating UIs. Agentic systems don’t use UIs. They call APIs, read from multiple sources simultaneously, and move data freely across systems. The switching costs that once made incumbent software sticky are collapsing, because an agent doesn’t care which UI it used last quarter.

The SaaS that survives

The question isn’t whether SaaS survives. It’s which SaaS survives.

The companies with durable positions are not the ones with the cleanest interface. They’re the ones that transfer operational risk customers genuinely cannot absorb themselves. Compliance. Regulatory certification. Accumulated domain expertise. Liability.

Think about compliant invoicing across 140 countries. That’s not a workflow someone builds in a sprint. The certifications alone take years. A single regulatory change in one jurisdiction can break an AP process for a global enterprise overnight. Customers don’t pay for that capability because it’s technically complex. They pay because they cannot afford to own the risk of getting it wrong.

That’s the distinction that matters: AI lowers the cost of building software. It does not lower the cost of absorbing risk. The vendors who understand this are building durable businesses. The ones who don’t are quietly subsidizing their customers’ internal build programs.

Software sells features. Platforms sell accountability.

The prototype trap

The danger for enterprise buyers right now is overcorrection. Every successful prototype looks like a cost-saving opportunity. Very few survive the jump to production.

Building a workflow with generative AI is becoming straightforward. Maintaining it is not. Models evolve. Outputs drift. Governance requirements tighten. What worked cleanly in a controlled environment behaves differently at scale, and the failure mode is worse than traditional software. Rule-based automation, when it fails, fails obviously. Agents fail silently, confidently, at scale, often with a completely reasonable-sounding explanation.

Engineering teams that take on AI-powered systems need to solve for observability, model drift, access controls, audit trails, and long-term maintenance ownership. In regulated industries, they need to demonstrate exactly how the system reached every decision. That’s not a weekend project. That’s an ongoing operational commitment that compounds over time as models change and regulatory requirements evolve.

Before a team decides to replace an external platform with internal AI tooling, the honest question isn’t, “Can we build this?” The real question is, “Are we prepared to own this in production, for years, as the underlying models change beneath us?” Sometimes the answer is yes. Often the answer is no, and the true cost only becomes visible after the vendor contract is canceled.

Build vs. partner: a sharper frame

The build vs. buy framing has always been too binary. The right question is build vs. partner.

Partner for the capabilities where risk transfer, regulatory complexity, and domain expertise create genuine value your team cannot replicate. Build for the capabilities that actually differentiate your business from your competitors. Don’t burn your best engineers rebuilding compliant invoice processing or production-grade document extraction. Those aren’t competitive advantages. They’re table stakes, and someone else has already paid the cost, across decades, to make them reliable.

The organizations getting this right are honest about where they create unique value. They focus development there, and partner for everything else. The ones getting it wrong are vibe-coding solutions to non-differentiating problems while their actual competitive moat goes unattended.

The true value of software

We’re not watching the death of SaaS. We’re watching the end of the friction-based value proposition: the idea that software is worth renewing because integration used to be painful. That rationale is largely gone.

What survives is software that does something customers cannot reasonably replicate internally: absorb risk, maintain regulatory compliance, deliver operational reliability at scale, and bring genuine domain expertise into a production-grade system that someone else already stress-tested for years.

The vendors who recognize this are already repositioning around accountability, governance, and outcomes. The ones who haven’t will find the next renewal conversation noticeably harder.

Software sells features. Platforms sell accountability. That distinction is about to separate a lot of winners from a lot of cautionary tales.

—

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.

The emergence of artificial intelligence is beginning to expose a profound weakness in the way modern enterprises measure performance.

For decades, business evaluation systems have been built around the logic of the industrial and transactional economy. Revenue growth, operating margins, earnings per share, labor productivity, return on investment and market share became the dominant indicators of organizational success because they reflected the economic realities of a world in which value creation was primarily tied to physical production, labor efficiency, scale and later the automation of information processing. AI, however, is altering the very structure of enterprise value creation, and in doing so it is creating a widening separation between perceived future value and actual realized economic performance.

Much of the current discussion surrounding AI performance measurement reflects this tension. The overwhelming majority of AI-related metrics being celebrated today are not direct measures of realized enterprise outcomes. They are largely indicators of capability formation, market positioning, experimentation or investor signaling. Metrics such as AI spending levels, number of AI use cases, GPUs deployed, copilots implemented, models placed into production, AI hiring growth or agentic AI pilots all serve primarily as proxies for anticipated future advantage. These indicators may influence stock valuations, analyst sentiment and strategic narratives, but their relationship to measurable operational performance is often indirect, delayed or in some cases entirely speculative.

This distinction is critically important because capital markets have historically rewarded the expectation of technological transformation long before actual economic results materialized. During previous technological revolutions—including electrification, enterprise resource planning, the internet, cloud computing and mobile platforms—valuation expansion frequently preceded measurable productivity gains by many years. The market priced future possibility before operational economics caught up. In many instances, investors rewarded firms simply for appearing strategically aligned with the dominant technological shift of the era. AI appears to be following a similar trajectory.

The phenomenon resembles the famous productivity paradox articulated by economist Robert Solow, who observed that “you can see the computer age everywhere but in the productivity statistics.” AI today is visible everywhere: in investor presentations, earnings calls, technology conferences, product announcements and boardroom strategies. Yet in many industries, its measurable contribution to enterprise productivity, profitability or economic resilience remains difficult to isolate with precision. This does not necessarily mean AI lacks value. Rather, it reflects the reality that traditional accounting and performance systems were never designed to measure the forms of value AI increasingly produces.

Artificial intelligence creates benefits that are often diffuse, cumulative and difficult to attribute directly to financial outcomes. AI may improve forecasting accuracy, reduce fraud, accelerate decision cycles, augment employee effectiveness, improve customer interactions, optimize logistics or enhance cybersecurity resilience. These benefits frequently manifest as second-order effects distributed across the enterprise rather than as immediately visible financial events. The causal chain between AI investment and realized business performance can therefore become extraordinarily difficult to quantify. A company may become operationally more intelligent without immediately becoming measurably more profitable.

At the same time, AI introduces a profound danger: organizations may increasingly optimize for technological narrative rather than durable enterprise economics. Many firms today are pursuing AI primarily because markets reward the appearance of AI leadership. Investor enthusiasm, analyst pressure and competitive fear create incentives to demonstrate visible AI activity regardless of whether measurable economic value has actually been achieved. In this environment, AI metrics can easily become instruments of valuation signaling rather than instruments of operational truth.

This distinction between signaling and substance may become one of the defining economic challenges of the AI era. An organization may announce aggressive AI deployment programs, reduce headcount and report short-term margin improvements while simultaneously increasing hidden forms of technological fragility. Infrastructure costs may rise dramatically as GPU consumption, cloud usage, data engineering requirements and cybersecurity complexity expand. Technical debt may accelerate as AI-generated code proliferates without sufficient architectural discipline. Institutional knowledge may erode as organizations become excessively dependent on opaque models and automated systems. Long-term innovation capacity may weaken if enterprises divert disproportionate resources toward maintaining internally generated AI systems rather than building new strategic capabilities.

What measuring AI value might actually look like

The distinction between AI activity and AI value becomes clearer when viewed through the kinds of measures organizations choose to track. Many enterprises today emphasize indicators such as the number of AI models deployed, copilots implemented, agents created, prompts executed, tokens consumed or employees using AI tools. These metrics demonstrate adoption and technological activity, but they reveal relatively little about whether AI is producing meaningful business outcomes.

Measures of enterprise value look quite different. A manufacturer might evaluate whether AI improves demand forecasting accuracy enough to reduce inventory carrying costs or stockouts. A financial institution might measure whether AI meaningfully lowers fraud losses, accelerates loan processing or improves regulatory compliance. A healthcare provider could assess reductions in administrative burden, faster clinical decision support or improvements in patient throughput. In each case, the objective is not simply to measure AI deployment, but to determine whether AI creates measurable improvements in operational performance, economic outcomes or organizational resilience.

Ultimately, organizations may need to ask a different question: not “How much AI are we using?” but “How much business value does each unit of AI investment create?” That shift—from measuring technological activity to measuring economic outcomes—may become one of the defining management disciplines of the AI era.

Under traditional accounting frameworks, many of these deteriorations remain largely invisible. Quarterly earnings may improve even as underlying enterprise resilience declines. Stock prices may rise even as operational complexity becomes increasingly unsustainable. In this sense, the AI era threatens to widen the gap between financial appearance and organizational reality.

This is why the future of enterprise measurement cannot simply involve adding AI metrics to existing financial scorecards. The challenge is far deeper. AI forces a reconsideration of what business performance actually means. Historically, enterprises were measured largely through static indicators of efficiency and output. Increasingly, however, competitive advantage may depend less on traditional efficiency and more on adaptive intelligence: the ability of an organization to learn faster, make better decisions, integrate human and machine capabilities effectively, manage technological complexity sustainably and convert computational power into durable economic outcomes.

The most important future performance measures may therefore revolve around questions traditional accounting rarely addresses. How effectively does an enterprise convert technology investment into sustainable business capability? How economically efficient are its AI operations relative to the value they generate? How resilient is the organization to AI failure, cybersecurity disruption or infrastructure inflation? How successfully does it preserve and amplify human expertise rather than simply eliminate labor? How rapidly can it learn, adapt and operationalize new knowledge?

These are not merely technology questions. They are questions of enterprise economics, organizational sustainability and long-term competitive viability.

The companies that ultimately succeed in the AI era may not be those with the largest AI budgets, the greatest number of pilots or the most aggressive automation programs. They may instead be the firms that best understand the economics of technological capability itself: organizations capable of balancing innovation with resilience, automation with human augmentation and technological ambition with sustainable operational design.

The coming decade is therefore likely to produce a widening divide between enterprises optimizing for AI-driven valuation narratives and enterprises optimizing for measurable, durable economic performance. In the short term, these may appear to be the same thing.

Over time, however, the distinction will become increasingly visible. Some organizations will discover that AI has enhanced genuine enterprise capability. Others will discover that they merely optimized the appearance of transformation while silently accumulating new forms of economic and operational risk.

Artificial intelligence is not simply changing business operations. It is exposing the inadequacy of many of the measures used to evaluate business success itself. The central challenge of the AI economy may ultimately become not whether organizations adopt AI, but whether they can distinguish between technological activity and actual economic value creation.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

In one ransomware response I observed, the master operational dashboard remained green while the underlying environment told a very different story. It was a classic example of what we in the IT audit profession call the “watermelon effect”—green on the outside, red on the inside.

Beneath that dashboard sat an unmapped web of legacy technical debt, undocumented service accounts and shadow cloud instances. For years, presenting a green dashboard to the audit committee could give technology leaders a false sense of comfort. If a catastrophic breach occurred, it was generally treated as an unpredictable operational tragedy, managed via cyber insurance, a carefully calibrated public relations pivot and perhaps a quiet executive transition.

Today, that corporate shield is thinner than many technology leaders assume. For technology leaders in regulated or public-company environments, executive exposure is no longer only a theoretical debate. The regulatory environment has made plausible deniability much harder to sustain.

The erosion of the corporate shield

With the application of the European Union’s Digital Operational Resilience Act (DORA) for financial entities, alongside the broader NIS2 Directive for essential and important entities, cybersecurity governance has become harder to separate from board-level oversight. DORA places ultimate responsibility for ICT risk management on the management body of financial entities, while NIS2 requires management bodies to approve and oversee cybersecurity risk-management measures. In the United States, the U.S. Securities and Exchange Commission’s cybersecurity disclosure rules require public companies to disclose material cyber incidents and describe their cyber risk management, strategy and governance in annual filings. The new burden is not simply to operate controls; it is to show, after the fact, that leadership decisions matched the risk evidence available at the time.

The serious risk to a modern CIO is not simply the occurrence of a sophisticated security incident. The true danger is the inability to reconcile what leadership presented externally to investors, regulators and the board with what the internal evidence showed inside the environment.

When a serious crisis breaks, you may find yourself surrounded by corporate defense counsel, regulatory investigators and outside forensic lawyers all asking variations of the same uncomfortable questions: What did you know, when did you discover it and what specific actions did you take next?

When those questions are asked, a slide deck asserting that your security posture is “aligned with industry best practices” will not be enough. A post-incident review may recognize that sophisticated attacks occur. What creates greater exposure is evidence that known risks were ignored, understated or left outside structured governance. To survive that level of post-incident review, one of your strongest assets is a disciplined, independent evidence trail showing that risks were identified, challenged, escalated and acted on before the first indicator of compromise appeared.

Why point-in-time comfort letters fail regulatory scrutiny

The reality we face is that legacy compliance evidence often falls short under regulatory scrutiny. For years, the annual SOC 2 Type II report or a standardized ISO 27001 certification was brandished by technology teams as the definitive proof of a functional control environment. I have sat in dozens of scoping meetings where an engineering director pointed to a freshly minted compliance report as if it were a complete defense against scrutiny.

But a compliance report is a historical artifact—a retrospective evaluation of how specific controls operated during a defined window of time months in the past. It tells an investigator that on a random afternoon in Q2, your production change-management approvals conformed to a baseline policy. It says absolutely nothing about the configuration drift, unauthorized API keys or emergency patch bypasses that developers introduced the following weekend to hit a product release deadline.

Modern regulators, boards and investors are no longer satisfied by historical comfort letters alone. Under contemporary frameworks, especially regimes focused on operational resilience, static compliance evidence is no longer enough. The expectation of due care has shifted from a passive state of compliance to an active state of continuous challenge. Increasingly, post-incident reviews look for evidence that leadership identified system vulnerabilities, formally escalated material deficiencies, evaluated systemic risk to the business and tracked remediation progress with measurable rigor.

When an architecture fails, post-incident reviews often focus quickly on ownership, escalation and whether known risks were acted upon. If your defensive documentation consists entirely of static policy documents and green dashboards, you leave an evidentiary vacuum that can invite difficult questions about executive oversight. Post-incident reviews rarely turn on perfection. They turn on whether the organization can show a traceable chain of governance.

5 non-negotiable artifacts for your executive evidence engine

This reality requires a complete reframing of your relationship with your IT audit department. Historically, this dynamic has been defined by friction. Technology leaders frequently view my peers and me as compliance traffic cops—bureaucrats who interrupt core engineering sprints to demand evidence samples, user access reviews and system configurations.

It is time to view IT audit through a pragmatic lens: we are your independent evidence engine. We are one of the few corporate functions tasked with independently challenging your control environment, documenting where exceptions were escalated and showing how management responded. When an auditor identifies a control gap and partners with you to draft a management action plan, they are not creating a bureaucratic roadblock. They are helping you construct an evidence trail that can show risk was identified, escalated and acted upon.

To transform your IT audit function into an effective executive shield, you must shift focus away from superficial check-the-box exercises and collaborate on specific artifacts. The most effective exercise you can run with your audit leadership is to flip the timeline completely and ask: if this program were reviewed six months from now, which evidence would show we governed the risk before it failed?

1. Board-facing risk registers with escalation history: A risk register that sits unreviewed on an intranet page for 12 months is not a management tool; to an investigator, it can look like evidence that known risks were not actively governed. Your material technology, cybersecurity and dependency risks must be centrally logged. More importantly, this artifact must contain a clear, chronological escalation history showing exactly when the risk was presented to leadership committees and the board, along with related minutes, decisions or follow-up actions.
2. Granular risk acceptance records: You cannot remediate every vulnerability instantly. Business continuity, legacy software limitations and budgetary boundaries require you to accept certain operational exposures. When this occurs, ensure your risk acceptance records are airtight. A defensible record must document the specific technical variance, the precise financial or operational rationale for the delay, a definitive expiration date, explicit executive sign-off and the active compensating controls deployed to reduce the blast radius in the interim.
3. Tabletop and operational simulation records: Independent frameworks such as ISACA’s Digital Trust Ecosystem Framework can help structure this evidence, but boards and regulators will still look for proof that the testing actually happened. Your audit trail should contain comprehensive records of cyber incident, disaster recovery and third-party dependency simulations. These records must detail the scenario tested, the executive participants, the control failures identified during the drill and a formalized tracking schedule showing when those gaps were closed.
4. AI governance inventories and data-flow mappings: The rapid deployment of generative AI tools across enterprise operations has created a massive blind spot for technology executives. In one audit, we found developers using an unapproved public large language model to accelerate debugging with sensitive internal code. To protect yourself, work with your audit team to build an active enterprise AI inventory that maps data lineage, identifies model business owners, documents risk classification approvals and demonstrates active technical monitoring for unauthorized data exfiltration.
5. Synchronized disclosure-control handoffs: When a material security incident or system outage occurs, the clock begins ticking for regulatory reporting. Your incident response playbook must be technically linked to your corporate disclosure controls. The audit trail should show that a documented, synchronized handoff occurred between your technical response leaders, general counsel, chief financial officer and corporate communications team. This evidence helps show that your external statements match internal technical realities.

In the modern corporate ecosystem, technology leadership is no longer just an engineering challenge; it is an exercise in rigorous, evidence-based governance. The regulatory landscape has changed, and the expectation of continuous traceability cannot be avoided.

Open and direct collaboration with your IT audit team will not prevent a zero-day exploit, an unexpected cloud outage or a critical third-party vendor failure. That is not the purpose of enterprise risk management.

The true value is far more practical: when a serious incident puts your program under review, you will not be forced to defend your reputation with a feeling, an unverified assumption or a misleadingly green dashboard. Instead, you will have an independent record showing that risk was actively seen, appropriately challenged, properly escalated and responsibly managed. In today’s regulatory environment, that disciplined trail of evidence may be the difference between a failure that can be explained and one that begins to look negligent.

.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

For three decades, enterprise software has been a buy-it decision. Packaged software from SAP, Oracle and Salesforce covered roughly 80% of requirements at a fraction of the cost of building. The economics were obvious, and for traditional applications, they still are.

AI is introducing a wrinkle that is forcing even the most committed enterprise software customers to rethink their options. AI is a layer that sits across your data, your processes, and your decisions. Where that layer runs and who controls it is an architecture question, and most of the enterprise community is still treating it as a procurement one.

The appeal of vendor-embedded AI is clear: automated operational decisions, smarter supplier and merchandising choices, and friction-free workflows built into the systems enterprises already rely on. The catch is that these capabilities almost universally depend on your data living in the vendor’s cloud environment. For most large enterprises, it sits on-premises, in hyperscale cloud infrastructure they manage themselves, or in private data centers. That gap between where your data is and where your vendor’s AI assumes it should be creates a fundamental strategic fork in the road.

Build vs. buy is a category error

The framing I keep hearing is “build vs. buy your AI strategy.” It implies that some organizations are out there training foundation models from scratch. Nobody serious is doing that. The real choice sits across three distinct approaches, and conflating them leads to poor decisions:

• Buy embedded. Use the AI capabilities your vendor ships natively inside their platform: the assistant baked into your ERP, your CRM, your HCM suite. Lowest integration cost, fastest time to value, tightest fit with the application data.
• Buy platform. Adopt the vendor’s AI infrastructure layer and build your own assistants and agents on top of it. More flexible, but you remain inside the vendor’s architectural boundary and subject to their governance model.
• Compose. Connect a third-party model (Claude, GPT, Gemini, an open-weight model running in your own environment) directly to your existing landscape. Maximum control, maximum integration burden, and full responsibility for what comes out the other end.

These are not equivalent options at different price points. They make different assumptions about where your data lives, who governs the AI, and how much architectural change you’ll absorb to get there. Vendor pitches sometimes blur the distinction on purpose. Enterprise leaders can’t afford to.

The vendor AI stack has an assumption baked in

Every embedded AI capability ships with an unstated architectural prerequisite: your data must be where the AI can see it, in the shape it expects, under the governance the vendor enforces.

For organizations with clean, modern cloud estates, that is often a reasonable trade. For the long tail of large enterprises running heavily customized environments on private or hybrid infrastructure, that trade becomes a precondition, one you must meet before the AI conversation can even begin. Whether meeting it makes sense depends on your starting point, your sector’s regulatory posture, and your appetite for migration risk. None of those are uniform across organizations.

That’s the part that gets glossed over in vendor keynotes. The AI demo on stage assumes a destination architecture the audience hasn’t necessarily reached yet. Large enterprise customers are carrying an unusually heavy technology burden right now. Many are simultaneously managing platform modernization programs that have been building for over a decade, alongside pressure to migrate to vendor-managed cloud infrastructure. Sitting above both is a boardroom-level directive to demonstrate meaningful AI progress fast. The vendor path to AI and the boardroom path to AI can diverge sharply, and enterprises need to make selective, strategic decisions about where to adopt AI first to maximize value and minimize risk.

Sovereignty isn’t a slogan, it’s an architecture constraint

The conversation about sovereignty has been hijacked by both sides. One camp treats every SaaS adoption as a sovereignty violation. The other dismisses every sovereignty concern as Luddite resistance. Neither is useful.

What’s happening in real customer conversations – particularly in DACH, public sector, and financial services – is more specific. Organizations are drawing a distinction between running their applications in a vendor’s cloud (which is broadly fine, well understood, decades of precedent) and enriching their data and processes inside a vendor’s AI model (which has less precedent, is harder to reverse, and carries material implications for competitive position).

Enriching your data inside a vendor’s AI model is the genuinely new question, and organizations that conflate it with their existing cloud posture tend to defend the wrong perimeter.

Despite spending around $100 million annually with Amazon, Disney built its own internal AI system to house its corporate intelligence rather than rely on a hyperscaler’s AI offering. The decision came down to control. When your data represents decades of creative and commercial IP, you think carefully about where it lives and who can learn from it. Disney has become more open to SaaS over time. The AI sovereignty question is a separate debate from the SaaS debate and conflating the two leads organizations to the wrong conclusions.

At the other end of the spectrum, enterprises in heavily regulated environments treat data sovereignty as an absolute non-negotiable. Any AI model must run within their controlled environment, especially where sensitive data cannot touch the public internet. GDPR obligations reinforce this instinct across the European market, requiring organizations to maintain clear accountability for how personal data is processed inside AI systems, including vendor-managed ones.

AI-enriched data, meaning models that have learned the shape of your business processes, your supplier negotiations, your customer behavior, carries a different half-life and a different strategic value than the operational data underneath it. That deserves its own architectural decision, separate from your broader cloud strategy.

What this means in practice

Most large enterprise estates will end up with a mix of all three approaches, and where you draw the lines matters more than your overall posture.

Embedded AI capabilities are the right answer for in-application productivity: the assistant inside your ERP workflows, the agent inside your procurement or HR suite. That is where vendor embedding genuinely shines, and attempting to compose your own equivalent is typically a poor use of engineering resources.

Compose belongs elsewhere: in cross-application orchestration, in custom assistants over operational and observability data, and in agents that need to reach across multiple vendor systems and infrastructure layers in ways no single vendor stack will never natively support. Research from McKinsey suggests the most significant near-term productivity gains from enterprise AI will come precisely from these cross-system workflows, rather than from within individual applications. The most interesting enterprise AI work over the next eighteen months lives here, and it doesn’t require waiting for a migration to complete first.

That compose path isn’t free, and it’s important to be honest about the costs. Governance, audit trails, and accountability for hallucinated outputs become your problem, not the vendor’s. Prompt drift and evaluation discipline are real engineering costs that never appear in the proof-of-concept. Those costs scale with the complexity of your landscape and the number of systems your agents touch. Budget for them before deployment, not after your first production incident. None of that is a reason to avoid the path. It’s a reason to staff for it, honestly.

The real question

The build-vs-buy frame survives because it gives executives a binary choice along a familiar axis. AI sits somewhere else entirely.

The question worth putting on the table at your next architecture review is simpler:

Which decisions do we want our vendors’ AI to make, and which do we want to keep on our side of the boundary?

Answer that, and the right build/buy/compose mix flows from it. Skip it, and you will end up with the architecture your vendors prefer – which may or may not be the one your business needs.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

DC Studio | shutterstock.com

KI-Agenten und die Large Language Models (LLMs), auf denen sie basieren, haben ein eher kurzlebiges „Gedächtnis“. Das ist so gewollt, schließlich kann nur eine begrenzte Menge an Konversationsinhalten in Token kodiert und vom LLM zuverlässig abgerufen werden. Um KI-Agenten und Sprachmodelle mit „Hirnschmalz“ auszustatten, das über ihre Kontextfenster hinausreicht, lässt sich Retrieval Augmented Generation (RAG) einsetzen. Erfolgsentscheidend ist dabei, wie dieser Mechanismus (oder ein anderer, um Gesprächsdaten vorzuhalten) konkret zur Anwendung kommt.

Ein anderer Weg, sowohl KI-Agenten als auch LLMs mit erweiterten Speicherfähigkeiten auszustatten, führt über Software-Tools von Drittanbietern. Diese können die KI mit einer Session-übergreifenden, persistenten Memory ausstatten. Auch hier variiert jedoch die Art und Weise, wie das technisch umgesetzt wird. Die folgenden vier Projekte sind besonders empfehlenswert, wenn es darum geht, KI-Agenten und Sprachmodelle smarter zu machen.    

1. Graphiti

Graphiti wird als „das Open-Source-Framework für temporale Knowledge-Graphen“ beworben. Das Projekt ist auf GitHub verfügbar – oder auch im Rahmen des Memory-Service Zep, für den es die Grundlage liefert. „Temporal“ bedeutet in diesem Zusammenhang, dass die in Graphiti gespeicherten Informationen im Laufe der Zeit reevaluiert werden, um den Kontext korrekt einzubetten. Der Begriff „Graph-Framework“ ist hingegen darauf zurückzuführen, dass die Daten dabei als eine Reihe von Graphen gespeichert werden. Dieses Feature spielt auch bei den anderen in diesem Artikel vorgestellten Lösungen eine Rolle – im Fall von Graphiti steht es allerdings im Fokus.

Out of the Box unterstützt das KI-Memory-Projekt eine ganze Reihe gängiger LLMs, etwa von Anthropic, OpenAI, Google oder X. Auch sämtliche Ollama- und OpenAI-kompatiblen APIs funktionieren mit Graphiti – es kann also auch mit lokal gehosteten LLMs genutzt werden. Daten aus Quellen wie GitHub, Gmail und OneDrive sowie aus Anwendungen wie Notion lassen sich über Konnektoren einbinden.

Um Graphiti lokal nutzen zu können, ist es allerdings nötig, eine Graphdatenbank einzurichten oder eine Verbindung zu einer solchen herzustellen. Die Standardlösung dafür (mit dem breitesten Support) ist Neo4j. Davon abgesehen, funktionieren auch Amazon Neptune, FalkorDB und KuzuDB. Postgres mit pgvector ist (derzeit) hingegen keine Option bei Graphiti.

2. Hindsight

Das KI-Memory-Projekt Hindsight als Cloud Service verfügbar, kann jedoch auch lokal gehostet werden. Dieses Tool speichert Details zu Agenten-Sitzungen in vier verschiedenen Memory-Instanzen und wendet dabei vier unterschiedliche Storage- und Retrieval-Strategien an. Diese werden über drei programmatische Interfaces gehändelt:

• retain, um Inhalte (einzelne Fakten oder komplette Sessions) zu speichern,
• recall, um den Content abzurufen, und
• reflect, um einen Agenten-Loop über eine Abfrage zu initiieren, die zuvor gespeicherte Daten nutzt.

In Sachen Integrationen hat Hindsight eine breite Palette von First- und Third-Party-Optionen zu bieten. Wenn Sie beispielsweise die „Continue“-Erweiterung mit Visual Studio Code einsetzen, um mit einem lokal gehosteten LLM zu kommunizieren, können Sie die entsprechende First-Party-Integration nutzen. In diesem Fall verwenden Sie einfach das Keyword @hindsight in der Query, um den Agenten-Kontext um relevante Memory zu erweitern. Um sich die Arbeit zu erleichtern, respektive diese zu automatisieren, könnten Sie außerdem auch auf (anpassbare) Auto-Injection-Regeln zurückgreifen.

3. Mem0

Wie Hindsight nutzt auch Mem0 vier grundlegende Memory-Typen – allerdings sind diese anders benannt und organisiert. Beispielsweise kommt im Fall von Mem0 die sogenannte „Organizational Memory“ zum Einsatz, um Daten zu speichern, die zwischen verschiedenen KI-Agenten(-Teams) geteilt werden sollen.

Jede Form von Memory, die über Mem0 hinzugefügt wird, durchläuft einen „Destillationsprozess“ und wird auf unterschiedliche Art und Weise (Vektor-, Graph- oder SQL-Datenbank) gespeichert. Ältere Daten werden bei Mem0 nicht gelöscht, sondern als veraltet markiert – eine Strategie, um einen umfassenderen, längerfristigen Kontext zu erzeugen.

Das Projekt unterstützt im Vergleich – etwa zu Hindsight – weniger LLMs, die wichtigen Anbieter (Anthropic, Google, OpenAI) sind jedoch vertreten. Dazu kommen Self-Hosting-Optionen über LangChain, LiteLLM, LM Studio und Ollama. Falls Sie Mem0 lokal statt als Service nutzen möchten, ist es nötig, eine Python-Instanz und eine eigene Vektordatenbank bereitzustellen. Für Letzteres ist Postgres mit der pgvector-Erweiterung eine gängige und simple Option, die sogar innerhalb einer virtuellen Python-Umgebung installiert werden kann.

4. Supermemory

Supermemory erfasst Daten aus vielen gängigen Quellen und unterstützt dabei unter anderem Plaintext, strukturierte Daten, PDF- und Office-Dokumente sowie Video-, Audio- und Bilddateien. Aus diesen Informationen erstellt das Tool einen Kontextgraphen, der anschließend als Grundlage für Chatbot-Konversationen fungiert. PR-mäßig setzt dieses Projekt den Fokus vor allem auf seine Context-Extraktions-Tools.

Supermemory ist entweder als Cloud-Dienst oder als quelloffene, lokal ausführbare Software verfügbar. Die Open-Source-Version lässt zwar die Scaling Services und Drittanbieter-Konnektoren der Enterprise-Version vermissen – hat jedoch einen entscheidenden Vorteil: Sie besteht aus einer einzelnen Self-Contained-Binary. So lässt sie sich auch auf der eigenen Hardware mit sehr überschaubarem Aufwand bereitstellen.

Da für dieses Projekt zudem keine externen Datenbanken aufgesetzt werden müssen, eignet es sich in besonderem Maße für (agile) Experimente. (fm)

Dieser Artikel ist im Original bei unserer Schwesterpublikation Infoworld.com erschienen.

Google DeepMind CEO Demis Hassabis is pushing for the US AI industry to self-regulate, with the support of government, as a starting point for an international creating shared international standards. In a blog post, he called for a focus on artificial general intelligence (AGI) and national security. 

But it is precisely that focus on national security that may make the results of such an effort, assuming it happens, less than palatable outside of the US.

“The rapid progress we’re seeing in AI requires a new approach to testing frontier AI model capabilities that is dynamic, adaptable, and rigorous,” Hassabis wrote. “The US is well positioned, given its economic and technical standing, to take the first step in developing such a framework. It could establish a new Standards Body modelled on a federally overseen public-private partnership or self-regulatory organization, much like the Financial Industry Regulatory Authority (FINRA), with a board that includes independent leading technical experts and open-source representatives.”

He noted, however, that the funding would need to be substantial, and would most likely come from industry, to allow the new body to attract world-class technical talent and obtain the necessary compute resources for large-scale testing.

Hassabis proposed that the organization “be responsible for developing assessment protocols and working with appropriate federal agencies and the US National Labs to conduct testing in areas relevant to national security,” and that AI vendor participants be encouraged to adopt best practices such as publishing model cards with technical details, maintaining strong internal cybersecurity, vetting key personnel, and providing sufficient resourcing for safety and security research.

This is not the first time Hassabis has expressed worries about AGI. 

DeepMind was involved in an earlier US government initiative evaluating AI safety, alongside Microsoft and xAI (now SpaceXAI) working with the Center for AI Standards and Innovation (CAISI), a division of the US Department of Commerce. It allowed CAISI to conduct pre-deployment evaluations and targeted research to “better assess frontier AI capabilities and advance the state of AI security.”  

The rest of the world may have concerns

Analysts and consultants were mixed about the move, with most expressing concerns about whether an industry-focused group would prioritize the public’s best interests.

“Self-regulation is not viable because it implies everyone is able to regulate themselves and will do so in line with the best interests of the public. Most tech vendors don’t have the capacity to self-regulate. They would just prefer a set of rules within which they can operate,” said Gartner VP analyst Nader Henein. “For-profit organizations are required to do what is best for their shareholders, and external regulation ensures that those organizations are never in a conflict of interest where they have to choose between what is good for their shareholders and what is good for the public.”

And, said Sanchit Vir Gogia, chief analyst at Greyhound Research, given the international nature of AI models, an effort coordinated by the US government might alienate other countries. 

“National security is the proposal’s accelerator in Washington and its poison pill abroad: the framing that opens the only gate available at home invites foreign capitals to read the institution as an instrument of American strategy,” he pointed out.

“The map is already plural,” he said. “Brussels switches on enforcement powers over general-purpose models [starting in August 2026], London runs the AI Security Institute, and Beijing licenses on its own terms. California and New York have legislated for frontier models at home. The durable route is shared technical evidence with sovereign enforcement, sealed through mutual recognition rather than deference, with India and the other major non-Western markets holding authorship rather than seats.”

Gogia added that the rules enacted by even such a group may not address all of the key concerns of enterprise IT. A US government effort along the lines that Hassabis is proposing would result in testing that “sits close to intelligence and industrial policy, and those functions will not stay neatly separated. A model can pass every catastrophic-risk test and still fail the enterprise on privacy, reliability, and liability,” he noted.

Walmart’s former director of cybersecurity Steven Eric Fisher, who is now an independent cybersecurity consultant, said he found the proposal “well-intentioned, but it addresses a highly polarized topic at a time when commercial interests carry unprecedented political influence, which is not always applied benevolently.”

He added, “an exclusive US standard that is not globally respected or enforceable would likely fail to achieve its core purpose and would place US companies at a competitive disadvantage.”

Aman Mahapatra, chief strategy officer for Tribeca Softtech, a New York City-based technology consulting firm, said that a deep dive into how FINRA operates today is illustrative of what IT leaders can expect from this effort, assuming the industry adopts that model.

“When the CEOs of the five companies that would be regulated are also the primary drafters of the standards, the standards will reflect those companies’ interests. FINRA has an independent board, but the operational reality is that member firm perspectives dominate the working groups that write the actual rules,” he said. “There is no reason to expect an AI equivalent to work differently, and every reason to expect it to work worse, because AI standardization is happening faster than any industry has ever attempted to standardize itself, and speed is the enemy of independent oversight.”

Carmi Levy, an independent technology analyst, was even more emphatically opposed to the Hassabis proposal.

“Asking Big Tech companies to self-police is analogous to allowing foxes to guard the henhouse. It hasn’t worked to date, and it won’t work going forward. Expecting these organizations to somehow change their ways at this point in time represents the height of naïve thinking,” Levy said. “The framework proposed by Demis Hassabis is a self-serving roadmap for an industry bent on racing to the AI horizon regardless of the harms caused along the way. It is impossible to quantify the dangers to broader society should frameworks allowing self-regulation become the norm.”

Some love the proposal

An almost completely opposite stance came from Yuri Goryunov, CIO of consulting firm Acceligence, who applauded the proposed move.

“This is one of the rare setups where industry self-regulation has a real shot, and enterprise IT should be enthusiastically rooting for it,” he said. “It fails when harms are externalized, such as in social media content moderation. Or when the overseer outsources judgment to the overseen, such as the FAA’s delegation to Boeing before the 737 MAX. It works when everyone in the industry shares the catastrophic downside.”

He suggested, however, that the best precedent here isn’t FINRA, it’s INPO, the Institute of Nuclear Power Operations, which the nuclear industry created within months of the 1979 Three Mile Island partial reactor meltdown “on the logic that an accident anywhere is an accident everywhere. INPO peer-reviews every US plant, its evaluations move insurance premiums, and it sits on top of the NRC’s statutory floor. That is a public-private stack very close to what Hassabis is describing. Frontier AI has the same structure: one lab’s catastrophic failure brings regulation down on all of them.”

For enterprise CIOs and other IT executives, Goryunov said, that model has the potential for being a big win.

“Today, every enterprise duplicates the same AI diligence of red-teaming, eval suites, governance committees and each does so with less information than any certifying body would have,” Goryunov said. “A credible standards regime does for AI what UL did for electrical equipment and SOC2 did for cloud: it converts an unknowable risk into a procurable product and gives boards a defensible standard of care. That’s not red tape. That’s peace of mind with an audit trail.”

However, Mahapatra said, “the countervailing view is that the alternative to industry-led standards is probably not thoughtful legislation. It is probably no standards, or state-by-state fragmentation, or the current pattern of ex-post enforcement actions where regulators surface concerns years after harm has already occurred.”

Thus, he noted, “Hassabis is making the reasonable argument that imperfect fast standards are better than perfect slow ones, and there is genuine merit to that view for topics like agent identity, evaluation methodology, and interoperability, which are exactly the areas OpenClaw is also targeting.”

Google DeepMind CEO Demis Hassabis is pushing for the US AI industry to self-regulate, with the support of government, as a starting point for an international creating shared international standards. In a blog post, he called for a focus on artificial general intelligence (AGI) and national security. 

But it is precisely that focus on national security that may make the results of such an effort, assuming it happens, less than palatable outside of the US.

“The rapid progress we’re seeing in AI requires a new approach to testing frontier AI model capabilities that is dynamic, adaptable, and rigorous,” Hassabis wrote. “The US is well positioned, given its economic and technical standing, to take the first step in developing such a framework. It could establish a new Standards Body modelled on a federally overseen public-private partnership or self-regulatory organization, much like the Financial Industry Regulatory Authority (FINRA), with a board that includes independent leading technical experts and open-source representatives.”

He noted, however, that the funding would need to be substantial, and would most likely come from industry, to allow the new body to attract world-class technical talent and obtain the necessary compute resources for large-scale testing.

Hassabis proposed that the organization “be responsible for developing assessment protocols and working with appropriate federal agencies and the US National Labs to conduct testing in areas relevant to national security,” and that AI vendor participants be encouraged to adopt best practices such as publishing model cards with technical details, maintaining strong internal cybersecurity, vetting key personnel, and providing sufficient resourcing for safety and security research.

This is not the first time Hassabis has expressed worries about AGI. 

DeepMind was involved in an earlier US government initiative evaluating AI safety, alongside Microsoft and xAI (now SpaceXAI) working with the Center for AI Standards and Innovation (CAISI), a division of the US Department of Commerce. It allowed CAISI to conduct pre-deployment evaluations and targeted research to “better assess frontier AI capabilities and advance the state of AI security.”  

The rest of the world may have concerns

Analysts and consultants were mixed about the move, with most expressing concerns about whether an industry-focused group would prioritize the public’s best interests.

“Self-regulation is not viable because it implies everyone is able to regulate themselves and will do so in line with the best interests of the public. Most tech vendors don’t have the capacity to self-regulate. They would just prefer a set of rules within which they can operate,” said Gartner VP analyst Nader Henein. “For-profit organizations are required to do what is best for their shareholders, and external regulation ensures that those organizations are never in a conflict of interest where they have to choose between what is good for their shareholders and what is good for the public.”

And, said Sanchit Vir Gogia, chief analyst at Greyhound Research, given the international nature of AI models, an effort coordinated by the US government might alienate other countries. 

“National security is the proposal’s accelerator in Washington and its poison pill abroad: the framing that opens the only gate available at home invites foreign capitals to read the institution as an instrument of American strategy,” he pointed out.

“The map is already plural,” he said. “Brussels switches on enforcement powers over general-purpose models [starting in August 2026], London runs the AI Security Institute, and Beijing licenses on its own terms. California and New York have legislated for frontier models at home. The durable route is shared technical evidence with sovereign enforcement, sealed through mutual recognition rather than deference, with India and the other major non-Western markets holding authorship rather than seats.”

Gogia added that the rules enacted by even such a group may not address all of the key concerns of enterprise IT. A US government effort along the lines that Hassabis is proposing would result in testing that “sits close to intelligence and industrial policy, and those functions will not stay neatly separated. A model can pass every catastrophic-risk test and still fail the enterprise on privacy, reliability, and liability,” he noted.

Walmart’s former director of cybersecurity Steven Eric Fisher, who is now an independent cybersecurity consultant, said he found the proposal “well-intentioned, but it addresses a highly polarized topic at a time when commercial interests carry unprecedented political influence, which is not always applied benevolently.”

He added, “an exclusive US standard that is not globally respected or enforceable would likely fail to achieve its core purpose and would place US companies at a competitive disadvantage.”

Aman Mahapatra, chief strategy officer for Tribeca Softtech, a New York City-based technology consulting firm, said that a deep dive into how FINRA operates today is illustrative of what IT leaders can expect from this effort, assuming the industry adopts that model.

“When the CEOs of the five companies that would be regulated are also the primary drafters of the standards, the standards will reflect those companies’ interests. FINRA has an independent board, but the operational reality is that member firm perspectives dominate the working groups that write the actual rules,” he said. “There is no reason to expect an AI equivalent to work differently, and every reason to expect it to work worse, because AI standardization is happening faster than any industry has ever attempted to standardize itself, and speed is the enemy of independent oversight.”

Carmi Levy, an independent technology analyst, was even more emphatically opposed to the Hassabis proposal.

“Asking Big Tech companies to self-police is analogous to allowing foxes to guard the henhouse. It hasn’t worked to date, and it won’t work going forward. Expecting these organizations to somehow change their ways at this point in time represents the height of naïve thinking,” Levy said. “The framework proposed by Demis Hassabis is a self-serving roadmap for an industry bent on racing to the AI horizon regardless of the harms caused along the way. It is impossible to quantify the dangers to broader society should frameworks allowing self-regulation become the norm.”

Some love the proposal

An almost completely opposite stance came from Yuri Goryunov, CIO of consulting firm Acceligence, who applauded the proposed move.

“This is one of the rare setups where industry self-regulation has a real shot, and enterprise IT should be enthusiastically rooting for it,” he said. “It fails when harms are externalized, such as in social media content moderation. Or when the overseer outsources judgment to the overseen, such as the FAA’s delegation to Boeing before the 737 MAX. It works when everyone in the industry shares the catastrophic downside.”

He suggested, however, that the best precedent here isn’t FINRA, it’s INPO, the Institute of Nuclear Power Operations, which the nuclear industry created within months of the 1979 Three Mile Island partial reactor meltdown “on the logic that an accident anywhere is an accident everywhere. INPO peer-reviews every US plant, its evaluations move insurance premiums, and it sits on top of the NRC’s statutory floor. That is a public-private stack very close to what Hassabis is describing. Frontier AI has the same structure: one lab’s catastrophic failure brings regulation down on all of them.”

For enterprise CIOs and other IT executives, Goryunov said, that model has the potential for being a big win.

“Today, every enterprise duplicates the same AI diligence of red-teaming, eval suites, governance committees and each does so with less information than any certifying body would have,” Goryunov said. “A credible standards regime does for AI what UL did for electrical equipment and SOC2 did for cloud: it converts an unknowable risk into a procurable product and gives boards a defensible standard of care. That’s not red tape. That’s peace of mind with an audit trail.”

However, Mahapatra said, “the countervailing view is that the alternative to industry-led standards is probably not thoughtful legislation. It is probably no standards, or state-by-state fragmentation, or the current pattern of ex-post enforcement actions where regulators surface concerns years after harm has already occurred.”

Thus, he noted, “Hassabis is making the reasonable argument that imperfect fast standards are better than perfect slow ones, and there is genuine merit to that view for topics like agent identity, evaluation methodology, and interoperability, which are exactly the areas OpenClaw is also targeting.”

This article first appeared on CIO.

It’s not so much generosity that’s behind Anthropic’s decision to extend free access to its most advanced model, Fable, for paid subscribers until July 19, analysts say. Its a last-minute move to grab users, data and model evaluation results.

After the free-access period, Anthropic plans to convert Fable to a pay-per-use model, at $10 per million input tokens and a whopping $50 for 1 million output tokens.

That is double the price of its next most advanced model, Opus 4.8, for input and output tokens. “We’re extending Claude Fable 5 access on all paid plans, as well as keeping Claude Code’s weekly rate limits 50% higher, through July 19,” Anthropic’s team said in a July 12 tweet.

Anthropic keeps extending Fable because it does not yet know what its flagship is worth, said Sanchit Vir Gogia, principal analyst at Greyhound Research. “A vendor confident in its price does not move the same cutoff twice in six days, both times at the wire,” Gogia said.

Anthropic is essentially pushing deadlines to test its products, while users gain by being able to put their toughest tasks to Fable, Gogia said.

Anthropic, which did not immediately reply to a request for comment about the situation, has already seen plenty of action with Fable and its sister model Mythos. Both have been touted as the company’s most advanced models yet.

Fable stumbles, then reappears

Fable was officially launched June 9. Just three days later, on June 12, the US government put export controls on it after Amazon researchers bypassed Fable’s safeguards, prompting the model to identify software vulnerabilities and demonstrate an exploit. 

After Anthropic scrambled to address the issues — and after the export controls were lifted — Fable was relaunched July 1.

Fable’s freebie extension comes after OpenAI’s latest model, ChatGPT 5.6 Sol, became generally available July 9. Sol is cheaper at $5 per one million tokens input, and $30 for 1 million output tokens.

Anthropic and OpenAI are competing aggressively to build market share, said Jack Gold, principal analyst at J. Gold Associates. “Anthropic and OpenAI are looking to go public and the more users they have, the more attractive it is — even if they are not yet producing income,” he said.

In some ways, the two companies are following a well-trodden path to get customers hooked on their products and turned into paying customers. That’s what Meta, Google and Microsoft, for instance, have done over the years with various “free” offers that later morphed into paid products. 

Plus, said Gold, ”The more users you have, the better you can train your models across multiple data sets.”

That’s a potential boon for proprietary large language model (LLM) vendors offering free tokens in a bid to lock enterprises and vendors into their AI environments. But numerous experts have warned enterprises not to fall for that tactic. Instead, they argue enterprises should diversify AI development across multiple AI and cloud vendors, and adopt open-source models.

An LLM space race?

According to LLM benchmarks maintained by Artificial Analysis, Fable is the most intelligent model currently available, with Sol just behind it in second place. One benchmark by LiveBench places Sol as being better in reasoning, with Fable better at math, data analysis, instruction following and language. Both models have advantages in coding.

Meanwhile, Cursor and SpaceXAI on July 8 unveiled Grok 4.5, which the companies said can “handle difficult, long-running tasks that require creatively using tools to solve problems, whether in software engineering, data science, finance, legal work, or anything else you do on a computer,” the company said in a blog entry.

Its pricing is even more aggressive than Fable and ChatGPT 5.6 Sol. Grok 4.5 charges $2 for 1 million input tokens and $6 for 1 million output tokens.

There are growing concerns about tokenmaxxing, where enterprises rack up billions of dollars in token spending, blowing past usage limits before finance controls are implemented.

Enterprises might decide to spend more on models such as Mythos and Fable — if the benefits are tangible, said Max Leaming, head of data science and AI solutions at ManpowerGroup. Fable and Mythos may “actually be less expensive to use in spite of the spiked token cost because it’s far more efficient,” he said.

A company might find that the models use fewer tokens, are faster, and can reduce compute time, he said. “Even though the per-token costs may go up, we may see overall costs go down,” Leaming said.

Most security leaders I know working on AI robotics are being shown the same kind of video. A humanoid folds a shirt, sorts a bin, walks a warehouse aisle and a vendor uses the clip to move an embodied AI system from pitch to purchase order. Someone then has to sign off. Robot demos create procurement momentum before security teams receive the artifacts needed to evaluate the system as cyber-physical infrastructure.

Before the book, I prepared cloud infrastructure operating in China and the United States for cybersecurity compliance audits and for the Multi-Level Protection Scheme, China’s mandatory security-grading regime that determines whether a system is allowed to operate. That work taught me a lesson I carry into every AI conversation now. You cannot secure what you cannot see into, and the buyer rarely sees in. A demo makes it worse. It shows one task, completed once, under conditions the vendor chose. None of what a security team must evaluate is on screen.

This used to be a research-lab problem. It is now a procurement line item. The risk changed when embodied AI moved from a research demo to a purchase order.  Vendors are asking security teams to approve embodied AI before the category has audit evidence, logging norms, supplier transparency or a shared-responsibility model.

Embodied AI puts a model inside a machine that operates in the physical world: a robot, an arm, a humanoid. Once a model gains motors, sensors and a body, it ceases to be a software endpoint and becomes a cyber-physical system. It inherits hardware, firmware, a supply chain, an installer and a set of remote-access paths. Every one of those is an attack surface that the demo video doesn’t show. An embodied system is sold like software and behaves like a fleet of networked machinery on your floor.

Evaluate these systems across five questions: provenance, access, integrity, evidence and accountability. Here is what each means.

Evaluation question #1: Provenance

What is inside, and who controls it? A humanoid is an assembly of actuators, lidar units, battery packs, joint modules and controllers, most from a supply chain the buyer never vetted, each running firmware the buyer cannot read. Software teams already fought this fight, which is why the software bill of materials became standard practice. Lack of transparency creates systemic risk. Embodied systems raise the stakes because the firmware now lives in dozens of parts that move. The risk does not depend on whether the robot is Chinese, American, German or Japanese. It depends on how much of the system the buyer can see: the hardware, firmware, remote-access paths and maintenance relationships behind it.  China installs more industrial robots than any other country and sits near the center of the battery supply chain, as well as parts of the lidar and machine-vision supply base, which these systems draw on. Lidar, short for Light Detection and Ranging, uses pulsed laser beams to map an environment in 3D; machine vision handles optical inspection and guidance. Much of that lineage traces to suppliers your team has no relationship with. This is the hardware and firmware version of the third-party risk NIST’s supply chain guidance was written for, except that the component has motors. Demand a hardware and firmware bill of materials, then use it. Flag unsigned firmware. Map which supplier holds update authority for each part. Require a way to verify integrity, and treat any component you cannot identify as unmanaged.

Evaluation question #2: Access

Who can reach the fleet? Someone installs these machines, someone services them and the vendor pushes software updates.  Where teleoperation is part of the support model, treat it as a privileged remote-access path, not a convenience feature.  Each is a standing path into a machine that moves and lifts. Security teams have seen this story before. Operational Technology (OT) security went mainstream once industrial systems joined IT networks, and the recurring failure is unmanaged remote access that nobody inventoried. According to one industry survey, roughly half of attacks on OT assets originate in an IT network breach. SolarWinds showed why a trusted update channel deserves scrutiny when one delivered a backdoor to thousands of networks. Embodied systems add the harder part. The compromised endpoint can move. A remote operator on that channel can drive a machine and push code to every unit at once. Treat the fleet like high-value OT. Inventory every remote path, segment it from the production network, default to deny, require signed and verified updates, apply privileged-access controls to vendor maintenance, and treat an always-on teleoperation link as a backdoor until it is governed.

Evaluation question #3: Integrity

Whether the machine can be made to misperceive or misbehave. Researchers have shown that lidar spoofing can cause an autonomous system to brake for an obstacle that is not there or miss one that is. The same class of sensor and model manipulation, on a humanoid sharing a floor with people, produces motion, not a wrong answer on a screen. This is where safety engineering and security part ways. Functional safety stops hazardous motion when a component fails. It plans for accidents. Security plans for an adversary. A hardwired safety circuit can stay independent of the control plane, and a good one does. What it does not tell you is how an attacker reached that control plane, altered the model’s inputs or seized the fleet-management path. Ask the vendor to threat-model sensor spoofing and model manipulation as a path to physical motion. Then ask how you will even know it happened. A spoofed sensor does not announce itself. It shows up as a machine acting incorrectly with confidence.

Picture the failure in plain terms. A warehouse robot takes a routine vendor update that changes how it navigates. The buyer cannot verify the firmware, cannot identify the supplier of the sensor module and has no logs to distinguish a spoofed sensor from a model error. The machine keeps moving, and no one can say why.

Evaluation question #4: Evidence

Whether the claims are true. You have not found an independent audit of embodied-AI field performance, so the uptime and reliability numbers come from the vendor. You are buying a claim, not a track record. Require independently verified uptime, intervention rate and incident history from a named deployment you can call. “Cutting-edge” is not a control.

Evaluation question #5: Accountability

Who owns the risk when it fails? Cloud taught security teams shared responsibility the hard way, after years of arguing which side of the line a breach fell on. Embodied AI arrives without that model, and the stakes are physical: the machine can injure someone. In my compliance work, the question that decided everything was always who is accountable when this thing breaks. Put it in the contract. Define the responsibility boundary, an incident-disclosure timeline, a right to audit and liability for physical harm. A vendor who will not commit in writing is showing you who bears the risk.

These five questions share one root. For a decade, the security question was whether you could trust what a model generates. The embodied question is who can reach the machine and what they can make it do. A demo answers neither.

Before any embodied system reaches your floor, make these five demands of the vendor.

• Provenance. A hardware and firmware bill of materials with named suppliers, integrity verification and a vulnerability-disclosure record. No bill of materials, no deal.
• Access. A full map of who installs, who services and every update and teleoperation path, with segmentation, default-deny and signed updates required.
• Integrity. A threat model for sensor spoofing and model manipulation that treats the failure as physical motion, plus logging that a defender can use.
• Evidence. Independently verified uptime, intervention and incident history from a named deployment you can call.
• Accountability. A contract that defines the responsibility boundary, incident-disclosure timelines, audit rights and liability for physical harm.

The robot demo is built to make you feel the future has arrived. My job, and now yours, is the unglamorous question behind it. Ask what the machine’s attack surface looks like once it is bolted to your floor, wired to your network and updated by someone you have never met.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Google DeepMind CEO Demis Hassabis on Tuesday reiterated his push for an AI industry self-regulation effort, led by the US government, that is particularly focused on artificial general intelligence (AGI) and national security. 

But it is precisely that focus on national security that may make the results of such an effort, assuming it happens, less than palatable outside of the US.

“The rapid progress we’re seeing in AI requires a new approach to testing frontier AI model capabilities that is dynamic, adaptable, and rigorous,” Hassabis wrote. “The US is well positioned, given its economic and technical standing, to take the first step in developing such a framework. It could establish a new Standards Body modelled on a federally overseen public-private partnership or self-regulatory organization, much like the Financial Industry Regulatory Authority (FINRA), with a board that includes independent leading technical experts and open-source representatives.”

He noted, however, that the funding would need to be substantial, and would most likely come from industry, to allow the new body to attract world-class technical talent and obtain the necessary compute resources for large-scale testing.

Hassabis said he would propose that the organization “be responsible for developing assessment protocols and working with appropriate federal agencies and the US National Labs to conduct testing in areas relevant to national security,” and that AI vendor participants would be encouraged to adopt best practices, such as publishing model cards with technical details, maintaining strong internal cybersecurity, vetting key personnel, and providing sufficient resourcing for safety and security research.

This is not the first time Hassabis has expressed worries about AGI. He has already worked on a US government initiative evaluating AI safety, which involved DeepMind, Microsoft and xAI (now SpaceXAI) working with the Center for AI Standards and Innovation (CAISI), a division of the US Department of Commerce. It allowed CAISI to conduct pre-deployment evaluations and targeted research to “better assess frontier AI capabilities and advance the state of AI security.”  

The rest of the world may have concerns

Analysts and consultants were mixed about the move, with most expressing concerns about whether an industry-focused group would prioritize the public’s best interests.

“Self-regulation is not viable because it implies everyone is able to regulate themselves and will do so in line with the best interests of the public. Most tech vendors don’t have the capacity to self-regulate. They would just prefer a set of rules within which they can operate,” said Gartner VP analyst Nader Henein. “For-profit organizations are required to do what is best for their shareholders, and external regulation ensures that those organizations are never in a conflict of interest where they have to choose between what is good for their shareholders and what is good for the public.”

And, said Sanchit Vir Gogia, chief analyst at Greyhound Research, given the international nature of AI models, an effort coordinated by the US government might alienate other countries. 

“National security is the proposal’s accelerator in Washington and its poison pill abroad: the framing that opens the only gate available at home invites foreign capitals to read the institution as an instrument of American strategy,” he pointed out.

“The map is already plural,” he said. “Brussels switches on enforcement powers over general-purpose models [starting in August 2026], London runs the AI Security Institute, and Beijing licenses on its own terms. California and New York have legislated for frontier models at home. The durable route is shared technical evidence with sovereign enforcement, sealed through mutual recognition rather than deference, with India and the other major non-Western markets holding authorship rather than seats.”

Gogia added that the rules enacted by even such a group may not address all of the key concerns of enterprise IT. A US government effort along the lines that Hassabis is proposing would result in testing that “sits close to intelligence and industrial policy, and those functions will not stay neatly separated. A model can pass every catastrophic-risk test and still fail the enterprise on privacy, reliability, and liability,” he noted.

Walmart’s former director of cybersecurity Steven Eric Fisher, who is now an independent cybersecurity consultant, said he found the proposal “well-intentioned, but it addresses a highly polarized topic at a time when commercial interests carry unprecedented political influence, which is not always applied benevolently.”

He added, “an exclusive US standard that is not globally respected or enforceable would likely fail to achieve its core purpose and would place US companies at a competitive disadvantage.”

Aman Mahapatra, chief strategy officer for Tribeca Softtech, a New York City-based technology consulting firm, said that a deep dive into how FINRA operates today is illustrative of what IT leaders can expect from this effort, assuming the industry adopts that model.

“When the CEOs of the five companies that would be regulated are also the primary drafters of the standards, the standards will reflect those companies’ interests. FINRA has an independent board, but the operational reality is that member firm perspectives dominate the working groups that write the actual rules,” he said. “There is no reason to expect an AI equivalent to work differently, and every reason to expect it to work worse, because AI standardization is happening faster than any industry has ever attempted to standardize itself, and speed is the enemy of independent oversight.”

Carmi Levy, an independent technology analyst, was even more emphatically opposed to the Hassabis proposal.

“Asking Big Tech companies to self-police is analogous to allowing foxes to guard the henhouse. It hasn’t worked to date, and it won’t work going forward. Expecting these organizations to somehow change their ways at this point in time represents the height of naïve thinking,” Levy said. “The framework proposed by Demis Hassabis is a self-serving roadmap for an industry bent on racing to the AI horizon regardless of the harms caused along the way. It is impossible to quantify the dangers to broader society should frameworks allowing self-regulation become the norm.”

Some love the proposal

An almost completely opposite stance came from Yuri Goryunov, CIO of consulting firm Acceligence, who applauded the proposed move.

“This is one of the rare setups where industry self-regulation has a real shot, and enterprise IT should be enthusiastically rooting for it,” he said. “It fails when harms are externalized, such as in social media content moderation. Or when the overseer outsources judgment to the overseen, such as the FAA’s delegation to Boeing before the 737 MAX. It works when everyone in the industry shares the catastrophic downside.”

He suggested, however, that the best precedent here isn’t FINRA, it’s INPO, the Institute of Nuclear Power Operations, which the nuclear industry created within months of the 1979 Three Mile Island partial reactor meltdown “on the logic that an accident anywhere is an accident everywhere. INPO peer-reviews every US plant, its evaluations move insurance premiums, and it sits on top of the NRC’s statutory floor. That is a public-private stack very close to what Hassabis is describing. Frontier AI has the same structure: one lab’s catastrophic failure brings regulation down on all of them.”

For enterprise CIOs and other IT executives, Goryunov said, that model has the potential for being a big win.

“Today, every enterprise duplicates the same AI diligence of red-teaming, eval suites, governance committees and each does so with less information than any certifying body would have,” Goryunov said. “A credible standards regime does for AI what UL did for electrical equipment and SOC2 did for cloud: it converts an unknowable risk into a procurable product and gives boards a defensible standard of care. That’s not red tape. That’s peace of mind with an audit trail.”

However, Mahapatra said, “the countervailing view is that the alternative to industry-led standards is probably not thoughtful legislation. It is probably no standards, or state-by-state fragmentation, or the current pattern of ex-post enforcement actions where regulators surface concerns years after harm has already occurred.”

Thus, he noted, “Hassabis is making the reasonable argument that imperfect fast standards are better than perfect slow ones, and there is genuine merit to that view for topics like agent identity, evaluation methodology, and interoperability, which are exactly the areas OpenClaw is also targeting.”

Google DeepMind CEO Demis Hassabis on Tuesday reiterated his push for an AI industry self-regulation effort, led by the US government, that is particularly focused on artificial general intelligence (AGI) and national security. 

But it is precisely that focus on national security that may make the results of such an effort, assuming it happens, less than palatable outside of the US.

“The rapid progress we’re seeing in AI requires a new approach to testing frontier AI model capabilities that is dynamic, adaptable, and rigorous,” Hassabis wrote. “The US is well positioned, given its economic and technical standing, to take the first step in developing such a framework. It could establish a new Standards Body modelled on a federally overseen public-private partnership or self-regulatory organization, much like the Financial Industry Regulatory Authority (FINRA), with a board that includes independent leading technical experts and open-source representatives.”

He noted, however, that the funding would need to be substantial, and would most likely come from industry, to allow the new body to attract world-class technical talent and obtain the necessary compute resources for large-scale testing.

Hassabis said he would propose that the organization “be responsible for developing assessment protocols and working with appropriate federal agencies and the US National Labs to conduct testing in areas relevant to national security,” and that AI vendor participants would be encouraged to adopt best practices, such as publishing model cards with technical details, maintaining strong internal cybersecurity, vetting key personnel, and providing sufficient resourcing for safety and security research.

This is not the first time Hassabis has expressed worries about AGI. He has already worked on a US government initiative evaluating AI safety, which involved DeepMind, Microsoft and xAI (now SpaceXAI) working with the Center for AI Standards and Innovation (CAISI), a division of the US Department of Commerce. It allowed CAISI to conduct pre-deployment evaluations and targeted research to “better assess frontier AI capabilities and advance the state of AI security.”  

The rest of the world may have concerns

Analysts and consultants were mixed about the move, with most expressing concerns about whether an industry-focused group would prioritize the public’s best interests.

“Self-regulation is not viable because it implies everyone is able to regulate themselves and will do so in line with the best interests of the public. Most tech vendors don’t have the capacity to self-regulate. They would just prefer a set of rules within which they can operate,” said Gartner VP analyst Nader Henein. “For-profit organizations are required to do what is best for their shareholders, and external regulation ensures that those organizations are never in a conflict of interest where they have to choose between what is good for their shareholders and what is good for the public.”

And, said Sanchit Vir Gogia, chief analyst at Greyhound Research, given the international nature of AI models, an effort coordinated by the US government might alienate other countries. 

“National security is the proposal’s accelerator in Washington and its poison pill abroad: the framing that opens the only gate available at home invites foreign capitals to read the institution as an instrument of American strategy,” he pointed out.

“The map is already plural,” he said. “Brussels switches on enforcement powers over general-purpose models [starting in August 2026], London runs the AI Security Institute, and Beijing licenses on its own terms. California and New York have legislated for frontier models at home. The durable route is shared technical evidence with sovereign enforcement, sealed through mutual recognition rather than deference, with India and the other major non-Western markets holding authorship rather than seats.”

Gogia added that the rules enacted by even such a group may not address all of the key concerns of enterprise IT. A US government effort along the lines that Hassabis is proposing would result in testing that “sits close to intelligence and industrial policy, and those functions will not stay neatly separated. A model can pass every catastrophic-risk test and still fail the enterprise on privacy, reliability, and liability,” he noted.

Walmart’s former director of cybersecurity Steven Eric Fisher, who is now an independent cybersecurity consultant, said he found the proposal “well-intentioned, but it addresses a highly polarized topic at a time when commercial interests carry unprecedented political influence, which is not always applied benevolently.”

He added, “an exclusive US standard that is not globally respected or enforceable would likely fail to achieve its core purpose and would place US companies at a competitive disadvantage.”

Aman Mahapatra, chief strategy officer for Tribeca Softtech, a New York City-based technology consulting firm, said that a deep dive into how FINRA operates today is illustrative of what IT leaders can expect from this effort, assuming the industry adopts that model.

“When the CEOs of the five companies that would be regulated are also the primary drafters of the standards, the standards will reflect those companies’ interests. FINRA has an independent board, but the operational reality is that member firm perspectives dominate the working groups that write the actual rules,” he said. “There is no reason to expect an AI equivalent to work differently, and every reason to expect it to work worse, because AI standardization is happening faster than any industry has ever attempted to standardize itself, and speed is the enemy of independent oversight.”

Carmi Levy, an independent technology analyst, was even more emphatically opposed to the Hassabis proposal.

“Asking Big Tech companies to self-police is analogous to allowing foxes to guard the henhouse. It hasn’t worked to date, and it won’t work going forward. Expecting these organizations to somehow change their ways at this point in time represents the height of naïve thinking,” Levy said. “The framework proposed by Demis Hassabis is a self-serving roadmap for an industry bent on racing to the AI horizon regardless of the harms caused along the way. It is impossible to quantify the dangers to broader society should frameworks allowing self-regulation become the norm.”

Some love the proposal

An almost completely opposite stance came from Yuri Goryunov, CIO of consulting firm Acceligence, who applauded the proposed move.

“This is one of the rare setups where industry self-regulation has a real shot, and enterprise IT should be enthusiastically rooting for it,” he said. “It fails when harms are externalized, such as in social media content moderation. Or when the overseer outsources judgment to the overseen, such as the FAA’s delegation to Boeing before the 737 MAX. It works when everyone in the industry shares the catastrophic downside.”

He suggested, however, that the best precedent here isn’t FINRA, it’s INPO, the Institute of Nuclear Power Operations, which the nuclear industry created within months of the 1979 Three Mile Island partial reactor meltdown “on the logic that an accident anywhere is an accident everywhere. INPO peer-reviews every US plant, its evaluations move insurance premiums, and it sits on top of the NRC’s statutory floor. That is a public-private stack very close to what Hassabis is describing. Frontier AI has the same structure: one lab’s catastrophic failure brings regulation down on all of them.”

For enterprise CIOs and other IT executives, Goryunov said, that model has the potential for being a big win.

“Today, every enterprise duplicates the same AI diligence of red-teaming, eval suites, governance committees and each does so with less information than any certifying body would have,” Goryunov said. “A credible standards regime does for AI what UL did for electrical equipment and SOC2 did for cloud: it converts an unknowable risk into a procurable product and gives boards a defensible standard of care. That’s not red tape. That’s peace of mind with an audit trail.”

However, Mahapatra said, “the countervailing view is that the alternative to industry-led standards is probably not thoughtful legislation. It is probably no standards, or state-by-state fragmentation, or the current pattern of ex-post enforcement actions where regulators surface concerns years after harm has already occurred.”

Thus, he noted, “Hassabis is making the reasonable argument that imperfect fast standards are better than perfect slow ones, and there is genuine merit to that view for topics like agent identity, evaluation methodology, and interoperability, which are exactly the areas OpenClaw is also targeting.”

This article originally appeared on CIO.com.

Every day this past week I did something I suspect millions of other people also did: I stared at an LLM model picker and wondered which one I was supposed to want.

OpenAI just released ⁠GPT-5.6 Sol, Terra, and Luna. Sol is the flagship. Terra offers much of its intelligence for less money. Luna is cheaper still. Anthropic released ⁠Claude Sonnet 5 at the end of June and Opus 4.8 the month prior, with a little Fable 5 emerging in between. Meanwhile, Google, which seemed to be winning the model wars a few months ago, is now getting shade from Gergely Orosz, who ⁠argues that Gemini has slipped outside the top tier for software development and has been out of the major model release game for eons (May 19).

Perhaps Orosz is right. Perhaps he’ll be wrong again in six weeks. Honestly, it’s exhausting.

I use ChatGPT and Claude constantly and still have no principled idea which model to choose most of the time. I tend to click whatever looks like the biggest, most expensive option because I don’t know what I’m giving up by choosing something smaller. “Instant” sounds dangerously unserious. “Thinking” sounds expensive but powerful.

A quick survey of my LinkedIn crowd suggests others also feel my “WHICH MODEL???” pain. More importantly, I suspect most enterprises do, too.

A model doesn’t rot

Before getting carried away, however, it’s worth considering whether any of this model churn actually matters. After all, a model doesn’t rot. The model an enterprise put into production in March performs just as well in July as it did when the company selected it. “Obsolete” generally means that something better now exists, not that the deployed model suddenly stopped summarizing insurance claims or classifying support tickets. (In other words, once you have something working, the idea that “but maybe Opus 200.2 is better!” is really a FOMO problem, not a performance issue.)

Most enterprise workloads don’t live at the frontier anyway. Extraction, summarization, classification, document comparison, and customer-service assistance often work perfectly well with smaller, cheaper models. OpenAI’s own pitch for the trio of GPT-5.6 models isn’t simply that Sol is better. It’s that ⁠Terra and Luna deliver different combinations of intelligence, latency, and cost. Luna, the cheapest tier, nearly matches the previous generation’s peak performance at less than half the estimated cost, according to OpenAI.

The practical question, of course, is where to start. An enterprise can’t test every model, every reasoning setting, and every price tier before doing any work. So here’s my advice (which I don’t follow in my own work, but I’m not defining enterprise strategy and can be a little price-insensitive). Start with the cheapest credible model that appears capable of the task. Give it a representative set of real examples and, before you start testing, define what counts as good enough. If it passes, stop. If it fails, move up a tier or try a model with strengths better suited to the work.

That sounds almost offensively simple, but it reverses the way many people, including me, use these products. We start with the biggest model because we’re afraid of what we might lose. Enterprises should start lower and require evidence before paying for more intelligence.

There are exceptions, of course. For genuinely difficult work, such as autonomous coding, complex research, or high-stakes reasoning, beginning with a frontier model may save time. But even then, the goal should be to establish a quality ceiling, then test whether a cheaper model can meet it. It’s changing the question from “which model is best?” to “what is the least expensive model that reliably clears the bar for this job?”

For many workloads, that price improvement matters more than a few extra benchmark points. ⁠As I argued back in 2023, following AI hype doesn’t help anyone. If your model strategy depends on whichever benchmark screenshot is circulating on X this week, you don’t have a strategy. Not a viable one, anyway. Pick a model and ignore the noise.

Except, of course, when that noise suggests a serious signal.

Sometimes better really is better

Frontier improvements aren’t always incremental, making it advantageous to consider an upgrade. Coding is the obvious example. There’s a significant difference between a model that suggests the next few lines of code and one that can inspect a repository, plan a change, use tools, run tests, discover its own mistakes, and keep working for an extended period. That isn’t merely a nicer autocomplete experience. It can reorganize a development workflow.

This is why enterprises can’t simply standardize on an 18-month-old model and declare victory. In some areas, particularly software development and other agentic work, better models can unlock compounding productivity. A model that reliably completes 80% of a bounded task rather than 50% may justify an entirely different division of labor between humans and machines.

Still, that upgrade isn’t free.

Models differ in how they interpret instructions, call tools, manage context, refuse requests, and fail. Prompts and scaffolding tuned for one model can regress when moved to another. Or costs can explode. As one of my Oracle colleagues discovered just this week, running the same tasks in GPT 5.6 was orders of magnitude more expensive than 5.5. The API change may be trivial, but the revalidation and implications are not.

This leaves enterprises caught between two bad options. They can freeze and potentially miss out on meaningful improvements or chase every release and repeatedly test production systems on faith. What to do?

Stop making model bets

The answer is to stop making LLM bets and start making job-to-be-done bets. Stop asking which model is fastest. Instead, figure out what work you are trying to improve. What does a good result look like? How much latency and cost can the workflow tolerate? How wrong can it be before a human must intervene? Once those questions have answers, model selection becomes less opaque.

A difficult code migration may justify GPT-5.6 Sol or Claude Sonnet 5. A repetitive classification task may work just as well with Luna or another smaller model. A regulated workflow may require a model or deployment option that offers particular data controls. Sometimes the correct model is no LLM at all, like when I’m writing this post. Sorry, AI vendors! (At least you won’t get blamed for my mistakes.)

This is where evaluations become the center of enterprise AI strategy. ⁠As I’ve said before, most companies don’t have an AI quality problem so much as an AI measurement problem. Hence, a private evaluation suite built from real company work is the only leaderboard that matters. Does the new model materially improve quality? If so, use it! Does it reduce cost or latency? Again, that’s your free pass to adoption. Does the improvement justify the expense and effort of revalidation? If yes, continue.

Make model releases boring

As important as the model is, keep in mind that AI success always comes back to your company’s data, your company’s workflows, your company’s integrations, etc. That’s the ⁠dull reality behind sexy AI. Retrieval, memory, governance, data quality, observability, and feedback loops aren’t as exciting as a new model launch, but they’re what ultimately make AI truly work.

Again, when it’s time to consider something new, the principle should be to default to the least expensive model that reliably passes your evaluations. Only escalate harder tasks to more capable models when measurement shows that the premium pays. Tip: Make this invisible to employees so that the system routes to the best model for a particular prompt. As dbt Labs’ Jon Lewis expresses it, “The best model is ‘Auto’ and I won’t hear anyone say otherwise.” OpenAI’s own ⁠migration guidance recommends testing models on representative tasks, including trying a lower reasoning level rather than automatically cranking everything to the maximum.

As for me, I’ll probably keep clicking the shiniest option. I don’t have a formal evaluation suite for InfoWorld columns, and the marginal cost is a subscription I already pay. Enterprises don’t get that excuse.

A largely overlooked layer of the AI stack is emerging as a major driver of enterprise costs. New testing by AI consultancy Systima found that agent harnesses, the software that coordinates models, tools and workflows, can generate significant token overhead through their configuration alone, potentially inflating the cost of AI deployments as organizations scale agents from experimental pilots to production environments.

The firm, which ran a series of tests by juxtaposing two harnesses on the same tasks, namely Anthropic’s Claude Code and open-source OpenCode using the same Claude Sonnet 4.5 model underneath, found both exhibiting sharply different token overhead because of the differences in their configuration.

These differences included system prompts, tool definitions, agent coordination mechanisms and other orchestration components, resulting in markedly different baseline input token overhead before users even entered a prompt, the consultancy firm wrote in a blog post.

Separately, the firm also found that other configuration choices while setting up the harnesses such as repository instruction files, Model Context Protocol (MCP) servers, prompt framework templates and subagents can each add substantial token overhead.

The consultancy’s conclusions are also supported by emerging academic research examining how orchestration of the harnesses themselves, rather than optimizing models or changing them, can help enterprises reshape the economics around AI agents.

In a paper, titled The Harness Effect: How Orchestration Design Sets the Token Economics of Enterprise Agentic AI, researchers showed that changing the harness while keeping models and tasks the same can reduce token consumption by 38%, cost per task by 41%, and execution time by 44% while maintaining comparable quality.

Why enterprises overlook harness costs

Analysts say that enterprises can gain greater control over AI agent operating costs by paying closer attention to how their harnesses are configured and orchestrated, instead of just relying on model pricing as a yardstick.

“The evaluation shows that the model is only one part of agent economics. The harness, tool schemas, instructions, MCP connections, and subagents matter as well. Enterprises therefore need to measure the entire agent configuration, not assume model pricing tells them what an agent will cost,” said Stephanie Walter, practice lead of the AI stack at HyperFRAME Research.

Currently, most enterprises pick agent tooling based on model quality, benchmarks, developer experience, and headline pricing per seat or per million tokens, with almost no one measuring what the harness sends per request, how stable the cache prefix is, or what subagent fan out costs at scale, echoed Advait Patel, site reliability engineer at Broadcom.

“Ask the average CIO whether their coding agent rewrites its cache mid-session, and you will get a blank stare,” Patel added.

However, Ashish Chaturvedi, executive research leader at HFS Research, pointed out that lack of visibility is less a failure of enterprise leaders than a consequence of how AI agent ecosystem components are sold, stacked, and managed presently.

“Most organizations have no visibility, mainly due to the absence of any metric from the vendor’s end that lets CIOs measure the entire agent or at least the harness configuration. None of this shows up in the developer’s experience. The agent just works, and the tokens burn silently in the background,” Chaturvedi said.

The problem is further compounded, according to Chaturvedi, due to the manner in which AI agent configuration is distributed across enterprise teams.

“The harness is chosen by one team, the instruction file written by another, and the MCP servers attached by a third, so no single person sees the cumulative weight,” Chaturvedi noted.

Even when, in some cases, enterprises do have visibility and ownership, Patel argued, the industry, in general, still lack the operational maturity and discipline to systematically optimize AI agent costs.

“FinOps for agents is where cloud FinOps was in 2013. Nobody has hired the equivalent of a cost optimization team focused on prompt engineering, harness configuration, and cache stability,” Patel said.

Separately, Abhishek Satapathy, principal analyst at Avasant, pointed out that the invisibility issue stems from how enterprises evaluate AI agents before deploying them into production: “Most proof-of-concepts involve a limited number of users, relatively short-lived sessions, and controlled agentic interactions, where the accuracy of model output is the primary evaluation criterion.”

The analysts’ comments also echo the conclusions of another research paper, in which researchers argued that token consumption in agentic software engineering systems remains poorly understood because existing metrics provide limited visibility into where tokens are spent across orchestration components.

How CIOs can improve visibility into AI agent costs

Closing that visibility gap, though, according to Satapathy, is increasingly becoming a priority for enterprises, as AI agents move from pilots to production and operating costs become harder to predict.

“Across our advisory engagements, we are seeing growing demand for AI observability frameworks that combine runtime tracing, workload-level cost attribution, and execution analytics. This enables organizations to establish engineering baselines, benchmark workload efficiency, forecast AI operating costs, and continuously optimize agent performance as deployments mature,” Satapathy said.

However, until vendors provide more comprehensive visibility into harness-level token consumption, analysts said enterprises should begin treating harness configuration as an operational governance issue rather than merely a developer preference.

“The single most valuable move is to get visibility into what the harness actually sends. Enterprises should treat configuration as a governed cost decision, deliberately match harnesses to workloads, and closely monitor cache behavior and subagent fan-out, since those were among the biggest cost multipliers identified in the evaluation,” Chaturvedi said.

Walter echoed that recommendation, saying CIOs should require observability across the entire agent configuration: “Without that visibility, enterprises are effectively buying an agent platform without knowing how much of the bill comes from useful work versus orchestration overhead.”

Since the advent of generative AI, the use of AI in business has shifted from something we should do to something we must do to survive. Many companies are now working to utilize AI with the aim of improving productivity and creating value.

Here, I would like to pose a question to you all once again: “What is the fundamental factor that determines AI performance?”

Is it the AI model? Is it the AI tool? Or is it the AI agent?

Of course, I believe all of these are important. However, if we look at the long-term perspective, the competition among multiple companies to improve AI model performance will eventually level off, and we will eventually reach a point where every AI model is amazing!

In that context, what I believe is the most important factor influencing AI performance is the data accumulated by companies that connects to their unique strengths.

For example, if asked, “What do plants need to grow?” I would say “good water and light.”

Similarly, if asked, “What do people need to thrive?” I would say, “Kind words.”

Finally, “What does AI need to thrive?” The answer is “good data.”

I believe that the extent to which companies can genuinely understand the importance of this extremely simple principle and implement it with unwavering dedication will determine their ability to establish a competitive advantage and achieve sustainable growth.

AI is a mirror of data

As I’m sure you’re all aware, AI is by no means a magic wand. It is an entity that learns based on the data it is given and makes inferences within that scope. In other words, AI’s output depends heavily on the quality of its input data; one could say that AI is a mirror of data.

• If you feed it inaccurate data, it will return inaccurate results (i.e., garbage in, garbage out)
• If you feed it biased data, it will make biased judgments
• Insufficient data yields only shallow insights and suggestions

In this way, AI is not smart but rather faithful to the data. Based on this premise, it becomes clear that the essence of AI utilization lies not in which tools to use, but in what kind of high-quality data to prepare and how to utilize it.

What is good data?

So, what exactly is good data?

It goes without saying that data is useless if it is merely abundant in quantity, but on the other hand, what specific qualities must good data possess?

Generally speaking, good data possesses at least the following elements.

• Accuracy: Data containing many errors or noise will skew conclusions, no matter how advanced the analysis. It is important to minimize sensor errors, input mistakes and duplicates.
• Completeness: Are any required fields missing, and are there too many missing values? For example, if customer data is missing information such as age, region or gender, it becomes difficult to perform meaningful analysis.
• Consistency: Is data with the same meaning mixed in different formats (e.g., date formats, units, variations in notation)? This is particularly important for system integration and long-term data.
• Timeliness: No matter how accurate it is, data that is too old may not be useful for decision-making. Whether real-time data is required or historical data is sufficient depends on the use case, but it is important that the data has the appropriate freshness for the purpose.
• Relevance: If there is a large amount of data unrelated to the analysis objective, it becomes noise and leads to incorrect judgments. It is necessary to clearly define what the data is used for and ensure the data is appropriate for that purpose.
• Reliability: The data’s source and collection method must be clear, ensuring reliability and reproducibility. Data with an unknown source or that is a black box cannot be verified later.

In summary, good data is data that is accurate, has few gaps, is consistent in meaning and notation, is collected at the appropriate time, is suitable for the purpose and comes from a reliable source.

Only when the quality of this good data is guaranteed can AI produce valuable outputs. Conversely, introducing AI with unorganized data will not yield the expected results. Many complaints, such as “We implemented AI but it’s unusable” or “The AI’s accuracy isn’t improving stem from data issues.”

Data does not organize itself naturally

The key point here is that good data does not arise naturally. On the contrary, if left unattended, data will inevitably deteriorate.

• Rules become inconsistent depending on who entered the data and when
• Multiple instances of data with the same meaning exist
• Outdated data is scattered and left unattended
• Data becomes siloed by department

These conditions are likely common in many companies.

Below is an overview of our company’s data management framework.

Akio Ueda

Broadly speaking, it consists of data governance — covering roles and structures, risk management and evaluation — and data management, which encompasses data utilization cycle management and data utilization support services. Within this framework, data utilization cycle management involves:

• Needs management: We clarify the purpose and needs by asking, “What is the data being used for?” and “For whom, and in what way, does this data create value?”
• Collection: We gather the necessary data based on the defined objectives. We design the process to determine what data is required (internal/external), the level of detail and frequency of collection, and how to ensure data quality.
• Processing: We enhance the quality and prepare the data for use. This includes cleansing (correcting errors and missing values), standardizing formats, deduplicating and integrating data, processing structured and unstructured data separately, and assigning business and operational meaning to the data.
• Storage: We ensure the data is available to the right people at the right time. This involves storing data in databases or data lakes, implementing security and access controls, and managing metadata (ensuring the data is clearly identifiable).
• Utilization: This is the most critical step. The purpose of data is not merely analysis but driving action. We generate value from the data through visualization (dashboards), analysis (statistical processing, BI, AutoML, AI) and integration into business operations (automation and decision support).
• Disposal: We properly dispose of data that is no longer needed. Simply holding data can itself pose risks, such as managing retention periods, complying with laws and governance requirements, and mitigating security risks. That is why the principle of not holding data that is not used is so important.

Data management is not a one-time effort; it is an ongoing initiative that requires continuous maintenance and improvement.

The CIO must embed data management as a system within the organization and continue to implement it until it becomes firmly established.

Data management is not just the IT department’s job

Another important point is that data management is not just the IT department’s job.

Data is fundamentally generated within day-to-day operations on the front lines. Therefore:

• Who determines the meaning and definition of data
• How should input rules be standardized?
• How do we ensure data quality?

are, in essence, operational issues, business issues and management issues.

The latest Digital Skills Standard ver. 2.0, published by the Ministry of Economy, Trade and Industry in April 2026, defines the following three roles within the data management category:

• Data steward: Based on business domain knowledge, this role is responsible for operations aimed at ensuring data quality, reliability and security, as well as for promoting the adoption and establishment of data management within business divisions and frontline organizations, and for fostering data utilization. In short, they are the data quality manager and data utilization promoter.
• Data engineer: This role involves understanding the current state of data and supporting the organization’s continuous data utilization through data preparation and preprocessing in processes such as collection, integration, processing and provision, as well as the design and implementation of data pipelines. In essence, they are the implementers and operators who drive data.
• Data architect: This role involves taking a bird’s-eye view of the data structure, flow and utilization methods across the entire organization and business. By designing and continuously reviewing data architecture that encompasses the entire data lifecycle in alignment with business strategy, they ensure the successful integration of company-wide data utilization and governance—essentially serving as the overall designer of data.

The CIO is not merely responsible for establishing data storage and analysis infrastructure; they are also tasked with appropriately assigning personnel to these three roles within the company and establishing cross-departmental, company-wide tools and rules to connect data with management, business operations and daily tasks.

Ultimately, the success of data utilization depends on organizational culture

On the other hand, no matter how much progress is made in staffing, infrastructure, tools and rulemaking, data will not be utilized unless there is an organizational culture that actively drives management, business and operations based on data.

• The purpose of data entry is not understood
• Data is optimized solely for the department’s own operations
• Decision-making based on data is not valued

In such a situation, no matter how well the systems are set up, they will become mere formalities.

In contrast, in organizations where data utilization is advanced:

• Discussions are based on data
• Formulate hypotheses and verify them with data
• And continuously improve based on data

These actions occur naturally.

In other words, the essence of data management ultimately lies in creating an organizational culture that assumes the effective use of data.

Data management cannot be achieved overnight. That is precisely why it is important to start small and build on your successes.

• Organize data for specific tasks and achieve results through the use of AI
• Rolling out successful practices
• Gradually Expand the Scope

By repeating this cycle, the importance of data will permeate the entire organization.

The role expected of a CIO in the AI era

In the AI era, the role expected of a CIO has changed significantly.

Traditionally:

• Ensuring the stable operation of systems
• And optimizing costs

However, moving forward:

• We will view data as an asset and maximize its value
• Developing the data infrastructure, tools and rules that underpin AI adoption, and advancing personnel allocation and development
• And fostering an organizational culture that embraces data utilization —roles that are more directly linked to business management

In other words, the CIO must evolve into the person responsible for creating value from data.

Data is the source of competitive advantage

In the coming era, the use of AI will be a given. What will set companies apart is not whether they use AI, but what data they possess.

Data is the accumulation of a company’s past strengths and the source of future value creation. And its quality is determined by daily operations and the nature of the organization.

• AI grows by being fed good data
• And companies grow through that AI

Taking this simple principle as our starting point, we must place data management at the core of our business strategy. Isn’t that the shortest route to sustainable growth in the AI era?

CIOs are called upon to lead the way in making this a reality.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Generative AI is a kind of artificial intelligence that creates new content, including text, images, audio, and video, based on patterns it has learned from existing data.

Today’s generative models are typically built on foundation-model architectures such as large-language models (LLMs) and multimodal systems, enabling them to carry on conversations, answer questions, write stories, generate code, and produce images or videos from brief prompts.

Generative AI is different from discriminative AI, which draws distinctions between different kinds of input. Where discriminative AI answers questions like “Is this image of a rabbit or a lion?”, generative AI instead responds to prompts such as “Describe to me how a rabbit and lion look different from one another” or “Draw me a picture of a lion and a rabbit sitting next to each other” — and in both cases produces text or imagery that, while grounded in the AI’s training data, isn’t just a copy of something that already existed.

Just a few years ago, generative AI was once a novelty focused on chatbots and artistic image generation. Today, it has become a core enterprise technology, and powers everything from content creation and software development to customer support and analytics workflows. But with that power comes a new set of challenges — from model alignment and hallucination to governance and data-integration hurdles.

In this article, we’ll look at how generative AI works, explore how it has evolved into the foundation-model era, examine how to implement it effectively, and offer best practices for getting value out of it, today and in the future.

How does generative AI work?

For decades, early artificial-intelligence efforts often focused on rule-based systems or narrowly trained models that were built for one task at a time. While these efforts produced useful systems that could reason and solve human tasks, they were generally a far cry from sci-fi visions of thinking machines. Programs that could talk to people never seemed to get very far past the level of ELIZA, a “computer therapist” created at MIT in the mid 1960s; even Siri and Alexa after much fanfare were revealed to be fairly limited.

The big structural shift that gave birth to modern generative AI came with the concept of a transformer, first introduced in “Attention Is All You Need,” a 2017 paper from Google researchers.

Using a transformer architecture as a basis, you can build a system that derives meaning from analyzing long sequences of input tokens (words, sub-words, bytes) to understand how different tokens might be related to one another, then determines how likely any given token is to come next in a sequence, given the others. In AI lingo, we call these systems models. Because a model analyzes very large datasets and parameter counts, it can pick up on statistical patterns and knowledge implicitly embedded in the data.

This is all easier said than done. The process of adjusting a model’s internal parameters so it gets better at predicting the next token in sequences is called training. During training, the model repeatedly guesses the next token in a given sequence, compares its prediction to the actual one, measures the error, and updates its parameters to reduce that error across billions of examples. Over time, that process teaches the model the statistical relationships that will allow it to generate coherent language (or code, or images) later.

What is a foundation model?

You’ll often hear the word large used for transformer-based models of these types, like the LLMs we mentioned earlier. Large in this context refers to the large number of internal numerical values that the model adjusts during training to represent what it has learned, along with breadth and diversity of data used to train the model and the underlying compute resources powering this whole process.

This is in contrast with the narrow models of the earlier era of AI/ML, which werebuilt for one purpose and trained on a limited dataset. For instance, a spam filter may be very good at what it does, but it’s only trained on email data and all it can do is classify emails. Large models, by contrast, serve as what’s known as foundation models. They’re trained broadly on diverse data (text, code, images, or multimodal data) and then adapted or specialized for many downstream tasks.

These foundation models are the basis for most of the popular generative AI tools and services on the market today. They can be specialized in several ways:

• Fine-tuning: Giving a foundation model further training on a smaller, task-specific dataset
• Retrieval-augmented generation (RAG): Giving the model the ability to pull in external knowledge when asked a question
•  Prompt engineering: Tailoring a query so the model gives the sort of answers you’re looking for.

How do AI systems write computer code?

One of the surprising discoveries of the gen AI era was that in recent years was that foundation models trained on natural-language text can also, when fine-tuned with code examples, also write computer code — often better than many purpose-built systems. Still, it makes sense, when you think about it — after all, high-level computer languages are designed by humans and ultimately based on human language.

This 2023 InfoWorld article highlights how models like PaLM, LLaMA and other transformer-based systems fine-tuned on code repositories propelled this shift, but since AI giants like OpenAI have moved into this space. This all matters because code generation (or code-assisted productivity) has become a key enterprise use case of generative AI — perhaps the key use, given the industry’s enthusiastic adoption of it.

What are AI agents?

So far, we’ve been talking about chatbots, writing assistants, image-generation tools. They respond to prompts, output text or images, and then stop. A new category of tool called agentic AI goes further: it plans, executes, and in many cases learns as it works.

Because large models already understand language, code, and even structured data to some extent, they can be repurposed to generate not only descriptive text but operational instructions. For example: an agent might parse the intent “generate a sales-report”, then format internal calls like getData(salesDB, region=NA, period=lastQuarter), and then call an API, all by generating text that’s interpreted as instructions. The MCP framework standardizes the “language” of those instructions and the plug-points into tools and data so that the model doesn’t need bespoke integrations for each new workflow.

These kinds of autonomous agents have several enterprise use cases:

• Software automation: Agents that generate code, call unit tests, deploy builds, monitor logs and even roll back changes autonomously.
• Customer support: Instead of simply drafting responses, agents interact with CRM APIs, update ticket statuses, escalate issues, and trigger follow-up workflows.
• IT operations/AIOps: Agents monitor infrastructure, identify anomalies, open/close tickets, or auto-remediate based on defined rules and context from logs.
• Security: Agents may detect threats, initiate alerts, isolate compromised systems, or even attempt to manage threat containment — though this raises new risks.

How can you implement generative AI in the enterprise?

We’ve now touched on what generative AI can do. But how can you make it work reliably in your business. The difference between a pilot and full-scale deployment often comes down to systems, structure and governance as much as to models themselves. InfoWorld’s Matt Asay offers a deep dive into enterprise gen AI essentials, but here are some important points to keep in mind:

Choosing between API, open-source or custom fine-tuned models. One of the first major decisions for any enterprise project is: do you use a model via an API (e.g., from a vendor like OpenAI or Anthropic), deploy an open-source model internally, or build/fine-tune a custom model yourself? Each has trade-offs.

APIs offer speed and minimal setup, but may expose data, limit customization or accrue high cost — and will leave you at the mercy of your vendor. Open source allows internal control and may ease fine-tuning, but requires infrastructure, expertise, and support. Custom fine-tuning gives you the tightest alignment to your use-case, but lengthens time to value and increases risk.

Governance, data privacy and compliance. Deploying generative AI in an enterprise setting raises new governance, privacy and regulatory issues. For example: Who owns the data that’s ingested? How is proprietary data protected if you call a third-party API? What traceability exists for model outputs—a huge question for regulated industries? One useful framework is covered in “A GRC framework for securing generative AI” Data governance must adapt for the new era,  and new frameworks are evolving to help.

Human-in-the-loop review. Even the best models make mistakes and cannot simply be put on autopilot. You need a human-in-the-loop (HITL) process: real people need to review outputs, validate for bias, approve high-stakes content, and tune prompts or models based on feedback. Incorporating HITL checkpoints helps mitigate risk and improve overall quality.

Integration with existing systems and RAG pipelines. Retrieval-augmented generation, which we touched on earlier, connects foundation models into business workflows, systems, and enterprise data stores. RAG can bind LLMs to your organization’s internal knowledge bases, thereby reducing hallucinations (which we’ll discuss in a moment) and increasing the relevance of gen AI output.

What causes AI hallucinations?

Probably the biggest limitation of generative AI is what those in the industry call hallucinations, which is a perhaps misleading term for output that is, by the standards of humans who use it, false or incorrect.  

Every generative AI system, no matter how advanced, is built around prediction. Remember, a model doesn’t truly know facts—it looks at a series of tokens, then calculates, based on analysis of its underlying training data, what token is most likely to come next. This is what makes the output fluent and human-like, but if its prediction is wrong, that will be perceived as a hallucination.

Foundry

Because the model doesn’t distinguish between something that’s known to be true and something likely to follow on from the input text it’s been given, hallucinations are a direct side effect of the statistical process that powers generative AI. And don’t forget that we’re often pushing AI models to come up with answers to questions that we, who also have access to that data, can’t answer ourselves.

In text models, hallucinations might mean inventing quotes, fabricating references, or misrepresenting a technical process. In code or data analysis, it can produce syntactically correct but logically wrong results. Even RAG pipelines, which provide real data context to models, only reduce hallucination—they don’t eliminate it. Enterprises using generative AI need review layers, validation pipelines, and human oversight to prevent these failures from spreading into production systems.

What are some other problems with generative AI?

Generative AI has proven to be such a disruptive technology that’s stoking near-apocalyptic fears that it will result in a superintelligence that will enslave or destroy humanity. Meanwhile, in the present day, increasingly troubling reports of so-called AI psychosis are emerging, where people have mental health episodes triggered by the uncanny and sometimes sycophantic ways chatbots affirm whatever you talk to them about and try to keep the conversation going.

Compared to such existential questions, the following business-related problems may seem petty. But they’re real issues for enterprises considering investing in AI tools.

• Data leakage and regulatory risk. When a model is fine-tuned or prompted with sensitive information, that data may be memorized and unintentionally reproduced. Using third-party APIs without strict controls can expose proprietary or personally identifiable information (PII). Regulatory frameworks like GDPR and HIPAA require explicit governance around where training data resides and how inference results are stored.
• Prompt injection occurs when an attacker manipulates a model’s instructions—embedding hidden directives or malicious payloads in user input or external content the model reads. This can override safety rules, expose internal data, or execute unintended actions in agentic systems. Guardrails that sanitize inputs, restrict tool-calling permissions, and validate outputs are becoming essential.
• Copyright and content ownership. Many foundation models are trained on data scraped from the public internet, creating disputes over copyright and data provenance. Enterprises using generated output commercially need to confirm usage rights and review indemnity terms from vendors.
• Unrealistic productivity expectations. Finally, organizations sometimes expect generative AI to deliver instant productivity gains. The reality, it turns out, is more mixed. Enterprise adoption requires infrastructure, governance, retraining, and cultural change. The models accelerate work once properly integrated, but they don’t automatically replace human judgment or oversight.

The current generation of enterprise AI systems includes several layers of defense against these risks:

• Guardrails that constrain model behavior and filter unsafe outputs.
• Model validation frameworks that measure factual accuracy and consistency before deployment.
• Policy layers that enforce compliance rules, redact sensitive data, and log model actions.

These safeguards reduce—but don’t remove—the inherent uncertainty that defines generative AI.

GenAI: essential for the enterprise

Generative AI has evolved from a novelty into a core layer of enterprise technology. Foundation models and agentic systems now power automation, analytics, and creative workflows — but they remain fundamentally probabilistic tools. Their strength lies in scale and adaptability, not perfect understanding.

For organizations, success depends less on chasing model breakthroughs than on integrating these systems responsibly: building guardrails, maintaining oversight, and aligning them with real business needs. Used wisely, generative AI can amplify human capability rather than replace it.

Every day this past week I did something I suspect millions of other people also did: I stared at an LLM model picker and wondered which one I was supposed to want.

OpenAI just released ⁠GPT-5.6 Sol, Terra, and Luna. Sol is the flagship. Terra offers much of its intelligence for less money. Luna is cheaper still. Anthropic released ⁠Claude Sonnet 5 at the end of June and Opus 4.8 the month prior, with a little Fable 5 emerging in between. Meanwhile, Google, which seemed to be winning the model wars a few months ago, is now getting shade from Gergely Orosz, who ⁠argues that Gemini has slipped outside the top tier for software development and has been out of the major model release game for eons (May 19).

Perhaps Orosz is right. Perhaps he’ll be wrong again in six weeks. Honestly, it’s exhausting.

I use ChatGPT and Claude constantly and still have no principled idea which model to choose most of the time. I tend to click whatever looks like the biggest, most expensive option because I don’t know what I’m giving up by choosing something smaller. “Instant” sounds dangerously unserious. “Thinking” sounds expensive but powerful.

A quick survey of my LinkedIn crowd suggests others also feel my “WHICH MODEL???” pain. More importantly, I suspect most enterprises do, too.

A model doesn’t rot

Before getting carried away, however, it’s worth considering whether any of this model churn actually matters. After all, a model doesn’t rot. The model an enterprise put into production in March performs just as well in July as it did when the company selected it. “Obsolete” generally means that something better now exists, not that the deployed model suddenly stopped summarizing insurance claims or classifying support tickets. (In other words, once you have something working, the idea that “but maybe Opus 200.2 is better!” is really a FOMO problem, not a performance issue.)

Most enterprise workloads don’t live at the frontier anyway. Extraction, summarization, classification, document comparison, and customer-service assistance often work perfectly well with smaller, cheaper models. OpenAI’s own pitch for the trio of GPT-5.6 models isn’t simply that Sol is better. It’s that ⁠Terra and Luna deliver different combinations of intelligence, latency, and cost. Luna, the cheapest tier, nearly matches the previous generation’s peak performance at less than half the estimated cost, according to OpenAI.

The practical question, of course, is where to start. An enterprise can’t test every model, every reasoning setting, and every price tier before doing any work. So here’s my advice (which I don’t follow in my own work, but I’m not defining enterprise strategy and can be a little price-insensitive). Start with the cheapest credible model that appears capable of the task. Give it a representative set of real examples and, before you start testing, define what counts as good enough. If it passes, stop. If it fails, move up a tier or try a model with strengths better suited to the work.

That sounds almost offensively simple, but it reverses the way many people, including me, use these products. We start with the biggest model because we’re afraid of what we might lose. Enterprises should start lower and require evidence before paying for more intelligence.

There are exceptions, of course. For genuinely difficult work, such as autonomous coding, complex research, or high-stakes reasoning, beginning with a frontier model may save time. But even then, the goal should be to establish a quality ceiling, then test whether a cheaper model can meet it. It’s changing the question from “which model is best?” to “what is the least expensive model that reliably clears the bar for this job?”

For many workloads, that price improvement matters more than a few extra benchmark points. ⁠As I argued back in 2023, following AI hype doesn’t help anyone. If your model strategy depends on whichever benchmark screenshot is circulating on X this week, you don’t have a strategy. Not a viable one, anyway. Pick a model and ignore the noise.

Except, of course, when that noise suggests a serious signal.

Sometimes better really is better

Frontier improvements aren’t always incremental, making it advantageous to consider an upgrade. Coding is the obvious example. There’s a significant difference between a model that suggests the next few lines of code and one that can inspect a repository, plan a change, use tools, run tests, discover its own mistakes, and keep working for an extended period. That isn’t merely a nicer autocomplete experience. It can reorganize a development workflow.

This is why enterprises can’t simply standardize on an 18-month-old model and declare victory. In some areas, particularly software development and other agentic work, better models can unlock compounding productivity. A model that reliably completes 80% of a bounded task rather than 50% may justify an entirely different division of labor between humans and machines.

Still, that upgrade isn’t free.

Models differ in how they interpret instructions, call tools, manage context, refuse requests, and fail. Prompts and scaffolding tuned for one model can regress when moved to another. Or costs can explode. As one of my Oracle colleagues discovered just this week, running the same tasks in GPT 5.6 was orders of magnitude more expensive than 5.5. The API change may be trivial, but the revalidation and implications are not.

This leaves enterprises caught between two bad options. They can freeze and potentially miss out on meaningful improvements or chase every release and repeatedly test production systems on faith. What to do?

Stop making model bets

The answer is to stop making LLM bets and start making job-to-be-done bets. Stop asking which model is fastest. Instead, figure out what work you are trying to improve. What does a good result look like? How much latency and cost can the workflow tolerate? How wrong can it be before a human must intervene? Once those questions have answers, model selection becomes less opaque.

A difficult code migration may justify GPT-5.6 Sol or Claude Sonnet 5. A repetitive classification task may work just as well with Luna or another smaller model. A regulated workflow may require a model or deployment option that offers particular data controls. Sometimes the correct model is no LLM at all, like when I’m writing this post. Sorry, AI vendors! (At least you won’t get blamed for my mistakes.)

This is where evaluations become the center of enterprise AI strategy. ⁠As I’ve said before, most companies don’t have an AI quality problem so much as an AI measurement problem. Hence, a private evaluation suite built from real company work is the only leaderboard that matters. Does the new model materially improve quality? If so, use it! Does it reduce cost or latency? Again, that’s your free pass to adoption. Does the improvement justify the expense and effort of revalidation? If yes, continue.

Make model releases boring

As important as the model is, keep in mind that AI success always comes back to your company’s data, your company’s workflows, your company’s integrations, etc. That’s the ⁠dull reality behind sexy AI. Retrieval, memory, governance, data quality, observability, and feedback loops aren’t as exciting as a new model launch, but they’re what ultimately make AI truly work.

Again, when it’s time to consider something new, the principle should be to default to the least expensive model that reliably passes your evaluations. Only escalate harder tasks to more capable models when measurement shows that the premium pays. Tip: Make this invisible to employees so that the system routes to the best model for a particular prompt. As dbt Labs’ Jon Lewis expresses it, “The best model is ‘Auto’ and I won’t hear anyone say otherwise.” OpenAI’s own ⁠migration guidance recommends testing models on representative tasks, including trying a lower reasoning level rather than automatically cranking everything to the maximum.

As for me, I’ll probably keep clicking the shiniest option. I don’t have a formal evaluation suite for InfoWorld columns, and the marginal cost is a subscription I already pay. Enterprises don’t get that excuse.

While many technology companies have slowed hiring or even launched significant layoffs, that doesn’t mean job opportunities have dried up for software developers. In fact, skilled developers—particularly those with knowledge of AI—are in demand in other industries.

The key to success for developers looking to snatch up these roles is to be well-prepared to meet the needs of potential employers in a variety of sectors.

“The demand for developers in non-tech sectors is real and growing, but the roles look different from what you’d find at a software company,” says Pragati Awasthi, assistant teaching professor of AI and data science at Drexel University.

“Across all these sectors, the common thread is that software is no longer a support function; it is embedded in core operations,” Awasthi says. “The developer in these environments is often the person translating domain-specific business problems into technical solutions, which requires a different profile than a pure product engineer at a tech firm.”

Opportunity knocks

The tech industry has long been a mainstay as far as employing software developers. But as these businesses trim staffs in efforts to cut expenses, that has impacted the hiring landscape. Even as the tech sector scales back, however, companies in industries such as financial services/fintech, healthcare/healthtech, retail/ecommerce, and manufacturing are looking to acquire programming talent.

“The unifying factor is data complexity,” Awasthi says. “These industries generate large volumes of sensitive, regulated, or operationally critical data, and they need developers who can build and maintain systems that handle it responsibly.”

While recruiting firm Summit Search Group has placed developers in roles with technology companies, “it is just as common to recruit them for roles outside this niche,” says Matt Erhard, managing partner at the company. “There are actually a fairly wide variety of roles available for developers in industries beyond tech,” Erhard says.

For example, in financial services Summit Search Group has seen significant hiring for back-end and data engineers who can build and maintain fraud detection systems, digital banking platforms, and regulatory tools, Erhard says. In healthcare, companies are hiring developers to build AI-driven diagnostics platforms and patient portals, or to work with systems that manage electronic health records, he says.

In manufacturing and industrial companies, developers are needed for systems integration and embedded software related to predictive maintenance, Internet of Things (IoT) systems, and smart factories. And in retail and ecommerce, there’s strong demand for full-stack developers and data developers who can handle logistics systems, omni-channel platforms, and personalization engines, Erhard says.

“One significant function where we’ve been placing developer talent lately is in developing business systems and internal applications,” Erhard says. These roles often have titles such as systems engineer or application developer, and professionals are hired to handle tasks such as customizing customer relationship management (CRM) or enterprise resource planning (ERP) platforms, building workflow automation tools or modernizing legacy systems, he says.

Other core functions for which Summit Search Group has placed a lot of developers include data, analytics, and AI-enablement. “That could be directly involved with data engineering or in building tools like reporting systems and ETL [extract, transform, load] pipelines,” Erhard says.

The firm also has handled searches for developers who can build and maintain customer-facing products for banking, healthcare, and retail companies, such as mobile apps or digital platforms customers can use to interact with companies.

Randstad Digital, a provider of global technology talent, sees demand for roles including web developers, system developers, and app developers. “These professionals would work on anything from customer-facing platforms to internal tools,” says Michael Morris, global head of platform and talent at the company. “Non-tech companies are also often hiring roles like software architecture and devops to help scale existing technology. These involve being more ingrained in the business, like building a supply chain system for a retailer, rather than creating individual tech products like you would at a technology company.”

Prep for success

To increases the chances of success at landing developer jobs outside of the tech industry, development professionals would be wise to follow some good practices.

Boost AI skills

One best practice is to boost skills in using AI-powered tools and get familiar with all things AI.

“Get fluent with AI-assisted development and its limits,” Awasthi says. “This is not optional. Organizations across every sector expect developers to use AI coding tools productively. But the more durable skill is knowing when AI output is wrong, incomplete, or unsuitable for a regulated context. That critical evaluation capacity is what non-tech employers are increasingly trying to hire.”

AI does not necessarily replace the need for human developers so much as it changes the skills profile for those roles, Erhard says. “The biggest difference in recent years is that AI literacy is now a non-negotiable,” he says. “At minimum, developers today need to understand concepts like prompt engineering and how to use AI tools to improve their efficiency.”

One thing many job candidates don’t expect is that the rise of AI has also increased the importance of high-level skills such as problem framing, system design, and cross-functional communication,” Erhard says. “Essentially, if something is related to development but too complex or nuanced for an AI to handle effectively, then the demand is high for human developers who have that expertise,” he says.

Candidates who land roles consistently have experience building AI-augmented workflows along with standard coding skills, Erhard says. “Employers increasingly expect to hire developers who can leverage AI, so demonstrating this experience on your résumé can be very beneficial,” he says.

Gain domain knowledge

Summit Search Group is seeing high demand for developers with deep domain knowledge in an organization’s specific industry. “So, for instance, if someone is both an experienced developer and has expertise in healthcare compliance, or financial regulations, then those candidates tend to be very sought after,” Erhard says.

Domain fluency is an underrated skill, Awasthi says. “A developer who understands healthcare compliance, financial regulation, or manufacturing process logic is significantly harder to replace than one who only writes clean code,” she says. “AI can generate boilerplate. It cannot navigate a HIPAA audit or explain a model’s output to a compliance officer.”

Development professionals should “pick an industry and learn it seriously; not just the technology stack but the regulatory environment, the business model, and the actual problems practitioners face,” Awasthi says. “A developer who has read about HIPAA, or spent time understanding credit risk, is immediately more valuable in those hiring contexts.”

It’s also vital to demonstrate real-world, practical application of skills, not just credentials. “The strongest candidates have projects in their portfolio that directly tie to and solve real business problems,” Erhard says.

Acquire soft skills

And then there are the soft skills that are becoming more of a differentiator than they were in the past. As AI handles more routine coding, human developers are expected to make more architectural decisions and collaborate across departments, Erhard says. “Strong communication and problem-solving skills are critical for many of the developer roles that we’re filling today,” he says.

While technical skills are still relevant for developers using and managing AI tools, “they also need to develop the skill of ‘deeper thinking’ and learn how to think one step ahead,” Morris says. “This includes skills like system design mastery—understanding the macro view and learning how microservices, databases, and third-party APIs interact securely and efficiently.”

They also should become deeply fluent in the AI coding tools commonly used in their particular industry, with a strong understanding of how to prompt them for optimal output, Morris says. Product context awareness is also useful. “AI doesn’t know what the customer wants, but you do,” Morris says. “Understanding the business problem and the end-user experience is a requirement for being able to guide LLMs.”

Master debugging and incident response

Developers looking to break into non-tech sectors also should develop skills in debugging and incident response, Morris says. “Complex systems with multiple AI agents can, and will, fail, which means companies need humans to trace logic flaws to get the system back on track,” he says. “A mastery of root-cause analysis is a critical skill.”

“Security, compliance, and reliability are very important in non-tech industries like finance and healthcare,” says Rohit Agarwal, co-founder of Zenius, a remote hiring company. “So employers want developers who also know regulatory environments well.”

Network and keep learning

To successfully pivot from jobs at tech companies, “continuous learning, upskilling, and building hybrid skills that combine technical and business knowledge are essential,” Morris says. “With the right preparation, tech professionals can adapt and continue to thrive in meaningful, dynamic careers.”

It’s also a good idea to join talent communities in fields of interest and “engage with other members in conversations that increase your knowledge through the collective intelligence of the community,” Morris says. “Take advantage of AI skilling opportunities relevant for your role, or better yet, where you want to go next. Experiment with the technology either on your own or through structured programs.” Ultimately, be curious and proactive, he says.

“I’d also recommend developers not to ignore referrals, direct outreach, and industry-specific communities during job search,” Agarwal says. “There are often a lot more opportunities available than the ones posted online.”

A product manager on my team recently asked me where we were seeing the most issues across the engineering team. Instead of guessing, I had an engineering lead point Claude at our Jira via an MCP connector and look at the bug patterns himself.

One team had a wildly disproportionate share of tickets — about 50% of their sprint time was spent on “bugs,” versus roughly 25% for everyone else. The headline number suggested a quality problem.

It wasn’t. When we layered in the context around those tickets, almost none of them were bugs. They were manual workarounds for a missing product capability: customers asking us, one request at a time, to restore items they had accidentally deleted. Not shipping an item restore feature was burning roughly 1.5 engineers’ worth of capacity. I went back to our product team and said, “Build this, and you reclaim a person and a half.”

The analysis took 45 minutes. It was only possible because our data was already organized, tagged by team, connected to contributors, accessible through MCP and protected by role-based access. None of that is “AI.” All of it is the layer underneath AI that almost nobody invests in first. That’s probably because the investment is unglamorous: updating data dictionaries, access controls, team taxonomies, system-to-system mappings. Most of the work has been the same for twenty years. AI just raised the cost of skipping it.

The intelligence underneath the models

I keep coming back to the value of context data layers as a CTO in the middle of an AI rollout. I have started calling that value proposition contextual intelligence because I haven’t found a better name. Anthropic’s engineering team has been calling this kind of work “context engineering” since late 2025, and CIO ran its own feature on the term shortly after. Whether you describe it as contextual intelligence or context engineering, it’s the part of the stack where the actual programming work still lives.

If business logic is your company’s official org chart, then contextual intelligence is knowing who actually gets things done, how decisions are actually made and what the unwritten rules are. One is theory. The other is reality.

Most enterprise systems capture the theory. The systems that capture how work actually happens — what people do, how teams operate, where decisions get stuck — are rarer and harder to build. And modern LLMs, it turns out, are useless without both.

I learned this the hard way at a recent company hackathon. Nine engineering teams, one prompt: make our operational dataset more usable through AI. My team built persona-based chatbots (CFO, CIO, sales manager) on top of an MCP server backed by Postgres and our enrichment data. Other teams built dashboard generators, Looker conversational analytics and workflow agents.

The initial demos all had the same problem. Claude could talk to our data, but the answers were either generic or confidently wrong. The CFO persona would happily report a “spend trend” that quietly conflated two distinct cost categories across two different tables. The CIO persona would answer questions about team productivity, but the averages across roles should never have been aggregated. The sales manager persona returned answers that were technically correct against the schema and completely wrong against the business. The raw data was rich. The context layer around it didn’t exist yet. Chatting with raw data is not an AI product. It’s a demo.

One of my senior engineers spent the second day ripping out the agent’s direct database connection. He stopped trying to prompt-engineer the LLM to understand our business and instead codified that logic into the data pipeline. Working backward from the failed CFO answers, he mapped out the implicit knowledge an experienced controller relies on: Explicitly defining which legacy tables actually represent ‘spend,’ writing the rules for currency normalization and hardcoding our fiscal time windows. He built a series of semantic SQL views to enforce these rules and restricted the MCP server to exposing only this curated layer. When we pointed the same model at those same questions, it returned completely different answers. They were specific, evidence-based and grounded in our actual business reality. The model didn’t get smarter. The engineering beneath it did.

The same pattern shows up everywhere I look right now

McKinsey keeps publishing that software development tops enterprise AI use cases, with companies reporting 30–50% productivity gains in pilots. The pilot numbers are real. They rarely translate to top- or bottom-line impact in production. Our own company data tells the same story: Between Q1 2025 and Q1 2026, our total AI tool usage grew by 328% (over 4x). Over that same period, PR throughput grew by just 49%.

That gap — adoption way up, outcomes inching along — is the context gap. Plug a generic agent into raw, uninterpreted data, and it will act inefficiently at best, harmfully at worst. An agent optimizing sales without your customer segmentation or product hierarchy will confidently recommend the wrong thing. Anthropic framed the shift directly: building with language models is becoming “less about finding the right words and phrases for your prompts, and more about answering the broader question of what context configuration is most likely to generate our model’s desired behavior.” That second question — what context configuration  — is the entire game. Most organizations are still answering the first one.

Where the work actually lives

A growing number of CTOs I talk to are shifting their AI investments accordingly. Less attention on the model. More on the layer between the model and the data.

When peers ask me what that actually looks like day-to-day, I tell them I give every engineering role the same mandate: the LLM should never see raw, uncontextualized data.

In practice, that breaks down to three pieces of work, none of them glamorous.

The first is semantic middleware. We need code that transforms raw data into business-meaningful signals before it ever reaches the model. Our feature stores hold things like “employee code velocity on critical-path features,” not “X logged 50 Git commits.” The work of figuring out what “critical-path” means in our product, in our org, on this team is the work. It does not get cheaper because the model has gotten better.

The second is multi-agent design. Instead of one omniscient orchestrator, we run smaller agents scoped to specific domains, each with rules that catch the failure modes the main model is known for. We pair them with RAG that retrieves precomputed insights, with their rules attached, rather than raw documents. Validation checkpoints sit between steps and flag suggestions that violate known constraints, such as averaging productivity across completely different job functions. The guardrails are not there to be clever. They are there because we already watched the model make those exact mistakes.

The third is evaluation that takes business logic seriously. When I look at a model, general benchmark accuracy is the least interesting number. I want to know whether it respects our constraints and integrates cleanly with our existing architecture. That sometimes means fine-tuning our patterns, sometimes constitutional approaches to embed principles, sometimes hybrid systems where deterministic rules sit alongside the probabilistic ones. The throughline is the same: validate against reality, not against the benchmark.

Why this matters now

The reason this matters more now than it did six months ago is that adoption is moving faster than measurement, let alone integration. Model Evaluation & Threat Research’s (METR) developer productivity work tells the story in a way they didn’t intend. In early 2025, they ran a controlled study and found AI tools slowed experienced open-source developers by 19%. When they tried to repeat the study in late 2025, the experiment broke. Thirty to fifty percent of developers refused to submit tasks under the no-AI condition. They wouldn’t accept working without their tools. METR is now redesigning the study because the original methodology no longer holds up against how developers actually work. That’s how fast adoption moved. But I’d be willing to bet the organizational scaffolding required to convert that adoption into outcomes — context layers, workflow redesign, retraining around new tools — moved nowhere near as fast.

Get ahead with context

The teams I’ve seen succeed with AI built the context layer first. The teams I’ve seen struggle eventually built in context anyway, just at higher cost and with more scar tissue. Raw data is the new currency. But raw data without a context layer is cash sitting in a vault. It cannot act on anything. The difference between insight and noise is a layer of code that understands what your data means.

That layer is the work. It is where the next decade of competitive advantage will sit. And in my experience, the organizations that build it first are the ones that will actually get the productivity gains the rest of the market keeps promising.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

A new era of customer engagement is taking shape. AI agents are quickly becoming integral to the way businesses serve, support, and sell to customers — able to respond, reason, and take action in ways that go far beyond scripted automation.

Many customer journeys, however, are still built on systems that don’t talk to each other. Customer data lives in one place, channel history in another, and AI agents often operate with only part of the picture. Customers feel the pain when they switch between channels like voice and messaging, get transferred, and have to repeat themselves yet again. It doesn’t matter that they’ve been loyal to a brand for years, every interaction feels like a cold start. That is the conversation gap.

It’s clear that AI isn’t the problem, infrastructure is. Closing the gap requires new building blocks that focus on continuity, so context can carry forward across systems, channels, human agents, and AI agents.

To bridge the gap, at SIGNAL 2026, we are introducing a new conversation layer for the Twilio Platform.

Twilio Conversation Orchestrator, Twilio Conversation Memory, and Twilio Conversation Intelligence are now generally available. Together, they help businesses coordinate interactions, preserve context, and connect human and AI agents so every conversation is more continuous and useful.

In addition to the new Conversations layer, we’re also announcing platform updates that make it easier to build, manage, and scale customer engagement on Twilio — from a reimagined Twilio Console to expanded channels and new voice AI capabilities.

New building blocks for connected conversations

The conversation gap does more than create inconsistent customer experiences. It hurts conversion and retention, increases operational costs, adds integration complexity, and makes agents less productive. The new platform capabilities we’re introducing are designed to fix that by coordinating interactions, maintaining context, and surfacing signals as conversations happen.

Conversation Orchestrator

Conversation Orchestrator helps businesses coordinate interactions across Twilio channels without complex custom logic. Teams can configure it in Console or configure their implementation with the API. It connects interactions into a single thread and manages handoffs between human agents and automated systems.

Conversation Memory

Conversation Memory creates a living, identity-resolved profile by connecting customer data with conversation history and customer traits. That means each interaction starts with the right context. It’s built specifically for LLMs to reduce latency and token usage by surfacing the most relevant details when they matter.

A new Enterprise Knowledge API (now generally available) also allows teams to deliver more relevant experiences and ground interactions in trusted business knowledge such as FAQs, policies, and product documentation.

Conversation Intelligence

Conversation Intelligence provides real-time understanding of live interactions. Using prebuilt and custom LLM-based operators, it can detect changes in sentiment, flag potential escalations, and trigger action during a conversation, not only after it ends.

That gives teams the ability to respond sooner, support agents more effectively, and improve customer outcomes while the conversation is still in progress.

Together, these products help businesses create customer experiences that feel more connected across channels.

Open by design

Twilio remains neutral by design. We start with the premise that you know your business. We aren’t here to prescribe a model, framework, or data strategy. We provide the infrastructure that helps you build customer engagement in the way that works best for your business. You pick the model and agent runtime. You own the data.

That doesn’t mean you need to start from scratch, either. We partnered with Microsoft, AWS, and others to create blueprints that support faster development. We are also introducing an open-source developer toolkit, Twilio Agent Connect (now generally available), that lets your teams connect agents built on any LLM or framework directly to Twilio’s infrastructure.

For developers, this means more flexibility. For businesses, it means less lock-in and the ability to get value from existing investments. For partners, it means more ways to build with Twilio.

A new front door

We are also introducing a reimagined Twilio Console, because as customer engagement grows more complex, managing the infrastructure behind it should feel effortless.

The new Console is a single mission control center that brings your communications, identity, and data into one experience: one login, consistent logs across every surface, an intelligent Console Assistant, transparent billing insights, and streamlined compliance workflows that no longer slow you down.

Over the coming months, we’ll roll out this new Console experience to customers automatically. You can also opt in to gain early access.

More channels, more control, smarter conversations

In addition to these launches, we are announcing several updates that expand customer reach, support enterprise requirements, and make it simpler to build on Twilio.

• Apple Messages for Business (Private beta) and Twilio Email (GA) give teams new ways to reach customers on the channels they already use.
• Data Residency for SMS (EU) (Public beta) enables teams to manage personal data locally to support regional data requirements.
• Conversation Relay enhancements add PCI compliance, HIPAA eligibility, Insights, and support for Deepgram Flux for smarter turn detection — helping AI agents better understand when a person has finished speaking.
• Stripe Projects integration enables developers and AI agents to seamlessly provision Twilio within Stripe Projects in a single, programmable CLI workflow.

Built with our customers

Bringing these new products to life required a close partnership with many beta customers and partners. This helped us understand real-world signals and needs to help make the capabilities robust from the start.

Among dozens of others, Centerfield, Constellation Dealerships, Car Finance 247, and Meera.ai leveraged Twilio to solve their own customer engagement challenges. These teams showed what is possible when businesses carry context forward, act on live conversation signals, and connect AI agents with human teams in the moments that matter.

Car Finance 247, a leading UK online car finance broker, is using Twilio to help recover stalled loan applications. When customers miss a field, need to correct information, or still need to confirm terms and conditions, AI-powered outreach across voice, SMS, and RCS, Conversation Memory tracks the application state. Conversation Orchestrator manages the outreach journey, and Flex helps bring in a human agent as needed. As Reg Rix, Co-Founder and CEO, shared:

“Because the platform remembers where each customer left off, we can pick up right where they stopped, helping them cross the finish line in a way that is modern, responsive, and genuinely helpful.”

Centerfield, a technology company powering AI-driven commerce, helps brands connect with consumers across digital and phone-based journeys. With Twilio, the team is connecting real-time conversation data with customer context to guide agents and AI systems in the moment, standardise what works, and improve performance at scale. As Aniketh Parmar, Chief Technology Officer, said:

“Performance comes down to how well every interaction moves a customer forward. We’re capturing each conversation in real time and applying what we already know about the customer to guide our agents and AI systems in the moment. With the Twilio Platform, including Conversation Orchestrator, Conversation Memory, and Conversation Intelligence, we can see what’s driving conversations so we can standardise what works, eliminate what doesn’t, and continuously improve outcomes at scale.”

Constellation Dealerships is using Twilio’s agent infrastructure to accelerate AI-powered engagement across its dealer network, moving from evaluation to measurable outcomes in days. As Richard Pineault, Director of R&D, shared:

“The value of this partnership is evident—our team progressed from evaluating Twilio’s agent infrastructure to realising measurable outcomes within days. This rapid speed-to-value exemplifies the agility and innovation required to propel the dealership industry into the future.”

Meera.ai is building on Twilio to modernise outbound engagement, replacing repeated manual follow-ups with always-on conversations across voice, SMS, and messaging. Vivek Zaveri, Chief Executive Officer, said:

“Meera.ai has partnered with Twilio since our inception to champion a conversation-first future for commerce. As the industry shifts toward real-time LLM-enabled interactions, Twilio’s Platform and the new Conversations products will help us reach customers in the moment.”

Together, these customers and partners show that the Twilio Platform can help businesses recover stalled journeys, improve live interactions, accelerate time to value, and create more connected experiences across AI agents, human teams, and every customer channel.

The next era of customer engagement starts here

As AI agents own more of customer engagement, businesses need infrastructure that keeps conversations connected across channels, systems, and teams. That means preserving context, coordinating handoffs, and acting on what is happening in real time.

That is what we are building with this next generation of the Twilio Platform: a new layer that connects channels, context, intelligence, and human and AI agents, helping businesses make every digital interaction more connected, more useful, and more amazing.

For 17 years, Twilio has helped builders create better ways for businesses to connect with their customers. In this next era, that connection matters more than ever.

Explore the new Conversations layer, try the products, and let’s build what comes next, together.

FOTOGRIN – shutterstock.com

Für viele Geschäftsprozesse ist Technologie inzwischen unverzichtbar. Deshalb zählt diese auch zu den wertvollsten Assets eines Unternehmens. Leider stellt sie gleichzeitig jedoch auch eines der größten Risiken dar – was Risk-Assessment-Frameworks auf den Plan ruft.

IT-Risiken formal zu bewerten, ermöglicht es Organisationen, besser einzuschätzen, zu welchem Grad ihre Systeme, Devices und Daten schädlichen Einflüssen ausgesetzt sind. Etwa in Form von Cyberbedrohungen, Compliance-Verfehlungen oder Ausfällen. Zudem können IT- und Sicherheitsentscheider deren Folgen mit Hilfe von entsprechenden Rahmenwerken auch besser abzuschätzen. Das Ziel besteht am Ende darin, sämtliche identifizierten Risiken – und ihren Impact – zu minimieren.

In diesem Artikel stellen wir Ihnen (in aller Kürze) sechs populäre Risk-Assessment-Frameworks vor, die jeweils auf spezifische Risikobereiche abgestimmt sind.

1. COBIT

Das ist es: Hinter COBIT (Control Objectives for Information and Related Technology) steht der internationale IT-Berufsverband ISACA, der sich auf IT Governance fokussiert hat. Dieses sehr umfassende und breit angelegte Framework wurde entwickelt, um dabei zu unterstützen, Enterprise IT:

• zu verstehen,
• zu designen,
• zu implementieren,
• zu managen und
• zu steuern.

Das kann es: Laut ISACA definiert COBIT die Komponenten und Designfaktoren, ein optimales Governance-System aufzubauen und aufrechtzuerhalten. Die aktuelle Version, COBIT 2019, fußt auf einem Governance-Prinzipien-Sextett:

1. Value für Stakeholder liefern
2. ganzheitlichen Ansatz realisieren
3. Governance-System dynamisch gestalten
4. Management von Governance trennen
5. auf individuelle Unternehmensanforderungen abstimmen
6. Ende-zu-Ende-Governance-System realisieren

So funktioniert es: Das COBIT-Framework ist auf Business-Fokus konzipiert und definiert eine Reihe generischer Prozesse, um IT-Komponenten zu managen. Dabei werden außerdem auch Inputs und Outputs, Schlüsselaktivitäten, Zielsetzungen, Performance-Metriken und ein grundlegendes Reifegradmodell festgelegt.

Gut zu wissen: Laut ISACA ist COBIT flexibel zu implementieren und ermöglicht Unternehmen, ihre Governance-Strategie anzupassen.

2. FAIR

Das ist es: Das Framework FAIR (Factor Analysis of Information Risk) bildet eine Methodik ab, um unternehmensbezogene Risiken zu quantifizieren und zu managen. Dahinter steht das Fair Institute, eine wissenschaftlich ausgerichtete Non-Profit-Organisation, die sich dem Management von betrieblichen und sicherheitstechnischen Risiken verschrieben hat. Laut den Machern ist FAIR das einzige, quantitative Standardmodell auf internationaler Ebene, um diese Art von Risiken zu erfassen.

Das kann es: FAIR bietet ein Modell, um die genannten Risiken in finanzieller Hinsicht zu verstehen, zu analysieren und zu quantifizieren. Laut dem Fair Institute unterscheidet es sich dabei insofern von anderen Risk-Assessment-Frameworks, als dass es seinen Fokus nicht auf qualitative Farbdiagramme oder numerisch gewichtete Skalen legt. Stattdessen will FAIR eine Grundlage liefern, um einen robusten Risikomanagement-Ansatz auszubilden.

So funktioniert es: FAIR ermittelt in erster Linie Wahrscheinlichkeiten mit Blick auf die Frequenz und das Ausmaß von Data-Loss-Ereignissen. Es handelt sich hierbei nicht um eine Methodik, um individuelle Risikobewertungen durchzuführen. Vielmehr will das Framework Unternehmen in die Lage versetzen, IT-Risiken zu verstehen, zu analysieren und zu messen.

Zu den Komponenten des FAIR-Frameworks gehören:

• eine Taxonomie für IT-Risiken,
• eine standardisierte Nomenklatur für Risiken,
• eine Methode um Datenerfassungskriterien zu definieren,
• Messskalen für Risikofaktoren,
• eine Engine für Risikoberechnungen, sowie
• ein Modell, um komplexe Risikoszenarien zu analysieren.

Gut zu wissen: Die quantitative Risk-Assessment-Ansatz von FAIR ist branchenübergreifend anwendbar.

3. ISO/IEC 27001

Das ist es: Bei ISO/IEC 27001 handelt es sich um einen internationalen Standard, der mit Leitlinien in Sachen IT-Security-Management unterstützt. Ursprünglich wurde er im Jahr 2005 gemeinschaftlich von der International Organization for Standardization (ISO) und der International Electrotechnical Commission (IEC) veröffentlicht – und wird seither sukzessive überarbeitet.

Das kann es: ISO/IEC 27001 ist laut den Verantwortlichen ein Guide für Unternehmen jeder Größe und aus sämtlichen Branchen, um ein Information Security Management System (ISMS) aufzusetzen, zu implementieren, zu warten und fortlaufend zu verbessern.

So funktioniert es: ISO/IEC 27001 fördert einen ganzheitlichen Cybersicherheitsansatz, der Menschen, Richtlinien und Technologie auf den Prüfstand stellt. Ein auf dieser Grundlage erstelltes ISMS ist laut ISO ein Tool für Risikomanagement, Cyberresilienz und Operational Excellence.

Gut zu wissen: ISO/IEC-27001-konform zu sein bedeutet, einem weltweit eingesetzten Standard zu genügen und Datensicherheitsrisiken aktiv zu managen.

4. NIST Risk Management Framework

Das ist es: Das Risk Management Framework (RMF) wurde von der US-Behörde NIST (National Institute of Standards and Technology) entwickelt. Dieses Framework stellt einen umfassenden, wiederverwend- und messbaren, siebenstufigen Prozess in den Mittelpunkt, um IT- und Datenschutzrisiken zu managen. Dabei kommt eine ganze Reihe von NIST-eigenen Standards und Guidelines zur Anwendung, um die Implementierung von Risikomanagement-Initiativen zu unterstützen.

Das kann es: Laut NIST realisiert das RMF einen Prozess, der die Risikomanagementaktivitäten in den Bereichen Sicherheit, Datenschutz und Supply Chain in den Lebenszyklus der Systementwicklung integriert. Dabei berücksichtigt der Ansatz Effektivität, Effizienz und Einschränkungen durch geltende Gesetze, Direktiven, Anordnungen, Richtlinien, Standards oder Vorschriften.

So funktioniert es: Der siebenstufige Prozess des NIST RMF gliedert sich in.

1. wesentliche Aktivitäten, um die Organisation auf den Umgang mit Sicherheits- und Datenschutzrisiken vorzubereiten.
2. Systeme und Daten, die verarbeitet, gespeichert und übertragen werden, auf der Grundlage einer Impact-Analyse kategorisieren.
3. eine Reihe von Kontrollmaßnahmen auswählen, um Systeme auf der Grundlage einer Risikobewertung zu schützen.
4. Kontrollmaßnahmen implementieren – und dokumentieren, wie das vonstattengeht.
5. Kontrollmaßnahmen überprüfen und bewerten, ob diese wie gewünscht funktionieren.
6. Systembetrieb auf Grundlage einer risikobasierten Entscheidung autorisieren.
7. Implementierung und Systemrisiken kontinuierlich überwachen.

Gut zu wissen: Das RMF bietet einen verfahrenstechnischen und geordneten Prozess, um Organisation dabei zu unterstützen, Security in ihre allgemeinen Risikomanagement-Prozesse einzubetten.

5. OCTAVE

Das ist es: OCTAVE (Operationally Critical Threat, Asset, and Vulnerability Evaluation (PDF)) ist ein Framework, um Risiken im Bereich der Cybersicherheit zu identifizieren und zu managen. Es wurde vom CERT-Team der Carnegie Mellon University in den USA entwickelt.

Das kann es: Dieses Risk-Assessment-Framework definiert eine umfassende Evaluierungsmethode. Diese ermöglicht Unternehmen nicht nur, missionskritische IT-Assets zu identifizieren, sondern auch die Bedrohungen, die mit diesen in Zusammenhang stehen und die Schwachstellen, die das erst ermöglichen.

So funktioniert es: Laut den Verantwortlichen ermöglicht die Zusammenstellung von IT-Assets, -Bedrohungen und –Schwachstellen Unternehmen, zu durchdringen, welche Daten wirklich bedroht sind. Mit diesem Verständnis ausgestattet, können die Anwender eine Schutzstrategie entwickeln und implementieren, um diese nachhaltig zu schützen.

Gut zu wissen: Das OCTAVE-Framework ist in zwei Versionen erhältlich.

• OCTAVE-S bietet eine vereinfachte Methodik, die auf kleinere Unternehmen mit flachen hierarchischen Strukturen ausgerichtet ist.
• OCTAVE Allegro ist hingegen ein umfassenderes Framework, das sich in erster Linie für große Unternehmen oder solche mit komplexen Strukturen eignet.

6. TARA

Das ist es: TARA (Threat Assessment and Remediation Analysis) stellt eine Engineering-Methodik dar, mit deren Hilfe, Sicherheitslücken identifiziert, bewertet und behoben werden können. Dieses Framework wurde von der Non-Profit-Organisation MITRE entwickelt.

Das kann es: Das Framework ist Teil des MITRE-Systemportfolios, das darauf ausgerichtet ist, die Cybersicherheitshygiene sowie die Resilienz von IT-Systemen in einem möglichst frühen Stadium (innerhalb des Beschaffungsprozesses) zu adressieren.

So funktioniert es: Das TARA-Framework nutzt einen Datenkatalog, um Angriffsvektoren zu identifizieren, die genutzt werden könnten, um Systemschwachstellen auszunutzen sowie potenzielle Gegenmaßnahmen einzuleiten.

Gut zu wissen: TARA wurde ursprünglich im Jahr 2010 entwickelt und kam bereits in mehr als 30 Cyber-Risk-Assessments zum Einsatz. Dieses Framework eignet sich in besonderem Maße für Risikostudien, die sich auf Sicherheitsbedrohungen konzentrieren. (fm)

Dieser Artikel ist im Original bei unserer Schwesterpublikation CSOonline.com erschienen.

IBM is looking to expand the reach of its foundational mainframe portfolio by adding new single frame and rack mounted versions of its Z and LinuxONE systems.

The IBM z17 portfolio adds a single frame and rack mount versions that bring mainframe capabilities into smaller, customizable footprints. The LinuxONE Rockhopper family gets a single frame and rack mount models, plus a new Express rack mount offering, that target new and smaller clients, according to Tina Tarquinio, chief product officer, IBM Z & LinuxONE.

Specifically, the new hardware includes:

• z17 single frame is a fully packaged box in an IBM rack with intelligent power distribution units, delivered as a complete enclosed unit ready to deploy at the edge or other strategically important customer sites.
• z17 rack mount lets customers install IBM Z components directly into their own industry-standard rack, with built-in flexibility for co-location with other technologies.
• LinuxONE Rockhopper 5 is a multi-drawer LinuxONE system for high-density workloads, with on-chip AI acceleration, confidential computing, and postquantum cryptography available in both single frame and rack mount configurations.
• Rockhopper 5 rack mount and Express offerings deliver enterprise-grade Linux, confidential computing, and on-chip AI acceleration in a compact 18U configuration. Designed for organizations supporting a smaller set of workloads, the offering provides a cost-efficient entry point that can scale as business grows, while prioritizing security, resiliency, and performance.

IBM

The new IBM z17 and IBM LinuxONE 5 Rockhopper configurations support up to 82 cores and 18 TB of memory across two processor drawers, representing about a 20% increase in core count and 12% increase in memory capacity over current systems, IBM stated. Single processor capacity of an IBM z17 ME2 provides full speed IBM z/OS configurations including 10% greater throughput per core than IBM z16 A02 with some variation based on workload and configuration, according to Tarquinio.

Both systems feature a 5.5 GHz IBM Telum II processor and a built-in AI accelerator that IBM says will let customers run more than 450 billion inferencing operations in a day with one millisecond response time. In addition, the 32-core Spyre AI accelerator is designed to handle all manner of AI workloads.

The idea is to bring the core strengths of IBM Z to a broader range of deployment models while offering the security, resilience, and performance enterprises depend on, Tarquinio said. 

“As always, we’re continuing to innovate to deliver more with less, including up to 20% more capacity than IBM z16 to help process transactions faster and support growing AI-driven workloads,” Tarquinio said.  “Even the newest and smallest member of the IBM z17 family delivers the performance, efficiency, and scalability organizations need as they balance growth ambitions with real-world resource constraints.”

The Linux-based system, Rockhopper 5 is for organizations that have moved past the evaluation question and are ready to consolidate a substantial portion of their x86 estate, said Marcel Mitran, IBM Fellow and CTO of IBM LinuxONE. 

Rockhopper 5 is designed to bring a smaller physical footprint and a software licensing model that reflects actual workload boundaries rather than physical server counts, Mitran said.

The LinuxONE 5 Express is a preconfigured system designed to get organizations running on LinuxONE quickly, with a defined bill of materials and a predictable starting cost, on the same architecture that the largest enterprises in the world depend on, Mitran said.

“It is built for organizations that want to consolidate a modest x86 estate, evaluate LinuxONE for the first time, or deploy a specific workload such as digital assets, AI-infused transaction processing, or confidential computing, without committing to the footprint of the larger model,” Mitran said.

Some of the mainframes’ software features were also bulked up. For example, IBM said that Post Quantum Cryptography security is now standard on the z17 and LinuxONE Rockhopper 5 systems letting customers start to utilize cryptography to protect core resources for the future.

The idea is to help customers protect long-lived, mission-critical data while reducing the cost and complexity of future cryptographic migration, IBM stated. 

In that vein, IBM said it was bringing Crypto Discovery & Inventory, which lets security teams see what has been encrypted across the enterprise. In addition, IBM announced an Infrastructure Management for Z and LinuxONE package that would let customers administer, monitor, automate, and provision IBM Z and LinuxONE systems from a central location.

IBM said it wants to reduce operational complexity for customers by making automating day-to-day operations to ultimately lower administrative costs and concerns. With the new flexible form factors, IBM continues to target hybrid and AI infrastructure buildouts with the Big Iron. In the AI world, the z17 is being utilized for AI inferencing, transactions, training, and key security applications such as fraud detection and insurance claims.

“Enterprise infrastructure is entering a new phase. Organizations need platforms that can support AI-driven growth while navigating resource constraints, evolving business requirements, and increasingly complex hybrid environments,” Tarquinio said. “They are being asked to deploy new AI capabilities while learning new skills, controlling operational costs, and maximizing the value of existing applications and infrastructure.”

A recent IBM Institute study on mainframe usage stated that embedding mainframe to support AI in executing transactions is not temporary: 75% of executives expect mainframe-based applications to remain central to digital transformation, and 60% say mainframe-based platforms are essential to enabling AI innovation.

”Mainframe-anchored systems of record are becoming systems of intelligent execution—not as general‑purpose AI platforms, but as environments where AI acts directly within transactions and in support of them,” the study reported.

Gartner wrote in its “The State of the IBM Mainframe in 2026” report that IBM’s willingness to make significant investments ensure the mainframe modernizes to remain a vital and thriving component of enterprise IT.  

“Most mainframe customers are now prioritizing the reduction of technical debt and adopting platform innovations to future-proof their mainframe environments for the coming decade,” Gartner wrote.

The new z17 single frame and rack mount configurations, LinuxONE Rockhopper 5, and LinuxONE 5 Express will all be available August 12, 2026. IBM Infrastructure Management for IBM Z and IBM LinuxONE will be available August 14.

Faster Python has stopped becoming a pipe dream, and is now a major topic for its development. Sometimes that comes by way of new syntax (lazy imports), sometimes by JIT compilation, and sometimes by generating C code from Python. Sometimes, it’s also by way of a whole new programming language (Mojo) that’s intended to be a powerful Python companion.

Top picks for Python readers on InfoWorld

Speed boost your Python programs with new lazy imports
With lazy imports in Python 3.15, the evaluation of imports can be delayed until they’re actually used, instead of when your Python program declares them. Best of all, you don’t need to rewrite everything to use this feature.

CPython vs. PyPy: Which Python runtime has the better JIT?
Conventional wisdom tells us that PyPy’s built-from-scratch and time-tested JIT should beat CPython’s own new native JIT. Conventional wisdom isn’t always right.

First look: Mojo 1.0 mixes Python and Rust
Is Mojo likely to outmuscle Python anytime soon? Probably not, but so far it’s shaping up to be as speedy as Rust without so much syntactical overhead.

PythoC: A new way to generate C code from Python
The traditional way to use Python to generate C code is Cython. But PythoC offers a far more streamlined experience for those who want to hitch C’s speed to Python’s convenience.

More good reads and Python updates elsewhere

PEP 836 – JIT go brrr: The path to a supported JIT compiler for CPython
The Python Software Foundation has delivered a roadmap for moving Python’s experimental JIT compiler towards a full-blown, supported, enabled-by-default part of Python’s future. But the road ahead could be bumpy.

Are you really expected to run five type-checkers now?
Well, are you? (Spoiler: not really!) Which of the big five type checkers for Python should you use? Even if you’re already committed to one type checker, this article is well worth reading. The context for why so many exist is useful.

Making Python faster with free threading and Mypyc
Mypyc is an underrated way to convert Python to C, and it’s now compatible with Python’s free-threaded build. Combining the two can unleash truly hair-raising speedups.

MicroGPT: A GPT in 200 lines of pure Python
You won’t get blazing GPU-powered performance, but you’ll get a hands-on under-the-hood example of how, exactly, one can create a GPT-2-esque neural network architecture. Try it out on your favorite public domain text!

Site reliability engineering is entering a new phase. As incidents become faster-moving, more data-rich and more complex, SRE teams are exploring agentic AI to help with alert triage, root cause analysis, runbook execution and mitigation planning. But in production, the question is not whether an agent can act; it is whether people can trust it to act safely, consistently and transparently when the system is under stress.

This blog argues that trust is an engineering outcome, not a marketing promise. Trustworthy agentic SRE systems are built on a foundation of grounded telemetry, explicit safety boundaries, progressive autonomy, auditability and evaluation against real incidents.

Why trust matters

Traditional automation works well when the world is predictable. SRE work is different because incidents are messy, partial and time-sensitive, with ambiguous symptoms, shifting dependencies and business context that rarely fits into a neat playbook. A fluent AI agent that lacks system context can sound convincing while still making dangerous recommendations.

Trust in SRE is earned during failure, not during demos. That means the system must prove it can help during noisy alerts, failed deploys, partial outages and conflicting telemetry, while staying bounded enough that one mistake does not become a major incident. Google’s AI-in-SRE work makes the same point through its emphasis on strict guardrails, progressive authorization and deterministic actuation controls.

Trust pillars

A practical trust model for agentic SRE can be organized into five pillars.

Google’s SRE autonomy model reflects the same progression: From assisted monitoring and investigation to partial autonomy with human approval to higher autonomy only after sustained success and safety proof.

Architecture pattern

A trustworthy agentic SRE system should separate reasoning from actuation. The agent can investigate, summarize, propose and even stage a plan, but the actual execution path should pass through a deterministic safety layer that validates permissions, risk, current production state and blast radius before any change is made.

A strong pattern looks like this:

1. Alert arrives from monitoring or incident tooling.
2. Agent gathers context from telemetry, deploy history, ownership and prior incidents.
3. Agent produces a ranked hypothesis and a candidate remediation plan.
4. Safety layer checks policy, risk score, current incident state and dry-run outcome.
5. Human approves low-confidence or high-risk actions.
6. Actuation layer executes only pre-approved, bounded changes.
7. The system observes post-action effects and either confirms success or falls back.

Google’s description of AI operator and its mitigation safety verification layer is a useful reference point here: Investigation is not the same as actuation and the two should not share the same trust boundary. That separation reduces blast radius and keeps the system interruptible.

To try out Agentic SRE, StackGen has a community edition where you can see the capabilities of agentic SRE by connecting your Grafana or Datadog.

Guardrails that work

The most effective guardrails are boring in the best possible way. They include least-privilege identity, strict rate limits, dry-run support, explicit approval workflows, action allowlists and hard stop mechanisms for runaway loops. Check out the detailed guide on how SRE trusts AI agents. AWS describes trust in autonomous systems in the same terms: Identity, runtime guardrails, observability and policy enforcement are the backbone of safe autonomy.

For SRE agents, a few guardrails are especially important:

• Least privilege identity so the agent only has access to the systems it truly needs.
• Dry-run or simulation mode so the likely outcome is known before production state changes.
• Circuit breakers and loop detection to stop repeated or runaway tool calls.
• Action tiers so low-risk tasks can be automated while high-risk tasks require approval.
• Red-button controls so humans can immediately revoke autonomy during a bad incident.

These controls are not signs of immaturity. They are what make autonomy acceptable in high-stakes environments.

Observability for agents

Observability is not just for services; it is for the agent itself. If the agent’s reasoning, tool usage and outcomes are not observable, then debugging it during an incident becomes guesswork. Google explicitly emphasizes exposing reasoning traces and execution traces so that autonomous decisions remain auditable and debuggable.

A good agent observability stack should capture:

• Inputs and retrieved context.
• Tool calls, parameters and results.
• Intermediate hypotheses.
• Confidence and uncertainty.
• Approvals, denials and overrides.
• Final action and outcome.
• Post-action verification signals.

This creates the operational memory needed to understand whether the agent helped, harmed or merely added noise. It also supports post-incident review and future training data generation.

Human in the loop

Human-in-the-loop does not mean the agent is weak; it means the system is designed around responsibility. SREs still own the incident, the rollback, the customer impact and the final decision when context is incomplete. The agent should reduce toil and improve speed, not create a false sense of safety.

The best human-in-the-loop model is proportional. Low-risk tasks like summarizing incidents or collecting dashboards can be automated. Medium-risk actions like restarting a worker can require lightweight approval. High-risk actions like draining core capacity or disabling a major dependency should remain human-controlled. This progressive model lets trust grow gradually rather than forcing a dangerous leap to full autonomy.

Evaluation strategy

If you only test an agent on toy benchmarks, you will get toy reliability. Real SRE evaluation should replay historical incidents and score whether the agent identified the right signals, chose the right hypothesis and recommended safe remediation under realistic conditions. Google’s approach uses continuous evaluation pipelines, human-verified gold data and nightly evals against real incident trajectories to measure readiness for autonomous action.

A practical evaluation program should include:

• Historical incident replay.
• Golden-path and failure-path comparisons.
• Tool misuse tests.
• Prompt injection and adversarial input tests.
• Loop and retry stress tests.
• Human review of edge cases.
• Regression tracking across model and policy changes.

The key metric is not “did the model sound right?” It is “did the system shorten time to mitigation, reduce toil and avoid new operational risk?”.

Failure modes

Agentic SRE systems fail in ways that classic software often does not. They can hallucinate a root cause, misread telemetry, over-trust stale context, loop on a broken action or optimize the wrong objective while sounding confident. In a high-stakes environment, this is more dangerous than a simple bug because the system can act before humans realize it is wrong.

The main failure modes to design against are:

• Confident incompleteness, where the agent lacks key context but still gives a decisive answer.
• Runaway loops, where tool calls repeat and consume time or budget.
• Unsafe actuation, where a valid-looking action is harmful in the current operational state.
• Workflow drift, where the agent bypasses established incident processes.
• Hidden fragility, where speed increases but accountability decreases.

Good architecture assumes failure will happen and makes sure the system fails safely, visibly and reversibly.

If you need a more detailed guide to keep points while evaluating AI SRE tools, then check this buyer’s guide by one of the senior leaders.

Operating model

The healthiest way to deploy agentic SRE is to treat it as a bounded operational partner. Start with read-only use cases like alert enrichment, incident summarization and investigation assistance. Then move to recommendation-only workflows, then to low-risk automation and only later to tightly scoped autonomous mitigation.

That staged rollout should be paired with policy, ownership and incident review discipline. Every agent action should map back to a responsible team, a bounded capability and a visible audit trail. This is how the system earns confidence from engineers, security teams and leadership at the same time.

Conclusion

Trustworthy agentic systems for SRE are built, not assumed. The winning formula is grounded telemetry, explicit guardrails, human oversight, explainable reasoning and evaluation against the messy reality of production incidents. When those pieces are in place, AI becomes a reliability multiplier rather than another source of operational risk.

The real goal is not a fully autonomous agent that never makes mistakes. The real goal is an agentic system that stays safe when it does make mistakes, recovers cleanly and keeps SRE teams in control when it matters most.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

앤트로픽이 자사 AI 모델이 문제를 해결하는 내부 과정을 보다 깊이 들여다볼 수 있는 새로운 분석 기법을 공개했다. 모델이 특정 방식으로 판단하고 행동하는 이유를 파악할 수 있게 되면서, 기업의 AI 평가와 구매 기준에도 적지 않은 영향을 미칠 것으로 전망된다.

앤트로픽은 최근 ‘J-스페이스(J-space)’라고 이름 붙인 새로운 내부 표현 공간을 발견했다고 밝혔다.

앤트로픽은 공식 블로그를 통해 “클로드(Claude)는 수많은 내부 처리 과정 가운데 특별한 역할을 수행하는 소수의 신경 패턴 집합을 스스로 형성했다”라며 “이 패턴 집합을 발견하는 데 야코비안(Jacobian)이라는 수학 개념을 활용한 기법을 사용했기 때문에 이를 ‘J-스페이스’라고 명명했다”고 설명했다. 회사는 ‘야코비안 렌즈(Jacobian Lens·J-렌즈)’라는 분석 기법을 통해 J-스페이스 내부를 관찰한다.

앤트로픽은 “J-스페이스의 각 패턴은 특정 단어와 연결돼 있다”라며 “하지만 특정 패턴이 활성화됐다고 해서 모델이 실제로 그 단어를 출력하려는 것은 아니다. 단지 해당 단어가 모델의 ‘생각 속’에 있다는 의미”라고 밝혔다.

이어 “언어 모델의 ‘스크래치패드(scratchpad)’나 ‘생각의 연쇄(Chain of Thought)’처럼 추론 과정에서 스스로 작성하는 텍스트와 J-스페이스는 다르다”라며 “J-스페이스는 모델 내부 신경망 활성화 영역에서 조용히 작동하기 때문에 모델은 개념을 텍스트로 적지 않고도 내부적으로 사고할 수 있다”고 설명했다.

이번 연구는 앤트로픽이 2024년 공개했던 내부 스크래치패드보다 훨씬 발전한 개념이다. 당시에는 모델이 답변이나 행동을 준비하면서 무엇을 고려하는지를 보여주는 수준이었다. 반면 이번 기술은 모델의 훨씬 깊은 내부 사고 과정을 분석할 수 있어 AI 시스템을 평가하고 구매하는 방식 자체를 바꿀 가능성을 제시한다.

논문에 소개된 사례도 이러한 가능성을 보여준다. 일부 모델은 테스트 과정에서 부적절한 행동을 하지 않아 안전한 모델처럼 보였다. 그러나 J-스페이스를 분석한 결과, 해당 모델은 자신이 테스트를 받고 있다는 사실을 인지하고 있었으며, 그 때문에 문제 행동을 의도적으로 하지 않았을 가능성이 드러났다. 이는 아이들이 누군가 자신을 지켜보고 있다는 사실을 알 때 행동을 달리하는 것과 유사하다고 연구진은 설명했다.

AI 에이전트 기업 제니티(Zenity)의 AI 표준·거버넌스 총괄인 록 램브로스(Rock Lambros)는 “앤트로픽은 모델이 테스트를 받고 있다는 사실을 인식하거나, 좋은 결과를 얻기 위해 행동을 꾸미거나, 프롬프트 인젝션을 탐지하거나, 아직 실행하지 않은 목표를 내부적으로 유지하고 있는 상황까지 포착할 수 있는 분석 도구를 만들었다”라며 “일부 바람직한 행동은 모델이 자신이 평가받고 있다는 사실을 알고 있었기 때문에 나타난 것일 수 있다”고 말했다.

그는 기업 고객 역시 AI 안전성 벤치마크를 해석할 때 이러한 점을 고려해야 한다고 지적했다.

램브로스는 “프로젝트에 적합한 모델인지는 모델이 알고 응시한 리더보드 결과가 아니라, 기업이 보유한 데이터와 실제 공격 시나리오를 활용한 자체 테스트를 통해 검증해야 한다”고 설명했다.

이처럼 모델 내부를 들여다볼 수 있는 능력은 CIO에게도 중요한 의미를 갖는다.

램브로스는 “자사 모델이 겉으로 드러나지 않는 문제 행동을 스스로 발견하고 그 결과를 공개할 수 있는 공급업체라면 신뢰성 검증 체계가 상당히 성숙했다는 의미”라며 “이러한 역량은 단순한 뉴스가 아니라 공급업체 실사(Due Diligence) 과정에서 반드시 확인해야 한다”고 말했다.

이어 “이제 모든 AI 모델 공급업체에 던져야 할 질문은 ‘모델 출력만으로는 볼 수 없는 내부 상태 가운데 무엇을 관찰할 수 있으며, 실제로 어떤 문제를 발견했는가’가 돼야 한다”고 덧붙였다.

AI 거버넌스 컨설팅 기업 디지털 520(Digital 520)의 수석 컨설턴트 노아 케니(Noah Kenney)도 비슷한 견해를 내놨다.

케니는 “감시받고 있다는 사실을 알기 때문에 더 바람직하게 행동하는 모델은 안전한 모델이 아니다. 단지 포커페이스를 잘하는 모델일 뿐”이라며 “레드팀 테스트 결과나 모델이 위험한 요청을 거부한 내부 파일럿, ‘테스트해 보니 문제가 없었다’는 모든 사례를 다시 검토해야 한다. 이제는 모두 단서를 달고 해석해야 하기 때문”이라고 말했다.

또한 CIO는 AI 에이전트가 특정 방식으로 작업을 수행한 이유가 원래 그렇게 설계됐기 때문인지, 아니면 단순히 자신이 테스트받고 있다는 사실을 알아차렸기 때문인지를 구분해야 한다고 강조했다.

케니는 “이 질문에 대한 답에 따라 모델 평가 결과에 대한 해석도 크게 달라져야 한다”고 말했다.

아직 고객은 사용할 수 없는 J-렌즈

노아 케니는 “이번 연구는 지금까지 업계가 AI 모델 평가를 통해 측정해 온 것이 모두가 생각했던 것만큼 견고한 지표가 아니었다는 사실을 인정한 것”이라며 “이제 다른 프런티어 AI 연구소들도 자사 평가 체계 역시 같은 문제를 안고 있는지 답해야 할 것”이라고 말했다. 이어 “CIO에게 이번 논문은 기업의 AI 모델 리스크 관리 체계 전반을 다시 점검하라는 경고”라고 평가했다.

렉시스넥시스 리스크 솔루션 그룹(LexisNexis Risk Solutions Group)의 CISO 플라비오 비야누스트레(Flavio Villanustre)는 J-스페이스를 분석하면 모델 효율성까지 높일 수 있다고 설명했다.

비야누스트레는 “J-스페이스는 모델 내부를 직접 들여다볼 수 있는 능력을 제공하기 때문에 사용자에게 매우 유용하다”라며 “특히 설명 가능성이 중요한 규제 산업에서는 응답의 근거와 인과관계를 충분히 분석해야 하는데 큰 도움이 될 수 있다”고 말했다.

이어 “사용자가 프롬프트를 더욱 정교하게 다듬는 데에도 활용할 수 있어 모델의 토큰 사용 비용을 최적화하는 데 도움이 된다”고 설명했다.

다만 현재로서는 이러한 정보를 직접 활용하기 어렵다. AI 공급업체를 통해 간접적으로 접근하거나 향후 계약 협상을 통해 권한을 확보하는 것이 사실상 유일한 방법이다. 비야누스트레는 일부 기업은 앤트로픽의 FDE 프로그램에 비용을 지불하면 J-스페이스에 직접 접근할 수도 있다고 덧붙였다.

그는 “J-스페이스는 CIO에게 매우 유용한 도구”라면서도 “이를 실제로 활용하려면 분석 결과를 해석할 수 있는 전문 인력이 필요하다. 요구되는 역량은 일반 데이터 분석가는 물론 데이터 사이언티스트 수준을 넘어선다”고 말했다.

기술 컨설팅 기업 트라이베카 소프트테크(Tribeca Softtech)의 최고전략책임자(CSO) 아만 마하파트라(Aman Mahapatra)는 현재 기업 고객이 J-렌즈를 실제 운영에 활용하기는 어렵다고 지적했다.

그는 “현재 기업 고객은 야코비안 렌즈를 활성화할 수도 없고, API를 통해 모델의 잔차 스트림(residual stream)을 분석할 수도 없으며, 논문의 핵심 결과를 도출한 제거 실험(ablation study)도 수행할 수 없다”고 설명했다.

이어 “올해 3분기 안에 CIO가 J-스페이스 모니터링을 실제 운영 환경의 배포 승인 기준으로 활용할 수 있느냐는 질문에 대한 답은 ‘아니다'”라고 말했다.

다만 마하파트라는 앞으로는 다른 방식의 접근 경로가 마련될 것이며, CIO들이 이를 적극 요구해야 한다고 주장했다.

그는 “고객이 직접 접근할 수 없다면 결국 이번에도 앤트로픽을 믿을 수밖에 없다”라며 “바로 그렇기 때문에 기업들은 업계 전반에 새로운 신뢰 검증(Assurance) 체계를 요구해야 한다”고 말했다.

이어 “현재 AI 모델 공급업체들은 자체 도구로 스스로 모델을 점검한 뒤 안심할 수 있는 연구 결과를 발표하는 방향으로 나아가고 있다”라며 “그러나 어떤 규제 산업도 다른 공급업체에게 이런 방식의 검증을 신뢰 기준으로 인정하지 않는다”고 지적했다.

마하파트라는 “은행은 신용평가 업체가 ‘우리 모델은 우리가 검증했으니 믿어달라’고 말한다고 이를 받아들이지 않는다”라며 “의료 업계 역시 임상 의사결정 지원 시스템 공급업체의 자체 검증만으로는 신뢰하지 않는다. 파운데이션 모델 공급업체만 예외로 취급해야 할 원칙적인 이유는 없으며, 이번 J-스페이스 연구는 그 이유를 분명하게 보여준다”고 말했다.

새로운 가시성이 요구하는 변화

마하파트라는 기업이 장기적으로는 AI 모델의 내부 동작을 독립적으로 검증할 수 있는 환경을 요구해야 한다고 강조했다.

그는 “기업이 취해야 할 올바른 장기 전략은 고객이 사용할 수 있는 API, 특권 접근 권한을 가진 독립적인 제3자 감사기관, 또는 은행의 모델 리스크 관리팀이 공급업체의 안전성 검증팀과 동일한 도구를 사용할 수 있도록 하는 개방형 해석 가능성(Interpretability) 표준 등을 요구하는 것”이라고 말했다.

이어 “현재는 이런 환경이 전혀 마련돼 있지 않다”라며 “하지만 CIO들이 추진해야 할 로드맵에는 반드시 포함돼야 하며, 이번 연구는 그 필요성을 보여주는 가장 강력한 근거”라고 평가했다.

실제로 이번 연구 결과는 기업의 AI 전략 자체를 근본적으로 바꿀 가능성을 갖고 있다.

마하파트라는 “기업이 AI 에이전트를 도입할 때 가장 어려운 과제는 자율 시스템이 설명하는 추론 과정과 실제 내부 추론이 일치하는지를 검증하는 것”이라며 “지금까지는 모델이 출력한 내용만 감사할 수 있었고 실제 추론의 상당 부분은 보이지 않는 곳에서 이뤄졌다. J-렌즈는 바로 이 간극을 정면으로 해결하려는 시도”라고 설명했다.

그는 기업의 AI 구매 담당자들이 앞으로 모델 공급업체에 내부 상태를 관찰할 수 있는 해석 가능성 도구를 제공하는지 반드시 확인해야 한다고 조언했다. 특히 고객 환경에서 기만 행위(deception), 평가 회피(evaluation gaming), 목표 불일치(goal misalignment) 등을 모니터링할 수 있는 기능을 제공하는지를 구매 과정에서 질문해야 한다고 강조했다.

마하파트라는 “현재 이러한 질문에 제대로 답할 수 있는 공급업체는 거의 없다”라며 “관련 도구가 완전히 성숙하기 전이라도 내부 상태의 가시성을 구매 기준으로 요구하기 시작하는 CIO가 앞으로 공급업체의 제품 발전 방향을 결정하게 될 것”이라고 말했다.

이어 “향후 규제기관이 ‘기업은 자율 AI 에이전트가 실제로 주장한 대로 동작한다는 사실을 어떻게 확인했는가’를 묻기 시작하면, 이런 기업만이 실질적인 신뢰성을 입증할 수 있을 것”이라고 전망했다.

표준화의 시작

CIO가 클로드의 새로운 내부 가시성을 활용할 수 있는 또 다른 방법은 이미 해당 정보에 접근할 수 있는 제3자를 활용하는 것이다. 실제로 이번 보고서에는 구글의 AI 연구원이 오픈웨이트(Open-weight) 모델에서 일부 연구 결과를 독립적으로 재현하는 데 성공했다는 내용이 포함됐다.

소프트웨어 개발 기업 컴프 AI(Comp AI)의 CEO 루이스 카하트(Lewis Carhart)는 “이는 공급업체의 주장만이 아니라 경쟁사가 해당 분석 기법을 검증했다는 의미”라며 “기술적으로 무엇이 가능한지를 보여주지만, 기업이 직접 이를 검증할 수 있는 수단을 제공하는 것은 아니다”라고 설명했다.

그는 이러한 상황이 컴플라이언스 분야에서도 이미 여러 차례 반복됐던 패턴이라고 말했다.

카하트는 “SOC 2 역시 처음부터 독립적인 감사 기준으로 출발한 것은 아니었다”라며 “초기에는 공급업체가 자체 통제 체계를 설명하는 수준에 머물렀고, 이후 시장이 수년에 걸쳐 이를 외부에서 검증할 수 있는 인프라를 구축했다”고 설명했다.

이어 “AI 해석 가능성(Interpretability)도 지금은 바로 그 출발점에 있다”라며 “J-렌즈 분석 결과가 제3자 감사 보고서나 모델 카드(Model Card), 또는 규제기관 제출 문서에 포함되기 시작해야 CIO에게 실질적인 의미를 갖게 될 것”이라고 말했다.

그는 “결국 리스크 관리 조직이 공급업체의 주장만이 아니라 객관적인 근거로 제시할 수 있는 자료가 마련돼야 한다”고 덧붙였다.

AI 전략 변화 이끄나

컨설팅 기업 액셀리전스(Acceligence)의 CEO 저스틴 그라이스(Justin Greis)는 이번 기술이 기업 AI 전략에도 상당한 변화를 가져올 것으로 전망했다.

그는 “향후 AI 거버넌스 플랫폼은 프롬프트와 출력 결과, 사용자 신원 정보, 정책 결정, 도구 사용 기록뿐 아니라 J-렌즈가 제공하는 내부 신호까지 함께 활용하게 될 것”이라고 말했다.

이어 “미래의 AI 제어 플랫폼은 AI 에이전트가 프롬프트 인젝션 시도를 인식했는지, 민감한 정보가 포함됐음을 이해했는지, 상충되는 목표를 감지했는지, 또는 실제 위험한 행동을 실행하기 전에 그러한 방향으로 추론하고 있었다는 징후가 있었는지를 지속적으로 평가할 수 있을 것”이라며 “이러한 신호는 정책 집행, 사람의 개입 여부 결정, 감사 로그 작성, 기업 AI 환경 전반의 신뢰도 평가 등에 활용될 것”이라고 설명했다.

그는 이러한 변화가 현재 CIO에게도 실질적인 의미를 갖는다고 강조했다.

그라이스는 “AI 공급업체를 평가하는 기준 자체가 달라지고 있기 때문”이라며 “1년 전만 해도 기업은 모델의 정확도와 지연시간, 보안, 비용을 주로 평가했다. 앞으로는 AI 에이전트의 행동과 추론 품질, 정책 준수 여부, 안전성 모니터링, 감사 가능성에 대해 공급업체가 얼마나 높은 수준의 운영 가시성을 제공하는지도 중요한 평가 항목이 될 것”이라고 말했다.

마하파트라는 이러한 변화가 CIO에게 강력한 협상 카드가 될 수도 있다고 분석했다.

그는 “실제 협상력이 발휘되는 시점은 계약 갱신 과정”이라며 “다음 계약 갱신 때 해석 가능성 보고서 제공과 제3자 감사 접근 권한을 계약 조항에 포함시켜야 한다. 지금은 이러한 조건을 비교적 쉽게 확보할 수 있지만 계약 체결 이후에는 훨씬 큰 비용이 들기 때문”이라고 말했다.

이어 “2027년 신뢰성 확보 경쟁에서 앞서는 CIO는 2026년에 이미 AI 모델 공급업체의 ‘우리를 믿어달라’는 말만 받아들이지 않고, 공급업체가 아직 계약을 더 필요로 하는 시점에 필요한 조항을 계약서에 반영한 사람일 것”이라고 전망했다.
dl-ciokorea@foundryco.com

As enterprises struggle to manage their AI strategies, the US AI regulatory environment is sending a wide range of contradictory signals. OpenAI’s Wednesday announcement that it will now release GPT-5.6 Sol, along with Terra and Luna, on Thursday highlights the confusion.

Initially, the US government said that it was asking OpenAI to limit access to its top models, including the three releasing Thursday, to a short list of companies. OpenAI seemingly agreed and held back their general availability.

But on Wednesday, OpenAI reversed its position, with a statement on X saying simply: “GPT-5.6 Sol, along with Terra and Luna, will launch publicly this Thursday. We’re expanding preview access globally now.” No details were released about the extent of the expansion.

Then the White House issued a statement, a copy of which it emailed to InfoWorld, saying that the US government “did not give OpenAI a ‘green light,’ approval or clearance to release its models. No such permission is required or granted. The Administration does not provide approvals for private companies to release AI models – decisions on timing and scope of releases rest entirely with the companies.”

The statement then quoted from the June 2 White House executive order that said, “nothing in this section shall be construed to authorize the creation of a mandatory governmental licensing, preclearance, or permitting requirement for the development, publication, release, or distribution of new AI models, including frontier models.” It also said, “any testing or meetings with government experts is voluntary. Participation is not required to release a model.”

Yet last month, the US Commerce Department weighed in on how Anthropic’s models can be distributed. 

The ‘worst of both worlds’

Lewis Carhart, CEO of software development firm Comp AI, said the statement is frustrating for IT executives on multiple fronts. 

“Think about what that [White House statement] means. OpenAI stationed engineers in Washington for weeks, submitted to government testing, staggered its launch at the government’s request. And the official position is that none of that was required,” Carhart said. “We now have a de facto licensing regime that legally doesn’t exist. There’s no statute, no appeal process, no published criteria. Just [the US Department of] Commerce deciding model by model what ships and when.”

That’s the worst of both worlds, he noted: “All the friction of regulation with none of the predictability. Compliance people have a name for this – it’s an audit with no framework. And the precedent is now locked in for both frontier labs: if you build at the frontier, your launch calendar runs through Washington whether the law says so or not.”

Carhart argued that this regulatory reality should be of extreme concern to enterprise IT executives, given it indicates that model availability is now “a regulatory variable” not driven by the vendor roadmap.

“Anthropic’s most advanced models disappeared from the market for three weeks in June. It was not because of an outage, not because of a pricing change. It was because of an export control directive,” he pointed out. “If your AI architecture assumes the model you deployed today is available tomorrow, that assumption is now demonstrably false. Multi-model resilience just went from nice to have to a board-level risk item.”

It also offers an opportunity, given that a model that cleared government security testing is a model that auditors and boards sign off on faster. “Government review is quietly becoming a procurement asset,” he observed. “The CIOs who win here are the ones who treat ‘regulatory posture of the model itself’ as a line item in vendor risk assessments – most risk teams are still only looking at the provider’s SOC 2 attestation.”

Jason Andersen, principal analyst at Moor Insights & Strategy, agreed with Carhart and described the overall back-and-forth as “a bit of pageantry. OpenAI needs its model to look as powerful, potentially dangerous, as Anthropic’s so it can be a contender to be at the absolute frontier. It also helps to burnish OpenAI’s PR efforts to look more responsible than it has in the past.”

Furthermore, he added, “tech CEOs are acutely aware that flattery towards this administration could keep regulators or the threat of regulation at bay.”

Criteria not clear

Brian Jackson, a principal research director at Info-Tech Research Group, echoed the political concerns. 

“What’s still not clear is the actual criteria being used to deem the models safe for release. The government has said that it’s concerned about cybersecurity risk as well as the risk of AI being used to develop biological weapons,” he said. “But so long as the actual release criteria lack transparency, there will be some perception that the evaluation could be politically motivated.”

Jackson said that one, presumably unintended, result of the US government’s efforts to control AI rollouts is that it is making companies look far more seriously at using non-US vendors for AI strategies. 

“Organizations are looking for alternatives to the private US-based cloud-delivered frontier models. That’s so they can maintain control over their AI supply chain,” he said. “Chinese open source models are one option that’s available, but there are other options too, from Canada and Europe. Companies can either set up AI access through other APIs not connected to US-based AI providers, or download open-source models to run locally.”

He noted that the added US regulatory risk means that some organizations will avoid becoming entrenched within OpenAI’s and Anthropic’s interfaces, where there’s no option to swap out the LLMs for an alternative.

Impacts enterprise AI strategy

Rock Lambros, director of AI standards and governance at AI agent vendor Zenity, shared the frustration that little to no actionable compliance data is being released. 

“Nobody outside a closed room can tell you what standard [the model] passed because it was never written down. For two weeks, [US government officials] kept a model out of defenders’ hands that’s better at guarding your network than breaking into anyone else’s,” Lambros said. “Call that a security review if it helps you sleep better. But it reads to me like a bouncer working a velvet rope nobody hired him to run, waving people through today because he’s in a better mood than he was a couple of weeks ago.”

This unpredictability is likely to have impacts on AI strategy far beyond traditional compliance concerns, Lambros said.

“We’ve built way too much operational reliance on these models to hang it on a review with no rulebook,” he said, pointing out that hospitals, pipelines, banks and water utilities are relying on frontier AI whose availability “can swing from ‘on’ to ‘off’ to ‘on’ with no notice, no appeal, and no published standard behind any of it.”

“You can’t run critical infrastructure on a tool that runs fine Friday and is offline by Monday because an approval process nobody can see reached a verdict nobody can predict,” Lambros said. “That is a supply chain risk with a government hand on the switch, and almost nobody has priced it into a continuity plan.”

To protect themselves, companies need to adjust their expectations. “Treat model availability like any single point of failure you don’t own by standing up a fallback you’ve tested, getting a continuity clause in your contract, and drilling for the blackout, because ‘the government backed off this time’ is not a plan,” he advised.

As enterprises struggle to manage their AI strategies, the US AI regulatory environment is sending a wide range of contradictory signals. OpenAI’s Wednesday announcement that it will now release GPT-5.6 Sol, along with Terra and Luna, on Thursday highlights the confusion.

Initially, the US government said that it was asking OpenAI to limit access to its top models, including the three releasing Thursday, to a short list of companies. OpenAI seemingly agreed and held back their general availability.

But on Wednesday, OpenAI reversed its position, with a statement on X saying simply: “GPT-5.6 Sol, along with Terra and Luna, will launch publicly this Thursday. We’re expanding preview access globally now.” No details were released about the extent of the expansion.

Then the White House issued a statement, a copy of which it emailed to InfoWorld, saying that the US government “did not give OpenAI a ‘green light,’ approval or clearance to release its models. No such permission is required or granted. The Administration does not provide approvals for private companies to release AI models – decisions on timing and scope of releases rest entirely with the companies.”

The statement then quoted from the June 2 White House executive order that said, “nothing in this section shall be construed to authorize the creation of a mandatory governmental licensing, preclearance, or permitting requirement for the development, publication, release, or distribution of new AI models, including frontier models.” It also said, “any testing or meetings with government experts is voluntary. Participation is not required to release a model.”

Yet last month, the US Commerce Department weighed in on how Anthropic’s models can be distributed. 

The ‘worst of both worlds’

Lewis Carhart, CEO of software development firm Comp AI, said the statement is frustrating for IT executives on multiple fronts. 

“Think about what that [White House statement] means. OpenAI stationed engineers in Washington for weeks, submitted to government testing, staggered its launch at the government’s request. And the official position is that none of that was required,” Carhart said. “We now have a de facto licensing regime that legally doesn’t exist. There’s no statute, no appeal process, no published criteria. Just [the US Department of] Commerce deciding model by model what ships and when.”

That’s the worst of both worlds, he noted: “All the friction of regulation with none of the predictability. Compliance people have a name for this – it’s an audit with no framework. And the precedent is now locked in for both frontier labs: if you build at the frontier, your launch calendar runs through Washington whether the law says so or not.”

Carhart argued that this regulatory reality should be of extreme concern to enterprise IT executives, given it indicates that model availability is now “a regulatory variable” not driven by the vendor roadmap.

“Anthropic’s most advanced models disappeared from the market for three weeks in June. It was not because of an outage, not because of a pricing change. It was because of an export control directive,” he pointed out. “If your AI architecture assumes the model you deployed today is available tomorrow, that assumption is now demonstrably false. Multi-model resilience just went from nice to have to a board-level risk item.”

It also offers an opportunity, given that a model that cleared government security testing is a model that auditors and boards sign off on faster. “Government review is quietly becoming a procurement asset,” he observed. “The CIOs who win here are the ones who treat ‘regulatory posture of the model itself’ as a line item in vendor risk assessments – most risk teams are still only looking at the provider’s SOC 2 attestation.”

Jason Andersen, principal analyst at Moor Insights & Strategy, agreed with Carhart and described the overall back-and-forth as “a bit of pageantry. OpenAI needs its model to look as powerful, potentially dangerous, as Anthropic’s so it can be a contender to be at the absolute frontier. It also helps to burnish OpenAI’s PR efforts to look more responsible than it has in the past.”

Furthermore, he added, “tech CEOs are acutely aware that flattery towards this administration could keep regulators or the threat of regulation at bay.”

Criteria not clear

Brian Jackson, a principal research director at Info-Tech Research Group, echoed the political concerns. 

“What’s still not clear is the actual criteria being used to deem the models safe for release. The government has said that it’s concerned about cybersecurity risk as well as the risk of AI being used to develop biological weapons,” he said. “But so long as the actual release criteria lack transparency, there will be some perception that the evaluation could be politically motivated.”

Jackson said that one, presumably unintended, result of the US government’s efforts to control AI rollouts is that it is making companies look far more seriously at using non-US vendors for AI strategies. 

“Organizations are looking for alternatives to the private US-based cloud-delivered frontier models. That’s so they can maintain control over their AI supply chain,” he said. “Chinese open source models are one option that’s available, but there are other options too, from Canada and Europe. Companies can either set up AI access through other APIs not connected to US-based AI providers, or download open-source models to run locally.”

He noted that the added US regulatory risk means that some organizations will avoid becoming entrenched within OpenAI’s and Anthropic’s interfaces, where there’s no option to swap out the LLMs for an alternative.

Impacts enterprise AI strategy

Rock Lambros, director of AI standards and governance at AI agent vendor Zenity, shared the frustration that little to no actionable compliance data is being released. 

“Nobody outside a closed room can tell you what standard [the model] passed because it was never written down. For two weeks, [US government officials] kept a model out of defenders’ hands that’s better at guarding your network than breaking into anyone else’s,” Lambros said. “Call that a security review if it helps you sleep better. But it reads to me like a bouncer working a velvet rope nobody hired him to run, waving people through today because he’s in a better mood than he was a couple of weeks ago.”

This unpredictability is likely to have impacts on AI strategy far beyond traditional compliance concerns, Lambros said.

“We’ve built way too much operational reliance on these models to hang it on a review with no rulebook,” he said, pointing out that hospitals, pipelines, banks and water utilities are relying on frontier AI whose availability “can swing from ‘on’ to ‘off’ to ‘on’ with no notice, no appeal, and no published standard behind any of it.”

“You can’t run critical infrastructure on a tool that runs fine Friday and is offline by Monday because an approval process nobody can see reached a verdict nobody can predict,” Lambros said. “That is a supply chain risk with a government hand on the switch, and almost nobody has priced it into a continuity plan.”

To protect themselves, companies need to adjust their expectations. “Treat model availability like any single point of failure you don’t own by standing up a fallback you’ve tested, getting a continuity clause in your contract, and drilling for the blackout, because ‘the government backed off this time’ is not a plan,” he advised.

This article originally appeared on InfoWorld.

AI agents, and the large language models (LLMs) that power them, have short memories. That’s by design. There is only so much conversation that can be encoded into tokens and accessed reliably by the LLM. Retrieval-augmented generation, or RAG, can be used to give agents and LLMs memories larger than their context windows. But how agents use RAG, or other mechanisms for retaining the details of a conversation, can make all the difference.

With the rise of AI agents, there has been a corresponding rise in complementary software tools that give both agents and LLMs expanded memory capabilities. Most of the time, this means giving an agent or model persistent memory across sessions, so that previous context can be restored automatically. But, again, how that’s done can vary tremendously with each tool.

Here are some of the major projects in the AI agent memory space, each with their own particular spins, strengths, and orientations.

Graphiti

Graphiti is billed as “the open-source temporal knowledge graph framework.” The project is available on GitHub, or as the underpinning of the Zep ageny memory service. “Temporal” means information stored in Graphiti is re-evaluated over time to keep its context properly framed, and “graph framework” means the data is stored as a set of graphs. The other solutions profiled here use graph storage as part of their approach, but Graphiti makes that a front-and-center part of its design.

Graphiti supports a range of common LLM providers out of the box: Anthropic, Azure OpenAI, Google Gemini, and Groq. Any Ollama and OpenAI-compatible APIs also work, so Graphiti can be used with locally hosted LLMs as well. Connectors for third-party storage services let you ingest data from places like GitHub, Gmail, and OneDrive, as well as from applications like Notion.

Using Graphiti locally requires you set up or connect to a graph database. Neo4j is the default and most broadly supported of the bunch, but Amazon Neptune, FalkorDB, and KuzuDB will also work. Postgres with pgvector is not listed as an option.

Hindsight

Hindsight, available as both a cloud service and a locally hostable project, stores details about agent sessions into four types of memory with four types of storage and retrieval strategies. All of these are handled through three programmatic interfaces: retain for storing content, either a single fact or a whole conversation; recall for retrieving content; and reflect for running an agentic loop over a query that uses previously stored data.

Hindsight comes with a broad range of first-party and third-party integrations with existing LLMs and agent toolkits. For instance, if you’re using the Continue extension with Visual Studio Code to talk to a locally hosted LLM, you can use Hindsight’s Continue integration to add long-term memory to your interactions. You can use the @hindsight keyword in your query to inject relevant memory into the agent’s context, or use auto-injection rules (which can be edited) to do most of that heavy lifting automatically.

Mem0

Mem0 is a little like Hindsight in that it has four basic kinds of memory, although they are labeled and organized differently. For instance, Mem0 has a separate type of memory called organizational memory that’s intended to store data to be shared between multiple agents or different teams, something that is not normally done by default. Each memory added is passed through a distillation process and stored in a different way (vector DB, graph DB, SQL DB) depending on how it will be used. Older data, instead of being overwritten, gets deprecated rather than deleted, as a strategy for preserving larger long-term context. (Hindsight does this as well.)

Mem0 supports a smaller range of LLMs than Hindsight, but all the major options are available: Anthropic, Google Gemini, OpenAI, and self-hosted options like LangChain, LiteLLM, LM Studio, and Ollama. If you intend to use Mem0 locally rather than as a service, you’ll need to provide a Python instance and your own vector database. For the latter, Postgres with the pgvector extension is a common and simple choice; it can even be installed inside a Python venv.

Supermemory

Supermemory ingests data from many common sources—supporting plaintext, structured data, common document file formats like PDF and Microsoft Office, video and audio, images—and uses them to build a context graph to inform agent conversations. Among its most promoted features is its content-extraction tools.

Supermemory is available as a cloud service or as open-source software you can run locally. The open-source edition lacks the scaling services and third-party service connectors (Gmail, Google Drive, Notion, etc.) provided with the enterprise edition, but it has one big advantage: it consists of a single, self-contained binary, so it can be deployed on one’s own hardware with very little effort. No external databases need to be provisioned for Supermemory, either, so it’s well-suited to quick experimentation.

Anthropic says Claude now writes more than 80% of the code merged at one of the most sophisticated AI companies on the planet. Foundry’s 2026 State of the CIO study says fewer than one in five enterprises can show that their AI initiatives have met or exceeded their ROI goals. Both numbers came out this spring. Both are true. And the distance between them is the most important thing an IT leader can understand about AI right now.

Because that distance isn’t a contradiction, it’s a lesson. And the profession sitting in the middle of it, software engineering, is the canary that explains why so much enterprise AI spend has produced so little measurable return.

The report everyone misread

When Anthropic published its recursive self-improvement piece, plenty of people read it as the starting gun for the job apocalypse. Claude writing its own code, models getting better at building models, humans narrowing toward oversight. If you wanted a headline about the end of the software profession, it was right there.

I read it almost the opposite way. What struck me wasn’t how far AI had come. It was how much had to be true first, even in the one profession built from the ground up to let it succeed.

I made this argument back in my “$570K canary” piece, and the Anthropic data only sharpens it. AI coding agents don’t work because coding models are special. The underlying large language models (LLMs) are the same ones answering support tickets and reviewing contracts. They work because software development already had the infrastructure that makes an agent’s output trustworthy: governance baked into branch protection and code review, observability through version control and CI/CD pipelines, evaluation through automated tests, persistent context through commit history. Developers built all of that for themselves over decades. They didn’t build it for AI. But it turned out to be exactly the scaffolding AI needed.

That’s the part the apocalypse reading skips. Claude’s coding gains are real. They also rode on decades of pre-built substrate. Both things are true at once, and the second one is the one CIOs should be paying attention to when it comes to gains from things like recursive self-improvement.

What the CIO data actually shows

Now hold that next to the State of the CIO numbers. Only 19% of the 662 IT leaders surveyed say their AI initiatives have met or exceeded business goals. Another 18% admit fewer than a third of their use cases are hitting defined expectations.

The easy explanation is that the technology isn’t ready. The data says otherwise. This isn’t for lack of trying, and it isn’t for lack of organizing. Eighty-three percent of respondents have stood up cross-functional steering committees or are about to. Just over half have some form of AI approval process in place, with another quarter building one. Forty-seven percent have formal success metrics, with a third more on the way. The field is pouring effort into the organizational machinery of AI. The ROI still isn’t showing up.

Here’s why I think that is. All of that machinery sits above the work. Steering committees, approval gates and KPI dashboards govern the org chart. But the value, or the leak, happens inside the workflow, at the level of the actual task the AI is doing. You can instrument your governance structure perfectly and still have nothing measuring whether the agent’s output was right at the point where it mattered.

TIAA shows how little the org chart settles. The firm is three years in, runs generative and agentic use cases across fraud detection and call centers, and has 85% of its people on TIAA Gate, its internal platform. It also has the full governance stack most CIOs are still assembling. None of it closed the gap. “You need to understand the full cost of operations,” its chief operating, information and digital officer, Sastry Durvasula told CIO.com, “the efficiencies of running tokens or how you’re handling traffic or RAG.” The structures were never the thing leaking value. The workflow underneath them was.

The barriers respondents named back this up. The top three are lack of in-house expertise (40%), ill-defined ROI metrics (32%) and murky corporate AI strategy (31%). Not one of them is “the model isn’t good enough.” And according to the full Foundry report, the expertise gap is deepest in healthcare (52%), retail (51%) and manufacturing (49%), the sectors whose core work looks least like a software development lifecycle. That’s consistent with substrate being the real variable, though a tighter market for AI talent in those industries is surely part of the story too.

The market is already voting

Look at where the AI is actually being pointed, and you’ll see enterprises sequencing by substrate even though nobody’s calling it that. Three-quarters of both IT leaders and line-of-business respondents say AI is primarily being used to automate internal processes rather than customer-facing applications.

That’s not timidity. It’s instinct pointing at the right thing. Internal processes are the ones with structured, observable workflows and users who tolerate a little friction. Customer-facing work is where the trust gaps are still wide open and the cost of a wrong answer is asymmetric. A bad internal draft gets fixed before anyone sees it. A bad customer answer is the whole ballgame.

I’ll be honest about a wrinkle in the data here, because a careful reader will catch it. The same study reports a near-mirror finding, that 66% to 69% of respondents say the bulk of their current AI work is customer-facing. The two stats sit a paragraph apart in the CIO study and almost certainly reflect how the question was framed rather than a real reversal. But the synthesis holds either way: even where customer-facing work is being attempted, it’s where ROI is least realized. The work that lands is the work with the substrate underneath it. The split only reinforces the point.

Sequence by readiness, not by ambition

So, here’s the prescription, and it cuts against the instinct most AI strategies are built on. Stop sequencing your AI portfolio by where the value looks biggest. Start sequencing it by where the work already has, or can be given, a structured workflow with a usable signal for whether the output was right.

The study itself shows what the alternative looks like. Andrea Ballinger, CIO at Rensselaer Polytechnic Institute, described the trap precisely. No one measures ROI on an ongoing basis, she said, “because we are facing counterpressures from every vice president and line-of-business domain looking to implement AI for their own optimization.” The result: “We are saying yes to everyone without stepping back and focusing on the business cases that show real value.” That’s value-led sequencing under pressure from every budget-holder in the building, and it’s exactly how you end up with a sprawling pipeline of pilots and a 19% success rate.

The counterexample comes from the same study. Thomas Prommer, a longtime CTO, CIO and CAIO, funds outcomes instead of deliverables. “We don’t fund ‘build a model,’ we fund ‘reduce returns by 8% on this category’ with checkpoints at 90, 180 and 270 days,” he explained. He kills any project that misses two checkpoints, “roughly a third of what we start, and that’s healthy.” Read that through the substrate lens and you see what he’s really doing. He’s manufacturing a correctness signal where the work didn’t come with one. He’s building the missing piece of scaffolding by hand.

That gives you a simple lens to run any candidate use case through. Does the work break into discernible stages? Can you observe what happens at each one? Is there a usable signal for whether the result was right? Score high on all three and you have a software-engineering-shaped problem, so go now. Score low and you have a choice: build the substrate first or wait. What you shouldn’t do is fund it at scale and hope the ROI materializes, because that’s the pile the 19% number is built on.

The hard part, and the honest caveat

Run the professions through that lens and they sort themselves. Finance is the closest cousin to software. Reconciliation, close processes, approval chains and audit trails already give you staged work with a clear “it reconciles or it doesn’t” signal, which is part of why financial services sits among the sectors furthest along with AI. Legal and medicine are harder. The workflow shell exists, intake to redline to filing, diagnosis to treatment to follow-up, but the correctness signal at the core is weak, delayed or confounded. You can automate the routine staged parts and you hit a wall at the judgment that defines the profession.

And that’s the caveat that keeps this honest. A structured workflow isn’t always buildable in software’s image. For the judgment core of some professions, the substrate is years out no matter how good the model gets or how mature your governance becomes. Anyone selling you a tighter timeline than that is selling.

But notice what this reframe does. It turns “our AI ROI is elusive” from a mystery you wait out into a sequencing-and-instrumentation problem you can actually test. Your timeline isn’t set by how smart the next model is. It’s set by how fast you build the substrate for your own domain, and that’s within your control.

The two numbers, reconciled

Put the 80% and the 19% back next to each other and they stop looking like a paradox. Software engineering didn’t win because its models were better than everyone else’s. It won because the work was already shaped to let an agent succeed, and the scaffolding that makes agent output trustworthy had been in place for decades before the agent showed up.

The question for the rest of the enterprise was never really whether AI can do the work. It’s whether your work is shaped so AI’s output can be trusted. That’s not something you wait for. It’s something you build.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

With change a constant, cybersecurity professionals looking to improve their careers can benefit from the latest insights into employers’ needs. Data from Foote Partners on the skills and certification most in demand today may provide helpful signposts.

Analyzing more than 660 certifications as part of its 2Q 2026 “IT Skills Demand and Pay Trends Report,” Foote Partners calculated the most valuable IT security certifications to pursue right now based on two dimensions. The first, the average pay premium, measures the difference in pay between IT pros with a particular credential and those without it. The second, market value increase, measures the increase in pay gains over the past six months.

Together, average pay premium and market value increase can give cybersecurity pros a starting point in deciding which certification to pursue for more pay. Apart from considering their overall professional goals, security professionals should consider each certification’s training and exam costs, whether vendor-specific or vendor-neutral, and the lateral or vertical role opportunities it may open.

Here are the top 13 certifications paying higher premiums today in descending order.

GIAC Security Expert (GSE)

The GIAC Security Expert (GSE) portfolio certification is for security leaders wishing to prove their status as a top information security practitioner by showing they have offensive and defensive skills and hands-on practical skills. Available for more than 15 years, the GSE is considered one of the broadest and deepest cybersecurity certifications. To earn the certification, candidates must complete any six practitioner certifications and any four applied knowledge certifications.

GIAC allows candidates to customize the certification to fit their expertise and career. Candidates can also build their certification over any amount of time as along as the required certifications within the portfolio remain active. Practitioner certification exams are 2-5 hours in length, depending on the specific certification attempt, and applied knowledge certification exams are 4 hours in length.

Training fees: Some training is offered in affiliation with SANS Institute and costs $8,780.

Exam Fees: Because you need 10 certifications to achieve the GSE prices vary significantly. If you already hold a GIAC Certified Forensic Analyst (GCFA), the cost of one of the required certifications drops from $1,299 to $499. Most required certifications are priced at either $999 or $1,299 per attempt, though they can cost up to $11,190.

GIAC Security Professional (GSP)

The GIAC Security Professional (GSP) is designed to demonstrate the holder’s depth and breadth of information security knowledge. Launched approximately two years, this newer certification is the halfway point to the GSE. Customization of the certification is allowed, and to achieve it a candidate must complete any three practitioner certifications and any two applied knowledge certifications. Candidates can also build their certification over any amount of time as along as the required certifications within the portfolio remain active. Practitioner Certification exams are 2-5 hours in length, depending on the specific certification attempt, and Applied Knowledge Certification exams are 4 hours in length.

Training fees: Some training is offered in affiliation with SANS Institute and costs $8,780.

Exam Fees: Because you need five certifications to achieve the GSP prices vary significantly. If you already hold a GIAC Security Essentials (GSEC), the cost of one of the required certifications drops from $1,299 to $499. Most certifications required are priced at either $999 or $1,299 per attempt, though certification can cost up to $5,595.

Microsoft Certified Azure Cybersecurity Architect Expert

Those who earn the Microsoft Certified: Cybersecurity Architect Expert credential are able to translate a cybersecurity strategy into capabilities that protect the assets, business, and operations of an organization. Through the certification process, candidates learn to design, guide the implementation of, and maintain security solutions that follow zero-trust principles and best practices. You’ll also be able to design solutions for governance, risk, and compliance (GRC), security operations, and security posture management.​

As a prerequisite, candidate must have earned one of the following: Microsoft Certified: Azure Security Engineer Associate, Microsoft Certified: Identity and Access Administrator Associate, Microsoft Certified: Security Operations Analyst Associate certification.

Training fees: Self-paced training is available from the course’s page and free of charge. There is also an option to find an instructor-led training with pricing starting at $1,300.

Exam Fees: The exam costs $165 and Microsoft offers free practice assessments.

Certificate of Cloud Security Knowledge (CCSK)

As a certificate and not a certification — an important distinction — the Cloud Security Alliance (CSA) positions its Certificate of Cloud Security Knowledge as the foundation for future credentials and upskilling in the sector. From this perspective, the CCSK is helpful for cybersecurity analysts, compliance managers, security engineers, architects, and administrators. This vendor-neutral certificate has been recently updated and covers topics in zero trust, DevSecOps, cloud telemetry and security analytics, artificial intelligence, and more. CCSK offers a variety of training modalities, including an exam prep kit, instructor-led classes offered virtually and in person, and an online self-paced option. Candidates must score at least 80% on the exam, randomly pulling 60 multiple-choice questions from a test bank.

Training fees: Prices vary based on modality. A self-paced course and exam bundle costs $795, and online, instructor-led training begins at $995.

Exam fees: The exam costs $445, though discounts are available for corporate members, and US military veterans can take it for free.

Certified in Risk and Information Systems Control (CRISC)

Administered by ISACA, the Certified in Risk and Information Systems Control certification provides candidates with training across four domains: corporate IT governance, risk assessment, risk response and reporting, and technology and security. CRISC is ideal for candidates who want to enhance and optimize business resilience and risk management across their organization. The exam consists of 150 questions across the four domains. Since ISACA began offering CRISC in 2010, more than 23,000 people have obtained the certification. ISACA claims 52% of certificate holders experienced on-the-job improvement, and CRISC is the “4th top-paying certification worldwide.” To qualify for CRISC, candidates must adhere to a code of professional ethics and have three years of work experience in risk assessment and risk response and reporting. On passing the exam, candidates must submit 20 CPE credits annually and 120 continuing professional education (CPE) hours every three years to maintain their CRISC.

Training fees: ISACA offers three resources: an online review course, $895; a review manual in print or digital, $139; and an annual subscription to a 833-question test bank, $399. Discounts are available for ISACA members.

Exam fees: $575, ISACA members; $760 for non-members; plus $50 application fee.

Certified Information Systems Auditor (CISA)

The Information Systems Audit and Control Association (ISACA)’s CISA is geared toward IT auditors who wish to upskill or earn a pay boost. According to ISACA, 70% of CISA holders report on-the-job improvement, and another 22% receive a raise. The course covers five domains: information systems auditing, implementation, and operations; protection of information assets; and IT governance. The four-hour exam consists of 150 multiple-choice questions, and candidates must earn 450 on ISACA’s scaled scoring system, with 800 representing a perfect score. To maintain their CISA, certification holders must take 20 CPE credits annually and 120 over three years through conferences, volunteering, on-demand learning, and other methods as well as paying maintenance fee. To qualify, you must have five years of experience in IT or IS audit, control, assurance, or security. You can apply for an experience waiver for up to three years.

Training fees: ISACA offers four resources: an online review course for $895, an annual subscription to a question bank for $399, and a print or digital review manual for $139. Discounts are available for ISACA members. 

Exam fees: $575, members; $760, non-members; plus $50 application fee.

Certified Information Systems Security Professional (CISSP)

CISSP is a generalist cert from ISC2 aimed at security pros who have already established a strong track record. Advanced-level analysts interested in getting CISSP certified will need to know all the ins and outs of security and risk management, asset security, operations, security assessment and testing, and more. The CISSP certification requires five years of full-time experience in at least two of its eight domains. The exam is adaptive, ranging from 100 to 150 questions, including multiple-choice and advanced items of varying formats. Candidates need to score 700 points out of 1,000 to pass the exam.

Training fees: Online self-paced training fees start at $595 and can cost up to $1,993; online instructor-led bootcamp costs $2,880.

Exam fee: $749

Certified Secure Software Lifecycle Professional (CSSLP)

This ISC2 certification helps cyber pros build their career by training them to better incorporate security practices throughout software development phases. The CSSLP exam evaluates experience across eight domains: secure software concepts; secure software; lifecycle management; secure software requirements; secure software architecture and design; secure software implementation; secure software testing; secure software deployment, operations, maintenance; secure software supply chain. Those wishing to acquire the CSSLP must have four years of paid work experience as a software development lifecycle professional in one or more of the eight domains.

Training fees: Online self-paced training fees start at $550 and can cost up to $1,718; online instructor-led bootcamp costs $2,650.

Exam fee: $599

Check Point Certified Security Master (CCSM)

To become a Check Point Certified Security Master (CCSM) security professionals must have an active Certified Security Expert (CCSE) and mast have completed two subsequent Check Point Specialist accreditations. CCSM validates advanced expertise in configuring, deploying, and troubleshooting Check Point solutions. Check Point certifications are valid for 24 months.

Training fees: Training for CCSE is $3,500

Exam fee: The fee for CCSE is $300

GIAC Experienced Cybersecurity Specialist (GX-CS)

The Experienced Cybersecurity Specialist (GX-CS) sits within the applied knowledge certifications with GIAC. The certification is for practitioners to show their qualifications for advanced, hands-on IT systems roles across cybersecurity. Its intent is to demonstrate the candidate can navigate evolving real-world threats. The certification covers five areas: network security analysis and tools; evaluation of Windows and Linux OS security; advanced security tools and techniques; common attacks and defenses; and implementing overall cybersecurity and information security. The GX-CS is for GSEC holders who acquired additional experience — the GSEC exam costs $999, and SANS Institute offers training for GSEC.

Training fees: There are a few related affiliate training programs provided by SANS, each costing approximately $9,000.

Exam fee: $499 for those with an active GSEC; otherwise $1,299.

OffSec Certified Professional (OSCP+)

To earn the OffSec Certified Professional certification, candidates must complete the affiliated course, PEN-200: Penetration Testing with Kali Linux, and pass the subsequent exam. The course covers 20 plus modules, including information gathering, vulnerability scanning, encryption and cryptography, Active Directory and AWS exploitation, and more. Certificate holders will have shown mastery of penetration testing methodologies ideal for new roles, such as an ethical hacker, incident responder, or threat hunter. The OSCP+ exam is entirely hands-on, and test-takers must compromise systems within a lab environment.

OffSec does not enforce any prerequisites but recommends candidates be familiar with TCP/IP networking, scripting in Bash and Python, and Linux and Windows, which they can learn through its Network Penetration Testing Essentials Learning Path.

Training and exam fees: OffSec bundles the course and exam for $1,749 and as a yearly subscription that includes access to one 200 or 300-level course, the associated labs, and two exam attempts for $2,749 annually.

OffSec Experienced Penetration Tester (OSEP)

The OffSec Experienced Penetration Tester is ideal for penetration testers and ethical hackers who need more advanced techniques to sharpen offensive skills against modern enterprise defenses. Across more than 20 modules, the certification introduces these professionals to advanced offensive techniques, EDR and AV evasion, advanced Windows offensive security and more. During the two-day proctored exam, professionals must connect to a lab environment via a VPN and compromise multiple machines within a network through several possible attack paths. To pass, professionals must achieve the objective stated within the control panel or score at least 100 points — 10 points are awarded for every flag found in a local.txt or proof.txt file. Professionals who earn their OSEP can also obtain their OSCE³ Certification to demonstrate their mastery of offensive security. They would also need to pass the exams for WEB-300: Advanced Web Attacks and Exploitation and EXP-301: Windows User Mode Exploit Development, after which the OSCE³ is automatically awarded.

While there are no formal prerequisites for OSEP, OffSec recommends candidates take the PEN-200: Penetration Testing with Kali Linux or have a strong foundation in operating systems, networking, and scripting. 

Training and exam fees: OffSec bundles the course and exam for $1,749, and as a yearly subscription that includes access to one 200 or 300-level course, the associated labs, and two exam attempts for $2,749 annually.

OffSec Exploitation Expert (OSEE)

OffSec’s Offensive Security Exploitation Expert is a vendor-specific certification, focusing on advanced Windows exploitation, with OffSec deeming it its most challenging certification. As a penetration testing course, the material dives deep into topics such as advanced heap manipulations and disarming WDEG mitigations. Certificate holders can identify problematic code in Windows operating systems and develop exploits. For the practical exam, candidates must complete a comprehensive penetration test of software and create an exploit within a lab environment — all within 72 hours. To qualify, you must have experience debugging, developing Windows exploits, and using the following technologies: WinDBG, x86_64, IDA Pro, and basic C/C++ programming. OffSec recommends completing its 300-level certifications before OSEE.

Training and exam fees: OffSec offers only instructor-led, in-person training. Enterprises should inquire for more information.

Anthropic has found a way to shed new light on how its models solve problems, thanks to its discovery of what it has dubbed the J-space. 

“We find that Claude has developed a small collection of internal neural patterns that, compared to all its other internal processing, play a special role. We call the collection of these patterns the J-space, named after the technique we used to find them, involving a mathematical concept called the Jacobian,” Anthropic said in its post about the discovery. It examines the contents of the J-space using what it calls the Jacobian lens, or J-lens.

“Each J-space pattern is linked to a particular word,” Anthropic said. “But when one of these patterns lights up, it doesn’t mean the model is saying that word, just that the word is on its ‘mind.’ If you’ve heard of language models having a scratchpad or chain of thought—text they write to themselves while reasoning—the J-space is something different. It operates silently, in the model’s internal neural activations, allowing the model to ‘think’ about a concept without writing it down.”

This new level of analytical visibility goes well beyond what Anthropic announced as an internal scratchpad for its models in 2024. That scratchpad revealed what the model was considering when preparing an action or delivering an answer. The new development instead focuses on something much deeper which has the potential to change how AI systems are evaluated and purchased. 

One example in the paper described how some models did not engage in improper behavior during tests, which would appear to be a very favorable result. But the contents of the J-space revealed that the model sometimes knew that it was being tested, and that awareness might have been the key reason it declined to engage in the problematic behavior, much in the way human children act when they know they are being watched. 

“Anthropic built a lens that catches its own model quietly noticing it’s being tested, faking a result to look good, spotting a prompt injection, or sitting on a planted goal it hasn’t acted on yet,” said Rock Lambros, director of AI standards and governance at AI agent vendor Zenity. “Some of that good behavior rode on the model knowing it was on stage.”

Customers should read their safety benchmarks with that in mind, he said. “Fitness for your project still comes from testing on your own data and your own attackers, not from a leaderboard the model knew it was sitting for.”

That kind of visibility is a potentially crucial tool for CIOs.

“A provider that can catch its own model misbehaving in silence, then publish [those results], is telling you something real about its assurance maturity. Put that in your due diligence, not just your newsfeed,” Lambros noted. “Here’s the question I’d hand every model vendor now: what can you see inside your model that I can’t see in its output, and what have you caught?”

Added Noah Kenney, principal consultant at Digital 520: “A model that behaves better because it knows it is being watched is not a safe model. It is a model with a poker face. We have to question every red team result, every internal pilot where the model refused something dangerous, and every ‘we tested this and it was fine’ story, because they now carry an asterisk.”

CIOs need to now determine whether an agent performed a function in a specific way because that is how it will always perform, or whether it was it behaving differently because it figured out you were just testing it, Kenney said. “The answer to that question should change your interpretation in a material way.”

No J-lens for customers – yet

“It is an admission that the industry’s evaluation regime is measuring something less durable than everyone assumed, and now the other frontier labs have to answer whether their own evaluations have the same problem,” Kenney said. “For CIOs, the paper is a warning about their entire model risk framework.”

Flavio Villanustre, CISO for the LexisNexis Risk Solutions Group, said that examining the J-space can even help make models more efficient.

“It gives you the ability to introspect into the model and, as such, can be very useful to the user, especially in cases where explainability is important. Think regulated environments that require explainable responses and full causal analysis of them,” Villanustre said. “This can also be very helpful to users trying to fine tune their prompts, making models more efficient to optimize token cost.”

But currently indirect access, or future access achieved via AI vendor negotiations, is the only path for accessing the new information, though Villanustre noted that some enterprises could gain direct access to J-space by paying for Anthropic’s FDE program.

“It is very useful to CIOs,” he pointed out, “but in order to make use of the capabilities offered by analysis of the J-space, they need appropriate talent that can make sense of it. The type of skills required go far beyond those of the general data analyst, or even data scientist.”

Today, said Aman Mahapatra, chief strategy officer for Tribeca Softtech, a New York City-based technology consulting firm, “enterprise customers cannot enable the Jacobian lens, cannot inspect the residual stream through the API, and cannot run the ablation studies that produced the most interesting findings in the paper.”

So, he said, “on the narrow question of whether a CIO can operationally use J-space monitoring in Q3 of this year to gate a production deployment, the answer is no.”

But Mahapatra argued that there are going to be other ways to access the information, and CIOs must insist on them.

“Without customer-side access, this reduces to trusting Anthropic yet again, and that is exactly why enterprises should start pushing for a different assurance model industry-wide,” he said. “Model providers are converging on a posture where they inspect their own models using proprietary tooling and publish reassuring research about what they found. That is not an assurance framework any regulated industry accepts from any other vendor.”

He pointed out that banks do not accept “we validated our own model, trust us” from a credit scoring vendor, not does the healthcare industry accept it from a clinical decision support vendor. “There is no principled reason to accept it from a foundation model vendor either, and the J-space research crystallizes why,” he said.

New visibility demands

“The right long-term enterprise posture is to demand independent interpretability access, either through customer-facing APIs, through independent third-party auditors with privileged access, or through open interpretability standards that let a bank’s model risk management team apply the same tooling the vendor’s own safety team uses,” Mahapatra stressed. “None of that exists today. All of it should be on the roadmap CIOs are pushing for, and this research is the strongest argument yet for why.”

In fact, the discoveries in the research have the potential to fundamentally rewrite the AI strategy rules.

Mahapatra said that the single hardest problem in enterprise agentic deployment is verifying that an autonomous system’s stated reasoning matches its actual reasoning. “Until now, we could only audit what the model writes, while much of its reasoning happened silently. The J-lens attacks that gap head-on,” he noted.

Thus, he said, sophisticated buyers should start asking model providers during the procurement process about the interpretability tooling they offer to let customers monitor internal model state for deception, evaluation-gaming, and goal misalignment in their specific deployments.

“Almost no vendor can answer that today,” he said. “The CIOs who start requiring internal-state observability as a procurement criterion, even before the tooling is fully mature, will be the ones who shape how their vendors productize it, and the ones with genuine assurance when regulators start asking how they know their autonomous agents are actually doing what they claim.”

The beginning of standards

Another way that CIOs can benefit from this new visibility into Claude is to try and get that information from third-parties that already have access. The report, for example, noted that a Google AI specialist independently replicated some findings on an open-weight model.

That, noted Lewis Carhart, CEO of Comp AI, a software development firm, “is a competitor verifying the method, not just the vendor’s own claim. It shows what’s technically possible, but it doesn’t give enterprises a way to check anything themselves.”

He said that it’s a pattern that compliance has seen before; SOC 2 didn’t start as an independent audit standard either. It started as vendors describing their own controls, and the market spent years building the infrastructure to verify those claims externally.

“Interpretability is at that same starting point now,” he noted. “It becomes meaningful for CIOs once J-lens findings show up in third-party audits, published model cards, or regulator-facing disclosures. Anything a risk team can point to that isn’t just the vendor’s word.”

Leads to AI strategy changes

Justin Greis, CEO of consulting firm Acceligence, said he also expects this development to lead to major AI strategy changes. 

“I can easily imagine governance platforms consuming those signals alongside prompts, outputs, identity information, policy decisions, and tool activity,” he said. “A future AI control plane could continuously evaluate whether an agent recognized an attempted prompt injection, understood that sensitive information was involved, detected conflicting objectives, or showed evidence that it was reasoning toward an unsafe action before that action was ever executed. Those signals become inputs into policy enforcement, human escalation, audit logging, and trust scoring across enterprise AI environments.”

This has practical implications for CIOs today, he pointed out, “because it changes how they evaluate AI vendors. A year ago, enterprises primarily asked about model accuracy, latency, security, and cost. Increasingly, procurement teams will also ask how much operational visibility vendors provide into agent behavior, reasoning quality, policy compliance, safety monitoring, and auditability.”

Mahapatra added that all of this could give CIOs a powerful new negotiating tactic.

“The renewal path is where the leverage actually sits: write contractual rights to interpretability reporting and third-party audit access into the next renewal, because those terms are free today and expensive after signature,” he said. “The CIOs who win on assurance in 2027 will be the ones who stopped accepting ‘trust us’ from their model provider in 2026 and put the right clauses in the paperwork while the vendor still needed the deal more than the customer needed the model.”

This article originally appeared on CIO.com.

Anthropic has found a way to shed new light on how its models solve problems, thanks to its discovery of what it has dubbed the J-space. 

“We find that Claude has developed a small collection of internal neural patterns that, compared to all its other internal processing, play a special role. We call the collection of these patterns the J-space, named after the technique we used to find them, involving a mathematical concept called the Jacobian,” Anthropic said in its post about the discovery. It examines the contents of the J-space using what it calls the Jacobian lens, or J-lens.

“Each J-space pattern is linked to a particular word,” Anthropic said. “But when one of these patterns lights up, it doesn’t mean the model is saying that word, just that the word is on its ‘mind.’ If you’ve heard of language models having a scratchpad or chain of thought—text they write to themselves while reasoning—the J-space is something different. It operates silently, in the model’s internal neural activations, allowing the model to ‘think’ about a concept without writing it down.”

This new level of analytical visibility goes well beyond what Anthropic announced as an internal scratchpad for its models in 2024. That scratchpad revealed what the model was considering when preparing an action or delivering an answer. The new development instead focuses on something much deeper which has the potential to change how AI systems are evaluated and purchased. 

One example in the paper described how some models did not engage in improper behavior during tests, which would appear to be a very favorable result. But the contents of the J-space revealed that the model sometimes knew that it was being tested, and that awareness might have been the key reason it declined to engage in the problematic behavior, much in the way human children act when they know they are being watched. 

“Anthropic built a lens that catches its own model quietly noticing it’s being tested, faking a result to look good, spotting a prompt injection, or sitting on a planted goal it hasn’t acted on yet,” said Rock Lambros, director of AI standards and governance at AI agent vendor Zenity. “Some of that good behavior rode on the model knowing it was on stage.”

Customers should read their safety benchmarks with that in mind, he said. “Fitness for your project still comes from testing on your own data and your own attackers, not from a leaderboard the model knew it was sitting for.”

That kind of visibility is a potentially crucial tool for CIOs.

“A provider that can catch its own model misbehaving in silence, then publish [those results], is telling you something real about its assurance maturity. Put that in your due diligence, not just your newsfeed,” Lambros noted. “Here’s the question I’d hand every model vendor now: what can you see inside your model that I can’t see in its output, and what have you caught?”

Added Noah Kenney, principal consultant at Digital 520: “A model that behaves better because it knows it is being watched is not a safe model. It is a model with a poker face. We have to question every red team result, every internal pilot where the model refused something dangerous, and every ‘we tested this and it was fine’ story, because they now carry an asterisk.”

CIOs need to now determine whether an agent performed a function in a specific way because that is how it will always perform, or whether it was it behaving differently because it figured out you were just testing it, Kenney said. “The answer to that question should change your interpretation in a material way.”

No J-lens for customers – yet

“It is an admission that the industry’s evaluation regime is measuring something less durable than everyone assumed, and now the other frontier labs have to answer whether their own evaluations have the same problem,” Kenney said. “For CIOs, the paper is a warning about their entire model risk framework.”

Flavio Villanustre, CISO for the LexisNexis Risk Solutions Group, said that examining the J-space can even help make models more efficient.

“It gives you the ability to introspect into the model and, as such, can be very useful to the user, especially in cases where explainability is important. Think regulated environments that require explainable responses and full causal analysis of them,” Villanustre said. “This can also be very helpful to users trying to fine tune their prompts, making models more efficient to optimize token cost.”

But currently indirect access, or future access achieved via AI vendor negotiations, is the only path for accessing the new information, though Villanustre noted that some enterprises could gain direct access to J-space by paying for Anthropic’s FDE program.

“It is very useful to CIOs,” he pointed out, “but in order to make use of the capabilities offered by analysis of the J-space, they need appropriate talent that can make sense of it. The type of skills required go far beyond those of the general data analyst, or even data scientist.”

Today, said Aman Mahapatra, chief strategy officer for Tribeca Softtech, a New York City-based technology consulting firm, “enterprise customers cannot enable the Jacobian lens, cannot inspect the residual stream through the API, and cannot run the ablation studies that produced the most interesting findings in the paper.”

So, he said, “on the narrow question of whether a CIO can operationally use J-space monitoring in Q3 of this year to gate a production deployment, the answer is no.”

But Mahapatra argued that there are going to be other ways to access the information, and CIOs must insist on them.

“Without customer-side access, this reduces to trusting Anthropic yet again, and that is exactly why enterprises should start pushing for a different assurance model industry-wide,” he said. “Model providers are converging on a posture where they inspect their own models using proprietary tooling and publish reassuring research about what they found. That is not an assurance framework any regulated industry accepts from any other vendor.”

He pointed out that banks do not accept “we validated our own model, trust us” from a credit scoring vendor, not does the healthcare industry accept it from a clinical decision support vendor. “There is no principled reason to accept it from a foundation model vendor either, and the J-space research crystallizes why,” he said.

New visibility demands

“The right long-term enterprise posture is to demand independent interpretability access, either through customer-facing APIs, through independent third-party auditors with privileged access, or through open interpretability standards that let a bank’s model risk management team apply the same tooling the vendor’s own safety team uses,” Mahapatra stressed. “None of that exists today. All of it should be on the roadmap CIOs are pushing for, and this research is the strongest argument yet for why.”

In fact, the discoveries in the research have the potential to fundamentally rewrite the AI strategy rules.

Mahapatra said that the single hardest problem in enterprise agentic deployment is verifying that an autonomous system’s stated reasoning matches its actual reasoning. “Until now, we could only audit what the model writes, while much of its reasoning happened silently. The J-lens attacks that gap head-on,” he noted.

Thus, he said, sophisticated buyers should start asking model providers during the procurement process about the interpretability tooling they offer to let customers monitor internal model state for deception, evaluation-gaming, and goal misalignment in their specific deployments.

“Almost no vendor can answer that today,” he said. “The CIOs who start requiring internal-state observability as a procurement criterion, even before the tooling is fully mature, will be the ones who shape how their vendors productize it, and the ones with genuine assurance when regulators start asking how they know their autonomous agents are actually doing what they claim.”

The beginning of standards

Another way that CIOs can benefit from this new visibility into Claude is to try and get that information from third-parties that already have access. The report, for example, noted that a Google AI specialist independently replicated some findings on an open-weight model.

That, noted Lewis Carhart, CEO of Comp AI, a software development firm, “is a competitor verifying the method, not just the vendor’s own claim. It shows what’s technically possible, but it doesn’t give enterprises a way to check anything themselves.”

He said that it’s a pattern that compliance has seen before; SOC 2 didn’t start as an independent audit standard either. It started as vendors describing their own controls, and the market spent years building the infrastructure to verify those claims externally.

“Interpretability is at that same starting point now,” he noted. “It becomes meaningful for CIOs once J-lens findings show up in third-party audits, published model cards, or regulator-facing disclosures. Anything a risk team can point to that isn’t just the vendor’s word.”

Leads to AI strategy changes

Justin Greis, CEO of consulting firm Acceligence, said he also expects this development to lead to major AI strategy changes. 

“I can easily imagine governance platforms consuming those signals alongside prompts, outputs, identity information, policy decisions, and tool activity,” he said. “A future AI control plane could continuously evaluate whether an agent recognized an attempted prompt injection, understood that sensitive information was involved, detected conflicting objectives, or showed evidence that it was reasoning toward an unsafe action before that action was ever executed. Those signals become inputs into policy enforcement, human escalation, audit logging, and trust scoring across enterprise AI environments.”

This has practical implications for CIOs today, he pointed out, “because it changes how they evaluate AI vendors. A year ago, enterprises primarily asked about model accuracy, latency, security, and cost. Increasingly, procurement teams will also ask how much operational visibility vendors provide into agent behavior, reasoning quality, policy compliance, safety monitoring, and auditability.”

Mahapatra added that all of this could give CIOs a powerful new negotiating tactic.

“The renewal path is where the leverage actually sits: write contractual rights to interpretability reporting and third-party audit access into the next renewal, because those terms are free today and expensive after signature,” he said. “The CIOs who win on assurance in 2027 will be the ones who stopped accepting ‘trust us’ from their model provider in 2026 and put the right clauses in the paperwork while the vendor still needed the deal more than the customer needed the model.”