Enterprise Architecture (EA) Tools unterstützen Unternehmen und Organisationen dabei, mit ihren IT-Strategien die Geschäftsziele optimal zu unterstützen. Sie sorgen ebenfalls dafür, dass Unternehmen ihre Roadmaps für die digitale Transformation geordnet vorantreiben können. EA Tools bieten dafür unter anderem Collaboration-, Reporting-, Testing- und Simulationsfunktionen. Mit deren Hilfe lassen sich Modelle implementieren, die Geschäfts- und IT-Prozesse gezielt verbessern.

Um die beste Lösung für Ihr Unternehmen zu finden, sollten Sie zuerst prüfen, ob sich das jeweilige Tool mit Ihrem Technologie-Stack integrieren lässt. Anschließend gilt es abzuwägen, ob die Informationen, Diagramme und Tabellen, die die Software zur Verfügung stellt, für das Unternehmen auch einen echten Nutzwert haben.

Empfehlenswerte Enterprise-Architecture-Tools

Nachfolgend finden Sie einen Überblick über die wichtigsten Enterprise-Architecture-Tools – in alphabetischer Reihenfolge. Sie stellen einen Mix aus Visualisierungs-, Collaboration- und Project-Management-Funktionen bereit und unterstützen eine Vielzahl von Enterprise Architecture Frameworks.

Ardoq

Nachdem zuerst über einfache Formulare Informationen von Usern, Entwicklern und sonstigen Stakeholdern im Unternehmen eingesammelt wurden, lässt sich mithilfe von Ardoq ein digitaler Zwilling der gesamten Organisation erstellen. Der Ansatz setzt also darauf, die Menschen, die in ihren Rollen mit den verschiedensten Systemen arbeiten, realistisch in ihrer Arbeitswelt abzubilden.

Jede Mitarbeiterin und jeder Mitarbeiter im Unternehmen kann später von den Netzwerkvisualisierungen und Datenfluss-Diagrammen profitieren, um seine eigene Rolle optimal zu unterstützen und den Arbeitsplatz immer wieder anzupassen und zu modernisieren. Das Tool lässt sich mit den wichtigsten Cloud-Plattformen integrieren. Es bietet eine API, die individuelle Anpassungen in allen wichtigen Programmiersprachen (Python, C#, Java, etc.) ermöglicht.

Wichtigste Use Cases:

• “Architektonischen Stress” bei Lastspitzen simulieren, falls größere Veränderungen bevorstehen;

• Verstehen, wie verändertes Nutzerverhalten neue Anforderungen generiert;

• Application Portfolio Management, um besser strategisch zu planen.

Atoll Group SAMU

Das EA-Tool SAMU macht die Enterprise Architecture sichtbar, indem es tiefe Verknüpfungen zwischen On-Premises-Systemen, dem Cloud-Layer und Tools für das Business Process Management aufzeigt. Das Tool der Atoll Group bietet vielfältige Integrationsmöglichkeiten, zum Beispiel mit Monitoring-Tools (etwa Tivoli, ServiceNow), Configuration-Management-Datenbanken (zum Beispiel CA, BMC) oder Service-Organisations-Tools (BMC, HPE). Alle Informationen fließen in ein zentrales Datenmodell ein, das um den zusätzlichen Input der Stakeholder weiter angereichert wird.

Wichtigste Use Cases:

• Enterprise-Architektur visualisieren;

• strategische Planungsprozesse und Architektur-Reviews mit Informationen unterfüttern;

• mithilfe einer visuellen Verständnisgrundlage die Kommunikation verbessern.

Avolution Abacus

Dieses Tool erfasst die Breite und den Umfang der Unternehmensarchitektur mit Hilfe eines auf Diagrammen basierenden Dashboards. Die Integration mit gängigen Tools wie SharePoint, Excel, Visio, Google Sheets, Technopedia oder ServiceNow vereinfacht die Nutzung. Abacus wurde inzwischen auch um einen Machine-Learning-Layer ergänzt, der es Anwendern ermöglicht, ein Modell zu trainieren, das ihnen beispielsweise hilft zu erkennen, wer im Unternehmen für welches System verantwortlich ist.

Wichtigste Use Cases:

• die IT für das gesamte Unternehmen “öffnen”, um ein allgemeines Verständnis der Datenflüsse zu erzeugen;

• umfassendes Enterprise Modeling, um eine Roadmap für künftige Entwicklungen zu erstellen;

• Business-Metriken tracken, die mit der Unternehmens-Performance zusammenhängen.

BOC Group ADOIT

ADOIT soll Teams dabei unterstützen, Ressourcen zu verwalten, Bedarfe vorherzusagen und Assets zu tracken. Dazu mappt das Tool jedes System oder Softwarepaket mit einem Objekt. Die Datenflüsse zwischen den Systemen werden in Beziehungen umgewandelt, die von diesen Objekten mithilfe eines anpassbaren Metamodells erfasst werden. Geschäftsprozesse können auf ähnliche Weise über ein gut integriertes Begleitprodukt namens ADONIS modelliert werden. ADOIT ist Web-basiert und lässt sich auch mit Tools wie Atlassian Confluence integrieren, um die Datenerfassung und -entwicklung zu beschleunigen.

Wichtigste Use Cases:

• ein unternehmensweites Modell erstellen, das bei sämtlichen Teammitgliedern ein Verständnis über den Stack schafft – und wie man diesen verbessern kann;

• vollständiger Zugriff auf EA-Daten über eine Mobile-Anwendung;

• bei Fusionen und Übernahmen den Tech-Bereich durch genaues Asset-Mapping orchestrieren.

Bizzdesign Hopex

Nach der Übernahme von Mega International zählt die Hopex-Plattform zum Portfolio von Bizzdesign. Sie soll dabei unterstützen, Unternehmensanwendungen zu modellieren und dabei ein Verständnis der von ihnen unterstützten Geschäfts-Workflows schaffen. Dabei liegt ein Schwerpunkt auf den Bereichen Data Governance und Risikomanagement. Hopex basiert auf Microsoft Azure und stützt sich auf eine Reihe offener Standards wie GraphQL und REST Queries, um Informationen aus Komponentensystemen zu sammeln. Das Reporting ist mit den Office-Tools von Microsoft sowie mit grafischen Lösungen wie Tableau und Qlik integriert.

Wichtigste Use Cases:

• datengestützte Erkenntnisse herbeiführen, um Cloud- und Anwendungsbereitstellung zu steuern;

• akkurate Nutzungsmodelle erstellen, um Architekturanforderungen zu verstehen;

• eine Bedarfsschätzung mit Umfragen und anderen Tools vornehmen, um für die Zukunft zu planen.

Bizzdesign Horizzon

Das Tool dient dazu, Business Workflows und den zugrundeliegenden Tech-Stack zu modellieren. Dazu bietet Horizzon ein Graph-basiertes Modell, das Daten von sämtlichen Stakeholdern einsammelt und diese an eine Analytics-Engine weitergibt. Im Ergebnis entstehen Diagramme, die den aktuellen Systemzustand widerspiegeln. Wichtige Schwerpunkte dieses Tools sind Change Management und Zukunftsplanung: Horizzon ist nicht zuletzt dafür konzipiert worden, die Risiken eines Redesigns zu minimieren. Das Toolset unterstützt die wichtigsten Frameworks ArchiMate, TOGAF und BPMN. Neben Mega hat Bizzdesign im Januar 2025 auch den EA-Geschäftsbereich der Software AG – Alfabet – übernommen.

Wichtigste Use Cases:

• Vorhersage zukünftiger Anforderungen durch Predictive Modeling;

• Orchestrieren von Workflows auf der Basis der technischen und der Business-Architektur;

• Antizipieren von Risiken sowie Security- und Governance-Problemen durch die Modellierung von Datensicherheitsanforderungen.

Capstera

Das Tool von Capstera fokussiert darauf, die Business Architecture selbst abzubilden. Value und Process Maps helfen dabei, die Rollen der verschiedenen Unternehmensbereiche zu definieren und nachzuverfolgen. Dabei können im laufenden Prozess Verknüpfungen mit den zugrundeliegenden Softwarprodukten und Tools hinzugefügt werden.

Wichtigste Use Cases:

• Reports erstellen, die sich erst einmal mit der Business-Architektur selbst beschäftigen;

• Beziehungen zwischen Menschen, Abteilungen und Rollen analysieren;

• die langfristige strategische Planung vorantreiben.

Clausmark Bee360

Teammitglieder, die Clausmarks Flaggschiffprodukt Bee360 (früher Bee4IT) verwenden, wollen eine einfache “Single Source of Truth” über die Workflows im Unternehmen. Ziel ist es, verschiedenen betrieblichen Rollen intelligentere Entscheidungen zu ermöglichen. Das Modul Bee360 FM (Finanzmanagement) bietet etwa die Möglichkeit, Kosten nachzuvollziehen und zuzuordnen. Die Anwender können verschiedene solcher Module miteinander verknüpfen, um EAM, Finanzmanagement, Portfolio Management und Agile Planning nahtlos zu integrieren – bei maximaler Transparenz.

Wichtigste Use Cases:

• C-Suite-Ebene befähigen, Projekte zu managen und Assets zuzuweisen;

• präzise digitale Zwillinge entwickeln, um ein Verständnis über Datenflüsse zu schaffen und künftige Erweiterungen zu planen;

• integrierte Wissensdatenbank aufbauen, um alle digitalen Workflows zu tracken.

EAS

Das Essential-Paket von EAS (Enterprise Architecture Solutions) nahm als Open-Source-Projekt seinen Anfang und hat sich inzwischen zu einer kommerziell verfügbaren Cloud-Lösung weiterentwickelt. Das Tool erstellt ein Metamodell, das die Interaktionen zwischen Systemen und Geschäftsprozessen beschreibt. Ebenfalls enthalten sind Pakete, um gängige Business Workflows wie Datenmanagement oder DSGVO-Compliance zu tracken.

Wichtigste Use Cases:

• den technischen Reifegrad der eigenen Architektur evaluieren;

• Sicherheit und Governance durch besseres Asset Tracking optimieren;

• wachsende Systemkomplexität kontrollieren und managen.

OrbusInfinity

Orbus Software hat Anfang 2025 die Akquisition seines Konkurrenten Capsifi abgeschlossen. Der Anbieter stellt mit OrbusInfinity eine Enterprise-Transformation-Plattform auf KI-Basis zur Verfügung, die schnellere, bessere Entscheidungen, Kosteinesparungen und Risikominimierung verspricht. Architecture-Teams sollen mit Hifle von OrbusInfinity mit einer Vielzahl von Stakeholdern interagieren können, um eine “digitale Blaupause” ihres Unternehmens zu generieren, die eine einheitliche Sicht auf das aktuelle und künftige Geschäft realisieren soll. Diverse Drittanbieter-Tools lassen sich außerdem mit der Plattform integrieren, darunter etwa von Microsoft, Flexera, ManageEngine oder ServiceNow.

Wichtigste Use Cases:

• Stakeholder-Management;

• Enterprise-Landschaften visualisieren;

• Entscheidungsfindung und Datenanalyse automatisieren.

Planview Enterprise One

Planview bietet eine ganze Reihe von Produkten, mit denen Unternehmen Teamwork, Prozesse und die Enterprise Architecture nachvollziehen können. Die Enterprise Tools sind in drei Kategorien unterteilt: strategisches Portfolio-Management, Produktportfolio-Management und Projektportfolio-Management. Im Zusammenspiel entstehen hardware- und Software-übergreifende Layer, die rollenbasierte Perspektiven für Führungskräfte und Teammitglieder eröffnen. Das Toolset integriert mit gängigen Ticket-Tracking-Systemen wie Jira, um Workflow-Analysen und Reports zu erstellen. Inzwischen hat Planview nach einer Übernahme neue Tools in sein Portfolio integriert, die früher unter den Namen Daptiv, Barometer und Projectplace bekannt waren.

Wichtigste Use Cases:

• eine langfristige, strategische Vision für die Architekturentwicklung aufbauen;

• Entwicklungsarbeit auf Projektebene tracken und in eine beliebige Strategie integrieren;

• mit Fokus auf die Customer Experience und die Produktstruktur den Change vorantreiben.

QualiWare Enterprise Architecture

Das Enterprise Architecture Tool von QualiWare ist Teil einer größeren Sammlung von Modellierungswerkzeugen, die darauf abzielt, sämtliche Geschäftsprozesse zu erfassen. Beispielsweise ist es möglich, einen digitalen Zwillinge zu bauen, mit dem sich Customer Journeys nachvollziehen lassen. Qualiware hat diverse KI-Algorithmen integriert, um Dokumentation und Process Discovery zu optimieren.

Wichtigste Use Cases:

• ein kollaboratives Ökosystem für Business Manager aufbauen, das ein Verständnis von der Enterprise Architecture vermittelt;

• architektonische Designelemente erfassen, um ein Wissens-Ökosystem rund um den Stack aufzubauen;

• eine breite Beteiligung in Sachen Dokumentationserstellung und -überprüfung fördern.

Quest Erwin Evolve

Das Erwin Evolve Tool von Quest hat sich von einem Datenmodellierungs-Tool zu einem System für Enterprise-Architecture- und Geschäftsprozess-Modellierung weiterentwickelt. Um die Komplexität moderner, ineinandergreifender Softwaresysteme und der von ihnen gemanagten Geschäftsprozesse zu durchdringen, können Anwender auf benutzerdefinierte Datenstrukturen zurückgreifen. Das Web-Tool erstellt Modelle, rollenbasierte Diagramme und andere Visualisierungen, die in allgemein zugängliche Dashboards einfließen. Zum Paket gehört ein KI-basiertes Modellierungs-Tool, das Whiteboard-Skizzen integrieren kann.

Wichtigste Use Cases:

• einen digitalen Zwilling für die strategische Modellierung der Enterprise Data Architecture erstellen;

• Customer Journeys verstehen;

• Services und Systeme mit Application Portfolio Management tracken.

SAP LeanIX Enterprise Architecture Suite

Die Tool-Sammlung von LeanIX umfasst unter anderem Enterprise Architecture Management und andere Bereiche, die für Aufgaben wie SaaS– und Value-Stream-Management wichtig sind – etwa um Cloud-Deployments und darauf laufende Services zu tracken. Die Daten die dabei über die IT-Infrastruktur gesammelt werden, fließen in ein grafisches Dashboard ein. Das Tool ist eng mit wichtigen Cloud-Workflow-Tools wie Confluence, Jira, Signavio und Lucidchart integriert. Das ist für Teams von Vorteil, die diese Tools bereits nutzen, um ihre Entwicklungsstrategien zu planen und umzusetzen. Seit November 2023 ist LeanIX Teil von SAP.

Wichtigste Use Cases:

• Anwendungsmodernisierung und Cloud-Migration managen;

• Obsoleszenz von Software-Services evaluieren;

• Kosten kontrollieren und managen.

ServiceNow

Die Tool-Sammlung von ServiceNow lässt sich auf verschiedene Architekturtypen herunterbrechen, darunter Assets, DevOps, Security und Service. Die Tools katalogisieren die unterschiedlichen Hardware- und Softwareplattformen, um Workflows und Datenflüsse im Unternehmen abzubilden und zu verstehen. Ausführliche Reportings und detaillierte Dashboards ermöglichen Analysen, auf deren Grundlage Risiken minimiert und die Ausfallsicherheit der Systeme erhöht werden können.

Wichtigste Use Cases:

• Tracken von Assets, Services und Systemen, die das Unternehmen ausmachen;

• Governance-Themen, Risikobegrenzung, IT-Management und Security Operations werden in einer Plattform zusammengeführt;

• durch die Integration von CRM-Tools lassen sich auch kundenorientierte Services managen.

Sparx Systems

Um Teams und Projekte verschiedener Größe und Komplexität zu unterstützen, hat Sparx vier Versionen seines EA-Tools entwickelt. Allen gemeinsam ist eine UML-basierte Modellierung, mit der sich die Komponenten komplexer Systeme tracken lassen. Eine Simulations-Engine ermöglicht “War Gaming” und vermittelt ein Verständnis darüber, wie sich Fehler ausbreiten und kaskadieren können. Sparx stellt zudem eine Vielzahl von vorgefertigten Design Patterns bereit, um Teams bei der Modellierung zu unterstützen.

Wichtigste Use Cases:

• Nachfrage- und Lastveränderungen zur Prognose künftiger Anforderungen simulieren;

• (potenzielle) Probleme durch eine Verbindungs-Matrix im Auge behalten;

• Dokumentation erstellen.

Unicom System Architect

System Architect ist eines der Angebote aus Unicoms Team Blue. Es handelt sich um ein Tool, das ein Metamodell verwendet, um automatisiert so viele Daten wie möglich über die laufenden Systeme zu sammeln – manchmal auch durch ein Reverse Engineering von Datenflüssen. Dieses systemweite Datenmodell kann über benutzerdefinierte Dashboards Teammitgliedern aller Rollen zugänglich gemacht werden. Ein weiteres erwähnenswertes Feature: Die Ressourcenzuweisung lässt sich mit Hilfe von Simulationen optimieren.

Wichtigste Use Cases:

• Was-wäre-wenn-Fragen zum Architekturmodell stellen;

• ein Metamodell von Daten und Systemen aufbauen;

• Migrations- und Transformationspläne erstellen.

ValueBlue BlueDolphin

Dieses EA-Tool sammelt Daten auf dreierlei Art:

1. Es importiert Basisdaten auf der Grundlage standardgesteuerter Automatisierung (ITSM, SAM).

2. Es arbeitet mit den Dateiformaten von Architekten und Systemdesignern – etwa ArchiMate oder BPMN.

3. Es gibt Fragebögen an andere Stakeholder heraus, die auf anpassbaren Vorlagen basieren.

Die aufbereiteten Informationen werden in einer visuellen Umgebung bereitgestellt, die Auskunft über die historische Entwicklung von Systemen gibt.

Wichtigste Use Cases:

• systemweite Daten von internen und externen Stakeholdern automatisiert und formularbasiert erfassen;

• zukunftsorientierte Reportings erzeugen, um den Change zu überwachen und voranzutreiben;

• Kooperation und Zusammenarbeit durch offenes Data Reporting fördern.

(fm)

Dieser Artikel ist im Original bei unserer Schwesterpublikation CIO.com erschienen.

Model Context Protocol (MCP), the emerging standard for connecting AI models to external tools and enterprise data, is undergoing its biggest architectural overhaul yet.

The latest release candidate, scheduled for release on July 28, removes protocol-level sessions in favor of a stateless architecture, a change which industry experts say is intended to make MCP easier to deploy across standard cloud infrastructure as enterprises move AI pilots into production.

“The session-based model made sense when MCP servers were local processes on a developer’s laptop. In production, it became an operational tax,” said Muskan Bandta, cloud associate at ZopDev.

“When your infrastructure team asks whether MCP services can scale like other cloud applications, the answer used to be ‘not quite.’ With the move to a stateless architecture, the answer is now yes,” Bandta added.

Earlier versions of the protocol maintained information about every client connection, meaning servers had to keep track of each session throughout an interaction. While that approach worked well for local development, it complicated deployments across multiple servers because requests often had to be routed back to the same machine, limiting scalability and making MCP a less natural fit for modern cloud architectures.

“Under the new stateless design, every request contains the information needed for any available server to process it independently. Applications that need to maintain context across multiple requests can still do so, but developers must now manage that state explicitly rather than relying on the protocol itself,” she said.

This transition to a stateless design goes beyond simplifying infrastructure by fundamentally changing how AI applications manage and share context across tools, according to Amit Jena, AI development manager at IT consulting firm Kanerika.

Instead of keeping application state hidden inside protocol sessions, the new design makes it explicit, allowing AI models to access, reason over, and pass that information between tools, giving developers greater control over how context is preserved and shared across tools, Jena said.

It should also make AI workflows more portable, resilient, and easier to orchestrate across distributed environments, he said.

MCP’s new features

Other changes to MCP include the addition of a Multi Round-Trip Requests (MRTR) mechanism that changes how AI agents request additional information they need to complete a task.

Instead of relying on a persistent connection between the client and server throughout the interaction, the new mechanism lets the server request additional input through a standard request-response exchange before continuing the task, Jena said.

Routable transport headers, another addition, enable API gateways and other networking infrastructure to identify and route MCP requests without inspecting their contents.

They reduce processing overhead, lower latency, and let enterprise teams enforce routing, rate-limiting and security policies more efficiently using existing API management infrastructure, Jena said.

MCP is also getting an updated authorization framework built around OAuth 2.1 and OpenID Connect; interactive MCP Apps; and deterministic caching of tool and resource listings to improve LLM prompt-cache hit rates, potentially saving on token costs.

Rebuilding the trust boundary

The MCP release steering committee also decided to deprecate some legacy features, including Roots, Sampling, Logging, the older HTTP+SSE transport and Dynamic Client Registration, although these will continue to work in this version and any other released over the next year.

The deprecation of Sampling is likely to have the biggest impact because it changes who is responsible for interacting with foundation models, said Jena.

“Sampling let MCP servers invoke the LLM through the client, which meant the server had a callback path into the model without owning that connection. Deprecating it means rebuilding that trust boundary,” Jena said. “Your server now calls the model provider directly. That changes your network architecture, your auth model, and depending on how you’ve built cost attribution, your billing flow.”

The year-long transition period will be enough for teams to audit their sampling dependencies now, said Jena: “The risk is that teams who haven’t implemented sampling themselves won’t know if a third-party MCP server they’re depending on uses it.”

Updated MCP SDKs

To accompany the protocol update, there are updated MCP SDKs for Python, Typescript, Go, and C#. These support both the old and new protocol versions, so new clients can continue communicating with older servers, while updated servers will also support older clients, reducing the risk of immediate disruptions.

That backward compatibility should make the transition largely incremental, except for enterprises that built custom infrastructure around MCP’s earlier session-based architecture, Bandta said.

Identifying and auditing those session dependencies may not be easy, Jena warned.

“Session management complexity tends to be hidden across multiple layers — the gateway config, the deployment scripts, the monitoring dashboards. The code change is small; finding everywhere the assumption lives is what takes time,” he said.

This article first appeared on InfoWorld.

Model Context Protocol (MCP), the emerging standard for connecting AI models to external tools and enterprise data, is undergoing its biggest architectural overhaul yet.

The latest release candidate, scheduled for release on July 28, removes protocol-level sessions in favor of a stateless architecture, a change which industry experts say is intended to make MCP easier to deploy across standard cloud infrastructure as enterprises move AI pilots into production.

“The session-based model made sense when MCP servers were local processes on a developer’s laptop. In production, it became an operational tax,” said Muskan Bandta, cloud associate at ZopDev.

“When your infrastructure team asks whether MCP services can scale like other cloud applications, the answer used to be ‘not quite.’ With the move to a stateless architecture, the answer is now yes,” Bandta added.

Earlier versions of the protocol maintained information about every client connection, meaning servers had to keep track of each session throughout an interaction. While that approach worked well for local development, it complicated deployments across multiple servers because requests often had to be routed back to the same machine, limiting scalability and making MCP a less natural fit for modern cloud architectures.

“Under the new stateless design, every request contains the information needed for any available server to process it independently. Applications that need to maintain context across multiple requests can still do so, but developers must now manage that state explicitly rather than relying on the protocol itself,” she said.

This transition to a stateless design goes beyond simplifying infrastructure by fundamentally changing how AI applications manage and share context across tools, according to Amit Jena, AI development manager at IT consulting firm Kanerika.

Instead of keeping application state hidden inside protocol sessions, the new design makes it explicit, allowing AI models to access, reason over, and pass that information between tools, giving developers greater control over how context is preserved and shared across tools, Jena said.

It should also make AI workflows more portable, resilient, and easier to orchestrate across distributed environments, he said.

MCP’s new features

Other changes to MCP include the addition of a Multi Round-Trip Requests (MRTR) mechanism that changes how AI agents request additional information they need to complete a task.

Instead of relying on a persistent connection between the client and server throughout the interaction, the new mechanism lets the server request additional input through a standard request-response exchange before continuing the task, Jena said.

Routable transport headers, another addition, enable API gateways and other networking infrastructure to identify and route MCP requests without inspecting their contents.

They reduce processing overhead, lower latency, and let enterprise teams enforce routing, rate-limiting and security policies more efficiently using existing API management infrastructure, Jena said.

MCP is also getting an updated authorization framework built around OAuth 2.1 and OpenID Connect; interactive MCP Apps; and deterministic caching of tool and resource listings to improve LLM prompt-cache hit rates, potentially saving on token costs.

Rebuilding the trust boundary

The MCP release steering committee also decided to deprecate some legacy features, including Roots, Sampling, Logging, the older HTTP+SSE transport and Dynamic Client Registration, although these will continue to work in this version and any other released over the next year.

The deprecation of Sampling is likely to have the biggest impact because it changes who is responsible for interacting with foundation models, said Jena.

“Sampling let MCP servers invoke the LLM through the client, which meant the server had a callback path into the model without owning that connection. Deprecating it means rebuilding that trust boundary,” Jena said. “Your server now calls the model provider directly. That changes your network architecture, your auth model, and depending on how you’ve built cost attribution, your billing flow.”

The year-long transition period will be enough for teams to audit their sampling dependencies now, said Jena: “The risk is that teams who haven’t implemented sampling themselves won’t know if a third-party MCP server they’re depending on uses it.”

Updated MCP SDKs

To accompany the protocol update, there are updated MCP SDKs for Python, Typescript, Go, and C#. These support both the old and new protocol versions, so new clients can continue communicating with older servers, while updated servers will also support older clients, reducing the risk of immediate disruptions.

That backward compatibility should make the transition largely incremental, except for enterprises that built custom infrastructure around MCP’s earlier session-based architecture, Bandta said.

Identifying and auditing those session dependencies may not be easy, Jena warned.

“Session management complexity tends to be hidden across multiple layers — the gateway config, the deployment scripts, the monitoring dashboards. The code change is small; finding everywhere the assumption lives is what takes time,” he said.

Model Context Protocol (MCP), the emerging standard for connecting AI models to external tools and enterprise data, is undergoing its biggest architectural overhaul yet.

The latest release candidate, scheduled for release on July 28, removes protocol-level sessions in favor of a stateless architecture, a change which industry experts say is intended to make MCP easier to deploy across standard cloud infrastructure as enterprises move AI pilots into production.

“The session-based model made sense when MCP servers were local processes on a developer’s laptop. In production, it became an operational tax,” said Muskan Bandta, cloud associate at ZopDev.

“When your infrastructure team asks whether MCP services can scale like other cloud applications, the answer used to be ‘not quite.’ With the move to a stateless architecture, the answer is now yes,” Bandta added.

Earlier versions of the protocol maintained information about every client connection, meaning servers had to keep track of each session throughout an interaction. While that approach worked well for local development, it complicated deployments across multiple servers because requests often had to be routed back to the same machine, limiting scalability and making MCP a less natural fit for modern cloud architectures.

“Under the new stateless design, every request contains the information needed for any available server to process it independently. Applications that need to maintain context across multiple requests can still do so, but developers must now manage that state explicitly rather than relying on the protocol itself,” she said.

This transition to a stateless design goes beyond simplifying infrastructure by fundamentally changing how AI applications manage and share context across tools, according to Amit Jena, AI development manager at IT consulting firm Kanerika.

Instead of keeping application state hidden inside protocol sessions, the new design makes it explicit, allowing AI models to access, reason over, and pass that information between tools, giving developers greater control over how context is preserved and shared across tools, Jena said.

It should also make AI workflows more portable, resilient, and easier to orchestrate across distributed environments, he said.

MCP’s new features

Other changes to MCP include the addition of a Multi Round-Trip Requests (MRTR) mechanism that changes how AI agents request additional information they need to complete a task.

Instead of relying on a persistent connection between the client and server throughout the interaction, the new mechanism lets the server request additional input through a standard request-response exchange before continuing the task, Jena said.

Routable transport headers, another addition, enable API gateways and other networking infrastructure to identify and route MCP requests without inspecting their contents.

They reduce processing overhead, lower latency, and let enterprise teams enforce routing, rate-limiting and security policies more efficiently using existing API management infrastructure, Jena said.

MCP is also getting an updated authorization framework built around OAuth 2.1 and OpenID Connect; interactive MCP Apps; and deterministic caching of tool and resource listings to improve LLM prompt-cache hit rates, potentially saving on token costs.

Rebuilding the trust boundary

The MCP release steering committee also decided to deprecate some legacy features, including Roots, Sampling, Logging, the older HTTP+SSE transport and Dynamic Client Registration, although these will continue to work in this version and any other released over the next year.

The deprecation of Sampling is likely to have the biggest impact because it changes who is responsible for interacting with foundation models, said Jena.

“Sampling let MCP servers invoke the LLM through the client, which meant the server had a callback path into the model without owning that connection. Deprecating it means rebuilding that trust boundary,” Jena said. “Your server now calls the model provider directly. That changes your network architecture, your auth model, and depending on how you’ve built cost attribution, your billing flow.”

The year-long transition period will be enough for teams to audit their sampling dependencies now, said Jena: “The risk is that teams who haven’t implemented sampling themselves won’t know if a third-party MCP server they’re depending on uses it.”

Updated MCP SDKs

To accompany the protocol update, there are updated MCP SDKs for Python, Typescript, Go, and C#. These support both the old and new protocol versions, so new clients can continue communicating with older servers, while updated servers will also support older clients, reducing the risk of immediate disruptions.

That backward compatibility should make the transition largely incremental, except for enterprises that built custom infrastructure around MCP’s earlier session-based architecture, Bandta said.

Identifying and auditing those session dependencies may not be easy, Jena warned.

“Session management complexity tends to be hidden across multiple layers — the gateway config, the deployment scripts, the monitoring dashboards. The code change is small; finding everywhere the assumption lives is what takes time,” he said.

This article first appeared on InfoWorld.

Leadership wants to scale AI. Budgets are tripling. Adoption is up.

Then the CFO asks the question every board now asks: which of these initiatives is actually profitable?

Most organizations cannot answer that question, not because they lack visibility into cost, but because the cost data they have was never designed to produce that answer.

Applying lessons learned from managing cloud spend won’t be a fix for the AI and ROI quandary. True, cloud taught a generation of CFOs that billing without business context is noise. So to get cloud ROI, they stitched two data sources together: cost data plus business data. AWS reveals which account, which region, which tag, which resource. Merge in customer and product mappings on top and the ROI of the cloud spend comes into focus.

But AI is harder. It requires three data sources: cost, business, and telemetry—the automatic collection of data from disparate sources that helps to clarify the whole picture of what happened and why. An executive or engineering lead can have AI invoices and customer revenue. But they have no way to connect them to business value. The token count on the OpenAI invoice does not specify which customer triggered which call, which feature it served, or whether the prompt produced a business outcome. That data does not exist in the provider’s billing.

AI providers won’t fix this problem

The situation is not likely to change anytime soon because AI providers are not in the business of attributing an enterprise’s costs to that enterprise’s customers. Instead, AI providers are in the business of selling tokens. The granularity they expose is the granularity their billing systems require, not the granularity a CFO requires.

Not convinced? Compare what AWS gives you to what an AI provider gives you.

AWS billing exposes resource IDs, account hierarchies, region, SKU, tag metadata, usage by the minute. Every dollar can be attributed to a workload, a team, a customer segment if it was tagged correctly. The data is rich enough that mature FinOps teams built unit economics on top of it years ago.

An AI provider invoice gives you tokens consumed by model, with optional grouping by API key. That is the resolution. No request-level attribution. No customer ID. No feature mapping. No prompt outcome. No retry identification. Multi-step agent workflows collapse into a token count. Imagine a large bank receives a multi-million dollar AI invoice each month. But it has no visibility into what parts of the business were responsible for what parts of the cost so cannot allocate them.

If an enterprise wants to know what AI cost drove which customer or feature, it has to capture that data itself, inside an application, before the call leaves it. 

Three required sources

Building AI ROI measurement requires three data sources, stitched together in a single model.

1. Cost data, normalized across providers. Every AI provider delivers cost differently. OpenAI invoices in one taxonomy, Anthropic in another, fine-tuning vendors and inference platforms each in their own. Cloud GPU costs sit in AWS or Azure billing. Vector database costs land in Pinecone or Snowflake invoices. None interoperate by default. Normalization is necessary but not sufficient. It will put all your AI costs in one schema. It does not tell you what they produced.
2. Application-layer telemetry. This is the source most organizations are missing, and the one that makes AI ROI structurally different from cloud ROI. It requires instrumenting AI calls inside your application across six categories: request-level tracing tied to a customer or session ID; feature attribution tied to the product surface that triggered the call; agent-step capture for multi-step workflows; retry and fallback identification so recovery costs don’t get attributed to primary calls; model selection logging that records which model was chosen and why; and outcome capture that ties each call to whether it produced business value. None of this data exists in the provider’s billing. All of it has to be captured at the moment the call is made and stored in a system that can be stitched to the cost data.
3. Business data. Revenue, customer segments, product hierarchies, and feature usage. The same business data already feeding your CRM and analytics stack, mapped to the customers and features the telemetry layer attributes calls to.

Stitched together, the three sources produce the unit economics every AI investment decision now requires: cost per customer interaction, margin per feature, profitability per agent workflow, ROI per model choice. None of these can be calculated from billing data alone. None can be calculated from telemetry alone. They require all three sources, modeled together in a way that maps cost to outcome.

Why agentic AI makes this urgent

Single-call inference is the easy case. One request, one cost, one customer, one outcome.

Agentic workflows are different. An agent decomposes a task into multiple steps. Each step calls a model. Some steps fall back to a different model when the first fails. Some steps retry on a poor result. Some steps invoke external tools that themselves cost money. A single user request can produce dozens of inference calls across multiple providers, with the cost compounding in ways the provider invoice cannot disaggregate.

If telemetry does not capture agent-step granularity, no one will know which steps are profitable. Aggregate costs will show up three weeks later in the invoice. By then, the workflow has been running at scale, customers are onboarded, and unprofitable paths have been retried thousands of times.

When agents make the calls, the volume of cost-generating events without business context attached grows by an order of magnitude. The window for instrumenting this before it becomes unmanageable is closing.

What changes when the three sources come together

Once the three sources are stitched together, the AI investment conversation changes.

Five different ways to build the same AI capability stop looking equivalent. They converge on adoption metrics and diverge by 10x on cost. The team picks the approach that delivers a similar business outcome at one-fifth the cost, because the team can finally see the difference. Product teams design features with margin awareness from the architecture phase, not from the post-launch budget review. Engineering teams choose model architectures with cost-per-outcome data alongside latency and quality. Leadership evaluates AI initiatives the way they evaluate any other capital allocation: on unit economics, not on the engagement chart. Aggregated invoices track the cost per customer interaction. Engagement metrics reveal margin per feature. Gut-instinct model selection is checked against real cost-per-outcome model selection results. 

Within seconds, everyone can see which AI features are profitable, which should scale, and which should be killed. This is the insight everyone is looking for and companies that achieve it will optimize the benefits of AI.

The build trap

AI costs are compounding now. The board is not waiting 18 months for an internal project to reach production.

The temptation to build it anyway has never been sharper. AI coding tools have changed what a small engineering team can ship in a quarter. The instrumentation layer looks tractable. The cost normalization looks like a weekend project. The semantic model feels like something a senior engineer could draft over a sprint.

It is a trap. Three reasons.

Volume is the first. A production AI footprint generates millions of telemetry events per hour, and that volume scales with agentic adoption. Real-time ingestion, correlation, and attribution at that scale is not the same problem as vibe coding a prototype in an afternoon. It is a permanent operational system that has to be right every minute of every day.

The vendor landscape is the second. Cost data arrives in delayed billing windows from providers with non-interoperable schemas. Schemas change without notice. New AI providers enter the landscape monthly, each with its own taxonomy and metering. The system is not built once. It is maintained against a moving target that moves faster than most internal release cycles.

The third is what the first two add up to: this is business-critical infrastructure. The CFO and the board are going to make capital allocation decisions on the data this system produces. When schema drift goes unnoticed for two weeks, when an agent telemetry stream stops correlating to a vendor that quietly changed its billing API, the cost of being wrong is not a sprint of cleanup. It is a quarter of misallocated capital.

The build-vs.-buy question for engineering leaders has changed. It’s not “can we build this?” The honest answer is yes. The real question is whether the marginal hour of your strongest engineers is best spent stitching cost data to telemetry to business outcomes, or building the AI products that produce the revenue the cost data is measuring.

The capability is reproducible in weeks. The choice is whether to spend the next 18 months building it, or the next 18 months acting on it.

—

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.

G# (GSharp) is moving forward as a programming language for Microsoft’s .NET platform, touted as bringing Go-, Kotlin-, and Swift-style ergonomics to the CLR (Common Language Runtime). The language is described by its creators as modern, simple, and accessible.

Although pre-1.0 and still growing, G# aims to be for people who want a small, predictable language with direct access to the .NET ecosystem. Developers will see imports, func, structs, slices, maps, channels, go, select, and for in iteration. Also important are nullable flow, direct calls into the CLR (Common Language Runtime), and built-in concurrency.

With G#, copyrighted in 2026, developers get value-oriented structs, reference-oriented classes, data structs, and data classes. For concurrency, G# uses scope for structured concurrency, async func and await for task-based asynchrony, and async sequence[T] for asynchronous streams. G# also makes use of the same Task and Task[T] types familiar from the .NET BCL (Base Class Library).

G# documentation is available on the GitHub site of Microsoft software engineer David Obando. “Every .NET type—your packages, third-party NuGet packages, the BCL—is callable from G# with the syntax you already know. CLR generics use G#’s bracket spelling, and method calls, properties, indexers, and for in over IEnumerable[T] all just work,” according to the website.

A Visual Studio Code extension for G3 can be found at marketplace.visualstudio.com. The extension adds syntax highlighting, language server features, build/run commands, and debugger configuration for .gs and .gsproj files. Developers can install the extension from within VS Code (search for “G#” in the Extensions view) or from the command line.

AI agents can turn a clear description into working software, the engineer’s judgement is what makes the difference: deciding what to build, catching the tradeoff the agent didn’t know to weigh, and owning the call on whether the result is right.

That judgement has always been the hard part of engineering. It just used to be bundled into the act of writing code, where a skilled engineer did it while typing. As agents take on more of the typing, that judgement separates out and becomes the clear center of the role. Leaders who adapt early will get their metrics, their talent pipelines, and their delivery models working with this shift rather than against it.

Judgement is defining, constraining, and deciding

Judgement is all about defining the problem precisely enough that an agent builds the right thing. It’s setting the constraints the agent won’t infer on its own. It’s spotting the tradeoff buried three layers down that only shows up if you understand the system. And it’s looking at a finished implementation and knowing whether it’s genuinely good enough to ship.

This is the harder part of the job and the real driver of quality. It was easy to underrate when it lived inside day-to-day coding. Now it’s what separates a strong team from an average one.

The shift changes where time, growth, and metrics go

If the high-value activity is intent, review, and judgement rather than raw output, a few assumptions are worth revisiting.

Where engineers spend their time. Less of the day goes to producing boilerplate and mechanical implementation, and more goes to the reasoning that used to get squeezed to the edges: framing the problem and owning the judgement calls that determine quality.

How teams grow their people. Defining problems well, spotting risk, and critically evaluating work you didn’t write yourself have always been senior skills. When agents handle more of the mechanical work, those skills become learnable earlier. That puts the emphasis on leaders to teach the reasoning: why a choice gets made and how to weigh the tradeoffs that come with it.

What you measure. Lines shipped, tickets closed, and velocity charts all measured throughput of the old scarce resource. They say very little about the new one. The teams that adapt will start measuring the quality of intent going in and the reliability of judgement coming out, because that’s where the results now live.

Reinvest the time you get back

The tempting response is to treat the freed-up capacity as pure speed: same work, same tooling, just faster. That captures the easy win and misses the real one. If engineers spend their reclaimed time reviewing a rising volume of agent output with no better context than before, review quietly becomes the new constraint, and you’ve moved the problem rather than solved it.

The organizations that get ahead will invest the reclaimed capacity into the judgement layer: creating stronger specs and acceptance criteria before work starts, building review practices that test agent output against intent, and capturing the reasoning behind decisions where the next person can find it, so it doesn’t have to be reconstructed every time. That’s how the shift becomes an advantage for your team.

The through-line for leaders

The engineering job is moving up a level, from executing the work to directing and validating it. That’s a more strategic role, and it rewards clarity of thought over speed of output. Leaders who see the shift early can help their engineers grow into the work that’s now most valuable.

See how leading engineering organizations are operationalizing this shift at jira.dev.

GitLab has released GitLab 19.2, an update to the company’s devsecops platform that allows teams to fix vulnerable dependencies automatically, use Security Review Flow to catch logic flaws that scanners miss, and run AI agents straight from the terminal, the company said.

Highlights in GitLab 19.2 include the following:

• Dependency Scanning Auto-Remediation, in public beta, uses AI to fix build-breaking changes and iterates until your pipeline passes, with every change governed by your existing gates and audit trail.
• Security Review Flow, also in public beta, analyzes code changes as a security engineer would and catches authorization gaps, business-logic errors, and race conditions that static scanners structurally cannot see.
• GitLab Duo CLI, now generally available, gives developers access to agents and multi-step agentic flows for all software life cycle tasks without leaving the terminal. 
• Custom Flows, now generally available, let teams replace manual multi-step workflows with agentic automations for software development, triggered by GitLab events.

“Coding agents made it possible to generate far more code and moved the bottleneck downstream to reviews and security,” said Manav Khurana, chief product and marketing officer at GitLab, in a statement. “GitLab 19.2 puts agents to work on that bottleneck: fixing vulnerable dependencies, catching the flaws scanners miss, and automating the steps in between with a person still approving what ships.”

Outsourcing worked – until it didn’t.

After Akirolabs achieved early market validation and onboarded its first enterprise customers, outsourcing began to create strategic limitations around scalability, intellectual property (IP) ownership, security and delivery execution.

The challenges started after the first enterprise customers confirmed product-market fit. At that point, delivery speed became directly tied to business growth. Product quality expectations increased. Infrastructure and security requirements became stricter. Investors started asking difficult but fair questions about IP ownership, operational dependencies and long-term scalability.

Most importantly, engineering execution was no longer just an operational function – it became part of the company’s strategic advantage. That was the moment when the founders decided the company needed dedicated technology leadership to address these challenges. This is how I joined the company at the beginning of 2023. As VP of Engineering and a bit later as CTO, I led the transformation (usually known as insourcing, repatriating or backsourcing) from an outsourced model to an internal engineering organization while maintaining product delivery continuity and preparing the company for the next growth stage. The process took roughly a year and involved not only technical migration, but also organizational design, hiring, process development, infrastructure modernization and cultural transformation – everything from the ground up.

Building an internal engineering organization while still delivering

One of the biggest misconceptions about insourcing is that it is primarily a technical project. It is a leadership and execution challenge.

When I joined the company, there was effectively no internal engineering structure, limited visibility into the existing system and no clear long-term technical strategy. My first months were dedicated to understanding reality and I began with a comprehensive assessment of the codebase, operational risks, documentation quality and knowledge dependencies to determine the most viable transition strategy.

Very early in the process, I faced a critical strategic decision: whether to gradually assume ownership of the existing platform or rebuild it internally. To make that decision, I evaluated four distinct transition models ranging from limited management insourcing to a complete internal rebuild.

After assessing the technical, operational and long-term business implications of each approach, I selected the most demanding option: rebuilding the product internally while maintaining uninterrupted delivery for existing customers. Although riskier in the short term, a full rebuild offered the clearest route to complete IP ownership, architectural flexibility and long-term scalability.

At the time, this decision ran counter to the approach typically taken by startups in similar situations. Most organizations gradually assume ownership of an existing codebase to minimize short-term risk and preserve delivery capacity. My assessment was that the accumulated architectural debt, fragmented knowledge distribution and long-term maintenance risks would ultimately make a phased takeover more expensive and less scalable than a controlled rebuild. The strategy required significantly higher execution discipline, but it allowed us to establish complete ownership of the platform, eliminate inherited constraints and create an architecture capable of supporting enterprise-scale growth.

The next challenge was hiring.

In Germany, hiring can easily take four to six months – mostly due to a typical 3-month notice period, which is incompatible with startup timelines. We solved this by building a hybrid organization structure early: a lean internal core team combined with carefully selected contractors. Instead of hiring only narrow specialists, we prioritized experienced generalists capable of operating across architecture, infrastructure, security and compliance discussions. Later, we evolved toward a product engineering model, where engineers owned broader product outcomes rather than narrowly defined technical functions.

During the first three months, we established a core engineering team of four senior engineers. Over the following nine months, the organization expanded to roughly fifteen engineers while I strategically designed and executed the transformation of the platform’s architecture to meet the rigorous deployment and compliance standards of our first enterprise clients, including Raiffeisen Bank International and Bertelsmann. This structural overhaul allowed the company to meet the deployment, security and compliance requirements of enterprise customers that had previously been inaccessible under the outsourced model. At that point, we had already achieved complete coverage across backend, frontend, DevOps, QA and security.

I also intentionally kept processes lightweight during the transition. Instead of introducing heavyweight frameworks, we focused on clarity of priorities, fast decision-making and execution discipline. We used Kanban over Scrum, eliminated unnecessary meetings, shortened the remaining ones and emphasized engineering culture over process overhead.

Another major challenge was project estimation. Because dual-track development was unavoidable until the in-house platform reached production readiness, estimation accuracy had a direct impact on budget efficiency. Despite all challenges, my initial estimate ultimately proved remarkably close to the final delivery date, differing by only about a week. Accurate forecasting under conditions of parallel development streams, ongoing customer commitments and active team formation became a critical leadership challenge. Maintaining this level of predictability throughout the transition helped align engineering execution with business planning, hiring decisions and investor expectations.

The engineering transformation enabled capabilities that contributed to Akirolabs being recognized as an IDC Innovator in Procurement in 2023, named amongst the Top 27 AI Startups in Germany in 2024, Sifted’s 100 Fastest-Growing Startups in DACH & CEE 2025 and inclusion in 2024-2026 in ProcureTech100 annual recognition of procurement technology providers shaping the future of digital procurement.

Managing risk without slowing down the business

The hardest part of insourcing is not writing code, selecting the technology stack, designing architecture or configuring infrastructure. It is avoiding disruption while the company is changing underneath the product. I successfully orchestrated the concurrent overhaul of product architecture, cross-functional engineering recruitment, infrastructure modernization and live customer operations under exceptionally tight margins.

To reduce delivery risk, we approached the transition in layers.

First, we focused on infrastructure reliability and operational readiness before feature expansion. Cloud architecture, recovery testing, permission segregation and incident management processes were implemented early, not after launch. We also introduced multiple testing stages and dedicated QA functions after learning the hard way that a “developers-only” quality control approach does not scale for complex web platforms and business domains.

Second, we established a structured knowledge-transfer process to rapidly onboard engineers and reduce external dependencies.

Third, we became extremely disciplined about scope management. One of the most common reasons insourcing initiatives fail is uncontrolled change during the rebuild phase. Every new feature request increases uncertainty non-linearly. We learned to separate strategic improvements from distractions and protect the core delivery roadmap aggressively. Throughout the transition, we successfully maintained uninterrupted customer operations by utilizing planned maintenance windows, achieved a near-zero-downtime migration and permanently doubled product velocity immediately following the migration.

Beyond the technical migration itself, the transition established a repeatable operating model for scaling technology organizations beyond the product-market-fit stage. The framework combined organizational redesign, controlled knowledge repatriation, architecture modernization and enterprise-grade operational practices while maintaining uninterrupted customer delivery throughout the transformation. While the implementation was specific to Akirolabs, the underlying principles are broadly applicable to organizations seeking to transition from outsourced development to internal product ownership without disrupting business operations.

By the time the new platform reached production readiness, I had established not only a functioning engineering organization, but also a stable operational model: internal ownership, production-grade infrastructure, security processes, scalable hiring practices and clear technology and product roadmaps.

A positive side effect of the transition was the creation of internal UI/UX and Data Science capabilities, which later became strategically important for AI product initiatives and created a foundation for the third version of the product, which we released in mid-2025.

My technical restructuring and migration to a secure proprietary platform reduced architectural risk, established full in-house ownership and helped strengthen investor confidence during the company’s successful €5M fundraising round in 2024.

The transition created a stronger foundation for scale and supported the company’s continued expansion among enterprise organizations operating at Fortune 500 scale, including Ahold Delhaize, Workday, IFF, Deutsche Bahn and others.

Lessons learned for CTOs considering insourcing

Looking back, several decisions made the transition successful, and several mistakes made it harder than necessary.

The first lesson is simple: decisiveness in strategic transition is paramount to maintaining business momentum. Rapidly evaluating insourcing frameworks and defining clear boundaries with the external partner allowed us to mitigate operational downtime and execute a highly efficient migration ahead of critical market deadlines.

Second, hire more senior people and do it as early as possible. Strong technical leaders multiply execution capacity far beyond their individual contribution. In our case, the quality of the first hires influenced architecture quality, hiring standards, delivery discipline and engineering culture for the entire organization.

Finally, culture matters more than frameworks. Processes can be added later. Ownership mentality cannot.

The biggest long-term advantage of bringing development in-house was not simply faster execution, not better code quality or operational cost optimization by over 30% after the transition which we also achieved. It was an alignment. Product strategy, engineering decisions, customer priorities and business goals became part of the same conversation instead of being separated by organizational boundaries. For technology companies operating in highly competitive markets, that alignment becomes a compounding advantage over time.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Microsoft and Mistral are betting that the future of enterprise AI is in sovereign infrastructure and model choice, rather than with one locked-in system. 

The companies have announced a “significant expansion” of their strategic partnership, which includes a multibillion dollar commitment from Microsoft. Mistral will add to its GPU infrastructure in Europe and extend access to its frontier multilingual models, while Microsoft will expand its sovereign cloud capabilities. The companies will also align on a joint go-to-market plan and will pursue enterprise opportunities together across Europe and globally, as well as funding proofs of concept (PoCs), offering Azure credits, and leading workshops to drive AI innovation with customers.

The partnership between the tech giant and the three-year-old French startup might seem an odd combination at first glance, analysts note, as both develop enterprise AI models and offer access as-a-service. But it reflects changing AI market dynamics.

“It’s possible to be both a competitor and a partner at the same time,” noted technology analyst Carmi Levy. Large cloud providers are becoming AI marketplaces in their own right, he pointed out, and are drifting away from exclusively promoting their own models. Building Mistral support into their infrastructure avoids platform lock-in and removes a “key objection for customers looking for options.”

“As much as Microsoft would want everybody standardizing on Copilot and Phi, it recognizes the simple fact that customers increasingly want to choose their own models,” said Levy.

Expands model access, sovereign cloud capabilities

As part of the agreement, Mistral will expand its Europe-based capacity with thousands of Nvidia Vera Rubin GPUs.

Mistral CEO and co-founder Arthur Mensch described a “slight gap” in compute capacity in Europe, noting that this expansion will provide more compute capability and support Microsoft’s cloud and AI services, providing a “shared platform for training, inference and large-scale deployment.” The companies call it a critical step to allow Microsoft customers to benefit from Mistral’s “scientific and compute innovations.”

In addition, Mistral Medium 3.5 and OCR 4 models are now available in Microsoft Foundry, and Mistral Medium 3.5 can be used in Microsoft Copilot Studio.

The partnership also extends Microsoft’s sovereign cloud infrastructure as well as combining Mistral’s frontier models with Microsoft’s security, compliance, and cloud-to-edge platform. This gives enterprises, particularly those in regulated markets, the ability to deploy AI where they see fit, while maintaining control over their data and workloads, according to the companies.

Further, customers will be able to build AI using the same models, tools, APIs, and workflows they’re used to, across Microsoft Foundry, Foundry Local, and Azure Local, and opt for fully Azure-hosted cloud environments; cloud-connected, controlled Azure Local environments that only use cloud-based Azure when necessary; and fully-disconnected environments that can operate independently for more sensitive scenarios.

“Europe should have access to the world’s most capable AI without compromising control over their data, operations or digital future,” said Brad Smith, vice chair and president, Microsoft, noting that with this partnership, the company is honoring its European digital commitments and giving customers a foundation for AI so they can “operate on their own terms.” Customers with “heightened sovereignty needs” will be able to exercise more control with “resilience and assurance” and continued access to Mistral’s open-weight models.

Enterprise credibility

Gartner distinguished VP analyst Arun Chandrasekaran noted that there’s no doubt that this agreement strengthens Microsoft’s sovereignty messaging and its position in regulated industries, and the tech giant benefits by expanding its AI portfolio with a “credible European frontier model provider”

He pointed to key differences from the initial partnership struck by the two companies in 2024; whereas originally Microsoft was hosting Mistral’s models, it is now consuming capacity built by Mistral in Europe.

Ultimately, the deal emphasizes European data centers, customer-controlled deployments, Azure Local, and fully-disconnected environments, addressing many of the concerns that surrounded the original Azure cloud only relationship, Chandrasekaran explained.

For Mistral, the partnership provides “enterprise credibility, and repeatable infrastructure revenue” that can fund continued AI platform development, he said. The combination of Microsoft’s enterprise AI platform with Mistral’s models and European AI infrastructure will give joint customers more deployment flexibility and expand options around data residency, sovereign AI deployments, and disconnected/on-premises environments.

“It also gives customers more model choice, reducing dependence on a single AI provider,” said Chandrasekaran.

A complementary partnership

Mistral continues to innovate with its frontier AI models and its chat and coding agent, Vibe (formerly Le Chat), yet it doesn’t attract as much attention as Claude or ChatGPT.

One of the company’s key differentiators is its targeted business model. Levy pointed out that not every workload requires “full-flight GPT.” For customers trying to rein in costs and limit exposure with on-premises deployments, Mistral’s “more focused capabilities can represent a cost-effective alternative.”

Chandrasekaran pointed to Mistral’s combination of high-performance open-weight models, strong multilingual capabilities, and a “focus on efficient inference that lowers deployment costs.”

Unlike many frontier AI companies, it offers customers greater flexibility to self-host and customize models; this makes it particularly attractive for enterprises and governments with sovereignty or regulatory requirements, he said. Its European roots also position it as the leading alternative for organizations seeking cutting-edge AI outside the US and Chinese ecosystems.

Bill Wong, research fellow at Info-Tech Research Group, also pointed to Mistral’s high-quality models and “adeptness as a sovereign AI leader.” There is growing demand for AI companies that comply with regional laws and data residency, and Mistral is established as “one of the most prominent European players.”

“Such a strategic position makes it a great partner for Microsoft to further expand its AI offerings beyond just being a single-model provider,” he said. Customers get freedom of choice while complying with data sovereignty and regulatory limitations without having to execute a separate AI deployment, while Mistral, for its part, can go beyond Europe and gain more visibility with international businesses.

Mistral brings both “technological and political advantages,” Levy noted. The startup’s European roots give Microsoft more credibility “at a fraught time for geopolitical relationships.” Customers in Europe and beyond are concerned about US exposure, and Mistral can provide a safer choice.

Meanwhile, Microsoft can deploy European-developed AI models running on European infrastructure, thus maximizing regulatory compliance while offering next-level enterprise marketing scale that Mistral “simply couldn’t achieve on its own,” said Levy. Mistral-based workloads deployed on Azure will also benefit from Microsoft’s “comprehensive security certifications, governance frameworks, and monitoring.”

Bottom line: Both companies can maximize their unique roadmaps through the partnership, he said. “As the rules of the AI economy continue to evolve, expect more eyebrow-raising deals like this to be signed.”

AWS has updated AWS Data Exports, its service for generating and managing cost and usage Reports (CURs), to include standardized Amazon Bedrock product metadata, making it easier for enterprise engineering teams to analyze AI usage and spending as they scale AI deployments spanning multiple foundation models.

The update extends billing exports with normalized fields for model provider, model name, inference type, inference mode, billing unit and Bedrock product family, and will enable enterprises to identify which models generated costs and compare spending across providers without relying on custom parsing or normalization of billing records, AWS wrote in a blog post.

That reduced reliance on custom parsing will reduce the engineering effort required to analyze billing data, analysts said.

“Before the update, a data engineer would typically need to maintain a model ID registry, write regex against usage type strings, or join AWS CloudTrail with CUR to figure out which provider generated which cost,” said Bhupendra Chopra, chief revenue officer at IT consulting firm Kanerika.

The new standardized fields “can be the difference between a billing pipeline that needs constant babysitting and one that doesn’t,” Chopra added.

That’s because custom parsing logic is more prone to break down or require maintenance when AWS adds new models or updates pricing in Bedrock, said Pareekh Jain, principal analyst at Pareekh Consulting.

Richer billing data to boost enterprise AI cost governance

Beyond reducing engineering overhead, the update could also help enterprises improve AI cost governance.

Before this update FinOps teams struggled to identify which model Bedrock related to because usage type fields were inconsistent, and there was no unified product family name that captured all Bedrock costs in one place, Chopra said.

“Now those attributes — model provider, model name, inference type, inference mode, pricing unit — are standardized and available by default. That’s the plumbing work no one talks about, but it’s what makes downstream reporting actually reliable,” Chopra added.

This, said Jain, makes it easier to build dashboards showing cost by model, provider, token type or inference mode while also identifying expensive workloads, unusual token growth and opportunities to move to cheaper models or batch processing.

It’s a timely update, especially in light of last week’s AWS billing issue that caused some customers to see incorrect cost estimates of services consumed in the AWS Management Console, said Muskan Bandta, cloud associate at FinOps services providing firm ZopDev.

“Anything that gives customers clearer, more granular and more trustworthy billing data is welcome when confidence in the numbers has just been shaken. It does not fix what went wrong, but better visibility into where spend is going is exactly what teams want more of after an episode like that,” Bandta added.

This article first appeared on InfoWorld.

AWS has updated AWS Data Exports, its service for generating and managing cost and usage Reports (CURs), to include standardized Amazon Bedrock product metadata, making it easier for enterprise engineering teams to analyze AI usage and spending as they scale AI deployments spanning multiple foundation models.

The update extends billing exports with normalized fields for model provider, model name, inference type, inference mode, billing unit and Bedrock product family, and will enable enterprises to identify which models generated costs and compare spending across providers without relying on custom parsing or normalization of billing records, AWS wrote in a blog post.

That reduced reliance on custom parsing will reduce the engineering effort required to analyze billing data, analysts said.

“Before the update, a data engineer would typically need to maintain a model ID registry, write regex against usage type strings, or join AWS CloudTrail with CUR to figure out which provider generated which cost,” said Bhupendra Chopra, chief revenue officer at IT consulting firm Kanerika.

The new standardized fields “can be the difference between a billing pipeline that needs constant babysitting and one that doesn’t,” Chopra added.

That’s because custom parsing logic is more prone to break down or require maintenance when AWS adds new models or updates pricing in Bedrock, said Pareekh Jain, principal analyst at Pareekh Consulting.

Richer billing data to boost enterprise AI cost governance

Beyond reducing engineering overhead, the update could also help enterprises improve AI cost governance.

Before this update FinOps teams struggled to identify which model Bedrock related to because usage type fields were inconsistent, and there was no unified product family name that captured all Bedrock costs in one place, Chopra said.

“Now those attributes — model provider, model name, inference type, inference mode, pricing unit — are standardized and available by default. That’s the plumbing work no one talks about, but it’s what makes downstream reporting actually reliable,” Chopra added.

This, said Jain, makes it easier to build dashboards showing cost by model, provider, token type or inference mode while also identifying expensive workloads, unusual token growth and opportunities to move to cheaper models or batch processing.

It’s a timely update, especially in light of last week’s AWS billing issue that caused some customers to see incorrect cost estimates of services consumed in the AWS Management Console, said Muskan Bandta, cloud associate at FinOps services providing firm ZopDev.

“Anything that gives customers clearer, more granular and more trustworthy billing data is welcome when confidence in the numbers has just been shaken. It does not fix what went wrong, but better visibility into where spend is going is exactly what teams want more of after an episode like that,” Bandta added.

Site reliability engineers (SREs) have the tough assignment of resolving thorny performance and reliability issues. But their primary mission is to provide devops teams with operational insights and to suggest implementation improvements on business system performance, security, and overall robustness.

Google introduced its SRE playbook in 2003, but it took some time for the role’s definition, tools, and techniques to become mainstream. Startups were the first to adopt observability for cloud-native applications and create dedicated SRE positions. As tools matured and SRE responsibilities became more clearly defined, larger enterprises assigned SREs to work as a bridge between devops and IT ops teams to improve resilience across a wider range of applications, APIs, and data pipelines.

SRE is a career path for multidisciplinary engineers with strong investigative instincts, sharp data analytics skills, and the temperament to perform under pressure. It has become a critical responsibility as tech became mission-critical for enterprises, and it is a growing role in the genAI era as more businesses deploy AI agents.

But the critical need for resiliency and greater technological complexity brings new challenges for SREs. According to the 2026 State of Production Reliability and AI Adoption report, 44% of respondents experienced an outage linked to ignored or suppressed alerts in the past year, and 35% report their engineers occasionally ignore or dismiss alerts due to alert fatigue. More than 70% of alerts received are not actionable, according to 57% of organizations.

So, is AI making the SRE’s role easier and helping businesses run more reliable technology operations? On the other hand, AI is also driving complexity, as companies deploy genAI tools and AI agents across more business functions and seek to automate more decision-making across operations.

AIops and agentic ops aid SREs

Over the past decade, SRE responsibilities have become somewhat easier through improvements in monitoring platforms, observability practices, tools for centralizing operational data, and AI applied in IT operations (AIops). But during the heat of resolving an outage or performance issue, it’s not easy to correctly identify what system triggered the issue versus other downstream systems impacted by it.

According to the Komodore 2025 Enterprise Kubernetes Report, 79% of production incidents originate from recent system changes, including deployments and changes to compute environments. But the other 21% of incidents stem from issues outside of the business’s control, including network failures, third-party changes, and cloud provider failures.

“SREs using AI capabilities succeed or fail in the moment an incident unfolds, when engineers are deciding what to investigate next,” says Itiel Shwartz, CTO at Komodor. “If the system streamlines root cause detection, connects signals to recent changes, and explains its reasoning in a way engineers recognize, it earns trust. If it adds uncertainty or demands extra validation, it gets sidelined, regardless of how bespoke the model behind it may be. What’s less obvious is what it takes to make AI for SREs work in production, and how different that reality is from prototypes, demos, or early internal builds.”

AIops is not a new capability, especially in using machine learning to correlate logs, metrics, and traces across monitoring and alerting systems. IT service management and SREs have been using AIops to reduce the mean time to resolve incidents and to perform accurate root cause analysis (RCA) efficiently. Agentic ops is the next wave of genAI operational capabilities, including tools for monitoring AI agents, managing their access rights, and detecting AI model accuracy drift.

 “AI is useful during major incidents because it can pull together a lot of context into a few clear sentences, which is exactly what an SRE needs in the moment,” suggests Shani Shoham, chief revenue officer at OpenObserve. “The complexity of architecture and the different tooling make it easier for AI than for a human, but autonomous resolution is still a way off.”

AI’s impact on people and burnout

The business pressure to keep systems up, secure, and performing well is a 24/7 stressful responsibility. According to The SRE Report 2025 from Catchpoint, 36% of SREs often or always experience elevated stress during an incident, and 28% said the stress persists even after the incident is resolved. AI capabilities may prove to be a game-changer in helping SREs avoid burnout and reduce stress.

“AI can improve RCA by taking in a much larger incident context than any engineer can hold at 3am, reasoning across traces, logs, metrics, deploys, config changes, alerts, ownership, and recent production behavior,” says Noam Levy, founding engineer and field CTO at Groundcover. “Beyond attempting a full RCA, its immediate value is distilling the signals that actually matter, reconstructing a clear timeline of cause and effect, and helping engineers separate correlation from likely causality. Once a fix is deployed, agents can also verify remediation by comparing pre- and post-fix behavior, but this depends on broad access to rich, correlated production signals and a cost model that does not discourage adoption or experimentation.”

Not only are incidents resolved faster and with less stress, but AI can also free up SRE time to focus on proactive work and create a career path for junior developers into SRE roles. Quais Taraki, CTO at EDB Postgres AI, adds, “AI reduces toil by automating repetitive tasks while accelerating incident resolution through copilots that correlate signals across distributed systems, allowing SREs to focus more on resilience strategies like chaos engineering and failure analysis.”

AI can have long-lasting operational impacts, especially for organizations looking to deploy more mission-critical technology and AI capabilities. Two longer-term benefits of AI for SREs are reducing the number of bridge calls needed for incident response and the number of engineers required in “war rooms” to coordinate root cause analyses.

“When something goes wrong, AI that guides SREs can do the full analysis, get to the root cause, and perform the remediation,” says Spiros Xanthos, founder and CEO of Resolve AI. “AI also helps avoid many escalations, and when escalations are needed, it targets the right people from the network, infrastructure, and the application teams. AI for SREs centralizes operational intelligence, exposes tribal knowledge, and can guide more junior developers.” 

AI agent reliability

While AI capabilities have been a net positive in helping SREs improve system reliability, the growth of AI code generators, vibe coding, and spec-driven development is adding to their workloads. According to one study, 41% of all global code is now AI-generated, and Gartner predicts that 40% of new enterprise production software will be created using vibe coding techniques by 2028.

But coding velocity is creating new issues for SREs as AI pull requests have 1.4 times more critical issues and 1.7 times more major issues, according to CodeRabbit. “AI-assisted development has created an unprecedented velocity of code reaching production, expanding surface area, edge cases, and failure rates faster than traditional SRE practices can absorb,” says Vinod Jayaraman, cofounder and CTO at NeuBird AI. “The speed of shipping has far outpaced the speed of understanding what breaks in production. To close this loop, SREs need enterprise agents that can capture precise diagnostic context, including correlated traces, service dependencies, and anomaly timelines, and structure it as actionable input for the engineers and AI coding tools responsible for the fix.”

The growing number of AI agents deployed to production creates new challenges. AI agents are not just code; they have multiple failure points. They are built using language models, connect to proprietary sources for context, and integrate with Model Context Protocol servers to support more complex workflows. Changes are ongoing and not deployment events, so the SRE’s job of identifying the source of performance and accuracy drifts isn’t trivial. 

“Traditional SRE was built for systems that fail in reproducible ways, but agents fail differently and drift when a model provider pushes an update, and behavior shifts silently with no baseline for comparison,” says Mohammed Aboul-Magd, vice president of product at SandboxAQ. “Most organizations can’t even answer the basics: how many agents are running, what they have access to, and whether they’re still doing what they were built to do.”

“Every time a senior engineer leaves, they take years of learned failure patterns with them, and the next outage starts from square one,” adds Ronak Desai, cofounder and CEO at Ciroos. “Using AI for compounding operational memory changes that, and every incident your system resolves, the AI learns it.”

SREs should take a leadership role in emerging best practices, including defining their standards for AI agent non-functional acceptance criteria, observability practices, and release-readiness criteria. SREs should update their service-level objectives (SLOs) and define error budgets for AI agents in production.

Ryan Downing, vice president and CIO of enterprise business solutions at Principal Financial Group, says, “Standard SLOs and error budgets give teams the guardrails, and AI helps interpret the telemetry against those targets, reducing noise so engineers can get to the real issue faster and automate parts of remediation before customers are impacted.”

AI raises the SRE’s business impact

The more dramatic shift in site reliability engineering is an evolution of its business scope. IT leaders focus on uptime, performance, and issue resolution, as well as understanding their impacts. Business leaders will look to IT and SREs to identify, determine root cause, and remediate a broader class of issues, including rogue AI agents and the impacts of rapidly deploying new agentic capabilities. 

“AI agents are handing SREs categories of problems they’ve never had to solve before, specifically failures defined in business terms, not technical ones,” says Blake Sherwood, distinguished technologist for AI and platform strategy at Smarsh. “Traditional reliability engineering is built around latency, errors, and crashes, but agents now fail due to skipped compliance steps or outcomes that looked fine technically but were wrong contextually. Most SRE teams aren’t wired for that yet.”

The question is whether SREs with AI-augmented tools can keep up with the velocity, complexity, and business urgency of deploying new AI business capabilities.

Every enterprise AI conversation seems to begin with the same question: Which model should we use?

I understand why. Models are visible. They have names, benchmarks, release notes, pricing pages and impressive demos. They are easy to compare in a leadership meeting. One model promises better reasoning. Another offers a larger context window. Another appears faster, cheaper or more specialized.

But after years of working around enterprise platforms, integration layers, cloud migration, middleware, production operations and mission-critical systems, I see the AI conversation differently.

The model matters. But it is not where most enterprises will struggle next.

The next AI bottleneck is the infrastructure behind the model.

I do not mean only GPUs, cloud capacity or data storage. I mean the full enterprise operating layer that allows AI to work safely in the real world: data pipelines, identity, APIs, messaging, observability, security controls, deployment automation, cost governance, auditability, support ownership and recovery design.

That layer is what determines whether AI remains an exciting experiment or becomes a trusted business capability.

Pilots hide the hard part

Most organizations can build an impressive AI pilot. A small team can connect a model to a dataset, create a workflow and show a use case that works well in a controlled setting.

The harder part starts when that pilot moves into a real production process.

That is when practical questions show up. Who owns the data quality? What systems can the AI access? How do we trace which prompt, policy or retrieval flow produced a specific answer? What happens when an API slows down, a queue backs up or a downstream system is unavailable?

To me, these are not model problems. They are infrastructure problems.

This is where many enterprises are now headed. The first phase of AI was experimentation. The next phase is operationalization, and that is where the real gap becomes clear.

McKinsey has made a similar point in its work on agentic AI, noting that the next phase of value depends less on isolated tools and more on redesigning workflows, operating models and enterprise execution around agents.

AI pilots can survive on enthusiasm. Production AI requires architecture.

AI is becoming an integration problem

The more I look at enterprise AI, the more it feels like an integration challenge.

In large organizations, I have seen how messaging platforms, integration gateways, deployment pipelines, monitoring tools and cloud infrastructure can decide whether a digital capability succeeds or fails. AI will be no different. Even the strongest model will struggle if the data, middleware, identity layer and operational controls around it are weak.

AI does not work in isolation. It needs context from systems of record, clean data from different business areas, secure access to APIs, event streams, workflows, knowledge repositories, monitoring tools and legacy systems.

That is why the CIO question is changing.

It is no longer just, “Which AI tool should we buy?”

It is becoming, “Can we safely operationalize intelligence across the business?”

This is where agentic AI matters. Autonomous AI only creates real value when the architecture around it can make its actions safe, traceable and useful.

A model can generate an answer. Infrastructure determines whether that answer is secure, timely, explainable, governed and connected to the right workflow.

For example, an AI assistant that summarizes customer or order information may look like a model use case. But underneath, it depends on access control, fresh data, reliable APIs, logging, encryption, monitoring and policy enforcement.

If the answer is wrong, people may blame the model. But the real failure may have started with stale data, weak integration, poor access design, missing observability or an unreliable downstream system.

That is why CIOs should not judge AI only by model capability. The enterprise system around the model matters just as much.

Latency will become a trust issue

In traditional technology operations, latency is often treated as a performance metric. In AI-enabled workflows, latency becomes a trust issue.

When an employee asks an AI assistant for help and the response takes too long, the employee stops using it. When a customer-facing workflow becomes slow, the customer abandons it. When an AI agent waits on multiple backend calls, the entire business process feels unreliable.

This becomes even more important as organizations move from simple chat interfaces to agentic workflows. A single AI-driven action may include identity checks, context retrieval, policy validation, model reasoning, API calls, business-rule execution, logging and human approval.

Each step adds latency. Each dependency adds a possible failure point.

A model may be fast in a benchmark but slow inside an enterprise process. That difference matters.

This is where platform engineering becomes essential. Enterprises need reusable patterns for AI workloads: approved connectors, secure retrieval methods, queue-based decoupling, caching strategies, deployment pipelines, monitoring dashboards and standard rollback procedures.

Without those patterns, every AI initiative becomes a custom build. Custom builds may work for pilots, but they do not scale across a large enterprise.

Observability has to expand

Traditional monitoring tells us whether infrastructure is healthy. Is the server up? Is CPU high? Is memory exhausted? Is the application returning errors?

AI needs that, but it also needs more.

We need to know what data was retrieved, which model was used, which prompt version was active, which user initiated the request, which policy was applied, how long each step took and whether the output passed validation.

We also need to detect new forms of risk: unusual usage patterns, repeated failed tool calls, unexpected cost spikes, sensitive data exposure, weak retrieval results or an AI workflow attempting actions outside its intended boundary.

In production AI, observability is not only about uptime. It is about confidence.

If a business leader, auditor, regulator or security team asks why an AI system made a recommendation, the answer cannot be, “The model said so.” The enterprise needs traceability. It needs evidence. It needs operational context that engineers, risk teams and business owners can understand.

This is one of the biggest gaps I see in AI strategy. Many organizations are investing in models and use cases, but not enough in the control plane required to manage them.

Data readiness is still underestimated

AI has exposed an uncomfortable truth: many enterprises are not as data ready as they think.

Data is often duplicated across platforms, described differently by each team, governed inconsistently and refreshed on different schedules. Access rules may be clear in one system but unclear in another. Even basic business definitions can change from department to department.

AI does not fix that automatically. In many cases, it makes the problem more visible.

A bad report may be questioned. A bad AI answer may sound confident enough to be trusted.

That is a real risk.

Being data-ready for AI is not just about connecting a vector database or indexing documents. It requires clear ownership, lineage, classification, quality checks, retention rules, access boundaries and a shared understanding of which data should be used for which purpose.

The same principle applies to resilient cloud-native design. In my IEEE TechRxiv paper, “Enabling Fault-Tolerant Multicast in Cloud-Native Architectures” I explored how reliability, observability and fault tolerance become foundational requirements when critical workloads stretch across hybrid and multi-cloud environments.

CIOs already understand this because they have lived through enterprise resource planning programs, cloud migration, integration modernization, cybersecurity transformation and analytics initiatives. The lesson is familiar: technology cannot outrun data discipline forever.

Security cannot be added later

As AI moves from answering questions to acting, security becomes much more important.

An assistant that summarizes information carries one level of risk. An agent that can open a ticket, update a record, trigger a workflow, approve a request or contact a customer carries a very different one.

The more AI can do, the more identity, authorization, least privilege, separation of duties and human approval matter.

Enterprises should be careful not to grant AI broad access just to speed up a pilot. That may seem harmless in development, but it can become dangerous at scale.

AI access should be treated like any other privileged enterprise capability: limited, logged, reviewed and easy to revoke.

The NIST AI Risk Management Framework is a useful reference point here because it frames AI risk as something organizations must govern, map, measure and manage continuously rather than something handled only at the end of deployment.

Security teams should be involved early, not at the end. The goal is not to slow innovation. The goal is to build a platform where safe innovation becomes repeatable.

The CIO has to define the operating model

AI is creating pressure from every direction. Boards want productivity. Business teams want automation. Employees want better tools. Vendors are pushing new features. Security teams are watching risk. Finance teams are watching cost. Customers expect faster, smarter experiences.

The CIO sits in the middle of all of it.

That is why the CIO’s role cannot stop at choosing tools or approving pilots. The CIO has to define how AI will actually operate across the enterprise.

That means answering practical questions. Which architecture is approved? Which data sources can be trusted? How are AI workflows deployed, monitored, supported and governed? How are costs controlled? How do teams reuse common patterns instead of rebuilding the same foundation each time?

This work may not be as exciting as a model demo, but it is what separates sustainable AI from short-term experimentation.

The winning organizations will not be the ones with the most pilots. They will be the ones with the strongest AI operating layer.

They will build reusable platform patterns, strengthen data governance, design access properly, monitor AI behavior end to end and measure success by business improvement, not only model performance.

The model still matters. But the enterprise behind the model matters more.

A powerful model on weak infrastructure will eventually disappoint the business. A capable model on strong infrastructure can deliver real value because it can be trusted, secured, scaled and improved.

That is the shift CIOs need to lead.

The next AI bottleneck is not the model. It is whether the enterprise behind the model is ready.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

When a goal goes in during a tournament quarter-final and a hundred million people watch it at the same time, what they feel is the goal. The roar, the replay, the disbelief.

They do not feel the contribution feeds traversing private media networks across continents, or the edge nodes absorbing a traffic spike that appeared without warning.

They just feel the moment.

And that’s exactly how it’s supposed to work.

And as live sports viewership pushes into territory that makes previous records look modest (driven by a generation that expects to watch anything, on any device, anywhere, without waiting), the gap between getting that delivery right and getting it wrong has never been more consequential, or more public.

As audiences moved to digital platforms, the margin for error disappeared.

There is a version of this conversation that is easy to have: audiences expect more, technology has to keep up. True, but incomplete.

Audiences have always expected live sport to work. What changed is what “working” means, and how quickly they find out when it doesn’t.

Viewers no longer sit in front of a single screen. During a FIFA World Cup match, a household might have the main feed on the living room television, while someone else streams the highlights on a second TV in the bedroom, all while phones flash with live stats and tablets run separate commentary. From the infrastructure’s perspective, that isn’t just one household watching a game; it’s a chaotic web of concurrent demands triggered by the exact same split-second on the pitch.

Multiply that across tens of millions of viewers, and the scale of the challenge becomes clear. Social media raises the stakes further. When a platform fails during a World Cup knockout match, audiences report it in real-time on the same platforms they use to discuss the game. The complaint travels faster than the fix.

Broadcasters no longer have the luxury of resolving an incident before people notice. The incident becomes the story, and in many cases, travels further than the match itself.

What these viewership numbers actually mean for infrastructure

The shift in how people watch live sport has moved well beyond trend territory.

EMARKETER forecasts that digital live sports audiences in the US will grow to 114.1 million viewers, while traditional pay TV audiences decline to 82.0 million, highlighting the continued shift toward streaming.

The concurrency numbers generated by major sporting events now sit in a territory that would have seemed implausible a decade ago.

During the 2026 FIFA World Cup, for instance, streaming platforms shattered every historical ceiling, highlighted by Brazil’s CazéTV repeatedly breaking global YouTube records for concurrent viewership during the group stage. Meanwhile, in the United States, Peacock and Telemundo’s digital platforms logged an unprecedented 13 million concurrent viewers for a single knockout window. 

When tens of millions of people tune into the same live stream at the same moment, it’s a challenge unlike regular web traffic.

Historically, massive global audiences were insulated by geography. The load was spread across distinct regional networks: antenna signals, satellite downlinks, and physical cable architectures. The physical infrastructure of traditional television inherently absorbed the impact. 

Digital streaming removes that buffer. Traffic spikes all at once, often at the most critical moment. The tighter the match, the deeper the stoppage time, the sharper the spike. Network infrastructure is forced to handle its heaviest, most volatile traffic exactly when it has zero margin for error.

Social media compounds the pressure operationally. The second a crucial goal is scored, a wave of real-time reactions floods the internet, instantly dragging a secondary “curiosity audience” into the app. These are people who weren’t even watching the match, but saw the hype and decided to tune in, meaning the network has to absorb a massive new rush of users precisely while the primary stream is already maxing out its capacity.

To survive these surges while satisfying a modern audience, the underlying broadcast playbook has undergone a massive structural shift. It’s no longer just about handling traffic; it’s also about using modern technology like AI to manage it intelligently.

According to an industry survey, 25% of broadcasters integrated AI into live production workflows in 2025, a massive leap from just 9% the previous year, with 64% identifying AI as the single largest impact driver over the next five years. 

The network is no longer just delivering content. AI is now generating highlights and short clips in real time, producing millions of videos that keep fans engaged long after the live moment has passed.

Ultimately, the technical demand is driven by a shift in what viewers expect. An IBM sports study revealed that 56% of fans now want AI-driven insights layered directly onto their content, while 33% point to real-time, automated translation as the feature that most impacts their experience.

Whether it’s one screen or several, viewers don’t notice the edge infrastructure or AI powering the experience. They just expect the game to play without interruption.

The planning mistake most organisations make

Capacity planning is where most organisations spend their time when preparing to stream a major event. Can the system handle a million concurrent streams? Can it scale on demand if the numbers exceed projections? These are real questions. 

The lesson is not unique to sports streaming. Every digital business now experiences moments where demand, visibility, and customer expectations collide. Peak traffic events such as flash sales, ticket releases, and viral campaigns can drive website traffic 2 to 25 times above normal levels within seconds. The infrastructure may be different, but the pressure is remarkably similar.

Large-scale system failures occur when multiple components, each functioning as expected on its own, are overwhelmed by a surge in demand, rising latency, or regional blind spots at the same time.

The problem isn’t the individual systems. It’s how they work together.

Latency is the factor most consistently underestimated. A few seconds of delay is not a minor inconvenience in live sport. It is a fundamentally broken experience. 

A viewer whose stream is running four seconds behind will see a notification before the decisive moment appears on screen. Someone watching a service from the privacy of their room may hear a celebration from another room before seeing it on their screen.

Geography is another planning gap. Streaming growth is increasingly being driven by emerging markets. In Southeast Asia alone, premium video streaming subscriptions grew 19% in 2025, led by Indonesia, while viewing hours continued to climb across the region. Yet much of the world’s media infrastructure was originally designed around North American and Western European demand. An architecture that looks robust on paper can deliver very different experiences depending on where the viewer is.

The reason is simple: physical distance still matters. Every extra hop between the viewer and the content adds latency, making it harder to deliver a consistent experience at global scale.

Then there is the timing question. The decisions that determine whether a platform holds during the most-watched minutes of the year are not made on event day. They are made months earlier through choices around architecture, redundancy, testing, and operational readiness.

Once an event is underway, it’s too late to redesign the architecture behind it. If your system isn’t designed to handle the pressure before the crowd arrives, it’s already too late.

The hidden chain behind every live event

When a streaming disruption becomes public, people naturally look for a single point of failure: the app, the platform, or the provider.

A live event depends on dozens of systems working together, and any one of them can become a problem.

And the experience is only as good as the weakest handoff between them.

It all starts with the live camera feed moving from the venue to the production studio. This is a real-time stream, not a file download. If you drop even a single packet at the wrong moment, everything down the line breaks, no matter how perfect the rest of your setup is.

Remote and cloud-based production workflows have redefined how live sports are produced, enabling broadcasters to operate with greater agility and scale. As production becomes more distributed, success increasingly depends on ensuring every stage of the delivery chain works together seamlessly.

Each transition is a potential failure point. Managing them requires visibility that extends across providers, platforms, and networks simultaneously.

Behind every live stream, technologies like encoding, transcoding, packaging, rights management, and ad insertion are constantly at work. If any one of them fails, the stream can go down altogether.

Global distribution introduces another layer of complexity. Viewers in Asia, Africa, and South America may all be watching the same match, but each stream travels across different networks and infrastructure. That means performance can vary by region, and issues may affect one audience without impacting another. 

AI is increasingly helping operators detect anomalies in real time, pinpoint affected regions and trigger corrective actions before disruptions become widespread. Combined with point-to-point monitoring, it provides the visibility needed to keep live events running smoothly at global scale.

Edge delivery is where the difference between preparation and improvisation becomes most apparent. Bringing content closer to users reduces latency, absorbs local traffic surges, and improves performance in markets with variable connectivity. 

The value of technology investments such as AI and Edge becomes clearest during the moments when demand is highest.

Monitoring is what turns visibility into action. With AI helping analyze telemetry and detect anomalies in real time, operations teams can identify issues sooner and respond before they affect viewers. By the time customers start reporting a problem, the opportunity to prevent it has already passed.

What reliability is actually worth

For most of early broadcast history, audience tolerance provided some buffer. Disruptions happened. People accepted them. There was nowhere else to go, and the story rarely escaped the room.

Neither of those things is true now.

A streaming failure during a major match becomes public within seconds. Viewers don’t distinguish between a network issue, a processing failure, or a distribution problem; they simply see a service that failed. That single experience can shape the broadcaster’s reputation, credibility and customer loyalty, influencing whether viewers come back for the next event or recommend the service to others.

The commercial implications are significant. Global tournaments such as the FIFA World Cup illustrate just how valuable live sports rights have become. Their return depends on reliably reaching the audience that was promised.

Advertisers invest in live sport for one reason: to reach a large, engaged audience at the exact moment it matters most. If the stream fails during that window, the opportunity is lost. Those viewers, impressions, and advertising value cannot be recovered once the moment has passed.

The same principle increasingly applies outside media. Customers rarely know nor care whether an outage originated in the application, the cloud environment, the network or a third-party dependency. They experience a failure of the brand. In a digital-first economy, reliability has become part of the customer experience itself.

For broadcasters and streamers, reliability is no longer just an operational KPI. It directly influences audience trust, advertising revenue, and the long-term value of premium sports rights.

The demands ahead are bigger

AI-assisted production is already changing how live events are created. Broadcasters are using AI to automate highlight generation, camera selection and real-time clip packaging for social media, with new AI-assisted workflows producing sports highlights up to 80% faster than traditional methods. 

All of this processing happens within the live delivery chain, where every additional task must be completed without adding latency or compromising the viewing experience.

Personalisation at scale is the next significant challenge. Not personalisation in a vague sense, but the specific technical reality of delivering multi-language commentary tracks, different languages, different statistical overlays, and different camera angles to different viewers watching the same event simultaneously. 

Instead of one stream per event, the infrastructure has to manage a matrix of concurrent variants, each with its own encoding, storage, and delivery requirements. 

Interactive experiences add bidirectional data flows: real-time polls, integrated second-screen data, live wagering. These move data from the viewer back through infrastructure that was primarily built to push content outward. Managing that at scale is a different engineering problem from managing delivery.

Higher-resolution formats (4K now becoming a standard expectation in premium markets, 8K moving into early deployment) are bandwidth-intensive at exactly the scale where bandwidth is already under pressure. Consumer devices are ready. Infrastructure in many high-growth markets is not uniformly there yet.

Many of these capabilities are already being deployed for major global sporting events. The organisations investing seriously in technology, innovation, and infrastructure now are building toward a standard that will be the baseline requirement within a few years. Those that are not will be closing the gap under the worst possible conditions.

The technology you never think about

The broadcasters that succeed don’t leave reliability to chance. They plan for it from the outset, designing their infrastructure to handle peak demand long before the audience arrives.

This reality hits hardest during massive global events. When a stream glitches, millions of people feel it simultaneously in a matter of seconds. Keeping those streams alive doesn’t happen by accident; it takes massive scale, intense discipline, and deep experience controlling everything from the stadium camera to the viewer’s screen.

The lesson extends well beyond live sports. Every enterprise is becoming a real-time digital business, whether it’s delivering AI-powered applications, launching digital products, processing financial transactions, or handling a sudden surge in customer demand. Different industries may face different triggers, but the expectation is the same: the experience has to work, even when demand is at its highest.

Delivering that level of reliability is why many of the world’s largest sports brands rely on Tata Communications. Supporting the broadcast, production, and management of 80% of the world’s sporting events, and reaching more than two billion viewers across 190+ countries, Tata Communications operates in the invisible layers that make every live moment possible. We call this the “Virtual Stadium of the World”, the technology and infrastructure that connects fans, broadcasters, rights-holders, and sporting moments at a truly global scale.

By managing the critical handoffs across contribution networks, edge processing, and global media infrastructure, we engineer the resilience required to keep 120,000 live events running flawlessly every year.

Live sport may be the most visible test of digital infrastructure, but it won’t be the last. As AI, personalisation and real-time experiences become the norm across industries, the ability to deliver reliably at scale will define far more than match day.

To learn more, visit us here.

Major AI labs are unleashing forward-deployed engineers (FDEs) to try and grab enterprise customers. Large consultancies are dishing out tokens and assembling armies of consultants — both human and agent — to do the same.

But smaller firms are in the mix now, as well. AI is helping 28Stone Consulting, a New York-based, 230-person technology consultancy for capital markets, punch above its weight against larger rivals in the rush to deliver FDEs.

In this Q&A, Thomas Dolan and Frank Erickson, founders of 28Stone, argue that agentic AI isn’t a one-size-fits-all solution in vertical markets; success takes discipline, deep domain expertise, and human involvement to mitigate risk.

Many enterprises continue to struggle with the use of AI agents, which is consultancies are stepping in to get projects off the ground. 28Stone is among those that have published blueprints and methodologies on the development and delivery of agentic AI workflows with humans in the loop.

Computerworld spoke with both founding partners about why companies are still stumbling with agentic AI rollouts, and what a disciplined delivery process actually looks like.

After 15 years of delivering software for capital markets firms, is ‘AI-first’ a real distinction or just positioning?

Dolan: “We’re not shying away from being AI-forward. What needs to shine through is AI done intelligently — not stuff you get by buying some tokens for somebody on the trading desk. We’re an AI-first firm.”

Erickson: “And it’s temporary. At some point, AI is going to be synonymous with software development.

“The whole idea of an AI SDLC (software development lifecycle) versus an SDLC is going to be one and the same, a lot like cloud computing today. To not include AI in your strategy, you’d look like a COBOL vendor.”

What does agentic AI delivery look like?

Dolan: “We’ve got several AI initiatives delivering a pure agentic approach. We’ve doubled down on the human expertise wrapper in the SDLC. That doesn’t mean sacrificing any of the benefits of the AI models — quite the opposite.

“You don’t achieve anywhere near the same level of value from applying AI without keeping that expertise — industry, functional and technical — throughout the process.”

Where do humans stay in the loop once agents are doing the work?

Dolan: “We’re believers in starting with requirements discovery. Someone who knows the analytical nuances of a good business analyst is critically important; shaping a product owner’s business information through a markup file that can be fed into a BA agent, then treating the output as if it came from a very fast junior BA. Only then is the story complete.

“The developer takes that story, transforms it into the most efficient input, then owns the output, because they’re accountable for that code. A developer should own the code on both the input and output side.

“Your product owner, who knows the business, that’s great. But expecting them to interact with an agent and output enterprise code is ridiculous. It’s not a great plan.“

Why not just put one do-everything person in charge of AI and agents?

Erickson: “Every analyst, programmer or software engineer isn’t a great requirements analyst. And a great domain analyst with some technical background won’t know if the agent’s code is garbage, maintainable, performant.

“It’s unrealistic to expect one individual to have that breadth across domain, software engineering, testing, deployment. Clients ask all the time, and we push back: ‘Great, if you can find that guy, they’re few and far between.’ To deliver at the enterprise level, you need the human expertise, at depth.“

Dolan: “There’s system speed and latency, important in parts of finance. Then there’s speed of delivery, because other areas evolve quickly and time-to-market is critical.

“Our human wrapper may at first pass come across as a little slowed down. Maybe it is. But [Erickson] has a good analogy about one of the dangers of AI: you can end up going really fast in the wrong direction. By the time you look up, you’re way off base and have to backtrack.“

What about AI in your sector do you think is overhyped?

Dolan: “The hype around the ease of use of AI and the democratization of enterprise software delivery — that ‘anybody could do it now, it’s all being done by machines’ — is another idea that could prove costly in the long run.

“This do-it-yourself reaction is dangerous for clients, and for trust in the overall AI benefit, which is real. We compare it to the beginning of offshoring 20, 30 years ago: a golden idea that was going to cure everything. A lot of firms did it thoughtlessly, thinking it’s just labor arbitrage, and it almost inevitably failed. That all-or-nothing mentality missed that offshoring is an amazing way of getting better value for your dollar, but it has to be done thoughtfully, so the delivery process — the thing that ties it all together — stays unsevered.

“We’re seeing that now. I’ve heard, ‘We’ll just push a button, the machine’s building the system.’ The machine is not building the system. It might be writing the code, the story, running the tests.

The system is built by a team of engineers you bring in and trust. My fear is that people will say, ‘We don’t need this vendor or this technology team. I’ve got a product team. They might not be able to code at all, but they know the business,’ and it fails dramatically. 

“Then people say, ‘We played with AI, it’s not ready yet,’ and throw it all away. One of the best things we can do is ensure clients know the benefit is real.“

Erickson: “The hype can be summed up in a single phrase: vibe coding. That has done AI a massive disservice, because there’s a huge difference between vibe coding and enterprise software development, and some of the loudest proponents of AI are too latched on to it. In our industry, the only way to succeed would be a stable of unicorns. It just doesn’t scale. I get perturbed when our people internally refer to AI tooling as vibe coding; if they think that’s what they’re doing, they’re misunderstood.“

When you engage clients at different levels of AI maturity, how do you get them to a understand what works?

Dolan: “95% of our take on an agentic approach is in line with everyone else’s, but that 5% matters, especially in requirements discovery, in who’s giving the requirements and how they’re thought of. It can set you up for dramatic errors, given the speed at which you’re moving.

“There’s a dangerous human tendency we’re seeing among clients to try and cut corners at the start of a project and — in lieu of having deep, expert driven discovery sessions — just summarize what they may want using AI.

“We would hope our clients are collaborative, everyone understanding it’s early days. If a client insists on doing something we feel strongly against, like a product owner completely owning everything right up to code generation, that’s an issue we have to either push back strongly on or step out of the accountability for.“

AI body shops — LLM providers and giant consultancies — are emerging to help enterprises deploy AI. Does that model work?

Dolan: “Whether you’re partnering with an LLM or with an AI-first, generic software provider — ‘Hey, we’re not industry guys, but we know AI delivery’ — you end up, if you’re a bank or a broker-dealer, saying: ‘All right, we know our business, these guys know the AI side of it. What could go wrong? Put us together and we’ll have quality engineering.’

“The problem is what you miss: the know-how of putting industry and technical expertise together and actually delivering financial services systems. The people working at the generic delivery firms, whether an AI-only firm or a body shop somewhere, don’t have that capability.“

Does AI change the economics for smaller consultancies like yours competing against the big firms, and does it cut both ways?

Dolan: “Over our 15 years pre-AI, there were two recurring reasons we’d lose a project. One: ‘We’d love to work with you guys, given your subject matter expertise, but the costs just aren’t there compared to my budgets. I’m being forced to go to a body shop or an [offshore] delivery center.’ The other side of that coin: ‘We love your capabilities, but you’re a firm of 230 people and I need 300, 400 people.’

“AI changes the options for clients. You don’t have to sacrifice the niche vendor who knows your space just because you need a larger team or a cost target. AI levels the playing field and should allow smaller firms to compete with the larger, big-box generic firms, the Accentures of the world.“

Erickson: “It redefines what scale means. You can look at velocity as a measure of your cost to deliver, not a rate card. Scale can’t be defined in terms of headcount anymore. It’s got to be defined in terms of output.

“There’s a threat in it, too. If you’re an Accenture with hundreds of thousands of low-cost software engineers, how do you train all those people? I feel for them. But for us, a couple hundred people with a specific domain focus, it’s a huge opportunity.“

How has the profile of the people you and others hire changed with this agentic process?

Erickson: “You’re still looking for people with strong engineering and design backgrounds, and communication skills, because they interact across the software development lifecycle more than in the past.

“Many take too much joy in typing out perfect code. Sorry, I don’t need you writing for-loops and classes anymore. I need you reviewing them, understanding them, operating at a higher level. That’s a different kind of person: an engineer, not a programmer or a coder. On the [business analyst] side it’s similar: people took great pride in detailed user stories covering every path. Now it’s conversations, prompts, reviewing output — less doing, more interacting.

“More than ever, they have to be interested in the domain. They can’t just be, ‘I want to learn everything there is to know about Java.’ That’s too narrow. They don’t have to be an expert; they have to be interested. In our case, capital markets is a specific niche. The biggest challenge is getting familiar with the tools — finding time, while delivering for customers, to ramp up and make the mistakes you need to without jeopardizing projects.“

What about governance? Who’s keeping AI delivery and its costs under control?

Erickson: “This is evolving rapidly. People aren’t sure how to put governance around this. The most obvious is financial governance. People are starting to get hefty bills. One of our clients spent a million dollars on tokens over the last eight weeks alone. Sticker shock. The token-maxing policies are starting to show their flaws. It’s wild west still: learn on the fly, then figure out what needs to be governed.“

Are CIOs actually opening their wallets? And when they do, what’s the smarter way to invest?

Erickson: “There’s still a lot of caution. Forecasts keep going down on how long something should take. So: ‘I could wait three months and maybe still get it delivered by the same date someone’s promising me now, but for half the price. I’m going to wait and see when equilibrium is met.’ We haven’t seen the wallets open up like crazy — it’s slow adoption.“

Dolan: “One of our clients is looking at it from a productivity-boost perspective: instead of doing the same for less, I can do much more for the same. AI lets clients pull the trigger on things they wouldn’t have in the past — projects that might not have been approved pre-AI, where the costs have come down to a point that’s palatable with the business.“

Erickson: “And that’s the story we’re hoping to hear more of. There isn’t a huge cost anymore to exploring a business opportunity. The time and money that would have gone to a return-on-investment study could be spent on a proof-of-concept with AI, and the project done a few weeks later. Maybe [there’s] a hint of things to come, where decisions start being made quicker. 

“There’s a little fear on our side, though: a lot of tiny little projects is tough for a consulting business.“

In one ransomware response I observed, the master operational dashboard remained green while the underlying environment told a very different story. It was a classic example of what we in the IT audit profession call the “watermelon effect”—green on the outside, red on the inside.

Beneath that dashboard sat an unmapped web of legacy technical debt, undocumented service accounts and shadow cloud instances. For years, presenting a green dashboard to the audit committee could give technology leaders a false sense of comfort. If a catastrophic breach occurred, it was generally treated as an unpredictable operational tragedy, managed via cyber insurance, a carefully calibrated public relations pivot and perhaps a quiet executive transition.

Today, that corporate shield is thinner than many technology leaders assume. For technology leaders in regulated or public-company environments, executive exposure is no longer only a theoretical debate. The regulatory environment has made plausible deniability much harder to sustain.

The erosion of the corporate shield

With the application of the European Union’s Digital Operational Resilience Act (DORA) for financial entities, alongside the broader NIS2 Directive for essential and important entities, cybersecurity governance has become harder to separate from board-level oversight. DORA places ultimate responsibility for ICT risk management on the management body of financial entities, while NIS2 requires management bodies to approve and oversee cybersecurity risk-management measures. In the United States, the U.S. Securities and Exchange Commission’s cybersecurity disclosure rules require public companies to disclose material cyber incidents and describe their cyber risk management, strategy and governance in annual filings. The new burden is not simply to operate controls; it is to show, after the fact, that leadership decisions matched the risk evidence available at the time.

The serious risk to a modern CIO is not simply the occurrence of a sophisticated security incident. The true danger is the inability to reconcile what leadership presented externally to investors, regulators and the board with what the internal evidence showed inside the environment.

When a serious crisis breaks, you may find yourself surrounded by corporate defense counsel, regulatory investigators and outside forensic lawyers all asking variations of the same uncomfortable questions: What did you know, when did you discover it and what specific actions did you take next?

When those questions are asked, a slide deck asserting that your security posture is “aligned with industry best practices” will not be enough. A post-incident review may recognize that sophisticated attacks occur. What creates greater exposure is evidence that known risks were ignored, understated or left outside structured governance. To survive that level of post-incident review, one of your strongest assets is a disciplined, independent evidence trail showing that risks were identified, challenged, escalated and acted on before the first indicator of compromise appeared.

Why point-in-time comfort letters fail regulatory scrutiny

The reality we face is that legacy compliance evidence often falls short under regulatory scrutiny. For years, the annual SOC 2 Type II report or a standardized ISO 27001 certification was brandished by technology teams as the definitive proof of a functional control environment. I have sat in dozens of scoping meetings where an engineering director pointed to a freshly minted compliance report as if it were a complete defense against scrutiny.

But a compliance report is a historical artifact—a retrospective evaluation of how specific controls operated during a defined window of time months in the past. It tells an investigator that on a random afternoon in Q2, your production change-management approvals conformed to a baseline policy. It says absolutely nothing about the configuration drift, unauthorized API keys or emergency patch bypasses that developers introduced the following weekend to hit a product release deadline.

Modern regulators, boards and investors are no longer satisfied by historical comfort letters alone. Under contemporary frameworks, especially regimes focused on operational resilience, static compliance evidence is no longer enough. The expectation of due care has shifted from a passive state of compliance to an active state of continuous challenge. Increasingly, post-incident reviews look for evidence that leadership identified system vulnerabilities, formally escalated material deficiencies, evaluated systemic risk to the business and tracked remediation progress with measurable rigor.

When an architecture fails, post-incident reviews often focus quickly on ownership, escalation and whether known risks were acted upon. If your defensive documentation consists entirely of static policy documents and green dashboards, you leave an evidentiary vacuum that can invite difficult questions about executive oversight. Post-incident reviews rarely turn on perfection. They turn on whether the organization can show a traceable chain of governance.

5 non-negotiable artifacts for your executive evidence engine

This reality requires a complete reframing of your relationship with your IT audit department. Historically, this dynamic has been defined by friction. Technology leaders frequently view my peers and me as compliance traffic cops—bureaucrats who interrupt core engineering sprints to demand evidence samples, user access reviews and system configurations.

It is time to view IT audit through a pragmatic lens: we are your independent evidence engine. We are one of the few corporate functions tasked with independently challenging your control environment, documenting where exceptions were escalated and showing how management responded. When an auditor identifies a control gap and partners with you to draft a management action plan, they are not creating a bureaucratic roadblock. They are helping you construct an evidence trail that can show risk was identified, escalated and acted upon.

To transform your IT audit function into an effective executive shield, you must shift focus away from superficial check-the-box exercises and collaborate on specific artifacts. The most effective exercise you can run with your audit leadership is to flip the timeline completely and ask: if this program were reviewed six months from now, which evidence would show we governed the risk before it failed?

1. Board-facing risk registers with escalation history: A risk register that sits unreviewed on an intranet page for 12 months is not a management tool; to an investigator, it can look like evidence that known risks were not actively governed. Your material technology, cybersecurity and dependency risks must be centrally logged. More importantly, this artifact must contain a clear, chronological escalation history showing exactly when the risk was presented to leadership committees and the board, along with related minutes, decisions or follow-up actions.
2. Granular risk acceptance records: You cannot remediate every vulnerability instantly. Business continuity, legacy software limitations and budgetary boundaries require you to accept certain operational exposures. When this occurs, ensure your risk acceptance records are airtight. A defensible record must document the specific technical variance, the precise financial or operational rationale for the delay, a definitive expiration date, explicit executive sign-off and the active compensating controls deployed to reduce the blast radius in the interim.
3. Tabletop and operational simulation records: Independent frameworks such as ISACA’s Digital Trust Ecosystem Framework can help structure this evidence, but boards and regulators will still look for proof that the testing actually happened. Your audit trail should contain comprehensive records of cyber incident, disaster recovery and third-party dependency simulations. These records must detail the scenario tested, the executive participants, the control failures identified during the drill and a formalized tracking schedule showing when those gaps were closed.
4. AI governance inventories and data-flow mappings: The rapid deployment of generative AI tools across enterprise operations has created a massive blind spot for technology executives. In one audit, we found developers using an unapproved public large language model to accelerate debugging with sensitive internal code. To protect yourself, work with your audit team to build an active enterprise AI inventory that maps data lineage, identifies model business owners, documents risk classification approvals and demonstrates active technical monitoring for unauthorized data exfiltration.
5. Synchronized disclosure-control handoffs: When a material security incident or system outage occurs, the clock begins ticking for regulatory reporting. Your incident response playbook must be technically linked to your corporate disclosure controls. The audit trail should show that a documented, synchronized handoff occurred between your technical response leaders, general counsel, chief financial officer and corporate communications team. This evidence helps show that your external statements match internal technical realities.

In the modern corporate ecosystem, technology leadership is no longer just an engineering challenge; it is an exercise in rigorous, evidence-based governance. The regulatory landscape has changed, and the expectation of continuous traceability cannot be avoided.

Open and direct collaboration with your IT audit team will not prevent a zero-day exploit, an unexpected cloud outage or a critical third-party vendor failure. That is not the purpose of enterprise risk management.

The true value is far more practical: when a serious incident puts your program under review, you will not be forced to defend your reputation with a feeling, an unverified assumption or a misleadingly green dashboard. Instead, you will have an independent record showing that risk was actively seen, appropriately challenged, properly escalated and responsibly managed. In today’s regulatory environment, that disciplined trail of evidence may be the difference between a failure that can be explained and one that begins to look negligent.

.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

I keep seeing a version of the same scene. A CEO has a question about AI. It is a real question, the kind that will shape where the company spends the next two years. The CEO does not bring it to the CIO. They bring it to a data leader two levels down, or to a vendor who presented at a conference, or to an AI specialist a board member recommended. The CIO finds out the strategy is forming when a slide shows up that they did not build. By then, the direction is already half-set, and the CIO is being asked to react to it rather than shape it.

I want to be precise about what is happening, because it is easy to misread. The CIO has not been removed from anything. Title intact, budget intact, seat at the table intact. What has changed is quieter. On one of the most consequential technology conversations the company will have this decade, the CIO is being routed around. The work still flows through them eventually. The thinking no longer starts with them.

I have watched this happen to capable people who would have given the CEO a better answer than the person who was asked. That is what makes it worth naming. This is not a competence gap. It is a positioning gap, and positioning gaps close in the wrong direction if you ignore them long enough.

How the routing actually starts

The routing does not begin with a decision to exclude anyone. It begins with a CEO who is anxious about AI and looking for someone who sounds certain. AI is moving fast enough that executives feel the pressure to have a point of view before they have earned one. That pressure usually arrives secondhand, from a board member or a peer on the golf course describing what is working at their company. So the CEO goes looking for someone who will confirm the answer they already want to hear, and they keep going back to whoever gives it to them.

Here is where many technology leaders lose the thread. For years, the safe posture in the CIO seat was measured caution. You raised the risks, you flagged the integration cost, you asked who owns the data and what the compliance exposure looks like. That posture built credibility in an era when the failure mode was moving too fast on technology nobody understood. With AI, the same posture reads as drag. A CEO who is being told by three vendors that “the future is already here” does not want to hear why they should slow down and be cautious. They hear caution as losing the race, and they go find a point of view somewhere else.

The data leaders, vendors and specialists who get the call are not necessarily more capable. They are more available with a confident answer. A vendor’s whole job is to arrive with conviction. A data scientist who has shipped one impressive model carries more apparent authority on AI, in that moment, than a CIO who runs the entire estate but talks about AI the way they talk about every other risk. The CEO is not weighing depth against depth. They are weighing the person who said yes against the person who said it depends.

Once that pattern sets, it compounds. The CEO who got a satisfying answer from the data leader goes back to the data leader. The vendor who shaped the first conversation gets invited into the second. Each loop the CIO is not in makes the next one easier to run without them. The org chart still says the CIO owns technology strategy. The actual conversation has relocated.

What it costs before anyone notices

The cost shows up late, which is exactly why it is dangerous. For a while nothing looks broken. The CIO is still delivering. The AI initiatives are still landing on their plate to execute. The damage is happening upstream, in the room where the bets get made, and the CIO is not in that room.

I have seen what arrives downstream when the strategy was set without the person who has to run it. A model gets championed that the data cannot actually support. A vendor commitment gets made that locks the company into an architecture the CIO would have flagged in the first meeting. An agent gets deployed inside a business unit, with executive blessing, and the CIO inherits accountability for it months later without ever having shaped how it was governed. The recent IBM finding that CIOs are increasingly held accountable for AI they do not fully control is the visible end of this. The invisible front end is the conversation the CIO was routed around, the one where the accountability got created in the first place.

What I find most corrosive is what it does to the CIO’s standing over time. Every initiative the CIO executes but did not shape reinforces a story about what the CIO is for. They become the person who runs the technology other people decided on. That is a fine description of an order taker and a poor description of a strategic leader, and CEOs do not promote, fund, or defend order takers when budgets tighten. The routing-around does not just cost the company a worse AI strategy. It quietly recasts the CIO as the implementer of everyone else’s thinking, and that recasting is hard to reverse once the executive team has internalized it.

What the leaders who stayed in the conversation did

The technology leaders I have watched hold their position on AI did one thing first. They stopped leading with caution and started leading with a point of view. Not a reckless one. A real, defensible position on where AI creates value in their specific business and where it does not, delivered with the same conviction the vendors bring, before the CEO went looking elsewhere for it. They made themselves the person with the clearest answer, which is the role the routing-around was filling with someone else.

That requires giving up a posture that felt safe for a long time. The CIOs who made the shift accepted that on AI, being right and cautious is worth less than being early and directional. They formed a view ahead of being asked. They walked into the CEO’s office with where we should place our AI bets and why, rather than waiting to be handed someone else’s bets to pressure-test. The difference is whether you are the author of the strategy or its editor, and CEOs route around editors.

They also changed how they talk about risk. Instead of presenting risk as the reason to slow down, they folded it into the recommendation. The data is not ready for that use case, so here is the use case where it is ready, and here is what we do in parallel to unlock the first one. That framing keeps the CIO inside the conversation as the person making AI happen responsibly, rather than the person standing outside it explaining why it is hard. Same expertise, opposite effect on whether the CEO keeps coming back.

None of this is about pushing the data leaders and specialists out. The strongest CIOs I know pulled those people closer and brought them into the room under their own framing, so that when the CEO wanted the specialist’s input, it arrived through the CIO rather than around them. They made themselves the orchestrator of the AI conversation instead of one of its casualties.

If you are a technology leader right now, the question worth sitting with is not whether you are good at AI. You probably are. The question is whether the most important AI conversations in your company are still starting with you, or whether you have quietly become the person they get handed to after the thinking is done. That answer is set in rooms you may not be in, and the only way to find out is to ask who your CEO called the last three times AI came up. If the answer is not you, the role is still yours. The conversation has already started leaving.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Chinese tech giant Huawei is reportedly entering the DRAM manufacturing business in a bid to cash in on the insane profitability of memory sales.

Three firms – Micron Technology, SK hynix, and Samsung Electronics — account for 95% of the DRAM on the market worldwide. The rest is small players, mostly in China. One of them, CXMT, is gearing to make a run for the market and try and take a little bit of their business. But it is a small player, especially compared to Huawei.

Huawei’s strategy is complex. It is working with various entities to circumvent the U.S. trade sanctions specifically targeting it. According to SemiconductorInsider on X.com, the Chinese government and DRAM chip maker Swaysure (formed in 2022) are planning to introduce a DRAM manufacturing plant with a capacity of 140,000 wafers per month.

For perspective, Samsung manufacturers approximately 500,000 wafers per month, and Micron makes about 250,000 wafers per month.

It all fits in perfectly with Beijing’s drive for semiconductor self-reliance uh especially after all of the U.S. sanctions. And it would be US interference, not Huawei’s inexperienced with making memory that will be its greatest challenge, says one analyst.

“Their biggest problem will be tooling up, since the US has a lot of sway over the world’s semiconductor equipment makers, and it’s using that sway to prevent tools from being shipped to China,” said Jim Handy, president of Objective Analysis.

Handy said he hears that Huawei and Swaysure are working to produce DRAM starting at 28nm.  With that process they ought to be able to make 8Gb chips that yield reasonably but he doubts that they will be able to make HBM with that process, though.

Since he doesn’t know how far along they are in the process, Handy doesn’t know when something could come to market.  “They may be on the cusp of shipping something, or they may be a couple of years away, but I don’t think they will wait any longer than that, so let’s say 2028.  If there’s still a shortage in 2028, they are likely to ramp as hard as they can to get to something like 2-5% of the market by 2030.  If the shortage ends before then, they will probably take a much longer time,” he said.

With AI increasingly tucked into every cranny of the enterprise, someone has had to step up and provide the tools necessary to discover, track, and monitor all the agents and LLMs and keep them humming along in their various workflows. Thankfully, the DevOps world answered the call, building the tools to support our new overlords in an emerging subdiscipline interchangeably called “AIOps,” “AgentOps,” and sometimes “agent observability.”

Many of the challenges involved in AgentOps are similar to those tackled by traditional DevOps tools and processes. After all, at their foundation, LLMs are just software running on hardware somewhere. Typical issues involving RAM and disk space are just as important in the agent world, maybe more so because AI operations are even more greedy about consuming storage than regular software is.

Many of the companies supporting agent observability are big names in DevOps circles, having adapted their stacks to address the idiosyncrasies of modern LLMs. IT teams maintaining enterprise agents can treat the LLMs as just one node in a big graph filled with services that are constantly swapping packets and triggering software jobs. Latency and resource constraints must be managed because end-users don’t care whether it’s an LLM, a database, or a plain-old Python script that’s failing, bringing their work to a grinding halt.

But new AI-specific challenges are opening the door to newcomers that are building tools with the peculiarities of LLMs in mind — for example, keeping deeper logs filled with records of prompts. LLMs are also often very non-deterministic by design, making it trickier to pinpoint failure modes. And then there’s the fact that an agent will give a perfectly intelligent answer one minute and hallucinate the next.

Relying on many of the same approaches that DevOps tools do, AgentOps tools watch for misbehavior and flag anything out of the ordinary for deeper analysis. This may be as simple as fixing slow responses, but it can also include AI hallucinations and other issues born of LLMs’ non-determanism.

Teams trying to choose which agent observability tools is best for their use case should look at the size and nature of their agentic systems and projects. Are they adding AI agent features to an existing product or application, or are they building agentic systems from scratch? Are they more focused on maintaining a stable LLM operation or iterating on new approaches? Is AI the center of attention or just an add-on that’s meant to improve an existing stack?

The AgentOps and agent observability options listed below share many of the same features but differ in their focus and their attention to the challenges organizations will encounter when incorporating agents into their stacks. Each tool offers a worthwhile place to start understanding how to care for the growing presence of AI in the production world.

AgentOps.ai

When teams of agents work together, tracking the conversations are essential for understanding and debugging what’s happening. The SDK from AgentOps.ai records events so that the creators can replay past behavior to track details such as token counts, spending, latency, and more. Available as a service and on-premises.

Pricing: Starts at $40 per month plus usage costs at $0.20 per 1M tokens

Standout feature: Replay analytics with “time-travel debugging”

Best suited for: Complex agent debugging

Arize Phoenix

Debugging prompts and LLM responses requires a nuanced understanding of just what’s happening, in part because of the non-determinism that often enters the process. Phoenix from Arize supports this process with robust tracing and the ability to score the results for more precise iteration. Their system can track the results and tool calls from a variety of major platforms (Anthropic, AWS, OpenAI, etc.) that are initiated by the major frameworks (LangChain, LlamaIndex, DSPy, etc.). The result is insight into what data is triggering what chain of responses.

Pricing: Small free tier; Pro plan starts at $50 per month plus costs tied to events

Standout feature: LLM-as-a-Judge metrics for tracking quality

Best suited for: Teams focusing on iterating for accuracy and quality

BigPanda

BigPanda has always offered solutions for tracking performance of complex systems. Now the company is drilling deeper into the challenge of detecting and ending the problems that come from models that go awry. BigPanda’s main system relies on historical data and machine learning algorithms to flag issues. Its own agent layer connects the problematic nodes and errant models while dispatching alerts to the right team members.

Pricing: “Value-based” table on request

Standout feature: Automated triage for faster response

Best suited for: Large teams seeking to reduce alert fatigue from large customer base

Braintrust

Setting up an effective improvement cycle for an AI agent requires a strong feedback loop from production data to the agent’s next generation. Braintrust watches the production workload and creates test vectors that expose how an agent may be drifting, regressing, or departing from its path. The tool automates much of the testing and scoring feedback loop so problematic patterns can be discovered and addressed. A core part of the offering is a specialized data store that can track large and sometimes deeply nested collections of tests and their results. Their approach may be summarized by one of their tag lines: “trace everything.”

Pricing: Free starter tier; Pro plan starts at $249 with some usage-based costs covered

Standout feature: Highly scalable trace ingestion

Best for: Teams developing strong guardrails through continuous testing

Chronicle Labs

When it’s time to release a new version of an agent into the wild, the platform from Chronicle Labs specializes in staging it and testing it with a collection of use tests and regression cases. The tools are also helpful during development cycles. “Backtest your agent against reality,” their sales material promises, with a set of tools that mines the production telemetry for solid test vectors that stress every part of the agent with prompts and challenges that the agent will encounter after leaving the safety of the lab.

Pricing: On request

Standout feature: Back-testing options for complex testing regimes

Best for: Teams chasing strong models with good fidelity to reality

Comet Opik

Building a dashboard for tracking every in-flow and out-flow to agents is one way to be ready to watch for and solve problems. Opik from Comet is just such a tool. The DevOps teams can track each call and add its own automated routines to examine the results, score them based on 30-plus metrics, and if desired, send it off to another LLM to evaluate the results. Agents that are constantly failing stand out. DevOps teams can also ask questions like, “Who is using this model and racking up all of the bills?” The same goes for MCP skills and other cogs in the machine.

Pricing: Free tiers for open source and small projects; Pro plan starts at $19 per month with usage limits

Standout feature: Auto-scoring with 30-plus metrics for evaluating traces

Best for: Teams focusing on RAG and agentic workflows

Datadog

DevOps teams that rely on Datadog to track logs across collections of services can also use it to track LLM operations, which are, of course, just another source and sink for data. It will track performance such as time to first token and offer insight into what might be causing an issue, such as lack of memory. Results then get plugged into the same cost-tracking mechanism so the bean counters can predict when the budget will run out. After all, the CFO likely doesn’t care whether the bill comes from an LLM or an old-school S3 storage bucket. Datadog integrates AI into their tools by treating these models as just another source of data.

Pricing: Small free tier with multiple paid tiers for various levels of enterprise monitoring

Standout feature: Large installed base with broad focus on more than LLMs

Best for: Large enterprise teams working with established infrastructure

Dynatrace

For more than 20 years, Dynatrace has been delivering tools that track dataflows across the full stack. Now that AIs are finding roles in many of the nodes in this complex graph, they’re expanding to track how various AI agents can interact. They want to build one platform that helps track the root cause and, often now, deploy solutions autonomously. They want to focus on being ready to support complex networks of agents that detect problems in either performance or security and then work within defined guardrails to fix them. Determining the right role for their own AI-powered agents is a key part of the product.

Pricing: Plans start at $7 per month with larger plans designed for full enterprise monitoring

Standout feature: High level of autonomous monitoring designed for large installations

Best for: Complex, hybrid environments mixing LLMs with traditional services

Galileo

Placing some AI systems into production is often a harrowing experience because the actual performance is impossible to predict, even with the most rigorous tests. Galileo offers guardrails that track performance and watch for any behavior that deviates from the ground truth. Their “LLM-as-judge” systems are distilled into compact models that can be run locally for lower costs and faster performance.

Pricing: Small free tier; Pro plans start at $50 per month with usage-based limits and costs

Standout feature: Real-time guardrails for deployed agents

Best for: Security-conscious installations that need to defend against hallucination and data leakage

Grafana Labs

Long the go-to source for open source telemetry, Grafana Labs now tracks performance of AI models in constellations of services. Grafana tracks the evolution of answers across the agentic network to recognize how small changes or hallucinations can spin out of control. It bills its system as “actually useful AI” and has even trademarked it. Its cloud assistant can configure and reconfigure the Grafana dash to offer the right level of observability. Its system includes AI-level analysis that can flag models that are responding quickly but offering bad answers because of problems such as model drift or context degradation.

Pricing: Basic free tier; Pro plan begins at $19 per month, includes better retention and some usage-based fees 

Standout feature: Full-stack tool with fully integrated LLM tools

Best for: Large, enterprise-scale system adding AI

Helicone

Sometimes shoehorning in another tool into the chain can be tricky. Helicone is designed as a smart network proxy that will route all model requests while keeping solid debugging records from the data as it goes by. The data it captures can be turned into nice charts that make it easy to spot latency issues or model failures. Naturally, tracking AI spend is also a feature in much demand as bills continue to climb.

Pricing: Small free tier; Pro plan starts at $79 per month, includes features such as team collaboration and improved querying

Standout feature: Proxy-based integration

Best for: Development teams who want to add better monitoring features quickly

Laminar

Tracking agents in development and production means building strong storehouses of data enumerating what happened. Laminar works closely with OpenTelemetry to follow agents operating in production so that flaws and failure modes can be understood from log files stored efficiently with their own compression scheme. Developers can search through traces with an SQL-ish language and Laminar’s transcript view illuminates what happened. When necessary, the traces can enable developers to scroll back in time and replay the same inputs for debugging. The goal is to offer deep insights with high-level visibility of how well the agents are meeting business objectives.

Pricing: Small free tier; “Hobby” tier that adds more features at $30; Pro level starts at $150 per month

Standout feature: Open-source license makes self-hosting a viable option

Best for: Teams fully able to leverage open-source responsibilities

LangChain LangSmith

Real-time data from agents is essential for managing any mutli-agent system in production. LangSmith from LangChain traces costs, tools, and progress toward solutions for a wide collection of agents using SDKs for Python, TypeScript, Go, and Java. The OpenTelemetry-based solution watches for anomalies, issuing warnings and alerts through dashboards and communication channels such as PagerDuty. Deeper analysis can reveal issues such as topic clustering or odd patterns of failure. Coordination with agent deployment platforms such as LangGraph and deepagents ensures greater focus on successful resolution of assignments.

Pricing: Free for solo developers; Pro teams start at $39 per person per month 

Standout feature: Systematic approach to regression testing of prompts

Best for: Teams relying on LangChain and LangGraph frameworks for supporting complex agentic behavior

Lunary

Watching the user experience is essential for building AI applications such as chatbots and assistants. Lunary offers a proxy that traces all interactions and then builds analytical dashboards for measuring metrics such as user satisfaction or model costs. One common usage is finding frequent topics and looking at the responses to ensure they deliver. When prompts aren’t perfect, Lunary lets teams iterate on the prompt text until the right answers are coming out. Its proxy structure and common API format enables Lunary to promise to work with “any LLM, any framework.”

Pricing: Free tier; Pro plan starts at $20 per month

Standout feature: Deep integration with humans for reviewing and optimizing results

Best for: Startups focused on rapid prompt innovation

NewRelic

The platform that began tracking performance of some web applications is now powerful enough to track the flows of data through complex agentic ecologies. NewRelic’s AI-driven monitoring watches for golden signals that can indicate misbehavior or worse throughout the entire lifecycle. It tracks every detail of the interactions through protocols such as MCP and then makes this available to the AI engineers responsible for performance. The dashboard provides the insights necessary to watch for toxic behavior, overt bias, drift, and overblown hallucinations. Predicting and maybe even controlling the cost is also a growing role as tokenomics becomes as important as response time.

Pricing: Free tier; Pro plan fees available through website

Standout feature: Full-stack support with hundreds of integrations with other tools

Best for: Established enterprise teams mixing in AI

Nova AI Ops

The goal of Nova AI Ops is to deliver a team of agents that watch over a cloud and make it, at least partially, self-healing. Each agent uses a mixture of predictive AI and machine learning to watch cloud telemetry reports for anomalies. Then they calculate the “blast radius” and decide whether this is a problem that can be fixed automatically “while you sleep” or saved for the human supervisors. These tools are aimed not just on LLM operations but on the stack as a whole.

Pricing: Small free tier; Standard pricing  begins at $40 per user per month with usage billing

Standout feature: Focus on software reliability engineering helps teams deliver stable stacks

Best for: Teams that want to integrate LLMs into incident response and stability management

Splunk

The platform that began delivering smart logging is now fully AI capable, offering solutions that can watch over agents with much the same way that it continues to track microservices. Splunk now includes a fairly large amount of predictive AI for learning from the information in the logs and then turning this learning into fast solutions. This AI assistant can track deployed AI models connected by protocols such as MCP and watch over behavior while delivering the ability for users to drill down and explore what’s working and what’s failing. Their AI Canvas is meant to offer a central hub where the AI scientists can track both the local behavior of the models as well as their role in a larger data ecosystem.

Pricing: Activity-based pricing tracks usage of LLM backends and storage

Standout feature: Ready to scale to large enterprise stacks

Best for: Teams with legacy systems that are folding in agentic options

SuperPenguin

One of the most important parts of an AI service is the bill. SuperPenguin is a product designed to track consumption and make predictions so that the CFO won’t be surprised. The goal is to provide solid estimates about the total cost of each product by allocating costs to customers, features, and teams. If there’s a sudden shift, a “spike detector” will raise an alarm so that dev teams can ensure that the AI spend is worth it.

Pricing: Small free tier for experimentation; Growth tier for teams, starting at $30 per month; Pro tier offers deeper options starting at $200 per month

Standout feature: Strong accounting with invoice reconciliation and PR-level usage tracking

Best for: Teams that need precise cost accounting

Vellum

Prompt engineers spend time fussing over the details of tweaking, improving, and enhancing the words that guide the LLM. Vellum started as a company that would provide the pipeline so that you could manage and improve the prompts that ran again and again. Now the system is growing more powerful, offering a higher level of automation that lets you meta-manage the prompt chain. They’ve also begun marketing it as a form of personal assistant with pre-built connections to many of the major services such as Gmail. Its llm-cost-optimizer can juggle multiple options while finding a cheaper way to execute a prompt, a process the company suggests can save 60% or more.

Pricing: Open-source free tier; Pro plan starts at $35 per month

Standout feature: Focus on multi-model pipelines for true agentic solutions

Best for: Product teams with complex prompt engineering workflows

For a decade, Kubernetes was the right answer. It organized containers, scaled services horizontally and gave platform teams a shared vocabulary for running software in production. It abstracted away enough of the underlying complexity that engineers could stop thinking about servers and start thinking about services. Most cloud-native infrastructure today is built on top of it, directly or in spirit, and EKS made that model the default for the majority of enterprise teams running workloads on AWS.

The workload that defined that era was the stateless HTTP request, fast in, fast out, disposable. A user action triggers a request, the request hits a service, the service returns a response and the container is done. Kubernetes was optimized for that pattern down to the scheduler internals: Bin-pack containers onto nodes, autoscale on CPU and memory, evict and reschedule when something goes wrong. The whole system is tuned around the assumption that individual units of work are short, stateless and interchangeable.

That assumption no longer holds for the workloads that matter most right now.

The agent workload is structurally different

Agents are long-running, stateful processes. They reason across time, call external tools, spawn subprocesses, write and execute code, and make decisions that depend on what happened five steps earlier in the same task. A single-agent workflow might run for minutes or hours, touching a dozen external systems and generating intermediate outputs that subsequent steps depend on. The compute layer for that kind of work needs to do things the old model was never asked to do. That is the new pattern: Execution infrastructure designed around agent semantics rather than request semantics.

The Kubernetes community itself has acknowledged this mismatch. In March 2026, Kubernetes SIG Apps published an introduction to Agent Sandbox, a new CRD-based abstraction designed specifically for singleton, stateful agent workloads. The framing is direct: The ecosystem is moving from short-lived, isolated tasks to deploying multiple, coordinated AI agents that run continuously, and mapping those workloads to traditional Kubernetes primitives requires an entirely new abstraction. The fact that the Kubernetes maintainers built a dedicated primitive for this, rather than recommending teams compose one from existing resources, is itself the clearest signal that agent execution does not fit the old model.

What agent execution actually requires

Concretely, it requires four things. First, isolated execution environments that provision in milliseconds, not minutes, so each agent task gets its own sandbox for code execution and tool calls without blocking the reasoning loop. The difference between a two-second environment and a two-minute environment is not a performance optimization; it determines whether the architecture is viable at all. Second, durable state management across the full task lifecycle, so an agent can pause, hand off or resume without re-initializing from scratch and burning tokens to reconstruct context it already built. Third, coordination primitives for multi-agent work: The ability to spawn subagents, pass structured outputs between them and track task dependencies across a graph of concurrent processes. Production agent systems are rarely single agents; they are pipelines of specialized agents with handoffs that need to be reliable and inspectable. Fourth, credentials and secrets management that travel with the execution context, so agents can authenticate to external services securely without exposing credentials in the task definition, logs or the environment variables of a shared container.

The mismatch shows up fast in production

Kubernetes and EKS expose the mismatch quickly in practice. Pod eviction terminates an agent mid-task with no clean recovery path. Autoscaling reads CPU utilization as the load signal, but an agent holding a long inference connection looks idle to the scheduler even when it is doing the most consequential work in the pipeline. Provisioning a new environment takes 45 seconds to two minutes on a well-tuned cluster; agent workloads need that in under two seconds or the reasoning loop stalls and the user experience degrades visibly. These are not edge cases or misconfigurations. They are the normal operating conditions for production agent workloads running on infrastructure that was not designed for them.

The utilization data makes the broader cost picture even starker. The 2026 State of Kubernetes Optimization Report from CAST AI, drawn from analysis of over 23,000 production clusters across AWS, Azure and GCP, found average CPU utilization at 8 percent, down from 10 percent the year prior. Memory utilization fell from 23 to 20 percent. CPU overprovisioning jumped from 40 to 69 percent year over year. These numbers reflect clusters running traditional workloads, and the pattern is worsening, not improving, as environments scale. Agent workloads compound this problem further. An agent holding an open inference connection or waiting on a tool call registers as idle to a scheduler that reads CPU and memory as the only meaningful load signals. The infrastructure responds to the wrong metric, overprovisioning capacity for demand it cannot measure, while the actual bottleneck, environment provisioning latency and state continuity, goes unaddressed.

Security is not the same problem it was before

Agent workloads change the threat model at the infrastructure level. A compromised stateless service exposes a narrow surface defined by its API contracts. A compromised agent exposes every system it can reach, every credential it holds and every action it is authorized to take on behalf of the user. Agents generate and execute their own code, make non-deterministic tool-call decisions and accumulate context across long-running sessions. Standard container namespacing does not contain that kind of risk. Kernel-level isolation, default-deny network egress, scoped credentials per session and agent-aware observability are not optional hardening steps. They are baseline requirements for running agents in production.

What teams that ship agents have already figured out

Some of the clearest evidence for this shift comes not from infrastructure vendors but from product engineering teams running agents at scale on their own code. In late 2025, Ramp’s engineering team published a detailed account of building Inspect, their internal background coding agent. Each Inspect session runs in a sandboxed VM with a full-stack development environment and deep integrations across their observability, CI, and deployment tooling. The architecture requirements map almost exactly to the four primitives above. Filesystem snapshots keep sessions starting in seconds rather than minutes. Sessions are isolated and stateful. The agent can run tests, review telemetry, query feature flags and visually verify frontend changes in a real browser. And the whole system supports unlimited concurrency, so engineers can spin up ten parallel sessions exploring different approaches to the same problem without contention.

The results speak for themselves. Within months of launch, roughly 30 percent of all pull requests merged to Ramp’s frontend and backend repositories were written by Inspect. That level of adoption was not mandated. It happened because the execution environment was fast enough, capable enough and well-integrated enough that the agent was strictly better than a local workflow for a meaningful share of tasks. The key insight from the Ramp case is not about the model. It is about the execution layer. As their team put it, session speed should only be limited by model-provider time-to-first-token; everything else, like cloning and installing, needs to be done before the session starts. That is a statement about infrastructure, not intelligence.

The ecosystem is catching up, but defaults are sticky

None of that is a criticism of the tools. Kubernetes solved exactly the problem it was designed for, and it solved it well. The issue is that infrastructure defaults are sticky. Teams inherit them, build on top of them and optimize within their constraints long after the underlying workload has changed. The Kubernetes community’s own response, the Agent Sandbox project under SIG Apps, validates the thesis that a new abstraction is necessary. The new primitives the community is building include warm pools for near-zero cold starts, lifecycle management for suspending and resuming idle agents without losing state, and pluggable kernel isolation for secure execution of untrusted code. These are not incremental improvements to existing resources. They are net-new abstractions that acknowledge the old model does not stretch to fit.

But adoption of purpose-built agent infrastructure remains early. Enterprises building agent pipelines today are largely running a request-oriented orchestration model against an execution-oriented workload, and the mismatch shows up in task failure rates, runaway costs and debugging cycles that have no good tooling because the observability layer was also designed for stateless services.

The structural advantage is available now

The infrastructure to close that gap exists now. The prerequisite is recognizing that agent execution is a first-class compute pattern with its own primitives and its own requirements, not a variant of the stateless service model that defined the last decade. Teams that make that shift early will have a meaningful structural advantage. The ones that do not will spend the next two years wondering why their agent systems are unreliable at a scale that should be tractable.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Data Products tragen dazu bei, die Art und Weise zu standardisieren, wie Rohdaten, Data-Warehouse-, sowie logische Data-Lake-Ansichten kombiniert und genutzt werden, um Analyse- und KI-Funktionen bereitzustellen. Indem sie Datenprodukte entwickeln, können Teams in Unternehmen einen Großteil der im Vorfeld erforderlichen Daten-Pipelines sowie Governance- und Management-Tasks optimieren. Darüber hinaus gewährleisten diese auch, dass Mensch und KI auf vertrauenswürdige Datenressourcen zugreifen.  

Kochen bietet an dieser Stelle eine hilfreiche Analogie: Sie könnten sich dazu entscheiden, für Ihr Lieblingsgericht ausschließlich auf frische Zutaten zu setzen. Dieser Ansatz funktioniert gut, wenn Sie sowohl die Zeit als auch die nötigen Fähigkeiten dafür mitbringen. Wenn nicht, setzen Sie eventuell lieber auf Convenience-Bestandteile – insbesondere unter Zeitdruck. Datenprodukte bieten eine vergleichbare Zeitersparnis – Analytics- und KI-Funktionen bauen in diesem Fall auf konsistenten, (vor)optimierten „Zutaten“ auf.

Die folgenden fünf Tipps sollten Sie bei Ihrer Data-Product-Initiative unbedingt verinnerlichen.

1. Data Products strategisch nutzen

Die meisten Unternehmen können es sich nicht leisten, für jede Datenvisualisierung, jedes Machine-Learning-Modell oder jeden KI-Agenten eigene Datenprodukte zu entwickeln. Schließlich ist das mit Kosten und Zeitaufwand verbunden. Dazu kommt: Sobald ein Data Product bereitgestellt ist, müssen die Produktmanager für fortlaufenden Support und ein entsprechendes Lifecycle-Management sorgen. Die erste entscheidende Frage ist also, in welchen Fällen es für agile Daten-Teams Sinn macht, Datenprodukte zu entwickeln – und wie dabei priorisiert werden sollte.   

Ein Ansatzpunkt besteht darin, das Data Product auf einen einzelnen Datensatz herunterzubrechen und sich zu überlegen, was es bedeutet, diesen zum Produkt zu machen. Danielle Ben-Gera, Vice President of Engineering bei Crunchbase, erklärt: „Ein Datensatz sollte erst dann zu einem Datenprodukt werden, wenn sich mehrere Teams bei Entscheidungen – oder zum Support von Anwendungen – darauf verlassen.“

Dabei seien eine angemessene Governance, klare Zuständigkeiten, Versionierungen und ein kontrollierter Lebenszyklus für Änderungen essenziell, warnt die Managerin: „Ansonsten liefert man nur instabile Pipelines aus, die die nachgelagerten Workflows zum Erliegen bringen.“

Eine andere Überlegung, die zu Data Products führt, ist die Nutzung von Daten außerhalb der Governance. An dieser Stelle kann ein Datenprodukt einen taktischen Ansatz darstellen, wie Yaad Oren, Global Head of Research and Innovation bei SAP, nahelegt: „Wenn Datensätze teamübergreifend ohne strenge Governance, klar definierte Prozesse oder eindeutige Zuständigkeiten genutzt werden, ist Unternehmen zu empfehlen, ein Data Product zu entwickeln. Datenprodukte, die in einer einheitlichen Datenbasis verankert sind, beseitigen Silos, schaffen ein gemeinsames Verständnis über die Daten und etablieren einen sicheren, standardisierten Zugriff auf diese.“

Eine dritte Möglichkeit, Datenprodukte strategisch zu nutzen, ist, diese für definierte Kunden in wiederverwendbarer Form zu entwickeln, um Effizienzgewinne einzufahren. Wenn ein solches Data Product erfordert, mehrere Datenquellen miteinander zu kombinieren, ist das Vision Statement und qualifizierter Business Value besonders wichtig. Christopher Zangrilli, Vice President of Technology Strategy beim Compliance-Dienstleister Vertex, erklärt: „Führungskräfte sollten sich fragen, ob die Daten die Cycle Times optimieren, die Entscheidungsgenauigkeit verbessern oder Compliance-Risiken mindern, um den Business Impact einzuordnen. Wenn Governance, Change Management, Qualität und Messverfahren von Beginn an integriert sind, wandeln sich Datenprodukte von experimentellen Tools zu strategischen Ressourcen.“

2. Datenprodukte standardisieren

Produkte im Supermarkt sind mit einer Verpackung versehen, auf der eine detaillierte Liste der Inhaltsstoffe, ein Verfallsdatum und ein Preis angegeben sind. Ganz ähnlich sollten Data-Governance-Verantwortliche vorgehen – und standardisieren, wie Data Products definiert, katalogisiert und gemanagt werden. Wie und warum, erklärt Abhi Sharma, Mitbegründer und CEO des KI-Anbieters Relyance AI: „Jedes moderne Datenprodukt sollte vier Fragen klar beantworten: Woher stammen die Daten, wie werden sie systemübergreifend transformiert, wer oder was nutzt sie und welche Governance-Verpflichtungen fallen dabei an? Ohne diesen durchgängigen Kontext entwickeln Teams Funktionen auf der Grundlage von Daten, die sie nicht vollständig verstehen.“

Obwohl Lebensmittelhersteller ihre Inhaltsstoffe veröffentlichen und mit Blick auf Gefahren wie allergische Reaktionen kennzeichnen, dokumentieren nur wenige die Herkunft ihrer Rohstoffe und welchen Weg diese vom Erzeuger zum Händler nehmen. Geht es darum, Data Products in streng regulierten Branchen zu entwickeln, kann es allerdings erforderlich sein, genau das zu tun – und die Data Lineage zu erfassen. Besonders wichtig ist das, wenn es darum geht, Datenquellen für KI-Applikationen zu standardisieren.

Carter Page, Executive Vice President of Research and Development beim Dev-Spezialisten Astronomer, weiß, was anderenfalls droht: „Ohne Data Lineage arbeiten Teams im Blindflug und Governance verkommt zu reaktiver Fehlerbehebung. Wenn Teams dagegen nachvollziehen können, woher die Daten stammen, wie sie transformiert wurden und welche Systeme darauf angewiesen sind, werden Aktualisierungen vorhersehbar, die richtigen Pipelines getestet, die betroffenen Stakeholder benachrichtigt und grundlegende Änderungen dokumentiert. Bevor es dadurch zu Incidents kommt.“

3. Data Products nachhaltig managen

Lebenszyklusmanagement erfordert bei APIs, Anwendungen oder KI-Modellen, einen Release-Plan für Optimierungen, Fehlerbehebungen und andere notwendige Updates festzulegen. Geht es hingegen um Datenprodukte, kommen mehrere, verwandte Disziplinen zusammen, wie Ulf Viney, EVP of Engineering beim KI-Datenspezialisten Precisely, erklärt: „Um den Lebenszyklus von Data Products zu managen, braucht es Versionierung, Testing, strukturierte Deployments und Stakeolder-Kommunikation.“

Ein weiterer grundlegender Unterschied bei Datenprodukten: Ihr Lifecycle Management ist eng damit verbunden, wie die zugrundeliegenden Datensätze wachsen – beziehungsweise, welche strukturelle Veränderungen diese durchlaufen. Ein Data Product, das zwar funktioniert, aber nicht veränderungsresistent ist oder keine Warnmeldungen ausgibt, wenn Fehlerbehebungen erforderlich sind, kann nachgelagerte Anwendungsfälle beeinträchtigen und das Vertrauen der Stakeholder und Nutzer in die Daten untergraben. Insbesondere letzteres gilt es zu verhindern. Wie, weiß Bethany Sehon, Senior Director of Enterprise Data bei Capital One: „Ein nachhaltiges und skalierbares Governance-Framework kann sicherstellen, dass Daten leicht zu finden, zu verstehen und zu nutzen sind.“

Teams, die geschäftskritische Echtzeit-Datenprodukte managen, die mehrere nachgelagerte Analytics- und KI-Anwendungsfälle unterfüttern, sind die folgenden DevOps- und Data-Governance-Praktiken zu empfehlen:

• Legen Sie unverhandelbare Data-Governance-Kriterien fest – insbesondere, wenn es darum geht, Datenqualitäts-Benchmarks zu setzen, etwaige Verzerrungen zu identifizieren und Datenschutzrichtlinien einzuhalten.
• Nutzen Sie fortschrittliche CI/CD-Pipelines, Continuous Deployment sowie Continuous Testing und automatisieren Sie Produktions-Deployments.
• Stellen Sie sicher, dass sämtliche Datenintegrationen über „observable“ DataOps verfügen, Datenqualitätsprobleme überprüft werden und Alerts ausgesendet werden, wenn die Pipelines zum Erliegen kommen. Um Requests und Incidents zu bearbeiten, sollten IT-Services zudem entsprechend definiert werden.
• Stützen Sie sich auf Plattform-Strategien wenn es um Datenmanagement geht – zum Beispiel im Hinblick auf Data Fabrics, DSPM, Dokumentenverarbeitung und Vektordatenbanken.

4. Datenprodukte verargumentieren

Ein Data Product auf die Beine zu stellen, ist leider kein Garant dafür, dass dieses auch angenommen wird. Das verdeutlichen auch die Beispiele von Reusable Code, API-Nutzung oder DevOps-Tools: Sie alle zielten darauf ab, Entwicklern das Arbeitsleben leichter zu machen und die Qualität zu verbessern. Trotzdem nahmen viele Teams lieber eine „Not invented here“-Haltung ein und setzten lieber auf Eigenentwicklungen statt die Standards anderer.

Datenprodukte stehen allerdings vor noch größeren Herausforderungen. Ganz besonders, wenn sie darauf abzielen, Datensilos zu konsolidieren oder Tabellenkalkulationen zu eliminieren. Um die Akzetanz zu fördern (und Feedback einzuholen), sollten die für die jeweiligen Data Products verantwortlichen Produktmanager deshalb ein Change-Management-Programm entwickeln. Förderlich ist dabei, darzulegen, wie das Datenprodukt auf den kulturellen Change und die KI-Strategie des Unternehmens einzahlt – etwa indem es die Demokratisierung von KI vorantreibt und die Kompetenz im Umgang mit der Technologie optimiert.

5. Data Products richtig evaluieren

Der Geschäftswert eines kundenorientierten Produkts wird häufig gemessen anhand der Auswirkungen auf den Umsatz, der Nutzungs-Metriken sowie der Kundenzufriedenheit. Interne, mitarbeiterorientierte Produkte lassen sich hingegen anhand ihrer Workflow-Effizienz, ihrem Potenzial für Produktivitätssteigerungen und der Mitarbeiterzufriedenheit evaluieren.

„Zu viele Unternehmen behandeln Datenprodukte immer noch als technische Outputs und nicht als strategische Assets“, kritisiert Daniel Ziv, Global Vice President of AI and Analytics beim KI-Anbieter Verint. Der wahre Wert von Data Products lasse sich daran ablesen, wie einzigartig die generierten Daten sind, wie viel messbaren Einfluss sie auf Entscheidungen nehmen, meint der Manager: „Wenn jedes Unternehmen Zugang zu denselben KI-Modellen hat, ergibt sich der Wettbewerbsvorteil aus ‚uniquen‘ Daten und der Geschwindigkeit, mit der diese in Maßnahmen umgesetzt werden können.“

Eine Best Practice auf die IT-Entscheider in diesem Zusammenhang zurückgreifen können, ist es, Metriken heranzuziehen, die Aufschluss über die Geschwindigkeit digitaler Transformationsvorhaben geben. Dazu gehören etwa:  

• „Time to Data“,
• „Time to Decision“,
• „Time to Innovation“, und
• „Time to Value“.

(fm)

Dieser Artikel ist im Original bei unserer Schwesterpublikation Infoworld.com erschienen.

Enterprises are facing two simultaneous challenges with AI: The risks associated with it are evolving faster than governance frameworks, while the business benefits are often difficult to measure.

This is one of the key findings of The Value of AI, a study commissioned by SAP from Oxford Economics. Now in its second year, the study surveyed 2,600 executives from 13 countries worldwide.

High expectations, limited preparation

On average, the enterprises surveyed plan to spend around $28 million on AI (up from $26.7 million last year), and expect a 21% ROI (from 16% last year). Expectations for AI agents are particularly high, with ROI expected to reach 17% this year, up from 10% last year. Furthermore, 83% of respondents worldwide said agentic AI has the potential to fundamentally transform their organization. On the other hand, only 3% of respondents said their enterprises were fully prepared for the deployment of AI agents.

There are gaps, particularly when it comes to governance:

• Only 12% of respondents said their skills or processes were able to govern AI effectively,
• 38% do not have human-in-the-loop processes in place for oversight of AI agents, and
• only 63% have established permissions and access controls for agents.

Other concerns include weaknesses in the organization of AI deployment, poor data quality, insufficient employee training, and the widespread use of shadow AI.

Governance is the bigger challenge

In an interview, Sean Kask, Chief AI Strategy Officer at SAP, commented on the study’s key findings.

Mr. Kask, in the study’s foreword, you write that companies are currently facing two challenges simultaneously: The risks associated with AI are evolving faster than governance, while the business benefits are often difficult to measure. Which of these poses the greater problem for companies?

Sean Kask: Measuring the business value of IT investments has never been easy. The same applies to AI. That’s why I currently consider the governance issue to be the greater challenge. While traditional governance principles and best practices for secure software development remain important even in the age of large language models and agent-based AI, entirely new risks are emerging at the same time.

For example, as soon as companies roll out AI on a broad scale, they suddenly discover hundreds or even thousands of so-called shadow agents that employees are using without central oversight. Or they find that a significant portion of the workforce is copying content into private ChatGPT accounts. Such risks often only become apparent once AI is already being used productively.

According to your study, German companies invest an average of nearly $40 million in AI, more than companies in all other countries surveyed. Why is that?

Kask: I was less surprised by the amount of investment than by the fact that, overall, the level of investment and the return on investment achieved have developed very similarly across the various countries. There’s no clear answer as to why Germany invests more. In part, it’s likely simply because costs here are higher than in India, for example.

However, we’re also seeing a high level of AI adoption among German companies. SAP has a dashboard that allows us to track how our customers are using AI features. Germany is among the countries with particularly high usage. Added to this are the strong industrial base and the political impetus from Europe, which are driving the use of AI. Accordingly, companies there are making targeted investments in building the necessary expertise.

According to the study, 47% of German companies are satisfied with the return on investment from their AI investments. At the same time, 77% say they are still far from realizing AI’s full potential. Isn’t that a contradiction?

Kask: No, we see this pattern worldwide. Companies initially invest in a few AI use cases and realize: This works; we’re creating added value. Accordingly, they’re satisfied with their investment.

But this is precisely what leads them to identify further use cases. They explore AI agents and want to utilize them as well. However, it is exactly at this point that many encounter new challenges in implementation and scaling.

The study therefore primarily highlights a learning curve: The more experience companies gain with AI, the greater their awareness of its previously untapped potential becomes.

According to the study, only 33% of companies surveyed have KPIs at the executive board level that are directly linked to the implementation of AI. In your view, which metrics should supervisory boards and CEOs definitely be tracking?

Kask: For us, a key indicator is employee enablement. How many employees have already successfully completed training or upskilling programs related to AI? Without the appropriate skills, AI adoption will fall short of its potential.

Transparency is equally important. Companies should know which AI agents are actually in use within their landscape. SAP offers the SAP AI Agent Hub for this purpose, which automatically discovers and inventories agents from SAP and third-party environments. Customers have already been able to identify thousands of agents this way, which highlights the need for centralized governance and transparency.

In addition, companies should have a complete overview of all AI use cases. A robust business case should be in place for each use case. We often see two extremes: Either the executive board is under pressure to implement AI as quickly as possible and allocates a lump-sum budget for this purpose. Or management initially takes a wait-and-see approach. This leads to independent pilot projects springing up throughout the company, with individual departments procuring their own tools and entering into their own contracts.

At SAP, we therefore follow a clearly structured selection process. Each idea first undergoes an assessment of its expected business value. We then examine technical feasibility, data availability, and ethical and governance aspects. From management’s perspective, it is crucial to maintain transparency regarding all ongoing AI projects at all times and to consistently prioritize them based on their business value.

Agents, too, need a ‘hire-to-retire’ lifecycle

Even with the introduction of dozens or even hundreds of AI agents, governance becomes increasingly complex. What capabilities do enterprise platforms need to manage AI agents securely and in a controlled manner at scale?

Kask: We make a conscious effort not to anthropomorphize AI too much. Nevertheless, the analogy is helpful: Agents require a complete hire-to-retire lifecycle. This begins with the detection and registration of an agent. It is then integrated into the enterprise environment, granted the necessary permissions, and given access to the data sources it needs to perform its tasks.

Observability is just as important. Companies must be able to track what an agent is actually doing in the system at all times. In addition, they should track key performance indicators: Is the agent achieving the desired results? How efficiently is it working? How many tokens does it consume? How many processing steps does it require for a task?

Ultimately, this involves several key components: a complete inventory of all agents, appropriate governance, risk, and compliance (GRC) mechanisms, transparency regarding agent behavior, and continuous monitoring. This is the only way to ensure that AI agents consistently operate within defined parameters and deliver the desired business value.

In your estimation, which business processes will companies actually delegate entirely to AI agents over the next two to three years?

Kask: Currently, such agents work particularly well in clearly defined use cases. SAP will release more than 50 (currently 34) specialized AI agents.

One example is periodic financial reporting. In this context, journal entries must be made based on numerous rules stored in documents, emails, or previous transactions. The agent analyzes these various sources of information, derives a recommendation from them, and suggests the appropriate journal entry to the user.

Based on what we’ve heard from customer projects, employees at medium-sized companies currently spend about twelve hours per month on these tasks. With the help of an AI agent, this effort can be reduced to two to three hours.

Another area of application is production planning. If delivery dates change or new orders come in at short notice, the entire production plan must be adjusted. It is precisely these kinds of complex optimization tasks that are ideally suited for AI agents.

In principle, there are virtually no limits to the narrowly defined business processes in which agents can be deployed. However, they will not operate completely autonomously at first.

Trust in AI begins with a stable foundation

Many companies still struggle to trust AI agents. After all, large language models operate probabilistically and can produce false information. This is particularly problematic in financial processes. How do you build trust?

Kask: Trust begins with a stable foundation. ERP systems remain the reliable system of record. They operate deterministically, contain the business logic, and hold the relevant company data. AI agents build upon this foundation. They do not replace it.

Equally important is the human-in-the-loop principle. Employees must be able to understand what the agent is doing, verify its results, and intervene if necessary. That’s why employee training also plays a crucial role. They must understand how generative AI works and where its limitations lie.

Of course, language models can hallucinate. At the same time, we must not forget that humans are not infallible either. The key lies in the collaboration between humans and AI. This allows us to improve both the efficiency and the quality of many business processes.

Another important component is transparency. Our global AI ethics policy, for example, stipulates that users must always be able to recognize when AI is involved. In Joule, it’s possible to trace which data sources the agent used and which steps it went through in reaching its decision. This traceability is an essential prerequisite for trust.

What distinguishes an SAP agent from a general AI agent that merely accesses an ERP system?

Kask: The key difference is that Joule and the SAP agents are directly embedded in the ERP system. There, for example, we’ve built a knowledge graph that describes the semantic relationships between all tables, business objects, and data fields.

To put this into perspective: The SAP S/4HANA Knowledge Graph is based on approximately 452,000 ABAP tables, 7.3 million data fields, and thousands of analytical views. The semantic relationships between these artifacts are modeled in the Knowledge Graph and made available for AI applications.

For example, if a user wants to view all open purchase orders, the agent does not first have to laboriously search for the relevant information. It immediately knows which tables and objects are relevant and also understands the relationships between a purchase order, a purchase requisition, the responsible approvers, and other business objects. As a result, the agent not only works much more precisely but also requires significantly fewer tokens because it can greatly narrow down the search space.

If, instead, one attempts to simply overlay AI onto an existing system or extract data from a relational ERP system, many of these relationships are lost. In a sense, this destroys the semantic context that is crucial for precise answers.

That is why we view the ERP system as an enormous strategic advantage. It has been the system of record for decades and contains roughly 50 years of codified business and process knowledge. This knowledge forms the foundation for what we call the autonomous enterprise. The agents build upon this knowledge and continue to develop it.

In the future, SAP agents will also communicate bidirectionally with agents from other providers via standards such as Agent-to-Agent (A2A).

According to your study, AI currently creates the greatest added value in decision-making, customer interaction, and gaining new insights, rather than in traditional productivity gains. Will this change the way companies justify AI investments in the future?

Kask: In our study, productivity was simply rated slightly lower than, for example, gaining new insights. In the long term, however, productivity remains the ultimate goal. Europe, in particular, has been suffering from comparatively weak productivity growth for years.

At SAP, we therefore first evaluate every new AI feature based on its specific business value. For all agents and AI features that we include in our AI Feature Catalog, we first conduct a value analysis. We ask: What benefit does the feature offer the user? Does it contribute to higher revenue? Does it increase productivity? Only then is it developed further.

At the moment, the greatest added value often still lies in consolidating information from structured and unstructured data sources and making it accessible via natural language. The next step, however, is to translate these insights directly into more efficient business processes. That is precisely where the greatest productivity gains will be realized in the future.

If you could give CIOs just one or two pieces of advice for the transition from generative AI to AI agents, what would they be?

Kask: In my view, the biggest mistake would be to try to transform the entire company all at once or to attempt to perfectly prepare all the data right from the start.

Instead, you should consider what kind of agent can create significant added value, and then implement it. Of course, this agent needs access to consistent and context-rich enterprise data. That’s exactly what we’re working on at SAP with technologies like the knowledge graph, which maps the semantic relationships within enterprise data.

In addition, with data products and the SAP Business Data Cloud, we provide tools that make data from various sources usable for AI agents. Thanks to zero-copy and data fabric approaches, information from legacy systems, Snowflake, or ERP systems can be consolidated without first having to extensively replicate the data. For a procurement agent, this makes it possible to provide exactly the relevant data for the specific use case.

The key point is this: Companies do not have to wait until they have fully migrated to the cloud or consolidated their entire data landscape. With the technologies available today, data can already be made usable for specific AI agents, managed in a controlled manner, and used to quickly generate initial business value. On the other hand, those who wait for the perfect starting point run the risk of falling behind.

This article is adapted from one first published by Computerwoche.

---

Every day this past week I did something I suspect millions of other people also did: I stared at an LLM model picker and wondered which one I was supposed to want.

OpenAI just released ⁠GPT-5.6 Sol, Terra, and Luna. Sol is the flagship. Terra offers much of its intelligence for less money. Luna is cheaper still. Anthropic released ⁠Claude Sonnet 5 at the end of June and Opus 4.8 the month prior, with a little Fable 5 emerging in between. Meanwhile, Google, which seemed to be winning the model wars a few months ago, is now getting shade from Gergely Orosz, who ⁠argues that Gemini has slipped outside the top tier for software development and has been out of the major model release game for eons (May 19).

Perhaps Orosz is right. Perhaps he’ll be wrong again in six weeks. Honestly, it’s exhausting.

I use ChatGPT and Claude constantly and still have no principled idea which model to choose most of the time. I tend to click whatever looks like the biggest, most expensive option because I don’t know what I’m giving up by choosing something smaller. “Instant” sounds dangerously unserious. “Thinking” sounds expensive but powerful.

A quick survey of my LinkedIn crowd suggests others also feel my “WHICH MODEL???” pain. More importantly, I suspect most enterprises do, too.

A model doesn’t rot

Before getting carried away, however, it’s worth considering whether any of this model churn actually matters. After all, a model doesn’t rot. The model an enterprise put into production in March performs just as well in July as it did when the company selected it. “Obsolete” generally means that something better now exists, not that the deployed model suddenly stopped summarizing insurance claims or classifying support tickets. (In other words, once you have something working, the idea that “but maybe Opus 200.2 is better!” is really a FOMO problem, not a performance issue.)

Most enterprise workloads don’t live at the frontier anyway. Extraction, summarization, classification, document comparison, and customer-service assistance often work perfectly well with smaller, cheaper models. OpenAI’s own pitch for the trio of GPT-5.6 models isn’t simply that Sol is better. It’s that ⁠Terra and Luna deliver different combinations of intelligence, latency, and cost. Luna, the cheapest tier, nearly matches the previous generation’s peak performance at less than half the estimated cost, according to OpenAI.

The practical question, of course, is where to start. An enterprise can’t test every model, every reasoning setting, and every price tier before doing any work. So here’s my advice (which I don’t follow in my own work, but I’m not defining enterprise strategy and can be a little price-insensitive). Start with the cheapest credible model that appears capable of the task. Give it a representative set of real examples and, before you start testing, define what counts as good enough. If it passes, stop. If it fails, move up a tier or try a model with strengths better suited to the work.

That sounds almost offensively simple, but it reverses the way many people, including me, use these products. We start with the biggest model because we’re afraid of what we might lose. Enterprises should start lower and require evidence before paying for more intelligence.

There are exceptions, of course. For genuinely difficult work, such as autonomous coding, complex research, or high-stakes reasoning, beginning with a frontier model may save time. But even then, the goal should be to establish a quality ceiling, then test whether a cheaper model can meet it. It’s changing the question from “which model is best?” to “what is the least expensive model that reliably clears the bar for this job?”

For many workloads, that price improvement matters more than a few extra benchmark points. ⁠As I argued back in 2023, following AI hype doesn’t help anyone. If your model strategy depends on whichever benchmark screenshot is circulating on X this week, you don’t have a strategy. Not a viable one, anyway. Pick a model and ignore the noise.

Except, of course, when that noise suggests a serious signal.

Sometimes better really is better

Frontier improvements aren’t always incremental, making it advantageous to consider an upgrade. Coding is the obvious example. There’s a significant difference between a model that suggests the next few lines of code and one that can inspect a repository, plan a change, use tools, run tests, discover its own mistakes, and keep working for an extended period. That isn’t merely a nicer autocomplete experience. It can reorganize a development workflow.

This is why enterprises can’t simply standardize on an 18-month-old model and declare victory. In some areas, particularly software development and other agentic work, better models can unlock compounding productivity. A model that reliably completes 80% of a bounded task rather than 50% may justify an entirely different division of labor between humans and machines.

Still, that upgrade isn’t free.

Models differ in how they interpret instructions, call tools, manage context, refuse requests, and fail. Prompts and scaffolding tuned for one model can regress when moved to another. Or costs can explode. As one of my Oracle colleagues discovered just this week, running the same tasks in GPT 5.6 was orders of magnitude more expensive than 5.5. The API change may be trivial, but the revalidation and implications are not.

This leaves enterprises caught between two bad options. They can freeze and potentially miss out on meaningful improvements or chase every release and repeatedly test production systems on faith. What to do?

Stop making model bets

The answer is to stop making LLM bets and start making job-to-be-done bets. Stop asking which model is fastest. Instead, figure out what work you are trying to improve. What does a good result look like? How much latency and cost can the workflow tolerate? How wrong can it be before a human must intervene? Once those questions have answers, model selection becomes less opaque.

A difficult code migration may justify GPT-5.6 Sol or Claude Sonnet 5. A repetitive classification task may work just as well with Luna or another smaller model. A regulated workflow may require a model or deployment option that offers particular data controls. Sometimes the correct model is no LLM at all, like when I’m writing this post. Sorry, AI vendors! (At least you won’t get blamed for my mistakes.)

This is where evaluations become the center of enterprise AI strategy. ⁠As I’ve said before, most companies don’t have an AI quality problem so much as an AI measurement problem. Hence, a private evaluation suite built from real company work is the only leaderboard that matters. Does the new model materially improve quality? If so, use it! Does it reduce cost or latency? Again, that’s your free pass to adoption. Does the improvement justify the expense and effort of revalidation? If yes, continue.

Make model releases boring

As important as the model is, keep in mind that AI success always comes back to your company’s data, your company’s workflows, your company’s integrations, etc. That’s the ⁠dull reality behind sexy AI. Retrieval, memory, governance, data quality, observability, and feedback loops aren’t as exciting as a new model launch, but they’re what ultimately make AI truly work.

Again, when it’s time to consider something new, the principle should be to default to the least expensive model that reliably passes your evaluations. Only escalate harder tasks to more capable models when measurement shows that the premium pays. Tip: Make this invisible to employees so that the system routes to the best model for a particular prompt. As dbt Labs’ Jon Lewis expresses it, “The best model is ‘Auto’ and I won’t hear anyone say otherwise.” OpenAI’s own ⁠migration guidance recommends testing models on representative tasks, including trying a lower reasoning level rather than automatically cranking everything to the maximum.

As for me, I’ll probably keep clicking the shiniest option. I don’t have a formal evaluation suite for InfoWorld columns, and the marginal cost is a subscription I already pay. Enterprises don’t get that excuse.

Microsoft has released TypeScript 7.0, a native port of the company’s strongly typed version of the JavaScript language based on the Go programming language.

With this new codebase, TypeScript 7.0 brings native code speed, shared memory multithreading, and a number of new optimizations that typically yield speedups between 8x and 12x on full builds, Microsoft said in a July 8 announcement. This port of TypeScript to native code was done as faithfully as possible, writing new code while maintaining the structure and logic of the original codebase to keep results consistent and compatible between the two compilers, according to Microsoft. 

TypeScript 7.0 is available via npm:

npm install -D typescript

To help with the TypeScript 6.0 to TypeScript 7.0 transition process, Microsoft has published a new compatibility package, @typescript/typescript6. This package provides an executable named tsc6, so that if needed, a developer can install TypeScript 7.0 (which ships its own tsc binary) side-by-side with TypeScript 6.0 without naming conflicts. The new package also re-exports the TypeScript 6.0 API, so that a developer can use tsc for TypeScript 7, while other tooling can continue to rely on TypeScript 6.0.

TypeScript 7.0 is a major milestone in the TypeScript project, Microsoft said. This port has been the primary focus for Microsoft’s team for more than a year. With TypeScript 7.0 now out, the team will return to new feature work, ergonomic improvements, more performance wins, and implementing a new API for the broader ecosystem. Microsoft expects a fairly similar timeline to releases prior to TypeScript 7.0, with new versions published every three to four months.

TypeScript 7.0 was announced as a release candidate on June 18.

Port has rolled out Port AI Builder, a vibe coding platform designed for software development and platform engineering teams.

Announced July 14, Port AI Builder lets teams create and run agentic workflows in natural language, with built-in human-in-the-loop review and approval. The platform lets organizations apply AI agents across the SDLC (software development life cycle), drawing on domain skills spanning site reliability engineering, devops, architecture, security, AI governance, data modeling, and UX, while maintaining governance and visibility, the company said. Port AI Builder is available through free and paid subscriptions.

Port AI Builder works on top of Port’s Agentic SDLC Platform, which provides the context lake, workflow orchestration, agent management, and governance that enterprises need to operationalize AI SDLC, according to the company. The new AI Builder lets teams build production-grade workflows in minutes, for use cases such as autonomous resolution, AI cost management, and engineering performance tracking, without losing control, Port said.

The first time I heard someone say their AI now wrote 80% of their tests, I asked the obvious question. Eighty percent of what?

After 20 years building and leading test automation for consumer-scale platforms, my honest answer turned out to be eighty percent of the typing, not eighty percent of the engineering. The remaining twenty was where the work still lived. Budgeting for two percent of leftover effort was the mistake. When the real number was closer to thirty, that gap was the difference between a pipeline that shipped and one that quietly built up a queue of half-trusted features nobody could rely on.

This piece is about that gap. As an independent research project on LLM-augmented testing methodology, I built a six-stage agentic pipeline that takes a design in Figma and produces running tests in WebDriverIO, connected end to end over the Model Context Protocol. It works. It has been useful. And the parts that broke surprised me, because they were not the parts the hype cycle tells you to worry about.

How I wired a six-stage pipeline over one protocol

The pipeline runs six stages in sequence, each owned by a different agent, with every handoff crossing MCP.

Six-stage agentic test pipeline: design capture → requirements writer → ticket opener → code generator → test-case writer → automation generator. Each stage carries an MCP handoff and a provenance stamp.

The end-to-end trace links a pull request back to a Jira ticket, a requirements section and a Figma frame. Each artifact is stamped with the agent that produced it, the model it used and the inputs it was given.

MCP is the boring middle that makes any of this work. The cliché is that MCP is “USB-C for AI”: one open protocol, any tool. Like most analogies, it is about eighty percent right. The part that matters is the eighty: I do not have to write a custom adapter for every system the agent talks to. One MCP server per tool and every agent talks to all of them the same way.

Typed handoffs between agents are my own architecture, layered on top of MCP rather than provided by it. Each agent writes a typed artifact the next agent reads. Each handoff is logged with provenance. When something went wrong six stages in, I could replay the chain. Without that discipline, a multi-agent pipeline is a debugger’s worst day. You know the test plan is wrong. You cannot tell whether the mistake came from the Figma read, the requirements interpretation or the ticket scaffolding. With it, I could point at exactly which stage went sideways and which inputs it was looking at when it did. The pattern lives in a public MIT-licensed reference implementation for any reader who wants to run it.

The sixteen-minute number is the marketing number. I ran the full chain end to end in about sixteen minutes on a synthetic net-new screen, Figma in, automation suite out. That repeated across my runs; it is not a demo trick. But sixteen minutes is the part of the story most fun to tell and least useful to learn from. It is what gets quoted in the all-hands. The hours that come after, when a human reviews each handoff, are where the work actually lives.

What actually broke in production-style runs

The failures that stalled my pipeline were rarely the ones I expected.

I expected hallucinated APIs. I got them: the agent confidently called endpoint names that sounded right but did not exist. I expected sparse-spec-in, sparse-spec-out, where a Figma frame with no annotations produced a requirements doc with vague acceptance criteria, every time. I expected locator drift, the common UI-automation failure mode where a renamed component silently breaks an entire test suite. There is solid outside writing on non-determinism in tests covering this whole family of failure modes, and the agent inherited every one.

What I did not expect, and what kept the pipeline down longer than any of the above, was the plumbing.

The model backend timed out under load. It lost credentials silently and started returning empty strings, which the agent then read as confidence. A duplicate consumer on a shared long-poll API endpoint produced an HTTP 409 conflict that broke delivery without throwing anything visible. One unguarded exception inside one agent aborted a whole shared scheduler run and took the other agents in the registry down with it. The single worst incident cost me three hours to find. An environment variable had silently rotated overnight; every agent in the fleet was returning structurally valid but semantically empty requirements docs; the downstream stages were dutifully generating tests against nothing.

None of those are model bugs. They are infrastructure. The agent literature, which is what I went looking through when I started this work, mostly does not talk about them.

The fix was not better prompts. It was circuit-breaker-style review checkpoints between stages and what I now call the four-guard discipline: four small guards I consider non-negotiable on any unattended agentic pipeline. The bulkhead pattern from microservices is the most consequential. An unhandled exception inside one agent can no longer abort the shared run; the offending agent fails fast with a structured error and the others keep going. Paired with that, a pure-data fallback ensures a model timeout produces a deterministic output explicitly marked as degraded mode, rather than an empty string the next stage will misread as confidence. A single-owner lease sits on every shared external endpoint, the cure for the duplicate-consumer incident that ate one of my Sunday afternoons. The cheapest guard was the last to arrive: a one-line synthetic canary every agent has to produce a known correct response to before any real work begins, so a credentials rotation or silent backend failure trips an alert before downstream stages have generated artifacts against garbage.

None of these guards is novel. They are textbook stability patterns at a new boundary: the seam between the LLM agent and the rest of the system, which most of the existing agent literature still treats as a solved problem.

The 20% you don’t see, and when not to do this

Here is the part the demo videos leave out. Even when the pipeline works, the human time per stage does not go to zero.

Human review time per ticket across five pipeline stages: code review 60-180 min, automation review and flaky-fix loop 30-90 min, ticket architecture and sequencing 30-60 min, test data and environment 15-30 min, requirements review 20-30 min. Net: the human still spends 20-30% of the original effort, almost all of it reviewing rather than creating.

Net of all that, the human still spends twenty to thirty percent of the original effort, almost all of it reviewing rather than creating. The pipeline saves seventy to eighty percent, not ninety-eight. The trap is budgeting for the two percent you do not save.

When does this kind of pipeline make sense? In my experience, when the Figma is richly annotated and acceptance criteria are clear up front; when there is review capacity to absorb the work the pipeline shifts onto humans; when the stack is well represented in the training data; and when the feature is net-new rather than a deep edit of legacy code. When does it not? When the design lives on a whiteboard. When the integration touches old code with hidden contracts. When the path is regulated or safety-critical. When there is no senior reviewer who can hold the line. When the work is exploratory and writing the spec is the actual point of the exercise.

Teams I have seen succeed with agentic pipelines budget for the rework explicitly, staff the review queue and treat the saved hours as capacity for harder problems rather than headcount they can release. Teams I have seen struggle did the opposite: declared victory at the demo and quietly accumulated a backlog of half-trusted features the next quarter had to clean up.

The right unit of measurement is not how much the pipeline generates. It is how much of what it generates a human still has to touch before you would ship it. Call it the 80/20 rework rule: measure the rework, not the generation. The teams that get the rework number right are the ones whose AI investments compound. The teams that stop counting at the headline percentage are the ones that own the cleanup six months later.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

These days, application developers can take their pick from a vast menu of architectural solutions. We can choose from the well-understood to the experimental, and from blended solutions in between. Several powerful middle-ground technologies that emerged during the cloud revolution have really come of age. Here we’ll take a look at putting together three of the most impressive: GitHub, Vercel, and Firestore.

Each of these is an important tool in its own right that can be used to attack specific problems. In combination, they not only meet the needs of several important application scenarios, but they have a superpower—the ability to dramatically shorten the distance between development and deployment.

There is nothing quite as gratifying as putting your hands on just the right mix of tools for a given need.

A ‘no-ops’ stack built for speed

If your primary goal is sheer development velocity, you would be hard-pressed to top this architecture. This “no-ops” stack collapses the distance between your local IDE and a globally distributed production environment. You are essentially trading the overhead of managing VMs and load balancers for the sheer speed of committing code and watching it deploy automatically.

While each component is highly flexible, adopting them requires a specific, event-driven mindset. There are a few finicky bits to manage, mostly around routing environment variables securely and designing around stateless back-end functions. But the constraints are obvious and well-documented.

Before we look more closely, let’s quickly identify the kinds of apps that are a perfect fit here, along with those that are workable and those that really merit a different approach.

• The sweet spot (deploy and go): AI-mediated applications, asynchronous game back ends, and real-time collaborative B2B dashboards. This architecture perfectly absorbs the unpredictable latency of LLM APIs and instantly syncs state across multiple clients without requiring you to build custom WebSocket infrastructure.
• The middle ground (workable, with trade-offs): Headless e-commerce, moderate IoT telemetry, and apps requiring scheduled batch processing. You will encounter friction if your catalog relies on deeply relational SQL constraints, or if your background reporting jobs take longer than a few minutes and hit serverless execution limits.
• The danger zone (look elsewhere): High-frequency trading, fast-paced action multiplayer games, heavy data ETL pipelines, and core financial ledgers. Serverless architectures cannot natively hold open the persistent WebSockets required for twitch-reflex data, and heavy compute tasks will abruptly time out.

We should mention that these categories are not mutually exclusive. Many enterprise applications, such as a full-scale e-commerce platform, straddle these lines. You might use Vercel and Firestore to build a lightning-fast, reactive storefront that handles ephemeral user state like shopping carts, while simultaneously “stitching in” a managed SQL database like Supabase or PlanetScale. This hybrid approach allows you to maintain the relational integrity required for back-office inventory and financial ledgers and pair it with the front-end velocity this stack provides.

GitHub: the bedrock

I don’t need to introduce you to GitHub. It is a central element of the development landscape. I still remember CVS and SVN with a certain nostalgia, but the enhancements of Git speak for themselves. When combined with the orchestration powers of GitHub, it is no wonder that virtually the whole industry has adopted this type of platform.

Git plus GitHub gives you an enormous amount of power already, in terms of how you can organize and automate your projects. But there is a next-level experience in combining GitHub and Vercel. For JavaScript-based projects, you can take simple GitHub pushes and turn them into instantly deployed clients and serverless functions. It is one of the cleanest and least fiddly ways to move from raw code on your local machine to a globally deployed, full-stack architecture.

Vercel: the nexus

Vercel is more than just a deployment host. It is a control plane that ties this high-velocity, no-ops architecture together. Alongside GitHub and Firestore, Vercel’s deeper strength is its ability to act as an orchestration layer between your reactive front end and external stateful services.

Vercel has a great amount of facility in fine-tuning what branches go to what environment and helpful features like instant rollback. You can just log into Vercel’s dashboard for your project and see the history of deployments and any errors and logs. It’s a simple menu choice to roll back to a historical version or compare one version against another.

When you “stitch in” third-party services (such as a managed SQL database like Supabase or a payment processor like Stripe), Vercel’s serverless functions become the lightweight interface, and Vercel’s the adapters handle the communication. You offload the integration logic (the service layer) to Vercel’s global Edge Network, keeping your UI and back end clean, responsive, and decoupled. 

In short, Vercel allows you to get the speed of the “no-ops” development life cycle without sacrificing the complex transactional integrity required for some applications like enterprise inventory systems.

Firestore: the datastore

Firestore is an extremely lightweight, NoSQL, cloud datastore. It has a great deal of add-on power, but its core value proposition is that it accepts virtually any data you stuff into it and it provides event-driven subscriptions to data changes.

These two capabilities together make Firestore about as straightforward a solution to a managed back end as you can imagine. You subscribe to collections or even fields and then you simply stick “unstructured” data (read: JSON with variable fields) in and the client waits for the changes it is interested in.

This is so streamlined that one can just point the browser (or native mobile app) directly at Firestore and listen for events. Which immediately raises the question of identity, for auth and for data visibility, but hold on—Firestore’s third superpower is that it has an authentication module that actually works. What I mean is, it is actually pretty simple and yet confidently secures your app.

Sometimes auth solutions seem either too simple (and yet opaque) or too mired in the nitty gritty. Firestore auth will let you do some basic configuration and start using a reasonable auth almost immediately. 

Not to belabor the point, but having a realistic and attainable auth solution elevates your stack to a production grade—one that can handle many real-world applications. Firestore auth plays nicely with other important APIs, like Stripe. Typically, auth is a major feature that feels like off-roading in a Honda Civic, but Firestore’s approach to auth, added to this particular stack, feels like a normal speed bump. It’s just another component you plug in, rather than a tentacled alien you weave into the your code.

The limits of the velocity stack

This architecture combines components that are optimized for flexibility. That same character also introduces distinct limitations. Understanding these is essential before committing production workloads.

The serverless life cycle

Serverless functions are spun up to handle requests. They close out soon afterward and lose any state. For that reason, they cannot natively hold open persistent WebSockets. If your system requires continuous, sub-millisecond, bidirectional streams—like a real-time multiplayer action game or a high-frequency trading dashboard—pure serverless will fight you all the way. You are forced to introduce a third-party managed WebSocket service to route messages back to your stateless endpoints via HTTP webhooks.

The execution time ceiling

Vercel (like all serverless platforms) enforces strict timeouts on operations. While enterprise tiers might grant you up to 15 minutes, standard functions often time out after 10 to 60 seconds. Long-running tasks like video transcoding, database scripts, or orchestrating multi-step AI agent workflows, which might take 20 minutes to resolve, will run up against these limits. Heavy-lifting tasks must be offloaded to a dedicated, long-running service like Google Cloud Run, or broken into smaller, asynchronous chunks via message queues.

The cold start reality

While the industry has made massive strides in minimizing initialization times—particularly with lightweight edge networks—traditional Node.js-based serverless functions still experience cold starts. If a function has not been invoked recently, or if traffic spikes require a new instance to spin up concurrently, the first request will take a noticeable latency hit as the container provisions and the code loads.

API instead of RAM

In a traditional server environment, you can store transient data in global RAM, allowing subsequent requests to access shared context instantly. In the serverless model, every request might hit a fresh container. Therefore, all shared context must be externalized. Although Firestore serves brilliantly as the state manager, relying on a database for high-frequency, sub-millisecond, ephemeral caching introduces network latency and per-operation costs. That said, using a shared RAM state on a server is non-trivial also, unless you are using a single app server and VM (because high-availability or fail-over requirements will lessen the RAM win on a traditional server).

Tuning for velocity and control

Every architectural decision is a trade-off. There are no cost-free choices. By adopting the GitHub, Vercel, and Firestore stack, you are explicitly maximizing feature velocity over fine-grained control.

You lose the ability to tweak the underlying operating system, hold open persistent sockets, or run hour-long back-end scripts. In exchange, you gain an architecture that scales from zero to global distribution instantly, requires virtually no devops maintenance, and perfectly absorbs the asynchronous, event-driven realities of modern application development.

For the right application—whether it is a fast-moving prototype or an enterprise AI copilot—this stack doesn’t just save time; it fundamentally changes how quickly a small team (or a single person) can impact the market. You stop worrying about build chains, load balancers, and server patches, and you focus on the central mission: shipping features.

This is Part 2 of a series on tech worker unionization. See Part 1: “A brewing battle: More IT workers want unions. The industry doesn’t.”

The best time for tech workers to unionize was 20 years ago, when they had plenty of leverage. The second-best time is now, when they don’t.

Mass layoffs, AI-driven displacement, corporate surveillance, workplace disillusionment have created conditions that have made organizing compelling for tech professionals. But the federal labor board that has historically protected workers’ right to organize has been weakened, and the companies that once feared it are openly defying it.

Here’s how organizers and labor experts describe the pros and cons to organizing — and how you can get started.

What unions can — and can’t — do for you

The single biggest benefit of a union contract for most tech workers isn’t pay — it’s protection against arbitrary termination, especially in the wake of recent mass layoffs in tech. In the United States, nonunion “at-will” workers can be fired at any time without a stated reason, while unionized workers negotiate protections written into their contracts.

“That fear of the company letting you go for anything at any time…with a union they just can’t do that,” says Zak Thompson, a senior software engineer at Kickstarter and union steward at Kickstarter United. Now that Kickstarter employees are unionized, people are less worried that saying something negative will result in termination.

“I’ve been shocked at the willingness of my co-workers to speak up against what they see as poor or controversial business decisions,” Thompson says.

Beyond job security, unions can deliver concrete material gains. Kickstarter United was formed in 2020, although getting there wasn’t easy: two employees were fired during the organizing campaign — which itself became a galvanizing event. And while the union hasn’t been able to prevent layoffs, it did negotiate better terms: four months of severance pay and four to six months of continued health insurance, versus the two to three weeks per year of work that management had initially proposed.

Other benefits include a four-day work week; AI protections; a minimum pay floor; and standards for raises, promotions, and time off for the company’s 59 employees.

Unions can give tech workers a voice in decisions that affect their daily work — including how AI tools are deployed. “Nobody I’ve spoken to is against new technology or getting trained in it,” says Max Belasco, a business systems analyst at the University of California Los Angeles School of Law and co-chair of the UCLA chapter of the University Professional and Technical Employees/Communications Workers of America (UPTE-CWA) Local 9119.

“But when new technology is being implemented, we want to know: what’s the five-year vision, the 10-year vision? Are we implementing this in a way that betters staffing, increases efficiency, or eases the lives of people already working? Or are we trying to take away jobs, automate people out of their pension or paycheck?” Belasco says.

The challenges are real. Tech professionals are less inclined to leave their jobs in the current market because wages haven’t been increasing as fast as they once were, and it can take longer to land another job.

“Tech moved from a very tight labor market in 2022 (1.85% unemployment rate) to a noticeably weaker one in 2024–2026 (3.49%),” although that’s still better than the national unemployment rate of 4.36% through May of this year, says Liya Palagashvili, senior research fellow and director of the Labor Policy Project at the Mercatus Center at George Mason University.

Flexibility is a concern. The more substantive challenge, raised by economists including Palagashvili, is that traditional union contracts impose uniform terms across an entire bargaining unit, limiting the flexibility that many tech workers — and their employers —currently enjoy. Tech firms need to move fast, adjusting teams, products, and roles on the fly.

“Collective bargaining agreements can make those adjustments much more difficult, whether by making them slower, costlier, or inconsistent with the contract,” she says.

Workers skeptical of unions in a survey of 1,900 tech professionals conducted by the career site Blind cited specific concerns: that unions are “not meritocratic,” “prevent innovation,” and “hold back earnings of top performers.”

Thompson from Kickstarter United pushes back: “We have nothing in our contract about ‘you can’t bend down and pick up a piece of trash because that’s someone else’s job.’ The company is free to give bonuses and individual raises as much as they like. This is all just up to the people who are bargaining the contract from the union side.”

Organizing carries potentially serious personal risks. During negotiations for a second three-year contract in 2025, Kickstarter United went on strike for 42 days. A few months later, the company announced layoffs.

“They let go strong union leaders, including a person who had bargained our last contract,” Thompson says. The union appealed, and the issue is now going to arbitration.

If you form a union, don’t expect much support from the National Labor Relations Board, the agency that certifies US labor unions and protects workers’ right to organize, in terms of prosecuting complaints of unfair labor practices, Thompson warns. “We’re in a political moment in this country with a pretty weakened NLRB. You have to be ready to organize and withhold worker power without any guarantee of safety.”

Organizers are up against an enormous union avoidance industry. Organizers can expect fierce pushback as soon as the business discovers that organizing is underway.

“There’s a multi-billion-dollar industry in union avoidance,” says Alan McAvinney, a Google software engineer and organizing chair, Alphabet Workers Union-CWA, a 1,400-member minority union of Alphabet employees. (Google is a subsidiary of Alphabet.)

US employers spend roughly $1.7 billion a year on union avoidance consultants and law firms, according to a May 2026 report by the Economic Policy Institute and LaborLab.

Expect hardball tactics. Management may play hardball during the time between when organizers announce their intention to unionize and the actual vote. For example, management can threaten to fire foreign-born workers in the US on H-1B visas if they support the union. Those workers would then have just 60 days to find a new sponsoring employer or lose their H-1B status, according to a recent Tech Workers Coalition blog post.

And at venture capital-backed startups, investment agreements sometimes require management to attest there is no union activity — meaning a public organizing drive can trigger funding withdrawal. Or, if a unionized company is acquired, the new management can dissolve the union overnight by reclassifying unionized workers as new hires.

With these sobering facts in mind, here is how organizers who have done it describe the process of creating a union.

Step 1: Start a conversation with your co-workers

At the University of California, a two-tier system had evolved where some tech workers were unionized and some weren’t, Belasco says. Management created new titles that fell outside the union even though they had similar job descriptions and responsibilities to those in the union. Those nonunion employees received lower pay and benefits than their unionized peers, which created resentment and instability.

Belasco and other organizers wanted to eliminate that division by bringing everyone under the same contract. But when they began their unionization drive, “the biggest barrier we faced wasn’t management opposition — it was that people felt this was just the best-case scenario realistically available: ‘We have this job at the university, we have concerns about automation and layoffs, but what can we really do about it?'” he says.

The antidote to that fatalism, organizers say, is simple: “Just start talking to your immediate co-workers. Are they experiencing the same challenges you are experiencing?” says McAvinney. “There’s no need to start talking about a union at this point.”

Just get a consensus and start building a group of like-minded individuals, Thompson advises. “Always start with one-on-one conversations, and that’s what you should do the whole time. That’s the key to organizing,” he says.

Tech workers often think they’re a special case, says Thompson, and therefore that unionization isn’t a good fit. “You’re not special. You are a company of workers, you are organizing, and there is a playbook for that. Trust the process, because it tends to work pretty well,” he says.

Step 2: Who’s on board, and who’s not? Map your workplace, but keep it quiet

Once there’s a consensus, continue to grow your network. Keep a list of everyone you’ve spoken with and note their disposition: “Is this person union-friendly or anti-union? Would they be a strong organizer?” Thompson says.

Maintaining secrecy early on is essential, because anti-union tactics will start immediately, and that can stop union organizing before it can gain momentum.

“Generally, employers do not want to share power with their workforce,” McAvinney says. Employers will deploy every means at their disposal to stop organizing efforts and peel away potential yes votes.

“If you look at historical examples, having 70% approval before the employer finds out about you results in a high percentage of wins when you actually cast the vote. Historically, that’s an effective buffer,” he says.

There’s a real threat of firing and layoffs. The traditional tech worker belief that job mobility makes collective action unnecessary is now being tested by a tighter job market, McAvinney says, noting that workers who many believe were fired for speaking out back in 2019 were a galvanizing factor in his union’s formation.

“You generally don’t want to be in a situation where the employer feels comfortable firing everyone. Part of that is thinking from a cynical standpoint about what the consequences would be to the employer if they did fire everyone,” he says.

One-on-one conversations that include personally asking co-workers to keep conversations confidential are key to keeping things quiet, Belasco says. When 2,100 UC tech workers voted to unionize in May, 96% voted in favor. To stay out of earshot of managers, avoid employee surveillance tools, and sidestep conference calls that could be recorded, organizers met with workers in their homes.

“That tactic is probably what made the difference between winning the election and getting the majority we got,” he says.

Step 3: Find the right union affiliation or go it alone

“Running a campaign against major employers requires the resources and expertise of the larger labor movement, even if workers publicly present as independent,” says Kate Bronfenbrenner, director of labor education research and senior lecturer emeritus at Cornell University’s School of Industrial and Labor Relations.

Options include the Communications Workers of America (CWA), Service Employees International Union (SEIU), and the Office and Professional Employees International Union (OPEIU), among others. Another resource, the Tech Workers Coalition (TWC), provides training on organizing tactics, AI-in-workplace issues, and contract negotiation, and can match workers to the right unions for their needs.

The Alphabet Workers Union decided early on to affiliate with CWA. “They gave us a bunch of support early on in our campaign with no strings attached,” McAvinney says.

Kickstarter is organized through OPEIU, Thompson says. “They’ll usually have resources and staff that can help you through the next steps: collecting signatures in support of a union, bringing that to management, holding a vote — the more formalized things that interact with US labor law. They’ll also help with organizing along the way,” he says.

For workers at institutions where a union already exists, there may be a faster path. Organizers at UCLA did what’s called a “unit modification,” aligning with UPTE. By organizing under UPTE, the workers didn’t have to negotiate a new contract from scratch — they joined an already-negotiated contract covering existing UPTE tech members, which put them in “a much stronger position” than starting fresh, Belasco says.

Step 4: Choose your union model: majority vs. pre-majority or minority

Assess what’s practical for your organizing effort. In a majority union, more than 50% of all workers in a defined bargaining unit must vote to join the union through an NLRB-supervised election in the private sector, or a Public Employment Relations Board (PERB)-supervised election for public sector workers.

The NLRB must certify the union, which then operates under its legal protections. This means, for example, that the employer must bargain, negotiated contracts are enforceable, violations must go to the NLRB or arbitration, and workers can’t be dismissed without just cause.

A pre-majority or minority union is a minority labor organization operating without NLRB protections or collective bargaining agreements. “Pre-majority means that workers are able to demonstrate majority support — through signed cards, petitions, a walkout, or everyone wearing solidarity T-shirts — without going through a formal election,” Bronfenbrenner says.

The Alphabet Workers Union-CWA (AWU-CWA) formed as a pre-majority union because achieving majority status across a globally distributed workforce of over 100,000 was not a realistic near-term goal. “An underground model where you try to reach 70% support across a workforce of over 100,000 people isn’t realistic,” McAvinney says.

A pre-majority union can still make a difference, he says. For example, the Alphabet Workers Union-CWA convinced management to offer voluntary exit packages — buyouts — prior to announcing layoffs.

For smaller organizations, a majority union may be the more practical option — it’s more attainable, McAvinney says. “I don’t think [the pre-majority union model] is the correct thing to do in all situations. I certainly would not recommend it to a 200-person shop.”

Kickstarter, which had fewer than 100 employees, was able to form a majority union, with 55% voting to organize.

Ultimately, says McAvinney, “there’s no inflection point where you go from being able to win nothing to winning everything, even with a contract and a supermajority. But the more people you have who are willing and able to fight for what they want, the more you’ll be able to get.”

Step 5: Who should — and should not — be in your union?

Belasco’s situation at UCLA illustrates a broader strategic choice that every organizing campaign must make. He had been in a union position in educational technology when he was told his role would be reclassified as a non-union position.

“I was given a choice: apply to the new non-union position to continue doing the work I’d trained for, or stay in my union position doing service desk work I wasn’t used to,” he says. “Essentially, it was a choice between job security and career progression.”

Belasco joined a “wall-to-wall” union, which represents a broad range of university professional and technical employees across the UC system rather than a single job category, such as engineers or tech professionals.

Kickstarter United is another example of a wall-to-wall union. “It’s not just the engineers who are unionized, but also customer support, designers — everyone,” Thompson says.

Wall-to-wall unions are more powerful, but they’re also more difficult to achieve. Under US labor law, “professionals have to vote separately on whether they want to be combined with other workers,” says Bronfenbrenner. “You can never have a wall-to-wall unit without giving professionals the chance to decide whether they want to be separate.”

The law’s “professional employees” category includes roles like software engineers and developers but not necessarily others. For example, customer support specialists and QA analysts would fall into the “non-professional workers” category.

“For decades, the pattern was either to organize everybody except the engineers, or manage to organize the engineers and fail to bring in everybody else — neither of which builds real worker power,” says Simone Robutti, an organizer with Tech Workers Coalition Global, an international branch of TWC based in Berlin.

Step 6: You won the vote. Get ready for what comes next

Winning a union vote means having a seat at the table, says Thompson. “Once the workers have come together and agreed they want that seat, you bring that to management, and they have a chance to voluntarily recognize a union,” he says.

But in most cases employers contest the results, which must be certified by the NLRB or PERB. That process, in which the employer uses various tactics to challenge the legitimacy of the outcome, can take weeks or months.

Unfortunately, the legal framework that is supposed to protect workers during this process has been significantly weakened in the last few years. In a potentially more ominous development, SpaceX, Amazon, Trader Joe’s, Starbucks, and the University of Southern California have in separate legal actions challenged the constitutionality of the NLRB, arguing that the agency’s structure violates the separation of powers. The Fifth Circuit Court of Appeals upheld injunctions against the NLRB in SpaceX’s case in August 2025 — a serious challenge to the agency’s authority.

In the meantime, some companies may disregard negotiated contracts, which can lead to lengthy legal appeals or extended arbitration.

“The NLRB can still force an election, but it can’t force a contract, and companies are saying they simply won’t comply,” Bronfenbrenner says. This is where the expertise and resources of affiliation with a major union can help, she adds.

As a result, contract negotiations can take far longer than workers might expect. At Kickstarter, for example, two years and four months elapsed from the time of the union vote to the first contract, and that was at a 59-person company with a relatively cooperative employer. At larger companies with more aggressive legal teams, the timeline will be longer.

Forming a union is hard work, Robutti says. “It’s not a service you pay for and they protect you. It doesn’t happen spontaneously, and it doesn’t happen magically. It’s the choice to take responsibility for improving your workplace.”

Gen AI spending is moving beyond the familiar software model of seats, licenses, and pilots. As AI shifts from copilots to embedded workflows and autonomous agents, one user request can trigger multiple model calls, retrieval steps, retries, orchestration layers, and infrastructure events. A tool that looks affordable in pilot may behave very differently once connected to production systems or allowed to act with less human supervision.

According to Michael Corrigan, CIO of World Insurance Associates, AI introduces a fundamentally different cost model — one that’s usage driven, non-linear, and tightly coupled to business activity. “Success requires shifting from traditional IT budgeting to FinOps-style discipline where consumption, value, and governance are actively managed in real time,” he says.

Here are five ways CIOs can build that discipline before AI costs spiral.

Forecast AI by workflow, not by user

At World, a top 25 insurance broker with about 3,000 employees across roughly 300 locations, AI use falls into three broad categories, Corrigan says. One is broad tools, such as copilots. Another is embedded AI inside SaaS platforms. And the third is bespoke AI built around specific workflows and manual processes.

“The bespoke is the area that’s growing the most right now,” he says. “And that’s where the model, from a cost perspective, has really been shifting from a license seat cost to a token consumption or token burn cost, or even a hybrid.”

Seat-based pricing is relatively easy to forecast whereas consumption-based AI isn’t. Costs may depend on prompt complexity, output length, model choice, workflow design, and whether the system calls a model once or many times in the background.

World tries to manage that uncertainty by defining the business problem, success criteria, and expected operational improvement upfront. Pilots help estimate consumption before scaling, but Corrigan says they don’t remove the ambiguity.

“We’ll try our best in the pilot to understand what the consumption rate is, what the token burn rate is,” he says. But once a consumption-based workflow goes into production, he adds, an estimate is put into place. That estimate is informed, but still rough.

Elmer Morales, founder and CEO of koder.com, an agentic AI coding startup, says CIOs should think less about headcount and more about workflow mechanics. Agentic AI costs are driven by the number of decisions an agent makes, how often it retrieves external data, how much context it carries, and how many systems it touches.

“CIOs should start by mapping workflows, not necessarily users,” he says. “The relevant variable isn’t going to be the headcount but how many decisions an agent makes per task.”

Model the failure path, not just the happy path

Pilots can mislead because they often test the cleanest version of an AI workflow. Morales says many enterprises model agentic AI costs around the happy path: the user gives a clear prompt, the system understands the request, the agent completes the task, and the process ends. Production is messier.

“They generally don’t model for situations where the agent is going to need to go back and check its work and redo things,” Morales says. “A lot of times, agents are wrong, either because they hallucinate or they understood the problem incorrectly.”

In an agentic workflow, the system may check its work, call another tool, retrieve more data, or redo a step. While that may improve quality, it also adds cost.

The difference between copilots and agents is central. A copilot interaction is often one prompt and one response. An agentic workflow may involve agents moving through a decision tree, executing tasks in sequence or in parallel, and calling sub-agents or external systems along the way. “By the time it’s achieved the original goal, the agent might have made 50 or 100 model calls, compared with a single call for a traditional copilot prompt,” Morales says.

That’s why CIOs should require teams to model the failure path before production, like how many retries are allowed, how much context is resent, which tools can be called, when a human should intervene, and what happens when the agent can’t complete the task.

Build cost controls into the architecture

Traditional FinOps practices still matter, but AI requires more than retrospective dashboards and chargebacks.

According to Pavan Madduri, senior cloud platform engineer at industrial supply company Graigner, looking backward at usage data, as traditional FinOps often does, can be too late. Costs are shaped by prompt design, model selection, agent behavior, orchestration choices, and runtime loops.

“Dashboards or chargebacks, those are historical accounting,” he says. “The money’s already gone.” For AI, he argues, cost controls need to be embedded into the architecture. That includes hard token caps, retry-depth limits, maximum runtime limits, workload prioritization, background-job throttling, and cluster-level controls that prevent runaway consumption.

“The real FinOps means you need to have the cost constraints embedded into your architecture framework,” Madduri says.

Those controls also extend to infrastructure. Expensive GPUs may sit warm between jobs because systems need capacity available when inference demand arrives. Teams may pass huge schemas, databases, or thousands of lines of code into frontier models when a smaller or more focused prompt would do.

Enterprises should also adopt event-driven autoscaling, Madduri says. “Use tools like KEDA to scale GPU nodes down to zero the moment inference demand drops, so teams only pay for the windows when the silicon is actively crunching tokens.”

Corrigan says World uses rate limits, spend limits, alerts, and approval gateways for consumption-based tools. When users approach token consumption limits, automated alerts allow IT and the business to review whether the continued spend is justified.

“If it’s not meeting the success criteria we expected, you have to have the control in place to say we’re going to move on or kill that process,” Corrigan says.

Route work to the right model

CIOs can also reduce AI bill shock by avoiding a default assumption that every task requires the most powerful model available. While some tasks need advanced reasoning, many others don’t. A simple support ticket, log-parsing task, or structured database transaction may be handled by a smaller or cheaper model. A complex architecture decision, legal analysis, or multi-step reasoning task may justify a more powerful one.

“Choosing the right model for the right prompt and right question — that’s where you leverage the maximum from that model, and you can decrease the costing,” Madduri says. “If you default every single call to a frontier model, that’s architectural laziness.”

Morales makes a similar point. Not every step in an agentic workflow requires a top-of-the-line model. Model routing, he says, is the discipline of determining the best model for the task, and providing the relevant context when the model needs it.

According to Jim Olsen, CTO of enterprise software company ModelOp, CIOs should use the least expensive model that can accomplish the business goal. Using the biggest model for everything is easier, but expensive. “It’s like hiring the most expensive engineer to change a few colors in a website’s CSS, or visual styling,” he says. “You wouldn’t do that. You use the appropriate tools for the task.”

Tie consumption to business value

For Olsen, the deeper enterprise problem is AI value shock, not just bill shock. Spending $200,000 in a quarter on AI is justified if it produces $2 million in business value. The problem is spending heavily on use cases that don’t generate a meaningful return.

“Are you actually getting that return on investment, or are you just blowing tokens for something that’s not delivering the value to your business?” Olsen asks. Tracking token usage by user or department may show who consumed AI, but not whether the consumption mattered.

For most enterprise AI systems, Olsen says costs should be tied back to business use cases. A model may be used for HR document search, customer support, code review, problem resolution, or other functions. Each use case may draw on the same underlying models or agents, but the business value can be very different.

That’s why he argues that companies need an AI inventory, a record of which business workflows use which models, agents, providers, workflows, and systems. Without that inventory, enterprises can’t connect consumption to value.

Corrigan takes a similar approach from a governance perspective. At World, new AI ideas go through an intake process. Business users propose improvements, and IT, finance, operations, sales, and business stakeholders evaluate, prioritize, and monitor them from pilot through production.

That may be where the next stage of AI FinOps is heading, toward a clearer understanding of which AI consumption deserves to scale, not just to lower bills. So the question, as Olsen puts it, isn’t whether someone used a million tokens. It’s what are they using them for.

If you spend enough time in Silicon Valley AI circles, you’ll hear the same message over and over again: AI needs context.

The statement is broadly true. The problem is that “context” has become one of the least precise terms in the industry.

Depending on who is using it, context can mean documents, dashboards, reports, metadata, business rules, policies, transaction histories, CRM records, knowledge bases or institutional expertise. The word has become a catch-all for virtually any information that might be made available to a model.

As a result, many organizations have started treating context as a volume problem. Conversations quickly turn to larger context windows, additional data sources and broader system access, while far less attention goes toward determining whether that information actually improves the quality of the outcome.

What we’re seeing in practice suggests a different way of thinking about the problem. The organizations making the most progress with enterprise AI are not necessarily the ones exposing the largest amount of information to their systems. They are the ones spending the most time understanding which information should influence a decision, which information should not and how to ensure that business logic is applied consistently.

That distinction matters because the industry is beginning to repeat a mistake enterprises already made once before.

Context has become the new ‘big data’

For much of the last two decades, organizations operated under the assumption that collecting more data would naturally produce better decisions. Massive investments were made in data warehouses, reporting platforms, analytics systems and business intelligence tools. Those investments created tremendous value, but they also exposed an important reality: Collecting information and creating clarity are not the same thing.

Today, AI is heading down a similar path.

Many enterprise AI projects measure progress by counting how much information a model can access. More documents become better than fewer documents. More systems become better than fewer systems. Larger context windows become better than smaller ones. The conversation often assumes that quantity and quality move together.

Well, they don’t.

According to Salesforce research, only 35% of business leaders say they are completely satisfied with their organization’s ability to use data effectively despite years of investment in data infrastructure and analytics. Enterprises learned long ago that information alone does not create understanding. The same lesson applies to AI.

When a model gains access to five versions of the same metric, conflicting definitions of a business process or documentation that has not been updated in years, it does not magically resolve those inconsistencies. It consumes them. More context can just as easily increase ambiguity as reduce it.

Simply exposing more information to a model does not guarantee better outcomes. What matters is whether the information available to the system helps it make the right decision at the right time.

Most AI failures are actually context failures

One of the more interesting things we’ve observed over the past year is how many AI projects are blamed for problems that have very little to do with AI.

The model answers a question incorrectly, and the immediate assumption is that the model failed. In reality, the underlying issue often sits elsewhere. The organization may have multiple definitions of the metric being requested. Customer information may exist across several systems with conflicting values. Business rules may be documented in one location, partially implemented in another and understood differently by different teams.

In many deployments, the issue is not that the AI lacks information. The issue is that it has access to several competing versions of the truth.

Anyone who has worked inside a large enterprise will recognize the pattern. Revenue means one thing to finance and something slightly different to sales. Product usage metrics evolve over time. Operational processes change while documentation remains frozen. Human employees learn how to navigate these inconsistencies through experience and institutional knowledge. AI systems inherit them immediately.

This is why the conversation around context often misses the point. The challenge is not simply providing more information. The challenge is determining which information should be trusted, how conflicts should be resolved and what business logic should govern the final answer.

A single trusted source can be more valuable than a hundred loosely connected ones. A clearly defined rule can be more useful than thousands of pages of documentation. The quality of the context matters far more than the volume.

Access does not create trust

Many organizations can tell you exactly how their AI systems retrieve information. They can explain retrieval pipelines, vector databases, ranking systems, semantic search architectures and context windows in extraordinary detail.

Far fewer can explain how they determine whether the answers produced are consistently correct.

That gap becomes especially important in enterprise environments where the cost of an incorrect answer can be substantial. A sales leader making a forecast, a finance team evaluating performance or an operations executive making a resource allocation decision does not care how many documents were retrieved. They care whether the answer is right.

Trust has always been one of the hardest problems in enterprise data. According to Accenture research on data trust and decision making, only about a quarter of employees report high confidence in their organization’s data when making decisions. That challenge does not disappear when AI enters the picture. If anything, it becomes more visible.

Organizations frequently measure access because access is easy to quantify. Reliability is harder. Reliability requires understanding whether an answer remains consistent across users, across prompts, across time periods and across changing business conditions. It requires understanding whether the same question produces the same answer and whether that answer reflects the business logic the organization intends to enforce.

Those are fundamentally different measurements, and they point to a different definition of success.

Context requires measurement

One reason this problem is becoming more pronounced is that enterprises accumulate information far faster than they eliminate it.

New systems are added, new reports are created, processes evolve. Teams develop local definitions and specialized workflows. Documentation grows continuously, while very little of it gets removed. Over time, organizations build large collections of information that contain years of historical decisions, exceptions, workarounds and competing interpretations.

We’ve yet to encounter an enterprise that doesn’t have some version of this problem.

That reality turns context into an operational challenge rather than a technical one.

Simply connecting AI systems to enterprise information does not improve the quality of that information. In some cases, it exposes longstanding inconsistencies that were previously hidden by human interpretation and tribal knowledge. Gartner has long identified poor data quality as one of the most significant obstacles to successful analytics and AI initiatives because bad inputs inevitably produce unreliable outputs, regardless of how sophisticated the technology becomes.

As AI becomes more deeply integrated into business operations, organizations will need new ways to evaluate the context their systems rely on. They will need visibility into how information is being used, where definitions conflict, which sources are trusted and how context quality affects outcomes. Context cannot be treated as a static asset. It must be measured, monitored and improved over time, just as organizations measure the quality of the models and applications built on top of it.

The shift from access to reliability

The industry has spent the last several years focused on access. How do we connect models to enterprise systems? How do we expose organizational knowledge? How do we give AI visibility into the information people use every day?

Those questions were important because they represented genuine technical barriers. Today, many of those barriers are disappearing.

Most enterprises can already connect AI systems to data warehouses, applications, dashboards, documents and knowledge repositories. The conversation is beginning to shift toward a more difficult problem: Determining whether those connections actually produce outcomes people trust.

That is where the next phase of enterprise AI will be decided.

Organizations that treat context as a quantity problem will continue adding more information and hoping accuracy improves. Organizations that treat context as a quality problem will focus on trust, consistency, governance and outcome reliability.

The difference between those approaches may sound subtle, but it has enormous implications. One produces systems that can access information. The other produces systems that people are willing to use to make decisions.

And in the enterprise, that distinction is ultimately what matters.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Security engineers play a pivotal role in enterprise cybersecurity, because they are the professionals who design, build, and deploy security systems to protect an organization’s data, applications, systems, networks, and other IT components against a variety of cyber threats.

Finding not just qualified security engineers, but the best and brightest available, needs to be a priority for CISOs and others overseeing security at their organizations. That’s especially true with the rapid rise of AI and the threats that brings to the enterprise.

Here are some of the key skills and traits of elite security engineers to look for when hiring — or to acquire in order to uplevel your cybersecurity career.

Acumen with AI-powered tools

These days, AI-related skills are in demand regardless of domain, and this certainly applies to security engineers. There’s a wealth of solutions leveraging AI in the market, tools that engineers can add to their defense arsenal.

“AI is transforming security engineering from reactive alerting to predictive threat detection,” says Praveen Margabandhu, digital engineering anchor at financial services firm Navy Federal Credit Union. “AI-driven anomaly detection now identifies behavioral patterns that indicate fraud or compromise before traditional threshold-based systems would fire. This shifts the security engineer’s role from incident responder to threat model designer.”

AI-powered tools have taken over a large portion of the detection and triage work that used to be the core of a security engineer’s day, says Maruf Ahmed, cofounder and CEO of global tech staffing firm Dexian. “Vulnerability scanning runs on its own now,” he says. “Threat flagging that used to require a team pulling through logs for hours happens in minutes.”

This has freed up capacity on most security teams and changed what the day-to-day work looks like, Ahmed says. “With detection increasingly automated, the engineer’s value sits more in interpreting what gets flagged and deciding what to do about it,” he says.

Keen understanding of emerging and established AI threats

Engineers must also have a thorough understanding of the risks AI presents, including AI-enhanced cyberattacks using large language models (LLMs) to automate and scale highly personalized social engineering attacks, craft sophisticated malware, and generate deepfakes.

Other AI threats they need to be aware of include prompt injections, data and model poisoning, disclosure of sensitive information, model theft, supply chain compromises, and excessive agency.

“The same generative tools that help security teams work faster are available to adversaries, and it shows,” Ahmed says. “Phishing campaigns read better and land more precisely than they did a year ago. Social engineering is harder to catch when the language is polished and tailored to the target, and security engineers are now defending against threats built with the same class of technology they use on the defensive side.”

That has raised the bar for what reliable detection looks like, Ahmed says. “The objective shift I hear most from clients is about trust in their own systems,” he says. “Two years ago, the priority was visibility — making sure you could see across your environment. Most organizations have that now. The harder problem is knowing whether what those tools are telling you holds up under scrutiny and having people on the team who can stand behind those findings in front of a regulator or a board.”

Appreciation of performance and business goals

The best security engineers understand how performance and security intersect, says Margabandhu, who leads performance engineering across Navy Federal Credit Union’s digital banking infrastructure, including real-time fraud detection, identity and access management, and cybersecurity infrastructure resilience.

“A fraud detection system that is secure but too slow to catch transactions in real-time is not secure at all,” Margabandhu says. “Elite engineers optimize for both simultaneously.”

Engineers must be able to put things in business context, Ahmed says. “An engineer who can work across domains, validate AI outputs, and learn new tools fast is valuable. But that value compounds when the person also understands what the organization is trying to protect and why,” he says.

Security engineers who understand the business make better risk decisions, write more effective policies, and generate less friction with the teams around them, Ahmed says. “That is the profile employers are hiring toward right now, and it is where the talent shortage is most pronounced,” he says.

Systems mindset

“One of the biggest misconceptions in cybersecurity hiring is that elite security engineers are defined purely by technical certifications or tool familiarity,” says Juan Mathews Rebello Santos, an independent cybersecurity researcher and ethical hacker.

“Technical skill absolutely matters, but the strongest engineers I’ve worked with consistently share a combination of analytical thinking, operational adaptability, communication ability, and deep systems understanding,” Santos says.

Elite security engineers understand how infrastructure, cloud services, identity systems, applications, APIs, networks, users, and business operations connect, Santos says.

“Modern attacks rarely target a single isolated component anymore,” he says. “Threat actors chain together weaknesses across environments. Engineers who can understand those relationships holistically are significantly more effective at both prevention and incident response.”

Cross-disciplinary fluency and broad stack know-how

Being an elite software engineer today means having a range of technology experience and knowledge. “Organizations want engineers who can work across more of the stack than they used to,” Ahmed says. “A role that might have asked for deep specialization in one area now expects someone who can move between cloud infrastructure, application security, and compliance without needing a handoff at every boundary.”

The attack surface has continued to get wider, and the job descriptions for security engineers has followed suit. “That cross-domain fluency matters because security incidents rarely stay contained in one layer,” Ahmed says. “The engineer who can follow a problem from the network through the application to the data governance framework resolves it faster, with fewer people involved.”

The strongest security engineers bridge infrastructure, application, and business domains, Margabandhu says. “They can speak to a CISO, a developer, and a cloud architect in the same conversation,” he says. “An engineer who can explain what an authentication problem means for fraud exposure moves faster in a room full of executives than one who can only describe it in infrastructure terms. I’ve watched technically brilliant people lose that race repeatedly.”

Having the ability to communicate technical risk clearly to non-technical leadership can mean the difference between success and failure of attacks.

“Many security failures today are not caused by lack of tooling, but by misalignment between technical teams and business decision-makers,” Santos says. “Elite engineers can explain operational risk, prioritization, and security tradeoffs in language executives understand.”

Deep understanding of third-party risk and non-human threats

Threats can come from anywhere, including supply chains and non-human combatants. Third-party cybersecurity risks are on the rise. The 2026 Global CISO Leadership Report by executive search firm Hitch Partners, based on a survey of more than 625 information security executives across the US and Canada, says 43% put third-party risks as the No. 1 priority.

“Most teams are still better at securing what they own than securing what they depend on,” Margabandhu says. “The mental shift from perimeter thinking to dependency thinking is real and not everyone has made it. The engineers who treat every API call, every credentialed vendor, every third-party model as part of their attack surface approach design differently.”

Another growing source of potential threats are not human. Machine identities now outnumber human identities by ratios exceeding 100 to 1 in most enterprise environments, with some sectors closer to 500 to 1, according to the ManageEngine Identity Security Outlook 2026 report.

This includes service accounts, API keys, automation tokens, and AI agents, any one of which can present data governance and security risks.

Many organizations are still managing machine identities through manual processes that weren’t designed for scale, Margabandhu says. “Engineers who understand non-human identity governance are rare and increasingly important. This is not a future problem.”

Willingness to keep learning

Security engineers need to have a desire to never stopped learning.

“That sounds obvious until you work with people who’ve been doing this for 15 years and are still operating from the same threat models they built in 2012,” Margabandhu says. “Security changes fast enough that standing still is the same as going backwards.”

The security engineers who keep up aren’t reading one report a year. “They’re genuinely curious about what attackers are doing right now, this month, and they adjust how they think accordingly,” Margabandhu says. “That quality is harder to hire for than most technical skills, because it’s not on a resume.”

With AI presenting new and more sophisticated threats, keeping up with the latest developments is perhaps more important than ever. “Strong engineers are naturally investigative,” Santos says. “They actively study attack techniques, test assumptions, reverse engineer failures, and continuously adapt their understanding of risk.”

The best security engineers are often the people who remain intellectually uncomfortable because they know the landscape is always evolving, Santos says.

Employers have started paying closer attention to how fast someone can learn, Ahmed says. “The threat landscape and the defensive toolkit are both moving faster than any certification program can track, so hiring managers are probing for adaptability in interviews: how candidates have responded to recent shifts, whether they have picked up unfamiliar platforms on their own, how they work through problems they have not seen before,” he says.