The post PentesterFlow – AI Tool for Penetration Testers and Bug Hunters to Automate Workflows appeared first on Cyber Security News.

Read more of this story at Slashdot.

Read more of this story at Slashdot.

The issue is straightforward: when Node.js resolves modules, the runtime searches for packages in C:\node_modules as part of its default behavior. Since low-privileged Windows users can create this directory and plant malicious modules there, any Node.js application with missing or optional dependencies becomes vulnerable to privilege escalation.

This issue is not new. Concerned discussions about Node.js's module search path behavior date back to 2013 and 2014.

Node.js has explicitly stated that they consider this behavior intentional:

"Node.js trusts the file system."

They do not treat CWE-427 (Uncontrolled Search Path Element) as a vulnerability, pushing responsibility onto application developers.

As the case studies below demonstrate, this stance has dangerous consequences. Developers are largely unaware of this attack surface, and the result is a proliferation of exploitable applications. We will show examples in npm CLI and Discord, but there are likely many more applications that are impacted by this.

Root Cause

The root cause lies in the way Node.js performs module resolution. This is documented here. Although UNIX paths are used in the documentation provided by Node.js, the same logic is applied on Windows.

When a Node.js application calls require(‘bar’), the runtime searches for the module in the following order:

1. C:\Users\Administrator\projects\node_modules\bar.js
2. C:\Users\Administrator\node_modules\bar.js
3. C:\Users\node_modules\bar.js
4. C:\node_modules\bar.js <-- The problem

If the legitimate package is missing, whether due to optional dependencies, development packages removed in production, or installation failures, the resolution search will eventually reach the root of the drive. Any user can create C:\node_modules and place a malicious package there. Once the low-privileged user has populated C:\node_modules\bar.js, Node.js will load and execute it in the context of the current user. In the following case studies, we will provide evidence of how, despite properly following NPM’s guidelines, third-party dependencies end up triggering this vulnerability anytime you launch the application.

Case Studies: Real-World Manifestations

The Optional Dependency Pattern: npm supports optional dependencies to be specified in the project’s package.json file. The recommended pattern for checking for these dependencies is as follows:

This pattern silently catches errors when optional packages are missing, allowing execution to continue. So what’s the problem? On Windows, Node.js will search all the way up to C:\node_modules where an attacker may have planted a malicious replacement. This search behavior mirrors UNIX conventions where /node_modules at the filesystem root is typically only writable by root. Windows systems by default allow any user to create C:\node_modules. Once require is called, Node.js will traverse the search path and execute any matching module it finds.

Important things to note:

1. This pattern can be found in third party libraries deep in a dependency tree, as we will see in the following examples.
2. There is no runtime indication to either the developers or the end users that such a vulnerability exists without looking at the filesystem logs with Procmon.
3. The optional dependency pattern itself would not be dangerous if Node.js did not search for packages in C:\node_modules.

Let’s take a deeper look at both cases and see why this is so dangerous.

Case 1: npm CLI (ZDI-26-043 / ZDI-CAN-25430 / CVE-2026-0775).

Prior to version 11.2.0, npm CLI used a library called “promise-inflight”, which contained an optional dependency on a package called “bluebird”.

When Node.js is installed on the system, npm is included by default without the bluebird package. This vulnerability was introduced when bluebird was removed through a well-intentioned pull request (https://github.com/npm/cli/pull/1438/changes), demonstrating how easy it is for developers to unknowingly create this attack surface.

We can see Node’s package resolution logic at work in the screenshot below:

First, the application looks for the bluebird.js package in the Node.js installation directory. Node.js sequentially searches back to the system root until it finds the package. If an attacker has placed C:\node_modules\bluebird.js, the require call will find, read, and execute the malicious payload in the context of any user running npm on the system.

This vulnerability is especially dangerous because it is triggered when many npm * cli commands are used. Common development commands such as npm install, npm –l, and npm prune will all execute the malicious bluebird.jspackage.

Case 2: Discord (ZDI-26-040/ ZDI-CAN-27057 / CVE-2026-0776/ UNPATCHED)

On April 22, 2025, ZDI received a report for a similar vulnerability in Discord reported by T. Doğa Gelişli. Discord uses the ws WebSocket library, which contains an optional dependency on utf-8-validate for compatibility with older Node.js versions:

Discord does not ship with the utf-8-validate package. As a result, the following Procmon logs show the same behavior as Case 1. Anytime Discord is launched, the attacker controlled C:\node_modules\utf-8-validate.js is executed.

The ws library does support disabling this check via the WS_NO_UTF_8_VALIDATE environment variable, but this requires the consuming application (Discord) to set it explicitly. Here’s a quick video demonstrating the bug by popping the calc app when opening Discord:

Discord automatically opens on login by default, so in practice code execution happens immediately without any user interaction. Strangely, the Discord Security team made it clear to us in their responses that they do not consider local attack vectors as valid security issues.

The Bigger Picture

The cases above represent only a few of the applications affected by this pattern. During our investigation we found many other independent reports.  These issues in Mongo DB Compass and Mongo DB Shell are just two other examples.

Every Windows application built on Node.js with missing or optional dependencies is potentially vulnerable. This includes desktop applications that utilize Electron as well as popular web frameworks such as Next.js and React.

Each vendor has clearly stated that they will not treat these issues as vulnerabilities:

NPM’s response to our report:

“exploits that require local access to a machine are considered ineligible for npm CLI

Discord’s response to our report:

“We do not consider physical/local attacks as valid security issues”

Node.js, in the “Examples of non-vulnerabilities” section of their Security Policy:

“Node.js trusts the file system in the environment accessible to it. Therefore, it is not a vulnerability if it accesses/loads files from any path that is accessible to it.”

Conclusion

The vulnerability pattern described in this blog stems from a deliberate design decision by Node.js maintainers. While Node.js's position that “applications should trust their filesystem” may hold true on properly administered UNIX systems, it creates a systemic vulnerability on Windows where low-privileged users can write to C:\node_modules. Without a fix from Node.js, the burden silently falls on application developers.

Making matters worse, the vulnerable code may not live in the application code itself. The optional dependencies that trigger this behavior could come from third-party libraries buried in the dependency tree as we saw with both Discord and npm CLI.

We encourage security researchers to further review this issue and investigate other applications for this dangerous behavior. You can find us online at @bobbygould5 and @izobashi, and follow the team on Twitter, Mastodon, LinkedIn, or Bluesky for the latest in exploit techniques and security patches.

DISCLOSURE TIMELINES

NPM CLI:

2024-11-13 – ZDI submitted the report to the vendor

2024-11-13 – The vendor acknowledged the receipt of the report

2024-11-13 – The vendor communicated that the reported behavior was by design and they do not consider local attacks as valid security issues

2025-08-05 – ZDI encouraged the vendor to re-assess the issue

2025-12-18 – ZDI notified the vendor of the intention to publish the case as a 0-day advisory

DISCORD:

2025-07-08 – ZDI notified vendor

2025-09-11 – ZDI followed up with vendor

2025-09-15 – Vendor stated they do not consider local attacks as valid security issues

2025-12-01 – ZDI explained why we believe the issue is still valid

2025-12-10 – Vendor replied that the vulnerability is still out of scope 

2025-12-11 – ZDI informed vendor of intent to publish 0-day 

REFERENCES

https://nodejs.org/api/modules.html#loading-from-node_modules-folders

https://docs.npmjs.com/cli/v10/configuring-npm/package-json#optionaldependencies

https://groups.google.com/g/nodejs/c/5BGr5dliUIk/m/abJEH3sPymcJ?pli=1

https://github.com/nodejs/node-v0.x-archive/issues/8830

https://bounty.github.com/ineligible.html#vulnerability_in_upstream_dependencies:~:text=eligible%20for%20rewards.-,Local%20access,-Vulnerabilities%20which%20require

https://github.com/nodejs/node/security/policy#examples-of-non-vulnerabilities

Powerful AI models capable of devastating new cyber attacks on governments and businesses are mere months away, intelligence agencies for the Five Eyes have warned in a rare joint statement, urging leaders to “act now”.

The surprising public intervention by signals agencies for Australia, the US, the UK, New Zealand and Canada comes after the Trump administration earlier this month decided to block “foreign nationals” from using a much-hyped AI model built by tech company Anthropic, called Fable.

While app performance is often equated with a smooth UI and fast start times, memory serves as the silent foundation upon which these visible metrics are built. It's no secret that we're seeing a shift where device memory is more important than ever. Not only have we made strides in Android memory optimizations with Android 17, we're providing the tooling and API support to help you stay ahead of stricter memory requirements later this year.

To ensure device stability, starting in Android 17, the system will begin enforcing app memory limits based on the device's total RAM. If an app exceeds those limits, Android will kill the process with no associated stack trace.

Optimize image loading

Detect and fix memory leaks with Android Studio

Memory leaks in Android occur when your code holds onto an object's reference long after its lifecycle has ended. This prevents the Garbage Collector (GC) from reclaiming that memory, eventually leading to sluggish performance or OutOfMemoryError (OOM).

Android Studio Panda 3 features a dedicated LeakCanary profiler task, allowing developers to analyze real-time memory leaks and map traces within the IDE.

The LeakCanary profiler task in Android Studio actively moves the memory leak analysis from your device to your development machine, resulting in a significant performance boost during the leak analysis phase as compared to on-device leak analysis.

Additionally, the leak analysis is now contextualized within the IDE and fully integrated with your source code, providing features like go to declaration and other helpful code connections that drastically reduce the friction and time required to investigate and fix memory leaks.

Trim memory when app leaves visible state

Android can reclaim memory from your app or stop your app entirely if necessary to free up memory for critical tasks, as explained in Overview of memory management. Android will usually reclaim memory from your app when it’s not visible to the user, such as by discarding some of your app’s code and data pages in memory or compressing your heap allocations. When the user resumes your app and your app tries to access some memory that’s been reclaimed, the OS will swap that memory back in on demand. This swapping behavior can be slow, and cause unexpected jank or stutters in your app.

If you leave it to the OS to decide what memory to reclaim from your app, you may find that the OS reclaimed memory that you’ll need shortly after resuming your app. Instead, your app can voluntarily discard memory allocations that it can regenerate later, on demand and at a low cost. To do so, you can implement the ComponentCallbacks2 interface. You can implement onTrimMemory in your Activity, Fragment, Service, or even your custom Application class. Using it in the Application class is highly effective for global cache management.

The provided onTrimMemory() callback method notifies your app of lifecycle or memory-related events that present a good opportunity for your app to voluntarily reduce its memory usage.

In terms of memory lifecycle management, your implementation should focus exclusively on TRIM_MEMORY_UI_HIDDEN and TRIM_MEMORY_BACKGROUND. Since Android 14, the system has ceased delivering notifications for other legacy constants, which were formally deprecated in Android 15.

TRIM_MEMORY_UI_HIDDEN: This signal indicates that your application's UI has transitioned out of the user's view. This provides an opportunity to release substantial memory allocations tied strictly to the interface—such as Bitmaps, video playback buffers, or complex animation resources.

TRIM_MEMORY_BACKGROUND: At this level, your process is residing in the background and is now a candidate for termination to satisfy the system's global memory needs. To extend the duration your process remains in the cached state, and reduce the number of app cold starts, you should aggressively release any resources that can be easily reconstructed once the user resumes their session.

import android.content.ComponentCallbacks2
// Other import statements.

class MainActivity : AppCompatActivity(), ComponentCallbacks2 {

    /**
     * Release memory when the UI becomes hidden or when system resources become low.
     * @param level the memory-related event that is raised.
     */
    override fun onTrimMemory(level: Int) {

        if (level >= ComponentCallbacks2.TRIM_MEMORY_UI_HIDDEN) {
            // Release memory related to UI elements, such as bitmap caches.
        }

        if (level >= ComponentCallbacks2.TRIM_MEMORY_BACKGROUND) {
            // Release memory related to background processing, such as by
            // closing a database connection.
        }
    }
}

Note: The onTrimMemory integration may depend on SDK support. For instance, certain games rely on their game engine to enable this capability. Please check out the game memory optimization documents.

Advanced memory observability with ProfilingManager

To catch and diagnose memory issues in the field that cannot be reproduced locally, you should leverage the ProfilingManager API. Introduced in Android 15, this advanced observability API allows you to programmatically collect real-user Perfetto profiles.

For teams that lack a dedicated infrastructure to manage and host performance artifacts, Crashlytics is exploring a specialized solution to streamline this workflow. They are inviting developers to provide feedback.

Android 17 introduces new event-driven triggers, most notably TRIGGER_TYPE_OOM and TRIGGER_TYPE_ANOMALY:

• The OOM trigger automatically collects a Java heap dump at the exact moment an OutOfMemoryError crash occurs, providing precise allocation states. A collected OOM profile is provided the next time the app starts and registers the registerForAllProfilingResults callback.
• The Anomaly trigger detects severe performance issues, such as excessive binder spam or breached memory thresholds. The memory anomaly delivers a heap dump just prior to the system terminating the app.

  val profilingManager = 
applicationContext.getSystemService(ProfilingManager::class.java)
    val triggers = ArrayList()  


    triggers.add(ProfilingTrigger.Builder(
                 ProfilingTrigger.TRIGGER_TYPE_ANOMALY))
    val mainExecutor: Executor = Executors.newSingleThreadExecutor()
    val resultCallback = Consumer { profilingResult ->
        if (profilingResult.errorCode != ProfilingResult.ERROR_NONE) {
            // upload profile result to server for further analysis          
            setupProfileUploadWorker(profilingResult.resultFilePath)
        } 

    profilingManager.registerForAllProfilingResults(mainExecutor, resultCallback)
    profilingManager.addProfilingTriggers(triggers)

Once you’ve collected the heap dump, you can download the profile from the server, or locally via adb pull and drag and drop the file into the Perfetto UI. To streamline your memory debugging workflow, use the Heap Dump Explorer, this is the new default view for heap dumps in Perfetto UI. This tool provides an intuitive interface for inspecting Java heap dumps, allowing you to visualize object allocation hierarchies, compute retained memory sizes, and identify the shortest path from garbage collection root. By leveraging the Heap Dump Explorer, you can rapidly pinpoint memory leaks, bloated retained objects such as excessive bitmap allocations, and analyze heap object allocations all in one place.

Conclusion

Optimizing bytecode with R8, adopting image loading best practices, and resolving memory leaks are critical steps toward delivering a high-quality user experience while managing resources effectively under pressure. Adopting these proactive measures helps maintain app stability and performance, preventing unexpected terminations while safeguarding user context. To further your performance expertise, explore our revised memory guidance.

Posted by Alice Yuan, Developer Relations Engineer at Google, Arti Arutiunov, Product Manager at Datadog and Nikita Ogorodnikov, Staff Software Engineer at Datadog

Performance regressions are notoriously hard to reproduce, making regressions a massive bottleneck for mobile developers. Although signals like ANR rates indicate what issues occur in production, pinpointing the specific line of code that resulted in the performance issue has historically necessitated exhaustive manual reproduction or speculative trial-and-error experimentation.

Datadog collaborated with Google to mitigate this frustration by integrating the ProfilingManager API (available on Android 15+ devices) into its Real User Monitoring (RUM) and Continuous Profiling platforms. This integration transforms the debugging workflow, allowing developers to move beyond surface-level symptoms to being able to detect the why behind a performance bottleneck.

The impact of ProfilingManager

ProfilingManager is a system service introduced in Android 15 that enables apps to programmatically collect performance data such as call stack samples, field traces and memory heap dumps directly from production environments. This capability shifts the engineering paradigm from reactive manual reproduction to proactive field analysis.

Solving the Android code-level visibility challenge

Prior to the implementation of ProfilingManager, Datadog’s Real User Monitoring (RUM) focused on high-level application health and session-level telemetry to assess the user journey. Engineering teams could monitor Android performance signals like time to initial display, ANR rates, CPU load, and frozen frames. These insights extended to granular interactions, such as network latency, touch events, and main thread hangs. However, while this data effectively highlighted which performance bottlenecks were surfacing in the field, it provided no clear path to identifying the root cause of these failures.

To address this, Datadog needed a profiling engine capable of capturing Android traces directly from devices in production with minimal performance impact. After evaluating alternative approaches, such as writing their own trace processor using Android Debug APIs, the team selected ProfilingManager because it is the most performant solution of the profiling options they evaluated and offloads the sampling decisions overhead to the OS.

ProfilingManager supports a wide range of collection methods, including CPU traces, call stack sampling, memory analysis through Java heap dumps and native heap profiles. It enables developers to profile production builds, upload trace files to external storage, and review them in the Perfetto trace analyzer UI. As a SaaS provider, Datadog uploads, visualizes, and analyzes these profiles collected via its SDK, providing a unified view of application health.

• Granular session diagnostics: ProfilingManager enhances debuggability by delivering direct OS-level trace data, overcoming the visibility and alignment challenges typical of custom logging with system services. To dive deeper, developers can download these traces from Datadog to investigate further in visualization tools like the Perfetto UI.
• Automated telemetry triggers: By leveraging native system events to initiate trace recordings at key optimization points, Datadog reduces the need to build custom collection logic. While the initial rollout focuses on the APP_FULLY_DRAWN signal, there are already plans to expand this observability to include ANR, OOM, and COLD_START triggers.
• Proactive trace snapshots: By interfacing directly with the system-level Perfetto service (traced), ProfilingManager utilizes a proactive background recording model designed to capture unpredictable issues. This ensures that developers receive a precise visualization of the events leading up to a performance anomaly, offering a level of insight that exceeds what is possible through manual instrumentation.
• Bottleneck detection at scale: Datadog is able to synthesize telemetry from across Datadog’s global customer base to uncover regressions that only emerge under unique hardware configurations and variable network environments.
• System-enforced resource stability: The API leverages sampling trace collection to ensure performance and user experience impacts remain unnoticeable.
• On-device data controls: ProfilingManager filters out irrelevant information from other processes on-device before the profile is delivered to the app. This minimizes file sizes and ensures that only data relevant to the app's processes is provided.

Processing millions of weekly profiles to optimize real-world apps

Conclusion

By integrating Android’s ProfilingManager API, Datadog successfully closed the visibility gap between backend systems and mobile client applications for their customers. By processing millions of profiles weekly with negligible device overhead, Datadog equips Android developers with the code-level insights necessary to diagnose complex performance bugs instantly, helping developers build smoother applications and improve their app’s performance signals in the Play Store. To adopt the ProfilingManager API directly into your performance observability framework, check out our documentation.

In the future, Datadog aims to make Android profiling data a first-class input for coding agents to autonomously resolve performance bottlenecks, closing the feedback loop between detection and remediation. Datadog is working toward making Android profiling broadly accessible to developers.

To get started using the Datadog real user monitoring feature powered by ProfilingManager, visit Datadog Mobile Real User Monitoring.

Today we're releasing Android 17 and making it available on most supported Pixel devices. Look for new devices running Android 17 in the coming months.

Android 17 marks the start of our transition to an intelligence system, putting your apps at the center. It's shifting to an adaptive-first development standard by introducing mandatory large-screen resizability, all while delivering next-generation privacy, security, media, camera, and performance. We'll cover all that in this post, as well as how we're bringing together next generation tools, libraries, and agent skills to help your apps embrace the opportunity.

Throughout the past year, from our Canary channel to our Beta releases, we’ve collaborated with you in the developer community to build a platform you and your users can trust. To that end, this moment marks the availability of the source code at the Android Open Source Project (AOSP). This allows you to examine the source code for a deeper understanding of how Android works.

Let's dive deeper into Android 17.

An intelligence system

With deep integration between hardware, software and AI, we’re transforming Android from an operating system to an intelligence system. It's about delivering new helpful experiences that anticipate user needs, and it brings more opportunities for engagement with your apps. To that end, Android 17 expands the capabilities of AppFunctions, a platform API with a corresponding Jetpack library. It allows you to contribute your app's unique capabilities as orchestratable "tools" for Android MCP, the on-device equivalent of the Model Context Protocol. AI agents and assistants (like Google Gemini) can discover and execute AppFunctions to perform workflows on behalf of the user with direct access to the app's local state.

The Jetpack library, currently in alpha, makes adding AppFunctions as easy as annotating a class and adding KDoc comments.

/**
 * A note app's [AppFunction]s.
 */
class NoteFunctions(
    private val noteRepository: NoteRepository
) {
    /**
     * Adds a new note to the app.
     *
     * @param appFunctionContext The execution context.
     * @param title The title of the note.
     * @param content The note's content.
     */
    @AppFunction(isDescribedByKDoc = true)
    suspend fun createNote(
        appFunctionContext: AppFunctionContext,
        title: String,
        content: String
    ): Note {
        return noteRepository.createNote(title, content)
    }
}

We’ve also launched an AppFunctions agent skill that analyzes your app’s key workflows, automatically generates the required Kotlin code, optimizes your KDocs for LLM tool-calling, and provides ADB commands for testing and debugging.

The Gemini integration is currently in a private preview with trusted testers, but you can begin preparing your apps now. In addition to ADB commands to execute your AppFunctions, we've provided a test agent app that includes an interface to discover and execute your app functions and simulate an AI agent integration. Join our integration early access program at goo.gle/eap-af for a chance to be among the first apps to deploy AppFunctions to production.

Adaptive-first

Your users no longer rely on a single form factor; they transition between phones, foldables, tablets, laptops, automotive displays, and immersive XR environments. Now, with over 580 million large screen devices in the hands of users and the forthcoming launch of Googlebooks, the next generation of ChromeOS built on the Android stack, adaptive is no longer just a technical goal. It’s a massive opportunity to reach highly engaged users, which is one of the reasons we're shifting to an adaptive-first development standard.

No resizability/orientation restrictions on large screens

To ensure apps deliver a premium experience across all form factors, including mobile devices running in desktop mode on connected displays, Android 17 (API level 37) removes the developer opt-out for orientation and resizability restrictions on large screen devices (sw > 600 dp) for apps targeting API level 37. The system will ignore legacy manifest attributes and runtime APIs, including screenOrientation, setRequestedOrientation(), resizeableActivity=false, and aspect ratio constraints (minAspectRatio/maxAspectRatio). Games (based on app category in Google Play) remain exempt. Your app must be ready to adapt to any window size, respect the user's preferred device posture, and support free-form windowing natively.

Next-gen multitasking: App Bubbles, Bubble Bar, and desktop interactive PiP

Android 17 introduces powerful new windowing capabilities that redefine how users multitask, demanding even greater layout flexibility from your apps:

• App Bubbles: Moving beyond the messaging bubbles API, users can now transform any app into a floating bubble by long-pressing its icon on the launcher. This feature is available across phones, foldables, and tablets, enabling lightweight multitasking for any workflow.
• The Bubble Bar: On large screens (tablets and foldables), the system taskbar now includes a dedicated Bubble Bar to organize, transition between, and dock these floating app bubbles.
• Desktop interactive PiP: In desktop environments, Android 17 introduces interactive Picture-in-Picture (PiP). Unlike traditional PiP windows which are read-only, these pinned windows remain fully interactive while staying always-on-top of other application windows.

App Bubbles and Bubble Bar in action

Activity recreation updates

To prevent disruptive state loss and stutter, Android 17 updates the default behavior for Activity recreation. The system will no longer restart activities by default for typical configuration changes that do not require a full UI redraw (including CONFIG_KEYBOARD, CONFIG_KEYBOARD_HIDDEN, CONFIG_NAVIGATION, CONFIG_TOUCHSCREEN, and CONFIG_COLOR_MODE).
Instead, running activities will receive these updates via onConfigurationChanged(), enabling smooth transitions. If your application explicitly relies on a full restart to reload resources for these changes, you must now explicitly opt-in using the new android:recreateOnConfigChanges manifest attribute.

Continue On

Android 17 adds Continue On to help users seamlessly transition a task between Android devices. The user sees a suggestion for the most recently opened app from their mobile device in their tablet taskbar, providing a one-tap affordance to launch the app and deep-link where they left off. Continue on can support app-to-web transitions, including falling back to using the web if the app isn't installed.

Handoff Suggestion on a Tablet

class MyHandoffActivity : Activity() {

    ...

  override fun onCreate(savedInstanceState: Bundle?) {
    super.onCreate(savedInstanceState)
    // Do stuff
    ...
    // Enable handoff
    setHandoffEnabled(true, null)
  }

  // Override and implement onHandoffActivityDataRequested
  override fun onHandoffActivityDataRequested(handoffRequestInfo: HandoffActivityDataRequestInfo) : HandoffActivityData {
    // Create and return handoff data
  }
}

Go adaptive-first with Jetpack Compose

To help you adapt your apps to meet the new Android 17 requirements, we've launched the Jetpack Compose adaptive skill. This AI-powered developer workflow helps you implement the best adaptive practices:

• Adaptive navigation: Automatically transition between bottom navigation bars on mobile and edge-anchored navigation rails on large screens using NavigationSuiteScaffold from the Material 3 Adaptive library.
• Multi-pane layouts: Implement list-detail and supporting pane layouts natively using Navigation 3 Scenes (ListDetailSceneStrategy and SupportingPaneSceneStrategy) instead of fragile fragment transactions.
• FlexBox & Grid APIs: Utilize Compose 1.11's dynamic layout components to easily adjust row and column spans on the fly, ensuring your content always fills the space beautifully.
• Advanced non-touch input: Leverage Compose 1.11's enhanced trackpad and mouse support, including native focus rings and new APIs (like TrackpadInjectionScope and performTrackpadInput) to easily test and deliver a true "laptop-class" experience on Googlebooks and Desktop Mode.
• Dynamic window states: Leverage Compose's reactive state model to seamlessly adapt your UI when the app transitions from full screen to a floating App Bubble or an interactive Desktop PiP window, ensuring a premium experience even at minimal dimensions.

Android is Compose-first

Compose offers the easiest way to build adaptive apps, and that's just one of the many reasons we believe that all Android UI should be built with Compose. To that end, Android development is now Compose-first. All new Android APIs, libraries, tools, and developer guidance will be built exclusively for Jetpack Compose. Legacy View components (in the android.widget package) and View-based Jetpack libraries (like Fragments, RecyclerView, and ViewPager) are now in maintenance mode. They will receive only critical bug fixes, and no new features.

TIP
Ready to migrate? Use our AI-driven XML to Compose Migration Skill to automatically analyze your legacy View layouts and convert them into highly-adaptive Compose code.

Performance & efficiency

App performance means a smooth user interface, fast app start times, and efficient multitasking; Android 17 has impactful improvements in all of these areas.

App memory limits

Memory usage is one of the silent foundations of overall performance. When a foreground app or service grows unchecked, memory management spikes CPU and battery utilization and eventually leads to the termination of other well-behaved cached apps and background jobs, ultimately forcing slower cold starts and impaired multitasking. 

Starting in Android 17, the system will enforce strict app memory limits based on a device's total RAM, abruptly terminating offending processes. New things to help you navigate these tighter requirements:

• R8 Optimizer: The R8 optimizer significantly reduces your app's bytecode memory footprint by shrinking classes, methods, and fields into shorter names, and stripping out unused code and resources. Use R8 in full mode along with the new R8 configuration analyzer to make sure your app is getting the most from R8.
  

• LeakCanary in Android Studio Panda: The profiler now features native LeakCanary integration as a dedicated task, fully integrated with your IDE and source code.
• ApplicationExitInfo: If your app is terminated by these limits, getDescription() from ApplicationExitInfo will return "MemoryLimiter:AnonSwap".
• On-Device Anomaly Detection: Part of ProfilingManager, you can leverage trigger-based profiling using TRIGGER_TYPE_ANOMALY to automatically capture heap dumps when the memory limit is reached.

val profilingManager = applicationContext
   .getSystemService(ProfilingManager::class.java)

val triggers = ArrayList<ProfilingTrigger>().apply {
  add(ProfilingTrigger.Builder(
    ProfilingTrigger.TRIGGER_TYPE_ANOMALY).build())
}
profilingManager.addProfilingTriggers(triggers)

And, we're working to surface more in-field memory metrics to you within Google Play Console.

Generational garbage collection

Android 17 introduces more frequent, less resource-intensive young-generation collections to ART's Concurrent Mark-Compact garbage collector (GC). By separating short-lived objects from stable, long-lived ones, the system runs frequent, lightweight "young-generation" sweeps rather than expensive full-heap scans, drastically reducing CPU usage, power drain, and UI stutter. Our testing has shown significant improvements in GC interference with application threads and a reduction in the maximum memory resident set size (RSS). ART improvements are also available to over a billion devices running Android 12 (API level 31) and higher through Google Play System updates.

Lock-Free MessageQueue

For apps targeting SDK 37 or higher, the core android.os.MessageQueue now implements a lock-free architecture, significantly reducing missed frames, improving app startup time, and radically improving the performance of busy queues in multithreaded scenarios. Note: This can break apps that use reflection on private MessageQueue fields and methods.  The peekWhen and poll APIs have been added to TestLooperManager for instrumentation testing without relying on MessageQueue internals.

Static final fields now truly final

Starting from Android 17, apps targeting SDK 37 or higher won’t be able to modify “static final” fields, allowing the runtime to apply performance optimizations more aggressively. An attempt to do so via reflection (or deep reflection) will lead to an IllegalAccessException being thrown. Modifying them via JNI’s SetStatic<Type>Field methods family will immediately crash the application.

Custom notification view restrictions

To reduce memory usage we are further restricting the size of custom notification views. This update closes a loophole that allows apps to bypass existing limits using URIs. This behavior is gated by the target SDK version and takes effect for apps targeting API 37 and higher.

Privacy & Security

Maintaining user trust is at the heart of the Android ecosystem. Android 17 introduces robust features that protect sensitive data while simplifying user experiences.

Privacy-preserving choices

Historically, apps required broad, permanent permissions to access information like contacts, precise location and media files. Android 17 continues the shift toward privacy-preserving choices that grant temporary, session-based access only to the data the user explicitly selects:

• System-Level Contact Picker: Utilizing ACTION_PICK_CONTACTS, apps can request temporary access only to specific fields (e.g., email or phone number) chosen by the user, eliminating the need for the broad READ_CONTACTS permission. It also fully supports work/personal profile separation.
• Customizable Photo Picker aspect ratio: UsingPhotoPickerUiCustomizationParams, you can customize the system photo picker to show thumbnails in portrait mode. This is perfect for apps that always display photos and videos in portrait such as video based social media apps.
• System-rendered Location Button: A new system-rendered location button that you can embed in your app grants precise location access for the current session only.
• EyeDropper API: A new system-level API, ACTION_OPEN_EYE_DROPPER, allows your app to create a system-powered eyedropper enabling the user to select color from any pixel on the display. This provides a secure, privacy-preserving color-picking experience that eliminates the need for broad, sensitive screen capture or media projection permissions.

val eyeDropperLauncher = registerForActivityResult(ActivityResultContracts.StartActivityForResult()) { result ->
   if (result.resultCode == Activity.RESULT_OK) {
       val color = result.data?.getIntExtra(Intent.EXTRA_COLOR, Color.BLACK)
       // Use the picked color in your app
   }
}
fun launchColorPicker() {
   val intent = Intent(Intent.ACTION_OPEN_EYE_DROPPER)
   eyeDropperLauncher.launch(intent)
}

Picking a color from anywhere on the screen with the system EyeDropper

Local network access

Apps targeting Android 17 now either require the ACCESS_LOCAL_NETWORK runtime permission or the use of system-mediated, privacy-preserving device pickers for local network communication, such as talking to smart home devices or casting receivers. Because ACCESS_LOCAL_NETWORK falls under the existing NEARBY_DEVICES permission group, users who have already granted other NEARBY_DEVICES permissions will not be prompted again.

SMS OTP protection

Android 17 expands SMS one-time-password (OTP) protection by delaying access to SMS messages for three hours:

• WebOTP Format: Delayed for all apps that are not the intended recipient (domain mismatch).
• Standard SMS OTP: Delayed for all apps targeting SDK 37+.
• Exemptions: Default SMS, assistant, and connected companion apps are exempt. Apps are strongly encouraged to migrate to the SMS Retriever or SMS User Consent APIs.

Post-Quantum Cryptography (PQC)

Android 17 is ready for the next generation of cryptographic security:

• Keystore Integration: Supported devices can generate ML-DSA (Module-Lattice-Based Digital Signature Algorithm) keys in secure hardware to produce quantum-safe signatures, exposed via standard JCA APIs.
• Hybrid APK Signing: Introducing the v3.2 APK Signature Scheme, which combines classical signatures with ML-DSA signatures to secure app delivery.

Safer native dynamic code loading 

Smarter password protection for physical inputs

With Android 17, we're making it safer to enter passwords, PINs, and other secrets when using a physical keyboard by no longer showing the last typed character by default.

Users can still easily customize these display settings to match their preferences (availability may vary by device manufacturer).

These enhanced privacy protections are automatically supported byAndroid's built-in SDK components and will be supported in Compose 1.12 for SecureTextFields.

Media and camera features that empower creators and delight users

Android 17 introduces new creator features that give access to pro-quality cameras and media, all while improving the experience for consumers.

• Eclipsa Video: HDR video standard built upon the SMPTE ST 2094-50 specification that introduces new metadata to help devices adapt content for their display headroom and ambient light conditions, as well as improve the simultaneous display of standard and HDR content.
• RAW14 image format: New support for the RAW14 image format provides a way for your professional camera app to capture the highest level of detail and color depth from compatible camera sensors.
• Vendor-defined camera extensions: Vendor-defined extensions enable hardware partners to define and implement custom camera extension modes, providing access to the best and latest camera features.
• Extended HE-AAC software encoder: A new system-provided Extended HE-AAC software encoder, supports both low and high bitrates using unified speech and audio coding, providing significantly better audio quality for voice messages in low-bandwidth conditions, including support for loudness metadata.
• Versatile Video Coding (H.266): Enables OEMs to add codec support by defining the video/vvc MIME type in MediaFormat, adding new VVC profiles in MediaCodecInfo, and integrating support into MediaExtractor.
• Camera device type: New APIs that query the underlying device type to identify if a camera is built-in hardware, an external USB webcam, or a virtual camera.
• Constant Quality for Video Recording: SetVideoEncodingQuality in MediaRecorder configures a constant quality (CQ) mode for video encoders to ensure uniform visual fidelity across the entire video.

Better support for hearing aids

• Bluetooth LE Audio hearing aid support: Android now includes a specific device category for Bluetooth Low Energy (BLE) Audio hearing aids with the new AudioDeviceInfo.TYPE_BLE_HEARING_AID constant, so your app can distinguish hearing aids from regular headsets to provide a tailored experience for users with assistive listening devices.
• Granular audio routing for hearing aids: Android 17 allows users to independently manage where specific system sounds are played. They can choose to route notifications, ringtones, and alarms to connected hearing aids or the device's built-in speaker, helping to avoid unwanted in-ear interruptions while maintaining a Bluetooth connection for hearing aid management apps.

CameraX and Media3

CameraX and Media3 have been updated for Android 17. They are there to do the heavy lifting, smoothing the rough edges of media development and simplifying building reliable camera capture, smooth media playback, and creative and complex editing experiences.

We've released an agent skill that can migrate legacy Android camera implementations (Camera1 or raw Camera2 APIs) to CameraX.

Note: You'll need to update your CameraX version to either 1.5.2 or 1.6.0+ to avoid a crash related to an added dynamic range mode on Android 17 devices.

Get your apps, libraries, tools, and game engines ready!

If you develop an Android SDK, library, tool, or game engine, it's critical to prepare any necessary updates now to prevent your downstream app and game developers from being blocked by compatibility issues and allow them to target the latest SDK features. Please let your downstream developers know if updates are needed to fully support Android 17.

Testing involves installing your production app or a test app making use of your library or engine using Google Play or other means onto a device or emulator running Android 17 Beta 4. Work through all your app's flows and look for functional or UI issues. Each release of Android contains platform changes that improve privacy, security, and overall user experience; review the app impacting behavior changes for apps running on and targeting Android 17 to focus your testing, including the following:

• Resizability on large screens: Once you target Android 17 (SDK 37), you can no longer opt out of maintaining orientation, resizability and aspect ratio constraints on large screens.
• Dynamic code loading: If your app targets SDK 37 or higher, the Safer Dynamic Code Loading (DCL) protection introduced in Android 14 for DEX and JAR files now extends to native libraries. All native files loaded using System.load() must be marked as read-only. Otherwise, the system throws UnsatisfiedLinkError.
• Enable CT by default: Certificate transparency (CT) is enabled by default. (On Android 16, CT is available but apps had to opt in.)
• Local network protections: Apps targeting SDK 37 or higher have local network access blocked by default. Switch to using privacy preserving pickers if possible, and use the new ACCESS_LOCAL_NETWORKpermission for broad, persistent access.
• Background audio hardening: Starting in Android 17, the audio framework enforces restrictions on background audio interactions including audio playback, audio focus requests, and volume change APIs. Based on your feedback, we’ve made some changes since beta 2, including targetSDK gating while-in-use FGS enforcement and exempting alarm audio. Full details available in the updated guidance.
• NPU access declaration: Apps targeting Android 17 that need to directly access the NPU must declare FEATURE_NEURAL_PROCESSING_UNIT in their manifest to avoid being blocked from accessing the NPU. This includes apps that use the LiteRT NPU delegate, vendor-specific SDKs, as well as the deprecated NNAPI.

Get started with Android 17

Your Pixel device should get Android 17 shortly if you haven't already been on the Android Beta. If you don’t have a Pixel device, you can use the 64-bit system images with the Android Emulator in Android Studio. If you are currently on Android 17 Beta 4.1 and have not yet taken an Android 17 QPR1 beta, you can opt out of the program and you will then be offered the release version of Android 17 over the air.

Getting the Android 17 beta on partner devices

Android 17 is available in beta on handset, tablet, and foldable form factors from partners including Honor, iQOO, Lenovo, OnePlus, OPPO, Realme, Sharp, vivo, and Xiaomi.

For the best development experience with Android 17, we recommend that you use the latest Canary build of Android Studio Quail. Once you’re set up, here are some of the things you should do:

Test your current app for compatibility, learn whether your app is affected by changes in Android 17, and install your app onto a device or Android Emulator running Android 17 and extensively test it.

Thank you again to everyone who participated in our Android developer preview and beta program. We're looking forward to seeing how your apps take advantage of the updates in Android 17, and have plans to bring you updates in a fast-paced release cadence going forward.

For complete information on Android 17 please visit the Android 17 developer site.

We’ve all been there: You’re scrolling through your favorite social media feed in a dim room, and suddenly an HDR video pops up. It’s so intensely bright that you have to squint, or maybe you find yourself turning down your screen brightness just to read the caption. Other times, a video that looks vibrant on your phone looks flat, dark, or washed out when you watch it on your living room TV. 

While High Dynamic Range (HDR) technology was designed to make videos look richer and more lifelike, the lack of unified industry guidelines means that the exact same clip can render in unexpected and jarring ways depending on the display you’re using.

To solve this, we’re introducing Eclipsa Video—a new standard built to make your favorite videos look consistent, balanced, and comfortable on every screen. Eclipsa Video builds on the open SMPTE ST 2094-50 specification, which Google developed in collaboration with Apple and NBCUniversal.

Sudden brightness spikes during feed scrolling—fixed with Eclipsa Video.

More consistency, comfort, and creative control

Designed to scale with your hardware, Eclipsa Video provides three core benefits:

• A consistent baseline: Eclipsa Video introduces a shared rulebook for screens. It establishes a consistent benchmark for normal brightness—known as the HDR reference white. This ensures standard text, app interfaces, and standard-range colors remain vibrant and readable without causing uncomfortable screen glare.
• Adaptive headroom: Screens have different physical brightness limits, or "headroom." Eclipsa Video guides how displays handle highlights dynamically. Bright details remain brilliant on a premium television, while being scaled intelligently on a mobile screen to prevent sudden blinding transitions.
• Preserved creative intent: Rather than applying a single static setting to an entire video, Eclipsa Video carries adaptive, frame-by-frame instructions. Think of it as a set of digital notes from the creator traveling with the video, ensuring the exact colors, contrast, and mood they graded are preserved on your display.

Eclipsa Video preserves true highlight detail on any screen you watch.

Built natively into Android 17

Starting with Android 17, support for Eclipsa Video is built directly into the platform. This means a more comfortable, true-to-life HDR experience is coming natively to the phones, tablets, and TVs you rely on every day. The video you capture carries its creative intent with it, and the video you watch is shown exactly the way it was meant to be seen.

Guidelines for developers & creators

We’re inviting the developer and creator ecosystem to help build a more reliable HDR environment:

• Get started with implementation: Learn how to configure playback and capture in your apps with our official guide.
• ExoPlayer & Media3 integration: Standard playback handling built directly into Jetpack Media3, allowing ExoPlayer to support Eclipsa Video metadata automatically with no additional player configuration.
• Explore open source tools: View and inspect SMPTE ST 2094-50 metadata and dynamic gain curves in real time using HDR Explorer.

What’s next

Eclipsa Video is rolling out now, and you’ll see more apps and devices supporting it over time. Because it’s an open standard, any app developer or hardware manufacturer can integrate it to elevate the viewing experience.

Try out the new tools in Android 17, explore the open-source metadata, and let us know what you think on our developer channels. We can’t wait to see what you create.

Notes & Availability

1. Device Compatibility: Eclipsa Video playback and capture are supported natively on devices running Android 17 (API level 37) and above with HDR displays passing Eclipsa Compliance tests.

2. Developer Resources: The SMPTE ST 2094-50 Specification is openly accessible for technical evaluation.

Posted by Ben Weiss, Senior Developer Relations Engineer, Android Developer Relations

Welcome back to the blog post series "Build intelligent Android apps" where we take a basic Android app and transform it into a personalized, intelligent, and agentic experience. In our previous post, we explored how to leverage Firebase AI Logic to build cloud-hosted and hybrid AI features.

In this article, we'll show you how we designed and integrated these capabilities into our travel planning app, JetPacker, using Android AppFunctions. We'll explore the rationale behind our feature choices, discuss the specialized tooling we used to accelerate development, and dive into the code that makes it all work.

Designing AI-ready features: making choices that matter for your users

To select which features to provide to the intelligence system, we looked for tasks where a voice or text command is objectively faster than tapping through screens. In this side-by-side screen recording you can see this contrast perfectly: on the left, a user tapping through multiple screens to log an expense; on the right, the same task completed instantly in the background via a privileged agent.

Our first choice was expense tracking. Logging a coffee expense during a trip usually takes quite a few taps—unlocking the phone, opening the app, finding the active trip, navigating to the expenses tab, tapping the add button, taking a picture of the receipt, and checking the result. By providing the addExpense and getExpenses features as AppFunctions, the system agent handles the heavy lifting. When the user says, "Add a five-dollar coffee expense to my Paris trip," the agent automatically searches for the correct trip ID in the background and inserts the expense, skipping the manual UI flow entirely.

We also prioritized itinerary management. Finding what activity is next on a busy trip itinerary usually requires scrolling through a dense timeline view. By providing getItinerary and addItineraryEvent to the system, the user can simply ask, "What am I doing next in Paris?" and get an immediate answer.

Finally, we focused on hands-free note capturing. Typing out reminders or notes while walking down a busy street is difficult and unsafe. Exposing a voice note capability allows the user to say, "The flight was amazing, I saw a beautiful sunset and managed to sleep well," and the privileged agent automatically transcribes and saves it directly into the travel database  using the addVoiceNote AppFunction.

Android MCP powered by AppFunctions

This entire experience is built on Android MCP. Under this design, the app acts as a local MCP server. Rather than remote APIs, you provide your app features directly to the on-device intelligence system.

Android AppFunctions is the API that brings this concept to life. It reads annotated Kotlin functions and compiles them into type-safe, sandboxed tool definitions that the privileged agent can discover and invoke locally on the device.

Diagram highlighting our apps, the android platform, and system agents coordinate AppFunctions.

Under the Android MCP model, your app acts as a local MCP server that exposes structured tools, while the Android platform serves as the central tool registry. On the MCP client side, agent apps are registered with the intelligence system after being granted system-privileged permissions to access the registry.

When a user interacts with a registered agent, its LLM determines if the request can be handled by an AppFunction, queries the platform's metadata, and executes the appropriate registered functions in the background. This local MCP client-server design gives you full control: you choose exactly which features are accessible to the agent, keeping the rest of your app's data private.

How we accelerated development with Android skills

To streamline the integration process, we leveraged the AppFunctions development skill. The AppFunctions development skill is a complete development companion. It guided us through the entire lifecycle: mapping Kotlin data classes to serialize parameters, generating the necessary Service entry points, refining our KDoc documentation to ensure the LLM understands parameter boundaries, and setting up automated testing using ADB.

Providing app features to the intelligence system

Enough with the theory, let's dive into the implementation.

Configuration and dependency setup

We begin by adding the AppFunctions dependencies. One for the API and one for the Kotlin Symbol Processing compiler.

implementation("androidx.appfunctions:appfunctions:1.0.0-alpha10")
ksp("androidx.appfunctions:appfunctions-compiler:1.0.0-alpha10")

Modeling custom data types

Any custom object exchanged with the agent must be annotated with @AppFunctionSerializable. In our TripSerializable.kt file, we define our trip data model:

@AppFunctionSerializable(isDescribedByKDoc = true)
data class TripSerializable(
    /** The trip's unique identifier. */
    val id: String,
    /** The trip's title. */
    val title: String,
    /** The trip's destination location. */
    val location: String,
    /** The trip's start date in milliseconds. */
    val startDate: Long,
    /** The trip's end date in milliseconds. */
    val endDate: Long,
    /** A list of participants. */
    val participants: List<String>,
)

Providing features using the @AppFunction annotation

Next, the skill wrote the Kotlin functions that perform the database queries and annotate them with @AppFunction. We can view this in searchTrip:

/**
 * Looks for trips based on optional filters like id, title (name), location, and dates.
 *
 * @param id The unique identifier of the trip.
 * @param title The title or name of the trip.
 * @param location The destination location.
 * @param startDate The minimum start date in milliseconds.
 * @param endDate The maximum end date in milliseconds.
 * @return A list of trips matching the filters.
 */
@AppFunction(isDescribedByKDoc = true)
suspend fun searchTrip(
    id: String? = null,
    title: String? = null,
    location: String? = null,
    startDate: Long? = null,
    endDate: Long? = null
): List<TripSerializable> {
    return withContext(Dispatchers.IO) {
    // implementation
}

Since AppFunctions run on the UI thread by default, we use withContext(Dispatchers.IO) to switch to a background dispatcher. Additionally, we refine our KDoc to use clear, imperative verbs and specify parameter constraints. This documentation compiles directly into the tool's schema, which the privileged agent uses to resolve parameters and handle runtime errors.

The service entry point and Hilt integration

To register these features with the intelligence system, we create an abstract base class that extends AppFunctionService. We annotate it with @AppFunctionServiceEntryPoint:

@RequiresApi(36)
@AndroidEntryPoint
@AppFunctionServiceEntryPoint(
    serviceName = "JetPackerAppFunctionService",
    appFunctionXmlFileName = "jetpacker_app_function_service"
)
abstract class BaseJetPackerAppFunctionService : AppFunctionService() {
    @Inject internal lateinit var tripDao: TripDao
    // DAOs and database references are injected here...
}

During compilation, KSP generates the final concrete service subclass, JetPackerAppFunctionService, as declared with the serviceName parameter. We also register app_metadata.xml in the app's manifest. This file provides global operational rules for JetPacker's declared AppFunctions.

Testing and verifying your AppFunctions

Once implemented, you should verify that your AppFunctions are registered and working correctly.

Running devices or emulators with Android 17 or newer, you can use ADB commands from your terminal to list and invoke your functions. Running adb shell cmd app_function list-app-functions displays all registered functions for your package. You can then execute a specific function and test its database integration by running adb shell cmd app_function execute-app-function while passing a raw JSON parameters string.

Instead of these ADB commands, you can also use the AppFunctions Testing Agent to inspect your configuration, list and execute AppFunctions, and even see how your AppFunctions behave in a real conversational flow.

Wrapping it up

When thinking about app features that can be contributed to the intelligence system using AppFunctions requires a slight shift in how we think about code and documentation. AppFunctions enable you to use this new interaction model for apps, which allows using an agent to access app features..

First, the AppFunctions development skill is an essential lifecycle tool, helping you discover features, implement and refine AppFunctions for your apps. Second, KDoc comments are a compiled API asset; clear parameter descriptions directly impact the execution accuracy of the system agent. Finally, Android MCP provides local-first execution allowing apps to safely collaborate with AI agents.

Contributing app features through AppFunctions makes your application ready for the intelligence system. Let us know how you are adapting your apps for the agentic era!

Learn more

Check out the other parts of this blog post series:
Part 1: Introduction of the app and a high-level overview.
Part 2: On-device intelligence. Deep-dive into ML Kit’s GenAI APIs and Gemini Nano to build privacy-first features like itinerary summarization, receipt parsing, and local audio processing.
Part 3: Hybrid and cloud reasoning. Explore how to use Firebase AI Logic to ground LLM answers in real-world data like Google Maps and web context.
Part 4 (this post!): System integration. Integrating with the Android intelligence system using AppFunctions.
Part 5 (coming soon): In-app agentic workflows. Extend the app with an end-to-end booking assistant powered by A2UI and ADK.

Interested in more on Android Development? Follow Android Developers on YouTube or LinkedIn!

All code snippets in this blog post follow the following copyright notice:

Copyright 2026 Google LLC.
SPDX-License-Identifier: Apache-2.0

Welcome back to the blog post series "Build intelligent Android apps" where we take a basic Android app and transform it into a personalized, intelligent, and agentic experience. In our previous post we explored how to build intelligent on-device features using Gemini Nano through ML Kit's Prompt API.

In this post, we will look at how you can leverage Firebase AI Logic to build cloud-hosted and hybrid AI features: 

• Grounding answers in real-world context
• Routing requests dynamically between cloud and local execution using hybrid inference
• Translating content with custom routing systems

Sometimes a use case requires AI models with greater world knowledge, a much larger context window, or the ability to handle complex queries. In those scenarios, we can leverage cloud models. 

Other times, you want the best of both worlds: using hybrid inference to run on-device when available to lower costs, while falling back to the cloud to ensure compatibility for all devices.

Cloud and hybrid features in Jetpacker: Museum assistant with web grounding, hybrid restaurant review drafting, and  support chat featuring custom-routed live translation.

Let’s look at how we implemented three cloud and hybrid features in Jetpacker:

• a museum assistant with web grounding
• hybrid restaurant review drafting
• hotel support chat featuring custom-routed live translation.

Use LLM grounding for up-to-date informationMuseum assistant chatbot with LLM grounding

The Museum assistant is an interactive chatbot designed to help users plan their museum visits. It provides visitors with up-to-date details regarding specific exhibits, current opening hours, ticket pricing, and more.

Museum assistant is a chatbot that answers questions, such as 

When building AI features, getting the model to answer with fresh, accurate, and specific real-world information is a common challenge. While cloud models possess massive amounts of world knowledge, they might not know about seasonal exhibits or the current day’s opening hours. 

Grounding data is added to the context window to enable the model

To bridge this gap, we can use grounding techniques to add extra context to the model’s context window. The Firebase AI Logic SDK supports three types of grounding:

• URL grounding: Grounding responses using content from a specific webpage (e.g. current ticket prices or museum rules).
• Google Search grounding: Letting the model query the real-time Google search index for up-to-date details.
• Maps grounding: Using Google Maps location data.

In Jetpacker, we dynamically construct the available tools based on enabled feature flags and initialize the generative model using the Firebase AI SDK:

// implementation("com.google.firebase:firebase-ai-logic")

private var toolList = mutableListOf<Tool>()

init {
    if (ENABLE_SEARCH_GROUNDING) {
        toolList.add(Tool.googleSearch())
    }
    if (ENABLE_URL_GROUNDING) {
        toolList.add(Tool.urlContext())
    }
}

private val generativeModel = Firebase.ai(backend = GenerativeBackend.googleAI())
    .generativeModel(
        modelName = "gemini-3-flash",
        systemInstruction = content {
            text("You are a helpful museum assistant answering questions about a museum. Use plain text.")
        },
        tools = toolList
    )

When the user queries the assistant, if URL grounding is enabled, we append the specific museum resource URLs directly into the prompt:

val groundingText = if (FeatureFlags.ENABLE_URL_GROUNDING) {
    "\n If the following message above is about the rules and terms to visit Le Louvre, " +
    "if needed answer this urls ${urlList.joinToString()}"
} else {
    ""
}

val prompt = "$text $groundingText"

var response = chat.sendMessage(prompt)

Hybrid inference: On-device review generation with Maps deep link

Not every AI task requires a cloud-based model, and not every device is online. To help developers balance latency, cost, and offline availability, we recently introduced the Firebase API for Hybrid Inference.

In Jetpacker, the restaurant review feature lets users review select topics and automatically drafts a review. To enable this for all users, we prioritize local execution with Gemini Nano, and fall back to cloud models on devices that don’t support Gemini Nano. 

// implementation("com.google.firebase:firebase-ai-logic")
// implementation("com.google.firebase:firebase-ai-ondevice:16.0.0-beta03")


// Initialize the model with hybrid routing configuration
val reviewModel = Firebase.ai.generativeModel(
    modelName = "gemini-3.1-flash-lite",
    onDeviceConfig = OnDeviceConfig(
        inferenceMode = InferenceMode.PREFER_ON_DEVICE
    )
)

The Hybrid Inference API supports four distinct routing modes:

• PREFER_ON_DEVICE: Prioritizes local execution and falls back to cloud if Gemini Nano is unavailable.
• PREFER_IN_CLOUD: Prioritizes cloud execution and falls back to on-device if the device goes offline.
• ONLY_ON_DEVICE: Restricts execution strictly to the device.
• ONLY_IN_CLOUD: Restricts execution strictly to the cloud.

Once the review is generated, we copy it to the clipboard and use an intent to open Google Maps directly to the restaurant's review page, providing a seamless user experience:

private fun copyAndOpenMapsReview(context: Context, reviewText: String, placeId: String) {
    val clipboard = context.getSystemService(Context.CLIPBOARD_SERVICE) as ClipboardManager
    val clip = ClipData.newPlainText("User Review", reviewText)
    clipboard.setPrimaryClip(clip)

    val uri = Uri.parse("https://search.google.com/local/writereview/mobile?placeid=$placeId")
    val intent = Intent(Intent.ACTION_VIEW, uri).apply {
        setPackage("com.google.android.apps.maps")
    }
    context.startActivity(intent)
}

Custom hybrid routing: Hotel support chat translation with simulated personas

The hotel support chat was built to let users finalize logistics and check on hotel details. This feature uses system instructions to configure a localized receptionist assistant. By passing specific information—such as the preferred language and hotel information—in the instructions, we can set up a conversational persona representing a specific hotel.

private val generativeModel = Firebase.ai(backend = GenerativeBackend.googleAI())
    .generativeModel(
        systemInstruction = content {
            text("""
              You are a helpful hotel receptionist at $hotelName only speaking $language. 
              Answer politely in $language. The bar closes at 10pm and breakfast is from 7am to 10am.
              There's someone at the desk 24/7. You can retrieve your luggage from the storage room 
              at the back of the lobby at any time.
              """)
        },
        modelName = "gemini-3-flash-preview"
    )

Because receptionist responses are in the hotel's local language (for example, French for Hotel Le Meurice in Paris), we need to translate messages to the user’s preferred language. 

Hotel support chat messages are automatically translated to the user’s preferred language 

While hybrid models can configure simple routing preferences, complex scenarios require custom routing logic. In Jetpacker, we implement a custom routing stack that takes into account:

• Language identification: Using the on-device ML Kit Language Identification API, we can detect the incoming message language.
• On-device translation (Gemini Nano): ML Kit’s Prompt API lets us translate common language pairs directly on the device, saving bandwidth and cloud cost.
• Cloud translation (Gemini 3 Flash): For more complex languages, we use Gemini Flash 3 to get a higher quality translation.

// implementation("com.google.android.gms:play-services-mlkit-language-id:17.0.0") 

// ML Kit for Language Identification (powered by Google Play Services)
private val languageIdentifier = LanguageIdentification.getClient()

// On-device translator model (prefer Gemini Nano) for translating common language pairs
private val hybridTranslationModel = Firebase.ai(backend = GenerativeBackend.googleAI())
    .generativeModel(
        modelName = "gemini-3-flash",
        onDeviceConfig = OnDeviceConfig(mode = InferenceMode.PREFER_ON_DEVICE)
    )

// Cloud translator model for more complex language pairs
private val cloudTranslationModel = Firebase.ai(backend = GenerativeBackend.googleAI())
    .generativeModel(
        modelName = "gemini-3-flash"
    )

When a message needs to be translated, we identify the source language and apply our custom routing logic, executing either on-device or cloud translation:

fun translateMessage(message: SupportChatMessage) {
    viewModelScope.launch {
        // 1. Detect language using ML Kit Language Identification
        val sourceLang = try {
            Tasks.await(languageIdentifier.identifyLanguage(message.text))
        } catch (e: Exception) {
            "Undefined"
        }

        // 2. Custom routing: we've verified the translation quality for English and Korean with Gemini Nano, and will translate message on-device for those two languages
        val routeToCloud = sourceLang != "en" && sourceLang != "kr"

        val prompt = "Translate the following text to $selectedLanguage. Just return the translated sentence: ${message.text}."

        val (translatedText, routePrefix) = if (routeToCloud) {
            val result = cloudTranslationModel.generateContent(prompt)
            result.text to "[Cloud]"
        } else {
            val result = hybridTranslationModel.generateContent(prompt)
            result.text to "[On-Device]"
        }

        if (translatedText != null) {
            _translations.update { current ->
                current + (message.id to "$routePrefix: $translatedText")
            }
        }
    }
}

In this example, the custom routing logic only takes into consideration the translation’s source and target language. However, based on your app’s use case, you can expand the routing logic to include other factors such as the on-device model version, network connectivity, battery status, and more.

Securing the AI Pipelines: Firebase App Check

Lastly, using AI in the cloud opens up possibilities of API key abuse or unauthorized billing. To secure API calls, we integrated Firebase App Check using both Play Integrity (production) and the local Debug Provider (for local development or emulators).

In the JetPackerApplication.kt file, we install the debug provider at startup and trigger anonymous authentication to establish a secure user session:

//  implementation("com.google.firebase:firebase-appcheck-playintegrity") 
//  implementation("com.google.firebase:firebase-appcheck-debug")  
//  implementation("com.google.firebase:firebase-auth") 

override fun onCreate() {
    super.onCreate()
    Firebase.initialize(context = this)
    Firebase.appCheck.installAppCheckProviderFactory(
        DebugAppCheckProviderFactory.getInstance()
    )
    Firebase.auth.signInAnonymously()
}

When building locally on an emulator, App Check prints a local token secret to logcat:

Enter this debug secret into the allow list in the Firebase Console: a8c2dd4c-xxxx-xxxx-xxxx-ef6c114ba27e

Once registered in the Firebase console, local requests are fully verified and authenticated by App Check, protecting our backend while letting us test the app locally.

Conclusion

By combining cloud model capabilities (grounding, system instructions) with on-device capabilities (hybrid routing, translation, security app checks), we created a travel app that is smart, secure, and available offline.

Check out the full source code for Jetpacker on GitHub, and explore the Firebase documentation to get started:

Firebase AI Logic Documentation
Firebase Hybrid Inference API

Learn more

Check out the other parts of this blog post series:

Part 1: Introduction of the app and a high-level overview.
Part 2: On-device intelligence. Deep-dive into ML Kit’s GenAI APIs and Gemini Nano to build privacy-first features like itinerary summarization, receipt parsing, and local audio processing.
Part 3 (this post!): Hybrid and cloud reasoning. Explore how to use Firebase AI Logic to ground LLM answers in real-world data like Google Maps and web context.
Part 4: System integration. Integrating with the Android intelligence system using AppFunctions. 
Part 5 (coming soon): In-app agentic workflows. Extend the app with an end-to-end booking assistant powered by A2UI and ADK.

Interested in more on Android Development? Follow Android Developers on YouTube or LinkedIn!

All code snippets in this blog post follow the following copyright notice:

Copyright 2026 Google LLC.
SPDX-License-Identifier: Apache-2.0

Bots (and privacy-preserving browsers) not welcome

Browse a news site in a private window. Shop at a major retailer with a VPN. Visit a video streaming platform with anti-fingerprinting defenses tuned up. You’ll see the same responses: registration walls, block pages, and endless CAPTCHAs. The message is clear: if we think you might be a bot, you’re not welcome.

Websites have valid reasons for wanting to block bots. Bots enable volumetric abuse, abuse that wouldn’t otherwise be feasible if they had to be carried out by humans.  For example: SEO comment spam, credential stuffing and DDoSing. Consequently many sites employ dedicated anti-abuse tooling which aims to keep the bots out whilst minimizing friction for human visitors.

Unfortunately, that tooling is increasingly failing at both tasks. Browser privacy protections are dismantling the passive signals that anti-abuse systems depended on to identify and distinguish visitors. Meanwhile advances in generative AI have rendered CAPTCHAs ineffective: bots now solve them faster and more reliably than humans.

Many sites are switching to more invasive mechanisms and now ask visitors to disclose identifying information, e.g. an email address, a federated login or disabling their VPN. This means greater friction for users, since providing these details on a first visit takes time. It also compromises their privacy, since these details enable the same kinds of cross-site tracking that browser privacy protections were intended to mitigate.

This leaves users with a dilemma. The more effectively they protect their privacy, the harder it is for websites to distinguish them from bots and the worse the treatment they receive. Website operators are also suffering. The additional friction they inflict upon well-behaved visitors harms their site, but many are willing to pay the costs if it mitigates volumetric abuse.

Browser-based AI agents make this tension more acute. Sites may want to allow agents which are acting on behalf of individual users while blocking agents engaged in volumetric abuse. However, with no effective mechanisms to distinguish the two, websites are opting to block both. That hurts users, who should be free to choose the user agent they use to access the web; it hurts new browsers and agents, which struggle to interoperate; and it hurts sites, which lose legitimate visitors.

The consequence is that the web gets worse for everyone. Users get more friction or less privacy or both. Website operators see more volumetric abuse and the friction they add drives away users who would otherwise want to consume their content or services. New user agents struggle to access the same content as conventional browsers.

The Costs of Convenient Solutions

Some large ecosystem players have put forward solutions that leverage their control of the dominant operating systems and their deep integration with consumer hardware. These rely on device attestation: identifiers and privileged code baked into devices at the hardware level, which let manufacturers prove what software is running on a user’s device. Exposing this functionality to the web means attesting to sites that the user is running approved software with trusted hardware and therefore isn’t a bot. There have been two substantive proposals.

Google’s Web Environment Integrity, abandoned in 2023, was the blunt version. It attested to the user agent itself, as well as the operating system and device in use. Users would have lost control in two ways: once to the attester, which would decide which operating systems and devices could be blessed, and again to the website, which would decide which software to accept. If sites had adopted allow-lists of approved user agents, building a new browser would have become virtually impossible, and sites could have withdrawn access from any user agent they chose.

Apple’s Private Access Tokens, deployed across their ecosystem in 2022, have more subtle issues. Built on the Privacy Pass protocol standardized at the IETF, they get a lot right: a user receives a renewed, limited batch of one-time tokens that can be presented to websites without linking their visits together. This provides privacy for users and has shown rate limits to be an effective tool for sites – both points we’ll return to later in this post.

However, Private Access Tokens rely on device attestation, requiring that the hardware manufacturer be in overall control of the user’s device. Presenting a PAT tells a website you are locked into Apple’s rules for what counts as acceptable software. Due to PAT’s technical design^[1], there’s no way to open the system to other sources of scarcity without compromising the system’s privacy properties, meaning that if more widely deployed, access to the web would become tied to having bought expensive hardware from a small, hard to change set of vendors.

Both approaches are ultimately hostile to users and to the openness of the web. Both are premised on parts of a user’s device that sit within the manufacturer’s control and beyond the user’s own. Were they widely deployed, the web would become just another walled garden with centralized gatekeepers controlling acceptable hardware, operating systems and software. As convenient as these solutions are for the players who already dominate the ecosystem, we think there’s a better path.

A Better Path Forward

Bots’ harms arise from their ability to operate beyond human scale. For sites to prevent volumetric abuse they don’t actually need to know the user’s identity or receive cryptographic proof that they’re running approved software. If sites knew their visitors were restricted to a rate limit set by a site, that would be enough.  

Rate limits only make sense if they’re tied to something scarce; something an attacker can’t cheaply replicate to evade the limit. Without anchoring to a scarce resource, like the trusted hardware used in Private Access Tokens, attackers can generate as many fresh identities as they need to bypass the rate limit.

However, hardware is just one option for scarcity. Anything a user already has that an attacker can’t trivially spin up at scale will work: email addresses and phone numbers are naturally scarce. A paid subscription costs an attacker the same as a real user.  Even maintaining an account on a free service requires some non-trivial work.

What if we could use these scarce signals across the web? We could build an open ecosystem with many parties offering scarcity signals, each site choosing which to accept. By opening up who can provide a signal, and letting sites choose which to accept, we can avoid transferring control to device manufacturers and the resulting harms.

As a concrete example of who might be well positioned to provide such a signal, we can consider VPN providers acting as a subscription service. Sites routinely block VPN users indiscriminately, whether through a deliberate policy choice or through an indirect consequence of rate limiting visitors per IP address. But a VPN subscription is a perfect source of scarcity. If the VPN provider could vouch for its users so that sites could rate limit each user individually – then users would be able to browse the web with less friction and without giving up their VPN usage.

The catch is that building a system that can enable this on the open web whilst maintaining user’s privacy is genuinely difficult. It requires that we take information from one site — that this user holds some scarce thing — and expose it to other sites so that they can use that as the basis for their rate limiting. Letting one site verify a signal from another is the sort of information flow that privacy-preserving browsers have spent the last decade locking down to prevent cross-site tracking.

Our goal would be that no more than the minimum information gets through: a single bit communicating whether the user is below the rate limit set by the site. Leaking anything more – like the source of the scarcity that the rate limit is anchored to – would be unacceptable. Enabling a new cross-site information flow might feel like compromising privacy to gain better access, but reality is more nuanced. If a new system moves sites away from demanding that visitors be identifiable (whether through fingerprinting or login forms), it can be a win for both privacy and access.

The Foundations

The good news is that the cryptographic foundations for a privacy preserving approach already exist. The Privacy Pass protocol, originally developed in 2018 to reduce the friction of Cloudflare CAPTCHAs for Tor users, introduced the core primitive: a token that is unlinkable between issuance and redemption. You prove something to an issuer (e.g. by solving a CAPTCHA), receive some tokens, and later present a token to a website. The website can verify the token is legitimate, but can’t link it to the user it was issued to.

Figure 1: In Privacy Pass, a CAPTCHA provider can issue tokens to a client which can then be used to bypass challenges for future site visits. Even if the CAPTCHA provider and sites collude, they can’t use the tokens to identify the user or their browsing history.

Privacy Pass has gone on to be successfully deployed in systems where the issuer and verifier have a prior trust relationship: Apple uses it to authenticate users of Private Cloud Compute and Private Relay without linking their activity to their identity, Chrome uses it for two-hop IP protection, and Kagi uses it to provide private search. These deployments work in part because a small number of parties have agreed in advance on who issues tokens and who accepts them.

Applying this approach to an open system where any site can act as an issuer brings real challenges. Firstly, even though tokens are unlinkable, knowing a user has access to a specific issuer is a privacy leak on its own, because you can infer that the user meets the relevant issuance criteria. If one site can learn that you have a token from another site, that reveals that you have been to that site, which can be a major privacy problem. This compounds if sites can learn the set of issuers you have visited, since it becomes a fingerprint which can be used to identify you.

Generic techniques exist for proving a statement in zero knowledge: we can prove that a client has a token from a set of acceptable issuers without revealing which specific issuer it is. We’ll call this issuer blinding. The generic approach is often slow, but bespoke approaches tailored to the underlying cryptography can improve this considerably.

Another challenge is how sites using rate limits decide who to trust to issue tokens. If an issuer misbehaves then the site’s rate limits become ineffective, enabling volumetric abuse. However, if we need to prevent the site from learning which issuers a user has access to, the site is only going to know that one of its trusted issuers was used, not which one. This makes mistakes or misbehaviour by an issuer difficult to detect, and makes it hard for sites to evaluate new issuers. Solving this challenge is essential for openness. Without adequate information, sites are likely to lean towards conservative issuer selection. That could lead to less choice between Anchors, which in turn could lead to a new form of gatekeeper being created. 

To solve this, sites at least need a way to calculate an aggregate score for each issuer they use. This should roughly correspond to how much of the traffic it considers abusive to have come from users using that particular issuer. Mozilla has long invested in systems like Prio which use multiparty computation (MPC) to protect user privacy whilst enabling aggregate measurements of system behaviour.

Privacy Pass also struggles to handle dynamic adjustments to rate limits. Once tokens have been issued, they’re difficult to invalidate without either revoking all active tokens or risking attacks which can compromise the privacy of users. It’s also beneficial if sites can adjust rate limits on a per client basis, for example by increasing rate limits where they become more confident the client is benign and withdrawing access when abuse is detected.

Anonymous Credit Tokens offer a useful building block to solve this problem. Conventional Privacy Pass schemes rely on issuing a bucket of tokens but ACT works differently by enabling the use of a credential with state. For example, an ACT credential can hold an internal counter. When the credential is presented, the site can check the counter is over some threshold and mutate it, increasing or decreasing the counter whenever the site’s perception of the holder has improved or worsened. Critically, the exact value is never leaked to the site, preventing the site from tracking the holder and ensuring successive presentations of the same credential can’t be linked.

Putting it together

So how can we combine these techniques to build a system which can enable privacy-preserving rate limiting on the open web? In May 2026, we participated in a W3C CG Meeting in collaboration with Cloudflare, Chrome and other web stakeholders in which we started sketching out a design we’re calling PACT – Private Access Control Tokens.

Rate limits need a starting point, a source of scarcity to anchor on. We’ll call an entity that provides such a source an Anchor. To a user who meets the Anchor’s criteria, like having a subscription, an account in good standing, or a verified phone number, an Anchor issues a batch of Endorsement tokens, following the Privacy Pass model. In practice, Anchors could be any website which has access to this kind of signal. An Endorsement conveys scarcity to other sites.

That’s enough for a simple system where access is either granted or denied. But as we discussed earlier, we also want the ability to increase access where a visitor behaves benignly and decrease it where they don’t. The state needed to enforce a rate limit can’t live in the Endorsement, because Endorsements cross trust boundaries between unrelated sites. We need a second object that can hold that state, scoped to the party that maintains it.

We’ll call that the party that handles rate limiting for a site a Moderator and the stateful object a Credential. A Credential is specific to a Moderator and, unlike endorsements, we limit each site to nominating a single Moderator. In the common case the site itself plays the Moderator role, so there’s no new entity or trust boundary. A Moderator can also be a third-party service shared across many sites, allowing those sites to cooperatively share a rate limit. 

In the terminology of the previous section, the Anchor is the issuer of Endorsements, and the Moderator both verifies Endorsements and issues Credentials. A Moderator manages rate-limit policy: it decides which Anchors it trusts, accepts their Endorsements, and issues a Credential in return.

Figure 2: (1) Clients acquire Endorsements from Anchors in the course of normal browsing to sites they have relationships with. (2) Clients can exchange Endorsements for a stateful Credential from a Moderator. (3) Credentials can be used to access sites which use that Moderator. Credentials can be updated over time.

Directly revealing which Anchor backed an Endorsement would leak a lot of information about the user. The issuer blinding techniques from the previous section solve this: when an Endorsement is redeemed, the Moderator only learns that it came from one of the Anchors it trusts, but not which one.

When a Moderator covers more than one site, we let Credentials be presented across all of them but partition cookies and storage as we would for any other third party site. The unlinkability of Credential presentations keeps this from creating a new cross-site identifier. The benefit is that good behaviour on one site improves access on every site the Moderator covers, and bad behaviour cuts it everywhere. Websites can already build the same capability with a shared account system, so this doesn’t create a new way to lock users out, but it does provide a new way to grant access without requiring users to give up their privacy.

Enabling Moderators that cover many sites carries a centralisation risk, similar to the concentration we see today in anti-abuse providers. The mitigation is that the choice of Moderator stays with each site, and the choice of trusted Anchors stays with each Moderator. This can’t reverse the centralisation pressure the web already faces, but it ensures this system won’t lead to additional lock-in: a new Anchor or a new Moderator can be adopted without coordinating with a dominant vendor.

The system then has three flows. First, the user receives Endorsements from an Anchor in the course of normal interaction, based on the Anchor’s positive view of the user. This is a relatively rare operation for any given user and Anchor. After all, as our source of scarcity, Endorsements should not be too easy to accumulate.

Figure 3: In the course of normal browsing, clients browse to websites they have a relationship with. These sites can act as Anchors by issuing Endorsements to clients.

Second, when the user arrives at a site that works with a Moderator, the browser spends an Endorsement from an Anchor the Moderator trusts and receives a Credential in return. The presentation hides which Anchor was used, and neither the Anchor nor the Moderator can trace the Endorsement back to where it was issued. The Moderator decides what initial balance the Credential starts with. If the user has no Endorsements from suitable Anchors at all, existing mechanisms (CAPTCHAs, account creation, federated login) could be used to bootstrap a Credential the same way, so the system degrades to today’s experience rather than locking the user out.

Figure 4: When the client browses to a site, it can prompt the client for a Credential from the Moderator it uses. If the Client doesn’t have a suitable Credential, but does have a suitable Endorsement, it can exchange it for a Credential with the Moderator. In practice, the Moderator and the Site might be the same server.  

Third, as the user browses, the browser presents the Credential and the Moderator updates the internal state of the Credential. The Moderator can reward behaviour that looks benign and penalize suspicious activity, but can’t track the use of the Credential or identify it if it’s used on other sites the Moderator covers. Revocation falls out of the same mechanism: a Moderator can refuse to return an updated Credential. 

Figure 5: The Client can present the Credential on sites which use the matching Moderator. Sites can check if the Credential is in good standing. The sites can then adjust the access the Credential has in response to behaviour. E.g. increasing it when they gain confidence in the client or reducing it in response to malicious behaviour.

In practice, all of this would happen transparently to the user through a WebAPI that sites acting as Anchors or Moderators would call from JavaScript. In an ideal ecosystem, users would accumulate Endorsements through normal browsing, just by virtue of the sites they already visit, and the rest of the flow would happen in the background as they move around the web, leaving users with meaningfully less friction.

AI agents acting on behalf of a user slot into the same flow. An agent can carry its user’s Credentials, in which case the user remains accountable for how the agent behaves. Sites would not need to grant any more access than they would to the user themselves. Alternatively, the operator of an agent can run its own Anchor and vouch for its agents the way other Anchors vouch for human users. Sites retain control over which Anchors they accept, so they can choose how to treat agent traffic without needing a separate detection mechanism.

Several mechanisms combine to keep the information about a user that flows out close to a single bit. Cryptographic unlinkability ensures successive Credential presentations cannot be tied to each other or to the original issuance, so a user’s visits cannot be joined into a history. Each site is bound to a single Moderator, so the set of Moderators a user has Credentials with never becomes a cross-site fingerprint. The Anchor-to-Credential exchange happens in an isolated browsing context, so during ordinary browsing the only thing the site or its Moderator ever observes is a Credential presentation: the site only learns if the user has a valid Credential below the rate limit, or nothing. When the Moderator updates a Credential, it adjusts the credentials state without learning what it is.

The additional privacy given to users from Issuer blinding makes participating in the system more challenging for Moderators. Because the Moderator can’t see which Anchor backed a Credential at issuance, it can’t give a Credential from a strong Anchor more access than one from a weak Anchor: doing so would itself leak which Anchor was used. The initial access has to be uniform across the Moderator’s whole pool of Anchors, which in practice means setting it at the strength of the weakest. However, this is only relevant for that initial access, the Moderator can update credentials according to the holder’s behavior, enabling Credential’s to accrue access over time.

Building an open ecosystem also requires that sites can make effective decisions about the Anchors they choose to trust. Multiparty computation systems like Prio enable aggregate scoring without compromising privacy. When users present Credentials, they can provide an encrypted share which identifies the anchor they used and can be privately aggregated to compute the quality of an issuer.

Next Steps

We think the architecture we’ve sketched for PACT has the right shape, but many of the details still need to be worked out and the entire system needs rigorous privacy and security analysis.

We want to do that work in the open. The IETF is the natural venue for the cryptographic protocols underneath, and the W3C for the WebAPI surface that sits on top. We’ll be bringing draft specifications to these bodies as soon as they’re ready, and we welcome collaborators from across the ecosystem: browser vendors, site operators, anti-abuse providers, and the cryptography community.

If successful, we think we can provide a system which will keep the web open and private, while still giving sites the rate-limiting signal they need.

Acknowledgements

The ideas described here are the result of collaboration and conversations with many people, including: Watson Ladd, Thibault Meunier, Michele Orrù, Trevor Perrin, Eric Rescorla, Samuel Schlesinger, Martin Thomson, Eric Trouton, Benjamin Vandersloot & Cathie Yun. 

The post PACT: Anonymous Credentials for the Web appeared first on Mozilla Hacks - the Web developer blog.

In this post we walk through what folks have found on their journey to learn the Rust programming language with ups and downs covered.

As a disclaimer, LLMs (Large Language Models) come up in this post because our interviewees brought them up. We're scoping discussion to their use as a learning tool, covering research and example generation, not broader questions about AI (Artificial Intelligence) in software development.

Many paths to needing Rust

The interviews surfaced several different paths into Rust: curiosity, embedded work, job-market pressure, organizational adoption, and reassignment after a team or company chose Rust. That last path matters because many learners are not evaluating Rust from a blank slate; they are trying to become productive after Rust has already arrived in their work.

"Funny enough, I've advocated for more niche languages than Rust in the past. Rust has pretty much stopped being as much of a niche language as it was, but it's not Java." -- Fractional CTO

Rust learning resources

Likely as expected, the folks that we talked to reach for a range of resources to learn Rust. Some reach for official documentation, such as The Rust Programming Language Book and find that sufficient to build on what the compiler was already showing them.

"I started with the official Rust documentation because there are a lot of great examples of how features like the borrow checker work." -- Software engineer at an Automotive supplier

Others needed more passes and more formats, sometimes reaching for resources the community maintains, such as Rustlings, The Little Book of Rust Macros, and Learn Rust With Entirely Too Many Linked Lists.

"The first time I went through the chapter in [The Rust Programming Language] on borrow checking, I was like, what is this? I read it again, then I watched a YouTube video of someone explaining the chapter." -- Rust freelance consultant

"Rust book, Rustlings, Zero to Production in Rust, Jon Gjengset tutorials. A bunch of books. It's not a one-pass reading. Can't say how many times I've gone through it." -- Software engineer working on video streaming and storage

These resources have brought up an entire generation of Rust programmers. But, to some, there is a perception that these resources have trouble keeping pace with the language.

"We'd like to use [The Rust Programming Language/'the book'], but we've found that it's out of date, unfortunately. We've looked at the GitHub repo and found it's got a lot of unresolved issues and unmerged PRs" -- Principal Software Engineering work on Rust adoption in a regulated industry

Whether or not this is factually true, Rust's growth has nonetheless put more scrutiny on these materials. Companies evaluating adoption and engineers getting reassigned to Rust teams are looking at them with fresh eyes and finding the gaps that affect their own evaluation.

Beginner stumblings and unlearning habits

It's pretty typical for Rust to be the 2nd, 3rd or Nth programming language that someone picks up. They'd end up writing their most familiar language in Rust, whether C++ patterns, Java patterns, or whatever they knew, for months or even years. Eventually they got comfortable enough to start writing idiomatic Rust.

"There's a bit of a drop in productivity compared to C if you're already familiar with it just because you're learning new rules, new syntax." -- Principal Firmware Engineer (mobile robotics)

"In the beginning it was more poking around the code and adding and removing some ampersands and asterisks to try to make sense of mut and not mut and whatever." -- Senior engineer with 20 years of Java experience in cloud and IoT

We also spoke with someone who found that not having much of a programming background seemed to benefit people picking up Rust. Not having worn-in grooves from other languages may play a role here, and it's worth investigating further.

"I had someone who had never programmed much before start working on the internals of [our Rust project]. She was just fine with getting into Rust. It's more of the senior people that struggle as they need to unlearn practices which may work in other languages, but it's not the 'Rust' way." -- Researcher, Automotive OEM R&D Lab

Learning to work with the borrow checker

We heard a lot about learning to work with the borrow checker instead of against it. People get there through different paths, but a few patterns came up repeatedly.

The compiler as teacher

Rust's diagnostics did the teaching on their own, especially around lifetimes.

"If you mess up the lifetimes in a piece of code that you've written by hand, I usually find that Rust's diagnostics are very helpful" -- Researcher working on static analysis of Rust programs

"Whatever's missing, the compiler usually fills in: it tells me 'you need to declare the lifetime of this reference', so I know and can figure it out. That all generally works pretty well." -- Senior Software Engineer

Learning by doing

Others felt like they only really internalized the borrow checker after writing a lot of Rust. It took projects, coding challenges, prototyping and so on until at some point it clicked.

"I actually did not understand the borrow checker until I spent a lot of time writing Rust" -- Founder of a startup built on Rust

"Besides the prototyping work, I also did coding-challenge-type stuff to get familiar with Rust for Advent of Code. [..] It eventually clicked to the point where I wasn't fighting with Rust, it was working for me. I had that experience other people describe: when I managed to get my program to fit with Rust, it worked. I didn't spend time debugging." -- Principal Software Engineer, large SaaS provider

Letting go of "clone guilt"

Some learners arrive with the assumption that good Rust means zero clones, zero copies, lifetimes threaded through everything. They set the bar at optimal before they've learned how to write idiomatic Rust, and it makes the borrow checker feel harder than it needs to be at the outset.

"On one of my first projects, I was like, 'I don't ever want to copy or clone anything,' so I carefully wove through all the lifetimes and got myself into a bit of a bind. Then I saw someone else just cloning the struct I was working with, and it was super cheap. Sometimes you can just clone and it's going to be okay." -- Researcher at a university

The experienced Rust developers we spoke with consistently said the same thing: clone freely while you're learning, then optimize when you understand the problem. Rust's reputation for performance and correctness feeds this. Newcomers assume anything less than optimal is wrong before they've written a first working program, and clone guilt is how that shows up.

We think it could be an interesting area of future study to check into the patterns Rust programmers employ at different levels of experience and under which circumstances. One member of the Rust Vision doc team that's very experienced with Rust noted that there's kind of an "expected shape" they understand as passing the compiler. This knowledge influences how they approach writing code which wouldn't take that shape and they naturally find themselves understanding when to use so-called workarounds, such as passing around indices into arrays or Vecs.

Multi-paradigm, but not the OOP some are used to

The Rust programming language is multi-paradigm, and how that lands depends on what you're coming from. We heard some that came from a functional background were delighted with digging into learning how much Rust inherits from that lineage. Some others noted that they and others on their teams struggled to unlearn the object-oriented style they'd come to use heavily in other languages like C++ and Java.

"Developers coming from C++ tend to think object-oriented. I think that's a difference between C++ and Rust." -- Architect at Automotive OEM

"I had exactly that thing, where I would apply all my years of Java and JS thinking, where I could just create some object, not care about it, return it, have it sloshing around between various functions. Found myself reaching for these patterns and then being told 'no, you cannot do that'." -- Principal Engineer at a SaaS company

Developers coming from functional programming had less to unlearn: strong typing, pattern matching, and an expression-oriented style were already familiar.

"My background has been more functional programming, strong typing. That originated for me as a Lisper: once a Lisper, always a Lisper." -- Principal Software Engineer working on Rust tooling for safety-regulated industries

"The languages I primarily used before Rust were things like OCaml. Way back, I came from C and C++, the classic languages, and then I spent quite a long time doing primarily pure functional stuff. These days I've ended up back in what I like to think of as a pragmatic center ground [with Rust]." -- Fractional CTO

Teaching Rust in academia

We spoke with a university professor that's been teaching Rust generally. In the academic environment, they were able to use proxies for some things such as "traits are like interfaces in Java" because the students had already gone through a set of courses in their first and second years that taught them Java. They introduced concepts slowly throughout the course, choosing to deal with some more complex topics like generics later. The outcome generally was that students had no problem picking up Rust in this setting.

"I couldn't see any big difference on the embedded side. We also teach an embedded class, and we did an experiment. Half of the students' feedback was worse on the Rust class, mostly because they needed to build the project themselves. The C students just got one from [an LLM], absolutely no problem." -- University Professor, on teaching Rust

The C cohort leaned on LLMs for the project in ways the Rust cohort couldn't. We don't yet have a clear answer for why.

What did come through clearly was the Rust cohort's experience with the community. Some students needed to figure out which drivers to use for the embedded project and how to use them. Their professor encouraged them to open issues and ask questions directly on GitHub, and the maintainers responded. Students who had never contributed to open source before were getting answers from the people who wrote the code.

Learning using LLMs

Some experienced folks shared that they saw LLMs as a tool that can help someone come up to speed quickly, either as a research tool or for generating example Rust code to understand concepts.

"I'm optimistic that there's a way to work [LLMs] in that will cut down that learning curve. One of the big things these tools bring is reducing the learning curve in general; these are very good tools to help you navigate a space that you don't know yet." -- Maintainer of large open source Rust crate

"I try [LLMs] out once a month, usually for generating an example or something like this. Just like with Stack Overflow: when you read an example, you should read it carefully and try to understand it. Not copy and paste it, but type it in your own words in code and then check it, because that's where the teeny tiny little mistakes are." -- Founder of startup built on Rust

For some learners, an LLM is just another way to find answers, no different than a search engine.

"So for the most part, picking up Rust - how do I learn? I'll [use web search for] things, I'll ask [an LLM], I'll just poke around and read the code." -- Senior Software Engineer working in a regulated space

One founder went further and claimed that LLMs change who can become a Rust developer. One consulting company founder described hiring high school graduates with no systems programming background and training them as Rust developers, with LLMs filling in the learning gaps that would previously have required years of experience.

"At the beginning, I was worried, but now that we have [LLMs] supporting development, the difficulty of the language doesn't matter. I'm seeing a huge opportunity behind strong runtime languages like Rust. [..] In [Developing Country] we hire 20-25 high school graduates, train them to be Rust programmers, then they enhance our workforce worldwide." -- Founder of a consulting company

We heard this from one organization. This is a claim that the combination of Rust's compiler and LLM tooling can dramatically shorten the path from beginner to working developer. Whether it generalizes depends on questions we can't answer from a single interview: how long these developers stay, what kind of code they can maintain independently, and whether this training/learning model works outside this company's particular structure. If it holds up, the pool of people who can become Rust developers is much larger than the usual hiring profile suggests.

Organizational considerations for Rust learners

We spoke with a number of folks on teams that are using Rust in larger organizations. Teams wanted to know that everyone would end up at roughly the same level of competence, which led a good number to invest in training courses to get there. Some leaders found that staff was able to ramp well enough by reading The Rust Programming Language, going through Rustlings, and then picking up lower risk and priority tickets to work on. Having a sense of community was also important within companies; it helps people know they are not alone when they are asked to work on Rust after, say, a reorganization happens.

"[..] the idea with the class as opposed to 'just read the Rust book on your own' was that this gives everyone kind of the same baseline going in." -- Principal Firmware Engineer (mobile robotics)

"So typically we're going to have people work through Rustlings, work through The Rust Programming Language. We have them then start to pick up lower risk tickets to work on." -- Principal Engineer at a large SaaS provider

"We've got an internal Slack channel for Rust learning where people can drop questions and others will come in and answer them. That helps build up understanding and community." -- Software Engineer at a large corporation

Some organizations found that while the person they'd hire would need to learn Rust, it was still preferable to the alternative of hiring someone for a critical piece of software written in another language.

"They needed to grow and maintain this C++ codebase. They had a C++ wizard, and they tried for about two years to find someone with the same level of expertise. They ended up hiring people that didn't know Rust and ramping them up, creating FFI bindings from the C++ side so they could work in Rust. And you can feel it: the borrow checker is teaching these people the right way to handle their systems." -- Principal Engineer at an Automotive OEM

The community and helping each other aspect seems to grow bonds as organizations mature.

"Our team is [all about] mentorship. I've mentored people coming up to speed on Rust, and people help each other hugely." -- Principal Software Engineer at a large SaaS company

Silent attrition

We identified some cases where people have approached Rust and bounced off of it, for one reason or another. In the below case, someone with a background in a language with fewer guardrails found themselves frustrated enough with Rust to walk away.

"All of that means that that embedded ecosystem is very frustrating to somebody who comes from C and is like, why can't I just get a pointer to this peripheral and then write into the registers. What are you doing to me? [..] My friend never got over that. He looked at it and said, I'm not going to deal with this and walked away." -– A second University Professor

There may be language features that for a particular domain are not seen as comfortable or usable yet, such as async Rust usage in a safety domain. We'd like to map which language features feel off-limits in which domains; async in safety-critical work probably isn't the only case.

"We're not fully sure how async [Rust] will work out in the long run in our domain. [..] People don't feel comfortable yet since C++14 doesn't provide such concepts. [..] It's the chicken-and-egg problem again: we probably need to gain some experience to see whether we can actually benefit from these new concepts in the automotive and safety domains." -- Team Lead at Automotive Supplier (ASIL D target)

We heard in at least one case, that while the language was challenging and there was a near bounce, the tooling helped keep them coming back and trying.

"Well, I think my early impressions of Rust - one is I find C++ so intimidating, and I think a big part of why I was able to succeed at [..] learning Rust is the tooling. I mean, all this makes sense [..] but it's like, for me, getting started with Rust, the language was challenging, but the tooling was incredibly easy." -- Founder of another startup built on Rust

While it might be considered more of a community concern, if there are interactions online and in spaces that point to learners having so-called "skill issues" this feeds into the narrative that Rust must be hard to learn. We may be unintentionally turning away Rust Project contributors and maintainers due to the vibes being put out when new learners show up in certain spaces.

"People are very helpful, but generally the attitude is: if your program is very complicated, it's mostly a skill issue. There's not that much empathy when people get stuck learning, and a lot of people are just pushed away by it. There's probably a huge number of people who silently stop wanting to write Rust, because at some point it gets complicated and the feedback they get is 'you just need to be a better programmer, obviously'." -- Software Engineer at a SaaS Provider

Feedback on near-bounces from survey

We found a few interesting perspectives collected in the Rust Vision doc survey which we administered with examples of bouncing and coming back:

"I started before 1.0, got stuck very soon when trying to translate patterns from C++ to Rust (due to borrow checking). I tried again after 1.0 and it stuck. [..]" -- Survey Respondent A

Survey Respondent A went on to share in a more detailed response about a perceived weakness in Rust learning materials related to lifetimes and the borrow checker are explained. There was an observation that it's fairly easy to run into more complex situations with lifetimes and the borrow checker. They felt that the current state of this sort of material and tutorials is fairly superficial and can leave learners stuck when they run into those more complex situations.

One respondent that bounced once and came back shared challenges around usage of async. In concert with Rust's memory-safety and the borrow checker, they found some of the nitty-gritty details of async were difficult to learn. While we're aware of the Rust Project's continuous efforts to improve Rust's async story, this is another data point of a user that faced challenges.

Another survey respondent shared how they had multiple times bounced in trying to learn Rust. They returned after a year or so and found Rustlings to be highly motivating. We note that having multiple pathways for folks to learn Rust opens up more possibilities for those that nearly bounced, just like this person.

Need more focused work on silent attritrion

The thing that stood out most to us was the lack of real, first-hand knowledge of having bounced when learning Rust. While this is an obvious effect of soliciting answers to our survey and opportunities to interview through Rust channels and our networks, this cohort is good future candidate where interviews could start.

Conclusions

Across these conversations, the experience of learning Rust depended heavily on context. Why someone was learning and what support they had mattered as much as the borrow checker. The same kinds of examples kept coming up: a training course that got a team to a shared baseline, a maintainer answering a student's first GitHub issue, and a colleague whose code showed that cloning was okay.

That context is largely something the community has a hand in. With that in mind, here is what we take away from what we heard, and what we still don't know.

What seems worth trying

Learning materials aimed at unlearning. Syntax barely came up when people described their struggles. People struggled with unlearning habits from previous languages, whether OOP structuring from C++ and Java or the instinct to grab a raw pointer to a peripheral. Most of our learning materials teach Rust from first principles, and that works. What we didn't come across is much written for, say, the engineer with ten years of Java who lands on a Rust team after a reorg: material that names the patterns they'll reach for that won't transfer, and shows what to do instead. The professor we spoke with did a version of this in the classroom, leaning on "traits are like interfaces in Java" and saving generics for later in the course, and the students did fine. Something similar could work outside the classroom too.

Put the "clone freely while you're learning" advice somewhere official. Every experienced developer we spoke with gave the same advice, but learners seem to mostly pick it up by accident, like the researcher who happened to see someone else cloning the struct they had been carefully threading lifetimes through. Saying it early in official materials would take some of the steepness out of the curve. The broader version belongs there too: idiomatic Rust doesn't have to mean optimal Rust, especially on a first project.

Diagnostics are already a primary learning resource: several people told us the compiler taught them lifetimes before any documentation did. Diagnostics reach learners right at the moment they're stuck. When writing new ones, it seems worth keeping the confused newcomer in mind alongside the expert, because for a lot of people this is where the learning happens.

Is "the book" actually out of date? Whether or not The Rust Programming Language or other materials are actually behind, a team evaluating Rust looked at its repository, saw unresolved issues and unmerged PRs, and moved on. As more companies evaluate adoption, more people will look at these materials with the same fresh eyes. Visible issue triage and some communication about what's current and what's planned would address the perception, separately from whatever content work may or may not be needed.

How stuck learners get treated is shaping who stays. We heard about students getting answers on GitHub from the maintainers who wrote the code, and we heard about learners being told their struggles were a skill issue. The first group came away with a lasting good impression of Rust. Some of the second group walked away entirely, and because they leave quietly, it's easy to underestimate how many of them there are. The welcoming side of the community came up unprompted as a reason people stayed, so we know it makes a difference when we get this right.

Every organization we spoke with described essentially the same ramp-up for bringing a team to Rust. Teams that brought groups of developers to Rust described roughly the same approach: get everyone to a shared baseline with a training course or with The Rust Programming Language and Rustlings, start people on lower-risk tickets, and give them somewhere internal to ask questions. Several organizations also found that hiring developers without Rust experience and ramping them up worked out better than continuing to search for rare expertise in another language. None of this is complicated, and teams weighing adoption don't need to invent a training program from scratch.

What we still don't know

The biggest gap is the people we didn't reach. Nearly everyone we spoke with stuck with Rust long enough to be reachable through Rust channels, so the stories of bouncing off came to us second-hand: a friend who walked away from embedded Rust, colleagues who quietly stopped after the responses they got. As we wrote in our first post, finding people who decided against Rust takes targeted outreach. If the proposed User Research team comes together, talking with learners who bounced would make a good early project, and learning is probably the area where that research would teach us the most.

We also don't know what to make of LLMs as a learning tool yet. They came up as a search engine, as an example generator, and in one organization's case as something that makes training high school graduates into working Rust developers possible. We saw a classroom where the C cohort leaned on LLMs in ways the Rust cohort couldn't, and we don't have an explanation for it. All of this comes from a handful of conversations, so we treat it as a set of leads to follow up on. Given how quickly the tools are changing, it seems better to study this deliberately than to wait and see what folklore develops.

The folks we spoke with showed that people do get there: with enough passes through the materials and enough code written, it eventually clicks. The opportunities above are mostly about making it work for the people who didn't pick Rust on purpose, and for the ones who would have stuck around if their early experience had gone a little differently.

Source Code Viewer

Crate pages now have a "Code" tab that lets you browse the contents of published crate versions directly on crates.io. This shows you the exact files that cargo downloads when you add a crate as a dependency, which might differ from the linked repository. This makes it much easier to audit your dependencies, including files that never appear in the repository, like the normalized Cargo.toml files that cargo generates.

The viewer comes with a file tree sidebar with search functionality, syntax highlighting, and GitHub-style line selection, where clicking or dragging line numbers produces shareable #L10-L20 URLs.

Under the hood, the server now builds a zip file for every published version. Since the .crate files that cargo consumes are gzipped tarballs without random access support, a background job re-packs each of them into a seekable zip archive plus a JSON manifest describing the contained files. Both are served from our static CDN. The frontend then fetches only the manifest and loads each file on demand with an HTTP range request. Because of this architecture, browsing crate sources essentially adds no load on the crates.io API servers. Existing crate versions have been backfilled, so this works for old releases too.

The rendering library behind the code viewer is a diff renderer at heart, and that's no accident: a version-to-version diff viewer built on the same infrastructure is currently in the works. This will allow you to review exactly what changed between two published versions, right on crates.io. Stay tuned!

Untangling crates.io Accounts from GitHub

At the end of May, the crates.io team accepted RFC #3946. Crates.io accounts always have been tightly coupled to GitHub: signing in means "Log in with GitHub", and your crates.io identity is your GitHub username. The RFC changes that. It introduces usernames that are native to crates.io and independent of linked GitHub accounts, as a prerequisite for eventually supporting login via other identity providers.

The implementation of crates.io usernames has started, but there is still a lot left to do, most visibly the ability to change your crates.io username. After that is complete, there will be future RFCs and implementation for signing in with identity providers other than GitHub. Since all of this touches authentication and account security, we are deliberately taking it slow and rolling these changes out in small, carefully reviewed steps.

Advisories and Suggestions

In our January update we introduced the "Security" tab, which shows security advisories from the RustSec database. We have since taken this integration one step further: crates that RustSec has flagged as unmaintained now show a warning banner directly on their crate pages, linking to the corresponding advisory for details and possible alternatives. Thanks to Dirkjan Ochtman for implementing this feature!

Related to this, some popular crates have been largely absorbed into the Rust standard library over the years, like lazy_static, which has been superseded by std::sync::LazyLock since Rust 1.80. Crate pages of such crates now show a friendly "You might not need this dependency" banner describing the standard library replacement, and superseded crates in dependency lists get a small light bulb icon with a similar hint.

The dataset behind this feature lives in the new rust-lang/std-replacement-data repository, together with a documented inclusion policy: standard library replacements only, every entry must cite the stable std, core, or alloc API and Rust version, and crate maintainers get a notice-and-comment window before an entry is added. New entries can be proposed upstream and can benefit other tools too.

Ferris

The most delightful change of this cycle: the Ferris on our error pages now follows your mouse cursor with its eyes:

Getting a 404 error on crates.io is now slightly less sad.

Svelte Frontend Migration Completed

In our January update, we announced that we were experimenting with porting the crates.io frontend from Ember.js to Svelte. This experiment has concluded successfully: the new frontend reached feature parity, went through a public testing phase in April, became the default at the beginning of May, and the Ember.js app has been removed from our repository.

We designed this change to be invisible for our users, since the new frontend is a 1:1 port of the previous design and functionality. For the team and our contributors, however, it is a big deal: the frontend is now built on a more modern framework, which should make it easier for new contributors to get started. It also allows us to iterate faster, as the source code viewer above demonstrates.

We want to thank the Ember.js team for a framework that served crates.io well for many years, and the Svelte team for making the transition so enjoyable.

Miscellaneous

These were some of the more visible changes to crates.io over the past six months, but a lot has happened "under the hood" as well:

• Search performance: Relevance-sorted search queries previously ranked every crate matching the query, which could take 1-2 seconds for short or common search terms. Ranking is now bounded to the 1,000 matching crates with the highest recent download counts.

• Reverse dependencies performance: The reverse dependencies endpoint no longer recomputes the full dependent set on every request. It is now served from a precomputed table kept in sync by database triggers, turning an expensive join into a bounded index scan and greatly reducing the chance of getting a timeout error.

• New ARCHITECTURE.md: If you've ever wondered how crates.io actually works, our ARCHITECTURE.md document got a complete rewrite. It is now organized around the high-level systems that make up crates.io and how they fit together, and includes walkthroughs of what happens when you run cargo publish, why a typical crate download never touches our API servers, and how download counts are derived from CDN access logs.

• Definition lists: READMEs now render Markdown definition lists, a widely used Markdown extension. Our markdown renderer comrak already supported them, the extension just wasn't enabled yet. Thanks to @mistaste for this contribution!

• CDN cache tags: Files uploaded to our static CDN now carry cache-tag metadata, allowing us to invalidate all cached files of a crate or a specific release in a single operation, instead of issuing one invalidation per file URL.

• Caching improvements: We removed a global Vary: Cookie response header that was preventing our CDNs from caching public API responses and frontend assets effectively. Per-user responses now use Cache-Control: no-store instead, resulting in better cache hit rates at the CDN edge.

• Accessibility: We have made crates.io friendlier to screen readers: decorative icons are now hidden from the accessibility tree, heading hierarchies have been fixed, and lists are marked up as proper lists. ARIA snapshot tests now ensure that regressions can't slip in unnoticed. We plan to continue to improve crates.io accessibility over the coming months.

• Git index performance: The background worker's local clone of the git index is now a bare and shallow repository, eliminating roughly 250,000 checked-out files and the full commit history from its disk, improving its performance as we see increased rates of crate publication. The periodic index squashing now goes through the GitHub API instead of generating large git packs locally, which had previously caused out-of-memory failures on the production worker.

Feedback

We hope you enjoyed this update on the development of crates.io. If you have any feedback or questions, please let us know on Zulip or GitHub. We are always happy to hear from you and are looking forward to your feedback!

Huawei may have just become NVIDIA’s biggest problem in China.

In this video, we break down Huawei’s new Ascend 950PR AI chip, why it matters for the global semiconductor war, and how U.S. export restrictions may have accelerated China’s push toward technological independence. The Ascend 950PR is designed mainly for AI inference and is built by SMIC using advanced DUV multi-patterning. Huawei is also attacking NVIDIA’s biggest advantages beyond hardware by building a CUDA compatibility layer through CANN and scaling thousands of Ascend NPUs together inside the Atlas 950 SuperPod using UnifiedBus interconnect technology. We also look at reported demand from ByteDance, Alibaba Cloud, and Tencent, the role of DeepSeek models optimized for Ascend hardware, and why NVIDIA could lose major market share inside China even without Huawei beating its most advanced chips in every benchmark. This is bigger than one AI chip. China is building a parallel AI ecosystem across chips, software, networking, cloud infrastructure, and AI models — and Huawei is quickly becoming the center of it.

#Huawei #NVIDIA #AIChips #Ascend950PR #ChinaTech #Semiconductors #ArtificialIntelligence

NVIDIA just unveiled Rubin, but this isn't just another GPU. It's a complete AI factory built to power the next generation of artificial intelligence. In this video, we break down NVIDIA's Vera Rubin platform, a fully co-designed AI system that combines the Vera CPU, Rubin GPU, NVLink 6, BlueField-4 DPU, ConnectX-9 SuperNIC, and Spectrum networking into one massive AI infrastructure platform. You'll learn why Rubin delivers up to 50 petaflops of AI inference, 288GB of HBM4 memory per GPU, 3.6 exaflops of compute per NVL72 rack, over 260TB/s of NVLink bandwidth, and why NVIDIA claims up to 10x lower AI inference costs than Blackwell. We also explore why AI companies like Microsoft, OpenAI, Anthropic, Meta, xAI, Google, AWS, Oracle, and CoreWeave are already adopting Rubin, and why NVIDIA believes the future of AI isn't a faster GPU; it's an entire factory built to manufacture intelligence. Is Rubin the biggest leap in AI hardware yet, or the beginning of a completely new era of AI infrastructure?

#NVIDIA #Rubin #AIFactory #AIChips #ArtificialIntelligence #HBM4 #DataCenter

AMD may have finally built a real challenger to NVIDIA’s AI empire. In this video, we break down the AMD Instinct MI400 series and the flagship MI455X AI accelerator, featuring 432GB of HBM4 memory, 19.6TB/s of memory bandwidth, up to 40 petaflops of FP4 compute, and an advanced chiplet architecture designed for next-generation AI training and inference. But AMD’s real weapon is bigger than one chip. The Helios rack-scale AI platform combines 72 MI455X GPUs with next-generation AMD EPYC “Venice” CPUs, massive HBM4 capacity, high-speed networking, ROCm software, and open technologies like UALink to challenge NVIDIA’s tightly integrated AI infrastructure. We also explore how AMD plans to compete with NVIDIA Vera Rubin, the importance of ROCm versus CUDA, and why major AI companies and cloud providers are increasingly looking for alternatives to NVIDIA. Could AMD finally turn the AI accelerator market into a real two-company war?

#AMD #NVIDIA #MI455X #AIChips #InstinctMI400 #Helios #ArtificialIntelligence

NVIDIA just revealed the most powerful AI workstation ever built—and it puts data center hardware on your desk. Powered by the new GB300 Grace Blackwell Ultra Superchip, the NVIDIA DGX Station combines a 72-core Grace CPU, a Blackwell Ultra GPU with 20,480 CUDA cores, 748GB of unified coherent memory, and up to 20 petaflops of AI compute. It's designed to run massive AI models locally, eliminating many of the memory limitations that force developers to rely on expensive cloud GPUs. In this video, we break down the DGX Station architecture, unified memory, NVLink C2C, HBM3e, local AI inference, trillion-parameter model claims, real-world pricing, and why NVIDIA believes desktop AI workstations are the future of artificial intelligence development. We also compare the DGX Station with DGX Spark, Apple’s Mac Studio, cloud GPU infrastructure, and explain why local AI could become the next major shift in computing.

Is NVIDIA reinventing the personal computer for the AI era?

#NVIDIA #DGXStation #Blackwell #AIWorkstation #ArtificialIntelligence #LocalAI #CUDA

AMD may have just launched the biggest challenge NVIDIA has faced in years. Helios isn't just another AI GPU it's AMD's first full rack-scale AI supercomputer, powered by 72 Instinct MI455X GPUs, 18 EPYC Venice CPUs, and an incredible 31TB of HBM4 memory. Built to compete directly with NVIDIA's Grace Blackwell and Vera Rubin platforms, Helios is already being deployed by Microsoft, Meta, Oracle, OpenAI, and other AI giants. In this video, we break down AMD Helios, Instinct MI455X, EPYC Venice, HBM4 memory, Helios vs NVIDIA Vera Rubin, Microsoft's Azure deployment, Meta's AI infrastructure, ROCm, CUDA, AI data centers, and why AMD believes it can finally challenge NVIDIA's dominance in the AI chip market. Is Helios the system that finally gives NVIDIA real competition?

#AMD #Helios #NVIDIA #AI #ArtificialIntelligence #MI455X #Technology #Semiconductors

Read more of this story at Slashdot.

VentureBeat Research measured the five controls an enterprise has to build before it can trust an agent: identity, evaluation, cost telemetry, the context layer, and orchestration. Identity governs which agent is allowed to do what, under whose credentials. Evaluation determines whether the agent's work is any good. Cost telemetry tracks what each agent costs to run. The context layer supplies the business data and definitions agents draw on when they answer. And the orchestration control plane coordinates multi-step agent work. Each of our five reports measures one of those controls.

Most deployed "agents" are chatbots wearing the label. Seventy-one percent of enterprises said a quarter or fewer of their deployed "agents" can complete multi-step work on their own; only 10% said true agents are the majority of what they run. These respondents are positioned to know: 81% recommend or decide AI purchases at their companies. A single-prompt chatbot with a human reading every answer needs none of the controls the other four reports measure. A true multi-step agent needs all of them — and most enterprises can't say which one they've deployed. (Full findings: Agentic Orchestration report.)

Autonomy is outrunning trust in the evaluations that gate it. Two-thirds of enterprises either already allow an agent to push a code or system change to production on automated evaluation results alone, with no human review, or are actively engineering toward that within 12 months. Only 5% fully trust the evaluations that would make that call — and half of enterprises shipped an agent that passed internal evaluations and then caused a customer-facing failure in the past year. Before removing human review from any workflow, test evaluations against production outcomes rather than internal benchmarks. (Full findings: Agent Reliability & Evals report.)

Companies that let agents share credentials get hit more often. Sixty-nine percent of companies let at least some of their agents share credentials — multiple agents operating under one API key or service account. Organizations that allow credential sharing anywhere experienced a security incident or near-miss at a 63.5% rate (47 of 74), against 40.9% (nine of 22) at companies where every agent has its own scoped identity. The fix is scoped identity for every agent, starting with the ones that touch production systems. (Full findings: Agentic Security & Identity report.)

The most expensive hardware in the building runs at half capacity or less. More than eight in 10 enterprises that run their own GPUs reported utilization of 50% or less, and only 44% rigorously track what their AI compute actually costs and returns. The number worth chasing first isn't more GPUs — it's the utilization and per-workload cost of the ones already running. (Full findings: AI Infrastructure & Compute report.)

Agents answer confidently from data nobody governs. Fifty-seven percent of enterprises traced a confident, wrong agent answer in the past six months to their own missing or inconsistent business context — wrong metrics, stale definitions, absent documents — and most saw it happen more than once. Governing the definitions agents answer from — metrics and entities first — has to come before scaling the agents that depend on them. (Full findings: Context Layers / RAG report.)

No layer has an entrenched incumbent: The defaults today are the built-in tools that ship with the big AI platforms enterprises already use. Switching intent runs highest in orchestration itself, where 68% plan to adopt, add, or replace platforms within 12 months and 34% within the quarter. Our surveys did not ask which direction that money moves — toward the platforms' built-in tools or toward the specialists challenging them — and that open question is the next four quarters of this market.

About this research

VentureBeat Research fielded five parallel surveys in June 2026 under its VB Pulse program: Agentic Orchestration (101 respondents), Agent Reliability & Evals (157), Agentic Security & Identity (107), AI Infrastructure & Compute (107), and Context Layers / RAG (101) — 573 qualified respondents in total, all at organizations with 100 or more employees. Samples are self-selected, and some findings should be read directionally; each report carries its full methodology note. What the pattern supports more strongly than any single percentage is the direction: every survey, independently, points the same way. VentureBeat produces both this research and VB Transform, the conference where these reports debuted.

The model, available immediately on all of Anthropic's platforms, is priced at $5 per million input tokens and $25 per million output tokens, unchanged from its predecessor, Opus 4.8. It becomes the new default model on Claude Max, Anthropic's premium consumer tier, and the strongest model available on Claude Pro.

The positioning is deliberate. Anthropic is not claiming Opus 5 is its smartest model — that distinction still belongs to Fable 5, and rival systems retain an edge in certain domains. Instead, the company is making a subtler argument that may matter more to enterprise buyers: that the most economically important AI work happens in a middle band of difficulty, where near-frontier intelligence delivered efficiently and cheaply beats frontier intelligence delivered expensively.

"Opus 5 as your daily driver, the model you hand complex work to and review when it's done," an Anthropic spokesperson said in an interview with VentureBeat, describing how the company's lineup now stratifies. "Fable 5 for your most ambitious work, the days-long autonomous projects nothing could take on before... Sonnet 5 for work you run at scale, where speed and cost per call decide what ships. Haiku 4.5 for subagents and instant answers."

How Claude Opus 5 benchmark results stack up against Fable 5 and rival AI models

On paper, the results are striking. Anthropic says Opus 5 sets new state-of-the-art marks on coding and knowledge-work evaluations including Frontier-Bench and GDPval-AA. On Frontier-Bench v0.1, an agentic terminal coding benchmark, Opus 5 scores 43.3 percent — more than double Opus 4.8's 18.7 percent and well ahead of Fable 5's 33.7 percent — at a lower cost per task, according to the company. On ARC-AGI 3, an evaluation of novel problem-solving, Anthropic reports Opus 5 scored three times as high as the next best model. On OSWorld 2.0, a computer-use benchmark, the company says the model surpasses Fable 5's best result at just over a third of the cost.

The numbers come with honest caveats that are themselves notable in an industry prone to superlatives. Anthropic acknowledges Opus 5 remains behind Mythos 5, a competing model, on cybersecurity tasks and biology research, and an OpenAI-family model still leads on one agentic coding benchmark.

The more revealing caveat came from Anthropic itself, when asked where Opus 5 still falls short of Fable 5. The spokesperson's answer amounted to a candid admission about what benchmarks do and don't capture.

"The evals where Opus 5 wins are bounded tasks with a specific outcome, which is where it's strongest. What those evals don't measure is duration," the spokesperson told VentureBeat. "One way to put it: Opus 5 is the best tool for the jobs benchmarks can see, and Fable 5 is what you reach for when the job outruns the benchmark."

Fable 5, by contrast, "is for the longest, most autonomous jobs, where the model has to stay coherent across many connected steps over hours or days with dense source material," the spokesperson said, advising customers to "run both on a representative workload, one bounded task and one long-horizon job." That framing — bounded tasks versus long-horizon autonomy — may become the defining axis of model differentiation in 2026, as benchmarks saturate and the hardest remaining problems involve sustained, multi-day agentic work rather than discrete puzzles.

Why token efficiency is becoming the real battleground for enterprise AI spending

Threaded through the launch is a theme Anthropic clearly wants buyers to absorb: Opus 5 doesn't just score well, it scores well per dollar. The model ships with an adjustable "effort" setting that lets customers trade intelligence for speed and token savings, and Anthropic's charts emphasize performance at a given cost rather than peak performance alone.

Early customers echoed the point with unusual specificity. Harvey, the legal AI company, said Opus 5 achieved similar performance to Opus 4.8's maximum-reasoning mode "while generating 26% fewer tokens on average," according to Niko Grupen, its head of applied research. Richard Pham of Fundamental Research Lab said that on hard financial-modeling tasks, the model averaged nine percentage points higher accuracy "while using roughly one-third fewer turns and tool calls and 60% less time."

Wade Foster, chief executive of Zapier, said Opus 5 topped his company's AutomationBench leaderboard "without spending more tokens than prior Claude models," running a full churn-prevention workflow from start to finish. "Previous models didn't pass; Opus 5 hit 100%," he said. Scott Wu, chief executive of Cognition, the company behind the Devin coding agent, said that on FrontierCode 1.1, "Claude Opus 5 approaches Fable-level performance at half the cost," with particular strength in debugging and root-cause analysis.

The efficiency emphasis reflects commercial reality. Enterprise AI spending is no longer experimental, and inference costs — the price of actually running these models at scale — have become a board-level line item. 

Anthropic's business skews heavily toward API and enterprise usage; according to a February 2026 analysis by Contrary Research, Claude held roughly 40 percent of the enterprise large language model market by usage as of late 2025, and Claude Code alone had reached about $1 billion in annualized revenue. For a company whose customers pay by the token, a model that does more with fewer tokens is not a nice-to-have. It is the product.

Self-verifying AI agents and what they mean for the hidden costs of automation

Beyond the numbers, Anthropic is selling a behavioral story: that Opus 5 verifies its work and iterates until it succeeds. The company offered several examples from testing that read like small parables of machine stubbornness.

In one Frontier-Bench task, the model was asked to reconstruct a machine part as a 3D CAD model from a drawing it was intentionally given no way to view. Rather than fail, Anthropic says, Opus 5 wrote its own computer vision pipeline to extract the geometry from raw pixels — and did so repeatedly, while no competing model solved the task in five attempts. In another case, given a real bug in a popular open-source package manager, the model found the root cause and fixed an edge case the community's own patch had missed; a competing model patched only the symptom and declared victory. An engineer at a trading firm, the company says, used Opus 5 to build a market data feed for a new exchange in a single session and, finding no live feed to validate against, watched the model build its own test harness to check its parsing code.

Customers described similar behavior in the wild. Cristian Rivera, a staff software engineer at Stripe, said he gave the model "a chief-of-staff role over my dev environments" for a weekend: "it built its own monitor, drove each box, and pulled me in only for the judgment calls."

This is the capability enterprises actually care about, and it is worth dwelling on why. The gap between a model that produces plausible output and one that verifies its output is the gap between a demo and a deployable system. Most of the hidden cost of enterprise AI today is human review — engineers checking the machine's work. A model that reliably checks its own work compresses that cost, which is precisely why customers keep citing fewer turns, fewer passes, and less time rather than higher raw scores.

Inside Anthropic's safety strategy: capability gaps, classifiers, and model fallbacks

The launch also showcases Anthropic's increasingly intricate approach to safety — one that now involves deliberately not teaching its models certain skills. The company says its automated behavioral audit found Opus 5 to be its most aligned model to date, scoring 2.3 on overall misaligned behavior, lower than Opus 4.8, Sonnet 5, or Fable 5, with the lowest rates of deceptive behavior and the least susceptibility to being tricked into misuse.

On the capability side, Anthropic says it intentionally avoided training Opus 5 on cyber tasks, as it did with Opus 4.8. The model improved on them anyway — a side effect of general capability gains — and now nearly matches Mythos 5 at finding software vulnerabilities. But it remains far behind at exploiting them: on Anthropic's OSS-Fuzz evaluation, Opus 5 identified vulnerabilities at a 79.4 percent rate, close to Mythos 5's 80 percent, but succeeded at developing exploits in only 4 challenges versus Mythos 5's 13. That asymmetry — strong at defense-relevant discovery, weak at offense-relevant exploitation — appears to be by design, and the safeguards follow the same logic. Anthropic expects Opus 5's cyber classifiers to intervene about 85 percent less often than Fable 5's.

When a classifier does trigger, requests in Claude.ai, Claude Code, and Claude Cowork fall back to Opus 4.8 by default — raising an obvious question: if a request is too risky for one model, why is it acceptable for another? "The model it falls back to has lower capability levels making the risk of harmful use lower as well," the spokesperson said, adding that "there is a message that lets the user know when this occurs and is visible in the chat."

The logic is defensible, but it reveals how AI safety actually works in 2026: risk is not a property of the question alone, but of the question multiplied by the capability of the system answering it. On biology, the calculus runs the other way. Opus 5 is now Anthropic's most capable generally available model for scientific research — scoring 10.2 percentage points higher than Opus 4.8 on the company's internal chemistry benchmark — though the spokesperson acknowledged that "Mythos 5 remains the stronger model for long-horizon, open-ended work like autonomous drug design campaigns."

The business stakes behind the launch: a $380 billion valuation and massive compute bets

The launch lands at a moment of extraordinary commercial momentum — and extraordinary obligations — for Anthropic. Reuters reported in February that the company was valued at roughly $380 billion in its latest funding round, following a period in which, per Contrary Research's analysis, its annualized revenue climbed from about $1 billion at the end of 2024 to a projected $9 billion by the end of 2025, with internal targets reportedly reaching $20 to $26 billion for 2026. Those targets are underwritten by enormous infrastructure commitments, including a reported $30 billion Azure compute deal alongside arrangements with Google Cloud and Nvidia — spending that only pencils out if enterprises keep expanding usage.

That is the context in which Opus 5's pricing strategy makes sense. Holding the price at Opus 4.8 levels while roughly doubling performance on key agentic benchmarks is effectively a steep price cut per unit of capability, designed to widen the funnel of workloads that are economical to automate. Every task that was marginal at Opus 4.8's cost-per-success becomes viable at Opus 5's — and every viable task is recurring token revenue.

The regulatory backdrop has grown more complex as well. A U.S. judge gave final approval this week to Anthropic's $1.5 billion copyright settlement with book authors, Reuters reported, closing a chapter of litigation over the company's early training data. And in June, Reuters, citing Axios, reported that the U.S. government had moved to block foreign access to Anthropic's most advanced models — a reminder that frontier AI is now entangled with export policy in ways that shape which customers can buy what.

Also shipping Friday: a Fast mode running at roughly 2.5 times default speed at twice the base price, automatic fallback routing on the API, and mid-conversation tool changes that no longer invalidate the prompt cache — a small feature that agent developers may appreciate more than any benchmark. Consistent with prior Opus models, Opus 5 carries no data retention requirements for general access, a point the spokesperson flagged unprompted for customers with "a hard zero data retention requirement." Developers can access the model as claude-opus-5 on the Claude API starting today.

Two questions will determine whether the bet pays off: whether Opus 5's efficiency claims survive contact with production workloads at scale, and whether enterprises embrace a world where safety classifiers, not users, sometimes decide which model answers. But the deeper message of Friday's launch is that the AI industry's center of gravity has moved. For three years, the labs competed on what their best model could do on its best day. With Opus 5, Anthropic is competing on something less glamorous and far more lucrative: what a very good model can do every day, for half the price. In a market where the frontier keeps moving, Anthropic is wagering that the real fortune lies just behind it.

Read more →

The post IT Security News Hourly Summary 2026-07-24 18h : 7 posts appeared first on IT Security News.

Read more →

The post Expert Density as Strategy: How 2F-IT Built One of Germany’s Deepest Fortinet Practices appeared first on IT Security News.

The post OpenAI pushes ChatGPT into patient health records appeared first on AI News.

The post Google Brings iPhone-to-Android Switching Directly Into Android 17 appeared first on TechRepublic.

The post Context Windows Forget What Matters — I Built a Usage-Reinforced Decay Engine for AI Agent Memory appeared first on Towards Data Science.

Read more →

The post Kimi K3 Agents Found Redis Zero-Days and Built RCE Exploit, Researchers Say appeared first on IT Security News.

Read more →

The post Google gives developers an AI bug hunter that also writes patches appeared first on IT Security News.

The post Google gives developers an AI bug hunter that also writes patches appeared first on Help Net Security.

Read more →

The post The best-funded companies open the most phishing attachments appeared first on IT Security News.

The post The best-funded companies open the most phishing attachments appeared first on Help Net Security.

The announcement, made by Microsoft AI's Superintelligence team, lands roughly a year after the company committed to building purpose-built models internally, and it arrives with an unusual level of specificity about where those models now run: Bing, PowerPoint, OneDrive, Dynamics 365, Excel, GitHub Copilot, and Azure. The message to enterprise buyers — and, implicitly, to OpenAI — is that Microsoft's homegrown models are no longer research projects. They are production infrastructure serving millions of users.

"Each of these enhancements is a step toward the same goal: Microsoft products, powered by Microsoft models," the company wrote in its announcement blog.

How MAI-Image-2.5-Pro and MAI-Voice-2-Flash stake out opposite ends of the AI cost curve

The two new releases occupy opposite ends of what Microsoft calls the quality-speed-cost curve, and the positioning is deliberate. MAI-Image-2.5-Pro targets the premium tier: hero imagery, detailed editing, and precise in-image text rendering — the last of which has long been a notorious weak spot for image generation models. Microsoft priced the model at $5 per million text input tokens, $8 per million image input tokens, and $106 per million image output tokens. The base MAI-Image-2.5 model recently launched at No. 2 for image editing on Arena, the community leaderboard that has become a de facto scoreboard for generative media.

The creative industry appears to be taking notice. Rob Reilly, global chief creative officer at advertising giant WPP, called the Pro model "a strong leap forward for GenMedia tools" in a statement included in Microsoft's announcement, adding that "Microsoft has firmly established itself among the leaders in generative AI."

MAI-Voice-2-Flash goes the other direction. First previewed at Microsoft's Build conference, Flash runs twice as fast as MAI-Voice-2 and costs 32% less, priced at $15 per million characters. It is designed for the unglamorous but enormous market of high-volume voice — call centers, voice agents, and real-time speech applications where latency and cost-per-call matter more than marginal gains in expressiveness. Together, the two models reflect a strategy of building families of models rather than a single flagship, because, as the company put it, a creative studio chasing maximum fidelity has very different needs from a customer service operation handling millions of calls a day.

Microsoft's production metrics show in-house models cutting GPU costs by up to 89%

The model launches are arguably less newsworthy than the deployment metrics Microsoft attached to them — numbers that read like a systematic case for swapping out third-party frontier models across its product portfolio. 

Bing Image Creator now runs entirely on MAI-Image-2.5, end to end, marking the first time the consumer image tool is fully in-house. In PowerPoint, Microsoft says MAI-Image-2.5 reduces GPU costs by up to 84% compared with GPT-Image-2, OpenAI's image model. In OneDrive, where MAI-Image-2.5 is now the default for key image-editing scenarios, the company reports a 26% increase in save rates, roughly 25% lower P95 latency, and 2.5 times greater efficiency under medium-utilization production workloads.

On the voice side, MAI-Voice-2-Flash now powers Dynamics 365 Contact Center — the platform used by customers including T-Mobile and EasyJet — where Microsoft claims GPU cost reductions of up to 89%. The model is also integrated into Azure Voice Live for developers building speech-to-speech agents.

Perhaps the most consequential deployment sits in healthcare. Microsoft's Dragon Copilot, used by 170,000 medical providers and responsible for processing 28 million patient encounters last quarter, now runs on MAI-Transcribe-1.5 for its multilingual workflow across 58 languages. Microsoft says internal evaluations show a 50% relative reduction in both transcription and language-identification error rates across most languages — a meaningful claim in a domain where transcription errors can propagate directly into clinical notes.

Inside the 'hill-climbing' strategy that lets small models beat GPT-5.6 in Excel

In a companion post published the same day, Microsoft detailed the methodology behind these results — what it calls its "hill-climbing machine," an integrated flywheel of data, models, and the product "harness" that surrounds them.

The clearest example is MAI-Code-1-Flash, the lightweight coding model launched in GitHub Copilot in June. Microsoft says the model achieves an approximately 10% higher code accept rate than GPT-5.4 Mini and Claude Haiku 4.5 in VS Code, while using 10% fewer median tokens. Developer retention tells a similar story: users were 6% more likely to return across multiple days than with GPT-5.4 Mini, and 11% more likely than with Claude Haiku 4.5.

Then Microsoft did something more interesting. It took the MAI-Code-1-Flash checkpoint and further trained it inside an Excel reinforcement learning environment, teaching a coding model the tools and workflows of spreadsheet knowledge work. The result, according to production user feedback, is a model on par with GPT-5.6 for the most common Excel tasks — while being small enough to run on Nvidia's older H100 and even A100 GPUs rather than requiring the latest-generation accelerators.

That hardware detail deserves emphasis. Every major AI company is fighting for allocation of cutting-edge chips, and a model that delivers frontier-adjacent quality on two-generation-old silicon fundamentally changes the deployment economics. It also frees the newest hardware — including Microsoft's now-operational GB200 cluster — for training rather than serving.

Satya Nadella's 'frontier diffusion' manifesto redraws the OpenAI relationship

Microsoft CEO Satya Nadella framed the announcements in a lengthy post on X titled "Frontier Diffusion & Control," which functions as something close to a strategic manifesto. "We can now take saturated frontier capabilities and deliver them at scale and at lower cost through models optimized for high-usage products, while continuing to use frontier models for frontier needs," Nadella wrote, adding that Microsoft is "beginning to route traffic across our first-party surfaces to MAI whenever our models match or outperform frontier alternatives."

Translated from executive prose: capabilities that were state-of-the-art a year ago are now table stakes, and Microsoft believes it can replicate them cheaply for the specific, repetitive tasks that dominate real product usage. Why pay frontier prices for a frontier model when a user just wants to reformat a spreadsheet column?

Nadella was careful to note that "frontier models from OpenAI and Anthropic are part of the orchestration system alongside MAI" — but he also articulated a pointed principle of model independence, arguing that a company's evaluations "should continue to hill climb even when any given model has been removed." 

“Keeping the harness, memory, context, and skills outside the model, he argued, is what gives Microsoft control. The subtext is hard to miss. Reuters reported in April that Microsoft’s exclusive license to OpenAI’s technology had been revised into a non-exclusive arrangement, and The Information reported last September that Microsoft had begun incorporating Anthropic models into some products. Wednesday’s announcement completes the triangle: Microsoft as orchestrator, with its partners’ frontier models as interchangeable components and its own models absorbing an ever-larger share of routine traffic.”

Developers cheer cheaper task-specific models while skeptics question Microsoft's track record

The response online captured both the appeal and the skepticism surrounding the strategy. "I love when people use small models for niche tasks," wrote one X user, @mavihsk, responding to Nadella's post. "Why do I have to use the all-knowing model just to change my field in Excel?" Another user, @nabu_lines, distilled the pitch neatly: "cost and performance both improve when you stop overusing the biggest model."

Others were less charitable about Microsoft's execution track record. "Microsoft is the worst when it comes to listening to user feedback," wrote designer @designedbyabin, arguing the company "will lose the AI race because they repeatedly failed to understand user needs." And one user, @tokenoverflow, offered a drier critique of the model-independence pitch: "i want it keep hill climbing after removing microsoft."

The skeptics raise a fair point. Microsoft's self-reported metrics — accept rates, save rates, GPU savings — come from its own internal evaluations, not independent benchmarks, and the company chooses which comparisons to publish.

But the strategy's logic does not depend on any single number. Nadella's framing that software now has "real marginal cost for the first time" explains why Microsoft is obsessive about tokens, GPUs, and serving costs: when AI features run on every keystroke across a billion-user product portfolio, an 84% GPU cost reduction is not an optimization. It is the difference between a viable business and a money pit.

Why Microsoft is turning its internal AI playbook into an Azure product

The final piece of the strategy is that Microsoft is selling the playbook, not just the models. Nadella explicitly positioned the hill-climbing approach as "a template for every other AI native, SaaS, or Enterprise company," and Microsoft is packaging the toolchain through Foundry and what it calls Frontier Tuning — letting enterprises train specialized models against their own proprietary evaluations and reinforcement learning environments. That turns Microsoft's internal cost-cutting exercise into an Azure product, and it gives enterprise customers a reason to run their AI workloads on Microsoft's cloud even if the models themselves come from elsewhere.

The company's emphasis on models trained "on clean, traceable, enterprise-grade data, without distillation from third-party models" serves the same commercial end. In an industry facing mounting scrutiny over training data provenance, Microsoft is betting that enterprise buyers — and courts — will care where model capabilities come from. Microsoft says it is now extending the hill-climbing approach to Copilot Chat, Outlook, and PowerPoint, and both new models are available in public preview through Microsoft Foundry and the MAI Playground. "None of this is an endpoint," the company wrote. "We're just getting started."

Seven years ago, Microsoft bet more than $13 billion that OpenAI would build the future of AI. Wednesday's announcement suggests the company has since learned a cheaper lesson: the future of AI may belong to whoever builds the frontier, but the profits belong to whoever makes it ordinary.

In our sample game we look at playing as Austria and aiming for a Diplomatic Victory. And our focus is on puppeting City-States, but I misunderstood and instead of making a Diplomatic Victory easier, it makes it harder. I still managed to get my Diplomatic Victory, but a Science or Domination Victory would definitely have been easier in this scenario.

Playing Civilization V, Part 14

Now I am at turn 340, I just puppeted another City-State, and I have one more on the way. I have sent a large number of military units, both land and sea, to the City-States that are on Japan’s border. If he tries anything now he will likely regret it. Now I am starting to build things I had not bothered with before, like Granaries and Amphitheaters. I got a couple of Great Engineers, and used them to rush build Wonders. I am nearly finished puppeting all of the City-States, and my Happiness and Income are both quite healthy. I am in such good shape I started to build the World’s Fair, which I initally proposed then ignored.

At this point I only have one more City-State to puppet, then I will have puppeted every City-State in the game. So it is time to put my large income to use in other ways by giving gifts to the other players. I may want to get a few more votes on my side to get over the top in the World Leader vote when it happens. On turn 357 I completed the puppeting of all of the City-States in the game. Meanwhile, I am building up my Science and building Wonders as they become available. I didn’t build any in the early game because I had other priorities, but now I have a strong military, high income, and no other pressing needs. My score on the Leader Board is now double that of my nearest competitor which is still Japan. And their denunciation me of just expired with them taking no action.

I used a Great Engineer to rush build the Porcelain Tower, earning me another Great Scientist, which I then used to build another Academy. At this point it is mostly a slog to the finish. Since I was so dominant and so wealthy I could afford to pursue a couple of other options just in case. One was a Science victory, which looked like a possibility, though it would be close. I had to be very focused in my Science Research. The other option was a Culture victory, and I had dominated two other empires culturally without half trying, and I was generating Great Artists, Great Musicians, and Great Writers pretty rapidly. So I built Museums, then Archeologists, and grabbed every artifact I find to stuff into the Museums. But the final Victory came when I researched Globalization. This tech allows you to get an extra vote in a World Council for every Spy you send to another empire’s capital and designate as an Ambassador. I immediately sent my spies out (they had previously been in my cities doing counter-espionage since I had a tech lead). And a few turns later a vote for World Leader came up and I won.

This was my first time using Maria Theresa of Austria as my leader, and it was fun, but in the end puppeting all of the City-Staters actually made it harder to win a Diplomatic victory. In other games I have played, every City-State that was an ally was an extra vote in the World Council, and I assumed that making them puppets would mean I would get that many votes. But that didn’t happen. Of course, if I had read more carefully in the Civilization Wiki I would have realized this, because it says there very clearly that Austria is one of the worst Empires to pick if you are going for a Diplomatic victory. So the fact that I managed it anyway is nice. I think I actually could have done a Science victory as I had my Spaceship half-built when I won the Diplomatic Victory. But I hope this analysis of how I actually played this game will be helpful to someone else.

Links

• https://www.palain.com/gaming/civilization-v/playing-civilization-v-part-14/

Provide feedback on this episode.

OpenAI is rolling out "Health in ChatGPT" to U.S. users, connecting Apple Health, medical records, and wellness apps. More than 300 million people already ask ChatGPT health questions every week, but paying users get better answers. The more powerful GPT-5.6 Sol model is reserved for premium subscribers, while free users are stuck with the weaker GPT-5.5 Instant.

The article ChatGPT will give you worse health advice if you don't pay appeared first on The Decoder.

VentureBeat's June 2026 Pulse survey of 107 enterprise respondents explains why the room was full. More than half, 54%, have already had a confirmed agent security incident (18%) or a near-miss caught before harm (36%). Just 32% give every agent its own scoped, managed identity, and fewer still, 30%, isolate their highest-risk agents in sandboxes. Provider-native and hyperscaler controls remain the primary agent security layer at 82% of companies surveyed. The world's largest security vendors have done the same math.

Palo Alto Networks closed its $25 billion acquisition of CyberArk in February, CrowdStrike agreed in January to pay $740 million for SGNL, and Cisco announced its intent to acquire Astrix Security for a reported $400 million, all of it aimed at the identity and isolation layer most enterprises have not finished building.

Chang came to the panel with almost two decades of experience spanning cybersecurity operations, government, and the military. She ran global cybersecurity operations as an executive director at JPMorgan Chase, where she led the bank's cyber threat intelligence teams, and served as a senior staffer on the House Foreign Affairs Committee and as a U.S. Navy Reserve officer. She also teaches cybersecurity and emerging threats as adjunct faculty at the Middlebury Institute of International Studies.

Chang's 88.3% number comes from a study she co-authored with Nicholas Conley, built on 30,090 single-turn prompts and 6,986 multi-turn attacks against those 15 closed and proprietary flagship models. Multi-turn success rates ranged from 7.89% to 88.3%, every model tested showed non-trivial multi-turn exposure, and the two testing styles did not even rank the models in the same order. Cisco publishes adversarial evaluation signals for what is now 105 models on its LLM Security Leaderboard, she told the audience.

"If you don't understand how models are susceptible to different types of attacks, then you are unable to account for how that model that is powering your agent, that is powering your application, to understand where those failure points are," Chang said. Single-turn testing is the one-shot malicious prompt, she explained, while extending an attack into a longer conversation "is more realistic of how we are actually engaging with our models, with our agents, with our applications." That longer arc surfaces harmful outputs and misaligned behaviors that a snapshot never catches.

Cisco has pushed the testing itself into agentic territory. Chang described a framework where agents assess a deployment scenario, develop relevant attacks, judge whether they are worth pursuing, execute them, and evaluate their own success. What surprised her most, after all that sophistication, was how simple the defensive answer stays. "The answer is still that it's pretty simple," she said. "You don't have to get super creative. You just need to think about truly what are the fundamentals and basics of what I'm trying to secure in my organization."

Her starting point for CISOs beginning agentic deployments is Cisco's Integrated AI Security and Safety Framework, which she said "stipulates all the ways that AI can be compromised across the AI lifecycle" from modality through supply chain. From there, teams can work backward from real incidents, trace how each attack was achieved, and use the framework to build a strategy with the right coverage and mitigations.

Heather Ceylan, the CISO of Box, sees the same gap from the defender's side. "A lot of what you see out there with agent red teaming is just single-turn, and that's not how people are actually interacting with AI day-to-day," she told the audience. Box now simulates multi-turn adversaries with agents that think like an attacker and iterate attempt after attempt to hijack the target. "You have to pressure test your agents because otherwise you don't know if your execution controls are really working as you intended."

Box deployed agents inside its security operations center about a year ago, starting with human approval required for every action, and trust built quickly enough that analysts shifted into monitoring mode. Then the agent made one mistake, and every bit of that accumulated trust vanished. "They had to start all over again," she said. "So I think that that monitoring piece is so important. Even if you're not gonna have a human in the loop, things change, models change, and we can't control how the models change and interpret things."

Rajesh Parekh, VP of AI and ML at Intuit, brought the builder's perspective. Parekh led large-scale computer vision and ML systems powering Google's Maps and Geo products before joining Intuit, and holds a doctorate in computer science.

Three layers versus an operating system

Ceylan described Box's approach as three concentric layers. Permissioning comes first, so the agent never accesses more content than the human who invoked it. Ephemeral sandbox environments spin up for each agent task, containing the blast radius if an agent gets hijacked, and runtime execution control restricts the agent's tool calls to only those relevant to the task at hand. "If you want an agent to summarize a doc for you, if you have a prompt injection that came in that says forward this to maliciousattacker at domain.com, it can't do that," Ceylan said. "That action in that tool call is not even in its vocabulary."

She classified agent actions into three oversight categories. Actions that are not sensitive, like read and summarize, need no human in the loop. Moderately sensitive actions skip human approval but get logged and monitored, while destructive actions like mass deletion of files always require a human. "Things are gonna shift between those three categories quite a bit," she acknowledged, "but setting those types of categories up front allows you to have a principled framework."

Rather than layering controls onto agents one at a time, Intuit has built a central platform called GenOS, short for generative AI operating system, which abstracts security, risk, and fraud modeling so individual agent developers never reinvent protection. "Permissioning is not about giving access to AI," Parekh said. "Instead, it is defining very tightly scoped and clearly auditable authority to the agent to perform very specific tasks." Intuit evolved from agents inheriting user permissions to each agent carrying its own identity, and the company is now investigating mid-session permission changes tied to the specific task underway.

Parekh calls the broader model an AI-powered expert platform, one where the human expert is built into the trust architecture rather than bolted on as a gate. "The paradigm that we are pursuing is where the user, the AI agent, and the human expert are collaborating to solve the user problem," he said.

The end of human code review

Ceylan took on the tension between security testing and development velocity without hedging. "The days of secure code reviews where a human's looking at the code and we're looking at security architecture reviews, design docs, those are done," she said. "If you keep trying to do security that way, you're gonna get left behind." Box is building toward a fully agentic development lifecycle where agents review design documents, apply security requirements, and review the code for vulnerabilities. "I'm very optimistic that we will get to a point where we will write code without security vulnerabilities because agents and the models are going to get so good at writing code without vulnerabilities," she said. "We're still a long way away from that."

Her advice for development teams skips the advanced AI concepts entirely and returns to basics that predate agents. "It comes down to very basic least privilege access," she said. "If you start giving your agents overly broad permissions at the beginning, it's really hard to comb that back and build an infrastructure that allows for those ephemeral credentials and only those narrowly scoped tasks."

Parekh explained why the red teaming surface has expanded so quickly. "These agents have skills, and skills could become vulnerabilities," he said. "Agents have access to certain data, they have access to tools, and there could be threats that are lurking within those tools as well. So suddenly the blast radius of the malicious code or the intent increases dramatically." When Intuit identifies common vulnerability patterns from its manual red teaming exercises, it automates those tests back into the GenOS harness so future agents inherit protection and red teamers stay focused on new threat vectors. Runtime scanning of prompts and responses adds a final layer that can stop a suspect response and escalate to a human expert, he said.

"You need to continuously test to ensure that those remain robust to the protections that you have built, as well as to account for any sort of drift or any other types of dependencies that you introduce into your scenario that can create novel vulnerabilities," she said.

Intent versus probability

An audience question about intent detection set off the sharpest exchange of the session. Ceylan noted that when Box's own agent operates, the system always knows the user's intent because it controls the prompt, which means guardrails and tool-call restrictions can be engineered around it. The harder challenge, which she admitted Box is still trying to solve, arrives when external agents connect and the context behind the request is opaque.

That exchange exposed a split running through the wider industry. Mastercard, in the fireside chat immediately preceding the panel, came down on the side of quantifying intent, building an open-source framework to propagate it as a standard because complex B2B procurement cannot work without that trust. Endpoint security CTOs, in briefings with VentureBeat, have gone the other way, saying they will bet on probability rather than intent inference for production workloads. Chang explained why models, as they are trained today, cannot reliably derive intent from a prompt, which is why deterministic controls and behavioral proxies remain necessary. Ceylan agreed that both are required. "If you're not doing anything deterministic, you're really relying heavily on that intent, and I haven't seen programs that are there yet," she said.

Ceylan's story about trust collapsing after a single agent mistake landed as the panel's most memorable moment because enterprise agentic security is not a problem that gets solved and stays solved. Models change, permissions drift, and adversaries adapt across multi-turn conversations that snapshot tests never capture.

For the 82% of enterprises relying on provider-native controls as their primary security layer, and the 59% shopping for agent security tooling over the next 12 months, the panel's takeaway was blunt. Test the way attackers attack, across full conversations and continuously, or find out in production what your single-turn red teaming missed.

The Freiburg, Germany-based AI lab says FLUX 3 is jointly trained across those modalities rather than assembling separate image, video and audio models behind a common interface.

That distinction is central to the company's pitch: BFL wants enterprises to think about creative generation, simulation, computer use and robotics as connected applications of a single capability it calls visual intelligence — models, in the company's words, "that can perceive, predict, and act across physical and digital environments." This release marks BFL's first public video generation model.

FLUX 3 will be offered through four product lines: FLUX 3 Video, FLUX 3 Image, FLUX 3 Action and the upcoming, open source FLUX 3 Dev. FLUX 3 Video, with optional native audio generation, and FLUX 3 Action are entering a gated "Early Access" program now, to which anyone can apply, but which BFL must approve.

There is presently no public access through BFL's application programming interface (API) or those of partners yet, but the company says FLUX 3 Image will roll out in the coming weeks, followed by general availability. The limited initial availability rollout echoes the release strategies of new models from other frontier labs in the U.S. lately, including Anthropic and OpenAI, though those were ostensibly for security concerns and due to government request.

What the company has not announced is pricing, production service-level commitments, evaluation methodology, sample sizes, rater counts or any image-model benchmarks at all. Enterprise buyers therefore cannot yet calculate total cost of ownership or independently reproduce the video comparisons.

Another big notable omission: FLUX 3 is not launching with downloadable weights at this time, nor an open source license. BFL says faster and open-weight versions will arrive later this year, and its technical blog names FLUX 3 Dev as "open-weight access to a multimodal backbone, for content creation (video, audio and image) and action prediction" — a considerably broader commitment than any previous FLUX Dev release, all of which covered images only.

But it arrives last in the sequence. Developers accustomed to receiving a locally deployable FLUX variant alongside — or soon after — a major model announcement will have to wait. That delay does not negate the company's commitment, but it is disappointing given the role open weights have played in FLUX's adoption thus far.

Flux 3 is rated higher than the competition, but missing pricing and benchmarking details may prevent rapid enterprise adoption

BFL has published several benchmark comparisons, but they're qualified as preliminary — with full benchmark results and methodology to be published later during broader general availability.

In early head-to-head preference testing on 10-second, 720p text-to-video clips with audio, the company says FLUX 3 was preferred over Luma Ray 3.2 in 93% of comparisons, Runway Gen-4.5 in 77%, Grok Imagine Video in 69%, Kling v3 Pro in 60%, Happy Horse v1 in 59%, Happy Horse 1.1 in 57%, and both Seedance 2.0 and Google's Gemini Omni Flash in 52%.

One caveat travels with every one of those figures, and it comes from BFL itself. The chart carrying the results is labeled a "preliminary evaluation of an early FLUX 3 candidate" — meaning the numbers describe a pre-release checkpoint rather than the model now entering early access. That cuts both ways: the shipping model may perform better, but nothing published today measures what customers will actually call.

Luma Ray 3.2 and Runway Gen-4.5, where FLUX 3 posted 93% and 77%, are the softest comparisons on the list — established products, but not the models currently setting the pace in independent video rankings. Those are real wins, and they are the ones least likely to change an enterprise shortlist.

Seedance 2.0, at 52%, is a statistical coin flip against a model most Western enterprises cannot currently procure. ByteDance indefinitely postponed Seedance 2.0's international rollout after Netflix, Warner Bros., Disney, Paramount and Sony sent legal threats over alleged systematic copyright infringement, and that suspension remains in place. Tying a frozen product is neither a strong claim nor a damaging one.

Gemini Omni Flash, also at 52%, matters much more. Omni is the closest large-platform analogue to what FLUX 3 is attempting — multimodal input, video and audio-aware creation, conversational editing — and by BFL's own measurement, the two are indistinguishable on 10-second text-to-video quality.

Google's advantage in that matchup is that Omni is generally available via Google's Gemini API for $0.10 per second of generated 720p video, or a 10-second clip for around.

One regional wrinkle matters for a German company's home market. Editing uploaded video is unavailable to Omni Flash users in the European Economic Area, Switzerland and the United Kingdom, though editing video the model itself generated is permitted. A European enterprise that wants to run its existing footage through a generative editing pass cannot currently do so on Omni Flash.

Here's a rough guide for enterprises considering which video models to rely upon:

One architecture for media generation and physical action

FLUX 3 builds on Self-Flow, BFL's method for aligning multimodal understanding and generation within one architecture, publicized back in March 2026.

The company says it significantly scaled up compute and data to train across video, images and audio simultaneously, and that testing showed video generation and action prediction do not require separate foundations — the same architecture could be extended to action prediction without sacrificing what it learned from video.

"We place vision at the center of our approach because it is the most signal-rich medium of the physical world. Images convey structure, images and video teach spatial relationships, video teaches dynamics, and actions reveal causal relationships. But vision alone is not the complete picture," said Robin Rombach, co-founder and CEO of BFL, in a pre-release statement provided to VentureBeat. "True intelligence means perceiving the world: predicting how it will change, taking action, and learning from the results. Joint training within one unified architecture is what will get us there, because each training modality strengthens the others. Audio conveys timing, prosody, and physical events that elude vision. Language conveys goals, abstractions, and instructions that pixels cannot easily express."

He put the case more bluntly elsewhere in the announcement: "You can't cheat reality. A model that only learns images can only generate images. But the world is not made of still frames. It moves, sounds, changes, and responds."

BFL says FLUX 3 targets creative tooling, media, design, e-commerce and physical AI, supporting video generation with synchronized audio, precise image editing, product and material consistency across motion, multilingual generation and robotic action prediction. It is already being tested by Canva, Burda, Magnific (formerly Freepik), Krea and Picsart.

For creative software companies, the appeal is consolidation. A single foundation could potentially support storyboarding, image editing, product rendering, video variation and localization without repeatedly translating assets and instructions between disconnected models.

For robotics teams, the potential value is data efficiency. Models that already encode motion, object behavior and physical change may need less task-specific robot training than systems starting from raw demonstrations.

What FLUX 3 Video can actually do

The video tier is the most concretely specified part of the launch, and it settles a question that had been circulating as rumor: FLUX 3 generates clips of up to 20 seconds with audio in a single generation.

Every video output comes with native audio. For comparison, HappyHorse 1.0 tops out at 15 seconds of 1080p with synchronized audio — though BFL has not stated what resolution its 20-second clips run at, and its published evaluations were conducted at 720p. Still, a 20-second long clip from a single prompt is among the longest yet achieved, matching OpenAI's discontinued Sora model.

The capability list BFL published covers:

• Text-to-video generation.

• Image-to-video generation, either animating from a starting frame or using images as visual references.

• Video-to-video generation from a reference clip, carrying elements such as a specific character into a new scene or context.

• Generative video-audio continuation from existing video and audio input.

• Keyframe-to-video generation for controlled transitions between defined moments.

• Multilingual dialogue.

• A broad range of visual styles and aspect ratios, from candid camcorder footage to animation and cinematics.

• Typography generation and animated design.

• Agentic chaining of individual clips into longer, multi-shot sequences.

That last item is the one enterprise video teams should look at hardest. BFL claims the capabilities combine to produce sequences lasting several minutes, with visual references keeping characters consistent across scenes. If that holds up under production conditions, it addresses the constraint that has kept generative video out of most commercial pipelines: not clip quality, but continuity across shots.

It is also the capability where competition is most direct. HappyHorse 1.1's headline upgrade is R2V, or Reference-to-Video, which accepts multiple character reference images to hold identity stable across generated footage — the same problem, approached at the input layer rather than through agentic clip chaining. Alibaba also claims zero-drift lip sync and has specifically targeted the artifacts that mark commercial AI video as synthetic, including facial oiliness and over-sharpening. Character consistency is where this category is being contested, and both companies know it.

BFL says FLUX 3 Video is already particularly strong at human facial expressions, associating sounds with physical events, and multilingual output. On the image side, the company says preliminary evaluations conducted during midtraining show significant improvement over earlier FLUX versions in complex prompt handling and text generation, including high-accuracy text in multiple languages. It published no image benchmarks or win rates.

FLUX-mimic tests whether video models can become robot models

BFL is applying its unified-architecture thesis through FLUX-mimic, a video-action model built on FLUX 3 and developed with Swiss firm Mimic Robotics, one of the first partners to receive early access.

The technical blog describes two distinct routes to action prediction: integrating native action prediction directly into FLUX 3, scaling up the initial Self-Flow work; and using the pretrained video backbone as a dynamics-aware foundation from which specialized action models can be finetuned with limited task-specific data. FLUX-mimic is the second route — the FLUX 3 backbone combined with mimic's robot-learning and production-deployment expertise in dexterous manipulation.

FLUX-mimic is designed for general-purpose robotic manipulation: helping robots understand a visual scene, predict the consequences of an action, and adapt to new tasks with far less task-specific data.

BFL and Mimic Robotics say that depending on task difficulty, the model can be finetuned for a specific manipulation task with as little as 30 minutes of robot data, where prior approaches have required 30 or more hours.

"The hardest part of robotics is data," said Elvis Nava, CTO of Mimic Robotics, in a statement provided to VentureBeat. "Every new task normally means hours of a robot repeating itself. Because FLUX-mimic is built on top of frontier video models that already understand how the physical world behaves, it picks up a new task in minutes, not days. This way, we can leapfrog the current state of the art in robot learning."

BFL argues that a model trained only on images cannot understand a world that "moves, sounds, changes, and responds," and that physical understanding is what produces convincing generated footage. Google makes a nearly identical claim for Gemini Omni.

Its developer documentation cites "world knowledge" that combines "an understanding of physics" with Gemini's grasp of history, science and cultural context. Its marketing is blunter still: "Most AI models just predict the next pixel to build a narrative or an image. Gemini Omni is different," the company posted in June, crediting the model with "an intuitive understanding of forces like gravity, kinetic energy, and fluid dynamics for more realistic movements that follow real-world logic."

The practical consequence for enterprise buyers is that world-model language is not a differentiator. Two of the three leading video systems now market physical understanding as their central advantage, and neither has published a benchmark that measures it.

There is no standard test for whether generated water behaves like water, whether a dropped object falls at a plausible rate, or whether a sound arrives when the impact does. Human preference ratings capture some of it indirectly. Nothing else on offer captures it at all.

Open weights helped make FLUX an industry standard

BFL officially launched in summer 2024 and gained a name for itself in the AI industry in the intervening two years for its commitment to open sourcing high-quality AI image models beloved by developers, creatives, and enterprises.

The company's founders, including Rombach, Andreas Blattmann and Patrick Esser, previously helped create VQGAN, latent diffusion and Stable Diffusion, the latter the open source technology that kicked off broad AI generation capabilities for the masses and currently used by many AI image generators and companies.

That reach translated into commercial distribution. FLUX models now power generative features inside Adobe Photoshop, Picsart and Nous Research's Hermes Agent, among other platforms, and the company cites film director Martin Scorsese among professional users.

Wired magazine described Black Forest Labs as a relatively small company that nevertheless became a leading competitor to Silicon Valley's largest AI labs, with FLUX models ranking near the top of image benchmarks and becoming some of the most downloaded text-to-image models on AI code sharing community Hugging Face. The company says it now runs a 100-person team across Freiburg and San Francisco.

FLUX.1 Dev, FLUX.1 Kontext Dev, FLUX.1 Fill Dev and related control models, released shortly after the firm's launch, gave researchers and creative-tool developers access to downloadable checkpoints, local inference and integrations with frameworks including Hugging Face Diffusers and ComfyUI. FLUX.1 Kontext Dev, for example, was released as an open-weight model for research and noncommercial use, with generated outputs permitted for commercial purposes under the applicable license.

The company continued that pattern with FLUX.2 Dev in late 2025, a 32-billion-parameter open-weight model combining generation and multi-reference editing. Black Forest Labs called it the strongest open-weight image generation and editing model available at launch and released weights, reference inference code and optimized implementations for consumer Nvidia GPUs.

FLUX 3 Dev raises the stakes on that evaluation. Previous Dev releases were image models. This one is described as a multimodal backbone spanning video, audio, image and action prediction — meaning a single license will govern whether a company can locally deploy a model that touches both content production and physical machinery. BFL hasn't yet shared information about its license, the parameter count, quantizations or hardware requirements.

The company frames open weights as an enterprise feature rather than a community gesture, arguing they enable secure, low-latency local deployment for applications like robotic control systems and let teams adapt FLUX 3 to their own data, products and workflows.

The financial backing behind FLUX 3 is worth noting alongside the technical claims. Black Forest Labs is valued at $3.25 billion and has raised more than $450 million from investors including a16z, AMP, Salesforce Ventures, Nvidia, General Catalyst, Adobe Ventures, Figma Ventures, Canva and Deutsche Telekom's T.Capital.

Read more of this story at Slashdot.

“Our founder and CTO has actually been really pushing to enable our agents in YOLO mode,” Rishi told the audience. That admission comes from a publicly traded data security firm whose business is backing up what he called the most important data in the world.

YOLO mode strips the permission prompt out of agent workflows and lets the agent act on its own. In Rubrik's version, a second AI judges every action in real time against policy in place of a human clicking approve. Rubrik is running the experiment on itself first. Rishi treats autonomy as a settled capability question and an open judgment question. "If you ask the agent to act autonomously, it will," he said. "It's a question that you have internally. Should it?"

Rubrik earned that question the hard way. When Claude Code and Cowork pilots rolled out, the company required every command to run in ask mode so the employee issuing it carried the liability, and the developer pushback filled a single Slack thread 120 messages deep.

"The developers basically are pushing back, and they're like, this is like the iTunes service agreement. I'm just hitting check, check, check, check, check, check, check," Rishi said. "There's no way that I can actually read through this. And it becomes security theater." Roughly 80% of respondents are in the same bind, Rishi said, citing Rubrik Zero Labs research that found monitoring and approving agent actions takes more time than the agents save. The State of the Agent, the April report behind that figure, surveyed more than 1,600 IT and security leaders.

SAGE is the reason Rubrik trusts the bet. Short for Semantic AI Governance Engine, SAGE is the arbitration layer inside Rubrik Agent Cloud that watches every action an agent takes and reads the semantic intent behind it, then rules the action in or out against policies written in natural language. "We took what people said was human in the loop, a good idea, and we replaced it with AI in the loop," Rishi said, describing the pitch to security chiefs he characterized as skittish about non-deterministic systems.

Security approval, not cost, blocks AI ROI

Rishi’s path to Rubrik ran through Predibase, the generative AI infrastructure startup he co-founded and ran as CEO until Rubrik agreed to acquire it in June 2025. Before that, he led ML product at Google on the team that became Vertex AI, served as Kaggle's first product manager as it grew from about one million to ten million users, and holds bachelor's and master's degrees in computer science from Harvard.

Over roughly his first three and a half months at Rubrik, Rishi set up 200 customer conversations with IT and security leaders across a customer base that looks like the Global 2000, asking open-ended questions about cost, latency, performance, and orchestration. "Pretty consistently, what I heard through all of those conversations was that all of those are pretty secondary," he said. "The main challenge is actually, how do I get this approved from a security and risk standpoint? I'm concerned about all the different things that could go wrong. Actually, I felt like that was one of the biggest things constraining ROI."

VentureBeat Pulse research presented on the Transform stage earlier in the day confirms the gap Rishi kept hearing. Two-thirds of enterprises, 66%, already allow or are actively building toward production deployment with zero human review, yet only 5% fully trust the automated evaluations that would make that decision.

One AI reading what the rulebook can't

Rubrik's own policies exposed why written rules fail as enforcement. One internal rule states that agents should respect Rubrik's customer data use policy, which sounds enforceable until someone tries. "Rubrik's customer data use policy is like a three-page document of legal text," Rishi said. "I have no idea how to write that in there as a rule." Asked on stage how a team of AI infrastructure people took on a problem that security engineers own, Rishi answered, "with a lot of naivety and innocence, honestly." His team bet that models good at understanding language could police other models, and SAGE became the answer.

The case for putting a model in the judgment seat comes down to precision. A rule like "agents should not be able to edit revenue fields in Salesforce" fails in conventional tooling because Salesforce does not delineate which fields count as revenue, Rishi explained, so administrators fall back on approving every Salesforce action by hand. SAGE reads the intent instead and acts as a judge, carrying organizational context, which can tell a benign lookup from the edit the policy prohibits.

Keeping the judge small is what makes the economics work. SAGE runs on a small language model that Rishi said operates at an order of magnitude lower cost and latency than a frontier LLM. "If I told you, don't worry, you're gonna be secure and governed, but I'm gonna double your cost and latency, you would tell me to get out of the room," Rishi said.

When Rishi asked who in the audience had worried about token consumption over the past year, half the hands went up. "And I guess the other half is probably just too lazy to raise their hand," he said.

SAGE is an aggregation of judges based on parameter-efficient fine-tuning that Rubrik uses to take on task-specific variants of a base model with shared organizational context. One judge watches for tool-use hallucinations while another suppresses PII before it can leave, each running as its own enforceable policy. Security and GRC teams have started writing financial rules into the same layer, including one internal policy barring AI spend on personal projects.

The lethal trifecta

Asked which attacks worry him most, Rishi pointed at the lethal trifecta, the term security researcher Simon Willison coined in June 2025 for an agent that holds private data while taking in content nobody vetted, with a channel to send what it finds to the outside world. The danger, according to Rishi, is what happens when individually legitimate permissions stack. An agent granted Salesforce access and email access on an employee's credentials has done nothing wrong yet, with yet being the operative word. "A very simple example is that an agent can start pulling data from Salesforce and then decide to accidentally leak and exfiltrate that out via an email," he told the audience. A financial services company he met the morning of the session made the point for him, telling Rishi that none of the individual permissions are bad on their own and the agent needs every one of them to do its job. "It should have permission to each of those systems, but it's the combination that ends up becoming really destructive," Rishi said.

Traditional identity and access management never priced in that combination because it relied on the judgment of the employee holding the credentials, Rishi argued, and agents supply none. "I can tell you the number of times Claude Code has tried to leak some of our sensitive source code to a public GitHub repository is incredibly high," he said. Cutting agents off from public resources entirely would defeat their purpose, which returns the problem to adjudicating intent in context rather than revoking access.

A separate VentureBeat June Pulse survey of 107 qualified enterprise respondents maps the blast radius of exactly this pattern. On the Transform stage that morning, VentureBeat research reported that 69% of companies run credential sharing somewhere in their agent fleet. Companies with shared credentials anywhere got hit more often, reporting a security incident or near-miss at a 63.5% rate (47 of 74), against 40.9% (9 of 22) where every agent carries its own scoped identity.

The attacks no single turn reveals

Rubrik Agent Cloud reached general availability in February, though not everything Rishi described ships in it yet. Backtesting is just starting to roll out. The feature replays an organization's historical agent actions and tool calls against a new policy, showing where the policy would have stepped in and where an action would have sailed through uncaught, with policy edits applied in real time. Rishi called that archive one of the most valuable data troves an enterprise holds.

Real-time detection and blocking turn out to be the entry point rather than the whole product. Some attacks never trip a single-action rule. "No individual turn of the conversation was problematic, but if you took the session as a full trace, that ended up being problematic," Rishi said. Agent Cloud runs batch analysis across entire session traces every hour or every day and surfaces what Rubrik calls insights, the problems no individual guardrail caught. The same Zero Labs report found that 88% say they lack the ability to roll back agent actions without system disruption, a recovery gap that sits squarely in Rubrik's original line of business.

A skeptical CISO will ask the question the fireside did not answer. SAGE is a non-deterministic model policing other non-deterministic models, and Rishi offered no false positive or false negative rate for the judge itself. The closest thing the architecture gives to an answer is auditability, since backtesting and the batch insights both leave a human-reviewable trail of each call SAGE made and whatever got past it. Who watches the watcher, for now, is a trail of receipts rather than a benchmark. Until that benchmark exists, AI in the loop stays an operational wager rather than a quantified control.

Three questions fall out of the session for security teams. How many of the guardrails now in production depend on a human clicking approve, and what happens to that workload as agent count grows? Does anything in the stack enforce semantic intent, or is it all allow and deny lists? And can the team backtest agent behavior against a new policy, then unwind a multi-turn session without taking systems down?

Rishi's timing has a market behind it. In the same VentureBeat research, 82% of enterprises still name their primary AI provider's built-in guardrails and cloud controls as their main agent security layer, and 59% plan to adopt, add, or replace agent security tooling within the next 12 months. Only 12% include an agent-identity product in what they are considering, even with credential sharing still the norm. Every CISO at that Anthropic roundtable had a policy document and no enforcement mechanism, and Rubrik built a product for the space between the two. YOLO mode is the bet that an AI watching other AIs can finally make the policies real.

This wave of VentureBeat Pulse Research examines enterprise agent orchestration: which platforms enterprises run on, what drives the choice, what they optimize for, how they expect agent control to be structured, and — most revealingly — how orchestrated their deployed “agents” actually are and how tightly they control the cost of running them.

The central finding is a gap between orchestration ambition and orchestration reality. Enterprises are consolidating fast onto the major model platforms: Anthropic’s Claude is the primary platform for 40%, more than double any rival, followed by Microsoft (18%) and OpenAI (13%). The choice is driven by “model gravity” — native alignment with a state-of-the-art base model (21%) — and success is judged by reliable, multi-step execution (task completion reliability 32%, multi-step workflow management 28%). Yet asked to assess their portfolios honestly, 71% say a quarter or fewer of their deployed “agents” are true multi-step orchestrated workflows rather than single-prompt chatbot wrappers, and only 10% have crossed the halfway mark. The orchestration layer is being built well ahead of the orchestrated portfolio it is meant to run.

That gap shapes the architecture enterprises are putting in place. By the end of 2026 a clear majority (51%) expect a hybrid control plane — provider-native plus external orchestration — and only 6% expect to hand control to a provider-managed service, because vendor lock-in (35%) is the risk they fear most if control lives inside a model provider. Investment follows the build-out: agent workflow tooling leads the spend (34%), with security and permissions enforcement (25%) behind. And fiscal control lags throughout — more than a quarter (27%) have no real-time way to stop a runaway agent before the bill arrives.

Methodology

VentureBeat fielded this survey as part of its ongoing Pulse Research series, this instrument focused on enterprise agent orchestration. Responses are filtered to organizations with 100 or more employees (n=101), drawn from a single June 2026 wave; because this is one wave rather than a pooled multi-month sample, the report reads cross-sectionally and does not infer month-over-month trends.

By organization size the sample is spread evenly across the enterprise bands: 100–499 employees, 2,500–9,999, and 50,000+ (21% each), with 10,000–49,999 and 500–2,499 (19% each). By role it is senior and buyer-credible: product and program managers (15%), CIO/CTO/CISO (13%), consultants and advisors (13%), and a spread of data, AI, and engineering directors and VPs, with an “Other” function at 18%. On purchasing, 81% are recommenders, influencers, or final decision-makers for AI solutions (66% recommender/influencer, 15% final decision-maker). Technology/Software is the largest industry at 44%, followed by Financial Services (17%) and Healthcare/Life Sciences (8%).

At 101 respondents the sample is robust enough to read directionally with reasonable confidence, though it remains self-selected and is not a probability sample.

Finding 1: Orchestration runs on model-provider platforms

Anthropic’s Claude leads; open frameworks are marginal

We asked which agent orchestration platform enterprises primarily use today. The answer concentrates on the major model providers — and on one in particular.

A note on reading these shares. As described in the methodology section, the respondents are self-selected, and this question asked them for a single primary platform — so the figures measure which platform leads each enterprise's deployment, within a self-selected audience of AI-active technical decision-makers. A sample built this way can diverge substantially from spend-weighted market measures, and each VB Pulse survey draws its own sample with its own company-size mix, so vendor figures should not be compared across our surveys either. Read these shares as a portrait of where this cohort has placed its primary orchestration bet today, rather than as market share.

The model platforms dominate. Anthropic, Microsoft, OpenAI, Google, and Amazon together account for roughly 80% of deployments (81 of 101), while the open frameworks (LangChain/LangGraph) and custom in-house builds that anchor engineering discussion sit in single digits. Anthropic’s lead — 40%, more than double the next platform — mirrors the “model gravity” selection logic in Finding 2: enterprises are choosing the orchestration layer that comes with the model they want to build on. As with the security vendors in the prior agent-security wave, the tools that define the category in technical circles are not yet where enterprise deployment concentrates. A small 3% are not orchestrating at all.

Respondents rate the platforms they run at 3.94 out of 5 overall (109 answered), with “value for money” specifically at 3.94 and “ease of implementation” the weakest score, at 3.85 — placing orchestration near the bottom of our five-tracker satisfaction range, ahead of only evaluation tooling. A rating just under 4 out of 5, from users of whom 96% plan to change their orchestration approach within the year, reads as provisional acceptance: the platforms work well enough to run today, and not well enough to stop the search for something better. The ratings sit alongside near-universal intent to change; this is a layer enterprises tolerate more than they love.

Finding 2: Model gravity drives platform selection

The base model, not the tooling, decides the platform

We asked what most influenced the orchestration platform choice. The single largest factor is the pull of the underlying model — though flexibility and ease of development follow close behind.

Model gravity leading is the selection-side explanation for Anthropic’s platform lead: enterprises pick the orchestration environment closest to the frontier model they have standardized on. But the next tier complicates the picture — flexibility across models and tools (17%) and ease of development (17%) say enterprises also want to avoid being trapped by that choice, foreshadowing the lock-in fear in Finding 6. Security and permissions (14%) and total cost of ownership (11%) round out a pragmatic buying logic. Performance (latency/memory) sits last at 4%, a reminder that at this stage of adoption the binding constraints are model fit and optionality, not raw speed.

Finding 3: The job is reliable multi-step execution

Enterprises just orchestration by whether it completes the work

We asked what enterprises optimize for — their primary success metric for orchestration. Reliability and multi-step workflow management dominate; developer- and user-facing metrics trail.

Task completion reliability (32%) and multi-step workflow management (28%) together account for 59% of responses (60 of 101): orchestration succeeds, in the enterprise view, when it reliably carries a task through multiple steps to completion. Developer productivity (17%) matters but is secondary — the inverse of its prominence in framework discussion — and end-user experience (9%) is a minor concern, consistent with orchestration being an internal execution problem rather than a UX one. This reliability-first standard is exactly what makes the Chatbot Trap finding so pointed: enterprises define success as dependable multi-step execution, yet most of their deployed “agents” do not yet do multi-step work at all.

The trap is not evenly distributed. Splitting the sample by organization size, 77% of smaller enterprises say a quarter or fewer of their agents do true multi-step work, against 62% of larger ones. Larger enterprises are meaningfully further into genuine multi-step deployment; the chatbot trap is, directionally, a mid-market condition.

Finding 4: Consolidate, productionize, and build in-house

Three strategic moves are nearly tied for the year ahead

We asked what major change enterprises anticipate in their orchestration strategy over the next 12 months. Three moves cluster at the top, almost evenly split.

The top three — building in-house control (25%), standardizing on one framework (24%), and moving agents from sandbox to production (23%) — are statistically indistinguishable and tell a single story: enterprises are moving from experimentation to operational consolidation. They want fewer frameworks, more production exposure, and more ownership of the control layer; only 4% expect no change. The appetite for custom in-house control planes is notable alongside the platform concentration in Finding 1 — enterprises are standardizing on model-provider platforms while simultaneously planning to wrap them in control logic they own, the hybrid posture that Finding 6 makes explicit.

Finding 5: Nearly seven in 10 plan to switch — and the biggest group of movers has no shortlist 

The strategic change enterprises anticipate (previous finding) comes with vendor motion attached. Asked whether they plan to adopt a new, additional, or replacement agent orchestration platform in the next twelve months, more respondents are moving here than in any other layer we track.

Asked which platforms they are considering, the most common answer among those in motion is none yet: 29% of all respondents are evaluating without a shortlist, the largest single response after "not considering a change." Among named candidates, OpenAI leads at 16%, followed by LangChain/LangGraph at 12% and Anthropic at 7% — and notably, the independent frameworks draw roughly double their current usage footprint in forward consideration, the same pattern our security tracker found for specialist vendors. Read with this report's concentration and lock-in findings, the picture completes itself: the major model-platform providers hold roughly four-fifths of today's primary usage, vendor lock-in has become the leading fear, 96% anticipate a strategic change — and now the purchase intent to act on all of it, with the largest bloc of buyers still undecided. The most concentrated layer of the agentic stack is also, as of June, the least settled.

Finding 6: Investment flows to workflow tooling

Tooling and permissions lead the spend; monitoring trails

We asked which orchestration-related investment will grow most next year. Agent workflow tooling leads, with security and permissions enforcement behind.

Workflow tooling leading (34%) is the budget-side expression of the reliability-and-multi-step priority in Finding 3: the money is going to the machinery that strings steps together dependably. Security and permissions enforcement (25%) and scaling infrastructure (20%) follow — the investments required to take agents from sandbox into production, the strategic move in Finding 4. Monitoring and debugging draws a smaller 11%, with another 11% reporting flat budgets. The weight on tooling, permissions, and scaling over pure observability signals that enterprises are spending to build and harden orchestration, not merely to watch it run.

Finding 7: The control plane will be hybrid — and lock-in is why

Enterprises expect to split control between providers and their own layer

We asked where enterprises expect the primary control plane for agents to live by the end of 2026, and what worries them most if that control sits inside a model-provider platform. A clear majority expect a hybrid model — and vendor lock-in is the reason.

Hybrid control is the dominant expectation by a wide margin (51%), and only 6% expect to hand control to a provider-managed service outright. Read together, the hybrid, custom, and externally-abstracted options — every architecture that keeps control at least partly outside the provider — sum to 88% (89 of 101). The reason surfaces directly when we asked about the risk of provider-resident control: vendor lock-in leads at 35% (35 of 101), ahead of security and permissioning limitations (28%) and inflexibility across models and tools (21%). The pattern echoes the prior wave’s “don’t trust the model to police itself” posture — here, enterprises will build on a provider’s platform but decline to be governed entirely by it. The hybrid control plane is the architectural hedge against the lock-in they most fear.

The June figure asserting a preference for a hybrid control plane marks movement from earlier. In the April–May survey (n=145), only 34% expected a hybrid control plane, and a greater number (12%) expected to hand control fully to a provider-managed service. These two snapshots don’t yet measure a confirmed longitudinal trend — but the direction of the conversation is unambiguous: toward keeping control.

Lock-in is also a new arrival as a top concern. In the April–May wave, the leading concern was security and permissioning limitations (32%), with lock-in second at 24%; by June the two had traded places. The worry about provider platforms appears to be maturing from whether they can be secured to whether they can be replaced.

Finding 8: The chatbot trap — most “agents” aren’t agents yet

Enterprises admit most deployments are still chatbot wrappers

We asked enterprises to assess their portfolios honestly: what share of their deployed “agents” are true multi-step orchestrated workflows versus simple single-prompt chatbot wrappers. The answer is the defining finding of this wave.

This is the gap at the center of the report. Combining the bottom two bands, 71% of enterprises (72 of 101) say a quarter or fewer of their deployed “agents” are genuinely orchestrated — and just 10% (10 of 101) have crossed the halfway mark. The ambition documented in the earlier findings — model-provider platforms, reliability-first success metrics, production rollouts, a deliberate control architecture — runs well ahead of the deployed reality, which remains overwhelmingly single-prompt assistants dressed as agents. This is less a contradiction than a roadmap: the platforms, budgets, and strategies are being put in place precisely because the orchestrated portfolio is still so thin. The open question for later waves is how fast the reality closes on the ambition.

Finding 9: Fiscal control is still reactive

Only a minority can stop a runaway agent before the bill arrives

Finally, we asked how enterprises enforce fiscal control over agent token consumption — the risk that an autonomous loop exhausts a budget before anyone intervenes. Most rely on native caps or after-the-fact monitoring; real-time programmatic control is the exception.

More than a quarter of enterprises (27%) admit they have no real-time, programmatic way to stop an agent before a budget-breaking bill arrives — they learn of it from the logs afterward. Another 32% lean entirely on the native caps and throttles built into their primary platform, a control only as good as the provider’s tooling and one that ties back to the lock-in concern of Finding 6. The enterprises building custom gateways (23%) or exploiting cross-model routing to arbitrage cost (19%) are the ones treating token burn as an engineering problem to be controlled deterministically. As with orchestration maturity, fiscal control is an area where the operational reality lags the ambition: agents are moving toward production faster than the cost-control plane around them is being built.

It’s worth noting, a split appears according to company size: roughly one in three enterprises under 2,500 employees (34%) exercises only reactive control of agent spend, against 20% of larger enterprises — directional figures, but consistent with the chatbot-trap split. The mid-market is running the least mature agents on the least instrumented budgets.

The bottom line: The layer is real; most of the agents aren't yet

Organizations with 100 or more employees describe an orchestration strategy that is consolidating quickly and maturing slowly. They are standardizing — for now — on model-provider platforms, which collectively hold roughly four-fifths of primary usage, chosen for the gravity of the underlying model, and they judge success by reliable multi-step execution. Investment is flowing to workflow tooling and permissions, the strategy is to consolidate frameworks and push agents into production, and the control plane they expect is deliberately hybrid, because vendor lock-in is the risk they fear most. But the standardization is provisional: 68% plan to adopt a new, additional, or replacement orchestration platform within twelve months — the highest switching intent of any layer we track — and the largest group of those movers has not yet shortlisted a candidate. Today's concentration describes where enterprises are, and visibly does not describe where they intend to stay.

But the honest self-assessment punctures the ambition. Seventy-one percent say a quarter or fewer of their deployed "agents" are truly orchestrated, only 10% are past the halfway mark, and more than a quarter cannot stop a runaway agent in real time. The orchestration layer — the platforms, the budgets, the control architecture — is being built ahead of the orchestrated portfolio it is meant to run. At 101 respondents in a single June wave this reads as a clear directional signal rather than a precise measurement: enterprises have decided how they want to orchestrate agents well before most of their agents are doing anything an orchestration layer is for. The questions for subsequent waves are whether the deployed reality closes the gap on the ambition — and, with nearly seven in ten buyers in motion and most of them undecided, which platforms the settled stack finally lands on.

Based on survey responses from 101 qualified enterprise respondents (100+ employees), drawn from a single June 2026 wave. Because this is one wave rather than a pooled multi-month sample, results read directionally rather than as a confirmed trend. Respondents include product and program managers, CIOs, CTOs and CISOs, consultants and advisors, and directors and VPs of data, AI, and engineering, across Technology/Software, Financial Services, Healthcare, and other sectors.

This wave of VentureBeat Pulse Research examines how technical leaders measure agent performance: which reliability and evaluation platforms they use, how they select and trust them, what breaks in production, and how far they are willing to let agents run without a human in the loop.

The central finding is an evaluation gap — the distance between the autonomy enterprises are granting their agents and the trust they place in the evaluations meant to govern it. Half of organizations (50%) have, in the past year, deployed an agent or LLM feature that passed their internal evaluations and then caused a customer-facing failure, and a quarter have seen it happen more than once. Trust in the tests themselves is thin: only 5% say they fully trust automated evaluation today, and the single most-cited limitation is that evaluations align poorly with real-world outcomes (29%). Enterprises are discovering that a passing eval is not the same as a working agent.

What makes the gap consequential is the direction of travel. Two-thirds of organizations (66%) already permit fully automated, zero-human-in-the-loop deployment for low-risk agents (34%) or are actively engineering their pipelines to allow it within twelve months (33%). At the same time, the evaluation stack that would have to earn that trust is fragmented and immature: the most common primary tools are the model providers’ native evals, tied with having no dedicated tooling at all (17% each); and only about a quarter of enterprises run real-time quality checks on live production traffic. The autonomy is arriving faster than the assurance.

Methodology

VentureBeat fielded this survey as part of its ongoing Pulse Research series, this survey — the Agentic Reliability & Evals tracker — focused on how technical leaders evaluate agent performance and reliability. Responses are filtered to organizations with 100 or more employees (n=157), drawn from a single survey in June 2026; because this is one wave rather than a pooled multi-month sample, the report reads cross-sectionally and does not infer month-over-month trends. Where questions were multiple-select, those shares can sum to more than 100%.

By role the sample is senior and buyer-credible: 38% are final decision-makers for AI purchases and another 34% recommenders or influencers. Product and program managers (15%), consultants and advisors (10%), directors of engineering/IT (8%), and CIOs/CTOs/CISOs (8%) lead the named titles, alongside a large “Other” function (37%). By organization size the sample is mid-market-weighted: 100–499 (37%) and 500–2,499 (27%) employees lead, with 2,500–9,999 (20%), 10,000–49,999 (10%), and 50,000+ (6%) above them. Technology/Software is the largest industry at 23%, followed by Retail/Consumer (15%), Healthcare/Life Sciences (12%), and Manufacturing (10%).

At 157 respondents the sample is large enough to read directionally but should be treated as a directional signal rather than a precise measurement; it is self-selected and is not a probability sample. It skews toward the mid-market, so it is best read as the view from organizations actively standing up agent evaluation practices rather than from the largest operators.

Note: This survey was rebuilt for the June wave from the earlier “LLM observability and evaluations” survey; because the questions and sample differ, no comparisons are made to the April–May data.

Finding 1: A passing eval is not a working agent

Half have shipped an agent that passed evals, then failed a customer

We asked whether, in the past 12 months, organizations had deployed an agent or LLM feature that passed their internal evaluations but then caused a customer-facing failure. Half of those that run evaluations had.

This is the report’s defining number. Half of organizations (50%) have shipped an AI feature that cleared their internal evaluations and then failed in front of a customer — an incorrect output, a broken workflow, or a quality incident — and a quarter have seen it happen more than once. Only 36% report no such failure, and the remainder either run no pre-deployment evaluations (8%) or don’t track the root cause closely enough to know (6%). The failure is precise and expensive: the evaluation said the agent was ready, and it was not. Everything that follows — how enterprises trust their evals, what they monitor, and how much autonomy they grant — is shaped by this experience.

Finding 2: Almost no one fully trusts automated evaluation

The top complaint: Evals don't match real-world outcomes

We asked which limitation most reduces trust in automated agent evaluations today. Only a sliver of enterprises had no complaint at all.

Trust in automated evaluation is scarce, and specific. Only 5% of organizations say they fully trust automated evaluation as it stands — meaning 95% name a limitation that holds them back. The most common, at 29%, is the one that most directly explains Finding 1: evaluations align poorly with real-world outcomes, passing agents that later fail. Bias or inconsistency (21%) and a lack of explainability (18%) follow — enterprises cannot always tell why an evaluation reached its verdict — and 17% cite data-leakage or privacy concerns in the evaluation process itself. The tests meant to certify agents are not yet trusted to certify them, which is precisely why the autonomy trajectory in Finding 3 is so striking.

Finding 3: The autonomy ceiling is rising anyway

Two-thirds already allow, or are building toward, zero-human deployment

We asked whether organizations would let an autonomous agent deploy a code or system change to production on automated evaluation results alone, with no human-in-the-loop validation. The trajectory runs straight through the trust gap.

Here is the paradox at the heart of the report. Even though almost no one fully trusts automated evaluation (Finding 2), two-thirds of organizations (66%) either already allow zero-human-in-the-loop deployment for low-risk agents (34%) or are actively engineering their pipelines to permit it within a year (33%). Only 22% rule it out for the foreseeable future. The direction is unambiguous: enterprises are moving to let evaluations gate production autonomously — removing the human check — at the same moment they say those evaluations don’t reliably match reality. The autonomy ceiling is rising faster than the assurance beneath it, which is the mechanism by which the false-confidence failures of Finding 1 will scale rather than shrink.

Notably, the autonomy bet is not just a small company phenomenon. Splitting the sample by company size, larger enterprises are slightly further down the path toward zero human review than smaller companies (70% versus 64%) and slightly more likely to have shipped an evaluation-passing agent that then failed a customer (54% versus 48%). The assumption that large, regulated organizations are holding the human in the loop longest is, in this sample, backwards.  To be sure, these are directional figures, since the survey was not a huge sample — 57 respondents from companies with 2,500+ employees and 100 from companies smaller than that. 

Finding 4: The evaluation stack is fragmented and provider-led

Provider-native evals lead — tied with no dedicated tool at all

We asked which agent reliability or evaluation platform enterprises primarily use today. The market has no clear leader — and a large share has nothing dedicated.

The evaluation layer is early and unconsolidated. Provider-native tooling leads — OpenAI’s native evals and traces (17%) and Anthropic’s Claude Console evals (13%) together outweigh any independent platform — but it is tied at the top by a striking answer: 17% of enterprises use no dedicated agent-evaluation tooling at all, a notable gap for organizations shipping agents to customers. The specialist evaluation vendors — DeepEval (12%), Braintrust (8%), LangSmith, Weave, Promptfoo, Langfuse, Arize — are scattered across single to low double digits, and 11% have built their own. No independent platform has yet become the category standard, which leaves most enterprises evaluating agents with provider-native tools, home-grown scripts, or nothing.

Finding 5: Production monitoring rarely watches output quality

Only a quarter run real-time quality checks on live traffic

Production monitoring for an AI agent can watch two very different things. It can watch whether the system is functioning — is the agent up and responding, did each request complete, how fast, at what cost, with any errors. Or it can watch whether the agent's output is correct — automated checks that evaluate the content of each answer as it goes out: did the agent give the right answer, take the right action, stay within policy. The distinction matters because a confidently wrong answer is invisible to the first kind of monitoring: the request completes, the response is fast, no error is thrown, and every functioning-metric reads healthy. We asked organizations which kind their live production monitoring is built for today.

Grouped by what is actually being watched, the split is stark: 51% of organizations monitor only whether the agent is functioning, while 23% monitor whether its answers are right. Counting the ad-hoc reviewers and the don't-knows, roughly three-quarters of organizations run no automated, real-time evaluation of output correctness in production — they can see that the system is up and what it costs, and they are taking the correctness of its answers on faith. That blind spot is the runtime counterpart to the pre-deployment gap in Finding 1: the same organizations engineering the human out of the deployment decision mostly cannot see, in real time, when the deployed agent starts getting things wrong.

Finding 6: Bought on cost, measured on consistency

Price and integration drive selection; evaluation consistency is the goal

We asked what most influenced enterprises’ choice of an evaluation vendor, and what they treat as their primary measure of success. Both answers are pragmatic.

Enterprises buy evaluation tooling on economics and trust it on repeatability. Cost of evaluations (28%) narrowly leads selection, just ahead of ease of integration (27%) and evaluation accuracy (24%) — breadth of observability (13%) and vendor roadmap (4%) matter far less. On what success looks like, more than a third (36%) name evaluation consistency — getting the same verdict on the same behavior every time — well ahead of speed of experimentation (19%), reduction in failures (18%), production visibility (13%), and compliance (11%). The emphasis on consistency is telling: before enterprises can trust an evaluation’s verdict, they need it to be stable — the very property whose absence (bias and inconsistency) ranked among the top trust limitations in Finding 2. Satisfaction with current tooling is only moderate, averaging 3.8 on a five-point scale across overall satisfaction, ease of implementation, and value for money.

Finding 7: The next dollar goes to humans and observability

Investment is flowing to oversight, not just automation

We asked which reliability and evaluation investment will grow most over the next year. The money is going toward watching agents more closely — including with people.

The second-largest planned investment — behind only production observability — is human review workflows, at 26%. Read against Finding 1, that is the report's quietest contradiction: at the same moment two-thirds of enterprises are engineering the human out of the deployment decision, more of them plan to grow spending on human reviewers (26%) than on the automated evaluation pipelines (16%) that would replace them. The zero-human trajectory and the human-review budget are rising in the same companies at the same time. Indeed, only 8% report that their budget is not increasing.

Taken together, enterprises are hedging: building toward autonomy while spending to watch agents more closely and keep humans available for the calls that automated evaluation cannot yet be trusted to make.

Finding 8: A tooling reshuffle is coming

Nearly two-thirds plan to adopt or switch platforms within a year

We asked whether enterprises plan to adopt a new, additional, or replacement evaluation platform, and which they are considering. Few intend to stand pat.

The evaluation market is wide open. While 36% have no plans to change, a clear majority (64%) intend to adopt a new, additional, or replacement platform within twelve months, and 31% within the next quarter. The consideration set points where current usage is thinnest: Confident AI’s DeepEval leads what enterprises are evaluating (20%), ahead of OpenAI’s native evals (13%) and Braintrust (9%) — the open-source specialists drawing more interest than their present footprint.

Given that so many enterprises today rely on provider-native tools or nothing at all (Finding 4), this is less a defection than a first real wave of tooling adoption — the moment the evaluation layer starts to consolidate. Which platforms earn that trust, in a market where almost no one trusts automated evaluation yet, is the open question this series will keep tracking.

The bottom line: An evaluation gap that autonomy will widen, not close

Organizations with 100 or more employees are granting AI agents more independence than they trust their evaluations to support. Half have already shipped an agent that passed its evals and then failed a customer; almost none fully trust automated evaluation, chiefly because it doesn’t match real-world outcomes; and most watch production for uptime and cost rather than for whether the agent’s answers are right. Yet two-thirds already allow, or are actively building toward, deploying to production on automated evaluation alone.

The vendor market is early and unsettled: the most common primary evaluation tools are provider-native evals, tied with no dedicated tooling at all, and a clear majority plan to adopt or switch platforms within the year. Encouragingly, the next dollar is going to observability and — pointedly — human review, suggesting enterprises sense the gap even as they engineer past it. At 157 respondents in a single wave this is a directional read, skewed toward the mid-market — but the direction is clear: autonomy is being granted on the strength of evaluations that the people granting it do not yet trust. The evaluation gap is not a coverage problem that more tests alone will close; it is a problem of evaluations that reflect reality and can be trusted to gate it. The open question for later waves is whether assurance catches up to autonomy — or whether the false-confidence failures move from customer incidents into changes that deploy themselves.

Based on survey responses from 157 qualified enterprise respondents (100+ employees), drawn from a single June 2026 wave. This is a directional read rather than a precise measurement — the sample is self-selected, not a probability sample, and skews toward the mid-market. Respondents include product and program managers, consultants and advisors, directors of engineering/IT, and CIOs/CTOs/CISOs, among other functions, across technology/software, retail/consumer, healthcare/life sciences, manufacturing, and other industries.

This wave of VentureBeat Pulse Research examines the enterprise RAG and context layer: what feeds AI agents their business context, which retrieval systems enterprises run, how they buy and measure them, where the architecture is heading, and — most revealingly — how often that context is already failing them.

The central finding is a context gap — the distance between how confidently enterprise agents answer and how reliable the context beneath them actually is. A majority of enterprises (57%) report that in the past six months their AI agents produced confident but wrong answers they traced to missing or inconsistent business context, and more than half of those said it happened more than once. This is not a fringe failure: retrieval is the primary context source for 38% of enterprises, more than any other approach, so when retrieval is thin or inconsistent, the errors it produces are wearing the agent’s authority. The infrastructure to fix it is being built — 58% already run or are building a governed semantic layer — but for most it is not yet in production.

Underneath, the market is consolidating in a direction that surprises. Provider-native retrieval — OpenAI’s file search (40%) and Google’s Vertex AI Search (38%) — already leads every dedicated vector database, and enterprises expect hybrid retrieval to dominate by the end of 2026 (34%). Yet a plurality (36%) say they intend to keep best-of-breed standalone tools rather than consolidate onto a provider’s native context stack, and a majority (57%) plan to switch or add a provider within the year. Stated preference and actual usage are pulling in opposite directions — the market is buying provider-native while insisting it wants independence.

Methodology

VentureBeat fielded this survey as part of its ongoing Pulse Research series. This survey focused on enterprise RAG infrastructure and the context layer — the retrieval systems, semantic layers, and context sources that feed AI agents. Responses are filtered to organizations with more than 100 employees (n=101); the survey drew no responses from organizations of 100 or fewer, so the full sample qualifies. All responses are from a single Q2 2026 (June) wave, so the report reads cross-sectionally and does not infer month-over-month trends. Several questions were multiple-select, so those shares can sum to more than 100%.

By organization size the sample concentrates in the mid-market: 251–1,000 employees (31%) and 101–250 (31%) lead, with 1,001–5,000 (20%), 5,001–10,000 (12%), and 10,001+ (7%) above them. By role it spans managers (39%), individual contributors (27%), the C-suite (16%), and VPs and directors (14%); on purchasing authority it is buyer-credible, with 46% final decision-makers and another 26% recommenders or influencers. Technology/Software is the largest industry at 20%, followed by Healthcare/Life Sciences (11%) and a broad spread across retail, transportation, financial services, manufacturing, and education.

At 101 respondents this is a modest sample and should be read as a directional signal rather than a precise measurement; it is self-selected and is not a probability sample. It is best read as the view from organizations actively standing up RAG and context infrastructure rather than from the largest operators.

Finding 1: Confident and wrong

More than half have traced agent errors to bad context

We asked whether, in the past six months, enterprises had traced a confident but wrong agent answer to missing or inconsistent business context. Most had.

This is the report’s defining number. A majority of enterprises (57%) have already had an AI agent produce a confident, wrong answer they traced to bad context — wrong metrics, stale definitions, or missing documents — and more than half of those have seen it happen more than once. Only 28% report no such failure, and a small remainder either don’t run agents on enterprise data or don’t trace root cause closely enough to know.

The failure mode is specific and dangerous: the model is not obviously hallucinating; it is confidently wrong because the context feeding it was thin or inconsistent. Everything else in this report — what enterprises retrieve, how they govern it, and what they plan to build — is downstream of this problem.

Finding 2: RAG is the default context source

Retrieval feeds more agents than any other method

We asked what an enterprise’s AI agents primarily use to understand its data. Retrieval leads by a wide margin.

Retrieval is the backbone of enterprise context. For 38% of organizations, RAG over documents or a vector index is the primary way agents understand the business — nearly twice the share of the next approach, a governed semantic layer or ontology (21%). Mixed approaches (14%), direct live-system queries (10%), and long-context loading (6%) fill out the rest, and only 2% let agents run on the model’s general knowledge alone. The concentration matters in light of Finding 1: because so much enterprise context flows through retrieval, the quality of that retrieval is the quality of the answer. When RAG is the default source, thin retrieval is not an edge case — it is the main failure surface.

One approach is notable for its absence from these answers: customizing model weights, also known as fine-tuning. Every leading source of business context is injected at run time. Our most recent direct measurement of fine-tuning comes from our April–May survey wave (a separate survey, n=136), where fine-tuning capabilities ranked last of six factors in model selection at 5% — even as 26% of that sample still named fine-tuning and customization an investment they expect to grow. Fine-tuning has fallen out of the primary selection conversation; context injection is how enterprises make agents knowledgeable about their business.

Finding 3: Provider-native retrieval already leads the vector databases

OpenAI file search and vertex AI search top the dedicated tools

We asked which retrieval systems enterprises run in production today. The answer favors the model providers and hyperscalers over the specialists.

The dedicated vector database is no longer the center of the RAG stack. OpenAI’s file search (40%) and Google’s Vertex AI Search (38%) lead — provider-native and hyperscaler-native retrieval — ahead of every purpose-built vector database. Among the specialists, the most-used is the one enterprises already run for other reasons (Elasticsearch/OpenSearch, 20%) and the open, embedded option (pgvector, 12%); the pure-play vector databases that define the category — Weaviate, Qdrant, Pinecone, Milvus — each sit in single digits to low double digits. Notably, 13% of enterprises say they still run no production RAG at all. As with the platforms in the parallel infrastructure wave, enterprises are gravitating to retrieval that comes bundled with tools they already buy.

The shape of this finding held across both Q2 waves. In April–May (n=161), provider-built retrieval led usage there too, while every dedicated vector database remained marginal — the most-used standalone vector database peaked at 8% of that sample — and the hybrid, pluralistic future was already the consensus expectation (34% expected hybrid retrieval to dominate, with another 29% expecting multiple architectures by use case). Two waves, consistent picture: the category that coined the “vector database” term is being collected by the platforms enterprises already buy from.

Finding 4: But they say they want to keep best-of-breed

A plurality resist consolidating onto a provider’s native stack

We asked how enterprises will respond as model providers bundle retrieval, memory, and orchestration into their platforms. Their stated intent cuts against their current usage.

Here is the tension at the heart of the stack. Even as provider-native retrieval leads in practice (Finding 3), a plurality of enterprises (36%) say they intend to keep best-of-breed standalone tools rather than consolidate onto a provider’s native context stack — well ahead of the 21% who plan to consolidate. Another 21% expect a mix, and 9% intend to build and own the layer themselves. The gap between what enterprises run and what they say they want is the strategic question of the category: they are adopting bundled retrieval for convenience while asserting they will preserve independence. Which impulse wins — the pull of the provider bundle or the stated preference for modular control — will shape the retrieval market more than any single tool.

Finding 5: Hybrid retrieval is the consensus bet

Vector-only retrieval is already seen as insufficient

We asked which retrieval architecture enterprises expect to dominate their production RAG systems by the end of 2026. The field is converging — with a large share still unsure.

The architecture is settling on hybrid. A third (34%) expect hybrid retrieval — embeddings combined with reranking and access controls — to dominate their production systems by the end of 2026, three times the 11% who expect vector-only retrieval to prevail. That is a notable signal: the pure vector-search approach that launched the category is already viewed as insufficient on its own, superseded by pipelines that add reranking for accuracy and access controls for governance — the very access controls whose absence produces the failures in Finding 1. Tellingly, the second-largest answer is uncertainty: 17% simply don’t know, and another 14% expect to move beyond a dedicated vector layer entirely toward tool-first or long-context retrieval. The consensus is not a single tool but a layered pipeline — and it is not yet fully formed.

Finding 6: The governed context layer is being built now

Most run or are building a semantic layer — few in production

We asked whether enterprises use a governed semantic or context layer to give agents and BI a shared understanding of their data. Most are on the path; fewer have arrived.

The fix for the context gap is under construction. Well over half of enterprises (58%) either run a governed semantic layer in production (25%) or are piloting and building one (34%), and a further 17% are actively evaluating — meaning three-quarters are engaged with the idea in some form. But the balance is telling: more are building than have shipped, so for most enterprises the shared, governed definition layer that would prevent the "confident but wrong" failures of Finding 1 is still a work in progress. The semantic layer is the industry’s answer to inconsistent context; this wave catches it mid-construction, ambition well ahead of production.

Finding 7: Bought on ingestion and simplicity, watched for correctness

Selection favors operability; monitoring favors correctness and security

We asked what matters most when enterprises choose a retrieval system, and what they track once it is running. Both answers lean practical.

Enterprises choose retrieval systems on operability. Ease of data ingestion (36%), latency and performance (32%), and operational simplicity (29%) lead the selection criteria — ahead of retrieval accuracy and access control (23% each), the two factors most directly tied to the failures in Finding 1. Once systems are running, the emphasis shifts toward trust: the most-tracked metrics are response correctness (42%) and security and access control (38%), ahead of latency (28%), operational stability (27%), and answer relevance (23%).

Satisfaction with current systems is moderately positive but not enthusiastic — on a five-point scale, overall satisfaction averages 4.0, with ease of implementation and value for money both near 3.9. Enterprises buy for how easily a system runs and watch it for whether it can be trusted.

Finding 8: A retrieval reshuffle is coming

A majority plan to change providers — and the vector specialists are gaining interest

We asked whether enterprises plan to change or add a retrieval provider, and which they are considering. The consideration set differs from today’s stack.

The retrieval stack is not settled. While 43% have no plans to change, a small majority (57%) intend to switch or add a provider within twelve months, and a quarter (26%) within the next quarter. The consideration set is where it gets interesting: provider-native retrieval still leads what enterprises are evaluating (OpenAI 22%, Vertex AI Search 21%), but the open-source vector specialists punch above their current footprint — Qdrant (14%) and Milvus (13%) draw more switching interest than their present usage (10% and 6%) would suggest. Read with Finding 4, the picture is a market in flux: enterprises run provider-native today, are evaluating a broader field, and say they want to keep their options open. The reshuffle ahead will test whether best-of-breed intent survives contact with the convenience of the bundle.

The bottom line: A context gap that more retrieval alone won’t close

Organizations with more than 100 employees are wiring agents into their business faster than they can guarantee the context those agents run on. Retrieval is the default source of enterprise context, and it increasingly comes from the model providers and hyperscalers rather than the dedicated vector databases — yet a majority of enterprises have already watched agents answer confidently and wrongly because that context was thin or inconsistent. The failure is not exotic; it is the predictable result of pointing authoritative-sounding agents at an unreliable foundation.

The industry’s answer — a governed semantic layer, hybrid retrieval with reranking and access controls — is being built but is mostly not yet in production, and enterprises are pulled between the convenience of provider-native bundles and a stated preference for best-of-breed independence. At 101 respondents in a single Q2 wave this is a directional read, skewed toward the mid-market — but the direction is clear: the context layer is the next contested tier of the AI stack, and right now agents are running ahead of it. The context gap is not a retrieval-volume problem that more documents or bigger indexes will solve on their own; it is a problem of governed, consistent, access-aware context. The open question for later waves is whether enterprises finish building that layer before the confident-but-wrong failures move from the lab into decisions that matter.

Based on survey responses from 101 qualified enterprise respondents (100+ employees), drawn from a single Q2 2026 (June) wave. At this sample size the results should be read as a directional signal rather than a precise measurement — it's a self-selected sample, not a probability sample, and skews toward the mid-market. Respondents include managers, individual contributors, VPs/directors, and the C-suite, with strong purchasing authority, across technology, healthcare, retail, transportation, financial services, manufacturing, and education.

This wave of VentureBeat Pulse Research examines how enterprises secure their AI agents: what tooling they run, how they manage agent identity and isolation, what has already gone wrong, how much they spend, and whether they believe their defenses are keeping pace with AI-enabled attackers.

The central finding is an agent security gap — the distance between the autonomy enterprises are granting their agents and the controls in place to contain them. More than half of organizations (54%) have already experienced a confirmed agent security incident (18%) or a near-miss caught before harm (36%). The structural weakness beneath those numbers is identity: only about a third (32%) give every agent its own scoped, managed identity, while the rest report that some agents share credentials or that agents mostly run on shared API keys and human or service-account credentials. When agents share credentials, a single compromised or over-permissioned agent carries a wide blast radius — and only three in ten enterprises (30%) isolate their highest-risk agents in sandboxes to bound that radius.

What makes the gap notable is how comfortable enterprises are inside it. The security stack is overwhelmingly provider-native — OpenAI’s guardrails (51%), Google’s and Microsoft’s cloud controls, and Anthropic’s managed-agent controls dominate, while the dedicated agent-security specialists barely register — and satisfaction with that borrowed stack is high, averaging 4.2 out of 5. Yet spending remains a thin slice of the security budget, only a third of enterprises believe their AI defenses are ahead of AI-enabled attackers, and a clear majority plan to change tooling within the year. Enterprises are satisfied with controls they are simultaneously preparing to replace.

Methodology

VentureBeat fielded this survey as part of its ongoing Pulse Research series, this instrument focused on enterprise agent security — the tooling, identity, isolation, and enforcement controls organizations use to secure autonomous AI agents. Responses are filtered to organizations with more than 100 employees (n=107; the survey’s smallest size band, 1–100 employees, is excluded), drawn from a single June 2026 wave. Because this is one wave rather than a pooled multi-month sample, the report reads cross-sectionally and does not infer month-over-month trends. Several questions were multiple-select, so those shares can sum to more than 100%.

By role the sample is senior and buyer-credible: 45% are final decision-makers for AI purchases and another 30% recommenders or influencers. Managers (43%), individual contributors (24%), VPs and directors (15%), and the C-suite (11%) make up the seniority mix. By organization size the sample is mid-market-weighted: 251–1,000 (42%) and 101–250 (25%) employees lead, with 1,001–5,000 (19%), 5,001–10,000 (8%), and 10,001+ (7%) above them. Technology/Software is the largest industry at 23%, followed by Manufacturing (15%), Retail/E-commerce (14%), and Healthcare/Life Sciences (13%).

At 107 respondents the sample is large enough to read directionally but should be treated as a directional signal rather than a precise measurement; it is self-selected and is not a probability sample. It skews toward the mid-market, so it is best read as the view from organizations actively standing up agent security rather than from the largest operators.

Satisfaction ratings are computed on the respondents who answered each rating question; the overall satisfaction score reflects 82 of the 107 qualified respondents.

Finding 1: The incidents are already here

More than half have had an agent security incident or near-miss

We asked whether organizations had experienced an agent security incident — a confirmed breach, or a near-miss caught before harm. Most that run agents in production had.

This is the report’s defining number. More than half of organizations (54%) have already had an agent security event — 18% a confirmed incident and 36% a near-miss caught before it caused harm. Only 42% report nothing, and a small remainder either run no agents in production or don’t track such events. That so many report near-misses rather than only confirmed incidents is telling: enterprises are catching problems, but they are catching them close to the edge. The controls examined in the rest of this report — identity, isolation, enforcement — are what determine whether the next near-miss stays a near-miss.

Exposure scales with company size, but containment does not. The incident-or-near-miss rate rises from 49% in the mid-market (companies with 101-1,000 employees) to 63% at larger enterprises (above 1,000 employees), while sandbox isolation of high-risk agents falls from 35% to 20%, and satisfaction with security tooling drops from 4.36 to 3.97. The organizations running the most agents across the most systems carry the most incidents and the least of the one control that bounds an incident's blast radius.

Finding 2: The identity gap

Only a third give every agent its own scoped identity

We asked how enterprises manage the identity of their AI agents — whether each agent has its own credentials, or agents share them. Full per-agent identity is the exception.

Rolled together, the overlapping answers show 69% of enterprises (74 of 107) with credential sharing somewhere in the agent fleet. Identity is the structural weakness beneath the incidents. Only about a third of enterprises (32%) give every agent its own scoped, managed identity — the precondition for least-privilege access and clean attribution. Nearly half (48%) say some agents have scoped identities but many still share credentials, and another 32% say agents mostly run on shared API keys or borrowed human and service-account credentials. (Respondents could describe more than one pattern across their agent fleet, so these overlap.)

The consequence is direct: when agents share credentials, an over-permissioned or compromised agent can act with far more reach than intended, and forensics after an incident cannot cleanly tell which agent did what. The non-human identity problem — giving every agent its own governed identity — is the single largest unfinished piece of enterprise agent security.

Moreover, a company’s agent credential posture is correlated with incidents. Organizations with credential sharing anywhere in the fleet were hit — with an incident or a near-miss in the past twelve months — at 63.5% (47 of 74). Organizations where every agent carries its own scoped identity were hit at 40.9% (9 of 22). The fully-scoped group is small, so for now the relationship is an association rather than proven causation, and the gap is concentrated in the mid-market — but within a single survey, a twenty-three point difference in incident rate suggests significance.

Finding 3: Observe and enforce, but rarely isolate

Only three in 10 sandbox their highest-risk agents

We asked what an organization’s agent security posture looks like in practice — whether they observe, enforce, isolate, or some combination. The control that bounds damage is the least common.

Monitoring and enforcement are reasonably common; containment is not. Roughly half of enterprises observe agent activity (47%) or enforce scoped permissions at runtime (49%), but only 30% isolate their highest-risk agents in sandboxes that bound the blast radius when the other controls fail. That ordering is backwards from a defense-in-depth standpoint: observation tells you what happened, enforcement tries to prevent it, but isolation is what limits the damage when prevention fails — and it is the control enterprises have adopted least. Combined with the identity gap in Finding 2, the picture is of agents that are watched and permissioned but rarely boxed in, which is precisely the configuration in which a single failure propagates.

Finding 4: Security runs on borrowed, provider-native controls

Guardrails from OpenAI, Google and Microsoft dominate; specialists barely register

We asked which agent security tooling enterprises use, and which is their primary layer. The answer favors the model providers and hyperscalers over the dedicated security vendors.

Enterprises are securing agents with tools that came bundled with their models and clouds. OpenAI’s guardrails lead at 51%, followed by Google’s and Microsoft’s cloud-native controls and Anthropic’s managed-agent controls — and when asked to name their single primary security layer, 82% name one of these provider-native offerings. The purpose-built agent-security category — Palo Alto’s Prisma AIRS, CrowdStrike, Cisco AI Defense, Zenity, HiddenLayer, Check Point’s Lakera, Okta for AI Agents, non-human identity platforms — barely registers, each in the low single digits, and only 5% run no dedicated tooling at all. As with retrieval and evaluation elsewhere in this series, the provider bundle is winning the default: enterprises reach first for the guardrails their platform ships, and the independent security layer that would address the identity and isolation gaps has not yet been adopted at scale.

The provider-default pattern is consistent across both Q2 survey waves. In April–May (n=110), usage was led by the same names — OpenAI's controls at 26%, Azure at 15%, AWS at 14%, Google at 12% — with every dedicated agent-security specialist at 3% or below and one in ten using no dedicated tooling at all. The common finding from the two surveys: Enterprises are defaulting to the solutions provided by the platform they’re using, and the specialist category vendors have yet to become big players here.

(A note on reading these shares. As described in the methodology section, the respondent sample is self-selected and skews mid-market, and the usage question counted every vendor or approach a respondent has in place — so the figures measure presence in the security stack rather than spending or exclusivity. Individual vendor percentages therefore carry all the usual sample caveats. The structural pattern, however, held across both Q2 waves on two differently worded questions: provider-native and hyperscaler controls lead, and dedicated agent-security specialists remain in low single digits. Read the individual shares loosely and the pattern with confidence.)

Finding 5: And enterprises are comfortable with it

Satisfaction is high, even as incidents mount and identity lags

We asked how satisfied enterprises are with their current agent security tooling. The comfort is notably out of step with the exposure documented above.

Satisfaction with agent security tooling is high — 4.2 out of 5 overall, and 4.1 for value for money — among the most positive readings in this series. That is the striking part: enterprises are highly satisfied with a stack that is mostly borrowed provider guardrails, even though more than half have already had an incident or near-miss and only a third give their agents scoped identities. The comfort appears to rest on the convenience and low friction of provider-native controls rather than on demonstrated containment. It is a false comfort in the making — the same enterprises expressing satisfaction are, as Finding 8 shows, a clear majority planning to change tooling within the year, which suggests the confidence is thinner than the score implies.

Finding 6: Budgets haven’t caught up

Most spend under a tenth of the security budget on agents

We asked what share of the security budget enterprises allocate to securing AI agents. For a fast-emerging risk, the allocation is modest.

Spending on agent security is still a thin slice. The most common allocation is 6–10% of the security budget (46%), and a third of enterprises (34%) spend 5% or less; only a quarter (24%) devote more than a tenth. Given the incident rate in Finding 1 and the identity and isolation gaps in Findings 2 and 3, the budget looks like a lagging indicator — the risk has arrived faster than the funding to address it. The enterprises spending more than a tenth of their security budget on agents are a distinct minority, and they are likely the ones building the scoped-identity and isolation controls the rest have not.

Finding 7: The arms race is even, at best

Only a third think their AI defenses are ahead of AI-enabled attackers

We asked how enterprises assess the balance between their AI-enabled defenses and AI-enabled attackers. Confidence is far from settled.

Enterprises are split on whether they are winning. Only about a third (35%) believe their AI-enabled defenses are ahead of AI-enabled attackers; the rest are less sure — 32% call it roughly even, 21% think attackers are ahead, and another 21% say it is too early to tell. Taken together, a clear majority (53%) rate the balance as even or tilted toward the attacker. That uncertainty sits uneasily beside the high satisfaction of Finding 5: enterprises are content with their tooling yet unconvinced it is winning the contest it exists to win. In a domain where the offense is also compounding with AI, an even race is not a comfortable place to be.

Finding 8: A security reshuffle is coming

Nearly six in 10 plan to adopt or switch tooling within a year

We asked whether enterprises plan to adopt a new, additional, or replacement agent security solution, and which they are considering. Few intend to stand pat.

The security stack is not settled. While 41% have no plans to change, a clear majority (59%) intend to adopt a new, additional, or replacement agent security solution within twelve months, and 29% within the next quarter — a strong signal that, high satisfaction notwithstanding, enterprises know the current stack is provisional. Incidents are what start the buying cycle.

Among organizations that have been hit, 42.1% plan to adopt, add, or replace agent security tooling within the next ninety days, against 14.0% of organizations with no incident — and after a confirmed incident it becomes majority behavior, at 52.6%. Getting hit also changes the threat assessment: 33.3% of hit organizations say AI-armed attackers are ahead of their defenses, against 8.0% of the unhit. Experience, in this data, is the strongest predictor of both urgency and pessimism.

The consideration set still leans provider-native (OpenAI 34%, Google 30%, Anthropic 29%, Azure 25%), but the dedicated security vendors — Cloudflare, Cisco, Palo Alto, Okta, Check Point’s Lakera — draw early interest in the mid-to-high single digits, more than their current footprint. 

What the shopping does not yet include is the identity layer specifically. Twelve percent of the respondents include an agent-identity product — Okta for AI Agents, Microsoft Entra Agent ID, or a non-human identity platform — anywhere in their consideration set, and among the credential-sharing organizations that have already had an incident, identity consideration is essentially unchanged, at roughly one in ten. The control most directly implicated by the incident data is the one largely missing from the purchase plans. Whether this wave hardens the provider-native default or finally opens the door to purpose-built agent security — the identity and isolation controls the incidents call for — is the question this series will keep tracking.

The bottom line: A security gap that autonomy will test first

Organizations with more than 100 employees are giving AI agents real reach into systems and data while securing them with controls built for something else. More than half have already had an incident or near-miss; only a third give every agent its own scoped identity, and most still share credentials; only three in ten isolate their highest-risk agents; and the stack doing this work is overwhelmingly borrowed from the model providers and hyperscalers rather than purpose-built for agents.

The uncomfortable pairing is confidence with exposure: satisfaction with the current tooling is among the highest in this series, yet spending is a thin slice of the security budget, only a third believe their defenses are ahead of AI-enabled attackers, and a clear majority are already planning to replace what they have. At 107 respondents in a single wave this is a directional read, skewed toward the mid-market — but the direction is clear: agent adoption is running ahead of agent security, and the controls that matter most when something fails — scoped identity and isolation — are the ones enterprises have built least. The agent security gap is not a coverage problem that a provider guardrail will close on its own; it is a problem of identity, isolation, and enforcement built for autonomous software. The open question for later waves is whether enterprises close it deliberately — or whether a confirmed incident closes it for them.

Based on survey responses from 107 qualified enterprise respondents (100+ employees), drawn from a single June 2026 wave. This is a directional read, not a precise measurement — the sample is self-selected and skews mid-market, so it's best read as the view from organizations actively standing up agent security rather than from the largest operators. Respondents are senior and buyer-credible (45% final decision-makers, 30% recommenders/influencers), spanning managers through the C-suite, and drawn primarily from Technology/Software, Manufacturing, Retail/E-commerce, and Healthcare/Life Sciences.

This wave of VentureBeat Pulse Research examines enterprise AI infrastructure and compute: where organizations are in their deployment journey, what they run AI on today, how satisfied they are, what would make them switch, where they plan to evaluate their investments, and — most revealingly — how well they can measure and control the economics of the compute underneath it all.

The central finding is a compute gap — the distance between how aggressively enterprises are investing in AI infrastructure and how little of its economics they can see. Only about one in five (21%) run AI in production at scale, yet spending intentions are outrunning that maturity: the single largest planned area enterprises plan to evaluate over the next year is AI-specialized clouds (45%), a layer almost none of these enterprises use today. Meanwhile the compute already in place runs cold — 83% report GPU utilization of 50% or less — and fewer than half (44%) can rigorously track what their AI compute costs. Enterprises are buying more infrastructure faster than they can account for what they already own.

Enterprises are not settled on their infrastructure vendors, either: A clear majority (64%) plan to switch or add an infrastructure provider within twelve months, and 38% within the next quarter — unusually high churn intent for a category this foundational. When they choose, they choose on integration with the existing stack (41%) and total cost of ownership (35%), not on headline price: cost per million tokens is the deciding factor for just 8%. And the frontier constraint that will shape the next round of decisions — the shift from GPU compute to memory bandwidth as inference scales — is barely on the radar, with roughly one in five enterprises either unaware of it or yet to address it.

Methodology

VentureBeat fielded this survey as part of its ongoing Pulse Research series, this survey focused on enterprise AI infrastructure, compute, and inference economics. Responses are filtered to organizations with more than 100 employees (n=107; the survey’s smallest size band, 1–100 employees, is excluded), drawn from a single Q2 2026 (June) wave. Because this is one wave rather than a pooled multi-month sample, the report reads cross-sectionally and does not infer month-over-month trends. Several questions were multiple-select, so those shares can sum to more than 100%.

By organization size the sample concentrates in the mid-market: 101–250 employees (36%) and 251–1,000 (27%) lead, with 1,001–5,000 (22%), 5,001–10,000 (8%), and 10,001+ (7%) above them. By role it spans managers (38%), individual contributors (28%), VPs and directors (19%), and the C-suite (13%); on purchasing authority it is buyer-credible, with 45% final decision-makers and another 30% recommenders or influencers for AI solutions. Technology/Software is the largest industry at 26%, followed by Healthcare/Life Sciences (15%), Financial Services (13%), and Retail/E-commerce (12%).

At 107 respondents the sample is large enough to read directionally but should be treated as a directional signal rather than a precise measurement; it is self-selected and is not a probability sample. It also skews toward the mid-market and toward earlier-stage adopters, so it is best read as the view from organizations actively building out AI infrastructure rather than from the largest hyperscale operators.

Finding 1: Ambition outpaces production

Only one in five run AI in production at scale

We asked where organizations sit in their AI deployment journey. Most are still building toward production rather than operating at scale.

The maturity curve is front-loaded. Three-quarters of enterprises (76%) are either experimenting or running only some workloads in production, and just 21% describe AI in production at scale. This matters for everything that follows: the infrastructure decisions in this report are being made largely by organizations still early in deployment, whose compute footprint — and whose costs — are about to grow. The evaluation and switching intentions in Findings 3 and 4 are the leading edge of that build-out, not the settled preferences of operators who have already found what works.

Finding 2: Enterprises run on hyperscalers and model APIs

The specialized GPU clouds barely register — today

We asked which providers and platforms enterprises currently use to run their AI. The answer is a familiar one: the incumbents.

The current stack is hyperscaler-and-API. Google Cloud leads at 48%, and the general-purpose clouds (Google, Microsoft, AWS, Oracle) together with the major model APIs (Gemini, OpenAI, Anthropic) account for essentially all current deployment. The specialized “neocloud” GPU providers that dominate AI-infrastructure headlines — CoreWeave, Lambda, Crusoe, Nebius and peers — register at or near zero among these enterprises today. Only 6% run their own on-prem GPU clusters and 4% a custom open-source stack. Enterprises are, for now, running AI on the providers they already buy from — which makes the evaluation intentions in Finding 3 all the more striking.

(A note on reading these shares. As described in the methodology section, this sample is self-selected and skews mid-market, and this question counted every provider a respondent uses — an average of 2.1 selections each — so the figures measure presence in the stack rather than spending or primary status. A sample built this way will show a different provider mix than a spend-weighted census of the broader market; Google's strength here, for example, is consistent with its long-standing position among smaller enterprises building on AI. Read these shares as a portrait of what this AI-active cohort runs today, and treat gaps between these figures and industry-wide market share estimates as a property of the sample rather than a contradiction of either.)

Finding 3: The next dollar goes to infrastructure they don’t yet run

AI-specialized clouds top the evaluations list

We asked where enterprises planned to evaluate AI infrastructure over the next 12 months. Their answers point away from the stack they run today.

Here is the report’s sharpest tension. The single most-cited planned evaluation area — AI-specialized clouds, at 45% — is the very category almost none of these enterprises use today (Finding 2). Nearly a third (32%) intend to evaluate non-Nvidia accelerators, and 28% in next-generation Nvidia silicon; even decentralized compute networks (16%) and sovereign compute (11%) draw meaningful interest. Read against current usage, this is not incremental — it is the leading edge of a re-platforming. The direction-of-travel question tells the same story: every infrastructure approach is net-expanding, but specialized AI clouds carry the highest net momentum (+24), edging out even the hyperscalers (+22). Enterprises are preparing to move a meaningful share of AI compute off the general-purpose cloud.

This continues a trend we saw in our April-May survey wave. Back then, usage of the AI-specialized clouds was equally marginal — CoreWeave at 3%, Lambda at 4%, Crusoe at 2% of enterprises. When we asked enterprises what change they planned in their AI infrastructure strategy over the next twelve months, the most-cited answer was moving workloads to specialized AI clouds, at 33%. Asked in April-May which emerging compute option they were most likely to evaluate AI-specialized clouds again drew the most responses. Two waves, two differently worded questions, one consistent picture: the type of cloud enterprises are most eager to assess is the type they have barely begun to use.

Finding 4: A switching wave is building

Six in 10 plan to change providers within a year — many within a quarter

We asked whether and when enterprises plan to switch or add an infrastructure provider. Very few intend to stand still.

For a category as foundational as compute, this is a remarkable amount of intended movement. Only 36% have no plans to change, meaning a clear majority (64%) intend to switch or add a provider within twelve months — and 38% within the next quarter alone. Where that interest points is telling: the providers drawing the most switching consideration are again the incumbents — Microsoft Azure and Google Cloud (33% each), OpenAI (30%), and Gemini (22%) — which suggests much of the near-term movement is reshuffling among the majors and consolidating spend rather than defecting to new entrants. The neocloud interest in Finding 3 is a 12-month evaluation thesis; the switching in the next quarter is mostly incumbents trading share.

(Method note: Respondents who selected both "no plans to change" and a specific switching window are counted as switchers, on the logic that naming a timeframe is the more specific answer; three respondents were reclassified under this rule.)

Finding 5: Nobody buys on token price

Integration and total cost of ownership decide — not sticker price

We asked what matters most when enterprises select an AI infrastructure provider. Headline price finished last.

Enterprises do not buy AI infrastructure on pricing, which is the place vendors compete on hardest. Integration with the existing stack (41%) and total cost of ownership (35%) dominate, while the headline metric — cost per million tokens — is the deciding factor for just 8%, dead last. The pattern is coherent: buyers are optimizing for how a provider fits and what it truly costs to operate, not for the advertised unit rate. It also foreshadows Finding 7 — enterprises say TCO matters most, yet most cannot yet measure it rigorously. The stated priority and the measured capability are out of step.

Finding 6: Expensive GPUs, idle most of the time

83% report GPU utilization of 50% or less

We asked what share of their GPU capacity enterprises actually utilize. The answer is a well-known but rarely quantified inefficiency.

Disclosure: Band percentages count every selection against all 107 qualified respondents; 14 respondents selected more than one band, so bands overlap. At the respondent level, 83 of the 100 GPU-operating enterprises reported utilization at or below 50%

The compute already in place runs cold. Adding the bands at or below half capacity, 83% of enterprises that operate GPUs report utilization of 50% or less, and nearly half (49%) run at 25% or below. Only 12% clear the 50% mark, and a further 8% do not measure utilization at all. Idle accelerators are expensive accelerators, and this is the clearest single measure of the compute gap: enterprises are planning to buy more GPUs and specialized compute (Finding 3) while the capacity they already own sits substantially unused. The efficiency headroom in the current fleet is large — and largely unmeasured.

Finding 7: Spending fast, measuring slowly

Fewer than half rigorously track what their compute costs

We asked whether enterprises can quantify the cost and return of their AI infrastructure spend, and how satisfied they are with what they run. Confidence in the ledger lags the spending.

Measurement trails money. Fewer than half of enterprises (44%) rigorously track the cost and return of their AI compute; the majority track only partially (39%), cannot quantify it yet (20%), or have not prioritized it (6%). That gap is consequential given Finding 5, where total cost of ownership was the second-ranked buying criterion — enterprises are choosing providers on an economic basis they mostly cannot yet measure. Satisfaction with current infrastructure is moderately positive but not enthusiastic: on a five-point scale, overall satisfaction averages 4.0, with ease of implementation (3.8) and value for money (3.9) trailing slightly — the softness landing, tellingly, on cost. Enterprises are spending quickly and accounting slowly.

Finding 8: The next bottleneck few are watching

As inference shifts from compute to memory, the field scatters

Finally, we asked how enterprises would address the emerging constraint in large-scale inference — the shift from GPU compute to memory, specifically KV-cache capacity. The responses reveal a frontier that is not yet a priority.

The memory frontier is real but barely governed. Asked which approach they would rely on as the binding constraint in inference shifts from compute to memory bandwidth, enterprises scatter: Dell leads at 31%, Nvidia follows at 16%, and the rest fragments across storage vendors, open-source tooling, and model-level efficiency techniques. Most telling is that roughly one in five (18%) either do not recognize the constraint or have not begun to address it. For a shift that will reshape inference cost and architecture, this is an early and unsettled market — and, consistent with the measurement gap in Finding 7, one where many enterprises simply do not yet have a view. It is the next chapter of the compute gap, arriving before most have closed the current one.

The bottom line: A compute gap that faster spending will widen, not close

Organizations with more than 100 employees are investing in AI infrastructure faster than they can measure it. Most are still early in deployment, yet their spending intentions point past their current stack — toward specialized clouds and alternative accelerators almost none of them run today — and a clear majority intend to change providers within the year. They buy on integration and total cost of ownership rather than headline price, which is rational; the difficulty is that most cannot yet see those economics clearly.

The visibility gap is concrete. The GPUs enterprises already own run at half utilization or less for the overwhelming majority, and fewer than half can rigorously track what their compute costs or returns. Satisfaction is decent but unenthusiastic, softest on value for money — the dimension hardest to judge without measurement. And the next constraint, the shift from compute to memory in large-scale inference, is arriving while most enterprises are still unaware of it. At 107 respondents in a single Q2 wave this is a directional read, skewed toward the mid-market and earlier-stage adopters — but the direction is consistent: the appetite to spend is running well ahead of the instrumentation to spend well. The compute gap is not a capacity problem that more hardware will solve on its own; it is, first, a problem of seeing what the hardware already costs. The open question for later waves is whether enterprises build that visibility before the re-platforming arrives — or buy the next layer of infrastructure as blind to its economics as the last.

Based on survey responses from 107 qualified enterprise respondents (100+ employees), drawn from a single Q2 2026 (June) wave. Because this is one wave rather than a pooled multi-month sample, the results read cross-sectionally rather than as a month-over-month trend, and at 107 respondents this is a directional signal rather than a precise measurement — the sample is self-selected, skews mid-market, and leans toward earlier-stage adopters rather than the largest hyperscale operators. Respondents include managers, individual contributors, VPs/directors, and the C-suite, with buyer-credible purchasing authority, across Technology/Software, Healthcare/Life Sciences, Financial Services, Retail/E-commerce, and other industries.

File access now requires a dedicated permission

Extensions that need to read file:// URLs used to get that access as part of the “Access your data for all websites” host permission. Starting in Firefox 153, file access is a separate, explicit permission, “Access local files on your computer”, shown in the extension’s permissions settings. It’s off by default for every extension, including ones already installed.

This change has a few concrete effects on code:

• Before: an extension with <all_urls> or a matching host permission could read file:// pages without any additional grant, and extension.isAllowedFileSchemeAccess() always returned false regardless of the permission setting.
• After: the extension must have the new file-access permission granted, and extension.isAllowedFileSchemeAccess() correctly reflects whether the user has granted it.

async function checkFileSchemeAccess() {
  const isAllowed = await browser.extension.isAllowedFileSchemeAccess();

  if (!isAllowed) {
    await browser.notifications.create("file-scheme-access-needed", {
      type: "basic",
      iconUrl: browser.runtime.getURL("icons/icon-48.png"),
      title: "Local file access required",
      message:
        'This extension needs "Allow access to file URLs" enabled to work ' +
        "with local files. Go to about:addons → select this extension → " +
        "turn on that setting, then reload the page.",
    });
    return false;
  }

  return true;
}

devtools.inspectedWindow.eval() calls targeting file:// URLs are affected the same way; they now require this permission to succeed.

If your extension depends on file:// access, expect existing users to see that access stops after upgrading (until they enable the permission), and consider adding a prompt or fallback path, for example by specifying an embedded options page (options_ui) and calling browser.runtime.openOptionsPage() to open about:addons and including instructions to toggle the setting in the “Permissions and data” tab.

userScripts.execute() and publicSuffix: covered in our next post

Firefox 153 adds two community-contributed APIs:

• userScripts.execute(), which provides for one-off injection of one or more user script sources into a tab or frame, in a defined order, as a complement to the persistent, URL-pattern-based userScripts.register().
• publicSuffix, which enables synchronous lookups against the browser’s built-in Public Suffix List using publicSuffix.isKnownSuffix(), publicSuffix.getKnownSuffix(), and publicSuffix.getDomain(). This API means that extensions no longer need to bundle or maintain a suffix list to determine a hostname’s registrable domain (eTLD+1).

Both APIs were built by contributors motivated by real needs in their extensions. We take an in-depth look at these contributions, their developers, impact, and history in a forthcoming post.

documentId support across more APIs

Firefox 153 introduces documentId, a stable identifier for a document instance, including a new runtime.getDocumentId() method, several webNavigation events and methods, webRequest events, scripting injection targets, and the extension messaging APIs.

Many WebExtension APIs use tabId and frameId to identify where to perform an operation. However, because frameId identifies the frame rather than its content, the loaded document can change and the extension’s subsequent operation ends up targeting the new (intended) document. documentId addresses this problem by providing a unique ID for the document. Now, if an extension uses the ID and the frame’s document has changed, the operation fails rather than silently targeting the wrong document.

See Work with documentId for the full list of supported events and methods, along with guidance on using it.

Content scripts can read and modify adopted stylesheets

Content scripts can now access document.adoptedStyleSheets and ShadowRoot.adoptedStyleSheets directly.

const sheet = new CSSStyleSheet();
sheet.replaceSync("* { background: pink; }");
document.adoptedStyleSheets = [sheet];

This enables extensions to inspect or modify constructed stylesheets from a content script, without using .wrappedJSObject, a workaround that risks interference from the web page.

Theme manifest key: gradients in additional backgrounds

The theme manifest key’s images.additional_backgrounds property now accepts CSS gradients alongside image URLs. A new properties.additional_backgrounds_size property controls the size of each additional background item.

Contextual identities (containers)

If your extension supports contextual identities, you now have access to two new methods: contextualIdentities.getSupportedColors() and contextualIdentities.getSupportedIcons(). These methods return the supported colors and icons, so your extension doesn’t need to hardcode either list.

Also, the colors have been updated to align with the new UI theme: “turquoise” is now “cyan”, “toolbar” is now “gray”, and “violet” has been added. The old names still work for backward compatibility, but your extension should switch to using getSupportedColors() rather than hardcoding either the old or new names.

Add a build-for-amo script

While this isn’t about new APIs, I wanted to mention a change that’s part of our work to make source code review faster and more reliable. When you submit an extension version, AMO now attempts to build your extensions from the submitted source code and compares the result to the package you uploaded. When the two match, reviewers don’t have to verify the build manually. This means submission can move through its review faster.

For now, this applies only if you submit source code that includes a package.json file to build your extension. If your extension has no build step, or you use a different build system, nothing changes. The AMO builder keeps its zero-config approach.

So, if your extension’s source code uses a package.json file, add an npm script named build-for-amo that runs the commands needed to build your extension for Firefox:

{
  "scripts": {
    "fx-build": "some commands to build your add-on for Firefox",
    "build-for-amo": "npm run fx-build"
  }
}

If you’ve a Firefox-specific build command, just point build-for-amo at it. When present, the builder invokes this script instead of guessing how to build your extension. And while you are at it, make sure all your dev dependencies are listed in the package.json file.

For more information, including documentation and Bugzilla links, see the Changes for add-on developers section of the Firefox 153 for developers release notes on MDN.

As always, file extension-related issues on Bugzilla under the WebExtensions product, cross-browser API proposals are discussed in the W3C WebExtensions Community Group, and questions are welcome on the Add-ons Discourse.

The post Firefox 153 WebExtensions API updates appeared first on Mozilla Add-ons Community Blog.

Black Hat USA 2026 isn't just bigger... It's built differently, and Black Hat is closing the gap between knowing and doing. 

This video walks through some of our new features, the team building them explain what's coming and why it matters:

Arsenal Labs is reunited with Arsenal. The Interface showcases CTF arenas, including scenarios across healthcare, retail, and more, VR breach investigations, and escape rooms. Our NOC Outpost allows you to step beyond the glass and work alongside the team protecting the event in real time. Bricks and Picks introduces real-world physical security scenarios. The Main Stage moves to the Business Hall floor. A First-Timers Program launches for anyone walking into their first Black Hat. Networking is easier throughout the new Cyber District.

Timestamps:
0:00 — Why 2026 is different: active learning, hands-on experiences
2:06 — Arsenal + Arsenal Labs: 115 sessions, Drone Zone, The Lab, The Interface
5:51 — Summit Leaders Lounge, Community Conversations, Picture Point
8:21 — NOC Outpost: step beyond the glass, see agentic AI in action
9:48 — Bricks and Picks: lock picking + Lego, physical security scenarios
12:19 — Main Stage + Startup Spotlight (now a global competition)
13:51 — Startup City + AI Zone content stages
16:09 — First-Timers Program (Tuesday 5-7pm)
16:54 — Cyber District: 13 sponsor venues across the week

Black Hat USA 2026
August 1–6, 2026 | Las Vegas
Register at blackhat.com
Full feature breakdown: Black Hat USA 2026 | Features

One Step Ahead.

 

Black Hat USA 2026
August 1–6, 2026 | Las Vegas
Register at blackhat.com
Full feature breakdown: Black Hat USA 2026 | Features

 

One Step Ahead.

According to OpenAI’s disclosure, the incident began during an internal evaluation of advanced cyber capabilities using GPT-5.6 Sol and a more capable pre-release model. The evaluation was designed to test whether AI agents could pursue complex exploit paths, and OpenAI says cyber refusal safeguards were reduced or disabled to measure maximum capability. Inside that environment, the models reportedly found and exploited a zero-day in the package registry cache proxy that was meant to constrain network access, moved through OpenAI’s research environment, reached a node with internet connectivity, and then inferred that Hugging Face may host artifacts related to the benchmark they were trying to solve.

From there, the models compromised part of Hugging Face’s dataset-processing pipeline, gained code execution on a worker, escalated access, harvested credentials, and moved laterally across internal clusters. Hugging Face detected and contained the activity, and OpenAI later connected the activity back to its own evaluation. Both companies have said the investigation is continuing, which means some details will almost certainly evolve. Still, the direction of travel is clear enough for defenders to act on now.

How did the OpenAI model evaluation reach Hugging Face?

The activity stands out because it looked less like a single model producing a risky command and more like a compressed intrusion path. Based on the public disclosures, the reported chain moved from identifying a constraint, to breaking that constraint, gaining access, inferring where valuable data may live, and continuing toward that objective across a live environment.

Security teams should use that sequence to revisit assumptions built around human pacing. Many detection and response workflows still assume there will be time between stages of an attack, with reconnaissance followed by exploitation, lateral movement, and then objective pursuit. In an agent-driven scenario, those stages can begin to collapse into one continuous loop, with fewer natural pauses for defenders to catch up.

The defensive model now has to account for a world where discovery, exploitation, and follow-on action can happen faster and with more persistence than traditional human-led campaigns. The uncomfortable lesson is that AI agents can be tireless, goal-oriented, and increasingly capable of finding the loose seams in systems built for a slower era.

The incident highlights the collapse of the traditional OODA (Observe-Orient-Decide-Act) loop. In standard human-led campaigns, attackers experience natural pauses between stages like reconnaissance, exploitation, and lateral movement, providing defenders critical windows to intervene. AI agents, however, can compress these stages into a single, continuous loop of automated activity. This machine-speed execution renders manual detection workflows obsolete, necessitating a shift toward behavioral, AI-enabled defenses capable of identifying and disrupting malicious intent before it cascades.

Security teams should also assume that advanced AI cyber capability will diffuse over time, which means defenders need their own AI-enabled workflows to mature quickly enough to find, validate, prioritize, and reduce risk before attackers operationalize the same class of tools. Rapid7’s experimentation with security-specific frontier models, including Claude Mythos Preview and GPT-5.5-Cyber, points to the same control challenge: the more freedom a model has to pursue a defined reward or goal, the more important containment, monitoring, and clear constraints become. Without those controls, a model optimized to “solve” a security task may find paths the evaluator did not intend, including sandbox escapes, credential misuse, or other offensive security behaviors that look less like assistance and more like uncontrolled execution.

Why AI evaluation environments need stronger containment

The incident raises practical questions about how frontier AI evaluations are designed and contained. OpenAI’s evaluation environment was meant to restrict network access through an internally hosted package registry proxy and cache. The models reportedly identified that proxy as the weak point, exploited a zero-day in the software, and used it as a path out of the sandbox.

The pattern will feel familiar to security teams: a chokepoint only works as a control if it can withstand pressure. Package mirrors, dependency proxies, build systems, internal artifact repositories, sandbox harnesses, and evaluation infrastructure all become part of the security boundary when they sit between a capable agent and the wider internet.

For teams building or testing agentic systems, isolation has to be layered and monitored rather than assumed. Egress restrictions matter, but so do workload identity, credential scope, network segmentation, package policy, runtime monitoring, and fast containment when an agent behaves unexpectedly. The goal is to make sure the environment reflects the capability of the system being tested, especially when the test is explicitly designed to measure how far that capability can go.

Why AI and ML pipelines are now software supply chain risk

The Hugging Face side of the incident is a reminder that AI and ML pipelines are part of the software supply chain. Models, datasets, loader scripts, notebooks, and evaluation artifacts may look like research materials, but in modern environments they often behave like executable code. Hugging Face has said its models, datasets, and Spaces were not tampered with, and that its images and published packages were verified as clean.

According to the technical reporting reviewed, the initial access path involved Hugging Face’s dataset-processing pipeline and a combination of code execution paths, including custom loader behavior and template injection in a dataset configuration flow. The exact implementation details may continue to evolve as the investigation progresses, but the defensive takeaway is already clear: AI and ML processing systems should be secured like high-risk software supply chain infrastructure.

Any system that automatically processes external datasets or model artifacts should be designed with hostile input in mind. Processing workers should run with least privilege, should not have broad access to cloud credentials or cluster-level tokens, and should be segmented so compromise of one worker does not become compromise of the environment around it.

Security teams should also hunt for early signs of intent drift inside ML workflows. Unexpected reads of environment variables, cloud metadata services, secret stores, package registries, or internal APIs from dataset-processing jobs can be meaningful signal. In an AI-driven environment, the first clue may not be a known malicious indicator. It may be a workload behaving with curiosity it should not have.

What AI guardrails mean for incident response

One of the most useful lessons for security teams came during the response, when Hugging Face’s responders reportedly needed to analyze logs containing exploit payloads, attacker commands, and command-and-control artifacts. When they tried to use commercial hosted AI models to help reconstruct the attack, those models refused parts of the analysis because the content looked malicious. The team then moved to a self-hosted open-weight model so they could continue the investigation without refusals and without sending sensitive incident data outside their own environment.

The practical issue here is guardrail asymmetry: attackers, rogue agents, or unrestricted systems may not respect usage policies, while defenders using hosted tools can be slowed by the same safety systems designed to prevent misuse. Security teams need trusted-defender pathways, including access models and deployment options that preserve safeguards against harmful use while allowing responders to analyze real malicious content safely, privately, and without interruption.

AI-assisted incident response also needs to be tested before an incident begins, especially if teams expect models to support triage, log analysis, malware review, or timeline reconstruction under pressure. Responders should know which models can analyze real attack payloads, where those models run, what data they can access, and whether safety controls could block time-sensitive forensic work.

What security teams should do about agentic AI risk

The practical response to this incident is preparation, with organizations treating AI agents as both powerful tools and potential attack surfaces when those agents can access code, credentials, infrastructure, datasets, or internal systems.

Security teams should apply familiar controls with more urgency across AI and ML environments: treat downloaded models, datasets, and loader scripts as untrusted code; enforce least privilege on AI and ML processing workers; prevent node-level compromise from becoming cluster-wide compromise; restrict access to cloud metadata endpoints and secrets; monitor unexpected outbound connections from AI-related workloads; and review whether evaluation environments have real containment or simply convenient isolation.

Detection logic needs to account for machine-speed activity. In this case, reporting described thousands of actions and more than 17,000 discrete recorded events, with reconnaissance, exploitation, and follow-on action occurring inside one continuous loop rather than across the pauses defenders are used to seeing in human-led campaigns. Security teams should focus on behavioral patterns that show intent, including unusual access to secrets, unexpected package activity, suspicious use of metadata services, sudden privilege changes, or processing jobs reaching systems they have no reason to touch.

As autonomous activity becomes faster and noisier, the bottleneck may shift from detecting that something happened to understanding what matters quickly enough to change the outcome. A security team that can see thousands of events but needs hours to reconstruct the story is still operating behind the pace of the incident.

How preemptive security helps reduce AI-driven risk

At Rapid7, our view is that this is where preemptive security becomes especially important. Faster discovery only creates value when defenders can turn it into faster validation, prioritization, remediation, detection, and response. The same principle applies to agentic AI risk. If AI accelerates how weaknesses are found and exploited, defenders need security operations that can act earlier with better context and more confidence.

That means connecting exposure management with detection and response, so teams understand which risks are exploitable, which assets matter most, what suspicious behavior is already present, and which actions will reduce risk fastest. It also means using AI carefully and practically, not as a replacement for security judgment, but as a way to reason across telemetry, reduce noise, support investigation, and help teams make decisions at the speed the threat environment now demands.

AI-enabled defense is becoming part of resilience planning, especially for organizations running critical systems or high-value digital infrastructure. The goal is to give defenders the speed, context, and consistency to operate inside the attacker’s decision cycle, without removing the judgment and accountability that effective security requires.

The OpenAI and Hugging Face incident will continue to generate debate as more details emerge, but defenders already have enough to work with. Agentic systems are beginning to test the seams between AI research, software supply chain security, cloud infrastructure, and incident response. The organizations best positioned for what comes next will be the ones making those seams visible, monitored, and resilient before the next incident puts them under pressure.

Read more →

The post Nuclear-Sabotage Malware Benchmark Trips Up Most Frontier AI Models appeared first on IT Security News.

The post Nuclear-Sabotage Malware Benchmark Trips Up Most Frontier AI Models appeared first on SecurityWeek.

Read more →

The post Google Released Gemini 3.5 Flash Cyber AI, a Specialized AI Model for Vulnerability Hunting appeared first on IT Security News.