Why Realtime Is the Future of Speech-to-Text

For most of the last decade, transcription was something you did after the fact.

You recorded a call, dumped the file into a queue, waited a few minutes (or a few hours), and got back a block of text. That was the deal. Batch, async, post-hoc—whatever you want to call it, the audio was already over by the time the model saw it. And for a long time, that was fine, because that's all the technology could reliably do.

Recently, the field has quietly crossed the line where the most interesting, highest-value speech-to-text work happens while people are still talking. Realtime isn't a niche feature you bolt on for a captions demo. It's becoming the default that everything else gets measured against.

Here's the argument.

The shift: the interesting STT workloads are now live

Look at where the energy is going in Voice AI right now.

that flags a compliance risk or a churn signal before the call ends, not in a report the next morning.

Every one of those is realtime-first. Not realtime-optional—realtime-first. The value evaporates if you make the user wait.

Compare that to the classic batch workloads: transcribing a podcast back catalog, indexing a media archive, generating subtitles for content that already exists. Those are real, they matter, and they're not going away. But notice the pattern—they're all about audio that's already done. The frontier, the stuff people are building companies around in 2026, is about audio that's happening.

So the question flips. It used to be "when do I actually need realtime?" Now it's closer to "why am I making this a two-step process at all?"

What changed: latency and accuracy crossed the usability line at the same time

For years you could have fast transcription or accurate transcription in the streaming setting, but picking both felt like a trap. Realtime models were noticeably worse than their batch cousins. So teams defaulted to batch when quality mattered and only reached for streaming when they absolutely had to.

That tradeoff is mostly gone. And it went away because two curves crossed the usability threshold at roughly the same moment.

The first is latency. , and it's a good illustration of the general principle: realtime often means being provisionally right now and definitively right slightly later.

None of this is a reason to avoid realtime. It's a reason to take it seriously as its own category—and to be skeptical of any provider that treats streaming as an afterthought to their batch product.

The business case: latency is starting to look like a revenue metric

Here's where it gets interesting for the people who sign the invoices.

We've spent years talking about transcription accuracy as the thing that drives business outcomes, and it does—we've written a whole piece on as a way to judge a model—it says nothing about whether the system feels alive.

Where batch still absolutely makes sense

I'd lose credibility fast if I told you batch is dead. It isn't, and I'd steer you toward it in plenty of cases.

If the audio is already recorded and nobody's waiting on the result, batch is the right tool. Transcribing an archive of ten thousand old calls. Subtitling a media library. Running analytics over a quarter of sales conversations overnight. There's no live interaction to preserve, so paying the complexity tax of streaming buys you nothing.

Cost is real, too. Batch .

Ask how turn detection works. Silence-timer only? Or does it read tonality and pacing? This single choice determines whether your product feels human.

Ask whether the model can take live context. Can you feed it the agent's current question or a set of key terms mid-stream? agent_context and keyterm prompting are the difference between a model that guesses at "my ID is A-as-in-apple, 4, 4, 9" and one that gets it right.

Ask about diarization and whether it revises. Live speaker labels that never get corrected will drift. The revise-at-end-of-turn pattern is what keeps them honest.

And ask about the production stuff—concurrency limits, EU data residency, a signable BAA for healthcare workloads. Universal-3.5 Pro Realtime runs with unlimited concurrency and no rate limits, which matters the day your traffic spikes and you don't want to file a ticket to serve it.

For the full picture of how we think about the live path, the digs into the numbers.

The close: the transcript stops being the product

Here's the forward-looking part, and it's the thing I've been circling the whole time.

As realtime becomes the default, the transcript itself stops being the deliverable. For decades the output of speech-to-text was a document—text you'd read, search, or file. But a realtime transcript isn't something a human reads. It's something another system consumes, instantly, to make a decision: route the call, surface the answer, flag the risk, generate the next turn.

That changes what "good" means. The winning realtime models won't just be the ones with the lowest word error rate. They'll be the ones that emit the most machine-actionable stream—context-aware partials, confident finals, live speaker turns, end-of-turn signals—in a shape the next system can act on without waiting.

In other words, speech-to-text is quietly turning from a transcription tool into a real-time interface between human speech and software. The transcript is becoming plumbing. And once you see it that way, batch-versus-realtime stops being a feature comparison and starts looking like the difference between reading history and participating in the present.

That's the future I'd bet on. The rest of the field is already building for it.

Frequently asked questions

Is realtime speech-to-text replacing batch transcription?

No—and anyone who says so is oversimplifying. Realtime is becoming the default for interactive workloads like voice agents, agent assist, live captions, and ambient scribes, where making a user wait kills the value. Batch still wins for already-recorded audio: archives, bulk media, and overnight analytics, where it's simpler and cheaper. Batch async runs $0.21/hr with Universal-3.5 Pro; streaming runs $0.45/hr base. Think two tools with clearer boundaries, not a replacement.

What's the difference between realtime and streaming speech-to-text?

They're the same thing. "Streaming" describes the mechanism—audio flows over a WebSocket connection and the model returns partial and final transcripts as it listens. "Realtime" describes the outcome—results arrive fast enough to act on while the conversation is still happening. AssemblyAI's streaming API uses wss://streaming.assemblyai.com/v3/ws.

How low does latency need to be for a good voice experience?

For a natural back-and-forth, you want transcripts arriving in a few hundred milliseconds and turn detection firing quickly enough that the system doesn't talk over people or leave dead air. Universal-3.5 Pro Realtime returns partials in a few hundred ms, and its end-of-turn detection reads tonality and pacing to fire in about 300ms rather than waiting on a silence timer. End to end, a well-built voice agent lands around one second.

Do you sacrifice accuracy by going realtime?

Much less than you used to. On Pipecat's open STT benchmark of real agent conversations, Universal-3.5 Pro Realtime posted a 6.99% pooled word error rate—ahead of Deepgram Flux (15.58%), ElevenLabs Scribe v2 (9.76%), and Google Chirp3 (9.04%). Passing agent context cut WER by a further 10.2% across 20,000 files. Batch can still edge ahead on the hardest audio because it sees the whole file, but the gap has narrowed dramatically.

What are the main realtime speech-to-text use cases?

Voice agents, live agent assist for contact center reps, real-time has the details.