I measured every millisecond of my real-time AI pipeline. The LLM was the fast part.

I'm building LiveSuggest, a real-time meeting assistant. It listens to your call and shows you written suggestions while you're still talking: a clarifying question to ask, a point you forgot to make, the definition of a term someone just dropped. No bot joins the call. Everything runs from your own browser.

The whole product lives or dies on one number: how long between someone saying something and a useful suggestion appearing on screen. A suggestion that arrives after the conversation has moved on isn't just useless, it's a distraction.

So I assumed the language model would be my enemy. LLMs are the slow, expensive part, right? I instrumented the entire pipeline in production, expecting to spend my life fighting token latency.

I was wrong about where the time goes. Here's what a week of production metrics actually showed me.

The pipeline

Before the numbers, the shape. A suggestion goes through five stages:

1. 
   Capture. The browser grabs audio in 100ms chunks and streams them over a WebSocket. (I wrote about the capture side in , a real-time meeting assistant that gives you suggestions during a call without a bot joining it. The audio-capture side is here if you want the earlier chapter.
   Vollständiger Original-Bericht

   Ausführliche Details, Code-Beispiele & Hersteller-Stellungnahme auf dev.to.

   ↗ Original-Artikel auf dev.to lesen