Most in-app onboarding tools tell users what to do. They pop a tooltip, queue a checklist, or open a chat box where an AI types a paragraph the user still has to translate into clicks. We wanted the opposite: a guide a user can talk to out loud, that answers in voice, and that moves the cursor to the exact button instead of describing where it is.
This is a write-up of the architecture decisions behind that guide, the tradeoffs we hit, and what we'd tell anyone building real-time voice features today.
Why voice-to-voice, not STT → LLM → TTS
The traditional voice pipeline is three hops: speech-to-text, then a chat completion, then text-to-speech. Each hop adds latency, and the seams show — the assistant feels like it's "thinking" because it is, serially.
We built on the OpenAI Realtime API, which collapses that into a single bidirectional stream: audio in, audio out, with the model reasoning in between. The practical win is latency. A user asking "where do I start?" gets a spoken answer that begins in well under a second, which is the difference between a guide that feels alive and one that feels like a kiosk.
If you're evaluating this path, the tradeoff is control: you give up the ability to inspect and rewrite the intermediate text on every turn. For a guidance product where speed and naturalness matter more than deterministic phrasing, that was an easy call. A longer, hands-on walkthrough of wiring up the Realtime API for voice + screen apps is here: — free to start.
Building in public — feedback and hard questions welcome.
SOCIAL SHARE CARD GENERATOR