Most RAG tutorials open with "set your OPENAI_API_KEY." This one doesn't need it. In won't save you here — that fallback calls ainvoke when no tokens stream, but here ainvoke is the empty result. The real mitigations are a larger/better tool-tuned local model, or accepting some flakiness as the price of fully offline. Worth knowing before you demo it live.
2. Embedded Qdrant locks the directory
Embedded mode keeps the store in one process. Run the ingest script while the server is up and you'll get a lock error. Order matters: ingest first → let it exit → then start the server. The ingest script even closes the client explicitly to avoid a noisy shutdown traceback.
3. Embedding dimensions must match end to end
bge-m3 is 1024-dim; OpenAI's text-embedding-3-small is 1536. If you ingest with one provider and query with another, the dimensions don't line up and search breaks. Switching embedding_provider means re-ingesting (--reset). The embed_query("probe") dimension check is exactly what keeps the collection honest per provider.
4. The first call is slow
Ollama loads the model into memory on first use. The first request eats that cost; subsequent ones are fast. Don't benchmark the cold start.
Why this matters
You can build, debug, and demo the entire RAG agent — graph, retrieval, citations — on a plane with no wifi. Then, for production, you flip two config values (CHAT_PROVIDER, QDRANT_URL) and the same code talks to a hosted model and a real Qdrant cluster. Part 1 claimed the provider boundary; Part 3 ran on both sides of it.
The flip side is honesty about local models: retrieval is rock-solid, but a 9B model's synthesis step is the weak link, and it'll occasionally hand you an empty answer. Know that going in.
Next: persisting conversation threads with a checkpointer — so the agent remembers across requests — and what that adds to the message log you just saw.
Part 3 of a series on running LangGraph in production. .
SOCIAL SHARE CARD GENERATOR