The Silent Vector Contamination Bug: Why Your Concurrent Embeddings Might Be Lying to You

TL;DR: If you run concurrent inference (e.g., via OpenVINO AsyncInferQueue or custom threading) for text/code embeddings, your tests might show 0 exceptions and 0 errors, while silently returning embeddings belonging to other inputs in the batch. Here is how we caught a subtle race condition using a cosine-similarity contamination test.

The Setup

We use OpenVINO with an INT8 quantized — an MCP server for codebase intelligence with incident memory and root cause prediction.