Race Condition in Concurrent AI Embeddings Can Silently Corrupt Vector Stores
A subtle race condition was discovered in concurrent text embedding pipelines using OpenVINO's AsyncInferQueue with an INT8 quantized E5-small model. The bug caused output tensors from one inference request to be mapped to the metadata of a different request, meaning vectors silently represented the wrong input text. Standard unit tests passed without errors since returned vectors were valid, non-zero, and correctly shaped, masking the corruption entirely. The flaw surfaced during end-to-end RAG retrieval tests, where semantically unrelated code chunks were returned with high confidence for unrelated queries. The root cause was a shared integer counter used as userdata keys across concurrent calls, which reset and overlapped between simultaneous batch operations; the fix required strict per-request isolation of the shared results dictionary.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in