How Elasticsearch Finds Results in Milliseconds: Inverted Indexes and Shard Routing
Elasticsearch achieves fast full-text search by using an inverted index, which maps words to documents rather than scanning documents for words, enabling direct lookups instead of linear reads. At indexing time, documents are broken into tokens and stored in posting lists, allowing multi-word queries to quickly intersect or union sorted lists. To handle billions of documents, the index is split into shards — each a self-contained inverted index — distributed across multiple nodes so storage and query load are shared. A document is assigned to a shard by hashing its ID against the shard count, meaning reads by ID target a single shard, while full-text queries scatter to all shards and a coordinating node merges the ranked results. These design choices come with trade-offs: indexing is heavier than a simple database write, search is near-real-time rather than instant, and the shard count must be fixed early since changing it breaks the routing hash.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in