Freshness Windows and Index Costs for Internal Engineering Documentation Retrieval
A DEV Community article argues that engineering documentation retrieval should use freshness windows tied to document change rate and incident risk, so only changed pages go through incremental indexing. Its example is a property-management watcher that alerts engineers, where a stale internal runbook could make an accurate alert operationally dangerous. The author recommends defining per-source service-level targets for how quickly changes appear in search, using conditional HTTP requests, and tracking fetch, index, and answer freshness as separate timestamps. The piece also advises indexing by content revision with idempotency keys to prevent stale chunks and duplicate deliveries, and estimating embedding costs from changed tokens.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in