Tracking Document Lifecycle to Control News Retrieval Index Costs

For an e-commerce news monitoring service, the article argues that retrieval observability should follow each document through ingestion, chunking, indexing, querying, and answer grounding. Index cost is driven mainly by retained chunks, so teams should measure document arrival rate, chunks per document, and retention period alongside relevance and latency. Duplicate syndicated stories and chunking changes that quietly multiply chunk counts are named as common causes of unexpected growth. The piece recommends tracking document versions with fingerprints, parser and chunker versions, and retention classes, and separating source categories in telemetry. It concludes that measuring these ingestion-side signals should come before changing the retrieval algorithm.