LLMs Belong Inside Recommendation Stacks, Not on Top of Them

Recent 2025-2026 research papers, including RecoChain and RRCM, argue that large language models should function within specific layers of a recommendation system rather than replacing the entire pipeline. A standard production recommender operates across four layers: data and features, candidate retrieval, ranking, and feedback, with LLMs contributing only to retrieval and semantic feature extraction. Strict latency requirements, typically around 100 milliseconds end-to-end, make it technically infeasible to place a generative model call in the critical request path for large catalogs. Engineers are advised to precompute or cache LLM outputs, such as item embeddings and query understanding, and always maintain a deterministic fallback when model calls fail. A commonly overlooked data practice is logging impressions alongside clicks, which is essential for computing click-through rates, correcting position bias, and enabling offline ranker evaluation.