Late Chunking in Spring AI Preserves Document Context for Better RAG Retrieval

Traditional RAG pipelines often split documents into fixed token chunks before embedding, which destroys cross-paragraph context and degrades retrieval quality. Late chunking addresses this by running a long-context transformer over the entire document first, then pooling embeddings from contextualized token states at chunk boundaries. In Spring AI, this approach requires a long-context model such as jina-embeddings-v3 with an 8k token window, with mean-pooling applied per text span after a single forward pass. The method eliminates the need for costly LLM-based context summarization or large chunk overlaps used to compensate for lost semantics. Existing vector store schemas in PgVectorStore or QdrantVectorStore require no changes, making adoption relatively low-effort for teams already using Spring AI.