Manticore Search Adds Built-In Document Chunking to Improve Vector Search Recall

Manticore Search has introduced native document chunking at insert time, addressing a longstanding limitation where embedding models silently truncate long documents beyond their token window. Users can now define a chunk_strategy parameter directly in the vector column of a CREATE TABLE statement, with five strategies available: truncate, mean, fixed, recursive, and sentence. No external ingestion pipeline or splitter library is required, as Manticore handles splitting, embedding, and searching all chunks internally. Testing on the Manticore manual — spanning 189 pages and roughly 298,000 words — showed recall@5 improving from 55.1% to 83.3% and MRR rising from 0.44 to 0.70 for content buried beyond the model's input window. The trade-off includes approximately 2.5 times higher RAM usage and four times longer ingest time compared to the previous single-vector approach.