RAG Systems Struggle to Accurately Read Tables, Footnotes in Documents

Retrieval Augmented Generation (RAG), a widely used method for querying long documents, performs poorly when those documents contain tables, according to emerging research. A 2026 benchmark called REAL-MM-RAG, designed around table-heavy documents, found that current retrieval models consistently fail to handle tabular data even when surrounding prose is processed correctly. A core issue is how RAG systems split documents into chunks — a table spanning multiple pages can be cut mid-way, separating header rows from their data. Footnotes and explanatory text placed away from a table are often retrieved independently, causing models to misrepresent figures that depend on that missing context. These limitations are especially acute in on-demand systems that process documents at query time, leaving no opportunity to pre-structure tables and cross-references in advance.