Reranker Lowers Memory Search Accuracy on Most Question Types in Retest
A retest of a personal memory search system found that an added cross-encoder reranker often reduced result quality. Across 184 questions on a roughly 112,700-passage index, reranking hurt opening-paragraph and linked-note retrieval, while note-title results showed no reliable change. The author notes that the index and pipeline had changed since an earlier test, so the comparison is within the current setup rather than a controlled rerun. Confidence intervals for the body and linked-note groups excluded zero, while the title group's interval crossed it, and the temporal group was too small to draw conclusions. The default configuration was kept and no new search change was deployed.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.


Discussion (0)
Log in to join the discussion and vote.
Log in