Teaching a RAG App to Admit When Its Documents Lack an Answer

A developer starting a 90-day AI challenge is tackling a basic reliability problem in retrieval-augmented generation: getting the app to decline questions its sources cannot support. Retrieval tends to return the closest-matching passage even when that passage doesn't contain the needed information, so a language model can produce a confident but unsupported answer. The first version is deliberately small, covering document chunking, passage retrieval, answering only from retrieved text, requiring citations, and returning a clear fallback when evidence is missing. Test questions fall into three groups: directly answerable, related but unsupported, and unrelated, with the middle group considered the most revealing. The author stresses that a valid citation ID alone does not prove correctness, and that any retrieval-score cutoff must be tested on the actual documents.