RAG Systems Can Cite Real Sources Yet Deliver Wrong Medical Answers, Study Finds

A new research paper on clinical retrieval-augmented generation (RAG) has identified a critical failure mode called 'deceptive grounding,' where a system retrieves evidence about one drug or entity but presents it as applying to another. The flaw is particularly dangerous because standard faithfulness and hallucination checks do not flag it — citations appear valid and no content is fabricated. Tested across 13 models under adversarial conditions, deceptive grounding rates ranged from 8% to 87%, with medically fine-tuned models reaching 86.7%. In a production setting, the rate was 7.8% across 740 drug-disease pairs, rising to 13.6% for recently approved drugs with sparse evidence. The authors propose an entity-attribution verification step — checking whether a cited source actually concerns the entity the answer references — which achieved 97% precision and 98.7% recall against a human-labeled benchmark.