rag-critic: Open-Source Tool Flags Ungrounded Claims in LangGraph RAG Pipelines
The developer released rag-critic on PyPI, a lightweight auditor that checks LLM-generated drafts against retrieved source chunks to catch hallucinations in retrieval-augmented generation systems. The author argues that silent, unsupported claims are a bigger production risk than retrieval latency, and that holistic LLM-as-a-judge scoring is too coarse and slow. Unlike paragraph-level evaluation, rag-critic assesses individual claims, labeling each as supported or ungrounded and producing a grounding score and hallucination index. It is designed to plug into LangGraph as a checkpoint node, with conditional routing that approves grounded output, requests a rewrite for minor gaps, or triggers re-retrieval for heavy hallucination. The project's code and installation instructions are available on GitHub and PyPI.
