How a calibration flaw taught a fraud detection AI to approve fraudulent transactions

A developer building an AI-powered fraud investigation system on TigerGraph discovered that a miscalibrated risk score threshold was causing the agent to incorrectly clear fraudulent transactions as legitimate. The system was designed as an agentic investigator using LangGraph and a TigerGraph graph database, where the AI gathers evidence iteratively before recommending an action and approval route. A key architectural decision kept all regulatory-critical logic — including the Bayesian ledger, policy engine, and approval routing — in a separate core module that is structurally prevented from importing any LLM library. To retrieve relevant past cases, the system fuses vector similarity search with graph traversal, addressing the shortcomings of using either method alone. The project highlights how measurement and calibration errors in underlying data can silently undermine even well-architected AI decision systems.