Silent failures in LLM redaction pipelines pose serious PII data risks

A developer benchmarking six PII redaction approaches found that large language models can fail silently, returning plausible-looking responses that leave sensitive data fully intact. Unlike parsers or schema validators, a failed redaction pass produces no error signal, meaning unredacted records appear processed and flow downstream undetected. Running rule-based and model-based redaction passes independently over original text, then reconciling results, proved more reliable than chaining them sequentially, which caused offset corruption and malformed tags. HTML markup in enterprise documents was found to suppress model recall entirely in several test cases, with text-node extraction lifting recall from 0% to above 80%. The analysis also highlighted that model refusals and low-confidence false positives from tools like Presidio must be explicitly handled, as neither condition raises an exception on its own.