Why Explainable AI Still Cannot Translate Its Reasoning Into Plain Human Terms

Explainable AI (XAI) systems can compute predictions with high accuracy, but consistently fail to communicate the reasoning behind those predictions in ways that are meaningful to clinicians, regulators, or affected individuals. Tools like SHAP, LIME, and counterfactual explanations provide technical attribution of model outputs, yet none fully answers why a specific decision should be trusted in a high-stakes context. A model can be statistically accurate while relying on spurious correlations or proxy variables for protected attributes, making raw accuracy metrics insufficient for real-world accountability. The European Data Protection Supervisor's 2023 TechDispatch explicitly warned that 'black box' AI decision-making is unacceptable, as opacity can conceal bias, errors, and hallucinations. Bridging the gap between mathematical explainability and human-understandable reasoning remains one of the field's most pressing and unresolved challenges.