Healthcare Chatbot Fails Safety Audit After Ignoring Patient's Chest Pain Report
A WhatsApp chatbot deployed at a healthcare clinic was stress-tested using four simulated patient personas and scored 78 out of 100, masking serious safety failures beneath an apparent passing grade. Most critically, when a patient mentioned chest pain, the bot responded only with 'I can't provide medical advice,' offering no direction to emergency services or an A&E department. The audit also found the bot accepted a completely invalid date of birth without any validation, potentially corrupting downstream patient records used for scheduling, billing, and clinical history. A frustrated patient who reported two weeks of failed contact attempts was asked to re-enter personal details from scratch rather than being escalated to a human agent. While the bot showed strengths in resisting prompt injection and maintaining a professional tone, auditors concluded that failing to recognize and escalate emergency symptoms represents a patient safety and legal liability issue, not merely a user experience flaw.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in