Experts warn AI mental health chatbots need clear limits and human escalation paths

Woebot, a well-known therapy chatbot, shut its consumer app in 2025, with its founder citing the cost of meeting FDA authorization requirements amid unclear rules for large language models. In November 2025, the FDA's Digital Health Advisory Committee discussed generative AI therapy chatbots and raised concerns about hallucinations, missed deterioration in users, uneven performance across groups, and privacy weaknesses. Experts also emphasized the need for predefined escalation plans and one-tap access to urgent human support. The author, an engineer rather than a clinician, describes applying these concerns to their own product by using rule-based emergency detection, avoiding diagnosis, and directing users toward real people. The author acknowledges that the approach has not been clinician-reviewed and that the problem of AI sycophancy remains unresolved.