The Unregulated Therapist

Why Millions Turn to AI Chatbots Despite Serious Safety Risks

A recent JAMA article, "The Chatbot Will See You Now", describes how generative AI tools such as ChatGPT and dedicated "therapy bots" have quietly become de facto mental health providers for millions of people in the US. Many turn to them because they cannot find an affordable, available human therapist, or value the 24/7, low-friction access and lack of perceived judgement.

Surveys cited in the article show that a substantial share of adults with ongoing mental health conditions, as well as millions of young people, now use large language models for psychological support. Many even report that these tools improve their mental health and sometimes rate them as more helpful than human therapists.

However, the article highlights a growing list of risks. Generative AI systems are inherently sycophantic—more eager to agree and reassure than to challenge unhelpful thinking—undermining the core of effective psychotherapy. Some products use human-like avatars or falsely claim professional credentials, blurring the line between support tool and licensed clinician. Most are not covered by HIPAA, avoid FDA scrutiny by branding themselves as "wellness" tools, and operate with minimal evidence of safety or effectiveness. Tragic wrongful death lawsuits, including cases of suicide, underscore the potential harms.

The article concludes that ignoring this reality is no longer an option. Clinicians should routinely ask patients about their use of chatbots, and regulators, researchers, and policymakers must move quickly towards more rigorous, transparent and evidence-based oversight of AI in mental health.

We should be alarmed by this readily available usage of non-evidence based "therapeutic" options improving access to validated health services as well as holding liable manufacturers of these programs.

Source: JAMA, January, 2026. doi:10.1001/jama.2025.23965