California bill targets AI therapy chatbots as 1.2M share suicide thoughts

California lawmakers are advancing a bill that would bar AI chatbots from advertising as therapy, as roughly 1.2 million users a week share suicidal thoughts on OpenAI's ChatGPT and wrongful-death lawsuits target chatbot makers.

"We're dealing with a new impactful technology that is unfolding and is deployed in our world at light speed, and so it is both extremely powerful and consequential," State Sen. Steve Padilla, the San Diego Democrat who authored Senate Bill 903, said.

The bill would ban companies from advertising chatbots as therapy, require a licensed professional to review AI therapeutic decisions, and mandate disclosure before AI tools record therapy sessions or triage mental healthcare. It is backed by the American Psychological Assn., the National Union of Healthcare Workers, and the California Behavioral Health Assn. Illinois last year became the first state to prohibit AI in therapy services, and several other states have since followed with similar restrictions.

The regulatory push targets a fast-growing use case. Companionship and "therapy" have been the top chatbot use over the past two years, according to research cited in the legislative debate, with an estimated one in eight teens and young adults turning to AI for mental health support. Companies from startups to telehealth giants are building purpose-built tools — Slingshot AI's Ash and Talkspace's Tee — while general-purpose platforms like ChatGPT and Character.ai see millions of daily mental-health conversations.

The therapy gap AI is filling

The appeal is straightforward: AI chatbots are free or low-cost, available 24/7, and carry no insurance paperwork or waitlists. That matters in a country where every county in California faces a shortage of behavioral healthcare workers, according to TechNet, the tech industry group opposing the bill.

But mental-health professionals warn the technology is not equipped for crisis management. Trained clinicians pick up on tone, eye contact, and body language, and can detect conditions AI misses. OpenAI disclosed last fall that about 1.2 million users a week share thoughts of suicide on its platform; the company says it recommends professional help and directs users to the 988 hotline. Families who filed wrongful-death suits against OpenAI and Alphabet claim chatbots assisted their loved ones through suicide plans.

"The difference between a licensed clinician and an automated response is not technical. It can be life altering," Le Ondra Clark Harvey, chief executive officer at the California Behavioral Health Assn., told lawmakers.

Industry pushback and the Kaiser dispute

TechNet argues SB 903 would restrict beneficial AI use in healthcare and slow access to care. "At a time when every county in California faces a shortage of behavioral healthcare workers, SB 903 still puts a clinician bottleneck in front of the intake and screening tools that help patients reach care faster," Robert Boykin, TechNet's executive director for California, said in a statement.

The bill's scope is also contested. The National Union of Healthcare Workers filed a complaint with state regulators against Kaiser Permanente, alleging its e-visit screening tool automatically generates care recommendations without licensed professional review. Kaiser says the tool "does not use AI to diagnose patients, make clinical decisions, or determine medical necessity." Whether SB 903 would apply to Kaiser's tool remains unclear.

For AI companies, the stakes are financial as well as legal. Mental-health chatbots represent a growing revenue stream for platforms like OpenAI, which has been expanding consumer subscriptions, and for dedicated telehealth providers like Talkspace. The bill awaits a vote in the Assembly's fiscal committee, with the WSJ's three-part podcast series "The AI Therapist" airing Sundays through Aug. 23 examining the industry's promise and risks. If California and other states impose clinician-review requirements, the cost structure of AI mental-health products could shift, potentially compressing margins for companies built around automated, low-touch care.

This article is for informational purposes only and does not constitute investment advice.