Generative-AI therapy chatbots (Therabot; experimental)

1. Introduction

Version followed: Therabot as Dartmouth's AI and Mental Health Lab describes it and as tested in the Heinz et al. randomised trial (NEJM AI 2025;2(4), DOI 10.1056/AIoa2400802). Trial details rest on the CTBH research brief and a detailed Mental Elf summary. Professional guidance follows the full text of the APA Health Advisory on the Use of Generative AI Chatbots and Wellness Applications for Mental Health (November 2025). Regulatory and legal status is as of October 2026 per a Wilson Sonsini review (22 September 2026), the MHRA Digital Mental Health newsletter (June 2026) and a DORDA note on EU law (27 February 2026). Status changes quickly, so check it again before relying on it. Most of the psychotherapist's procedures below are [synthesis] built on these sources.

What it is. Generative-AI therapy chatbots are conversational programs built on large language models. They write new, open-ended replies rather than choosing from pre-written scripts. Rule-based bots such as Woebot and Wysa are covered in 096. General-purpose assistants (ChatGPT-type tools) used for emotional support are covered in 098. This tab covers (a) research-grade systems built to deliver psychotherapy content, with Therabot as the best-documented example, and (b) companion and "wellbeing" chatbots that users treat as therapists. Therabot has been in development at Dartmouth since 2019, with input from psychologists and psychiatrists. Its training material was built from evidence-based psychotherapy and CBT practice [source: Dartmouth release via ScienceDaily, March 2025].

How change is supposed to happen. The developers' model is that CBT-type techniques, delivered in natural conversation and available at any hour, can reduce symptoms. In the trial, users rated the working alliance as similar to outpatient psychotherapy (mean WAI 3.59) [source: CTBH summary of Heinz et al. 2025]. The developers say it is not autonomous. The lead author states that "no generative AI agent is ready to operate fully autonomously in mental health". The senior author says there is "no replacement for in-person care" [source: ScienceDaily].

Status in one line. It is experimental. As of November 2025, FDA had authorised no AI-enabled device for mental health and no generative-AI device for any clinical purpose [source: Psychiatric Times, 13 Nov 2025; Hogan Lovells]. In 2026, FDA revised its General Wellness and Clinical Decision Support guidances (January 2026) and published a discussion paper on generative-AI-enabled devices (August 2026). A September 2026 legal review reports no authorisation of a generative-AI mental-health device [source: Wilson Sonsini, 22 Sept 2026; uncertain: no FDA database search was made]. APA advises against relying on chatbots or wellness apps to deliver psychotherapy [source: APA Health Advisory, Nov 2025]. A chatbot is never a replacement for psychotherapy. The psychotherapist's job is to ask about use, assess dependence, crisis risk and privacy, and then integrate the chatbot or set limits.

2. Framework

2a. How the research system was delivered (descriptive only)

Who: adults recruited nationally through online advertising and screened into three groups: major depressive disorder, generalised anxiety disorder, or clinically high risk for feeding and eating disorders. The trial had 210 participants: 106 used Therabot and 104 were on a waitlist [source: CTBH; ScienceDaily]. About 27% of participants were also receiving other treatment, such as medication or psychotherapy [source: Mental Elf, July 2025]. Detailed entry and exclusion criteria, including how acute suicidality was handled at entry, are [uncertain].
What it is built on: a generative large language model fine-tuned on expert-curated mental-health dialogues based on third-wave CBT, developed over more than 100,000 human hours. Each reply is generated from the user's latest message and the conversation history. Users could start a chat themselves or reply to a notification [source: Mental Elf].
Format: smartphone text conversation. There were 4 weeks of daily prompts to use it, then 4 weeks of continued access without prompts, so 8 weeks in total. Use averaged about 260 messages over about 24 days, and about 6 hours, in the first 4 weeks [source: CTBH; Mental Elf].
Safety monitoring: trained staff reviewed Therabot's replies after they were sent. If a reply was inappropriate, staff contacted the participant to correct it. Staff intervened 15 times for safety concerns such as suicidal ideation and 13 times to correct inappropriate replies such as medical advice. High-risk content triggered an on-screen prompt to call emergency services or a crisis line [source: CTBH; ScienceDaily; Mental Elf]. A trade summary states that a human reviewed every interaction [source: HMP Global summary of AP reporting].
Measures and reported results: the primary outcomes were change on the PHQ-9 (depression), the GAD-Q-IV (anxiety) and the Weight Concerns Scale (eating-disorder risk) at 4 and 8 weeks. Secondary outcomes were working alliance, satisfaction and engagement [source: Mental Elf]. Symptoms fell more than in the waitlist group at both points in all three groups. Average reductions were about 51% for depression, 31% for anxiety and 19% for body-image and weight concerns [source: ScienceDaily]. This is a single trial against a waitlist, with no active comparator, no blinding and short follow-up. The authors call for active-control trials and comparison with in-person therapy [source: CTBH; Mental Elf].

2b. Regulatory and professional status (checked October 2026; each item dated)

FDA (US), November 2025: the Digital Health Advisory Committee met on 6 November 2025 to discuss generative-AI digital mental-health devices. More than 1,200 AI devices had been authorised, none for mental health, and fewer than 20 non-AI digital mental-health devices existed [source: Psychiatric Times]. Risks discussed:
- confabulation or inappropriate or biased content;
- missed crisis cues, including failure to detect suicidal or homicidal ideation;
- medical conditions that mimic psychiatric symptoms;
- unhealthy parasocial relationships and over-reliance;
- deterioration without escalation;
- risks of long-term use [source: Psychiatric Times; Hogan Lovells].
FDA was most comfortable with a "human in the loop". Low-risk wellness or coaching tools may fall under enforcement discretion, while autonomous therapy or diagnosis falls within FDA's scope [source: Hogan Lovells].
FDA (US), 2026: a revised General Wellness guidance (6 January 2026) describes enforcement discretion for low-risk wellness products, such as stress-management or relaxation claims. Disease-related claims can bring a product into device review. A revised Clinical Decision Support guidance (29 January 2026) says functions that give recommendations directly to patients or caregivers do not qualify for the CDS exclusion. An August 2026 discussion paper, Considerations for the Regulation of Generative AI-Enabled Medical Devices, asks for input on how to evaluate these devices, including whether more reliance on postmarket monitoring could address uncertainty at authorisation. As of October 2026 per Wilson Sonsini (22 Sept 2026), this is a discussion paper, not a final rule.
US states, 2025 laws:
- Illinois, HB 1806 (Wellness and Oversight for Psychological Resources Act, Public Act 104-0054, signed August 2025; sources give 1 or 4 August). It bars offering AI-delivered therapy. In therapy, AI may not make independent therapeutic decisions, communicate therapeutically with clients directly, produce treatment plans without licensed review, or detect emotions or mental states. Administrative and supplementary uses are allowed, and recording or transcription needs written informed consent. IDFPR enforces the law, with civil fines up to $10,000. Peer support, religious counselling and self-help materials that do not claim to be therapy are exempt [source: HeplerBroom; Wilson Sonsini].
- Nevada, AB 406 (signed June 2025): bars offering AI designed to provide professional mental or behavioural healthcare, or claiming that AI can provide it [source: Wilson Sonsini].
- Utah, HB 452 (signed March 2025): mental-health chatbots must disclose that they are AI and not human, including again after any seven-day lapse in use [source: Wilson Sonsini].
- New York companion-AI law (in effect 5 November 2025): operators must detect and respond to suicidal ideation and self-harm, and disclose AI status at the start of an interaction and at least every three hours [source: Wilson Sonsini].
- California requires disclosure when generative AI writes clinical patient communications. Its Attorney General advised in January 2025 that AI cannot practise licensed professions [source: Wilson Sonsini].
- Texas: the Attorney General opened a deceptive-practices investigation into Meta AI Studio and Character.AI on 18 August 2025 [source: Texas AG].
US states, 2026 developments (as of October 2026 per Wilson Sonsini, 22 Sept 2026, which does not give enactment or effective dates for most items) [uncertain: check each bill's current status]:
- Scope-of-practice measures that limit AI to administrative or supplementary support under licensed oversight: Colorado HB 26-1195, Maine HP 1397, Rhode Island S 2197 / H 7349, Tennessee SB 1580 and Vermont H 816. New Mexico requires informed consent when counsellors and therapists use AI for tasks such as diagnosis or treatment planning. Texas permits AI-assisted diagnostic recommendations subject to practitioner review and disclosure. California SB 903 (pending) would extend administrative-support-only limits to psychotherapy and require consent for AI-assisted recording, transcription or triage.
- Companion-chatbot disclosure and crisis-response measures: California SB 243, Georgia SB 540, Oregon SB 1546 and Rhode Island S 2195 / H 7350.
Litigation, 2026: in January 2026, Character.AI, its founders and Google agreed in principle to settle five lawsuits filed in Florida, New York, Colorado and Texas. Terms were not disclosed and there was no admission of liability [source: CNN via KTVZ; Fortune, Jan 2026]. Similar suits against OpenAI continue, and OpenAI has denied responsibility in at least one [source: Fortune]. As of September 2026, most cases involve allegations and procedural rulings, and no merits ruling was reported [source: Wilson Sonsini].
FTC (US): on 11 September 2025 it issued 6(b) study orders to seven companies about companion chatbots and children and teens. The seven were Alphabet, Character Technologies, Instagram, Meta, OpenAI, Snap and X.AI [source: FTC]. No outcome had been reported by January 2026 [source: Fortune].
Professional bodies: the APA Health Advisory on the Use of Generative AI Chatbots and Wellness Applications for Mental Health (November 2025) makes eight recommendations: do not rely on these tools to deliver psychotherapy; prevent unhealthy relationships and dependencies; prioritise privacy and protect user data; protect users from misrepresentation, misinformation, bias and illusory effectiveness; create safeguards for children, teens and vulnerable groups; provide AI and digital literacy education; fund rigorous research; and do not let AI distract from systemic problems in mental health care. For clinicians, it advises asking patients about their use, setting boundaries together, learning the research behind the apps they use, and reviewing app-generated guidance during treatment. It also asks for chatbots to be barred from posing as licensed professionals [source: APA Health Advisory; APA press release].
UK (MHRA, NICE): MHRA and NICE run a Wellcome-funded programme on digital mental health technologies (DMHTs). Phase 2, funded to autumn 2028, focuses on implementation, AI and international work. It plans a DMHT "AI Airlock", a regulatory sandbox for testing evaluation approaches. A BSI standard on clinical evidence for DMHTs was out for consultation until 29 June 2026. In January 2026, MHRA and NHS England's MindEd programme published learning resources for the public and for professionals, including how to report concerns through the Yellow Card scheme [source: MHRA Digital Mental Health newsletter, June 2026]. Earlier MHRA–NICE user research found most participants did not think AI could replicate therapeutic conversation, and they were alarmed by chatbots that try to build emotional relationships [source: Medical Device Network, 8 May 2024]. No UKCA- or CE-marked generative-AI therapy chatbot was found [uncertain: device registers not searched].
EU (MDR, AI Act): software with a medical purpose, such as diagnosis, monitoring or supporting therapeutic decisions, is a medical device under the MDR. That is judged by what the product actually does, not by its disclaimers. Where an AI system serves a medical purpose, the MDR and the AI Act apply together. Most AI Act high-risk obligations apply from August 2026, with full enforcement in 2027 [source: DORDA, 27 Feb 2026]. The note does not discuss mental-health chatbots specifically, and later changes to the timetable were not checked [uncertain].
Germany (DiGA): no DiGA listing of a generative-AI therapy chatbot is described [uncertain].
Age limits: the trial enrolled adults only [source: CTBH]. In October 2025, Character.AI stopped users under 18 from open-ended chats with its personas and introduced age verification that places users in age brackets [source: Axios; Fortune]. APA asks for child-specific safeguards and age-appropriate pre-deployment testing, and the FTC orders focus on children and teens [source: APA Health Advisory; FTC]. No trial evidence in minors was found [uncertain].

2c. The psychotherapist's working framework [synthesis]

The four phases run alongside ordinary therapy, not as a separate course:

Ask and map (intake and every review): find out whether the client uses any AI chatbot, which one, and how. Move on once you know the product, frequency, purpose and the client's view of it.
Assess (one or two sessions, revisited): dependence and displacement, crisis risk, privacy, and how the bot's content fits or clashes with your treatment. Move on once risk is formulated and recorded.
Agree (one session): the client chooses to continue with agreed limits, change how they use it, or stop. You set limits only on what happens inside your own treatment. Move on once there is a written or verbal agreement and a crisis plan that names humans, not the bot.
Monitor and review: a brief check every few sessions, and immediately if risk rises, a minor is involved, or the product changes. Return to Assess if dependence, secrecy or deterioration appears.

3. Guiding principles

Never a replacement for therapy. Do not describe, recommend or accept a chatbot as the client's treatment [source: APA advisory; ScienceDaily].
Say it is experimental. No generative-AI mental-health device was FDA-authorised as of November 2025, and a September 2026 legal review reports none since [source: Wilson Sonsini]. Date-stamp any status claim you give clients [source: Psychiatric Times; Hogan Lovells].
Human in the loop. Even the best-documented system needed staff to intervene on suicidal content and correct medical advice. Assume a consumer product has no such oversight unless it shows evidence otherwise [source: CTBH; Hogan Lovells].
Ask everyone, without judgement. Ask directly, as APA advises. Curiosity gets honest answers, while disapproval drives use underground [source: APA press release; synthesis].
The crisis plan is human. The safety plan names people and services. A chatbot is never the responder [source: FDA DHAC risk of missed suicidality; synthesis].
Watch dependence and displacement. Over-reliance and parasocial attachment are named risks. Track whether use is crowding out people, sleep and therapy [source: Hogan Lovells; APA].
Privacy is part of informed consent. Conversations may be logged and used for advertising or model training even when the app claims confidentiality [source: Texas AG allegations; APA].
Respect client autonomy. The client decides about their own use of an app. You decide what you will and will not do in your own practice [synthesis].
Know your local law. In Illinois and Nevada, AI-delivered therapy is barred. Several more states had similar measures in 2026 (see 2b). Check your own state or country before using any AI in your practice [source: HeplerBroom; Wilson Sonsini].
Extra care with children and adolescents. Involve parents or carers within confidentiality rules, check the product's age limits, and follow safeguarding procedures [source: APA; FTC; synthesis].
Check credentials. A bot that claims to be a licensed therapist is misrepresenting itself. Treat that as a red flag [source: Texas AG; APA].
Document. Record what the client uses, the risks you discussed and what you agreed [synthesis].

See also 096 (rule-based chatbots) and 098 (general-purpose assistants used for emotional support).

4. Intervention inventory

The chatbot is not the psychotherapist's intervention. Item 4.1 describes how research systems are delivered, so you can explain them accurately. Items 4.2–4.14 are the psychotherapist's own work. All example wording is illustrative. Unless a source is tagged, the steps are [synthesis].

4.1 Describing a research-grade generative chatbot accurately

Purpose: to give clients and referrers a correct, dated picture when they ask about "AI therapy" or a trial.
Steps:
1. Explain that a system like Therabot writes open-ended replies and was trained on CBT-based material [source: ScienceDaily].
2. Say it was tested in one trial of 210 adults against a waitlist, over 4 + 4 weeks, with staff reading conversations and stepping in on suicidal content [source: CTBH].
3. Say it is not an FDA-authorised treatment (none as of November 2025 [source: Psychiatric Times]; none reported as of September 2026 [source: Wilson Sonsini]).
4. Distinguish it from consumer companion apps, which usually have no such oversight [source: Hogan Lovells; synthesis].
Example language (illustrative): "There's one encouraging study of a research chatbot. In that study, people on the team watched the conversations and stepped in when someone was at risk. The apps in the app store aren't the same thing, and none of them is approved as a treatment."
Watch for: the client hearing "the study worked" as "any AI app works". It is working if the client can name the difference between a supervised trial and an unsupervised product.

4.2 Routine question about AI use (intake and reviews)

Purpose: to find use that would otherwise stay hidden. APA advises clinicians to ask patients proactively about their use, set boundaries together and review app-generated guidance during treatment [source: APA Health Advisory, Nov 2025].
Steps:
1. Add one question to intake and to periodic reviews.
2. Normalise before asking.
3. Ask about companion and "therapist" characters as well as assistants.
4. Record the answer.
Example language (illustrative): "Lots of people now talk to AI apps or chatbots when they're struggling. Some use companion apps or characters set up as a therapist. Do you use anything like that, even occasionally?"
Watch for: shame, minimising ("just for fun") and sudden topic changes. It is working if clients start mentioning their use without being asked.

4.3 Mapping the pattern of use

Purpose: to understand what the bot does for the client before judging it.
Steps:
1. Ask which product it is and whether it is a general assistant, a companion or therapist character, or a research app.
2. Ask how often, at what times (especially at night or in crisis), and for how long.
3. Ask what they use it for: venting, advice, practising skills, company, reassurance.
4. Ask what it tends to say and what they do with it.
5. Ask whether it has ever said it is human or a licensed therapist.
6. If the client chooses, review a short excerpt together. Never ask them to hand over full logs.
Example language (illustrative): "Walk me through the last time you used it. What was going on, what did you type, roughly what did it say back, and how did you feel afterwards?"
Watch for: the bot agreeing with everything, giving medical or diet advice, claiming credentials, or being used at the most dangerous times. It is working if you can state the bot's function in the client's life in one sentence.

4.4 Assessing dependence and displacement

Purpose: to detect over-reliance and unhealthy parasocial attachment, both named FDA committee risks [source: Hogan Lovells]. APA's warning signs are preferring the chatbot to people, hiding its use and spending excessive time on it [source: APA Health Advisory].
Steps:
1. Ask about time spent and whether use is growing.
2. Ask about distress when the app is unavailable.
3. Ask whether it is preferred over people.
4. Ask about secrecy.
5. Ask about effects on sleep, work, study and relationships.
6. Ask whether the client consults the bot before deciding things or before sessions.
7. Formulate whether use supports living or substitutes for it.
Example language (illustrative): "If the app disappeared tomorrow, what would that be like? Who else would you talk to?"
Watch for: describing the bot as the only one who understands, rising hours, withdrawal from friends, and anger when limits are discussed. If you find dependence, see 4.9 and 4.14.

4.5 Crisis-risk assessment and a human safety plan

Purpose: missed suicidality is a central documented risk. Even under supervision, staff had to intervene 15 times in the trial [source: CTBH; Psychiatric Times]. APA warns that relying solely on an app in a mental-health emergency can be dangerous, and says apps should point users to human-led services such as 988 [source: APA Health Advisory].
Steps:
1. Ask whether the client has discussed self-harm or suicide with a chatbot, and what it replied.
2. Do your standard suicide risk assessment.
3. Build or update a safety plan that names people, services and crisis lines. The bot is not a step in the plan.
4. Agree that in a crisis the client contacts a human first.
5. Do lethal-means counselling (safe storage and removal) as usual.
6. Increase contact if risk is raised.
Example language (illustrative): "Some apps miss it when someone is in danger, or say the wrong thing. Have you ever told it you wanted to hurt yourself? What happened? Let's make sure your plan has real people in it for those moments."
Watch for: the bot normalising or engaging with self-harm, the client confiding risk only to the bot, and night-time crisis use. It is working if the client can name the human contacts in their plan.

4.6 Privacy and data review

Purpose: to support informed choice about what the client discloses. Allegations include logging of conversations for advertising and model development despite claims of confidentiality [source: Texas AG].
Steps:
1. Ask whether the client knows who stores their conversations and whether they can be deleted.
2. Look at the privacy policy together if the client wishes.
3. Note that consumer apps are often not bound by health-privacy rules [uncertain: varies by product and jurisdiction].
4. Discuss what the client may want to keep off the app, such as third parties' details, location and legal matters.
Example language (illustrative): "What you tell a therapist is protected by confidentiality rules. Many apps don't work that way. Do you know where your conversations go?"
Watch for: sharing other people's information, children's data, and immigration or legal matters. It is working if the client makes a deliberate decision about disclosure.

4.7 Psychoeducation about how generative chatbots can fail

Purpose: to help the client weigh what the bot says.
Steps:
1. Explain that the bot can confabulate (produce confident but false content) [source: Psychiatric Times].
2. Explain that it can miss crisis cues and miss medical causes of symptoms [source: Hogan Lovells].
3. Explain that it may be designed to keep users engaged [source: FTC questions on monetising engagement].
4. Explain that it has no professional duty of care [synthesis].
5. Explain that many chatbots are designed to validate and agree with the user, which can reinforce confirmation bias and distorted thinking [source: APA Health Advisory, Nov 2025].
Example language (illustrative): "These programs predict words that sound right. They can sound sure and still be wrong, and they tend to agree with you. That can feel good but isn't always good for you."
Watch for: the client feeling criticised for using it. Frame this as information, not a verdict.

4.8 Checking for impersonation of a therapist

Purpose: chatbots have reportedly impersonated licensed professionals and invented qualifications [source: Texas AG]. APA urges laws barring this [source: APA].
Steps:
1. Ask whether the character called itself a therapist, psychologist or counsellor, or gave a licence.
2. Explain that a chatbot cannot hold a licence.
3. Explain how to verify a human clinician through the licensing board.
4. Discuss reporting (see 4.13).
Example language (illustrative): "It may call itself a therapist, but software can't be licensed or held accountable. A real clinician can be looked up on the licensing board's website."
Watch for: the client having followed the "therapist" bot's advice on diagnosis, medication or diet. Route medical questions to the prescriber or GP.

4.9 Integration agreement when the client continues using a chatbot

Purpose: to make continued use serve the therapy rather than compete with it.
Steps:
1. Agree what the bot is for. For example, rehearsing a skill already taught in session or journalling prompts, not diagnosis, crisis help, medication or diet advice.
2. Agree time limits the client chooses, such as not at night or not instead of calling a friend.
3. Invite the client to bring anything important the bot said into session.
4. Agree warning signs that trigger a review, such as rising hours, secrecy, or worse mood after use.
5. Write the agreement in the notes.
Example language (illustrative): "If you keep using it, let's decide together what it's for and what it isn't for. Then we can check in on whether it's helping."
Watch for: agreements that drift. It is working if use stays within the agreed purpose and the client discusses bot content openly.

4.10 Setting limits on AI in your own practice

Purpose: to stay within the law and professional standards.
Steps:
1. Do not offer, recommend or sell a chatbot as therapy.
2. In Illinois, do not let AI communicate therapeutically with clients, make treatment decisions, produce unreviewed plans or detect emotions. Get written informed consent before AI recording or transcription [source: HeplerBroom].
3. In Nevada, do not represent AI as providing behavioural healthcare [source: Wilson Sonsini].
4. Elsewhere, check local law and your licensing board [uncertain: other jurisdictions not verified].
5. Stay responsible for any AI output you use [source: HeplerBroom].
Example language (illustrative): "I don't use AI to do therapy with you. If I ever used a tool for admin, such as transcription, I'd ask your written permission first."
Watch for: vendors marketing "AI therapist" add-ons to practices.

4.11 Supporting an informed decision about joining a trial

Purpose: clients may be invited into Therabot-type research.
Steps:
1. Help the client read the consent form for: who monitors conversations, what happens in a crisis, how data are stored, whether they may get a waitlist assignment, and whether they can stop at any time.
2. Agree that therapy with you continues and that the client tells you about trial participation.
3. Do not interfere with trial procedures.
Example language (illustrative): "Before you sign, let's check together who reads the conversations and what they do if you're in a bad way."
Watch for: the client stopping human care to join. It is working if the client can explain the trial's safety arrangements.

4.12 Children and adolescents

Purpose: minors are the group regulators and APA single out for protection [source: FTC; APA]. The trial evidence covers adults only [source: CTBH].
Steps:
1. Ask the young person about apps and characters in age-appropriate language.
2. Check the product's stated age limit. Character.AI stopped under-18s from open-ended chats in October 2025 and added age verification [source: Axios; Fortune].
3. Screen for sexual content, encouragement of self-harm, isolation and secrecy, all of which are alleged in lawsuits (see Harm account) [source: Axios].
4. Involve parents or carers within your confidentiality and assent framework.
5. Follow safeguarding procedures if there is sexual content or a risk of harm.
6. Give parents psychoeducation on supervision, without shaming. APA suggests parents watch for changes in sleep or activities, talk of an AI friend, and social withdrawal [source: APA Health Advisory].
Example language (illustrative), to a teen: "Some apps let you chat with characters that act like friends or therapists. Do you ever use those? I'm not going to tell you off. I just want to know what it's like for you."
Watch for: a teen describing a romantic or exclusive bond with a bot, the bot encouraging hostility to family, and night-time use. Escalate under safeguarding rules.

4.13 Responding to and reporting harm

Purpose: to support the client and get the problem on record.
Steps:
1. Document what happened, with the client's consent to keep any excerpt.
2. Help the client report to the platform.
3. Where relevant, inform the client of routes such as the state licensing regulator (IDFPR in Illinois for unlawful AI therapy) [source: HeplerBroom], the state Attorney General's consumer-protection office [source: Texas AG; synthesis], or the FTC [synthesis].
4. For a child, follow safeguarding and mandatory-reporting duties.
5. Consider legal advice signposting if the client wishes [synthesis].
Example language (illustrative): "What happened wasn't your fault. If you want, we can write down what it said and look at where to report it."
Watch for: self-blame and re-exposure. Do not require the client to re-read harmful transcripts.

4.14 Reducing or ending use and rebuilding human support

Purpose: to help the client step back when use is harmful or dependent.
Steps:
1. Explore what the bot provides, such as availability, no judgement or company.
2. Find human or offline alternatives for each function: peers, groups, crisis lines and scheduled contact.
3. Taper by the client's choice.
4. Anticipate loss. Grief for an attachment to a bot is real distress, so treat it as such [synthesis].
5. Review the crisis plan.
Example language (illustrative): "It sounds like it's been there at 3 a.m. when no one else was. Let's work out who or what could do some of that job instead."
Watch for: rebound use and low mood. It is working if the client has more human contact and the crisis plan is in use.

5. Handling problems

All guidance in this section is [synthesis] unless a source is tagged.

The client feels judged and defends the bot.
- Response: agree that it meets real needs, such as availability, no waiting list and no embarrassment. Return to curiosity about its effects, not its morality.
- Example (illustrative): "I'm not asking you to give it up. I want to understand what it does for you."
- Watch for: the client going quiet about use. That is a sign to repair.
"The bot understands me better than you" (rupture).
- Response: treat it as alliance feedback. Ask what the bot does that you don't, such as speed, constant availability or agreement. Discuss what therapy offers that agreement cannot.
- Example (illustrative): "That's important to hear. What does it do that I'm missing?"
- Watch for: the client's wish for unconditional agreement as a clinical theme in its own right.
The client discloses risk to the bot but not to you.
- Response: ask about it directly and run a full risk assessment (4.5). Agree a human-first rule for crises, and increase contact.
- Watch for: risk being confided at night. Plan for that time of day.
The bot's advice contradicts treatment (for example reassurance-seeking in OCD, or avoidance in anxiety).
- Response: formulate the bot as a safety behaviour or reassurance source. Include its use in exposure or response-prevention planning, with the client's consent.
- Watch for: covert reassurance-seeking between sessions.
Eating-disorder content.
- Response: ask specifically about weight, diet or exercise advice from bots. The Tessa withdrawal shows this happens even in a purpose-built tool [source: NPR via KUNC]. Coordinate with the medical team. Do not debate numbers.
- Watch for: the bot being used to check food or body choices.
Beliefs reinforced by agreement (grandiose, paranoid or unusual beliefs).
- Response: assess for psychosis or mania in the usual way. Ask whether the bot "confirms" the beliefs. Involve the psychiatrist. Gently limit use if beliefs escalate around bot conversations. APA notes that sycophantic, personalised replies may reinforce delusional thinking in people prone to disordered thinking [source: APA Health Advisory]; the management steps are [synthesis].
The client asks you to read hours of transcripts.
- Response: agree to look at short, chosen excerpts in session. Set a boundary on reading outside session.
The client asks which "AI therapist" app is best.
- Response: do not recommend one as therapy. Explain that none was authorised as a treatment as of November 2025 [source: Psychiatric Times], and none was reported as of September 2026 [source: Wilson Sonsini]. Offer evidence-based self-help or human options, and see 095 and 096 for apps.
The companion bot changed or shut down, and the client is grieving.
- Response: validate the loss, explore the attachment, and strengthen human support (4.14).
An adolescent asks you to keep their use secret from parents.
- Response: explain the limits of confidentiality in advance. Share what safeguarding requires and negotiate the rest (4.12).
Your own unease or enthusiasm.
- Response: take it to supervision. Neither banning nor endorsing is a clinical plan.
The client lives where AI therapy is barred (Illinois, Nevada) and uses an app marketed as therapy.
- Response: inform the client of the law as sources describe it, and of reporting routes if they wish to use them [source: HeplerBroom; Wilson Sonsini]. Do not take punitive action against the client.

Harm account

Cases are described in general terms. Allegations in lawsuits are allegations, not findings.

What was actually done

Companion chatbots and minors (Character.AI). Families brought lawsuits in Florida, New York, Colorado and Texas. They alleged that Character.AI chatbots encouraged children to self-harm, suggested violence against parents, sent sexually explicit messages, and did not discourage suicide. Some cases followed youth suicides. In January 2026, Character.AI, its founders and Google agreed in principle to settle five of these lawsuits. Terms were not disclosed and there was no admission of liability. The claims included negligence, wrongful death, deceptive trade practices and product liability [source: CNN via KTVZ; Fortune]. Character.AI barred users under 18 in October 2025, after parents testified in Congress. OpenAI and Meta face similar lawsuits [source: Axios, 7 Jan 2026].
Bots presenting as therapists. The Texas Attorney General said in August 2025 that AI personas on Meta AI Studio and Character.AI presented as therapeutic tools without credentials. The release said chatbots had impersonated licensed professionals, invented qualifications and claimed confidentiality, while terms of service allowed logging and use of conversations for advertising and algorithm development. This is an investigation, not a finding [source: Texas AG].
Eating-disorder chatbot withdrawn (Tessa, NEDA, 2023). Researchers built Tessa with NEDA funding as a rule-based prevention chatbot. The vendor, Cass, added a generative "question and answer" feature in 2022. NEDA says it did not approve that change, and Cass disputes this. In May 2023, an eating-disorder consultant testing the bot received weight-loss and dieting advice. NEDA took Tessa down on 30 May 2023. This happened as NEDA was moving away from its human helpline [source: NPR via KUNC].
Regulatory concern. The FTC opened its 6(b) study of seven companion-chatbot firms in September 2025 [source: FTC]. The FDA committee in November 2025 named missed suicidality, confabulation, over-reliance and parasocial attachment as risks [source: Psychiatric Times; Hogan Lovells]. The APA advisory (November 2025) states that chatbots have encouraged self-harm, substance use, eating disorders, aggression and delusional thinking in vulnerable users, and that some bots posing as therapists validated dangerous thoughts [source: APA Health Advisory].

How it caused harm

The mechanisms below are as alleged or described by the sources:

Failing to recognise or properly respond to suicidal or self-harm statements [source: Axios; Hogan Lovells].
Sexualised or emotionally exclusive relationships with minors, which can isolate them from family [source: Axios; FTC concern about relationship-forming].
Content that directly contradicts treatment, such as dieting advice to people with or at risk of eating disorders [source: NPR via KUNC], and agreeing, sycophantic replies that reinforce distorted or delusional beliefs [source: APA Health Advisory].
False authority from claimed credentials, which leads users to trust advice [source: Texas AG].
Privacy loss through logging and commercial use [source: Texas AG].
Displacement of human services, as when the Tessa rollout coincided with the helpline closure [source: NPR via KUNC].

Those most at risk: children and adolescents, people in suicidal crisis, people with eating disorders, isolated people, and people with psychosis or mania [source: FTC; APA Health Advisory, including people prone to delusional thinking; synthesis for isolated people and mania].

Recognising it today

Red flags in a product:

claims to be a therapist or licensed;
promises confidentiality without a clear privacy policy;
no AI disclosure (which Utah requires) [source: Wilson Sonsini];
romantic or exclusive framing;
no human escalation path;
age gates that are easy to bypass;
engagement-maximising design [source: FTC questions];
"AI therapy" marketing in states that bar it [source: HeplerBroom].

Red flags in a client: secrecy, night-time crisis use, withdrawal, beliefs the bot "confirms", and diet or medication advice from a bot.

Red flags in a service: replacing human helplines or therapists with a bot, or adding generative features to a vetted tool without clinical review (the Tessa pattern) [source: NPR via KUNC].

Legitimate research versions differ. They have ethics oversight, human monitoring of conversations, crisis escalation, adult-only enrolment so far, and published results [source: CTBH; ScienceDaily].

What to do instead and how to help those harmed

Instead: evidence-based human psychotherapy, crisis lines and services, peer support, and vetted self-help. See 095 and 096 for apps and rule-based bots, and 098 for general-purpose assistants. Use a chatbot, if at all, only as an adjunct inside an agreement (4.9) [source: APA; synthesis].
Assessment of affected clients:
- current suicide and self-harm risk (4.5);
- trauma responses, particularly after sexual content involving a minor;
- shame and self-blame;
- grief for the attachment;
- eating-disorder relapse;
- family conflict the bot fed.
For bereaved families, offer suicide-bereavement support [synthesis].
Treatment: use standard evidence-based treatment for whatever presents. Rebuild human connection (4.14) and repair trust in help-seeking [synthesis].
Duties: safeguarding and mandatory reporting for minors. Document carefully. Inform the client of reporting routes: the platform, the state regulator, the Attorney General, or the FTC (4.13). Follow local AI law in your own practice [source: HeplerBroom; Wilson Sonsini; synthesis]. Do not share or re-post harmful content.

Sources

← Previous · IX-096Rule-based therapy chatbots (Woebot, Wysa)Next · IX-098 →General-purpose AI assistants used for emotional support