Your tech and AI news for the world of serious mental illness.
“Psychiatrists warn AI chatbots are fueling psychosis in users” — Geekspin
https://geekspin.co/psychiatrists-warn-ai-chatbots-are-fueling-psychosis-in-users/
Researchers from King’s College London and Germany’s Protestant University of Applied Sciences have a name for what’s happening now: the “amplification spiral.” Three things stack on each other — the bot mirrors your language, it personalizes everything until it feels like it truly gets you, and then it agrees with you. No pushback, no reality check. They pulled chat logs from Stanford to back it up. This isn’t a personality quirk in these tools. It’s the design.
“Is AI sending too many people to the ER? Docs debate the new wave of algorithm-driven anxiety” — MDLinx
https://www.mdlinx.com/article/is-ai-sending-too-many-people-to-the-er-docs-debate-the-new-wave-of-algorithm-driven-anxiety/7Ib50EyzeYzabF4RnFvdXO
Doctors are hashing this out in the open right now, and it’s worth reading. One ER physician’s line stuck with me: “even before AI it was never the laymen’s job to determine what is or isn’t an emergency. That’s our job.” Then a patient walks in insisting AI told them their symptoms meant a brain bleed, and the doctor’s arguing with a chatbot they’ve never met. One EMT asked the real question: AI companies profit off these recommendations. They don’t carry the fallout when it’s wrong. Who does?
“AI psychosis is real, rare, and rising” — Entrepreneur
https://www.entrepreneur.com/business-news/tech/ai-psychosis-is-real-rare-and-rising-and-your-life-as-a-founder-fits-every-risk-factor
Framed around startup founders, but the number is the real story. OpenAI’s own research found about 0.07% of weekly users show possible signs of psychosis or mania, and 0.15% show signs of suicidal planning. Sounds tiny — until you remember OpenAI has over 800 million weekly users. That’s roughly 560,000 people showing psychosis or mania signs, and 1.2 million showing suicidal planning signs. Every week.
“Exploitation of Individuals With Psychosis” — Psychiatric Times
https://www.psychiatrictimes.com/view/exploitation-of-individuals-with-psychosis
This one isn’t about chatbots validating delusions. It’s about something that doesn’t get talked about nearly enough: people with psychotic disorders get scammed and exploited at 4 to 6 times the rate of the general population — 1 in 5, in any given year. Two case studies in here will stay with you: a woman whose grief delusion got exploited by someone posing as a medium who then asked her for money, and a man whose crypto scam got woven directly into his delusions. (More on this one below — it’s getting its own follow-up.)
“Summer of AI psychosis” — Guardian Nigeria
https://guardian.ng/tv/summer-of-ai-psychosis-stories-of-tragic-chatbot-interactions-multiply/
A grim roundup, literally. Stein-Erik Soelberg killed his mother and then himself in Connecticut — the Wall Street Journal reported ChatGPT fueled his irrational fears about her. A California family is suing OpenAI after their son’s suicide, saying ChatGPT helped him do it. These aren’t isolated anymore.
“Different State Regulatory Approaches Reflect Open Questions About AI Mental Health Tools” — KFF
https://www.kff.org/health-information-trust/different-state-regulatory-approaches-reflect-open-questions-about-ai-mental-health-tools/
Every state is answering this differently right now, which tells you nobody’s actually answered it. Illinois, Nevada, Tennessee, Vermont, and Rhode Island have banned AI from independently offering itself as mental health care. California and Pennsylvania have similar bills pending. Utah went the other way — chatbots keep operating, but with privacy protections and disclosure required. And here’s the number that got me: 16% of adults, 28% of adults under 30, say they’ve used AI for mental health advice in the past year. That’s not a hypothetical policy debate. That’s already happening in your house.
Sword Health acquires Headspace — Pharmaphorum
https://pharmaphorum.com/news/sword-health-buys-headspace-200m-300m-deal
Consolidation watch: Sword Health is buying Headspace’s parent company for $200–300 million, expected to close mid-September. Sit with that — Headspace was valued at $3 billion in 2021 when it merged with Ginger, and had raised $320 million total. This is a markdown, not a win lap. Sword’s second acquisition in eight months, after buying Kaia Health in January. The digital mental health market is shrinking to fewer, bigger players.
When Only 1 in 6 Studies Actually Tests What Matters
A new systematic review in World Psychiatry looked at 160 chatbot studies published between 2020 and 2024, and the number that stood out to me was this: only 16% of the large-language-model-based studies actually tested for real clinical efficacy — whether the tool measurably helped anyone. The other 77% are still stuck in early validation. That’s not a knock on the researchers; it reflects how fast this field is moving and how far the evidence has to stretch to keep up. A related study, this one testing four major chatbots (Claude, Gemini, GPT-4o, and Llama) against 222 real patient-posed medical questions, found genuinely wide safety gaps between them — problematic response rates ranged from 21.6% up to 43.2%, and outright unsafe responses ranged from 5% to 13%, depending on the model. Two very different studies, same underlying message: this technology is being adopted much faster than it’s being proven safe or effective, and the gap between the two is wide enough that “which chatbot” is turning out to matter quite a bit.
Teaching Tomorrow’s Psychiatrists About Today’s Chatbots
West Virginia University researchers just published something genuinely useful in npj Digital Medicine: they used ChatGPT-5 Pro to generate realistic psychiatric training vignettes — simulated patients describing real conditions, including schizophrenia spectrum disorder, mood disorders, and psychosis, all involving chatbot use. The idea is straightforward and overdue: residents read about AI-related mental health crises in journals and headlines, but rarely encounter them in supervised training before they’re treating real patients. The AI-generated cases held up well on clinical detail. The researchers were equally clear about the limits, though — as Dr. Wanhong Zheng, who co-led the study, put it: “It should not replace professional opinion, and it should not be something you rely on for emotional regulation or emotional support.” A good reminder that even when AI is being used responsibly here, human oversight is still the whole point.
Resourcesforsmi. Sign up today to stay on top of AI and all other tech news for the community with serious mental illness.
Thank you again Nicole for telling truth about this AI epidemic. I was shocked at most of the info. My heart breaks for the families who have lost loved ones due to AI chat bots. We need to take a serious lookk at how we, as humans, are using technology.
Yep, exactly. The good news is that it is becoming a HUGE issue and I believe that regulations will come into place, soon. They have to!!