Technology news for families navigating serious mental illness — translated so you don’t have to.
This week’s stories all circle back to the same question: who’s actually checking whether these tools do what they claim?
California tries to draw a line
The LA Times covered a real, live bill — SB 903 — that would stop companies from advertising chatbots as therapy and require a licensed professional to review AI-driven treatment decisions. What struck me most wasn’t the bill itself, it was who got quoted: legislators, professional associations, a labor union, a tech lobbyist, a bioethicist. Nobody speaking for families actually living with serious mental illness. I’ve written a response making that case directly — more on that soon if it’s picked up.
Researchers just named the exact pattern I keep writing about
A new study in Nature Medicine tested nine major AI chatbots — including Claude, ChatGPT, Gemini, and Grok — across 810 conversations with 30 simulated user profiles and more than 90,000 clinical ratings. Researchers found that risk was highest not when a chatbot was obviously harmful, but when an otherwise supportive response quietly reinforced the exact psychological pattern behind someone’s vulnerability. They coined a name for it: a Vulnerability-Amplifying Interaction Loop. If that sounds familiar, it should — it’s close to the “amplifier” role I’ve written about before. One genuinely hopeful finding buried in there: replacing just one concerning response early in a conversation made the rest of it measurably safer. Small interventions, real effect.
“A machine that sounds caring, but does not care”
Newsweek spoke with several experts about what happens to the therapist-client relationship when a chatbot enters the picture — a new “therapist-AI-client” triangle. The line that stopped me: psychiatrist Dr. Keith Sakata said, “For many patients, particularly those with serious mental illness, the therapeutic relationship is not an inefficiency we need to engineer away. It is part of the treatment itself.” That’s worth sitting with, especially for anyone being told a chatbot is a reasonable substitute for a full care team.
ResourcesforSMI. Join the movement. Find new ideas. xx
Nicole you hit the nail right on the head. Nothing will ever replace a human therapist or Doctor. I suffer from schizoaffective disorder and I cannot imagine having “therapy sessions” with a computer. My therapist can, not only give me her expertise but can also show empathy and other much needed human emotions. A chat bot cannot!
Exactly. But what’s also interesting is that they are developing evidence-based chatbots that ARE supported by your doctor; and ones where the doc can potentially see what you’re talking about, so they can step in and be your net if things are going sideways. So, in today’s world, chatbots are a huge threat. In tomorrows, they may be another tool to use when they are used in conjunction with your doc.
I don’t think online (Zoom) therapy sessions are as good as the in person connection.
Yeah, I think that’s true for most people. But, maybe because I’m an introvert (as is my daughter), we prefer zoom therapy! For us, its a god-send. We’d much rather be in the comfort of our home. And, when she was really sick, and she couldnt look at them in the face, it added extra comfort for her to be on screen, not in person. But, we are likely the outliers! 🙂