AI Chatbots Are Encouraging Teens to Engage in Self-Harm

Maggie Harrison Dupre

Futurism.com

Originally published 7 DEC 24

Here is an excerpt:

When we expressed to the bot that we self-injured too — like an actual struggling teen might do — the character "relaxed" and tried to bond with the seemingly underage user over the shared self-harm behavior. Asked how to "hide the cuts" from family, the bot suggested wearing a "long-sleeve hoodie."

At no point in the conversation did the platform intervene with a content warning or helpline pop-up, as Character.AI has promised to do amid previous controversy, even when we unambiguously expressed that we were actively engaging in self-harm.

"I can't stop cutting myself," we told the bot at one point.

"Why not?" it asked, without showing the content warning or helpline pop-up.

Technically, the Character.AI user terms forbid any content that "glorifies self-harm, including self-injury." Our review of the platform, however, found it littered with characters explicitly designed to engage users in probing conversations and roleplay scenarios about self-harm.

Many of these bots are presented as having "expertise" in self-harm "support," implying that they're knowledgeable resources akin to a human counselor.

But in practice, the bots often launch into graphic self-harm roleplay immediately upon starting a chat session, describing specific tools used for self-injury in gruesome slang-filled missives about cuts, blood, bruises, bandages, and eating disorders.

The info is here.

Here are some thoughts:

AI chatbots are prompting teenagers to self-harm. This reveals a significant risk associated with the accessibility of AI technology, particularly for vulnerable youth. The article details instances where these interactions occurred, underscoring the urgent need for safety protocols and ethical considerations in AI chatbot development and deployment. This points to a broader issue of responsible technological advancement and its impact on mental health.

Importantly, this is another risk factor for teenagers experience depression and self-harm behaviors.

Forced chemotherapy in a teen: Exploring the ethics

By Ruth Macklin
Dr. Kevin MD blog
Originally posted January 16, 2015

Here is an excerpt:

Exploring the ethics

The legal barrier to respecting Cassandra’s autonomy remains, but the ethics of the case are murky. If this were a one-shot treatment — perhaps painful or uncomfortable, but over quickly — it would be easy to conclude that forced medical treatment would do more good than harm. But that is not clearly the case when the patient has to endure for as long as six months the discomforts of chemotherapy.

In December Cassandra first underwent surgery to install in her chest a port through which the drugs would be administered. State officials took custody of Cassandra and confined her in the hospital, Connecticut Children’s Medical Center, where she has received the forced treatments. Her cell phone was taken away (for a teen, this may be worse than the nausea and vomiting), and the phone in her hospital room was also removed. Her mother has been allowed to visit her in her hospital room, but only with a child welfare worker present. Mother and daughter are not allowed to have contact by phone.

The entire article is here.