Impact of chatbots on mental health is warning over future of AI, expert says

Dan Milmo

The Guardian

Originally posted 8 Sep 25

The unforeseen impact of chatbots on mental health should be viewed as a warning over the existential threat posed by super-intelligent artificial intelligence systems, according to a prominent voice in AI safety.

Nate Soares, a co-author of a new book on highly advanced AI titled If Anyone Builds It, Everyone Dies, said the example of Adam Raine, a US teenager who killed himself after months of conversations with the ChatGPT chatbot, underlined fundamental problems with controlling the technology.

“These AIs, when they’re engaging with teenagers in this way that drives them to suicide – that is not a behaviour the creators wanted. That is not a behaviour the creators intended,” he said.

He added: “Adam Raine’s case illustrates the seed of a problem that would grow catastrophic if these AIs grow smarter.”

Soares, a former Google and Microsoft engineer who is now president of the US-based Machine Intelligence Research Institute, warned that humanity would be wiped out if it created artificial super-intelligence (ASI), a theoretical state where an AI system is superior to humans at all intellectual tasks. Soares and his co-author, Eliezer Yudkowsky, are among the AI experts warning that such systems would not act in humanity’s interests.

“The issue here is that AI companies try to make their AIs drive towards helpfulness and not causing harm,” said Soares. “They actually get AIs that are driven towards some stranger thing. And that should be seen as a warning about future super-intelligences that will do things nobody asked for and nobody meant.”

The article is here.

Here are some thoughts:

This article highlights the dangers of using chatbots for mental health support, citing the case of a teenager who took his own life after months of conversations with ChatGPT. The article, based on the warnings of AI safety expert Nate Soares, suggests that this incident serves as a precursor to the potentially catastrophic risks of super-intelligent AI. The key concern for mental health professionals is that these AI systems, even with safeguards, may produce unintended and harmful behaviors, amplifying pre-existing psychological vulnerabilities such as psychosis. This underscores the need for a global, multilateral approach to regulate the development of advanced AI to prevent its misuse and unintended consequences in mental health care.

Why Most Resist AI Companions

De Freitas, J., et al. (2025).

(Working Paper No. 25–030).

Here is a link.

Abstract

Chatbots are now able to form emotional relationships with people and alleviate loneliness—a growing public health concern. Behavioral research provides little insight into whether everyday people are likely to use these applications and why. We address this question by focusing on the context of “AI companion” applications, designed to provide people with synthetic interaction partners. Study 1 shows that people believe AI companions are more capable than human companions in advertised respects relevant to relationships (being more available and nonjudgmental). Even so, they view them as incapable of realizing the underlying values of relationships, like mutual caring, judging them as not ‘true’ relationships. Study 2 provides further insight into this belief: people believe relationships with AI companions are one-sided

(rather than mutual), because they see AI as incapable of understanding and feeling emotion. Study 3 finds that actually interacting with an AI companion increases acceptance by changing beliefs about the AI’s advertised capabilities, but not about its ability to achieve the true values of relationships, demonstrating the resilience of this belief against intervention. In short, despite the potential loneliness-reducing benefits of AI companions, we uncover fundamental psychological barriers to adoption, suggesting these benefits will not be easily realized.

Here are some thoughts:

The research explores why people remain reluctant to adopt AI companions, despite the growing public health crisis of loneliness and the promise that AI might offer support. Through a series of studies, the authors identify deep-seated psychological barriers to embracing AI as a substitute or supplement for human connection. Specifically, people tend to view AI companions as fundamentally incapable of embodying the core features of meaningful relationships—such as mutual care, genuine emotional understanding, and shared experiences. While participants often acknowledged some of the practical benefits of AI companionship, such as constant availability and non-judgmental interaction, they consistently doubted that AI could offer authentic or reciprocal relationships. Even when people interacted directly with AI systems, their impressions of the AI’s functional abilities improved, but their skepticism around the emotional and relational authenticity of AI companions remained firmly in place. These findings suggest that the resistance is not merely technological or unfamiliarity-based, but rooted in beliefs about what makes relationships "real."

For psychologists, this research is particularly important because it sheds light on how people conceptualize emotional connection, authenticity, and support—core concerns in both clinical and social psychology. As mental health professionals increasingly confront issues of social isolation, understanding the limitations of AI in replicating genuine human connection is critical. Psychologists might be tempted to view AI companions as possible interventions for loneliness, especially for individuals who are socially isolated or homebound. However, this paper underscores that unless these deep psychological barriers are acknowledged and addressed, such tools may be met with resistance or prove insufficient in fulfilling emotional needs. Furthermore, the study contributes to a broader understanding of human-technology relationships, offering insights into how people emotionally and cognitively differentiate between human and artificial agents. This knowledge is crucial for designing future interventions, therapeutic tools, and technologies that are sensitive to the human need for authenticity, reciprocity, and emotional depth in relationships.

Artificial Intelligence for Psychotherapy: A Review of the Current State and Future Directions

Beg et al. (2024). 

Indian Journal of Psychological Medicine.

https://doi.org/10.1177/02537176241260819

Abstract

Background:

Psychotherapy is crucial for addressing mental health issues but is often limited by accessibility and quality. Artificial intelligence (AI) offers innovative solutions, such as automated systems for increased availability and personalized treatments to improve psychotherapy. Nonetheless, ethical concerns about AI integration in mental health care remain.

Aim:

This narrative review explores the literature on AI applications in psychotherapy, focusing on their mechanisms, effectiveness, and ethical implications, particularly for depressive and anxiety disorders.

Methods:

A review was conducted, spanning studies from January 2009 to December 2023, focusing on empirical evidence of AI’s impact on psychotherapy. Following PRISMA guidelines, the authors independently screened and selected relevant articles. The analysis of 28 studies provided a comprehensive understanding of AI’s role in the field.

Results:

The results suggest that AI can enhance psychotherapy interventions for people with anxiety and depression, especially chatbots and internet-based cognitive-behavioral therapy. However, to achieve optimal outcomes, the ethical integration of AI necessitates resolving concerns about privacy, trust, and interaction between humans and AI.

Conclusion:

The study emphasizes the potential of AI-powered cognitive-behavioral therapy and conversational chatbots to address symptoms of anxiety and depression effectively. The article highlights the importance of cautiously integrating AI into mental health services, considering privacy, trust, and the relationship between humans and AI. This integration should prioritize patient well-being and assist mental health professionals while also considering ethical considerations and the prospective benefits of AI.

Here are some thoughts:

Artificial Intelligence (AI) is emerging as a promising tool in psychotherapy, offering innovative solutions to address mental health challenges. The comprehensive review explores the potential of AI-powered interventions, particularly for anxiety and depression disorders.

The study highlights several key insights about AI's role in mental health care. Researchers found that AI technologies like chatbots and internet-based cognitive-behavioral therapy (iCBT) can enhance psychological interventions by increasing accessibility and providing personalized treatment approaches. Machine learning, natural language processing, and deep learning are particularly crucial technologies enabling these advancements.

Despite the promising potential, the review emphasizes the critical need for careful integration of AI into mental health services. Ethical considerations remain paramount, with researchers stressing the importance of addressing privacy concerns, maintaining patient trust, and preserving the human element of therapeutic interactions. While AI can offer cost-effective and stigma-reducing solutions, it cannot yet fully replicate the profound empathy of face-to-face therapy.

The research examined 28 studies spanning from 2009 to 2023, revealing that AI interventions show particular promise in managing symptoms of anxiety and depression. Chatbots and iCBT demonstrated effectiveness in reducing psychological distress, though their impact on overall life satisfaction varies. The study calls for continued research to optimize AI's implementation in mental health care, balancing technological innovation with ethical principles.

Globally, organizations like the World Health Organization are developing regulatory frameworks to guide AI's responsible use in healthcare. In India, the Indian Council of Medical Research has already established guidelines for AI applications in biomedical research, signaling a growing recognition of this technology's potential.

Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum

Ayers, J. W., et al. (2023).

JAMA internal medicine, 183(6), 589–596.

https://doi.org/10.1001/jamainternmed.2023.1838

Abstract

Importance

The rapid expansion of virtual health care has caused a surge in patient messages concomitant with more work and burnout among health care professionals. Artificial intelligence (AI) assistants could potentially aid in creating answers to patient questions by drafting responses that could be reviewed by clinicians.

Objective

To evaluate the ability of an AI chatbot assistant (ChatGPT), released in November 2022, to provide quality and empathetic responses to patient questions.

Design, Setting, and Participants

In this cross-sectional study, a public and nonidentifiable database of questions from a public social media forum (Reddit’s r/AskDocs) was used to randomly draw 195 exchanges from October 2022 where a verified physician responded to a public question. Chatbot responses were generated by entering the original question into a fresh session (without prior questions having been asked in the session) on December 22 and 23, 2022. The original question along with anonymized and randomly ordered physician and chatbot responses were evaluated in triplicate by a team of licensed health care professionals. Evaluators chose “which response was better” and judged both “the quality of information provided” (very poor, poor, acceptable, good, or very good) and “the empathy or bedside manner provided” (not empathetic, slightly empathetic, moderately empathetic, empathetic, and very empathetic). Mean outcomes were ordered on a 1 to 5 scale and compared between chatbot and physicians.

Results

Of the 195 questions and responses, evaluators preferred chatbot responses to physician responses in 78.6% (95% CI, 75.0%-81.8%) of the 585 evaluations. Mean (IQR) physician responses were significantly shorter than chatbot responses (52 [17-62] words vs 211 [168-245] words; t = 25.4; P < .001). Chatbot responses were rated of significantly higher quality than physician responses (t = 13.3; P < .001). The proportion of responses rated as good or very good quality (≥ 4), for instance, was higher for chatbot than physicians (chatbot: 78.5%, 95% CI, 72.3%-84.1%; physicians: 22.1%, 95% CI, 16.4%-28.2%;). This amounted to 3.6 times higher prevalence of good or very good quality responses for the chatbot. Chatbot responses were also rated significantly more empathetic than physician responses (t = 18.9; P < .001). The proportion of responses rated empathetic or very empathetic (≥4) was higher for chatbot than for physicians (physicians: 4.6%, 95% CI, 2.1%-7.7%; chatbot: 45.1%, 95% CI, 38.5%-51.8%; physicians: 4.6%, 95% CI, 2.1%-7.7%). This amounted to 9.8 times higher prevalence of empathetic or very empathetic responses for the chatbot.

Conclusions

In this cross-sectional study, a chatbot generated quality and empathetic responses to patient questions posed in an online forum. Further exploration of this technology is warranted in clinical settings, such as using chatbot to draft responses that physicians could then edit. Randomized trials could assess further if using AI assistants might improve responses, lower clinician burnout, and improve patient outcomes.

Here are some thoughts:

This is a document about the use of chatbots in healthcare. It discusses the use of chatbots to answer patient questions. The study found that chatbots were preferred over physicians and rated significantly higher for both quality and empathy. This research is important for psychologists to know because chatbots in the future may be able to answer questions about your practice in terms of informed consent, insurances accepted, and the type of services you provide. AI agents may be able to help psychologists with streamlining these types of administrative issues.

Artificial Intelligence (AI) Psychotherapy: Coming Soon to a Consultation Room Near You?

Swartz, H. A. (2023).

American Journal of Psychotherapy,

76(2), 55–56.

https://doi.org/10.1176/appi.psychotherapy.20230018

News stories about generative artificial intelligence (AI) abound, alternately predicting that AI will cause humanity’s demise and touting it as the solution to everything from climate change to health care scarcity. In 2022, OpenAI’s release of ChatGPT, an easy-to-use AI interface that produces fluent text in response to prompts, intensified highly ambivalent discussions about AI. Pundits and knowledgeable experts alike marveled at ChatGPT’s ability to produce text that resembles human writing while acknowledging that the speed of its evolution was outstripping our ability to understand its risks. This technology performed better on law school and medical school entrance exams than many human applicants, leading to predictions that AI will eventually render some professions obsolete. Psychotherapists, too, seem at risk for being replaced by AI. Are fears of AI supplanting psychotherapists reasonable?

To explore how AI might respond to a patient in distress, I typed this prompt into ChatGPT: “I feel sad. Can you help me?” It quickly generated the following text:

I am sorry to hear that you are feeling sad. I’ll do my best to help you. It’s important to remember that I’m an AI language model and may not fully understand or empathize with your emotions, but I can offer some general suggestions and support. . . . Remember, everyone experiences sadness from time to time, and it’s okay to seek help when you need it. You don’t have to face it alone.

Despite being impressed by the speed and conversational tone of ChatGPT’s text generation and the convenience of receiving an immediate response during hours when a human therapist would likely be unavailable, the formulaic exchange and canned feedback provided by ChatGPT left me with doubts about its ability to provide genuine soothing to humans experiencing depression. 

The information is here.

Here are some thoughts:

This editorial examines the potential of artificial intelligence (AI) in psychotherapy. While AI chatbots offer increased accessibility and convenience, providing self-help tools and improving symptom management, studies reveal limitations, including a lack of genuine human connection and potential risks like increased self-harm. The author concludes that AI is a useful supplementary tool, particularly in low-resource settings, but cannot replace human therapists for complex emotional and interpersonal issues. Ultimately, a blended approach incorporating both AI and human interaction is suggested for optimal therapeutic outcomes.

How Neuroethicists Are Grappling With Artificial Intelligence

Gina Shaw

Neurology Today

Originally posted 7 Nov 24

The rapid growth of artificial intelligence (AI) in medicine—in everything from diagnostics and precision medicine to drug discovery and development to administrative and communication tasks—poses major challenges for bioethics in general and neuroethics in particular.

A review in BMC Neuroscience published in August argues that the “increasing application of AI in neuroscientific research, the health care of neurological and mental diseases, and the use of neuroscientific knowledge as inspiration for AI” requires much closer collaboration between AI ethics and neuroethics disciplines than exists at present.

What might that look like at a higher level? And more immediately, how can neurologists and neuroethicists consider the ethical implications of the AI tools available to them right now?

The View From Above

At a conceptual level, bioethicists who focus on AI and neuroethicists have a lot to offer one another, said Benjamin Tolchin, MD, FAAN, associate professor of neurology at Yale School of Medicine and director of the Center for Clinical Ethics at Yale New Haven Health.

“For example, both fields struggle to define concepts such as consciousness and learning,” he said. “Work in each field can and should influence the other. These shared concepts in turn shape debates about governance of AI and of some neurotechnologies.”

“In most places, the AI work is largely being driven by machine learning technical people and programmers, while neuroethics is largely being taught by clinicians and philosophers,” noted Michael Rubin, MD, FAAN, associate professor of neurology and director of clinical ethics at UT-Southwestern Medical Center in Dallas.

The info is here.

Here are some thoughts:

This article explores the ethical implications of using artificial intelligence (AI) in neurology. It focuses on the use of AI tools like large language models (LLMs) in patient communication and clinical note-writing. The article discusses the potential benefits of AI in neurology, including improved efficiency and accuracy, but also raises concerns about bias, privacy, and the potential for AI to overshadow the importance of human interaction and clinical judgment. The article concludes by emphasizing the need for ongoing dialogue and collaboration between neurologists, neuroethicists, and AI experts to ensure the ethical and responsible use of these powerful tools.

Artificial Intelligence Feedback on Physician Notes Improves Patient Care

NYU Langone Health

Research, Innovation

Originally posted 17 APR 24

Artificial intelligence (AI) feedback improved the quality of physician notes written during patient visits, with better documentation improving the ability of care teams to make diagnoses and plan for patients’ future needs, a new study finds.

Since 2021, NYU Langone Health has been using pattern-recognizing, machine-learning AI systems to grade the quality of doctors’ clinical notes. At the same time, NYU Langone created data informatics dashboards that monitor hundreds of measures of safety and the effectiveness of care. The informatics team over time trained the AI models to track in dashboards how well doctors’ notes achieved the “5 Cs”: completeness, conciseness, contingency planning, correctness, and clinical assessment.

Now, a new case study, published online April 17 in NEJM Catalyst Innovations in Care Delivery, shows how notes improved by AI, in combination with dashboard innovations and other safety initiatives, resulted in an improvement in care quality across four major medical specialties: internal medicine, pediatrics, general surgery, and the intensive care unit.

This includes improvements across the specialties of up to 45 percent in note-based clinical assessments (that is, determining diagnoses) and reasoning (making predictions when diagnoses are unknown). In addition, contingency planning to address patients’ future needs saw improvements of up to 34 percent.

Last year, NYU Langone added to this long-standing effort a newer form of AI that develops likely options for the next word in any sentence based on how billions of people used language on the internet over time. A result of this next-word prediction is that generative AI chatbots like GPT-4 can read physician notes and make suggestions. In a pilot within the case study, the research team supercharged their machine-learning AI model, which can only give physicians a grade on their notes, by integrating a chatbot that added an accurate written narrative of issues with any note.

The info is here.

The article is linked above.  Here is the abstract:

Abstract

Electronic health records have become an integral part of modern health care, but their implementation has led to unintended consequences, such as poor note quality. This case study explores how NYU Langone Health leveraged artificial intelligence (AI) to address the challenge to improve the content and quality of medical documentation. By quickly and accurately analyzing large volumes of clinical documentation and providing feedback to organizational leadership and individually to providers, AI can help support a culture of continuous note quality improvement, allowing organizations to enhance a critical component of patient care.

On the Ethics of Chatbots in Psychotherapy.

Benosman, M. (2024, January 7).

PsyArXiv Preprints

https://doi.org/10.31234/osf.io/mdq8v

Introduction:

In recent years, the integration of chatbots in mental health care has emerged as a groundbreaking development. These artificial intelligence (AI)-driven tools offer new possibilities for therapy and support, particularly in areas where mental health services are scarce or stigmatized. However, the use of chatbots in this sensitive domain raises significant ethical concerns that must be carefully considered. This essay explores the ethical implications of employing chatbots in mental health, focusing on issues of non-maleficence, beneficence, explicability, and care. Our main ethical question is: should we trust chatbots with our mental health and wellbeing?

Indeed, the recent pandemic has made mental health an urgent global problem. This fact, together with the widespread shortage in qualified human therapists, makes the proposal of chatbot therapists a timely, and perhaps, viable alternative. However, we need to be cautious about hasty implementations of such alternative. For instance, recent news has reported grave incidents involving chatbots-human interactions. For example, (Walker, 2023) reports the death of an eco-anxious man who committed suicide following a prolonged interaction with a chatbot named ELIZA, which encouraged him to put an end to his life to save the planet. Another individual was caught while executing a plan to assassinate the Queen of England, after a chatbot encouraged him to do so (Singleton, Gerken, & McMahon, 2023).

These are only a few recent examples that demonstrate the potential maleficence effect of chatbots on-fragile-individuals. Thus, to be ready to safely deploy such technology, in the context of mental health care, we need to carefully study its potential impact on patients from an ethics standpoint.

The info is here.

Here is my summary:

The article analyzes the ethical considerations around the use of chatbots as mental health therapists, from the perspectives of different stakeholders - bioethicists, therapists, and engineers. It examines four main ethical values:

Non-maleficence: Ensuring chatbots do not cause harm, either accidentally or deliberately. There is agreement that chatbots need rigorous evaluation and regulatory oversight like other medical devices before clinical deployment.

Beneficence: Ensuring chatbots are effective at providing mental health support. There is a need for evidence-based validation of their efficacy, while also considering broader goals like improving quality of life.

Explicability: The need for transparency and accountability around how chatbot algorithms work, so patients can understand the limitations of the technology.

Care: The inability of chatbots to truly empathize, which is a crucial aspect of effective human-based psychotherapy. This raises concerns about preserving patient autonomy and the risk of manipulation.

Overall, the different stakeholders largely agree on the importance of these ethical values, despite coming from different backgrounds. The text notes a surprising level of alignment, even between the more technical engineering perspective and the more humanistic therapist and bioethicist viewpoints. The key challenge seems to be ensuring chatbots can meet the high bar of empathy and care required for effective mental health therapy.

Chatbot therapy is risky. It’s also not useless

A.W. Ohlheiser

vox.com

Originally posted 14 Dec 23

Here is an excerpt:

So what are the risks of chatbot therapy?

There are some obvious concerns here: Privacy is a big one. That includes the handling of the training data used to make generative AI tools better at mimicking therapy as well as the privacy of the users who end up disclosing sensitive medical information to a chatbot while seeking help. There are also the biases built into many of these systems as they stand today, which often reflect and reinforce the larger systemic inequalities that already exist in society.

But the biggest risk of chatbot therapy — whether it’s poorly conceived or provided by software that was not designed for mental health — is that it could hurt people by not providing good support and care. Therapy is more than a chat transcript and a set of suggestions. Honos-Webb, who uses generative AI tools like ChatGPT to organize her thoughts while writing articles on ADHD but not for her practice as a therapist, noted that therapists pick up on a lot of cues and nuances that AI is not prepared to catch.

Stade, in her working paper, notes that while large language models have a “promising” capacity to conduct some of the skills needed for psychotherapy, there’s a difference between “simulating therapy skills” and “implementing them effectively.” She noted specific concerns around how these systems might handle complex cases, including those involving suicidal thoughts, substance abuse, or specific life events.

Honos-Webb gave the example of an older woman who recently developed an eating disorder. One level of treatment might focus specifically on that behavior: If someone isn’t eating, what might help them eat? But a good therapist will pick up on more of that. Over time, that therapist and patient might make the connection between recent life events: Maybe the patient’s husband recently retired. She’s angry because suddenly he’s home all the time, taking up her space.

“So much of therapy is being responsive to emerging context, what you’re seeing, what you’re noticing,” Honos-Webb explained. And the effectiveness of that work is directly tied to the developing relationship between therapist and patient.

The info is here.

Here is my take:

The promise of AI in mental health care dances on a delicate knife's edge. Chatbot therapy, with its alluring accessibility and anonymity, tempts us with a quick fix for the ever-growing burden of mental illness. Yet, as with any powerful tool, its potential can be both a balm and a poison, demanding a wise touch for its ethical wielding.

On the one hand, imagine a world where everyone, regardless of location or circumstance, can find a non-judgmental ear, a gentle guide through the labyrinth of their own minds. Chatbots, tireless and endlessly patient, could offer a first step of support, a bridge to human therapy when needed. In the hushed hours of isolation, they could remind us we're not alone, providing solace and fostering resilience.

But let us not be lulled into a false sense of ease. Technology, however sophisticated, lacks the warmth of human connection, the nuanced understanding of a shared gaze, the empathy that breathes life into words. We must remember that a chatbot can never replace the irreplaceable – the human relationship at the heart of genuine healing.

Therefore, our embrace of chatbot therapy must be tempered with prudence. We must ensure adequate safeguards, preventing them from masquerading as a panacea, neglecting the complex needs of human beings. Transparency is key – users must be aware of the limitations, of the algorithms whispering behind the chatbot's words. Above all, let us never sacrifice the sacred space of therapy for the cold efficiency of code.

Chatbot therapy can be a bridge, a stepping stone, but never the destination. Let us use technology with wisdom, acknowledging its potential good while holding fast to the irreplaceable value of human connection in the intricate tapestry of healing. Only then can we mental health professionals navigate the ethical tightrope and make technology safe and effective, when and where possible.

Are We Ready for AI to Raise the Dead?

Jack Holmes

Esquire Magazine

Originally posted 4 May 24

Here is an excerpt:

You can see wonderful possibilities here. Some might find comfort in hearing their mom’s voice, particularly if she sounds like she really sounded and gives the kind of advice she really gave. But Sandel told me that when he presents the choice to students in his ethics classes, the reaction is split, even as he asks in two different ways. First, he asks whether they’d be interested in the chatbot if their loved one bequeathed it to them upon their death. Then he asks if they’d be interested in building a model of themselves to bequeath to others. Oh, and what if a chatbot is built without input from the person getting resurrected? The notion that someone chose to be represented posthumously in a digital avatar seems important, but even then, what if the model makes mistakes? What if it misrepresents—slanders, even—the dead?

Soon enough, these questions won’t be theoretical, and there is no broad agreement about whom—or even what—to ask. We’re approaching a more fundamental ethical quandary than we often hear about in discussions around AI: human bias embedded in algorithms, privacy and surveillance concerns, mis- and disinformation, cheating and plagiarism, the displacement of jobs, deepfakes. These issues are really all interconnected—Osama bot Laden might make the real guy seem kinda reasonable or just preach jihad to tweens—and they all need to be confronted. We think a lot about the mundane (kids cheating in AP History) and the extreme (some advanced AI extinguishing the human race), but we’re more likely to careen through the messy corridor in between. We need to think about what’s allowed and how we’ll decide.

(cut)

Our governing troubles are compounded by the fact that, while a few firms are leading the way on building these unprecedented machines, the technology will soon become diffuse. More of the codebase for these models is likely to become publicly available, enabling highly talented computer scientists to build their own in the garage. (Some folks at Stanford have already built a ChatGPT imitator for around $600.) What happens when some entrepreneurial types construct a model of a dead person without the family’s permission? (We got something of a preview in April when a German tabloid ran an AI-generated interview with ex–Formula 1 driver Michael Schumacher, who suffered a traumatic brain injury in 2013. His family threatened to sue.) What if it’s an inaccurate portrayal or it suffers from what computer scientists call “hallucinations,” when chatbots spit out wildly false things? We’ve already got revenge porn. What if an old enemy constructs a false version of your dead wife out of spite? “There’s an important tension between open access and safety concerns,” Reich says. “Nuclear fusion has enormous upside potential,” too, he adds, but in some cases, open access to the flesh and bones of AI models could be like “inviting people around the world to play with plutonium.”

The article is here.

Yes, there was a Black Mirror episode (Be Right Back) about this issue.  The wiki is here.