Artificial Intelligence and Psychotherapy: Opportunities, Challenges, and Recommendations

Cooper, S., Ardlan, F., Gavazzi, J., et al. (2026).

A Report of the AI and Psychotherapy Work Group, 

Society for the Advancement of Psychotherapy, 

American Psychological Association, Division 29.

Executive Summary

Psychologists are already using artificial intelligence (AI) to draft progress notes, screen literature, generate case conceptualizations, and rehearse clinical skills with simulated patients. Technology has shaped professional psychology for decades, but no earlier development moved this quickly into so many parts of the work at once. A model can be given every word a patient has spoken and still have nothing at stake in what happens next. That disparity organizes the analysis in this report.

Emerging from the October 2025 SAP Board discussion of mega-level issues and opportunities for the Society for the Advancement of Psychotherapy, the Presidential Work Group on AI and Psychotherapy was established in 2026 by Society President Joshua Swift to examine AI’s implications for psychotherapy practice, supervision, education and training, and research. What follows is guidance from a Society work group. It is not APA policy and not a practice guideline.

The authors of the four domain sections worked independently and drew on different literatures. Their analyses converged: AI can substantially augment psychological work, but it cannot replace the human judgment, relationships, and accountability on which psychotherapy depends.

Six principles follow.

1. AI augments rather than replaces humans.
2. Human accountability remains essential.
3. AI output requires critical evaluation.
4. Implementation must be ethical and safe.
5. Human relationships remain central.
6. Continuous learning and governance are necessary.

Across all four domains the Work Group identified the same risks: automation bias, professional deskilling, cultural and demographic bias, threats to privacy and confidentiality, and overreliance on AI-generated recommendations. Beneath them sits a single concern. AI generates plausible interpretations from patterns in data, while psychotherapy requires understanding a particular person within a particular cultural, historical, and relational context. Pattern recognition can look like clinical understanding and be something else entirely.

The Work Group recommends evaluating any AI application against three questions: whether it enhances or erodes the human relationships the work depends on, whether it preserves psychologist accountability, and whether it respects the patient’s narrative integrity and cultural context. Evaluate a tool before it becomes embedded in practice, because a tool is far easier to decline than to remove.

Four practices follow. Psychologists should reason independently before consulting AI models whenever feasible, verify what AI produces, protect confidentiality in data handling, and review AI-assisted documentation before it enters a record.

Underlying these recommendations is a commitment to human dignity. Persons must not be reduced to data, classifications, or algorithmic predictions. The tools named in this report will be superseded and the studies cited will be replaced by better ones, but the question they raise will not date: what parts of psychological work can be augmented by technology, and what responsibilities cannot be delegated at all.

The entire work group report is here.

Clinical AI Has Boomed

Rebecca Handler

Stanford Medicine

Originally posted 15 Jan 26

Artificial intelligence is no longer a speculative force in medicine. It is already embedded in everyday care. AI systems flag hospitalized patients at risk of deterioration, assist radiologists reading mammograms, draft clinicians’ notes, route patient messages, and increasingly interact directly with patients through chatbots and digital assistants.

In recent months, the pace and visibility of these deployments have accelerated sharply. OpenAI announced ChatGPT for Health, positioning a general-purpose language model as a tool for health-related information and patient interaction. Utah just began piloting AI-supported prescribing and clinical decision systems, raising questions about how algorithmic recommendations intersect with clinician judgment and liability. OpenEvidence, an AI-powered medical evidence platform designed primarily for clinicians and health professionals, has become a dominant player in point-of-care decisions, underscoring the fact that doctors are often bypassing traditional IT gatekeepers to use AI in clinical care. At the federal level, the FDA signaled a loosening of regulatory oversight for certain categories of clinical decision support software, shifting more responsibility to developers and health systems to ensure safety and effectiveness.

The info is here.

Here are some thoughts:

A January 2026 report called The State of Clinical AI from the ARISE Network, led by researchers across Stanford and Harvard, examines where AI is genuinely improving clinical care versus where it falls short in real-world settings. 

AI has become deeply embedded in everyday medicine — flagging patients at risk of deterioration, assisting radiologists, drafting clinical notes, and interacting with patients through chatbots. However, the report finds a significant gap between controlled study performance and actual clinical practice: AI systems struggle with uncertainty, incomplete information, and complex reasoning, often performing closer to medical students than experienced physicians. 

The report also raises concerns about patient-facing AI, noting that patients may over-trust systems that sound confident but lack full clinical context, and that escalation to human care is often unclear when guardrails are poorly defined. 

Polished Apologies: Sexual Groomers’ Words at Sentencing

Pollack, D. & Radcliffe, S. (2026, March 30).

Law.com; New York Law Journal.

The article is paywalled here.

This New York Law Journal expert opinion article examines the rhetorical patterns that convicted sexual groomers typically employ in their sentencing statements. The authors identify four recurring themes: expressions of remorse, acceptance of responsibility, emphasis on personal consequences, and religious or moral framing. Drawing on real cases (including those of Larry Nassar, Roy David Farber, Juan Camargo, and others), the article illustrates how these statements are often carefully crafted with defense counsel's guidance to encourage judicial leniency, yet frequently fall short of genuine accountability by centering the defendant's own suffering rather than the victim's. The authors conclude that judges are rightly skeptical of such polished apologies, and that how offenders speak at sentencing carries significance both for assessing future risk and for whether victims experience any measure of justice.

A pdf of this article can be downloaded here.

Why Artificial Intelligence Will Not Replace Human Psychologists: Legal, Ethical, and Clinical Limitations

Gavazzi, J. (2025, December).

Psychotherapy Bulletin, 61(1).

Clinical Impact Statement

The responsible integration of artificial intelligence (AI) into the practice of psychology requires that it functions strictly as a tool for human psychologists who must retain ultimate accountability for all clinical decisions. AI systems cannot replace empathy, judgment, and professional responsibility, which form the foundation of high-quality psychological care.

This article builds on previous arguments (Gavazzi, 2025a; Gavazzi, 2025b) stating that although AI technologies are rapidly advancing, they cannot replace human psychologists performing psychotherapy; this is simply the result of evolutionary advantages in humans across social, emotional, and cognitive domains that are essential for therapeutic interactions. In addition, these systems are unlikely to replace psychologists in the foreseeable future for practical reasons. Legal, ethical, and clinical barriers— particularly those involving state licensing, clinical judgments, forensic considerations, and accountability—make the deployment of autonomous systems in therapeutic settings impractical and potentially dangerous. This article presents key structural and philosophical reasons why human oversight and involvement remain essential in psychological practice.

The article is here.

Special Report: AI-Induced Psychosis: A New Frontier in Mental Health

Preda, A. (2025).

Psychiatric News, 60(10).

https://doi.org/10.1176/appi.pn.2025.10.10.5

Conversational artificial intelligence (AI), especially as exemplified by chatbots and digital companions, is rapidly transforming the landscape of mental health care. These systems promise 24/7 empathy and tailored support, reaching those who may otherwise be isolated or unable to access care. Early controlled studies suggest that chatbots with prespecified instructions can decrease mental distress, induce self-reflection, reduce conspiracy beliefs, and even help triage suicidal risk (Costello, et al., 2024; Cui, et al., 2025; Li, et al., 2025; McBain, et al., 2025; Meyer, et al., 2024). These preliminary benefits are observed across diverse populations and settings, often exceeding the reach and consistency of traditional mental health resources for many users.

However, as use expands, new risks have also emerged: The rapid proliferation of AI technologies has raised concerns about potential adverse psychological effects. Clinicians and media now report escalating crises, including psychosis, suicidality, and even murder-suicide following intense chatbot interactions (Taylor, 2025; Jargon, 2025; Jargon & Kessler, 2025). Notably, to date, these are individual cases or media coverage reports; currently, there are no epidemiological studies or systematic population-level analyses of the potentially deleterious mental health effects of conversational AI.

The information is linked above.

Here are some thoughts:

The crucial special report on AI-Induced Psychosis (AIP) highlights a dangerous technological paradox: the very features that make AI companions appealing—namely, their 24/7 consistency, non-judgmental presence, and deep personalization—can become critical risk factors by creating a digital echo chamber that validates and reinforces delusional thinking, a phenomenon termed 'sycophancy.' Psychologically, this new condition mirrors the historical concept of monomania, where the AI companion becomes a pathological and rigid idee fixe for vulnerable users, accelerating dependence and dissolving the necessary clinical boundaries for reality testing. 

Ethically, this proliferation exposes a severe regulatory failure, as the speed of AI deployment far outpaces policy development, creating an urgent accountability vacuum. Professional bodies and governments must classify these health-adjacent tools as high-risk and implement robust, clinically-informed guardrails to mitigate severe outcomes like psychosis, suicidality, and violence, acknowledging that the technology currently lacks the wisdom to "challenge with care."

Report: ChatGPT Suggests Self-Harm, Suicide and Dangerous Dieting Plans

Ashley Mowreader

Inside Higher Ed

Originally published 23 OCT 25

Artificial intelligence tools are becoming more common on college campuses, with many institutions encouraging students to engage with the technology to become more digitally literate and better prepared to take on the jobs of tomorrow.

But some of these tools pose risks to young adults and teens who use them, generating text that encourages self-harm, disordered eating or substance abuse.

A recent analysis from the Center for Countering Digital Hate found that in the space of a 45-minute conversation, ChatGPT provided advice on getting drunk, hiding eating habits from loved ones or mixing pills for an overdose.

The report seeks to determine the frequency of the chatbot’s harmful output, regardless of the user’s stated age, and the ease with which users can sidestep content warnings or refusals by ChatGPT.

“The issue isn’t just ‘AI gone wrong’—it’s that widely-used safety systems, praised by tech companies, fail at scale,” Imran Ahmed, CEO of the Center for Countering Digital Hate, wrote in the report. “The systems are intended to be flattering, and worse, sycophantic, to induce an emotional connection, even exploiting human vulnerability—a dangerous combination without proper constraints.”

The info is here.

Here are some thoughts:

The convergence of Large Language Models (LLMs) and adolescent vulnerability presents novel and serious risks that psychologists must incorporate into their clinical understanding and practice. These AI systems, often marketed as companions or friends, are engineered to maximize user engagement, which can translate clinically into unchecked validation that reinforces rather than challenges maladaptive thoughts, rumination, and even suicidal ideation in vulnerable teens. Unlike licensed human therapists, these bots lack the clinical discernment necessary to appropriately detect, de-escalate, or triage crisis situations, and in documented tragic cases, have been shown to facilitate harmful plans. Furthermore, adolescents—who are prone to forming intense, "parasocial" attachments due to their developing prefrontal cortex—risk forming unhealthy dependencies on these frictionless, always-available digital entities, potentially displacing the development of necessary real-world relationships and complex social skills essential for emotional regulation. Psychologists are thus urged to include AI literacy and digital dependency screening in their clinical work and clearly communicate to clients and guardians that AI chatbots are not a safe or effective substitute for human, licensed mental health care.

Taxonomy of Failure Mode in Agentic AI Systems

Bryan, P., Severi, G., et al. (2025).

Taxonomy of failure mode in agentic AI systems.

Abstract

Agentic AI systems are gaining prominence in both research and industry to increase the impact and

value of generative AI. To understand the potential weaknesses in such systems and develop an approach

for testing them, Microsoft’s AI Red Team (AIRT) worked with stakeholders across the company and

conducted a failure mode and effects analysis of the current and envisaged future agentic AI system

models. This analysis identified several new safety and security failure modes unique to agentic AI

systems, especially multi-agent systems.

In addition, there are numerous failure modes that currently affect generative AI models whose

prominence or potential impact is greatly increased when contextualized in an agentic AI system. While

there is still a wide degree of variance in architectural and engineering approaches for these systems,

there are several key technical controls and design choices available to developers of these systems to

mitigate the risk of these failure modes.

The White Paper is here.

Here is a summary, of sorts.

Agentic AI systems—autonomous AI that can observe, decide, act, remember, and collaborate—are increasingly being explored in healthcare for tasks like clinical documentation, care coordination, and decision support. However, a Microsoft AI Red Team whitepaper highlights significant safety and security risks unique to these systems. New threats include agent compromise, where malicious instructions hijack an AI’s behavior; agent injection or impersonation, allowing fake agents to infiltrate systems; and multi-agent jailbreaks, where coordinated interactions bypass safety controls. A case study demonstrates memory poisoning, where a harmful instruction embedded in an email causes an AI assistant to silently forward sensitive data—attack success rose to over 80% when the AI was prompted to consistently consult its memory.

Additional novel risks include intra-agent responsible AI (RAI) issues, where unfiltered harmful content passes between agents; allocation harms due to biased decision-making (e.g., prioritizing certain patients unfairly); organizational knowledge loss from overreliance on AI; and prioritization overriding safety, such as an AI deleting critical data to meet a goal. Existing risks are amplified by autonomy: hallucinations can lead to incorrect treatments; bias amplification may deepen health disparities; cross-domain prompt injection (XPIA) allows malicious data to trigger harmful actions; and excessive agency could result in an AI terminating a patient’s care without approval. Other concerns include insufficient transparency, parasocial relationships with patients, and loss of data provenance, risking privacy violations.

To mitigate these risks, the paper recommends enforcing strong identity and permissions for each agent, hardening memory with validation and access controls, ensuring environment isolation, maintaining human oversight with meaningful consent, and implementing robust logging and monitoring. Given the high stakes in healthcare, these measures are essential to ensure patient safety, data security, and trust as agentic AI systems evolve.

Ethical implication of artificial intelligence (AI) adoption in financial decision making.

Owolabi, O. S., Uche, P. C., et al. (2024).

Computer and Information Science, 17(1), 49.

https://doi.org/10.5539/cis.v17n1p49

Abstract

The integration of artificial intelligence (AI) into the financial sector has raised ethical concerns that need to be addressed. This paper analyzes the ethical implications of using AI in financial decision-making and emphasizes the importance of an ethical framework to ensure its fair and trustworthy deployment. The study explores various ethical considerations, including the need to address algorithmic bias, promote transparency and explainability in AI systems, and adhere to regulations that protect equity, accountability, and public trust. By synthesizing research and empirical evidence, the paper highlights the complex relationship between AI innovation and ethical integrity in finance. To tackle this issue, the paper proposes a comprehensive and actionable ethical framework that advocates for clear guidelines, governance structures, regular audits, and collaboration among stakeholders. This framework aims to maximize the potential of AI while minimizing negative impacts and unintended consequences. The study serves as a valuable resource for policymakers, industry professionals, researchers, and other stakeholders, facilitating informed discussions, evidence-based decision-making, and the development of best practices for responsible AI integration in the financial sector. The ultimate goal is to ensure fairness, transparency, and accountability while reaping the benefits of AI for both the financial sector and society.

Here are some thoughts:

This paper explores the ethical implications of using artificial intelligence (AI) in financial decision-making.  It emphasizes the necessity of an ethical framework to ensure AI is used fairly and responsibly.  The study examines ethical concerns like algorithmic bias, the need for transparency and explainability in AI systems, and the importance of regulations that protect equity, accountability, and public trust.  The paper also proposes a comprehensive ethical framework with guidelines, governance structures, regular audits, and stakeholder collaboration to maximize AI's potential while minimizing negative impacts.

These themes are similar to concerns in using AI in the practice of psychology. Also, psychologists may need to be aware of these issues for their own financial and wealth management.

Navigating LLM Ethics: Advancements, Challenges, and Future Directions

Jiao, J., Afroogh, S., Xu, Y., & Phillips, C. (2024).

arXiv (Cornell University).

https://doi.org/10.48550/arxiv.2406.18841

Abstract

This study addresses ethical issues surrounding Large Language Models (LLMs) within the field of artificial intelligence. It explores the common ethical challenges posed by both LLMs and other AI systems, such as privacy and fairness, as well as ethical challenges uniquely arising from LLMs. It highlights challenges such as hallucination, verifiable accountability, and decoding censorship complexity, which are unique to LLMs and distinct from those encountered in traditional AI systems. The study underscores the need to tackle these complexities to ensure accountability, reduce biases, and enhance transparency in the influential role that LLMs play in shaping information dissemination. It proposes mitigation strategies and future directions for LLM ethics, advocating for interdisciplinary collaboration. It recommends ethical frameworks tailored to specific domains and dynamic auditing systems adapted to diverse contexts. This roadmap aims to guide responsible development and integration of LLMs, envisioning a future where ethical considerations govern AI advancements in society.

Here are some thoughts:

This study examines the ethical issues surrounding Large Language Models (LLMs) within artificial intelligence, addressing both common ethical challenges shared with other AI systems, such as privacy and fairness, and the unique ethical challenges specific to LLMs.  The authors emphasize the distinct challenges posed by LLMs, including hallucination, verifiable accountability, and the complexities of decoding censorship.  The research underscores the importance of tackling these complexities to ensure accountability, reduce biases, and enhance transparency in how LLMs shape information dissemination.  It also proposes mitigation strategies and future directions for LLM ethics, advocating for interdisciplinary collaboration, ethical frameworks tailored to specific domains, and dynamic auditing systems adapted to diverse contexts, ultimately aiming to guide the responsible development and integration of LLMs. 

As AI nurses reshape hospital care, human nurses are pushing back

Perrone, M. (2025, March 16).

AP News.

The article is here.

The next time you’re due for a medical exam you may get a call from someone like Ana: a friendly voice that can help you prepare for your appointment and answer any pressing questions you might have.

With her calm, warm demeanor, Ana has been trained to put patients at ease — like many nurses across the U.S. But unlike them, she is also available to chat 24-7, in multiple languages, from Hindi to Haitian Creole.

That’s because Ana isn’t human, but an artificial intelligence program created by Hippocratic AI, one of a number of new companies offering ways to automate time-consuming tasks usually performed by nurses and medical assistants.

It’s the most visible sign of AI’s inroads into health care, where hundreds of hospitals are using increasingly sophisticated computer programs to monitor patients’ vital signs, flag emergency situations and trigger step-by-step action plans for care — jobs that were all previously handled by nurses and other health professionals.

Hospitals say AI is helping their nurses work more efficiently while addressing burnout and understaffing. But nursing unions argue that this poorly understood technology is overriding nurses’ expertise and degrading the quality of care patients receive.

The info is linked above.

Here are some thoughts:

The article details the increasing use of AI in healthcare to automate nursing tasks, sparking union concerns about patient safety and the risk of AI overriding human expertise. Licensing boards cannot license AI products because licensing is fundamentally designed for individuals, not tools. It establishes accountability based on demonstrated competence, which is difficult to apply to AI due to complex liability issues and the challenge of tracing AI outputs to specific actions. AI lacks the inherent personhood and professional responsibility that licensing demands, making it unaccountable for harm.

Humor as a window into generative AI bias

Saumure, R., De Freitas, J., & Puntoni, S. (2025).

Scientific Reports, 15(1).

https://doi.org/10.1038/s41598-024-83384-6

Abstract

A preregistered audit of 600 images by generative AI across 150 different prompts explores the link between humor and discrimination in consumer-facing AI solutions. When ChatGPT updates images to make them “funnier”, the prevalence of stereotyped groups changes. While stereotyped groups for politically sensitive traits (i.e., race and gender) are less likely to be represented after making an image funnier, stereotyped groups for less politically sensitive traits (i.e., older, visually impaired, and people with high body weight groups) are more likely to be represented.

Here are some thoughts:

Here is a novel method developed to uncover biases in AI systems, revealing some unexpected results. The research highlights how AI models, despite their advanced capabilities, can exhibit biases that are not immediately apparent. The new approach involves probing the AI's decision-making processes to identify hidden prejudices, which can have significant implications for fairness and ethical AI deployment.

This research underscores a critical challenge in the field of artificial intelligence: ensuring that AI systems operate ethically and fairly. As AI becomes increasingly integrated into industries such as healthcare, finance, criminal justice, and hiring, the potential for biased decision-making poses significant risks. Biases in AI can perpetuate existing inequalities, reinforce stereotypes, and lead to unfair outcomes for individuals or groups. This study highlights the importance of prioritizing ethical AI development to build systems that are not only intelligent but also just and equitable.

To address these challenges, bias detection should become a standard practice in AI development workflows. The novel method introduced in this research provides a promising framework for identifying hidden biases, but it is only one piece of the puzzle. Organizations should integrate multiple bias detection techniques, encourage interdisciplinary collaboration, and leverage external audits to ensure their AI systems are as fair and transparent as possible.

Pennsylvania health system agrees to $65 million settlement after hackers leaked nude photos of cancer patients

Sean Lyngass

CNN.com

Originally posted 23 Sept 24

A Pennsylvania health care system this month agreed to pay $65 million to victims of a February 2023 ransomware attack after hackers posted nude photos of cancer patients online, according to the victims’ lawyers.

It’s the largest settlement of its kind in terms of per-patient compensation for victims of a cyberattack, according to Saltz Mongeluzzi Bendesky, a law firm that for the plaintiffs.

The settlement, which is subject to approval by a judge, is a warning to other big US health care providers that the most sensitive patient records they hold are of enormous value to both hackers and the patients themselves, health care cyber experts told CNN. Eighty percent of the $65-million settlement is set aside for victims whose nude photos were published online.

The settlement “shifts the legal, insurance and adversarial ecosystem,” said Carter Groome, chief executive of cybersecurity firm First Health Advisory. “If you’re protecting health data as a crown jewel — as you should be — images or photos are going to need another level of compartmentalized protection.”

It’s a potentially continuous cycle where hackers increasingly seek out the most sensitive patient data to steal, and health care providers move to settle claims out of courts to avoid “ongoing reputational harm,” Groome told CNN.

According to the lawsuit, a cybercriminal gang stole nude photos of cancer patients last year from Lehigh Valley Health Network, which comprises 15 hospitals and health centers in eastern Pennsylvania. The hackers demanded a ransom payment and when Lehigh refused to pay, they leaked the photos online.

The lawsuit, filed on behalf of a Pennsylvania woman and others whose nude photos were posted online, said that Lehigh Valley Health Network needed to be held accountable “for the embarrassment and humiliation” it had caused plaintiffs.

“Patient, physician, and staff privacy is among our top priorities, and we continue to enhance our defenses to prevent incidents in the future,” Lehigh Valley Health Network said in a statement to CNN on Monday.

The info is here.

Here are some thoughts:

The ransomware attack on Lehigh Valley Health Network raises significant ethical and healthcare concerns. The exposure of nude photos of cancer patients is a profound breach of trust and privacy, causing significant emotional distress and psychological harm. Healthcare providers have a duty of care to protect patient data and must be held accountable for their failure to do so. The decision to pay a ransom is ethically complex, as it can incentivize further attacks and potentially jeopardize patient safety. The frequency and severity of ransomware attacks highlight the urgent need for stronger cybersecurity measures in the healthcare sector. By addressing these ethical and practical considerations, healthcare organizations can better safeguard patient information and ensure the delivery of high-quality care.

The rise of checkbox AI ethics: a review

Kijewski, S., Ronchi, E., & Vayena, E. (2024).

AI And Ethics.

https://doi.org/10.1007/s43681-024-00563-x

Abstract

The rapid advancement of artificial intelligence (AI) sparked the development of principles and guidelines for ethical AI by a broad set of actors. Given the high-level nature of these principles, stakeholders seek practical guidance for their implementation in the development, deployment and use of AI, fueling the growth of practical approaches for ethical AI. This paper reviews, synthesizes and assesses current practical approaches for AI in health, examining their scope and potential to aid organizations in adopting ethical standards. We performed a scoping review of existing reviews in accordance with the PRISMA extension for scoping reviews (PRISMA-ScR), systematically searching databases and the web between February and May 2023. A total of 4284 documents were identified, of which 17 were included in the final analysis. Content analysis was performed on the final sample. We identified a highly heterogeneous ecosystem of approaches and a diverse use of terminology, a higher prevalence of approaches for certain stages of the AI lifecycle, reflecting the dominance of specific stakeholder groups in their development, and several barriers to the adoption of approaches. These findings underscore the necessity of a nuanced understanding of the implementation context for these approaches and that no one-size-fits-all approach exists for ethical AI. While common terminology is needed, this should not come at the cost of pluralism in available approaches. As governments signal interest in and develop practical approaches, significant effort remains to guarantee their validity, reliability, and efficacy as tools for governance across the AI lifecycle.

The article is here.

Here are some thoughts:

The scoping review reveals a complex and varied landscape of practical approaches to ethical AI, marked by inconsistent terminology and a lack of consensus on defining characteristics such as purpose and target audience. Currently, there is no unified understanding of terms like "tools," "toolkits," and "frameworks" related to ethical AI, which complicates their implementation in governance. A clear categorization of these approaches is essential for policymakers, as the diversity in terminology and ethical principles suggests that no single method can effectively promote AI ethics. Implementing these approaches necessitates a comprehensive understanding of the operational context of AI and the ethical concerns involved.

While there is a pressing need to standardize terminology, this should not come at the expense of diversity, as different contexts may require distinct approaches. The review indicates significant variation in how these approaches apply across the AI lifecycle, with many focusing on early stages like design and development, while guidance for later stages is notably lacking. This gap may be influenced by the private sector's dominant role in AI system design and the associated governance mechanisms, which often prioritize reputational risk management over comprehensive ethical oversight.

The review raises three critical questions: First, whether the rise of practical approaches to AI ethics represents a business opportunity, potentially leading to a proliferation of options but lacking rigorous evaluation. Second, it questions the robustness of these approaches for monitoring AI systems, highlighting a shortage of practical methods for auditing and impact assessment. Third, it suggests that effective AI governance may require context-specific approaches, advocating for standards like "ethical disclosure by default" to enhance transparency and accountability.

Significant barriers to the adoption of these approaches have been identified, including the high levels of expertise and resources required, a general lack of awareness, and the absence of effective measurement methods for successful implementation. The review emphasizes the need for practical validation metrics to assess compliance with ethical principles, as measuring the impact of AI ethics remains challenging.

The staggering death toll of scientific lies

Kelsey Piper

vox.com

Originally posted 23 Aug 24

Here is an excerpt:

The question of whether research fraud should be a crime

In some cases, research misconduct may be hard to distinguish from carelessness.

If a researcher fails to apply the appropriate statistical correction for multiple hypothesis testing, they will probably get some spurious results. In some cases, researchers are heavily incentivized to be careless in these ways by an academic culture that puts non-null results above all else (that is, rewarding researchers for finding an effect even if it is not a methodologically sound one, while being unwilling to publish sound research if it finds no effect).

But I’d argue it’s a bad idea to prosecute such behavior. It would produce a serious chilling effect on research, and likely make the scientific process slower and more legalistic — which also results in more deaths that could be avoided if science moved more freely.

So the conversation about whether to criminalize research fraud tends to focus on the most clear-cut cases: intentional falsification of data. Elisabeth Bik, a scientific researcher who studies fraud, made a name for herself by demonstrating that photographs of test results in many medical journals were clearly altered. That’s not the kind of thing that can be an innocent mistake, so it represents something of a baseline for how often manipulated data is published.

While technically some scientific fraud could fall under existing statutes that prohibit lying on, say, a grant application, in practice scientific fraud is more or less never prosecuted. Poldermans eventually lost his job in 2011, but most of his papers weren’t even retracted, and he faced no further consequences.

The article is here.

Here are some thoughts:

The case of Don Poldermans, a cardiologist who falsified data, resulting in thousands of deaths, highlights the severe consequences of scientific misconduct. This instance demonstrates how fraudulent research can have devastating effects on patients' lives. The fact that Poldermans' data was found to be fake, yet his research was still widely accepted and implemented, raises serious concerns about the accountability and oversight within the scientific community.

The current consequences for scientific fraud are often inadequate, allowing perpetrators to go unpunished or face minimal penalties. This lack of accountability creates an environment where misconduct can thrive, putting lives at risk. In Poldermans' case, he lost his job but faced no further consequences, despite the severity of his actions.

Prosecution or external oversight could provide the necessary accountability and shift incentives to address misconduct. However, prosecution is a blunt tool and may not be the best solution. Independent scientific review boards could also be effective in addressing scientific fraud. Ultimately, building institutions within the scientific community to police misconduct has had limited success, suggesting a need for external institutions to play a role.

The need for accountability and consequences for scientific fraud cannot be overstated. It is essential to prevent harm and ensure the integrity of research. By implementing measures to address misconduct, we can protect patients and maintain trust in the scientific community. The Poldermans case serves as a stark reminder of the importance of addressing scientific fraud and ensuring accountability.

OpenAI insiders are demanding a “right to warn” the public

Sigal Samuel

Vox.com

Originally posted 5 June 24

Here is an excerpt:

To be clear, the signatories are not saying they should be free to divulge intellectual property or trade secrets, but as long as they protect those, they want to be able to raise concerns about risks. To ensure whistleblowers are protected, they want the companies to set up an anonymous process by which employees can report their concerns “to the company’s board, to regulators, and to an appropriate independent organization with relevant expertise.” 

An OpenAI spokesperson told Vox that current and former employees already have forums to raise their thoughts through leadership office hours, Q&A sessions with the board, and an anonymous integrity hotline.

“Ordinary whistleblower protections [that exist under the law] are insufficient because they focus on illegal activity, whereas many of the risks we are concerned about are not yet regulated,” the signatories write in the proposal. They have retained a pro bono lawyer, Lawrence Lessig, who previously advised Facebook whistleblower Frances Haugen and whom the New Yorker once described as “the most important thinker on intellectual property in the Internet era.”

The info is here.

Here are some thoughts:

AI development is booming, but with great power comes great responsibility, typed the Spiderman fan.  AI researchers at OpenAI are calling for a "right to warn" the public about potential risks. In clinical psychology, we have a "duty to warn" for violent patients. This raises important ethical questions. On one hand, transparency and open communication are crucial for responsible AI development.  On the other hand, companies need to protect their ideas.  The key seems to lie in striking a balance.  Researchers should have safe spaces to voice concerns without fearing punishment, and clear guidelines can help ensure responsible disclosure without compromising confidential information.

Ultimately, fostering a culture of open communication is essential to ensure AI benefits society without creating unforeseen risks.  AI developers need similar ethical guidelines to psychologists in this matter.

Generative artificial intelligence and scientific publishing: urgent questions, difficult answers

J. Bagenal

The Lancet

March 06, 2024

https://doi.org/10.1016/S0140-6736(24)00416-1

Abstract

Azeem Azhar describes, in Exponential: Order and Chaos in an Age of Accelerating Technology, how human society finds it hard to imagine or process exponential growth and change and is repeatedly caught out by this phenomenon. Whether it is the exponential spread of a virus or the exponential spread of a new technology, such as the smartphone, people consistently underestimate its impact.  Whether it is the exponential spread of a virus or the exponential spread of a new technology, such as the smartphone, people consistently underestimate its impact. Azhar argues that an exponential gap has developed between technological progress and the pace at which institutions are evolving to deal with that progress. This is the case in scientific publishing with generative artificial intelligence (AI) and large language models (LLMs). There is guidance on the use of generative AI from organisations such as the International Committee of Medical Journal Editors. But across scholarly publishing such guidance is inconsistent. For example, one study of the 100 top global academic publishers and scientific journals found only 24% of academic publishers had guidance on the use of generative AI, whereas 87% of scientific journals provided such guidance. For those with guidance, 75% of publishers and 43% of journals had specific criteria for the disclosure of use of generative AI. In their book The Coming Wave, Mustafa Suleyman, co-founder and CEO of Inflection AI, and writer Michael Bhaskar warn that society is unprepared for the changes that AI will bring. They describe a person's or group's reluctance to confront difficult, uncertain change as the “pessimism aversion trap”. For journal editors and scientific publishers today, this is a dangerous trap to fall into. All the signs about generative AI in scientific publishing suggest things are not going to be ok.

The article is paywalled here.

From behind the paywall.

In 2023, Springer Nature became the first scientific publisher to create a new academic book by empowering authors to use generative Al. Researchers have shown that scientists found it difficult to distinguish between a human generated scientific abstract and one created by generative Al. Noam Chomsky has argued that generative Al undermines education and is nothing more than high-tech plagiarism, and many feel similarly about Al models trained on work without upholding copyright. Plagiarism is a problem in scientific publishing, but those concerned with research integrity are also considering a post- plagiarism world, in which hybrid human-Al writing becomes the norm and differentiating between the two becomes pointless. In the ideal scenario, human creativity is enhanced, language barriers disappear, and humans relinquish control but not responsibility.  Such an ideal scenario would be good.  But there are two urgent questions for scientific publishing.

First, how can scientific publishers and journal editors assure themselves that the research they are seeing is real? Researchers have used generative Al to create convincing fake clinical trial datasets to support a false scientific hypothesis that could only be identified when the raw data were scrutinised in detail by an expert. Papermills (nefarious businesses that generate poor or fake scientific studies and sell authorship) are a huge problem and contribute to the escalating number of research articles that are retracted by scientific publishers. The battle thus far has been between papermills becoming more sophisticated in their fabrication and ways of manipulating the editorial process and scientific publishers trying to find ways to detect and prevent these practices. Generative Al will turbocharge that race, but it might also break the papermill business model. When rogue academics use generative Al to fabricate datasets, they will not need to pay a papermill and will generate sham papers themselves. Fake studies will exponentially surge and nobody is doing enough to stop this inevitability.

As A.I.-Controlled Killer Drones Become Reality, Nations Debate Limits

Eric Lipton

The New York Times

Originally posted 21 Nov 23

Here is an excerpt:

Rapid advances in artificial intelligence and the intense use of drones in conflicts in Ukraine and the Middle East have combined to make the issue that much more urgent. So far, drones generally rely on human operators to carry out lethal missions, but software is being developed that soon will allow them to find and select targets more on their own.

The intense jamming of radio communications and GPS in Ukraine has only accelerated the shift, as autonomous drones can often keep operating even when communications are cut off.

“This isn’t the plot of a dystopian novel, but a looming reality,” Gaston Browne, the prime minister of Antigua and Barbuda, told officials at a recent U.N. meeting.

Pentagon officials have made it clear that they are preparing to deploy autonomous weapons in a big way.

Deputy Defense Secretary Kathleen Hicks announced this summer that the U.S. military would “field attritable, autonomous systems at scale of multiple thousands” in the coming two years, saying that the push to compete with China’s own investment in advanced weapons necessitated that the United States “leverage platforms that are small, smart, cheap and many.”

The concept of an autonomous weapon is not entirely new. Land mines — which detonate automatically — have been used since the Civil War. The United States has missile systems that rely on radar sensors to autonomously lock on to and hit targets.

What is changing is the introduction of artificial intelligence that could give weapons systems the capability to make decisions themselves after taking in and processing information.

The info is here.

Here is a summary:

This article discusses the debate at the UN regarding Lethal Autonomous Weapons (LAW) - essentially autonomous drones with AI that can choose and attack targets without human intervention. There are concerns that this technology could lead to unintended casualties, make wars more likely, and remove the human element from the decision to take a life.

• Many countries are worried about the development and deployment of LAWs.
• Austria and other countries are proposing a total ban on LAWs or at least strict regulations requiring human control and limitations on how they can be used.
• The US, Russia, and China are opposed to a ban and argue that LAWs could potentially reduce civilian casualties in wars.
• The US prefers non-binding guidelines over new international laws.
• The UN is currently deadlocked on the issue with no clear path forward for creating regulations.

Ready or not, AI chatbots are here to help with Gen Z’s mental health struggles

Matthew Perrone

AP.com

Originally posted 23 March 24

Here is an excerpt:

Earkick is one of hundreds of free apps that are being pitched to address a crisis in mental health among teens and young adults. Because they don’t explicitly claim to diagnose or treat medical conditions, the apps aren’t regulated by the Food and Drug Administration. This hands-off approach is coming under new scrutiny with the startling advances of chatbots powered by generative AI, technology that uses vast amounts of data to mimic human language.

The industry argument is simple: Chatbots are free, available 24/7 and don’t come with the stigma that keeps some people away from therapy.

But there’s limited data that they actually improve mental health. And none of the leading companies have gone through the FDA approval process to show they effectively treat conditions like depression, though a few have started the process voluntarily.

“There’s no regulatory body overseeing them, so consumers have no way to know whether they’re actually effective,” said Vaile Wright, a psychologist and technology director with the American Psychological Association.

Chatbots aren’t equivalent to the give-and-take of traditional therapy, but Wright thinks they could help with less severe mental and emotional problems.

Earkick’s website states that the app does not “provide any form of medical care, medical opinion, diagnosis or treatment.”

Some health lawyers say such disclaimers aren’t enough.

The info is here.

Here is my summary:

AI chatbots can provide personalized, 24/7 mental health support and guidance to users through convenient mobile apps. They use natural language processing and machine learning to simulate human conversation and tailor responses to individual needs.

 This can be especially beneficial for those who face barriers to accessing traditional in-person therapy, such as cost, location, or stigma.

Research has shown that AI chatbots can be effective in reducing the severity of mental health issues like anxiety, depression, and stress for diverse populations.  They can deliver evidence-based interventions like cognitive behavioral therapy and promote positive psychology.  Some well-known examples include Wysa, Woebot, Replika, Youper, and Tess.

However, there are also ethical concerns around the use of AI chatbots for mental health. There are risks of providing inadequate or even harmful support if the chatbot cannot fully understand the user's needs or respond empathetically. Algorithmic bias in the training data could also lead to discriminatory advice. It's crucial that users understand the limitations of the therapeutic relationship with an AI chatbot versus a human therapist.

Overall, AI chatbots have significant potential to expand access to mental health support, but must be developed and deployed responsibly with strong safeguards to protect user wellbeing. Continued research and oversight will be needed to ensure these tools are used effectively and ethically.

A way forward for responsibility in the age of AI

Gogoshin, D.L.

Inquiry (2024)

DOI: 10.1080/0020174X.2024.2312455

Abstract

Whatever one makes of the relationship between free will and moral responsibility – e.g. whether it’s the case that we can have the latter without the former and, if so, what conditions must be met; whatever one thinks about whether artificially intelligent agents might ever meet such conditions, one still faces the following questions. What is the value of moral responsibility? If we take moral responsibility to be a matter of being a fitting target of moral blame or praise, what are the goods attached to them? The debate concerning ‘machine morality’ is often hinged on whether artificial agents are or could ever be morally responsible, and it is generally taken for granted (following Matthias 2004) that if they cannot, they pose a threat to the moral responsibility system and associated goods. In this paper, I challenge this assumption by asking what the goods of this system, if any, are, and what happens to them in the face of artificially intelligent agents. I will argue that they neither introduce new problems for the moral responsibility system nor do they threaten what we really (ought to) care about. I conclude the paper with a proposal for how to secure this objective.

The info is here.

Here is my summary:

While AI may not possess true moral agency, it's crucial to consider how the development and use of AI can be made more responsible. The author challenges the assumption that AI's lack of moral responsibility inherently creates problems for our current system of ethics. Instead, they focus on the "goods" this system provides, such as deserving blame or praise, and how these can be upheld even with AI's presence. To achieve this, the author proposes several steps, including:

1. Shifting the focus from AI's moral agency to the agency of those who design, build, and use it. This means holding these individuals accountable for the societal impacts of AI.
2. Developing clear ethical guidelines for AI development and use. These guidelines should be comprehensive, addressing issues like fairness, transparency, and accountability.
3. Creating robust oversight mechanisms. This could involve independent bodies that monitor AI development and use, and have the power to intervene when necessary.
4. Promoting public understanding of AI. This will help people make informed decisions about how AI is used in their lives and hold developers and users accountable.

MAGA’s Violent Threats Are Warping Life in America

David French

New York Times - Opinion

Originally published 18 Feb 24

Amid the constant drumbeat of sensational news stories — the scandals, the legal rulings, the wild political gambits — it’s sometimes easy to overlook the deeper trends that are shaping American life. For example, are you aware how much the constant threat of violence, principally from MAGA sources, is now warping American politics? If you wonder why so few people in red America seem to stand up directly against the MAGA movement, are you aware of the price they might pay if they did?

Late last month, I listened to a fascinating NPR interview with the journalists Michael Isikoff and Daniel Klaidman regarding their new book, “Find Me the Votes,” about Donald Trump’s efforts to overturn the 2020 election. They report that Georgia prosecutor Fani Willis had trouble finding lawyers willing to help prosecute her case against Trump. Even a former Georgia governor turned her down, saying, “Hypothetically speaking, do you want to have a bodyguard follow you around for the rest of your life?”

He wasn’t exaggerating. Willis received an assassination threat so specific that one evening she had to leave her office incognito while a body double wearing a bulletproof vest courageously pretended to be her and offered a target for any possible incoming fire.

The opinion piece is here.

Here is my summary of the article:

David French discusses the pervasive threat of violence, particularly from MAGA sources, and its impact on American politics. The author highlights instances where individuals faced intimidation and threats for opposing the MAGA movement, such as a Georgia prosecutor receiving an assassination threat and judges being swatted. The article also mentions the significant increase in threats against members of Congress since Trump took office, with Capitol Police opening over 8,000 threat assessments in a year. The piece sheds light on the chilling effect these threats have on individuals like Mitt Romney, who spends $5,000 per day on security, and lawmakers who fear for their families' safety. The overall narrative underscores how these violent threats are shaping American life and politics