Educational Strategies for Clinical Supervision of Artificial Intelligence Use

Abdulnour, R. E., Gin, B., & Boscardin, C. K. (2025).

New England Journal of Medicine, 393(8), 786–797.

https://doi.org/10.1056/nejmra2503232

Human–computer interactions have been occurring for decades, but recent technological developments in medical artificial intelligence (AI) have resulted in more effective and potentially more dangerous interactions. Although the hype around AI resonates with previous technological revolutions, such as the development of the Internet and the electronic health record, the appearance of large language models (LLMs) seems different. LLMs can simulate knowledge generation and clinical reasoning with humanlike fluency, which gives them the appearance of agency and independent information processing. Therefore, AI has the capacity to fundamentally alter medical learning and practice. As in other professions, the use of AI in medical training could result in professionals who are highly efficient yet less capable of independent problem solving and critical evaluation than their pre-AI counterparts.

Such a challenge presents educational opportunities and risks. AI can enhance simulation-based learning, knowledge recall, and just-in-time feedback and can be used for cognitive off-loading of rote tasks. With cognitive off-loading, learners rely on AI to reduce the load on their working memory, a strategy that facilitates mental engagement with more-demanding tasks. However, off-loading of complex tasks, such as clinical reasoning and decision making, can potentially lead to automation bias (overreliance on automated systems and risk of error), “deskilling” (loss of previously acquired skills), “never-skilling” (failure to develop essential competencies), and “mis-skilling” (reinforcement of incorrect behavior due to AI errors or bias). These risks are especially troubling because LLMs operate as unpredictable black boxes; they generate probabilistic responses with low reasoning transparency, which limits assessment of their reliability. For example, in one study, more than a third of advanced medical students missed erroneous LLM answers to clinical scenarios.

The article is paywalled, unfortunately.

Here is my brief summary: 

This article addresses the challenge of supervising medical trainees who use artificial intelligence (AI) tools, particularly large language models (LLMs), in clinical reasoning. It notes that while AI can enhance learning through simulation, knowledge recall, and cognitive off-loading of rote tasks, off-loading complex tasks like clinical reasoning risks automation bias, "deskilling," "never-skilling," and "mis-skilling." LLMs are unpredictable black boxes with low reasoning transparency, making their reliability hard to assess.

The authors argue that critical thinking is foundational to adaptive practice in the age of AI. They propose the DEFT-AI framework (Diagnosis, Evidence, Feedback, Teaching, and recommendation for AI engagement) as a structured approach for educators to promote critical thinking during learner-AI interactions. The framework guides educators to probe the learner's clinical reasoning and AI use, evaluate supporting and opposing evidence, foster self-reflection, provide targeted teaching, and recommend safe AI engagement.

The article also describes two human-AI collaboration behaviors: centaur (strategic division of tasks, with human judgment leading) and cyborg (tight intertwining of user and AI throughout a task). Adaptive AI practice requires shifting between these modes based on task complexity and risk. The authors emphasize promoting AI literacy through evidence-based evaluation of AI tools and outputs, effective prompt engineering, and a "verify and trust" paradigm, concluding that AI interactions are here to stay and must be met with critical thinking and structured educational strategies.

Auditing the AI Auditors: A Framework for Evaluating Fairness and Bias in High Stakes AI Predictive Models

Landers, R. N., & Behrend, T. S. (2023).

American Psychologist, 78(1), 36–49.

https://doi.org/10.1037/amp0000972

Abstract

Researchers, governments, ethics watchdogs, and the public are increasingly voicing concerns about unfairness and bias in artificial intelligence (AI)-based decision tools. Psychology’s more-than-a-century of research on the measurement of psychological traits and the prediction of human behavior can benefit such conversations, yet psychological researchers often find themselves excluded due to mismatches in terminology, values, and goals across disciplines. In the present paper, we begin to build a shared interdisciplinary understanding of AI fairness and bias by first presenting three major lenses, which vary in focus and prototypicality by discipline, from which to consider relevant issues: (a) individual attitudes, (b) legality, ethicality, and morality, and (c) embedded meanings within technical domains. Using these lenses, we next present psychological audits as a standardized approach for evaluating the fairness and bias of AI systems that make predictions about humans across disciplinary perspectives. We present 12 crucial components to audits across three categories: (a) components related to AI models in terms of their source data, design, development, features, processes, and outputs, (b) components related to how information about models and their applications are presented, discussed, and understood from the perspectives of those employing the algorithm, those affected by decisions made using its predictions, and third-party observers, and (c) meta-components that must be considered across all other auditing components, including cultural context, respect for persons, and the integrity of individual research designs used to support all model developer claims.

Public Significance Statement

Although artificial intelligence (AI) is now being used to make decisions about people’s employment, education, healthcare, and experiences with law enforcement, external evaluators do not often agree on what is necessary to show that an AI is “unbiased” or “fair.” This is in part because “bias” and “fairness” mean different things to different people. We created a framework for auditing that respects these differences in pursuit of better, fairer AI.

Practical Frameworks for Integrating Artificial Intelligence in Forensic Psychology

Rilen, S. (2026).

Journal of Forensic Psychology Research

and Practice, 1–39.

https://doi.org/10.1080/24732850.2026.2630383

The article is paywalled. Please contact the author for a copy of this article.

Abstract

The American Psychological Association (APA) provides broad guidance for generative Artificial Intelligence (gAI) use, yet forensic practice requires a specialized context for adversarial settings. This article proposes a four-step operational framework for ethical gAI integration grounded in forensic admissibility. It establishes operational mandates and readiness assessments followed by a risk-stratification model for tool selection distinguishing convergent from divergent tasks. It differentiates substantive from peripheral uses to guide disclosure decisions, and addresses admissibility considerations under Daubert and Frye. Sample disclosure language and visual aids help experts preserve credibility through methodological verification and systematic procedural rigor, rather than algorithmic trust.

Here are some thoughts:

The article establishes that while generative AI offers significant potential to enhance forensic psychology, its integration must be governed by a strict, four-step framework that prioritizes practitioner accountability, methodological transparency, and data security. Central to this is the principle of non-delegation of cognition, which mandates that forensic experts retain full ownership of all analytical reasoning and conclusions, using gAI only as an augmentative tool rather than a substitute for professional judgment. All gAI output must be treated with a zero-trust verification approach, requiring independent validation against primary sources to guard against hallucinations, omissions, and algorithmic bias. Furthermore, the framework draws a critical distinction between substantive use—which materially influences opinions and requires proactive disclosure in court reports—and peripheral use, which involves administrative support and only requires informed consent. Given the adversarial legal context, practitioners must demonstrate technical competence to explain gAI's probabilistic nature and limitations, ensuring their methodology can withstand scrutiny under Daubert or Frye standards. Data privacy is paramount, necessitating enterprise-grade tools with robust security pillars, including data quarantine and encryption, to preserve legal privilege. Ultimately, the article emphasizes that forensic practitioners are the gatekeepers of their methodology, and by embracing rigor, transparency, and continuous self-assessment, they can harness gAI's benefits while safeguarding the integrity of the justice system.

An AI Perspective on Counseling Supervision

Brinck, E. A., et al. (2026).

Behavioral Sciences, 16(6), Article 1038. 

https://doi.org/10.3390/bs16061038

Abstract

The increased use of technology-assisted distance counseling practices is one result of COVID’s impact on behavioral health, including in counselor education and the delivery of supervision. First, technology-assisted distance supervision needed for “real time” communication grew. Furthermore, there is an emergence of artificial intelligence (AI) technologies that have the potential to contribute to aspects of supervision; however, current evidence remains emerging, context-dependent, and at times mixed, warranting cautious interpretation of their effectiveness. The article offers an overview of using AI in clinical supervision, examines the benefits and potential concerns of AI from different perspectives, and considers the significance of using AI in counseling supervision. The role of AI is discussed as applied to counseling supervision including the use of AI tools, such as chatbots and reasoning AI, to detect and track sessions, note behavioral and emotional cues, aid/monitor communication and feedback, while also attending to ethical and legal consideration for its use. The article will report a range of benefits for supervisors and trainees using AI—for example, by enhancing data-driven supervision decisions, analyzing feedback trends, providing more efficient administrative monitoring, flexible/remote support, skill development, and promoting ethical decisions and self-reflection. Special attention is given to the challenges of using AI in supervision, including risks of undervaluing intuition and qualitative insights, potential for algorithms to reinforce systemic biases, risks of replacing human interaction, as well as non-compliance with HIPAA, FERPA, and ethical guidelines in data storage and privacy. The article will discuss privacy concerns, depersonalized feedback, and increased judgment-driven anxiety despite needed empathy when using AI as a tool for clinical supervision. Recommendations will also be offered for effective, ethical integration of AI in counseling supervision.

Here are some thoughts:

This is a comprehensive review, and its most useful feature for a clinical audience is how thoroughly it maps existing professional guidance onto the specific problem of AI-assisted supervision. The authors walk through the NBCC's 2024 clinical tenets and the ACA's 13 recommendations in detail, which gives the piece real practical value as a reference document, something a supervisor could actually consult when drafting an informed consent form or a supervision contract that addresses AI use. The case example of Lexi and Jalen, where AI-flagged session notes surfaced a pattern of the trainee talking over a client's frustration, is a nice illustration of AI functioning as intended: not replacing supervisory judgment but generating material for the supervisor to process relationally with the supervisee.

The empirical base here is thinner than the framework suggests, and the authors are reasonably candid about this.

The most clinically substantive contribution is the discussion of the supervisory working alliance (SWA) as the issue actually at risk. The authors argue that AI-mediated feedback, delivered without the moderating effect of a trusted relationship, risks landing as more punitive or deficit-focused than the same content delivered by a supervisor who knows the trainee's developmental stage and history. That's a sharper articulation than most AI-and-supervision pieces manage, and it connects naturally to the Integrated Developmental Model framework they invoke early on: a supervisee's readiness to receive AI-flagged feedback without a supervisor's relational buffering plausibly varies by developmental level in ways this piece gestures at but doesn't fully develop.

Where I'd flag some limitation: the literature search was explicitly broad and non-systematic ("AI in human services" across a handful of databases), which the authors acknowledge as a scope limitation given how new this area is. And several of the ethical and legal considerations, informed consent language, data ownership, HIPAA and FERPA compliance, are more thorough in their treatment of what should happen than in documenting what is currently happening in supervision practice. The piece reads as a strong, well-organized synthesis and practical guide rather than an evidence base, which is an entirely fair thing for a review in this stage of the literature to be, but worth naming for readers expecting empirical weight.

Benchmarking Large Language Models Against Psychiatry Residents Using Traditional Institutional Assessments

Sethi, M. I. S. et al. (2026).

Indian Journal of Psychological Medicine, 

02537176261435658.

https://doi.org/10.1177/02537176261435658

Background:Artificial intelligence (AI) models demonstrate remarkable capabilities in healthcare applications, yet their performance compared to medical trainees in psychiatric education remains unexplored. This study evaluated the comparative performance of large language models (LLMs) against first-year psychiatry residents in standardized assessments at a premier Indian medical educational institute.

Methods:For this study, the already-scored answer sheets for Theory Papers I and II, as well as unmanned, non-interactive Objective Structured Clinical Examinations (OSCEs) with image-based tasks, from all 25 first-year psychiatry residents (March 2024 exam) were obtained from the examination section of the institute. The same question papers were then uploaded into three AI models (ChatGPT−3.5, Gemini Advanced, and Claude Sonnet). Four blinded faculty members evaluated the responses generated by the AI models. Final, the scores of the AI models and psychiatry residents were analyzed for comparison. Statistical analysis employed Kruskal–Wallis tests with post hoc Mann–Whitney U comparisons.

Results:AI models outperformed residents in theoretical assessments. In Paper I (theory), AI models achieved mean scores (standard deviation) of Claude Sonnet 67.88 (10.63), ChatGPT−3.5 70.38 (3.95), and Gemini Advanced 71.25 (3.86), compared to residents’ 58.0 (2.58). Paper II (theory) assessments showed even larger gaps, with AI models scoring Claude Sonnet 72.88 (3.77), ChatGPT−3.5 71.0 (3.56), and Gemini Advanced 69.63 (12.86), compared to residents’ 50.96 (2.49). OSCE performance patterns differed markedly. Paper I OSCEs showed equivalent performance: AI: 13.0; residents’: 13.16 (1.49), while Paper II OSCEs revealed variable results: Claude Sonnet excelled at 20.0 (1.41), but ChatGPT−3.5 underperformed at 15.0 (0.50), compared to residents at 16.6 (1.55). Inter-rater reliability coefficients remained excellent ( intraclass correlation coefficients [ICC]: 0.810–0.934).

Conclusions:While AI demonstrated superior theoretical knowledge, equivalent or variable practical skills performance reveals fundamental limitations in clinical reasoning and contextual understanding. These findings necessitate reconceptualizing psychiatric education to emphasize uniquely human competencies while leveraging AI’s capabilities for knowledge synthesis.

Here are some thoughts:

This study compared three large language models (LLMs) to first-year psychiatry residents using real institutional exams in India. The LLMs consistently outperformed residents on theoretical assessments (by 17–43%) but showed equivalent or inconsistent performance on practical OSCEs, revealing critical gaps in clinical reasoning and cultural contextualization. The authors conclude that psychiatric education should shift focus toward uniquely human skills like empathy and judgment, while using AI as a tool for knowledge synthesis.

The moon, the ghetto and artificial intelligence: Reducing systemic racism in computational algorithms.

Fountain, J. (2022).

Government Information Quarterly, 39(2), 101645.

https://doi.org/10.1016/j.giq.2021.101645 

Abstract

Computational algorithms and automated decision making systems that include them offer potential to improve public policy and organizations. But computational algorithms based on biased data encode those biases into algorithms, models and their outputs. Systemic racism is institutionalized bias with respect to race, ethnicity and related attributes. Such bias is located in data that encode the results and outputs of decisions that have been discriminatory, in procedures and processes that may intentionally or unintentionally disadvantage people based on race, and in policies that may discriminate by race. Computational algorithms may exacerbate systemic racism if they are not designed, developed, and used–that is, enacted–with attention to identifying and remedying bias specific to race. Advancing social equity in digital governance requires systematic, ongoing efforts to assure that automated decision making systems, and their enactment in complex public organizational arrangements, are free from bias.

Highlights

• Computational algorithms are powerful tools but may replicate biases.

• Biases, including systemic racism, in underlying data bias algorithms

• Automated decision making systems that discriminate harm people.

• Careful scrutiny of data, processes, variables and algorithms may reduce bias.

AI in psychoeducational assessment: a study of report generation.

Farmer, R. L., et al. (2025).

Online preprint. 

https://doi.org/10.13140/RG.2.2.31345.95844

Abstract

Artificial intelligence (AI) is poised to reshape school psychology, with report writing as a primary area of impact. A national sample of school psychologists using AI at work (n = 100) reported on its role in documentation. Of those using AI for report writing (n = 45), most applied it to discrete tasks such as rewriting sections for clarity (69%) or generating recommendations (67%). Far fewer used AI for interpretive or diagnostic purposes, and none relied on it to generate entire reports. Nearly all users (94%) edited AI-generated content before use. On average, practitioners who used AI for report writing saved 6.3 hours per week (95% CI: 4.6–8.1), more than double the savings of those using AI for other tasks (≈3 hours). AI shows promise for reducing documentation burden and reclaiming time for direct services, though its use raises ethical concerns requiring further guidance and oversight.

Here are some thoughts:

This preprint is important to practicing psychologists because AI is already being widely adopted in school psychology (roughly half of practitioners report using it at work) and those using it specifically for report writing are saving an estimated six hours per week, meaningful relief in a profession burdened by high caseloads and heavy documentation demands. However, the study also reveals real risks: a subset of practitioners are entering student-identifying information into AI platforms in likely violation of FERPA, and some are using AI to assist with diagnostic and eligibility decisions that currently lack any empirical validation for AI use. The research draws a clear distinction between acceptable applications (drafting, editing, improving clarity) and problematic ones (interpreting test scores, rendering classifications), and while it is encouraging that 94% of users edit AI-generated content before use, the authors argue this oversight needs to be formalized rather than left to individual habit, making this study a timely and practical reference for any psychologist navigating AI in their work.

Ethical implication of artificial intelligence (AI) adoption in financial decision making.

Owolabi, O. S., Uche, P. C., et al. (2024).

Computer and Information Science, 17(1), 49.

https://doi.org/10.5539/cis.v17n1p49

Abstract

The integration of artificial intelligence (AI) into the financial sector has raised ethical concerns that need to be addressed. This paper analyzes the ethical implications of using AI in financial decision-making and emphasizes the importance of an ethical framework to ensure its fair and trustworthy deployment. The study explores various ethical considerations, including the need to address algorithmic bias, promote transparency and explainability in AI systems, and adhere to regulations that protect equity, accountability, and public trust. By synthesizing research and empirical evidence, the paper highlights the complex relationship between AI innovation and ethical integrity in finance. To tackle this issue, the paper proposes a comprehensive and actionable ethical framework that advocates for clear guidelines, governance structures, regular audits, and collaboration among stakeholders. This framework aims to maximize the potential of AI while minimizing negative impacts and unintended consequences. The study serves as a valuable resource for policymakers, industry professionals, researchers, and other stakeholders, facilitating informed discussions, evidence-based decision-making, and the development of best practices for responsible AI integration in the financial sector. The ultimate goal is to ensure fairness, transparency, and accountability while reaping the benefits of AI for both the financial sector and society.

Here are some thoughts:

This paper explores the ethical implications of using artificial intelligence (AI) in financial decision-making.  It emphasizes the necessity of an ethical framework to ensure AI is used fairly and responsibly.  The study examines ethical concerns like algorithmic bias, the need for transparency and explainability in AI systems, and the importance of regulations that protect equity, accountability, and public trust.  The paper also proposes a comprehensive ethical framework with guidelines, governance structures, regular audits, and stakeholder collaboration to maximize AI's potential while minimizing negative impacts.

These themes are similar to concerns in using AI in the practice of psychology. Also, psychologists may need to be aware of these issues for their own financial and wealth management.

The impact of AI errors in a human-in-the-loop process

Agudo, U., Liberal, K. G., et al. (2024).

Cognitive Research Principles and 

Implications, 9(1).

https://doi.org/10.1186/s41235-023-00529-3

Abstract

Automated decision-making is becoming increasingly common in the public sector. As a result, political institutions recommend the presence of humans in these decision-making processes as a safeguard against potentially erroneous or biased algorithmic decisions. However, the scientific literature on human-in-the-loop performance is not conclusive about the benefits and risks of such human presence, nor does it clarify which aspects of this human–computer interaction may influence the final decision. In two experiments, we simulate an automated decision-making process in which participants judge multiple defendants in relation to various crimes, and we manipulate the time in which participants receive support from a supposed automated system with Artificial Intelligence (before or after they make their judgments). Our results show that human judgment is affected when participants receive incorrect algorithmic support, particularly when they receive it before providing their own judgment, resulting in reduced accuracy. The data and materials for these experiments are freely available at the Open Science Framework: https://osf.io/b6p4z/ Experiment 2 was preregistered.

Here are some thoughts:

This study explores the impact of AI errors in human-in-the-loop processes, where humans and AI systems collaborate in decision-making.  The research specifically investigates how the timing of AI support influences human judgment and decision accuracy.  The findings indicate that human judgment is negatively affected by incorrect algorithmic support, particularly when provided before the human's own judgment, leading to decreased accuracy.  This research highlights the complexities of human-computer interaction in automated decision-making contexts and emphasizes the need for a deeper understanding of how AI support systems can be effectively integrated to minimize errors and biases.    

This is important for psychologists because it sheds light on the cognitive biases and decision-making processes involved when humans interact with AI systems, which is an increasingly relevant area of study in the field.  Understanding these interactions can help psychologists develop interventions and strategies to mitigate negative impacts, such as automation bias, and improve the design of human-computer interfaces to optimize decision-making accuracy and reduce errors in various sectors, including public service, healthcare, and justice. 

On bullshit, large language models, and the need to curb your enthusiasm

Tigard, D. W. (2025).

AI And Ethics.

https://doi.org/10.1007/s43681-025-00743-3

Abstract

Amidst all the hype around artificial intelligence (AI), particularly regarding large language models (LLMs), generative AI and chatbots like ChatGPT, a surge of headlines is instilling caution and even explicitly calling “bullshit” on such technologies. Should we follow suit? What exactly does it mean to call bullshit on an AI program? When is doing so a good idea, and when might it not be? With this paper, I aim to provide a brief guide on how to call bullshit on ChatGPT and related systems. In short, one must understand the basic nature of LLMs, how they function and what they produce, and one must recognize bullshit. I appeal to the prominent work of the late Harry Frankfurt and suggest that recent accounts jump too quickly to the conclusion that LLMs are bullshitting. In doing so, I offer a more level-headed approach to calling bullshit, and accordingly, a way of navigating some of the recent critiques of generative AI systems.

Here are some thoughts:

This paper examines the application of Harry Frankfurt's theory of "bullshit" to large language models (LLMs) like ChatGPT. It discusses the controversy around labeling AI-generated content as "bullshit," arguing for a more nuanced approach. The author suggests that while LLM outputs might resemble bullshit due to their lack of concern for truth, LLMs themselves don't fit the definition of a "bullshitter" because they lack the intentions and aims that Frankfurt attributes to human bullshitters.

For psychologists, this distinction is important because it asks for a reconsideration of how we interpret and evaluate AI-generated content and its impact on human users. The paper further argues that if AI interactions provide tangible benefits to users without causing harm, then thwarting these interactions may not be necessary. This perspective encourages psychologists to weigh the ethical considerations of AI's influence on individuals, balancing concerns about authenticity and integrity with the potential for AI to enhance human experiences and productivity.

Can AI replace psychotherapists? Exploring the future of mental health care.

Zhang, Z., & Wang, J. (2024).

Frontiers in psychiatry, 15, 1444382.

https://doi.org/10.3389/fpsyt.2024.1444382

In the current technological era, Artificial Intelligence (AI) has transformed operations across numerous sectors, enhancing everything from manufacturing automation to intelligent decision support systems in financial services. In the health sector, particularly, AI has not only refined the accuracy of disease diagnoses but has also ushered in groundbreaking advancements in personalized medicine. The mental health field, amid a global crisis characterized by increasing demand and insufficient resources, is witnessing a significant paradigm shift facilitated by AI, presenting novel approaches that promise to reshape traditional mental health care models (see Figure 1 ).

Mental health, once a stigmatized aspect of health care, is now recognized as a critical component of overall well-being, with disorders such as depression becoming leading causes of global disability (WHO). Traditional mental health care, reliant on in-person consultations, is increasingly perceived as inadequate against the growing prevalence of mental health issues. AI’s role in mental health care is multifaceted, encompassing predictive analytics, therapeutic interventions, clinician support tools, and patient monitoring systems. For instance, AI algorithms are increasingly used to predict treatment outcomes by analyzing patient data. Meanwhile, AI-powered interventions, such as virtual reality exposure therapy and chatbot-delivered cognitive behavioral therapy, are being explored, though they are at varying stages of validation. Each of these applications is evolving at its own pace, influenced by technological advancements and the need for rigorous clinical validation.

The article is linked above.

Here are some thoughts: 

This article explores the evolving role of artificial intelligence (AI) in mental health care, particularly its potential to support or even replace some functions of human psychotherapists. With global demand for mental health services rising and traditional care systems under strain, AI is emerging as a tool to enhance diagnosis, personalize treatments, and provide therapeutic interventions through technologies like chatbots and virtual reality therapy. While early research shows promise, particularly in managing conditions such as anxiety and depression, existing studies are limited and call for larger, long-term trials to determine effectiveness and safety. The authors emphasize that while AI may supplement mental health care and address gaps in service delivery, it must be integrated responsibly, with careful attention to algorithmic bias, ethical considerations, and the irreplaceable human elements of psychotherapy, such as empathy and nuanced judgment.

Large Language Models and User Trust: Consequence of Self-Referential Learning Loop and the Deskilling of Health Care Professionals

Choudhury, A., & Chaudhry, Z. (2024).

Journal of medical Internet research, 26, e56764.

https://doi.org/10.2196/56764

Abstract

As the health care industry increasingly embraces large language models (LLMs), understanding the consequence of this integration becomes crucial for maximizing benefits while mitigating potential pitfalls. This paper explores the evolving relationship among clinician trust in LLMs, the transition of data sources from predominantly human-generated to artificial intelligence (AI)–generated content, and the subsequent impact on the performance of LLMs and clinician competence. One of the primary concerns identified in this paper is the LLMs’ self-referential learning loops, where AI-generated content feeds into the learning algorithms, threatening the diversity of the data pool, potentially entrenching biases, and reducing the efficacy of LLMs. While theoretical at this stage, this feedback loop poses a significant challenge as the integration of LLMs in health care deepens, emphasizing the need for proactive dialogue and strategic measures to ensure the safe and effective use of LLM technology. Another key takeaway from our investigation is the role of user expertise and the necessity for a discerning approach to trusting and validating LLM outputs. The paper highlights how expert users, particularly clinicians, can leverage LLMs to enhance productivity by off-loading routine tasks while maintaining a critical oversight to identify and correct potential inaccuracies in AI-generated content. This balance of trust and skepticism is vital for ensuring that LLMs augment rather than undermine the quality of patient care. We also discuss the risks associated with the deskilling of health care professionals. Frequent reliance on LLMs for critical tasks could result in a decline in health care providers’ diagnostic and thinking skills, particularly affecting the training and development of future professionals. The legal and ethical considerations surrounding the deployment of LLMs in health care are also examined. We discuss the medicolegal challenges, including liability in cases of erroneous diagnoses or treatment advice generated by LLMs. The paper references recent legislative efforts, such as The Algorithmic Accountability Act of 2023, as crucial steps toward establishing a framework for the ethical and responsible use of AI-based technologies in health care. In conclusion, this paper advocates for a strategic approach to integrating LLMs into health care. By emphasizing the importance of maintaining clinician expertise, fostering critical engagement with LLM outputs, and navigating the legal and ethical landscape, we can ensure that LLMs serve as valuable tools in enhancing patient care and supporting health care professionals. This approach addresses the immediate challenges posed by integrating LLMs and sets a foundation for their maintainable and responsible use in the future.

The abstract provides a sufficient summary.

Why Most Resist AI Companions

De Freitas, J., et al. (2025).

(Working Paper No. 25–030).

Here is a link.

Abstract

Chatbots are now able to form emotional relationships with people and alleviate loneliness—a growing public health concern. Behavioral research provides little insight into whether everyday people are likely to use these applications and why. We address this question by focusing on the context of “AI companion” applications, designed to provide people with synthetic interaction partners. Study 1 shows that people believe AI companions are more capable than human companions in advertised respects relevant to relationships (being more available and nonjudgmental). Even so, they view them as incapable of realizing the underlying values of relationships, like mutual caring, judging them as not ‘true’ relationships. Study 2 provides further insight into this belief: people believe relationships with AI companions are one-sided

(rather than mutual), because they see AI as incapable of understanding and feeling emotion. Study 3 finds that actually interacting with an AI companion increases acceptance by changing beliefs about the AI’s advertised capabilities, but not about its ability to achieve the true values of relationships, demonstrating the resilience of this belief against intervention. In short, despite the potential loneliness-reducing benefits of AI companions, we uncover fundamental psychological barriers to adoption, suggesting these benefits will not be easily realized.

Here are some thoughts:

The research explores why people remain reluctant to adopt AI companions, despite the growing public health crisis of loneliness and the promise that AI might offer support. Through a series of studies, the authors identify deep-seated psychological barriers to embracing AI as a substitute or supplement for human connection. Specifically, people tend to view AI companions as fundamentally incapable of embodying the core features of meaningful relationships—such as mutual care, genuine emotional understanding, and shared experiences. While participants often acknowledged some of the practical benefits of AI companionship, such as constant availability and non-judgmental interaction, they consistently doubted that AI could offer authentic or reciprocal relationships. Even when people interacted directly with AI systems, their impressions of the AI’s functional abilities improved, but their skepticism around the emotional and relational authenticity of AI companions remained firmly in place. These findings suggest that the resistance is not merely technological or unfamiliarity-based, but rooted in beliefs about what makes relationships "real."

For psychologists, this research is particularly important because it sheds light on how people conceptualize emotional connection, authenticity, and support—core concerns in both clinical and social psychology. As mental health professionals increasingly confront issues of social isolation, understanding the limitations of AI in replicating genuine human connection is critical. Psychologists might be tempted to view AI companions as possible interventions for loneliness, especially for individuals who are socially isolated or homebound. However, this paper underscores that unless these deep psychological barriers are acknowledged and addressed, such tools may be met with resistance or prove insufficient in fulfilling emotional needs. Furthermore, the study contributes to a broader understanding of human-technology relationships, offering insights into how people emotionally and cognitively differentiate between human and artificial agents. This knowledge is crucial for designing future interventions, therapeutic tools, and technologies that are sensitive to the human need for authenticity, reciprocity, and emotional depth in relationships.

The Moral Psychology of Artificial Intelligence

Bonnefon, J., Rahwan, I., & Shariff, A. (2023).

Annual Review of Psychology, 75(1), 653–675.

https://doi.org/10.1146/annurev-psych-030123-113559

Abstract

Moral psychology was shaped around three categories of agents and patients: humans, other animals, and supernatural beings. Rapid progress in artificial intelligence has introduced a fourth category for our moral psychology to deal with: intelligent machines. Machines can perform as moral agents, making decisions that affect the outcomes of human patients or solving moral dilemmas without human supervision. Machines can be perceived as moral patients, whose outcomes can be affected by human decisions, with important consequences for human–machine cooperation. Machines can be moral proxies that human agents and patients send as their delegates to moral interactions or use as a disguise in these interactions. Here we review the experimental literature on machines as moral agents, moral patients, and moral proxies, with a focus on recent findings and the open questions that they suggest.

Here are some thoughts:

This article delves into the evolving moral landscape shaped by artificial intelligence (AI). As AI technology progresses rapidly, it introduces a new category for moral consideration: intelligent machines.

Machines as moral agents are capable of making decisions that have significant moral implications. This includes scenarios where AI systems can inadvertently cause harm through errors, such as misdiagnosing a medical condition or misclassifying individuals in security contexts. The authors highlight that societal expectations for these machines are often unrealistically high; people tend to require AI systems to outperform human capabilities significantly while simultaneously overestimating human error rates. This disparity raises critical questions about how many mistakes are acceptable from machines in life-and-death situations and how these errors are distributed among different demographic groups.

In their role as moral patients, machines become subjects of human moral behavior. This perspective invites exploration into how humans interact with AI—whether cooperatively or competitively—and the potential biases that may arise in these interactions. For instance, there is a growing concern about algorithmic bias, where certain demographic groups may be unfairly treated by AI systems due to flawed programming or data sets.

Lastly, machines serve as moral proxies, acting as intermediaries in human interactions. This role allows individuals to delegate moral decision-making to machines or use them to mask unethical behavior. The implications of this are profound, as it raises ethical questions about accountability and the extent to which humans can offload their moral responsibilities onto AI.

Overall, the article underscores the urgent need for a deeper understanding of the psychological dimensions associated with AI's integration into society. As encounters between humans and intelligent machines become commonplace, addressing issues of trust, bias, and ethical alignment will be crucial in shaping a future where AI can be safely and effectively integrated into daily life.

High-reward, high-risk technologies? An ethical and legal account of AI development in healthcare

Corfmat, M., Martineau, J. T., & Régis, C. (2025).

BMC Med Ethics 26, 4

https://doi.org/10.1186/s12910-024-01158-1

Abstract

Background

Considering the disruptive potential of AI technology, its current and future impact in healthcare, as well as healthcare professionals’ lack of training in how to use it, the paper summarizes how to approach the challenges of AI from an ethical and legal perspective. It concludes with suggestions for improvements to help healthcare professionals better navigate the AI wave.

Methods

We analyzed the literature that specifically discusses ethics and law related to the development and implementation of AI in healthcare as well as relevant normative documents that pertain to both ethical and legal issues. After such analysis, we created categories regrouping the most frequently cited and discussed ethical and legal issues. We then proposed a breakdown within such categories that emphasizes the different - yet often interconnecting - ways in which ethics and law are approached for each category of issues. Finally, we identified several key ideas for healthcare professionals and organizations to better integrate ethics and law into their practices.

Results

We identified six categories of issues related to AI development and implementation in healthcare: (1) privacy; (2) individual autonomy; (3) bias; (4) responsibility and liability; (5) evaluation and oversight; and (6) work, professions and the job market. While each one raises different questions depending on perspective, we propose three main legal and ethical priorities: education and training of healthcare professionals, offering support and guidance throughout the use of AI systems, and integrating the necessary ethical and legal reflection at the heart of the AI tools themselves.

Conclusions

By highlighting the main ethical and legal issues involved in the development and implementation of AI technologies in healthcare, we illustrate their profound effects on professionals as well as their relationship with patients and other organizations in the healthcare sector. We must be able to identify AI technologies in medical practices and distinguish them by their nature so we can better react and respond to them. Healthcare professionals need to work closely with ethicists and lawyers involved in the healthcare system, or the development of reliable and trusted AI will be jeopardized.

The research is here.

Here are some thoughts:

This article explores the ethical and legal challenges surrounding artificial intelligence (AI) in healthcare. The authors identify six critical categories of issues: privacy, individual autonomy, bias, responsibility and liability, evaluation and oversight, as well as work and professional impacts.

The research highlights that AI is fundamentally different from previous medical technologies due to its disruptive potential and ability to perform autonomous learning and decision-making. While AI promises significant improvements in areas like biomedical research, precision medicine, and healthcare efficiency, there remains a significant gap between AI system development and practical implementation in healthcare settings.

The authors emphasize that healthcare professionals often lack comprehensive knowledge about AI technologies and their implications. They argue that understanding the nuanced differences between legal and ethical frameworks is crucial for responsible AI integration. Legal rules represent minimal mandatory requirements, while ethical considerations encourage deeper reflection on appropriate behaviors and choices.

The paper suggests three primary priorities for addressing AI's ethical and legal challenges: (1) educating and training healthcare professionals, (2) providing robust support and guidance during AI system use, and (3) integrating ethical and legal considerations directly into AI tool development. Ultimately, the researchers stress the importance of close collaboration between healthcare professionals, ethicists, and legal experts to develop reliable and trustworthy AI technologies.

AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking

Gerlich, M. (2025).

Societies, 15(1), 6.

https://doi.org/10.3390/soc15010006

Abstract

The proliferation of artificial intelligence (AI) tools has transformed numerous aspects of daily life, yet its impact on critical thinking remains underexplored. This study investigates the relationship between AI tool usage and critical thinking skills, focusing on cognitive offloading as a mediating factor. Utilising a mixed-method approach, we conducted surveys and in-depth interviews with 666 participants across diverse age groups and educational backgrounds. Quantitative data were analysed using ANOVA and correlation analysis, while qualitative insights were obtained through thematic analysis of interview transcripts. The findings revealed a significant negative correlation between frequent AI tool usage and critical thinking abilities, mediated by increased cognitive offloading. Younger participants exhibited higher dependence on AI tools and lower critical thinking scores compared to older participants. Furthermore, higher educational attainment was associated with better critical thinking skills, regardless of AI usage. These results highlight the potential cognitive costs of AI tool reliance, emphasising the need for educational strategies that promote critical engagement with AI technologies. This study contributes to the growing discourse on AI’s cognitive implications, offering practical recommendations for mitigating its adverse effects on critical thinking. The findings underscore the importance of fostering critical thinking in an AI-driven world, making this research essential reading for educators, policymakers, and technologists.

Here are some thoughts:

"De-skilling" is a concern regarding LLMs. Gerlich explores the critical relationship between AI tool usage and critical thinking skills. The study investigates how artificial intelligence technologies impact cognitive processes, with a specific focus on cognitive offloading as a mediating factor.

Gerlich conducted a comprehensive mixed-method research involving 666 participants from diverse age groups and educational backgrounds. The study employed surveys and in-depth interviews, analyzing data through ANOVA and correlation analysis, alongside thematic interview transcript analysis. Key findings revealed a significant negative correlation between frequent AI tool usage and critical thinking abilities, particularly pronounced among younger participants.

The research highlights several important insights. Younger participants demonstrated higher dependence on AI tools and correspondingly lower critical thinking scores compared to older participants. Conversely, individuals with higher educational attainment maintained better critical thinking skills regardless of their AI tool usage. These findings underscore the potential cognitive costs associated with excessive reliance on AI technologies.

The study's broader implications are important. It emphasizes the need for educational strategies that promote critical engagement with AI technologies, warning against the risk of cognitive offloading—where individuals delegate cognitive tasks to external tools, potentially reducing their capacity for deep, reflective thinking. By exploring how AI tools influence cognitive processes, the research contributes to the growing discourse on technology's impact on human cognitive development.

Gerlich's work is particularly significant as it offers practical recommendations for mitigating adverse effects on critical thinking in an increasingly AI-driven world. The research serves as essential reading for educators, policymakers, and technologists seeking to understand and address the complex relationship between artificial intelligence and human cognitive skills.

Augmenting research consent: Should large language models (LLMs) be used for informed consent to clinical research?

Allen, J. W., et al. (2024).

Research Ethics, in press.

Abstract

The integration of artificial intelligence (AI), particularly large language models (LLMs) like OpenAI’s ChatGPT, into clinical research could significantly enhance the informed consent process. This paper critically examines the ethical implications of employing LLMs to facilitate consent in clinical research. LLMs could offer considerable benefits, such as improving participant understanding and engagement, broadening participants’ access to the relevant information for informed consent, and increasing the efficiency of consent procedures. However, these theoretical advantages are accompanied by ethical risks, including the potential for misinformation, coercion, and challenges in accountability. Given the complex nature of consent in clinical research, which involves both written documentation (in the form of participant information sheets and informed consent forms) and in-person conversations with a researcher, the use of LLMs raises significant concerns about the adequacy of existing regulatory frameworks. Institutional Review Boards (IRBs) will need to consider substantial reforms to accommodate the integration of LLM-based consent processes. We explore five potential models for LLM implementation, ranging from supplementary roles to complete replacements of current consent processes, and offer recommendations for researchers and IRBs to navigate the ethical landscape. Thus, we aim to provide practical recommendations to facilitate the ethical introduction of LLM-based consent in research settings by considering factors such as participant understanding, information accuracy, human oversight and types of LLM applications in clinical research consent.

A pdf is here.

Here are some thoughts:

This paper examines the ethical implications of using large language models (LLMs) for informed consent in clinical research. While LLMs offer potential benefits, including personalized information, increased participant engagement, and improved efficiency, they also present risks related to accuracy, manipulation, and accountability. The authors explore five potential models for LLM implementation in consent processes, ranging from supplementary roles to complete replacements of current methods. Ultimately, they propose a hybrid approach that combines traditional consent methods with LLM-based interactions to maximize participant autonomy while maintaining ethical safeguards.

Advancements in AI-driven Healthcare: A Comprehensive Review of Diagnostics, Treatment, and Patient Care Integration

Kasula, B. Y. (2024, January 18).

International Journal of Machine Learning

for Sustainable Development.

Volume 6 (1).

Abstract

This research paper presents a comprehensive review of the recent advancements in AI-

driven healthcare, focusing on diagnostics, treatment, and the integration of AI technologies in

patient care. The study explores the evolution of artificial intelligence applications in medical

imaging, diagnosis accuracy, personalized treatment plans, and the overall enhancement of

healthcare delivery. Ethical considerations and challenges associated with AI adoption in

healthcare are also discussed. The paper concludes with insights into the potential future

developments and the transformative impact of AI on the healthcare landscape.

A pdf of the research is here.

Here are some thoughts:

This research paper provides a comprehensive review of recent advancements in AI-driven healthcare, focusing on diagnostics, treatment, and the integration of AI technologies in patient care. The study explores the evolution of artificial intelligence applications in medical imaging, diagnosis accuracy, personalized treatment plans, and the overall enhancement of healthcare delivery. It discusses the transformative impact of AI on healthcare, highlighting key achievements, challenges, and ethical considerations associated with its widespread adoption.

The paper examines AI's role in improving diagnostic accuracy, particularly in medical imaging, and its contribution to developing personalized treatment plans. It also addresses the ethical dimensions of AI in healthcare, including patient privacy, data security, and equitable distribution of AI-driven healthcare benefits. The research emphasizes the need for a holistic approach to AI integration in healthcare, calling for collaboration between healthcare professionals, technologists, and policymakers to navigate the evolving landscape successfully.

It is important for psychologists to understand the content of this article for several reasons. Firstly, AI is increasingly being applied in mental health diagnosis and treatment, as mentioned in the paper's references. Psychologists need to be aware of these advancements to stay current in their field and potentially incorporate AI-driven tools into their practice. Secondly, the ethical considerations discussed in the paper, such as patient privacy and data security, are equally relevant to psychological practice. Understanding these issues can help psychologists navigate the ethical challenges that may arise with the integration of AI in mental health care.

Moreover, the paper's emphasis on personalized medicine and treatment plans is particularly relevant to psychology, where individualized approaches are often crucial. By understanding AI's potential in this area, psychologists can explore ways to enhance their treatment strategies and improve patient outcomes. Lastly, as healthcare becomes increasingly interdisciplinary, psychologists need to be aware of technological advancements in other medical fields to collaborate effectively with other healthcare professionals and provide comprehensive care to their patients.

Beyond rating scales: With targeted evaluation, large language models are poised for psychological assessment

Kjell, O. N., Kjell, K., & Schwartz, H. A. (2023).

Psychiatry Research, 333, 115667.

https://doi.org/10.1016/j.psychres.2023.115667

Abstract

In this narrative review, we survey recent empirical evaluations of AI-based language assessments and present a case for the technology of large language models to be poised for changing standardized psychological assessment. Artificial intelligence has been undergoing a purported “paradigm shift” initiated by new machine learning models, large language models (e.g., BERT, LAMMA, and that behind ChatGPT). These models have led to unprecedented accuracy over most computerized language processing tasks, from web searches to automatic machine translation and question answering, while their dialogue-based forms, like ChatGPT have captured the interest of over a million users. The success of the large language model is mostly attributed to its capability to numerically represent words in their context, long a weakness of previous attempts to automate psychological assessment from language. While potential applications for automated therapy are beginning to be studied on the heels of chatGPT's success, here we present evidence that suggests, with thorough validation of targeted deployment scenarios, that AI's newest technology can move mental health assessment away from rating scales and to instead use how people naturally communicate, in language.

Highlights

• Artificial intelligence has been undergoing a purported “paradigm shift” initiated by new machine learning models, large language models.

• We review recent empirical evaluations of AI-based language assessments and present a case for the technology of large language models, that are used for chatGPT and BERT, to be poised for changing standardized psychological assessment.

• While potential applications for automated therapy are beginning to be studied on the heels of chatGPT's success, here we present evidence that suggests, with thorough validation of targeted deployment scenarios, that AI's newest technology can move mental health assessment away from rating scales and to instead use how people naturally communicate, in language.

Here are some thoughts:

The article underscores the transformative role of machine learning (ML) and artificial intelligence (AI) in psychological assessment, marking a significant shift in how psychologists approach their work. By integrating these technologies, assessments can become more accurate, efficient, and scalable, enabling psychologists to analyze vast amounts of data and uncover patterns that might otherwise go unnoticed. This is particularly important in improving diagnostic accuracy, as AI can help mitigate human bias and subjectivity, providing data-driven insights that complement clinical judgment. However, the adoption of these tools also raises critical ethical and practical considerations, such as ensuring client privacy, data security, and the responsible use of AI in alignment with professional standards.

As AI becomes more prevalent, the role of psychologists is evolving, requiring them to collaborate with these technologies by focusing on interpretation, contextual understanding, and therapeutic decision-making, while maintaining their unique human expertise.

Looking ahead, the article highlights emerging trends like natural language processing (NLP) for analyzing speech and text, as well as wearable devices for real-time behavioral and physiological data collection, offering psychologists innovative methods to enhance their practice. These advancements not only improve the precision of assessments but also pave the way for more personalized and timely interventions, ultimately supporting better mental health outcomes for clients.

Artificial Intelligence for Psychotherapy: A Review of the Current State and Future Directions

Beg et al. (2024). 

Indian Journal of Psychological Medicine.

https://doi.org/10.1177/02537176241260819

Abstract

Background:

Psychotherapy is crucial for addressing mental health issues but is often limited by accessibility and quality. Artificial intelligence (AI) offers innovative solutions, such as automated systems for increased availability and personalized treatments to improve psychotherapy. Nonetheless, ethical concerns about AI integration in mental health care remain.

Aim:

This narrative review explores the literature on AI applications in psychotherapy, focusing on their mechanisms, effectiveness, and ethical implications, particularly for depressive and anxiety disorders.

Methods:

A review was conducted, spanning studies from January 2009 to December 2023, focusing on empirical evidence of AI’s impact on psychotherapy. Following PRISMA guidelines, the authors independently screened and selected relevant articles. The analysis of 28 studies provided a comprehensive understanding of AI’s role in the field.

Results:

The results suggest that AI can enhance psychotherapy interventions for people with anxiety and depression, especially chatbots and internet-based cognitive-behavioral therapy. However, to achieve optimal outcomes, the ethical integration of AI necessitates resolving concerns about privacy, trust, and interaction between humans and AI.

Conclusion:

The study emphasizes the potential of AI-powered cognitive-behavioral therapy and conversational chatbots to address symptoms of anxiety and depression effectively. The article highlights the importance of cautiously integrating AI into mental health services, considering privacy, trust, and the relationship between humans and AI. This integration should prioritize patient well-being and assist mental health professionals while also considering ethical considerations and the prospective benefits of AI.

Here are some thoughts:

Artificial Intelligence (AI) is emerging as a promising tool in psychotherapy, offering innovative solutions to address mental health challenges. The comprehensive review explores the potential of AI-powered interventions, particularly for anxiety and depression disorders.

The study highlights several key insights about AI's role in mental health care. Researchers found that AI technologies like chatbots and internet-based cognitive-behavioral therapy (iCBT) can enhance psychological interventions by increasing accessibility and providing personalized treatment approaches. Machine learning, natural language processing, and deep learning are particularly crucial technologies enabling these advancements.

Despite the promising potential, the review emphasizes the critical need for careful integration of AI into mental health services. Ethical considerations remain paramount, with researchers stressing the importance of addressing privacy concerns, maintaining patient trust, and preserving the human element of therapeutic interactions. While AI can offer cost-effective and stigma-reducing solutions, it cannot yet fully replicate the profound empathy of face-to-face therapy.

The research examined 28 studies spanning from 2009 to 2023, revealing that AI interventions show particular promise in managing symptoms of anxiety and depression. Chatbots and iCBT demonstrated effectiveness in reducing psychological distress, though their impact on overall life satisfaction varies. The study calls for continued research to optimize AI's implementation in mental health care, balancing technological innovation with ethical principles.

Globally, organizations like the World Health Organization are developing regulatory frameworks to guide AI's responsible use in healthcare. In India, the Indian Council of Medical Research has already established guidelines for AI applications in biomedical research, signaling a growing recognition of this technology's potential.