The Illusion of Competence: How AI Tools Can Mask the Erosion of Clinical Judgment

Gavazzi, J. (2026, July).

Psychotherapy Bulletin, 61(4).

Clinical Impact Statement:

Psychologists who integrate AI tools without deliberate attention to their clinical consequences risk producing an illusion of competence: the capacity to generate sophisticated clinical language without the depth of reasoning that the language is meant to reflect. Maintaining the sequencing of independent judgment before AI consultation, treating AI outputs as objects of critical analysis, and preserving documentation as a reflective practice are essential safeguards for the integrity of quality psychological care.

The article is here.

Here is a snippet:

Practical Recommendations

None of this argues against using AI in psychological practice. LLMs offer genuine value as consultation resources, prompts for critical analysis, and tools for broadening the range of hypotheses a clinician considers. The argument is about sequencing and stance. Several principles follow.

1. AI-generated formulations should follow rather than precede independent clinical reasoning. The psychologist who develops her own differential formulation and then consults an LLM to examine what she may have missed is doing something different from the psychologist who queries the LLM first. The first sequence sharpens clinical thinking. The second quietly replaces it. This is a choice worth making consciously rather than letting convenience decide.
2. AI-generated outputs should be treated as objects of critical analysis, not as drafts to be refined. Before accepting an LLM’s formulation, ask what it assumed, what it excluded, and how it compares to your own reasoning. This turns an AI interaction into a reflective exercise rather than a shortcut.

Automation bias and assistive AI.

Khera, R., Simon, M. A., & Ross, J. S. (2023).

JAMA, 330(23), 2255. 

https://doi.org/10.1001/jama.2023.22557

At the point of care, artificial intelligence (AI) algorithms have been developed to augment diagnostic decisions and suggest appropriate care pathways, by leveraging complex information in a patient’s electronic health record, such as imaging, documentation, and diagnostic testing. With an increasing number of technologies integrated into the diagnosis, management, and even treatment of patients, the promise of AI to enhance accuracy, reduce errors, reduce clinician burnout, and improve clinical workflows may appear imminent.

MostAI algorithms aredesigned tobe assistive technologies—augmenting, not replacing, clinicians’

decision-making. AI models are imperfect and lack the broader clinical context that may be relevant for patient care. The expectation is that the diagnostic performance of clinicians supported by AI will exceed those of clinicians without such support.

A pdf is here.

Here are some thoughts:

This article highlights a critical problem with artificial intelligence in medicine: automation bias. This is when clinicians trust an AI’s recommendation too much, even when it is clearly wrong or contradicts their own judgment. The authors show that biased AI models can significantly lower the quality of patient care, and simply explaining how the AI works does not fix the issue. Clinicians, often working under time pressure, may defer to the tool instead of using their own expertise, which can lead to direct patient harm.

The key takeaway is that keeping a human “in the loop” is not enough to ensure safety. Current regulatory approaches focus too much on the AI’s technical accuracy and not enough on how real clinicians actually use these tools in practice. The authors argue that better training, higher safety standards, and truly interpretable AI are needed. Without these changes, the excitement around medical AI risks overshadowing its primary goal: improving patient care, not undermining it.

Ethical challenges and evolving strategies in the integration of artificial intelligence into clinical practice

Weiner, E. B.,  et al. (2025).

PLOS digital health, 4(4), e0000810.

https://doi.org/10.1371/journal.pdig.0000810

Abstract

Artificial intelligence (AI) has rapidly transformed various sectors, including healthcare, where it holds the potential to transform clinical practice and improve patient outcomes. However, its integration into medical settings brings significant ethical challenges that need careful consideration. This paper examines the current state of AI in healthcare, focusing on five critical ethical concerns: justice and fairness, transparency, patient consent and confidentiality, accountability, and patient-centered and equitable care. These concerns are particularly pressing as AI systems can perpetuate or even exacerbate existing biases, often resulting from non-representative datasets and opaque model development processes. The paper explores how bias, lack of transparency, and challenges in maintaining patient trust can undermine the effectiveness and fairness of AI applications in healthcare. In addition, we review existing frameworks for the regulation and deployment of AI, identifying gaps that limit the widespread adoption of these systems in a just and equitable manner. Our analysis provides recommendations to address these ethical challenges, emphasizing the need for fairness in algorithm design, transparency in model decision-making, and patient-centered approaches to consent and data privacy. By highlighting the importance of continuous ethical scrutiny and collaboration between AI developers, clinicians, and ethicists, we outline pathways for achieving more responsible and inclusive AI implementation in healthcare. These strategies, if adopted, could enhance both the clinical value of AI and the trustworthiness of AI systems among patients and healthcare professionals, ensuring that these technologies serve all populations equitably.

Here are some thoughts:

This article is important for psychologists because it highlights the critical ethical challenges surrounding patient trust, consent, and human-AI interaction in clinical settings—areas central to psychological practice. It details how patient demographics influence trust in AI and emphasizes the need for empathetic, transparent communication from AI systems to address patient anxieties and perceptions of "uniqueness neglect." Furthermore, it discusses "automation bias," where clinicians may overly rely on AI, a phenomenon psychologists must understand to support ethical decision-making and preserve the human-centered, therapeutic aspects of care.

The impact of AI errors in a human-in-the-loop process

Agudo, U., Liberal, K. G., et al. (2024).

Cognitive Research Principles and 

Implications, 9(1).

https://doi.org/10.1186/s41235-023-00529-3

Abstract

Automated decision-making is becoming increasingly common in the public sector. As a result, political institutions recommend the presence of humans in these decision-making processes as a safeguard against potentially erroneous or biased algorithmic decisions. However, the scientific literature on human-in-the-loop performance is not conclusive about the benefits and risks of such human presence, nor does it clarify which aspects of this human–computer interaction may influence the final decision. In two experiments, we simulate an automated decision-making process in which participants judge multiple defendants in relation to various crimes, and we manipulate the time in which participants receive support from a supposed automated system with Artificial Intelligence (before or after they make their judgments). Our results show that human judgment is affected when participants receive incorrect algorithmic support, particularly when they receive it before providing their own judgment, resulting in reduced accuracy. The data and materials for these experiments are freely available at the Open Science Framework: https://osf.io/b6p4z/ Experiment 2 was preregistered.

Here are some thoughts:

This study explores the impact of AI errors in human-in-the-loop processes, where humans and AI systems collaborate in decision-making.  The research specifically investigates how the timing of AI support influences human judgment and decision accuracy.  The findings indicate that human judgment is negatively affected by incorrect algorithmic support, particularly when provided before the human's own judgment, leading to decreased accuracy.  This research highlights the complexities of human-computer interaction in automated decision-making contexts and emphasizes the need for a deeper understanding of how AI support systems can be effectively integrated to minimize errors and biases.    

This is important for psychologists because it sheds light on the cognitive biases and decision-making processes involved when humans interact with AI systems, which is an increasingly relevant area of study in the field.  Understanding these interactions can help psychologists develop interventions and strategies to mitigate negative impacts, such as automation bias, and improve the design of human-computer interfaces to optimize decision-making accuracy and reduce errors in various sectors, including public service, healthcare, and justice.