Khera, R., Simon, M. A., & Ross, J. S. (2023).
JAMA, 330(23), 2255.
At the point of care, artificial intelligence (AI) algorithms have been developed to augment diagnostic decisions and suggest appropriate care pathways, by leveraging complex information in a patient’s electronic health record, such as imaging, documentation, and diagnostic testing. With an increasing number of technologies integrated into the diagnosis, management, and even treatment of patients, the promise of AI to enhance accuracy, reduce errors, reduce clinician burnout, and improve clinical workflows may appear imminent.
MostAI algorithms aredesigned tobe assistive technologies—augmenting, not replacing, clinicians’
decision-making. AI models are imperfect and lack the broader clinical context that may be relevant for patient care. The expectation is that the diagnostic performance of clinicians supported by AI will exceed those of clinicians without such support.
Here are some thoughts:
This article highlights a critical problem with artificial intelligence in medicine: automation bias. This is when clinicians trust an AI’s recommendation too much, even when it is clearly wrong or contradicts their own judgment. The authors show that biased AI models can significantly lower the quality of patient care, and simply explaining how the AI works does not fix the issue. Clinicians, often working under time pressure, may defer to the tool instead of using their own expertise, which can lead to direct patient harm.
The key takeaway is that keeping a human “in the loop” is not enough to ensure safety. Current regulatory approaches focus too much on the AI’s technical accuracy and not enough on how real clinicians actually use these tools in practice. The authors argue that better training, higher safety standards, and truly interpretable AI are needed. Without these changes, the excitement around medical AI risks overshadowing its primary goal: improving patient care, not undermining it.
