Welcome to the Nexus of Ethics, Psychology, Morality, Philosophy and Health Care

Welcome to the nexus of ethics, psychology, morality, technology, health care, and philosophy

Wednesday, January 22, 2025

Cognitive biases and artificial intelligence.

Wang, J., & Redelmeier, D. A. (2024).
NEJM AI, 1(12).

Abstract

Generative artificial intelligence (AI) models are increasingly utilized for medical applications. We tested whether such models are prone to human-like cognitive biases when offering medical recommendations. We explored the performance of OpenAI generative pretrained transformer (GPT)-4 and Google Gemini-1.0-Pro with clinical cases that involved 10 cognitive biases and system prompts that created synthetic clinician respondents. Medical recommendations from generative AI were compared with strict axioms of rationality and prior results from clinicians. We found that significant discrepancies were apparent for most biases. For example, surgery was recommended more frequently for lung cancer when framed in survival rather than mortality statistics (framing effect: 75% vs. 12%; P<0.001). Similarly, pulmonary embolism was more likely to be listed in the differential diagnoses if the opening sentence mentioned hemoptysis rather than chronic obstructive pulmonary disease (primacy effect: 100% vs. 26%; P<0.001). In addition, the same emergency department treatment was more likely to be rated as inappropriate if the patient subsequently died rather than recovered (hindsight bias: 85% vs. 0%; P<0.001). One exception was base-rate neglect that showed no bias when interpreting a positive viral screening test (correction for false positives: 94% vs. 93%; P=0.431). The extent of these biases varied minimally with the characteristics of synthetic respondents, was generally larger than observed in prior research with practicing clinicians, and differed between generative AI models. We suggest that generative AI models display human-like cognitive biases and that the magnitude of bias can be larger than observed in practicing clinicians.

Here are some thoughts:

The research explores how AI systems, trained on human-generated data, often replicate cognitive biases such as confirmation bias, representation bias, and anchoring bias. These biases arise from flawed data, algorithmic design, and human interactions, resulting in inequitable outcomes in areas like recruitment, criminal justice, and healthcare. To address these challenges, the authors propose several strategies, including ensuring diverse and inclusive datasets, enhancing algorithmic transparency, fostering interdisciplinary collaboration among ethicists, developers, and legislators, and establishing regulatory frameworks that prioritize fairness, accountability, and privacy. They emphasize that while biases in AI reflect human cognitive tendencies, they have the potential to exacerbate societal inequalities if left unchecked. A holistic approach combining technological solutions with ethical and regulatory oversight is necessary to create AI systems that are equitable and socially beneficial.

This topic connects deeply to ethics, values, and psychology. Ethically, the replication of biases in AI challenges principles of fairness, justice, and equity, highlighting the need for responsible innovation that aligns AI systems with societal values to avoid perpetuating systemic discrimination. Psychologically, the biases in AI reflect human cognitive shortcuts, such as heuristics, which, while useful for individual decision-making, can lead to harmful outcomes when embedded into AI systems. By leveraging insights from psychology to identify and mitigate these biases, and grounding AI development in ethical principles, society can create technology that is both advanced and aligned with humanistic values.