Garcia, B., Chua, E. Y. S., & Brah, H. S. (2025).
Journal of Medical Ethics, jme-2025.
Abstract
Large language models (LLMs) are increasingly proposed as scalable solutions to the global mental health crisis. But their deployment in psychiatric contexts raises a distinctive ethical concern: the problem of atypicality. Because LLMs generate outputs based on population-level statistical regularities, their responses—while typically appropriate for general users—may be dangerously inappropriate when interpreted by psychiatric patients, who often exhibit atypical cognitive or interpretive patterns. We argue that standard mitigation strategies, such as prompt engineering or fine-tuning, are insufficient to resolve this structural risk. Instead, we propose dynamic contextual certification (DCC): a staged, reversible and context-sensitive framework for deploying LLMs in psychiatry, inspired by clinical translation and dynamic safety models from artificial intelligence governance. DCC reframes chatbot deployment as an ongoing epistemic and ethical process that prioritises interpretive safety over static performance benchmarks. Atypicality, we argue, cannot be eliminated—but it can, and must, be proactively managed.
The article is linked above.
Here are some thoughs:
This article presents a critically important and nuanced argument that identifies a fundamental, structural flaw in deploying large language models (LLMs) in psychiatry: the problem of atypicality. The authors compellingly argue that because LLMs generate responses based on statistical regularities from a general population ("model-typicality"), their outputs are inherently mismatched for psychiatric patients, who often possess "interpretation-atypicality" due to conditions like paranoia or cognitive distortion. This misalignment is not merely a technical bug but a core ethical risk, where a model's factually accurate or conventionally appropriate response can inadvertently reinforce delusions or cause harm, as tragically illustrated in the case studies.
The paper's robust critique demonstrates why common technical solutions like prompt engineering and fine-tuning are insufficient, as they cannot anticipate the infinite contextuality of individual crises or "atypically atypical" presentations. In response, the proposed framework of Dynamic Contextual Certification (DCC) is a prudent and practical pathway forward, rightly reframing LLM deployment as a phased, evidence-building process akin to clinical trials, which prioritizes iterative safety and contextual fit over rapid scaling.
This work successfully bridges clinical wisdom, ethical reasoning, and technology critique, insisting that for AI to be human-centered in mental healthcare, it must be governed by a standard of therapeutic appropriateness, not just factual truth.
