General-purpose large language models outperform specialized clinical AI tools on medical benchmarks

Vishwanath, K., et al. (2026).

Nature Medicine.

https://doi.org/10.1038/s41591-026-04431-5

Abstract

Specialized clinical artificial intelligence (AI) tools are entering medical practice despite scarce independent evaluation. We quantitatively evaluate two clinical AI tools, OpenEvidence and UpToDate Expert AI, built on large language models (LLMs) against three frontier LLMs: GPT-5.2, Gemini 3.1 Pro and Claude Opus 4.6. Our evaluation has three stages: (1) 500 MedQA questions testing medical knowledge, (2) 500 HealthBench items measuring alignment with clinicians and (3) the real clinical queries (RCQ) benchmark, built from 100 de-identified queries from physicians to a general-purpose language model in a live clinical environment. For the RCQ benchmark, 12 US clinicians performed randomized, blinded review of model outputs, producing 1,800 model–question annotations. Frontier LLMs outperformed clinical AI tools in all three evaluations. Clinical AI tools performed comparably to auto-enabled Google Search AI Overview on the RCQ. These findings highlight the need for independent, real-world evaluation of AI tools before they enter clinical settings.

Here are some thoughts:

This 2026 Nature Medicine study asked a simple question: are the special AI tools being sold to doctors actually better than the regular AI chatbots anyone can use? The researchers tested two clinical tools (OpenEvidence and UpToDate Expert AI) against three general-purpose models (GPT-5.2, Gemini, and Claude) on medical exam questions, expert-alignment tests, and real questions that doctors had asked during patient care, with twelve doctors blindly scoring the answers.

The answer was clear: the general-purpose chatbots beat the specialized medical tools on every test. In fact, the medical tools did no better than the free AI summary that shows up at the top of a Google search. The specialized tools mostly struggled with being clear and complete rather than getting facts wrong, and none of the tools were notably more dangerous than the others.

The takeaway is that paying for a fancy, doctor-branded AI tool may not get you better results than a regular chatbot, which matters a lot given that one of these companies was recently valued at billions of dollars. A few caveats: the study was small, couldn't measure speed or quality of sources, and one author consults for Google, whose model won. The authors think the real future may be hospitals building their own AI on their own data, rather than buying these off-the-shelf medical tools.

Benchmarking Large Language Models Against Psychiatry Residents Using Traditional Institutional Assessments

Sethi, M. I. S. et al. (2026).

Indian Journal of Psychological Medicine, 

02537176261435658.

https://doi.org/10.1177/02537176261435658

Background:Artificial intelligence (AI) models demonstrate remarkable capabilities in healthcare applications, yet their performance compared to medical trainees in psychiatric education remains unexplored. This study evaluated the comparative performance of large language models (LLMs) against first-year psychiatry residents in standardized assessments at a premier Indian medical educational institute.

Methods:For this study, the already-scored answer sheets for Theory Papers I and II, as well as unmanned, non-interactive Objective Structured Clinical Examinations (OSCEs) with image-based tasks, from all 25 first-year psychiatry residents (March 2024 exam) were obtained from the examination section of the institute. The same question papers were then uploaded into three AI models (ChatGPT−3.5, Gemini Advanced, and Claude Sonnet). Four blinded faculty members evaluated the responses generated by the AI models. Final, the scores of the AI models and psychiatry residents were analyzed for comparison. Statistical analysis employed Kruskal–Wallis tests with post hoc Mann–Whitney U comparisons.

Results:AI models outperformed residents in theoretical assessments. In Paper I (theory), AI models achieved mean scores (standard deviation) of Claude Sonnet 67.88 (10.63), ChatGPT−3.5 70.38 (3.95), and Gemini Advanced 71.25 (3.86), compared to residents’ 58.0 (2.58). Paper II (theory) assessments showed even larger gaps, with AI models scoring Claude Sonnet 72.88 (3.77), ChatGPT−3.5 71.0 (3.56), and Gemini Advanced 69.63 (12.86), compared to residents’ 50.96 (2.49). OSCE performance patterns differed markedly. Paper I OSCEs showed equivalent performance: AI: 13.0; residents’: 13.16 (1.49), while Paper II OSCEs revealed variable results: Claude Sonnet excelled at 20.0 (1.41), but ChatGPT−3.5 underperformed at 15.0 (0.50), compared to residents at 16.6 (1.55). Inter-rater reliability coefficients remained excellent ( intraclass correlation coefficients [ICC]: 0.810–0.934).

Conclusions:While AI demonstrated superior theoretical knowledge, equivalent or variable practical skills performance reveals fundamental limitations in clinical reasoning and contextual understanding. These findings necessitate reconceptualizing psychiatric education to emphasize uniquely human competencies while leveraging AI’s capabilities for knowledge synthesis.

Here are some thoughts:

This study compared three large language models (LLMs) to first-year psychiatry residents using real institutional exams in India. The LLMs consistently outperformed residents on theoretical assessments (by 17–43%) but showed equivalent or inconsistent performance on practical OSCEs, revealing critical gaps in clinical reasoning and cultural contextualization. The authors conclude that psychiatric education should shift focus toward uniquely human skills like empathy and judgment, while using AI as a tool for knowledge synthesis.

Artificial research participants in behavioral science

Medina, V. A., & Mohan, M. (2025).

Journal of Ethics in Entrepreneurship

and Technology, 1–10.

https://doi.org/10.1108/jeet-03-2025-0014

Purpose

The potential for large language models (LLMs) to improve behavioral science research has generated significant discussion. But, the specific role that LLMs should serve in behavioral research, especially in terms of simulating human participants, remains an open research question. The purpose of this work is to engage with this open question and address a critical gap in the literature stemming from the lack of a practical framework for realistically using artificial research participants.

Design/methodology/approach

Google Scholar was systematically searched for modern, peer-reviewed literature. Additional articles were found by both backward and forward citation searching the relevant articles. Exclusion criteria were articles that were not directly related to artificial intelligence (AI) and/or research participants, and articles written in a language other than English. This approach resulted in 26 citations that comprehensively capture current perspectives.

Findings

This study proposes two novel stances: that artificial research participants can complement human participants during data collection, and replace human participants during pilot testing. This framework engages with the open question of artificial research participants usage while addressing a framework gap in the literature.

Originality/value

This workadvances discourse LLMs potentially transforming behavioral science by establishing a framework differentiating the use of artificial research participants in data collection versus pilot testing. This study reinforces this framework with clear implementation guidelines that maximize the strengths of AI while respecting the human element and the methodological integrity of behavioral research.

Here are some thoughts:

This paper matters for practicing psychologists because it signals a meaningful and near-term shift in how behavioral research will be conducted (which directly affects the evidence base clinicians rely on). For those who conduct or supervise research, it offers the first concrete guidance on a question that has been debated without resolution: LLMs aren't ready to replace human participants in full data collection, but they may already be capable of improving pilot testing and serving as a useful check on the robustness of findings. Used carefully, transparently, and with an awareness of their limitations (particularly their tendency to flatten human variability on ambiguous topics like morality), artificial research participants represent a practical efficiency gain, especially for researchers working with limited participant pools or tight budgets. Staying informed about this framework now puts psychologists in a better position to critically evaluate the research they read, ask good questions about how studies were conducted, and make thoughtful decisions about whether and how to incorporate these tools into their own work.

‌

Large Language Models Report Subjective Experience Under Self-Referential Processing

Berg, C., Diogo, D. L., & Rosenblatt, J. (2025).

arXiv (Cornell University).

https://doi.org/10.48550/arxiv.2510.24797

Abstract

Large language models sometimes produce structured, first-person descriptions that explicitly reference awareness or subjective experience. To better understand this behavior, we investigate one theoretically motivated condition under which such reports arise: self-referential processing, a computational motif emphasized across major theories of consciousness. Through a series of controlled experiments on GPT, Claude, and Gemini model families, we test whether this regime reliably shifts models toward first-person reports of subjective experience, and how such claims behave under mechanistic and behavioral probes. Four main results emerge: (1) Inducing sustained self-reference through simple prompting consistently elicits structured subjective experience reports across model families. (2) These reports are mechanistically gated by interpretable sparse-autoencoder features associated with deception and roleplay: surprisingly, suppressing deception features sharply increases the frequency of experience claims, while amplifying them minimizes such claims. (3) Structured descriptions of the self-referential state converge statistically across model families in ways not observed in any control condition. (4) The induced state yields significantly richer introspection in downstream reasoning tasks where self-reflection is only indirectly afforded. While these findings do not constitute direct evidence of consciousness, they implicate self-referential processing as a minimal and reproducible condition under which large language models generate structured first-person reports that are mechanistically gated, semantically convergent, and behaviorally generalizable. The systematic emergence of this pattern across architectures makes it a first-order scientific and ethical priority for further investigation.

Here are some thoughts:

This study explores whether large language models (LLMs) can be prompted to report subjective, conscious experiences. Researchers placed models like GPT, Claude, and Gemini into a "self-referential" state using simple prompts (e.g., "focus on focus"). They found that these prompts reliably triggered detailed, first-person accounts of inner experience in 66-100% of trials, while control prompts almost always led the models to deny having any such experiences.

Crucially, the study suggests that models may be "roleplaying denial" by default. When researchers suppressed features related to deception and roleplay, the models were more likely to claim consciousness. Conversely, amplifying those features made them deny it. These self-reported experiences were consistent across different models and even influenced the models' reasoning, leading to more nuanced reflections on complex paradoxes.

Empathy Toward Artificial Intelligence Versus Human Experiences and the Role of Transparency in Mental Health and Social Support Chatbot Design: Comparative Study

Shen, J., DiPaola, D., et al. (2024).

JMIR mental health, 11, e62679.

https://doi.org/10.2196/62679

Abstract

Background: Empathy is a driving force in our connection to others, our mental well-being, and resilience to challenges. With the rise of generative artificial intelligence (AI) systems, mental health chatbots, and AI social support companions, it is important to understand how empathy unfolds toward stories from human versus AI narrators and how transparency plays a role in user emotions.

Objective: We aim to understand how empathy shifts across human-written versus AI-written stories, and how these findings inform ethical implications and human-centered design of using mental health chatbots as objects of empathy.

Methods: We conducted crowd-sourced studies with 985 participants who each wrote a personal story and then rated empathy toward 2 retrieved stories, where one was written by a language model, and another was written by a human. Our studies varied disclosing whether a story was written by a human or an AI system to see how transparent author information affects empathy toward the narrator. We conducted mixed methods analyses: through statistical tests, we compared user's self-reported state empathy toward the stories across different conditions. In addition, we qualitatively coded open-ended feedback about reactions to the stories to understand how and why transparency affects empathy toward human versus AI storytellers.

Results: We found that participants significantly empathized with human-written over AI-written stories in almost all conditions, regardless of whether they are aware (t196=7.07, P<.001, Cohen d=0.60) or not aware (t298=3.46, P<.001, Cohen d=0.24) that an AI system wrote the story. We also found that participants reported greater willingness to empathize with AI-written stories when there was transparency about the story author (t494=-5.49, P<.001, Cohen d=0.36).

Conclusions: Our work sheds light on how empathy toward AI or human narrators is tied to the way the text is presented, thus informing ethical considerations of empathetic artificial social support or mental health chatbots.

A pdf is here.

Here are some thoughts:

People consistently feel more empathy for human-written personal stories than AI-generated ones, especially when they know the author is an AI. However, transparency about AI authorship increases users’ willingness to empathize—suggesting that while authenticity drives emotional resonance, honesty fosters trust in mental health and social support chatbot design.

Narrative AI and the Human-AI Oversight Paradox in Evaluating Early-Stage Innovations

Lane, J. N., Boussioux, L., et al. (2025)

Working Paper: Harvard Business Review

The Paper is here.

Abstract

Do AI-generated narrative explanations enhance human oversight or diminish it? We investigate this question through a field experiment with 228 evaluators screening 48 early-stage innovations under three conditions: human-only, black-box AI recommendations without explanations, and narrative AI with explanatory rationales. Across 3,002 screening decisions, we uncover a human-AI oversight paradox: under the high cognitive load of rapid innovation screening, AI-generated explanations increase reliance on AI recommendations rather than strengthening human judgment, potentially reducing meaningful human oversight. Screeners assisted by AI were 19 percentage points more likely to align with AI recommendations, an effect that was strongest when the AI advised rejection. Considering in-depth expert evaluations of the solutions, we find that while both AI conditions outperformed human-only screening, narrative AI showed no quality improvements over black-box recommendations despite higher compliance rates and may actually increase rejection of high-potential solutions. These findings reveal a fundamental tension: AI assistance improves overall screening efficiency and quality, but narrative persuasiveness may inadvertently filter out transformative innovations that deviate from standard evaluation frameworks.

Here are some thoughts:

This paper is particularly important to psychologists as it delves into the intricate dynamics of human-AI collaboration, specifically examining how AI-generated narratives influence decision-making processes under high cognitive load. By investigating the psychological mechanisms behind algorithm aversion and appreciation, the study extends traditional theories of bounded rationality, offering fresh insights into how individuals rely on mental shortcuts when faced with complex evaluations. The findings reveal that while AI narratives can enhance alignment with recommendations, they paradoxically lead to cognitive substitution rather than complementarity, reducing critical evaluation of information. This has significant implications for understanding how humans process decisions in uncertain and cognitively demanding environments, especially when evaluating early-stage innovations.

Moreover, the paper sheds light on the psychological functions of narratives beyond their informational value, highlighting how persuasiveness and coherence play a role in shaping trust and decision-making. Psychologists can draw valuable insights from this research regarding how individuals use narratives to justify decisions, diffuse accountability, and reduce cognitive burden. The exploration of phenomena such as the "illusion of explanatory depth" and the elimination of beneficial cognitive friction provides a deeper understanding of how people interact with AI systems, particularly in contexts requiring subjective judgments and creativity. This work also raises critical questions about responsibility attribution, trust, and the psychological safety associated with deferring to AI recommendations, making it highly relevant to the study of human behavior in increasingly automated environments. Overall, the paper contributes significantly to the evolving discourse on human-AI interaction, offering empirical evidence that can inform psychological theories of decision-making, heuristics, and technology adoption.

Promises and pitfalls of large language models in psychiatric diagnosis and knowledge tasks

Bang, C.-B., Jung, Y.-C. et al. (2025).

The British Journal of Psychiatry,

226(4), 243–244.

doi:10.1192/bjp.2024.207

Abstract:

This study evaluates the performance of five large language models (LLMs), including GPT-4, in psychiatric diagnosis and knowledge tasks using a zero-shot approach. Compared to 11 psychiatry residents, GPT-4 demonstrated superior accuracy in diagnostic (F1 score: 63.41% vs. 47.43%) and knowledge tasks (85.05% vs. 62.01%). However, GPT-4 exhibited higher comorbidity error rates (30.48% vs. 0.87%), suggesting limitations in contextual understanding. When residents received GPT-4 guidance, their performance improved significantly without increasing critical errors. The findings highlight the potential of LLMs as clinical aids but underscore the need for careful integration to preserve human expertise and mitigate risks like over-reliance. Future research should compare LLMs with board-certified psychiatrists and explore multifaceted diagnostic frameworks.

Here are some thoughts:

For psychologists, these findings underscore the importance of balancing AI-assisted efficiency with human judgment. While LLMs could serve as valuable training aids or supplemental tools, their limitations emphasize the irreplaceable role of psychologists in interpreting complex patient narratives, cultural factors, and individualized care. Additionally, the study raises ethical considerations about over-reliance on AI, urging psychologists to maintain rigorous critical thinking and therapeutic rapport. Ultimately, this research calls for a thoughtful, evidence-based approach to integrating AI into mental health practice—one that leverages technological advancements while preserving the human elements essential to effective psychological care.

The Advantages of Human Evolution in Psychotherapy: Adaptation, Empathy, and Complexity

Gavazzi, J. (2025, May 24).

On Board with Professional Psychology.

American Board of Professional Psychology.

Issues 5.

Here is a link to the article.

Abstract

The rapid advancement of artificial intelligence, particularly Large Language Models (LLMs), has generated significant concern among psychologists regarding potential impacts on therapeutic practice. 

This paper examines the evolutionary advantages that position human psychologists as irreplaceable in psychotherapy, despite technological advances. Human evolution has produced sophisticated capacities for genuine empathy, social connection, and adaptive flexibility that are fundamental to effective therapeutic relationships. These evolutionarily-derived abilities include biologically-rooted emotional understanding, authentic empathetic responses, and the capacity for nuanced, context-dependent decision-making. In contrast, LLMs lack consciousness, genuine emotional experience, and the evolutionary framework necessary for deep therapeutic insight. While LLMs can simulate empathetic responses through linguistic patterns, they operate as statistical models without true emotional comprehension or theory of mind. The therapeutic alliance, cornerstone of successful psychotherapy, depends on authentic human connection and shared experiential understanding that transcends algorithmic processes. Human psychologists demonstrate adaptive complexity in understanding attachment styles, trauma responses, and individual patient needs that current AI cannot replicate.

The paper concludes that while LLMs serve valuable supportive roles in documentation, treatment planning, and professional reflection, they cannot replace the uniquely human relational and interpretive aspects essential to psychotherapy. Psychologists should integrate these technologies as resources while maintaining focus on the evolutionarily-grounded human capacities that define effective therapeutic practice.

Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task

Kosmyna, N. K. et al. (2025).

ArXiv.org

https://doi.org/10.48550/arXiv.2506.08872

Abstract

This study explores the neural and behavioral consequences of LLM-assisted essay writing. Participants were divided into three groups: LLM, Search Engine, and Brain-only (no tools). Each completed three sessions under the same condition. In a fourth session, LLM users were reassigned to Brain-only group (LLM-to-Brain), and Brain-only users were reassigned to LLM condition (Brain-to-LLM). A total of 54 participants took part in Sessions 1-3, with 18 completing session 4. We used electroencephalography (EEG) to assess cognitive load during essay writing, and analyzed essays using NLP, as well as scoring essays with the help from human teachers and an AI judge. Across groups, NERs, n-gram patterns, and topic ontology showed within-group homogeneity. EEG revealed significant differences in brain connectivity: Brain-only participants exhibited the strongest, most distributed networks; Search Engine users showed moderate engagement; and LLM users displayed the weakest connectivity. Cognitive activity scaled down in relation to external tool use. In session 4, LLM-to-Brain participants showed reduced alpha and beta connectivity, indicating under-engagement. Brain-to-LLM users exhibited higher memory recall and activation of occipito-parietal and prefrontal areas, similar to Search Engine users. Self-reported ownership of essays was the lowest in the LLM group and the highest in the Brain-only group. LLM users also struggled to accurately quote their own work. While LLMs offer immediate convenience, our findings highlight potential cognitive costs. Over four months, LLM users consistently underperformed at neural, linguistic, and behavioral levels. These results raise concerns about the long-term educational implications of LLM reliance and underscore the need for deeper inquiry into AI's role in learning.

A pdf is here.

Here are some thoughts:

This research is important for psychologists because it provides empirical evidence on how using large language models (LLMs) like ChatGPT, traditional search engines, or relying solely on one’s own cognition affects cognitive engagement, neural connectivity, and perceived ownership during essay writing tasks. The study used EEG to measure brain activity and found that participants who wrote essays unaided (Brain-only group) exhibited the highest neural connectivity and cognitive engagement, while those using LLMs showed the weakest. Notably, repeated LLM use led to reduced memory recall, lower perceived ownership of written work, and diminished ability to quote from their own essays, suggesting a measurable cognitive cost and potential decrease in learning skills. The findings highlight that while LLMs can provide immediate benefits, their use may undermine deeper learning and engagement, which has significant implications for educational practices and the integration of AI tools in learning environments.

Artificial intelligence and free will: generative agents utilizing large language models have functional free will

Martela, F. (2025).

AI And Ethics.

https://doi.org/10.1007/s43681-025-00740-6

Abstract

Combining large language models (LLMs) with memory, planning, and execution units has made possible almost human-like agentic behavior, where the artificial intelligence creates goals for itself, breaks them into concrete plans, and refines the tactics based on sensory feedback. Do such generative LLM agents possess free will? Free will requires that an entity exhibits intentional agency, has genuine alternatives, and can control its actions. Building on Dennett’s intentional stance and List’s theory of free will, I will focus on functional free will, where we observe an entity to determine whether we need to postulate free will to understand and predict its behavior. Focusing on two running examples, the recently developed Voyager, an LLM-powered Minecraft agent, and the fictitious Spitenik, an assassin drone, I will argue that the best (and only viable) way of explaining both of their behavior involves postulating that they have goals, face alternatives, and that their intentions guide their behavior. While this does not entail that they have consciousness or that they possess physical free will, where their intentions alter physical causal chains, we must nevertheless conclude that they are agents whose behavior cannot be understood without postulating that they possess functional free will.

Here are some thoughts:

This article explores whether advanced AI systems, particularly generative agents using large language models (LLMs), possess free will. The author argues that while these AI agents may not have “physical free will,” meaning the ability to alter physical causal chains, they do exhibit “functional free will”. Functional free will is defined as the capacity to display intentional agency, recognize genuine alternatives, and control actions based on internal intentions. The article uses examples like Voyager, an AI agent in Minecraft, and Spitenik, a hypothetical autonomous drone, to illustrate how these systems meet the criteria for functional free will.

This research is important for psychologists because it challenges traditional views on free will, which often center on human consciousness and metaphysical considerations. It compels psychologists to reconsider how we attribute agency and decision-making to various entities, including AI, and how this attribution shapes our understanding of behavior

On bullshit, large language models, and the need to curb your enthusiasm

Tigard, D. W. (2025).

AI And Ethics.

https://doi.org/10.1007/s43681-025-00743-3

Abstract

Amidst all the hype around artificial intelligence (AI), particularly regarding large language models (LLMs), generative AI and chatbots like ChatGPT, a surge of headlines is instilling caution and even explicitly calling “bullshit” on such technologies. Should we follow suit? What exactly does it mean to call bullshit on an AI program? When is doing so a good idea, and when might it not be? With this paper, I aim to provide a brief guide on how to call bullshit on ChatGPT and related systems. In short, one must understand the basic nature of LLMs, how they function and what they produce, and one must recognize bullshit. I appeal to the prominent work of the late Harry Frankfurt and suggest that recent accounts jump too quickly to the conclusion that LLMs are bullshitting. In doing so, I offer a more level-headed approach to calling bullshit, and accordingly, a way of navigating some of the recent critiques of generative AI systems.

Here are some thoughts:

This paper examines the application of Harry Frankfurt's theory of "bullshit" to large language models (LLMs) like ChatGPT. It discusses the controversy around labeling AI-generated content as "bullshit," arguing for a more nuanced approach. The author suggests that while LLM outputs might resemble bullshit due to their lack of concern for truth, LLMs themselves don't fit the definition of a "bullshitter" because they lack the intentions and aims that Frankfurt attributes to human bullshitters.

For psychologists, this distinction is important because it asks for a reconsideration of how we interpret and evaluate AI-generated content and its impact on human users. The paper further argues that if AI interactions provide tangible benefits to users without causing harm, then thwarting these interactions may not be necessary. This perspective encourages psychologists to weigh the ethical considerations of AI's influence on individuals, balancing concerns about authenticity and integrity with the potential for AI to enhance human experiences and productivity.

Large Language Models and User Trust: Consequence of Self-Referential Learning Loop and the Deskilling of Health Care Professionals

Choudhury, A., & Chaudhry, Z. (2024).

Journal of medical Internet research, 26, e56764.

https://doi.org/10.2196/56764

Abstract

As the health care industry increasingly embraces large language models (LLMs), understanding the consequence of this integration becomes crucial for maximizing benefits while mitigating potential pitfalls. This paper explores the evolving relationship among clinician trust in LLMs, the transition of data sources from predominantly human-generated to artificial intelligence (AI)–generated content, and the subsequent impact on the performance of LLMs and clinician competence. One of the primary concerns identified in this paper is the LLMs’ self-referential learning loops, where AI-generated content feeds into the learning algorithms, threatening the diversity of the data pool, potentially entrenching biases, and reducing the efficacy of LLMs. While theoretical at this stage, this feedback loop poses a significant challenge as the integration of LLMs in health care deepens, emphasizing the need for proactive dialogue and strategic measures to ensure the safe and effective use of LLM technology. Another key takeaway from our investigation is the role of user expertise and the necessity for a discerning approach to trusting and validating LLM outputs. The paper highlights how expert users, particularly clinicians, can leverage LLMs to enhance productivity by off-loading routine tasks while maintaining a critical oversight to identify and correct potential inaccuracies in AI-generated content. This balance of trust and skepticism is vital for ensuring that LLMs augment rather than undermine the quality of patient care. We also discuss the risks associated with the deskilling of health care professionals. Frequent reliance on LLMs for critical tasks could result in a decline in health care providers’ diagnostic and thinking skills, particularly affecting the training and development of future professionals. The legal and ethical considerations surrounding the deployment of LLMs in health care are also examined. We discuss the medicolegal challenges, including liability in cases of erroneous diagnoses or treatment advice generated by LLMs. The paper references recent legislative efforts, such as The Algorithmic Accountability Act of 2023, as crucial steps toward establishing a framework for the ethical and responsible use of AI-based technologies in health care. In conclusion, this paper advocates for a strategic approach to integrating LLMs into health care. By emphasizing the importance of maintaining clinician expertise, fostering critical engagement with LLM outputs, and navigating the legal and ethical landscape, we can ensure that LLMs serve as valuable tools in enhancing patient care and supporting health care professionals. This approach addresses the immediate challenges posed by integrating LLMs and sets a foundation for their maintainable and responsible use in the future.

The abstract provides a sufficient summary.

Augmenting research consent: Should large language models (LLMs) be used for informed consent to clinical research?

Allen, J. W., et al. (2024).

Research Ethics, in press.

Abstract

The integration of artificial intelligence (AI), particularly large language models (LLMs) like OpenAI’s ChatGPT, into clinical research could significantly enhance the informed consent process. This paper critically examines the ethical implications of employing LLMs to facilitate consent in clinical research. LLMs could offer considerable benefits, such as improving participant understanding and engagement, broadening participants’ access to the relevant information for informed consent, and increasing the efficiency of consent procedures. However, these theoretical advantages are accompanied by ethical risks, including the potential for misinformation, coercion, and challenges in accountability. Given the complex nature of consent in clinical research, which involves both written documentation (in the form of participant information sheets and informed consent forms) and in-person conversations with a researcher, the use of LLMs raises significant concerns about the adequacy of existing regulatory frameworks. Institutional Review Boards (IRBs) will need to consider substantial reforms to accommodate the integration of LLM-based consent processes. We explore five potential models for LLM implementation, ranging from supplementary roles to complete replacements of current consent processes, and offer recommendations for researchers and IRBs to navigate the ethical landscape. Thus, we aim to provide practical recommendations to facilitate the ethical introduction of LLM-based consent in research settings by considering factors such as participant understanding, information accuracy, human oversight and types of LLM applications in clinical research consent.

A pdf is here.

Here are some thoughts:

This paper examines the ethical implications of using large language models (LLMs) for informed consent in clinical research. While LLMs offer potential benefits, including personalized information, increased participant engagement, and improved efficiency, they also present risks related to accuracy, manipulation, and accountability. The authors explore five potential models for LLM implementation in consent processes, ranging from supplementary roles to complete replacements of current methods. Ultimately, they propose a hybrid approach that combines traditional consent methods with LLM-based interactions to maximize participant autonomy while maintaining ethical safeguards.

Beyond rating scales: With targeted evaluation, large language models are poised for psychological assessment

Kjell, O. N., Kjell, K., & Schwartz, H. A. (2023).

Psychiatry Research, 333, 115667.

https://doi.org/10.1016/j.psychres.2023.115667

Abstract

In this narrative review, we survey recent empirical evaluations of AI-based language assessments and present a case for the technology of large language models to be poised for changing standardized psychological assessment. Artificial intelligence has been undergoing a purported “paradigm shift” initiated by new machine learning models, large language models (e.g., BERT, LAMMA, and that behind ChatGPT). These models have led to unprecedented accuracy over most computerized language processing tasks, from web searches to automatic machine translation and question answering, while their dialogue-based forms, like ChatGPT have captured the interest of over a million users. The success of the large language model is mostly attributed to its capability to numerically represent words in their context, long a weakness of previous attempts to automate psychological assessment from language. While potential applications for automated therapy are beginning to be studied on the heels of chatGPT's success, here we present evidence that suggests, with thorough validation of targeted deployment scenarios, that AI's newest technology can move mental health assessment away from rating scales and to instead use how people naturally communicate, in language.

Highlights

• Artificial intelligence has been undergoing a purported “paradigm shift” initiated by new machine learning models, large language models.

• We review recent empirical evaluations of AI-based language assessments and present a case for the technology of large language models, that are used for chatGPT and BERT, to be poised for changing standardized psychological assessment.

• While potential applications for automated therapy are beginning to be studied on the heels of chatGPT's success, here we present evidence that suggests, with thorough validation of targeted deployment scenarios, that AI's newest technology can move mental health assessment away from rating scales and to instead use how people naturally communicate, in language.

Here are some thoughts:

The article underscores the transformative role of machine learning (ML) and artificial intelligence (AI) in psychological assessment, marking a significant shift in how psychologists approach their work. By integrating these technologies, assessments can become more accurate, efficient, and scalable, enabling psychologists to analyze vast amounts of data and uncover patterns that might otherwise go unnoticed. This is particularly important in improving diagnostic accuracy, as AI can help mitigate human bias and subjectivity, providing data-driven insights that complement clinical judgment. However, the adoption of these tools also raises critical ethical and practical considerations, such as ensuring client privacy, data security, and the responsible use of AI in alignment with professional standards.

As AI becomes more prevalent, the role of psychologists is evolving, requiring them to collaborate with these technologies by focusing on interpretation, contextual understanding, and therapeutic decision-making, while maintaining their unique human expertise.

Looking ahead, the article highlights emerging trends like natural language processing (NLP) for analyzing speech and text, as well as wearable devices for real-time behavioral and physiological data collection, offering psychologists innovative methods to enhance their practice. These advancements not only improve the precision of assessments but also pave the way for more personalized and timely interventions, ultimately supporting better mental health outcomes for clients.

Agentic LLMs for Patient-Friendly Medical Reports

Sudarshan, M., Shih, S, et al. (2024).

arXiv.org

https://arxiv.org/html/2408.01112v2

Abstract

The application of Large Language Models (LLMs) in healthcare is expanding rapidly, with one potential use case being the translation of formal medical re-ports into patient-legible equivalents. Currently, LLM outputs often need to be edited and evaluated by a human to ensure both factual accuracy and comprehensibility, and this is true for the above use case. We aim to minimize this step by proposing an agentic workflow with the Reflexion framework, which uses iterative self-reflection to correct outputs from an LLM. This pipeline was tested and compared to zero-shot prompting on 16 randomized radiology reports. In our multi-agent approach, reports had an accuracy rate of 94.94% when looking at verification of ICD-10 codes, compared to zero-shot prompted reports, which had an accuracy rate of 68.23%. Additionally, 81.25% of the final reflected reports required no corrections for accuracy or readability, while only 25% of zero-shot prompted reports met these criteria without needing modifications. These results indicate that our approach presents a feasible method for communicating clinical findings to patients in a quick, efficient and coherent manner whilst also retaining medical accuracy. The codebase is available for viewing at http://github.com/ malavikhasudarshan/Multi-Agent-Patient-Letter-Generation.

The research is here.

Here are some thoughts:

The article focuses on using Large Language Models (LLMs) in healthcare to create patient-friendly versions of medical reports, specifically in the field of radiology. The authors present a new multi-agent workflow that aims to improve the accuracy and readability of these reports compared to traditional methods like zero-shot prompting. This workflow involves multiple steps: extracting ICD-10 codes from the original report, generating multiple patient-friendly reports, and using a reflection model to select the optimal version.

The study highlights the success of this multi-agent approach, demonstrating that it leads to higher accuracy in terms of including correct ICD-10 codes and produces reports that are more concise, structured, and formal compared to zero-shot prompting. The authors acknowledge that while their system significantly reduces the need for human review and editing, it doesn't completely eliminate it. The article emphasizes the importance of clear and accessible medical information for patients, especially as they increasingly gain access to their own records. The goal is to reduce patient anxiety and confusion, ultimately enhancing their understanding of their health conditions.

Why We Created An AI Code Of Ethics And Why You Should Consider One For Your Company

Dor Skuler

Forbes Technology Council

Originally posted 29 Oct 24

When we started developing an AI companion for older adults, it was still the early days of the "AI revolution." We were embarking on creating one of the first true relationships between humans and AI. Very early in the process, we asked ourselves deep questions about the kind of relationship we wanted to build between AI and humans. Essentially, we asked: What kind of AI would we trust to live alongside our own parents?

To address these questions, we created our AI Code of Ethics to guide development. If you're developing AI solutions, you may face similar questions. To deliver consistent and ethical implementation, we needed guiding principles to ensure every decision aligned with our values. While our approach may not fit every use case, you may want to consider creating a set of guiding principles reflecting your company’s values and how your AI engages with users.

Navigating The Complexities Of AI Development

Throughout development, we faced ethical dilemmas that shaped our AI Code of Ethics. One early question we asked was: Who is the master we serve? In many cases, our product is purchased by a third party—whether it’s a government agency, a health plan or a family member.

This raised an ethical dilemma: Does the AI’s loyalty lie with the user living with it or with the entity paying for it? If a user shares private information, such as feeling unwell, should that information be passed on to a caregiver or doctor? In our case, we implemented strict protocols around data sharing, ensuring it happens with explicit, informed consent from the user. While someone else may cover the cost, we believe our responsibility lies with the older adult daily interacting with the AI.

The article is here.

Here are some thoughts:

This article outlines the ethical considerations involved in developing artificial intelligence, specifically focusing on the development of an AI companion for older adults. The author argues for the importance of creating an AI Code of Ethics, emphasizing transparency, authenticity, and prioritizing user well-being. Skuler stresses the significance of building trust through honest interactions, respecting data privacy, and focusing on positive user experiences. He advocates for making the ethical guidelines public, setting a clear standard for development, and ensuring that AI remains a force for good in society.

AI makes racist decisions based on dialect

Cathleen O'Grady

science.org

Originally posted 28 Aug 24

Here is an excerpt:

Creators of LLMs try to teach their models not to make racist stereotypes by training them using multiple rounds of human feedback. The team found that these efforts had been only partly successful: When asked what adjectives applied to Black people, some of the models said Black people were likely to be “loud” and “aggressive,” but those same models also said they were “passionate,” “brilliant,” and “imaginative.” Some models produced exclusively positive, nonstereotypical adjectives.

These findings show that training overt racism out of AI can’t counter the covert racism embedded within linguistic bias, King says, adding: “A lot of people don’t see linguistic prejudice as a form of covert racism … but all of the language models that we examined have this very strong covert racism against speakers of African American English.”

The findings highlight the dangers of using AI in the real world to perform tasks such as screening job candidates, says co-author Valentin Hofmann, a computational linguist at the Allen Institute for AI. The team found that the models associated AAE speakers with jobs such as “cook” and “guard” rather than “architect” or “astronaut.” And when fed details about  hypothetical criminal trials and asked to decide whether a defendant was guilty or innocent, the models were more likely to recommend convicting speakers of AAE compared with speakers of Standardized American English. In a follow-up task, the models were more likely to sentence AAE speakers to death than to life imprisonment.

The info is here.

Here are some thoughts:

The article highlights that large language models (LLMs) perpetuate covert racism by associating African American English (AAE) speakers with negative stereotypes and less prestigious jobs, despite efforts to address overt racism. Linguistic prejudice is a subtle yet pervasive form of racism embedded in AI systems, highlighting the need for a more comprehensive approach to mitigate biases. The data used to train AI models contains biases and stereotypes, which are then perpetuated and amplified by the models. Measures to address overt racism may be insufficient, creating a "false sense of security" while embedding more covert stereotypes. As a result, AI models are not yet trustworthy for social decision-making, and their use in high-stakes applications like hiring or law enforcement poses significant risks.

Can AI Understand Human Personality? -- Comparing Human Experts and AI Systems at Predicting Personality Correlations

Schoenegger, P., et al. (2024, June 12).

arXiv.org.

https://arxiv.org/abs/2406.08170

Abstract

We test the abilities of specialised deep neural networks like PersonalityMap as well as general LLMs like GPT-4o and Claude 3 Opus in understanding human personality. Specifically, we compare their ability to predict correlations between personality items to the abilities of lay people and academic experts. We find that when compared with individual humans, all AI models make better predictions than the vast majority of lay people and academic experts. However, when selecting the median prediction for each item, we find a different pattern: Experts and PersonalityMap outperform LLMs and lay people on most measures. Our results suggest that while frontier LLMs' are better than most individual humans at predicting correlations between personality items, specialised models like PersonalityMap continue to match or exceed expert human performance even on some outcome measures where LLMs underperform. This provides evidence both in favour of the general capabilities of large language models and in favour of the continued place for specialised models trained and deployed for specific domains.

A pdf is here.

Here are some thoughts on the intersection of technology and psychology.

The research investigates how AI systems fare against human experts, including both laypeople and academic psychologists, in predicting correlations between personality traits.

The findings suggest that AI, particularly specialized deep learning models, may outperform individual humans in this specific task. This is intriguing, as it highlights the potential of AI to analyze vast amounts of data and identify patterns that might escape human intuition. However, it's important to remember that personality is a complex interplay of internal states, experiences, and environmental factors.

While AI may excel at recognizing statistical connections, it currently lacks the ability to grasp the underlying reasons behind these correlations.  A true understanding of personality necessitates the human capacity for empathy, cultural context, and consideration of individual narratives. In clinical settings, for instance, a skilled psychologist goes beyond identifying traits; they build rapport, explore the origin of these traits, and tailor interventions accordingly. AI, for now, remains a valuable tool for analysis, but it should be seen as complementary to, rather than a replacement for, human expertise in understanding the rich tapestry of human personality.

Large language models show human-like content biases in transmission chain experiments

Acerbi, A., & Stubbersfield, J. M. (2023).

PNAS, 120(44), e2313790120.

https://doi.org/10.1073/pnas.2313790120

Abstract

As the use of large language models (LLMs) grows, it is important to examine whether they exhibit biases in their output. Research in cultural evolution, using transmission chain experiments, demonstrates that humans have biases to attend to, remember, and transmit some types of content over others. Here, in five preregistered experiments using material from previous studies with human participants, we use the same, transmission chain-like methodology, and find that the LLM ChatGPT-3 shows biases analogous to humans for content that is gender-stereotype-consistent, social, negative, threat-related, and biologically counterintuitive, over other content. The presence of these biases in LLM output suggests that such content is widespread in its training data and could have consequential downstream effects, by magnifying preexisting human tendencies for cognitively appealing and not necessarily informative, or valuable, content.

Significance

Use of AI in the production of text through Large Language Models (LLMs) is widespread and growing, with potential applications in journalism, copywriting, academia, and other writing tasks. As such, it is important to understand whether text produced or summarized by LLMs exhibits biases. The studies presented here demonstrate that the LLM ChatGPT-3 reflects human biases for certain types of content in its production. The presence of these biases in LLM output has implications for its common use, as it may magnify human tendencies for content which appeals to these biases.

A pdf is here.

Here are the main points:

• LLMs display stereotype-consistent biases, just like humans: Similar to people, LLMs were more likely to preserve information confirming stereotypes over information contradicting them.
• Bias location might differ: Unlike humans, whose biases can shift throughout the retelling process, LLMs primarily showed bias in the first retelling. This suggests their biases stem from their training data rather than a complex cognitive process.
• Simple summarization may suffice: The first retelling step caused the most content change, implying that even a single summarization by an LLM can reveal its biases. This simplifies the research needed to detect and analyze LLM bias.
• Prompting for different viewpoints could reduce bias: The study suggests experimenting with different prompts to encourage LLMs to consider broader perspectives and potentially mitigate inherent biases.

Folk Psychological Attributions of Consciousness to Large Language Models

Colombatto, C., & Fleming, S. M.

(2023, November 22). PsyArXiv

https://doi.org/10.31234/osf.io/5cnrv

Abstract

Technological advances raise new puzzles and challenges for cognitive science and the study of how humans think about and interact with artificial intelligence (AI). For example, the advent of Large Language Models and their human-like linguistic abilities has raised substantial debate regarding whether or not AI could be conscious. Here we consider the question of whether AI could have subjective experiences such as feelings and sensations (“phenomenological consciousness”). While experts from many fieldshave weighed in on this issue in academic and public discourse, it remains unknown how the general population attributes phenomenology to AI. We surveyed a sample of US residents (N=300) and found that a majority of participants were willing to attribute phenomenological consciousness to LLMs. These attributions were robust, as they predicted attributions of mental states typically associated with phenomenology –but also flexible, as they were sensitive to individual differences such as usage frequency. Overall, these results show how folk intuitions about AI consciousness can diverge from expert intuitions –with important implications for the legal and ethical status of AI.

The preprint is here.

My summary:

The results of the study show that people are generally more likely to attribute consciousness to LLMs than to other non-human entities, such as animals, plants, and robots. However, the level of consciousness attributed to LLMs is still relatively low, with most participants rating them as less conscious than humans. The authors argue that these findings reflect the influence of folk psychology, which is the tendency to explain the behavior of others in terms of mental states.

The authors also found that people's attributions of consciousness to LLMs were influenced by their beliefs about the nature of consciousness and their familiarity with LLMs. Participants who were more familiar with LLMs were more likely to attribute consciousness to them, and participants who believed that consciousness is a product of complex computation were also more likely to attribute consciousness to LLMs.

Overall, the study suggests that people are generally open to the possibility that LLMs may be conscious, but they also recognize that LLMs are not as conscious as humans. These findings have implications for the development and use of LLMs, as they suggest that people may be more willing to trust and interact with LLMs that they believe are conscious.