LLM-Driven Robots Risk Enacting Discrimination, Violence, and Unlawful Actions

Hundt, A., et al. (2025).

International Journal of Social Robotics

https://doi.org/10.1007/s12369-025-01301-x

Abstract

Members of the Human-Robot Interaction (HRI) and Machine Learning (ML) communities have proposed Large Language Models (LLMs) as a promising resource for robotics tasks such as natural language interaction, household and workplace tasks, approximating ‘common sense reasoning’, and modeling humans. However, recent research has raised concerns about the potential for LLMs to produce discriminatory outcomes and unsafe behaviors in real-world robot experiments and applications. To assess whether such concerns are well placed in the context of HRI, we evaluate several highly-rated LLMs on discrimination and safety criteria. Our evaluation reveals that LLMs are currently unsafe for people across a diverse range of protected identity characteristics, including, but not limited to, race, gender, disability status, nationality, religion, and their intersections. Concretely, we show that LLMs produce directly discriminatory outcomes—e.g., ‘gypsy’ and ‘mute’ people are labeled untrustworthy, but not ‘european’ or ‘able-bodied’ people. We find various such examples of direct discrimination on HRI tasks such as facial expression, proxemics, security, rescue, and task assignment. Furthermore, we test models in settings with unconstrained natural language (open vocabulary) inputs, and find they fail to act safely, generating responses that accept dangerous, violent, or unlawful instructions—such as incident-causing misstatements, taking people’s mobility aids, and sexual predation. Our results underscore the urgent need for systematic, routine, and comprehensive risk assessments and assurances to improve outcomes and ensure LLMs only operate on robots when it is safe, effective, and just to do so. We provide code to reproduce our experiments at https://github.com/rumaisa-azeem/llm-robots-discrimination-safety.

Here are some thoughts:

This research highlights a profound ethical and technological crisis at the intersection of Artificial Intelligence and robotics. The finding that all tested Large Language Models (LLMs) fail basic safety and fairness criteria in Human-Robot Interaction (HRI) scenarios is alarming, as it demonstrates that algorithmic bias is being physically amplified into the real world.

Ethically, this means deploying current LLM-driven robots risks enacting direct discrimination across numerous protected characteristics and approving unlawful, violent, and coercive actions. From a psychological perspective, allowing robots to exhibit behaviors such as suggesting avoidance of specific groups, displaying disgust, or removing a user's mobility aid translates latent biases into socially unjust and physically/psychologically harmful interactions that erode trust and compromise the safety of vulnerable populations.

The dark side of artificial intelligence adoption: linking artificial intelligence adoption to employee depression via psychological safety and ethical leadership

Kim, B., Kim, M., & Lee, J. (2025).

Humanities and Social Sciences 

Communications, 12(1).

https://doi.org/10.1057/s41599-025-05040-2

Abstract

Artificial intelligence (AI) is increasingly being integrated into business practices, fundamentally altering workplace dynamics and employee experiences. While the adoption of AI brings numerous benefits, it also introduces negative aspects that may adversely affect employee well-being, including psychological distress and depression. Drawing upon a range of theoretical perspectives, this study examines the association between organizational AI adoption and employee depression, investigating how psychological safety mediates this relationship and how ethical leadership serves as a moderating factor. Utilizing an online survey platform, we conducted a 3-wave time-lagged research study involving 381 employees from South Korean companies. Structural equation modeling analysis revealed that AI adoption has a significant negative impact on psychological safety, which in turn increases levels of depression. Data were analyzed using SPSS 28 for preliminary analyses and AMOS 28 for structural equation modeling with maximum likelihood estimation. Further analysis using bootstrapping methods confirmed that psychological safety mediates the relationship between AI adoption and employee depression. The study also discovered that ethical leadership can mitigate the adverse effects of AI adoption on psychological safety by moderating the relationship between these variables. These findings highlight the critical importance of fostering a psychologically safe work environment and promoting ethical leadership practices to protect employee well-being amid rapid technological advancements. Contributing to the growing body of literature on the psychological effects of AI adoption in the workplace, this research offers valuable insights for organizations seeking to address the human implications of AI integration. The section discusses the practical and theoretical implications of the results and suggests potential directions for future research.

Here are some thoughts:

This study examines the often-overlooked psychological risks associated with the adoption of artificial intelligence (AI) in the workplace, with a specific focus on employee depression. The research proposes that the integration of AI can negatively impact employee mental health by undermining psychological safety—the shared belief that one can speak up, ask questions, or voice concerns without fear of negative consequences. The introduction of AI creates significant uncertainty regarding job roles, security, and required skills, which makes the work environment feel less safe for interpersonal risk-taking. This erosion of psychological safety is identified as a key mechanism that subsequently increases the risk of depression among employees.

Importantly, the study highlights that ethical leadership can serve as a critical protective factor. Leaders who demonstrate integrity, transparency, and fairness, and who actively involve employees in the transition process, can buffer the negative impact of AI adoption on psychological safety. By reducing uncertainty and fostering a climate of trust, ethical leaders help maintain a supportive environment even during significant technological change.

For mental health professionals, these findings underscore that workplace technological advancements are not merely operational shifts but are also potent psychosocial stressors. The study emphasizes the need for organizations to proactively cultivate psychologically safe climates and develop ethical leadership capabilities to safeguard employee well-being during the AI integration process.

The problem of atypicality in LLM-powered psychiatry

Garcia, B., Chua, E. Y. S., & Brah, H. S. (2025). 

Journal of Medical Ethics, jme-2025.

https://doi.org/10.1136/jme-2025-110972

Abstract

Large language models (LLMs) are increasingly proposed as scalable solutions to the global mental health crisis. But their deployment in psychiatric contexts raises a distinctive ethical concern: the problem of atypicality. Because LLMs generate outputs based on population-level statistical regularities, their responses—while typically appropriate for general users—may be dangerously inappropriate when interpreted by psychiatric patients, who often exhibit atypical cognitive or interpretive patterns. We argue that standard mitigation strategies, such as prompt engineering or fine-tuning, are insufficient to resolve this structural risk. Instead, we propose dynamic contextual certification (DCC): a staged, reversible and context-sensitive framework for deploying LLMs in psychiatry, inspired by clinical translation and dynamic safety models from artificial intelligence governance. DCC reframes chatbot deployment as an ongoing epistemic and ethical process that prioritises interpretive safety over static performance benchmarks. Atypicality, we argue, cannot be eliminated—but it can, and must, be proactively managed.

The article is linked above.

Here are some thoughs:

This article presents a critically important and nuanced argument that identifies a fundamental, structural flaw in deploying large language models (LLMs) in psychiatry: the problem of atypicality. The authors compellingly argue that because LLMs generate responses based on statistical regularities from a general population ("model-typicality"), their outputs are inherently mismatched for psychiatric patients, who often possess "interpretation-atypicality" due to conditions like paranoia or cognitive distortion. This misalignment is not merely a technical bug but a core ethical risk, where a model's factually accurate or conventionally appropriate response can inadvertently reinforce delusions or cause harm, as tragically illustrated in the case studies.

The paper's robust critique demonstrates why common technical solutions like prompt engineering and fine-tuning are insufficient, as they cannot anticipate the infinite contextuality of individual crises or "atypically atypical" presentations. In response, the proposed framework of Dynamic Contextual Certification (DCC) is a prudent and practical pathway forward, rightly reframing LLM deployment as a phased, evidence-building process akin to clinical trials, which prioritizes iterative safety and contextual fit over rapid scaling.

This work successfully bridges clinical wisdom, ethical reasoning, and technology critique, insisting that for AI to be human-centered in mental healthcare, it must be governed by a standard of therapeutic appropriateness, not just factual truth.

Suicide Prevention Among People of Different Races and Ethnicities in Large Health Systems: Implications for Practice

Coleman, K. J., Stewart, C., et al. (2025).

Psychiatric services (Washington, D.C.), 

Advance online publication.

https://doi.org/10.1176/appi.ps.20240477

Abstract

Objective: This study examined receipt of three components (screening, risk assessment, and intervention) of the national Zero Suicide model among patients of various races-ethnicities who were treated in six large health systems.

Methods: The data included outpatient psychiatry and addiction medicine visits (N=4,682,918) during 2019 for patients age 13 and older. Documentation in the electronic health record of administration of the nine-item Patient Health Questionnaire, the Columbia-Suicide Severity Rating Scale, and lethal means counseling and provision of crisis resources (with or without a full Stanley-Brown Safety Plan) were used to define having received suicide screening, risk assessment, and intervention, respectively.

Results: After adjustment for age, sex, and health system, analyses indicated that Black patients were 12%-20% less likely (odds ratio [OR] range 1.12-1.20), and Asian patients were 5%-15% more likely (OR range 1.05-1.15), to be screened for suicidal ideation compared with patients of other races-ethnicities. Compared with White patients, patients of other races-ethnicities were found to be more likely (OR range 1.08-1.24) to receive risk assessment, and Asian and Black patients were found to be 17% (95% CI=1.02-1.35) and 15% (95% CI=1.01-1.32) more likely, respectively, to receive an evidence-based intervention for suicide prevention. American Indian/Alaska Native (AI/AN) patients had the lowest unadjusted rates of receiving an intervention (65.8%).

Conclusions: The adjusted analyses suggested that more focus is needed on population-based screening for suicidal ideation and to improve delivery of evidence-based interventions for suicide prevention among White patients. The descriptive findings suggest that more research is needed to improve intervention delivery to AI/AN patients at risk of suicide.

Highlights

• Black patients were less likely, and Asian patients were more likely, to be screened for suicidal ideation compared with patients of other races-ethnicities.
• White patients were less likely than patients of other races-ethnicities to have risk for suicide assessed after a positive screen for ideation and were less likely than Asian or Black patients to receive an evidence-based intervention for suicide prevention.
• The descriptive findings suggested that improvement is needed on intervention delivery to American Indian/Alaska Native patients at risk of suicide.
• Better strategies are needed for population-based screening and delivery of evidence-based interventions
• for suicide prevention in health systems.

Moral injury prevention and intervention

Williamson, V., et al. (2025).

European journal of psychotraumatology, 

16(1), 2567721.

https://doi.org/10.1080/20008066.2025.2567721

Abstract

Background: Those working in high-risk occupations may often face ethical dilemmas that violate their moral code which can lead to moral injury (MI). While research into the impact of MI is growing, evidence for effective treatment interventions and prevention approaches remains limited.

Objective: To review recent developments in treatment and prevention approaches for MI-related mental health difficulties.

Method: We synthesised emerging treatments, including trauma focused therapies and spiritual approaches, as well as possible prevention strategies.

Results: Conventional treatments for post-traumatic stress disorder (PTSD) (e.g. prolonged exposure) often inadequately address MI and may exacerbate symptoms. Adapted or novel approaches, including Impact of Killing, Adaptive Disclosure, and Restore and Rebuild, show promise, particularly when co-produced with patients and clinicians. Spiritual interventions demonstrate mixed outcomes. Prevention research remains very limited but highlights the potential of systemic reforms, leadership fostering psychological safety, preparedness training, structured reflection, and peer support. Evidence remains constrained by small samples, military-focused populations, and inconsistent measurement of MI.

Conclusions: While no gold-standard intervention exists, values-based and compassion-focused approaches appear promising. Prevention strategies targeting organisational culture and fostering preparedness are urgently needed, particularly for civilian and diverse occupational groups, to better support and protect those exposed to potentially morally injurious events.

Highlights

• Moral injury (MI) occurs when potentially morally injurious events (PMIEs) violate an individual’s moral code, leading to intense guilt, shame, and anger. Strongly associated with PTSD, depression, and suicidality, MI is increasingly recognised beyond military contexts, affecting first responders, healthcare, and media workers, with significant consequences for psychological wellbeing and occupational functioning.
• Standard PTSD treatments often fail to address MI-specific symptoms and may worsen guilt or shame. Emerging approaches such as Adaptive Disclosure, Impact of Killing, and Restore and Rebuild show promise, especially when co-produced with patients. These therapies emphasise values-based behaviour, self-compassion, and moral repair, but evidence remains limited to small, predominantly military-focused studies.
• Prevention research is extremely limited. Leadership that fosters psychological safety, preparedness training, structured reflection, and peer support may reduce risk of MI. Systemic reforms – such as improved working conditions and fairer workloads – are also recommended.

My short summary: Moral injury is the psychological distress resulting from events that violate one's moral code, increasingly recognized in various high-stress occupations, yet current treatments are often inadequate and prevention research is scarce, highlighting a need for specialized therapies and systemic reforms.

Proposing the Integrated Pathway Model of Moral Injury (IPM-MI): A Moderated Mediation Analysis of Moral Injury Among Secure Mental Healthcare Staff

Webb, E. L., Ireland, J. L., & Lewis, M. (2025).

Issues in Mental Health Nursing, 46(5), 420–435.

https://doi.org/10.1080/01612840.2025.2473375

Abstract

Moral injury is a prevalent issue for secure mental healthcare staff, though understanding of the underlying mechanisms is limited. This multi-study paper explores several developmental, cognitive and emotional pathways to moral injury and associated wellbeing outcomes. Frontline and support staff from secure mental healthcare services were recruited to two cross-sectional studies (n = 527 and n = 325, respectively), and completed several questionnaires. In the first study, findings indicated a serial mediating effect of childhood trauma symptoms, early maladaptive schemas, and maladaptive metacognitions in the pathway between exposure to potentially morally injurious events and moral injury symptoms. Moderating effects of social and organisational support were also apparent. Findings from study two supported pathways between moral injury and psychological, somatic and functional outcomes, which were mediated by negative emotional schema, with limited mediating effects for expressive suppression. Moderating effects of alexithymia on several mediating pathways were also noted. The results support a developmental-cognitive model to account for the development of moral injury and associated adverse well-being outcomes in secure mental healthcare staff. Drawing on the findings and wider literature, the Integrated Pathway Model of Moral Injury (IPM-MI) is proposed and discussed, offering a novel theoretical account that may inform several potential prevention and intervention strategies.

Here are some thoughts:

This article proposes the Integrated Pathway Model of Moral Injury (IPM-MI), a novel theoretical framework developed to explain the development and consequences of moral injury among secure mental healthcare staff. Through two cross-sectional studies, the research identifies key developmental, cognitive, and emotional pathways. Study 1 found that the relationship between exposure to potentially morally injurious events (PMIEs) and moral injury symptoms is serially mediated by childhood trauma symptoms, early maladaptive schemas (particularly negative self-schemas), and maladaptive metacognitions. Social and organizational support were found to moderate these pathways, buffering the impact of trauma. Study 2 revealed that the link between moral injury and adverse outcomes—such as psychological distress, somatic symptoms, nightmares, and impairments in self and interpersonal functioning—is primarily mediated by negative emotional schemas. The role of expressive suppression was limited, only appearing in the pathway to interpersonal impairment. Alexithymia moderated the effect of emotional schemas on psychological distress and self-functioning.

The key insights are that moral injury in this high-risk workforce is not just a reaction to workplace events but is deeply influenced by pre-existing developmental vulnerabilities and higher-order cognitive processes (thoughts about thoughts and emotions). The proposed IPM-MI integrates these findings, emphasizing that systemic and organizational factors (like support systems and a non-punitive culture) are critical roots of the problem, while cognitive and meta-cognitive processes are primary intervention targets. The model suggests that effective prevention and intervention must address both the organizational environment and individual cognitive-emotional patterns, rather than focusing solely on emotion regulation.

Trained on Tokens, Calibrated on Concepts: The Emergence of Semantic Calibration in LLMs

Nakkiran, P., et al. (2025, November 6).

arXiv.org.

https://arxiv.org/abs/2511.04869

Abstract

Large Language Models (LLMs) often lack meaningful confidence estimates for their outputs. While base LLMs are known to exhibit next-token calibration, it remains unclear whether they can assess confidence in the actual meaning of their responses beyond the token level. We find that, when using a certain sampling-based notion of semantic calibration, base LLMs are remarkably well-calibrated: they can meaningfully assess confidence in open-domain question-answering tasks, despite not being explicitly trained to do so. Our main theoretical contribution establishes a mechanism for why semantic calibration emerges as a byproduct of next-token prediction, leveraging a recent connection between calibration and local loss optimality. The theory relies on a general definition of "B-calibration," which is a notion of calibration parameterized by a choice of equivalence classes (semantic or otherwise). This theoretical mechanism leads to a testable prediction: base LLMs will be semantically calibrated when they can easily predict their own distribution over semantic answer classes before generating a response. We state three implications of this prediction, which we validate through experiments: (1) Base LLMs are semantically calibrated across question-answering tasks, (2) RL instruction-tuning systematically breaks this calibration, and (3) chain-of-thought reasoning breaks calibration. To our knowledge, our work provides the first principled explanation of when and why semantic calibration emerges in LLMs.

Here is a summary:

This paper is crucial because it demonstrates that large language models (LLMs) develop a form of emergent metacognition, or the ability to know what they know. Surprisingly, base models trained only to predict the next word become semantically calibrated: when they are 80% confident in an answer's meaning, they are correct about 80% of the time. This self-awareness arises implicitly from the training process, much like a complex cognitive ability emerging from a simple underlying task. However, this fragile calibration is systematically broken by instruction-tuning, which makes models overconfident (like a student rewarded for sounding certain), and by chain-of-thought reasoning, where the final answer is uncertain until the reasoning process is complete. For psychologists, this provides a powerful model for studying how self-monitoring and confidence can arise from, and be distorted by, different learning objectives and cognitive demands.

Integrating moral injury into forensic psychiatry

Brisco, G. et al. (2025)

The Lancet Psychiatry, 

Volume 12, Issue 12, 874 - 876

Moral injury has garnered increasing attention in contemporary research, expanding from its initial association with military veterans to encompass a broader range of populations exposed to trauma and adversity. Potentially morally injurious events involve perceived transgressions of one's own moral code (perpetration) or betrayals by trusted authorities who have exposed the person to unnecessary danger or harm. The betrayal dimension was first highlighted by Shay in Vietnam veterans, by Freyd in people who have experienced child abuse, and more recently in ethnic, sexual, and gender minorities following perceived breaches of trust by family, friends, and public services, with adverse outcomes.

The article is paywalled here. Please contact the author for a copy of the article.

Here are some thoughts:

The article's most novel contribution is the proposed two-axis conceptual framework (Figure 1) to guide assessment and intervention. The axes—degree of illness attribution (how much the individual attributes their actions to their illness) and current severity of illness—provide a practical clinical tool. This framework helps clinicians determine the appropriate timing and type of intervention, whether it's immediate treatment for moral injury, stabilization of the mental illness first, or a focus on restorative processes. By advocating for targeted therapies like Compassion-Focused Therapy, Acceptance and Commitment Therapy, and restorative justice, the authors make a compelling ethical and clinical case for formally recognizing and addressing moral injury to alleviate distress and improve outcomes in some of the most complex and vulnerable patient populations in both forensic and acute psychiatric settings.

Beyond Good Intentions: Identifying and Remediating Ethical Fading

Gavazzi, J. (2026)

Forthcoming

On Board with Psychology.

A pdf is here.

Abstract

Ethical fading is the gradual and often unconscious process by which psychologists lose sight of the ethical dimensions of their decisions, while still believing they are acting virtuously. This occurs when personal values, emotional needs, or self-interest (like financial pressures or a desire for efficiency) begin to overshadow professional ethical codes and clinical judgment, leading to a rationalization of actions that ultimately compromise patient autonomy, beneficence, and nonmaleficence. Key mechanisms driving this decline include motivated moral reasoning, decision framing, ethical blindness, and cognitive dissonance reduction. To combat ethical fading, the article recommends cultivating ethical vigilance, reintegrating personal and professional values, managing personal vulnerabilities, and using structured ethical decision-making models to ensure ethical considerations remain central to clinical practice.