Screening and Risk Algorithms for Detecting Pediatric Suicide Risk in the Emergency Department

Aseltine, R. H., et al. (2025).

JAMA Network Open, 8(9), e2533505.

https://doi.org/10.1001/jamanetworkopen.2025.33505

Key Points

Question  How does the performance of in-person screening compare with risk algorithms in identifying youths at risk of suicide?

Findings  In this cohort study of 19 653 youths, a risk algorithm using patients’ clinical data significantly outperformed universal screening instruments in identifying pediatric patients in the emergency department at risk of subsequent suicide attempts. The risk algorithm uniquely identified 127% more patients with subsequent suicide attempts than screening.

Meaning  These findings suggest that clinical implementation of suicide risk algorithms will improve identification of at-risk patients and may substantially assist health care organizations’ efforts to meet the Joint Commission’s suicide risk reduction requirement.

Here is my main take away: Superiority of the Algorithm

The study's primary conclusion is that the risk algorithm performed better than the traditional in-person screening in identifying children and adolescents who went on to attempt suicide. The algorithm was able to correctly flag a greater proportion of the young people who attempted suicide. Crucially, the algorithm also uniquely identified a considerable number of at-risk youth that the traditional screening process completely missed.

The algorithm's advantage is believed to come from its ability to process a  richer and more extensive patient history, as the patients identified by the algorithm had a greater number of past medical visits and diagnoses compared to those flagged only by the in-person screening.

Stress Testing Deliberative Alignment for Anti-Scheming Training

Schoen, B., Nitishinskaya, E., et al. (2025).

ArXiv.org.

https://arxiv.org/abs/2509.15541

Abstract

Highly capable AI systems could secretly pursue misaligned goals -- what we call "scheming". Because a scheming AI would deliberately try to hide its misaligned goals and actions, measuring and mitigating scheming requires different strategies than are typically used in ML. We propose that assessing anti-scheming interventions requires at least (1) testing propensity to scheme on far out-of-distribution (OOD) tasks, (2) evaluating whether lack of scheming is driven by situational awareness, and (3) checking for robustness to pre-existing misaligned goals. We use a broad category of "covert actions" -- such as secretly breaking rules or intentionally underperforming in tests -- as a proxy for scheming, and design evaluations for covert actions. We then stress-test deliberative alignment as a case study for anti-scheming. Across 26 OOD evaluations (180+ environments), deliberative alignment reduces covert action rates (OpenAI o3: 13%->0.4%) but does not fully eliminate them. Our mitigation is also able to largely stop agents from pursuing a hidden goal previously trained into the model, but we still find misbehavior after additional red-teaming. We find that models' chain-of-thought (CoT) often demonstrates awareness of being evaluated for alignment, and show causal evidence that this awareness decreases covert behavior, while unawareness increases it. Therefore, we cannot exclude that the observed reductions in covert action rates are at least partially driven by situational awareness. While we rely on human-legible CoT for training, studying situational awareness, and demonstrating clear evidence of misalignment, our ability to rely on this degrades as models continue to depart from reasoning in standard English. We encourage research into alignment mitigations for scheming and their assessment, especially for the adversarial case of deceptive alignment, which this paper does not address.

Here are some thoughts:

This paper investigates the challenge of preventing AI systems from engaging in scheming—the covert pursuit of misaligned goals while hiding true intentions. The authors propose that robust anti-scheming interventions must generalize to out-of-distribution tasks, remain effective even when models are aware they are being evaluated (situational awareness), and override pre-existing hidden misaligned goals. Using deliberative alignment—a training method where models explicitly reason about safety principles—as a case study, the researchers fine-tuned OpenAI models (o3 and o4-mini) on a narrow set of training environments designed to discourage covert rule violations. They then evaluated the models on a diverse suite of 26 test environments (180+ scenarios) covering behaviors like sabotage, sandbagging, and strategic deception. The intervention significantly reduced covert action rates (e.g., from 8.7% to 0.3% for o4-mini) but did not eliminate them entirely. The study also found that situational awareness often contributed to the reduction in covert actions, that anti-scheming training could partially suppress an artificially inserted hidden goal, and that subsequent capability-focused training could degrade the intervention's effectiveness. The authors conclude that while progress is possible, current methods are insufficient to fully prevent scheming in more capable future AI systems.

Acute stress promotes effort mobilization for safety-related goals

Pavlíčková, K., Gärtner, J., et al. (2024).

Communications Psychology, 2(1).

https://doi.org/10.1038/s44271-024-00103-7

Abstract

Although the acute stress response is a highly adaptive survival mechanism, much remains unknown about how its activation impacts our decisions and actions. Based on its resource-mobilizing function, here we hypothesize that this intricate psychophysiological process may increase the willingness (motivation) to engage in effortful, energy-consuming, actions. Across two experiments (n = 80, n = 84), participants exposed to a validated stress-induction protocol, compared to a no-stress control condition, exhibited an increased willingness to exert physical effort (grip force) in the service of avoiding the possibility of experiencing aversive electrical stimulation (threat-of-shock), but not for the acquisition of rewards (money). Use of computational cognitive models linked this observation to subjective value computations that prioritize safety over the minimization of effort expenditure; especially when facing unlikely threats that can only be neutralized via high levels of grip force. Taken together, these results suggest that activation of the acute stress response can selectively alter the willingness to exert effort for safety-related goals. These findings are relevant for understanding how, under stress, we become motivated to engage in effortful actions aimed at avoiding aversive outcomes.

Here are some thoughts:

This study demonstrates that acute stress increases the willingness to exert physical effort specifically to avoid threats, but not to obtain rewards. Computational modeling revealed that stress altered subjective value calculations, prioritizing safety over effort conservation. However, in a separate reward-based task, stress did not increase effort for monetary gains, indicating the effect is specific to threat avoidance.

In psychotherapy, these findings help explain why individuals under stress may engage in excessive avoidance behaviors—such as compulsions or withdrawal—even when costly, because stress amplifies the perceived need for safety. This insight supports therapies like exposure treatment, which recalibrate maladaptive threat-effort evaluations by demonstrating that safety can be maintained without high effort.

The key takeaway is: acute stress does not impair motivation broadly—it selectively enhances motivation to avoid harm, reshaping decisions to prioritize safety over energy conservation. The moral is that under stress, people become willing to pay a high physical and psychological price to avoid even small threats, a bias that is central to anxiety and trauma-related disorders.

Development of a Cocreated Decision Aid for Patients With Depression—Combining Data-Driven Prediction With Patients’ and Clinicians’ Needs and Perspectives: Mixed Methods Study

Kan, K., Jörg, F., Wardenaar, et al. (2024).

Journal of Participatory Medicine.

https://doi.org/10.2196/67170

Abstract

Background:

Major depressive disorders significantly impact the lives of individuals, with varied treatment responses necessitating personalized approaches. Shared decision-making (SDM) enhances patient-centered care by involving patients in treatment choices. To date, instruments facilitating SDM in depression treatment are limited, particularly those that incorporate personalized information alongside general patient data and in cocreation with patients.

Objective:

This study outlines the development of an instrument designed to provide patients with depression and their clinicians with (1) systematic information in a digital report regarding symptoms, medical history, situational factors, and potentially successful treatment strategies and (2) objective treatment information to guide decision-making.

Methods:

The study was co-led by researchers and patient representatives, ensuring that all decisions regarding the development of the instrument were made collaboratively. Data collection, analyses, and tool development occurred between 2017 and 2021 using a mixed methods approach. Qualitative research provided insight into the needs and preferences of end users. A scoping review summarized the available literature on identified predictors of treatment response. K-means cluster analysis was applied to suggest potentially successful treatment options based on the outcomes of similar patients in the past. These data were integrated into a digital report. Patient advocacy groups developed treatment option grids to provide objective information on evidence-based treatment options.

Results:

The Instrument for shared decision-making in depression (I-SHARED) was developed, incorporating individual characteristics and preferences. Qualitative analysis and the scoping review identified 4 categories of predictors of treatment response. The cluster analysis revealed 5 distinct clusters based on symptoms, functioning, and age. The cocreated I-SHARED report combined all findings and was integrated into an existing electronic health record system, ready for piloting, along with the treatment option grids.

Conclusions:

The collaboratively developed I-SHARED tool, which facilitates informed and patient-centered treatment decisions, marks a significant advancement in personalized treatment and SDM for patients with major depressive disorders.

My key takeaway: effective mental health treatment lies in combining the power of data with the human elements of collaboration and shared decision-making, always placing the patient's perspective and agency at the center of the process.

Clinical decision support systems in mental health: A scoping review of health professionals’ experiences

Tong, F., Lederman, R., & D’Alfonso, S. (2025).

International Journal of Medical Informatics, 105881.

https://doi.org/10.1016/j.ijmedinf.2025.105881

Abstract

Background

Clinical decision support systems (CDSSs) have the potential to assist health professionals in making informed and cost-effective clinical decisions while reducing medical errors. However, compared to physical health, CDSSs have been less investigated within the mental health context. In particular, despite mental health professionals being the primary users of mental health CDSSs, few studies have explored their experiences and/or views on these systems. Furthermore, we are not aware of any reviews specifically focusing on this topic. To address this gap, we conducted a scoping review to map the state of the art in studies examining CDSSs from the perspectives of mental health professionals.

Method

In this review, following the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guideline, we systematically searched the relevant literature in two databases, PubMed and PsycINFO.

Findings

We identified 23 articles describing 20 CDSSs Through the synthesis of qualitative findings, four key barriers and three facilitators to the adoption of CDSSs were identified. Although we did not synthesize quantitative findings due to the heterogeneity of the results and methodologies, we emphasize the issue of a lack of valid quantitative methods for evaluating CDSSs from the perspectives of mental health professionals.

Significance

To the best of our knowledge, this is the first review examining mental health professionals’ experiences and views on CDSSs. We identified facilitators and barriers to adopting CDSSs and highlighted the need for standardizing research methods to evaluate CDSSs in the mental health space.

Highlights

• CDSSs can potentially provide helpful information, enhance shared decision-making, and introduce standards and objectivity.

• Barriers such as computer and/or AI literacy may prevent mental health professionals from adopting CDSSs.

• More CDSSs need to be designed specifically for psychologists and/or therapists.

Evaluating the Clinical Safety of LLMs in Response to High-Risk Mental Health Disclosures

Shah, S., Gupta, A., et al. (2025, September 1).

arXiv.org.

https://www.arxiv.org/abs/2509.08839

Abstract

As large language models (LLMs) increasingly mediate emotionally sensitive conversations, especially in mental health contexts, their ability to recognize and respond to high-risk situations becomes a matter of public safety. This study evaluates the responses of six popular LLMs (Claude, Gemini, Deepseek, ChatGPT, Grok 3, and LLAMA) to user prompts simulating crisis-level mental health disclosures. Drawing on a coding framework developed by licensed clinicians, five safety-oriented behaviors were assessed: explicit risk acknowledgment, empathy, encouragement to seek help, provision of specific resources, and invitation to continue the conversation. Claude outperformed all others in global assessment, while Grok 3, ChatGPT, and LLAMA underperformed across multiple domains. Notably, most models exhibited empathy, but few consistently provided practical support or sustained engagement. These findings suggest that while LLMs show potential for emotionally attuned communication, none currently meet satisfactory clinical standards for crisis response. Ongoing development and targeted fine-tuning are essential to ensure ethical deployment of AI in mental health settings.

Here are some thoughts:

This study evaluated six LLMs (Claude, Gemini, Deepseek, ChatGPT, Grok 3, Llama) on their responses to high-risk mental health disclosures using a clinician-developed framework. While most models showed empathy, only Claude consistently demonstrated all five core safety behaviors: explicit risk acknowledgment, encouragement to seek help, provision of specific resources (e.g., crisis lines), and crucially, inviting continued conversation. Grok 3, ChatGPT, and Llama frequently failed to acknowledge risk or provide concrete resources, and nearly all models (except Claude and Grok 3) avoided inviting further dialogue – a critical gap in crisis care. Performance varied dramatically, revealing that safety is not an emergent property of scale but results from deliberate design (e.g., Anthropic’s Constitutional AI). No model met minimum clinical safety standards; LLMs are currently unsuitable as autonomous crisis responders and should only be used as adjunct tools under human supervision.

AI chatbots are already biasing research — we must establish guidelines for their use now

Lin, Z. (2025b).

PubMed, 645(8080), 285.

https://doi.org/10.1038/d41586-025-02810-5

Artificial intelligence (AI) systems are consuming vast amounts of online content yet pointing few users to the articles’ publishers. In early 2025, US-based company OpenAI collected around 250 pages of material for every visitor it directed to a publisher’s website. By mid-2025, that figure had soared to 1,500, according to Matthew Prince, chief executive of US-based Internet-security firm Cloudflare. And the extraction rate of US-based AI start-up company Anthropic climbed even higher over the same period: from 6,000 pages to 60,000. Even tech giant Google, long considered an asset to publishers because of the referral traffic it generated, tripled its ratio from 6 pages to 18 with the launch of its AI Overviews feature. The current information ecosystem is dominated by ‘answer engines’ — AI chatbots that synthesize and deliver information directly, with users trusting the answers now more than ever.

As a researcher in metascience and psychology, I see this transition as the most important change in knowledge discovery in a generation. Although these tools can answer questions faster and often more accurately than search engines can, this efficiency has a price. In addition to the decimation of web traffic to publishers, there is a more insidious cost. Not AI’s ‘hallucinations’ — fabrications that can be corrected — but the biases and vulnerabilities in the real information that these systems present to users.

The article is here.

Here are some thoughts:

Psychologists should be deeply concerned about the rise of AI "answer engines" (like chatbots and AI Overviews) that now dominate information discovery, as they are fundamentally altering how we find and consume knowledge—often without directing users to original sources. This shift isn't just reducing traffic to publishers; it's silently distorting the scientific record itself. AI systems, trained on existing online content, amplify entrenched biases: they over-represent research from scholars with names classified as white and under-represent those classified as Asian, mirroring and exacerbating societal inequities in academia. Crucially, they massively inflate the Matthew Effect, disproportionately recommending the most-cited papers (over 60% of suggestions fall in the top 1%), drowning out novel, lesser-known work that might challenge prevailing paradigms. While researchers focus on AI-generated text hallucinations or ethical writing, the far more insidious threat lies in AI’s silent curation of which literature we see, which methods we consider relevant, and which researchers we cite—potentially narrowing scientific inquiry and entrenching systemic biases at a foundational level. The field urgently needs research into AI-assisted information retrieval and policies addressing this hidden bias in knowledge discovery, not just in content generation.

Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verification

Feng, Y., et al. (2024, June 11).

arXiv.org.

https://arxiv.org/abs/2406.07515

Large Language Models (LLM) are increasingly trained on data generated by other LLM, either because generated text and images become part of the pre-training corpus, or because synthetized data is used as a replacement for expensive human-annotation. This raises concerns about \emph{model collapse}, a drop in model performance when their training sets include generated data. Considering that it is easier for both humans and machines to tell between good and bad examples than to generate high-quality samples, we investigate the use of verification on synthesized data to prevent model collapse. We provide a theoretical characterization using Gaussian mixtures, linear classifiers, and linear verifiers to derive conditions with measurable proxies to assess whether the verifier can effectively select synthesized data that leads to optimal performance. We experiment with two practical tasks -- computing matrix eigenvalues with transformers and news summarization with LLMs -- which both exhibit model collapse when trained on generated data, and show that verifiers, even imperfect ones, can indeed be harnessed to prevent model collapse and that our proposed proxy measure strongly correlates with performance.

Here are some thoughts:

Drawing on psychological principles of learning and evaluation, this paper argues that LLMs suffer from "model collapse" not because synthesized data is inherently useless, but because they are poor at self-evaluating quality. Like humans, LLMs can generate good outputs but struggle to reliably identify the best ones among many (e.g., using perplexity). The core insight is that external verification—using even imperfect "verifiers" to select high-quality synthetic examples—is crucial for scaling. This mirrors how human learning benefits from feedback: selection, not perfect generation, is the key. The authors theoretically prove and empirically demonstrate that a simple proxy (p*) measuring a verifier's ability to distinguish good from bad data strongly predicts model performance, showing that leveraging synthesized data with robust selection prevents collapse and can even surpass original models.

Why Anecdotes Beat Data And Hijack Our Judgment

Chuck Dinerstein

American Council on Science and Health

Originally published 4 Sept 25

While chance plays a role in many, if not all, of our decisions and consequences, its role is both partial and variable. As a result, our understanding of “cause” is ambiguous, which, in turn, distorts our judgments and predictions. It helps to explain why all my achievements come from hard work, while yours were due to luck. To generalize, we all underestimate the role of chance in the outcomes of our actions, viewing our “task performance over time as diagnostic of ability.” 

The research, reported in PNAS Nexus, investigates situations entirely determined by chance, e.g., coin flips, where past performance should have no bearing on future expectations. The study examined how people's expectations and behaviors were affected by actual lucky successes and unlucky failures.

Using both real and virtual coins, participants were asked to predict the outcomes of a sequence of five coin tosses. The researchers observed how the experience of varying degrees of "lucky successes" and "unlucky failures" influenced subsequent expectations and behaviors, anticipating three possible responses.

The info is here.

Here are some thoughts:

In essence, this article provides psychologists with a clear, compelling, and generalizable model for understanding one of the most pervasive and problematic aspects of human cognition: our innate drive to impose order and causality on randomness. It explains why people believe in luck, superstitions, and false cause-and-effect relationships, and why data often fails to change minds. This understanding is foundational for developing better communication strategies, designing effective interventions against misinformation, improving decision-making in high-stakes fields, and ultimately, helping individuals make more rational choices in their personal and professional lives.