How to Navigate the Pitfalls of AI Hype in Health Care

Suran M, Hswen Y.

JAMA.

Published online January 03, 2024.

doi:10.1001/jama.2023.23330

What is AI snake oil, and how might it hinder progress within the medical field? What are the inherent risks in AI-driven automation for patient care, and how can we ensure the protection of sensitive patient information while maximizing its benefits?

When it comes to using AI in medicine, progress is important—but so is proceeding with caution, says Arvind Narayanan, PhD, a professor of computer science at Princeton University, where he directs the Center for Information Technology Policy.

The interview and the video is here.

Here is my summary:

• AI has the potential to revolutionize healthcare, but it is important to be aware of the hype and potential pitfalls.
• One of the biggest concerns is bias. AI algorithms can be biased based on the data they are trained on, which can lead to unfair or inaccurate results. For example, an AI algorithm that is trained on data from mostly white patients may be less accurate at diagnosing diseases in black patients.
• Another concern is privacy. AI algorithms require large amounts of data to work, and this data can be very sensitive. It is important to make sure that patient data is protected and that patients have control over how their data is used.
• It is also important to remember that AI is not a magic bullet. AI can be a valuable tool, but it is not a replacement for human judgment. Doctors and other healthcare professionals need to be able to critically evaluate the output of AI algorithms and make sure that it is being used in a safe and ethical way.

Overall, the interview is a cautionary tale about the potential dangers of AI in healthcare. It is important to be aware of the risks and to take steps to mitigate them. But it is also important to remember that AI has the potential to do a lot of good in healthcare. If we develop and use AI responsibly, it can help us to improve the quality of care for everyone.

Here are some additional points that were made in the interview:

• AI can be used to help with a variety of tasks in healthcare, such as diagnosing diseases, developing new treatments, and managing chronic conditions.
• There are a number of different types of AI, each with its own strengths and weaknesses.
• It is important to choose the right type of AI for the task at hand.
• AI should always be used in conjunction with human judgment.

Cruel, Immoral Behavior Is Not Mental Illness

James L. Knoll & Ronald W. Pies
Psychiatric Times
Originally posted August 19, 2019

Here is an excerpt:

Another way of posing the question is to ask—Does immoral, callous, cruel, and supremely selfish behaviors constitute a mental illness? These socially deviant traits appear in those with and without mental illness, and are widespread in the general population. Are there some perpetrators suffering from a genuine psychotic disorder who remain mentally organized enough to carry out these attacks? Of course, but they are a minority. To further complicate matters, psychotic individuals can also commit violent acts that were motivated by base emotions (resentment, selfishness, etc.), while their psychotic symptoms may be peripheral or merely coincidental.

It bears repeating that reliable, clinically-based data or complete psychological autopsies on perpetrators of mass public shootings are very difficult to obtain. That said, some of the best available research on mass public shooters indicates that they often display “rigidness, hostility, or extreme self-centeredness.” A recent FBI study found that only 25% of mass shooters had ever had a mental illness diagnosis, and only 3 of these individuals had a diagnosis of a psychotic disorder. The FBI’s cautionary statement in this report is incisive: “. . . formally diagnosed mental illness is not a very specific predictor of violence of any type, let alone targeted violence…. declarations that all active shooters must simply be mentally ill are misleading and unhelpful."

Psychiatric and mental health treatment has its limits, and is not traditionally designed to detect and uncover budding violent extremists. It is designed to work together with individuals who are invested in their own mental health and seek to increase their own degrees of freedom in life in a pro-social manner. This is why calls for more mental health laws or alterations in civil commitment laws are likely to be low-yield at best, with respect to preventing mass killing—and stagnating to mental health progress at worst.

The info is here.

The 3 faces of clinical reasoning: Epistemological explorations of disparate error reduction strategies.

Sandra Monteiro, Geoff Norman, & Jonathan Sherbino
J Eval Clin Pract. 2018 Jun;24(3):666-673.

Abstract

There is general consensus that clinical reasoning involves 2 stages: a rapid stage where 1 or more diagnostic hypotheses are advanced and a slower stage where these hypotheses are tested or confirmed. The rapid hypothesis generation stage is considered inaccessible for analysis or observation. Consequently, recent research on clinical reasoning has focused specifically on improving the accuracy of the slower, hypothesis confirmation stage. Three perspectives have developed in this line of research, and each proposes different error reduction strategies for clinical reasoning. This paper considers these 3 perspectives and examines the underlying assumptions. Additionally, this paper reviews the evidence, or lack of, behind each class of error reduction strategies. The first perspective takes an epidemiological stance, appealing to the benefits of incorporating population data and evidence-based medicine in every day clinical reasoning. The second builds on the heuristic and bias research programme, appealing to a special class of dual process reasoning models that theorizes a rapid error prone cognitive process for problem solving with a slower more logical cognitive process capable of correcting those errors. Finally, the third perspective borrows from an exemplar model of categorization that explicitly relates clinical knowledge and experience to diagnostic accuracy.

A pdf can be downloaded here.

Complexities for Psychiatry's Identity As a Medical Specialty

Mohammed Abouelleil Rashed
Kan Zaman Blog
Originally posted November 23, 2018

Here is an excerpt:

Doctors, researchers, governments, pharmaceutical companies, and patient groups each have their own interests and varying abilities to influence the construction of disease categories. This creates the possibility for disagreement over the legitimacy of certain conditions, something we can see playing out in the ongoing debates surrounding Chronic Fatigue Syndrome, a condition that “receives much more attention from its sufferers and their supporters than from the medical community” (Simon 2011: 91). And, in psychiatry, it has long been noted that some major pharmaceutical companies influence the construction of disorder in order to create a market for the psychotropic drugs they manufacture. From the perspective of medical anti-realism (in the constructivist form presented here), these influences are no longer seen as a hindrance to the supposedly objective, ‘natural kind’ status of disease categories, but as key factors involved in their construction. Thus, the lobbying power of the American Psychiatric Association, the vested interests of pharmaceutical companies, and the desire of psychiatrists as a group to maintain their prestige do not undermine the identity of psychiatry as a medical specialty; what they do is highlight the importance of emphasizing the interests of patient groups as well as utilitarian and economic criteria to counteract and respond to the other interests. Medical constructivism is not a uniquely psychiatric ontology, it is a medicine-wide ontology; it applies to schizophrenia as it does to hypertension, appendicitis, and heart disease. Owing to the normative complexity of psychiatry (outlined earlier) and to the fact that loss of freedom is often involved in psychiatric practice, the vested interests involved in psychiatry are more complex and harder to resolve than in many other medical specialties. But that in itself is not a hindrance to psychiatry’s identity as a medical speciality.

The info is here.

This AI Just Beat Human Doctors On A Clinical Exam

Parmy Olson
Forbes.com
Originally posted June 28, 2018

Here is an excerpt:

Now Parsa is bringing his software service and virtual doctor network to insurers in the U.S. His pitch is that the smarter and more “reassuring” his AI-powered chatbot gets, the more likely patients across the Atlantic are to resolve their issues with software alone.

It’s a model that could save providers millions, potentially, but Parsa has yet to secure a big-name American customer.

“The American market is much more tuned to the economics of healthcare,” he said from his office. “We’re talking to everyone: insurers, employers, health systems. They have massive gaps in delivery of the care.”

“We will set up physical and virtual clinics, and AI services in the United States,” he said, adding that Babylon would be operational with U.S. clinics in 2019, starting state by state. “For a fixed fee, we take total responsibility for the cost of primary care.”

Parsa isn’t shy about his transatlantic ambitions: “I think the U.S. will be our biggest market shortly,” he adds.

The info is here.

Goldwater Rule: Red Line or Guideline?

Scott O. Lilienfeld, , Joshua D. Miller, Donald R. Lynam
Perspectives on Psychological Science 
Vol 13, Issue 1, pp. 33 - 35
First Published October 13, 2017

The decades following Miller’s (1969) call for psychological scientists to “give psychology away” have witnessed a growing recognition that we need to do more to communicate our knowledge to the general public (Kaslow, 2015; Lilienfeld, 2012). But should there be limits on the nature of this communication? The Goldwater Rule, which expressly forbids psychiatrists from commenting on the mental health of public figures whom they have not directly examined, answers this query in the affirmative; as we observed in our article (Lilienfeld, Miller, & Lynam, 2017), this rule has been de facto adopted by psychology.

We appreciate the opportunity to respond to two commentators who raise thoughtful qualifications and objections to our thesis, which holds that the Goldwater Rule is antiquated and premised on dubious scientific assumptions.  We are pleased that both scholars concur with us that the direct interview assumption—the principal empirical linchpin of the Goldwater Rule—is contradicted by large bodies of psychological research.

(cut to the conclusion)

Psychologists should typically refrain from proffering diagnostic judgments regarding public figures. Such judgments boost the risk of inaccurate ‘arm chair' diagnoses and of damaging the reputation of public figures and the profession at large.  At the same time, there is scant justification for a categorical ban on this practice, especially because psychologists can at times offer diagnostic information that bears to some degree on the question of individuals’ suitability for high public office.  We therefore recommend reformulating the 'Goldwater Rule” as the 'Goldwater Guideline.’  Such a change would underscore the wisdom of discretion with respect to statements concerning the diagnostic status of public figures but remind psychologists that such statements can be useful and even advisable within limits.

The article is here.

President Trump’s Mental Health — Is It Morally Permissible for Psychiatrists to Comment?

Claire Pouncey
The New England Journal of Medicine
December 27, 2107

Ralph Northam, a pediatric neurologist who was recently elected governor of Virginia, distinguished himself during the gubernatorial race by calling President Donald Trump a “narcissistic maniac.” Northam drew criticism for using medical diagnostic terminology to denounce a political figure, though he defended the terminology as “medically correct.” The term isn’t medically correct — “maniac” has not been a medical term for well over a century — but Northam’s use of it in either medical or political contexts would not be considered unethical by his professional peers.

For psychiatrists, however, the situation is different, which is why many psychiatrists and other mental health professionals have refrained from speculating about Trump’s mental health. But in October, psychiatrist Bandy Lee published a collection of essays written largely by mental health professionals who believe that their training and expertise compel them to warn the public of the dangers they see in Trump’s psychology. The Dangerous Case of Donald Trump: 27 Psychiatrists and Mental Health Experts Assess a President rejects the position of the American Psychiatric Association (APA) that psychiatrists should never offer diagnostic opinions about persons they have not personally examined. Past APA president Jeffrey Lieberman has written in Psychiatric News that the book is “not a serious, scholarly, civic-minded work, but simply tawdry, indulgent, fatuous tabloid psychiatry.” I believe it shouldn’t be dismissed so quickly.

The article is here.

Facial recognition may reveal things we’d rather not tell the world. Are we ready?

Amitha Kalaichandran
The Boston Globe
Originally published October 27, 2017

Here is an excerpt:

Could someone use a smartphone snapshot, for example, to diagnose another person’s child at the playground? The Face2Gene app is currently limited to clinicians; while anyone can download it from the App Store on an iPhone, it can only be used after the user’s healthcare credentials are verified. “If the technology is widespread,” says Lin, “do I see people taking photos of others for diagnosis? That would be unusual, but people take photos of others all the time, so maybe it’s possible. I would obviously worry about the invasion of privacy and misuse if that happened.”

Humans are pre-wired to discriminate against others based on physical characteristics, and programmers could easily manipulate AI programming to mimic human bias. That’s what concerns Anjan Chatterjee, a neuroscientist who specializes in neuroesthetics, the study of what our brains find pleasing. He has found that, relying on baked-in prejudices, we often quickly infer character just from seeing a person’s face. In a paper slated for publication in Psychology of Aesthetics, Creativity, and the Arts, Chatterjee reports that a person’s appearance — and our interpretation of that appearance — can have broad ramifications in professional and personal settings. This conclusion has serious implications for artificial intelligence.

“We need to distinguish between classification and evaluation,” he says. “Classification would be, for instance, using it for identification purposes like fingerprint recognition. . . which was once a privacy concern but seems to have largely faded away. Using the technology for evaluation would include discerning someone’s sexual orientation or for medical diagnostics.” The latter raises serious ethical questions, he says. One day, for example, health insurance companies could use this information to adjust premiums based on a predisposition to a condition.

The article is here.

Cultivating Humility and Diagnostic Openness in Clinical Judgment

John R. Stone
AMA Journal of Ethics. October 2017, Volume 19, Number 10: 970-977.

Abstract
In this case, a physician rejects a patient’s concerns that tainted water is harming the patient and her community. Stereotypes and biases regarding socioeconomic class and race/ethnicity, constraining diagnostic frameworks, and fixed first impressions could skew the physician’s judgment. This paper narratively illustrates how cultivating humility could help the physician truly hear the patient’s suggestions. The discussion builds on the multifaceted concept of cultural humility as a lifelong journey that addresses not only stereotypes and biases but also power inequalities and community inequities. Insurgent multiculturalism is a complementary concept. Through epistemic humility—which includes both intellectual and emotional components—and admitting uncertainty, physicians can enhance patients’ and families’ epistemic authority and health agency.

The article is here.

Will Trump Be the Death of the Goldwater Rule?

Jeannie Suk Gersen
The New Yorker
Originally posted August 23, 2017

Here is an excerpt:

The class of professionals best equipped to answer these questions has largely abstained from speaking publicly about the President’s mental health. The principle known as the “Goldwater rule” prohibits psychiatrists from giving professional opinions about public figures without personally conducting an examination, as Jane Mayer wrote in this magazine in May. After losing the 1964 Presidential election, Senator Barry Goldwater successfully sued Fact magazine for defamation after it published a special issue in which psychiatrists declared him “severely paranoid” and “unfit” for the Presidency. For a public figure to prevail in a defamation suit, he must demonstrate that the defendant acted with “actual malice”; a key piece of evidence in the Goldwater case was Fact’s disregard of a letter from the American Psychiatric Association warning that any survey of psychiatrists who hadn’t clinically examined Goldwater was invalid.

The Supreme Court denied Fact’s cert petition, which hoped to vindicate First Amendment rights to free speech and a free press. But Justice Hugo Black, joined by William O. Douglas, dissented, writing, “The public has an unqualified right to have the character and fitness of anyone who aspires to the Presidency held up for the closest scrutiny. Extravagant, reckless statements and even claims which may not be true seem to me an inevitable and perhaps essential part of the process by which the voting public informs itself of the qualities of a man who would be President.”

These statements, of course, resonate today. President Trump has unsuccessfully pursued many defamation lawsuits over the years, leading him to vow during the 2016 campaign to “open up our libel laws so when they write purposely negative and horrible and false articles, we can sue them and win lots of money.” (One of his most recent suits, dismissed in 2016, concerned a Univision executive’s social-media posting of side-by-side photos of Trump and Dylann Roof, the white supremacist who murdered nine black churchgoers in Charleston, South Carolina, in 2015; Trump alleged that the posting falsely accused him of inciting similar acts.)

The article is here.

Psychiatry Group Tells Members They Can Ignore ‘Goldwater Rule’ and Comment on Trump’s Mental Health

Sharon Begley
Global Research
Originally published July 25, 2017

A leading psychiatry group has told its members they should not feel bound by a longstanding rule against commenting publicly on the mental state of public figures — even the president.

The statement, an email this month from the executive committee of the American Psychoanalytic Association to its 3,500 members, represents the first significant crack in the profession’s decades-old united front aimed at preventing experts from discussing the psychiatric aspects of politicians’ behavior. It will likely make many of its members feel more comfortable speaking openly about President Trump’s mental health.

The impetus for the email was “belief in the value of psychoanalytic knowledge in explaining human behavior,” said psychoanalytic association past president Dr. Prudence Gourguechon, a psychiatrist in Chicago.

“We don’t want to prohibit our members from using their knowledge responsibly.”

That responsibility is especially great today, she told STAT, “since Trump’s behavior is so different from anything we’ve seen before” in a commander in chief.

An increasing number of psychologists and psychiatrists have denounced the restriction as a “gag rule” and flouted it, with some arguing they have a “duty to warn” the public about what they see as Trump’s narcissism, impulsivity, poor attention span, paranoia, and other traits that, they believe, impair his ability to lead.

The article is here.

Psychiatry’s “Goldwater Rule” has never met a test like Donald Trump

Brian Resnick
Vox.com
Originally published May 25, 2017

Here is an excerpt:

Some psychiatrists are saying it’s time to rethink this core ethical guideline. The rule, they say, is acting like a gag order, preventing qualified psychiatrists from giving the public important perspective on the mental health of a president whose behavior is out of step with any other president in history.

“The public has a right to medical and psychiatric knowledge about its leaders — at least in a democracy,” Nassir Ghaemi, a Tufts University psychiatrist, recently argued at an APA conference. “Why can’t we have a reasoned scientific discussion on this matter? Why do we just have complete censorship?”

The controversy is sure to rage on, as many psychiatrists stand by the professional precedent. The rule itself has even been expanded recently. But just the existence of the debate is an incredible moment not only in the field of psychiatry but in American politics. It’s not just armchair psychiatrists who are concerned about Trump’s mental health — some of the real ones are even willing to rethink their professional ethics because of it.

The article is here.

The power thinker

Colin Koopman

aeon.co

Originally posted March 15, 2017

Here is an excerpt:

Foucault’s work shows that disciplinary power was just one of many forms that power has come to take over the past few hundred years. Disciplinary anatomo-politics persists alongside sovereign power as well as the power of bio-politics. In his next book, The History of Sexuality, Foucault argued that bio-politics helps us to understand how garish sexual exuberance persists in a culture that regularly tells itself that its true sexuality is being repressed. Bio-power does not forbid sexuality, but rather regulates it in the maximal interests of very particular conceptions of reproduction, family and health. It was a bio-power wielded by psychiatrists and doctors that, in the 19th century, turned homosexuality into a ‘perversion’ because of its failure to focus sexual activity around the healthy reproductive family. It would have been unlikely, if not impossible, to achieve this by sovereign acts of direct physical coercion. Much more effective were the armies of medical men who helped to straighten out their patients for their own supposed self-interest.

Other forms of power also persist in our midst. Some regard the power of data – that is the info-power of social media, data analytics and ceaseless algorithmic assessment – as the most significant kind of power that has emerged since Foucault’s death in 1984.

The article is here.

Is It Time to Call Trump Mentally Ill?

Richard A. Friedman
The New York Times
Originally published February 17, 2017

Here are two excerpts:

A recent letter to the editor in this newspaper, signed by 35 psychiatrists, psychologists and social workers, put it this way: "We fear that too much is at stake to be silent." It continued, "We believe that the grave emotional instability indicated by Mr. Trump's speech and actions makes him incapable of serving safely as president."

But the attempt to diagnose a condition in President Trump and declare him mentally unfit to serve is misguided for several reasons.

First, all experts have political beliefs that probably distort their psychiatric judgment. Consider what my mostly liberal profession said of Senator Barry Goldwater, the Republican nominee for president in 1964, right before the election. Members of the American Psychiatric Association were surveyed about their assessment of Goldwater by the now-defunct Fact magazine. Many savaged him, calling him "paranoid," "grossly psychotic" and a "megalomaniac." Some provided diagnoses, like schizophrenia and narcissistic personality disorder.

They used their professional knowledge as a political weapon against a man they had never examined and who certainly would never have consented to their discussing his mental health in public.

Goldwater sued (successfully) and, as a result, in 1973 the A.P.A. developed the Goldwater Rule. It says that psychiatrists can discuss mental health issues with the news media, but that it is unethical for them to diagnose mental illnesses in people they have not examined and whose consent they have not received.

(cut)

There is one last reason we should avoid psychiatrically labeling our leaders: It lets them off the moral hook. Not all misbehavior reflects psychopathology; the fact is that ordinary human meanness and incompetence are far more common than mental illness. We should not be in the business of medicalizing bad actors.

The article is here.

Psychiatrist diagnosed local optician without meeting him

By Kelly Bennett
CBC News
Originally posted: Aug 25, 2016

A Burlington optician is outraged after discovering a psychiatrist he'd never met wrote a critical two-page psychiatric evaluation about him without ever seeing or talking to him.

The optician, Jay Hakim, filed a complaint with the provincial medical regulator, which concluded the psychiatrist's conduct was appropriate.

Hakim appealed the regulator's decision; that appeal was held in a hearing downtown Hamilton on Wednesday.

The case raises "some very serious consequences for society" if it's allowed to stand, Hakim argued.

It also parallels issues raised in the United States over whether psychiatrists can ethically provide opinions on the mental health of presidential candidates they've never met.

The article is here.

How Artificial Intelligence Could Help Diagnose Mental Disorders

Joseph Frankel
The Atlantic
Originally posted August 23, 2016

Here is an excerpt:

In addition to the schizophrenia screener, an idea that earned Schwoebel an award from the American Psychiatric Association, NeuroLex is hoping to develop a tool for psychiatric patients who are already being treated in hospitals. Rather than trying to help diagnose a mental disorder from a single sample, the AI would examine a patient’s speech over time to track their progress.

For Schwoebel, this work is personal: he thinks this approach may help solve problems his older brother faced in seeking treatment for schizophrenia. Before his first psychotic break, Schwoebel’s brother would send short, one-word responses, or make cryptic to references to going “there” or “here”—worrisome abnormalities that “all made sense” after his brother’s first psychotic episode, he said.

The article is here.

Should Therapists Analyze Presidential Candidates?

by Robert Klitzman
The New York Times
Originally published March 6, 2016

Many psychologists have been quick to offer diagnoses, calling him and other presidential candidates "narcissists," and even providing thoughts about possible treatments.

I wondered what, if anything, to say. I've watched Mr. Trump on TV like everyone else, but never met him. So, I hesitated -- for ethical reasons. The American Psychiatric Association (A.P.A.) prohibits its members from giving professional opinions about public figures we have not interviewed.

This ban stems from a bad incident in my field. In 1964, Fact magazine published an article, announced on its cover as "1,189 Psychiatrists say Goldwater is Psychologically Unfit to be President". The magazine surveyed these professionals, and 49 percent of respondents said Barry M. Goldwater was unfit for the job, describing him as "unbalanced," "immature," "paranoid," "psychotic" and "schizophrenic," and questioning his "manliness." Leading psychiatrists were among those quoted. A famous Johns Hopkins professor said Mr. Goldwater's utterances should "disqualify him from the presidency."

The article is here.

When Doctors Should Say 'I Don't Know'

By Julie Beck
The Atlantic
Originally published February 29, 2016

Here is an excerpt:

Doctors’ tools, knowledge, and treatments have improved since the bloodletting days, and we now have the ability to scan and analyze the body down to the cellular level. But “precision is not the same thing as certainty,” Hatch writes, and often, doctors are just making guesses based on the best evidence they have—a measuring of risks and benefits and probabilities that can be easily influenced by their preconceptions.

Medicine is a high-stakes game of uncertainty, complicated by the fact that people are naturally predisposed to seek certainty whenever possible. If you don’t know what something is, it could be a threat, out there on the ancient savannah of evolutionary psychology logic. That goes for patients and doctors alike, and if both parties are in agreement that certainty is best, it’s possible that they’ll just blow past the risks of a treatment, or the dubiousness of a diagnosis, for the sake of having an answer.

The article is here.

New Jersey Psychology Practice Revealed Patients’ Mental Disorders in Debt Lawsuits

By Charles Ornstein
ProPublica, Dec. 23, 2015

When a New Jersey lawyer named Philip received legal papers last year informing him that his former psychologist’s practice was suing him over an unpaid bill, he was initially upset they could not work out a payment arrangement outside of court.

It was only later, Philip said in an interview, that he scanned the papers again and realized something else: The psychology group to which he’d confided his innermost feelings had included his mental health diagnosis and treatments he received in publicly filed court documents.

The greatest fear of many patients receiving therapy services is that somehow the details of their private struggles will be revealed publicly. Philip, who requested his last name not be used to protect his privacy, said he felt “betrayed” by his psychologist. He worried that his legal adversaries would find the information and try to use it against him in court.

“It turned my life upside down,” he said.

The article is here.

The Autism Paradox

Kathleen K. Miller
AMA Journal of Ethics. April 2015, Volume 17, Number 4: 297-298

Through the process of researching this issue, I became fascinated with recent research on neurodiversity as it applies to autism. To oversimplify vastly, neurodiversity is the idea that neurological differences (including autism) are the result of normal variation. In other words, we may be creating pathology where there is none. Should we consider autism a disease? Or is it a variation of normal? This question has been the source of heated debate, with many autism advocates arguing that autism should not be considered a disease or disorder.

As a pediatrician, this is a question I struggle with. I support autism advocacy and the rights of people with autism. I believe we need to adjust our mental framework and see people with autism as more than their diagnoses.

The entire article is here.