Patients aren’t being told about the AI systems advising their care

Rebecca Robbins and Erin Brodwin
statnews.com
Originally posted 15 July 20

Here is an excerpt:

The decision not to mention these systems to patients is the product of an emerging consensus among doctors, hospital executives, developers, and system architects, who see little value — but plenty of downside — in raising the subject.

They worry that bringing up AI will derail clinicians’ conversations with patients, diverting time and attention away from actionable steps that patients can take to improve their health and quality of life. Doctors also emphasize that they, not the AI, make the decisions about care. An AI system’s recommendation, after all, is just one of many factors that clinicians take into account before making a decision about a patient’s care, and it would be absurd to detail every single guideline, protocol, and data source that gets considered, they say.

Internist Karyn Baum, who’s leading M Health Fairview’s rollout of the tool, said she doesn’t bring up the AI to her patients “in the same way that I wouldn’t say that the X-ray has decided that you’re ready to go home.” She said she would never tell a fellow clinician not to mention the model to a patient, but in practice, her colleagues generally don’t bring it up either.

Four of the health system’s 13 hospitals have now rolled out the hospital discharge planning tool, which was developed by the Silicon Valley AI company Qventus. The model is designed to identify hospitalized patients who are likely to be clinically ready to go home soon and flag steps that might be needed to make that happen, such as scheduling a necessary physical therapy appointment.

Clinicians consult the tool during their daily morning huddle, gathering around a computer to peer at a dashboard of hospitalized patients, estimated discharge dates, and barriers that could prevent that from occurring on schedule.

The info is here.