AI Tool May Reduce Risk for Unexpected Hospital Deaths

A tool called CHARTwatch may reduce the risk for unexpected death among hospitalized patients by serving as an early warning system for rapidly deteriorating health, according to a new study.

The artificial intelligence (AI)–based system watches real-time data from patients’ electronic medical records to identify those who may have unplanned admission to the intensive care unit (ICU) and creates a clinical pathway for high-risk patients. The tool helped reduce deaths by 26% in a general internal medicine unit by sending real-time alerts to doctors, twice-daily emails to nursing teams, and daily emails to the palliative care team.

“AI tools hold great promise for helping us improve the quality of healthcare by improving the accuracy and efficiency of diagnosis, assisting with the personalization of treatment decisions for individual patients, enhancing our ability to predict and prevent future health events, and improving the efficiency of healthcare operations,” lead author Amol Verma, MD, a clinician-scientist at St. Michael’s Hospital and Temerty professor of AI research and education in medicine at the University of Toronto, Toronto, Ontario, Canada, told Medscape Medical News.

“It’s essential that these tools be researched and implemented carefully,” he said. “Like other healthcare interventions, AI tools may have benefits or unintended consequences, and we need to ensure that they are safe and effective.”

The study was published online on September 16 in CMAJ.

Watching CHARTwatch

During a 3-year period, Verma and colleagues developed and tested CHARTwatch, which includes a machine learning prediction model, a series of tools to alert doctors and other relevant medical professionals when a patient faces a high risk for deteriorating, and a suggested care pathway for high-risk patients. They tested the tool at St. Michael’s Hospital, which is an inner-city academic health center in Toronto, and focused on the general internal medicine unit, which includes 70 beds.

The tool was implemented between August and October 2020. Before then, the critical care response team would respond to patients based on a doctor’s or nurse’s judgment, but a formal deterioration detection score system had not been established. The primary goal of implementing CHARTwatch was to reduce nonpalliative hospital deaths by starting interventions sooner and consulting palliative care specialists when appropriate.

In this study, the research team conducted a nonrandomized but controlled evaluation of the association between the implementation of CHARTwatch as an early warning system and clinical outcomes. They looked at deaths among 9626 patients in the general internal medicine unit before the tool was used (November 2016 to June 2020) and among 4023 patients in the postimplementation phase (November 2020 to June 2022). They also compared the combined 13,649 patients in the general internal medicine unit with 8470 subspecialty patients in cardiology, respirology, and nephrology, where the tool wasn’t used.

In general, the rate of nonpalliative deaths was significantly lower among general internal medicine patients when the tool was used in 2020-2022 (1.6%) than the period before it was used (2.1%). The adjusted relative risk (RR) for death was 0.74. In in the subspecialty cohorts, the rate of nonpalliative deaths wasn’t significantly different during the intervention period (1.9% vs 2.1%; adjusted RR, 0.89).

Among high-risk general internal medicine patients with at least one tool-based alert, the proportion of nonpalliative deaths was 7.1% in 2020-2022 vs 10.3% in 2016-2020, yielding an adjusted RR of 0.69. Among subspecialty cohorts, nonpalliative deaths didn’t differ significantly during those time periods (10.4% vs 10.6%; adjusted RR, 0.98).

Across analyses and subgroups, there also weren’t significant differences in overall deaths, palliative deaths or transfers, ICU transfers, or length of hospital stay during those periods.

After a high-risk alert, patients in the intervention group were more likely to have vital signs measured more frequently and receive antibiotics and systemic glucocorticoids, the research team found. However, there didn’t appear to be any changes in imaging use, code status orders, or intravenous fluid orders.

Implementing Across Hospitals

For the tool to be used more widely, Verma said, researchers need large-scale health datasets for AI tools to accurately make predictions across hospital settings, particularly among patients from diverse backgrounds.

“It is important for tools that appear to be effective in a single context to be tested in a wider range of settings,” said Verma. “This is particularly true for AI tools, which perform best in the patient populations that were used for their development.”

The investigators are involved with a 35-hospital data sharing network, called GEMINI, which is now building a large, inclusive data resource to allow for this type of broad AI-based development and testing.

“In healthcare, it is crucial to continuously focus on system improvements to elevate the quality of care, patient safety, and efficiency. As demands grow and the complexity of care increases, researching and adopting innovative strategies is essential to meet these challenges and improve care delivery,” said Rabia Shahid, MD, associate professor of medicine at the University of Saskatchewan in Saskatoon, Saskatchewan, Canada.

Shahid, who wasn’t involved with this study, has researched early warning systems for patient deterioration in hospitals. She and colleagues found that tools can improve patient outcomes and communication among medical providers — but need to be refined and have better stakeholder buy-in to work effectively.

“These tools, particularly those driven by technology such as AI and machine learning, are vital to research and develop because they significantly enhance the quality of care across critical domains, including safety, effectiveness, timeliness, patient-centeredness, and efficiency,” she said. “By enabling early detection of patient deterioration and supporting clinical decision-making, these tools help ensure that care remains consistent and of high quality, even in the most demanding and high-pressure hospital settings.”

The study was funded in part by the Vector Institute Pathfinder Project and the AMS Healthcare Compassion and AI Fellowship. Verma is supported by the Temerty Professorship of AI Research and Education in Medicine, received travel support from the Alberta Machine Intelligence Institute, and is a part-time employee of Ontario Health outside of this study. Verma and several authors co-invented CHARTwatch, which was acquired by a startup, and may be able to acquire minority interest in the company in the future. Shahid reported no relevant financial relationships.

Carolyn Crist is a health and medical journalist who reports on the latest studies for Medscape Medical News, MDedge, and WebMD.