Generative AI enhances emergency department decision-making

Generative artificial intelligence (AI), exemplified by models like GPT-4, shows promise in predicting emergency department (ED) patient admissions with minimal training, according to a study by the Icahn School of Medicine at Mount Sinai. Published in the online issue of the Journal of the American Medical Informatics Association (JAMIA), the study highlights the potential of AI to transform clinical decision-making in high-pressure medical settings.

In a retrospective analysis involving seven hospitals within the Mount Sinai Health System, researchers utilized both structured data, such as vital signs, and unstructured data, like nurse triage notes, from over 864,000 ED visits. The data, stripped of identifiable patient information, revealed that 159,857 visits (18.5%) resulted in hospital admissions.

The study aimed to evaluate GPT-4's effectiveness against traditional machine-learning models, specifically Bio-Clinical-BERT for text analysis and XGBoost for structured data. The generative AI model was tested both independently and in combination with these conventional methods to predict hospital admissions.

"We were motivated by the need to test whether generative AI, specifically large language models (LLMs) like GPT-4, could improve our ability to predict admissions in high-volume settings such as the emergency department," said Eyal Klang, MD, co-senior author and director of the Generative AI Research Program in the Division of Data-Driven and Digital Medicine at Icahn Mount Sinai, in a statement. "Our goal is to enhance clinical decision-making through this technology. We were surprised by how well GPT-4 adapted to the ER setting and provided reasoning for its decisions. This capability of explaining its rationale sets it apart from traditional models and opens up new avenues for AI in medical decision-making."

While traditional machine-learning models require extensive data for training, LLMs can learn effectively from a limited number of examples. The study demonstrated that LLMs could incorporate predictions from traditional models, thereby enhancing performance.

Girish N. Nadkarni, MD, MPH, co-senior author and director of The Charles Bronfman Institute of Personalized Medicine, emphasized the supportive role of AI in a statement: "Our research suggests that AI could soon support doctors in emergency rooms by making quick, informed decisions about patient admissions. This work opens the door for further innovation in health care AI, encouraging the development of models that can reason and learn from limited data, like human experts do. However, while the results are encouraging, the technology is still in a supportive role, enhancing the decision-making process by providing additional insights, not taking over the human component of health care, which remains critical."

The research team is exploring further applications of LLMs in health care, aiming for seamless integration with traditional machine-learning methods to tackle complex clinical decision-making challenges in real-time.

"Our study informs how LLMs can be integrated into health care operations," said Brendan Carr, MD, study co-author, emergency room physician, and CEO of Mount Sinai Health System, in a statement. "The ability to rapidly train LLMs highlights their potential to provide valuable insights even in complex environments like health care. Our study sets the stage for further research on AI integration in health care across the many domains of diagnostic, treatment, operational, and administrative tasks that require continuous optimization."