Artificial Intelligence-powered tiered early warning framework addressing high false alarm rates for in-hospital mortality prediction

Abstracts Alert fatigue remains a major barrier to the effective deployment of predictive models in emergency care, particularly in the context of rare but critical outcomes such as in-hospital mortality (IHM), which often occurs in less than 5.0% of patients admitted from the emergency department (ED). Severe class imbalance leads to low positive predictive value (PPV), undermining the clinical utility of even high-performance predictive models. To address this issue, we propose AI-TEW (Artificial Intelligence-powered Tiered Early Warning), a novel two-stage early warning framework designed to reduce false alarms and improve clinical interpretability. In Stage 1, a robust machine learning model was developed and validated using data from 174,292 ED visits across three hospitals in China and the United States. The model demonstrated strong discriminative ability for IHM prediction, achieving AUROCs ranging from 0.84 (95% CI, 0.81–0.86) to 0.91 (95% CI, 0.90–0.91) in internal and external validation cohorts. In Stage 2, AI-TEW implements a tiered risk stratification strategy by optimizing decision thresholds to prioritize high-risk patients, thereby increasing PPV from baseline levels of 9.8–18.8% to 32.5–40.5% across sites, while maintaining a high negative predictive value (NPV) of over 98% for low-risk individuals. To further refine alert precision, a knowledge-based filtering layer is introduced, leveraging large language models (LLM) to interpret patient-specific risk factors derived from SHAP (Shapley Additive exPlanations) method. Integrating explainable AI with clinical reasoning enhances contextual understanding and reduces spurious alerts, leading to an 11.53% increase in PPV in external validation (p = 0.0092 for MedGemma). By integrating improved predictive efficiency with interpretable, knowledge-informed filtering, AI-TEW reduces alert burden while supporting timely clinical intervention, demonstrating a promising approach to mitigating the impact of class imbalance in emergency risk prediction.