Prospective Evaluation of a Point-of-Care Artificial Intelligence Model in Critical Care Outcomes
MetroWest Artificial Intelligence Research Workgroup
Summary
This is a prospective, unmasked, randomized, multicenter clinical trial evaluating the impact of point-of-care large language model (LLM)-based decision support on diagnostic accuracy and clinical outcomes in adult medical intensive care unit (MICU) patients. Consecutive adult ICU admissions at participating community hospitals (initially MetroWest Medical Center and St. Vincent Hospital) will be screened for eligibility. Eligible patients will be randomized 1:1 to standard care or an AI-assisted group. In both arms, initial evaluation and management will follow usual practice. For patients randomized to AI assistance, de-identified admission data (history and physical, labs, imaging reports, and other relevant documentation) will be formatted and submitted to a state-of-the-art LLM (ChatGPT-5) at the time of admission. The AI-generated differential diagnosis and therapeutic recommendations will be provided to the admitting team for consideration. For the standard care arm, LLM output will be generated but not shared with clinicians. After discharge, a masked chart review will determine the "ground truth" primary diagnosis and extract outcomes including: Primary Outcome - a composite of medical errors (from time of ICU admission through day 7 of ICU stay, or ICU discharge, whichever comes first); Secondary Outcomes - 90-day mortality, ICU and hospital length of stay, and ventilator-free days.
Description
The rapid development of large language models (LLMs) such as ChatGPT has created new opportunities and risks for their use in medicine. Although early studies suggest high diagnostic accuracy in complex clinical scenarios and ICU admissions, the impact of LLMs on real-world clinical outcomes and the optimal mode of physician-AI interaction remain uncertain. Published work from our group showed that ChatGPT-4 achieved diagnostic accuracy comparable to board-certified intensivists for ICU admissions in a retrospective study. However, prospective, randomized data on clinical outcomes are lacking…
Eligibility
- Age range
- 18+ years
- Sex
- All
- Healthy volunteers
- No
Inclusion Criteria: 1. Adult patients (≥ 18 years) admitted to the medical intensive care unit (MICU) at participating hospitals. 2. Direct admissions from the emergency department or transfers from medical wards to the MICU. 3. Critically ill patients meeting local ICU admission criteria. Exclusion Criteria: 1. Transfers to the MICU from outside hospitals, operating room, or post-anesthesia care unit. 2. Age \< 18 years. 3. Incomplete or missing essential clinical information at admission (e.g., key labs or documentation not yet available). 4. Primary surgical or cardiac (e.g., STEMI) pati…
Interventions
- OtherPoint-of-care large language model decision support (ChatGPT-5)
Use of a large language model (ChatGPT-5) to analyze de-identified ICU admission data (history, physical examination, laboratory results, imaging reports, and other documentation) at the time of admission. The model generates diagnostic and therapeutic recommendations that are shared with clinicians in the AI-assisted arm only.
Location
- Framingham Union Hospital/MetroWest Medical CenterFramingham, Massachusetts