Beth Israel Deaconess Medical Center
The purpose of the TEACH-AI study is to assess whether a brief, structured workshop on artificial intelligence can improve the performance of medicine doctors in training (i.e. residents) in their diagnostic and management reasoning. In this multi-site randomized controlled trial, internal medicine and family medicine residents are assigned either to receive an in-person workshop on safe, effective LLM use before a standardized AI-assisted assessment, or to complete the same assessment before receiving the workshop. Residents will review clinical cases that are fully synthetic, no protected health information is used, using a password protected LLM interface.
Large language models (LLMs) have rapidly entered routine use in medical education and clinical practice. Prior randomized trials have shown that, although LLMs can outperform individual clinicians on some reasoning benchmarks, providing physicians with access to an LLM does not necessarily improve their diagnostic or management performance, and LLMs alone may perform better than physician-plus-LLM teams. These findings suggest that human-AI collaboration is fraught, non-trivial and may require explicit training. TEACH-AI is a pragmatic, multi-site, two-arm randomized controlled trial embedde…
Inclusion Criteria: * Participants must be licensed physicians and have started at least post-graduate year 1 (PGY1) of medical training. * Training in Internal medicine or family medicine or emergency medicine. * Able to provide informed consent and complete assessments in English. Exclusion Criteria: * Not currently practicing clinically. * Resident directly involved in the design of the study, development of the workshop, or creation/piloting of the assessment vignettes. * Resident who declines or withdraws consent.
Participants will attend a workshop during their didactic time that will review various aspects of large language models including fundamentals and best practices of interacting and interpreting their outputs.