Harvard Medical School (HMS and HSDM)
This study will evaluate whether artificial intelligence (AI) can enhance clinicians' accuracy, efficiency, and confidence in distinguishing lung adenocarcinoma (LUAD) from lung squamous cell carcinoma (LUSC) and kidney renal papillary cell carcinoma (KIRP) from kidney renal clear cell carcinoma (KIRC) using digitized pathology slides. These subtype classifications are routinely performed by pathologists but can be challenging and time-consuming, particularly in difficult cases. During the study, participating clinicians will review lung and kidney pathology slides under three different conditions: * Unaided Review: Diagnosis without AI assistance. * AI as Double-Check: The clinician first makes an independent diagnosis, after which the AI-generated diagnosis (prediction only or prediction with explanation) is revealed for review. * AI as First-Look: The AI-generated diagnosis (prediction only or prediction with explanation) is presented before the clinician begins the review. Clinicians will be randomly assigned to different review sequences to minimize potential order effects. This study design will enable us to assess the impact of AI assistance on diagnostic accuracy, interpretation time, and clinician confidence.
This study aims to evaluate the effect of artificial intelligence (AI) assistance on clinicians' diagnostic performance in distinguishing lung adenocarcinoma (LUAD) from lung squamous cell carcinoma (LUSC) and kidney renal papillary cell carcinoma (KIRP) from kidney renal clear cell carcinoma (KIRC) using digitized hematoxylin and eosin (H\&E)-stained whole-slide images (WSIs). ENLIGHT (Explainable Neoplasm Learning In Grounded Histology Terms) will serve as the AI system under evaluation. This is a single-session, within-reader, between-case study in which each reader evaluates distinct sets…
Inclusion Criteria for Pathology Slides (i.e., Cases): * Hematoxylin and eosin (H\&E)-stained pathology slides * Final diagnosis confirmed through molecular testing in conjunction with expert pathology evaluation Exclusion Criteria for Pathology Slides (i.e., Cases): * Poor-quality or unreadable slides * Cases used in AI training Inclusion Criteria for Readers (i.e., Participants): * Board-certified or board-eligible pathologists * Willingness to complete both unaided and AI-assisted review sessions
Readers first complete Block X (Unaided) on their assigned subset SX. They then complete Block Y1 (AI as Double-Check) on two separate subsets: SY1a (AI prediction-only as Double-Check) and SY1b (AI prediction-with-explanation as Double-Check). Within Block Y1, the order of SY1a and SY1b is randomized. They then complete Block Y2 (AI as First-Look) on two separate subsets: SY2a (AI prediction-only as First-Look) and SY2b (AI prediction-with-explanation as First-Look). Within Block Y2, the order of SY2a and SY2b is randomized. For each reader, SX, SY1a, SY1b, SY2a, and SY2b are disjoint.
Readers first complete Block X (Unaided) on their assigned subset SX. They then complete Block Y2 (AI as First-Look) on two separate subsets: SY2a (AI prediction-only as First-Look) and SY2b (AI prediction-with-explanation as First-Look). Within Block Y2, the order of SY2a and SY2b is randomized. They then complete Block Y1 (AI as Double-Check) on two separate subsets: SY1a (AI prediction-only as Double-Check) and SY1b (AI prediction-with-explanation as Double-Check). Within Block Y1, the order of SY1a and SY1b is randomized. For each reader, SX, SY1a, SY1b, SY2a, and SY2b are disjoint.
Readers first complete Block Y1 (AI as Double-Check) on two separate subsets: SY1a (AI prediction-only as Double-Check) and SY1b (AI prediction-with-explanation as Double-Check). Within Block Y1, the order of SY1a and SY1b is randomized. They then complete Block Y2 (AI as First-Look) on two separate subsets: SY2a (AI prediction-only as First-Look) and SY2b (AI prediction-with-explanation as First-Look). Within Block Y2, the order of SY2a and SY2b is randomized. Then readers complete Block X (Unaided) on their assigned subset SX. For each reader, SX, SY1a, SY1b, SY2a, and SY2b are disjoint.
Readers first complete Block Y2 (AI as First-Look) on two separate subsets: SY2a (AI prediction-only as First-Look) and SY2b (AI prediction-with-explanation as First-Look). Within Block Y2, the order of SY2a and SY2b is randomized. They then complete Block Y1 (AI as Double-Check) on two separate subsets: SY1a (AI prediction-only as Double-Check) and SY1b (AI prediction-with-explanation as Double-Check). Within Block Y1, the order of SY1a and SY1b is randomized. Then readers complete Block X (Unaided) on their assigned subset SX. For each reader, SX, SY1a, SY1b, SY2a, and SY2b are disjoint.