P3391 - Automated Documentation of Colonoscopy Quality Measures With Machine Learning Is More Accurate Than Manual Human Input

Amelie Tiritilli, MD¹, Dmitriy V. Korchev, PhD², James Requa, BS³, William Karnes, FACG, MD^1
1UC Irvine, Orange, CA; ²DocBot, Irvine, CA; ³Docbot, Irvine, CA

Introduction: Machine learning (ML) is poised to positively impact patient care and clinical workflows. Beyond assisting polyp detection during colonoscopy, ML has the potential to automatically document procedure reports and record key quality measures, including tool detection, cecal intubation rate, and withdrawal time. We previously developed and reported algorithms to detect times of insertion, cecum, out, and tools used. Here we report video validation of these algorithms.

Methods: We utilized videos of colonoscopies recorded as part of the multi-center randomized trial of CADe for polyp detection (AIDA). Ground truth (GT) was established by consensus video review by two blinded gastroenterologists marking the times of insertion (gtIN), first clear view of cecum (gtCEC), exit from the colon (gtOUT), and tools and tool type relative to the first video frame (time 0). GT insertion time, withdrawal time, and total time were calculated from gtIN, gtCEC, and gtOUT and compared to insertion, withdrawal, and total durations recorded in real-time by study coordinators in the AIDA trial (manIN, manWT, and manOUT, respectively). ML algorithms for IN, CECUM, OUT, tools, and tool types were subsequently run on all videos to record times of mlIN, mlCEC, mlOUT, and tool predictions, respectively. Relative to GT values, mean absolute errors (MAEs) in seconds were determined for real-time human and ML-derived values.


Results: A total of 48 videos of complete colonoscopies with adequate prep were selected for analysis. The MAEs for mlIN, mlCEC, and mlOUT were 1.49 (range 0.65, 2.32), 48.68 (31.58, 65.78), and 5.11 (3.3, 2.32) seconds, respectively. The MAEs for calculated durations of insertion, withdrawal, and total time were significantly shorter for ML compared to human real-time-derived values (Table 1). Figure 1 illustrates these findings. Compared to GT, ML detected 80 of 80 tool usages and correctly identified 54 out of 56 snares and 22 out of 24 forceps.

Discussion: This video validation study demonstrates the feasibility of automated and accurate recording of cecal intubation time, tool usage, and withdrawal duration using ML. Its accuracy exceeds live manual recording during colonoscopy. ML has the potential to serve as a powerful and transformative documentation tool for video-guided procedures in the future.



Disclosures:


Amelie Tiritilli, MD¹, Dmitriy V. Korchev, PhD², James Requa, BS³, William Karnes, FACG, MD¹. P3391 - Automated Documentation of Colonoscopy Quality Measures With Machine Learning Is More Accurate Than Manual Human Input, ACG 2023 Annual Scientific Meeting Abstracts. Vancouver, BC, Canada: American College of Gastroenterology.