P0712 - The Utility of ChatGPT in the Reporting of Idiopathic Inflammatory Bowel Disease

Matt Andrew Paz, DO, Kevin Turner, DO
University of Minnesota, Minneapolis, MN

Introduction: Inflammatory Bowel Disease (IBD) is a group of chronic inflammatory diseases of the bowel associated with a dysregulated response to normal gut microflora. The two main types of IBD, ulcerative colitis (UC) and Crohn’s disease (CD) are characterized by a combination of clinical, pathologic, and radiologic features and that may present similarly in up to 10% of patients, making a definitive diagnosis difficult. The aim of this study was to evaluate the capabilities of ChatGPT, an artificial intelligence chatbot by OpenAI, in characterizing the two entities given a patient’s history and diagnosis.

Methods: The free research version of ChatGPT was used to generate a set of pathology reports for fictitious patients exhibiting typical and atypical presentations of IBD. Each prompt consisted of patients with different demographics, clinical presentations, endoscopic findings, and histologic findings. The first output for each prompt was retained, and no response was regenerated. Each response was analyzed for content and accuracy.

Results: For each prompt, ChatGPT generated a multi-sentence response incorporating the supplied patient information. ChatGPT generated generally reasonable comments when typical presentations were entered. ChatGPT performed less well when provided atypical presentations of UC or CD. Instead of issuing an indeterminate diagnosis for example, ChatGPT formulated erroneous conclusions. Finally, ChatGPT outputs were seemingly heavily influenced by “buzz words” within the prompts resulting in definitive results that should have been indefinite.

Discussion: Differentiating the presentation of UC and CD can be difficult in certain patients as the two diseases can present with similar clinical symptoms, similar endoscopic findings, and inconclusive microscopic findings. In this study, we used ChatGPT to review a patient’s clinical history, combined with endoscopic and microscopic findings, and write a pathology report with a final diagnosis. Although there were errors, ChatGPT performed relatively well in classifying patients with a typical presentation as having UC or CD. Our findings are supportive of the notion that as these technologies mature they will be useful in the generation of pathology reports.


Disclosures:


Matt Andrew Paz, DO, Kevin Turner, DO. P0712 - The Utility of ChatGPT in the Reporting of Idiopathic Inflammatory Bowel Disease, ACG 2023 Annual Scientific Meeting Abstracts. Vancouver, BC, Canada: American College of Gastroenterology.