P1223 - Understanding the Landscape: The Emergence of AI, ChatGPT, and Google BARD in Gastroenterology

Award: Outstanding Research Award in the Practice Management Category (Trainee)

Award: Presidential Poster Award

Rajmohan Rammohan, MD, Melvin Joy, MD, Dilman Natt, MD, Sai Greeshma Magam, MD, Achal Patel, MD, Tulika Saggar, MD, Abhishek Tadikonda, MD, Jiten Desai, MD, Ana Rivera, MD, Paul Mustacchia, MD, MBA
Nassau University Medical Center, East Meadow, NY

Introduction: Artificial Intelligence (AI) has significantly revolutionized healthcare, particularly in gastroenterology, enhancing diagnosis and treatment of digestive diseases. Key AI applications like ChatGPT and Google Bard interpret endoscopic images, analyze various samples, streamline administrative tasks, and aid in medical image evaluation and device automation. By personalizing treatments and predicting side effects, they have significantly improved digestive disorder management. The incorporation of these AI tools enables patients to make informed decisions by assessing tailored treatment options.

Methods: The research aimed to assess the accuracy of two prevalent Chat Bots - Chat GPT and Google BARD, - in answering medical management. Each bot was given a set of questions, with responses rated on a 1-10 Likert scale, where 1 represented high accuracy. Two impartial evaluators scrutinized each bot's responses to maintain objective judgment. This study intended to shed light on these Chat Bots' abilities through a systematic performance assessment in terms of accuracy and reliability. The use of two evaluators and the Likert scale method contributed to reducing potential bias, hence validating the results

Results: Our research contrasted the effectiveness of Chat GPT and Google BARD in the realm of medical management. Chat GPT excelled, scoring 61% in accuracy compared to Google BARD's 29% (p = 0.022) and 38% vs.19% in medical information reliability (p = 0.012). This study emphasizes the critical role of accuracy, and dependability, in developing gastrointestinal-oriented chatbots. While Chat GPT displayed impressive performance, indicating its potential as a reliable tool, it also underscored the necessity for continued research and development in this field.

Discussion: AI has notably transformed healthcare, especially in gastroenterology, facilitating diagnosis and disease management. ChatGPT and Google Bard are pivotal AI applications that interpret medical images, analyze samples, and streamline tasks. Our study assessed these chatbots' accuracy and reliability in medical management. ChatGPT outperformed Google Bard, achieving 61% accuracy and 38% information reliability. Despite ChatGPT's promising potential as a dependable AI tool in gastroenterology, the findings also emphasize the ongoing need for research and development in this field.




Disclosures:


Rajmohan Rammohan, MD, Melvin Joy, MD, Dilman Natt, MD, Sai Greeshma Magam, MD, Achal Patel, MD, Tulika Saggar, MD, Abhishek Tadikonda, MD, Jiten Desai, MD, Ana Rivera, MD, Paul Mustacchia, MD, MBA. P1223 - Understanding the Landscape: The Emergence of AI, ChatGPT, and Google BARD in Gastroenterology, ACG 2023 Annual Scientific Meeting Abstracts. Vancouver, BC, Canada: American College of Gastroenterology.