P1787 - Do Voice Assistants and Chatbots Provide Equally Appropriate Advice Regarding Colorectal Cancer Screening?

Atul Sinha, MD¹, Tulika Saggar, MD¹, Rajmohan Rammohan, MD¹, James Pellegrini, MD¹, Jenoj Gnana, MD², Paul Mustacchia, MD, MBA^1
1Nassau University Medical Center, East Meadow, NY; ²Nassau University Medical Center, Astoria, NY

Introduction: The prominence of artificial intelligence in our lives is growing everyday. Two forms of artificial intelligence that are available to the public are chatbots and voice assistants (VA). A chatbot allows users to have human-like conversations via manual entry whereas a voice assistant is a voice enabled artificial intelligence that allows users to communicate with a device in a conversation-like manner. One of the many uses of digital searches is to seek out medical information, begging the question, are these artificial intelligences providing medically accurate answers to such questions? Web searches for colorectal cancer screening have steadily increased over the past 10 years which means there is increasing interest in finding information about colorectal cancer by the public. We did this study to determine if chatbots and virtual assistants provide clinically appropriate advice regarding colorectal cancer screening.

Methods: Two chatbots: OpenAI ChatGPT and Google Bard were tested. Four voice assistants: Apple Siri, Amazon Alexa, Google Assistant, and Microsoft Cortana were tested. 5 commonly asked questions regarding colorectal cancer screening were used. Questions were manually typed for the chatbots and voice recordings were done for the virtual assistants. The authors decided if clinically appropriate advice was given.



Results: We found that clinically appropriate advice was provided 80% by OpenAI ChatGPT, 100% by Google Bard, 100% by Apple Siri, 60% by Amazon Alexa, 100% by Google Assistant, and 40% by Microsoft Cortana (Table 1).

Discussion: Apple Siri, Google Assistant, and Google Bard provided clinically appropriate advice 100% of the time, however, the top results for Google Assistant were advertisements for the services and products of private companies. We understand that the VAs generate income from advertising, however, we believe it is amoral to show these advertisements prior to providing meaningful results in this situation. All of the VAs and one chatbot provided accurate advice about when to start colorectal cancer screening, even including the May 2021 USPSTF recommendation to start screening at age 45. Both of the chatbots, but none of the VAs, instructed users to speak to a healthcare provider, which we believe is a vital aspect to any medically related search result. Most of the artificial intelligences performed well in our study but we believe there is a need for improvement, especially with how technology is becoming more ingrained in our everyday lives.


Disclosures:


Atul Sinha, MD¹, Tulika Saggar, MD¹, Rajmohan Rammohan, MD¹, James Pellegrini, MD¹, Jenoj Gnana, MD², Paul Mustacchia, MD, MBA¹. P1787 - Do Voice Assistants and Chatbots Provide Equally Appropriate Advice Regarding Colorectal Cancer Screening?, ACG 2023 Annual Scientific Meeting Abstracts. Vancouver, BC, Canada: American College of Gastroenterology.