Rajmohan Rammohan, MD, Melvin Joy, MD, Sai Greeshma Magam, MD, Dilman Natt, MD, Achal Patel, MD, Abhishek Tadikonda, MD, Rucha Jiyani, MD, Tulika Saggar, MD, Jiten Desai, MD, Sandra Gomez, MD, Saher Sheikh, MD, Paul Mustacchia, MD, MBA Nassau University Medical Center, East Meadow, NY
Introduction: Artificial intelligence plays a significant role in the medical field by delivering healthcare information. AI systems can understand and respond to human speech, enabling them to perform various tasks upon command. In this study, four popular voice assistants—Apple Siri, Amazon Alexa, Google Assistant, and Microsoft Cortana—were evaluated. Voice recordings were created for five frequently asked questions about Ulcerative colitis and Crohn's disease, in four languages: English, Spanish, Mandarin, and French. The accuracy of the responses given via audio output and the reliability of the sources supporting each answer was assessed. The researchers determined whether the four voice assistants offered clinically sound advice.
Methods: This study assessed the reliability of medical information from four AI voice assistants—Apple Siri, Amazon Alexa, Google Assistant, and Microsoft Cortana—across English, Spanish, Mandarin, and French for Ulcerative colitis and Crohn's questions. Two blinded authors graded responses in a two-stage comparison process. The research evaluated response accuracy, supporting source quality, and determined clinically reliable advice while using native speakers and multiple logistic regression analysis.
Results: When comparing individual languages, Apple Siri delivered significantly more reliable information in English than Alexa, Google Assistant, and Cortana (98% vs. 88% vs. 45% vs. 13%, P=0.023, 95% CI). Both Alexa and Apple Siri provided dependable information in French compared to other AI systems (88% vs. 78% vs. 45% vs. 10%, P< 0.01, 95% CI). Google Assistant and Apple Siri proved to be more reliable than their counterparts in Spanish (81% vs. 84% vs. 33% vs. 5%, P=0.023, 95% CI) and in Mandarin (91% vs. 88% vs. 21% vs. 1%, P=0.023, 95% CI). AI systems in English yielded significantly better responses compared to those in other languages (88% vs. 57%, P< 0.01).
Discussion: This study assessed the reliability of four AI voice assistants—Apple Siri, Amazon Alexa, Google Assistant, and Microsoft Cortana—in providing healthcare information on Ulcerative colitis and Crohn's disease. Results indicated Siri outperformed others in English, while Siri and Alexa were reliable in French, and Google Assistant and Siri in Spanish and Mandarin. Further research is needed to enhance AI reliability across languages for diverse populations.
Figure: Artificial Intelligence Study Outcome
Disclosures:
Rajmohan Rammohan indicated no relevant financial relationships.
Melvin Joy indicated no relevant financial relationships.
Sai Greeshma Magam indicated no relevant financial relationships.
Dilman Natt indicated no relevant financial relationships.
Achal Patel indicated no relevant financial relationships.
Abhishek Tadikonda indicated no relevant financial relationships.
Rucha Jiyani indicated no relevant financial relationships.
Tulika Saggar indicated no relevant financial relationships.
Jiten Desai indicated no relevant financial relationships.
Sandra Gomez indicated no relevant financial relationships.
Saher Sheikh indicated no relevant financial relationships.
Paul Mustacchia indicated no relevant financial relationships.
Rajmohan Rammohan, MD, Melvin Joy, MD, Sai Greeshma Magam, MD, Dilman Natt, MD, Achal Patel, MD, Abhishek Tadikonda, MD, Rucha Jiyani, MD, Tulika Saggar, MD, Jiten Desai, MD, Sandra Gomez, MD, Saher Sheikh, MD, Paul Mustacchia, MD, MBA. P0774 - Revolutionizing Irritable Bowel Disease Management With the Help of Artificial Assistance Technology, ACG 2023 Annual Scientific Meeting Abstracts. Vancouver, BC, Canada: American College of Gastroenterology.
