Skip to main content
ACG 2023 Annual Meeting Posters Main banner
Back
Rate ePoster

Monday Poster Session

Category: General Endoscopy

P1986 - Inter- and Intra-Observer Variability of Endoscopic Scoring Systems in Ulcerative Colitis and Crohn’s Disease: A Systematic Review and Meta-Analysis

Monday, October 23, 2023
10:30 AM – 4:15 PM PT
Location: Exhibit Hall
Has Audio
Jana G.. Hashash, MD, MSc, FACG1, Francis A. Farraye, MD, MSc, MACG1, Yeli Wang, PhD2, Daniel Colucci, BS2, Shrujal Baxi, MD2, Sadaf Muneer, MS2, Faye Yu Ci Ng, BS2, Pratik Shingru, MD2, Gil Melmed, MD, MS3
1Mayo Clinic, Jacksonville, FL; 2Iterative Health Inc., Cambridge, MA; 3Center for Inflammatory Bowel Diseases, Cedars-Sinai Medical Center, Los Angeles, CA

Introduction: Accurate and reproducible endoscopic assessment is critical to the diagnosis and monitoring of patients with inflammatory bowel disease (IBD). Furthermore, endoscopic scoring is used to determine IBD clinical trial eligibility and measurement of treatment effect. However, the agreement rates of standard endoscopic scoring indices commonly used in IBD are not presently clear. In this study, we aimed to assess inter- and intra-observer variability of endoscopic scoring systems for IBD in a systematic review and meta-analysis.

Methods: A search was conducted in PubMed/MEDLINE, EMBASE, and by screening of references of selected publications. We included observational studies that evaluated the inter- and intra-observer variability of either UC indices (endoscopic Mayo Score [eMS] and Ulcerative Colitis Endoscopic Index of Severity [UCEIS]), or CD indices (Crohn's Disease Endoscopic Index of Severity [CDEIS] and Simple Endoscopic Score for Crohn's Disease [SES-CD]), among adults (≥18 years) and published in English. Metrics used to assess agreement included Intra-class correlation (ICC), agreement rate, and kappa (k). The strength of agreement was categorized as fair (0.21-0.4), moderate (0.41-0.6), good (0.61-0.80), and very good (0.81-1.00).

Results: A total of 6,003 records were identified. After screening, 13 studies were included in the final analysis. The overall inter-observer agreement rate for eMS was 0.58 (95% CI: 0.52-0.65) (n=7), 0.66 (0.34-0.99) (n=2) for UCEIS, 0.80 (0.73-0.87) (n=6) for CDEIS, and 0.78 (0.64-0.92) (n=7) for SES-CD. The overall heterogeneity (I2) for these indices ranged from 93.2% to 99.2%.

A few studies assessed the intra-observer agreement rate. The overall effect size was 0.75 (0.73-0.77) (n=1) for eMS,, 0.87 (0.84-0.91) (n=2) for UCEIS, 0.89 (0.86-0.93) (n=1) for CDEIS, and 0.91 (0.89-0.95) (n=1) for SES-CD.

Discussion: The inter-observer agreement rate for eMS, UCEIS, CDEIS, and SES-CD ranged from moderate to good. Despite advancements in validating endoscopic scoring, there is room for further improvement to reduce the variability and maximize the reproducibility of endoscopic indices to facilitate standardization. The intra-observer agreement rate for eMS, UCEIS, CDEIS, and SES-CD based on the available data ranged from good to very good. Solutions to improve inter-rater agreement could allow for more accurate patient assessments, leading to richer, more accurate clinical trial data and potentially improved patient outcomes.

Figure: Figure 1. Forest Plot of Studies Examining Inter-observer Variability of Endoscopic Scoring Systems in Scoring Endoscopic Disease Activity in Ulcerative Colitis and Crohn’s Disease using Random-Effects Meta-analysis


Disclosures:
Jana Hashash: Iterative Health – Grant/Research Support.
Francis Farraye: AbbVie – Advisory Committee/Board Member. Avalo Therapeutics – Advisory Committee/Board Member. BMS – Advisory Committee/Board Member. Braintree Labs – Advisory Committee/Board Member. Fresenius Kabi – Advisory Committee/Board Member. GI Reviewers – Independent Contractor. GSK – Advisory Committee/Board Member. IBD Educational Group – Independent Contractor. Iterative Health – Advisory Committee/Board Member. Janssen – Advisory Committee/Board Member. Pfizer – Advisory Committee/Board Member. Pharmacosmos – Advisory Committee/Board Member. Sandoz Immunology – Advisory Committee/Board Member. Sebela – Advisory Committee/Board Member. Viatris – Advisory Committee/Board Member.
Yeli Wang: Iterative Health – Employee, Stock Options.
Daniel Colucci: Iterative Health – Employee, Stock Options.
Shrujal Baxi: Iterative Health – Employee, Stock Options.
Sadaf Muneer: Iterative Health – Employee, Stock Options.
Faye Yu Ci Ng: Iterative Health – Employee.
Pratik Shingru: Iterative Health – Employee, Stock Options.
Gil Melmed: AbbVie – Consultant. Amgen – Consultant. Arena – Consultant. BI – Consultant. BMS – Consultant. Dieta – Consultant, Owner/Ownership Interest. Eli Lilly – Employee. Ferring – Consultant. Fresenius Kabi – Consultant. Genetech – Consultant. Gilead – Consultant. Janseen – Consultant. Oshi – Consultant, Owner/Ownership Interest. Pfizer – Consultant, Grant/Research Support. Prometheus Labs – Consultant. Samsung Bioepis – Consultant. Takeda – Consultant. Techlab – Consultant. Viatris – Consultant.


Jana G.. Hashash, MD, MSc, FACG1, Francis A. Farraye, MD, MSc, MACG1, Yeli Wang, PhD2, Daniel Colucci, BS2, Shrujal Baxi, MD2, Sadaf Muneer, MS2, Faye Yu Ci Ng, BS2, Pratik Shingru, MD2, Gil Melmed, MD, MS3. P1986 - Inter- and Intra-Observer Variability of Endoscopic Scoring Systems in Ulcerative Colitis and Crohn’s Disease: A Systematic Review and Meta-Analysis, ACG 2023 Annual Scientific Meeting Abstracts. Vancouver, BC, Canada: American College of Gastroenterology.