Inter-observer agreement of a new endoscopic score for ulcerative colitis activity: Preliminary experience

Ulcerative colitis (UC) endoscopic scores translate mucosal damage into values standardizing image analysis. Due to potential limits of current endoscopic activity indexes, we have elaborated on a new score, the “Extended Mayo Endoscopic Score (EMES),” and evaluated its inter-observer agreement in a multicenter endoscopy team, comparing concordance with the Mayo subscore. Sixteen UC consecutive patients underwent follow-up colonoscopy. Recorded videos were anonymously loaded on a web platform. Thirteen expert endoscopists evaluated UC activity using both Mayo and EMES. EMES was described in every colon segment: erythema (0: absent, 1: mild, 2: moderate, 3: severe), vascular pattern (0: normal, 1: reduction, 2: disappearance), erosions and ulcers (0: absent, 1: from 1 to 5, 2: 6 to 10, 3: >10). Weighted Fleiss’ kappa with 95% confidence interval (CI) and p-value defined inter-rater agreement. Global inter-observer agreement of EMES was moderate (kappa = 0.56, 95% CI = 0.46–0.67, p < 0.001). The evaluation of each colonic segment showed moderate agreement for all segments: ascending (kappa = 0.46, 95% CI = 0.32–0.60, p < 0.001), transverse (kappa = 0.48, 95% CI = 0.29–0.67, p < 0.001); descending (kappa = 0.49, 95% CI = 0.35–0.64, p < 0.001), sigmoid (kappa = 0.52, 95% CI = 0.39–0.65, p < 0.001) and rectum (kappa = 0.55, 95% CI = 0.42–0.69, p < 0.001). Mayo subscore agreement was similar to global EMES (kappa = 0.53, 95% CI = 0.39–0.66, p = 0.001). Therefore, our report emphasizes the importance of assessing inter-observer agreement for EMES, but also for other known scoring systems, including the Mayo subscore.