INTRODUCTION

Colorectal cancer (CRC) is the third most common cancer worldwide, and high-quality colonoscopy is paramount to effective prevention of CRC. The adenoma detection rate (ADR) is the dominant colonoscopy quality metric, and it has been validated as an independent predictor for risk of postcolonoscopy CRC (1–4).

A variety of educational and behavioral interventions have been proposed to monitor and improve colonoscopy quality at the operator and institutional levels, with inconsistent results. In a study at a single academic center, Keswani et al (5) assessed the effect of physician report cards and implementing institutional standards of practice on ADRs. There were 20 endoscopists who had done 12,894 colonoscopies over a nearly 2-year period; mean ADR increased by 3% after report card distribution, with an additional 8% increase after standard-of-practice implementation. Conversely, Shaukat et al (6). reported the results of an educational initiative to increase ADR among community-based physicians, which included scheduled reporting of ADR, didactic sessions, direct feedback to the lowest ADR detectors, and financial penalties for inadequate withdrawal times. During their 3-year study period, there was no significant improvement in ADR. We have previously reported the effect of a quarterly report card with our group's anonymized quality data (7). This intervention was associated with an increase in pooled ADR from 44.7% to 53.9%, which was driven by increased detection of proximal adenomas. The reasons for these inconsistent findings include different study settings, baseline endoscopist ADRs, and heterogeneity of interventions that may be coadministered with the report card.

A recent systematic review and meta-analysis examined studies of educational interventions to improve ADR (8). Interventions included interactive modules on polypectomy with ADR feedback, live endoscopy courses and training, didactic presentations, and report cards with standard-of-practice feedback to the lowest detectors. The pooled baseline ADR was 26.5% and 35.4% after intervention, with education associated with a 29% relative increase in ADR (relative risk 1.29, 95% confidence interval 1.22–1.37). However, the durability of the effect of interventions was uncertain.

There remain important areas of uncertainty for using report cards as a colonoscopy quality monitoring modality. Notably, most studies, including ours, reported the short-term effect of quality interventions on ADR (7). An important unresolved question is whether any positive effect on ADR and other quality metrics has long-term durability. This study aims to report our long-term experience with the quarterly colonoscopy quality report card. We hypothesized that long-term implementation of report cards would be paralleled by additional positive effects on ADRs, similar to that seen with initial implementation.

METHODS

The study was deemed not to involve human subject research by the Institutional Review Board at Indiana University-Purdue University at Indianapolis and by the Richard L. Roudebush Veterans Affairs Medical Center Research and Development Committee. We conducted a retrospective study of prospectively administered quarterly colonoscopy quality report cards at the Roudebush Veteran's Affairs Medical Center (VAMC) between April 1, 2012, and August 31, 2019. The start date was chosen to follow the time frame of our previous study describing initial implementation of the quality report card initiative at our center (7). In 2020, the COVID-19 pandemic significantly disrupted normal operations, including suspension of elective procedures and subsequent resumption at a restricted pace. The report card was not administered regularly during that phase, and it was reinstated in 2021. We, therefore, opted for 2012–2019 because it covers a period when the quarterly card was administered continuously, without interruptions.

Report cards

We have previously described the implementation and structure of the quality report card at our center (7). Briefly, in 2009, physicians who performed endoscopic procedures at our VAMC started to receive quarterly report cards, which included individual endoscopists' cecal intubation rates, withdrawal times, and ADRs. The ADR used here is a composite metric derived from screening, surveillance, and diagnostic colonoscopies. We have previously shown that this overall ADR was not significantly different from the conventional ADR based on screening colonoscopies, with the added advantage of being easier to derive, and increasing sample sizes for more precise ADR estimates (9). The data forming the basis of the report cards are obtained from the VAMC endoscopic and clinical electronic databases, then tabulated, anonymized, and provided to the entire group by the gastroenterology section chief. Individual physicians then received their personal data confidentially so they could compare their performance with others, but the rest of the report remained deidentified. The procedures were performed by attending physicians or gastroenterology fellows under direct supervision. To be included in this follow-up study, endoscopists should have performed at least 50 procedures per year and contributed at least 4 consecutive quarters of data. The 17 attending physicians who were part of this study included 16 board-certified gastroenterologists and one colorectal surgeon.

Statistical methods

Linear regression models were used to determine and test slopes over time for each metric by physician and to determine whether slopes differed for data above vs below the median for each outcome. Analyses were performed for quarterly and yearly data to assess whether ADR trends varied based on frequency of calculation. The yearly data were not a simple average of the data from 4 quarters, but they took into account the different numbers of patients from each quarter (thus, quarterly and annual ADRs were different because the number of patients was not constant across quarters). The main analysis assessed ADR trends as linear; we also used the locally estimated scatterplot smoothing (LOESS) method to analyze ADR trends within the study time frame. Each physician has multiple ADR measurements over time, calculated using either yearly or quarterly measurements. The variability of the ADRs within a physician can be represented by the SD of the ADRs; low SDs indicate stable ADRs, whereas physicians with less consistency in their ADRs over time have higher SDs. The difference in the SDs calculated using the yearly and quarterly ADR measurements were evaluated to determine whether the consistency in the ADRs over time was measured differently using yearly compared with quarterly ADR measurements. A 2-sided 5% significance level was used for all tests. Analyses were performed using SAS version 9.4 (SAS Institute, Cary, NC) and R (https://www.R-project.org/).

RESULTS

A total of 24,361 colonoscopies were performed by 17 endoscopists over a mean (range) of 18 quarters (6–28). The mean quarterly ADR (±SD) was 51.7% (±11.7%) while the mean yearly ADR was 47.2% (±13.8%). Over the study time frame, there was a small increase in overall ADR based on both quarterly and yearly measurements (slope + 0.6%, P = 0.02; and slope +2.7%, P < 0.001, respectively). However, most of the endoscopists had no statistically significant change in their ADRs (Table 1). The LOESS approach was used to examine whether the overall increase in ADR was constant or plateaued after an initial increase. In the LOESS analysis, most of the overall ADR increase appeared in the first 2 years of follow-up, with subsequent plateauing (Figure 1).

Table 1. - Mean adenoma detection rate and slopes of adenoma detection rate between yearly and quarterly measurements Quarterly Yearly Physician Mean (± SD) Slope SE P value Mean (± SD) Slope SE P value All 51.7% (± 11.7%) 0.6% 0.2% 0.021 47.2% (± 13.8%) 2.7% 0.4% <0.001 Physician A 48.5% (± 7.8%) 1.1% 2.6% 0.689 49.3% (± 5.7%) −3.0% 2.3% 0.314 Physician B 39.4% (± 5.4%) 0.2% 0.5% 0.628 36.8% (± 9.4%) 1.6% 1.4% 0.306 Physician C 66.3% (± 6.0%) 1.6% 0.7% 0.022 64.1% (± 7.6%) 2.3% 1.2% 0.113 Physician D 53.4% (± 9.2%) 1.2% 0.8% 0.155 47.9% (± 13.3%) 3.5% 1.7% 0.084 Physician E 33.8% (± 11.5%) 4.7% 10.8% 0.692 21.0% (± 15.0%) 13.6% 6.3% 0.275 Physician F 46.4% (± 6.3%) 0.6% 2.0% 0.761 48.2% (± 4.8%) 1.8% 2.3% 0.531 Physician G 48.0% (± 11.5%) 0.1% 1.1% 0.948 39.6% (± 9.1%) 1.9% 1.3% 0.201 Physician H 60.3% (± 8.6%) 1.2% 0.8% 0.118 54.0% (± 13.3%) 3.6% 1.6% 0.070 Physician I 55.1% (± 6.2%) −1.3% 0.9% 0.167 53.2% (± 9.1%) 1.4% 2.3% 0.589 Physician J 51.8% (± 7.5%) 0.4% 1.5% 0.802 48.9% (± 12.2%) 3.6% 3.9% 0.431 Physician K 38.5% (± 11.0%) −1.4% 1.3% 0.290 35.5% (± 12.2%) 1.6% 2.4% 0.533 Physician La 50.1% (± 8.1%) 14.2% 3.4% 0.052 35.9% (± 14.9%) −21.0% Physician M 48.2% (± 11.3%) 11.8% 3.8% 0.026 38.6% (± 15.3%) 7.1% 13.6% 0.692 Physician N 60.0% (± 7.5%) 2.0% 2.9% 0.506 59.7% (± 2.8%) 0.8% 2.7% 0.819 Physician O 52.9% (± 6.4%) 1.3% 1.2% 0.277 52.1% (± 5.2%) 1.9% 1.6% 0.302 Physician P 59.1% (± 10.4%) 6.0% 2.5% 0.043 59.2% (± 8.6%) 5.6% 2.6% 0.160 Physician Q 55.2% (± 10.7%) 0.4% 1.1% 0.747 48.5% (± 13.6%) 3.0% 1.9% 0.169

The bolded entries represent significant P values (P < 0.05).

SE, standard error.

aPhysician L performed colonoscopies over 6 consecutive quarters, so there were insufficient data to assess yearly measures over time.

Figure 1.:

LOESS plot for overall ADR trends during the study time frame. ADR, adenoma detection rate; LOESS, locally estimated scatterplot smoothing.

Overall mean quarterly and yearly cecal intubation rates (99.6% ± 0.9%; 99.6% ± 0.5%) and withdrawal times (16.3 ± 5.8 minutes; 16.7 ± 5.9 minutes) did not change significantly. In addition, the slopes of quarterly and yearly cecal intubation rates (slope 0.0%, P = 0.20; slope 0.0%, P = 0.217) and withdrawal times (slope +0.1%, P = 0.732; slope −0.2%, P = 0.587) also did not significantly change. Individual endoscopist data for cecal intubation rates and withdrawal times are provided in Supplementary Digital Content (see Supplementary Tables 1 and 2, https://links.lww.com/CTG/A978).

The SDs of ADRs showed no significant difference when calculated using yearly compared with quarterly ADR measurements (P = 0.064). Individual endoscopists' ADR SD differences between yearly and quarterly measurements ranged from −4.7% to +6.8% (Table 2). Individual endoscopists' cecal intubation rate differences between yearly and quarterly measurements ranged from −1.3% to 0.0% (see Supplementary Table 3, https://links.lww.com/CTG/A978), which were statistically significant (P < 0.001) but clinically similar. Individual endoscopists' withdrawal time differences between yearly and quarterly measurements ranged from −1.9 to +3.1 (see Supplementary Table 4, https://links.lww.com/CTG/A978), with no significant difference (P = 0.356).

Table 2. - Individual endoscopists' adenoma detection rate SD differences between yearly and quarterly measurements Physician Quarterly Yearly Yearly-quarterly All 8.7% 10.8% 2.1% Physician A 7.8% 5.7% −2.1% Physician B 5.4% 9.4% 4.0% Physician C 6.0% 7.6% 1.6% Physician D 9.2% 13.3% 4.1% Physician E 11.5% 15.0% 3.5% Physician F 6.3% 4.8% −1.5% Physician G 11.5% 9.1% −2.4% Physician H 8.6% 13.3% 4.7% Physician I 6.2% 9.1% 2.9% Physician J 7.5% 12.2% 4.6% Physician K 11.0% 12.2% 1.3% Physician L 8.1% 14.9% 6.8% Physician M 11.3% 15.3% 4.0% Physician N 7.5% 2.8% −4.7% Physician O 6.4% 5.2% −1.2% Physician P 10.4% 8.6% −1.7% Physician Q 10.7% 13.6% 2.8%
DISCUSSION

We found that the administration of a quarterly colonoscopy quality report card was associated with a modest increase in overall pooled ADR, but most endoscopists did not experience a significant change in their individual ADRs. This observation was consistent, regardless of baseline ADR. The overall ADR increase occurred mostly in the first 2 years of follow-up. To our knowledge, this is the first study to report a long-term experience with regularly administered anonymized colonoscopy quality report cards. Our previous study had shown that implementation of the report card was independently associated with a short-term increase in ADR and cecal intubation rates (7). Taken together, the previous study and the current one suggest that after implementation of monitoring, there is initial improvement in endoscopists' performance that tends to stabilize over the long term. This initial improvement may have been due to a Hawthorne effect, where endoscopists' knowledge of being monitored leads to a significant increase in ADR (10). It is also possible that temporal trends are also contributing to the ADR increase, in addition to an independent causal effect of the report card.

It is important to note that 10 of 17 endoscopists in our study had aspirational quarterly ADRs (ADR >50%), and thus, it is plausible that the lack of further improvement is due to a ceiling effect. Conversely, there were 2 endoscopists with ADRs < 40% and neither improved significantly over the study period. While both exceeded the minimum threshold of 30% for a male population (11), their performance can still be viewed as inferior compared with that of the rest of the group. Aiming for aspirational ADRs should be a goal for every endoscopist, especially given the known inverse association between ADR and postcolonoscopy CRC (12,13). In this context, our study suggests that providing a deidentified quality report card alone, without other interventions, may not be sufficiently impactful to achieve aspirational targets.

Our analysis of SD of ADRs to represent variability over time showed no significant difference between yearly and quarterly measurements, implying stability of ADR estimates regardless of whether they are calculated quarterly or annually. These findings are broadly consistent with those of El Rahyel et al (14), who evaluated a group of 11 endoscopists and assessed their ADR trends, without provision of quarterly report cards. They found that all but one met the recommended threshold ADR of 25% continuously over the study period and that 6 had stable ADRs while 5 had an increase in ADR. The authors concluded that for endoscopists who meet minimum ADR thresholds, performance tends to remain stable over time and thus resources could be diverted to target other quality measures. Our findings also show that frequent monitoring for endoscopists with adequate or high performance was not associated with significant changes in the long term.

An additional important observation from our study is that there was a continued upward trend in ADR for the first 2 years of the long-term follow-up phase, followed by plateauing and stability. This suggests that the report card intervention could motivate endoscopists to reach, and then stay at, their colonoscopy quality potential. We view the long-term findings broadly as a success because there was no deterioration in ADR or regression to the mean. Our findings also suggest that it may be reasonable to monitor and report quality metrics intensively early on, with subsequent de-escalation of the frequency of monitoring once ADRs have reached satisfactory and stable levels. Additional studies are needed to determine optimal duration of intensive monitoring before de-escalation because this is expected to vary significantly based on specifics of different endoscopy groups and baseline ADRs. It is also critical to note that our findings do not mean that monitoring ADRs or quality in general is less important, but rather that it could be less hawkish when performance is satisfactory or aspirational as is the case for several of the endoscopists in this study. Our unit is considering less frequent report card administration as a result.

Strengths of our study include the unique and curated data set of quality report cards assessing colonoscopist performance over a long observation period. There are, however, some limitations: Some endoscopists joined our group during the study time frame and others left. Thus, the observation period is not uniform for all endoscopists. Second, this group of endoscopists had relatively high ADRs, and it not clear whether the results can be extrapolated to endoscopists with lower quality benchmarks. Third, we cannot exclude the possibility of endoscopist disengagement with this specific type of monitoring: There were no instances of suboptimal ADRs over the study time frame, and hence, additional educational or upskilling interventions that could have influenced engagement were not required nor administered. We also cannot assess whether alternative report card formats (e.g., deanonymized) could have spurred additional competition between endoscopists to further improve ADRs. Additional study is warranted in other practice settings and in groups with wider ranges of ADRs. One more consideration is whether technology or unit practice changes could have affected the findings. However, our unit has used Olympus 190 series colonoscopes since 2013 and implemented split bowel preparation regimens since 2008, essentially spanning the study time frame. Finally, we cannot determine the optimal report card administration frequency because answering this question with precision requires assessing endoscopists' ADRs in response to different reporting intervals.

In conclusion, we found that long-term quarterly colonoscopy quality monitoring did not significantly change individual ADRs but had a modest and stable improvement in overall pooled ADR. For endoscopists with baseline high ADR and who abide by the precepts of high-quality colonoscopy, frequent monitoring and reporting of colonoscopy quality metrics may not be necessary; however, the optimal frequency requires additional study.

CONFLICTS OF INTEREST

Guarantor of the article: Charles Kahi, MD, MS.

Specific author contributions: All authors of this research paper have directly participated in the planning, execution, or analysis of the study, and all authors have read and approved the final version submitted herein.

Financial support: None to report.

Potential competing interests: None to report.

Study Highlights

WHAT IS KNOWN ✓ Adenoma detection rate (ADR) is a validated measure of colonoscopy quality. ✓ We have previously shown that a quarterly report card led to an increase in ADR over the short term. ✓ The long-term effect of such reporting on ADR trends is not known.

WHAT IS NEW HERE ✓ Long-term quarterly colonoscopy quality monitoring was paralleled by a modest and stable improvement in overall pooled ADR. ✓ There were no significant changes in individual ADRs and other quality metrics over the long term. ✓ For endoscopists with baseline high ADR, frequent monitoring of quality metrics may not be necessary. REFERENCES 1. Kaminski MF, Regula J, Kraszewska E, et al. Quality indicators for colonoscopy and the risk of interval cancer. N Engl J Med 2010;362(19):1795–803. 2. Kaminski MF, Wieszczy P, Rupinski M, et al. Increased rate of adenoma detection associates with reduced risk of colorectal cancer and death. Gastroenterology 2017;153(1):98–105. 3. Wieszczy P, Waldmann E, Løberg M, et al. Colonoscopist performance and colorectal cancer risk after adenoma removal to stratify surveillance: Two nationwide observational studies. Gastroenterology 2021;160(4):1067–74.e6. 4. Corley DA, Jensen CD, Marks AR, et al. Adenoma detection rate and risk of colorectal cancer and death. N Engl J Med 2014;370(14):1298–306. 5. Keswani RN, Yadlapati R, Gleason KM, et al. Physician report cards and implementing standards of practice are both significantly associated with improved screening colonoscopy quality. Am J Gastroenterol 2015;110(8):1134–9. 6. Shaukat A, Oancea C, Bond JH, et al. Variation in detection of adenomas and polyps by colonoscopy and change over time with a performance improvement program. Clin Gastroenterol Hepatol 2009;7(12):1335–40. 7. Kahi CJ, Ballard D, Shah AS, et al. Impact of a quarterly report card on colonoscopy quality measures. Gastrointest Endosc 2013;77(6):925–31. 8. Causada-Calo NS, Gonzalez-Moreno EI, Bishay K, et al. Educational interventions are associated with improvements in colonoscopy quality indicators: A systematic review and meta-analysis. Endosc Int Open 2020;08(10):E1321–31. 9. Kaltenbach T, Gawron A, Meyer CS, et al. Adenoma detection rate (ADR) irrespective of indication is comparable to screening ADR: Implications for quality monitoring. Clin Gastroenterol Hepatol 2021;19(9):1883–9.e1. 10. Fletcher RFS, Wagner E. Clinical Epidemiology: The Essentials. 3rd edn. Lippincott Williams and Wilkins: Philadelphia, PA, 1996. 11. Rex DK, Schoenfeld PS, Cohen J, et al. Quality indicators for colonoscopy. Gastrointest Endosc 2015;110(1):72–90. 12. Rex DK. Detection measures for colonoscopy: Considerations on the adenoma detection rate, recommended detection thresholds, withdrawal times, and potential updates to measures. J Clin Gastroenterol 2020;54(2):130–5. 13. Shaukat A, Rector TS, Church TR, et al. Longer withdrawal time is associated with a reduced incidence of interval cancer after screening colonoscopy. Gastroenterology 2015;149(4):952–7. 14. El Rahyel A, Vemulapalli KC, Lahr RE, et al. Implications of stable or increasing adenoma detection rate on the need for continuous measurement. Gastrointest Endosc 2022;95(5):948–53.e4.