The objective of this study was to, first, to quantify the frequency, types, and temporal trends of clinical substitutions among pediatric clerkship students. Second, the study investigated the associations between clinical substitutions and students’ performance on the NBME and H&P.

From 2018 to 2023, over 90% of students had two or fewer clinical substitutions. Moreover, the frequency of substitutions did not vary by clerkship sites with or without pediatric GME programs. As medical schools aim to broaden their pediatric clerkship programs to include more diverse hospital locations, the USU pediatric clerkship provides an example. The results indicate that the USU pediatric clerkship can effectively deliver numerous, high-yield, real patient encounters at 12 varied sites, regardless of GME programs. When comparing across blocks (ie, time of year), findings showed only one significant difference in total number of clinical substitutions (Table 2). There are several anecdotal reasons for the increase in substitutions during July and August, such as decrease in pediatric illness while children are out of school or the start of the new GME year when Attendings’ focus is primarily on the incoming residents. Further investigation is needed to identify more definitive reasons for this increase.

Results showed a statistically significant increase in clinical substitutions from 2019 to 2020. This is in line with a substantial decrease in pediatric patient census during the COVID-19 pandemic [17, 18]. Additionally, at the USU, COVID-19 precautions in 2020 shortened the length of core clerkships (from five weeks to four), thereby reducing students’ overall time to care for patients with a variety of conditions. Furthermore, during 2020 and 2021, the students’ themselves may have been exposed to or contracted COVID-19, requiring transition to an isolated, virtual learning environment. Unexpectedly, the elevated numbers of clinical substitutions persisted throughout 2023, despite a learning environment that more closely resembled the pre-pandemic clerkships. For instance, there was a post-pandemic rise in pediatric patient census [19], a reduction in COVID severity and quarantine restrictions, as well as USU medical students returning to a regular clerkship schedule in 2022. One potential reason for the continued higher frequency of clinical substitutions through 2023 is learners’ attitudes toward online learning. Specifically, recent research found that many medical students in clerkships expressed a positive response to online learning and the autonomy it permitted within their schedules [20]. Consequently, the authors hypothesize that some medical students may have chosen to complete screen-based simulations at their convenience rather than waiting for a real-patient encounter.

The relatively high percentage of clinical substitutions for acute gastroenteritis may be surprising to most pediatricians considering this illness accounts “for 1.5 million office visits, 200,000 hospitalizations, and 300 deaths” of children in the United States each year [21]. And, while the rates of pediatric patients visiting both inpatient and outpatient for acute gastroenteritis decreased within the Military Health System during 2020 and 2021 [22], this does not explain the consistently high percentage of substitutions throughout the study period. The authors surmise that one potential reason for substitutions of these experiences may be related to misinterpretation of the clinical experience. For example, if medical students saw children with vomiting or diarrhea but failed to classify it as acute gastroenteritis, they may have felt the need to utilize the Aquifer case specifically labeled "acute gastroenteritis" to satisfy the requirement.

By 2020, the second most frequently substituted clinical experience became AOM. Although AOM has traditionally been one of the most common illnesses seen by pediatricians [23], recent research out of the United Kingdom has shown that from 2003–2019, occurrence of AOM declined by over 40% with the introduction of pneumococcal conjugate vaccines [24]. Additionally, researchers suspect reduced viral illness during the COVID period [25] may have impacted, and continues to impact, the number of visits for secondary infections like AOM. Consequently, pediatric clerkship students may not have had the opportunity to see children with AOM.

The small, but significant negative correlation between clinical substitutions and NBME scores supports Kolb’s experiential learning theory that active clinical participation is crucial for knowledge and skill development associated with this assessment [4]. This paradigm is reflected in a study that found third year medical students’ OSCE scores were positively correlated to the total number of patients they cared for during their clerkship [11]. On the other hand, while screen-based simulations may help students to develop knowledge of systems (eg, immune, musculoskeletal, respiratory) as well as their application of concepts, diagnostic knowledge, interventions and management, there is less accountability for how actively engaged students are in learning. For instance, a student may "click through” a screen-based simulation without engaging with the material. Nevertheless, it must be emphasized that the magnitude of the substitution effect was extremely small – about 6% of one standard deviation – suggesting that, although in-person clinical encounters remain ideal, the modest decrement in NBME performance may be an acceptable tradeoff when clinical substitutions are needed to ensure coverage of required experiences.

The results of there being no significant relationship between clinical substitutions and performance on the H&P written exam are readily explicable. Namely, the open-resource nature of the retrospectively written H&P assignment measures students’ documentation, information literacy, and synthesis skills in an untimed setting, and it allows for review and refinement before submission. Consequently, in-person clinical experiences with patients likely do not contribute as much to students’ ability to perform on the written H&P. This is because the format of the H&P assignment effectively decouples raw data collection from the final presentation, allowing cognitive skills like organization and editing to dominate the score. This contrasts with the NBME, which assesses integrated clinical knowledge and reasoning in a timed setting without the use of external resources (eg, notes, textbooks), a format where the rich, integrated knowledge gained from varied, real-time patient encounters provides a critical and unique advantage for rapid pattern recognition and decision-making. All in all, the results of minimal association between clinical substitutions and performance outcomes suggests that the use of screen-based simulations is still a reasonable alternative to in-person patient encounters in a pediatrics clerkship.

There are limitations to this study. First, the data on the frequency and type of clinical substitutions were self-reported by the students, which may have led to an over- or under-estimation of the true use of clinical substitutions. The study team omitted all clinical trackers for which it was unclear if the student had done a clinical substitution or if it was incomplete. Second, due to changes in assignments and clinical performance grading policies within the USU Department of Pediatrics from 2018–2023, only two performance outcomes (ie, NBME and H&P scores) remained as valid and comparable for our analysis. Since 2018 the H&P has allowed students to resubmit the assignment for a potentially higher score, which may have inflated the means for the H&P. Lastly, given the Military Health System and the USU are nestled in a unique environment with a well-developed pediatrics clerkship curriculum, in addition to this study being limited to only the pediatrics clerkship, results may not be generalizable to a broad scope of institutions or other clerkships.