The Impact of Tele-Education on Family Medicine Clerkship Students’ Learning Outcomes

The COVID-19 pandemic necessitated changes to medical education content and delivery, ^1-3 but how these changes affected student learning, particularly the clinical components in the third and fourth years of medical school, is not clear. Given the broad potential impact of pandemic curriculum changes, conducting evidence-based assessments to understand the effect on medical education is important. However, much of the relevant literature is international, ^4-8 focuses on preclinical education outcomes, ^9-13 or examines interpersonal or psychological outcomes instead of educational ones. ^14-18 Other studies examine online versus in-person education prior to the pandemic or speculate about the long-term impact. ^19-22 The few relevant studies primarily focus on surgical education. ^23-25 Virtually nothing has been published on the effect of the pandemic on primary care medical education.

This paper focuses on third-year medical students (MS3s) completing their family medicine clerkship ²⁶ at West Virginia University (WVU) and aims to (a) describe changes made due to the pandemic, and to (b) compare educational outcomes for students completing the clerkship in person, virtually, or with a hybrid model of delivery.

The clerkship educates students on the physical, psychological, social, and spiritual needs of patients and families. In addition to supervised interactions with patients, MS3s practice taking patient histories and physical write ups (H&Ps), present patient cases, and participate in various workshops. At WVU, the 8-week clerkship includes participation in 4 weeks of clinical assignments both on campus and at a community site.

This retrospective study examined the impact content modifications and delivery had on MS3 educational outcomes from two campuses of the WVU Department of Family Medicine: Morgantown and Charleston (Appendix 1). Teams on both campuses collaborated to ensure equivalent modification and content delivery using a shared timeline.

Participants included 386 MS3 students completing their clerkship at either campus between July 2, 2018, and June 17, 2022. We adapted the content and delivery of the five major clerkship components, including didactics, clinical experience, case logs, community experience, and standardized patients.

Following national guidelines on the timeline for educational content delivery from the Association of American Medical Colleges, ²⁷ we delivered clerkship content in three ways:

1. In person. Until February 28, 2020, students participated in traditional in-person clinical training in ambulatory, inpatient, and community settings.

2. Virtual only. From March 20, 2020, to August 30, 2020, students moved to virtual-only learning. This included activities such as participating in inpatient rounds virtually, viewing electronic health records remotely, interacting with standardized patients for H&Ps, and participating virtually in ambulatory telehealth visits with faculty and residents.

3. Hybrid. We transitioned to hybrid learning September 1, 2020, following students’ return to clinic with restrictions (ie, mandatory mask, hygiene, and social distancing protocols) and present outcomes for hybrid clerkship students up to June 17, 2022.

We examined three educational outcomes associated with the family medicine clerkship: (a) the midblock assessment score, (b) the National Board of Medical Examiners (NBME) subject exam, and (c) the final numeric score (FNS). Two are standardized exams and one is the clerkship final grade. We used the same grading structure across content delivery types but adjusted the relative weights of the grading components (Table 1 ).

We analyzed data using SAS version 9.4 (SAS Institute). We assessed normality with the Shapiro-Wilk test. We performed two sample t tests or Wilcoxon rank sum tests to compare students’ learning outcomes between campuses. We used one-way analysis of variance (ANOVA) or the Kruskal-Wallis test to examine differences in educational outcomes stratified by content delivery type. All tests were two-sided, and P values <.05 were considered statistically significant. This study was exempted from review by the WVU Institutional Review Board (Protocol #2209650754).

Of 386 MS3s, 232 (60.1%) attended the Morgantown campus and 154 (39.9%) attended the Charleston campus. A comparison for equivalence revealed no significant differences in students’ midblock assessment scores and NBME exam scores between campuses. However, students from Charleston (mean [M]=89.2, standard deviation [SD]=2.5) had significantly higher FNSs than students from Morgantown (M=88.4, SD=3.0). Table 2 presents the characteristics of the student sample.

With respect to content delivery type, 164 (42.5%) received clerkship content in person, 36 (9.3%) virtually, and 186 (48.2%) in hybrid format. An ANOVA followed by Tukey’s HSD test showed a significant difference in mean student midblock assessment scores across content delivery type (F=8.06, df=2, P=.0004). Specifically, students receiving virtual-only training (M=76.4, SD=9.1) had significantly higher midblock assessment scores than students receiving hybrid (M=71.7, SD=8.8) or in-person training (M=74.5, SD=7.2). A Kruskal-Wallis test revealed no significant differences in students’ NBME exam scores or FNSs across the three types of content delivery (Table 3 ).

We found no significant differences in clerkship students’ NBME exam scores and FNSs with respect to content delivery type. This finding is consistent with limited literature on the impact of the pandemic on student learning outcomes, which found no significant differences in NBME exam scores for MS3s in their surgical clerkship pre- and postpandemic. ^{23, 25} In addition, virtual-only students’ midblock assessment scores were significantly higher than those of in-person and hybrid students. Because this exam is based on Aquifer cases that take a significant amount of time to review, we suspect virtual students had more time to complete these cases than their in-person and hybrid peers.

Finding no significant differences in students’ NMBE exam scores or FNSs based on educational content delivery type is noteworthy. Medical educators worldwide had to pivot quickly and change the way curricula were delivered with no specialized training in tele-education. To ensure that students were prepared for standardized exams and clinical practice, educators created innovative ways of simulating clinical experience. This study provided one snapshot of how we attempted to provide equivalent education across delivery types during our family medicine clerkship.

This study had known limitations. The cross-sectional design prevented us from making causal conclusions. Further, the metrics we present overlap in the students they summarize (ie, the midblock exam score and the NBME exam score made up a combined total of 30% of the FNS). Despite overlap, the data are critical indicators of clerkship performance. Additionally, only 9.3% of the sample received virtual-only training. Variability in scores was relatively low, however, increasing confidence that these results are reliable. Despite limitations, this study contributes to our understanding of how medical students’ clinical education was affected by the pandemic.

These data suggest that our clerkship students received at least equivalent clinical instruction across in-person, virtual-only, and hybrid delivery types as reflected in three educational outcomes. Students’ performance satisfied national requirements for content mastery across delivery types. Perhaps this says as much about our students’ resilience as it does about curricula changes. More research is needed to understand what characteristics or factors mediated or moderated student learning. We also need to set standards for increasingly technology-based medical education going forward to ensure that the next generation of physicians provides high-quality care to patients.