Improvement of Medical Student Performance in Telemedicine Standardized Patient Encounters Following an Educational Intervention

Telemedicine, “the use of electronic information and communications technologies to provide and support health care when distance separates the participants,” ¹ is an increasingly used form of health care delivery. ² The Association of American Medical Colleges (AAMC) identified the need for telemedicine training to ensure a supply of competent providers. ³ Prior assessments have demonstrated gaps in trainee telemedicine skills, ^4-6 highlighting the need for foundational education early in training. Although nearly 90% of surveyed medical schools reported incorporating telemedicine education into their 2020-2021 curricula, ⁷ limited data on learner-specific needs are available.

Proposed methods for telemedicine education include asynchronous teaching, discussion groups, standardized patient (SP) encounters, and hands-on experience. ⁸ Though which educational interventions can improve performance is unclear, several studies have demonstrated increased self-reported preparedness and confidence with telemedicine curricula. ^9- 11

We aimed primarily to assess the effect of an educational intervention on performance outcomes in SP telemedicine encounters and secondarily to assess differences in performance across two educational interventions.

We performed a prospective cohort study of second-year medical students at Johns Hopkins University School of Medicine during their required longitudinal ambulatory clerkship, which consists of weekly 4-hour ambulatory clinics that include in-person and telemedicine encounters. Sixty students participated in the virtual telemedicine sessions during the clerkship (Figure 1).

With support from the Johns Hopkins Telemedicine Consortium, the clerkship course directors developed the materials for the sessions, including three unique cases (ie, two cases for the pre- and postintervention telemedicine SP encounters and a teaching case), performance checklists for the SP encounters, and educational interventions. Performance checklists for each SP encounter included content items within six domains: (1) history of present illness (HPI); (2) review of systems (ROS); (3) medical history, including past medical, social, and family histories (Hx); (4) physical exam (PE); (5) assessment/plan; and (6) interpersonal communication skills (ICS). Three nonaffiliated ambulatory physicians reviewed the material to ensure that the cases were equivalent in difficulty with appropriate content items. The educational interventions, role-play, and demonstration were selected because they are appropriate educational methods for skills-based objectives and require minimal resources. ¹²

Students completed the preintervention SP encounter in October 2020. Immediately following the encounter, they were randomly assigned to two intervention groups to complete a virtual teaching case. Group 1 completed a role-play session and Group 2 completed a faculty-led demonstration. During the faculty-led demonstration, a facilitator completed a telemedicine demonstration with a patient and a parent. During the role-play session, students were assigned roles as student doctor, patient, or parent and completed the role-play with the help of facilitators. All students then completed a postintervention SP encounter in December 2020. Following the pre- and postintervention SP encounters, SPs completed the performance checklist. In a 4-hour session, SPs were trained by education specialists to standardize scoring.

We described the characteristics of the two intervention groups using descriptive statistics and calculated the students’ median scores and interquartile ranges (IQR) across the six performance domains in the pre- and postintervention cases. We used a Wilcoxon signed rank test to compare performance outcomes pre- and postintervention, and a Wilcoxon rank sum test to compare the medians of the two intervention groups. We used Pearson correlation coefficient to measure the association between telemedicine experience and postintervention score. We used SAS software, version 9.4, for analysis. Two-tailed P<.05 was considered significant. The project was approved by the Johns Hopkins Medicine Institutional Review Board (IRB00270053).

Sixty students participated in the virtual telemedicine sessions between October and December 2020; the students completed 57 total clinical telemedicine encounters as a part of the longitudinal ambulatory clerkship during this time (Table 1).

In the preintervention case, students received the highest median scores in HPI, ROS, Hx, and ICS, and the lowest median scores in PE and assessment/plan (Table 2). Postintervention, the scores of 73.3% of students increased and 23.3% decreased. The median postintervention score significantly increased in PE (ie, median score difference 2, IQR 1-3.5, P<.001), assessment/plan (ie, median score difference 0.5, IQR 0-2, P=.005), and overall performance (ie, median score difference 3, IQR 0-5, P<.001). We found no significant difference in median scores between intervention Groups 1 and 2 for all performance domains. We also found no significant correlation between the overall postintervention performance and the number of patient telemedicine encounters (r=0.0040, P=.98).

The AAMC states that, by graduation, a competent medical student “obtains history,” “conducts [an] appropriate physical examination” and provides a “clinical assessment and treatment plan during a telehealth encounter.^{” 3} In our population, second-year medical students achieved these competencies in history-taking but not in PE or assessment/plan domains at baseline. Performance in these domains improved with role-play and faculty-led demonstrations but was not associated with clinical telemedicine experience. Because both interventions were effective, either can be feasibly implemented in low-resource, virtual learning environments.

The PE is an important educational domain, especially because the virtual PE differs from the in-person PE. ¹³ Students previously identified PE content in a telemedicine curriculum as the most useful component. ⁹ We note that while student performance in both PE and assessment/plan improved postintervention, median scores remained low. Similarly, in non-telemedicine objective structured clinical examinations, second-year students scored the lowest in their differential diagnosis. ¹⁴ Further research is needed to determine whether student performance is due to telemedicine-specific deficiencies or low general competency at this stage of training.

Our study has several limitations. We studied one class of medical students in one institution with a moderate sample size. Case specificity may have impacted performance; we addressed this by using matched samples and independent physician review of the cases, but alternating the case order in future studies could better address this limitation. The use of unique cases also confers a benefit, because it eliminates case memorization from contributing to performance. Additional strengths include the fact that students demonstrated improved performance 3 months after the intervention, indicating longer-term retention of these skills.

In conclusion, we are the first to describe educational interventions to improve medical student performance during telemedicine SP encounters. Future studies should assess performance trajectory throughout medical training, particularly in more advanced domains.