Improvement of Medical Student Performance in Telemedicine Standardized Patient Encounters Following an Educational Intervention

Emily M. Murphy, MD | Ariella Stein, MPH | Amit Pahwa, MD | Maura McGuire, MD | Tina Kumra, MD

Fam Med. 2023;55(6):400-404.

DOI: 10.22454/FamMed.2023.523442

Return to Issue


Background and Objectives: The Association of American Medical Colleges identifies telemedicine competence as an important skill for graduating medical students, but which educational methods are effective in improving student performance is unclear. We aimed to assess the impact of two educational interventions on student performance in telemedicine standardized patient encounters.

Methods: Sixty second-year medical students participated in the telemedicine curriculum during their required longitudinal ambulatory clerkship. Students first completed a preintervention telemedicine standardized patient (SP) encounter in October 2020. They subsequently were assigned to two intervention groups (ie, a role-play intervention, N=30; a faculty demonstration, N=30) and completed a teaching case. In December 2020, they completed a postintervention telemedicine SP encounter. Each case was a unique clinical scenario. SPs scored the encounters across six domains based on a standardized performance checklist. We compared the median scores for these domains and the median total score pre- and postintervention (using Wilcoxon signed rank and rank-sum tests) and the difference in median score by intervention type.

Results: Students scored highly in history-taking and communication performance but had low physical exam (PE) and assessment/plan scores. Postintervention, median scores in PE (ie, median score difference 2, interquartile ranges [IQR] 1-3.5, P<.001), assessment/plan (ie, median score difference 0.5, IQR 0-2, P=.005), and overall performance improved significantly (ie, median score difference 3, IQR 0-5, P<.001).

Conclusions: Early medical students had low performance at baseline in telemedicine PE and assessment/plan skills, but both a role-play intervention and faculty demonstration led to significant increases in student performance.


Telemedicine, “the use of electronic information and communications technologies to provide and support health care when distance separates the participants,” 1 is an increasingly used form of health care delivery. 2 The Association of American Medical Colleges (AAMC) identified the need for telemedicine training to ensure a supply of competent providers. 3 Prior assessments have demonstrated gaps in trainee telemedicine skills, 4-6 highlighting the need for foundational education early in training. Although nearly 90% of surveyed medical schools reported incorporating telemedicine education into their 2020-2021 curricula, 7 limited data on learner-specific needs are available.

Proposed methods for telemedicine education include asynchronous teaching, discussion groups, standardized patient (SP) encounters, and hands-on experience. 8 Though which educational interventions can improve performance is unclear, several studies have demonstrated increased self-reported preparedness and confidence with telemedicine curricula. 9- 11

We aimed primarily to assess the effect of an educational intervention on performance outcomes in SP telemedicine encounters and secondarily to assess differences in performance across two educational interventions.


We performed a prospective cohort study of second-year medical students at Johns Hopkins University School of Medicine during their required longitudinal ambulatory clerkship, which consists of weekly 4-hour ambulatory clinics that include in-person and telemedicine encounters. Sixty students participated in the virtual telemedicine sessions during the clerkship (Figure 1).

With support from the Johns Hopkins Telemedicine Consortium, the clerkship course directors developed the materials for the sessions, including three unique cases (ie, two cases for the pre- and postintervention telemedicine SP encounters and a teaching case), performance checklists for the SP encounters, and educational interventions. Performance checklists for each SP encounter included content items within six domains: (1) history of present illness (HPI); (2) review of systems (ROS); (3) medical history, including past medical, social, and family histories (Hx); (4) physical exam (PE); (5) assessment/plan; and (6) interpersonal communication skills (ICS). Three nonaffiliated ambulatory physicians reviewed the material to ensure that the cases were equivalent in difficulty with appropriate content items. The educational interventions, role-play, and demonstration were selected because they are appropriate educational methods for skills-based objectives and require minimal resources. 12

Students completed the preintervention SP encounter in October 2020. Immediately following the encounter, they were randomly assigned to two intervention groups to complete a virtual teaching case. Group 1 completed a role-play session and Group 2 completed a faculty-led demonstration. During the faculty-led demonstration, a facilitator completed a telemedicine demonstration with a patient and a parent. During the role-play session, students were assigned roles as student doctor, patient, or parent and completed the role-play with the help of facilitators. All students then completed a postintervention SP encounter in December 2020. Following the pre- and postintervention SP encounters, SPs completed the performance checklist. In a 4-hour session, SPs were trained by education specialists to standardize scoring.

We described the characteristics of the two intervention groups using descriptive statistics and calculated the students’ median scores and interquartile ranges (IQR) across the six performance domains in the pre- and postintervention cases. We used a Wilcoxon signed rank test to compare performance outcomes pre- and postintervention, and a Wilcoxon rank sum test to compare the medians of the two intervention groups. We used Pearson correlation coefficient to measure the association between telemedicine experience and postintervention score. We used SAS software, version 9.4, for analysis. Two-tailed P<.05 was considered significant. The project was approved by the Johns Hopkins Medicine Institutional Review Board (IRB00270053).


Sixty students participated in the virtual telemedicine sessions between October and December 2020; the students completed 57 total clinical telemedicine encounters as a part of the longitudinal ambulatory clerkship during this time (Table 1).

In the preintervention case, students received the highest median scores in HPI, ROS, Hx, and ICS, and the lowest median scores in PE and assessment/plan (Table 2). Postintervention, the scores of 73.3% of students increased and 23.3% decreased. The median postintervention score significantly increased in PE (ie, median score difference 2, IQR 1-3.5, P<.001), assessment/plan (ie, median score difference 0.5, IQR 0-2, P=.005), and overall performance (ie, median score difference 3, IQR 0-5, P<.001). We found no significant difference in median scores between intervention Groups 1 and 2 for all performance domains. We also found no significant correlation between the overall postintervention performance and the number of patient telemedicine encounters (r=0.0040, P=.98).


The AAMC states that, by graduation, a competent medical student “obtains history,” “conducts [an] appropriate physical examination” and provides a “clinical assessment and treatment plan during a telehealth encounter. 3 In our population, second-year medical students achieved these competencies in history-taking but not in PE or assessment/plan domains at baseline. Performance in these domains improved with role-play and faculty-led demonstrations but was not associated with clinical telemedicine experience. Because both interventions were effective, either can be feasibly implemented in low-resource, virtual learning environments.

The PE is an important educational domain, especially because the virtual PE differs from the in-person PE. 13 Students previously identified PE content in a telemedicine curriculum as the most useful component. 9 We note that while student performance in both PE and assessment/plan improved postintervention, median scores remained low. Similarly, in non-telemedicine objective structured clinical examinations, second-year students scored the lowest in their differential diagnosis. 14 Further research is needed to determine whether student performance is due to telemedicine-specific deficiencies or low general competency at this stage of training.

Our study has several limitations. We studied one class of medical students in one institution with a moderate sample size. Case specificity may have impacted performance; we addressed this by using matched samples and independent physician review of the cases, but alternating the case order in future studies could better address this limitation. The use of unique cases also confers a benefit, because it eliminates case memorization from contributing to performance. Additional strengths include the fact that students demonstrated improved performance 3 months after the intervention, indicating longer-term retention of these skills.

In conclusion, we are the first to describe educational interventions to improve medical student performance during telemedicine SP encounters. Future studies should assess performance trajectory throughout medical training, particularly in more advanced domains.


  1. Institute of Medicine (US) Committee on Evaluating Clinical Applications of Telemedicine; Field MJ, ed. Telemedicine: A Guide to Assessing Telecommunications in Health Care. National Academies Press; 1996.
  2. Samson LW, Tarazi W, Turrini G, Sheingold S. Medicare Beneficiaries’ Use of Telehealth in 2020: Trends by Beneficiary Characteristics and Location. Office of the Assistant Secretary for Planning and Evaluation. December 3, 2021. Accessed September 27, 2022.
  3. Association of American Medical Colleges. Telehealth Competencies Across the Learning Continuum. AAMC; 2021.
  4. Kumra T, McGuire M, Stein A, Pahwa AK. Telemedicine clinical skills needs assessment in early medical students. Fam Med. 2022;54(4):294-297.
  5. Sartori DJ, Olsen S, Weinshel E, Zabar SR. Preparing trainees for telemedicine: a virtual OSCE pilot. Med Educ. 2019;53(5):517-518. doi: 10.1111/medu.13851
  6. Kosak Z. Incorporating telehealth into family medicine training: An emerging need. Fam Med. 2022;54(4):316-317. doi: 10.22454/FamMed.2022.201063
  7. Association of American Medical Colleges. Content documentation in required courses and elective courses. Accessed September 27, 2021.
  8. Jumreornvong O, Yang E, Race J, Appel, J. Telemedicine and medical education in the age of COVID-19. Academic Medicine. 2022;95(12):1,838-1,843.
  9. Cornes S, Gelfand JM, Calton B. Foundational telemedicine workshop for first-year medical students developed during a pandemic. MedEdPORTAL. 2021;17:11,171. doi: 10.15766/mep_2374-8265.11171
  10. Mulcare M, Naik N, Greenwald P, et al. Advanced communication and examination skills in telemedicine: a structured simulation-based course for medical students. MedEdPORTAL. 2020;16:11,047. doi: 10.15766/mep_2374-8265.11047
  11. Frankl, SE, Joshi, A, Onorato, S, et al. Preparing future doctors for telemedicine: an asynchronous curriculum for medical students implemented during the COVID-19 pandemic. Academic Medicine. 2021;96(12):1,696-1,701. doi: 10.1097/ACM.0000000000004260
  12. Chen, BY, Hughes, MT, Kern, DE, Thomas, PA. Curriculum Development for Medical Education: A Six-Step Approach. 3rd ed. Johns Hopkins University Press. doi:10.1353/book.44600
  13. Ansary AM, Martinez JN, Scott JD. The virtual physical exam in the 21st century. J Telemed Telecare. 2021;27(6):382-392. doi: 10.1177/1357633X19878330
  14. Hamann C, Volkan K, Fishman MB, Silvestri RC, Simon SR, Fletcher SW. How well do second-year students learn physical diagnosis? Observational study of an objective structured clinical examination (OSCE). BMC Med Educ. 2002;2:1. doi: 10.1186/1472-6920-2-1

Lead Author

Emily M. Murphy, MD

Affiliations: Department of General Internal Medicine and Department of Pediatrics, Johns Hopkins University School of Medicine, Baltimore, MD


Ariella Stein, MPH - Department of General Internal Medicine, Johns Hopkins University School of Medicine, Baltimore, MD

Amit Pahwa, MD - Department of General Internal Medicine and Department of Pediatrics, Johns Hopkins University School of Medicine, Baltimore, MD

Maura McGuire, MD - Department of General Internal Medicine, Johns Hopkins University School of Medicine, Baltimore, MD

Tina Kumra, MD - Department of Pediatrics, Johns Hopkins University School of Medicine, Baltimore, MD

Corresponding Author

Emily M. Murphy, MD

Fetching other articles...

Loading the comment form...

Submitting your comment...

There are no comments for this article.

Downloads & Info


Related Content