Telemedicine Clinical Skills Needs Assessment in Early Medical Students

International organizations overseeing medical education acknowledge the need for competence in telemedicine, however most trainees are inadequately prepared to care for patients using telemedicine.1-3 Telemedicine is defined as the use of two-way, real-time interactive audio or audiovisual communication between a patient and a practitioner at a distant site, for example by video visits.4 While 57% of medical schools have telemedicine curricula, only one-third of schools have curricula in the preclerkship phase.5,6 In a review of 17 schools, 12 had some didactic learning coupled with either patient encounters or standardized patients (SPs) during the preclinical years.7 However, the conceptual frameworks and learning theories underlying these curricula are currently unknown. A systematic curriculum development approach is needed to effectively teach telemedicine clinical skills to early medical students.

First-year medical students learn foundational skills in history-gathering, communication, physical examination (PE), and clinical reasoning.8 Many students can implement these skills upon completion of their second year.9 However, the degree to which students demonstrate these skills during video visits is unknown. Student telemedicine self-efficacy, defined as one’s perceived ability for learning or achieving actions at designated levels, has not yet been reported.10 We utilized Kerns’ six-step approach to develop a telemedicine clinical skills curriculum for medical students.11 The six steps include problem identification and needs assessment, targeted needs assessment, goals and objectives, educational strategies, implementation, and feedback and evaluation.11 Specifically, we aimed to assess the baseline performance of second-year medical students during a simulated telemedicine case and their self-reported efficacy in a telemedicine visit in order to define the problem and needs of students.

Preclinical students at Johns Hopkins University School of Medicine complete a required ambulatory clerkship from January of first year to December of second year. Second-year students completed a video visit exercise on an SP with hypertension near the end of the clerkship. Prior to the exercise, students read a handout on the ambulatory management of hypertension based on the Joint National Committee practice guideline and completed a hypertension case in the electronic medical record.12 In the first year, students completed an in-person SP exercise on obtaining a blood pressure. Course directors used the practice guidelines to develop the case and checklist (Table 1). The PE checklist was adapted from materials created by the Johns Hopkins Telemedicine Education Consortium and was reviewed by three nonaffiliated ambulatory physicians.

Four groups of 30 students were randomly assigned to complete the session during the same week. The exercise was modeled on the United States Medical Licensing Examination Step 2 Clinical Skills Exam, a national standardized assessment of clinical skills in which medical students complete an SP encounter in fifteen minutes. The SP completed a checklist directly following the encounter. Students conducted these video visits either at home or on campus while the SP connected from home. Fifteen SPs completed a 4-hour training led by an SP education specialist, including review of the case and checklist. The students were provided with two prompts during the encounter: a prompt to conduct the vital signs at 10 minutes and a prompt to provide the diagnosis and plan at 13 minutes.

Students completed an anonymous, voluntary post-exercise questionnaire. We utilized the three recommended principals in constructing questions measuring student self-efficacy: the measures were future-oriented, focused on beliefs about the student’s ability to carry out the action necessary for success, and focused on specific domains.13 For example, to assess self-efficacy in history gathering, students were asked to rate their confidence level in their ability to gather a history in a telemedicine visit using a 5-point Likert scale. We used SAS Software, version 9.4 (SAS Inc, Cary, NC) to conduct our analyses. The Johns Hopkins Medicine Institutional Review Board approved this study (IRB00270053).

During the 2020 calendar year, 120 students were enrolled in the clerkship. The number of telemedicine sessions completed prior to the SP exercise made no difference on student performance in any of the five domains. Students received the highest scores in the history gathering and interpersonal skills sections (Figure 1). Students received their lowest scores in the PE domain (2% scored either a 4 or 5 out of 5). Thirty percent of students did not complete any PE element during the encounter. Students obtained the lowest score in the neurologic portion of the PE and highest on the cardiovascular and respiratory portions; 88% of students responded to the anonymous postexercise evaluation (Figure 2). Students reported the lowest self-efficacy in PE skills.

Second-year students performed well on history-gathering and communication, however they performed poorly on gathering objective data over video visits, including vital signs and in PE maneuvers. Students reported the lowest self-efficacy in telemedicine PE skills compared to other domains. Notably, student self-efficacy often overestimates their abilities.14,15 The number of telemedicine sessions completed with patients prior to the exercise did not impact performance on the PE domain.

Second-year students tend to perform equivalently on history-taking and PE domains during in-person SP encounters.9 However, students performed more poorly than expected in the PE domain in a video visit encounter. Second-year students in the previous year in the same clerkship scored 81.5% on the PE domain in a similar in-person case with a SP while their performance in history and interpersonal domains were comparable to the video visit performance found in our study.

Limitations of this study include an assessment of students during a single year at a single institution. The exercise was simulated and estimates, but does not equate to, actual clinical performance. Lastly, we delivered the exercise across 2 different days, which may have led to some communication between groups regarding the SP case, leading to higher scores on day two. Study strengths include a large class size and robust early assessment of telemedicine clinical skills performance paired with high response rate of self-reported student efficacy in telemedicine skills.

Medical student performance and self-efficacy in video visit encounters has not previously been reported. Our findings suggest telemedicine curricula with goals and objectives focusing on PE skills are needed starting in early medical school. Lesser emphasis on telemedicine history-gathering and communication skills may be needed, given that they seem to be more easily translated from in-person to telemedicine encounters.