— BRIEF REPORTS —

Telemedicine Clinical Skills Needs Assessment in Early Medical Students

Tina Kumra, MD, MPH | Maura McGuire, MD | Ariella Stein, MPH | Amit K. Pahwa, MD

Fam Med. 2022;54(4):294-297.

DOI: 10.22454/FamMed.2022.904873

Abstract

Background and Objectives: As clinicians increasingly rely on telemedicine, medical students will need to learn how to appropriately use telemedicine in patient care. A formal approach to curriculum development is needed to identify gaps and needs in early medical student performance.

Methods: In October 2020, 120 second-year medical students completed a telemedicine visit with a standardized patient with chronic essential hypertension. Students were assessed across five domains (history-gathering, communication, vitals, physical exam, and assessment/management). An anonymous, voluntary survey was distributed to assess self-efficacy in telemedicine skills.

Results: Students perform well in history-gathering and communication (98% of student scored 4 or 5 out of 5 on history, 100% of students received a 7 or 8 out of 8 on communication). Students perform poorly in obtaining vital signs (23% scored 3 or 4 out of 4) and assessment/management (14% scored 3 or 4 out of 4). Students received their lowest scores in physical examination (2% score 4 or 5 out of 5). The number of telemedicine visits completed with patients prior to the standardized patient exercise had no impact on student performance during the exercise. Student response rate on the postexercise survey was 88%. Self-efficacy was lowest in physical examination telemedicine skills compared to other domains.

Conclusions: Findings suggest that early medical students are able to gather history and communicate over telemedicine, but perform poorly on telemedicine physical examination skills. More robust curriculum development addressing telemedicine physical examinations skills is needed early in medical training.


International organizations overseeing medical education acknowledge the need for competence in telemedicine, however most trainees are inadequately prepared to care for patients using telemedicine.1-3 Telemedicine is defined as the use of two-way, real-time interactive audio or audiovisual communication between a patient and a practitioner at a distant site, for example by video visits.4 While 57% of medical schools have telemedicine curricula, only one-third of schools have curricula in the preclerkship phase.5,6 In a review of 17 schools, 12 had some didactic learning coupled with either patient encounters or standardized patients (SPs) during the preclinical years.7 However, the conceptual frameworks and learning theories underlying these curricula are currently unknown. A systematic curriculum development approach is needed to effectively teach telemedicine clinical skills to early medical students.

First-year medical students learn foundational skills in history-gathering, communication, physical examination (PE), and clinical reasoning.8 Many students can implement these skills upon completion of their second year.9 However, the degree to which students demonstrate these skills during video visits is unknown. Student telemedicine self-efficacy, defined as one’s perceived ability for learning or achieving actions at designated levels, has not yet been reported.10 We utilized Kerns’ six-step approach to develop a telemedicine clinical skills curriculum for medical students.11 The six steps include problem identification and needs assessment, targeted needs assessment, goals and objectives, educational strategies, implementation, and feedback and evaluation.11 Specifically, we aimed to assess the baseline performance of second-year medical students during a simulated telemedicine case and their self-reported efficacy in a telemedicine visit in order to define the problem and needs of students.

Methods

Preclinical students at Johns Hopkins University School of Medicine complete a required ambulatory clerkship from January of first year to December of second year. Second-year students completed a video visit exercise on an SP with hypertension near the end of the clerkship. Prior to the exercise, students read a handout on the ambulatory management of hypertension based on the Joint National Committee practice guideline and completed a hypertension case in the electronic medical record.12 In the first year, students completed an in-person SP exercise on obtaining a blood pressure. Course directors used the practice guidelines to develop the case and checklist (Table 1). The PE checklist was adapted from materials created by the Johns Hopkins Telemedicine Education Consortium and was reviewed by three nonaffiliated ambulatory physicians.

Four groups of 30 students were randomly assigned to complete the session during the same week. The exercise was modeled on the United States Medical Licensing Examination Step 2 Clinical Skills Exam, a national standardized assessment of clinical skills in which medical students complete an SP encounter in fifteen minutes. The SP completed a checklist directly following the encounter. Students conducted these video visits either at home or on campus while the SP connected from home. Fifteen SPs completed a 4-hour training led by an SP education specialist, including review of the case and checklist. The students were provided with two prompts during the encounter: a prompt to conduct the vital signs at 10 minutes and a prompt to provide the diagnosis and plan at 13 minutes.

Students completed an anonymous, voluntary post-exercise questionnaire. We utilized the three recommended principals in constructing questions measuring student self-efficacy: the measures were future-oriented, focused on beliefs about the student’s ability to carry out the action necessary for success, and focused on specific domains.13 For example, to assess self-efficacy in history gathering, students were asked to rate their confidence level in their ability to gather a history in a telemedicine visit using a 5-point Likert scale. We used SAS Software, version 9.4 (SAS Inc, Cary, NC) to conduct our analyses. The Johns Hopkins Medicine Institutional Review Board approved this study (IRB00270053).

Results

During the 2020 calendar year, 120 students were enrolled in the clerkship. The number of telemedicine sessions completed prior to the SP exercise made no difference on student performance in any of the five domains. Students received the highest scores in the history gathering and interpersonal skills sections (Figure 1). Students received their lowest scores in the PE domain (2% scored either a 4 or 5 out of 5). Thirty percent of students did not complete any PE element during the encounter. Students obtained the lowest score in the neurologic portion of the PE and highest on the cardiovascular and respiratory portions; 88% of students responded to the anonymous postexercise evaluation (Figure 2). Students reported the lowest self-efficacy in PE skills.

Discussion

Second-year students performed well on history-gathering and communication, however they performed poorly on gathering objective data over video visits, including vital signs and in PE maneuvers. Students reported the lowest self-efficacy in telemedicine PE skills compared to other domains. Notably, student self-efficacy often overestimates their abilities.14,15 The number of telemedicine sessions completed with patients prior to the exercise did not impact performance on the PE domain.

Second-year students tend to perform equivalently on history-taking and PE domains during in-person SP encounters.9 However, students performed more poorly than expected in the PE domain in a video visit encounter. Second-year students in the previous year in the same clerkship scored 81.5% on the PE domain in a similar in-person case with a SP while their performance in history and interpersonal domains were comparable to the video visit performance found in our study.

Limitations of this study include an assessment of students during a single year at a single institution. The exercise was simulated and estimates, but does not equate to, actual clinical performance. Lastly, we delivered the exercise across 2 different days, which may have led to some communication between groups regarding the SP case, leading to higher scores on day two. Study strengths include a large class size and robust early assessment of telemedicine clinical skills performance paired with high response rate of self-reported student efficacy in telemedicine skills.

Medical student performance and self-efficacy in video visit encounters has not previously been reported. Our findings suggest telemedicine curricula with goals and objectives focusing on PE skills are needed starting in early medical school. Lesser emphasis on telemedicine history-gathering and communication skills may be needed, given that they seem to be more easily translated from in-person to telemedicine encounters.

References

  1. Sartori DJ, Olsen S, Weinshel E, Zabar SR. Preparing trainees for telemedicine: a virtual OSCE pilot. Med Educ. 2019;53(5):517-518. doi:10.1111/medu.13851
  2. World Health Organization Global Observatory for eHealth. Accessed January 8, 2020. https://www.who.int/observatories/global-observatory-for-ehealth
  3. Telehealth Competencies Across the Learning Continuum. In: AAMC New and Emerging Areas in Medicine Series. Washington, DC: Association of American Medical Colleges; 2021.
  4. Telemedicine and Telehealth. HealthIT.gov. Accessed June 15, 2021. https://www.healthit.gov/topic/health-it-health-care-settings/telemedicine-and-telehealth
  5. Content Documentation in Required Courses and Elective Courses. Curriculum Reports Web site. Association of American Medical Colleges. Accessed November 23, 2021. https://www.aamc.org/data-reports/curriculum-reports/interactive-data/content-documentation-required-courses-and-elective-courses
  6. Khullar D, Mullangi S, Yu J, et al. The state of telehealth education at US medical schools. Healthc (Amst). 2021;9(2):100522. doi:10.1016/j.hjdsi.2021.100522
  7. Waseh S, Dicker AP. Telemedicine training in undergraduate medical education: mixed-methods review. JMIR Med Educ. 2019;5(1):e12515. doi:10.2196/12515
  8. Uchida T, Park YS, Ovitsh RK, et al. Approaches to teaching the physical exam to preclerkship medical students: results of a national survey. Acad Med. 2019;94(1):129-134. doi:10.1097/ACM.0000000000002433
  9. Hamann C, Volkan K, Fishman MB, Silvestri RC, Simon SR, Fletcher SW. How well do second-year students learn physical diagnosis? Observational study of an Objective Structured Clinical Examination (OSCE). BMC Med Educ. 2002;2(1):1. doi:10.1186/1472-6920-2-1
  10. Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev. 1977;84(2):191-215. doi:10.1037/0033-295X.84.2.191
  11. Thomas PA, Kern DE, Hughes MT, Chen BY. Curriculum development for medical education: a six-step approach. JHU Press; 2016.
  12. James PA, Oparil S, Carter BL, et al. 2014 evidence-based guideline for the management of high blood pressure in adults: report from the panel members appointed to the Eighth Joint National Committee (JNC 8). JAMA. 2014;311(5):507-520. doi:10.1001/jama.2013.284427
  13. Klassen RM, Klassen JRL. Self-efficacy beliefs of medical students: a critical review. Perspect Med Educ. 2018;7(2):76-82. doi:10.1007/s40037-018-0411-3
  14. Pajares F. Self-efficacy beliefs in academic settings. Rev Educ Res. 1996;66(4):543-578. doi:10.3102/00346543066004543
  15. Artino AR Jr. Academic self-efficacy: from educational theory to instructional practice. Perspect Med Educ. 2012;1(2):76-85. doi:10.1007/s40037-012-0012-5

Lead Author

Tina Kumra, MD, MPH

Affiliations: Department of Pediatrics, Johns Hopkins University School of Medicine, Baltimore, MD

Co-Authors

Maura McGuire, MD - Division of General Internal Medicine, Johns Hopkins University School of Medicine, Baltimore, MD

Ariella Stein, MPH - Division of General Internal Medicine, Johns Hopkins University School of Medicine, Baltimore, MD

Amit K. Pahwa, MD - Department of Pediatrics, Johns Hopkins University School of Medicine, Baltimore, MD

Fetching other articles...

Mendeley

Loading metrics from Mendeley...

Loading the comment form...

Submitting your comment...

There are no comments for this article.