— LEARNER RESEARCH —

Using the EPAs to Evaluate the Clinical Experience of Medical Students

Cameron Bosinski, MD, MS | Marissa Rice, MD | Matthew Mason, MD | Lauren J. Germain, PhD, MEd

PRiMER. 2022;6:13.

Published: 5/20/2022 | DOI: 10.22454/PRiMER.2022.873914

Abstract

Introduction: Although there are standardized assessments of medical students throughout medical school, the clinical experience of each student may vary widely. Currently, medical schools do not have a systematic method to ensure that students perform clinical skills during clinical clerkships. Our study used the Entrustable Professional Activities for Entering Residency (EPAs) framework to assess the ways in which medical students meaningfully participate in patient care at various clinical sites in each required clerkship and over time.

Methods: Over the 2020-2021 academic year, 170 third-year medical students were asked to report the number of times they performed EPA-1 (gather history and perform physical examination), EPA-5 (document clinical encounter), and EPA-6 (provide oral presentation of clinical encounter) at the end of each required clinical rotation (emergency medicine, internal medicine, surgery, obstetrics/gynecology, pediatrics, neurology, psychiatry) at a single medical school. We used descriptive statistics and t tests to compare frequency of these EPAs by campus type, site type, clerkship, and time.

Results: One thousand, two hundred sixty-one surveys met inclusion criteria. Students performed EPA-5 more often at an academic medical center, and EPA-1 more frequently in the outpatient setting. Students performed EPA-1 and EPA-6 most often during emergency medicine and EPA-5 most often during internal medicine. Performance of all three EPAs increased over time.

Conclusion: This reporting system produced a robust data set that allowed for EPA performance comparisons by campus, site type, clerkship, and time. EPA performance varied by rotation, site type, clerkship, and time.

Introduction

The clinical experience of medical students varies widely across institutions, and also within an institution.1 Traditionally, Step 2 Clinical Skills has acted as a standardized evaluation of medical student clinical competence, though it is no longer being administered.2 Without a formal, standardized assessment of clinical skills, it may be worthwhile to consider quantifying authentic clinical experiences. The entrustable professional activities (EPAs) are a list of 13 activities published by the Association of American Medical Colleges (AAMC) that medical students should be able to perform upon entering residency.3-5 Quantification of EPA performance may be important for medical schools to measure meaningful clinical participation by medical students. Our study measured EPA performance during required clerkships to better understand how frequently students perform these tasks over time and how performance differs by clinical setting.

Methods

Our study was a single institution, retrospective cross-sectional survey of the frequency with which third-year medical students performed three of the 13 EPAs defined by the AAMC (1: Gather history and perform physical examination, 5: Document clinical encounter in patient record, 6: Provide oral presentation of clinical encounter). We chose these EPAs because they can be readily tracked and are performed in all clerkships. All third-year medical students enrolled in at our institution during the 2020 academic year (n=170) were invited to participate and were informed of the three EPAs included in this study. Third-year students rotate through required clerkships in surgery, internal medicine, pediatrics, obstetrics/gynecology, neurology, family medicine, emergency medicine, and psychiatry. Most rotations were 4 weeks long, except for surgery (6 weeks), emergency medicine (2 weeks), and internal medicine (8 weeks). Approximately half of time is outpatient and half inpatient, though some rotations were entirely inpatient (emergency medicine) or entirely outpatient (family medicine). Seventy-five percent of students stay at the university campus for clerkships, though 25% of the class transitions to a community-based satellite campus. 

At the end of each site rotation, students normally receive an anonymous electronic site evaluation through MedHub as part of ongoing clerkship evaluations. For this study, students were also asked to report the number of times they performed each EPA at that specific site and number of days they rotated there on a survey at the bottom of each site evaluation. Site evaluations are not used by family medicine, so no data were gathered from this clerkship. We excluded missing, incomplete, and nonnumeric responses. We analyzed data for differences between clinical campuses, clerkships, and inpatient/outpatient sites. We also compared performance early versus late in the year. We defined students who completed a clerkship within the first/last 2 months of the academic year as the early/late cohort respectively. Cohorts were then compared within each clerkship (ie, “early” surgery vs “late” surgery). We performed statistical analysis with t tests (α<0.05). This study was approved for institutional review board exemption.

Results

Over a 10-month period from July 2020 to May 2021, a total of 2,298 surveys were sent to third-year medical students; 55% were eligible for analysis, with 168 (13%) surveys from the community campus and 1,093 (87%) from the academic medical center campus.

On average, EPA-5 was performed least frequently and EPA-1 most frequently (Table 1). Over time, students performed greater numbers of all EPAs. Students performed EPA-5 significantly more frequently at the university campus. EPA-1 was performed significantly more frequently in the outpatient setting. EPA performance varied widely by clerkship (Table 2).

Discussion

To date, there are few studies that describe student performance of the EPAs in the context of traditional clinical rotations. This study demonstrated the feasibility of collecting data from students and tracking EPA performance in each clerkship, thus providing insight into how the student experience varies within an institution. 

 

The data showed that students tended to perform EPAs more frequently as the clinical year progressed. This may be due to many factors, including increased student participation or entrustability by faculty. Further investigation may shed light on the progression of student EPA performance.6,7 The reduced performance of EPA-5 in the community campus setting may result from limited electronic health record access or other factors, such as faculty workload, time constraints or site culture.8 Qualitative analysis may identify factors limiting note-writing in the community setting. EPA-1 was performed more frequently in the outpatient setting. This is likely due to the inherent differences in inpatient and outpatient medicine. In the inpatient setting, third-year medical students are typically assigned one to two patients per day. In the outpatient setting, third-year students often have the opportunity to see many more patients in succession throughout the day.9 Several factors may account for the differences in EPA performance by clerkship. Differential performance of EPA-1 and EPA-6 may be related to differences in patient volume, team size, preceptor preferences, or other factors.10 Additional analysis of these differences may highlight opportunities to increase student participation.

While the EPA concept was created to aid faculty in entrustment decisions and guide self-directed learning among students, several concerns regarding its implementation and measurement have been cited in the literature.4,11 There is a dearth of literature regarding quantification of EPA performance as it relates to competence, and more work is needed to relate the frequency and quality of EPA performance to student competence in these areas. The results of this study have implications for developing robust observation systems that may improve clinical rotations at a given institution and maximize clinical opportunities for students. Future studies will need to be performed to assess whether institutional changes can have a positive impact on the clinical experience of medical students.

Study limitations include a single-center design, reliance on self-report, and lack of measurement of preceptor entrustability. Qualitative analysis could provide a more complete understanding of EPA performance. Student clinical experience was likely affected by the COVID-19 pandemic. At this institution, the family medicine clerkship does not use site evaluations, which prevented students from reporting EPA data during this clerkship, but can be included in future studies.

This study provides a method for tracking authentic performance of three EPAs. Adopting this method may give medical education programs greater insight into the clinical experience of medical students.

References

  1. Rattner SL, Louis DZ, Rabinowitz C, et al. Documenting and comparing medical students’ clinical experiences. JAMA. 2001;286(9):1035-1040. doi:10.1001/jama.286.9.1035
  2. Murphy B. USMLE Step 2 CS canceled: what it means for medical students. Accessed May 17, 2022. https://www.ama-assn.org/residents-students/usmle/usmle-step-2-cs-canceled-what-it-means-medical-students
  3. Lyss-Lerman P, Teherani A, Aagaard E, Loeser H, Cooke M, Harper GM. What training is needed in the fourth year of medical school? Views of residency program directors. Acad Med. 2009;84(7):823-829. doi:10.1097/ACM.0b013e3181a82426
  4. Lomis K, Amiel JM, Ryan MS, et al; AAMC Core EPAs for entering residency pilot team. Implementing an entrustable professional activities framework in undergraduate medical education: early lessons from the AAMC Core Entrustable Professional Activities for Entering Residency Pilot. Acad Med. 2017;92(6):765-770. doi:10.1097/ACM.0000000000001543
  5. Langdale LA, Schaad D, Wipf J, Marshall S, Vontver L, Scott CS. Preparing graduates for the first year of residency: are medical schools meeting the need? Acad Med. 2003;78(1):39-44. doi:10.1097/00001888-200301000-00009
  6. Amiel JM, Andriole DA, Biskobing DM, et al; Association of American Medical Colleges Core EPAs for Entering Residency Pilot Team. Revisiting the Core Entrustable Professional Activities for Entering Residency. Acad Med. 2021;96(7S):S14-S21. doi:10.1097/ACM.0000000000004088
  7. Forte M, Morson N, Mirchandani N, Grundland B, Fernando O, Rubenstein W. How teachers adapt their cognitive strategies when using entrustment scales. Acad Med. 2021;96(11S):S87-S92. doi:10.1097/ACM.0000000000004287
  8. Cuddy MM, Foster LM, Wallach PM, Hammoud MM, Swanson DB. Medical student experiences with electronic health records nationally: a longitudinal analysis including school-level effects. Acad Med. 2021. doi:10.1097/ACM.0000000000004290
  9. Jacobson EW, Keough WL, Dalton BE, Giansiracusa DF. A comparison of inpatient and outpatient experiences during an internal medicine clerkship. Am J Med. 1998;104(2):159-162. doi:10.1016/S0002-9343(97)00343-4
  10. Levy BT, Merchant ML. Factors associated with higher clinical skills experience of medical students on a family medicine preceptorship. Fam Med. 2005;37(5):332-340.
  11. O’Connor P, Lydon S, O’Dowd E, Byrne D. Entrustable professional activities for early health professionals: what’s next? Med Educ. 2021;55(9):984-986. doi:10.1111/medu.14594

Lead Author

Cameron Bosinski, MD, MS

Affiliations: Department of Anesthesiology, Perioperative and Pain Medicine, Brigham and Women’s Hospital, Boston, MA

Co-Authors

Marissa Rice, MD - College of Medicine, SUNY Upstate Medical University, Syracuse, NY

Matthew Mason, MD - Department of Urology, SUNY Upstate Medical University, Syracuse, NY

Lauren J. Germain, PhD, MEd - Department of Academic Affairs, SUNY Upstate Medical University, Syracuse, NY

Corresponding Author

Cameron Bosinski, MD, MS

Correspondence: Department of Anesthesiology, Perioperative and Pain Medicine, Brigham and Women’s Hospital, 75 Francis St, Boston, MA 02115-6110. 617-732-8210

Email: cbosinski@bwh.harvard.edu

Fetching other articles...

Mendeley

Loading metrics from Mendeley...

Loading the comment form...

Submitting your comment...

There are no comments for this article.

Downloads & Info

Share