Using the EPAs to Evaluate the Clinical Experience of Medical Students

The clinical experience of medical students varies widely across institutions, and also within an institution.¹ Traditionally, Step 2 Clinical Skills has acted as a standardized evaluation of medical student clinical competence, though it is no longer being administered.² Without a formal, standardized assessment of clinical skills, it may be worthwhile to consider quantifying authentic clinical experiences. The entrustable professional activities (EPAs) are a list of 13 activities published by the Association of American Medical Colleges (AAMC) that medical students should be able to perform upon entering residency.^3-5 Quantification of EPA performance may be important for medical schools to measure meaningful clinical participation by medical students. Our study measured EPA performance during required clerkships to better understand how frequently students perform these tasks over time and how performance differs by clinical setting.

Our study was a single institution, retrospective cross-sectional survey of the frequency with which third-year medical students performed three of the 13 EPAs defined by the AAMC (1: Gather history and perform physical examination, 5: Document clinical encounter in patient record, 6: Provide oral presentation of clinical encounter). We chose these EPAs because they can be readily tracked and are performed in all clerkships. All third-year medical students enrolled in at our institution during the 2020 academic year (n=170) were invited to participate and were informed of the three EPAs included in this study. Third-year students rotate through required clerkships in surgery, internal medicine, pediatrics, obstetrics/gynecology, neurology, family medicine, emergency medicine, and psychiatry. Most rotations were 4 weeks long, except for surgery (6 weeks), emergency medicine (2 weeks), and internal medicine (8 weeks). Approximately half of time is outpatient and half inpatient, though some rotations were entirely inpatient (emergency medicine) or entirely outpatient (family medicine). Seventy-five percent of students stay at the university campus for clerkships, though 25% of the class transitions to a community-based satellite campus. 

At the end of each site rotation, students normally receive an anonymous electronic site evaluation through MedHub as part of ongoing clerkship evaluations. For this study, students were also asked to report the number of times they performed each EPA at that specific site and number of days they rotated there on a survey at the bottom of each site evaluation. Site evaluations are not used by family medicine, so no data were gathered from this clerkship. We excluded missing, incomplete, and nonnumeric responses. We analyzed data for differences between clinical campuses, clerkships, and inpatient/outpatient sites. We also compared performance early versus late in the year. We defined students who completed a clerkship within the first/last 2 months of the academic year as the early/late cohort respectively. Cohorts were then compared within each clerkship (ie, “early” surgery vs “late” surgery). We performed statistical analysis with t tests (α<0.05). This study was approved for institutional review board exemption.

Over a 10-month period from July 2020 to May 2021, a total of 2,298 surveys were sent to third-year medical students; 55% were eligible for analysis, with 168 (13%) surveys from the community campus and 1,093 (87%) from the academic medical center campus.

On average, EPA-5 was performed least frequently and EPA-1 most frequently (Table 1). Over time, students performed greater numbers of all EPAs. Students performed EPA-5 significantly more frequently at the university campus. EPA-1 was performed significantly more frequently in the outpatient setting. EPA performance varied widely by clerkship (Table 2).

To date, there are few studies that describe student performance of the EPAs in the context of traditional clinical rotations. This study demonstrated the feasibility of collecting data from students and tracking EPA performance in each clerkship, thus providing insight into how the student experience varies within an institution. 

The data showed that students tended to perform EPAs more frequently as the clinical year progressed. This may be due to many factors, including increased student participation or entrustability by faculty. Further investigation may shed light on the progression of student EPA performance.^6,7 The reduced performance of EPA-5 in the community campus setting may result from limited electronic health record access or other factors, such as faculty workload, time constraints or site culture.⁸ Qualitative analysis may identify factors limiting note-writing in the community setting. EPA-1 was performed more frequently in the outpatient setting. This is likely due to the inherent differences in inpatient and outpatient medicine. In the inpatient setting, third-year medical students are typically assigned one to two patients per day. In the outpatient setting, third-year students often have the opportunity to see many more patients in succession throughout the day.⁹ Several factors may account for the differences in EPA performance by clerkship. Differential performance of EPA-1 and EPA-6 may be related to differences in patient volume, team size, preceptor preferences, or other factors.¹⁰ Additional analysis of these differences may highlight opportunities to increase student participation.

While the EPA concept was created to aid faculty in entrustment decisions and guide self-directed learning among students, several concerns regarding its implementation and measurement have been cited in the literature.^4,11 There is a dearth of literature regarding quantification of EPA performance as it relates to competence, and more work is needed to relate the frequency and quality of EPA performance to student competence in these areas. The results of this study have implications for developing robust observation systems that may improve clinical rotations at a given institution and maximize clinical opportunities for students. Future studies will need to be performed to assess whether institutional changes can have a positive impact on the clinical experience of medical students.

Study limitations include a single-center design, reliance on self-report, and lack of measurement of preceptor entrustability. Qualitative analysis could provide a more complete understanding of EPA performance. Student clinical experience was likely affected by the COVID-19 pandemic. At this institution, the family medicine clerkship does not use site evaluations, which prevented students from reporting EPA data during this clerkship, but can be included in future studies.

This study provides a method for tracking authentic performance of three EPAs. Adopting this method may give medical education programs greater insight into the clinical experience of medical students.