Identifying Community-Based Entrustable Professional Activities for Medical Students Through a Modified Delphi Process

An entrustable professional activity (EPA) is defined as “a unit of professional practice that can be fully entrusted to a trainee, once he or she has demonstrated the necessary competence to execute this activity unsupervised.” ¹ EPAs have been recognized as a valuable concept in both undergraduate and graduate medical education, given that they help ground competencies and milestones. ^{2, 3}

A key aspect of EPAs is the requirement of a workplace curriculum. ¹ Typically, EPAs have focused on patient care delivery and the clinical environment as the workplace. However, recognition of the need for physicians also to possess skills in improving population health is growing. ⁴ This recognition is sparking calls for medical education efforts to be more socially accountable and community-engaged ^{5, 6} as well as calls for inclusion of accreditation standards that address community and population health among requirements. ^{7, 8}

The MD curriculum at the University of Wisconsin School of Medicine and Public Health (UWSMPH) emphasizes community engagement principles, such as having a strong understanding of engagement goals and the community itself, fostering long-term relationship-building, and recognizing community self-determination. ⁹ With a statewide campus, UWSMPH students can help address statewide issues contributing to noted downward trends related to health outcomes and determinants. ¹⁰ The goal of our project was to identify new community-based tasks that physician graduates of UWSMPH should be entrusted to do without direct supervision.

The effort to identify community-based EPAs was initiated during a 1-hour workshop in May 2022. Session participants included four faculty physicians, three staff members, and two session facilitators. Session objectives included sharing curricular goals for community-based learning across the UWSMPH medical curriculum and identifying discrete community-based tasks important for all UWSMPH physician graduates.

We then employed a three-stage online modified-Delphi process, a known strategy for elaborating EPAs, ¹ to identify community-based EPAs for UWSMPH medical students. We selected 34 key stakeholders through a snowball referral process and asked them to participate in surveys to further identify and reach consensus on community-based EPAs. Individuals were chosen based on their relationship with UWSMPH and their potential to represent one of four stakeholder groups: (a) community partners, key points of contact at community organizations that support UWSMPH community-engaged curriculum; (b) medical education curricular leaders; (c) faculty/staff working with community-engaged medical education programs; and (d) community-engaged physician faculty.

We employed three stages of online surveys using the Qualtrics XM (Qualtrics, LLC) survey application. During the first survey stage, participants were given information defining EPAs and contrasting them with competencies. Participants then were asked to list discrete community-based tasks that they felt UWSMPH graduates should be expected to perform. We used identified tasks and comments from the first stage of surveys, along with tasks identified during the workshop, to inform the second stage of surveys.

During the second stage, participants were asked to rate their level of agreement that each identified task was part of a physician’s job; rating was on a five-point anchored Likert scale (1=strongly disagree, 2=disagree, 3=neither agree nor disagree, 4=agree, 5=strongly agree). For the third stage, average scores for tasks and all available comments about each item were shared with participants; subsequently, participants were asked to evaluate each task again using the same Likert scale. During the first and third survey stages, participants were asked to identify which of the four stakeholder categories they most strongly identified with.

In stages two and three, we calculated the mean and standard deviation for each task rating. The a priori definition of consensus was predetermined as a mean rating of ≥4 and 80% of participants stating ≥4 agreement (agree or strongly agree) with the final list of stage three survey items; we used consensus determining methodology consistent with that previously described. ¹¹ We examined the results from all respondents, including the subset that identified as community member/other. Subsequently, we grouped all the tasks identified by respondents by common themes.

We completed the University of Wisconsin–Madison Self-Certification Tool, characterizing the project as quality improvement and/or program evaluation, which does not require review by the institutional review board.

Survey respondents were made up of 18 (53%) for stage one, 18 (53%) for stage two, and 14 (41%) for stage three. The participant information collected in the first and third survey stages indicated that all four stakeholder groups were represented in these stages (Table 1).

Combining data collected from the workshop and the first survey stage, participants identified 38 separate tasks as potential EPAs. After stage two, 26 tasks met the consensus threshold. After stage three, 22 tasks reached final consensus with a mean rating of ≥4 and 80% of participants rating them ≥4. We categorized the 22 tasks that met consensus into general themes, which included (a) community needs, assets, and priorities; (b) community connections; and (c) education and advocacy. The task with the highest mean rating was “addressing trust issues with the medical community amongst the local population” (mean=4.71, SD=0.45; 100% agree or strongly agree; Table 2).

When examining the subset (n=2) of respondents that identified as community member/other, 14 tasks met consensus, 12 of which met consensus by the larger group. The two tasks that met consensus by the community member/other subset but not the larger group, included “developing a project that helps a local organization” and “developing an elevator pitch for a community health initiative.”

Our project identified 22 tasks as potential UWSMPH community-based EPAs. Our work builds on existing efforts to define EPAs that move beyond individual patient care and outside of health care settings, including those focused on systems thinking and improvement, and global health. ^{3, 12} The identified tasks appear to have applicability across physician training programs and are not community specific. Currently the 13 core EPAs for entering residency, as described by the Association of American Medical Colleges, include 12 EPAs focused on clinically oriented activities and one focused on identifying system failures and contributing to a culture of safety and improvement. ³ In contrast, the core outcomes of family medicine training are made up of activities impacting communities, with outcomes such as “model professionalism and be trustworthy for patients, peers and communities” and “effectively lead, manage, and participate in teams that provide care and improve outcomes for the diverse populations and communities they serve.” ¹³ Thus, the findings reported here can help bridge this gap by offering additional community-based EPAs to support the transition from medical school to residency.

Of note, tasks do not themselves ensure adherence to community-engagement principles. ⁹ Therefore, efforts must be made to elevate community voices when operationalizing these tasks, potentially through consultation with community members during the curriculum development process. Other limitations include that surveyed individuals were recruited based on their relationship with UWSMPH, and many of the final participants identified themselves as faculty/staff involved with community-engaged medical education programs, which might have resulted in participants who shared similar ideas. Community members were identified through roles with organizations connected with UWSMPH, which might not represent a broader patient population. Few respondents identified as community members, and how findings would differ had this group been more robustly represented is unclear.

Overall, findings from this project have provided guidance for the UWSMPH community-based educators to novelly augment existing and developing curriculum and to consider the progression of skills through graduate medical education. Similar efforts could be adapted by other medical education programs as they work to refine community-based curriculum.

Parts of this study' findings were presented at the following meetings:

• American Medical Association, Health Systems Science Summit. Chicago, IL. December 2022.

• Association of American Medical Colleges, Central Region Group on Educational Affairs Spring Meeting, Indianapolis, IN. April 2023.