BRIEF REPORTS

Identifying Community-Based Entrustable Professional Activities for Medical Students Through a Modified Delphi Process

Parvathy Pillai, MD, MPH | Amina Maamouri | Jonathan Temte, MD, PhD

Fam Med. 2024;56(8):501-504.

DOI: 10.22454/FamMed.2024.818912

Return to Issue

Abstract

Background and Objectives: Recognition of the need for medical education to train physicians who are skilled at supporting population health and work beyond traditional health care settings is growing. Entrustable professional activities (EPAs) for medical students typically have centered around activities taking place in the clinical workplace; however, EPAs that involve working with community members in community contexts have not been clearly established.

Methods: We used a three-stage online modified-Delphi method to identify community-based EPAs for University of Wisconsin School of Medicine and Public Health medical students. We recruited key stakeholders to participate and asked them, based on their experience, to generate a list of community-based tasks that they believed graduates should be trusted to perform. Subsequently, using a five-point anchored Likert scale (1=strongly disagree to 5=strongly agree), we asked participants to rate their level of agreement with each identified task becoming an EPA. An a priori definition of consensus was established.

Results: Twenty-two tasks reached consensus as potential community-based EPAs. The tasks with the highest mean ratings were “addressing trust issues with the medical community amongst the local population” (mean=4.71), “meeting with community members around a health topic” (mean=4.64), “identifying opportunities for disease prevention” (mean=4.64), and “identifying policies that impact community outcomes” (mean=4.57).

Conclusions: The identified community-based tasks can support the augmentation of existing community-based curriculum and help identify areas for novel curriculum and assessment development. Lessons learned from this local effort could be helpful to other programs seeking to establish and refine community-based curricula.

INTRODUCTION

An entrustable professional activity (EPA) is defined as “a unit of professional practice that can be fully entrusted to a trainee, once he or she has demonstrated the necessary competence to execute this activity unsupervised.” 1 EPAs have been recognized as a valuable concept in both undergraduate and graduate medical education, given that they help ground competencies and milestones. 2, 3

A key aspect of EPAs is the requirement of a workplace curriculum. 1 Typically, EPAs have focused on patient care delivery and the clinical environment as the workplace. However, recognition of the need for physicians also to possess skills in improving population health is growing. 4 This recognition is sparking calls for medical education efforts to be more socially accountable and community-engaged 5, 6 as well as calls for inclusion of accreditation standards that address community and population health among requirements. 7, 8

The MD curriculum at the University of Wisconsin School of Medicine and Public Health (UWSMPH) emphasizes community engagement principles, such as having a strong understanding of engagement goals and the community itself, fostering long-term relationship-building, and recognizing community self-determination. 9 With a statewide campus, UWSMPH students can help address statewide issues contributing to noted downward trends related to health outcomes and determinants. 10 The goal of our project was to identify new community-based tasks that physician graduates of UWSMPH should be entrusted to do without direct supervision.

METHODS

The effort to identify community-based EPAs was initiated during a 1-hour workshop in May 2022. Session participants included four faculty physicians, three staff members, and two session facilitators. Session objectives included sharing curricular goals for community-based learning across the UWSMPH medical curriculum and identifying discrete community-based tasks important for all UWSMPH physician graduates.

We then employed a three-stage online modified-Delphi process, a known strategy for elaborating EPAs, 1 to identify community-based EPAs for UWSMPH medical students. We selected 34 key stakeholders through a snowball referral process and asked them to participate in surveys to further identify and reach consensus on community-based EPAs. Individuals were chosen based on their relationship with UWSMPH and their potential to represent one of four stakeholder groups: (a) community partners, key points of contact at community organizations that support UWSMPH community-engaged curriculum; (b) medical education curricular leaders; (c) faculty/staff working with community-engaged medical education programs; and (d) community-engaged physician faculty.

We employed three stages of online surveys using the Qualtrics XM (Qualtrics, LLC) survey application. During the first survey stage, participants were given information defining EPAs and contrasting them with competencies. Participants then were asked to list discrete community-based tasks that they felt UWSMPH graduates should be expected to perform. We used identified tasks and comments from the first stage of surveys, along with tasks identified during the workshop, to inform the second stage of surveys.

During the second stage, participants were asked to rate their level of agreement that each identified task was part of a physician’s job; rating was on a five-point anchored Likert scale (1=strongly disagree, 2=disagree, 3=neither agree nor disagree, 4=agree, 5=strongly agree). For the third stage, average scores for tasks and all available comments about each item were shared with participants; subsequently, participants were asked to evaluate each task again using the same Likert scale. During the first and third survey stages, participants were asked to identify which of the four stakeholder categories they most strongly identified with.

In stages two and three, we calculated the mean and standard deviation for each task rating. The a priori definition of consensus was predetermined as a mean rating of ≥4 and 80% of participants stating ≥4 agreement (agree or strongly agree) with the final list of stage three survey items; we used consensus determining methodology consistent with that previously described. 11 We examined the results from all respondents, including the subset that identified as community member/other. Subsequently, we grouped all the tasks identified by respondents by common themes.

We completed the University of Wisconsin–Madison Self-Certification Tool, characterizing the project as quality improvement and/or program evaluation, which does not require review by the institutional review board.

RESULTS

Survey respondents were made up of 18 (53%) for stage one, 18 (53%) for stage two, and 14 (41%) for stage three. The participant information collected in the first and third survey stages indicated that all four stakeholder groups were represented in these stages (Table 1).

Combining data collected from the workshop and the first survey stage, participants identified 38 separate tasks as potential EPAs. After stage two, 26 tasks met the consensus threshold. After stage three, 22 tasks reached final consensus with a mean rating of ≥4 and 80% of participants rating them ≥4. We categorized the 22 tasks that met consensus into general themes, which included (a) community needs, assets, and priorities; (b) community connections; and (c) education and advocacy. The task with the highest mean rating was “addressing trust issues with the medical community amongst the local population” (mean=4.71, SD=0.45; 100% agree or strongly agree; Table 2).

When examining the subset (n=2) of respondents that identified as community member/other, 14 tasks met consensus, 12 of which met consensus by the larger group. The two tasks that met consensus by the community member/other subset but not the larger group, included “developing a project that helps a local organization” and “developing an elevator pitch for a community health initiative.”

CONCLUSIONS

Our project identified 22 tasks as potential UWSMPH community-based EPAs. Our work builds on existing efforts to define EPAs that move beyond individual patient care and outside of health care settings, including those focused on systems thinking and improvement, and global health. 3, 12 The identified tasks appear to have applicability across physician training programs and are not community specific. Currently the 13 core EPAs for entering residency, as described by the Association of American Medical Colleges, include 12 EPAs focused on clinically oriented activities and one focused on identifying system failures and contributing to a culture of safety and improvement. 3 In contrast, the core outcomes of family medicine training are made up of activities impacting communities, with outcomes such as “model professionalism and be trustworthy for patients, peers and communities” and “effectively lead, manage, and participate in teams that provide care and improve outcomes for the diverse populations and communities they serve.” 13 Thus, the findings reported here can help bridge this gap by offering additional community-based EPAs to support the transition from medical school to residency.

Of note, tasks do not themselves ensure adherence to community-engagement principles. 9 Therefore, efforts must be made to elevate community voices when operationalizing these tasks, potentially through consultation with community members during the curriculum development process. Other limitations include that surveyed individuals were recruited based on their relationship with UWSMPH, and many of the final participants identified themselves as faculty/staff involved with community-engaged medical education programs, which might have resulted in participants who shared similar ideas. Community members were identified through roles with organizations connected with UWSMPH, which might not represent a broader patient population. Few respondents identified as community members, and how findings would differ had this group been more robustly represented is unclear.

Overall, findings from this project have provided guidance for the UWSMPH community-based educators to novelly augment existing and developing curriculum and to consider the progression of skills through graduate medical education. Similar efforts could be adapted by other medical education programs as they work to refine community-based curriculum.

Presentations

Parts of this study' findings were presented at the following meetings:

  • American Medical Association, Health Systems Science Summit. Chicago, IL. December 2022.

  • Association of American Medical Colleges, Central Region Group on Educational Affairs Spring Meeting, Indianapolis, IN. April 2023.

Acknowledgments

The authors are grateful to the Wisconsin Partnership Program of the University of Wisconsin School of Medicine and Public Health and the Herman and Gwendolyn Shapiro Foundation for funding this project and to all the survey participants for sharing their time and expertise.

References

  1. Ten Cate O, Chen HC, Hoff RG, Peters H, Bok H, van der Schaaf M. Curriculum development for the workplace using entrustable professional activities (EPAs): AMEE Guide No. 99. Med Teach. 2015;37(11):983-1,002. doi:10.3109/0142159X.2015.1060308
  2. Chen HC, van den Broek WE, ten Cate O. The case for use of entrustable professional activities in undergraduate medical education. Acad Med. 2015;90(4):431-436. doi:10.1097/ACM.0000000000000586
  3. Association of American Medical Colleges. Core Entrustable Professional Activities for Entering Residency: Curriculum Developers’ Guide. 2014. Accessed April 14, 2024. https://www.aamc.org/what-we-do/mission-areas/medical-education/cbme/core-epas/publications
  4. Johnson SB, Fair MA, Howley LD, et al. Teaching public and population health in medical education: an evaluation framework. Acad Med. 2020;95(12):1,853-1,863. doi:10.1097/ACM.0000000000003737
  5. Boelen C, Woollard R. Social accountability: the extra leap to excellence for educational institutions. Med Teach. 2011;33(8):614-619. doi:10.3109/0142159X.2011.590248
  6. Strasser R, Worley P, Cristobal F, et al. Putting communities in the driver’s seat: the realities of community-engaged medical education. Acad Med. 2015;90(11):1,466-1,470. doi:10.1097/ACM.0000000000000765
  7. Liaison Committee on Medical Education. Functions and Structure of a Medical School: Standards for Accreditation of Medical Education Programs Leading to the MD Degree. Association of American Medical Colleges and American Medical Association; 2023. Accessed April 14, 2024. https://lcme.org/publications
  8. Byrne LM, Nasca TJ. Population health and graduate medical education: updates to the ACGME’s common program requirements. J Grad Med Educ. 2019;11(3):357-361. doi:10.4300/JGME-D-19-00267.1
  9. Agency for Toxic Substances and Disease Registry. Principles of Community Engagement. 2nd ed.  ATSDR; August 2011. Accessed April 14, 2024. https://www.atsdr.cdc.gov/communityengagement/index.html
  10. Ezenwanne O, Crawford R, Remington PL. The race to the bottom: Wisconsin’s long-term trends in health rankings. WMJ. 2020;119(2):119-121.
  11. Waggoner J, Carline JD, Durning SJ. Is there a consensus on consensus methodology? descriptions and recommendations for future consensus research. Acad Med. 2016;91(5):663-668. doi:10.1097/ACM.0000000000001092
  12. Steeb DR, Brock TP, Dascanio SA, et al. Entrustable professional activities (EPAs) for global health. Acad Med. 2021;96(3):402-408. doi:10.1097/ACM.0000000000003856
  13. Newton W, Magill M, Barr W, Hoekzema G, Karuppiah S, Stutzman K. Implementing competency based ABFM board eligibility. J Am Board Fam Med. 2023;36(4):703-707. doi:10.3122/jabfm.2023.230201R0

Lead Author

Parvathy Pillai, MD, MPH

Affiliations: Department of Population Health Sciences, School of Medicine and Public Health, University of Wisconsin Madison, Madison, WI

Co-Authors

Amina Maamouri - School of Medicine and Public Health, University of Wisconsin, Madison, WI

Jonathan Temte, MD, PhD - Department of Family Medicine and Community Health, School of Medicine and Public Health, University of Wisconsin, Madison

Corresponding Author

Parvathy Pillai, MD, MPH

Correspondence: Department of Population Health Sciences, School of Medicine and Public Health, University of Wisconsin Madison, Madison, WI

Email: ppillai@wisc.edu

Fetching other articles...

Loading the comment form...

Submitting your comment...

There are no comments for this article.

Downloads & Info

Share

Related Content