Background and Objectives: The Accreditation Council for Graduate Medical Education requires all residents be trained in population health, but the most effective training strategies to impact care of patients and populations are not well established. The purpose of this study is to assess resident self-efficacy and expected application of population management skills through iterative experiential, longitudinal, team-based training in the office and community settings.
Methods: Using a prospective longitudinal curricular evaluation, we surveyed residents at a single institution from 2014-2020, evaluating self-efficacy in population health skills as well as perceived impact on patient care and future practice. We collected surveys before and after participating in a 3-year, longitudinal, team-based, experiential population health curriculum that integrates clinic-based quality improvement and community engagement projects.
Results: Fifty-nine of 68 residents (87%) responded to the presurvey, and 42/56 (75%) responded to the postsurvey. We observed significant increases in resident self-efficacy in all population health skills. All respondents reported finding common population health skills that were applicable in both office and community settings; 81% reported care of their continuity clinic patients changed because of taking part in the curriculum. Finally, 94% of respondents reported the intention to use population health skills and incorporate quality improvement (75%) and community engagement (100%) in future practice.
Conclusions: Teaching population health management skills in both office and community settings allows residents to integrate and apply these skills across settings and may enhance their use in patient care and future practice.
Population health management is an essential component of the triple aim, which seeks to improve US health care through improving patient care experience, health of populations, and cost of health care.1 The Accreditation Council for Graduate Medical Education (ACGME) competency-based Milestones,2–4 Common Program Requirements,5,6 and Clinical Learning Environment Review7–9 recognize the importance of resident population health education. New strategies are needed to train physicians to lower health care costs and address the health care needs of communities. While there is consistent agreement regarding the importance of population health in residency training, the most effective teaching strategies and how these skills translate into clinical practice are less studied.
Research in residency training has shown that population health skills can be enhanced through a variety of teaching methods including a flipped classroom,10 didactics,11–14 workshops,15,16 block rotations,17,18 and real-world practice.13,19 Current research has shown that residents value population health training that has a clear vision, valued resident contribution, dedicated time, faculty support, a structured curriculum, involves interprofessional teamwork, is experiential, longitudinal, and is horizontally integrated with the clinic and larger institution.19–30 The majority of population health residency training literature has focused on resident quality improvement (QI) efforts within the clinic setting or hospital, with less research in education of residents in the community.
Community engagement (CE), is defined as “the process of working collaboratively with and through groups of people affiliated by geographic proximity, special interest, or similar situations to address issues affecting the well-being of those people.”31 Training residents in this discipline can address community health needs, allow residents to reach at-risk populations, expand the breadth of interdisciplinary teamwork, and increase exposure to social determinants of health.32,33 Both CE and QI are founded on population management skills, including use of population data and plan-do-study-act (PDSA) cycles.34 Offering residents longitudinal, experiential training in these skills via both community and clinic settings presents an opportunity to reinforce the broad applicability of these skills in current and future practice.
The purpose of this study is to assess resident self-efficacy and expected application of population management skills through iterative, experiential, longitudinal, team-based training in office and community settings.
This study was a prospective, longitudinal curricular evaluation of residents from one residency program at a single institution from 2014-2020. We collected pre- and postsurveys at orientation and before graduation. Two resident cohorts had classes of 12 residents each, with eight residents per class in each of the subsequent cohorts. Demographics that describe the residency program for residents with graduation dates between 2017-2022 are detailed in Table 1.
All residents participated in a required longitudinal curriculum that integrated QI and CE in team-based, experiential learning. Residents learned and used PDSA cycles, leadership skills, and community assessment through longitudinal projects both in the clinic and in partnership with organizations in the community. Table 2 presents a description and timeline of the curriculum.
We invited all residents to voluntarily participate in the survey evaluating self-efficacy—defined as someone’s belief in their capacity to execute certain behaviors35,36—before and after completing the educational program. We structured questions to assess the degree to which learners found application for population health training in their clinical practice and future careers in accordance with the evaluation framework, based on the Kirkpatrick Model, for teaching population health in medical education proposed by Johnson, et al.37 Self-efficacy has been shown to be a strong predictor of behavior change (Kirkpatrick level 3),38 and has been used to assess likelihood of changing future practice among clinicians.36,39 We performed data collection using REDCap (Research Electronic Data Capture).40 We deidentified data for descriptive analysis. We used descriptive anlysis due to small sample size. Univariate statistical tests compared characteristics of residents who completed the survey. We used Student t tests to compare continuous characteristics and we used c2 tests to compare binary and categorical characteristics. The Penn State College of Medicine Institutional Review Board deemed this study exempt (STUDY#00002431).
We collected survey responses annually from 2014 through 2020. Fifty-nine of 68 (87%) responded to the presurvey, and 42/56 (75%) responded to the postsurvey.
Table 3 summarizes resident self-efficacy in population health skills assessed at the start and end of residency. Significant increases were seen in the percentage of residents expressing confidence in all population health skills.
All respondents reported finding common population health skills that were applicable in both office and community settings, as shown in Table 4. Most (81%) reported that the care of continuity clinic patients had changed due to taking part in the curriculum. Finally, most respondents reported the intention to use population health skills and incorporate QI and CE in future practice.
Our data support the effectiveness of this integrated curriculum in enhancing resident self-efficacy with population health skills, with a majority of respondents reporting that the curriculum has application to current and future practice. These results are in keeping with existing literature suggesting exposure to QI in training leads to increased implementation of QI into practice.41 All respondents found common population health skills that were applicable in both the office and community settings. Previous work by Knox et al42 demonstrated improved competency in related milestones and satisfaction among residents, but the translation of these skills between settings is a novel contribution to curricula in this area.
Limitations of this study include a small cohort at a single suburban program and may not be generalizable to other programs and settings. During the 3-year training experience, multiple factors outside this curriculum may have also influenced the participants’ responses.
Future studies may explore how exposure to a synergized QI and CE curriculum directly affects patient care outcomes, changes future practice, and impacts future population health behaviors.
Teaching population health management skills in both office and community settings allows residents to integrate and apply these skills across settings and may enhance their use in patient care and future practice.
Arthur Berg, PhD, provided statistical support for the project.
Financial Support: This work was partially supported by grant funding from Health Resources and Services Administration (HRSA) Primary Care Training and Enhancement 1T0BHP30010-01-11: A Campaign for Primary Care: Transforming Medical Education Today, to Develop the Leaders of Tomorrow, 2016-21.
Presentations: This work has been previously presented in the following settings:Poster at the EdVenture Conference in Hershey, PA in April, 2020 and Poster at STFM Annual Conference, May 2021.
- Berwick DM, Nolan TW, Whittington J. The triple aim: care, health, and cost. Health Aff (Millwood). 2008;27(3):759-769. doi:10.1377/hlthaff.27.3.759
- Accreditation Counsel for Graduate Medical Education. Family Medicine Milestones. 2019. Accessed May 29, 2021. https://www.acgme.org/Portals/0/PDFs/Milestones/FamilyMedicineMilestones.pdf
- Lane-Fall MB, Davis JJ, Clapp JT, Myers JS, Riesenberg LA. What every graduating resident needs to know about quality improvement and patient safety: a content analysis of 26 sets of ACGME Milestones. Acad Med. 2018;93(6):904-910.
- Duke Community and Family Medicine. Population Health Graduate Education Milestones: A Report to the Centers for Disease Control and Prevention and the Association of American Medical Colleges. Durham, NC; 2015. Accessed May 29, 2021.https://fmch.duke.edu/files/field/attachments/Population%20Health%20Milestones%20in%20Graduate%20Medical%20Education_web_0.pdf
- Accreditation Counsel for Graduate Medical Education. ACGME Common Program Requirements Common Program Requirements. 2017. Accessed April 14, 2018. http://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/CPRs_2017-07-01.pdf
- Byrne LM, Nasca TJ. Population health and graduate medical education: updates to the ACGME’s Common Program Requirements. J Grad Med Educ. 2019;11(3):357-361. doi:10.4300/JGME-D-19-00267.1
- Myers JS, Nash DB. Graduate medical education’s new focus on resident engagement in quality and safety: will it transform the culture of teaching hospitals? Acad Med. 2014;89(10):1328-1330. doi:10.1097/ACM.0000000000000435
- Tess AV, Rosen C, Tibbles C. Mapping quality improvement and safety education to drive change across training programs. J Grad Med Educ. 2015;7(2):275-276. doi:10.4300/JGME-D-14-00709.1
- Accreditation Counsel for Graduate Medical Education. CLER Pathways To Excellence: Expectations for an Optimal Clinical Learning Environment to Achieve Safe and High-Quality Patient Care, Version 2.0. Chicago, IL; ACGME 2019. doi:10.35425/ACGME.0003
- Bonnes SL, Ratelle JT, Halvorsen AJ, et al. Flipping the quality improvement classroom in residency education. Acad Med. 2017;92(1):101-107. doi:10.1097/ACM.0000000000001412
- Pensa M, Frew P, Gelmon SB. Integrating improvement learning into a family medicine residency curriculum. Fam Med. 2013;45(6):409-416.
- Bartman T, Heiser K, Bethune A, et al. Interprofessional QI training enhances competency and QI productivity among graduates: findings from nationwide children’s hospital. Acad Med. 2018;93(2):292-298. doi:10.1097/ACM.0000000000001862
- Duello K, Louh I, Greig H, Dawson N. Residents’ knowledge of quality improvement: the impact of using a group project curriculum. Postgrad Med J. 2015;91(1078):431-435. doi:10.1136/postgradmedj-2014-132886
- Pohl SD, Van Hala S, Ose D, Tingey B, Leiser JP. A longitudinal curriculum for quality improvement, leadership experience, and scholarship in a family medicine residency program. Fam Med. 2020;52(8):570-575. doi:10.22454/FamMed.2020.679626
- Jacobs C, Seehaver A, Skiold-Hanlin S. A longitudinal underserved community curriculum for family medicine residents. Fam Med. 2019;51(1):48-54. doi:10.22454/FamMed.2019.320104
- Tudiver F, Click IA, Ward P, Basden JA. Evaluation of a quality improvement curriculum for family medicine residents. Fam Med. 2013;45(1):19-25.
- Catalanotti J, Popiel D, Johansson P, Talib Z. A pilot curriculum to integrate community health into internal medicine residency training. J Grad Med Educ. 2013;5(4):674-677. doi:10.4300/JGME-D-12-00354.1
- Byrne JM, Hall S, Baz S, et al. Quality and safety training in primary care: making an impact. J Grad Med Educ. 2012;4(4):510-515. doi:10.4300/JGME-D-11-00322.1
- Moushey E, Shomo A, Elder N, O’Dea C, Rahner D. Community partnered projects: residents engaging with community health centers to improve care. Fam Med. 2014;46(9):718-723.
- Butler JM, Anderson KA, Supiano MA, Weir CR. “It feels like a lot of extra work”: resident attitudes about quality improvement and implications for an effective learning health care system. Acad Med. 2017;92(7):984-990. doi:10.1097/ACM.0000000000001474
- Jiménez J, Andolsek KM, Martinez-Bianchi V, Michener JL. A framework for Resident participation in population health. Acad Med. 2019;94(1):42-46. doi:10.1097/ACM.0000000000002471
- Vickery KD, Rindfleisch K, Benson J, Furlong J, Martinez-Bianchi V, Richardson CR. Preparing the next generation of family physicians to improve population health: A CERA Study. Fam Med. 2015;47(10):782-788.
- Potts S, Shields S, Upshur C. Preparing future leaders: an integrated quality improvement residency curriculum. Fam Med. 2016;48(6):477-481. PMID:27272426
- Courtlandt C, Noonan L, Koricke MW, Zeskind PS, Mabus S, Feld L. Pediatrics residents’ confidence and performance following a longitudinal quality improvement curriculum. J Grad Med Educ. 2016;8(1):74-79. doi:10.4300/JGME-D-15-00032.1
- Craig MS, Garfunkel LC, Baldwin CD, et al. Pediatric resident education in quality improvement (QI): a national survey. Acad Pediatr. 2014;14(1):54-61. doi:10.1016/j.acap.2013.10.004
- Philibert I, Gonzalez Del Rey JA, Lannon C, Lieh-Lai M, Weiss KB. Quality improvement skills for pediatric residents: from lecture to implementation and sustainability. Acad Pediatr. 2014;14(1):40-46. doi:10.1016/j.acap.2013.03.015
- Wilder V, Gagnon M, Olatunbosun B, et al. Community health needs assessment as a teaching tool in a family medicine residency. Fam Med. 2016;48(8):635-637.
- Bodenheimer T, Dickinson WP, Kong M. Quality improvement models in residency programs. J Grad Med Educ. 2019;11(1):15-17. doi:10.4300/JGME-D-18-00556.1
- Thibault GE. Resident empowerment as a driving theme of graduate medical education reform. Acad Med. 2018;93(3):357-359. doi:10.1097/ACM.0000000000001935
- Gold B, England D, Riley W, Jacobs-Halsey G, Webb C, Daniels B. Integrating quality improvement and continuing professional development at an academic medical center: a partnership between practice plan, hospital, and medical school. J Contin Educ Health Prof. 2016;36(4):307-315. doi:10.1097/CEH.0000000000000118
- Clinical and Translational Science Awards Consortium Community Engagement Key Function Committee Task Force. Principles of Community Engagement - Second Edition. 2011. Accessed May 29, 2021. https://www.atsdr.cdc.gov/communityengagement/pdf/PCE_Report_508_FINAL.pdf
- Wilkins CH, Alberti PM. Shifting academic health centers from a culture of community service to community engagement and integration. Acad Med. 2019;94(6):763-767. doi:10.1097/ACM.0000000000002711
- Talib Z, Palsdottir B, Briggs M, et al. Defining Community-Engaged Health Professional Education: A Step Toward Building the Evidence. NAM Perspectives.Discussion Paper, Washington, DC: National Academy of Medicine; 2017. https://doi.org/10.31478/201701a
- Kaprielian VS, Silberberg M, McDonald MA, et al. Teaching population health: a competency map approach to education. Acad Med. 2013;88(5):626-637. PMID:23524919 doi:10.1097/ACM.0b013e31828acf27
- Bandura A. Self-Efficacy: The Exercise of Control. New York: W.H. Freeman; 1997.
- Klassen RM, Klassen JRL. Self-efficacy beliefs of medical students: a critical review. Perspect Med Educ. 2018;7(2):76-82.
- Johnson SB, Fair MA, Howley LD, et al. Teaching public and population health in medical education: an evaluation framework. Acad Med. 2020;95(12):1853-1863.
- Kirkpatrick JD, Kirkpatrick WK, eds. Kirkpatrick’s four levels of training evaluation. Association for Talent Development Press; 2016.
- Klassen RM, Klassen JRL. Self-efficacy beliefs of medical students: a critical review. Perspect Med Educ. 2018;7(2):76-82. doi:10.1007/s40037-018-0411-3
- Williams BW, Kessler HA, Williams MV. Relationship among practice change, motivation, and self-efficacy. J Contin Educ Health Prof. 2014;34(suppl 1):S5-S10. doi:10.1002/chp.21235
- Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377-381. doi:10.1016/j.jbi.2008.08.010
- Diaz VA, Carek PJ, Johnson SP. Impact of quality improvement training during residency on current practice. Fam Med. 2012;44(8):569-573.
- Knox KE, Lehmann W, Vogelgesang J, Simpson D. Community Health, Advocacy, and Managing Populations (CHAMP) Longitudinal Residency Education and Evaluation. J Patient Cent Res Rev. 2018;5(1):45-54. Published 2018 Jan 30. doi:10.17294/2330-0698.1580