A Longitudinal Curriculum for Quality Improvement, Leadership Experience, and Scholarship in a Family Medicine Residency Program

Susan D. Pohl, MD | Sonja Van Hala, MD, MPH | Dominik Ose, DrPH, MPH | Benjamin Tingey, BS | Jennifer P. Leiser, MD

Fam Med. 2020;52(8):570-575.

DOI: 10.22454/FamMed.2020.679626


Background and Objectives: The Accreditation Council for Graduate Medical Education (ACGME) requires all residents be trained in quality improvement (QI), and that they produce scholarly projects. While not an ACGME requirement, residents need leadership skills to apply QI knowledge. We developed the Skills-based Experiential Embedded Quality Improvement (SEE-QI) curriculum to integrate training in QI, leadership, and scholarship.

Methods: The University of Utah Family Medicine Residency Program began using the novel curriculum in 2012. The aim of the curriculum is to tie didactic teaching in quality improvement, leadership, and scholarship with skills application on multidisciplinary QI teams. Coaching for resident leaders is provided by faculty. Third-year resident leaders prepare academic presentations. Results of the ACGME Practice-Based Learning and Improvement (PBLI) 3 scores and number of scholarship presentations are described as a measure of efficacy.

Results: Two cohorts of residents completed the curriculum and all competency assessments. The average initial and final competency scores for competency PBLI-3 showed improvement and the average final competency for each cohort was above the proficient level. The residency requirements for QI scholarship did not change with introduction of the curriculum, but the amount of optional curricular QI scholarship and independent QI scholarship increased.

Conclusions: The SEE-QI curriculum resulted in a high level of resident QI competency, opportunity for leadership training, and an increase in scholarship. We studied the results of this curriculum at one institution. Efforts to tie QI, leadership, and scholarship training should be evaluated at other programs.

Residency programs must prepare physicians to meet the health care needs of our nation. The Accreditation Council for Graduate Medical Education (ACGME) requires that residents in all medical specialties be trained in quality improvement (QI), and that they complete scholarly projects.1 Physicians must be trained in leadership skills in order to achieve quality and safety goals of patient care.2,3 Residents, however, have described QI training as confusing, difficult to manage, and disconnected from clinical experience.4 Integrating QI training into clinical practice is challenging. Educational interventions that accomplish multiple educational goals simultaneously are desirable for their efficiency, as residency programs are challenged to meet all educational directives.5 A strategy for linking resident training in QI with scholarship has been published,6 but descriptions of existing curricula describe training that occurs in a single year,7 expect residents to work in isolation,8 and lack scholarship support.7-9

The aim of this study was to evaluate a Skills-based Experiential Embedded Quality Improvement (SEE-QI) curriculum that can be used for multiple missions: to train residents in QI, leadership, and scholarship. The goals of the curriculum are to achieve high resident competency in QI, provide resident leadership experience, and increase scholarship. This report describes the curriculum and its outcomes.


The University of Utah Family Medicine Residency Program (UUFMRP) developed the SEE-QI curriculum as a 3-year longitudinal skills-based QI curriculum. Key principles of the curriculum are: (1) longitudinal participation in QI teams throughout residency, with advancing responsibility each year; (2) leadership coaching for resident leaders on QI teams; and (3) scholarship training to help residents achieve QI scholarship (Figure 1).

The SEE-QI curriculum contains didactic sessions and application that is embedded in clinical operations at two residency clinics. Since 2012, residents learn by participating on and leading multidisciplinary teams through QI methodology for the advancement of clinical practice and disseminating their findings through scholarship.

Foundational Training

Training includes 12 didactic sessions repeated each year. QI methods are based on the Model for Improvement (Figure 2).10,11

Example topics in the didactic sessions are process definition, team functionality, change management, and Squire guidelines (Table 1).12

QI in Clinical Practice: Implementing Change

Residents apply QI and leadership training by leading a QI team. All teams in this program are multidisciplinary, consisting of faculty, residents, nurses, clinical pharmacists, behavioral health providers, medical assistants, and reception staff. The teams meet monthly for 1 year and choose a topic related to chronic disease, screening, or immunization metrics.

Leading Teams

All teams are led by third-year residents with direct supervision by faculty. First- and second-year residents participate as team members. Faculty leadership coaches debrief with residents after each team meeting and record qualitative feedback.

Advancing Scholarship

Resident leaders are required to disseminate the results of their team’s work in an oral presentation, and are encouraged to create an optional scholarship poster for external presentation. A faculty member serves as a scholarship coach on each team. To support scholarship, we created yearly scholarship time lines of submission deadlines, implemented an umbrella institutional review board approval, and offered staff assistance for poster preparation.


We based evaluation of the curriculum on a descriptive, explorative approach and focused on residency competency in QI and scholarship. We reviewed Practice-Based Learning and Improvement Milestone 3 (PBLI-3) “Improve systems in which the physician provides care,” developed by the ACGME for family medicine. Milestone competency levels range from Level 1 (novice) to Level 5 (expert).13 The Milestones were implemented in academic year 2015; competency levels were assessed by our Clinical Competency Committee, based on the ACGME rubric. For statistical analysis, we compared the mean scores achieved in PBLI-3 at the beginning and at the end of the residency program by using a Wilcoxon Matched-Pairs Sign-Rank Test. Because the Milestones were implemented in academic year 2015, data for the classes of 2017 and 2018 were available; only classes that completed all 3 years of measurement were included in statistical analysis.

In order to evaluate scholarship, we compared yearly resident-led scholarship products from 2008 to 2018. We constructed a stacked bar chart to show the number of products by type (required curricular QI scholarship, optional curricular QI scholarship, independent non-QI scholarship, and independent QI scholarship) by year.

The University of Utah IRB approved this study (#00124825).


Over the course of the curriculum, the mean scores for PBLI-3 increased significantly in the class of 2017 from 1.4 to 4.6 (P<.05) ,and in the class of 2018 from 1.4 to 4.4 (P<.05; Figure 3). Prior to implementing the curriculum, the baseline QI scholarship rate was eight required oral QI presentations per year with no optional or independent QI scholarship. Once the curriculum was implemented, optional curricular QI scholarship increased, independent QI scholarship increased and there was no decrease in other scholarship (Figure 4). In addition to the scholarship presentations, four residents published peer-reviewed articles of their required SEE-QI project after completing residency.14-17


The SEE-QI curriculum leveraged the ACGME requirements for QI for multiple educational purposes. Prior to the curriculum, resident QI work was an isolated endeavor, not embedded in clinical operations, with minimal scholarship. The curriculum resulted in a high level of resident competency, provided leadership experience in a clinical environment, and increased resident scholarship. By the end of the curriculum, every resident was able to lead a multidisciplinary team through the QI process. As measured by Milestone PBLI-3, residents demonstrated advancement in their QI skills over the course of the 3-year family medicine residency program.

Although residency scholarship requirements did not change, scholarship increased with implementation of the curriculum. The increase in scholarship was most notable in optional poster presentations, but new independent QI scholarship was also produced. We attribute this to creating an understanding of the QI framework as a scholarship opportunity and increased programmatic support.

This evaluation has several limitations. First, ACGME Milestones were implemented in academic year 2015. Prior to 2015, resident competency was assessed by direct faculty observation; we did not compile data on these observations. The QI competency scores described may be biased due to faculty involvement in the curriculum, but Milestone scores were determined by unanimous decision of the Clinical Competency Committee, following standard residency procedures. The Milestone evaluative process is likely to limit individual faculty bias. Second, no formal evaluation of resident leadership skills was conducted, as the curricular design followed a coaching model based on observation, reflection and discussion. Third, because this evaluation is based on a descriptive, explorative approach, the results are not generalizable.

The SEE-QI curriculum is an innovative approach that combines training in advanced QI skills, expectation for leadership, as well as a structure for supporting required and optional scholarship. The curriculum should be tested for its applicability in other programs and specialties.


  1. Accreditation Council for Graduate Medical Education. Common Program Requirements. Chicago, IL: ACGME; 2018. Accessed July 7, 2020.
  2. Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system—rationale and benefits. N Engl J Med. 2012;366(11):1051-1056. doi:10.1056/NEJMsr1200117
  3. Committee on Quality of Health Care in America. Crossing the quality chasm: a new health system for the 21st century. Institute of Medicine. Washington, DC: National Academy Press; 2001.
  4. Butler JM, Anderson KA, Supiano MA, Weir CR. “It Feels Like a Lot of Extra Work”: Resident Attitudes About Quality Improvement and Implications for an Effective Learning Health Care System. Acad Med. 2017;92(7):984-990. doi:10.1097/ACM.0000000000001474
  5. Frey K, Edwards F, Altman K, Spahr N, Gorman RS. The ‘Collaborative Care’ curriculum: an educational model addressing key ACGME core competencies in primary care residency training. Med Educ. 2003;37(9):786-789. doi:10.1046/j.1365-2923.2003.01598.x
  6. Neeman N, Sehgal NL. Perspective: a road map for academic departments to promote scholarship in quality improvement and patient safety. Acad Med. 2012;87(2):168-171. doi:10.1097/ACM.0b013e31823f3c2c
  7. Hall Barber K, Schultz K, Scott A, Pollock E, Kotecha J, Martin D. Teaching Quality Improvement in Graduate Medical Education: An Experiential and Team-Based Approach to the Acquisition of Quality Improvement Competencies. Acad Med. 2015;90(10):1363-1367. doi:10.1097/ACM.0000000000000851
  8. Cheung L. SAFE QI - a framework to overcome the challenges of implementing a quality improvement curriculum into a residency program. Adv Med Educ Pract. 2017;8:779-784. doi:10.2147/AMEP.S150718
  9. Potts S, Shields S, Upshur C. Preparing Future Leaders: An Integrated Quality Improvement Residency Curriculum. Fam Med. 2016;48(6):477-481.
  10. Langley GL, Moen R, Nolan KM, Nolan TW, Norman CL, Provost LP. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. 2nd ed. San Francisco: Jossey-Bass Publishers; 2009.
  11. Farrer K, Donaldson E, Blackett B, et al. Nutritional screening of elderly patients: a health improvement approach to practice. J Hum Nutr Diet. 2014;27(2):184-191. doi:10.1111/jhn.12073
  12. Ogrinc G, Mooney SE, Estrada C, et al. The SQUIRE (Standards for Quality Improvement Reporting Excellence) guidelines for quality improvement reporting: explanation and elaboration. Qual Saf Health Care. 2008;17 Suppl 1(Suppl_1):i13–i32.
  13. The family medicine milestone project. J Grad Med Educ. 2014;6(1)(suppl 1):74-86.
  14. Prince VL, McNamara S, Gondor K, Pohl S, Sanyer O, Frame KA. Use of a Hepatitis C Screening Order Set to Improve Screening Rates and Measure Intervention Effectiveness. PRiMER Peer-Rev Rep Med Educ Res. 2019;3:13. doi:10.22454/PRiMER.2019.282710
  15. Jones KB, Gren LH, Backman R. Improving pediatric immunization rates: description of a resident-led clinical continuous quality improvement project. Fam Med. 2014;46(8):631-635.
  16. Jones KB, Spain C, Wright H, Gren LH. Improving Immunizations in Children: A Clinical Break-even Analysis. Clin Med Res. 2015;13(2):51-57. doi:10.3121/cmr.2014.1234
  17. Badarane D, Knox J, Camacho A, Magill MK, Van Hala S, Jones JL. Increasing Chlamydia Testing Rates via Targeted Outreach. PRiMER Peer-Rev Rep Med Educ Res. 2019;3:17. doi:10.22454/PRiMER.2019.669190

Lead Author

Susan D. Pohl, MD

Affiliations: University of Utah Department of Family and Preventive Medicine, University of Utah Medical School, Salt Lake City, UT


Sonja Van Hala, MD, MPH - Department of Family and Preventive Medicine, University of Utah School of Medicine, Salt Lake City, UT

Dominik Ose, DrPH, MPH - Department of Family and Preventive Medicine, University of Utah, Salt Lake City, UT

Benjamin Tingey, BS - Department of Family and Preventive Medicine, University of Utah, Salt Lake City, UT

Jennifer P. Leiser, MD - Department of Family and Preventive Medicine, University of Utah, Salt Lake City, UT

Corresponding Author

Susan D. Pohl, MD

Correspondence: 375 Chipeta Way, Suite A, Salt Lake City, UT 84108. 801- 585-5287. Fax: 801-581-2759.


Fetching other articles...


Loading metrics from Mendeley...

Loading the comment form...

Submitting your comment...

There are no comments for this article.

Downloads & Info


Related Content