The Experiential Improvement Curriculum: Teaching Improvement Science to Preclinical Medical Students in Primary Care

Systems improvement is an important domain in medical education,^1,2 but best practices for engaging medical students in improvement work within primary care remain limited.^3-7 At our institution, a medical student designed, implemented, and evaluated the Experiential Improvement Curriculum (EIC) for preclinical medical students to conduct improvement projects at primary care sites. The EIC’s inception relied on local experts in improvement curriculum design⁸ and primary care faculty who coordinated clinical sites for curriculum and project implementation.

The curriculum is nested within a 15-month alternative preceptorship program, the Student Navigator Project (SNaP), where first- and second-year students serve as patient navigators in academic primary care clinics.⁹ EIC itself spans 10 months to include small group didactics, a clinical improvement project, and scholarly synthesis of work (Figure 1). With the support of a faculty coach, learners utilize the IHI Open School Quality Improvement (QI) Practicum Handbook¹⁰ to create a project charter, develop a cause-and-effect diagram, conduct plan-do-study-act (PDSA) cycles, construct a run chart, and create a poster. Clinic staff are recruited as project champions to optimize stakeholder buy-in.

The Quality Improvement Knowledge Assessment Tool–Revised (QIKAT-R),¹¹ delivered before and after the curriculum, evaluated the learners’ ability to define an aim, measure, and change idea based on three clinical scenarios. Three faculty physicians with credentialed improvement experience scored the QIKAT-R responses in a blinded fashion using the established rubric.¹¹ Learners also completed a pre/postsurvey assessing their perceived skills and attitudes toward improvement work.¹² To maintain anonymity, each learner created a deidentified set of numbers and letters to link their pre- and postresponses.

We conducted a paired t test to analyze the QIKAT-R and survey responses. We calculated intraclass correlation coefficients to assess interrater reliability on the QIKAT-R. We set a significance level of 0.05 for both assessment tools. The QIKAT-R analyses were performed using SAS 9.4 software (SAS Institute Inc., Cary, NC). We performed the survey analyses using Minitab Statistical Software v18.1 (Minitab LLC, State College, PA). This work was conducted while the first author was a medical student. The Institutional Review Board deemed this study exempt from full review. The assessment tools are available on the STFM Resource Library.¹² 

Learner projects addressed the medication reconciliation and food insecurity screening processes.¹² In a separate survey completed at the conclusion of the SNaP program, students reported appreciating the hands-on experience of conducting a project, developing teamwork skills, and feeling empowered to improve patient outcomes.⁹

Seven out of eight learners had no exposure to improvement work prior to medical school. We obtained complete QIKAT-R results from seven learners. In the 27-point assessment, there was an average improvement of 4.5 points (95% confidence interval of 0.55, 8.49), from 15.3 points (standard deviation of 4.5) to 19.8 points (standard deviation of 2.8) (P=.03). Interrater reliability among the three graders was determined by pre- and post-intraclass correlation coefficient, which were 0.46 and 0.13, respectively. We observed significant improvement in mean scores in the majority of survey items (Table 1). 

The student-led Experiential Improvement Curriculum is a model for preclinical medical students to apply improvement science knowledge towards real-world settings, particularly primary care. EIC’s core feature is its placement within a 15-month program that utilizes experiential learning methods in a primary care environment. Once embedded in the clinic setting, learners alternate between the clinic and classroom over ten months to conduct projects.

The curriculum generated favorable outcomes, as reflected by the improvement in total QIKAT-R scores and the learners’ perceived ability to demonstrate key skills. Their increased perceived ability to execute a PDSA cycle can be attributed to the four to 5-month period available to lead multiple PDSA cycles followed by the time to synthesize and communicate the implications of their work. The lack of significant improvement in their perceived ability to develop aims and measures could represent a curricular deficit. The overall mean improvement in QIKAT-R scores is also qualified by a wide 95% confidence interval, although the individual components had minimal overlap in standard deviation. Additionally, the poor pre- and post-interrater reliability indicate a need for improved training with utilizing the QIKAT-R rubric.

Our generalizability is limited as a single site study with a small cohort of self-selected learners. As we move forward with further curricular modifications, we strive to maintain the core intention of EIC: by using the primary care setting as a longitudinal space for change, we can integrate experiential improvement learning early in medical education with the hope that students continue to build on these principles throughout their training.