Evaluation of a Quality Improvement Experience for Family Medicine Clerkship Students

While the importance of systems-based practice (SBP) knowledge is emphasized in graduate medical education, limited work on systems-based practice such as quality improvement (QI) has been done in undergraduate medical education (UME) settings. Teaching SBP is often limited to didactics and case review with little experiential learning.1-3 QI can be taught effectively in primary care, but students may not perceive relevancy.3 In addition, lack of faculty expertise and competing educational demands may limit student exposure to this topic.4 Here, we describe an evaluation of our approach to teaching QI for family medicine (FM) clerkship students.

We identified existing resources, developed an online curriculum, and implemented QI experiences into our 5-week, third-year FM clerkship. Initially, we tested providing QI experiences with small cohorts of subinternship students. Based on feedback, we modified self-directed learning, improved assigned learning, and focused the QI experience. We developed a five-module online course using our institution’s web platform (Table 1). This self-directed curriculum was anchored on the Institute for Healthcare Improvement (IHI) Open School QI 102 module, because of its usability and to stimulate interest in obtaining a certificate in quality and safety.5 Additional content and tools connected didactics to the experiential curriculum in an online QI toolkit.

After completing the self-directed modules, students had 1 half-day to meet in groups of two to six to make practice observations (ie, shadowing medical assistants), speak with stakeholders, conduct chart reviews, and develop process maps. Initially, students were given freedom to identify any process in the clinical environment to study. Later, we guided students to work on particular processes and measures based on the practice QI team needs (eg, hemoglobin A1c control). The students’ objective was to work through planning steps leading up to the plan-do-study-act (PDSA) cycle (choosing a focus area, aim setting, process mapping, brainstorming, and prioritization) and to choose one change idea to implement in practice (Figure 1).

Students pitched change ideas to faculty at the end of their rotation in a format modeled after ABC’s Shark Tank television show.6 Faculty provided hypothetical funding for projects, guided discussions about principles of QI, reinforced measurement issues, feasibility, and balancing measures.

We developed a nine-question survey of self-reported QI knowledge and self-efficacy for stated learning objectives. Students completed the questionnaire before and after completing the rotation but survey results were not matched by student. Postsurveys assessed satisfaction with the curriculum and had prompts for curriculum improvement.

We used descriptive statistics to characterize the sample. We excluded responses if they were incomplete or if the respondents were not third-year medical students. Histograms of responses were highly skewed for both pre- and postsurveys so change in self-reported knowledge and self-efficacy pre- and postcurriculum was assessed. We collapsed responses for items one through eight into two categories: students who chose “agree” or “strongly agree” (agreed) versus “neutral,” “disagree” or “strongly disagree” and the proportion of students choosing “agree” or “strongly agree” pre- and postcurriculum were compared using χ2 tests. We reviewed open-ended responses to refine the curriculum; authors reviewed comments and sorted them into themes. The Thomas Jefferson University Institutional Review Board approved all activities.

One hundred percent of eligible students (399 total) completed the presurvey over 18 blocks from April 2017 to March 2019. Ten responses were excluded: two incomplete records, seven fourth-year students, and one physician assistant student who participated in the curriculum but was not part of the study population. Students completed 248 postsurveys; six were excluded (four fourth-year students and two physician assistant students) for a total of 242 postsurveys. Table 2 shows the percentages of students who agreed for survey items one through nine. There was an increase in respondents who agreed that they had the stated QI knowledge and skills (P<.001). Most reported feeling reasonably confident or confident in their ability to create change in healthcare and this increased in the post survey (P<.001).

Fifty-two percent (n=126; 52.2%) agreed/strongly agreed that the toolkit facilitated learning, and 45.6% (n=110) would recommend it to others. One-third (n=81; 32.7%) intend to follow up and complete the remaining IHI modules. Most (n=158; 65.4%) reported the toolkit taking 1-3 hours to complete. In open-ended responses, two themes emerged: usefulness of the content and the opportunity to shadow medical assistants and front office staff. One said, “This was a valuable exercise that allowed us to look at health care in a different way,” another commented that “(I) actually struck a new interest for myself as a career goal and plan to do the rest of the IHI modules.”

We implemented a blended model to teach QI principles using self-directed online modules, collaborative group learning, and presentation to faculty in a classroom setting. We showed that this group-based flipped-classroom7,8 curriculum increased self-reported knowledge and confidence in the PDSA process for FM clerkship students. We are unaware of similar published QI curricula for FM clerkship students. By utilizing existing resources and focusing on aim setting and planning for PDSA interventions rather than full QI projects, we created a QI experience that could fit within a clinical rotation. The Shark Tank format was engaging for participants while reducing faculty time commitment. Our program is flexible, allowing leadership to direct students toward processes in need of improvement, aligning student efforts with practice needs. Students valued both QI knowledge and working with interdisciplinary team members, highlighting the importance of learning the practice of medicine in the context of a system-based approach.

This study was limited to a single program, so findings may not be generalizable. Outcomes assessed student understanding of the PDSA process; unfamiliarity with jargon in the presurvey may overestimate the impact of our intervention. We had a low response rate in the postcurriculum survey, which may skew our results.

The FM clerkship provides a rich environment for learning about QI and the complexity of implementing best practices in the real world. Having students engage with stakeholders and map processes that underlie quality measures, patient safety topics, and the user experience provides a deep understanding of how didactic learning relates to the practice of medicine. We show that a flipped-classroom curriculum can improve QI knowledge and comfort with implementing QI in FM clerkship students. We hope to build on this curriculum and study the impact of implementing student change ideas on clinical quality measures, patient satisfaction, and patient outcomes.