Introduction: Teaching quality improvement (QI) in undergraduate medical education to meet entrustable professional activities (EPA) requirements is a challenge. We describe a model where first-year medical students learn QI methods using online modules and then apply their knowledge by leading an interprofessional project in a clinical setting. This model project, set in an outpatient family medicine clinic, sought to improve patient compliance with the preventive care metric of annual serum potassium and creatinine monitoring for patients taking a diuretic, angiotensin-converting enzyme inhibitor (ACEI), or angiotensin receptor blocker (ARB) medication.
Methods: A first-year medical student joined an interprofessional team of one primary care physician and three pharmacy residents. The student led the team in reviewing patient charts to identify root causes and implementing a multifaceted strategy to improve patient outreach and refill policies. The effects of these interventions were monitored weekly using an electronic health record population health tool (Epic’s Healthy Planet) over the course of 10 months.
Results: At baseline, 76.7% (477/622) of patients taking a diuretic and 79.4% (752/947) taking an ACEI or ARB were compliant with monitoring, approximately 10% below the Healthcare Effectiveness Data and Information Set (HEDIS) recommendations. Within 6 months, interventions resulted in both patient cohorts meeting the HEDIS metric with sustained success for the study duration of 10 months.
Conclusion: This report demonstrates that a first-year medical student can make meaningful contributions to preventive care while gaining clinically relevant QI experience. Further evaluation is needed to determine generalizability and scalability of this model.
The Association of American Medical Colleges’ (AAMC) Core Entrustable Professional Activities (EPAs) recommend that all medical students be able to “identify system failures and contribute to a culture of safety and improvement” upon entering residency.1 However, teaching quality improvement (QI) in undergraduate medical education remains a challenge because of perceived inability of early students to contribute and limited curricular time.2 Students benefit from engaging in projects from beginning to end, but such involvement is often not possible due to conflicting responsibilities.3 Previous studies found that with proper support, preclinical students can make effective contributions in a clinical setting.4-6 In one study, medical students identified bottlenecks in clinic flow in a medical student-run clinic and implemented interventions to decrease visit times.4 This paper describes a novel education model where a first-year medical student learned QI methods and led a project as part of an interprofessional team, fostering competency in both the interprofessional and QI Core EPAs numbers 9 and 13, respectively.7
Sample and Design
This model is part of the Stanford Healthcare Innovations and Experiential Learning Directive (SHIELD),8 a program that empowers preclinical students to actively participate in patient care through value-added medical education.9 The goal of SHIELD is to immerse preclinical students as contributing members of interprofessional teams. Students spend approximately 5 hours completing the Institute of Healthcare Improvement (IHI) online modules 101 through 10510 to learn QI principles before developing and executing a project. One to two students are matched with one faculty mentor who commits 2 to 4 hours per week. Students join the mentor’s existing clinical team and commit 3 to 5 hours per week for at least 1 year to allow time for iterative tests of change, a key component of experiential QI learning.11 For this study, one first-year medical student (CK) worked with one primary care physician (PCP) mentor (AS) and three pharmacy residents, who each rotated with the team for 3 months. This project was determined by the Institutional Review Board of Stanford University School of Medicine to constitute a quality improvement project and not human subjects research.
Preventive Care Gap Identification
The team identified a preventive care gap in an outpatient family medicine clinic using Epic’s Healthy Planet, an embedded population health tool that tracks the clinic’s performance on utilization and preventive care metrics. The tool shows the percentage of patients meeting each metric compared with the Healthcare Effectiveness Data and Information Set (HEDIS) benchmarks.
Root Cause Analysis and Tests of Change
The team generated a list of possible root causes, which are the reasons why patients are noncompliant with a preventive care metric, based on observations in clinic. Team members reviewed a total of 33 patient charts, 3 from each of the 11 clinic providers. Team members reviewed lab orders and notes, spending approximately 10 minutes per chart. The frequency of occurrence for each of the root causes was recorded, with a frequency of 2 or above qualifying as a root cause. Based on identified key drivers, the team implemented 4 iterative plan-do-study-act (PDSA) cycles, which refer to a 4-step method to design a change, carry it out, assess the consequences, and determine what modifications are needed for the next cycle. One cycle occured approximately every 2 months.
PDSA 1: Phone Calls
The team used the electronic health record (EHR) system Epic to identify overdue patients, and called those who were long overdue (noncompliant for at least 18 months) or recently overdue (noncompliant within the past 3 to 6 months). For patients who had switched PCPs, the team updated their charts to reflect this change. For those who were still under the clinic’s care, the team counseled patients about the recommended tests and submitted the necessary orders through Epic.
PDSA 2: Personalized Electronic Messaging
Due to inconsistent success reaching patients via phone calls, the team started using personalized electronic messaging, sending secure messages to 70 patients through the EHR patient portal (Epic’s MyHealth). These personalized messages suggested patients complete their tests on a specific date, selected to coincide with an upcoming clinic appointment to promote compliance.
PDSA 3: Bulk Electronic Messaging and Ordering
Based on the success of messaging, the team initiated bulk outreach by sending generic electronic messages and ordering monitoring tests for approximately 230 overdue patients.
PDSA 4: Standardized Refill Protocol
Because refill encounters are the most consistent contact with patients not seen in clinic, the team revised the clinic’s medication refill protocol to include checking for preventive care gaps, consulted with staff to review protocol drafts, and implemented the standardized protocol in the clinic.
Root causes were analyzed using a Pareto chart (a bar graph that organizes root causes in order of the magnitude of their effect). The team monitored the impact of PDSA cycles using Epic’s Healthy Planet to identify the compliance percentage every week. A run chart (a line graph used to chart data during the study period) was used to track compliance over 10 months and to assess the long-term effect of each PDSA cycle.
Preventive care gap identification showed that the clinic was failing to meet the preventive care metric for annual serum potassium and creatinine monitoring for patients on a diuretic, angiotensin-converting enzyme inhibitor (ACEI), or angiotensin receptor blocker (ARB) medication. At the start of the study, 76.7% (477/622) of patients on a diuretic and 79.4% (752/947) on an ACEI/ARB were compliant with annual potassium and creatinine monitoring, approximately 10% below the HEDIS recommendation for both.12 Root cause analysis identified eight key drivers underlying patient noncompliance (Figure 1). The most frequent causes were no follow-up appointments scheduled (23.0%, 28/122 total occurrences), no tests ordered (17.2%, 21/122), and no clinic appointments in the past year (15.6%, 19/122). Phone calls (PDSA 1) led to an increase of 0.2% and 2.3% of average compliance percentage compared with average baseline percentage for diuretics and ACEI/ARBs, respectively (Figure 2). Personalized electronic messaging (PDSA 2) led to an increase of 3.4% and 1.9%. Bulk electronic messaging and ordering (PDSA 3) led to an increase of 3.6% and 2.8%. The standardized refill protocol (PDSA4) led to an increase of 2.1% and 1.1%. Within 6 months, the team reached the HEDIS goal for both diuretic (86.8%, 528/608) and ACEI/ARB (89.0%, 824/925) cohorts. To maintain success, the clinic staff is using the standardized refill protocol and conducting quarterly bulk electronic messaging and ordering. The first-year medical student was able to apply QI principles in a clinical setting and work as a meaningful member of an interprofessional team based on feedback from the PCP mentor and clinic staff.
A first-year medical student successfully applied QI principles to lead an interprofessional team in meeting the HEDIS medication monitoring metric and maintaining success for the duration of the 10-month study period. These findings suggest that preclinical students can help PCPs fulfill population health demands that require over 7 hours per working day.13 These results add to the growing literature demonstrating that combined didactic and experiential learning provides meaningful QI education.14-16
This study is limited by its small participant number and its dependence on close faculty mentorship and staff buy-in. To assess generalizability, more students will need to be involved in similar opportunities over time through SHIELD following this general model (Figure 3). To overcome the potential barrier of close faculty mentorship, a cascading mentorship structure can be created in which teams of 3 to 4 first-year students work together under the supervision of a third or fourth-year medical student; 2 to 3 of these student teams could be mentored under one faculty member. Regarding staff buy-in, the team found that pharmacy residents, nurses, and other health care practitioners were excited to participate in PDSA implementation if the need for improving patient care was clearly communicated, and staff was incorporated into decision-making as much as possible, for example during departmental meetings. Creating an interprofessional team helps address this barrier since staff are already included as core members. Future studies should assess the impact of this program on medical student learning by using a validated pre- and postassessment questionnaire to assess students’ familiarity with QI learning tools, QI perspectives, and QI competencies.
In conclusion, this report demonstrates that first-year medical students can make meaningful contributions to patient care while gaining clinical QI experience. This model exemplifies learner-centered education by immersing students in every step of the QI process and empowering them to lead interprofessional teams.
The authors thank the pharmacy residents (Walter Domingo, PharmD; Benny Yau, PharmD; Daniel Polchinski, PharmD), the population health nurse (Mary Christensen, RN), the Healthy Planet Working Group, and the Stanford Family Medicine clinic staff who contributed to this project.
Part of the data from this study was presented at the Institute for Healthcare Improvement’s 18th Annual Summit in Orlando, Florida from April 20-22, 2017, and at the Society of Teachers of Family Medicine's 2018 Conference on Medical Student Education in Austin, Texas from February 1-4, 2018.
- Lomis K, Amiel JM, Ryan MS, et al; AAMC Core EPAs for Entering Residency Pilot Team. Implementing an Entrustable Professional Activities Framework in Undergraduate Medical Education: Early Lessons From the AAMC Core Entrustable Professional Activities for Entering Residency Pilot. Acad Med. 2017;92(6):765-770. https://doi.org/10.1097/ACM.0000000000001543
- Gonzalo JD, Caverzagie KJ, Hawkins RE, Lawson L, Wolpaw DR, Chang A. Concerns and Responses for Integrating Health Systems Science Into Medical Education. Acad Med. 2018;93(6):843-849. https://doi.org/10.1097/ACM.0000000000001960
- Gould BE, Grey MR, Huntington CG, et al. Improving patient care outcomes by teaching quality improvement to medical students in community-based practices. Acad Med. 2002;77(10):1011-1018. http://www.ncbi.nlm.nih.gov/pubmed/12377677. Accessed July 2, 2017. https://doi.org/10.1097/00001888-200210000-00014
- Chen CA, Park RJ, Hegde JV, et al. How we used a patient visit tracker tool to advance experiential learning in systems-based practice and quality improvement in a medical student clinic. Med Teach. 2016;38(1):36-40. https://doi.org/10.3109/0142159X.2014.975193
- Remus KE, Honigberg M, Tummalapalli SL, Cohen LP, Fazio S, Weinstein AR. A Chronic Disease Management Student-Faculty Collaborative Practice: Educating Students on Innovation in Health Care Delivery. Acad Med. 2016;91(7):967-971. https://doi.org/10.1097/ACM.0000000000001147
- Gonsenhauser I, Beal E, Shihadeh F, Mekhjian HS, Moffatt-Bruce SD. Development and assessment of quality improvement education for medical students at The Ohio State University Medical Center. J Healthc Qual. 2012;34(6):36-42. https://doi.org/10.1111/j.1945-1474.2012.00160.x
- Association of American Medical Colleges. The Core Entrustable Professional Activities (EPAs) for Entering Residency. https://www.aamc.org/initiatives/coreepas/. Accessed November 11, 2017.
- Lin S, Osborn K, Sattler A, et al. Creating the medical school of the future through incremental curricular transformation: the Stanford Healthcare Innovations and Experiential Learning Directive (SHIELD). Educ Prim Care. 2017;28(3):180-184. https://doi.org/10.1080/14739879.2016.1259965
- Lin SY, Schillinger E, Irby DM. Value-added medical education: engaging future doctors to transform health care delivery today. J Gen Intern Med. 2015;30(2):150-151. https://doi.org/10.1007/s11606-014-3018-3
- Institue for Healthcare Improvement (IHI). IHI Open School Online Courses. http://www.ihi.org/education/IHIOpenSchool/Courses/Pages/default.aspx. Accessed November 11, 2017.
- Wong BM, Levinson W, Shojania KG. Quality improvement in medical education: current state and future directions. Med Educ. 2012;46(1):107-119. https://doi.org/10.1111/j.1365-2923.2011.04154.x
- National Committee for Quality Assurance. HEDIS Measures. http://www.ncqa.org/hedis-quality-measurement/hedis-measures. Accessed November 3, 2017.
- Yarnall KSH, Pollak KI, Østbye T, Krause KM, Michener JL. Primary care: is there enough time for prevention? Am J Public Health. 2003;93(4):635-641. https://doi.org/10.2105/AJPH.93.4.635
- Headrick LA, Moore SM, Alemi F, et al. Using PDSA (Plan-Do-Study-Act) to establish academic-community partnerships: the Cleveland experience. Qual Manag Health Care. 1998;6(2):12-20. https://doi.org/10.1097/00019514-199806020-00002
- Dean Cleghorn G, Baker R. What faculty need to learn about improvement and how to teach it to others. J Interprof Care. 2000;14(2):147-159. https://doi.org/10.1080/jic.22.214.171.124
- Ogrinc G, Headrick LA, Morrison LJ, Foster T. Teaching and assessing resident competence in practice-based learning and improvement. J Gen Intern Med. 2004;19(5 Pt 2):496-500. https://doi.org/10.1111/j.1525-1497.2004.30102.x