Experiential QI Activity for Residents to Improve Women’s Preventive Services

To meet Accreditation Council for Graduate Medical Education (ACGME) requirements, residents are required to complete a quality improvement (QI) project.¹ Impactful QI residency training programs involve clinical outcomes and metrics, direct role modeling, and evaluations of patient impact.^2,3 Most QI curricula focus on learner knowledge.^2-4 Primary care-based QI education models should incorporate experiential learning and QI application.^4-8 More QI curricula that focus on evaluation of clinical outcomes while educating learners are needed.^9,10

We developed a QI project to improve women’s preventive health services using experiential learning activities for FM residents who practice in an underserved clinic. We conducted a quasi-experimental project to determine: (1) how women’s receipt of vaccines and cancer screenings compared to benchmarks, (2) physician and staff knowledge of processes and barriers to receiving preventive services, and (3) whether an intervention for physicians and staff improved patients’ receipt of preventive services.

The project was developed for the Parkland Family Medicine Clinic, the training site for family medicine residents. The clinic conducts approximately 20,000 visits per year among medically underserved residents in Dallas, Texas. More than 60% of patients have limited or no English proficiency.

Preintervention, we randomly selected electronic medical records (EMRs, N=505) from 2,500 female patient (ages 40 years and older) visits during a 4-month period, based on a clinic report. First-year residents reviewed 10 charts while second- and third-year residents reviewed 20 charts. We evaluated charts for: (1) mammogram in past 2 years, (2) Pap test in past 3 years, (3) colon cancer screenings (ages 50 years or more), and (4) pneumonia vaccine (ages 65 years or more) based on recommendations.^10-12 We compared estimates to local (Dallas County),¹³ state (Texas),¹⁴ and Healthy People 2020 estimates/goals.¹⁵

During the intervention, preliminary results and an overview of recommendations were presented to physicians and staff. We collected surveys assessing knowledge and. Two questions were asked for each preventive service. Participants were asked, “what do you know about the process for a patient at our clinic to receive [insert preventive service] after a provider has ordered it?” Then, participants were asked “what do you think are some potential barriers to patients at our clinic getting a mammogram/Pap test/colonoscopy/pneumonia vaccine?”

Postintervention, a new random sample of records (n=100) were selected from patient visits the following year. Our postintervention sample size was smaller because data could only be collected by three residents, who used this project to meet research and scholarly activity requirements, instead of by all residents. We compared pre- and post results.

We used descriptive statistics to report age and receipt of preventive services. We developed themes to describe processes/barriers. We entered open-ended responses into a Microsoft Excel database, and identified key words related to processes and barriers, and we counted recurring words (ie, scheduling, language). We used McNemar tests to determine statistically significant differences pre-/postintervention. This project was reviewed and approved as QI by our institutional review board.

Preintervention, most women ages 40 years and older received mammograms (72%) and Pap tests (65%). Findings were lower than Healthy People 2020 goals (mammograms=81%; Pap tests=93%) and local (mammograms=75%; Pap tests=77%) estimates. Clinic estimates for mammograms (72%) were higher than statewide estimates (66%). Clinic estimates for Pap tests (65%) were lower than statewide estimates (76%). Among women ages >50 years, over half (56%) received a colon cancer screening, with 53% receiving up-to-date screening. Most (81%) women ages 65 years or more received a pneumonia vaccine, which was lower than the Healthy People 2020 goal (90%, Table 1).¹⁵

We found several inconsistencies when we measured processes for receiving services (Table 2). For mammograms, 34% reported that patients had the responsibility to set up the appointment, while 15% reported that patients were called for appointments. Similar inconsistencies were found for colonoscopy. For Pap testing, 41% reported that patients had to set up separate appointments, whereas 20% indicated that they could be done during already-scheduled appointments. Health care providers and staff reported communication barriers with patients about pneumonia vaccines. The most commonly reported processes were physicians ordering the vaccine (32%) and discussing the vaccine with patients (11%). Using descriptive data, fear and lack of knowledge were identified as top barriers. Scheduling difficulties were identified as barriers to completing mammograms (42%), colonoscopies (50%), and Pap tests (15%). Limited stock was a barrier for pneumonia vaccines (19%).

Despite not reaching statistical significance (P<.05), we found improvements for colon cancer screenings and pneumonia vaccines postintervention (Table 1). Mammograms decreased only slightly. Pneumonia vaccines met Healthy People 2020 targets (90%). We found a statistically significant increase (P=.0013) in the proportion of women who received Pap tests despite not meeting Healthy People 2020’s goal. Up-to-date colon cancer screenings (70.0%) were higher than Texas estimates (62.6%) and similar to Healthy People 2020 (70.5%) goals.

Our goal was to develop an activity for family medicine residents to learn how to conduct QI projects while improving women’s preventive health services. In accordance with recent QI practice, another focus was to shed light on clinic performance in providing women’s preventive services. We observed the following results.

Prior to completing this project, our clinic had poor performance on Pap tests and colon cancer screenings. Postintervention, our clinic met Healthy People 2020’s goal for pneumonia vaccines (90%) among women aged 65 years and older. Furthermore, we observed a significant increase in Pap testing (20.4% increase, Table 1) after intervention. Our results are similar to previous studies that found that Pap testing improved after the development of a women’s preventive health clinic for residents.^16,17 We observed a slight decrease in mammogram uptake (preintervention 72.2%; postintervention 69.1%). In addition to the smaller sample size, one possible explanation may be a difference the patients’ age. Preintervention, 20% of the sample was aged 40-49 years, versus 30% of the sample postintervention.

During our intervention with physicians and staff, several inconsistencies in the process for obtaining screenings and vaccines were reported. There was lack of agreement on the process of setting up appointments. This inconsistency may have led to scheduling difficulties being reported as a top barrier for all screenings. Patient-provider communication and system-level coordination have been identified as strategies for developing best practices and streamlining processes.^18,19

Strengths of this study include a large sample of patients’ records available to review and high participation from residents and faculty. Future QI projects should engage more staff and evaluate patient perceptions. A limitation was that the charts selected postintervention were not the same as preintervention. The patients whose charts were selected preintervention may not have been due, or may have completed a follow-up visit, when postintervention data were collected. Since our goal was to determine improvements on the clinic level, we do not believe this affected our results or the residents’ ability to learn from this activity. Another limitation was that we only conducted follow-up data once. Due to a lack of resources and changes in QI project priorities (eg, focus on diabetes 1 year later), we were unable to conduct additional follow-ups. Future QI activities should include longitudinal approaches.

This experiential learning activity equipped residents with tools necessary for implementing QI projects after residency. In conjunction with hands-on education in QI methodology, residents also gained experience working in interprofessional teams, applying guidelines to practice, and utilizing EMRs to better understand patient health. We believe this experiential learning activity provided a valuable framework for residents to gain hands-on QI experience while impacting health outcomes among their patients. Our focus on women’s preventive services can be modified to meet other health metrics.