Culture of Safety Quality Improvement Project: Longitudinal AHRQ Survey Results From a Family Medicine Residency Program

Patient safety has come to the forefront since the publication of the Institute of Medicine's report To Err is Human: Building a Safer Health System in 1999.¹ Tracking patient safety and clinic culture allows for continual improvement.^2–6 The Family Medicine Center (FMC) used the Agency for Healthcare Research and Quality (AHRQ) Culture of Safety Survey to measure the clinic’s patient safety culture by assessing physician and staff perceptions of clinic patient safety and quality outcomes. Teamwork is a core feature of successful health care quality improvement, yet there is little evidence on best practices for implementation and support of teamwork initiatives that directly impact safety and quality.^2,7

The Family Medicine Center (FMC) clinic is a recognized patient-centered medical home (PCMH) for a community-based family medicine residency program. Senior leadership within the FMC sought to improve the culture of safety and quality through targeted, evidence-based systems improvement efforts. These efforts included error reporting mechanisms, regular safety meetings by department leaders, and the creation of a resident safety chief position.^5,8 The goal of these initiatives has been to maintain high-quality health care despite regular resident turnover.⁹ The FMC utilized the AHRQ Culture of Safety Survey as a benchmark for internal and external comparison. The AHRQ’s Medical Office Survey on Patient Safety Culture is broken down into several categories such as “Communication About Error” and “Office Processes and Standardization” with additional questions on patient safety issues and exchanging patient information with other entities, which are respectively referred to as the “A” and “B” series of questions in the survey tool. The FMC has been working to balance initiatives to enhance patient care and graduate medical education experience with staff’s fatigue associated with frequent changes to processes, workflow, and reporting requirements in a fast-paced clinic.^10,11

The purpose of this study was to longitudinally evaluate the net impact on patient safety metrics when the organization’s leadership set its improvement as a company-wide goal. As a community family medicine residency program, it is crucial to include a focus on patient safety outcomes as part of our mandate to graduate practice-ready physicians.

This study was conducted at the FMC with responses provided by physicians, nurses, pharmacists, and other clinic staff members. Printed surveys were made available throughout the facility.

The medical office survey, an expansion of AHRQ’s Hospital Survey on Patient Safety Culture, was designed to measure an office’s patient safety culture. The survey consisted of 26 questions in five different series and an additional 22 survey questions about organizational culture. Paper surveys were made available to the approximately 80 FMC employees in 2015, 2017, and 2019. The printed surveys were made available throughout the clinic and administrative spaces. We used the Kruskal-Wallace test to analyze the data with an α of 0.05. This project was approved by the Lutheran Health Network Institutional Review Board (LHN# 18-511).

A total of 112 surveys were completed by FMC employees. Statistically significant improvements in patient centeredness, effectiveness, timeliness, efficiency, equity, and overall patient safety were seen from baseline results to final results. In the 2015 baseline assessment 33 participants completed the survey, and in 2017 and 2019 48 and 44 participants, respectively, submitted the survey.

In the “A” series question about access to care within 48 hours, 55% in 2015, 65% in 2017, and 91% in 2019 of employees had a positive response. The FMC leadership set the goal of having a mean value ≥5 for the “A” series questions (Figure 1). We met these goals for access to health care providers, chart access, and properly working medical equipment. This study demonstrated significant improvement in five the of the nine areas, while only four the nine areas met the leadership’s target.

Tables 1 and 2 represent the results of the next 11 areas of the AHRQ survey. The bold font entries in the tables show where the FMC leadership’s goal was achieved. There were instances where significant changes were observed even if the leadership’s goal wasn’t achieved. 

In the “B” series of questions, values of ≤2 were desired for negative questions, and ≤4 for positive questions. Results were below the national average for interfacing with various external medical settings. From 2015 to 2019 we saw significant improvement in teamwork, with three out of four responding “very good” to the prompt “when someone in this office gets really busy, others help out.” We also saw significant improvement in employee perceptions of patient care tracking and follow-up. The questions concerning organization learning showed there was a significant decrease in positive responses.

Figure 2 shows the results of employee’s rating their clinic on six different areas. The leadership set a score of 4 out of 5 or more as the target for each category. All six areas showed a significant increase in the employee’s rating of their office.

This study had several limitations. First, we had a small number of participants, and some of the participants did not complete all components of the survey. Some parts of the survey inexplicably had as few as five participants respond (Table 1, 2017 data set). Missing data and nonresponse bias may be present, and those with strong negative or positive feelings may have been more likely to respond. Additionally, there were some survey questions with interpretation variances leading to inconsistent results from similar questions. Also, we were unable to gather specific baseline characteristics regarding employees’ role in the clinic and years of practice at the clinic as many of the surveys were left blank for these questions. Therefore, the anonymous paper-based survey format did not allow for intra-cohort analysis between physicians, nurses and other staff participants.  Finally, since we followed the AHRQ representative’s advice on how to make the surveys available to the employees we were not able to determine the actual response rate. 

We improved patients’ access to care at the FMC, interclinical communication, and team and interprofessional interactions. Overall perception of patient safety and quality has shown statistically significant improvement in four out of four responses. However, multiple areas still need improvement to reach benchmarks and the national average for top box responses. The organizational learning survey indicated a discordance between questions F1 and F5 (change management perceptions). These discordances could be attributed to inconsistent interpretation of survey questions or could be due to safety inertia, where easily modified issues are addressed but larger, global issues are not addressed. Another area where we noted a discordance was between questions D8 and D12. Providers and staff do not reportedly speak openly about problems in the office in the first question. However, staff are willing to report mistakes they observe in the office in the second question. The wording of “talk openly about office problems” allows for variations in interpretation, eg, venting dissatisfaction openly versus discussing concerns in appropriate settings.

In addition to the learning opportunities for our family medicine residents, the quality improvement projects focused on identified workflow inefficiencies such as implementation of a new electronic medical record and telephone system, along with robust screening protocols. Key activities that affected the described changes included daily team huddles to foster communication, clinical staff uniforms to build a sense of community, development of an FMC committee with diverse participants, and the creation of a resident quality safety chief to identify and correct errors. Finally, to the best of our knowledge there are no previous publications that used AHRQ’s Culture of Safety Survey for longitudinal patient care benchmarks as part of a family medicine residency’s physician training. 

The results from our study helped compare patient safety culture survey results from the FMC with other medical offices. Our clinic has progressed significantly from 2015 to 2019, but there is room for continued improvement through more focused projects.