Incorporating the Patient Voice Into Practice Improvement:

Many primary care clinics rely on providers and staff to generate quality improvement (QI) ideas without explicitly including patients.1 However, they may miss important areas for improvement. Several clinics in Australia and the United Kingdom have harnessed narrative-based patient experiences to drive practice transformation,2-6 and report significant differences between provider and patient QI priorities,2 and positive associations between incorporating the patient voice and QI success.4,6 In a similar parallel, many clinics in the United States have adopted patient advisory councils (PACs) as a mechanism to give patients a seat at the table.7-9

At the same time, there is growing recognition of the importance of a robust QI curriculum within medical education.10,11 Notably, scholars have found that active participation of medical trainees in QI, as opposed to passive classroom-based didactics, produces more meaningful learning about QI principles and methods.12

While the areas of patient-centered practice improvement and QI within medical education have expanded, little is known regarding the inclusion of patients in QI with the help of medical trainees, and whether provider perceptions are consistent with patients’ concerns. Our pilot sought to address this gap in research. We identified QI opportunities using a medical student-driven pilot of QI design that incorporates the patient voice and explored provider/staff perceptions of patient perspectives.

Stanford Coordinated Care (SCC) is an intensive primary care (IPC) practice that delivers team-based, interprofessional care to Stanford University employees and their dependents who are in the top 20% of high-risk, high-cost patients.13

To determine patient selection criteria, a medical student asked SCC’s care coordinators (ie, staff who interacted with patients most frequently) for their recommendations of the types of patients for whom the clinic could improve care and would be valuable for SCC’s QI process. This resulted in the following selection criteria: patients who showed enthusiasm for improving their health, but for whom the practice struggled to engage in following through with their action plans. Of the 16 patients identified by their care coordinators and invited to participate, 8 agreed to interview.

We developed a narrative-based3,14 interview guide, iteratively edited by three physicians (S.L., K.V., N.C.), with open-ended questions regarding: (1) opportunities for QI (eg, “Are there any areas that are lacking at SCC based on prior health care settings you’ve experienced?” “Any suggestions for how SCC can improve its quality of care?” “How has the care at SCC been not-so-helpful?”), and (2) what patients valued most about their care (eg, “How has the care you have received at SCC most helped you feel better?”). Participants were provided a $10 incentive. A medical student (K.H.) trained in QI methodology15, 16 conducted semistructured, in-depth (60 minutes) interviews. All patients provided written informed consent for study participation.

We derived a codebook that characterized common elements of clinic quality by conducting directed2,5,17 content analysis with a deductive coding approach.18 We used Dedoose 4.3 software (Los Angeles, CA: SocioCultural Research Consultants, LLC) to facilitate content analysis of interview transcripts. Two students (K.H., V.C.) independently coded transcript excerpts (total excerpts: N=231; pilot 1: n=21 excerpts, k=0.71; pilot 2: n=60 excerpts, k=0.75; final: n=150 excerpts, k=0.84), reached consensus for disagreements, and revised the codebook iteratively (final codebook: 11 themes). Reviewers (K.H., V.C.) independently assigned each excerpt a weight that indicated whether it expressed a QI opportunity or what patients value about the practice (pilot: n=21 excerpts, k=0.95; final: n=210 excerpts, k=0.95).

We surveyed 11 providers/staff on their perceptions of patient concerns by providing them an abridged codebook (ie, code names and definitions without specific patient examples) and frequencies of mentions for the top five themes regarding QI opportunities (eg, #1 had 10 mentions, #2 had 8 mentions, etc), and what patients value about their care. Each provider/staff selected 5 of the 11 themes and placed them in rank order based on their perception of patient responses. We scored surveys by assigning each provider/staff response a weight based on their perception of frequency of mentions, then determined the average provider/staff perceived frequency for each theme and aggregated surveys into one ranked list. We explored differences between the provider/staff ranked list of themes and patient list using a Likert-type scale19 (very concordant if ≤1 ranking difference; somewhat concordant if >1 and ≤3 ranking difference; or not concordant if >3 ranking difference). The Stanford University Institutional Review Board determined the study protocols were exempt.

Table 1 describes the final codebook containing 11 thematic elements of care quality with representative quotations. Patients most frequently identified the following themes as QI opportunities: relationship (frequency=10), specialty care (frequency=7), convenience (frequency=7), sustainability (frequency=5), and goal follow-up (frequency=5).

Figure 1 illustrates the themes for QI opportunities by frequency of mention and average perceived frequency of mention for patients and providers, respectively. While patients most frequently mentioned (frequency=10; rank=1) relationship (ie, feeling of personal connection with providers/staff) as a QI opportunity, the aggregate provider/staff list ranked this theme second to last (average perceived frequency=0.5; rank=10). Goal setting and follow-up was also one of patients’ top five (frequency=5; rank=3) themes for QI opportunities, but missing in the provider/staff top five list (average perceived frequency=1.9; rank=7).

Our work provides a medical student-driven pilot of QI design that may benefit and facilitate learning among multiple stakeholders (Figure 2). By explicitly incorporating the patient voice, we found new opportunities for practice improvement, such as strengthening personal connection and optimizing goal setting with patients.

This pilot contributes to extant literature on patient-centered QI assessments2,3,5 and QI within medical education.10-12 By leveraging the role of medical students, we were able to utilize a low-cost ($80 spent on interview incentives) pilot with limited disruption to clinic workflow, that mirror the provider-patient differences seen by large-scale (>$9,000 per site)5 patient-centered QI studies conducted internationally.2

One study limitation is that our pilot was conducted at a single IPC practice with only eight patients and two physicians, and may not be generalizable, particularly given the complexity of a medical system with diverse patients, multidisciplinary staff, and medical students. The integration of medical students into patient-centered QI processes requires thoughtful strategies in order to facilitate productive information sharing among stakeholders and meaningful clinic improvement. This study provides a preliminary model of how medical students might participate in clinics to incorporate patients’ perspectives into practice improvement. Next, our pilot focused on patients who demonstrated interest in improving their health, which may not be generalizable, for example, to those in a precontemplation stage of health behavior change.20 While there is value in seeking ideas for patient selection criteria from stakeholders who interact with patients most frequently (eg, care coordinators), future studies may consider using random sampling. Additionally, our qualitative analysis relied on the assumption that frequency of mentions is correlated with perception of importance, which may not be true in all settings. Finally, we relied on a crude comparison of provider perceptions with actual patient responses (eg, providers had a limited number of options whereas patients had an infinite number of options). Future work may consider validating a measure of provider-patient concordance.

Future studies may consider expansion of this work, for example having teamlets of medical, nursing, pharmacy, and social work trainees collaborate to incorporate the patient voice into QI processes or incorporation of active roles for trainees within existing PACs. This study may serve as a springboard for active learning and interprofessional collaboration early on in QI education, providing the potential to benefit patients, professionals, and learners alike.