Perceived Access and Appropriateness: Comparison of Teaching and Resident Family Physicians’ Patients

Teaching clinics train the next generation of family physicians in community-based care that is comprehensive, high quality, and timely, with continuity assured by a most responsible health care provider and care team. These are important pillars of the patient medical home (PMH) model promoted by the College of Family Physicians of Canada, ¹ which accredits family medicine residency programs. In Canada, all family medicine teaching clinics are affiliated with a medical school and offer academic training to family medicine residents, medical students, and other health care professionals.

The 2-year family medicine residency curriculum integrates outpatient family practice with rotations in hospital (eg, obstetrics or internal medicine) and formal teaching activities. During the 4-week hospital rotations, these physicians in training, called residents, spend 1 day per week at the teaching clinic to provide care to their patients. Residents assume full primary care responsibility for a panel of patients right from their entry into the program, ² enabling them to experience continuity of care. ^{3, 4} Residents are supervised by teaching family physicians, also called staff, who typically also have a panel of patients at the teaching clinic. With this role model residents are learning what it is like to work in a community-based practice versus hospital rotations. These residents are working in an environment that supports the patient’s medical home. ¹ Since the residency program is short and resident attendance at the clinic is irregular, it is a challenge to achieve both continuity of care and timely access. ^{5, 6} Consequently, many teaching clinics implement advanced access scheduling systems that attempt to meet goals of both timely access and continuity of care. ^7-9 However, it is not clear whether residents’ patients experience the same timely access as staff patients.

We examined how access is perceived by patients in teaching clinics with two main objectives: (1) to compare the experience of timely access by patients of residents versus staff, and (2) to determine if there was a difference between both groups in reported appropriateness and patient-centeredness of the visit. To our knowledge, no studies have been conducted in teaching clinics where residents are responsible for a panel of patients and use advanced access.

This was a cross-sectional survey of patients visiting nine family medicine teaching clinics with advanced access in place, at the University of Montreal and McGill University in Quebec (Canada) in 2018. Ethical approval was received from Centre Intégré de Santé et Services Sociaux de Laval, as well as from the ethics boards associated with participating clinics (Number 2017-2018 / 04-01-E).

To help us conceptualize access to health care and develop our survey instrument, we adopted Levesque’s Patient Centered Accessibility Framework (2013), ^{10, 11} which synthesizes access into five dimensions. This framework is well suited to primary health care and describes characteristics of both the organization and patients that interact to produce appropriate access to care. This article presents results related to two of the five dimensions: availability and accommodation, which refers to the ease of obtaining services in a timely manner, and appropriateness, which refers to how well the services provided meet patient needs.

Patients attending the clinic for their own care, either scheduled or walk-in (urgent), were eligible if they were: (1) 18 years of age or over, (2) registered with a clinician at the clinic, and (3) able to read and answer a questionnaire in French or English on their own. Patients were excluded if they were on their first visit to the clinic or had already completed the questionnaire on a previous visit.

The questionnaires were developed by selecting specific questions from validated instruments that mapped onto concepts in the Accessibility Framework. ^12-16 Questions were adapted to our care context and translated into French. A patient partner verified the relevance and the clarity of the questions. The self-administered and anonymous questionnaire package consisted of two parts: (1) a longer previsit questionnaire to be completed while waiting for their consultation; and (2) a short postvisit questionnaire completed following their consultation.

The previsit questionnaire of 33 questions elicited patient experience with access to care in the teaching clinic, both usual and for that specific day’s experience access for scheduled or urgent care. There were seven questions on clinic approachability related to patient health care needs, nine on clinic availability and accommodation in response to patient health care seeking, and two on economic affordability of using health care. The questionnaire also elicited affiliation with a professional (three questions), the reason for the appointment, and sociodemographic characteristics (nine questions). Finally, two open-ended questions asked patients to indicate what needed improvement and what was appreciated in the delivery of health care services.

The short postvisit questionnaire assessed appropriateness dimensions of the patients’ experience in their medical appointment. Validated questions were selected and adapted slightly to reflect patient-centered communication (Q1 to Q5), how well needs had been met (Q8),^{14, 17} patient enablement (Q7), ¹⁸ and one question about the visit duration, taking into account the time spent on supervision which may lengthen the visit (Q6).

The researchers trained reception staff and provided a standardized script to recruit patients. Reception staff noted on the questionnaire the type of professional (resident, staff, nurse practitioner) being seen that day. Pre and postvisit questionnaires were given to patients upon their arrival at the clinic. Patients could refuse to participate in the study either explicitly or by leaving blank questionnaires in the sealed box in the waiting room.

For individual questions, differences between residents and staff patients were tested using a χ² test. Analysis of the previsit questions compared experience by type of most responsible physician (staff vs resident) reported by patients; postvisit questionnaire analysis compared type of physician seen noted by the reception staff, who also differentiated between first-year and second-year residents. Because of the multiple group comparisons between first-year and second-year residents and staff, we applied Bonferonni correction ¹⁹ to the significance level and set at 0.017. We performed analyses using SPSS Statistics 26 (IBM Corp, Version 26.0).

Seven of the nine clinics were located in urban areas, each with between 4,400 and 29,435 registered patients (median=11,921). In the five clinics that kept careful recruitment logs, the refusal rate ranged from 4% to 10%. A total of 1,979 patients participated, with 201 to 239 completed questionnaires per participating clinic. The analytic sample for this study consists of the 1,676 patients who answered the previsit questionnaire and identified their primary care provider as either a resident (21%, n=409) or staff (64%, n=1,267). We excluded from previsit analysis 169 patients (9%) whose identified primary care provider was a nurse practitioner, 33 (2%) who were uncertain, and 101 (5%) who did not answer the question.

A total of 1,651 of 1,979 (83%) responded to the postvisit questionnaires, of which 1,387 (1,387/1,651; 84%) had complete information on the physician status (especially resident training level) and were used in the secondary comparison by level of experience of treating physician.

As shown in Table 1, the only statistically significant sociodemographic difference between patients of residents and staff is occupation, where resident patients are less likely to be retired or students. Self-reported overall health status was similar. The length of affiliation with the clinic did not differ between groups: 43% (n=172) of resident patients had been enrolled in the clinic for more than 5 years, compared to 48 % (n=601) of staff patients.

In both groups most patients (69%) had an appointment with their own primary care physician. In each group around 83% of patients were there for a routine or follow-up visit, and 18% for a minor urgent problem (Table 2).

Table 2 shows there is no difference between resident and staff patients in the wait for this appointment; in both groups 26% waited 1 day or less (including urgent care) and more than one-third had waited 14 or more days. When asked to rate the usual wait time for an appointment with their usual responsible physician, staff patients were more likely than resident patients to rate the wait time as very good or excellent (46% vs 35%) whereas resident patients were more likely to rate the wait time as poor or fair (23% vs 19%; P=.001). There was also a slight difference between groups in the ease of obtaining an appointment sooner than the usual wait time frame, with resident patients being more likely to report difficulty getting an appointment sooner than staff patients (41% vs. 34%; P=.47).

To better capture potential problematic access, we asked patients if they consulted other providers in the past year and if so, for what reasons (Table 3). Although the overall proportion seeking urgent care from another clinic was similar in both groups (23% vs 24%), resident patients were more likely than staff patients to make two or more visits to another clinic: 73% compared to 56% (P=.01). A much higher proportion of resident patients than staff patients invoked lack of physician availability or long wait for next appointment as the reason, although the difference did not reach statistical significance. The proportion of patients who reported seeking care at the hospital emergency department was higher for resident patients (35%) compared to staff patients (29%; P=.01).

Patient perceptions of appropriateness dimensions are reported in Table 4. The results reveal a tendency for staff patients to rate dimensions of appropriateness more highly than resident patients. All three statistically significant differences were in favor of staff patients. The results show significant differences between the two groups on the patient-centered communication dimension, with fewer resident patients believing that their physician had adequately explained their problem or condition (67% vs 76%) or questionned whether the recommended treatment or advice was realistic compared to staff (72% vs 78%). Surprisingly, only 1% of the resident patients considered their consultation too long, despite the added time for supervision.

This finding led us to further secondary analysis of appropriateness by training level of the resident (Table 5). Results showed that second-year (senior) residents had a tendency to have better postvisit results than first-year (junior) residents and junior residents scored statistically significantly lower than staff on four out of eight questions. Patients of junior residents were less likely (53%) to feel that the visit completely meets their needs (Q6) compared to senior residents (66%) and staff (69%).

Our study compared the perceptions of resident and staff patients on dimensions of access to care and on the experienced appropriateness of care. As expected, residents’ patients do rate more poorly the usual wait time for an appointment, and they also more frequently seek care at other clinics or the hospital emergency room. However, despite advanced access, staff patients wait as long as resident patients for an appointment, and more than half rate the usual wait time as poor, fair, or good, suggesting that timeliness is also an issue for staff patients. Finally, our findings suggest that training impacts positively on visit appropriateness as reported by patients, with senior residents often achieving higher scores than junior residents and not statistically different from those reported by staff patients.

Our study confirms the expectation that resident patients experience more difficult access than staff patients. These results can be explained, in part, by the irregular presence of residents in the clinic. When they are on off-site rotations, they are only present at the clinic to see their patients about 1 day per week.

Another explanation for patients still seeking care elsewhere is not being comfortable with the assigned professional or an imbalance between supply (service offered either by the resident or by the clinic's team of professionals) and demand (patients' need for service or wants for their services). These findings confirm previous data on the challenge of implementing advanced access, based on resident availability and training needs. ^{6, 20} We assume, with our clinical experience in these teaching clinics, that a team-based approach is compensating, in part, for resident reduced availability. However, we did not measure this type of care in the questionnaire. While team-based care is a pillar of advanced access, PMH 2019 also recognizes “that a patient will not be able to see their personal family physician at every visit.” ¹ To propose team-based care ^{21, 22} to ensure timely access for more urgent needs during times when the responsible professional is not available is part of the solution when facing the challenge of balancing continuity and timeliness of care. This challenge is not specific to family medicine residents; it is also shared by other disciplines. ³

Our results reveal that timeliness of access is an issue for staff as well, despite the implementation of advanced access. Staff patients wait as long as resident patients to receive care, and a significant proportion of them have consulted elsewhere either in another clinic or in the emergency room, in the past year. This may be particular to the Quebec context, where policies requiring a versatile practice limit the availability of staff. All new family physicians have an obligation to dedicate approximately 20% of their time to providing clinical activities such as obstetric care, palliative care, in-patient and long-term care according to regional family medicine workforce plans ²³; staff additionally have teaching responsibilities. As developers of advanced access scheduling have suggested, “continuity is difficult to achieve for providers who work in continuing care less than 6 out of 10 half days per work week.” ⁶ Moreover, one of the selection criteria for the clinics was to have implemented advanced access. However, experience with advanced access and the length of time residents had been using it varied greatly between clinics. Today, we know that if the pillars of the model are not monitored, it is easy to get overwhelmed and reach an imbalance between supply and demand. ^{24, 25} These facts could probably explain the lack of access, given the combination of multiple clinical and pedagogical duties (direct and indirect supervision of students, courses, workshops, journal clubs), or insufficient collaborative practice, which we did not explore in this study.

Despite the brevity of the family medicine residency program, our results suggest progress in appropriateness by level of resident training and especially in patient-centered communication, another important pillar of the PMH. ¹ It is good news that such a short curriculum can train residents to develop a patient-centered approach to care.

To our knowledge, our study is the first to report comparison in patients’ perception of timely access and appropriateness between patients of residents and patients of staff, in an academic primary care setting. Results in the few studies concerning postgraduate residents are difficult to compare. ^26-33 Either they did not compare patients’ opinion with staff ^{27, 28, 32} or were not in a primary care setting. ^{29, 31-33} Moreover, residents in some study settings provide episodic care to patients of staff, whereas our residents assume full responsibility for a panel of patients over 2 years. This may also account for the gradient in visit appropriateness observed between first and second-year residents. We observed a similar gradient in a paediatric study, where patients of junior residents received lower satisfaction scores than senior residents, who in turn were rated lower than staff. ²⁹ Another study in a pediatric setting had a similar gradient with no comparison with staff. ³² Two studies in internal medicine, however, report divergent results. ^{30, 31} Our study also elicited patient experience of access dimensions while some studies asked about satisfaction. ^{26, 29, 31} Experience-based measures are considered more comparable across respondents because the patient reports what happened whereas the evaluator judges whether a satisfactory benchmark was achieved, as reflected in our reporting of percentage achieving only the best response option (eg, “completely”). This is consistent with the recommendation in satisfaction studies that any patient rating below “excellent” implies room for improvement. ^{34, 35}

Our analysis was based on a large sample of patients in multiple settings. Our results are generalizable to teaching clinics in our context because the sample includes both urban and rural areas and response rates over 90%, and an 83% response rate to the postvisit questionnaire. This large sample size gives adequate statistical power to detect even small differences between staff and resident patients. The similarities between these two groups give confidence that our comparisons are not biased by differences in patient characteristics. However, a selection bias for dimensions of access is possible because we only surveyed patients who reach the primary care clinic, so those with the most access difficulty are less likely to be sampled. But this bias is not likely to differ by type of professional, making our comparisons internally valid. Our results do show that resident patients are more likely than staff patients to seek care elsewhere more often for minor urgencies, which may explain the few statistically significant differences on the accessibility indicators.

We acknowledge some limitations. Although we used previously validated, patient-reported experience measures that had been shown to be equivalent in French and English, ³⁶ we sometimes selected a subset of items from a construct to reduce the response burden and also adapted the statements for relevance to our primary care context. However, the selection and editing was based on our intimate knowledge of individual performance and discriminability of these items within the construct. ³⁷ This study was conducted in one province in Canada. The structure of family medicine practices is similar, but also different across the country with variations in their funding, solo versus group practice, and the adoption of team-based care.

Our study reveals that in teaching clinics, residents’ patients do experience more difficulties with timely access, but staff also face challenges in providing services in a timely manner. The particular challenges of the residency program and the burden of balancing clinical duties and teaching activities for staff, suggest the need to develop robust strategies to improve timely access for both staff and residents in teaching clinics. Finally, a novel and promising finding of this study is the observed improvement in appropriateness between visits to first- and second-year residents that affirms the value of training efforts in patient-centered practice.