Identifying and Remediating Quality Issues at Clinical Teaching Sites: A CERA Clerkship Directors Survey

While medical schools increasingly rely on community-based teaching sites and preceptors for clinical training, placing medical students in high-quality community-based clinical training sites is becoming more difficult.1 This difficulty stems from an increase in matriculating students at US medical schools, the growing emphasis on clinical learning in ambulatory-based settings, and other health profession students also needing clinical experiences. Thus, clerkship directors face increasing competition for a limited number of clinical teaching sites.1 It is likely that the shortage of preceptors will significantly impact outpatient learning opportunities for the foreseeable future.

Beyond seeking to establish a greater number of sites for community-based teaching, national organizations like the Alliance for Clinical Education have also made recommendations to enhance the quality of learning experiences.2 Characteristics of quality clinical teaching sites, such as allowing autonomy and providing feedback, have been described in the literature.3,4 The Liaison Committee on Medical Education (LCME) requires that medical schools provide ongoing faculty development for preceptors so students receive excellent training.4

The academic community agrees on the need for evidence-based faculty development and training that provides a supportive learning environment, maintains enthusiasm for teaching, and promotes understanding and implementation of curriculum plans for community-based faculty.5-8

Since 2015 the Society of Teachers of Family Medicine (STFM) has established a Preceptor Expansion Initiative with a specific action plan addressing multiple tactics to identify, develop, and disseminate resources for community-based preceptors. The overarching program goals of the initiative are to decrease the percentage of primary care clerkship directors who report difficulty finding clinical preceptor sites, and increase the percentage of students completing clerkships at high-functioning sites.9 Though the literature describes a search for more precepting sites and for higher quality learning experiences, it has not examined how clerkship directors monitor teaching quality at community-based clinical sites, and how best to achieve remediation when performance issues are identified. Remediation of struggling clinical educators has been discussed in the graduate medical education literature, with one group of authors providing guidance on making the diagnosis through collecting data, developing an intervention, and following up to monitor improvement.10 This group divides quality challenges into four categories: knowledge, attitudes/behaviors, skills, and system problems, and suggests that clinicians can improve their performance once they are made aware of their deficits.

The Alliance for Clinical Education has released expectations for clerkship directors, indicating that a minimum of a 50% effort of a full-time equivalent position, as well as a full-time administrative assistant, are necessary for successful oversight of a clerkship. Specific clerkship director responsibilities include reporting on the sufficiency and comparability of clerkship experiences at all teaching sites, and orienting teachers which requires the time and resources to visit other sites where teaching occurs.11 To our knowledge, no guidance on the best methods for monitoring the quality of the clinical experience at sites, or best practices on remediating quality issues have been described in the literature.

This CERA survey explored how family medicine clerkship directors identify community-based clinical sites with performance challenges, types of challenges, and whether a remediation option exists for struggling clinical sites or preceptors. The central hypothesis was that clerkship directors who struggle to establish a sufficient number of community-based sites or preceptors will be less likely to remove, remediate, or even track underperforming sites or preceptors. An additional hypothesis was that informal methods for identifying and remediating teaching sites and preceptors are likely used in most cases. We anticipate that clerkship directors feel they can benefit from additional institutional support to track and remediate clinical sites and preceptors with performance issues. This survey identifies a gap in the literature by understanding the types of performance issues affecting clinical sites and community-based preceptors, and describes trends in remediation opportunities.

Data were gathered and analyzed as part of the 2018 Council of Academic Family Medicine’s (CAFM) Educational Research Alliance (CERA) survey of family medicine clerkship directors. CAFM is a joint initiative of four major academic family medicine organizations: STFM, North American Primary Care Research Group, Association of Departments of Family Medicine, and Association of Family Medicine Residency Directors. The cross-sectional survey is distributed annually to clerkship directors or their designee at qualifying medical schools accredited by LCME or Committee on Accreditation of Canadian Medical Schools within the United States and Canada. In 2018, 125 US and 16 Canadian individuals were identified as family medicine educators directing family medicine or primary care clerkships. CAFM members were invited to propose survey questions for inclusion in the CERA survey. Approved projects were assigned a CERA research mentor to help refine questions. Survey questions included multiple-choice questions describing clerkship and medical school characteristics, a listing of challenging issues from which respondents could choose multiple answers, and a 5-point Likert scale question regarding perceived difficulty in identifying clinical sites. Final survey questions were modified following review by the CERA management team and authors.

The survey was distributed via email between June 28, 2018 and August 4, 2018. Two US emails were undeliverable, reducing the final sample size to 139. Invitations to participate included a personalized greeting and letter signed by the presidents of each of the four sponsoring organizations with a link to the online SurveyMonkey survey. During survey administration, 17 clerkship director changes were identified; 14 through contact with the survey director and three in the survey. All new clerkship directors were invited to participate. Nonrespondents received weekly requests and were contacted through personal email to verify their status as clerkship directors, confirm email addresses, and encourage participation. The American Academy of Family Physicians Institutional Review Board approved the study.

The deidentified data set received from CERA was imported into SPSS statistical software, version 23, (IBM SPSS Inc, Armonk, NY) for data analysis. We conducted descriptive and bivariate statistical analysis. Variables were converted to binary variables for analysis. Comparisons between paying and not paying preceptors (paying no preceptors vs paying some or all), in difficulty identifying sites (very difficult/difficult vs neutral/easy/very easy) and the number of underperforming sites removed in the last 24 months (0/1-2/>3) were performed using Pearson χ2 test or Fisher exact test if a cell contained a quantity less than 5. These variables were compared to one another, as well as to clerkship director and medical school characteristics, most challenging issues with sites, existence of a remediation system, and most-needed remediation support. Significance was set at P<.05. All percentages are based on the number of valid responses; missing values were omitted.

A total of 99 out of 144 clerkship directors (68.8%) responded to the survey. Of the 96 respondents who answered the question, 94 (97.9%) report having less than 50% of their time protected for clerkship director responsibilities. Regarding clerkship design, 71 out of 97 (73.2%) respondents report using a block clerkship design only. When asked about providing payment as outlined in Table 1, 57 out of 95 respondents (60%) reported paying 0% of their community-based preceptors, while 28 (29.5%) pay at least 76% of their community preceptors. Table 1 lists characteristics of clerkship directors, clerkship structures, and medical schools reported in the survey.

Ninety-four clerkship directors rated their experiences identifying clinical teaching sites ranging from easy to very difficult, with 77 (81.9%) describing the process as difficult or very difficult. When asked about removing clinical teaching sites or teachers or limiting their roles in response to performance or quality concerns, 53 of 94 (56.4%) reported eliminating one to two preceptors or sites over the last 24 months, while 14 (14.9%) reported removal of three or more preceptors or sites over the last 24 months. Fifty-two of 92 (56.5%) clerkship directors reported their institutions do not have systems in place to assist in the remediation of teachers or clinical sites with performance issues. A lower proportion of sites where a system is in place for remediation reported it being difficult to identify clinical sites (70.0% vs 92.2%, respectively, P=.011). Having a remediation system in place was also associated with less removal of sites (2.5% removed three or more sites vs 25% removed three or more sites, respectively, P=.005).

Table 2 lists the most challenging performance or quality concerns clerkship directors face with clinical sites or preceptors, stratified by whether preceptors receive payment. The most common was late or insufficient evaluations (43, 43.4%), followed by too many learners on site at one time (36, 36.4%), and students not being allowed to adequately participate in patient care (35, 35.4%). A statistical relationship was found between poor student access to the EHR and preceptors not being paid (P=.011). Additionally, lack of payment toward preceptors was significantly linked to perceived difficulty of identifying clinical sites, and not having a system in place for remediation (P=.032, P=.012).

Student evaluations provided clerkship directors the majority of data when it came to identifying challenges with either preceptors or clinical sites (77 of 95, 81.1%). However, the most critical information about site issues was obtained somewhat more evenly between student evaluations (46 of 94, 48.9%) and informal student reporting (30 of 94, 31.9%; Table 3). When asked what kind of additional institutional support is most needed to assist in remediation, 34 of the 93 (36.6%) clerkship directors who responded selected preceptor development activities offered from the school, 18 (19.4%) selected preceptor or clinical site stipends, and 13 (14%) selected additional clerkship director time (Table 4). There was a significant relationship between ease/difficulty of identifying clinical sites and paying preceptors (P=.032).

No significant relationship was found between ease/difficulty in identifying sites and the following measures: protected time as a clerkship director, number of students in each class at the school, clerkship design, or clerkship length. Similarly, a relationship was not found when comparing schools that removed sites or teachers in last 24 months and the variables of protected time as clerkship director, number of students in each class, clerkship design, or clerkship length (number of weeks).

To our knowledge, this study is the first to describe how clerkship directors identify concerns about quality of clinical sites, the types of such concerns, whether these concerns result in the removal or limited use of teaching sites, and whether there is support for remediating preceptors or sites with performance issues. By providing more support, clerkship sites could be saved, rather than removing them or limiting their teaching roles, requiring less recruiting of new sites in the future. These data provide guidance for medical schools and educational leaders supporting and developing community-based preceptors as educators, in order to provide an adequate pool of high-quality preceptors for students.

In terms of difficulty identifying clinical sites, fewer sites with remediation systems in place reported difficulty in identifying sites. Additionally, having a remediation system in place was also associated with less removal of sites. These clerkship directors may be making the investment in improving sites for retention, thus decreasing the need to identify new sites. It is also possible that these clerkships are more attractive to community-based preceptors when support is available to improve teaching. Providing clerkship directors protected time, class size, and clerkship length or design did not impact the ease of identifying clinical sites or likelihood of removing clinical sites.

We have also found that clerkship directors are relying on student evaluations and informal reporting to identify performance issues at clinical sites. Through the existing survey format it was not possible to determine which types of monitoring clerkship directors utilize. It is possible that clerkship directors do not have time to perform site visits or engage in formal conversations with clinical sites, despite wanting to identify issues through these methods. That site visits were not a large source for useful quality data in our survey may be a result of these visits not occurring, or it could be that site visits are not a useful tool for monitoring the quality of clinical training sites. This may help clerkship directors to advocate for additional protected time, or institutional support to develop and institute formal monitoring strategies, and to provide remediation support where appropriate. At a time when we need to both secure an adequate number of teaching sites, as well as maintain teaching quality, an appropriate monitoring program and resources for quality improvement are essential. Our findings demonstrate a correlation between paying preceptors and ease of identifying teaching sites. Paying preceptors was also related to having a system in place for remediation. Providing financial incentives may allow schools to expect higher quality and encourage remediation when necessary. There may also be a relationship between paying preceptors and valuing their contributions, leading to a culture where recognition, support for, and development of teaching skills are offered.

The growing need for community-based faculty to take part in medical education is one cause of the current preceptor shortage.2 Medical schools are increasingly reliant on community-based faculty to offer clinical experiences not only in family medicine clerkships, but also in other specialties such as internal medicine, pediatrics, and obstetrics and gynecology. These same preceptors may also be providing clinical experiences to other health professions learners, such as physician assistants or advanced practice nurses. Without the participation of community-based preceptors, most medical schools would not have enough clinical training sites for students. As opposed to inpatient medicine where the structure allows one attending to teach multiple students and residents, the fast-paced nature of outpatient medicine works best in a one-to-one student to preceptor ratio. This increases the need for more preceptors in outpatient educational settings leading to the utilization of community preceptors. Perhaps in response to this impending crisis, providing payment to community-based preceptors for teaching activities is becoming more common. The percentage of medical schools paying community-based preceptors for family medicine clerkship teaching was 23% in 2014, increased to 35% in 2015, and to 40% in 2017.12

This study has several limitations. The survey was directed only to family medicine or primary care clerkship directors, and the findings may relate more to outpatient rather than inpatient practice. The survey did not distinguish between clerkships that assign learners to community-based offices vs hospital settings. However, most of the findings relating to monitoring and remediating performance issues are applicable to any specialty that engages preceptors in community settings. This study may also have been limited by response bias. Only those clerkship directors who chose to respond to the survey completed the series of 10 questions on this topic. A response rate of 68%, however, argues for good representation of medical schools. Though a survey-based study proves association and not causation, the descriptions and correlations may still provide ideas for implementing performance monitoring and improvement initiatives and further evaluation.

More formal methods of tracking the quality of teaching at community practices, as well as remediation plans for struggling offices, can support current sites, reducing the need to recruit new training sites. Offering centralized preceptor development activities from medical schools geared toward the importance of evaluations, balancing learners’ needs and opportunities for student engagement, can help overcome some of the identified challenges. Medical schools should also consider providing additional time and support for clerkship directors to assist with tracking teaching quality at sites, and to assist struggling preceptors prior to removing them from teaching. Medical education leaders can explore payment as an option to incentivize community-based preceptors in schools where identifying clinical sites is a challenge. Future areas of study include strategies for remediation, criteria for limiting teaching roles and whether these differ with the type of performance issue, as well as the effect of paying preceptors on preceptor recruitment and retention.