Results of a Preceptor Improvement Project

Most US medical students receive some of their training in community settings,1-4 where they work with community preceptors who have had minimal formal training in teaching.5,6 Community preceptors—largely volunteer physicians—are keenly interested in improving their teaching effectiveness.5-7 Thus, improving preceptor teaching is an important goal. We developed a new preceptor improvement program and studied it to determine its effectiveness and acceptability for our community preceptors at the University of Minnesota.

To recruit potential participants for this project, we sent out 120 emails and postcards to all community physicians who had taken at least one University of Minnesota clerkship student in the last 2 years. Forty physicians expressed initial interest, of which 23 physicians enrolled in the project, and 13 physicians completed the project.

Interested preceptors completed an online form using Qualtrics software that included a self-assessment tool (pretest). The survey included questions about 21 teaching competencies across five domains (learner centeredness, interpersonal and communication skills, professionalism and role modeling, practice-based reflection and improvement, and learner evaluation) that the first author of this paper had previously validated and reported.8 The preceptors were asked to evaluate how often they demonstrated each competency along a 5-point scale (from never to always). After completing the survey, the physicians were asked to choose one of the five domains to work on during this self-improvement project.

A standardized medical student9 then contacted the preceptors about setting up an onsite visit. This was a second-year medical student who had been trained to portray an early third-year clerkship student on the first day in the clinic. The student gave a scripted oral presentation to the preceptor about a predetermined case, deliberately omitting some information. The student listened for questions, clarification, and feedback on the presentation from the preceptor. The student then noted the presence or absence of eight teaching behaviors (Table 1),10 and also gave the preceptor a global precepting score, ranging from 1 to 10. These three markers—the student’s global score, the number of observable behaviors, and the physician’s self-assessment—represented the preintervention condition.

After establishing the baseline, the physicians were provided an overview of the project. Each physician was given one of five different posters (Figure 1) that summarized the skill that they had chosen to work on. The participants were next emailed a link to a website that housed the content about each of the five possible areas of improvement. For each domain, there was a short video that explained and provided demonstrations of the skills related to that domain. In addition, each preceptor was required to review a module on the 1-minute preceptor microskills. The participants were given a laminated card that summarized the 1-Minute Preceptor approach. After completing these items, the participants were provided a link to an online module with further content related to the domain that they had chosen.

The preceptors’ progress on completing the online modules was verified through Qualtrics. After the preceptor completed the online content, the standardized student returned to the practice site and presented the same mock patient to the preceptor. Again, the preceptor was evaluated on the presence or absence of eight teaching behaviors and given an overall global score between 1 and 10. The preceptor again completed the self-assessment (posttest). The preceptor also evaluated the program’s strengths and weaknesses and indicated their likelihood of recommending the course to other preceptors. The time commitment for this performance improvement project was, on average, 3 hours in total over the course of 4 weeks (range 2 weeks to 3 months).

The University of Minnesota Institutional Review Board deemed this study exempt (IRB ID: STUDY00003529).

In the self-assessments, preceptors showed statistically significant improvement in 12 of the 21 teaching competencies (Table 2) on average. On average, one additional key teaching behavior was demonstrated during the precepting encounter after the intervention (P=.001, Table 3). On average, there was a 1.15-point improvement in the overall teaching effectiveness score (P=.001; Table 4). In the exit survey, all participants indicated they were either satisfied or highly satisfied with the program, and all indicated that they would recommend this program to a colleague.

Preceptors of medical students want to improve as teachers. We implemented a mixed-modality performance improvement project that was well received by our preceptors. The time investment required for it was similar to that of other performance improvement projects. Participants not only reported feeling better prepared to teach medical students, but also demonstrated higher scores in a postintervention self-assessment. They also demonstrated more positive teaching behaviors and scored higher on a global teaching effectiveness scale after the intervention. Although the preceptors received content in only one of the five domains, there was significant improvement across the five domains regardless of which domain they had chosen. It may be that reviewing the list of competencies is itself didactic, as the preceptor was reminded of core desired teaching behaviors.

Nearly every participant listed the interaction with the standardized student as the most valuable aspect of the project. It is unclear whether the exposure to the list of competencies, the poster, the online module, the video, or the time with the student played the most important role; likely it was the reinforcement of a message delivered via multiple modalities.

Several aspects of this project may be of value to other institutions designing teaching improvement programs. We used a set of previously vetted competencies that are clear and observable. Clear goals make for clearer instruction. Our use of a student as evaluator and a standardized student evaluation form is easy to replicate.

This study does have methodological limitations. We had a small sample size. We had no control group; any number of external inputs that our preceptors received during the study period could have played a role in the improvement seen and measured. As these preceptors all opted in, there is a high likelihood of bias; those who want to be better teachers completed this intervention. Also, 10 of the preceptors who opted in did not complete the project. While we did email them to confirm that they had withdrawn from the project, we did not inquire why they had not finished. The follow-up evaluation occurred 2 weeks to 3 months after the intervention. A longer interval was desirable as we wanted to know how long these concepts and skills had stuck. However, as participants were able to complete the content at their own pace, some completed it shortly before the student’s second visit was scheduled. Finally, short-term improvement in medical student teaching does not imply lasting improvement.

Further (and larger) studies are needed both to replicate our results and to better understand which aspects of this program were key to its effectiveness. In addition, future research could be done to see if similar effectiveness could be achieved with predominantly online content.