Residency Faculty Teaching Evaluation: What Do Faculty, Residents, and Program Directors Want?

The Accreditation Council for Graduate Medical Education Common Residency Program Requirements stipulate that each faculty member’s performance must be evaluated annually, and must include “a review of the faculty member’s clinical teaching abilities.”1 Feedback is an essential element in this process. Yet, the culture of medicine poses challenges to developing effective feedback systems: valid bidirectional feedback can be challenging in a hierarchical educational structure; patient care takes priority over teaching; and learners often have limited contact with multiple instructors.2 Likewise, no uniform expectations are universally accepted for clinical instruction,3-5 despite existing for learners.6,7 Without well-defined expectations regarding instruction, feedback provided to faculty members may not be adequately focused. This paper explores current and ideal characteristics of faculty teaching evaluation systems from the perspectives of faculty, residents, and residency program directors (PDs).

We utilized two qualitative approaches. First, we conducted confidential semistructured telephone interviews with PDs from a convenience sample of eight family medicine (FM) residency programs. The interview guide for the semistructured interviews (Table 1) was formulated after conducting a literature review and was developed by a panel of faculty experts in the fields of graduate medical education, behavioral science, and qualitative data collection. The expert panel also included a former residency program director. Questions explored the current process for faculty teaching evaluation with probes related to challenges, successes, barriers, and gaps within the current system. One member of the research team (H.B.) conducted the interviews. The interviewer asked each question of the program directors, along with appropriate probes in order to ensure a thorough exploration of each question. Interviews were recorded with permission of the respondents and transcribed verbatim.

Second, we emailed an anonymous online survey link to faculty and residents from the same eight programs with questions related to participants’ perceptions of the current faculty teaching evaluation system. The survey was primarily quantitative, but concluded with the open-ended question: “In your ideal world, what would the faculty teaching evaluation process look like?” A postimplementation survey was also administered to programs following piloting of a mobile application to collect point of teaching feedback from learners, and the quantitative data from pre- and postimplementation will be analyzed together and published separately.

The research team used an inductive thematic analysis approach8 to analyze the interview transcripts and narrative survey responses; one research team member reviewed the data and developed initial codes, sorting quotes and salient text by code. The other team members then reviewed the initial codes and related text and developed additional codes that emerged from the data. The team then met to resolve coding discrepancies and discuss emergent themes. It is important to note that rather than conducting data collection and analysis to the point of saturation, the research team analyzed the data with the intention of identifying and developing salient themes mentioned across programs. This is a result of the data collection being a part of a small implementation study of eight family medicine programs pilot-testing the aforementioned mobile application.

The University of North Carolina Chapel Hill’s Institutional Review Board approved this study (IRB #17-2052).

All eight (100%) PDs completed interviews. The following themes emerged across interviews.

Culture of Evaluation. PDs discussed programmatic conventions that foster a culture of evaluation, including having both residents and faculty involved in the feedback process, maintaining a high degree of mutual trust, and a shared understanding of feedback as an important element of performance improvement rather than just a task to be completed.

Faculty Behavior Change. The majority of PDs described the importance of an evaluation system that fosters faculty behavior change; several indicated they believed their programs succeeded in this regard.

Evaluation Fatigue. Many PDs identified the large number of evaluations residents are required to complete as a barrier; several also indicated they have attempted to address this barrier within their programs.

Quality of Feedback. Several of the PDs noted that the feedback to faculty members is often of low quality, provides no actionable information, and is often received out of context or too long after the event.

Table 2 provides illustrative quotes of each theme.

Survey response rates for faculty and residents were 79% (99/126) and 70% (152/216), respectively. Forty-nine percent (62/126) of faculty and 28% of residents (61/216) responded to the specific open-ended question described in this paper.

As shown in Table 3, faculty and residents expressed several themes in common regarding the ideal feedback system. Half of respondents in both groups noted that feedback should be specific enough to enable behavior change. As separate groups, similar proportions of faculty and residents described an evaluation process that captured feedback in real time. Roughly equal proportions of faculty and residents used the word “timely,” but without specifying what that meant. Similar proportions of faculty and residents indicated a desire for feedback at set time intervals rather than constant or ongoing. Faculty and residents differed in the extent to which they mentioned ease of use of the feedback mechanism, with faculty mentioning this less often than residents. This is also the case for ability for feedback to be provided anonymously, with residents mentioning this more often than faculty.

Eight themes were unique to faculty, with the following three being the most salient: regular frequency of feedback (29%), easy accessibility of feedback (21%), and feedback from peers (10%). Another eight themes were unique to residents, with the following three being the most salient: a preference for providing in-person feedback (21%), the opportunity to provide narrative feedback (15%), and a feedback process that does not result in retribution for negative feedback (13%). The rest of the themes are listed in Table 3.

Many of our findings are consistent with commonly accepted characteristics of effective feedback.9 Throughout the data collection process, we did not explicitly distinguish between feedback and evaluation, and results reflect this conceptual overlap among participants. Both PD and faculty responses identified a desire for actionable, real-time, frequent feedback that can be used to foster continued professional development. Residents indicated that feedback should be meaningful and with a level of specificity that helps faculty improve. Residents also indicated that faculty feedback should mirror the resident feedback process (ie, with explicit, shared understanding of what skill(s) the faculty member is trying to address).

These results highlight important process and contextual issues. All stakeholders indicated that feedback for faculty should not be burdensome. PDs emphasized the need for buy-in from both residents and faculty and for a culture that promotes multidirectional feedback as a part of continuous improvement. In addition, while the theme of anonymity was salient for residents, few faculty members mentioned this as important. Of particular note is the seemingly contradictory finding that residents preferred in-person feedback, yet they also expressed the need for anonymity. This reflects an important tension in the way residents perceive feedback, and while open, bidirectional feedback may be a program’s goal, it may be important for programs to be sensitive to the realities of their program context and recognize that their program’s culture may not yet support open feedback (eg, a culture where bidirectional feedback is welcomed and there is no fear of retribution). This is consistent with previous findings10 and may also reflect the cultural challenges of graduate medical education noted by Ramani11 and Watling.2

This research is not without limitations. Participating programs were self-selected members of a pilot study to implement a mobile faculty feedback system; additionally, while they represent a mix of university- and community-based programs and regional variation, all are FM programs, which may limit transferability of themes.

PDs, faculty, and residents share a desire to provide faculty with meaningful, specific, real-time feedback. Programs should strive to provide a culture in which feedback is an integral part of the learning process for both residents and faculty. Next steps include piloting a mobile feedback tool to facilitate point-of-observation feedback from residents to faculty, incorporating the principles identified in this study.