Narrative Feedback to Family Medicine Faculty: A Content Analysis

Peer and learner evaluations of faculty in graduate medical education are critical for performance improvement, promotion, and resource allocation.¹ Although data are limited, studies show that feedback from residents may help faculty improve teaching.^2,3 Narrative feedback, in particular, can provide important contextual information.⁴ Existing reviews of narrative feedback from residents to faculty show that the feedback is often relevant, but not specific. This limits the utility of feedback to improve teaching.^5-7

Multiple feedback tools exist to evaluate clinical teaching. However the likelihood of capturing narrative feedback varies among these tools.¹ Barriers to the use of feedback tools include time-consuming processes, issues with accessibility, reliance upon rating scales, and the retrospective nature of solicited feedback. Mechanisms to solicit feedback are more effective when they do not require significant changes to workflow.⁸ To address these barriers, a mobile application, the Faculty Feedback Facilitator (F3App), was developed to allow for real-time capture of narrative feedback for faculty in the medical education setting. In 2017 to 2018, the F3App was piloted across eight family medicine residency programs. Training and technical assistance were provided to programs on implementation and best practices for feedback. Participating programs reviewed the F3App positively.⁹

This study expands the initial pilot⁹ by qualitatively examining the narrative feedback from the same eight programs.

We included narrative feedback observations entered about faculty by residents and peers from July 1, 2019 to June 5, 2020 across the eight programs in the analysis. Program characteristics are shown in Table 1.  Resident observations were anonymous by default per the Accreditation Council for Graduate Medical Education (ACGME), however learners were able to deselect this option.¹⁰ Faculty peer feedback was not anonymous in order to foster transparency and trust, important elements in a robust culture of feedback.¹¹ Programs were offered training on the Situation-Behavior-Impact feedback model and tools were implemented at the discretion of the program.¹² 

Using a deductive content analysis approach, we coded each observation as “positive,” “constructive,” and “actionable.”¹³ The categories were not mutually exclusive and are based on previous work.¹⁴ We also coded observations according to perceived environment: “clinic,” “inpatient,” or “general/unknown.” Table 2 includes coding definitions.

The four coders conducted an initial independent coding of a subset of observations and then reviewed the codes as a team to create a shared understanding of the coding scheme. Next, the data set was divided in half and two team members coded each half. Upon completion, the full coding team discussed coding discrepancies to achieve consensus.

The University of North Carolina Institutional Review Board approved this study (IRB # 20-0324).

Three hundred thirty unique observations were generated during the study. Table 3 shows the coding summary. Thirty-three percent (110/330) of the observations were made by peer faculty and 67% (220/330) were made by residents. Most observations were positive (98%) and actionable (88%). Observations were made in clinic, inpatient, and general educational settings. Sample entries for each observation category are listed in Table 2.

Table 4 compares the types and settings of observations by faculty and residents. There was little variability in the types of observations recorded by faculty vs residents.

Table 5 compares observations submitted anonymously vs nonanonymously by residents. Sixty-one percent (134/220) of the observations were submitted anonymously. All eleven (100%) constructive comments were submitted anonymously.

Based on literature review, this is the first qualitative analysis of feedback to medical faculty from residents and peers using a mobile application. The findings demonstrate that the mobile interface yielded primarily positive and actionable/specific narrative feedback. This type of feedback encourages and reinforces positive behaviors and is consistent with best practices for providing feedback.^15-18 Prior studies show that narrative feedback is often not specific.^5-6

The few constructive comments from learners were submitted anonymously. Learners want the option to give anonymous feedback.^19-20 Learners did choose to deselect the anonymous default for some positive feedback.

This study has limitations. First, the design does not allow for a conclusion about why the majority of narrative observations were actionable/specific (ie, causality). Second, we only included residencies that participated in the pilot, some of which incorporated training on providing feedback. This may have positively skewed the number of actionable entries.

In conclusion, the use of a mobile application-based tool allows for collection of narrative feedback by learners and peers for faculty in a variety of settings. The feedback provides faculty with actionable and constructive suggestions for how to improve and/or continue their current teaching methods.