Resident Physician Interactions and Engagement With Written Assessments of Performance

Resident feedback is critical for improvement and excellent performance, yet residents report that they infrequently get it.^1-3 The Accreditation Council for Graduate Medical Education (ACGME) requires faculty to directly observe, evaluate, and provide feedback on resident performance during each rotation.⁴ Residents can use this ongoing, formative feedback to improve their learning in the context of provision of patient care or other educational opportunities.⁴ Feedback can also be summative in that faculty evaluate a resident’s learning by comparing residents against the objectives, goals, and standards stated by the program. Additionally, formal assessments/evaluations are utilized for advancement and resident promotion.⁴ To accomplish both summative and formative feedback, faculty assess resident performance on multiple facets using numerical milestones.⁴ Typically, the assessment forms also include a comment portion for narrative feedback that may include feedback on specific patient encounters, corrective information, and alternative strategies. Effective assessments with milestones are crucial to residency training for ongoing development of individual learners and continual quality improvement of training programs.^2- 8

Faculty report multiple barriers to providing residents with meaningful feedback, including time constraints, the often-brief duration of relationship between preceptor and trainees, inconsistencies among evaluations, quantity of evaluations, lack of training for faculty, and typical concerns of how certain statements may affect resident progression.^9, 10 Trainees also perceive that assessors become preoccupied with form completion and box checking as opposed to providing meaningful advice. Residents find greater value from feedback derived from natural conversations compared to interpreting numbers from a checklist questionnaire.¹¹ Yet, clinical competency committees use both narrative and numeric assessments for promotion decisions.^2, 12 Thus, there may be fear from residents that any suboptimal performance on a single assessment may lead to remediation or impact their career prospects, causing residents to avoid actively seeking feedback or remain disengaged with the evaluative process.¹¹ Evidence exists regarding residents’ attitudes toward written assessments, including concerns for quantity over quality, lack of understanding on how to utilize feedback, and adversity to criticism, but little is known about the actual process that residents use when reviewing their assessments. ^{1, 6, 7, 8, 9, 11}

The objective of our study was to understand residents’ approach to reviewing, interpreting, and incorporating the feedback provided in faculty assessments. Our study explored what residents reflect on prior to viewing their evaluations, the procedures they followed when reviewing them, the value they placed on the numerical milestone scales, and the emphasis they put on written feedback.

Utilizing an interview-based constructivist grounded-theory approach, this qualitative study explored how residents interact with their assessments. ¹³ The interviews were conducted with a target population of residents in an ACGME-accredited family medicine residency program. Fourteen of the 20 eligible residents voluntarily participated. The interviews followed a semistructured guide (Table 1) intended to elicit evidence on residents’ practice, attitudes, and knowledge of the assessment process. The interview guide, created through a collaboration (T.H., B.F.), was piloted with a resident (coauthor O.B.) and slightly revised for clarity. Interviews were conducted by the primary author (B.F.) in order to preserve confidentiality from the faculty who may be involved in resident training. All interviews were conducted via WebEx in January of 2021. Following the Standards for Reporting Qualitative Research and data analysis 14 and consistent with the tenets of constructivist grounded-theory methodology, ¹⁵ we began with independent data analysis and initial coding of five transcripts. This provided the foundation for a coding meeting where examples of themes were discussed and agreed upon through consensus. Two researchers (B.F., O.B.) then analyzed an additional five transcripts; common themes were discussed at the following coding meeting of the entire team. B.F. coded the final four transcripts, and O.B. reviewed them. We conducted a final coding meeting to finalize themes through consensus and discussion (Figure 1). The Central Michigan University Institutional Review Board determined the study to be exempt.

Our study revealed most residents (71%) viewed the comment section first, and all but two residents (85%) viewed the comment section as the most important part of the assessment (Appendix Table A). The study outcomes revealed themes focused on the assessments, the assessed, and the assessor.

As a process, most residents opened their evaluations, scrolled past the numerical data, and looked at the comments. As a focus, residents tended to scrutinize the narrative on the evaluation first, and most considered the comments more useful than the numerical portion of the evaluation, whether they viewed that section first or last. As for value, residents appreciated feedback in the comments when it was timely, constructive, and consistent across evaluations (Appendix Table A).

As residents opened their evaluations, many reported that they took the opportunity to reflect on their performance, yet some reported anxiety when thinking about what the faculty thought of their performances. The reflections included wondering if their actions were interpreted differently by faculty than the resident remembered, and reactions to faculty perceptions that included nervousness, anxiety, and even fear (Appendix Table A).

Overall, residents reported that formal assessments did not provide tangible and timely feedback that was meaningful. Residents reported frustration with evaluations that were not available immediately. The residents described feedback given only at the end of a rotation as “having no value,” especially when they did not have another similar rotation soon so they could act on this information. Therefore, they expressed a desire for midrotation assessments to allow them time to immediately correct their practice. Recall bias for both residents and faculty were reported as well; with too much time passing between the interactions and the assessment, residents would forget when the observed clinical interactions occurred or the interactions therein. Residents expressed concerns with imprecise feedback due to faculty forgetting the resident interactions and performances between the time the actions occurred and the time when faculty completed their assessment. Residents in this study also expressed concern about inconsistencies among evaluators, including issues with the feedback provided when the duration of the interaction was short or there was no direct observation of the resident’s clinical performance. Residents disregarded evaluations containing number straight-lining (choosing the same number throughout the assessment) since this indicated the faculty’s lack of mindful assessment of their performance (Appendix Table A).

The intention of the ACGME assessment system is both formative and summative assessment⁴ and this study determined how residents engage with this evaluation system. The residents in our study reported that narrative feedback, more than numeric ratings, gave more insight to performance and led to actionable changes in behaviors. Comments provide constructive feedback and informed self-assessment, and they detail information for program leadership on resident performance.^3, 16

This study supported the importance of faculty building rapport with residents as an important part of the evaluation process.¹⁷ Similar to prior studies, residents discount faculty evaluations when they perceive lack of engagement and find value in feedback from faculty that have observed their skills on multiple occasions.^{9, 16, 18, 19}

Residents noted that self-reflection is critical to the evaluation process. This is supported by the R2C2 model (relationships, reactions, content, coaching) where the “reactions” phase encourages residents to reflect on and compare their view of their performance against the feedback received from faculty. ¹⁷ Recognizing that residents are reflective while reviewing their evaluations is important to understanding how residents engage in feedback. Residency programs should facilitate growth of informed self-assessment skills during training since independent physicians must identify their own knowledge and skill gaps to continue professional growth and enhance patient care.^17, 20 The residents in our study also reported that time between performance and feedback can lead to discrepancies between the faculty members’ and residents’ memory of patient encounters, thus delaying implementation of practice change and impairing growth. Residents desire real-time feedback when content, context, and perceptions are available for immediate recall.²¹

This study has several limitations, including sample size (the subjects may not represent all residents), and generalizability (the study involved a single site and specialty). The ACGME Milestones are designed to personalize the competency of each resident on their journey toward the independent practice of medicine. ⁴ Future studies should focus on response time for feedback and teaching approaches to incorporating feedback. Programs should educate faculty on how to provide meaningful and actionable feedback to improve the usefulness of the ACGME Milestones.¹⁶ Residency programs should also understand that the milestones were designed to facilitate specialty goals and curriculum improvement for guiding residents.²

In conclusion, a program’s approach to formative and summative assessments should include quality, useful data. Until these changes are implemented, the comment portion of evaluations will likely remain the focus of residents’ interaction with their assessments. Faculty should therefore engage residents in the feedback process and reinforce areas of strength while also providing constructive criticism supported by specific examples within the narratives and suggestions for improvement.^3, 18, 4