Video Review Process: Enhancing Impact and Resident/Faculty Satisfaction

Video review (VR) is a commonly employed teaching method to help residents reflect upon their strengths and areas for development in communicating with patients.¹ Research suggests review of videos and feedback by experts can improve resident performance.^2,3 Although over two-thirds of family medicine residency (FMR) programs use VR to teach communication skills,⁴ the levels of faculty and resident satisfaction and perceived impact of the process are unknown. Anecdotally, some residents in our FMR program have expressed dread and anxiety surrounding VRs.

Historically, the VR process at our FMR program involved a pair of faculty (a physician and a behaviorist) who met with one resident for 40 minutes and viewed a recent video-recorded session. Faculty completed the Patient Centered Observation Form (PCOF)⁵ as a structure for resident feedback, but no follow-up occurred.

Sensing an opportunity to enhance satisfaction and impact, we made the following three procedural changes to the existing process. First, we developed a new VR feedback form (available from the first author) in which residents and faculty discuss what communication skill the resident has been working on. Then, based on the reviewed session, they jointly identify and document three strengths and three areas for improvement. Second, shortly after the VR, the behaviorist emails the resident and his/her advisor the feedback form and a tip sheet corresponding to a skill the resident identified to work on (eg, agenda setting, shared decision-making). Third, the behaviorist shadows the resident on one clinic-based patient encounter within a month of the VR. The behaviorist checks in with the resident before the visit about the communication skill he/she is working on, observes the visit, and provides brief, specific feedback after the patient encounter. The intent of the shadowing is to reinforce the skills discussed in the VR and to offer immediate feedback after practice of a specific skill.

The objective of this study was to evaluate the impact of these changes in the VR curriculum. We hypothesized that the changes would be associated with increased satisfaction, perceived value, and educational impact of the VR process from the perspectives of both faculty and residents.

FMR faculty and residents in a 10/10/10 urban program were recruited to participate via email and by reminders in clinic meetings. Two randomly selected participants received a $10 gift card to a local restaurant. Participants completed an online survey in REDCap⁶ eliciting their perspectives about VR during the preceding 9 months. The survey was administered twice, first prior to implementing any changes and then 9 months after the changes were instituted. All surveys were anonymous; we did not collect any names or identifying information with the surveys. REDCap uses unique identifiers to link pre- and postsurveys without compromising anonymity. 

The 20-item survey consisted primarily of 5-point Likert scale items and assessed satisfaction, perceived value, and impact. Questions were parallel for the residents and faculty, using almost verbatim wording to elicit perspectives. The cognizant university institutional review board determined that this project was exempt from review because a review of departmental/educational processes does not meet the definition of human subjects research.

Likert-type variables were treated as continuous. Wilcoxon signed-rank tests (faculty) and linear mixed models with a random intercept for ID (residents) were used to assess change in the Likert-type scales from pre to post. Linear mixed models were chosen to analyze resident data in order to include all residents, even those that did not complete both of the surveys (n=14). McNemar’s exact tests were used to assess change for questions with binary response options for residents that completed both the pre- and postsurveys. SAS v 9.4 (SAS Institute Inc, Cary, NC) was used for analysis with alpha set at 0.05.

All faculty completed both the baseline and postsurveys. Of the 11 faculty members, seven (64%) were male, nine (82%) were white.

Thirty-four unique residents completed at least one survey during the 18-month evaluation period (four residents extended their training slightly). Response rates for both the baseline and postsurveys for all residents in our program at the time of the survey were 100%. Most residents were female (74%) and white (74%). Of the 34 residents, 20 completed both the pre- and postsurveys (59%), four (12%) only completed the presurvey (third-year residents who finished the program prior to the postsurvey), and 10 (29%) only completed the postsurvey (interns who were not in the program during the presurvey).

As shown in Table 1, the percentage of respondents who reported high levels of satisfaction with the VR process and the percentage who described it as valuable or very valuable were significantly higher at postsurvey than at baseline (among both faculty and residents). At baseline, less than 10% of residents indicated that they remembered VR skills 1 month afterward a fair amount or a great deal; these rates rose significantly at postsurvey among both residents and faculty, but only to 50% among residents. Finally, faculty perceptions about the improvement in residents’ communication skills based on VR rose significantly from the baseline to the postsurvey. However, at postsurvey, slightly over half (53%) of residents perceived the VR process as resulting in considerable improvements in their communication, which was not a significant change from levels at baseline.

As the VR process can be stressful, we also examined changes in resident emotions (see Table 2). At the postsurvey, significantly more residents reported being both aware of skills they want to work on and equipped with specific tips to improve the identified skill than at baseline. Some residents added qualitative comments about their experience of VR on the postsurvey, many of which correspond to our quantitative findings, such as: “I always come away with a new skill/thing to work on,” and “making a goal for 1-2 things to work on is helpful.”

Most residents (89%) reported that the behaviorist consistently debriefed with them after the observed patient visit. Of residents who had debriefings, 96% indicated they received feedback about a strength of their communication style, and 92% had received constructive feedback.

Consistent with our expectation based on anecdotal feedback from residents, only one-third of residents were satisfied with the VR process at baseline, and 40% reported feeling embarrassed after a VR at baseline. However, significant improvements emerged in both residents’ and faculty members’ satisfaction and perceptions of value of the VR process after our procedural changes. Furthermore, significantly more residents reported being both aware of skills they wanted to work on and equipped with specific tips to improve the identified skill at postsurvey than at baseline. Finally, almost all residents who experienced follow-up shadowing by a BH provider reported having received praise and constructive feedback regarding their communication skills.

Although these findings are promising, more work is needed to improve memory of skills and impact. In addition to the follow-up email, tip sheet, and shadowing, faculty may reinforce the skills during precepting and during quarterly reviews. We realize residents vary in what form of learning is most impactful. Although some may not immediately perceive the benefits of VR, we hope that further process modification will help to enhance value. For example, we also increased the frequency of VRs from annually to approximately five times per year.

Study limitations include data collection at one site, reliance on self-report measures, and impossibility of discerning impact of each procedural change due to simultaneous implementation. Future research could develop and validate measures to accurately assess communication skills for resident physicians, thereby allowing future research to supplement self-report data with objective observational data from independent, blind raters.

Despite these limitations, this study documented significant improvements in satisfaction and perceived value in a VR process associated with numerous procedural changes. Anecdotally, many residents report that they keep the feedback forms and tip sheets, and go back to them when struggling with a communication issue. Similarly, many faculty note that they review their residents’ feedback forms in preparation for quarterly reviews. All of these changes can be implemented in other residencies, and the VR feedback form and tip sheets are available from the first author upon request. Given the busy schedules of faculty and residents and the vital role of enhancing communication skills during residency, it is important for family medicine educators to develop and implement creative educational innovations to maximize the impact of the VR process.