Teaching on the Fly: A Needs Assessment for an Ambulatory Resident-as-Teacher Workshop

Residents serve a key role in medical student education, however the art of teaching is often left to the hidden curriculum without formal training. While residents spend up to 20% of their time teaching,^1,2 most feel unprepared to teach, a barrier shared by clinical educators in the ambulatory setting.^3,4 Resident-as-teacher programs across specialties increase self-reported confidence, enthusiasm, and teaching skills, as well as students’ ratings of resident teaching ability.^5-12 As residency programs formalize teaching curricula, ambulatory teaching and feedback are critical topics to include as identified by consensus guidelines.¹³

While self-assessment is commonly used as an end point in evaluating interventions, it frequently has little correlation to objective measures.^14-16 While teaching interventions often result in increased self-reported teaching behaviors, they do not directly correlate with learner outcomes.¹⁷

With no formal curriculum established at our program and most teaching occurring in outpatient clinic, we identified an opportunity to improve ambulatory teaching with a focused workshop to improve residents’ self-confidence and teaching skills, gathering data from both residents and medical students to assess workshop efficacy.

A 2.5-hour workshop was integrated into resident didactics attended by all year groups. The workshop included a rotating small group discussion (Gallery Walk) discussing attitudes toward teaching, pocket references, and lectures on utilizing microskills and giving effective feedback.^18-21 Residents then practiced specific microskills and elements of feedback through role play. Finally, barriers to teaching in the ambulatory setting and potential solutions were discussed.

Surveys utilized a 5-point Likert scale to evaluate outcomes on various levels of the New World Kirkpatrick Model of evaluation.²² An immediate postintervention survey captured resident reactions to the workshop (Kirkpatrick level 1). Follow-on resident surveys evaluated confidence and implementation of teaching behavior in clinic (Kirkpatrick level 2 and self-reported level 3). Medical student surveys assessed changes in teaching behaviors from the student perspective (Kirkpatrick level 3). Resident preintervention data were collected just prior to the workshop and postintervention data were collected 3 months after. Anonymous end-of-rotation preintervention surveys were provided to students rotating through the Family Medicine Residency at Naval Hospital Camp Pendleton for 3 months prior to the workshop, while postintervention surveys were collected for 5 months following the workshop. Due to the nature of the clerkship schedule, there was no overlap between medical students in the preintervention group and those in the postintervention group. Likewise, due to away rotations at other local hospitals, some residents in the postintervention group may have been different than those in the preintervention group.

We converted Likert scale results into numerical values and compared via two-sample t test with unequal variance. This project was determined to be exempt from review by the Naval Hospital Camp Pendleton Clinical Investigation Program, under exemption #2. Full intervention materials are available on the STFM Resource Library.²³

There were 13 preintervention and 12 postintervention resident responses, as well as four pre-intervention and eight postintervention student responses. Immediate postworkshop reactions showed residents felt the content was useful, particularly lectures and roleplay scenarios (Tables 1 and 2). Resident postintervention surveys showed significant increases in self-reported preparedness, confidence, and utilization of microskills. Self-reported use of effective feedback skills increased but was not significant (Table 3). Student responses reflected improved resident preparedness to teach and improved frequency and quality of feedback but did not reflect increases in other teaching behaviors (Table 4).

Our intervention had positive impacts at Kirkpatrick levels 1 and 2 with good initial reaction and increased resident self-confidence but failed to translate into significantly changed behavior in student responses. Role-play scenarios during the workshop appeared to increase preparedness and confidence, suggesting this would be helpful in future iterations. Conversely, the initial small group discussion was not perceived as helpful and should be changed in future iterations.

Several insights were only made possible through triangulating resident self-report with medical student responses, emphasizing the importance of moving beyond self-report alone in evaluating the efficacy of teaching programs. While residents self-reported increases in teaching behaviors, students did not recognize this change. Among teaching microskills, students only noted increased learning plan development. While other microskills such as pausing to allow time for response may not be recognized by students, establishing goals before clinic may have been apparent. Although residents reported an increase in giving learners the opportunity for feedback, students disagreed, again raising questions on the correlation between self-reporting and actual behavior. Possible explanations include that residents learned the importance of feedback but failed to implement it regularly, or medical students lacked recognition of feedback when given. The data collected do not allow further examination of the discrepancy, revealing a gap in a survey-only collection model.

Our findings are consistent with a growing trend in medical education literature recognizing the inadequacies of self-reporting and use of student evaluations as the only metrics for curricula efficacy, instead shifting toward more objective evaluation of teaching efficacy from multiple sources.^24-26 While we attempted to mitigate the inadequacies of self-report with student surveys, there are other limitations, such as asking for aggregated assessments on resident teaching at end of rotation instead of individual teaching experiences. Use of survey data alone does not explain why there are discrepancies between resident self-report and learner responses. Future iterations should include direct observation of teaching from peers or faculty who can provide feedback on teaching technique. While this is more time-consuming, we believe it could provide a more complete assessment on intervention effectiveness.

The data from our study are limited by the small number of participants and may be underpowered to detect significant changes or reach actionable quantitative conclusions. In addition, having different students involved in the pre- and postintervention surveys makes it challenging to determine if changes were due to the intervention or from personal student factors. Overall, our use of learner data and resident input emphasizes the risks of relying on self-assessment data and demonstrates the importance of capturing data from learners and teachers simultaneously to improve the experiences of both groups. Future iterations should further work on translating knowledge into learner-focused outcomes.