Introduction: Many residency programs utilize passive didactic lectures despite mixed evidence for knowledge retention. This prospective study investigated the efficacy of “relay,” an active-learning technique, as measured by residents' knowledge retention and attitudes compared to traditional format.
Methods: Faculty presented lectures to four family medicine residency programs on a randomized schedule. Lectures were followed by a 15-minute question-and-answer (Q&A) session or relay session. A relay is a collaborative, question-based game. The primary outcome was knowledge retention at 3 months, comparing Q&A to relay sessions as measured by a multiple-choice assessment. Responses were only included if a given resident completed knowledge assessments for both Q&A and relay sessions, to allow for intraresident adjustments, in addition to program, training year, and lecturer/topic. Secondary outcomes included residents' self-perceived knowledge and engagement as surveyed by an ordinal scale immediately following the learning session.
Results: The primary analysis included 51 responses from 18 unique residents. The adjusted mean knowledge assessment score at 3 months was not statistically different after the relay sessions compared to Q&A (67% vs 60%, respectively; 7% difference, 95% CI: -4 to 18%, P=.20). For the secondary outcomes of learner attitudes (n=143 responses), learners reported greater engagement after the relay sessions compared to Q&A (51% vs 28% "very engaged"; overall P=.003), but self-perceived knowledge was not significantly different (overall P=.05, rounded down).
Conclusions: The relay technique did not show significant difference in 3-month knowledge retention, nor immediate self-perceived knowledge, despite greater learner self-perceived engagement.
Many family medicine educators use passive didactic lectures despite mixed evidence for knowledge retention.1–6 Adult learners have a limited attention span, and the transfer of new knowledge to long-term memory requires information manipulation and application.7 Active-learning techniques increase learner engagement and cognitive processing. Such techniques include group discussion, test-enhanced learning, hands-on lectures, audience response systems, simulation, and medical gaming.8–15 These methods increase recall and retention, though often at the cost of increased preparatory time or expensive technology.7,8,15–18
This study investigated a novel active-learning technique called "relay." Relay is low cost and easily implemented. It involves learners participating in three rounds of activity to answer questions that are increasingly open ended and require higher-order cognitive skills. As the rounds progress, learners correct their answers by accessing additional resources, including peers and point-of-care, evidence-based sources. The relay activity was designed to combine the active-learning techniques of test-enhanced learning, gamification, and collaborative learning to promote knowledge retention. These techniques have been shown to improve knowledge retention, critical thinking, and transfer to clinical situations, as well as learner satisfaction and engagement.9,15,19–23
This study investigated the efficacy of relay as measured by knowledge retention at 3 months among family medicine residents, as compared to traditional methods of lecture followed by question-and-answer (Q&A). Secondary outcomes included learners’ self-perceived knowledge and engagement, again as compared to Q&A. An example of a relay activity is available in the STFM Resource Library.24
This study was approved by the University of Minnesota’s Institutional Review Board. The study involved residents from four family medicine residency programs of sizes 18, 18, 24, and 28. During established structured educational sessions, family medicine faculty presented a 45-minute traditional passive lecture followed by either a 15-minute Q&A session or a 15-minute relay activity.
Each of the four residency programs received the same set of four lecture topics. The topics were chosen to be relevant but not overly familiar to resident-level physicians. Specifically, the topics were progesterone use to prevent premature birth, thalassemia evaluation and management, hepatitis C treatment in the primary care setting, and care of patients postbariatric surgery. These were based on the unique subject matter expertise of the faculty. A given topic was presented by a single lecturer, who delivered the same 45-minute content lecture to each program. The 45-minute lecture was followed by either a Q&A or relay activity. An independent statistician generated a balanced randomization schedule such that each program received two Q&A sessions and two relay sessions, and each faculty delivered his/her topic twice as a Q&A session and twice as a relay session (Table 1).
The primary outcome was knowledge retention at 3 months, comparing Q&A vs relay activities (Table 2). This was assessed by a 6-question multiple-choice knowledge assessment sent via email to lecture attendees. The questions aligned with the pre-specified learning objectives for the lecture. The primary analysis included all residents who completed retention assessments for both types of sessions (Q&A and relay activity). This allowed for adjustment within-resident, as well as training year, program, and lecturer/topic.
Secondary outcomes included self-perceived learning and engagement, measured by 4-item subjective ordinal scales administered as an electronic survey immediately following a session (Table 2). We included all session participants who completed the survey. For learner engagement, the residents were asked, "How engaged were you during the [Q&A/relay] portion of the session?" As a quality control measure, residents were also asked about their engagement during the portion that was identical for all learners: "How engaged were you during the lecture portion of the session?" Similarly, to assess self-perceived knowledge, residents were asked two questions: "How knowledgeable were you about the topic after the session?" and "How knowledgeable were you about the topic before the session?" The four-item choices on the scales were "minimally," "somewhat," "moderately," or "very" engaged/knowledgeable.
We performed statistical analysis using R Project for Statistical Computing, version 3.6.25 We summarized resident characteristics with frequencies and compared using Pearson’s c2 test. We summarized knowledge retention as mean scores with 95% confidence intervals, which were compared using a linear mixed-effects regression model. We measured secondary outcomes of self-perceived learning and engagement by 4-item ordinal scales, which we compared using the nonparametric Mann-Whitney Test. All comparative tests used a two-tailed null hypothesis of equivalence. We considered a value of ≤.05 statistically significant.
The primary analysis of 3-month knowledge retention included 18 unique residents (overall response rate: 20% of the programs' 88 residents), and their associated 51 responses. There were no statistically significant differences in the respondents’ training year, program, and attended lecturer/topics (Table 3). The mean scores for the Q&A and relay sessions were 60% and 67%, respectively (7% difference, 95% CI: -4 to 18%, P=.20), adjusting within-resident and for resident training year, program, and lecturer/topic (Table 4).
For the secondary outcomes of immediate follow-up attitude surveys, the response rate was 79% (143 responses from 181 attendees; Figure 1). Compared to the Q&A sessions, the relay sessions’ attendees reported greater engagement (overall P=.003). Specifically, 51% of respondents felt “very engaged” during the relay session, compared to 28% during the Q&A session. Self-perceived knowledge, however, was not statistically different between the relay and Q&A sessions (overall P=.05, rounded down). Additionally, the self-perceived engagement and knowledge during the identical lecture portion (quality control questions) were not significantly different (P=.51 engagement, P=.39 knowledge).
This prospective multiresidency study compared the effects of relay activity to Q&A sessions following a traditional lecture. The primary outcome of 3-month knowledge retention was not significantly different, nor was immediate self-perceived knowledge. The only significant difference was self-perceived learner engagement. This work is consistent with existing mixed evidence regarding various active-learning techniques. It provides another option for active-learning techniques.
A major limitation of the primary outcome of 3-month knowledge retention was the low response rate. Despite a pool of 88 residents across the four programs, only 18 residents (20%) responded to knowledge assessments across both the relay and Q&A sessions. One program in particular (Program B) did not have any residents respond to both session types (Table 3). This limits the generalizability of the results, and it is uncertain what the effect would be across the larger resident population. There would have been more responses if residents who only responded to one session type were included. However, it was felt that individual resident differences presented the greatest source of potential variability and need for adjustment across Q&A and relay sessions.
Another limitation of the primary outcome involved the measurement of knowledge retention. A multiple-choice assessment does not directly align with resource utilization, which is a focus of the relay activity. That is, the measurement does not directly measure the intervention. Also, the assessments were not validated with other audiences (eg, medical students, specialists) to ensure appropriate complexity for the resident learner level, though mean retention scores of 60%-67% at 3 months are consistent with the existing literature.3–5,8,12,13,16,17,20 Ideally, the randomized schedule of the topics across relay and Q&A sessions reduced the incidence of bias secondary to any flawed questions. Finally, in an effort to reduce survey fatigue, the knowledge assessment was limited to six items; this resolution is not fine enough to discern small effect sizes.
As for the secondary outcomes of attitudes, there was potential nonresponse bias if the surveys were more likely to be completed by residents with stronger knowledge and/or more positive attitudes toward the intervention.
Further work is needed to investigate this study in a larger population. It is also possible that the efficacy of the relay activity may be related to a learner’s familiarity with a topic, which would require additional studies with a greater range of topics. Finally, the relay technique could also be explored in additional educational contexts, such as priming learners before a lecture, as a stand-alone activity, or as reemphasis after some time has passed.
- Picciano A, Winter R, Ballan D, Birnberg B, Jacks M, Laing E. Resident acquisition of knowledge during a noontime conference series. Fam Med. 2003;35(6):418-422.
- Warner S, Williams DE, Lukman R, Powell CC II, Kundinger K. Classroom lectures do not influence family practice residents’ learning. Acad Med. 1998;73(3):347-348. doi:10.1097/00001888-199803000-00029
- Tawfik MMR, Fayed AA, Dawood AF, Al Mussaed E, Ibrahim GH. Simulation-based learning versus didactic lecture in teaching bronchial asthma for undergraduate medical students: a step toward improvement of clinical competencies. Med Sci Educ. 2020;30(3):1061-1068. doi:10.1007/s40670-020-01014-y
- Taglieri C, Schnee D, Dvorkin Camiel L, et al. Comparison of long-term knowledge retention in lecture-based versus flipped team-based learning course delivery. Curr Pharm Teach Learn. 2017;9(3):391-397. doi:10.1016/j.cptl.2017.01.007
- Bleske BE, Remington TL, Wells TD, Klein KC, Tingen JM, Dorsch MP. A randomized crossover comparison between team-based learning and lecture format on long-term learning Outcomes. Pharmacy (Basel). 2018;6(3):81. doi:10.3390/pharmacy6030081
- Telner D, Carroll JC, Regehr G, Tabak D, Semotiuk K, Freeman R. Teaching primary care genetics: a randomized controlled trial comparison. Fam Med. 2017;49(6):443-450.
- Cooper AZ, Richards JB. Lectures for adult learners: breaking old habits in graduate medical education. Am J Med. 2017;130(3):376-381. doi:10.1016/j.amjmed.2016.11.009
- Sawatsky AP, Berlacher K, Granieri R. Using an ACTIVE teaching format versus a standard lecture format for increasing resident interaction and knowledge achievement during noon conference: a prospective, controlled study. BMC Med Educ. 2014;14(1):129. doi:10.1186/1472-6920-14-129
- Green ML, Moeller JJ, Spak JM. Test-enhanced learning in health professions education: A systematic review: BEME Guide No. 48. Med Teach. 2018;40(4):337-350. doi:10.1080/0142159X.2018.1430354
- McFee RM, Cupp AS, Wood JR. Use of case-based or hands-on laboratory exercises with physiology lectures improves knowledge retention, but veterinary medicine students prefer case-based activities. Adv Physiol Educ. 2018;42(2):182-191. doi:10.1152/advan.00084.2017
- Roze E, Worbe Y, Louapre C, et al. Miming neurological syndromes improves medical student’s long-term retention and delayed recall of neurology. J Neurol Sci. 2018;391:143-148. doi:10.1016/j.jns.2018.06.003
- Duggan PM, Palmer E, Devitt P. Electronic voting to encourage interactive lectures: a randomised trial. BMC Med Educ. 2007;7(1):25. doi:10.1186/1472-6920-7-25
- Arcoraci V, Squadrito F, Altavilla D, et al. Medical simulation in pharmacology learning and retention: A comparison study with traditional teaching in undergraduate medical students. Pharmacol Res Perspect. 2019;7(1):e00449. doi:10.1002/prp2.449
- Gorbanev I, Agudelo-Londoño S, González RA, et al. A systematic review of serious games in medical education: quality of evidence and pedagogical strategy. Med Educ Online. 2018;23(1):1438718-. doi:10.1080/10872981.2018.1438718
- McCoy L, Lewis JH, Dalton D. Gamification and multimedia for medical education: a landscape review. J Am Osteopath Assoc. 2016;116(1):22-34. doi:10.7556/jaoa.2016.003
- Raleigh MF, Wilson GA, Moss DA, et al. Same content, different methods: comparing lecture, engaged classroom, and simulation. Fam Med. 2018;50(2):100-105. doi:10.22454/FamMed.2018.222922
- Feldman M, Fernando O, Wan M, Martimianakis MA, Kulasegaram K. Testing test-enhanced continuing medical education: a randomized controlled trial. Acad Med. 2018;93(11S Association of American Medical Colleges Learn Serve Lead: Proceedings of the 57th Annual Research in Medical Education Sessions):S30-S36. doi:10.1097/ACM.0000000000002377
- Warren JN, Luctkar-Flude M, Godfrey C, Lukewich J. A systematic review of the effectiveness of simulation-based education on satisfaction and learning outcomes in nurse practitioner programs. Nurse Educ Today. 2016;46:99-108. doi:10.1016/j.nedt.2016.08.023
- Oakes DJ, Hegedus EM, Ollerenshaw SL, Drury H, Ritchie HE. Using the jigsaw method to teach abdominal anatomy. Anat Sci Educ. 2019;12(3):272-283. doi:10.1002/ase.1802
- Thrall GC, Coverdale JH, Benjamin S, Wiggins A, Lane CJ, Pato MT. A randomized controlled trial of team-based learning versus lectures with break-out groups on knowledge retention. Acad Psychiatry. 2016;40(5):755-760. doi:10.1007/s40596-016-0501-7
- Faezi ST, Moradi K, Ghafar Rahimi Amin A, Akhlaghi M, Keshmiri F. The effects of team-based learning on learning outcomes in a course of rheumatology. J Adv Med Educ Prof. 2018;6(1):22-30.
- Giuliodori MJ, Lujan HL, DiCarlo SE. Collaborative group testing benefits high- and low-performing students. Adv Physiol Educ. 2008;32(4):274-278. doi:10.1152/advan.00101.2007
- Vázquez-García M. Collaborative-group testing improves learning and knowledge retention of human physiology topics in second-year medical students. Adv Physiol Educ. 2018;42(2):232-239. doi:10.1152/advan.00113.2017
- Freeman K. Relay: a collaborative, active-learning technique. STFM Resource Library. Published 2021. Accessed May 6, 2021. https://resourcelibrary.stfm.org/viewdocument/relay-a-collaborative-active-lear?CommunityKey=2751b51d-483f-45e2-81de-4faced0a290a&tab=librarydocuments
- Hothorn T, Everitt BS. A Handbook of Statistical Analyses Using R.Boca Raton, FL: CRC Press; 2006. doi:10.1201/9781420010657.