Knowledge Retention Using “Relay”: A Novel Active-Learning Technique

Many family medicine educators use passive didactic lectures despite mixed evidence for knowledge retention.^1–6 Adult learners have a limited attention span, and the transfer of new knowledge to long-term memory requires information manipulation and application.⁷ Active-learning techniques increase learner engagement and cognitive processing. Such techniques include group discussion, test-enhanced learning, hands-on lectures, audience response systems, simulation, and medical gaming.^8–15 These methods increase recall and retention, though often at the cost of increased preparatory time or expensive technology.^7,8,15–18

This study investigated a novel active-learning technique called "relay." Relay is low cost and easily implemented. It involves learners participating in three rounds of activity to answer questions that are increasingly open ended and require higher-order cognitive skills. As the rounds progress, learners correct their answers by accessing additional resources, including peers and point-of-care, evidence-based sources. The relay activity was designed to combine the active-learning techniques of test-enhanced learning, gamification, and collaborative learning to promote knowledge retention. These techniques have been shown to improve knowledge retention, critical thinking, and transfer to clinical situations, as well as learner satisfaction and engagement.^9,15,19–23

This study investigated the efficacy of relay as measured by knowledge retention at 3 months among family medicine residents, as compared to traditional methods of lecture followed by question-and-answer (Q&A). Secondary outcomes included learners’ self-perceived knowledge and engagement, again as compared to Q&A. An example of a relay activity is available in the STFM Resource Library.²⁴

This study was approved by the University of Minnesota’s Institutional Review Board. The study involved residents from four family medicine residency programs of sizes 18, 18, 24, and 28. During established structured educational sessions, family medicine faculty presented a 45-minute traditional passive lecture followed by either a 15-minute Q&A session or a 15-minute relay activity.

Each of the four residency programs received the same set of four lecture topics. The topics were chosen to be relevant but not overly familiar to resident-level physicians. Specifically, the topics were progesterone use to prevent premature birth, thalassemia evaluation and management, hepatitis C treatment in the primary care setting, and care of patients postbariatric surgery. These were based on the unique subject matter expertise of the faculty. A given topic was presented by a single lecturer, who delivered the same 45-minute content lecture to each program. The 45-minute lecture was followed by either a Q&A or relay activity. An independent statistician generated a balanced randomization schedule such that each program received two Q&A sessions and two relay sessions, and each faculty delivered his/her topic twice as a Q&A session and twice as a relay session (Table 1).

The primary outcome was knowledge retention at 3 months, comparing Q&A vs relay activities (Table 2). This was assessed by a 6-question multiple-choice knowledge assessment sent via email to lecture attendees. The questions aligned with the pre-specified learning objectives for the lecture. The primary analysis included all residents who completed retention assessments for both types of sessions (Q&A and relay activity). This allowed for adjustment within-resident, as well as training year, program, and lecturer/topic.

Secondary outcomes included self-perceived learning and engagement, measured by 4-item subjective ordinal scales administered as an electronic survey immediately following a session (Table 2). We included all session participants who completed the survey. For learner engagement, the residents were asked, "How engaged were you during the [Q&A/relay] portion of the session?" As a quality control measure, residents were also asked about their engagement during the portion that was identical for all learners: "How engaged were you during the lecture portion of the session?" Similarly, to assess self-perceived knowledge, residents were asked two questions: "How knowledgeable were you about the topic after the session?" and "How knowledgeable were you about the topic before the session?" The four-item choices on the scales were "minimally," "somewhat," "moderately," or "very" engaged/knowledgeable.

We performed statistical analysis using R Project for Statistical Computing, version 3.6.²⁵ We summarized resident characteristics with frequencies and compared using Pearson’s c² test. We summarized knowledge retention as mean scores with 95% confidence intervals, which were compared using a linear mixed-effects regression model. We measured secondary outcomes of self-perceived learning and engagement by 4-item ordinal scales, which we compared using the nonparametric Mann-Whitney Test. All comparative tests used a two-tailed null hypothesis of equivalence. We considered a value of ≤.05 statistically significant.

The primary analysis of 3-month knowledge retention included 18 unique residents (overall response rate: 20% of the programs' 88 residents), and their associated 51 responses. There were no statistically significant differences in the respondents’ training year, program, and attended lecturer/topics (Table 3). The mean scores for the Q&A and relay sessions were 60% and 67%, respectively (7% difference, 95% CI: -4 to 18%, P=.20), adjusting within-resident and for resident training year, program, and lecturer/topic (Table 4).

For the secondary outcomes of immediate follow-up attitude surveys, the response rate was 79% (143 responses from 181 attendees; Figure 1). Compared to the Q&A sessions, the relay sessions’ attendees reported greater engagement (overall P=.003). Specifically, 51% of respondents felt “very engaged” during the relay session, compared to 28% during the Q&A session. Self-perceived knowledge, however, was not statistically different between the relay and Q&A sessions (overall P=.05, rounded down). Additionally, the self-perceived engagement and knowledge during the identical lecture portion (quality control questions) were not significantly different (P=.51 engagement, P=.39 knowledge).

This prospective multiresidency study compared the effects of relay activity to Q&A sessions following a traditional lecture. The primary outcome of 3-month knowledge retention was not significantly different, nor was immediate self-perceived knowledge. The only significant difference was self-perceived learner engagement. This work is consistent with existing mixed evidence regarding various active-learning techniques. It provides another option for active-learning techniques.

A major limitation of the primary outcome of 3-month knowledge retention was the low response rate. Despite a pool of 88 residents across the four programs, only 18 residents (20%) responded to knowledge assessments across both the relay and Q&A sessions. One program in particular (Program B) did not have any residents respond to both session types (Table 3). This limits the generalizability of the results, and it is uncertain what the effect would be across the larger resident population. There would have been more responses if residents who only responded to one session type were included. However, it was felt that individual resident differences presented the greatest source of potential variability and need for adjustment across Q&A and relay sessions.

Another limitation of the primary outcome involved the measurement of knowledge retention. A multiple-choice assessment does not directly align with resource utilization, which is a focus of the relay activity. That is, the measurement does not directly measure the intervention. Also, the assessments were not validated with other audiences (eg, medical students, specialists) to ensure appropriate complexity for the resident learner level, though mean retention scores of 60%-67% at 3 months are consistent with the existing literature.^{3–5,8,12,13,16,17,20} Ideally, the randomized schedule of the topics across relay and Q&A sessions reduced the incidence of bias secondary to any flawed questions. Finally, in an effort to reduce survey fatigue, the knowledge assessment was limited to six items; this resolution is not fine enough to discern small effect sizes.

As for the secondary outcomes of attitudes, there was potential nonresponse bias if the surveys were more likely to be completed by residents with stronger knowledge and/or more positive attitudes toward the intervention.

Further work is needed to investigate this study in a larger population. It is also possible that the efficacy of the relay activity may be related to a learner’s familiarity with a topic, which would require additional studies with a greater range of topics. Finally, the relay technique could also be explored in additional educational contexts, such as priming learners before a lecture, as a stand-alone activity, or as reemphasis after some time has passed.