Longitudinal Interleaved Residency Training: A Consensus Definition

There has been much discussion in graduate medical education over the best way to structure residency training to optimally train physicians for the future health care workforce. ^{1, 2} Through the years, some programs have departed from the traditional structure of 1-month block rotations and have moved toward a longitudinal approach to their curriculum. ^{3, 4} Recent advancements in cognitive science lend credence to the notion that longitudinal curricula may have advantages over a block structure in that repetitive exposure to the same content distributed over time leads to enhanced knowledge retention compared to closely timed massed learning. ^5- 9 This spaced, or distributed, learning has been demonstrated to be effective not only in medical education but also in many other fields. ¹⁰

Taking spaced exposure further, interleaving is a process in which learners switch between multiple related subjects instead of mastering one topic thoroughly prior to moving onto the next topic (Figure 1 ). ¹¹ Interleaving allows learners to compare and contrast new concepts, creating essential connections that would be missed by studying each component independently. Studies have demonstrated that interleaving is more effective than blocked practice in helping learners differentiate, or discriminate, among similar concepts in math, art, sports, and other disciplines, thereby improving memory associations. ⁷ Interleaving has specifically been shown to improve inductive reasoning, ^{12, 13} a skill physicians use to make predictions about novel cases based on existing knowledge. ¹⁴ While evidence suggests benefits from interleaving, ^15-19 the technique remains understudied and underutilized at many levels of education, including graduate medical education. ^{20, 21}

In 2001, several articles in Family Medicine introduced the idea of a longitudinal approach to structuring residency training. ^22-26 The tenets of this paradigm emphasized increased continuity of care by scheduling residents in clinic nearly every day, supplemented by intense focused experiences in the hospital and other practice settings. ²² The proposed benefits of a longitudinal curriculum included more role modeling by family physicians, increased continuity of care across the continuum, more emphasis on clinic and primary care, and the practice of lifelong learning. A program director survey found that 3.6% of respondents described their curriculum as “mostly longitudinal” and 15% indicated interest in moving toward that style in the next 2 years. ²³ Evidence showed that structured repetitive learning experiences improved in-training exam scores significantly ²⁴ and improved patient access to their assigned primary care physician. ²⁶ A critique of these studies highlighted the need to standardize the definition of longitudinal curriculum before outcomes could be analyzed effectively in terms of resident training, knowledge, and performance. ²²

Despite advancements in cognitive science supporting longitudinal and interleaved education and the call for more research about longitudinal curricula in graduate medical education more than 20 years ago, no standardized definition has emerged. Published studies in family medicine are scarce and have focused on applying a longitudinal structure to specific aspects of the curriculum rather than the entire residency structure but conclude that a longitudinal format increases resident knowledge. ^27-29 More recently, however, descriptions of Clinic First ^{3, 30, 31} and High-Functioning Primary Care Residency Clinics ^{32, 33} have renewed interest in the clinic as the curriculum in primary care, and in longitudinal approaches as a means to increase time in clinic. A review of evidence-based education discussed the potential advantages of interleaving in anesthesiology residency, ⁵ and spaced repetition implemented in urology residency increased transfer and long-term retention of diagnostic skills. ³⁴ As such, we believe that longitudinal, interleaved residency training can be implemented in many disciplines once a standardized definition has been accepted.

To this end, we describe the results of a national workgroup that undertook an exercise in definition of Longitudinal Interleaved Residency Training (LIRT), a nomenclature consistent with Longitudinal Integrated Clerkships (LIC) being implemented in undergraduate medical education globally. ^35-38 The objective was to arrive at a standardized, functional definition of the term that will then allow consistency in future research.

Project leaders (T.Z., K.B.) set an initial goal of between 10 and 20 members, with representation from at least four programs self-described as utilizing a comprehensive longitudinal curriculum and at least four programs using a traditional block curriculum. The American Academy of Family Physicians’ (AAFP) Residency Directory classification nomenclature was utilized to create a workgroup representative of the national diversity in family medicine residency programs regarding population density (urban/suburban/rural, small/large programs) and academic affiliation (university based/university affiliated/community based). ³⁹ Geographic diversity was sought using established constructs. ⁴⁰ To test significance of differences in the composition of the workgroup and national-level family medicine program diversity, we applied extended Fisher’s exact tests. The study was determined to be Not Human Subjects Research by Prisma Health’s Institutional Review Board.

The workgroup engaged in a series of five surveys and five virtual meetings between October 2021 and March 2022. The Delphi method was utilized to arrive at expert consensus through rounds of participation while promoting anonymity and independent thought amongst participants. ^41-43 Surveys collected individual written definitions from workgroup members, and a research assistant deidentified and distributed definitions prior to each virtual meeting to blind the authorship of a particular definition (Figure 2). The deidentified definitions were reviewed during the virtual meetings to reach consensus around definition components and nuances, which then informed further rounds of definitions. Surveys were conducted using REDCap software. The project leaders led recorded 1-hour virtual meetings on Zoom. Recordings and chat transcripts were made available online so that members unable to attend could watch the group discussion. Consensus was defined as unanimous agreement among workgroup members.

Eighteen members accepted invitations to join the workgroup and completed the initial demographic survey. Thirteen workgroup members contributed consistently to the Delphi method by attending meetings and completing definition surveys. Both initial and final workgroup composition was representative of the nationwide geographic location and population density diversity of family medicine residency programs, but not academic affiliation (Table 1). The workgroup represented a diversity of program structure and program size. One workgroup member was not affiliated with a residency program but had extensive experience working with different residency programs on various scheduling models.

Longitudinal Interleaved Residency Training (LIRT) in family medicine is a curricular design and program structure that offers graduated, concurrent clinical experiences in the core competencies of the specialty. LIRT:

• Models the comprehensive scope of practice and continuity that defines the specialty.

• Applies training methods that enhance long term retention of knowledge, skills, and attitudes across all dimensions and locations of care delivery.

• Accomplishes program objectives through employment of longitudinal curricular scheduling and interleaving with spaced repetition (a process in which learners mix multiple subjects or topics frequently to improve their learning).

  a. Longitudinal: The majority of core curricular components, as determined by the program, are taught throughout at least two years of a 3-year program or three years of a 4-year program.
  b. Interleaved: The majority of core curricular components are scheduled as interleaved experiences that recur several times per year. Learners have substantial exposure to at least three different curricular components in any given 4-week span, and no single curricular component is taught exclusively for more than 2 weeks.

This consensus definition (Table 2) was meticulously crafted with deliberate word choices to highlight integral components of LIRT as a paradigm. The goal was a definition that balances generalizability for residency programs with different educational goals and patient populations while still capturing the identifying features of LIRT that clearly differentiate it from other program structures. The workgroup initially intended to use the term “integrated” rather than “interleaved,” given the use of “integrated” in undergraduate medical education, ^{21, 35-38} but with further discussion shifted to “interleaved” as it is a more specific description of the training structure and has a robust presence in the pedagogical education and psychology literature. ^{6, 7, 8, 10, 11}

First, LIRT is designated as both a curricular design and program structure. The structure of training defines which teachers and learners are together in which settings and with what frequency, thus forming the scaffolding upon which curriculum is built. “Graduated” and “concurrent” refer to the features of spaced repetition and interleaving for which operational details are provided later in the definition. The LIRT structure prioritizes continuity of care between residents and patients within and across the major domains of residency training and emphasizes continuity between the learner and core faculty who serve as teachers, role models, and evaluators.

The longitudinal nature of the training is an integral part of the definition. It allows for recurrent exposures to core content during an overall period long enough for residents’ skills and knowledge to grow in multiple domains during the interstices between exposures. For enough exposures to accumulate in a given core curricular area, the overall length of time during which exposures occur must be at least two-thirds of the total residency training duration.

The workgroup chose to require interleaving with spaced repetition by including technical criteria for frequency, concurrence, spacing, and length of curricular component exposures that are intended to be both flexible and adequately explicit. To meet LIRT criteria, a program-defined core curricular exposure should recur several times per year to provide sufficient repetition. Sufficient recurrence is crucial to the growth of a learner’s baseline accessible clinical knowledge and skills and avoids the pattern of ingurgitating content and subsequently forgetting it when it is no longer required.

Interleaving was operationalized by specifying that at least three different components should take place within a 4-week span, which is differentiated from the traditional block model of one major rotation with scattered half-days of continuity clinic (constituting two core curricular components). Additionally, no single curricular component should be taught exclusively (eg, without mixing with another core curricular exposure) for more than 2 weeks, to add structure for the sufficient mixing of topics.

Consideration was given to specifying minimum quantities of time (eg, hours, half-days, etc) for each exposure to be considered substantial, but the workgroup realized that, depending on the clinical area and setting, there were a variety of ways to create a substantial educational exposure. The term “substantial” was retained to represent a need for the exposure to require significant engagement of the learner with the clinical material.

The workgroup valued inclusivity where possible, allowing flexibility to encompass curricula that fit within the defined structure while acknowledging that a variety of institutional constraints and training goals exist across a specialty. While striving to be explicit and create a functional working definition, the workgroup avoided being overly prescriptive around logistical details such that it created barriers to meeting the definition if the core intent of interleaving remained intact. The workgroup conceptualized LIRT in the context of family medicine but took pains to craft the definition in such a way that it could be transferable to other specialties, with each specialty substituting relevant overarching rationale and core curricular components as necessary.

Limitations include that the definition was arrived at via dialogue and consensus without the availability of any pre-existing technical definitions in the context of residency curricula. Care was taken to bring together a diverse group of family medicine educators. A larger group would have been more unwieldy, but it is possible that important input was missed by virtue of the limited workgroup membership.

In response to past calls for clarity on the subject, a representative national workgroup was convened and crafted a consensus definition of LIRT in family medicine, a program structure with a basis in emerging evidence-based cognitive science.

With a consensus definition of LIRT in hand, the workgroup will now turn to establishing a national baseline prevalence of this program structure in the specialty of family medicine, potentially utilizing the Council for Academic Family Medicine’s (CAFM) Education Research Alliance (CERA) survey platform. Work is already underway to re-envision family medicine residency training in order to better prepare primary care physicians for the practice of the future. ^44-46 Subsequent high-quality comparative effectiveness studies measuring outcomes such as in-training examination scores, board examination pass rates, readiness for practice at graduation, and satisfaction with training could lead to the emergence of LIRT as an approach capable of producing physicians better prepared to handle the medical complexity and variety that physicians routinely face in their roles within a community.