Longitudinal Interleaved Residency Training: A Consensus Definition

T. Aaron Zeller, MD | Katherine Beben, MD | Marianna Kong, MD | Ildi Martonffy, MD | Seth Patterson, MD | Weldon Deas, MD | Moonseong Heo, PhD | Drew M. Keister, MD

Fam Med. 2023;55(5):311-316.

DOI: 10.22454/FamMed.2023.378423

Return to Issue


Background and Objectives: Cognitive benefits of longitudinal curricula and interleaving have been demonstrated in several disciplines. However, most residency curricula are structured in a block format. There is no consensus definition as to what constitutes a longitudinal program, making comparative research on curricular efficacy a challenge. The objective of our study was to arrive at a consensus definition of Longitudinal Interleaved Residency Training (LIRT) in family medicine.

Methods: A national workgroup was convened and utilized a Delphi method between October 2021 and March 2022 to arrive at a consensus definition.

Results: Twenty-four invitations were sent, and 18 participants initially accepted. The final workgroup (n=13) was representative of the nationwide diversity of family medicine residency programs in terms of geographic location (P=.977) and population density (P=.123). The following definition was approved: “LIRT is a curricular design and program structure that offers graduated, concurrent clinical experiences in the core competencies of the specialty. LIRT models the comprehensive scope of practice and continuity that defines the specialty; applies training methods that enhance long-term retention of knowledge, skills, and attitudes across all dimensions and locations of care delivery; and accomplishes program objectives through employment of longitudinal curricular scheduling and interleaving with spaced repetition.” Additional technical criteria and definitions of terms are elucidated in the body of this article.

Conclusions: A representative national workgroup crafted a consensus definition of Longitudinal Interleaved Residency Training (LIRT) in family medicine, a program structure with a basis in emerging evidence-based cognitive science.


There has been much discussion in graduate medical education over the best way to structure residency training to optimally train physicians for the future health care workforce. 1, 2 Through the years, some programs have departed from the traditional structure of 1-month block rotations and have moved toward a longitudinal approach to their curriculum. 3, 4 Recent advancements in cognitive science lend credence to the notion that longitudinal curricula may have advantages over a block structure in that repetitive exposure to the same content distributed over time leads to enhanced knowledge retention compared to closely timed massed learning. 5- 9 This spaced, or distributed, learning has been demonstrated to be effective not only in medical education but also in many other fields. 10

Taking spaced exposure further, interleaving is a process in which learners switch between multiple related subjects instead of mastering one topic thoroughly prior to moving onto the next topic (Figure 1 ). 11 Interleaving allows learners to compare and contrast new concepts, creating essential connections that would be missed by studying each component independently. Studies have demonstrated that interleaving is more effective than blocked practice in helping learners differentiate, or discriminate, among similar concepts in math, art, sports, and other disciplines, thereby improving memory associations. 7 Interleaving has specifically been shown to improve inductive reasoning, 12, 13 a skill physicians use to make predictions about novel cases based on existing knowledge. 14 While evidence suggests benefits from interleaving, 15-19 the technique remains understudied and underutilized at many levels of education, including graduate medical education. 20, 21

In 2001, several articles in Family Medicine introduced the idea of a longitudinal approach to structuring residency training. 22-26 The tenets of this paradigm emphasized increased continuity of care by scheduling residents in clinic nearly every day, supplemented by intense focused experiences in the hospital and other practice settings. 22 The proposed benefits of a longitudinal curriculum included more role modeling by family physicians, increased continuity of care across the continuum, more emphasis on clinic and primary care, and the practice of lifelong learning. A program director survey found that 3.6% of respondents described their curriculum as “mostly longitudinal” and 15% indicated interest in moving toward that style in the next 2 years. 23 Evidence showed that structured repetitive learning experiences improved in-training exam scores significantly 24 and improved patient access to their assigned primary care physician. 26 A critique of these studies highlighted the need to standardize the definition of longitudinal curriculum before outcomes could be analyzed effectively in terms of resident training, knowledge, and performance. 22

Despite advancements in cognitive science supporting longitudinal and interleaved education and the call for more research about longitudinal curricula in graduate medical education more than 20 years ago, no standardized definition has emerged. Published studies in family medicine are scarce and have focused on applying a longitudinal structure to specific aspects of the curriculum rather than the entire residency structure but conclude that a longitudinal format increases resident knowledge. 27-29 More recently, however, descriptions of Clinic First 3, 30, 31 and High-Functioning Primary Care Residency Clinics 32, 33 have renewed interest in the clinic as the curriculum in primary care, and in longitudinal approaches as a means to increase time in clinic. A review of evidence-based education discussed the potential advantages of interleaving in anesthesiology residency, 5 and spaced repetition implemented in urology residency increased transfer and long-term retention of diagnostic skills. 34 As such, we believe that longitudinal, interleaved residency training can be implemented in many disciplines once a standardized definition has been accepted.

To this end, we describe the results of a national workgroup that undertook an exercise in definition of Longitudinal Interleaved Residency Training (LIRT), a nomenclature consistent with Longitudinal Integrated Clerkships (LIC) being implemented in undergraduate medical education globally. 35-38 The objective was to arrive at a standardized, functional definition of the term that will then allow consistency in future research.


Workgroup Formation

Project leaders (T.Z., K.B.) set an initial goal of between 10 and 20 members, with representation from at least four programs self-described as utilizing a comprehensive longitudinal curriculum and at least four programs using a traditional block curriculum. The American Academy of Family Physicians’ (AAFP) Residency Directory classification nomenclature was utilized to create a workgroup representative of the national diversity in family medicine residency programs regarding population density (urban/suburban/rural, small/large programs) and academic affiliation (university based/university affiliated/community based). 39 Geographic diversity was sought using established constructs. 40 To test significance of differences in the composition of the workgroup and national-level family medicine program diversity, we applied extended Fisher’s exact tests. The study was determined to be Not Human Subjects Research by Prisma Health’s Institutional Review Board.

Workgroup Process

The workgroup engaged in a series of five surveys and five virtual meetings between October 2021 and March 2022. The Delphi method was utilized to arrive at expert consensus through rounds of participation while promoting anonymity and independent thought amongst participants. 41-43 Surveys collected individual written definitions from workgroup members, and a research assistant deidentified and distributed definitions prior to each virtual meeting to blind the authorship of a particular definition (Figure 2). The deidentified definitions were reviewed during the virtual meetings to reach consensus around definition components and nuances, which then informed further rounds of definitions. Surveys were conducted using REDCap software. The project leaders led recorded 1-hour virtual meetings on Zoom. Recordings and chat transcripts were made available online so that members unable to attend could watch the group discussion. Consensus was defined as unanimous agreement among workgroup members.


Workgroup Composition

Eighteen members accepted invitations to join the workgroup and completed the initial demographic survey. Thirteen workgroup members contributed consistently to the Delphi method by attending meetings and completing definition surveys. Both initial and final workgroup composition was representative of the nationwide geographic location and population density diversity of family medicine residency programs, but not academic affiliation (Table 1). The workgroup represented a diversity of program structure and program size. One workgroup member was not affiliated with a residency program but had extensive experience working with different residency programs on various scheduling models.

Consensus Definition

Longitudinal Interleaved Residency Training (LIRT) in family medicine is a curricular design and program structure that offers graduated, concurrent clinical experiences in the core competencies of the specialty. LIRT:

  • Models the comprehensive scope of practice and continuity that defines the specialty.

  • Applies training methods that enhance long term retention of knowledge, skills, and attitudes across all dimensions and locations of care delivery.

  • Accomplishes program objectives through employment of longitudinal curricular scheduling and interleaving with spaced repetition (a process in which learners mix multiple subjects or topics frequently to improve their learning).

    a. Longitudinal: The majority of core curricular components, as determined by the program, are taught throughout at least two years of a 3-year program or three years of a 4-year program.
    b. Interleaved: The majority of core curricular components are scheduled as interleaved experiences that recur several times per year. Learners have substantial exposure to at least three different curricular components in any given 4-week span, and no single curricular component is taught exclusively for more than 2 weeks.


This consensus definition (Table 2) was meticulously crafted with deliberate word choices to highlight integral components of LIRT as a paradigm. The goal was a definition that balances generalizability for residency programs with different educational goals and patient populations while still capturing the identifying features of LIRT that clearly differentiate it from other program structures. The workgroup initially intended to use the term “integrated” rather than “interleaved,” given the use of “integrated” in undergraduate medical education, 21, 35-38 but with further discussion shifted to “interleaved” as it is a more specific description of the training structure and has a robust presence in the pedagogical education and psychology literature. 6, 7, 8, 10, 11

First, LIRT is designated as both a curricular design and program structure. The structure of training defines which teachers and learners are together in which settings and with what frequency, thus forming the scaffolding upon which curriculum is built. “Graduated” and “concurrent” refer to the features of spaced repetition and interleaving for which operational details are provided later in the definition. The LIRT structure prioritizes continuity of care between residents and patients within and across the major domains of residency training and emphasizes continuity between the learner and core faculty who serve as teachers, role models, and evaluators.

The longitudinal nature of the training is an integral part of the definition. It allows for recurrent exposures to core content during an overall period long enough for residents’ skills and knowledge to grow in multiple domains during the interstices between exposures. For enough exposures to accumulate in a given core curricular area, the overall length of time during which exposures occur must be at least two-thirds of the total residency training duration.

The workgroup chose to require interleaving with spaced repetition by including technical criteria for frequency, concurrence, spacing, and length of curricular component exposures that are intended to be both flexible and adequately explicit. To meet LIRT criteria, a program-defined core curricular exposure should recur several times per year to provide sufficient repetition. Sufficient recurrence is crucial to the growth of a learner’s baseline accessible clinical knowledge and skills and avoids the pattern of ingurgitating content and subsequently forgetting it when it is no longer required.

Interleaving was operationalized by specifying that at least three different components should take place within a 4-week span, which is differentiated from the traditional block model of one major rotation with scattered half-days of continuity clinic (constituting two core curricular components). Additionally, no single curricular component should be taught exclusively (eg, without mixing with another core curricular exposure) for more than 2 weeks, to add structure for the sufficient mixing of topics.

Consideration was given to specifying minimum quantities of time (eg, hours, half-days, etc) for each exposure to be considered substantial, but the workgroup realized that, depending on the clinical area and setting, there were a variety of ways to create a substantial educational exposure. The term “substantial” was retained to represent a need for the exposure to require significant engagement of the learner with the clinical material.

The workgroup valued inclusivity where possible, allowing flexibility to encompass curricula that fit within the defined structure while acknowledging that a variety of institutional constraints and training goals exist across a specialty. While striving to be explicit and create a functional working definition, the workgroup avoided being overly prescriptive around logistical details such that it created barriers to meeting the definition if the core intent of interleaving remained intact. The workgroup conceptualized LIRT in the context of family medicine but took pains to craft the definition in such a way that it could be transferable to other specialties, with each specialty substituting relevant overarching rationale and core curricular components as necessary.

Limitations include that the definition was arrived at via dialogue and consensus without the availability of any pre-existing technical definitions in the context of residency curricula. Care was taken to bring together a diverse group of family medicine educators. A larger group would have been more unwieldy, but it is possible that important input was missed by virtue of the limited workgroup membership.


In response to past calls for clarity on the subject, a representative national workgroup was convened and crafted a consensus definition of LIRT in family medicine, a program structure with a basis in emerging evidence-based cognitive science.

With a consensus definition of LIRT in hand, the workgroup will now turn to establishing a national baseline prevalence of this program structure in the specialty of family medicine, potentially utilizing the Council for Academic Family Medicine’s (CAFM) Education Research Alliance (CERA) survey platform. Work is already underway to re-envision family medicine residency training in order to better prepare primary care physicians for the practice of the future. 44-46 Subsequent high-quality comparative effectiveness studies measuring outcomes such as in-training examination scores, board examination pass rates, readiness for practice at graduation, and satisfaction with training could lead to the emergence of LIRT as an approach capable of producing physicians better prepared to handle the medical complexity and variety that physicians routinely face in their roles within a community.


The Longitudinal Interleaved Residency Training Definition Workgroup includes the authors listed and Elizabeth Baxley, John Emerson, James Jarvis, Carl Morris, Viviane Sachs, Andrew Slattengren and Keith Stelter.


  1. Rosenblum MJ, Hinchey KT. Rapid resident cycling: the 14 day mini-block. Acad Intern Med Insight. 2009;7(4):10-11.
  2. Weinberger SE, Smith LG, Collier VU; Education Committee of the American College of Physicians. Redesigning training for internal medicine. Ann Intern Med. 2006;144(12):927-932. doi:10.7326/0003-4819-144-12-200606200-00124
  3. Barnes K, Morris CG. Clinic first: prioritizing primary care outpatient training for family medicine residents at Group Health Cooperative. J Gen Intern Med. 2015;30(10):1557-1560. doi:10.1007/s11606-015-3272-z

  4. Warm EJ, Schauer DP, Diers T, et al. The ambulatory long-block: an accreditation council for graduate medical education (ACGME) educational innovations project (EIP). J Gen Intern Med. 2008;23(7):921-926. doi:10.1007/s11606-008-0588-y
  5. Martinelli SM, Isaak RS, Schell RM, Mitchell JD, McEvoy MD, Chen F. Learners and Luddites in the Twenty-first Century: Bringing Evidence-based Education to Anesthesiology. Anesthesiology. 2019;131(4):908-928. doi:10.1097/ALN.0000000000002827
  6. Roediger HL, Pyc MA. Inexpensive techniques to improve education: applying cognitive psychology to enhance educational practice. J Appl Res Mem Cogn. 2012;1(4):242-248. doi:10.1016/j.jarmac.2012.09.002
  7. Rohrer D. Interleaving helps students distinguish among similar concepts. Educ Psychol Rev. 2012;24(3):355-367. doi:10.1007/s10648-012-9201-3
  8. Brown PC, Roediger HL, McDaniel MA. Make It Stick: The Science of Successful Learning. Harvard University Press; 2014.
  9. Sundar KR. The Gift of Empanelment in a “Clinic First” Residency. Ann Fam Med. 2018;16(6):563-565. doi:10.1370/afm.2307
  10. Schmidt HG, Mamede S. How cognitive psychology changed the face of medical education research. Adv Health Sci Educ Theory Pract. 2020;25(5):1025-1043. doi:10.1007/s10459-020-10011-0
  11. The University of Arizona. L2L strategy - Interleaving. Academic Affairs. Published July 25, 2019. Accessed November 15, 2021. https://academicaffairs.arizona.edu/l2l-strategy-interleaving
  12. Kornell N, Bjork RA. Learning concepts and categories: is spacing the “enemy of induction”? Psychol Sci. 2008;19(6):585-592. doi:10.1111/j.1467-9280.2008.02127.x
  13. Birnbaum MS, Kornell N, Bjork EL, Bjork RA. Why interleaving enhances inductive learning: the roles of discrimination and retrieval. Mem Cognit. 2013;41(3):392-402. doi:10.3758/s13421-012-0272-7
  14. Hayes BK, Heit E. Inductive reasoning 2.0. Wiley Interdiscip Rev Cogn Sci. 2018;9(3):e1459. doi:10.1002/wcs.1459
  15. Carvalho PF, Goldstone RL. The benefits of interleaved and blocked study: different tasks benefit from different schedules of study. Psychon Bull Rev. 2015;22(1):281-288. doi:10.3758/s13423-014-0676-4
  16. Eglington LG, Kang SHK. Interleaved Presentation Benefits Science Category Learning. J Appl Res Mem Cogn. 2017;6(4):475-485. doi:10.1016/j.jarmac.2017.07.005

  17. Carpenter SK, Mueller FE. The effects of interleaving versus blocking on foreign language pronunciation learning. Mem Cognit. 2013;41(5):671-682. doi:10.3758/s13421-012-0291-4
  18. Taylor K, Rohrer D. The effects of interleaved practice. Appl Cogn Psychol. 2010;24(6):837-848. doi:10.1002/acp.1598
  19. Cepeda NJ, Pashler H, Vul E, Wixted JT, Rohrer D. Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychol Bull. 2006;132(3):354-380. doi:10.1037/0033-2909.132.3.354
  20. Brunmair M, Richter T. Similarity matters: A meta-analysis of interleaved learning and its moderators. Psychol Bull. 2019;145(11):1029-1052. doi:10.1037/bul0000209
  21. Pan S. The Interleaving Effect: Mixing It Up Boosts Learning. Scientific American. Published online August 4, 2015. Accessed November 15, 2021. https://www.scientificamerican.com/article/the-interleaving-effect-mixing-it-up-boosts-learning/ 
  22. Weiss BD. Longitudinal residency training in family medicine: not ready for prime time. Fam Med. 2001;33(10):762-765.
  23. Reust CE. Longitudinal residency training: a survey of family practice residency programs. Fam Med. 2001;33(10):740-745.
  24. Steinweg KK, Cummings DM, Kelly SK. Are some subjects better taught in block rotation? A geriatric experience. Fam Med. 2001;33(10):756-761.
  25. Merenstein D, Damico F, Devine B, et al. Longitudinal versus traditional residencies: a study of continuity of care. Fam Med. 2001;33(10):746-750.
  26. Neher JO, Kelsberg G, Oliveira D. Improving continuity by increasing clinic frequency in a residency setting. Fam Med. 2001;33(10):751-755.
  27. Knox KE, Lehmann W, Vogelgesang J, Simpson D. Community Health, Advocacy, and Managing Populations (CHAMP) longitudinal residency education and evaluation. J Patient Cent Res Rev. 2018;5(1):45-54. doi:10.17294/2330-0698.1580
  28. Schiefer R, Devlaeminck AV, Hofkamp H, Levy S, Sanchez D, Muench J. A Family Systems Curriculum: back to the Future? Fam Med. 2017;49(7):558-562.
  29. Jacobs C, Seehaver A, Skiold-Hanlin S. A Longitudinal Underserved Community Curriculum for Family Medicine Residents. Fam Med. 2019;51(1):48-54. doi:10.22454/FamMed.2019.320104
  30. Gupta R, Barnes K, Bodenheimer T. Clinic First: 6 Actions to Transform Ambulatory Residency Training. J Grad Med Educ. 2016;8(4):500-503. doi:10.4300/JGME-D-15-00398.1

  31. Kim JG, Morris CG, Ford P. Teaching Today in the Practice Setting of the Future: Implementing Innovations in Graduate Medical Education. Acad Med. 2017;92(5):662-665. doi:10.1097/ACM.0000000000001510
  32. Bodenheimer T, Gupta R, Dubé K, et al. High-functioning Primary Care Residency Clinics: Building Blocks for Providing Excellent Care and Training. Association of American Medical Colleges; 2016.
  33. Profiles of Three High-Performing Primary Care Residency Clinics. Association of American Medical Colleges; 2018. Accessed December 2, 2022. https://cepc.ucsf.edu/sites/cepc.ucsf.edu/files/aamc-profiles-three-high-performing-primary-care-residency-clinics.pdf
  34. Kerfoot BP, Fu Y, Baker H, Connelly D, Ritchey ML, Genega EM. Online spaced education generates transfer and improves long-term retention of diagnostic skills: a randomized controlled trial. J Am Coll Surg. 2010;211(3):331-337.e1. doi:10.1016/j.jamcollsurg.2010.04.023
  35. Norris TE, Schaad DC, DeWitt D, Ogur B, Hunt DD; Consortium of Longitudinal Integrated Clerkships. Longitudinal integrated clerkships for medical students: an innovation adopted by medical schools in Australia, Canada, South Africa, and the United States. Acad Med. 2009;84(7):902-907. doi:10.1097/ACM.0b013e3181a85776
  36. Hudson JN, Poncelet AN, Weston KM, Bushnell JA, A Farmer E. Longitudinal integrated clerkships. Med Teach. 2017;39(1):7-13, 7-13. doi:10.1080/0142159X.2017.1245855
  37. Worley P, Couper I, Strasser R, et al; Consortium of Longitudinal Integrated Clerkships (CLIC) Research Collaborative. A typology of longitudinal integrated clerkships. Med Educ. 2016;50(9):922-932. doi:10.1111/medu.13084
  38. Walters L, Greenhill J, Richards J, et al. Outcomes of longitudinal integrated clinical placements for students, clinicians and society. Med Educ. 2012;46(11):1028-1041. doi:10.1111/j.1365-2923.2012.04331.x
  39. AAFP Residency Directory. American Academy of Family Physicians. Accessed July 15, 2021. https://www.aafp.org/medical-education/directory/residency/search
  40. Shappell CN, Farnan JM, McConville JF, Martin SK. Geographic Trends for United States Allopathic Seniors Participating in the Residency Match: a Descriptive Analysis. J Gen Intern Med. 2019;34(2):179-181. doi:10.1007/s11606-018-4686-1
  41. Dalkey N, Helmer O. An experimental application of the Delphi method to the use of experts. Manage Sci. 1963;9(3):458-467. doi:10.1287/mnsc.9.3.458
  42. Linstone HA, Turoff M. The Delphi Method: Techniques and Applications. Addison-Wesley Publishing Company; 1975.
  43. Jünger S, Payne SA, Brine J, Radbruch L, Brearley SG. Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: recommendations based on a methodological systematic review. Palliat Med. 2017;31(8):684-706. doi:10.1177/0269216317690685
  44. Newton WP, Mitchell KB, Magill MK. The Future of Family Medicine Residency Education: The Specialty Has Spoken. Ann Fam Med. 2021;19(2):185-187. doi:10.1370/afm.2677
  45. Newton WP, Magill M, Biggs W, et al. Re-Envisioning Family Medicine Residencies: The End in Mind. J Am Board Fam Med. 2021;34(1):246-248. doi:10.3122/jabfm.2021.01.200604
  46. Newton WP, Bazemore A, Magill M, Mitchell K, Peterson L, Phillips RL. The Future of Family Medicine Residency Training is Our Future: A Call for Dialogue Across Our Community. J Am Board Fam Med. 2020;33(4):636-640. doi:10.3122/jabfm.2020.04.200275

Lead Author

T. Aaron Zeller, MD

Affiliations: Prisma Health/University of South Carolina School of Medicine–Greenville (Seneca) Family Medicine Residency, Greenville, SC | Prisma Health, Greenville, SC | Clemson University School of Health Research (CUSHR), Clemson, SC | University of South Carolina School of Medicine–Greenville, Greenville, SC


Katherine Beben, MD - Prisma Health/University of South Carolina School of Medicine–Greenville (Seneca) Family Medicine Residency, Greenville, SC | Prisma Health, Greenville, SC | University of South Carolina School of Medicine–Greenville, Greenville, SC

Marianna Kong, MD - University of California San Francisco

Ildi Martonffy, MD - University of Wisconsin Department of Family Medicine and Community Health, Madison, WI

Seth Patterson, MD - Prisma Health, Greenville, SC | University of South Carolina School of Medicine–Greenville, Greenville, SC | Prisma Health/University of South Carolina School of Medicine-Greenville (Greer) Family Medicine Residency, Greenville, SC

Weldon Deas, MD - Prisma Health/University of South Carolina School of Medicine–Greenville (Seneca) Family Medicine Residency, Greenville, SC | Prisma Health, Greenville, SC

Moonseong Heo, PhD - Department of Public Health Sciences, Clemson University, Clemson, SC

Drew M. Keister, MD - Lehigh Valley Health Network Department of Family Medicine, Allentown, PA | University of South Florida Morsani College of Medicine, Tampa, FL

Corresponding Author

T. Aaron Zeller, MD

Correspondence: Prisma Health/University of South Carolina School of Medicine–Greenville (Seneca) Family Medicine Residency, Greenville, SC; Prisma Health, Greenville, SC; Clemson University School of Health Research (CUSHR), Clemson, SC; University of South Carolina School of Medicine–Greenville, Greenville, SC

Email: aaron.zeller@prismahealth.org

Fetching other articles...

Loading the comment form...

Submitting your comment...

There are no comments for this article.

Downloads & Info


Related Content