Background and Objectives: The optimal length of residency training in family medicine is under debate. This study compared applicant type, number of applicants, match positions filled, matched applicant type, and ranks to fill between 3-year (3YR) and 4-year (4YR) residencies.
Methods: The Length of Training Pilot (LOTP) is a case-control study comparing 3YR (seven residencies) and 4YR (six residencies) training models. We collected applicant and match data from LOTP programs from 2012 to 2018 and compared data between 3YR and 4YR programs. National data provided descriptive comparisons. An annual resident survey captured resident perspectives on training program selection. Summary statistics and corresponding t-tests and χ2 tests of independence were performed to assess differences between groups. We used a linear mixed model to account for repeated measures over time within programs.
Results: There were no differences in the mean number of US MD, US DO, and international medical graduate applicants between 3YR and 4YR programs. Both the 3YR and 4YR programs had a substantially higher number of US MD and DO applicants compared to national averages. The percentages of positions filled in the match and positions filled by US MDs, DOs and IMGs were not different between groups. The percentage of residents in 4YR programs who think training in family medicine requires a fourth year varied significantly during the study period, from 35% to 25% (P<.001). The predominant reasons for pursuing training in a 4YR program was a desire for more flexibility in training and a desire to learn additional skills beyond clinical skills.
Conclusions: The applicant pool and match performance of the residencies in the LOTP was not affected by length of training. Questions yet to be addressed include length of training’s impact on medical knowledge, scope of practice, and clinical preparedness.
The optimal length of residency training needed to best prepare a family physician to function in both current and future health care environments is under debate.1 Reasons proposed to extend training to 4 years include inadequate clinical preparation during medical school,2-5 reduction in available educational time due to duty hour restrictions,6-8 and new skills needed to practice in complex health systems while providing patient-centered care.9-10 Despite these arguments for adding a year, concerns remain that doing so will further erode student interest in choosing family medicine at a time when fewer students are entering primary care disciplines.11 Additionally, some educators advocate for more flexibility in residency education to address clinical preparedness rather than more time.12-13
Student and resident interest in a 4-year (4YR) residency has been studied over the past few decades. In 2002, the concept of a 4YR family medicine residency was supported by 32% of first-year residents when surveyed shortly after starting training.14 A 2006 survey of residency applicants indicated that the addition of another year of training would have a neutral or positive effect on choosing family medicine.15 More recently, a 2014 study found that slightly less than half of graduating family medicine residents expressed interest in a fourth year of training.16 Additionally, a study of over 6,000 soon-to-graduate residents in 2014 and 2015 found little overlap between residents intending fellowship training and those who would consider a longer residency, suggesting that the desire to pursue a fellowship does not necessarily equate to a desire for more core training.17
The residency placement process has been undergoing marked change in the past decade, with the number of applications submitted and the number of residencies ranked per US senior applicant increasing significantly.18 The average number of applications to family medicine programs per seniors from US allopathic medical schools has risen from 18.7 in 2012 to 33.3 in 2017, a 78% increase.19 Despite this increase in the number of programs each applicant applies to, match rates for allopathic US graduates remain steady and unchanged.18,20
Research requirements, rural training tracks, curriculum innovations, and individualized education have been studied as factors that may affect program match performance in family medicine through the National Resident Matching Program (NRMP).21-23 The only previous report of the effect of a 4YR training model on applicants and the match was from a single program in the Preparing the Personal Physician for Practice (P4) Project that reported a 77% increase in the number of US senior applicants between 2007 and 2011, with no adverse effect on match performance.24
The purpose of this study is to determine, among programs enrolled in the Length of Training Pilot (LOTP) in family medicine, what effect length of training has on applicant and match variables (applicant type, number of applicants, match positions filled, matched applicant type, ranks to fill). We specifically explored the hypothesis that no differences in applicant and match outcomes between 4YR programs and 3-year (3YR) programs would be found.
The LOTP is a prospective case-control study (2013-2022) exploring the influence of length of family medicine residency training on several learner and program outcomes.1,25 Evaluation of the pilot is overseen by a team of educational researchers in the Department of Family Medicine at Oregon Health & Science University (OHSU). Residency programs that had or were planning to transition to a 4YR model applied for the pilot in 2012. Selection was based on components of programs’ innovations and designs of their site-specific evaluation plans. Four of the selected 4YR programs had a required full 4 years of training and two had an optional additional fourth year of training that residents could choose to undertake, the timing of which varied. Enrolled 4YR programs were matched to 3YR control programs based on size, region, and clinical setting. The LOTP includes 17 residencies (seven 3YR civilian programs, six 4YR civilian programs, and four military programs with a 4YR track). The military programs were excluded from these analyses because residency applicant and selection processes are quite different from those used by civilian residencies. Because of the size of one 4YR program, two 3YR programs were matched to it to ensure equivalent numbers of residents in 3YR and 4YR groups.
All LOTP programs obtained Institutional Review Board (IRB) approval with waivers of documented consent or exemptions to participate in the study as well as to conduct program-specific evaluations at their respective institutions. OHSU’s IRB granted the evaluation team an educational exemption to obtain data from the study sites (IRB #9770).
Data Collection and Management
Match and applicant data were collected from multiple sources. Participating programs provided data for years 2012-2018 on applicants (type, interviews completed, number of applicants ranked) and match results (positions filled, matched applicant type). Applicants were categorized as graduates of allopathic schools (US MD), graduates of osteopathic schools (US DO), or international medical graduates (IMG)—both US and foreign citizenship. These data were collected annually immediately following Match Day and then verified, cleaned, and entered into a secure database. Programs provided retrospective data for 2012 and 2013 immediately following the 2014 match. Comparative national data were obtained from the Electronic Residency Application Service (ERAS)26 and the NRMP27 for years 2012-2018. Published data from ERAS and NRMP includes only aggregate data so we were unable to undertake statistical comparisons of national data with LOTP programs.
Resident characteristics, number of programs where they interviewed, number of programs ranked, and perspectives on training program selection were obtained from an annual resident survey administered in the fall of each year to all residents in LOTP programs. The 29-question instrument was updated from the P4 Resident Survey to reflect current trends in family medicine training and attitudes on 3 versus 4 years of training. The survey was revised accordingly after its first year of use in the study, which served as a pilot test of the items. Variables included in these analyses were from 2015 and later. We used survey responses from postgraduate year 1 (PGY-1) residents only since this group was closest to the applicant and match experience. Response rates to these surveys ranged from 88% to 96% for 3YR programs, and from 97% to 98% for 4YR programs.
Summary statistics including means, standard deviations, frequencies, and corresponding t-tests and χ2 tests of independence were performed to assess differences between 3-year and 4-year residents and resident perspectives on residency programs.
To estimate trends in applicant pool and match performance, means and standard deviations were calculated for each year from 2012-2018 by program type. To evaluate if 3- vs 4-year programs differed in trends over time, we utilized a linear mixed model to account for repeated measures over time within programs. For each outcome, we considered fixed effects of program type (3 vs 4 years), a set of indicators for categorical years and their interaction. The interaction terms identified any potential differences in mean outcomes over time (eg, interaction terms identified any deviation of parallel trends over time where P values >.05 suggest similar trends over time between 3- and 4-year programs). We included a random effect of program ID to account for temporal correlation of outcome measures within each program.
For a few outcomes (US MD applicants, US DO applicants, IMG applicants, and ranks to fill per position offered in the match), we observed missing outcome data for some years. To account for missingness, we performed multiple imputations by chained equations (MICE) to maximize all observed data without removing programs with missing data. For the MICE procedure, we included as important covariates: number of US senior applicants, percent of positions filled by US seniors (MDs and DOs) and IMGs, region of the country, single vs multiple specialty residency status, continuity clinic in a community health center, number of interviews conducted, total number of applicants, and total percent of positions filled. For all imputed models, we performed 20 imputations with 100 iterations and used Rubin’s Rules to combine parameter estimates.28 For these imputed outcomes, we do not report standard deviations, as they are not recommended for reporting when producing estimates by MICE.29
In addition, we conducted a sensitivity analysis where one analysis was run with all six 4YR programs included, and a second analysis was run with the two programs that had optional additional training removed from the data set. This allowed us to evaluate the impact that an optional additional training had on study outcomes. Because the sensitivity analysis showed only minor changes to the results, none of which affected statistical significance, the data presented here include all six programs that offered additional training beyond the traditional 3 years. All analyses were performed using Stata version 15. All statistical tests were two-sided and type I error was set at 5%.
Six of the seven 3YR programs are community-based and four of the six 4YR programs are community-based, the remainder being university-based. The characteristics of the PGY-1 residents in 3YR (n=254) and 4YR programs (n=308) in the LOTP are presented in Table 1. We found no statistical differences between residents in 3YR and 4YR programs for age, race, ethnicity, marital status, parental status, or attending a US medical school. The percentage of women in 3YR programs was significantly higher than 4YR programs (68.5% vs 56.5%, P=.017).
Table 2 illustrates trends in the applicant pool for 3YR and 4YR programs from 2012 to 2018 as well as national comparator data from the ERAS system. There were no differences in the mean number of US MD applicants between 3YR and 4YR programs, and the upward trend in both groups parallels the national trend over the 7-year study period. Both the 3YR and 4YR programs had a substantially higher number of applicants from both allopathic and osteopathic schools compared to national averages. There was variability in the mean number of US DO applicants and IMG applicants during the study period in both 3YR and 4YR programs, with no significant differences between the two groups. There were no statistical differences in the total number of applicants per program between the two groups.
Data from the NRMP (Table 3) show no statistical differences for positions filled in the match between 3YR and 4YR programs. The mean percentage of positions filled by US MDs, US DOs, and IMGs was not different for 3YR vs 4YR programs. Both 3YR and 4YR programs exceeded the national averages for percent of positions filled by US MDs in all study years except 2012. The only significant difference between groups was in the mean number of ranks to fill per position offered in the match (a higher ranks to fill per position indicates the program went farther down the match list to fill its last position) with the 3YR programs having higher means in 2012, 2013, and 2014, and the 4YR programs having higher means in the last 4 years of data collection (P=.027).
PGY-1 resident views of the residency selection process from residents in 3YR vs 4YR programs (Table 4) illustrate no statistical differences in resident responses from 2014 to 2017 in how they considered length of training in the interview and match process. Notably, one-third of residents in 3YR programs in 2014 to 2016, and nearly half in 2017 had also interviewed at 4YR programs. Residents in 3YR and 4YR programs ranked a similar number of 3YR programs. There was a significant upward trend in the percentage of residents in 3YR programs who did not think a fourth year of training is necessary in family medicine (P<.001). The percentage of residents in 4YR programs who think training in family medicine requires a fourth year varied significantly from 35% to 25% over the years analyzed (P<.001). The predominant reason each year for pursuing training in a 4YR program was a desire for more flexibility in training, followed by a desire to learn additional skills beyond clinical skills. The percentage of residents in 4YR programs who considered doing a fellowship after 3 years but chose a 4YR program because of a preference to gain skills in an area of concentration through a different model varied between 36% and 43% without a significant trend across years.
Our findings confirm our hypothesis that the length of training does not adversely affect applicant and match outcomes in LOTP programs, and thus offering 4 years vs 3 years of training does not appear to adversely affect the applicant or match pool of potential candidates. We found similar fill rates and similar percentages of applicants and filled positions who were US MDs and US DOs in the 3YR and 4YR programs in the LOTP. Notably, the number of applicants from both allopathic and osteopathic schools was higher in the LOTP programs compared to national averages, indicating that this group of residencies may be historically more competitive or attractive to students. It is possible that any differences between 3YR and 4YR applications and match outcomes were muted because of the quality and appeal of both groups of programs and confounded by other appealing program attributes and strengths.
Some significant changes in the national graduate medical education (GME) environment occurred during our period of data collection and are worth noting. First, over the past decade, students from allopathic schools were applying to a larger number of residencies because too few slots exist for the number of medical school graduates.18 The LOTP programs mirrored this upward national trend. The second notable national trend occurred in 2014 when the Accreditation Council for Graduate Medical Education (ACGME), the American Osteopathic Association (AOA), and the American Association of Colleges of Osteopathic Medicine (AACOM) resolved to institute a single, unified GME accreditation system by July 1, 2020.30 In family medicine, this has increased the overall number of ACGME residencies, as AOA-only accredited programs are now receiving ACGME accreditation through the Single Accreditation System process.31 Subsequently, programs are seeing an increase in number of DO applicants. This trend was apparent in LOTP programs regardless of length of training, especially in 2017 and 2018.
In the LOTP, residents in both 3YR and 4YR programs reported that factors other than length of training were more important in their decision making about program selection. This is consistent with other reports demonstrating that factors such as location, work/life balance, resident satisfaction and curriculum are more often selected as influencing residency ranking.32,33 A notable finding is that 4YR programs have statistically fewer women residents. It is not known whether this difference is due to longer training being less attractive to women or other program features of the 4YR residencies that lead to more men than women matching with these particular programs. Further study is needed to look carefully at outcomes in the LOTP to determine if any gender differences are emerging.
In a study of family medicine residents about to graduate in 2014, approximately one-fifth were moderately or extremely likely to have undertaken an additional year of residency, with factors including gender, debt load, and planned scope of practice influencing the decision.16 Interestingly, only one-quarter to one-third of residents in LOTP 4YR programs think training in family medicine requires a fourth year. However, an increasing majority of them want more flexibility in their training, and nearly half pursued a 4YR residency to expand their skill set beyond clinical skills. This may include skills needed for practice leadership/management, research, or health policy/advocacy work. Of future importance will be any notable differences in the level of preparedness for independent practice for those receiving 3 vs 4 years of training, which will be conducted closer to the end of the study. It may be that the 4YR model provides opportunities for customized training that align with individual career paths leading to a greater sense of control over one’s education.
Strengths of this study include successfully enrolling well-matched residency programs with diverse geographic, size, and clinical training settings representation across the United States; having comparison data from both 3YR and 4YR programs over an 8-year period; and including national data for descriptive comparisons over this same time period. Additionally, we analyzed a large sample of 475 residents to ascertain their perspectives on length of training. Weaknesses of our study include that we are only analyzing data from 13 residencies that may not be generalizable to the applicant and match experience of all residencies across the United States. The 4YR programs were carefully chosen to be in this pilot and likely do not represent average family medicine residencies. Another potential weakness is that the 4YR programs were implementing their new model of training during the period of analysis with varying curricula from year to year, which may have influenced their applicant pool and match results. National attention to the LOTP may have contributed to changes in the applicant pool for 4YR programs. The study is also limited by a small number of applicant variables analyzed, and there could be other differences found looking at other variables or using qualitative assessments.
In conclusion, extending residency training in family medicine to 4 years appears not to adversely affect the applicant and match outcomes of these programs compared to their matched 3YR programs. Given ongoing problems with lower student interest in primary care careers, continued monitoring of factors influencing student interest in family medicine is warranted. Additional important questions need to be addressed regarding length of training, including its impact on medical knowledge, continuity of care, scope of practice and clinical preparedness. The LOTP is well positioned to provide some answers to those questions in the next few years.
Financial Support: The Length of Training Pilot is sponsored by the Accreditation Council for Graduate Medical Education and funded by the American Board of Family Medicine Foundation. None of the authors have a conflict of interest for this manuscript.
- Carek PJ. The length of training pilot: does anyone really know what time it takes? Fam Med. 2013;45(3):171-172.
- Lyss-Lerman P, Teherani A, Aagaard E, Loeser H, Cooke M, Harper GM. What training is needed in the fourth year of medical school? Views of residency program directors. Acad Med. 2009;84(7):823-829. https://doi.org/10.1097/ACM.0b013e3181a82426
- Walling A, Merando A. The fourth year of medical education: a literature review. Acad Med. 2010;85(11):1698-1704. https://doi.org/10.1097/ACM.0b013e3181f52dc6
- Stevens CD. Commentary: Taking back Year 4: a call to action. Acad Med. 2010;85(11):1663-1664. https://doi.org/10.1097/ACM.0b013e3181f53487
- Sklar DP. Making the fourth year more meaningful. Acad Med. 2014;89(4):527-528. https://doi.org/10.1097/ACM.0000000000000184
- Kozakowski S, Abercrombie S, Carek P, et al. Perceived impact of proposed institute of medicine duty hours on family medicine residency programs. Ann Fam Med. 2009;7(3):276-277. https://doi.org/10.1370/afm.1007
- Antiel RM, Thompson SM, Reed DA, et al. ACGME duty-hour recommendations - a national survey of residency program directors. N Engl J Med. 2010;363(8):e12. https://doi.org/10.1056/NEJMp1008305
- Harris JD, Staheli G, LeClere L, Andersone D, McCormick F. What effects have resident work-hour changes had on education, quality of life, and safety? A systematic review. Clin Orthop Relat Res. 2015;473(5):1600-1608. https://doi.org/10.1007/s11999-014-3968-0
- Bohmer RM. Managing the new primary care: the new skills that will be needed. Health Aff (Millwood). 2010;29(5):1010-1014. https://doi.org/10.1377/hlthaff.2010.0197
- Pugno PA. One giant leap for family medicine: preparing the 21st-century physician to practice patient-centered, high-performance family medicine. J Am Board Fam Med. 2010;23(suppl 1):S23-S27. https://doi.org/10.3122/jabfm.2010.S1.090291
- Kozakowski SM, Travis A, Bentley A, Fetter G Jr. Results of the 2016 National Resident Matching Program: 1986–2016: a comparison of family medicine, E-ROADS, and other select specialties. Fam Med. 2016;48(10):763-769.
- Fields KB. More on the 4-year FM residency program. Fam Med. 2005;37(1):8.
- Scherger JE. Residencies: heal thyself before extending. Fam Med. 2006;38(3):158-159.
- Duane M, Green LA, Dovey S, Lai S, Graham R, Fryer GE. Length and content of family practice residency training. J Am Board Fam Pract. 2002;15(3):201-208.
- Smits AK, Walsh E, Ross RG, Gillanders WR, Saultz JW. Residency applicants’ perspectives on family medicine residency training length. Fam Med. 2006;38(3):172-176.
- Carney PA, Eiff MP, Waller E, Peterson LE. Factors associated with interest in pursuing a fourth year of family medicine residency training. Fam Med. 2017;49(5):339-345.
- Sairenji T, Dai M, Eden AR, Peterson LE, Mainous AG III. Fellowship or further training for family medicine residents? Fam Med. 2017;49(8):618-621.
- Gruppuso PA, Adashi EY. Residency placement fever: is it time for a reevaluation? Acad Med. 2017;92(7):923-926. https://doi.org/10.1097/ACM.0000000000001468
- Association of American Medical Colleges. Table C-4: Residency Applicants from US M.D.-Granting Medical Schools by Specialty, 2012-2013 through 2017-2018. https://www.aamc.org/data/facts/. Accessed June 7, 2018.
- Weissbart SJ, Kim SJ, Feinn RS, Stock JA. Relationship between the number of residency applications and the yearly match rate: time to start thinking about an application limit? J Grad Med Educ. 2015;7(1):81-85. https://doi.org/10.4300/JGME-D-14-00270.1
- Carek PJ, Steyer TE, Dickerson L. Does a research requirement affect match rates for family medicine residency programs? Fam Med. 2007;39(7):483-487.
- Malaty W, Pathman DE. Factors affecting the match rate of rural training tracks in family practice. Fam Med. 2002;34(4):258-261.
- Garvin RD, Eiff MP, Pugno P, et al. A p⁴ report: effect of curriculum innovation on residency applications and match performance. Fam Med. 2011;43(7):472-479.
- Douglass AB, Rosener SE, Stehney MA. Implementation and preliminary outcomes of the nation’s first comprehensive 4-year residency in family medicine. Fam Med. 2011;43(7):510-513.
- Length of Training Pilot Project. http://www.lotpilot.org. Accessed Aug 16, 2018.
- Association of American Medical Colleges. Electronic Residency Application Service (ERAS), Applicant Data Years 2012-2017. https://www.aamc.org/services/eras/stats/359278/stats.html. Accessed June 7, 2018.
- National Resident Matching Program. Match Results, Historical Reports Years 2012-2017. http://www.nrmp.org/report-archives/. Accessed June 7, 2018.
- Little RJ, Rubin DB. Statistical Analysis with Missing Data. 2nd ed. Hoboken, NJ: John Wiley & Sons; 2002. https://doi.org/10.1002/9781119013563
- White IR, Royston P, Wood AM. Multiple imputation using chained equations: issues and guidance for practice. Stat Med. 2011;30(4):377-399. https://doi.org/10.1002/sim.4067
- Ahmed AH, Schnatz PF, Adashi EY. Allopathic and Osteopathic Medicine Unify GME Accreditation: A Historic Convergence. Fam Med. 2017;49(5):374-377.
- Accreditation Council for Graduate Medical Education. List of Newly Accredited Programs, Academic Year 2017-2018, United States. https://apps.acgme.org/ads/Public/Reports/Report/8 Accessed August 15, 2018.
- Wright KM, Ryan ER, Gatta JL, Anderson L, Clements DS. Finding the perfect match: factors that influence family medicine residency selection. Fam Med. 2016;48(4):279-285.
- Nuthalapaty FS, Jackson JR, Owen J. The influence of quality-of-life, academic, and workplace factors on residency program selection. Acad Med. 2004;79(5):417-425. https://doi.org/10.1097/00001888-200405000-00010