A Comparison of Residency Applications and Match Performance in 3-Year vs 4-Year Family Medicine Training Programs

The optimal length of residency training needed to best prepare a family physician to function in both current and future health care environments is under debate.1 Reasons proposed to extend training to 4 years include inadequate clinical preparation during medical school,2-5 reduction in available educational time due to duty hour restrictions,6-8 and new skills needed to practice in complex health systems while providing patient-centered care.9-10 Despite these arguments for adding a year, concerns remain that doing so will further erode student interest in choosing family medicine at a time when fewer students are entering primary care disciplines.11 Additionally, some educators advocate for more flexibility in residency education to address clinical preparedness rather than more time.12-13

Student and resident interest in a 4-year (4YR) residency has been studied over the past few decades. In 2002, the concept of a 4YR family medicine residency was supported by 32% of first-year residents when surveyed shortly after starting training.14 A 2006 survey of residency applicants indicated that the addition of another year of training would have a neutral or positive effect on choosing family medicine.15 More recently, a 2014 study found that slightly less than half of graduating family medicine residents expressed interest in a fourth year of training.16 Additionally, a study of over 6,000 soon-to-graduate residents in 2014 and 2015 found little overlap between residents intending fellowship training and those who would consider a longer residency, suggesting that the desire to pursue a fellowship does not necessarily equate to a desire for more core training.17

The residency placement process has been undergoing marked change in the past decade, with the number of applications submitted and the number of residencies ranked per US senior applicant increasing significantly.18 The average number of applications to family medicine programs per seniors from US allopathic medical schools has risen from 18.7 in 2012 to 33.3 in 2017, a 78% increase.19 Despite this increase in the number of programs each applicant applies to, match rates for allopathic US graduates remain steady and unchanged.18,20

Research requirements, rural training tracks, curriculum innovations, and individualized education have been studied as factors that may affect program match performance in family medicine through the National Resident Matching Program (NRMP).21-23 The only previous report of the effect of a 4YR training model on applicants and the match was from a single program in the Preparing the Personal Physician for Practice (P4) Project that reported a 77% increase in the number of US senior applicants between 2007 and 2011, with no adverse effect on match performance.24

The purpose of this study is to determine, among programs enrolled in the Length of Training Pilot (LOTP) in family medicine, what effect length of training has on applicant and match variables (applicant type, number of applicants, match positions filled, matched applicant type, ranks to fill). We specifically explored the hypothesis that no differences in applicant and match outcomes between 4YR programs and 3-year (3YR) programs would be found.

The LOTP is a prospective case-control study (2013-2022) exploring the influence of length of family medicine residency training on several learner and program outcomes.1,25 Evaluation of the pilot is overseen by a team of educational researchers in the Department of Family Medicine at Oregon Health & Science University (OHSU). Residency programs that had or were planning to transition to a 4YR model applied for the pilot in 2012. Selection was based on components of programs’ innovations and designs of their site-specific evaluation plans. Four of the selected 4YR programs had a required full 4 years of training and two had an optional additional fourth year of training that residents could choose to undertake, the timing of which varied. Enrolled 4YR programs were matched to 3YR control programs based on size, region, and clinical setting. The LOTP includes 17 residencies (seven 3YR civilian programs, six 4YR civilian programs, and four military programs with a 4YR track). The military programs were excluded from these analyses because residency applicant and selection processes are quite different from those used by civilian residencies. Because of the size of one 4YR program, two 3YR programs were matched to it to ensure equivalent numbers of residents in 3YR and 4YR groups.

All LOTP programs obtained Institutional Review Board (IRB) approval with waivers of documented consent or exemptions to participate in the study as well as to conduct program-specific evaluations at their respective institutions. OHSU’s IRB granted the evaluation team an educational exemption to obtain data from the study sites (IRB #9770).

Match and applicant data were collected from multiple sources. Participating programs provided data for years 2012-2018 on applicants (type, interviews completed, number of applicants ranked) and match results (positions filled, matched applicant type). Applicants were categorized as graduates of allopathic schools (US MD), graduates of osteopathic schools (US DO), or international medical graduates (IMG)—both US and foreign citizenship. These data were collected annually immediately following Match Day and then verified, cleaned, and entered into a secure database. Programs provided retrospective data for 2012 and 2013 immediately following the 2014 match. Comparative national data were obtained from the Electronic Residency Application Service (ERAS)26 and the NRMP27 for years 2012-2018. Published data from ERAS and NRMP includes only aggregate data so we were unable to undertake statistical comparisons of national data with LOTP programs.

Resident characteristics, number of programs where they interviewed, number of programs ranked, and perspectives on training program selection were obtained from an annual resident survey administered in the fall of each year to all residents in LOTP programs. The 29-question instrument was updated from the P4 Resident Survey to reflect current trends in family medicine training and attitudes on 3 versus 4 years of training. The survey was revised accordingly after its first year of use in the study, which served as a pilot test of the items. Variables included in these analyses were from 2015 and later. We used survey responses from postgraduate year 1 (PGY-1) residents only since this group was closest to the applicant and match experience. Response rates to these surveys ranged from 88% to 96% for 3YR programs, and from 97% to 98% for 4YR programs.

Summary statistics including means, standard deviations, frequencies, and corresponding t-tests and χ2 tests of independence were performed to assess differences between 3-year and 4-year residents and resident perspectives on residency programs.

To estimate trends in applicant pool and match performance, means and standard deviations were calculated for each year from 2012-2018 by program type. To evaluate if 3- vs 4-year programs differed in trends over time, we utilized a linear mixed model to account for repeated measures over time within programs. For each outcome, we considered fixed effects of program type (3 vs 4 years), a set of indicators for categorical years and their interaction. The interaction terms identified any potential differences in mean outcomes over time (eg, interaction terms identified any deviation of parallel trends over time where P values >.05 suggest similar trends over time between 3- and 4-year programs). We included a random effect of program ID to account for temporal correlation of outcome measures within each program.

For a few outcomes (US MD applicants, US DO applicants, IMG applicants, and ranks to fill per position offered in the match), we observed missing outcome data for some years. To account for missingness, we performed multiple imputations by chained equations (MICE) to maximize all observed data without removing programs with missing data. For the MICE procedure, we included as important covariates: number of US senior applicants, percent of positions filled by US seniors (MDs and DOs) and IMGs, region of the country, single vs multiple specialty residency status, continuity clinic in a community health center, number of interviews conducted, total number of applicants, and total percent of positions filled. For all imputed models, we performed 20 imputations with 100 iterations and used Rubin’s Rules to combine parameter estimates.28 For these imputed outcomes, we do not report standard deviations, as they are not recommended for reporting when producing estimates by MICE.29

In addition, we conducted a sensitivity analysis where one analysis was run with all six 4YR programs included, and a second analysis was run with the two programs that had optional additional training removed from the data set. This allowed us to evaluate the impact that an optional additional training had on study outcomes. Because the sensitivity analysis showed only minor changes to the results, none of which affected statistical significance, the data presented here include all six programs that offered additional training beyond the traditional 3 years. All analyses were performed using Stata version 15. All statistical tests were two-sided and type I error was set at 5%.

Six of the seven 3YR programs are community-based and four of the six 4YR programs are community-based, the remainder being university-based. The characteristics of the PGY-1 residents in 3YR (n=254) and 4YR programs (n=308) in the LOTP are presented in Table 1. We found no statistical differences between residents in 3YR and 4YR programs for age, race, ethnicity, marital status, parental status, or attending a US medical school. The percentage of women in 3YR programs was significantly higher than 4YR programs (68.5% vs 56.5%, P=.017).

Table 2 illustrates trends in the applicant pool for 3YR and 4YR programs from 2012 to 2018 as well as national comparator data from the ERAS system. There were no differences in the mean number of US MD applicants between 3YR and 4YR programs, and the upward trend in both groups parallels the national trend over the 7-year study period. Both the 3YR and 4YR programs had a substantially higher number of applicants from both allopathic and osteopathic schools compared to national averages. There was variability in the mean number of US DO applicants and IMG applicants during the study period in both 3YR and 4YR programs, with no significant differences between the two groups. There were no statistical differences in the total number of applicants per program between the two groups.

Data from the NRMP (Table 3) show no statistical differences for positions filled in the match between 3YR and 4YR programs. The mean percentage of positions filled by US MDs, US DOs, and IMGs was not different for 3YR vs 4YR programs. Both 3YR and 4YR programs exceeded the national averages for percent of positions filled by US MDs in all study years except 2012. The only significant difference between groups was in the mean number of ranks to fill per position offered in the match (a higher ranks to fill per position indicates the program went farther down the match list to fill its last position) with the 3YR programs having higher means in 2012, 2013, and 2014, and the 4YR programs having higher means in the last 4 years of data collection (P=.027).

PGY-1 resident views of the residency selection process from residents in 3YR vs 4YR programs (Table 4) illustrate no statistical differences in resident responses from 2014 to 2017 in how they considered length of training in the interview and match process. Notably, one-third of residents in 3YR programs in 2014 to 2016, and nearly half in 2017 had also interviewed at 4YR programs. Residents in 3YR and 4YR programs ranked a similar number of 3YR programs. There was a significant upward trend in the percentage of residents in 3YR programs who did not think a fourth year of training is necessary in family medicine (P<.001). The percentage of residents in 4YR programs who think training in family medicine requires a fourth year varied significantly from 35% to 25% over the years analyzed (P<.001). The predominant reason each year for pursuing training in a 4YR program was a desire for more flexibility in training, followed by a desire to learn additional skills beyond clinical skills. The percentage of residents in 4YR programs who considered doing a fellowship after 3 years but chose a 4YR program because of a preference to gain skills in an area of concentration through a different model varied between 36% and 43% without a significant trend across years.

Our findings confirm our hypothesis that the length of training does not adversely affect applicant and match outcomes in LOTP programs, and thus offering 4 years vs 3 years of training does not appear to adversely affect the applicant or match pool of potential candidates. We found similar fill rates and similar percentages of applicants and filled positions who were US MDs and US DOs in the 3YR and 4YR programs in the LOTP. Notably, the number of applicants from both allopathic and osteopathic schools was higher in the LOTP programs compared to national averages, indicating that this group of residencies may be historically more competitive or attractive to students. It is possible that any differences between 3YR and 4YR applications and match outcomes were muted because of the quality and appeal of both groups of programs and confounded by other appealing program attributes and strengths.

Some significant changes in the national graduate medical education (GME) environment occurred during our period of data collection and are worth noting. First, over the past decade, students from allopathic schools were applying to a larger number of residencies because too few slots exist for the number of medical school graduates.18 The LOTP programs mirrored this upward national trend. The second notable national trend occurred in 2014 when the Accreditation Council for Graduate Medical Education (ACGME), the American Osteopathic Association (AOA), and the American Association of Colleges of Osteopathic Medicine (AACOM) resolved to institute a single, unified GME accreditation system by July 1, 2020.30 In family medicine, this has increased the overall number of ACGME residencies, as AOA-only accredited programs are now receiving ACGME accreditation through the Single Accreditation System process.31 Subsequently, programs are seeing an increase in number of DO applicants. This trend was apparent in LOTP programs regardless of length of training, especially in 2017 and 2018.

In the LOTP, residents in both 3YR and 4YR programs reported that factors other than length of training were more important in their decision making about program selection. This is consistent with other reports demonstrating that factors such as location, work/life balance, resident satisfaction and curriculum are more often selected as influencing residency ranking.32,33 A notable finding is that 4YR programs have statistically fewer women residents. It is not known whether this difference is due to longer training being less attractive to women or other program features of the 4YR residencies that lead to more men than women matching with these particular programs. Further study is needed to look carefully at outcomes in the LOTP to determine if any gender differences are emerging.

In a study of family medicine residents about to graduate in 2014, approximately one-fifth were moderately or extremely likely to have undertaken an additional year of residency, with factors including gender, debt load, and planned scope of practice influencing the decision.16 Interestingly, only one-quarter to one-third of residents in LOTP 4YR programs think training in family medicine requires a fourth year. However, an increasing majority of them want more flexibility in their training, and nearly half pursued a 4YR residency to expand their skill set beyond clinical skills. This may include skills needed for practice leadership/management, research, or health policy/advocacy work. Of future importance will be any notable differences in the level of preparedness for independent practice for those receiving 3 vs 4 years of training, which will be conducted closer to the end of the study. It may be that the 4YR model provides opportunities for customized training that align with individual career paths leading to a greater sense of control over one’s education.

Strengths of this study include successfully enrolling well-matched residency programs with diverse geographic, size, and clinical training settings representation across the United States; having comparison data from both 3YR and 4YR programs over an 8-year period; and including national data for descriptive comparisons over this same time period. Additionally, we analyzed a large sample of 475 residents to ascertain their perspectives on length of training. Weaknesses of our study include that we are only analyzing data from 13 residencies that may not be generalizable to the applicant and match experience of all residencies across the United States. The 4YR programs were carefully chosen to be in this pilot and likely do not represent average family medicine residencies. Another potential weakness is that the 4YR programs were implementing their new model of training during the period of analysis with varying curricula from year to year, which may have influenced their applicant pool and match results. National attention to the LOTP may have contributed to changes in the applicant pool for 4YR programs. The study is also limited by a small number of applicant variables analyzed, and there could be other differences found looking at other variables or using qualitative assessments.

In conclusion, extending residency training in family medicine to 4 years appears not to adversely affect the applicant and match outcomes of these programs compared to their matched 3YR programs. Given ongoing problems with lower student interest in primary care careers, continued monitoring of factors influencing student interest in family medicine is warranted. Additional important questions need to be addressed regarding length of training, including its impact on medical knowledge, continuity of care, scope of practice and clinical preparedness. The LOTP is well positioned to provide some answers to those questions in the next few years.