New Tools for Resident Recruiting Season: A CERA Survey of Family Medicine Residency Program Directors

John E. Snellings, MD | Miranda A. Moore, PhD | Daniel L Meyer, PhD (ret.)

Fam Med. 2023;55(8):525-529.

DOI: 10.22454/FamMed.2023.504726

Return to Issue


Background and Objectives: As the family medicine community continues to adapt to interview season changes secondary to the effects of the COVID-19 pandemic, discussions are underway regarding new options to improve the overall success and satisfaction of resident recruiting. Tools such as preference signaling, interview capping, and supplementary applications are options that have been investigated, and in some cases implemented, by other specialties for their recruiting seasons. Family medicine as a specialty is now actively scrutinizing the benefits and drawbacks of these tools.

Methods: The fall 2021 CERA program directors’ omnibus online cross-sectional survey invited family medicine program directors to provide their perceptions of these tools. Descriptive statistics and multivariate logistic regressions were conducted.

Results: Two-thirds of the 262 program director respondents (42% response rate, n=184) supported policies for preference signaling and a national interview cap; however, support was mixed for the use of supplemental applications.

Conclusions: Because the survey results indicated a high level of support for using these innovative new tools during recruitment season, family medicine should take action to implement these programs/policies.


Residency recruitment strategies continue to evolve after two cycles in which virtual interviewing models were strongly recommended by governing bodies. While the use of virtual interviewing models has been proven a successful option, the convenience of the model has led to concerns regarding applicants’ overapplying and overinterviewing for residency positions. Electronic Residency Application Service (ERAS) data has shown a continued upward trend in the total number of applications per applicant for family medicine programs. 1 This trend persists despite the most recent match data from the National Resident Matching Program (NRMP) showing that US medical school seniors (MDs & DOs) have approximately 90% match success when they rank 5 to 6 programs and near 100% success when they rank 12 to 13 programs. After 13, the applicant has no increased benefit from additional program ranking. 2, 3

Additional interview season strategies that have been suggested or trialed since the start of the pandemic in other specialties include capping the number of residency interviews an applicant can accept, 4 along with preference signaling, where an applicant has the ability to communicate to a select few programs a more sincere interest in training there. 5, 6 A supplemental application within the ERAS system also has been used and has proliferated among multiple specialties over recent interviewing cycles. 7-9 While these formalized models have yet to be trialed in family medicine residency programs, ongoing discussions are taking place among stakeholders as to the potential use of these tools as early as the 2023-2024 residency recruiting season. 10 Of particular interest are the use of a supplementary application and preference signaling.

In 2021, the Council of Academic Family Medicine (CAFM) Educational Research Alliance (CERA) fielded a survey module to family medicine program directors focusing on the use of supplemental applications, preference signaling, and interview capping as standardized components of the specialty’s recruiting season. Data from the survey provide initial insights into residency program directors’ perceptions of these issues. The purpose of our study was to measure program directors’ perspectives on these potential new components to resident interviewing and to assess for variables, including demographics, that may influence program directors’ opinions on these topics.


Survey Development and Sample

CAFM is a leadership and research collaborative among the leading academic family medicine organizations: the Society of Teachers of Family Medicine, the Association of Departments of Family Medicine, the Association of Family Medicine Residency Directors (AFMRD), and the North American Primary Care Research Group. CERA’s program director survey methodology has previously been described in detail. 11 Questions were modified based on feedback received. We were given ethical approval for the study by the American Academy of Family Physicians Institutional Review Board in September 2021. The survey was fielded from September 14 to October 8, 2021, and we conducted data analysis November 2021 through March 2022.

The directors of all US family medicine residency programs accredited by the Accreditation for Graduate Medical Education, as identified by the AFMRD, served as the survey sampling frame. SurveyMonkey (Momentive Inc) was used to deliver email invitations and reminders to participants and collect survey responses. Three weekly follow-up emails were sent to nonrespondent participants, and a fourth reminder was sent 2 days before the survey closed. Of the 696 program directors on the distribution list, 30 emails returned a bounced message. Programs with fewer than three resident classes were excluded (n=35) because those programs have yet to experience a graduating class of residents. The final sample size was 631.


We obtained residency program demographic data from the recurring questions of the survey. We combined the underrepresented in medicine (UriM) residency percentage (0%) group with the less than 6% group because very few in the sample reported 0%. Additionally, we dichotomized support for each topic by grouping “Strongly agree” and “Agree” responses to the survey question.

Statistical Analysis

The frequency of responses and percentages are reported for categorical variables, and mean and standard deviation are reported for continuous variables. We performed bivariate analysis among the three module questions to determine significant, either statistically or of high magnitude, associations. For the variables with a significant association, we conducted multivariate logistic regression to examine impacts adjusted for demographic factors. We used Stata version 14.0 (StataCorp) to perform the statistical analyses, with a P value of.05 as statistically significant.


The overall response rate to the survey was 44% (275/631). The response rate for our module questions was 42% (n=262). In the program directors’ module, certain demographic responses were particularly relevant to experience with virtual interviewing: the mean years in their current position (5.6); a minority (8%) identifying as Hispanic; and the majority of programs being community-based, university affiliated (58%, see Appendix A).

Univariate Analysis

We found strong support for capping the number of applications per applicant and for applicant preference signaling. A significant majority of program directors were in favor of a national interview cap and preference signaling (both 68%, Table 1). In contrast, only 44% of respondents were in favor of making a supplemental application a standard component of the recruitment process.

Bivariate and Multivariate Analysis

We found a statistically significant relationship between favor for preference signaling and favor for supplemental applications. For these two questions, we combined those who responded “neither agree nor disagree” with those who responded “somewhat disagree” or “strongly disagree” to create a “not agree” category. More program director respondents who favored preference signaling also favored supplemental applications than program director respondents who did not favor preference signaling (52% vs 29%, Table 2). Overall, 241 program director respondents provided complete demographic information and were included in the multivariate logistic regression. When adjusting for residency demographic factors, the program directors who supported supplemental applications were three times more likely to favor preference signaling (odds ratio [OR]: 3.16; 95% confidence interval [CI]: 1.51-6.60). Compared to program director respondents whose residency was located in communities with less than 30,000 residents, all other program director respondents were less supportive of preference signaling, ranging from 81% to 88% less likely (all significant at the 10% level).

Compared to program director respondents whose residency consisted of less than 19 residents, program director respondents with a larger residency size were less likely to favor preference signaling (19-31 OR: 0.47; 95% CI: 0.21-1.01; more than 31 OR: 0.57; 95% CI: 0.19-1.74). Additionally, compared to female program director respondents, male program director respondents were less likely to favor preference signaling (OR: 0.36; 95% CI: 0.17-0.76).


This CERA survey’s responses provided early insight into the perspective of family medicine program directors for whom new tools would be helpful to counteract some of the challenges of the ever-evolving residency recruitment season. Supplemental applications have been used by family medicine residency programs in the past, but at a program level, and not necessarily in direct coordination with ERAS. At the time this survey was collecting data in September to October 2021, only otolaryngology residencies had completed an interview season using preference signaling in their recruiting; dermatology, internal medicine, and surgery were starting their first experiences during that time frame. 11

Because some questions have been raised about the legality of a national interview cap as it pertains to trade laws, 12 the use of supplemental applications and preference signaling would seem to hold the highest potential for family medicine residency programs in the upcoming match seasons.

Our statistical analyses showed program directors generally supporting both supplemental applications and preference signaling, as opposed to supporting one tool independently. With specific regard to preference signaling, program directors in smaller communities or with smaller class sizes showed greater support for this system; that finding could be due to these program directors’ desire to use preference signaling as a tool to refocus their recruiting resources on applicants with particular interest in training at their site. The most relevant data on preference signaling came from the 2021 survey of program directors’ and applicants’ experiences in otolaryngology, where program directors found it most helpful when a preference was expressed from an applicant when it was not otherwise evident (eg, from a different geographical region). 13, 14 Further investigation of this relationship would be beneficial, particularly to programs in smaller or rural communities. Additionally, the preference difference we found between male and female program directors may simply be an incidental finding and requires further study.

One potential limitation of this study was the simplistic framing of the questions, without specific descriptions of what each question’s subject would entail. This limitation was especially true for the questions involving a national interview cap and supplemental application; the latter particularly would have the potential to change responses based on a more detailed overview of what the application would require.

In conclusion, family medicine program directors showed support for use of all three proposed recruiting season tools, with preference signaling and interview capping receiving the greatest backing, and support for all tools more likely than isolated support for one tool. When these responses were further analyzed, program director gender, residency community size, and residency class size were variables that significantly affected support.


  1. Association of American Medical Colleges. ERAS statistics by applicant: Family medicine. 2022. https://www.aamc.org/media/39346/download
  2. National Resident Matching Program. Charting outcomes in the match: US allopathic seniors. 2020:73
  3. National Resident Matching Program. Charting outcomes in the match: US osteopathic seniors. 2020:76.
  4. Morgan HK, Winkel AF, Standiford T, et al. The case for capping residency interviews. J Surg Educ. 2021;78(3):755-762. doi:10.1016/j.jsurg.2020.08.033
  5. Pletcher SD, Chang CWD, Thorne MC, Malekzadeh S. The otolaryngology residency program preference signaling experience. Acad Med. 2022;97(5):664-668. doi:10.1097/ACM.0000000000004441
  6. Salehi PP, Azizzadeh B, Lee YH. Preference signaling for competitive residency programs in the NRMP. J Grad Med Educ. 2019;11(6):733-734. doi:10.4300/JGME-D-19-00695.1
  7. Rozenshtein A, Griffith BD, Paladin A. More signal, less noise: the electronic residency application program supplemental application in radiology match. J Am Coll Radiol. 2022. doi:10.1016/j.jacr.2022.05.016
  8. Ahmed AM, Helfrich YR. The ERAS supplemental application: current status and recommendations for dermatology applicants and programs. Cutis. 2022;109(6):306-308. doi:10.12788/cutis.0536
  9. Association of American Medical Colleges. AAMC supplemental ERAS® application: key findings from the 2022 application cycle. 2022. https://www.aamc.org/media/58891/download
  10. Association of Family Medicine Residency Directors. 23-24 interview season listening sessions. August 2022.
  11. Seehusen DA, Mainous AG III, Chessman AW. Creating a centralized infrastructure to facilitate medical education research. Ann Fam Med. 2018;16(3):257-260. doi:10.1370/afm.2228
  12. Carmody JB, Rosman IS, Carlson JC. Application fever: reviewing the causes, costs, and cures for residency application inflation. Cureus. 2021;13(3):e13804. doi:10.7759/cureus.13804
  13. Society of University Otolaryngologists. Signaling updates. https://opdo-hns.org/mpage/signaling-updates
  14. Pletcher SD, Chang CW, Thorne MC, Malekzadeh S. The otolaryngology residency program preference signaling experience. Acad Med. 2022;97(5):664-668. doi:10.1097/ACM.0000000000004441

Lead Author

John E. Snellings, MD

Affiliations: Department of Family and Community Medicine, Eastern Virginia Medical School, Norfolk, VA


Miranda A. Moore, PhD - Department of Family and Preventive Medicine, Emory University School of Medicine, Atlanta, GA

Daniel L Meyer, PhD (ret.) - Maine-Dartmouth Family Medicine Residency, Brunswick, ME

Corresponding Author

John E. Snellings, MD

Correspondence: Department of Family and Community Medicine, Eastern Virginia Medical School, Norfolk, VA

Email: snelliJE@evms.edu

Fetching other articles...

Loading the comment form...

Submitting your comment...

There are no comments for this article.

Downloads & Info


Related Content