Use of Preference Signals in Family Medicine Residency Recruitment

An increase in graduate medical education applications,^1,2 secondary to the widespread introduction of virtual interviewing during the COVID-19 pandemic, led to the genesis of preference signaling, a tool where an applicant can communicate a more sincere interest to a limited number of select programs within the Electronic Residency Application Service (ERAS) system. First piloted by otolaryngology during the 2021–2022 interview season,³ early analysis showed program directors’ assigning value to preference signals in their recruitment efforts⁴; however, that analysis also found that most signals followed a geographic pattern with applicants’ home or medical school, providing some question regarding their added value to the recruitment process beyond generally accepted patterns of application regionality.^5-7

Preference signaling initially was available to family medicine residency programs during the 2023–2024 interview season. Their arrival was met with mixed opinions, including with regard to how programs intended to use them.^7,8 After the completion of the initial recruitment cycle where family medicine residency programs and applicants had access to preference signals, our research team sought to better understand how one group of stakeholders within family medicine residency recruitment—program directors—used this new tool. Comparing how preference signals were used as sway factors during the review of residency applicants at all stages of the recruitment process—including application review, interview offer, candidate comparison, and ranking—is an important step in understanding the value of preference signals and whether they are providing the benefits for which they were initially designed.

The methodology employed by the Council of Academic Family Medicine Educational Research Alliance (CERA) survey has been thoroughly documented in earlier work.⁹ Our study was approved by the American Academy of Family Physicians Institutional Review Board. Data collection for the spring 2024 program director survey took place between April 30 and June 7, 2024. Alongside demographic questions, the survey included 10 items developed by our research team to evaluate family medicine residency program directors’ opinions on interview format and preference signaling (Appendix). The CERA Steering Committee reviewed these 10 questions to ensure that they aligned with the subproject’s goals and were clear, reliable, and valid based on existing evidence. Family medicine educators outside the target population tested the survey prior to distribution, informing adjustments to improve clarity, timing, and flow.

The Association of Family Medicine Residency Directors provided the contact list of Accreditation Council for Graduate Medical Education (ACGME)–accredited US family medicine residency program directors. CERA distributed an initial email invitation through SurveyMonkey (SurveyMonkey Inc), followed by three weekly reminder emails and one final reminder sent the day before the survey concluded. At the time of distribution, 767 program directors, excluding those with undeliverable email addresses, were on the list. To ensure that directors have comprehensive program knowledge, the CERA methodology does not include directors whose programs have not graduated a class of residents or had fewer than three cohorts who experienced all training components.

We conducted χ² tests to evaluate the association between the use of preference signaling and its potential role as a deciding factor in candidate selection. Our analysis considered variations across residency program types and demographic characteristics of family medicine residency program directors. A significance threshold of P<0.05 was applied. All statistical analyses were performed using Stata (StataCorp) software.

Survey data were collected from 382 respondents; 308 of the 382 responded fully to our survey questions for an overall response rate of 40.2% (308 valid surveys received from a possible 767 programs).

Signals were seen as positive components within a holistic review, but did not yield automatic interviews for applicants, nor did they independently influence the ranking process in a significant fashion (Table 1). In comparison with other classic sway factors, such as residency location or experience with a residency (eg, visiting rotations), preference signaling was reported to be a lesser factor (Table 2). Additionally, we found that 19 program directors in communities of less than 75,000 population reported that they did not use signals at all (Table 3).

Our study shows that while preference signals were used by most program directors in the immediate recruitment season after their debut to family medicine, they did not supersede the importance of other factors in all stages of decision-making regarding residency applicants. Preference signals have become another tool in the overall applicant review process, yet their usage has been inconsistent among program directors. While overall application volumes per applicant have decreased in many specialties, including a modest decrease in family medicine,¹⁰ the decrease in volume has not transformed the way residency recruitment seasons have operated.

Rotation experiences remain the top factor for influencing a program’s opinion on an applicant. Long seen as equivalent to an extended interview or audition to the program,¹¹ rotation experience continued to be reported as a top factor by family medicine program directors during our study.

Secondary to rotation experiences was residency geographic location with regard to its proximity to either the applicant’s identified home or the applicant’s medical school. The strength of the geography factor continued to be illustrated during the 2025 Match data review.^12,13 When primary and secondary sway factors were combined, an increased importance of residency location was seen; however, location did not exceed rotation experience as a determining factor.

An area for future exploration would be a deeper look at the perspectives of program directors in rural communities regarding preference signal utilization. Within our data set, 19 program directors from residencies within communities with less than a population of 75,000 indicated that they did not use signals. While these numbers did not rise to statistical significance in the data set, the data do pose the question of why these program directors would not use this new tool to enhance the recruitment process for programs that have historically struggled to recruit trainees to their locales.¹⁴

Our study had limitations, including some perceived confusion among respondents regarding interpretation of survey questions. For example, 66 program directors selected “Did not use” as their primary usage type, yet 35 of these respondents indicated in other questions that they incorporated preference signals into some aspect of their recruitment process. This inconsistency represents approximately 11.7% of the analytic sample and suggests that a subset of respondents misunderstood the question. Such misclassification may have influenced the distribution of responses in Table 1 and should be considered when interpreting these findings. Additionally, a possibility of response bias existed, and the survey questions were limited to only 10.

Given that signals are expected to remain part of the family medicine residency match process, this study demonstrates a need for future qualitative studies on the best use of signals for residency applicants.

Miranda A. Moore reports receiving funding from Agency for Healthcare Research and Quality, Health Resources and Services Administration, Ardmore Institute of Health, the Alzheimer’s Association, and the Georgia Department of Human Services.