Do Residency Signals Actually Signal Intent? Insights From the 2024 Family Medicine National Resident Survey

In the setting of the growing need to train more family physicians and the growth of available residency slots, there is also increasing concern about the decreasing percentage of family medicine (FM) residency positions being filled in the primary Match process and the increased reliance on the Supplemental Offer and Acceptance Program (SOAP) to fill residency positions.^1,2 The American Academy of Family Physicians announced in April 2025 their Residency Selection Improvement Initiative to evaluate residency selection and recruitment in FM with one of the goals being to improve the efficiency and function of the FM residency selection process.³ One of the tools identified for evaluation are the Association of American Medical Colleges (AAMC) Electronic Residency Application Service (ERAS) program signals and geographic preferences.⁴ These ERAS tools were created as an aid for applicants to highlight their interests and preferences and for programs to more easily identify interested applicants who are best suited for their program as part of a holistic application review. After being piloted by several other specialties, FM adopted program signaling where applicants could use up to five signals of interest to specific programs and signal geographic preferences in the 2023-2024 cycle.

While the associations of programs’ use of signals and interview invitations^5,6; the number of programs and residents who matched via SOAP⁷; and signaling usability^8,9 have been reported, there is a lack of information on whether signals are associated with residency selection. Faculty career advisors and students seek more guidance on how to best use these new tools.¹⁰ The objective of this study was to inform applicants and their advisors, programs, and policy makers about how to best utilize these ERAS tools by analyzing the relationship between first-year FM residents’ use of program signals and geographic preferences with where they matched.

The American Board of Family Medicine (ABFM) administered the National Resident Survey (NRS) to all FM residents in October 2024 after the ABFM In-Training Exam (ITE). The questions on the NRS were created by ABFM staff to evaluate changes in residency training requirements and initial certification eligibility. Each class of residents received different questions with PGY-1 FM residents being asked about their use of ERAS signals and geographic preferences and whether they indicated their current program with either system. PGY-1 FM residents were also asked about how they entered their current program: Main Match (applied only to FM), Main Match (applied to multiple specialties with FM highest), Main Match (applied to multiple specialties with other specialty highest), SOAP, or After SOAP. These categories refer to how residents report submitting their rank lists to the Match. Resident demographic data were obtained from ABFM administrative databases.

ABFM staff with access to deidentified data conducted all analyses. We described the data and performed bivariate analyses by degree type, site of medical school training, and when in the Match process they entered their current program. This study was approved by the American Academy of Family Physicians Institutional Review Board.

Our sample included 5,237 of 5,293 (99% response rate) eligible PGY-1 FM residents who completed items related to signals; 4,681 (89.4%) stated that they used program signals and 4,160 (79.4%) used geographic preferences. Of the residents who used program signals, 66.0% stated they signaled their current program (59.0% of all residents surveyed) and of those who use geographic preferences, 91.0% stated they indicated the geographic region of their current program (72.0% of all residents surveyed; Figure 1).

We found differences in program signaling and geographic preferences by degree type and international medical graduate (IMG) status (Table 1). IMG residents were less likely to use geographic preferences compared to United States medical graduate (USMG) residents and were less likely to be in a program they signaled or geographically preferred (Figure1). Meanwhile DO residents were more likely to use both program signals and geographic preferences compared to USMG-MD residents.

We also observed differences by how in the Match process they entered FM, with those who entered through the SOAP process being less likely to report that they used program signals (64%) and geographic preferences (62%) in their applications (Table 2). Surprisingly, 10% of these residents reported signaling their current program (Figure 3) and 45% reported being in a program in their geographic preference (Figure 4).

In the first year of FM using ERAS signal system, the majority of PGY-1 residents reported that they matched into a program that they signaled and/or geographically preferred. Our findings suggest that ERAS program signals and geographic preferences may reflect applicants’ true intentions. Most PGY-1 FM residents matched to a program they had signaled and to a preferred region, indicating these tools may help improve alignment between applicant preferences and match outcomes.

We found that 10% of residents who matched through SOAP reported that they signaled the program they are currently attending, suggesting that they did not receive an interview in the Main Match from this program, despite giving them one of their signals and the program not filling all their positions in the Main Match. This finding suggests that some program directors did not fully utilize program signals to identify applicants to interview who were more likely to attend their program. This is consistent with a qualitative study of Midwest FM program directors that suggested some program directors did not heavily utilize program signals in the first year.⁹ While Brown and Mitchell⁷ found that the introduction of these tools was not associated with improvement of programs matching more of their residents through the Main Match,⁷ our study shows that a large majority of residents report using signals and preferences for the programs they eventually attend, and there is even alignment for those who entered through SOAP.

Residents who entered through the Main Match were more likely to use program signals and geographic preferences than those entering through the SOAP. This was also true for IMG residents. Encouraging broader use of these tools, particularly among IMGs who often apply widely, may help applicants target programs more effectively and improve alignment in the Match process.

A major strength of this study is the high response rate and sample size, both of which increase generalizability to nearly all PGY-1 FM residents. The timing of the survey in October of the first year of training also helps to minimize recall bias. A major limitation of this study is the use of cross-sectional descriptive methods, which do not enable determination of causality. Additionally, we are only gaining the perspectives of those who successfully matched into FM, so we cannot draw conclusions about how signals and preferences influence the overall chance that an applicant matches into FM. Finally, the data are self-reported and there may be social desirability bias where residents are more likely to report signaling their current program. The NRS will continue to ask PGY-1 FM residents about their Match experience so we can monitor trends over time as students and programs gain more experience with these new tools.

Our findings indicate that FM residency applicants are using signaling and geographic preferences to broadcast their true preferences to programs. Programs should see these signals and preferences as a reliable indication of applicant’s true preferences. If both occur, FM has an opportunity to improve the efficiency of the Match through better alignment of applicant and program preferences, potentially reducing SOAP reliance.