Impact of Virtual Recruitment on Costs, Time Spent, and Applicant Perspectives Within a Family Medicine Residency Program

Prior to 2020, few residency programs used virtual recruitment to fill incoming classes,^1,2 despite recruitment being an historically important, time intensive, and costly effort.³ During the COVID-19 pandemic, the Accreditation Council for Graduate Medical Education (ACGME) advised all programs to conduct interviews virtually for the 2021 recruitment season.⁴ Virtual recruitment was then strongly encouraged for the 2022 season. Subsequently,  the ACGME called for research on the virtual interview process.⁵ Studies examining virtual recruitment within family medicine residency programs remains scarce.

This study aimed to analyze the impact of virtual recruitment within one university-based program in three areas: financial cost to the program, time spent by applicants with the program, and applicant satisfaction with the virtual process. Additionally, given this program’s focus on increasing equity for applicants from diverse backgrounds,⁶ this study analyzed the impact of virtual recruitment on applicants who self-identify as being underrepresented in medicine (URiM).

We assessed recruitment data at the University of Utah Family Medicine Residency program for the 2019 through 2022 recruitment seasons. Expenses incurred during calendar years 2018 and 2019 (covering 2019 and 2020 seasons), were reviewed and compared to the expenses incurred during calendar years 2020 and 2021 (covering 2021 and 2022 seasons). We compared staff, resident, and faculty time on interview day for the same years.

For the 2022 season, an anonymous survey was sent to interviewed applicants via SurveyMonkey (Momentive), after applicants certified their rank lists but before match day. The survey consisted of multiple choice, Likert scale, and free response questions. Questions examined the program’s ability to represent itself virtually and applicant satisfaction with the virtual interview format (see Appendix). For comparing data between URiM and non-URiM interviewees, we used Pearson χ², continuous, and t tests. We set statistical significance at .05. All statistical tests were two-sided. This study was included under an umbrella exemption from review, provided by the University of Utah Institutional Review Board. 

Program Expenditures. Recruitment expenses for in-person seasons of 2019 and 2020 were $70,355 and $70,604, respectively. Recruitment expenses for virtual recruiting years of 2021 and 2022 were $21,949 and $9,932, respectively. Most expenses for in-person interviews were related to lodging and meals for interviewees. Additional expenses for both in-person and virtual interviews were for travel for program representatives to out-of-state recruitment events, and meals for program representatives (Table 1).

Time Spent by Applicants With the Program. Applicants spent 515 minutes directly engaging the program on in-person interview days and 345 minutes on virtual days (Table 2). The reduced time was due to the elimination of in-person facility tours, lunch, and travel time between sites and shorter welcome session and interview times. Virtual interview-day interactions were conducted via the Zoom platform. In 2019 and 2020 there were 9 interview days, compared to 11 days in 2021 and 2022.

Out of 60 responders (response rate 67%), 70% (n=42) were female, and 20% were URiM (Table 3). Regarding participation, 3.5% stated they would not have interviewed if interviews were held in person, 80.4% indicated that virtual interviews allowed them to interview with more programs, and 82.8% indicated they would have needed to fly to interview with the program. In addition, 45.2% indicated that finances would have been a barrier to interviewing in person. Respondents were split on preferred format, with 38.6% preferring virtual, 35.1% in-person, and 26.3% no preference. However, URiM candidates had a higher preference for the virtual interview format (54.5%) compared to non URiM candidates (34.8%; Table 4).

Respondents interviewed at, on average, 16.6 programs virtually and 0.4 programs in-person. URiM respondents interviewed virtually at four more programs on average than their non-URiM counterparts (19.9 vs 15.9; P=.052). Non-URiM respondents on average attended 0.5 in-person interviews, whereas URiM respondents attended no interviews in person (0.5 vs 0; P=.033). This suggests a statistically significant difference between non-URiM and URiM in-person interviews (Table 5).

The financial benefits of virtual recruitment were substantial; transitioning from in-person to virtual format saved the program about $50,000 annually. Cost savings due to reduced air travel were noted by 82.8% of respondents. Nearly half of respondents indicated that costs would have been a barrier interviewing with this program. The average cost of a roundtrip flight within the United States during quarter 3 of 2021 was $314.39.⁷ The cost of flying to 16.6 in-person interviews could have totaled $5,218.87 per applicant in flights alone. Prior studies suggest applicants are particularly interested in virtual interviews when distance is a variable.¹ The reduced costs to applicants of virtual interviews may allow them to interview with more programs.

However, virtual recruitment is associated with the loss of some interview day activities, including physical hospital, clinics, and community tours, and in-person meals. Eliminating the tours and lunch resulted in lost opportunities for applicants to gain insight into the program by seeing program and community facilities and sharing casual conversations.

In the 2022 recruitment season, this program did not fill during the initial Match. As a result, the program participated in the Supplemental Offer and Acceptance Program through the National Resident Matching Program, which required added time commitments from staff, faculty, and residents. It is unclear if the failure to fill is linked to virtual recruiting. Further study comparing program personnel engagement in in-person and virtual recruitment, and related success in the match, will be beneficial.

Overall, survey respondents were satisfied with the virtual process, and reported sufficient understanding of the program, but also reported less ability to distinguish between programs. The virtual format may allow applicants to consider more programs. Whether the increased number of interviews is beneficial for applicants remains to be determined; if individual applicants interview at extremely high numbers of programs, there are fewer interview opportunities for other applicants. Additionally, the difference between URiM and non-URIM in-person attendance should be further studied.

A limitation of this project is that data were collected from a single program. Additionally, the URiM applicant sample size was small. Strengths include the longitudinal nature of objective outcome measures. While the COVID-19 pandemic created challenges, it provided opportunities to expand understanding of virtual recruitment options for family medicine residencies.