Impact of Virtual Recruitment on Costs, Time Spent, and Applicant Perspectives Within a Family Medicine Residency Program

Kara A. Frame, MD | Katherine T. Fortenberry, PhD | Susan Cochella, MD, MPH | Osman Sanyer, MD | Eliza Taylor, MPH, BS, CHES | Dominik Ose, DrPH, MPH | Kirsten Stoesser, MD

PRiMER. 2023;7:38.

Published: 11/29/2023 | DOI: 10.22454/PRiMER.2023.262017


Background: Virtual recruitment for all residency programs was endorsed by the Accreditation Council for Graduate Medical Education (ACGME) for the 2021 and 2022 recruitment seasons. This study assesses the impact of virtual recruitment on cost and outcome in a family medicine residency program.

Methods: We assessed program recruitment costs and interview-day time with applicants in one program for the 2019 to 2022 recruitment seasons, and we sent an anonymous survey to interviewed applicants (n=98) for the 2022 match year.  In-person interviews were conducted in 2019 and 2020. Virtual interviews were conducted in 2021 and 2022.

Results: Program recruitment costs decreased from over $70,000 annually for in-person interview seasons to between $10,000 and $20,000 annually for the virtual interview years. Applicant time with the program on interview days decreased from 515 minutes when held in-person, to 345 minutes when virtual. Applicants expressed that they were generally satisfied with the virtual interview format though their preference for the virtual format was only slightly greater than for in-person interviews (38.6% and 35.1%, respectively); 26.3% of the responding applicants had no preference for either format. During virtual interview years, applicants interviewed at an average of 16.6 programs with 80% indicating that virtual interviews allowed for consideration of more programs.

Conclusion: The virtual interview format was associated with decreased interview-day costs for programs and interviewees, and decreased time on interview days for both groups. It allowed applicants to consider more programs.  


Prior to 2020, few residency programs used virtual recruitment to fill incoming classes,1,2 despite recruitment being an historically important, time intensive, and costly effort.3 During the COVID-19 pandemic, the Accreditation Council for Graduate Medical Education (ACGME) advised all programs to conduct interviews virtually for the 2021 recruitment season.4 Virtual recruitment was then strongly encouraged for the 2022 season. Subsequently,  the ACGME called for research on the virtual interview process.5 Studies examining virtual recruitment within family medicine residency programs remains scarce.

This study aimed to analyze the impact of virtual recruitment within one university-based program in three areas: financial cost to the program, time spent by applicants with the program, and applicant satisfaction with the virtual process. Additionally, given this program’s focus on increasing equity for applicants from diverse backgrounds,6 this study analyzed the impact of virtual recruitment on applicants who self-identify as being underrepresented in medicine (URiM).


We assessed recruitment data at the University of Utah Family Medicine Residency program for the 2019 through 2022 recruitment seasons. Expenses incurred during calendar years 2018 and 2019 (covering 2019 and 2020 seasons), were reviewed and compared to the expenses incurred during calendar years 2020 and 2021 (covering 2021 and 2022 seasons). We compared staff, resident, and faculty time on interview day for the same years.

For the 2022 season, an anonymous survey was sent to interviewed applicants via SurveyMonkey (Momentive), after applicants certified their rank lists but before match day. The survey consisted of multiple choice, Likert scale, and free response questions. Questions examined the program’s ability to represent itself virtually and applicant satisfaction with the virtual interview format (see Appendix). For comparing data between URiM and non-URiM interviewees, we used Pearson χ2, continuous, and t tests. We set statistical significance at .05. All statistical tests were two-sided. This study was included under an umbrella exemption from review, provided by the University of Utah Institutional Review Board. 


Finances and Time

Program Expenditures. Recruitment expenses for in-person seasons of 2019 and 2020 were $70,355 and $70,604, respectively. Recruitment expenses for virtual recruiting years of 2021 and 2022 were $21,949 and $9,932, respectively. Most expenses for in-person interviews were related to lodging and meals for interviewees. Additional expenses for both in-person and virtual interviews were for travel for program representatives to out-of-state recruitment events, and meals for program representatives (Table 1).

Time Spent by Applicants With the Program. Applicants spent 515 minutes directly engaging the program on in-person interview days and 345 minutes on virtual days (Table 2). The reduced time was due to the elimination of in-person facility tours, lunch, and travel time between sites and shorter welcome session and interview times. Virtual interview-day interactions were conducted via the Zoom platform. In 2019 and 2020 there were 9 interview days, compared to 11 days in 2021 and 2022.

Survey Results

Out of 60 responders (response rate 67%), 70% (n=42) were female, and 20% were URiM (Table 3). Regarding participation, 3.5% stated they would not have interviewed if interviews were held in person, 80.4% indicated that virtual interviews allowed them to interview with more programs, and 82.8% indicated they would have needed to fly to interview with the program. In addition, 45.2% indicated that finances would have been a barrier to interviewing in person. Respondents were split on preferred format, with 38.6% preferring virtual, 35.1% in-person, and 26.3% no preference. However, URiM candidates had a higher preference for the virtual interview format (54.5%) compared to non URiM candidates (34.8%; Table 4).

Respondents interviewed at, on average, 16.6 programs virtually and 0.4 programs in-person. URiM respondents interviewed virtually at four more programs on average than their non-URiM counterparts (19.9 vs 15.9; P=.052). Non-URiM respondents on average attended 0.5 in-person interviews, whereas URiM respondents attended no interviews in person (0.5 vs 0; P=.033). This suggests a statistically significant difference between non-URiM and URiM in-person interviews (Table 5).


The financial benefits of virtual recruitment were substantial; transitioning from in-person to virtual format saved the program about $50,000 annually. Cost savings due to reduced air travel were noted by 82.8% of respondents. Nearly half of respondents indicated that costs would have been a barrier interviewing with this program. The average cost of a roundtrip flight within the United States during quarter 3 of 2021 was $314.39.7 The cost of flying to 16.6 in-person interviews could have totaled $5,218.87 per applicant in flights alone. Prior studies suggest applicants are particularly interested in virtual interviews when distance is a variable.1 The reduced costs to applicants of virtual interviews may allow them to interview with more programs.

However, virtual recruitment is associated with the loss of some interview day activities, including physical hospital, clinics, and community tours, and in-person meals. Eliminating the tours and lunch resulted in lost opportunities for applicants to gain insight into the program by seeing program and community facilities and sharing casual conversations.

In the 2022 recruitment season, this program did not fill during the initial Match. As a result, the program participated in the Supplemental Offer and Acceptance Program through the National Resident Matching Program, which required added time commitments from staff, faculty, and residents. It is unclear if the failure to fill is linked to virtual recruiting. Further study comparing program personnel engagement in in-person and virtual recruitment, and related success in the match, will be beneficial.

Overall, survey respondents were satisfied with the virtual process, and reported sufficient understanding of the program, but also reported less ability to distinguish between programs. The virtual format may allow applicants to consider more programs. Whether the increased number of interviews is beneficial for applicants remains to be determined; if individual applicants interview at extremely high numbers of programs, there are fewer interview opportunities for other applicants. Additionally, the difference between URiM and non-URIM in-person attendance should be further studied.

A limitation of this project is that data were collected from a single program. Additionally, the URiM applicant sample size was small. Strengths include the longitudinal nature of objective outcome measures. While the COVID-19 pandemic created challenges, it provided opportunities to expand understanding of virtual recruitment options for family medicine residencies.


The authors acknowledge the assistance of the following individuals: Heather McComber, Becky Reynolds, Jessica Bickley, and Andrew Curtin.


  1. Seifi A, Mirahmadizadeh A, Eslami V. Perception of medical students and residents about virtual interviews for residency applications in the United States. PLoS One. 2020;15(8):e0238239. doi:10.1371/journal.pone.0238239
  2. Vadi MG, Malkin MR, Lenart J, Stier GR, Gatling JW, Applegate RL II. Comparison of web-based and face-to-face interviews for application to an anesthesiology training program: a pilot study. Int J Med Educ. 2016;7:102-108. doi:10.5116/ijme.56e5.491a
  3. Nilsen K, Callaway P, Phillips JP, Walling A. How much do family medicine residency programs spend on resident recruitment? A CERA study. Fam Med. 2019;51(5):405-412. doi:10.22454/FamMed.2019.663971
  4. The Coalition for Physician Accountability’s Work Group on Medical Students in the Class of 2021, Moving Across Institutions for Post Graduate Training. Final Report and Recommendations for Medical Education Institutions of LCME-Accredited, U.S. Osteopathic, and Non-U.S. Medical School Applicants. Accessed November 17, 2023.
  5. The Coalition for Physician Accountability’s Work Group on Medical Students in the Class of 2022, Moving Across Institutions for Interviews for Postgraduate Training. Recommendations on 2021-22 Residency Season Interviewing for Medical Education Institutions Considering Applicants from LCME-Accredited, U.S. Osteopathic, and Non-U.S. Medical Schools. Accessed November 17, 2023.
  6. Stoesser K, Frame KA, Sanyer O, et al. Increasing URiM Family Medicine Residents at University of Utah Health. PRiMER Peer-Rev Rep Med Educ Res. 2021;5:42. doi:10.22454/PRiMER.2021.279738
  7. Average Domestic Air Fares. United States Department of Transportation.

Lead Author

Kara A. Frame, MD

Affiliations: Department of Family and Preventive Medicine, School of Medicine, University of Utah, Salt Lake City, UT


Katherine T. Fortenberry, PhD - Department of Family and Preventive Medicine, School of Medicine, University of Utah, Salt Lake City, UT

Susan Cochella, MD, MPH - Department of Family and Preventive Medicine, School of Medicine, University of Utah, Salt Lake City, UT

Osman Sanyer, MD - Department of Family and Preventive Medicine, School of Medicine, University of Utah, Salt Lake City, UT

Eliza Taylor, MPH, BS, CHES - Department of Family and Preventive Medicine, School of Medicine, University of Utah, Salt Lake City, UT

Dominik Ose, DrPH, MPH - Department of Family and Preventive Medicine, School of Medicine, University of Utah, Salt Lake City, UT

Kirsten Stoesser, MD - Department of Family and Preventive Medicine, School of Medicine, University of Utah, Salt Lake City, UT

Corresponding Author

Kara A. Frame, MD

Fetching other articles...

Loading the comment form...

Submitting your comment...

There are no comments for this article.

Downloads & Info


Related Content