The Impact of Virtual Interviews on the Resident Candidate: A Before-and-After Comparison

The COVID-19 pandemic created challenges to recruitment for residency programs. The Association of American Medical Colleges (AAMC), as a member of the Coalition for Physician Accountability, encouraged residency programs to transition to virtual interviewing in May 2020.1 The academic family medicine community followed this recommendation.2 The transition from in-person visits led to creative recruitment practices. While there has been discussion regarding approaches that are most engaging,3-5 there is minimal research that examines the impact virtual interviews have had from the applicant’s perspective. Our study investigates the impact this transition had on applicants’ interview experiences and decisions in applying with a family medicine department at a large research university.

Our program recruits 13 interns per year, matching them into two continuity clinics. During the 2020-2021 and 2021-2022 seasons, all interviews were held virtually with all prior interviews being in-person. Each interviewee was invited to join an informal, resident-led/resident-only informal virtual get-together prior to their interview. Interview days were structured with opening introductions, five 20-minute one-on-one interviews with faculty and residents, followed by filmed virtual tours and open discussion with residents. Our local graduate medical education (GME) office created virtual content about our community, and residents increased content posted to social media. Following interviews, a virtual second-look event was held in conjunction with our GME community.

Candidates voluntarily completed a postinterview survey as part of an internal quality improvement effort, with questions specific to their perceptions of their interview experience. Applicants who completed virtual interviews were asked if national ranking, social media, or prior experiences at the institution influenced their decision to apply. These residents were additionally asked about their preferences for in-person or virtual interviews. We examined survey results for trends across the pre- and postvirtual interviewing recruitment cycles. We combined recruitment cohorts 2018-2019 and 2019-2020 to form the in-person interview group, and we combined 2020-2021 and 2021-2022 to form the virtual interview group.

We conducted statistical analyses using SPSS Statistics software, version 26 (IBM Corp, Armonk, NY). We tested differences between applicant years for significance using χ² tests for all survey questions.

Our institutional review board policies do not require approval for postinterview data that is for quality improvement, and our study also falls outside of Common Rule and Food and Drug Adminsitration definitions of research.6

Five hundred twelve interviews were conducted over 4 years (2018-2019: n=124, 2029-2020: n=131, 2020-2021: n=129, 2021-2022: n=128). The total number of surveys collected over 4 years was 256 (2018-1019: n=70; 2019-2020, n=60; 2020-2021, n=78; 2021-2022, n=48). We examined demographic and geographic data between cohorts. The only significant difference found was an increase in Hispanic applicants, which coincided with the creation of a Spanish language track. The top factors influencing an applicant’s decision to apply across all 4 years included university setting, location, fit, and feel based on the program website. Among residents who completed a virtual interview, national ranking was the second-most selected factor (74.3%; Table 1). The number of applicants who selected an urban site significantly increased from 50% (in-person) to 66.3% (virtual) as a factor influencing one’s decision to apply, as did community setting, with an increase from 44.6% to 63.3%.

Outside of the pandemic, 48.7% of the virtual applicants indicated a virtual interview as their preference and 41.6% indicated preference for an in-person interview (this question was not asked of in-person applicants). We found no significant differences in candidate perceptions of the management of the interview process. Both in-person and virtual interview candidates reported that the process was managed very well (84.7% and 80.4%, respectively). Similarly, 87.6% of the in-person interview candidates and 83.8% of the virtual candidates described their interview as a positive experience.” Candidates from both in-person and virtual cohorts did not perceive themselves to be strongly evaluated after the interview (43.7% and 41.2%, respectively; Table 2).

Both groups reported adequate contact with clinical staff, residents, faculty, and residency leadership. No significant differences were identified.

There were no significant differences in how applicants—both in-person and virtual interviewees—characterized their experience, suggesting that the virtual model of recruitment was neither superior nor inferior, and as effective as in-person recruitment. Candidates from the virtual interviews expressed more interest in an urban clinic, suggesting a potential change in how sites are portrayed within the virtual process. Candidates from virtual interviews expressed more interest in obstetrics training. Perhaps these changes are a result of a larger candidate pool due to fewer financial and time restrictions.7 This could also reflect a change in applicant interests.

Over half of applicants in all years did not feel they were assessed as a strong candidate. Candidates perceived positively may have been inadvertently provided with indirect feedback. Our residency has attempted to reduce bias by standardizing interview questions, which may have resulted in less feedback to applicants. Future research may track candidates’ ratings by the recruitment team to determine accuracy in candidate perceptions.

A limitation of this study is that it reflects the experience of a single site, which impacts generalizability, and results may be more applicable to large academic programs as compared to community-based programs. Study findings represent a small number of candidates responding to unvalidated surveys that were constructed for the purpose of feedback. There was also no way to contact applicants who only considered programs offering in-person interviews, potentially skewing the data. Future research may also benefit from comparing how our program rated candidates and applicants’ perception of being viewed strongly.

It is possible to conduct a virtual interviewing process that is as equivalent to in-person interviewing in terms of reported applicant experience and factors leading to decisions in applying to our program. Advantages of a virtual recruitment process include saving costs of travel and lodging. However, virtual-only options limit how applicants adequately learn about programs. Looking to the future, we need to carefully consider how this could impact the size and diversity of our applicant pool.