Effectiveness of Virtual Residency Interviews: Interviewer Perspectives

The COVID-19 pandemic led to widespread adoption of virtual residency program interviews. Little is known about the effectiveness, advantages, barriers, and acceptability of the virtual interview format. Studies of these factors are needed to inform decisions about future interview formats.

A growing number of studies have assessed effectiveness, perceived advantages, disadvantages, and acceptability of virtual interviews.1-36 Most studies were limited to single programs or specialties. In our current study, we analyzed the results of a multiprogram, single-institution survey of interviewers’ perceptions of effectiveness, advantages, and barriers related to virtual residency interviews. We also assessed preferences for future interview seasons.

We sent an online survey to interviewers and applicants in anesthesiology, clinical psychology internship and PhD programs, emergency medicine, family medicine, general surgery, obstetrics and gynecology, pediatrics, and psychiatry programs following virtual interviews at one academic institution in the Pacific Northwest during the 2020-2021 match cycle. We previously reported survey development methods and applicant perspectives.37 The current analysis assesses interviewer responses.

As previously described, a multidisciplinary group created a survey using expert opinion and literature review for content validity. We piloted the survey among a group of 10 people to ensure clarity. The instrument assesses effectiveness, advantages, barriers, and future preferences for interview format (Appendix Box 1). Although this was not a detailed analysis of virtual interview equity, we included several equity-related items such as pregnancy, breastfeeding, dependent care, disability, and mental health conditions.

We generated descriptive statistics of survey data. All analyses were conducted using SAS 9.4 software (SAS Institute Inc, Cary, NC). This study was exempted by the Oregon Health & Sciences University Institutional Review Board.

A total of 136 out of 312 interviewers completed the survey (response rate of 44%; Table 1). Interviewers identified predominately as female (58%) and White (73%). The majority (67%) were under age 45 years, with very few respondents over the age of 55 years (13%); 83% reported they were interviewing from an urban location.

As shown in Table 2, the majority of interviewers rated virtual interviews as very or extremely effective in creating a comfortable setting (79%) and answering interviewee questions (86%). A smaller majority of interviewers rated virtual interviews as very or extremely effective for establishing a sense of connection (59%), evaluating interviewee strengths via the formal interview (64%), and communicating program culture (51%), although an additional 30% to 38% of interviewers felt virtual interviews were moderately effective in these areas. Over half of interviewers (51%) felt that virtual interviews were either not effective at all or only slightly effective for evaluating interviewee strengths via informal interactions. Similar portions of interviewers agreed or strongly agreed the virtual format was effective at providing adequate tours (44%) and providing adequate information about the institutional social environment (50%).

A large portion of participants identified more advantages than barriers of virtual interviews (Figure 1). Top-perceived advantages were time efficiency (81%), reduced carbon footprint (61%), cost savings (56%), and less burdensome to schedule (49%). Interviewers selected technological issues (21%), caregiving duties (18%), access to an appropriate interview setting (16%), and increased stress/anxiety (9.9%) as top barriers. Few selected advantages or barriers related to disabilities (full data available in Appendix Table 1).

Most interviewers (91%) preferred to incorporate a virtual format in future interview cycles. The most common preference selected was “Two stages: Interviews are conducted virtually with programs having the option of inviting a subset of applicants for subsequent face to face visits” (49%). Some preferred providing an option of virtual or in-person (29%) or remaining entirely virtual (13%). The least-preferred format was entirely face to face (9%).

Our study demonstrated an overall positive perception of the virtual interview platform. Most interviewers rated virtual interviews as effective in most aspects, although the virtual format was relatively less effective for evaluating interviewee strengths via informal interactions, providing adequate tours of the facilities, and providing adequate information about the institutional social environment. Interviewers endorsed advantages more frequently than barriers, with positive perceived impacts on time efficiency, carbon footprint, cost, and scheduling burden. The majority supported virtual platforms for the future, with many endorsing a two-tiered approach (eg, virtual followed by in-person for a selected few).

Our study results are in line with a growing body of literature suggesting that a majority of interviewers find the virtual platform effective and acceptable.1-36 Concerns expressed in most studies related to providing information to applicants about the physical and social environment of a program, and an ability to assess applicants’ interpersonal interactions. The majority of studies available for comparison were single program or single specialty, whereas our study included several specialties. Results were still consistently supportive of virtual platforms.

Few studies have identified specific advantages and barriers to the virtual interview format.7,8,14,16,20,21,34 Our findings identified cost savings and time efficiency as significant advantages,in line with other studies. Novel to our study, carbon footprint savings was the second-most selected advantage. This is an important factor to consider during the current climate crisis, and in line with studies showing substantial carbon emissions related to interview travel.38-41 We uniquely identified caregiving duties and access to appropriate interview setting as top barriers, warranting future, more detailed, exploration into these concerns.

Limitations include that this was a single institution survey and had a response rate of 44%. The majority of respondents were under age 45 years, and as such may have biases related to comfort with the use of technology. Our inclusion of equity-related items was limited, potentially missing important issues such as impacts of racial bias. We also did not directly compare virtual interviews to the alternatives of in-person or hybrid interviews.

Overall, interviewers perceived virtual interviews as acceptable, effective, and advantageous with a positive impact on time efficiency, cost and carbon footprint. The majority indicated that they preferred virtual interviews be part of future recruitment cycles. Further research may focus on methods to improve evaluation of applicants’ informal interactions and tours/social activities from a virtual platform, and how to ensure equitable match results for applicants.