On Screen or On Site? Hybrid Interviews for Flexibility Without Added Bias

In the ever-evolving landscape of medical education and training, the interview process for selecting medical residents stands as a pivotal juncture that shapes the future of the residents and their programs. Traditionally conducted in person, medical resident interviews underwent significant transformation in 2020–2021 due to the pandemic.¹ The traditional in-person interview has long been the gold standard, fostering interpersonal connections and allowing interviewers to gauge a candidate’s demeanor, communication skills, and emotional intelligence. Though virtual interviews became the predominant mode of interviewing for infection control, they have continued because they are most often perceived to reduce disparities; travel-related time and direct expenses for both applicants and programs are minimized in this format.^2-4 However, some programs have struggled to tell their story effectively in the virtual format,⁵ and some applicants have struggled to get a true picture of the residency training environment.^1,2,6

A hybrid approach, defined by the American Association of Medical Colleges as allowing applicants to choose between virtual or in-person interview formats,⁷ offers flexibility but has raised concerns about bias. The National Resident Matching Program’s (NRMP’s) 2024 code of conduct for programs suggests conducting interviews in-person or virtually, but not both, to reduce bias—specifically, the chance that students who interview in person may be perceived more favorably and thus ranked higher than those who interview virtually because they have the opportunity for more interaction with current residents and faculty.⁸ In contrast, the American Association of Colleges of Osteopathic Medicine has strongly encouraged hybrid interviews for the same reason of decreasing bias, but with the opposite logic—suggesting that letting students decide how best to present themselves to programs was more fair and equitable.⁹ These contradicting recommendations underscore the need for data around interview format—specifically, whether offering a hybrid format creates bias in ranking that disadvantages applicants in matching to their desired program. Unlike prior studies that have examined costs, time, and perceptions associated with interview format,¹⁰ our study explored applicant interview and match outcomes in a hybrid approach. This included the concept of opportunity to match, which considers only the applicants ranked up to the final matched position on the program rank list and thus may provide a better way to assess the program’s interview ranking process and the possibility of bias in that process as opposed to any applicant bias that may have led to the choice of in-person versus virtual interview.

As such, the objectives of this study were to explore, first, whether a hybrid interview structure might create bias on the residency program rank list; second, whether the hybrid structure influenced how the program matched; and third, what programs’ experiences were with the process of organizing a hybrid interview approach.

The Family Medicine Residency Network (FMRN) comprises 32 family medicine residency programs and 10 rural training tracks across Washington, Wyoming, Alaska, Montana, and Idaho (WWAMI). Programs are mostly community-based and have different sponsoring institutions, but all are educationally affiliated with the University of Washington. Each program interviews and ranks applicants independently. Due to its collaborative culture, FMRN has been able to gather sensitive data such as applicant interview data, rank list data, and match list data for multiple residency programs that are not typically available for external analysis.

Six programs (five core and one rural track) offered a hybrid interview format in 2022–2023 and 2023–2024, and all participated in this study. Programs submitted de-identified rank lists sharing data on each interviewee, including interview format (in-person or virtual), subinternship participation, and other applicant contact (eg, second look). In the 2022–2023 season, these programs also completed a brief survey on their hybrid process (included as a supplemental appendix), including anticipated interview format ratios, whether applicants could choose between formats, screening processes, rationale for offering hybrid interviews, whether they collected applicant feedback, and bias-mitigation strategies. This study was retrospective, that is, program decisions about interview structure were made prior to the study’s design.

Data were collected via email after the 2023–2024 NRMP Match and the Supplemental Offer and Acceptance Program (SOAP) process. We aggregated and analyzed quantitative data descriptively and with bivariate tests using SPSS version 28.0.0.0 (IBM). Rank lists varied in length, so for analysis we normalized them on a 100 ranked position scale (ie, divided the individual’s rank number by the total ranked in that program and multiplied by 100 to get the normalized rank position). To assess bias, our analyses focused on applicants ranked up to the final matched position (ie, opportunity to match). We used exploratory content analysis to identify themes from survey open-ended responses.

With guidance from the University of Washington Human Subjects Division, this study was determined not to be human subjects research, and institutional review board review was not sought.

All six programs submitted both rank list data and program director surveys (100% response); program characteristics are shared in Table 1. In 2022–2023,, programs anticipated roughly equal mixes of interview formats; in practice, 42.9% were in person and 57.1% virtual (n = 560 interviews for 46 positions). In 2023–2024,, programs shifted toward more virtual interviews (33.8% in person, 66.2% virtual, n = 610). Spanning both years, 38.1% of interviews were in person and 61.9% virtual. Description of the sample, the rank list, and match outcomes overall are shown in Table 2.

A total of 52 interviewees were not ranked across all programs; half of those individuals (n = 26) interviewed in person and half interviewed virtually. Because of the greater number of virtual interviews overall, the percentage not ranked was greater among those who interviewed in person than virtually (5.8% vs 3.6%, χ² = 3.26, P = 0.071).

In comparing individual applicant rank spot by type of interview conducted, those who interviewed in person had a higher average rank location compared to those who interviewed virtually (45 vs 49 out of 100 positions; independent samples two-sided t test, P = 0.022 assuming unequal variances; Table 3). Those who participated in a subinternship were ranked higher on the rank list, at an average position of 34 out of 100 ranked positions vs 49 out of 100 ranked positions for those who did not participate in a subinternship (independent samples two-sided t test, P<0.001 assuming unequal variances). Subinternships were also significantly more likely to interview in person (n = 58/95, 61.1%) than virtually (n = 37/95, 38.9%; χ², P<0.001).

To eliminate the impact of subinternship participation in average rank list position by interview type, we then performed the same calculations excluding those who participated in subinternships. When subinternships were excluded from the sample, the average rank locations were closer; 47 out of 100 for in-person interviews and 50 out of 100 for virtual interviews (independent samples one-sided t test, P = 0.090 assuming unequal variances; Table 3).

Most (88/91, 96.7%) of the positions being interviewed for were filled in the main match; spanning both years, three of the 91 total positions (3.3%) were filled in the SOAP process, and none went unfilled after the match process was complete.

Across the programs, 747 positions had the opportunity to match, that is, applicants that were ranked above the lowest spot on the rank list that was a match. Of these, 267 (35.7%) were in-person interviews and 480 (64.3%) were virtual. This finding was not statistically different from the overall interview ratio of 38.1% virtual and 61.9% in person (χ² = 1.102, P = 0.29).

Applicants who interviewed in person were significantly more likely to match (n = 48/446, 10.8%) than those who interviewed virtually (n = 40/724, 5.5%; χ², P<0.001; Table 3).

Additionally, those who had participated in a subinternship at a program (n = 95) were significantly more likely to match (17/95, 18%) than those who had not participated in a subinternship (71/1075 = 7%) (χ² = 15.996, P<0.001). When we excluded applicants who had participated in subinternship experiences from the analysis, those who interviewed in-person were still significantly more likely to match (n = 35, 9.0%) than those who interviewed virtually (n = 36, 5.2%, χ², P = 0.017; Table 3).

All six programs allowed applicants to choose which interview format they wanted; one required supplemental questions before an applicant could schedule a virtual interview. Reasons for conducting hybrid interviews included flexibility, accommodating out-of-state applicants, and not wanting to lose out on applicants by requiring an in-person interview. Programs also desired to give interested applicants an in-person opportunity to see where they would be living/working and experience the culture of the training program. Programs viewed these factors as important to overcome applicant bias against program locations. Programs reported using standard rubrics and interview questions and locking scoring after interviews to mitigate any potential bias between interview types. Programs received feedback that applicants valued the choice between in-person and virtual interviews.

Overall, this unique dataset has provided several insights into outcomes of the hybrid interview structure. First, we found that once those who participated in a subinternship were excluded, the type of interview was not associated with an applicant’s position on the rank list, which suggests that the hybrid interview approach was not introducing programmatic bias into the interview process, at least in aggregate. Although subinternships could be before or after the interview process, these data suggest that students who engaged in subinternships and became known to the program were more likely to receive higher ranking and more likely to match to the program.

When considering match outcomes, we found that even when those who participated in a subinternship were excluded, those who matched were more likely to have interviewed in person. We hypothesize that this was due to an applicant’s initial interest in a particular program. That is, applicants who are highly interested in a program may choose an in-person interview over a virtual option. This choice reflects applicant preference. However, the opportunity to match reflects a program’s interview ranking process. In this study, we found that opportunity to match was not significantly affected by the interview type, which suggests that any difference in rank position was a reflection of applicant attributes and not programmatic bias from interview type.

In terms of program experiences, both programs and students (based on feedback the programs received) appreciated the flexibility of the hybrid approach. This finding aligns with other data showing that medical students prefer the hybrid approach to interviews so that they can decide for themselves on the best format.¹¹ Additionally, whether the hybrid approach might affect programs’ likelihood of not matching and needing to participate in the SOAP process is not yet clear. Three of 46 (6.5%) positions were filled in the SOAP among the programs in the study in the 2022–2023 interview season and none (0%) in the 2023–2024 interview season. In comparison, national SOAP rates in those years were 11.3% and 12.2%, respectively, and the rates among other FMRN programs were close to the national numbers at 10.9% and 9.6%, respectively.^12,13 For programs considering whether to offer a hybrid interview format, the FMRN programs all ended up at around 60% virtual, 40% in-person interview ratio, which is a good place to start. Programs, however, should note that because they will be more likely to match from the in-person interviews than the virtual ones, the number of in-person interviews needed to generate one match in person may be fewer.

Limitations of this study included a small sample size in terms of programs, and heterogeneity among the location and structure of these programs. Data aggregation may have masked individual differences by program type or location (eg, urban vs rural). However, the opportunity to pool data from more than one program’s rank list was unique, and we were able to include six programs in this study. The study focused on a specific network of family medicine residency programs within a five-state region (WWAMI), and five of the six had their primary continuity clinic in a federally qualified health center, which may limit the generalizability of findings to other regions or specialties; although no data suggested that the WWAMI region nor family medicine residency recruiting differed significantly from other specialty recruiting practices. The findings reflected only two interview seasons (2022–2023 and 2023–2024), which may not capture all potential effects of hybrid interviewing over time. Additionally, beginning in the 2023–2024 interview season, applicants were given the ability to signal their top five programs in the application platform; that change may have impacted applicant and program interview behaviors, but we did not collect data on that process because it was not a factor in the first year of our study.

This study also raised a number of unanswered questions. Future research should explore the student experience and perspectives on interviewing with programs that offer hybrid interview formats. That research would provide insights into applicant satisfaction, perceived fairness, and potential preferences for one interview mode over another. Expanding similar research with other specialties and geographic locations also could be explored to determine whether the effects of hybrid interviews differ by specialty or region. Research on how programs organize their interview processes in general also would be additive because programs look for best practices and norms across a broader spectrum of residency programs. Continuing to collect outcomes from programs offering a hybrid interview format would help determine whether our findings hold true over time and provide insight into the longitudinal impact on resident performance, retention, and program satisfaction.

In this study, after controlling for those who participated in a subinternship, programs that offered applicants the choice of virtual or in-person interviews did not appear to influence rank lists or match results.

Early data from this project were presented at the Rural Training Track Collaborative Annual Meeting, April 2024, Asheville, NC; at the North American Primary Care Research Group (NAPCRG) Annual Conference, November 2024, Quebec City, Quebec, Canada; and at the Society of Teachers of Family Medicine Annual Conference, May 2025, Salt Lake City, UT. We have an abstract published on the first year of data only, tied to the NAPCRG presentation: Ormsby M, Evans D, Shih G, Weidner A. Face-to-face or digital space: which is more favorable for a residency applicant? Ann Fam Med. 2024;22(suppl 1):6166. doi:10.1370/afm.22.s1.6166