Surveying Applicants to Improve the Family Medicine Residency Interview Day

The investment family medicine residency programs make in recruiting is significant, though outcomes and trends are unassessed. Applicant investment in the process is also significant, with 62% of applicants reporting expenditures ranging between $1,000 to $4,000.¹

The American Academy of Family Physicians (AAFP) publication “Strolling Through the Match” serves as a guide for residency applicants.² However, there is no reference to published data or quantified survey results. One study examining the characteristics of the family medicine residency process is over 30 years old.³ Several other medical specialties have published data on the interview process, ranging from the preinterview dinner, number of interviews, and financial and educational costs for these specialties.^4-9

Program directors cite the interview as the top factor for ranking¹⁰ and candidates rely heavily on the quality of interactions and the interview day as a whole as key factors.¹⁰ 

Given the impact of the interview day and the resources expended by applicants and programs, it is prudent for programs to structure an experience that suits the needs of both parties. While programs may know their own needs, they do not necessarily know the needs and expectations of applicants.

This study suggests a method whereby individual programs can adapt the interview day to the preferences of the type of applicants it attracts. Doing so can facilitate programs emulating or improving upon practices used elsewhere, satisfy applicant preferences, and justify program expenditures. This study also identifies preferences and trends among applicants interviewing at one program during the 2014-2015 and 2015-2016 application cycles, and may suggest broader trends and preferences of applicants in general.

The Penn State College of Medicine Institutional Review Board approved this study (study number: 00001266). Family medicine residency applicants’ experiences and opinions regarding the residency interview process were investigated via a paper-based, anonymous, multiple-choice survey. There were no personal identifying questions. Questions were relevant to the preinterview dinner, interview day, and postinterview day communication methods. 

The survey was distributed to family medicine residency program applicants during the interview day at Penn State Health Milton S. Hershey Medical Center in 2 consecutive years, from October 2014 through January 2015, and October 2015 through January 2016. All applicants who attended an interview were given the paper-based survey on the day of the interview during a scheduled break, and all were directed to complete the survey by the end of the interview day.

Applicants placed completed surveys into a locked box until the conclusion of interview season to ensure anonymity. Surveys were analyzed by the principal investigator and coinvestigator only. Using Microsoft Excel computation, the responses for each question were totaled and percentages calculated.

A total of 132 surveys were analyzed, with 67 surveys collected from 68 eligible candidates in 2014-2015, and 65 surveys from 65 eligible candidates in 2015-2016, response rates of 98.5% and 100% respectively. Table 1 gives a description of the sample demographics. Figure 1 shows results represented as a percentage of the 132 applicants who selected the corresponding multiple choice answer.

Figure 2 shows additional survey results based on applicant views on the importance of each item presented. The majority of applicants felt it “essential” to visit the outpatient clinical site (73.5%), interview with a current resident (59.8%), and interview with the program director (71.2%). Meeting the department chair of the academic institution was “important” to 58.3% and “essential” to 15.2%.

Applicants preferred a restaurant for the preinterview dinner (49%), while 40% had no preference, and 11% preferred the dinner to be held at a current resident’s apartment/home. Only 37% of those surveyed had ever attended a preinterview dinner at a current resident apartment/home.

Table 2 shows responses to supplemental questions. The majority of applicants (82%) felt that a total of three interviews during a visit was appropriate. Email communication was the preferred postinterview day communication follow-up method (79%). No applicants favored telephone communication follow-up.

This study offers two resources of value to residency programs. It suggests that individual programs can utilize surveys of their own applicants to ensure that the structure of their interview process and the resources they deploy in recruiting are aligned with the expectations and preferences of that applicant pool. The study also provides some limited information on what those expectations and preferences are for one program’s pool of applicants, which can provide a thumbnail sketch to programs more generally and lay the groundwork for future inquiries.

A limitation of this study is that it was performed within a single specialty at one academic East Coast medical center, and the surveyed population was solely students of Liaison Committee on Medical Education-accredited medical schools, limiting its generalizability. Another limitation is the nature of the survey administration, having been completed on a rolling basis throughout the interviewing season. This meant that those applicants who interviewed and completed the survey earlier in the year had much less experience to draw upon than those applicants who interviewed and completed the survey late in the season.

This approach allowed Penn State Hershey to improve its interview day structure based on applicant preferences. Following the conclusion of the first collection cycle, a resident interviewer was added to the agenda based on applicant preferences. Similarly, the program decided to continue to sponsor hotel accommodations for applicants and to host the preinterview day meal at a restaurant rather than a current resident’s apartment/home.

Future work should include a collaborative effort across family medicine residency programs to expand the pool of programs or cross-specialty studies of the use and impact of interview day preference surveys. Future work may also include a redesign of the survey instrument to allow more response options at the low end of resource utilization to reflect that applicants may encounter “none” or “very few” programs offering resources such as dinner or hotel accommodations.

Surveying applicants along the residency interview trail can be an effective method to explore trends and adapt the interview day experience to the preferences of the applicants the program attracts.