Efforts to Recruit Diverse Resident Complements in Family Medicine

Family medicine (FM) is the most diverse specialty in contemporary graduate medical education (GME); however, at a national scale, the demographics of FM residency have not achieved parity with the general US population. ¹ This study explored what efforts FM program directors reported making to diversify their resident complements in order to definitively roster the range of options that programs may consider when they contemplate their next steps toward greater inclusivity. To do this, we began with the annual update reports that programs are required to file with the Accreditation Council for Graduate Medical Education (ACGME). For those reports, program directors were prompted to describe their efforts to recruit and retain diverse resident complements. We then correlated the program directors’ statements with recommendations on diversifying America’s health workforce from two blue-ribbon committees. We hypothesized that the recommendations of the blue-ribbon committees would be reflected in the program directors’ statements to the ACGME.

Two blue-ribbon commissions each produced a groundbreaking report. The Sullivan Commission’s Missing Persons: Minorities in the Health Professions ² and the Institute of Medicine’s In the Nation’s Compelling Interest: Ensuring Diversity in the Health Care Workforce ³ were published in 2004 with significant input from leading figures in medicine, advocacy, government, and business. Since their publication, many of the country’s leading medical organizations have echoed the recommendations. For example, the American Academy of Family Physicians (AAFP) has focused on related themes, such as bringing more women and racial or ethnic minorities into medicine. ⁴ The Association of American Medical Colleges (AAMC) urged policymakers to prioritize research and initiatives for increasing diversity in the physician workforce, noting that doing so would contribute to providing care in underserved areas and remediate population health disparities. ^{5, 6} More recently, the American Association of Colleges of Osteopathic Medicine adopted a consensus statement on diversity, equity, and inclusion, which addressed workforce development. ⁷

The effectiveness of these recommendations has been supported in scholarly literature. For example, Stoesser and her colleagues instituted an array of recruiting activities, which together resulted in increasing numbers of Hispanic and Asian residents at their institution. ⁸ These initiatives included ensuring diversity on interview panels, recognizing applicants for overcoming hardships, engaging their organization’s diversity office, and priority interviewing for candidates of disadvantaged backgrounds. Similarly, Wusu et al reported increasing minority matches to their program through strategic outreach, carefully worded interview questions, and monitoring of outcome data. ⁹

Similarly, Emery et al ¹⁰ found effectiveness in removing organizational barriers, building pipelines with K-12 or baccalaureate levels of education to foster student attachment to the study of science, and helping minority students overcome stereotypes or self-perceptions that hinder academic achievement.

The Sullivan Commission and Institute of Medicine (IOM) reports, however, were unique in their authoritative contributors and in the breadth of their recommendations. To date, no effort has been made to synthesize their recommendations and contrast them with what residency programs are doing en masse. This synthesis could provide a single menu to serve as a resource for programs that may need guidance on diversifying their resident complements. Some attempts have been made through survey research to quantify diversity, equity, and inclusion (DEI) initiatives currently in use. ¹¹ We, instead, relied on statements that program directors made to the ACGME. The ACGME’s common program requirements (CPRs) ¹² mandate that accredited programs “engage in practices that focus on mission-driven, ongoing, systematic recruitment and retention of a diverse and inclusive workforce of residents and fellows” (p. 5). The annual updates prompt program directors to explain how they fulfill that CPR. ¹² The annual updates provide the widest coverage of the steps programs take to fulfill this ACGME CPR of diversifying resident complements.

In this paper, we report on the FM program directors’ annual update responses addressing how they recruited residents. With 701 programs, FM makes up the greatest proportion (13%) of residency programs across all specialties. In 2020, FM had the second largest proportion of total residents (11.5%, with 13,725 residents), and FM has achieved parity in terms of gender (53.5% of residents were female). As such, FM should offer the widest possible variety of initiatives in this regard and therefore serve as a template for other disciplines.

This paper is both a qualitative comparison of the FM program directors’ responses from annual updates and the recommendations of the Sullivan Commission and IOM. In the nearly 20 years since the two commissions’ recommendations were published, we thought that programs may have undertaken initiatives of their own creation, which our methods allowed us to document. Since 2019, the ACGME’s annual updates have prompted program directors to "describe how the program will achieve/ensure diversity in trainee recruitment, selection, and retention." This item corresponds with the ACGME’s CPR that programs engage systematic efforts to recruit and retain a diverse resident workforce. Since 2019, the annual updates have been reported by program directors, or their staff, and the results are stored in the ACGME’s accreditation data system.

Our analytical methods were rooted in template analysis. ^{13, 14} Template analysis organizes open-ended statements by coding them into important and recurring themes. ¹⁵ First, our project team received an electronic file containing program directors’ responses from the annual update. Data fields specifically designed to record the program’s name, the name of the sponsoring institution, location, and the name of the program director were excluded from the file delivered to the project team. However, some authors of annual update responses included identifying information in their narratives. Those responses were left intact when delivered to the research team.

Second, we imported the annual update into NVivo, a software application commonly used in qualitative research. There we used a system of open coding to create an initial classification of recurring themes at a granular level. ¹⁶ For example, one program director reported, “Quarterly Equity, Diversity, and Inclusion sessions required for all residents and faculty were integrated into the didactic schedule,” while another said, “Formal training will continue to educate all learners and faculty about unconscious bias, racial disparities, and cultural competence and their effects on healthcare outcomes.” We coded comments like these under the theme DEI curriculum.

Two members of the project team independently reviewed and coded each of the annual update responses to document recurring themes. Then we combined those two lists into one preliminary template documenting the recurring themes that both team members found.

Next, all three members of the project team worked in tandem to code a randomly selected set of 20 program directors’ responses using this preliminary template. This exercise allowed us to (a) eliminate redundant or extraneous codes, (b) add new codes, (c) ensure that the project team was like-minded about the coding, and (d) ensure that we were of like mind about any restated activity theme. After that, two of us independently coded the remaining entries. Next, the two coders together reviewed initial coding and attempted to resolve discrepancies. We sent those that we could not reconcile to a third member of our team to decide.

By coding the annual update statements before aligning them to the two reports, we were sure to acknowledge any efforts that had not been recommended by the Sullivan Commission or IOM. Consequently, our initial coding was more granular than the two reports. For example, the Sullivan Commission’s recommendation 4.7 advised that learners from disadvantaged backgrounds should be afforded “an array of support services, including mentoring, test-taking skills, counseling on application procedures, and interviewing skills.” We coded learning support, such as test-taking skills and mentoring, as two distinct categories. Thus, our coding produced more than one activity corresponding to a single Sullivan Commission or IOM recommendation.

Only after coding the annual update responses did we apply the principles of thematic analysis to develop a crosswalk of the FM program directors’ responses to the recommendations from the Sullivan Commission and IOM reports. ^{17, 18} We did this by collectively identifying those Sullivan Commission and IOM recommendations that had a theme that was parallel to our restatements of the themes. Figure 1 depicts our process. The way the themes were crosswalked is shown in Appendix Table A.

We excluded some items in the two reports and the annual updates from study because they did not relate to processes in resident recruiting. For example, the two reports recommended addressing governmental regulation, leadership development, and collaborations with community organizations. Some of the annual update statements referred to demographic characteristics. We focused only on those report recommendations or annual update activities that directly addressed resident recruiting and retaining processes.

This project was exempted from review by the Institutional Review Board of the American Institutes for Research.

The ACGME accredited 701 FM programs in 2020. Of the 701 programs, 689 (98%) provided responses to the prompt about efforts to recruit and retain diverse resident complements in the annual update. Their responses varied in terms of the number of words used to describe their efforts, how directly they responded to the prompt, the variety of efforts, and the goals to be achieved. We identified 31 unique themes in the annual updates. Twenty-three had a counterpart in either or both the Sullivan Commission and IOM reports, and another eight were unique to the annual updates. As we reviewed the crosswalk, we realized that individual themes could fit into higher-level classifications. For example, program directors indicated that they emphasized their organizations’ training in nondiscriminatory hiring policies, appointment of a DEI officer, and DEI committees. These are discreet activities that could be classified into a larger domain for organizational commitments to DEI.

Table 1 illustrates how frequently each of the restatements of the Sullivan Commission and IOM recommendations are mentioned in the annual updates.

Next, we discuss the themes, along with illustrative quotes. Appendix Table B provides examples from the annual update for each of the restatements.

Some program directors reported systematically reviewing and modifying their initiatives based on analysis of data. These data typically included the number of underrepresented minority (URM) candidates who applied or interviewed, or the results of candidate feedback surveys. For example, one program director reported, “We analyze data available from NRMP (National Resident Matching Program) and ERAS (Electronic Residency Application Service) to determine whether our recruitment efforts are effective in recruiting and interviewing a diverse applicant group.”

For some programs, the recruiting process was motivated by an ethical purpose, such as improving patient care, improving the educational experience, or reflecting the population served. One program director indicated,

Many programs offered a system of DEI curriculum and learning supports. That curriculum included cultural competency, second languages (medical Spanish), reflective practices, or readings about the experiences of marginalized individuals. One program director indicated,

Supports included interventions like mentoring, tutoring, or social services that encouraged the residents’ professional development. One program director summarized the array of supports:

Many program directors wrote that the recruiting process was aligned with an institutional mission. For example, one program director indicated,

Program directors often wrote that their processes included a commitment of resources to advance DEI in recruiting. That may have included offsetting candidate travel costs, protecting time for DEI committee participation, training staff on nondiscrimination policy, or appointing an ombudsman to investigate claims of discrimination.

Many programs included efforts to attract prospective residents to the field of medicine. Those efforts included pathways, or pipelines, which are systematic efforts introducing high school and college students to medicine. Outreach included updating promotional materials or recruiting at conferences. One program director reported that the institution makes

Selection procedures is a category of diverse initiatives. Examples include training recruiting committees on hiring policy, diversifying recruiting committees, and blinding the program to applicant demographic information in ERAS. Programs frequently used holistic review, by which applicants are credited for attributes like foreign-language fluency, perseverance, overcoming adversities, attachment to the community, or relevant experiences. Some programs integrated holistic and merit criteria. For example, one program director wrote, “The review uses a point system for letters of recommendation, research, publications, academics, life experiences, rural-community involvement, and leadership.” At the same time, many other programs construed merit criteria to include only academic qualifications, suggesting that merit is a nondiscriminatory recruiting practice. For example,

Appendix Table B provides examples of how program directors’ statements corresponded to the themes just described.

We found that contemporary FM programs, in our total sample, have implemented an even wider array of efforts than the Sullivan Commission or IOM recommended. Except for engaging in public awareness campaigns, all the recommendations from the Sullivan Commission and IOM have been implemented. We rostered those activities and aligned them with the Sullivan Commission and IOM recommendations, presenting a comprehensive menu of options from which to choose when identifying next steps on the journey toward a recruiting process that includes a robust array of DEI elements.

This menu can be helpful to programs because a growing body of research has demonstrated the efficacy of DEI interventions. Diversity in the learning environment has been linked to fewer experiences of discrimination and harassment against trainees. As Hartman and colleagues¹⁹ found, merit criteria, such as the US Medical Licensing Examination, COMLEX scores, or class rankings, were weakly correlated with residency performance. ^{19, 20} The reliance on test scores may lead to fewer URM candidates receiving invitations to interview. ^{21, 22}

Diversity in the medical profession is important because it leads to improved patient experiences. Patients place more trust in doctors who share their racial or ethnic background, have a greater sense of partnership with their physicians, experience elevated patient satisfaction, have an enhanced understanding of their health care needs, and improve their compliance with treatment regimens. ^{23, 24} To achieve the goal of improving patient care, the recommendations of the two reports have been cited in the extant literature that attends to developing a more diverse health care workforce (eg, see Toledo et al ²⁵; Bonini and Matias ²⁶; and Velez-McEvoy ²⁷).

The Association of Family Medicine Residency Directors Health Equity Task Force developed a DEI Competency Milestones framework, ²⁸ which uses a rubric akin to the ACGME’s Milestones. That DEI assessment tool can help programs evaluate the degree to which faculty and residents incorporate DEI practices in learning environments and patient encounters.

This paper is a first step that may prompt further work to collate disparate research on the efficacy of various interventions aimed at diversifying the medical workforce. Medical education still needs a better understanding of the effectiveness of individual interventions, the obstacles to and facilitators of implementation, and the importance of the context of individual programs and their sponsoring organizations. Additional work would help describe how programs decide which initiatives to put into action and how applicants view these initiatives when considering their rankings for a residency match.

How completely and accurately program directors reported their DEI recruiting initiatives in the annual update is unclear. Some programs may have offered only a minimally satisfactory answer to the ACGME’s mandatory paperwork, while others may have overstated their efforts, hoping to leave a favorable impression on an accrediting agency. Program directors may have been thorough in their responses to the prompt in this reporting cycle because health and social disparities were particularly salient amid reports of police misconduct against minorities and the uneven effects of the COVID-19 pandemic during this reporting cycle. ^29-32 The annual update did not prompt programs to align their efforts with the Sullivan Commission or IOM recommendations; still, we see nearly all of them being mentioned. A multiple-choice questionnaire format would more clearly quantify the use of each intervention.

The two blue-ribbon commission reports spoke more to the admissions and progress of students than to the recruitment and retention of residents. This analysis was an act of retrofitting those recommendations to align with the experience of GME. We mitigated the risk of individual interpretations misaligning those concepts first by relying on several coders who independently and then collectively coded the annual update responses, secondly by separately and then collectively aligning the Sullivan Commission and IOM recommendations, and finally by comparing the annual update responses to the recommendations.

We found that many programs have implemented recommendations from the Sullivan Commission and IOM reports. Many FM residency programs have instituted recruiting activities beyond those recommended in either of the reports. Together these sources provide a menu of options worthy of consideration by GME programs in any discipline thinking about its next steps in resident recruiting and retention.