Evaluation of Family Medicine Residency Programs for Diversity, Equity, and Inclusion Milestones

Santina J.G. Wheat, MD, MPH | Katherine M. Wright, PhD, MPH | A. Ildiko Martonffy, MD | Suki Tepperberg, MD, MPH | Paul A. Ravenna, MD | John B. Waits, MD, CPE | Fadya El Rayess, MD, MPH

Fam Med. 2024;56(1):24-29.

DOI: 10.22454/FamMed.2023.919199

Return to Issue


Background and Objectives: In 2020, the Association of Family Medicine Residency Directors (AFMRD) Diversity and Health Equity (DHE) Task Force developed and piloted a framework to measure diversity, equity, and inclusion (DEI) initiatives in medical residencies across five domains: curriculum, evaluation, institution, resident pathway, and faculty pathway. The objectives were (1) to measure DEI initiatives across multiple domains in family medicine residencies using the DEI milestones and (2) to obtain current national baseline data providing criteria against which to measure effectiveness of initiatives and create tailored benchmarks.

Methods: We developed a cross-sectional survey of 12 quantitative residency characteristic items and the five DEI milestone ratings, and distributed the survey to program directors of family medicine residencies accredited by the Accreditation Council for Graduate Medical Education in the spring of 2022. We generated descriptive statistics, including item frequencies and cross-tabulations, and completed subgroup comparisons with analysis of variance.

Results: We collected aggregate milestone data for 194 family medicine residencies of 588 eligible programs (33% response rate). Respondents represented 48 states and US territories: 107 community-based, university-affiliated; 48 community-based; 34 university-based; and 5 military/other programs. Overall, the curriculum milestone was rated the highest (mean=2.54, SD=1.03), whereas the faculty pathway (mean=1.94, SD=1.04) and resident pathway (mean=2.02, SD=1.06) milestones were rated lowest.

Conclusions: DEI milestone data may support residency programs as they assess their institution’s developmental progress across five key domains. Additionally, aggregate data may shed light on collective strengths and areas for improvement.


With the increasing recognition of the long-term effects of racism on systemic health and health care inequities, medical education organizations have placed greater attention on diversity, equity, and inclusion (DEI). In 2020, the Accreditation Council for Graduate Medical Education (ACGME) released Common Program Requirements for residency programs to address DEI 1 without providing strategies to measure programming or specific benchmarks. Additionally, while recent studies have documented underrepresented in medicine (URiM) resident and faculty recruitment efforts in various residency specialties 2-10 and offered racial justice and health equity curricula, 11-13 no large evaluations of DEI efforts in graduate medical education programs have been conducted. Smaller assessments have included the development of a climate assessment tool at a single program 14 or institution, 15 and the Racial Justice Report Cards developed by White Coats for Black Lives for medical schools and residencies, which were used to evaluate 26 medical institutions. 16 Another survey of family medicine program directors 17 asked questions about discrimination complaints in the training environment.

In 2022, the Association of American Medical Colleges (AAMC) released a report on DEI competencies across the learning continuum as a way to measure competencies of individuals, not programs. 18 Available DEI templates for business and nonprofit organizations 19-22 do not meet the needs of graduate medical education, especially in terms of strategies to reduce bias in learner assessment, 23-26 and faculty promotion. 27 They also do not provide a framework to measure progress.

The Association of Family Medicine Residency Directors (AFMRD) Diversity and Health Equity (DHE) Task Force launched an initiative in 2019 to advance antiracism, diversity, and inclusion in family medicine residency training and developed a set of DEI milestones as the first known available tool to help residency programs identify areas of strength and opportunity within their programs. 28 The DEI milestones provide a framework for programs to evaluate themselves across multiple DEI domains, including curriculum, resident and faculty recruitment, resident evaluation, and institutional learning environment. After creating these milestones using a process that mirrored the creation of the ACGME family medicine milestones, committee members piloted these milestones within their programs to evaluate for usefulness and understanding. The committee sought to document the status of DEI initiatives in family medicine residencies nationally by asking program directors to rate their progress on the milestones.28 The findings establish a baseline that allows the AFMRD to tailor resource development to meet the needs of its members. The findings also provide individual programs with a benchmark to focus their activities and measure the effectiveness of initiatives undertaken to improve performance in the DEI domains, though individual programs may go about the evaluation process differently. This article describes the results of a baseline survey of US family medicine residency program directors implementing the DEI milestones.


Setting and Participants

We developed a cross-sectional survey and distributed it to program directors of ACGME-accredited family medicine residencies through the AFMRD member listserv. The survey was delivered online through the SurveyMonkey (SurveyMonkey Inc.) platform via a universal link; all responses were anonymous. Following an initial invitation to participate, an announcement was made at the annual meeting of the professional organization, and reminder emails were sent 2, 8, and 12 weeks later.

Survey Instrument

The survey instrument consisted of 12 quantitative items capturing residency program characteristics and the 5 DEI milestone ratings. The program characteristics questions mirrored those included in the Council of Academic Family Medicine Educational Research Alliance’s biannual survey, which is sent to program directors of family medicine residency programs to become familiar with them and to allow for future comparisons.

The curriculum milestone assesses implementation of DEI principles within the curricular content, such as the use of judgment-free terminology, social determinants of health (SDOH), and inclusion of antiracism content. The evaluation milestone assesses programs’ efforts to remove bias and increase objectivity within the promotion process along with inclusion of evaluation of residents’ ability to assess SDOH impact. The institutional milestone assesses the sponsoring institution’s integration with the community, hiring practices, and DEI policies and their application. The two pathways milestones for resident and faculty personnel, respectively, assess the programs’ recruitment, mentorship, and retention processes for residents and faculty. The full milestone framework has previously been described in detail. 28

The AFMRD Diversity and Health Equity Committee, consisting of 11 members with extensive experience as residency program directors or associate program directors, convened and drafted the first iteration of the milestone framework. 28 This framework was presented to more than 400 family medicine residency leaders for feedback. After refining items based on their input to improve readability and content validity, the survey launched on March 21, 2022, and closed on June 20, 2022.

Data Analysis

We generated descriptive statistics, including item frequencies, percentages, and cross-tabulations, for all survey items. Analysis of variance (ANOVA) and independent samples t tests were used for subgroup comparisons. Statistical analysis was performed using SPSS Version 28 for Windows (IBM Corp.). A P value of <.05 was determined to be statistically significant.

The American Academy of Family Physicians (AAFP) Institutional Review Board (IRB) reviewed and approved this project (protocol #22-432) on February 26, 2022.


Program Characteristics

Of 588 eligible program directors, 194 (33%) completed the survey. Respondents represented 48 states and US territories; additional characteristics of respondents’ residency programs are reported in Table 1.

Milestone Data

We calculated mean ratings (range 0.5–5.0) for all five DEI milestones (Figure 1). Overall, the curriculum milestone was rated the most advanced along the scoring rubric (mean=2.54, SD= 1.03), whereas the faculty pathway (mean=1.94, SD=1.04) and resident pathway (mean=2.02, SD=1.06) milestones were rated less advanced. The faculty and resident pathway milestones demonstrated the highest inter-item correlation (r=0.81, P<.01).

On average, the 100 programs with 25% or higher URiM residents reported significantly higher ratings on the curriculum (mean=2.71, SD=1.04, t[192]=–2.45, P=.02), faculty (mean=2.19, SD=1.02, t[192]=–3.51, P<.01), and resident (mean=2.26, SD=1.12, t[192]=–3.40, P<.01) pathway milestones compared to their counterparts with fewer URiM residents, as reported in Table 2.

We performed one-way ANOVA to compare average milestone ratings among residency programs of different complements. We observed significant differences for the curriculum (F[2,191]=3.95, P=.02) and faculty pathway milestones (F[2,191]=3.70, P=.03). Tukey’s HSD test for multiple comparisons revealed that larger programs (with greater than 31 residents) were rated significantly higher on the curriculum milestone than smaller programs with fewer than 19 residents (d=0.57, CI=0.09–1.06, P=.02). Regarding the faculty pathway milestone, larger programs rated themselves higher, on average, compared to smaller programs (d=.57, CI=0.07–1.06, P=.02), as reported in Table 3.

Similarly, one-way ANOVA revealed significant differences among residency programs of different institutional affiliations. On average, university-based programs reported higher milestone ratings compared to their community-based counterparts on all but the resident evaluation milestone, as denoted in Table 4.


This DEI milestone survey of US family medicine program directors found that programs that were university-based, located in larger communities, and had a large number of residents scored themselves higher on most but not all milestones when compared to their counterparts. Additionally, programs with 25% or higher URiM residents reported significantly higher scores on curriculum, faculty pathway, and resident pathway milestones. Programs overall rated themselves highest on the curriculum milestone.

Programs located in larger communities and those that were university-based scored higher on milestones. This may be because they drew a more diverse resident and faculty base that was more focused on DEI. Also possible is that institutional DEI initiatives at the university level provided greater access to DEI resources, thereby advancing DEI efforts.

We found higher scores in the curriculum, faculty pathway, and resident pathway milestones in programs with more URiM residents. Residents who identified as URiM could have been drawn to these programs, and programs might have been motivated to prioritize issues encompassed in these milestones to better meet the needs of their URiM residents who might enhance the program’s demographic diversity.

The scores for the curriculum milestone were higher than scores for other milestones. One explanation is that curriculum is more easily modified—with the effect of changes seen more quickly—than more complex domains like pathway and institutional domains, especially in larger institutions. In addition, resident feedback might have a more immediate and actionable effect on curriculum as opposed to the other domains.


The response rate was 33% of surveyed programs, and some programs may have program directors who are not represented in the AFMRD membership. Program characteristics are self-reported, and categories were not defined; however, these categories are used in biannual Council of Academic Family Medicine surveys and are familiar to program directors. Additionally, no public source of program characteristic data exists to determine whether this is a representative sample. Possibly, program directors from programs more attuned to DEI issues were more likely to complete the survey, which might have either over-sampled higher scoring programs or conversely over-sampled programs that rated themselves more critically. Milestone scores were self-rated, and programs with similar processes in place may have rated themselves differently. Because our survey is meant to be a tool that allows programs to rate themselves over time, the likelihood is that programs will continue to rate themselves in the same way, contributing to the validity of the rating rubric within each program. Also, a desirability bias may have impacted the ratings despite the anonymity of the survey. A survey completed by raters external to the program could overcome this potential bias.

The findings of this study provide a baseline assessment of the status of DEI in family medicine residencies. Next steps include a follow-up study of family medicine residencies at 1-year or every-other-year intervals to track changes in milestone scores over time. A mixed methods approach that includes qualitative surveys of programs to see how they are using the milestones and what interventions they have undertaken also would be useful. This information could include where the milestones are being discussed, such as among leadership only, with the program evaluation committee, or with all program members. Assessment of milestone progress also will allow organizations such as the AFMRD and the Society of Teachers of Family Medicine to focus on areas of demonstrated need, such as faculty development. Performing DEI milestone studies across other specialties and studies to assess reliability of the milestones among programs will be important as well.


Programs scored highest in the curriculum milestone and lowest on the faculty and resident pathway milestones. Larger and university-based programs as well as those in larger cities scored higher on several milestones. Milestone data may be useful in supporting residency programs as they assess their institution’s developmental progress across key DEI domains.


Information from this article has been presented at or accepted to be presented at the ACGME Equity Matters Symposium, the ACGME Annual Education Conference, the AAFP Resident Leadership Summit 2023, and the 2023 Society of Teachers of Family Medicine Annual Meeting.

Funding Sources

The application fee for the AAFP IRB was paid for by the AFMRD.


  1. Accreditation Council for Graduate Medical Education. Common Program Requirements (Residency). ACGME; September 17, 2022. Accessed October 4, 2023. https://www.acgme.org/globalassets/pfassets/programrequirements/cprresidency_2023.pdf
  2. Wusu MH, Tepperberg S, Weinberg JM, Saper RB. Matching our mission: a strategic plan to create a diverse family medicine residency. Fam Med. 2019;51(1):31-36. doi:10.22454/FamMed.2019.955445
  3. Cheng JL, Dibble EH, Baird GL, Gordon LL, Hyun H. Diversity, equity, and inclusion in academic nuclear medicine: national survey of nuclear medicine residency program directors. J Nucl Med. 2021;62(9):1,207-1,213. doi:10.2967/jnumed.120.260711
  4. Mendiola M, Modest AM, Huang GC. Striving for diversity: national survey of OB-GYN program directors reporting residency recruitment strategies for underrepresented minorities. J Surg Educ. 2021;78(5):1,476-1,482. doi:10.1016/j.jsurg.2021.02.007
  5. Whitley JA, Joyner BD, Asante PG, Waters WB, Kieran K. Exploring barriers to the inclusion of underrepresented minorities in urology: the temporal and financial costs of residency interviews. Urology. 2022;162:91-98. doi:10.1016/j.urology.2021.08.060
  6. Kearse LE, Jensen RM, Schmiederer IS, et al. Diversity, equity, and inclusion: a current analysis of general surgery residency programs. Am Surg. 2022;88(3):414-418. doi:10.1177/00031348211048824
  7. Gallegos M, Landry A, Alvarez A, et al. Holistic review, mitigating bias, and other strategies in residency recruitment for diversity, equity, and inclusion: an evidence-based guide to best practices from the council of residency directors in emergency medicine. West J Emerg Med. 2022;23(3):345-352. doi:10.5811/westjem.2022.3.54419
  8. Parsons M, Caldwell MT, Alvarez A, et al. Physician pipeline and pathway programs: an evidence-based guide to best practices for diversity, equity, and inclusion from the council of residency directors in emergency medicine. West J Emerg Med. 2022;23(4):514-524. doi:10.5811/westjem.2022.2.54875
  9. Roulier JP, Sung J. What will it take to recruit and train more underrepresented minority physicians in family medicine? a CERA survey analysis. Fam Med. 2020;52(9):661-664. doi:10.22454/FamMed.2020.453883
  10. Davenport D, Alvarez A, Natesan S, et al. Faculty recruitment, retention, and representation in leadership: an evidence-based guide to best practices for diversity, equity, and inclusion from the Council of Residency Directors in Emergency Medicine. West J Emerg Med. 2022;23(1):62-71. doi:10.5811/westjem.2021.8.53754
  11. Gagliardi JP, Smith CM, Simmons KL, Tweedy DS. Racial justice beyond the curriculum: aligning systems of care with anti-racist instruction in graduate medical education. J Grad Med Educ. 2022;14(4):403-406. doi:10.4300/JGME-D-22-00056.1
  12. Wusu MH, Baldwin M, Semenya AM, Moreno G, Wilson SA. Racial justice curricula in family medicine residency programs: a CERA survey of program directors. Fam Med. 2022;54(2):114-122. doi:10.22454/FamMed.2022.189296
  13. Edgoose J, Davis S. et. al. A Guidebook to the Health Equity Curricular Toolkit. Health Equity Team for Family Medicine for America’s Health; 2018. Accessed December 31, 2022. https://www.aafp.org/family-physician/patient-care/the-everyone-project/health-equity-tools.html
  14. Smith R, Johnson A, Targan A, Piggott C, Kvach E. Taking our own temperature: using a residency climate survey to support minority voices. Fam Med. 2022;54(2):129-133. doi:10.22454/FamMed.2022.344019
  15. Tesfaye Y, Keppley M, Harchanko E, et al. Climate survey development to assess the DEI gap across surgical specialties in an academic institution. Am J Surg. 2022;223(2):425-427. doi:10.1016/j.amjsurg.2021.08.009
  16. White Coats for Black Lives: Racial Justice Report Card 2020-2021. Accessed December 11, 2022. https://whitecoats4blacklives.org/rjrc
  17. Potts SE, McLean I, Saba GW, Moreno G, Edgoose J, Candib LM. Diversity and facing discrimination in family medicine residencies: a CERA survey of program directors. Fam Med. 2021;53(10):871-877. doi:10.22454/FamMed.2021.482291
  18. Association of American Medical Colleges. Diversity, Equity, and Inclusion Competencies Across the Learning Continuum. AAMC; July 2022. https://store.aamc.org/diversity-equity-and-inclusion-competencies-across-the-learning-continuum.html
  19. Oyetunde T, Boulin A, Holt J. Equity Diversity Inclusion Action Toolkit for Organizations. American Public Health Association; 2020. Accessed December 31, 2022. https://www.apha.org/-/media/Files/PDF/affiliates/equity_toolkit.ashx
  20. Joe M, Waddy R. Racial Equity Toolkit: Applying a Racial Equity Lens to Your Organization. Housing Development Consortium. Accessed December 31, 2022. https://www.housingconsortium.org/wp-content/uploads/2016/11/Racial-Equity-Toolkit-Downloadable.pdf
  21. Organizational assessment tools and resources. Racial Equity Tools. Accessed December 31, 2022. https://www.racialequitytools.org/resources/plan/informing-the-plan/organizational-assessment-tools-and-resources
  22. Jackson B, Hardiman R. Continuum on becoming an anti-racist multicultural organization. Crossroads Ministry. Accessed February 1, 2022. https://philanos.org/resources/Documents/Conference%202020/Pre-Read%20PDFs/Continuum_AntiRacist.pdf
  23. Boatright D, Ross D, O’Connor P, Moore E, Nunez-Smith M. Racial disparities in medical student membership in the Alpha Omega Alpha Honor Society. JAMA Intern Med. 2017;177(5):659-665. doi:10.1001/jamainternmed.2016.9623
  24. Low D, Pollack SW, Liao ZC, et al. Racial/ethnic disparities in clinical grading in medical school. Teach Learn Med. 2019;31(5):487-496. doi:10.1080/10401334.2019.1597724
  25. Rojek AE, Khanna R, Yim JWL, et al. Differences in narrative language in evaluations of medical students by gender and under-represented minority status. J Gen Intern Med. 2019;34(5):684-691. doi:10.1007/s11606-019-04889-9
  26. Zhang N, Blissett S, Anderson D, O’Sullivan P, Qasim A. Race and gender bias in internal medicine program director letters of recommendation. J Grad Med Educ. 2021;13(3):335-344. doi:10.4300/JGME-D-20-00929.1
  27. Nunez-Smith M, Ciarleglio M, Sandoval-Schaefer T, et al. Institutional variation in the promotion of racial/ethnic minority faculty at US medical schools. Am J Pub Health. 2012;102(5):852–858. doi:10.2105/AJPH.2011.300552
  28. Ravenna PA, Wheat S, El Rayess F, et al. Diversity, equity, and inclusion milestones: creation of a tool to evaluate graduate medical education programs. J Grad Med Educ. 2022;14(2):166-170. doi:10.4300/JGME-D-21-00723.1

Lead Author

Santina J.G. Wheat, MD, MPH

Affiliations: Northwestern University Feinberg School of Medicine, Chicago, IL | Northwestern McGaw Family Medicine Residency at Humboldt Park, Chicago, IL


Katherine M. Wright, PhD, MPH - Northwestern University Feinberg School of Medicine, Chicago, IL

A. Ildiko Martonffy, MD - Department of Family Medicine and Community Health, Madison Residency, University of Wisconsin, Madison, WI

Suki Tepperberg, MD, MPH - Family Medicine Residency, Boston University Chobanian and Avedisian School of Medicine, Boston, MA

Paul A. Ravenna, MD - Northwestern Lake Forest Family Medicine Residency, Chicago, IL

John B. Waits, MD, CPE - Cahaba and University of Alabama at Birmingham Family Medicine Residency, Cahaba Medical Care Foundation, Centreville, AL

Fadya El Rayess, MD, MPH - Cahaba and University of Alabama at Birmingham Family Medicine Residency, Cahaba Medical Care Foundation, Centreville, AL

Corresponding Author

Santina J.G. Wheat, MD, MPH

Correspondence: Northwestern University Feinberg School of Medicine, Chicago, IL

Email: swheat@eriefamilyhealth.org

Fetching other articles...

Loading the comment form...

Submitting your comment...

There are no comments for this article.

Downloads & Info


Related Content


Searching for articles...