Departmental Metrics to Guide Equity, Diversity, and Inclusion for Academic Family Medicine Departments

Academic family medicine departments across the United States have set goals and processes in motion to improve equity in their practices, workplaces, and learning environments. ¹ Departments have been comparing notes and learning from one another, particularly since the recent acceleration of efforts to address structural racism and advance social justice. In addition, family medicine is the most diverse medical specialty in terms of leadership and membership.^{2, 3} In the summer of 2021, the Association of Departments of Family Medicine (ADFM) Diversity, Equity, and Inclusion Committee learned that departments were interested in ways to measure equity, diversity, and inclusion (EDI) baselines and monitor progress within departments over time. Knowing how to assess outcomes across this broad EDI space has been a challenge to departments. The Committee envisioned framing a broad range of metrics that included areas for practice, workforce, learning environment, and research in which EDI efforts take place. Such metrics would allow departments to assess their current state and measure how successful they are in achieving their own EDI strategic aims. A metrics framework shared by departments could be valuable internally and for its potential to aggregate across departments to see overall progress more clearly.

Different departments or institutions use different terms or acronyms for the same or similar core ideas. For example, some use different juxtapositions of diversity, equity, inclusion, and justice (DEI, EDI, JEDI); use the term belonging instead of inclusion; or add a particular emphasis such as health equity in the context of workplace, learner, and research equity; in the end, they all point to equity for patients, learners, and all health care team members. Emphasis on “isms” such as racism, in the context of sex, gender identity, age, disability, or another characteristic also may be the subject of EDI work. These different acronyms or terms may arise from different traditions, ways of thinking, purposes, or emphases. Importantly, these terms are a constellation of interrelated concepts that together capture the purposes and subject matter (ie, becoming increasingly fair and safe for all), recognizing that no single term captures it all. Diversifying is one aspect, being inclusive is another, achieving belonging another, and having these all add up to being equitable is yet another aspect of this overarching goal. Different people at different times may emphasize or feature various aspects and arrange their acronyms accordingly. For example, some think of equity as more of an end goal, with diversity and inclusion as approaches to achieving that goal; so, they capture this with EDI (as the authors do in this paper). For others, diversity is the starting point, so they emphasize this ideal with DEI instead. Another approach has justice as a priority goal enabled by equity, diversity, and inclusion; and this is captured with JEDI. For others, belonging is experienced more like a result than inclusion (ie, felt to be more like a means), or belonging may be the heart of how equity, diversity, and inclusion interact. Hence, different phrases and acronyms are housed under this broad constellation of interrelated concepts with many specific terms and glossaries for departments to use accordingly.^{4 ,5, 6}

Different acronyms and phrases can peacefully coexist; nailing down a term or arguing about which one is better is not necessary. Different academic institutions already use many different acronyms at different college and department levels, and all appear to be aiming for the same general goals. All these words exist in local contexts and add up to something greater and more integrated than any one of them alone. 

The intention is to provide a broad metrics framework, not to tell people which specific data elements to monitor in their own settings. The framework depicts a range of metric areas and invites departments to decide what is timely and important for them to measure, and then to look for specific data elements that are available or can be created.

The framework creates an organized space for different kinds of observables distributed across department missions. But it does not offer universally defined data elements. These are up to departments, which indeed makes cross-department comparison challenging. Having a shared framework for different kinds of observables is useful even if the specific data elements used are not quite comparable. Over time, departments can begin to define data elements that are comparable across departments.

Table 1 and Table 2 are an example of such a framework, or dashboard, adapted from one family medicine department and broadened for presentation by the authors. The data elements shown here are merely examples used by one department and serve to evoke individual department thinking on what should appear for them in the cells of the table and what they can look for in the world of observables to monitor their own progress. This framework emerged from facilitated conversations among family medicine faculty as they set out to form a department-wide approach to improving EDI. ⁷ First, the scope of work was cast as pillars of action in EDI, with work groups and goals formed in each one, with the intent to populate the pillars as a metrics framework with data that became available. This framework has helped accommodate quantitative and qualitative information across the spectrum, from simple counts to changed processes (process measures) to actual results (outcome measures). Second, the framework helps show the distribution of diversity, inclusion, and equity measures in use. Last, it creates data cells that do not yet have measurable data in them, as a reminder that metrics may not yet exist for everything. Selected metrics can align with individual departmental priorities to demonstrate progress and help show the value in EDI work. Therefore, Tables 1 and 2 are intended to provide a broad overview, with examples from which departments might construct their own EDI dashboards. Such measures can have significant overlap. For example, inclusion can be the actions taken to improve diversity and equity, in a means–ends relationship. The authors acknowledge challenges and limitations surrounding measurement, not only in finding particular data elements to reflect important observables, but in considering the social context such as controversies around affirmative action, identity-linked metrics, quotas that are illegal and not suggested here, and local legislation that can set boundaries around an institution’s ability to act.

Measurements, like EDI paths themselves, take time. Departments are unlikely to be able to immediately measure the ultimate outcomes they want to see, such as reduced health disparities, a representative workforce that experiences belonging, or a diverse thriving learner cohort. The journey is longitudinal, getting to those outcomes because of changing many things over time; the results departments want rarely suddenly appear. Measuring ultimate outcomes is challenging because convenient observables expressed in numbers are often not readily available. Many institutions have human resources or equity, diversity, and inclusion offices with access to shareable institutional and departmental data that could supply baseline information.

At the outset, departments may just start counting things, even though such counts are not the goal. Admittedly, the numbers of faculty or staff from underrepresented or minoritized backgrounds, or the number of community-engaged research projects may be low; when looking at these data, small changes to low numbers result in high percentage changes. Over time, the contents of a dashboard can evolve from counts to indicators of the ultimate desired outcomes. Some faculty may be impatient with what is on the dashboard early on, viewing metrics as only superficial process indicators, not real outcomes. While the authors recognize this critique, taking a developmental view not only of EDI progress but of metrics development itself can be helpful. One example to illustrate the concept and process of metrics development is the Association of Family Medicine Residency Directors milestones.¹⁵ These highlight a five-level sequence of development for residents and faculty that spans the evolution from recruitment to leadership, the resident evaluation process, and the curriculum. Showing progress along this pathway could further support the EDI journey at your institution; for example, a shared metrics framework could be used during recruitment to inform prospective residents when ranking programs. The authors recognize that the underrepresentation of Black, Indigenous, and other People of Color (BIPOC) in medical schools adds a challenge in diversifying residency programs and departmental faculty. Departmental engagement in family medicine interest groups, primary care educational panels, and dissemination of information on family medicine can increase BIPOC participation in pathway programs and serve as an additional metric for measurement. Leaders can quantify the number of faculty or residents that participate in the process to recruit or foster growth for diverse talent. The framework of Tables 1 and 2 is adaptable to initial counts, simple measures, qualitative information, and more outcome-oriented measures when available. Departments can start with what is observable in their current environment and then build onto their dashboard from there.

This framework portrays a range of possible metrics that a department could measure on its EDI journey—an organized set, or menu, of options that cover the broad range of EDI actions. Departments choose which areas are important for them to measure and find specific metrics suitable to their situations. The framework offers examples of what could be measured, not what must be measured, and serves as a template with different options from which to choose. The framework of Tables 1 and 2 has examples, but is not populated here with specific, definitive, validated measures or tools, some of which may not yet be available or universally used. Departments can use this dashboard framework to build and adapt existing metrics for EDI concepts that are meaningful and feasible for them, and then allow their system to evolve over time. Rather than repeating some concepts in each pillar, some measures, such as climate or resident competencies, are shown in Table 2 as cutting across all the pillars of Table 1—in clinics, teaching, the working environment, and research.

The value of a shared general metrics framework is that departments can more easily see what is relevant to measure at any given point on their EDI path; accordingly, they may have to create specific measures within their chosen cells of the table, even if their institutions already have available or recommended metrics. Beyond benefit to individual departments, having a shared framework across departments (such as member departments of ADFM) facilitates comparing notes and aggregating information on what areas are being measured, on goals for specific measures, and ultimately on progress with EDI over time and across departments. A shared framework helps set the stage for measuring aggregate progress.

The following general reminders are designed to help departments be realistic and avoid common pitfalls in measurement in the EDI area. Overall, treat metrics as ways of assessing progress toward expressed goals, not as ends in themselves. Metrics can help measure progress with intermediate goals but may fall short of the overall goal. For example, measuring diversity statistics will reveal how many historically excluded people work in a department, but will not disclose whether the workspace is equitable, diverse, and inclusive enough. Metrics also may not measure the extent the workforce is representative of the community or whether everyone can be at the table. Expressing such priority goals simply helps ground measurement to what really matters, in ordinary language, and can be used to show progress.

1. Include the perspective of people who may feel excluded in choice of EDI goals and measures. Some people may have (and accept) low expectations based on previous experience. So, consider further qualifying these perspectives with expectations of the goals or measures being fair, safe, equitable, diverse, and inclusive as part of baseline data.

2. Focus observations on key priorities important to your department. The focus of the dashboard should be on a few metrics for items that are truly important. Including multiple or peripherally relevant measures, just because they are available, might not add value in illuminating priority goals.

3. Honor qualitative information. Do not insist on objective or numerical data. A measure also can be subjective information involving experiences, which can be gathered and analyzed in the spirit of good qualitative traditions to compare over time.

4. Be willing to collect new data for the dashboard. This data could get at what a department may now see but did not previously look at. Do not insist that new measures must already be validated if that comes at the expense of collecting anything at all in a new important area. Data elements to get at an important kind of observable can be investigated and refined over time, while temporarily remaining provisional.

5. Favor plain and simple because it is powerful. Simple framing communicates what is being evaluated without the need for multiple-choice questions that leave little room for interpretation.

The following reminders are specific to patients.

1. Patients may have different views of what makes a practice equitable or inclusive. Consider including patient perspectives of inclusion. For example, consider how closely the practice or providers resemble them or their community. These views are important to ponder when striving for the overarching goal.

2. Patients self-identify as part of a minority group in different ways. Capturing patient-reported experience can become part of the information on the diversities of the communities served, which can be measured for changes over time. 

3. Portray patient composition of practice in different ways. In addition to race or ethnicity, consider looking at segments of insurance, primary language, income, or chronic condition burden. These may be indicators for socioeconomic complexities that affect the sense of identification with the practice or the perception of it being equitable.

This general framework is presented to departments to stimulate development of their own EDI measures or dashboards for recording baselines and improvements over time on this longitudinal path. The hope is that the creation of common intellectual frameworks and measures in the wide scope of family medicine can help all departments create their own dashboards and demonstrate progress with comparable concepts across departments. The framework is offered as an organized space from which to choose areas for measurement and as a prompt for developing (or adopting existing) specific metrics in those areas, knowing that the use of different locally available data elements can make comparing departments difficult.

The authors recognize that many of the priority goals may be difficult to measure in simple ways with metrics that already are in place. Surveys have limitations, whether for patients or department leaders. Being aware of institutional history and the context or baseline the department is addressing is important. To address these potential limitations, consider piloting the dashboard across one clinic site (or departmental division) in collaboration with the organization to obtain existing data, and use these results to evaluate and adapt the dashboard accordingly so that it is aligned with collective priorities. Departmental baseline and level of progress may depend on whether the department is working from historical strengths and momentum or from limited support or even possible underlying resistance. This status should be elucidated when appraising progress.

Addressing both the current state of EDI and future goals aligned with the academic mission is integral for departments to move forward toward a more inclusive and just system. Considering patient perspective is important as well as which measures will be most beneficial to evaluating the organization. ADFM invites departments to share their measurement goals, methods, metrics, questions, and findings so that departments may not only learn from one another through shared best practices, but also evolve as a national cohort of medical professionals and organizations working to advance justice, equity, diversity, and inclusion across collective communities.