An Evaluation of STFM’s Academic Family Medicine Antiracism Learning Collaborative

As the US population and family medicine workforce become increasingly racially and ethnically diverse, ^{1, 2} academic family medicine has an imperative to implement initiatives that advance racial equity and reduce the prevalence of bias and racism within their institutions. ^{3, 4} Historically, medical educational institutions often have perpetuated misinformation regarding race and failed to create safe educational environments for trainees, thus contributing to decreased intention of students to work in underserved or minority communities after graduation. ^{5, 6} In particular, trainees and faculty, especially those racially underrepresented in medicine (URM), have been affected by biased policies and practices within their academic institutions that led to or enabled discrimination. ^7-11 Ultimately, these unjust practices and biases have severely negatively impacted URM students’ medical school admission and academic performance, ⁹ as well as URM faculty and trainees’ grant-funding opportunities. ¹²

Addressing systemic racism in medicine can resolve or mitigate these negative experiences and have profound positive impacts on both patients and clinicians. For example, research has shown that having a culturally competent ^{13, 14} and racially and ethnically diverse workforce can lead to better patient and clinician communication, better medical adherence, and higher patient satisfaction. ^{3, 4} For URM students and trainees, mentorship and allyship are mitigating factors that can increase career and personal satisfaction, provide opportunities for career advancement, and encourage productivity in research. ^{15, 16} Arguably, the most important factor for enabling sustainable change is buy-in from leadership to support diversity and equity related efforts in academic medical institutions; leaders denote prioritization and set the tone for organizational culture. ^{17, 18} Thus, medical institutions are central to addressing systemic racism.

For these reasons, the Society of Teachers of Family Medicine (STFM) appointed an antiracism task force for the purpose of advancing racial equity and reducing the prevalence of racism in academic family medicine. The antiracism task force created an action plan to help operationalize and address concepts such as antiracism, racist structures, and allyship. ¹⁹ Existing constructs, ²⁰ combined with STFM’s antiracism and health equity resources, ²¹ contributed to one of the task force’s creations: the Antiracism Learning Collaborative (ALC). This 2-year collaborative focused on supporting longitudinal projects that helped reduce racism within academic medical institutions. The task force solicited applications through a variety of electronic communications to STFM members and other academic family medicine organizations nationally. Applications consisted of preparticipation surveys, a project work plan proposal, and at least one letter of support from the department chair, dean, or other leader within the participants’ institution. The task force members then reviewed, ranked, and scored 57 applications; in the end, they selected 20 dyads, each with at least one dyad member being URM. The task force members also recruited mentors to provide guidance and expertise to participants; they received 31 applications and settled on 10 mentors, each of which they then paired with two dyads. ALC activities took place from January 2022 through September 2023.

The Robert Graham Center for Policy Studies in Family Medicine and Primary Care (RGC), on behalf of the American Academy of Family Physicians (AAFP), evaluated ALC activities to determine whether ALC achieved the five goals described here. RGC was uniquely positioned to evaluate the ALC project because of its extensive work in supporting the primary care pathway for graduate medical education, as well as its work in promoting health equity. Ensuring equitable opportunities and removing barriers for learners are critical to growing the primary care workforce, and thus, central to RGC’s mission.

ALC provided content that emphasized how to recognize racist structures and behaviors embedded in institutions and what actions to take to enable a shift toward antiracist constructs for individuals and organizations. Specifically, topical content included training upstanders, promoting precise use of language, explaining dominant cultural norms, outlining biases and power in institutions, finding allies, maximizing impact, and managing expectations. The ALC task force shared content via presentations, resources, and tools during virtual and in-person meetings and on the STFM Connect platform. Additionally, participants completed an online course on leading change (www.stfm.org/leadingchange). Discussions focused on experiences and learnings of task force members, presenters, mentors, and participants; these were shared throughout the ALC program.

ALC promoted allyship in numerous ways. First, ALC required all dyads to include one URM and one ally. Teams also were required to develop SMART (specific, measurable, achievable, relevant, and time-bound) objectives and describe how these objectives promoted allyship. Individual project objectives varied from generalized descriptions to specific aspects of allyship promotion and enhancement. Lastly, presentations and discussions focused on or referred to the importance of allyship in partnerships and collaborations.

To help participants disseminate their project findings and affect sustainable, effective change at their institutions, ALC provided a loose framework based on five key components.

• Know institutional history, understand previous work in local environments, and build on existing initiatives.

• Cultivate, sustain, and enhance relationships.

• Utilize external institutions to foster local community engagement.

• Encourage and provide participants with different opportunities to disseminate their project findings via channels such as presentations, teaching seminars, and publications.

• Encourage participants to be pragmatic in their approach, align their project objectives with their institutions’ missions, and incorporate sustainability plans.

Through the cultivation and dissemination of information and resources, ALC encouraged participants to develop sustainable strategies that could be applied at other institutions. ALC stressed the importance of engagement with their stakeholders, whether they were community partners, leadership, or those affected by interventions (ie, students, residents, and/or faculty). ALC also noted the importance of being selective and intentional about who to work with and the need to secure time investments from leadership stakeholders up front.

ALC included five overarching components that provided the foundation for the project: infrastructure, dyads, project plans, mentorship, and meetings. ALC infrastructure was flexible so that it could respond to unforeseen needs. A dedicated STFM staffer served in a management and administrative capacity, which was critical to program implementation. As for mentorship, the task force tried to pair dyads with mentors based on availability, geography, and skill set. ALC held both virtually and in-person meetings.

The project evaluation used multiple data sources: project plans; four survey sets (baseline, 7-month, 14-month, and 1-month post program) that contained both quantitative and qualitative assessment questions; four anecdotal meeting notes (in-person and virtual); five mentor-mentee meeting form sets; and 17 final reports and presentations from the dyads. RGC performed quantitative and qualitative analyses, and an STFM staff member, an outside consultant specialized in antiracism, and an ALC task force member reviewed results. For the quantitative analysis, we included only those individuals who participated in all four survey sets. The study received AAFP Institutional Review Board approval. Table 1 provides a breakdown of how the measures under each goal were analyzed qualitatively and quantitatively.

The qualitative data sources consisted of all the aforementioned data sources that included open-ended responses. A member from the research team summarized the data for each dyad and then extrapolated conclusions based on the findings. These findings were shared with the larger research team to ensure alignment with the quantitative data. The team also discussed the significance and meaning of the data. Emergent themes related to each of the five evaluation goals were extracted from the findings and discussions with the team. Any discrepancies were resolved through consensus with the research team at-large.

The quantitative data source consisted solely of the survey sets, excluding the open-ended responses. Summary statistics on personal demographics and institutional characteristics were derived from the baseline survey. Outcome variables included confidence, six antiracism perceptions, institutional rating, progress, perception of mentor support, and barriers, attributes, and support. Institutional rating, progress, and mentor perception outcome measures were analyzed via a stacked bar chart and two-way contingency table. The mentor perception outcome measures were analyzed via a group mean rating chart. The remaining outcome measure—barriers, attributes, and support—was analyzed via stacked bar charts.

At the conclusion of ALC, 20 dyad spots were filled and 17 of 20 dyads (total of 34 participants) completed the program. For the purposes of the quantitative analysis, only those who participated in all surveys were included, which resulted in a total of 31 participants analyzed (Table 2). Most participants were from the West (35.48%), had an MD (77.42%), were Black or African American (41.94%), and/or were from a community-based, university residency program (35.48%).

At each survey stage, participants were asked about confidence, that is, “How confident are you in your skills to lead meaningful change within your institution?” The mean confidence rating declined at the 7-month survey, then increased for the remaining surveys. By the conclusion of ALC, 38% of participants noted increased confidence in their skills to lead meaningful change in their institution, and 100% of participants believed they incorporated strategies that led to positive outcomes when implementing their projects. See Table 3 for exemplar quotes associated with each goal.

Participants also were asked to “Review the Continuum on Becoming an Anti-Racist Multicultural Organization. ²² Based on the descriptions, where would you rate your institution?” Responses fluctuated over time, as seen in Figure 1. At the 14-month survey, the passive change category (club institution) increased substantially, and the structural change category (transforming institution) decreased, meaning that more participants rated their institutions lower on the continuum, that is, monocultural, compared to the baseline and 7-month surveys. In the post survey, participants rated more institutions in the structural change category (transforming institution) than the 14-month survey.

Two projects listed allyship as the central focus for antiracism promotion and institutional change; a handful of others focused on informing and guiding education for allies or specifically referred to getting buy-in from allies for successful project implementation; and the remaining ones recognized the importance of allyship for project implementation. By the conclusion of ALC, 94% of participants said that ALC met its goal to promote allyship.

Participants indicated that understanding institutional history was beneficial in gaining buy-in from leadership. Participants who developed mutually beneficial relationships and discussed managing expectations in their institution reported more buy-in and facilitated change. Additionally, participants saw value in exploring potential connections outside of their department because others may have been working on similar initiatives. Participants who engaged local community partners in their projects thought that initiating and/or extending relationships to drive action were well worth their efforts. Participants indicated that following a practical and critically thought-out plan made using their data to tell a compelling story easier, as well as to garner institutional support.

The most reoccurring and effective strategy the participants acclaimed centered on engagement with stakeholders. Buy-in from institutional leadership was particularly important to participants, as noted in the survey question, “Which of the following institutional supports can you credit with any success you’ve had implementing your project?” (Figure 2). To secure chances of buy-in, participants claimed demonstrating value to their institutions as a critical strategy to employ.

Key indicators that led to effective projects included incorporating existing resources and tools and integrating sustainability into project planning. Utilizing existing resources and tools, whether from ALC or elsewhere, saved time and effort. Similarly, incorporating antiracist language and concepts, such as changing language from determinants to drivers and integrating structural racism into previous concepts, into existing curricula were appealing to leadership due to the streamlined approach and structure, which increased the chances of a sustainable project. Obtaining funding to support current and future efforts also laid the groundwork for future success.

Several topics emerged that contributed to effective dyad relationships: logistics, soft skills, and ability to pivot. Prioritizing and organizing their projects helped dyads meet deadlines and maintain a sense of accountability. To overcome barriers and delays to project plans, dyads had to use soft skills, such as creative thinking and patience. Lastly, their ability to pivot in difficult situations was critical in moving their projects forward.

Regarding working in a dyad, participants overwhelmingly praised the structure because it allowed them to pair with another at a different career stage and lived experience. However, some dyads expressed challenges with consistently meeting with their partner due to external commitments. Most found meetings with their mentors to be beneficial, because the meetings helped clarify next steps and provided an opportunity for mentees to share their needs, frustrations, and experiences. Overall, a direct, positive, association emerged between the number of times participants met with their mentors and program progression. Overall, attendees found the in-person meetings to be more beneficial and effective for engaging with other participants. However, virtual meetings were still helpful in that they provided an opportunity for leadership to check in with participants and for participants to touch base with their fellow participants. When participants expressed a desire for more meetings to sustain ALC connectivity and discuss successes and challenges, ALC responded by implementing virtual office hours and unstructured time to talk; however participation in both was extremely limited, and these were discontinued.

Our evaluation focused on whether ALC met the five goals it set out to achieve; overall, ALC met each goal, however the degree of effectiveness of each achievement is open to interpretation because the data were subjective. Generally, participants learned antiracism concepts and how to counter racist structures and behaviors, such as identifying dominant cultural norms or tactics used to suppress others within institutions, implementing actionable approaches (ie, being an agent of change), and using antiracist language. Strengths of the learning collaborative were the tools and resources offered to the dyads for individual project implementation, such as the discussion platform and the leading change course.

Over the duration of ALC, participants encountered both expected and unexpected challenges; and depending on the strength of their influence over a given challenge, they either were able to adapt and complete their projects or were forced to pivot their project plans entirely. For example, when participants encountered “anti-antiracism/anti-diversity, equity and inclusion (DEI)” legislative mandates enacted in some states, as well as various levels of pushback from their institutions, dyads had to completely alter their projects to comply with state law and/or discuss the intentions of their projects with institutional stakeholders, which may have inspired future change in those institutions. Participants also faced challenges regarding the lack of protected time to implement their projects, even though prior to being selected for participation, applicants were required to obtain a letter of support from their department chair, dean, or other leader indicating how they and the department, program, or institution would support the dyad member and their change efforts.

Several program and evaluation limitations existed. First, a small number of institutions and dyads participated in the project, yielding limited data. Increasing the sample size would help with generalizability. Second, each institution had its own viewpoint on the importance of implementing antiracist projects, so participating dyads received varying levels of institutional support. For the evaluation team, demarcating specific cutoffs regarding what constituted goal achievement was difficult due to different starting and ending points. In future projects, the project leadership and evaluation team should define overall metrics for success and develop a rubric participants can use to gauge their individual project’s success. Third, the overall project evaluation was based primarily on self-reported data. These data are prone to biases, such as limited flexibility in responses, misinterpreted questions, and self-serving responses. Hence, developing an overall metrics of success and a defined rubric can help mitigate this issue.

To best serve participants and institutions in their journeys to reduce racist structures and behaviors, enhancements to future collaboratives could maximize the return on effort and investment. Considerations for future endeavors include potentially conducting a longitudinal study of institutional outcomes to see whether changes continue or project implementation is sustainable; encouraging participants to seek funding to support their projects (eg, through implementation and data analysis) and providing ideas for funding sources; providing more opportunities for in-person collaboration, communication, and structured networking between dyads; and providing suggestions to mentors on how to engage with their mentees in a more structured path that detailed what to do and how to move forward.

Our evaluation of STFM’s ALC pilot program concludes that the project was overall successful. ALC provided subject matter experts, tools, resources, mentors, and structural support (eg, funding for travel to meetings) that enabled participants to use their expanding knowledge and skills to take actionable steps to identify and mitigate racist structures and behaviors within their institutions. Participants were supportive and appreciative of the opportunity to be part of the collaborative, and lessons learned during the implementation and evaluation process can be used to shape ongoing ALC efforts to reduce racism in family medicine education.

Preliminary findings were presented in-person via a poster at the 2023 North American Primary Care Research Group Annual Meeting, October 30 to November 3, in San Francisco, California. Final results were presented at the 2024 Society of Teachers of Family Medicine Annual Spring Conference, May 4 to 8, in Washington, DC.

This study was partially financially supported by the Adtalem Global Education Foundation.