Efforts to Reduce Bias in Clerkship Evaluations: A CERA Study

Disparities and bias in evaluations and grading impact medical students’ careers. Several studies on clerkship grades show disparities related to gender and underrepresented minority status, leading to a concern that bias in evaluation affects final grades.^1-3 Grading inequities are further worsened by the added pressures that students who are underrepresented in medicine (URiM) face, such as confronting stereotype threat and supervisors’ implicit biases.⁴  To students, the grading process can feel biased and inconsistent, leading to an unfavorable learning environment.⁴

Several studies have explored ways to reduce bias in the evaluation process, including using grading committees,⁵ minimizing descriptive words that can lead to bias,^1,6,7 and using competency-based judgment of medical school performance.^8-11 However, a paucity of studies exist related to actual implementation of antibias evaluation strategies.

Our study examines the status of antibias initiatives within family medicine (FM) clerkships. We hypothesized that clerkships in medical schools that have current antiracism or diversity initiatives would have more strategies to reduce bias in evaluation and grading. Additionally, we hypothesized that being a female or URiM clerkship director (CD) would be positively associated with implementation strategies to reduce bias in evaluations.

Our survey on antibias initiatives was included in the Council of Academic Family Medicine Educational Research Alliance (CERA) study of FM CDs. The data was collected from June 7, 2022, through July 8, 2022. The methodology of the survey has been previously described.¹² The survey is distributed annually to CDs of qualifying medical schools that are accredited by the Liaison Committee on Medical Education or the Committee on Accreditation of Canadian Medical Schools within the United States (n=148) and Canada (n=18), respectively. Our antibias survey included 10-questions about the use of strategies to reduce bias in medical student evaluations, antiracism initiatives, perceptions on effectiveness of the antibias initiatives, and type and cadence of faculty development on antibias evaluations. CD responses about faculty participation in development sessions and receiving evaluation instructions are based on their best estimates. The project was approved by the American Academy of Family Physicians Institutional Review Board.

All analyses were performed using STATA 13.1 (StataCorp). We used univariate statistics to described CD and program characteristics. We used bivariate statistics to examine the relationships between X and Y. A P-level of <.05 defined statistical significance. We conducted bivariate analysis using ꭓ² tests to calculate the associations between being at a medical school that conducted an analysis of grades and each of the following: implementing strategies to reduce bias in evaluations, effectiveness of an evaluation system designed to reduce bias, prioritizing strategies to reduce bias in evaluations, using standard behavior-focused anchors, annual faculty development for preceptors on how to write less biased descriptions, and attendance to faculty development of preceptors (community and faculty). We also examined the bivariate relationships between use of strategies to reduce bias with each gender, years graduated from residency, race, years as director, and protected time.

The response rate for the survey was 59.12% (94/159). All respondents completed our survey module. Respondents were primarily female (59.6%) and White (76.6%). Approximately 11% identified as URiM and, on average, had been a CD for 8 years (Table 1). The majority (70%, strongly agree or agree) had implemented strategies to reduce bias in evaluations and thought their bias-reducing strategies were at least somewhat effective (63%, somewhat or very effective; Table 2). Forty-two percent of community preceptors reported never attending faculty development on reducing bias in evaluations compared to 17% of the academic preceptors (Table 3). Table 2 shows that the majority of medical schools 89/91 (95%) had a current social justice, diversity, or antiracism initiative. One-third of the medical schools analyzed their bias in clerkship grading efforts while 15% were in the process of doing so.

In our bivariate analyses, CDs were more likely to implement strategies to reduce bias in evaluations if their medical school performed an analysis of grades compared to those that did not (90% vs 62%, P=.005). Conducting a grade analysis was associated with an evaluation system designed to reduce bias (81% vs 56 %, P=.02), prioritizing strategies to reduce bias in evaluations (100% vs 74 %, P=.002), using standard behavior-focused anchors (81% vs 60%, P=.045), and annual faculty development for preceptors on how to use less biased descriptors in their written evaluations (77% vs 34%, P=.045).

Academic preceptors (71% vs 40%, P<.005) and community preceptors (35% vs 11% (P<.005) were more likely to attend faculty development on reducing bias in evaluations at least once a year if the medical school had performed an analysis of grades.

We found no statistically significant association between gender of CD nor race (White vs URiM) and use of various strategies to reduce bias in evaluations. CDs who agreed that they implemented strategies to reduce bias in evaluations had more recently graduated from residency (7.14 vs 10.44 years, P=.03) and had a lower average number of years in their current director role (6.79 vs 10.24 years, P=.02), as well as less protected time (30.2 vs 40.7, P=.04).

In this study, we identified a positive correlation between medical school initiatives related to grade analysis and efforts to reduce bias in evaluations in FM clerkships. Reducing bias in evaluations was a priority for most FM CDs. Two-thirds of CDs believed their efforts were effective at reducing bias in evaluation. Faculty development on reducing bias in evaluation was less available to community preceptors compared to academic preceptors.

Our hypothesis that being a female or URiM CD would be associated with implementing more strategies to reduce bias was not supported by the data. We also found that CDs who were in the role for fewer years and had less protected time implemented strategies to reduce bias more often than those who had more protected time and had been a CD for a longer time.

The Alliance for Academic Internal Medicine recommends a comprehensive approach to reducing bias in medical education.¹⁰ To our knowledge, an evidence-based faculty development program aimed at reducing bias in evaluation and grading does not currently exist. Such programs would be invaluable for reducing bias in evaluations for both academic and community preceptors.¹³

Our study’s limitations included limited diversity in respondent demographics,^14,15 a modest response rate, recall bias of CDs with answers based on estimations, and inability to control for socially desirable answers. We did not ask about specific biases CDs were addressing, and some terms were not clearly defined for the respondents, such as “standard, behavior-focused anchors” and “antiracism and diversity initiatives.” Also, given that the evaluations were presumed to be FM-specific, the results are not generalizable across all types of clerkships, and variation is likely across institutions. To the best of our knowledge, our study is among the first that explores FM CDs’ initiatives to reduce bias in medical school evaluations. The results show that efforts can be impactful when medical schools have specific initiatives to analyze bias in clerkship grades and when academic faculty can more easily access training about bias reduction in evaluations. As strategies to address bias in evaluation are implemented, assessing for their intended effect is important. Future studies should evaluate long-term outcomes.