Clerkship Grading, USMLE Step 1, and Student Distinction: A CERA Study

In January 2022, US Medical Licensing Examination (USMLE) Step 1 numeric scoring changed to pass/fail. Until then, that numeric score had been the top factor in applicants receiving an offer to interview at prospective residency programs.¹ The 2024 US residency match was the first to engage applicants who did not have a Step 1 numeric score. Little was known about what factors—noteworthy characteristics of students, referred to here as distinction—residency programs might consider in selecting applicants to interview in the absence of a numeric Step 1 score.^2,3 Several research groups sought to predict which factors might be considered for this decision. Among them were the USMLE Step two clinical knowledge exam score, letters of recommendation, research, leadership, school reputation, and clinical clerkship grades.^4-8

As clerkship faculty at a large medical school that recently changed its clerkship grading methodology from tiered (eg, high pass, pass, fail) to pass/fail, we were particularly interested in the predictions about how clinical clerkship grades would be considered in residency applications. We investigated the impact of two conditions on a family medicine clerkship director’s (FMCD’s) perception of a student’s ability to distinguish themselves for residency applications: first, the impact of the new Step 1 pass/fail paradigm; and second, the impact of a school’s choice of preclerkship and clerkship grading methodology—tiered or pass/fail. We surveyed FMCDs in the United States and Canada. We investigated four associative hypotheses. We hypothesized that FMCDs would be divided in their opinions about whether students could still adequately distinguish themselves for residency applications since the Step 1 pass/fail transition and the impact (harm, neutral, or benefit) of this transition. First, we postulated that those FMCDs who felt students could no longer distinguish themselves adequately would view the Step 1 transition as harmful. Second, we hypothesized that FMCDs at schools with pass/fail methodologies would perceive the Step 1 pass/fail change to be more harmful to students than FMCDs at schools with tiered grading. Similarly, we postulated that FMCDs who perceived increased student stress would prefer tiered-grading methodologies; supposing that FMCD might perceive tiered grading as a solution to the issue at hand. We hypothesized that FMCDs at schools with pass-fail methodologies throughout the entire (clinical and preclinical curriculum) would report less satisfaction with students’ ability to distinguish themselves. Finally, in a post-hoc analysis, we investigated whether pass/fail methodology in the preclerkship phase correlated with greater FMCD-perceived student stress. We postulated that the students who did not have an established record of academic success in preclerkship courses might feel more stress in the clerkship curriculum without a numeric Step 1 score. As we were unable to directly measure student perceptions, we used FMCD perceptions of student stress as a surrogate measure.

We included 10 questions in the omnibus 2024 Council of Academic Family Medicine (CAFM) Educational Research Alliance (CERA) survey of clerkship directors, which was conducted using previously described methodology.⁹ Briefly, we were accepted by a peer-reviewed application process to propose survey questions, which were then reviewed by the survey committee and revised based on their feedback. The sample frame was initially sent to 179 FMCDs, updated by the survey committee from the prior 2023 distribution list. Surveys were conducted using SurveyMonkey (SurveyMonkey, Inc) between June 4, 2024, and July 12, 2024, and six reminders were sent to partial respondents and nonrespondents. After troubleshooting the distribution for invalid email addresses, removing one participant who previously opted out of SurveyMonkey surveys, and further updating the distribution list due to changes in clerkship directorships, a final sampling frame included 173 survey recipients.

We conducted statistical analyses in R version 4.4.1 (R Foundation). Descriptive statistics were calculated using standard methods. Percentages for descriptive statistics were calculated using all nonmissing responses. Associations between variables were tested using the Kruskal-Wallis test. We conducted the Wilcox signed-rank test to assess whether opinions were significantly different from neutral as to whether the change of USMLE Step 1 from pass/fail was beneficial or harmful. The survey was reviewed and approved by the American Academy of Family Physicians Institutional Review Board.

The CERA survey received 91 valid responses, and the response rate was 52.6% (91/173) overall. Of these, eight left our subset of questions pertaining to changes in grading structure blank and were removed from the analysis, leaving 83 (48% response rate) respondents included in our analysis. Demographics of respondents are summarized in Table 1. Notice that 59% of respondents were women and were most commonly White (76%), with 9% of FMCDs_self-identifying as underrepresented in medicine. Most responding FMCDs directed clerkships at public institutions (72%). Clerkships were drawn from all four United States census regions (Northeast, Midwest, South, West) and Canada.

The distribution of responses to each of the 10 survey questions is shown in Tables 2 and 3.

Most schools represented by respondents in our survey used a pass-fail grading methodology in the preclerkship phase (59%, n = 49), and most used a tiered grading system in the clerkship phase (78%, n = 65). A large range was noted in the FMCDs’ report of how many students at their institutions receive the top tier score. Among schools with tiered clerkship grading system, 28% reported that more than 40% of the class earn the highest grade, whereas at some schools, more than 90% earn the highest grade.

Since the 2022 Step 1 pass/fail transition, the majority of respondents (58%, n= 48) had not changed or were not considering a change to their clerkship grading methodology. However, 42% indicated either that a change had been implemented (20%, n = 17) or was under consideration (22%, n = 18). Among clerkships that had changed or were considering change, change toward a less-tiered (eg, pass/fail) system was more likely than change to a more-tiered pass/fail clerkship grading, and those who preferred tiered clerkship grading were similar with a preference for pass/fail clerkship grading (57% vs 43%, respectively).

FMCDs’ perception of the impact of the Step 1 pass/fail transition on students’ abilities to distinguish themselves was mixed: 48% felt the impact was neutral, 37% felt the impact was harmful, and 14% felt the impact was beneficial. The difference between FMCDs who perceived the transition as having caused harm as compared to having benefited student distinction efforts was statistically significant (Wilcox signed-rank test P=0.003). Specific descriptions of these harms or benefits were not further elucidated in our study. We found that 78% of FMCDs perceived increased student stress since 2022 about their ability to distinguish themselves.

Most FMCDs (60%, n = 50) were satisfied, however, that students could still appropriately distinguish themselves for residency applications.

We interrogated four prespecified hypotheses (see Introduction) that FMCDs at schools with tiered or pass/fail grading systems in the clerkship year would have different perceptions of students’ distinction and the USMLE Step 1 pass/fail transition. These hypotheses were not supported by the data (Kruskal-Wallis P>0.05). We found no association between FMCDs who were dissatisfied with students’ ability to distinguish themselves and a perception that the Step 1 transition was harmful. We found no association between the presence of a pass/fail clerkship grading methodology and the following:

• FMCD perception that the Step 1 pass/fail transition was harmful to student distinction; and

• FMCD satisfaction that students could still distinguish themselves.

We found no association between FMCDs who perceived increased students’ stress about distinction and a preference for tiered grading.

Nearly one-third of FMCDs in this study felt that the Step 1 pass/fail transition in 2022 was harmful to student distinction, and more than three-quarters perceived increased student stress about distinguishing themselves. Nevertheless, most FMCDs were satisfied that students still could appropriately distinguish themselves for residency applications. FMCDs’ satisfaction was not different whether they came from clerkships with pass/fail or tiered grading. Thus, FMCDs do not view tiered clerkship grading as a clear solution to student distinction post-2022. They do not view pass/fail grading as harmful to student distinction in this context either. Nevertheless, 57% of FMCDs in our survey preferred pass/fail grading for clerkships. This finding may reflect the culture of family medicine, which comparatively may use more holistic assessment and less emphasis on grades as compared to other specialties.

Indeed, both pass/fail and tiered grading methodologies have strengths and weaknesses. We found that most clerkships represented in our study (78%) used tiered grading methodologies in the clerkship phase. Among those, we found great variability in what percentage of students earn the top tier grade among the family medicine clerkships represented in our study. These inconsistencies echo prior reports from the Association of American Medical Colleges (AAMC) and others, which questioned the reliability and meaning of tiered clerkship grading across and even within institutions.^10-16 These concerns, of course, are not wholly mitigated by adopting pass/fail clerkship grading either. Pass/fail methodologies still contain the risk of bias.¹⁷ Studies on the effects of pass/fail grading on motivation, learning, and achievement remain inconclusive.¹⁷

Despite inconsistencies in grading across institutions, results from the 2024 National Resident Matching Program–Program Director Survey (NRMP-PDS), which is the only NRMP-PDS since the Step 1 pass/fail transition, revealed that clerkship grades remained a top 10 factor used by programs both in offering interviews and in ranking applicants.¹⁸ Given that both tiered and pass/fail systems have inherent strengths and weaknesses, our findings underscore the need for more standardized, competency-based approaches to student assessment across institutions and the educational continuum.

Before the 2022 Step 1 pass/fail transition, voices in medical education identified the moment as an opportunity to reimagine and strengthen the meaningfulness of assessment in medical education. We heard calls to redesign the transition to residency.^2,3 Those opportunities remain to strengthen the link between undergraduate and graduate medical education programs. Our findings provide support for such work. We found that 43% of respondents in our study had either already changed their clerkship grading methodology since 2022 or were considering a change. These changes in assessment and communication between undergraduate and graduate medical education should be coordinated. Clerkship grading methodology should be standardized or more clarity provided on the meaning of these distinctions as related to competencies. The AAMC Medical Student Performance Evaluation guidance provides a model for standardization. Undergraduate medical education must devise methods for more accurately and meaningfully communicating student competence and distinction to residency programs.

Distinction opportunities indeed exist outside a Step 1 numeric score and a tiered clerkship grade, but these are as yet insufficiently described or measured in the new Step 1 paradigm. A few small studies, mostly specialty-specific, have investigated what distinguishing factors programs considered during the 2024 and 2025 residency application seasons.^19,20 Early evidence supports that the Step two clinical knowledge score may have risen in importance but not to the level Step 1 numeric scores previously held. Using these data in combination with the NRMP-PDS, we have early, but limited insight into the distinguishing characteristics sought after by residency programs. Other reported distinguishing factors included letters of recommendation, awards (eg, American Osteopathic Association), research, medical school reputation, and, of course, clerkship grades.^19,20 As long as the phenomenon of student distinction remains unclear under the new paradigm, clerkship director perception of student stress may be expected to remain elevated. Studies investigating the direct measurement of student stress after the Step 1 pass/fail transition are also limited or do not extend to the residency application phase.^21,22 This area represents an opportunity for future research.

Limitations of this project included the response rate, a respectable 48%, which still limits the generalizability of the findings. The hypotheses assumed that FMCDs are curriculum leaders at their schools, which, if erroneous, would jeopardize their reliability as reporters of their institution’s plans for grading schema. The project also assumed that FMCDs are reliable reporters of their students’ experiences and perceptions. If they are not, they may not be reliable reporters of student stress. Importantly, this survey was sent to family medicine respondents. The perceptions of faculty leaders in medical schools from other disciplines possibly would be different. Our prespecified hypotheses were not borne out in the statistical data. However, the findings of this study provide insight into the next steps that should be undertaken to study this complex time in medical education. Further investigation into student perspectives, interview, and match data, where available, and perspectives of leaders of other disciplines would expand these conversations in important ways.

In the wake of the USMLE Step 1 transition to pass/fail scoring, our study reveals that FMCDs perceive increased student stress but remain largely confident in students’ ability to distinguish themselves for residency applications. The lack of strong associations between grading methodology and perceptions of distinction or stress suggests that neither tiered nor pass/fail systems offer a definitive solution. Instead, these findings highlight the need for more standardized, competency-based approaches to assessment and clearer communication of student achievement across institutions. As medical education continues to evolve, coordinated efforts to define and measure student distinction—beyond traditional metrics—will be essential to support learners, ensure fairness in the residency selection process, and foster successful transitions to residency.