Introduction: The United States Medical Licensing Examination (USMLE) Step 1 will transition to a pass-fail format in 2022. This is likely to result in an increased focus on Step 2 Clinical Knowledge (CK) scores. Thus, academic advisors must provide evidence-based guidance for preparing students. While prior research has examined the utility of academic indicators to predict student performance on the USMLE exams, no significant scholarly effort has described or evaluated students' study approaches. The research study's goal was to understand what strategies and resources students utilized when preparing for the Step 2 CK exam and investigate the relationship(s) between these approaches and performance.
Methods: Students at a single US medical school were surveyed about their Step 2 CK preparation. We analyzed self-reported exam preparation strategies and the use of specific resources to determine their relationship with Step 2 CK score.
Results: Student performance on Step 2 CK was correlated with performance on previous exams, including school-specific examinations, National Board of Medical Examiners clerkship shelf exams, and Step 1. Two study strategies were positively correlated with Step 2 CK score in preliminary analyses: completing more working practice questions, and the proportion of a question bank completed. In hierarchical regression, only completing more working questions remained predictive, after controlling for demographic variables and Step 1 performance.
Conclusions: Faculty and staff can optimize students' Step 2 CK performance by encouraging them to work through case-based, clinically-focused questions. Further study is needed to describe optimal preparation strategies better.
Academic advisors counsel medical students on study approaches and resources that promote student success on the United States Medical Licensing Examination (USMLE) Step 2 Clinical Knowledge (CK) exam. Over time, the pressure on students to perform optimally on this exam has increased due to the importance of exam scores in the residency selection processes.1,2 With the 2020 announcement that score reporting for Step 1 will move to pass/fail, students will likely experience increased pressure to achieve a competitive score on their Step 2 CK exam.3-5 Moreover, because of the changes planned for scoring Step 1, students may not have learned and implemented effective exam preparation strategies previously. Thus, additional Step 2 CK preparation may be necessary.
While prior research has examined the utility of various academic indicators to predict student performance on the USMLE exams,6,7 no significant scholarly effort has been conducted to investigate students' Step 2 CK study approaches. This study aimed to describe what approaches and resources medical students utilized when preparing for the Step 2 CK exam and investigate the relationship(s) between various resources and strategies on Step 2 CK performance.
The humanist framework of self-directed learning theory guided this research.8 Students engaged in self-directed learning to diagnose their learning needs, formulate goals, identify resources, implement learning strategies, and evaluate outcomes.9 Two hundred-five students in the class of 2020 at a single public, allopathic US medical college were invited to participate in a survey focused on preparation for the Step 2 CK exam. Fifty-one students responded (25% response rate). The average USMLE Step scores for respondents was 232 on Step 1 and 248 on Step 2 CK. Survey invitations were sent on a rolling basis to students approximately 2 weeks after completing their exam. The survey included 102 multiple-choice and short-answer items. Responses were linked to demographic and academic performance information using the college's honest broker process to deidentify data before analysis. The study was exempted from review by the Michigan State University Institutional Review Board.
We analyzed Step 2 CK preparation strategies and the use of specific resources to determine their relationship with Step 2 CK scores. We used Pearson correlations for linear variables, and independent samples t tests for dichotomous variables. We analyzed results descriptively, using counts and proportions. In some cases, we combined variables. We initially intended to analyze the relationships between study approaches and passing or failing the Step 2 CK and CS exams, but the relatively small sample size, and the small number of failures, prohibited these analyses. We performed hierarchical multiple regression analysis to evaluate whether any specific study strategies and resources continued to be positively associated with Step 2 CK score, after controlling for Step 1 score, MCAT score percentile, and entry pathway into medical school.
Students used a heterogenous mix of study strategies and resources (Tables 2, 3, and 4). In basic analyses, both the earliest school-administered Comprehensive Clinical Skills Exam (CCSE) scores and the most recent CCSE scores were positively correlated with Step 2 CK scores (Table 5). Each National Board of Medical Examiners' clerkship exam score was also predictive of the Step 2 CK score. Additionally, practice test questions (using the average percentage of correct questions) and the predicted Step 2 CK score from the most recent practice test were correlated. Among preparation strategies and resources, only completing more practice questions and the proportion of a question bank completed were positively correlated with Step 2 CK score. Prioritizing specific content areas for study, using coaching, and using some specific resources (First Aid, Master the Boards, OnlineMedEd, and Step 2 Secrets) were negatively predictive of score (Table 4). None of the other study resources and strategies were associated with Step 2 CK score. The number of different study resources used was also not predictive.
Demographic variables (gender, race/ethnicity, underrepresented in medicine status, rural origin, and age) were not predictive of CK score (Table 1). Medical school entry pathway and MCAT score percentile were associated with Step 2 CK score (correlation coefficients 0.290 and 0.374, respectively; P=.020 and .007, respectively). In hierarchical regression, only completing more questions remained predictive of Step 2 CK score, after controlling for entry pathway, MCAT percentile, and Step 1. However, the impact was small.
Although students used a heterogenous mix of study strategies and resources, few were positively correlated with examination performance. Notably, students performed better if they focused on working through case-based, clinically-focused questions. By incorporating the regular review of case-based questions, students engaged in a form of practice testing, which has been found to enhance learning and long-term retention.10 Studying based on focused content areas did not lead to improved performance. This may be because students who study broad content areas are missing core foundational knowledge due to ineffective self-monitoring and the inability to identify and implement effective learning strategies.9
Results confirmed that student performance on clerkship exams was associated with performance on the Step 2 CK exam. Thus, students can reliably use these assessments as indicators of expected performance on Step 2 CK. This affirms results of a previous study with similar findings.7 Researchers were unable to identify significant positive associations using specific resources (eg, texts, videos, study aids, question banks, practice tests) and performance on the USMLE Step 2 CK exam.
The results of this analysis are limited by the small sample size and evaluation of a single class at a single allopathic US institution. Further research into students' Step 2 CK study habits is needed to identify and develop evidence-based best practices. Based on results of this study, students should be encouraged to work through case-based, clinically-focused questions as they prepare. Additional research focused on students' ability to self-direct learning could also inform optimal preparation approaches to Step exams and appropriate educational interventions.
The authors thank Ann Taft, statistical analyst for the Office of Medical Education Research and Development, College of Human Medicine, Michigan State University, for her advice and service as an honest broker.
Presentations: This study has been presented as a poster at the Society for Teachers of Family Medicine Conference on Medical Student Education (virtual), on February 1–3, 2021.
- National Resident Matching Program. Data Release and Research Committee: Results of the 2020 NRMP Program Director Survey. Washington, DC: NRMP; 2020. Accessed August 11, 2021. https://mk0nrmp3oyqui6wqfm.kinstacdn.com/wp-content/uploads/2020/08/2020-PD-Survey.pdf
- National Resident Matching Program. Charting Outcomes in the Match: U.S. Allopathic Seniors, 2018. Washington, DC: NRMP; 2018. Accessed August 11, 2021. https://mk0nrmp3oyqui6wqfm.kinstacdn.com/wp-content/uploads/2019/10/Charting-Outcomes-in-the-Match-2018_Seniors-1.pdf
- Humphrey HJ, Woodruff JN. The Pass/Fail Decision for USMLE Step 1-Next Steps. JAMA. 2020;323(20):2022-2023. doi:10.1001/jama.2020.3938
- McDade W, Vela MB, Sánchez JP. Anticipating the impact of the USMLE Step 1 pass/fail scoring decision on underrepresented-in-medicine students. Acad Med. 2020;95(9):1318-1321. doi:10.1097/ACM.0000000000003490
- Nolen L, Goshua A, Farber, ON, Tiako MJ. Cheers and jeers as med school's Step 1 test becomes pass/fail. STAT. February 14, 2020. Accessed August 11, 2021. https://www.statnews.com/2020/02/14/cheers-and-jeers-as-med-schools-step-1-test-becomes-pass-fail/
- Guiot HM, Franqui-Rivera H. Predicting performance on the United States Medical Licensing Examination Step 1 and Step 2 Clinical Knowledge using results from previous examinations. Adv Med Educ Pract. 2018;9:943-949. doi:10.2147/AMEP.S180786
- Monteiro KAG, George P, Dollase R, Dumenco L. Predicting United States Medical Licensure Examination Step 2 clinical knowledge scores from previous academic indicators. Adv Med Educ Pract. 2017;8:385-391. doi:10.2147/AMEP.S138557
- Torre DM, Daley BJ, Sebastian JL, Elnicki DM. Overview of current learning theories for medical educators. Am J Med. 2006;119(10):903-907. doi:10.1016/j.amjmed.2006.06.037
- Loeng S. Self-directed learning: a core concept in adult education. Educ Res Int. 2020;2020:1-12. doi:10.1155/2020/3816132
- Dunlosky J, Rawson KA, Marsh EJ, Nathan MJ, Willingham DT. Improving students’ learning with effective learning techniques: promising directions from cognitive and educational psychology. Psychol Sci Public Interest. 2013;14(1):4-58. doi:10.1177/1529100612453266