The Methodological Challenges With Assessing Primary Care Pathways in US Medical Schools

“Using Propensity-Score Matched Cohorts to Evaluate Career Outcomes for Medical Students Completing the Underserved Pathway” by Shimkin et al¹ demonstrates a rigorous approach for assessing the role of a longitudinal educational pathway on career outcomes. Authors found that medical student participation in the pathway was significantly associated with an increased rate of primary care practice in rural settings. This article brings to the forefront two key issues for educators seeking to improve primary care interest: the role of longitudinal pathways and modernizing educational research methods for evaluating their efficacy.

A popular approach for undergraduate medical schools to improve this interest is implementation of a primary care pathway. Usually enrolling a subset of self-selected medical students, these tracks incorporate longitudinal experiences that expose students to the challenges and insights of primary care alongside the traditional curriculum.² Across multiple reviews, these pathways have been reported to increase medical student interest in primary care.^2-4 However, because of student self-selection, we might instead say that these pathways improve or maintain interest among a biased sample of students already drawn to the pathways’ missions; we have little data to suggest that these pathways increase primary care career choice among medical students generally. This is a subtle but important distinction. Though cultivating interest among students is necessary and valuable, this is unlikely to address the growing need for primary care physicians in the United States. We need to think about avenues to both reach the broader medical student population apart from students already seeking to engage with primary care and critically evaluate our current interventions.²

Causal inference methodologies are key to both accurately estimating the effects of nonrandomized educational interventions like primary care pathway programs and addressing issues with concomitant self-selection and generalization. Causal inference estimates the effect of a nonrandomized intervention on an outcome, trying to emulate the gold-standard randomized controlled trial in situations where this is impossible or impractical.⁵ In Shimkin et al,¹ authors employ a popular causal inference method called propensity-score matching. Using the full student population at their medical school, authors estimated the chances (ie, propensity score) that a student would enroll in the pathway based on relevant factors including gender, race/ethnicity, and primary care interest. From there, they created pairs (ie, matches) of students that were similarly likely to be in the pathway or not, trying to simulate an experiment with balance across these two groups. Thus, their results can be generalized to the broader student population rather than just the impact on students within the pathway.

We wanted to highlight the utility of causal inference methodology as a necessity for educational research. This is one of many possible ways that we can incorporate causal inference into how we conduct medical education research. Following the example of Shimkin and colleagues, improving how we causally assess educational interventions will help us better identify effective methods for increasing interest in primary care that can be used for all medical students, not only those who are already aligned with the missions of primary care.