Using Propensity-Score Matched Cohorts to Evaluate Career Outcomes for Medical Students Completing the Underserved Pathway

Rural communities in the United States continue to face physician shortages ¹ despite decades of efforts to increase the supply of physicians working in these areas.² Rural communities face higher rates of chronic disease burden compared to their urban counterparts,³ and the lack of primary care doctors in rural regions has been linked to patients having lengthier travel times to receive health care. ⁴

For more than 15 years, the Underserved Pathway (UP) at the University of Washington School of Medicine (UWSOM) has provided mentorship, training, and skill-building to medical students interested in working in rural and underserved communities. Because UWSOM is a regional medical school, UP enrolls students from across six campuses in Washington, Wyoming, Alaska, Montana, and Idaho (WWAMI), a geographic area covering 27% of the US landmass.⁵ The WWAMI region is a hugely diverse area with many different underserved communities and populations. The UP works to extend and complement students’ curriculum by offering opportunities for customized training in specific underserved communities. ⁶

While each WWAMI state includes sizable areas designated as health professional shortage areas,⁷ we see distinct shortages in primary care specialties, which include family medicine. This reality is especially troubling because family physicians are the backbone of primary care in rural areas.⁸ In addition to providing outpatient primary care services, family physicians support the workforce by extending the services of psychiatrists,⁹ obstetrician-gynecologists,¹⁰ pediatricians,¹¹ and emergency physicians.¹² Many family physicians also provide inpatient care at rural and critical access hospitals. For example, a study of rural Idaho family physicians found that 88.9% of respondents provided inpatient admissions.¹³ A study of Washington family physicians found that across the state, 81% had admitted a patient to the hospital in the previous 2 weeks. ¹⁴ Indeed, a recent analysis found that later-career rural-trained family physicians had a broader scope of practice compared to their urban-trained peers. ¹⁵

Previous research has identified specific characteristics and exposures associated with specialty choice and rural practice location; our study looked at the combination of these factors to determine which factors predicted rural primary care and family medicine practice. For the purposes of this study, primary care is defined to include family medicine, general internal medicine, general pediatrics, and medicine-pediatrics without further specialization. Evidence has suggested that students who come from a rural background are more likely to practice in rural communities, compared with students who do not have a rural background. ^{16, 17}

Evidence also suggests that students who complete a rural training track are more likely to practice in a rural site compared with those who do not.¹⁸ In addition, factors such as early and repeated exposure to family medicine¹⁹ and participation in immersive rural/underserved clinical training²⁰ seem to increase student likelihood of practicing primary care, regardless of location of practice. ²¹

Important to note is that UWSOM is also home to the Targeted Rural and Underserved Track (TRUST). That program began in 2008 and offers a customized curriculum that maximizes learning time in rural communities for students recruited from rural WWAMI backgrounds.²² During the period covered by this study, TRUST scholars were required to complete the UP. As a result, their numbers drove much of the UP enrollment at WWAMI campuses in the early years of the program. Since 2010, more than half of UP students came from WWAMI campuses; the majority were TRUST scholars. Historically, the campus with the highest proportion of its students enrolled in the UP was Montana, which also has the largest TRUST cohort.

The UP is managed by the Department of Family Medicine and welcomes students from all specialty interests. With programming and curriculum spanning all 4 years of medical school, the UP gives students frequent touchpoints that aim to maintain and nurture their interest in working in underserved communities. Figure 1 shows UP components across UWSOM’s three-phase curriculum. The UP’s asynchronous curriculum, mentorship, service learning, scholarship, and clinical requirements provide scaffolding that allows students to customize their experience and explore different dimensions of medicine in underserved communities. A 2017 qualitative study of UP graduates from the class of 2013 found that while no one curricular component predicted specialty choice, 78.6% reported that the overall UP experience did influence their career choice.²³ Because the University of Washington is dedicated to meeting the workforce needs of our region, including serving underserved communities, the current study sought to understand the specific characteristics of students in the UP and whether UP completeion was associated with rural practice, particularly in primary care specialties.

Students are encouraged to join the UP early in their medical school careers; the application is noncompetitive and no limit is placed on the number of students who can join the program. Further, no consequences exist for students who join the UP and do not complete all the requirements to graduate from the program. Between 2009 and 2016, attrition averaged 11 students per year (range 3–14) who participated in UP activities but did not graduate from the program. As the number of UP graduates has continued to grow, the attrition rate has declined as a percentage of UP enrollees.

Using an innovative propensity-score matching model, we compared 8 years of UP graduates to a matched cohort of UWSOM students who did not complete the UP. This model allowed us to isolate specific variables and determine which factors were associated with practice in rural areas and/or primary care specialties. Our results can help inform our approach to UP curriculum and programming based on student characteristics as well as offer considerations for outreach and recruitment of students with interests in rural medicine and primary care. Specifically, we hoped to demonstrate that UP graduates were more likely to practice primary care and more likely to practice in rural areas than their non-UP peers.

The study population included 2,027 students who matriculated at UWSOM between 2004 and 2012; 243 of these students graduated from the UP. Of the non-UP graduates, 918 had complete data for propensity-score matching on key variables. These variables included age, self-identified person of color versus not, sex, whether the graduate reported family medicine as their top choice of specialty at matriculation, interest in urban underserved care, interest in rural underserved care, whether the student had a physician parent, and whether they self-reported disadvantaged status. Though much of the research in academic medicine focuses specifically on racialized groups considered underrepresented in medicine, we use a broader category of person of color, which includes groups that experience race-based discrimination and are not considered underrepresented (eg, Asian and Pacific Islander). In work with students of color at UWSOM, this practice has emerged as their preferred language and categorization. The person of color variable was drawn from the matriculation survey where students self-identified their race.

We did propensity-score matching using nearest neighbor and a 0.2 caliper for UP graduation. Of the 243 UP graduates, 211 had complete data for matching and were matched to 206 non-UP graduates for a total of 417. Between January and March 2022, the research team performed web searches on this sample to identify specialty and practice location. Where possible, the team identified physician practice site through a health system or clinic website rather than an aggregator or ratings website. Once practice location (zip code) was determined, we calculated the Rural-Urban Commuting Area (RUCA)²⁴ codes using the Federal Information Processing Standards in February 2023. RUCA codes classify census tracts based on rurality using numbers 1 to 10; higher numbers indicate more rural areas, while lower numbers indicate more metropolitan/urban areas. Of the sample of 417 propensity-matched graduates, we identified specialty, practice location, and RUCA code for 191 UP graduates and 193 non-UP graduates, or 384 total graduates.

To determine the success of the propensity-score matching, we compared means on the key demographic matching variables using analysis of variance. Five outcome measures were selected: primary care practice, family medicine practice, rural primary care practice, rural family medicine practice, and rural practice. Rural was defined as a RUCA code of 4 or greater. We used multiple logistic regression to predict whether graduation from the UP was associated with these five outcome variables, controlling for demographics that influence specialty choice, Step 1 score as a marker of academic performance, and matriculation into the TRUST program (which has a mission of training physicians for rural underserved careers). We also controlled for physician parent because that was found to be significantly different between the UP and non-UP graduates, despite the propensity-score matching. Alpha was set at 0.05. All analyses were performed with SPSS version 28 (IBM). The University of Washington Human Subjects Division found this study to be exempt from formal review.

A total of 384 UWSOM graduates were included in the analysis. Table 1 summarizes the demographics of the total sample and shows the similarities between UP graduates and their propensity-matched peers. The only variable that was statistically significantly different between the two groups was having a physician parent: 13% of UP graduates had a physician parent versus 31% of non-UP graduates. To determine differences between UP and non-UP graduates, and to determine which factors predicted rural practice or family medicine specialty choice, we used a series of analyses with unique variables and outcome measures. Our analyses found that overall, older student age and lower Step 1 score predicted family medicine specialty choice, and UP graduates and White students were more likely to practice in rural sites. We found that UP graduates were three times as likely to practice rural family medicine compared to non-UP graduates and three and a half times more likely to practice rural primary care.

Table 2 shows multiple logistic regression for the outcome of primary care practice and, separately, the outcome of family medicine practice. Only female sex uniquely predicted primary care practice (OR 1.758, P=.017), while older age (OR 1.067, P=.049) was associated with family medicine practice. Lower Step 1 score was associated with both outcomes: primary care practice (OR 0.986, P=.017) and family medicine practice (OR 0.972, P<.001). These findings echo previously published research findings on the relationship between Step 1 score and specialty choice²⁵ before the change to pass/fail grading on the exam.

Our analyses did find significant relationships between UP graduation and rural practice, and between race and rural practice. Table 3 shows that UP graduates entered rural practice more often than non-UP graduates (OR 2.469, P=.018), and graduates who identified as people of color entered rural practice less often than White graduates (OR 0.447, P=.029).

Looking at the outcomes of rural primary care practice and rural family medicine practice, Table 4 shows that UP graduates were almost three times more likely to enter rural family medicine practice than non-UP graduates (OR 2.972, P=.041). Graduates who identified as people of color entered rural family medicine practice less often than White graduates (OR 0.247, P=.016). UP graduates had even higher odds of entering rural primary care practice compared to non-UP graduates (OR 3.584, P=.015). Important to note is that these associations held even after controlling for enrollment in the TRUST program. TRUST enrollment drove a majority of the UP enrollment at WWAMI sites; therefore that variable also captured variation in UP enrollment state to state.

This study demonstrated that students who completed a cocurricular longitudinal program focused on care of underserved communities were more likely than their matched peers who did not complete the program to practice rural primary care and rural family medicine. Even when controlling for other variables, UP graduates were over three and a half times more likely than non-UP graduates to practice rural primary care. Programs like the UP are an important offering for schools looking to promote primary care and rural practice among their graduates.

This cross-sectional study adds to a body of literature demonstrating the importance of longitudinal programming in undergraduate medical education for addressing workforce shortages in rural primary care. While more longitudinal research is needed to understand which components of the UP had the greatest impact on graduates, the use of a propensity-matched cohort clearly demonstrated the increased likelihood of UP graduates practicing rural primary care. These results highlight the need for continued investment in longitudinal programs that encourage students to explore careers in rural and underserved communities, particularly in geographic areas with physician shortages.

Other medical schools may find certain components of the UP program easy to replicate for their students; the online curriculum is publicly available, and visitors are welcome to peruse the module content. The curriculum of the UP leverages existing structures in the UWSOM to make completing program requirements through their required training easy for students. For example, most students at UWSOM will complete at least one core rotation at an underserved site, and those rotations count toward UP completion. All UWSOM students must satisfy a scholarship requirement, so students in the UP can choose to focus their scholarship on health equity topics to meet that requirement. These components give students the structure and flexibility to tailor their experience and meet UP requirements within their required curriculum. Other schools may find that working within their existing structures to develop program components that provide additional exposure or opportunities to engage with underserved populations makes sense.

With a mission to serve the WWAMI region’s rural and underserved communities, the University of Washington continues to invest in programs like the UP to prepare graduates for practice across the region and beyond. Programs like the UP support this mission and translate it into practice by increasing the likelihood that our graduates will serve our mission-defined populations. However, programs like the UP are not enough to close the provider gap in rural communities. In addition to opt-in curricular and extracurricular activities, medical schools can and should offer more exposure to primary care, underserved communities, and rural practice to spark and maintain student interest.

This study used an innovative method of propensity-score matching to create a control group. By developing a set of variables and selecting non-UP graduates who closely resembled program graduates, we were able to use rigorous statistical analyses on a sufficiently large sample. Selection bias is always an issue when attempting to evaluate the effects of voluntary and extracurricular programs. Having a nonparticipating control cohort with matched backgrounds at entry significantly strengthened the study and adds to the evidence that longitudinal opt-in programs may have benefits for the rural and primary care workforce.

Another variable worth further exploration is race. The UP uses Dr Camara Jones’ definition of race as a guiding framework: “Race is a social construct, a social classification based on phenotype, that governs the distribution of risks and opportunities in our race-conscious society.”²⁶ As medical schools in the United States continue to explore avenues for both increasing diversity in the physician workforce and addressing physician shortages in rural communities, better understanding the relationship between physician race and rural practice will be important. While many studies have explored demographic characteristics that influence or predict rural practice, none seem to have found a clear relationship with race, which is often used as a crude proxy for socioeconomic status or other social determinants of health.

This is a single-institution study and may not be generalizable to other institutions or training programs. In addition, the research team relied on Internet searches to determine graduates’ current practice location; when possible, the location was cross-checked with multiple data sources, with a particular focus on finding websites belonging to health systems themselves rather than aggregator or review sites. However, we could not independently verify each data point, and we recognize that the data represent a point-in-time snapshot of the cohort. Because this work focused on location of practice and primary specialty, identifying scope of practice was beyond the ability of the team. Additionally, of the 2,027 identified school of medicine graduates, 866 did not have complete information and could not be included in developing the control cohort. Furthermore, web searches failed to identify practice location or specialty for some graduates; this was usually due to circumstances such as name changes (ie, marriage), international practice (no RUCA code), or the choice not to pursue residency training or clinical practice. We do not know what effect was created by this lack of information. Despite this limitation, creating a matched cohort seeks to control for any effect.

Finally, the UP itself is a possible source of selection bias. As a voluntary program housed in family medicine, students who join the UP possibly are already more likely to pursue family medicine and/or rural practice. Despite their similarity to other nonparticipating students, they may be different based on characteristics we cannot identify or measure. Also possible is that programs like the UP do not convert students to enter rural primary care or family medicine, but that they do, through scaffolding students’ experiences, support and nurture those interests.

Programs like the UP are an effective way to engage students in curriculum, service, and professional development. Our students gain the language and skills to work in a broad range of specialties and communities. By comparing UP students with other students who did not complete the UP, this study highlighted the fact that programs like the UP support students who ultimately will choose to practice rural primary care.