Impact of the Rural Physician Associate Program on Workforce Outcomes

The United States continues to face a physician workforce problem. There is an ongoing and predicted shortage of rural providers across the country.1 This imbalance in the distribution of physicians is an important factor contributing to health disparities seen in rural areas.2 The United States also suffers from a shortage of primary care providers. By 2033, there will be an estimated shortage of between 21,400 and 55,200 primary care providers across the country.1

In order to address the growing physician workforce shortages, medical education programs have attempted various strategies to increase the rural and primary care physician workforce, such as adopting an admission policy that favors students coming from rural backgrounds or expressing an interest in primary care, and implementing rural training experiences during medical school and residency.3-5 As part of a land-grant institution, the University of Minnesota Medical School (UMMS) has a responsibility to meet the physician workforce needs of the state. Seventy percent of the active physician workforce in Minnesota has trained at UMMS for either undergraduate or graduate medical education (unpublished internal UMMS data). To specifically address Minnesota’s workforce needs, UMMS aims to increase the number of graduates who practice in Minnesota, specialize in primary care (particularly family medicine), and practice in rural settings. Strategies to increase the rural physician workforce have been implemented at UMMS,6 and this study evaluates the long-term outcomes, spanning nearly 50 years, of implementing an undergraduate rural longitudinal integrated clerkship (LIC) training model.

As part of the effort to address the workforce needs of Minnesota, the UMMS Rural Physician Associate Program (RPAP) LIC was established in 1971 by the medical school in collaboration with the Minnesota legislature “to redistribute physicians into the medically underserved rural areas of Minnesota.”7 RPAP has a mission to increase the number of graduates entering family medicine, and who will practice in rural settings. RPAP is a 9-month, rural, community-based LIC designed to nurture third-year medical student interest in rural medicine and comprehensive primary care.6,8-10 Students from either the Duluth or Twin Cities campus apply to the program during year 2 of medical school. Every effort is made to place all interested students in the program. Students enter the program with varied specialty interests, practice location intentions, and backgrounds. During the program, students live and train in rural communities across Minnesota where they complete core clinical clerkships and are mentored by both academic and community faculty. RPAP class size varies year to year, ranging from 19 to 47 students, averaging 35 students. While participating in RPAP, students form and sustain strong relationships and connections within the rural community. Students spend the entire 9 months maintaining continuity with a primary care practice, almost all with family medicine preceptors. Community physicians from multiple specialties serve as mentors for students as they evaluate and treat patients in the clinic, hospital, emergency department and long-term care centers, assist in surgery, consult with other health care professionals, participate in educational activities including online modules, and carry out community service projects. Students who do not participate in RPAP complete a majority of their clerkships in the Minneapolis/St Paul metro area or in the Duluth area. Performance of students in RPAP has been shown to be equivalent to students rotating in traditional metro-based block clerkships.11,12

Previously published work suggests the RPAP experience is associated with students choosing primary care practice, especially family medicine, and choosing to practice in a rural location.9 At the time this previous work was published, no direct comparison of practice location could be made between RPAP and non-RPAP UMMS graduates, due to limitations in tracking the practice demographics of these students over time. The purpose of this study was to examine the relationships between participation in the RPAP program and the desired workforce outcomes of (1) practice in Minnesota, (2) primary care specialty (particularly family medicine), and (3) rural practice. In contrast to previous work, this study directly compares practice outcomes between RPAP and non-RPAP UMMS graduates via outcomes data from national physician databases in order to evaluate the success of the RPAP program in meeting its workforce development objectives.

Ethical approval for this research was granted by an Institutional Review Board of the University of Minnesota (study number STUDY00006864).

Participants in this study included 9,145 graduates of UMMS who completed graduate medical education training between 1975-2017. Because RPAP was first established in 1971, we began our analysis for graduates who had completed graduate training in 1975 allowing for completion of medical school and a three-year residency. We included all graduates when a link could be made between their educational data and practice outcomes data (see the following “Linking Data Sets” section). Of those, 1,217 graduates (13.3%) participated in the RPAP program and 7,928 (86.7%) did not.

This study was conducted by the UMMS Medical Education Outcomes Center (MEOC), an interdisciplinary center that connects learner data to health and workforce outcomes.13 In order to obtain recent practice and training information, including current practice locations, specialty, training milestone dates, and certifications, we used the American Medical Association (AMA) Physician Masterfile. In 2019, UMMS purchased a subset of the Masterfile containing 13,092 UMMS graduates from Medical Marketing Service (Schaumburg, IL), which was licensed by the AMA to distribute these data. Of the 13,092 UMMS graduates in the Masterfile, National Provider Identifier (NPI) numbers were available for 10,443 individuals. NPI numbers are issued by the Center for Medicare and Medicaid Services and are used by Medicare and commercial insurers to identify the specific provider of health care services.

We linked the 10,443 AMA Masterfile records with NPI numbers to the publicly-available National Plan and Provider Enumeration System (NPPES) NPI data set. The NPPES NPI data set contains additional data points, including provider name. We then linked the combined AMA Physician Masterfile and NPPES NPI data to internal UMMS student databases through a ranked matching algorithm, based on matches between data points including name, birth city and state, birth year, medical school graduation year, residency location, year, and specialty. We resolved incomplete matches manually. Utilizing internal UMMS databases, we identified 1,312 UMMS students in the combined data set who had participated in the UMMS RPAP LIC (RPAP graduates). After using the matching algorithm and a manual matching second-pass, 95 RPAP graduates were not identified in our linked AMA Physician Masterfile and NPI data set or were excluded from analysis because they were outside the chosen date range. Because the AMA Physician Masterfile was purchased in 2019 and NPPES NPI data was collected in 2018, we excluded 2018 and beyond to ensure data available for each year was complete. Fifteen non-RPAP graduates did not have a practice location available in the data. The final total number of graduates in our sample was 9,145 (RPAP n=1,217, non-RPAP n=7,928).

To identify the practice specialty of UMMS graduates, we used the primary specialty field within the AMA Physician Masterfile. To improve the consistency of the data, we aligned the specialty data in the Masterfile with the American Board of Medical Specialties (ABMS) specialty designations.14 We reviewed the primary and secondary specialty type for each graduate and assigned primary care status to those listing adolescent medicine-pediatrics, family medicine, family medicine/geriatric medicine, general practice, general preventative medicine, internal medicine, internal medicine - family medicine, internal medicine - geriatrics, internal medicine - pediatrics, pediatrics, and psychiatry - family medicine as their primary specialty. We excluded graduates with a secondary specialty other than one of the listed primary specialties, an addiction medicine specialty, or a family medicine/sports medicine specialty.

For practice location, we utilized the NPI data set’s state field in order to identify graduates who practice within the state of Minnesota versus those who do not, and the zip code field for identifying those who practice in a rural or urban setting. The practice settings were designated either urban or rural based on the Rural-Urban Commuting Area Codes (RUCA) definition of urban (RUCA codes 1-3) and rural (RUCA codes 4-10).15

We analyzed our data using two-sample difference-of-proportions right-tailed normal-distribution hypothesis tests,15 executed in Python 3.7 (Python Software Foundation, Beaverton, OR). For each desired outcome (eg, practice in-state), we computed a contingency table of counts for both the RPAP and non-RPAP groups, comparing the proportion of those who participated in the desired outcome (eg, practice in state) in the RPAP group to the same proportion in the non-RPAP group. We set statistical significance at a P value of 0.01. We additionally calculated 99% confidence intervals (CIs) for the difference of these two proportions.

A visualization of the relationships between the variables under study are shown in Figure 1. Statistics for each workforce outcome are shown in Table 1.

Among UMMS graduates, RPAP graduates practice in state at a higher rate (799/1,217 [65.7%]) than non-RPAP graduates (4,302/7,913 [54.4%]; P<.01, 99% CI 7.3%, 15.2%).

RPAP graduates practice in primary care at a higher rate (840/1,217 [69.0%]) than non-RPAP graduates (2,649/7,928 [33.4%]; P<.01, 99% CI 31.8%, 39.5%).

Within the primary care group of specialties, RPAP graduates practice family medicine at a higher rate (743/840 [88.5%]) than non-RPAP graduates (1,370/2,649 [51.7%]); P<.01, 99% CI 40.4%, 47.1; Table 1). Overall, 61.1% of all RPAP graduates specialize in family medicine (743/1,217), versus 17.3% of all non-RPAP graduates (1,370/7,928).

A higher percentage of RPAP graduates are practicing in rural areas (502/1,217 [41.2%]) than non-RPAP graduates (1,103/7,928 [13.9%]; P<.01, 99% CI 24.3%, 30.4%). A visual representation of rural and urban regions in the state of Minnesota where graduates are practicing is shown in Figure 2.

In addition, the UMMS targeted workforce outcomes trends are sustained from graduates who completed postgraduate training from 1975-2017. These temporal trends are shown in Figure 3.

Participation in the RPAP LIC, a specialized family medicine-based rural track during third year of medical school, is associated with a higher percentage of graduates staying to practice in Minnesota, choosing a career in primary care (particularly family medicine), and practicing in a rural area, compared to UMMS graduates not participating in RPAP. Using novel integration methods and analysis of multiple data sets, these results update previous reports on the impact of RPAP on rural physician workforce across nearly 50 years, demonstrating the sustainability of this program in producing and retaining rural family physicians.6,9

Rural programs, as recently defined and listed by Longenecker et al, are offered at about 20% of US medical schools.17 Recent reports of established programs align with our findings that rurally-trained medical students are more likely to join and sustain the rural, primary care, and family medicine workforce.17-20 Compared to UMMS matched peers, RPAP graduates have contributed significantly to the Minnesota rural primary care workforce, primarily as family physicians, supporting the findings of Wendling et al, that choosing family medicine may be the best predictor of enduring impact on the primary care workforce.17

UMMS supports a rural mission in multiple ways. The UMMS Duluth medical school campus (UMN-Duluth) is a program that educates first- and second-year students, and has an admissions process and mission to develop rural physicians. A recent study of UMN-Duluth students showed 47% chose family medicine and 37% chose a rural first practice location.21

Previous analysis demonstrated both UMN-Duluth and RPAP experiences significantly increase the outcomes of primary care specialty and rural practice, the effects are additive, and greater than being raised in a rural community.6,9 Increasing the number of students who participate in RPAP is likely to further improve workforce outcomes. RPAP recruitment strategy includes strongly supporting the rural-focused admission process to UMN-Duluth, identifying and increasing participation in RPAP among students who expressed commitment to rural practice at matriculation, and welcoming students from urban backgrounds to explore rural practice via RPAP.

Consistent with other rural programs, RPAP emphasizes cohort experiences for students to develop rural identity and promotes experiences to engage deeply with their host community.22 Key components of RPAP include a family medicine focus, living in the community for 9 months, longitudinal patient relationships, and completion of a community health assessment project. This project requires a longitudinal relationship with community partners and development of rural community engagement skills.23 A community engagement component has also been shown to positively impact rural career choice in rural training tracks during residency.4

Research into workforce outcomes of graduates who participated in long-standing rural educational programs compared to their peer medical school classmates is challenging, as it requires accurate pairing of practicing physician data sets to their historic student data sets. Wendling and colleagues used the AMA Masterfile as a primary tool for determining practice location.24 Our study links multiple data sets including the AMA Masterfile, NPPES, and UMMS student databases to efficiently compare student education pathways and practice outcomes and responds to the recent call to use practice data rather than residency match data to determine primary care contribution.17

Expanding rural medical school programs as a strategy to meet rural workforce needs, and identifying other factors which predict eventual primary care and rural practice, is an area of ongoing research.25-27 Parlier and colleagues examined factors in successfully recruiting and retaining rural primary care physicians.4 These investigators concluded that rural exposure during medical school or a rural emphasis of a school including such factors as admission criteria, location of the school in a rural location, and use of rural faculty can increase the number of graduates going into rural practice, but pointed out the confounding effect of self-selection. Growing up in a rural community and expressing an interest in primary care, specifically family medicine, before the start of medical school are also predictors of future rural primary care practice.5,10,20,27 These factors are also likely to influence a student’s choice to participate in a rural training program.

Our study contributes to the literature regarding the long-term and sustained impact of a rural LIC on workforce outcomes. While the rural health care workforce shortage remains, it is important to identify modifiable factors that may be at least partially effective countermeasures. These countermeasures exist at different time points along the education and practice continuum and this study sought to measure the relative effectiveness of an educational experience on workforce outcomes. By comparing RPAP with non-RPAP outcomes, we feel confident that we have demonstrated an educationally sensitive workforce outcome (ESWO). We define ESWOs as workforce outcomes that are sensitive to changes and innovations in educational programs and thus a means to modify workforce outcomes through educational interventions. Our ongoing work is focused on developing a prospective model of determinants of rural workforce outcomes, but this is still in progress. Nonetheless we feel the long-term outcomes reported here add value to the existing literature in this area and are an important part of a comprehensive approach to understanding the factors that lead to (or mitigate) workforce shortages and disparities.

This study has several important limitations. In this study, the RPAP and the non-RPAP group are products of a shared admissions process. After matriculation, students self-select to the RPAP program, and the influence of demographic variables that could have impacted choice to participate in RPAP, and rural or family medicine practice were not evaluated. This project could not discern the influence of potential selection bias.

Additionally, the NPPES NPI data set provides only the current practice location for graduates. We know some graduates move from an initial rural practice to urban practice, and some communities have grown from rural to urban over the decades. In this study we did not examine the percentage of years that graduates spent in rural practice. Through the process of joining our AMA Physician Masterfile data set with NPPES NPI data and our internal UMMS student databases, not all UMMS graduates were identified in the practicing physician data set. Reasons include those who are currently in residency or training, providers who do not have an NPI number, individuals who are practicing outside of the United States, and providers who are not practicing. Based on medical school graduation year, the difference between UMMS internal databases and the AMA Physician Masterfile was small (1%-3% difference each year) and unlikely to affect the overall findings. Data regarding Minnesota rural workforce needs over the period of time RPAP has existed are not available so it is difficult to determine the proportion of these needs that were met by the RPAP LIC.

We found significant relationships between participation in the RPAP program and postgraduation practice in a rural setting, family medicine, primary care, and in-state practice, but more investigation is needed to demonstrate causality. Our current work is focused on developing a comprehensive approach to modeling and predicting workforce outcomes based on learner and educational factors in order to design educational programs that proactively meet workforce needs.

This study demonstrates a significant association between participation in RPAP and a career in family medicine, rural practice, and primary care—all outcomes that promote meeting urgent rural workforce needs. We believe these to be educationally sensitive workforce outcomes that serve as the basis for further exploration into causality and predictive analysis to determine which learners are most likely to participate in a rural family medicine training program, and the extent to which participation in a rural family medicine training program independently predicts the desired practice and workforce outcomes.