Objective Admissions Data and In-State Practice: What Can We Really Predict?

Medical school admissions committees commonly struggle to identify which applicants are most likely to practice in-state following training. This struggle is of particular interest for institutions committed to addressing regional health care shortages and for specialties that serve in areas of health care shortage, such as family medicine. Although research exists on predicting academic performance in medical school applicants using cognitive and noncognitive variables, ^1-4 research concerning applicant variables affecting future practice location is sparse.

Studies have reported that residency training location is a stronger predictor than medical school of future practice location. ^5-8 Reportedly, 52.9% of residents practice in the state of residency training. ⁹ Another study reported that ties to the state and primary care residency are associated with in-state practice. ¹⁰ Pipeline programs also play a role in shaping graduates’ career choices and increasing the likelihood of pursuing primary care or practicing in-state.^11-13 One other study showed that rural origin and intent to practice in rural areas and/or family medicine were strongly correlated with rural practice, but did not address the question of in-state practice. ¹⁴

The University of New Mexico School of Medicine (UNM SOM) was established to address the critical shortage of medical care across the state and is strongly committed to admitting students from New Mexico. Through a holistic review process, the UNM SOM admissions committee considers objective data, personal attributes, goals, and experiences, aiming to assess the likelihood of an applicant’s future in-state practice and thereby advance the school’s mission.¹⁵ This holistic review process complements other UNM SOM programs similarly aimed at enhancing future in-state practice, such as pipeline programs aimed at groups underrepresented in medicine (URiM) and/or from rural backgrounds. ^{11, 13}

This study evaluated whether objective applicant data (ie, academic, geographic, demographic, and educational) are associated with future in-state practice.

This study examined applicant objective data of eight medical student cohorts between 2006 and 2013 (N=617) that graduated between 2010 and 2017 and completed graduate medical education by the time of analysis. Individuals who withdrew from medical school (n=24) or residency (n=8) do not have a current medical license (n=5, including 2 deceased) and current students (n=2), residents (n=36) or fellows (n=33) were excluded. The final dataset consisted of 509 practicing physicians.

Applicant objective data included high school location (rural or urban), Medical College Admissions Test (MCAT) percentile, grade point average (GPA), college location, college major, graduate school concentration, race/ethnicity, and gender. In-state versus out-of-state practice locations were identified using public information from the Centers for Medicare & Medicaid Services National Plan and Provider Enumeration System National Provider Identifier registry, state medical boards, and additional online resources.

Initial data analysis was done using χ² for proportions or t test of means to compare practice groups (in-state vs out-of-state). Next, using variables for which a statistically significant difference in the univariate analysis existed, path modeling using multiple regression was performed.

Table 1 shows that of the 509 graduates, 212 (41.7%) practice in New Mexico. Average age at the time of application differed by 1 year, in-state practicing physicians being older (P<.001). One-hundred (47%) graduates practicing in New Mexico identified as URiM compared to 98 practicing out-of-state (33%, P<.01). In addition, graduation from a rural high school significantly differed as 40 (19%) practicing in-state attended rural high schools, while 83 (28%) practicing out-of-state attended rural high schools (P<.05). Attending a New Mexico institution of higher education (NMIHE) was more frequent for in-state practicing MD graduates: 140 (66%) versus 163 (55%) out-of-state, (P<.05). Finally, mean MCAT percentiles differed, in-state 56th percentile versus 67th percentile for out-of-state (P<.001). No significant differences were found for gender, undergraduate major, graduate school area of concentration, or GPA.

A multiple regression was done to determine independent effects of statistically significant predictors from univariate analysis. Figure 1 illustrates the path model of statistically significant applicant characteristics (URiM status, age, rural high school, MCAT score percentile, and NMIHE). Relationships as indicated by the coefficients and model R ² are standardized estimates with significant relationships indicated by solid arrows and nonsignificant by dotted arrows. Of the five relationships, three were significant: age (0.14, P<.05), graduating from a rural high school (-0.11, P=.02), and MCAT percentile (-0.11, P<=.05). These data indicated, holding all other variables constant, the following: As age increases by 0.14 years, in-state practice increases 1%; as proportion graduating from rural high school decreases 0.011, in-state practice increases by 1%; and as MCAT percentile decreases by 0.011%, in-state practice increases by 1%. URiM and NMIHE drop out of statistical significance in the multiple regression.

The R ² is significant (P<.05) with an estimate of 0.064; this model explains 6.4% of the variance. Considering the substantial number of characteristics that could explain practicing in NM, this R ² provides some, but limited, insight.

Univariate analysis of applicant objective data that would predict applicants’ likelihood of practicing in-state in the future revealed that older, minority, urban high school or in-state institution of higher education graduates, or those with lower MCAT percentiles were more likely to practice in-state. However, multiple regression modeling showed that only age, not attending a rural high school and MCAT percentile, was associated with future in-state practice. Identifying as URiM and graduating from a NMIHE were not.

Age being significantly associated with future in-state practice may suggest that older applicants have different personal or professional motivations. Explanations for associations between lower MCAT percentile and urban high school with future in-state practice are not evident.

The most important conclusion is the limited value of applicant objective attributes predicting future in-state practice. Multiple regression modeling explained only 6.4% of variance, indicating the existence of factors not captured in our analysis. These findings suggest a holistic approach to admissions may aid in predicting future in-state practice by considering applicant experiences, values, goals, motivations, and lifestyle preferences.

Although the study focused on informing admissions committees, important to acknowledge is that factors evolve throughout subsequent training years (ie, health care policies, financial and workforce incentives, family circumstances, partner and family priorities, personal priorities, economic conditions), influencing choice of practice location by reinforcing or diminishing the importance of preexisting factors. While these factors may impact graduate physicians’ practice location, this future information is not available to admissions committees.

These applicant objective data do not include the subjective elements (ie, narrative responses, community service, and interview data), which may offer greater value to admissions committees regarding future practice location. Additionally, the absence of data on specialty choices and in-state practice is also a limitation. Specialty information could clarify the alignment between admissions data, future specialty choice, and practice location. Further, although the use of eight cohorts is a strength of the analysis, generalizability is limited because they represented a single state.

These findings underscore challenges admissions committees face in predicting applicants most likely to practice in-state. Further research might explore the value of specific subjective applicant data in this role. Incorporating specialty data could provide deeper insights into how admissions data may be associated with meeting in-state needs. Identifying applicants likely to practice in-state and to address health care needs is clearly an important goal for many admissions committees. Nevertheless, committees are limited in judging future practice location from the snapshot presented in applications. Applicant objective data alone does not appear to effectively predict practice location.