Background and Objectives: Schools of medicine in the United States may overstate the placement of their graduates in primary care. The purpose of this project was to determine the magnitude by which primary care output is overestimated by commonly used metrics and identify a more accurate method for predicting actual primary care output.
Methods: We used a retrospective cohort study with a convenience sample of graduates from US medical schools granting the MD degree. We determined the actual practicing specialty of those graduates considered primary care based on the Residency Match Method by using a variety of online sources. Analyses compared the percentage of graduates actually practicing primary care between the Residency Match Method and the Intent to Practice Primary Care Method.
Results: The final study population included 17,509 graduates from 20 campuses across 14 university systems widely distributed across the United States and widely varying in published ranking for producing primary care graduates. The commonly used Residency Match Method predicted a 41.2% primary care output rate. The actual primary care output rate was 22.3%. The proposed new method, the Intent to Practice Primary Care Method, predicted a 17.1% primary care output rate, which was closer to the actual primary care rate.
Conclusions: A valid, reliable method of predicting primary care output is essential for workforce training and planning. Medical schools, administrators, policy makers, and popular press should adopt this new, more reliable primary care reporting method.
The population of the United States is facing an accelerating shortage of primary care physicians.1-3 Schools of medicine commonly report their primary care output as 100% of graduates who match in internal medicine, family medicine, pediatrics, and medicine-pediatrics residencies. This Residency Match Primary Care method (Match-PC) is commonly used by medical school deans and is widely cited in public media, routinely showing many schools producing 40%, 50%, or higher percentages of their graduates entering primary care. However, such data are known to be significantly overstated, being pejoratively termed “the Dean’s lie” by some.4-5 Previous studies have found only 21% of third-year internal medicine residents,6 10% of those entering internal medicine residencies,7 and 53% of pediatrics residents8 say they intend to practice primary care, as compared to 95% of family physicians applying for initial board certification.9
An adequate supply of primary care physicians is vital to the quality and cost-effectiveness of the US health care system,10,12 which lags behind the rest of the industrialized world as fewer health professions’ graduates choose primary care,13 despite the expansion of medical school class sizes.14
Given the breadth of information documenting that the shortage of primary care physicians in the United States is harmful, and the fact that medical schools are not adequately alleviating the shortage, it is essential that an accurate way to identify true primary care output be available for workforce planning. A more accurate point in time to determine who really chooses primary care practice is after residency completion. Determining which physicians actually practice primary care after residency completion can be difficult and tedious. Commonly used databases like the American Medical Association (AMA) database do not contain scope of practice information. This study is the first of its kind to:
- Identify a large number of medical school graduates from a variety of schools over a long period of time who would be classified as primary care based on the Match-PC method used by deans and public media,
- Determine who among them are actually practicing primary care after completion of residency and entry into practice,
- Calculate the difference in the two methods of measuring primary care output, and
- Suggest an alternative, less resource-intensive method of predicting a school of medicine’s primary care output that more accurately and reliably identifies true primary care practice.
This information and methodology will assist those responsible for health care workforce tracking and planning.
We conducted a retrospective study using a convenience sample of US schools of medicine granting the MD degree. We invited known faculty at 38 schools, including a detailed description of the study objective, data collection protocol, and shared data plan. Over the course of one year, 20 campuses from 14 universities agreed to participate. At each school, a collaborating faculty member provided the data. The target medical students were graduates between 2003 and 2014, to allow time for completion of residency and entry into practice. Graduate information obtained included residency match specialty, residency completion year, and current practice specialty. Total number of graduates per medical school during the time period were obtained from the Association of American Medical Colleges published data. To determine how representative these schools are of all US allopathic medical schools, we compared participating and nonparticipating schools by their published primary care production ranking.15 The institutional review boards at collaborating sites approved the study.
Definitions
The definitions for primary care used in this study are outlined in Table 1. The Match-PC Method reflects how medical school deans and the media report primary care match data and excludes those in subspecialty residencies, surgery, emergency medicine, as well as transitional and preliminary internal medicine residencies. The definition used for Actual Primary Care is based on commonly accepted criteria, as exemplified by the World Health Organization16 and defined by the National Academy of Medicine as
the provision of integrated, accessible health care services by clinicians who are accountable for addressing a large majority of personal health care needs, developing a sustained partnership with patients, and practicing in the context of family and community.17
This definition excludes medical and surgical specialties that do not address the large majority of the patient’s health care needs. It also excludes urgent care, emergency medicine, and hospitalists because they do not aim for or provide a sustained continuity partnership with patients and families.
We created a new method, the Intent to Practice Primary Care Method (Intent-PC Method) that includes only those graduates who entered residencies in medicine-primary, pediatrics-primary, family medicine, and medicine-pediatrics.
Deceased graduates were classified based on practice specialty before death. Deceased graduates who had not completed residency at time of death were excluded. Individuals for whom residency match specialty or current practice information was missing were excluded from the analysis.
Primary care status was determined using a variety of methods, including internet search methods such as Doximity, National Provider Identifier (NPI), LinkedIn, Google/Yahoo searches, or searches of alumni databases, medical board records, or direct contact.
We calculated descriptive statistics to determine the proportion of graduates designated as practicing in primary care by the Match-PC Method, Intent-PC Method, and Actual Primary Care Method both as part of the total sample and by individual medical school. Additionally, we completed sensitivity analyses to assess the impact of missing data on the proportions. We conducted all analyses using SAS v.9.4 or Microsoft Excel.
The majority of schools found most graduates using a generalized search method, through Google or Yahoo searches (Table 2). Schools may have used more than one search method for the same graduate if not all relevant information could be located with one.
Of the 38 institutions invited to participate, 24 did not respond or declined to participate (n=20), or their programs were ineligible because they were too new (n=2), or osteopathic (n=2). Comparing the eligible participating and nonparticipating schools to published primary care rankings15 showed five of the participating schools in the top third, six in the middle third, and three in the bottom third. Nine of the nonparticipating schools ranked in the top third, seven in the middle third, and four in the bottom third of these rankings.
The final study population consisted of 17,509 medical students from 14 institutions across 20 campuses (Table 3). Of these students, 7,206 were considered primary care by the Match-PC Method and were followed to determine their current practice type after residency completion. Medical schools included in the study were predominantly public universities (86%). Most campuses were urban (89%). Participating medical schools’ US geographic locations were: Northeast (1), Southeast (5), Midwest (4), and West (4). All but one institution provided data for 7 to 10 years of graduating classes.
The Match-PC Method yielded primary care rates from 29.1% to 50.5% and averaged 41.2% of the entire study cohort of 17,509 graduates (N=7,206, Table 3). Among the Match-PC Method group, Actual Primary Care rates ranged from 32.0% to 87.1% and averaged 54.1% (n=3,901). Thus, nearly half of the 7,206 graduates labeled as primary care by the Match-PC Method were not actually in primary care, overestimating primary care output by 3,305 physicians or 18.9% of the entire study cohort. Primary care status could not be identified for about 4.5% of the entire study cohort.
The new Intent-PC Method yielded rates from 6.8% to 36.8% and averaged 17.1% of the entire study cohort of 17,509 graduates (Table 3). Among just the 3,001 Intent-PC graduates, Actual Primary Care rates ranged from 36.5% to 97.3% and averaged 76.5%. The Intent-PC Method underestimated primary care output by 900 physicians, or 5.1% of the entire study cohort.
The 3,901 graduates who became Actual Primary Care constitute 22.3% of the entire study cohort of 17,509 graduates (Table 3). The 3,001 number generated by the Intent-PC Method yields a predicted primary care rate of 17.1%.
Of the graduates in the Match-PC Method, family medicine and pediatrics-primary had the greatest percentage of graduates become Actual Primary Care (Table 4); 92.8% of graduates from family medicine residencies became Actual Primary Care, comprising 47.8% of all primary care physicians in the sample and making the largest contribution to the primary care workforce. Approximately 93 % of graduates from pediatrics-primary residencies became Actual Primary Care, comprising 1.1% of all primary care physicians. Only 30.3% of graduates from internal medicine (categorical) residencies and only 29.5% of medicine-primary residency graduates became Actual Primary Care, making up 19.9% and 4.5%, respectively of all primary care graduates in the sample. About 61.1% of all medicine-pediatrics graduates became Actual Primary Care, accounting for about 5.4% of all primary care physicians.
The goals of this study were twofold. First, we wanted to analyze a large sample of graduates from US schools of medicine and compare the actual primary care output at time of entry into practice to the primary care output commonly cited by deans and public media at the time of medical school graduation. Second, knowing that the Match-PC Method overestimates primary care output,4-5 we wanted to identify a more accurate and easily accessible method of predicting actual primary care output without the need to track graduates down at the time of residency completion and entry into practice. This more accurate and easily accessible method is Intent-PC, calculated by using only the medical school match categories of medicine-primary, pediatrics-primary, family medicine, and medicine-pediatrics. Of these categories, family medicine is the largest contributor to total Actual Primary Care workforce at 47.8%.
This study is unique in terms of the level of detail with which the actual practices of physicians were determined. The sample included over 17,000 graduates from 20 different medical school campuses throughout all regions of the United States, both private and public, and widely distributed across the spectrum of medical schools ranked previously15 for primary care output. Participating and nonparticipating schools were similarly distributed across published primary care rankings.
We found that only 22.3% of our cohort of 17,509 graduates actually entered primary care, rather than 41.2% predicted by the Match-PC Method. Thus, 58.4% of the graduates predicted to be in primary care by the Match-PC Method were not actually practicing primary care. We also found the Intent-PC Method predicted an overall 17.1% primary care rate, which was closer to the Actual Primary Care overall rate of 22.3%.
In aggregate, the Intent-PC Method was 5.2 percentage points off from Actual Primary Care, making it overall a more accurate predictor than the Match-PC Method, which was off by 18.9 percentage points. The Intent-PC Method predicted a slightly higher primary care output for three of the 14 schools, and a slightly lower primary care output for 11 of the 14 schools. There were no obvious geographic trends in high versus low predictions, or in the magnitude of difference between the Intent-PC Method and Actual Primary Care. Because our sample is widely distributed across the United States geographically and by previous primary care ranking,15 the Intent-PC Method can be used not only in aggregate but also individually by most medical schools. An individual medical school could perform analyses of a cohort of graduates to adjust the predictive value of the Intent-PC Method to their particular school.
The Association of American Medical Colleges’ 2018 and 2019 reports attribute the accelerating primary care shortage to the trend of physicians, nurse practitioners (NPs), and physician assistants (PAs) entering subspecialties rather than remaining in primary care.18 Our findings confirm this trend for physicians. It is not the purpose of this study to address the relative roles of physicians, PAs, and NPs in fulfilling US primary care needs.
Family medicine (92.8%) and pediatrics-primary (93.5%) had the greatest percentage of graduates become primary care physicians, but family medicine (47.8%) comprised a much greater proportion of the overall primary care workforce as compared to pediatric-primary (1.1%). This study did not have any graduates with combined residencies in family medicine and another subspecialty, but other studies have included those residency programs in their analyses of primary care output.19 Previous work has found that graduates of combined degree programs—specifically BA-BS/MD programs—have a greater percentage of their graduates match into family medicine residencies.20 Future studies may examine how graduates from combined family medicine residencies and combined degree programs differ in their contributions to the primary care physician workforce.
Our study has limitations. The final study population included only data from schools granting the MD degree, so results may only be generalizable to allopathic programs. Some previous studies have found that a higher proportion of graduates from osteopathic programs go on to primary care practice as compared to allopathic programs.21-22 Additionally, the vast majority (86%) of participating schools were public institutions. Previous research has shown that public universities may contribute more to primary care specialties such as family medicine as compared to private institutions.22 Both of these factors may have impacted the actual primary care rates observed in this study.
Although schools from most US geographic regions were included in the study, the Mid-Atlantic/New England region was the least represented, with only one Northeastern institution participating. It is possible that there are regional differences in rates of primary care physician production. However, the study sample was well-distributed among previously-published primary care rankings.15 Although small numbers in this study made subanalyses uninformative, future studies with larger sample sizes should seek to understand these regional differences.
Data on graduates ranged in source. Some information was obtained using specialized search engines such as National Provider Identifier and Doximity; others were identified using more general internet search engines, like Google. More recent graduates tended to have more complete online presences, such as through user-created professional profiles from LinkedIn. To assess how differences in search method may have impacted study results, we conducted a sensitivity analysis on 10% of the graduates missing an actual primary care status designation (analysis excluded schools that did not provide graduate names). These graduates were followed up using a general internet search to see if they could be identified. Overall, less than 10% of the missing graduates were located in this sensitivity analysis. Additionally, there was no significant difference by school in whether or not the sensitivity analysis findings matched the original findings (Fisher exact test P>.05). This lends support to the overall quality of the data, regardless of search method.
We obtained information only for graduates defined by the Match-PC Method and could therefore miss graduates who enter preliminary residencies and subsequently enter primary care. In a pilot study at the University of Colorado, we followed up on graduates who entered internal medicine preliminary residencies and found that only 1.4% went into primary care.23 In an ancillary analysis at Case Western Reserve that followed all graduates over 10 years, only 1.5% later switched to a primary care residency and eventually practiced primary care.
Because data were collected on graduates between 2003 and 2014, the point in time when practice type was determined may have influenced the study results. Individuals who graduated in 2003 were 15 years from medical school completion, as compared to those who graduated in 2014 or 2015 who were only 3-4 years out of school. It is possible some graduates who were in primary care practice at the time they were identified will later enter fellowships or residencies in subspecialties, or otherwise leave primary care, resulting in an overrepresentation of actual primary care physicians within this group.24-25
Our study is limited by missing data both in the original match information (9.0%) and in the follow-up information (4.5%). We determined the potential extent of bias on the proportions by imputing all missing values to practicing primary care as “Yes” and separately “No”; doing so only changed the proportions by 2.3%, thus showing the internal validity of the findings. Additionally, one participant did not distinguish between graduates who entered categorical or primary care residencies in internal medicine and pediatrics. The Intent-PC Method for this university was calculated excluding all internal medicine and pediatrics residents. To determine the impact that this lack of subcategorization had on this university’s results, we calculated a range based on the range of proportions graduates in internal medicine, primary care and pediatrics, primary care residencies across the rest of the sample. We found that the potential range of the proportion of graduates classified as Intent-PC for this particular university was 21.4% to 31.4%. This potential misclassification did not significantly change the regression findings and only increased the difference between the two primary care match methods, thus strengthening the conclusions of the study.
The Intent-PC Method slightly underestimated actual primary care output. This underestimate is the net result of missing some categorical internal medicine and pediatrics residents who actually practice primary care, and counting some family medicine, medicine-pediatrics, primary internal medicine and primary pediatrics residents who do not. The exact proportion of graduates in these residencies that eventually practiced primary care varied widely by institution, but the missed resident numbers from these two residencies have the largest effect on the Intent-PC estimate. For future use, each school could adjust their Intent-PC formula based on evidence about their own graduates to produce a revised local estimate. In the case of the University of Colorado, 19% of categorical internal medicine and 47% of categorical pediatrics residents entered primary care (72 and 68 individuals, respectively). Adding these percentages of categorical internal medicine and pediatrics residences increases the percentage of graduates meeting the Intent-PC definition from 11.4% to 20%, which is closer to the actual rate of 18.6%. Regardless of these adjustments, the Intent-PC method is still considerably closer to the actual primary care rate than the Match-PC method.
Our findings confirm that the current metrics reported by medical schools in the United States significantly overestimate the number of primary care physicians expected to enter the workforce. The Intent to Practice Primary Care Method documented in this study more accurately predicts how medical schools are, or are not, addressing the shortage of primary care physicians in the United States. Medical schools, administrators, policy makers, and the popular press should change their primary care reporting to this new, more predictive method.
Acknowledgments
The authors thank all of the medical schools that participated and the medical students and research assistants who worked on the project. The authors particularly acknowledge James Leeper, PhD, University of Alabama; Susan Schmidt, MD, East Carolina University; and Randall Longnecker, MD; Ohio University, for their support, participation in the study, and data collection efforts; and Larry Green, MD, University of Colorado for his careful review of the manuscript.
Financial Support: This research was predominantly unfunded work. Authors Mark Deutchman, MD, Katherine A. James, PhD, MS, MSPH, and Francesca Macaluso, MPH, were supported in part by a Colorado Health Foundation grant (ID #4526).
References
- Association of American Medical Colleges. New Findings Confirm Predictions on Physician Shortage. https://news.aamc.org/press-releases/article/2019-workforce-projections-update/. Published April 23, 2019. Accessed July 19, 2019.
- Bodenheimer T, Pham HH. Primary care: current problems and proposed solutions. Health Aff (Millwood). 2010;29(5):799-805. https://doi.org/10.1377/hlthaff.2010.0026
- Dall T, West T, Chakrabarti R, Iacobucci W, Brunec P. The Complexities of Physician Supply and Demand: Projections from 2013 to 2025. Washington, DC: IHS Markit Ltd; 2015.
- Bernstein K. Future of Family Medicine: The “Dean’s Lie” About Medical School Primary Care Production. Future of Family Medicine Blog. April 2011. http://futureoffamilymedicine.blogspot.com/2011/04/deans-lie-about-medical-school-primary.html. Accessed July 19, 2019.
- Schwitzer, G. “The Dean’s Lie” – About New Docs Going Into Primary Care. MedPage Today. https://www.medpagetoday.com/garyschwitzer/38533. Published May 1, 2013. Accessed July 1, 2019.
- West CP, Dupras DM. General medicine vs subspecialty career plans among internal medicine residents. JAMA. 2012;308(21):2241-2247. https://doi.org/10.1001/jama.2012.47535
- Hauer KE, Durning SJ, Kernan WN, et al. Factors associated with medical students’ career choices regarding internal medicine. JAMA. 2008;300(10):1154-1164. https://doi.org/10.1001/jama.300.10.1154
- Harris MC, Marx J, Gallagher PR, Ludwig S. General vs subspecialty pediatrics: factors leading to residents’ career decisions over a 12-year period. Arch Pediatr Adolesc Med. 2005;159(3):212-216. https://doi.org/10.1001/archpedi.159.3.212
- Coutinho AJ, Cochrane A, Stelter K, Phillips RL Jr, Peterson LE. Comparison of intended scope of practice for family medicine residents with reported scope of practice among practicing family physicians. JAMA. 2015;314(22):2364-2372. https://doi.org/10.1001/jama.2015.13734
- Basu S, Berkowitz SA, Phillips RL, Bitton A, Landon BE, Phillips RS. Association of primary care physician supply with population mortality in the United States, 2005-2015. JAMA Intern Med. 2019;179(4):506-514. https://doi.org/10.1001/jamainternmed.2018.7624
- Daly MR, Mellor JM, Millones M. Do Avoidable hospitalization rates among older adults differ by geographic access to primary care physicians? Health Serv Res. 2018;53(S1)(suppl 1):3245-3264. https://doi.org/10.1111/1475-6773.12736
- Starfield B, Shi L, Macinko J. Contribution of primary care to health systems and health. Milbank Q. 2005;83(3):457-502. https://doi.org/10.1111/j.1468-0009.2005.00409.x
- Health Resources and Services Administration. Council on Graduate Medical Education Twentieth Report: Advancing Primary Care. Rockville, MD: HRSA; 2010.
- Dalen JE, Ryan KJ, Alpert JS. Where have the generalists gone? They became specialists, then subspecialists. Am J Med. 2017;130(7):766-768. https://doi.org/10.1016/j.amjmed.2017.01.026
- Mullan F, Chen C, Petterson S, Kolsky G, Spagnola M. The social mission of medical education: ranking the schools. Ann Intern Med. 2010;152(12):804-811.https://doi.org/10.7326/0003-4819-152-12-201006150-00009
- World Health Organization. Primary health care. World Health Organization. https://www.who.int/news-room/fact-sheets/detail/primary-health-care. Published February 27, 2019. Accessed August 13, 2019.
- Institute of Medicine. Primary Care: America’s Health in a New Era. Washington, DC: The National Academies Press; 1996.
- IHS Markit Ltd. The Complexities of Physician Supply and Demand: Projections from 2017 to 2032. Washington, DC: Association of American Medical Colleges; 2019.
- Kozakowski SM, Travis A, Bentley A, Fetter G Jr. Entry of US medical school graduates into family medicine residencies: 2015-2016. Fam Med. 2016;48(9):688-695.
- Ballejos M. Combined baccalaureate/medical degree students match into family medicine residencies more than similar peers. Fam Med. 2019;51(10):854-857. https://doi.org/10.22454/FamMed.2019.110812
- Malouin RA, Keenum A. Contribution of osteopathic training to the primary care workforce. Fam Med. 2017;49(4):311.
- Phillips JP, Wendling A, Bentley A, Marsee R, Morley CP. Trends in US medical school contributions to the family physician workforce: 2018 update from the American Academy of Family Physicians. Fam Med. 2019;51(3):241-250. https://doi.org/10.22454/FamMed.2019.395617
- Deutchman M, Priester T, Wills S. Who REALLY goes into primary care? Presented at the February 24, 2016 Rural Medical Educator’s Meeting of the National Rural Health Association.
- Crawford RL, McCormack RC. Reasons physicians leave primary practice. J Med Educ. 1971;46(4):263-268.
- Shanafelt TD, Boone S, Tan L, et al. Burnout and satisfaction with work-life balance among US physicians relative to the general US population. Arch Intern Med. 2012;172(18):1377-1385. https://doi.org/10.1001/archinternmed.2012.3199
There are no comments for this article.