Academic Achievement, Professionalism, and Burnout in Family Medicine Residents

Physicians in training experience high levels of burnout with associated higher rates of stress, anxiety, depression, suicidal ideation, and substance abuse.1-4 Physician burnout also affects patients by predisposing doctors to make medical errors and provide poorer quality of care.5,6 Residency training can be a particularly stressful time in a physicians’ career, with prior work showing that residents at highest risk of reporting burnout are female, of non-Hispanic White ethnicity, younger, and graduates of US medical schools.1,2,7-10 Studies have also found that burnout is more prevalent in physicians who are unmarried and who experience higher levels of family stress.11,12

Recent literature has also linked physician burnout to psychological and interpersonal contributors like resilience, social support, and self-care practice. Regular self-care, healthy personal habits, higher levels of resilience, psychological flexibility, and robust social supports may insulate physicians from the effects of burnout.1,3,4,8,12-17 Within family medicine, residency programs assess their trainees on four professionalism milestones throughout training. The milestone measures include specific descriptors of behaviors and competencies that signify progression. The professionalism milestone domain includes four measures that reflect professionalization, professional conduct and accountability, humanism and cultural proficiency, and self-care.18 The relationship between these milestones ratings and burnout has not yet been assessed. However, previous research suggests that engaging in healthy habits and self care activities like exercise, spending time with family and friends, and prioritizing recreational pursuits insulates resident physicians from burnout, as does approaching work with a greater level of empathy and a humanist perspective.1,4,16 Conversely, positive correlations have been noted between burnout and both the tendency to commit medical errors and the endorsement of racially biased perspectives, suggesting that lack of professionalism attainment as defined in the milestone data may predispose residents to burnout.5,28 We thus hypothesize that residents lagging in professionalism milestone achievement will be more likely than their peers to report burnout.

Burnout in residents may also be associated with lower academic achievement due to less attention and focus on studying. Previous studies used data from in-training examinations (ITEs) and found a correlation between poor performance on ITEs and burnout in internal medicine, general surgery, and orthopedic surgery residents, although these findings were not replicated in emergency medicine residents.19-22

In this study, we examined modifiable contributors to resident family physician burnout. To address gaps in this body of knowledge, we used a large sample of graduating resident family physicians. Specifically, we investigated the relationship between burnout and milestone attainment as a proxy for professionalism and capacity for self-care. We also studied the impact of academic struggle, as measured by ITE performance, on resident burnout, hypothesizing that academic struggle and failure to attain professionalism milestones would be positively associated with burnout.

We used data from the 2019 American Board of Family Medicine (ABFM) Family Medicine Certification Examination Registration Questionnaire for initial certification candidates. The questionnaire is a mandatory component of registration and achieves a 100% response rate.23 Initial certification candidates complete the questionnaire 3-4 months prior to their examination date, which is usually in December to February of their last year of training for the April examination, or in July to September for the November examination. For initial certification candidates, the questionnaire captures information about candidates’ intended scope of practice, future practice organization and ownership (if known), fellowship intentions, and, beginning in 2019, burnout.

We obtained data on all four professionalism milestone ratings on all family medicine residents seeking to obtain ABFM certification in 2019 back to their first year of training. Milestones ratings from 0 to 5 are assigned to each resident twice per year, midyear and end of year, which are roughly December and June. The first milestone (PROF-1, or “professionalization”) measures completion of a process of professionalization. Some indicators of high performance in this milestone include “embracing the professional responsibilities of being a family physician,” and “recognizing an obligation to self-discipline and self-regulate.” The second milestone (PROF-2 or “professional conduct and accountability”) indicates professional conduct and accountability. Examples of advanced achievement in this milestone include “exhibiting self-awareness, self-management, social awareness and relationship management.” The third milestone (PROF-3, or “cultural proficiency”) measures humanism and cultural competency, demonstrated by “leadership in cultural proficiency, understanding of health disparities and social determinants of health.” The fourth (PROF-4, or “self-care”) measures ability to maintain emotional, physical, and mental health, especially when challenged.

Resident demographic information and ITE scores came from ABFM administrative databases. The ITE is administered in the fall of each year and is reported on a scaled score of 200 to 800. There is no passing score as the ITE is intended to be a formative evaluation.

We obtained first-year resident salary by review of residency databases on the American Academy of Family Physicians website. To model stress of meeting costs of living, we obtained 2019 county-level median home prices from the National Association of Realtors website and created a ratio of PGY-1 resident salary to median home prices. We geocoded residency contact addresses on file with the ABFM to the county level, and we linked geographic location of residency to Census Region. We calculated residency size from ABFM administrative data by counting the number of third-year residents and categorized as less than 6, 6 to 10, and 11 or more.

We assessed burnout using two items that correspond to the emotional exhaustion and depersonalization scales of the Maslach Burnout Inventory.24 Following past work,25-27 we defined burnout as responses to either the emotional exhaustion or depersonalization items at “once a week or more” or more frequently. We characterized whether the resident was meeting expectations on the professionalism milestones at each evaluation starting with a 4 at the end of year PGY-3 rating and removing 0.5 points for each of the preceding ratings. We calculated a distribution of ITE scores for each administration and created a variable reflecting whether the resident was in the bottom 25% of scores for their class in that year. We dichotomized age as above the 75th percentile or lower for analysis. We categorized the salary to cost of living ratio by quartile.

To create our sample, we merged examination registration data from graduating residents in 2019 with milestones, ITE scores, demographics, and residency location. Residents without a complete set of data on milestones, ITE scores, examination scores, and salary information, as well as those who transferred programs, or were in combined programs or 4-year training programs were excluded. Then we examined distributions of variables and assessed correlations of low-performing ITE scores with meeting milestones ratings. We conducted bivariate testing using t tests or χ2 tests for associations of variables with burnout. We graphed the percentage of professionalism milestone scores the resident met at each rating by whether they reported burnout at examination registration. We calculated the intraclass correlation for burnout at the residency level. Finally, we conducted multilevel logistic regression controlling for clustering at the residency level to assess associations of burnout with professionalism milestones rating at end of residency and number of ITE scores below the 25th percentile as main exposures. We additionally controlled for resident demographics, residency size and region, and ratio of salary to cost of living. We conducted all analyses in SAS v 9.2 (Cary, NC). The American Academy of Family Physician Institutional Review Board approved this study.

The 2019 ABFM Family Medicine Certification Examination Registration Questionnaire included 3,668 candidates, with 1,159 removed for exclusion criteria, leaving a final sample size of 2,509. The most common reasons for exclusion were being in a 4-year program (29.2% of excluded), lack of salary data (24.5% of excluded), and missing ITE score (18.6% of excluded; see Supplemental Table 1). DO degree holders were slightly less likely to be included, due to lack of ITE scores.

Respondents had an overall mean age of 32.7 years (Table 1). The majority of respondents held an MD degree (81.7% vs 18.3%), were female (54.4% vs 45.6%), and were US graduates (64.8% vs 35.2%). For all four of the professionalism milestones, the majority of respondents met expectations at graduation. Over half of respondents (54.4%) did not have any ITE scores in the bottom quartile. The mean first-year resident salary was $48,582. The overall prevalence of burnout among respondents was 36.8%.

In bivariate analysis, we found that females were significantly more likely to report burnout (38.8% vs 34.5%; P=.03; Table 2). This bivariate model also suggested a significant protective relationship between international medical graduate (IMG) status and burnout (29.6% of IMGs reported burnout vs 40.8% of US graduates; P<.001). Finally, there was a correlation between burnout and self-identified White race (38.3% of White respondents reported burnout vs 31.9% of Black respondents, 32.7% of Asian respondents, and 41.8% of respondents who identified as another race, P=.02).

In our bivariate model, poor performance (defined as a resident who is assessed as “below expectations”) on three of the professionalism milestones, professionalization, professional conduct and accountability, self-care) were each positively correlated with burnout (Table 3). For professionalization (41.8%), professional conduct and accountability (43.1%), and self-care (42.1%) the percentage of residents below expectations experienced burnout at a statistically significant higher rate than the overall mean of 36.8%. We found a small but statistically significant relationship between meeting expectations on professionalism milestones and PGY-3 ITE quartile (Figure 1). Over six ratings, we found few differences in milestones ratings between residents who reported burnout at the end of residency until PGY-3 ratings on professionalization, professional conduct and accountability, and self-care (Figure 2).

We first fit a null model solely controlling for clustering of residents within residencies and found an intraclass correlation of 6.9% for burnout by residency program. In multivariable multilevel models controlling for individual resident and program characteristics, there was no statistically significant association found between burnout and female gender (OR 0.84, 95% CI 0.71-1.01), degree type (OR 0.98, CI 0.78-1.26), or older age (OR 1.09, CI 0.88-1.34; Table 4). No racial category had different odds of burnout compared to the reference group (White race). There remained a statistically significant association between lower odds of burnout and being an IMG versus a US graduate (OR 0.61, CI 0.49-0.76). We found no association between residency program region or program size with odds of resident burnout. Lower odds of burnout were associated with respondents who made adjusted salaries in the 75th percentile or higher versus those who had adjusted salaries in the 25th percentile or below (OR 0.67 CI, 95% CI 0.49-0.95).

For educational assessments, we found that residents not meeting expectations for the professional conduct and accountability milestone at graduation had higher odds of burnout, (OR 1.41, 95% CI 1.07-1.86). No statistically significant associations were found between burnout and not meeting expectations on other professionalism milestones. There was no increase in the odds of burnout with increasing numbers of low performances on the ITE versus those who had no low performances.

Our study of more than 2,500 family physicians completing residency training in 2019 found a significant adjusted association between failing to meet expectations on a milestone reflecting professional conduct and accountability and self-reported burnout. There was no relationship between burnout and medical knowledge as measured by lower ITE performance. Together these findings indicate resident burnout is more indicative of attitudes of professionalism and commitments to patient care than measures of medical knowledge. Although further study is required to determine directionality, our findings may prompt program directors to consider targeting remediation efforts on professionalism as a way to promote wellness.

The family medicine milestones assess professionalism in four domains. Of these, our bivariate results found significant relationships between professionalization, professional conduct and accountability, and self-care milestones and burnout. However, only the relationship between professional conduct and accountability and burnout was robust to regression analysis controlling for other milestones, medical knowledge, and multiple resident and residency program characteristics. That neither humanism and cultural proficiency nor self-care were ultimately associated with burnout was unexpected in light of a body of literature indicating relationships between burnout and lower resilience, higher stress levels, lower psychological flexibility, and lower rates of meaning salience.1,14 Instead, a resident performing poorly on a measure of self-care was no more likely to experience burnout than a peer who was perceived as proficient in self-care. Some of this discrepancy may originate from the fact that the professionalism milestones are assessed by program leadership rather than the residents themselves, and perhaps do not reflect the resident’s own perception of competency in self-care.

Unlike self-care, perceived deficiency in professional conduct and accountability was found to be positively correlated with self-reported burnout. Contained within the professional conduct and accountability milestones definitions are proficiencies such as maintaining appropriate professional behavior without external guidance, and consistently recognizing the limits of one’s own knowledge and asking for assistance. By looking at these component parts, it is evident that the finding of correlation between failure of professional conduct and accountability and burnout may corroborate previous research that suggests a correlation between burnout and physician-reported medical error.5 Those failing to recognize the limits of their own knowledge or struggling to ask for assistance are almost definitely at higher risk of committing a medical error out of a failure to appropriately recognize their own ignorance, and both the professionalism failure and the tendency toward medical error have a positive relationship to burnout. This finding suggests potentially concerning implications for patient care and contributes to our understanding of the risks that physician burnout poses not just to the individual but to the patients and community served.

Regression analysis did not bear out our hypothesized positive relationship between lower medical knowledge and burnout, but instead found that scoring in the lowest quartile of ITE score was unrelated to burnout. Research from internal medicine, general surgery, and orthopedic surgery suggests that residents who score in the lowest ITE percentiles relative to their peers are more likely to report burnout,19-21 but a study of ITE scores among 30 emergency medicine residents in one military training program failed to demonstrate a similar correlation.22 Our study, including a large national sample of family medicine residents applying for ABFM certification in 2019, replicates the results of the emergency medicine study within a considerably larger population, suggesting that low academic performance does not predispose family medicine residents to burnout.

Our findings on the relationship between resident characteristics and burnout were largely consistent with previous literature.1,2,7-10 Our bivariate models found that female physicians were slightly more likely than their male counterparts to report burnout (39.0% vs 34.5%) and that physicians who identify as White had much higher rates of reported burnout (38.3%) than their colleagues who identify as Black (31.9%), Asian (32.7%), or of another race (41.8%) but neither of these results were robust to regression analysis. We found that IMG status was associated with lower rate of reported burnout in both bivariate and adjusted models. However, it is important to note in all of these cases that experiencing burnout is not entirely synonymous with reporting burnout, and that certain demographic or socioeconomic identities may confer more or less ease with reporting professional challenges. This finding is consistent with a previous result that resident family physicians educated in the United States experience burnout at higher rates than their IMG colleagues, which may be related to reporting greater work pressure and less subjective feeling of control at work.10 Recent work also suggests that both explicit and implicit racial bias toward Black people are correlated with higher levels of burnout among resident physicians,28 a finding that merits close examination in light of the tendency of physicians from racial and ethnic majority groups to report higher levels of burnout in our study and the others referenced above.

We also found that higher salary compared to cost of living was associated with lower odds of burnout in both bivariate and regression analysis. This result suggests that financial strain may contribute to burnout in resident physicians, a possibility that merits further research given the increasing average medical school debt burden for new physicians.29

While our study’s strengths lie in its large sample size and high response rate, it is also subject to limitations. First, our assessment of burnout comes from a one-time metric collected at the end of residency and we cannot determine directionality in the relationship between academic performance and burnout. Instead of our hypothesized relationship where poor academic performance leads to burnout, it is plausible that a persistent state of burnout in residency can instead cause lower performance. Second, the burnout measures in our data represent only emotional exhaustion and depersonalization screeners, and not the full Maslach Burnout Inventory. In addition, unmeasured residency characteristics and culture could be associated with burnout in ways that were not captured in our data.30 Moreover, the professionalism milestones we used in our data set are assessed by program leadership rather than by residents themselves, and may be an inadequate proxy for resident self-assessment of self-care and well-being. Finally, our sample includes only those residents who registered for the ABFM Certification Examination and excludes those whose academic or personal struggles preclude them seeking certification. However, the take rate for the ABFM examination has been over 95%, which limits this potential bias.31

In conclusion, although ITE scores and self-care milestone attainment were not found to be significantly related to resident family physician burnout, subpar performance in professional conduct and accountability predicted higher odds of burnout and IMG status conferred some protection against burnout. Further study is needed to explain the contribution of gender-based and race-based discrimination, both that which the resident experiences and that which he or she perpetuates, and individual financial stress to the experience of burnout. Future research with this population could also incorporate program-level factors like leadership quality and availability of burnout prevention curriculum to explain the resident experience and ultimately seek to reduce burnout among family medicine residents.