Variation in Family Medicine Resident Reports of Burnout by Survey Context

Family physicians are the backbone of primary care in the United States.^1,2 Given the importance of maintaining the pipeline of family physicians, increased rates of burnout and staffing shortages in primary care³ have drawn attention to the magnitude of resident family physician burnout as a public health concern. The COVID-19 pandemic and the resultant stress on the health care workforce prompted further investigation of and concern regarding resident physician burnout.^4-6

Recent estimates of resident burnout in family medicine have ranged from 33%⁵ to 36%.^7,8 Characteristics associated with resident burnout include higher levels of debt, domestic medical education, identifying as female, identifying as White non-Hispanic, and larger residency class size.⁷ A resident’s odds of burnout are not attributable to their residency program, indicating that resident burnout is not a problem of low-performing programs; rather, it is a system-wide issue.⁷ Burned out family medicine residents are more likely to fail to meet milestones for professional conduct and accountability,⁸ to report less empathy,⁹ and are less likely to plan to practice primary care after graduation.⁷ Other specialties have assessed resident burnout using a survey administered after their In-Training Exam (ITE) due to ease of administration and minimal selection bias. For example, in internal medicine, higher rates of burnout were found among domestic medical graduates and residents with higher debt.¹⁰ In surgery, two studies found that mistreatment was frequent, more common among women, and associated with burnout,¹¹ and that unionization was not correlated with a decrease in burnout.¹² Among 62 orthopedic surgery residents at one institution, burnout was associated with poor exam performance;¹³ however among 1385 vascular surgery residents, no relationship was found between burnout and exam score.¹⁴

Examinations are stressful events. Even though ITEs are intended to be formative, they require prolonged focus in an exam setting and are often used by residents and programs to gauge performance. Some specialties make promotion decisions and fellowship selections based on ITE performance.¹⁵ Stress management interventions such as mindfulness have been found to reduce reports of burnout^16,17 without addressing the underlying systems issues that contribute to burnout, which begs the question whether the reverse may hold true; brief superficial stressors, such as an ITE, may increase reports of burnout.

The majority of research on occupational burnout uses a version of the Maslach Burnout Inventory (MBI).¹⁸ The MBI has demonstrated test-retest reliability in a variety of groups. Among graduate students and health care administrators, test-retest reliability has been demonstrated at 0.82 for emotional exhaustion and 0.60 for depersonalization over an interval of 2 to 4 weeks.¹⁹ In a study of physical therapy students, emotional exhaustion and depersonalization demonstrated test-retest reliability with a reliability coefficient of 0.850 and 0.907, respectively.²⁰ However, these assessments occurred in similar contexts and did not assess whether measures of burnout are reliable after brief stressors.

In 2023, the American Board of Family Medicine (ABFM) began administering a National Resident Survey (NRS) immediately after its ITE in October. For final-year residents, the registration period for the ABFM Initial Certification Examination is open from October to February, which includes an Initial Certification Questionnaire (ICQ). Because both instruments included the same questions about burnout,²¹ this unique overlap in timing created a natural experiment to assess the potential bias in burnout reporting by survey context. We hypothesized that reports of burnout would be higher after the ITE than at other times. Therefore, our objective was to assess whether situational context, such as just having completed an exam, altered residents’ self-reports of burnout measured just days or weeks apart. We were interested in this question because significant differences in reports of burnout among the same population measured in overlapping time periods would indicate that timing of survey administration should be considered when interpreting resident reports of burnout.

Our analysis was done in 2025. We used data from multiple sources. First, we obtained basic demographic information, examination scores, and residency training data from ABFM administrative databases. Second, we used data from the 2025 ABFM ICQ, which is a business requirement with a 100% response rate.²² The ICQ asks about residency training, future practice plans, and wellness. Third, we used data from the 2024 ABFM NRS, which is offered to all family medicine residents after the ITE.

Our primary outcome was burnout. We used two items that measure the emotional exhaustion and depersonalization domains of the MBI. In line with other work in the field, we defined symptoms of burnout as responding “once a week or more” or more frequently to either question.^23,24 The ABFM ITE consists of 200 multiple-choice questions administered over 4 hours in a proctored residency setting.²⁵ ITE scores are reported from 200 to 800, and residents are given a detailed score report with subscores by content area and a comparison to the national mean for their training year. Additionally, the ABFM has a Bayesian Score Predictor²⁶ that programs can use, along with the ITE score, to predict eventual success on the ABFM Initial Certification Examination. To capture whether worry about low examination performance contributed to burnout, we created a variable indicating whether the resident was in the bottom quartile of examination performance the year prior to the current ITE. For 2023 this was a score of 350, which correlates with a 14% chance of failure on the Initial Certification Examination using the Bayesian Score Predictor. To capture current exam performance, we created a variable indicating whether the resident was in the bottom quartile of examination performance for the ITE taken the same day as the NRS. For our data, this was a score of 420. To model the potential temporal spillover of stress from the ITE, we created variables to capture time between the ITE (NRS) and completing the ICQ as within 1 week of the ITE, 1 to 3 weeks after the ITE, and more than 3 weeks after the ITE.

Resident characteristics were reflected by self-reported gender and race/ethnicity. For multivariate analysis, gender was used in the binary due to low sample size. Race/ethnicity was used to characterize residents as underrepresented in medicine (URiM) if they self-identified as Black/African American, Hispanic/Latinx, Native Hawaiian or Pacific Islander, Alaskan Native/American Indian, Filipino or Vietnamese, or multiple race/ethnicity including at least one of these, which is consistent with the Association of American Medical Colleges definition.²⁷ Medical training was characterized as domestic allopathic (MD), osteopathic (DO), or international medical graduate (IMG).

We limited our sample to residents who registered for the spring ABFM Initial Certification Examination and thus had ICQ data that overlapped with the NRS. We began by graphing the change in burnout between the two instruments by weekly increments. Then we conducted bivariate testing of differences in resident characteristics by burnout change state (burnout; burnout, no burnout; burnout; etc.) using χ² testing. P values were adjusted for multiple comparisons via the false discovery method. To further evaluate the residents who reported burnout on the NRS but not the ICQ, we first restricted our sample to those who reported burnout on the NRS, then conducted multivariate logistic regression to identify characteristics that predict not being burned out on the ICQ, including time between NRS and ICQ and ITE performance, controlling for clustering by residency. We then performed multivariate analysis to calculate the odds of burnout on the ICQ among those reporting burnout on the NRS by respondent characteristics. This study was approved as exempt by the American Academy of Family Physicians Institutional Review Board. All analyses were conducted with SAS version 9.4 (SAS Institute).

The NRS response rate for final year residents was 99.9%. We excluded 420 residents without overlapping ICQ data. We excluded an additional 379 residents for missing relevant data, ending with 3,990 residents in our analytic sample. Burnout reported on the NRS (post-ITE) was higher than on the ICQ (ABFM certification exam registration; 49.9% vs 30.4%, χ², P<0.0001). Of the 1,990 residents who reported burnout on the NRS, 897 (45.1%) did not report burnout on the ICQ (Table 1). In contrast, 119 (6.0%) of the 2,000 respondents who did not report burnout on the NRS reported burnout on the ICQ. In bivariate analysis restricted to residents who reported burnout on the NRS, those not reporting burnout on the ICQ were more likely to be male, IMGs, and to have scored in the bottom quartile for their last ITE (Table 2). Residents who reported burnout on the NRS were more likely to report burnout on the ICQ in the week preceding, the week of, and the week following the ITE compared to 2 to 13 weeks after the ITE (Figure 1).

In multivariate analysis, resident characteristics positively associated with reporting burnout on the NRS (post-ITE) but not the ICQ (ABFM certification exam registration) were being a DO (aOR 1.4, CI 1.1–1.8) or IMG (aOR 2.2, CI 1.9–3.2) compared to US MDs, and registering for the certification exam 1 to 3 weeks after the ITE (aOR 2.4, CI 1.8–3.1) and more than 3 weeks after the ITE (aOR 2.6, CI 2.1–3.2) compared to registering within 1 week of the ITE (Table 3). Contrary to the bivariate results, in multivariate analysis of residents who reported burnout on the NRS, neither prior nor current ITE performance was associated with not reporting burnout on the ICQ.

Burnout estimates were highly sensitive to survey context, even when measured among the same residents during overlapping time periods. Residents had significantly higher odds of reporting burnout when surveyed after an ITE than during certification examination registration. Nearly one-quarter of residents reported burnout after the ITE but not during certification registration, whereas relatively few showed the opposite pattern. These findings suggest that reports of burnout are sensitive to survey context and that estimates obtained immediately after an examination may not be directly comparable to estimates obtained in other settings.

The significant difference between reported burnout when registering for the ABFM Initial Certification Examination versus when completing the NRS after taking the ITE suggests that burnout measures may be sensitive to survey context, specifically when the survey is paired with a stressor like the ITE. The significantly higher rates of reported burnout among the same population of residents when surveyed twice in an overlapping time period on the NRS (post-ITE) compared to the ICQ (ABFM certification exam registration) suggest that taking the ITE may prime respondents to report more burnout. ITEs offer a valuable opportunity to systematically survey residents with minimal selection bias, and our results should not be interpreted to discredit surveying residents in this way. Rather, our results demonstrate that residents may be primed unintentionally by the circumstances of survey administration to report burnout, and this should be considered in interpreting survey results.

Furthermore, multivariate analysis showed that, among those who reported burnout on the NRS, DOs and IMGs had higher odds of not reporting burnout on the ICQ compared to domestically trained MDs. This finding raises the concern that administering a burnout survey with an ITE may overestimate burnout in DOs and IMGs relative to other respondents.

In bivariate analysis (Table 2), among residents who reported burnout post-ITE, ITE performance was associated with likelihood of reporting burnout on the ICQ. However, on multivariate regression analysis (Table 3), this effect did not persist, suggesting that poorer examination performance was not independently associated with differences in burnout reporting after accounting for demographic and training-related factors.

While our findings raise questions about the situational sensitivity of burnout measures, they do not discredit prior research on burnout using ICQ data. Because the ICQ can be completed at any time during the registration window, variation in the circumstances of survey completion may reflect real-world differences in the lived experience of burnout across groups. If certain demographic groups or residents at specific programs are more likely to complete the survey during moments of heightened stress, this may itself reflect meaningful differences in their overall burden of burnout. For example, prior research has suggested that women experience disproportionate responsibilities at home, even when working full time.²⁸ Therefore, female residents may be more likely to respond to the ICQ after completing a household or childcare tasks because they spend more of their time doing these things. However, this is in fact what the burnout measure is intending to capture—genuine differences in the frequency or chronicity of burnout symptoms—and not a measurement or confounding bias.

Our study was subject to limitations. First, our measure of burnout was highly correlated with the MBI but is not able to capture the severity of burnout. Moreover, the MBI test-retest reliability among medical trainees has not been demonstrated. Second, we were unable to control for personality traits, stressors, and life circumstances that may confound our findings. For example, residents who feel especially vulnerable may feel unsafe reporting burnout. Other personality, life experiences, and cultural factors also may influence the way a resident understands and would describe their own experience of burnout and respond to the survey questions. Third, we limited our sample to residents who registered for the spring ABFM Initial Certification Examination because those residents had ICQ data that overlapped with the NRS. This included the majority of residents but excluded a minority (89 residents) who registered for the November certification exam. Those residents may have delayed graduation due to leave, program transfer, or academic difficulty, which may affect the way their reports of burnout would be situationally influenced in a way we did not measure.

Future research should investigate whether other high-stress or emotionally salient investigatory contexts similarly influence burnout reporting. Researchers relying on these measures should consider the timing and context of survey delivery when interpreting trends or evaluating interventions.

Dr Doe was supported by the American Board of Family Medicine Foundation.

An earlier draft of this work was presented at the North American Primary Care Research Group Annual Meeting, November 2025, Atlanta, Georgia. As part of that submission process, an earlier draft of the abstract was published in a supplemental meeting report in the Annals of Family Medicine.