Making Sense of Family Medicine Resident Wellness Curricula: A Delphi Study of Content Experts

Lauren Penwell-Waines, PhD | Christine Runyan, PhD, ABPP | Irina Kolobova, PhD | Aaron Grace, PsyD | Julie Brennan, PhD, RD, LD | Katherine Buck, PhD, LMFT | Valerie Ross, MS | Jill Schneiderhan, MD

Fam Med. 2019;51(8):670-676.

DOI: 10.22454/FamMed.2019.899425


Background and Objectives: The Association of Family Medicine Residency Directors (AFMRD) Physician Wellness Task Force released a comprehensive Well-Being Action Plan as a guide to help programs create a culture of wellness. The plan, however, does not offer a recommendation as to which elements may be most important, least resource intensive, or most feasible. This study sought to identify the most essential components of the AFMRD’s Well-Being Action Plan, as rated by expert panelists using a modified Delphi technique.

Methods: Sixty-eight selected experts were asked to participate; after three rounds of surveys, the final sample included 27 participants (7% residents, 38% MD faculty, 54% behavioral science faculty).

Results: Fourteen elements were rated as essential by at least 80% of the participants. These components included interventions at both the system and individual level. Of those elements ranked in the top five by a majority of the panel, all but one do not mention specific curricular content, but rather discusses the nature of a wellness curriculum.

Conclusions: The expert consensus was that an essential curriculum should begin early, be longitudinal, identify a champion, and provide support for self-disclosure of struggles.

Medical residency is a high-stress period characterized by long hours, increased responsibilities, lack of control, and fatigue. Such stressors can predispose residents to decreased self-care behaviors and well-being and an elevated risk of distress and burnout.1-5 Family medicine has among the highest rates of burnout when compared to all specialties. 6 Burnout is associated with myriad negative consequences including higher attrition, decreased productivity, unprofessional behaviors, lower empathy, difficulty concentrating, lower medical knowledge, suicidal ideation, alcohol abuse or dependence, relationship stress, and motor vehicle incidents.7

The increase of burnout among physicians in training and in practicing physicians is a threat not only to individual physicians but also to the health care system. From national organizations to individual residency program faculty, substantial efforts are being made to improve the well-being of family medicine residents. Evidence has demonstrated that a multidimensional approach addressing personal, system, and organizational factors is most helpful to mitigate the elements that contribute to burnout.7,8

From a systems level, the Accreditation Council of Graduate Medical Education (ACGME) initially attempted to improve resident well-being and decrease burnout through duty hour restrictions.9 This change has yielded mixed results, with some studies citing improvement in the rate of physician burnout but others indicating negative educational consequences (eg, lower attendance at didactic sessions, lower perceived quality of education).10-12 In short, limiting clinical and educational work hours is not a complete solution to improving physician well-being. Most recently, ACGME took another step by revising the Common Program Requirements in 2017 to emphasize the importance of promoting well-being in residency curricula and mandating procedures and programming to support well-being.9 It is now expected that programs and institutions address the system factors that contribute to burnout and teach well-being as a competency.

Emphasizing well-being in residency is especially important as work habits and patterns surrounding self-care begin to solidify during this time. Moreover, there are concrete ways that resilience can be taught and enhanced.13-15 Residency faculty have been developing and implementing programs to combat burnout and fatigue and to promote well-being for many years. In the past few years, conference presentations and articles discussing resident burnout and well-being have proliferated, but few methodologically rigorous published trials of residency-based wellness curricula exist. Most programs have focused on teaching skills including mindfulness and stress management in small groups.10,16 Many of these programs also emphasize the importance of cultivating a culture of wellness.17-19 Some programs are now linking these initiatives to resident evaluation and/or milestone attainment as well as programmatic evaluation on the efficacy of such efforts to reduce burnout. Initial evidence suggests that these programs are well-received by residents, can reduce burnout, and can be part of the solution.10,16-18 However, it is also clear that individual residency programs that develop and deliver these interventions are highly variable, and there are many mediating and moderating variables in these findings. Different definitions* and ways to measure the constructs of burnout and well-being make it difficult to reach conclusions about what is most effective.20,21

Given the importance of supporting resident physicians’ well-being, the Association of Family Medicine Residency Directors (AFMRD) Physician Wellness Task Force released its Well-Being Action Plan22 to help programs create cultures of wellness. The action plan was developed using a review of literature on existing wellness programs in family medicine residencies. The AFMRD action plan offers an extensive list of suggested wellness program elements for residency programs to implement; however, the list does not offer a recommendation as to which elements may be most important, least resource intensive, or most feasible given certain program restrictions. In other words, it is an excellent summary, but it is largely aspirational, and many programs are seeking more specific guidance about which elements are likely to have the biggest impact on resident well-being. In light of this, a Task Force on Resident Wellness Curriculum was formed through the Society of Teachers of Family Medicine (STFM). The aim of this task force was to provide an expert-informed set of guidelines on core elements of a curriculum addressing resident well-being and burnout prevention. To this end, the group conducted a Delphi study with wellness experts in family medicine education to refine and identify the most essential elements from the AFMRD Well-being Action Plan.


A modified Delphi method23,24 was used to develop consensus regarding the most essential elements from the AFMRD Well-being Action Plan.22 The Delphi method was selected for its ability to gather feedback from experts anonymously and without influence from group thinking. Three rounds of online surveys were completed via SurveyMonkey between December 2017 and August 2018. An overview of the participants and procedures is provided in Figures 1 and 2.


Seventy-six potential expert panelists were identified by membership in the STFM Professional Well-being Collaborative, scholarly record of national presentations or publications on family medicine resident wellness, and referral from known experts on family medicine residency wellness. Despite also having expertise in resident well-being and wellness programming, no members of the task force were included as experts. Sixty-eight individuals were contacted by email and asked to participate if they met the following inclusion criteria: family medicine educators in accredited residency programs who have expertise in implementing resident wellness programs and/or education for at least 2 years or recent chief residents who had a role in leading resident wellness (eg, education, administration). Twenty-seven participants (40% of those initially contacted) provided responses to all three surveys; of these, 7% were residents, 38% were physician faculty, and 54% were behavioral science faculty. No additional demographic information was collected. The composition of the final sample is not significantly different from the role composition of the first-round sample.


In the first of three online survey rounds, investigators asked participants to indicate the importance of elements included in the AFMRD Well-being Action Plan. The survey included 19 items to be rated on a 5-point scale. The response options were “essential,” “very important,” “somewhat important,” “less important,” or “not important.” Participants were given an opportunity to comment on each element (eg, specific activities that are recommended, barriers to implementing that element). The first-round survey also included open-ended questions for participants to provide input regarding additional elements that should be included in residency wellness programs. Items were retained for the subsequent round of surveys if they were rated as very important or essential by 80% or more of respondents.24 Thematic analysis was applied to the feedback gathered from the comments and open-ended questions. Themes were initially identified by one researcher and then confirmed or modified by the research team. These themes were then developed into statements for the next round of the survey. Statements served as either clarification of a previously presented element or were written as new elements.

In the round two survey, participants were asked to rate the newly formed items using the same 5-point scale noted previously. The same process of using 80% criteria as very important or essential to retain items was again employed.

For the third round of the survey, the elements retained from rounds one and two were presented back to participants for their final input regarding whether the item was essential or not. Then each participant was asked to rank the top five most important items out of the elements they had marked as essential. Ranks were weighted as follows: five points for each time the element was ranked as a participant’s first/most essential element, four points for each time the element was ranked as a participant’s second, three points for each time the element was ranked as a participant’s third, two points for each time the element was ranked as a participant’s fourth, and one point for each time the element was ranked as a participant’s fifth.

The University of Massachusetts Institutional Review Board reviewed and approved the study (IRB Docket #H00014101); it was conducted in the United States.


Round One Survey

The average rating of each element and the percentage of participants who rated that element as very important or essential in the round one survey is presented in Table 1. Of the 19 elements included in the round one survey, nine items were rated as essential or very important by at least 80% of participants. Three items were rated as nonessential and were eliminated from subsequent rounds. An additional seven items had mixed ratings with some confusion noted in the comments sections and thus were reworded for inclusion in the round two survey. Five additional items were identified through analysis of the qualitative data gathered from the comments and open-ended questions and were added to the round two survey for participants to score.

Round Two Survey

The average rating of each element and the percentage of participants that rated that element as very important or essential for each item included in the round two survey is featured in Table 2. Of the 12 elements included in the round two survey, seven identified were rated as essential or very important by at least 80% of the participants and were retained for the round three survey. The remaining five items were eliminated.

Round Three Survey

Of the 16 elements presented in round three, 14 elements were rated as essential by at least 80% of the participants (Table 3). The five items with the highest ratings are marked with an asterisk in Table 3.


Over recent years there has been increased interest in physician well-being and an accompanying effort to develop residency wellness curricula. However, little is known about best practices for these curricula. Although the AFMRD Well-being Action Plan provided a list of multiple components of a successful wellness curriculum, implementation of all of these components is not feasible in most programs. Indeed, the size, setting, and makeup of residencies vary widely, along with an individual program’s ability to supply funding and staff support for wellness programing. As such, faculty interested in establishing or improving their residency’s wellness curriculum with limited resources or expertise would have little direction about which components are potentially most likely to produce positive outcomes in resident well-being. This study sought to identify these most essential components as rated by expert panelists by using a modified Delphi technique.

Fourteen components were rated as essential by 80% of the expert panel across three surveys (Table 3). These components varied widely and included interventions at both the system level (eg, optimize work schedules to promote wellness, such as mechanisms to attend medical visits) and individual level (faculty model wellness behaviors). This is consistent with multiple recent systematic reviews25,26 that have found both levels of intervention to be vital for impacting physician well-being. Due to the wide range of programmatic and curricular changes represented by these items, most programs, even those with relatively robust wellness curricula, will likely be able to identify one or more changes to implement.

Of the 14 components rated essential by 80% of the expert panel, five items were ranked highly by a majority of the panel. These included making wellness part of the residency vocabulary early and maintaining conversations about wellness throughout residency; creating a safe culture that allows for confidential disclosure of distress and impairment; providing access to mental health services; implementing a longitudinal wellness curriculum; and identifying one or more wellness champions with clear support from leadership. What is notable about these items is that with one exception (access to mental health services), they do not mention specific curricular content, but rather discuss the nature of a wellness curriculum.

Also notable is what the expert panel found less essential. In the first round of surveys, the lowest performing items pertained to providing physical wellness space, academic support, and gathering anonymous feedback. In round two, the lowest performing items pertained to having a wellness committee, measuring resident wellness through survey instruments, optimizing time spent with patients, focusing on high-functioning care teams, and including structured reflection activities. This suggests that perhaps the ways in which well-being is carried out from program to program (eg, use of space, measurement) matter less than support for ideas and programmatic buy-in. Another possible interpretation is that although in some highly resourced organizations an emphasis on optimization of time spent with patients has produced measurable results, those results might be harder to duplicate across all settings, leaving some experts to focus more on attainable items. As researchers, we were disappointed to see neither “measure wellness through quantitative methods” nor “engage in systematic evaluation and quality improvement,” reach the final list of essential items, particularly given that the dearth of objective outcome data is one of the problems currently plaguing the field of medical education wellness research. However, review of the qualitative comments revealed that panelists did not find these items unimportant but rather less vital than other aspects of the curriculum, cited a lack of standardized definitions and measurement tools,27 and were sensitive to survey fatigue among their learners. As with other screening measures, it is also important to consider the systems that are in place at residency institutions that would support an adequate response to surveys indicating a high level of resident burnout.

Taken together, these results may indicate that some amount of flexibility is allowable in a robust wellness curriculum, provided it has the core, underlying components described herein. Inexperienced faculty or programs that are newly establishing a wellness curriculum may wish to focus first on the areas rated most highly by our expert panel.

This study was limited by the reliance on subjective methods. The method for recruiting expert panelists could have resulted in failing to identify some individuals, as it relied on membership in one society, publication, or being professionally known by others. Another limitation is that the study design relied on panelists to rate the perceived importance of items but did not require that they support their ratings with evidence. This means that the resulting list of most essential items is based on expert opinion, not objective outcome data. However, these limitations are tempered by the fact that our expert panel is from a national sample of programs, each participant was blinded to the responses of the other participants at each stage, and we used objective criteria for item selection at each round. Another limitation was the small size of our expert panel and the attrition of our panel across the three rounds. However, samples of this size have been shown to produce reliable and valid results when used in Delphi method studies.28

This study fills a gap in the existing literature in that it presents recommendations of national experts of the most essential components of a wellness program. The expert consensus was that a robust wellness curriculum would be one that begins early, is longitudinal, identifies a champion, and provides support for self-disclosure of struggles. Faculty play an important role in promoting resident well-being in their advising roles and by modeling wellness behaviors themselves. With these results, residency programs, faculty, and residents interested in developing or improving their wellness curricula have some direction for where to focus their limited energy and resources. Equipped with this information, they can move tomorrow’s physician workforce toward greater wellness.


*Extant literature uses the terms of well-being, wellness, burnout prevention, and resilience virtually interchangeably. However, we distinguish well-being as an outcome and use wellness to refer to elements of a culture or curriculum.


The authors acknowledge Amy Romain, LMSW, ACSW, and Mary Talen, PhD, for their contributions to the conceptualization of the Delphi study. The authors thank Steven Brown, MD, FAAFP, for providing consultation on the work of the AFMRD Wellness Task Force.

Financial Support: This project was supported by special project funding from the STFM Foundation.

Presentations: Results from the study have been accepted for presentation at the STFM 52nd Annual Spring Meeting in Toronto, ON, CA.


  1. Dyrbye LN, West CP, Satele D, et al. Burnout among U.S. medical students, residents, and early career physicians relative to the general U.S. population. Acad Med. 2014;89(3):443-451. https://doi.org/10.1097/ACM.0000000000000134
  2. Ishak WW, Lederer S, Mandili C, et al. Burnout during residency training: a literature review. J Grad Med Educ. 2009;1(2):236-242. https://doi.org/10.4300/JGME-D-09-00054.1
  3. Thomas NK. Resident burnout. JAMA. 2004;292(23):2880-2889. https://doi.org/10.1001/jama.292.23.2880
  4. Lebensohn P, Dodds S, Benn R, et al. Resident wellness behaviors: relationship to stress, depression, and burnout. Fam Med. 2013;45(8):541-549.
  5. West CP, Dyrbye LN, Shanafelt TD. Physician burnout: contributors, consequences and solutions. J Intern Med. 2018;283(6):516-529. https://doi.org/10.1111/joim.12752
  6. Shanafelt TD, Boone S, Tan L, et al. Burnout and satisfaction with work-life balance among US physicians relative to the general US population. Arch Intern Med. 2012;172(18):1377-1385. https://doi.org/10.1001/archinternmed.2012.3199
  7. Dyrbye L, Shanafelt T. A narrative review on burnout experienced by medical students and residents. Med Educ. 2016;50(1):132-149. https://doi.org/10.1111/medu.12927
  8. Lee J, Brown J, Stewart M. Exploring family physician stress. Can Fam Physician 2009;55(3):288-289.e1-6.
  9. Accreditation Council for Graduate Medical Education. Common Program Requirements. https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/CPRs_2017-07-01.pdf. Accessed April 17, 2019.
  10. Busireddy KR, Miller JA, Ellison K, Ren V, Qayyum R, Panda M. Efficacy of Interventions to Reduce Resident Physician Burnout: A Systematic Review. J Grad Med Educ. 2017;9(3):294-301. https://doi.org/10.4300/JGME-D-16-00372.1
  11. Ripp JA, Bellini L, Fallar R, Bazari H, Katz JT, Korenstein D. The impact of duty hours restrictions on job burnout in internal medicine residents: a three-institution comparison study. Acad Med. 2015;90(4):494-499. https://doi.org/10.1097/ACM.0000000000000641
  12. Bolster L, Rourke L. The effect of restricting residents’ duty hours on patient safety, resident well-being and resident education: an updated systematic review. J Grad Med Educ. 2015;7(3):349-363. https://doi.org/10.4300/JGME-D-14-00612.1
  13. Howe A, Smajdor A, Stöckl A. Towards an understanding of resilience and its relevance to medical training. Med Educ. 2012;46(4):349-356. https://doi.org/10.1111/j.1365-2923.2011.04188.x
  14. Epstein RM, Krasner MS. Physician resilience: what it means, why it matters, and how to promote it. Acad Med. 2013;88(3):301-303. https://doi.org/10.1097/ACM.0b013e318280cff0
  15. Jensen PM, Trollope-Kumar K, Waters H, Everson J. Building physician resilience. Can Fam Physician. 2008;54(5):722-729.
  16. West CP, Dyrbye LN, Erwin PJ, Shanafelt TD. Interventions to prevent and reduce physician burnout: a systematic review and meta-analysis. Lancet. 2016;388(10057):2272-2281. https://doi.org/10.1016/S0140-6736(16)31279-X
  17. Runyan C, Savageau JA, Potts S, Weinreb L. Impact of a family medicine resident wellness curriculum: a feasibility study. Med Educ Online. 2016;21(1):30648. https://doi.org/10.3402/meo.v21.30648
  18. Brennan J, McGrady A. Designing and implementing a resiliency program for family medicine residents. Int J Psychiatry Med. 2015;50(1):104-114. https://doi.org/10.1177/0091217415592369
  19. Place S, Talen M. Creating a culture of wellness: conversations, curriculum, concrete resources, and control. Int J Psychiatry Med. 2013;45(4):333-344. https://doi.org/10.2190/PM.45.4.d
  20. Kalani SD, Azadfallah P, Oreyzi H, Adibi P. Interventions for physician burnout: a systematic review of systematic reviews. Int J Prev Med. 2018;9(1):81. https://doi.org/10.4103/ijpvm.IJPVM_255_18
  21. Raj KS. Well-being in residency: a systematic review. J Grad Med Educ. 2016;8(5):674-684. https://doi.org/10.4300/JGME-D-15-00764.1
  22. Association of Family Medicine Residency Directors. Well-being action plan for family medicine residencies: creating a culture of wellness. April 2017. Available to members at www.afmrd.org.
  23. Dalkey NC, Helmer O. An experimental application of the Delphi method to the use of experts. Publication RM-727-PR. Santa Monica, CA: RAND, 1962.
  24. Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs. 2000;32(4):1008-1015.
  25. West CP, Dyrbye LN, Erwin PJ, Shanafelt TD. Interventions to prevent and reduce physician burnout: a systematic review and meta-analysis. Lancet. 2016;388(10057):2272-2281. https://doi.org/10.1016/S0140-6736(16)31279-X
  26. Panagioti M, Panagopoulou E, Bower P, et al. Controlled interventions to reduce burnout in physicians: A systematic review and meta-analysis. JAMA Intern Med. 2017;177(2):195-205. https://doi.org/10.1001/jamainternmed.2016.7674
  27. Eckleberry-Hunt J, Kirkpatrick H, Barbera T. The problems with burnout research. Acad Med. 2018;93(3):367-370. https://doi.org/10.1097/ACM.0000000000001890
  28. Akins RB, Tolson H, Cole BR. Stability of response characteristics of a Delphi panel: application of bootstrap data expansion. BMC Med Res Methodol. 2005;5(1):37. https://doi.org/10.1186/1471-2288-5-37

Lead Author

Lauren Penwell-Waines, PhD

Affiliations: Novant Health Family Medicine Residency Program, Cornelius, NC


Christine Runyan, PhD, ABPP - Department of Family Medicine, University of Massachusetts Medical School, Worcester, MA

Irina Kolobova, PhD - Merck & Co, Inc, Kenilworth, NJ

Aaron Grace, PsyD - Waukesha Family Medicine Residency Program, Medical College of Wisconsin, Waukesha, WI

Julie Brennan, PhD, RD, LD - Department of Family Medicine, University of Toledo Medical Center, Maumee, OH

Katherine Buck, PhD, LMFT - John Peter Smith Family Medicine, Residency, Ft Worth, TX

Valerie Ross, MS - University of Washington Family Medicine Residency Program, Seattle, WA

Jill Schneiderhan, MD - Department of Family Medicine, University of Michigan Medical School, Ann Arbor, MI

Corresponding Author

Lauren Penwell-Waines, PhD

Correspondence: 19475 Old Jetton Rd, Ste 200, Cornelius, NC 28031. 704-316-1847.

Email: Lmpenwell-waines@novanthealth.org

Fetching other articles...


Loading metrics from Mendeley...

Loading the comment form...

Submitting your comment...

There are no comments for this article.

Downloads & Info