Background and Objectives: We sought to develop and validate a self-assessment of foundational leadership skills for early-career physicians.
Methods: We developed a leadership self-assessment from a compilation of materials on health care leadership skills. A sequential exploratory study was conducted using qualitative and quantitative analysis for face, content, and construct validity of the self-assessment. First, two focus groups were conducted with leaders in medicine and family medicine residents, to refine the pilot self-assessment. The self-assessment pilot was then tested with family medicine residents across the country, and the results were quantitatively evaluated with principal component analysis. This data was used to reduce and group the statements into leadership domains for the final self-assessment.
Results: Twenty-two invited family medicine residency programs agreed to distribute the survey. A total of 163 family medicine residents completed the survey, representing 16 to 20 residency programs from 12 states (response rate 28.9% to 34.8%). Analysis showed important differences by residency year, with more advanced residents scoring higher. The analysis reduced the number of items from 33 on the pilot assessment to 21 on the final assessment, which the authors titled the Foundational Healthcare Leadership Self-assessment (FHLS). The 21 items were grouped into five leadership domains: accountability, collaboration, communication, team management, and self-management.
Conclusions: The FHLS is a validated 21-item self-assessment of foundational leadership skills for early career physicians. It takes less than 5 minutes to complete, and quantifies skill within five domains of foundational leadership. The FHLS is a first step in developing educational and evaluative assessments for training medical residents as clinician leaders.
Physicians in all disciplines are called upon to demonstrate leadership skills even if they do not hold a formal leadership role. Additionally, organizations led by physicians perform better than organizations with nonphysician leaders.1 For these reasons, leadership skill development must become a routine part of medical education at all levels and in all specialties.1-20
National organizations recognize the need for effective leadership training. The Interprofessional Education Collaborative defined competencies in interprofessional collaboration for a varity of medical professionals, including physicians, that include the use of leadership skills to support collaborative practice and team effectiveness.21 The Institute of Medicine’s July 2014 report Graduate Medical Education that Meets the Nation’s Health Needs recommended “production of a physician workforce better prepared to work in, help lead, and continually improve an evolving health care delivery system that can provide better individual care, better population health, and lower cost.”19 The Accreditation Council for Graduate Medical Education (ACGME) Milestones for Family Medicine specify competencies in team leadership, collaboration with public health and community agencies, advocacy, and leadership of systems and organizational strategies.22 The ACGME Milestones for Internal Medicine and Pediatrics identify competencies in team management, communication, practice improvement, and advocacy.23,24
While many innovative curricula have been developed, few target all learners, use a validated assessment, or show meaningful impact.25-30 Validated assessment is needed to demonstrate impact and compare curriculum.9,14,31-42 A systematic review of leadership training in health care teams found that defining best practices is difficult due to lack of a standard definition of leadership, supporting frameworks, and robust assessments.35
Though assessments do exist for health care administrators, practicing physicians, and surgeons’ nontechnical skills in the operating room (including leadership), there are no published assessment instruments of leadership skills for physicians in residency training programs.17-20,22,30,43-45 Knowing that leadership development covers a broad spectrum, we specifically wanted to describe the foundational skills that early-career physicians, specifically residents, need to learn to be effective team leaders. Residents routinely participate in interprofessional teams focused on clinical care and quality improvement. Theoretically though, foundational leadership skills should translate across a variety of settings.
The purpose of this study was to develop a health care leadership self-assessment of foundational competencies and validate the tool among family medicine residents.
Faculty at the University of Utah Family Medicine Residency Program developed the initial leadership self-assessment from a compilation of materials on general leadership skills, including those for health care executives.18,30,36,42-60 Our intent was to identify leadership competencies for early-career physicians, specifically resident physicians, to be effective team leaders.
To validate the self-assessment, we used a sequential exploratory study design, which utilizes “an initial phase of qualitative data collection and analysis followed by a phase of quantitative data collection and analysis.”61 The overall purpose of this approach is “to explore a phenomenon.” This strategy may also be useful when developing and testing a new instrument.61 The Institutional Review Board at the University of Utah approved this study.
To refine the self-assessment, we conducted two focus groups. The primary focus group question was: “What are the foundational leadership skills that a family medicine physician needs to be an effective leader of interprofessional teams?” The focus group participants discussed the family medicine physician’s role in interprofessional teams and described the foundational skills that family medicine physicians need to be effective leaders. Participants then completed the self-assessment, and discussed its comprehensiveness, relevance, clarity, and ease of use. Each focus group lasted approximately 1 hour, was audio recorded, and detailed notes were taken by the facilitators. This qualitative data was analyzed using open coding to identify emerging themes from the data as they linked to the assessment.
Face Validity. In March 2015, we conducted the first focus group with leaders in medicine to elicit expert opinion on the skills needed for leadership in family medicine and to identify whether the devised items captured the noted leadership skills. This cross-disciplinary group included experts in community medicine, academic medicine, family medicine residencies, and team practice. The insights from this focus group were used to guide revisions of the self-assessment.62
Content Validity. The purpose of the second focus group (June 2015) was to further refine the self-assessment with eight third-year residents from our home institution. We followed a focus group process, similar to the expert focus group, to explore resident perspective on leadership competencies. The residents reviewed the self-assessment, which had been revised based on information from the first focus group. We elicited their views on the clarity and comprehensiveness of the self-assessment, and used their responses to clarify and simplify the self-assessment items.
For the second phase of this study, we contacted 30 family medicine residency directors to distribute the pilot self-assessment to their residents. The directors were selected by purposive sample, based on geographic region and type of program (community versus university), in order to increase diversity of representation. The author (SVH) contacted the directors by email with an invitation to participate in a validation study of a leadership self-assessment. We sent an email survey link to the participating directors who were instructed to forward the survey to their entire resident cohort. Two weeks later we sent a second email to improve participation. REDCap, a secure web application, was the survey platform.63
We collected basic demographic information and asked residents to complete the 33-item self-assessment on a scale from beginner (1) to expert (5). We compared the demographics of our sample to those of US family medicine residents (as reported by the Accreditation Council for Graduate Medical Education, 2013-2014) using chi-square analysis and Fisher’s exact computation for P value.64
Construct Validity. We conducted a principal component analysis with varimax rotation, to reduce the number of assessment items, and group these items into domains using Stata (v 13.1 College Station, TX).65 We retained individual items with eigenvalues over 1.0. Factor loadings of at least 0.30 were used to group items into domains. Once the domains were established, we calculated and compared mean scores overall and for each domain across demographic variables using linear regression analysis.
Principal component analysis grouped the self-assessment statements into domains. Once the domains were established, we named the domains. Our research team completed a comprehensive review of the scholarly and academic literature to determine the vocabulary and categorization of leadership behaviors.38,45,47,66,67 We drew from our collective experience in teaching medical, public health, and social work students about leadership. We named the leadership domains to represent the statements contained within each domain.
Initially, 30 program directors were contacted to inquire whether they would be interested in participating in this validation study. Of these, 22 programs agreed to send the survey link out to their residents. Of these, 13 program directors confirmed via email that the link was sent out to residents. For the remaining programs for which no confirmation was obtained, it is not possible to determine which programs did distribute the link to their residents, as survey responses were anonymous and respondents were asked only to identify their state, but not their program. Of the programs that initially agreed but did not confirm sending the link, three programs must have sent the link, as there were responses from that state and their program was the only one contacted in the state. Together, these 16 programs are considered verified participants. The participation of four programs cannot be determined, as there were multiple programs contacted in the same state, preventing researchers from delineating whether these programs did in fact send the link. Finally, two programs likely did not send the link out to residents, as the program was the only program contacted in that state and no responses from that state were received (Figure 1). Overall, responses were received from between 16 and 20 programs with a broad geographic distribution across 12 states (West: California, Colorado, Utah, Washington; Midwest: Illinois, Michigan, Minnesota; Northeast: New York, Pennsylvania; South: Missouri, North Carolina, Texas).
The maximum number of residents potentially included in the sample across the 20 programs that most likely sent out the link was 564, while the number of residents in the 16 programs verified to have circulated the assessment was 468. We received responses from 163 residents, giving a response rate of 28.9% across all 20 programs, or 34.8% across the 16 programs with verified participation. The majority of respondents were female (64.4%) and white (65.6%). Demographics for respondents, US family medicine residents, and all US medical residents are presented in Table 1.64 We compared the demographics of our sample to all US family medicine residents, and found significant differences by sex (P=0.015), race/ethnicity (P=0.001) and residency year (P=0.000).
The principal component analysis reduced the number of items from 33 on the pilot assessment to 21 on the final assessment, which was titled the Foundational Healthcare Leadership Self-assessment (FHLS). The 21 items of the FHLS were assigned to five leadership domains: accountability, collaboration, communication, team management, and self-management (Table 2).
An average score for each domain was calculated by summing the self-assessment values (from 1=beginner to 5=expert) from the appropriate items, and then dividing by the number of items in the domain. The average scores from our survey are presented in Table 3. Regression analysis of average scores found no significant differences by sex, but did find important differences by residency year, with more advanced residents scoring higher (P<0.01). Additionally, there was suggestive evidence that nonwhite residents reported higher leadership scores than their white peers (P=0.055, Table 4). Regression analysis for each of the five domains revealed similar patterns, with residency year remaining the strongest predictor of average score (results not shown).
We created and validated the FHLS as a self-assessment of foundational leadership skills for early-career physicians. We validated this assessment using two groups—experts and users—using qualitative and quantitative analysis for face, content, and construct validity. A self-assessment can be used longitudinally as a formative tool during residency to help residents engage in personalized leadership development.32,68,69 The FHLS is the first step in the development of educational and evaluative assessments for training clinician leaders.
The FHLS was developed with a focus on foundational skills for leading interprofessional teams. During residency, residents work in and lead teams, in both formal and informal roles,7 providing clinical care in ambulatory and inpatient settings, and conducting quality improvement. We imagine these foundational skills are generalizable across different team environments.
The scores from this tool showed discernable differences by year of residency, and no gender or age differences. Scores increased by year of residency, which is an expected progression with advancing training. As residents develop more experience, they feel more skilled.
The majority of respondents to this survey were female and white. Compared to family medicine and all residents nationally, respondents to the survey were disproportionately female. However, our analysis showed no difference in response by gender. Nonwhite respondents scored higher in the total score than white respondents (P=0.055). This phenomenon should be studied further. While respondents represented both academic and community-based residency programs, we did not ask them to identify the type of program, in order to maintain anonymity. Type of program could be evaluated in future studies with the FHLS.
The FHLS is a brief self-assessment that takes less than 5 minutes to complete and quantifies skill within five domains of foundational leadership (Table 2).
The strength of our study is in the use of both qualitative and quantitative methods to validate the FHLS. In our qualitative analysis, we involved experts and the users of the tool, with input from both groups. This process provided face and content validity for the items presented in the initial tool, as we used the collective expertise of the leaders, and feedback from the intended users. Construct validity was assessed in our quantitative analysis, and resulted in a streamlined tool containing five domains. For construct validity, we had broad geographic representation. This purposive sample targeted residents in all US geographic regions and included respondents in both academic and community-based residencies.
The FHLS was initially developed using qualitative methods at a single geographic location. The initial focus group included physician leaders in various leadership positions in the region. Therefore, a bias may have occurred due to the geographic location and practices of the experts who designed and gave feedback on the tool. However, the tool was designed using published materials from a variety of locations, and the group of experts had a collectively broad set of life and leadership experiences. If there were geographic differences, we would expect to see differences in response by geography, which we did not.
To cultivate leadership skills in medical residents, residency faculty need validated instruments to measure progress and to target educational interventions.9,14,31-42 Such a tool now exists. The FHLS is intended to be a formative tool for resident leadership development within a leadership curriculum. It can inform individualized learning plans and tailored educational experiences.32,68-69 The FHLS can be used to guide residency curricular development for educational needs in a resident cohort.
Our assessment addresses the ACGME Milestone for Family Medicine competency in team leadership, but does not address the other Milestones of collaboration with public health and community agencies, advocacy, and leadership of systems and organizational strategies.21 These items were not identified as foundational in our validation process, but are present in many conceptual models, and should be considered as more advanced skills on the leadership spectrum.
Although the FHLS was validated in family medicine, we imagine the foundational leadership skills are relevant across many disciplines. Future research is needed to validate the FHLS with residents in other specialties and providers from other disciplines such as physician assistant, nursing, pharmacy, and social work.
The spectrum of leadership skills that clinicians need across their career is broad, with one end of the spectrum being foundational skills. To complete the spectrum, intermediate and advanced leadership skills need to be defined, and validated tools developed.
This work was supported by the Health Studies Fund (Department of Family and Preventive Medicine, University of Utah). The authors thank Jessica Bickley for her support in manuscript preparation.
- Goodall AH. Physician-leaders and hospital performance: is there an association? Soc Sci Med. 2011;73(4):535-539.
- Angood P. Cultivating Physician Leadership Is Necessary for Lasting and Meaningful Change. Acad Med. 2015;90(10):1290.
- Blumenthal DM, Bernard K, Bohnen J, Bohmer R. Addressing the leadership gap in medicine: residents’ need for systematic leadership development training. Acad Med. 2012;87(4):513-522.
- Ciampa EJ, Hunt AA, Dermody TS. Leadership can and should be taught. Acad Med. 2010;85(12):1814.
- Deane K, Ringdahl E. The family medicine chief resident: a national survey of leadership development. Fam Med. 2012;44(2):117-120.
- Dhaliwal G, Sehgal NL. Demystify leadership in order to cultivate it. Acad Med 2014;89(11):1441.
- Gabel S. Expanding the scope of leadership training in medicine. Acad Med. 2014;89(6):848-852.
- Goleman D. Leadership that gets results. Harvard Business Review Mar-Apr 2000;78-90.
- Jardine D, Correa R, Schultz H, et al. The need for a leadership curriculum for residents. J Grad Med Ed. Jun 2015; 307-309.
- Kotter JP. What leaders really do. Harv Bus Rev. 1990;68(3):103-111.
- Magill MK. Becoming an effective physician leader. Fam Pract Manag. 1999;6(5):35-37.
- Stoller JK. Commentary: recommendations and remaining questions for health care leadership training programs. Acad Med. 2013;88(1):12-15.
- Taylor RB. Leadership is a learned skill. Fam Pract Mgt. Oct 2003:43-48.
- Verma AA, Bohnen JD. Bridging the leadership development gap: recommendations for medical education. Acad Med. 2012;87(5):549-550.
- Mittwede PN. On leadership and service during medical training. Acad Med. 2015;90(4):399.
- Wiencke MC. Building a strong foundation. Dartmouth Medicine 2014 Winter-Spring. 22-25.
- Warren OJ, Carnall R. Medical leadership: why it’s important, what is required, and how we develop it. Postgrad Med J. 2011;87(1023):27-32.
- Healthcare Leadership Competency Model. National Center for Healthcare Leadership. http://www.nchl.org/static.asp?path=2852,3238. Accessed March 10, 2016
- Institute of Medicine. (2014). Graduate Medical Education that Meets the Nation’s Health Needs. https://www.nationalacademies.org/hmd/~/media/Files/Report%20Files/2014/GME/GME-RB.pdf. Accessed March 10, 2016.
- Snyderman CH, Eibling DE, Johnson JT. The physician as team leader: new job skills are required. Acad Med. 2011;86(11):1348.
- Interprofessional Education Collaborative. Core competencies for interprofessional collaborative practice: 2016 update. Washington, DC: Interprofessional Education Collaborative; 2016.
- Accreditation Council on Graduate Medical Education and the American Board of Family Medicine. (2013). The Family Medicine Milestone Project. http://www.acgme.org/acgmeweb/Portals/0/PDFs/Milestones/FamilyMedicineMilestones.pdf. Accessed March 10, 2016.
- Accreditation Council on Graduate Medical Education and the American Board of Internal Medicine. (2015). The Internal Medicine Milestone Project. http://www.acgme.org/acgmeweb/portals/0/pdfs/milestones/internalmedicinemilestones.pdf. Accessed March 10, 2016.
- Accreditation Council on Graduate Medical Education and the American Board of Pediatrics. (2015). The Pediatrics Milestone Project. https://www.acgme.org/acgmeweb/Portals/0/PDFs/Milestones/PediatricsMilestones.pdf. Accessed March 10, 2016
- Ackerly DC, Sangvai DG, Udayakumar K, et al. Training the next generation of physician-executives: an innovative residency pathway in management and leadership. Acad Med. 2011;86(5):575-579.
- Donnelly EF. A leadership development curriculum for radiology residency. J Grad Med Ed. Jun 2015;296-297.
- Eubank D, Geffken D, Orzano J, Ricci R. Teaching adaptive leadership to family medicine residents: what? why? how? Fam Syst Health. 2012;30(3):241-252.
- Nambudiri VE, Adler DS, Wright SC, et al. Training the physician executives of the future” the medical management leadership track at Brigham and Women’s Hospital Department of Medicine. Physician Executive J. May-Jun 2013. p96-98.
- Sheline B, Tran AN, Jackson J, Peyser B, Rogers S, Engle D. The Primary Care Leadership Track at the Duke University School of Medicine: creating change agents to improve population health. Acad Med. 2014;89(10):1370-1374.
- Accreditation Council on Graduate Medical Education and the Council of Review Committee Residents – Leadership Subcommittee. (2012) Leadership Development Curriculum for Chief Residents in Medicine. http://www.acgme.org/Portals/0/PDFs/ChiefResidents/2015-BROCHURE-FINAL-CHIEF-RES-PROGRAMS.pdf. Accessed March 10, 2016
- Buckingham M. Leadership development in the age of the algorithm. Harv Bus Rev. 2012;90(6):86-92, 94, 144.
- Flood SC. Using qualitative self-evaluation in rating physician performance. Fam Pract Manag. 1998;5(5):22-24, 27-30, 33-34.
- Gillam S. Teaching doctors in training about management and leadership. BMJ. 2011;343(sep13 3):d5672.
- Jortberg BT, Fernald DH, Dickinson LM, et al. Curriculum redesign for teaching the PCMH in Colorado Family Medicine Residency programs. Fam Med. 2014;46(1):11-18.
- Rosenman ED, Shandro JR, Ilgen JS, Harper AL, Fernandez R. Leadership training in health care action teams: a systematic review. Acad Med. 2014;89(9):1295-1306.
- Mintz LJ, Stoller JK. A systematic review of physician leadership and emotional intelligence. J Grad Med Ed. March 2014: 21-31.
- Straus SE, Soobiah C, Levinson W. The impact of leadership training programs on physicians in academic medical centers: a systematic review. Acad Med. 2013;88(5):710-723.
- Stoller JK. Developing physician-leaders: a call to action. J Gen Intern Med. 2009;24(7):876-878.
- Stoller JK. Developing physician-leaders: key competencies and available programs. J Health Adm Educ. 2008;25(4):307-328.
- Webb AM, Tsipis NE, McClellan TR, et al. A first step toward understanding best practices in leadership training in undergraduate medical education: a systematic review. Acad Med. 2014;89(11):1563-1570.
- Tsen LC, Borus JF, Nadelson CC, Seely EW, Haas A, Fuhlbrigge AL. The development, implementation, and assessment of an innovative faculty mentoring leadership program. Acad Med. 2012;87(12):1757-1761.
- Yule S, Flin R, Paterson-Brown S, Maran N. Non-technical skills for surgeons in the operating room: a review of the literature. Surgery. 2006;139(2):140-149.
- Yule S1, Flin R, Paterson-Brown S, Maran N, Rowley D. Development of a rating system for surgeons’ non-technical skills. Med Ed. 2006;40:1098-1104.
- American Academy of Family Physicians. (2008). Recommended curriculum guidelines for Family Medicine Residents: Leadership. http://www.aafp.org/dam/AAFP/documents/medical_education_residency/program_directors/Reprint292_Leadership.pdf. Accessed March 10, 2016.
- University of Aberdeen. Non-Technical Skills for Surgeons (NOTSS) Handbook, Version 1.2. June 2012.
- Dickinson WP. Leadership for change: a vital area for faculty development, continuing education, and resident education. Fam Med. 2010;42(9):659-660.
- Lingard L, Vanstone M, Durrant M, et al. Conflicting messages: examining the dynamics of leadership on interprofessional teams. Acad Med. 2012;87(12):1762-1767.
- Heifetz RA, Linsky M. Managing yourself: a survival guide for leaders. Harvard Business Review: Leadership 2002. June;65-74.
- Souba WW. The 3 essential responsibilities: a leadership story. Arch Surg. 2010;145(6):540-543.
- Stock R, Mahoney E, Carney PA. Measuring team development in clinical care settings. Fam Med. 2013;45(10):691-700.
- Useem M. Four lessons in adaptive leadership. Harv Bus Rev. 2010;88(11):86-90, 149.
- American College of Healthcare Executives (ACHE). ACHE Healthcare Executive 2012 Competencies Assessment Tool. Chicago, IL: American College of Healthcare Executives; 2011.
- Bentley JC. Leadership Development for Physician Executives Syllabus. Salt Lake City, UT: University of Utah Eccles School of Business; 2007.
- Conner DR. Managing at the Speed of Change. New York, NY: Villiard Books; 1992.
- Collins J. Good to Great. New York, NY: Harper-Collins; 2001.
- Patterson K, Grenny J, McMillan R, Switzler, A. Crucial Conversations: Tools for Talking When Stakes are High. New York, NY: McGraw-Hill; 2002.
- Studer Q. Hardwiring Excellence. Gulf Breeze, FL: Fire Starter Publishing; 2003.
- Taking You and Your Organization to the Next Level with Quint Studer @ 2005. The Studer Group, LLC.
- Sotile QM, Sotile MO. The Resilient Physician: Effective Emotional Management for Doctors and Their Medical Organizations. United States of America: American Medical Association Press; 2002.
- Whetten DA, Cameron KS. Developing Management Skills. 8th ed. Upper Saddle River, NJ: Prentice Hall; 2011.
- Creswell JW, Plano Clark VL. Designing and conducting mixed methods research. Thousand Oaks, CA: Sage Publications; 2007.
- King WJ. The Unwritten Laws of Business. New York, NY: Doubleday; 2007.
- Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377-381.
- Accreditation Council for Graduate Medical Education. Data Resource Book: Academic Year 2013-2014. Chicago, IL: ACGME; 2014.
- Dunteman GH. Principal Components Analysis (Quantitative Applications in the Social Sciences), 1st edition. Los Angeles, CA: Sage Publications; 1989.
- Calhoun JG, Dollett L, Sinioris ME, et al. Development of an interprofessional competency model for healthcare leadership. J Healthc Manag. 2008;53(6):375-389.
- Serio CD, Epperly T. Physician leadership: a new model for a new generation. Fam Pract Manag. 2006;13(2):51-54.
- Eva KW, Regehr G. Self-assessment in the health professions: a reformulation and research agenda. Acad Med. 2005;80(10)(suppl):S46-S54.
- Jones KB, Gren LH, Backman R. Improving pediatric immunization rates: description of a resident-led clinical continuous quality improvement project. Fam Med. 2014;46(8):631-635.