— RESEARCH BRIEF —

Teaching Self-directed Learning Skills Through a Resident-Led Milestones Process

Katherine T. Fortenberry, PhD | Susan Cochella, MD, MPH | Kirsten Stoesser, MD | Dominik Ose, DrPH, MPH | Eliza Taylor, MPH, BS, CHES | Kara A. Frame, MD | Sonja Van Hala, MD, MPH

PRiMER. 2022;6:40.

Published: 9/21/2022 | DOI: 10.22454/PRiMER.2022.321624

Abstract

Introduction: The Accreditation Council for Graduate Medical Education allows flexibility for resident roles in the Milestone assessment process. The University of Utah Family Medicine Residency implemented a resident-led Milestones process to cultivate the skill of self-assessment and promote resident ownership of their learning.

Methods: Residents were provided comprehensive evaluation data and asked to self-assess their competency on each Milestone, with input from their advisor. Residents presented their self-assessment to the Clinical Competency Committee, who then determined the final score for each Milestone. A 10-question survey examined perceptions of the resident-led Milestones process by residents and faculty. We calculated means and standard deviations (SD).

Results: A total of 16 of 24 residents (67% response rate) and 12 of 14 faculty (86% response rate) completed the survey. Residents agreed most highly with the following statements: “I have good support from my advisor in being prepared to lead my Milestones meeting,” “I am actively engaged in guiding the development of my own Milestones ratings,” and “Leading my Milestones meeting assists me in accurately self-assessing my progress.” Residents showed high agreement that “My final Milestones scores accurately reflect my behavior and level of knowledge.” Residents rated the stress as low, in response to the statement, “My Milestones meeting is stressful for me.” Faculty responses were similar but tended toward lower scores than residents.

Conclusion: The resident-led Milestones process engages residents actively in self-assessment. Residents and faculty believe the process provides accurate assessment results without undue stress; this process potentially increases residents’ ability to understand their own learning needs and direct their own learning process.

Introduction

Milestones assessment processes were introduced into the family medicine specialty in 2014.1 The overall goal of the Milestones assessment process is to provide a common language and understanding for defining core competencies in graduate medical training,2 and is intended to be a formative pathway that supports residents’ professional development.3 The Milestones assessment process can promote effective self-assessment and self-directed learning as part of an overall goal of strengthening outcomes-focused competency-based medical education.4

When the Accreditation Council for Graduate Medical Education (ACGME) tasked residency programs to implement specialty-specific milestones, it allowed flexibility for resident roles in milestone assessment processes.5–7 Resident roles differ between programs,8 spanning a spectrum from no involvement to leadership. On the “no involvement” extreme, residents simply receive the Clinical Competency Committee’s (CCC) assessment of their performance. “Limited involvement” allows residents to submit self-assessments for the committee’s consideration. Alternatively, residents may serve in a leadership role during their Milestone assessment process, actively participating in the CCC discussion of their performance. While systems with less resident involvement may appear more efficient at face value, they likely have an incomplete picture of resident performance, as many resident activities may not be visible to the CCC. Further, limited resident involvement may create stress by not giving the resident a voice in their evaluation, and the absence of resident input places the bulk of administrative burden on the faculty.

The ACGME recognizes that physicians need to become lifelong learners with accurate self-assessment skills.9 However, there is a relative paucity of literature on resident self-assessment within the Milestones evaluation process, with most occurring in surgical and specialty programs.10,11 Existing evidence supports resident self-assessments as beneficial for self-directed learning and correlates with faculty evaluations,4,10–13 In 2014, the University of Utah’s Family Medicine Residency program implemented a resident-led Milestones process to cultivate the skill of self-assessment and promote resident ownership of their learning. This article describes resident and faculty perceptions as evidenced by structured surveys. 

Methods

Resident-led Milestones Process

The resident-led Milestone process places each resident at the center of their own evaluation process. Residents review their evaluations and performance data, which is drawn from numerous sources (Table 1) including observation-based feedback organized by ACGME competency. The residency program provides a dashboard of graduation requirements with a visual summary of resident progress. Residents self-assess along each Milestone, meeting with their faculty advisor to receive support and feedback and present their assessment and rationale to the CCC (Figure 1, Table 2). The CCC then makes the final determination for the level of each Milestones competency, collaboratively incorporating resident self-assessment.

Evaluation

We developed and administered a 10-question, 5-point Likert scale (1=strongly disagree; 5=strongly agree) survey examining perceptions of the resident-led Milestones process by residents and faculty. The questions addressed engagement of residents and faculty, self-assessment, accuracy of scores, and perceived support during the process. During a single 6-month period in 2016, a full academic year after implementing this process, residency faculty received the survey by email and residents completed a confidential and anonymous paper survey immediately following their Milestones meeting. We calculated means and standard deviations (SD). The University of Utah Institutional Review Board (IRB, IRB #00130505) acknowledged the project as nonresearch under an Umbrella IRB for educational quality improvement projects.

Results

A total of 16 of 24 residents (67%) and 12 of 14 faculty (86%) completed the survey (Table 3). Residents agreed most highly that they had good support from their advisors to be prepared to lead their Milestones meeting (Q9; mean: 4.6). Residents also reported active engagement in guiding the development of their Milestones ratings (Q1; mean: 4.5) and agreed that leading their meeting supported accurate self-assessment (Q10; mean: 4.4). Residents agreed that the final Milestones scores accurately reflected their behavior and knowledge (Q3; mean: 4.2). Residents rated the stress as low (Q8; mean: 2.5).

Residents tended toward agreeing that their individual learning plan guided studying and learning goals (Q7; mean: 3.6) and that leading their meeting assisted in accurately self-assessing progress (Q6; mean: 3.9).

Faculty responses were similar but tended toward lower scores than residents (Table 3). 

Conclusion

Self-awareness of strengths and weaknesses, as well as ability to direct one’s own learning process, are essential skills that family medicine residents must develop to become lifelong learners.14 A resident-led Milestones process actively engages residents in self-assessing their progress, with residents perceiving the final evaluations as accurate, and experiencing the evaluation process as low stress. Making the residents’ view central in this process may be an important component in keeping stress low.

Program faculty reported similarly positive perceptions of resident engagement and accurate self-reflection with the resident-led Milestones process. Although faculty perceived that residents experience higher stress and lower support with the resident-led Milestones process, overall, faculty view the resident-led Milestones process as a way to promote residents’ skills in self-directed assessment and learning (Table 3).

Residency programs have implemented the ACGME Milestones through a variety of processes, with varying degrees of administrative burdens.16 We suspect that there is a high level of administrative burden when CCC faculty complete Milestones ratings without active resident participation. The resident-led Milestones process reduces inefficiencies by incorporating resident self-reflections that fill in gaps missed by data reports and making sure residents are present for communications about their performance.

Limitations

This assessment involved a brief, nonvalidated survey with a limited number of respondents in one residency program. The survey was completed with one cohort of residents over 1 year relatively early in the implementation of Milestones assessments; this may have added stress as the process still felt new. Residents were not required to participate in the survey, making selection bias possible. Faculty who created and implemented this process were among those who completed the faculty survey, making confirmation bias possible.

Next Steps

An important next step will be to compare different models of Milestones implementation across different programs and specialties. A qualitative look at the resident-led Milestones process could explore sources of stress and support and impact on learner engagement. The ACGME Milestones can be an opportunity for residents to practice, in a supervised environment, the skills of defining desirable behaviors, marking progress toward competency, and setting guideposts for their training. By adopting a resident-led process, residency programs can operationalize the Milestones process to be learner-centered as a mechanism to promote the core competency of resident self-directed learning.

References

  1. Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system--rationale and benefits. N Engl J Med. 2012;366(11):1051-1056. doi:10.1056/NEJMsr1200117
  2. Green ML, Aagaard EM, Caverzagie KJ, et al. Charting the road to competence: developmental milestones for internal medicine residency training. J Grad Med Educ. 2009;1(1):5-20. doi:10.4300/01.01.0003
  3. Peabody MR, O’Neill TR, Peterson LE. Examining the functioning and reliability of the Family Medicine Milestones. J Grad Med Educ. 2017;9(1):46-53. doi:10.4300/JGME-D-16-00172.1
  4. Holmboe ES. The transformational path ahead: competency-based medical education in family medicine. Fam Med. 2021;53(7):583-589. doi:10.22454/FamMed.2021.296914
  5. Holmboe ES, Edgar L, Hamstra S. The Milestones Guidebook. Accreditation Council for Graduate Medical Education. Accessed September 15, 2022. https://www.acgme.org/globalassets/milestonesguidebook.pdf
  6. Hawkins RE, Welcher CM, Holmboe ES, et al. Implementation of competency-based medical education: are we addressing the concerns and challenges? Med Educ. 2015;49(11):1086-1102. doi:10.1111/medu.12831
  7. ten Cate O, Scheele F. Competency-based postgraduate training: can we bridge the gap between theory and clinical practice? Acad Med. 2007;82(6):542-547. doi:10.1097/ACM.0b013e31805559c7
  8. Heath JK, Dine CJ. ACGME Milestones within subspecialty training programs: one institution’s experience. J Grad Med Educ. 2019;11(1):53-59. doi:10.4300/JGME-D-18-00308.1
  9. ACGME Common Program Requirements (Residency). Accreditation Council for Graduate Medical Education; July 1, 2022. Accessed September 15, 2022. https://www.acgme.org/globalassets/PFAssets/ProgramRequirements/CPRResidency_2022v2.pdf
  10. Meier AH, Gruessner A, Cooney RN. Using the ACGME Milestones for resident self-evaluation and faculty engagement. J Surg Educ. 2016;73(6):e150-e157. doi:10.1016/j.jsurg.2016.09.001
  11. Watson RS, Borgert AJ, O Heron CT, et al. A multicenter prospective comparison of the Accreditation Council for Graduate Medical Education Milestones: Clinical Competency Committee vs Resident Self-Assessment. J Surg Educ. 2017;74(6):e8-e14. doi:10.1016/j.jsurg.2017.06.009
  12. Srikumaran D, Tian J, Ramulu P, et al. Ability of ophthalmology residents to self-assess their performance through established milestones. J Surg Educ. 2019;76(4):1076-1087. doi:10.1016/j.jsurg.2018.12.004
  13. Lyle B, Borgert AJ, Kallies KJ, Jarman BT. Do attending surgeons and residents see eye to eye? An evaluation of the Accreditation Council for Graduate Medical Education Milestones in General Surgery Residency. J Surg Educ. 2016;73(6):e54-e58. doi:10.1016/j.jsurg.2016.07.004
  14. Martin JC, Avant RF, Bowman MA, et al; Future of Family Medicine Project Leadership Committee. The Future of Family Medicine: a collaborative project of the family medicine community. Ann Fam Med. 2004;2(suppl 1):S3-S32. doi:10.1370/afm.130
  15. Stefanou CR, Perencevich KC, DiCintio M, Turner JC. Supporting autonomy in the classroom: ways teachers encourage student decision making and ownership. Educ Psychol. 2004;39(2):97-110. doi:10.1207/s15326985ep3902_2
  16. Goroncy A, Goldstein H, Goodnow K, Pallerla H, Regan S. Evaluating resident home visit performance: introducing a feedback form linked to ACGME Milestones. J Am Geriatr Soc. 2020;68(3):E1-E3. doi:10.1111/jgs.16280

Lead Author

Katherine T. Fortenberry, PhD

Affiliations: Department of Family and Preventive Medicine, School of Medicine, University of Utah, Salt Lake City, UT

Co-Authors

Susan Cochella, MD, MPH - Department of Family and Preventive Medicine, School of Medicine, University of Utah, Salt Lake City, UT

Kirsten Stoesser, MD - Department of Family and Preventive Medicine, School of Medicine, University of Utah, Salt Lake City, UT

Dominik Ose, DrPH, MPH - Department of Family and Preventive Medicine, School of Medicine, University of Utah, Salt Lake City, UT

Eliza Taylor, MPH, BS, CHES - Department of Family and Preventive Medicine, School of Medicine, University of Utah, Salt Lake City, UT

Kara A. Frame, MD - Department of Family and Preventive Medicine, School of Medicine, University of Utah, Salt Lake City, UT

Sonja Van Hala, MD, MPH - Department of Family and Preventive Medicine, School of Medicine, University of Utah, Salt Lake City, UT

Corresponding Author

Katherine T. Fortenberry, PhD

Correspondence: 375 Chipeta Way Suite A, Division of Family Medicine, Department of Family and Preventive Medicine, University of Utah, Salt Lake City, UT 84108. 801-587-3381. Fax: 801-587-2771.

Email: Katie.Fortenberry@hsc.utah.edu

Fetching other articles...

Mendeley

Loading metrics from Mendeley...

Loading the comment form...

Submitting your comment...

There are no comments for this article.

Downloads & Info

Share