Learning From the Implementation of Milestones

In 1999, the Accreditation Council for Graduate Medical Education (ACGME) and the American Board of Medical Specialties (ABMS) adopted six core competencies—patient care, medical knowledge, systems-based practice, practice-based learning and improvement, professionalism and interpersonal and communication skills—to improve the quality and safety of patient care in medicine. The Outcomes Project was launched in 2001 to help operationalize these core competencies. As residency programs continued to struggle with the core competencies, the Milestones were developed as part of the Next Accreditation System (NAS) to improve graduate medical education’s demonstration of the trajectory of an individual resident’s path toward competent, unsupervised practice.1

As we think about how to proceed with the next level of competency-based medical education, what can we learn from the implementation of the Milestones? I write from the perspective of the chair of the committee that developed the first version of Milestones and then sat on the Review Committee as they were implemented. Family medicine (FM) began using our Milestones in 2014. A committee with representation across the FM organizations developed the Milestones, received feedback from a broad group of stakeholders, and pilot tested the Milestones before all FM residency programs began using them. Clinical competency committees (CCC) were also introduced as part of the NAS. CCCs review the many different sources of information for each resident and make judgements regarding each resident’s progress across the Milestones. At the time FM residency programs began using Milestones and CCCs, there was uncertainty regarding who should be on the CCCs, what information should be shared with the CCCs, and whether evaluations should change to more closely represent the Milestones.

The Milestones are not competencies. Rather, the Milestones help us have a shared mental model of the trajectory of learning and professional growth in FM. The information needed to make judgements about where learners are on the trajectory of the Milestones include evaluations, assessments of competencies, patient surveys, and clinical data. The ACGME’s Milestone website has links to the current FM Milestones, the supplemental guide to the FM Milestones, as well as the Milestones National Reports from 2016-2020 and other articles about the Milestones and assessments.2 The Milestones National Reports include benchmarking data for FM and all other specialties.

The Milestones have forced FM educators to have conversations through the CCCs about the multiple sources of information collected regarding each individual resident. As a specialty, we have worked together, taught one another, and shared best practices regarding the CCCs and completing the Milestones. Because of this, FM, unlike other specialties, has not used straight-lining based on the year a resident is in training to complete the Milestones. Straight-lining is defined in the Milestones National Report as a string of identical Milestones ratings for a learner across all subcompetencies within that specialty.3 In the 2020 Milestones National Report, FM residency programs showed straight-lining in only 8.3% of PGY-1, 4.5% of PGY-2, and 5.6% of PGY-3 Milestones (range for straight-lining for all specialties 0.0%–50.0%).4 Completing the Milestones twice per year has helped programs identify individual residents who are struggling globally or in a specific area earlier in residency. This process has also helped programs identify areas where the curriculum of the residency program may need to be changed or improved for all residents in their program.

The Milestones have also shown FM residency programs and educators where challenges remain. For instance, evaluations being completed by faculty do not always provide specific enough information for the CCC to draw conclusions regarding a resident’s progress. The assessments programs use do not always accurately assess the competencies. Faculty have not all been trained to teach competencies, provide feedback in a learner-centered way, complete evaluations or do assessments in ways that help residents move along the trajectory towards competent, unsupervised practice. For the Milestones to accurately show a resident’s trajectory, the multiple sources of information used to make judgements about resident progress in the Milestones must be appropriate, accurate, and contain useful information to help residents maximize their professional growth.

FM educators can also strengthen and improve education in our residency programs through the use of data. The aggregate data shared in the Milestones National Report can help the specialty understand where we are behind in training our residents compared to other specialties in the common competencies of systems-based practice, practice-based learning and improvement, professionalism, and interpersonal and communication skills. We should reach out to our colleagues in other specialties to learn best practices to help improve FM residency education.

As FM residency education continues to embrace competency-based education, faculty development in assessment of the competencies, appropriate completion of evaluations, interpretation of other pertinent data, and educating residents and fellows about their own learning and professional growth trajectory is important. We must collaborate by sharing best practices and ensuring all faculty have been trained to use this system. FM residency program directors must also be trained to utilize program and national data from the Milestones to improve the education and assessments for the program, and we as a specialty must do the same.

The Milestones are not designed to be forgotten at the end of residency. The trajectory of the Milestones is not designed for graduating residents to be experts in all competencies. All practicing physicians should continue their professional growth trajectories toward being experts. Family physicians can continue using the Milestones to help develop individual learning plans for continued professional growth and development throughout their careers.

This work is important. Moving to competency-based education and assessment requires us to have goals and specific outcomes to measure our success. After 5 years of work, FM residencies have learned to teach and assess competencies. To complete this process, it will take dedicated time for faculty and program directors to develop and implement competency-based teaching and assessment. We can accomplish this goal.

Family physicians care for all ages of individuals, in urban, suburban, small city and rural communities across the United States. FM has a responsibility to train and graduate family physicians ready to provide competent, unsupervised care, and who will continue their learning and professional growth to provide high-quality, safe care to their community.