Implementing Competency-Based Medical Education in Family Medicine: A Scoping Review on Residency Programs and Family Practices in Canada and the United States

Competency-based medical education (CBME) is an outcomes-based approach to the design, implementation, assessment, and evaluation of an educational program using an organized framework of competencies, which ultimately may lead to better care for patients.1 Promoting CMBE across the medical education continuum from undergraduate to residency training and continuing professional development (CPD) is envisioned to improve quality and patient safety, considering all stages of professional development contribute to physicians’ knowledge, skills, attitudes, behaviors, and competencies.1-3 Much of the recent discussion around CBME implementation focuses on residency programs,1,4-8 including family medicine residency programs in Canada9-15 and the United States.16-19

Family medicine is uniquely positioned to improve the health of patients and advocate for health equity and patients’ social well-being.20 As family medicine gains importance in the delivery of health care, it is imperative that family physicians are trained in accordance with international educational trends and societal expectations.9,20-22 In order to enhance the quality of residency medical education, the College of Family Physicians of Canada (CFPC) transitioned all family medicine programs to a competency-based Triple C Curriculum.9,23 The Triple C Curriculum emphasizes enabling competencies within seven roles: medical expert, communicator, collaborator, health advocate, scholar, manager, and professional. In the Triple C Curriculum Report, 11 recommendations to change family medicine residency training included the following statement: “A competency-based approach should be used to guide curriculum development and planning.”9 A year later, the Future of Medical Education in Canada: Postgraduate Education Project24 recommended all residency training in Canada “develop, implement, and evaluate competency-based, learner-focused education to meet the diverse learning needs of residents and the evolving healthcare needs of Canadians.”24 The transition from time-based to competency-based education residency programs was intended to explicitly teach and assess the specific competencies needed for practice.

Similarly, in the United States, in 1998, the Accreditation Council for Graduate Medical Education (ACGME) Outcomes Project was initiated to emphasize educational outcome assessment in residency programs and the accreditation process.25 The ACGME and American Board of Medical Specialties (ABMS) jointly endorsed six competency domains: patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice.26 Similar to residency training, CPD based on competency frameworks is envisioned to promote lifelong learning, practice-based change, and improve clinical outcomes.2,8,27-29 As the CBME education systems evolve, there is a need to understand how CBME implementation practices have been adopted across residency education in order to inform the future design and operationalization of CBME in CPD.

Family medicine was one of the earliest adopters of CBME in Canada and the United States with its programs being implemented into postgraduate training on a nationwide scale.4,9,19,30,31 This scoping review represents a timely foundational summation of the existing evidence around the nature of CBME implementation activities in family medicine. The purpose of this study was to use a scoping review methodology to examine the extent, range, and nature of CBME implementation practices in family medicine residency and CPD programs across Canada and the United States. In this review, we addressed the broad question: “What is the scope and nature of the CBME implementation practices in family medicine residency and CPD in the North American scholarly research?” We aimed to address this broad question through first addressing the following subset questions:

1. What is the range and prevalence of CBME implementation practices in family medicine residency and CPD across Canada and the United States discussed in scholarly literature?
2. What are similarities and differences in the CBME implementation practices between residency and CPD?

We organized this scoping review32,33 into six stages: (1) identifying the research question, (2) identifying relevant studies, (3) study selection, (4) charting the data, (5) collating, summarizing, and reporting results, and (6) consultation (optional). We increased methodological rigor by making consultation with content experts the first step and an ongoing process throughout the review.

We circulated the scoping review protocol draft for feedback from experts/stakeholders in three domains: CPD, knowledge translation, and family medicine. These content experts were engaged at four critical points, (1) developing the methodology, (2) reviewing search results, (3) discussing emerging themes and gaps, and (4) reviewing the manuscript.

The scoping review focused on answering the following two questions:

1. What is the range and prevalence of CBME implementation practices in family medicine residency programs and family practices across Canada and the United States?
2. What are similarities and differences in the CBME implementation practices between residency and CPD?

A University of Ottawa senior librarian developed our search strategy, and the project team subsequently revised the strategy. We searched five electronic databases (MEDLINE, ERIC, PsycINFO, Embase, and EdSource) using the following search terms:

“competency-based medical education” OR “CBME” OR “competency-based education” OR “competence,” OR “CanMEDS” OR “the Institute of Medicine competencies” OR “ACGME competencies” OR “outcomes project” OR “ACGME”; AND “continuing professional development” OR “CPD” OR “continuing medical education” OR “CME” OR “education, medical, continuing” OR “continuing education”; AND “residency” OR “residency program” OR “residency programme” OR “residency training” OR “postgraduate medical education” OR “PGME” OR “postgraduate training” OR “graduate medical education” OR “GME” OR “education, medical, graduate” OR “internship and residency” OR “medical residency”; AND “family medicine” OR “family practice” OR “general practice” OR “family physician(s)” OR “general practitioner(s)” OR “primary care physician(s)”; AND Canada OR Canadian OR CanMEDS OR United States OR American OR North America.

We selected the keywords based on a review of published literature and several consultations with the librarian, and this process yielded 470 records (after removing duplicates). The search strategy included manual review of key journals, bibliographies of identified articles and websites of relevant organizations (see the full search strategy in Appendix 1 at https://journals.stfm.org/media/3007/hendry-appendices-1-3.pdf). We included all original research, review articles, editorials/commentaries, and regulatory papers.

The inclusion criteria included articles that

1. were focused exclusively on family medicine residency or CPD programs;
2. discussed CBME;
3. were American or Canadian in origin;
4. were published in English;
5. were available in a full-text version;
6. were published between January 1, 2000 (as the ACGME Outcome Project started in 1999) and April 30, 2017.

The article selection process was guided by the approach advocated by Reeves et al.43 We also drew on our team’s extensive experience in scoping reviews.34-39

Step 1: Articles Screened by Title and Abstract. One member of the project team (N.D.), in consultation with the principal investigator (S.K.) screened 470 titles and abstracts based on inclusion criteria. In case of ambiguities, a full-text review (N.D. and S.K.) was conducted using the same inclusion criteria. The decision to include or exclude an article was recorded to ensure procedural rigor.

Step 2: Full-text Review. After completion of Step 1, 114 articles underwent full-text review. N.D. read all articles and S.K. checked 15% of the articles. The project team then discussed any discrepancies, and disagreements were resolved by consensus. Of the 114 articles, we excluded 77 for the following reasons:

1. No evidence of the CBME concept (n=13);
2. No relevance to CBME implementation practices (n=28);
3. Irrelevant to family medicine (n=20);
4. Assessment tools description (n=8)
5. Undergraduate education (n=5);
6. Non-Canadian or US study (n=3).

Thirty-seven articles were included in the final data set (Figure 1).

Criterion #4 (excluding assessment tools description) was selected given the large corpus of research literature covering CBME. As previously reported, one of the important challenges scoping review teams face is the heterogeneity and the expansive volume of the existing literature included in review, which can prohibit effective and meaningful synthesis or quality assessment. In turn, this creates difficulties in performing in-depth analysis and/or information synthesis.40-42

We completed data extraction using a standardized data extraction form using Microsoft Excel. To ensure rigor of the process and credibility of the findings, we followed Reeves et al’s protocol43 for two or more independent reviewers (C.C., P.H., D.D.) with quality checks from a third party (N.D., S.K.). A detailed description of this step is provided in Appendix 1. We revised the coding manual and specific coding decisions during the coding process based on input from the expert group (C.C., D.D., P.H.; coding manual is provided in Appendix 2 at https://journals.stfm.org/media/3007/hendry-appendices-1-3.pdf).

The final data charting form consisted of two steps. The first step involved gathering article metadata, such as year of publication, title, country where the research was conducted, publication type, research paradigm, and target population. The second step included data relevant to three coding categories: (1) foundation for implementation activity (defined by study authors as documents that serve to support teachers and other educators as they implement the competency-based curriculum framework, including outcomes and standards), (2) implementation activity (defined by Fixsen et al,46 as “a specified set of activities designed to put into practice an activity or program of known dimensions”), and (3) level of implementation practices (defined as “target” or “action target” of interventions).47,48 Using this data extraction form, each expert independently extracted the relevant data from assigned articles in accordance with the criteria defined in Stages 4 and 5. Any ambiguous item was reviewed by the principal investigator (S.K.), who made a final decision. The results were checked by the project team (S.K. and N.D.) for coding consistency. Any changes suggested by the project team were discussed with the experts using a consensus process.

Step 1: Analyzing the Data. We synthesized the data according to themes and shared with the expert panel for their feedback.34 Data analysis involved mixed methods, such as quantitative frequency analysis and qualitative content analysis.

Step 2: Reporting Results. We summarized findings in tabular and narrative forms. In order to preserve clarity of reporting strategy, we applied a consistent approach to reporting the findings.32,42,44 This implies that after consultation with the expert panel, we developed a template and applied it to each level of learning continuum (family medicine residency training and CPD). The template included a table summarizing the objectives previously listed in the initial sections of this protocol.

Step 3: Transferability. The final step of the review framework focused on transferability,35 including the applicability of the findings to other similar contexts and the relevance of the findings to knowledge, policy, practice, and research.45

The original search conducted in April 2017 yielded 891 potentially relevant citations. After screening for duplication and relevance, 114 citations met eligibility criteria based on title and abstract, and the corresponding full-text articles were procured for review. Following full-text review, 37 articles were retained for analysis. Figure 1 represents the flow of the articles from initial identification to final inclusion.

Table 1 reports the general characteristics of the articles included in this study. All included papers (n=37) were published between January 2003 and April 2017, with 76% (n=28) published after 2010 (Figure 2). Almost three-quarters (25/37; 68%) of included studies originated from Canada. Research articles (12/37; 32%), commentary/reflective papers (11/37; 30%), and regulatory (10/37; 27%) comprised most documents included in the review. The number of review articles (4/37; 11%) was underrepresented and there were no editorial opinion articles in the data set. Among 16 studies eligible for classification by a type of research paradigm, 75% (12/16) used a qualitative research approach, 19% (3/16) used a quantitative approach, and one article used mixed methods (1/16; 6%).

Most included studies (27/37; 73%) were identified in published literature, the remaining studies (10/37; 27%) were found in the gray literature, such as websites of CFPC (6/37; 16%), ACGME (2/37; 5%), and American Academy of Family Physicians (2/37’ 5%). The scholarly articles (n=27) were published in 11 different journals with 15 (57%) papers appearing in Canadian Family Physician (10/27; 37%) and Family Medicine (5/27; 19%). Approximately 81% (30/37) of all studies reported residents as their target population, followed by faculty (5/37; 14%) and family physicians (2/37; 5%).

The majority of studies (32/37; 86%) included data coded under the category “foundation for implementation activity” (Table 2). Some of those studies (18/37; 49%) focused solely on “foundation for implementation activity,” but 14 papers (38%) discussed both “foundation for implementation activity” and “implementation activity.” Only a small proportion of the studies (5/37; 13%) concentrated exclusively on the coding category “implementation activity.”

The most commonly described type of foundations for implementation activities was “designing competency-based curriculum” (18/37; 48%), followed by “describing how evaluation/review of competency-based educational program has to be done” (4/37; 11%) and “reviewing competency-based curriculum” (4/37; 11%). The least-used foundation elements were “defining competency-based curriculum outcomes that reflect CanMEDS-FM” (3/37; 8%) and “developing competency-based evaluation objectives” (3/37; 8%).

The coding category “implementation activities” was discussed in 19 papers and fell under two major groups. Almost two-thirds of the studies (11/19; 58%) were assigned to the faculty development (FD) theme with the remaining 42% (8/19) allocated to the curriculum-related activities. The most commonly described FD activity was “training faculty about CBME” (6/19; 33%). Among the curriculum-related activities, the most frequent was “curriculum partial adaptation/implementation” (7/19; 37%) and the least prevalent was “program evaluation” (1/19; 5%).

Over half (20/37; 54%) of the implementation practices were delivered at the program level, followed by institutional/organization level (16/37; 43%). In contrast, only one study (3%) described multiple levels of intervention with individual level of implementation practices being absent (Table 2).

Table 2 highlights the frequency of the three coding categories (foundation for implementation activity, implementation activity, and level of implementation practice) between residency and CPD articles. CBME foundation elements were discussed almost twice as much in residency as in CPD papers, 93% and 57%, respectively. While residency articles focused considerably less (12/30; 40%) on implementation activities, the entire set of CPD studies focused more on FD activities (5/7; 71%) than curriculum activities (2/7; 29%). In contrast, implementation activities in residency papers were equally divided between the two themes: FD and curriculum-related activities. The program level of implementation practices was used more often in residency (15/30; 50%) and CPD (5/7; 71%) studies, followed by institutional/organization level (14/30; 47% and 2/7; 49%, respectively).

The purpose of this study was to identify the range, extent, and nature of CBME implementation practices in family medicine residency and CPD. The major findings of this scoping review were that the vast majority (86%) of included articles concentrated on foundational elements related to designing competency-based curriculum and various implementation strategies, rather than program evaluation or other outcome measures. One-fifth of the articles related to CPD focused on the role for faculty within residency programs rather than the broader issues of how competency frameworks will guide, support, or enable learning and practice improvement initiatives. In addition, there was an exclusive focus on implementation at the program and/or institution/organization levels without a single article focused on the impact of CBME on the individual level.

The review identified three themes: (1) a focus on foundational elements, (2) underevaluated implementation activities, and (3) CBME in CPD as a nascent field.

After ACGME’s and CFPC’s decisions concerning the strategic initiatives for residency education, scholarship in CBME in family medicine saw an increase. Twenty-eight (76%) articles included in this scoping review were published after 2010. As a complex educational intervention, CBME in residency programs is in its relative infancy. Therefore, our findings that implementation practices concentrated more on foundational elements (Table 2) was anticipated. However, considering the inclusion of milestones (observable markers of an individual’s ability along a developmental continuum) within the CanMEDS 2015 framework,49 and the introduction of entrustable professional activities (EPAs) to guide assessment, it was surprising there were only three articles on defining competency-based curriculum outcomes that reflect CanMEDS-FM.

Nineteen studies (51%) discussed specific activities to implement CBME studies divided between residency (n=12) and CPD articles (n=7). Given that implementation of CBME in residency training is recent and evolving, implementation activities mainly focused on FD educational strategies (11/19; 58%). These included the concepts, principles, or intended outcomes for transitioning to CBME (6/19; 33%) with only a limited focus on training curriculum planners about CBME, designing educational meetings, or interprofessional/interdisciplinary education training activities (n=1 each). There were no articles on training initiatives to teach faculty new approaches to clinical supervision of residents within CBME. From a curricular change perspective, seven articles (37%) focused on how CBME could be adapted within traditional clinical rotations.

Another interesting finding was the paucity of formal evaluation studies with only one article focused on approaches to program evaluation. Given that CBME is a complex medical education innovation in the early stages of implementation, the absence of articles to assess the “fidelity of implementation,” defined as the “proportion of program components that were implemented,”50 did not come as a surprise. It has been shown previously that evaluating the fidelity of design elements in CBME can be hindered by the lack of a common framework of what constitutes a CBME program.51,52 Additionally, we identified that the current status of CBME implementation was exclusively focused at the program or institution/organization, rather than individual level. This finding is in parallel with the overall lack of evaluation studies (eg, the impact or outcomes of CBME on individuals) in family medicine residency education and CPD.53

Finally, there were only seven (19%) articles related to CBME for CPD. Most of these articles focused on FD activities, such as preparing faculty to teach or assess residents in new or different ways. Although this focus is clearly relevant to CPD, there was essentially no scholarship on broader questions of the applicability or relevance of CBME to physicians in practice. This finding corresponds to the previous studies by Lockyer et al,54 Nousiainen et al,55 and Kitto et al53 reporting an overall lack of studies on CBME implementation practices within CPD.

Overall, there was a lack of pertinent or transferable knowledge regarding the applicability of CBME within CPD. The current focus of scholarship in CBME for residency training provided limited insights into how competences or a competency framework will guide physician learning and practice improvement initiatives, or contribute to career development, or what infrastructure will be required to sustain a competency-based approach once residents enter practice.

To our knowledge, this is the first study to examine competency-based implementation activities in family medicine, an academic discipline known for advancing CBME implementation. The strengths of this scoping review included the assembly of content and methodological expertise from diverse backgrounds.41,42,56 The article selection process (eg, our inclusion/exclusion criteria) followed established methods, integrating the experience of key content expert team members. We acknowledge several limitations of this review. Although we sought to search multiple databases including the gray literature, we limited the scope of the search to articles in English published in Canada and the United States, which have similar residency programs in family medicine.57 Given the exclusive focus on family medicine, the applicability or transferability to other disciplines may be limited.

This study should be a call to action for all those involved in the development and implementation of CBME and the advancement of health professions education. The pattern of implementation activities found in this scoping review reflected a limited focus on a broad range of issues related to fidelity of implementation—that is, whether CBME is being implemented as intended.49 Perhaps the most alarming finding was a lack of studies on the applicability or relevance of CBME to physicians in practice. With the recent drive to create and implement CBME in CPD to improve the quality of care and patient safety, this lack of evidence could seriously hinder implementation practices well into the future.

Drawing on this scoping review and the knowledge gaps identified, we advise that the paucity of literature assessing the application of CBME to practicing family physicians requires our close attention now. As one avenue to explore, we recommend a closer collaboration among researchers in the areas of continuing education, knowledge translation, patient safety, and quality improvement, as well as within stakeholder organizations responsible for each level of medical training and practice. Integration of efforts across all participants should help to develop successful interdisciplinary interventions to improve professional performance and patient outcomes.53 Before the wave of implementation activities truly commences in earnest across Canada and the United States, there are still many questions that need to be answered, including:

1. How will competencies guide physician learning or practice improvement within their scope of practice?
2. What outcomes or outcome measures can competency-based medical education enable physicians to pursue, either individually or collectively?
3. How can competency-based CPD enable or enhance interprofessional CPD?

We hope this collated resource will help educators become more aware of the existing menu of implementation practices and the gaps in family medicine residency and CPD. There is a need for ongoing and rigorous evidence-gathering to enable the implementation of CBME in CPD.