Purpose: The patient-centered medical home (PCMH) model has been proposed as the ideal model for delivering primary care and is focused on improving patient safety and quality, reducing costs, and enhancing patient satisfaction. The mandated Accreditation Council for Graduate Medical Education educational milestones for evaluation of resident competency represent the skills graduates will utilize after graduation. Many of these skills are reflected in the PCMH model. We sought to determine if residency programs whose main family medicine (FM) practice sites have achieved PCMH recognition are therefore more prepared to evaluate milestones.
Method: A national Council of Academic Family Medicine Educational Research Alliance (CERA) survey of family medicine program directors (PDs) was conducted during June and July 2015 to determine if PCMH recognition influences PDs’ ability to evaluate training methods and their level of preparedness to evaluate milestones.
Results: The response rate for the survey was 53.3% (252/473). Nearly two-thirds of the PDs (62.7%) reported that their main FM practice site had earned PCMH recognition. There was no statistical difference between non-PCMH-recognized vs PCMH-recognized programs in how PDs perceived that their program was prepared to assess residents’ milestone levels overall (P=0.414). Residents of PCMH-recognized programs were more likely to receive training for team-based care (P=0.009), system improvement plans (P<0.001), root-cause analysis (P=0.002), and health behavior change (P=0.003).
Conclusions: PCMH recognition itself did not improve preparedness of FM residency programs to assess milestones. Residents from programs whose main FM practice site is PCMH-recognized are more likely to be trained in the key concepts and tasks associated with the PCMH model, tools that they are expected to utilize extensively after graduation.
The Accreditation Council for Graduate Medical Education (ACGME) requires that milestones be used to assess the competency of residents.1 The ACGME Milestones were developed and written to embody key attributes physicians are expected to possess and to reflect real-world practice. They were designed to be “a logical trajectory of professional development in essential elements of competency,”2 representing stages in the progression of resident competency through training and even into their postgraduation practice environment.1 Emerging data suggests the Milestones are a reliable tool for assessing where learners are on this trajectory.3,4
Family medicine residency programs (FMRPs) are expected and required to prepare their graduates to provide safe and effective health care in that future practice environment.5 Today, this means that FMRPs must train residents to successfully practice and lead in the PCMH model.6
The patient-centered medical home (PCMH) model emphasizes patient safety, quality, cost reduction, and patient satisfaction.7,8 It is the currently favored primary care model, making it imperative that programs prepare residents to practice within this framework.9 By extension, this means FMRPs need to evaluate residents on their competency within the PCMH framework. Many FMRPs have therefore sought to gain PCMH recognition status for their primary family medicine site.10 Certain ACGME Milestones, such as under the competencies of Practice-Based Learning and Improvement and Systems-Based Practice, describe vital skills to practicing in a PCMH model.
The authors hypothesized that those program directors whose main family medicine practice site is a certified PCMH environment would report being better able to assess their residents’ skills at PCMH tasks, and would report being better able to assess the ACGME Milestones, or at least a select subset, because their residents would be observed practicing in the PCMH model regularly.
Some research has been done on the impact of training residents in the PCMH model. It has been demonstrated that residents trained at PCMH-recognized sites perform more PCMH-type skills.11 The creation of a block rotation in a PCMH environment increases residents’ skills and knowledge about the model, but alone does not make them experts on PCMH.12,13 How increased exposure to the PCMH model relates to a residency program’s ability to evaluate competency at these skills or evaluate milestones that relate to these skills has not been studied. This study sought to address that question.
A focused survey was developed and included as part of an omnibus survey of family medicine residency program directors (PDs) conducted by the Council of Academic Family Medicine Educational Research Alliance (CERA) following methods previously described.14 The survey covered program demographics, status of PCMH recognition, preparedness to assess Milestones, training on PCMH topics, and preparedness to utilize Milestone data to influence residency curricula. The study was approved by the American Academy of Family Physicians Institutional Review Board. The study was conducted from June to July 2015.
Descriptive analyses using frequencies and proportions was used for all categorical data, while means and standard deviations were used for continuous data to summarize the overall demographic data and to describe the PDs’ perceptions of preparedness to assess resident Milestone levels and the amount of training in various tasks related to PCMH. Bivariate analyses examined the relationship between preparation and PCMH status along with other variables. Variables with P-values less than .20 were included in a prediction model. Analyses were conducted with SAS, version 9.4 (SAS Institute, Cary, NC).
A total of 252 of 473 (53.3%) PDs responded (Table 1), and of these, 62.7% reported their main family medicine clinic had earned PCMH recognition. Most (87.7%) of the programs received their PCMH recognition from the National Committee for Quality Assurance (NCQA), 3.4% from the Joint Commission, 1.4% from the Utilization Review Accreditation Commission, and 0.7% from the Accreditation Association for Ambulatory Health Care. PDs reported that they had been certified for an average of 4.5+1.9 years.
No difference in preparedness to assess Milestones based on recognition status was found (Table 2). Residents of PCMH-recognized programs were more likely to receive training in team-based care, system improvement to promote patient safety, root-cause analysis, and motivational interviewing. PCMH-recognized programs were more likely to receive training on core PCMH principles. More PDs of PCMH-recognized programs felt that their residents were prepared to lead a clinic through the process of PCMH transformation than of PDs from non-PCMH-recognized programs.
Multivariate logistic regression identified determinants of preparedness to assess Milestones (Table 3). The proportion of international medical graduates (IMGs) was inversely associated with preparedness. PCMH recognition from the NCQA was associated with preparedness compared to recognition from another organization.
Contrary to our hypothesis, we found no benefit of having PCMH recognition in terms of preparedness to assess Milestones overall, for system-based practice, practice-based learning, or interpersonal and communication skills. It may be that assessing milestones is such a complex process that exposure to the ideal practice environment itself was not enough to make program directors feel more comfortable with the process.
PDs of PCMH-recognized programs reported more training for team-based care, system improvement plans, root-cause analysis, health behavior change, core PCMH principles, and gaining PCMH recognition. It seems intuitive that programs applying for recognition would train residents in PCMH concepts.9,15 Previous research has shown that exposing residents to a certified PCMH clinic increases knowledge and skills about PCMH principles, although perhaps not their ability to lead in the PCMH model.16 The optimal method of teaching these principles is not known, although residencies are experimenting with various methods.17 Moreno et al found that family medicine residents want to learn more about the PCMH model during residency. The authors concluded that residents may benefit from experiential learning focused on the PCMH model.12
University-based and military programs were less likely to have recognition than university-affiliated and community-based programs. This suggests that recognition may be more valued, or easier to achieve, in community settings than in complex systems. University-based programs may have more bureaucracy, making it difficult to promote change (eg, to alter a clinic process to accommodate an educational curricular requirement). The PDs in non-university-based programs are more likely to direct all aspects of the education of their residents. Thus, the PDs in non-university-based programs may have greater autonomy to implement curricular and assessment changes compared to PDs in university-based programs.
We found significant differences in recognition rates based on geographic distribution, with more programs in the New England, Middle Atlantic, and South Atlantic states having PCMH recognition. This finding may be unique to our data set, or could represent regional differences regarding the perceived importance of PCMH recognition.
Logistic regressions found two variables statistically associated with preparedness. NCQA recognition increased preparedness relative to other organizations, suggesting there may be something unique about the NCQA application. Fewer IMGs was associated with preparedness. This finding could be unique to our data set. If it is a true association, we believe it may be a marker for another, unidentified factor impacting preparedness.
There are some limitations to this study. Our effort evaluated only the perceptions of PDs. It is also possible that PDs feel compelled (either consciously or subconsciously) to exaggerate their actual ability to accomplish this ACGME-mandated task. Therefore, the actual preparedness of a program may be greater or less than PDs reported. Additionally, we looked only at a subset of ACGME Milestones; other milestones could be associated with PCMH recognition. Lastly, our study looked only at FMRPs. Evaluation of the PCMH status in other primary care specialties could yield different results.
While PCMH recognition does not appear to enhance program preparedness to assess milestones, residents of these programs do receive more training in key elements of the PCMH model. The recognition process itself may therefore better prepare residents to function within the PCMH environment after graduation, making it worth the time and effort it takes to gain recognition.
Prior presentation: Andrianakos R, Wilkins T, Wonsuk Y, Gillies R, Dahl-Smith J, DuBose J, Hobbs J, Smith S, Seehusen DA. Patient Centered Medical Home Status and Preparedness to Implement Evaluation of Resident Milestones in U.S. Family Medicine Residencies: A CERA Study. NAPCRG Annual Meeting. Colorado Springs, CO. November 2016.
- Allen S. Development of the family medicine milestones. J Grad Med Educ. 2014;6(1)(suppl 1):71-73.
- Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system--rationale and benefits. N Engl J Med. 2012;366(11):1051-1056.
- Peabody MR, O’Neill TR, Peterson LE. Examining the Functioning and Reliability of the Family Medicine Milestones. J Grad Med Educ. 2017;9(1):46-53.
- Mainous AG III, Fang B, Peterson LE. Competency Assessment in Family Medicine Residency: Observations, Knowledge-Based Examinations, and Advancement. J Grad Med Educ. 2017;9(6):730-734.
- Bhuyan N, Miser WF, Dickson GM, et al. From family medicine milestones to entrustable professional activities (EPAS). Ann Fam Med. 2014;12(4):380-381.
- American Academy of Family Physicians. Residency Program Solutions Criteria for Excellence. 7th ed. Leawood, KS: AAFP; 2008.
- Peikes D, Zutshi A, Genevro JL, Parchman ML, Meyers DS. Early evaluations of the medical home: building on a promising start. Am J Manag Care. 2012;18(2):105-116.
- Brown CM, Cronholm PF, Wright J, Warning WJ II, Radosh L, Gabbay R. A collaborative approach to achieving NCQA PCMH recognition within family medicine residency practices. Fam Med. 2014;46(1):19-27.
- Jortberg BT, Fernald DH, Dickinson LM, et al. Curriculum redesign for teaching the PCMH in Colorado Family Medicine Residency programs. Fam Med. 2014;46(1):11-18.
- Buscaj E, Hall T, Montgomery L, et al. Practice Facilitation for PCMH Implementation in Residency Practices. Fam Med. 2016;48(10):795-800.
- Block L, LaVine N, Verbsky J, et al. Do medical residents perform patient-centered medical home tasks? A mixed-methods study. Med Educ Online. 2017;22(1):1352434.
- Anandarajah G, Furey C, Chandran R, et al. Effects of adding a new PCMH block rotation and resident team to existing longitudinal training within a certified PCMH: primary care residents’ attitudes, knowledge, and experience. Adv Med Educ Pract. 2016;7:457-466.
- El Rayess F, Goldman R, Furey C, Chandran R, Goldberg AR, Anandarajah G. Patient-Centered Medical Home Knowledge and Attitudes of Residents and Faculty: Certification Is Just the First Step. J Grad Med Educ. 2015;7(4):580-588.
- Mainous AG III, Seehusen D, Shokar N. CAFM Educational Research Alliance (CERA) 2011 Residency Director survey: background, methods, and respondent characteristics. Fam Med. 2012;44(10):691-693.
- Carney PA, Eiff MP, Saultz JW, et al. Assessing the impact of innovative training of family physicians for the patient-centered medical home. J Grad Med Educ. 2012;4(1):16-22.
- Moreno G, Gold J, Mavrinac M. Primary care residents want to learn about the patient-centered medical home. Fam Med. 2014;46(7):539-543.
- Chandran R, Furey C, Goldberg A, Ashley D, Anandarajah G. Training family medicine residents to build and remodel a patient centered medical home in Rhode Island: a team based approach to PCMH education. R I Med J (2013). 2014;98(4):35-41.