— ORIGINAL ARTICLES —

Simulation-Based Medical Education in Family Medicine Residencies: A CERA Study

Johnny C. Tenegra, MD, MS | M. Rebecca Hoffman, MD, MSPH | Georgia S. Mueller Luckey, PhD, MS | Lisabeth F. DiLalla, PhD | Christy J.W. Ledford, PhD

Fam Med. 2022;54(4):264-269.

DOI: 10.22454/FamMed.2022.731975

Abstract

Background and Objectives: To better understand the current use of simulation and barriers to its use in family medicine resident education, we surveyed US family medicine residency (FMR) program directors (PDs) about opinions and use of simulation-based medical education (SBME) in their programs. A number of specialties have incorporated or required simulation-based educational techniques in residency education over the past 10 years, but little is known about the current use of SBME in family medicine graduate medical education. We also evaluated associations between program characteristics and the use of SBME in FMR education.

Methods: Questions were included on the 2019 Council of Academic Family Medicine Education Research Alliance (CERA) survey of US FMR PDs. The survey included questions regarding current use of SBME along with questions to identify barriers to its use in family medicine programs.

Results: Thirty-nine percent (n=250) of PDs completed the survey; 84.5% reported using simulation. PDs reporting they did not use simulation were less likely to view simulation as valuable for education or assessment. Unexpectedly, residency program size was not associated with simulation use or access to a dedicated location for SBME.

Discussion: Use of SBME in family medicine resident education has increased since 2011. PDs value simulation for education and remediation, and most programs have introduced some degree of simulation despite barriers. The results of this study can inform resources to support the continued integration of SBME into family medicine resident education.


To prepare family medicine residents for the increased spectrum of clinical skills they must acquire within the constraints of reduced duty hours, limited patient encounters, and increased focus on patient safety, simulation education has been proposed as a method to expose residents to common and uncommon clinical situations.1 Simulation-based medical education (SBME) leads to retention of clinical knowledge 2 and improved performance in rare, high-stakes situations,3,4 and exposes residents to clinical scenarios of varying difficulty.5 SBME can improve a resident’s self-confidence,6 medical knowledge, clinical skills, communication,7-9 critical thinking,10 team building, and leadership11 —all items evaluated on resident Accreditation Council for Graduate Medical Education (ACGME) milestone evaluations.12 When theoretically-grounded, SBME can help residents achieve more specific clinical skill acquisition goals as compared to traditional clinical medical education.13 SBME is also an effective tool for gathering professional assessment feedback14 and for the evaluation of clinical skills in residency training environments.15,16

Though simulation has been a staple of graduate medical education (GME) for more than 40 years in the fields of surgery, anesthesiology, and emergency medicine, family medicine (FM) historically has not fully leveraged the potential of simulation. A 2011 AAMC survey regarding simulation showed that only 12% of teaching hospitals and 58% of medical schools utilized simulation for family medicine resident education.17 Potential barriers to the use of simulation in FMR programs include lack of fiscal and physical resources, such as simulation models or simulated patient actors, and lack of faculty simulation experience.10

Since the AAMC study in 2011, no studies have investigated the scope of SBME in family medicine residencies. This study aims to fill that gap. The objectives of this study were (1) to describe the current use of SBME in FMR education, (2) to identify FMR program characteristics associated with the use of simulation, and (3) to assess the relationships among program director (PD)-perceived barriers, PD-perceived value, and the use of SBME.

Methods

To assess the current state of simulation education in family medicine and to better understand the relationships between program characteristics, resources, and PD characteristics and the use of and barriers to simulation use, we developed questions for inclusion in the 2019 Council of Academic Family Medicine (CAFM) Educational Research Alliance (CERA) survey of FMR program directors. The methodology of the CERA Program Director Survey has previously been described in detail.18 The American Academy of Family Physicians Institutional Review Board approved this project in September 2019.

Setting and Participants

The population surveyed was all ACGME-accredited US FMR PDs as identified by the Association of Family Medicine Residency Directors (AFMRD). Email invitations to participate were delivered along with the survey, using the online program SurveyMonkey. Between September and November 2019, six follow-up emails were sent after the initial invitation to encourage nonrespondents to participate. At the time of the survey, 668 PDs met the inclusion criteria. Forty had previously opted out or blocked surveys; therefore, the survey was emailed to 628 individuals.

Survey Development

The research team developed survey questions to assess the current state of simulation and submitted the questions through the CERA survey process. The CERA Steering Committee, along with a group of expert peer reviewers, evaluated questions for consistency with the overall subproject aim, readability, and existing evidence of reliability and validity. Pretesting, conducted with family medicine educators who were not included in the sampling frame, evaluated questions for flow, timing, and readability. The final larger omnibus survey included 53 survey items: 10 demographic/characteristic core items and five topic-specific sections. The simulation section included nine questions.

Main variables of interest included use of simulation and perceived barriers to simulation use in resident education, assessment, and remediation. We provided the following definition of simulation to respondents: “a method used in health care education to replace or amplify real patient experiences with scenarios using lifelike mannequins, physical models, standardized patients, or computers.” Likert-type items assessed the value of the use of simulation in education, assessment, and remediation. Four questions addressed the specific uses, how often simulation was used, and which competencies simulations assessed. Four questions addressed the types of simulations used to create realistic simulation experiences. Table 2 provides definitions of each of these simulation types. PDs were also asked to identify barriers to the use of simulation from a pre-defined list developed from previous literature and to identify their access to dedicated simulation location, defined as “a space or building that has a primary role/purpose to provide simulation-based medical education.”

In the core questions, PDs reported their residency program’s general characteristics (location and size of program, size of community, proportion of international medical graduates (IMGs) and individual characteristics (PD gender, race, and length of time in PD role).

Data Analysis

We coded, aggregated, and analyzed data using Statistical Analysis System (SAS) software version 9.4. We compared differences in simulation use, how often simulation is utilized, how programs utilize simulation, competencies, barriers, and value of simulation between type of residency program and size of residency program using a χ2 test of independence and/or Fisher’s exact test as appropriate. In the direct comparison between community-based, university-affiliated, university nonaffiliated, and university-based FMR programs, “military” and “other” residency program types were excluded (n=10), due to not being able to confidently distinguish them as community-based or university-based programs. In direct comparisons between community-based and university-based FMR programs, community-based programs were defined to include both university and nonuniversity-affiliated community programs. We used a Spearman rank test post hoc to explore the association between value of simulation for education and assessment and frequency of simulation utilization.

Results

Overall Results

The overall response rate for the survey was 39.1% (250/628). Table 1 presents program characteristics of responding PDs.

Overall, 84.5% of respondents reported that their programs use simulation. A majority of responding PDs (67%) report using simulation for both education and assessment. The core competency that was assessed the most through simulation was patient care, followed by medical knowledge and interpersonal communication skills (Table 2). PDs reported that they use a variety of methods of simulation, including standardized patients (56%), full-scale mannequins (61%), and part or partial task trainers (59%). Screen-based simulation was utilized the least; only 22% of PDs reported use of this method of simulation (Table 3).

A significant, positive linear relationship existed between perceived value of simulation for education and assessment and frequency of simulation utilization. The more frequently simulation was utilized in a program, the more a PD valued simulation as a technique for resident education, including remediation (sr=0.35, P<.0001), and assessment (sr=0.22, P=.0014).

Comparisons Among Residency Program Type and Size

Residency programs of different types and sizes utilized SBME at comparable rates. Similarly, there were no significant differences between university- and community-based programs regarding how simulation is used, how often it is used, and which of the ACGME core competencies are assessed using SBME (Table 2). Access to a dedicated location for simulation did not differ significantly between sizes of family medicine programs (79% and 77%). Slightly more university-based programs utilized simulation for education, including remediation, compared to community-based programs (37% vs 21%), although this difference was not significant.

PDs of community-based programs (18%) were as likely to report barriers as PDs of university-based programs (23%), although the most prominent barrier differed significantly P=.012). For university-based programs, the most prominent reported barrier was lack of faculty teaching time. However, for community-based programs, the most prominent reported barrier was lack of support from teaching hospital or academic institution.

Findings for Programs That Did Not Utilize Simulation

Fifteen percent of PD respondents reported that their program did not utilize simulation. Participants who did not utilize simulation reported significantly lower rates of agreement with the statement “simulation is a valuable technique for education” (71% vs 92%, P=.001) and “… for assessment” (66% vs 82%, P=.0286) compared to PDs that utilized simulation, respectively (Table 4). Of PDs who did not utilize simulation, 61% reported not having access to a dedicated location for simulation compared to 18% of programs that utilized simulation (P<.0001). Among PDs who did not utilize simulation, 95% reported a barrier (ie, lack of resident and/or faculty time, interest, or training; lack of support from teaching institution or colleagues) when attempting to provide simulation-based education compared to 80% of programs that utilized simulation (P=.034).

Discussion

In this 2019 CERA survey, a majority of responding FMR PDs in this survey reported that they utilized simulation as an educational tool, for both education and assessment. PDs also reported access to dedicated locations to utilize SBME. Most FMR programs using simulation utilize it for clinical skills and medical knowledge core competencies. Although studies in other fields have highlighted the value of simulation in improving professionalism,19,20 executive leadership,21 and skills in systems-based practices,22 such as addressing patient safety,21, 23 FMR PDs here do not report using SBME to affect these competencies. In undergraduate medical education, simulation is also successfully being used to teach interprofessional teamwork.24

These results suggest that FMRs are not leveraging SBME to its greatest potential. In this sample, PD perceptions of value were related to the use of simulation. This may indicate one potential target for intervention. FMR educators may not be aware of these possible applications or may not be trained to adapt simulation to include these competencies. More faculty development may be needed to share the evidence on the positive effects of SBME.

A majority of respondents, even those utilizing SBME, reported barriers to simulation. A lack of support from sponsoring institutions can be related to lack of investment in simulation infrastructure, changing educational culture, or training faculty. In this study, residency programs with characteristics that are associated with more resources, such as larger size and university affiliation, were just as likely as their counterparts to report barriers to simulation use. The top-three barriers reported from FMR programs include lack of resident and faculty time, lack of support from teaching hospitals, and lack of faculty training. Effective SBME requires time investment for proper coordination, development, and execution.1 Time and resources are also needed for organized faculty development, including training faculty in SBME-specific skills such as debriefing. 25 Organizational leaders can enable implementation of SBME by resourcing spaces, equipment, and staff. 26

Findings are limited by the self-selection of respondents to participate. Although the response rate was below the recommended 60% threshold,27 one strength is that respondents represent a wide distribution of residency types and sizes. Respondents may not have had a shared mental model of the difference in simulation for assessment versus for education, which could limit application of findings. It is possible that the respondent group was systematically more or less interested in simulation than non-respondents; however, the simulation questions were among others about unrelated subjects in family medicine education, lessening this concern. This study was also conducted just before the release of a new milestone evaluation system by the ACGME in July 2020,28 which may subjectively change responders’ interpretation of their use of simulation to evaluate milestones.

Future intervention and implementation studies are needed to identify how to overcome barriers and how to make SBME more sustainable and effective for FMR programs. Data on patient safety and interprofessional team care would greatly improve the understanding of how SBME can be effective.

References

  1. Motola I, Devine LA, Chung HS, Sullivan JE, Issenberg SB. Simulation in healthcare education: a best evidence practical guide. AMEE Guide No. 82. Med Teach. 2013;35(10):e1511-e1530. doi:10.3109/0142159X.2013.818632
  2. Raleigh MF, Wilson GA, Moss DA, et al. Same content, different methods: comparing lecture, engaged classroom, and simulation. Fam Med. 2018;50(2):100-105. doi:10.22454/FamMed.2018.222922
  3. Moazed F, Cohen ER, Furiasse N, et al. Retention of critical care skills after simulation-based mastery learning. J Grad Med Educ. 2013;5(3):458-463. doi:10.4300/JGME-D-13-00033.1
  4. Singer BD, Corbridge TC, Schroedl CJ, et al. First-year residents outperform third-year residents after simulation-based education in critical care medicine. Simul Healthc. 2013;8(2):67-71. doi:10.1097/SIH.0b013e31827744f2
  5. Issenberg SB, McGaghie WC, Petrusa ER, Lee Gordon D, Scalese RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach. 2005;27(1):10-28. doi:10.1080/01421590500046924
  6. Cooke JM, Larsen J, Hamstra SJ, Andreatta PB. Simulation enhances resident confidence in critical care and procedural skills. Fam Med. 2008;40(3):165-167.
  7. Gardner AK, Lim G, Minard CG, Guffey D, Pillow MT. A cross-specialty examination of resident error disclosure and communication skills using simulation. J Grad Med Educ. 2018;10(4):438-441. doi:10.4300/JGME-D-17-00603.1
  8. Peterson EB, Porter MB, Calhoun AW. A simulation-based curriculum to address relational crises in medicine. J Grad Med Educ. 2012;4(3):351-356. doi:10.4300/JGME-D-11-00204
  9. Ledford CJ, Seehusen DA, Canzona MR, Cafferty LA. Using a teaching OSCE to prompt learners to engage with patients who talk about religion and/or spirituality. Acad Med. 2014;89(1):60-65. doi:10.1097/ACM.0000000000000062
  10. Cooke JM, Wimsatt, L. Simulation in Family Medicine. In Levine AI, et al. eds., The Comprehensive Textbook of Healthcare simulation. New York: Springer Science+Business Media, 2013: 341-351. doi:10.1007/978-1-4614-5993-4_22
  11. Burden AR, Pukenas EW, Deal ER, et al. Using simulation education with deliberate practice to teach leadership and resource management skills to senior resident code leaders. J Grad Med Educ. 2014;6(3):463-469. doi:10.4300/JGME-D-13-00271.1
  12. Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system—rationale and benefits. N Engl J Med. 2012;366(11):1051-1056. doi:10.1056/NEJMsr1200117
  13. McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med. 2011;86(6):706-711. doi:10.1097/ACM.0b013e318217e119
  14. Issenberg SB, McGaghie WC, Hart IR, et al. Simulation technology for health care professional skills training and assessment. JAMA. 1999;282(9):861-866. doi:10.1001/jama.282.9.861
  15. Wendling A, Baty P. A step ahead—evaluating the clinical judgment skills of incoming interns. Fam Med. 2009;41(2):111-115.
  16. Klee D. The Roles of High-Fidelity Simulation in a Family Medicine Residency. STFM Educational Column. Published January 2016. Accessed March 3, 2020. https://www.stfm.org/publicationsresearch/publications/educationcolumns/2016/january/
  17. Passiment M, Sacks H, Huang G. Medical simulation in medical education: results of an AAMC survey. Washington, DC: Association of American Medical Colleges; 2011.
  18. Mainous AG III, Seehusen D, Shokar N. CAFM Educational Research Alliance (CERA) 2011 Residency Director survey: background, methods, and respondent characteristics. Fam Med. 2012;44(10):691-693.
  19. Larkin AC, Cahan MA, Whalen G, et al. Human Emotion and Response in Surgery (HEARS): a simulation-based curriculum for communication skills, systems-based practice, and professionalism in surgical residency training. J Am Coll Surg. 2010;211(2):285-292. doi:10.1016/j.jamcollsurg.2010.04.004
  20. Guerrasio J, Aagaard EM. Long-term outcomes of a simulation-based remediation for residents and faculty with unprofessional behavior. J Grad Med Educ. 2018;10(6):693-697. doi:10.4300/JGME-D-18-00263.1
  21. Rosen MA, Goeschel CA, Che XX, et al. Simulation in the executive suite: lessons learned for building patient safety leadership. Simul Healthc. 2015;10(6):372-377. doi:10.1097/SIH.0000000000000122
  22. Crawford SB, Monks SM, Mendez M, Quest D, Mulla ZD, Plavsic SK. A Simulation-based workshop to improve residents’ collaborative clinical practice. J Grad Med Educ. 2019;11(1):66-71. doi:10.4300/JGME-D-18-00209.1
  23. Weis JJ, Croft CL, Bhoja R, Lee JH, Scott DJ. Multidisciplinary simulation activity effectively prepares residents for participation in patient safety activities. J Surg Educ. 2019;76(6):e232-e237. doi:10.1016/j.jsurg.2019.07.015
  24. Cyr PR, Schirmer JM, Hayes V, Martineau C, Keane M. Integrating interprofessional case scenarios, allied embedded actors, and teaching into formative observed structured clinical exams. Fam Med. 2020;52(3):209-212. doi:10.22454/FamMed.2020.760357
  25. Cheng A, Grant V, Dieckmann P, Arora S, Robinson T, Eppich W. Faculty development for simulation programs: five issues for the future of debriefing training. Simul Healthc. 2015;10(4):217-222. doi:10.1097/SIH.0000000000000090
  26. Ferguson J, Astbury J, Willis S, Silverthorne J, Schafheutle E. Implementing, embedding and sustaining simulation based education: what helps, what hinders. Med Educ. 2020; published online ahead of print, 2020; 54:915–924. doi:10.1111/medu.14182
  27. Johnson TP, Wislar JS. Response rates and nonresponse errors in surveys. JAMA. 2012;307(17):1805-1806. doi:10.1001/jama.2012.3532
  28. Edgar L, Roberts S, Holmboe E. Milestones 2.0: a step forward. J Grad Med Educ. 2018;10(3):367-369. doi:10.4300/JGME-D-18-00372.1

Lead Author

Johnny C. Tenegra, MD, MS

Affiliations: Department of Family and Community Medicine, Southern Illinois University School of Medicine, Springfield, IL

Co-Authors

M. Rebecca Hoffman, MD, MSPH - Department of Family and Community Medicine, Southern Illinois University School of Medicine, Springfield, IL

Georgia S. Mueller Luckey, PhD, MS - Department of Family and Community Medicine, Southern Illinois University School of Medicine, Springfield, IL

Lisabeth F. DiLalla, PhD - Department of Family and Community Medicine, Southern Illinois University School of Medicine, Springfield, IL

Christy J.W. Ledford, PhD - Department of Family Medicine, Medical College of Georgia, Augusta, GA

Fetching other articles...

Mendeley

Loading metrics from Mendeley...

Loading the comment form...

Submitting your comment...

There are no comments for this article.

Downloads & Info

Share

Tags

Searching for articles...