Introduction: Despite near-universal utilization of electronic health records (EHRs) by physicians in practice, medical students in most ambulatory settings gain limited experience with placing EHR orders. In this study, an individual preceptor site investigated the usefulness of a targeted curriculum in improving students’ EHR confidence and clinical reasoning skills.
Methods: Family medicine clerkship students assigned to one community health center were invited to participate in this prospective, survey-based study. In their first week, students observed a preceptor performing EHR tasks. For the remainder of the 4-week clerkship, students utilized decision support tools, assigned a working diagnosis, entered unsigned orders in the EHR, proposed an assessment, and discussed a plan with a preceptor. Students completed weekly questionnaires to self-report confidence across several EHR domains while preceptors synchronously evaluated students’ accuracy with entering orders correctly.
Results: From February 2017 to March 2020, all 49 eligible students completed the study. One hundred percent of students reported that placing EHR orders was beneficial to their medical education. The difference over time in learner confidence with placing EHR orders was statistically significant across every domain (eg, writing prescriptions, ordering labs and imaging). Preceptors’ evaluations of students’ accuracy with placing orders also showed significant improvement between each week.
Conclusion: Clerkship-wide EHR training may be limited by multiple sites with multiple EHR products. This pilot study suggests that committed faculty at an individual preceptor site can offer a targeted curriculum to help students develop EHR confidence. We propose other preceptors similarly offer students this opportunity to maximize clerkship education.
To improve care coordination, patient safety, and health outcomes, more than 85% of office-based physicians have adopted an electronic health record (EHR).1 Fittingly, the Association of American Medical Colleges (AAMC), Alliance for Clinical Education (ACE), and Society of Teachers of Family Medicine (STFM) agree that basic EHR proficiency is essential in medical education.2-4
A recent search offered several general studies on student EHR access.5-7 These publications suggest many family medicine clerkship students have EHR access and can write notes.8,9 However, only one-third of students enter electronic orders and even fewer enter prescriptions.8 Although class-wide workshops occur with simulated EHRs, we are not aware of published studies that train ambulatory clerkship students to perform higher-order EHR tasks on real patients.10 Differing EHR products across multiple clinical sites present an ongoing challenge to integrating EHR instruction into clerkship-wide education. Preceptors at individual sites may be uniquely poised to address this gap. This study aims to determine whether a targeted EHR curriculum offered by an individual preceptor site can improve students’ confidence with placing EHR orders. The objective of this curricular intervention is to enable participating students to independently and accurately enter orders for diagnostic tests, prescriptions, and other management plan items.
The Institutional Review Board of Baylor College of Medicine (BCM) approved this research study. Participation in the EHR training intervention and completion of weekly surveys were voluntary and not a clerkship requirement. As students were in an established educational environment and evaluating a learning activity, documentation of written consent was waived.
The BCM Family and Community Medicine (FCM) clerkship is a required 4-week ambulatory rotation for second- or third-year students. BCM students enter clinical rotations 18 months into their medical education. Preceptors at one community site initiated the idea of providing a targeted Epic curriculum and intentionally guiding students to gain EHR proficiency. All 49 clerkship students assigned to the clinic during the study period were invited to participate.
Description of the Intervention
To prepare the other nine faculty at the site, two lead preceptors (A.A. and N.M.) held an orientation session during a monthly clinic meeting. Total training time for nonlead faculty on how to teach students to enter EHR orders was 2 hours. The two lead preceptors observed the students directly and completed weekly 7-item surveys to evaluate order entry accuracy. Preceptors estimated the overall percentage of correct orders placed per week and recorded these observations on a quintile scale. On days that students were assigned to nonlead preceptors, order accuracy was not recorded. The questionnaires used in this study were developed by the authors.
In a second-year course entitled “Transitions to Clerkships,” all BCM students receive a general Epic EHR orientation on chart review and note writing. The FCM clerkship does not offer a standardized EHR orientation as different products are used at individual preceptor sites. In this study, students received a refresher on ambulatory Epic functionality and a basic overview on order entry on their first day.”
In their first week, enrolled students performed a history and physical, presented findings to a preceptor, and observed the preceptor complete EHR tasks. In the ensuing 3 weeks, students conducted a history and physical, utilized decision support tools (eg, UpToDate and Epic SmartSets), assigned a working diagnosis, entered unsigned orders in the EHR, proposed an assessment, and reviewed a plan with the preceptor. Students completed weekly 10-item questionnaires to self-report confidence across eight EHR domains using a 10-point Likert scale. Two additional items queried the benefit of entering EHR orders and using decision support tools to students’ overall education.
We compared students’ confidence in each domain at different time points using a Friedman test. We followed this by pairwise comparisons with a Bonferroni correction for multiple comparisons. We also analyzed preceptors’ evaluation of student accuracy over time using a Friedman test followed by pairwise comparisons with a Bonferroni correction. We carried out analyses using SPSS software version 26 (SPSS, Chicago, IL).
From February 2017 to March 2020, 100% of the 49 eligible students completed the study and surveys; 100% of participants reported that placing EHR orders positively impacted their medical education (Table 1). Three students’ clerkships were shortened to 3 weeks due to the COVID-19 pandemic. Questionnaires from the remaining 46 of 49 (94%) respondents were utilized in the 4-week analysis. Increases over time in student confidence with placing EHR orders across every domain were statistically significant (P values <.001 for all comparisons). Weekly increases in student accuracy via preceptor evaluation (Table 2) were also statistically significant (P values <.03 for all comparisons). Trainees’ initial confidence was lowest in ordering outpatient therapy and linking orders with associated diagnoses. These two domains experienced the greatest increase in confidence over the study period.
This study describes an individual preceptor site offering a targeted EHR curriculum to clerkship students. Preceptors intentionally gave students opportunities to utilize decision support tools, place pended EHR orders, and receive frequent feedback. Students reported increased levels of confidence in writing prescriptions, ordering labs and imaging, and using the EHR overall. Preceptor-rated student accuracy in completing orders similarly increased. By the clerkship’s end, students unanimously reported that placing EHR orders was beneficial to their medical education.
One of the AAMC’s Core Entrustable Professional Activities for Entering Residency is to “enter and discuss orders and prescriptions.”11 To this end, several inpatient clerkships have reported variable success with training students to learn computerized order entry.12-15 Alternatively, some studies propose a simulated EHR to teach students to write orders on a standardized patient.16,17 Nevertheless, more effort is needed to train students to use EHRs and enter orders in the ambulatory setting on actual patients. Our findings suggest EHR training at an individual preceptor site can be useful in increasing students’ confidence and accuracy in entering orders. Evaluation of students' skill in order entry was not incorporated into their clerkship assessment or grade.
The two lead preceptors provided weekly feedback to students on order entry accuracy in addition to traditional history and exam skills. Faculty reported that by week 2, students’ pended orders saved time that could be reinvested to precepting efforts. Developing additional curricula on order entry could be useful beyond education to help students contribute to the practices where they learn.
This study demonstrates it is possible for committed preceptors at an individual site to train students in EHR order entry beyond what is offered in the general clerkship curriculum. Although two lead preceptors oriented and evaluated the participants, the engagement of nine other faculty at the site promoted a consistent and continuous EHR training experience.
There is limited literature on how individual preceptor sites contribute to the learning of family medicine clerkship students. This study indicates an individual preceptor site can contribute to EHR education beyond what is offered in the general clerkship curriculum.
The two lead preceptors who evaluated the students are authors of this manuscript. The study offered EHR training for all students and did not feature control participants. Although all students reported increased confidence and all preceptors reported corresponding increased accuracy with orders, correlations between the students’ and preceptors’ responses could not be performed due to limited sample size. We did not specifically analyze differences in EHR confidence between second- and third-year clerkship students. Preceptor questionnaires were completed by two individuals; we did not perform interrater reliability studies. Finally, this study was completed at a single site and may not be generalizable to other practice sites or institutions.
Despite these limitations, this study demonstrates that a single-site initiative wherein preceptors intentionally train and guide students in EHR use can improve students’ confidence with higher-order EHR tasks. Encouraging preceptor sites to develop similar approaches may be useful for clerkships with multiple sites and various EHRs. Further study into the efficacy of this approach will enable family medicine educators to determine the best way to help students actively participate in patient care and employ the underutilized EHR arena.
Presentations: Portions of this study were presented as “Electronic Order Entry as an Educational Tool in a Family Medicine Preceptor Site” at the 2018 Society of Teachers of Family Medicine Medical Student Education Conference, February 1-4, in Austin, Texas.
*Authorship Note: Dr Arindam Sarkar and Dr Anjali Aggarwal contributed equally to this work.
- National Electronic Health Records Survey: Percentage of office-based physicians using any electronic health record (EHR)/electronic medical record (EMR) system. CDC National Center for Health Statistics. Published 2017. Accessed August 28, 2020. https://www.cdc.gov/nchs/fastats/electronic-medical-records.htm
- Compliance Advisory: Electronic Health Records (EHRs) in Academic Health Centers. Association of American Medical Colleges. Published 2014. Accessed August 28, 2020. https://www.aamc.org/em/aamc/compliance_advisory.pdf.
- Alliance for Clinical Education. Letter of Support to LCME emphasizing Standard 7.8 Curricular Content - Communication Skills. Published 2015. Accessed August 28, 2020. http://allianceforclinicaleducation.org.
- Working with medical students using electronic health records: guidelines and best practices for clinical preceptors. Society of Teachers of Family Medicine. Published June 2013. Accessed August 28, 2020. http://www.stfm.org/Resources/StudentUseofElectronicHealthRecords.
- Rouf E, Chumley HS, Dobbie AE. Electronic health records in outpatient clinics: perspectives of third year medical students. BMC Med Educ. 2008;8(1):13. doi:10.1186/1472-6920-8-13
- Tierney MJ, Pageler NM, Kahana M, Pantaleoni JL, Longhurst CA. Medical education in the electronic medical record (EMR) era: benefits, challenges, and future directions. Acad Med. 2013;88(6):748-752. doi:10.1097/ACM.0b013e3182905ceb
- Habboush Y, Hoyt R, Beidas S. Electronic health records as an educational tool: viewpoint. JMIR Med Educ. 2018;4(2):e10306. doi:10.2196/10306
- Huang WY, Grigoryan L, Aggarwal A. Predictors of student use of an electronic record. Clin Teach. 2019;16(2):131-137. doi:10.1111/tct.12785
- White J, Anthony D, WinklerPrins V, Roskos S. Electronic medical records, medical students, and ambulatory family physicians: a multi-institution Study. Acad Med. 2017;92(10):1485-1490. doi:10.1097/ACM.0000000000001673
- Zavodnick J, Kouvatsos T. Electronic health record skills workshop for medical students. MedEdPORTAL. 2019;15(1):10849. doi:10.15766/mep_2374-8265.10849
- Core Entrustable Professional Activities for Entering Residency Drafting Panel. Core entrustable professional activities for entering residency: faculty and learner’s guide. Association of American Medical Colleges. Published 2015. Accessed August 28, 2020. https://store.aamc.org/downloadable/download/sample/sample_id/66/%20.
- Stair TO, Howell JM. Effect on medical education of computerized physician order entry. Acad Med. 1995;70(6):543. doi:10.1097/00001888-199506000-00019
- Patterson R, Harasym P. Educational instruction on a hospital information system for medical students during their surgical rotations. J Am Med Inform Assoc. 2001;8(2):111-116. doi:10.1136/jamia.2001.0080111
- Knight AM, Kravet SJ, Kiyatkin D, Leff B. The effect of computerized provider order entry on medical students’ ability to write orders. Teach Learn Med. 2012;24(1):63-70. doi:10.1080/10401334.2012.641490
- Wack M, Puymirat E, Ranque B, et al. Evaluating the impact of computerized provider order entry on medical students training at bedside: a randomized controlled trial. PLoS One. 2015;10(9):e0138094. doi:10.1371/journal.pone.0138094
- Milano CE, Hardman JA, Plesiu A, Rdesinski RE, Biagioli FE. Simulated electronic health record (Sim-EHR) curriculum: teaching EHR skills and use of the EHR for disease management and prevention. Acad Med. 2014;89(3):399-403. doi:10.1097/ACM.0000000000000149
- Biagioli FE, Elliot DL, Palmer RT, et al. The electronic health record objective structured clinical examination: assessing student competency in patient interactions while using the electronic health record. Acad Med. 2017;92(1):87-91. doi:10.1097/ACM.0000000000001276