Using Social Media to Teach About and Engage Residents in Evidence-Based Medicine

Teaching evidence-based medicine (EBM) has been studied in medical education and is a key component in the Accreditation Council for Graduate Medical Education (ACGME) milestones. ^1-6 However, program directors have reported issues protecting resident time for teaching EBM, having faculty unprepared to deliver feedback on EBM skills, and being skeptical overall about EBM. ⁷ When EBM is delivered in a structured, longitudinal way, learners have significant increases in comfort and confidence using EBM through a validated assessment tool. ^{8, 9}

Social media (SM) is a technology-based platform where users create and interact with content on a global scale.¹⁰ SM use in medical education is growing with an exponential rise in publications since 1996.^10-12 A recent review found X (formally Twitter) was used in 157 publications, Facebook in 103, and blogs and podcasts in 176. ¹¹ Learners use SM for professional networking, acquisition of procedural skills, and recruitment. ^12-16 Studies have shown that SM is an effective adjunct to didactic content, improving learner engagement and amplifying publication views. ^17-22 Our purpose was to create a novel method of EBM teaching that uses SM to disseminate content and to evaluate its effectiveness.

We aimed to describe learners’ (a) current SM use; (b) current engagement, confidence, and comfort with EBM principles and resources; and (c) change in knowledge of EBM concepts and emerging clinical information. UPMC has eight family medicine residency programs, with a total of 196 learners in the 2021-2022 academic year.

This project was supported by the American Board of Family Medicine through a $5,000 grant to promote its National Journal Club initiative where recipients were asked to innovate EBM teaching. The Institutional Review Board of the University of Pittsburgh approved this study.

In April 2022, the cohort of 196 learners was emailed study information and asked to complete an anonymous preintervention survey. Starting in May 2022, our video intervention was posted weekly for 6 weeks to our program’s SM, which included YouTube, Instagram, Facebook, and X (formerly known as Twitter). Additionally, the cohort was emailed the same links weekly for rolling recruitment. Because anyone could access our SM, emails ensured that only members of the cohort responded to the surveys. The final video was shared in June along with the postintervention survey. Participants received a $5 gift card for completing each survey; those completing both were entered into a raffle for $100 gift cards at study completion.

Each video was divided into two parts: case scenarios and EBM teaching. During the case portion, learners followed an investigation of clinical questions using the latest evidence. Each incident is investigated using storytelling to illustrate trial outcomes. During the EBM teaching portion, the video used an article as a basis to review concepts. Table 1 provides an overview of the articles reviewed, concepts taught, and overall viewership.

This quasi-experimental study used a pre- and postintervention survey. Surveys were reviewed by researchers at the University of Pittsburgh. Respondent data from the two surveys were linked with a unique identifier using Qualtrics (Qualtrics LLC). The preintervention survey included (a) demographics; (b) SM usage in general and for medical education; (c) EBM engagement; (d) EBM comfort/confidence; and (e) questions pertaining to each video (a clinical concept and an EBM concept). EBM comfort/confidence was identified via an adapted validated scale. ⁸ The postintervention survey consisted of (a) the same questions related to the videos for recall; (b) an assessment of the intervention; and (c) overall engagement.

Forty-four of a possible 196 learners (22.4%) participated in the preintervention survey. Respondents included 13 postgraduate year 1 family medicine residents, 11 postgraduate year 2 family medicine residents, 10 postgraduate year 3 family medicine residents, 3 fellows, and 7 pharmacy residents. Respondents identified residency didactics as the main resource to review primary medical literature (86%) followed by journal websites (61%), applications (52%), podcasts (34%), and SM (23%). For EBM engagement, 89% of respondents reported spending at least 30 minutes daily using secondary resources to review literature for learning purposes. Most respondents (55%) strongly agreed that EBM was part of their residency’s culture, and 80% identified EBM as important. EBM confidence/comfort is reported in Table 2. Respondents were least comfortable with critically appraising study methods. SM respondent use is reported in Figure 1. Of note, 52% used SM and podcasts 15 or fewer minutes per week for EBM.

Overall, respondents scored 64% cumulatively on the preintervention survey of the EBM questions that were subsequently covered in the video series. Question results with the lowest scores were about the effects of egg consumption on cardiovascular disease (11%), the effects of lower blood pressure on mortality (36%), and the purpose of propensity scoring (43%).

Videos averaged 334 views (200 to 427 views). Following the videos, seven respondents completed the postintervention survey. Respondents used YouTube and Instagram for video access. Overall, respondents scored 85% cumulatively on the postintervention survey of the video-related questions. Only one respondent’s score decreased from the preintervention survey and four participants’ scores improved. All respondents agreed or strongly agreed that the series enhanced their EBM knowledge and that they would share a similar video series with coworkers/learners.

To our knowledge, this is one of the first studies on engaging learners in family medicine residency programs using SM to disseminate EBM teaching. Respondents noted that their primary source of emerging literature comes from didactics. Given that ACGME milestones include EBM concepts, we feel that learners should practice finding and evaluating the best available evidence. Our project encouraged SM use for EBM content and taught concepts through educational videos. Importantly, postintervention results showed improvement, which is encouraging, for the use of SM to teach.

Respondents to the preintervention survey preferred to engage with medical education on SM built around videos (YouTube and Instagram). This may represent a shift from older platforms like X and Facebook where videos are less common and shorter.^{28, 29} Given this finding, we hypothesize that content for learners may best be created in video format. Average video views (334) exceeded the total number in the cohort (196), meaning that videos either were watched multiple times or reached beyond our cohort (Table 1). Anecdotally, applicants to our residency program mention the video series in interviews. SM can be accessed asynchronously and without intent to learn. Educators may want use SM to capitalize on these attributes.

Respondents’ lowest self-assessment in the EBM comfort/confidence scale focused on their ability to apply EBM to articles (Table 2). These included the ability to critically appraise evidence, evaluate treatment effectiveness, and determine the relevance to patients. Our videos attempted to influence these areas by equally distributing allotted time to education and critical appraisal. Learners felt most confident identifying their own knowledge gaps.

Limitations include low postintervention response rate and not evaluating changes in EBM comfort/confidence in this survey. To remove the burden of completing the scale a second time, we decided not to collect these data because the survey was distributed in the final week of an academic year, where residents’ attention to it may have been diminished. These data may have provided insight on EBM comfort/confidence following the series. Nonetheless, postintervention scores on the video-related questions increased, perhaps showing that confidence/comfort in critical appraisal increased.

The results of this project describe EBM comfort and use of SM in medical education. Further studies are needed to understand the relationship SM may have with learner retention and engagement.