Educational Podcast Impact on Student Study Habits and Exam Performance

By offering on-demand access to educational content, podcasts have featured more prominently in curricula as mobile and asynchronous learning grow in medical education.^1-7 Podcasts are well suited for self-directed learning (SDL), with learners independently selecting resources to meet their needs.8 Research suggests podcast listeners feel connected to the larger professional community, creating a social context that is often lacking in SDL and providing an environment for the informal lessons at the core of sociocultural learning theory.^8-10

The literature on podcasts in medical education has grown in recent years, but several gaps remain.^11, 12 Studies consistently find that podcasts are widely used, engaging, and preferable to text resources because of ease of use and ability to multitask while listening.^10-15 Knowledge retention from podcasts appears comparable to traditional tools, but these studies were performed in controlled settings that may not correspond to real-world podcast use.^15, 16, 17 Few reports on podcasts in pediatrics have been published and none have evaluated educational outcomes.^18- 21

We examined how typical use of an educational podcast impacted students’ learning during pediatrics clerkship. We surveyed students’ study habits, assessed perceptions of the podcast, and compared users’ and nonusers’ scores on the National Board of Medical Examiners (NBME) pediatrics examination.

We conducted a nonrandomized prospective study of the impact of an educational podcast during pediatrics clerkship. All third-year medical students at our institution in the 2019-2020 academic year were eligible. The Medical College of Wisconsin Institutional Review Board approved the study.

We developed a podcast titled “Peds Soup,” drawing topics from the American Board of Pediatrics content outline.^22, 23 Scripts were based on relevant literature then recorded and edited using Audacity software.²⁴ Episodes were 10-20 minutes long and met widely-accepted quality criteria for educational podcasts.²⁵

The study was introduced during clerkship orientation with instructions for accessing the podcast. The podcast was freely available to students regardless of their participation in the study and all faculty were blinded to students’ participation status. We distributed surveys via email during the last week of the clerkship.

We collected data using a Qualtrics survey adapted from previous studies of podcast utility and medical students’ study habits.^21, 26 We asked all participants whether they used Peds Soup during the clerkship, their likelihood of pursuing a pediatrics career, their perceived utility of common study resources, and the average number of hours they spent studying each week. Listeners rated the podcast’s overall quality and impact on their knowledge and clinical performance and answered questions about their perceptions of Peds Soup and where and when they listened to the podcast. Participants provided their United States Medical Licensing Examination numbers, which were used to locate exam scores and demographic information. We mapped survey data to evaluate a Kirkpatrick Level 1 reaction impact and exam scores to evaluate a Kirkpatrick Level 2 knowledge gain impact.²⁷

We classified participants as podcast users and nonusers by their self-reported use of Peds Soup and conducted χ² tests for demographic analysis. We calculated free-response answers for common themes and refined into categories by the authors until consensus was reached. All analyses were performed with R software version 4.0.0 (Vienna, Austria) and IBM SPSS 24.0 (Armonk, NY) with a significance threshold of P<.05.

Out of 204 eligible students, 83 (41%) participated in the study. Forty-three students (52%) reported using the Peds Soup podcast. There were no significant differences between users and nonusers in the available demographic data (Table 1).

There was no significant difference in mean NBME examination scores between the groups with podcast users recording a mean score of 78.0 and standard deviation of 7.6 compared to a mean of 77.1 and standard deviation of 8.8 among nonusers (P=.609). Podcast users rated podcasts as the second most useful study resource and rated podcasts significantly higher than nonusers (Table 2). The two groups rated all other resources similarly.

Users were more interested in pediatrics careers than nonusers (Table 3). Median interest ratings corresponded to “Neutral” for users and “Somewhat Unlikely” for nonusers. Users also reported significantly more independent study hours per week (M=16.5, SD=9.0 vs M=12.4, SD=9.2, P=.009).

Listeners gave Peds Soup high scores for overall rating (M=8.1, SD=1.4) and impact on knowledge of pediatrics (M=7.3, SD=1.5; Table 4). Ratings on perception questions were generally high, with only “Is something I plan to continue listening to” receiving a median score below “Strongly Agree.”

Responses for where and when users listened to podcasts fell primarily into the “driving” category (65%), followed by “exercising” (30%), “doing chores” (23%), and “at home” (9%). Percentages total more than 100% because responses could be counted in multiple groups.