Brief Video-Based Suicide Prevention Training for Primary Care

Suicide is the tenth leading cause of death among adults in the United States, and rates are rising.¹ In 2015, an estimated 1.3 million US adults attempted suicide.² About one-third of individuals who die by suicide had primary care contact in the month before their death.^3,4 Screening for depression and suicide risk is increasingly common in primary care, and family medicine faculty have advocated for improved detection, documentation, and response to suicide risk.⁵ More than one-quarter of states have policies that require or encourage suicide prevention for health care professionals to improve engagement, assessment, and response to suicide concerns.⁶ Unfortunately, most medical education provides minimal training in suicide prevention, even in mental health settings,⁷ and to our knowledge, no postgraduate or continuing education training for primary care has yet been evaluated.

Time constraints in primary care education present a key challenge for suicide prevention training. Training content must be engaging and practical for use in brief patient encounters, and delivery must fit within busy schedules and competing educational demands. Online learning is a viable option for delivering new resources in this context, for example through video-based instruction, webinars, online office hours, and hosted text-based materials.⁸

Here we describe the evaluation of Commitment to Living-Primary Care (CTL-PC), an innovative, brief online training for primary care providers. This training draws on visual concept mapping, an education technique based on constructivist learning theory^9,10 and successfully tested elsewhere in health care training.^11-16 CTL-PC was adapted from an evidence-based training for behavioral health professionals in which learners from diverse disciplines and training levels showed improvements in knowledge, confidence, and objectively-rated risk assessments.¹⁷ The primary care version was first developed in 2010 in the context of pediatric residency training,¹⁸ and was refined into video-based modules in 2015 as part of a broader patient safety project studying the impact of video-based instruction and patient simulation.¹⁹ The professionally filmed videos consisted of instruction interspersed with concept diagrams, with key practice elements added to the visual concept map as each was discussed (Table 1). A pocket card based on this mapping and distributed to learners is shown in Figure 1.

The participants (n=132) were second-year University of Rochester (UR) School of Nursing nurse practitioner trainees (NPs, n=67) and second-year medical residents at the UR Medical Center (n=65). The demographics and suicide-related experiences of the participant group are summarized in Table 2. Nineteen of the residents were drawn from family medicine and 46 from pediatrics and medicine-pediatrics programs. Overall, the mean hours of suicide risk assessment training per participant at baseline was 0.86 (SD=2.38, range=0-20 hours). Among those with previous training, the mean was 3.05 hours. We chose these groups because most learners were preparing for careers in primary care. The training directors of the nurse practitioner and residency programs agreed to incorporate suicide prevention training into their curriculums for the duration of the study and to support randomization of learners. All trainees from these programs received the training and engaged in a consent process (conducted by study personnel) for analysis of their data. One hundred forty trainees received the training, of whom 132 consented to have their data analyzed, and eight declined. Of the 132 learners who consented, four dropped out and play rate data for one learner was lost to a technical error. Results for 127 (90.7%) of the 140 eligible learners are included in our study.

We assessed participants’ knowledge of suicide prevention in primary care before and after training using 17 question items. An example question is:

No suicide contracts (contract for safety) are recommended to: A. prevent self-injury; B. protect the clinician from legal challenge if the patient later harms him/herself; C. A and B; D. None of the above (contracts are not recommended)

We also assessed learners’ self-efficacy before and after training, using a 20-item Likert scale from 1 (strongly disagree) to 8 (strongly agree; α=.97). Example prompts included “I feel confident in my ability to determine a follow-up plan for patients who are suicidal”; and “I feel confident that I can ask directly about suicide.” After training, we assessed participants’ satisfaction with the modules (4-item Likert scale: 1=not at all satisfied to 5=very satisfied, eg: “It is clear to me that the developers of the training understand how I will use what I learn”; “I will be able to use this training in my job”), and their perception of their ability to transfer learning to a clinical setting using the Perception of Training Transfer scale^17,18 (8-item Likert scale from 1=strongly disagree to 5=strongly agree, eg, satisfaction with video quality, duration, realism of examples, overall satisfaction). We established levels of engagement with the educational material using direct measures of how much of each video participants watched, and levels of satisfaction with the material through self-report.

Following baseline assessments, participants received a link to a website hosting the videos. We presented participants with 21 professionally-produced, brief instructional videos (six modules, total 48 minutes). It was compulsory to load the page hosting each video, but learners could choose whether to click “play” and how much of each video to watch. Some programs instructed participants to watch the complete sequence of videos in one sitting while others allowed 2 weeks. The learning system retained aggregated data for play rate and time watched for each video. At completion, learners were assessed for knowledge, self-efficacy, perception of training transfer, and satisfaction with the modules. Open-ended comments were gathered 6 months later as part of the broader study within which this evaluation was embedded.

We calculated play rate and average engagement in aggregate across all learners. Play rate is defined as the proportion of participants who go on to click play after landing on a video host page, calculated by dividing the total number of unique plays by the number of unique page views. We calculated average engagement by dividing the total number of minutes of a video watched across learners by total plays X video duration. To statistically assess change in knowledge and self-efficacy, we examined paired t tests and effect sizes (Cohen’s d – [post μ – baseline μ/pooled α]). To assess for group differences in change (NPs versus residents), we utilized repeated measures analyses with time (pre/post), learner group (residents/NP), and the time by learner group interaction as the independent variables. We analyzed the data using IBM SPSS Statistics for Windows, Version 24.0.

The University of Rochester Institutional Review Board approved the curriculuim and evaluation in December 2016, as study number RSRB00061161.

The level of engagement, measured by the proportion of each video watched, was high. The mean play rate for videos was 79.62%. The play rate for the videos in the first three modules varied considerably from 62%-84%, but stabilized between 79%-88% for the final three modules. When a video was started, a mean of 83.19% was watched. Of the 21 videos, 17 were watched for at least 80% of their length.

At baseline, learners correctly answered a mean of six of 17 questions (35.6%, range: 11.7%-70.6%), rising to 11 out of 17 (64.3%, range: 5.8%-100%) after the training. For knowledge, change was statistically significant (t_{(131 df)}=19.91, P<.001), with an effect size of 1.73. Repeated measures analysis demonstrated that these gains were comparable across NP and resident learner groups, with 82% and 80% gains, respectively (time by group interaction F=.84, P=.36). Knowledge scores among residents started and ended higher (the main effect for learner group was statistically significant, F=5.75, P=.02).

Confidence in ability to assess and manage suicide risk increased (P<.001; Cohen’s d=1.42). The mean baseline score was 3.57 (moderate disagreement), rising to 5.07 (moderate agreement) after the video-based training. Gains were statistically significant (t_{(131 df)}=16.31, P<.001), with an effect size of 1.42. Repeated measures analysis showed higher gains among NPs (repeated measures time by group interaction F=9.74, P=.002). NPs rose from a baseline mean of 3.29 to 5.07, for an average gain of 1.78; residents rose from a higher baseline of 3.86 to an identical mean of 5.07, for a gain of 1.21. The between learner groups main effect was not statistically significant (F=2.19, P=.14).

Participants rated their perception of their ability to transfer the training into practice with a mean score of 3.32 (moderate agreement; SD=.90), in line with the findings of our previous evaluations.17 There was no statistically significant difference (F=.43, P=.52) between residents (mean=3.27, SD=.92) and NPs (mean=3.37, SD=.88). In open-ended comments, participants asked for more examples of plans and standardized responses to patients.

Satisfaction with the modules was high; 118 participants (89.5%) rated their satisfaction with the overall quality of the modules between 3 and 5 (satisfied to very satisfied). Only one participant (0.8%) scored the overall quality with a 1 (not at all satisfied). One hundred three participants (78.1%) reported being satisfied to very satisfied with the level of engagement/interest in the modules; 113 (85.6%) were satisfied to very satisfied with the clarity of visuals/graphics and an identical number with the amount of information presented. Differences in satisfaction between NPs and residents were small but statistically significant (F=5.94, P=.02; d=.42); NPs reported higher levels of satisfaction (mean=3.71, SD=.81) than did medical residents (mean=3.33, SD=.93).

This training successfully engaged primary health care trainees in suicide prevention education. Self-report and direct measures indicate, respectively, high levels of satisfaction and high levels of engagement with the training videos. We found increases in knowledge and self-efficacy among both learner groups. Many participants judged that they could put their training into practice, but moderate scores for perception of training transfer indicated the need to strengthen the program to emphasize application. To better understand these findings, we examined open-ended feedback from participants at 6-month follow-up. Feedback focused on the need for patient scenarios and examples of the skills in use, including examples of wording (“it would be helpful if there was a more standardized approach to what you are actually supposed to say to a suicidal patient”; “It would’ve been helpful … to go through the steps and appropriate questions to ask”; “concrete examples of safety plans would be appreciated”).

Based on these results, we identified steps to improve the training. First, we will integrate skill demonstrations by primary care providers with patient actors, and “voice of the patient” teaching from the perspective of a suicide attempt survivor and patient advocate. Second, to assist with retention and use in real practice, modules will be reorganized into a memorable framework and mnemonic: CARE: Connect, Assess, Respond, Extend (Connect and collaborate with patients and families; Assess the severity of suicidal thoughts and the context in which planning must occur; Respond to suicide concerns with appropriate interventions and plans; Extend the impact of plans and referrals so they take hold in patients’ lives and support networks). Third, a revised version will include material on effectively responding to risk identified in routine primary care screening, which has become more common since the study was carried out. Finally, we are completing the parent study in which this innovation is embedded, which will examine the added benefit of simulated patient practice and feedback compared to video-based instruction alone, with standardized patient interactions coded for assessment and other related skills.19 A replication of the present study, delivering the training to practicing physicians, is also ongoing.²⁰ These next steps are underway. The next-generation video-based curriculum, along with supporting electronic materials, will be available for wider evaluation and dissemination in January 2020.

While the modules evaluated here focus on primary care, the content and conceptual mapping can be adapted for medical disciplines that have a less intensive psychosocial focus, but still need to respond to suicide risk. Since the development of this program, the Joint Commission has signalled that requirements for education in suicide prevention will be greatly expanded across health care.²¹ Versions of this training are already available for behavioral health and youth and community services contexts, and modifications for emergency departments and other health care contexts are being examined.

Several limitations should be noted. While the study covered a number of different residencies and NP programs at the University of Rochester, we cannot be sure that these findings can be generalized to learners at an earlier training stage or at other universities. In addition, while some learners were instructed by their program staff to watch the videos together in one sitting, others were given 2 weeks to work through them. These differences were not applied systematically across programs, so we were unable to differentiate learners on this basis and analyze whether the varying time frames had any effect on results. We measured engagement with the educational materials by recording the proportion of learners who clicked play on each video and the proportion of each video watched in aggregate. Since learners were not required to run the video for any specific length of time and could click off it if they wanted to rush through, we consider the free choice to run the video and to leave it running to be indicative of engagement. However, these measures are unable to control for possible distraction (eg, from multitasking or from running the videos in the background but ignoring the content). More direct measures of engagement, such as eye tracking or task analysis, were impractical given the flexibility allowed to learners to watch in their own time and outside controlled settings.

This was a pre/post study with no control group, so we cannot compare the effectiveness of the training with other possible approaches or conclude securely that the improvements in knowledge and self-efficacy are attributable to the innovative pedagogical strategies deployed. Finally, while we measured engagement behaviorally through click and watch rates, future studies should include direct measures of practice behavior and patient outcomes. We recommend that future studies continue to track self-reported self-efficacy because this is a valuable outcome in itself and suicide prevention self-efficacy correlates with use of best practices in assessing and treating suicide risk.²²

This video-based curriculum shows promise for engaging medical and nursing trainees, and for enhancing their knowledge and confidence in this important but neglected area. More study is needed to determine if these findings generalize to people at earlier and later stages of professional development, and to measure further the effect on clinical behavior and outcomes.