Long-term Effects of a Health Literacy Curriculum for Family Medicine Residents

Health literacy is the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions.¹ Low health literacy affects over one-third of US adults,² and is associated with numerous adverse health outcomes.³ Health literacy is considered an educationally sensitive issue, in that improved training in health literacy and clear communication for health care professionals is expected to improve care for patients.⁴ However, physicians, including family medicine residents, are often inadequately prepared to provide effective care for patients with low health literacy.^5-10 Improved training for health care professionals in health literacy and clear communication (written or oral communication that helps patients understand and act on health care information)¹¹ techniques has been widely recommended at the national level.^12,13 Many US medical schools¹⁴ and residency programs^15,16 now teach about health literacy and clear communication techniques. In addition, health literacy and clear communication educational competencies have been identified through a consensus process,¹⁷ and a rank-ordered set of target behaviors for health professionals was recently published.¹⁸ Training in health literacy has been associated with the self-reported use of health literacy best practices,¹⁹ and prospective studies have shown short-term improvements in self-reported knowledge and planned behaviors among medical students,²⁰ nonphysician health professionals,^21,22 and family physicians.²² One observational study reported increased knowledge scores and use of plain language among internal medicine residents within 1 week of training.²³ In the only known health literacy educational study to include long-term follow-up, 48 first-year medical students reported significant gains in self-perceived knowledge, and perceived and planned clear communication behaviors immediately following a one-time didactic and skill-building training, however, after 12 months nearly all of these gains had returned to baseline levels, possibly due to the limited clinical experiences of preclerkship medical students.²⁰ Our pilot study aimed to build on this prior study, to determine the long-term effects of a longitudinal didactic and experiential skill-building health literacy and clear communication curriculum on knowledge and practices among family medicine residents. We hypothesized that exposure to such a curriculum would result in residents reporting increased use of clear communication techniques over time.

All 12 first-year family medicine residents at an academic residency program in the Pacific Northwest participated. Participants engaged in a series of four 60- to 90-minute required health literacy and clear communication trainings, which were conducted by one of the authors (CC) over an 11-month period. The sessions included didactic and experiential skill-building elements based on consensus-derived educational competencies, which were available to the authors, but published at a later date.¹⁷ The training program was consistent with previously described curricula,^24-26 and is outlined in Table 1.  

Demographic data were collected. Immediately before and after each of the four trainings, participants anonymously completed pre- and postassessments (except before training two, for which only the postassessment was administered), which were modified versions of the tools developed by Mackert and colleagues.²¹ While these survey tools have been used in two subsequent studies,^20,22 they have not been validated. We modified the original pre- and posttests²¹ by adding four items based on published best practices/competencies,¹⁷ for a total of five knowledge items and 11 perceived/planned behavior pairs (Figures 1-3; for exact wording of pre/posttest items, please contact the corresponding author). In addition, we added two self-confidence items (“I am confident in my ability to explain things clearly when speaking,” and “I am confident in my ability to explain things in writing”), and the posttests included an item from the original tool, assessing respondents’ potential overestimation of their health literacy knowledge on the pretest. All items used a 7-point Likert scale (1=strongly disagree, 7=strongly agree for knowledge and attitude items; 1=very unlikely, 7=very likely for practice items). For the fourth training (writing workshop) the posttest only included three of the relevant practices that had been emphasized during that training. A power calculation done for an earlier study indicated the need for at least 10 participants in order to detect a practically significant effect.²²

Frequencies for demographic variables were calculated. Mean scores for pre- and posttraining item pairs were compared using two-tailed t-tests with a significance level of .05. Pretraining data for session two were not available for analysis (indicated in Figures 1-3 by the lack of a P value for the interval between postassessment one and postassessment two). We omitted three perceived/planned behavior item pairs in our analysis: two because the stems of the pre- and post- questions differed (identifying low health literate patients, and knowing when patients understand), and one systems issue which residents were felt to have limited influence over (creating a shame-free environment). This study was approved by the Institutional Review Board at Oregon Health & Science University.

All 12 residents participated in the first three trainings; two missed the fourth training. The majority were white women, ages 26 to 30 (Table 2).

All five self-perceived knowledge items showed statistically significant increases from baseline after the initial training, and these increases were sustained at each subsequent pre- and postassessment across the entire 11-month study period (Figure 1).The two items assessing confidence in spoken and written communication showed nonstatistically significant positive trends across the 11-month study period (data not shown).  

Six of the eight perceived/planned behavior pairs that we analyzed showed a pronounced “saw-tooth” pattern, with statistically significant improvements after each training, but returning fully back to baseline between trainings, such that baseline averages were not statistically significantly higher at any subsequent preassessments, indicating no improvements in self-perceived behaviors for these items (Figures 2 and 3). Only two perceived/planned behavior pairs showed sustained statistically significant improvement across the 11-month study period. The first was average ratings for the item asking “what questions” rather than “do you have any questions?”, which increased from 4.3 to 6.1 (P=.01) after the first training, and remained statistically significantly improved above baseline at each subsequent pre- and postassessment. The second was average ratings for the item “Use teach-back or show-me”, which increased from 3.1 to 5.3 (P<.00) after the first training, and remained statistically significantly higher than baseline at each subsequent pre- and postassessment as well, but with statistically significant declines toward (but not back to) baseline between each training (Figure 3).  

For the posttest item, “I originally overestimated my own understanding of health literacy,” mean agreement was statistically significantly higher following the second training than it was following the first (3.5 and 5.7, respectively; P=.002), with a nonstatistically significant trend downward after the third and fourth trainings (4.8 and 4.1, respectively).

Little is known about optimal methods for teaching physicians about health literacy and clear communication.²⁴ This adequately powered²² pilot study is important for two reasons. First, to our knowledge, it is the only such study to follow physicians (family medicine residents) beyond the immediate posttraining period. While we showed short-term improvements in knowledge and planned behaviors, similar to prior studies,^5,20-22 we were also able to demonstrate robust long-term improvements in knowledge after a single didactic training. This is significant because 30% of family medicine residency programs with a required health literacy curriculum reported using a single lecture instructional format.¹⁵ Second, the posttraining gains we observed for six out of eight planned behaviors regressed to baseline at each of the subsequent pretraining assessments (similar to patterns observed among medical students)²⁰ despite the use of experiential skill-building instructional methods designed to influence clinical practice.²⁹ These findings suggest that residents were able to learn material related to health literacy and clear communication easily, but that they were largely unable to translate this learning into practice.

There are several possible explanations for the saw-tooth pattern observed for seven of the eight perceived/planned behavior pairs analyzed. First, improvements after individual trainings could reflect social response bias, or response shift bias due to the immediate pre/post design. Second, participants may have developed a deeper and more nuanced appreciation of health literacy issues over time, which could result in more realistic self-assessments at each successive preassessment. For example, more participants reported having originally overestimated their understanding of health literacy after the second, more in-depth training than they did after the first training. Third, there may have been environmental barriers to implementing planned behaviors, such as potential lack of faculty role modeling, time constraints, and/or pressure to address multiple issues during clinic visits.

Potential limitations include study at a single training site, meaning that the results may not be generalizable. The sample size was small, but this study was adequately powered to detect a practically significant effect size, based on a previous power calculation.²² However, the original assessment tools had been used in several other studies,^20-22 but had not been previously validated. We modified the original tools, but did not attempt to validate the assessments used in this pilot study. In addition, the self-reported nature of these data may be susceptible to social response bias, and the pre/postassessment method may be susceptible to response shift bias. The use of a different assessment tool^30,31 may have yielded different results.  

Future studies should include an objective knowledge assessment, observational skills assessment, such as an observed structured clinical examination, and objective assessment of clinical behaviors, such as through patient surveys, as well as assessment of potential environmental barriers to translating skills into practice, such as the degree of faculty role modeling.