Response to “Artificial Intelligence-Prompted Explanations of Common Primary Care Diagnoses”

The recent study by Mr Kattih and colleages¹ is an intriguing examination of the capabilities of several AI models to develop learning materials for patients with common conditions. However, we wish to note some concerns and suggestions for changes, both in terms of substance and approach with this study.

First, while Flash Reading Ease (FRE) and Flesch-Kincaid Grade Level (FKGL) are widely employed to assess reading difficulty, they may not adequately capture the intricacies of medical language or the cognitive needs of patients with widely varying degrees of health literacy. Relying solely on these measurements can be problematic, as they fail to address issues such as medical terminology or patients' ability to understand complex medical concepts. Furthermore, while the PEMAT tool is functionally effective and valuable for evaluation, the study does not clarify how the AI-generated content is used. A prior study found that the quality of the material for the PEMAT instrument determines its success.² This study raises a critical question: do AI-generated learning materials actually improve patient outcomes, even when they demonstrate good readability and comprehension scores? For example, do these tools improve patient adherence to treatment or reduce confusion regarding diagnostic and treatment options? These are broad concerns that may inspire a larger discussion in health care AI research.

The study by Kattih et al focusses largely on the technical aspects of readability and functionality in AI-generated materials for patient education. It is essential to address additional barriers to effective education, such as emotional support, personalized counseling, and the inclusion of patient preferences and concerns. These factors are critical to ensuring that content is not only readable, but also beneficial and durable for patients. In clinical settings, AI may assist practitioners in providing individualized advice, emphatic care, and communication styles that are tailored to each patient.³

Future innovation could involve exploring the use of additional assessment measures beyond readability scores. For instance, incorporating patient feedback through qualitative studies or evaluating the real-world impact of AI-generated materials on patient behavior would provide a more comprehensive understanding of their effectiveness. Furthermore, integrating AI-generated material with interactive features like videos, quizzes, or individualized care plans may boost patient engagement and retention. These approaches can overcome the limitations of static text and ensure that patient education materials are interesting and suited for a variety of learning styles.⁴ Deshpande et al studied teaching tools for patients and found improvements in both process and outcomes across a range of chronic conditions.⁴ According to their findings, video-based aids were the most effective at increasing patient knowledge.⁴

Finally, we suggest that adopting an interdisciplinary approach could improve future research. Involving health care practitioners, linguists, and health communication experts would provide a diversified view of what makes good patient education.⁵ It would also be beneficial to evaluate AI-generated educational materials across different demographic groups, considering variables such as age, cultural background, and health literacy. This approach could assist researchers in identifying and addressing disparities in health education, ensuring that AI technologies serve a broader spectrum of patients and health care settings.