Deprescribing Guidelines: Value of an Interactive Mobile Application

Polypharmacy and inappropriate medication use is prevalent in older adults and is associated with significant morbidity.^1-3 Withdrawal and/or dose reduction (deprescribing) of medications that are no longer needed, or may be causing more harm than benefit, is an essential part of optimizing medication use and is necessary to reduce medication-related harm.^4,5 However, clinicians have reported significant barriers to deprescribing in regular practice; a lack of resources (such as guidelines) to identify medications that are suitable for deprescribing, and provide guidance on how to stop them has been highlighted as an important barrier.^6,7 Evidence-based deprescribing guidelines support clinicians in conducting shared decision-making about when and how to reduce or stop medications; members of our team collaborated in the development of five such guidelines.^8-12 Accompanying two-page open-access decision-support PDF algorithms were developed to support implementation of the guidelines¹³ (www.deprescribing.org). These algorithms contain only highlights of the guideline presented in a static, noninteractive format.

Clinical decision support mobile apps can instantly connect clinicians to a wealth of information, with the advantages of being portable, easy to use, customizable, low cost and accessible at the point of care on a device that is ubiquitous.^14,15 These apps have the potential to improve decision making and clinical outcomes.^14-16 In response to requests from users of our guidelines, we developed a Deprescribing Channel on the Information Assessment Method (IAM) Medical Guidelines app to augment the use of the deprescribing guidelines, maximize point-of-care uptake by clinicians, and provide the ability to connect to the more detailed published guideline content and supporting resources as continuing education. The aim of this study was to evaluate the value of this Deprescribing Channel. Here, we present data reflecting the reach and adoption of the channel, as well as user feedback on the value of this information.

In 2015, Roland Grad, MD, developed the IAM Medical Guidelines app to enhance access to a curated selection of recommendations from guidelines relevant to clinicians in primary health care. This free app consists of multiple channels. Each channel is designed specifically for one clinical practice guideline. Our team chose this existing platform for the deprescribing guidelines because of reputation, low cost, and an ongoing collaboration with the developers. Together, we created a Deprescribing Channel containing five algorithms and information on the five drug class-specific deprescribing guidelines.

Beginning with the proton pump inhibitor guideline, a content expert worked with a programmer to develop a logic flow. This consisted of selecting content and response options for personalization across pages, including pop-up boxes with concise deprescribing advice and hyperlinks to full guideline content (Figure 1).

Next, we sought feedback on content, functionality, and user experience from stakeholders including guideline development team members, three individuals with expertise in app design, and 15 attendees at two health care conferences. The next four guideline topics (benzodiazepine receptor agonists, antipsychotics, antihyperglycemics, cholinesterase inhibitors/memantine) were designed with a similar logic flow to ensure a consistent navigation experience. Content review proceeded with members from each relevant guideline development team and representatives from the Canadian Agency for Drugs and Technologies in Health. 

The Deprescribing Channel was launched in March 2019 on the App Store and Google Play stores. Users downloaded the IAM Medical Guidelines app to their smartphones and then created an account, after which they could download the Deprescribing Channel and access any of the five algorithms and related deprescribing guidelines information.

A multifaceted promotion strategy included highlighting the app on a dedicated webpage at deprescribing.org, an email banner in research staff signature blocks, social media announcements through the deprescribing.org Facebook and Twitter accounts to over 9,500 followers, quarterly newsletters and e-blasts to subscribers and targeted emails to stakeholder organizations inviting them to share details with their networks, and showcasing the Deprescribing Channel at conferences and events through poster presentations and exhibitor booths.

We conducted data collection from March 2, 2019 to November 30, 2019. Using Google Analytics and page view metrics, we tracked app reach by the number of people accessing the web page and their residing country.

We counted the total number of downloads of the Deprescribing Channel as well as active users (users who accessed the channel at least once during each calendar month). We also examined page views for each guideline algorithm to determine how frequently guideline content was accessed.

The IAM Questionnaire (IAM-v 2014 for clinicians¹⁷) added a theory-based and validated measure to assess the value of this deprescribing information. This questionnaire was embedded into the app during the design phase and presented to the user via a link called “Feedback about this algorithm.” This feedback link appeared each time the user tapped the “Finish” button. Demographic questions (including professional designation) and a question asking why the information was sought were added. Other than completion of viewing content within a guideline algorithm, there were no other restrictions to accessing the IAM questionnaire. Users could complete the questionnaire any time they used the app in order to capture their experience in each unique context (eg, different patient, different moment in time, or different guideline leading to a new experience with this information). Additional information and versions of the IAM questionnaire can be found in the STFM Resource Library.¹⁷

We indirectly assessed outcomes at the four levels of the Kirkpatrick model,¹⁸ with levels 2-4 measured through the IAM questionnaire: Level 1: Reaction—at this level, we measured app downloads and page views; Level 2: Learning—we measured Level 2 through the IAM question on “cognitive impact.” This subconstruct of the IAM questionnaire includes an item on learning. Level 3: Behavior—we measured  through the IAM question on the “use or application of information in clinical practice.” Level 4: Results—we measured through the IAM question on “expected benefits for patients,” conditional on this information being used in clinical practice. 

The Bruyère Continuing Care Research Ethics Board reviewed and approved this research to use implied consent. Whenever a user completed an IAM questionnaire, they were shown a statement identifying the IAM questionniare as an evaluation tool.

Most views of the web page promoting the app occurred in North America (3,499 in the United States, 2,569 in Canada), with international interest demonstrated by page views across six continents and 40 countries in total (Table 1).

The Deprescribing Channel was downloaded 3,256 times (based on unique user ID numbers). We documented an average of 464 monthly active users. Of the active users, 78% were new downloads and 22% were returning visitors (those who had initiated a prior session within the Channel).

The proton pump inhibitor topic was viewed most frequently, while the cholinesterase inhibitors/memantine topic was viewed the least (Table 2).

A mixed group of 34 app users completed 37 questionnaires; the group breakdown was physicians (15), pharmacists (13), nurses (2) and others (7). Three of those users completed the IAM questionnaire on two occasions. The top three reasons for seeking deprescribing information were to (1) exchange information with colleagues (18), (2) satisfy curiosity (17), and (3) fulfill a personal educational objective (16; Figure 2). In 32 of the 37 IAM questionnaires, respondents reported that the information was partially or totally relevant for at least one of their patients. The majority said that they learned something new (n=22), were motivated to learn more (n=22), or confirmed their actions were correct (n=19) as a result of the information (Figure 3). In 29 questionnaires, respondents said they would use or possibly use this information for a patient, with 18 reporting that they expected a health benefit by applying this information.

We developed an interactive app containing information from five deprescribing guidelines and achieved international dissemination with almost 500 active monthly users, over a 9-month period. Feedback from a small number of users suggests the app content is relevant, educational, and applicable to patients. To our knowledge, this is the first evaluation of a mobile app providing guideline-based information on deprescribing.

While the number of app downloads was high, the number of downloads was less (approximately one-third) than the web page views. The website is designed to have broad appeal and it seems unlikely that all viewers would download a deprescribing guideline app designed for health care professionals. For example, nonclinical members of the general public have full access to the site, but it is unlikely those who are not health professionals would download the application. We are unable to calculate an exact conversion rate (defined as the ratio of website visitors who complete a desired action, in this case downloading the app). However, conversion rates in the e-commerce environment are typically low, with the top 10% of companies typically achieving an 11% conversion rate, at most.¹⁹

The IAM questionnaire provided the opportunity to systematically document users’ reflections on deprescribing information in the app. In other work, the IAM questionnaire has enhanced reflective learning, evaluation of knowledge resources, and two-way knowledge exchange between information users and information providers.²⁰ Our results suggest the app provides medical learners with new information for their continuing education and motivates them to learn more about deprescribing. However, only a small proportion of those who downloaded the Deprescribing Channel completed the IAM questionnaire. Two reasons may explain this observation. First, the questionnaire link was restricted to users who clicked on the “Finish” button and it may not have been obvious that this linked to an evaluation questionnaire. Secondly, we provided no incentive for questionnaire completion, such as continuing medical education credits or a contest to win a prize. With such a small sample size, and due to the self-reported nature of the questionnaire, our data may be biased if it was received from those who were more engaged, or had more favorable opinions. In the future, a usability study of this app should consider how and when to better present surveys to maximize user feedback; for example, the questionnaire link button could be made more appealing to click (using wording like “Help us improve the app”). The process also did not allow a measurement of learning through pre- or posttests of knowledge.

Promising avenues for future research include identifying minimal numbers of completed questionnaires needed to calculate a relevance index using the IAM questionnaire,²¹ approaches to maximize response, measuring app retention through continued visits, and exploring how the use of the Deprescribing Channel in clinical practice compares to the traditional static presentation of text in a guideline. Additionally, a recently published instrument designed to be used by health professional educators could assist in rating the value of the app to support learning in four thematic areas: teaching and learning, user-centered, professional measures and usability.²²

The Deprescribing Channel achieved international reach and adoption with app users making sustained page views over a 9-month study period. Feedback from a small number of users was positive, with most finding the content to be educational and applicable to patient care. Despite the low number of completed questionnaires, the value of information in this app can be inferred from a sustained number of monthly users.