Digital Learning Tools in Postgraduate Family Medicine Training: A Scoping Review

Digital learning tools are reshaping the landscape of medical education, yet their integration into family medicine residency training remains poorly understood. These tools have the potential to enhance residency workflows by providing flexible, location-independent, and individualized learning experiences; offering simulations for skills development; enabling access to evidence-based resources; supporting reflective practice; and fostering knowledge sharing, collaborative learning, and ongoing professional development.^1,2

Digital learning tools take many forms and can refer to any electronic application, game, or resource that supports education, including online or offline computer-based resources, mobile apps, or electronic games.^3,4 They offer a variety of benefits, such as increased learner engagement, access to timely feedback, resource efficiency, and more detailed assessments, making them a valuable support for medical education in a digital age.⁵

Several literature reviews have been conducted on digital learning tools in the education of health professionals.⁶ However, to our knowledge, none have explored the type of digital learning tools available to postgraduate family medicine trainees. As postgraduate family medicine education evolves, knowing which digital learning tools are being used to help family medicine trainees attain the competencies required for a comprehensive scope of practice is essential. Competency-based medical education (CBME) has become a dominant approach to postgraduate medical education in many countries.⁷ Mapping the current landscape of digital tools can help identify potential knowledge gaps and inform educational practice, research, and development of new tools.

This scoping review aims to identify existing digital learning tools in postgraduate family medicine education and describe how they are used.

We used the scoping review framework developed by Arksey and O’Malley⁸ and recommendations from Levac and colleagues.⁹ Digital learning tools were defined as any online or offline computer-based resource, mobile application, electronic game, or other resource that supports, enhances, or contributes to medical education by facilitating the achievement of specific learning outcomes. We involved knowledge users throughout the project, including clinicians, medical educators, digital learning tool developers, and medical students.¹⁰ The manuscript was drafted according to the PRISMA-ScR checklist.¹¹

Ethics approval was not applicable for this literature review.

The detailed methods for this scoping review were predefined in a published protocol⁶ and registered with the OSF Registries (osf.io/wju4k).

To be included, articles had to:

• focus on the design, development, implementation, or evaluation of digital learning tools for postgraduate family medicine;

• include postgraduate family medicine trainees, at any year of training;

• feature at least one digital learning tool and one or more specific learning outcomes;

• have full texts available in English or French; and

• be published from 2010 onward.

Questions regarding eligibility were resolved through discussion with knowledge users.

The full search strategy is described in detail in our protocol.⁶ Briefly, the major search terms were “family medicine training” and “digital learning tools.” For “digital learning tools,” we searched both database-specific subject headings (eg, computer-assisted instruction, mobile applications, simulation training, virtual reality, social media) and a wide range of related keywords (eg, digital learning, elearning, gamification, web-based, smartphone, tablet, Twitter). For “family medicine training”, we used only keywords, which included combinations of terms related to family medicine (eg, family medic*, primary care, general practi*) and training (eg, educat*, residen*, post?graduate*, student*, train*). We searched MEDLINE, ERIC, Education Source, and Embase using subject heading terms and keywords. Scopus and Web of Science do not use subject headings, so we searched these using keywords alone. We also searched the reference lists of the included articles. The results of the academic literature searches were imported into Covidence (Covidence Inc) software for deduplication and title/abstract screening. The academic literature search was conducted in August 2021 and updated in August 2022.

The gray literature search was conducted in August 2021 and updated in December 2022 using keywords identified in the academic literature search. We used an advanced Google (Alphabet Inc) search to identify relevant resources from university websites, family medicine organizations, medical school and residency organizations, and relevant conferences. Each search was limited to the first 10 pages of results.

Two reviewers (H.Y., T.D., and/or R.H.) independently screened the title and abstract of each academic article according to the eligibility criteria. Gray literature was screened by one reviewer (H.Y.).

If either reviewer included an article, or if eligibility was unclear, the article underwent full-text screening. Full texts were independently screened by two reviewers (H.Y., T.D., and/or R.H.), who discussed any disagreements. If an agreement could not be reached, a knowledge user (L.S.) and a senior author (C.L.) were consulted.

A data charting form was created in Microsoft Excel software and revised iteratively by the study team, including knowledge users, throughout the charting process. The data items included title, author(s), publication year, country of origin, publication type, study design, data analysis methods, target population and size, name and description of the digital learning tool(s), type of the tool, specific learning outcome(s) facilitated by the tool, whether the tool is readily accessible (eg, free to use), usage location (eg, classroom, clinical, or personal), learning objective(s), outcome measure(s) assessed, users’ perspectives, and main findings.

Characteristics and findings from all included literature were tabulated and summarized. The results were reported descriptively, with the analysis anchored in the data charting spreadsheet.

To answer the research question “What digital learning tools exist for postgraduate family medicine training?” and to capture the diversity of tools and learning approaches in the studies, we classified the tools by modality (type of application or technology) and learning type (digital, blended, traditional). Given the inconsistent terminology in the articles, we adapted working definitions for tool types and learning context (Tables 1 and 2) from other sources describing digital learning in medical education.^12–14

To describe how the tools were used in family medicine training, we classified them according to the learning outcomes they were designed to facilitate, based on domains of clinical skills¹⁵ and knowledge user feedback:

1. Electronic medical record (EMR)/technology-based knowledge,

2. Physical examination and procedural skills,

3. Pathology knowledge/skills,

4. Pharmacological knowledge,

5. Communication skills,

6. Research skills, and

7. Social accountability skills.

To relate the learning outcomes facilitated by the digital learning tools to the competencies expected to be acquired through postgraduate family medicine training, we classified the tools according to the eight core domains of professional responsibility identified by the College of Family Physicians of Canada:¹⁶

1. Primary care in the community/office setting,

2. Maternal and newborn care,

3. Home and long-term care,

4. Emergency care,

5. Hospital care,

6. Advocacy,

7. Leadership, and

8. Scholarship.

To synthesize the diverse outcomes reported in the studies, including user perceptions, which were particularly relevant to our research question, we adapted the extended Kirkpatrick’s educational outcomes model.^17–19 This model assesses the effectiveness of training programs at four levels: learner reaction or experience of learning; acquisition of knowledge, skills, and attitude; behavioral change; and impact on clinical outcomes. Definitions of each level are provided in Table 3.

Our search yielded 1,446 potentially relevant academic and 612 gray literature sources (Figure 1). After screening titles and abstracts, 506 academic and 147 gray literature citations met eligibility criteria and were subjected to full-text review. Following this process, 39 articles (34 from academic and five from gray literature) were retained for analysis.

Because the five records retrieved through the gray literature search were academic, peer-reviewed articles, we examined all 39 articles together (Table 4). Of the 39 studies, 28 (72%) were undertaken in North America (19 in the United States and nine in Canada), 6 (15.4%) in Europe (two in France, 1 in the Netherlands, 1 in Sweden, 1 in the United Kingdom, and one in Spain), 2 (5.1%) in Asia (both in Singapore), 1 (2.6%) in South America (Brazil), 1 (2.6%) in Africa (multiple Sub-Saharan countries including Botswana, Congo, Ghana, Nigeria, Sierra Leone), and 1 (2.6%) in Oceania (Australia). One to six articles were published yearly. Combined, the articles included 5,282 participants.

We classified the digital learning tools according to the type of technology and the learning context (Tables 1 and 2). Twenty-six studies (66.6%) described online computer-based tools such as modules, courses, or curricula. Simulations, including virtual and augmented reality, were featured in seven studies (17.9%) and mobile applications in three studies (7.7%). Among other types of tools (7.7%), we identified one serious game, one digital toolkit, and one social media application. In 25 studies (64.2%), the digital learning tool was applied in the context of fully digital learning; whereas in 14 studies (35.8%), the digital learning tool was integrated with traditional learning and included an in-person/face-to-face component (eg, skill sessions, in-class seminar, webinar, or lectures, small group debriefing, immersive simulations including role playing, and hands-on simulator training).

The digital learning tools featured in 13 of the included studies (33%) had more than one learning outcome identified. The highest proportion of studies featured tools designed to facilitate physical examination and procedural skills (35.9%) and pathology knowledge and skills (33.3%), followed by pharmacology knowledge (23.1%) and communication skills (17.9%). Very few studies featured tools that facilitated research skills (7.7%), social accountability skills (7.7%), and EMR/technology-based knowledge (2.6%).

We aligned the learning outcomes facilitated by the digital tools with the eight core family medicine domains of professional responsibility and their core professional activities (CPAs). In 32 studies (82%), the digital tools targeted competencies relevant to the practice of primary care in the community/office setting. CPAs encountered most frequently in this category included providing comprehensive primary care and continuity of care for adults, performing medical procedures in all settings, and providing mental health and addiction care. Very few digital learning tools targeted maternal and newborn care (3 of 39, or 7.7%) and scholarship (3 of 39, or 7.7%). No digital learning tools were designed to facilitate acquiring skills and competencies pertinent to the practice of family medicine in home and long-term care, hospital care, and emergency care, as well as advocacy and leadership.

The largest proportion of the studies assessed acquisition of knowledge/skills (28, or 71.8%), followed by reaction, or learning experience (23, or 58.9%), and modification of attitudes/perceptions (18, or 46.2%). Very few studies reported on behavioral change, as evidenced by the application of newly learned skills into clinical practice (2, or 5.1%), only one (2.6%) mentioned change in organizational practice, and none assessed impact on patient care.

To gain more insight into the user experience, we further characterized the reaction outcomes, which represent learners’ perspectives on the learning experience with the digital learning tools. Of the 23 studies (58.9%) that examined reactions, all reported positive reactions, including satisfaction with the tools and their educational value, interactive elements that learners found engaging, autonomy to control the pace and place of learning, and design features including immediate feedback. Negative reactions were reported in 17 out of the 23 studies that evaluated reactions; these included technical difficulties accessing the tools, lack of interactive features, distracting features, missing or hard-to-read facial expressions or body language in virtual patient simulations, and difficulty finding time to complete the digital tool activities.

Our scoping review revealed a small number of studies on digital learning tools for postgraduate family medicine trainees, with 39 highly heterogeneous studies published between 2010 and 2022. We discovered that most digital learning tools focused on competencies relevant to primary care in community and office settings, with limited attention to maternal and newborn care, emergency care, and other key areas in family medicine. Alarmingly, no tools targeted competencies essential for the provision of home and long-term care, hospital care, or the advocacy and leadership roles of family physicians. These gaps are particularly concerning given the increasing demands placed on family physicians to address the needs of an aging population, whose care requirements cannot be met by specialists alone.²⁰ Digital learning tools present an opportunity to equip family medicine trainees with the skills and knowledge needed to meet these evolving demands, particularly in the context of CBME, an approach adopted by national medical regulatory bodies in Canada, the United States, and other countries.⁷ CBME requires trainees to demonstrate proficiency across a broad range of skills, yet the current digital tools fail to address these core expectations. While the classification of digital learning tools in this review was based on the Canadian framework of CPAs, these domains broadly align with competency-based milestones and entrustable professional activities used internationally to guide postgraduate training in family medicine. Developing new digital learning tools targeting a wider array of competencies is essential to align family medicine training programs with the realities of modern practice and to ensure that graduates are prepared to meet the diverse needs of the populations they serve. Our findings also revealed a gap in digital learning tools designed to develop essential competencies that are increasingly vital for modern medical practice, such as research skills, social accountability, and technology-based knowledge. Digital platforms offer flexible, interactive, and accessible learning experiences that promote critical appraisal of evidence and scholarship, thereby enhancing research skills, a core competency of residency training.²¹ Similarly, digital tools such as virtual simulations and case-based learning modules can expose trainees to real-world challenges involving health disparities, thereby fostering cultural competence, advocacy, and social accountability.^22,23 Finally, as health care continues to evolve through digitization, family medicine curricula must incorporate training in technology-based knowledge, such as EMRs, clinical decision support systems, and artificial intelligence (AI) tools.²⁴ Proficiency in these areas is now essential for modern physicians, is well-suited to digital learning, and can contribute to improved clinical outcomes.²⁵

Our review demonstrates that despite the advantages that immersive learning approaches such as simulations, virtual and augmented reality, gamification, or AI offer, family medicine training is lagging in incorporating all of these advanced technologies except simulation.²⁶ These cutting-edge tools are reshaping medical education, offering dynamic and interactive opportunities for hands-on learning and practice.^27,28 The benefits of simulation-based training are well-documented, including accelerated skill acquisition and reduced risk for clinical practice errors.²⁸ For example, realistic patient scenarios in virtual simulations enable trainees to refine their clinical decision-making and learn from mistakes in a safe environment, with instant feedback and competency-based scoring to guide improvement.²⁹

Consistent with other literature appraising interventions in medical education,^25,30 most studies we reviewed evaluated educational outcomes at the lower levels of Kirkpatrick’s model, focusing on learners’ knowledge, skills, and learning experience. While a few studies assessed behavioral change among trainees, only one explored change in organizational practice, and none examined the impact on patient care or clinical outcomes. This finding represents an important gap, because evaluation of educational outcomes, both short- and long-term, is essential for curriculum development and improvement of educational processes. Though resource-intensive, we recommend that family medicine educators prioritize evaluation of digital learning tools at higher levels of Kirkpatrick’s model, such as change in behavior and practice. Carefully designed studies are essential to develop digital learning tools that measure improved patient outcomes, enhance health care delivery, or increase the competence required for trainees to incorporate specific skills into their future practice. Where feasible, evaluations should incorporate objective measures, such as patient health indicators or practice metrics, to provide robust evidence of the tools’ effectiveness. Study designs such as longitudinal, mixed-methods, or quasi-experimental approaches (eg, pre-post with a control group) offer a way to assess impacts on specific clinical outcomes, patient safety indicators (eg, adverse events, medication errors), or quality of care metrics (eg, patient satisfaction, readmission rates), though such evaluations may be challenging to implement in postgraduate training contexts.

Faculty preparedness and implementation challenges are important considerations for the successful integration of digital tools into postgraduate family medicine education. Some of the common barriers include limited training, inadequate resources, and lack of institutional support.^2,31 Effective solutions may include (a) offering targeted faculty development programs to build digital teaching competencies; (b) institutional investing in reliable infrastructure, technical support, and relevant digital platforms; and (c) fostering an institutional culture that values innovation by establishing clear strategies for digital learning and encouraging peer collaboration.^2,31 Ultimately, sustained institutional commitment to these solutions is essential for the successful adoption of digital learning in family medicine training.

Our study had several limitations, including those inherent to all scoping reviews. We did not perform critical appraisal beyond the inclusion/exclusion criteria, because we aimed to explore the literature rather than analyze any intervention effects. We relied on the availability of relevant information, which restricts the applicability of our findings to practice or policy. Additionally, we observed significant heterogeneity among the studies and ambiguous definitions, which made synthesizing the results challenging. Gray literature was reviewed by a single author, which may have introduced some selection bias. However, predefined inclusion and exclusion criteria, informed by discussions with knowledge users, guided the process, and full-text selection decisions were documented to support transparency. Finally, while our search concluded in December 2022, the more recent literature does not yet suggest that developments in large language models (LLMs), including the launch of ChatGPT (OpenAI), have had a significant impact on postgraduate family medicine education that would alter our conclusions. Most studies published since then have focused on the theoretical potential, possible misuse, or performance testing of LLMs rather than on their integration and evaluation within curricula, particularly in the context of family medicine, and therefore would not have met our inclusion criteria.^32–35 Future reviews may be warranted to examine how these technologies evolve and whether they get incorporated into postgraduate family medicine education.

Our scoping review highlights several important directions for future research and development in postgraduate family medicine training. First, new tools should be designed to address gaps in key competencies, including research skills, social accountability, technology-based knowledge, and areas such as home and long-term care, hospital care, emergency care, and advocacy and leadership. Second, a need exists to leverage more immersive and innovative digital learning technologies to enhance hands-on learning and skill development in family medicine trainees. Finally, rigorous evaluation is essential to accurately assess the impact of both existing and newly developed tools on behavioral changes, improvements in clinical practice, and, critically, patient care outcomes and to inform curriculum development for the digital age. By addressing these gaps, family medicine educators can develop evidence-based digital learning tools that prepare trainees to meet the challenges of an evolving health care landscape. This review provides a road map for advancing learning in postgraduate family medicine education and underscores the urgency of closing existing gaps to equip the next generation of family physicians with the competencies needed to deliver comprehensive, high-quality care (Appendix A).