Knowledge Syntheses in Medical Education

As the volume of medical education literature grows, educators face challenges in interpreting, contextualizing, and applying relevant findings in their daily practices, whether in classrooms, clinics, or conference rooms. Knowledge syntheses, commonly known as literature reviews, provide rigorous methods for identifying, appraising, and integrating this expanding body of publications. By combining findings from multiple publications and offering evidence-informed guidance, knowledge syntheses help bridge academic evidence and real-world application. Moreover, by distilling nuanced and technical research into accessible formats, such as summary tables, conceptual models, graphical representations, or practical recommendations, syntheses empower individuals to apply findings more effectively.¹ Considering these benefits, medical education scholars have asserted that knowledge syntheses are as important as, if not more than, primary studies.²

In this era of exponential knowledge growth, knowledge synthesis has emerged as a critical methodological approach within medical education.² Reflecting this growth, the number of knowledge syntheses published in medical education has increased by over 2,000% between 1999 and 2019.³ Moreover, compared to other publication types, knowledge syntheses are highly cited, frequently featured on social media, and heavily downloaded, indicating that they play an important role in the field’s discourse.³

This article provides an overview of knowledge synthesis in medical education to familiarize readers with the methodology, raise points of concern, and highlight future opportunities. It is structured to describe: (a) what knowledge synthesis is; (b) types of knowledge syntheses, (c) common challenges; and (d) future directions.

Knowledge synthesis is the process of integrating existing findings from multiple publications to develop a more comprehensive understanding of a topic. Unlike traditional literature reviews, which often appear as introductions of academic articles and may be selective or narrative in nature,⁴ knowledge syntheses follow structured, transparent methodologies explicitly designed to promote reproducibility and enhance the reliability of findings. Knowledge synthesis can take many forms,^5,6 such as systematic reviews, scoping reviews, and realist reviews, each with distinct methodologies tailored to specific types of questions, data, and objectives. Whether focused on measuring effectiveness, mapping the field, or exploring how and why interventions work, each synthesis type contributes unique value to medical education.

A knowledge synthesis typically follows a multistep process. First, researchers define a clear, focused research question, which can be guided by frameworks such as PICO (Population, Intervention, Comparison, Outcome)⁷ or SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type),⁸ depending on the synthesis type. Next, researchers develop and execute a literature search, often across multiple information resources, to identify relevant publications. The literature search is followed by screening titles and abstracts, and full texts against predefined inclusion and exclusion criteria. Next is data extraction, in which key information from the selected studies is systematically collected. The extracted data are then analyzed and synthesized using quantitative and/or qualitative approaches. Finally, findings are interpreted in the context of the research question and existing literature, with implications for educational practice and policy discussed. While these steps provide a general structure, in practice, they will vary based on the knowledge synthesis type being conducted.

Knowledge synthesis encompasses a diverse array of types, each developed to serve different purposes, accommodate various research questions, and engage with different evidence types. Understanding the types of knowledge syntheses is essential, because choosing the right type ensures that the conclusions drawn are both valid and useful.

Across fields, researchers have identified 25 knowledge synthesis types.⁶ Specific to medical education, medical education journals have published 21 types of knowledge syntheses, with systematic reviews, scoping reviews, and narrative reviews being the most prevalent.³ While multiple types are conducted, this article provides a brief overview of systematic reviews, scoping reviews, meta-syntheses, and realist reviews, which are commonly used or emerging in medical education.

Systematic reviews address focused research questions by comprehensively identifying, appraising, and synthesizing the evidence on a particular topic.⁹ These reviews seek to answer specific, often narrowly framed research questions related to effectiveness, causality, or measurement. For example, one systematic review examined the effectiveness of using virtual patients to provide feedback on clinical reasoning skills.¹⁰ Systematic reviews are especially valuable for informing evidence-based decision-making in educational policy and practice. Systematic reviews also can include a meta-analysis. A meta-analysis is a quantitative approach that involves the statistical pooling of quantitative results across included studies to estimate the overall effect size of an intervention.¹¹ For example, one systematic review with meta-analysis examined the effects of synchronous distance education versus traditional education, finding no significant difference in knowledge acquisition or satisfaction.¹²

While powerful, systematic reviews are also resource-intensive, requiring careful planning, methodological expertise, and significant time investment. Fortunately, the medical education field offers several resources to support their development, including step-by-step guides.^13,14 Many journals require systematic reviews to follow the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) statement, which provides a 27-item checklist to promote transparency and completeness in reporting.¹⁵

In medical education, the publication of scoping reviews is on the rise.¹⁶ Researchers typically conduct a scoping review to (a) examine the nature of the literature on a topic, (b) identify gaps in the evidence base, (c) determine whether undertaking a systematic review is appropriate, and/or (d) summarize and disseminate the findings of the topic’s literature.^17,18 For example, in a scoping review on artificial intelligence (AI), the authors described their rationale: “We chose a scoping review methodology, mapping out the landscape of the existing literature, to delineate the evidence bases, classify methodologies, and highlight thematic content ripe for systematic investigation.”¹⁹ As in this example, scoping reviews are often described as “mapping” a topic’s literature landscape.¹⁷ The importance of this mapping function is highlighted in an editorial from this journal: “Our discipline [family medicine education] needs more scoping reviews, because their value is unique. Without maps, how will we know where to go?”²⁰

To create a complete map of a topic’s literature, scoping reviews are intentionally broad and flexible in the publication types they include. In addition to peer-reviewed articles, they may incorporate alternative publication types such as policy documents, theses, meeting abstracts, book chapters, or websites. Additionally, authors are increasingly involving stakeholders or community members in the review process to increase the usability and relevance of this mapping exercise. For example, in a scoping review on the hidden curriculum, the authors shared preliminary findings with clinical teachers to examine how the review’s findings aligned with their related lived experience.²¹ This engagement helps validate the findings and increases the scoping review’s practical impact.

Scoping reviews require a systematic and comprehensive approach,²² which can be resource- and time-intensive. To guide this process, many researchers rely on the widely used six-step framework developed by Arkey and O’Malley¹⁷ and updated by Levac and colleagues.²³ However, for those conducting medical education scoping reviews, Mak and Thomas’ step-by-step guide provides practical, field-specific recommendations.²⁴ Additionally, authors of scoping reviews should consult the PRISMA-ScR extension for scoping reviews, which facilitates thorough reporting.²⁵

As the number of knowledge syntheses has grown, a new challenge has emerged. What was once a problem of too many primary studies has now become one of having too many knowledge syntheses. This abundance can challenge educators and researchers to interpret and apply the growing body of synthesized evidence. In response, the meta-synthesis has gained traction. A meta-synthesis, not to be confused with a meta-analysis, integrates findings across multiple existing knowledge syntheses, providing a higher-level summary of evidence on a topic.²⁶ For example, researchers examining the impact of continuing professional development on patient outcomes encountered 63 preexisting reviews. To make sense of them, they conducted a meta-synthesis, ultimately distilling the findings from the numerous individual reviews into a single comprehensive overview.²⁷ This approach allowed the authors to identify patterns, highlight consistent findings, and uncover gaps in the evidence in one accessible synthesis.

One form of meta-synthesis that has gained popularity is the umbrella review, which is “a systematic collection of multiple systematic reviews and meta-analyses on a specific research topic.”²⁸ By aggregating and comparing the conclusions of several reviews, umbrella reviews can identify areas of consensus, highlight inconsistencies, and assess the overall strength of evidence. For example, Onyura and colleagues investigated the evidence for curricular approaches by synthesizing the findings of 36 systematic reviews and concluding that educational interventions produce mixed effects on learning.²⁹

Meta-syntheses are increasingly valuable tools for guiding decision-making for topics where the literature is abundant and increasingly complex. However, because meta-reviews require the existence of multiple prior syntheses, they are less suited to emerging or underdeveloped topics.³⁰ Additionally, while meta-reviews are becoming more popular, no medical education-specific resources exist for those hoping to conduct one. Authors may find the seminal work by Aromataris et al useful, as it outlines best practices for conducting and reporting umbrella reviews, which can be extrapolated to other meta-review types.³¹

A realist review, a theory-driven form of knowledge synthesis, aims to understand not just whether an intervention works, but how, for whom, and under what circumstances it works.^32,33 Rooted in realist philosophy, this approach is particularly well-suited to complex interventions in dynamic settings (eg, clinical placements, academic health centers).³⁴ Unlike many knowledge synthesis types, which focus on aggregating findings across studies, realist reviews explore the underlying mechanisms of interventions and how these interact with contextual factors to produce specific outcomes. The goal is to generate or refine a “program theory,” which is a conceptual model that explains how and why an intervention leads to specific results in certain contexts. To strengthen their program theory, authors often integrate community members into the review process, which ensures that the perspectives of those affected by the intervention are considered. For example, in a realist review on interprofessional education (IPE), the authors developed a program theory, which incorporated 124 contexts, mechanisms, and outcomes, and conducted listening sessions with community members to gauge their relevance. The resulting model led to a practical resource for educators developing, delivering, and assessing IPE.³⁵

By focusing on real-world complexity, realist reviews can inform policy and practice in nuanced and complex settings where one-size-fits-all conclusions may be insufficient. To guide the conduct of realist reviews, researchers are encouraged to consult the RAMESES publication standards,³³ which provide a structured approach for developing, conducting, and reporting realist reviews. Recent field-specific guidance also has emerged in medical education.³⁶

Across the four knowledge synthesis types, several common challenges can undermine the quality and usefulness of the resulting work, though they may manifest in different ways depending on the approach. These challenges include underestimating the time needed, not including community members, producing findings not ready for real-world use, and not looking beyond usual information sources.

Conducting a knowledge synthesis is resource-intensive, requiring significant time, team coordination, and methodological expertise. For example, systematic reviews have been estimated to take on average 67 weeks³⁷ and 881 person hours³⁸ to complete. While time estimates for other knowledge synthesis types are unavailable, they are likely comparable. Unfortunately, authors may underestimate the scope and time commitment. To avoid this pitfall, authors should start with a clear and focused research question. They also may consider registering a protocol of their knowledge synthesis to help them maintain scope and direction. For guidance on protocols, consult Pieper and Rombey’s work,³⁹ which describes available protocol registers, registration benefits, and protocol characteristics. Engaging a librarian for help with searching and project management tools also can streamline the process. Finally, setting realistic timelines and budgeting sufficient time for each phase of the process can ensure that the knowledge synthesis remains manageable and methodologically sound.

Involving community members (aka stakeholders) in a knowledge synthesis enhances the relevance, reach, and real-world applicability of the findings.⁴⁰ It also fosters transparency, builds trust, and increases the likelihood that the synthesis will inform meaningful action or decision-making. Yet, the inclusion of community members in knowledge synthesis is an exception rather than a rule, with their inclusion primarily in scoping and realist reviews, despite their value across synthesis types.

Community members, such as patients, trainees, educators, clinicians, policymakers, or other end users, can contribute to knowledge syntheses as coauthors (when appropriate) or consultants throughout the process. Their involvement can help shape and refine the research question to ensure that it reflects real-world concerns, provide input on inclusion and exclusion criteria to capture diverse evidence forms, and assist in interpreting findings through the lens of lived experience. For example, in a scoping review on social media, the authors interviewed topical experts regarding whether the findings were consistent with their experience as social media researchers.⁴¹ Community members also may help identify important gray literature, codevelop dissemination strategies to reach intended audiences, and offer feedback on the clarity and accessibility of draft manuscripts. Despite these benefits, engaging community members can also introduce challenges, such as an extended project timeline, the need for training and support, and the difficulty of identifying and matching the right individuals to the appropriate knowledge synthesis steps.⁴⁰ Thoughtful planning and clear communication can help address these challenges and facilitate meaningful community member engagement.

Knowledge syntheses in medical education have been criticized for lacking relevance to, and for not being ready for translation into, practice.^2,42 When syntheses fail to address educators’ practical realities, they risk becoming disconnected from the learning environments where they are meant to be used. Similarly, knowledge syntheses that lack clear, practical recommendations may leave educators unsure how to implement findings.⁴³ For example, a knowledge synthesis that concludes “more research is needed” but does not help a reader make sense of the evidence presented is both frustrating and of limited use. One approach to address these issues is for authors to draw on the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework,⁴⁴ which facilitates the translation of research into practice.⁴⁵ While this framework can guide multiple knowledge synthesis steps, it is particularly helpful for data extraction. For this purpose, authors can use the published RE-AIM extraction template that facilitates extracting data relevant to the dimensions of the framework.^46,47 For example, a review on diabetes health coaching used the RE-AIM template to “elucidate the critical aspects of an intervention to ensure the adoption, scaling, and maintenance of an intervention.”⁴⁸

Identifying medical education literature can be challenging due to its dispersion across biomedical, education, and social science journals, as well as conference proceedings and gray literature sources. Compounding this difficulty is the inconsistency in indexing terms used to describe relevant publications, which can hinder comprehensive searching.^49,50 To ensure broad and inclusive retrieval, authors should look beyond usual sources and adopt a multiple database search strategy that includes MEDLINE, ERIC, EMBASE, PsycINFO, CINAHL, Scopus, and others, as appropriate. Additionally, preprint servers should be considered, because medical education literature is increasingly being disseminated as preprints prior to peer review.⁵¹ For example, medRxiv includes a dedicated category for medical education and can be a valuable source of emerging literature. Importantly, authors also should make intentional efforts to include non-English-language publications. Restricting knowledge syntheses to English-only sources, which is a common tendency,^52,53 can result in a narrow understanding of educational practices, privileging certain systems, values, and assumptions.

Medical education knowledge syntheses, like those in other fields,⁵⁴ have been criticized for being infrequently updated,⁴² which means they can be outdated and inaccurate. One approach has been to use living reviews. Living reviews are a form of knowledge synthesis that is continually updated to incorporate new evidence as it becomes available, ensuring that the review remains current and relevant over time.⁵⁵ Unlike traditional reviews, which represent the available evidence at a single point, living reviews are particularly useful in rapidly evolving fields or when timely guidance is needed for policy or practice.⁵⁵ For example, multiple living reviews are aimed at answering critical questions related to COVID-19.⁵⁶ Living reviews can enhance the usefulness of syntheses by reducing the lag between evidence generation and use, but they also demand sustained resources, clear protocols for when and how updates are made, and effective communication strategies to signal changes to end users.⁵⁷ Despite these challenges, living reviews hold promise for improving the responsiveness and relevance of medical education research, especially in areas where guidance must evolve alongside emerging data.

Knowledge syntheses are traditionally disseminated as journal articles, which can be jargon-dense and inaccessible behind subscription paywalls, limiting their practical impact. To enhance uptake, authors should consider how to move beyond traditional dissemination by using knowledge translation strategies tailored to their audiences. For example, visual summaries such as infographics, conceptual frameworks, or summary tables can distill complex findings into formats more usable by time-constrained educators.¹ Similarly, plain-language summaries can help bridge the gap between research and practice, making evidence more understandable to nonspecialist stakeholders, including learners or institutional leaders.⁵⁸ Dissemination via podcasts and social media platforms like Twitter/X or LinkedIn also can expand reach and stimulate discussion.⁴¹ More interactive approaches, such as developing searchable databases of studies or cohosting workshops to interpret findings with stakeholders, can foster deeper engagement and cocreation of actionable outcomes.²⁵ Toolkits and practical implementation guides are additional formats that translate findings into actionable steps.⁵⁹

The use of AI to conduct knowledge syntheses can streamline resource-intensive tasks such as literature searching and screening, data extraction, and evidence summarization.⁶⁰ For example, one author team reported completing a systematic review in 2 weeks using AI⁶¹ to search for and determine inclusion of articles and draft the manuscript, saving over a year compared to typical timelines for human-only conducted syntheses.³⁷ While these efficiencies are compelling, AI has limitations, and individuals are encouraged to use it thoughtfully and appropriately.⁶² AI tools may introduce bias, lack contextual understanding, or misinterpret nuanced data, especially in the oftentimes complex and nuanced step of data extraction.⁶³ Thus, adopting an approach in which AI supplements, but does not replace human involvement is critical. Humans remain vital for interpreting context, assessing relevance, and ensuring methodological rigor. Without human oversight, AI-generated outputs may appear convincing but lack the depth, accuracy, or educational relevance needed for meaningful knowledge translation. Additionally, authors should select AI tools carefully, ensuring that what is selected is appropriate for their task. As the use of AI continues to evolve, researchers should consider using AI but remain attentive to both its possibilities and limitations.

Knowledge syntheses are vital to medical education. When conducted thoughtfully, they not only inform curriculum design, teaching strategies, and policy decisions but also bridge the gap between research and practice by translating complex evidence into actionable insights. To realize their full potential, knowledge syntheses must be inclusive, methodologically sound, and focused on relevance and impact. As the field evolves, integrating innovations such as community engagement, AI, and living reviews will help ensure that syntheses remain timely and applicable.