Background and Objectives: Nurturing a research culture in family medicine practice enables identification of health problems and their solutions at the grassroots level. Mandatory research during family medicine residency can serve this purpose. The Saudi Board Family Medicine (SBFM) program in Qassim provides comprehensive research training and mandates the submission of a full research project for residency completion. The aim of this study was to evaluate the research training at the SBFM program in Qassim, applying Kirkpatrick’s evaluation model.
Methods: We evaluated the research training of five cohorts, enrolled 2019 to 2023. To evaluate the research training, we analyzed the research course’s pretest/posttest, cumulative results, and satisfaction survey; residents’ perceived research skills; research project completion; and publication rate. We used descriptive and comparative statistics for data analysis.
Results: During the years 2019 to 2023, 94 residents attended the research course. For all cohorts, the difference in the pretest and posttest mean scores was statistically significant, and the mean score of cumulative results exceeded 90%. The mean scores for perceived research skills increased after completion of the research course, and the level of satisfaction remained consistently high throughout the years. Among the three cohorts completing residency, 100% (n = 51) of the research projects were completed, and 45 (88%) were published in international, peer-reviewed journals.
Conclusions: The research training has been successfully implemented, and the high publication rate demonstrates its impact on research productivity. The study highlights the importance of research training during family medicine residency in cultivating a research culture and promoting evidence-based practice in primary care.
Primary health care is the cornerstone of health systems, and family medicine is its key discipline beginning with the patient’s first contact. The broad spectrum of health issues at primary health care makes it an ideal setting for research.1-3 Systems where clinicians are actively involved in research show improved outcomes and satisfaction.4
Critical thinking is an essential component of research. Thus, nurturing a research culture in family medicine improves physicians’ skills and strengthens their role in health care systems.5-7 However, family medicine is generally perceived as a less research-oriented environment,8-10 and research during training is considered to compromise the more important task of clinical activities learning,11 resulting in lower publication output in family medicine compared to other medical specialties.12,13
Although research is encouraged in Saudi health care, gaps exist in addressing primary health care problems, leaving most evidence-based care reliant on international studies. These gaps can be filled by conducting local research.2 The more family medicine research is supported, the better the health care and population health will be.3,14
The Saudi Commission for Health Specialties curriculum requires mandatory submission of a research proposal to the Saudi Board Family Medicine (SBFM) program, while the complete research project is left to the discretion of individual institutions.15 The Family Medicine Academy in Qassim mandates completion of a full research project to attain the training certificate. To facilitate the residents’ research, a 2 week research course is conducted in the first year of residency. The research course follows the Commission’s guidelines.15 Residents submit proposals in their second year and complete research projects in their third year. The top three projects are recognized each year by a consultant panel using predefined criteria, including topic importance to family medicine, relevance to primary health care administration, and journal impact.
The SBFM program in Qassim includes a training course on comprehensive research and completion of full projects. This study evaluates the training using Kirkpatrick’s evaluation model,16 assessing residents’ satisfaction, perceived knowledge and skills, course outcomes, project completion, and publications.
We conducted a retrospective study using data from training records and surveys completed by residents at the Family Medicine Academy in Qassim. Ninety-four residents enrolled in the SBFM program during the years 2019 to 2023 attended the research course. For analysis, we categorized residents into five cohorts based on their year of enrollment (2019–2022, 2020–2023, 2021–2024, 2022–2025, and 2023–2026).
The Research Course
The Family Medicine Academy in Qassim conducts a 2 week annual on-site research course each January for first-year residents, covering epidemiology, study designs, statistics, and screening, and using lectures, discussions, assignments, tutorials, and practical group work. The objectives of the research course are in accordance with those outlined by the 2024 SBFM curriculum.15
Assessment Tools and Measures
At the conclusion of each research course day, residents completed a satisfaction questionnaire evaluating the session on a 5-point Likert scale. Perceived research skills were self-rated (1–10) before the beginning and after the end of the course. Pretest and posttest (10 multiple-choice questions each), weekly quizzes, and both graded and ungraded assignments were used to assess the knowledge of the residents. Thus, a comprehensive evaluation of residents’ learning outcomes was conducted.
Evaluation Model
We applied Kirkpatrick’s model to evaluate the research training. As a standard assessment tool, this model is widely used to evaluate training.17-19 The model focuses on four levels: (1) reaction of participants (the satisfaction of participants during the training); (2) participants’ learning (their knowledge and skills obtained from the training); (3) behaviors of participants (practically applying knowledge and skills); (4) the overall results (impact of the training on the institution).
Data Management and Analysis
We used descriptive and inferential statistics to analyze the data, which were collected via Google Forms and analyzed in SPSS 21 (IBM). We analyzed the pre- and postcourse score differences using a paired t test (P<0.05 significant).
Ethical Considerations
All data were anonymized prior to analysis. Ethical approval for the study was obtained from the Qassim Research Ethics Committee (Approval No. 607-46-012704).
A total of 94 residents attended the research course across five cohorts: 12 (2019–2022), 17 (2020–2023), 22 (2021–2024), 23 (2022–2025), and 20 (2023–2026). All residents (100%) successfully completed the course.
Level 1: Reaction
Residents’ reactions were assessed through daily satisfaction surveys, and research skills were self-rated before and after the course. The survey evaluated session objectives, teaching methods, interactivity, knowledge improvement, and feedback. Mean satisfaction scores remained consistently high throughout the years, with the majority of the scores above 4 (Table 1). Perceived research skills also increased significantly in all except one cohort (2019–2022). Standard deviations were smaller postcourse, reflecting more consistent perceived skill gain (Table 2).
Cohort |
Research course year |
Week 1a |
Week 2a |
|
|
Day 1 |
Day 2 |
Day 3 |
Day 4 |
Day 1 |
Day 2 |
Day 3 |
Day 4 |
2019–2022 |
2020 |
4.92±0.06 |
4.91±0.05 |
4.86±0.09 |
4.88±0.11 |
4.81±0.14 |
4.82±0.07 |
4.99±0.02 |
4.69±0.15 |
2020–2023 |
2021 |
4.56±0.33 |
4.68±0.39 |
4.54±0.40 |
4.51±0.48 |
4.58±0.16 |
4.62±0.21 |
4.79±0.12 |
4.89±0 |
2021–2024 |
2022 |
4.68±0.09 |
4.71±0.08 |
4.19±0.43 |
4.04±0.22 |
4.47±0.29 |
4.33±0.24 |
4.68±0.19 |
4.86±0.1 |
2022–2025 |
2023 |
4.14±0.79 |
4.12±0.82 |
4.2±0.71 |
4.16±0.68 |
4.24±0.72 |
4.15±0.9 |
4.33±0.65 |
3.94±1.15 |
2023–2026 |
2024 |
4.51±0.81 |
4.96±0.23 |
4.9±0.31 |
4.97±0.2 |
4.62±0.29 |
4.71±0.7 |
4.76±0.27 |
4.29±0.8 |
Cohort |
Research course year |
Residents’ perceived research skills |
|
|
|
|
Min |
Max |
Mean score |
SD |
P value |
2019–2022 |
2020 |
Before the research course |
1 |
9 |
3.7 |
2.5 |
.0518 |
|
|
After the research course |
3 |
8 |
5.8 |
1.7 |
2020–2023 |
2021 |
Before the research course |
1 |
5 |
3.1 |
1.4 |
.0001 |
|
|
After the research course |
4 |
10 |
7.6 |
1.6 |
2021–2024 |
2022 |
Before the research course |
1 |
10 |
4.7 |
4 |
<.0001 |
|
|
After the research course |
5 |
10 |
7.7 |
1.3 |
2022–2025 |
2023 |
Before the research course |
1 |
10 |
4.6 |
1.9 |
<.001 |
|
|
After the research course |
3 |
10 |
7 |
1.5 |
2023–2026 |
2024 |
Before the research course |
1 |
10 |
5.5 |
2.4 |
.006 |
|
|
After the research course |
4 |
10 |
7.65 |
1.7 |
Level 2: Learning
Learning outcomes were measured from pre-/posttests, assignments, and final course results. Pretest means ranged from 5.2 (±1.5) to 7.11 (±1.45). Posttest scores improved substantially throughout the years. Differences between pre- and posttest scores were statistically significant across all cohorts (Table 3). Final cumulative scores exceeded 90% for every cohort, with the 2021–2024 cohort scoring the highest at 99.3%.
Cohort |
Research course year |
|
Minimum |
Maximum |
Mean score |
SD |
P value |
2019–2022 |
2020 |
Pretest |
4 |
9 |
7.1 |
1.4 |
<.0001 |
|
|
Posttest |
10 |
10 |
10 |
0 |
2020–2023 |
2021 |
Pretest |
3 |
8 |
5.2 |
1.5 |
<.0001 |
|
|
Posttest |
9 |
10 |
9.6 |
0.5 |
2021–2024 |
2022 |
Pretest |
4 |
10 |
6.2 |
1.7 |
<.0001 |
|
|
Posttest |
9 |
10 |
9.8 |
0.7 |
2022–2025 |
2023 |
Pretest |
2 |
9 |
6 |
1.8 |
<.0001 |
|
|
Posttest |
4 |
10 |
9.1 |
1.5 |
2023–2026 |
2024 |
Pretest |
1 |
9 |
5.7 |
2.1 |
<.0001 |
|
|
Posttest |
6 |
10 |
8.8 |
1.1 |
Level 3: Behavior
Residents’ behavior was assessed by participation in assignments, selection of a supervisor, and signed research agreement. Completion rates for assignments ranged from 80%–100%. By the end of the research course, all residents chose supervisors and signed agreements stating tentative topics.
Level 4: Results
We evaluated program outcomes based on research project completion and publications. Of the three cohorts completing residency, all 51 projects (100%) were completed, and 45 (88%) were published in international peer-reviewed journals. The research projects covered topics related to family medicine and primary health care.
This study showed that the residents were highly satisfied, and their knowledge and perceived research skills improved significantly after the 2 week research course. Moreover, they successfully applied acquired skills to complete and publish their research projects.
Pre- and posttest results confirmed significant knowledge gains. This outcome may be attributed to residents’ medical background and their recognition of the importance of research in health care. Studies have suggested that personal experiences and trainees with higher degrees often experience enhanced self-efficacy after training.20-2 The residents in this study were motivated to gain new skills, consistent with previous findings that trainees’ attitudes and behaviors influence training performance.22,23 A previous study showed that the residents completing projects showed greater confidence and research involvement and that structured research programs enhance the perceived value of projects.24 The Family Medicine Academy achieves a high level of research completion and publication rate through ongoing supervision, mandatory research submission, publication incentives, and annual recognition of the top three projects.
Publication of findings is crucial for scientific advancement. We were encouraged to find that a substantial proportion of projects in this study were published in peer-reviewed journals.25-29 In contrast, a study among Saudi Board residents in the Eastern Province found that only 26% of junior level residents had one publication.30 In the United States, only 16.9% of family medicine programs reported at least 25% of residents publishing.31
Limitations of our study included small cohort sizes and absence of a comparable control group or an example from literature describing a similar population for comparison.
The research training improved residents’ knowledge and perceived skills, highlighting the importance of such training during family medicine residency in cultivating a research culture and promoting evidence-based practice in primary care. Future research should assess the impact of the published research based on citations, presentations, and policy outputs, as well as continuation of the research activities 5 to 10 years after residents’ graduation. We recommend an increase in research opportunities and resources within residency programs. Strengthening family medicine research at the primary care level will lead to improved population health in Saudi Arabia.
DATA AVAILABILITY STATEMENT
Data are available upon request via e mailing the corresponding author.
The authors express their sincere gratitude to the faculty members of the Family Medicine Academy in Qassim for their invaluable support and dedication throughout the research training. Special thanks also are extended to the secretarial staff for their administrative assistance and coordination, which greatly contributed to the success of the research training.
References
-
Bonfim D, Belotti L, de Almeida LY, et al. Challenges and strategies for conducting research in primary health care practice: an integrative review.
BMC Health Serv Res. 2023;23(1). doi:10.1186/s12913-023-10382-1
-
Norman WV. Do you have a secret researcher inside?: High-quality family medicine research at Family Medicine Forum.
Can Fam Physician. 2014;60(7):602–604. https://www.cfp.ca/content/60/7/602
-
-
Brandenburg C, Noble C, Wenke R, et al. Relationship between research culture and research activity of medical doctors: a survey and audit.
J Multidiscip Healthc. 2021;14:2137–2150. doi:10.2147/JMDH.S319191
-
Del Mar C, Askew D. Building family/general practice research capacity.
Ann Fam Med. 2004;2 Suppl 2(Suppl 2):S35–40. doi:10.1370/afm.146
-
Ameh PO, McGuire CM, Van Waes A, et al. Research activity, facilitators and barriers amongst trainee and early-career family physicians in sub-Saharan Africa: A cross-sectional survey.
Afr J Prim Health Care Fam Med. 2022;14(1):e1–e10. doi:10.4102/phcfm.v14i1.3367
-
van Weel C, Rosser WW. Improving health care globally: a critical review of the necessity of family medicine research and recommendations to build research capacity.
Ann Fam Med. 2004;2 Suppl 2(Suppl 2):S5–16. doi:10.1370/afm.194
-
De Maeseneer JM, van Driel ML, Green LA, van Weel C. The need for research in primary care.
Lancet. 2003;362(9392):1314–1319. doi:10.1016/S0140-6736(03)14576-X
-
Friedman RH, Wahi-Gururaj S, Alpert J, et al. The views of U.S. medical school deans toward academic primary care.
Acad Med. 2004;79(11):1095–1102. doi:10.1097/00001888-200411000-00018
-
Svab I. Changing research culture.
Ann Fam Med. 2004;2 Suppl 2(Suppl 2):S30–4. doi:10.1370/afm.150
-
Koo J, Bains J, Collins MB, Dharamsi S. Residency research requirements and the CanMEDS-FM scholar role: perspectives of residents and recent graduates.
Can Fam Physician. 2012;58(6):e330–6. https://www.cfp.ca/content/58/6/e330
-
-
-
McGuire CM, Fatusin BB, Kodicherla H, et al. implementation of online research training and mentorship for sub-Saharan African family physicians.
Ann Glob Health. 2021;87(1). doi:10.5334/aogh.3171
-
-
Kirkpatrick DL, Kirkpatrick JD. Evaluating Training Programs: The Four Levels. 3rd ed. Berrett-Koehler; 2006.
-
Almeneessier AS, AlYousefi NA, AlWatban LF, et al. evaluation of educational workshops for family medicine residents using the Kirkpatrick framework.
Adv Med Educ Pract. 2021;12:371–382. doi:10.2147/AMEP.S283379
-
Dorri S, Akbari M, Dorri Sedeh M. Kirkpatrick evaluation model for in-service training on cardiopulmonary resuscitation.
Iran J Nurs Midwifery Res. 2016;21(5):493–497. doi:10.4103/1735-9066.193396
-
-
Maxwell AE, Danao LL, Cayetano RT, Crespi CM, Bastani R. Evaluating the training of Filipino American community health advisors to disseminate colorectal cancer screening.
J Community Health. 2012;37(6):1218–1225. doi:10.1007/s10900-012-9557-9
-
Bandura A. Self-efficacy: toward a unifying theory of behavioral change.
Psychological Review. 1977;84(2):191–215. doi:10.1037/0033-295X.84.2.191
-
Gao P, Xiang H, Liu S, et al. Evaluation of a consulting training course for international development assistance for health.
BMC Med Educ. 2018;18(1). doi:10.1186/s12909-018-1339-3
-
Wang C, Xiang H, Xu Y, et al. Improving emergency preparedness capability of rural public health personnel in China.
Public Health. 2010;124(6):339–344. doi:10.1016/j.puhe.2010.02.020
-
Leahy N, Sheps J, Tracy CS, Nie JX, Moineddin R, Upshur REG. Family physicians’ attitudes toward education in research skills during residency: findings from a national mailed survey.
Can Fam Physician. 2008;54(3):413–414. https://www.cfp.ca/content/54/3/413
-
AlSaeed AA, Rabbani U. Explaining COVID-19 vaccine rejection using social cognitive theory in qassim, Saudi Arabia.
Vaccines (Basel). 2021;9(11). doi:10.3390/vaccines9111304
-
Alabdullgader A, Rabbani U. Prevalence and risk factors of falls among the elderly in unaizah City, Saudi Arabia.
Sultan Qaboos Univ Med J. 2021;21(1):e86–e93. doi:10.18295/squmj.2021.21.01.012
-
Alobaysi HM, Jahan S. Infant care practices among mothers attending well-baby clinics at primary health care centers in Unaizah city.
J Family Med Prim Care. 2022;11(8):4766–4772. doi:10.4103/jfmpc.jfmpc_329_22
-
Abumismar A, Jahan S. Assessing functional status in geriatric patients attending primary healthcare centers in Qassim Region, Saudi Arabia.
Cureus. 2024;16(6). doi:10.7759/cureus.62012
-
Alharbi A, Almogbel F, Rabbani U, Memish ZA. Long COVID-19 and coexistence of fatigue and depression: a cross-sectional study from Saudi Arabia.
J Epidemiol Glob Health. 2024;14(4):1602–1608. doi:10.1007/s44197-024-00312-7
-
Saud AlEnazi A, Alamri AS, AlGhamdi AS, et al. Perceptions, barriers, and attitudes toward research among in-training physicians in Saudi Arabia: a multicenter survey.
Sci Prog. 2021;104(2). doi:10.1177/00368504211010604
-
Ringwald BA, Taylor M, Seehusen DA, Middleton JL. Family medicine resident scholarly activity infrastructure, output, and dissemination: a CERA Survey.
Ann Fam Med. 2024;22(5):400–409. doi:10.1370/afm.3160
There are no comments for this article.