A Curriculum to Increase Resident Scholarly Activity

Robert P. Lennon, MD, JD | Roselyn W. Clemente Fuentes, MD | Christine Broszko, MD | John J. Koch, MD | Kristian Sanchack, MD, MHA | James W. Keck, MD, MBA

Fam Med. 2020;52(8):557-561.

DOI: 10.22454/FamMed.2020.257274

Return to Issue


Background and Objectives: Scholarly activity (SA) is an Accreditation Council of Graduate Medical Education (ACGME) requirement for family medicine residents. Engaging residents in scholarly activity can be challenging. Naval Hospital Jacksonville Family Medicine Residency (NHJ) pioneered a curriculum that led to a dramatic, sustained increase in resident SA. We sought to implement the curriculum in other family medicine residency programs.

Methods: The curriculum was implemented at two additional family medicine residencies. Three curricular interventions were identified: a 3-hour case report workshop, a written practical guide to scholarly activity, and a resident peer research leader. One program implemented all three elements. The other implemented the workshop and written guide, but did not identify a resident peer leader. SA was measured using the annual ACGME program director report and compared the intervention year to the previous 3 years of SA using a 2-sample test for equality of proportions with continuity correction. We used pre- and postintervention surveys to evaluate resident attitudes about SA.

Results: The program implementing all three interventions increased residents’ conference presentation 302% (n=34, P<.001). The program that did not identify a resident peer leader had no significant change in SA as reported to the ACGME.

Conclusions: The curriculum was implemented in two additional residencies with promising results. We recommend further implementation across multiple sites to determine the extent to which the results are generalizable.

The Accreditation Council for Graduate Medical Education (ACGME) requires all programs to demonstrate evidence of scholarly activity.1 Refinement of resident requirements for scholarly activity are left to each specialty. Family medicine residents must participate in scholarship and should complete two scholarly activities prior to graduation, at least one of which should be a quality improvement project.2 To be recognized as scholarship, contributions must be shared with peers at a local or higher level and be subject to peer review.3 Widely recognized barriers to resident research include an absence or lack of protected time, access to mentors, a formal research curriculum, technical support, venues to present scholarship, and funding.4 Other barriers include a lack of interest5 and uncertainty.6 Myriad strategies have been developed within specific programs to overcome barriers to scholarly activity.7,8 Most of these strategies focus on overcoming the specific barriers listed above, and were well described by Seehusen and Weaver in 2009 across many programs,9 and Alweis et al in 2015 as iterative improvements within a single program.10

One successful alternative approach has been a scholarly activity point system, first described by Seehusen et al in 2009,11 and later shown to be successful at several other programs.12 Others include an improved journal club, a Wall of Fame, a writing seminar, and a cash incentive for residents.10 More recently, Seehusen et al have described the success of the Council of Academic Family Medicine Educational Research Alliance (CERA) efforts to build a research infrastructure for individuals external to their own programs in order to facilitate medical education research.13

While innovative and successful, we felt that greater change might be achieved if interventions were resident-driven, and developed a resident-driven curricular change at Naval Hospital Jacksonville Family Medicine Residency Program (NHJ). This resulted in a 13-fold increase in research activity and a seven-fold increase in resident participation in research.14 These improvements were sustained over the 5 years following implementation.15 In this study, we evaluate the outcomes after implementing this resident-driven curriculum at two additional family medicine residencies.


This was a prospective, multisite trial. The Naval Medical Center Portsmouth Institutional Review Board, deemed this project exempt, and it was funded through a grant from the Uniformed Services Academy of Family Physicians.

We distilled three core elements that led to NHJ’s success: a resident peer leader in research (the resident research coordinator, or RRC), a 3-hour case report workshop (CRW), and a guidebook to scholarly activity. The RRC is a self-selected resident who undertakes the position to champion research within their program, and is described in detail elsewhere.14,16 RRCs provide encouragement, direction, and support to residents pursuing scholarly activity. The CRW is a 3-hour, hands-on workshop available for use or adaptation from the Society of Teachers of Family Medicine Resource Library,17 and is described in detail elsewhere.14 The goal of the workshop is the creation of a near-complete abstract for submission. The guidebook used is The Recipe: A Practical Guide to Scholarly Activity, and was developed to provide residents with step-by-step instructions on scholarly activities from performing a literature search to conducting a process improvement project to writing a grant proposal. It is freely available for use online through both the Uniformed Services Academy of Family Physicians and the Society of Teachers of Family Medicine.16 Electronic versions of The Recipe were provided to all residents and faculty in fall of 2018.

Two full-scope ACGME-accredited military family medicine residency programs were invited to participate. Site A adopted all three intervention elements, identifying two residents to serve as research peer leaders, hosting a CRW and providing residents and faculty with The Recipe. Site B hosted the CRW and provided residents and faculty with The Recipe, but did not identify any residents to serve as research peer leaders.

Site A is a 12/12/12 program, and Site B is a 13/13/13 program. Each self-identified as a program desiring to increase their residents’ scholarly activity. While similarly sized military residencies, they are different at baseline in faculty research engagement and resident research time allotment. Several faculty at Site A are actively engaged in research; their residents receive no dedicated research time. Faculty at Site B are less actively engaged in research; their residents are given a 4-week block for research during their PGY-3 year.

Primary Outcome Measures

It is challenging to define scholarly activity in a way that is rigorous, precise, generalizable, and timely. We have previously defined it as any activity generating scholarly activity points,14 or more rigorously, as presentation at a regional or higher conference or publication in a PubMed-indexed, peer-reviewed journal with a Medline number.15 To make measurements more generalizable, in this study we use data reported on the scholarly activity report by the program director (PD) to the ACGME. The reportable categories in the ACGME resident scholarly activity report include conference presentations, textbook chapters, research participation, teaching presentations and scholarly activity participation. Virtually all residents report teaching and some form of scholarly activity every year. For this reason, our primary outcome measures were limited to changes to conference presentation, textbook chapters and research participation. “Conference presentations” are abstracts, posters, or oral presentations given at regional or higher level conferences. “Research participation” includes participation in a funded or nonfunded basic science or clinical outcomes research project.

The PD at each site provided their ACGME-reported resident scholarly activity for academic years 2015-2016, 2016-17 and 2017-18 as baseline data. The intervention occurred in the 2018-19 academic year. In August 2019, the PD at each site provided their ACGME reported resident scholarly activity for the 2018-19 academic year. We compared pre- and postintervention ACGME-reported data using a 2-sample test for equality of proportions with continuity correction to 95% confidence, and confirmed with an alternate critical ratio test.

Secondary Outcome Measures

We collected resident perceptions about case reports before the CRW. We obtained these secondary outcomes to compare their perceptions of scholarly activity participation to their objective performance (measured as primary outcomes). Mismatches between resident perception of performance and objective measures of performance may highlight learning opportunities or knowledge gaps that hinder research. Residents were surveyed before the CRW, and resurveyed 6 months later for comparison. We assessed these changes with a 2-tailed Mann-Whitney U test with significance set at P<.05.


Primary Outcomes

Postintervention conference presentation at Site A increased 302% compared to its previous 3-year average (n=34, P<.001), and 91% compared to its preintervention year (n=34, P<.001; Tables 1 and 2). Residents at site A had no significant change to reported textbook chapters or research participation. Residents at site B had no significant change in any scholarly activity reported to the ACGME.

Secondary Outcomes

Site A residents perceived a 217% increase in case submissions (n=12, P=.0198) after the CRW, but no other significant increases in how they viewed cases (Table 3). Site B residents perceived a 217% increase in case consents (n=12, P=.0404), a 163% increase in case reports drafted (n=12, P=.0316), and a 1,000% increase in cases submitted (n=12, P=.0383), but no other significant increases in how they viewed cases.

External constraints prevented our planned statistical analysis of resident perceptions. We intended to use a Wilcoxon Signed Rank test. However, because of technical constraints, we were unable to pair pre and postsurveys. That technical limitation also means that we cannot say for certain that all respondents to the postsurvey in fact participated in the CRW and completed a presurvey; for this reason our survey data does not meet strict criteria for any statistical test. However, it is unlikely that any resident who did not participate in the CRW and complete the presurvey would have completed a postsurvey. We therefore report the survey data using a Mann-Whitney U test, with the caveat that technical criteria for using that test are not met.


This study shows a correlation between the curriculum intervention and increased scholarly output at the residency site implementing all three components. Further study is needed to identify a causal link to this observed correlation, address the limitations of this study, and confirm that the correlation observed was not caused by the baseline differences between the programs.

Those baseline differences in residency characteristics and lack of randomization preclude claiming a correlation between the RRC position and an increase in scholarly activity. However, it is noteworthy that the program that adopted the RRC had significant objective improvement in resident productivity, while the site without an RRC improved only in resident perceptions. Perhaps filling knowledge gaps with interventions like the CRW and The Recipe guidebook are necessary but insufficient to increase resident scholarship, and resident peer leaders allow residents to overcome the uncertainty barriers to scholarly activity described by Ledford et al.6 A qualitative study may help identify a correlation, and would also help identify other characteristics of the culture of scholarship that appears to be a common theme across successful scholarly activity interventions.8 Other limitations to our study include short follow-up time, which may explain why conference participation increased, but longer lead time outcomes like textbook chapters and research participation did not.

Because of the technical limitations in secondary outcomes described above, the survey data must be considered anecdotal. The surveys were deidentified, so it is not possible to directly compare resident self-assessment with actual output. Further limitations include small sample size and implementation only in larger military residencies. However, as none of these interventions are military-specific, and each program operates under the same ACGME and American Board of Family Medicine requirements, we think it likely that the interventions are generalizable to civilian programs.

Another limitation is the larger size of these programs; smaller programs may find it more challenging to identify and support a resident peer leader. However, a strength of the study is the low cost of these interventions in time and money, and hence their utility even to programs with limited resources. The CRW requires a 3-hour time commitment, and has materials for presentation freely available. The Recipe guidebook is also freely available. Resident peer leaders require no direct compensation. NHJ has given an average of 2 weeks of time over the course of a year to its resident peer leaders. This is usually distributed as one half day per month, with extra time around submission and conference deadlines, and is counted as a research elective.

Another strength of this study is its use of ACGME-reported scholarly activity to measure improvements. This overcomes prior study limitations using “quality” or “high quality” measures that lacked universal use, and helps to demonstrate the generalizability of our results. Another strength of this study is its prospective comparison of similar programs.

The correlation between the curriculum and scholarly output at two additional programs is promising. Future opportunities for research include longevity evaluations of site A’s successes and larger randomized controlled trials including smaller programs and civilian programs to address the limitations outlined above. Including a qualitative component to these studies may also help determine causative elements of changes to programs’ research outputs.


The authors thank Drs Joshua Hodge, Christy Ledford, Greg Jones, Noa Hammer, Stephanie Fulleborn, and Sarah Carroll for their support of this project.

FUNDING SUPPORT: Funding support was provided through a grant from the Uniformed Services Academy of Family Physicians, grant number 18LG-1.

PRIOR PRESENTATION: This study was presented as a work-in-progress at Society of Teachers of Family Medicine Annual Spring Conference, May, 2019, in Toronto, Canada.

This study was accepted for presentation at Uniformed Services Academy of Family Physicians Annual Conference, 2020, and Society of Teachers of Family Medicine Annual Conference, 2020.

DISCLAIMER: The views expressed in this article are those of the authors and do not necessarily reflect the official policy or position of the US Department of the Air Force, the US Department of the Navy, the Department of Defense, or the United States Government.


  1. Accreditation Council for Graduate Medical Education. ACGME Common Program Requirements (Residency). Scholarship. 2019. https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/CPRResidency2019.pdf. Accessed November 1, 2019.
  2. Accreditation Council for Graduate Medical Education. ACGME Program Requirements for Graduate Medical Education in Family Medicine. Scholarship. 2019. https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/120_FamilyMedicine_2019_TCC.pdf?ver=2019-06-13-074503-640. Accessed November 1, 2019.
  3. Accreditation Council for Graduate Medical Education. Scholarly Activity Guidelines. Review Committee for Family Medicine. 2019. https://www.acgme.org/Portals/0/PFAssets/ProgramResources/120_scholarly-activity-guidelines.pdf. November 1, 2019.
  4. Rothberg MB. Overcoming the obstacles to research during residency: what does it take? JAMA. 2012;308(21):2191-2192. doi:10.1001/jama.2012.14587
  5. Senf JH, Campos-Outcalt D, Kutob R. Family medicine specialty choice and interest in research. Fam Med. 2005;37(4):265-270.
  6. Ledford CJ, Seehusen DA, Villagran MM, Cafferty LA, Childress MA. Resident scholarship expectations and experiences: sources of uncertainty as barriers to success. J Grad Med Educ. 2013;5(4):564-569. doi:10.4300/JGME-D-12-00280.1
  7. Stevenson MD, Smigielski EM, Naifeh MM, Abramson EL, Todd C, Li ST. Increasing scholarly activity productivity during residency: a systematic review. Acad Med. 2017;92(2):250-266. doi:10.1097/ACM.0000000000001169
  8. Wood W, McCollum J, Kukreja P, et al. Graduate medical education scholarly activities initiatives: a systematic review and meta-analysis. BMC Med Educ. 2018;18(1):318. doi:10.1186/s12909-018-1407-8
  9. Seehusen DA, Weaver SP. Resident research in family medicine: where are we now? Fam Med. 2009;41(9):663-668.
  10. Alweis R, Wenderoth S, Donato A. Effectiveness of iterative interventions to increase research productivity in one residency program. J Community Hosp Intern Med Perspect. 2015;5(6):10.3402
  11. Seehusen DA, Asplund CA, Friedman M. A point system for resident scholarly activity. Fam Med. 2009;41(7):467-469.
  12. Seehusen DA, Ledford CJ, Grogan S, et al. A Point System as Catalyst to Increase Resident Scholarship: an MPCRN Study. Fam Med. 2017;49(3):222-224.
  13. Seehusen DA, Mainous AG III, Chessman AW. Creating a centralized infrastructure to facilitate medical education research. Ann Fam Med. 2018;16(3):257-260. doi:10.1370/afm.2228
  14. Lennon RP, Oberhofer AL, McNair V, Keck JW. Curriculum changes to increase research in a family medicine residency program. Fam Med. 2014;46(4):294-298.
  15. Seales SM, Lennon RP, Sanchack K, Smith DK. Sustainable Curriculum to Increase Scholarly Activity in a Family Medicine Residency. Fam Med. 2019;51(3):271-275. doi:10.22454/FamMed.2019.906164
  16. Rutherford AL, Lennon RP, Seehusen DA, eds. The recipe: a practical guide to scholarly activity. Richmond, VA: USAFP, 2019. https://www.usafp.org/research/wp-content/uploads/2018/12/the-recipe-2018.pdf. Accessed November 1, 2019.
  17. Bunt C, Oberhofer A, Keck JW, Latimer K, Patterson P, Lennon RP. Case Report Workshop STFM 2014. https://resourcelibrary.stfm.org/resourcelibrary/viewdocument/case-report-workshop. Accessed November 19, 2019.

Lead Author

Robert P. Lennon, MD, JD

Affiliations: Penn State College of Medicine, Department of Family and Community Medicine, Hershey, PA


Roselyn W. Clemente Fuentes, MD - Department of Flight Medicine, Eglin Air Force Base Hospital, 96th Medical Group, Eglin AFB, FL

Christine Broszko, MD - Eglin Air Force Base Family Medicine Residency Program, Eglin Air Force Base, FL

John J. Koch, MD - Naval Hospital Camp Pendleton, Department of Family Medicine, Camp Pendleton, CA

Kristian Sanchack, MD, MHA - Naval Hospital Jacksonville, Department of Family Medicine, Jacksonville, FL

James W. Keck, MD, MBA - Naval Hospital Jacksonville, Department of Family Medicine, Jacksonville, FL

Corresponding Author

Robert P. Lennon, MD, JD

Correspondence: Penn State College of Medicine, Department of Family and Community Medicine, 700 HMC Crescent Road, Hershey, PA 17033. 904-588-2621. Fax: 717-531-4327.

Email: rlennon@pennstatehealth.psu.edu

Fetching other articles...

Loading the comment form...

Submitting your comment...

There are no comments for this article.

Downloads & Info


Related Content