Infrastructure Features Associated With Increased Department Research Capacity

Academic medical departments juggle multiple missions, balancing the classic three-legged stool: educating the next generation, delivering high-quality clinical care, and producing valuable research. ^1- 3 With more prioritization of educational and clinical demands and limited resources, academic departments commonly struggle with research productivity. ^{4, 5} This struggle is amplified within primary care specialties, in which clinical care and education are historically more valued than scholarship. ^6-10

Academic departments and residency programs that are productive at research tend to have robust infrastructure, including the presence of dedicated research faculty and staff, dedicated research time for faculty, embedded fellowship programs, and departmental strategic research plans. ^11- 14 However, for departments aiming to increase research productivity, the best path to success is unclear. Across specialties, a variety of interventions designed to increase department research have been published. ^{5, 7, 15-17} Most published interventions occurred at single institutions and provide limited understanding of how academic departments can generally increase research productivity over time.

Within the specialty of family medicine, few departments have highly productive research programs. ^{10, 11, 18} The differences between highly productive family medicine departments and less productive departments have been previously described. ^{11, 19} Potential mechanisms for increasing family medicine departmental research productivity include increasing protected faculty time, creating access to expertise in statistics or grant preparation, and developing interdisciplinary research teams. ^{5, 20, 21}

The Association of Departments of Family Medicine (ADFM) created a Research Capacity Scale (RCS) that classifies departments into one of five categories to describe departmental research capacity in a single, self-assessed survey item. ^{10, 11} shows the criteria for each research capacity level as defined by the ADFM and used internally for over a decade. These can be used to track changes in departmental research capacity across time. To enable comparison to these previous inquiries, we therefore chose to use the RCS as our departmental measure.

The five levels of family medicine department research capacity as defined by the ADFM. These levels have been used internally, without change, by ADFM for over 10 years.

Our study aimed to describe the distribution of infrastructure features available to departments at each of the five levels. Additionally, our study aimed to examine how the addition of infrastructure features is associated with movement along the scale across time in an exploratory manner. Specifically, we examined infrastructure features over which department chairs have substantial direct control and could be feasibly implemented within the department to increase research productivity.

Our study was survey research as part of an omnibus survey conducted by the Council of Academic Family Medicine Educational Research Alliance (CERA). CERA survey methodology has previously been described in detail. ²² After that we created survey items informed by the existing literature, the CERA Steering Committee (CERA-SC) evaluated questions for consistency with the overall subproject aim, readability, and consistency with existing evidence of reliability and validity. CERA-SC then conducted survey pretesting for flow, timing, and readability with family medicine educators who were not part of the target population. The rest of the CERA omnibus survey was not about research. The project was approved by the American Academy of Family Physicians Institutional Review Board. Data were collected from August 6 to 31, 2021.

We identified several candidate infrastructure features to include in the survey based upon prior literature. ^{5, 12- 15} Among the candidate features, we chose to include those mentioned most often and those that would be available to most academic family medicine departments. Among these, five were identified as substantially within the department chair’s control: including research in the department’s mission statement, identifying an individual other than the chair to be responsible for departmental research, developing a standardized mechanism for accessing statistical expertise, developing a standardized mechanism for accessing grant writing expertise, and employing a PhD researcher on faculty within the department. We focused most of our analysis on features within the control of chairs to modify because one of our goals was to provide concrete recommendations for chairs hoping to increase their research output.

Recognizing that most infrastructure changes require time to make a quantifiable impact, we defined a feature as long-term if it had been present for at least 6 years. We defined a feature as recently added if it had been added within the past 6 years. This time frame was chosen to balance providing enough time for a factor to have an impact on research productivity and a short enough time that most chairs will be able to adequately recall sufficiently. The one likely exception to this is protected faculty time for research. Due to its potential for more immediate impact, we defined a 3-year time frame. These time frames were chosen based upon our personal experiences with adding research infrastructure and the usual time frames results are likely to be experienced when factors such as training personnel, grant submission timelines, subject enrollment, and publication timelines are all factored in.

The sampling frame was all US family medicine department chairs. Email invitations to participate were delivered through the web-based interface Survey Monkey. Three follow-up emails to encourage nonresponders to participate were sent weekly after the initial email invitation and a fourth reminder was sent 1 day before the survey closed. On behalf of CERA, ADFM identified 200 department chairs at the time of the survey. Two chairs had previously opted out of CERA surveys so the survey was emailed to 198 chairs. Six invitations were returned undelivered, leaving 192 invitations successfully delivered.

Of 110 participants who initiated survey completion, six survey responses were abandoned after answering the first question and thus removed from analysis. The other 104 responses were included in the analysis (overall response rate of 54.2%).

We used descriptive statistics to describe the number of departments at each research capacity level and their available research infrastructure features. We used nonparametric tests to examine the impact of adding modifiable infrastructure features. We used unpaired t test for continuous variables. In the analysis of change over time, we focused on departments that were not at the highest level of research capacity 3 years prior to survey completion because we were most interested in how adding features was associated with increased capacity, defined as moving up at least one level. Additionally, we divided departments into quartiles by faculty size in order to further investigate how department size influences capacity.

Table 2 shows the distribution of family medicine departments classified by their chairs into 2021 research capacity levels and associated infrastructure features grouped by how much control the department chair would be expected to have over each. Most family medicine departments (83%) fall into the middle three research capacity levels (levels 2-4). Only 11% of departments met the criteria of the top level (level 1: extensive research). Conversely, only 6% of departments were at level 5 with essentially no research activity. For all the features examined, more departments at higher levels have the feature in place than departments at the lower levels. This was true for all features, regardless of the degree of chair control over the feature.

For the historical items, 95 respondents reported their department’s research level in 2018. Among the 86 departments in levels 2-5 in 2018, 37 (43%) climbed at least one level by 2021, indicating an increase in research capacity. Of these 37, 6 (16.2%) did so without adding any of the infrastructure features examined in this study. Another 6 (16.2%) added one infrastructure feature while 20 (54.1%) added three or more infrastructure features. Of the 74 departments in levels 1-4 in 2018, only 4(5.4%) reported dropping one or more levels.

Table 3 shows the associated outcomes of adding specific infrastructure features on a department’s research capacity. For each infrastructure feature, more departments increased research capacity from 2018 to 2021 if the feature had been added within 6 years (within 3 years for increasing dedicated faculty time). Two additions associated with a statistically significant increased research capacity were adding at least one PhD researcher (P<.001) and adding an institutional affiliation with a Clinical and Translational Science Award (CTSA; P=.01). An association of increased department research capacity was noted for the 58% of departments that added protected research time versus 37% of departments that did not add protected time (P=.07).

To explore potential differences in the effect of resources for departments at different stages, we comparing level 4-5 programs (lowest research capacity) to level 2-3 programs (moderate research capacity). The results were similar to results with all four levels combined with one exception: adding a standardized mechanism for accessing an individual with grant-writing experience was significantly associated with level 4-5 programs increasing their research capacity (P=.04).

Department size, as measured by full-time faculty, heavily influences RCS level. As shown in Table 2, the RCS top-level departments were, on average, five times larger than the bottom-level departments. In order to analyze this trend further, we divided departments roughly into quartiles according to size measured by faculty full-time equivalents (FTE; Table 4). Most of the departments in the lowest FTE quartile were in the lowest capacity levels while most of the highest FTE quartile departments were in the highest capacity levels. The middle quartiles by size had a much greater distribution across research capacity levels. Table 5 shows the results of adding infrastructure features that are within the chair’s control for the middle two quartiles of department size only. None of these reached statistical significance, which may be due to the smaller segmented sample size represented.

Chair-reported survey data reveal a wide variation in the research capacity of family medicine departments in the United States. Similar to prior research, our results show that academic department size is heavily associated with research capacity. ²¹ One potential interpretation of this finding is that not all family medicine departments are likely to be highly successful at producing research. Chairs need to consider where to invest resources and research might not be the best investment for the smallest departments. For the smallest departments, focusing on other forms of scholarship to promote a culture of inquiry may make more sense.

Our findings also suggest that at least some of the research capacity variation may be due to variation in infrastructure features, which also mirrors the conclusions of several prior studies of the variation in research productivity among academic medicine departments. ^{11, 18}

Larger departments are more likely located within institutions with more research infrastructure, while smaller departments are located within institutions that are less able to invest in research. In our analysis of the middle 50% of departments by size, few features reached statistical significance for increasing capacity. This may be due to the small sample size. The noted trends imply that adding a mechanism to consult for statistical expertise, adding a mechanism to consult for grant writing expertise, and adding a PhD researcher were more likely be associated with climbing up the scale. Our findings suggest that among resources that are within a department chair’s control, it is most likely that adding these infrastructure features to increase the capacity for research is most feasible and impactful for midsized departments. Further research on adding infrastructure features for medium-sized departments is warranted because these seem to be the departments with the largest variation in research productivity.

This study extends prior research in its analysis of how adding individual infrastructure features can increase research capacity. Nearly half of departments not already at the highest research capacity level managed to climb at least one level across just 3 years. Adding certain infrastructure features was statistically associated with increasing department research capacity. The most encouraging finding of this study is that one impactful infrastructure feature, adding a PhD researcher, is a feature over which department chairs have considerable influence. Still, 25% of departments that added a PhD researcher did not move up a capacity level in 3 years. Given what we know from prior literature, ⁵, ¹⁸ it is likely that how a particular researcher’s time is utilized makes a large difference. If a PhD researcher is added to increase research capacity, their role must be committed in a way to support that end. The largest potential challenge of this strategy is finding qualified PhD researchers willing to come to work in a family medicine department. Alternatively, some departments with a heavy focus on clinical productivity may find it logistically difficulty to hire a PhD with no clinical effort.

In line with previous suggestions that CTSAs offer an avenue to increase family medicine research, ²³ our findings demonstrate that aligning with a CTSA was associated with a climb on the scale. Aligning with a CTSA is much less within the control of a single department chair, but this information could be useful when trying to convince institutional leadership that seeking such an affiliation is valuable to increase research capacity. The location and missions of all CTSAs can be found online at the CTSA Program Hub Directory. ²⁴

Strengths of our study include a high response rate (>50%), compared to similar survey studies. Our study also sampled department chairs: a key, but difficult to access, group of informants. Our study utilized an existing scale of department research capacity that is validated against a series of empirical measures of research capacity. ¹⁰ Additionally, this study included measures of change over 6 years to provide a longitudinal description of infrastructure features.

Findings are also limited by the design of the study. The survey examined the research capacity of one specialty; results may not be entirely generalizable to other specialties. However, struggles in the area of research productivity are common across specialties, and the infrastructure features studied are applicable to all departments. Data collected in a survey is subject to response, social desirability, and recall biases. As with all CERA surveys, no information is available about nonresponding departments. The RCS, while useful, is at least partially subjective. It is likely the scale was not interpreted the same by all respondents. Each infrastructure feature was examined in isolation while, in reality, some of these features are linked. For example, adding a PhD researcher may also equate to adding a mechanism to obtain grant writing expertise. Lastly, because the time frame studied includes years 2020 and 2021, it is possible that our findings have been impacted by in unpredictable ways by the COVID-19 pandemic.

Our findings have important implications for department chairs and other institutional leaders attempting to increase departmental research capacity. To increase their departments’ research capacity, chairs should actively add resources. In fact, most departments increasing their research capacity had added multiple additional infrastructure features. Department chairs should focus on infrastructure features within their control that are associated with increased research capacity. For chairs of departments without a PhD researcher, this is the additional resource that is the most likely to increase research capacity in a relatively short period of time for the departments included in this study. At the institutional level, obtaining an affiliation with a CTSA was associated with increased family medicine department research capacity and should be widely advocated for.