The Current State of Research Capacity in US Family Medicine Departments

Medical specialties are characterized by a definable body of medical knowledge applicable to patient care, and a scientific foundation with active investigation and new knowledge generation.1 Family medicine has defined an emerging body of knowledge, has quintessential applicability to patient care, and research capacity has grown incrementally since the founding of the specialty in 1969.2 There have been numerous calls, rationales, and strategies put forth for training more researchers, building greater research capacity in the discipline, and establishing a more robust peer-reviewed knowledge base in family medicine.2-9 However, achievement of these goals has been measured only sporadically and inconsistently. Past work has documented definite strides forward but also often noted that the family medicine research enterprise falls short of what is needed to address intractable problems and optimally improve the health of the nation.2,9,10 While studies have reported on individual faculty productivity,11,12 residency program productivity,13-16 numbers of publications,17-24 amount of funding from national sources,25-28 publication quality,29 and research infrastructure and barriers to productivity,30 the family medicine department research landscape has not been systematically assessed using a consistent conceptual framework.6,10 The growth in capacity for research in family medicine is therefore difficult to describe as a longitudinal enterprise.

To better understand the current state and to establish a baseline from which to monitor future growth of research capacity in family medicine departments, Family Medicine for America’s Health (FMAHealth), the Association of Departments of Family Medicine (ADFM), and the Council of Academic Family Medicine (CAFM) Educational Research Alliance (CERA) collaborated to develop a survey tool in 2016, guided by the Bland model29,30 to measure research capacity in family medicine departments in the United States. We describe the findings of this survey in this report.

A guiding goal of this survey was to generate a minimum set of well-established research capacity measures associated with research productivity from an easily replicated and reliable data source. Our expectation is that establishing a baseline of measures will provide a beginning to guide current efforts to build research capacity, in particular the Building Research Capacity (BRC) initiative,5 and create an opportunity to track long-term trends.

ADFM has defined core characteristics that meaningfully reflect departmental research capacity. We have operationalized those characteristics using measures that would generate some of the minimum data needed to mark meaningful progress over time. ADFM (authors A.W. and B.E.), FMAHealth Research Tactic Team (L.P.), and CERA (A.M.) investigators developed survey items on research capacity, attitudes toward research, and barriers and facilitators of research success at the department and institution level. The findings from these latter measures will be reported in a subsequent document.

Surveys were sent by email to chairs of family medicine departments who were members of ADFM in September 2016 using the standard CERA survey methodology.31 ADFM includes virtually all allopathic departments, many regional branch campuses, several osteopathic departments, and several departments in large regional medical centers that have research and medical education missions. Respondents were instructed to complete all data with reference to the 2015 calendar or 2015 fiscal year only. During the study period, nonresponders received four reminder messages. The survey was closed in November 2016.

The Institutional Review Board of the American Academy of Family Physicians approved this study.

The survey included questions about the chair, department, and institutional background and demographics. Survey items were constructed to empirically measure variables from four general categories known to characterize research-productive departments in the Bland model.32,33 The four categories selected for this study were: Trained research faculty: (1) number of individual faculty with dedicated research effort, and (2) number of full-time equivalents (FTE) of faculty dedicated to research. Infrastructure: (3) number of research laboratories providing sources of data for faculty investigators, and (4) types of research laboratories providing sources of data (primary or secondary). Research leadership: (5) department faculty roles as principal investigators on active grants and contracts, and (6) number of faculty who currently serve on federal research peer-review panels. Funding: (7) proportion of FTEs of research faculty externally funded for salary support; (8) amount of internal/in-kind department research funding; (9) number of external awards (grants, contracts, philanthropy); (10) sources of external awards; and (11) amounts of external awards. Variable definitions are available upon request.

Each chair was also asked to designate one of five stages of research capacity of their department: none, minimal, moderate, significant, or extensive, based on their own assessment (self-assessed capacity). These designations were subsequently collapsed to minimal, moderate, and high capacity for the analyses.

Analyses were completed using SPSS 19.0.0 and SAS 9.3. Categorical variables were compared via the χ2 test. Continuous variables were compared across groups via Student t test or analysis of variance (ANOVA) as appropriate, except where data were not normally distributed. In these cases, a nonparametric Kruskal-Wallis test was employed. Categorical variables were collapsed into fewer values where necessary to have adequate cell counts for statistical analysis, or where data revealed that combining values would be more informative.

A total of 109 of 142 invited departments completed the survey (77% response rate). The majority of responding departments were located on the main campuses of allopathic medical schools (74%) and were public institutions (63%; Table 1). The oldest was founded in 1964 and the youngest in 2013, with a median of 39.0 years (interquartile range or IQR=19.5 years) since the year of founding as of 2015. Data were missing on year of founding for 16 departments (15%). The mean number of total doctoral-level (MD, DO, PhD, EdD, etc) faculty reported per department was 41.0 (SD=45.3), among whom there was a mean of 32.4 FTE (SD=32.4).

Thirty-six (33%) of the department chairs had been serving for less than 3 years, 30 (28%) for 3-10 years, and 32 (29%) for more than 10 years. Thirty (28%) self-identified as researchers for “most or all” of their career before becoming a chair, though 41 (38%) never identified as a researcher and many had less than 10% personal effort (FTE) for research and scholarship prior to becoming a chair (39%; Table 1).

Forty-eight (44%) respondents reported having none or minimal research in their department (minimal capacity); 23 (21%) considered their department research capacity as moderate (moderate capacity), and 38 (34%) considered their departmental research as significant or extensive (high capacity). All of the 38 self-assessed high-capacity departments were on the main campus of allopathic medical schools. There was no difference in research capacity stage between departments in public versus private medical schools. High-capacity departments tended to be slightly older (median=41.5 years, IQR=12.3 years) than moderate-capacity (median=37.0 years, IQR=31.0 years) or minimal-capacity (median=40.0 years, IQR=15.5 years) departments (P=.042).

Trained Research Faculty. High-capacity departments were far more likely to have five or more individual research faculty (87%) and more than four FTE faculty time devoted to research (82%) than either minimal capacity or moderate capacity departments (P<.001; Table 2).

Infrastructure. High-capacity departments were more likely to use more than one primary research laboratory (74%), including practice-based research networks (PBRN), local practices, community-based surveys, or other primary data sources (P=.024). High-capacity departments were also much more likely to use multiple secondary research laboratories (63%), including clinical data networks, enterprise-level data warehouses, large secondary databases, or existing narrative data than departments with either minimal or moderate-capacity departments (P<.001). Almost half of departments indicated that they used a PBRN as a research laboratory (48%), though minimal capacity departments trended toward being more likely to use a PBRN (65% vs 44% for moderate and 42% for high-capacity departments, P=.074).

Research Leadership. Slightly less than half of the departments (45%) reported one or more faculty who had served on a federal peer-review panel for research or research training proposals in 2015. Seventy-nine percent of high-capacity departments had at least one faculty serve on a federal grant review panel in 2015 (P<.001; Table 2). Department faculty served as the principal investigator (PI) on two-thirds of reported sources of funding for all departments in 2015. High-capacity departments were significantly more likely to report having more than two award sources for which the PI was in their department (87%) than minimal or moderate-capacity departments (P<.001; Table 2). The mean award sources with a PI from the department reported for high-capacity departments was 3.7 (SD=2.2) compared to 2.0 (SD=1.2) for moderate-capacity departments and 0.75 (SD=1.0) for minimal-capacity departments (P<.001).

High-capacity departments were far more likely to have more than 50% external funding for salary recovery of research faculty time devoted to research (63%, P<.001). In addition, the majority of high-capacity departments invested more than $100,000 of internal/in-kind department support for research in 2015 (Table 2). Approximately one-third (N=11) of the high-capacity departments provided more than $300,000 of internal/in-kind support.

Of the 109 responding departments, 89 (82%) reported that at least one department faculty member had externally-funded research effort. Among those with funding, the National Institutes of Health (NIH) was the single most common source of funding, followed by other federal and foundation funding (Table 3). Industry accounts for a very small percentage of department funding sources. Of note, most funding sources provided a relatively small amount of support directly to the department (less than $100,000 for the year). The federal government accounted for the great majority of large ($100,000 to $500,000 annual direct support to the department) and very large awards (more than $500,000 annual direct support; Table 4). Including awards for which the principal investigator was outside of the family medicine department, high-capacity departments had a mean of 4.4 external award sources, including an average of two to three large or very large award sources providing annual direct costs to the department. In contrast, minimal-capacity departments had an average just over one award per department, almost all of which were small (Figure 1).

As described above, self-assessed stage of research capacity was strongly associated with every empirical measure of research capacity (see below, Tables 2-4, and Figure 1) including those reflecting research-trained faculty, infrastructure (research laboratories), research leadership, and funding.

In this national survey, roughly one-third of family medicine departments reported empirical measures of research capacity known to be predictive of research productivity and were classified by their chairs as having high capacity for research. These departments have developed a research-trained faculty, utilized numerous research “laboratories” and data sets, devoted more faculty effort and internal funding to research, and garner more funding from more award sources, particularly federal sources, than the remaining two-thirds of departments. This represents progress from prior estimates of research capacity across the discipline of family medicine. Although it is not possible from this data to establish the growth of research capacity precisely over time, there can be no question that family medicine research capacity has developed meaningfully since the recognition as a specialty 50 years ago and the opening of new departments of family medicine that followed.

The need for research training in family medicine has been well described in the literature, and numerous initiatives and programs have been developed to address major gaps that still exist.34-38 Our results are consistent with Bland’s findings that there is a “critical mass” of research faculty and research FTE associated with research capacity. Other findings that support these associations included the presence of researchers within the department with the capacity to function as leaders of research teams in the role of principal investigator, as well as having nationally recognized faculty to serve on federal review panels for research or research training proposals.

Utilizing a wide range of research laboratory types—both primary and secondary—was also associated with departments with high research capacity in this study. Notably, secondary sources of data were less commonly used in lower-capacity departments despite being more readily available and often free or low cost. For departments looking to grow their capacity, the use of secondary data sources should be considered as a lower-cost way to boost productivity. PBRNs are widely utilized by departments of all capacity levels, offer an excellent option for the generation of new knowledge, and are likely among the best settings for asking and answering primary care questions. However, simply having a PBRN as a laboratory does not lead directly to a high-capacity research enterprise.

Compared to other specialties, family medicine receives very little funding from the National Institutes of Health (NIH).25-28,39-42 That said, funding from the NIH is the primary source of financing for family medicine research, though funding from other federal sources such as the Agency for Healthcare Research and Policy (AHRQ), the Patient Centered Outcomes Research Institute (PCORI), the National Science Foundation (NSF), the Centers for Disease Control and Prevention (CDC), and the Centers for Medicare and Medicaid Services (CMS), as well as funding from private foundations, is also crucial. Detrimental to family medicine research has been the dramatic decline in funding from the Health Resources and Services Administration (HRSA), whose Title VII mechanism was previously used to support research, research training, and research faculty development in the past several decades.43

This study provides a snapshot of research capacity via a sample with a robust response rate and delivers more targeted and distinct measures of research capacity in family medicine. One of our primary goals was to compare well-established empirical measures of research capacity with a measure of self-assessed research capacity. We found that the 11 empirical variables were highly and consistently associated with the self-assessed classifications. The self-assessment question can therefore be considered to have sufficient face validity to be used as a single-item tool going forward. However, even the 11 variables used as empirical measures of research capacity do not generate an excessive response burden, as was demonstrated by the 77% response rate. This study provides a beginning for reporting a minimum set of metrics that can be tracked longitudinally through periodic survey of departments of family medicine. For example, the self-assessed measure could be used more frequently, perhaps every 3 years, whereas the empirical measures could be used less frequently, such as every 5 to 10 years. Research capacity generally changes very slowly (either increasing or declining) or not at all for most departments in the span of 3 to 5 years.

A limitation of this study is that not much is known about the nonresponding departments, including departments in osteopathic medical schools, of which very few were ADFM members at the time of the survey. Our sample excludes residency programs13-16 with faculty not employed in academic departments as well as family medicine researchers based in research institutes, research centers, and research agencies outside of departments of family medicine. Therefore, some research capacity in the specialty has likely gone unmeasured in this survey. However, the majority of family medicine research is generated from academic departments of family medicine.19

As a trade-off for creating a survey that was brief and straightforward to answer, and given that this work was being done in tandem with other efforts to measure research publication productivity, we did not report on publications or other scholarly outputs. In addition, our methodology lacks the detail and depth of more intensive studies of productivity.11,17-21,29 There is also an enormous amount of scholarly activity not reflected by grants and high-impact peer-reviewed original research publications. Measurement of the scholarly activity of departments and residency programs broadly conceptualized would be an important complement to the current study.

Many family medicine departments, if not most, rightly prioritize clinical service, residency training and medical student education and devote precious resources to these mission-critical areas. Some departments are sponsored by institutions that either do not aspire to have a research culture or have not been successful in creating one. These departments may not identify externally-funded original research as a core mission, may lack the necessary leadership experience, or simply have not had the resources to feasibly increase their capacity for research. Additionally, trying to make all departments similar given their diverse structures and missions is not a good use of resources, nor does it recognize the strengths and contributions of departments that focus primarily on patient care, education and training, and community service. However, many departments with limited research capacity do aspire to grow their capacity. This study provides more context for these departments and the initiatives designed to facilitate the development of research resources for the discipline. Future evaluation will help track growth going forward and future research should examine the different perspectives, missions, and goals of departments and residency programs within the context of research as well as capacity and productivity for scholarly activity defined more broadly.44

The implications of these findings with respect to strategic approaches to building research capacity are beyond the scope of this study. However, it should be clear that new chairs (or existing chairs) charged with building research capacity would be well advised to negotiate a robust financial package to make the investments necessary to achieve these well established empirical elements.

In summary, the research capacity of family medicine departments in the United States has progressed substantially since their founding in the past 5 decades, though research capacity in family medicine departments is modest among all disciplines. There is a substantial unmet need for sources of funding for training family medicine researchers and supporting their research agendas. This survey was undertaken to establish a baseline from which to guide current efforts such as the Building Research Capacity initiative6 and determine the impact on increasing future research capacity more broadly in departments that have the desire, leadership, and resources to grow.