The Scholarly Output of Faculty in Family Medicine Departments

Research is critical to family medicine’s legitimacy, evolution, and future.1 Today, this research enterprise spans the spectrum of health care delivery, from wellness and disease prevention to management of multiple chronic conditions and social determinants of health. The knowledge generated affects change at the individual, population, community, and health system levels. In addition to clinical research, family medicine investigators address key problems in myriad fields including health policy, medical education, health informatics, health services research, dissemination and implementation, and payment reform.2-5 Over time, family medicine research has expanded beyond academic walls, broadening its impact. Practice-based research networks (PBRNs) reach 15% of the US population, and numerous family medicine researchers engage communities through participatory research.6

Previous attempts to measure family medicine research output have alluded to the challenge of capturing its scope and highlighted others.7-9,4,10 For example, family medicine scholars often collaborate with experts in other fields (eg, psychology, social work) and publish in nonclinical fields (eg, information technology) while experts in other disciplines and specialties also publish in leading family medicine journals.4,8 Prior work has used a wide range of inclusion criteria, databases, and methods to capture research productivity in family medicine.7-9,4,10 Though heterogeneous, most have pointed toward a general increase in family medicine scholarly production over time and noted that the field lacks a well-accepted methodology for identifying its works. One prior assessment found that most family medicine articles were published without external funding.10 Within this context, we sought to quantify the publications by faculty in departments of family medicine in 2015 based on a bibliometric analysis, identify the journals in which they were published, determine the percentage that were original research articles, quantify the impact of these publications based on citations in 2017, identify the funders supporting these publications, and identify factors associated with publications, citations, and funding.

In this descriptive, bibliometric analysis, we identified faculty within departments of family medicine, then determined the scholarly publications they produced in 2015.

We used department websites to obtain the names, academic degrees, and titles of faculty. To identify scholarly publications and citations, we used Web of Science (WOS), a subscription-based database which covers biomedical, social science, arts, and humanities journals. Its science collection contains 33,000 journals and is broader than that of PubMed.11,12 In addition to journal articles, WOS also covers conference proceedings and books. To identify US departments of family medicine, we used a database from the Association of Departments of Family Medicine (ADFM), that includes accredited allopathic and osteopathic medical schools with family medicine departments. To determine additional characteristics of family medicine departments, we used data from the Association of American Medical Colleges (AAMC) website.13

We assessed the accuracy of these data sources in a related, yet-to-be-published study conducted in partnership with the ADFM. In that study, we asked chairs from 13 departments of family medicine to list the faculty in their department who met criteria outlined below. Of the unique names identified through websites and reported by departments, website searches identified 87%.14 Chairs also reported publications from their faculty in 2015, against which we compared the WOS-based method presented here and another method using the PubMed author affiliation field. Across all publications identified from these three methods, the WOS approach identified 58.6%.14

In the spring of 2017, we used Google internet searches to identify webpages for each included department. Within each department webpage, we recorded faculty member names, academic degrees, and titles. Because we were interested in the output of faculty responsible for scholarship within the department, we limited the academic appointments to and categorized appointments as assistant professor, associate professor, professor, clinical assistant professor, clinical associate professor, clinical professor, and chair, or their respective equivalent ranks. Therefore, we excluded those with titles such as adjunct faculty, clinician educator, and instructor. We included all divisions within departments, including sports medicine, geriatrics, population health, occupational medicine, and public health.

WOS categorizes publications into 43 groups, from which we developed two outcomes: all publications and original research articles only. For “all publications,” we included articles, books, book chapters, book reviews, early access articles, editorial material, fiction, film reviews, items about individuals, letters, poetry, proceedings papers (including conference proceedings), and reviews. The second outcome, “original research articles only,” was defined by WOS as reports of research on original work. For both outcomes, we excluded the remaining WOS categories, which included art exhibit reviews, dance performance reviews, corrections, and reprints. We identified the top 10 family medicine journals by summing the number of times faculty published in each journal and identifying the journals with the 10 highest sums. Additionally, we calculated the number of times faculty published in four of the highest impact journals (Journal of the American Medical Association, British Medical Journal, New England Journal of Medicine, and Lancet). To assess productivity, we calculated the number of times each faculty published and unique publications for the specialty as a whole.

We matched faculty names and institutions with authors and their affiliations for publications in WOS that were published between January 1, 2015 and December 31, 2015. In addition to name, the WOS fields we used for matching included author country, state, city, department name, and institution. We included all of the publications, which matched name, department, and institution. For those publications that only matched institution (eg, did not include family medicine), state, or country, we reviewed the publications and names individually, excluding partial matches where the faculty could not be directly linked to the author affiliation institution. For example, we excluded publications that matched names if the author affiliation institution was the Cleveland Clinic but the faculty’s appointment was with Case Western Reserve University.

To identify factors associated with our outcomes, we assessed individual faculty and department characteristics. We grouped departments into four geographic regions (Midwest, Northeast, South, and West), excluding the medical schools in Puerto Rico. We used information from the AAMC and American Association of Colleges of Osteopathic Medicine to determine whether the institutions were public or private.13,15 We also grouped departments in size quartiles based on the number of faculty. We used four categories for academic degrees: (1) MD or DO, (2) PhD, (3) combined MD or DO and PhD, and (4) other (eg, pharmacists, dietitians, and social workers).

We assessed funding by using the information provided within the WOS metadata and grouped sources into four categories (university, federal, other, and none). We considered a publication to be funded by university sources if the authors reported internal funding from a university. Federal funding included funding from US agencies, including the National Institutes of Health, Veterans Administration, Agency for Healthcare Research and Quality, and Patient-Centered Outcomes Research Institute. Sources that could not be placed in the first two categories were placed in the other group, which included foundations, state agencies, pharmaceutical companies, and international funders. The remaining publications were listed as having no external funding. Papers could receive funding from multiple sources.

We conducted univariate and bivariate analyses to capture publications (all publications and original research articles only), journals, citations, and funding. In bivariate analyses, we assessed these outcomes for region, department size, public/private status, faculty academic degrees, and faculty titles.

The American Academy of Family Physicians Institutional Review Board approved this protocol.

We included 134 schools and 6,736 faculty in our analysis (Table 1). Nearly 81% of these were MDs or DOs only while 10% were PhDs only. Thirty-six percent were assistant professors, while this figure was 14% and 13% for associate professors and professors, respectively.

With respect to the number of times faculty published in specific journals, half of the top 10 journals are family medicine-centric (Table 2). The remainder are related to academic medicine, public health, primary care, and occupational and environmental medicine. Overall, family medicine faculty published 24% of the time in these top 10 journals and 16% of the time in the five family medicine-centric journals. In total, faculty published in 856 unique journals, or in 2.6% of the journals available through WOS.

Using all publications as the outcome, family medicine faculty published over 3,000 times in 2015 (Table 3). These correspond to 2,313 unique publications due to multiple family medicine authors contributing to the same publications. Among those publishing, the mean number of times that faculty published was 2.9. The mean number of publications was higher for faculty in departments in the West, in the third quartile for size, with professor titles, and with combined MD or DO and PhD degrees. Assistant professors, associate professors, professors, and chairs accounted for 93% of all of the times that faculty published. Thus, despite comprising 35% of all faculty, faculty with clinical appointments (clinical assistant, clinical associate, and clinical professors) accounted for only 7% of all the times faculty published. Fifteen percent of all faculty had any publications. Of all of the times that faculty published, 83% were original research articles.

By 2017, faculty were cited 13,548 times, or a mean of 4.5 times cited per faculty (Table 4). These correspond to 10,140 unique citations, or a mean of 5.9 citations per unique article. Over 90% of publications had at least one citation. Professors had the highest mean citations per faculty and accounted for 55% of all citations. In contrast, clinical professors had the lowest mean citations. Of all the times that faculty published, two-thirds reported external funding (Table 5), and over half reported federal funding. The percentage of faculty reporting federal funding was highest for those in departments in the West, in the largest quartile for department size, private institutions, professors, and PhDs only.

In 2015, family medicine faculty published over 3,000 times and were cited over 13,500 times 2 years after publication. These correspond to 2,313 unique publications and 10,140 unique citations. Using different methods and including articles that systematically gathered and analyzed data, Pathman et al identified 790 articles published by family medicine scholars in 2003, and at the time, heralded the contribution as “larger and more productive than generally recognized.”4 While a portion of this growth may be attributable to an increase in the number of journals, the annualized growth rate in family medicine production (9.4%) still outpaces the annualized growth rate in records within PubMed (5.6%).16 In this study, two-thirds of faculty publications reported federal funding, in contrast to a 1992 study demonstrating that the majority were completed without external funding. The number of faculty we calculated (6,738) is comparable to a previous estimate calculated by Weaver (8,433).17 Weaver’s figure is likely higher due to the inclusion of community-based family medicine residency faculty who do not have formal academic titles such as the ones we used as inclusion criteria.

Our results provide insights into the scope of family medicine research. Family medicine faculty published in 856 unique journals, the vast majority (84%) of which were non-family medicine journals. In his declaration of general practice as an academic discipline, Ian McWhinney foresaw the opportunity for learning in the “twilight zones” where disciplines meet and viewed generalism as the intersection of medicine, psychology, and sociology.1 By spanning domains and journals, these results support McWhinney’s vision of the generalist research enterprise.

Several factors could account for the increase in publications over time. To reduce widespread gaps in quality, stakeholders implemented strategies to systematically improve care.18 Practices, particularly those applying for patient centered medical home certification, embraced quality improvement principles.19 On the education side, specialty boards transitioned to maintenance of certification and encouraged continuous learning.20 Academic health centers sought to integrate delivery and research silos by applying principles of learning health care systems.21 Research and quality improvement were built into residency milestones.22 The Accreditation Council for Graduate Medical Education mandated that faculty must engage in scholarship that includes publication and presentations.23 All of these movements pushed faculty toward evaluating whether their care improves outcomes and may have contributed to the growth in publications. Simultaneously, the available outlets for these manuscripts has grown. With the internet reducing barriers to entry, the number of journals has steadily increased over time.16

These data may prove useful for guiding promotion decisions and providing benchmarks. For example, they can be updated over time to quantify the return on investments in research infrastructure. One such investment is Family Medicine for America’s Health (FMAHealth), a national initiative to integrate family medicine organizations to chart the future of the discipline.24 In addition to launching this effort, the FMAHealth Research Tactic Team sought to understand the state of family medicine research and stimulate conversations about how to bolster its research enterprise. Using the data presented in this work, we hope to identify and disseminate lessons from highly productive departments so that all departments can learn from effective strategies. Further, our data highlight opportunities for improvement as only 15% of faculty published anything. As clinical faculty appointments expand, these trends will need to be monitored.25 Nonpublishing faculty likely have research questions and insights that need to be shared with the broader academic community. Understanding the characteristics of nonpublishing faculty and their barriers to scholarship will be critical to increasing participation. It remains to be seen whether the aforementioned initiatives to enhance scholarship will lead to broader participation. Finally, these figures should be reproduced and tracked at regular intervals. For educators and workforce planners, the annual publication that updates the number of available and filled family medicine residency slots provides important guidance, allowing stakeholders to track and respond to trends.26 A similar effort for family medicine faculty, publications, citations, and funding could prove critical to building on the successes documented here.

There are numerous limitations that should be considered when assessing our results. First, given our desire to develop a reproducible method, we only captured scholarly publications produced by a subset of faculty. However, we contend that these were the faculty that were most likely to publish. Second, we limited our publications to those listed in WOS. Therefore, we missed articles that were in nonindexed journals and scholarly publications not included in WOS. These presentations, curricula, and articles have tremendous value to the discipline and are not captured by this method. Third, the accuracy of faculty lists on websites is limited by the frequency with which those websites are updated. We assessed faculty names and appointments in 2017 while the actual roster of faculty may have differed in 2015. Fourth, since we lack a unique identifier that spans faculty rosters and WOS publications, we matched a subset of publications manually. As noted in the methods, we used names and the author affiliation institutions to match authors. Imprecision in names (eg, the use of a nickname when publishing) and affiliations (eg, identifying only a hospital rather than an institution) limited the certainty of some matches. Unfortunately, it is possible that we missed publications of faculty who listed nonuniversity affiliations and that we included publications of faculty not in departments of family medicine. While an approach that combined multiple methods and allowed faculty to add, modify, or remove publications from this list would be an improvement, this approach would be more resource intensive and could introduce bias if only a subset of faculty participated in verification. Fifth, we did not attempt to evaluate the quality of publications by assessing the study designs or the importance of the findings. Finally, we omitted a wide range of family medicine scholars not primarily affiliated with departments of family medicine, including those working at the American Academy of Family Physicians, American Board of Family Medicine, and health systems like Kaiser and the Veterans Administration. Taken together, these limitations strongly suggest that our figures are conservative. As previously mentioned, in a companion ADFM study, we found that this WOS approach discovered only 58.6% of the publications identified across three methods. Extrapolating from this figure, it is possible that the method we used missed up to 2,121 additional times that faculty published. Faculty may have published as many as 5,123 times.

In conclusion, family medicine faculty published over 3,000 times and were cited over 13,500 times. Eighty-four percent of these works were published in non-family medicine journals, underscoring the broad scope of family medicine research, and over half of the faculty publications were linked to federal funding. These figures are higher than prior estimates, are most likely underestimates of the scholarly output, and should be tracked over time.