Personal Manuscript Acceptance Rates: Metrics for Self-assessment in Scholarship

Manuscript rejection is a common barrier to dissemination of scholarly output.^1-4 Despite the awareness that many journals accept far fewer papers than they reject,^5-8 there are no manuscript rejection rate baselines for clinical researchers and rejections are often met with negative emotional responses including discouragement, disillusionment, alienation, and damaged egos by academic clinicians.^1,3,9 These predictable reactions to rejection can contribute to decreased professional satisfaction, abandonment of the manuscript, and may even dissuade the author from future manuscript preparation.^1,3-5,9,10 Further, negative responses to rejection have led promising scholars to abandon careers in academic medicine.¹⁰

As many departments regard the number of articles published by faculty as a metric of success, it is an expectation for faculty to overcome the challenges of rejection. Despite this expectation, the metrics typically used by academic review committees only provide information about eventual success (ie, publications, citations),¹¹ and accordingly, may fail to provide information that could be useful for tailored mentoring and self-assessment.^12,13 For example, a common metric adopted by academic review committees is an author’s h-index, introduced by Jorge Hirsch to give a better estimate of the significance and impact of a scientist’s cumulative research contribution.¹¹ As this index requires time for citations to accumulate, and can only increase over time, it provides no real-time feedback and does little to assess academic production for early-career researchers.^12,13

Because there are no manuscript rejection rate norms for clinicians at different career stages, and individual metrics such as the h-index provide ways of tracking an author’s ultimate success in publication, little help is available for faculty to put their own evolving experience with rejection into context. However, if authors were to calculate their own personal acceptance rates per submission and eventual acceptance rates of their manuscripts, tracking these dynamic rates might provide meaningful context when compared to peers at various career stages. We wished to determine the feasibility and potential utility of calculating these metrics among primary care faculty at an academic health center.

Faculty deal with two types of rejection regarding manuscript submissions: the per-submission decision (ie, the editorial decision after an author submits a particular manuscript on a given attempt), and the eventual acceptance or rejection of a given paper (ie, whether that paper is eventually published somewhere). We propose the following metrics:

• Acceptance per submission (APS)=acceptances/submissions: percent of the time a manuscript is accepted for any given submission to a unique journal.
  • An APS of 1 indicates that every manuscript submitted is accepted for publication by the first journal it was submitted to.
• Acceptance per manuscript (APM)=acceptances/manuscripts: percent of manuscripts submitted that were eventually accepted by a journal.
  • An APM of 1 indicates that every manuscript is eventually accepted for publication.

The purpose of this study was to determine the feasibility of obtaining personal acceptance per submission (APS) and acceptance per manuscript (APM) for individual faculty members and whether they differ by academic title.

We developed a voluntary cross-section survey pilot study to determine the feasibility of obtaining these novel metrics among clinical faculty in two departments (family medicine and pediatrics). We administered the survey in 2017 after the University of Minnesota Institutional Review Board determined that the study met the criteria for exemption from IRB review. Participants included departmental clinical faculty and did not include adjunct faculty or clinical professors.

Research staff created a list of publications for each individual faculty using Manifold, a web-accessible interface that generates reports of scholarly output for faculty in the University of Minnesota Medical School. Research staff downloaded article data from Scopus (www.scopus.com, a large abstract and citation database of peer-reviewed literature), to verify that the faculty were from our institution and to assess the type of article published.

We provided faculty members with an individualized link to an internet-based survey (Qualtrics, Provo, UT) that included demographics (gender identity, academic rank, professional designation, percent clinical time) and a list of their published manuscript titles listed in the Scopus database from the prior 2 years. For each listed article, faculty indicated the number of journals to which each manuscript was submitted prior to acceptance. We also asked participants to list any manuscripts from the last 2 years not indexed in Scopus and provide the number of journals those manuscripts were submitted to. Finally, participants were asked to report manuscripts that had been submitted in the last 2 years but not yet accepted by a journal, and to report the number of submissions for each.

A research staff member not involved in academic advancement decision making in either department (D.F.) linked the survey responses to the publication statistics available in Scopus. Only this research team member had access to the link to identify responders. Prior to data analysis, we reviewed the survey responses for outliers, duplicate records, and inconsistent responses. We included all responses where there were multiple authors within our institution because the goal was to calculate the APS and APM for each faculty member. Responses that included the response “I have no idea” regarding the number of submissions before publication were modified to include the accurate number of submissions when that data was available from the responses of coauthors. If that information was not available from the coauthors’ responses, the manuscript was removed from the calculation of APS and APM. Faculty who indicated “I have no idea” were included in a subanalysis to determine if author position or number of authors on the paper correlated with the author being unaware of manuscript submission decisions.

The data were then provided to authors A.S. and M.P. without direct identifiers for analysis. We excluded book chapters, notes, letters to the editor, editorials, newsletter publications, blog posts and any other similar publications from the analysis. Individuals who had 0% clinical time and those who had not submitted any manuscripts in the 2 years prior to the survey were also excluded as we were specifically focused on the experience of clinic faculty with regards to their manuscript acceptances.

We calculated the APS and APM for each faculty member, summarizing with descriptive statistics, assessing differences in demographics using t tests (continuous and ordinal variables) and Fisher exact tests (categorical variables).

Sixty-eight of 136 eligible faculty (50%) completed the questionnaire, with respondents equally distributed by department. Sixty-three faculty members met inclusion criteria, with demographic breakdowns similar to the makeup of the departments (Table 1).

Table 2 summarizes the experience with rejection and acceptance. Academic clinicians in the sample eventually published 80% of the manuscripts submitted, yet manuscript rejection (1-APS) was common, with 39% of papers rejected per submission attempt. Associate professors had the highest APS (0.71) and APM (0.88). Associate professors also had the highest average number of submissions (11.3) and published manuscripts (9.6) in the 2-year period, followed by professors (9.1 APS; 8.1 APM) and assistant professors (4.9 APS; 3.8 APM). The differences were significant for both the average number of manuscript submissions and number of published manuscripts (P=.001). The APS and APM were similar between clinical psychologists with PhD, physicians with additional graduate training (PhD and masters), and physicians without additional training. Manuscripts that included both physicians and clinical psychologists had similar APS and APM to those that had only physicians or only PhDs.

A total of 402 manuscripts were submitted; 214 (53.2%) were uploaded by study staff prior to survey distribution. Sixty-three (15.7%) published articles and 50 (12.4%) in-press articles were added by survey respondents. Seventy-five (18.7%) manuscripts that were submitted but not accepted by a journal to date were also added by survey respondents. Thirty-five (9%) of the manuscripts did not have a response by the faculty when asked “How many submissions did this manuscript have prior to acceptance?” and were removed from the analysis of APS and APM.

Subanalysis was done on the 13 faculty who answered “I have no idea” for the number of journal submissions on at least one of their manuscripts. In all, this subgroup submitted 93 manuscripts and did not know the number of journals that their manuscripts were submitted to 38% (n=35) of the time. When further asked what position author they were in those manuscripts, 80% (n=28) were a middle author or did not know what author position they held on that manuscript. Having more than five authors on a manuscript was associated with a small, nonsignificant difference in whether the faculty knew the number of journals their manuscript was submitted to (63% vs 59%).

We proposed two novel metrics, the APS and APM, that may help academics track their progress from day one of their publishing careers, rather than having to wait for citations to accumulate. We generated department- and rank-specific averages for manuscript acceptance and rejection, and confirmed that rejection is common, yet surmountable among academic clinicians.

There is growing awareness that the overall health of the United States could benefit from increased primary care research, yet that has been underdeveloped in part due to limited and concentrated federal funding.^14-19 Despite these barriers, the percentage of academic faculty who have published manuscripts has increased, often as the result of publications of research presented at major conferences.^20,21 The primary care research enterprise is positioned to build on that past work and capitalize on emerging opportunities through the contributions and initiatives of national organizations.^22-24 Academic departments are vital to this process and best practices for departments that do this well include faculty who are supported by mentorship and resources.^23,25,26 The manuscript metrics APS and APM provide data previously unavailable and have the potential to enhance discussions with the individual faculty member regarding their evolving experience with manuscript acceptance and rejection. By comparing these metrics to peers at various career stages, departments will be able to provide feedback, mentoring, and appropriate resources to clinical faculty. In contrast to previously used metrics that relied on manuscript publication numbers and citation counts, these dynamic metrics could enhance ongoing faculty development efforts aimed at increasing scholarly productivity and have the added benefit of destigmatizing rejection.

In this pilot, we analyzed two departments within a single academic center, which limits the generalizability of our APS and APM norms. Although we have presented benchmark data in this pilot, broader collection of these metrics would allow for the establishment of norms at various career stages within an academic center or medical specialty. Additionally, further qualitative research is warranted to determine how faculty decide where to submit their manuscripts and what specific actions and behaviors researchers use to overcome manuscript rejection. This is especially important considering that in our sample additional graduate training (PhD, MD/DO plus PhD or masters) did not affect the outcome of a manuscript, but academic rank did.

The retrospective nature of data reporting in this pilot is also a limitation, as evidenced by 13 of the 63 faculty answering “I have no idea” to the number of journals their manuscript was submitted to for at least one of their manuscripts, which included 9% of the manuscripts in our analysis. This gap in the level of awareness was most evident in manuscripts where faculty were not the first or last author. Further, our data includes the responses from half of the faculty who received the survey, and those who did not respond may have had different experiences with their manuscript acceptances. Removing the retrospective data collection is one method that could decrease the response bias inherent in our methods. We are working to create a platform to enable personal tracking of manuscript fates at the time of submission. We have already demonstrated that such prospective reporting is feasible on a web-based platform, as part of work we have done to bring gamification to academic pursuits where participants log their submissions and acceptances in real time.^27,28 Even before institutional infrastructure is in place, however, individuals can begin tracking their manuscript submissions and calculate their personal manuscript acceptance rates as a method of self-assessment.