Evaluating the Evidence: Scientometric Analysis of Highly Cited Neurofibromatosis 1 Publications

The study of Neurofibromatosis 1 (NF1) is progressing rapidly. This study aimed to identify historical trends in publications focusing on NF1, to find the top 100 most cited publications on this topic, and to evaluate their level of evidence. This study identifies historical trends in publication regarding NF1 with the aim of providing readers useful information about the areas of research being performed, an educational guide to facilitate novice researchers in conducting effective evidence-based medical research, and unique insight into developments and trends of NF 1 research. This study also evaluates the evidence of highly cited papers on NF1. A search of all databases and journals accessible within Elsevier's Scopus was performed on June 27th, 2020, using combinations of the Boolean queries "Neurofibromatosis 1," "Von Recklinghausen," and "NF1," which yielded 13,599 documents. The top 100 most-cited papers were identified, analyzed, and evaluated for level of evidence. Evidence was assessed using the GRADE guidelines. The top 100 most-cited articles span years 1963-2010 and are published in 50 different journals. The average number of citations per publication was 366.5 (range 189-1527). The most cited article is "Neurofibromatosis: Conference Statement" (Stumpf et al., 1988). In this study, the top 100 most-cited works in NF1 are identified, characterized, and analyzed. This study will serve as a historical point of reference for future research, a jumping point for those unfamiliar with the topic, and an educational foundation for future NF1 specialists and researchers.


Introduction And Background
Many medical specialties use bibliometrics to compile, publish, and review the most-cited works within their respective fields [1]. This is especially useful as the exponential rise in publications and resources makes it difficult for learners to process information efficiently. It likewise identifies and assesses the impact of publications, journals, and authors. Furthermore, bibliometrics takes a snapshot in time of objective metrics, which can highlight scientific progression, historical trends, and prolific individuals.
This study aimed to identify historical trends in publications focusing on Neurofibromatosis 1 (NF1), to find the top 100 most cited publications on this topic, and to evaluate their level of evidence. Our literature search revealed this had not previously been done. Reflecting not only on the trends of this research, but also their level of evidence, provides readers useful information about the areas of research being performed, provides an educational guide to facilitate novice researchers in performing effective evidence-based medical research, and provides unique insight into developments and trends of NF1 research.

Review Methods
A search of all databases and journals accessible within Elsevier's Scopus was performed on June 27th, 2020. Document search was performed using the Boolean query "[TITLE-ABS-KEY ( "Neurofibromatosis type 1" ) OR TITLE-ABS-KEY ( "Neurofibromatosis 1" ) OR TITLE-ABS-KEY ( "Von Recklinghausen's disease" ) OR TITLE-ABS-KEY ( "Von Recklinghausen" )]," without limitations on year or article type; and, which yielded 13,599 documents. Documents were ordered by the highest citations and screened for those papers whose primary focus was on NF1 or the NF1 gene, as well as the disease's complications, incidence, management, pathogenesis, treatment, and diagnostic evaluation. The top 100 most-cited papers from Elsevier's database were identified, and data were extracted. Data about these articles were collected and sorted. All references contained within the top 100 articles were obtained, sorted, and counted. Statistical analysis was performed using a combination of R-Studio and Bibliometrix [2]. The spectrographic analysis was performed using CRExplorer [3]. Two separate reviewers evaluated papers for the level of evidence using the GRADE system described by . Any discrepancies in scoring were resolved by a third reviewer and discussion to reach a consensus. Papers that did not provide new evidence such as review papers and conference statements were excluded from this evaluation. Graphs and tables were drawn to illustrate the relationships between factors.

Results
The top 100 most-cited articles for NF1 span between 1963-2010 and are published in 50 journals. The average years from publication was 29.8, and the average number of citations per publication was 366.5, with an average of 12.3 citations per year. The total number of references contained within the top 100 articles was 3,852. Of the top 100 most-cited articles, the total number of citations ranges from 189-1527 ( Table 1). The most cited article is "Neurofibromatosis: Conference Statement" by Stumpf           Overall, there were 470 authors, with 655 total author appearances. Seven of the publications were from single authors, but most publications averaged 4.7 authors per document.
The historical roots of NF1 research were identified using spectrographic analysis according to the method of Marx et al. 2014 [8]. The largest peak occurs in the year 1990, which is indicative of the year when NF1 research took its firmest foothold ( Figure 3).

3,852 references, with references going as far back as the year 1768
Spectrographic analysis reveals quantitatively which historical papers are of particular interest in this specific research topic. This mode of analysis combats "obliteration by incorporation," where novel ideas are "rapidly absorbed into the body of scientific knowledge and their origins thus quickly forgotten due to familiarity," and "palimpsestic syndrome," where an "idea is covered by ascribing it to a more recent author who cites the original work" [8].
The institution affiliated with the most publications in the top 100 was the University of Michigan with 25 publications (Figure 4).   Table 3 lists the authors that the top 100 most-cited articles cited most in their references. These authors all appear as authors of the top 100 most-cited articles. Table 4    The geographic distribution of corresponding authors with NF1 publications was made up of 59% publications from the US, 13% from the UK, 5% from Canada, 3% from Italy, 2% from Australia, Belgium, and Germany each, and 1% from Denmark, France, and Sweden ( Figure 5). The top eight institutions contributing to the top 100 NF1 publications were all from the US, with the University of Michigan, University of California, Howard Hughes Medical Institute University, and Harvard Medical School, each being involved with more than 20 articles (Figure 4).
The distribution of publications by author ( Figure 6) (Lotka's Law) [9] shows that 381 authors published one paper, and 51 authors published two papers within the top 100 most-cited. Another 38 authors published more than three papers, with the most with 13 articles. The historical origins of NF1 research were also traced using a direct citation network (Figure 7) to compare to the spectrographic analysis seen in Figure 3.   Table 5) showed that the level of evidence for the remaining articles was very low for 19.7%, low for 68.4%, moderate for 11.8%, and high for 0% of the articles (Figure 8).

Discussion
Academic inquiries in the modern era are being pursued by a greater number of individual investigators than ever before, at around 7.8 million worldwide in 2013 [10]. A greater number of scientists correlates with a more considerable amount of scientific data. While an increased quantity of academic works leads to an overall increase in significant advancement in scientific understanding for the general public, it is worthwhile to analyze which articles seem to have risen above the rest. Such "foundational studies" are studies that have significantly impacted the course of academic progress in a particular field, resulting in frequent citation [11].
However, the argument can be made that an elevated number of overall citations may not be the best metric for determining an article's influence. Instead, the best method may be to measure an article's citations per year. If the paper continues to be cited over time, this demonstrates a continued impact on the scientific community. Bohl and Ponce 2017 found that when ranking articles based on citations per year as opposed to total citations, the citations that were in the top 100 were more recent publications that focused less on the understanding of the root cause or progression of a disease and more on surgical management and disease outcomes [12].
From our data of articles covering NF1 from 1963-2010, if all citations organize the articles, the premier publication is Neurofibromatosis: Conference Statement at 1527 citations since 1988 [7]. By this ranking, the second-most cited article is Wallace et al. 1990 at 1102 citations [13].  [14].
The Neurofibromatosis Conference Statement by Stumpf et al. in 1988 is referenced extensively throughout Neurofibromatosis literature as it is an early statement that set out to present information for the diagnosis of and management options for NF1, NF2, and NF variants [7]. Von Recklinghausen identified the disease in 1882, and over 100 years later, this panel consensus set forth the guidelines for clinical diagnosis of NF disorders based on physical characteristics [14][15][16][17]. Additionally, it discussed weak points in the current knowledge base, calling for academic research into topics that would serve as a springboard for future publications [7]. Some of the specific topics cited for future research include looking for possible genetic heterogeneity of NF patient families, isolating the NF genes to understand the pathophysiological progression of the disease, and acquiring prognostic data for survival and disease progression. One notable proposition that may have increased the amount this article was referenced is that the consensus panel calls for continued NF tumor DNA analysis. They state that NF2 tumor analysis had already shown potential for increasing understanding of all neoplasia, not simply neoplasia limited to NF [7].
Williams et al. 2009 is a review article that covers the history of NF1 disease presentation and makes recommendations for treatment based upon more recent findings [14]. The reason for its heightened citation count per year likely has to do with how recently the paper was published. A well-written review article summarizing the most relevant recent data about disease progression as well as the progress that has been made since foundational articles first laid forth the groundwork for treatment, is bound to be cited frequently within a certain duration of time. This is especially true immediately after publication, prior to new publications taking its place. Ultimately, guidelines will be updated as academic understanding progresses, and more review articles summarizing modern findings will be produced. As for specific findings of this review article, one notable statement was that there is interest in using small animals to study NF1 therapeutic methodologies. This is something that was briefly suspected as a future avenue for research in the NIH Conference Statement [7]. The article states that in combination with small animal imaging modalities, these models could allow researchers to more easily observe disease progression [14].
Our Lotka's law analysis ( Figure 6) found that the most prolific author within the top 100 most cited articles on NF1 was D.H. Gutmann, producing 13 articles within the top 100. Gutmann's work often focused on defining the clinical basis for the management of NF1 and its individual characteristics. Some of his works include statements on peripheral nerve sheath tumors, optic gliomas, and cardiovascular manifestations of NF1, [18][19][20] genetic pathways integral to NF1 disease progression, [21,22] as well as an article identifying the NF1 gene product, NF1GRP [23]. His most cited work at 1017 citations is "The diagnostic evaluation of multidisciplinary management of neurofibromatosis 1 and neurofibromatosis 2", an article that updated management guidelines based on recent evidence in a similar manner to that of the 1988 NIH Conference Statement before it, and the Williams 2009 review article after it [7,14,24]. Research that takes several bodies of work and synthesizes them into one cohesive statement for the most appropriate management of a disease is a springboard for future works, leading to multiple citations according to our findings.
Most articles in the top 100 are published within Cell ( Figure 2). Notably, the year that had the most articles within the top 100 for NF1 was 1990 ( Figure 1) with ten articles; six of which were from Cell. The cell had 11 articles within the top 100, meaning that these six articles may have had a significant influence not only on Cell's high amount of top 100 NF1 articles but also 1990's abundance of articles. It was in 1990 that the NF1 gene was cloned, and sections of its cDNA product were sequenced, resulting in each of these articles focusing on the NF1 gene and its GAP-related protein product [25][26][27][28][29][30]. These works likely had substantial collaboration, with researchers such as Viskochil, Cawthon, White, and Xu appearing in several publications. The cell is a journal focused on molecular biology, so, understandably, a year with an emphasis on discoveries related to the molecular pathway of NF1 would create a spike in publications of interest in Cell specifically. Besides an NF1 and NF2 update by Mulvihill et al., [31] the other publications in 1990 outside of Cell discussed the pathophysiology of malignant NF1 tumors [32,33] and added more information about the NF1 gene [13]. Of Gutmann's 13 publications in the top 100, only his work on Cui et al. in 2008 was published in Cell [22].
Despite being highly cited, the majority of the articles found within this study held the GRADE score low. None of the studies in the top 100 most cited held the GRADE high. In the case of Neurofibromatosis 1, citation number is not a good surrogate for a quality paper according to the GRADE system [4][5][6]. One possible reason for this is the number of basic science articles (36) found within the top 100, which often fail to randomize and blind their studies, as well as a lack of randomized, controlled, and double-or tripleblinded studies.
Our analysis was not without limitations. Total citations and citations per year are not fool-proof methods for calculating publication impact. While total citations can often be skewed due to a more extended period of circulation, the inverse can be said with how our top article based on citations per year was still within its first and most relevant year. H-indices are not a perfect stand-in either, as the metric has potentially skewed results that can ignore the works of researchers who published several moderately successful publications or even a handful of outstanding articles [34]. Self-citations can additionally alter apparent citation counts when authors working together on multiple projects reciprocate references. While collaboration on top NF1 publications was noted, numerous studies of the effect of self-citation find that said citations usually account for less than 10% of citations and do not influence outcomes [35][36][37].

Conclusions
In this study, we identified, characterized, and analyzed the top 100 most-cited works in NF1. This will serve as a historical point of reference for future research, a jumping point for those not familiar with the topic, and an educational foundation for future NF1 specialists and researchers. Citation count did not correlate with the quality of evidence. We suggest that this study be replicated every five years to assess the progress of NF1 research and to identify historical trends.

Conflicts of interest:
In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.