"Never doubt that a small group of thoughtful, committed citizens can change the world. Indeed, it is the only thing that ever has."

Margaret Mead

Review article

Bibliometric Trends in Open Surgical and Endovascular Cerebrovascular Research


The last decade has witnessed a major expansion in endovascular interventions concurrent with a contraction of open neurovascular surgeries. Whether research efforts have also shifted from open to endovascular neurosurgery is an effect that has not been explored extensively. Understanding the bibliometric trend is important for researchers, funding agencies, and publishing journals. The aim of this review is to explore this potential shift.

We compared the bibliometrics of open cerebrovascular and endovascular research articles published in two neurosurgical journals (Journal of Neurosurgery, Neurosurgery) and two neuroradiological journals (Journal of Neurointerventional Surgery, American Journal of Neuroradiology). Data were collected between September 26, 2021, and October 18, 2021. Articles published in 2011, 2013, 2015, 2017, and 2019 from the journals were screened. Neurovascular articles were classified into open surgical, endovascular, or mixed. Bibliometric parameters were collected via SCOPUS and journals’ websites. A total of 8,018 articles were screened, of which 1,551 were included (16.2% open, 62.2% endovascular, 21.5% mixed). Most articles were related to aneurysms (76%). Open-access status correlated with increased citations (p<0.001) and Altmetric (p<0.001), which measures online activity. Comparing 2011 and 2019, the article distribution (open/endovascular/mixed) has changed significantly (χ2 test, p=0.002), with open articles dropping from 23.6% (68/288) to 12.9% (44/342) and endovascular articles rising from 56.6% (163/288) to 65.8% (225/342). Using the Kruskal-Wallis test, the citation distribution is different across the three groups in 2019 (p<0.001), favoring endovascular articles, but not in the other years. Our study suggests a trend of diminishing open neurovascular research output and increasing endovascular research output, in terms of both the number of articles and the citations. More time for citation accumulation may be required to verify this trend.

Introduction & Background

With advances in minimally invasive endovascular techniques and technologies, the volume of open cerebrovascular surgeries has become increasingly smaller and subspecialized [1]. Initially, for brain aneurysms, this shift was accelerated by trials such as the ISAT (International Subarachnoid Aneurysm Trial) that demonstrated the safety and value of endovascular treatment [2,3]. Data from the national database and large volume centers also showed a marked increase in endovascular treatments for aneurysms relative to open treatments [4,5]. More recently, with emerging evidence of the efficacy of thrombectomy treatments in ischemic stroke, the field of neurovascular therapy has expanded dramatically over the past decade [6-8]. Whether or not research effort has also shifted from open cerebrovascular surgeries toward endovascular interventions is an effect that has not been studied extensively.

This is an interesting and important question. Not only does answering this question help inform researchers of the ongoing trend of research, one (but not the only) metrics in resource allocation for research is also often based on academic productivity, which is important for researchers and funding agencies. In addition, the bibliometric studies provide researchers with objective evidence to assess the impact of their research relative to other publications in a similar domain. Therefore, understanding the trend in research is important for researchers, gatekeepers of research funding, and publishing journals [9].

To better inform academic clinicians and other researchers, in the current study we compare the citation numbers and Altmetric indices (measurement of online activity such as social media) [10] of open surgical and endovascular cerebrovascular research articles published in four prominent journals, of which two are general neurosurgical journals (Journal of Neurosurgery [JNS] and Neurosurgery) and two are neuroradiological journals (Journal of Neurointerventional Surgery [JNIS] and American Journal of Neuroradiology [AJNR]). These four journals were chosen as they are among the most prominent journals in the field with a high volume of published cerebrovascular studies according to the authors’ experience. In addition, they have similar journal impact factors - JNS (5.115), Neurosurgery (4.654), JNIS (5.836), AJNR (3.825) [11] - which makes it a more homogenous comparison.



Data were collected between September 26, 2021, and October 18, 2021. Every article from the four journals published in 2011, 2013, 2015, 2017, and 2019 was screened manually by the authors (J.Y., M.S.J.), mostly using the title and abstract. Full articles were assessed if necessary. Articles with a cerebrovascular theme were included, except those that focused on purely diagnostic methods (no interventional component), trauma (including subdural hematomas), oncology, other head and neck pathologies (e.g., mandibular arteriovenous malformations), microvascular decompression, idiopathic intracranial hypertension, radiotherapy treatment (unless in combination with open and/or endovascular interventions), stroke prevention (e.g., inferior vena cava filter insertion), and peripheral intravenous thrombolysis, and those with no open or endovascular interventions. Editorials, letters, conference abstracts, and guidelines were also excluded. Scoping reviews were also excluded (systematic reviews and meta-analyses are included in the study).

Bibliometric data were collected via the SCOPUS electronic database [12] and websites of individual journals [13-16]. Neurovascular articles were classified into open surgical vascular articles, endovascular articles, or mixed. The following data were also collected: year of publication, whether the article is open access (OA), number of citations, Altmetric score, country of corresponding author’s institute, neurosurgical theme of the article, and type of research. OA status can be subclassified in multiple ways [17], e.g., “green” and “gold,” but in the current study, we adopted a pragmatic classification where articles that can be viewed by the public with no extra cost are classified as OA.

Statistical analysis and graph constructions were performed using SPSS 25.0 for Mac (IBM Corp, Armonk, NY, USA). Year-to-year comparison of distribution of articles (open/endovascular/mixed) was compared using χ2 test. Differences in citation number and Altmetric between groups (open/endovascular/mixed; OA vs non-OA) was compared using the Kruskal-Wallis test due to the non-parametric nature of the data. Correlation of citation count and Altmetric was calculated using the two-tailed Spearman rank correlation coefficient (ρ). Statistical significance was set at p<0.05.


Overall, 1,551 articles were selected out of 8,018. A detailed breakdown is shown in Table 1. The majority of articles are endovascular-related (965/1551; 62.2%). In addition, almost half of the articles are related to aneurysm/subarachnoid hemorrhage (760/1551; 49.0%). The majority of articles in all journals are original clinical articles.

  Neurosurgery JNS AJNR JNIS Total
 No. of articles screened 2472 2255 2201 1090 8018
No. of articles included
  Open 99 (27.4%) 147 (35.3%) 1 (0.4%) 5 (1.0%) 252 (16.2%)
  Endo. 121 (33.5%) 110 (26.4%) 254 (95.8%) 480 (94.3%) 965 (62.2%)
  Mixed 141 (39.1%) 159 (38.2%) 10 (3.8%) 24 (4.7%) 334 (21.5%)
  Total 361 416 265 509 1551
 Average of citations ± SEM (range) 26.5±1.4 (1-192) 23.2±1.6 (0-320) 34.8±2.4 (0-314) 18.2±0.8 (0-146) 24.3±0.7 (0-320)
 Average Altmetric ± SEM (range) 1.7±0.2 (0-42) 4.7±0.9 (0-323) 3.2±0.4 (0-74) 5.8±0.8 (0-299) 4.1±0.4 (0-323)
 Percentage of OA 8.9% 3.6% 14.0% 14.1% 10.1%
Distribution of topics
  Aneurysm/SAH 211 (58.4%) 237 (57%) 157 (59.2%) 155 (30.5%) 760 (49.0%)
  Fistula/AVM 48 (13.3%) 56 (13.5%) 24 (9.1%) 47 (9.2%) 175 (11.3%)
  Cavernoma 18 (5.0%) 20 (4.8%) 0 (0%) 0 (0%) 38 (2.5%)
  Stroke 24 (6.6%) 17 (4.1%) 78 (29.4%) 257 (50.5%) 376 (24.2%)
  Venous thrombosis 3 (0.8%) 7 (1.7%) 0 (0%) 5 (1.0%) 15 (1.0%)
  Carotid artery dissection or stenosis 27 (7.5%) 29 (7.0%) 1 (0.4%) 8 (1.6%) 65 (4.2%)
  Vertebral artery dissection or stenosis 4 (1.1%) 3 (0.7%) 1 (0.4%) 2 (0.4%) 10 (0.6%)
  Moyamoya 19 (5.3%) 30 (7.2%) 0 (0%) 2 (0.4%) 51 (3.3%)
  General open 2 (0.6%) 7 (1.7%) 0 (0%) 0 (0%) 9 (0.6%)
  General endo. 5 (1.4%) 8 (1.9%) 4 (1.5%) 32 (6.3%) 49 (3.2%)
  General mixed 0 (0%) 2 (0.5%) 0 (0%) 1 (0.2%) 3 (0.2%)
Type of article
  Clinical original 296 (82.0%) 320 (76.9%) 223 (84.2%) 377 (74.1%) 1216 (78.4%)
  Case report/technical note 30 (8.3%) 58 (13.9%) 4 (1.5%) 100 (19.6%) 192 (12.4%)
  Basic science 25 (6.9%) 23 (5.5%) 21 (7.9%) 18 (3.5%) 87 (5.6%)
  Systematic review/meta-analysis 10 (2.8%) 15 (3.6%) 17 (6.4%) 14 (2.8%) 56 (3.6%)
 Top three most popular countries (corresponding author) USA (55.1%); Japan (11.4%); Germany (5.5%) USA (46.6%); Japan (13.9%); China (7.2%) USA (27.9%); France (15.5%); Germany (13.2%) USA (59.3%); Germany (7.9%); China (5.3%) USA (49.6%); Japan (8.1%); Germany (7.7%)

Breaking down into the subspecialty groups (open surgical/endovascular/mixed), the citation count (across all years) does not differ significantly across the group (p = 0.444), but the Altmetric appears to be lower in the endovascular and mixed groups compared to the open surgical group (p<0.001) (Table 2). However, considering the progression over the years, comparing 2011 and 2019 alone, the article distribution (open/endovascular/mixed) has changed significantly (χ2 test, p=0.002), with open articles dropping from 23.6% (68/288) to 12.9% (44/342) and endovascular articles rising from 56.6% (163/288) to 65.8% (225/342). A similar trend also applies when considering solely the two neurosurgical journals. Here, the proportion of open articles dropped from 38% (67/176) to 27% (43/161), and the proportion of endovascular articles was approximately constant, changing from 31% (54/176) to 30% (48/161) (χ2 test, p=0.034).

  Open Endovascular Mixed p-Value
Average number of citations ± SEM (range) 23.0±1.7 (0-195) 25.1±1.0 (0-314) 23.1±1.6 (0-320) 0.444
Altmetric ± SEM (range) 4.3±1.3 (0-323) 4.4±0.4 (0-299) 3.1±0.4 (0-52) <0.001
Average number of authors per article ± SEM (range) 6.4±0.2 (1-15) 7.8±0.1 (1-31) 7.3±0.2 (1-25) <0.001

In addition, with all four journals, using the Kruskal-Wallis test, both the citation distribution and Altmetric score are significantly different across the three groups in 2019 (both p<0.001), favoring endovascular articles, but not in the other years (except the Altmetric score in 2017, which is also statistically significant with p=0.001, favoring endovascular articles). These are demonstrated in Figures 1, 2. The citation count fell over time (as citation takes time to accumulate), but the Altmetric score increased from 2011 to 2019.

As shown in Figure 3, there is no statistically significant correlation with the Altmetric scores (ρ=-0.025; p=0.326). OA articles have statistically higher citation counts (p<0.001) and Altmetric scores (p<0.001), as demonstrated in Figure 4.


Our study adds to the current literature by assessing the relative changes in research focus between open surgical and endovascular cerebrovascular research. Our results suggest a trend where open neurovascular research output is diminishing while endovascular research output is increasing, as evidenced by the changes in publication number. Furthermore, the impact of open surgical research has also reduced relative to endovascular research, as demonstrated by the changes in citation counts and Altmetric scores.

This potential decline in the open surgical research output and impact may be explained by the decline in open vascular case volume and the fact that a large proportion of these cases are now concentrated in a few highly specialized centers [18]. This may lead to challenges such as patient recruitment and generalization of results, and therefore such research is harder to be conducted. For these reasons, a multicenter approach is often adopted, which, however, can be very costly. This is reflected in the fact that most studies are published in more wealthy developed countries and those with high patient volume, such as the USA, Germany, Japan, and China [18].

However, it is difficult to ascertain the temporal relationship on how research may reflect surgical volume, as research output could both be a leading and a lagging indicator of treatment paradigm shift. Early rise in research output may signal an upcoming change in the clinical practice, while once that change has been implemented it may also serve to further increase the research output (for example, increasing number of patients and operators may further popularize the treatment modality).

Another possible explanation for the relative decline in open surgical output and impact may be the relative lack of new open techniques or tools over the last decades (unlike endovascular devices), which couples with the evolving costs of endovascular procedures and explosion of commercial investment in endovascular research. Furthermore, the emergence of dual-trained open/endovascular neurosurgeons and shift in practice from radiology to neurosurgery (especially in the United States) may also be part of the reasons, particularly since there is a large demand for publications among academic neurosurgeons. However, further studies are needed to confirm that.


Citation of an article accumulates over time as it is cited by other publications. Hence, articles published closer to the data collection date tends to have lower citation count. However, interestingly, Altmetric, an algorithm that takes into account activity on the Internet, such as Wikipedia, newspaper, and social media [10], is actually higher in the later years. This is likely to be caused by the increasing popularity of smart phones, as well as social media use such as Twitter [19]. Furthermore, the Altmetric score bears no correlation with the citation count. A previous study suggests that the impact factor, which is based on citation count, is strongly correlated with other newer indices such as SCImago Journal indicator, SCOPUS h-index, Google h-index, Eigenfactor, and Article Influence Score (detailed description of these indices can be found in the study by Yuen [20]). None of these capture the Internet activity in the same way Altmetric does, which provides another dimension to how information is obtained by the audience. Therefore, authors and editors should consider evaluating their research impact using this in conjunction with traditional metrics, especially given its increasing popularity and importance.

Is the cost for open access worth it?

In the current study, we demonstrated that when an article is free to view (sometimes costing the authors a fee or sometimes at the discretion of the editors), their citation counts and Altmetric scores are higher. However, the cost of publishing OA can be very high. With the four journals here, the cost to publish an original article can vary between $750 and $4,269 (although there are discounts available for researchers from developing countries) [13-16]. Given the importance to disseminate research, this should incentivize more governmental support to subsidize researchers in opting for this option. Furthermore, having more citations (and hence a higher h-index) is associated with increased success in the National Health of Institute (NIH) grant application, and therefore there are also reasons for the academic institutes to support OA costs [21].


There are certain limitations to this study. In the current study, high profile but more general medical journals such as New England Journal of Medicine and Lancet Neurology were excluded. Although these journals generally have a higher impact factor and important landmark papers are sometimes published there, we do not want them to skew our results. In particular, a small number of recently published endovascular articles (such as the study by Nogueira et al. [22]) in these journals are outliers compared to the majority of the endovascular literature. Inclusion of journals that regularly publish cerebrovascular studies provides a more balanced view. Hence, we decided on the four chosen journals to achieve our objectives since they are all highly regarded neurosurgical/neuroradiological journals with similar impact factors and excellent track record in publication of cerebrovascular studies. Furthermore, the two neuroradiological journals do not publish open cerebrovascular research in general, which is the reason why we assess citation counts and other bibliometric metrics to compare the impact of articles.

The citation counts, particular those in the later years, accumulate over time. Hence, the deduction that endovascular research is overtaking open surgical research probably cannot be confirmed until years later. Nonetheless, the results here are suggestive that this is the case.

We have only sampled the odd number years in this study but given that more than 1,500 articles are included and the general trend is the focus, the risk of significant bias is low on balance. Increasing the number of years in the study would increase the duration of data collection, which can introduce bias as bibliometric data changes over time.


The current study sampled articles from four prominent journals in the field of neurosurgery and neuroradiology across five years (2011 to 2019). It demonstrated a trend where open neurovascular research output is diminishing while endovascular research output is increasing in terms of the number of articles. Furthermore, considering the citation counts, endovascular research has also been cited progressively more. We have also demonstrated OA status does increase the impact in terms of citations and Altmetric scores, providing incentives for researchers to opt for that option. However, more time for citation accumulation may be required to verify this trend.


  1. Lawton MT, Lang MJ: The future of open vascular neurosurgery: perspectives on cavernous malformations, AVMs, and bypasses for complex aneurysms. J Neurosurg. 2019, 130:1409-25. 10.3171/2019.1.JNS182156
  2. Qureshi AI, Vazquez G, Tariq N, Suri MF, Lakshminarayan K, Lanzino G: Impact of International Subarachnoid Aneurysm Trial results on treatment of ruptured intracranial aneurysms in the United States. Clinical article. J Neurosurg. 2011, 114:834-41. 10.3171/2010.6.JNS091486
  3. Molyneux AJ, Kerr RS, Yu LM, Clarke M, Sneade M, Yarnold JA, Sandercock P: International subarachnoid aneurysm trial (ISAT) of neurosurgical clipping versus endovascular coiling in 2143 patients with ruptured intracranial aneurysms: a randomised comparison of effects on survival, dependency, seizures, rebleeding, subgroups, and aneurysm occlusion. Lancet. 2005, 366:809-17. 10.1016/S0140-6736(05)67214-5
  4. Golnari P, Nazari P, Garcia RM, et al.: Volumes, outcomes, and complications after surgical versus endovascular treatment of aneurysms in the United States (1993-2015): continued evolution versus steady-state after more than 2 decades of practice. J Neurosurg. 2020, 134:848-61. 10.3171/2019.12.JNS192755
  5. Spiotta AM, Hui FK, Moskowitz SI: Trends in device-assisted aneurysm treatment at a single high-volume tertiary care center. J Neurointerv Surg. 2013, 5:117-20. 10.1136/neurintsurg-2012-010258
  6. Ciccone A, Valvassori L, Nichelatti M, Sgoifo A, Ponzio M, Sterzi R, Boccardi E: Endovascular treatment for acute ischemic stroke. N Engl J Med. 2013, 368:904-13. 10.1056/NEJMoa1213701
  7. Goyal M, Demchuk AM, Menon BK, et al.: Randomized assessment of rapid endovascular treatment of ischemic stroke. N Engl J Med. 2015, 372:1019-30. 10.1056/NEJMoa1414905
  8. Kidwell CS, Jahan R, Gornbein J, et al.: A trial of imaging selection and endovascular treatment for ischemic stroke. N Engl J Med. 2013, 368:914-23. 10.1056/NEJMoa1212793
  9. Durieux V, Gevenois PA: Bibliometric indicators: quality measurements of scientific publication. Radiology. 2010, 255:342-51. 10.1148/radiol.09090626
  10. Altmetric. (2021). Accessed: September 12, 2021: https://www.altmetric.com/.
  11. Clarivate Analytics: 2020 Journal Impact Factor in Journal Citation Reports. (2021). Accessed: September 12, 2021: https://clarivate.com/.
  12. Scopus. (2021). Accessed: September 12, 2021: https://www.scopus.com/.
  13. Journal of NeuroInterventional Surgery. (2021). Accessed: 9/12/21: https://jnis.bmj.com/.
  14. Journal of Neurosurgery. (2021). Accessed: September 12, 2021: https://thejns.org/.
  15. American Journal of Neuroradiology. (2021). Accessed: September 12, 2021: http://www.ajnr.org/.
  16. Neurosurgery. (2021). Accessed: September 12, 2021: https://academic.oup.com/neurosurgery.
  17. Yuen J, Muquit S, Whitfield PC: Correlation between cost of publication and journal impact. comprehensive cross-sectional study of exclusively open-access surgical journals. J Surg Educ. 2019, 76:107-19. 10.1016/j.jsurg.2018.06.029
  18. Leake CB, Brinjikji W, Kallmes DF, Cloft HJ: Increasing treatment of ruptured cerebral aneurysms at high-volume centers in the United States. J Neurosurg. 2011, 115:1179-83. 10.3171/2011.7.JNS11590
  19. Twitter. (2021). Accessed: November 5, 2021: https://twitter.com/.
  20. Yuen J: Comparison of impact factor, Eigenfactor metrics, and SCImago Journal Rank Indicator and h-index for neurosurgical and spinal surgical journals. World Neurosurg. 2018, 119:e328-37. 10.1016/j.wneu.2018.07.144
  21. Rezek I, McDonald RJ, Kallmes DF: Is the h-index predictive of greater NIH funding success among academic radiologists?. Acad Radiol. 2011, 18:1337-40. 10.1016/j.acra.2011.06.017
  22. Nogueira RG, Jadhav AP, Haussen DC, et al.: Thrombectomy 6 to 24 Hours after Stroke with a Mismatch between Deficit and Infarct. N Engl J Med. 2018, 378:11-21. 10.1056/NEJMoa1706442

Review article

Bibliometric Trends in Open Surgical and Endovascular Cerebrovascular Research

Author Information

Jason Yuen Corresponding Author

Neurological Surgery, Mayo Clinic, Rochester, USA

Mohamed Sobhi Jabal

Radiology, Mayo Clinic, Rochester, USA

Luis E. Savastano

Neurological Surgery, Mayo Clinic, Rochester, USA

David F. Kallmes

Radiology, Mayo Clinic, Rochester, USA

Ethics Statement and Conflict of Interest Disclosures

Conflicts of interest: In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.

Review article

Bibliometric Trends in Open Surgical and Endovascular Cerebrovascular Research

Figures etc.


Scholarly Impact Quotient™ (SIQ™) is our unique post-publication peer review rating process. Learn more here.