The Hirsch Index (h-index) and m-index are often utilized to assess academic productivity and have been widely found to have a positive association with academic promotion and grant selection. The aim of this study was to assess the relationship between these indices and academic ranks among Canadian orthopaedic surgery faculty members.
Five hundred and sixty-seven Canadian orthopaedic surgery faculty members associated with residency training programs were included in the study. H-indices of individual faculty members were obtained through Elsevier’s Scopus database. Faculty members’ year of residency graduation was recorded from their respective licensing body database and was utilized as a surrogate for the start of their academic career to determine career duration and calculate the m-index. Faculty members were divided based on their academic rank (assistant, associate and full professors) and subspecialty.
Increased h-index, m-index and long career duration were associated with increased academic rank, while gender did not demonstrate an association. Overall, males had a significantly higher h-index compared to females, but no significant difference was observed when comparing the m-index between genders. The m-index varied between subspecialties among senior faculty, but not among junior-ranked faculty.
Bibliometric academic productivity using h-index and m-index is associated with academic ranking among Canadian orthopaedic surgeons at training institutions. Although these indices may provide insight into the academic merits of faculty members, caution must be taken about utilizing it indiscriminately and their limitations must be strongly considered.
Assessing the quality of academic performance has always been an important consideration for funding agencies, grant allocation and committees evaluating faculty recruitment and promotion [1,2]. The utilization of bibliometric indices to assess academic productivity was first popularized by Eugene Garfield in 1955 with the introduction of the Science Citation Index . This innovation subsequently led to his creation of the “Impact Factor”, which was utilized to assess both author and journal impact . For over four decades, an individual’s academic contribution was classified based on their cumulative citation count, total publication count and/or a journal’s impact factor. Due to the variability in the utilization of these numbers by different institutions, there was a tremendous need for the development of a standardized and objective tool to measure academic performance.
To combat this void, Jorge Hirsch introduced his novel bibliometric index, the h-index . He defined it as follows: “A scientist has index h if h of his or her Np papers have at least h citations each and the other (Np- h) papers have ≤h citations” . For example, if a scientist has an h-index of 9, this means that they have published nine papers where each has at least nine citations. Hirsch further described the m-index, which takes into account an individual’s academic career duration in relation to their h-index. The m-index is derived by dividing the h-index by career duration. This, therefore, permitted the comparison of scientists within the same field but at different stages of their academic careers [5,6].
Since its conception, the h-index has been widely calculated and popularized by databases such as Thomson Reuters’ Web of Science, Google Scholar and Elsevier’s Scopus [1,7,8]. It has also been studied in various medical specialties such as anesthesiology , general surgery , neurosurgery [10,11], ophthalmology , otolaryngology , radiation oncology , radiology  and urology . These studies have largely demonstrated a positive association between the h-index and academic promotions. Furthermore, it has been shown that the h- and m-indices increase with academic rank among faculty members at academic orthopaedic surgery institutions in the United States [6,17].
Our study aimed to evaluate the academic productivity of clinical and academic orthopaedic surgeons at training programs across Canada through the h-index and m-index. Study participants were further evaluated based on their academic rank and respective subspecialties.
Materials & Methods
To obtain the list of all orthopaedic surgery faculty members, the Canadian Resident Matching Service (CaRMS) was utilized to outline the 17 orthopaedic surgery training programs across Canada. Each program’s website provided a list of faculty members and their clinical or academic ranking. Programs’ websites were accessed between February and March 2020. Provincial licensing websites, such as that of the College of Physicians and Surgeons of Ontario (CPSO), were utilized to obtain the year of graduation of each faculty member from their respective training programs. This was used as a surrogate for the start of their respective academic career. This value was subtracted from the current year (2020) to determine their academic career duration in years.
Elsevier’s Scopus database (www.scopus.com) was accessed through our institution’s library subscription. A custom search utilizing the name search field was completed and the h-index for each faculty member was recorded. The overall cohort of orthopaedic surgeons was subdivided into clinical and academic surgeons based on their department/university website listing. Those with the wording of adjunct, affiliate, clinical and lecturer were also included within the clinical category. Faculty members hired under academic ranks were divided based on their ranks. Junior academic faculty members included assistant professors, while senior academic faculty members included associate and full professors. This complemented the ranking methodology used in a similar analysis of orthopaedic surgeons in the United States . Furthermore, academic faculty members’ subspecialties were recorded. Exclusion criteria included faculty members without physician degrees, those retired from orthopaedic surgery practice and those with an expired or inactive license. One university was excluded as an updated list of their faculty was not available.
All analyses were completed using IBM SPSS Statistics software (IBM, Armonk, NY). The median and interquartile range (IQR) were calculated for the h-index, career duration and m-index of all faculty members. These metrics were compared between faculty type (academic vs. clinical), academic rank (junior faculty vs. senior faculty), gender (female vs. male) and subspecialty (divided into arthroplasty, foot and ankle, oncology, pediatrics, spine, sports, trauma and upper extremity). Discrete variables were compared using the chi-square test. Non-normally distributed data were compared using the Wilcoxon-Mann-Whitney test for two-group analysis and the Kruskal-Wallis test for analysis of multiple groups. Multivariate regression analysis was completed using nominal logistic regression to assess h-index, career duration and m-index. A p-value of <0.05 was considered statistically significant.
Faculty rank and distribution
A total of 567 Canadian orthopaedic surgery faculty members were included in the analysis. Of these, 14.5% were clinical faculty (n = 82) and 85.5% were academic faculty (n = 485). Among academic faculty, 58% were assistant professors (n = 280) and were designated as junior faculty. The remaining 42% (n = 205) included 132 associate professors and 73 full professors and were classified as senior faculty.
Clinical and academic faculty
Academic faculty had significantly higher publication productivity metrics when compared with clinical faculty. The median h-index was 8 (IQR: 3 - 16.5) for academic faculty compared to a median of 2 (IQR: 1 - 4) for clinical faculty (p: <0.001). The median career duration was four years longer for academic faculty compared with clinical faculty, with 19 years (IQR: 12 - 28 years) for academic faculty compared with 15 years (IQR: 9-24 years) for clinical faculty (p = 0.031). The median m-indices were 0.50 (IQR: 0.19 - 0.92) and 0.17 (IQR: 0.03 - 0.40) for academic faculty and clinical faculty respectively (p: <0.001).
Academic orthopaedic surgeons
The median h-indices were 4 (IQR: 2 - 9) for assistant professors, 12 (IQR: 6 - 18) for associate professors, and 28 (IQR: 19 - 40) for full professors (p: <0.001). The median career duration was 10 years (IQR: 8 - 21 years) for assistant professors, 23 years (IQR: 17 - 30 years) for associate professors, and 30 years (IQR: 24 - 36 years) for full professors (p: <0.001). The median m-indices were 0.33 (IQR: 0.10 - 0.73) for assistant professors, 0.60 (IQR: 0.25 - 0.89) for associate professors and 0.96 (IQR: 0.53 - 1.52) for full professors (p: <0.001). Productivity metrics for juniors compared with senior academic faculty can be found in Table 1. A higher h-index, m-index and longer career duration demonstrated an independent association with senior academic rank. Meanwhile, gender did not demonstrate such an association (Table 2).
Differences between subspecialties
Of the 485 academic orthopaedic surgeons, 455 had specified subspecialties listed. Of these subspecialists, 53 (11.6%) were female. Among these subspecialists, arthroplasty was the most common subspecialty among males (24.4%) while pediatrics was the most common among females (34%). Meanwhile, oncology represented the least common specialty among both females (1.9%) and males (3.7%). The subspecialty with the largest proportion of females was pediatrics (31.6%), while spine had the least (3.3%). Between all subspecialists, there was a statistically significant difference in the h-index (p = 0.004) and m-index (p = 0.001), but not in career duration (Table 3). Upon analysis of junior-ranked subspecialists, there was a statistically significant difference in h-index (p = 0.048) but no difference within career duration (p = 0.840) or m-index (p = 0.125) among the various subspecialties. However, there were statistically significant differences in h-index (p = 0.013) and m-index (p = 0.011) between subspecialties among senior faculty members.
Gender and academic rank
Gender data were analyzed for 567 surgeons, of whom 71 (12.5%) were female. Fifty-five females were academic faculty (77.5% of all female faculty) compared with 430 males (86.7% of all male faculty) (p = 0.039). Among academic faculty, the proportion of females in senior positions was 30.9% (n = 17) compared to 43.7% (n = 188) of males in these positions (p = 0.07). Females represented 13.6% of the assistant professors, 11.4% of the associate professors, and 2.4% of the full professors. Female academic surgeons had a significantly lower median h-index of 5 compared with male academic surgeons (median h-index of 8) (p: <0.001); however, there were no significant differences in the m-index between males and females overall (p = 0.052), male and female junior faculty (p = 0.394) or male and female senior faculty (p = 0.191) (Table 1).
In the last decade, the h-index has become increasingly recognized as a powerful metric of an individual’s academic productivity [18,19]. Prior to its inception, the total number of publications and/or citations was utilized to assess a faculty member’s productivity within their respective field. As a single numeric measure, the h-index provides a more complete overview of the authors’ research impact, and institutions have subsequently adopted it for determining superiority among applicants for awards, grants, promotions and for hiring purposes [2,10,20]. This study assessed academic productivity using this metric and its variants among Canadian orthopaedic surgeons. Increased h-index, m-index and career duration were associated with increased academic rank, while gender did not demonstrate an association.
Similar to the findings of Ence et al. in the United States, we found that the h-index differs significantly among clinical and academic orthopaedic surgery faculty at Canadian training programs. Canadian academic faculty had slightly higher h-index median (8, IQR: 3 - 16.5) and m-index median (0.50, IQR: 0.20 - 0.94) compared to their colleagues in the United States (5, IQR: 1 - 12 and 0.37, IQR: 0.13 - 0.60 respectively) . This study also demonstrated a significant correlation between faculty rank and their respective h-index, career duration and m-index, which is consistent with findings in various other medical and surgical specialties [6,10-13,16]. This correlation suggests that these academic achievement metrics have had a role in the advancement of academic ranks among Canadian orthopaedic surgeons. Furthermore, among academic orthopaedic surgeons, gender was not an independent predictor for their academic rank.
To our knowledge, this is the first study to analyze the academic productivity of various orthopaedic subspecialties. Statistical significance among the subspecialties was noted in junior and senior ranks. Pediatrics and foot and ankle tended to have lower h-indices and this is in line with the findings of the previous studies that subspecialties with lower industry support tended to have lower academic productivity [21,22]. Forrester et al. suggested that the median industry compensation of male surgeons was higher than that of their female colleagues . Their analysis also exhibited disparity among orthopaedic subspecialties. Lieber et al. further demonstrated an association between increased industry consulting payments and academic productivity among orthopaedic surgeons in the United States .
Gender disparities in medical and surgical specialties have been extensively studied over the past few decades. These studies have demonstrated an underrepresentation of females across a number of medical and surgical specialties, including orthopaedic surgery . Although Martinez et al. demonstrated no differences in h-index between female and male musculoskeletal oncology surgeons, our study demonstrated higher h-indices for male orthopaedic surgeons compared to their female colleagues. This difference, along with the shorter career duration among females, is also consistent with the demographics of orthopaedic surgeons in the United States . Nonetheless, it is important to note that this h-index disparity is not evident when examining the m-index within the same cohort. It has been previously hypothesized that female academic productivity based on the h-index is lower compared to their male colleagues at the early stages of their careers, due to the period coinciding with child-bearing years, and it subsequently balances out in senior years . Our study suggests that this disparity that is balanced by the m-index may in fact be due to an age disparity as the m-index takes into account their academic duration. The Canadian Medical Association’s profile of Canadian orthopaedic surgeons in 2018 determined that 62.6% of females were under the age of 45 while only 32.3% of their male colleagues were within that age group . This likely accounts for the disparity that is only seen when comparing h-indices and not m-indices. The significant career duration difference may also be attributed to a younger female cohort, rather than an overall shorter career, as a number of studies have demonstrated an increased representation of females in orthopaedic surgery in recent years [23,26].
Despite the wide utilization of the h-index as a marker for academic productivity, its limitations should be thoroughly considered. Since its inception, Hirsch has discouraged its use in comparing individuals of different specialties as the number of readers and researchers varies between specialties . Also, Svider et al. demonstrated differences in h-indices among academic surgical specialties . This phenomenon could account for the variance noted in our study among orthopaedic surgery subspecialties. Although they are all within the same “specialty”, certain subspecialties may include readership among other disciplines. For example, a medical oncology journal would be more likely to host orthopaedic oncology publications in comparison to foot and ankle studies. This should also caution surgical departments about utilizing generalized h-index cut-offs for promotion or other evaluative purposes between all surgical specialties or subspecialties. Also, the h-index does not compensate for confounders such as the “Matthew effect” whereby more notable researchers may be cited at an increased rate in comparison to their less established colleagues . It has also been suggested that the practice of “gratuitous authorship” can confound calculated h-indices as each co-author receives equal credit when utilizing this metric. Lee et al. disputed this notion as their study of academic neurosurgical faculty did not yield a change in h-index at the individual level when assessing the position of authorship . Lastly, the muddling of self-citation has also been a criticism of the h-index . Engqvist et al. examined this issue and found a minimal effect on the h-index after removing self-citations from the authors studied .
Aside from the known limitations of the indices utilized, this study also has a number of other limitations. The utilization of the database, Elsevier’s Scopus, to determine an individual’s h-index has the potential to credit a publication to the wrong author and we would not be able to correct that. Nonetheless, studies have demonstrated no significant difference in calculated h-index when comparing Google Scholar and Elsevier’s Scopus and therefore its use as the only source was justified . As for other limitations, the analysis of this database may not take into account faculty who have changed their names in the midst of their careers. This may potentially disadvantage female faculty who change their name in relation to marital status. Nonetheless, a study assessing the academic productivity of radiation oncologists noted that 12.4% of their female faculty had published using more than one name during their academic career and this had affected the h-index in approximately 8.2% of the cases .
Overall, the difference in h-indices between male and females, and junior and senior faculty among Canadian orthopaedic surgeons was similar to their colleagues in the United States. After adjusting for career duration by utilizing the m-index, the data for both countries did not demonstrate differences in terms of gender. Although these indices correlate with academic advancement, caution should be taken in their utilization for the establishment of advancement criteria as they are prone to bias, confounding and do not necessarily represent the full academic potential of an individual.
- Bornmann L, Daniel HD: The state of h index research. Is the h index the ideal way to measure research performance?. EMBO Rep. 2009, 10:2-6. 10.1038/embor.2008.233
- Atasoylu AA, Wright SM, Beasley BW, et al.: Promotion criteria for clinician-educators. J Gen Intern Med. 2003, 18:711-716. 10.1046/j.1525-1497.2003.10425.x
- Garfield E: Citation indexes for science; a new dimension in documentation through association of ideas. Science. 1955, 122:108-111. 10.1126/science.122.3159.108
- Garfield E: The history and meaning of the journal impact factor. JAMA. 2006, 295:90-93. 10.1001/jama.295.1.90
- Hirsch JE: An index to quantify an individual's scientific research output. Proc Natl Acad Sci U S A. 2005, 102:16569-16572. 10.1073/pnas.0507655102
- Bastian S, Ippolito JA, Lopez SA, Eloy JA, Beebe KS: The use of the h-Index in academic orthopaedic surgery. J Bone Joint Surg Am. 2017, 99:e14. 10.2106/JBJS.15.01354
- Sharma B, Boet S, Grantcharov T, Shin E, Barrowman NJ, Bould MD: The h-index outperforms other bibliometrics in the assessment of research performance in general surgery: a province-wide study. Surgery. 2013, 153:493-501. 10.1016/j.surg.2012.09.006
- Bakkalbasi N, Bauer K, Glover J, Wang L: Three options for citation tracking: Google Scholar, Scopus and Web of Science. Biomed Digit Libr. 2006, 3:7. 10.1186/1742-5581-3-7
- Bunting AC, Alavifard S, Walker B, Miller DR, Ramsay T, Boet S: Research productivity and rankings of anesthesiology departments in Canada and the United States: the relationship between the h-index and other common metrics [RETRACTED] (Epub ahead of print). Anesth Analg. 2018, 10.1213/ANE.0000000000002508
- Lee J, Kraus KL, Couldwell WT: Use of the h index in neurosurgery. Clinical article. J Neurosurg. 2009, 111:387-392. 10.3171/2008.10.JNS08978
- Spearman CM, Quigley MJ, Quigley MR, Wilberger JE: Survey of the h index for all of academic neurosurgery: another power-law phenomenon?. J Neurosurg. 2010, 113:929-933. 10.3171/2010.4.JNS091842
- Thiessen CR, Venable GT, Ridenhour NC, Kerr N: Publication productivity for academic ophthalmologists and academic ophthalmology departments in the United States: an analytical report. J Clin Acad Ophthalmol. 2016, 8:19-29. 10.1055/s-0036-1581111
- Svider PF, Choudhry ZA, Choudhry OJ, Baredes S, Liu JK, Eloy JA: The use of the h-index in academic otolaryngology. Laryngoscope. 2013, 123:103-106. 10.1002/lary.23569
- Choi M, Holliday EB, Jagsi R, Wilson LD, Fuller CD, Thomas CR Jr: Citation-based estimation of scholarly activity among domestic academic radiation oncologists: five-year update. J Radiat Oncol. 2014, 3:115-122. 10.1007/s13566-013-0103-x
- Rad AE, Brinjikji W, Cloft HJ, Kallmes DF: The h-index in academic radiology. Acad Radiol. 2010, 17:817-821. 10.1016/j.acra.2010.03.011
- Benway BM, Kalidas P, Cabello JM, Bhayani SB: Does citation analysis reveal association between h-index and academic rank in urology?. Urology. 2009, 74:30-33. 10.1016/j.urology.2008.10.045
- Ence AK, Cope SR, Holliday EB, Somerson JS: Publication productivity and experience: factors associated with academic rank among orthopaedic surgery faculty in the United States. J Bone Joint Surg Am. 2016, 98:e41. 10.2106/JBJS.15.00757
- Ball P: Achievement index climbs the ranks. Nature. 2007, 448:737. 10.1038/448737a
- Gasparyan AY, Yessirkepov M, Duisenova A, Trukhachev VI, Kostyukova EI, Kitas GD: Researcher and author impact metrics: variety, value, and context. J Korean Med Sci. 2018, 33:e139. 10.3346/jkms.2018.33.e139
- Zhu E, Shemesh S, Iatridis J, Moucha C: The association between scholarly impact and National Institutes of Health funding in orthopaedic surgery. Bull Hosp Jt Dis (2013). 2017, 75:257-263.
- Forrester LA, Seo LJ, Gonzalez LJ, Zhao C, Friedlander S, Chu A: Men receive three times more industry payments than women academic orthopaedic surgeons, even after controlling for confounding variables (Epub ahead of print). Clin Orthop Relat Res. 2020, 10.1097/CORR.0000000000001132
- Lieber AM, Kirchner GJ, Zavras AG, Kerbel YE, Khalsa AS: Industry consulting payments to orthopedic surgeons are associated with increased publications. Orthopedics. 2019, 42:137-142. 10.3928/01477447-20190424-03
- Day CS, Lage DE, Ahn CS: Diversity based on race, ethnicity, and sex between academic orthopaedic surgery and other specialties: a comparative study. J Bone Joint Surg Am. 2010, 92:2328-2335. 10.2106/JBJS.I.01482
- Myers SP, Reitz KM, Wessel CB, Neal MD, Corbelli JA, Hausmann LRM, Rosengart MR: A systematic review of gender-based differences in Hirsch index among academic surgeons. J Surg Res. 2019, 236:22-29. 10.1016/j.jss.2018.10.015
- Canadian Medical Association: orthopaedic surgery profile. (2020). Accessed: March 21, 2020: https://www.cma.ca/sites/default/files/2019-01/orthopedic-surgery-e.pdf.
- Achkar E: Will women ever break the glass ceiling in medicine?. Am J Gastroenterol. 2008, 103:1587-1588. 10.1111/j.1572-0241.2008.01988.x
- Svider PF, Pashkova AA, Choudhry Z, et al.: Comparison of scholarly impact among surgical specialties: an examination of 2429 academic surgeons. Laryngoscope. 2013, 123:884-889. 10.1002/lary.23951
- Merton RK: The Matthew effect in science. The reward and communication systems of science are considered. Science. 1968, 159:56-63. 10.1126/science.159.3810.56
- Engqvist L, Frommen JG: The h-index and self-citations. Trends Ecol Evol. 2008, 23:250-252. 10.1016/j.tree.2008.01.009
- Holliday EB, Jagsi R, Wilson LD, Choi M, Thomas CR Jr, Fuller CD: Gender differences in publication productivity, academic position, career duration, and funding among U.S. academic radiation oncology faculty. Acad Med. 2014, 89:767-773. 10.1097/ACM.0000000000000229
Publication Productivity Among Academic Orthopaedic Surgeons in Canada
Ethics Statement and Conflict of Interest Disclosures
Human subjects: All authors have confirmed that this study did not involve human participants or tissue. Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue. Conflicts of interest: In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.
Cite this article as:
Atwan Y, Charron B P, Sidhu S, et al. (June 04, 2020) Publication Productivity Among Academic Orthopaedic Surgeons in Canada. Cureus 12(6): e8441. doi:10.7759/cureus.8441
Received by Cureus: May 15, 2020
Peer review began: May 20, 2020
Peer review concluded: May 26, 2020
Published: June 04, 2020
© Copyright 2020
Atwan et al. This is an open access article distributed under the terms of the Creative Commons Attribution License CC-BY 4.0., which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.