Outcome Measure Utilization Demonstrates No Relation to Journal Rank or Level of Evidence for Superior Labral Tears of the Shoulder

Introduction Superior labrum anterior to posterior (SLAP) tears are injuries of the shoulder's glenoid labrum involving biceps tendon insertion. We describe the scope of outcome measures used in the SLAP tear literature over the last decade and evaluate whether objective study metrics relate to level of evidence (LOE) and journal rank. Methods A comprehensive review of the literature was performed to identify all articles on the outcomes of the treatment of patients with SLAP tears published over the period of a decade (2007-2016). Studies were reviewed for LOE, SCImago Scientific Journal Ranking (SJR), study characteristics, and patient demographics. The utilization frequencies of outcome measures were recorded. Outcome measures were categorized as clinician-measured outcomes (CMOs), outcome scores/patient-reported outcomes (PROs), or other reported outcomes (OROs). Univariate analyses compared demographics and outcome-measure utilization among studies grouped by LOE (high: I/II vs. low: III/IV) and SJR (high: >2 vs. low: <2). Results The literature review identified 86 studies meeting inclusion criteria from an initial search of 582 articles. The mean study SJR was 2.02 and 0.85 +/- 0.79 CMOs were included. Twenty different outcome scores were utilized in the reviewed literature with a mean of 2.66 +/- 1.61 PROs/study. The most common outcome scores were ASES score, pain Visual Analog Scale (VAS), and constant score. High SJR studies had longer follow-up (p=0.036). High LOE studies utilized more CMOs (p=0.008) and more commonly included physical exam findings (p=0.0015). Revision surgery rate was less commonly reported in high LOE studies (p=0.036). None of the other outcome measures studied were associated with high study LOE or SJR. Conclusions Considerable variability exists in outcome-measure utilization within the recent SLAP tear literature. ASES score, pain VAS, and constant score are the most commonly reported PROs. CMOs and OROs are infrequently reported. There is little objectivity distinguishing higher and lower quality studies based on the proxy metrics used. Future SLAP tear research should focus on improving the consistency of outcome-measure reporting to enable more meaningful cross-study comparisons.


Introduction
Awareness and interest in the pathoanatomy and management of superior labral tears of the shoulder anterior to posterior (SLAP) have grown since their first description and early classification in the late 20th century [1][2]. Coincident with this trend has been the evolution of measures of patient outcomes within the orthopedic literature. Basic clinical and radiographic findings are no longer a sufficient barometer of success. Orthopedics has widely adopted additional outcome tools focused on the patient and his or her individual response to treatment: patient-reported outcomes (PROs) [3]. Patient-reported measures and outcome scores have become a fundamental component of outcome studies and a cornerstone of highlevel research, adding to the vast array of traditional measures such as range of motion (ROM), return-to-work/sport rate, and complication rates.
With the breadth of available measures and instruments and without the standardization of use, variability in outcome-measure reporting can make meaningful cross-study comparison difficult and lead to inconsistent interpretations of the literature [4][5][6]. Such variability has been well-documented in other highly studied areas of orthopedics such as rotator cuff repairs and anterior cruciate ligament injuries and, more recently, within the SLAP tear literature [7][8][9]. However, what remains unclear is whether the breadth and depth of outcome-measure reporting relate to a study's potential to affect change in clinical practice. Level of evidence (LOE) and the rank of the journal in which a study is published serve as easily accessible metrics that are likely related to the impactfulness of a given study. Thus, determining whether they hold any association with outcome-measure utilization is of interest in answering this lingering question.
The aim of this study was to rigorously evaluate the outcome-based SLAP tear literature over the past decade in order to aid in navigating future research efforts through better-informed study design and selection of outcome measures. Our primary purposes were the following: (1) to describe the scope of outcome measures utilization throughout the SLAP tear literature over the last decade; and (2) to evaluate whether objective study metrics and outcome-measure usage relate to proxy quality measures of research: LOE and journal impact factor/scientific journal rank (SJR). We hypothesized that greater outcome-measure utilization would be seen in a higher level of evidence studies and those published in higher-ranked journals.

Materials And Methods
A comprehensive, systematic literature review was performed of the PubMed and Cochrane databases, spanning a decade-long period, identifying SLAP tear outcome studies published over a decade from January 1, 2007, to December 31, 2016. The keywords used for the database searches were "shoulder," "SLAP," "superior labrum," and "superior labral." Inclusion criteria consisted of outcome studies, including patients with superior labral pathology of any type. Studies containing patients with concomitant pathology and comparison groups with other pathology were also included if the percentage of patients with superior labral pathology was greater than 20%. This cutoff was selected in order to be maximally inclusive of studies while still examining literature where the management of patients with SLAP tears was a significant component. Exclusion criteria included review articles, systematic reviews, meta-analyses, case reports, surgical technique articles, and studies without full text available in English.
The title and abstract review was completed for all articles found in the primary database searches. The screening was completed by two authors. Articles were selected for a full-text review according to the aforementioned inclusion and exclusion criteria. The full-text review was then used to identify those matching study parameters. As additional cross-referencing, a hand search was performed of the works cited of 13 narrative and systematic review articles identified during our literature search. These were surveyed by title, abstract, and full-text review for additional studies eligible for inclusion. Two or more authors participated in all phases of the review process and data extraction. Consensus over any discrepancies was reached by discussion and mutual agreement between the first and senior authors.
Included studies were reviewed for LOE, SCImago Scientific Journal Ranking (SJR), study characteristics, as well as subject and labral pathology demographics. All outcome measures that were reported in patient follow-ups were documented and grouped as clinician-measured outcomes (CMOs), PROs, and other reported outcomes (OROs). Measures including a clinicianmeasured component in the scoring (e.g., the American Shoulder and Elbow Surgeons (ASES), Constant, and the University of California at Los Angeles (UCLA) scores) were grouped as PROs, as these are a form of outcome scores. Among the PROs, clearly defined and/or previously validated outcome measures were tallied individually. However, multiple studies included PRO measures for pain and activity that were described in detail in the studies even if they were not routinely used in the literature or validated scores. These scores were grouped and tallied separately based on their area of assessment as "not otherwise specified (NOS)." All collected study variables and identified outcome measures are summarized in Table 1. Bias (as per the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines) was not included in the assessment because it was not relevant to the purpose of this study, which examined the reporting of outcome measures and did not compare the results of the measures themselves [10].

Variables
Outcome Measures  Individual ROM assessments were only documented as included if post-treatment degree values or scores were reported. ROM assessments that did not include degree measurements were tallied separately as "not otherwise specified (NOS)." Strength assessments and other physical exam findings (e.g., tenderness, signs, or maneuvers) were included if the presence or absence was reported in patient follow-up.

Journal and Study
For articles not clearly designating LOE, the level was derived from the Journal of Bone & Joint Surgery LOE standards and confirmed by two authors [11]. The SJR was obtained for each journal once all included articles had been identified [11][12].
Statistical analyses were performed using JMP® PRO Version 12.0.1 (SAS Institute, Cary, NC). Descriptive statistics were used to analyze the gathered study variables and the inclusion frequency of each of the outcome measures gathered (individually and grouped). In order to assess for study bias based on the inclusion of studies that were not exclusively examining patients with superior labral pathology, a subgroup analysis was performed to compare the studied variables among isolated SLAP tear studies and those including other pathology. Univariate analyses were then performed comparing study demographics and outcome-measure utilization among studies grouped by LOE (high: I/II vs. low: III/IV) and SJR (high: >2 vs. low: <2). Continuous variables were compared using the student t-test while nominal variables were compared with chi-square testing. A p-value of <0.05 was considered significant.

Study and patient demographics
The initial search identified 582 unique articles, of which 86 remained after the completion of our systematic-style comprehensive review over a decade-long period ( Figure 1). Included studies were published in 24 different journals, the most common being the American Journal of Sports Medicine (28), Knee Surgery Sports Traumatology Arthroscopy (10), Arthroscopy (9), and the Journal of Shoulder and Elbow Surgery (7)   Twenty different PROs were utilized in the reviewed literature; however, 13 of these were used in less than 10% of studies. The most common outcome scores used were American Shoulder and Elbow Surgeon Standardized Shoulder Assessment Form (ASES) (59.3%), VAS-Pain (41.9%), and Constant score (32.6%). Return to activity/sport was the most commonly included ORO. SLAP re-tear rates were reported in only roughly one-quarter of the studies, but re-tears were infrequently documented as being confirmed on advanced imaging (CT or MRI). Inclusion frequencies of the commonly reported CMOs, PROs, and OROs are detailed in Table 3.

Discussion
The principal findings of this work demonstrate that studies of a higher level of evidence and those published in higher-ranked journals demonstrate little difference in outcome-measure utilization and reporting compared to other studies. Persistent variability in outcome-measure reporting was identified over the last decade in the SLAP tear literature in addition to the low reporting rates of critical measures of treatment success. This information highlights critical areas for improvement for any future researcher examining treatment options for this patient population.
This study identified that the vast majority of SLAP studies are of lower LOE (level III and IV), and use roughly three PROs for measuring patient outcomes, most commonly being ASES, VAS pain, and Constant scores [13][14][15]. CMOs are not consistently reported, with even the most common (ROM) described in fewer than 50% of studies. Other regularly reported outcomes, notably complication and revision surgery rates, are even less frequently reported. This lack of consistency makes interpretation and cross-study comparison within the literature difficult, restricting the meaningful evaluation of emerging and evolving treatment options in the management of SLAP tears of the shoulder.
Although the exploration of outcome-measure reporting practices within the orthopedic literature has become a popular topic recently, only one prior study has explored this issue in the realm of SLAP tears. Similar findings of variability were identified in the recent work by Steinhaus et al. with some overlap in the study period [9]. However, this work did not examine the relationship between outcome-measure utilization and other study metrics and instead focused more on return-to-play reporting. LOE was only briefly discussed and journal ranking/impact was not examined. Also of interest, during the overlapping time periods of the two studies available for comparison (2010-2014), the Steinhaus work identified 35 studies while our study included 47. This is likely due to the more lenient inclusion of studies on combined lesions and those including other shoulder pathology. It was also felt to be more representative of the entire SLAP literature, which often includes concomitant diagnoses. The present study confirms many of the findings of the Steinhaus work regarding the variability in outcome-measure utilization but in a larger and more recent cohort of studies. Additionally, it provides an in-depth exploration of LOE and journal ranking as it relates to the reporting of outcome measures. Our findings are unique in that this is the first study to our knowledge to examine outcome-measure reporting in the orthopedic literature as it relates to these proxy measures of study quality and impactfulness.
In selecting metrics of study quality, we sought to use objective, quantitative measures that were easily accessible and interpretable by the common orthopedist. While standardized methodologic quality checklists exist, they are impractical to use for the casual interaction with the medical literature [10,[16][17]. Additionally, most physicians are not actively engaged in clinical research and are likely unlikely to be familiar with them. Thus, the quality and potential impact of a study is more commonly judged by more readily available parameters such as the LOE and the ranking of the journal of publication. These metrics, although less rigorous, carry tremendous implications for a study's potential to influence the decision-making of the majority of physicians and ultimately alter the practice of orthopedics. We hypothesized that these higher-quality studies would have some objective differences in terms of study design, particularly in the realm of outcome-measure reporting. Interestingly, this was not the case. Studies from higher-ranked journals only demonstrated longer patient follow-up. Higher LOE studies did show some additional differences among CMO utilization and revision surgery rate reporting. However, in regards to outcome-measure reporting overall, we found little that distinguishes higher and lower quality studies based on the metrics utilized.
Based on our study findings, we have identified essential areas for improvement in the SLAP tear literature. First and foremost, we feel that reporting of complication rates and revision surgery rates is an essential component of any outcome study in the surgical literature.
Reporting rates of these factors were dismally low in the literature reviewed, 41.9% and 37.2%, respectively. And similar to the findings of Steinhaus et al., return to activity/sports reporting also has substantial room for improvement, particularly for a pathology prevalent in an athletic population [9].
Another change that would be of tremendous benefit to the SLAP tear literature is a move toward standardizing the utilization of PRO measures. In our review of the past decade, we identified 20 different measures in circulation, only seven of which were reported in more than 10% of studies.
In a recent survey of surgeons in three large sports medicine and arthroscopy-focused organizations, Provencher et al. evaluated the use of outcomes instruments in clinical practice [18]. Consistent with the recent literature examined in this study, the ASES and Constant scores were among the three most preferred measures for the management of SLAP tear patients. The UCLA score was also a popular measure (second) among the surveyed surgeons for this patient population, whereas this score was the fifth most commonly utilized measure in the recent SLAP literature. Preference for these three scores was similar in the evaluation of rotator cuff and anterior instability patients, who are often present in cohorts examining SLAP tear outcomes. Establishing formal consensus and/or publisher guidelines for the use of these scores in future SLAP tear studies may assist in improving the quality and consistency of the literature base. Clear and structured outcome reporting of measures useful from a research and clinical care standpoint would enable meaningful cross-study comparisons and facilitate the progression of evidence-based patient care.
In a move towards this goal, Schmidt et al. examined the existing cohort of shoulder-specific PRO outcome measures objectively utilizing the Evaluating Measures of PROs (EMPRO) tool [3]. Their findings suggested that the ASES, SST, and Oxford Shoulder Score should be the primary options considered for measuring function and disability in patients with shoulder disorders. The Constant Score, of note, was not included in this analysis because aspects of this score require clinician response. Taking these findings into account along with the results of our study, investigators are empowered to select PROs for inclusion in future work that are reliable, valid, responsive, and have a basis for comparison within the existing literature. And although the Patient-Reported Outcomes Measurement Information System (PROMIS) has recently emerged within the orthopedic literature and carries the potential for creating a common language for outcomes reporting, limitations remain with respect to its implementation, and little evidence exists supporting its use in the athletic population [19]. Thus, it is likely too early to make this a standard measure for reporting, but researchers should stay attuned to its development as it may aid in improving the efficiency of shoulder outcome data collection in the near future.
Ultimately, journals may use this study and others to drive change in outcome reporting by setting publication standards. This is important not only within the shoulder literature but all orthopedic literature as the use of PRO measures expands. Journals often establish minimums of reporting for eligibility for publication (e.g., 2-year follow-up). Through structured evaluation in a manner similar to Schmidt et al., objectively selected essential sets of outcome measures could be instituted in popular areas of research [3].

Limitations
This study had several important limitations. First, despite performing a systematic review of two separate databases, other databases exist that were not included; thus, it was possible to have missed studies meeting study criteria. In a similar manner, exclusion of studies unavailable in the English literature may have biased our results towards outcome measures developed or more commonly utilized in English-speaking countries. Second, our utilization of the selected metrics of study quality in this manner has not been previously described and other important criteria for assessing study quality exist that were not examined before in our analysis [10,[16][17]. Finally, our methods of selecting measures for inclusion introduce a source of potential variation from other similar works. Many studies may gather a considerable amount of patient data but limit their reporting to notable or significant outcome findings. We opted to only include a measure as "reported" if objective values were included in the results, i.e., stating that a measure was assessed in the methods alone was not sufficient for inclusion. True outcome-measure acquisition may also be different from what is reported in the literature.

Conclusions
This study identified ongoing variability in outcome-measure reporting within the SLAP tear literature concurrent with prior investigations. Additionally, it was demonstrated that there is little objectively differentiating higher and lower quality studies in regard to outcome-measure reporting using the metrics described. Based on these findings, we recommend that SLAP investigators seek to standardize outcome-measure inclusion in future research efforts to enable better cross-study comparisons. Greater attention should be paid to the inclusion of CMOs, complication rates, and revision surgery rates. Ideally, standards for study design should be raised for publication in high-impact journals to drive this change.

Additional Information Disclosures
Human subjects: All authors have confirmed that this study did not involve human participants or tissue. Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue. Conflicts of interest: In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: Joseph J. King declare(s) non-financial support from Exactech. Unpaid consultant. Joseph J. King declare(s) stock/stock options from Pacira Phamaceuticals.