Poor Readability of Online Patient Resources Regarding Sentinel Lymph Node Biopsy for Melanoma

Background Established guidelines recommend that patient educational materials should be set at no higher than a sixth-grade reading level to be considered adequately comprehensible to the general public. Our study objective was to assess the readability of online patient resources related to sentinel lymph node biopsy (SLNB) as part of treatment for melanoma. Materials and methods The top 50 results from a Google search (search terms: “sentinel lymph node biopsy melanoma”) were analyzed using seven established readability formulae in order to determine their level of adherence to current guidelines. Results We found that the readability of available online patient resources is currently very poor, with only 12% of the websites meeting the sixth-grade reading level criteria according to at least one measure, and 0% meeting the criteria according to all seven assessment tools. Furthermore, half of search results were peer-reviewed academic journal articles not intended for the general public. Discussion and conclusions Online patient resources related to SLNB carried out as part of melanoma treatment have poor readability. Several simple measures may be taken in order to make these resources more accessible and comprehensible to a broader audience. These resources should undergo ongoing evaluation, with the ultimate goal being improved readability and patient education.


Introduction
In recent years, the Internet has become a ubiquitous source of information for patients who are seeking to learn more about their symptoms, conditions, upcoming procedures or any aspect of subjects relevant to their own, or to a friend or family member's care [1,2]. Powerful 1 2 internet search engines such as Google are often queried for medical advice well before a patient actually sees a medical professional in consultation [3]. In many ways, an online consultation with "Doctor Google" is understandable due to the appeal of the easily accessible, and seemingly limitless, healthcare-related information the Internet provides. However, while the Internet does have an important role to play in patient education, it is its unfiltered and unvetted nature that makes it imperative that all users carefully scrutinize the accuracy and reliability of the information it provides. Even a resource with a credible author may not be of adequate quality or accessibility for the general public. Readability is a key aspect of accessibility that is often ignored, and guidelines formulated by the American Medical Association (AMA) and the US Department of Health and Human Services (USDHHS), make it clear that patient reading material should be set at no higher than a sixth-grade reading level in order to be considered accessible and comprehensible to the majority of the general public [4]. Readability may be readily evaluated using several validated formulae.
Melanoma, also referred to as malignant melanoma or cutaneous melanoma, is the least common but most lethal form of skin cancer. However, it is amongst the most prevalent cancer types that is diagnosed in young and middle-aged adults, and therefore is responsible for a substantial burden of disease [5]. Early melanoma detection, staging and treatment may also significantly improve patient outcomes. Sentinel lymph node biopsy (SLNB) is a surgical lymph node mapping technique that is commonly utilized in order to identify, for sampling, the first lymph node(s) to which a particular malignancy would metastasize [5]. SLNB also allows for reduction in procedure-related morbidity when compared to the performance of routine lymphadenectomy. It helps the clinician to stage, prognosticate, and select appropriate treatment for melanoma patients [6].
The objective of this study was to assess the readability of online patient resources that are specifically related to SLNB being carried out as part of melanoma treatment, and evaluate these observations within the context of currently established readability guidelines. Furthermore, recommendations that ultimately will make reliable, evidence-based, highquality and easily understandable health-related information available to a broader audience, will also be reviewed. Several recently published articles have reported that online patient resources for a variety of surgical procedures, and other health conditions including a variety of cancer types, tend to be written at too high of a grade level to allow for adequate comprehensibility [7][8][9][10][11][12][13][14], and we hypothesize that this finding will also hold true for resources related to SLNB that is being performed as part of melanoma treatment.

Materials And Methods
The Google search engine was used to run our search, as it has been reported to be the most prominent and most popular search engine utilized by patients [15]. Prior to conducting the search, the browser data (including cookies and cache) was cleared, and we also made sure it was not logged into any specific user account. The following search terms were entered into a Google Chrome web browser, using the incognito browsing mode: "sentinel lymph node biopsy melanoma". The search was performed on July 1st, 2017. The top 50 unique web pages yielded from the search that met study inclusion criteria and did not meet study exclusion criteria, were investigated. The study inclusion criteria required the website to be: (1) written completely in English, (2) free-to-access, and (3) specifically contain information regarding SLNB being performed for melanoma. The study exclusion criteria required the website not to: (1) have obvious financial conflicts of interest (i.e., specifically sponsored by an external organization or company, or advertising products for sale), (2) be a news article, or (3) solely be a video.
The readability of each website was evaluated using seven different validated readability formulae ( Figure 1). The Flesch-Kincaid Grade Level (FKGL) and the Flesch Reading Ease (FRE) both primarily take into account average sentence length, and average syllables per word [16,17]. The Gunning-Fog Score (GFS) is based on average sentence length and the number of polysyllabic words (words containing three or more syllables) [18]. The Coleman-Liau Index (CLI) is based on the variables of average number of letters per 100 words (L), and the average sentence length (S) [19]. The Automated Readability Index (ARI) is similar to the CLI in that it also considers the number of characters per word [20]. The Simple Measure of Gobbledygook (SMOG) index takes three 10-sentence samples near the beginning, middle and end of a piece of text. The number of polysyllabic words are counted and used to calculate a specific grade level. If there are fewer than 30 sentences, the formula includes a correction factor [21]. The New Dale-Chall Score (NDC) includes a list of 3000 words an American fourth-grader can reliably understand, and any word not on this list is considered a "difficult word" [22]. The FRE is a 100-point scale with higher scores indicating more easily understood text. Precise scoring is outlined in Table 1. The FKGL, GFS, CLI, ARI and SMOG indicate the US academic grade level, or number of years of education, that are required in order to comprehend the text. The NDC has a similar but distinct scoring system that is outlined in Table 2.

or lower
Easily understood by an average 4 th grade student or lower 5.0-5.9 Easily understood by an average 5 th or 6 th grade student 6.0-6.9 Easily understood by an average 7 th or 8 th grade student 7.0-7.9 Easily understood by an average 9 th or 10 th grade student 8.0-8.9 Easily understood by an average 11 th or 12 th grade student 9.0-9.9 Easily understood by an average 11 th or 12 th grade student In order to minimize the risk of bias and human error during calculations, and for ease of use, all seven readability tests were administered using a specific online readability calculator recommended by the National Institutes of Health (NIH) (https://readability-score.com/). Prior to analyzing the web pages, the "ideal" readability criteria for online resources were established. The USDHHS recommends health-related materials be written at a sixth-grade level. Thus, for this study, the level of acceptable readability was determined to be greater than or equal to 80.0 for the FRE, less than or equal to 5.9 for the NDC, and less than or equal to 6.9 for the FKGL, GFS, CLI, ARI and SMOG.
Data entry and analyses were performed using a Microsoft® Excel spreadsheet (Microsoft®, Redmond, Washington). Standard independent upper-tailed hypothesis tests were conducted for each readability index, comparing the mean score of websites that were identified in our search with the grade level, or other measure of readability, recommended by the AMA and USDHHS. Differences were considered statistically significant when they yielded a p-value equal to or less than 0.05.

Results
The first 50 websites yielded by the search that met all of the inclusion criteria, and none of the exclusion criteria, were analyzed for their readability ( Table 3). A webpage was categorized as "specialty" if the root website focused on any of the following topics: melanoma, lymph node biopsy, skin cancer, dermatology, or general surgery (10 out of 50); otherwise, it was categorized as "general" (40 out of 50). Moreover, half of the 50 websites were themselves, or contained links to peer-reviewed journal articles, or to guidelines that were derived from them.   All seven assessment tools reported statistically significant results in that the calculated pvalue was less than the standard alpha value of 0.05. In other words, all seven average readability scores ("average" meaning average of the top 50 websites), as calculated by the assessment tools, were significantly different (worse) than their respective "acceptable" readability scores previously outlined in the Methods section. This observation remained unchanged whether or not journal articles, a subgroup that could potentially bias observations towards worse readability scores, were included in the analysis. Distributions of readability scores of the health information websites were outlined in box-and-whisker plots (Figure 2a,  2b). A comparison of mean readability scores between the websites that were classified as "general", and the websites that were classified as "specialty" is shown in Table 4. Specialty websites tended to have better readability than general websites, but these differences were not statistically significant. A comparison of mean readability scores between peer-reviewed journal articles, and all other websites, is shown in Table 5. Journal articles were significantly less readable than non-journal articles. A comparison of mean readability scores between the top 10 websites that were identified by the search, and the remaining 40 websites, is shown in Table 6. There was no appreciable difference in readability identified by this comparison. Of the 50 websites evaluated, only six (12.0%) were within the limits of the recommended sixth-grade reading level as evaluated by at least one assessment tool, while there were no websites either at or under the sixth-grade reading level according to all seven assessment tools ( Table 7). This observation did not change when journal articles were either included or excluded from the analysis.

Discussion
Despite readability scores varying between the different formulae utilized, a consistent observation emerged from the analysis. None of the 50 websites were consistently written at a reading level below the sixth-grade level, as recommended by the AMA and USDHHS guidelines, and this observation overwhelmingly highlights a need for major change. The rationale behind these readability guidelines is based upon evidence that the average reading level among American adults is between grades eight and nine, and that approximately one in five of these people read below a grade five level and are considered functionally illiterate [23]. Although according to a report by Statistics Canada, the Canada-United States literacy gap is sizable, with the average Canadian adult more literate than the average American adult by almost a full year of schooling, 48% of Canadian adults 16 years of age or older did not possess the literacy skills required by the current workforce, with many of them being new immigrants [24]. This is especially problematic as literacy is known to be highly positively correlated with socioeconomic status (SES) [25]. Those people with lower literacy rates, and thereby more commonly of lower SES, are also more likely to suffer from a plethora of chronic health conditions that include: type 2 diabetes, hypertension, ischemic heart disease, mental illness, and cancer. Populations with low literacy rates also have many risk factors for these conditions that include: smoking, alcohol and illegal drug use, adverse childhood experiences, and others [26,27]. Interestingly, one study found that the relative risk of developing melanoma was actually decreased in people with lower annual household incomes, but despite this observation those individuals with lower SES were more likely to have a later-stage disease at diagnoses. This observation could potentially be due to substantial barriers that these individuals face when accessing health care (e.g., waitlists in Canada and the lack of universal health care in the United States) [27]. This was an important observation because the prognosis of melanoma is highly dependent on its stage at the time of diagnosis. Table 4 suggests that there were no significant differences in the readability scores of general medical websites as compared to specialty-specific websites when evaluated by any of the measurements utilized. However, specialty websites tended to be more readable, perhaps because the authors may be experts on the subjects, and are able to better present the relevant information in terms that are more readily comprehended by the general public. Table 5 identifies a significant discrepancy (p-values well below 0.01 across all measures) between the readability of journal articles compared with non-journal articles. This observation is not surprising because journal articles are written for an academic audience that would be expected to have higher literacy scores than the general population. Regardless, these journal articles are identified in the top online search results that are accessible by the general population. Interestingly, articles published in peer-reviewed journals, or links to such articles, represented half of the top 50 search results, most likely because the search phrase was very specific. This suggests that the breadth of online educational resources is actually even worse for people with limited literacy than was previously believed. Table 6 shows that there was no significant difference or trend in the readability scores of the top 10 search results, when compared to the next 40. This is important information because the top 10 websites are much more likely to be accessed by patients, in part because they comprise the first page of search results. The top 10 websites were evenly split between five general websites and five specialty websites. The specialty websites were all authored by organizations or research institutes focused on the study and/or treatment of melanoma ( Table 3).
It is important for individuals with either a suspected or confirmed melanoma diagnosis to learn about their cancer, and to understand their recommended management plan, that often includes SLNB. It is expected that members of the patients' healthcare management team, especially the surgeon, will review with them the risks and benefits of SLNB. However, online resources do still represent an important and readily available source of supplementary educational information; it is crucial that the treating physician is cognizant of website readability when recommending or directing patients toward particular resources. This is the first study to evaluate the readability of online resources for SLNB, and one of few to assess the proportion of journal articles amongst the search results.
The current study has several limitations that must be reviewed. The scope and coverage of available online patient resources may not have been ideal, as only a single search phrase and a single search engine were utilized, and only 50 websites were analyzed in detail. However, we found that variations on the search phrase did not appreciably change the search results, and that Google is currently the most favored search engine for finding health-related information [15], often cited as being the most reliable and least redundant [28]. Furthermore, Google metrics and analytics for the United States suggest that most users evaluate a median of two to four webpages (20-40 search results) for a given search, thus providing a good rationale for our 50-website focus. The study also is limited in the applicability of its observations to more ethnically and geographically diverse patient populations. Only English websites were included and analyzed in the current study. People living in other countries, who may speak other languages, may utilize other search engines that yield different search results, as well as have unique measures or standards of reading literacy. The cross-sectional nature of our study must also be considered, as the Internet is dynamic and is constantly evolving. The same search performed even months earlier or later would have likely yielded somewhat different results; in fact, a repeat search was performed on November 6th, 2017, and the list of top 50 websites the search yielded was 86% similar to the original list. Despite these limitations, we intended this study to inform others about the current state of online patient resource readability, and to provide suggestions that could help facilitate much-needed change.
The observation that the websites' readability scores were inadequate amongst multiple measures suggests that complex sentence structure, semantics, and vocabulary are all key issues that must all be addressed in the future. When editing or developing online resources, medical jargon must also be avoided whenever possible. This is especially crucial in the first few lines of text, or in any sort of overview paragraph, because if a user is unable to initially comprehend a resource they may cease reading altogether [29]. If using medical terminology is unavoidable, it would be helpful and intuitive to add either a hyperlink to an understandable definition, or a mouse-over pop-up box that contains the definition [30]. Alternatively, dictionary, thesaurus or translation support could be provided, though this would require more time and effort by the user [30]. Keywords and phrases should also be highlighted and emphasized [30]. Educational objectives should be targeted at the sixth-grade reading level, and should also be carefully reviewed so that the authors have a better understanding of the most appropriate language structures to utilize. If re-writing a resource is necessary, authors should focus on utilizing plainspoken words, simple sentence structures, and the active voice, whenever possible. Other areas for future study include investigating whether the timing and character of patient internet searches (e.g., before versus after meeting with the treating surgeon) is impacted by preoperative counselling. Projects such as Health on the Net Foundation show promise in addressing other important dimensions of health information accessibility, such as accuracy and quality, which were not the focus of the current study.