Readability Assessment of Online Patient Education Materials on Atrial Fibrillation

Health literacy is emerging as an important factor for medical outcomes as more patients turn to the internet for information about their disease. However educational materials on complex conditions such as atrial fibrillation tend to still be esoteric and result in compromised patient autonomy. We add to the current literature by examining the reading level of websites of major healthcare intuitions and general medicine websites. An online Google search using the term “atrial fibrillation” was used to collect patient educational material from the first 20 academic health institutions (AHI) and 20 non-affiliated general medicine websites (GMW). The materials were assessed for readability using nine (9) tests from the analysis software Readability Studio (Oleander Software Solutions Ltd., Maharashtra, India). The patient education materials from the AHI and GMW websites were written at a college freshman reading grade level (13.050 ± 0.845) and high school junior year reading level (11.64 ± 0.789) respectively. The GMW tend to have a wider range of readability levels, and many were scored at the 6th-grade level. In conclusion, the readability levels of patient education materials on atrial fibrillation from both the AHI and GMW are well above the 6th-grade level recommended by the NIH and AMA, posing a risk to the patients’ understanding of the materials. The high readability scores found across all websites and the differences between the groups have been attributed to the various goals and target audiences of the material.


Introduction
Although the increased utilization of the internet has made healthcare information more accessible, poor healthcare literacy can be a major deterrent to patient autonomy [1,2]. Patient autonomy relies on the ability for patients to obtain and comprehend basic health information and services needed to make appropriate medical decisions [3]. This is not an easy task as medical terminology is nuanced and sometimes requires higher-level education to understand. As a result, many patients have little understanding of their diseases and even worse, little understanding of their treatment plan resulting in poor medical adherence, worse disease status, and early hospital readmission and mortality [4,5].
This inadequate comprehension of healthcare material is compounded when acquiring information about complex diseases like atrial fibrillation, a disease that affects between 2 and 6 million people in the US [6]. Previous studies have demonstrated that 25% of the international adult population is unable to adequately explain atrial fibrillation, and these knowledge deficits are more prevalent in individuals with less education [7,8]. In addition to knowledge of the disease process, atrial fibrillation requires an adequate comprehension of multifaceted treatment plans, which require dangerous medications with the propensity to cause major bleeding [9,10]. Therefore, internet sources relaying uncomplicated and accurate information on atrial fibrillation would benefit millions of people and alleviate some of the burden faced by the healthcare industry due to a misguided understanding of atrial fibrillation treatment plans. Previous studies for cardiovascular disease have been developed to assess and evaluate education material [11]; however, few studies have analyzed the internet-based healthcare material for atrial fibrillation. Preliminary studies have shown that atrial fibrillation internet material requires a 10th-grade reading level, which is well above the recommended 6th-grade reading level [11,12].
As health care literacy is a fundamental component of patient education and patient shareddecision making, we hope to add to the current literature and to analyze the education level of major healthcare institutions as well as general medicine websites. Our evaluation will better delineate the current landscape of healthcare information and will uncover areas where healthcare information can be more appropriately presented.

Materials And Methods
An online search on Google was conducted on March 19, 2019 using the single key term "atrial fibrillation". The search was limited to the United States and to articles written in English. Patient educational materials were only included if they satisfied the following criteria: they were over 100 words; included a full overview of the disease including causes, risks, symptoms, diagnoses, and treatments; and were created specifically for patient use and information. Any articles or blogs were excluded from the analysis. The first 20 eligible academic institutions that were associated with a hospital were categorized as academic hospital institutions (AHI). The first 20 eligible general medicine websites that were not associated with any universities or hospitals were categorized as non-affiliated general medicine websites (NAGMW).
The materials were downloaded from the websites and reformatted into Microsoft Word (Microsoft Corp., Redmond, WA, USA). These texts were then edited to exclude any hyperlinks, advertisements, or extraneous information. The texts were also edited to reflect the original paragraph or bullet formats presented on the websites. The materials were then assessed for readability using the analysis software Readability Studio (Oleander Software Solutions Ltd., Maharashtra, India). A series of nine (9) previously validated scales for readability were tested on each study, including: Coleman Liau, Flesch Kincaid, FORCAST, Fry, Gunning Fog, Raygor Estimate, Simple Measure of Gobbledygook (SMOG), New Dale Chall, and Flesch Reading Ease ( Table 1).

TABLE 1: Readability Assessment Scales
The letters designate the following: A = average # of letters per 100 words; B = average # of sentences per 100 word; C = number of "easy" words within sample; D = average number of words per sentences; E = number of single syllable words in a 150-word sample; F = average number of sentences; G = average number of words with 3 or more syllables; H= number of polysyllabic words in sample; I = percentage of difficult words in text.

SMOG = Simple Measure of Gobbledygook
The mean readability grade and age levels were computed for each of the websites. The mean overall reading grade level was calculated for both categories and compared using statistical analysis in Statistical Package for the Social Sciences (SPSS) (IBM Corp, Armonk, NY, USA). The data was treated as nonparametric and compared with a simple independent samples T-test with significance set at p<0.05. Maryland, and Temple, failed the assessment by the Fry and Raygor Estimate readability scales for having too many 6+ syllable words and long sentences in their texts ( Table 2 and Figure 1).   For the non-academic medicine websites, the patient education materials were written at a high school junior year reading level (11.64 ± 0.789 CL). Online information from VeryWellHealth and eMedicine health were written at the highest grade level with a mean of 14.7. WebMD's material required the lowest reading level of all the websites, with a mean grade level of 7.8. In addition, the website scored at the recommended 6th-grade reading level on two of the readability scales, the Flesch Kincaid and Raygor Estimate. Only one website, Medtronic, failed to be assessed by the Fry or Raygor Estimate readability scales for having too many 6+ syllable words and long sentences ( Table 3 and Figure 2).   Overall, the materials produced by the general medicine websites were significantly easier to read than those produced by the academic hospital institutions (11.64 vs 13.050 mean grade level, p value 0.015) ( Table 4). The range of average grade levels was larger for the NAGMW group, and the AHI group showed a more even distribution for its scores (Figure 3). In both groups, the FORCAST readability scale produced the most precise results, with all scores falling within two to three grade levels from each other within each group. The Fry, Gunning Fog, and Raygor Estimates scales displayed the most variability in the scores and highlighted most of the differences in readability levels within the groups, especially with the general medicine websites (Figure 4 and Figure 5).

Discussion
Health literacy is quickly becoming a key subject of analysis by the medical community as numerous studies have emerged revealing the adverse effect of poor health literacy on medical outcomes [13,14]. It is defined by the Institute of Medicine as "the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions" [15]. Therefore, information received by patients has a profound effect on their medical decisions, and in order for patients to control their own healthcare, healthcare-related information must be both informative and comprehensible.
An important part of this health information is the ease at which the text can be read and understood, otherwise known as its readability. In the United States, it is typically measured by school grade levels, ranging from kindergarten to post-graduate reading proficiency. Many tests have been developed to assess readability of documents by analyzing components of the text including the complexity of the words, length of sentences, and use of high-level jargon ( Table  1). Recent surveys show that only 12% of Americans have proficient health literacy [16], and over 93 million Americans have basic or below basic literacy skills [17].
A 2018 survey conducted by the US Census Bureau determined that 99% of the US population can read at a 5th-to 6th-grade level [18]. Therefore, many institutions including the AMA and NIH have recommended that patient educational materials be written at a 6th-grade reading level to ensure that the majority of the population will be able to comprehend the material [19,20].
As the use of the internet has increased in the last decade, more patients are turning to Google to access these educational materials. Nearly 60% of Americans say they use the internet as their first choice to get medical information, and about 80% of those individuals will start with a simple Google search of their disease [21]. This exposes patients to a wide variety of websites with medical information from many different sources. Although the information presented in the materials must be accurate, it should also be easily understandable as studies show that misinterpretation of online health information can lead to negative healthcare outcomes [22].
Atrial fibrillation is one of the more complicated diseases of the heart as it requires an understanding of electrophysiology, cardiovascular mechanics, and its relation to stroke risk. A variety of tests and treatments are available for patients and can often be confusing for newly diagnosed patients, especially since most procedures involve electrical and chemical ablation [11]. A previous survey has shown that nearly 50% of physicians felt that their patients were unable to explain their diagnosis sufficiently [7]. Other surveys also demonstrate that many of the deficits in knowledge about treatment of atrial fibrillation are more pronounced in patients with lower educational backgrounds and poor health literacy [8,23]. Educational interventions have proven in the past to increase patients' knowledge about atrial fibrillation [24], and therefore all sources of patient education should be scrutinized including materials from online websites.
This study reveals that materials written by both academic institutions and medicine websites required reading levels well above the recommendations made by most public health institutions. This finding is consistent with similar studies conducted in different specialties [25]. Patients with low levels of literacy will most likely have a difficult time understanding these websites. The internet allows the patient to have greater autonomy over their care and therefore, they may choose what resources they use to access educational materials. This may lead to the patient receiving confusing and inaccurate information about their disease which may adversely affect their medical decision-making capacity [26].
Materials from the general medicine websites were significantly easier to read than ones from the academic institutions, which was also found in a similar analysis on online education for pancreatic cancer [27]. The main difference in these medical websites from the academic hospital institutions is most likely the use of medical terminology. The nine readability scales calculate the scores based on the number and complexity of the sentences, words, and syllables used in the text. Since most of the texts between the two groups were similar in length, the amount of medical jargon likely had the largest impact on the readability levels of these texts. A previous readability analysis demonstrated that replacement of the medical terminology with simpler words reduced the required reading level to more acceptable levels for nearly all websites examined in the study [28]. Without sacrificing the accuracy of the materials, reducing the amount of medical jargon should be a focus when reviewing these texts [29].
The authors recognize that with atrial fibrillation, many of these polysyllable words are unavoidable including terms related to electrophysiology. Therefore, other areas for the improvement of readability should be considered. Many of the websites in this study contained videos, images, and charts that were not considered in the analysis. The use of multimedia may not alter the readability level of the text but may help patients better understand the information [30]. This may be particularly used for atrial fibrillation as it involves visualizing the abnormal electrical rhythm and its path in the heart. In addition, many guides have been published specifically to improve healthcare materials. Recommendations from these websites include limiting sentences to 8-10 words, paragraphs to 3-5 sentences, and using more active phrases rather than passive ones [19].
Many of the limitations of this study are inherent to the readability scales used in the analysis. They only measure the complexity of the words, sentences, and paragraphs of the text, but not the specific format or arrangements of the material. As noted earlier, they also do not measure the use of multimedia which may supplement the reading material. While the scales do provide the best quantitative measure and allow for comparison of different materials, analysis of multimedia in tandem with the reading scales would give a better picture of the readability of the website. In addition, while some websites may be easier to read it is important to also conduct an analysis of the accuracy of the material presented to ensure that the information the patients receive is correct. This study focused on websites from academic institutions and non-affiliated general medicine websites, and while these categories encompassed most of the websites on the first 10 pages of the Google search, there are other websites from hospital networks and independent physicians that appeared and can be analyzed in future studies. The word atrial fibrillation may inherently inflate the reading level. The word can be replaced in the future with "this disease" or "AF" or something similar. We mitigated this issue by using multilpe reading scale metrics that focus on different components of the material. However, we maintained the word in our analysis because other studies have shown that complex words make text esoteric because of their complex meaning and their length, syllable count, and familiarity. Therefore, the word atrial fibrillation may be an important component when measuring readability. Future sources written about atrial fibrillation should work to use replacement words as they may make reading the information easier helping patients better understand the disease and treatment plan.

Conclusions
Patient education materials on atrial fibrillation from academic hospital institutions and nonaffiliated general medicine websites are written at grade levels higher than the 6th-grade level recommended by many major public health institutions, including the NIH and the AMA. The increasing number of patients who refer to these materials may not be able to comprehend these texts and may be less likely to make informed decisions about their medical care. In addition, materials from general medicine websites are significantly easier to read than ones from academic institutions, indicating that the selection of materials is important. Future interventions should place emphasis on the readability of online patient education materials to improve outcomes for common and complex cardiovascular diseases.

Additional Information Disclosures
Human subjects: All authors have confirmed that this study did not involve human participants or tissue. Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue. Conflicts of interest: In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.