1. Introduction

Chronic (or persistent) pain is common worldwide with an estimated prevalence to be as high as 55%, with higher rates in low-income and middle-income countries.20,38 For example, chronic pain prevalence in Australia is 20%,11 27% in Mexico,9 and 53% in Nepal.35 Chronic musculoskeletal pain conditions are the number one cause of disability globally, with a 32% increase in the past 3 decades,15 estimated financial costs at $73.2 billion in 2018 in Australia alone.10 Effective management of chronic pain requires accurate pain science education.43

Misconceptions about pain are common and are associated with poorer health outcomes. For example, some common misconceptions in people with back pain are that the spine is fragile and needs protection, a wrong movement can cause serious back pain, imaging such as x-rays can always identify the cause of back pain, and that bed rest is needed to treat back pain.8,16,25 Pain science education aims to address pain-related misconceptions allowing individuals to reconceptualize pain, take greater control of their pain, and make more informed decisions regarding their pain care.1,11 Understanding that pain is a biopsychosocial experience, the context in which it is experienced, and the behaviour associated with pain can influence how one can experience pain.7 Three major target concepts have been identified by patients to reconceptualize pain and manage it: (1) “Pain does not mean my body is damaged,” (2) “Thoughts, emotions, and experiences affect pain,” and (3) “I can retrain my overprotective pain system.”22,34 We consider these as core concepts of pain essential to self-management strategies.

Given the positive effects that pain science education can have on different aspects of life of people living with chronic pain,40,43 an important endeavour is to study how people acquire information about pain and to examine the sources available. People primarily use the internet to obtain health information.5,37 Health information should be evidence based, accurate, comprehensible, and accessible to prevent misunderstandings, especially when health and web literacy are poor, which is a common phenomenon.3,27 This is especially true for complex conditions such as chronic pain. The aim of this study was to assess the content of websites providing information about chronic pain. We analyzed the readability, credibility, and accuracy of their content and mapped the inclusion of the 3 core concepts of pain described above. To gain this insight into how pain information is provided on the web worldwide, we conducted the search using the Google search engine and analyzed the information available from 3 different countries representing 3 continents and economies (Australia, Mexico, and Nepal).

2. Methods

This study was a cross-sectional evaluation of the content of websites providing information about chronic pain. We searched 2 keywords to select the websites to evaluate “chronic pain” and “persistent pain.” We included the keyword “persistent pain” to ensure that we captured as many sites as possible related to the topic because this term is gaining acceptance by the public and in health care.4The evaluation was conducted in 3 countries: Australia, Mexico, and Nepal. The keywords were translated into Nepali and Spanish to search on Google, Nepalese, and Mexican government websites.

2.1. Website selection criteria

Websites were included if they contained a definition of chronic pain and included treatment pathways/recommendations. Websites were excluded if they focused on chronic pain arising from concurrent comorbidities contributing to continual or actual tissue damage (eg, cancer pain), purely for advertising or promotional purposes (ie, they explicitly promoted a product and 70% or more of the content was centered on the product), news and/or journal articles, pages unrelated to chronic pain or those only providing a list of website links, “for profit” pages, unless they attempted to educate readers about pain, pages specifically for health professionals only, and pages that were personal stories or blogs.

2.2. Search strategy

We used Google as our primary search engine. The browser history, cache, and cookies were cleared prior to running the search, and an incognito window was used to prevent the search engine algorithm from altering the results based on the user's search history. The first 20 results available for each country were collected for each search term (n = 20 chronic pain, n = 20 persistent pain) as the first 20 results receive the most traffic, and this number is commonly used.21,42

Two reviewers then independently read and filtered the sources through the inclusion and exclusion criteria described above. The same search methodology was used to assess government health websites for each country. For Australia, we searched Australian Federal, State, and Territory health websites (eg, https://www.betterhealth.vic.gov.au/). For Mexico, we searched the official search engine provided by the Mexican government (eg, https://www.gob.mx/busqueda?utf8=%E2%9C%93). For Nepal, we searched the official health websites provided by the Nepalese government (eg, https://www.mohp.gov.np/eng/).

2.3. Readability

For websites in English, the FKRE was used as a measure to quantify readability. Scores range from 0 to 100 with a score of 0 indicating the text is difficult to read, and a score of 100 is easier to comprehend.13 A score of 70 to 80 is equivalent to the reading level of a seventh grader (termed “fairly easy”), 60 to 70 is the equivalent of an eighth or ninth grader (“plain English”), and a score of less than 60 is at a reading level of a 10th grader and above (considered “fairly difficult” to “very difficult”).13 To obtain the score for the websites in English, we used the automatic tool provided by Microsoft Word Office. We copied all the content of the website into a Word document and obtained the scores from the “document stats” in the Editor function. In the case of the websites in Spanish, we used an equivalent measure called INFLEZ.2 To get the score estimates, we used the website https://legible.es/. We considered the readability score of 60 to 90, the reading age of 12 to 14 years, and equivalent to grades 6 to 9 to have adequate readability. A score below 60 would be considered as having low readability.18,41

2.4. Credibility 2.4.1. Journal of the American Medical Association benchmark criteria

Websites were evaluated against the 4 standards set by the Journal of the American Medical Association (JAMA) benchmark criteria to assess the reliability and transparency of health information online.14 The benchmark criteria are as follows: (1) authorship—sites provide details about authors (whether individual or group); (2) attribution—cites references containing journal articles or other published sources such as books; (3) disclosure—website ownership and potential conflicts of interests are disclosed; and (4) currency—sites provide dates when written or updated.24,36 According to the JAMA measure, websites were given 1 point per criterion met, where a score of 4 indicates a more credible source.

2.4.2. The health on the net code

The Health on the Net Code (HONcode) seal of approval is an ethical code awarded to websites that comply with the following 8 principles: authority, privacy policy, complementarity, attribution and date, transparency, justifiability, advertising policy, and financial disclosure.6,24 The code addresses the reliability and credibility of a website to ensure that trustworthy health and medical information has been established.

2.5. Accuracy 2.5.1. Core concepts of pain science education

To assess the quality of pain information provided on the websites, we adopted 3 core concepts reported by Leake et al.22 that were deemed the most essential based on the review of literature,29,31,33,34 and the research team that has a combined extensive experience in pain science research. Furthermore, these concepts, although not exhaustive, are consistently present in pain science education programs (eg, pain revolutio28 and Zoe and Zak's Pain Hacks).30 Each concept was considered present if it met the following criteria:

Concept 1 (“Pain does not mean my body is damaged”)—this concept highlights the importance of dissociating a direct relationship between pain and actual structural damage;

Concept 2 (“Thoughts, emotions and experiences affect pain’)—this concept acknowledges the multifactorial nature of pain;

Concept 3 (“I can retrain my overactive pain system”)—this concept emphasizes the neuroplastic nature of pain and that for those with chronic pain, this overly protective mechanism can be reduced.

A score of 1 was given for each theme found on each website (range 0–3). A trial run was performed with all assessors present to develop a standardized baseline for each theme, increasing the objectivity of the tool and reducing the risk of cognitive bias.

2.6. Data analysis

To analyze the readability scores of each source as a function of the country, individual scores, boxplots, and kernel density estimates of the readability scores are presented. Levene test for heterogeneity, mean scores, and standard deviations were also reported. All hypothesis testing was considered statistically significant at an alpha level <0.05. Due to the small sample size and the difference in each of the groups, we used a Kruskal–Wallis nonparametric test to compare readability scores by country. We then compared the number of JAMA benchmark criteria found in each source, the proportion of sources certified with the HONcode seal, and the number of core concepts of pain science education found in each of the sources as a function of the country. For these analyses, we reported absolute counts and percentages. To compare the counts of each category as a function of the country, we used χ2 tests. We also reported absolute counts and percentages of JAMA benchmark criteria met, HONcode seal–certified sources, and the core concepts of pain science education from all the web sources pooled together, irrespective of the country. All analyses were conducted using Jeffrey Amazing Statistics Program 0.16.3 version (JASP) software.37

3. Results

We identified 120 Google websites, 301 Australian government websites, and 20 Mexican government websites. Of these sources, 71 Google websites (57 in English, 14 in Spanish, none in Nepali), and 15 Government websites (14 Australian, 1 Mexican, and none in Nepali) met the inclusion criteria (Fig. 1).

Figure 1.:

Flow diagram for the inclusion and exclusion of results from Google and Government search engines. ACT, Australian Capital Territory; AUS, Australian Federal Government; NSW, New South Wales; NT, Northern Territory; QLD, Queensland; SA, South Australia; TAS, Tasmania; VIC, Victoria.

3.1. Readability

For websites searched using chronic pain, the mean readability scores for the 3 countries were similar: Australia (M = 52.62, SD = 15.44), Nepal (M = 54.07, SD = 10.05), and Mexico (M = 55.38, SD = 5.17) (Fig. 2, panel A). The readability scores for websites in Mexico were less variable as opposed to those for other countries. Levene test suggests no heterogeneity of variance, F(3, 39) = 3.09, P = 0.06. A Kruskal–Wallis test showed no differences in readability scores of each website as a function of country, H (2) = 0.07, P = 0.97. Readability scores fell into the “fairly difficult to read” category (50–60), regardless of the country.

Figure 2.:

Readability ease scores (a score of 60–90 was considered to have adequate readability). Individual scores, boxplots, and kernel density estimates of the obtained readability scores by country. (A and B) The readability of websites by keyword. (C) The readability of Australian government health websites pooled for “chronic pain” and “persistent pain.” AU, Australia; Mex, Mexico.

For the persistent pain search, Australia's (M = 56.35, SD = 11.96), Nepal’s (M = 57.66, SD = 9.82), and Mexico's (M = 55.89, SD = 5) search results all had mean scores that were very similar (Fig. 2, panel B). Again, we found no evidence for homogeneity of variance, F (2, 26) = 0.43, P = 0.65, or differences between countries (H2 = 0.49, P = 0.78). For the Australian search of government databases, there were 14 websites including (M = 51.91 with a SD = 11.37) 1 from Mexico (41.31 readability score) and none for Nepal (Fig. 2, panel C). The rank order of websites according to readability is given in Supplementary file 1 (available at https://links.lww.com/PR9/A197).

3.2. Quality of content 3.2.1. Journal of the American Medical Association benchmark criteria

We found no significant differences in the number of JAMA benchmark criteria for website credibility reported among the countries websites when searched for chronic pain (X2 (8) = 12.84, P = 0.12) and persistent pain (X2 (8) = 11.08, P = 0.20) keywords (Table 1). Only 26.2% websites met the 4 areas of JAMA benchmark criteria when searching chronic pain.Figure 3 panel A shows the number of JAMA benchmark criteria met by all the sources irrespective of country. Disclosure and currency were more likely found than attribution and authorship (Fig. 3, panel A). For persistent pain, only 10.3% websites met 4 areas of JAMA benchmark criteria with only 1 website from all 3 countries having the 4 areas of JAMA benchmark criteria. Again, disclosure and currency were the most common (Fig. 3). For government websites from Australia 42.8% had 3, and 57.1% had 4 JAMA benchmark criteria. The single Mexican government website had 3 criteria. Attribution was the least met JAMA benchmark criteria on government websites, whereas the other 3 were met 100% of the time.

Table 1 - Number of Journal of American Medical Association benchmark criteria met by the information sources per country. Keyword Chronic pain AU Mexico Nepal χ2 (df) P No. of JAMA criteria Total 0 4.76% (2) 0% (0) 22.22% (2) 0% (0) 12.84 0.12 1 7.14% (3) 0% (0) 11.11% (1) 12.5% (2) −8 2 28.57% (12) 41.18% (7) 11.11% (1) 25% (4) 3 33.33% (14) 41.18% (7) 22.22% (2) 31.25% (5) 4 26.19% (11) 17.65% (3) 33.33% (3) 31.25% (5) Total 42 17 9 16 Keyword Persistent pain 0 17.24% (5) 0% (0) 60% (3) 15.38% (2) 11.08 0.2 1 24.14% (7) 27.27% (3) 0% (0) 33.77% (4) −8 2 20.69 (6) 27.27% (3) 0% (0) 23.08% (3) 3 27.59% (8) 36.36% (4) 20% (1) 23.08% (3) 4 10.34% (3) 9.09% (1) 20% (1) 7.69% (1) Total 29 11 5 13

AU, Australia; JAMA, Journal of American Medical Association.

Figure 3.:

Credibility and accuracy analyses. (A) JAMA Benchmark Criteria analysis of all studied sources. Absolute counts are reported inside parenthesis. (B) Percentage of HONcode-certified and HONcode-noncertified websites as a function of country for both keywords. (C) Percentage of core concepts of pain of all studied sources. Absolute counts are reported inside parenthesis. HONcode, health on the net code.

3.2.2. Health on the net code

For all countries, the majority of the included websites were not HONcode certified, with Mexico having the highest proportion of noncertified sources (88.9%). There were no differences in the percentages of certification as a function of the country, X2 (2) = 1.98, P = 0.37 (Fig. 3, panel B). When government sources were analyzed, 10 of 14 (71.4) of the Australian government sources were not certified. The single source of the Mexican government was also uncertified.

3.3. Core concepts of pain science education

We found no significant differences between the counts of the core concepts of pain as a function of the country for both keyword searches: chronic pain (X2 (8) = 12.84, P = 0.12) and persistent pain (X2 (6) = 9.24, P = 0.16) (Table 2). Only 11.9% of websites provided all 3 core concepts of pain science education for all 3 countries with 17.6% of Australian web and 12.5% Nepal websites providing all 3 concepts. None of the websites from Mexico reported all 3 pain science education concepts (Table 2). Figure 3, panel C shows the percentage of the core concepts of pain found in all sources irrespective of country. For chronic pain, we found that the 2 most common concepts were concept 2 (thoughts, emotions, and experiences affect pain; 73.8%) and concept 3 (I can retrain my overactive pain system; 73.8%), in comparison with concept 1 (pain does not mean my body is damaged) (Fig. 3, panel C). By contrast, persistent pain returned more pages with concept 1 (58.6%) and concept 2 (51.7%) rather than concept 3 (34.4%) (Fig. 3, panel C). Government websites in Australia 71.4% webpages with all 3 core concepts of pain science education. None of the core concepts of pain science education was provided in the Mexican government website.

Table 2 - Number of core concepts of pain science education found in the information sources per country. Keyword Total AU México Nepal χ2 (df) P Chronic pain  0 16.67% (7) 11.76% (2) 33.33% (3) 12.50% (2) 12.84 (8) 0.12  1 30.95% (13) 29.41% (5) 33.33% (3) 31.25% (5)  2 40.48% (17) 41.18% (7) 33.33% (3) 43.75% (7)  3 11.90% (5) 17.65% (3) 0% (0) 12.5% (2) Total 42 17 9 16 Persistent pain  0 31.03% (9) 18.18% (2) 80% (4) 23.07% (3) 9.24 (6) 0.16  1 20.70% (6) 18.18% (2) 20% (1) 23.07% (3)  2 20.70% (6) 18.18% (2) 0% (0) 30.77% (4)  3 27.59% (8) 45.45% (5) 0% (0) 23.07% (3) Total 29 11 5 13
4. Discussion

In this first investigation of the readability, credibility, and accuracy of online chronic pain information in 3 different countries and continents, we found that websites on chronic pain retrieved from Google search generally have low readability, mixed credibility, and variable accuracy of pain science information. Moreover, there were no significant between-country differences in online information on chronic pain regarding readability, credibility, and accuracy. For government websites, the Australian government websites had low readability but were credible, and the majority provided all 3 core concepts in pain science education. The single Mexican government website had low readability without any core concepts but was credible.

We found that regardless of the country, Google-based websites and government websites on chronic pain were in the “fairly difficult to read” category, which means the content is accessible to those aged 15 to 17 years and grades 10 to 12 or greater (Fig. 1). This does not meet the recommendations of the National Institute of Health, American Medical Association, and Australian state government recommendations that health information should be delivered at a reading age of 12 to 14 years, equivalent to grades 6 to 8.18,26,41 It was not surprising to find that of the websites assessed, most were fairly difficult to read given that previous studies report similar findings,12,17,24 which suggests that online health information is currently not readily accessible. Our finding of poor readability of online pain information is concerning because this is a significant barrier to providing health information that could help people challenged by chronic pain.

Only 24% of websites on chronic pain showed evidence of good credibility by meeting at least 3 of the 4 JAMA benchmarks. A score greater than 3 indicates a high-quality credible source.32,36 Authorship and attribution were reported on few sites (less than 50% average), providing little detail about who wrote the information or cited references for published sources. Lack of authorship and attribution for websites providing information is a common finding.23,24 Although authorship was reported in 100% of government websites (Australia and Mexico), attribution was found in approximately half (53%) of the websites. It was surprising to find that government pages on chronic pain did not consistently provide attribution or references to content, which would be consistent with evidence-based reporting of health information. It is crucial for health-based websites to document authorship and attribution to the original source to help individuals access the original information so that they can make informed decisions about their health.

Another unexpected finding was that most websites found through the Google search and on the government health websites were not HONcode certified. The HONcode certification indicates that a third party has evaluated and confirmed the credibility of information published. A lack of HONcode registration might put into question the credibility of a health-oriented website. Although currently not a requirement, a HONcode seal is one of the few ways to insure the quality and credibility of online content and may reduce misinformation available to the general public.6 Of note, the absence of a HONcode certification does not imply that a website lacks credibility.

Interestingly, few Google-based websites provided all 3 of the core concepts on pain that have been identified as being important to people challenged by chronic pain. Majority of Australian government websites provide all 3 core concepts, whereas the single Mexican government website had none of the core concepts in pain science education. Considering that in Mexico, almost one-quarter of the general population experiences chronic pain,9 this result highlights the need for more sources with quality pain information provided by the Mexican government to the general population.

The first core concept, “pain does not mean my body is damaged,” was present higher on Government websites (73%) than on Google websites (∼53%). This concept is an important message in pain science education because it can assist in alleviating fears that hurt equals harm and aid in reducing pain as a barrier for participating in daily activities. The second concept, “thoughts, emotions, and experiences affect pain” was present in government websites more (73%) than those found through the Google search (63%). This concept highlights that pain is complex and multifactorial, and recognition of this in chronic pain science education is helpful in explaining how pain can persist or worsen due to experiences other than somatic injury. The third concept, “I can retrain my overprotective pain system” asserts that pain is a protective mechanism, but one that can become overprotective (sensitization), which helps to explain why pain persists after normal healing times and that the pain system is not broken but needs to and can be reset. Although this concept is important to recognize that change is possible, it was the least reported concept on Google websites.

4.1. Strengths and limitations

This study has several strengths. First, this is only study we are aware of to evaluate the readability, credibility, and accuracy of online information on chronic pain across 3 continents with significantly different economic levels. Next, we actively attempted to minimize the risk of bias by different members of the research team independently extracting and reviewing the websites. Finally, we used multiple established tools to evaluate the websites comprehensively.

Despite the strengths, this study also has limitations. First, we chose to use only the Google search engine. Although 52% of the world population uses Google, in some Asian countries like China and Korea, Google is far behind in the search market share.39 Therefore, the findings may not be generalizable to countries that do not use Google as a primary search engine. Second, we included the first 20 websites in Google assuming that most consumers searching for online information would not look past the first 20 results (often the first page of results).37 Thus, it is possible that websites with credible, accurate, and accessible pain information may have been excluded. Third, although we assessed the accuracy of information through our chosen themes, we did not assess the extent to which inaccurate information was present. This is an important consideration because a website may be accurate based on our criteria, but it may also contain inaccurate information about pain that does not fall into 1 of the 3 main categories we selected. Fourth, we recognize that the 3 core pain concepts used may not be a comprehensive representation of all core pain concepts. Although we based the core concepts on 3 valuable aspects of pain science education as seen by people challenged by pain,22 this does not necessary mean that these are only the most important ones.34,44 Finally, given that online information can change quickly, the accuracy of these data may have a short life. Therefore, the data should be considered in light of ever-evolving online content.

4.2. Implications

The main implication of this study is that health organizations and government should aim to make information about chronic pain available and accessible. Concurrently, our data highlight the need for people to critically review the available online information, looking for important indicators such as the HON-code seal to trust a source. More generally, government, clinicians, and researchers should strive to ensure that information, ie, published online is (1) at an appropriate reading level, (2) based upon the most up-to-date and critically appraised evidence, and (3) includes documentation that supports the credibility of information (ie, authorship, attribution, and disclosure currency).

As Australian government sites provide credible and reliable information on chronic pain, it would be ideal for Mexican and Nepali government websites to develop in a similar way, with better readability, to ensure delivering quality information to their people. Given that only 1 Mexican government website discussed chronic pain and not finding a single website on Nepalese government websites, there is an urgent need to create readable, credible, and accurate chronic pain information. Thus, this study points to a major opportunity for Mexican and Nepalese governments and health organizations to develop information on chronic pain in the local language at an appropriate reading level first and then to create quantifiable tools to assess the readability and credibility of the developed information to ensure accessibility.

5. Conclusion

We found that websites, both through Google and on government health sites, had low readability and thus pose a barrier to accessing quality health information about pain. The accuracy and credibility of websites about chronic pain were mixed, with some providing good information but inconsistent across the board. The Australian government health websites were credible, and most provided all 3 core concepts in pain science education. Thus, these could be used as examples of how Mexican and Nepali government websites could be developed to deliver quality information to their people where education levels and health literacy are suboptimal. Given that online information about pain is likely to continue to be a first-line source, these data suggest that it is important to improve that information.

Disclosures

S.S. is supported by the John J. Bonica Postdoctoral Fellowship from the International Association for the Study of Pain. The remaining authors have no conflicts of interest to declare.

Appendix A. Supplemental digital content

Supplemental digital content associated with this article can be found online at https://links.lww.com/PR9/A197.

Acknowledgements

The authors thank Lucy Edwards, Matthew Gleeson, Jack Nuske, Joanna Piscitello-Gangi, and Dean Zalunardo for their assistance in collecting data in Australia.

Author contributors: D. S. Kennedy, J. W. Pate, M. O'Keeffe, and E. O'Hagan contributed to conceptualization of the study. D. S. Kennedy, R. Basnet, D. R. Mendez, I. Lugo-González contributed to data curation and investigation. D. S. Kennedy, R. Basnet, D. R. Mendez contributed