Figures
Abstract
Background
There is no special instrument to measure skills-based health literacy where it concerns infectious respiratory diseases. This study aimed to explore and evaluate a new skills-based instrument on health literacy regarding respiratory infectious diseases.
Methods
This instrument was designed to measure not only an individual’s reading and numeracy ability, but also their oral communication ability and their ability to use the internet to seek information. Sixteen stimuli materials were selected to enable measurement of the skills, which were sourced from the WHO, China CDC, and Chinese Center of Health Education. The information involved the distribution of epidemics, immunization programs, early symptoms, means of disease prevention, individual’s preventative behavior, use of medications and thermometers, treatment plans and the location of hospitals. Multi-stage stratified cluster sampling was employed to collect participants. Psychometric properties were used to evaluate the reliability and validity of the instrument.
Results
The overall degree of difficulty and discrimination of the instrument were 0.693 and 0.482 respectively. The instrument demonstrated good internal consistency reliability with a Cronbach’s alpha of 0.864. As for validity, six factors were extracted from 30 items, which together explained 47.3% of the instrument’s variance. And based on confirmatory factor analysis, the items were grouped into five subscales representing prose, document, quantitative, oral and internet based information seeking skills (χ2 = 9.200, P>0.05, GFI = 0.998, TLI = 0.988, AGFI = 0.992, RMSEA = 0.028).
Citation: Sun X, Chen J, Shi Y, Zeng Q, Wei N, Xie R, et al. (2013) Measuring Health Literacy Regarding Infectious Respiratory Diseases: A New Skills-Based Instrument. PLoS ONE 8(5): e64153. https://doi.org/10.1371/journal.pone.0064153
Editor: Hamid Reza Baradaran, Tehran University of Medical Sciences, Islamic Republic of Iran
Received: November 27, 2012; Accepted: April 9, 2013; Published: May 28, 2013
Copyright: © 2013 Sun et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This study was funded by China-US Collaborative Program on Emerging and Re-emerging Infectious Diseases (5U2GGHH000018-02). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
In recent years, the viruses SARS, influenza (H5N1 and H1N1) have broken out, and drug-resistant tuberculosis has made a resurgence, attracting public attention to infectious respiratory diseases. The most important feature of these diseases is that they can be prevented, controlled and cured. Correct preventive measures and health behavior can stop their spread to a very large extent. So health education and health promotion is both efficient and important in this field. Low health literacy can restrict and impact on the efficiency of health education, adoption of preventative measures and health behaviors.
After more than 20 years’ development, health literacy has several different definitions. The Ratzan and Parker (2000) definition has been widely used. As stated in Healthy People 2010, health literacy was defined as “The degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions.” [1] Berkman in 2010 gave some suggestions for minor modifications to the definition: “The degree to which individuals can obtain, process, understand, and communicate about health-related information needed to make informed health decisions.” [2].
Different definitions adopt different measurement indicators and assessment systems. The most common measurement and assessment tools include Rapid Estimate of Adult Literacy in Medicine (REALM) [3], Test of Functional Health Literacy in Adults (TOFHLA) [4], and the Department of Education’s 2003 National Assessment of Adult Literacy (NAAL) [5]. REALM was developed as a quick screening tool to assist physicians in identifying patients with limited reading skills, and to help estimate patients’ reading levels. It contains 66 common medical terms and nouns representing body parts and diseases. REALM emphasizes language capability, which reflects people’s ability to recognize terms, but ignoring their understanding and skill in health literacy. TOFHLA measured patients’ numeracy and reading levels in a healthcare environment, and assessed their health literacy. The NAAL scale included information about medicines and prevention and guidance in health care, aimed at assessing subjects’ position and understanding of health-related information and skills, but it did not take into account respondents’ ability to communicate with doctors, or their understanding of medical terms. In 2009, McCormack designed a new skills-based instrument, titled the Health Literacy Skills Instrument (HLSI) to measure health literacy more comprehensively. It measured not only print literacy but also oral and internet-based information seeking skills. [6].
China carried out a National Health Literacy Survey in 2007 to understand Chinese residents’ level of basic health commonsense and skills. [7] A composite index and percentages were adopted to assess health literacy. However, the project mainly indicated residents’ knowledge levels and health-related behaviors, and did not sufficiently reflect their skills.
All these instruments involved all respects of health. Some studies tried to measure health literacy in the field of infectious diseases, but they were knowledge or perception-based. [8], [9] There is no special instrument to measure skills-based health literacy where it concerns infectious respiratory diseases. Given that health literacy assessment in certain fields would be more particular, comprehensive and specific, our research aimed only to assess health literacy as it relates to respiratory infectious diseases.
This study was supported by the China-US Collaborative Program which focused on emerging and re-emerging infectious diseases. Due to the project’s background, this study aimed to develop a skills-based instrument to measure the health literacy concerned only with infectious respiratory diseases. The development of the instrument and its reliability and validity are described in this article.
Compared with other health literacy measurement tools mentioned above, this instrument used audiovisual stimuli to measure oral literacy, and computer and internet questions to measure internet-based information seeking skills, other than print stimuli to measure print literacy, so as to explore the feasibility and reliability of a skills-based measuring method. Additionally, in terms of content domains, it reflects health related issues for disease prevention, health care maintenance and treatment, and the available health system utilization, which can be used to assess people’s health literacy not only in a clinical setting but also in a public environment. This kind of assessment can provide directions or methods of how to conduct patient education and public health education on the treatment and prevention of infectious respiratory diseases.
Methods
Instrument Development
The questionnaire is a self-administered test, consisting mainly of single choice questions. It takes 20 to 50 minutes to finish. The average finishing time is 30 minutes. The first part of the questionnaire establishes the social demographic characteristics of the participant. It measures age, gender, ethnicity, household registration status, marital status, education, occupation and income.
The main part of the questionnaire measures and assesses health literacy. We used Ratzan and Parker’s (2000) definition of health literacy: “The degree to which individuals can obtain, process, understand, and communicate about health-related information needed to make informed health decisions”. [1].
We selected appropriate health information to measure according to the study’s objectives using the following selection criteria:
- It covers all prevention and control information about infectious respiratory diseases among the public, without regional and sex differences.
- It covers all transmission vectors for infectious respiratory diseases related to the general public.
- The health information should be objective, scientific, direct and vivid.
- It should not take more than one minute to read each piece of information.
Sixteen stimuli materials (See Table S1) were included in the questionnaire, sourced from the World Health Organization (WHO), the Chinese Center of Disease Control and Prevention and the Chinese Center of Health Education. Information involved the distribution of epidemics, immunization programs, early symptoms, means of disease prevention, preventative behavior for individuals (for example, washing hands, wearing a face mask, sneezing), use of medications and thermometers, treatment plans and the locations of hospitals. The media forms contained brochures, books, posters, newspapers, website information, a hotline number and items including a medicine box, thermometer and hospital map. We categorized tasks into three skills set areas as follows: print, oral and internet information seeking. Moreover, the ‘print’ skill set included three aspects. The print-prose scale measured the knowledge and skills needed to search, comprehend, and use information from texts that were organized in sentences or paragraphs, while the print-document scale measured from non-continuous texts in various formats. And the print-quantitative scale measured the knowledge and skills needed to identify and perform computations using numbers embedded in printed materials. [5].
The questionnaire consisted of 30 items, aimed to assess individuals’ capacity to obtain, interpret and use information. Tasks were classified as follows: understanding health-related text; interpreting information in the form of pictures, symbols, and maps videos and recordings; interpreting information in the form of videos and recordings; completing computations; applying information to a specific scenario; and utilizing the internet to obtain health information. These selective questions had one correct answer, a correct reply was worth one point, while an incorrect reply scored zero. Therefore, the maximum possible score for this part of the questionnaire was 30 points.
Target Population and Sampling
The author believes that the rising trend of infectious respiratory diseases is probably related to greater population mobility. Due to the accelerating process of urbanization in China, the number of internal migrants in cities is growing. They are living and working in closer confines, with facilities shared between more people. These people aid the outbreak and spread of contagious diseases. Therefore, cities characterized by dense populations and rapid population growth should be the crucial locations for research about health literacy related to infectious respiratory diseases. [10].
Field work in Beijing (The capital of China), Datong (in Shanxi province, North China) and Shenzhen (in Guangdong province, South China) was conducted from May to December 2011. Sample size was calculated by the function n = Z21-a/2P (1 − P)/d2 × deff. The National Health Literacy Survey in 2008 regarding health literacy towards infectious diseases showed the expected percentage was 16% (P = 0.16). [11] The confidence interval was ±10%, d = 0.1*P = 0.016, α = 0.05, Z1-α/2 = 1.96, deff is design effect, defined as 2 due to cluster sampling. Therefore the minimum sample size is 2581. If recovery rates and efficiency rates of the questionnaire are to be both 90%, then the actual sample size should be at least 3186. So the sample size in each of the three cities surveyed would have to be about 1062. In the end, 3222 residents responded to the survey.
Multi-stage sampling was employed to collect data from each city. The target population was first stratified into local and non-local residents, with an equal sample size for each group. Then, based on the principle of balancing samples among factors like age and occupation, cluster sampling was conducted in six places where local residents gather (including communities, government organizations, factories and other institutions) and six places where non-local residents gather (including building sites, hotels, employment medical examination centers and assembly shops).
Ethics Statement
The study protocol was reviewed and approved by the Peking University Institutional Review Board and it was in accordance with Helsinki Declaration. Written informed consent was obtained for all study participants.
Statistical Analysis
Questionnaires with more than 10% of items unanswered were considered ineligible and removed before analysis in order to ensure the quality of data. EPI Data 3.0 was used for data double entry and SPSS 13.0 for data analysis. Descriptive statistics were employed to examine demographic characteristics. Discrimination index (D) was calculated by the score of the top 27% of participants (PH) minus that of the bottom 27% (PL). Scale and factor analyses were conducted to verify the scale’s reliability and construct validity. Confirmatory factor analysis of a half randomly selected sample was implemented by the Statistical Analysis System (SAS 6.12). Covariance Analysis of Linear Structural Equations (CALIS) was performed to examine the model. In addition, maximum likelihood estimation was used to appraise the parameters with a covariance matrix. Participants were classified into three categories based on their health literacy levels: proficient, basic, and below basic. Cut-points were determined through receiver operating characteristic (ROC) analyses based on their educational level. Generally, the α level was set at 0.05. Initial eigenvalue >1 was the criterion in the factor analysis.
Results
Participant Characteristics
Table 1 shows participants’ mean health literacy scores by characteristics. Health literacy score increased as education levels and income rose, but declined along with age. There was a significant difference in scores across three categories of occupation. The first was between students and scientific and technical workers/teachers/doctors, the second included office workers, service providers, general workers and others, and the third included the retired and farmers.
Difficulty, Discrimination, Reliability and Validity of the Instrument
The average percentage of correct answer was 69.3%, that is, the overall degree of difficulty (P) was 0.693. The overall degree of discrimination was 0.482. The instrument had good internal consistency, while the Cronbach’s alpha coefficient was 0.863 (See Table 2). By analyzing these together with the exploratory factors, six factors were extracted from 30 items (the communality of each item is shown in Table 3), which explained 47.3% of total variance. The six factors were named f1: internet, f2: oral, f3: quantitative, f4: document, f5: prose and f6: chart. As f3 and f6 were both about numeracy, and f6 had only two items, they were combined. So the items were regrouped into five subscales representing print-prose, print-document, print-quantitative, oral and internet-based information seeking skills, and each subscale had good internal consistency (See Table 3). The correlation between subscales is shown in Table 4. Based on confirmatory factor analysis (See Figure 1), the items were grouped into five subscales representing prose, document, quantitative, oral and internet based information seeking skills very well (χ2 = 9.200, P>0.05, GFI = 0.998, TLI = 0.988, AGFI = 0.992, RMSEA = 0.028). The latent variable “print” was extracted from “prose”, “document” and “quantitative”, that measured the health literacy to search, comprehend, and use information from print media. The latent variable “multimedia” was extracted from “oral” and “internet”, that measured the health literacy to search, comprehend, and use information from multimedia.
Note: “e1” to “e5” were the residuals of variables.
ROC Analysis
An ROC curve was used to analyze the classification of health literacy levels. Comparisons by education level indicated a cut-point of 24 for differentiating respondents with college education versus no college education (sensitivity = 0.64, specificity = 0.71), and a cut-point of 21 for differentiating respondents with an education of at least high school versus less than high school (sensitivity = 0.70, specificity = 0.69). Based on these analyses, we classified participants into three groups: proficient literacy (score >24), basic literacy (score of 21–24), and below basic literacy (score <21). In the sample, 33.5% of participants were found to have proficient literacy, 24.5% basic literacy, and 42.0% below basic literacy.
Simplified Measurement Tool
Based on these results, a simplified version of the infectious respiratory disease health literacy measurement tool has been developed. One representative item from the prose (P3), quantitative (Q3), oral (O3) and internet (I5) category, and two items from the document (D4, D6) category were selected. The particular items selected were all answered with a moderate accuracy rate (range: 61.0%–74.6%, average: 68.0%) with a good discrimination (range of D: 0.488–0.760, average: 0.542). The items’ importance and practical value for the prevention of infectious respiratory diseases was also taken into account. Therefore, the simplified measurement tool had six items from six stimuli. The Cronbach’s alpha coefficient was 0.571. The simplified version with six items significantly correlated with the full version (r = 0.843, P<0.001).
Discussion
The methods of measuring health literacy have improved with the development of its definition. The earliest prevalent measuring tool of health literacy was the REALM [3], which merely measured adult patients’ ability to read and spell common medical terms, expressing the name of body parts and diseases. As it is not able to comprehensively evaluate patients’ abilities, this measuring tool is no longer fundamental to interpretations of health literacy. Likewise, the 2008 investigation into citizens’ health literacy in China has similar limitations. It tested a wide range of basic health knowledge and skills, but did not measure subjects’ ability to learn and apply new skills [11]. Another commonly used measuring tool named TOFHLA was the first way of measuring functional health literacy. It measures subjects’ efficiency in learning, thinking and processing health information, and evaluates their ability in both counting and reading comprehension. RTI International has explored a measurement tool for health literacy based on skills [6], which expands the measurement to three dimensions, namely print, oral and network information seeking. Interpreting health literacy reasonably and comprehensively, this measuring tool has been chosen as the initial chief source of this study design.
As a measurement tool for functional health literacy, this study’s primary feature is a questionnaire. The answers to all questions, included in corresponding health information materials, can only be obtained after the information is comprehended and processed. This is an implementation of the concept that health literacy is the individual’s ability in obtaining, comprehending and processing basic health information or services, and making right decisions related to health. Using this tool, subjects’ health literacy level will not depend on their previous health knowledge, but rather, on their ability to interpret and understand health information on the spot.
Due to the background of the project’s fund, the information materials used to measure health literacy in this study mainly come from the field of respiratory infectious disease prevention and treatment, causing field limitations in measured health literacy, but increasing the value of information on disease control in this specific field. Comprehensive measurement of health literacy can be achieved by continuing this study while expanding the range of health information materials to other health-related fields.
The instrument has moderate degree of difficulty, good discrimination, good reliability and validity. Using this instrument, we supposed that 58% of Chinese adults had basic literacy on respiratory infectious diseases or better. This is much higher than the data (16%) obtained from the National Health Literacy Survey in 2008 regarding health literacy towards infectious diseases [11]. However, the two studies tested very different aspects of health literacy, using different methods and evaluation criteria.
In accordance with the underlying theory of RTI, this study categorizes health information into five classes, namely reading, picture comprehension, listening, computing and network information searching. This way it can imitate all kinds of information media accessible in daily life, and evaluate subjects’ multi-aspect ability to process information. In this research, the difference in accuracy rates between reading and other skills indicates that the measurement result of reading skills alone cannot completely and objectively reflect subjects’ ability to process all kinds of health information in real life. This substantiates the necessity to measure health literacy from many dimensions. The health information materials used in this study are all officially circulated promotional materials.
Very little time is available for the measurement of health literacy in the health promotion and medical services environment, but health literacy measurement in across some districts and populations is still necessary. Therefore, simplifying the health literacy measurement tool will be necessary in the future. Currently, the simplified versions of many universal measurement tools have been developed so as to adapt for their practical requirements. Taking S-TOFHLA for example, the 17 calculation problems and three reading comprehension questions in the original version have been reduced to four calculation problems and two reading comprehension questions. This has served to enhance the tool’s practical applicability by shortening the test’s duration from around 22 minutes to around 12 minutes [12]. In this study, 30 items were reduced to only six items. The duration of the test was shortened from 20–50 minutes to four-to-10 minutes. Subjects’ decreased tolerance for taking long tests may seriously influence the quality of measurement; therefore, the simplified version may yield more believable results other than the advantage of an expanded applicable scope (conducting quick assessment of patients’ literacy in a medical environment).
Because the sampling method was not strictly randomized, the sample, whose demographic characteristics may be different from that of the total population, cannot perfectly represent the health literacy of all citizens all in mainland China. Limited by the project’s background, this study only measures health literacy where it concerns infectious respiratory diseases. A comprehensive measurement concerning other diseases still needs to be developed. In the pilot survey, wireless internet was used to test internet-based information seeking skills. Due to the low speed of wireless internet, the survey took 10–15 minutes longer to complete than otherwise expected. This affected participants’ enthusiasm for completing the test. Therefore, in the formal study, we used a series of questions to test the ability of internet-based information seeking. In future study, if possible, the test should be conducted online, by then the problem will be solved naturally.
Supporting Information
Table S1.
Health literacy skill area by task and by health domain.
https://doi.org/10.1371/journal.pone.0064153.s001
(DOC)
Acknowledgments
We would like to acknowledge the CDC China Office and RTI international for their support and valuable comments. We also thank all health care workers in Beijing, Shenzhen and Datong for their help in data collection.
Author Contributions
Conceived and designed the experiments: XYS CC WJD. Performed the experiments: XYS YHS NFW RQX CC WJD. Analyzed the data: XYS JC QQZ. Contributed reagents/materials/analysis tools: NFW RQX WJD. Wrote the paper: XYS JC QQZ. Obtained funding: NFW RQX WJD. Provided supervision: CC NFW.
References
- 1.
Ratzan SC, Parker RM (2000) Introduction, in National Library of Medicine current bibliographies in medicine: Health literacy, M.X. C. R. Selden, S. C. Ratzan,& R. M. Parker Editor. Bethesda: NLM Pub.
- 2.
Berkman ND, Davis TC, McCormack L (2010) Health Literacy: What Is It? J Health Commun (Suppl 2): 9–19.
- 3. Davis TC, Williams MV, Marin E, Parker RM, Glass J (2002) Health literacy and cancer communication. CA Cancer J Clin 52: 134–49.
- 4. Parker RM, Baker DW, Williams MV, Nurss JR (1995) The test of functional health literacy in adults: a new instrument for measuring patients’ literacy skills. J Gen Intern Med 10(10): 537–41.
- 5.
Kutner M, Greenberg E, Jin Y, Paulsen C (2006) The health literacy of America’s adults: Results from the 2003 National Assessment of Adult Literacy (NCES 2006–483). Washington, DC: National Center for Education Statistics.
- 6.
McCormack L, Bann C, Squiers L, Berkman ND, Squire C, et al.. (2010) Measuring Health Literacy: A Pilot Study of a New Skills-Based Instrument. Journal of Health Communication (suppl 2): 51–71.
- 7. Zheng YD, Shi JH, Cao RX (2010) Confirmative factor analysis in the health literacy questionnaire and its applications in Chinese residents. Beijing Da Xue Xue Ba 42(3): 314–7.
- 8. Wu SS, Yang P, Li HY, Ma CN, Zhang Y, et al. (2012) Study on the index system in evaluating the adult health literacy situation related to infectious disease in Beijing. Zhonghua Liu Xing Bing Xue Za Zhi 33(3): 301–4.
- 9. Pappas G, Siozopoulou V, Saplaoura K, Vasiliou A, Christou L, et al. (2007) Health literacy in the field of infectious diseases: The paradigm of brucellosis. Journal of Infection 54(1): 40–45.
- 10. Barrera-Anderson T, Olives T, Larsen K, Pereira A (2007) Health literacy in native Spanish-speaking immigrants in Minneapolis. Travel Medicine and Infectious Disease 5(6): 410.
- 11. Wang P, Mao Q, Tao M, Tian X, Li Y, et al. (2010) Survey on the status of health literacy of Chinese residents in 2008. Chinese Journal of Health Education 26(4): 243–246 (in Chinese)..
- 12. Baker DW, Williams MV, Parker RM, Gazmararian JA, Nurss J (1999) Development of a brief test to measure functional health literacy. Patient Educ Couns 38(1): 33–42.