Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Evaluation of Risk Perception and Risk-Comparison Information Regarding Dietary Radionuclides after the 2011 Fukushima Nuclear Power Plant Accident

  • Michio Murakami ,

    michio@fmu.ac.jp

    Affiliation Department of Health Risk Communication, Fukushima Medical University School of Medicine, 1 Hikarigaoka, Fukushima City, 960-1295, Japan

  • Jun Nakatani,

    Affiliation Department of Urban Engineering, Graduate School of Engineering, The University of Tokyo, 7-3-1 Hongo, Bunkyo, Tokyo, 113-8656, Japan

  • Taikan Oki

    Affiliation Institute of Industrial Science, The University of Tokyo, 4-6-1 Komaba, Meguro, Tokyo, 153-8505, Japan

Correction

9 Feb 2017: Murakami M, Nakatani J, Oki T (2017) Correction: Evaluation of Risk Perception and Risk-Comparison Information Regarding Dietary Radionuclides after the 2011 Fukushima Nuclear Power Plant Accident. PLOS ONE 12(2): e0172248. https://doi.org/10.1371/journal.pone.0172248 View correction

Abstract

In the wake of the 2011 Fukushima Daiichi Nuclear Power Station accident, to facilitate evidence-based risk communication we need to understand radiation risk perception and the effectiveness of risk-comparison information. We measured and characterized perceptions of dread risks and unknown risks regarding dietary radionuclides in residents of Fukushima, Tokyo, and Osaka to identify the primary factors among location, evacuation experience, gender, age, employment status, absence/presence of spouse, children and grandchildren, educational background, humanities/science courses, smoking habits, and various types of trustworthy information sources. We then evaluated the effects of these factors and risk-comparison information on multiple outcomes, including subjective and objective understanding, perceived magnitude of risk, perceived accuracy of information, backlash against information, and risk acceptance. We also assessed how risk-comparison information affected these multiple outcomes for people with high risk perception. Online questionnaires were completed by people (n = 9249) aged from 20 to 69 years in the three prefectures approximately 5 years after the accident. We gave each participant one of 15 combinations of numerical risk data and risk-comparison information, including information on standards, smoking-associated risk, and cancer risk, in accordance with Covello’s guidelines. Dread-risk perception among Fukushima residents with no experience of evacuation was much lower than that in Osaka residents, whereas evacuees had strikingly higher dread-risk perception, irrespective of whether their evacuation had been compulsory or voluntary. We identified location (distance from the nuclear power station), evacuation experience, and trust of central government as primary factors. Location (including evacuation experience) and trust of central government were significantly associated with the multiple outcomes above. Only information on “cancer risk from radiation and smoking risk” enhanced both subjective and objective understanding without diminishing trust in all participants and in the high dread-risk perception group; use of other risk-comparison information could lead the public to overestimate risk.

Introduction

The Fukushima Daiichi Nuclear Power Station accident after the Great East Japan Earthquake on 11 March 2011 spread radionuclides and contaminated foods and drinking water. Although the additional effective doses of radionuclides from dietary intakes were limited (e.g., adults in Fukushima City, 0.062 mSv in the first year [1]) and minor in comparison with additional inhalation and external exposures [2,3] or natural exposure [4], public anxiety about radionuclides in the diet has not been dispelled. Consequently, some people in not only Fukushima but also other prefectures a long way away have refused to eat foods from Fukushima and surrounding areas [5,6], resulting in continuing economic damage to farmers and fishery workers. A decline in subjective well-being after the 2011 accident has also been found in people with strong levels of anxiety about food safety [7]. Similarly, high levels of psychological distress have been observed in those people who believe that radiation exposure very likely has adverse health effects [8].

Anxiety about radionuclides can be explained by the characteristics of human perception of radiation risks and distrust of government and experts. Slovic [9] mentioned that risk perception can be divided into two psychological dimensions, namely “dread risk” and “unknown risk”. Members of the public, including the Japanese public, perceive radiation risk and nuclear power plant accidents as high dread risks [9,10]. Although various risk-communication activities were implemented after the 2011 accident to enhance public understanding of radiation risks, controversial “experts’” opinions on radiation risks may have led to confusion and increased distrust of government and experts [11].

The US National Research Council (NRC) [12] stated that “risk communication is an interactive process of exchange of information and opinion among individuals, groups, and institutions,” and that “risk communication is successful to the extent that it raises the level of understanding of relevant issues or actions and satisfies those involved that they are adequately informed within the limits of available knowledge”. Fischhoff [13] identified seven stages of risk communication and pointed out that it is important to start risk communication by informing recipients with numerical data. Hino et al. [14] found empirically that delivering objective data at explanatory meetings helped to reduce anxiety and improve comprehension and satisfaction. The concept of risk, however, demands highly complex numerical skills for communication [15]. Since the 2011 accident, many experts have tried to convey numerical information by using risk-comparison methods (e.g., by comparison with standards, natural and artificial radiation doses, and smoking risk [16,17]). Levels of understanding, perception, and acceptance of risk, or of trust of the information provided, are considered to differ according to the risk-comparison information provided. On the basis of their personal experience, Covello et al. [18] proposed risk-comparison guidelines that ranked people’s acceptance of risk-information sources into five categories, i.e., from (1) most acceptable: comparison with a standard etc. to (5) rarely acceptable: comparison with unrelated risks. However, little quantified evidence is available on the effects and applicability of the guidelines, with the exception of a few studies focusing on chemicals [19,20]; moreover, application of the guidelines to different cultural situations is especially poorly understood [21]. In particular, multiple outcomes, including levels of understanding and risk acceptance in addition to trust of the information provided, should be empirically evaluated to facilitate evidence-based risk communication. Furthermore, because the effects of risk-comparison information likely depend on individual risk perceptions, which are primary factors in decision-making [9,22], analysis of the relationship between effects and risk perception would be useful for providers in choosing their risk-comparison information. This would promote effective risk communication and would help recipients to understand numerical risk information, provided that the providers’ risk communication was fair and justifiable.

The logical steps are therefore to understand the factors involved in radiation risk perception and to then investigate the effects of risk-comparison information on different risk-perception groups by using multiple outcomes, including level of understanding, perception and acceptance of risk, and trust of information. Although there have been advances in our understanding of the general factors involved in risk perception (e.g., gender, age, cultural worldview) [9,23,24], the effects of disaster-related factors (e.g. evacuation experience) on radiation risk perception have not been studied, except in a limited report [8]. Nor has the effect of risk-comparison information on comprehensive outcomes been unraveled well.

This study had two objectives. First, we investigated factors involved in risk perception of dietary radionuclides among residents of Fukushima, Tokyo, and Osaka on the basis of Slovic’s two psychological dimensions [9]. In addition to general individual attributes and trustworthy information sources, we evaluated evacuation experiences associated with the 2011 accident. Second, we assessed the effects of risk-comparison information on level of understanding, perceived magnitude and acceptance of risk, and trust of information. In the case of level of understanding of risk, both subjective and objective understanding were evaluated. We also investigated the relationship between Slovic’s two dimensions of risk perception and the risk-comparison information.

Methods

Participants

Ethical approval for the study was granted by the Fukushima Medical University Ethics Committee (Ethics Committee approval number: 2489). Online questionnaires were completed in December 2015 by members of the public aged from 20 to 69 years in three prefectures, namely Fukushima, Tokyo (~230 km from the Fukushima Daiichi Nuclear Power Station), and Osaka (~580 km). Survey participants in Tokyo and Osaka were selected for consistency with actual composition ratios for gender and age (20s, 30s, 40s, 50s, and 60s). In the case of Fukushima, selection based on composition ratios for gender and age was not applied because of limitations on the number of panelists. The distributions of respondents’ gender and age in Fukushima were as follows (Table 1): men, 53.8%; women, 46.2%: 20s, 9.9%; 30s, 21.0%; 40s, 29.2%; 50s, 25.0%; and 60s, 14.9%. The gender distributions were close to the actual distributions of all residents in Fukushima Prefecture (men, 51.1%; women, 48.9% in September 2015) [25], but there were fewer respondents in their 20s and 60s (20s, 14.0%; 30s, 18.2%; 40s, 20.5%; 50s, 21.9%; 60s, 25.4%). According to the number of types of risk-comparison information provided (see details in “Questionnaires”), participants from Fukushima, Tokyo, and Osaka were divided into four, 15, and nine groups, respectively. Approximately 300 participants were allocated to each group; in total, 9249 people (Fukushima, 1458; Tokyo, 4856; Osaka, 2935) participated in the survey.

The participant pool consisted of members of the general public who had registered as survey panelists with INTAGE Research Inc. INTAGE Research Inc. is one of the biggest survey companies in Japan: at the time of the survey it had 1.32 million panelists. The reliability and advantages of online surveys have been described by ref [26]. The reliability of the panelists was ensured by identifying the panelists through mail-outs to physical addresses. Responses were excluded if the response time was too short, or if there was a discrepancy of gender or age between the survey response and the register information (± 1-year age difference was accepted), or if multiple responses from the same IP address were found. Because participants received reward points that could be exchanged for cash and commercial products after the survey, the participants were motivated to respond to the questionnaires.

Questionnaires

We first asked the participants to respond regarding their individual attributes and radiation risk perceptions (Tables 1 and 2). Individual attributes included gender, age, employment status, absence/presence of spouse, children and grandchildren, educational background, humanities/science courses, smoking habits, and perceived trustworthy information sources. Multiple answers were allowed in the case of trustworthy information sources. Fukushima respondents were also asked their current area of residence, namely Hamadori (the east coast of Fukushima Prefecture), Nakadori (the central region of Fukushima Prefecture), or Aizu (the western mountainous region of Fukushima Prefecture). Questions were also asked regarding evacuation experience in relation to the 2011 accident. There were four choices: “evacuated now,” “evacuated in the past,” “voluntarily evacuated,” and “not evacuated.” Radiation risk perception was assessed by using a 4-point Likert scale (4: strongly agree; 3: agree; 2: disagree; 1: strongly disagree). Neutral points were not provided: participants were required to choose whether they (strongly) agreed or disagreed.

thumbnail
Table 2. Arithmetic mean, standard deviation, and factor pattern matrix for perception of radiation risk, and their interpretation.

KMO: 0.918, P < 0.001 (Bartlett). Bold font: >0.40 or <–0.40. Cronbach’s α: 0.878 (eight items in bold font for factor 1); 0.578 (three items in bold font for factor 2).

https://doi.org/10.1371/journal.pone.0165594.t002

We then provided each participant with one risk information case out of a total of 15 (made up of a variety of combinations of three items of numerical risk data and 10 items of risk-comparison information) (Table 3). The three items of numerical risk data were as follows: A, current dietary radiocesium doses; B, current dietary radiocesium doses and corresponding lifetime cancer mortality rates; and C, current dietary radiocesium doses and corresponding loss of life expectancy (LLE). Increase of cancer mortality at low-doses is unclear and statistically undetectable [27], whereas mortality risk is useful as an ‘indicator’ for regulation- or decision-making [28]. Current dietary doses of radiocesium in the target prefecture were derived by using the market basket method [29] and were 0.0016 mSv/year for Fukushima, 0.0010 mSv/year for Tokyo, and 0.0007 mSv/year for Osaka. The lifetime cancer mortality rates corresponding to 1 year of intake were calculated from the doses given above and a 4%/Sv unit risk of cancer mortality [30] under the concept of radiological protection. Because of differences in perception expressed as frequencies and that expressed as percentages [31,32], these values were provided in terms of both the proportional increase in the number of affected individuals per 100,000 people and percentage values. They were, for Fukushima, 0.0064 out of 100,000 people (0.0000064%); Tokyo, 0.0040 out of 100,000 people (0.0000040%); and Osaka, 0.0028 out of 100,000 people (0.0000028%). The LLE corresponding to 1 year of intake was estimated from the coefficient of LLE per dose (= 5.5 × 10−4 year/mSv), with a dose- and dose-rate effectiveness factor of 2 [33]. The values were, for Fukushima, 27 seconds; Tokyo, 17 seconds; and Osaka, 12 seconds.

thumbnail
Table 3. Risk-comparison information provided, and its explanation.

“+” represents risk-comparison information used in this study.

https://doi.org/10.1371/journal.pone.0165594.t003

The 10 items of information provided for risk-comparison were: 1. No comparison information; 2. Food standard dose (corresponding to Rank 1 in Covello’s guideline [18]); 3. Results for 100-mSv [34] (Rank 1); 4. 1960s dose [35] (Rank 1); 5. Doses in other prefectures [29] (Rank 2); 6. Natural radiation dose [4] (Rank 3); 7. Total cancer mortality rate [36] (Rank 3); 8. Airplane dose [37] (Rank 4); 9. Arsenic risk [30,38,39] (Rank 4); and 10. Smoking risk [17,40] (Rank 5). These items of risk-comparison information were selected because they have often been used since the 2011 accident [16,17]. Out of four cases (A1, A2, A6, and A10), one case of risk data and risk-comparison information was provided for participants in Fukushima. Similarly, one case of risk data and risk-comparison information was provided out of 15 cases (A1 to A6, A8 to A10, B1, B7, B9, B10, C1, C10) for those in Tokyo, and one out of nine cases (A1 to A6, A8 to A10) was provided for those in Osaka.

After the risk data and risk-comparison information had been provided to participants, we asked participants to respond to the following five index questions under the definition that “risk” represented the likelihood or probability of an unfavorable event: (1) “From the above information, do you intuitively understand the level of risk currently posed by dietary radiocesium in your prefecture?” (2) “Do you think that the risks currently posed by dietary radiocesium in your prefecture are large?” (3) “Do you think that the information provided on the level of risk currently posed by dietary radiocesium in your prefecture is accurate?” (4) “How do you feel about the risk currently posed by dietary radiocesium in your prefecture?” and (5) “Compare the level of risk currently posed by dietary radiocesium in your prefecture with the risk of dying in a traffic accident. How do you think the possibility of dying from cancer as a result of 1 year’s current intake of dietary radiocesium in your prefecture compares with the annual possibility of dying in a traffic accident?”

For index questions (1) to (3), 5-point Likert scales were used: a low score represented “not comprehensible,” “small,” or “inaccurate,” whereas a high score represented “comprehensible,” “large,” or “accurate.” For index question (4), four choices were provided: “I am not particularly concerned about it, because there are factors other than radiation (e.g., smoking, heavy drinking, and being underweight)” (hereinafter “do not mind”); “I can accept it if it is less than the standard, because the risk is lower than those from other factors,” (hereinafter “acceptable”); “I cannot accept it even if it is less than the standard, because it could still increase the risk of cancer, even slightly;” and “I cannot evaluate it because there is not enough information.” For index question (5), five choices were provided: “higher than for a traffic accident,” “comparable to that of a traffic accident,” “about 1/10 of that of a traffic accident,” “about 1/100 of that of a traffic accident,” and “about 1/1000 of that of a traffic accident.”

Index questions (1) and (5) were asked to assess subjective understanding and objective understanding, respectively, under the respective concepts of “System I (fast, intuitive, and emotional way of thinking)” and “System II (slower, more deliberative, and more logical way of thinking)” [41,42]. According to statistics on annual traffic death rates [43], the mortality rate from 1 year of intake of dietary radiocesium, as calculated on the basis of linear non-threshold (LNT) theory, was about 1/1000 of the annual traffic death rate. Because LNT theory is used as radiological protection concept and the estimated risk was conservative (i.e. overestimated) [44], the correct answer was “about 1/1000 of that of a traffic accident.” Index question (2) was asked to determine the perceived magnitude of risk. Index question (3) was asked to evaluate trust of the information in terms of the outcomes of “perceived accuracy of information” and “backlash against information.” Index question (4) was derived from a previous report [6] and was used to evaluate the outcome of risk acceptance.

Statistical analyses

To evaluate perception of radiation risk, we performed a factor analysis using the maximum likelihood method and Promax rotation. Factors with an eigenvalue of 1 or more were extracted, and factor scores were obtained by using the regression method (Table 2). No floor or ceiling effect was observed.

To evaluate the factor scores for perception of radiation risk, we used a t-test for two groups and an analysis of variance (ANOVA) for more than two groups. As a post hoc test along with ANOVA we used the Tukey-Kramer or Games-Howell test.

To identify the primary factors associated with perception of radiation risk, we performed a multiple regression analysis with the factor scores as the objective variables and individual attributes as the explanatory variables. Dummy variables were created for each individual attribute parameter. Reponses from Fukushima were further divided into two groups on the basis of evacuation experience, i.e. “not evacuated” and “evacuated (including “evacuated now,” “evacuated in the past,” and “voluntarily evacuated”).” A stepwise approach was used to add significant explanatory variables (P < 0.05) and remove non-significant explanatory variables (P > 0.10).

To assess the effects of the primary factors governing perception of radiation risk and of risk-comparison information, we used a multivariate binomial logistic regression analysis with subjective and objective understanding, perceived magnitude of risk, perceived accuracy of information, backlash against information, and risk acceptance as the objective variables, and individual attributes and risk-comparison information as the explanatory variables. For subjective understanding, perceived magnitude of risk and perceived accuracy of information ≥4 and <4 on the Likert scale for index questions (1) to (3) were valued at 1 and 0, respectively. For objective understanding, “about 1/1000 of that of a traffic accident” and the other answers to index question (5) were valued at 1 and 0, respectively. For backlash against information, 1 (highly inaccurate) and >1 on the Likert scale for index question (3) were valued at 1 and 0, respectively. For risk acceptance, two responses (i.e., “do not mind” and “acceptable”) and the other responses were valued at 1 and 0, respectively. A supplementary multivariate logistic regression analysis was also performed with the intersection of “high perceived magnitude of risk (≥4 on the Likert scale)” and “risk acceptance (“do not mind” and “acceptable”)” as the objective variables. For risk-comparison information, dummy variables were created with A1 information (radiation dose only) as a reference. The multivariate logistic regression analysis was done for not only all participants and participants in each prefecture, but also for participants with higher risk perception [Factor 1 (dread risk): factor score >0; Factor 2 (unknown risk (reversed): factor score <0]. Interactions between smoking habits and information on risk-comparison using smoking (A10, B10, and C10) were also investigated for all participants by using a multivariate logistic regression analysis; no significant interactions were observed (P >0.05).

All variance inflation factors (VIF) in the multiple regression analysis and the multivariate logistic regression analysis were ≤3.15; values less than 10 indicate that multicollinearity is not a concern. IBM SPSS Version 22 was used for analysis.

Results

Radiation risk perception and primary associated factors

Consistent with the results of a previous report [9], two factors were extracted from the analysis of the perception of radiation risk (Table 2). Bartlett’s test of sphericity was P < 0.001 and the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy was 0.918. Cronbach’s α for eight and three representative items in Factor 1 and Factor 2 was 0.878 and 0.578, respectively. The lower value in Cronbach’s α for Factor 2 probably occurred because only three items were considered. Overall, the results obtained from the factor analysis were judged to be reliable. Factor 1 was typified by cancer risk, a fatal effect on health, effects on future generations, and intuitive dread, whereas Factor 2 represented whether the health effects of radiation were scientifically elucidated and whether these effects were immediate. In accordance with the previous study [9], we named these factors “dread risk” and “unknown risk (reversed),” respectively. Items for the unknown risk factor were presented in reverse, such that lower factor scores indicated stronger perception of unknown risk. The arithmetic mean ± standard deviation of the factor scores was 0.000 ± 0.953 for dread risk and 0.000 ± 0.854 for unknown risk (reversed). The Pearson’s correlation coefficient r between the two factor scores was 0.671 (P < 0.001). This significant positive correlation indicated that people with a higher dread-risk perception regarding radiation tended to perceive that the radiation risk was scientifically known and its health effects were immediate.

We investigated the relationships between the factor scores for dread risk or unknown risk (reversed) and individual attributes (Fig 1). Factor scores for dread risk in people in Tokyo and Osaka were significantly higher than those in people in Fukushima (P < 0.01), whereas those in people who had had evacuation experience (“evacuated now” and “voluntarily evacuated”) were significantly and strikingly higher than those in people who had not been evacuated (P < 0.05), irrespective of whether the evacuation had been compulsory or voluntary. Factor scores for dread risk in women, elderly people, those with a spouse, children, or grandchildren present, and those who trusted direct or online information from researchers and friends as information sources were significantly higher than those in men, those with no spouse, children, or grandchildren present, and people who did not trust direct or online information from researchers and friends (P < 0.01 for all other attributes). Slight but significant differences were also found in terms of employment status (P < 0.05) and whether a respondent had a characteristic of a humanities or science course (P < 0.01). In contrast, people who trusted central government as an information source had significantly much lower factor scores for dread risk than people who did not trust central government (P < 0.01). There were no significant differences in factor scores for dread risk in terms of current resident area, educational background, or existence and non-existence of a smoking habit (P > 0.05). For the factor scores for unknown risk (reversed), patterns similar to those for dread risk were observed. Fukushima respondents had a higher perception of unknown risk than did those in Tokyo or Osaka. The differences between dread risk and unknown risk (reversed) were that, in the case of unknown risk, significant differences were found between existence and non-existence of a smoking habit and between presence/absence of trust of information from TV and radio (P < 0.01); moreover, in the case of unknown risk there were no significant differences among age groups (P > 0.05).

thumbnail
Fig 1. Factor scores for (a) dread risk and (b) unknown risk (reversed). Error bars represent standard errors.

* P < 0.05, ** P < 0.01. Different letters represent significant differences among groups upon further analysis (P < 0.05).

https://doi.org/10.1371/journal.pone.0165594.g001

To assess the primary factors associated with the perception of dread and unknown risks, we compared standardized partial regression coefficients (β) among individual attributes by using a multiple regression analysis (Table 4). The absolute values of β for the dread-risk factor were highest for location (Fukushima, not evacuated), followed by trust of central government as an information source, and gender (women). Similarly, those for unknown risk factor (reversed) were highest for location (Fukushima, not evacuated), followed by trust of central government and trust of information on TV and radio. The differences in β between existence or non-existence of evacuation experience in people living in Fukushima were strikingly high for both risk-perception factors. In terms of trustworthy information sources, whereas trust of central government contributed negatively to perceptions of dread risk and unknown risk (reversed), trust of information from TV/radio and friends, and of online information from sources other than researchers, contributed positively to these perceptions.

thumbnail
Table 4. Regression coefficients for perception of radiation risk.

B: unstandardized regression coefficient; CI: confidential interval; β: standardized partial regression coefficient.

https://doi.org/10.1371/journal.pone.0165594.t004

Effects of primary factors associated with risk perception and risk-comparison information on respondents’ attitudes

The distributions of respondents’ subjective and objective understanding, perceived magnitude of risk, perceived accuracy of information, backlash against information, and risk acceptance are briefly summarized in S1 Table. Depending on the location and the risk-comparison information provided, these multiple outcomes ranged as follows: subjective understanding, 15.4% to 42.0%; objective understanding, 26.1% to 42.3%; perceived magnitude of risk, 8.5% to 32.5%; perceived accuracy of information: 16.4% to 28.9%; backlash against information, 1.5% to 8.6%; and risk acceptance, 42.1% to 60.8%. The results for risk acceptance were comparable to those of a previous report (46.1%) [6].

We performed a multivariate logistic regression analysis for all participants in order to evaluate the associations of individual attributes and risk-comparison information with these multiple outcomes. The adjusted odds ratios (AOR) and 95% confidential interval (CI) are shown for primary factors associated with dread and unknown risk perception (i.e., location and trust of central government) and risk-comparison information in Table 5, and for other parameters in S2 Table. Results for each prefecture are shown in S3 to S5 Tables.

thumbnail
Table 5. Adjusted odds ratios for location (including evacuation experience), trust of central government, and risk-comparison information provided for respondents’ attitudes to risk, as determined by using a multivariate logistic analysis (all respondents).

https://doi.org/10.1371/journal.pone.0165594.t005

Subjective understanding was significantly and positively associated with location (Fukushima, not evacuated), trust of central government, and risk-comparison information (P < 0.05 for Fukushima, not evacuated; P < 0.01 for other). Among the risk-comparison information, the AOR (95% CI) was the highest for “smoking risk” at 3.08 (2.50 to 3.79), followed by “results for 100 mSv” at 2.94 (2.32 to 3.71) and “cancer risk from radiation and smoking risk” at 2.81 (2.10 to 3.78).

Objective understanding was also significantly and positively associated with trust of central government, and “cancer risk from radiation and smoking risk” as items of risk-comparison information (P < 0.01). A significant negative association was found with location (Tokyo) (P < 0.05).

Perceived magnitude of risk was significantly and positively associated with location (Tokyo; Fukushima, not evacuated; and Fukushima, evacuated) and “doses in other prefectures” and “cancer risk from radiation and arsenic risk” as risk-comparison information items; it was negatively associated with trust of central government (P < 0.01).

Perceived accuracy of information was significantly and positively associated with location (Tokyo; Fukushima, not evacuated; and Fukushima, evacuated) and trust of central government (P < 0.05 for Fukushima, evacuated; P < 0.01 for others). The results for backlash against information were similar to those for perceived accuracy of information; the only difference was that a significant negative association was found with trust of central government (P < 0.01). Importantly, there were no significant associations between perceived accuracy of information or backlash against information and any of the risk-comparison information (P > 0.05).

For risk acceptance, there were significant positive associations with location (Fukushima, not evacuated), trust of central government, and “results for 100-mSv” and “cancer risk from radiation” as risk-comparison information (P < 0.05 for “results for 100-mSv” and “cancer risk from radiation”; P < 0.01 for other). For the intersection of “perceived greatness of risk” and “risk acceptance,” a significant positive association were observed with any location (P < 0.01): the AOR (95% CI) values were 1.74 (1.36 to 2.24) for Tokyo, 3.15 (2.29 to 4.35) for Fukushima (not evacuated), and 4.57 (2.94 to 7.11) for Fukushima (evacuated).

We also conducted a multivariate logistic regression analysis of the high risk-perception groups to evaluate the relationship between risk perception and risk-comparison information and to identify how the risk-comparison information affected people who perceived the risk as more dread or unknown than the average (Tables 6 and 7 and S6 and S7 Tables). For groups with high dread-risk perception the results were close to those for all participants taken together, except that there was a lack of significant association between perceived magnitude of risk and “cancer risk from radiation and arsenic risk” and between risk acceptance and “results for 100-mSv” (P > 0.05). For groups with high unknown-risk perception, the differences from the results for all participants together were a significant and negative association between objective understanding and “results for 100-mSv” (P < 0.05) and a positive association between perceived accuracy/risk acceptance and “arsenic risk” (P < 0.05), in addition to a lack of significant associations between risk acceptance and “results for 100-mSv” and “cancer risk from radiation” (P > 0.05).

thumbnail
Table 6. Adjusted odds ratios of risk-comparison information provided regarding respondents’ attitudes to risk, as determined by using a multivariate logistic analysis (high dread-risk perception group).

https://doi.org/10.1371/journal.pone.0165594.t006

thumbnail
Table 7. Adjusted odds ratios of risk-comparison information provided regarding respondents’ attitudes to risk, as determined by using a multivariate logistic analysis (high unknown-risk perception group).

https://doi.org/10.1371/journal.pone.0165594.t007

Discussion

Radiation risk perception and its primary determinants

Among individual attributes, location (including evacuation experience) and trust of central government were recognized as primary factors influencing both dread-risk perception and unknown-risk perception. This is consistent with the results of a previous study reporting a strong relationship between trust and risk perception [45]. In the US, cultural worldviews are an important determinant of risk perception [24]. Trust of central government is thought to reflect cultural worldview and is therefore potentially involved in risk perception.

In addition to trust of central government, location (including evacuation experience) was another primary determinant of both dread-risk perception and unknown-risk perception. Dread-risk perception among people in Fukushima (not evacuated) was lower than that in people in an area remote from Fukushima (i.e. Osaka); however, people in Fukushima who had had evacuation experience had greater dread-risk perception than those in Osaka, irrespective of whether the evacuation was compulsory or voluntary. People in Fukushima (not evacuated) had greater unknown-risk perception than those in Osaka, whereas the unknown-risk perception in people who had experienced evacuation was comparable to that of people in Osaka.

For decision-making, dread risk perception is a more important factor than unknown risk perception [9]. Dread-risk perception among Fukushima residents clearly differed depending on people’s evacuation experience. For people in Fukushima (not evacuated), the lower dread-risk perception than in Osaka residents might be attributable to habituation to radiation or increased knowledge after the 2011 accident. Notably, the dread-risk perception of people who had experienced compulsory evacuation was similar to that of people who had evacuated voluntarily. These results indicated that high dread-risk perception in evacuees was not explainable only by the fact that people who had originally had a higher dread-risk perception were evacuated; instead, disaster-related experience, including the evacuation itself, might have enhanced people’s dread-risk perception. This finding is consistent with that of a previous study that disaster-related stressors, including living arrangements, were associated with another perception indicator, namely that radiation health effects were likely after the 2011 accident [8].

General public stigma affects disaster victims’ self-image, owing to interactions between in-groups and out-groups [46]. Traumatic memories may have been associated with dread-risk perception after the 2011 accident and were likely much stronger for evacuees than for people who had not experienced evacuation. Because trust of information from TV/radio and friends, and of online information from sources other than researchers, contributed positively to high dread-risk perception, it is likely that dread-risk perception in Fukushima evacuees increased dynamically, reflecting the information supplied by media and advice from relatives and friends (especially those living outside Fukushima), who generally had greater dread-risk perception than those in Fukushima (not evacuated). There were gaps in media information between inside and outside Fukushima [47], and the consolidated information from out-groups, obtained just after the 2011 accident, increased the gap in risk-related attitudes between disaster victims and others. It was a natural reaction to be anxious about radiation after the 2011 accident, and high dread-risk perception should not be therefore blamed. It is, however, a fact that anxiety about radiation is associated with a decline in subjective well-being [7] and an increase in psychological distress [8], which, along with mood disorders, is a risk factor for increased suicide or death from other causes [48,49]. These findings highlight the importance of delivering updated information on radiation risk, not only to evacuees but also to those living outside Fukushima. The delivery of updated information on radiation risk, and associated risk communication, to people outside Fukushima will help disaster victims.

Effects of primary determinants of risk perception on respondents’ attitudes

People in Fukushima (not evacuated) had greater subjective understanding of the risks posed by dietary radiocesium than did those in Osaka, but there were no differences in objective understanding. This suggested that Fukushima residents were used to radiation risk information and that their System I abilities had made them able to judge that they could understand radiation risks. People in Fukushima—especially those who had experienced evacuation—perceived that the radiation risk from dietary radiocesium was greater than did people in Tokyo and Osaka, irrespective of the slight differences in doses among the three prefectures (0.0016 mSv/year for Fukushima; 0.0010 mSv/year for Tokyo; 0.0007 mSv/year for Osaka). Despite this, people in Fukushima (not evacuated) accepted the risk more than did those in Osaka; this was supported by our finding that the AOR (95% CI) for the intersection of “perceived magnitude of risk” and “risk acceptance” compared with that in Osaka was 3.15 (2.29 to 4.35) for Fukushima (not evacuated) and 4.57 (2.94 to 7.11) for Fukushima (evacuated). This consistency between perceived magnitude and risk acceptance indicates that the people of Fukushima could not help but adapt to the current situation after the 2011 accident. Note that the AORs for both perceived accuracy of information and backlash against information were significantly and positively higher for Fukushima residents than for Osaka residents. This result indicated that opinions on accuracy regarding radiation risk were polarized.

People who trusted the central government had a greater subjective and objective understanding of radiation risk than did those who did not trust the central government; the former perceived that radiation risks were lower, believed that the information was adequate, did not doubt the information, and more readily accepted the risks. Trust of central government contributed both negatively to high dread-risk perception and positively to high unknown-risk perception (Table 4). Attitudes to radiation risk, as demonstrated above by the multiple outcomes, were generally consistent with the characteristics of dread-risk perception rather than unknown-risk perception. This is similar to the results of past studies highlighting the fact that dread-risk perception is more likely to have a primary influence on decision-making than is unknown-risk perception [9].

Effect of risk-comparison information on respondents’ attitudes

Among the risk-comparison information, the use of information on “smoking risk,” “results for 100-mSv,” and “cancer risk from radiation and smoking risk” gave the greatest improvements in subjective understanding. Only the risk-comparison information on “cancer risk from radiation and smoking risk” gave an improvement in objective understanding in all participants overall and in the high dread-risk perception group (in which evacuees were likely involved). Because an increase in the level of understanding is an essential element in the success of risk communication, as stated by the US NRC [12], using “cancer risk from radiation and smoking risk” as risk-comparison information is the most prospective approach toward the overall general public. On the other hand, the use of “results for the 100-mSv” should be done with care, because using this information could worsen the objective understanding of the high unknown-risk perception groups, as typified by Fukushima residents without evacuation experience. Importantly, the use of “cancer risk from radiation and smoking risk” did not worsen perceived accuracy of information, backlash against information, perceived magnitude of risk or risk acceptance. Contrary to what was expected from Covello’s guidelines [18], the use of “cancer risk from radiation and smoking risk” did not cause respondents to doubt or distrust the risk information. The lack of correlations between the preference of risk-comparison information and Covello’s guidelines was consistent with previous findings on chemicals [19,20]. The reasons for the advantages of using information on “cancer risk from radiation and smoking risk” are not clear; however, this risk-comparison information provides both an actual risk (cancer risk from radiation) and a relative risk indicator that is known in daily life (i.e. smoking). This likely works well from the perspective of both System I and System II [41,42].

Notably, we set the option with the smallest risk (“about 1/1000 of that of a traffic accident”) as the correct answer to evaluate objective understanding: i.e., choosing other options implied overestimation of the risk. It should be remembered that there is no neutrality in the way we provide information; this approach is a fundamental concept of nudge theory [50]. In this theory, to deliver risk, a provider can choose any item of risk-comparison information out of various options, such as information on radiation dose only or on a combination of radiation dose, cancer risk from radiation, and smoking risk. Use of radiation dose information apart from cancer risk, as typically applied in practice after the 2011 accident, makes the public perceive that radiation risks are higher, in comparison with supplying risk-comparison information on “cancer risk from radiation and smoking risk.” The use of cancer risk and smoking risk as risk-comparison information has been regarded as taboo [18]. It is true that risk acceptance differs between voluntary risk (e.g., from smoking) and involuntary risk (radiation risk owing to an accident) [9,51]. However, this does not mean that the risk-comparison information itself is invalid for raising people’s understanding of the level of risk. Similar to the finding that narrative messages affect respondents’ decisions [52], a context that has the intention of persuasion rather than providing risk-comparison information probably causes public distrust. Use of cancer risk from radiation assumes absence of “zero risk.” Providers may think that this attitude is unacceptable; however, we highlight the fact that avoiding the use of information on “cancer risk from radiation and smoking risk” can lead the public to perceive that the risk is greater than the actual level.

Limitations of this study, and future perspectives

Our study had some limitations. The first potential limitation is participant bias. We used an online questionnaire survey; this is a possible creator of bias, although online questionnaires have advantages in that participants, who receive reward points, have incentives to respond to the questionnaire, irrespective of their level of interest in the topic: level of interest can be a source of potential bias in the case of mail surveys or central location testing. Moreover, the numbers of Fukushima respondents aged in their 20s and 60s were limited, and this is another possible creator of bias. To limit the bias, we categorized factors such as individual attributes to evaluate and relate the strengths of contributions to, or associations with, outcomes. Second, we performed the questionnaire surveys approximately 5 years after the 2011 accident; the situation was therefore a post-crisis one. There is room to discuss whether our findings can be adapted to risk communication in emergencies (crisis communications). Third, because we limited the radiation risk to dietary radiocesium, caution should be taken in extrapolating the results to overall radiation risk, including from external exposure. Fourth, we assumed that, in the initial stage of risk communication, numerical risk data would be delivered via a group-based approach; therefore, the effects of interactions among stakeholders through two-way communications within small focus groups remain debatable. Further studies based on empirical studies covering comprehensive risk at different time stages, from pre-crisis to crisis and post-crisis, are required.

Here, we found that the evacuation experience, as well as trust, is an important determinant of radiation risk perception. Moreover, use of “cancer risk from radiation and smoking risk” enhances subjective and objective understanding without diminishing trust, and use of other risk-comparison information can lead the public to overestimate risk. Providers must choose appropriate risk-comparison information. Evidence-based risk communication using fair and justifiable risk-comparison information will help the public to increase their levels of understanding and make their own decisions regarding risk.

Supporting Information

S1 Table. Distributions of respondents’ subjective and objective understanding, perceived magnitude of risk, perceived accuracy of information, backlash against information, and risk acceptance.

https://doi.org/10.1371/journal.pone.0165594.s001

(PDF)

S2 Table. Adjusted odds ratios of other factors affecting respondents’ attitudes to risk, as determined by a multivariate logistic analysis (all respondents).

Values in parenthesis represent 95% CI. * P < 0.05, ** P < 0.01. Ref = reference. Adjusted by location including evacuation experience, trust on central government, and risk-comparison information provided (see Table 5).

https://doi.org/10.1371/journal.pone.0165594.s002

(PDF)

S3 Table. Adjusted odds ratios of determinants of respondents’ attitudes to risk, as determined by using a multivariate logistic analysis (Fukushima respondents).

Values in parenthesis represent 95% CI. * P < 0.05, ** P < 0.01. Ref = reference.

https://doi.org/10.1371/journal.pone.0165594.s003

(PDF)

S4 Table. Adjusted odds ratios of determinants of respondents’ attitudes to risk, as determined by a multivariate logistic analysis (Tokyo respondents).

Values in parenthesis represent 95% CI. * P < 0.05, ** P < 0.01. Ref = reference.

https://doi.org/10.1371/journal.pone.0165594.s004

(PDF)

S5 Table. Adjusted odds ratios of determinants of respondents’ attitudes to risk, as determined by a multivariate logistic analysis (Osaka residents).

Values in parenthesis represent 95% CI. * P < 0.05, ** P < 0.01. Ref = reference.

https://doi.org/10.1371/journal.pone.0165594.s005

(PDF)

S6 Table. Adjusted odds ratios of other determinants of respondents’ attitudes to risk, as determined by using a multivariate logistic analysis (high dread-risk perception group).

Values in parenthesis represent 95% CI. * P < 0.05, ** P < 0.01. Ref = reference. Adjusted by risk-comparison information provided (see Table 6).

https://doi.org/10.1371/journal.pone.0165594.s006

(PDF)

S7 Table. Adjusted odds ratios of other determinants of respondents’ attitudes to risk, as determined by using a multivariate logistic analysis (high unknown-risk perception group).

Values in parenthesis represent 95% CI. * P < 0.05, ** P < 0.01. Ref = reference. Adjusted by risk-comparison information provided (see Table 7).

https://doi.org/10.1371/journal.pone.0165594.s007

(PDF)

Author Contributions

  1. Conceptualization: MM JN TO.
  2. Data curation: MM.
  3. Formal analysis: MM.
  4. Funding acquisition: MM JN TO.
  5. Investigation: MM JN TO.
  6. Methodology: MM JN TO.
  7. Visualization: MM JN TO.
  8. Writing – original draft: MM JN TO.
  9. Writing – review & editing: MM JN TO.

References

  1. 1. Murakami M, Oki T (2014) Estimated dietary intake of radionuclides and health risks for the citizens of Fukushima City, Tokyo, and Osaka after the 2011 nuclear accident. PLos One 9: e112791. pmid:25390339
  2. 2. Harada KH, Niisoe T, Imanaka M, Takahashi T, Amako K, Fujii Y, et al. (2014) Radiation dose rates now and in the future for residents neighboring restricted areas of the Fukushima Daiichi nuclear power plant. P Natl Acad Sci USA 111: E914–E923. pmid:24567380
  3. 3. United Nations Scientific Committee on the Effects of Atomic Radiation (2014) Sources, effects and risks of ionizing radiation. UNSCEAR 2013 Reports to the General Assembly with Scientific Annexes. United Nations, New York.
  4. 4. Nuclear Safety Research Association (2011) Sinban seikatu kankyo houshasen (kokumin senryo no santei). Nuclear Safety Research Association. [in Japanese]
  5. 5. Tsubokura M, Kato S, Nomura S, Morita T, Sugimoto A, Gilmour S, et al. (2015) Absence of internal radiation contamination by radioactive cesium among children affected by the Fukushima Daiichi nuclear power plant disaster. Health Phys 108: 39–43. pmid:25437518
  6. 6. Consumer Affairs Agency Government of Japan (2016) Fuhyo higai nikansuru shohisha isiki no jittaityosa (dai 7 kai) nituite: http://www.caa.go.jp/earthquake/understanding_food_and_radiation/pdf/160310kouhyou_1.pdf. Accessed: July, 2016 [in Japanese]
  7. 7. Hommerich C (2012) Trust and subjective well-being after the Great East Japan Earthquake, tsunami and nuclear meltdown: Preliminary results. Int J Jpn Sociol 21: 46–64.
  8. 8. Suzuki Y, Yabe H, Yasumura S, Ohira T, Niwa S, Ohtsuru A, et al. (2015) Psychological distress and the perception of radiation risks: the Fukushima health management survey. Bull World Health Organ 93: 598–605. pmid:26478623
  9. 9. Slovic P (1987) Perception of risk. Science 236: 280–285. pmid:3563507
  10. 10. Kleinhesselink RR, Rosa EA (1991) Cognitive representation of risk perceptions: A comparison of Japan and the United States. J Cross-Cult Psychol 22: 11–28.
  11. 11. Science Council of Japan Daiichibu Fukushima Genpatsu saigai gono kagaku to shakai no arikata wotou bunkakai (2014) Kagaku to Shakai no yoriyoi kankei ni mukete: fukushima genpatsu saigaigo no sinraisousitu wo fumaete: http://www.scj.go.jp/ja/info/kohyo/pdf/kohyo-22-t195-6.pdf. Accessed: May, 2016 [in Japanese]
  12. 12. National Research Council (1987) Improving risk communication. Washington D.C., National Academy Press. https://doi.org/10.17226/1189 pmid:25032320
  13. 13. Fischhoff B, Slovic P, Lichtenstein S (1978) Fault trees: Sensitivity of estimated failure probabilities to problem representation. J Exp Psychol Human 4: 330–344.
  14. 14. Hino Y, Murakami M, Midorikawa S, Ohtsuru A, Suzuki S, Tsuboi K, et al. (2016) Explanatory meetings on thyroid examination for the "Fukushima Health Management Survey" after the Great East Japan Earthquake: Reduction of anxiety and improvement of comprehension, and satisfaction. Tohoku J Exp Med 239: 333–343. pmid:27535010
  15. 15. Apter AJ, Paasche-Orlow MK, Remillard JT, Bennett IM, Ben-Joseph EP, Batista RM, et al. (2008) Numeracy and communication with patients: they are counting on us. J Gen Intern Med 23: 2117–2124. pmid:18830764
  16. 16. National Institute of Radiological Sciences (2013) http://www.nirs.go.jp/data/pdf/hayamizu/j/20130502.pdf. Accessed: July, 2016 [in Japanese]
  17. 17. National Cancer Center Japan (2014) http://www.ncc.go.jp/jp/shinsai/index.html#03. Accessed: July, 2016 [in Japanese]
  18. 18. Covello VT, Sandman PM, Slovic P (1988) Risk communication, risk statistics, and risk comparisons: A manual for plant managers. Washington, DC: Chemical Manufacturers Association.
  19. 19. Roth E, Morgan MG, Fischhoff B, Lave L, Bostrom A (1990) What do we know about making risk comparisons? Risk Anal 10: 375–387.
  20. 20. Johnson BB (2003) Are some risk comparisons more effective under conflict?: a replication and extension of Roth et al. Risk Anal 23: 767–780. pmid:12926569
  21. 21. Sato A (2015) Understanding effective risk communication in the context of a radiological accident. Fukushima Global Communication Programme Working Paper Series Number 07.
  22. 22. Orita M, Hayashida N, Urata H, Shinkawa T, Endo Y, Takamura N (2013) Determinants of the return to hometowns after the accident at Fukushima Dai-ichi nuclear power plant: a case study for the village of Kawauchi. Radiat Prot Dosim 156: 383–385. pmid:23547241
  23. 23. Boholm A (1998) Comparative studies of risk perception: a review of twenty years of research. J Risk Res 1: 135–163.
  24. 24. Kahan DM, Braman D, Gastil J, Slovic P, Mertz CK (2007) Culture and identity-protective congnition: explaining the while-male effect in risk perception. J Empir Legal Stud 4: 465–505.
  25. 25. Fukushima Prefecture (2015) https://www.pref.fukushima.lg.jp/sec/11045b/15859.html. Accessed: July, 2015 [in Japanese]
  26. 26. Oki S, Nakayachi K (2012) Paradoxical effects of the record-high Tsunamis caused by the 2011 Tohoku Earthquake on public judgments of danger. Int J Disaster Risk Reduction 2: 37–45.
  27. 27. National Research Council Of The National Academies (2005) Assessment of the scientific information for the radiation exposure screening and education program. The National Academies Press. Washington, D.C. https://doi.org/10.17226/11279
  28. 28. Murakami M (2016) Risk analysis as regulatory science: Toward the establishment of standards, Radiat Prot Dosim, 171(1), 156–162, 2016. pmid:27475751
  29. 29. Ministry of Health Labour and Welfare (2015) http://www.mhlw.go.jp/file/04-Houdouhappyou-11134000-Shokuhinanzenbu-Kijunshinsaka/2015051502.pdf. Accessed: July, 2016 [in Japanese]
  30. 30. ICRP (2007) The 2007 recommendations of the international commission on radiological protection. ICRP Publication 103. Ann ICRP 37.
  31. 31. Yamagishi K (1997) When a 12.86% mortality is more dangerous than 24.14%: Implications for risk communication. Appl Cognitive Psych 11: 495–506.
  32. 32. Slovic P, Monahan J, MacGregor DG (2000) Violence risk assessment and risk communication: The effects of using actual cases, providing instruction, and employing probability versus frequency formats. Law and Human Behavior 24: 271–296. pmid:10846372
  33. 33. Oka T (2014) Cost-benefit analysis of the regulation of food contamination with radioactive caesiums within a year after the Fukushima accident: The cases of vegetables and rice. Jpn J Risk Anal 24: 101–110. [in Japanese]
  34. 34. United Nations Scientific Committee on the Effects of Atomic Radiation (2011) UNSCEAR 2008 Report Volume II. United Nation, New York.
  35. 35. National Institute of Radiological Sciences (1965) Radioactivity Survey Data in Japan. Number 7.
  36. 36. National Cancer Center Center for Cancer Control and Information Services (2016) http://ganjoho.jp/reg_stat/statistics/stat/summary.html. Accessed: July 2016 [in Japanese]
  37. 37. Ministry of Education Culture Sports Science and Technology (2011) http://www.mext.go.jp/component/b_menu/other/__icsFiles/afieldfile/2011/11/04/1313005_10_1.pdf. Accessed: July, 2016 [in Japanese]
  38. 38. Joint FAO/WHO Expert Committee on Food Additives (2011) WHO food additives series 63/FAO/ JECFA monographs 8.
  39. 39. Oguri T, Yoshinaga J, Tao H, Nakazato T (2012) Daily intake of inorganic arsenic and some organic arsenic species of Japanese subjects. Food Chem Toxicol 50: 2663–2667. pmid:22617717
  40. 40. Inoue M, Hanaoka T, Sasazuki S, Sobue T, Tsugane S (2004) Impact of tobacco smoking on subsequent cancer risk among middle-aged Japanese men and women: data from a large-scale population-based cohort study in Japan—the JPHC study. Prev Med 38: 516–522. pmid:15066353
  41. 41. Stanovich KE, West RF (2000) Individual differences in reasoning: implications for the rationality debate? Behav Brain Sci 23: 645–665. pmid:11301544
  42. 42. Kahneman D (2011) Thinking, fast and slow: Farrar, Straus and Giroux.
  43. 43. National Police Agency Traffic Bureau (2015) https://www.npa.go.jp/toukei/koutuu48/before/30/PDF/H26.30.pdf. Accessed: July, 2016 [in Japanese]
  44. 44. ICRP (1959) Recommendations of the International Commission on Radiological Protection. ICRP Publication 1.
  45. 45. Visschers VHM, Siegrist M (2013) How a nuclear power plant accident influences acceptance of nuclear power: Results of a longitudinal study before and after the Fukushima Disaster. Risk Anal 33: 333–347. pmid:22762151
  46. 46. Corrigan PW (2005) On the stigma of mental llness: Practical strategies for research and social change. Washington, DC. American Psychological Association.
  47. 47. Ikeda K, Yasuda Y, Tanaka A, Inamasu K, Shibanai Y, Aizu I (2015) Sinsai kara mieru jouhou media to network. Tokyo: Toyo Keizai Inc. [in Japanese]
  48. 48. Pratt LA (2009) Serious psychological distress, as measured by the K6, and mortality. Ann Epidemiol 19: 202–209. pmid:19217003
  49. 49. Mattisson C, Bogren M, Bradvik L, Horstmann V (2015) Mortality of subjects with mood disorders in the Lundby community cohort: A follow-up over 50 years. J Affect Disord 178: 98–106. pmid:25801522
  50. 50. Thaler RH, Sunstein CR (2008) Nudge: Improving decisions about health, wealth, and happiness: Yale University Press.
  51. 51. Starr C (1969) Social benefit versus technological risk. Science 165: 1232–1238. pmid:5803536
  52. 52. Steinhardt J, Shapiro MA (2015) Framing effects in narrative and non-narrative risk messages. Risk Anal 35: 1423–1436. pmid:25809457