http://orcid.org/0000-0002-8514-2563
Figures
Abstract
Background
In January 2020, the coronavirus disease 2019 (COVID-19) started to spread in Italy. The Italian government adopted urgent measures to slow its spread. Enforcing compliance with such measures is crucial in order to enhance their effectiveness. Engaging citizens in the COVID-19 preventive process is urgent today both in Italy and around the world. However, to the best of our knowledge, no previous studies have investigated the role of health engagement in predicting citizens’ compliance with health emergency containment measures.
Method
An online survey was administered between February 28 and March 4, 2020 on a representative sample of 1000 Italians. The questionnaire included a measure of health engagement (Patient Health Engagement Scale), a 5-item Likert scale ranging from 1 to 7, resulting in 4 positions that describe the psychological readiness to be active in one’s own health management, and a series of ad hoc items intended to measure citizens’ perceived susceptibility and severity of the disease, orientation towards health management, trust in institutional bodies, health habits and food consumption. To investigate the relationship between health engagement and these variables, ANOVA analysis, logistic regression and contingency tables with Pearson’s chi-squared analysis have been carried out.
Results
Less engaged people show higher levels of perceived susceptibility to the virus and severity of the disease; they are less trustful of scientific and healthcare authorities, they feel less self-effective in managing their own health—both in normal conditions and under stress—and are less prone to cooperate with healthcare professionals. Low levels of health engagement also are associated with a change in the usual purchase behavior.
Conclusions
The Patient Health Engagement model (PHE) provides a useful framework for understanding how people will respond to health threats such as pandemics. Therefore, intervention studies should focus on raising their levels of engagement to increase the effectiveness of educational initiatives intended to promote preventive behaviors.
Citation: Graffigna G, Barello S, Savarese M, Palamenghi L, Castellini G, Bonanomi A, et al. (2020) Measuring Italian citizens’ engagement in the first wave of the COVID-19 pandemic containment measures: A cross-sectional study. PLoS ONE 15(9): e0238613. https://doi.org/10.1371/journal.pone.0238613
Editor: Wen-Jun Tu, Chinese Academy of Medical Sciences and Peking Union Medical College, CHINA
Received: April 18, 2020; Accepted: August 20, 2020; Published: September 11, 2020
Copyright: © 2020 Graffigna et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: This study was conducted within the CRAFT project, funded by Fondazione Cariplo & Regione Lombardia.
Competing interests: The authors have declared thata no competing interests exist.
Introduction
In January 2020, the coronavirus disease 2019 (COVID-19) started to spread in Italy. The virus and its associated disease were given the designation of coronavirus disease 2019 (COVID-19) in February 2020, distinguishing this syndrome from the acute respiratory syndromes associated with 2 other betacoronaviruses (SARS-CoV and Middle East respiratory syndrome corona-virus) that caused earlier outbreaks of severe disease [1,2]. As of March 17, 2020, a total of 31,506 COVID-19 cases with 2503 deaths and 2941 recovered had been reported in Italy (updated Italian situation available at http://www.salute.gov.it/portale/nuovocoronavirus/dettaglioContenutiNuovoCoronavirus.jsp?area=nuovoCoronavirus&id=5351&lingua=italiano&menu=vuoto).
On Jan 30, 2020, the World Health Organization (WHO) declared the coronavirus outbreak a public health emergency of international concern (PHEIC). Starting March 7, 2020, the Italian government adopted very urgent and restrictive measures to slow the virus spread and reduce its potential impact on the population (available at https://www.gazzettaufficiale.it/eli/id/2020/03/08/20A01522/sg). Several cities—identified as “red areas”—have been put under quarantine, hoping to stop the disease from spreading to other parts of the country. This situation is globally unprecedented at least for two main reasons. First, to control the COVID-19 outbreak, governmental authorities have suddenly adopted very extreme public measures such as locking down cities, deeply reorganizing healthcare services to cope with the rapid increasing demand for acute care, imposing school and university closures, suggesting—where possible—smart-working solutions and transportation restrictions, deploying thousands of healthcare workers to more heavily affected regions, and running wide public health messaging campaigns for consumers’ education. Second, consumers are overwhelmed by rather mixed and confounding information, partly because scientific discovery related to COVID-19 disease is constantly evolving with the course of the disease outbreak, and partly due to the rapid increase in misleading or false news. Therefore, all these measures are currently having a deep impact on Italian people’s attitudes, daily habits and consumption behaviors [3].
As in other similar situations, prior to the availability of an effective vaccination therapy, strategies to mitigate and control the impact of the pandemic typically involve “non pharmacological” interventions [4,5], and rely on citizens’ autonomous responses to public health preventive measures. In particular, past literature suggested that people appear to respond to an epidemic by voluntarily undertaking specific behaviors in order to protect themselves [6,7]. However, in some cases, these behaviors may not correspond to an objective evaluation of risk [8,9], but depend on individual subjective evaluations, thus becoming potentially counterproductive. For this reason, there has been a rapid rise of interest in understanding the determinants of people’s behavioral change that may influence the adequacy of the response to health emergencies [10,11]. People dealing with these situations, indeed, may experience negative attitudes, feelings of uncertainties and alarmism [12]. These reactions might potentially end in risky habits and inadequate and disorganized behaviors, both for individuals and the community [13,14], affecting public health outcomes. Therefore, to study the subjective factors implied in such reactions is of much relevance to effectively sensitize the general public and identify high-risk targets [15]. Along with structural and immutable factors such as socio-demographics, scholars have previously attempted to understand the subjective determinants of citizens’ changing attitudes and behaviors in a pandemic emergency. In particular, authors identified risk perception as one of the most relevant variables in determining citizens’ response to global pandemic disease [16,17], among a series of factors such as the perception of economic impact [18]; efficacy beliefs related to health [19]; level of literacy and knowledge elaboration [20,21]. Another important factor identified is the level of subjective anxiety, which influences both citizens’ attitudes towards the emergency disease and consequent preventive behaviors [22–24]. Other subjective factors accounting for the changes in peoples’ habits in pandemic emergencies include those related to the perceived effect of one’s individual behavior, such as perceived costs and benefits of preventive behaviors on the disease spread [25,26] or perceived impact of an individual's behavior on other individuals’ outcomes [24,27].
Among other variables accounting for a change in citizens’ attitudes, habits and behaviors, scholars recently have shown the role of health engagement in affecting health-related behaviors and preventive habits [28–34]. More specifically, people with high levels of health engagement have been identified as more likely to adopt behavioral change suggestions and to adhere to medical prescriptions [35–39]. However, previous literature has also demonstrated how individuals may be in different phases of their process of engagement [29,31,40,41], thus being more or less ready to enact a change in the way they cope with a critical event and comply with prescribed preventive conduct [42–45].
In the current COVID-19 outbreak in Italy–as well as in other countries–citizens are experimenting with a life-threatening situation, leading to profound changes in attitudes, habits and behaviors, which also are potentially negative for consumers’ health and virus containment. Making citizens aware of their crucial role in avoiding the rapid spread of the virus and engaging them in the COVID-19 disease preventive process is urgent today both in Italy and around the world.
However, to the best of our knowledge, no previous studies have investigated the role of people’s health engagement in determining citizen attitudes, habits and compliance with containment measures in the event of an health emergency; moreover, previous literature has highlighted the need to apply a validated theoretical framework to the study of these phenomena in order to effectively predict people’s responses to the event and adherence to prescriptions [11].
For these reasons, we conducted a study aimed at understanding citizens’ attitudes and behavioral responses to the current spread of COVID-19 disease in Italy and determining how they changed their daily habits and behaviors according to their level of health engagement. Results of this study will contribute to informing public health communication and targeted consumer education activities.
Theoretical background
The Patient Health Engagement model [29,46] is a recently developed psychological framework that theorizes how individual health engagement results from a continuous emotional and motivational reframing of an individual’s own role perception in the management of a disease (i.e. from passive user of services to active partner of the healthcare system). According to this model, to become engaged means to be emotionally resilient and able to adjust to the health risks and specific requirements. This model also features unique ways of coping with a health crisis, which the Covid-19 disease can be considered today, with the necessary adaptation to the specific context. In particular, the model features four positions: the first position (“Blackout”) is complete disengagement, typically occurring when people feel vulnerable and without control over the perceived risk, psychologically frozen and behaviorally paralyzed. In the case of COVID-19 disease, this position can explain the psychological reaction of all the citizens to the very initial moment of pandemic spread, where the sudden recognition of the uncontrollable disease diffusion changed people’s lives in the most hit countries. Moreover, the sense of helplessness also exposed the citizens to a psychological vulnerability and risk. Next follows the psychological position of “Arousal,” in which people have acquired an initial awareness about their actual situation of health risk but don’t yet have enough knowledge and skills to manage it. They do not accept the impact of preventive requirements on the modification of their daily habits and appear hyper-vigilant over their body signals, disorganized and confused when seeking information on the health situation. In the situation here described, each unexpected news or change in the epidemic situation causes emotional alert and overwhelming emotional response, with disorganized actions and behaviors that can be dysfunctional for health prevention. When individuals succeed in the process of emotional regulation and coping with the stressful condition, they achieve a position of “Adhesion.” In this phase, patients have developed a good psychological adaptation to the critical situation and appear able to manage their psychological distress connected–in this case—to the COVID-19 health emergency. They appear more motivated to comply with medical and preventive prescriptions. In this phase, moreover, patients acquire further skills to effectively manage their risk condition. Finally, when people achieve a complete awareness of the characteristics and consequences of the critical situation, and assume a more responsible position in their behaviors and risk management, they reach the “Eudaimonic project” phase, which features a better, positive and optimistic approach to the situation, with an increased ability to deal with the uncertainty of the moment and a strong motivation to psychologically achieve the sense of a “new normality” (Fig 1), to overcome the current emergency. The model has been translated into a psychometric scale used to measure Patient Health Engagement level (PHE-S), which has been validated in different countries [47–49]. The PHE-S also has been cited by different research and clinical groups [34,50,51].
Methods
Study design and participants
The study here described is a part of a broader project (entitled: “Italian citizens’ food habits monitoring from a consumer psychology perspective”) aimed at monitoring Italian consumers’ habits. In particular, food consumption behaviors, health behaviors, information gathering and trust towards institutions are tracked over time. It is delivered through online surveys repeated and adjusted over time to track changes in consumers’ orientations in relation to the evolving socio-economic situation of the country. In this case, we adapted the survey to explore people’s reactions to the COVID-19 disease emergency and how different levels of health engagement correspond to unique patterns of behaviours. The survey is based on a cross-sectional design and was carried out between February 28 and March 4, 2020. A panel provider company. Norstat s.r.l. (https://norstat.it/), was in charge of the participants’ selection through stratified random sampling, a sampling method that splits the population in smaller groups (strata) based on sociodemographic characteristics and then samples from these subgroups in order to obtain a sample with the same proportions of, for instance, genders as in the general population [52]. To become part of the panel, people are usually first contacted using random digit dialing that is a technique for drawing a sample of households from the frame or set of telephone numbers. Respondents were rewarded through the Norstat system. A sample of 1000 Italians was involved and weighed to be representative of the Italian population for gender, age, employment, geographical area and dimension of urban center of residence, residents from all the different regions of Italy. To be included in the survey, participants are over 18 years old, are able to read and understand Italian and live in Italy. The percentages relating to the Italian population were taken from the website of ISTAT[53]. People belonging to the online panel were carefully screened for authenticity and legitimacy via digital fingerprint and geo-IP-validation from the panel provider. In this study, in order to guarantee data quality, respondents were asked to confirm their demographics. From the 1000 recruited subjects, 32 were excluded because demographic data provided by the respondent and those provided from the panel were inconsistent (there were discrepancies between reported and known gender and/or age). Statistical analyses were hence carried out on a dataset composed of the answers of 968 respondents. All analyses have been carried out with IBM SPSS 23 (release 23.0.0.0).
Ethical statement
Each participant was instructed about the aims of the research and gave written informed consent before starting the questionnaire. By agreeing to start the compilation, participants accepted the informed consent. They were also allowed to drop out from the compilation at any time. As a part of a panel, the GDPR compliance for the participants here involved is guaranteed by Norstat s.r.l. We received the anonymous database for analysis. No participants’ identification detail was provided to researchers. This study has been performed in accordance with the Declaration of Helsinki and has been approved by an independent ethics committee of Università Cattolica del Sacro Cuore in Milan (CERPS—IRB#02–20).
Study measures
As a part of a broader study (see section 2.1), the survey is composed of a core of fixed measures and a pull of ad-hoc items that change according to the contingency occurring during the specific data collection period. In the wave of data collection we report on in this article, ad hoc items related to individuals’ affective and behavioral responses to the COVID-19 pandemic were added. Following are the specific measures:
- Health engagement: we adopted a revised version of the Patient Health Engagement Scale (PHE-s®) to measure this construct. This measure, developed according to the Patient Health Engagement model [46], assesses the consumers health engagement level, defined as the “people’s psychological readiness and sense of mastery to become active players in their own health management and health risk prevention.” Previous studies demonstrated its robust psychometric proprieties [29], also in other languages [47–49]. This scale features five ordinal items reflecting the continuum of engagement described in the four levels of the PHE model. According to the ordinal nature of the PHE-s®, the median score is considered the more reliable index to calculate the final patients’ scoring [29]. According to the score obtained, each respondent result is in one of the four levels of health engagement as described in the PHE model (i.e. blackout, arousal, adhesion, eudaimonic project). The scale is based on the assumption that the score obtained by the person should reflect his/her actual health engagement level. For this study purposes, the PHE- s® was slightly revised in order to adapt the items’ formulation to the specific context of the health emergency. The incipit was revised in order to adapt coherently to the Covid-19 emergency (from “thinking about your health” to “thinking about your health in this emergency”; the formulation of the fifth item was also revised to adapt it to the nature of the subject. For this reason, the psychometric characteristics of the revised version were tested.
- Attitudinal response towards to the COVID-19 health emergency. In particular, in light of studies on risk processing [54], two elements of risk judgment were measured: (a) risk severity (the perceived potential severity of COVID-19 infection for their own health) on a scale from 1 (not concerned at all) to 10 (very concerned); and (b) risk susceptibility (the perceived likelihood to get COVID-19) on a scale from 1 (very little) to 5 (a lot). Moreover, participants were asked to rate their agreement (from 1, completely disagree, to 5, completely agree) with a series of statements regarding their self-responsibility, self-efficacy in health management, self-efficacy in stress management, the value of partnership in healthcare, trust in science, trust in the National Healthcare System (NHS) and media reliability.
- Behavioral responses: involving frequency of information seeking from various different medias (TV, newspapers, social networks, scientific journals etc.) on a scale from 1 (never) to 5 (usually). An average was then calculated, in order to obtain an indicator of how much a certain subject was searching for information regarding the virus. Moreover, a series of dichotomous, yes/no questions were asked regarding changes in consumer habits and, in particular, asking whether they had reduced restaurant meals, ethnic restaurant meals, and the purchase of products coming from “red” zones. They were also asked if they “stockpiled” food and first need products. Finally, they were asked a series of questions surveying whether the buying of different products (i.e., fresh food, frozen food, canned food, products for personal disinfection and care, and products for house disinfection) had diminished/remained the same/increased.
A list of all the items included in the present study has been made available in both English and original language in the Supplementary Information along with the dataset.
Statistical analysis
The revised form of PHE scale evaluation.
To evaluate the psychometric properties of the revised PHE-s® scale, a Partial Credit Rasch Model (PCM) was performed to check uni-dimensionality and the fit of each item at the construct of interest [55]. In the family of Rasch Models, PCM was chosen because the revised PHE-s® items had more than two response options and they showed different patterns of usage [56,57]. It is reasonable to assume that since the thresholds are different for all the items, i.e., each item has its own unique rating scale structure, the PCM appears the most appropriate model. The parameters of the model are estimated by the maximum likelihood method [58]. Then, the Person Separation Index (PSI) was calculated to evaluate the reliability in the Rasch Model. The PSI is an indicator of the quality of measures and refers to the reproducibility of the measured location of the persons. PSI indicates the degree to which study participants can be differentiated into certain groups (range 0–1). Values for PSI superior to 0.8 are acceptable [59,60].
In particular, to check whether the items fitted the expected model, two items fit mean square (MNSQ) statistics (Infit and Outfit) were computed. If the data fit the Rasch model, the fit statistics should be between 0.6 and 1.4 [61]. Analyses of difficulty and step parameters were conducted to guarantee a sufficient ranking of the different categories of response and to respect the monotonic order. The internal consistency of the items of the revised PHE-s® was assessed using Ordinal Alpha via Empirical Copula Index [62] due to the ordinal nature of the items. A reliability index superior to 0.7, 0.8, 0.9 can be interpreted as acceptable, good and excellent, respectively [63].
Finally, a Confirmatory Factor Analysis (CFA) was performed. Goodness-of-fit indexes (i.e. comparative fit index CFI, root mean square residual RMR and root mean square error of approximation RMSEA) were evaluated. A CFI > .90 was considered a good model fit [64], a RMR < .05 was desirable [65], whereas a RMSEA < .08 indicated an acceptable fit [66].
Attitudinal response towards the COVID-19 health emergency.
To explore differences in the perceived Risk Severity and Risk Susceptibility between different health engagement groups, two factorial ANOVA with Risk Severity and Risk Susceptibility as dependent variables and health engagement and “Coming from red zones” as independent variables were carried out, followed by Tuckey HSD post-hoc tests. Tukey HSD post-hoc test was preferred since it is conservative when there are unequal sample sizes.
To explore the difference in self-responsibility, self-efficacy in health management, self-efficacy in stress management, the value of partnership in healthcare, trust in science, trust in the National Healthcare System (NHS) and perceived media reliability among different health engagement groups, a series of univariate Welch’s ANOVA with health engagement as independent variable was carried out followed by Games-Howell post-hoc comparisons. Welch’s ANOVA and G-H post-hoc comparisons were preferred over a classic ANOVA approach to provide a more robust method for data analysis [67] since some dependent variables were violating the assumption of homoscedasticity.
Behavioral responses.
A Welch’s ANOVA with Health Engagement as independent variable and Information Seeking as dependent was carried out, followed by Games-Howell post-hoc comparisons to investigate whether people in different health engagement positions show different amounts of media access.
Dichotomous variables were used as dependent variables in a series of multi-variable logistic regressions, with Health Engagement, Risk Susceptibility and Risk Severity as independent variables Wald forward method was selected to automatically exclude non-significant predictors. Health Engagement was used as a categorical variable and hence dummy coded: Eudaimonic Project was used as the 0, the baseline of comparison, for the other two levels. Dependent variables were coded so that “No” was used as the comparison level for “Yes”. Hence, an Odds Ratio > 1 should be interpreted as “more likely to answer yes” and vice-versa.
To assess the association between change in consumer purchase behaviors and different health engagement levels, a series of contingency tables was created. Pearson’s chi-square and Fisher’s exact tests were also carried out to reject the null hypothesis that data are randomly distributed across health engagement levels. As post-hoc, standardized residuals were inspected: standardized residuals are calculated as the difference between observed and expected counts of a cell divided by an estimate of its standard deviation. Since they are asymptotically normally distributed with a mean of 0 and standard deviation of 1 under the null hypothesis of independence, as a general rule of thumb, cells with an absolute value of standard residuals above 2 can be considered to significantly contribute to the general chi-square value [68]. For stockpiling behavior, groups were way too unbalanced to proceed with a logistic regression (Yes = 5.6%); hence, an approach based on contingency tables was preferred.
Results
Sample
Male participants were 473 (48.9%). Mean age was 44 years (SD = 14; range 18–70). For a more detailed description of the study sample, see Table 1.
Descriptive statistics
There were no missing data in our dataset. For dichotomous and multiple-choice questions, answer frequencies and “I don’t know” answers are reported -where provided- in Table 2. However, in the following analyses, “I don’t know” was considered as a missing value. For other variables, descriptive statistics are reported in Table 3. Since very few participants resulted being in “Blackout” position, they were grouped together with participants in “Arousal” to facilitate statistical analyses.
Psychometric proprieties of the PHE-s® revised version
Table 4 shows the results of the Rasch Analysis to test the psychometric properties of the PHE-s® revised version.
The item statistics ranged from .674 to 1.085 for the infit MSQ and from .616 to 1.187 for the outfit MSQ. These values indicate an acceptable fit of the Rasch Model. The distances between subsequent thresholds showed acceptable distinction between the response options and measurement model fit. The PSI for revised PHE-s® was equal to .851. Rasch Model confirmed the unidimensionality of the revised PHE-s® scale and the fit of each item of the scale to the data.
The revised PHE-s® had a quite good internal consistency, since the value of the Ordinal Alpha via Empirical Copula was equal to .788. Each item contributed significantly to the revised PHE-s® scale score. So, the internal consistency of the revised PHE-s® was satisfactory.
CFA showed reasonable goodness of fit indices. The fit indices met the criteria of fit for the hypothesized one-factor structure. All goodness of fit indices (CFI = 0.994, RMR = 0.008, RMSEA = 0.066) suggested that the model is coherent with the data. The analysis of modification indices did not find any relation between the error covariance of the items. All the standardized to factor loadings ranged from .532 to .820.
Attitudinal response towards to the COVID-19 health emergency
Risk severity.
ANOVA results show a significative main effect of Health Engagement on Risk Susceptibility [F(2, 1048) = 185.709; p < .001; η2p = .262]. No other significant main effect or interaction was found. Tukey post-hoc comparisons show that the Arousal group (M = 8.00; SD = 1.71) was more concerned than either the Adhesion group (M = 5.98; SD = 2.09) or the Eudaimonic Project group (M = 3.51; SD = 2.39) with a significance level of 99.9%. Also, the means difference of Adhesion and Eudaimonic Project groups was found to be statistically significant with p < .001.
Risk susceptibility.
Results show a significative main effect of Health Engagement on Risk Susceptibility [F(2, 1040) = 150.890; p < .001; η2p = .225]. No other significant main effect or interaction was found. Tukey post-hoc comparisons revealed that the Arousal group (M = 3.73; SD = .87) perceived themselves as more at risk than either the Adhesion group (M = 2.94; SD = .92) or the Eudaimonic Project group (M = 1.97; SD = .906) with a significance of 99.9%. Also the means difference of Adhesion and Eudaimonic Project groups was found to be statistically significant with p < .001.
Orientation towards health management and trust in authorities.
ANOVA results show a significant main effect of Health Engagement on Self-Responsibility [F(2, 322.257) = 3.700; p = .026; η2 = .009], Self-Efficacy in Health Management [F(2, 339.819) = 57.382; p < .001; η2 = .113], Self-Efficacy in Stress Management [F(2, 355.911) = 16.497; p < .001; η2 = .032], Value of Partnership in Healthcare [F(2, 344.585) = 9.568; p < .001; η2 = .022], Trust in Science [F(2, 335.022) = 8.158; p = .001; η2 = .018], Trust in NHS [F(2, 337.641) = 9.575; p < .001; η2 = .021] and Media Reliability [F(2, 344.288) = 28.664; p < .001; η2 = .060]. Results of Games-Howell comparisons are reported in Table 5.
Behavioral responses
Information seeking.
ANOVA results show a significant main effect of Health Engagement on Information Seeking [F(2, 334.095) = 29.344; p < .001; η2 = .064]. G-H comparisons showed that the amount of Information Seeking differed significantly among all the different levels: in particular, results showed that people in Arousal search significantly more information (M = 2.79; SD = .74) than people in either Adhesion (M = 2.47; SD = .68) or Eudaimonic project (M = 2.20; SD = .77). The comparison between Adhesion and Eudaimonic project was significantly different as well.
Consumer habits and purchasing behaviors.
Results of the logistic regressions are reported in Table 6. In particular, results show that higher levels of Risk Severity and Risk Susceptibility are associated with a higher probability of having reduced meals outside in both generic and ethnic restaurants. Perceived Risk Severity is also a predictor of the willingness to buy products coming from “red zones” (higher perceived Severity increases the probability of not being willing to buy). Results also show that Health Engagement (HE) levels predict having reduced meals outside (lower levels of engagement have a higher probability) and of being willing to buy “red zone” products (lower engagement, lower probability).
Results of contingency tables are reported in Table 7. Pearson’s chi-squared analysis and the inspection of standardized residuals show that different levels of Health Engagement are associated with different consumer behaviors: in particular, our results show that lower levels of engagement are more frequently associated with stockpiling, and with an increased consumption of fresh, canned and frozen food, and with products for disinfection when compared with average and high levels of engagement.
Discussion
By the end of February 2020, the diffusion of the COVID-19 epidemics in northern Italy had forced health authorities to embrace restrictive preventive measures that impacted Italian citizens’ daily habits and consumption behaviors. Enforcing compliance with such measures was crucial at that time in order to enhance their effectiveness and to sustain the sustainability of the healthcare system. However, this sudden change caused huge reactions by Italian citizens: many of them experienced panic and enacted maladaptive behaviors (for example the migration from north to south Italy immediately after the Lombardy region lockdown, which was initially considered a “red zone”; also, food stockpiling happened soon after the first cases of Covid-19 disease came out, which created negative consequences for the food chain organization). In this scenario, the Italian citizens’ reactions to the COVID-19 emergency measures, from the scientific perspective, is an interesting and unique platform to demonstrate the value of making citizens engaged as actual partners of the healthcare system to safeguard both individual and collective health. Therefore, we consider the current COVID-19 outbreak in Italy as a valuable “testing ground” for consumer education initiatives aimed at sustaining their health engagement and compliance with the prescribed behavioral changes. Existing research has focused on demographic and immutable and subjective factors that influence how people are likely to behave in a pandemic [69–71]. Furthermore, previous research on responses to pandemics has been largely a-theoretical [11]. Therefore, all these studies provide valuable insights into how different segments of the population are likely to respond, but do not tell us why they respond in this way. The current study adopted the theoretical lens of the Patient Health Engagement Model (PHE) to explain–from a psychosocial perspective—people’s responses through the first wave of the COVID-19 pandemic in Italy. This theory states that individuals are more or less likely to change their behaviors according to their own subjective perceptions about the role (more or less active) they might play in their health and care [46].
The Patient Health Engagement Model (PHE) provides a potentially useful framework for understanding how people will respond to health threats such as pandemics and related prescribed preventive measures imposed by healthcare authorities. The PHE model proposes that people’s adaptive behavioral and emotional responses to protect themselves from a health threat are influenced by their level of health engagement–that is a progressive reframing of individuals’ own roles within the healthcare system (i.e. from passive users of services to active partners of the healthcare system) [46]. In this study, we employed and evaluated the psychometric properties of a revised version of the PHE-s® to measure citizens’ health engagement. This revised version showed good psychometric properties for our representative sample.
According to the study results, Italian citizens seems to be more concerned about the health emergency than not, even though not extremely worried (on a scale from 1 to 10, the average is around 6) and not feeling exceedingly at risk of being infected (the 5-point Likert shows a normaloid distribution with mean around the central point), confirming previous studies in other similar settings [9,72,73]. Nevertheless, it is important to notice how different health engagement profiles are associated with different levels of both perceived risk severity and susceptibility: indeed, less engaged people (rated as in “Blackout” and “Arousal”) show significantly higher levels of perceived susceptibility to and perceived risk of the infection when compared with highly engaged ones (rated as in “Adhesion” and “Eudaimonic Project”), regardless of the geographical area of origin (“red zone” or not), which surprisingly wasn’t found to be associated with different levels of susceptibility and severity. This seems to support that people differ in their ability to psychologically master their worries related to the COVID-19 epidemic, and this explains the consequent more or less adherence to the change in behaviors imposed by the health authorities. This interpretation is confirmed also by the fact that people with different levels of health engagement show different attitudinal responses to the emergency: in particular, when compared to people with higher levels of health engagement, less engaged people are less trusting of scientific and healthcare authorities, they feel less self-effective in managing their own health—both in normal conditions and under stress—and are less prone to cooperate with healthcare professionals [74]. These results confirm previous studies on Influenza A (H1N1), which demonstrated that if perceived severity and susceptibility are high but response and self-efficacy are low, maladaptive responses (e.g. denying the existence of a threat) are likely to ensue [75]. The perceived self-efficacy in health management and a sentiment of mistrust towards authorities may actually help in understanding why a less engaged person feels more concerned and worried about the new COVID-19 emergency: they seek more information, potentially exposing themselves to fake or over-hyped news, since they are also prone to feel that news regarding the emergency is reliable; nevertheless, they mistrust scientific research and the capacity of the NHS to cope with the pandemic and feel less capable of taking care of themselves. Furthermore, low levels of health engagement may demonstrate that people do not consider themselves ready to be active partners of the healthcare systems, being more focused on their own health interests and need and not inclined to collaborate and trust the healthcare system to achieve a common public health goal.
The health engagement construct also seems to be a predictor of behavioral responses to the emergency. Generally speaking, a substantial part of our sample reported a change in their habits: one out of three Italian citizens reported having fewer meals outside and/or meals in ethnic restaurants, while 20% declared that they would not buy products coming from “red zones.” Indeed, while risk severity and risk susceptibility are clearly strong predictors, logistic models show that people with lower levels of health engagement are more than twice as likely as people with higher level of health engagement to have reduced either meals or ethnic meals outside their home. It’s important to notice that data have been collected at a moment when the emergency was still away from its peak and guidelines were not forbidding people from moving freely or from having meals in restaurants. These results could be interpreted as in line with previous studies underlining that when unknown diseases are thought to be lethal, people are inclined to blame the outbreaks on someone, or some group of people, who live outside of their own social sphere, as a mechanism to cope with fear and risk perception [76]. In this research, it appears clear that this form of “moral panic” [77] had a halo effect also on products and restaurants people naively thought were guilty in the Covid-19 disease spread, or that were related to the “infected zone.” Such lay interpretation of disease transmission, together with the difficulty of finding reliable information in a first phase of health emergency, has an impact on people’s habits and consumptions, and clear consequences for the local enterprises’ economy. A similar case occurred with the H5N1 Avian Influenza on food consumption, when the poultry industry suffered severe losses due to a sort of “halo effect” in consumer perception of risk, even after the emergency was over [78,79].
Despite these results, with respect to buying behaviours, our data show that generally, most people didn’t actually change their habits, in line with other studies [80]: most people didn’t stockpile goods or increase the purchase of the goods we considered in our survey. Nevertheless, crosstabs show that amongst those who stockpiled goods and increased the purchase of food (fresh, frozen or canned) and disinfection products (in particular regarding home disinfection), there is a significantly higher presence of lower engaged consumers. This evidence is in line with other studies [81–83] that showed how personal reaction to the critical event can feed behavioral changes, with many people making significant changes in their consumption behaviors like anticipating the purchasing of goods [84,85]. As food consumption is recognized as a primary need for individuals, it is strongly influenced by the subjective interpretation of risk and the possible scarcity [86]. For this reason, these results appear interesting in giving a sense of how people orient their food purchase in the case of emergency in relation to their engagement level [87,88]. Furthermore, it appears evident that people with a low level of health engagement, not being psychologically ready to consider the social and public health consequences of their conduct, appear more focused on their own health interests and less keen to rely on health authorities’ guidelines to orient their behaviors [3,89]. For instance, the behavior of stockpiling goods carried out by the less engaged Italian citizens had a negative organizational impact on food supplies, which further compromised the delicate situation of the Italian population. Furthermore, the overcrowding at superstores in the situation of the COVID-19 epidemic was highly counterproductive and contributed to spreading the risk of contagion.
Limitations and future studies
The study measured a specific population’s views at a specific point in time; their beliefs and attitudes reflect the information available at the time and therefore are not stable. Second, results were self-reported and data were collected through a broader continuative online-based survey: measurement errors, unreliable answers and social desirability bias may have partially altered results, as is generally the case in these kinds of studies. Finally, this study relies on several ad hoc items, specifically developed for our research questions but not validated; regardless of our effort to make them clear and non-ambiguous, it is still possible that some participants may have misinterpreted them. Two items in particular (those regarding reduced meals in restaurants and ethnic restaurants) may raise some concerns as participants were not given the option to answer that they never go to (ethnic) restaurants.
Future research should test the Patient Health Engagement Model as a predictor of particular preventive behaviors in different socio-cultural contexts. This model is indeed relatively young and current evidence about its applicability has been carried out mainly by the same research group who developed it. Further studies are needed to consolidate it and to confirm the reliability of the results on the larger Italian population. In addition, it is important to carry out further behavioral research where actual behavior can be measured, not only self-reported.
Practical implications
This study has provided evidence about the role of health engagement as a determinant of citizens’ behavioral change, which is key for controlling the spread of pandemic disease, and has described a conceptual framework–i.e. the Patient Health Engagement Model—in which to better understand these behaviors [90]. In sum, the study shows that health engagement levels are predictive of different responses, both affective and behavioral: playing an active role in health management is associated with a higher chance of performing specific behaviors. In particular, the psychological readiness to assume a proactive role in their own health prevention depends on the individuals’ tendency to be more or less able to comply with health authorities’ prescriptions and to perceive themselves as mainly responsible for their own health and the health of their community. Furthermore, the psychological readiness to engage in health is a crucial factor for explaining the different way in which individuals can cope with their worries about a health emergency. The findings suggest that intervention studies should focus on particular groups and on raising their levels of engagement to increase the effectiveness of educational initiatives designed to promote preventive behaviors. Communication strategies should maximize their impact by targeting messages according to the health engagement levels of citizens. For instance, in order to improve the levels of engagement of citizens in a “psychological blackout,” reassuring messages aimed at sustaining the emotional elaboration of the emergency and related worries would be particularly needed. For those citizens, psychological counselling and positive emotions facilitated by a social campaign also are suggested. To enhance the motivation to stay engaged, citizens in a situation of “psychological adherence” would need positive stories of other persons who succeeded in adhering to the prescribed containment measures. For instance, video testimony of peers able to describe how they successfully coped with the emergency, sharing tips and advice. Finally, people in the position of “Eudaimonic Project,” who were able to develop a new sense of normality despite the serious emergency, can be involved in peer-to-peer communication initiatives, becoming advocates for the correct engagement in adhering to the prescribed measures to face the COVID-19 epidemic. Furthermore, this target group could be further engaged in an open and accountable debate with healthcare authorities to better understand the rationale of some decisions about containment measures and to contribute raising their voice to orient them. Furthermore, fostering the psychological readiness to get engaged in health prevention appears to be a crucial goal for educational and communication initiatives in the event of a health emergency. Carrying out this work now will be invaluable in preparing for this and future pandemics. Listening to consumers’ concerns and expectations in an emergency situation is the base for building a collaborative space where health authorities and civic communities can all contribute to the best management of the situation. Measuring the levels of health engagement of citizens may be considered as a vital parameter for healthcare authorities in order to best orient educational initiatives and supports able to sustain citizens’ adherence to the preventative measures.
Supporting information
S1 Appendix. Survey coronavirus Eng.
Study survey–English version,
https://doi.org/10.1371/journal.pone.0238613.s001
(DOCX)
S2 Appendix. Survey coronavirus ITA.
Study survey–Italian version.
https://doi.org/10.1371/journal.pone.0238613.s002
(DOCX)
S1 File. Coronavirus Eng.
Study original dataset.
https://doi.org/10.1371/journal.pone.0238613.s004
(SAV)
References
- 1. Cao J, Hu X, Cheng W, Yu L, Tu WJ, Liu Q. Clinical features and short-term outcomes of 18 patients with corona virus disease 2019 in intensive care unit. Intensive Care Medicine. 2020.
- 2. Cao J, Tu WJ, Cheng W, Yu L, Liu YK, Hu X, et al. Clinical Features and Short-term Outcomes of 102 Patients with Corona Virus Disease 2019 in Wuhan, China. Clin Infect Dis. 2020.
- 3. Barello S, Nania T, Dellafiore F, Graffigna G, Caruso R. ‘Vaccine hesitancy’ among university students in Italy during the COVID-19 pandemic. Eur J Epidemiol. 2020.
- 4. Godoy P, Castilla J, Delgado-Rodríguez M, Martín V, Soldevila N, Alonso J, et al. Effectiveness of hand hygiene and provision of information in preventing influenza cases requiring hospitalization. Prev Med (Baltim). 2012;54: 434–439.
- 5. Ferguson NM, Cummings DAT, Fraser C, Cajka JC, Cooley PC, Burke DS, et al. Strategies for mitigating an influenza pandemic. Nature. 2006.
- 6. Fenichela EP, Castillo-Chavezb C, Ceddiac MG, Chowellb G, Gonzalez Parrae PA, Hickling GJ, et al. Adaptive human behavior in epidemiological models. Proc Natl Acad Sci U S A. 2011.
- 7. Funk S, Salathé M, Jansen VAA. Modelling the influence of human behaviour on the spread of infectious diseases: A review. Journal of the Royal Society Interface. 2010.
- 8. Fenichel EP, Kuminoff N V., Chowell G. Skip the Trip: Air Travelers’ Behavioral Responses to Pandemic Influenza. PLoS One. 2013;8.
- 9. Leppin A, Aro AR. Risk perceptions related to SARS and avian influenza: Theoretical foundations of current empirical research. Int J Behav Med. 2009;16: 7–29.
- 10. Kass NE, Otto J, O’Brien D, Minson M. Ethics and severe pandemic influenza: Maintaining essential functions through a fair and considered response. Biosecurity and Bioterrorism. 2008.
- 11. Bish A, Michie S. Demographic and attitudinal determinants of protective behaviours during a pandemic: A review. Br J Health Psychol. 2010;15: 797–824.
- 12. Jin Y, Pang A, Cameron GT. Toward a Publics-Driven, Emotion-Based Conceptualization in Crisis Communication: Unearthing Dominant Emotions in Multi-Staged Testing of the Integrated Crisis Mapping (ICM) Model. J Public Relations Res. 2012.
- 13. Zhang Y, Yang H, Cheng P, Luqman A. Predicting consumers’ intention to consume poultry during an H7N9 emergency: an extension of the theory of planned behavior model. Hum Ecol Risk Assess. 2020.
- 14. Mossong J, Hens N, Jit M, Beutels P, Auranen K, Mikolajczyk R, et al. Social contacts and mixing patterns relevant to the spread of infectious diseases. PLoS Med. 2008.
- 15. Lau JTF, Griffiths S, Au DWH, Choi KC. Changes in knowledge, perceptions, preventive behaviours and psychological responses in the pre-community outbreak phase of the H1N1 epidemic. Epidemiol Infect. 2011;139: 80–90.
- 16. Brug J, Aro AR, Richardus JH. Risk perceptions and behaviour: Towards pandemic control of emerging infectious diseases: Iional research on risk perception in the control of emerging infectious diseases. Int J Behav Med. 2009;16: 3–6.
- 17. Poletti P, Ajelli M, Merler S. The effect of risk perception on the 2009 H1N1 pandemic influenza dynamics. PLoS One. 2011.
- 18. Smith RD. Responding to global infectious disease outbreaks: Lessons from SARS on the role of risk perception, communication and management. Soc Sci Med. 2006;63: 3113–3123.
- 19. De Zwart O, Veldhuijzen IK, Elam G, Aro AR, Abraham T, Bishop GD, et al. Perceived threat, risk perception, and efficacy beliefs related to SARS and other (emerging) infectious diseases: Results of an international survey. Int J Behav Med. 2009.
- 20. Brug J, Aro AR, Oenema A, De Zwart O, Richardus JH, Bishop GD. SARS risk perception, knowledge, precautions, and information sources, the Netherlands. Emerg Infect Dis. 2004.
- 21. Vartti AM, Oenema A, Schreck M, Uutela A, De Zwart O, Brug J, et al. SARS knowledge, perceptions, and behaviors: A comparison between finns and the dutch during the SARS outbreak in 2003. Int J Behav Med. 2009.
- 22. Cowling BJ, Ng DMW, Ip DKM, Liao Q, Lam WWT, Wu JT, et al. Community Psychological and Behavioral Responses through the First Wave of the 2009 Influenza A(H1N1) Pandemic in Hong Kong. J Infect Dis. 2010;202: 867–876.
- 23. Jones JH, Salathé M. Early assessment of anxiety and behavioral response to novel swine-origin influenza a(H1N1). PLoS One. 2009;4: 2–9.
- 24. Rubin GJ, Amlôt R, Page L, Wessely S. Public perceptions, anxiety, and behaviour change in relation to the swine flu outbreak: Cross sectional telephone survey. BMJ. 2009;339: 156.
- 25. Chapman GB, Coups EJ. Predictors of influenza vaccine acceptance among healthy adults. Prev Med (Baltim). 1999.
- 26. Ritvo P, Wilson K, Willms D, Upshur R. Vaccines in the public eye. Nature Medicine. 2005.
- 27. Ibuka Y, Chapman GB, Meyers LA, Li M, Galvani AP. The dynamics of risk perceptions and precautionary behavior in response to 2009 (H1N1) pandemic influenza. BMC Infect Dis. 2010.
- 28. Coulter A. Patient engagement-what works? J Ambul Care Manage. 2012.
- 29. Graffigna G, Barello S, Bonanomi A, Lozza E. Measuring patient engagement: Development and psychometric properties of the patient health engagement (PHE) scale. Front Psychol. 2015;6: 1–10.
- 30. Barello S, Graffigna G, Vegni E. Patient Engagement as an Emerging Challenge for Healthcare Services: Mapping the Literature. Nurs Res Pract. 2012.
- 31. Gruman J, Rovner MH, French ME, Jeffress D, Sofaer S, Shaller D, et al. From patient education to patient engagement: Implications for the field of patient education. Patient Educ Couns. 2010.
- 32. Tzeng H-M, Pierson JM. What are the highly important and desirable patient engagement actions for self-care as perceived by individuals living in the southern United States? Patient Prefer Adherence. 2017;Volume 11: 181–191.
- 33. Al-Tannir M, AlGahtani F, Abu-Shaheen A, Al-Tannir S, AlFayyad I. Patient experiences of engagement with care plans and healthcare professionals’ perceptions of that engagement. BMC Health Serv Res. 2017;17: 853.
- 34. Pekonen A, Eloranta S, Stolt M, Virolainen P, Leino-Kilpi H. Measuring patient empowerment–A systematic review. Patient Educ Couns. 2020;103: 777–787.
- 35. Graffigna G, Barello S, Bonanomi A. The role of Patient Health Engagement model (PHE-model) in affecting patient activation and medication adherence: A structural equation model. PLoS One. 2017;12.
- 36. Graffigna G, Barello S, Bonanomi A, Riva G. Factors affecting patients’ online health information-seeking behaviours: The role of the Patient Health Engagement (PHE) Model. Patient Educ Couns. 2017.
- 37. Hill AM, Etherton-Beer C, Haines TP. Tailored Education for Older Patients to Facilitate Engagement in Falls Prevention Strategies after Hospital Discharge-A Pilot Randomized Controlled Trial. PLoS One. 2013.
- 38. Woehrle H, Arzt M, Graml A, Fietze I, Young P, Teschler H, et al. Effect of a patient engagement tool on positive airway pressure adherence: analysis of a German healthcare provider database. Sleep Med. 2018;41: 20–26.
- 39. Zullig LL, Bosworth H. Engaging Patients to Optimize Medication Adherence. NEJM Catal. 2017;14: 1–14. Available: https://catalyst.nejm.org/optimize-patients-medication-adherence/
- 40. Hibbard JH, Mahoney E. Toward a theory of patient and consumer activation. Patient Educ Couns. 2010.
- 41. Laforge RG, Rossi JS, Prochaska JO, Velicer WF, Levesque DA, McHorney CA. Stage of regular exercise and health-related quality of life. Prev Med (Baltim). 1999.
- 42. McDonald EM, Frattaroli S, Edsall Kromm E, Ma X, Pike M, Holtgrave D. Improvements in health behaviors and health status among newly insured members of an innovative health access plan. J Community Health. 2013.
- 43. Harvey L, Fowles JB, Xi M, Terry P. When activation changes, what else changes? The relationship between change in patient activation measure (PAM) and employees’ health status and health behaviors. Patient Educ Couns. 2012.
- 44. Fowles JB, Terry P, Xi M, Hibbard J, Bloom CT, Harvey L. Measuring self-management of patients’ and employees’ health: Further validation of the Patient Activation Measure (PAM) based on its relation to employee characteristics. Patient Educ Couns. 2009.
- 45. Lubetkin EI, Lu WH, Gold MR. Levels and correlates of patient activation in health center settings: Building strategies for improving health outcomes. J Health Care Poor Underserved. 2010.
- 46. Graffigna G, Barello S. Spotlight on the patient health engagement model (PHE model): A psychosocial theory to understand people’s meaningful engagement in their own health care. Patient Preference and Adherence. 2018.
- 47. Zhang Y, Graffigna G, Bonanomi A, Choi K, Barello S, Mao P, et al. Adaptation and Validation of a Chinese Version of Patient Health Engagement Scale for Patients with Chronic Disease. Front Psychol. 2017;8.
- 48. Magallares A, Graffigna G, Barello S, Bonanomi A, Lozza E. Spanish adaptation of the Patient Health Engagement scale (S.PHE-s)in patients with chronic diseases. Psicothema. 2017;29: 408–413.
- 49. Usta D, Korkmaz F, Akyar İ, Bonanomi A. Hasta Katılımı Ölçeği: Kronik hastalığı olan hastalarda Türkçe geçerlik ve güvenirlik çalışması. Cukurova Med J. 2019;44: 1055–1063.
- 50. Kondylakis H, Bucur A, Crico C, Dong F, Graf N, Hoffman S, et al. Patient empowerment for cancer patients through a novel ICT infrastructure. J Biomed Inform. 2020.
- 51. Golubinski V, Oppel E-M, Schreyögg J. A systematic scoping review of psychosocial and psychological factors associated with patient activation. Patient Educ Couns. 2020.
- 52.
Parsons VL. Stratified Sampling. Wiley StatsRef: Statistics Reference Online. Chichester, UK: John Wiley & Sons, Ltd; 2017. pp. 1–11. https://doi.org/10.1002/9781118445112.stat05999.pub2
- 53.
ISTAT. No Title.
- 54. Griffin RJ, Dunwoody S, Neuwirth K. Proposed model of the relationship of risk information seeking and processing to the development of preventive behaviors. Environmental Research. 1999.
- 55. Andrich D. A structure of index and causal variables. Rasch Measurement Transactions. 2014. pp. 1475–1477.
- 56. Masters GN. A rasch model for partial credit scoring. Psychometrika. 1982;47: 149–174.
- 57. Bonanomi A, Osmetti SA. THE RASCH MODEL FOR VICTIMIZATION ANALYSIS: A PROPOSAL OF AN INSECURITY PERCEPTION INDEX. Electron J Appl Stat Anal Decis Support Syst Serv Eval. 2012;3: 75–85.
- 58.
Andrich D, Sheridan B, Lyne A, Luo G. RUMM: A windows-based item analysis program employing Rasch unidimensional measurement models. Perth, Australia: Murdoch University. Perth, WA; 2000.
- 59.
Wright BD, Masters GN. Rating Scale Analysis. Advances in Measurement in Educational Research and Assessment. Chicago, IL: MESA Press; 1982.
- 60. Prieto L, Alonso J, Lamarca R. Classical Test Theory versus Rasch analysis for quality of life questionnaire reduction. Health Qual Life Outcomes. 2003;1: 27.
- 61. Wright BD, Linacre JM, Gustafson JE, Martin-Lof P. Reasonable mean-square fit values. Rasch Meas Trans. 1994;8: 370.
- 62. Bonanomi A, Cantaluppi G, Nai Ruscone M, Osmetti SA. A new estimator of Zumbo’s Ordinal Alpha: a copula approach. Qual Quant. 2015;49: 941–953.
- 63.
Gliem JA, Gliem RR. Calculating, Interpreting, and Reporting Cronbach’s Alpha Reliability Coefficient for Likert-Type Scales. Midwest Research-to-Practice Conference in Adult, Continuing, and Community Education. Columbus, OH, USA: The Ohio State University press; 2003.
- 64. Bentler PM. Comparative fit indexes in structural models. Psychol Bull. 1990;107: 238.
- 65.
Jöreskog KG, Sörbom D, Lisrel VI. Analysis of linear structural relationship by maximum likelihood. Chicago: Scientific Software; 1984.
- 66. MacCallum RC, Browne MW, Sugawara HM. Power analysis and determination of sample size for covariance structure modeling. Psychol Methods. 1996;1: 130.
- 67. Delacre M, Leys C, Mora YL, Lakens D. Taking Parametric Assumptions Seriously: Arguments for the Use of Welch’s F-test instead of the Classical F-test in One-Way ANOVA. Int Rev Soc Psychol. 2019.
- 68. Haberman SJ. The Analysis of Residuals in Cross-Classified Tables. Biometrics. 1973.
- 69. F J., S P., K. MC. Gender, race, and perception of environmental health risks. Risk Anal. 1994.
- 70. Balinska M, Rizzo C. Behavioural responses to influenza pandemics. PLoS Currents. 2009.
- 71. Tooher R, Collins JE, Street JM, Braunack-Mayer A, Marshall H. Community knowledge, behaviours and attitudes about the 2009 H1N1 Influenza pandemic: A systematic review. Influenza Other Respi Viruses. 2013.
- 72. Kristiansen IS, Halvorsen PA, Gyrd-Hansen D. Influenza pandemic: Perception of risk and individual precautions in a general population. Cross sectional study. BMC Public Health. 2007.
- 73. Balkhy HH, Abolfotouh MA, Al-Hathlool RH, Al-Jumah MA. Awareness, attitudes, and practices related to the swine influenza pandemic among the Saudi public. BMC Infect Dis. 2010.
- 74. Barello S, Palamenghi L, Graffigna G. The Mediating Role of the Patient Health Engagement Model on the Relationship Between Patient Perceived Autonomy Supportive Healthcare Climate and Health Literacy Skills. Int J Environ Res Public Health. 2020;17: 1741.
- 75. Barello S, Palamenghi L, Graffigna G. Burnout and somatic symptoms among frontline healthcare professionals at the peak of the Italian COVID-19 pandemic. Psychiatry Res. 2020.
- 76. McCauley M, Minsky S, Viswanath K. The H1N1 pandemic: Media frames, stigmatization and coping. BMC Public Health. 2013;13: 1–16.
- 77. Muzzatti SL. Bits of falling sky and global pandemics: Moral panic and Severe Acute Respiratory Syndrome (SARS). Illness Crisis and Loss. 2005.
- 78. Akben E, Özertan G, Spaulding AD, Saghaian SH. Consumer responses of the H5N1 Avian Influenza: The case of Turkey. Econ Bull. 2008;4.
- 79. de Krom MPMM Mol APJ. Food risks and consumer trust. Avian influenza and the knowing and non-knowing on UK shopping floors. Appetite. 2010;55: 671–678.
- 80. Van D, McLaws ML, Crimmins J, MacIntyre CR, Seale H. University life and pandemic influenza: Attitudes and intended behaviour of staff and students towards pandemic (H1N1) 2009. BMC Public Health. 2010;10.
- 81. Goodwin R, Haque S, Neto F, Myers LB. Initial psychological responses to Influenza A, H1N1 (“Swine flu”). BMC Infect Dis. 2009;9: 166.
- 82. Bults M, Beaujean DJMA, De Zwart O, Kok G, Van Empelen P, Van Steenbergen JE, et al. Perceived risk, anxiety, and behavioural responses of the general public during the early phase of the Influenza A (H1N1) pandemic in the Netherlands: Results of three consecutive online surveys. BMC Public Health. 2011.
- 83. Kamate SK, Agrawal A, Chaudhary H, Singh K, Mishra P, Asawa K. Public knowledge, attitude and behavioural changes in an Indian population during the Influenza A (H1N1) outbreak. J Infect Dev Ctries. 2010.
- 84. Fischhoff B, De Bruin WB, Perrin W, Downs J. Travel risks in a time of terror: Judgments and choices. Risk Anal. 2004.
- 85. Kim HK, Niederdeppe J. The Role of Emotional Response during an H1N1 Influenza Pandemic on a College Campus. J Public Relations Res. 2013;25: 30–50.
- 86. Gstraunthaler T, Day R. Avian influenza in the UK: Knowledge, risk perception and risk reduction strategies. Br Food J. 2008;110: 260–270.
- 87. Jribi S, Ben Ismail H, Doggui D, Debbabi H. COVID-19 virus outbreak lockdown: What impacts on household food wastage? Environ Dev Sustain. 2020;22: 3939–3955.
- 88. Addo PC, Jiaming F, Kulbo NB, Liangqiang L. COVID-19: fear appeal favoring purchase behavior towards personal protective equipment. Serv Ind J. 2020;40: 471–490.
- 89. Nania T, Dellafiore F, Caruso R, Barello S. Risk and protective factors for psychological distress among Italian university students during the COVID-19 pandemic: The beneficial role of health engagement. Int J Soc Psychiatry. 2020; 20764020945729.
- 90. Graffigna G, Barello S, Riva G, Corbo M, Damiani G, Iannone P, et al. Italian consensus statement on patient engagement in chronic care: Process and outcomes. Int J Environ Res Public Health. 2020.