Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

An integrated view on society readiness and initial reaction to COVID–19: A study across European countries

Expression of Concern

PLOS ONE conducted a post-publication assessment of this article [1] due to questions about aspects of the peer review process. In the post-publication assessment, which involved a member of PLOS ONE’s Editorial Board and an external reviewer, concerns were raised about the rigor and appropriateness of the statistical analyses and whether the article’s conclusions are well-supported by the data.

This research was conducted in the first year of the global COVID pandemic. This does not alter the journal’s requirements with regard to research reporting standards and publication criteria, but in this context and given the objectives of the study the PLOS ONE Editors understand that there were constraints which may reasonably have impacted study design decisions. The authors commented that when the article was written, “many insights [had] yet to be developed and data were scarce. This fundamentally influenced our conceptual framework and the chosen methodological approach.”

Even so, based on the outcome of the post-publication assessment, PLOS ONE has concerns about the robustness of the results and the strength of evidence supporting the article’s conclusions due to the study design limitations. Specific concerns include: (a) limited variables were used to represent readiness structures, responses, and outcomes, (b) the data analysis approach did not include a robust statistical analysis, and (c) potential biases in the dataset and study design were not adequately addressed or discussed in the article [1].

Overall, the outcome of the assessment was that the results reported in the original published article did not adequately support the conclusions. The PLOS ONE Editors issue this Expression of Concern to notify readers of these issues and additional information and analyses that were provided in post-publication discussions.

As is discussed below and in S1 File, the authors provided additional context, clarifications, and analyses in post-publication discussions that partially resolved the concerns and lend additional support for the article’s main claims. With this provided, the journal considers that this body of work makes a valid contribution to the published literature, although not all concerns have been fully resolved.

Primary data underlying the study’s results and other supplementary materials (referred to below as “OSF Supp. Materials”) are available at https://osf.io/g69hy/?view_only=9c54713bd17c4dac8c8c1a6104d67d45.

Below, we outline the main concerns raised in this case and the authors’ response to each. For additional information, see S1 File.

  1. The conceptual framework is limited in its scope of what determines society readiness and reactions. The article did not adequately address the rationale for aspects of the study design, or provide evidence to support that the included factors provide a reasonable, reliable representation of the context. For example, concerns were raised about the scientific validity and appropriateness of representing health risk factors by a singular variable (the percentage of population aged 65 or more), and why health risk of age was used as an indicator of society readiness for a public health emergency.
    With regard to health risk factors, the authors commented that when the study was conducted–during early stages of the pandemic–unambiguous research and data on risk factors were limited, the authors viewed age as the only reliable indicator known at that time. The article’s conclusions (final paragraph of [1] discuss this limitation and note that the choice of dimensions should be revisited in the future; for example, future studies could combine characteristics of population (age, gender, obesity) and medical conditions (prevalence of diabetes, cardiovascular diseases, etc.).
    The authors also explained that use of age as a health risk indicator in the study was in line with the conceptual framework of the study: societal readiness seen as demographic predisposition for a more severe outcome of the pandemics and societal reaction in terms of occurrence of government interventions targeting elderly population.
  2. The statistical analyses presented in the article are not suitable to address the aims of this study, or for use in analyzing outcomes that represent a cumulative distribution function. The analyses did not adequately address key confounds, such as under detection of COVID cases, different case definitions, and the timing of epidemic waves and first reports of COVID cases within the country being analyzed. Also, the study did not include sensitivity analyses, and Tables 3 and 5 did not present analysis of variance across the countries in a transparent way. Due to the lack of statistical rigor applied in these data analyses, the claims reported in the article based on the correlation analysis are not adequately supported.
    The authors explained that their approach to this study was not rooted in statistical modeling or an approach where independent variables are evaluated to see how they affect outcomes. Instead, the study was designed based on an approach similar to theory-based evaluation which strives to systematically link concept (dimensions within proposed conceptual framework) and data (indicators derived in operationalization phase). This conceptual analytical model relies on so called ’empirical indicants’; a small subset of possible indicators used to represent a particular concept [2]. The authors further noted that the results are not dependent on or sensitive to the order to dimension inclusions, and that the correlation analysis (Table 4) should be understood only as a “showcase performed to initially check possible redundancy between variables” i.e. Table 4 is not intended to demonstrate relationships between variables based on statistical modeling analyses.
    The authors also provided the following clarifications about the analysis method used in the study:
    • ELECTRE MLO was used as an analytical support because it has been shown to be effective for evaluation of multidimensional phenomena and as an alternative to common composite indexes.
    • This approach is suitable for mix-data type and has meaningful interpretation of performance differences, and ELECTRE based analysis have been previously applied in epidemic and pandemic research [36]).
    • The method was used to demonstrate the usability of proposed framework, and to obtain relative positioning of countries according to the depicted dimensions of readiness and reaction. Only basic elements of ELECTRE-MLO were used due to constraints of the study design (e.g. variables used, data quality, time frame). The outcome reflected in three color-coded groups of countries was used to monitor movements in hierarchy obtained by outranking method and was not intended to offer a “forecasting tool in terms of the initial severity of the COVID-19”.
  3. The article did not adequately address biases in the study design and/or analyses, discuss what steps were taken to address, minimize, or otherwise mitigate effects of bias, or to discuss impacts of potential biases as limitations of the study. For example, the article does not address how global or regional responses might have affected country-level responses, preparedness, or outcomes; the data collected for each country may have been biased due to the relative timings within the study period of within-country COVID exposures and waves of positivity and COVID-related mortality; and issues pertaining to under-detection and differences in case definition need to be addressed in analyses to examine the reliability and robustness of the results.
    The authors commented that per the reported analysis countries’ responses were not uniform despite the global context and information available about responses in other countries. Some governments started with early measures well before their first case. For example, Croatia, Bulgaria, and Norway introduced measures (GovI1) as early as the end of January, whereas others did not take action until their first cases were reported. In an effort to account for differences in country-level exposure timings, the authors defined three sub-indicators related to government measures: timeliness, strictness, and duration of government interventions. The authors noted that although COVID deaths are a lagging indicator, since all countries from the sample were at the peak of infection at a similar time (March-April), death counts declined during May (OSF Supp. materials/Examples/picture 4.).
    Country-level COVID timelines are in OSF Supp. materials/Timeline.
  4. Apple mobility data do not provide a consistent representation of population-level mobility across countries due to differences in iPhone usage. The authors noted that data about iPhone users in the sample range from the lowest in Poland (5% of all mobile phone users i.e. approx. 1.5–2 million users) and Serbia (10% of all mobile phone users i.e. approx. 400.000–500.000 users) to the highest Sweden (50% of all mobile phone users) and UK (50% of all mobile phone users) [OSF Supp. Materials/Apple mobility trends]. They conducted a comparison of Google and Apple tracking data for Poland and Serbia (countries with the lowest portion of iPhone users in the sample) and found a correlation of ~94% (OSF Supp. materials/Examples/figure 6.)). These data should be interpreted with caution since Apple and Google use different methodologies to track movement, but this finding supports that users of iPhones vs. other devices may have comparable mobility patterns.
  5. The study’s main conclusions reported in the Abstract and the ‘Discussion and conclusions’ section of [1] overstate what can be drawn from this study’s data. The penultimate sentence of the Abstract (“Our findings confirm…”) is to be disregarded. The ‘Discussion and conclusions’ section of the article is hereby updated with the following new text, which replaces paragraph 2 (with all three bullet points) and paragraphs 6 to 10 (from “If we recall our correlation analysis…” to the paragraph ending with reference 44).

Discussion and conclusions

Bringing together the diverse aspects which shape society readiness, reaction and outcomes for the COVID-19 pandemic is a challenging research task as many relevant aspects are not yet understood. Following the course of theory-based evaluation we have systematically derived an integrated approach to study preparedness and initial reaction to COVID-19. Based on the body of knowledge and available data in the first phase of the outbreak (from January to May of 2020) we have extracted in total six dimensions, of which three were related to society preparedness and another three to initial reaction to the pandemics. We have proposed a way to operationalize these dimensions by extracting corresponding indicators and conducting a pilot exercise using a sample of 23 European countries. It is important to stress that the proposed approach produced only one view on the initial phase of the COVID-19 outbreak across the observed European countries. Due to many limitations associated with available data and body of knowledge in the early phase of the pandemic outbreak, conclusions of this study should be taken with caution. As the body of knowledge on COVID-19 is continuously growing, but still not released of ambiguity, it is reasonable to say that findings themselves are more in the domain of discussion points than strongly supported standpoints.

The conclusions are built around the finding that in the first phase of the COVID-19 outbreak, society reaction could compensate for its lower preparedness to pandemics. There are some indications that this pattern of reaction was more associated with Eastern European countries in the early days of COVID-19 outbreak. For updated discussion on possible reasons for differences in reactions to the COVID-19 outbreak between Eastern and Western European countries please see S1 File.

Although our six-dimensional framework is not designed to be a forecasting tool in terms of the expected severity of the COVID-19, the chosen structural and process dimensions gave a fair reflection of the actual outcome in the first phase of the outbreak. Still, it should not be seen as a rigid structure. The major challenge is about the operationalisation of the proposed conceptual framework–choosing adequate indicators for each dimension. Even though, we selected indicators based on the results of systematic search, the need to reconcile drawn dimensions with available indicators and data unavoidably simplified the concept. Limitations of the research are outlined further below and in S1 File.

Regarding indicators and data used in this study it is important to emphasize that they were collected in the early stage of the pandemics which brings several limitations of the study. The first one is about the outcome data (morbidity and mortality) used to classify countries in three color coded groups (red, green and yellow, Table 6). As noted in several studies [7, 8], countries use incompatible methodologies for counting infected and deceased patients, which makes comparing data problematic. Similar to other early COVID-19 studies we relied on data from official national sources, but their validity may be questioned by the fact that they were unadjusted and many countries later updated numbers on cases and deaths as reporting caught up or case definitions were amended. As highlighted by WHO [9], excess mortality, even though not unambiguous, is a more objective and comparable measure that accounts for both the direct and indirect impacts of the pandemic (undetected cases, lives lost due the overwhelm of health care system, etc). But, in early stage of COVID-19 outbreak these data records were lagging even more. WHO published first estimates on May 20th and they are still being updated. However, our follow-up analysis based on excess mortality showed that grouping of countries from our sample (yellow, green and red) was not disturbed comparing to unadjusted data (please see S1 File for further elaboration).

Another limitation also coming from scarce data in early COVID-19 analysis is related to the absence of adjusting caseloads to testing ratios. True infection rates remain questionable, since data based on the number of positive tests per day was to some extent influenced by a country’s willingness and ability to undertake testing. This calls for adjusting caseloads to testing ratios. As we have already explained, many of the data at that time were not yet available or valid for all countries (such as prevalence studies for antibodies) and official data without any adjustments were often used in the early scholar papers. We have however acknowledged the importance of testing by including it as one dimension in our framework. Still, this remains the strong limitation of our study and with new and more reliable records being developed (such as prevalence studies for antibodies), another direction for improving outcome indicator is adjusting caseloads to testing ratio.

Another limitation that can be associated with our study refers to the timeline of the outbreak. In an infectious disease where death lags behind cases, the amount of time where vulnerable (nonimmune populations) are exposed to a disease is of great importance. Using the same length of time (16 weeks, Feb—May 2020) for all countries in this study did not significantly impact the outcomes and findings. The peak of infections and deaths in the first wave of pandemics in Europe had occurred during April, so by the end of May, all countries in the sample were at the end of the first wave of epidemic were at the peak of infection at a similar time (see S1 file). Using the same length of time for all countries also brings to discussion the impact of global or regional responses on country-level reaction and outcomes. As we discussed above we were aware that the reaction of the countries could not be observed individually, i.e. that it was related to cases in other countries and that it affected the reaction of the countries. Global response was not fully included in our analysis since it is still a very debatable issue, even labelled as a “silent factor of COVID-19” [10]. For example, as elaborated in [11] several studies on COVID-19 concluded that domestic factors strongly shape policymaking, even when faced with a global threat and when problem pressure is high. Another study indicated that from a comparative perspective, managing the COVID-19 crisis in Europe has been predominantly shaped by distinct country-specific approaches [12]. Despite the complexity and unambiguity, global response remains an important aspect which should be accounted for in further research on holistic frameworks of society response to COVID-19.

To sum up, the availability of data and confirmed factors strongly influenced the operationalization of our proposed framework. However, this study offers a valuable platform that is flexible for further development and adaptations. As new relevant factors are unambiguously confirmed and data records become more reliable the choice of indicators should be enriched in future applications. The choice of dimensions should also be revisited in the future. Pandemics are complex situations and it takes time to collect all the pieces of the puzzle, especially if the analysis is to be expanded to other regions and a larger set of countries.

  1. 6. The article cited a publication (Reference 32) that was retracted prior to the publication of the PLOS ONE article. The sentence that cites this reference (“Moreover, the increased risk of hospital death was over the age of 65 (mortality of 10.0%, vs. 4.9% among those aged ≤65) and current smoking [32].”) is supported by other published works including articles cited in the original article [13,14] and [1517].

Supporting information

S1 File. Supplementary discussion.

Additional discussion of points raised in the post-publication assessment.

https://doi.org/10.1371/journal.pone.0294386.s001

(DOCX)

9 Nov 2023: The PLOS ONE Editors (2023) Expression of Concern: An integrated view on society readiness and initial reaction to COVID–19: A study across European countries. PLOS ONE 18(11): e0294386. https://doi.org/10.1371/journal.pone.0294386 View expression of concern

Abstract

With the wake of the COVID-19 pandemic, the question of society’s capability to deal with an acute health crisis is, once again, brought to the forefront. In the core is the need to broaden the perspective on the determinants of a country’s ability to cope with the spread of the virus. This paper is about bringing together diverse aspects of readiness and initial reaction to a COVID-19 outbreak. We proposed an integrated evaluation framework which encapsulates six dimensions of readiness and initial reaction. Using a specific multi-level outranking method, we analysed how these dimensions affect the relative positioning of European countries in the early stages of the COVID-19 outbreak. The results revealed that the order of countries based on our six-dimensional assessment framework is significantly reminiscent of the actual positioning of countries in terms of COVID-19 morbidity and mortality in the initial phase of the pandemic. Our findings confirm that only when a country’s readiness is complemented by an appropriate societal reaction we can expect a less severe outcome. Moreover, our study revealed different patterns of performance between former communist Eastern European and Western European countries.

Introduction

A major health crisis, such as the COVID–19 pandemic declared by WHO on March 11 2020, is a huge challenge for any society. Such events become even more dangerous if the society itself is in a state of political, cultural or social crisis. This means that public health capacity to respond to such serious threats are significantly limited. Since government measures can have broad societal impacts, it is necessary that members of society comply with them as much as possible [1]. Therefore, pandemics are not only health issues but also political, economic and cultural matters. As elaborated in Lofredo [2] such circumstances can trigger waves of social confusion and panic, uncontrolled waves of contagion, and political and social unrest. Protests in the summer of 2020 against the government’s restriction of coronavirus in Germany, Serbia, Romania, Bulgaria and Israel confirm these fears.

As the number of cases grows, the health system losses its resilience which calls for urgent limitation of the spread of the virus during the early stages of a pandemic. In such a situation, government interventions to support physical distancing take a crucial role in mitigating the severity of the outbreak [35]. This is especially important in the early days of a pandemic when additional capacity still needs to be provided and coping strategies revised. However, it is necessary to broaden the perspective on what makes society capable to manage such an acute health crisis. In line with this, several studies dealing with preparedness and vulnerability in the context of COVID-19 have emphasised the importance of social factors [68]. Our study is nested in this research domain and adds to previous findings by offering an integrated view on countries’ readiness and reaction to the initial outbreak of COVID-19 in Europe. We use the term initial outbreak to highlight that we focused on the first four months of the outbreak (February, March, April and May 2020), given that during this period government interventions in European countries reached its peak and then started to decline.

Instead of focusing on the impact of a particular factor (e.g. social distancing) we gather different aspects of readiness and initial reaction to a pandemic. In order to be able to perform a cross-country analysis on this topic, we have transformed these aspects into a set of measurable dimensions. Our quantitative assessments were compiled based on a prominent evaluation framework for public health emergency preparedness and review of existing COVID-19 related impact studies. To make our conceptual framework operational, we combined well-established country-level indicators and depicted some specific variables associated with COVID-19 on the basis of recently established daily trackers. We ended up with six dimensions. Three of them are about the readiness of the society to fight the outbreak of the virus, while the rest reflect its initial reaction to COVID-19. Finally, we used the multi-level outranking method to investigate the extent to which these dimensions reflect the severity of the initial outbreak of COVID-19 in 23 European countries. This allowed us to rigorously analyse how the European countries stand relative to each other in terms of readiness and reaction to COVID-19.

This paper raises several research questions. The first is whether a better prepared health system of the country and trustworthy institutions could reflect the real outcome. The second is whether the application of more rigorous measures can complement the preparedness capacity of a country. The third question of interest concerns the existence of performance patterns across European countries.

Materials and methods

The conceptual framework

The conceptual basis of our study is inspired by the classic framework "structure-process-outcome (SPO)" [9] which was previously applied in the context of public health preparedness in emergency situations [10]. Based on the SPO framework and the findings of previous studies, we presented the outline of multidimensional analysis as given in Table 1. By structure we refer to the structural elements that existed before the pandemic and were crucial in determining society’s readiness to cope with the pandemic. In the first raw, we draw attention to the health system and its preparedness for a health crisis, but also to the ability of society to cope with such events, as well as to the health risk factors that are present in a certain country. In turn, by process we refer to aspects of the reaction to the pandemic in order to reduce the virus transmission and mortality. In addition to government interventions and testing policies that have been recognized as instruments of the country’s reaction to COVID-19, we have also included behaviour change as an important element of society reaction to the crisis. Finally, by outcome we refer to the severity of the COVID-19 outbreak in the initial phase. The importance of outcome within the SPO is to show whether the system has achieved intended impacts on morbidity and mortality. Since the emergencies in public health are rare, there are few opportunities to include all three elements in assessment of society’s response to the crisis [10]. Also, it takes time to see the whole picture, leaving researchers with narrowed insights while the pandemic is still on the run. Therefore, we chose to access societal readiness and reaction in the initial phase. This gave us the opportunity to connect the outcome of the pandemic (in terms of morbidity and mortality) together with readiness structure and reaction process.

thumbnail
Table 1. Framework for multidimensional assessment of country readiness and reaction to COVID-19 outbreak.

https://doi.org/10.1371/journal.pone.0242838.t001

As we can see from Table 1, the readiness structure and the reaction process include three aspects each, leading to a total of six dimensions. The proposed framework includes both a macro and micro perspective. Institutional aspects are related to the preparedness and reaction of institutions and as such belong to the analysis at the macro level analysis. Societal aspects indicate the behaviour and perception of individuals and they are a phenomenon at the micro level. In turn, medical factors are related to institutions and their policies, but they are also related to the health conditions of individuals.

The choice of relevant dimensions for this study was guided primarily by what we know so far. In this regard, we decided to include those components whose effects have been confirmed in numerous studies on COVID-19. As our research interest was European countries, some aspects such as economic, political or technological are not explicitly included. Some of them partially overlap with the included dimensions (economical as part of preparedness, democracy in high correlation with trust), while some do not have sufficient power of differentiation (such as the similar uptake of information and communication technologies (ICT) across Europe).

The literature is growing, but it is still limited when it comes to the influence of medical and non-medical factors on the spread of COVID-19. In the spotlight are government interventions [11] the preparedness of the healthcare system [12], risk factors [13], economic [14], political [15] and social factors. Among social factors, the one that received the most attention in the context of COVID-19 is the phenomenon of social distancing [16, 17]. Several other authors have focused on social trust [8], social behaviour [18] and the well-being of individuals [19]. Some other non-medical aspects, such as informational overload [20] and the role of ICT [21] also received considerable attention, but their significance has yet to be assessed.

We now proceed to elaborate on the relevance associated with the structure and process dimensions.

Dimensions of the readiness structure–preparedness, trust and health risks.

The main question in a pandemic is what makes a society capable of dealing with such an acute crisis. Answers are usually built around the concept of preparedness and response to virus outbreaks [22]. Although generally considered to be the responsibility of the health system to respond to a crisis, there is a greater understanding that preparedness requires the involvement not only of the health sector but of society as a whole [23]. In a broader sense, health emergency preparedness can be seen as a capability of the public health and health systems, communities, and individuals to prevent, protect against, quickly respond to, and recover from emergencies, particularly those whose scale, timing, or unpredictability threatens to overwhelm routine capabilities [22].

The outbreak of COVID-19 called in question the ability of well-prepared health systems to respond to the epidemic [24]. But the events of March and April 2020 in Italy and Spain reflected that the high level of preparedness was just one piece of the COVID-19 puzzle that should be followed by other compatible pieces, if the puzzle was to be solved.

The willingness of the public to comply to the measure, as proposed by the government, may be crucial for controlling the spread of COVID-19 and accordingly, personal rather than government action could be the most important issue [25]. Therefore, one of the key social elements for the successful implementation of government measures in a state of health crisis is the level of citizens’ trust in the government and its health system. As pointed out by Legido-Quigley et al. [24] "the trust of patients, healthcare professionals, and society as a whole is of paramount importance for meeting health crises". Trust is an important factor because it can influence the public’s judgments of risks and benefits. Decreased trust in the government’s ability to deal with the threat can result in scepticism about public health warnings. The great health crisis of the past has confirmed that without trust in the government, the chances of society overcoming this type of crisis are significantly reduced [3]. Furthermore, trust may indirectly affect the acceptance of implemented measures, as people with higher levels of confidence in institutions are more likely to accept the recommended measures than those with lower confidence levels [26]. Studies related to the influenza A (H1N1) pandemic have already proven that the level of government trust and risk perception of the general public have been of great importance for the adoption of protective measures [3]. Similar conclusions can be found in early studies on COVID–19 [8, 27]. However, the link between trust in government and citizens’ responses to government interventions is not as simple as might be expected. There is some evidence that there could be a positive relationship between low social confidence in government and a strong, determined government response to the pandemic [8].

In addition to preparedness and trust, another important element of the country readiness is the health condition of the population. The literature on health factors associated with COVID-19 is already extensive and covers many different medical problems. Special attention has been paid to various health risk factors that are fertile ground for COVID-19 eruption and spread, such as gender [28], age [13, 29], obesity and race [30], and comorbidities [31]. One of the earliest studies on risk factors for COVID-19 mortality was conducted at the epicentre of the COVID-19 outbreak in Wuhan and included 191 patients [13]. The results of this study showed an increasing chance of hospital death associated with older age. In another study [29], older patients (age ≥64 years) had higher mortality than younger patients (age ≤63 years) (36% vs. 15%; difference). Moreover, the increased risk of hospital death was over the age of 65 (mortality of 10.0%, vs. 4.9% among those aged ≤65) and current smoking [32]. Mortality rates for those who received mechanical ventilation in the age groups 18 to 65 and older than 65 years were 76.4% and 97.2%, respectively [31]. The overall case fatality rate was much higher among the elderly in China, Korea, and Italy [33]. Several statistical sources show that in European countries the hospitalized and deceased patients were elderly people (France: the mean age of hospitalized patients was 68 years and the mean age of the deceased was 79, Italy: 52.7% of COVID-19 cases were >60 years, while 95.4% of the deaths were people aged 60 and over [34, 35], UK: 40% of COVID-19 cases were aged 60 and over; 94% of the victims were people aged 70 and over [36, 37], Spain: 68.5% of hospitalized patients with COVID-19 were >60 years old, while 86.4% of deaths were people aged 80 and over [38, 39]. The data presented on COVID-19 of this large European population support aging as a major risk factor in both morbidity and mortality.

Dimensions of the reaction process-government interventions, testing and mobility reduction.

Timely implementation of control measures is the key to their success, but it is also important to find a balance between early enough implementation to reduce the peak of the epidemic ensuring that they can be sustained in a timely manner [40]. It is fair to say that the rapid and strong reaction of the Chinese government and its success in fighting the virus in February 2020 influenced many other countries in terms of their strategies related to the outbreak of COVID-19. What has happened in China has shown that quarantine, social distancing, and isolation of the infected population can contain an epidemic [25]. Today, in response to COVID-19, many governments use a combination of containment and mitigation activities with the intention of delaying large patient surges and levelling the demand for hospital beds, while protecting the most vulnerable from infection, including the elderly and those with comorbidities [41]. This includes different ranges of contact tracing and self-isolation or quarantine but also varying levels of social distancing and promotion of public health measures. However, it is not easy to detect the individual effect of each government intervention. For example, one study finds that only the effect of lockdown is identifiable, and that it had a substantial effect of 81% reduction in virus transmission [42].

Along with government interventions, extensive testing may also play a major role in the fight against COVID-19. In his introductory speech at a press briefing on COVID-19 in the early days of raging pandemics, the WHO Director General Dr Tedros said: "We have a simple message for all countries: test, test, test" [43]. Mass testing leads to rapid case identification, rapid treatment of these people, and immediate isolation to prevent the spread of the virus [44]. Further, mass and timely testing is crucial to identify people who have come in contact with infected sources so they too can be treated quickly. In this way, it is possible to avoid the need for non-selective quarantines. Piguillem and Shui [45] argue that random testing can even replace the lockdown policy and eliminate the need for nonselective quarantine. Regular screening regardless of symptoms has also been analysed by Grassly et al. [46], who claim that if performed for health care and other key workers, it can prevent a third of transmissions and greatly complement lockdown interventions.

Studies conducted in China [47, 48] and the United States [49] have provided early evidence that social distancing can successfully reduce infection transmission. Reduction of mobility is at the heart of social distancing [17]. It is important that the decision to stay at home is not only the result of an imposed government restriction, but also reflects the self-imposed COVID-19 strategies of avoiding individuals [16, 50, 51]. Several early studies have shown that citizens stay at home voluntarily, rather than in response to stay-at-home orders. Mehari [8] found that the voluntary choice to stay at home was associated to the spread of news about the first COVID-19 case. Engle et al. [52], report that an increase in the local infection rate from 0% to 0.003% reduces mobility by 2.31%, while a government restriction order to stay-at-home reduces mobility by 7.87%. A study from the United States [53] found that every resident, even without government mandates, was able to help slow the spread of COVID-19. Espinoza et al. [54], also argue that individual patterns of mobility established by people may be sufficient to contain epidemics.

Methodological approach

Based on established conceptual framework we developed methodological approach which includes four steps. The first one is on operationalisation of dimensions and entails two sub-steps: 1) establishing indicators and 2) depicting variables for each indicator The step that follows is on sampling and extracting data for each variable. In the third step we have performed a pre-scanning analysis of data based on correlation, analysis. Finally, in the fourth step we have conducted a cross-country analysis using a specific multiple criteria decision making method. Besides data pre-processing and calculations, this final step includes customized visualization of country positioning. Each step is presented in separate subsection.

Operationalisation of dimensions.

In this subsection, we define indicators, variables and data sources for the operationalisation of the dimensions. The proposed approach is summarised in Table 2 and further elaborated for each dimension.

Preparedness is operationalised on the basis of the e-SPAR database, which contains assessments of the country’s capacity to meet IHR—International Health Regulations. e-SPAR stands for Electronic State Parties Self-Assessment Annual Reporting Tool and provides data on global public health security at the country level. It has been used in several studies to provide insight into countries’ preparedness for the COVID-19 outbreak [12, 61]. The 2019 version of e-Spar contains 13 capacity ratings. Each capacity is measured by one to three indicators, which makes to a total of 24 indicators. Similar to Gilbert et al. [12] we excluded capacities for zoonoses, food safety, chemical events, and radiation emergency.

Trust is operationalised around two standpoints: holding a positive perception might increase the efficiency and effectiveness of government operations [62] and sense of confidence in the health system will lead people to comply with the recommendations [63, 64]. Accordingly, we have included two pillars of vertical trust—trust in the government and trust in the public health system; both quantified using data from the last wave of the European Values Study [56].

As we opted for age as a health risk factor, we used data on the proportion of people aged 65 or over. As previously elaborated, at this stage of medical research it is difficult to establish unambiguous conclusions on COVID-19 health risk factors. That is why we decided to use the one that has been proved to be most directly associated with the severity of COVID-19.

In order to operationalise government interventions, our study uses a composite measure that includes eight indicators from the category containment and closure into the so-called Stringency index (SI). SI is part of the Oxford COVID-19 Government Response Tracker [65], a daily updated database on composite measures covering three categories: containment and closure, the economy and the health system. This tracker allowed researchers to extend the analysis to a larger set of countries and explore the link between government measures and other aspects (such as social distancing in several studies [15, 66, 67]). To expand the daily tracker into an indicator of government intervention, we depicted three variables. As we focus on the initial reaction, these three variables (Table 2) represent the timeliness, strictness and duration of government interventions within a defined time frame (from the first case in each country to June 1). The standpoint behind this is that interventions should be implemented early enough, strict enough and long enough.

When it comes to the testing policy, this study does not address who was tested, but only deals with the number of tests performed in each country. Data on the scope of testing are scarce and difficult to obtain systematically. A data source that can be helpful is Our World in Data. This open-source platform contains data on the scale of testing in relation to the scale of the outbreak. It allowed us to build our indicator around WHO recommendation that proposes about 10–30 tests per confirmed case as a general benchmark of adequate testing [68]. In respect to the nature of the available data, we opted for the number of days during the initial outbreak, in which testing was below the suggested benchmark.

Apple’s Mobility Trends Reports were used to quantify the degree of mobility reduction in European countries during the initial phase of the COVID-19 outbreak. Similar to the approach we used in the case of government interventions, two variables are created to reflect strength and promptness of mobility change relative to the first case in each country (Table 2).

Sampling and data.

Regarding the country sample our aim was to collect data for at least 10 Eastern European and 10 Western European countries, depending on data availability. After exploring data sources for the selected indicators and variables, we were able to perform our analysis for 23 European countries (12 Western European and 11 Eastern European) (Table 3). Data on structure (IHR compliance, trust, and risk factors) are the latest available, while data on reaction process (government interventions, testing, and mobility reduction) refer to the period of the initial response to COVID-19 (February, March, April, and May 2020).

Correlation among dimensions.

The correlation, matrix in Table 4 indicates several interesting relationships between dimensions. As expected, there is a positive correlation between government interventions (GI) and several other dimensions. The strongest relationship is between timeliness of GI and promptness of mobility reduction (MR). There is also a strong correlation between duration of GI and promptness of MR and strictness of GI with the strength of MR. This tells us that the MR is a direct consequence of government interventions. However, we can also notice that the promptness of MR is in a strong negative correlation with trust in the health care system. This means that countries with higher level of trust have not reduced their mobility as much as those countries with lower level of trust. Therefore, it is fair to say that mobility reduction is not only a consequence of government interventions, as we discussed above, but also a spontaneous reduction of citizen’s mobility as a personal precaution (in the context of their low trust in the health care system).

It can further be observed that the timeliness of GI is strongly negative correlated with country’s preparedness (IHRc) and with trust in the health system (TrPHS), meaning that countries with lower preparedness and lower trust have implemented measures more promptly. The strength and duration of the GI are also negatively correlated with trust in the government which supports the assumption that government with lower trust reacted quickly and strictly because they feared that their citizens would not comply with the imposed measures. Both trust in the government and in the public health system are strongly positively correlated with the IHRc. In other words, countries with better preparedness have a more trustworthy government and the health care system.

A method for cross-country analysis.

Our study proceeds with fusing different aspects of countries’ preparedness and response to COVID-19 to gain an integrated view of their performance. We use a multiple-criteria decision making approach (MCDA) based on outranking relations. Within this area of MCDM we decided on a specific method and software solution, ELECTRE MLO developed in Petrovic et al. [69]. The method itself has been shown to be easily applicable for cross-country comparative reporting on various topics [7075].

A common idea of the outranking method is to perform pair-wise comparisons and establish binary relations between items (alternatives) being observed. In the family of the ELECTRE outranking methods, these relations determine whether an item is at least as good as another, given that there may be arguments to reject such a claim. An item is better than another if it performs better on a sufficient majority of criteria, which is verified by the concordance threshold. At the same time, the outranking condition may not hold if there is strong opposition to this claim among the remaining minority of criteria. This is controlled by the discordance threshold.

Among the many upgrades of the ELECTRE method, ELECTRE MLO stands out as a method that arranges items into hierarchical performance levels. It introduces a specific procedure by which preferable items are repeatedly extracted. After removing the best performers from the set, in the next iteration, the best among the others are searched, etc. As a result, the items are distributed according to the level of their performance which is visualised in the form of a so-called relation tree. A higher position (level) on a relation tree implies better performance.

In this study, we first arrange countries according to their structural readiness to cope with COVID-19. We then separately include response process indicators to outline the individual impacts of each. Eventually, we include all dimensions and analyse to what extent obtained performance levels correspond to the actual outcome.

In addition to the degree of matching of expected and achieved performance at the level of the entire set, a series of relation trees will outline useful information for each country. These data refer to the vertical mobility of countries—an increase to a higher or a decrease to a lower performance. To analyse this, we have created additional ELECTRE MLO output—visualization of the vertical mobility of countries.

In order to provide the data collected from the ELECTRE MLO (given in Table 3), it is necessary to transform it into performance scores. The scale from 1 to 9 is most often used, 1 is the lowest grade and 9 is the highest. In order to differentiate the topics and avoid interference among related indicators, each dimension is represented by one criterion and equally weighted. For this purpose, trust, government interventions and mobility related indicators are averaged. Concordance threshold is set to reflect that dominance is declared when the unit (here the country) exhibits at least the same performance by 70% of the criteria while and discordance threshold is set to q = 0,3.

Performance scoring is shown in Table 5.

In order to simultaneously monitor and compare the performance obtained by the ELECTRE MLO (expected performance) with the actual outcome (achieved performance), the countries are marked with colours (Table 6). We used three colours—red, yellow and green, each representing the severity level of the outcome (high, middle and low, retrospectively). The level of severity of outcomes is arbitrarily set as follows: countries with less than 1000 reported cases per million residents and less than 30 deaths per million residents are "green"; those with 1000–2000 reported cases and 30–100 deaths are "yellow", while countries with more than 2000 reported cases per million residents and more than 100 deaths per million residents are marked as "red". Data on the number of confirmed cases and deaths related to COVID-19 were extracted from the European Centre for Disease Prevention and Control and refer to cumulative data from the beginning of the COVID-19 pandemic until June 1, 2020 [76].

thumbnail
Table 6. Countries in terms of severity of initial outbreak.

https://doi.org/10.1371/journal.pone.0242838.t006

Results

According to the ELECTRE MLO procedure five relation trees were obtained and presented on Figs 1 and 2. Each country is represented by its flag with a border line coloured according to the outcome of the outbreak at an early phase. The left relation tree in Fig 1 visualises the positioning of countries only on the basis of the tree readiness dimensions while the right is obtained on the basis of all six dimensions. The three relation trees in Fig 2 picture relative position of countries when only one reaction dimension is added to the three readiness dimensions: mobility reduction (relation tree a) in Fig 2), government interventions (relation tree b) in Fig 2) and testing policy (relation tree c) in Fig 2).

thumbnail
Fig 1. Hierarchical order of countries based on readiness dimensions (left) and structure +reaction dimensions (right).

https://doi.org/10.1371/journal.pone.0242838.g001

thumbnail
Fig 2. Hierarchical order of countries after adding different dimensions of the process.

https://doi.org/10.1371/journal.pone.0242838.g002

Relation trees allowed us to investigate how the inclusion of different dimensions affect the positioning of countries with different levels of outbreak severity in the early phase (green, yellow and red countries according to Table 6).

As we noted the first relation tree in Fig 1 reflects the positioning of countries in terms of their readiness: preparedness, trust and the share of the elderly population. Countries with better functional capacities to deal with the acute health crisis, which have more trustworthy institutions (government and public health system) and a younger population are expected to be less sensitive to the outbreak of COVID-19. However, this first relation tree implies that this is not the case. Four of the eight countries that occupy the first three levels are red (Switzerland, Great Britain, Netherlands and Sweden). Although they had a comparative advantage in terms of their structural capacity, they had a more severe initial outcome. The same is true for the bottom levels, where three countries with a less severe outcome can be found (green Czech Republic, Croatia and Bulgaria). Slovakia (green) and Italy (red) are the only two countries in which the structural capacity to deal with an acute health crisis coincides with the severity of the initial outbreak of COVID-19.

We will now look at the effects of the reaction process criteria. When we introduce the criterion of mobility reduction, as expected, the red countries begin move downwards, while the green ones move in the opposite direction (Fig 2a). These changes are slow but evident. With exception of Poland all green countries have slightly improved their relative position. There are still four red countries on the first three levels. Spain is new here due to large reduction in mobility. Low performance by the same criteria ‘pushed down’ the Netherlands and Germany. The addition of government interventions affected the hierarchy more than the change in mobility. Only two red countries retained their positions (Great Britain and Spain), while the rest went down the relation tree. The same goes for the green countries. Due to quick and strict government measures, two managed to maintain their position while the others climbed up. When testing is added (as the fourth dimension) number of levels is reduced from 10 to 8 which implies that it has smoothened differences in structural capacities (it has technically helped some countries to be released from the dominance of their counterparts). For example, this is the case with Poland which was dominated by Great Britain. Due to the significant number of days with testing below the proposed benchmark the red countries fell through the hierarchy. The exception is Germany which climbed tree because its scope of testing is significantly higher compared to many other countries. Again green countries have improved their relative position indicating that the testing criteria are in favour of these countries.

We now turn to the final result which signifies the cumulative effect of societal reaction aspects. When we include all six dimensions, the relation tree becomes most reflective in terms of the severity of the initial COVID-19 outcome. Namely, the positioning of countries in relation to the six dimensions is closer to the positioning in terms of the outbreak severity. As can be seen from the right graph in Fig 1 there are no red countries on the first two levels, nor green on the four bottom levels. Although we did not end up with a perfect ‘reverse traffic light’, the final hierarchy (Fig 2) corresponds well to the actual outcome.

To illustrate the changes in the positions of countries caused by the cumulative effect of different aspects of societal reaction we created a summary graph presented in Fig 3. The graph visualises ‘vertical mobility’ of countries from the initial position in relation to the three dimensions of readiness (dots on the graph) to the final position induced by all six dimensions (country flags). The higher end position on the graph reflects better country performance. Both dots and flags were coloured (in red, yellow and green) to reflect the actual outcome—the severity of COVID-19 at an early stage (Table 6).

thumbnail
Fig 3. Summary graph—Vertical mobility of countries induced by three reaction dimensions.

https://doi.org/10.1371/journal.pone.0242838.g003

Looking at this overall image, we notice that almost all countries have changed their placements on the relation tree. Green and yellow migrated upwards, while the reds slid downwards, thus confirming the simultaneous effect of readiness and reaction variables.

As for the green countries, some of them had an unfavourable starting position due to poorer readiness performance. Despite their unsatisfactory readiness for the epidemic, they managed to improve their position with a prominent societal reaction. This is especially true for Bulgaria, Croatia and the Czech Republic. Comparatively better placement was achieved by those countries that have more favourable initial conditions expressed by the assessment of structural variables, such as Poland and Slovakia.

When it comes to the red countries, we may say that they generally had a good predisposition to cope with COVID-19, but their reactive measures were relatively worse. If we look at green Bulgaria and red Italy as the countries with the lowest rank according to structure dimensions—we can see that Bulgaria has managed to improve its relative position. Unlike Italy, strong government interventions and testing policy has allowed Bulgaria to move five levels higher. The most obvious drop among red countries came from Sweden and Great Britain. Both experienced a cumulative effect of all three reaction dimensions. Neither of the three reaction dimensions had the power to significantly reduce the relative position of these red countries (Fig 2). However, their joint contribution resulted in a significant slide in the hierarchy.

Germany is the only red country which managed to improve its position due to reaching a benchmark in testing. But this was not enough for Germany to move to the upper positions, which also corresponds to the real outcome.

Three (yellow) countries—Norway, Finland and Austria retained their positions, which implies that these countries had a balanced performance in terms of readiness and reaction to the pandemic.

Another observation can be made from the sum graph (Fig 3). All Eastern European countries have progressed thanks to rapid societal reaction. In contrast, almost all Western societies ended low in the final hierarchy, despite good readiness. The exception are Scandinavian countries. Excluding Sweden, they all managed to maintain their relative position.

Discussion and conclusions

There is no doubt that the current situation regarding the outbreak of the COVID-19 is a multidimensional concern. Our study supports this view with an integrated view of impact factors with the focus on the initial outbreak in 23 European countries. At the heart of our analysis is the joint contribution of the six dimensions in dealing with the initial outbreak of a pandemic, which are divided into the structural readiness and the societal reaction. The structural readiness reflects the state in which society awaits the pandemic. It includes three dimensions: health system preparedness, trust in institutions, and health risk factors. The process of societal reaction to an outbreak is also characterized by three dimensions: government interventions, testing policy, and behavioural change. We came up with a set of indicators which reconcile several criteria: the essence of each dimension, specifics of the research area and the time frame of the analysis, as well as the feasibility of data collecting. Our analysis is built on the idea that only when these different aspects come together we can move further to scrutinising the discrepancy between expected and achieved outcome (society’s readiness/reaction vs. severity of the outcome in the early stage of a pandemic).

The outcome of the initial phase of the COVID-19 outbreak clearly showed that having a better prepared health system and trustworthy institutions is not enough to count on success in dealing with the spread of virus such as COVID-19. With the help of specific multiple-level outranking method, we came to the following main findings.

  • The process of reaction to the outbreak to the virus was more important than the structural predispositions of the country. What was even more striking, societies with lower preparedness capacities and untrusted institutions performed better in the initial phase of the COVID-19 outbreak.
  • Government interventions are considered to be the most influential. However, it alone cannot reflect the actual outcome. Only when other dimensions of societal reaction are included, the expected and achieved performance is fairly aligned.
  • The differences between former communist Eastern European countries and Western European countries in reaction to COVID-19 outbreak are evident.

Our findings reveal the full complexity of efforts to bring COVID-19 pandemic under control. Although we have presented the most important dimensions of the societal reaction to COVID-19 they can only be understood in relation to other dimensions. For example, countries with better prepared health systems are also countries with similar political traditions and cultural values. Therefore, it is not easy to explain why their reaction to COVID-19 was weaker compared to countries with different political and cultural background.

We can also question methodologies and metrics for assessing and comparing preparedness. In the era of globalisation and migration, country-level metrics may fail to reflect preparedness at the national-level [6]. What also rises the mist around e-SPAR and similar indicators (Global Health Security Index, Infectious Disease Vulnerability Index) is the rarity of epidemics, which in combination with the exclusive self-evaluation results, and neglect of demographic, socio-economic, and political factors may blur the reflection of a country’s preparedness and vulnerability [6, 12, 77].

There are similar dilemmas when it comes to understanding government interventions. They are mainly focused on social distancing and their scope is actually a measure of the effectiveness of an uptake of the government reaction. However, this is not simple, because social distancing comes not only as a consequence of lockdown measures, but also as a spontaneous reaction of people to the news of the first case, as we discussed in opening sections.

If we recall our correlation analysis, we can see that the strength of government interventions is related to trust in institutions, but in the opposite direction than might have been expected. We found that countries with lower trust in institutions had more stringent and prompt interventions, followed by a better outcome in the initial phase of the outbreak. Our data showed (Table 3) that countries with less severe initial outbreaks had an average trust score of 2.6 (on a four step reverse scale) compared with a score 2.06 in countries with a more severe initial outbreak. It seems that less trustworthy governments had to count on quick lockdown interventions instead of relying on the conscience of citizens’ (to follow recommendations), which in turn could lead to a better result compared to more trustworthy countries. However, although low social trust plays a positive role in the initial months of the pandemic, it is unclear whether this will continue to be the case as the current situation evolves. Low social trust can lead to catastrophic effects when countries reopening is characterized by uncertainty and lack of clear direction, as manifested in the US and some Balkan countries during the summer of 2020.

Another interesting finding is a comparison between Western European and Eastern European countries. In terms of structural readiness, Western countries were in a much better position, but the Eastern European countries had a better reaction to the outbreak. One of the highlighted reasons for this is swift lockdowns and bans evident mostly in central and Eastern European countries also known as former communist countries (Poland, Bulgaria, Hungary, the Czech Republic, Slovakia). The governments of these countries were aware of its weaknesses (vulnerable health system, lower initial testing capacities, and low trust in government) and had to implement early lockdown in order to prevent a pandemic eruption. According to Pancevski and Hinshaw [78], the main cause of this discrepancy is the fact that "the poorer countries of Central and Eastern Europe, fearing their relatively weak healthcare systems would be overwhelmed by the virus, moved more quickly to enact strict social-distancing rules and restrict movement to contain outbreaks".

Unlike Western European countries, Eastern countries began measures long before they reached the 1000th case. Perception of the weaknesses of health system and how trustworthy it is, have also made people in these countries feel more vulnerable and follow the lockdown measures.

However, a better overall preparedness score for Western health systems does not necessarily mean that they are better prepared to fight viruses like COVID-19. For example, when it comes to the number of hospital beds per 100 000 inhabitants, in 2018 the former communist countries ranked 8 out of the top 10 and 11 out of the top 15 in Europe [79]. This is important because about 1/5 of patients with COVID-19 who have moderate or severe disease require hospitalisation [80, 81]. Among hospitalised patients with COVID-19, the percentage of patients who required intensive care units (ICU) care varied from 5% to 10% in Europe [29, 82]. Because COVID-19 treatment does not require ICU for a large number of patients, the former communist countries, which had better hospital facilities, were able to provide the necessary public health care for all patients. Even with regard to the needs of the ICU, Eastern European countries did not face the Spanish or Italian scenario, as they had sufficient primary and secondary care capacity to handle with the moderate and severe outcomes of COVID-19 [83].

There are some other factors which can be important for understanding the different initial outcomes between these countries. There is some speculation about protection from BCG vaccine, which is still mandatory in Eastern European countries [84]. Also, what requires caution in the analysis is the issue of testing, because fewer tests mean fewer identified cases [44].

However, for economic reasons, lockdown is not a long-term solution. Several studies evidenced that European countries will face socio-economic losses due to COVID-19 epidemics [85, 86] and that the consequences will be different in Eastern and Western European countries [87]. Hence, the Eastern European countries have yet to see the seriousness of COVID-19, as measures are relaxed and the capacity of the health system loses support in lockdowns and bans [88]. This is already obvious based on data from the end of July 2020, which show that COVID-19 is again very active in Serbia and some other neighbouring countries [89].

On the other hand, the Western European countries, although not rigid in the first response, will probably cope better with COVID-19 in the future. It is important to remember that this reading of the data is set around statistics on the initial outbreak and that a number of countries have yet to reach the peak of their outbreaks, as implied in reports from October 2020 [90].

Although our six-dimensional framework is not designed to be a forecasting tool in terms of the initial severity of the COVID-19 we can see that the chosen structural and process indicators gave a fair prediction of the actual outcome.

However, our study is not without limitations. The strongest one is about the data. This is especially true for outcome data (morbidity and mortality). As noted in several studies [91, 92], countries use incompatible methodologies for counting infected and deceased patients, which makes comparing data problematic. Moreover, newly established daily trackers of government measures and reduced mobility are still in the process of being redefined and established. Once the data limitation is overcome, we can focus our research on a more detailed analysis of the relationship between the expected and actual outcome. The choice of dimensions should also be revisited in the future. As we explained earlier, pandemics are complex situations and it takes time to collect all the pieces of the puzzle, especially if the analysis is to be expanded to other regions and a larger set of countries.

References

  1. 1. Uslaner E. M. Disasters, trust, and social cohesion. Ritsumeikan language culture study. 2016; 28(2).
  2. 2. Lofredo, M. P. P. Social Cohesion, Trust, and Government Action Against Pandemics. 2020
  3. 3. van der Weerd W., Timmermans D. R., Beaujean D. J., Oudhoff J., & van Steenbergen J. E. Monitoring the level of government trust, risk perception and intention of the general public to adopt protective measures during the influenza A (H1N1) pandemic in the Netherlands. BMC public health. 2011;11(1), 575. pmid:21771296
  4. 4. Chowdhury R., Heng K., Shawon M. S. R., Goh G., Okonofua D., Ochoa-Rosales C., et al. Dynamic interventions to control COVID-19 pandemic: a multivariate prediction modelling study comparing 16 worldwide countries. European journal of epidemiology. 2020; 35(5), 389–399. pmid:32430840
  5. 5. Islam N., Sharp S. J., Chowell G., Shabnam S., Kawachi I., Lacey B., et al. Physical distancing interventions and incidence of coronavirus disease 2019: natural experiment in 149 countries. bmj, 2020 jul; 370. pmid:32669358
  6. 6. Aitken T., Chin K. L., Liew D., & Ofori-Asenso R. Rethinking pandemic preparation: Global Health Security Index (GHSI) is predictive of COVID-19 burden, but in the opposite direction. Journal of Infection. 2020 May 8. pmid:32437727
  7. 7. Wong MC, Teoh JY, Huang J, Wong SH. The potential impact of vulnerability and coping capacity on the pandemic control of COVID-19. The Journal of Infection. 2020 May 28.
  8. 8. Mehari, Y. The Role of Social Trust in Citizen Mobility During COVID-19. Available at SSRN 3607668. May 20.
  9. 9. Donabedian A. Evaluating the quality of medical care. The Milbank memorial fund quarterly. 1966 Jul 1;44(3):166–206. pmid:5338568
  10. 10. Nelson C, Lurie N, Wasserman J. Assessing public health emergency preparedness: concepts, tools, and challenges. Annu. Rev. Public Health. 2007 Apr 21;28:1–8. pmid:17129174
  11. 11. Fang Y, Nie Y, Penny M. Transmission dynamics of the COVID-19 outbreak and effectiveness of government interventions: A data-driven analysis. Journal of medical virology. 2020 Jun;92(6):645–59. pmid:32141624
  12. 12. Gilbert M, Pullano G, Pinotti F, Valdano E, Poletto C, Boëlle PY, et al. Preparedness and vulnerability of African countries against importations of COVID-19: a modelling study. The Lancet. 2020 Mar 14;395(10227):871–7. pmid:32087820
  13. 13. Zhou F, Yu T, Du R, Fan G, Liu Y, Liu Z, et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. The lancet. 2020 Mar 11. pmid:32171076
  14. 14. Fernandes N. Economic effects of coronavirus outbreak (COVID-19) on the world economy. Available at SSRN 3557504. 2020 Mar 22.
  15. 15. Frey CB., Chen C, Presidente G. Democracy, Culture, and Contagion: Political Regimes and Countries Responsiveness to Covid-19. Covid Economics 2020 May 13; 18:1–20.
  16. 16. Morita H, Kato H, Hayashi Y. International comparison of behavior changes with social distancing policies in response to COVID-19. Available at SSRN 3594035. 2020 May 6.
  17. 17. McGrail DJ, Dai J, McAndrews KM, Kalluri R. Enacting national social distancing policies corresponds with dramatic reduction in COVID19 infection rates. PloS one. 2020 Jul 30;15(7):e0236619. pmid:32730356
  18. 18. Morita H, Nakamura S, Hayashi Y. Changes of urban activities and behaviors due to COVID-19 in Japan. Available at SSRN 3594054. 2020 May 6.
  19. 19. Sibley CG, Greaves LM, Satherley N, Wilson MS, Overall NC, Lee CH, et al. Effects of the COVID-19 pandemic and nationwide lockdown on trust, attitudes toward government, and well-being. American Psychologist. 2020 Jun 4.
  20. 20. Tangcharoensathien V, Calleja N, Nguyen T, Purnat T, D’Agostino M, Garcia-Saiso S, et al. Framework for managing the COVID-19 infodemic: methods and results of an online, crowdsourced WHO technical consultation. Journal of medical Internet research. 2020;22(6):e19659 pmid:32558655
  21. 21. Arshad M. COVID-19: It’s time to be thankful to our ICT professionals. Information Technology & Electrical Engineering. 2020;9(2):23–31.
  22. 22. Nelson C, Lurie N, Wasserman J, Zakowski S. Conceptualizing and defining public health emergency preparedness.
  23. 23. World Health Organization. Pandemic influenza preparedness and response: a WHO guidance document. World Health Organization. 2020. https://www.who.int/influenza/resources/documents/pandemic_guidance_04_2009/en/
  24. 24. Legido-Quigley H, Asgari N, Teo YY, Leung GM, Oshitani H, Fukuda K, et al. Are high-performing health systems resilient against the COVID-19 epidemic? The Lancet. 2020 Mar 14;395(10227):848–50. pmid:32151326
  25. 25. Anderson RM, Heesterbeek H, Klinkenberg D, Hollingsworth TD. How will country-based mitigation measures influence the course of the COVID-19 epidemic? The Lancet. 2020 Mar 21;395(10228):931–4. pmid:32164834
  26. 26. Taylor-Clark K, Blendon RJ, Zaslavsky A, Benson J. Confidence in crisis? Understanding trust in government and public attitudes toward mandatory state health powers. Biosecurity and bioterrorism: biodefense strategy, practice, and science. 2005 Jun 1;3(2):138–47. pmid:16000045
  27. 27. Chan HF, Torgler B, Brumpton M, Macintyre A, Arapoc J, Savage DA, et al. How confidence in health care systems affects mobility and compliance during the COVID-19 pandemic. Center for Research in Economics, Management and the Arts (CREMA). 2020. pmid:33057450
  28. 28. Jin JM, Bai P, He W, Wu F, Liu XF, Han DM, et al. Gender differences in patients with COVID-19: Focus on severity and mortality. Frontiers in Public Health. 2020 Apr 29; 8:152. pmid:32411652
  29. 29. Grasselli G, Zangrillo A, Zanella A, Antonelli M, Cabrini L, Castelli A, et al. Baseline characteristics and outcomes of 1591 patients infected with SARS-CoV-2 admitted to ICUs of the Lombardy Region, Italy. Jama. 2020 Apr 28;323(16):1574–81. pmid:32250385
  30. 30. Raifman MA, Raifman JR. Disparities in the population at risk of severe illness from covid-19 by race/ethnicity and income. American journal of preventive medicine. 2020 Apr 27. pmid:32430225
  31. 31. Richardson S, Hirsch JS, Narasimhan M, Crawford JM, McGinn T, Davidson KW, et al. Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the New York City area. Jama. 2020 Apr 22.
  32. 32. Mehra MR, Desai SS, Kuy S, Henry TD, Patel AN. Cardiovascular disease, drug therapy, and mortality in COVID-19. New England Journal of Medicine. 2020 May 1.
  33. 33. Kang SJ, Jung SI. Age-related morbidity and mortality among patients with COVID-19. Infection & chemotherapy. 2020 Jun;52(2):154. pmid:32537961
  34. 34. STATISTA. Coronavirus (COVID-19) death rate in Italy as of August 11, 2020, by age group, https://www.statista.com/statistics/1106372/coronavirus-death-rate-by-age-group-italy/
  35. 35. STATISTA. Distribution of coronavirus (COVID-19) deaths in Italy as of August 11, 2020, by age group, https://www.statista.com/statistics/1106367/coronavirus-deaths-distribution-by-age-group-italy/
  36. 36. STATISTA. Number of coronavirus (COVID-19) cases in England as of July 30, 2020, by age and gender, https://www.statista.com/statistics/1115083/coronavirus-cases-in-england-by-age-and-gender/
  37. 37. UK Office for National Statistics. Deaths involving COVID-19, UK: deaths occurring between 1 March and 30 April 2020. https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/deaths/bulletins/deathsinvolvingcovid19uk/deathsoccurringbetween1marchand30april2020
  38. 38. INED. Spain summary information. 2020 https://dc-covid.site.ined.fr/en/data/spain/
  39. 39. STATISTA. Percentage distribution of people hospitalized due to complications from coronavirus (COVID-19) in Spain as of April 29, 2020, by age group, https://www.statista.com/statistics/1106425/covid-19-mortality-rate-by-age-group-in-spain-march/
  40. 40. Imai N, Gaythorpe KA, Abbott S, Bhatia S, van Elsland S, Prem K, et al. Adoption and impact of non-pharmaceutical interventions for COVID-19. Wellcome Open Research. 2020;5. pmid:32529040
  41. 41. Bedford J, Enria D, Giesecke J, Heymann DL, Ihekweazu C, Kobinger G, et al. COVID-19: towards controlling of a pandemic. The Lancet. 2020 Mar 28;395(10229):1015–8. pmid:32197103
  42. 42. Flaxman S, Mishra S, Gandy A, Unwin HJ, Mellan TA, Coupland H, et al. Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe. Nature. 2020 Aug;584(7820):257–61. pmid:32512579
  43. 43. WHO Director-General’s opening remarks at the media briefing on COVID-19–16 March 2020, https://www.who.int/dg/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19—16-march-2020
  44. 44. COVID-19 science: Why testing is so important By Dr. Eduardo Sanchez, American Heart Association Chief Medical Officer for Prevention https://www.heart.org/en/news/2020/04/02/covid-19-science-why-testing-is-so-important
  45. 45. Piguillem F., & Shi L. The optimal COVID-19 quarantine and testing policies. Einaudi Institute for Economics and Finance (EIEF). 2020.
  46. 46. Grassly N, Pons Salort M, Parker E, White P, Ainslie K, Baguelin M, et al. Role of testing in COIVD-19 control. Imperial College London. 2020 April.
  47. 47. Zhao S, Zhuang Z, Cao P, Ran J, Gao D, Lou Y, et al. Quantifying the association between domestic travel and the exportation of novel coronavirus (2019-nCoV) cases from Wuhan, China in 2020: a correlational analysis. Journal of travel medicine. 2020 Mar;27(2):taaa022. pmid:32080723
  48. 48. Chinazzi M, Davis JT, Ajelli M, Gioannini C, Litvinova M, Merler S, et al. The effect of travel restrictions on the spread of the 2019 novel coronavirus (COVID-19) outbreak. Science. 2020 Apr 24;368(6489):395–400. pmid:32144116
  49. 49. Siedner MJ, Harling G, Reynolds Z, Gilbert RF, Haneuse S, Venkataramani AS, et al. Social distancing to slow the US COVID-19 epidemic: Longitudinal pretest–posttest comparison group study. PLoS medicine. 2020 Aug 11;17(8):e1003244. pmid:32780772
  50. 50. Teslya A, Pham TM, Godijk NG, Kretzschmar ME, Bootsma MC, Rozhnova G. Impact of self-imposed prevention measures and short-term government intervention on mitigating and delaying a COVID-19 epidemic. PLoS Med. 2020;17(7): e1003166. pmid:32692736
  51. 51. Gao S, Rao J, Kang Y, Liang Y, Kruse J. Mapping county-level mobility pattern changes in the United States in response to COVID-19. SIGSPATIAL Special. 2020 Jun 3;12(1):16–26.
  52. 52. Engle S, Stromme J, Zhou A. Staying at home: mobility effects of covid-19. Available at SSRN. 2020 Apr 2.
  53. 53. Badr HS, Du H, Marshall M, Dong E, Squire MM, Gardner LM. Association between mobility patterns and COVID-19 transmission in the USA: a mathematical modelling study. The Lancet Infectious Diseases. 2020 Jul 1. pmid:32621869
  54. 54. Espinoza B, Castillo-Chavez C, Perrings C. Mobility restrictions for the control of epidemics: When do they work?. Plos one. 2020 Jul 6;15(7):e0235731. pmid:32628716
  55. 55. WHO (2019), eSPAR Database, https://extranet.who.int/e-spar
  56. 56. EVS (2018), European Values Study, https://europeanvaluesstudy.eu/methodology-data-documentation/survey-2017/
  57. 57. OWiD (2020), Our world in Data database, https://covid.ourworldindata.org/data/owid-covid-data.xlsx
  58. 58. OxCGRT (2020), Oxford COVID-19 Government Response Tracker, https://www.bsg.ox.ac.uk/research/research-projects/coronavirus-government-response-tracker
  59. 59. OWiD (2020), Our world in Data database, https://ourworldindata.org/grapher/tests-per-confirmed-case-daily-smoothed?time=latest
  60. 60. Apple Community Tracker (2020), Data stream ‘walking’, https://www.apple.com/covid19/mobility
  61. 61. Kandel N, Chungong S, Omaar A, Xing J. Health security capacities in the context of COVID-19 outbreak: an analysis of International Health Regulations annual report data from 182 countries. The Lancet. 2020 Mar 18. pmid:32199075
  62. 62. OECD, OECD. Trust in government, policy effectiveness and the governance agenda. Government at a Glance 2013. 2013.
  63. 63. Gille F, Smith S, Mays N. Why public trust in health care systems matters and deserves greater research attention. Journal of health services research & policy. 2015 Jan;20(1):62–4. pmid:25038059
  64. 64. Bargain O. and Aminjonov U. Trust and Compliance to Public Health Policies in Times of COVID-19. Groupe de Recherche en Economie ThéoriqueetAppliquée (GREThA). 2020.
  65. 65. Hale T, Webster S, Petherick A, Phillips T, Kira B. Oxford covid-19 government response tracker. Blavatnik School of Government. 2020 Apr;25. https://www.bsg.ox.ac.uk/research/research-projects/oxford-covid-19-government-response-tracker
  66. 66. Hussain AH. Stringency in policy responses to Covid-19 pandemic and social distancing behavior in selected countries. Stringency in Policy Responses to Covid-19 Pandemic and Social Distancing Behavior in Selected Countries (April 20, 2020). 2020 Apr 20.
  67. 67. Barberia LG, Cantarelli L, Claro ML, de Paula Moreira N, Rosa IS, Schmalz P, et al. The Effect of State-Level Social Distancing Policy Stringency Measures (SDPS) on Willingness to Distance in the States of Brazil.
  68. 68. OWiD, (2020), Our World in Data—Coronavirus (COVID-19) Testing, https://ourworldindata.org/coronavirus-testing
  69. 69. Petrović M, Bojković N, Anić I, Petrović D. Benchmarking the digital divide using a multi-level outranking framework: Evidence from EBRD countries of operation. Government Information Quarterly. 2012 Oct 1;29(4):597–607.
  70. 70. Petrović M, Bojković N, Anić I, Stamenković M, Tarle SP. An ELECTRE-based decision aid tool for stepwise benchmarking: An application over EU Digital Agenda targets. Decision Support Systems. 2014 Mar 1; 59:230–41.
  71. 71. Stamenković M, Anić I, Petrović M, Bojković N. An ELECTRE approach for evaluating secondary education profiles: evidence from PISA survey in Serbia. Annals of Operations Research. 2016 Oct 1;245(1–2):337–58.
  72. 72. Petrović M, Bojković N, Stamenković M, Anić I. Supporting performance appraisal in ELECTRE based stepwise benchmarking model. Omega. 2018 Jul 1; 78:237–51.
  73. 73. Petrovic M, Bojkovic N, Stamenkovic M, Anic I. A sensitivity analysis of ELECTRE based stepwise benchmarking for policy: The case of EU digital agenda. Management: Journal of Sustainable Business and Management Solutions in Emerging Econo. 2018 Jul 1.
  74. 74. Lazarevic I. Using the ELECTRE MLO Multi-Criteria Decision-Making Method in Stepwise Benchmarking–Application in Higher Education. Operational Research in Engineering Sciences: Theory and Applications. 2019 Dec 16;2(3):77–91.
  75. 75. Petrovic M., Bojkovic, N., & Jeremic, V. Methods for cross-national performance evaluation in logistics, In Quantitative methods in logistics, Faculty of Transport and Traffic Engineering, Serbia.2020; 25–48.
  76. 76. ECDC (2020), COVID-19 data, https://www.ecdc.europa.eu/en/covid-19/data
  77. 77. WHO (2005), IHR Monitoring and Evaluation Tool. https://apps.who.int/iris/bitstream/handle/10665/204368/9789241510172_eng.pdf?sequence=1
  78. 78. Pancevski B and Hinshaw D. (2020) Poorer Nations in Europe’s East Could Teach the West a Lesson on Coronavirus. The Wall Street Journal, 12. April 2020. Available at: shorturl.at/oBI67
  79. 79. EUROSTAT. Data on hospital beds. 2020. https://ec.europa.eu/eurostat/web/products-datasets/-/tps00046
  80. 80. Weiss P, Murdoch DR. Clinical course and mortality risk of severe COVID-19. The Lancet. 2020 Mar 28;395(10229):1014–5. pmid:32197108
  81. 81. Baud D, Qi X, Nielsen-Saines K, Musso D, Pomar L, Favre G. Real estimates of mortality following COVID-19 infection. The Lancet infectious diseases. 2020 Mar 12. pmid:32171390
  82. 82. Murillo-Cabezas F, Vigil-Martín E, Raimondi N, Pérez-Fernández J. Covid-19 pandemic and digital transformation in critical care units. Medicina intensiva. 2020 Apr 15. pmid:32416947
  83. 83. Mishra BK, Keshri AK, Rao YS, Mishra BK, Mahato B, Ayesha S, et al. COVID-19 created chaos across the globe: Three novel quarantine epidemic models. Chaos, Solitons & Fractals. 2020 May 25:109928. pmid:32501378
  84. 84. Curtis N, Sparrow A, Ghebreyesus TA, Netea MG. Considering BCG vaccination to reduce the impact of COVID-19. The Lancet. 2020 May 16;395(10236):1545–6. pmid:32359402
  85. 85. Kraus S, Clauss T, Breier M, Gast J, Zardini A, Tiberius V. The economics of COVID-19: initial empirical evidence on how family firms in five European countries cope with the corona crisis. International Journal of Entrepreneurial Behavior & Research. 2020 May 26.
  86. 86. Lenzen M, Li M, Malik A, Pomponi F, Sun Y-Y, Wiedmann T, et al. Global socio-economic losses and environmental gains from the Coronavirus pandemic. PLoS ONE. 2020;15(7): e0235654. pmid:32645023
  87. 87. Avrămescu TC. Assessments on the Effects of the Coronavirus Pandemic on the Economies of the Central and South-East European Union Countries. Ovidius University Annals, Economic Sciences Series, 2020;20(1), 105–112
  88. 88. Loblova, O., Borbath, E., Rone, J. Coronavirus: why central and eastern European countries seem to be running out of luck, The Conversation, (September 18, 2020). https://theconversation.com/coronavirus-why-central-and-eastern-european-countries-seem-to-be-running-out-of-luck-146349
  89. 89. Associated Press. COVID-19: Surge in new coronavirus cases in Eastern Europe prompts new restrictions, Euro News. July, 12, 2020. https://www.euronews.com/2020/07/12/covid-19-surge-in-new-coronavirus-cases-in-eastern-europe-prompts-new-restrictions
  90. 90. Sangameswaran S, Ahluwalia S. Europe records 100,000 daily coronavirus cases for first time, Reuters, (October, 9, 2020). https://www.reuters.com/article/us-health-coronavirus-europe-cases-idUSKBN26U1Q0
  91. 91. Wolkewitz M, and Puljak L. Methodological challenges of analysing COVID-19 data during the pandemic. BMC Medical Research Methodology. 2020. pmid:32290816
  92. 92. Bartoszek K, Guidotti E, Iacus SM, Okrój M. Are official confirmed cases and fatalities counts good enough to study the COVID-19 pandemic dynamics? A critical assessment through the case of Italy. arXiv preprint arXiv:2005.07271. 2020 May 14.