Agency Preparedness and Regulator Concern

Our expectation that experts would differ significantly as to how prepared regulators are for controlling the risks of nanotechnologies was tested by presenting each respondent with 14 brief descriptions of nanotechnology scenarios. They were asked to rate each using the following scale: “Please indicate whether you strongly disagree, disagree, agree, or strongly agree that current US government agencies are adequately prepared for controlling risks from nanotechnologies in the following categories”. This four-point likert scale indexed 1 as ‘strongly disagree’ through 4 as ‘strongly agree’; also provided were the options: “not familiar with relevant agency or its regulations / can’t answer” and “don’t know / not sure”. Figure 1 illustrates the results for each of 14 scenarios, where points on color-coded lines indicate the mean score on agency preparedness for each expert group (NSE, NEHS, and NREG).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. 'Agency Preparedness' ratings for NSE, NEHS, and NREG expert groups.

Mean scores for each group are indicated with points on respective color-coded lines capturing 14 different nanotechnology scenarios. The dotted grey line indicates the mid or neutral-point between ‘strongly disagree’ and ‘strongly agree’. Significant differences in means were determined using a one-way ANOVA with Games-Howell post hoc analysis, and are indicated with a, b, and c markings as outlined in the legend.

https://doi.org/10.1371/journal.pone.0080250.g001

Across 10 of 14 scenarios, the mean scores for all three groups lie to the left of the centerline, demonstrating consistent disagreement with the claim that federal agencies are ‘adequately prepared’ to control risks from nanotechnologies. Agreement is demonstrated in one case: ‘computers and electronic devices’, by just one group (NSE). This result indicates a prevailing trend towards disagreement (agencies are not prepared for controlling risks) for a majority of the 14 scenarios presented. The NSE and NEHS groups also visibly vary from scenario to scenario in much closer agreement with one another than with NREG, and differ significantly from one another on only one scenario, again, computers and electronic devices. The NSE and NEHS groups are also proximate to the neutral center for several items (medical devices and treatments, pharmaceuticals, industrial workplaces, chemicals and product additives). More striking, however, are the low mean scores for the NREG group, all of which lie consistently to the left of the NEHS and NSE groups, and largely to the left of the ‘disagree’ point on scale. This suggests that those most fully responsible for managing the risks of nanotechnologies regard government agencies as unprepared, more so than their counterparts outside of government regulatory and research agencies.

To confirm that the differences in opinion observed between expert groups are statistically significant, we conducted a between-subjects analysis of variance (ANOVA), wherein significant differences in means were found for 12 of 14 scenarios (p<0.05). This finding confirms that NREG and NSE groups are most dissimilar in their opinions on a majority of items. Differences in opinion are also observed between the NREG and NEHS groups with fewer significant differences across scenarios, and smaller difference in the magnitude of their mean responses. Detailed results are provided in Tables S2 and S3.

To determine whether the difference in mean responses across groups is significant when all 14 scenarios are aggregated, we also created a composite measure (hereafter referred to as ‘Preparedness Index’) using Principal Component Analysis (PCA) with orthogonal rotation (Varimax). One component accounting for 56% of the variance was selected using a scree test [36], and index scores were calculated using the Anderson-Rubin method. The resulting Preparedness Index was found to constitute a consistent and highly reliable scale (Chronbach’s alpha = 0.98). We found a statistically significant difference between expert groups using an ANOVA test with the Preparedness Index (F(2, 251) = 10.216, p = < .001), a finding that is consistent with ANOVA results for individual nanotechnology scenarios reported above. A Tukey HSD post hoc analysis revealed that the Preparedness Index score was significantly higher for NSE (N=114, 0.21 +/- 0.91; p < .001) and NEHS (N=86, 0.02 +/- 0.85; p < .001) groups than for the NREG (N=54, -0.47 +/- 0.99) group. However, there was no statistically significant difference between NSE and NEHS groups (p = .33). Thus preparedness judgments can be said to differ significantly across NSE and NREG, and NEHS and NREG groups only.

Drivers of Perceived Agency Preparedness

Several hypotheses (2 through 5 above) were posed for why experts might differ on their views of agency preparedness. Among those most important to policy discussions is whether or not nanomaterials really are seen as different from their bulk form and so necessitate additional capacity and expertise at the level of regulation. Seeing nanotechnologies as ‘new’ or ‘novel’ for the purpose of testing here (hypothesis 2) was operationalized using an index developed for this purpose, measuring seven dimensions of novelty (listed in Table 1 below). The first component labeled “New and Uncertain Risks” (Cronbach’s α = 0.65) is highly correlated with the variables ‘Properties Cannot be Anticipated’, ‘New Risks’, ‘Risks are Not Well Known’, ‘Risks Cannot be Determined’, and ‘More Uncertainty than Bulk Materials’. The second component is associated with ‘New Benefits’ and ‘Novel Properties’, and is labeled accordingly as “Novel Benefits and Properties” (Cronbach’s α = 0.74). The questions designed to elicit these judgments were developed using face-to-face interviews with US and Canadian nanotechnology experts [31], and are summarized in the table. The two orthogonal components were isolated using a PCA with Varimax rotation followed by a scree test, and orthogonal factor scores were calculated using the Anderson-Rubin method. Factor loadings from the PCA are shown in Table 1. The component “New and Uncertain Risks” is moderately correlated with Agency Preparedness (Pearson’s r = -.479), indicating its potential as a predictor of preparedness perceptions, while “Novel Benefits and Properties” is only slightly correlated (r = -0.01). Both factors are included in the regression analysis below to examine their influence on perceptions of ‘Agency Preparedness’.

Next we tested trust (hypothesis 3) as explaining different views of agency preparedness. Trust has been found to be highly predictive of expert and lay views of the risk and regulation of new technologies in general. We developed a comprehensive index based on responses to the question: “Please indicate how trustworthy you feel the following government agencies are for effectively managing nano-specific environmental health and safety risks from: very untrustworthy, somewhat untrustworthy, somewhat trustworthy, very trustworthy”. Federal regulatory agencies presented included those expected to play a central role in nanotechnology regulation[37], including: US Environmental Protection Agency (EPA), Food and Drug Administration (FDA), Occupational Safety and Health Administration (OSHA), Consumer Product Safety Commission (CPSC). By evaluating trustworthiness of several regulatory agencies in the aggregate this variable provides a robust measure of trust in regulatory agencies in general. As before, the index was created with all four items using a PCA with orthogonal (varimax) rotation, then calculated using the Anderson-Rubin method. Only one component is needed to explain the correlations between all four items, accounting for 69% of the variance (Cronbach’s α = 0.86). The trust index is moderately correlated with preparedness perceptions (Pearson’s r = .263), and so was also included in the regression.

The degree of responsibility expected from, or attributed to, various stakeholders constituted our fourth predictor of views of preparedness (hypothesis 4). To test this hypothesis we developed an index based on responses to the question: “For the following list of groups or stakeholders, please indicate whether they should: Bear none of the responsibility, some of the responsibility, most of the responsibility, or all of the responsibility, for managing risks that emerge from nanotechnologies.” Stakeholder groups included:

1. •. Academic basic sciences and R&D laboratories (ie. Physics, chemistry, engineering)
2. •. Academic environmental and health sciences laboratories (ie. toxicology, epidemiology)
3. •. Private research and development laboratories
4. •. Smaller companies developing nanotechnology products
5. •. Larger companies developing nanotechnology products
6. •. Government agencies (eg. EPA, FDA)
7. •. Environmental groups and non-governmental organizations (NGOs)
8. •. Consumers, through their product purchasing decisions

The ‘responsibility’ index provides a measure of whether responsibility is attributed to a single or narrow set of stakeholders (resulting in a lower responsibility score), versus many or all stakeholder groups (higher responsibility score). This ‘responsibility’ index was created as above with all eight items using a PCA with varimax rotation and the Anderson-Rubin method. Only one component is needed to explain the correlations between all eight items, accounting for 42% of the variance (Cronbach’s α = 0.79). The responsibility index is correlated with preparedness perceptions (r = -.182), and hence was included in the regression analysis to test as a predictor variable.

Finally, we tested the influence of experts’ ‘socio-political values’, measured as social and economic conservatism, on their views of agency preparedness (hypothesis 5). Socio-political values were measured using the following two questions: “The terms ‘liberal’ and ‘conservative’ may mean different things to different people, depending on the kind of issue one is considering. In terms of economic issues, would you say you are: 1- Very Liberal, 2 – Somewhat Liberal, 3 – Somewhat Conservative, 4- Very Conservative, 5 - Don’t Know/Not sure”. The question was then repeated, using social issues. A ‘Social/Economic Conservatism’ index was created based on the standardized z-score of the combined mean responses for these two questions (Cronbach’s α= 0.64). The index indicates potential as a predictive variable (Pearson’s r = .127), and so was included in the regression analysis.

Table 2 provides a summary of descriptive statistics for all four driver variables (novelty, trust, responsibility, and socio-political values) and controls (demographics, domain of expertise) across all three expert groups.

To test the influence of the above four constructed independent variables on perceptions of regulatory agency preparedness for managing risks, a hierarchical ordinary least squares (OLS) multivariate regression was conducted. Using our composite ‘Preparedness Index’ as the dependent variable, independent variables were entered in six steps to investigate their relative power in predicting agency preparedness. Step 1 introduces ‘expert group’ along with demographic variables gender, education, and year of highest degree (as a proxy for age) as control variables. Step 2 introduces ‘domain of expertise’ control variables, including disciplinary field, and Institutional Affiliation. Steps 3 through 6 introduce the socio-political values, trust, responsibility, and novelty variables respectively. Table 3 presents the results of the hierarchical regression.

The resulting final model explained 32% of the variance (adjusted R²) and revealed significant contributions (at the p < .05 level) from four variables: “Novelty: New and Uncertain Risks” (β = -0.40; p < .001, ΔR² = 14%), “Trust” (β = 0.21; p < .001, ΔR² = 6%), “Responsibility” (β = -0.14; p < .01, ΔR² = 3%), and “Social/Economic Conservatism” (β = 0.15; p = .02, ΔR² = 2%). We see that respondents judged agencies are more prepared when they were more conservative, and when they had more trust in regulatory agencies. Conversely, respondents judged agencies are less prepared when they attributed responsibility more uniformly across stakeholder groups, and when they perceived nanotechnology risks as new and more uncertain. Experts relied strongly upon framing of risks (as novel) as a heuristic cue, a finding that empirically demonstrates the link between novelty and risk perceptions, and expands upon recent interview-based research showing substantial differences in experts’ framing of the novelty of nanotechnology risks [32]. The framing of benefits and properties as novel however was not utilized as a heuristic cue, and little variation in views of the novelty of benefits were found between groups (see Table 2). Thus hypothesis 2, that perceptions of novelty significantly affect preparedness perceptions, is supported for the novelty of nanomaterial risks, but not for novelty of properties and benefits.

Trust in regulatory agencies was also a strong driver of preparedness perceptions, supporting hypothesis 3, and reinforcing findings in the risk literature that demonstrate a significant inverse relationship between trust and perceived risk [25]. However, no significant difference in means was observed across groups for this variable (see Table 2). This suggests that trust is limited to ‘within-group’ variation and does not drive observed differences in preparedness perceptions between groups. ‘Trust in regulatory agencies’ can be understood to reflect several possible trust judgments, so it is important to assess what aspect of trust is being invoked. Trust in regulatory agencies to manage risks may reflect, among others, 1) trust in regulatory agencies’ intent to manage risks, 2) trust that regulatory authority and regulatory mechanisms are adequate for the task, or 3) trust that regulators have adequate evidence and a sound scientific basis to take action. A strong inverse correlation between trust and novelty of risks would provide evidence for cases 2 and 3, where the novelty of nanotechnology risks challenge the adequacy of evidence, or appropriateness of existing regulatory mechanisms and authority. We found a small but significant negative correlation between the aggregate metrics of trust and novelty of risk (Pearson’s r = -.128, p < .05, 2-tailed), suggesting that our trust metric is based in part on judgments of regulatory adequacy for managing nanomaterial risks.

Attribution of collective stakeholder responsibility was also found to relate significantly to views on preparedness, supporting hypothesis 4. The attributed responsibility index provides insight into expert’s expectations for stakeholders to manage risks. A high score on the attributed responsibility scale indicates that a greater degree of responsibility is expected from stakeholders overall. It also reflects the judgment that a wide range of stakeholders should play a role in the management of nanotechnologies, rather than one or a narrow set of stakeholders. Attributed responsibility can thus be seen as a proxy measurement for perceived magnitude or complexity of the risk management challenge, where a greater challenge requires greater attention from a number of stakeholders. Hence, when attributed responsibility is high, the management challenge is seen as great, and regulatory agencies (among other stakeholder groups) are perceived as less prepared for managing those risks on their own. Nonetheless, attributed responsibility played only a minor role in overall variance explained by the model (ΔR² = 3%).

The finding on the significant role of social-political views (conservatism) is somewhat contrary to theory suggesting that experts draw upon their expertise and experience, and not upon heuristic cues and value predispositions, when making judgments on risk and regulatory policy [23,24]. The range of socio-political differences across the three groups is small with mean responses roughly half-way between ‘very liberal’ and ‘liberal’. Nonetheless, the regression results weakly support hypothesis 5, and reflect longstanding findings that cultural worldviews, including political ideology, influence expert judgment [10]. Our results also echo recent findings in the nano-risk perceptions literature, where Corley et al. found economic conservatism is inversely related to experts’ support for regulation of nanotechnology [22].