http://orcid.org/0000-0002-1856-0546
Figures
Abstract
Managing rivers in society’s best interest requires data on river condition. However, the complexity of river ecosystems, combined with finite budgets for river monitoring and modeling, mean difficult choices are necessary regarding what information will be available. Typically, decisions of "what to measure" are left to natural scientists. However, knowledge of public appetite for different types of information helps ensure river data is useful to society. We investigated public interest in rivers directly, engaging nearly one hundred urban and rural participants in a combination of focus groups and semi-structured interviews. Drawing on concepts of "final" ecosystem services developed in environmental economics, we moved discussions past commonly mentioned stressors, such as pollution, to actual river features important in and of themselves. Participant feedback reflected extensive thought on river issues, in contrast to a stereotype that the public is ambivalent about environmental conditions. Interests were also broad, encompassing water quality and quantity, fish and wildlife, vegetation, and human features. Results show consolidation around relatively few themes despite diverse sociodemographics. Themes were interpreted into distilled, specific metrics to make public feedback as useful as possible for water resources monitoring, modeling, and management. Our research provides detailed, methodically generated hypotheses regarding river themes and metrics of public interest that should be considered as part of the tradeoffs inherent in river monitoring design. Results compared reasonably well to river attributes emphasized in river restoration environmental valuation reviews, with some differences. Future research could test our hypotheses with large-sample surveys.
Citation: Weber MA, Ringold PL (2019) River metrics by the public, for the public. PLoS ONE 14(5): e0214986. https://doi.org/10.1371/journal.pone.0214986
Editor: Miguel Cañedo-Argüelles Iglesias, Universitat de Barcelona, SPAIN
Received: March 21, 2018; Accepted: March 25, 2019; Published: May 8, 2019
This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Data Availability: Complete transcriptions of all focus groups and validation interviews (with name changes and limited redactions to protect confidentiality) are available through OSF at: DOI 10.17605/OSF.IO/2DU5W; https://osf.io/7w59g/.
Funding: Funded by United States Environmental Protection Agency. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
River ecosystems are multi-attribute public goods. Decisions about what to manage and report on are often left up to biophysical specialists. For example, water resources journals are replete with technical monitoring guidance and insights about attributes to measure. However, the goal of managing public goods necessary involves decisions about public interests in these resources. Our premise is that decisions about how to describe rivers to the public and therefore guide management should be informed by increased knowledge of the specific elements of public interests.
Our approach in this article is to collect preference data by asking members of the general public what is important to them about rivers, via established qualitative research techniques used in such fields as sociology, anthropology, and marketing. This helps document public perspectives so that they can be considered in the difficult choices involved in targeting measurements used to represent river condition. This increases communications potential since the public is more likely to be interested in tailored, accessible information. Likewise, management decisions in the public interest are more likely when a foundation of knowledge is available regarding what is important to a constituency. For example, public preferences could inform priorities for restoration budgets given the broad range of potential outcomes restoration programs might deliver.
We wish to be clear that technical counsel is not the input we sought from our research participants, nor do we believe the public would (generally) be qualified in this regard. Instead, we asked the public to comment, in lay terms, on measurable features ultimately important to them about rivers and streams. Given information on such “goalposts,” technical approaches to measure, model, or predict the desired information could be formulated by natural scientists. If matching information is found to already be available, public feedback helps tailor that information for public audiences. Natural scientists have expressed interest in incorporating public interest in stream monitoring design and monitoring in general [1–3].
Our approach is aligned with the “final” ecosystem goods and services conceptual model [4– 9], hereafter referred to as Final EGS. Whereas ecosystems are composed of multiple interconnected processes and functions, the Final EGS hypothesis is that this complexity can be reduced to a smaller subset of ecological attributes highly relevant to people. These attributes rely on myriad underlying or "intermediate" ecological factors, yet provide a way of focusing natural science and social science attention on key ecological outcomes. People do not necessarily need to understand ecological processes in order to describe the final ecological outcomes directly relevant to them. These attributes are then in turn the currency of interdisciplinary work. Social science efforts of multi-objective planning, decision science, cost-effectiveness analysis, nonmarket valuation, and benefit-cost analysis all rely on articulated attributes; these tools can be deployed to assist in managing public goods for broad public interests, rather than special interest groups.
Piecemeal insight on publicly-valued river metrics is available from multiple sources. For example, nonmarket valuation case studies featuring rivers tend to carefully define metrics and changes to be valued. For an example of a US national scope study, see [10], and for local examples see studies included in the review [11]. However, despite this literature, it can be difficult to gauge the public relevance of metrics used, since an explanation of how metrics were selected and validated, from all the possibilities, is typically incomplete. For example, [10] simply use water quality metrics derived from early US Environmental Protection Agency National Water Quality Inventory metrics [12]. This is emblematic of the nonmarket valuation literature: instead of metrics emerging from exploratory empirical social science study, metrics are most commonly derived from needs of the funder, researcher-driven objectives for the experiment, or information that happens to be available. Thus, although metrics from any given study may seem sensible, it remains unknown which metrics would be the most relevant for the population surveyed. There may be important metrics left out, or people may attempt to extrapolate given metrics to those more relevant to them [7].
Certainly, many nonmarket valuation studies do rigorously test metrics, even taking pains to modify attributes to improve the experiment based on qualitative research findings, e.g. [13, 14]. However, even with such high standards, the number of variables that can be included in any single experiment is limited by human cognition [15]. Furthermore, the manner in which attributes are validated through pre-testing is by no means standardized [16]. Thus, despite availability of case-study literature carefully defining river metrics, there remains a need for a broad, exploratory research.
We are aware of only scant prior research investigating public interest across the spectrum of river attributes. An early study by [17] presented a small number of research participants with existing US Environmental Protection Agency monitoring data for rivers and other ecosystems, and used their input to propose ways of repackaging existing data. The study [18] is the first of which we are aware attempting to build the breadth of publicly valued river attributes de novo, i.e. considering what data to collect without constraint. They convened an expert workshop of natural and social scientists to hypothesize a list of Final EGS for rivers, for a variety of beneficiaries such as industrial or recreational users, and these results also appear in [19]. A subset of these Final EGS were described in more detail for a subset of beneficiaries in [6]. Later, [20] documented Final EGS as defined by the general public, engaging urban residents of southern Arizona.
Our goal in this paper is to continue the example of these studies, providing broad insight and hypotheses regarding river themes important to the respondent population, unconstrained by the agenda of any specific river management project. Furthermore, each theme of public interest is interpreted into one or more specific river metrics. These metrics extend public input to foresee and assist with the decisions needed to collect data at an actual field site, to make the results as relevant as possible for water resource monitoring, modeling, and management. Thus, our results create a bridge between social science and natural science arenas within the water resources discipline. This article continues with the tested empirical techniques of [20], extending the research into the Pacific Northwest, and for the first time including both urban and rural participants. We synthesize findings across the two empirical studies in the Overall discussion, and also provide a broad comparison with the nonmarket valuation literature on river restoration.
Methods
The study setting is the Willamette watershed in western Oregon. Approximately 2.5 million people reside in the watershed, comprising the bulk (65%) of Oregon’s population. The Willamette River drains 2.97 million hectares between the Coast Range and Cascade Mountains before meeting the Columbia River northwest of Portland. The valley is known for agricultural production of grass seed, fruits, vegetables, and grains, as well as significant industrial and public timberlands. The region is noted for temperate, wet winters and dry summers. The Willamette River is ranked 19th in US rivers by annual flow, averaging 1.06 thousand cubic meters per second [21].
Focus groups and semi-structured interviews were utilized for field data collection. These in-depth techniques are ideal to explore and document river attributes important to the sampled residents. Back-and-forth opportunities afforded by qualitative research methods were indispensable for ensuring input on specific, measurable, attributes important in and of themselves (i.e., Final EGS, rather than proxies). The US Environmental Protection Agency Human Subjects Research Review Office reviewed the study and approved it (exempt) with reference number C10-013CS. To recruit participants, we utilized a purposive sampling strategy which allowed us to gather perspectives from typical residents, but also explore similarities and differences between population segments. We used five mutually exclusive segments as follows: urban low-income; urban recreationalist (and not low-income); urban non-recreationalist (and not low-income); rural farmers; and rural non-farmers. Any given person could qualify for one and only one of these segments. Two geographic areas of recruitment within the watershed were designated for each of the five population segments to keep the segments from reflecting solely a single area. Urban recruits came from both Portland and Corvallis, and rural recruits came from both the Linn-Benton and the Clackamas-Marion county areas.
The rationale for separating urban from rural groups was to investigate whether the so-called “urban-rural divide” would manifest in divergent preferred river attributes. Within the urban population, recreationalists were separated in the event they might have more specific river preferences due to their activities, or would tend to overwhelm input from others. We separated low-income persons in order to examine whether there were any particular river preferences particular to disadvantaged persons. Urban non-recreationalists were separated since it is difficult to hypothesize what river attributes this group might prefer since they have no apparent special engagement with the resource. In considering rural residents, our anecdotal information was that farmers felt over-regulated by environmental policies designed to protect rivers, thus they might have a different perspective. Rural farmers were separated from non-farmers as a way of distinguishing two recognizable but separate segments of rural life, since most rural residents of Oregon do not partake in or gain significant income from farming (personal communication, Bruce Weber, Oregon State University Rural Studies Program).
Population segment definitions were as follows. The low-income criterion was defined as 185% or less of the US Census poverty threshold by size of household [22, 23]. Recreationalists were defined as persons being on or near bodies of fresh water for the purposes of recreation on average once per month. Rural was defined using the US Census definition of residence in a community of 2,500 persons or less in open country. Farmers were defined as those self-identifying as receiving significant household income from farming, with non-farmers self-identifying as not receiving such income. Table 1 summarizes the population segments, screening criteria, and recruiting locations.
A marketing firm was contracted to recruit the participants, relying mainly on cold calls from a random sample phone list for each defined region. Internet notices and flyers in local libraries were used to supplement cold call recruiting. Farmer groups proved especially difficult to recruit; only for this population segment were a few referrals permitted (for other farmers). To reduce self-selection of those with a particular interest in river and stream issues, the recruiter offered an incentive payment of $75 to attendees. A screening script was used in which the recruiter was not permitted to describe the topic of the meeting as being rivers and streams. Our rigorous recruiting measures are a strength of the study, contrasting with “open-door” meetings, or inviting predefined “engaged” stakeholders. The intention was that input received would be more likely to reflect broad-based issues within diverse population segments rather than being overtaken by potentially narrow interests. To further diversify input, there were at least two groups per segment and at least one for each geographic area. All groups were moderated by the lead author for consistency, and convened in local library branches, to offer comfortable and neutral surroundings for participants.
An open-ended script comprised of relatively few questions structured the meetings (S1 Appendix). The script follows a common “funnel” design of warm-up, intermediate, and main questions [24]. The questions asked participants about rivers and streams in western Oregon in general, as well as about the Willamette River in particular. Focus groups were approximately two hours in length. Brief background as well as photo visual aids for the Willamette River were provided halfway through the meetings (S2 and S3 Appendices). Photos were selected to display a variety of conditions on the Willamette River mainstem, to seed discussion. Photo A is by Miles Hochstein (https://portlandground.net), used with permission. Photos B, C, D, and F were from US Environmental Protection Agency stream monitoring files. Photo E is from [25], used with permission. We carefully selected background information and photos to offer various portraits of the resource, yet any choice of information provided necessarily incurs a bias. We believed the advantages of visual and text information to seed respondent reactions would outweigh the disadvantages, since many participants might not be closely familiar with the resource. A worthwhile avenue for future work would be to repeat the study with different background material and photos.
Focus groups were tape-recorded and transcribed, providing a tremendous dataset of over 300,000 words for analysis. Participant input was distilled through an iterative process of reading transcripts to identify emergent themes corresponding with measurable attributes of rivers and streams. In all, there were more than 2,500 code occurrences for measurable features. We report code frequencies to convey a sense of the relative importance of different themes, although there is debate regarding this practice [26] (p.185). We employ qualitative techniques for their hypothesis generating potential [27, 28]; our code count results are not meant to imply an ability to statistically test hypotheses or generalize findings to the broader population. Like many qualitative studies, we provide a depth of input with an intensively studied small sample, and investigate variation by deliberately engaging diverse participants. We view our study as part of a vanguard of exploratory research on empirically derived river metrics, to provide a reference for other research deriving or utilizing river metrics, including large-sample quantitative approaches to verify public priorities suggested by our code count frequencies. For example, choice experiments techniques can be used to rate the relative importance of different attributes, rather than quantify the value of changes in given attributes [29, 30].
To systematically code transcripts for frequency analysis, turns of talk were used as a unit of text [31] (p.105). A block of text in which a participant was briefly interrupted, but then continued speaking was treated as a single unit of text for that speaker. A given theme could be coded only once for each unit, but could be coded multiple times if the speaker returned to a topic repeatedly throughout a transcript. Many separate themes could be coded within a given paragraph, for example the phrase “endangered fish” would require citing an occurrence for both the fish theme and the endangered species theme. Rare measurement themes raised only by one person or arising in only one focus group are not reported. Certain recreational activities mentioned by participants clearly imply river measurements; such cases are noted in the results. To provide additional detail in how different measurement-oriented themes were raised, several emergent context codes were tracked (as opposed to measurement codes). These context codes are not tabulated to save space, but are described in the text, with full results available from the authors upon request. Coding and analysis were facilitated by The Ethnograph (v6) software [32].
After initial analysis of focus groups, themes of public interest were interpreted into specific metrics of sufficient detail to guide river data collection. Subsequently, an additional phase of data collection we refer to as validation interviews occurred to apply a form of "member-checking" to the metrics [33, 34] (p.201). These new participants were provided with a summary sheet of river measurements for their reaction. In addition to their open-ended reactions, interviewees were asked to highlight five measurements that they would emphasize, as well as five measurements they would drop if resources were constrained, to assist in identifying priorities. Participants were again recruited with the assistance of a marketing firm with the same protocol as with focus groups. The semi-structured validation interviews were approximately one hour, conducted by telephone. Interviews were transcribed and coded using the same techniques as for focus groups.
Results and discussion
Sociodemographics for all population segments are shown in Table 2, illustrating participant diversity. Final EGS themes (Table 3) were organized in a structure as follows. The first branch of the hierarchy separates inert and live features, with Water and Channel for the former, and Vegetation, Fish and Wildlife, and overarching Biota issues for the latter. Water is further split into subcategories of Quality versus Quantity. These categories reflect the breadth of participant input. For example, the Biota category was needed due to the emergence of themes such as nuisance species and native species that apply across different types of wildlife and vegetation. No attempt was made to limit participants to input on ecological, or ‘natural’ features, and we also coded input on numerous anthropogenic river attributes. To distinguish these, results are divided into Final EGS versus Human features. This helps isolate the ecological component of rivers important to people as opposed to onsite human modifications serving as either amenities or disamenities. The methodology flow is represented in Fig 1.
Each segment sums to 100%.
Code frequencies (Table 3) are normalized to within-segment percentages for each theme, for focus group and validation interview participants combined. Normalization puts data from each population segment on equal footing as compared with other segments. Measurement themes and corresponding metrics and shorthand are listed in descending frequency within category in Table 3. Frequencies for urban versus rural participants are graphed in Fig 2; validation interview feedback is included in the appropriate urban or rural segment, and also broken out as a separate series. Frequencies for the top ten themes shown in Fig 3. In the ensuing discussion, additional context is given for each theme, with attention to where population segments differed, as well as any relevant insights from validation interviews.
Validation interviews included in urban and rural results and also broken out as a separate series. Bars sum to 100% for each series.
These themes account for the majority of combined input (bars sum to 51%).
Water
Water quality.
We isolate nine distinct Final EGS themes and metrics associated with water quality. Foremost, participants were highly aware that rivers are a source of drinking water, and showed strong concern about water supply contamination. Despite sophisticated water treatment, concern remained for treatment system vulnerabilities (WQL2). There were also a number of comments pertaining to the safety of drinking water directly from the stream, tabulated as a separate metric (WQL3). A “drinkable” river water quality is the top tier in the classic water quality “ladder” used in economic valuation of water quality improvements such as US Environmental Protection Agency regulatory impact assessments [35], however achieving this tier has not been prominent in actual valuation studies.
Nearly equal to the combined interest in water supply quality found in WQL2 and WQL3, was concern about the safety of recreational contact, represented in WQL1. Nearly all of these comments reference swimming or submersion, with a few quotations each for boating, wading, or unspecified contact. Swimmable and boatable water quality levels are familiar attributes in the literature, also reflected in the classic water quality ladder [35, 36]. A majority of validation interviewees selected safety of recreational contact as one of their top five: far more votes than any other metric received.
There were several comments having to do with visible water quality. Water clarity was preferred, with many comments on esthetics (WQL4). Clarity as a desired surface water attribute is supported by many studies, e.g. [37, 38]. However, in our data it was apparent that clarity was also thought of as signaling other water quality concerns, such as the safety of the water supply or the safety of body contact; in other words, other Final EGS. In such cases the final outcome of interest was tallied instead of clarity.
A separate theme for visible flow was coded due to negative comments regarding “stagnant” rivers (WQL5). A final theme regarding visible water quality was appreciation of a view of water (WQL7), including waterfalls.
Remaining water quality themes have to do with perceptions other than health or visible quality concerns (WQL6, WQL7, WQL8, and WQL9). When temperature of river water was discussed (WQL6), this included recognition of cold water being a more natural water quality status. Notably, there are more than 3,000 US waterbodies listed as impaired due to temperature pollution [39].
Water quantity.
The single most prevalent water theme was concern for property damage by flooding (WQN1), as easily seen in Fig 2. This was especially true of farmer participants, who described firsthand experience with high flows impacting their land alongside rivers, and perceived lack of ability to take erosion control measures. There were also numerous mentions of property damage by urban groups, yet it was elevated to such a degree for rural groups that we mark it as an urban/rural difference.
About 20% of flooding-related quotations did not involve property concerns. Instead, the emphasis was on flooding as a positive ecosystem feature or simply as a notable phenomenon. These comments led to our inclusion of a metric gauging the extent of flooding as compared to past conditions (WQN3). Farmers never described positive aspects of flooding, despite some non-farmers perceiving flooding as a boon to farmland productivity. There were numerous comments from non-farmers regarding “poor planning” of investing in infrastructure such as crops or buildings within a flood zone, usually in reaction to photos E and D. In contrast, farmers were more likely to view photo E as a situation out of the landowner’s control.
A separate theme within water quantity was oriented to water supply (WQN2). The necessity of water for municipal needs such as drinking water was most prevalent, closely followed by the need to supply agriculture, with industrial water needs last.
The last theme under water quantity reflects a dozen comments on the importance of rivers for supplying hydropower opportunities (WQN4). There were also anti-dam comments; upon follow-up from the moderator these were found to be connected to Final EGS themes such as fish, or flooding as a positive characteristic.
Channel
The largest channel theme was suitability for navigation (C1), mainly in association with recreation. The context was participants expressing an interest in activities such as boating, rafting, canoeing, kayaking, or tubing, which implied an interest in navigability. There were less than a dozen quotations describing actual boating hazards, which tended to be from rural participants. About 25% of the C1 quotations mention the importance of rivers for commercial transport such as commodities.
Swimming was another frequently mentioned activity, voiced twice as often by rural as urban participants. Interest in swimming implies a measurement of sufficient channel depth (C2). A safety concern was hazardous currents for swimmers (C6). People also frequently described difficulty accessing the water’s edge: places with an easier natural access such as a beach-type shoreline were preferred for recreational activities (C4). The presence of areas suitable for camping was appreciated, such as flat ground on the riverbank (C5). Comments on developed campground facilities were coded under Human features.
There were numerous comments regarding erosion that were not related to property damage or water clarity concerns (C3). Reactions to photo E and F account for most occurrences of this theme. The context was usually unspecified. Some simply disliked the esthetic, others worried that erosion would be bad for the “health” of the river. Downcutting and reduced bank stability from human impacts is a well-known issue among river ecologists [40].
Germane to channel preferences were a number of comments regarding downed wood in the waterway. Due to the importance of woody debris for habitat purposes and river restoration efforts, public perception of downed wood has been a topic of survey research [41–43]. In general, our qualitative findings support previous results for Oregon, although our quotations are useful for understanding detailed perceptions. All of the positive comments regarding downed wood occurred after photos were distributed and directly referenced the woody debris on the shoreline in photo B. The context was that the wood signaled a natural condition, e.g. fish habitat. For the people noting downed wood as a negative characteristic, some thought that woody debris could impede fish passage, for others it represented a form of property damage, and two people wondered whether it indicated logging in the area (and would thus be unnatural). The code frequency of woody debris as a Final EGS and associated metric (C7) is reserved for when it was not a proxy for other attributes, but perceived as a natural condition.
Last in terms of channel characteristics, rocks and outcrops in association with rivers were of interest to several persons (C8).
Biota
Several themes apply to both fauna and flora; thus the overarching category of Biota was required. The central finding was strong interest in native species. Native species was coded not just for the term “native” but also terms or phrases such as “indigenous,” or wildlife “supposed to be there.” There was an even larger number of comments disliking invasive species, with a worry of them “taking over” native species. In addition, there were numerous references to endangered or extinct species, by definition a reference to native species. Finally, there were several occasions of disliking nonnative species regardless of whether they were invasive or not. All these comments were distilled into two themes and metrics, one theme on invasive species concerns and native species appreciation (B1), and one theme for endangered species (B4).
The context of native species appreciation comments was about one third generalized, one third in regards to fish in particular, and one third split across various plants and wildlife. For invasive species, most comments pertained to invasive wildlife, mainly fish, followed closely by invasive plant references, followed by comments unspecific to either animals or plants. Endangered or extinct species were mentioned by name infrequently. The few mentioned were salmon, birds, and mammals. More than half of the references to endangered or extinct species were made by validation interviewees. Furthermore, all interviewees were asked whether they were more interested in information on wildlife that could be hunted or fished, or endangered wildlife (S1 Appendix): twice as many chose the latter. Endangered species issues are sometimes considered controversial, and it is possible that the individual interview data collection phase allowed participants more freedom to voice their opinion [44]. It should also be noted that the background information (S2 Appendix) mentions nonnative fish and endangered fish. This likely influenced the frequency of these codes.
There was a recurrent but minority view to the effect that native, invasive, and endangered species concerns were not that important. The frequency of this view, tracked as a context code, was at 10% of the corresponding themes. These minority sentiments declared historical status was not necessarily preferred today; it was a lost cause; and "survival of the fittest" regardless of which species arrived first.
Overall public support for protecting endangered species has been documented by other authors relying on survey results [45, 46]. Public affinity for native species in general has also been documented elsewhere [47, 48]. Studies investigating invasive species preferences have shown mixed opinions [49–51], whereas our results show only a few comments dissenting from an anti-invasive stance.
As if to balance the affinity for biota, there was also a theme for biota that was a nuisance or dangerous to people or human property, or that depleted game herds (B2). Most of the nuisance comments referenced wildlife, mainly predators such as wolves or cougars, followed by birds such as seagulls and geese that were annoying or were perceived as leaving large amounts of waste, followed by mosquitoes. There were only about a dozen nuisance plant comments. Overall, the sentiment on nuisance species was not that they should be eradicated, but rather: (1) they should be kept away from areas where they could harm people, (2) populations should be kept under control, or (3) one may simply wish to know what conditions to expect when considering recreation in a particular area.
A metric for sensitive species was created due to numerous concerns about the physical condition of animals or plants (B3). Almost all of these references were in regards to pollution-induced mutations in fish or amphibians such as frogs. In keeping with our coding protocol, such references were cross-coded, that is, concern about fish mutations would appear in the sensitive species theme, as well as the fish theme.
Interest in biodiversity is represented, but with a very small number of quotations (B5). The context of the comments seems to be related more to species diversity than genetic diversity, thus metric B5 represents the sum of appreciated species from Vegetation, and Fish and Wildlife categories. However, the topic was not prevalent overall, and further research is warranted on this complicated issue. Other authors have found that the public in general supports protecting biodiversity when the question is posed directly in both survey and focus group formats [52, 53].
Fish and wildlife
Aquatic wildlife.
Fish was by far the largest category for any type of aquatic life. Within this broad topic, game fish was the largest Final EGS metric (AW1), with a conspicuous code frequency (Figs 2 and 3). Most of the quotations were general references to the activity of fishing, implying an interest in game fish. Approximately half of these comments referred to salmon in particular. Salmon were also frequently mentioned in contexts besides fishing, indicating interest in salmon as a charismatic species, although we include such comments within the game fish Final EGS. Within game fish, there were also comments on hatchery fish. Rural participants tended to describe their preference for natives but their support for hatcheries in providing fishing opportunities. Urban participants tended to see hatchery fish as unnatural and suspected they harmed the natural ecosystem.
There was significant concern regarding safe consumption of fish (AW3). There were several references to varying fish taste (AW5).
A separate fish theme was made for comments mentioning fish that were not game fish, or generalized references to fish that lacked a clear fishing or fish consumption motivation. The "All fish" metric (AW2) was relatively large even at half the frequency of game fish, ranking second within aquatic life.
Various aquatic life besides fish was also mentioned, but to a much lesser degree, necessitating the relatively broad metric AW4. There were several mentions of unspecific aquatic life, but since these could not be associated with a specific metric, they were not tabulated.
Land wildlife.
There were numerous vague references to “animals” or “wildlife,” or even just “habitat.” While this displays interest in wildlife, just as vague references to aquatic life were not tabulated, neither were these. Only when the moderator was able to probe participants for something more specific could input guiding a specific measurement be captured.
The most frequently cited specific form of land wildlife was birds (LW1). The largest specific subtheme was birds of prey, such as eagles and ospreys. References to other large birds such as herons, ducks, and geese also occurred but less often. The remainder, i.e. most of the bird codes, were unspecific, with only a few bird types mentioned.
Discussed nearly as frequently as birds were mammals (LW2). Interest in insects (LW3) and miscellaneous other forms of land wildlife (LW4) were minor themes.
When people were specific, large wildlife was more prevalent, as seen in the fish, bird, and mammal categories. There are at least three explanations for this. One is that larger wildlife is more visible and thus more charismatic. Indeed, the endangered species treated in nonmarket valuation studies tend to be easily visible and well-known types of animals [45]. Second, larger wildlife is more useful to people as a source of game. There were less than a dozen occasions of people mentioning hunting; however, it is possible that the association is subconscious. A final explanation is that larger species are often higher on food chain and thus represent keystone species, representing a healthy ecosystem overall. For mammals, we tracked interest in predators as a context code, which applied to several people. This included discussion about how predators keep ecosystems balanced. The perspective of predators playing a key role is well-supported in ecology [54].
Vegetation
As with wildlife, there were many references to “plants” or “vegetation” with no further specifics, thus these vague mentions are not tabulated here. Only when the moderator succeeded in eliciting more detailed feedback could a measurable Final EGS be tallied. The most prevalent specific vegetation category was trees (V1). About a third of these quotations simply expressed liking trees. Positive impact on home value with increasing canopy vegetation has been documented in the area [55, 56]. About a fifth of the comments expressed a dislike of logging. The Pacific Northwest is relatively famous within the US for environmental controversy surrounding logging and clear-cutting [57], with a debate including but not limited to maintaining habitat for the endangered Northern Spotted Owl. There were a dozen occasions of trees mentioned as important buffer vegetation along streams. There were several mentions of the importance of trees shading streams for ecological reasons, several preferences for big, old trees, and a handful of comments appreciating the shade trees provide for people. Notably only ten tree comments mentioned specific tree types.
The second theme under vegetation, bank vegetation (V2), captures diverse comments. Appreciation of the greenery or lush vegetation along rivers was the most common (typically referencing natural vegetation, but also manicured), followed by citing the importance of riparian vegetation in providing a buffer, and handful of mentions of naturally occurring grasses. Relatively small themes were dislike of algae (V3), appreciation of wildflowers (V4), and interest in edible plants (V5).
Human features
Many of the measurable features of rivers and streams that people described were clearly human controlled rather than ecological. Dislike of pollution permeated participant feedback, however, pollution is a stressor; thus our follow-up questions were designed to learn what ecological characteristics people worried about being damaged by pollution. Such follow-up interactions account for many of the Final EGS code occurrences inventoried in this article. Although we do not tabulate pollution as a theme, we did track several pollution context codes. Industrial, municipal, and agricultural pollution were all common, with agricultural pollution being somewhat less cited.
Recreational amenities were the most frequent measurable human feature of interest (H1). Activities of hiking, walking, or biking were taken to imply an interest in trails. Direct and implied interest in trails was the largest subtheme, constituting half of the amenity quotations, coming almost entirely from low-income and recreationalist segments. About two-thirds of the trail-related comments imply unpaved surfaces, e.g. references to hiking, while the remainder imply paved surfaces, e.g. urban “riverwalk” areas and bike paths. The second half of the recreational amenity quotations were split among several subthemes, also listed under the H1 metric. There were a large number of negative comments regarding trash (H2). Several people described carrying out other peoples’ garbage.
There was strong interest in legal access to rivers (H3). Most comments had the context of there not being enough access. However there were nearly as many comments simply interested in knowing where access was legal, expressing a need to balance access with private landholders, and a need to balance access with over-usage.
There were many references to humans themselves. These comments included both positive and negative statements (H4), with about twice as many of the latter. Urban participants in particular were concerned about over-crowding or desiring isolation at recreational sites. Crowding is a long-running theme in the recreation literature [58]. Comments in support of other users were in the spirit of wanting to keep opportunities open for others to connect with rivers. Several of these were associated with the easy access portrayed in photo A. There were also a few comments regarding crime (H9).
Human infrastructure besides recreational amenities was important. Nearly all such comments were in reaction to the photos distributed halfway through the focus groups, especially photo D, which shows a multifamily complex in close proximity to the Willamette River. Most comments viewed human infrastructure as a stressor, i.e. representing active or potential pollution. The prevalent context was that the buildings were “too close,” as well as suspicion of what pollution the culvert in the foreground could be carrying. Comments with a clearly esthetic context, such as “ugly” buildings were tabulated under an esthetics metric (H5). Also represented in this metric were comments that human-altered rivers could be attractive; about a dozen people had positive esthetic reactions to the city scene in photo A.
The themes of unnatural odor and unnatural noise (e.g. jet skis) also emerged as themes, for which we created metrics H6 and H7, respectively.
The metric for Land Use (H8) has a low code count, but is unique in that it emerged during the validation interview phase. As interviewees looked over the provisional list of metrics, and imagined actual stream information applying to different sites, several people desired land use information as a landscape level to contextualize stream data.
Overall discussion
In broad strokes, several things stand out in the results. First, people discussed topics far outside of the river channel, ranging across vegetation, wildlife, even human features. Transcript narratives are also more sophisticated than might have been supposed, showing evidence of extensive thought on river issues, in contrast to a stereotype that the public is ambivalent about environmental conditions. At the same time, feedback was often generalized. Trees, fish, and birds were highly prevalent themes, yet people rarely mentioned specific types. We constrain metrics when we have sufficient evidence to do so, e.g. for birds we focus on birds of prey and other large birds (LW1). Whereas species-level disaggregation was not typically important for the metrics, other distinctions did matter. For example, interest in native species and nuisance species was relatively high. Metrics for endangered species and biodiversity also appear, but at low levels, despite being common topics in the ecological literature. In the case of endangered species, people may be reluctant to take a stand in a focus group on something perceived to be controversial; most of the endangered species code frequency occurred during individual validation interviews rather than group meetings.
The data collection was carefully designed to examine differences between segments, especially urban versus rural interest in rivers. We were prepared to record entirely different metrics and themes for different segments, but on the whole this was not necessary. Surprisingly, the main outcome from comparing results across segments was the similarity of Final EGS. That said, there was evidence of differences in attitudes–for example, a recurring rural complaint was the phenomenon of combined sewer overflows from cities into rivers, while agriculture faced unnecessarily strict runoff regulations. Yet despite attitudinal differences, in only a few instances were notable variations found in Final EGS. Namely, there was higher rural interest in flood property damage, game fish, and swimming. Urban interest was higher in how flooding has changed over time, mammals, invasive/native species, sensitive species, recreational amenities, and esthetics of human developments along rivers. There were a few additional themes with exclusively urban participation, but all sparse in frequency.
Overall, there was a remarkable coalescence of feedback. Similar river measurements repeatedly arose across ninety-nine diverse participants, and the thirty distinct focus groups and validation interview sessions we conducted. We only excluded measurement themes mentioned by only one person or one group–even so, we generated just 49 metrics from essentially unlimited possibilities. We also tracked the frequency of interest in a given theme relative to other themes, yielding evidence of further prioritization: the top ten themes account for over 50% of theme frequency (Fig 3). We provide supporting evidence for familiar attributes in the social science literature regarding appreciated ecosystem features, such as safe water contact, water clarity, fish, trees, and mammals. However, there were also a few surprises. There was a remarkable amount of concern regarding the safety of water supplies, despite sophisticated treatment and testing infrastructure. In a similar vein, there was interest in whether river water would be safe to drink, something we would not have thought many people would consider. A large number of participants had interest in boating or swimming, activities well represented even by the non-recreationalist segment, leading to high frequencies for navigation and adequate swimming depth metrics. Interest in native species (as combined with concern regarding invasive species eliminating native species) was the second ranked theme overall. Concern for endangered species was an additional theme reflecting additional interest related to native species. Finally, we found substantial interest in Human attributes, accounting for a bit less than 20% of total feedback. This includes some relatively large themes, such as appreciation of recreational amenities, and dislike of garbage.
This study is similar to previous research in southern Arizona [20]. Despite marked landscape and sociodemographic contrasts, there are numerous parallel results between the two studies. This provides evidence that what matters to people about rivers may resonate across the western US, perhaps even beyond. Yet two notable differences demonstrate the need to consider regional context. In arid southern Arizona, the top water theme was the mere presence of water in the river channel. This idea only tangentially appeared in temperate western Oregon, under the theme of appreciating river views. Second, a significant theme in Oregon was fish edibility. In Arizona it was considered interesting if fish were present at all: fish edibility does not appear as a theme. This underscores the need to consider local context when attempting to apply metrics from either study to a new location.
Our results can also be compared to river attributes appearing in the global literature. A detailed comparison is beyond our scope, but we draw preliminary insights from a limited analysis, comparing with [11], a recent review of the nonmarket valuation literature on river restoration, and [59], a meta-analysis of nonmarket river restoration values. Both studies categorize river restoration ecosystem services, and both lists bear much resemblance to our top themes. The first study [11] finds that the vast majority of studies reviewed sought to restore or protect fish populations, including many threatened or endangered fish populations. Numerous studies also focused on improving wildlife habitat, and water quality for boating. The bulk of studies included in the meta-analysis [59] valued changes in wildlife habitat (a fisheries focus is possible but is not mentioned). Second in valuation frequency was landscape esthetics, with water recreation third.
The similarity of our results with this sample of the literature is encouraging; if our results are accurate, the nonmarket valuation literature on river restoration appears to be largely on the right track. We had multiple themes of public interest regarding wildlife, especially fisheries, as well as numerous themes covering recreation, esthetics, and endangered species. However, there are also some interesting contrasts. For example, property damage from flooding, navigability of waterways, large tree vegetation, and recreational amenities are all among our top themes (Fig 3), but are not prominent in the ecosystem service categorizations of [11] or [59]. There are multiple potential explanations for this. The categories in the comparison studies are broad—two papers included in [11] do actually value tree vegetation changes [14, 47] although vegetation is not given a separate category. The river restoration focus may bias the sample towards ecological attributes rather than human features such as recreational amenities. In addition, there is some bias towards attributes that can feasibly be changed—e.g. navigability would be difficult to change for many waterways since it is highly dependent on gradient and geology. Finally, some topics such as property damage from flooding involve market impacts, and thus logically appear in a separate literature [60–62]. The flood damage example is indicative of how broad the literature review and synthesis would need to be in order to execute a detailed comparison with our results, an undertaking we must reserve for future research.
A comparison with the literature should consider metric formulations as well as river themes of public interest. A primary goal of our research was to bring the specificity of metrics to general themes such as "wildlife." Metric comparison cannot be done without close examination of measurable features explicit or implied in each individual paper. Yet one widespread type of metric formulation deserves mention here, raised by an anonymous reviewer. Often, metrics embed a comparison of river status to a "reference condition." As noted previously, our participants commonly considered what was "natural" when commenting on preferred river attributes, such as when discussing background photos. Accordingly, several of our metrics include a direct or implied comparison with natural condition, such as the invasive species metric, and the garbage metric. Whether or not a comparison with a baseline is made is a critical decision, with important philosophical consequences: there are also numerous ways to actually calculate such a comparison, leading to potential for bias in how a metric is interpreted by the public [63]. Furthermore, it should be noted that there can be challenges in establishing a reference condition, e.g. situations where human activity has left little if any evidence of natural state [64]. The issues regarding reference condition are just one example of the challenges that remain in formulating metrics, after a theme of public interest has been identified.
Our results provide strategic, detailed insights on river metrics people care about, using interactive discussion techniques best suited to isolate what really matters. It was a constant challenge for the moderator to move past commonly stated issues such as "pollution" to specific measurable attributes, requiring repeated use of follow-up and probe questions uniquely suited to qualitative methods. Our results provide high-level insight on river themes and metrics of public interest, an arena with surprisingly little exploratory research. Our provisional metrics seed future research, and also provide a reference for situations in which river metric insight is needed but funds for original research are lacking. We suggest the following three applications of our results in particular:
- Reference for design of a river monitoring program, e.g. a river ’report card.’
- Reference for how to adapt already available river information for improved public relevance.
- Reference for priority areas of river management in the public interest.
In addition to information on metrics of public interest we provide, river monitoring should also consider the costs of acquiring different kinds of information, as well as the costs and benefits of different management actions as based on river monitoring data. For example, certain kinds of information may be extremely expensive to monitor, and thus it could be optimal for these to remain poorly known despite high public interest. Information will also be a platform for management in at least some cases. Metrics feasibly influenced through management actions, and with relatively high public value per unit change, should be prioritized when designing a river monitoring plan, although these factors are not necessarily easy to evaluate a priori.
Our goal was to formulate hypotheses of public interest across the spectrum of river attributes, yet our study has limitations. With an intensively studied but only moderately sized sample, split across five segments, we cannot conduct statistical tests or claim generalizability of results to the underlying population. Complementary quantitative research is a potential avenue for further work, i.e. a mixed-methods approach [34]. Survey work to test the hypotheses of priority themes and metrics we offer in this article would be needed to precisely gauge the absolute ranking of themes and metrics for a population (e.g., "best-worst scaling" methods, [29, 30]). Follow-up research could also measure any bias resulting from different background materials provided to human subjects. Of course, any quantitative survey effort should receive dedicated qualitative pre-testing before full deployment [65].
Conclusions
We painstakingly document publicly relevant river metrics based on detailed feedback from almost one hundred participants, recruiting both urban and rural residents of western Oregon. Our research adds further flesh and blood to the concept of Final EGS developed within environmental economics, with results oriented towards practical monitoring, modeling, and management usage for water resources professionals. We utilize qualitative research techniques of focus groups and interviews common to social science, methods preferred for eliciting human perspectives on complex questions. Of particular importance in our case, back-and-forth moderation allowed crucial follow-up opportunities to distinguish proxy issues from real concerns, e.g. interest in water clarity was not tallied as a Final EGS when the ultimate concern was actually safety of recreational water contact.
An important outcome is that meetings with diverse research participants led to highly repetitious feedback regarding what river metrics matter. We found remarkable coalescence across numerous groups and individuals, emphasizing relatively few themes overall. Furthermore, our results are similar to previous research, both prior qualitative research on river metrics for a contrasting US geography, and the global literature on nonmarket valuation. We thus speculate that there are relatively few river themes that account for the bulk of public interest, applying across diverse geographic contexts, with some allowance for regional context. We also speculate that some themes, such as those within the Biota category, would appear for studies investigating public interest in other types of ecosystem, such as oceans or forest.
The insights we provide on river metrics are important to help achieve focus in environmental study and management. In various ways, both natural scientists and social scientists work with or collect environmental data without necessarily knowing how publicly relevant the selected indices are. Thus, efforts are potentially inefficient if the ultimate goal is to manage resources in the public interest. Our results spotlight the information that effectively bridges the biophysical and social realms, and thus a potentially powerful shared set of objectives. In some cases publicly derived river metrics may imply new measurements, in other cases information may already exist or simply require changes in presentation to make the data more accessible. By providing publicly-relevant river information, we hope to make progress towards focused outcomes for river management, rather than fracturing limited resources. Our rigorously generated hypotheses regarding publicly relevant river metrics may be refined with future research, or utilized directly in appropriate circumstances. In deciding what data to collect, river monitoring should also consider the costs of acquiring different forms of information, as well as the costs and benefits of potential river management actions based on monitoring data.
Supporting information
S1 Appendix. Focus group script and validation interview specific questions.
https://doi.org/10.1371/journal.pone.0214986.s001
(DOCX)
Acknowledgments
The authors would like to thank David Morgan, Kirsten Winters, Editor Miguel Cañedo-Argüelles Iglesias, anonymous reviewers, and ecosystem services co-panelists of the Joint Aquatic Sciences Meeting, 2014, for useful comments and suggestions. The lead author would like to thank the US Environmental Protection Agency, Office of Research and Development, for postdoctoral support (Western Ecology Division, and National Risk Management Research Laboratory). Thanks to the The Henne Group (Contract EP-G12D-00336) for participant recruitment and scheduling. Finally, we extend gratitude to the numerous human subjects whose cooperation made this research possible.
References
- 1.
Hughes, R.M., 1993. Stream Indicator and Design Workshop. Environmental Protection Agency. Report no. EPA/600/R-93/138.
- 2.
National Research Council. Ecological Indicators for the Nation. National Academy Press, Washington, D.C. 2000.
- 3.
Jackson LE, Kurtz JC, Fisher WS (Eds.). Evaluation guidelines for ecological indicators. EPA/620/R-99/005. US Environmental Protection Agency, Office of Research and Development, Research Triangle Park, NC. 2000.
- 4. Boyd J, Banzhaf S. What are ecosystem services? The need for standardized environmental accounting units. Ecological economics. 2007 Aug 1;63(2–3):616–26.
- 5. Johnston RJ, Russell M. An operational structure for clarity in ecosystem service values. Ecological Economics. 2011 Oct 15;70(12):2243–9.
- 6. Ringold PL, Boyd J, Landers D, Weber M. What data should we collect? A framework for identifying indicators of ecosystem contributions to human well‐being. Frontiers in Ecology and the Environment. 2013 Mar;11(2):98–105.
- 7. Boyd J, Krupnick A. Using ecological production theory to define and select environmental commodities for nonmarket valuation. Agricultural and Resource Economics Review. 2013 Apr;42(1):1–32.
- 8. Boyd JW, Ringold P, Krupnick A, Johnston RJ, Weber MA, Hall K. Using Ecosystem Services Indicators: Improving the Linkage between Biophysical and Economic Analyses. International Review of Environmental and Resource Economics 2016 8(3–4):359–443. http://dx.doi.org/10.1561/101.00000073, accessed 2018.
- 9. Weber MA, Caplan S, Ringold P, Blocksom K. Rivers and streams in the media: a content analysis of ecosystem services. Ecology and Society. 2017 Sep 1;22(3).
- 10. Viscusi WK, Huber J, Bell J. The economic value of water quality. Environmental and Resource Economics. 2008 Oct 1;41(2):169–87.
- 11. Bergstrom JC, Loomis JB. Economic valuation of river restoration: An analysis of the valuation literature and its uses in decision-making. Water Resources and Economics. 2017;17:9–19.
- 12.
United States Environmental Protection Agency. National water quality inventory, report to congress.1994-2002. https://www.epa.gov/waterdata/national-water-quality-inventory-report-congress, accessed 2018.
- 13. Moore C, Guignet D, Dockins C, Maguire KB, Simon NB. Valuing Ecological Improvements in the Chesapeake Bay and the Importance of Ancillary Benefits. Journal of Benefit-Cost Analysis. 2018; 9(1):1–26.
- 14. Weber MA, Meixner T, Stromberg JC. Valuing instream-related services of wastewater. Ecosystem Services. 2016 Oct 1;21:59–71.
- 15.
Louviere JJ, Hensher DA, Swait JD. Stated choice methods: analysis and applications. Cambridge university press; 2000.
- 16. Jeanloz S, Lizin S, Beenaerts N, Brouwer R, Van Passel S, Witters N. Towards a more structured selection process for attributes and levels in choice experiments: A study in a Belgian protected area. Ecosystem services. 2016 Apr 1;18:45–57.
- 17. Schiller A, Hunsaker CT, Kane MA, Wolfe AK, Dale VH, Suter GW, Russell CS, Pion G, Jensen MH, Konar VC. Communicating ecological indicators to decision makers and the public. Conservation Ecology. 2001 Jun 1;5(1).
- 18.
Ringold PL, Boyd J, Landers D, Weber MA, Meeting Date: July 13 to 16, 2009. Report from the Workshop on Indicators of Final Ecosystem Services for Streams. EPA/600/R-09/137. http://www.epa.gov/nheerl/arm/streameco/index.html, accessed 2016.
- 19.
Landers DH, Nahlik AM. A national ecosystem service classification system for final ecosystem goods and services (NESCS). US Environmental Protection Agency, Office of Research and Development, Office of Research and Development, Western Ecology Division, Corvallis, OR. 2013.
- 20. Weber MA, Ringold PL. Priority river metrics for residents of an urbanized arid watershed. Landscape and Urban Planning. 2015 Jan 1;133:37–52.
- 21.
Kammerer JC. Largest Rivers in the United States. US Geological Survey. May 1990. http://pubs.usgs.gov/of/1987/ofr87-242/, accessed 2016.
- 22.
United States Department of Agriculture. National School Lunch Program. 2013. http://www.fns.usda.gov/sites/default/files/NSLPFactSheet.pdf, accessed 2016.
- 23.
United States Department of Agriculture. Women Infants and Children Eligibility Requirements. 2015. http://www.fns.usda.gov/wic/wic-eligibility-requirements, accessed 2016.
- 24.
Morgan DL, Krueger RA. The Focus Group Kit (6 volumes). Sage Publications. 1998.
- 25.
Hulse D, Gregory S, Baker J (Eds.). Willamette River Basin planning atlas. Trajectories of environmental and ecological change. Corvallis, OR: Oregon State University Press. 2002.
- 26.
Creswell JW. Qualitative inquiry and research design: Choosing among five approaches. Third Edition. Sage Publications. 2013.
- 27. Thorne S. Data analysis in qualitative research. Evidence-based nursing. 2000 Jul 1;3(3):68–70.
- 28. Johnson RB, Onwuegbuzie AJ. Mixed methods research: A research paradigm whose time has come. Educational researcher. 2004 Oct;33(7):14–26.
- 29. Auger P, Devinney TM, Louviere JJ. Using best–worst scaling methodology to investigate consumer ethical beliefs across countries. Journal of Business Ethics. 2007 Feb 1;70(3):299–326.
- 30. Flynn TN, Louviere JJ, Peters TJ, Coast J. Best–worst scaling: what it can do for health care research and how to do it. Journal of health economics. 2007 Jan 1;26(1):171–89.
- 31.
Krippendorff K. Content Analysis: An Introduction to its Methodology. Sage Publications. 2013.
- 32.
Qualis Research. The Ethnograph v6 (software). 610 Popes Valley Drive, Colorado Springs, CO 80919. http://www.qualisresearch.com, accessed 2018.
- 33. Creswell JW, Miller DL. Determining validity in qualitative inquiry. Theory into practice. 2000 Aug 1;39(3):124–30.
- 34.
Creswell JW. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Fourth Edition. Sage Publications. 2014.
- 35.
Mitchell RC, Carson RT. The Use of Contingent Valuation Data for Benefit/Cost Analysis in Water Pollution Control. CR-810224-02. Prepared for US EPA, Office of Policy Planning and Evaluation. Washington, DC. 1986. http://yosemite1.epa.gov/ee/epa/eerm.nsf/vwRepNumLookup/EE-0150?OpenDocument, accessed 2016.
- 36. Carson RT, Mitchell RC. The Value of Clean Water: Willingness to Pay for Boatable, Fishable, and Swimmable Water quality. Water Resources Research, July. 1993.
- 37. Gibbs JP, Halstead JM, Boyle KJ, Huang JC. An hedonic analysis of the effects of lake water clarity on New Hampshire lakefront properties. Agricultural and Resource Economics Review. 2002 Apr;31(1):39–46.
- 38. Keeler BL, Wood SA, Polasky S, Kling C, Filstrup CT, Downing JA. Recreational demand for clean water: evidence from geotagged photographs by visitors to lakes. Frontiers in Ecology and the Environment. 2015 Mar;13(2):76–81.
- 39.
United States Environmental Protection Agency. National Summary of Impaired Waters and TMDL Information. 2015. http://iaspub.epa.gov/waters10/attains_nation_cy.control?p_report_type=T#imp_water_by_state, accessed 2016.
- 40. Wohl E. Compromised rivers: understanding historical human impacts on rivers in the context of restoration. Ecology and Society. 2005 Dec 1;10(2).
- 41. Piégay H, Gregory KJ, Bondarev V, Chin A, Dahlstrom N, Elosegi A, Gregory SV, Joshi V, Mutz M, Rinaldi M, Wyzga B. Public perception as a barrier to introducing wood in rivers for restoration purposes. Environmental Management. 2005 Nov 1;36(5):665–74. pmid:16215648
- 42. Chin A, Daniels MD, Urban MA, Piégay H, Gregory KJ, Bigler W, Butt AZ, Grable JL, Gregory SV, Lafrenz M, Laurencio LR. Perceptions of wood in rivers and challenges for stream restoration in the United States. Environmental Management. 2008 Jun 1;41(6):893–903. pmid:18305987
- 43. Chin A, Laurencio LR, Daniels MD, Wohl E, Urban MA, Boyer KL, Butt A, Piégay H, Gregory KJ. The significance of perceptions and feedbacks for effectively managing wood in rivers. River Research and Applications. 2014 Jan;30(1):98–111.
- 44. Kaplowitz MD. Statistical analysis of sensitive topics in group and individual interviews. Quality and Quantity. 2000 Nov 1;34(4):419–31.
- 45. Richardson L, Loomis J. The total economic value of threatened, endangered and rare species: an updated meta-analysis. Ecological Economics. 2009 Mar 15;68(5):1535–48.
- 46. Wallmo K, Lew DK. Public willingness to pay for recovering and downlisting threatened and endangered marine species. Conservation Biology. 2012 Oct;26(5):830–9. pmid:22827248
- 47. Weber MA, Stewart S. Public values for river restoration options on the Middle Rio Grande. Restoration Ecology. 2009 Nov;17(6):762–71.
- 48. Morrison M, Bennett J. Valuing New South Wales rivers for use in benefit transfer. Australian Journal of Agricultural and Resource Economics. 2004 Dec;48(4):591–611.
- 49. Sharp RL, Larson LR, Green GT. Factors influencing public preferences for invasive alien species management. Biological Conservation. 2011 Aug 1;144(8):2097–104.
- 50. García-Llorente M, Martín-López B, González JA, Alcorlo P, Montes C. Social perceptions of the impacts and benefits of invasive alien species: implications for management. Biological Conservation. 2008 Dec 1;141(12):2969–83.
- 51. Selge S, Fischer A, van der Wal R. Public and professional views on invasive non-native species–A qualitative social scientific investigation. Biological Conservation. 2011 Dec 1;144(12):3089–97.
- 52. Christie M, Hanley N, Warren J, Murphy K, Wright R, Hyde T. Valuing the diversity of biodiversity. Ecological economics. 2006 Jun 15;58(2):304–17.
- 53. Fischer A, Young JC. Understanding mental constructs of biodiversity: implications for biodiversity management and conservation. Biological Conservation. 2007 Apr 1;136(2):271–82.
- 54. Ripple WJ, Estes JA, Beschta RL, Wilmers CC, Ritchie EG, Hebblewhite M, Berger J, Elmhagen B, Letnic M, Nelson MP, Schmitz OJ. Status and ecological effects of the world’s largest carnivores. Science. 2014 Jan 10;343(6167):1241484.
- 55. Netusil NR, Chattopadhyay S, Kovacs KF. Estimating the demand for tree canopy: a second-stage hedonic price analysis in Portland, Oregon. Land Economics. 2010 May 1;86(2):281–93.
- 56. Kadish J, Netusil NR. Valuing vegetation in an urban watershed. Landscape and Urban Planning. 2012 Jan 1;104(1):59–65.
- 57. Steel BS, List P, Shindler B. Conflicting values about federal forests: A comparison of national and Oregon publics. Society & natural resources. 1994 Mar 1;7(2):137–53.
- 58. Tarrant MA, Cordell HK, Kibler TL. Measuring perceived crowding for high‐density river recreation: the effects of situational conditions and personal factors. Leisure Sciences. 1997 Apr 1;19(2):97–112.
- 59. Brouwer R, Sheremet O. The economic value of river restoration. Water Resources and Economics. 2017;17:1–8.
- 60. Smith DI. Flood damage estimation- A review of urban stage-damage curves and loss functions. Water S. A.. 1994 Jul 1;20(3):231–8.
- 61. Dutta D, Herath S, Musiake K. A mathematical model for flood loss estimation. Journal of hydrology. 2003 Jun 1;277(1–2):24–49.
- 62. Merz B, Kreibich H, Schwarze R, Thieken A. Review article" Assessment of economic flood damage". Natural Hazards and Earth System Sciences. 2010 Aug 18;10(8):1697–724.
- 63. Fu B, Dyer F, Kravchenko A, Dyack B, Merritt W, Scarpa R. A note on communicating environmental change for non-market valuation. Ecological indicators. 2017 Jan 1;72:165–72.
- 64. Stoddard JL, Larsen DP, Hawkins CP, Johnson RK, Norris RH. Setting expectations for the ecological condition of streams: the concept of reference condition. Ecological Applications. 2006 Aug;16(4):1267–76. pmid:16937796
- 65.
Dillman DA, Smyth JD, Christian LM. Internet, Mail and Mixed-Mode Surveys. Vol 3. John Wiley and Sons. 2009.