Alluvial substrate mapping by automated texture segmentation of recreational-grade side scan sonar imagery

Daniel Hamill; Daniel Buscombe; Joseph M. Wheaton

doi:10.1371/journal.pone.0194373

Abstract

Side scan sonar in low-cost ‘fishfinder’ systems has become popular in aquatic ecology and sedimentology for imaging submerged riverbed sediment at coverages and resolutions sufficient to relate bed texture to grain-size. Traditional methods to map bed texture (i.e. physical samples) are relatively high-cost and low spatial coverage compared to sonar, which can continuously image several kilometers of channel in a few hours. Towards a goal of automating the classification of bed habitat features, we investigate relationships between substrates and statistical descriptors of bed textures in side scan sonar echograms of alluvial deposits. We develop a method for automated segmentation of bed textures into between two to five grain-size classes. Second-order texture statistics are used in conjunction with a Gaussian Mixture Model to classify the heterogeneous bed into small homogeneous patches of sand, gravel, and boulders with an average accuracy of 80%, 49%, and 61%, respectively. Reach-averaged proportions of these sediment types were within 3% compared to similar maps derived from multibeam sonar.

Citation: Hamill D, Buscombe D, Wheaton JM (2018) Alluvial substrate mapping by automated texture segmentation of recreational-grade side scan sonar imagery. PLoS ONE 13(3): e0194373. https://doi.org/10.1371/journal.pone.0194373

Editor: Judi Hewitt, University of Waikato, NEW ZEALAND

Received: December 7, 2017; Accepted: March 1, 2018; Published: March 14, 2018

Copyright: © 2018 Hamill et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All of the data and scripts presented in this paper are available at https://github.com/danhamill/ss_texture_analysis (DOI: 10.5281/zenodo.820662).

Funding: This work was funded by the Glen Canyon Dam Adaptive Management Program administered by the U.S. Bureau of Reclamation. The lead author (DH) and last author (JW) were supported by the U.S. Geological Survey to Utah State University (USGS Agreement G14AC00369; USU Award 150155).

Competing interests: The authors have declared that no competing interests exist.

Introduction

The grain size of bed sediment is a fundamental attribute of rivers and streams [1], and an important independent variable in studies of river adjustment [2], river classification [3], sediment transport [4], hydraulic roughness [5], and aquatic habitat [6] and therefore is an essential component of habitat suitability models [7–10].

Riverbeds are often arranged in sediment patch structures or facies of like-sediment [11] providing a diverse range of spatially coherent yet mobile micro-topographies [12]. Some sediment patches remain stationary because of large-scale topographic or obstruction-driven hydraulics [13], whereas others migrate freely in response to variable water and sediment supply [14]. Field studies have demonstrated how spatial variations in grain size affect the longitudinal organization of benthic community structure [15]. To develop suitable management and conservation strategies it is necessary to identify what controls the spatial variability in benthic habitats. The hierarchical organization of aquatic ecosystems [16] and the variable nature of grain size create a complex situation where, if relationships between aquatic organisms and grain size exist, these are highly non-linear linkages that are established across multiple scales [17]. Direct (and linear) relationships between aquatic organisms and grain size are often difficult to establish because grain size alone does not uniquely describe complex physical habitat [18], or alternatively because relating animal behavior in a spatially continuous sense to sediment relies on spatially continuous substrate maps, which traditionally are difficult to construct [17].

In recent years, side scan sonar within commodity ‘fishfinder’ systems have become an increasingly popular low-cost sensor for qualitative mapping of riverbed sediment and benthic environments [19–22]. We term these relatively low quality sonar systems, ‘recreational-grade’ to distinguish them from relatively higher quality, relatively expensive, ‘survey-grade’ side scan sonar systems [22]. In contrast to survey-grade side scan sonar, recreational-grade systems are typically operated on personal water crafts with out high-quality positioning and boat attitude (heave, pitch, roll, etc.) information. Kaeser et al. [23] demonstrated that recreational-grade side scan sonar imagery, called echograms, collected using a recreational-grade system in a riverine environment had sufficient detail to map locations of large woody debris. Subsequent studies have established that the resolution and quality of the echogram is sufficient to visually identify sediment facies [19–21] over reaches up to hundreds of kilometers in length, and these sonar have enjoyed a proliferation of use among aquatic ecologists [19–21, 23–30].

Despite allowing rapid collection high-resolution echograms across large areas, from which it is possible to visually identify sediment groupings, methods to automatically post-process and interpret data collected with a recreational-grade system are currently limited [22]. Recreational-grade systems are designed for providing images of the bed from a vessel, and do not record data to a hydrographic standard, or in standard data formats. Kaeser et al. [26] created a semi-automated, open-source GIS routine to create a georeferenced echogram by ‘rubbersheeting’ overlapping screenshots from the topside unit within a geographical information system platform, for subsequent visual interpretation. This methods works fairly well, but is labor intensive, subjective, and not practically applied to large-volumes of data. In addition, it does not correct for geometric or radiometric distortions present within the data. Buscombe [22] developed an open source program to automate the production of geometrically and radiometrically corrected georectified echograms directly from the binary files recorded by recreational-grade systems. Automated approaches to extracting and processing the data also presents the opportunity to draw upon automated side scan imaging processing literature [31–33] and to develop more objective approaches for carrying out spatially distributed substrate classification [34–37]. For highly heterogeneous sedimentary deposits, such as mixed-alluvial riverbeds it is unlikely that each substrate type is associated with a sufficiently narrow distribution of sidescan backscatter intensities to establish direct relations between sidescan backscatter intensity and substrate [36]. For finer substrates such as sand, this is due to large variations in slope, and bedform heights and wavelengths that collectively cause variations in backscattering strength. For coarser substrates such as cobbles, boulders and bedrock, the variation in backscattering of sound is caused by acoustic shadows that scale with both the height of the individual roughness elements but also the relative angle with the sonar [36]. Therefore, the majority of approaches to automated objective classification of substrates from echograms have used analyses of textural or variance properties of patterns that correspond to sedimentologically distinct regions [32, 34, 36, 38, 39]. An accurate substrate map might even provide a means with which to further correct echogram from which it is derived for radiometric distortions [37].

The word ‘texture’ has been used to describe variations in bed form morphologies [40], to map spatial arrangements of riverbed sediment [41], as synonymous with grain size distributions [11], and as a term to describe any surface roughness, rugosity, or waviness without strict definition [42]. The word is often used to substitute for a suite of variables related to the grain size, roughness, and the spatial arrangement of those quantities on the bed, where information on these quantities is lacking [43]. Roughness often refers to 1st-order metrics, such as the standard deviation of riverbed elevations [44], whereas texture often refers to 2nd-order metrics that take into occur the spatial arrangement of roughness elements or spatially continuous areas of like-roughness [43]. The word texture is often used in remote sensing in situations where the actual scale of interest, such as the bed form or grain scale, exist at the sub-pixel scale which is not resolvable but results in supra-pixel spatial arrangements of pixel intensities that indicate the presence and/or magnitude of the features of interest [45]. In a similar vein, here the word texture is used here in a qualitative sense to describe the spatial arrangement of surface roughness, that itself is the product of both supra-pixel (i.e. grouping of pixels) and sub-pixel (i.e. single pixel) grain size and morphologies, and in the quantitative sense as the value of a particular spatial statistic that is indicative of a particular band of grain sizes.

The objective of this study is to develop an unsupervised classification of substrates based on their corresponding textures in side scan echograms. This paper builds on Buscombe [22] by evaluating how echograms collected with a recreational-grade sonar system can be used to objectively identify classify and map (e.g. delineate) riverbed sediment. Perceptually homogeneous textures in an echogram are each characteristic of a different substrate, therefore discriminating among these textures using statistical techniques, creates a substrate map. We use a case study of multiple side scan sonar images a canyon riverbed to evaluate optimal texture metrics for broad-scale (a coverage of hundreds to thousands of square meters at a resolution of meters to decimeters) substrate classification. First, we examine textural characteristics of echograms from visually identified areas of interpreted substrate types. We then test and evaluate two classification approaches of differing complexity. Each classification approach is calibrated to a particular riverbed, before it is then applied to entire datasets from that riverbed in an unsupervised manner. We further evaluate the ability of the sidescan-derived substrate maps to reproduce reach-averaged proportions of substrates in calibrated acoustical substrate maps derived from multibeam sonar. Finally, we discuss how these methods could be applied to echograms of other mixed alluvial beds with a different sedimentary and morphological character, and the implications of using recreational-grade side sonar systems for characterizing riverbed sediment for physical benthic habitat assessment.

Methods

This work was conducted with assistance from the U.S. Geological Survey Grand Canyon Monitoring and Research Center under research permits issued by the National Park Service. Our approach to building an unsupervised classification using statistical descriptors requires manual delineation of different substrates within the data. The manually delineated substrate classes serve as training zones to identify statistical descriptors that can discriminate substrate classes from each other. We then optimize the discriminatory power of a classification approach before applying the classification in an unsupervised way to larger portions of the data set. We then validate the classification by evaluating the unsupervised classification’s ability to classify manually delineated substrate classes.

Data collection & study area

We collected side scan sonar data at a fish monitoring site spanning a 1.6-km canyon-bound reach of the Colorado River [46–48]. The study reach is located 98-km downstream of Lees Ferry in Marble Canyon, Arizona, directly upstream from the confluence of the Little Colorado River, and covers multiple pool-riffle sequences. Data were collected during five river trips between May 2012 and April 2015 (Table 1) between fish sampling activities during quarterly fish sampling trips by various operators and boatmen, who had little or no prior knowledge collecting these data. Data were not quality-controlled in the field, and no repeat surveys were conducted. This protocol was intentionally designed to mimic rapid, opportunistic sampling. At a minimum, data were collected over the entire fish sampling reach, but trip-by-trip survey extents were dictated by the availability of operators and the logistical requirements of the fish sampling activities.

Download:

Table 1. Echogram inventory.

https://doi.org/10.1371/journal.pone.0194373.t001

Additionally, Grams et al. [49] extensively mapped this study reach with multibeam sonar. The multibeam sonar data provide high resolution bathymetry for validating the positional accuracy of georectfied echograms, and independently derived sediment classification maps derived from the recorded acoustic backscatter [50] for evaluating side scan sonar sediment classifications. The riverbed of the study reach is well studied [36, 41, 50, 51], composed of non-cohesive sediment, with grain size ranging from fine sand to boulders, and containing no submerged vegetation.

Continuous side scan sonar recordings and positional information were collected with a Humminbird ^® 998c recreational-grade side sonar. The sonar was mounted to a pole off the starboard bow or abeam to starboard of a small (2.75-m long) aluminum-hulled boat with an outboard motor. Positional errors associated with poor GPS fix, and the lack of boat attitude information, were significant in this canyon setting with limited visibility of satellite constellations and areas with swift moving water. However, by collecting data in the middle of the channel and at low speeds the effects of canyon walls, boat pitch, heave and dynamic draft were minimized. The boat operator avoided crabbing, to ensure that the direction of progress best estimated the boat’s true heading, by either motoring with, or directly against, the main current.

Sonar data processing

All of the echograms analyzed in this paper were processed using PyHum [22], an open-source toolbox for decoding the file formats associated with side scan sonar recordings from a Humminbird^® Side Imaging Systems. PyHum is a python-based, modular toolbox that currently supports multiple models from the HD-SI (i.e. 700, 800, 900 and 1100 series), HELIX, MEGA, and ONIX side imaging systems. This study used a HD-SI 998c. The data collected within this study were processed using the ‘read’, ‘correct’, ‘remove shadows’, and ‘map’ modules. Continuous side scan recordings are encoded in proprietary file formats that consist of a DAT file and set of SON files. Using the meta-data encoded within DAT file, the read module decodes the raw data contained in the SON files to produce a time series of data (i.e. scan lines) that represent the ensonified water column and sediment-water interface. The timing and strength of each ping within a scan line are then reoriented using a simplified sonar geometry model to establish the most likely longitudinal orientation of returned signals. The scan lines collected during a continuous recording are compiled into an echogram using the positional data recorded by the GPS antenna. The correct module applies basic geometric and radiometric corrections to account for the effects of environmental conditions (e.g. sound absorption) and sonar settings (e.g. signal strength and beam pattern). The remove shadows module was used to visually segment and remove areas devoid of texture (i.e. water column and acoustic shadows) that exist in the near-field and far-fields of an echogram. The map module was then used to project the corrected and filtered echogram to a known coordinate system using the positional and navigational information collected with the supplied GPS antenna in the units of decibel watts (dBW). Buscombe [22] has detailed the data processing assumptions and acoustic corrections encoded within the software. PyHum differs from other available recreational-grade side scan sonar processing software [52, 53] because it radiometrically corrects the backscatter data, and projects each pixel in the echogram as a point in a point cloud using instantaneous position and heading, rather than rubbersheeting the raster using image rectification, which can lead to greater positional errors.

The resulting georeferenced side scan intensity points clouds are very spatially dense, with up to thousands of points per square meter. When resampling large point clouds consisting of millions of points, [43] found that a nearest-neighbor approach using a K-dimensional (K-D tree) was the fastest algorithm, even less computationally expensive than computing a mean of all points in the cell. This is because the former involves only two operations per grid node: 1) finding the nearest points to the node and 2) ascribes its value to the node. The mean (or other summary statistic) involves three operations: 1) finding the points, 2) computing the mean, and, 3) ascribing the mean to the node. The PyHum program does in fact offer three different ways to resample the data: 1) nearest neighbor (the approach we used for this manuscript); 2) inverse distance weighting, which is an average of nearest neighbors weighted inversely according to distance from grid node; and 3) average of nearest neighbors weighted by a Gaussian kernel. The latter two approaches often result in less noisy grids, but are significantly slower. The resampled side scan intensity points clouds were then converted to a raster format in Arizona Central State Plane, NAD83. The grid size 0.25 x 0.25 m was chosen to ensure each cell had multiple data points. The side scan intensity images were then processed outside of PyHum, using a program written by the authors, to derive the textural properties identified in this paper.

Visually identified sediment patches

Echograms consist of 8-bit digital integers representing the backscattering strength of the bed, called ‘grey levels’. The georectfied side scan sonar images were used for the visual classification of substrates into broad Wentworth-style groupings of similar sediment to develop calibration and validation data sets. Based on the quality of the available data (Table 1), visual delineation of echograms into three sediment classes was deemed appropriate to establish a data set to evaluate pixel-by-pixel classification by automated analyses. At least three distinct substrate types could always be reliably be distinguished. The textures shown within the echogram are created by sedimentary and morphologic features. Smooth (i.e. low contrast), highly ordered textures were associated with mixtures of sand. Rough, disorderly textures are associated with boulders and bolder-dominant mixtures of gravel/boulders and sand/boulders. Textures that vary between smooth, orderly and rough, and disorderly were associated with gravel and gravel-dominant mixtures of sand/gravel and gravel/boulders. Hereafter, these classes are referred to as sand, boulders, and gravel, respectively. The visual delineation was carried out in a Geographic Information System at a fixed scale of 1:600. Over-saturated regions of the echogram and apparent morphologic features were excluded from the delineation (Fig 1). These over-saturated regions are portions of the echogram directly beneath the boats track line where the first (nadir) returns are so much greater in intensity than subsequent returns that the 8-bit quantization is insufficient to capture the full dynamic range of backscattered sound.

Download:

Fig 1. An example of georectfied echogram.

A: indicated an apparent morphology that was ignored during visual delineation. B: is an example of an over-saturated region where the textural signatures are difficult to interpret.

https://doi.org/10.1371/journal.pone.0194373.g001

Texture metrics

First-order statistics.

First-order statistical signatures of sediment types were developed using zonal statistics calculated from the visually mapped substrate patches and georectfied echograms. Both statistics of central tendency (i.e. mean, quartiles) and statistics that describe the distributions shape (i.e. standard deviation (σ), coefficient of variation (CV), kurtosis (γ), and skewness) were considered.

GLCM texture metrics.

The Grey Level Co-Occurrence Matrix (GLCM) is a second-order (i.e. quantifying spatial relationships) statistical method and has been found suitable to describe textures in echograms [31, 32] because it statistically describes spatial relationships between pixels within a local area, and because a number of objective measures can be computed from it. A GLCM is a matrix within which the frequency of tonal patterns between pixel pairs within a computational window are tabulated [54]. For a reference pixel in a computational window of size L × L, a GLCM is calculated by specifying a reference angle θ, distance d and number of gray levels N to quantize the original image. Values within a co-occurrence matrix are typically normalized such that the values represent a probability rather than a frequency of a particular pixel-pair relationship [55]. A GLCM (P_{(i, j)}) for a given computational window: (1) where V_{(i, j)} is the co-occurrence matrix and i, j are reference and neighboring pixel values, respectively. If a value in a GLCM is large, the specific tonal pattern it represents is common and associated with textures within the computational window that are repetitive. When a value in a GLCM is low that specific tonal pattern is uncommon and textures within that computational window are random. Orderly, repetitive patterns of grey levels are interpreted as being created by features which exist at the pixel (25 cm) or sub-pixel scale and are interpreted as finer substrates (i.e. sand), whereas disorderly patterns of gray levels are created by supra-pixel scale features and are interpreted as coarser substrates (i.e. coarse gravel, boulders) [32].

Haralick [54] proposed 14 scalar metrics for description of textural patterns encoded in a GLCM. These properties are amenable to a spatially explicit analysis, whereby each scalar coefficient is computed from each co-occurrence matrix and assigned to the computational window it represents. The Haralick texture descriptors have been shown to be applicable to vary a wide spectrum of natural and artificial textures, but each can be thought of as belonging to one of three groups: namely, contrast, orderliness, and descriptive statistics [55]. Blondel [32] was the first to identify Entropy (group: orderliness) and Homogeneity (group: contrast) properties as useful for classifying echogram textures. They are defined as, respectively: (2) (3)

Blondel et al. [33] further suggested that if E and H have a strong negative correlation the end members of such a relationship represent boulders and sand, respectively. H is a useful indicator of image contrast because the term (i − j)² eliminates the diagonal terms of a co-occurrence matrix and therefore is weighted using only of the off-diagonal (i.e. i ≠ j) matrix elements. Therefore, highly contrasted textures produce low H values and textures with low contrast are characterized by high values of H. Entropy characterizes the orderly components of a GLCM. Large values of E occur when there is a wide distribution of grey levels.

Through a process of elimination and evaluation of the remaining 12 Haralick metrics, we found GLCM variance (group: descriptive statistics) to be another potentially useful GLCM property for sediment discrimination. GLCM variance () and E are related, since they both quantify the dispersion of differences in intensity between pixel pairs, but GLCM variance is a statistic computed from the GLCM itself, given by: (4) where GLCM mean is: (5)

A sliding 2D window approach was used to calculate GLCMs over small regions of the image. Neighboring windows had no overlap in either direction and a GLCM was only computed when at least 75% of the window contained data. To determine how various GLCM calculation parameters affect echogram texture segmentations, GLCMs were calculated with a parameter space with varying window sizes, search distances, and reference angles (Table 2).

Download:

Table 2. Variables tested for GLCM calculations.

https://doi.org/10.1371/journal.pone.0194373.t002

Texture segmentation and sediment classification

Linear least-squares.

A linear least-squares classification was developed to classify sediment into q sediment types using N classifying vectors V consisting of statistical measures of image texture. The process of assigning a scalar value to each sediment type results in the loss of a significant amount of information because each sediment type represents sediment of various sizes and is best described by a distribution of values. The proportion of variance explained by each of q sediment type is: (6) where: (7) (8) The ∥ indicates an Euclidian norm. The resulting probability of each sediment type is estimated using [50]: (9)

Since ∑ u_q = 1, α_q = 1 for a particular sediment type indicates zero confidence in all other sediment types and we therefore have complete confidence in that particular sediment type. In the unusual case where for n sediment types u_q ≈ 1/n for all n sediment types, a classification is considered indeterminate because equal confidence would exist in all sediment types. After the model is calibrated, a weighting (w_q) can be applied to each variable to produce the highest classification accuracy. Optimal weightings were identified using an optimization technique where the weights were evaluated in increments of 0.1 and constrained such that ∑ u_q ∑ w_q = 1. In the situation where all sediment types had low confidence (i.e. 0.15 < α_q < 0.25) the classification was considered indeterminate and is assigned a null sediment type [50]. Representing a substrate by the mean of an associated texture metric is simplistic, however this linear least-squares classification approach allows us to determine the viability and parsimony of more sophisticated classification approaches.

Gaussian Mixture Model.

With an expectation that there exists a distribution of each texture metric associated with each substrate type, we considered a Gaussian Mixture Model (GMM) approach to classification. A GMM, which has been used in a recent study [56] to classify riverbed substrates from populations of multibeam backscatter, is a model for non-normal distribution as a a mixture of continuous distributions consisting of a finite number of Gaussian density functions [57]. Each Gaussian density function in this case represents a distribution of texture values from a given metric associated with a discrete substrate class.

A GMM is a weighted sum of q components (substrates) within a distribution of any suitable texture measure, v, expressed as (10) subject to: (11) where is an individual Gaussian density function, described by covariance matrix Σ_x and weightings assigned to each model component, w_k, and calculated as: (12) where μ_x is the mean vector of the X, D is the dimension of the vector X, and E[(x − μ)^T (x − μ)] is the covariance matrix. The model parameters, λ = [μ_x, Σ_x, w_x], are estimated using the Expectation-Maximization (E-M) algorithm [58]. The likelihood of the model given the training data is maximized by iteratively evaluating candidate parameters λ. The conditional probability of the sequence of T training vectors V = [v_i, …, v_T] given a parameter set, λ, is (13)

Beginning with an initial proposed λ (typically GMMs are initialized by estimating the mean and variance of V and unit weighting), a new model λ′ is proposed and accepted if p(V|λ′) > p(V|λ). This process is repeated until the E-M algorithm converges on the solution that best represents the data. The Expectation step involves assigning data points to Gaussian density functions by maximizing the likelihood probability a data point came from a particular distribution. Current λ is used to estimate posterior probability, given by (14)

The Maximization step is where λ′ is re-estimated using the probabilities calculated during the E-step. Since posterior probabilities are computed per-pixel and per-substrate, they offer a ready means with which to evaluate classification uncertainty in a spatially distributed sense, or define acceptance criteria for a given classification.

We considered several covariance models, including ‘full’ (), constrained to be diagonal (), or spherical (symmetrical in all directions, or , where D is the number of model parameters). Additionally, we considered a common covariance matrix for all q component substrates, termed a ‘tied’ covariance model where a full covariance matrix is shared among all of the Gaussian density functions. To determine the optimal number of substrates and form of the covariance model, an optimization was performed using the Bayesian Information Criterion (BIC, [59]) as a cost function. BIC scores are used to identify a best fitting model with the lowest number of model components. Models with too many components are prone to over-fitting the data and are assigned a higher BIC score than models with fewer components. Similarly model with too few components under-fit the data and are assigned higher BIC scores than models with more components. Thus, the optimal value of q and covariance model that collectively resulted in the lowest BIC score.

Substrate classification skill.

Each unsupervised classification algorithm was evaluated using accuracy (true positives) as well as precision and recall metrics that are commonly used to accounting for Type 1 (false positive) and Type 2 (false negative) errors. An F₁ score is a weighted average of precision and recall, taking values between 0 and 1, and is given by (15) where precision, P, is the number of true positives in the classification divided by the sum of true and false positives, and recall, R, is the number of true positives divided by the sum of true positives and false negatives.

Results

Sediment texture signatures

In the following subsections, the utility of first and second order (GLCM) statistics are evaluated to identify objective metrics that could be used for the development of automated pixel-by-pixel sediment classification algorithms.