Probabilistic segmentation

These algorithms involve prior models that describe the signal distributions of each brain structure. Ng et al. [7] segmented MR brain images using the unsupervised K-means clustering of signals and an improved watershed algorithm. A similar approach by Xue et al. [5] employed a parametric Gaussian density estimation with an expectation-maximization (EM) algorithm and constrained spatial homogeneity of the MR images with a Markov random field (MRF) prior. Partial volume averaging effects have been eliminated by predicting the misclassification (e.g., of an “averaged” CSF and GM into an intensity similar to WM). An automated MRI brain segmentation method by Mayer et al. [8] combined spatial and intensity features into a high-dimensional feature space. An adaptive mean-shift classifier extracted a set of convergence modes, i.e. high-density points of a feature space, being good candidates for intensity-based classification. Brain tissues were classified by an intensity-based mode clustering. This approach was very effective with non-convex clustering. Fang et al. [9] developed a tree metrics (TM)-based graph cut algorithm to segment the MRI brain tissues. After a brain MR image is classified using the TM, the goal labeling is inferred by “tree-cutting”. In contrast to most of conventional iterative methods like the EM-based ones, which produce only locally optimal labeling, this algorithm needs no more than one sweep to generate the globally optimal labeling with respect to the TM. An automated segmentation approach by Ortiz et al. [10] classified the brain tissue with no prior information. The segmentation consisted of feature extraction and classification. Extracted first order (pixel/voxel-wise), second order (pair-wise), moment, and scale-invariant features were classified by growing hierarchical self-organizing maps (GHSOM). Li et al. [11] proposed a 3D MGRF model for the segmentation of brain MR images to avoid the shortcomings of the 2D model which is not able to fully capture the spatial information, especially among the slices. An initial segmentation was first obtained by k-means clustering in order to reduce the extensive computations required by the MGRF model. The Iterated Conditional Modes (ICM) algorithm was finally applied to obtain the optimal solution under maximum a posteriori (MAP) criterion. A non-parametric adaptive mean-shift algorithm was proposed by Janney et al. [12] for brain tissue segmentation. The method clustered the joint spatial-intensity feature space, followed by a phase of intensity-based mode clustering into the brain tissue types. Weber et al. [13] segmented the brain tissues using FSL-FAST free software that is based on K-means clustering which provided initial segmentation, followed by the EM algorithm for bias field correction. In order to speed up the process, parallelization to any eligible parts of the software was applied, which needed some adaptation to the algorithms in order to maintain the accuracy obtained by the software package. Mahmood et al. [14] proposed an unsupervised framework for brain tissue segmentation using a combination of Bayesian-based adaptive mean shift that clustered the tissues in the joint spatial-intensity feature space, and fuzzy c-means that is initialized with a priori spatial tissue probability maps to assign the clusters into three tissue types; WM, GM, and CSF.

Infant brain segmentation using statistical-based methods was also addressed in literature. Automated segmentation of brain structures, such as WM, CSF, central GM (CEGM), and cortical GM (COGM) was conducted by Anbeek et al. [15] using T2-weighted and inversion recovery (IR) MRI of the neonatal brains. Probability maps to segment each brain tissue class with a K-nearest neighbor (KNN) classifier using voxel intensities and coordinates as features were constructed manually. A multi-label segmentation process combined the obtained classes. Wang et al. [16] employed a random forest technique to integrate features from different modalities for brain tissue segmentation in infants along with probability maps of GM and CWM. Zhang et al. [17] proposed a deep convolutional neural networks (CNNs) approach for segmenting the neonatal brains from multi-modal MR images, generating the segmentation maps as outputs. The multiple intermediate layers included many operations such as convolution, pooling, and normalization in order to capture the highly nonlinear mappings between inputs and outputs. Moeskops et al. [18] segmented neonatal brains into WM, GM, and CSF using supervised voxel classification.

To recapitulate, statistical-based techniques are generally fast to implement compared to other segmentation methods. However, the fact that actual intensity distributions of brain structures are greatly affected by several factors, such as the unique patient and scanner along with scanning parameters, makes the segmentation hard. Also, due to the similar intensities for the different brain tissue structures of the infant MR brain images, segmentation techniques only based on the intensity remain inaccurate.

Atlas-based segmentation

Atlas-based approaches have emerged as powerful segmentation tools. These approaches are based on a priori knowledge about brain structures, and treat the segmentation problem as a registration one. Ashburner et al. [19] introduced a generative framework that combined image registration, tissue classification, and bias correction. Their framework incorporated a smooth intensity variation and nonlinear registration with tissue probability maps using mixture of Gaussians. Pohl et al. [20] introduced a Bayesian model for simultaneous segmentation and registration. Their framework tried to exploit complementary aspects of registration and segmentation problems. In order to account for different physiological (patient size and weight) and scanning (scanner type and data acquisition protocol) parameters, Han et al. [21] introduced an intensity re-normalization procedure to adjust the prior atlas intensity model to new input data to overcome the problems stemming from using training data acquired from a different scanner that was used for the test data. Artaechevarria et al. [22] proposed a generalized local weighting voting scheme in which the fusion weights were adapted for each voxel based on local estimation of the segmentation performance. The local weighting voting outperformed traditional global strategies that estimate a single value for the segmentation accuracy for the whole image. Sabuncu et al. [23] proposed an automated, label fusion segmentation technique. In order to capture greater inter-subject anatomical variability, each training data set was individually co-registered to the test data set. Then, a nonparametric probabilistic model was employed to fuse the training labels to compute the final segmentation. Morin et al. [24] presented an atlas-based segmentation framework using random walks that combined registration and labeling propagation steps. They used a generative model to provide pixel label probabilities to improve the segmentation for high-confidence labels. To match the target images with atlas images, they used the Affine-Scale Invariant Feature Transform (ASIFT) [25] and Speeded Up Robust Features (SURF) [26] registration techniques. In order to avoid segmentation errors produced by registration imperfection, Lötjönen et al. [27] introduced an optimized pipeline for multi-atlas brain MRI segmentation. They introduced two approaches that combine multi-atlas segmentation and intensity modeling based on using EM and graph cuts for optimization. First, they registered all atlases to the target data and a majority voting was applied to predict the segmentation of the target image. Then, the segmentation was improved using the intensity modeling as a post-processing step. Lijn et al. [28] introduced a segmentation method based on the combination of spatial features and appearance models. They generated a spatial probability map that was obtained from multiple atlas-target image registrations to implement the spatial model. The tissue appearance was modeled by a KNN classifier using Gaussian scale-space features. Then, a Bayesian framework was used to combine both spatial and appearance models and a graph-cut approach [29] was used for optimization. Ledig et al. [30] introduced a framework for labeling whole brain scans by incorporating a global and stationary MRF to ensure consistency of the neighborhood relations between structures with an a priori defined model.

Segmentation of neonatal brains was also conducted in the literature using atlas-based techniques. Segmentation of axial neonatal brain MRI that combined multi-atlas-based segmentation and supervised voxel classification was proposed, [31], in order to segment eight different tissue classes, namely cortical grey matter (CoGM), unmyelinated white matter (UWM), brainstem, cerebellum, ventricles, cerebrospinal fluid in the extra-cerebral space (CSF), basal ganglia, and myelinated white matter (MWM). Some approaches use longitudinal scans at a late-time-point age, where the contrast is much better between different tissue types, from which probabilistic atlases are constructed to guide segmentation of neonatal images [32, 33]. Cherel et al. [34] employed a subject-specific atlas that is based on manually segmented data for brain tissue classes segmentation. The atlases were incorporated with single-atlas expectation maximization (EM) method. Neonatal brains were segmented into 50 regions by [35], where structural hierarchy along with anatomical constraints were employed. Infant brain segmentation using shape priors was also addressed in the literature [36–38].

Atlas-based segmentation techniques show more accuracy compared to statistical-based techniques. Nevertheless, they are still challenged by atlas selection, combination, and the associated heavy computation time. Another major drawback of atlas-based segmentation algorithms is their dependency on the selected features that will be used to link between the test subject and the prior (training) data used in the construction of the atlas. This may lead to inaccurate segmentation, as signals vary due to many factors such as the patient’s age, and the scanning protocol.

Hybrid methods

The literature also shows methods that exploited both probabilistic models along with shape prior (atlas-based) ones. Song et al. [39] proposed a probabilistic neural network (PNN) for segmenting the brain MRI. Probability density functions of the brain tissues were estimated from reference vectors generated by a self-organizing map (SOM). To reduce the partial volume averaging effects, weighting factors were added to the summation layer’s patterns in a weighted probabilistic neural network (WPNN) and soft labeling was performed by a supervised Bayesian classifier. Patenaude et al. [40] proposed a method that used manually labeled image training data, where the principles of both the active shape and appearance models were utilized within a Bayesian framework, allowing probabilistic relationships between shape and intensity to be fully used. Serag et al. [41] employed high-dimensional feature vectors for segmenting brain subjects using a sliding window approach along with a multi-class random forest classifier. Wang et al. [42] segmented T1, T2, and diffusion-weighted brain images using a sparse representation of the complementary tissue distribution. Initially, the brain tissue was segmented into different structures using a patch-based technique with a library of multi-modality images, having been aligned with their ground-truth segmentation maps. Then the segmentation was refined by integrating geometric constraints.

Deformable-model based segmentation

Deformable models have also been employed for brain segmentation in the literature. Angelini et al. [43] introduced a multi-phase level set framework for automated segmentation of brain MRIs. The segmentation of the brain tissues (WM, GM and CSF) was solely based on homogeneity (average grey level) measures. To avoid the need for any prior information and to speed up numerical calculation, a random seed for initialization of the deformable boundaries was used. Colliot et al. [44] proposed a deformable-model based approach that used spatial constraints, represented as fuzzy subsets of the 3D image space, as an external force to control the boundary evolution. To avoid manual selection of the model parameters, a training step was required to estimate the spatial constraints parameters. Miri et al. [45] introduced a topology-preserving deformable model framework for the segmentation of brain MRIs. They employed photometric constraints to guide the deformable model deformations to iteratively reclassify the points located at the evolving boundaries. A deformable model approach for the segmentation of brain regions from MR images was proposed by Liu et al. [46]. The deformable contour was implicitly represented by a set of Wendland’s radial basis functions (RBFs) and was evolved by iterative updates of the locations of the RBFs. Huang et al. [47] introduced an automated, hybrid deformable model framework that integrated both image edge geometry and voxel statistics features to regularize the convergence of the deformable contour. Del Fresno et al. [48] described a hybrid method that combined region growing and deformable models for segmentation of different structures in head MRI and Computed Tomography CT scans. Their approach used a Region-Growing (RG) algorithm to compute an approximation of the objects. This was followed by generating closed and oriented surface meshes to enclose the region of interest. The deformable model method geometry was constructed using the RG-list of boundary voxels generating a hole-free surface mesh. To better detect the structures of interest, the user could select few seeds for RG initial segmentation. Wang et al. [49] proposed a multi-phase level set framework to segment brain MR images with intensity inhomogeneity. They modeled the local image intensities using Gaussian distributions with different means and variances. Then, a variational approach minimized an energy function to compute the means and variances that would guide the contour evolution towards the target boundaries. Bourouis et al. [50] developed a level set framework for segmenting brain tissues. Their framework employed an image registration step and a classification step for the initialization of the deformable boundary. The boundary evolution was controlled by a speed function that accounted for both boundary- and region-based properties. Ciofolo et al. [51] developed an automated framework based on level sets for simultaneous segmentation of multiple structures from brain MRIs. The evolution of each level set was driven by a fuzzy decision system that combined three factors: intensity distribution of the 3D MR volume, the relative position of the evolving contours, and a priori knowledge provided by an anatomical atlas. Wang et al. [52] proposed a multi-layer background subtraction technique with a seed region growing approach which used local texture features represented by local binary patterns (LBP) and photometric invariant color measurements in RGB color space for brain segmentation. Zhao et al. [53] segmented brain tissues using a method adapted from Chan and Vese model, named automatic threshold level set without edges. Thresholds were obtained by fuzzy c-mean algorithm.

Segmentation using deformable models was also exploited in infant brains, [16, 32, 33, 42].

The main advantage of deformable-model based segmentation techniques is the ability to segment connected (non-scattered) objects more accurately than the other segmentation methods. However, the accuracy of this method is based on the accurate design of the guiding forces (statistical, geometric, etc.) as well as the model initialization. A summary for all cited related work is provided in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Summary of brain segmentation related work.

https://doi.org/10.1371/journal.pone.0187391.t001

Deep-learning based segmentation

De Brebisson et al. [54] proposed a deep artificial neural network for brain image segmentation from MR scans that assigns each voxel to its corresponding anatomical region. The information to the network is obtained at different scales around the voxel of interest: 3D and orthogonal 2D intensity patches capture a local spatial context. Also, large compressed 2D orthogonal patches and distances to the regional centroids are used so that they would enforce global spatial consistency. Zhang et al. [17] proposed a method that is based on deep convolutional neural networks for multi-modality isointense infant brain image segmentation. Multi-modality information from T1, T2, and fractional anisotropy (FA) images were exploited as inputs to generate the segmentation maps as outputs. The multiple intermediate layers applied convolution, pooling, normalization, and other operations to capture the highly nonlinear mappings between inputs and outputs. Chen et al. [55] proposed a deep voxelwise residual network, referred as VoxResNet, which handles volumetric data. It integrated the low-level image appearance features, implicit shape information, and high-level context together for improving the volumetric segmentation performance.

In summary, brain segmentation work found in the literature suffer from many drawbacks as mentioned above. Moreover, infant brain segmentation techniques depend on either multiple modalities which lengthens the processing time, or on longitudinal studies which are not always available for research purposes. Also the reduction in contrast between CWM and other structures hardens the issue of preserving its edges by most of the available techniques.

To overcome the aforementioned limitations, this paper proposes a novel technique for brain segmentation from MR images. Adaptive probabilistic shape models for the shape and first-order visual appearance of MRI data are employed to initialize the segmentation. This is then combined with a novel higher-order Markov Gibbs random field (MGRF) spatial interaction model (up to fourth order) with analytic estimation of potentials. This joint model guarantees increasing the segmentation accuracy by accounting for the large-scale inhomogeneities and noise in infant brain MRI data. Also, the analytic estimation of potentials generalizes the proposed model to different MRI subjects, unlike using empirical values in most of the work present in literature which would require manual setting for each subject. The strength of the proposed algorithm lies in the fact that it neither depends on multiple modalities for acquiring images nor on longitudinal studies.

Preprocessing

Before segmentation takes place, bias correction is applied to the brain volumes using the nonparametric approach proposed by Tustison at al. [56] to account for nonuniform intensities. This is followed by applying a 3D generalized Gauss-Markov random field (GGMRF) model [57] that reduces noise effects and removes inconsistencies of the scans.

The skull is then removed before segmentation takes place. This work uses the automated approach proposed by our group in [58], which refines the skull stripping results from the brain extraction tool (BET). This is achieved using an additional processing step that is based on the geometric features of the brain. Since the non-brain tissues are brighter than brain tissue, this step exploits the visual appearance features of the MR brain data. Namely, an evolving iso surface-based approach is proposed to remove the non-brain tissues, which is guided by the MR data visual appearance features. First, a set of nested iso-surfaces are generated by the fast marching level sets (FMLS), using the calculated distance map of the extracted brain from the BET step. Then, classification of voxels as brain or non-brain is conducted. Results on both infants and adults using this approach showed the capability of the proposed approach, outperforming four widely used brain extraction tools: BET, BET2, brain surface extractor (BSE), and infant brain extraction and analysis toolbox (iBEAT).

Joint MGRF model of MR brain images

Let R = {(x, y, z) : 0 ≤ x ≤ X − 1, 0 ≤ y ≤ Y − 1, 0 ≤ z ≤ Z − 1}; Q = {0, 1, …, Q − 1}; and L = {0, …, L} denote a finite 3D arithmetic lattice of the size of XYZ supporting grayscale images and their region (segmentation) maps, a finite set of Q integer gray values, and a set of region labels L, respectively. Let g = {g_x,y,z : (x, y, z) ∈ R; g_x,y,z ∈ Q} and m = {m_x,y,z : (x, y, z) ∈ R; m_x,y,z ∈ L} be a grayscale image taking values from Q, i.e., g : R → Q, and a region map taking values from L, i.e., m : R → L, respectively. An input brain image, g, co-aligned to the training data base, and its map, m, are described with a joint probability model: P(g, m) = P(g|m)P(m), which combines a conditional distribution of the images given the map P(g|m), and an unconditional probability distribution of maps P(m) = P_sp(m)P_V(m). Here, P_sp(m) denotes a weighted shape prior, and P_V(m) is a Gibbs probability distribution with potentials V, which specifies a MGRF model of spatially homogeneous maps m. Details of the model’s components are outlined below.

Adaptive shape model P_sp(m).

To start the segmentation process, a database is created, where expected shapes of each brain label are constrained with an adaptive probabilistic shape prior. To create the atlases, a training set of images, collected for different subjects (not included as test subjects), are co-aligned by 3D affine transformations with 12 degrees of freedom (3 for the 3D translation, 3 for the 3D rotation, 3 for the 3D scaling, and 3 for the 3D shearing) in a way that maximizes their Mutual Information (MI) [59]. An atlas of 10 subjects containing the three labels to be segmented (WM, GM, and CSF) was constructed for each age group (infants, children, and adults) as described. For each input MR data to be segmented, the shape prior is constructed by an adaptive process guided by the visual appearance features of the input MRI data [60–62]. The shape prior is a spatially variant independent random field of region labels for the co-aligned data: (1) where p_sp:x,y,z(l) is the voxel-wise empirical probabilities for each brain label l ∈ L. First, the normalized cross correlation similarity coefficient is used to select the subject from the shape database that has the best match with the input subject (i.e., highest similarity). The selected subject is then used as a reference prototype to co-align the input subject using the 3D affine transformation described above. In order to estimate the shape prior probabilities for each voxel in the test subject, the steps summarized in Algorithm 1 are followed. Fig 3 show the calculated probabilistic maps for each structure using the proposed adaptive shape prior.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. The calculated shape probability for the CSF(a), GM(b), and WM(c).

https://doi.org/10.1371/journal.pone.0187391.g003

Algorithm 1 Steps of the shape prior segmentation.

1. Align the test subject with the shape database to get the 3D affine transformation matrix T.

2. For each slice i, i = 1 to n

  I. For each voxel v in slice i

   (a) Transform v to the atlas domain using the transformation matrix T.

   (b) Initialize a 3D cube, C, of size N_1i × N_2i × N_3i centered around the mapped voxel (v_mapped).

   (c) Search C for voxels with corresponding grey level in all training sets with equalized intensities that fall within a predefined tolerance ±τ in w.

   (d) If no voxels are found using Step (c), increase size of C and repeat step (c) until correspondences are found or the maximum size allowed for C is reached.

   (e) Calculate the shape probability for each structure at location r based on the found voxels and their labels.

   End for

  End for

3. Return the constructed 4D shape probabilities.

First-order intensity model P(g|m).

The first-order visual appearance of each brain label is modeled by separating a mixed distribution of voxel intensities of the brain scans into individual components associated with the dominant modes of the mixture. The latter is precisely approximated with a Linear Combinations of Discrete Gaussians (LCDG) [63] with positive and negative components, which is based on a modified version of the classical Expectation-Maximization (EM) algorithm.

Let Ψ_θ = (ψ(q|θ) : q ∈ Q) denote a discrete Gaussian (DG) with parameters θ = (μ, σ), integrating a continuous 1D Gaussian density with mean μ and variance σ² over successive gray level intervals. The LCDG with four dominant positive DGs and C_p ≥ 4 positive and C_n ≥ 0 negative subordinate DGs is [63]: (2) where all the weights w = [w_p:k, w_n:κ] are non-negative and meet an obvious constraint . All LCDG parameters, including C_p and C_n, are estimated from the mixed empirical distribution to be modeled using the modified EM algorithm [63]. Fig 4 shows an example of the empirical density for different brain tissues using the LCDG model for an infant subject, Fig 4(a), and an adult one Fig 4(b).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Normalized empirical density using the LCDG model for an infant subject (a), and an adult one (b).

Note that dashed = empirical, red = CSF component, green = GM component, blue = WM component.

https://doi.org/10.1371/journal.pone.0187391.g004

MGRF model with second- and higher–order cliques P_V(m).

In addition to the first-order visual appearance model, the spatial interactions between the brain voxels are also taken into account. Using spatial models that are only second-order-clique based, (e.g., [64]), will not enable accounting for the spatial inhomogeneity of brain MR images, especially for infants. Therefore, in this paper we propose a higher-order Markov-Gibbs Random Field (MGRF) spatial interaction model that adds the families of the triple and quad cliques to the pairwise cliques (Fig 5(b) and 5(c)), along with analytical estimation of the potentials. The proposed approach accounts for the spatial inhomogeneity of the brain scans, especially for those of infants, thus, reducing noise effects and increasing segmentation accuracy. Details of the proposed higher-order MGRF model are described below.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Samples of the second- (a), third- (b), and fourth-order (c) cliques for the 26-neighborhood (graph cliques are shown in different colors for visualization purpose).

https://doi.org/10.1371/journal.pone.0187391.g005

Let C_a denote a family of s-order cliques of an interaction graph with nodes in the 3D lattice sites (x, y, z) and edges connecting the interacting, or interdependent, sites (see Fig 5). To account for the scan inhomogeneities, especially with infant MRI, the label interactions are modeled by a spatially homogeneous MGRF with up to fourth-order interactions over the nearest 26-neighborhoods of voxels: (3) where A clique families describe the geometry of the graph interactions, V = [V_a : {0, …, L} → (−∞, ∞) : a = 1, …, A] is a collection of Gibbs potential functions V_a for the families C_a, and the partition function Z_V normalizes the probabilities over the parent population of all maps. An initial region map m, obtained by the voxel-wise classification, allows for analytically approximating the maximum likelihood estimates of the potentials and computing the voxel-wise probabilities of the region labels. For symmetry sake, only equality or inequality of the labels in clique c is taken into account. The second- third- and forth–order potentials are given by Eqs (4), (5) and (6), respectively: (4) where , and F(m°) = [ρ_aF_a(μ₁, …, μ_s|m°) : (μ₁, …, μ_s) ∈ {0, …, L}^s; a = 1, …, A] is the collection of scaled relative frequencies of co-occurrences of configurations (μ₁, …, μ_s) of the labels in the cliques of each family C_a over a given training map m°. (5) where (6) where and where m_pi is the region map label at the voxel p_i = (x_i, y_i, z_i). The proposed analytical approximation of the Gibbs potentials from a given map m extends earlier second-order MGRFs (e.g., [64]) to the higher-order models. The complete proof for the higher-order MGRF model is provided in the supplement.

Finally, the region map m is improved using Iterative Conditional Mode (ICM) algorithm [65] that maximizes the probabilities of the 3D joint model. The complete steps of our segmentation framework are summarized in Algorithm 2. Also Fig 2 illustrates the whole proposed framework.