Introduction

Cell classification via image processing has recently gained interest from the point of view of building computer assisted diagnostic tools for hematological malignancies. The computer assisted image processing tools can evaluate morphological features that are not discernable with human eyes. If automated, these tools can be used to analyze large number of cells in an objective manner for reliable assessment of specific cell populations of interest. The process of ‘Cell Segmentation’ is a precursor to cell classification implemented via image processing and hence, is the first stage of any computer assisted diagnostic tool. Several methods for cell segmentation have been described in the literature and often multiple methods are combined to achieve reasonable results depending on the type of cell images. Broad categories of segmentation methods include intensity thresholding methods, region-based segmentation methods, machine learning based methods and active contour methods [1].

Intensity thresholding based segmentation is one of the simplest and fastest methods of image segmentation. Dorini et al. [2] used intensity thresholding to segment nuclei of mature lymphocytes. Sharif et al. [3] utilized information contained in YCBr color space along with intensity thresholding, morphological operations, and watershed segmentation to segment red blood cells from the microscopic images. The method of Dorrini et al. [2] fails to delineate the regions of interest (ROI) and the method of Sharif et al. [3] does not accommodate spatial intensity variation in images as it depends on the structuring element chosen. Hence, both the methods do not yield robust results, especially, when cells are present in clusters.

Region-based segmentation approaches look for connected components on the basis of properties such as texture and brightness. These approaches include seed based region growing and merging approaches [4–6]. In general, region growing methods are computationally expensive, are sensitive to noise, require correct identification of seeds, are local in nature without any global view, and at times have problem with the stopping criterion.

Machine learning based methods carry out segmentation via grouping of similar pixels (e.g. based on Euclidean distance on intensity) into clusters or by using other methods that learn pixel characteristics. Watershed, k-means clustering, and Support Vector Machines (SVM) are some of the most often used algorithms in segmentation [3, 7–10]. However, none of the above methods are able to segment cells of interest from cell clusters.

Active contour approach works on deformable curves that change their shapes according to the boundaries of targeted objects in an image using internal and external forces defining the motion of closed 2D contours [11–17]. Sadeghian et al. [12] carried out edge detection using Canny filter followed by geodesic snake contour method to segment leukocytes. However, these methods work on single cell windows extracted from the full microscopic image. Also, edge detection fails whenever intensity of nucleus and cytoplasm are similar.

With the focus of our study on building a robust automated pipeline for residual disease estimation in Multiple Myeloma (MM), a type of plasma cell (PC) cancer, the segmentation of the plasma cells was initiated as the first step. The pre-determined challenges specific to plasma cell segmentation are as follows (Fig 1): 1) Plasma cell segmentation requires segmentation of both nucleus and cytoplasm. At times, the color contrast of nucleus and cytoplasm and, more often, of the cytoplasm with the adjacent background is less due to overstaining or understaining. This poses difficulty in cell segmentation; 2) Plasma cells may be clustered together and hence, segmentation of the overlapping/touching cells is required. Generally, this is difficult because of different configurations as a) nuclei of different cells are touching, b) nuclei of one and cytoplasm of another cell are touching, or c) the cytoplasm of different cells are touching; and 3) Presence of more than one type of stained and unstained cells poses another challenge in extracting plasma cells of interest.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Challenges associated with plasma cell segmentation; numbers in boxes indicate the following image regions: nucleus of plasma cells, cytoplasm of plasma cells, unstained cells, and background.

Three challenges are highlighted via this Fig: 1) At times, the color difference between the cytoplasm with the adjacent background is less; 2) Plasma cells may be clustered together and hence, segmentation of the overlapping/touching cells is required; and 3) there may be more than one type of stained and unstained cells posing difficulty in extracting plasma cells of interest.

https://doi.org/10.1371/journal.pone.0207908.g001

Although region growing and machine learning based methods have largely been used in cell segmentation, these methods are not effective in cluster segmentation [4–6, 8]. Contour based approaches such as snake models, level set models, and their variants are increasingly being used for segmentation in medical microscopic images [12–15, 17]. For example, Yang et al. [13] incorporated a color based gradient in the standard Gradient Vector Flow (GVF) model, a contour based approach to exploit the crucial information present in different histological components such as nucleus and cytoplasm of lymphocytes, follicle and mantle cells. Zamani and Safabakhsh [14] worked on a similar approach using GVF based on color gradients with the gradient flow initialized with the nuclei contours to identify nuclei using adaptive histogram thresholding to perform segmentation of lymphocytes. However, the accuracy of segmentation depends on the preliminary step of locating nuclei using histogram thresholding that is generally not robust. Also, this approach fails to segment cell clusters. Yu et al. [15] used level set by Chan-Vese [16] to first segment only the nuclei of nerve cells and later used another level set to segment complete cells (nucleus and cytoplasm). Recently, Lu et al. [17] proposed a joint level set initialized with cell nuclei for pap smear cell segmentation. However, this approach fails in regions of low contrast between the nucleus and the cytoplasm.

From the above literature review, it appears that a contour based method may be able to provide a clear boundary of cells compared to morphological, thresholding, or clustering techniques. Since most of the above contour based methods are deterministic, a probabilistic level set formulation may be able to capture intra-subject and inter-subject related intensity variations within biological components such as within the cytoplasm. Region based and machine learning based methods have largely been used in cell segmentation but these methods are not observed to be effective in cluster segmentation. Contour based approaches such as snake models or level set models are the state-of-the-art medical image segmentation methods that are increasingly being used for segmentation in medical microscopic images [12–15, 17–19] as well as in other medical imaging applications, say, CT segmentation and brain MRI segmentation [20–23].

This motivates us to explore level set formulation within the probabilistic framework for plasma cell segmentation including cluster segmentation from microscopic images. In the present study, the existing methods as well as a recently described method by Saeedizadehet al. [24], using combination of thresholding, modified bottleneck algorithm, and watershed to segment plasma cells, was evaluated for segmentation of plasma cells in our set-up. The ultimate purpose of our work is to build an automated multiple myeloma residual disease detection tool for deployment in the hospital. Incorrect segmentation or partial segmentation of PCs will hinder the development of the subsequent classifier. Thus, we were motivated to explore the problem of PC segmentation afresh for robust results.

Materials and methods

Microscopic images were captured from bone marrow aspirate slides of patients diagnosed with multiple myeloma as per the standard guidelines [25]. Slides were stained using Jenner-Giemsa stain. Images were captured at 1000x magnification using Nikon Eclipse-200 microscope equipped with a digital camera. Images were captured in raw BMP format with a size of 2560x1920 pixels. In all, our dataset consisted of 85 images. We trained our pipeline on 15 images. All the images were stain normalized, using the methodology proposed earlier [26], before being used for segmentation.

Written informed consent was obtained from all the subjects as per the guidelines of the Institute Ethics Committee (IEC) of All India Institute of Medical Sciences (AIIMS), New Delhi, India (Approval No. IEC/NP-145/2013 & RP-32/06.05.2013). Subsequently, a waiver for written informed consent for obtaining photomicrographs from the bone marrow aspirate slides was taken from IEC (approval No. OP-06/01.12.2017). One of the co-authors (RG) had access to the patient identifying information which was completely removed from the image data sets before sharing of data with the other co-authors for building up the PCSeg tool presented in this paper. The dataset is available at the public repository [37].

For the purpose of segmentation, an MM image can be divided into four regions of interest (ROI): (1) nucleus of PC, (2) cytoplasm of PC, (3) unstained cells, and (4) background (Fig 1). For efficient segmentation of all the four ROI, PCSeg Tool has been designed with the following four steps:

1. Step-1: Statistical modeling and computation of separability index of the four ROI in the images
2. Step-2: Removal of unstained cells
3. Step-3: Extraction of nucleus and cytoplasm of plasma cells using the proposed multiphase level set methodology
4. Step-4: Cluster cell segmentation using watershed and circular Hough transform

Step-1: Statistical modeling and computation of separability index of regions of interest

First, the statistical characterization of the four ROI of MM images, i.e., nucleus of PC, cytoplasm of PC, unstained cells, and background (Fig 1) was done and the intensity profile of these regions was studied in different color spaces. A set of fifteen reference MM images, representative of the color histograms, were chosen and the histograms of RGB, HSV, and Lab color channels of the four ROI were marked in these reference images as shown in Fig 1. Since images were at a very high resolution of 2560 x 1920, sufficient numbers of pixels were available for computing the histogram. Histograms of RGB, HSV, and Lab color channels of the four ROI and their corresponding Gaussian probability density functions (PDFs) were fitted to the normalized histograms.

Fitted PDFs are drawn in Fig 2 in the intensity ranges of the original histograms. It is evident from these histograms that 1) nucleus and cytoplasm overlap in every color channel, although this overlap is considerably less in blue (B), hue (H), and value (V) channels; 2) nucleus and cytoplasm appear considerably separated from the background in red (R) and green (G) channels; and 3) the unstained cells do not overlap with nucleus and cytoplasm of plasma cells in the hue (H) channel. Thus, although it may be possible to remove unstained cells using the intensity profile in H-channel, nucleus and cytoplasm of plasma cells cannot be discerned using any single color channel. Rather, a combination of color channels would be required to separate these.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2.

Histogram of four image regions (nucleus, cytoplasm, background, and unstained cells) in:(a)-(c) RGB, (d)-(f) HSV, and (g)-(i): Lab color spaces; Corresponding fitted probability density functions (PDFs): (a′)-(c′) RGB, (d′)-(f′) HSV, and (g′)-(i′): Lab color spaces.

https://doi.org/10.1371/journal.pone.0207908.g002

In order to quantify the separability of different image regions using probability distributions in RGB, HSV, and Lab color spaces, we used Bhattacharyya distance (D_B) as a metric that quantifies separation between two PDFs p and q [27] as:

Prior to computing D_B, we applied contrast stretching on the RGB image such that 1% of lower and higher intensity values are saturated to 0 and 255, respectively. Next, we converted this contrast stretched image (Fig 3b) to HSV and Lab color spaces. The distance between all required combinations of two ROI in RGB, HSV, and Lab color spaces were computed (Table 1). These values are indicative of the separability between different regions and are used in the proposed modified multiphase level set method.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Contrast stretching followed by unstained cell removal, image patch size 2560 × 1920 this figure shows (a) an original image that is (b) contrast stretched such that 1% of lower and higher intensity values are saturated to 0 and 255, respectively.

From the probability density functions of the four regions of interest (nucleus of PC, cytoplasm of PC, unstained cells, and background) of the resulting contrast stretched images, it is observed that nucleus and cytoplasm of plasma cells have maximum separability with unstained cells in H color channel. Unstained cells are removed by replacing intensity of pixels having values less than 120 in the H-channel with the background pixel intensity leading to (c).

https://doi.org/10.1371/journal.pone.0207908.g003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Bhattacharyya distance calculated between different image regions using the ground truth data.

https://doi.org/10.1371/journal.pone.0207908.t001

Step-3: Stained cell extraction using the proposed modified multiphase level set method

We modified the multiphase level set formulation by utilizing statistical information of the four ROI in the image, i.e., nucleus, cytoplasm, unstained cells, and the background, wherein the four ROI were modeled via four-phases of two level sets. Each of the ROI was assigned one phase and the corresponding label. We assigned labels Ω₁₁ and Ω₁₀ to the first level set ϕ₁ for the nucleus and cytoplasm, respectively. Likewise, we assigned labels Ω₀₁ and Ω₀₀ to the second level set ϕ₂ for the background and the remaining unstained cells, respectively. Next, four probability maps of the entire image were created corresponding to each of the four phases of the level set (one each for their respective regions of interest, namely, nucleus, cytoplasm, unstained cells, and the background) as below: (1) where U₀ is the contrast stretched image and c corresponds to color channels with c = 1, 2, 3, 4, …, 8, 9 for channels R, G, B, H, S, V, L, a, and b, respectively. U_0,c is the c^th color channel of the image U₀, Ω_ij is one of the four ROI with i, j ∈ {0, 1}, are the weights, p(U_0,c|Ω_ij) is the conditional probability (over the ROI Ω_ij) of the color channel image, and p(U₀|Ω_ij) is the conditional probability of the original image over all four phases constructed using the weighted probability in all color channels.

Bhattacharyya distance D_B in Table 1 was used to determine weights because it provides an appropriate metric for discerning ROI. Weights for a channel were assigned based on the ability of discerning the desired ROI from all other ROI in that channel. Since Bhattacharyya distance would be higher for larger separation, it can be used as the weight, provided this distance is larger than some minimum threshold. For example, nucleus is discernible from both cytoplasm and background in blue channel with D_B > 1 for each ROI (Table 1). Hence, the maximum of the two distances (distance between nucleus and cytoplasm, and distance between nucleus and background) is chosen as the weight for nucleus in blue channel. On the other hand, nucleus cannot be separated from cytoplasm in the red channel, although it is widely separated from the background in this color channel. This implies that nucleus cannot be extracted from all other ROI in red channel and hence, a zero weight is chosen for nucleus in this channel. Likewise, weights were chosen in stepwise manner for all the ROI, as detailed below.

For determining weights, , in (1) for nucleus, a channel was chosen (R, G, B, H, S, V, L, a, or b) and if the Bhattacharya distance D_B (Table 1) between both 1) nucleus and cytoplasm, and 2) nucleus and background was greater than 1, the maximum distance of the above two was chosen as the weight for nucleus in that channel. Else a value of zero was assigned to the weight in that channel (2). The process was repeated for all the channels. (2)

Similarly, weights were assigned for cytoplasm based on the Bhattacharya distance D_B between both 1) cytoplasm and nucleus, and 2) cytoplasm and background in each of the channels (3). (3)

Weights were assigned for background based on the Bhattacharya distance D_B between both 1) background and nucleus, and 2) background and cytoplasm. Weights were assigned for unstained cells based on the Bhattacharya distance D_B between both 1) unstained cells and nucleus, and 2) unstained cells and cytoplasm. Since plasma cells are required to be clearly delineated from the background and unstained cells, a greater threshold of 3 was considered for background (4) and unstained cells (5). (4) (5)

Final weights () obtained using the above scheme are summarized in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Weights of each ROI for level set equations.

https://doi.org/10.1371/journal.pone.0207908.t002

Next, we defined an energy functional E_p(ϕ₁, ϕ₂) for the multiphase level set formulation, where ϕ₁ and ϕ₂ are the two level set functions that capture the curves of cell boundaries. The energy functional E_p(ϕ₁, ϕ₂) utilizes the above constructed probability maps and was added to the overall functional required to be minimized for the derivation of level set equations. (6) where H is the Heaviside function, is an open and bounded domain, is the given bounded function representing the initial image, and C as the closed subset in Ω made up of finite set of smooth curves.

We also defined another energy functional, namely, distance energy functional E_d(ϕ₁, ϕ₂) that measures the intensity difference of a given pixel from each of the region’s mean color value. To this end, we first defined and computed the distance images for each of the regions in (7) as: (7) where is the mean color value of the distribution in region Ω_ij and is the weight of color channel c in region Ω_ij as tabulated in Table 2. Accordingly, we defined the distance energy functional E_d(ϕ₁, ϕ₂) in (8) as: (8)

Adding the regularization terms of length and area, the proposed modified multiphase level set energy functional is defined in (9) as: (9) where α is a constant regularizer that controls the area inside the contour C, β controls the length of the contour and, η₁ and η₂ are the constants that control relative weighting of the two energy functionals. The level set (ϕ₁, ϕ₂) is periodically re-initialized to the signed distance function [28].

Fig 4 presents Steps 1 to 3 of the proposed method including the modified multiphase level set formulation. The extracted mask of Ω₁₁ ∪ Ω₁₀ from the level set output provides the segmented plasma cells. Fig 5 presents the segmentation results after processing the input image with Steps-1 to 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4.

Schematic diagram of Steps 1 to 3 of the proposed method of PCSeg tool: Regions of interest (ROI) are: Nuclei of Plasma Cells (Ω₁₁), Cytoplasm of Plasma Cells (Ω₁₀), unstained cells (Ω₀₁), and background (Ω₀₀).

https://doi.org/10.1371/journal.pone.0207908.g004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Output of the modified multiphase level set method after Step-3 on an image patch.

It was noted that some unwanted stained cells (e.g. lymphocytes) were segmented to final output. In addition, some small disconnected components that were noisy patches owing to faulty manual staining were also captured by the multiphase level set. Also plasma cell clusters were not segmented.

https://doi.org/10.1371/journal.pone.0207908.g005

On evaluation of the output of the multiphase level set step in Fig 5, it was noted that some unwanted stained cells, such as lymphocytes, were segmented to final output. On careful observation, we noted that the cytoplasm of stained PCs covered large cell area compared to unwanted stained cells and therefore, unwanted cells could be rejected using a threshold on cytoplasm cell area. In addition, some small disconnected components that were noisy patches owing to faulty manual staining were also captured by the multiphase level set. These small noisy disconnected components that are too small to form any ROI were rejected at the output of level set by thresholding on the size of the component.

Although all the four ROI were captured by the multiphase level set, plasma cell clusters were not segmented as observed from Fig 5. To address this problem, Step-4 was added to the tool as detailed below.

Step-4: Cluster cell segmentation using watershed and circle Hough transform

Since stained plasma cells are approximately circular in shape, a combination of watershed and circular Hough transform (CHT) was applied to segment PC clusters. First, it is necessary to segment nuclei as the cases of touching nuclei will lead to improper cell segmentation. The nucleus of a plasma cell has a few distinct features as: 1) the nucleus is dark colored, 2) it is differently colored than the background, and 3) it is always encapsulated within the cytoplasm. Thus, the center of the nucleus could serve as an ideal seed for the watershed algorithm and the nuclei mask obtained as an output of Ω₁₁ phase of the level set could be used to compute the distance transform required by the watershed algorithm.

However, due to the variability in cytoplasm staining, the Ω₁₁ phase was observed to capture nuclei regions more liberally in some images as shown in Fig 6b. This led to unnecessary rejections of some PCs as shown in Fig 6d. Since it is vital for any medical imaging work to segment as many correct cells as possible, k-means (instead of Ω₁₁ phase of the level set) was used to extract the nuclei mask (Fig 6c) and distance transform was applied on this mask. This basin was used by the watershed algorithm to segment the nuclei from clusters. From the watershed output, only those segmented nuclei regions were retained that were circular in nature, i.e., segmented regions that contained a center point of CHT. The non-circular regions identified as nuclei were discarded.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6.

Output of cluster cell segmentation on an image patch Subfigure (a) shows the level set output wherein red boundary shows nucleus of PC being captured by levelset and green boundary shows cytoplasm of PC being captured. However, the cluster of cells are not segmented. (b) shows nuclei identified by levelset phase Ω₁₁ and the problem therein of some extra mask of cytoplasm in nucleus. (d) shows cluster cell segmentation using nuclei mask identified by level set Ω₁₁ in (b). One cell is falsely rejected. (c) shows nuclei identified by k-means on the levelset phase Ω₁₁ and (e) shows correct cluster cell segmentation using the mask of (c).

https://doi.org/10.1371/journal.pone.0207908.g006

Following this, the distance transform of the mask of the stained portions (Ω₁₁ ∪ Ω₁₀) was obtained. The centers of the segmented nuclei obtained from k-means above were used to impose a minima on this basin and subsequently used by the watershed algorithm to segment full plasma cells from clusters. Again, we retained only those segmented regions as cells that were circular in nature, i.e., segmented regions that contained a center point of CHT.

We have named the developed tool as PCSeg Tool-1 for the complete pipeline with k-means based nuclei mask for cluster cell segmentation in Step-4 and named the developed tool as PCSeg Tool-2 for the complete pipeline with Ω₁₁ phase based nuclei mask for cluster cell segmentation in Step-4. The complete process pipeline of cluster cell segmentation with PCSeg Tool-1 is shown in Fig 7.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Schematic diagram of cluster cell segmentation (Step 4 of PCSeg tool) using watershed and circle Hough transform (CHT).

https://doi.org/10.1371/journal.pone.0207908.g007

We noted that while most PCs were segmented, some of the stained lymphocytes and unwanted regions were also retained. In general, the amount of cytoplasm in lymphocytes is considerably less in comparison to PCs. Thus, for each segmented region, the ratio of the nucleus area to the total cell area was used to detect and discard these unwanted cells. As stated earlier, it was observed that the use of nuclei segmented from the phase Ω₁₁ led to inadvertent rejection of some PCs (Fig 6d), while the use of nuclei segmented from k-means helped us in retaining such cells of interest (Fig 6e). Hence, PCSeg Tool-1 based segmentation pipeline appeared more useful. Fig 6e shows the correctly segmented cluster, while the complete process pipeline is shown in Fig 7.

Experimental set-up

All experiments were performed on a Ubuntu 14.04 system with an Intel^® Xeon(R) CPU E5-2630 v2 @ 2.60GHz 12 processor and a GeForce GTX 980/PCIe/SSE2 graphics card supporting CUDA. Level set results depend on the initialization of ϕ’s and therefore, initial contour was set to small circles covering the entire image to ensure faster convergence. The initialization and the energy functionals were calculated on the CPU. The level set propagation and the re-initialization of ϕ’s was implemented on GPU using MEX compiled files containing CUDA code. While the implementation was memory bound, the code achieved upto 75x speed up as compared to the MATLAB R2015b implementation. The rest of the pipeline was implemented in MATLAB. The GPU versus CPU computational speed results are shown in Fig 8.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Running times on CPU and GPU.

https://doi.org/10.1371/journal.pone.0207908.g008

Evaluation metrics

A total of 85 images were considered, where parameters were fine tuned on 15 randomly chosen images. For our experiments, the level set parameters α₁ = α₂ = 0, β₁ = 2, β₂ = 1, η₁ = 1 and η₂ = 3 provided the best results. Parameters α₁ and α₂ are related to the compactness and the area of level set phase captured at the end. Since our images contained both isolated single cells as well as cluster of cells, putting constraints on the area provided poor results. Hence, we chose these parameters to be zero. Parameters β₁ and β₂ control the tightness of the boundary. Since Level set 1 captures the desired plasma cell, we wanted its boundary to be tighter compared to that of Level set 2. Hence, a higher β value was chosen for Level set 1 compared to Level set 2, i.e., β₁ was chosen to be greater than β₂. Since we used stain normalized images, the mean color vector based energy functional captured cells of interest neatly, while the probability based energy functional term took care of the slight color variations owing to subject variability and/or other variability. Hence, η₂ ≥ η₁ provided us best results.

The image dataset evaluated contained 260 single cells and 45 clusters in total. These 45 clusters had a total of 102 cells, with each cluster having two or more cells. For a quantitative assessment of the proposed pipeline and the method, we used TPR (True Positive Rate) or Recall rate defined as

PPV (Positive Predictive Value) or Precision defined as and F₁-Score defined as where TP, TN, FP, and FN stand for true positive (PC detected and segmented as PC), true negative (non-PC rejected), false positive (non-PC segmented as PC) and false negative (PC rejected as non-PC), respectively. In TP or true positives, we only considered those plasma cells that were completely segmented from the image. Any plasma cell that was over-segmented or segmented out with partial portion was discarded. This is to note that F₁- Score is same as the Dice coefficient that is used as a standard metric to assess the performance of segmentation.

Since the rate of detection of false positives is also crucial in medical applications, we evaluated False Discovery Rate (FDR) in our samples as

Results

We compared the results with the traditional levelset, multiphase levelset and k-means in Fig 9. From Fig 9, we notice that the output of these methods do not yield correctly segmented cells (including both nucleus and cytoplasm). Hence, quantification of results will not yield any accuracy with these methods that is worth comparison. Since the work by Saeedizadeh et al. [24] specifically addresses the problem of plasma cell segmentation, its pipeline is tuned to this cell type. Hence, of the existing methods including levelset, we could quantify results of this method only and thus, used the method by Saeedizadeh et al. [24] as the state-of-the-art work as of today for comparison on the given problem statement.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9.

Qualitative comparison of MM cell segmentation using different methods: (a) Gold standard (showing cells of interest with white outlines), (b) k-means, (c) Chan-Vese active contour method [16], (d) Chan-Vese multiphase method [29], (e) Saeedizadeh et al. [24] method, (f) PCSeg Tool-1, and (g) PCSeg Tool-2. All white outlines in (b)-(g) denote the outlines of regions segmented out. These regions are required to be compared with the regions contained in the Gold Standard shown in (a).

https://doi.org/10.1371/journal.pone.0207908.g009

Quantitative results obtained on 260 numbers of isolated single PC and 45 clusters (with 102 PC) are tabulated in Table 3 and the statistical quantities on TPR, PPV, F₁-score, and FDR are tabulated in Table 4. The PCSeg Tool-1 correctly segmented 83.5% of single and isolated plasma cells and 93.3% of PC clusters (71.1% complete clusters and 22.2% partial clusters). PCSeg Tool-2 performed slightly inferior and segmented 55.8% of single plasma cells and 64.5% of PC clusters (48.9% complete clusters and 15.6% partial clusters). Further, the number of false positives detected with PCSeg Tool-1 were about 90 cells leading to an FDR of 23.44% (Table 4). Compared to this, PCSeg Tool-2 detected 102 false positives and thus, had a higher FDR of 34.11% (Table 4). Thus, the PCSeg Tool-1 performed better than PCSeg Tool-2 in terms of both TPR and FDR.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Quantitative results on plasma cell segmentation.

https://doi.org/10.1371/journal.pone.0207908.t003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Statistical results on a total of 364 (260+102) plasma cells.

https://doi.org/10.1371/journal.pone.0207908.t004

Further, we compared both the proposed methods, i.e., PCSeg Tool-1 (PCSeg Tool with k-means as nuclei mask for cluster cell segmentation) and PCSeg Tool-2 (PCSeg Tool with Ω₁₁ phase of level set as nuclei mask for cluster cell segmentation) with existing cell segmentation methods relevant to our problem. These existing methods included k-means method, Chan-Vese active contour, Chan-Vese multiphase methods, and a recently described method by Saeedizadeh et al. [24]. k-means is a widely used method, while the proposed multiphase level set method in PCSeg Tool-1 and PCSeg Tool-2 is a refinement over standard active contour methods of Chan-Vese active contour [16] and Chan-Vese multiphase [29] methods. All these methods, i.e., k-means and standard active contour methods, have been used earlier on segmentation of cells other than plasma cells. The method of Saeedizadeh et al. [24] addresses specifically the plasma cell segmentation. Hence, all the above four methods were chosen for qualitative comparison with PCSeg Tool-1 and PCSeg Tool-2.

Discussion

The k-means method provided many false positives, missed many PCs, and could not segment clusters of PCs (Fig 9). Similar was the case with standard active contour methods of Chan-Vese active contour and Chan-Vese multiphase methods. Sincek-means, Chan-Vese active contour [16], and Chan-Vese multiphase [29] methods performed poorly on PC segmentation, quantitative results have been presented on only the rest of the three methods (Tables 3 and 4).

As compared to Saeedizadeh et al. [24], PCSeg Tool-1, and PCSeg Tool-2 performed far more superior to k-means and standard active contour methods (Tables 3 and 4). While PCSeg Tool-2 did not outperform [24] in the number correct PC cell segmentation, it did better with FDR compared to [24] which led to a very high number of 192 false positives with an FDR of 47.52% (Table 4). The method by [24] performed second best by segmenting 62% of single plasma cells and 64.5% of PC clusters (57.8% complete clusters and 6.7% partial clusters). However, one of the 4 plasma cells present in Fig 9 has been incorrectly segmented by [24]. On the other hand, PCSeg Tool-1 and PCSeg Tool-2 captured all 4 cells. This result shows that both the variants of PCSeg Tool (1 and 2) performed better in capturing the cells of interest.

This is an expected result because the methods specifically designed for plasma cell segmentation, i.e., PCSeg Tools 1 and 2 and [24] take into account the problems inherent to plasma cell segmentation. Further, these results establish that the method applicable on one cell type cannot be ported for segmentation of another cell type directly, i.e., no single method or segmentation pipeline can be applied to all the different cell types.

We also compared the performance of the above three segmentation tools with reference to Precision, Recall, and F1-score. While recall rate quantifies a tool’s performance with respect to false negatives (how many plasma cells were missed out in segmentation), precision rate informs us about the performance of the tool with respect to false positives, i.e., other cells segmented out as plasma cells. Thus, recall informs what ratio of correct plasma cells could be segmented, while precision tells how many of the cells segmented as plasma cells were erroneous. Thus, both precision and recall have a significance and both these are imbibed in the F₁-score that should be as high as possible and informs about the overall performance of the segmentation tool. It is noted that recall of PCSeg Tool-1 is good as 81.66% compared to 54.72% of PCSeg Tool-2 and 58.88% of [24]. Precision of PCSeg Tool-1 is also good as 76.56% compared to 65.88% of PCSeg Tool-2 and 52.47% of [24]. Thus, as expected, PCSeg Tool-1 provides best F1-score of 79% compared to 59.78% of PCSeg Tool-2 and 55.49% of [24]. Thus, although PCSeg Tool-2 could not segment as many PCs as [24], it yielded more correct segmented cells compared to [24].

Overall, PCSeg Tool-1 performed best and provided better TPR, PPV, and smaller FDR. It was not only able to reduce the number of false negatives; it was also able to reduce false positives. For use in real clinical treatment, false negative rate of PCs should be low as well as false positives should be low because less than required number of chemotherapy sessions due to poor recall (or missing of large number of true PCs) or more than required number of chemotherapy sessions on aged people owing to high false discovery can prove to be fatal.

Fig 10 presents qualitative (visual) comparison of these methods on some more images. The good performance of PCSeg Tool-1 with k-means identified nuclei mask for cluster cell segmentation in the modified level set formulation implies that perhaps the proposed probability based and mean color vector based energy functionals in the multiphase level set, added with the robustness of the k-means on the level set output for nuclei mask for subsequent cluster cell segmentation, is playing a key role in the robust segmentation of PCs. Thus, we name the PCSeg Tool-1 as the final PCSeg Tool product for use with plasma cell segmentation.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10.

Qualitative comparison of MM cell segmentation using different methods over five images: (a) Gold standard (showing cells of interest with white outlines), (b) Saeedizadeh et al. [24] method, (c) PCSeg Tool-1, and (d) PCSeg Tool-2. White outlines in all figures (b)-(d) denote the outlines of regions segmented out. These regions are required to be compared with the regions contained within the white boundaries in Gold Standard shown in (a).

https://doi.org/10.1371/journal.pone.0207908.g010

Conclusions and future work

In this paper, we designed, described, and implemented PCSeg tool for the segmentation of plasma cells from microscopic images. This tool has a robust pipeline consisting of modified multiphase level set method that utilizes statistical information about the probability densities of regions of interest (ROI) and the mean color vector of ROI in the color spaces in the multiphase level set. The level set stage removed the background and most of the unwanted cells. Only the stained single cells or the clusters of cells were retained after its application. Next, we tried two variations of PCSeg Tool: Tool-1 that utilized k-means based nuclei mask in the cluster cell segmentation and Tool-2 that utilized one of the phases of the level set for nuclei mask in cluster cell segmentation, where cluster segmentation was carried out with watershed and circular Hough transform and unwanted cells are completely removed in the post-processing stage of PCSeg Tool. PCSeg Tool-1 provided best results with better recall, precision, and F1-score. Further, the implemented PCSeg Tool-1 provided good results on segmentation of single isolated plasma cells as well as segmentation of plasma cells from cell clusters.

Recently, cell segmentation with deep learning (DL) has started picking up pace. However, as of now there are only a few papers with DL on cell segmentation [30–35]. Most of these methods have dealt with nucleus segmentation and so far, there is no paper on plasma cell segmentation using deep learning. Plasma cell segmentation is a more challenging problem compared to nucleus segmentation because (i) it requires both nucleus and cytoplasm segmentation, (ii) the color contrast of cytoplasm is sometimes very near to background, and (iii) cluster segmentation is also a problem because it can include a cluster of touching nuclei, touching nucleus with cytoplasm, touching cytoplasm of different cells, etc. Hence, recently proposed DL methods cannot be directly ported on this dataset. Solving the problem of plasma cell segmentation using DL is a challenging research problem that we plan to attempt in the near future. In addition, recent optimization/regularization based methods used in other domains of medical image segmentation similar to low rank and sparse decomposition method of [36] can also be explored in cell segmentation.

Acknowledgments

Authors gratefully acknowledge the research funding support (Grant Number: 1(7)/2014-ME&HI) from the Ministry of Communication and IT, Govt. of India and funding support (Grant: EMR/2016/006183) from the Department of Science and Technology, Govt. of India for this research work. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. We would also like to thank the support of Dr. Hossein and Mrs. Saeedizadeh [24] for providing their code for the experiments.