https://orcid.org/0000-0002-5630-7461
Figures
Abstract
Background
The SARS-CoV-2 pandemic began in early 2020, paralyzing human life all over the world and threatening our security. Thus, the need for an effective, novel approach to diagnosing, preventing, and treating COVID-19 infections became paramount.
Methods
This article proposes a machine learning-based method for the classification of chest X-ray images. We also examined some of the pre-processing methods such as thresholding, blurring, and histogram equalization.
Citation: Giełczyk A, Marciniak A, Tarczewska M, Lutowski Z (2022) Pre-processing methods in chest X-ray image classification. PLoS ONE 17(4): e0265949. https://doi.org/10.1371/journal.pone.0265949
Editor: Zahid Mehmood, University of Engineering and Technology Taxila Pakistan, PAKISTAN
Received: November 25, 2021; Accepted: March 10, 2022; Published: April 5, 2022
Copyright: © 2022 Giełczyk et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data available for free at the Kaggle repository. www.kaggle.com/amanullahasraf/covid19-pneumonia-normal-chest-xray-pa-dataset.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
1 Introduction
The occurrence of the COVID-19 pandemic in 2020 has shaken up the modern world. It has caused societies to close, crowded streets to become deserted, pubs and clubs to be silenced, and popular meeting places to die down. Currently, people all over the world are doing their best to overcome the pandemic’s impact on the social, medical, psychologic, economic, and industrial aspects of society. Currently, the main screening method for detecting COVID-19 infections is reverse transcriptase-polymerase chain reaction (RT-PCR) testing. The RT-PCR test can detect SARS-CoV-2 ribonucleic acid (RNA) from respiratory specimens (collected through nasopharyngeal or oropharyngeal swabs). In addition, patients suffering from COVID-19 can also present with abnormalities on chest X-ray images that are characteristic of infection [1]. This imaging modality is highly available and accessible in many clinical locations, and it is considered standard equipment in most healthcare systems. Moreover, CXR imaging is more widely available than CT imaging, especially in developing countries due to high equipment and maintenance costs. However, X-ray analysis can be time-consuming and requires highly educated specialists to interpret. But, the use of machine learning (ML)-based methods can improve efficiency, support medics in the diagnosis of COVID-19, speed up the time to diagnosis, and lighten the already burdened health care system.
At the same time, modern technologies have gathered more interest. Artificial intelligence (AI) and ML can be used in numerous applications such as cybersecurity [2], pedestrian detection [3], telemedicine [4], biometrics [5] or sports analytics [6]. Thus, the implementation of AI and ML in COVID-19 and other lung diseases seems to be the desired natural progression.
In this article, we present the impact pre-processing can have on the results of a classification system. We tested 5 different pre-processing methods and investigated their effect on the final classification. We conducted the study using a large public dataset. The proposed ML-based method was able to classify chest X-rays into 3 classes: normal (healthy), COVID-19, and pneumonia, which can be similar to images of patients infected by COVID-19.
2 Related work
As the COVID pandemic intensified, more investigators focused on automatic lung disease recognition. Milestones in pre-processing, feature extraction, and assigning a classification were required to achieve the required results. In addition, improvements were made at each step of the workflow.
The authors in [7] used a method based on U-NET and ResNet to perform the segmentation of CT images with an accuracy reaching 95%. The main obstacle in overcoming the segmentation problem is imperfect datasets. As mentioned in [8], medical image segmentation datasets suffer from scarce and weak annotations. In addition, acquiring the medical image’s data and annotations can be extremely difficult and expensive. In the article by [9], the authors proposed the use of a multi-level CNN-based preprocessor. The main reason for using this preprocessor was to dynamically enhance the lung regions that are useful in detecting COVID-19. Experiments using ReCoNet for differentiating COVID vs Pneumonia vs Normal were shown to have an accuracy > 97%. Authors in [10] proposed a novel, hybrid, multimodal deep learning system. With the use of Contrast-Limited Adaptive Histogram Equalization (CLAHE) and a Butterworth bandpass filter, the authors were able to enhance the contrast of X-ray images and eliminate the noise leading to an accuracy of 99.93%. The article by [11] highlights that pre-processing can improve a system’s accuracy. In this publication, the visibility of the diaphragm on the chest X-ray was mentioned. It was observed as a very light object in the bottom part of the chest. However, experiments using a convolutional neural network (CNN) reported improved results when the diaphragm was removed from the sample. In [12], an interesting and efficient approach based on the Bayes-SqueezeNet method was proposed. What is more, the authors described some details concerning data augmentation. In this specific study, the augmentation was performed offline using Gaussian blur, sheering, and decreasing/increasing brightness. The presented experiments provided promising results, namely an F1 score of 0.983. Mahdy et al. in [13], proposed a method to automatically classify COVID-19 chest X-rays using a multi-level threshold based on the Otsu algorithm and support vector machine (SVM). A SVM was also utilized in the article by [14]. In the presented approach, image enhancement was performed by increasing contrast, the Histogram of Oriented Gradients was used for features extraction, and Linear Regression and SVM were implemented for X-Ray classification resulting in an accuracy of 96%.
Models are using neural networks to analyze lung X-rays in cancer [15], pneumonia [16], and other lung diseases [17]. In the wake of the recent pandemic, deep learning methods have been used to analyze X-rays of patients potentially infected with the SARS-CoV-2 virus. A standard state-of-art approach using pre-trained CNNs was presented by authors in [18]. In the evaluation of AlexNet, VGG-16, VGG-19, SqueezeNet, GoogleNet, MobileNet-V2, ResNet-18, ResNet-50, ResNet-101, and Xception CNNs, the best results were achieved by Xception and ResNet-101, with an AUC of 0.994 for both networks and an accuracy of 99.02% and 99.51%, respectively. The authors in [19] connected pre-trained deep CNN models with various kernels SVM classifiers. This approach was compared with end-to-end training CNN models that performed worse. The best results (accuracy of 94.7%) were obtained by ResNet50 and SVM with linear kernels. Authors of [20], compared 3 deep-learning based CNN models, InceptionV3, Xception, and ResNeXt. Analysis was performed on 6,432 X-ray scans collected from the Kaggle repository. The highest accuracy was obtained for the Xception model (97.97%). Authors in [21] also tested InceptionV3 and compared the results with 8 more pre-trained CNNs. In that study, the overall accuracy for InceptionV3 was 54.41%, whereas the highest accuracy was achieved by the VGG16 model (95.88%). CoroDet, a novel CNN model, was developed by the authors of [22]. It allows X-ray images and CT scans to be classified into 2, 3, or 4 classes (COVID, Normal, non-COVID viral pneumonia, and non-COVID bacterial pneumonia) with an accuracy of 99.1%, 94.2%, and 91.2%, respectively.
As the COVID-19 pandemic started, researchers focused on providing datasets for performing scientific experiments. Selected datasets are listed in Table 1. As one can see, they contain not only COVID-19 chest X-ray images but also images of other lung diseases like pneumonia and SARS. In ML, the dataset must meet certain requirements. It should be representative of the disease and population being studied, consist of a large sample of data points, and be well balanced. Unfortunately, some of the listed datasets do not meet all of the above-mentioned requirements.
3 Materials and methods
In this research, we used the dataset available for the public at www.kaggle.com/amanullahasraf/covid19-pneumonia-normal-chest-xray-pa-dataset. It consists of images collected from the GitHub repository, Kaggle, Radiopedia, Italian Society of Radiology (SIRM), and Figshare data repository websites. The dataset was organized into 3 classes (COVID-19, pneumonia, and normal) containing posteroanterior (PA) chest X-ray images. A total of 6,939 samples were used in the experiment, and 2,313 samples were used for each class. Some examples of images used are presented in Fig 1.
To verify how the selected pre-processing method affects the final classification result, we proposed a baseline system. The general overview of this system is presented in Fig 2. The black box visible in Fig 2 marks the selected pre-processing method. The pre-processing step is an important element in the image analysis schema. It can enhance the original image and reduce noise or unwanted details. In our research, we examined 6 different approaches to pre-processing:
- None—the baseline approach is not to use any method apart from size reduction.
- Histogram equalization—this method extends the pixel’s intensity range from the original range to 0 to 255. Thus, the enhanced image has a wider range of intensity and slightly higher contrast.
- Hist. eq. + Gaussian blur—this filter reduces some noise and unwanted details that can be confusing for the neural network; the filter kernel size was experimentally set to 5 × 5 size.
- Hist. eq. + bilateral filter—this filter also reduces some noise and unwanted details that can be confusing for the neural network, but its main feature is to preserve edges; the experimentally set up parameters of the filter: diameter = 5, σcolor = σspace = 75.
- Adaptive masking—in [11] the authors proved that by removing the diaphragm from the sample it is possible to improve the classification results. In this proposed pre-processing method, we first found the maximum (max) and minimum (min) intensity of pixels and then applied the binary thresholding using the threshold expressed in Eq 1. The next step used morphologic closing. This creates the adaptive mask that after bitwise operation removes the diaphragm from the source image.
- Adaptive masking + hist. eq. + Gaussian blur—this method joins adaptive masking with histogram equalization and Gaussian blur.
(1)
At image classification, a CNN was implemented. The CNN model can provide human-like accuracy in classifying various images (Fig 2). A convolution network can be described as a chain of convolution layers, with rectified linear unit activation functions, pooling layers, and batch normalization operations. The architecture of a CNN is analogous to that of the connectivity pattern of neurons in the human brain. In fact, it was inspired by the organization of the visual cortex. Individual neurons respond to stimuli only in a restricted region of the visual field known as the receptive field. A collection of these fields overlap to encompass the entire visual area. The hierarchical network provides high-level feature maps, reduced computation complexity, and improved generalization ability. The advantages of CNNs have led to their wide implementation in image processing. For chest X-ray analysis, CNN was implemented in [23–27]. The neural network used in this paper consists of 12 layers:
- Conv2D—it is a convolution layer with 64 filters with dimensions of 3 × 3 and the activation function ReLU, the dimensions of the input data is also introduced
- MaxPooling with a size of 2 × 2
- Conv2D—it is a convolution layer with 64 filters with dimensions of 3 × 3 and the activation function ReLU
- MaxPooling with a size of 2x2
- Conv2D—it is a convolution layer with 128 filters with dimensions of 3 × 3 and the activation function ReLU
- MaxPooling with a size of 2x2
- Conv2D—it is a convolution layer with 128 filters with dimensions of 3 × 3 and the activation function ReLU
- MaxPooling with a size of 2x2
- Flatten—it is a data flattening layer, it has no additional parameters
- Dropout layer—randomly sets input units to 0 with a frequency rate at each step during training time, which helps prevent overfitting. Inputs not set to 0 are scaled up by 1/(1—rate) such that the sum over all inputs are unchanged, in this case, the rate equals 0.2
- Dense layer—in which each neuron is connected to each neuron of the previous layer with the unit parameters (positive integer, dimensionality of the output space) equal to 512, with the activation function ReLU
- Dense—with unit parameters equal to 3, with softmax activation function.
The output from the neural network shows the probability of an image belonging to one of the three classes thanks to the softmax activation function in the last layer. The network selects the classification with the highest probability and identifies it as the final result. All the experiments were executed using the online Kaggle notebook. There were almost 7,000 samples in the dataset. We decided to divide the dataset into three disjoint subsets: training-65%, validating-15%, and testing-20%. All of the experiments were executed 3 times to prove their independence from the learning data. Due to a balanced dataset, we did not need any sample augmentation.
4 Results
The above-mentioned experiments provided some promising results. We used 4 parameters for the evaluation methods—accuracy, precision, recall, and F1-score. The parameters were calculated using a confusion matrix (presented in Fig 3) reporting the number of true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN). The evaluation parameters were calculated using Eqs 2–5.
(2)
(3)
(4)
(5)
Confusion matrices for the experiments varying by the pre-processing method—A: none, B: histogram equalization, C: histogram equalization + Gaussian blur, D: histogram equalization + bilateral filter, E: adaptive mask, F: adaptive mask + histogram equalization + Gausssian blur.
The results without any additional pre-processing resulted in an accuracy of 93% and a F1-score in the range of 91% to 96% for the three evaluated classes. Introducing a pre-processing method improved the parameters, for instance, applying histogram equalization raised the precision, recall, and F1-score by 2%. Nevertheless, the most promising approach was joining histogram equalization with Gaussian blurring and adaptive masking. This approach ensured all evaluated parameters exceeded 97%. The results for each pre-processing method are presented in Table 2.
5 Discussion
5.1 Threads to validity
The presented method is very powerful in diagnosing COVID-19 and pneumonia. However, there are some issues to keep in mind if this modality is implemented in patient care. The first issue is responsibility—who is going to be responsible for the ML-based decision? Thus, we have to specify that the proposed method is not a tool for replacing the educated specialist but to improve his/her work and support the diagnostic process. Furthermore, to implement the proposed method, the explainability of the module must be added to the described pipeline. Explanation of the decisions would be the main task for such a module. It would give the reasons why the sample was classified to a specific class and would be very helpful in marking the part of the image responsible for the decision. A few explainable ML-based methods have been published recently (see: [28–30]).
The second issue is the quality of images used in the learning process. In the presented approach, the dataset was obtained from numerous sources: Github, Kaggle, Radiopedia, SIRM, and Figshare data repository websites. We deeply trust that the images provided are labeled correctly and submitted by an expert. The labeling process seems to be the most challenging, expensive, and time-consuming part of the chest X-ray image analysis system. Due to the relatively recent identification of COVID-19, the number of samples in the datasets are limited. Some commercial projects are working on gathering COVID-19 data (including chest X-rays), but these datasets are currently not available to the public.
5.2 Comparison to SOTA
The detection of COVID-19 and other lung diseases using chest X-ray imaging has recently been widely investigated. Table 3 provides detailed results from the current literature. Unfortunately, not all authors evaluate the same set of parameters as in our study, namely: Accuracy, Precision, Recall, and F1-score. Even though, the provided results prove that our solution is comparable to other SOTA methods.
6 Conclusions
COVID-19 is a highly infectious disease caused by the most recently discovered coronavirus and is considered a pandemic according to the World Health Organization. Even though vaccines were introduced at the beginning of 2021, there is a strong need for fast and accurate tools to improve the efficiency of the healthcare system.
In this article, we proposed a novel approach for the fully automated analysis of COVID-19 chest X-ray images using a neural network. Our approach was successful in distinguishing images into three classes: COVID-19, pneumonia, and normal (healthy). We also presented an improvement in the proposed method, namely the pre-processing part of the ML-based system. In this early step of image analysis, a few crucial operations are performed: adaptive masking (the part of the image that is very light is removed), histogram equalization, and Gaussian blur (removes noise and some unwanted details). We proved that the proposed pre-processing method increases the efficiency of the system as the F1-score raised from 93% to over 97%. Our results are comparable to other similar ML-based approaches in the literature, but there are plenty of pre-processing methods that can improve the efficiency of the system and be implemented in future work.
References
- 1. Ng Ming-Yen, et al. “Imaging profile of the COVID-19 infection: radiologic findings and literature review.” Radiology: Cardiothoracic Imaging 2.1 (2020): e200034. pmid:33778547
- 2. Pawlicki Marek, Michał Choraś, and Rafał Kozik. “Defending network intrusion detection systems against adversarial evasion attacks.” Future Generation Computer Systems 110 (2020): 148–154.
- 3. Nowosielski Adam, et al. “Embedded night-vision system for pedestrian detection.” IEEE Sensors Journal 20.16 (2020): 9293–9304.
- 4. Szymkowski Maciej, et al. “A novelty approach to retina diagnosing using biometric techniques with SVM and clustering algorithms.” IEEE Access 8 (2020): 125849–125862.
- 5.
Giełczyk Agata, Michał Choraś, and Rafał Kozik. “Lightweight verification schema for image-based palmprint biometric systems.” Mobile Information Systems 2019 (2019).
- 6. Ćwiklinski Bartosz, Agata Giełczyk, and Michał Choraś. “Who will score? a machine learning approach to supporting football team building and transfers.” Entropy 23.1 (2021): 90.
- 7. Saood Adnan, and Iyad Hatem. “COVID-19 lung CT image segmentation using deep learning methods: U-Net versus SegNet.” BMC Medical Imaging 21.1 (2021): 1–10.
- 8. Tajbakhsh Nima, et al. “Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation.” Medical Image Analysis 63 (2020): 101693. pmid:32289663
- 9. Ahmed Sabbir, et al. “Reconet: Multi-level preprocessing of chest x-rays for covid-19 detection using convolutional neural networks.” medRxiv (2020).
- 10. Al-Waisy A. S., et al. “COVID-DeepNet: hybrid multimodal deep learning system for improving COVID-19 pneumonia detection in chest X-ray images.” Computers, Materials and Continua 67.2 (2021): 2409–2429.
- 11. Heidari Morteza, et al. “Improving the performance of CNN to predict the likelihood of COVID-19 using chest X-ray images with preprocessing algorithms.” International journal of medical informatics 144 (2020): 104284. pmid:32992136
- 12. Ucar Ferhat, and Deniz Korkmaz. “COVIDiagnosis-Net: Deep Bayes-SqueezeNet based diagnosis of the coronavirus disease 2019 (COVID-19) from X-ray images.” Medical hypotheses 140 (2020): 109761. pmid:32344309
- 13. Mahdy Lamia Nabil, et al. “Automatic x-ray covid-19 lung image classification system based on multi-level thresholding and support vector machine.” MedRxiv (2020).
- 14.
Ullah Syed Irfan, et al. “COVID-19 lung image classification based on logistic regression and support vector machine.” European, Asian, Middle Eastern, North African Conference on Management & Information Systems. Springer, Cham, 2021.
- 15. Woźniak Marcin, et al. “Small lung nodules detection based on local variance analysis and probabilistic neural network.” Computer methods and programs in biomedicine 161 (2018): 173–180. pmid:29852959
- 16. Jaiswal Amit Kumar, et al. “Identifying pneumonia in chest X-rays: a deep learning approach.” Measurement 145 (2019): 511–518.
- 17. Bharati Subrato, Prajoy Podder, and Mondal M. Rubaiyat Hossain. “Hybrid deep learning for detecting lung diseases from X-ray images.” Informatics in Medicine Unlocked 20 (2020): 100391. pmid:32835077
- 18. Abbasian Ardakani Ali, et al. “Application of deep learning technique to manage COVID-19 in routine clinical practice using CT images: Results of 10 convolutional neural networks.” Computers in biology and medicine 121 (2020): 103795.
- 19. Ismael Aras M., and Abdulkadir Şengür. “Deep learning approaches for COVID-19 detection based on chest X-ray images.” Expert Systems with Applications 164 (2021): 114054. pmid:33013005
- 20. Jain Rachna, et al. “Deep learning based detection and analysis of COVID-19 on chest X-ray images.” Applied Intelligence 51.3 (2021): 1690–1700. pmid:34764553
- 21.
Makris Antonios, Ioannis Kontopoulos, and Konstantinos Tserpes. “COVID-19 detection from chest X-Ray images using Deep Learning and Convolutional Neural Networks.” 11th hellenic conference on artificial intelligence. 2020.
- 22. Hussain Emtiaz, et al. “CoroDet: A deep learning based classification for COVID-19 detection using chest X-ray images.” Chaos, Solitons & Fractals 142 (2021): 110495. pmid:33250589
- 23. Tawsifur Rahman, et al. “Transfer learning with deep convolutional neural network (CNN) for pneumonia detection using chest X-ray.” Applied Sciences 10.9 (2020): 3233.
- 24. Jia Guangyu, Hak-Keung Lam, and Yujia Xu. “Classification of COVID-19 chest X-Ray and CT images using a type of dynamic CNN modification method.” Computers in biology and medicine 134 (2021): 104425. pmid:33971427
- 25.
Tobias Rogelio Ruzcko NMI, et al. “CNN-based deep learning model for chest X-ray health classification using tensorflow.” 2020 RIVF International Conference on Computing and Communication Technologies (RIVF). IEEE, 2020.
- 26. Rahman Mohammad Marufur, et al. “HOG+ CNN Net: Diagnosing COVID-19 and pneumonia by deep neural network from chest X-Ray images.” Sn Computer Science 2.5 (2021): 1–15. pmid:34254055
- 27. Pathan Sameena, Siddalingaswamy P. C., and Ali Tanweer. “Automated Detection of Covid-19 from Chest X-ray scans using an optimized CNN architecture.” Applied Soft Computing 104 (2021): 107238. pmid:33649705
- 28. Luca Brunese, et al. “Explainable deep learning for pulmonary disease and coronavirus COVID-19 detection from X-rays.” Computer Methods and Programs in Biomedicine 196 (2020): 105608.
- 29. Singh Rajeev Kumar, Rohan Pandey, and Babu Rishie Nandhan. “COVIDScreen: explainable deep learning framework for differential diagnosis of COVID-19 using chest X-rays.” Neural Computing and Applications 33.14 (2021): 8871–8892. pmid:33437132
- 30. Hou Jie, and Terry Gao. “Explainable DCNN based chest X-ray image analysis and classification for COVID-19 pneumonia detection.” Scientific Reports 11.1 (2021): 1–15.