Brain cone beam computed tomography image analysis using ResNet50 for collateral circulation classification

ABSTRACT


INTRODUCTION
Cerebrovascular disease is a common brain injury that can cause life-threatening to humans [1]. Stroke is a typical symptom of cerebrovascular disease, and it is the leading cause of death and severe disability [2]. Over 85% of stroke patients lose their lives as a result of ischemic occlusions [3], with intracerebral hemorrhage contribute the remaining 15%. Ischemic stroke happens when the brain experiences insufficient blood flow due to two situations; thrombotic and embolic [4]. When blood flow to the brain is disrupted within a blood artery owing to vascular dysfunction, causing harm to patients in many aspects [5]. Wastages from other parts of the body obstruct the blood flow via the afflicted artery during an embolic event [6]. This results in cerebral hypertension, or a shortage of oxygen in the brain, which leads to brain tissue death or ischemic stroke [7], [8]. When a patient comes to the hospital after trauma, the emergency department would assess the patient. A detailed neurological examination must be completed after primary and secondary surveys. However, endovascular treatment is unsuitable for many stroke patients due to the risks. Adequate collateral circulation is a vital criterion for successful endovascular treatment [9].
In the case of acute brain ischemia, collateral circulation has an imminent role in the compensatory mechanisms for treatment decision-making as shown in Figure 1. It is an alternate arterial route that comes into play when the blood in the primary vessel is clogged [10]- [13]. Some methods that are applied to visually measure collateral circulation include American Society of Interventional and Therapeutic Neuroradiology/Society of Interventional Radiology (ASITN/SIR) collateral score, miteff system, mass system, modified tan scale, and Alberta stroke program early computerized tomography score (ASPECTS) [14]. Table 1 highlights our proposed collateral circulation grading system as well as the existing study grading systems. The performance of a radiologist heavily relies on their experience, training, and specialization with these approaches, which can contribute to inter-and intra-rater inconsistency, as reported in several studies [9], [15], [16].  CT angiography 1: Good (100% collateral supply of the occluded MCA territory); 2: Intermediate (collateral supply filling >50% but <100% of the occluded MCA territory) or 3: Poor (collateral supply filling ≤50% but >0% of the occluded MCA territory) Su et al. [23] CT angiography 0: absent collaterals (0% filling in occluded territory) 1: poor collaterals (>0% and 50% filling in occluded territory) 2: moderate collaterals (>50% and <100% in occluded territory) 3: good collaterals (100% filling in occluded territory) Proposed method cone beam computed tomography (CBCT) 1: good collaterals (collateral supply >50% and <100%) 2: poor collaterals (collateral supply >0% and 50%)

5845
Some medical imaging techniques, such as X-ray, cone beam computed tomography (CBCT) as shown in Figure 2, and magnetic resonance imaging (MRI), provide precise details regarding the flow of blood to various parts of the brain. CBCT is one of the most popular methods applied to assess many diseases, particularly the collateral circulation in the brain [24]. After the CBCT scan, the neuroradiologist would review the case [25]. In a normal situation, not all cases are reviewed immediately after the scan because they need to handle other modalities in the radiology department. In some cases, subtle ischemic may be missed by them due to the highly variable appearance of intracranial hemorrhage depending on two factors; age and location [26]. Therefore, CBCT is imminent to administer thrombolytic therapy safely in acute ischemic stroke. Brain operations involve high mobility and a slight error would put the patient in a life-threatening situation [6].

Figure 2. The CBCT image
Machine learning is particularly deep learning. It is a kind of machine learning that utilizes multi-layered neural networks and has risen in popularity these recent years [9], [27]. It has a huge potential in extracting important information from medical images [28]- [30]. Deep learning has been applied to reliably examine the identification of metastases in lymph node histologic sections, categorize the images of skin cancer, and detect tumors in some medical domains [31]. It has been proven accurate in image classification and processing tasks, mainly using a convolutional neural network (CNN). A similar method can be implemented for head CBCT scan images. This involves image segmentation, and the processed images will be trained by using the deep learning approach to detect hemorrhage in the scan [32]. The result can be classified into cases with hemorrhage and without hemorrhage at high accuracy [28], [31].
Datasets are transformed via deep learning by mapping them with high-dimension space. For image classification, the CNN in deep learning offers huge support with advanced techniques [33], [34]. The CNN extracts semantic features and network-fused features of the dataset to classify the images. Due to the nature of the network, high-resolution image classification is well supported by CNN. This is important as most medical data must be in high-quality resolution to prevent missed diagnoses of diseases that could harm the patient as well as delay disease management [33], [34].
The contribution of this paper is to classify collateral circulation of brain CBCT images. This paper proposed a method for CBCT image classification using a ResNet50 deep learning method. The proposed method for classifying based on its features provides a sustainable performance and could help neuroradiologists to speed up the treatment decision.

METHOD
In this research, the research flow for the proposed method which is ResNet50 can be described using the flowchart in Figure 3. The training and testing data were divided by 80:20 (80% training and 20% testing data). The accuracy is achieved after completing seven epochs. The testing value is crucial to keep the model built from overfitting/underfitting properties. The wider the variance between training accuracy and testing accuracy, the model reflection of the real situation is lower.

Convolutional neural network
Convolutional neural network (CNN) is commonly deployed in the field of computer vision due to its efficacy in interacting with image analysis. Its ability to overcome classification issues, and its capability to enhance the efficiency of many machine-learning processes [35]- [37] has become an advantage. As a result, CNN has become a robust and adaptable deep learning model [38]- [40]. The deep learning approach of CNN has been used for a wide spectrum of applications, such as the classification and analysis of images, medical analytics, object detection, and pattern recognition [41]. With the widespread use of digital images as raw data in hospitals, medical image databases are rapidly expanding [42]. Digital images have a crucial role in identifying the intensity of a patient's disease, while medical images have numerous applications in both diagnosis and research areas. The advancement in imaging technology has enabled the automatic identification of raw images in the medical field and this subject has become an open research topic among computer vision scholars [42].
A robust deep-learning framework is essential to identify raw images in the medical field based on their relevant classifications. Classification of images is effective for predicting the appropriate category of unknown images. The main drawbacks of low-level characteristics are relatively limited classification capability and context-specific classification. A wide gap exists between low-level qualities (data analysis and reporting) and high-level perceptual features (human understanding). Therefore, this study proposes a unique image representation approach, where by the algorithm is employed to classify raw images in the medical field via a deep learning algorithm. A pre-trained deep CNN technique with a fine-tuned technique was implemented in the last three layers of the deep neural cable network [43].
CNN applies two forms of learning: unsupervised and supervised learning [44], which are common trends in image analysis-based challenges. In techniques of supervised learning, algorithm and data are in the form of a model to calculate output unknown value and input known values [45]. For techniques of unsupervised learning, unlabeled data are used for classifications of testing and training. The CNN utilizes the transfer function of tangent hyperbolic. The MLP is an example of a neural network with numerous layers [46], [47].
The CNN is a biologically inspired good spatial cognitive method that involves an input layer, full connection layers, pooling layers, convolutional layers, and an output layer as shown in Figures 4 and 5. The CNN features are composed of two elements. The first element refers to the connection between the neurons in the convolutional layer that are not entirely linked. The connected weight between specific data and information is shared in the same layer, which denotes the second element. The network complexity is reduced by the minimal connection and the weight-sharing architecture [38].

Architecture of ResNet50
The ResNet50 is widely used and has demonstrated promising performance [48], [49]. Segregating the acquired data into training and testing sets is a vital step in the classification process. In this architecture, the training set includes one target value and multiple attributes for each data. The ResNet50 determines how to handle the problem in a simpler manner, thus resulting in accurate results and a simpler training process. It has 50 layers for data processing. To avoid gradient dispersion/explosion and network degradation issues caused by excessively deep networks, ResNet50 has the advantage of using a jumping layer connection to a deep neural network [50].
The concept of ResNet50 is illustrated in Figure 6. The layers are introduced first with the input image. The first layer has a combination of convolution and rectified linear units (ReLU). The max-pooling layer is next. The pooling procedure is executed by selecting as many elements as possible from the region of filter-covered layer depths. Each layer output in the residual block is passed on to the next layer and hops take place across the identity connections [51].

Data sets description
Thirty patients diagnosed with ischemic stroke were assessed. In total, 4368 CBCT images were acquired from the patients using an angiography system machine. Neurointerventionist (Philips Allura; Biplane FD20/15) who performs CBCT scans is required to ensure that the output image is reviewed and reported. The VasoCT acquisitions were performed with a motorized rotational C-arm movement and this resulted in an isotropic stack of VasoCT images, which can be visualized in any random position without image quality loss. All patients had their medical records verified by neuroradiologists. Images were encoded in digital imaging and communications in medicine (DICOM) format. This study focused on the classification task by using the ResNet50 via Python as the computational tool. Hence, clinical representation, patient history, historical findings, and solutions for lesions were excluded.

Implementation details
Based on the collected data, automatic classification is implemented using ResNet50. The research mainly focuses on the process of classification by using ResNet50 using Python as the computational tool. The deep learning framework is PyTorch. The Jupyter Notebook compiler that belongs to the anaconda package was used. In addition to some other basic python libraries such as NumPy, Pillow, Augmantor, and OpenCV. The sample image that has been used for the ResNet50 is shown in Figure 2.
Collateral circulation classification was performed on the dataset with the ResNet50. The training and testing data were divided by 80:20 (80% training and 20% testing data). Based on the seven epochs, Table 2 shows that the ResNet50 classification attained 76.79% accuracy. The testing value is crucial to keep the model built from overfitting properties. The wider the variance between training accuracy and testing accuracy, the model reflection of the real situation is lower.
Data sets of 3,411 images were trained with the ResNet50, while the remaining 957 images were used to test the classification performance of the model. The ResNet50 for residual learning displayed the advantages of simple optimization and minimal computing load. The residuals addressed both degradation and gradient issues so that the performance could be enhanced when network depth increased. A fully linked layer and 49 convoluted layers make up the ResNet50. The residual blocks continuously muddled the image data. The image pixel matrix had an increasing number of channels. The size of the image pixel matrix was modified to 224 after passing through the flat layer. The probability of the suitable category was generated through the SoftMax layer after it was fed into the complete connection layer. The following is the step-by-step implementation of ResNet50 in this project.
Step 1. Each DICOM file has been converted to PNG Step 2. Data preparation Step 3. Label and augment the datasets (normal and abnormal) Step 4. Make a prediction using the ResNet50 mode Step 5. Identify class Step 6. Result displayed Step 7. Performance efficiency by evaluating the performance of the system using (1)-(3). Table 2 shows that the testing accuracy. The testing data denote a collection of new data points, which are new to the method. Figure 7 illustrates the performance graph for testing accuracy.

5849
Based on the calculation using (2) and (3), it is calculated that sensitivity is 0.7933 and specificity is 0.9602. The performance of the collateral circulation classifier using the ResNet50 was expressed in terms of true positive (TP), true negative (TN), false positive rate (FP), and false negative (FN). Clinical samples that the constructed classifier accurately identified as benign are referred to as true positives (TP). Clinical samples where the suggested classifier has accurately identified the malignant clinical data are referred to as TN. False-negative and false-positive cases occur when the suggested design incorrectly assigns the data to the benign class or the malignant class, respectively. It shows the classification error made. A good classifier can diagnose all the samples properly. Unfortunately, due to the uncertainty of the classifier, a method cannot be employed in clinics if it properly predicts real negative samples but is unable to locate the true positive ones. Thus, it is very important to achieve high accuracy, sensitivity, and specificity values. The graph of testing accuracy is shown in Figure 7.
Saliency maps have been created as indicated in Figure 8 since deep learning has been demonstrated to be dominant in learning high-level semantic representation. To study better examples from a deep perspective, several higher-level features extracted by the CNN with additional adaptive layers were used. From a broad perspective, some visually equivalent neighbors were offered to successfully reduce the common background regions.

Performance evaluation
Some measures (accuracy, sensitivity, and specificity) were included to assess the performance of the ResNet50. These are the numerical measurements of the performance, where accuracy is defined as the proportion of accurately detected samples to the total number of samples. Specificity and sensitivity are measurements of correctly identifying two classes, namely negative and positive.

CONCLUSION
The ResNet50 of CNN was used in this study to propose a fully automatic method that classifies the various stages of collateral circulation from CBCT images. The 4,368 CBCT images were increased via the data augmentation method. Normal and abnormal collateral circulation stages were distinguished. The accuracy result of 76.79% was attained as well as a sensitivity of 79.33% and specificity of 96.02%. This method rapidly identified collateral circulation classes and its treatment process was more thorough. Some enhancements are still feasible despite the success recorded in this study. These problems will be addressed in a domain-specific study in near future, perhaps with a larger sample size and improved transfer learning model design.