An algorithm for detection of Tuberculosis bacilli in Ziehl-Neelsen sputum smear images

ABSTRACT


INTRODUCTION
The automation of the optical analysis of sputum samples inked with the Ziehl-Neelsen method [1] is still quite required in different medical centers in different parts of the world that use microscopy to evaluate the samples and determine the presence of tuberculosis bacilli and their respective concentration [2,3]. The automation process seeks to reduce the time of obtaining results and improve the precision with respect to that obtained by visual inspection by a medical specialist [4].
The evaluation of the detection algorithms is done through the parameters of sensitivity and specificity. These parameters allow to compare the results of the algorithm with those obtained by coincidence of more than one specialist or observer. Sensitivity measures the algorithm's ability to correctly detect objects that are actually bacilli, while specificity measures the ability to correctly detect objects that are not bacilli. Both parameters are expressed as a percentage, and the closer to 100% they are, the greater the performance of the algorithm will be. Several computational algorithms have been proposed in the literature in order to be able to execute the analysis of the samples and provide results in the same scheme that a specialist would do. Costa et al. [5] were the first to propose an automatic TB bacilli identification method for images acquired with bright field microscopy based on image processing techniques. In their work, they used images that resulted from subtracting the R component minus the G component (R-G) from the RGB color format. Their analysis shows that bacilli are more visible in images (R-G) than in RGB images or HSI images. To segment the bacilli, the authors used the histogram from the R-G image and a threshold value was calculated. Morphological and size filters were subsequently applied to remove artifacts from the segmented images. The results presented in the paper were promising; however, his method showed a low sensitivity (76.65%).
A Bayesian segmentation method was used in Sadaphal et al. [6] to segment the bacilli. Once the object of interest was segmented, the objects were classified as TB bacilli or non-TB bacilli according to their shape characteristics. The authors used shape descriptors such as axis ratio and eccentricity. However, they do not indicate results obtained from the application of their proposal.
Osman et al [7] used a single-layer feed-forward neural network trained by the Extreme Learning Machine (ELM) technique. The neural network genetic algorithm (GA-NN) approach was used in [8] and the single-layer compact occult feed-forward neural network (C-SLFN) trained by an improved ELM method was applied in [9]. Although the above methods produced acceptable results, the efficacy of the methods was not demonstrated with images with different backgrounds. The nature of the background is an important factor affecting the performance of most autofocus and TB detection methods. Thus, the best result was obtained in [8] where accuracy of test classification was achieved ranging from 84.64% to 88.21% with an average of 86.32%.
In 2010, Zhai et al. [10] proposed an automatic image capture system and the application of methods for the detection of TB bacilli. An automatic approach was used for the capture system, using a moving plate that can be moved in the x, y and z coordinates with the help of a stepper motor. The authors performed twostage segmentation such as coarse and fine segmentation by converting RGB images into HSV and CIE Lab color spaces. Its detection process for each sputum sample takes from 3 to 5 minutes. The results obtained for 95% of the images processed are more than 80% accurate. The images with an accuracy between 81% and 90% represent 72% of the 100 processed images.
CostaFilho et al. developed a neural network approach in [11]. This method initially classifies the image into a low-density or high-density background image using the image hue component of the HSI color model and a set of color characteristics for segmentation and classification. For high-density background images, the color ratio was used as an additional parameter for efficient sorting. The overall sensitivity obtained was 91.53%, while the false detection was 8.51%.
Again Costa et al. [12] provides an image database to test for the detection of tuberculosis bacilli in 2014. The database consists of two parts: an autofocus database with 1200 images and a segmentation and classification database with 120 images. The images with different backgrounds in the database will help conduct extensive experiments to see how existing and new methods work under various conditions. These marked objects could be used as gold standard to calculate the accuracy, sensitivity and specificity of bacillus recognition. In this case no sensitivity and specificity results of bacillus detection are presented.
Ayas et al. [13] proposed a segmentation and classification method based on the randon forest (RF) learning method. The experimental analysis proposed by the method performs better than other conventional classification methods, such as artificial neural networks (ANNs) and support vector machines (SVMs). Although many methods were presented for segmentation of TB bacilli from conventional microscopic images, a common drawback of most methods is their inability to classify superimposed bacilli as true bacilli. The sensitivity and specificity of the proposed classifier are greater than 75.77 and 96.97% for pixel segmentation, respectively. It is also mentioned that sensitivity is increased to more than 93% when staining is performed according to the procedure.
Panicker et al. [14] in 2018, present an automatic method for the detection of Tuberculosis (TB) bacilli by image binarization and subsequent classification of regions detected using a convolutional neural network. They evaluated the algorithm using a data set of 22 images with different backgrounds (high density and low density images). The experimental results show that the proposed algorithm achieves 78.4% accuracy and 86.76% F-score for TB detection. In the present work, although the image processing and classification techniques are well known, they have been adapted and improved aiming to obtain the best results for each preparation method in Ziel Neelsen bacilocopy: direct method, Pellet and diluted Pellet. Figure 1 shows the block diagram of the proposed algorithm. The image of the sputum sample prepared by a specific method (direct, pellet or diluted pellet) is digitized through a digital camera installed in an optical microscope. The format of the acquired image is RGB (Red-Green-Blue). The obtained image is

Sample preparation
Three types of ZN inked sputum baciloscopy are used in this paper. First, there is direct smear staining, which consists of staining sputum samples on a slide as shown in Figure 2(a) without treatment to purify and concentrate them refer Figure 2

Acquisition of images
All samples, under the three preparation methods, were obtained using a MiniVID microscope camera with CMOS sensor. The images were acquired in RGB color model (8 bits per band) with spatial resolution of 2592 x 1944 pixels and stored in bitmap format (.bmp). This RGB image is made up of component R expressed as ( , ), component G expressed as G( , ) and component B expressed as B( , ).

Image enhancement
The image enhancement process consists of two parts. The first part is to apply consecutively an enhancement filter and a contrast adjustment process. This will allow the edges of the bacilli to be more intense.
To apply the enhancement filter, you must first convert from the RGB color model to the CIE color model * * * [15]. The components of this image are the L component expressed as ( , ) and the components a * and b * defined as * ( , ) and * ( , ) respectively. To perform the conversion (1), (2), (3) and (4)   (1) where 0 , 0 , 0 tristimulus values of the reference white [16].
Highlight filtering on a color image is applied only on the L component; the a * and b * components are not modified. The ( , ) image is then filtered with a 9x9 MG Gaussian mask and standard deviation ( ) of 2, which is obtained by (5). The image resulting from the Gaussian filtering process is the image ( , ), which is obtained by (6).
Applying (7) you get ′( , ), which is the L component filtered with the enhancement filter.
Finally, we return to the RGB color model but with the L component modified, expressed as ′( , ), obtaining in RGB color model a new image. The result of the enhancement filtering process is shown in Figure  3(b). After applying the enhancement filter, a contrast enhancement is applied. Once the enhanced RGB image is obtained, its three primary color components are separated from the image. That is, component R expressed as ′( , ), component G expressed as ′( , ) and component B expressed as ′( , ). As can be seen in Figure 4, the most salient information regarding the bacilli can be found in R and G components of the enhanced RGB image. The transformation functions shown in (8), (9) and (10) are then applied to each component making up enhanced image to highlight the bacilli present in the image. This is shown in Figure 5.

Segmentation
Once the image has been enhanced, we proceed to the segmentation process. In order to extract the objects of interest (because they are concentrated in the components mentioned above), the improved components R and G are subtracted as expressed in (11). The result of this operation is shown in Figure 6.
In order to improve the display of all objects in the image, the dynamic range is reduced by applying (12) over the ( , ) image. The result is shown in Figure 7. The image 1 ( , ) is filtered with a median filter [17] using a 5x5 window to standardize the image obtaining 2 ( , ). The result can be seen in Figure 8. As can be seen, the image has been segmented obtaining the objects of interest and some unwanted artifacts. The image obtained is then subjected to a threshold process applying (13). The threshold is fixed and was obtained after performing multiple tests. Figure 9(a) shows an image with a color bar, where the objects of interest are taken with a threshold from 0.75. The result of the threshold is the ( , ) image shown in Figure 9(b). The tagging algorithm [18] is then applied to the ( , ) binary image in order to remove tags containing less than 40 pixels; this in order to remove objects that because of their size do not qualify to be bacilli. The result of this process is 1 ( , ). Likewise, in order to give solidity to the obtained objects, the morphological closing process [19] is applied with a structural element EE. This process is indicated in (14). where and constitute the matrices that represent the images 1 ( , ) and 2 ( , ) respectively. The symbols "⊕" and "⊝" represent the morphological operations of dilation and erosion respectively. The matrix obtained from this process is which represents the image 2 ( , ) and is shown in Figure 10. The labeling algorithm is applied to the image 2 ( , ), obtaining all the final objects to be able to determine which are possible bacilli. The image of the tagged objects is shown in Figure 11. Figure 10. Image 2 ( , ) Figure 11. Image of all tagged objects

Descriptors calculation
For the detection process of bacilli in images obtained by ZN bacilloscopy in the different methods, direct, pellet and dilute pellet, the detected objects were analyzed and the geometric and photometric descriptors were calculated.

Area
The area , of the nth object ( , ) belonging to the binary image 2 ( , ) is equal to the sum of the number of pixels corresponding to the object.
where and correspond to the number of rows and columns of the nth object respectively.

Perimeter
The perimeter , of the nth object ( , ) belonging to the binary image 2 ( , ) left, right, up or down (4-next-door).

Eccentricity
The eccentricity , of an ellipse of the nth object ( , ) belonging to the binary image 2 ( , ) is equal.
where and are the size of the major and minor axis of object ( , ) respectively

Solidity
The solidity , of the nth object ( , ) belonging to the binary image 2 ( , ) is obtained in the following way.
= (18) where is the convex area of the nth object.

Photometric Descriptors
Six photometric descriptors [21] where calculated for each objet obtained from 2 ( , ) a. Percentile 35 in the red band (PR35 n ). b. Percentile 65 in the green band (PG65 n ). c. Standard deviation in the red band (STDR ) d. Standard deviation in the green band (STDG ) e. Standardized energy in the red band (ER ). f. Standardized energy in the red band ( ). To obtain the a-th percentile ( , ) the histogram [22] of the n-th object ( , ) will be defined first.
where is the k-th intensity level of the object and is the number of pixels in the object that has the intensity level.
Then the index for the desired percentile is obtained, being and number of rows and columns respectively of the nth object ( , ) that belongs to the image of interest. The value of obtains the number of pixels of the desired percentile.
Once the number of pixels for this percentile is obtained, the values of the histogram are added until the number of calculated pixels is reached.
Therefore, in (22) the percentile a is obtained that is equal to that complies with (21).

Clasification of objects
For the classification of the objects, a Support Vector Machine (SVM) was implemented. The SVM finds the hyperplane that maximizes the margin between the TB-positive and TB-negative classes in the characteristics space [23][24][25]. The SVM dimension used is +1, where is the number of descriptors that the SVM input has for each preparation method. In this case, the dimension is the same for all preparation methods.
The descriptors used are shown in Table 1 and are the same for all preparation methods. With these chosen descriptors the best results were obtained, both in sensitivity and specificity. The SVM input vectors corresponding to each object obtained by each preparation method are defined by (23), (24) and (25) for the direct method, pellet and diluted pellet respectively. Table 1 shows the number of vectors used to train SVM for each preparation method, and the number of vectors for validation purposes. For the implementation of the SVM a kernel was applied, because the data were not linearly separable. For all preparation methods a grade 6 polynomial was used. In (26) and (27) the following operation is shown.  (26) where is the SVM support vector for each p separation method.
where is the Lagrange multiplier and has the same dimensions as its corresponding support vector and b is the bias. Figures 12-14 show how SVM classifies objects in images from the different preparation methods.

RESULTS
For the present work, 225 ZN paving images, 75 images by the direct method, 75 images by the pellet method and 75 images by the diluted pellet method were analyzed. All images were obtained from different patients. Sample preparation is a standardized process and this is indicated in [26]. In order to validate, we worked with more than 350 objects between bacilli and non-bacilli obtained from the different processed images. Table 2  Bacilli that are not detected by the software as bacilli, but have been recognized and accepted by the specialist. Two medical technologists from the Tuberculosis Laboratory of the Universidad Peruana Cayetano Heredia, both with extensive experience in the preparation, analysis and diagnosis of TB, participated in the validation of the proposed method. The specialists evaluated separately all the images that were submitted to the proposed algorithm (without knowing the previous results of the software). Tables 3-5 show the results obtained for sensitivity and specificity. Specialist No. 1 is denoted as E1, specialist No. 2 as E2, E1 or E2 as E3 and E1 and E2 as E4. Figure 15 shows the best results obtained for sensitivity and specificity from Tables 3-5.  It must be taken into account that the images obtained in any of the three preparation methods must be well focused so as not to cause error. Likewise, it is important that the samples are correctly prepared, that is to say, that there is no leakage of the fuchsin dye on the image, as well as the bacilli must be well marked with this dye to improve the sensitivity and specificity, and also avoid that a bacillus is split and can affect the counting of these.
For the description of the objects considered as bacilli, both shape and photometric descriptors were taken into account, and specifically that comply with fuchsin staining, because of this the 35th and 65th percentiles were used in the R and G components of the image respectively, given that component B was set to 1. The best result in sensitivity is obtained in the direct method, however, all three methods exceed 90% sensitivity. Likewise, the best specificity result is achieved in direct method, almost 90%, and the lowest index in pellet method. This is because there are objects that without being bacilli can be marked by fuchsin product of the sample preparation process. The most common method of preparation, in any health center, is the direct method.