A novel algorithm for detection of tuberculosis bacilli in sputum smear fluorescence images

Erwin Dianderas, Christian del Carpio , Mirko Zimic, Patricia Sheen, Jorge Coronel, Roberto Lavarello, Guillermo Kemper Faculty of Electrical and Electronic Engineering, Universidad Nacional de Ingeniería, Peru Laboratory of Bioinformatics and Molecular Biology-Faculty of Sciences-Universidad Peruana Cayetano Heredia, Peru Tuberculosis Laboratory-Universidad Peruana Cayetano Heredia, Peru Pontificia Universidad Católica del Perú, Peru


INTRODUCTION
Tuberculosis (TB) is an infectious disease caused by mycobacterium tuberculosis (MTB). According to the World Health Organisation (WHO) in 2017 there were an estimated of 10 million new cases of TB and over 1.3 million deaths, making TB the leading cause of death world-wide from a single infectious agent. More than 95% of TB deaths occur in the developing world, however despite this disproportionate disease burden, rate of diagnosis in these countries remains inadequate [1,2].
Currently, the direct sputum smear microscopy is the first-choice screening method and has become the indisputable pillar of active TB detection in the developing world [3]. The Ziehl-Neelsen (ZN) staining method is a simple, rapid, and low-cost assay, that is, the first-screening tool worldwide [4]. Despite its advantages, ZN is not very sensitive (30-70%) in comparison with culture, especially in sputum samples with low-bacterial load commonly found in pediatric and HIV-positive populations.
Comparably, the auramine-fluorescence (AF) smear microscopy has a higher sensitivity (40-80%) and consumes less time, since it allows for ×400 magnification microscopy inspection, which reduces to 2-3  ISSN: 2088-8708 Int J Elec & Comp Eng, Vol. 10, No. 6, December 2020 : 5665 -5677 5666 min the reading time, compared to the 10-15 min in ZN inspected at ×1000 magnification. The recent development of inexpensive, robust, and prolonged lifespan fluorescence microscopes based on the light-emitting diodes motivated the WHO to recommend the auramine smear microscopy as an alternative to ZN [5,6]. The high workload in TB endemic countries leads to technical-fatigue, which reduces the accuracy of microscopy inspection [2].
According to the WHO, a technician should not analyze more than 20 samples per day [7] due to worker fatigue causing a decrease in reliability of interpretation [8]. In developing countries there is often an insufficient number of these skilled laboratory staff to cope with the high number of sputum samples requiring analysis. Furthermore, in these settings, access to modern diagnostic methods using gene analysis is often restricted to tertiary centres and the necessary equipment is expensive to purchase and maintain. Also has been claimedthat the current screening methods may miss 33-50 % of active cases and automation may help to increase the accuracy, as machines can screen more fields and can detect cases in the initial stage of the disease itself [9]. Thus, the development of computational digital image processing algorithms that allow automated visual analysis of samples and generate a reliable report of bacilli concentration, would be desirable.
For the automatic detection of TB, different algorithms of image processing have already been developed. For example, in [10], Forero et al, took a generative approach, representing the TB-bacilli class with a Gaussian mixture model (GMM) and using Bayesian classification techniques. whilst in [11], Chang et al., proposed a method that consisted of using geometrical and photometric descriptors, Hu moments and the oriented gradient histogram to be able to characterize the bacilli. Once the parameters described were calculated, each object was classified automatically using support vector machines (SVM). In this case, the method worked with a specific microscope that was developed by the Berkley University in the United States. Finally in [12], Zheng et al, proposed a method similar to the aforementioned articles [10,11] with the difference being that for the object level classification, they used random forest. Although these studies present encouraging results, they only analyze digitized images using the direct sample preparation method, which compared to Diluted Pellet has less sensitivity and specificity. Therefore, it is still necessary to develop a computational algorithm capable of processing sputum samples for the detection of bacilli in detection methods that have better sensitivity and specificity.
In order to identify the bacilli within the images obtained by fluoroscopy, an algorithm was developed that first eliminates background noise and then obtains the candidate objects. Secondly, descriptors were implemented (Hu moments, geometric and photometric descriptors), which serve as the input to train support vector machines (SVM) helping to discern whether the object analyzed is a bacillus or not.

RESEARCH METHOD
The block diagram of the proposed algorithm is showed in Figure 1. The details of each processing stage are described in the following sections.

Sample preparation
Two types of sputum smear procedures for fluoroscopy were used:  Standard direct smear microscopy, which consists of staining the sputum samples on a glass slide see Figure 2 (a) without any further treatment see Figure 2 (b) [13,14].  Diluted sputum smear microscopy or diluted pellet: 2 mL of sputum was collected in a 15 mL Falcon tube and 2 mL of decontaminating solution using the standard method of NaOH-Sodium Citrate-N-acetyl-Lcysteine (NALC) was added. The sample was mixed with a vortex for 20 seconds and then was left to rest for 15 min. After that, 10 mL of phosphate buffer pH 6.8 was added to neutralize the alkaline conditions and centrifuged at 3000 × g for 15 min. Supernatants were discarded, and the sediment resuspended with  Figure 2 (c) [13].

Acquisition of images
With fluorescent microscopy, the only requirement for the microscope is that it has fluorescent light. For this, a Zeiss Primo Start model was used with a total magnification of 400X. A 5 Megapixels (Mpx) digital camera (MiniVid) was placed in one of the eyepieces. The images were acquired in RGB color model (radiometric resolution of 8 bits per band) with a spatial resolution of 2592 x 1944 pixels and stored in bitmap format (.bmp).
Every sample used in the three methods of preparation was digitized using the same configuration. The acquired image was conformed by its 3 primary components RGB defined as , ( , ), , ( , ) and , ( , ), where and are the spatial coordinates of the image, and indicates the preparation method ( = 1 for the direct method and = 2 for diluted pellet). In the images, can take a value from 0 to 1943 and can take a value from 0 to 2591. A camera of 5 Mpx was chosen because it allowed the bacilli to have a size of no less than 150 pixels, which allowed their correct appreciation for the human eye, which was necessary to carry out a correct validation.

Background elimination
Regardless of which sample preparation method was used, the background of the images was not black as shown in Figures 3 (a), and 4 (a). So, in order to make the background black, the histograms of the images were analyzed and the result was that the green color was predominant when there are bacilli, but when there are no bacilli, the red band and the blue band were predominant. To eliminate the background when the sample preparation method is direct and diluted pellet, the algorithm is the same. The procedure first involves the elimination of the blue band, making it equal to zero. Then, the pixels in the red band whose values were less than 30 were also eliminated. Finally, to finish eliminating the background, the pixels  ISSN: 2088-8708 Int J Elec & Comp Eng, Vol. 10, No. 6, December 2020 : 5665 -5677 5668 whose summed value (red band + green band) is close to zero (less than 30) are eliminated. This is because when the histogram of bacilli was analyzed, the value added in the green band plus the red band was greater than 100. These three steps were implemented in (1), (2) and (3).
The consecutive closing process can be expressed as follows: where the operators ⨁ and ⊖ represent the dilation and erosion processes respectively [15]. Based on the results obtained, the new primary components were generated for the direct/diluted pellet cases: 4 , ( , ) = 0 (12) Figure 5 shows the results for the direct and diluted pellet method. It can be seen that in the final image, the object is segmented properly unifying the entire region of interest that will enter the classification process.

Object segmentation
Before an object could be described, it was necessary to segment each of them in order to extract their geometric and photometric characteristics. For the object segmentation, the image was thresholded to be able to label each of the objects of interest. The thresholding applied for the direct and diluted pellet method can be expressed as: For segmentation, the labeling algorithm with connectivity 8 [16] was applied to binary image ( , ). From the results, each labeled object was extracted in a sub-image and then its geometric

Descriptors calculation
For the adequate detection of the bacillus that were in the image, we considered the geometric features, photometric features and Hu moments. With these descriptors, the best results of sensitivity and specificity were obtained.

Geometric descriptors
Seven geometric descriptors were calculated for each object obtained from ( , ) [17,18]. a. Area The number of pixels that conform an object (bacillus or not). In this case, the area of the n-th object extracted from the ( , ) image, = 1 for the direct method and = 2 for diluted pellet, was defined as: where , ( , ) is the sub-image containing the n-th object of ( , ); , and , constitute the number of rows and the number of columns of the sub-image respectively. b. Equivalent diameter A straight line joining two points of a circumference. In this case, the diameter of a circle was calculated with the same area as the region. The equivalent diameters for the nth object obtained from ( , ) were calculated by applying the following expression: c. Eccentricity For each object extracted from ( , ), the ellipse that more accurately houses the object was determined. Then, the size of the semimajor axis was calculated (in number of pixels) and the distance between the center of the ellipse and one of its focuses (in number of pixels). With this the eccentricity of the object is obtained in the following way: where , is the distance from the center of the ellipse to one of its foci for the n-th object of ( , ) and , is the size of the corresponding semimajor axis.
e. Ratio Relationship between the width and length of the bacilli. In this case, the quotient between the semi-major axis and the semi-minor axis of the ellipse obtained in the calculation of the eccentricity was calculated. Therefore, for objects obtained from ( , ) this gave: where , is the size of the semi-minor axis of the corresponding ellipse (in number of pixels). f. Perimeter Sum of the lengths of the sides of the object. In this case we defined the perimeter (in number of pixels) of the n-th object of ( , ) as , . g. Solidity The quotient between the area of the object in number of pixels and the convex area also expressed in number of pixels. To obtain the convex area of an object, the area of the polygon that exactly housed the object inside it was determined. The number of pixels that made up this polygon was defined as the convex area of the object. Therefore, for an object of ( , ) the solidity was obtained in the following way: where , is the convex area of the n-th object.

Photometric descriptors
Six photometric descriptors were calculated for each object obtained from ( , ).  Greater tonality in the red band ( , ).  Lower tonality in the red band ( , ).  Average tone of the bacillus in the red band ( , ).  Greater tonality in the green band ( , ).  Lower tonality in the green band ( , ).  Average tonality of the bacillus in the green band ( , ).

Hu moments
The Hu moments are moments that meet the requirement of being invariant to rotation, translation or scaling, so would describe the bacilli regardless of whether they are inclined or are larger than others. In total, the first 7 moments and the moment 11 were considered. The 8 Hu moments ( 1 , , 2 , , 3 , , 4 , , 5 , , 6 , , 7 , and 11 , ) were calculated for the n-th object extracted from ( , ) and the equations to calculate them are found in [19].

Object classification
For the object-level classification, an SVM algorithm (Equation (30)) was implemented. It was employed SVM because it can find the hyperplane that maximizes the margin between the TB-positive and TB-negative classes in the feature space [20][21][22].  Dimension: The dimension of the SVM used is + 1, where is the number of descriptors that the input of the SVM has for each preparation method p. Table 1 shows the descriptors selected for each method and the corresponding value of . For each case, the descriptors which obtained the best results for sensitivity and specificity were selected.  Table 1.
Each descriptor, photometric or geometric, was chosen based on its contribution to the classification process and maximization of sensitivity and specificity. The descriptors that were used as input were first normalized using maximum-minimum standardization.  Input vectors: The input vectors to the SVM corresponding to each object obtained by each preparation method can be defined with (20) and (21)  Number of Vectors: Table 2 shows the number of vectors used to train the SVM for each preparation method. It also shows the number of test vectors for validation purposes and the number of support vectors that were taken into account to determine the hyperplane solution for each SVM. For this last case we defined the support vectors in the following way: i,1 : Support vector for the direct method, where = 0,1, … ,356. The vector dimensions for this case is 14x1.
i,2 : Support vector for the diluted pellet method, where = 0,1, … ,369. The vector dimensions for this case is 12x1.  Kernel: An SVM with kernel was applied, because the data was not linearly separable. For each preparation method, the kernels with the best classification results were also selected. In that sense, for the direct and pellet methods a polynomial of grade 6 was selected as the kernel and for the pellet method a radial basis function (RBF) was selected (22): The value of γ that obtained the best results for specificity and sensitivity was = 1.
Finally, the expressions that defined the SVM for the direct, pellet and diluted pellet methods were the following: In (23), , is the Lagrange multiplier and has the same dimensions as its corresponding support vectors.

RESULTS
The sputum samples used for validation were obtained from different medical centers, the patients were randomly selected regardless of age or gender, since these parameters are indifferent when analyzing the sample. Each sputum sample was separated into 2 and each sub-sample was delivered to two specialists in order for them to perform the staining process independently (direct and diluted pellet). The sample preparation process is standardized in Peru and the steps indicated in [13] must be followed.
For the validation of the direct method a total of 72 images were used corresponding to 18 patients, for the validation of the pellet method a total of 56 images were used corresponding to 14 patients and for the validation of the diluted pellet method a total of 76 images were used corresponding to 19 patients. For each method of sample preparation there were more than 300 objects (bacilli and not bacilli) for validation. The data configuration is shown in Table 2.
Where means "True Positive" and are the objects that the algorithm and the specialist mark as bacilli; means "True negative" and are the objects that both algorithm and the specialist mark as NOT bacilli; F means "False Positive" and are the objects that the algorithm detects as bacilli but are NOT bacilli for the specialist; means "False Negative" and are the objects that the algorithm marks as not bacilli, but the specialist marks as bacilli.
Two technicians from the Tuberculosis Laboratory of the University Peruana Cayetano Heredia (UPCH) participated blindly in the validation of the proposed algorithm, both with extensive experience in the preparation, analysis and diagnosis of TB. The specialists were analyzed bythe kappa index to validate the agreement between them. The result obtained was an approximate value of 0.9, which indicates that they agreed to a great extent when marking the objects. but this means it was notpossible to obtain a sensitivity and specificity of 100%, if these serious values were obtained it will be by over fitting. That is why the sensitivity will be taken when specialist Nº 1 (E1) or Nº 2(E2) marked the object as a bacillus and specificity when the two specialists indicate that it is a bacillus. Therefore, in order to calculate these parameters, "two new specialists" were created E1 and E2. -Specialist Nº 3(E3): If E1 or E2 (E1 | E2) mark the object as a bacillus, it is considered a bacillus. -Specialist Nº 4 (E4): If E1 and E2 (E1 & E2) marked the object as a bacillus, it is considered a bacillus.
The specialists independently evaluated all the images that were submitted, without knowing the results of the algorithm. Sensitivity and specificity were then calculated by each specialist and the average of both values was obtained. The Figure 8 shows the best sensitivity and specificity results obtained from Tables 3 and 4. The diluted pellet method yielded the best results, with the highest sensitivity and specificity see Figure 8. This is not surprising, as this method generates cleaner images compared to the direct method. The results obtained for the direct and pellet methods were acceptable since they allowed differentiation of the bacilli within the image, with a sensitivity of 91.24% and 93.79% respectively. Tables 3-5 E1  E2  E3  E4  Bacilli  151  160  174  137  Artefacts  209  200  186  223  TP  135  135  145  125  TN  181  172  168  185  FP  28  28  18  38  FN  16  25  29  12 Figure 8. Sensitivity and specificity analysis by object

CONCLUSION
A greater sensitivity was obtained using the diluted pellet method, compared to the direct method. However, the diluted pellet method also achieved the highest specificity, this is because the image generated was cleaner as the artifacts were already removed. Hence, this would be the recommended method of preparation when using automated analysis in the future.
There is a greater error in the measurement when the brightness of the bacillus is very dim or they are out of focus in the image. Conversely, if the image overall is very bright, it is not possible to eliminate the noise, generating error in the measurement. It has been demonstrated that it is not necessary to use all the descriptors developed and that some descriptors worked better according to the type of sample preparation.
The algorithms proposed in this paper will work whenever the preparation methods are the ones indicated by Coronel  analysis of sputum samples may provide a useful adjunct in resource-poor settings where rate of diagnosis remains sub-standard and disease burden remains high. According Veropoulos et al., Sputum microscopy remains the most widely used method for diagnosis in many of these countries despite insufficient skilled personnel to analyze samples from the high number of suspected cases. Thus, the development of automated sputum analysis may have the power to ease pressure on over-worked staff, reduce time to diagnosis and increase the overall rate of diagnosis.