Estimation of object location probability for object detection using brightness feature only

ABSTRACT


INTRODUCTION
Research for detecting objects in images is one of the most important fields in image processing. In general, the first thing for object detection is analyzing the features of the object. Therefore, if an object does not have a consistent size, position, shape, contour, color, etc., object detection becomes difficult. In particular, when the relative brightness is the only usable feature, an additional problem of threshold setting is occurred, and the detection result is different depending on the used threshold value [1,2].
Since most existing object detection methods use features such as color, shape, and contour, if there are no consistent features in a gray image or the object can be detected by only the difference in brightness with the surroundings, we need a new object detection method. Therefore, in this paper, we propose a new method to estimate the probability that an object can be located in an image using only brightness feature, and explain a method of generating an object location probability map that expresses the estimated probability as an image.
In order to estimate the object location probability, we enhance the contrast using the ends-in search stretching [3] and use the color quantization method using color importance-based self-organizing map [4]. The proposed method can estimate the location of an object have no consistent features in gray image even it cannot be detected by using existing object detection methods. In addition, since there is no parameter or threshold setting, constant result is generated and constant execution time is required. Therefore, it is suitable for real-time image processing.
Ultrasonograms are gray images and there is no color information. Because of the features of organs in ultrasonogram are very different depending on the proficiency of the operator [5], the organs appearing in ultrasonograms are should be detected only by the difference in brightness [6]. Particularly, the gallbladder  [7][8][9][10]. Therefore, it is a good field to apply the proposed object location probability map, so the proposed method will be explained using ultrasonograms which is containing gallbladder. A related studies can be found in Reference [11][12][13][14]. Figure 1 shows the process of estimating object location probability and generating a probability map. We estimate the area where the object can be located, and generate 100 multi-location information. The object location probability is estimated based on the multi-location information, and the probability map is generated. Figure 1. The process of estimating object location probability and generating a probability map

Preprocessing
The preprocessing uses bilateral filter [15,16] and median filter [17,18] to remove noise. Figure 2 shows the result of applying bilateral filter and median filter. The bilateral filter is non-linear and is well known as a noise-reducing smoothing filter that removes noise while preserving edge of an image. The median filter has the effect of removing the impulse noise while also preserving the edge. Therefore, by using the above two filters, it is possible to preserve the edge and remove noises.

Object location probability representation using ends-in search stretching
In this paper, we assume the target object's brightness is lower than the surrounding area. If the brightness of the object is higher than the surrounding area, set it to the opposite of the description below. Figure 3 shows how to represent the object location probability by brightness. The object location probability is defined as the probability that the object can be located. Since the object is assumed to have a low brightness, if the brightness T low included in the brightness range of the object, the range below T low becomes a range in which the object can be located and defines it as a minimum locatable range R min . The object can be located in R min with a high probability. Contrastively, if we know the minimum brightness Thigh that the object cannot be located, then a brightness greater than T high is a range in which the object never can be located. Therefore, the range excluding greater than Thigh is defined as the maximum locatable range R max . Since R max has higher brightness than R min , R max has a relatively low object location probability.  Ends-in search stretching [3] is one of the contrast stretching methods in which a certain amount of pixels is set to white or black and the remaining pixels are stretched to a value between 0 and 255. The (1) explains the ends-in search stretching.
where i(x, y) is the brightness of the x, y coordinate pixel of the image, low is a threshold for setting pixels with a lower brightness than low to black (0), and high is a threshold for setting pixels with a higher brightness than high to white (255). If low = 50 and high = 150, pixels with a brightness lower than 50 in the image are set to black, and pixels with a brightness higher than 150 are set to white. And pixels with the brightness between 51 and 149 is stretched to between 0 and 255.
If the parameter of ends-in search stretching low is set to T low and high is set to T high , we can obtain the result that the brightness below T low is black, the brightness above T high is white, and the brightness between T low and T high is 0 to 255. Since we assumed that the object's brightness is low, the object location probability increases as the brightness decreases, and decreases as the brightness increases. Therefore, by enhancing the contrast by using ends-in search stretching, the object location probability can be appropriately expressed by the brightness.

Determining object locatable range using color quantization
Since the contrast enhanced image still has a brightness between 0 and 255, there is a problem that the brightness range in which the object can be located finally is determined. The proposed method determines the brightness range by using color quantization method. Color quantization [19][20][21][22][23][24][25] is a method to represent the colors of original image with limited number of colors. It clusters or learns the original colors and converts the colors within a certain range into optimized representative color. That is, by applying the quantization, the brightness of original image is divided into specific ranges, and the pixels corresponding to each range are converted into representative colors.
In the proposed method, we quantize the contrast enhanced image with 8 colors (C 1 ~ C 8 ) by using color quantization method using color importance-based self-organizing maps [4]. Figure 4 shows the progress of determining the object locatable ranges. Since we assumed the object is dark, it is also expressed in dark color in the quantization result. Therefore, the brightness range of the two darkest representative colors (C 1 , C 2 ) is a high probability that the object can be located, and this is defined as the minimum object locatable range LR min . On the contrary, the brightness range except for the brightest color (C 8 ) is a relatively low probability that the object can be located, and is defined as a maximum object locatable range LR max .  Figure 4. Determining the brightness range of an object locatable region using quantization

Estimating multiple-location information
The location information means an image in which the object locatable regions are marked, and it is estimated by applying AND operation to the minimum locatable image LI min and the maximum locatable image LI max . LI min is an image obtained by converting pixels included in LR min into black (0) and the rest pixels are converted into white (255), and LI max is an image converted as above using LR max .
The proposed method generates LI min and LI max repeatedly (20 times) while changing the low and high values of the ends-in search stretching parameters. Since it is impossible to know the exact values of T low and T high for estimating the object's brightness range, it is intended to estimate the object location probability in various brightness conditions by assuming T low and T high as various values. Figure 5 shows an example of LI min and LI max . The parameter values used to generate LI min and LI max are shown in Figure 5. AND operation extracts only the regions indicated by black in LI min and LI max . In the two images, black means that the object locatable region, so it is the same as extracting the regions where indicated as object locatable. Because LI min and LI max are generated using the assumed T low and T high values, so it may be inaccurate, AND operation extracts the region where the object location probability is higher by extracting the common regions. In addition, the combination of LI min and LI max is intended to estimate more location information in more various cases and to estimate more accurate object location probability thereafter.
In the estimated location information, the regions too big or too small are removed using size filter because the object has a certain size. Figure 6 shows the process of estimating a location information of the gallbladder in ultrasonogram.
(a) (b) (c) (d) Figure 6. The process of estimating a location information, (a) LI min , (b) LI max , (c) AND operation, (d) Location information (size filter is applied) The proposed method estimates a total of 100 pieces of position information by applying an AND operation to each of 10 LI min and LI max alternately, and defines it as multi-location information. Figure 7 shows 25 of the total 100 multi-location information images for detecting gallbladder in ultrasonogram. The actual gallbladder region is included in 21 of 25 images. However, the non-gallbladder regions can be seen only in some images.

Object location probability estimation and probability map generation
By averaging the estimated multiple-location information, it is possible to indirectly estimate the probability that an object can be located at each pixel position using brightness. This probability is defined as the object location probability, and the image representation is defined as the probability map. Figure 8 shows an example of object location probability map for detecting gallbladder in ultrasonogram. In object location probability map, the dark regions indicate that the object can be highly located, and the actual gallbladder region appears darker than other regions.

EXPERIMENTAL RESULTS AND ANALYSIS
The experimental environment for evaluating the proposed object location probability map is as follows. We used PC with Intel i5-4460 3.20 GHz CPU and 8.0 GB RAM, and the method is implemented in Visual Studio 2015 with MFC-based C ++ language. OpenCV 3.10 was used as an image processing library. The 98 ultrasonograms used in the experiment were taken using PHILIPS IU 22 Ultrasound equipment from November 2013 to July 2016. All ultrasonograms are used in the tertiary hospital.

Ground truth for test images
To evaluate the performance of the proposed method, ground truths were generated for the collected ultrasonograms. The ground truth is composed of the center point of the gallbladder, the length of the major axis and the length of the minor axis, and the area of the gallbladder. Table 1 shows examples of generated ground truth. The gallbladder area was directly marked by the person, and the center point, length, and area of the gallbladder were obtained using the program. If the gallbladder looks like separated [26,27] as shown in Table 1 (b), the area is calculated as the sum of the two areas, and the center point and length are calculated as the average of the two area.

Performance evaluation of object location probability
We experimented how many the actual object regions is included in the generated probability map to evaluate the performance of the estimated object location probability. To do this, the probability map is binarized with the threshold of 85 (= 255 × 1/3) to generate a candidate image. If the candidates contain more than 70% of the gallbladder region in ground truth, it is judged that the object location probability estimation is successful. A total of 270 candidates were generated from the candidate images. Since there are 3 images of gallbladder divided into two in the 98 experiment images, there are 101 actual objects. The result shows 99 of the 101 object were included in the candidates, which showed 98.02% object success rate, and 99 of the 270 candidates are estimated as actual objects, and the hit rate is 36.67%.
In addition, we calculated the difference in brightness between the object (gallbladder) region and the non-object (non-gallbladder) region in the generated probability map. The object region is based on ground truth. Table 3 shows the average brightness of the object region and the non-object region in the probability map. In a probability map, brightness means the object location probability, and the lower the brightness, the higher the probability. The average brightness of the object region is about 35% of the average brightness of the non-object region, so that it confirmed that the estimated object location probability is high in the region where the actual object is located. When the estimated object location is not accurately, the average brightness of the object region increases up to 248.47. It is a counterevidence that the object location probability is accurately estimated.

CONCLUSION
In this paper, we proposed a new method for estimating the probability that an object can be located for object detection and generating an object location probability map using only brightness in a gray image where information such as color, shape, and contour cannot be used. The proposed method represented the probability that an object can be located using the brightness using the ends-in search stretching and quantization, and generated the multi-location information. Based on multi-location information, the proposed method estimated the object location probability and generated the location probability map. To evaluate the performance of the proposed method, we generated the ground truths of the 98 ultrasonograms of gallbladder and applied the proposed method to gallbladder detection. Experimental results showed 98.02% success rate by generating the candidates including 99 of 101 gallbladder regions. The proposed method effectively located gallbladder in ultrasonogram and can be applied in various fields.