Modified JSEG algorithm for reducing over-segmentation problems in underwater coral reef images

ABSTRACT


INTRODUCTION
Within the last decades, the application image and video information processing, analysis and understanding has grown tremendously. These applications spread ranging from remote sensing to microscopic biomedical imaging [1]. Although image analysis can be performed manually but such analysis is rather difficult, time-consuming and painstakingly to be carried manually. Furthermore, considering that data are massive regarding the amount and the volume that keeps increasing each single day, the manual analysis approach may even be impracticable to do. Therefore, a system that could automatically and efficiently analyze imagery for various applications needs to be established [1].
In the past few years, image processing methods has been applied to process underwater images. The primary reasons for the challenging issues in underwater image processing is owing to the absorption and scattering effects of the light in an underwater environment and the underwater noise exhibit due to different water quality. This has led to the increasing exploration, understanding and investigation of image processing technique for underwater images. In the case of monitoring and survey of coral reef distribution, marine scientists require an automatic approach to estimate the coral reef distribution. Current method involves task that is time consuming, tedious and not accurate. Thus, automatic image segmentation that could separate a particular type of coral reef species from other is sought. Image segmentation is one of the vital process that involves the partitioning an image into non-overlapping differing regions with identical attributesthat have homogenous features or have the common characteristics [2][3][4][5]. Image segmentation is a critical process in many image processing applications such as shape recognition, object detection and optical character recognition [6,7].The primary goal of segmentation is to make the image more simple and  [8]. Segmentation of color image has been very popular for various applications and owing to its diverse application types. Still up to today, image segmentation remains a challenging subject [9,10]. One of the most robust methods of segmenting natural imagesis the JSEG algorithm [11]. This original JSEG algorithm has been provento be very useful and robust in variety of image segmentation. The original JSEG algorithm is a fully unsupervised segmentation algorithm capable to properly segment images of natural scenes.However, when this algorithm is applied to the underwater image environment, it produces over-segmentation problem. The main reasonfor this problem to occur is due to the varying shades and varying illumination in an underwater environment. Therefore, this paper proposed a method to reduce this over-segmentation problem by replacing the color histogram computation in region merge stage of the original JSEG algorithm with the new computation of color and texture features in the similarity measurement. The proposed method in this paper could automatically segment the regions better than the original JSEG algorithm. The main objective in this paper is to enhance the segmentation process by reducing the over-segmentation region when applying to the underwater coral reef images. JSEG algorithm [12,13] is among the most popular and effective methods for color image segmentation that is entirely unsupervised. JSEG automatically segments the color-texture region within a given image or video. JSEG algorithm works based on the premise that an image consists of homogenous (uniformly distributed) color-texture patterns or regions where it is aimed to quantize color information into several color regions while the color within the two adjacent regions is distinct perceptually. Such presumptions cause inability to JSEG algorithm in making a distinction between two adjacent regions which share similarities in color but different in terms of texture [14].
JSEG algorithm has proven its strong capacity in dealing with many types of natural images [15]. Somehow, there are several major problems relating to this algorithm, such as varying shades due to the illumination and over segmentation problem. Furthermore, as it was mentioned in the work by Deng et al. [12], the original JSEG has issues with spatially varying illumination. Aloun et al. [15] reported that the performance of the original JSEG method was poor for underwater image segmentation because of the aforementioned limitations and it appears to have difficulty in making a distinction between regions with color textures. It should be noted that within the underwater setting, as the water gets deeper, the light becomes more absorbed and scattered. Hence there is a pressing need for a robust and reliable method for underwater image analysis and segmentation [16,17].
The problem of coral reef thrives in countless colors, textures and shapes make segmentation even more challenging. Furthermore, coral reefs of the same species will appear different in terms of color at a differing depth of water. [18,19]. The problems of JSEG algorithm are openly recognized in the original paper [12], i.e. over-segmentation due to overlapping color-texture similarity in neighboring region. Accordingly, some alterations and comparisons were made with the original JSEG algorithm within the last decade. Jing et al. [20] added the H measure in the spatial segmentation process of the original algorithm and called them HSEG. In a related study Zheng et al. [21] added a fuzzy mechanism to JSEG in the construction of a soft class map. This soft class map strengthen JSEG functioning in an unsupervised image segmentation. Chang et al. [22] presented another modified JSEG method and named it as the improved contrast JSEG or ICJSEG. The method proposed by Wang et al. [23] allows the segmentation of the image by color and texture. Furthermore, JSEG segmentation algorithm had been enhanced by Kibria and Islam [24] through the addition of an edge detector. Komati et al. [25] introduced a method which combines the original JSEG algorithm with a local fractal operator.
This paper is organized as follows: Section 2 provides a related works. Section 3 presents the JSEG segmentation algorithm. Section 4 presents implementation of proposed method. Section 5 presents the experimental results and discussion and the final conclusion is given in Section 6.

RESEARCH METHOD
JSEG stands for J-segmentation, is an unsupervised color-texture segmentation algorithm [12]. It comprises of two computational phases that are successively applied which are color quantization and spatial segmentation. The first phase entails the sampling of the color information of the input image using a reduced set of significant colors (ranging from 10-20 prototypes) acquired following the technique of a peer-group filtering color quantization. This process maps out the image into a structure within which every pixel is provided a class label. This is followed by a phase where the algorithm uses spatial composition of the provided class labels employing a segmentation criterion, namely J value by sampling the local homogeneity. Applying the criterion to local image windows results in the J-images which can be segmented using a multi-scale region growing method. In this process, the original seeds needed by the region growing procedure align with minima of local J values. Moreover, the growing process of multi-scale region frequently produces an over-segmented outcome and to resolve this issue, a method of post-processing is used to combine the adjacent areas on the basis of color similarity and the Euclidian distance. It is important to state that the acquired outcomes hinge on the optimal choice of three parameters that should be a priori user-specification in terms of color quantization threshold, number of scales and the merge threshold [12,26]. The general flow of the JSEG algorithm is presented in Figure 1.

Color quantization
The primary aim of the color quantization is to reduce the number of colors in the original image to decrease the complexity of the algorithm. The provided image colors are decreased using Peer Group Filtering (PGF) algorithm [13,27] where the region is quantized with similar color and differentiated through classes. The PGF algorithm is described as a non-linear algorithm used to smoothen a given image by deleting impulsive noise while maintaining the edge as well as information details. Let xo(n) indicate an image pixel vector, the color information at position n centered in a wxw window and arrange all the pixels in the window according to their distances to xo(n) in ascending order and indicate them as xi(n), i = 0, ...,k = w2 -1 . The Euclidean distance measure is used where, The peer group P(n) of size m(n) for xo(n) is defined asfollows: It consists of 0 (n) itself and its neighboring points of similar colors [27]. Therefore, the next color quantization step entails the appropriation of weight to every pixel with smooth area were given more weight than the textured ones. In the third step, a modified General Lloyd Algorithm (GLA) [28] is applied to vector quantize the pixel colors. Lastly, cluster centroid are exposed to agglomerative clustering algorithm [29] to combine clusters located in the neighboring area of a specific preset threshold value that is user-defined.

Spatial segmentation algorithm
In this spatial segmentation process, the J-value concept is used. For instance, if Z is a set (x, y) image pixels in a particular class map and it is classified into C classes Zi, with i representing 1, ….C, m denoting spatial mean of the whole Z points, and mi denotes the spatial mean of pixels in Zi, then the total variance of the whole Z points are represented as follows; The total variance of points that belong to one class is represented as: A distribution measure of color classes is given by J-value is defined as follows: The J-values concept is used in the class map local area to indicate a specific area in the region center or its boundaries. J-images concept refers to a grayscale image with pixels having J values acquired through calculating over local windows focused on the pixels. The J values are computed at different scales.The next step in spatial segmentation is the application of region growing algorithm in the region and region merging. In the first one (region growing), there are two interactive process involved namely valley determination and valley growing. At the beginning of the region growing process, there is one initial region (the entire image). Figure 2 shows the spatial segmentation step.

The proposed method
To overcome the over-segmentation problem in an underwater coral reef images, we replaced the merge stage in the main JSEG original algorithm with a new technique based on the similarity measure using mean and covariance as illustrated in Figure 3.

Region merge in the original JSEG algorithm
In the original JSEG algorithm, region growing results in an initial segmentation of the image, thus over-segmented regions appear to be an issue. As a solution to these over-segmented regions, these regions are merged according to their color similarity. In this regard, color histogram characterizes the region's color information. Meanwhile, the color histogram bins are based on the quantized colors acquired from the process of color quantization. The Euclidean distance between two color histograms, DCH at i and j can be defined as follows: where P represents the vector of color histogram. The region merge method is grounded upon the agglomerative method in Duda and Hart [29]. By using this method, firstly, a distance table comprising the distances between the color histogram of any two nearby regions is constructed. In the process, the pair of regions comprising the least amount of distance is merged together. This is followed by the computation of the new color feature vector of the new region. Accordingly, the distance table is renewed together with the nearby associations. The process of region merge ensues till the highest threshold for the distance is attained [12].

The proposed region merge using similarity measure
Segmentation is executed to segment objects within an image that is very much comparable to human perception. As mentioned by Deng and Manjunath [12], JSEG algorithm has over-segmentation issue caused by the spatially fluctuating illumination. Such problem causes change to color image components in terms of intensity and chrominance [30]. After the stage of region growing, in the original JSEG algorithm, the following stage is region merging. Region growing involves the application of multiple seeds and some of these seeds may originate from the exact region. Accordingly, one region may contain more than one partition and region like this is not desirable. In this paper, the issue of over-segmentation is reduced via the merging of these partitions in accordance to the certain criterion of similarity. The procedure is detailed as follows: a. Create the mean and covariance measures of all regions alongside their adjacent neighbors. b. Merge regions which contain less than a minimum pixel count with their neighbors. Then, update the new region-neighborhood statistics. c. Calculate the similarity measure, S of two regions.
Taking into account two regions namely region A and B, the similarity measure between the two regions with color mean, µA, and µB, and the color co-variances, covA and covB. The similarity measure of two regions can be mathematically expressed as follows: The measure of similarity elucidated in (8) appears to be more robust measure as opposed to the widely employed color measure of means. This owes to the fact that this measure of similarity employs both the mean and the co-variances of the pixels within the regions to be merged. Hence, the comparison is made to the regions not only for their content of color but also to their content of texture prior to the establishment of the merging decision. d. Neighboring regions containing the smallest S value matches with the highly similar regions. For these regions, they are the earliest to be merged. For the next step of region merging, the update is performed on region-neighborhood statistics of S. This process iteratively ensues until the merging threshold, MThresh whose value determines the merging magnitude, is attained. Higher values of MThresh lead to more regions to be merged, and vice versa. The outcome comprises the final segmented image.

RESULTS AND ANALYSIS
In this paper, the execution of the proposed method has been tested on the coral reef images obtained from the Institute of Oceanography and Environment, Universiti Malaysia Terengganu (INOS), Malaysia. The experiment aims to automatically reduce the over-segmentation in the coral reef image. Usually, this segmentation process is being carried out manually by the marine scientists to estimate the coral reefs population growth. Figure 4 and Figure 5 show the results of segmentation method by the original JSEG algorithm (on the left column) and the respective modified JSEG technique (on the right column). Figure 6 shows the results of segmentation method by the original JSEG algorithm (on the first row) and ISSN: 2088-8708  Modified JSEG algorithm for reducing over-segmentation problems in… (Mohammad Sameer Aloun) 5249 the respective modified JSEG technique (on the seconde row) to reduce over-segmentation problem on 3 selected samples of coral reefs images at different underwater environment. By applying the original JSEG algorithm, it can be apparently observed that when the JSEG scale of the segmentation is set too low, it leads to under-segmentation. Whereas, if the JSEG scale is set to too high, it resulted in over-segmentation of the coral reefs. Therefore, it is a challenging process to find the correct JSEG scale that leads to the best segmentation results of different coral reef types by using the original JSEG algorithm. Although this algorithm can automatically segment the objects, it has the problem of either under or over-segmentation. The user has to find the correct setting of the JSEG scale to be used for region segmentation in these figures. In contrast to the original JSEG algorithm, the modified JSEG algorithm operates by directly reduces the number of regions that have common color-texture similarity in the image. The merged region stage in original JSEG algorithm has been replaced with region merging using the newly proposed similarity measurement. The rest of the algorithm is still following the original JSEG algorithm as shown in Figure 3. The effect of this modified method is, it reduces the number of regions for tested image as compared to the original JSEG algorithm. By applying the color means and covariance at the neighboring regions, it managed to reduce the over-segmentation issue in the original JSEG algorithm as opposed to the widely employed means color measure. As can be observed from these figures, the number of regions are depend on the preset threshold values, which we already used to merge the regionsby experiments. The final image refers to the last segmented image and can be compared directly to their respective counterpart in the original JSEG algorithm.

CONCLUSION
In this paper, a modified JSEG algorithm has been proposed where the merge region phase in the original JSEG algorithm has been replaced with region merging using similarity measure of mean and covariance of color statistics. The modified algorithm could automatically segment the regions of the test images better than the original JSEG algorithm. The new implementation has alleviated the problem of over-segmentation in the original JSEG algorithm. Even though the modified algorithm provide direct segmentation, however some of the regions still been incorrectly segmented. This problem arises due to the complex nature of underwater coral reef images, and we intend to solve this problem by using the composition of texture, color, and shape in the future research.