Pine wilt disease spreading prevention system using semantic segmentation

Pine wilt disease is a disease that affects ecosystems by rapidly killing trees in a short period of time due to the close interaction between three factors such as trees, mediates, and pathogens. There are no 100% mortality infectious forest pests. According to the Korea Forest Service survey, as of April 2019, the damage of pine re-nematode disease was about 490,000 dead trees in 117 cities, counties and wards across the country. It's a fatal condition. In order to prevent this problem, this paper proposes a system that detects dead trees, early infection trees, and the like, using deep learning-based semantic segmentation. In addition, drones were used to photograph the area of the forest, and a separate pixel segmentation label could be used to identify three levels of transmission information: Suspicion, attention, and confirmation. This allows the user to grasp information such as area, location, and alarm to prevent the spread of re-nematode disease.


INTRODUCTION
Recently, with the development of artificial intelligence, research on deep learning technology using satellite photographs and aerial photographs has been actively conducted. In particular, the interest in the technology for recognizing objects by using images or photos has been very high. Object detection is one of the deepest technologies for deep learning that can identify multiple objects in one image and identify their location. It is also applied as a technology for detecting pine objects to solve pine re-nematode disease. However, the existing system is an object recognition-based system that learns whether a pine tree is infected by learning a specific part of the image, but it is effective in simply recognizing an object and determining a target. It is more efficient than the existing system. In addition, it showed high accuracy in relatively close pictures, but low accuracy in high-altitude aerial and satellite photographs [1][2][3][4][5][6][7].
In order to solve this problem, this paper proposes a pine re nematode spreading prevention system using semantic splitting. Semantic segmentation is a field of object detection that recognizes and labels a set of pixels constituting different categories. Conventional object detection detects objects based on what are inside a specific bounding box, but semantic segmentation is more precise than other forms of object detection because it uses a segmentation label applied to pixels in the image. In this paper, a separate pixel segmentation label was applied to the pine re-nematode data set photographed by the drone to identify the 2668 three stages of infectious information such as suspicion, attention, and confirmation. This allows the user to grasp information such as area, location, and alarm to prevent the spread of re-nematode disease.

DETRECTION OF SUSPECTED PINE WILT DISEASE 2.1. Doubt detection with supervised classification
S. K. Lee et al. [8] acquired high resolution unmanned aerial vehicle (UAV) images in relapsing areas of pine nematodes, and conducted suspicions of suspected pine nematodes through supervised classification techniques such as SVM and ANN. Based on the acquired learning data, we applied two techniques, ANN and SVM, to calculate 99.43% and 84.60% of user accuracy and manufacturer accuracy using ANN, and 99.24% and 95.35% accuracy using SVM.

Extracting suspect space locations using RGB values
To identify the damage of pine re-nematode disease, the experiments were conducted by specifying the damage RGB values (Red 139, Green 113, Blue 123) and increasing the range values corresponding to RGB values from 1 to 10 [9][10][11][12]. In addition, when a clustering phenomenon occurs among the calculated pixels, it is recognized as an object as a colony. The purpose of this study is to present an extraction methodology that can extract damages using RGB values of images taken at high altitudes. However, due to the characteristics of RGB values, it is difficult to calculate certain accuracy simply by changing the range of RGB values, such as the RGB values varying due to various factors such as weather conditions, shadow presence, shooting height, and pixel size [13][14].

SYSTEM DESIGN 3.1. Data set preprocessing
The data set used in this study is an aerial photograph with both infected and normal trees. The pine tree infected with the pine re-nematode is characterized by browning and clustering. Clustering is a characteristic of re-nematode disease, it spreads the media to surrounding pines, so it can be confirmed that clustering phenomenon with surrounding pines occurs around the early tree of infection. Therefore, the data set was preprocessed using this feature.
As shown in the circle in Figure 1, the focusing effect and noise reduction are performed around the browned and clustered group to improve the accuracy of the data set. In addition, the noise generated during the process is removed to improve the quality of the data set. As the preprocessing of the image proceeds, it can be distinguished by the unaided eye and the boundary with the normal tree becomes clear. To evaluate the accuracy of the data set preprocessing, the dataset was stored in a separate directory and compared with the image without preprocessing.

Overall system design
The purpose of this system is to prevent the spread of pine nematode disease. It aims to detect pine tree infected with re-nematode disease and isolate it from normal tree based on the data set taken by aerial photograph. Therefore, the area of infected trees, the area of normal trees, and the area of suspected suspicions are required. The proposed system applies the preprocessed data set to the semantic partitioning model based on SegNet and returns each area. It also visualizes the area returned and provides it to the user Figure 2.
The system is a two-stage structure. The first step is to collect, classify and correct the data. The process is as: In the collecting phase, the user collects image data of the area suspected of pine re-nematode disease Figure 3. Collected objects can effectively collect a wide range of aerial photographs and satellite images. In the classification phase, you create a training set by labeling images so that you can classify areas based on the collected data set. The criteria to classify the area is a label classified based on the RGB value of Int J Elec & Comp Eng ISSN: 2088-8708  Pine wilt disease spreading prevention system using semantic segmentation (Chulhyun Hwang) 2669 the image. Since the classification targets of this system are infection trees, suspicions, and normal trees, they are defined based on the classification targets. In the correction step, the noise is removed from the image and the focus is applied to the infected tree to improve the overall accuracy. The second step is to learn the data set and the training set, and to derive the results. It is divided into modeling, segmentation, and output phases.
In the modeling stage, models based on SegNet can be selected and applied. In the segmentation step, the image is segmented based on the learned results. The output step derives the visualized image based on the divided image in the dividing step and returns the value of the divided area. It also returns the accuracy of the segmented area and provides it to the user.

Image classification area definition
Deep learning model is a model that maximizes the effect when built on a large amount of data. Therefore, it is important to secure a large amount of data in semantic partitioning. In addition, since semantic segmentation configures a data set by applying a label according to a classification target, an image to be classified must be labeled by category.
Image labeling was done using MATLAB's ImageLabeler. After loading the image, specify an area and a line to assign a label area to the image. Therefore, the classification targets were labeled as infectious, suspicious and normal. In addition, infected trees, suspicious trees, and normal trees were labeled and labeled as orange, blue, and yellow, respectively. This designates the labeled data set as the training set and the partition used for training.

Model selection and training
Semantic partitioning is based on the encoder-decoder architecture. Each encoder has a downsampling decoder and upsampling the structure. Among the models with the above structure, representative models with high accuracy are FCN, Unet and DeepLabV3, which are similar to the existing SegNet, but with high accuracy in a specific image. Therefore, in this paper, we designed to evaluate the accuracy using the test image set trained and prepared based on the training data set prepared for all the above models to compare the accuracy in a specific image [15][16][17].

Image region segmentation
Image segmentation is the segmentation of test data using trained weights after model selection. Classification areas are defined in a separate CSV file to provide visualization effects to users. Figure 4 shows the image inside the CSV file to define the RGB color. Infected, suspected infection, and uninfected are defined to be infected, suspicious, and normal, respectively, and marked in red, yellow, and green. The other Sky and Sod mean sky and earth, respectively. Void means an unspecified area and is marked in black.

FCN design
The biggest feature of FCN is the lack of a fully connected layer. Change the fully connected layer to 1x1 convolution layer in the existing CNN structure. The reason for deleting the fully connected layer is to obtain location information. The key to semantic segmentation is to get location information because the location information disappears through the fully connected layer. Therefore, the last layer was replaced with a 1x1 convolution layer in the VGG16-based model. We then upsampled using transposed convolution and created a map the same size as the input. Also, in the upsampling process, feature maps of lower layers were added to extract features by combining them with feature maps above. After that, all the features were recombined through the skip connection to derive the results [9].

DeepLabV3 design
DeepLabV3 is based on ResNet learned from ImageNet. ResNet feature extractors can be used to extract features of varying sizes. At the core of DeepLabV3 is the Atruos convolution layer. You can drill holes in a typical convolution layer to capture larger features with the same amount of computation. Therefore, in this paper, we designed the Atrous convolution layers in parallel to capture more features [18-22].

U-net design
U-Net is a U-shaped network with repeated contracting layers and upscaling. We can see that U-Net divides the image into several images while gradually reducing the size of the image during the contract process. Thereafter, the upsampling process is performed to obtain the divided image. In this paper, U-Net is designed by adopting the above structure [23][24][25].

SYSTEM IMPLEMENTATION
This chapter deals with the implementation of a pine nematode disease prevention system. In addition, we verify the effectiveness of the proposed system through consideration. The system building environment is shown in Table 1. This paper builds Anaconda 4.5.11 environment in Window10 Pro environment to build deep learning environment. GPU acceleration is used to improve training speed, and CUDA version is 10.0. Also, the implementation of the system uses Tenserflow 1.14.0 based on Python, and the library dependencies are OpenCV and numpy. The development tool used PyCharm and MATLAB's Image Labeler to label the training set.

System implementation
The system trains the semantic segmentation network model based on the labeled data set and returns information such as suspicion, normal tree, and infected tree of the test set using the weight. Provides the user with a visualized image and returns the area of the segmented area. The internal directory is divided into a data set directory and a model directory. When you run train.py and select a model, it trains based on the training set stored in the '../dataSet/train_Label' directory. test.py performs semantic partitioning of the data sets in the '../dataSet/test' directory based on the training weights trained in the above process. The result is stored in test_Label. To apply semantic splitting, you need a labeled training set. In this paper, a labeled image was created based on suspicious, normal and infected trees, and training weights were generated by applying semantic segmentation network models. Figure 5 shows the data set stored in the test directory. Once all the training weights have been generated, run test.py. After selecting the model and importing the above data set, the semantic segmentation proceeds and the network model is evaluated together. Figure 6 shows the visualized image of the semantic segmentation being completed. Green is an uninfected normal tree, and red is an infected pine tree. Yellow is the suspicious tree and black is the unspecified area. Figure 7 shows the result screen showing the share of each classification model in the total area. According to the area classification, 27.38% of infected trees, 35.52% of suspicious trees, 25.14% of normal trees, 3.32% of voids, and 8.64% of land. Pixel classification accuracy is about 87%.

Discussion
Most suspected pine re-nematode suspicion detection systems used supervised classification techniques to directly assign classification targets or to extract and detect RGB values of suspicious trees. However, the above method is an object recognition model that learns a specific part of the image. However, the accuracy of the object recognition model is observed, but the position and area of the object cannot be returned due to the characteristics of the object recognition model. Therefore, this study proposes a system that improves accuracy and returns the position and area of an object by using semantic segmentation, an object detection model, for suspicious detection. This section verifies the efficiency by comparing the classification accuracy of the existing system with that of the proposed system. Also, we analyze the best model by comparing the accuracy of semantic segmentation models applied to the proposed system.
The experimental method trains each model using a training set, extracts training weights, and measures pixel classification accuracy and area accuracy using a test set. Table 2 is a learning set containing the environment in which the experiment will be conducted. The experimental environment used for the experiment is shown in Table 2. Using the models of FCN, DeepLabV3, SetNet, and U-Net, Batch Size is set to 50 and Epoch is 50. The number of training sets to be used is three. The semantic segmentation is performed by applying the trained model weights using the above test set. Figure 8 is a graphical representation of the training data for each model. As a result, FCN, U-Net, DeepLabV3 and SegNet have 87%, 88%, 91% and 85% classification accuracy, respectively. When the experiment was conducted with three test sets, the accuracy is shown in Table 3. Test set 1 and test set 2, which are relatively easy to classify, have high accuracy with average accuracy of 87.75% and 91.25%, but 81.75% for test set that is difficult to identify, such as test set 3. In addition, DeepLabV3 was selected as the model with the highest accuracy with an average accuracy of 90%.

CONCLUSION
Pine nematode disease is a forest pest epidemic with a 100% mortality rate. Therefore, early detection is very important, and it is important to harvest the infected tree as quickly as possible. The Forest Service operates a monitoring center to take control of the complaints. However, due to the nature of the forest area, it is very difficult for humans to survey all areas. Recently, a system that combines machine learning and drones to effectively detect a wide range has been studied. Due to the nature of pine nematode disease, the location and area detection of suspicion is important. However, the object recognition model, which is one of the weak points of the existing system, is effective in recognizing objects but cannot return the position and area of the object.
To solve this problem, this paper proposes a prevention system to prevent the spread of pine nematode disease using aerial photographs and satellite images. In this paper, semantic segmentation, an object detection model, is used to classify infected trees, suspicious trees, and normal trees, visualize them, and return the area and accuracy of the separated areas to the user. In addition, the average classification accuracy of each model was about 86.9%, and the maximum classification accuracy was about 90%. Through this, we could verify that the object detection model has similar classification accuracy without falling classification accuracy compared with the existing object recognition model. Therefore, it was concluded that the object recognition model, such as the related research in Section 2.3, is fast and effective for simple object recognition, but that the object detection model is suitable for aeronautics and satellite images, which have a lot of information.