An effective identification of crop diseases using faster region based convolutional neural network and expert systems

P. Chandana, G. S. Pradeep Ghantasala, J. Rethna Virgil Jeny, Kaushik Sekaran, Deepika N., Yunyoung Nam, Seifedine Kadry Department of Computer Science and Engineering, Vignan Institute of Technology and Science, India Department of Computer Science and Engineering, Malla Reddy Institute of Technology and Science, India Research Scholar, Department of Computer Science and Engineering, Vignan's Foundation for Science Technology and Research, India Department of Computer Science and Engineering, Soonchunhyang University, South Korea Department of Mathematics and Computer Science, Faculty of Science, Beirut Arab University, Lebanon


INTRODUCTION
Among various revenue generating sources for the economy of any country, agriculture is a sector which plays a vital role in the economic development of any country. In other words, we can say that agriculture is the backbone for economy by providing basic ingredients to the mankind and the raw material for the industrialization. As it acts as a backbone for the country economy, the advancements of technology used in agriculture need to be made in order to increase the output which is directly proportional to the country economy. The advancements in the techniques and technology used for agriculture is represented with certain names like smart agriculture or digital agriculture or climate-smart Agriculture, and the strategies followed under this scheme include activities with respect to the actions performed in the agriculture, which may represent percentage of moisture in soil, predicting the crop yield, and suggesting the crops basing on the parameters like moisture, durability and strength of the soil.
The idea of the smart agriculture is to help the agriculture industry by guiding actions required to modify and reorient agricultural systems by supporting the development and providing the food security in spite of ever-changing climate helping the country by increasing the productivity and income. The area of smart agriculture can be considered as a major application area under the research study of Cloud computing, Big data, IOT applications and various modules of design that can be implemented using IOT with respect to smart agriculture is shown in Figure 1 and the modules that can be designed with respect to the crop is shown in Figure 2. Key characteristics of smart agricultures: -Handle climate change: In contrast with traditional agricultural development, Smart agriculture integrates climate change systematically into the planning and development of sustainable agricultural systems. -Combines multiple goals and control trade-offs: Ideally, these systems produce triple outcomes i.e, increased productivity, enhanced resilience and reduced emissions but may not achieve all three at once. When there is a need of implementation, trade-offs need to be made. -CSA maintains ecosystems services: These systems adopt a landscape approach which builds upon the principles of sustainable agriculture maintaining the ecosystem. -CSA has multiple entry points at different levels: These systems can have multiple entry points, ranging from the development of technologies and practices to the elaboration of climate change models and scenarios, information technologies, insurance schemes, and supply-value chains. Expert Systems is the area of Artificial Intelligence, which is a computer system or software design that provides the decision making capability of a human expert. These systems are designed to solve the complex problems by reasoning through bodies of knowledge, represented using IF-Then rules instead of conventional notations of source code. The expert system is divided into subsystems namely inference engine and knowledge base, in which the inference engines apply the rules to the known facts to obtain new facts by including debugging abilities, where as the knowledge base systems represents the facts and rules. The model of an expert systems is shown in Figure 3.

RELATED WORKS
Tan et al., [1], stated that the research on AI has the tremendous effects on the various fields like militaries, medicals, chemistry, engineering, management ,manufacturing etc. They also mentioned that the AI presents the wide scope in the technology development by defining various forms of implementations like fuzzy logic, neural network, rule-based expert system, frame-based expert system, genetic algorithm, etc. They presented the statements that AI research and applications is going to replace the human experts with the latest innovations in the technology.
Bassem et al., [2], proposed an Expert System design which is used to help farmers for diagnosing watermelon diseases namely Powdery mildew,Downy mildew, Alternaria leaf spot, Anthracnose, Fusarium wilt and Cucumber mosaic disease. They designed an expert system which presents an information on watermelon diseases, causes and the treatment of disease using E-clips Expert System language and evaluated with a group of farmers to identify the performance.
Sabzi et al., [3], discussed about the Precision Agriculture where the goals is to identify and to remove weeds intelligently, where herbicides can be sprayed only in weeds existing areas by reducing the dosage of herbicides to avoid harmful effects. They presented a computer vision based expert system for identifying potato plants and three different kinds of weeds to perform site-specific spraying. They presented visualization of agriculture through longitude and latitude positioning for identifying the objects using image processing, they extracted 3459 objects to train and test the classifiers with 126 color features and 60 texture features from each object. They used two metaheuristic algorithms to optimize the performance of a neural network classifier: namely, the cultural algorithm, harmony search algorithm for the optimal configuration of the network and then a statistical method based on linear discriminant analysis is used for the experimental evaluation.
Bannerjee et al., [4], presented a survey on the applications of artificial intelligence techniques in agriculture addressing several issues like crop management, soil management, weed management etc, which leads to severe crop loss along with environmental hazards may be due to excessive use of chemicals. Several researches have been conducted to address these issues. They presented various contributions of artificial intelligence related to agriculture covering the domains and domains to provide information on agrointelligent systems.
Ravisankar et al., [5], presented an web based expert system design for the identification of insect /pest management on the crop by diagnosing the problem to minimize the yield losses. The experimentation is carried out on the tobacco crop. The model designed by them is presented as Agridaksh, using ontology based methods for the effective identification and management of pests. The knowledge model is designed for the efficient knowledge acquisition, knowledge retrieval process.
Nipun et al., [6], presented the smart agriculture concept using IOT, using Arduino kit. They proposed a model with temperature sensors, radar sensors for optimizing water usage and yield and also adopted smart water management systems with consistent monitoring of weather conditions. The setup also represents a part of grid which utilizes solar energy to prevent battery loss with a miniature solar farm. They adopted machine learning algorithms and IOT to handle the problems related to water and energy.
Sarker et al., [7] adopted CNN models from machine to handle complex image detection problems. They used a region-based, fully convolutional network, for processing fast and accurate object detection on the crops. They stated that the deep residual networks (ResNets) can process the training process faster and attain high accuracy than traditional conventional neural networks. They used deep residual network for handling the over-fitting problem as well as data augmentation with ResNet to identify the weeds in the crop.
Hawashin et al., [8], presented the importance of smart agriculture and provided the ideology for managing the water resource management in the agriculture. They adapted the IOt with the vision of multimedia i.e, Internet of multimedia things (IoMT) to optimize the irrigation process with the help of multimedia sensors. They included image processing and machine learning strategies along with IOT to perform the optimized irrigation process.
Fiehn et al., [9], discussed the role of Smart farming as an economic way of agriculture adopting the latest technologies like IOT allowing more number of choices in designing the optimal solution for various challenges. They designed the strategical solution using machine learning to handle the water supply irrespective of climate changes. They used RF, Solar energy with various sensors and actuators to perform the monitoring process. Later they used Convolutional neural network for performing the data flow control. Jha, et al., [10,11], presented the survey on various problems that will arise for a crop in agriculture by choosing the grape crop. They also discussed about various challenges that are recognized in the crop management and various machine learning strategies that can be applied for effective or smart agriculture application. They gave the ideology of using multiple strategies to invoke solutions for handling different forms of challenges.

ARCHITECTURE
The architecture of the proposed work is presented in Figure 4. The Architecture can be explained in 4 stages namely acquisition, validation, query processing and presentation. a. Acquisition: This stage is considered for obtaining the crop images from various sources, which can be either captured or maintained as a dataset by the administrator or the farmer who captures the image of effected area of the crop using capturing devices. The images captured from various sources are identified with the infected areas and annotated with the information of infection and maintained as a dataset using object recognition [12] techniques

RESEARCH METHOD
The methodology used in proposed work collaborates two areas of research study namely image processing, expert systems. In this work, we adopt the object recognition methods for the effective identification of infections on the image, then it is annotated with the terminologies of infections identified. These annotations play a major role in classifying the type of infections obtained which may be due to weeds or due to climatic conditions and soil quality. The methodology can be classified into two strategies namely, Rot Identification and disease predictor. The methodology in the paper is evaluated and carried out on the maize crop which contains several forms of infections and diseases which may be due to soil consistency, climatic conditions, weeds and bugs.

Rot identification
The term rot represents the infections obtained for the plant which may be occurred due to bugs or weeds and the climatic conditions, soil quality. Process of Rot identification [13,14] deals with the identification of infected area on the leaf or plant due to any of the aforementioned cases. The process of identification adapts the object recognition strategy of image processing and uses machine learning techniques [15,16] to effectively identify the affected areas. In this approach, the faster Region-based Convolutional neural network (R-CNN) is used to process the detection phase faster and store the annotations for the processed image as a text file. The concept of faster RCNN is given in the preceding sections.

Convolutional neural networks (CNN)
The Idea of including CNN is to train the data that is the images of various types of diseases in the crop such as infections and other possible rots. In The convolutional neural networks each layer will be considered for the training the data. The CNN includes main layers namely convolutional neural networks layer, Max-pooling layer and soft-max layer for performing effective classification of the dataset. Each layers includes set of calculations to perform the mapping of larger datasets with the given input test set.

Convolutional layer
The objective of the convolution layer is to map the upper level features to the current layer using convolution operation. It can be expressed as shown in (1).
where n means the nth layer, Cpq represents the convolutional kernel, Bq represents the bias, and Fq represents the feature maps.

Max-pooling layer
In this layer, the sampling is performed by reducing the size of feature maps to achieve the spatial mapping for the faster and better performance. Also, the feature map at each layer is passed on to the next consecutive layer using the following max function in order to maintaining as a pool as given in (2).
where, PRq denotes pooling region in the layer and q denotes in feature map and also interestingly, the pooling layer of this could be represented by Lp, and p represents the index of every element in the pool and MP denotes the pooled feature maps. The processing of images in the proposed work requires faster and efficient mechanism for training and providing the outputs with respect to farmer queries. Therefore, the advanced convolutional network for processing images namely, region based convolutional network has been included. The extension of the RCCN has been discussed below.

Soft-max layer
Soft max layer is used to perform classification with respect to dataset availability. It is intended to create the training dataset by applying the cost function θ. Also, for each test set, it verifies the training dataset for the appropriate match. If the Loss value obtained is less, then with respect to cost function the value could be changed for the appropriate match. If this default case fails, then it is a mismatch. This scenario has been explained through the regression model of the soft max layer as represented in (3).
where h and w represent the pixel positions of the images in the training set, θ represents the cost function and finally T represents training set.

Faster RCNN
It is the model, obtained using the selective search on the bounding boxes (regions) identified using region based convolutional network [17] by applying deep models for object detection. In this method, the generation of region proposal and object detection are performed using same convolutional networks [18] which leads to faster object detection. The entire image is passed to ConvNet for generating region of interests rather than extracted regions from the image. It uses a single model which extracts the features from the regions, and classifies them into different classes by returning the bounding boxes. It adapts region proposal network (RPN) rather than selective search approach. The feature maps from the input image are extracted using ConvNet and the maps are passed through the RPN for returning the object proposals, and these maps are finally classified and the bounding boxes are predicted. The transformation of mapping is shown in Figure 5.
The Algorithm for the Faster RCNN [19] can be illustrated as follows: -The input image is passed to the ConvNet and extracts the feature maps of the image.
-The object proposals are obtained on these feature maps by applying Region Proposal Network (RPN) using a sliding window mechanism on the feature maps. -For each location, k, (k=9) anchor boxes are used in a scales of 128,256,512 and aspect ratios of 1:1, 1:2 and 2:1 sizes by applying ROI pooling layer to generate all the proposals to uniform size. -A layers named cls and reg layers were used to identify the object for the k boxes and to generate the box coordinates with a uniform width and height with a scores of 2k and 4k and then the WXH feature map is generating WHk anchors in total at the RPN layer, see Figure 6. -Finally, these proposals are passed to a fully connected layer in order to classify and predict the bounding boxes in the image.

Disease predictor engine
After identifying the classification of rots in the rot identification phase, the set of questionnaires is generated with the basic symptoms of the identified rot, the decision rule logic is followed by extracting the basic ontologies from the set of symptoms generated for the type of rots. The ontologies with highest precedence of words will be considered as an important symptoms and presented to the farmer in the form of questionnaire consisting true or false/ yes or no logic using the IF-THEN principle., if the set of answers are determined as true, then the information about the symptom is presented to farmer. The text to voice generation application can also be integrated to support the kissan application. The IOT [20] operations can be incorporated to provide the information via SMS or a phone call.

RESULTS
The experimentation is carried out on the maize crop which has a greater number of classifications of infections and diseases [21][22][23][24][25] due to bacteria, fungus, pests, and climatic conditions. The results of rot identification using faster regional convolutional neural network (R-CNN) are shown in Figure 7

COMPARATIVE STUDY
The Proposed methodology is experimented on the dataset of maize crop and compared with other approaches of object detection algorithms namely Region based Convolutional Neural network(R-CNN), Fast R-CNN, Spatial Pyramid Pooling Network (SPP-Net) algorithms. The comparative analysis is made for analyzing the performance of algorithms in terms of speed and Accuracy. Table 1 presents the speed of recognizing the objects like rots and weeds in the crop by including and excluding the region proposals in terms of Test-Time Speed. The Graphical analysis of the performance is presented in Figure 11, with the analysis of optimal speed in recognizing objects on the crop.  Figure 11. Test time speed analysis of algorithms

CONCLUSION AND FUTURE SCOPE
The proposed work focus on integrating the machine learning strategies to the expert system to provide the exclusive information to the farmer by helping them with the information about the infections occurred to their crops. The experimentation in the proposed work is carried out on the maize crop as it contains the various classifications of diseases which might be due to climatic conditions, fungal infections etc. The process simplifies the task of knowing the details about the infections occurred by simply uploading the image of the infection, which is fed to the image processing strategy using Machine Learning for identifying the rots using Faster R-CNN and further classifies the infection and presented to the disease predictor engine and then the set of questionnaire is generated to the farmer on the type of infection plotted asking for the answers in the form of Yes/No, the set of yes's presents the information about the infection occurred along with the reason of occurrence and preventive measures. The methodology of Faster R-CNN presents the efficiency in terms of performance by operating within a fraction of 0.2 secs. The work can be extended by employing the RGB configuration in the bounding box to predict the object more accurately with respect to the color of the surface of a plant and leaf color and integrate sms and call gateway API's, text to voice translation systems which would help in more precise way to the farmer by supporting illiterate farmers providing the precautions and preventive measures in the form of voice commands using IOT operations.