Soil moisture level prediction using optical technique and artificial neural network

Received Aug 3, 2020 Revised Sep 4, 2020 Accepted Sep 15, 2020 This research describes the use of an optical system combined with artificial neural network (ANN) for wireless and nondestructive prediction of soil moisture level. The former system comprising of near infrared (NIR) emitters of wavelengths 1200 nm and 1450 nm, and a photodetector for near real time soil moisture measurement in loams and peats holding different amount of water. There were 63 and 90 sets of data from loams and peats, respectively, used in the development of the dual stage-multiclass ANN model, wherein measurement of light attenuation (from nondestructive system) was correlated with percent soil moisture (from destructive gold standard approach) in pre-measurement stage. The result revealed a relatively good performance in the training of the NN with regression, R, of 0.8817 and 0.8881, and satisfactory error performance of 0.7898 and 1.172, for loams and peats, respectively. The testing of the system on 50 new samples of loam and peat showed a considerably high mean accuracy of 92% for loams while 82% was observed for peats. This study attributes the poorer performance of the system used on peats to the detection resolution of percent soil moisture, and structure and properties of the corresponding soil. This work concluded that the developed technology may be feasible for use in the future design and improvement of agricultural soil management.


INTRODUCTION
There is no doubt that the human population is increasing rapidly. The demand for natural resources such as water would increase due to this population growth. Agriculture is an important source of food security for human beings. Soil moisture, which is one of the important key factors to provide sustainable yield production, is corresponded to soil water content and it plays an essential role in regulating soil surface temperature and photosynthesis. In a nutshell, smart farming is optimization of agricultural inputs such as water, fertilizers, pesticides and tools to enhance yield, quality and productivity.
The most popular technique to measure soil moisture condition [1][2][3] is near-infrared (NIR) reflection technique due to its nondestructive attribute. Yin et al. [4] designed a NIR sensor for soil moisture measurement. This system used two light-emitting diodes (LEDs) of wavelength, λ, 1940 nm and 1800 nm to calculate for the relative soil moisture reflectance. The soil moisture was predicted using a calibration model. The results revealed a strong linear correlation between soil moisture and relative absorption depths for the different soils tested to give regression (R) of 0.642 and root mean square error (RMSE) of 8 parameter R is normally taken to be analogous to coefficient of determination, ρ [5]. A portable soil moisture sensor based on NIR spectroscopy was developed by An et al. [6]. The optical unit included a near-infrared lamp emitting light at λ=1450 nm, a shared lamp-house drive circuit, a Y-type optical fiber shared by incidence and reflectance, a probe and an Indium gallium arsenide (InGaAs) photoelectric sensor. The soil moisture data was first collected by this portable soil moisture detector. Next, a forecasting model of soil moisture established using a linear regression method was used in the prediction based on the collected data. The model was shown to generate ρ of 0.88. The author [6] further presented the limitations of the developed system, which include the mandatory mechanical support and space for the system. Machine learning techniques such as artificial neural network (ANN) is an innovative technology that is able to establish a relationship of regression between two variables. Besides, it also performs as a predictive model to predict the response of a new set of data. Arif et al. [7] estimated soil moisture by using ANN. In the corresponding work, soil moisture was measured by using a 5-TE sensor every 30 minutes, which was selected as the input of ANN. The result showed the prediction model performance with ρ of 0.80. In another study, Zeng et al. [8] presented measurement using AgriSpec portable spectroradiometer (ASD) across spectral range of 350-2500 nm to predict near surface moisture content. The calculated results showed relatively good performance with ρ=0.932.
Jaya et al. [9] developed an ANN based algorithm for soil moisture estimation using soil hydra probes. The ANN model was trained with soil moisture data to produce R=0.768. It was reported that the limitation in the use of the probe includes destruction of the soil structure and plants. Qing et al. [10] proposed an approach to estimate soil moisture in agricultural using ANN on GF-3 and Landsat-8 satellite data. The data were collected using optical land imager (OLI) sensor and thermal infrared sensor (TIRS) in training the neural network (NN) model. The result showed a good linear correlation and satisfactory results with R of 0.736 and supporting RMSE of 0.042. However, remote sensing technique is expensive in its operation and required extensive analysis [11,12].
Since the previous reported soil moisture systems were either bulky or destructive, and the operation of remote sensing technique is both costly and exhaustive, there remains a need to develop a portable and reliable system for soil moisture measurement. The aim of this work is to develop a near-infrared reflectance system involving two light wavelengths to predict soil moisture level using ANN technique.

RESEARCH METHOD
This work involved with two major parts namely destructive sampling of soil moisture and nondestructive measurement of soil reflectance properties under different soil water content. Within the destructive framework, data were collected using a soil moisture sensor in developing a reference data. These results were used to associate with that obtained from optical technique in the development of a prediction model described in the following.

Soil moisture sensor: destructive technique
A soil moisture module (HL-69) is used to provide the value of soil moisture level; an illustration of the destructive system in operation is shown in Figure 1. The power supplied to this sensor is by Arduino ESP8266 (Model no. NodeMCU Lua WIFI Controller Board V3) connected to a computing system. The measurement from this destructive system is to establish the relationship between the amount of water added into the soil and the corresponding moisture level at soil dep th of 1 cm. The selection of this soil depth (i.e. probe penetration) is following the report in Haubrock [13], which demonstrated feasibility of NIR ray to penetrate soil up to a depth of 1 cm for soil moisture measurement. Since this work aims to relate the measurement between the destructive systems with that of optical (NIR), this probe placement depth is considered herein. The sensor used in destructive system in Figure 1 (bottom left) is a 10-bit analog to digital converter device, which output indicates the soil dryness that can be displayed on Arduino's serial monitor. The lower the output value, the higher soil moisture content given by [14]: where represents percent soil moisture, while is the output of the sensor in units of bytes. The result from (1) is used in the development of a look up table before associating it with that from the optical system.

Nondestructive optical based sensing system
The arrangement and use of the NIR reflectance system in this study is shown at the center of Figure 1. This system used two NIR LEDs emitting light of center wavelength, λ, 1200 nm (HSE 1200-L504) and 1450 nm (HSE 1450-L504) for illumination of the chosen sample owing to their high water absorption characteristic [15,16] while detection of the reflected signals is via an InGaAs sensor (HIPD800-A1B). The detected signals are processed for the calculation of soil light reflectance. Similar to that used in destructive operation, this optical system is powered using Arduino ESP8266. The center wavelength of intensity peak of LEDs used in this work was first verified and measured using a power meter (PM100A) as 1100 nm and 1380 nm. The photodiode sensor, placed in between these NIR LEDs, is positioned at 90° angle, 2 cm from the soil sample while the emitters are at an angle of approximately 45º from normal during irradiation. The spatial distance between the photodetector connected in reverse-biased mode and each LED is approximately 1 cm to allow good detection of reflected signals. Based on the measured reference data, light attenuation, A, is calculated as [17]: where and d are white and dark reference, respectively, and is the reflectance from soil sample. All these parameters are in units of volts. The white and dark data from (three consecutive measurement) reflectance of a white paper and for measurement in the dark, respectively, are measured as 145.03±0.23 mV and 0 mV. Using the dark and white reference values, the attenuation of light detected from an investigated sample would be calculated and used for further processing. It must also be mentioned that, since the responsivity range of the detector used is between 800 to 1700 nm, the measurements are minimally affected by surrounding light condition. Thus, experiments on white reference and soil samples were conducted in an enclosed room lighted by fluorescent lamps. During the operation, the sensor output signals would continuously be updated in internet of things (IoT) server (ThingSpeak) at a rate of 20 times per second. The results are recorded and stored in the server, which can be accessed and saved electronically in Excel data file format for offline processing using MATLAB. Figure 1 shows an overview of the use and combination of measurement from destructive and optical systems for near real-time prediction of percent soil moisture. Also shown on the bottom right of this diagram is the employed NN architecture.

Soil samples preparation and measurement
This study carried out experiments on two different types of soil namely loam and peat as they are most commonly used for plantation [18,19]; even though both soil types have high water-holding capacity [19][20][21], the mainr difference between these soils is the different water flow resistance in the medium [22]. The experiment was conducted in laboratory with an average humidity value is 77% and an average temperature of 26 o C. Loam consists of large number of pores with high permeability hence allowing water and air to move easily through the pores [23]. Meanwhile peat has comparatively smaller pore sizes with lower permeability, rendering a slower flow of water through the soil [24]. Since loam and peat provide different absorption strength of water, different addition of water (volume) was chosen in this work. Soil samples (loam and peat) were carefully prepared in round shape containers of radius 0.06 m and volume 588 cm 3 . The plastic container was selected due to its high availability and minimal light reflection during measurement [25]. Water was spread evenly onto the soil sample using a water spray with nozzle diameter 1 mm and at a flow rate of 0.86 mL/s. The containers containing different soil samples of 350 grams were leftover for one minute after the water spraying to allow water flow therethrough before the measurement. This work found the maximum water volume that can be added into the container containing loam and peat (determined through visual inspections of water flooded on the soils surface) is given by 140 mL and 20 mL, respectively. Thus the considered volumes of water added to loam were arbitrarily selected as 20 mL, 40 mL, 60 mL, 80 mL, 100 mL, 120 mL and 140 mL, whereas a smaller water volumes i.e. 2 mL, 4 mL, 6 mL, 8 mL, 10 mL, 12 mL, 14 mL, 16 mL, 18 mL and 20 mL were chosen for peat [26,27]. For each soil, nine consecutive measurements were conducted for each added water volume given rise to 63 and 90 sets of data from loam and peat, respectively. These 153 sets of data were collected using both destructive and optical systems shown in Figure 2 for investigation of changes in moisture level for different amount of water held within the considered soils. These data were used in the training of the NN system described in section 2.4.

Dual-stage multiclass (DSMC) system
Dual-stage, multiclass system is a classifier consists of two independent phases, which information from each component in Figure 1 created a general process model in the prediction, and specifically in classifying instances into two or more classes (i.e. multiclass). The task of classification typically involves two groups and is on the basis of a classification rule. The DSMC system presented herein can be regarded as two phases NN architecture systems correlating measurement from destructive and nondestructive system as shown in Figure 2. The measurement of destructive system (in section 2.1) is included in the training of the DSMC since this technique is able to reveal soil moisture level, which is not feasible with the use of optical technique. In the first stage of DSMC system, regression based fitting approach is used to determine the relationship between soil moisture level and water volume. While former is selected as the input variable the latter is chosen as its target value. In this work 70%, 15% and 15% of data (i.e. 63 and 90 sets for loam and peat, respectively) were randomly chosen for training, testing and validation, respectively. The second stage of DSMC system correlates measurement from nondestructive system (section 2.2) with the added water volume. Similarly, the reflectance signals in voltage (mV) is chosen as input while water volume as the target. By integrating these two above mentioned stages (through their target variables), the trained model would be able to predict both the soil moisture level and soil water content through measurement of reflectance signals (from optical technology). Both stages took approximately 30 seconds each for the training. The final leg of this work is to validate the prediction of soil moisture level using data collected from optical system in near real-time. A summary of block diagram of DSMC system is shown in Figure 2. Figures 3(a) and 3(b) show the percent soil moisture content, σ, measured at 1 cm soil depth for the prepared loams and peats. Referring to these figures, the highest and lowest mean and standard deviation (SD) of σ is recorded as 36.95±1.19% and 0.34±0.11%, and 39.65±0.67% and 0.56±0.20% for loams and peats, respectively. The graphs revealed a relatively linear change in soil moisture level with the water added, wherein larger water content gives rise to higher percent soil moisture measured. Also shown in these figures is the best fitted line for the data for readers' reference. The training of the NN model using these data revealed R and MSE given by 0.9868 and 0.0636, and 0.9939 and 0.0903, for loams and peats, respectively. These trained models (i.e. from the first stage of DSMC) are then used concurrently with the measurements from nondestructive technique as elaborated in section 2.4. show the plots between the amount of water, ℓ, added into the investigated soils and the measured light attenuation, A. Similar to that shown in Figure 3 positive correlation is observed for ℓ and A. These changes in the measured A with soil water content is due to the absorption of light energy by moisture that traps within soil pores, which decreases the detected reflected signals and hence an increase in the calculated light attenuation. In addition to the medium's light absorption, soil surface color and texture would also affect the measured attenuation, wherein darker soil surface (such as that of the peat shown in Figure 2 renders a reduced light reflection. This, therefore, explains the similarity in the magnitude of the measured light attenuation for loams and peats (eventhough under different water content level) in Figures 4(a) and 4(b). The highest and lowest mean ± standard deviation (SD) attenuation value given by the optical system is measured as 1.5769±0.0337 optical density (OD) and 1.1830±0.0691 OD for loams in Figure 4(a) whereas Figure 4(b) showed 1.4261±0.1004 OD and 1.0410±0.0321 OD for peats; the greater range observed for loams (before the saturated water uptake) as compared to that of the peats can be attributed by water holding capacity. Water holding capacity is the amount of water that can be physically hold by soil for crop use, which properties are controlled by its texture, composition and amount of organic matter it contains.

RESULTS AND DISCUSSIONS
An investigation is carried out to correlate the results from the Figures 3 and 4 to better understand their relationship. Since the amount of water added into the soils is made consistent for both experiments, percent soil moisture (from destructive experiment) is plotted against the measured light attenuation (from optical experiment) as shown in Figure 5. Eventhough the plots are of generally linear relationship, albeit with some outliers, with correlation values given by 0.9729 and 0.9515, respectively, for loams and peats, it is interesting to note on the high similarity in their functions (i.e. fitted lines). These considerably high consistencies in their light attenuation and percent soil mo isture ranges may result in discernible NN in distinguishing the soil type. The sensing technique used in destructive and optical systems are based on changes in resistance and light reflectance with soil moisture level, and the results and trends recorded in Figure 3 and Figure 4 agreed reasonably well with that reported in [28,29]. The result in Figure 5 revealed that, although both destructive and nondestructive techniques adopted different working principle, both the techniques showed a significant positive relationship between the water volume added and their measurable. This is with exception for measurement performed under lower water content using destructive technique, in which notable outliers or high consistency can be observed for both soil samples in Figures 3(a) and 3(b) (i.e. ℓ of less than 40 mL in loams and 6 mL in peats). This observation is not apparent for measurement conducted using optical approach in Figures 4(a) and 4(b), hence the reason of these outliers could possibly be due to the measurement sensitivity of the sensor used in destructive system. This may also be compounded by the low amount of water introduced into the medium, which is insufficient to reach the sensor for measurement of the soil dielectric permittivity.
It was discussed in section 2.3 that both the considered soils have different water holding capacity, rendering different water uptake in the medium. Since water is trapped within the peats for a longer time than that of the loams, plateaued values are observed under ℓ value of as low as 18 mL in Figures 3(b) and 4(b). For the same reason, eventhough peats have darker color that may increase light attenuations, higher water content (i.e. higher water volume addition) in loams contributed to higher measured attenuations compared to that of the peats in Figure 5.
Since the measured σ and A are not able to show any significant changes beyond ℓ of 18 mL in Figures 3(b) and 4(b), this yields fluctuations in relationship between σ and A in Figure 5 (i.e. σ value in between 38-40% while A in the range 1.4-1.45 OD). The results highlighted (as transparent turquoise color) in Figure 5 revealed that light attenuation measured for peats and loams under optimum moisture condition ranged in between 1.25-1.4 OD; this finding can loosely be used as an indicator for efficient agricultural water management.
While the training and validation of first stage of DSMC system revealed R and MSE of 0.9868 and 0.0636, and 0.9939 and 0.0903, for loams and peats, respectively, second stage showed R and MSE of 0.8817 and 0.8881, and 0.7898 and 1.172 for these soils. These poorer performances in the validation of second stage DSMC system could be contributed by higher inconsistencies in the value of the measurable (by optical system) shown in Figures 4(a) and 4(b). Since measurement of white reference showed remarkably consistent reflectance value, i.e. SD of 0.16-0.39 mV, the fluctuations in the value in Figures 4(a) and 4(b) could be due to differences in soils particle size and surface roughness. Different soil texture and particle sizes (loams composed of particles of size around 0.06 mm, while 5 mm for peats) lead to changes in light scattering and hence the detected reflectance [30,31].
Meanwhile the use of trained neural net model on samples that have no role in the training of the model has been used to evaluate the performance of the developed optical system and prediction model. There were 50 samples comprised of peats added with ten different volume of water (i.e. 2 mL, 4 mL, 6 mL, 8 mL, 10 mL, 12 mL, 14 mL, 16 mL, 18 mL and 20 mL). Each amount of water was added into five prepared soils give rises to 50 prepared sets. Meanwhile in the case of loam, seven different volume of water (i.e. 20 mL, 40 mL, 60 mL, 80 mL, 100 mL, 120 mL and 140 mL) were added into the corresponding soil. Similarly, each amount of water was added into seven different prepared soil, this is with exception to water volume 20 mL, which was introduced into 8 soil samples. The results revealed a relatively good accuracy of 92% and 82% of the prediction measurement for loams and peats, respectively. The inadequacy in the performance of the model for measurement on peats is likely owing to high similarity in the value of the measured reflectance under the added water volume ranged between 4 mL and 12 mL. The time required for the measurement from optical technology to prediction of soil moisture level is approximately 45 seconds. The prediction result from second stage DSMC is considered relatively good result while compared to Li et al. [32] who obtained the accuracy prediction value of 90% by using spectral reflectance and Cai et al. [33] who used NN regression model to predict soil moisture result with accuracy of 92.56%.
It must be emphasized that the results and performance of DSMC presented herein are only valid for the soil types and preparation procedures described above. A revised procedure of soil preparation, water addition and NN system were to be introduced should investigations be conducted on other soil type. The limitation of the project is the nondestructive measurement is only can measure the soil surface but it is limited for further deeper penetration of the illumination measurement.

CONCLUSION
This study attributes the poorer performance of the optical system on peats to the detection resolution of percent soil moisture gathered in destructive experiemnt, and structure and properties of the corresponding soil. This study concluded that the developed system is able to provide nondestructive monitoring the soil moisture condition, for agricultural water management and to increase the productivity. This system would reduce the cost of water irrigation through the constant and real time detection of the soil moisture level without damaging the soil structure.