An assessment of stingless beehive climate impact using multivariate recurrent neural networks

ABSTRACT


INTRODUCTION
Stingless bees (Meliponies) are eusocial insects like honeybees (Apis), which produce honey, propolis, and beeswax [1].Over 600 stingless bee species with 56 different names of genera live in tropical and subtropical areas of the world [2].Meliponiculture is stingless beekeeping with supportable activities suitable for honey production and keeping excellent pollinators of the environmental and agricultural fields [3].An example of issues studied in the area of beekeeping includes the foraging activity pattern [1], [4], the pollinations factors and effects [5], the environmental influences [6], beekeeping and rearing techniques [7]- [9], and the quality of honey and propolis production [10].The increase in worldwide temperature may uncomfortably pressure the colonies while residing in their habitat.To prevent overheating, bee colonies attempt to regulate the internal hive temperature between 33 °C and 36 °C [11].High temperature may lead Int J Elec & Comp Eng ISSN: 2088-8708  An assessment of stingless beehive climate impact using multivariate … (Noor Hafizah Khairul Anuar) 2031 to situations includes colony death [11], swarming event [12], [13], low quality of honey [14], [15], and colonies disappearance [16].Temperature and humidity are the two crucial determined parameters to analyze the beehive, as well as hive weight and foraging activity data to resemble colonies' health level [17]- [22], swarming events [15], [23]- [25], and yields productions status [26], [27].
The neural network (NN) approach uses computational methods to learn the information directly from massive data without relying on a specific equation.For example, a study [28] used a NN on environmental data, hive temperature, and weight to forecast the beehive health status in beekeeping.The study [28] indicates an imminent collapsing state for beekeepers using classification algorithms such as k-nearest neighbors (k-NN), random forest (RF), and NN.The approach, like support vector machine (SVM) and machine learning methods, was used in the detection of the queen bee state condition as reported in [28], [29].In addition, Arruda et al. [30] applied an algorithm (RF, multilayer perceptron (MP), and support vector machine (SVM)) to the foraging pattern data from radio frequency identification (RFID) system to classify the stingless bee species.In [31], a fuzzy logic approach is used on temperature data to identify the honeybee hive state, either health, death, or weak.
Recurrent neural networks (RNN) add feedback within the neurons that allows the network to handle the variable-length sequences.RNN presents the ability to store internal memory to manage dynamic temporal behavior naturally.In [32], image processing through the fast-region-based convolutional neural networks (fast-RCNN) approach amounts to the pollen carried by the honeybee forager into the hive.As an improvement from the RNN concept, Hochreiter and Schmidhuber [33] introduced long short term memory (LSTM) cells that benefit from long term dependencies.Kachole et al. [34] used image processing to count the bee traffic data at the hive entrance and forecast the activity and health using the LSTM method.Meanwhile, the gated recurrent units (GRU) and LSTM approach are used in [35] to predict the bee activity levels through activity data and environmental parameters.
This research paper focuses on a method/approach for processing and analyzing stingless beehive climate data impact to forecast the beehive yields.The deep learning RNN approach was compared using LSTM and GRU to establish the best topology.Several forecasting window sizes have also been tested to improve model performance for future development.The forecast output assessed in this paper is the hive weight concerning the other environmental parameters (temperature, humidity, and sum of incoming and outgoing bees at the hive funnel).The output helps beekeepers estimate the yields produced and colony growth.The hive weight forecast output helps to determine the best time range for harvesting and the process of colony propagation (to separate the colonies if an overcrowded situation happens).On the other hand, the study may determine the correlations between observed parameters that most influence the colony's health.The suggested model also predicts unusual symptoms, which assists beekeepers in determining appropriate actions for their hives to improve the colony's propagation process.

RESEARCH METHOD 2.1. Data collection
Data collection using wireless stingless bee management and monitoring system for ~36 h from Dec 7, 2018, to Dec 9, 2018.In this paper the data stamps are set up at intervals of seven seconds.The monitoring system consists of a weight sensor (load cell), temperature and humidity sensors (DHT22), and bee counter controlled using a NodeMCU microcontroller.The bee counter system used two pairs of infrared sensors for counting the incoming and outgoing bees through the hive entrance funnel.The data from a microcontroller were pushed to Firebase's real-time database and can be accessed through web or android applications.The technical detail of the monitoring system is as discussed in previous works reported in [36].The data were continuously collected in real-time and saved to google firebase's real-time database.The extracted dataset is from a single hive of Heterotrigona itama.Figure 1 shows the overall architecture of the monitoring system used for this study.

The input variables
The sensors system observed the hive weight in kg, inside and outside hive temperature and humidity in °C and % respectively, and bee traffic at hive entrance (total number of bees going in and out from funnel).The bee counter consists of 2 pairs of infrareds (IRs) emitters and photoresistors.The bee counter system recognizes a departing bee when the first pair of IR is detected, followed by the second IR pair.Then, the value "1" is added to the current total bee count in the programming by the microprocessor.Meanwhile, an arriving bee is detected when the second IR pair is detected, followed by the first IR pair, and then a value of "-1" is added to the current total count.
Dewpoint temperature is a calculated variable accesses in this project.The dewpoint is the temperature at which the air is the most saturated.The inside hive relative humidity (RHin) and the inside hive temperature (Tin) need to be determined to calculate the dew point.Condensation occurs when the inside hive temperature is below the dewpoint (Tdp).Depending on the surface types, condensation is estimated to occur at |Tdp -Tin< 1.5 °C | [37] for the greenhouse area and at |Tdp -Tin< 3 °C| [38] for the standard building area.The dewpoint temperature was calculated using the inside hive temperature and humidity data [39]; as represented in (1).
where RH is inside hive relative humidity in %, Tin is inside temperature in °C, and Tdew is dewpoint temperature in °C

Data preprocessing
In this project, the scientific computing and data analysis process was performed using Numpy and Pandas libraries in Python software.The main subprocess involved data cleaning, data normalization, supervised learning data frame, segmentation, and network design using LSTM/GRU units for training and testing.Firstly, the data cleaning process involves cleaning, indexing (drop the date and time column), resizing, and searching for the missing data.Next, data normalization was performed.This process is essential in stabilizing the supervised learning process by ensuring the input and output ranges are of the same scale.The process rearranges the entire variable data range between zero and one by using the MinMaxScaler features of the scikit-learn library.The normalization scale was interpreted by (2).
where i is the number of variables, k is the number of data (index), and   * () is the sequence after the normalization process.
Supervised learning is a key component of machine learning and data mining [40].After the normalization process, the generation of a supervised learning data frame took place.One-dimensional vectors were alternately applied as an output.This project evaluated variables including hive weight and bee count as outputs, run one by one, rather than simultaneously.The chosen output variable is to compare the forecast performance and relevance between both variables.In contrast, the rest of the variables or vectors were set as the influenced input variables (temperature, humidity, dewpoint).Next, the data frame was segmented into train and test sections.In this experiment, the train and test sections were segmented approximately by 75% and 25%, respectively.The data samples to be processed in the train and test sections were 2,500 and 464, respectively.The next steps are to manipulate the number of neurons in the hidden and output layers and a loss function, and an optimizer were determined.In this work, the model is evaluated using the root mean square error (RMSE), which can be obtained from (3).
where   is the actual output,  ̂ is the output forecast, and n is the size of the test dataset.This study forecasts a single variable, influenced by the seven input variables.Finally, the training and test losses were tracked by setting the validation data argument in the fit() function.The training, test loss, and correct and predicted outputs were plotted at the end of the process.

The model
The LSTM and GRU [38] networks are superior RNNs that have the competencies for learning long-term dependencies [37].The Keras model implements multivariate time series forecasting using LSTM/GRU units.Keras is a high-level neural network API written in the python platform and capable of running with several backend packages such as Tensorflow, cognitive toolkit (CNTK), and Theano.In the process of finding the best RNN topology for hive weight and bee activity forecast, six different topologies considered a different number of the hidden layer and neurons.The designed network utilizes the Keras sequential model The constructed topologies included one hidden layer with thirty-two recurrent units in each layer (1×32), one hidden layer with fifty recurrent units (1x50), two hidden layers with thirty-two recurrent units in each layer (2×32), and two hidden layers with fifty recurrent units per layer (2×50).Hence, the six topologies models are: LSTM 1×32, LSTM 1×50, LSTM 2×32, LSTM 2×50, GRU 2×32, and GRU 2×50.The input shape is a single timestep with seven features, and the batch size is set as 32.The loss function and optimizer used are the mean absolute error and the efficient Adam version of stochastic gradient descent, respectively [40].The output layer consists of a dense, fully connected layer to compile the forecast output.The example of potential architectures for a single hidden layer with n-numbers of neurons is shown in Figure 2. In this illustration, seven input variables (at t=0) and one output variable (at t=t+1) are forecast.The evaluated forecast output will be hive weight or bee activity that happens independently, and the unit of the hidden layer for this investigation is LSTM and GRU

The multiple input vector window size
The previous evaluation sought to identify the best RNN topology for the data.The best topology from the previous research was employed in the second inquiry, which was aimed at determining the optimal input vector window size.There are four types of window size was chosen to be investigated which are {w1=t 0 -1}, {w2=t 0 -2}, {w3=t 0 -9}, and {w4=t 0 -14}.Referring to Table 1, t 0 denotes the current time, t 0 -1 denotes one minute before the event to be predicted, t0-2 is for two minutes before the event forecast, t0-9 is for nine minutes before the event to forecast, and lastly t0-14 is for fourteen minutes before the event to forecast.In this study, the optimal architecture discovered in the previous stage was used to analyze alternative widths for the input window, resulting in varying quantities of preceding data being used to anticipate the next level.It necessitates the RNN's capacity to retain useful information over time.The evaluations were conducted one minute, two minutes, nine minutes, and fourteen minutes before predicting the event.

Statistical exploration
The initial analysis using basic statistical techniques was performed to familiarize with the datasets.Then, the mean, standard deviation, maximum and minimum value, and interquartile range were compute.Table 2 represents the total, mean, standard deviation, minimum and maximum count of the dataset.The total data were 2,965, consisting of the hive (Kg), inside and outside hive temperature (°C), inside and outside hive humidity (%), the dewpoint (°C), and the bee count.In addition, the data show a huge variable range where the minimum is 16, and the maximum is 21,282 for the bee count variable.

Correlation analysis
A correlation analysis is performed on each observed variable to determine the relationship between them in the first evaluation.Based on the correlation analysis result, the beehive weight as an output variable shows a strong correlation with the inside temperature, outside temperature, and dewpoint variables (approximately 70%).The study also indicated that, if bee count is considered as the output variable, other examined variables have a weak relationship (less than 20%).Even though the bee count variable was also being run as an output on the second investigation, to compare between forecast output from hive weight and bee count.Within all the access variables, the inside humidity has the weakest correlation with other vectors.Figure 3 represents the features correlation heatmap for all observed variables.The blue region on the heatmap denotes a positive correlation, implying that the cross-correlation variables are proportional to each other.The red region, on the other hand, denotes a negative correlation, which signifies that all crosscorrelation variables are negatively proportional to one another.Brighter areas are considered natural correlations, whereas darken areas are considered a strong correlation.The RNN model evaluations in this study have considered all observed variables as input vectors.

RNN topologies
The second evaluation aims to investigate the best topology of RNN models for forecasting the selected variables (hive weight and bee count) based on all observed variables as input vectors.Initially, the first experiment forecast the hive weight variable, the designed network was trained and evaluated with 50 times epoch, and a batch size of 32.In the second experiment (bee count variable), the epoch evaluation is set to 200, and the same batch size of 32 for network design.The bee count data consists of a massive range for the maximum-minimum value (from 16 to 20,000) and require more epoch evaluation to stabilize the output fluctuations.
The most significant model found in the investigation of the hive weight forecast is LSTM 1X50 with the lowest RMSE (0.012).Bee count variable represents the high value of RMSE output (≈ 74 − 160).

2035
In the first evaluation, bee count shows a weak correlation with other access vectors as well as the input data range consists of a massive maximum and minimum peak.Figure 4 presents the error RMSE for all experimented network topologies for hive weight forecast.Meanwhile, Figure 5 shows the RMSE output graph of train vs. test for a hive weight forecast using the LSTM 1×50 topology.The forecast output fluctuation is stable approximately at epoch 50 times evaluations.

Investigations multiple window size on beehive weight forecast
The final evaluation seeks to establish the most appropriate size for the input window.The output of beehive weight forecast based on all input attributes using RNN topology of LSTM1X50 was used since this topology is the best found in the previous evaluation.Table 3 shows the first 5 rows of the supervised learning data frame for the window size of 1 minute before the event forecast (1 =  − 1).From Table 3, the data were grouped into an 8-column table, with the first and seventh columns representing the RNN input vectors and the eighth column representing the anticipated output.The var1 represents the value of hive weight, while the var2 until var7 represents the value of inside temperature, outside temperature, outside humidity, inside humidity, dewpoint, and bee count consequently.The window size of w1 (1 minute before the event forecast) resulted in the best RMSE of 0.012.Figure 6 represents the RMSE value obtained after evaluation for 50 times epoch for each different forecast window size.The larger the window, the more time-series information can be considered.It may, however, reduce the system's sensitivity, resulting in overly smooth forecasts.In our opinion, it is critical to find the best window size that corresponds to the fractal data dimension.

CONCLUSION
This study proposed a forecast model for stingless beehive weight output.The observed variables (features) are the internal hive temperature, internal hive humidity, outside hive temperature, outside hive humidity, dewpoint temperature, and bee count.The investigated models were run by using the Keras library in Python software.All features were tested for their significance using the correlation matrix method.
In future works, it is recommended to vary the dataset season for example data in sunny, rainy, windy, fruity, or flowering.Hence, different locations and landscapes of the dataset such as urban, rural, highland, and lowland could also be considered.It is also recommended to investigate other potential attributes such as light intensity, solar irradiance, gas content, and air pressure.

Figure 1 .
Figure 1.The general architecture of wireless stingless beehive monitoring system Int J Elec & Comp Eng ISSN: 2088-8708  An assessment of stingless beehive climate impact using multivariate … (Noor Hafizah Khairul Anuar)

Figure 2 .
Figure 2. One of the developed topologies consists of n-neurons grouped in a single hidden layer

Figure 6 .
Figure 6.The RMSE of different window sizes for hive weight forecast

Table 1 .
Evaluated input vectors structure of hive weight forecast

Table 2 .
The statistical description of the datasets

Table 3 .
The example of the first 5 rows of an input-output data frame for the beehive weight variable forecast