Neural Network Model Development with Soft Computing Techniques for Membrane Filtration Process

Received Feb 10, 2018 Revised May 28, 2018 Accepted May 7, 2018 Membrane bioreactor employs an efficient filtration technology for solid and liquid separation in wastewater treatment process. Development of membrane filtration model is significant as this model can be used to predict filtration dynamic which is later utilized in control development. Most of the available models only suitable for monitoring purpose, which are too complex, required many variables and not suitable for control system design. This work focusing on the simple time seris model for membrane filtration process using neural network technique. In this paper, submerged membrane filtration model developed using recurrent neural network (RNN) train using genetic algorithm (GA), inertia weight particle swarm optimization (IWPSO) and gravitational search algorithm (GSA). These optimization algorithms are compared in term of its accuracy and convergent speed in updating the weights and biases of the RNN for optimal filtration model. The evaluation of the models is measured using three performance evaluations, which are mean square error (MSE), mean absolute deviation (MAD) and coefficient of determination (R2). From the results obtained, all methods yield satisfactory result for the model, with the best results given by IW-PSO. Keyword:


INTRODUCTION
Membrane filtration is a promising technology in many separation and purification process. This technology is able to achieve high performance in term of permeate quality. The application of membrane filtration is widely applied in many industrial applications such as water treatment and wastewater treatment process, food processing, desalination, medical application and many more. Despite the advance in the current technology, membrane filtration still experiencing numerous drawbacks in terms of fouling, high energy consumption and a significantly higher operational cost compared to conventional techniques [1].
To improve the membrane filtration process, one of the options is to obtain accurate modeling of the filtration process. With the obtained model, an accurate prediction can be made for future action of the process in order to improve the filtration performance of hydraulic cleaning involving aeration airflow, backwash, relaxation and chemical cleaning. From the obtained model, the best setting for the filtration permeate flux set point and the transmembrane pressure (TMP) dynamic behavior can also be determined. However, developing a working model for membrane filtration process is not an easy task, due to the nonlinear chaotic behavior of fouling. In literature, several mechanisms were developed to obtain an optimum filtration model. Darcy's laws for example, several terms of fouling resistance are added in series to represent the fouling layer in the membrane filtration process to develop the filtration model. Similar to other mechanistic or first principle model, tedious calibration process involving many parameters is required. An alternative technique to represent a dynamic behavior of the filtration process is using artificial neural network (ANN), which has high accuracy of prediction capability. The ANN technique is employed in numerous areas including chemical process, mechanical applications, financial forecasting and trending, environmental prediction and others. On the area of membrane filtration modeling, Geissler et al [2] has developed models which are semi empirical model and ANN based model for permeate flux modeling in submerged capillary MBR. The ANN model is based on Elman neural network structure to predict the permeate flux. The techniques yield very good results, where the semi empirical model required small input variables compared to ANN. However, the ANN model gave high accuracy with the average error of 2.7%.
In [3], a flat sheet submerged membrane bioreactor (SMBR) filtration for wastewater filtration is modeled using ANN model to represent the backwash effect to the permeate flux. Multilayer feed forward neural network is used to model the system. Various backwashed and filtration interval are used as the input to the model, meanwhile flux is used as the output. With the given backwashed to permeate ratio, the model is able to predict the permeate flux performance. Nevertheless, the application of ANN using conventional back propagation (BP) algorithm for training has faced several problems such as slow convergent and the algorithm has tendency to tarp in the local minima. Thus, the application of heuristic search optimization technique is one of the solutions.
Several works has found in literature to find an optimal weight and bias value in the training of the neural network. Among the widely used heuristic search algorithm for ANN model is genetic algorithm (GA).this algorithm is among the pioneer in heuristic optimization which is inspired from the population evolutionary theory. GA was successful used in [4], [5] and [6] to train the neural network model in various applications. This algorithm is able to produce a global solution for a given optimization problems. Particle swarms optimizations (PSO) in particular, have become a very effective optimization technique in various fields. The algorithm is fast and very reliable in searching for minimization. Several well known advancement of PSO algorithm such as random PSO (RPSO), constriction factor PSO (CPSO) and inertia weight PSO (IW PSO) have been developed throughout the years. These algorithms have been tested in previous neural networks application as reported in [7], [8] and [9]. Gravitational search algorithm (GSA) [10] is another type of optimization technique. This method is inspired by the law of gravity and the mass interaction in universe. The GSA was reported to yield faster convergent rate and is reliable to meet the objective function in many applications as reported in [10]- [12].
This work focuses on model development of membrane filtration process using recurrent neural network (RNN) model train by three type's of soft computing optimization techniques which are GA, PSO and GSA. These three algorithms will be used to search for the best weights and biases of the recurrent neural network model. The trained mode is then compared in term of its accuracy on the training and testing of membrane filtration data set.

EXPERIMENT SETUP
The data set is collected from the developed membrane bioreactor pilot plant located in Process Control Lab, Faculty of Electrical Engineering, Universiti Teknologi Malaysia (UTM). Random magnitude steps input were given to the suction pump to stimulate the dynamic behavior of the process. Figure 1 shows the plant schematic diagram while Figure 2 shows the data collected from the pilot plant. The experiments were carried out in single tank submerged membrane bioreactors, with working volume of 20 L Palm Oil Mill Effluent (POME) taken from Sedenak Palm Oil Mill Sdn. Bhd. in Johor, Malaysia. The aeration during filtration is set around 6 to 8 litter per minute (LPM) to maintain the dissolved oxygen level to be more than 2 ppm during the experiment. In this work, voltage input of the permeate pump is used for the input, mean while the flux and TMP are the measurement outputs.

MODEL DEVELOPMENT
Recurrent neural network (RNN) model is a mathematical model developed based on the past input and past output of the system. The RNN is developed based on the basic feed forward neural network (FFNN). The training algorithm is employed to obtain suitable weights and biases of the network in order to minimize the error in the training procedure. Figure 3 shows the basic structure of the neural network model. The neural network equation can be represented as: where ̂ ( )is the prediction output. F i is the function of the network, φ is the input vector, W ij and w ij represent the network connection layer weights and biases respectively. In this work three and five hidden neurons are tested with basic three layers neural networks which are the input layer, single hidden layer and output layer. This work employed nonlinear autoregressive with exogenous input (NARX) ANN structure which shows in Figure 4. The optimization techniques applied in the ANN model training is optimize the weights and biases of the ANN structure by minimizing the mean square error (MSE) of the model

Genetic algorithm (GA)
Optimization algorithm is a developing area in the soft computing study. GA algorithm is one of the pioneers in evolution based optimization techniques and is a reliable technique in optimization. This optimization technique is widely applied in many industrial applications such as in [13] and [14]. The flow chart of the algorithm can be represented in Figure 5.

Particle swarm optimization
Particle swarm optimization (PSO) is inspired by behavior of a group of animals hunting. This heuristic search optimization is very effective in finding optimal solution for many problems. In PSO algorithm, number of swarm must be selected to search for the solution. Each swarm contains an individual calls a particle. The PSO main algorithm is to update the position of each particle with an estimated velocity. Each of the components of the velocity equation represents the exploration ability and capability of individual learning as well as social learning. The PSO algorithm execution flow chart is shown in Figure 6.

Gravitational search algorithm (GSA)
The GSA algorithm is an effective optimization algorithm developed in [10]. It is inspired from gravity law and mass interaction in the universe. Figure 7 presents the flow chart of the GSA optimization algorithm. The details development of the algorithm is presented in [10].

RESULTS AND ANALYSIS
Generally, all the training methods using soft computing techniques are able to produce acceptable result for the model obtained. From the training result it shows that PSO algorithm performs much better when compared with GA and GSA. From Figure 8(a) it can be observed that GA has a much faster searching for the right weights and biases of the neural network compared with the other. However, when approaching the first 50 iterations, PSO shows superiority in finding global minimum from the given cost function. Meanwhile, Figure 8(b) shows the zooming convergent area of the training algorithms. In term of modeling performance criteria, the PSO algorithm performs better compared with the other methods.    The validation of the model is done by using data not utilized for the model development. From the result given in the Table 2(a) and (b), it is proven that the training performances of all the algorithms are able to model the system. Generally, the testing result is less accurate compared with the training. However, from the testing performance it showed that the models are able to replicate the dynamic of the membrane filtration system as shown in Figure 10.  The result of the increment of the hidden neurons of the neural network has shown significant difference from the initial (three hidden neurons) training. It is an interesting finding where some of the training algorithm shows improvement of the model while the other is not able to find suitable weights and biases of the neural network model until the end of the iterations. Figure 11(a) shows the performance for each of the training methods in term of it fitness and iterations. Figure 11(b) shows the zoomed image at the convergent area.
(a) (b) Figure 11. (a) The performance of the training algorithm for five hidden neuron (b) Zoom at the convergent area of the training algorithm for five hidden neuron The overall of the modeling performance for the training result is shows in Table 3(a) and Table 3(b). Permeate flux modeling had shown the PSO and GA can be used for the RNN training. Figure 12(a) and Figure 12(b) represents the training result for the permeate flux and TMP using the PSO-RNN modeling technique.
Generally, the testing results have shown that the result is much improved compared with the three hidden neuron. The PSO and GA training algorithm are able to maintain the accuracy of the model with minimum error increment compared with the training dataset. The testing result reveal similar trend of accuracy with the training result where the PSO algorithm obtains higher accuracy of the model followed with GA and GSA. Table 4(a) and Table 4(b) present the testing performance of the models with the %R 2 , MSE and MAD performance. The testing result for the permeate flux and TMP were shown in Figure 13(a) and Figure 13

CONCLUSION
This paper presents the dynamic modeling technique for submerged membrane filtration process using RNN with various soft computing techniques. From the result it showed that the training algorithm using soft computing technique is able to produce high accuracy of RNN model. This work compares optimization algorithms of GA, PSO and GSA method to train the RNN model. It is observed that the PSO algorithm is able to gives higher accuracy compared with the others followed with GA and GSA. The performance of the PSO and GA algorithm are almost similar especially for the TMP model. Increment from three to five hidden neuron can increase the accuracy of the model, however, it is can also be troublesome for GSA algorithm to find an optimum weights and biases value for the model. Finally this model will be useful in order to do prediction performance and controller design strategy in the future.