Generative adversarial deep learning in images using Nash equilibrium game theory

ABSTRACT


INTRODUCTION
The perturbations are created to deep learning (DL) models to make the original image perturbed and resulted in an adversarial image that deceives and confuses the DL model.The DL model got more secure and alerted to adversarial attacks by training the model with creating perturbations on the original image.A generative adversarial network (GAN)-based on the DL model was presented to overcome the problem of the low-quality image which causes poor performance.The low-quality defect images are reconstructed by using the GAN model, and to recognize the reconstructed images a VGG-16 network is developed [1].The GAN model is mainly used to differentiate between fake and real samples to improve the performance of the model [2].The process of capturing high-dimensional images was complex in the steganography task hence the AdvSGAN which evaluates the restricted neural coder represents the steganography of images by performing an adversarial game between the adversary model and neural code.The adversarial GAN networks provide high performance in steganography tasks [3].The GAN based DL methods is also used for tumor classification in magnetic resonance (MR) images to extract the related features and structure of MR images known by the convolutional layers to improve the performance in data augmentation of medical MR images [4].The quality of the image and the accuracy of computed tomography (CT) of cone-beam computed tomography (CBCT) is improved using DL methods with GAN networks [5].The automated medical diagnostics of brain images is performed by a novel method to know high-resolution generative models of brain magnetic resonance imaging (MRI) images using the texture transformation and deformation field together with deep neural networks [6].The abnormal-to-normal translation GAN (ANT-GAN) model is used in medical images which consist of medical imaging information like classification and lesion segmentation to produce the improved lesion image [7].Medical images are timeconsuming and expensive hence to reduce them and to exploit the required information, a novel semisupervised DL method that trains the adversarial images itself with specific regularization is presented for medical images in large-scale classification [8].
The DL models mainly depend on the number of training samples which affects the performance of the model hence a multitask generative adversarial network (MTGAN) to extract the highly required information from unlabeled data.The MTGAN performs the classification and reconstruction tasks [9].The DL models do not give suitable solutions for sophisticated adversaries in image transformation due to their non-differential nature, and filtering of orientation hence non-deep learning approaches are presented.The non-deep learning approach performs well in image transformation like discrete sine transform (DST) and supports vector-based classifiers [10].Due to the limited availability of labeled samples, traditional deep learning (DL) algorithms were not appropriate.As a solution, a feature-oriented adversarial active learning (FAAL) approach was developed to extract high-level features from an intermediate layer of a DL classifier.These features were then used to design a heuristic-based GAN algorithm.The fake features are evaluated and differentiated between the features of real and fake [11].DL algorithms have shown impressive performance in various machine learning tasks but they are vulnerable to adversarial attacks, particularly in the form of adversarial images.To address this issue, experts have suggested a new approach called the spatial-frequency ensemble relation network based on GANs.This approach aims to improve the performance of DL models against adversarial attacks on images.The ensemble relation network extracts the features of training images, extracts the relation between images, and transforms the relationship into related categories by using GAN [12].The representation of features for targeted images was performed by  3  by using channel-wise attention and geo-metric attention to result in improved performance [13].The effectiveness of traditional image segmentation is performed by a novel two-stage image augmentation architecture which results in the synthetic mask and image pairs.The effectiveness of image segmentation is improved by increasing the size of the training dataset in synthesized image mask pairs [14].The Nash equilibrium game theory has been applied to Canadian Institute for Advance Research (CIFAR) dataset in the research, to implement generative adversarial learning algorithm (NEGT-GAL) on images to overcome adversarial data manipulations [15].
Yang et al. [16] presented a network security task method depending on adversarial DL.The deep auto-encoder-deep neural network (AEDNN) method was proposed by using deep auto-encoder (DAE) for feature extraction and deep neural network (DNN) for binary classification of attacks.AEDNN can manage huge amount of network data and the efficiency, robustness, and generalization of the network was improved.The limitations of AEDNN consists of imbalanced data of various categories in dataset and accuracy need to be improved in finding the minority classes.Further, the calculation method of security needs to be optimized and need to validate the model in detail.Jeong et al. [17] presented the accuracy for classification model using DL and decreased the malicious attackers.The Modified National Institute of Standards and Technology (MNIST) dataset with image samples and NSL-KDD dataset consists the data of network was taken as the input datasets.The accuracy was calculated by giving the adversarial samples into convolutional neural network (CNN) and auto-encoder classification models which were designed using the libraries of PyTorch and TensorFlow.The exploited adversarial samples consist of insufficient data can cause to a great damage to the performance of model.Further, the accuracy was improved by applying the learning method of recurrent neural network (RNN) and deep fool method.Ma et al. [18] presented the analysis of issues on adversarial attacks on DL depending on medical image analysis.The experiment was conducted by four various detection methods and attacks on three medical image datasets.The DNN models of medical images more vulnerable for adversarial attacks compared to natural images.The medical adversarial attacks detection is simple and can evaluate 98% detection accuracy.The wrong decision in medical images leads to difficulties in finding the adversarial attacks on medical images compared to natural images.Further, defense approaches need to improve the robustness of adversarial medical images.Huang et al. [19] presented the synthetic aperture radar (SAR) image recognition methods depended on DL models.The adversarial examples of SAR images are generated which was used to attack the classical DL models.The performance of the adversarial attacks was tested using SAR images and the advantages evaluated are automatic train classifiers and learning features which improves the performance of image recognition methods.The drawbacks of SAR image recognition were overfitting and it was not suitable for other attack models except iterative least likely class method (ILCM) algorithm.

6353
Mockingbird in machine learning domains related to adversarial attacks of website fingerprinting (WF) which was a traffic analysis attack, improves the accuracy up to 98%.Initially the straight forward techniques were used to save the traces against adversarial attacks but it does not give robust classification.Hence the Mockingbird technique evaluated for traces and gives adversarial training by moving in the space of traces where the predictable gradients are not followed.The main advantage of it is having high security of the data against adversarial attacks.There are limitations associated with high computational time as well as the rapid transmission of large amounts of information.Further, leverage Mockingbird for server-side and make it more robust against other models.According to Pasini et al. [21], deep convolutional (DC-GANs) model training could be distributed.The task distribution relies on categorizing the data, and it is decided by the qualities that differ between the classes of data, which take precedence over the features that differ between data points of identical classes.The pattern of the wall-clock time shows that the training of distributed DC-CGANs on all four datasets resulted in less scaling as long as the computational workload for every message passing interface (MPI) procedure stays unchanged.Because, the computational time required to complete the training was not affected by the increase in the number of data classes.Meanwhile, the distributed DC-CGANs produced results that were almost relevant and already included in the CIFAR10 dataset.A distributed method to train DC-CGANs models has been shown by Pasini et al. [21].By dividing the training data into groups based on data labels, this strategy lessens the disparity among the generator and discriminator and improves scalability by doing parallel training on many generators, each of which was trained with a single data label in mind.The variation across classes is eliminated during data splitting based on labels, which also corrects the imbalance in normal GAN training.Since everyone is independent of the others, the generators were trained simultaneously, which improves scalability.Quicker data processing and steady training may not constantly translate into faster convergence.Karim et al. [22] presented an adversarial transform network (ATN) model to attack different time series classification models and a distilled model was used to observe the behavior of classification models of time series.The student-teacher framework was used in proxy attacks on a specific model.The capability of generalizing adversarial models performed well on the samples which do not see before by the adversarial models.The disadvantage of ATN was not suitable for time series classification models which influences model robustness and performance evaluation.Further, time series classification models need to be developed for the targeted adversaries.
The main contributions involved in this research are given as follows: i) a Nash equilibrium game theory (NEGT) is used to attain the equilibrium condition as a result, of more similar manipulation images; ii) a generative adversarial learning (GAL) algorithm is used to generate additional data which helps to improve the training data; iii) also, the wall clock time is reduced by generating the additional data using the GAL algorithm.Moreover, the structure of this paper is given as follows: the overall concept of proposed methodology and the mathematical equations for NEGT are explained in section 2 and section 3. Whereas the experimental results of the proposed methodology are presented section 4. Finally, the conclusion of the overall work is given in section 5.

PROPOSED METHOD
Nash equilibrium game theory (NEGT) [23] has been developed on the CIFAR dataset to identify a manipulative change in the data that can influence the decision boundaries of the learner.Through this approach, it is possible to adjust many favorable labels to negative values.The Nash equilibrium assists a player in determining the optimum incentive in a circumstance based not just on their actions but also on the choice of the other people involved.NEGT identifies a set of tactics that are mutually optimal for both the learner and the opponent [24].Here, neither the learner nor the opponent has any incentive to deviate from these tactics.The learner would then be retrained across all adversarial data manipulations made by many players to suggest a secure CNN that is resistant to future adverse data manipulations.Figure 1 (see in appendix) shows the flow chart demonstrating a two-player games.

Dataset validation
A labelled input training data X train is used to train CNN original and then is tested using testing data X test.To obtain manipulated CNN, the adversarial data manipulation function ( * ) is added to X test data.The data is considered from the CIFAR dataset [25], [26].

NASH EQUILIBRIUM GAME THEORY
The adversarial learning is simulated by the training algorithm as a fixed sum Stackelberg game among two players, an adversary who plays leader (L) role and learner who plays the follower (F) role.The game begins with the leader taking the first action/move/play.The adversary's gain is thought to represent  ISSN: 2088-8708 Int J Elec & Comp Eng, Vol. 13, No. 6, December 2023: 6351-6360 6354 the learner's loss in a constant sum game, and vice versa.During each encounter, each of the opponents and the learner executes a move [27].A target CNN is tested on Xtest after being trained on Xtrain.The objective of the game is to identify  * such that  +  * reduces this CNN's performance found on Xtest.

Adversary (leader)
The adversary is supposed to be looking for genuine positives with only an understanding of the learner's class label errors.The adversary seeks data changes that increase classification error, error(ω) using an evolutionary algorithm.In the evolutionary algorithm,   (, ) is defined as the fitness function which improves the game's progresses.The game has converged when the opponent does not notice an improvement in the payout function or when the highest number of repetitions is achieved.The game convergence criteria are determined by the evolutionary algorithm's search and optimization criteria in each game round.The game eventually devolves into an aggressive data manipulation game using weights ω on the learner.Each player is assigned to the L and F strategy areas A and W, respectively [28].The strategy space is a set of possible moves for each participant.The reward function   and   of the player determines the result of a strategy.For a specific statement of ω∈W, the best strategy  * ∈  for the leader is expressed in (1), For labelled input training data Xtrain, Xtest available during the game, the adversary seeks a move that maximizes the fitness function   (), where, error (ω) is the categorization error evaluated by recall for the current adversary data.The term cost is the `2 norm for the current α.Hence, the (2) to ( 4) is written as: The negative cost (α) term ensures that the adversary makes as little alteration to the current α while maximizing the positive error (ω) term.By necessity, the fitness function maximizes error (ω) by minimizing the associated recall (ω).For each iteration of the game, recall (ω) is computed on the manipulated training data  +  and the iteration that produces the highest value for   (, ) is chosen for subsequent iterations.cost(α) is improved by an empirically determined weighting term for each dataset.To provide a positive fitness value in the evolutionary process, a constant 1 is added to   ().

Learner (follower)
CNN acts as the learner.The input and output layers of the CNN architecture are available in TensorFlow as the CIFAR10 model.Convolution layers, max pooling layers, regularization layers, and activation units comprise CNN's input layers.The CNN has a softmax probability distribution function output layer.The CNN's input and output layers define the learner's overall loss function.
Following the adversary's attack, the learner retrains the model.At equilibrium, the adversary can uncover examination data that is notably distinct from the dataset, while the learner can modify its model with antagonistic data to account for new threats.For a given observation of ω∈W, For L's move α, F's best strategy is formulated in (5).
The empirical difference among the given input training data Xtrain and the adversary testing distribution of data is characterized by  +  * terms of the attacker's cost cost(α) as well as the learner's error error(α).Throughout the game's versions, we can find  that maximizes the adversary's payoff   (, ) by manipulating the training data distribution Xtrain into  + .After game convergence, we can find the  * that minimizes the learner's payoff   (, ) by manipulating the dispersion of testing data Xtest into  +  * .The CNN is re-trained on the new bridge sample to modify DL processors for hostile data [29].

RESULTS AND DISCUSSION
This segment provides the results and analysis of the proposed NEGT-GAL [30] model where is implemented and simulated using Python 3.7 software whereas the computer is powered by the INTEL i5 processor running at 2.4 GHz with 16 GB RAM on Windows 10 OS.Where, Figure 2  Generative adversarial deep learning in images using Nash equilibrium game theory (Syeda Imrana Fatima) 6355 integration on strong labels that seem to be unfavorable in the harmful procedure.Adding and removing pixels, as well as modifying the shape and scale of the picture, are examples of changes that prevent detection.

Figure 2. Original images
In Figure 3, the images from the CIFAR dataset have undergone a form of alteration where additional pixels have been added to make them appear similar.This manipulation can impact the accuracy of image recognition algorithms trained on the dataset, as they may struggle to distinguish between artificially modified images.In Figure 4, the animal face has been altered to seem the same by altering the thickness and shape.In Figure 5, overall images from the CIFAR dataset images generated by adding and removing pixels.Figure 5 shows that the CIFAR landscape images were modified to appear as if 9 pixels had simply been added without altering the geometry of the images.iteration fast gradient method (AI-FGM), and adversarial transform network (ATN) models do not suitable to provide high security in adversary manipulations.The adversarial manipulations were analyzed and the features were recognized by using the SAR image on DL algorithms.The SAR images was well suitable to report the manipulations that occurred but the overfitting and not suitable for algorithms except ILCM algorithm [19].The AI-FGM method uses a gradient searching process in an iterative method related to adversarial attacks on DL to result in high success in adversary attacks by the modifications done in pixels of the image but it does not give suitable performance [21].An ATN model was utilized to launch attacks against various time series classification models.However, this approach may not yield precise outcomes for time series classification [22].Hence to result in a secured clear image that can give high performance in any manipulations, a NEGT is applied to CIFAR dataset in GAL algorithm.The loss function of generator and discriminator performance on CIFAR data manipulations with Nash equilibrium has been plotted against each other for its monotonically generated images and classification in Figure 6.
Here, Tables 1 to 4 represents the comparison analysis between various existing methods.The following are the methods used for the comparison (i.e.) DC-GAN, deep convolutional conditional GANs (DC-CGAN's), distributed DC-CGANs [21] and the proposed NEGT-GAL method.By using those methods, various performance parameters such as Wall-clock time, mean, standard deviation (SD), Fréchet inception distance (FID) are evaluated.9.41 Proposed NEGT-GAL 6.9 Table 1 shows that the wall clock time consumed by the DC-GAN is 375,520 sec, DC-CGAN's is 393,100 sec and distributed DC-CGANs [21] is 39,011 sec whereas the proposed NEGT-GAL method consumes wall clock time of 25,243 sec.Therefore, the proposed NEGT-GAL takes less time while compared to the existing methods and outperforms the existing methods.Figure 7 illustrates the graphical comparison of the proposed NEGT-GAL with existing methods in terms of wall clock time.  2 shows mean value of the DC-GAN, DC-CGAN's, and distributed DC-CGANs [21] as 4.39, 5.69, and 6.43 respectively.Whereas, the proposed NEGT-GAL method achieved a maximum mean value of 7.92 which is greater than the compared existing methods.Here also the proposed NEGT-GAL method outperforms the compared existing methods.The graphical comparison of the proposed NEGT-GAL with existing methods in terms of mean value is illustrated in Figure 8.  3 shows the achieved SD value of the DC-GAN, DC-CGAN's, and distributed DC-CGANs [21] as 0.28, 0.31, and 0.25 respectively whereas, the proposed NEGT-GAL method achieved a SD of 0.18.So, the proposed NEGT-GAL achieved a less SD value and outperforms the compared existing methods.Figure 9 illustrates the graphical comparison of the proposed NEGT-GAL with existing methods in terms of SD.
Finally, the DC-GAN, DC-CGAN's, and distributed DC-CGANs [21] achieved FID of 14.13, 11.12 and 9.41 respectively which is given in Table 4 whereas, the proposed NEGT-GAL method achieved a minimum FID of 6.90.Therefore, the results demonstrate that the proposed NEGT-GAL outperforms the DC-GAN, DC-CGAN's, and distributed DC-CGANs [21] in terms of FID.The graphical comparison of the proposed NEGT-GAL with existing DC-GANs, DC-CGAN's and distributed DC-CGANs [21] using various parameters is illustrated in Figure 10.

CONCLUSION
The complex task of adversarial data manipulations was reduced by presenting a GAL algorithm using NEGT.NEGT is applied to the CIFAR dataset as the input dataset whereas a fitness function is employed in a sequential game.During the sequential game, an adversary manipulates the input CIFAR dataset multiple times, which affects the learner's assessment results.In the game theory the adversary generates the manipulations on data and the learner retains all the manipulations held by the adversary and resulting in the secured CNN as output.The generative adversarial algorithm converges the affecting performance of testing on adversarial manipulations in DL networks which improves the security of adversarial manipulations.The Generative adversarial algorithm including sequential games with both players and stochastic games in deep neural networks resulted in an improved performance in secured CNN.Moreover, to evaluate the results of the proposed NEGT-GAL algorithm, it is compared with the conventional approaches such as DC-GAN, DC-CGAN's, and distributed DC-CGANs.The proposed NEGT-GAL achieved a greater mean value of 7.92, minimal SD of 0.18, minimal FID of 6.9 and less wall clock time of 25,243, which are superior when compared to the existing methods.In the future, some modifications will be included in the GAL algorithm to improve the classification performance.

Figure 7 .
Figure 7. Graphical comparison in terms of wall clock time

Figure 8 .
Figure 8. Graphical comparison in terms of mean

Figure 9 .
Figure 9. Graphical comparison in terms of SD Figure 10.Graphical comparison in terms of FID

APPENDIXFigure 1 .
Figure 1.A flow chart demonstrating a two-player game

Table 1 .
Comparison of wall-clock time

Table 2 .
Comparison of mean value

Table 3 .
Comparison of SD

Table 4 .
Comparison of FID