A constrained convolutional neural network with attention mechanism for image manipulation detection

Abstract

The information disseminated by online media is often presented in the form of images, in order to quickly captivate readers and increase audience ratings. However, these images can be manipulated for malicious purposes, such as influencing public opinion, undermining media credibility, disrupting democratic processes or creating conflict within society. Various approaches, whether relying on manually developed features or deep learning, have been devised to detect falsified images. However, they frequently prove less effective when confronted with widespread and multiple manipulations. To address this challenge, in our study, we have designed a model comprising a constrained convolution layer combined with an attention mechanism and a transfer learning ResNet50 network. These components are intended to automatically learn image manipulation features in the initial layer and extract spatial features, respectively. It makes possible to detect various falsifications with much more accuracy and precision. The proposed model has been trained and tested on real datasets sourced from the literature, which include MediaEval and Casia. The obtained results indicate that our proposal surpasses other models documented in the literature. Specifically, we achieve an accuracy of 87% and a precision of 93% on the MediaEval dataset. In comparison, the performance of methods from the literature on the same dataset does not exceed 84% for accuracy and 90% for precision.