Securing Visual Integrity: An Efficient NetB4-Based Solution with Attention Layers and Siamese Training for Face Manipulation Detection in Videos
Keywords:
Convolutional neural networks, Deepfake, Digital media forensics, Efficient netB4, Face forensicsAbstract
The public can now easily create video deepfakes because of the growth of machine learning and artificial intelligence in the current digital era. The system of society is severely affected by this technology. The phrase "Deep Fake" implies digital representations created by advanced artificial intelligence that are adapted to make erroneous sounds and sights that appear real. The identification of these motion pictures presents an important obstacle because of the occasional development of progressively realistic deepfake generating techniques. FaceSwap and deepfake are two programs that have made it easier for anyone to realistically alter faces in videos in recent years. Technological advances can be helpful, but they may also be misused, which may result in difficulties like the dissemination of misleading data or online bullying. For this reason, being able to recognize when a video has been altered is important. This research tackles the problem of face alteration detection in video sequences that aim to target modern facial manipulation methods in this research. Specifically, the research looks at a set of several Convolutional Neural Network (CNN) models that were successfully trained. The suggested method uses two separate concepts to generate multiple models starting from the fundamental network (EfficientNetB4): Layers of attention and instruction in Siamese. Thus, by such a structure this model attains an accuracy of 94% on FaceForensics and DFDC dataset.
Downloads
References
S. Agarwal, H. Farid, Y. Gu, M. He, K. Nagano, and H. Li, “Protecting World Leaders Against Deep Fakes.”
B. Dolhansky et al., “The DeepFake Detection Challenge (DFDC) Dataset,” Jun. 2020, doi: https://doi.org/10.48550/arXiv.2006.07397.
S. Verma, “Classification of Spoofing Attack Detection using Deep Learning Algorithms MSc Research Project Data Analytics.”
K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” Sep. 2014, doi: https://doi.org/10.48550/arXiv.1409.1556.
S. Sakib, M. Tarid, and A. Abid, “Deepfake detection Using Neural Networks,” 2021.
S. R. Krishnan and P. Amudha, “International Journal of INTELLIGENT SYSTEMS AND APPLICATIONS IN ENGINEERING Hybrid ResNet-50 and LSTM Approach for Effective Video Anomaly Detection in Intelligent Surveillance Systems.” [Online]. Available: www.ijisae.org
Y. Li, X. Yang, P. Sun, H. Qi, and S. Lyu, “Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics,” Sep. 2019, doi: https://doi.org/10.48550/arXiv.1909.12962.
A. Rossler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, and M. Niessner, “FaceForensics++: Learning to detect manipulated facial images,” in Proceedings of the IEEE International Conference on Computer Vision, 2019. doi: 10.1109/ICCV.2019.00009.
A. A. Pokroy and A. D. Egorov, “EfficientNets for DeepFake Detection: Comparison of Pretrained Models,” in Proceedings of the 2021 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering, ElConRus 2021, Institute of Electrical and Electronics Engineers Inc., Jan. 2021, pp. 598–600. doi: 10.1109/ElConRus51938.2021.9396092.
X. Yang, Y. Li, and S. Lyu, “Exposing Deep Fakes Using Inconsistent Head Poses,” in ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, 2019. doi: 10.1109/ICASSP.2019.8683164.
T. Wang, H. Cheng, K. P. Chow, and L. Nie, “Deep Convolutional Pooling Transformer for Deepfake Detection,” Sep. 2022, doi: 10.1145/3588574.
N. Bonettini, L. Bondi, E. D. Cannas, P. Bestagini, S. Mandelli, and S. Tubaro, “Video face manipulation detection through ensemble of CNNs,” in Proceedings - International Conference on Pattern Recognition, Institute of Electrical and Electronics Engineers Inc., 2020, pp. 5012–5019. doi: 10.1109/ICPR48806.2021.9412711.
A. Gironi, M. Fontani, T. Bianchi, A. Piva, and M. Barni, “A video forensic technique for detecting frame deletion and insertion,” in ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, 2014. doi: 10.1109/ICASSP.2014.6854801.
Y. Al-Dhabi and S. Zhang, “Deepfake Video Detection by Combining Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN),” in 2021 IEEE International Conference on Computer Science, Artificial Intelligence and Electronic Engineering, CSAIEE 2021, Institute of Electrical and Electronics Engineers Inc., Aug. 2021, pp. 236–241. doi: 10.1109/CSAIEE54046.2021.9543264.
I. Ilhan, E. Bali, and M. Karakose, “An Improved DeepFake Detection Approach with NASNetLarge CNN,” in 2022 International Conference on Data Analytics for Business and Industry, ICDABI 2022, Institute of Electrical and Electronics Engineers Inc., 2022, pp. 598–602. doi: 10.1109/ICDABI56818.2022.10041558.
M. Tan and Q. V. Le, “EfficientNetV2: Smaller Models and Faster Training,” Apr. 2021, [Online]. Available: http://arxiv.org/abs/2104.00298
A. Kohli and A. Gupta, “Detecting DeepFake, FaceSwap and Face2Face facial forgeries using frequency CNN,” Multimed Tools Appl, vol. 80, no. 12, pp. 18461–18478, May 2021, doi: 10.1007/s11042-020-10420-8.
V. Bazarevsky, Y. Kartynnik, A. Vakunov, K. Raveendran, and M. Grundmann, “BlazeFace: Sub-millisecond Neural Face Detection on Mobile GPUs,” Jul. 2019, [Online]. Available: http://arxiv.org/abs/1907.05047
A. Seth and A. K. Gogineni, “Detection of Deep-fakes in Videos using CNN and Transformers”, doi: 10.13140/RG.2.2.23238.60480.
L. Bondi, E. Daniele Cannas, P. Bestagini, and S. Tubaro, “Training Strategies and Data Augmentations in CNN-based DeepFake Video Detection,” in 2020 IEEE International Workshop on Information Forensics and Security, WIFS 2020, Institute of Electrical and Electronics Engineers Inc., Dec. 2020. doi: 10.1109/WIFS49906.2020.9360901.
H. T. Duong, V. T. Le, and V. T. Hoang, “Deep Learning-Based Anomaly Detection in Video Surveillance: A Survey,” Sensors, vol. 23, no. 11. 2023. doi: 10.3390/s23115024.
J. Deng, W. Dong, R. Socher, L.-J. Li, Kai Li, and Li Fei-Fei, “ImageNet: A large-scale hierarchical image database,” 2010. doi: 10.1109/cvpr.2009.5206848.
H. Dang, F. Liu, J. Stehouwer, X. Liu, and A. Jain, “On the Detection of Digital Face Manipulation.”
S. Ganguly, A. Ganguly, S. Mohiuddin, S. Malakar, and R. Sarkar, “ViXNet: Vision Transformer with Xception Network for deepfakes based video and image forgery detection,” Expert Syst Appl, vol. 210, Dec. 2022, doi: 10.1016/j.eswa.2022.118423.
X. Cheng, L. Yuan, Z. Liu, and F. Guo, “Comparative analysis of video anomaly detection algorithms,” 2022. doi: 10.1117/12.2641049.
A. Berroukham, K. Housni, M. Lahraichi, and I. Boulfrifi, “Deep learning-based methods for anomaly detection in video surveillance: a review,” Bulletin of Electrical Engineering and Informatics, vol. 12, no. 1, 2023, doi: 10.11591/eei.v12i1.3944.
L. D’Amiano, D. Cozzolino, G. Poggi, and L. Verdoliva, “A PatchMatch-Based Dense-Field Algorithm for Video Copy-Move Detection and Localization,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 29, no. 3, 2019, doi: 10.1109/TCSVT.2018.2804768.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
