Hybrid ResNet-50 and LSTM Approach for Effective Video Anomaly Detection in Intelligent Surveillance Systems
Keywords:
Anomaly detection, Deep Learning, LSTM, Rescale, ResNet-50, Video surveillanceAbstract
Modern intelligent surveillance systems have given video anomaly detection much attention. Interior and outdoor monitoring devices are widespread in modern public places and smart cities. Due to the limited modelling capabilities and difficulty in capturing complicated relationships, conventional approaches have difficulty in recognizing video anomalies. This research tries to solve the problem using a hybrid strategy combining ResNet-50 (Residual Network-50) and Long Short-Term Memory (LSTM) algorithms for detecting anomaly video activity. Videos of normal and anomaly actions datasets enhanced the video clip's unique features, reducing the data required for storage and transmission. There are separate frames in each video. According to the proposed procedure, each frame is rescaled into a different pixel size before being fed into the ResNet-50 technique for feature extraction. Following feature extraction, the frames are then fed into the classification layer. The values of the new feature vectors are calculated by adding the original feature vectors acquired by ResNet-50. Finally, the LSTM model classifies the video as normal or abnormal using the information extracted from a sequence of frames. The LSTM assigns a classification to the retrieved images. The hybrid technique ResNet-50 and LSTM obtained 96.48% accuracy using the UCSD Ped 1 dataset. The proposed models outperformed the equivalent deep learning models and showed a noticeably higher performance accuracy.
Downloads
References
G. Sreenu, & S. Durai, “Intelligent video surveillance: a review through deep learning techniques for crowd analysis,” Journal of Big Data, vol. 6, no. 1, pp. 1-27, 2019.
A. Alshammari, & D. B. Rawat, “Intelligent multi-camera video surveillance system for smart city applications,” In 2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC) IEEE, pp. 0317-0323, 2019, January.
Y. Feng, Y. Yuan, & X. Lu, “Learning deep event models for crowd anomaly detection,” Neurocomputing, vol. 219, pp. 548-556, 2017.
P. Khaire, & P. Kumar, “A semi-supervised deep learning-based video anomaly detection framework using RGB-D for surveillance of real-world critical environments,” Forensic Science International: Digital Investigation, vol. 40, p. 301346, 2022.
Y. Zhou, B. Li, J. Wang, E. Rocco, & Q. Meng, “Discovering unknowns: Context-enhanced anomaly detection for curiosity-driven autonomous underwater exploration,” Pattern Recognition, vol. 131, p. 108860, 2022.
X. Cheng, L. Yuan, Z. Liu, & F. Guo, “Comparative analysis of video anomaly detection algorithms,” In International Conference on Computer, Artificial Intelligence, and Control Engineering (CAICE 2022) SPIE, vol. 12288, pp. 431-436, 2022, December.
R. F. Mansour, J. Escorcia-Gutierrez, M. Gamarra, J. A. Villanueva, & N. Leal, “Intelligent video anomaly detection and classification using faster RCNN with deep reinforcement learning model,” Image and Vision Computing, vol. 112, p. 104229, 2021.
L. A. Saleem, & E. V. Reddy, “A Survey on Deep Learning based Video Surveillance Framework,” In 2023 International Conference on Computer Communication and Informatics (ICCCI), IEEE, pp. 1-6, 2023, January.
K. Singh, S. Rajora, D. K. Vishwakarma, G. Tripathi, S. Kumar, & G. S. Walia, “Crowd anomaly detection using aggregation of ensembles of fine-tuned convents,” Neurocomputing, vol. 371, pp. 188-198, 2020.
N. L. Lavanya, N. Vijayananda, N. N. Sattigeri, R. K. Nisarga, & N. M. Pooja, “Survey on Abnormal Event Detection and Signalling in Multiple Video Surveillance Scenes Using CNN,” International Journal of Human Computations & Intelligence, vol. 2, no. 3, pp. 159-168, 2023.
J. Ren, F. Xia, Y. Liu, & I. Lee, “Deep video anomaly detection: Opportunities and challenges,” In 2021 international conference on data mining workshops (ICDMW) IEEE, pp. 959-966, 2021, December.
G. Pang, C. Shen, L. Cao, & A. V. D. Hengel, “Deep learning for anomaly detection: A review,” ACM computing surveys (CSUR), vol. 54, no. 2, pp. 1-38, 2021.
E. Şengönül, R. Samet, Q. Abu Al-Haija, A. Alqahtani, B. Alturki, & A. A. Alsulami, “An Analysis of Artificial Intelligence Techniques in Surveillance Video Anomaly Detection: A Comprehensive Survey,” Applied Sciences, vol. 13, no. 8, p. 4956, 2023.
A. Berroukham, K. Housni, M. Lahraichi, & I. Boulfrifi, “Deep learning-based methods for anomaly detection in video surveillance: a review,” Bulletin of Electrical Engineering and Informatics, vol. 12, no. 1, pp. 314-327, 2023.
M. Arunachalam, V. Sanghavi, Y. A. Yao, Y. A. Zhou, L. A. Wang, Z. Wen, & F. Mohammad, “Strategies for Optimising End-to-End Artificial Intelligence Pipelines on Intel Xeon Processors,” arXiv preprint arXiv:2211.00286, 2022.
H. T. Duong, V. T. Le, & V. T. Hoang, “Deep Learning-Based Anomaly Detection in Video Surveillance: A Survey,” Sensors, vol. 23, no. 11, p. 5024, 2023.
F. Mumcu, K. Doshi, & Y. Yilmaz, “Adversarial Machine Learning Attacks Against Video Anomaly Detection Systems,” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 206-213, 2022.
Z. K. Abbas, & A. A. Al-Ani, “A Comprehensive Review for Video Anomaly Detection on Videos,” In IEEE 2022 International Conference on Computer Science and Software Engineering (CSASE), pp. 1-1, 2022, March.
Y. Hao, J. Li, N. Wang, X. Wang, & X. Gao, “Spatiotemporal consistency-enhanced network for video anomaly detection,” Pattern Recognition, vol. 121, p. 108232, 2022.
R. Nawaratne, D. Alahakoon, D. De Silva, & X. Yu, “Spatiotemporal anomaly detection using deep learning for real-time video surveillance,” IEEE Transactions on Industrial Informatics, vol. 16, no. 1, pp. 393-402, 2019.
Y. Kim, J. Y. Yu, E. Lee, & Y. G. Kim, “Video anomaly detection using Cross U-Net and cascade sliding window,” Journal of King Saud University-Computer and Information Sciences, vol. 34, no. 6, pp. 3273-3284, 2022.
J. Arunnehru, “Deep learning-based real-world object detection and improved anomaly detection for surveillance videos,” Materials Today: Proceedings, vol. 80, pp. 2911-2916, 2023.
W. Shao, R. Kawakami, & T. Naemura, “Anomaly Detection Using Spatio-Temporal Context Learned by Video Clip Sorting,” IEICE TRANSACTIONS on Information and Systems, vol. 105, no. 5, pp. 1094-1102, 2022.
A. A. Khan, M. A. Nauman, M. Shoaib, R. Jahangir, R. Alroobaea, M. Alsafyani, & C. Wechtaisong, “Crowd Anomaly Detection in Video Frames Using Fine-Tuned AlexNet Model,” Electronics, vol. 11, no. 19, p. 3105, 2022.
S. W. Khan, Q. Hafeez, M. I. Khalid, R. Alroobaea, S. Hussain, J. Iqbal, & S. S. Ullah, “Anomaly detection in traffic surveillance videos using deep learning,” Sensors, vol. 22, no. 17, p. 6563, 2022.
A. Chriki, H. Touati, H. Snoussi, & F. Kamoun, “Deep learning and handcrafted features for one-class anomaly detection in UAV video,” Multimedia Tools and Applications, vol. 80, pp. 2599-2620, 2021.
S. Ul Amin, M. Ullah, M. Sajjad, F. A. Cheikh, M. Hijji, A. Hijji, & K. Muhammad, “EADN: An Efficient Deep Learning Model for Anomaly Detection in Videos,” Mathematics, vol. 10, no. 9, p. 1555, 2022.
W. Ullah, A. Ullah, I. U. Haq, K. Muhammad, M. Sajjad, & S. W. Baik, “CNN features bi-directional LSTM for real-time anomaly detection in surveillance networks,” Multimedia tools and applications, vol. 80, pp. 16979-16995, 2021.
T. Ganokratanaa, S. Aramvith, & N. Sebe, “Video anomaly detection using deep residual-spatiotemporal translation network,” Pattern Recognition Letters, vol. 155, pp. 143-150, 2022.
Meneses-Claudio, B. ., Perez-Siguas, R. ., Matta-Solis, H. ., Matta-Solis, E. ., Matta-Perez, H. ., Cruzata-Martinez, A. ., Saberbein-Muñoz, J. ., & Salinas-Cruz, M. . (2023). Automatic System for Detecting Pathologies in the Respiratory System for the Care of Patients with Bronchial Asthma Visualized by Computerized Radiography. International Journal on Recent and Innovation Trends in Computing and Communication, 11(2), 27–34. https://doi.org/10.17762/ijritcc.v11i2.6107
Kwame Boateng, Machine Learning in Cybersecurity: Intrusion Detection and Threat Analysis , Machine Learning Applications Conference Proceedings, Vol 3 2023.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
