VADNet: A Novel Deep Learning Architecture for Automatic Detection and Classification of Abnormalities from Public Surveillance Videos
Keywords:
Video Abnormality Detection, Public Surveillance, Deep Learning, Artificial Intelligence, Novel Deep Learning ArchitectureAbstract
Usage of public surveillance cameras in cities and other places of public importance has resulted in reduction of crimes besides helping in establishing evidences of certain incidents. Thus, video analytics has become indispensable research area. With technological innovations like cloud computing, Internet of Things (IoT) and Artificial Intelligence (AI), storage and analysis of videos in real time is made possible. At the same time, emerging techniques in deep learning have paved way for processing of image content leading efficient video analytics. Abnormality detection from surveillance videos has assumed significance. The existing research in this area based on Convolutional Neural Network (CNN) has limitations because of the fact that it cannot bestow optimal performance unless, it is customized to solve the problem in hand. Towards this end, in this paper, we proposed a novel deep learning architecture known as Video Abnormality Detection Net (VADNet) for detection of abnormalities from surveillance videos. VADNet is a CNN variant designed for leveraging detection performance. We proposed an algorithm named Learning based Video Abnormality Detection (LbVAD) which exploits VADNet for efficient detection of video abnormalities. UCF-Crime is the benchmark dataset used for our empirical study. Our experimental results revealed that VADNet outperforms existing CNN variants like MobileNetV1, ResNet50 and VGG19 models with highest accuracy 95.64%.
Downloads
References
Khaleghi Ali and Moin Mohammad Shahram (2018). Improved anomaly detection in surveillance videos based on a deep learning method, IEEE, pp.73–81. doi:10.1109/RIOS.2018.8406634
Rashmika Nawaratne, Damminda Alahakoon and Daswin De Silvaand Xinghuo Yu. (2019). Spatiotemporal Anomaly Detection Using Deep Learning for Real-Time Video Surveillance. IEEE, pp.1-10.
Kavikuil K. and Amudha J. (2018). Leveraging Deep Learning for Anomaly Detection in Video Surveillance. First International Conference on Artificial Intelligence and Cognitive Computing, pp.239–247. doi:10.1007/978-981-13-1580-0_23
Khosro Rezaee, Sara Mohammad Rezakhani, Mohammad R. Khosravi and Mohammad Kazem Moghimi. (2021). A survey on deep learning-based real-time crowd anomaly detection for secure distributed video surveillance. Personal and Ubiquitous Computing, doi:10.1007/s00779-021-01586-5
Nayak Rashmiranjan, Pati Umesh Chandra and Das Santos Kumar (2021). A comprehensive review on deep learning-based methods for video anomaly detection. Image and Vision Computing, 106, 104078–. doi:10.1016/j.imavis.2020.104078
Sardar Waqar Khan, Qasim Hafeez, Muhammad Irfan Khalid and Roobaea Alrooba. (2022). Anomaly Detection in Traffic Surveillance Videos Using Deep Learning. MDPI, pp.1-28.
John Gatara Munyua, Geoffrey Mariga Wambugu and Stephen Thiiru Njenga. (2021). A Survey of Deep Learning Solutions for Anomaly Detection in Surveillance Videos. International Journal of Computer and Information Technology. 10(5), pp.1-8.
Aberkane Sabrina and Elarbi Mohamed (2019). Deep Reinforcement Learning for Real-world Anomaly Detection in Surveillance Videos, IEEE, pp.1–5. doi:10.1109/ISPA48434.2019.8966795
Sabah Abdulazeez Jebur 1,2, Khalid A. Hussein 3 , Haider Kadhim Hoomod 3 , Laith. (2023). Review on Deep Learning Approaches for Anomaly Event Detection in Video Surveillance. MDPI, pp.1-22.
Cosar, Serhan; Donatiello, Giuseppe; Bogorny, Vania; Garate, Carolina; Alvares, Luis O. and Bremond, Francois (2016). Towards Abnormal Trajectory and Event Detection in Video Surveillance. IEEE Transactions on Circuits and Systems for Video Technology, pp.1–14. doi:10.1109/TCSVT.2016.2589859
Giannakeris, Panagiotis; Kaltsa, Vagia; Avgerinakis, Konstantinos; Briassouli, Alexia; Vrochidis, Stefanos; Kompatsiaris, Ioannis. (2018). Speed Estimation and Abnormality Detection from Surveillance Cameras. , IEEE, pp.93–936. doi:10.1109/CVPRW.2018.00020
A. Balasundaram, Golda Dilip, M. Manickam, Arun Kumar Sivaraman and K. Gu. (2022). Abnormality Identification in Video Surveillance System using DCT. Intelligent Automation & Soft Computing, pp.1-12.
Ahmed, Sk. Arif; Dogra, Debi Prosad; Kar, Samarjit; Roy, Partha Pratim (2018). Surveillance scene representation and trajectory abnormality detection using aggregation of multiple concepts. Expert Systems with Applications, 101, pp.43–55. doi:10.1016/j.eswa.2018.02.013
Lin, Weiyao; Zhang, Yihao; Lu, Jiwen; Zhou, Bing; Wang, Jinjun; Zhou, Yu (2015). Summarizing surveillance videos with local-patch-learning-based abnormality detection, blob sequence optimization, and type-based synopsis. Neurocomputing, 155, pp.84–98. doi:10.1016/j.neucom.2014.12.044
Meng, B., Zhang, L., Jin, F., Yang, L., Cheng, H., & Wang, Q. (2017). Abnormal Events Detection Using Deep Networks for Video Surveillance. Cognitive Systems and Signal Processing, pp.197–204. doi:10.1007/978-981-10-5230-9_22
Mujtaba Asad;He Jiang;Jie Yang;Enmei Tu;Aftab Ahmad Malik; (2021). Multi-Stream 3D latent feature clustering for abnormality detection in videos . Applied Intelligence, pp.1–18. doi:10.1007/s10489-021-02356-9
Ye, Ou; Deng, Jun; Yu, Zhenhua; Liu, Tao; Dong, Lihong (2020). Abnormal Event Detection via Feature Expectation Subgraph Calibrating Classification in Video Surveillance Scenes. IEEE Access, 8, pp.97564–97575. doi:10.1109/access.2020.2997357
Ko, Kwang-Eun; Sim, Kwee-Bo (2018). Deep convolutional framework for abnormal behaviour detection in a smart surveillance system. Engineering Applications of Artificial Intelligence, 67(), 226–234. doi:10.1016/j.engappai.2017.10.001
Prawiro, Herman; Peng, Jian-Wei; Pan, Tse-Yu; Hu, Min-Chun (2020). Abnormal Event Detection in Surveillance Videos Using Two-Stream Decoder, IEEE, pp.1–6. doi:10.1109/ICMEW46912.2020.9105987
Li, Ang; Miao, Zhenjiang; Cen, Yigang; Zhang, Xiao-Ping; Zhang, Linna; Chen, Shiming (2020). Abnormal event detection in surveillance videos based on low-rank and compact coefficient dictionary learning. Pattern Recognition, 108, pp.1–16. doi:10.1016/j.patcog.2020.107355
Paul, Manoranjan; Haque, Shah M E; Chakraborty, Subrata (2013). Human detection in surveillance videos and its applications - a review. EURASIP Journal on Advances in Signal Processing, 2013(1), pp.1–16. doi:10.1186/1687-6180-2013-176
Mabrouk, Amira Ben; Zagrouba, Ezzeddine (2017). Abnormal behaviour recognition for intelligent video surveillance systems: a review. Expert Systems with Applications, pp.1–55. doi:10.1016/j.eswa.2017.09.029
Yan Shiyang, Smith Jeremy S., Lu Wenjin and Zhang Bailing. (2018). Abnormal Event Detection from Videos using a Two-stream Recurrent Variational Autoencoder. IEEE Transactions on Cognitive and Developmental Systems, pp.1–14. doi:10.1109/TCDS.2018.2883368
Wang, Siqi; Zeng, Yijie; Liu, Qiang; Zhu, Chengzhang; Zhu, En; Yin, Jianping (2018). [ACM Press 2018 ACM Multimedia Conference - Seoul, Republic of Korea (2018.10.22-2018.10.26)] 2018 ACM Multimedia Conference on Multimedia Conference - MM '18 - Detecting Abnormality without Knowing Normality, pp.636–644. doi:10.1145/3240508.3240615
Chong, Y. S., and Tay, Y. H. (2017). Abnormal Event Detection in Videos Using Spatiotemporal Autoencoder. Lecture Notes in Computer Science, pp.189–196. doi:10.1007/978-3-319-59081-3_23
Tong Li;Xinyue Chen;Fushun Zhu;Zhengyu Zhang;Hua Yan; (2021). Two-stream deep spatial-temporal auto-encoder for surveillance video abnormal event detection. Neurocomputing, pp.1–15. doi:10.1016/j.neucom.2021.01.097
Leyva, Roberto; Sanchez, Victor; Li, Chang-Tsun (2017). The LV dataset: A realistic surveillance video dataset for abnormal event detection, IEEE, pp.1–6. doi:10.1109/IWBF.2017.7935096
Zhou, Fuqiang; Wang, Lin; Li, Zuoxin; Zuo, Wangxia; Tan, Haishu (2019). Unsupervised Learning Approach for Abnormal Event Detection in Surveillance Video by Hybrid Autoencoder. Neural Processing Letters, pp.1–15. doi:10.1007/s11063-019-10113-w
Cui, Lili; Li, Kehuang; Chen, Jiapin; Li, Zhenbo (2011). Abnormal event detection in traffic video surveillance based on local features, IEEE, pp.362–366. doi:10.1109/CISP.2011.6099933
Dong, Nan; Jia, Zhen; Shao, Jie; Xiong, Ziyou; Li, Zhipeng; Liu, Fuqiang; Zhao, Jianwei; Peng, PeiYuan (2010). Traffic Abnormality Detection through Directional Motion Behavior Map, IEEE, pp.80–84. doi:10.1109/avss.2010.61
Leyva, Roberto; Sanchez, Victor; Li, Chang-Tsun (2017). Abnormal event detection in videos using binary features, IEEE, pp.621–625. doi:10.1109/TSP.2017.8076061
Varadarajan, Jagannadan; Odobez, Jean-Marc (2009). Topic models for scene analysis and abnormality detection, IEEE, pp.1338–1345. doi:10.1109/iccvw.2009.5457456
Leyva, Roberto; Sanchez, Victor; Li, Chang-Tsun (2018). Fast Detection of Abnormal Events in Videos with Binary Features, IEEE, pp.1318–1322. doi:10.1109/ICASSP.2018.8461759
Sharma, M. K.; Sheet, D.; Biswas, P. K. (2020). Spatiotemporal deep networks for detecting abnormality in videos. Multimedia Tools and Applications, pp.1–32. doi:10.1007/s11042-020-08786-w
Dr. Rajesh Sharma and Dr. Akey Sungheetha. (2021). An Efficient Dimension Reduction based Fusion of CNN and SVM Model for Detection of Abnormal Incident in Video Surveilla. Journal of Soft Computing Paradigm (JSCP). 3(2), pp.55-69.
Ravanbakhsh, Mahdyar; Nabi, Moin; Sangineto, Enver; Marcenaro, Lucio; Regazzoni, Carlo; Sebe, Nicu (2017). Abnormal event detection in videos using generative adversarial nets, pp.1577–1581. doi:10.1109/ICIP.2017.8296547
Yihao Zhang, ; Weiyao Lin, ; Guangwei Zhang, ; Chuanfei Luo, ; Dong Jiang, ; Chunlian Yao, (2014). A new approach for extracting and summarizing abnormal activities in surveillance videos, IEEE, pp.1–5. doi:10.1109/ICMEW.2014.6890537
A.Niranjil Kumar and Dr. C.Sureshkumar. (2014). Abnormal Crowd Detection and Tracking in Surveillance Video Sequences. International Journal of Advanced Research in Computer and Communication Engineering. 3(9), pp.1-5.
Popoola, O. P.; Kejun Wang, (2012). Video-Based Abnormal Human Behavior Recognition A Review. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 42(6), pp.865–878. doi:10.1109/tsmcc.2011.2178594
Abdulamir A. Karim and Narjis Mezaal Shati. (2017). Abnormality Detection using K-means Data Stream Clustering Algorithm in Intelligent Surveillance System. Journal of AL-Qadisiyah for computer science and mathematics. 9(1), pp.82 – 98.
UCF-Crime Dataset. Retrieved from https://www.crcv.ucf.edu/projects/real-world/
Waqas Sultani, Chen Chen, Mubarak Shah, Real-world Anomaly Detection in Surveillance Videos, Cornell University Library, arXiv:1801.04264 [cs.CV], [v1] Fri, 12 Jan 2018
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
