Overcoming Occlusion Challenges in Human Motion Detection through Advanced Deep Learning Techniques
Keywords:
Human motion detection, Occlusion, Recurrent Neural Network, Mask Region-based Convolutional Neural Network, Multiple Hypothesis model, WOA-RDAAbstract
Researchers engaged in Human Motion Detection (HMD) grapple with a primary challenge related to occlusion, where individuals or their body parts become obscured within image or video frames. Occlusion manifests in two distinct forms: Self-occlusion, occurring when one part of the human body hides another, and External occlusion, arising from external objects obstructing humans. This proposed work specifically focuses on self-occlusion and partial-occlusion. To discern human motion from visual data, three fundamental methods are deployed. The initial method, motion segmentation, entails identifying the moving object in a video. The second method, Object Classification, determines whether the moving object is human. The final method, the Tracking algorithm, is employed for identifying human gestures. Occlusion persists as a central concern in HMD. In our proposed methodology, we employ a Mask Region-based Convolutional Neural Network (Mask R-CNN) for motion segmentation to address the occlusion challenge. Object classification utilizes a Recurrent Neural Network (RNN), and for tracking human motion, even during self-occlusion, Multiple Hypothesis Tracking (MHT) is applied. This study presented an innovative hybrid algorithm, the Whale Optimization Algorithm and Red Deer Algorithm (WOA-RDA), demonstrating superior convergence speed coupled with high accuracy. Our HMD approach incorporates an RNN trained with 2D representations of 3D skeletal motion. Diverse datasets, encompassing scenarios with and without occlusion, are integrated into our proposed work. The experimental findings underscore the effectiveness of our approach in accurately identifying human motion under varied conditions, including both with and without occlusion scenarios.
Downloads
References
P. Wang, W. Li, P. Ogunbona, J. Wan, and S. Escalera, “RGB-D-based human motion recognition with deep learning: A survey,” Computer Vision and Pattern Recognition, 2017.
P. Pareek and A. Thakkar, “A survey on video-based Human Action Recognition: recent updates, datasets, challenges, and applications,” Artif. Intell. Rev., vol. 54, no. 3, pp. 2259–2322, 2021.
M. Sima, “Key frame extraction for Human Action Videos in dynamic spatio-temporal slice clustering,” J. Phys. Conf. Ser., vol. 2010, no. 1, p. 012076, 2021.
L.-H. Chen and C.-W. Su, “Video caption extraction using spatio-temporal slices,” Int. J. Image Graph., vol. 18, no. 02, p. 1850009, 2018.
M. Kocabas, S. Karagoz, and E. Akbas, “Self-supervised learning of 3D human pose using multi-view geometry,” 2019.
Y. Cheng, B. Yang, B. Wang, and R. T. Tan, “3D human pose estimation using spatio-temporal networks with explicit occlusion training,” 2020.
K. Wang, L. Lin, C. Jiang, C. Qian, and P. Wei, “3D human pose machines with self-supervised learning,” IEEE Trans. Pattern Anal. Mach. Intell., pp. 1–1, 2019.
S. Ghazal, U. S. Khan, M. Mubasher Saleem, N. Rashid, and J. Iqbal, “Human activity recognition using 2D skeleton data and supervised machine learning,” IET Image Process., vol. 13, no. 13, pp. 2572–2578, 2019.
B. Wandt and B. Rosenhahn, “RepNet: Weakly supervised training of an adversarial reprojection network for 3D human pose estimation,” Computer Vision and Pattern Recognition, 2019.
Lv, J. Li, and J. Tian, “Key frame extraction for sports training based on improved deep learning,” Sci. Program., vol. 2021, pp. 1–8, 2021.
M. Jian, S. Zhang, L. Wu, S. Zhang, X. Wang, and Y. He, “Deep key frame extraction for sport training,” Neurocomputing, vol. 328, pp. 147–156, 2019.
“Human movement detection using recurrent convolutional neural networks,” Special Issue-IJITEE, vol. 8, no. 12S, pp. 348–351, 2019.
L. Zhang, “Applying deep learning-based human motion recognition system in sports competition,” Front. Neurorobot., vol. 16, 2022.
Kumar and V. Kumar, “Human Motion Detection Based Upon CNN with Pruning and Edge Detection”,” ISSN, pp. 6766–6772, 2021.
N. Nair, C. Thomas, and D. B. Jayagopi, “Human activity recognition using temporal convolutional network,” in Proceedings of the 5th International Workshop on Sensor-based Activity Recognition and Interaction, 2018.
Y. A. Andrade-Ambriz, S. Ledesma, M.-A. Ibarra-Manzano, M. I. Oros-Flores, and D.-L. Almanza-Ojeda, “Human activity recognition using temporal convolutional neural network architecture,” Expert Syst. Appl., vol. 191, no. 116287, p. 116287, 2022.
M. S. Alzahrani, S. K. Jarraya, H. Ben-Abdallah, and M. S. Ali, “Comprehensive evaluation of skeleton features-based fall detection from Microsoft Kinect v2,” Signal Image Video Process., vol. 13, no. 7, pp. 1431–1439, 2019.
A. Franco, A. Magnani, and D. Maio, “A multimodal approach for human activity recognition based on skeleton and RGB data,” Pattern Recognit. Lett., vol. 131, pp. 293–299, 2020.
N. Gaud, M. Rathore, and U. Suman, “Human gait analysis and activity recognition: A review,” in 2023 IEEE Guwahati Subsection Conference (GCON), 2023.
G. Yang, C. Li, and H. Chen, “Research on deep correlation filter tracking based on channel importance,” EURASIP J. Adv. Signal Process., vol. 2022, no. 1, 2022.
Y. Yuan, J. Chu, L. Leng, J. Miao, and B.-G. Kim, “A scale-adaptive object-tracking algorithm with occlusion detection,” EURASIP J. Image Video Process., vol. 2020, no. 1, 2020.
H. Chu, K. Wang, and X. Xing, “Target tracking via particle filter and convolutional network,” J. Electr. Comput. Eng., vol. 2018, pp. 1–9, 2018.
A. Hayat, F. Morgado-Dias, B. Bhuyan, and R. Tomar, “Human activity recognition for elderly people using machine and Deep Learning approaches,” Information (Basel), vol. 13, no. 6, p. 275, 2022.
R. Ma, Z. Zhang, and E. Chen, “Human motion gesture recognition based on computer vision,” Complexity, vol. 2021, pp. 1–11, 2021.
F. Ababsa, H. Hadj-Abdelkader, and M. Boui, “3D human pose estimation with a catadioptric sensor in unconstrained environments using an annealed particle filter,” Sensors (Basel), vol. 20, no. 23, p. 6985, 2020.
L. C. Chang et al., “An intelligent automatic human detection and tracking system based on weighted resampling particle filtering,” Big Data Cogn. Comput., vol. 4, no. 4, p. 27, 2020.
R. Verma and A. K. Verma, “Particle Filter Based Visual Tracking: A Review,” International Journal of Advances in Engineering and Management, vol. 3, pp. 3660–3680, 2021.
S. Liu, D. Liu, G. Srivastava, D. Połap, and M. Woźniak, “Overview and methods of correlation filter algorithms in object tracking,” Complex Intell. Syst., 2020.
S. S. Moghaddasi and N. Faraji, “A hybrid algorithm based on particle filter and genetic algorithm for target tracking,” Expert Syst. Appl., vol. 147, no. 113188, p. 113188, 2020.
X. Guo, A. Hamdulla, and T. Tohti, “Research on target tracking algorithm based on correlation filtering,” J. Phys. Conf. Ser., vol. 2024, no. 1, p. 012043, 2021.
M. Ramesh and K. Mahesh, “Sports video classification framework using enhanced threshold based keyframe selection algorithm and customized CNN on UCF101 and Sports1-M dataset,” Comput. Intell. Neurosci., vol. 2022, pp. 1–15, 2022.
N. Tasnim and J.-H. Baek, “Deep learning-based human action recognition with key-frames sampling using ranking methods,” Appl. Sci. (Basel), vol. 12, no. 9, p. 4165, 2022.
G. A. S. Surek, L. O. Seman, S. F. Stefenon, V. C. Mariani, and L. dos S. Coelho, “Video-based human activity recognition using deep learning approaches,” Sensors (Basel), vol. 23, no. 14, p. 6384, 2023.
M. Rapczyński, P. Werner, S. Handrich, and A. Al-Hamadi, “A baseline for cross-database 3D human pose estimation,” Sensors (Basel), vol. 21, no. 11, p. 3769, 2021.
S. Chen, Y. Xu, and B. Zou, “Prior-knowledge-based self-attention network for 3D human pose estimation,” Expert Syst. Appl., vol. 225, no. 120213, p. 120213, 2023.
K. de Langis, M. Fulton, and J. Sattar, “Video Diver Dataset (VDD-C), 100,000 annotated images of divers underwater.” Data Repository for the University of Minnesota (DRUM), 2021.
M. Nithya and K. S. Gautam, “An alternative frame selection approach for motion detection,” Int. J. Adv. Sci. Technol., vol. 29, no. 7, pp. 13408–13415, 2020.
Y. Cheng, B. Yang, B. Wang, and R. T. Tan, “3D human pose estimation using spatio-temporal networks with explicit occlusion training,” Proc. Conf. AAAI Artif. Intell., vol. 34, no. 07, pp. 10631–10638, 2020.
Y. Cheng, B. Yang, B. Wang, Y. Wending, and R. Tan, “Occlusion-aware networks for 3D human pose estimation in video,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 2019.
F. Angelini, Z. Fu, Y. Long, L. Shao, and S. M. Naqvi, “2D pose-based real-time human action recognition with occlusion-handling,” IEEE Trans. Multimedia, vol. 22, no. 6, pp. 1433–1446, 2020.
“Pose landmark detection guide,” Google for Developers. [Online]. Available: https://developers.google.com/mediapipe/solutions/vision/pose_landmarker. [Accessed: 27-Jan-2024].
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
