Predicting Pedestrian Behavior at Zebra Crossings using Bottom-up Pose Estimation and Deep Learning
Keywords:
Pedestrians’ pose estimation, behavioral analysis, Advanced Driver Assistance Systems, autonomous vehicles, Pedestrian Behavior classification..Abstract
Anticipating pedestrian behavior is critical for traffic management, developing Advanced Driver Assistance Systems (ADAS), and creating autonomous vehicles. However, the unpredictability of pedestrians at zebra crossings poses a significant challenge in designing systems that can aid drivers or enable self-driving. Existing studies often overlook pedestrian behavior and intentions when predicting motion, and there is no integrated system that connects perception and decision-making tasks. To address these challenges, we propose a new bottom-up pedestrian Pose Estimation model based on a CNN network that is trained with the deep learning VGG-19 Pretrained model. This model allows for the analysis of videos captured at zebra crossings and enables the detection and classification of pedestrian poses and movements such as walking, standing, hand signals, crossing, and not crossing. We train and evaluate our models on the pedestrian intention estimation (PIE) dataset using the COCO-18 key point model. Our approach provides a comprehensive solution for predicting pedestrian behavior at zebra crossings. Machine learning-based classifiers are used to compare classification performance across different prediction horizon values, resulting in improved accuracy and efficiency. Our proposed solution has significant implications for traffic management, ADAS, and autonomous vehicles, as it enables them to better anticipate and respond to pedestrian actions. Overall, this study highlights the importance of integrating perception and decision-making tasks in predicting pedestrian behavior and provides a promising solution for addressing this critical problem.
Downloads
References
“Road Traffic Injuries.” https://www.who.int/news-room/fact-sheets/detail/road-traffic-injuries, www.who.int/news-room/fact-sheets/detail/road-traffic-injuries. Accessed 12 Nov. 2022.
Veena Hosamani, & Dr. H S Vimala. (2018). Data Science: Prediction and Analysis of Data using Multiple Classifier System. International Journal of Computer Engineering in Research Trends, 5(12), 216–222.
Sethuraman, R., Sellappan, S., Shunmugiah, J., Subbiah, N., Govindarajan, V., & Neelagandan, S.An Optimized AdaBoost Multi-class Support Vector Machine for Driver Behavior Monitoring in the Advanced Driver Assistance Systems. Expert Systems with Applications, 118618 (2022).
https://doi.org/10.1016/j.eswa.2022.118618
Schwalb, E. Analysis of Hazards for Autonomous Driving. Journal of Autonomous Vehicles and Systems, 1(2).(2021) https://doi.org/10.1115/1.4049922
A, B. Improvised Autonomous Car for a Safe Travel. Current Trends in Biomedical Engineering & Biosciences, 6 (2017).
https://doi.org/10.19080/ctbeb.2017.06.555678
Zhang, H., Liu, Y., Wang, C., Fu, R., Sun, Q., & Li, Z. Research on a Pedestrian Crossing Intention Recognition Model Based on Natural Observation Data. Sensors, 20, 1776(2020) https://doi.org/10.3390/s20061776
S D, V. S., & C J, P. (2023). A Study on Vision Based Lane Detection Methods for Advanced Driver Assistance Systems. International Journal of Computer Engineering in Research Trends, 10(8), 1–10.
https://doi.org/10.22362/ijcert.v10i8.750
Zhang, S., Klein, D. A., Bauckhage, C., & Cremers, A. B. Fast moving pedestrian detection based on motion segmentation and new motion features. Multimedia Tools and Applications, 75, 6263–6282(2015).
doi:10.1007/s11042-015-2571-z
D.Sreedevi, Prof.K.Samatha, & Prof.M.P.Rao. (2019). A Review on Typical and Modern Brain MRI Image Segmentation Methods and Challenges. International Journal of Computer Engineering in Research Trends, 6(5), 322–329.
M Bhavsingh, & S.Jan Reddy. (2023). Enhancing Safety and Security: Real-Time Weapon Detection in CCTV Footage Using YOLOv7. International Journal of Computer Engineering in Research Trends, 10(6), 1–8.
https://doi.org/10.22362/ijcert.v10i6.855
Chen, W., Jiang, Z., Guo, H., & Ni, X. Fall Detection Based on Key Points of Human-Skeleton Using OpenPose. Symmetry, 12, 744(2020).
https://doi.org/10.3390/sym12050744
Wu, Y., Zhang, X., & Fang, F. Automatic Fabric Defect Detection Using Cascaded Mixed Feature Pyramid with Guided Localization. Sensors, 20, 871(2020). https://doi.org/10.3390/s20030871
Zheng, H., & Liu, Y. Lightweight Fall Detection Algorithm Based on AlphaPose Optimization Model and ST-GCN. Mathematical Problems in Engineering, 1–15(2022). https://doi.org/10.1155/2022/9962666
S, N., N, P., & P, N. (2023). A Study on Flower Classification Using Deep Learning Techniques. International Journal of Computer Engineering in Research Trends, 10(4), 161–166.
Li, B., Ji, Y., Li, Y., Xu, Y., & Liu, C.Pose Knowledge Transfer for multi-person pose estimation. Signal, Image and Video Processing, 16, 321–328(2021). https://doi.org/10.1007/s11760-021-01922-5
Han, Y., & Wang, G. Skeletal bone age prediction based on a deep residual network with spatial transformer. Computer Methods and Programs in Biomedicine, 197, 105754(2020).
https://doi.org/10.1016/j.cmpb.2020.105754
Angadi, S., & Nandyal, S. Human Identification Using Histogram Of Oriented Gradients (Hog) And Non-Maximum Suppression (Nms) For Atm Video Surveillance. International Journal of Innovative Research in Computer Science and; Technology, 9,(2021)https://doi.org/10.21276/ijircst.2021.9.3.1
M Bhavsingh, B.Pannalal, & K Samunnisa. (2022). Review: Pedestrian Behavior Analysis and Trajectory Prediction with Deep Learning. International Journal of Computer Engineering in Research Trends, 9(12), 263–268
Zhang, Y., Liu, W., Yang, X., & Xing, S. Hidden Markov Model-based Pedestrian Navigation System using MEMS Inertial Sensors. Measurement Science Review, 15,35–43(2015). https://doi.org/10.1515/msr-2015-0006
Martinez-Gonzalez, A. N., Villamizar, M., Canevet, O., & Odobez, J. M. Efficient Convolutional Neural Networks for Depth-Based Multi-Person Pose Estimation. IEEE Transactions on Circuits and Systems for Video Technology, 30,4207–4221(2020).
https://doi.org/10.1109/tcsvt.2019.2952779
Fang, Z., & Lopez, A. M. Intention Recognition of Pedestrians and Cyclists by 2D Pose Estimation. IEEE Transactions on Intelligent Transportation Systems, 1–11(2019).
Munea, T. L., Jembre, Y. Z., Weldegebriel, H. T., Chen, L., Huang, C., & Yang, C. The Progress of Human Pose Estimation: A Survey and Taxonomy of Models Applied in 2D Human Pose Estimation. IEEE Access, 8, 133330–133348(2020). https://doi.org/10.1109/access.2020.3010248
Cheng, Y., Yang, B., Wang, B., & Tan, R. T. 3D Human Pose Estimation Using Spatio-Temporal Networks with Explicit Occlusion Training. Proceedings of the AAAI Conference on Artificial Intelligence, 34, 10631–10638 (2020). https://doi.org/10.1609/aaai.v34i07.6689
Liu, W., & Mei, T. Recent Advances of Monocular 2D and 3D Human Pose Estimation: A Deep Learning Perspective. ACM Computing Surveys.(2022) https://doi.org/10.1145/3524497
Soccolich, S. A., Fitch, G. M., Perez, M. A., & Hanowski, R. J. Comparing Handheld and Hands-free Cell Phone Usage Behaviors While Driving. Traffic Injury Prevention, 15(sup1), S21–S26(2014). https://doi.org/10.1080/15389588.2014.934958
Zhang, S., Abdel-Aty, M., Wu, Y., & Zheng, O. Pedestrian Crossing Intention Prediction at Red-Light Using Pose Estimation. IEEE Transactions on Intelligent Transportation Systems, 23, 2331–2339(2022). https://doi.org/10.1109/tits.2021.3074829
Ju, H., & Park, C. G. A pedestrian dead reckoning system using a foot kinematic constraint and shoe modeling for various motions. Sensors and Actuators A: Physical, 284, 135–144(2018). https://doi.org/10.1016/j.sna.2018.09.043
Ahmed, T., Moeinaddini, M., Almoshaogeh, M., Jamal, A., Nawaz, I., & Alharbi, F. A New Pedestrian Crossing Level of Service (PCLOS) Method for Promoting Safe Pedestrian Crossing in Urban Areas. International Journal of Environmental Research and Public Health, 18, 8813(2021). https://doi.org/10.3390/ijerph18168813
A., S. R., S., A., R., H., & S., S. K. Stacking Deep learning and Machine learning models for short-term energy consumption forecasting. Advanced Engineering Informatics, 52,101542(2022). https://doi.org/10.1016/j.aei.2022.101542
Hartwig, L., Kaufmann, C., Risser, R., Erbsmehl, C., Landgraf, T., Urban, M., & Schreiber, D. Evaluating Pedestrian and Cyclist Behaviour at a Level Crossing. Transactions on Transport Sciences, 11, 41–54(2020). https://doi.org/10.5507/tots.2019.009
Hariyono, J., & Jo, K. H. . Detection of pedestrian crossing road: A study on pedestrian pose recognition. Neurocomputing, 234, 144–153(2017). https://doi.org/10.1016/j.neucom.2016.12.050
LI, W., ZHAO, X., & SUN, D. Prediction of trajectory based on modified Bayesian inference. Journal of Computer Applications, 33, 1960–1963(2013). https://doi.org/10.3724/sp.j.1087.2013.01960
Min, J., Lim, E., & Son, S. The effect of airline kiosk service on behavior intention : Apply the theory of planned behavior(TPB). International Journal of Tourism and Hospitality Research, 36, 187–197(2022). https://doi.org/10.21298/ijthr.2022.1.36.1.187
Kalatian, A., Sobhani, A., & Farooq, B. Analysis of distracted pedestrians’ waiting time: Head-Mounted Immersive Virtual Reality application. Collective Dynamics, 5.(2020) https://doi.org/10.17815/cd.2020.32
Rasouli, Amir & Kotseruba, Iuliia & Kunic, Toni & Tsotsos, John. PIE: A Large-Scale Dataset and Models for Pedestrian Intention Estimation and Trajectory Prediction. 6261-6270(2019). 10.1109/ICCV.2019.00636.
Cao, Z., Hidalgo, G., Simon, T., Wei, S. E., & Sheikh, Y. OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43, 172–186(2021). https://doi.org/10.1109/tpami.2019.2929257.
[ Diniesh, V. C. ., Prasad, L. V. R. C. ., Bharathi , R. J. ., Selvarani, A., Theresa, W. G. ., Sumathi, R. ., & Dhanalakshmi, G. . (2023). Performance Evaluation of Energy Efficient Optimized Routing Protocol for WBANs Using PSO Protocol. International Journal on Recent and Innovation Trends in Computing and Communication, 11(4s), 116–121. https://doi.org/10.17762/ijritcc.v11i4s.6314
Mr. A. Kingsly Jabakumar. (2019). Enhanced QoS and QoE Support through Energy Efficient Handover Algorithm for UMTS Architectures. International Journal of New Practices in Management and Engineering, 8(01), 01 - 07. https://doi.org/10.17762/ijnpme.v8i01.73
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
