Predicting Pedestrian Behavior at Zebra Crossings using Bottom-up Pose Estimation and Deep Learning


  • Pannalal Boda Research Scholar, CSE Department Osmania, University,Hyderabad, Telangana, India.
  • Y. Ramadevi Professor, CSE Dept, Chaitanya Bharathi Institute of Technology, Osmania University, Hyderabad, Telangana, India.


Pedestrians’ pose estimation, behavioral analysis, Advanced Driver Assistance Systems, autonomous vehicles, Pedestrian Behavior classification..


Anticipating pedestrian behavior is critical for traffic management, developing Advanced Driver Assistance Systems (ADAS), and creating autonomous vehicles. However, the unpredictability of pedestrians at zebra crossings poses a significant challenge in designing systems that can aid drivers or enable self-driving. Existing studies often overlook pedestrian behavior and intentions when predicting motion, and there is no integrated system that connects perception and decision-making tasks. To address these challenges, we propose a new bottom-up pedestrian Pose Estimation model based on a CNN network that is trained with the deep learning VGG-19 Pretrained model. This model allows for the analysis of videos captured at zebra crossings and enables the detection and classification of pedestrian poses and movements such as walking, standing, hand signals, crossing, and not crossing. We train and evaluate our models on the pedestrian intention estimation (PIE) dataset using the COCO-18 key point model. Our approach provides a comprehensive solution for predicting pedestrian behavior at zebra crossings. Machine learning-based classifiers are used to compare classification performance across different prediction horizon values, resulting in improved accuracy and efficiency. Our proposed solution has significant implications for traffic management, ADAS, and autonomous vehicles, as it enables them to better anticipate and respond to pedestrian actions. Overall, this study highlights the importance of integrating perception and decision-making tasks in predicting pedestrian behavior and provides a promising solution for addressing this critical problem.


Download data is not yet available.


“Road Traffic Injuries.”, Accessed 12 Nov. 2022.

Veena Hosamani, & Dr. H S Vimala. (2018). Data Science: Prediction and Analysis of Data using Multiple Classifier System. International Journal of Computer Engineering in Research Trends, 5(12), 216–222.

Sethuraman, R., Sellappan, S., Shunmugiah, J., Subbiah, N., Govindarajan, V., & Neelagandan, S.An Optimized AdaBoost Multi-class Support Vector Machine for Driver Behavior Monitoring in the Advanced Driver Assistance Systems. Expert Systems with Applications, 118618 (2022).

Schwalb, E. Analysis of Hazards for Autonomous Driving. Journal of Autonomous Vehicles and Systems, 1(2).(2021)

A, B. Improvised Autonomous Car for a Safe Travel. Current Trends in Biomedical Engineering & Biosciences, 6 (2017).

Zhang, H., Liu, Y., Wang, C., Fu, R., Sun, Q., & Li, Z. Research on a Pedestrian Crossing Intention Recognition Model Based on Natural Observation Data. Sensors, 20, 1776(2020)

S D, V. S., & C J, P. (2023). A Study on Vision Based Lane Detection Methods for Advanced Driver Assistance Systems. International Journal of Computer Engineering in Research Trends, 10(8), 1–10.

Zhang, S., Klein, D. A., Bauckhage, C., & Cremers, A. B. Fast moving pedestrian detection based on motion segmentation and new motion features. Multimedia Tools and Applications, 75, 6263–6282(2015).


D.Sreedevi, Prof.K.Samatha, & Prof.M.P.Rao. (2019). A Review on Typical and Modern Brain MRI Image Segmentation Methods and Challenges. International Journal of Computer Engineering in Research Trends, 6(5), 322–329.

M Bhavsingh, & S.Jan Reddy. (2023). Enhancing Safety and Security: Real-Time Weapon Detection in CCTV Footage Using YOLOv7. International Journal of Computer Engineering in Research Trends, 10(6), 1–8.

Chen, W., Jiang, Z., Guo, H., & Ni, X. Fall Detection Based on Key Points of Human-Skeleton Using OpenPose. Symmetry, 12, 744(2020).

Wu, Y., Zhang, X., & Fang, F. Automatic Fabric Defect Detection Using Cascaded Mixed Feature Pyramid with Guided Localization. Sensors, 20, 871(2020).

Zheng, H., & Liu, Y. Lightweight Fall Detection Algorithm Based on AlphaPose Optimization Model and ST-GCN. Mathematical Problems in Engineering, 1–15(2022).

S, N., N, P., & P, N. (2023). A Study on Flower Classification Using Deep Learning Techniques. International Journal of Computer Engineering in Research Trends, 10(4), 161–166.

Li, B., Ji, Y., Li, Y., Xu, Y., & Liu, C.Pose Knowledge Transfer for multi-person pose estimation. Signal, Image and Video Processing, 16, 321–328(2021).

Han, Y., & Wang, G. Skeletal bone age prediction based on a deep residual network with spatial transformer. Computer Methods and Programs in Biomedicine, 197, 105754(2020).

Angadi, S., & Nandyal, S. Human Identification Using Histogram Of Oriented Gradients (Hog) And Non-Maximum Suppression (Nms) For Atm Video Surveillance. International Journal of Innovative Research in Computer Science and; Technology, 9,(2021)

M Bhavsingh, B.Pannalal, & K Samunnisa. (2022). Review: Pedestrian Behavior Analysis and Trajectory Prediction with Deep Learning. International Journal of Computer Engineering in Research Trends, 9(12), 263–268

Zhang, Y., Liu, W., Yang, X., & Xing, S. Hidden Markov Model-based Pedestrian Navigation System using MEMS Inertial Sensors. Measurement Science Review, 15,35–43(2015).

Martinez-Gonzalez, A. N., Villamizar, M., Canevet, O., & Odobez, J. M. Efficient Convolutional Neural Networks for Depth-Based Multi-Person Pose Estimation. IEEE Transactions on Circuits and Systems for Video Technology, 30,4207–4221(2020).

Fang, Z., & Lopez, A. M. Intention Recognition of Pedestrians and Cyclists by 2D Pose Estimation. IEEE Transactions on Intelligent Transportation Systems, 1–11(2019).

Munea, T. L., Jembre, Y. Z., Weldegebriel, H. T., Chen, L., Huang, C., & Yang, C. The Progress of Human Pose Estimation: A Survey and Taxonomy of Models Applied in 2D Human Pose Estimation. IEEE Access, 8, 133330–133348(2020).

Cheng, Y., Yang, B., Wang, B., & Tan, R. T. 3D Human Pose Estimation Using Spatio-Temporal Networks with Explicit Occlusion Training. Proceedings of the AAAI Conference on Artificial Intelligence, 34, 10631–10638 (2020).

Liu, W., & Mei, T. Recent Advances of Monocular 2D and 3D Human Pose Estimation: A Deep Learning Perspective. ACM Computing Surveys.(2022)

Soccolich, S. A., Fitch, G. M., Perez, M. A., & Hanowski, R. J. Comparing Handheld and Hands-free Cell Phone Usage Behaviors While Driving. Traffic Injury Prevention, 15(sup1), S21–S26(2014).

Zhang, S., Abdel-Aty, M., Wu, Y., & Zheng, O. Pedestrian Crossing Intention Prediction at Red-Light Using Pose Estimation. IEEE Transactions on Intelligent Transportation Systems, 23, 2331–2339(2022).

Ju, H., & Park, C. G. A pedestrian dead reckoning system using a foot kinematic constraint and shoe modeling for various motions. Sensors and Actuators A: Physical, 284, 135–144(2018).

Ahmed, T., Moeinaddini, M., Almoshaogeh, M., Jamal, A., Nawaz, I., & Alharbi, F. A New Pedestrian Crossing Level of Service (PCLOS) Method for Promoting Safe Pedestrian Crossing in Urban Areas. International Journal of Environmental Research and Public Health, 18, 8813(2021).

A., S. R., S., A., R., H., & S., S. K. Stacking Deep learning and Machine learning models for short-term energy consumption forecasting. Advanced Engineering Informatics, 52,101542(2022).

Hartwig, L., Kaufmann, C., Risser, R., Erbsmehl, C., Landgraf, T., Urban, M., & Schreiber, D. Evaluating Pedestrian and Cyclist Behaviour at a Level Crossing. Transactions on Transport Sciences, 11, 41–54(2020).

Hariyono, J., & Jo, K. H. . Detection of pedestrian crossing road: A study on pedestrian pose recognition. Neurocomputing, 234, 144–153(2017).

LI, W., ZHAO, X., & SUN, D. Prediction of trajectory based on modified Bayesian inference. Journal of Computer Applications, 33, 1960–1963(2013).

Min, J., Lim, E., & Son, S. The effect of airline kiosk service on behavior intention : Apply the theory of planned behavior(TPB). International Journal of Tourism and Hospitality Research, 36, 187–197(2022).

Kalatian, A., Sobhani, A., & Farooq, B. Analysis of distracted pedestrians’ waiting time: Head-Mounted Immersive Virtual Reality application. Collective Dynamics, 5.(2020)

Rasouli, Amir & Kotseruba, Iuliia & Kunic, Toni & Tsotsos, John. PIE: A Large-Scale Dataset and Models for Pedestrian Intention Estimation and Trajectory Prediction. 6261-6270(2019). 10.1109/ICCV.2019.00636.

Cao, Z., Hidalgo, G., Simon, T., Wei, S. E., & Sheikh, Y. OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43, 172–186(2021).

[ Diniesh, V. C. ., Prasad, L. V. R. C. ., Bharathi , R. J. ., Selvarani, A., Theresa, W. G. ., Sumathi, R. ., & Dhanalakshmi, G. . (2023). Performance Evaluation of Energy Efficient Optimized Routing Protocol for WBANs Using PSO Protocol. International Journal on Recent and Innovation Trends in Computing and Communication, 11(4s), 116–121.

Mr. A. Kingsly Jabakumar. (2019). Enhanced QoS and QoE Support through Energy Efficient Handover Algorithm for UMTS Architectures. International Journal of New Practices in Management and Engineering, 8(01), 01 - 07.




How to Cite

Boda, P. ., & Ramadevi , Y. . (2023). Predicting Pedestrian Behavior at Zebra Crossings using Bottom-up Pose Estimation and Deep Learning. International Journal of Intelligent Systems and Applications in Engineering, 12(4s), 527–544. Retrieved from



Research Article