Enhancing Driver Drowsiness Detection: A Fusion of Facial Landmarks and Modified YOLOv5 Architecture
Keywords:
Drowsiness Detection, You Only Look Once (YOLO) v5, Face Detection, Facial Landmark, Data AugmentationAbstract
The driver drowsiness detection system aims to enhance road safety by preventing accidents caused by driver fatigue. Despite progress in drowsiness detection using various approaches, existing methods often lack accuracy in capturing subtle aspects like facial expressions, eye movement patterns, micro head gestures, changes in blink frequency, and variations in steering control of driver behavior can often provide crucial insights into their level of alertness and potential drowsiness while operating a vehicle. This work introduces a novel framework that combines facial landmarks and the Yolov5 architecture to enhance drowsiness detection. By extracting relevant facial features using a modified Yolov5 architecture, the system gains a comprehensive understanding of the driver's state during operation, enabling it to detect even subtle indicators of drowsiness. The framework's integration with facial landmarks allows for the observation of minute changes in facial expressions, providing valuable insights into the driver's level of alertness. The framework was evaluated on the benchmark UTA and custom dataset, where the proposed model achieved an accuracy of 95.5% and 96.4% respectively. In comparison with the state-of-the-art techniques, the proposed system achieves an improvement of 3.2%.
Downloads
References
Wörle, J., Metz, B., Thiele, C., & Weller, G. (2019). Detecting sleep in drivers during highly automated driving: The potential of physiological parameters. IET Intelligent Transport Systems, 13(8), 1241-1248.
De Mello, M. T., Narciso, F. V., Tufik, S., Paiva, T., Spence, D. W., BaHammam, A. S., ... & Pandi-Perumal, S. R. (2013). Sleep disorders as a cause of motor vehicle collisions. International journal of preventive medicine, 4(3), 246.
Dong, Y., Hu, Z., Uchimura, K., & Murayama, N. (2010). Driver inattention monitoring system for intelligent vehicles: A review. IEEE transactions on intelligent transportation systems, 12(2), 596-614.
Ziebarth, M. (2020). NY v. national highway traffic safety admin., 974 F. 3d 87 (2d Cir. 2020). Transp. LJ, 47, 87.
Kumar, A., Kalita, D. J., & Singh, V. P. (2020, February). A modern pothole detection technique using deep learning. In 2nd International Conference on Data, Engineering and Applications (IDEA) (pp. 1-5). IEEE.
Capitaine, M. P., & Cárdenas, H. M. G. (2023). Artificial intelligence and advanced driver assistance systems absorption (ADAS) in Mexico. Ciencia Nicolaita, (88).
Tibrewal, M., Srivastava, A., & Kayalvizhi, R. (2021). A deep learning approach to detect driver drowsiness. Int. J. Eng. Res. Technol, 10, 183-189.
Panwar, P., Roshan, P., Singh, R., Rai, M., Mishra, A. R., & Chauhan, S. S. (2022). DDNet-A Deep Learning Approach to Detect Driver Distraction and Drowsiness.
Cao, Y., Li, F., Liu, X., Yang, S., & Wang, Y. (2023). Towards reliable driver drowsiness detection leveraging wearables. ACM Transactions on Sensor Networks, 19(2), 1-23.
Yang, C., Yang, Z., Li, W., & See, J. (2022). FatigueView: A Multi-Camera Video Dataset for Vision-Based Drowsiness Detection. IEEE Transactions on Intelligent Transportation Systems, 24(1), 233-246.
Bassil, M., Perrine, K. A., & Machemehl, R. B. (2023). Synthesis of Automated Multimodal Data Collection Techniques and Applications in the US. In International Conference on Transportation and Development 2023 (pp. 480-488).
Kotseruba, I., & Tsotsos, J. K. (2022). Attention for vision-based assistive and automated driving: a review of algorithms and datasets. IEEE transactions on intelligent transportation systems.
Rajawat, A. S., Goyal, S. B., Bhaladhare, P., Bedi, P., Verma, C., Florin-Emilian, Ț., & Candin, M. T. (2023, May). Real-Time Driver Sleepiness Detection and Classification Using Fusion Deep Learning Algorithm. In Proceedings of International Conference on Recent Innovations in Computing: ICRIC 2022, Volume 1 (pp. 447-457). Singapore: Springer Nature Singapore.
Ayyasamy, S. (2022). A Comprehensive Review on Advanced Driver Assistance System. Journal of Soft Computing Paradigm, 4(2), 69-81.
Ibrahim, M. S., Kamat, S. R., & Shamsuddin, S. (2023). THE APPLICATION OF DRIVING FATIGUE DETECTION AND MONITORING TECHNOLOGIES IN TRANSPORTATION SECTOR: A REVIEW. International Journal of Technology Management and Information System, 5(2), 30-42.
Zhang, Z., Ning, H., & Zhou, F. (2022). A systematic survey of driving fatigue monitoring. IEEE transactions on intelligent transportation systems.
Manfreda, A., Presbury, R., Richardson, S., Melissen, F., & King, J. (2023). Walking the talk: A High Engagement Research implementation framework in the qualitative study of tourism and hospitality experiences. Tourism Management Perspectives, 48, 101142.
Mohsan, S. A. H., Khan, M. A., Noor, F., Ullah, I., & Alsharif, M. H. (2022). Towards the unmanned aerial vehicles (UAVs): A comprehensive review. Drones, 6(6), 147.
Ansari, S., Du, H., Naghdy, F., & Stirling, D. (2022). Automatic driver cognitive fatigue detection based on upper body posture variations. Expert Systems with Applications, 203, 117568.
El-Nabi, S. A., El-Shafai, W., El-Rabaie, E. S. M., Ramadan, K. F., Abd El-Samie, F. E., & Mohsen, S. (2023). Machine learning and deep learning techniques for driver fatigue and drowsiness detection: a review. Multimedia Tools and Applications, 1-37.
Li, K., Yu, R., Liu, Y., Wang, J., & Xue, W. (2023). Correlation analysis and modeling of human thermal sensation with multiple physiological markers: An experimental study. Energy and Buildings, 278, 112643.
Fan, J., Yang, S., Liu, J., Zhu, Z., Xiao, J., Chang, L., ... & Zhou, J. (2022). A High Accuracy & Ultra-Low Power ECG-Derived Respiration Estimation Processor for Wearable Respiration Monitoring Sensor. Biosensors, 12(8), 665.
Sharma, G., Joshi, A. M., Yadav, D., & Mohanty, S. P. (2023). A Smart Healthcare Framework for Accurate Detection of Schizophrenia using Multi-Channel EEG. IEEE Transactions on Instrumentation and Measurement.
Sharma, M., Darji, J., Thakrar, M., & Acharya, U. R. (2022). Automated identification of sleep disorders using wavelet-based features extracted from electrooculogram and electromyogram signals. Computers in Biology and Medicine, 143, 105224.
Farooq, H., Jain, A., & Shukla, M. K. (2022, August). Classification of Sleep using Polysomnography. In Journal of Physics: Conference Series (Vol. 2327, No. 1, p. 012064). IOP Publishing.
Sar, I., Routray, A., & Mahanty, B. (2023). A Review on Existing Technologies for the Identification and Measurement of Abnormal Driving. International Journal of Intelligent Transportation Systems Research, 21(1), 159-177.
Ponnan, S., Theivadas, J. R., HemaKumar, V. S., & Einarson, D. (2022). Driver monitoring and passenger interaction system using wearable device in intelligent vehicle. Computers and Electrical Engineering, 103, 108323.
Bandele, O. O. (2022). Real-time drowsiness detection using computer vision and deep learning techniques (Doctoral dissertation, Dublin, National College of Ireland).
Jan, M. T., Hashemi, A., Jang, J., Yang, K., Zhai, J., Newman, D., ... & Furht, B. (2022, October). Non-intrusive drowsiness detection techniques and their application in detecting early dementia in older drivers. In Proceedings of the Future Technologies Conference (pp. 776-796). Cham: Springer International Publishing.
Guo, B., Hua, Q., Jin, L., Xie, X., Huo, Z., & Wang, H. (2022). Analysis of driving control characteristics in typical road types. Sustainability, 14(2), 782.
Shaik, M. E. (2023). A systematic review on detection and prediction of driver drowsiness. Transportation Research Interdisciplinary Perspectives, 21, 100864.
Mu, Z., Jin, L., Yin, J., & Wang, Q. (2022). Research on a driver fatigue detection model based on image processing. HUMAN-CENTRIC COMPUTING AND INFORMATION SCIENCES, 12.
Basit, M. S., Ahmad, U., Ahmad, J., Ijaz, K., & Ali, S. F. (2022, December). Driver Drowsiness Detection with Region-of-Interest Selection Based Spatio-Temporal Deep Convolutional-LSTM. In 2022 16th International Conference on Open Source Systems and Technologies (ICOSST) (pp. 1-6). IEEE.
Krishna, G. S., Supriya, K., & Vardhan, J. (2022). Vision transformers and YoloV5 based driver drowsiness detection framework. arXiv preprint arXiv:2209.01401.
Jilani, U., Asif, M., Rashid, M., Siddique, A. A., Talha, S. M. U., & Aamir, M. (2022). Traffic congestion classification using GAN-Based synthetic data augmentation and a novel 5-layer convolutional neural network model. Electronics, 11(15), 2290.
Krishna, G. S., Supriya, K., & Vardhan, J. (2022). Vision transformers and YoloV5 based driver drowsiness detection framework. arXiv preprint arXiv:2209.01401.
Lee, C., & An, J. (2023). LSTM-CNN model of drowsiness detection from multiple consciousness states acquired by EEG. Expert Systems with Applications, 213, 119032.
Ghosh, L., Dewan, D., Chowdhury, A., & Konar, A. (2021). Exploration of face-perceptual ability by EEG induced deep learning algorithm. Biomedical Signal Processing and Control, 66, 102368.
Jegham, I., Alouani, I., Khalifa, A. B., & Mahjoub, M. A. (2023). Deep learning-based hard spatial attention for driver in-vehicle action monitoring. Expert Systems with Applications, 219, 119629.
Safarov, F., Akhmedov, F., Abdusalomov, A. B., Nasimov, R., & Cho, Y. I. (2023). Real-time deep learning-based drowsiness detection: leveraging computer-vision and eye-blink analyses for enhanced road safety. Sensors, 23(14), 6459.
Barakoti, S. (2023). Enhancing driving safety using artificial intelligence technology.
Li, M. L., Sun, G. B., & Yu, J. X. (2023). A pedestrian detection network model based on improved YOLOv5. Entropy, 25(2), 381.
S. Bakheet and A. Al-Hamadi, “A framework for instantaneous driver drowsiness detection based on improved hog features and na¨ıve bayesian classification,” Brain Sciences, vol. 11, p. 240, 02 2021
S. Mittal, S. Gupta, Sagar, A. Shamma, I. Sahni, and N. Thakur, “Driver drowsiness detection using machine learning and image processing,” in 2021 9th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO), 2021, pp.
R. Tamanani, R. Muresan, and A. Al-Dweik, “Estimation of driver vigilance status using real-time facial expression and deep learning,” IEEE Sensors Letters, vol. 5, no. 5, pp. 1–4, 2021
A.-C. Phan, N.-H.-Q. Nguyen, T.-N. Trieu, and T.-C. Phan, “An efficient approach for detecting driver drowsiness based on deep learning,” Applied Sciences, vol. 11, no. 18, 2021
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
