Design of an Ensemble learning Model to Recognize Multilingual Hand Gestures via Interlinked Key Point analysis
Keywords:
Gesture Recognition, Multilingual Hand Gestures, Ensemble Learning, Interlinked Key Point Analysis, Ant Lion OptimizationAbstract
This paper addresses the urgent need for multilingual hand gesture detection that is accurate and efficient for use in real-time applications. The precise recognition of hand gestures across several languages becomes essential with the growing integration of gesture-based interfaces in a variety of applications, such as virtual reality, augmented reality, and human-computer interaction. When dealing with continuous gestures and many characters per gesture, present models struggle to achieve high precision, accuracy, and recall. The suggested solution employs a unique ensemble learning model that takes advantage of interconnected key point analysis of single hand motions to get around the shortcomings of the existing algorithms. A complete set of 441 distance characteristics is achieved by modeling hand motions using 20 key points, including 4 key points for each finger and one for the hand's center. Then, linguistic characters in Hindi, English, and Marathi are attached to these features. Using Ant Lion Optimization, which ensures high variance feature sets are used in the ensuing training of the ensemble learning model, the selection procedure is carried out in order to maintain the most discriminative and informative features. A variety of classifiers, including Naive Bayes (NB), k-Nearest Neighbors (kNN), Support Vector Machines (SVM), Multilayer Perceptron (MLP), and Logistic Regression (LR), are combined in the ensemble learning approach. This combination of classifiers strengthens the model's general robustness and increases its capacity to generalize successfully across many languages. Additionally, by incorporating a state machine that permits the seamless processing of continuous gestures, the model successfully handles characters that require numerous gestures. The suggested model's remarkable performance metrics and computational effectiveness are its main benefits. The results of the experiments show a remarkable precision of 99.8%, accuracy of 99.5%, and recall of 99.4% with delay lower than 2.5 ms for different use cases. These outcomes outperform the effectiveness of current approaches, making the suggested paradigm very effective for real-time scenarios and enabling fluid interaction with gesture-based interfaces. The paper concludes by presenting a novel ensemble learning model that, through interconnected key point analysis, efficiently recognizes multilingual hand motions. The model performs remarkably well because to the integration of many classifiers and Ant Lion Optimization feature selection, making it ideal for real-time applications and making a substantial addition to the study of gesture detection process.
Downloads
References
Q. Gao, Y. Chen, Z. Ju and Y. Liang, "Dynamic Hand Gesture Recognition Based on 3D Hand Pose Estimation for Human–Robot Interaction," in IEEE Sensors Journal, vol. 22, no. 18, pp. 17421-17430, 15 Sept.15, 2022, doi: 10.1109/JSEN.2021.3059685.
N. P. Sable, M. D. Salunke, V. U. Rathod and P. Dhotre, "Network for Cross-Disease Attention to the Severity of Diabetic Macular Edema and Joint Retinopathy," 2022 International Conference on Smart Generation Computing, Communication and Networking (SMART GENCON), Bangalore, India, 2022, pp. 1-7, doi: 10.1109/SMARTGENCON56628.2022.10083936.
D. G. León et al., "Video Hand Gestures Recognition Using Depth Camera and Lightweight CNN," in IEEE Sensors Journal, vol. 22, no. 14, pp. 14610-14619, 15 July15, 2022, doi: 10.1109/JSEN.2022.3181518.
N. P. Sable, V. U. Rathod, P. N. Mahalle, and P. N. Railkar, “An Efficient and Reliable Data Transmission Service using Network Coding Algorithms in Peer-to-Peer Network”, IJRITCC, vol. 10, no. 1s, pp. 144–154, Dec. 2022.
P. Kang, J. Li, B. Fan, S. Jiang and P. B. Shull, "Wrist-Worn Hand Gesture Recognition While Walking via Transfer Learning," in IEEE Journal of Biomedical and Health Informatics, vol. 26, no. 3, pp. 952-961, March 2022, doi: 10.1109/JBHI.2021.3100099.
R. R. Sharma, K. A. Kumar and S. H. Cho, "Novel Time-Distance Parameters Based Hand Gesture Recognition System Using Multi-UWB Radars," in IEEE Sensors Letters, vol. 7, no. 5, pp. 1-4, May 2023, Art no. 6002204, doi: 10.1109/LSENS.2023.3268065.
H. Zhou, C. Tawk and G. Alici, "A 3D Printed Soft Robotic Hand With Embedded Soft Sensors for Direct Transition Between Hand Gestures and Improved Grasping Quality and Diversity," in IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 30, pp. 550-558, 2022, doi: 10.1109/TNSRE.2022.3156116.
V. U. Rathod and S. V. Gumaste, “Role of Deep Learning in Mobile Ad-hoc Networks”, IJRITCC, vol. 10, no. 2s, pp. 237–246, Dec. 2022.
Y. Zou, L. Cheng, L. Han, Z. Li and L. Song, "Decoding Electromyographic Signal With Multiple Labels for Hand Gesture Recognition," in IEEE Signal Processing Letters, vol. 30, pp. 483-487, 2023, doi: 10.1109/LSP.2023.3264417.
T. Wang, Y. Zhao and Q. Wang, "Hand Gesture Recognition With Flexible Capacitive Wristband Using Triplet Network in Inter-Day Applications," in IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 30, pp. 2876-2885, 2022, doi: 10.1109/TNSRE.2022.3212705.
G. Park, V. K. Chandrasegar and J. Koh, "Accuracy Enhancement of Hand Gesture Recognition Using CNN," in IEEE Access, vol. 11, pp. 26496-26501, 2023, doi: 10.1109/ACCESS.2023.3254537.
Z. Wang, G. Li and L. Yang, "Dynamic Hand Gesture Recognition Based on Micro-Doppler Radar Signatures Using Hidden Gauss–Markov Models," in IEEE Geoscience and Remote Sensing Letters, vol. 18, no. 2, pp. 291-295, Feb. 2021, doi: 10.1109/LGRS.2020.2974821.
Q. Gao, Z. Ju, Y. Chen, Q. Wang and C. Chi, "An Efficient RGB-D Hand Gesture Detection Framework for Dexterous Robot Hand-Arm Teleoperation System," in IEEE Transactions on Human-Machine Systems, vol. 53, no. 1, pp. 13-23, Feb. 2023, doi: 10.1109/THMS.2022.3206663.
J. Shen, J. Dudley, G. Mo and P. O. Kristensson, "Gesture Spotter: A Rapid Prototyping Tool for Key Gesture Spotting in Virtual and Augmented Reality Applications," in IEEE Transactions on Visualization and Computer Graphics, vol. 28, no. 11, pp. 3618-3628, Nov. 2022, doi: 10.1109/TVCG.2022.3203004.
J. Shen, J. Dudley, G. Mo and P. O. Kristensson, "Gesture Spotter: A Rapid Prototyping Tool for Key Gesture Spotting in Virtual and Augmented Reality Applications," in IEEE Transactions on Visualization and Computer Graphics, vol. 28, no. 11, pp. 3618-3628, Nov. 2022, doi: 10.1109/TVCG.2022.3203004.
X. Chen, Y. Li, R. Hu, X. Zhang and X. Chen, "Hand Gesture Recognition based on Surface Electromyography using Convolutional Neural Network with Transfer Learning Method," in IEEE Journal of Biomedical and Health Informatics, vol. 25, no. 4, pp. 1292-1304, April 2021, doi: 10.1109/JBHI.2020.3009383.
G. Chen et al., "A Novel Illumination-Robust Hand Gesture Recognition System With Event-Based Neuromorphic Vision Sensor," in IEEE Transactions on Automation Science and Engineering, vol. 18, no. 2, pp. 508-520, April 2021, doi: 10.1109/TASE.2020.3045880.
C. Zhu, J. Yang, Z. Shao and C. Liu, "Vision Based Hand Gesture Recognition Using 3D Shape Context," in IEEE/CAA Journal of Automatica Sinica, vol. 8, no. 9, pp. 1600-1613, September 2021, doi: 10.1109/JAS.2019.1911534.
H. Wang, P. Kang, Q. Gao, S. Jiang and P. B. Shull, "A Novel PPG-FMG-ACC Wristband for Hand Gesture Recognition," in IEEE Journal of Biomedical and Health Informatics, vol. 26, no. 10, pp. 5097-5108, Oct. 2022, doi: 10.1109/JBHI.2022.3194017.
N. P. Sable, V. U. Rathod, P. N. Mahalle and D. R. Birari, "A Multiple Stage Deep Learning Model for NID in MANETs," 2022 International Conference on Emerging Smart Computing and Informatics (ESCI), Pune, India, 2022, pp. 1-6, doi: 10.1109/ESCI53509.2022.9758191.
J. E. Lara, L. K. Cheng, O. Röhrle and N. Paskaranandavadivel, "Muscle-Specific High-Density Electromyography Arrays for Hand Gesture Classification," in IEEE Transactions on Biomedical Engineering, vol. 69, no. 5, pp. 1758-1766, May 2022, doi: 10.1109/TBME.2021.3131297.
J. E. Lara, L. K. Cheng, O. Röhrle and N. Paskaranandavadivel, "Muscle-Specific High-Density Electromyography Arrays for Hand Gesture Classification," in IEEE Transactions on Biomedical Engineering, vol. 69, no. 5, pp. 1758-1766, May 2022, doi: 10.1109/TBME.2021.3131297.
T. -Y. Pan, W. -L. Tsai, C. -Y. Chang, C. -W. Yeh and M. -C. Hu, "A Hierarchical Hand Gesture Recognition Framework for Sports Referee Training-Based EMG and Accelerometer Sensors," in IEEE Transactions on Cybernetics, vol. 52, no. 5, pp. 3172-3183, May 2022, doi: 10.1109/TCYB.2020.3007173.
V. U. Rathod and S. V. Gumaste, "Role of Routing Protocol in Mobile Ad-Hoc Network for Performance of Mobility Models," 2023 IEEE 8th International Conference for Convergence in Technology (I2CT), Lonavla, India, 2023, pp. 1-6, doi: 10.1109/I2CT57861.2023.10126390.
X. Zeng, C. Wu and W. -B. Ye, "User-Definable Dynamic Hand Gesture Recognition Based on Doppler Radar and Few-Shot Learning," in IEEE Sensors Journal, vol. 21, no. 20, pp. 23224-23233, 15 Oct.15, 2021, doi: 10.1109/JSEN.2021.3107943.
Z. Yang and X. Zheng, "Hand Gesture Recognition Based on Trajectories Features and Computation-Efficient Reused LSTM Network," in IEEE Sensors Journal, vol. 21, no. 15, pp. 16945-16960, 1 Aug.1, 2021, doi: 10.1109/JSEN.2021.3079564.
D. S. Breland, A. Dayal, A. Jha, P. K. Yalavarthy, O. J. Pandey and L. R. Cenkeramaddi, "Robust Hand Gestures Recognition Using a Deep CNN and Thermal Images," in IEEE Sensors Journal, vol. 21, no. 23, pp. 26602-26614, 1 Dec.1, 2021, doi: 10.1109/JSEN.2021.3119977.
M. Garg, D. Ghosh and P. M. Pradhan, "Multiscaled Multi-Head Attention-Based Video Transformer Network for Hand Gesture Recognition," in IEEE Signal Processing Letters, vol. 30, pp. 80-84, 2023, doi: 10.1109/LSP.2023.3241857.
N. P. Sable, R. Sonkamble, V. U. Rathod, S. Shirke, J. Y. Deshmukh, and G. T. . Chavan, “Web3 Chain Authentication and Authorization Security Standard (CAA)”, IJRITCC, vol. 11, no. 5, pp. 70–76, May 2023.
Y. Wang, Y. Shu, X. Jia, M. Zhou, L. Xie and L. Guo, "Multifeature Fusion-Based Hand Gesture Sensing and Recognition System," in IEEE Geoscience and Remote Sensing Letters, vol. 19, pp. 1-5, 2022, Art no. 3507005, doi: 10.1109/LGRS.2021.3086136.
W. Qi, S. E. Ovur, Z. Li, A. Marzullo and R. Song, "Multi-Sensor Guided Hand Gesture Recognition for a Teleoperated Robot Using a Recurrent Neural Network," in IEEE Robotics and Automation Letters, vol. 6, no. 3, pp. 6039-6045, July 2021, doi: 10.1109/LRA.2021.3089999.
W. Dong, L. Yang, R. Gravina and G. Fortino, "Soft Wrist-Worn Multi-Functional Sensor Array for Real-Time Hand Gesture Recognition," in IEEE Sensors Journal, vol. 22, no. 18, pp. 17505-17514, 15 Sept.15, 2022, doi: 10.1109/JSEN.2021.3050175.
S. Khetavath et al., "An Intelligent Heuristic Manta-Ray Foraging Optimization and Adaptive Extreme Learning Machine for Hand Gesture Image Recognition," in Big Data Mining and Analytics, vol. 6, no. 3, pp. 321-335, September 2023, doi: 10.26599/BDMA.2022.9020036.
B. Qiang et al., "SqueezeNet and Fusion Network-Based Accurate Fast Fully Convolutional Network for Hand Detection and Gesture Recognition," in IEEE Access, vol. 9, pp. 77661-77674, 2021, doi: 10.1109/ACCESS.2021.3079337.
Pallavi Parlewar et al., “Depth Estimation using FCNN”, Bioscience Biotechnology Research Communications, Vol. 13, Iss.14, pp 372-375 DOI:10.21786/bbrc/13.14/86.
N. P. Sable, V. U. Rathod, R. Sable and G. R. Shinde, "The Secure E-Wallet Powered by Blockchain and Distributed Ledger Technology," 2022 IEEE Pune Section International Conference (PuneCon), Pune, India, 2022, pp. 1-5, doi: 10.1109/PuneCon55413.2022.10014893.
Prof. Arun Pawar, Mr. Dharmesh Dhabliya. (2018). Intelligent Modulation Recognition System and its Implementation using MATLAB. International Journal of New Practices in Management and Engineering, 7(01), 08 - 14. https://doi.org/10.17762/ijnpme.v7i01.63
Rodríguez, M., Jovanović, A., Petrova, M., Merwe, M. van der, & Levi, S. Predicting Customer Lifetime Value with Regression Models. Kuwait Journal of Machine Learning, 1(4). Retrieved from http://kuwaitjournals.com/index.php/kjml/article/view/144
Jain, V., Beram, S. M., Talukdar, V., Patil, T., Dhabliya, D., & Gupta, A. (2022). Accuracy enhancement in machine learning during blockchain based transaction classification. Paper presented at the PDGC 2022 - 2022 7th International Conference on Parallel, Distributed and Grid Computing, 536-540. doi:10.1109/PDGC56933.2022.10053213 Retrieved from www.scopus.com
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
