Effects of Different Datasets, Models, Face-parts on Accuracy and Performance of Intelligent Facial Expression Recognition Systems
Keywords:
Facial behavior analysis, Facial expression recognition, Datasets, Tools, Models, Complete facial recognition, incomplete facial recognitionAbstract
Facial expression recognition is a crucial area of study in the field of computer vision. Research on nonverbal communication has shown that a significant amount of deliberate information is sent via facial expressions. Facial expression recognition is a crucial field in computer vision that deals with the significant impact of nonverbal communication. Expression recognition has lately been extensively used in the medical and advertising sectors. Difficulties in Facial Emotion Recognition. Facial emotion recognition is a technique that examines facial expressions in static photos and videos to uncover information about an individual's emotional state. The intricacy of facial expressions, the versatile use of the technology in any setting, and the incorporation of emerging technologies like artificial intelligence pose substantial privacy hazards. Facial expressions serve as non-verbal cues, offering indications of human emotions. Deciphering emotional expressions has been a focal point of study in psychology for many years. This study will examine several prior studies that have undertaken comprehensive facial analysis, including both total and partial face recognition, to identify expressions and emotions. The datasets and models used in previous studies, as well as the findings gained, show that employing the whole face yields more accuracy compared to using specific face-parts, which result in lower accuracy ratios. However, emotional identification often does not rely only on the whole face, since it is not always feasible to have the full face available. Contemporary research is now prioritising the identification of facial expressions based on certain facial features. Efficient deep learning algorithms, particularly the CNN algorithm, can do this task.
Downloads
References
S. LokeshNaik, A. Punitha, P. Vijayakarthik, A. Kiran, A. N. Dhangar, B. J. Reddy, et al., "Real Time Facial Emotion Recognition using Deep Learning and CNN," in 2023 International Conference on Computer Communication and Informatics (ICCCI), 2023, pp. 1-5.
D. Duncan, G. Shine, and C. English, "Facial emotion recognition in real time," Computer Science, pp. 1-7, 2016.
A. Patwal, M. Diwakar, A. Joshi, and P. Singh, "Facial expression recognition using DenseNet," in 2022 OITS International Conference on Information Technology (OCIT), 2022, pp. 548-552.
H. Dino, M. B. Abdulrazzaq, S. Zeebaree, A. B. Sallow, R. R. Zebari, H. M. Shukur, et al., "Facial expression recognition based on hybrid feature extraction techniques with different classifiers," TEST Engineering & Management, vol. 83, pp. 22319-22329, 2020.
S. M. Saleem, S. R. Zeebaree, and M. B. Abdulrazzaq, "Real-life dynamic facial expression recognition: a review," in Journal of Physics: Conference Series, 2021, p. 012010.
S. Meriem, A. Moussaoui, and A. Hadid, "Automated facial expression recognition using deep learning techniques: an overview," International Journal of Informatics and Applied Mathematics, vol. 3, pp. 39-53, 2020.
I. Talegaonkar, K. Joshi, S. Valunj, R. Kohok, and A. Kulkarni, "Real time facial expression recognition using deep learning," in Proceedings of international conference on communication and information processing (ICCIP), 2019.
D. Dukić and A. Sovic Krzic, "Real-time facial expression recognition using deep learning with application in the active classroom environment," Electronics, vol. 11, p. 1240, 2022.
M. Sajjad, F. U. M. Ullah, M. Ullah, G. Christodoulou, F. A. Cheikh, M. Hijji, et al., "A comprehensive survey on deep facial expression recognition: challenges, applications, and future guidelines," Alexandria Engineering Journal, vol. 68, pp. 817-840, 2023.
A. Y. Maghari, "Recognition of partially occluded faces using regularized ICA," Inverse Problems in Science and Engineering, vol. 29, pp. 1158-1177, 2021.
S. Ziccardi, F. Crescenzo, and M. Calabrese, "“What is hidden behind the mask?” facial emotion recognition at the time of COVID-19 pandemic in cognitively normal multiple sclerosis patients," Diagnostics, vol. 12, p. 47, 2021.
Y. Nan, J. Ju, Q. Hua, H. Zhang, and B. Wang, "A-MobileNet: An approach of facial expression recognition," Alexandria Engineering Journal, vol. 61, pp. 4435-4444, 2022.
[C. Bisogni, A. Castiglione, S. Hossain, F. Narducci, and S. Umer, "Impact of deep learning approaches on facial expression recognition in healthcare industries," IEEE Transactions on Industrial Informatics, vol. 18, pp. 5619-5627, 2022.
G. Zhao, H. Yang, and M. Yu, "Expression recognition method based on a lightweight convolutional neural network," IEEE Access, vol. 8, pp. 38528-38537, 2020.
J. D. Bodapati, U. Srilakshmi, and N. Veeranjaneyulu, "FERNet: a deep CNN architecture for facial expression recognition in the wild," Journal of The institution of engineers (India): series B, vol. 103, pp. 439-448, 2022.
M. Akhand, S. Roy, N. Siddique, M. A. S. Kamal, and T. Shimamura, "Facial emotion recognition using transfer learning in the deep CNN," Electronics, vol. 10, p. 1036, 2021.
K. Mohan, A. Seal, O. Krejcar, and A. Yazidi, "FER-net: facial expression recognition using deep neural net," Neural Computing and Applications, vol. 33, pp. 9125-9136, 2021.
S. Bellamkonda, N. Gopalan, C. Mala, and L. Settipalli, "Facial expression recognition on partially occluded faces using component based ensemble stacked cnn," Cognitive Neurodynamics, vol. 17, pp. 985-1008, 2023.
A. S. F. Rodrigues, J. C. Lopes, R. P. Lopes, and L. F. Teixeira, "Classification of facial expressions under partial occlusion for VR games," in International Conference on Optimization, Learning Algorithms and Applications, 2022, pp. 804-819.
R. Magherini, E. Mussi, M. Servi, and Y. Volpe, "Emotion recognition in the times of COVID19: Coping with face masks," Intelligent Systems with Applications, vol. 15, p. 200094, 2022.
D. Pamod, C. Joseph, V. Palanisamy, and S. Lekamge, "Emotion Analysis of Occluded Facial Expressions-A Review of Literature," in 2022 ASU International Conference in Emerging Technologies for Sustainability and Intelligent Systems (ICETSIS), 2022, pp. 423-429.
P. Bambharolia, "Overview of Convolutional Neural Networks," in Proceedings of the International Conference on Academic Research in Engineering and Management, Monastir, Tunisia, 2017, pp. 8-10.
N. Omar, S. R. Zeebaree, M. A. Sadeeq, R. R. Zebari, H. M. Shukur, A. Alkhayyat, et al., "License plate detection and recognition: A study of review," in AIP Conference Proceedings, 2023.
A. Khan, A. Sohail, U. Zahoora, and A. S. Qureshi, "A survey of the recent architectures of deep convolutional neural networks," Artificial intelligence review, vol. 53, pp. 5455-5516, 2020.
A. Ghosh, A. Sufian, F. Sultana, A. Chakrabarti, and D. De, "Fundamental concepts of convolutional neural network," Recent trends and advances in artificial intelligence and Internet of Things, pp. 519-567, 2020.
C. Manresa-Yee, S. Ramis, and J. M. Buades, "Analysis of Gender Differences in Facial Expression Recognition Based on Deep Learning Using Explainable Artificial Intelligence," 2023.
N. Michael, "Artificial intelligence a guide to intelligent systems," ed: Addison Wesley, 2005.
A. Pannu, "Artificial intelligence and its application in different areas," Artificial Intelligence, vol. 4, pp. 79-84, 2015.
J. X.-Y. Lek and J. Teo, "Academic Emotion Classification Using FER: A Systematic Review," Human Behavior and Emerging Technologies, vol. 2023, 2023.
B. S. Tahir, Z. S. Ageed, S. S. Hasan, and S. R. Zeebaree, "Modified Wild Horse Optimization with Deep Learning Enabled Symmetric Human Activity Recognition Model," Computers, Materials & Continua, vol. 75, 2023.
D. Sarkar, R. Bali, and T. Sharma, "Practical machine learning with Python," Book" Practical Machine Learning with Python, pp. 25-30, 2018.
C. Dalvi, M. Rathod, S. Patil, S. Gite, and K. Kotecha, "A survey of ai-based facial emotion recognition: Features, ml & dl techniques, age-wise datasets and future directions," Ieee Access, vol. 9, pp. 165806-165840, 2021.
D. S. Shakya, "Analysis of artificial intelligence based image classification techniques," Journal of Innovative Image Processing, vol. 2, pp. 44-54, 2020.
B. Jena, S. Saxena, G. K. Nayak, L. Saba, N. Sharma, and J. S. Suri, "Artificial intelligence-based hybrid deep learning models for image classification: The first narrative review," Computers in Biology and Medicine, vol. 137, p. 104803, 2021.
J. Saeed and S. Zeebaree, "Skin lesion classification based on deep convolutional neural networks architectures," Journal of Applied Science and Technology Trends, vol. 2, pp. 41-51, 2021.
P. Wang, E. Fan, and P. Wang, "Comparative analysis of image classification algorithms based on traditional machine learning and deep learning," Pattern Recognition Letters, vol. 141, pp. 61-67, 2021.
J. Luo, Z. Xie, F. Zhu, and X. Zhu, "Facial expression recognition using machine learning models in fer2013," in 2021 IEEE 3rd International Conference on Frontiers Technology of Information and Computer (ICFTIC), 2021, pp. 231-235.
D. A. Zebari, H. Haron, S. R. Zeebaree, and D. Q. Zeebaree, "Enhance the mammogram images for both segmentation and feature extraction using wavelet transform," in 2019 International Conference on Advanced Science and Engineering (ICOASE), 2019, pp. 100-105.
S. Kim, G. H. An, and S.-J. Kang, "Facial expression recognition system using machine learning," in 2017 international SoC design conference (ISOCC), 2017, pp. 266-267.
A. N. Dixit and T. Kasbe, "A survey on facial expression recognition using machine learning techniques," in 2nd international conference on data, engineering and applications (IDEA), 2020, pp. 1-6.
B. Mahesh, "Machine learning algorithms-a review," International Journal of Science and Research (IJSR).[Internet], vol. 9, pp. 381-386, 2020.
S. M. S. A. Abdullah, S. Y. A. Ameen, M. A. Sadeeq, and S. Zeebaree, "Multimodal emotion recognition using deep learning," Journal of Applied Science and Technology Trends, vol. 2, pp. 52-58, 2021.
M. J. Mohammed Jasim, B. K. Hussan, S. R. Zeebaree, and Z. S. Ageed, "Automated Colonic Polyp Detection and Classification Enabled Northern Goshawk Optimization with Deep Learning," Computers, Materials & Continua, vol. 75, 2023.
K. B. Obaid, S. Zeebaree, and O. M. Ahmed, "Deep learning models based on image classification: a review," International Journal of Science and Business, vol. 4, pp. 75-81, 2020.
M. Usama, J. Qadir, A. Raza, H. Arif, K.-L. A. Yau, Y. Elkhatib, et al., "Unsupervised machine learning for networking: Techniques, applications and research challenges," IEEE access, vol. 7, pp. 65579-65615, 2019.
M. Karnati, A. Seal, D. Bhattacharjee, A. Yazidi, and O. Krejcar, "Understanding deep learning techniques for recognition of human emotions using facial expressions: a comprehensive survey," IEEE Transactions on Instrumentation and Measurement, 2023.
S. R. Zeebaree, O. Ahmed, and K. Obid, "Csaernet: An efficient deep learning architecture for image classification," in 2020 3rd International Conference on Engineering Technology and its Applications (IICETA), 2020, pp. 122-127.
K. DONUK, A. Ali, M. F. ÖZDEMİR, and D. HANBAY, "Deep feature selection for facial emotion recognition based on BPSO and SVM," Politeknik Dergisi, vol. 26, pp. 131-142, 2023.
D.-H. Lee and J.-H. Yoo, "CNN Learning Strategy for Recognizing Facial Expressions," IEEE Access, 2023.
S. B. Punuri, S. K. Kuanar, M. Kolhar, T. K. Mishra, A. Alameen, H. Mohapatra, et al., "Efficient net-XGBoost: an implementation for facial emotion recognition using transfer learning," Mathematics, vol. 11, p. 776, 2023.
T. Winyangkun, N. Vanitchanant, V. Chouvatut, and B. Panyangam, "Real-Time Detection and Classification of Facial Emotions," in 2023 15th International Conference on Knowledge and Smart Technology (KST), 2023, pp. 1-6.
B. Houshmand and N. M. Khan, "Facial expression recognition under partial occlusion from virtual reality headsets based on transfer learning," in 2020 IEEE Sixth International Conference on Multimedia Big Data (BigMM), 2020, pp. 70-75.
K. Soumya and S. Palaniswamy, "Emotion recognition from partially occluded facial images using prototypical networks," in 2020 2nd International Conference on Innovative Mechanisms for Industry Applications (ICIMIA), 2020, pp. 491-497.
H. Ding, P. Zhou, and R. Chellappa, "Occlusion-adaptive deep network for robust facial expression recognition," in 2020 IEEE International Joint Conference on Biometrics (IJCB), 2020, pp. 1-9.
B. Yang, W. Jianming, and G. Hattori, "Face mask aware robust facial expression recognition during the COVID-19 pandemic," in 2021 IEEE International conference on image processing (ICIP), 2021, pp. 240-244.
M. El Barachi, M. AlKhatib, and S. S. Mathew, "A Hybrid Machine Learning Approach for Sentiment Analysis of Partially Occluded Faces," in 2021 IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA), 2021, pp. 389-392.
D. Poux, B. Allaert, N. Ihaddadene, I. M. Bilasco, C. Djeraba, and M. Bennamoun, "Dynamic facial expression recognition under partial occlusion with optical flow reconstruction," IEEE Transactions on Image Processing, vol. 31, pp. 446-457, 2021.
R. Khoeun, W. Yookwan, P. Chophuk, A. Rodtook, and K. Chinnasarn, "Emotion Recognition of Partial Face using Star-Like Particle Polygon Estimation," IEEE Access, 2023.
Y. Chen, S. Liu, D. Zhao, and W. Ji, "Occlusion facial expression recognition based on feature fusion residual attention network," Frontiers in Neurorobotics, vol. 17, 2023.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
