Comparative Analysis of Deep learning Models for Various Optimizer Embedded with Gradient Centralization
Keywords:
Deep learning models, Densenet, Gradient Centralization, Mobile net, NasnetAbstract
Gradient Centralization (GC)emerges out an powerful optimization technique in area of Deep Convolutional neural network. It shows remarkable improvement in the execution time of deep learning models and opens up the scope of analyzing gradient vector to improve optimizer performance. It directly works upon the gradients and centralizes the gradient vector to have zero mean. One of the key factors which drives the attention of researchers is its embedding factor which allow its functionality to be explored with existing DNN optimizer. Our research works draws out individual and comparative analysis of GC embedded with RMS prop (Root Mean Square Propagation), Adam, Adagrad and Adadelta for three deep learning models: Mobile net, Nasnet and Densenet 201. Experiments are carried out with lung disease dataset. Highly motivating results are achieved through this embedding and accuracy of models has been enhanced up to 99%. Improved trends are also projected for Loss and Execution time.
Downloads
References
Elshamy, R., Abu-Elnasr, O., Elhoseny, M., & Elmougy, S. (2023). Improving the efficiency of RMSProp optimizer by utilizing Nestrove in deep learning. Scientific Reports, 13(1), 8814.
Z. Zhang, "Improved Adam Optimizer for Deep Neural Networks," 2018 IEEE/ACM 26th International Symposium on Quality of Service (IWQoS), Banff, AB, Canada, 2018, pp. 1-2, doi: 10.1109/IWQoS.2018.8624183.
Okewu, E., Adewole, P., Sennaike, O. (2019). Experimental Comparison of Stochastic Optimizers in Deep Learning. In: Misra, S., et al. Computational Science and Its Applications – ICCSA 2019. ICCSA 2019. Lecture Notes in Computer Science(), vol 11623. Springer, Cham. https://doi.org/10.1007/978-3-030-24308-1_55
S. R. Dubey, S. Chakraborty, S. K. Roy, S. Mukherjee, S. K. Singh and B. B. Chaudhuri, "diffGrad: An Optimization Method for Convolutional Neural Networks," in IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 11, pp. 4500-4511, Nov. 2020, doi: 10.1109/TNNLS.2019.2955777.
Yaqub, M., Feng, J., Zia, M. S., Arshid, K., Jia, K., Rehman, Z. U., & Mehmood, A. (2020). State-of-the-art CNN optimizer for brain tumor segmentation in magnetic resonance images. Brain Sciences, 10(7), 427.
A. M. Taqi, A. Awad, F. Al-Azzo and M. Milanova, "The Impact of Multi-Optimizers and Data Augmentation on TensorFlow Convolutional Neural Network Performance," 2018 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR), Miami, FL, USA, 2018, pp. 140-145, doi: 10.1109/MIPR.2018.00032.
Babu, D. V., Karthikeyan, C., & Kumar, A. (2020, December). Performance analysis of cost and accuracy for whale swarm and RMSprop optimizer. In IOP Conference Series: Materials Science and Engineering (Vol. 993, No. 1, p. 012080). IOP Publishing.
Yong, H., Huang, J., Hua, X., & Zhang, L. (2020). Gradient centralization: A new optimization technique for deep neural networks. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part I 16 (pp. 635-652). Springer International Publishing.
Fuhl, W., & Kasneci, E. (2020). Weight and gradient centralization in deep neural networks. arXiv preprint arXiv:2010.00866
Yong, H., Huang, J., Hua, X., & Zhang, L. (2020). Gradient centralization: A new optimization technique for deep neural networks. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part I 16 (pp. 635-652). Springer International Publishing.
Zang, H., Foo, S. Y., Bernadin, S., & Meyer-Baese, A. (2021). Facial emotion recognition using asymmetric pyramidal networks with gradient centralization. IEEE Access, 9, 64487-64498.
Sadu, S., Dubey, S. R., & Sreeja, S. R. (2023). Moment Centralization-Based Gradient Descent Optimizers for Convolutional Neural Networks. In Computer Vision and Machine Intelligence: Proceedings of CVMI 2022 (pp. 51-63). Singapore: Springer Nature Singapore.
Roy S. K., Paoletti, M. E., Haut, J. M., Dubey, S. R., Kar, P., Plaza, A., & Chaudhuri, B. B. (2021). Angulargrad: A new optimization technique for angular convergence of convolutional neural networks. arXiv preprint arXiv:2105.10190.
Lv, N., Xiang, X., Wang, X., Yang, J., & Abdein, R. (2022). Efficient person search via learning-to-normalize deep representation. Neurocomputing, 495, 169-177.
Narayan, Vipul, et al. "7 Extracting business methodology: using artificial intelligence-based method." Semantic Intelligent Computing and Applications 16 (2023): 123
Narayan, Vipul, et al. "A Comprehensive Review of Various Approach for Medical Image Segmentation and Disease Prediction." Wireless Personal Communications 132.3 (2023): 1819-1848.
Mall, Pawan Kumar, et al. "Rank Based Two Stage Semi-Supervised Deep Learning Model for X-Ray Images Classification: AN APPROACH TOWARD TAGGING UNLABELED MEDICAL DATASET." Journal of Scientific & Industrial Research (JSIR) 82.08 (2023): 818-830.
Narayan, Vipul, et al. "Severity of Lumpy Disease detection based on Deep Learning Technique." 2023 International Conference on Disruptive Technologies (ICDT). IEEE, 2023.
Saxena, Aditya, et al. "Comparative Analysis Of AI Regression And Classification Models For Predicting House Damages İn Nepal: Proposed Architectures And Techniques." Journal of Pharmaceutical Negative Results (2022): 6203-6215.
Kumar, Vaibhav, et al. "A Machine Learning Approach For Predicting Onset And Progression"“Towards Early Detection Of Chronic Diseases “." Journal of Pharmaceutical Negative Results (2022): 6195-6202.
Chaturvedi, Pooja, Ajai Kumar Daniel, and Vipul Narayan. "Coverage Prediction for Target Coverage in WSN Using Machine Learning Approaches." (2021).
Chaturvedi, Pooja, A. K. Daniel, and Vipul Narayan. "A Novel Heuristic for Maximizing Lifetime of Target Coverage in Wireless Sensor Networks." Advanced Wireless Communication and Sensor Networks. Chapman and Hall/CRC 227-242.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
