Evolutionary Strategies for Parameter Optimization in Deep Learning Models

Authors

  • Murthy D. H. R. Assistant Professor & Research Scholar, School of Computer Science and Information Science Presidency University, Bangalore, India.
  • Shalini S. Associate Professor, DSATM, Bangalore
  • Aditya Kumar Gupta Associate Professor, School of Management Sciences Varanasi, UP
  • Kiran Mayee Adavala Faculty, Kakatiya Institute of Technology & Science
  • Ahmad Tasnim Siddiqui Associate Professor, (School of Computer Science & Engineering) Sandip University, Nashik, Maharashtra
  • Rohan Shinkre Research Consultant, Central Research wing, KLE Society's Institute of Dental Sciences, Bangalore
  • Prasanna P. Deshpande Assistant Professor (Electronics and Communication Engineering) Shri Ramdeobaba College of Engineering and Management, Nagpur (India)
  • Manoj Pareek Associate Professor, Bennett University, Greater NOIDA, India

Keywords:

swarm optimization, evolutionary algorithms, hyperparameters, deep learning

Abstract

Evolutionary algorithms (EAs) have gained significant optimization techniques for deep learning model parameter tuning. Deep learning models often contain many parameters, and finding the optimal values for these parameters. EAs, inspired by natural evolution and natural selection processes, provide a promising approach for automatically searching and optimizing the parameter space. This study explores the application of EAs for deep learning model parameter tuning. We present an overview of deep learning and the challenges associated with parameter tuning. Next, we briefly introduce evolutionary algorithms and their key components, such as population initialization, reproduction, and selection operators. We discuss various strategies for integrating EAs into the parameter tuning process, including using different genetic operators, such as mutation and crossover, and techniques for handling constraints and incorporating prior knowledge. We aim of our present work is to optimize these hyperparameters using swarm optimization algorithms and evolutionary algorithms.

Downloads

Download data is not yet available.

References

J. H. Holland, Adaption in Natural and Artificial Systems. Cambridge, Massachusetts: The MIT Press, 1992.

N. E. Nawa and T. Furuhashi, “Fuzzy system parameters discovery by bacterial evolutionary algorithm,” IEEE Transactions on Fuzzy Systems, vol. 7, no. 5, pp. 608–616, Oct. 1999.

R. Storn and K. Price, “Differential evolution - a simple and efficient heuristic for global optimization over continuous spaces,” Journal of Global Optimization, vol. 11, no. 4, pp. 341–359, 1997.

Mehrabian and C. Lucas, “A novel numerical optimization algorithm inspired from weed colonization,” Ecological Informatics, vol. 1, no. 4, pp. 355–366, 2006.

J. Kennedy and R. C. Eberhart, “Particle swarm optimization,” in Proceedings of the IEEE International Conference on Neural Networks, Perth, Australia, 1995, pp. 1942–1948.

Bae, W.-C. Yeh, N. Wahid, Y. Chung, and Y. Liu, “A new simplified swarm optimization (SSO) using exchange local search scheme,” Int. J. Innovative Computing, Information and Control, vol. 8, no. 6, pp. 4391–4406, 2012.

Goodfellow, Y. Bengio, and A. Courville, Deep Learning. MIT Press, 2016, http://www.deeplearningbook.org.

P. I. Frazier, “A tutorial on Bayesian optimization,” www.arxiv.org/pdf/1807.02811.pdf, 2018.

R. Hecht-Nielsen, Neurocomputing. Addison-Wesley, 1990.

Y. LeCun and C. Cortes, “MNIST handwritten digit database,” 2010. [Online]. Available: http://yann.lecun.com/exdb/mnist/

K. Simonyan and A. Zisserman, “VGGNet,” www.arxiv.org/pdf/1409.1556.pdf, 2015.

H. Xiao, K. Rasul, and R. Vollgraf, “Fashion-MNIST,” www.arxiv.org/pdf/1708.07747.pdf, 2017.

P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” www.arxiv.org/pdf/1412.6980.pdf, 2014.

Supratak, H. Dong, C. Wu, and Y. Guo, “Deepsleepnet: A model for automatic sleep stage scoring based on raw single-channel eeg,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 25, no. 11, pp. 1998–2008, 2017.

Sherje, D. N. . (2021). Content Based Image Retrieval Based on Feature Extraction and Classification Using Deep Learning Techniques. Research Journal of Computer Systems and Engineering, 2(1), 16:22. Retrieved from https://technicaljournals.org/RJCSE/index.php/journal/article/view/14

Ch.Sarada, C., Lakshmi, K. V. ., & Padmavathamma, M. . (2023). MLO Mammogram Pectoral Masking with Ensemble of MSER and Slope Edge Detection and Extensive Pre-Processing. International Journal on Recent and Innovation Trends in Computing and Communication, 11(3), 135–144. https://doi.org/10.17762/ijritcc.v11i3.6330

Keerthi, R.S., Dhabliya, D., Elangovan, P., Borodin, K., Parmar, J., Patel, S. K. Tunable high-gain and multiband microstrip antenna based on liquid/copper split-ring resonator superstrates for C/X band communication (2021) Physica B: Condensed Matter, 618, art. no. 413203, .

Downloads

Published

27.10.2023

How to Cite

D. H. R., M. ., S., S. ., Gupta, A. K. ., Adavala, K. M. ., Siddiqui, A. T. ., Shinkre, R. ., Deshpande, P. P. ., & Pareek, M. . (2023). Evolutionary Strategies for Parameter Optimization in Deep Learning Models. International Journal of Intelligent Systems and Applications in Engineering, 12(2s), 371–378. Retrieved from https://ijisae.org/index.php/IJISAE/article/view/3636

Issue

Vol. 12 No. 2s (2024)

Section

Research Article

License

Creative Commons License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.

IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.