Maximum Power Point Tracking Control for Photovoltaic Battery Systems using Deep Q Network Algorithm
Keywords:
Solar Energy, Deep Reinforcement Learning, Deep Q Network, Maximum Power Point Tracking, Photovoltaic Systems, DC - DC converterAbstract
Currently, many countries around the world have taken specific steps to gradually replace traditional fossil energy sources with renewable energy sources, of which solar energy is an appropriate choice. Power generation using photovoltaic (PV) batteries is becoming increasingly important because this is a renewable energy source with many advantages such as no fuel costs, no environmental pollution, requiring little maintenance, and does not emit noise compared to other energy sources. However, PV modules when working with inappropriate load impedance still have low conversion efficiency, so maximum power point tracking (MPPT) for PV is essential in a PV system. The amount of electricity generated depends on the operating voltage of the PV. On the and characteristics of PV, there exists only one maximum power point (MPP), this MPP point changes depending on radiation and environmental temperature. The MPPT's mission is to find and maintain the most efficient working mode. Therefore, many MPPT methods have been studied to determine the optimal working point. In this article, we propose to use the Deep Q Network (DQN) algorithm to maximize the energy from solar panels when there are changes in radiation intensity and environmental temperature. The results have been simulated and verified on MATLAB/SIMULINK, showing the feasibility and quality of the response when applying the new algorithm.
Downloads
References
Xia, Lei et al. “A model-based optimal control strategy for ground source heat pump systems with integrated solar photovoltaic thermal collectors.” Applied Energy (2018): vol. 228, pp. 1399–1412.
B. Subudhi and R. Pradhan, "A Comparative Study on Maximum Power Point Tracking Techniques for Photovoltaic Power Systems," in IEEE Transactions on Sustainable Energy, vol. 4, no. 1, pp. 89-98, Jan. 2013, doi: 10.1109/TSTE.2012.2202294.
Chen, P.-C., Chen, P.-Y., Liu, Y.-H., Chen, J.-H., & Luo, Y.-F. (2015). A comparative study on maximum power point tracking techniques for photovoltaic generation systems operating under fast changing environments. Solar Energy, 119, 261–276. doi:10.1016/j.solener.2015.07.006.
S. Pant and R. P. Saini, "Comparative Study of MPPT Techniques for Solar Photovoltaic System," 2019 International Conference on Electrical, Electronics and Computer Engineering (UPCON), Aligarh, India, 2019, pp. 1-6, doi: 10.1109/UPCON47278.2019.8980004.
Raj, Neha et al. “Comparative Analysis of Different MPPT Techniques.” 2019 3rd International conference on Electronics, Communication and Aerospace Technology (ICECA) (2019): 1091-1096.
A. K. Abdelsalam, A. M. Massoud, S. Ahmed and P. N. Enjeti, "High-Performance Adaptive Perturb and Observe MPPT Technique for Photovoltaic-Based Microgrids," in IEEE Transactions on Power Electronics, vol. 26, no. 4, pp. 1010-1021, April 2011, doi: 10.1109/TPEL.2011.2106221.
R. I. Jabbar, S. Mekhilef, M. Mubin and K. K. Mohammed, "A Modified Perturb and Observe MPPT for a Fast and Accurate Tracking of MPP Under Varying Weather Conditions," in IEEE Access, vol. 11, pp. 76166-76176, 2023, doi: 10.1109/ACCESS.2023.3297445.
Devi, V. Kamala et al. “A modified Perturb & Observe MPPT technique to tackle steady state and rapidly varying atmospheric conditions.” Solar Energy 157 (2017): 419-426.
Kjær, Søren Bækhøj. “Evaluation of the “Hill Climbing” and the “Incremental Conductance” Maximum Power Point Trackers for Photovoltaic Power Systems.” IEEE Transactions on Energy Conversion 27 (2012): 922-929.
M. A. Elgendy, B. Zahawi and D. J. Atkinson, "Assessment of the Incremental Conductance Maximum Power Point Tracking Algorithm," in IEEE Transactions on Sustainable Energy, vol. 4, no. 1, pp. 108-117, Jan. 2013, doi: 10.1109/TSTE.2012.2202698.
F. Liu, S. Duan, F. Liu, B. Liu and Y. Kang, "A Variable Step Size INC MPPT Method for PV Systems," in IEEE Transactions on Industrial Electronics, vol. 55, no. 7, pp. 2622-2628, July 2008, doi: 10.1109/TIE.2008.920550.
M. A. Abdourraziq, M. Maaroufi and M. Ouassaid, "A new variable step size INC MPPT method for PV systems," 2014 International Conference on Multimedia Computing and Systems (ICMCS), Marrakech, Morocco, 2014, pp. 1563-1568, doi: 10.1109/ICMCS.2014.6911212.
P. Suwannatrai, P. Liutanakul and P. Wipasuramonton, "Maximum power point tracking by incremental conductance method for photovoltaic systems with phase shifted full-bridge dc-dc converter," The 8th Electrical Engineering/ Electronics, Computer, Telecommunications and Information Technology (ECTI) Association of Thailand - Conference 2011, Khon Kaen, Thailand, 2011, pp. 637-640, doi: 10.1109/ECTICON.2011.5947920.
Y. -H. Ji, D. -Y. Jung, J. -G. Kim, J. -H. Kim, T. -W. Lee and C. -Y. Won, "A Real Maximum Power Point Tracking Method for Mismatching Compensation in PV Array Under Partially Shaded Conditions," in IEEE Transactions on Power Electronics, vol. 26, no. 4, pp. 1001-1009, April 2011, doi: 10.1109/TPEL.2010.2089537.
Hussan, Reyaz and Adil Sarwar. “Maximum Power Point Tracking Techniques under Partial Shading Condition- A Review.” 2018 2nd IEEE International Conference on Power Electronics, Intelligent Control and Energy Systems (ICPEICES) (2018): 293-298.
Chen, Woei-Luen and Chung-Ting Justine Tsai. “Optimal Balancing Control for Tracking Theoretical Global MPP of Series PV Modules Subject to Partial Shading.” IEEE Transactions on Industrial Electronics 62 (2015): 4837-4848.
S. K. Kollimalla and M. K. Mishra, "Adaptive perturb & observe MPPT algorithm for photovoltaic system," 2013 IEEE Power and Energy Conference at Illinois (PECI), Urbana, IL, USA, 2013, pp. 42-47, doi: 10.1109/PECI.2013.6506032.
A. Mohapatra, B. Nayak and C. Saiprakash, "Adaptive Perturb & Observe MPPT for PV System with Experimental Validation," 2019 IEEE International Conference on Sustainable Energy Technologies and Systems (ICSETS), Bhubaneswar, India, 2019, pp. 257-261, doi: 10.1109/ICSETS.2019.8744819.
S. Xiao and R. S. Balog, "An improved adaptive perturb & observe maximum power point tracking technique," 2018 IEEE Texas Power and Energy Conference (TPEC), College Station, TX, USA, 2018, pp. 1-6, doi: 10.1109/TPEC.2018.8312066.
I. Chtouki, P. Wira and M. Zazi, "Comparison of several neural network perturb and observe MPPT methods for photovoltaic applications," 2018 IEEE International Conference on Industrial Technology (ICIT), Lyon, France, 2018, pp. 909-914, doi: 10.1109/ICIT.2018.8352299.
W. Hayder, D. Sera, E. Ogliari and A. Abid, "Hybrid MPPT method based on Neural Network and Perturb & Observe for PV systems," 2022 19th International Multi-Conference on Systems, Signals & Devices (SSD), Sétif, Algeria, 2022, pp. 1401-1408, doi: 10.1109/SSD54932.2022.9955860.
B. Bendib, F. Krim, H. Belmili, M. F. Almi and S. Bolouma, "An intelligent MPPT approach based on neural-network voltage estimator and fuzzy controller, applied to a stand-alone PV system," 2014 IEEE 23rd International Symposium on Industrial Electronics (ISIE), Istanbul, Turkey, 2014, pp. 404-409, doi: 10.1109/ISIE.2014.6864647.
K. Y. Yap, C. R. Sarimuthu and J. M. -Y. Lim, "Artificial Intelligence Based MPPT Techniques for Solar Power System: A review," in Journal of Modern Power Systems and Clean Energy, vol. 8, no. 6, pp. 1043-1059, November 2020, doi: 10.35833/MPCE.2020.000159.
P. Q. Dzung, Le Dinh Khoa, Hong Hee Lee, Le Minh Phuong and Nguyen Truong Dan Vu, "The new MPPT algorithm using ANN-based PV," International Forum on Strategic Technology 2010, Ulsan, Korea (South), 2010, pp. 402-407, doi: 10.1109/IFOST.2010.5668004.
Dounis, Anastasios I. et al. “A direct adaptive neural control for maximum power point tracking of photovoltaic system.” Solar Energy 115 (2015): 145-165.
A. A. Jai and M. Ouassaid, "Machine Learning-Based Adaline Neural PQ Strategy for a Photovoltaic Integrated Shunt Active Power Filter," in IEEE Access, vol. 11, pp. 56593-56618, 2023, doi: 10.1109/ACCESS.2023.3281488.
M. Manas, A. Kumari and S. Das, "An Artificial Neural Network based Maximum Power Point Tracking method for photovoltaic system," 2016 International Conference on Recent Advances and Innovations in Engineering (ICRAIE), Jaipur, India, 2016, pp. 1-6, doi: 10.1109/ICRAIE.2016.7939482.
R. B. Roy, J. Cros, A. Nandi and T. Ahmed, "Maximum Power Tracking by Neural Network," 2020 8th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO), Noida, India, 2020, pp. 89-93, doi: 10.1109/ICRITO48877.2020.9197882.
S. Premrudeepreechacharn and N. Patanapirom, "Solar-array modelling and maximum power point tracking using neural networks," 2003 IEEE Bologna Power Tech Conference Proceedings,, Bologna, Italy, 2003, pp. 5 pp. Vol.2-, doi: 10.1109/PTC.2003.1304587.
A. Musafa, P. W. Purnawan, A. S. Tri Yulianto, A. Priyadi, M. Pujiantara and M. H. Purnomo, "Fuzzy Logic Based MPPT with Feedback Control for Photovoltaic Application," 2018 International Seminar on Intelligent Technology and Its Applications (ISITIA), Bali, Indonesia, 2018, pp. 169-174, doi: 10.1109/ISITIA.2018.8711272.
M. A. Abdourraziq, M. Ouassaid and M. Maaroufi, "A fuzzy logic MPPT for photovoltaic systems using single sensor," 2014 International Renewable and Sustainable Energy Conference (IRSEC), Ouarzazate, Morocco, 2014, pp. 52-56, doi: 10.1109/IRSEC.2014.7059774.
H. Aprillia, A. -S. A. Amalia and A. Giyantara, "Maximum Power Point Tracking Design Based on Fuzzy Logic Algorithm Under Uncertain Weather Condition," 2022 International Conference and Utility Exhibition on Energy, Environment and Climate Change (ICUE), Pattaya, Thailand, 2022, pp. 1-8, doi: 10.1109/ICUE55325.2022.10113495.
J. A. Nemours and S. Chowdhury, "Performance Analysis of Fuzzy Logic Maximum Power Point Tracking Scheme for Solar PV System Under Varying Load and Atmospheric Conditions," 2019 IEEE PES/IAS PowerAfrica, Abuja, Nigeria, 2019, pp. 134-139, doi: 10.1109/PowerAfrica.2019.8928895.
N. H. Jabber, A. A. Ridha and M. A. R. S. Al-Baghdadi, "Design and Simulation of Intelligent Maximum Power Point Tracking (MPPT) based on Fuzzy Logic Controller for PV System," 2022 International Conference on Computer Science and Software Engineering (CSASE), Duhok, Iraq, 2022, pp. 157-162, doi: 10.1109/CSASE51777.2022.9759793.
D. Canny and F. Yusivar, "Maximum Power Point Tracking (MPPT) Algorithm Simulation Based on Fuzzy Logic Controller on Solar Cell with Boost Converter," 2018 2nd International Conference on Smart Grid and Smart Cities (ICSGSC), Kuala Lumpur, Malaysia, 2018, pp. 117-121, doi: 10.1109/ICSGSC.2018.8541325.
V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, et al., “Human-level control through deep reinforcement learning,” Nature, vol. 518, no. 7540, pp. 529–533, 2015.
D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den Driessche, J. Schrittwieser, I. Antonoglou, et al., “Mastering the game of go with deep neural networks and tree search,” Nature, vol. 529, no. 7587, pp. 484–489, 2016.
S. Levine, C. Finn, T. Darrell, and P. Abbeel, “End-to-end training of deep visuomotor policies,” J. Mach. Learning Res., vol. 17, no. 39, pp. 1–40, 2016.
P. Abolghasemi and L. Bölöni, "Accept Synthetic Objects as Real: End-to-End Training of Attentive Deep Visuomotor Policies for Manipulation in Clutter," 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France, 2020, pp. 6506-6512, doi: 10.1109/ICRA40945.2020.9197552.
S. Levine, P. Pastor, A. Krizhevsky, and D. Quillen, “Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection,” in Proc. Int. Symp. Experimental Robotics, 2016, pp. 173–184.
Humada, A. M., Hojabri, M., Mekhilef, S., & Hamada, H. M. (2016). Solar cell parameters extraction based on single and double-diode models: A review. Renewable and Sustainable Energy Reviews, 56, 494–509. doi:10.1016/j.rser.2015.11.051.
Kofinas, P., Doltsinis, S., Dounis, A. I., & Vouros, G. A. (2017). A reinforcement learning approach for MPPT control method of photovoltaic sources. Renewable Energy, 108, 461–473. doi:10.1016/j.renene.2017.03.008.
L. Avila, M. De Paula, I. Carlucho and C. Sanchez Reinoso, "MPPT for PV systems using deep reinforcement learning algorithms," in IEEE Latin America Transactions, vol. 17, no. 12, pp. 2020-2027, December 2019, doi: 10.1109/TLA.2019.9011547.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
