Design of an AI-Driven Feedback and Decision Analysis in Online Learning with Google BERT

Authors

  • Gaurav Srivastav Department of Computer Science and Engineering, Sharda University, Greater Noida,UP, India, 201010
  • Shri Kant Department of Cyber Security and Cryptology, Sharda University, Greater Noida, UP, India, 201010
  • Durgesh Srivastava Department of Computer Science and Engineering, Chitkara University, Rajpura, India, 140401

Keywords:

Automated Feedback, Natural Language Processing, Deep Learning, Hybrid Approaches, Educational Assessment, Text Analysis, Language Models

Abstract

The global COVID-19 pandemic has significantly altered educational practices. The enforcement of social distancing rules led to the widespread closure of schools, prompting a shift towards remote and online learning modalities. This transition has been challenging for both educators and students. Teachers have struggled to create and deliver online content that meets student needs, while students have faced difficulties adapting to new technologies and resource constraints. The pandemic has also disrupted traditional academic schedules, delaying admission processes, examinations, and academic calendar events. Research is underway to understand the impact of these shifts on student performance and educational outcomes. Interestingly, the pandemic has highlighted the necessity for greater investment in teacher training programs and digital infrastructure to support distance learning. This study introduces an automated feedback assessment model that utilizes Google's Bidirectional Encoder Representations from Transformers (BERT). The model generates a quality score for inputs in a Virtual Learning Environment (VLE) during the pandemic. It was trained using a dataset comprising 10,000 feedback entries, categorized as either "good" or "bad". Further refinement was done on the Open University Learning Analytics (OULA) dataset across 50 epochs. The model achieved a 93.4% accuracy rate on the validation set, indicating its proficiency in evaluating the quality of feedback. The implications of this model are far-reaching. It can be applied in various sectors, including education, performance assessment, and customer service, offering a means to decrease the time and subjectivity involved in human evaluations. This study not only addresses the immediate challenges posed by the pandemic in the educational sector but also provides a forward-looking solution with versatile applications.

Downloads

Download data is not yet available.

References

Torres Martín, César, Acal, Christian, El Homrani, Mohammed, and Ángel C. Mingorance Estrada. "Impact on the Virtual Learning Environment Due to COVID-19." Sustainability 13, no. 2 (2021): 582. Accessed April 19, 2023. https://doi.org/10.3390/su13020582.

Del Olmo-Muñoz, J., González-Calero, J.A., Diago, P.D. et al. Intelligent tutoring systems for word problem solving in COVID-19 days: could they have been (part of) the solution?. ZDM Mathematics Education 55, 35–48 (2023). https://doi.org/10.1007/s11858-022-01396-w

Bozkurt, Aras, Karakaya, Kadir, Turk, Murat, Karakaya, Özlem, and Daniela Castellanos-Reyes. "The Impact of COVID-19 on Education: A Meta-Narrative Review." Techtrends 66, no. 5 (2022): 883-896. Accessed April 19, 2023. https://doi.org/10.1007/s11528-022-00759-0.

Nayak, B., Bhattacharyya, S.S., Goswami, S. et al. Adoption of online education channel during the COVID-19 pandemic and associated economic lockdown: an empirical study from push–pull-mooring framework. J. Comput. Educ. 9, 1–23 (2022). https://doi.org/10.1007/s40692-021-00193-w

Li, Xia, Huali Yang, Shengze Hu, Jing Geng, Keke Lin, and Yuhai Li. "Enhanced hybrid neural network for automated essay scoring." Expert Systems 39, no. 10 (2022): e13068.

Tashu, Tsegaye Misikir, Chandresh Kumar Maurya, and Tomas Horvath. "Deep Learning Architecture for Automatic Essay Scoring." arXiv preprint arXiv:2206.08232 (2022).

Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) (pp. 4171-4186).

Soam, DMilind, and Sanjeev Thakur. "Sentiment Analysis Using Deep Learning: A Comparative Study." In 2022 Second International Conference on Computer Science, Engineering and Applications (ICCSEA), pp. 1-6. IEEE, 2022.

Ali, Omar, Wiem Abdelbaki, Anup Shrestha, Ersin Elbasi, Mohammad Abdallah Ali Alryalat, and Yogesh K. Dwivedi. "A systematic literature review of artificial intelligence in the healthcare sector: Benefits, challenges, methodologies, and functionalities." Journal of Innovation & Knowledge 8, no. 1 (2023): 100333.

Torres Martín, César, Acal, Christian, El Homrani, Mohammed, and Ángel C. Mingorance Estrada. "Impact on the Virtual Learning Environment Due to COVID-19." Sustainability 13, no. 2 (2021): 582. Accessed April 19, 2023. https://doi.org/10.3390/su13020582.

Coman, Claudiu, Țîru, Laurențiu G., Stanciu, Carmen, and Maria C. Bularca. "Online Teaching and Learning in Higher Education during the Coronavirus Pandemic: Students’ Perspective." Sustainability 12, no. 24 (2020): 10367. Accessed April 19, 2023. https://doi.org/10.3390/su122410367.

Mutizwa, Melissa R., Ozdamli, Fezile, and Damla Karagozlu. "Smart Learning Environments during Pandemic." Trends in Higher Education 2, no. 1 (2023): 16-28. Accessed April 19, 2023. https://doi.org/10.3390/higheredu2010002.

Amin, M. R., Islam, M. S., Islam, M. R., & Mahmud, M. (2021). Natural language processing techniques for hate speech detection and analysis: A comprehensive review. ACM Computing Surveys, 54(6), 1-39.

Burstein, J., Kukich, K., Wolff, S., Lu, C., & Chodorow, M. (1998). Using NLP to Identify Misconceptions in Students' Writing. IEEE Intelligent Systems, 13(5), 46-54.

Chen, Y., Cheng, W., & Huang, Y. (2016). Automated Feedback Assessment for Improving Students' Writing. IEEE Transactions on Learning Technologies, 9(3), 197-210.

Kukich, K., J.& Becker, S.& R., & Mostow, J. (1993). An evaluation of an automated response evaluation system. Intelligent Tutoring Systems, 4, 4-12.

Attali, Yigal, and Jill Burstein. "Automated essay scoring with e-rater® v.2." Journal of Technology, Learning, and Assessment 4, no. 3 (2006): 1-30.

Qian, Y., Liu, J., Jia, J., & Xie, X. (2019). An Automated Feedback Assessment Model for Online Discussions Based on Multiple-Aspect Analysis. IEEE Access, 7, 93596-93608.

Page, E. B. (1966). The use of the computer in analyzing student essays. Journal of Educational Measurement, 3(4), 211-220.

Landauer, T. K., Laham, D., & Foltz, P. W. (2003). Automated essay scoring: A cross-disciplinary perspective. Lawrence Erlbaum Associates Publishers.

Amado, S., & Dill, K. E. (1998). Automated essay grading: An application of n-gram analysis. In Proceedings of the 21st Annual Conference of the Cognitive Science Society (pp. 105-110).

Burstein, J., Leacock, C., & Swartz, R. (2003). CRATER: using natural language processing to provide automated feedback on student writing. In Proceedings of the 8th International Conference on Intelligent Tutoring Systems (pp. 115-124). Springer.

Alsariera, Y. A., Baashar, Y., Alkawsi, G., Mustafa, A., Alkahtani, A. A., & Ali, N. (2022). Assessment and evaluation of different machine learning algorithms for predicting student performance. Computational Intelligence and Neuroscience, 2022, 1–11.

Zhang, Y., Li, B., Hu, Y., Zhang, Y., & Li, X. (2019). Automated feedback assessment of writing performance based on lexical and syntactic features. Journal of Educational Computing Research, 56(1), 144-164.

Amin, A., Singh, P., & Arora, S. (2021). Automated feedback generation for programming assignments: A survey. Journal of Educational Computing Research, 59(6), 1489-1531.

Xiaodong Liu, Yelong Shen, Jingjing Liu, and Jianfeng Gao. "Automatic Dialogue Assessment with Sequence to Sequence Learning." In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 968-977. 2018.

Madnani, N., Dabre, R., & Mott, B. (2018). Automated writing evaluation for feedback on draft essays. Handbook of Automated Essay Evaluation, 121-140.

Wilson, T., Wiebe, J., & Hoffman, P. (2017). Recognizing contextual polarity in phrase-level sentiment analysis. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (pp. 347-352).

Taghipour, K., & Ng, H. T. (2016). A neural approach to automated essay scoring. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (pp. 1882-1891).

Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., & Hovy, E. (2016). Hierarchical attention networks for document classification. Proceedings of the 2016 conference of the North American chapter of the Association for Computational Linguistics: Human language technologies, 1480-1489.

Jamil H. and Mou X., "Automated Feedback and Authentic Assessment for Online Computational Thinking Tutoring Systems," 2022 International Conference on Advanced Learning Technologies (ICALT), 2022, pp. 53-55, doi: 10.1109/ICALT55010.2022.00024.

Lu, W., Vivekananda, GN & Shanthini, A. Supervision system of english online teaching based on machine learning. Prog Artif Intell (2022). https://doi.org/10.1007/s13748-021-00274-y

Nayak S., Agarwal R. and Khatri S., "Review of Automated Assessment Tools for grading student SQL queries," 2022 International Conference on Computer Communication and Informatics (ICCCI), 2022, pp. 1-4, doi: 10.1109/ICCCI54379.2022.9740799.

Zhang, Y., Liu, H., & Yang, P. (2021). An Automated Feedback Assessment Model for EFL Writing Based on Sentiment Analysis and Hierarchical Attention Network. IEEE Access, 9, 16571-16581. https://doi.org/10.1109/ACCESS.2021.3051497

Wang, Y., Li, L., Li, H., Zhang, X., & Li, L. (2021). An Automated Feedback Assessment Model for Online Courses Based on Machine Learning and Rule-Based Techniques. IEEE Access, 9, 39712-39720. https://doi.org/10.1109/ACCESS.2021.3064022

Burstein, J., Deane, P., & Chodorow, M. (2013). The e-rater® automated essay scoring system. Handbook of Automated Essay Evaluation: Current Applications and New Directions, 55-67.

ttali, Y., & Powers, D. E. (2015). Hybrid Scoring Model (HSM): A new approach to automated text scoring combining syntactic, semantic, and discourse processing. Educational and Psychological Measurement, 75(2), 185-206.

Rudner, L. M., & Liang, S. (2002). An overview of Bayesian adaptive testing. Journal of Educational and Behavioral Statistics, 27(4), 323-340.

Wright, H., Kemp, J., & Varner, L. K. (2016). The Writing Mentor: A Comprehensive Writing Tool for the Middle Grades. Journal of Educational Technology Development and Exchange, 9(1), 1-14.

Bai, Yunpeng, Linjie Li, Jingjing Xu, Xu Sun, Weiwei Liu, and Qi Su. "PARS-BERT: Pre-training of Parsimonious Language Model for Adapting BERT to Resource-poor Domains." arXiv preprint arXiv:2010.16024 (2020).

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.

Li, Pengfei, Yuxian Deng, and Shuai Wu. "Semantics-aware BERT for Language Understanding." arXiv preprint arXiv:2106.13139 (2021).

Kumar Agarwal A, Angeline Ranjithamani D, Pavithra M, Velayudham A, Shunmugam A, et al., (2021) Machine Learning Technique for the Assembly-Based Image Classification System. J Nucl Ene Sci Power Generat Techno 10:9.

R. G. Tiwari, A. Misra, V. Khullar, A. K. Agarwal, S. Gupta and A. P. Srivastava, "Identifying Microscopic Augmented Images using Pre-Trained Deep Convolutional Neural Networks," 2021 International Conference on Technological Advancements and Innovations (ICTAI), Tashkent, Uzbekistan, 2021, pp. 32-37, doi: 10.1109/ICTAI53825.2021.9673472.

A. K. Agarwal, G. Khan, S. Qamar, and N. Lal, “Localization and correction of location information for nodes in UWSN-LCLI,” Adv. Eng. Softw., vol. 173, p. 103265, 2022, doi: 10.1016/j.advengsoft.2022.103265.

Tarun Sharma, Ambuj Kumar Agarwal, Danish Ather and Ashendra Saxena, "SEARCH BASED SOFTWARE ENGINEERING IN REQUISITE PHASE OF SDLC: A SURVEY", Technical journal of LBSIMDS.

A. K. . Agarwal, V. . Kiran, R. K. . Jindal, D. . Chaudhary, and R. G. . Tiwari, “Optimized Transfer Learning for Dog Breed Classification”, Int J Intell Syst Appl Eng, vol. 10, no. 1s, pp. 18–22, Oct. 2022.

R. Sharma, H. Pandey, and A. K. Agarwal, “Exploiting artificial intelligence for combating COVID-19 : a review and appraisal,” vol. 12, no. 1, pp. 514–520, 2023, doi: 10.11591/eei.v12i1.4366.

D. Srivastava, H. Pandey, and A. K. Agarwal, “Complex predictive analysis for health care : a comprehensive review,” vol. 12, no. 1, pp. 521–531, 2023, doi: 10.11591/eei.v12i1.4373.

Downloads

Published

07.01.2024

How to Cite

Srivastav, G. ., Kant, S. ., & Srivastava, D. . (2024). Design of an AI-Driven Feedback and Decision Analysis in Online Learning with Google BERT. International Journal of Intelligent Systems and Applications in Engineering, 12(10s), 629 –. Retrieved from https://ijisae.org/index.php/IJISAE/article/view/4465

Issue

Section

Research Article