Abstractive Long Text Summarization using Large Language Models

Authors

  • Gunjan Keswani Department of Computer Science and Engineering Shri Ramdeobaba College of Engineering and Management (RCOEM) Nagpur, India
  • Wani Bisen Dept. of Computer Science and Engineering Shri Ramdeobaba College of Engineering and Management (RCOEM) Nagpur, India
  • Hirkani Padwad Dept. of Computer Science and Engineering Shri Ramdeobaba College of Engineering and Management (RCOEM) Nagpur, India
  • Yash Wankhedkar Dept. of Computer Science and Engineering Shri Ramdeobaba College of Engineering and Management (RCOEM) Nagpur, India
  • Sudhanshu Pandey Dept. of Computer Science and Engineering Shri Ramdeobaba College of Engineering and Management (RCOEM) Nagpur, India
  • Ayushi Soni Dept. of Computer Science and Engineering Shri Ramdeobaba College of Engineering and Management (RCOEM) Nagpur, India

Keywords:

Abstractive summarization, Large Language Models, LangChain, Natural Language Processing, Retrieval-Augmented Generation

Abstract

Large Language Models (LLMs) have made significant strides in processing human-written texts. However, a major challenge persists - the retention of context over extensive texts or multiple documents. The current approach of LLMs to retain context is often inefficient, both in terms of storage and time. To address this issue, this paper proposes a novel approach for two key tasks - Summarization and Question Answering. The methodology ensures that the LLM is not overwhelmed with unrelated, repetitive, or redundant data, thereby saving considerable time and resources. This approach facilitates the generation of effective summaries and answers for the user, enhancing the overall performance and efficiency of the LLM.

Downloads

Download data is not yet available.

References

Rani Horev (2018). BERT Explained: State of the language model for NLP. https://towardsdatascience.com/bert-explained-state-of-the-art-language-model- for-nlp-f8b21a9b6270

“Open Sourcing BERT: State-of-the-Art Pre-training for Natural Language Processing.” Google Research, 2 Nov. 2018, ai.googleblog.com/2018/11/open-sourcing- bert-state-of-art-pre.html

Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, Luke Zettlemoyer (2019). “BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension”. arXiv:1910.13461.https://arxiv.org/abs/1910.13461v1

S. Pal, “The Why, What, Who and Where of Vector Databases”, May 10, 2023. Available: https://www.eckerson.com/articles/why- what-who-and-where-of-vector-databases

Touvron, H.et al. (2023) Llama 2: Open Foundation and Fine-Tuned Chat Models, arXiv.org. Available at: https://arxiv.org/abs/2307.09288

Keivalya Pandya, Mehfuza Holia (2023). “Automating Customer Service using LangChain: Building custom open-source GPT Chatbot for organizations.” arXiv: 2310.05421 [cs.CL].

K.Martineau, “What is retrieval- augmented generation?,” IBM Research Blog, Aug.2023,[Online]. Available: https://research.ibm.com/blog/retrieval- augmented-generation-RAG

Atlassian, “What is a Knowledge Base?” Available:https://www.atlassian.com/itsm/knowledge- management/what-is-a-knowledge-base

S. Na, L. Xumin and G. Yong, “Research on k-means Clustering Algorithm: An Improved k-means Clustering Algorithm,” 2010 Third International Symposium on Intelligent Information Technology and Security Informatics, Jian, China, 2010, pp. 63-67

Nan, F., Santos, C. N. dos, Zhu, H., Ng, P., McKeown, K., Nallapati, R., Zhang, D., Wang, Z., Arnold, A. O., Xiang, B. (2021). Improving factual consistency of abstractive summarization via question answering. arXiv. http://arxiv.org/abs/2105.04623

P. Sitikhu, K. Pahi, P.Thapa, S. Shakya (2019). “A Comparison of Semantic Similarity Methods for Maximum Human Interpretability.” arXiv: 1910.09129 [cs.IR]. http://arxiv.org/abs/1910.09129

C.-Y. Lin, “ROUGE: A Package for Automatic Evaluation of Summaries,” in Text Summarization Branches Out, Barcelona, Spain, 2004, pp. 74-81. [Online]. Available: https://www.aclweb.org/anthology/W04- 1013

Max Grusky. 2023. Rogue Scores. In Pro- ceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1914–1934, Toronto, Canada. Association for Computational Linguistic.

Downloads

Published

12.01.2024

How to Cite

Keswani, G. ., Bisen, W. ., Padwad, H. ., Wankhedkar, Y. ., Pandey, S. ., & Soni, A. . (2024). Abstractive Long Text Summarization using Large Language Models. International Journal of Intelligent Systems and Applications in Engineering, 12(12s), 160–168. Retrieved from https://ijisae.org/index.php/IJISAE/article/view/4500

Issue

Vol. 12 No. 12s (2024)

Section

Research Article

License

Creative Commons License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.

IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.