Sequence-to-Sequence Abstractive Text Summarization Model for Headline Generation with Attention
Keywords:
Abstractive text summarization, Attention mechanism, Deep learning, Encoder-Decoder architecture, LSTM, Sequence-to-Sequence model, Single sentence summarization, Text preprocessing techniquesAbstract
Text summarization focuses on creating a brief and concise summary from source text while preserving the main idea and eliminating unnecessary details. Generating summaries through manual efforts by humans is a tedious, tiresome, and expensive process. Hence, this study’s objective is to build an automated abstractive text summarizer that can minimize manual efforts and generate concise summaries swiftly. The aim is to develop a text summarizer model using deep learning to form a single-line abstractive summary resembling a headline. It also explores the impact of adjusting the model's hyperparameters on the generated summary to achieve better results. A subset of instances from the Gigaword dataset is utilized to develop the model. The proposed summarizer is a sequence-to-sequence model with an LSTM-driven encoder-decoder architecture. It incorporates a Bahdanau attention mechanism and utilizes the Adam optimizer. Based on experimental analysis and the results obtained after adjusting hyperparameters and selecting the optimal values as final, the proposed architecture attained scores as 24.27, 8.57, and 23.13, for ROUGE-1, ROUGE-2, and ROUGE-L respectively.
Downloads
References
Suleiman D, Awajan A. Deep learning based abstractive text summarization: approaches, datasets, evaluation measures, and challenges. Mathematical problems in engineering. 2020 Aug 24;2020:1-29.
Dilawari A, Khan MU, Saleem S, Shaikh FS. Neural Attention Model for Abstractive Text Summarization Using Linguistic Feature Space. IEEE Access. 2023 Feb 27;11:23557-64.
Jobson E, Gutiérrez A. Abstractive text summarization using attentive sequence-to-sequence rnns.
Rush AM, Chopra S, Weston J. A neural attention model for abstractive sentence summarization. arXiv preprint arXiv:1509.00685. 2015 Sep 2.
Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. 2014 Sep 1.
Parker, Robert, et al. English Gigaword Fifth Edition LDC2011T07. Web Download. Philadelphia: Linguistic Data Consortium, 2011.
Resnik P, Niv M, Nossal M, Schnitzer G, Stoner J, Kapit A, Toren R. Using intrinsic and extrinsic metrics to evaluate accuracy and facilitation in computer-assisted coding. InPerspectives in health information management computer assisted coding conference proceedings 2006 Aug (pp. 2006-2006).
Liu F, Liu Y. Exploring correlation between ROUGE and human evaluation on meeting summaries. IEEE Transactions on Audio, Speech, and Language Processing. 2009 Jun 10;18(1):187-96.
Jiang J, Zhang H, Dai C, Zhao Q, Feng H, Ji Z, Ganchev I. Enhancements of attention-based bidirectional lstm for hybrid automatic text summarization. IEEE Access. 2021 Sep 3;9:123660-71.
Li Z, Peng Z, Tang S, Zhang C, Ma H. Text summarization method based on double attention pointer network. IEEE Access. 2020 Jan 10;8:11279-88.
Hanunggul PM, Suyanto S. The impact of local attention in lstm for abstractive text summarization. In2019 International Seminar on Research of Information Technology and Intelligent Systems (ISRITI) 2019 Dec 5 (pp. 54-57). IEEE.
Nallapati R, Zhou B, Gulcehre C, Xiang B. Abstractive text summarization using sequence-to-sequence rnns and beyond. arXiv preprint arXiv:1602.06023. 2016 Feb 19.
Wong KF, Wu M, Li W. Extractive summarization using supervised and semi-supervised learning. InProceedings of the 22nd international conference on computational linguistics (Coling 2008) 2008 Aug (pp. 985-992).
Moratanch N, Chitrakala S. A survey on extractive text summarization. In2017 international conference on computer, communication and signal processing (ICCCSP) 2017 Jan 10 (pp. 1-6). IEEE.
Mridha MF, Lima AA, Nur K, Das SC, Hasan M, Kabir MM. A survey of automatic text summarization: Progress, process and challenges. IEEE Access. 2021 Nov 22;9:156043-70.
See A, Liu PJ, Manning CD. Get to the point: Summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368. 2017 Apr 14.
Shi T, Keneshloo Y, Ramakrishnan N, Reddy CK. Neural abstractive text summarization with sequence-to-sequence models. ACM Transactions on Data Science. 2021 Jan 3;2(1):1-37.
Sakhare DY. A Sequence-to-Sequence Text Summarization Using Long Short-Term Memory Based Neural Approach. International Journal of Intelligent Engineering & Systems. 2023 Mar 1;16(2).
Singh S, Singh JP, Deepak A. Deep Learning based Abstractive Summarization for English Language. InWorking Notes of FIRE 2022-Forum for Information Retrieval Evaluation, Kolkata, India 2022 Dec 9.
Kouris P, Alexandridis G, Stafylopatis A. Abstractive text summarization: Enhancing sequence-to-sequence models using word sense disambiguation and semantic content generalization. Computational Linguistics. 2021 Dec 23;47(4):813-59.
Wazery YM, Saleh ME, Alharbi A, Ali AA. Abstractive Arabic text summarization based on deep learning. Computational Intelligence and Neuroscience. 2022 Jan 11;2022.
Siddiqui T, Shamsi JA. Generating abstractive summaries using sequence to sequence attention model. In2018 International Conference on Frontiers of Information Technology (FIT) 2018 Dec 17 (pp. 212-217). IEEE.
Masum AK, Abujar S, Talukder MA, Rabby AS, Hossain SA. Abstractive method of text summarization with sequence to sequence RNNs. In2019 10th international conference on computing, communication and networking technologies (ICCCNT) 2019 Jul 6 (pp. 1-5). IEEE.
Rahman MM, Siddiqui FH. An optimized abstractive text summarization model using peephole convolutional LSTM. Symmetry. 2019 Oct 14;11(10):1290.
Anvitha Aravinda, Gururaja H S, Padmanabha J, Unique Combinations of LSTM for Text Summarization, International Journal of Engineering Research & Technology (IJERT) ICEI – 2022 (Volume 10 – Issue 11)
Sanjabi N. Abstractive text summarization with attention-based mechanism (Master's thesis, Universitat Politècnica de Catalunya).
Huang L, Wu H, Gao Q, Liu G. Attention Localness in Shared Encoder-Decoder Model For Text Summarization. InICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2023 Jun 4 (pp. 1-5). IEEE.
Parker, Robert, et al. English Gigaword Fifth Edition LDC2011T07. Web Download. Philadelphia: Linguistic Data Consortium, 2011.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
