Transformer Based Implementation for Automatic Book Summarization
Keywords:
Summarization, Extractive, Abstractive, TransformerAbstract
Document Summarization is the procedure of generating a meaningful and concise summary of a given document with the inclusion of relevant and topic-important points. There are two approaches- one is picking up the most relevant statements from the document itself and adding it to the Summary known as Extractive and the other is generating sentences for the Summary known as Abstractive Summarization. Training a machine learning model to perform tasks that are time-consuming or very difficult for humans to evaluate is major challenge. Book summarization is one of the complex tasks which is time consuming as well. Traditional machine learning models are getting modified with pre-trained transformers. Transformer based Language models trained in a self-supervised fashion gaining a lot of attention when fine-tuned for Natural Language Processing(NLP) downstream task like text summarization. This work is an attempt to use Transformer based technique for Book Summarization.
Downloads
References
N. Andhale and L. A. Bewoor, “An overview of text summarization techniques,” 2017, doi: 10.1109/ICCUBEA.2016.7860024.
N. S. Shirwandkar and S. Kulkarni, “Extractive Text Summarization Using Deep Learning,” Proc. - 2018 4th Int. Conf. Comput. Commun. Control Autom. ICCUBEA 2018, 2018, doi: 10.1109/ICCUBEA.2018.8697465.
S. Singhal, “Abstractive Text Summarization,” J. Xidian Univ., vol. 14, no. 6, pp. 1–11, 2020, doi: 10.37896/jxu14.6/094..
A. Vaswani et al., “Attention is all you need,” Adv. Neural Inf. Process. Syst., vol. 2017-December, no. Nips, pp. 5999–6009, 2017.
S. Singh and A. Mahmood, “The NLP Cookbook: Modern Recipes for Transformer Based Deep Learning Architectures,” IEEE Access, vol. 9, pp. 68675–68702, 2021, doi: 10.1109/ACCESS.2021.3077350.
J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” NAACL HLT 2019 - 2019 Conf. North Am. Chapter Assoc. Comput. Linguist. Hum. Lang. Technol. - Proc. Conf., vol. 1, no. Mlm, pp. 4171–4186, 2019.
R. Mihalcea and H. Ceylan, “Explorations in automatic book summarization,” EMNLP-CoNLL 2007 - Proc. 2007 Jt. Conf. Empir. Methods Nat. Lang. Process. Comput. Nat. Lang. Learn., no. June, pp. 380–389, 2007.
J. Wu et al., “Recursively Summarizing Books with Human Feedback,” 2021, Available: http://arxiv.org/abs/2109.10862.
H. P. Luhn, “The Automatic Creation of Literature Abstracts,” IBM J. Res. Dev., vol. 2, no. 2, pp. 159–165, 2010, doi: 10.1147/rd.22.0159.
A. Kumar, A. Sharma, S. Sharma, and S. Kashyap, “Performance analysis of keyword extraction algorithms assessing extractive text summarization,” 2017 Int. Conf. Comput. Commun. Electron. COMPTELIX 2017, pp. 408–414, 2017, doi: 10.1109/ COMPTELIX.2017.8004004.
R. Bhargava and Y. Sharma, “Deep Extractive Text Summarization,” Procedia Comput. Sci., vol. 167, no. 2019, pp. 138–146, 2020, doi: 10.1016/j.procs.2020.03.191.
S. Syed, Abstractive Summarization of Social Media Posts: A case Study using Deep Learning, Master’s thesis, Bauhaus University, Weimar, Germany, 2017.
I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” Adv. Neural Inf. Process. Syst., vol. 4, no. January, pp. 3104–3112, 2014.
K. Lopyrev, Generating news headlines with recurrent neural networks, p. 9, 2015, https://arxiv.org/abs/1512.01712.
K. Al-Sabahi, Z. Zuping, and Y. Kang, Bidirectional Attentional Encoder-Decoder Model and Bidirectional Beam Search for Abstractive Summarization, Cornell University, Ithaca, NY, USA, 2018, http://arxiv.org/abs/1809.06662.
Tomer, M., Kumar, M. Improving Text Summarization using Ensembled Approach based on Fuzzy with LSTM. Arab J Sci Eng 45, 10743–10754 (2020). https://doi.org/10.1007/s13369-020-04827-6
Song, S., Huang, H. & Ruan, T. Abstractive text summarization using LSTM-CNN based deep learning. Multimed Tools Appl 78, 857–875 (2019). https://doi.org/10.1007/s11042-018-5749-3
D. Miller, “Leveraging BERT for Extractive Text Summarization on Lectures,” 2019, Available: http://arxiv.org/abs/1906.04165.
J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” NAACL HLT 2019 - 2019 Conf. North Am. Chapter Assoc. Comput. Linguist. Hum. Lang. Technol. - Proc. Conf., vol. 1, no. Mlm, pp. 4171–4186, 2019.
I. Tenney, D. Das, and E. Pavlick, “BERT rediscovers the classical NLP pipeline,” ACL 2019 - 57th Annu. Meet. Assoc. Comput. Linguist. Proc. Conf., pp. 4593–4601, 2020, doi: 10.18653/v1/p19-1452.
S. Zoupanos, S. Kolovos, A. Kanavos, O. Papadimitriou, and M. Maragoudakis, “Efficient comparison of sentence embeddings,” 2022. Available: http://arxiv.org/abs/2204.00820.
V. Sanh, L. Debut, J. Chaumond, and T. Wolf, “DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter,” pp. 2–6, 2019, [Online]. Available: http://arxiv.org/abs/1910.01108.
Lin, Chin-Yew. ROUGE: A Package for Automatic Evaluation of Summaries. In Proceedings of the Workshop on Text Summarization Branches Out (WAS 2004), Barcelona, Spain, July 25 - 26, 2004.
D. Suleiman and A. Awajan, “Deep Learning Based Abstractive Text Summarization: Approaches, Datasets, Evaluation Measures, and Challenges,” Math. Probl. Eng., vol. 2020, 2020, doi: 10.1155/2020/9365340.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
