Transformer Based Implementation for Automatic Book Summarization



Summarization, Extractive, Abstractive, Transformer


Document Summarization is the procedure of generating a meaningful and concise summary of a given document with the inclusion of relevant and topic-important points. There are two approaches- one is picking up the most relevant statements from the document itself and adding it to the Summary known as Extractive and the other is generating sentences for the Summary known as Abstractive Summarization. Training a machine learning model to perform tasks that are time-consuming or very difficult for humans to evaluate is major challenge. Book summarization is one of the complex tasks which is time consuming as well.  Traditional machine learning models are getting modified with pre-trained transformers. Transformer based Language models trained in a self-supervised fashion gaining a lot of attention when fine-tuned for Natural Language Processing(NLP) downstream task like text summarization. This work is an attempt to use Transformer based technique for Book Summarization.


Download data is not yet available.


N. Andhale and L. A. Bewoor, “An overview of text summarization techniques,” 2017, doi: 10.1109/ICCUBEA.2016.7860024.

N. S. Shirwandkar and S. Kulkarni, “Extractive Text Summarization Using Deep Learning,” Proc. - 2018 4th Int. Conf. Comput. Commun. Control Autom. ICCUBEA 2018, 2018, doi: 10.1109/ICCUBEA.2018.8697465.

S. Singhal, “Abstractive Text Summarization,” J. Xidian Univ., vol. 14, no. 6, pp. 1–11, 2020, doi: 10.37896/jxu14.6/094..

A. Vaswani et al., “Attention is all you need,” Adv. Neural Inf. Process. Syst., vol. 2017-December, no. Nips, pp. 5999–6009, 2017.

S. Singh and A. Mahmood, “The NLP Cookbook: Modern Recipes for Transformer Based Deep Learning Architectures,” IEEE Access, vol. 9, pp. 68675–68702, 2021, doi: 10.1109/ACCESS.2021.3077350.

J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” NAACL HLT 2019 - 2019 Conf. North Am. Chapter Assoc. Comput. Linguist. Hum. Lang. Technol. - Proc. Conf., vol. 1, no. Mlm, pp. 4171–4186, 2019.

R. Mihalcea and H. Ceylan, “Explorations in automatic book summarization,” EMNLP-CoNLL 2007 - Proc. 2007 Jt. Conf. Empir. Methods Nat. Lang. Process. Comput. Nat. Lang. Learn., no. June, pp. 380–389, 2007.

J. Wu et al., “Recursively Summarizing Books with Human Feedback,” 2021, Available:

H. P. Luhn, “The Automatic Creation of Literature Abstracts,” IBM J. Res. Dev., vol. 2, no. 2, pp. 159–165, 2010, doi: 10.1147/rd.22.0159.

A. Kumar, A. Sharma, S. Sharma, and S. Kashyap, “Performance analysis of keyword extraction algorithms assessing extractive text summarization,” 2017 Int. Conf. Comput. Commun. Electron. COMPTELIX 2017, pp. 408–414, 2017, doi: 10.1109/ COMPTELIX.2017.8004004.

R. Bhargava and Y. Sharma, “Deep Extractive Text Summarization,” Procedia Comput. Sci., vol. 167, no. 2019, pp. 138–146, 2020, doi: 10.1016/j.procs.2020.03.191.

S. Syed, Abstractive Summarization of Social Media Posts: A case Study using Deep Learning, Master’s thesis, Bauhaus University, Weimar, Germany, 2017.

I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” Adv. Neural Inf. Process. Syst., vol. 4, no. January, pp. 3104–3112, 2014.

K. Lopyrev, Generating news headlines with recurrent neural networks, p. 9, 2015,

K. Al-Sabahi, Z. Zuping, and Y. Kang, Bidirectional Attentional Encoder-Decoder Model and Bidirectional Beam Search for Abstractive Summarization, Cornell University, Ithaca, NY, USA, 2018,

Tomer, M., Kumar, M. Improving Text Summarization using Ensembled Approach based on Fuzzy with LSTM. Arab J Sci Eng 45, 10743–10754 (2020).

Song, S., Huang, H. & Ruan, T. Abstractive text summarization using LSTM-CNN based deep learning. Multimed Tools Appl 78, 857–875 (2019).

D. Miller, “Leveraging BERT for Extractive Text Summarization on Lectures,” 2019, Available:

J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” NAACL HLT 2019 - 2019 Conf. North Am. Chapter Assoc. Comput. Linguist. Hum. Lang. Technol. - Proc. Conf., vol. 1, no. Mlm, pp. 4171–4186, 2019.

I. Tenney, D. Das, and E. Pavlick, “BERT rediscovers the classical NLP pipeline,” ACL 2019 - 57th Annu. Meet. Assoc. Comput. Linguist. Proc. Conf., pp. 4593–4601, 2020, doi: 10.18653/v1/p19-1452.

S. Zoupanos, S. Kolovos, A. Kanavos, O. Papadimitriou, and M. Maragoudakis, “Efficient comparison of sentence embeddings,” 2022. Available:

V. Sanh, L. Debut, J. Chaumond, and T. Wolf, “DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter,” pp. 2–6, 2019, [Online]. Available:

Lin, Chin-Yew. ROUGE: A Package for Automatic Evaluation of Summaries. In Proceedings of the Workshop on Text Summarization Branches Out (WAS 2004), Barcelona, Spain, July 25 - 26, 2004.

D. Suleiman and A. Awajan, “Deep Learning Based Abstractive Text Summarization: Approaches, Datasets, Evaluation Measures, and Challenges,” Math. Probl. Eng., vol. 2020, 2020, doi: 10.1155/2020/9365340.

Workflow of the Model




How to Cite

Porwal, S. ., Bewoor, L. ., & Deshpande, V. . (2022). Transformer Based Implementation for Automatic Book Summarization. International Journal of Intelligent Systems and Applications in Engineering, 10(3s), 123–128. Retrieved from



Research Article