Automatic Headline Generation for Hindi News using Fine-tuned Large Language Models
Keywords:
Text Summarization, Fine-tuning, BART, indicBART, mT5Abstract
Generating news headlines is one use case for automated text summarization. With only a few text lines, it creates a summary of the longer content, cutting down on reading time. Text summarization is a very challenging task and it is very difficult to generate summaries as a human being. Most summary tools available in the market primarily concentrate on summarizing English material, resulting in a scarcity of summarizers for other languages. In this study, we used two datasets gathered from Dainik Bhaskar, NavBharat Times, and one publically available dataset to fine-tune four pre-trained language models: Someman/bart-hindi, facebook/mbart-large-50, indicBART, and mT5. To conduct a full performance assessment, a variety of evaluation metrics are utilized. Our comprehensive examination consistently reveals that the facebook/mbart-large-50 model exhibits superior performance compared to other models in terms of these metrics. This highlights its potential to enhance automated summarization systems and facilitate enhanced content retrieval and comprehension within the Hindi-speaking community.
Downloads
References
A. P. Widyassari et al., “Review of automatic text summarization techniques & methods,” J. King Saud Univ. - Comput. Inf. Sci., vol. 34, no. 4, pp. 1029–1046, Apr. 2022, doi: 10.1016/J.JKSUCI.2020.05.006.
G. Sharma and D. Sharma, “Automatic Text Summarization Methods: A Comprehensive Review,” SN Comput. Sci., vol. 4, no. 1, pp. 1–18, Jan. 2023, doi: 10.1007/S42979-022-01446-W/METRICS.
Y. Kumar, K. Kaur, and S. Kaur, “Study of automatic text summarization approaches in different languages,” Artif. Intell. Rev., vol. 54, no. 8, pp. 5897–5929, Dec. 2021, doi: 10.1007/S10462-021-09964-4/METRICS.
A. V. Pradeepika Verma, “Accountability of NLP Tools in Text Summarization for Indian Languages,” J. Sci. Res., vol. 64, no. 1, pp. 358–363, 2020.
W. S. El-Kassas, C. R. Salama, A. A. Rafea, and H. K. Mohamed, “Automatic text summarization: A comprehensive survey,” Expert Syst. Appl., vol. 165, p. 113679, Mar. 2021, doi: 10.1016/J.ESWA.2020.113679.
A. K. Yadav et al., “Extractive text summarization using deep learning approach,” Int. J. Inf. Technol., vol. 14, no. 5, pp. 2407–2415, Aug. 2022, doi: 10.1007/S41870-022-00863-7/METRICS.
E. R. Mahalleh and F. S. Gharehchopogh, “An automatic text summarization based on valuable sentences selection,” Int. J. Inf. Technol., vol. 14, no. 6, pp. 2963–2969, Oct. 2022, doi: 10.1007/S41870-022-01049-X/METRICS.
S. Mandal, G. K. Singh, and A. Pal, “Single document text summarization technique using optimal combination of cuckoo search algorithm, sentence scoring and sentiment score,” Int. J. Inf. Technol., vol. 13, no. 5, pp. 1805–1813, Oct. 2021, doi: 10.1007/S41870-021-00739-2/METRICS.
P. J. Goutom, N. Baruah, and P. Sonowal, “An abstractive text summarization using deep learning in Assamese,” Int. J. Inf. Technol., vol. 15, no. 5, pp. 2365–2372, Jun. 2023, doi: 10.1007/S41870-023-01279-7/METRICS.
G. B. Mohan and R. P. Kumar, “Lattice abstraction-based content summarization using baseline abstractive lexical chaining progress,” Int. J. Inf. Technol., vol. 15, no. 1, pp. 369–378, Jan. 2023, doi: 10.1007/S41870-022-01080-Y/METRICS.
K. Rudra, S. Banerjee, N. Ganguly, P. Goyal, M. Imran, and P. Mitra, “Summarizing situational tweets in crisis scenario,” HT 2016 - Proc. 27th ACM Conf. Hypertext Soc. Media, pp. 137–147, Jul. 2016, doi: 10.1145/2914586.2914600.
I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to Sequence Learning with Neural Networks,” Adv. Neural Inf. Process. Syst., vol. 4, no. January, pp. 3104–3112, Sep. 2014, doi: 10.48550/arxiv.1409.3215.
V. Dalal and L. Malik, “Semantic graph based automatic text summarization for hindi documents using particle swarm optimization,” Smart Innov. Syst. Technol., vol. 84, pp. 284–289, 2018, doi: 10.1007/978-3-319-63645-0_31/COVER.
M. L. Joshi, N. Joshi, and N. Mittal, “SGATS: Semantic Graph-based Automatic Text Summarization from Hindi Text Documents,” Trans. Asian Low-Resource Lang. Inf. Process., vol. 20, no. 6, Sep. 2021, doi: 10.1145/3464381.
A. Jain, A. Arora, J. Morato, D. Yadav, and K. V. Kumar, “Automatic Text Summarization for Hindi Using Real Coded Genetic Algorithm,” Appl. Sci., vol. 12, no. 13, 2022, doi: 10.3390/app12136584.
S. Dhankhar and M. K. Gupta, “A statistically based sentence scoring method using mathematical combination for extractive Hindi text summarization,” J. Interdiscip. Math., vol. 25, no. 3, pp. 773–790, 2022, doi: 10.1080/09720502.2021.2015096.
S. S. Aote, A. Pimpalshende, A. Potnurwar, and S. Lohi, “Binary Particle Swarm Optimization with an improved genetic algorithm to solve multi-document text summarization problem of Hindi documents,” Eng. Appl. Artif. Intell., vol. 117, p. 105575, 2023, doi: https://doi.org/10.1016/j.engappai.2022.105575.
R. Bhargava, G. Sharma, and Y. Sharma, “Deep Text Summarization using Generative Adversarial Networks in Indian Languages,” Procedia Comput. Sci., vol. 167, pp. 147–153, Jan. 2020, doi: 10.1016/J.PROCS.2020.03.192.
M. Singh and V. Yadav, “Abstractive Text Summarization Using Attention-based Stacked LSTM,” pp. 236–241, Oct. 2022, doi: 10.1109/CCICT56684.2022.00052.
A. Shah, D. Zanzmera, and K. Mehta, “Deep Learning based Automatic Hindi Text Summarization,” Proc. - 6th Int. Conf. Comput. Methodol. Commun. ICCMC 2022, pp. 1455–1461, 2022, doi: 10.1109/ICCMC53470.2022.9753735.
S. S. Aote, A. Pimpalshende, A. Potnurwar, and S. Lohi, “Binary Particle Swarm Optimization with an improved genetic algorithm to solve multi-document text summarization problem of Hindi documents,” Eng. Appl. Artif. Intell., vol. 117, p. 105575, Jan. 2023, doi: 10.1016/J.ENGAPPAI.2022.105575.
S. Bandari and V. V. Bulusu, “Feature extraction based deep long short term memory for Hindi document summarization using political elephant herding optimization,” Int. J. Intell. Robot. Appl., vol. 7, no. 1, pp. 103–118, 2023, doi: 10.1007/s41315-022-00237-z.
R. Bhansali, A. Bhave, G. Bharat, V. Mahajan, and M. L. Dhore, “Abstractive Text Summarization of Hindi Corpus Using Transformer Encoder-Decoder Model,” Smart Innov. Syst. Technol., vol. 333, pp. 171–185, 2023, doi: 10.1007/978-981-19-8094-7_13/COVER.
D. Taunk and V. Varma, “Summarizing Indian Languages using Multilingual Transformers based Models,” CEUR Workshop Proc., vol. 3395, pp. 435–442, Mar. 2023, Accessed: Sep. 29, 2023. [Online]. Available: https://arxiv.org/abs/2303.16657v1
and S. S. Agarwal, Arjit, Soham Naik, “Abstractive Text Summarization for Hindi Language using IndicBART,” Work. Notes FIRE 2022-Forum Inf. Retr. Eval., 2022.
“Hindi Text Short Summarization Corpus | Kaggle.” https://www.kaggle.com/datasets/disisbig/hindi-text-short-summarization-corpus (accessed Feb. 02, 2023).
G. Arora, “iNLTK: Natural Language Toolkit for Indic Languages,” pp. 66–71, Sep. 2020, doi: 10.18653/v1/2020.nlposs-1.10.
R. Dabre, H. Shrotriya, A. Kunchukuttan, R. Puduppully, M. M. Khapra, and P. Kumar, “IndicBART: A Pre-trained Model for Indic Natural Language Generation,” Proc. Annu. Meet. Assoc. Comput. Linguist., vol. 2, pp. 1849–1863, 2022, doi: 10.18653/V1/2022.FINDINGS-ACL.145.
L. Xue et al., “mT5: A massively multilingual pre-trained text-to-text transformer,” NAACL-HLT 2021 - 2021 Conf. North Am. Chapter Assoc. Comput. Linguist. Hum. Lang. Technol. Proc. Conf., pp. 483–498, Oct. 2020, doi: 10.18653/v1/2021.naacl-main.41.
C.-Y. Lin, “ROUGE: A Package for Automatic Evaluation of Summaries,” in Text Summarization Branches Out, Barcelona, Spain: Association for Computational Linguistics, Jul. 2004, pp. 74–81. [Online]. Available: https://aclanthology.org/W04-1013
K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a Method for Automatic Evaluation of Machine Translation,” Proc. 40th Annu. Meet. Assoc. Comput. Linguist. - ACL ’02, pp. 311–318, 2002, doi: 10.3115/1073083.1073135.
T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, and Y. Artzi, “Bertscore: Evaluating text generation with bert,” arXiv Prepr. arXiv1904.09675, 2019.
“GitHub - TechyNilesh/SSEM: SSEM is a semantic similarity-based evaluation library for natural language processing (NLP) text generation tasks. It supports various similarity metrics and evaluation levels, and is compatible with any Hugging Face pre-trained transformer model.” https://github.com/TechyNilesh/SSEM (accessed Sep. 25, 2023).
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
