Exploring the Efficacy of LSTM Networks in Machine Translation: A Survey of Techniques and Applications
Keywords:
Machine Translation, Neural Machine Translation, Recurrent Neural Network (RNN), Multi-Layered Perceptron, Long Short-term Memory (LSTM)Abstract
Machine Translation (MT) has significantly advanced with the advent of neural network architectures, among which Long Short-Term Memory (LSTM) networks have garnered substantial attention. This paper presents a comprehensive survey on the LSTM networks in MT tasks. This Paper delve into the architecture of MLPs, RNNs and LSTMs, advantages of LSTMs over traditional recurrent neural networks (RNNs), and their suitability for capturing long-range dependencies. Furthermore, this paper examines various approaches of gates adopted in leveraging LSTM networks for MT. This work emphasizes the benefits, drawbacks, and possible directions for further research in this field through a critical review of the body of existing work.
Downloads
References
Almeida, L. B. (1987). A learning rule for asynchronous perceptrons with feedback in a combinatorial environment. In IEEE 1st International Conference on Neural Networks, San Diego, volume 2, pages 609-618
Marino, D. L., Amarasinghe, K., and Manic, M., Building energy load forecasting using deep neural networks. in IECON 2016-42nd Annual Conference of the IEEE Industrial Electronics Society, 2016, pp. 7046– 7051.
Cherrier, N., Castaings, T., and Boulch, A., Deep sequence-to-sequence neural networks for ionospheric activity map prediction. in International Conference on Neural Information Processing, 2017, pp. 545–555.
Lindemann, B., Jazdi, N., and Weyrich, M., Detektion von Anomalien zur Qualitätssicherung basierend auf Sequence-to-Sequence LSTM Netzen. atAutomatisierungstechnik, vol. 67, no. 12, 2019, pp. 1058–1068.
Du, S., Li, T., and Horng, S.-J., Time Series Forecasting Using Sequenceto-Sequence Deep Learning Framework. in 2018 9th International Symposium on Parallel Architectures, Algorithms and Programming (PAAP), 2018, pp. 171–176.
Gu, J., Lu, Z., Li, H., and Li, V. O. K., Incorporating copying mechanism in sequence-to-sequence learning. arXiv preprint arXiv:1603.06393, 2016.
Jaitly, N., Le, Q. V., Vinyals, O., Sutskever, I., Sussillo, D., and Bengio, S., An online sequence-to-sequence model using partial conditioning. in Advances in Neural Information Processing Systems, 2016, pp. 5067– 5075.
Cinar, Y. G., Mirisaee, H., Goswami, P., Gaussier, E., Aït-Bachir, A., and Strijov, V., Position-based content attention for time series forecasting with sequence-to-sequence rnns. in International Conference on Neural Information Processing, 2017, pp. 533–544.
Author, F., Author, S.: Title of a proceedings paper. In: Editor, F., Editor, S. (eds.) CONFERENCE 2016, LNCS, vol. 9999, pp. 1–13. Springer, Heidelberg (2016).
Author, F., Author, S., Author, T.: Book title. 2nd edn. Publisher, Location (1999).
Author, F.: Contribution title. In: 9th International Proceedings on Proceedings, pp. 1–2. Publisher, Location (2010).
LNCS Homepage, http://www.springer.com/lncs, last accessed 2016/11/21.
ALMANSOR, E. H. 2018. Translating Arabic as low resource language using distribution representation and neural machine translation models.
ANJU, E. & MANOJ KUMAR, K. 2014. Malayalam to English machine translation: An EBMT system. IOSR Journal of Engineering (IOSRJEN), 4, 18-23.
BIGALHI94 2022. Language-translation-with-transformer-model. BROWNLEE, J. 2017. What Are Word Embeddings for Text? Available from: https://machinelearningmastery.com/what-are-word-embeddings/.
COLAH. 2015. Understanding LSTM Networks. Available from: https://colah.github.io/posts/2015-08- Understanding-LSTMs/.
GARG, A. & AGARWAL, M. 2018. Machine translation: a literature review. arXiv preprint arXiv:1901.01122.
KHAN, N. J., ANWAR, W. & DURRANI, N. 2017. Machine translation approaches and survey for Indian languages. arXiv preprint arXiv:1701.04290.
KLAPPENBACH, A. 2022. The 12 most spoken languages in the world [Online]. Available: https://blog.busuu.com/most-spoken-languages-in-the-world/ [Accessed].
MOREL, R. 2019. Why Do We Need Translation? [Online]. Available: https://lighthouseonline.com/blog-en/why-do-we-need-translation/ [Accessed].
MOSES-SMT 2019. Nonbreaking prefixes.
NGUYEN, T. T. 2019. Machine translation with transformers.
REDDIT 2017. Question about Positional Encodings used in "Attention is all you need" paper.
SCIONOFTECH 2019. English-Telugu-Bilingual-Sentence-Pairs.
SINGHAL, G. 2020a. Getting Started with RNN. Available from: https://www.pluralsight.com/guides/getting-started-with-rnn.
SINGHAL, G. 2020b. Introduction to LSTM Units in RNN. Available from: https://www.pluralsight.com/guides/introduction-to-lstm-units-in-rnn.
VASWANI, A., SHAZEER, N., PARMAR, N., USZKOREIT, J., JONES, L., GOMEZ, A. N., KAISER, Ł. & POLOSUKHIN, I. 2017. Attention is all you need. Advances in neural information processing systems, 30.
WEAVER, W. Translation. Proceedings of the Conference on Mechanical Translation, 1952.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
