Improving Document Categorization Models using Explanatory MLP and Batch Normalization: A Novel Methodology Featuring Logistic Regression Weights Transfer
Keywords:
Deep learning, Explanatory MLP, Multi-Layer Perceptron, Linear Knowledge Transfer, Linear Regression.Abstract
In this era of increasing textual documents on various platforms, it is important to have a text classification system that can categorize the text documents. We extracted the Reuters8 dataset and reduced its dimensions by using information gain. Later we applied the MLP model on the dataset to classify the text documents. The MLP model is applied by modifying in four different ways, first MLP is applied with assumed weights and to tune the hyperparameters GridSearchCV is used. Then batch normalization was performed where input of each layer is normalized by adjusting the activations. Next explanatory MLP was performed where the weights are taken from linear regression. Finally, linear knowledge was performed where no. of neurons in a hidden layer are taken in a sequence based on the number of categories in the dataset. In the Reuters8 dataset there are 8 classes. Out of all the variations it is found that explanatory MLP has given the best results.
Downloads
References
X. Feng, G. Ma, S.-F. Su, C. Huang, M. K. Boswell, and P. Xue, “A multi-layer perceptron approach for accelerated wave forecasting in Lake Michigan,” Ocean Eng., vol. 211, p. 107526, Sep. 2020, doi: 10.1016/j.oceaneng.2020.107526.
Nagaraja, U. Boregowda, K. Khatatneh, R. Vangipuram, R. Nuvvusetty, and V. Sravan Kiran, “Similarity Based Feature Transformation for Network Anomaly Detection,” IEEE Access, vol. 8, pp. 39184–39196, 2020, doi: 10.1109/ACCESS.2020.2975716.
N. Kamath, S. S. Bukhari, and A. Dengel, “Comparative Study between Traditional Machine Learning and Deep Learning Approaches for Text Classification,” in Proceedings of the ACM Symposium on Document Engineering 2018, Halifax NS Canada: ACM, Aug. 2018, pp. 1–11. doi: 10.1145/3209280.3209526.
M. P. Akhter, Z. Jiangbin, I. R. Naqvi, M. Abdelmajeed, A. Mehmood, and M. T. Sadiq, “Document-Level Text Classification Using Single-Layer Multisize Filters Convolutional Neural Network,” IEEE Access, vol. 8, pp. 42689–42707, 2020, doi: 10.1109/ACCESS.2020.2976744.
S. Lakhotia and X. Bresson, “An Experimental Comparison of Text Classification Techniques,” in 2018 International Conference on Cyberworlds (CW), Singapore: IEEE, Oct. 2018, pp. 58–65. doi: 10.1109/CW.2018.00022.
M. Jiang et al., “Text classification based on deep belief network and softmax regression,” Neural Comput. Appl., vol. 29, no. 1, pp. 61–70, Jan. 2018, doi: 10.1007/s00521-016-2401-x.
R. Valupadasu and B. R. R. Chunduri, “Automatic Classification of Cardiac Disorders Using MLP Algorithm,” in 2019 Prognostics and System Health Management Conference (PHM-Paris), Paris, France: IEEE, May 2019, pp. 253–257. doi: 10.1109/PHM-Paris.2019.00050.
I. A. Cahyani, P. I. Ashuri, and C. S. K. Aditya, “Stunting Disease Classification Using Multi-Layer Perceptron Algorithm with GridSearchCV,” Sinkron, vol. 9, no. 1, pp. 392–401, Jan. 2024, doi: 10.33395/sinkron.v9i1.13245.
M. Bounabi, K. El Moutaouakil, and K. Satori, “A Probabilistic Vector Representation and Neural Network for Text Classification,” in Big Data, Cloud and Applications, vol. 872, Y. Tabii, M. Lazaar, M. Al Achhab, and N. Enneya, Eds., Cham: Springer International Publishing, 2018, pp. 343–355. doi: 10.1007/978-3-319-96292-4_27.
M. Zhang, “Applications of Deep Learning in News Text Classification,” Sci. Program., vol. 2021, pp. 1–9, Aug. 2021, doi: 10.1155/2021/6095354.
T. Dönicke, M. Damaschk, and F. Lux, “Multiclass Text Classification on Unbalanced, Sparse and Noisy Data,” in Proceedings of the First NLPL Workshop on Deep Learning for Natural Language Processing, J. Nivre, L. Derczynski, F. Ginter, B. Lindi, S. Oepen, A. Søgaard, and J. Tidemann, Eds., Turku, Finland: Linköping University Electronic Press, Sep. 2019, pp. 58–65. Accessed: Apr. 14, 2024. [Online]. Available: https://aclanthology.org/W19-6207
“Microstrip bandpass filters with ultra-broad rejection band using stepped impedance resonator and high-impedance transformer,” in IEEE MTT-S International Microwave Symposium Digest, 2005., Long Beach, CA, USA: IEEE, 2005, pp. 683–686. doi: 10.1109/MWSYM.2005.1516699.
Y. Han, P. Han, B. Yuan, Z. Zhang, L. Liu, and J. Panneerselvam, “Novel Transformation Deep Learning Model for Electrocardiogram Classification and Arrhythmia Detection using Edge Computing,” J. Grid Comput., vol. 22, no. 1, p. 7, Mar. 2024, doi: 10.1007/s10723-023-09717-3.
K. Sabanci, M. Koklu, and M. F. Unlersen, “Classification of Siirt and Long Type Pistachios (Pistacia vera L.) by Artificial Neural Networks,” Int. J. Intell. Syst. Appl. Eng., vol. 3, no. 2, p. 86, Apr. 2015, doi: 10.18201/ijisae.74573.
Y. Gao, T. Zhu, and X. Xu, “Bone age assessment based on deep convolution neural network incorporated with segmentation,” Int. J. Comput. Assist. Radiol. Surg., vol. 15, no. 12, pp. 1951–1962, Dec. 2020, doi: 10.1007/s11548-020-02266-0.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
