Investigating the Influence of Feature Normalization on Spoken Language Understanding Performance for the Classification Function
Keywords:
WOZ 2.0, grammatically, SLU, HCI, experimentation, Z-score, matrix, utteranceAbstract
Deep learning models are used for improving the performance of many applications such as image processing, natural language processing, video processing, human-computer interaction (HCI), etc. Commercially available HCI systems such as Apple Siri, Microsoft Cortana, and Alexa incorporate deep learning models to enhance their system performance. One of the tasks of HCI is to classify user utterance to a predefined domain-specific slot such as a request for food, area, etc. This task is accompanied by the spoken language understanding (SLU) unit of HCI. The deep learning model in SLU classifies user utterance to understand the user’s intention. The performance of the classification learning model depends upon the quality of the input feature matrix. These feature matrices for SLU are high dimensional and each feature is not on the same scale. Thus, there is no equal contribution from each feature. Therefore, there is a need for applying to feature normalizing techniques to give equal weights to each feature and enhance the classification task in the SLU model. Feature quality in SLU can be improved by pre-processing techniques such as feature normalization, and it will aid to improve the user utterance classification of SLU. The work in this paper investigates the impact of feature normalization techniques on SLU performance for the classification task. The feature normalization techniques investigated for SLU are Z-score, mean-centered, variable stability scaling, min-max normalization, max normalization, decimal scaling normalization, tanh-based normalization, and sigmoidal normalization. The experimentation was done on a publicly available WOZ 2.0 dataset. The feature normalization methods which were more effective in reducing classification error are Z-score and min-max normalization techniques. The less effective techniques in reducing classification errors are decimal scaling, scaling, and log normalization. can result in a page being rejected by search engines. Ensure that your abstract reads well and is grammatically correct.
Downloads
References
Park, T., Yi, S. G., Kang, S. H., Lee, S. Y., Lee, Y. S., & Simon, R. (2003). Evaluation of normalization methods for microarray data. BMC Bioinformatics, 4, 1–13. https://doi.org/10.1186/1471-2105-4-33
Patro, S. G. K., & sahu, K. K. (2015). Normalization: A Preprocessing Stage. Iarjset, 20–22. https://doi.org/10.17148/iarjset.2015.2305
Reinhold, J. C., Dewey, B. E., Carass, A., & Prince, J. L. (2018). Evaluating the Impact of Intensity Normalization on MR Image Synthesis. ArXiv. https://doi.org/10.1117/12.2513089
Cochran, D. (1988). Consequence of Signal Normalization in Spectrum Analysis. ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, 2388–2391. https://doi.org/10.1109/icassp.1988.197121
Pathak, J., Bailey, K. R., Beebe, C. E., Bethard, S., Carrell, D. S., Chen, P. J., Dligach, D., Endle, C. M., Hart, L. A ., Haug, P. J., Huff, S. M., Kaggal, V. C., Li, D., Liu, H., Marchant, K., Masanz, J., Miller, T., Oniki, T. A., Palmer, M., … Chute, C. G. (2013). Normalization and standardization of electronic health records for high-throughput phenotyping: The sharpn consortium. Journal of the American Medical Informatics Association, 20(E2). https://doi.org/10.1136/amiajnl-2013-001939
Giricz, O., Lauer-Fields, J. L., & Fields, G. B. (2008). The normalization of gene expression data in melanoma: Investigating the use of glyceraldehyde 3-phosphate dehydrogenase and 18S ribosomal RNA as internal reference genes for quantitative real-time PCR. Analytical Biochemistry, 380(1), 137–139. https://doi.org/10.1016/j.ab.2008.05.024
Chen, L., Reeve, J., Zhang, L., Huang, S., Wang, X., & Chen, J. (2018). GMPR: A robust normalization method for zero-inflated count data with application to microbiome sequencing data. PeerJ, 2018(4), 1–20. https://doi.org/10.7717/peerj.4600
Pathak, J., Bailey, K. R., Beebe, C. E., Bethard, S., Carrell, D. S., Chen, P. J., Dligach, D., Endle, C. M., Hart, L. A., Haug, P. J., Huff, S. M., Kaggal, V. C., Li, D., Liu, H., Marchant, K., Masanz, J., Miller, T., Oniki, T. A., Palmer, M., … Chute, C. G. (2013). Normalization and standardization of electronic health records for high-throughput phenotyping: The sharpn consortium. Journal of the American Medical Informatics Association, 20(E2). https://doi.org/10.1136/amiajnl-2013-001939
Wei Rao, Man-Wai Mak, Kong-Aik Lee “ Normalization of total variability matrix for i-vector/PLDA speaker verification”,
Published 2015 Mathematics, Computer Science 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).
Yin, Z., Wan, B., Yuan, F., Xia, X., & Shi, J. (2017). A Deep Normalization and Convolutional Neural Network for Image Smoke Detection. IEEE Access, 5, 18429–18438.
Talasila, V., Madhubabu, K., Mahadasyam, M. C., Atchala, N. J., & Kande, L. S. (2020). The prediction of diseases using rough set theory with recurrent neural network in big data analytics. International Journal of Intelligent Engineering and Systems, 13(5), 10–18.
Sumpavakup, C., Mongkoldee, K., Ratniyomchai, T., & Kulworawanichpong, T. (2020). Touch and step voltage evaluation based on computer simulation for a mass rapid transit system in Thailand. International Journal of Intelligent Engineering and Systems, 13(5), 159–169.
Kusuma, E. J., Shidik, G. F., & Pramunendar, R. A. (2020). Optimization of Neural Network using Nelder Mead in Breast Cancer Classification. International Journal of Intelligent Engineering and Systems, 13(6), 330–337.
Fibriani, I., Widjonarko, W., Prasetyo, A., Raharjo, A., & Irawan, D. (2020). Multi Deep Learning to Diagnose COVID-19 in Lung X-Ray Images with Majority Vote Technique. International Journal of Intelligent Engineering and Systems, 13(6), 560–568.
Annamalai, P. (2020). Automatic face recognition using enhanced firefly optimization algorithm and deep belief network. International Journal of Intelligent Engineering and Systems, 13(5), 19–28.
Celikyilmaz, A., 2010, “Convolutional Neural Network Based Semantic Tagging with Entity Embeddings,” Microsoft research , pp. 1–7.lligent Engineering and Systems, 13(6), 560–568.
Dragone, P., 2015, “Non-Sentential Utterances in Dialogue: Experiments in Classification and Interpretation.” In Proc. of the 19th Workshop on the Semantics and Pragmatics of Dialogue.
Sarikaya, R., Hinton, G. E., and Deoras, A., 2014, “Application of Deep Belief Networks for Natural Language Understanding,” IEEE Trans. Audio, Speech Lang. Process., 22(4), pp. 778–784.
Sarikaya, R., Celikyilmaz, A., Deoras, A., Jeong, M., and Corporation, M., 2011, “Shrinkage Based Features for Slot Tagging with Conditional Random Fields,” Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH 2011, 1309-1312.
Macherey, K., Bender, O., Ney, H., and Member, S., 2009, “Applications of Statistical Machine Translation Approaches to Spoken Language Understanding,” IEEE Transactions on Audio, Speech, and Language Processing ( Volume: 17, Issue: 4, May 2009 ) 17(4), pp. 803–818.
Barahona, L. M. R., Gasic, M., Mrkšić, N., Su, P.-H., Ultes, S., Wen, T.-H., and Young, S., 2016, “Exploiting Sentence and Context Representations in Deep Neural Models for Spoken Language Understanding,” Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, The COLING 2016 Organizing Committee, Osaka, Japan, pp. 258--267.
Henderson, M. S., 2015, “Discriminative Methods for Statistical Spoken Dialogue Systems,” Phd thesis University of Cambridge.
Celikyilmaz, A., 2013, “Enriching Word Embeddings Using Knowledge Graph for Semantic Tagging in Conversational Dialog Systems.” AAAI Spring Symposium.
Sarikaya, R., Celikyilmaz, A., Deoras, A., Jeong, M., and Corporation, M., 2011, “Shrinkage Based Features for Slot Tagging with Conditional Random Fields,” Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH 2011, 1309-1312.
Luai Al Shalabi, Zyad Shaaban, Normalization as a preprocessing enginefor data mining and the approach of preference matrix, in: Interna-tional Conference on Dependability of Computer Systems, IEEE, 2006, pp.207–214.
2Robert A van den Berg, Huub CJ HoefslootJohan A Westerhuis, Age K Smilde, Mariët J van der Werf, Centering, scaling, and transformations: improving the biological information content of metabolomics data, BMCGenomics 7 (1) (2006) 142.
Shih-Wei Lin, Kuo-Ching Ying, Shih-Chieh Chen, Zne-Jung Lee, Particle optimization for parameter determination and feature selection ofsupport vector machines, Expert Syst. Appl. 35 (4) (2008) 1817–1824.
Luai Al Shalabi, Zyad Shaaban, Normalization as a preprocessing enginefor data mining and the approach of preference matrix, in: Interna-tional Conference on Dependability of Computer Systems, IEEE, 2006, pp.207–214.
Robert A van den Berg, Huub CJ HoefslootJohan A Westerhuis, Age K Smilde, Mariët J van der Werf, Centering, scaling, and transformations:improving the biological information content of metabolomics data, BMCGenomics 7 (1) (2006) 142.
Antonio Reverter, Wes Barris, Sean Mcwilliam, Keren A Byrne, Yong HWang, Siok H Tan, Nick Hudson, Brian P Dalrymple, Validation of al-ternative methods of data normalization in gene co-expression studies,Bioinformatics 21 (7) (2004) 1112–1120.
Olav M. Kvalheim, Frode Brakstad, Yizeng Liang, Preprocessing of analyticalprofiles in the presence of homoscedastic or heteroscedastic noise, Anal.Chem. 66 (1) (1994) 43–51.
Kevin L. Priddy, Paul E. Keller, Artificial neural networks: an introduction,vol. 68, SPIE press, 2005.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
