A Pragmatic Review of Learning Models Used for Unsupervised Analysis of Existing Cyber Physical Deployments from an Empirical Perspective
Keywords:
Neural, Network, Cyber, Physical, Unsupervised, Scalability, Empirical, Complexity, Automation, ControlAbstract
Cyber-physical deployments include game engines, multimedia systems, internet of Things (IoT) systems, etc. Each of these models has certain inputs, several processing layers, and certain outputs. Monitoring & control of such deployments can be automated via their unsupervised analysis, which requires deep learning & pattern analysis methods. A wide variety of such models are proposed by researchers and system designers, but each of them has its own nuances, advantages, limitations, & future research scopes. Moreover, these models have different performance characteristics, that vary in terms of analysis accuracy, precision, recall, fMeasure, delay of analysis, response time, computational complexity, etc. Thus, while deploying such learning models, researchers & system designers are required to perform manual analysis, validation, and testing for automation & control. Due to this cumbersome process, the cost & time to market for these unsupervised control models is very high, which limits their scalability, and deployment capabilities. To overcome this issue, a detailed characteristic discussion of these models is done in this text. Based on this discussion, researchers will be able to identify existing unsupervised & semi-supervised learning models, which closely match their deployments. These models are further analyzed in terms of their performance metrics, that includes, accuracy of analysis, response time needed for control, delay needed for analysis, precision of analysis, computational complexity, and cost of deployment. Using these metrics, researchers can evaluate best performing models for their deployments, which will assist them in reducing cost, and time needed for automating their cyber physical systems. This text also discusses certain future prospects that can be explored by researchers in order to further enhance quality of their deployments.
Downloads
References
A. -J. Gallego, J. Calvo-Zaragoza and R. B. Fisher, "Incremental Unsupervised Domain-Adversarial Training of Neural Networks," in IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 11, pp. 4864-4878, Nov. 2021, doi: 10.1109/TNNLS.2020.3025954.
L. Xiang, G. Zhao, Q. Li, W. Hao and F. Li, "TUMK-ELM: A Fast Unsupervised Heterogeneous Data Learning Approach," in IEEE Access, vol. 6, pp. 35305-35315, 2018, doi: 10.1109/ACCESS.2018.2847037.
Y. Feng, Y. Yuan and X. Lu, "Person Reidentification via Unsupervised Cross-View Metric Learning," in IEEE Transactions on Cybernetics, vol. 51, no. 4, pp. 1849-1859, April 2021, doi: 10.1109/TCYB.2019.2909480.
M. Usama et al., "Unsupervised Machine Learning for Networking: Techniques, Applications and Research Challenges," in IEEE Access, vol. 7, pp. 65579-65615, 2019, doi: 10.1109/ACCESS.2019.2916648.
H. Chang, J. Han, C. Zhong, A. M. Snijders and J. -H. Mao, "Unsupervised Transfer Learning via Multi-Scale Convolutional Sparse Coding for Biomedical Applications," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, no. 5, pp. 1182-1194, 1 May 2018, doi: 10.1109/TPAMI.2017.2656884.
C. Shang, C. -Y. Chang, G. Chen, S. Zhao and J. Lin, "Implicit Irregularity Detection Using Unsupervised Learning on Daily Behaviors," in IEEE Journal of Biomedical and Health Informatics, vol. 24, no. 1, pp. 131-143, Jan. 2020, doi: 10.1109/JBHI.2019.2896976.
X. Li, H. Zhang, R. Zhang and F. Nie, "Discriminative and Uncorrelated Feature Selection With Constrained Spectral Analysis in Unsupervised Learning," in IEEE Transactions on Image Processing, vol. 29, pp. 2139-2149, 2020, doi: 10.1109/TIP.2019.2947776.
C. S. Wickramasinghe, K. Amarasinghe, D. L. Marino, C. Rieger and M. Manic, "Explainable Unsupervised Machine Learning for Cyber-Physical Systems," in IEEE Access, vol. 9, pp. 131824-131843, 2021, doi: 10.1109/ACCESS.2021.3112397.
J. Gao, F. Li, B. Wang and H. Liang, "Unsupervised nonlinear adaptive manifold learning for global and local information," in Tsinghua Science and Technology, vol. 26, no. 2, pp. 163-171, April 2021, doi: 10.26599/TST.2019.9010049.
G. Chakraborty and B. Chakraborty, "A novel normalization technique for unsupervised learning in ANN," in IEEE Transactions on Neural Networks, vol. 11, no. 1, pp. 253-257, Jan. 2000, doi: 10.1109/72.822529.
C. Zhu, L. Cao and J. Yin, "Unsupervised Heterogeneous Coupling Learning for Categorical Representation," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 1, pp. 533-549, 1 Jan. 2022, doi: 10.1109/TPAMI.2020.3010953.
Sedaghat, Nafiseh & Fathy, Mahmood & Modarressi, Mohammad H. & Shojaie, Ali. (2017). Combination of Supervised and Unsupervised Approaches for miRNA Target Prediction. IEEE/ACM Transactions on Computational Biology and Bioinformatics. PP. 1-1. 10.1109/TCBB.2017.2727042.
G. Pu, L. Wang, J. Shen and F. Dong, "A hybrid unsupervised clustering-based anomaly detection method," in Tsinghua Science and Technology, vol. 26, no. 2, pp. 146-153, April 2021, doi: 10.26599/TST.2019.9010051.
I. Muñoz-Martín, S. Bianchi, G. Pedretti, O. Melnic, S. Ambrogio and D. Ielmini, "Unsupervised Learning to Overcome Catastrophic Forgetting in Neural Networks," in IEEE Journal on Exploratory Solid-State Computational Devices and Circuits, vol. 5, no. 1, pp. 58-66, June 2019, doi: 10.1109/JXCDC.2019.2911135.
S. Joung, S. Kim, K. Park and K. Sohn, "Unsupervised Stereo Matching Using Confidential Correspondence Consistency," in IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 5, pp. 2190-2203, May 2020, doi: 10.1109/TITS.2019.2917538.
Fuchs, Franz & Hjelmervik, Jon. (2014). Interactive Isogeometric Volume Visualization with Pixel-Accurate Geometry. IEEE Transactions on Visualization and Computer Graphics. 10.1109/TVCG.2015.2430337.
C. Shang, C. -Y. Chang, G. Chen, S. Zhao and H. Chen, "BIA: Behavior Identification Algorithm Using Unsupervised Learning Based on Sensor Data for Home Elderly," in IEEE Journal of Biomedical and Health Informatics, vol. 24, no. 6, pp. 1589-1600, June 2020, doi: 10.1109/JBHI.2019.2943391.
T. Liao, L. Xue, W. Hu and L. Yi, "Unsupervised Learning for Neural Network-Based Blind Equalization," in IEEE Photonics Technology Letters, vol. 32, no. 10, pp. 569-572, 15 May15, 2020, doi: 10.1109/LPT.2020.2985307.
H. Jia, Y. -m. Cheung and J. Liu, "A New Distance Metric for Unsupervised Learning of Categorical Data," in IEEE Transactions on Neural Networks and Learning Systems, vol. 27, no. 5, pp. 1065-1079, May 2016, doi: 10.1109/TNNLS.2015.2436432.
S. Wang, J. Cai, Q. Lin and W. Guo, "An Overview of Unsupervised Deep Feature Representation for Text Categorization," in IEEE Transactions on Computational Social Systems, vol. 6, no. 3, pp. 504-517, June 2019, doi: 10.1109/TCSS.2019.2910599.
Lake, Brenden & Vallabha, Gautam & Mcclelland, James. (2009). Modeling Unsupervised Perceptual Category Learning. IEEE T. Autonomous Mental Development. 1. 35-43. 10.1109/DEVLRN.2008.4640800.
G. Huang, S. Song, J. N. D. Gupta and C. Wu, "Semi-Supervised and Unsupervised Extreme Learning Machines," in IEEE Transactions on Cybernetics, vol. 44, no. 12, pp. 2405-2417, Dec. 2014, doi: 10.1109/TCYB.2014.2307349.
F. Wang, Q. Wang, F. Nie, Z. Li, W. Yu and R. Wang, "Unsupervised Linear Discriminant Analysis for Jointly Clustering and Subspace Learning," in IEEE Transactions on Knowledge and Data Engineering, vol. 33, no. 3, pp. 1276-1290, 1 March 2021, doi: 10.1109/TKDE.2019.2939524.
Michael Ustaszewski (2021) Towards a machine learning approach to the analysis of indirect translation, Translation Studies, 14:3, 313-331, DOI: 10.1080/14781700.2021.1894226
Kim, Wonjik & Kanezaki, Asako & Masayuki, Tanaka. (2020). Unsupervised Learning of Image Segmentation Based on Differentiable Feature Clustering. IEEE Transactions on Image Processing. 29. 1-1. 10.1109/TIP.2020.3011269.
J. G. Wolff, "Autonomous Robots and the SP Theory of Intelligence," in IEEE Access, vol. 2, pp. 1629-1651, 2014, doi: 10.1109/ACCESS.2014.2382753.
J. Gao, C. Zhong, X. Chen, H. Lin and Z. Zhang, "Unsupervised Learning for Passive Beamforming," in IEEE Communications Letters, vol. 24, no. 5, pp. 1052-1056, May 2020, doi: 10.1109/LCOMM.2020.2965532.
F. Liu, G. Zhang and J. Lu, "Heterogeneous Domain Adaptation: An Unsupervised Approach," in IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 12, pp. 5588-5602, Dec. 2020, doi: 10.1109/TNNLS.2020.2973293.
Y. Wang and Y. Xu, "Unsupervised Learning of Accurate Camera Pose and Depth From Video Sequences With Kalman Filter," in IEEE Access, vol. 7, pp. 32796-32804, 2019, doi: 10.1109/ACCESS.2019.2903871.
Phan, Ha & Kumar, Ashnil & Fulham, Michael & Kim, Jinman. (2019). Unsupervised Two-Path Neural Network for Cell Event Detection and Classification Using Spatiotemporal Patterns. IEEE Transactions on Medical Imaging. 38. 1477-1487. 10.1109/TMI.2018.2885572.
S. -H. Cheng, Y. -T. Shih and K. -C. Chang, "Proactive Power Control and Position Deployment for Drone Small Cells: Joint Supervised and Unsupervised Learning," in IEEE Access, vol. 9, pp. 126735-126747, 2021, doi: 10.1109/ACCESS.2021.3111964.
J. Xie, J. Fang, C. Liu and L. Yang, "Unsupervised Deep Spectrum Sensing: A Variational Auto-Encoder Based Approach," in IEEE Transactions on Vehicular Technology, vol. 69, no. 5, pp. 5307-5319, May 2020, doi: 10.1109/TVT.2020.2982203.
K. Zheng et al., "Coupled Convolutional Neural Network With Adaptive Response Function Learning for Unsupervised Hyperspectral Super Resolution," in IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 3, pp. 2487-2502, March 2021, doi: 10.1109/TGRS.2020.3006534.
H. Song, M. Zhang, J. Gao and C. Zhong, "Unsupervised Learning-Based Joint Active and Passive Beamforming Design for Reconfigurable Intelligent Surfaces Aided Wireless Networks," in IEEE Communications Letters, vol. 25, no. 3, pp. 892-896, March 2021, doi: 10.1109/LCOMM.2020.3041510.
Z. Gong, P. Zhong and W. Hu, "Unsupervised Deep Feature Learning With Iteratively Refined Pseudo Classes for Scene Representation," in IEEE Access, vol. 7, pp. 94779-94792, 2019, doi: 10.1109/ACCESS.2019.2928328.
A. Gupta, P. P. Roy and V. Dutt, "Evaluation of Instance-Based Learning and Q-Learning Algorithms in Dynamic Environments," in IEEE Access, vol. 9, pp. 138775-138790, 2021, doi: 10.1109/ACCESS.2021.3117855.
C. Zhao, G. G. Yen, Q. Sun, C. Zhang and Y. Tang, "Masked GAN for Unsupervised Depth and Pose Prediction With Scale Consistency," in IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 12, pp. 5392-5403, Dec. 2021, doi: 10.1109/TNNLS.2020.3044181.
N. Rathi and K. Roy, "STDP Based Unsupervised Multimodal Learning With Cross-Modal Processing in Spiking Neural Networks," in IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 5, no. 1, pp. 143-153, Feb. 2021, doi: 10.1109/TETCI.2018.2872014.
J. H. Seong and D. H. Seo, "Selective Unsupervised Learning-Based Wi-Fi Fingerprint System Using Autoencoder and GAN," in IEEE Internet of Things Journal, vol. 7, no. 3, pp. 1898-1909, March 2020, doi: 10.1109/JIOT.2019.2956986.
Pan, Y. and Wollack, J.A. (2021), An Unsupervised-Learning-Based Approach to Compromised Items Detection. Journal of Educational Measurement, 58: 413-433. https://doi.org/10.1111/jedm.12299
Eltouny, KA, Liang, X. Bayesian-optimized unsupervised learning approach for structural damage detection. Comput Aided Civ Inf. 2021; 36: 1249– 1269. https://doi.org/10.1111/mice.12680
Jo, Taeho. (2021). Machine Learning Foundations: Supervised, Unsupervised, and Advanced Learning. 10.1007/978-3-030-65900-4.
Xueli Wang, Jingjuan Gao, "Optimization Model of Logistics Task Allocation Based on Genetic Algorithm", Security and Communication Networks, vol. 2022, Article ID 5950876, 13 pages, 2022. https://doi.org/10.1155/2022/5950876
JingJing Zhao, "Risk Assessment Method of Agricultural Management Investment Based on Genetic Neural Network", Security and Communication Networks, vol. 2022, Article ID 2373363, 10 pages, 2022. https://doi.org/10.1155/2022/2373363
Zhenglei Zhang, "The Application of PSO-SVM Algorithm in the Evaluation System of Sports Competition Events", Security and Communication Networks, vol. 2022, Article ID 6865425, 8 pages, 2022. https://doi.org/10.1155/2022/6865425
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
