Failure Modes of AI Systems in Regulated Environments A Systems Architecture Perspective
Keywords:
Artificial Intelligence, AI System Architecture, Failure Modes, Regulated Environments, Compliance Risk, AuditabilityAbstract
Automated systems of artificial intelligence are gradually penetrating the controlled activities in finance, healthcare, and public services. A lot of the failures of these systems are considered as model or data failures. The thesis that is presented in this paper suggests that the majority of compliance failures are caused by flaws in the architecture of a system and not the errors in algorithms. The study is based on a quantitative, architecture-level analysis to find out the prevalence of failure modes in data pipelines, model lifecycle management, inference systems, and monitoring architectures. The findings indicate that there are evident trends between architectural design decisions and audit failures and audit governance risks. The paper brings out compliance-native architectural solutions which lessen risk by tracking, determinism and governance controls.
Downloads
References
Manheim, D. (2019). Multiparty dynamics and failure modes for machine learning and artificial intelligence. Big Data and Cognitive Computing, 3(2), 21. https://doi.org/10.3390/bdcc3020021
Stadler, J. J., & Seidl, N. J. (2013). Software failure modes and effects analysis. Software Failure Modes and Effects Analysis, 1–5. https://doi.org/10.1109/rams.2013.6517710
Kumar, R. S. S., O’Brien, D. R., Albert, K., Viljöen, S., & Snover, J. (2019). Failure modes in machine learning systems. arXiv (Cornell University). https://doi.org/10.48550/arxiv.1911.11034
Meng, D., Zhou, W., & Zhan, J. (2009). Multidimensional analysis of system logs in large-scale cluster systems. arXiv (Cornell University). https://doi.org/10.48550/arxiv.0906.1328
Güdemann, M., & Ortmeier, F. (2010). Probabilistic Model-Based Safety analysis. Electronic Proceedings in Theoretical Computer Science, 28, 114–128. https://doi.org/10.4204/eptcs.28.8
Snee, R. D., & Rodebaugh, W. F. (2007). Failure Modes and Effects Analysis. Encyclopedia of Statistics in Quality and Reliability. https://doi.org/10.1002/9780470061572.eqr411
Huang, G., Wang, W., Liu, T., & Mei, H. (2011). Simulation-based analysis of middleware service impact on system reliability: Experiment on Java application server. Journal of Systems and Software, 84(7), 1160–1170. https://doi.org/10.1016/j.jss.2011.02.008
VPatil, M., & Yogi, A. M. N. (2011). Importance of data collection and validation for systematic software development process. International Journal of Computer Science and Information Technology, 3(2), 260–278. https://doi.org/10.5121/ijcsit.2011.3220
Kaur, S., & Kumar, D. (2011). Quality prediction of object oriented software using density based clustering approach. In IACSIT International Journal of Engineering and Technology, IACSIT International Journal of Engineering and Technology: Vol. No.4. https://www.ijetch.org/papers/267-T781.pdf
Tekinerdogan, B., Sozer, H., & Aksit, M. (2007). Software architecture reliability analysis using failure scenarios. Journal of Systems and Software, 81(4), 558–575. https://doi.org/10.1016/j.jss.2007.10.029
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
