Explainable Artificial Intelligence (XAI): Shedding Light on AI's Black Box
Keywords:
Artificial Intelligence, Natural Language Generation, Explainable Artificial IntelligenceAbstract
The rise of AI, especially in critical domains, has raised transparency and accountability concerns due to opaque black-box algorithms. This article explores Explainable AI (XAI) and its application, ‘focusing on Remote Sensing and Signal Processing. AI is increasingly used in sectors like autonomous driving, healthcare, and finance, necessitating transparent decision-making.’ Opaque AI models impact trust, bias, and accountability, driving the need for XAI. XAI provides insights into AI decision rationale, supported by GDPR's right to explanations. In Reinforcement Learning (RL), XAI faces unique challenges due to RL's sequential nature and the lack of human-labeled data. XAI methods include model interpretation, post-hoc explanations, interactive explanations, and hybrid approaches. ‘XAI categories include transparent models, opaque models, model-agnostic and model-specific approaches, explanation by simplification, explanation by feature relevance, visual explanation, and local explanation. Applications span healthcare, criminal justice, natural language processing, autonomous systems, agriculture, finance, computer vision, forecasting, remote sensing, social media, and transportation, enhancing trust and fairness. Challenges in natural language generation involve evaluating, handling ambiguous language, constructing narratives, and communicating data quality. This article highlights XAI's role in making AI transparent, addressing black-box algorithm challenges, and fostering trust and accountability in AI decision-making.
Downloads
References
Glass, A., McGuinness, D. L., & Wolverton, M. (2008). Toward establishing trust in adaptive agents. In Proceedings of the 13th International Conference on Intelligent User Interfaces (pp. 227–236). New York, NY. doi: 10.1145/1378773.1378804.
Carey, P. (2018). Data Protection: A Practical Guide to UK and EU Law. Oxford University Press, Inc.
Anjomshoae, S., Najjar, A., Calvaresi, D., & Främling, K. (2019). Explainable agents and robots: results from a systematic literature review. In 18th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2019), Montreal, Canada, May 13–17, 2019. International Foundation for Autonomous Agents and Multiagent Systems (pp. 1078–1088).
Explainable and Transparent AI and Multi-Agent Systems: Third International Workshop, EXTRAAMAS 2021, Virtual Event, May 3–7, 2021, Revised Selected PapersTowards Explainable Practical Agency: A Logical Perspective.
Adebayo, J., Gilmer, J., Mueller, M., Goodfellow, I., Hardt, M., & Kim, B. (2018). Sanity checks for saliency maps. arXiv [Preprint] arXiv:1810.03292.
Mohanty, S., & Vyas, S. (2018). How to Compete in the Age of Artificial Intelligence: Implementing a Collaborative Human-Machine Strategy for Your Business. Apress. doi: 10.1007/978-1-4842-3808-0.
Baker, B., Kanitscheider, I., Markov, T., Wu, Y., Powell, G., McGrew, B., et al. (2019). Emergent tool use from multi-agent autocurricula. arXiv [Preprint] arXiv:1909.07528.
Adadi, A., & Berrada, M. (2018). Peeking inside the black-box: A survey on explainable artificial intelligence (XAI). IEEE Access, 6, 52138–52160.
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018). Explaining explanations: An overview of interpretability of machine learning. In 2018 IEEE 5th International Conference on data science and advanced analytics (DSAA). IEEE (pp. 80–89).
Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1, 206–215.
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023). Explainable Artificial Intelligence (XAI): What we know and what is left to attain Trustworthy Artificial Intelligence. Information Fusion, 99, 101805. https://doi.org/10.1016/j.inffus.2023.101805.
Amann, J., Blasimme, A., Vayena, E., et al. (2020). Explainability for artificial intelligence in healthcare: A multidisciplinary perspective. BMC Medical Informatics and Decision Making, 20, 310. https://doi.org/10.1186/s12911-020-01332-6.
Adadi, A., & Berrada, M. (2018). Peeking inside the black-box: A survey on explainable artificial intelligence (XAI). IEEE Access, 6, 52138–52160.
Pasquale, F. (2015). The Black Box Society: The Secret Algorithms That Control Money and Information. Harvard University Press. https://doi.org/10.4159/harvard.9780674736061.
Dieber, J., & Kirrane, S. (2020). Why model why? Assessing the strengths and limitations of LIME. arXiv preprint arXiv:2012.00093.
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015). On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS One, 10, e0130140.
Tritscher, J., Ring, M., Schölr, D., Hettinger, L., & Hotho, A. (2020). Evaluation of post-hoc XAI approaches through synthetic tabular data. In International Symposium on Methodologies for Intelligent Systems (pp. 422–430). Springer.
Chen, H., Lundberg, S., & Lee, S.-I. (2019). Explaining models by propagating Shapley values of local components. arXiv preprint arXiv:1911.11888.
Chattopadhay, A., Sarkar, A., Howlader, P., & Balasubramanian, V. N. (2018). Grad-CAM++: Generalized gradient-based visual explanations for deep convolutional networks. In 2018 IEEE Winter conference on applications of computer vision (WACV) (pp. 839–847)
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017). Grad-CAM: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision (pp. 618–626).
Burkart, N., & Huber, M. F. (2020). A survey on the explainability of supervised machine learning. arXiv preprint arXiv:2011.07876.
Adadi, A., & Berrada, M. (2018). Peeking inside the black-box: A survey on explainable artificial intelligence (XAI). IEEE Access, 6, 52138–52160.
Phillips, P. J., Hahn, C. A., Fontana, P. C., Broniatowski, D. A., & Przybocki, M. A. (2020). Four principles of explainable artificial intelligence.
Lundberg, S. M., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30, 4765–4774.
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016). Learning deep features for discriminative localization.
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017). Grad-CAM: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision (pp. 618–626).
Chattopadhay, A., Sarkar, A., Howlader, P., & Balasubramanian, V. N. (2018). Grad-CAM++: Generalized gradient-based visual explanations for deep convolutional networks. In 2018 IEEE Winter conference on applications of computer vision (WACV) (pp. 839–847).
Ibrahim, M., Louie, M., Modarres, C., & Paisley, J. (2019). Global explanations of neural networks: Mapping the landscape of predictions. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, AIES '19 (pp. 279–287). https://doi.org/10.1145/3306618.3314230.
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013). Deep inside convolutional networks: Visualizing image classification models and saliency maps. arXiv:1312.6034 [cs].
Ancona, M., Ceolini, E., Öztireli, C., & Gross, M. (2018). Towards better understanding of gradient-based attribution methods for deep neural networks. http://arxiv.org/abs/1711.06104.
Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., & Sayres, R. (2021). Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). http://arxiv.org/abs/1711.11279.
Ghorbani, A., Wexler, J., Zou, J., & Kim, B. (2019). Towards automatic concept-based explanations. http://arxiv.org/abs/1902.03129.
Dieber, J., & Kirrane, S. (2020). Why model why? Assessing the strengths and limitations of LIME. arXiv preprint arXiv:2012.00093.
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015). On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS One, 10, e0130140.
Noh, H., Hong, S., & Han, B. (2015). Learning deconvolution network for semantic segmentation. In Proceedings of the IEEE international conference on computer vision (pp. 1520–1528).
Angelov, P., & Soares, E. (2020). Towards explainable deep neural networks (xDNN). Neural Networks, 130, 185–194.
Pedreschi, D., Giannotti, F., Guidotti, R., Monreale, A., Ruggieri, S., & Turini, F. (2019). Meaningful explanations of black box AI decision systems. In Proceedings of the AAAI conference on artificial intelligence (Vol. 33, pp. 9780–9784).
Holzinger, A., Biemann, C., Pattichis, C. S., & Kell, D. B. (2017). What do we need to build explainable AI systems for the medical domain? arXiv preprint arXiv:1712.09923.
Soares, E. A., Angelov, P. P., Costa, B., Castro, M., Nageshrao, S., & Filev, D. (2020). Explaining deep learning models through rule-based approximation and visualization. IEEE Transactions on Fuzzy Systems, 1, 1–10.
Couteaux, V., Nempont, O., Pizaine, G., & Bloch, I. (2019). Towards interpretability of segmentation networks by analyzing DeepDreams. In Interpretability of machine intelligence in medical image computing and multimodal learning for clinical decision support (pp. 56–63). Springer.
Dressel, J., & Farid, H. (2018). The accuracy, fairness, and limits of predicting recidivism. Science Advances, 4, eaao5580.
Smith-Renner, A., Rua, R., & Colony, M. (2019). Towards an explainable threat detection tool. In IUI workshops.
Stilgoe, J. (2020). Who Killed Elaine Herzberg? In: Who’s Driving Innovation?. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-030-32320-2_1
Soares, E., Angelov, P., Costa, B., & Castro, M. (2019). Actively semi-supervised deep rule-based classifier applied to adverse driving scenarios. In 2019 international joint conference on neural networks (IJCNN). IEEE (pp. 1–8).
Jahmunah, V., Ng, E. Y. K., Tan, R. S., Oh, S. L., & Acharya, U. R. (2022). Explainable detection of myocardial infarction using deep learning models with Grad-CAM technique on ECG signals. Computers in Biology and Medicine, 146, 105550. https://doi.org/10.1016/j.compbiomed.2022.105550
Wei, K., Chen, B., Zhang, J., Fan, S., Wu, K., Liu, G., & Chen, D. (2022). Explainable Deep Learning Study for Leaf Disease Classification. Agronomy, 12, 1035. https://doi.org/10.3390/agronomy12051035
De, T., Giri, P., Mevawala, A., Nemani, R., & Deo, A. (2020). Explainable AI: A Hybrid Approach to Generate Human-Interpretable Explanation for Deep Learning Prediction. Procedia Computer Science, 168, 40-48. https://doi.org/10.1016/j.procs.2020.02.255
Joshi, G., Walambe, R., & Kotecha, K. (2021). A Review on Explainability in Multimodal Deep Neural Nets. IEEE Access, 9, 59800-59821. doi: 10.1109/ACCESS.2021.3070212
Naeem, H., Alshammari, B. M., & Ullah, F. (2022). Explainable Artificial Intelligence-Based IoT Device Malware Detection Mechanism Using Image Visualization and Fine-Tuned CNN-Based Transfer Learning Model. Computational Intelligence and Neuroscience, Volume 2022, Article ID 7671967, 17 pages. https://doi.org/10.1155/2022/7671967
Roanec, J. M., Fortuna, B., Mladeni, D. (2022). Knowledge graph-based rich and confidentiality preserving Explainable Artificial Intelligence (XAI). Information Fusion, 81, 91-102. https://doi.org/10.1016/j.inffus.2021.11.015
Kim, D., & Lee, J. (2022). Predictive evaluation of spectrogram-based vehicle sound quality via data augmentation and explainable artificial Intelligence: Image color adjustment with brightness and contrast. Mechanical Systems and Signal Processing, Volume 179, 109363. https://doi.org/10.1016/j.ymssp.2022.109363.
Kakogeorgiou, I., & Karantzalos, K. (2021). Evaluating explainable artificial intelligence methods for multi-label deep learning classification tasks in remote sensing. International Journal of Applied Earth Observation and Geoinformation, Volume 103, 102520. https://doi.org/10.1016/j.jag.2021.102520.
Lim, S.-Y., Chae, D.-K., & Lee, S.-C. (2022). Detecting Deepfake Voice Using Explainable Deep Learning Techniques. Appl. Sci., 12, 3926. https://doi.org/10.3390/app12083926.
Szczepański, M., Pawlicki, M., Kozik, R., et al. (2021). New explainability method for BERT-based model in fake news detection. Sci Rep, 11, 23705. https://doi.org/10.1038/s41598-021-03100-6.
Kim, H.-S., & Joe, I. (2022). An XAI method for convolutional neural networks in self-driving cars. PLoS ONE, 17(8), e0267282. https://doi.org/10.1371/journal.pone.0267282.
Reiter, E. (2019). Natural Language Generation Challenges for Explainable AI. In Proceedings of the 1st Workshop on Interactive Natural Language Technology for Explainable Artificial Intelligence (NL4XAI 2019), pages 3–7. Association for Computational Linguistics.
Sai, A. M. A., Sai Eswar, K. L., Sai Harshith, K. S., Raghavendra, P., Kiran, G. Y., & M. V. (2022). Study of Lasso and Ridge Regression using ADMM. In 2022 2nd International Conference on Intelligent Technologies (CONIT), Hubli, India, pp. 1-8. doi: 10.1109/CONIT55038.2022.9847706.
Van Deemter, K. (2010). Not exactly: In praise of vagueness. OUP Oxford.
Daniel, K. (2017). Thinking, fast and slow.
Reiter, E. (2019). Natural Language Generation Challenges for Explainable AI. In Proceedings of the 1st Workshop on Interactive Natural Language Technology for Explainable Artificial Intelligence (NL4XAI 2019), pages 3–7. Association for Computational Linguistics.
Karakülah, G., Dicle, O., Koşaner, O., et al. (2014). Computer-based extraction of phenotypic features of human congenital anomalies from the digital literature with natural language processing techniques. Stud Health Technol Inform, 205, 570–574.
Dam, H. K., Tran, T., & Ghose, A. (2018). Explainable software analytics. In Proceedings of the 40th International Conference on Software Engineering: New Ideas and Emerging Results, Gothenburg, Sweden, 27 May–3 June 2018, pp. 53–56.
Lipton, Z. C. (2018). The mythos of model interpretability. Commun. ACM, 61, 36–43.
Olden, J. D., Joy, M. K., & Death, R. G. (2004). An accurate comparison of methods for quantifying variable importance in artificial neural networks using simulated data. Ecological Modelling, Volume 178, Issues 3–4, 389–397. https://doi.org/10.1016/j.ecolmodel.2004.03.013.
Adadi, A., & Berrada, M. (2020). Explainable AI for Healthcare: From Black Box to Interpretable Models. In Advances in Intelligent Systems and Computing, Volume 1076, pp. 327–337. Springer.
Kleinbaum, D. G., & Kleinbaum, D. G. (1994). Logistic Regression. Springer.
Lundberg, S. M., & Lee, S. I. (2017). A unified approach to interpreting model predictions. Adv. Neural Inf. Process. Syst., 2017, 4766–4775.
Alvarez-Melis, D., & Jaakkola, T. S. (2017). A causal framework for explaining the predictions of black-box sequence-to-sequence models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark, 7–11 September 2017, pp. 412–421.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
