Bias Detection and Mitigation within Decision Support System: A Comprehensive Survey
Keywords:
Bias, Decision support system, Fairness Metrics, MitigationAbstract
In decision support system biases plays a vital role to lead unfair and discriminatory outcomes within the system, which can have serious consequences for people and society as a whole. While significant work has been done in classical machine learning and deep learning to address these difficulties, still there is a need for extensive surveys that evaluate various real-world applications and causes of bias in decision support systems. In this paper, we present a comprehensive survey that explores the biases, detection of biases, mitigation of biases, and fairness metrics to measure the degree of fairness in decision support systems. Further, we also identified several challenges related to biases such as minimizing biases when working with inadequate datasets, ensuring proper representation of protected attributes, developing efficient and direct methods for bias detection, identifying effective approaches for mitigating biases at various stages of the model, developing strategies to effectively mitigate multiple biases in the system to build a fair prediction model and at last, exploring and refining fairness metrics to achieve more fair results. We have also provided the research questions based on these challenges with the solutions and interesting future research avenues that might help to alleviate the problem of bias in decision support systems. We hope that this poll will motivate scholars to confront these issues and help the creation of more equitable systems.
Downloads
References
Power DJ. Understanding data-driven decision support systems. Information Systems Management. 2008 Mar 28;25(2):149-54.
Dastin J. Amazon scraps secret AI recruiting tool that showed bias against women. In Ethics of data and analytics 2018 Oct 10 (pp. 296-299). Auerbach Publications.
Holstein T, Dodig-Crnkovic G. Avoiding the intrinsic unfairness of the trolley problem. In Proceedings of the International Workshop on Software Fairness 2018 May 29 (pp. 32-37).
Power DJ. Decision support systems: concepts and resources for managers. Greenwood Publishing Group; 2002.
Power D. What are the features of a data-driven DSS? DSS News. 2007 Feb;8(4):6.
Balayn A, Lofi C, Houben GJ. Managing bias and unfairness in data for decision support: a survey of machine learning and data engineering approaches to identify and mitigate bias and unfairness within data management and analytics systems. The VLDB Journal. 2021 Sep;30(5):739-68.
Srinivasan R, Chander A. Biases in AI systems. Communications of the ACM. 2021 Jul 26;64(8):44-9.
Torralba A, Efros AA. An unbiased look at dataset bias. InCVPR 2011 2011 Jun 20 (pp. 1521-1528). IEEE.
Buolamwini J, Gebru T. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability, and transparency 2018 Jan 21 (pp. 77-91). PMLR.
Srinivasan R, Chander A. Crowdsourcing in the Absence of Ground Truth–A Case Study. arXiv preprint arXiv:1906.07254. 2019 Jun 17.
Plous S. The psychology of judgment and decision making. Mcgraw-Hill Book Company; 1993.
Kahneman D. Evaluation by moments: Past and future. Choices, values, and frames. 2000 Sep 25:693-708.
Hao K. This is how AI bias happens—and why It’s so hard to fix, 2019.
Elwert F, Winship C. Endogenous selection bias: The problem of conditioning on a collider variable. Annual review of sociology. 2014 Jul 30;40:31-53.
Quinn T. Judea Pearl and Dana Mackenzie: THE BOOK OF WHY: The new science of cause and effect. TLS. Times Literary Supplement. 2018 Sep 21(6025):31-2.
Kilbertus N, Ball PJ, Kusner MJ, Weller A, Silva R. The sensitivity of counterfactual fairness to unmeasured confounding. In Uncertainty in artificial intelligence 2020 Aug 6 (pp. 616-626). PMLR.
Friedman B, Nissenbaum H. Bias in computer systems. ACM Transactions on Information Systems (TOIS). 1996 Jul 1;14(3):330-47.
Mehrabi N, Morstatter F, Saxena N, Lerman K, Galstyan A. A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR). 2021 Jul
;54(6):1-35.
Amatriain X. What does the concept of presentation-feedback bias refer to in the context of machine learning? quora, 2015.
Austin PC, Platt RW. Survivor treatment bias, treatment selection bias, and propensity scores in observational research. Journal of clinical epidemiology. 2010 Feb 1;63(2):136-8.
Schick, Timo, Sahana Udupa, and Hinrich Schu¨tze. ”Self-diagnosis and self-debiasing: A proposal for reducing corpus-based bias in NLP.” Transactions of the Association for Computational Linguistics 9 (2021): 1408-1424.
Sirotkin, Kirill, Pablo Carballeira, and Marcos Escudero-Vin˜olo. ”A study on the distribution of social biases in self-supervised learning visual models.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.
Field, Anjalie, and Yulia Tsvetkov. ”Unsupervised discovery of implicit gender bias.” arXiv preprint arXiv:2004.08361 (2020).
Lim, Sora, et al. ”Annotating and analyzing biased sentences in news articles using crowdsourcing.” Proceedings of the Twelfth Language Resources and Evaluation Conference. 2020.
Terrin, Norma, et al. ”Adjusting for publication bias in the presence of heterogeneity.” Statistics in medicine 22.13 (2003): 2113-2126.
Waller, Niels G., Jane S. Thompson, and Ernst Wenk. ”Using IRT to separate measurement bias from true group differences on homogeneous and heterogeneous scales: an illustration with the MMPI.” Psychological Methods 5.1 (2000): 125.
Altmann, Andr´e, et al. ”Permutation importance: a corrected feature importance measure.” Bioinformatics 26.10 (2010): 1340-1347.
Smith, James F., and Thomas Kida. ”Heuristics and biases: Expertise and task realism in auditing.” Psychological bulletin 109.3 (1991): 472.
Abusitta, Adel, Esma A¨ımeur, and Omar Abdel Wahab. ”Generative adversarial networks for mitigating biases in machine learning systems.” arXiv preprint arXiv:1905.09972 (2019).
Darlow, Luke, Stanis law Jastrzebski, and Amos Storkey. ”Latent adversarial debiasing: Mitigating collider bias in deep neural networks.” arXiv preprint arXiv:2011.11486 (2020).
Nascimento, Francimaria RS, George DC Cavalcanti, and Marjory Da CostaAbreu. ”Unintended bias evaluation: An analysis of hate speech detection and gender bias mitigation on social media using ensemble learning.” Expert Systems with Applications 201 (2022): 117032.
Tarawneh, Ahmad S., et al. ”Smotefuna: Synthetic minority over-sampling technique based on furthest neighbour algorithm.” IEEE Access 8 (2020): 59069-59082.
Elhassan, T., and M. Aljurf. ”Classification of imbalance data using tomek link (t-link) combined with random under-sampling (rus) as a data reduction method.” Global J Technol Optim S 1 (2016): 2016.
Wang, Zhibo, et al. ”Fairness-aware adversarial perturbation towards bias mitigation for deployed deep models.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.
Nikolaev, Nikolay Y., and Hitoshi Iba. ”Regularization approach to inductive genetic programming.” IEEE Transactions on evolutionary computation 5.4 (2001): 359-375.
Hufthammer, Knut T., et al. ”Bias mitigation with AIF360: A comparative study.” Norsk IKT-konferanse for forskning og utdanning. No. 1. 2020.
Zhou N, Zhang Z, Nair VN, Singhal H, Chen J, Sudjianto A. Bias, Fairness, and Accountability with AI and ML Algorithms. arXiv preprint arXiv:2105.06558. 2021 May 13.
Mehrabi N, Morstatter F, Saxena N, Lerman K, Galstyan A. A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR). 2021 Jul
;54(6):1-35.
Ritov YA, Sun Y, Zhao R. On conditional parity as a notion of non-discrimination in machine learning. arXiv preprint arXiv:1706.08519. 2017 Jun 26.
Verma S, Rubin J. Fairness definitions explained. In Proceedings of the international workshop on software fairness 2018 May 29 (pp. 1-7).
Russell C, Kusner MJ, Loftus J, Silva R. When worlds collide: integrating different counterfactual assumptions in fairness. Advances in neural information processing systems. 2017;30.
Afzal W, Torkar R, Feldt R. A systematic review of search-based testing for non-functional system properties. Information and Software Technology. 2009 Jun 1;51(6):957-76.
Yahav I, Shehory O, Schwartz D. Comments mining with TF-IDF: the inherent bias and its removal. IEEE Transactions on Knowledge and Data Engineering. 2018 May 24;31(3):437-50.
Speicher T, Heidari H, Grgic-Hlaca N, Gummadi KP, Singla A, Weller A, Zafar MB. A unified approach to quantifying algorithmic unfairness: Measuring individual group unfairness via inequality indices. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery data mining 2018 Jul 19 (pp. 2239-2248).
Xu Y, Yang Y, Han J, Wang E, Zhuang F, Xiong H. Exploiting the sentimental bias between ratings and reviews for enhancing recommendation. In2018 IEEE international conference on data mining (ICDM) 2018 Nov 17 (pp. 1356-1361). IEEE.
Lauw HW, Lim EP, Wang K. Bias and controversy in evaluation systems. IEEE Transactions on Knowledge and Data Engineering. 2008 Apr 25;20(11):1490-504
Feelders AJ, Chang S, McLachlan GJ. Mining in the Presence of Selectivity Bias and its Application to Reject Inference. InKDD 1998 Aug 27 (pp. 199-203).
Norori, Natalia, et al. “Addressing bias in big data and AI for healthcare: A call for open science.” Patterns 2.10 (2021): 100347.
Zhao, Jieyu, and Kai-Wei Chang. “LOGAN: Local group bias detection by clustering.” arXiv preprint arXiv:2010.02867 (2020)
Zliobaite, Indre. “A survey on measuring indirect discrimination in machine learning.” arXiv preprint arXiv:1511.00148 (2015).
Kruse, Clemens Scott, et al. ”Challenges and opportunities of big data in health care: a systematic review.” JMIR medical informatics 4.4 (2016): e5359.
Heudecker N. “Hype Cycle for Big Data.” Gartner. URL: https://www.gartner.com/doc/2574616/ hype-cycle-big-data- [accessed 2016-11-08] [WebCite Cache ID 6lsI6Sxxr] 2013 Jul 31.
Chawla, Nitesh V., and Darcy A. Davis. “Bringing big data to personalized healthcare: a patient-centered framework.” Journal of general internal medicine 28.3 (2013): 660-665.
Jee, Kyoungyoung, and Gang-Hoon Kim. “Potentiality of big data in the medical sector: focus on how to reshape the healthcare system.” Healthcare informatics research 19.2 (2013): 79-85
Akbari, Ali, et al. ”A flatter loss for bias mitigation in cross-dataset facial age estimation.” 2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 2021.
Hammond, M. Elizabeth H., et al. ”Bias in medicine: lessons learned and mitigation strategies.” Basic to Translational Science 6.1 (2021): 78-85.
Mahabadi, Rabeeh Karimi, Yonatan Belinkov, and James Henderson. ”Endto-end bias mitigation by modelling biases in corpora.” arXiv preprint arXiv:1909.06321 (2019).
Bender, Emily M., and Batya Friedman. ”Data statements for natural language processing: Toward mitigating system bias and enabling better science.” Transactions of the Association for Computational Linguistics 6 (2018): 587-604.
Wang, Tianlu, et al. ”Balanced datasets are not enough: Estimating and mitigating gender bias in deep image representations.” Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019.
Maudslay, Rowan Hall, et al. ”It’s all in the name: Mitigating gender bias with name-based counterfactual data substitution.” arXiv preprint arXiv:1909.00871 (2019).
Hort, Max, et al. ”Fairea: A model behaviour mutation approach to benchmarking bias mitigation methods.” Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 2021.
Tae KH, Roh Y, Oh YH, Kim H, Whang SE. Data cleaning for accurate, fair, and robust models: A big data-AI integration approach. In Proceedings of the 3rd International Workshop on Data Management for End-to-End Machine Learning 2019 Jun 30 (pp. 1-4).
Zhang L, Wu Y, Wu X. Causal modelling-based discrimination discovery and removal: criteria, bounds, and algorithms. IEEE Transactions on Knowledge and Data Engineering. 2018 Sep 30;31(11):2035-50.
Hajian S, Domingo-Ferrer J. A methodology for direct and indirect discrimination prevention in data mining. IEEE Transactions on Knowledge and data engineering. 2012 Apr 3;25(7):1445-59.
Elbassuoni S, Amer-Yahia S, Ghizzawi A, El Atie C. Exploring fairness of ranking in online job marketplaces. In22nd International Conference on Extending Database Technology (EDBT) 2019 Mar 26.
Kamishima T, Akaho S, Asoh H, Sakuma J. Model-based and actual independence for fairness-aware classification. Data mining and knowledge discovery. 2018 Jan;32:258-86.
P´erez-Suay A, Laparra V, Mateo-Garc´ıa G, Mun˜oz-Mar´ı J, G´omez-Chova L, Camps-Valls G. Fair kernel learning. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2017, Skopje, Macedonia, September 18–22, 2017, Proceedings, Part I 2017 Dec 30 (pp. 339-355). Cham: Springer International Publishing.
Kamishima T, Akaho S, Sakuma J. Fairness-aware learning through regularization approach. In2011 IEEE 11th International Conference on Data Mining Workshops 2011 Dec 11 (pp. 643-650). IEEE.
Mancuhan K, Clifton C. Discriminatory decision policy aware classification. In2012 IEEE 12th International Conference on Data Mining Workshops 2012 Dec 10 (pp. 386-393). IEEE.
Luong BT, Ruggieri S, Turini F. k-NN as an implementation of situation testing for discrimination discovery and prevention. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining 2011 Aug 21 (pp. 502-510).
Kamishima T, Akaho S, Asoh H, Sakuma J. The independence of fairness-aware classifiers. In2013 IEEE 13th International Conference on Data Mining Workshops 2013 Dec 7 (pp. 849-858). IEEE.
Zliobaite I, Kamiran F, Calders T. Handling conditional discrimination. In2011ˇ IEEE 11th international conference on data mining 2011 Dec 11 (pp. 992-1001). IEEE.
Kamiran F, Karim A, Zhang X. Decision theory for discrimination-aware classification. In2012 IEEE 12th international conference on data mining 2012 Dec 10 (pp. 924-929). IEEE.
Norori N, Hu Q, Aellen FM, Faraci FD, Tzovara A. Addressing bias in big data and AI for health care: A call for open science. Patterns. 2021 Oct 8;2(10):100347.
Prakhar, Jyoti, and Md Tanwir Uddin Haider. “Automated Detection of Biases within the Healthcare System Using Clustering and Logistic Regression.” 2023 15th International Conference on Computer and Automation Engineering (ICCAE). IEEE, 2023.
Rayavarapu, S. M. ., Prashanthi, T. S. ., Kumar, G. S. ., Lavanya, Y. L. ., & Rao, G. S. . (2023). A Generative Adversarial Network Based Approach for Synthesis of Deep Fake Electrocardiograms . International Journal on Recent and Innovation Trends in Computing and Communication, 11(3), 223–227. https://doi.org/10.17762/ijritcc.v11i3.6340
Mr. Ather Parvez Abdul Khalil. (2012). Healthcare System through Wireless Body Area Networks (WBAN) using Telosb Motes. International Journal of New Practices in Management and Engineering, 1(02), 01 - 07. Retrieved from http://ijnpme.org/index.php/IJNPME/article/view/4
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
