Rejection Threshold Optimization using 3D ROC Curves: Novel Findings on Biomedical Datasets - Uyar, Asli; Atilgan Sengül, Yasemin (International Journal of Intelligent Systems and Applications in Engineering, 2021)
Reject option is introduced in classification tasks to prevent potential misclassifications. Although optimization of error-reject trade-off has been widely investigated, it is shown that error rate itself is not an appropriate performance measure, when misclassification costs are unequal or class distributions are imbalanced. ROC analysis is proposed as an alternative approach to performance evaluation in terms of true positives (TP) and false positives (FP). Considering classification with reject option, we need to represent the tradeoff between TP, FP and rejection rates. In this paper, we propose 3D ROC analysis to determine the optimal rejection threshold as an analogy to decision threshold optimization in 2D ROC curves. We have demonstrated our proposed method with Naive Bayes classifier on Heart Disease dataset and validated the efficiency of the method on Pima Indians Diabetes dataset. Our experiments reveal that classification with optimized rejection threshold significantly improves true positive rates in medical datasets. Furthermore, false positive rates remain the same with rejection rates below 10%.