Random Search-Based Parameter Optimization on Binary Classifiers for Software Defect Prediction

Abstract

Machine learning classifiers consist of a set of parameters. The efficiency of these classifiers in the context of software defect prediction is greatly impacted by the parameters chosen to execute the classifiers. These parameters can be optimized to achieve more accurate results. In this research, the efficiency of binary classifiers for software defect prediction is analyzed through parameter optimization using random search technique. Three heterogeneous binary classifiers i.e., Decision tree, Support vector machine, and Naïve Bayes are selected to examine the results of parameter optimization. The experiments were performed on seven publicly available NASA Datasets. The dataset was split into 70-30 proportions with class preservation. To evaluate the performance; five statistical measures have been implemented i.e., precision, recall, F-Measure, the area under the curve (AUC), and accuracy. The findings of the research revealed that there is significant improvement in accuracy for each classifier. On average, decision tree improved from 88.1% to 95.4%; support vector machine enhanced the accuracy from 94.3% to 99.9%. While Naïve Bayes showed an accuracy boost from 74.9% to 85.3%. This research contributes to the field of machine learning by presenting comparative analysis of accuracy improvements using default parameters and optimized parameters through random search. The results presented that he performance of binary classifiers in the context of software prediction can be enhanced to a great extent by employing parameter optimization using random search.