Performance Accuracy and Matthew Correlation Coefficient
The models were analyzed using the SK learn library after dropping the features with p-values greater than 0.05. Table 1 presents the performance accuracy and the MCC results for the different models tested. Compared to the gradient descent and the ensemble classifier, the benchmark logistic regression model did not fear much. The ensemble classifier, a combination of multiple models, showed the best performance in terms of accuracy and MCC. The superior performance of the ensemble classifiers is because they can capture different patterns in the data and weigh them appropriately, resulting in more accurate predictions 41,56. In terms of computational cost, gradient descent was the most expensive model to run, while logistic regression was the least expensive 34. Therefore, if computational cost is a concern, logistic regression may be a better option despite its slightly lower accuracy. Overall, all the models performed reasonably well, with ensembles being the best performers in terms of accuracy and MCC.
As shown in Table 6, the MCC had poorer performance overall than the model accuracy. The random forest model had the highest performance accuracy (.89) and MCC (.78) on the test set, although it showed signs of overfitting. The next best performing classifier was the decision tree classifier, with a performance accuracy of .86 and an MCC of .75. The gradient descent classifier had a similar accuracy (.82); but a lower MCC (.67). These results suggest that while the random forest model is more likely to overfit the data, it may still be the best option for this dataset due to its higher accuracy. However, further tuning of hyperparameters may be necessary to improve its performance. These results suggest that, while gradient descent may be a computationally efficient method, it is not as effective as other methods in predictive power. In conclusion, the random forest model is the best-performing model in terms of accuracy and MCC.
Table 6: Algorithms Performance