Overfitting in Machine Learning: A Comparative Analysis of Decision Trees and Random Forests.

Halabaku, Erblin; Bytyçi, Eliot

Machine learning has emerged as a pivotal tool in deciphering and managing this excess of information in an era of abundant data. This paper presents a comprehensive analysis of machine learning algorithms, focusing on the structure and efficacy of random forests in mitigating overfitting—a prevalent issue in decision tree models. It also introduces a novel approach to enhancing decision tree performance through an optimized pruning method called Adaptive Cross-Validated Alpha CCP (ACV-CCP). This method refines traditional cost complexity pruning by streamlining the selection of the alpha parameter, leveraging cross-validation within the pruning process to achieve a reliable, computationally efficient alpha selection that generalizes well to unseen data. By enhancing computational efficiency and balancing model complexity, ACV-CCP allows decision trees to maintain predictive accuracy while minimizing overfitting, effectively narrowing the performance gap between decision trees and random forests. Our findings illustrate how ACV-CCP contributes to the robustness and applicability of decision trees, providing a valuable perspective on achieving computationally efficient and generalized machine learning models.

RANDOM forest algorithms; DECISION trees; DECISION making; ARTIFICIAL intelligence; TREE pruning

Intelligent Automation & Soft Computing, 2024, Vol 39, Issue 6, p987

1079-8587

Academic Journal

10.32604/iasc.2024.059429