Automatic selection of classification learning algorithms for data mining practitioners

Abstract

Classification algorithm selection is an open research problem whose solution has tremendous value for practitioners. In recent years, metalearning has emerged as a viable approach. Unfortunately, the ratio of examples to classes is small at the metalevel for any reasonable number of algorithms to choose from, and there are serious risks of overfitting due to underlying similarities among algorithms. To alleviate these problems, we propose to 1) cluster algorithms based on behavior similarity, and 2) redefine the metalearning task as mapping classification tasks to clusters of behaviorally-similar algorithms. Experiments with a wide range of classification tasks and algorithms demonstrate that the clustering-based selection model yields better results than typical selection models.