Metalearning Approaches for Algorithm Selection II

Cognitive technologies, pp. 77–102

Abstract

Summary This chapter discusses different types of metalearning models, including regression, classification and relative performance models. Regression models use a suitable regression algorithm, which is trained on the metadata and used to predict the performance of given base-level algorithms. The predictions can in turn be used to order the base-level algorithms and hence identify the best one. These models also play an important role in the search for the potentially best hyperparameter configuration discussed in the next chapter. Classification models identify which base-level algorithms are applicable or non-applicable to the target classification task. Probabilistic classifiers can be used to construct a ranking of potentially useful alternatives. Relative performance models exploit information regarding the relative performance of base-level models, which can be either in the form of rankings or pairwise comparisons. This chapter discusses various methods that use this information in the search for the potentially best algorithm for the target task.

Authors 4

1. Pavel Brazdil corresponding

   Universidade do Porto

   Affiliation as printed

   Laboratory of Artificial Intelligence and Decision Support, University of Porto, Porto, Portugal

2. Jan N. van Rijn Aachen Leiden Institute of Advanced Computer Science

   Leiden University

   Affiliation as printed

   Leiden Institute of Advanced Computer Science, Leiden University, Leiden, The Netherlands

3. Carlos Soares

   Universidade do Porto

   Affiliation as printed

   Porto Business School, Porto, Portugal

4. Joaquin Vanschoren

   Eindhoven University of Technology

   Affiliation as printed

   Department of Mathematics and Computer Science, Technische Universiteit Eindhoven, Eindhoven, The Netherlands