Abstract
This chapter discusses some approaches that exploit metalearning methods in ensemble learning. It starts by presenting a set of issues, such as the ensemble method used, which affect the process of ensemble learning and the resulting ensemble. In this chapter we discuss various lines of research that were followed. Some approaches seek an ensemble-based solution for the whole dataset, others for individual instances. Regarding the first group, we focus on metalearning in the construction, pruning and integration phase. Modeling the interdependence of models plays an important part in this process. In the second group, the dynamic selection of models is carried out for each instance. A separate section is dedicated to hierarchical ensembles and some methods used in their design. As this area involves potentially very large configuration spaces, recourse to advanced methods, including metalearning, is advantageous. It can be exploited to define the competence regions of different models and the dependencies between them.
Chapter PDF
References
Britto, A. S., Sabourin, R., and Oliveira, L. E. (2014). Dynamic selection of classifiers—A comprehensive review. Pattern Recognition, 47(11):3665–3680.
Cachada, M., Abdulrahman, S., and Brazdil, P. (2017). Combining feature and algorithm hyperparameter selection using some metalearning methods. In Proc. of Workshop AutoML 2017, CEUR Proceedings Vol-1998, pages 75–87.
Cruz, R. M., Sabourin, R., and Cavalcanti, G. D. (2018). Dynamic classifier selection: Recent advances and perspectives. Information Fusion, 41:195–216.
Dietterich, T. G. (2000). Ensemble methods in machine learning. In Multiple Classifier Systems. MCS 2000, volume 1857 of LNCS. Springer.
Feurer, M., Klein, A., Eggensperger, K., Springenberg, J., Blum, M., and Hutter, F. (2015a). Efficient and robust automated machine learning. In Cortes, C., Lawrence, N., Lee, D., Sugiyama, M., and Garnett, R., editors, Advances in Neural Information Processing Systems 28, NIPS’15, pages 2962–2970. Curran Associates, Inc.
Feurer, M., Klein, A., Eggensperger, K., Springenberg, J. T., Blum, M., and Hutter, F. (2019). Auto-sklearn: Efficient and robust automated machine learning. In Hutter, F., Kotthoff, L., and Vanschoren, J., editors, Automated Machine Learning: Methods, Systems, Challenges, pages 113–134. Springer.
Feurer, M., Springenberg, J., and Hutter, F. (2015b). Initializing Bayesian hyperparameter optimization via meta-learning. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, pages 1128–1135.
Houeland, T. G. and Aamodt, A. (2018). A learning system based on lazy metareasoning. Progress in Artificial Intelligence, 7(2):129–146.
Khiari, J., Moreira-Matias, L., Shaker, A., Ženko, B., and Džeroski, S. (2019). MetaBags: Bagged meta-decision trees for regression. In Proceedings of ECML/PKDD 2018, pages 637–652. Springer.
KordÃk, P., ÄŒerný, J., and Frýda, T. (2018). Discovering predictive ensembles for transfer learning and meta-learning. Machine Learning, 107(1):177–207.
Kuncheva, L. I. and Whitaker, C. J. (2003). Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy. Machine Learning, 51(2):181–207.
Mendes-Moreira, J., Soares, C., Jorge, A. M., and Sousa, J. F. D. (2012). Ensemble approaches for regression. ACM Computing Surveys, 45(1):1–40.
Narassiguin, A., Elghazel, H., and Aussem, A. (2017). Dynamic Ensemble Selection with Probabilistic Classifier Chains. In Machine Learning and Knowledge Discovery in Databases, European Conference, ECML PKDD 2017, Lecture Notes in Computer Science, volume 10534 LNAI, pages 169–186. Springer, Cham.
Peterson, A. H. and Martinez, T. (2005). Estimating the potential for combining learning models. In Proc. of the ICML Workshop on Meta-Learning, pages 68–75.
Pinto, F., Soares, C., and Mendes-Moreira, J. (2014). An empirical methodology to analyse the behavior of bagging. In Advanced Data Mining and Applications. ADMA 2014. Lecture Notes in Computer Science, volume 8933. Springer, Cham.
Pinto, F., Soares, C., and Mendes-Moreira, J. (2015). Pruning bagging ensembles with metalearning. In International Conference on Advanced Data Mining and Applications, Lecture Notes in Computer Science, volume 9132, pages 64–75. Springer, Cham.
Pinto, F., Soares, C., and Mendes-Moreira, J. (2016). CHADE: Metalearning with classifier chains for dynamic combination of classifiers. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2016, Lecture Notes in Computer Science, volume 9851 LNAI, pages 410–425. Springer, Cham.
Read, J., Pfahringer, B., Holmes, G., and Frank, E. (2019). Classifier chains: A review and perspectives. arXiv preprint arXiv:1912.13405.
Todorovski, L. and Džeroski, S. (2000). Combining multiple models with meta decision trees. In Zighed, D. A., Komorowski, J., and Zytkow, J., editors, Proc. of the Fourth European Conf. on Principles and Practice of Knowledge Discovery in Databases, pages 255–264. Springer-Verlag.
Waegeman, W., Dembczýnski, K., and Hüllermeier, E. (2019). Multi-target prediction: a unifying view on problems and methods. Data Mining and Knowledge Discovery, 33(2):293–324.
Wistuba, M., Schilling, N., and Schmidt-Thieme, L. (2017). Automatic Frankensteining: Creating complex ensembles autonomously. In Proceedings of the 2017 SIAM International Conference on Data Mining, pages 741–749. Society for Industrial and Applied Mathematics, Philadelphia, PA.
Wolpert, D. (1996). The lack of a priori distinctions between learning algorithms. Neural Computation, 8:1341–1390.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2022 The Author(s)
About this chapter
Cite this chapter
Brazdil, P., van Rijn, J.N., Soares, C., Vanschoren, J. (2022). Metalearning in Ensemble Methods. In: Metalearning. Cognitive Technologies. Springer, Cham. https://doi.org/10.1007/978-3-030-67024-5_10
Download citation
DOI: https://doi.org/10.1007/978-3-030-67024-5_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-67023-8
Online ISBN: 978-3-030-67024-5
eBook Packages: Computer ScienceComputer Science (R0)