Avoid common mistakes on your manuscript.
Taking inspiration from natural behaviors to devise new optimization algorithms has played an important role in the history of the field of metaheuristics (Sörensen et al. 2017). Unfortunately, in the last two decades we have been witnessing a new trend by which dozens of metaphor-based metaheuristics based on the most diverse possible set of natural, artificial, social, and sometimes even supernatural phenomena and behaviors are proposed, without a clear motivation beyond the desire of their authors to publish their papers. Despite several attempts (Sörensen 2015; Sörensen et al. 2019; Weyland 2010; Piotrowski et al. 2014; Fong et al. 2016; Camacho Villalón et al. 2019; Camacho Villalón et al. 2020; Swan et al. 2015; Lones 2020; Tzanetos and Dounias 2020) to put an end to the flood of increasingly outlandish, “novel” metaphor-centered metaheuristics, there is still a steady inflow of such papers submitted, and often accepted for publication. For many reasons, we believe this is detrimental to the field of metaheuristics. This letter describes some of the negative effects of publishing such papers in the literature and proposes a necessary action to try to put a limit to this highly undesirable phenomenon.
Useless metaphors The usefulness of the metaphors that inspired the myriad of metaheuristics already published somewhere in the literature is arguably the most questionable aspect of these methods. As demonstrated by the long list of “novel” algorithms gathered by Campelo and Aranha (2021), inventing a metaheuristic that loosely mimics a real-world process is a trivial exercise that does not in itself justify inclusion in the scientific body of literature. Also, it is often the case that the mathematical models derived from the metaphors are modified or omitted in the implementation of the metaheuristics because they result in poorly performing implementations (Melvin et al. 2012; Piotrowski et al. 2014). Indeed, it is not only the fact that using these new metaphors lacks any sound, scientific motivation; but also that, in most cases, the metaphors themselves are not well represented by the resulting computational algorithm (i.e., the metaphor is invariably oversimplified or modified to look like an optimization process even when it is not). Very alarming is the fact that there are numerous examples of metaphor-based metaheuristics in which the metaphor, the mathematical model derived from the metaphor and the implementation of the algorithm are three (almost completely) different things (Camacho Villalón et al. 2020; Camacho Villalón et al. 2021).
Lack of novelty One argument often used to justify introducing a new metaphor-based metaheuristic is that there are novel concepts in the behavior inspiring the algorithm that can be used to solve optimization problems. Unfortunately, there is plenty of evidence that this is rarely the case. In fact, it has become increasingly common to discover, sometimes a few years after publication, that the exact same concepts proposed in the “novel” metaheuristic were proposed before in previously published works (Weyland 2010; Simon et al. 2011; Piotrowski et al. 2014; Camacho Villalón et al. 2019; Camacho Villalón et al. 2020; Tzanetos and Dounias 2020; Camacho Villalón et al. 2021). However, what is invariably new is the non-standard terminology used to present the “novel” metaheuristics. Publishing papers that do not propose anything more than a new terminology to refer to already well-known concepts has pernicious effects, such as (i) creating confusion in the literature, (ii) hindering our understanding of the existing metaphor-based metaheuristics, and (iii) making extremely difficult to compare metaheuristics both theoretically and experimentally.
Poor experimental validation and comparison Biased computational experimentation, the so-called apples to oranges comparison, often comparing “novel” metaphor-based metaheuristics run on recent computers against methods run on older computers, and/or limited experimentation often show a false picture of the performance of a “novel” method (Sörensen 2015; Weyland 2010). In addition to this, rather than testing the “novel” metaheuristic against the best performing algorithms for the considered problem, the comparison is often made with other versions of the very same metaheuristic that is being evaluated or with old algorithms whose performance is far from the state-of-the-art (García-Martínez et al. 2017).
It is important to stress that, even in those cases in which a metaphor-based metaheuristic can be shown to have a very good performance on some optimization problems, expressing an optimization method in the non-standard, metaphor-based vocabulary only serves to obfuscate its functioning, making it close to impossible to position the “novel” method in the literature and to discern how it relates to existing methods.
Metaphor-based optimization in application-oriented journals Because of the measures that the field of optimization has been increasingly taking against papers whose only contribution to the literature is a new metaphor upon which an otherwise trivial optimization algorithm is based (see below), authors of such papers are resorting to different outlets, such as journals that publish papers on the topic of the metaphor. For instance, if the metaphor is the mating behaviour of bats, authors will attempt to publish it in specialised journals on bats, or on animal mating behaviour; metaheuristics inspired on improvising musicians will go for journals on music. If the metaphor is the spread of a virus, they will turn to journals on biomedical informatics. The additional advantage of submitting a manuscript to what can only be considered an off-topic journal is that editors and reviewers might not be sufficiently knowledgeable in the specific field of optimization to evaluate the true merits of the paper. Fortunately, this strategy usually does not work and the editors and reviewers quickly realize that the paper is out of scope for their journal. In some cases, regrettably, a paper manages to slip through the net of peer review and gets published. This may happen, e.g., when reviewers are selected from the authors’ bubble (based on the list of references or, in some cases, on recommendations of the authors themselves); or when reviewers become blindsided by the metaphor and fail to recognize that the manuscript lacks any meaningful contribution to the state-of-the-art.
The appearance of papers proposing algorithms inspired by often ludicrous processes that do not—by any stretch of the imagination—optimize anything indicates poor scientific housekeeping and, consequently, radiates badly on the research community on metaheuristics. We strongly feel that it is in the best interest of this community that scientific standards in the field are improved, and that papers whose only claim to scientific contribution is having found a new source of inspiration are no longer published. For this reason, we call upon all editors-in-chief in the field to adapt their editorial policies—like several journals such as the Journal of Heuristics (Journal of Heuristics 2015), Swarm Intelligence (Dorigo 2016), and the ACM Transactions on Evolutionary Learning and Optimization (ACM 2021) have already done—and add a statement to the following effect to their submission guidelines:
This journal will not publish papers that propose “novel” metaphor-based metaheuristics, unless the authors (i) present their method using the normal, standard optimization terminology; (ii) show that the new method brings useful and novel concepts to the field; (iii) motivate the use of the metaphor on a sound, scientific basis; and (iv) present a fair comparison with other state-of-the-art methods using state-of-the-art practices for benchmarking algorithms.
Signed by (in alphabetical order; first authors of this letter are indicated by an *):
Claus Aranha*, University of Tsukuba, Japan
Claudia Archetti, ESSEC Business School, France
Anne Auger, Institut National de Recherche en Informatique et en Automatique, France
Thomas Bäck, Universiteit Leiden, The Netherlands
Prasanna Balaprakash, Argonne National Laboratory, United States
Thomas Bartz-Beielstein, Technische Hochschule Köln, Germany
Lucas Batista, Universidade Federal de Minas Gerais, Brazil
Kris Braekers, Universiteit Hasselt, Belgium
Juergen Branke, University of Warwick, United Kingdom
Alexander E. I. Brownlee, University of Stirling, United Kingdom
Christian L. Camacho Villalón*, Université Libre de Bruxelles, Belgium
Felipe Campelo*, Aston University, United Kingdom
Eduardo Carrano, Universidade Federal de Minas Gerais, Brazil
Daniele Catanzaro, Université Catholique de Louvain, Belgium
Marco Chiarandini, Syddansk Universitet, Denmark
Francisco Chicano, Universidad de Málaga, Spain
Carlos A. Coello Coello, CINVESTAV-IPN, Mexico
Ángel Corberán, Universitat de València, Spain
Oscar Cordón, Universidad de Granada, Spain
Yves Crama, Université de Liège, Belgium
Patrick De Causmaecker, Katholieke Universiteit Leuven, Belgium
Javier Del Ser, Universidad del País Vasco, Spain
Gianni Di Caro, Carnegie Mellon University in Qatar, Qatar
Benjamin Doerr, École Polytechnique, France
Carola Doerr, Sorbonne Université, France
Marco Dorigo*, Université Libre de Bruxelles, Belgium
Yu Du, University of Colorado Denver, United States
Anikó Ekárt, Aston University, United Kingdom
Andries Engelbrecht, Stellenbosch University, South Africa
Carlos M. Fonseca, Universidade de Coimbra, Portugal
Alberto Franzin, Université Libre de Bruxelles, Belgium
Margaretha Gansterer, Universität Klagenfurt, Austria
Pablo García Sánchez, Universidad de Granada, Spain
Fred Glover, Meta-Analytics Inc., United States
Maria Gomes, NOVA School of Science and Technology, Portugal
Nikolaus Hansen, INRIA, France
Jin-Kao Hao, Université d’Angers, France
Richard F. Hartl, Universität Wien, Austria
Alfredo G. Hernández-Díaz, Universidad Pablo de Olavide, Spain
Francisco Herrera, Universidad de Granada, Spain
Alain Hertz, Polytechnique Montréal, Canada
Holger Hoos, Universiteit Leiden, The Netherlands
Colin G. Johnson, University of Nottingham, United Kingdom
Ahmed Kheiri, Lancaster University, United Kingdom
Gary A. Kochenberger, Meta-Analytics Inc., United States
Manuel Laguna, University of Colorado Boulder, United States
Adam Letchford, Lancaster University, United Kingdom
Mark Lewis, Missouri Western State University, United States
Xiaodong Li, The University of Melbourne, Australia
Andrea Lodi, Polytechnique Montréal, Canada
Michael Lones, Heriot-Watt University, United Kingdom
Manuel López-Ibáñez, Universidad de Málaga, Spain
Jose A. Lozano, Universidad del País Vasco, Spain
Vittorio Maniezzo, Università di Bologna, Italy
Silvano Martello, Università di Bologna, Italy
Rafael Martí, Universitat de València, Spain
Pedro Martins, Coimbra Business School, Portugal
John McCall, Robert Gordon University, United Kingdom
Belen Melian, Universidad de La Laguna, Spain
Zbigniew Michalewicz, The University of Adelaide, Australia
Gerardo Minella, Universitat Politecnica de València, Spain
Nenad Mladenovic, Khalifa University United, Arab Emirates
José A. Moreno Pérez, Universidad de La Laguna, Spain
Aneta Neumann, The University of Adelaide, Australia
Frank Neumann, The University of Adelaide, Australia
Thiago Noronha, Universidade Federal de Minas Gerais, Brazil
Gabriela Ochoa, University of Stirling, United Kingdom
Ender Özcan, University of Nottingham, United Kingdom
Federico Pagnozzi, Université Libre de Bruxelles, Belgium
Gisele Lobo Pappa, Universidade Federal de Minas Gerais, Brazil
Luís Paquete, Universidade de Coimbra, Portugal
Eduardo G. Pardo, Universidad Rey Juan Carlos, Spain
Michael Patriksson, Chalmers Tekniska Högskola, Sweden
Federico Perea, Universidad de Sevilla, Spain
Leslie Pérez Cáceres, Pontificia Universidad Católica de Valparaíso, Chile
Riccardo Poli, University of Essex, United Kingdom
Helena Ramalhinho, Universitat Pompeu Fabra, Spain
Andrea Roli, Università di Bologna, Italy
Rubén Ruiz*, Universitat Politècnica de València, Spain
Marc Schoenauer, INRIA, France
Marc Sevaux*, Université Bretagne Sud, France
Christine Solnon, Institut National des Sciences Appliquées de Lyon, France
Michael M. Sørensen, Aarhus Universitet, Denmark
Kenneth Sörensen*, Universiteit Antwerpen, Belgium
Thomas Stützle*, Université Libre de Bruxelles, Belgium
Jerry Swan, NNAISENSE S.A., Switzerland
Eva Vallada, Universitat Politècnica de València, Spain
Greet Vanden Berghe, Katholieke Universiteit Leuven, Belgium
Pieter Vansteenwegen, Katholieke Universiteit Leuven, Belgium
Markus Wagner, The University of Adelaide, Australia
Elizabeth Wanner, Centro Federal de Educação Tecnológica de Minas Gerais, Brazil
Darrell Whitley, Colorado State University, United States
Marino Widmer, Université de Fribourg, Switzerland
John Woodward, Queen Mary University of London, United Kingdom
Rui Zhang, University of Colorado Boulder, United States
References
ACM Transactions on Evolutionary Learning and Optimization. Guidelines for Authors. https://dl.acm.org/journal/telo/author-guidelines (2021). Version visited last on March 26, 2021
Camacho Villalón, C.L., Dorigo, M., & Stützle, T. (2018). Why the intelligent water drops cannot be considered as a novel algorithm. In: M. Dorigo, M. Birattari, C. Blum, A.L. Christensen, A. Reina, V. Trianni (eds.) Swarm Intelligence, 11th International Conference, ANTS 2018, Lecture Notes in Computer Science, vol. 11172, (pp. 302–314). Springer.
Camacho Villalón, C. L., Dorigo, M., & Stützle, T. (2019). The intelligent water drops algorithm: why it cannot be considered a novel algorithm. Swarm Intelligence, 13(3–4), 173–192.
Camacho Villalón, C. L., Stützle, T., & Dorigo, M. (2020). Grey wolf, firefly and bat algorithms: Three widespread algorithms that do not contain any novelty. In: International Conference on Swarm Intelligence, (pp. 121–133). Springer (2020)
Camacho Villalón, C. L., Stützle, T., & Dorigo, M (2021). Cuckoo search \(\equiv \) (\(\mu +\lambda \))–evolution strategy — A rigorous analysis of an algorithm that has been misleading the research community for more than 10 years and nobody seems to have noticed. Technical Report TR/IRIDIA/2021-006, IRIDIA, Université Libre de Bruxelles, Belgium.
Campelo, F., & Aranha, C. (2021). Evolutionary computation bestiary. https://github.com/fcampelo/EC-Bestiary (2021). Version visited last on 26 March.
Dorigo, M. (2016). Swarm intelligence: A few things you need to know if you want to publish in this journal. https://www.springer.com/cda/content/document/cda_downloaddocument/Additional_submission_instructions.pdf (2016). Uploaded in November 2016
Fong, S., Wang, X., Xu, Q., Wong, R., Fiaidhi, J., & Mohammed, S. (2016). Recent advances in metaheuristic algorithms: Does the makara dragon exist? The Journal of Supercomputing, 72(10), 3764–3786.
García-Martínez, C., Gutiérrez, P. D., Molina, D., Lozano, M., & Herrera, F. (2017). Since CEC 2005 competition on real-parameter optimisation: a decade of research, progress and comparative analysis weakness. Soft Computing, 21(19), 5573–5583.
Journal of Heuristics. Policies on Heuristic Search Research. https://www.springer.com/journal/10732/updates/17199246 (2015). Version visited last on March 26, 2021.
Melvin, G., Dodd, T. J., & Groß, R. (2012). Why GSA: a gravitational search algorithm is not genuinely based on the law of gravity. Natural Computing, 11(4), 719–720.
Lones, M. A. (2020). Mitigating metaphors: A comprehensible guide to recent nature-inspired algorithms. SN Computer Science, 1(1), 1–12.
Piotrowski, A. P., Napiorkowski, J. J., & Rowinski, P. M. (2014). How novel is the novel black hole optimization approach? Information Sciences, 267, 191–200.
Simon, D., Rarick, R., Ergezer, M., & Du, D. (2011). Analytical and numerical comparisons of biogeography-based optimization and genetic algorithms. Information Sciences, 181(7), 1224–1248.
Sörensen, K. (2015). Metaheuristics–the metaphor exposed. International Transactions in Operational Research, 22(1), 3–18.
Sörensen, K. Sevaux, M., & Glover, F. (2017). A history of metaheuristics. arXiv preprint arXiv:1704.00853.
Sörensen, K., Arnold, F., & Palhazi Cuervo, D. (2019). A critical analysis of the improved Clarke and Wright savings algorithm. International Transactions in Operational Research, 26(1), 54–63.
Swan, J., Adriaensen, S., Bishr, M., & Burke, et al. (2015). A research agenda for metaheuristic standardization. In: Proceedings of the XI Metaheuristics International Conference, pp. 1-3.
Tzanetos, A., & Dounias, G. (2020). Nature inspired optimization algorithms or simply variations of metaheuristics? Artificial Intelligence Review,1–22,
Weyland, D. (2010). A rigorous analysis of the harmony search algorithm: How the research community can be misled by a novel methodology. International Journal of Applied Metaheuristic Computing, 12(2), 50–60.
Weyland, D. (2015). A critical analysis of the harmony search algorithm: How not to solve Sudoku. Operations Research Perspectives, 2, 97–105.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
All authors contributed equally to this letter.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Aranha, C., Camacho Villalón, C.L., Campelo, F. et al. Metaphor-based metaheuristics, a call for action: the elephant in the room. Swarm Intell 16, 1–6 (2022). https://doi.org/10.1007/s11721-021-00202-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11721-021-00202-9