GECCO 2021 logo

Workshop on Understanding Reproducibility in Evolutionary Computation (Benchmarking@GECCO2021)

July 10th-14th 2021

Call for Papers

We have the pleasure of announcing the 1st Workshop on Understanding Reproducibility in Evolutionary Computation co-organised with Good Benchmarking Practices for Evolutionary Computation (Benchmarking@GECCO-2021), to be held online as part of GECCO 2021.

Experimental studies are prevalent in Evolutionary Computation (EC), and concerns about the reproducibility and replicability of such studies has increased in recent years, following similar discussions in other scientific fields. In this workshop, we want to raise awareness of the reproducibility issue, shed light on the obstacles when trying to reproduce results, and discuss best practices in making results reproducible as well as reporting reproducibility results.

We invite submissions of papers repeating an empirical study published in a journal or conference proceedings, either by re-using, updating or reimplementing the necessary codes and datasets, irrespectively of whether this code was published in some form at the time or not.

We expect in the submitted paper:

In the end, there may be various possible outcomes, and all are acceptable for a paper: you are unable to run or compile the code, you are able to run the code but it does not give you the expected results (or no result at all), the program crashed regularly (before getting results), you do not remember the parameter settings used, etc. All these are valid conclusions. We care more about the description of the process, challenges to reproduce results, and the lessons to be learned, than about whether you have actually been able to reproduce the study.

Submission Instructions

In addition to the instructions above, authors should refer to the GECCO Submission Instructions.

Please note that GECCO 2021 will be held as an electronic-only conference. All accepted papers will be required to be presented in the form of a pre-recorded talk. More details about this will be provided soon.

Important Dates