TACO: Toolkit for Automatic Comparison of Optimizers TACO Comparison Toolkit TACO Compare algorithms

TACO: Toolkit for Automatic Comparison of Optimizers

TACO is a new website to allow researchers to compare easily the results of different algorithms between them. It is a very common and consuming time, and this platform online (without require any installation of software) allow to make comparisons.

As already mentioned, one of the core characteristics of TACO is its modularity and flexibility. One of the best examples of this modular design is the reporting system. As mentioned before, each benchmark included in the database has a variable number of associated reports. These reports can belong to two different categories:

  • General purpose reports. Some analyses are independent of the particular benchmark under consideration. Examples of these reports are summary tables with common statistics (minimum, maximum, mean, median, standard deviation), convergence plots (fitness vs. fitness evaluations) or non-parametric statistical tests. All these reports are generic and can be applied to any selection of algorithms.
  • Benchmark/Special Session specific reports. On the other hand, some benchmarks may require of special reports that conduct analyses too specific for the problems of that particular set of problems that would not be applicable to problems of a different domain. For example, if the benchmark contains some real-world problems (from the scientific or the engineering domain) specific analyses paying attention to features different from the fitness value itself (physical characteristics of the models adjusted, robustness of the solutions with regards to any additional considerations, factibility of the solutions reported, etc.) might be necessary. Another scenario in which these specific reports are frequent is in the context of a special session. A good example of this (which is actually included in the current version of the application) is the F1 criterion followed by CEC large-scale global optimization sessions. This report sorts algorithms according to the error reported for each function and assigns points to each algorithm according to the F1 criterion (25 points to the first algorithm, 18 to the second, etc.). This report is too specific of a particular community of users and thus it makes little sense to make it available to other benchmarks.

Tables and Export format

The website allow several interesting comparative tables:
  • Mean Table: Table with the mean for each algorithm and function for milestone, remarking for function which algorithm obtains the best results. Then, the number of functions each algorithm is better than the others is counted, and summarize for each table. Finally, an additional table with a summarize is also shown.
  • Ranking Table: For each algorithm and functions, the are sorted the algorithms by its mean. Then, the average ranking is shown, and summarize for each table. Finally, an additional table with a summarize (average ranking for milestone) is also shown.

Because this website is growning in features, if you want any other feature or suggestion (or better, to contribute), please, take contact with authors.

For each table, the best value is remarked with a color blue (also in the Excel format).

Also, these tables can be exported to:

  • Excel format, maintining the remarks that can be shown in the online version.
  • Latex format, to be easily included in your latex papers, with the right precision.

Plots and Figures

Lorem ipsum dolor sit ameng consectetuer adipiscing elit. Donec hendrerit tempor tellus. Donec pretium posuere tellus. Proin quam nisl, tincidunt et, mattis eget, convallis nec, purus. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Nulla posuere. Donec vitae dolor. Nullam tristique diam non turpis. Cras placerat accumsan nulla. Nullam rutrum. Nam vestibulum accumsan nisl.

Private Data

Nullam eu ante vel est convallis dignissim. Fusce suscipit, wisi nec facilisis facilisis, est dui fermentum leo, quis tempor ligula erat quis odio. Nunc porta vulputate tellus. Nunc rutrum turpis sed pede. Sed bibendum. Aliquam posuere. Nunc aliquet, augue nec adipiscing interdum, lacus tellus malesuada massa, quis varius mi purus non odio. Pellentesque condimentum, magna ut suscipit hendrerit, ipsum augue ornare nulla, non luctus diam neque sit amet urna. Curabitur vulputate vestibulum lorem. Fusce sagittis, libero non molestie mollis, magna orci ultrices dolor, at vulputate neque nulla lacinia eros. Sed id ligula quis est convallis tempor. Curabitur lacinia pulvinar nibh. Nam a sapien.

Pellentesque dapibus suscipit ligula. Donec posuere augue in quam. Etiam vel tortor sodales tellus ultricies commodo. Suspendisse potenti. Aenean in sem ac leo mollis blandit. Donec neque quam, dignissim in, mollis nec, sagittis eu, wisi. Phasellus lacus. Etiam laoreet quam sed arcu. Phasellus at dui in ligula mollis ultricies. Integer placerat tristique nisl. Praesent augue. Fusce commodo. Vestibulum convallis, lorem a tempus semper, dui dui euismod elit, vitae placerat urna tortor vitae lacus. Nullam libero mauris, consequat quis, varius et, dictum id, arcu. Mauris mollis tincidunt felis. Aliquam feugiat tellus ut neque. Nulla facilisis, risus a rhoncus fermentum, tellus tellus lacinia purus, et dictum nunc justo sit amet elit.