Benchmark set reduction for cheap empirical algorithmic studies
Yükleniyor...
Dosyalar
Tarih
2021
Yazarlar
Dergi Başlığı
Dergi ISSN
Cilt Başlığı
Yayıncı
IEEE
Erişim Hakkı
info:eu-repo/semantics/closedAccess
Özet
The present paper introduces a benchmark set reduction strategy that can degrade the experimental evaluation cost for the algorithmic studies. Algorithm design is an iterative development process within in a test-revise loop. Starting a devised algorithm's initial version, it needs to be tested for revealing its advantages and drawbacks. Its shortcomings lead the designers to modify the algorithm. Then, the modified algorithm is again tested. This two-step pipeline is repeated until the algorithm meets the expectations such as delivering the state-of-the-art results on a set of benchmarks for a specific problem. That recurring testing step can be a burden for whom with limited computational resources. The mentioned computational cost can mainly occur due either high per instance runtime cost or having a large benchmark set. This study focuses on the cases when a target algorithm needs to be assessed across large benchmark sets. The idea is to automatically extract problem instance representation through an instance-algorithm performance data. The derived representation in the form of latent features is utilized to determine a small yet a representative subset of a given large instance set. The proposed strategy is investigated on the Traveling Thief Problem of 9720 instances. The corresponding performance data is collected by the help of 21 TTP algorithms. The resulting computational analysis showed that the proposed method is capable of substantially minimizing the benchmark instance set size.
Açıklama
Anahtar Kelimeler
Benchmark Set Reduction, Empirical Analysis, Traveling Thief Problem
Kaynak
2021 IEEE CONGRESS ON EVOLUTIONARY COMPUTATION (CEC 2021)
WoS Q Değeri
N/A
Scopus Q Değeri
N/A
Cilt
Sayı
Künye
Mısır, M. (2021, June). Benchmark Set Reduction for Cheap Empirical Algorithmic Studies. In 2021 IEEE Congress on Evolutionary Computation (CEC) (pp. 871-877). IEEE.
