We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
The (black) art of runtime evaluation: Are we comparing algorithms or implementations?
- Authors
Kriegel, Hans-Peter; Schubert, Erich; Zimek, Arthur
- Abstract
Any paper proposing a new algorithm should come with an evaluation of efficiency and scalability (particularly when we are designing methods for 'big data'). However, there are several (more or less serious) pitfalls in such evaluations. We would like to point the attention of the community to these pitfalls. We substantiate our points with extensive experiments, using clustering and outlier detection methods with and without index acceleration. We discuss what we can learn from evaluations, whether experiments are properly designed, and what kind of conclusions we should avoid. We close with some general recommendations but maintain that the design of fair and conclusive experiments will always remain a challenge for researchers and an integral part of the scientific endeavor.
- Subjects
ALGORITHMS; METHODOLOGY; LITERATURE; DATA mining; C++
- Publication
Knowledge & Information Systems, 2017, Vol 52, Issue 2, p341
- ISSN
0219-1377
- Publication type
Article
- DOI
10.1007/s10115-016-1004-2