We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
Evaluating three approaches to binary event-level agreement scoring. A reply to Friedman (2020).
- Authors
Zemblys, Raimondas; Niehorster, Diederick C.; Holmqvist, Kenneth
- Abstract
Third, the imperfections in the data sets used for training and evaluation should logically only lead to reduced performance of the evaluated event classifiers, which would be reflected in lower agreement between the event classifiers' output and the (hand-coded) event labels in the evaluation data sets. Event-level agreement analysis There is one final "error" discussed by Friedman et al. ([2], "Error 3"), which concerns the implementation of our per-event event-related agreement analysis and the logic behind it. The procedure for the overall agreement score was described in the "Novel event-level evaluation" section on page 845 of Zemblys et al., ([21]), but we have discovered that the procedure for per-event (binary) event-level agreement score computation was not described in the methods section. We opted to not use the F1 score for assessing per-event agreement for the same reason of being able to report a per-event event-level agreement score that is comparable to the overall event-level agreement score.
- Subjects
CONVOLUTIONAL neural networks; BEHAVIORAL research
- Publication
Behavior Research Methods, 2021, Vol 53, Issue 1, p325
- ISSN
1554-351X
- Publication type
Article
- DOI
10.3758/s13428-020-01425-0