We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
Efficiently measuring recognition performance with sparse data.
- Authors
Schooler, Lael J.; Shiffrin, Richard M.
- Abstract
We examine methods for measuring performance in signal-detection-like tasks when each participant provides only a few observations. Monte Carlo simulations demonstrate that standard statistical techniques applied to a d′ analysis can lead to large numbers of Type I errors (incorrectly rejecting a hypothesis of no difference). Various statistical methods were compared in terms of their Type I and Type II error (incorrectly accepting a hypothesis of no difference) rates. Our conclusions are the same whether these two types of errors are weighted equally or Type I errors are weighted more heavily. The most promising method is to combine an aggregate d′ measure with a percentile bootstrap confidence interval, a computer-intensive nonparametric method of statistical inference. Researchers who prefer statistical techniques more commonly used in psychology, such as a repeated measures t test, should use γ (Goodman & Kruskal, 1954), since it performs slightly better than or nearly as well as d′. In general, when repeated measures t tests are used, γ is more conservative than d′: It makes more Type II errors, but its Type I error rate tends to be much closer to that of the traditional .05 α level. It is somewhat surprising that), performs as well as it does, given that the simulations that generated the hypothetical data conformed completely to the d′ model. Analyses in which H -- FA was used had the highest Type I error rates. Detailed simulation results can be downloaded from www.psychonomic.org/archive/Schooler-BRM-2004.zip.
- Subjects
SPARSE matrices; SIGNAL detection; MONTE Carlo method; ERROR; STATISTICAL hypothesis testing; STATISTICAL tolerance regions
- Publication
Behavior Research Methods, 2005, Vol 37, Issue 1, p3
- ISSN
1554-351X
- Publication type
Article
- DOI
10.3758/BF03206393