We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
Beyond subjective judgments: Predicting evaluations of creative writing from computational linguistic features.
- Authors
Zedelius, Claire M.; Mills, Caitlin; Schooler, Jonathan W.
- Abstract
The question of how to evaluate creativity in the context of creative writing has been a subject of ongoing discussion. A key question is whether something as elusive as creativity can be evaluated in a systematic way that goes beyond subjective judgments. To answer this question, we tested whether human evaluations of the creativity of short stories can be predicted by: (1) established measures of creativity and (2) computerized linguistic analyses of the stories. We conducted two studies, in which college students (with and without interest and experience in creative writing) wrote short stories based on a writing prompt. Independent raters (six in Study 1, five in Study 2) assessed the stories using an evaluation rubric specifically designed to assess aspects of creativity, on which they showed high interrater reliability. We provide evidence of convergent validity, in that the rubric evaluations correlated with established creativity measures, including measures of divergent thinking, associative fluency, and self-reported creative behavior and achievements. Linguistic properties of the short stories were analyzed with two computerized text analysis tools: Coh-Metrix, which analyzes aspects of text cohesion and readability, and Linguistic Inquiry and Word Count, which identifies meaningful psychological categories of the text content. Linguistic features predicted the human ratings of creativity to a significant degree. These results provide novel evidence that creative writing can be evaluated reliably and in a systematic way that captures objective features of the text. The results further establish our evaluation rubric as a useful tool to assess creative writing.
- Subjects
CREATIVE writing; READABILITY (Literary style); LINGUISTIC analysis; SHORT story writing; DIVERGENT thinking; WORD frequency; INTER-observer reliability
- Publication
Behavior Research Methods, 2019, Vol 51, Issue 2, p879
- ISSN
1554-351X
- Publication type
Article
- DOI
10.3758/s13428-018-1137-1