We found a match
Your institution may have rights to this item. Sign in to continue.
- Title
An Integrated Neural Decoder of Linguistic and Experiential Meaning.
- Authors
Anderson, Andrew James; Binder, Jeffrey R.; Fernandino, Leonardo; Humphries, Colin J.; Conant, Lisa L.; Raizada, Rajeev D. S.; Feng Lin; Lalor, Edmund C.
- Abstract
The brain is thought to combine linguistic knowledge of words and nonlinguistic knowledge of their referents to encode sentence meaning. However, functional neuroimaging studies aiming at decoding language meaning from neural activity have mostly relied on distributional models of word semantics, which are based on patterns of word co-occurrence in text corpora. Here, we present initial evidence that modeling nonlinguistic "experiential" knowledge contributes to decoding neural representations of sentence meaning. We model attributes of peoples' sensory, motor, social, emotional, and cognitive experiences with words using behavioral ratings. We demonstrate that fMRI activation elicited in sentence reading is more accurately decoded when this experiential attribute model is integrated with a text-based model than when either model is applied in isolation (participants were 5 males and 9 females). Our decoding approach exploits a representation-similarity-based framework, which benefits from being parameter free, while performing at accuracy levels comparable with those from parameter fitting approaches, such as ridge regression. We find that the text-based model contributes particularly to the decoding of sentences containing linguistically oriented "abstract" words and reveal tentative evidence that the experiential model improves decoding of more concrete sentences. Finally, we introduce a cross-participant decoding method to estimate an upper bound on model-based decoding accuracy. We demonstrate that a substantial fraction of neural signal remains unexplained, and leverage this gap to pinpoint characteristics of weakly decoded sentences and hence identify model weaknesses to guide future model development.
- Subjects
WORD frequency; SEMANTICS; CORPORA; CRIMINAL sentencing
- Publication
Journal of Neuroscience, 2019, Vol 39, Issue 45, p8969
- ISSN
0270-6474
- Publication type
Article
- DOI
10.1523/JNEUROSCI.2575-18.2019