We found a match
Your institution may have rights to this item. Sign in to continue.
- Title
A simple computational algorithm of model-based choice preference.
- Authors
Toyama, Asako; Katahira, Kentaro; Ohira, Hideki
- Abstract
A broadly used computational framework posits that two learning systems operate in parallel during the learning of choice preferences-namely, the model- free and model- based reinforcement-learning systems. In this study, we examined another possibility, through which model-free learning is the basic system and model-based information is its modulator. Accordingly, we proposed several modified versions of a temporal-difference learning model to explain the choice-learning process. Using the two-stage decision task developed by Daw, Gershman, Seymour, Dayan, and Dolan (2011), we compared their original computational model, which assumes a parallel learning process, and our proposed models, which assume a sequential learning process. Choice data from 23 participants showed a better fit with the proposed models. More specifically, the proposed eligibility adjustment model, which assumes that the environmental model can weight the degree of the eligibility trace, can explain choices better under both model-free and model-based controls and has a simpler computational algorithm than the original model. In addition, the forgetting learning model and its variation, which assume changes in the values of unchosen actions, substantially improved the fits to the data. Overall, we show that a hybrid computational model best fits the data. The parameters used in this model succeed in capturing individual tendencies with respect to both model use in learning and exploration behavior. This computational model provides novel insights into learning with interacting model-free and model-based components.
- Subjects
INSTRUCTIONAL systems; COMPUTATIONAL acoustics; HYBRID integrated circuits; LEARNING; SOCIAL influence
- Publication
Cognitive, Affective & Behavioral Neuroscience, 2017, Vol 17, Issue 4, p764
- ISSN
1530-7026
- Publication type
Article
- DOI
10.3758/s13415-017-0511-2