| Paper: | PS-1A.24 |
| Session: | Poster Session 1A |
| Location: | Symphony/Overture |
| Session Time: | Thursday, September 6, 16:30 - 18:30 |
| Presentation Time: | Thursday, September 6, 16:30 - 18:30 |
| Presentation: |
Poster
|
| Publication: |
2018 Conference on Cognitive Computational Neuroscience, 5-8 September 2018, Philadelphia, Pennsylvania |
| Paper Title: |
Noisy inference of value signals in frontal cortex drives exploration during reward-guided learning |
| Manuscript: |
Click here to view manuscript |
| DOI: |
https://doi.org/10.32470/CCN.2018.1196-0 |
| Authors: |
Charles Findling, Vasilisa Skvortsova, Ecole normale supérieure, France; Rémi Dromnelle, Université Pierre et Marie Curie, France; Nicolas Chopin, École nationale de la statistique et de l'administration économique, France; Stefano Palminteri, Valentin Wyart, Ecole normale supérieure, France |
| Abstract: |
When tracking rewarded stimulus-response associations in volatile environments, humans make a surprisingly large number of seemingly suboptimal decisions, which do not maximize expected outcome. These ‘exploratory’ decisions have been assigned either to information seeking or to stochasticity in response selection. We reasoned that a fraction of exploratory decisions could be due to random noise in the inference process driving learning, noise which is otherwise assumed to be negligible. Accounting simultaneously for these different sources of exploration in reinforcement learning revealed that more than half of exploratory decisions are due to inference noise alone. This computational dissection of exploration is supported by neuroimaging data, which shows a dissociation in the relationship between choice behavior and two brain regions associated with exploration: fluctuations in anterior cingulate activity co-vary with inference noise during learning, whereas frontopolar activity drives exploration during choice. Together, these findings indicate that exploration in reward-guided learning is driven to a large part by random errors in inference, unbeknownst to the decision-maker. |
