Note to users. If you're seeing this message, it means that your browser cannot find this page's style/presentation instructions -- or possibly that you are using a browser that does not support current Web standards. Find out more about why this message is appearing, and what you can do to make your experience of our site the best it can be.

Science 25 November 2005:
Vol. 310. no. 5752, pp. 1337 - 1340
DOI: 10.1126/science.1115270
Prev | Table of Contents | Next

Reports

Representation of Action-Specific Reward Values in the Striatum

Kazuyuki Samejima,1*{dagger} Yasumasa Ueda,2 Kenji Doya,1,3 Minoru Kimura2*

The estimation of the reward an action will yield is critical in decision-making. To elucidate the role of the basal ganglia in this process, we recorded striatal neurons of monkeys who chose between left and right handle turns, based on the estimated reward probabilities of the actions. During a delay period before the choices, the activity of more than one-third of striatal projection neurons was selective to the values of one of the two actions. Fewer neurons were tuned to relative values or action choice. These results suggest representation of action values in the striatum, which can guide action selection in the basal ganglia circuit.

1 Department of Computational Neurobiology, ATR Computational Neuroscience Laboratories, 619-0288 Kyoto, Japan.
2 Department of Physiology, Kyoto Prefectural University of Medicine, 602-8566 Kyoto, Japan.
3 Initial Research Project, Okinawa Institute of Science and Technology, 904-2234 Okinawa, Japan.

{dagger} Present address: Brain Science Research Center, Tamagawa University Research Institute, 194-8610 Tokyo, Japan.

* To whom correspondence should be addressed. E-mail: samejima{at}lab.tamagawa.ac.jp (K.S.); mkimura{at}koto.kpu-m.ac.jp (M.K.)

Read the Full Text


THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES:
Brain Hemispheres Selectively Track the Expected Value of Contralateral Options.
S. Palminteri, T. Boraud, G. Lafargue, B. Dubois, and M. Pessiglione (2009)
J. Neurosci. 29, 13465-13472
   Abstract »    Full Text »    PDF »
Human Reinforcement Learning Subdivides Structured Action Spaces by Learning Effector-Specific Values.
S. J. Gershman, B. Pesaran, and N. D. Daw (2009)
J. Neurosci. 29, 13524-13531
   Abstract »    Full Text »    PDF »
Ventral Striatal Neurons Encode the Value of the Chosen Action in Rats Deciding between Differently Delayed or Sized Rewards.
M. R. Roesch, T. Singh, P. L. Brown, S. E. Mullins, and G. Schoenbaum (2009)
J. Neurosci. 29, 13365-13376
   Abstract »    Full Text »    PDF »
Neural computations underlying action-based decision making in the human brain.
K. Wunderlich, A. Rangel, and J. P. O'Doherty (2009)
PNAS 106, 17199-17204
   Abstract »    Full Text »    PDF »
Stable Encoding of Task Structure Coexists With Flexible Coding of Task Events in Sensorimotor Striatum.
Y. Kubota, J. Liu, D. Hu, W. E. DeCoteau, U. T. Eden, A. C. Smith, and A. M. Graybiel (2009)
J Neurophysiol 102, 2142-2160
   Abstract »    Full Text »    PDF »
Neural Activity in the Human Brain Signals Logical Rule Identification.
K. Tachibana, K. Suzuki, E. Mori, N. Miura, R. Kawashima, K. Horie, S. Sato, J. Tanji, and H. Mushiake (2009)
J Neurophysiol 102, 1526-1537
   Abstract »    Full Text »    PDF »
Validation of Decision-Making Models and Analysis of Decision Variables in the Rat Basal Ganglia.
M. Ito and K. Doya (2009)
J. Neurosci. 29, 9861-9874
   Abstract »    Full Text »    PDF »
A basal ganglia-forebrain circuit in the songbird biases motor output to avoid vocal errors.
A. S. Andalman and and M. S. Fee (2009)
PNAS 106, 12518-12523
   Abstract »    Full Text »    PDF »
Lateral Intraparietal Cortex and Reinforcement Learning during a Mixed-Strategy Game.
H. Seo, D. J. Barraclough, and D. Lee (2009)
J. Neurosci. 29, 7278-7289
   Abstract »    Full Text »    PDF »
Model-based reinforcement learning under concurrent schedules of reinforcement in rodents.
N. Huh, S. Jo, H. Kim, J. H. Sul, and M. W. Jung (2009)
Learn. Mem. 16, 315-323
   Abstract »    Full Text »    PDF »
Behavioral and Neural Changes after Gains and Losses of Conditioned Reinforcers.
H. Seo and D. Lee (2009)
J. Neurosci. 29, 3627-3641
   Abstract »    Full Text »    PDF »
Encoding of Probabilistic Rewarding and Aversive Events by Pallidal and Nigral Neurons.
M. Joshua, A. Adler, B. Rosin, E. Vaadia, and H. Bergman (2009)
J Neurophysiol 101, 758-772
   Abstract »    Full Text »    PDF »
Explicit neural signals reflecting reward uncertainty.
W. Schultz, K. Preuschoff, C. Camerer, M. Hsu, C. D Fiorillo, P. N Tobler, and P. Bossaerts (2008)
Phil Trans R Soc B 363, 3801-3811
   Abstract »    Full Text »    PDF »
Cortical mechanisms for reinforcement learning in competitive games.
H. Seo and D. Lee (2008)
Phil Trans R Soc B 363, 3845-3857
   Abstract »    Full Text »    PDF »
The role of moral utility in decision making: An interdisciplinary framework.
P. N. Tobler, A. Kalis, and T. Kalenscher (2008)
Cogn Affect Behav Neurosci 8, 390-401
   Abstract »    PDF »
Moment-to-Moment Tracking of State Value in the Amygdala.
M. A. Belova, J. J. Paton, and C. D. Salzman (2008)
J. Neurosci. 28, 10023-10030
   Abstract »    Full Text »    PDF »
A Local Circuit Model of Learned Striatal and Dopamine Cell Responses under Probabilistic Schedules of Reward.
C. O. Tan and D. Bullock (2008)
J. Neurosci. 28, 10062-10074
   Abstract »    Full Text »    PDF »
Distinctive Roles for the Ventral Striatum and Ventral Prefrontal Cortex during Decision-Making.
L. T. Hunt (2008)
J. Neurosci. 28, 8658-8659
   Full Text »    PDF »
Evidence for Segregated and Integrative Connectivity Patterns in the Human Basal Ganglia.
B. Draganski, F. Kherif, S. Kloppel, P. A. Cook, D. C. Alexander, G. J. M. Parker, R. Deichmann, J. Ashburner, and R. S. J. Frackowiak (2008)
J. Neurosci. 28, 7143-7152
   Abstract »    Full Text »    PDF »
Understanding the Neural Computations of Arbitrary Visuomotor Learning through fMRI and Associative Learning Theory.
A. Brovelli, N. Laksiri, B. Nazarian, M. Meunier, and D. Boussaoud (2008)
Cereb Cortex 18, 1485-1495
   Abstract »    Full Text »    PDF »
From 'Understanding the Brain by Creating the Brain' towards manipulative neuroscience.
M. Kawato (2008)
Phil Trans R Soc B 363, 2201-2214
   Abstract »    Full Text »    PDF »
Neurocomputational mechanisms of reinforcement-guided learning in humans: A review.
M. X. COHEN (2008)
Cogn Affect Behav Neurosci 8, 113-125
   Abstract »    PDF »
Attaching Values to Actions: Action and Outcome Encoding in the Primate Caudate Nucleus.
C. H. Donahue and H. Seo (2008)
J. Neurosci. 28, 4579-4580
   Full Text »    PDF »
Action and Outcome Encoding in the Primate Caudate Nucleus.
B. Lau and P. W. Glimcher (2007)
J. Neurosci. 27, 14502-14514
   Abstract »    Full Text »    PDF »
Inactivation of the Lateral But Not Medial Dorsal Striatum Eliminates the Excitatory Impact of Pavlovian Stimuli on Instrumental Responding.
L. H. Corbit and P. H. Janak (2007)
J. Neurosci. 27, 13977-13981
   Abstract »    Full Text »    PDF »
Order-Dependent Modulation of Directional Signals in the Supplementary and Presupplementary Motor Areas.
J.-W. Sohn and D. Lee (2007)
J. Neurosci. 27, 13655-13666
   Abstract »    Full Text »    PDF »
History- and Current Instruction-Based Coding of Forthcoming Behavioral Outcomes in the Striatum.
H. Yamada, N. Matsumoto, and M. Kimura (2007)
J Neurophysiol 98, 3557-3567
   Abstract »    Full Text »    PDF »
Encoding of Action History in the Rat Ventral Striatum.
Y. B. Kim, N. Huh, H. Lee, E. H. Baeg, D. Lee, and M. W. Jung (2007)
J Neurophysiol 98, 3548-3556
   Abstract »    Full Text »    PDF »
Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning.
M. J. Frank, A. A. Moustafa, H. M. Haughey, T. Curran, and K. E. Hutchison (2007)
PNAS 104, 16311-16316
   Abstract »    Full Text »    PDF »
Dynamic Signals Related to Choices and Outcomes in the Dorsolateral Prefrontal Cortex.
H. Seo, D. J. Barraclough, and D. Lee (2007)
Cereb Cortex 17, i110-i117
   Abstract »    Full Text »    PDF »
Understanding Neural Coding through the Model-Based Analysis of Decision Making.
G. Corrado and K. Doya (2007)
J. Neurosci. 27, 8178-8180
   Abstract »    Full Text »    PDF »
Temporal Filtering of Reward Signals in the Dorsal Anterior Cingulate Cortex during a Mixed-Strategy Game.
H. Seo and D. Lee (2007)
J. Neurosci. 27, 8366-8377
   Abstract »    Full Text »    PDF »
Direct Instrumental Conditioning of Neural Activity Using Functional Magnetic Resonance Imaging-Derived Reward Feedback.
S. Bray, S. Shimojo, and J. P. O'Doherty (2007)
J. Neurosci. 27, 7498-7507
   Abstract »    Full Text »    PDF »
Adaptation of Prefrontal Cortical Firing Patterns and Their Fidelity to Changes in Action-Reward Contingencies.
W. J. Kargo, B. Szatmary, and D. A. Nitz (2007)
J. Neurosci. 27, 3548-3559
   Abstract »    Full Text »    PDF »
Individual differences and the neural representations of reward expectation and reward prediction error.
M. X Cohen (2007)
Soc Cogn Affect Neurosci 2, 20-30
   Abstract »    Full Text »    PDF »
Shaping of Motor Responses by Incentive Values through the Basal Ganglia.
B. Pasquereau, A. Nadjar, D. Arkadir, E. Bezard, M. Goillandeau, B. Bioulac, C. E. Gross, and T. Boraud (2007)
J. Neurosci. 27, 1176-1183
   Abstract »    Full Text »    PDF »
Reinforcement Learning Signals Predict Future Decisions.
M. X Cohen and C. Ranganath (2007)
J. Neurosci. 27, 371-378
   Abstract »    Full Text »    PDF »
A biophysically based neural model of matching law behavior: melioration by stochastic synapses..
A. Soltani and X.-J. Wang (2006)
J. Neurosci. 26, 3731-3744
   Abstract »    Full Text »    PDF »


To Advertise     Find Products

Science. ISSN 0036-8075 (print), 1095-9203 (online)