Related Content
Search Google Scholar for:
More Information
Related Jobs from ScienceCareers
|
|
Science 14 March 1997: Vol. 275. no. 5306, pp. 1593 - 1599 DOI: 10.1126/science.275.5306.1593
|
|
Articles
A Neural Substrate of Prediction and Reward
Wolfram Schultz,
Peter Dayan,
P. Read Montague
*
The capacity to predict future events permits a creature to detect,
model, and manipulate the causal structure of its interactions with its
environment. Behavioral experiments suggest that learning is driven by
changes in the expectations about future salient events such as rewards
and punishments. Physiological work has recently complemented these
studies by identifying dopaminergic neurons in the primate whose
fluctuating output apparently signals changes or errors in the
predictions of future salient and rewarding events. Taken together,
these findings can be understood through quantitative theories of
adaptive optimizing control.
W. Schultz is at the Institute of Physiology, University of Fribourg,
CH-1700 Fribourg, Switzerland. E-mail:
Wolfram.Schultz{at}unifr.ch P. Dayan is in the Department of
Brain and Cognitive Sciences, Center for Biological and Computational
Learning, E-25 MIT, Cambridge, MA 02139, USA. E-mail:
dayan{at}ai.mit.edu P. R. Montague is in the Division of Neuroscience,
Center for Theoretical Neuroscience, Baylor College of Medicine, 1 Baylor Plaza, Houston, TX 77030, USA. E-mail: read{at}bcm.tmc.edu
*
To whom correspondence should be addressed.
Read the Full Text
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES:
- A neurochemical approach to valuation sensitivity over gains and losses.
- S. Zhong, S. Israel, H. Xue, P. C. Sham, R. P. Ebstein, and S. H. Chew (2009)
Proc R Soc B
276, 4181-4188
| Abstract »
| Full Text »
| PDF »
- Reinforcement learning, conditioning, and the brain: Successes and challenges.
- T. V. Maia (2009)
Cogn Affect Behav Neurosci
9, 343-364
| Abstract »
| PDF »
- Aging and the neuroeconomics of decision making: A review.
- S. B. R. E. Brown and K. R. Ridderinkhof (2009)
Cogn Affect Behav Neurosci
9, 365-379
| Abstract »
| PDF »
- Caudate Nucleus Is Critically Involved in Trace Eyeblink Conditioning.
- L. C. Flores and J. F. Disterhoft (2009)
J. Neurosci.
29, 14511-14520
| Abstract »
| Full Text »
| PDF »
- How Humans Integrate the Prospects of Pain and Reward during Choice.
- D. Talmi, P. Dayan, S. J. Kiebel, C. D. Frith, and R. J. Dolan (2009)
J. Neurosci.
29, 14617-14626
| Abstract »
| Full Text »
| PDF »
- Neural Components Underlying Behavioral Flexibility in Human Reversal Learning.
- D. G. Ghahremani, J. Monterosso, J. D. Jentsch, R. M. Bilder, and R. A. Poldrack (2009)
Cereb Cortex
| Abstract »
| Full Text »
| PDF »
- Pharmacological modulation of subliminal learning in Parkinson's and Tourette's syndromes.
- S. Palminteri, M. Lebreton, Y. Worbe, D. Grabli, A. Hartmann, and M. Pessiglione (2009)
PNAS
106, 19179-19184
| Abstract »
| Full Text »
| PDF »
- Neural representation of time in cortico-basal ganglia circuits.
- D. Z. Jin, N. Fujii, and A. M. Graybiel (2009)
PNAS
106, 19156-19161
| Abstract »
| Full Text »
| PDF »
- Disrupted Effective Connectivity Between the Medial Frontal Cortex and the Caudate in Adolescent Boys With Externalizing Behavior Disorders.
- K. E. Shannon, C. Sauder, T. P. Beauchaine, and L. M. Gatzke-Kopp (2009)
Criminal Justice and Behavior
36, 1141-1157
| Abstract »
| PDF »
- Transient Firing of Dorsal Raphe Neurons Encodes Diverse and Specific Sensory, Motor, and Reward Events.
- S. P. Ranade and Z. F. Mainen (2009)
J Neurophysiol
102, 3026-3037
| Abstract »
| Full Text »
| PDF »
- Human Reinforcement Learning Subdivides Structured Action Spaces by Learning Effector-Specific Values.
- S. J. Gershman, B. Pesaran, and N. D. Daw (2009)
J. Neurosci.
29, 13524-13531
| Abstract »
| Full Text »
| PDF »
- Neural mechanisms for learned birdsong.
- R. Mooney (2009)
Learn. Mem.
16, 655-669
| Abstract »
| Full Text »
| PDF »
- Genetic variation in dopaminergic neuromodulation influences the ability to rapidly and flexibly adapt decisions.
- L. K. Krugel, G. Biele, P. N. C. Mohr, S.-C. Li, and H. R. Heekeren (2009)
PNAS
106, 17951-17956
| Abstract »
| Full Text »
| PDF »
- The Importance of Failure: Feedback-Related Negativity Predicts Motor Learning Efficiency.
- J. van der Helden, M. A. S. Boksem, and J. H. G. Blom (2009)
Cereb Cortex
| Abstract »
| Full Text »
| PDF »
- Corticostriatal Interactions during Learning, Memory Processing, and Decision Making.
- C. M. A. Pennartz, J. D. Berke, A. M. Graybiel, R. Ito, C. S. Lansink, M. van der Meer, A. D. Redish, K. S. Smith, and P. Voorn (2009)
J. Neurosci.
29, 12831-12838
| Abstract »
| Full Text »
| PDF »
- Neural computations underlying action-based decision making in the human brain.
- K. Wunderlich, A. Rangel, and J. P. O'Doherty (2009)
PNAS
106, 17199-17204
| Abstract »
| Full Text »
| PDF »
- Do Substantia Nigra Dopaminergic Neurons Differentiate Between Reward and Punishment?.
- M. J. Frank and D. J. Surmeier (2009)
J Mol Cell Biol
1, 15-16
| Abstract »
| Full Text »
| PDF »
- Adenylyl Cyclase Type 5 Contributes to Corticostriatal Plasticity and Striatum-Dependent Learning.
- M. A. Kheirbek, J. P. Britt, J. A. Beeler, Y. Ishikawa, D. S. McGehee, and X. Zhuang (2009)
J. Neurosci.
29, 12115-12124
| Abstract »
| Full Text »
| PDF »
- Reward-learning and the novelty-seeking personality: a between- and within-subjects study of the effects of dopamine agonists on young Parkinson's patients.
- N. Bodi, S. Keri, H. Nagy, A. Moustafa, C. E. Myers, N. Daw, G. Dibo, A. Takats, D. Bereczki, and M. A. Gluck (2009)
Brain
132, 2385-2395
| Abstract »
| Full Text »
| PDF »
- Restriction of dopamine signaling to the dorsolateral striatum is sufficient for many cognitive behaviors.
- M. Darvas and R. D. Palmiter (2009)
PNAS
106, 14664-14669
| Abstract »
| Full Text »
| PDF »
- Uncertainty during Anticipation Modulates Neural Responses to Aversion in Human Insula and Amygdala.
- I. Sarinopoulos, D. W. Grupe, K. L. Mackiewicz, J. D. Herrington, M. Lor, E. E. Steege, and J. B. Nitschke (2009)
Cereb Cortex
| Abstract »
| Full Text »
| PDF »
- Validation of Decision-Making Models and Analysis of Decision Variables in the Rat Basal Ganglia.
- M. Ito and K. Doya (2009)
J. Neurosci.
29, 9861-9874
| Abstract »
| Full Text »
| PDF »
- Which Way Do I Go? Neural Activation in Response to Feedback and Spatial Processing in a Virtual T-Maze.
- T. E. Baker and C. B. Holroyd (2009)
Cereb Cortex
19, 1708-1722
| Abstract »
| Full Text »
| PDF »
- Controls of Tonic and Phasic Dopamine Transmission in the Dorsal and Ventral Striatum.
- L. Zhang, W. M. Doyon, J. J. Clark, P. E. M. Phillips, and J. A. Dani (2009)
Mol. Pharmacol.
76, 396-404
| Abstract »
| Full Text »
| PDF »
- Single-Cell and Population Coding of Expected Reward Probability in the Orbitofrontal Cortex of the Rat.
- E. van Duuren, G. van der Plasse, J. Lankelma, R. N. J. M. A. Joosten, M. G. P. Feenstra, and C. M. A. Pennartz (2009)
J. Neurosci.
29, 8965-8976
| Abstract »
| Full Text »
| PDF »
- The Computation of Social Behavior.
- T. E. J. Behrens, L. T. Hunt, and M. F. S. Rushworth (2009)
Science
324, 1160-1164
| Abstract »
| Full Text »
| PDF »
- Role of 5-Hydroxytryptamine2C Receptors in Ca2+-Dependent Ethanol Potentiation of GABA Release onto Ventral Tegmental Area Dopamine Neurons.
- J. W. Theile, H. Morikawa, R. A. Gonzales, and R. A. Morrisett (2009)
J. Pharmacol. Exp. Ther.
329, 625-633
| Abstract »
| Full Text »
| PDF »
- Near-Term Liability of Exploitation: Exploration and Exploitation in Multistage Problems.
- C. Fang and D. Levinthal (2009)
Organization Science
20, 538-551
| Abstract »
| PDF »
- Dysconnection in Schizophrenia: From Abnormal Synaptic Plasticity to Failures of Self-monitoring.
- K. E. Stephan, K. J. Friston, and C. D. Frith (2009)
Schizophr Bull
35, 509-527
| Abstract »
| Full Text »
| PDF »
- The Dopamine Hypothesis of Schizophrenia: Version III--The Final Common Pathway.
- O. D. Howes and S. Kapur (2009)
Schizophr Bull
35, 549-562
| Abstract »
| Full Text »
| PDF »
- A Dual Role for Prediction Error in Associative Learning.
- H. E.M. den Ouden, K. J. Friston, N. D. Daw, A. R. McIntosh, and K. E. Stephan (2009)
Cereb Cortex
19, 1175-1185
| Abstract »
| Full Text »
| PDF »
- Functional Dissociations of Risk and Reward Processing in the Medial Prefrontal Cortex.
- G. Xue, Z. Lu, I. P. Levin, J. A. Weller, X. Li, and A. Bechara (2009)
Cereb Cortex
19, 1019-1027
| Abstract »
| Full Text »
| PDF »
- Different Pedunculopontine Tegmental Neurons Signal Predicted and Actual Task Rewards.
- K.-i. Okada, K. Toyama, Y. Inoue, T. Isa, and Y. Kobayashi (2009)
J. Neurosci.
29, 4858-4870
| Abstract »
| Full Text »
| PDF »
- Dopamine Signaling Differences in the Nucleus Accumbens and Dorsal Striatum Exploited by Nicotine.
- T. Zhang, L. Zhang, Y. Liang, A. G. Siapas, F.-M. Zhou, and J. A. Dani (2009)
J. Neurosci.
29, 4035-4043
| Abstract »
| Full Text »
| PDF »
- Human Substantia Nigra Neurons Encode Unexpected Financial Rewards.
- K. A. Zaghloul, J. A. Blanco, C. T. Weidemann, K. McGill, J. L. Jaggi, G. H. Baltuch, and M. J. Kahana (2009)
Science
323, 1496-1499
| Abstract »
| Full Text »
| PDF »
- Activity of Neurochemically Heterogeneous Dopaminergic Neurons in the Substantia Nigra during Spontaneous and Driven Changes in Brain State.
- M. T. C. Brown, P. Henny, J. P. Bolam, and P. J. Magill (2009)
J. Neurosci.
29, 2915-2925
| Abstract »
| Full Text »
| PDF »
- Distinct Subtypes of Basolateral Amygdala Taste Neurons Reflect Palatability and Reward.
- A. Fontanini, S. E. Grossman, J. A. Figueroa, and D. B. Katz (2009)
J. Neurosci.
29, 2486-2495
| Abstract »
| Full Text »
| PDF »
- Synaptic Overflow of Dopamine in the Nucleus Accumbens Arises from Neuronal Activity in the Ventral Tegmental Area.
- L. A. Sombers, M. Beyene, R. M. Carelli, and R. Mark Wightman (2009)
J. Neurosci.
29, 1735-1742
| Abstract »
| Full Text »
| PDF »
- Striatal Dopamine Predicts Outcome-Specific Reversal Learning and Its Sensitivity to Dopaminergic Drug Administration.
- R. Cools, M. J. Frank, S. E. Gibbs, A. Miyakawa, W. Jagust, and M. D'Esposito (2009)
J. Neurosci.
29, 1538-1543
| Abstract »
| Full Text »
| PDF »
- Encoding of Probabilistic Rewarding and Aversive Events by Pallidal and Nigral Neurons.
- M. Joshua, A. Adler, B. Rosin, E. Vaadia, and H. Bergman (2009)
J Neurophysiol
101, 758-772
| Abstract »
| Full Text »
| PDF »
- CNTRICS Final Task Selection: Long-Term Memory.
- J. D. Ragland, R. Cools, M. Frank, D. A. Pizzagalli, A. Preston, C. Ranganath, and A. D. Wagner (2009)
Schizophr Bull
35, 197-212
| Abstract »
| Full Text »
| PDF »
- Complementary roles for amygdala and periaqueductal gray in temporal-difference fear learning.
- S. Cole and G. P. McNally (2008)
Learn. Mem.
16, 1-7
| Abstract »
| Full Text »
| PDF »
- Changes in Control of Saccades during Gain Adaptation.
- V. Ethier, D. S. Zee, and R. Shadmehr (2008)
J. Neurosci.
28, 13929-13937
| Abstract »
| Full Text »
| PDF »
- Anticipatory affect: neural correlates and consequences for choice.
- B. Knutson and S. M Greer (2008)
Phil Trans R Soc B
363, 3771-3786
| Abstract »
| Full Text »
| PDF »
- The role of the striatum in aversive learning and aversive prediction errors.
- M. R Delgado, J. Li, D. Schiller, and E. A Phelps (2008)
Phil Trans R Soc B
363, 3787-3800
| Abstract »
| Full Text »
| PDF »
- Explicit neural signals reflecting reward uncertainty.
- W. Schultz, K. Preuschoff, C. Camerer, M. Hsu, C. D Fiorillo, P. N Tobler, and P. Bossaerts (2008)
Phil Trans R Soc B
363, 3801-3811
| Abstract »
| Full Text »
| PDF »
- Neuroethology of reward and decision making.
- K. K Watson and M. L Platt (2008)
Phil Trans R Soc B
363, 3825-3835
| Abstract »
| Full Text »
| PDF »
- The Neuromodulatory System: A Framework for Survival and Adaptive Behavior in a Challenging World.
- J. L. Krichmar (2008)
Adaptive Behavior
16, 385-399
| Abstract »
| PDF »
- Conceptual representations in goal-directed decision making.
- N. Shea, K. Krug, and P. N. Tobler (2008)
Cogn Affect Behav Neurosci
8, 418-428
| Abstract »
| PDF »
- Decision theory, reinforcement learning, and the brain.
- P. Dayan and N. D. Daw (2008)
Cogn Affect Behav Neurosci
8, 429-453
| Abstract »
| PDF »
- A Role for Dopamine in Temporal Decision Making and Reward Maximization in Parkinsonism.
- A. A. Moustafa, M. X. Cohen, S. J. Sherman, and M. J. Frank (2008)
J. Neurosci.
28, 12294-12304
| Abstract »
| Full Text »
| PDF »
- From Fear to Safety and Back: Reversal of Fear in the Human Brain.
- D. Schiller, I. Levy, Y. Niv, J. E. LeDoux, and E. A. Phelps (2008)
J. Neurosci.
28, 11517-11525
| Abstract »
| Full Text »
| PDF »
- Midbrain Dopaminergic Neurons and Striatal Cholinergic Interneurons Encode the Difference between Reward and Aversive Events at Different Epochs of Probabilistic Classical Conditioning Trials.
- M. Joshua, A. Adler, R. Mitelman, E. Vaadia, and H. Bergman (2008)
J. Neurosci.
28, 11673-11684
| Abstract »
| Full Text »
| PDF »
- The Spatial Attention Network Interacts with Limbic and Monoaminergic Systems to Modulate Motivation-Induced Attention Shifts.
- A. Mohanty, D. R. Gitelman, D. M. Small, and M. M. Mesulam (2008)
Cereb Cortex
18, 2604-2613
| Abstract »
| Full Text »
| PDF »
- A cAMP Pathway Underlying Reward Prediction in Associative Learning.
- M. A. Kheirbek, J. A. Beeler, Y. Ishikawa, and X. Zhuang (2008)
J. Neurosci.
28, 11401-11408
| Abstract »
| Full Text »
| PDF »
- Relation Between Obesity and Blunted Striatal Response to Food Is Moderated by TaqIA A1 Allele.
- E. Stice, S. Spoor, C. Bohon, and D. M. Small (2008)
Science
322, 449-452
| Abstract »
| Full Text »
| PDF »
- Moment-to-Moment Tracking of State Value in the Amygdala.
- M. A. Belova, J. J. Paton, and C. D. Salzman (2008)
J. Neurosci.
28, 10023-10030
| Abstract »
| Full Text »
| PDF »
- A Local Circuit Model of Learned Striatal and Dopamine Cell Responses under Probabilistic Schedules of Reward.
- C. O. Tan and D. Bullock (2008)
J. Neurosci.
28, 10062-10074
| Abstract »
| Full Text »
| PDF »
- Tripartite Mechanism of Extinction Suggested by Dopamine Neuron Activity and Temporal Difference Model.
- W.-X. Pan, R. Schmidt, J. R. Wickens, and B. I. Hyland (2008)
J. Neurosci.
28, 9619-9631
| Abstract »
| Full Text »
| PDF »
- Reward Processing in Schizophrenia: A Deficit in the Representation of Value.
- J. M. Gold, J. A. Waltz, K. J. Prentice, S. E. Morris, and E. A. Heerey (2008)
Schizophr Bull
34, 835-847
| Abstract »
| Full Text »
| PDF »
- Reinforcement and Reversal Learning in First-Episode Psychosis.
- G. K. Murray, F. Cheng, L. Clark, J. H. Barnett, A. D. Blackwell, P. C. Fletcher, T. W. Robbins, E. T. Bullmore, and P. B. Jones (2008)
Schizophr Bull
34, 848-855
| Abstract »
| Full Text »
| PDF »
- Distinctive Roles for the Ventral Striatum and Ventral Prefrontal Cortex during Decision-Making.
- L. T. Hunt (2008)
J. Neurosci.
28, 8658-8659
| Full Text »
| PDF »
- Dynamic changes in accumbens dopamine correlate with learning during intracranial self-stimulation.
- C. A. Owesson-White, J. F. Cheer, M. Beyene, R. M. Carelli, and R. M. Wightman (2008)
PNAS
105, 11957-11962
| Abstract »
| Full Text »
| PDF »
- Influence of Neuronal Nicotinic Receptors over Nicotine Addiction and Withdrawal.
- M. De Biasi and R. Salas (2008)
Experimental Biology and Medicine
233, 917-929
| Abstract »
| Full Text »
| PDF »
- Opioid Receptor PET Reveals the Psychobiologic Correlates of Reward Processing.
- M. Schreckenberger, A. Klega, G. Grunder, H.-G. Buchholz, A. Scheurich, R. Schirrmacher, E. Schirrmacher, C. Muller, G. Henriksen, and P. Bartenstein (2008)
J. Nucl. Med.
49, 1257-1261
| Abstract »
| Full Text »
| PDF »
- Influence of Reward Delays on Responses of Dopamine Neurons.
- S. Kobayashi and W. Schultz (2008)
J. Neurosci.
28, 7837-7846
| Abstract »
| Full Text »
| PDF »
- Evidence for Segregated and Integrative Connectivity Patterns in the Human Basal Ganglia.
- B. Draganski, F. Kherif, S. Kloppel, P. A. Cook, D. C. Alexander, G. J. M. Parker, R. Deichmann, J. Ashburner, and R. S. J. Frackowiak (2008)
J. Neurosci.
28, 7143-7152
| Abstract »
| Full Text »
| PDF »
- What's in a Smile? Maternal Brain Responses to Infant Facial Cues.
- L. Strathearn, J. Li, P. Fonagy, and P. R. Montague (2008)
Pediatrics
122, 40-51
| Abstract »
| Full Text »
| PDF »
- Preferential Reactivation of Motivationally Relevant Information in the Ventral Striatum.
- C. S. Lansink, P. M. Goltstein, J. V. Lankelma, R. N. J. M. A. Joosten, B. L. McNaughton, and C. M. A. Pennartz (2008)
J. Neurosci.
28, 6372-6382
| Abstract »
| Full Text »
| PDF »
- Methylphenidate Has Differential Effects on Blood Oxygenation Level-Dependent Signal Related to Cognitive Subprocesses of Reversal Learning.
- C. M. Dodds, U. Muller, L. Clark, A. van Loon, R. Cools, and T. W. Robbins (2008)
J. Neurosci.
28, 5976-5982
| Abstract »
| Full Text »
| PDF »
- Neurocomputational mechanisms of reinforcement-guided learning in humans: A review.
- M. X. COHEN (2008)
Cogn Affect Behav Neurosci
8, 113-125
| Abstract »
| PDF »
- Dissociating the Role of the Orbitofrontal Cortex and the Striatum in the Computation of Goal Values and Prediction Errors.
- T. A. Hare, J. O'Doherty, C. F. Camerer, W. Schultz, and A. Rangel (2008)
J. Neurosci.
28, 5623-5630
| Abstract »
| Full Text »
| PDF »
- Opposing Patterns of Signaling Activation in Dopamine D1 and D2 Receptor-Expressing Striatal Neurons in Response to Cocaine and Haloperidol.
- J. Bertran-Gonzalez, C. Bosch, M. Maroteaux, M. Matamales, D. Herve, E. Valjent, and J.-A. Girault (2008)
J. Neurosci.
28, 5671-5685
| Abstract »
| Full Text »
| PDF »
- Reward-Dependent Modulation of Neuronal Activity in the Primate Dorsal Raphe Nucleus.
- K. Nakamura, M. Matsumoto, and O. Hikosaka (2008)
J. Neurosci.
28, 5331-5343
| Abstract »
| Full Text »
| PDF »
- Disconnecting force from money: effects of basal ganglia damage on incentive motivation.
- L. Schmidt, B. F. d'Arc, G. Lafargue, D. Galanaud, V. Czernecki, D. Grabli, M. Schupbach, A. Hartmann, R. Levy, B. Dubois, et al. (2008)
Brain
131, 1303-1310
| Abstract »
| Full Text »
| PDF »
- Attaching Values to Actions: Action and Outcome Encoding in the Primate Caudate Nucleus.
- C. H. Donahue and H. Seo (2008)
J. Neurosci.
28, 4579-4580
| Full Text »
| PDF »
- Extracellular Signal-Related Kinase Activation During Natural Reward Learning: A Physiological Role for Phasic Nucleus Accumbens Dopamine?.
- J. J. Day (2008)
J. Neurosci.
28, 4295-4297
| Full Text »
| PDF »
- Distinct error-correcting and incidental learning of location relative to landmarks and boundaries.
- C. F. Doeller and N. Burgess (2008)
PNAS
105, 5909-5914
| Abstract »
| Full Text »
| PDF »
- Drosophila Egg-Laying Site Selection as a System to Study Simple Decision-Making Processes.
- C.-h. Yang, P. Belawat, E. Hafen, L. Y. Jan, and Y.-N. Jan (2008)
Science
319, 1679-1683
| Abstract »
| Full Text »
| PDF »
- Motor Adaptation as a Process of Reoptimization.
- J. Izawa, T. Rane, O. Donchin, and R. Shadmehr (2008)
J. Neurosci.
28, 2883-2891
| Abstract »
| Full Text »
| PDF »
- BOLD Responses Reflecting Dopaminergic Signals in the Human Ventral Tegmental Area.
- K. D'Ardenne, S. M. McClure, L. E. Nystrom, and J. D. Cohen (2008)
Science
319, 1264-1267
| Abstract »
| Full Text »
| PDF »
- Decoding of Temporal Intervals From Cortical Ensemble Activity.
- M. A. Lebedev, J. E. O'Doherty, and M. A. L. Nicolelis (2008)
J Neurophysiol
99, 166-186
| Abstract »
| Full Text »
| PDF »
- Cerebrovascular responses to incremental exercise during hypobaric hypoxia: effect of oxygenation on maximal performance.
- A. W. Subudhi, M. C. Lorenz, C. S. Fulco, and R. C. Roach (2008)
Am J Physiol Heart Circ Physiol
294, H164-H171
| Abstract »
| Full Text »
| PDF »
|
|