Note to users. If you're seeing this message, it means that your browser cannot find this page's style/presentation instructions -- or possibly that you are using a browser that does not support current Web standards. Find out more about why this message is appearing, and what you can do to make your experience of our site the best it can be.

Site Tools

  • AAAS
  • Subscribe
  • Feedback

Site Search

Search Advanced

Science 14 March 1997:
Vol. 275. no. 5306, pp. 1593 - 1599
DOI: 10.1126/science.275.5306.1593
Prev | Table of Contents | Next

Articles

A Neural Substrate of Prediction and Reward

Wolfram Schultz, Peter Dayan, P. Read Montague *

The capacity to predict future events permits a creature to detect, model, and manipulate the causal structure of its interactions with its environment. Behavioral experiments suggest that learning is driven by changes in the expectations about future salient events such as rewards and punishments. Physiological work has recently complemented these studies by identifying dopaminergic neurons in the primate whose fluctuating output apparently signals changes or errors in the predictions of future salient and rewarding events. Taken together, these findings can be understood through quantitative theories of adaptive optimizing control.

W. Schultz is at the Institute of Physiology, University of Fribourg, CH-1700 Fribourg, Switzerland. E-mail: Wolfram.Schultz{at}unifr.ch  P. Dayan is in the Department of Brain and Cognitive Sciences, Center for Biological and Computational Learning, E-25 MIT, Cambridge, MA 02139, USA. E-mail: dayan{at}ai.mit.edu  P. R. Montague is in the Division of Neuroscience, Center for Theoretical Neuroscience, Baylor College of Medicine, 1 Baylor Plaza, Houston, TX 77030, USA. E-mail: read{at}bcm.tmc.edu
*   To whom correspondence should be addressed.

Read the Full Text


THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES:
Which Way Do I Go? Neural Activation in Response to Feedback and Spatial Processing in a Virtual T-Maze.
T. E. Baker and C. B. Holroyd (2009)
Cereb Cortex 19, 1708-1722
   Abstract »    Full Text »    PDF »
The Computation of Social Behavior.
T. E. J. Behrens, L. T. Hunt, and M. F. S. Rushworth (2009)
Science 324, 1160-1164
   Abstract »    Full Text »    PDF »
Reward-learning and the novelty-seeking personality: a between- and within-subjects study of the effects of dopamine agonists on young Parkinson's patients.
N. Bodi, S. Keri, H. Nagy, A. Moustafa, C. E. Myers, N. Daw, G. Dibo, A. Takats, D. Bereczki, and M. A. Gluck (2009)
Brain
   Abstract »    Full Text »    PDF »
Role of 5-Hydroxytryptamine2C Receptors in Ca2+-Dependent Ethanol Potentiation of GABA Release onto Ventral Tegmental Area Dopamine Neurons.
J. W. Theile, H. Morikawa, R. A. Gonzales, and R. A. Morrisett (2009)
J. Pharmacol. Exp. Ther. 329, 625-633
   Abstract »    Full Text »    PDF »
Near-Term Liability of Exploitation: Exploration and Exploitation in Multistage Problems.
C. Fang and D. Levinthal (2009)
Organization Science 20, 538-551
   Abstract »    PDF »
Dysconnection in Schizophrenia: From Abnormal Synaptic Plasticity to Failures of Self-monitoring.
K. E. Stephan, K. J. Friston, and C. D. Frith (2009)
Schizophr Bull 35, 509-527
   Abstract »    Full Text »    PDF »
The Dopamine Hypothesis of Schizophrenia: Version III--The Final Common Pathway.
O. D. Howes and S. Kapur (2009)
Schizophr Bull 35, 549-562
   Abstract »    Full Text »    PDF »
A Dual Role for Prediction Error in Associative Learning.
H. E.M. den Ouden, K. J. Friston, N. D. Daw, A. R. McIntosh, and K. E. Stephan (2009)
Cereb Cortex 19, 1175-1185
   Abstract »    Full Text »    PDF »
Functional Dissociations of Risk and Reward Processing in the Medial Prefrontal Cortex.
G. Xue, Z. Lu, I. P. Levin, J. A. Weller, X. Li, and A. Bechara (2009)
Cereb Cortex 19, 1019-1027
   Abstract »    Full Text »    PDF »
Different Pedunculopontine Tegmental Neurons Signal Predicted and Actual Task Rewards.
K.-i. Okada, K. Toyama, Y. Inoue, T. Isa, and Y. Kobayashi (2009)
J. Neurosci. 29, 4858-4870
   Abstract »    Full Text »    PDF »
Dopamine Signaling Differences in the Nucleus Accumbens and Dorsal Striatum Exploited by Nicotine.
T. Zhang, L. Zhang, Y. Liang, A. G. Siapas, F.-M. Zhou, and J. A. Dani (2009)
J. Neurosci. 29, 4035-4043
   Abstract »    Full Text »    PDF »
Human Substantia Nigra Neurons Encode Unexpected Financial Rewards.
K. A. Zaghloul, J. A. Blanco, C. T. Weidemann, K. McGill, J. L. Jaggi, G. H. Baltuch, and M. J. Kahana (2009)
Science 323, 1496-1499
   Abstract »    Full Text »    PDF »
Activity of Neurochemically Heterogeneous Dopaminergic Neurons in the Substantia Nigra during Spontaneous and Driven Changes in Brain State.
M. T. C. Brown, P. Henny, J. P. Bolam, and P. J. Magill (2009)
J. Neurosci. 29, 2915-2925
   Abstract »    Full Text »    PDF »
Distinct Subtypes of Basolateral Amygdala Taste Neurons Reflect Palatability and Reward.
A. Fontanini, S. E. Grossman, J. A. Figueroa, and D. B. Katz (2009)
J. Neurosci. 29, 2486-2495
   Abstract »    Full Text »    PDF »
Synaptic Overflow of Dopamine in the Nucleus Accumbens Arises from Neuronal Activity in the Ventral Tegmental Area.
L. A. Sombers, M. Beyene, R. M. Carelli, and R. Mark Wightman (2009)
J. Neurosci. 29, 1735-1742
   Abstract »    Full Text »    PDF »
Striatal Dopamine Predicts Outcome-Specific Reversal Learning and Its Sensitivity to Dopaminergic Drug Administration.
R. Cools, M. J. Frank, S. E. Gibbs, A. Miyakawa, W. Jagust, and M. D'Esposito (2009)
J. Neurosci. 29, 1538-1543
   Abstract »    Full Text »    PDF »
Encoding of Probabilistic Rewarding and Aversive Events by Pallidal and Nigral Neurons.
M. Joshua, A. Adler, B. Rosin, E. Vaadia, and H. Bergman (2009)
J Neurophysiol 101, 758-772
   Abstract »    Full Text »    PDF »
CNTRICS Final Task Selection: Long-Term Memory.
J. D. Ragland, R. Cools, M. Frank, D. A. Pizzagalli, A. Preston, C. Ranganath, and A. D. Wagner (2009)
Schizophr Bull 35, 197-212
   Abstract »    Full Text »    PDF »
Complementary roles for amygdala and periaqueductal gray in temporal-difference fear learning.
S. Cole and G. P. McNally (2008)
Learn. Mem. 16, 1-7
   Abstract »    Full Text »    PDF »
Changes in Control of Saccades during Gain Adaptation.
V. Ethier, D. S. Zee, and R. Shadmehr (2008)
J. Neurosci. 28, 13929-13937
   Abstract »    Full Text »    PDF »
Anticipatory affect: neural correlates and consequences for choice.
B. Knutson and S. M Greer (2008)
Phil Trans R Soc B 363, 3771-3786
   Abstract »    Full Text »    PDF »
The role of the striatum in aversive learning and aversive prediction errors.
M. R Delgado, J. Li, D. Schiller, and E. A Phelps (2008)
Phil Trans R Soc B 363, 3787-3800
   Abstract »    Full Text »    PDF »
Explicit neural signals reflecting reward uncertainty.
W. Schultz, K. Preuschoff, C. Camerer, M. Hsu, C. D Fiorillo, P. N Tobler, and P. Bossaerts (2008)
Phil Trans R Soc B 363, 3801-3811
   Abstract »    Full Text »    PDF »
Neuroethology of reward and decision making.
K. K Watson and M. L Platt (2008)
Phil Trans R Soc B 363, 3825-3835
   Abstract »    Full Text »    PDF »
The Neuromodulatory System: A Framework for Survival and Adaptive Behavior in a Challenging World.
J. L. Krichmar (2008)
Adaptive Behavior 16, 385-399
   Abstract »    PDF »
Conceptual representations in goal-directed decision making.
N. Shea, K. Krug, and P. N. Tobler (2008)
Cogn Affect Behav Neurosci 8, 418-428
   Abstract »    PDF »
Decision theory, reinforcement learning, and the brain.
P. Dayan and N. D. Daw (2008)
Cogn Affect Behav Neurosci 8, 429-453
   Abstract »    PDF »
A Role for Dopamine in Temporal Decision Making and Reward Maximization in Parkinsonism.
A. A. Moustafa, M. X. Cohen, S. J. Sherman, and M. J. Frank (2008)
J. Neurosci. 28, 12294-12304
   Abstract »    Full Text »    PDF »
From Fear to Safety and Back: Reversal of Fear in the Human Brain.
D. Schiller, I. Levy, Y. Niv, J. E. LeDoux, and E. A. Phelps (2008)
J. Neurosci. 28, 11517-11525
   Abstract »    Full Text »    PDF »
Midbrain Dopaminergic Neurons and Striatal Cholinergic Interneurons Encode the Difference between Reward and Aversive Events at Different Epochs of Probabilistic Classical Conditioning Trials.
M. Joshua, A. Adler, R. Mitelman, E. Vaadia, and H. Bergman (2008)
J. Neurosci. 28, 11673-11684
   Abstract »    Full Text »    PDF »
The Spatial Attention Network Interacts with Limbic and Monoaminergic Systems to Modulate Motivation-Induced Attention Shifts.
A. Mohanty, D. R. Gitelman, D. M. Small, and M. M. Mesulam (2008)
Cereb Cortex 18, 2604-2613
   Abstract »    Full Text »    PDF »
A cAMP Pathway Underlying Reward Prediction in Associative Learning.
M. A. Kheirbek, J. A. Beeler, Y. Ishikawa, and X. Zhuang (2008)
J. Neurosci. 28, 11401-11408
   Abstract »    Full Text »    PDF »
Relation Between Obesity and Blunted Striatal Response to Food Is Moderated by TaqIA A1 Allele.
E. Stice, S. Spoor, C. Bohon, and D. M. Small (2008)
Science 322, 449-452
   Abstract »    Full Text »    PDF »
Moment-to-Moment Tracking of State Value in the Amygdala.
M. A. Belova, J. J. Paton, and C. D. Salzman (2008)
J. Neurosci. 28, 10023-10030
   Abstract »    Full Text »    PDF »
A Local Circuit Model of Learned Striatal and Dopamine Cell Responses under Probabilistic Schedules of Reward.
C. O. Tan and D. Bullock (2008)
J. Neurosci. 28, 10062-10074
   Abstract »    Full Text »    PDF »
Tripartite Mechanism of Extinction Suggested by Dopamine Neuron Activity and Temporal Difference Model.
W.-X. Pan, R. Schmidt, J. R. Wickens, and B. I. Hyland (2008)
J. Neurosci. 28, 9619-9631
   Abstract »    Full Text »    PDF »
Reward Processing in Schizophrenia: A Deficit in the Representation of Value.
J. M. Gold, J. A. Waltz, K. J. Prentice, S. E. Morris, and E. A. Heerey (2008)
Schizophr Bull 34, 835-847
   Abstract »    Full Text »    PDF »
Reinforcement and Reversal Learning in First-Episode Psychosis.
G. K. Murray, F. Cheng, L. Clark, J. H. Barnett, A. D. Blackwell, P. C. Fletcher, T. W. Robbins, E. T. Bullmore, and P. B. Jones (2008)
Schizophr Bull 34, 848-855
   Abstract »    Full Text »    PDF »
Distinctive Roles for the Ventral Striatum and Ventral Prefrontal Cortex during Decision-Making.
L. T. Hunt (2008)
J. Neurosci. 28, 8658-8659
   Full Text »    PDF »
Dynamic changes in accumbens dopamine correlate with learning during intracranial self-stimulation.
C. A. Owesson-White, J. F. Cheer, M. Beyene, R. M. Carelli, and R. M. Wightman (2008)
PNAS 105, 11957-11962
   Abstract »    Full Text »    PDF »
Influence of Neuronal Nicotinic Receptors over Nicotine Addiction and Withdrawal.
M. De Biasi and R. Salas (2008)
Experimental Biology and Medicine 233, 917-929
   Abstract »    Full Text »    PDF »
Opioid Receptor PET Reveals the Psychobiologic Correlates of Reward Processing.
M. Schreckenberger, A. Klega, G. Grunder, H.-G. Buchholz, A. Scheurich, R. Schirrmacher, E. Schirrmacher, C. Muller, G. Henriksen, and P. Bartenstein (2008)
J. Nucl. Med. 49, 1257-1261
   Abstract »    Full Text »    PDF »
Influence of Reward Delays on Responses of Dopamine Neurons.
S. Kobayashi and W. Schultz (2008)
J. Neurosci. 28, 7837-7846
   Abstract »    Full Text »    PDF »
Evidence for Segregated and Integrative Connectivity Patterns in the Human Basal Ganglia.
B. Draganski, F. Kherif, S. Kloppel, P. A. Cook, D. C. Alexander, G. J. M. Parker, R. Deichmann, J. Ashburner, and R. S. J. Frackowiak (2008)
J. Neurosci. 28, 7143-7152
   Abstract »    Full Text »    PDF »
What's in a Smile? Maternal Brain Responses to Infant Facial Cues.
L. Strathearn, J. Li, P. Fonagy, and P. R. Montague (2008)
Pediatrics 122, 40-51
   Abstract »    Full Text »    PDF »
Preferential Reactivation of Motivationally Relevant Information in the Ventral Striatum.
C. S. Lansink, P. M. Goltstein, J. V. Lankelma, R. N. J. M. A. Joosten, B. L. McNaughton, and C. M. A. Pennartz (2008)
J. Neurosci. 28, 6372-6382
   Abstract »    Full Text »    PDF »
Methylphenidate Has Differential Effects on Blood Oxygenation Level-Dependent Signal Related to Cognitive Subprocesses of Reversal Learning.
C. M. Dodds, U. Muller, L. Clark, A. van Loon, R. Cools, and T. W. Robbins (2008)
J. Neurosci. 28, 5976-5982
   Abstract »    Full Text »    PDF »
Neurocomputational mechanisms of reinforcement-guided learning in humans: A review.
M. X. COHEN (2008)
Cogn Affect Behav Neurosci 8, 113-125
   Abstract »    PDF »
Dissociating the Role of the Orbitofrontal Cortex and the Striatum in the Computation of Goal Values and Prediction Errors.
T. A. Hare, J. O'Doherty, C. F. Camerer, W. Schultz, and A. Rangel (2008)
J. Neurosci. 28, 5623-5630
   Abstract »    Full Text »    PDF »
Opposing Patterns of Signaling Activation in Dopamine D1 and D2 Receptor-Expressing Striatal Neurons in Response to Cocaine and Haloperidol.
J. Bertran-Gonzalez, C. Bosch, M. Maroteaux, M. Matamales, D. Herve, E. Valjent, and J.-A. Girault (2008)
J. Neurosci. 28, 5671-5685
   Abstract »    Full Text »    PDF »
Reward-Dependent Modulation of Neuronal Activity in the Primate Dorsal Raphe Nucleus.
K. Nakamura, M. Matsumoto, and O. Hikosaka (2008)
J. Neurosci. 28, 5331-5343
   Abstract »    Full Text »    PDF »
Disconnecting force from money: effects of basal ganglia damage on incentive motivation.
L. Schmidt, B. F. d'Arc, G. Lafargue, D. Galanaud, V. Czernecki, D. Grabli, M. Schupbach, A. Hartmann, R. Levy, B. Dubois, et al. (2008)
Brain 131, 1303-1310
   Abstract »    Full Text »    PDF »
Attaching Values to Actions: Action and Outcome Encoding in the Primate Caudate Nucleus.
C. H. Donahue and H. Seo (2008)
J. Neurosci. 28, 4579-4580
   Full Text »    PDF »
Extracellular Signal-Related Kinase Activation During Natural Reward Learning: A Physiological Role for Phasic Nucleus Accumbens Dopamine?.
J. J. Day (2008)
J. Neurosci. 28, 4295-4297
   Full Text »    PDF »
Distinct error-correcting and incidental learning of location relative to landmarks and boundaries.
C. F. Doeller and N. Burgess (2008)
PNAS 105, 5909-5914
   Abstract »    Full Text »    PDF »
Drosophila Egg-Laying Site Selection as a System to Study Simple Decision-Making Processes.
C.-h. Yang, P. Belawat, E. Hafen, L. Y. Jan, and Y.-N. Jan (2008)
Science 319, 1679-1683
   Abstract »    Full Text »    PDF »
Motor Adaptation as a Process of Reoptimization.
J. Izawa, T. Rane, O. Donchin, and R. Shadmehr (2008)
J. Neurosci. 28, 2883-2891
   Abstract »    Full Text »    PDF »
BOLD Responses Reflecting Dopaminergic Signals in the Human Ventral Tegmental Area.
K. D'Ardenne, S. M. McClure, L. E. Nystrom, and J. D. Cohen (2008)
Science 319, 1264-1267
   Abstract »    Full Text »    PDF »
Decoding of Temporal Intervals From Cortical Ensemble Activity.
M. A. Lebedev, J. E. O'Doherty, and M. A. L. Nicolelis (2008)
J Neurophysiol 99, 166-186
   Abstract »    Full Text »    PDF »
Cerebrovascular responses to incremental exercise during hypobaric hypoxia: effect of oxygenation on maximal performance.
A. W. Subudhi, M. C. Lorenz, C. S. Fulco, and R. C. Roach (2008)
Am J Physiol Heart Circ Physiol 294, H164-H171
   Abstract »    Full Text »    PDF »
Affect, Anticipation, and Adaptation: Affect-Controlled Selection of Anticipatory Simulation in Artificial Adaptive Agents.
J. Broekens, W. A. Kosters, and F. J. Verbeek (2007)
Adaptive Behavior 15, 397-422
   Abstract »    PDF »
Reinforcement Learning Signals in the Human Striatum Distinguish Learners from Nonlearners during Reward-Based Decision Making.
T. Schonberg, N. D. Daw, D. Joel, and J. P. O'Doherty (2007)
J. Neurosci. 27, 12860-12867
   Abstract »    Full Text »    PDF »
The Neural Substrate of Disappointment Revealed?.
J. T. Morra (2007)
J. Neurosci. 27, 10647-10648
   Full Text »    PDF »
Solving the Distal Reward Problem through Linkage of STDP and Dopamine Signaling.
E. M. Izhikevich (2007)
Cereb Cortex 17, 2443-2452
   Abstract »    Full Text »    PDF »
Prospection: Experiencing the Future.
D. T. Gilbert and T. D. Wilson (2007)
Science 317, 1351-1354
   Abstract »    Full Text »    PDF »
Statistics of Midbrain Dopamine Neuron Spike Trains in the Awake Primate.
H. M. Bayer, B. Lau, and P. W. Glimcher (2007)
J Neurophysiol 98, 1428-1439
   Abstract »    Full Text »    PDF »
Activation of Nigral and Pallidal Dopamine D1-Like Receptors Modulates Basal Ganglia Outflow in Monkeys.
M. A. Kliem, N. T. Maidment, L. C. Ackerson, S. Chen, Y. Smith, and T. Wichmann (2007)
J Neurophysiol 98, 1489-1500
   Abstract »    Full Text »    PDF »
Association study of four dopamine D1 receptor gene polymorphisms and clozapine treatment response.
R. Hwang, T. Shinkai, V. De Luca, Xingqun Ni, S. G. Potkin, J. A. Lieberman, H. Y. MeLtzer, and J. L. Kennedy (2007)
J Psychopharmacol 21, 718-727
   Abstract »    PDF »
Blunted response to feedback information in depressive illness.
J.D. Steele, P. Kumar, and K.P. Ebmeier (2007)
Brain 130, 2367-2374
   Abstract »    Full Text »    PDF »
Neural Antecedents of Financial Decisions.
B. Knutson and P. Bossaerts (2007)
J. Neurosci. 27, 8174-8177
   Abstract »    Full Text »    PDF »
Lateral Habenula Stimulation Inhibits Rat Midbrain Dopamine Neurons through a GABAA Receptor-Mediated Mechanism.
H. Ji and P. D. Shepard (2007)
J. Neurosci. 27, 6923-6930
   Abstract »    Full Text »    PDF »
Neural Responses to Taxation and Voluntary Giving Reveal Motives for Charitable Donations.
W. T. Harbaugh, U. Mayr, and D. R. Burghart (2007)
Science 316, 1622-1625
   Abstract »    Full Text »    PDF »
Shifting Attention Using a Temporal Difference Prediction Error and High-Dimensional Input.
W. H. Alexander (2007)
Adaptive Behavior 15, 121-133
   Abstract »    PDF »
Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration.
J. D Cohen, S. M McClure, and A. J Yu (2007)
Phil Trans R Soc B 362, 933-942
   Abstract »    Full Text »    PDF »
Neural signature of fictive learning signals in a sequential investment task.
T. Lohrenz, K. McCabe, C. F. Camerer, and P. R. Montague (2007)
PNAS 104, 9493-9498
   Abstract »    Full Text »    PDF »
Time Discounting for Primary Rewards.
S. M. McClure, K. M. Ericson, D. I. Laibson, G. Loewenstein, and J. D. Cohen (2007)
J. Neurosci. 27, 5796-5804
   Abstract »    Full Text »    PDF »
Severity of arterial hypoxaemia affects the relative contributions of peripheral muscle fatigue to exercise performance in healthy humans.
M. Amann, L. M. Romer, A. W. Subudhi, D. F. Pegelow, and J. A. Dempsey (2007)
J. Physiol. 581, 389-403
   Abstract »    Full Text »    PDF »
Differential Encoding of Losses and Gains in the Human Striatum.
B. Seymour, N. Daw, P. Dayan, T. Singer, and R. Dolan (2007)
J. Neurosci. 27, 4826-4831
   Abstract »    Full Text »    PDF »
From prediction error to psychosis: ketamine as a pharmacological model of delusions.
P.R. Corlett, G.D. Honey, and P.C. Fletcher (2007)
J Psychopharmacol 21, 238-252
   Abstract »    PDF »
Ethanol Inhibits Persistent Activity in Prefrontal Cortical Neurons.
Y. Tu, S. Kroener, K. Abernathy, C. Lapish, J. Seamans, L. J. Chandler, and J. J. Woodward (2007)
J. Neurosci. 27, 4765-4775
   Abstract »    Full Text »    PDF »
Conditioned Dopamine Release in Humans: A Positron Emission Tomography [11C]Raclopride Study with Amphetamine.
I. Boileau, A. Dagher, M. Leyton, K. Welfeld, L. Booij, M. Diksic, and C. Benkelfat (2007)
J. Neurosci. 27, 3998-4003
   Abstract »    Full Text »    PDF »
The Nucleus Accumbens and Pavlovian Reward Learning.
J. J. Day and R. M. Carelli (2007)
Neuroscientist 13, 148-159
   Abstract »    PDF »
Neural Coding of Reward-Prediction Error Signals During Classical Conditioning With Attractive Faces.
S. Bray and J. O'Doherty (2007)
J Neurophysiol 97, 3036-3045
   Abstract »    Full Text »    PDF »
Dynamics of Prefrontal and Cingulate Activity during a Reward-Based Logical Deduction Task.
C. Landmann, S. Dehaene, S. Pappata, A. Jobert, M. Bottlaender, D. Roumenov, and D. Le Bihan (2007)
Cereb Cortex 17, 749-759
   Abstract »    Full Text »    PDF »


To Advertise     Find Products

Science. ISSN 0036-8075 (print), 1095-9203 (online)