Note to users. If you're seeing this message, it means that your browser cannot find this page's style/presentation instructions -- or possibly that you are using a browser that does not support current Web standards. Find out more about why this message is appearing, and what you can do to make your experience of our site the best it can be.

Science 21 March 2003:
Vol. 299. no. 5614, pp. 1898 - 1902
DOI: 10.1126/science.1077349
Prev | Table of Contents | Next

Reports

Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons

Christopher D. Fiorillo,* Philippe N. Tobler, Wolfram Schultz

Uncertainty is critical in the measure of information and in assessing the accuracy of predictions. It is determined by probability P, being maximal at P = 0.5 and decreasing at higher and lower probabilities. Using distinct stimuli to indicate the probability of reward, we found that the phasic activation of dopamine neurons varied monotonically across the full range of probabilities, supporting past claims that this response codes the discrepancy between predicted and actual reward. In contrast, a previously unobserved response covaried with uncertainty and consisted of a gradual increase in activity until the potential time of reward. The coding of uncertainty suggests a possible role for dopamine signals in attention-based learning and risk-taking behavior.

Institute of Physiology, University of Fribourg, CH-1700 Fribourg, Switzerland, and Department of Anatomy, University of Cambridge, Downing Street, Cambridge CB2 3DY, UK.
*   To whom correspondence should be addressed. E-mail: cdf28{at}cam.ac.uk

Read the Full Text


THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES:
Reinforcement learning, conditioning, and the brain: Successes and challenges.
T. V. Maia (2009)
Cogn Affect Behav Neurosci 9, 343-364
   Abstract »    PDF »
Aging and the neuroeconomics of decision making: A review.
S. B. R. E. Brown and K. R. Ridderinkhof (2009)
Cogn Affect Behav Neurosci 9, 365-379
   Abstract »    PDF »
At-risk for pathological gambling: imaging neural reward processing under chronic dopamine agonists.
B. Abler, R. Hahlbrock, A. Unrath, G. Gron, and J. Kassubek (2009)
Brain 132, 2396-2402
   Abstract »    Full Text »    PDF »
Single-Cell and Population Coding of Expected Reward Probability in the Orbitofrontal Cortex of the Rat.
E. van Duuren, G. van der Plasse, J. Lankelma, R. N. J. M. A. Joosten, M. G. P. Feenstra, and C. M. A. Pennartz (2009)
J. Neurosci. 29, 8965-8976
   Abstract »    Full Text »    PDF »
Functional Dissociations of Risk and Reward Processing in the Medial Prefrontal Cortex.
G. Xue, Z. Lu, I. P. Levin, J. A. Weller, X. Li, and A. Bechara (2009)
Cereb Cortex 19, 1019-1027
   Abstract »    Full Text »    PDF »
Stimulus-outcome learnability differentially activates anterior cingulate and hippocampus at feedback processing.
P. F. Rodriguez (2009)
Learn. Mem. 16, 324-331
   Abstract »    Full Text »    PDF »
The Hippocampus Codes the Uncertainty of Cue-Outcome Associations: An Intracranial Electrophysiological Study in Humans.
G. Vanni-Mercier, F. Mauguiere, J. Isnard, and J.-C. Dreher (2009)
J. Neurosci. 29, 5287-5294
   Abstract »    Full Text »    PDF »
D1/D5 Modulation of Synaptic NMDA Receptor Currents.
J. A. Varela, S. J. Hirsch, D. Chapman, L. S. Leverich, and R. W. Greene (2009)
J. Neurosci. 29, 3109-3119
   Abstract »    Full Text »    PDF »
Encoding of Probabilistic Rewarding and Aversive Events by Pallidal and Nigral Neurons.
M. Joshua, A. Adler, B. Rosin, E. Vaadia, and H. Bergman (2009)
J Neurophysiol 101, 758-772
   Abstract »    Full Text »    PDF »
Variation in dopamine genes influences responsivity of the human reward system.
J.-C. Dreher, P. Kohn, B. Kolachana, D. R. Weinberger, and K. F. Berman (2009)
PNAS 106, 617-622
   Abstract »    Full Text »    PDF »
The role of the striatum in aversive learning and aversive prediction errors.
M. R Delgado, J. Li, D. Schiller, and E. A Phelps (2008)
Phil Trans R Soc B 363, 3787-3800
   Abstract »    Full Text »    PDF »
Explicit neural signals reflecting reward uncertainty.
W. Schultz, K. Preuschoff, C. Camerer, M. Hsu, C. D Fiorillo, P. N Tobler, and P. Bossaerts (2008)
Phil Trans R Soc B 363, 3801-3811
   Abstract »    Full Text »    PDF »
Neuroethology of reward and decision making.
K. K Watson and M. L Platt (2008)
Phil Trans R Soc B 363, 3825-3835
   Abstract »    Full Text »    PDF »
Cortico-limbic-striatal circuits subserving different forms of cost-benefit decision making.
S. B. Floresco, J. R. St. Onge, S. Ghods-Sharifi, and C. A. Winstanley (2008)
Cogn Affect Behav Neurosci 8, 375-389
   Abstract »    PDF »
Midbrain Dopaminergic Neurons and Striatal Cholinergic Interneurons Encode the Difference between Reward and Aversive Events at Different Epochs of Probabilistic Classical Conditioning Trials.
M. Joshua, A. Adler, R. Mitelman, E. Vaadia, and H. Bergman (2008)
J. Neurosci. 28, 11673-11684
   Abstract »    Full Text »    PDF »
A Local Circuit Model of Learned Striatal and Dopamine Cell Responses under Probabilistic Schedules of Reward.
C. O. Tan and D. Bullock (2008)
J. Neurosci. 28, 10062-10074
   Abstract »    Full Text »    PDF »
From the Cover: Age-related changes in midbrain dopaminergic regulation of the human reward system.
J.-C. Dreher, A. Meyer-Lindenberg, P. Kohn, and K. F. Berman (2008)
PNAS 105, 15106-15111
   Abstract »    Full Text »    PDF »
The placebo treatments in neurosciences: New insights from clinical and neuroimaging studies.
N. J. Diederich and C. G. Goetz (2008)
Neurology 71, 677-684
   Abstract »    Full Text »    PDF »
Influence of Reward Delays on Responses of Dopamine Neurons.
S. Kobayashi and W. Schultz (2008)
J. Neurosci. 28, 7837-7846
   Abstract »    Full Text »    PDF »
Motor Adaptation as a Process of Reoptimization.
J. Izawa, T. Rane, O. Donchin, and R. Shadmehr (2008)
J. Neurosci. 28, 2883-2891
   Abstract »    Full Text »    PDF »
Expected Value, Reward Outcome, and Temporal Difference Error Representations in a Probabilistic Decision Task.
E. T. Rolls, C. McCabe, and J. Redoute (2008)
Cereb Cortex 18, 652-663
   Abstract »    Full Text »    PDF »
BOLD Responses Reflecting Dopaminergic Signals in the Human Ventral Tegmental Area.
K. D'Ardenne, S. M. McClure, L. E. Nystrom, and J. D. Cohen (2008)
Science 319, 1264-1267
   Abstract »    Full Text »    PDF »
Neural correlates of trust.
F. Krueger, K. McCabe, J. Moll, N. Kriegeskorte, R. Zahn, M. Strenziok, A. Heinecke, and J. Grafman (2007)
PNAS 104, 20084-20089
   Abstract »    Full Text »    PDF »
Altered Reward Processing in Women Recovered From Anorexia Nervosa.
A. Wagner, H. Aizenstein, V. K. Venkatraman, J. Fudge, J. C. May, L. Mazurkewicz, G. K. Frank, U. F. Bailer, L. Fischer, V. Nguyen, et al. (2007)
Am J Psychiatry 164, 1842-1849
   Abstract »    Full Text »    PDF »
History- and Current Instruction-Based Coding of Forthcoming Behavioral Outcomes in the Striatum.
H. Yamada, N. Matsumoto, and M. Kimura (2007)
J Neurophysiol 98, 3557-3567
   Abstract »    Full Text »    PDF »
Medication-Related Impulse Control and Repetitive Behaviors in Parkinson Disease.
V. Voon and S. H. Fox (2007)
Arch Neurol 64, 1089-1096
   Abstract »    Full Text »    PDF »
Lateral Habenula Stimulation Inhibits Rat Midbrain Dopamine Neurons through a GABAA Receptor-Mediated Mechanism.
H. Ji and P. D. Shepard (2007)
J. Neurosci. 27, 6923-6930
   Abstract »    Full Text »    PDF »
On Cotransmission & Neurotransmitter Phenotype Plasticity.
L.-E. Trudeau and R. Gutierrez (2007)
Mol. Interv. 7, 138-146
   Abstract »    Full Text »    PDF »
Processing of Temporal Unpredictability in Human and Animal Amygdala.
C. Herry, D. R. Bach, F. Esposito, F. Di Salle, W. J. Perrig, K. Scheffler, A. Luthi, and E. Seifritz (2007)
J. Neurosci. 27, 5958-5966
   Abstract »    Full Text »    PDF »
Gene gene interaction associated with neural reward sensitivity.
J. Yacubian, T. Sommer, K. Schroeder, J. Glascher, R. Kalisch, B. Leuenberger, D. F. Braus, and C. Buchel (2007)
PNAS 104, 8125-8130
   Abstract »    Full Text »    PDF »
Differential Encoding of Losses and Gains in the Human Striatum.
B. Seymour, N. Daw, P. Dayan, T. Singer, and R. Dolan (2007)
J. Neurosci. 27, 4826-4831
   Abstract »    Full Text »    PDF »
From prediction error to psychosis: ketamine as a pharmacological model of delusions.
P.R. Corlett, G.D. Honey, and P.C. Fletcher (2007)
J Psychopharmacol 21, 238-252
   Abstract »    PDF »
The Nucleus Accumbens and Pavlovian Reward Learning.
J. J. Day and R. M. Carelli (2007)
Neuroscientist 13, 148-159
   Abstract »    PDF »
Adaptation of Prefrontal Cortical Firing Patterns and Their Fidelity to Changes in Action-Reward Contingencies.
W. J. Kargo, B. Szatmary, and D. A. Nitz (2007)
J. Neurosci. 27, 3548-3559
   Abstract »    Full Text »    PDF »
Menstrual cycle phase modulates reward-related neural function in women.
J.-C. Dreher, P. J. Schmidt, P. Kohn, D. Furman, D. Rubinow, and K. F. Berman (2007)
PNAS 104, 2465-2470
   Abstract »    Full Text »    PDF »
Reward Value Coding Distinct From Risk Attitude-Related Uncertainty Coding in Human Reward Systems.
P. N. Tobler, J. P. O'Doherty, R. J. Dolan, and W. Schultz (2007)
J Neurophysiol 97, 1621-1632
   Abstract »    Full Text »    PDF »
Shaping of Motor Responses by Incentive Values through the Basal Ganglia.
B. Pasquereau, A. Nadjar, D. Arkadir, E. Bezard, M. Goillandeau, B. Bioulac, C. E. Gross, and T. Boraud (2007)
J. Neurosci. 27, 1176-1183
   Abstract »    Full Text »    PDF »
Reinforcement Learning Signals Predict Future Decisions.
M. X Cohen and C. Ranganath (2007)
J. Neurosci. 27, 371-378
   Abstract »    Full Text »    PDF »
Feeding behavior of Aplysia: A model system for comparing cellular mechanisms of classical and operant conditioning.
D. A. Baxter and J. H. Byrne (2006)
Learn. Mem. 13, 669-680
   Abstract »    Full Text »    PDF »
Modulation of Dorsolateral Prefrontal Delay Activity during Self-Organized Behavior..
E. Procyk and P. S Goldman-Rakic (2006)
J. Neurosci. 26, 11313-11323
   Abstract »    Full Text »    PDF »
Variations in the human pain stress experience mediated by ventral and dorsal Basal Ganglia dopamine activity..
D. J. Scott, M. M. Heitzeg, R. A. Koeppe, C. S. Stohler, and J.-K. Zubieta (2006)
J. Neurosci. 26, 10789-10795
   Abstract »    Full Text »    PDF »
Dissociable systems for gain- and loss-related value predictions and errors of prediction in the human brain..
J. Yacubian, J. Glascher, K. Schroeder, T. Sommer, D. F. Braus, and C. Buchel (2006)
J. Neurosci. 26, 9530-9537
   Abstract »    Full Text »    PDF »
Role of brain dopamine in food reward and reinforcement.
R. A Wise (2006)
Phil Trans R Soc B 361, 1149-1158
   Abstract »    Full Text »    PDF »
Neural Representation of Information Measure in the Primate Premotor Cortex.
K. Nakamura (2006)
J Neurophysiol 96, 478-485
   Abstract »    Full Text »    PDF »
Apathy and the Functional Anatomy of the Prefrontal Cortex-Basal Ganglia Circuits.
R. Levy and B. Dubois (2006)
Cereb Cortex 16, 916-928
   Abstract »    Full Text »    PDF »
Reward Encoding in the Monkey Anterior Cingulate Cortex.
C. Amiez, J.P. Joseph, and E. Procyk (2006)
Cereb Cortex 16, 1040-1055
   Abstract »    Full Text »    PDF »
Prospective prevalence of pathologic gambling and medication association in Parkinson disease.
V. Voon, K. Hassan, M. Zurowski, S. Duff-Canning, M. de Souza, S. Fox, A. E. Lang, and J. Miyasaki (2006)
Neurology 66, 1750-1752
   Abstract »    Full Text »    PDF »
Higher-dimensional neurons explain the tuning and dynamics of working memory cells..
R. Singh and C. Eliasmith (2006)
J. Neurosci. 26, 3667-3678
   Abstract »    Full Text »    PDF »
Different roles for amygdala central nucleus and substantia innominata in the surprise-induced enhancement of learning..
P. C. Holland and M. Gallagher (2006)
J. Neurosci. 26, 3791-3797
   Abstract »    Full Text »    PDF »
Neural Coding of Distinct Statistical Properties of Reward Information in Humans.
J.-C. Dreher, P. Kohn, and K. F. Berman (2006)
Cereb Cortex 16, 561-573
   Abstract »    Full Text »    PDF »
Individual differences in dopamine efflux in nucleus accumbens shell and core during instrumental learning..
J. Cheng and M. G.P. Feenstra (2006)
Learn. Mem. 13, 168-177
   Abstract »    Full Text »    PDF »
Neuronal Activity in Primate Orbitofrontal Cortex Reflects the Value of Time.
M. R. Roesch and C. R. Olson (2005)
J Neurophysiol 94, 2457-2471
   Abstract »    Full Text »    PDF »
The Role of Ventral Frontostriatal Circuitry in Reward-Based Learning in Humans.
A. Galvan, T. A. Hare, M. Davidson, J. Spicer, G. Glover, and B. J. Casey (2005)
J. Neurosci. 25, 8650-8656
   Abstract »    Full Text »    PDF »
The Primate Thalamus Is a Key Target for Brain Dopamine.
M. A. Sanchez-Gonzalez, M. A. Garcia-Cabezas, B. Rico, and C. Cavada (2005)
J. Neurosci. 25, 6076-6083
   Abstract »    Full Text »    PDF »
Dopamine Cells Respond to Predicted Events during Classical Conditioning: Evidence for Eligibility Traces in the Reward-Learning Network.
W.-X. Pan, R. Schmidt, J. R. Wickens, and B. I. Hyland (2005)
J. Neurosci. 25, 6235-6242
   Abstract »    Full Text »    PDF »
Electrophysiological correlates of reward prediction error recorded in the human prefrontal cortex.
H. Oya, R. Adolphs, H. Kawasaki, A. Bechara, A. Damasio, and M. A. Howard III (2005)
PNAS 102, 8351-8356
   Abstract »    Full Text »    PDF »
Pedunculopontine Tegmental Nucleus Controls Conditioned Responses of Midbrain Dopamine Neurons in Behaving Rats.
W.-X. Pan and B. I. Hyland (2005)
J. Neurosci. 25, 4725-4732
   Abstract »    Full Text »    PDF »
Distributed Neural Representation of Expected Value.
B. Knutson, J. Taylor, M. Kaufman, R. Peterson, and G. Glover (2005)
J. Neurosci. 25, 4806-4812
   Abstract »    Full Text »    PDF »
Reinforcement in an in vitro analog of appetitive classical conditioning of feeding behavior in Aplysia: Blockade by a dopamine antagonist.
F. D. Reyes, R. Mozzachiodi, D. A. Baxter, and J. H. Byrne (2005)
Learn. Mem. 12, 216-220
   Abstract »    Full Text »    PDF »
Decisions under Uncertainty: Probabilistic Context Influences Activation of Prefrontal and Parietal Cortices.
S. A. Huettel, A. W. Song, and G. McCarthy (2005)
J. Neurosci. 25, 3304-3311
   Abstract »    Full Text »    PDF »
The Roles of the Caudate Nucleus in Human Classification Learning.
C. A. Seger and C. M. Cincotta (2005)
J. Neurosci. 25, 2941-2951
   Abstract »    Full Text »    PDF »
Adaptive Coding of Reward Value by Dopamine Neurons.
P. N. Tobler, C. D. Fiorillo, and W. Schultz (2005)
Science 307, 1642-1645
   Abstract »    Full Text »    PDF »
How Visual Stimuli Activate Dopaminergic Neurons at Short Latency.
E. Dommett, V. Coizet, C. D. Blaha, J. Martindale, V. Lefebvre, N. Walton, J. E. W. Mayhew, P. G. Overton, and P. Redgrave (2005)
Science 307, 1476-1479
   Abstract »    Full Text »    PDF »
Noradrenaline and Dopamine Elevations in the Rat Prefrontal Cortex in Spatial Working Memory.
Z. L. Rossetti and S. Carboni (2005)
J. Neurosci. 25, 2322-2329
   Abstract »    Full Text »    PDF »
Disconnection and Cognitive Dysmetria in Schizophrenia.
K. Friston (2005)
Am J Psychiatry 162, 429-432
   Full Text »    PDF »
Learned Predictions of Error Likelihood in the Anterior Cingulate Cortex.
J. W. Brown and T. S. Braver (2005)
Science 307, 1118-1121
   Abstract »    Full Text »    PDF »
Independent Coding of Movement Direction and Reward Prediction by Single Pallidal Neurons.
D. Arkadir, G. Morris, E. Vaadia, and H. Bergman (2004)
J. Neurosci. 24, 10047-10056
   Abstract »    Full Text »    PDF »
Correlation Between Dopamine D2 Receptors in the Ventral Striatum and Central Processing of Alcohol Cues and Craving.
A. Heinz, T. Siessmeier, J. Wrase, D. Hermann, S. Klein, S. M. Grusser-Sinopoli, H. Flor, D. F. Braus, H. G. Buchholz, G. Grunder, et al. (2004)
Am J Psychiatry 161, 1783-1789
   Abstract »    Full Text »    PDF »
Motivation-dependent Responses in the Human Caudate Nucleus.
M.R. Delgado, V.A. Stenger, and J.A. Fiez (2004)
Cereb Cortex 14, 1022-1030
   Abstract »    Full Text »    PDF »
Human Midbrain Sensitivity to Cognitive Feedback and Uncertainty During Classification Learning.
A. R. Aron, D. Shohamy, J. Clark, C. Myers, M. A. Gluck, and R. A. Poldrack (2004)
J Neurophysiol 92, 1144-1152
   Abstract »    Full Text »    PDF »
Presynaptic mechanisms of motor fluctuations in Parkinson's disease: a probabilistic model.
R. de la Fuente-Fernandez, M. Schulzer, E. Mak, D. B. Calne, and A. J. Stoessl (2004)
Brain 127, 888-899
   Abstract »    Full Text »    PDF »
Dopamine Neurons Mediate a Fast Excitatory Signal via Their Glutamatergic Synapses.
N. Chuhma, H. Zhang, J. Masson, X. Zhuang, D. Sulzer, R. Hen, and S. Rayport (2004)
J. Neurosci. 24, 972-981
   Abstract »    Full Text »    PDF »
Magnitude of Dopamine Release in Medial Prefrontal Cortex Predicts Accuracy of Memory on a Delayed Response Task.
A. G. Phillips, S. Ahn, and S. B. Floresco (2004)
J. Neurosci. 24, 547-553
   Abstract »    Full Text »    PDF »
Coding of Predicted Reward Omission by Dopamine Neurons in a Conditioned Inhibition Paradigm.
P. N. Tobler, A. Dickinson, and W. Schultz (2003)
J. Neurosci. 23, 10402-10410
   Abstract »    Full Text »    PDF »
Contrasting Neuronal Activity in the Supplementary and Frontal Eye Fields During Temporal Organization of Multiple Saccades.
M. Isoda and J. Tanji (2003)
J Neurophysiol 90, 3054-3065
   Abstract »    Full Text »    PDF »
Correlated Coding of Motivation and Outcome of Decision by Dopamine Neurons.
T. Satoh, S. Nakai, T. Sato, and M. Kimura (2003)
J. Neurosci. 23, 9913-9923
   Abstract »    Full Text »    PDF »
Hyperdopaminergic Mutant Mice Have Higher "Wanting" But Not "Liking" for Sweet Rewards.
S. Pecina, B. Cagniard, K. C. Berridge, J. W. Aldridge, and X. Zhuang (2003)
J. Neurosci. 23, 9395-9402
   Abstract »    Full Text »    PDF »


To Advertise     Find Products

Science. ISSN 0036-8075 (print), 1095-9203 (online)