Note to users. If you're seeing this message, it means that your browser cannot find this page's style/presentation instructions -- or possibly that you are using a browser that does not support current Web standards. Find out more about why this message is appearing, and what you can do to make your experience of our site the best it can be.

Published Online November 4, 2004
Science DOI: 10.1126/science.1102941

Reports

Submitted on July 19, 2004
Accepted on October 27, 2004

By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism

Michael J. Frank 1*, Lauren C. Seeberger 2, Randall C. O'Reilly 1*

1 Department of Psychology and Center for Neuroscience, University of Colorado Boulder, Boulder, CO 80309-0345, USA.
2 Colorado Neurological Institute Movement Disorders Center, Englewood, CO 80113, USA.

* To whom correspondence should be addressed.
Michael J. Frank , E-mail: frankmj{at}psych.colorado.edu
Randall C. O'Reilly , E-mail: oreilly{at}psych.colorado.edu

To what extent do we learn from the positive versus negative outcomes of our decisions? The neuromodulator dopamine plays a key role in these reinforcement learning processes. Patients with Parkinson's disease, who have depleted dopamine in the basal ganglia, are impaired in tasks that require learning from trial and error. Here we show, using two cognitive procedural learning tasks, that Parkinson's patients off medication are better at learning to avoid choices that lead to negative outcomes than they are at learning from positive outcomes. Dopamine medication reverses this bias, making patients more sensitive to positive than negative outcomes. This pattern was predicted by our biologically-based computational model of basal ganglia/dopamine interactions in cognition, which has separate pathways for "Go" and "NoGo" responses that are differentially modulated by positive and negative reinforcement.


THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES:
Aging and the neuroeconomics of decision making: A review.
S. B. R. E. Brown and K. R. Ridderinkhof (2009)
Cogn Affect Behav Neurosci 9, 365-379
   Abstract »    PDF »
Pharmacological modulation of subliminal learning in Parkinson's and Tourette's syndromes.
S. Palminteri, M. Lebreton, Y. Worbe, D. Grabli, A. Hartmann, and M. Pessiglione (2009)
PNAS 106, 19179-19184
   Abstract »    Full Text »    PDF »
Right Ventromedial and Dorsolateral Prefrontal Cortices Mediate Adaptive Decisions under Ambiguity by Integrating Choice Utility and Outcome Evaluation.
A. Christakou, M. Brammer, V. Giampietro, and K. Rubia (2009)
J. Neurosci. 29, 11020-11028
   Abstract »    Full Text »    PDF »
Reward-learning and the novelty-seeking personality: a between- and within-subjects study of the effects of dopamine agonists on young Parkinson's patients.
N. Bodi, S. Keri, H. Nagy, A. Moustafa, C. E. Myers, N. Daw, G. Dibo, A. Takats, D. Bereczki, and M. A. Gluck (2009)
Brain 132, 2385-2395
   Abstract »    Full Text »    PDF »
Decision-Making Impairment in a Patient With New Concomitant Diagnoses of Parkinson's Disease and HIV.
M. Poletti, F. Baldacci, A. Nuti, C. Lucetti, C. Berti, and U. Bonuccelli (2009)
J Neuropsychiatry Clin Neurosci 21, 352-353
   Full Text »    PDF »
Proactive Inhibitory Control and Attractor Dynamics in Countermanding Action: A Spiking Neural Circuit Model.
C.-C. Lo, L. Boucher, M. Pare, J. D. Schall, and X.-J. Wang (2009)
J. Neurosci. 29, 9059-9071
   Abstract »    Full Text »    PDF »
Human Substantia Nigra Neurons Encode Unexpected Financial Rewards.
K. A. Zaghloul, J. A. Blanco, C. T. Weidemann, K. McGill, J. L. Jaggi, G. H. Baltuch, and M. J. Kahana (2009)
Science 323, 1496-1499
   Abstract »    Full Text »    PDF »
Striatal Dopamine Predicts Outcome-Specific Reversal Learning and Its Sensitivity to Dopaminergic Drug Administration.
R. Cools, M. J. Frank, S. E. Gibbs, A. Miyakawa, W. Jagust, and M. D'Esposito (2009)
J. Neurosci. 29, 1538-1543
   Abstract »    Full Text »    PDF »
CNTRICS Final Task Selection: Long-Term Memory.
J. D. Ragland, R. Cools, M. Frank, D. A. Pizzagalli, A. Preston, C. Ranganath, and A. D. Wagner (2009)
Schizophr Bull 35, 197-212
   Abstract »    Full Text »    PDF »
Mesolimbic Functional Magnetic Resonance Imaging Activations during Reward Anticipation Correlate with Reward-Related Ventral Striatal Dopamine Release.
B. H. Schott, L. Minuzzi, R. M. Krebs, D. Elmenhorst, M. Lang, O. H. Winz, C. I. Seidenbecher, H. H. Coenen, H.-J. Heinze, K. Zilles, et al. (2008)
J. Neurosci. 28, 14311-14319
   Abstract »    Full Text »    PDF »
The Neuromodulatory System: A Framework for Survival and Adaptive Behavior in a Challenging World.
J. L. Krichmar (2008)
Adaptive Behavior 16, 385-399
   Abstract »    PDF »
A Role for Dopamine in Temporal Decision Making and Reward Maximization in Parkinsonism.
A. A. Moustafa, M. X. Cohen, S. J. Sherman, and M. J. Frank (2008)
J. Neurosci. 28, 12294-12304
   Abstract »    Full Text »    PDF »
Habenula: Crossroad between the Basal Ganglia and the Limbic System.
O. Hikosaka, S. R. Sesack, L. Lecourtier, and P. D. Shepard (2008)
J. Neurosci. 28, 11825-11829
   Abstract »    Full Text »    PDF »
Midbrain Dopaminergic Neurons and Striatal Cholinergic Interneurons Encode the Difference between Reward and Aversive Events at Different Epochs of Probabilistic Classical Conditioning Trials.
M. Joshua, A. Adler, R. Mitelman, E. Vaadia, and H. Bergman (2008)
J. Neurosci. 28, 11673-11684
   Abstract »    Full Text »    PDF »
A Psychological and Neuroanatomical Model of Obsessive-Compulsive Disorder.
E. D. Huey, R. Zahn, F. Krueger, J. Moll, D. Kapogiannis, E. M. Wassermann, and J. Grafman (2008)
J Neuropsychiatry Clin Neurosci 20, 390-408
   Abstract »    Full Text »    PDF »
Schizophrenia: A Computational Reinforcement Learning Perspective.
M. J. Frank (2008)
Schizophr Bull 34, 1008-1011
   Full Text »    PDF »
Lesions of the Medial Striatum in Monkeys Produce Perseverative Impairments during Reversal Learning Similar to Those Produced by Lesions of the Orbitofrontal Cortex.
H. F. Clarke, T. W. Robbins, and A. C. Roberts (2008)
J. Neurosci. 28, 10972-10982
   Abstract »    Full Text »    PDF »
Acute effects of cocaine on the neurobiology of cognitive control.
H. Garavan, J. N Kaufman, and R. Hester (2008)
Phil Trans R Soc B 363, 3267-3276
   Abstract »    Full Text »    PDF »
Reward Processing in Schizophrenia: A Deficit in the Representation of Value.
J. M. Gold, J. A. Waltz, K. J. Prentice, S. E. Morris, and E. A. Heerey (2008)
Schizophr Bull 34, 835-847
   Abstract »    Full Text »    PDF »
Emotion, Motivation, and Reward Processing in Schizophrenia Spectrum Disorders: What We Know and Where We Need to Go.
D. M. Barch (2008)
Schizophr Bull 34, 816-818
   Full Text »    PDF »
Role of Dopamine in the Motivational and Cognitive Control of Behavior.
R. Cools (2008)
Neuroscientist 14, 381-395
   Abstract »    PDF »
Human Medial Frontal Cortex Activity Predicts Learning from Errors.
R. Hester, N. Barre, K. Murphy, T. J. Silk, and J. B. Mattingley (2008)
Cereb Cortex 18, 1933-1940
   Abstract »    Full Text »    PDF »
Neurocomputational mechanisms of reinforcement-guided learning in humans: A review.
M. X. COHEN (2008)
Cogn Affect Behav Neurosci 8, 113-125
   Abstract »    PDF »
The human ventromedial frontal lobe is critical for learning from negative feedback.
E. Z. Wheeler and L. K. Fellows (2008)
Brain 131, 1323-1331
   Abstract »    Full Text »    PDF »
Disconnecting force from money: effects of basal ganglia damage on incentive motivation.
L. Schmidt, B. F. d'Arc, G. Lafargue, D. Galanaud, V. Czernecki, D. Grabli, M. Schupbach, A. Hartmann, R. Levy, B. Dubois, et al. (2008)
Brain 131, 1303-1310
   Abstract »    Full Text »    PDF »
Focal basal ganglia lesions are associated with impairments in reward-based reversal learning.
C. Bellebaum, B. Koch, M. Schwarz, and I. Daum (2008)
Brain 131, 829-841
   Abstract »    Full Text »    PDF »
Genetically Determined Differences in Learning from Errors.
T. A. Klein, J. Neumann, M. Reuter, J. Hennig, D. Y. von Cramon, and M. Ullsperger (2007)
Science 318, 1642-1645
   Abstract »    Full Text »    PDF »
Hold Your Horses: Impulsivity, Deep Brain Stimulation, and Medication in Parkinsonism.
M. J. Frank, J. Samanta, A. A. Moustafa, and S. J. Sherman (2007)
Science 318, 1309-1312
   Abstract »    Full Text »    PDF »
Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning.
M. J. Frank, A. A. Moustafa, H. M. Haughey, T. Curran, and K. E. Hutchison (2007)
PNAS 104, 16311-16316
   Abstract »    Full Text »    PDF »
Towards an executive without a homunculus: computational models of the prefrontal cortex/basal ganglia system.
T. E Hazy, M. J Frank, and R. C O'Reilly (2007)
Phil Trans R Soc B 362, 1601-1613
   Abstract »    Full Text »    PDF »
Understanding decision-making deficits in neurological conditions: insights from models of natural action selection.
M. J Frank, A. Scheres, and S. J Sherman (2007)
Phil Trans R Soc B 362, 1641-1654
   Abstract »    Full Text »    PDF »
Ageing and early-stage Parkinson's disease affect separable neural mechanisms of mesolimbic reward processing.
B. H. Schott, L. Niehaus, B. C. Wittmann, H. Schutze, C. I. Seidenbecher, H.-J. Heinze, and E. Duzel (2007)
Brain 130, 2412-2424
   Abstract »    Full Text »    PDF »
Medication-Related Impulse Control and Repetitive Behaviors in Parkinson Disease.
V. Voon and S. H. Fox (2007)
Arch Neurol 64, 1089-1096
   Abstract »    Full Text »    PDF »
A Physiologically Plausible Model of Action Selection and Oscillatory Activity in the Basal Ganglia.
M. D. Humphries, R. D. Stewart, and K. N. Gurney (2006)
J. Neurosci. 26, 12921-12942
   Abstract »    Full Text »    PDF »
Dissociable systems for gain- and loss-related value predictions and errors of prediction in the human brain..
J. Yacubian, J. Glascher, K. Schroeder, T. Sommer, D. F. Braus, and C. Buchel (2006)
J. Neurosci. 26, 9530-9537
   Abstract »    Full Text »    PDF »
The Basal Ganglia in Human Learning.
C. A. Seger (2006)
Neuroscientist 12, 285-290
   Abstract »    PDF »
Perspectives on Neuroscience and Behavior.
(2005)
Neuroscientist 11, 269-270
   PDF »


To Advertise     Find Products

Science. ISSN 0036-8075 (print), 1095-9203 (online)