Note to users. If you're seeing this message, it means that your browser cannot find this page's style/presentation instructions -- or possibly that you are using a browser that does not support current Web standards. Find out more about why this message is appearing, and what you can do to make your experience of our site the best it can be.

Science 26 May 1995:
Vol. 268. no. 5214, pp. 1158 - 1161
DOI: 10.1126/science.7761831
Prev | Table of Contents | Next

Articles

Science, Vol 268, Issue 5214, 1158-1161
Copyright © 1995 by American Association for the Advancement of Science

articles

The "wake-sleep" algorithm for unsupervised neural networks

GE Hinton, P Dayan, BJ Frey, and RM Neal

Department of Computer Science, University of Toronto, Ontario, Canada.

An unsupervised learning algorithm for a multilayer network of stochastic neurons is described. Bottom-up "recognition" connections convert the input into representations in successive hidden layers, and top-down "generative" connections reconstruct the representation in one layer from the representation in the layer above. In the "wake" phase, neurons are driven by recognition connections, and generative connections are adapted to increase the probability that they would reconstruct the correct activity vector in the layer below. In the "sleep" phase, neurons are driven by generative connections, and recognition connections are adapted to increase the probability that they would produce the correct activity vector in the layer above.


THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES:
Learning to represent visual input.
G. E. Hinton (2010)
Phil Trans R Soc B 365, 177-184
   Abstract »    Full Text »    PDF »
Relevance Realization and the Emerging Framework in Cognitive Science.
J. Vervaeke, T. P. Lillicrap, and B. A. Richards (2009)
J Logic Computation
   Abstract »    PDF »
Why does the brain predict sensory consequences of oculomotor commands? Optimal integration of the predicted and the actual sensory feedback..
S. Vaziri, J. Diedrichsen, and R. Shadmehr (2006)
J. Neurosci. 26, 4188-4197
   Abstract »    Full Text »    PDF »
A theory of cortical responses.
K. Friston (2005)
Phil Trans R Soc B 360, 815-836
   Abstract »    Full Text »    PDF »
Interactions Between Membrane Conductances Underlying Thalamocortical Slow-Wave Oscillations.
A. DESTEXHE and T. J. SEJNOWSKI (2003)
Physiol Rev 83, 1401-1453
   Abstract »    Full Text »    PDF »
Cortical region interactions and the functional role of apical dendrites..
M. W. Spratling (2002)
Behav Cogn Neurosci Rev 1, 219-228
   Abstract »    PDF »
Sleep, Learning, and Dreams: Off-line Memory Reprocessing.
R. Stickgold, J. A. Hobson, R. Fosse, and M. Fosse (2001)
Science 294, 1052-1057
   Abstract »    Full Text »    PDF »
Song Replay During Sleep and Computational Rules for Sensorimotor Vocal Learning.
A. S. Dave and D. Margoliash (2000)
Science 290, 812-816
   Abstract »    Full Text »
Replay and Time Compression of Recurring Spike Sequences in the Hippocampus.
Z. Nadasdy, H. Hirase, A. Czurko, J. Csicsvari, and G. Buzsaki (1999)
J. Neurosci. 19, 9497-9507
   Abstract »    Full Text »    PDF »
A Model for Encoding Multiple Object Motions and Self-Motion in Area MST of Primate Visual Cortex.
R. S. Zemel and T. J. Sejnowski (1998)
J. Neurosci. 18, 531-547
   Abstract »    Full Text »    PDF »
Replicator Neural Networks for Universal Optimal Source Coding.
R. Hecht-Nielsen (1995)
Science 269, 1860-1863
   Abstract »    PDF »


To Advertise     Find Products

Science. ISSN 0036-8075 (print), 1095-9203 (online)