Happy or not? It all depends on the pattern of random noise over The Mona Lisa's mouth.

Making Mona Lisa Frown

Poets and songwriters will be devastated to hear it: The eyes don't smile after all. Neither do they sadden, according to new research using Mona Lisa's portrait. Those two emotions are the domain of the mouth, it turns out. Similar research could help pinpoint other emotions, or help determine the brain defects in people with visual problems such as the inability to recognize faces.

Human facial expressions are subtler than they look, and it's hard to determine the source of emotional information. One technique researchers have used is to show volunteers parts of a face and ask what emotion they see. A different technique allows expressions to "evolve" on an image, allowing researchers access to the whole expression at once.

To determine what features make people look happy or sad, visual neuroscientists Christopher Tyler and Leonid Kontsevich of the Smith-Kettlewell Eye Research Institute in San Francisco evolved expressions on The Mona Lisa. The researchers added noise to her ambiguous expression, making the image look like a fuzzy TV screen, then asked subjects to rate her emotion on a four-point scale of happy to sad. The researchers then averaged all of the noisy images in each category of emotion, revealing the subtleties in the facial features needed to spell out happiness or sadness. To determine whether the eyes and the mouth worked together to evoke emotion, the team laid the composite happy or sad noise on just the upper or lower half at a time. Overlaying the mouth half caused Mona to grin or frown, but overlaying the eyes conveyed no emotion. The researchers replicated the finding with a photograph of a woman with an ambiguous expression.

"The eyes are well-known to be the window on the soul, so we expected to see an effect of the eyes," says Tyler. Instead, the eyes might express intensity, he speculates. "Perhaps they are the window on the spirit." Visual psychophysicist Richard Murray of the University of Pennsylvania in Philadelphia is also surprised. In addition to learning about expressions, "we can use this technique on people with visual deficits to determine what's going wrong in their brain," he says.