Social Neuroscience Key Readings

SOCIAL NEUROSCIENCE
Key Readings in Social Psychology
General Editor: ARIE W. KRUGLANSKI, University of Maryland at College Park
The aim of this series is to make available to senior undergraduate and graduate students key articles in
each area of social psychology in an attractive, user-friendly format. Many professors want to encourage
their students to engage directly with research in their fields, yet this can often be daunting for students
coming to detailed study of a topic for the first time. Moreover, declining library budgets mean that articles
are not always readily available, and course packs can be expensive and time-consuming to produce. Key
Readings in Social Psychology aims to address this need by providing comprehensive volumes, each one
of which will be edited by a senior and active researcher in the field. Articles will be carefully chosen to
illustrate the way the field has developed historically as well as current issues and research directions. Each
volume will have a similar structure to include:
• an overview chapter, as well as introduction to sections and articles

• questions for class discussion
• annotated bibliographies
• full author and subject indexes
Published Titles
The Self in Social Psychology Roy F. Baumeister
Stereotypes and Prejudice Charles Stangor
Motivational Science E. Tory Higgins and Arie W. Kruglanski
Social Psychology and Human Sexuality Roy F. Baumeister
Emotions in Social Psychology W. Gerrod Parrott
Intergroup Relations Michael A. Hogg and Dominic Abrams
The Social Psychology of Organizational Behavior Leigh L. Thompson
Social Psychology: A General Reader Arie W. Kruglanski and E. Tory Higgins
Social Psychology of Health Peter Salovey and Alexander J. Rothman
The Interface of Social and Clinical Psychology Robin M. Kowalski and Mark R. Leary
Political Psychology John T. Jost and James Sidanius
Close Relationships Harry T. Reis and Caryl E. Rusbult
Titles in Preparation
Attitudes Richard E. Petty and Russell Fazio
Group Processes John Levine and Richard Moreland
Language and Communication Gün R. Semin
Persuasion Richard E. Petty and Russell Fazio
Social Cognition David L. Hamilton
Social Comparison Diederik Stapel and Hart Blanton
Social Neuroscience John T. Cacioppo and Gary Berntson
For continually updated information about published and forthcoming titles in the Key Readings in Social Psychology
series, please visit: www.keyreadings.com
SOCIAL NEUROSCIENCE
Key Readings
Edited by
John T. Cacioppo
University of Chicago
Gary G. Berntson
Ohio State University
Psychology Press
New York and Hove
RT0996_C00.qxd 11/8/04 1:56 PM Page iv
Published in 2005 by
Psychology Press
270 Madison Avenue
New York, NY 10016
www.psypress.com
Published in Great Britain by

Psychology Press
27 Church Road
Hove, East Sussex
BN3 2FA
www.psypress.co.uk
Copyright © 2005 by Taylor & Francis Books, Inc.
This edition published in the Taylor & Francis e-Library, 2005.

“To purchase your own copy of this or any of Taylor & Francis or Routledge’s
collection of thousands of eBooks please go to www.eBookstore.tandf.co.uk.”
Psychology Press, is an imprint of the Taylor & Francis Group.
All rights reserved. No part of this book may be reprinted or reproduced or utilized in any form or by any
electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording,
or in any information storage or retrieval system, without permission in writing from the publishers.
Library of Congress Cataloging-in-Publication Data
Social neuroscience: key readings/edited by John T. Cacioppo and Gary G. Berntson.

p. cm. — (Key readings in social psychology)
Includes bibliographical references and index.
ISBN 1-84169-098-8 (hardback: alk. paper) — ISBN 1-84169-099-6 (pbk.: alk. paper)
1. Neuropsychology. 2. Social psychology. I. Cacioppo, John T. II. Berntson, Gary G. III. Series.
QP360.S595 2004
153—dc22
2004009959
ISBN 0-203-49619-1 Master e-book ISBN
ISBN 0-203-59512-2 (Adobe eReader Format)

RT0996_C00.qxd 11/8/04 1:56 PM Page v
Contents
•
About the Editors ix

Acknowledgments x
Preface xiii
P A R T 1
Volume Overview: Analyses of the Social Brain through the Lens
of Human Brain Imaging 1
John T. Cacioppo & Gary G. Berntson
P A R T 2
The Brain Determines Social Behavior: The Story of
Phineas Gage 19
READING 1
The Return of Phineas Gage: Clues about the Brain from the Skull
of a Famous Patient 21
Hanna Damasio, Thomas Grabowski, Randall Frank, Albert M. Galaburda & Antonio
R. Damasio
READING 2
Impairment of Social and Moral Behavior Related to Early
Damage in Human Prefrontal Cortex 29
Steven W. Anderson, Antoine Bechara, Hanna Damasio, Daniel Tranel & Antonio R. Damasio
v
RT0996_C00.qxd 11/8/04 1:56 PM Page vi
vi ■ Contents
P A R T 3
Dissociable Systems for Attention, Emotion, and Social
Knowledge 41
READING 3
Dissociable Prefrontal Brain Systems for Attention and
Emotion 43
Hiroshi Yamasaki, Kevin S. LaBar & Gregory McCarthy
READING 4
Distinct Neural Systems Subserve Person and Object
Knowledge 53
Jason P. Mitchell, Todd F. Heatherton & C. Neil Macrae
READING 5
Functional Networks in Emotional Moral and Nonmoral Social
Judgments 63
Jorge Moll, Ricardo de Oliveira-Souza, Ivanei E. Bramati & Jordan Grafman
P A R T 4
Dissociable Systems for Face and Object Processing 73
READING 6
Stages of Processing in Face Perception: An MEG Study 75
Jia Liu, Alison Harris & Nancy Kanwisher
READING 7
Distributed and Overlapping Representations of Faces and Objects
in Ventral Temporal Cortex 87
James V. Haxby, M. Ida Gobbini, Maura L. Furey, Alumit Ishai, Jennifer L. Schouten & Pietro
Pietrini
P A R T 5
Dissociable Systems for the Perception of Biological
Movement 97
READING 8
Brain Areas Active during Visual Perception of Biological
Motion 101
Emily D. Grossman & Randolph Blake
RT0996_C00.qxd 11/8/04 1:56 PM Page vii
Contents ■ vii
READING 9
Electrophysiology and Brain Imaging of Biological Motion 115
Aina Puce & David Perrett
P A R T 6
Biological Movement: From Perception to Imitation to
Emotion 131
READING 10
Action Observation Activates Premotor and Parietal Areas in a
Somatotopic Manner: An fMRI Study 133
G. Buccino, F. Binkofski, G.R. Fink, L. Fadiga, L. Fogassi, V. Gallese, R.J. Seitz, K. Zilles,
G. Rizzolatti & H.-J. Freund
READING 11
Neural Mechanisms of Empathy in Humans: A Relay from Neural
Systems for lmitation to Limbic Areas 143
Laurie Carr, Marco Iacoboni, Marie-Charlotte Dubeau, John C. Mazziotta & Gian Luigi Lenzi
P A R T 7
Animacy, Causality, and Theory of Mind 153
READING 12
Movement and Mind: A Functional Imaging Study of Perception and
Interpretation of Complex Intentional Movement Patterns 155
Fulvia Castelli, Francesca Happé, Uta Frith & Chris Frith
READING 13
People Thinking about Thinking People: The Role of the
Temporo-Parietal Junction in “Theory of Mind” 171
R. Saxe & N. Kanwisher
P A R T 8
Social Perception and Cognition: Multiple Routes 183
READING 14
Neural Correlates of the Automatic Processing of Threat Facial
Signals 185
Adam K. Anderson, Kalina Christoff, David Panitz, Eve De Rosa & John D.E. Gabrieli
RT0996_C00.qxd 11/8/04 1:56 PM Page viii
viii ■ Contents
READING 15
Automatic and Intentional Brain Responses during Evaluation of
Trustworthiness of Faces 199
J.S. Winston, B.A. Strange, J. O’Doherty & R.J. Dolan
P A R T 9
Decision-Making 211
READING 16
The Neural Basis of Economic Decision-Making in the Ultimatum
Game 215
Alan G. Sanfey, James K. Rilling, Jessica A. Aronson, Leigh E. Nystrom & Jonathan
D. Cohen
READING 17
Exploring the Neurological Substrate of Emotional and Social
Intelligence 223
Reuven Bar-On, Daniel Tranel, Natalie L. Denburg & Antoine Bechara
P A R T 1 0
Biological Does Not Mean Predetermined: Reciprocal Influences of
Social and Biological Processes 239
READING 18
Social Dominance in Monkeys: Dopamine D2 Receptors and
Cocaine Self-Administration 243
Drake Morgan, Kathleen A. Grant, H. Donald Gage, Robert H. Mach, Jay R. Kaplan, Osric
Prioleau, Susan H. Nader, Nancy Buchheimer, Richard L. Ehrenkaufer & Michael A. Nader
READING 19
Rethinking Feelings: An fMRI Study of the Cognitive Regulation of
Emotion 253
Kevin N. Ochsner, Silvia A. Bunge, James J. Gross & John D.E. Gabrieli
Appendix: How to Read a Journal Article in Social

Psychology 271
Christian H. Jordan and Mark P. Zanna
Author Index 281

Subject Index 289
RT0996_C00.qxd 11/8/04 1:56 PM Page ix
About the Editors

•
John T. Cacioppo is the Tiffany and Margaret Blake Distinguished

Service Professor of psychology at the University of Chicago. He is the
author of over 270 articles, and among his books are Attitudes and
Persuasion, Emotional Contagion, Principles of Psychophysiology,
Foundations in Social Neuroscience, and Handbook of Psychophysiology.
He is a recipient of the National Academy of Sciences Troland Research
Award, the Society for Personality and Social Psychology’s Campbell
Award, the Society for Psychophysiological Research’s Award for
Distinguished Scientific Contributions to Psychophysiology, and the
American Psychological Association’s Distinguished Scientific
Contribution Award.
Gary G. Berntson is a professor of psychology, psychiatry, and

pediatrics, as well as a member of the Neuroscience Graduate Faculty at
the Ohio State University. His research is highly interdisciplinary and
ranges from clinical studies in humans to neurophysiological
investigations in animals.
ix
RT0996_C00.qxd 11/8/04 1:56 PM Page x
Acknowledgments
•
The authors and publishers are grateful to the Emotional Moral and Nonmoral Social Judg-
following for permission to reproduce the articles ments. NeuroImage, 16 (3-1), 696–703. Copy-
in this book: right © 2002 by Elsevier. Reprinted with permis-
sion.
Reading 1: H. Damasio, T. Grabowski, R. Frank, Reading 6: J. Liu, A. Harris, and N. Kanwisher,
A. M. Galaburda, and A. R. Damasio, The Return Stages of Processing in Face Perception: An MEG
of Phineas Gage: Clues about the Brain from Study. Nature Neuroscience, 5 (9), 910–916.
the Skull of a Famous Patient. Science, New Copyright © 2002 by Nature Publishing Group.
Series, 264, 1102–1105. Copyright © 1994 by the Reprinted with permission.
American Association for the Advancement of Reading 7: J. V. Haxby, M. I. Gobbini, M. L.
Science. Reprinted with permission. Furey, A. Ishai, J. L. Schouten, and P. Pietrini,
Reading 2: S. W. Anderson, A. Bechara, H. Distributed and Overlapping Representations of
Damasio, D. Tranel, and A. R. Damasio, Impair- Faces and Objects in Ventral Temporal Cortex.
ment of Social and Moral Behavior Related to Science, 293, 2425–2430. Copyright © 2001 by
Early Damage in Human Prefrontal Cortex. Nature the American Association for the Advancement of
Neuroscience, 2 (11), 1032–1037. Copyright © Science. Reprinted with permission.
1999 by Nature Publishing Group. Reprinted with Reading 8: E. D. Grossman and R. Blake, Brain
permission. Areas Active during Visual Perception of Biolog-
Reading 3: H. Yamasaki, K. S. LaBar, and ical Motion. Neuron, 35, 1167–1175. Copyright
G. McCarthy, Dissociable Prefrontal Brain Sys- © 2002 by Elsevier. Reprinted with permission.
tems for Attention and Emotion. Proceedings of Reading 9: A. Puce and D. Perrett, Electrophysi-
the National Academy of Sciences, 99 (17), ology and Brain Imaging of Biological Motion.
11447–11451. Copyright © 2002 by the National Philosophical Transactions of the Royal Society
Academy of Sciences. Reprinted with permission. of London Series B-Biological Sciences, 358,
Reading 4: J. P. Mitchell, T. F. Heatherton, and 435–445. Copyright © 2003 by the Royal Society
C. N. Macrae, Distinct Neural Systems Subserve of London. Reprinted with permission.
Person and Object Knowledge. Proceedings of Reading 10: G. Buccino, F. Binkofski, G. R.
the National Academy of Sciences, 99 (23), Fink, L. Fadiga, L. Fogassi, V. Gallese, R. J. Seitz,
15238–15243. Copyright © 2002 by the National K. Zilles, G. Rizzolatti, and H. J. Freund, Action
Academy of Sciences. Reprinted with permission. Observation Activates Premotor and Parietal
Reading 5: J. Moll, R. de Oliveira-Souza, I. E., Areas in a Somatotopic Manner: An fMRI Study.
Bramati, and J. Grafman, Functional Networks in European Journal of Neuroscience, 13 (2),
x
RT0996_C00.qxd 11/8/04 1:56 PM Page xi
Acknowledgments ■ xi
400–404. Copyright © 2001 by Blackwell Pub- Trustworthiness of Faces. Nature Neuroscience,

lishing. Reprinted with permission. 5 (3), 277–283. Copyright © 2002 by Nature Pub-
Reading 11: L. Carr, M. Iacoboni, M. Dubeau, lishing Group. Reprinted with permission.
J. C. Mazziotta, and G. L. Lenzi, Neural Mecha- Reading 16: A. G. Sanfey, J. K. Rilling, J. A.
nisms of Empathy in Humans: A Relay from Aronson, L. E. Nystrom, and J. D. Cohen, The
Neural Systems for Imitation to Limbic Areas. Neural Basis of Economic Decision-Making in
Proceedings of the National Academy of Sciences, the Ultimatum Game. Science, 300, 1755–1758.
100 (9), 5497–5502. Copyright © 2003 by the Copyright © 2003 by the American Association
National Academy of Sciences. Reprinted with for the Advancement of Science. Reprinted with
permission. permission.
Reading 12: F. Castelli, F. Happé, U. Frith, and Reading 17: R. Bar-On, D. Tranel, N. L. Denburg,
C. Frith, Movement and Mind: A Functional and A. Bechara, Exploring the Neurological Sub-
Imaging Study of Perception and Interpretation of strate of Emotional and Social Intelligence. Brain,
Complex Intentional Movement Patterns. Neu- 126, 1790–1800. Copyright © 2003 by Oxford
roImage, 12 (3), 314–325. Copyright © 2000 by University Press. Reprinted with permission.
Elsevier. Reprinted with permission.
Reading 18: D. Morgan, K. A. Grant, H. D. Gage,
Reading 13: R. Saxe and N. Kanwisher, People R. H. Mach, J. R. Kaplan, O. Prioleau, S. H.
Thinking about Thinking People: The Role of the Nader, N. Buchheimer, R. L. Ehrenkaufer, and M. A.
Temporo-Parietal Junction in “Theory of Mind.” Nader, Social Dominance in Monkeys: Dopamine
NeuroImage, 19 (4), 1835–1842. Copyright © D2 Receptors and Cocaine Self-Administration.
2003 by Elsevier. Reprinted with permission. Nature Neuroscience, 5 (2), 169–174. Copyright
Reading 14: A. K. Anderson, K. Christoff, D. © 2002 by Nature Publishing Group. Reprinted
Panitz, E. De Rosa, and J. D. E. Gabrieli, Neural with permission.
Correlates of the Automatic Processing of Threat Reading 19: K. N. Ochsner, S. A. Bunge, J. J.
Facial Signals. The Journal of Neuroscience, 23, Gross, and J. D. E. Gabrieli, Rethinking Feelings:
5627–5633. Copyright © 2003 by the Society for An fMRI Study of the Cognitive Regulation of
Neuroscience. Reprinted with permission. Emotion. Journal of Cognitive Neuroscience,
Reading 15: J. S. Winston, B. A. Strange, J. O. 14 (8), 1215–1229. Copyright © 2002 by the
O’Doherty, and R. J. Dolan, Automatic and Inten- Massachusetts Institute of Technology. Reprinted
tional Brain Responses during Evaluation of with permission.
RT0996_C00.qxd 11/8/04 1:56 PM Page xii
RT0996_C00.qxd 11/8/04 1:56 PM Page xiii
Preface
•
Neuroscientists and cognitive scientists have col- mortality. One is as likely to find relevant animal
laborated for more than a decade with the com- and computational models as human studies, and
mon goal of understanding how the mind works. the methodologies—which support measure-
These collaborations have helped unravel puzzles ments ranging from the gene to cultures—are
of the mind including aspects of perception, even more diverse. Attempting to canvas the full
imagery, attention, and memory. Many aspects of scope of social neuroscience would deprive the
the mind, however, require a more comprehensive reader of the rich depth and coherence that can
approach to reveal the mystery of mind-brain con- come from multilevel analyses of some of the
nections. The topics of attraction, altruism, ag- most fascinating questions humanity has asked
gression, affiliation, attachment, and attitudes rep- about itself and the human mind. We, therefore,
resent a small sampler from the top of the limited our scope to human studies of a single
alphabet alone. Social neuroscience, therefore, question in one area—“Is there anything special
has emerged to address fundamental questions about social” cognition?”—although we also
about the mind and its dynamic interactions with included several studies of brain lesions where
the biological systems of the brain and body and appropriate.
the social world in which it resides. It is con- The readings are organized to reveal a series of
cerned with the relationship between neural and facets of what might be special about social in-
social processes, including the intervening infor- formation processing. Following an overview,
mation processing components and operations at readings in the next three sections show that the
both the neural and the computational levels of brain determines social behavior, that there are
analysis. As such, work in social neuroscience dissociable systems in the brain for social and
builds on work in the neurosciences, cognitive nonsocial information processing, and that there
sciences, and social sciences. The premise under- may be dissociable systems in the brain for face
lying this book is that more complex aspects of and object processing. In the next two sections,
the mind and behavior will benefit from yet a readings demonstrate that there are dissociable
broader collaboration of neuroscientists, cogni- systems in the brain for the perception of biologi-
tive scientists, and social scientists. cal and nonbiological movement, and that the
The field of social neuroscience encompasses perception of biological movement can automati-
studies ranging from social cognition, motiva- cally elicit imitation (mimicry) and empathy.
tion, and emotion to interpersonal and group Readings in the next three sections address
processes, and to social influences on health and possible means by which the brain constructs
xiii
RT0996_C00.qxd 11/8/04 1:56 PM Page xiv
xiv ■ Preface
inferences about the mental states of others, social, cognitive, and neurosciences for many
utilizes information about others from multiple years; the support of Paul Dukes at Psychology
parallel streams of information processing, and Press; and the assistance of Judy Runge in the
ultimately formulates preferences and decisions preparation of the manuscript. We hope the
to negotiate the social world. We end with two reader will enjoy getting to know the field of
readings to illustrate that research showing social social neuroscience and its potential in the
cognition is instantiated in biological (e.g., brain) century ahead.
processes does not mean that it is fixed and
immutable by an individual’s genotype. John T. Cacioppo
Work on this volume was made possible by University of Chicago
funding from the National Science Foundation
BCS-0086314, an institution that has tirelessly Gary G. Berntson
supported efforts to bridge the abyss across the The Ohio State University
RT0996_P01.qxd 11/8/04 2:13 PM Page 1
PA R T 1
Volume Overview:
Analyses of the Social Brain through
the Lens of Human Brain Imaging
•
John T. Cacioppo • University of Chicago
Gary G. Berntson • The Ohio State University
Among the major evolutionary advances in humans is the striking

development of the human cerebral cortex, especially the frontal and
temporal regions. The cerebral cortex is a mantle of between 2.6 to
16 billion neurons with each neuron receiving 10,000 to 100,000 synapses
in their dendritic trees (e.g., Pakkenberg, 1966). The frontal and temporal
lobes constitute 32% and 23% of the cerebral cortex, respectively,
rendering the sensorimotor cortices that dominate most mammalian brains
to minority status in the human brain. The expansion of the frontal regions
in the human brain is largely responsible for the human capacity for
reasoning, planning, and performing mental simulations, and an intact
frontal region contributes to the human ability to reason, remember, and
work together. The temporal regions, in turn, play essential roles in social
perception and communication. The neocortex, in particular, is a recent
development in evolutionary time, and the means for guiding behavior
through the environment, albeit in a more rigid and stereotyped fashion,
emerged prior to neocortical expansion. These evolutionarily older systems
likely also play a role in human information processing and behavior
(Berntson, Boysen, & Cacioppo, 1993; Smith et al., 2003).
1
RT0996_P01.qxd 11/8/04 2:13 PM Page 2
2 ■ Social Neuroscience: Key Readings
Human information processing capacities remain their strengths and underestimate their faults (Ross
woefully insufficient even with the expansion of the & Sicoly, 1979); they overestimate how important is
cortices, however. The sensory load from the their input, how pervasive are their beliefs, and how
physical environment is minor in comparison to the likely is a desired event to occur (McGuire, 1981),
quantity and complexity of the information that all while underestimating the contributions of others
comes from other individuals, groups, alliances, and the likelihood that risks in the world apply to
and cultures—as well as the potential for them (Ross, Greene, & House, 1977). Events that
benevolence or treachery from each. It is perhaps unfold unexpectedly are not reasoned as much as
understandable why social cognition is not an they are rationalized (Aronson, 1968), and the act of
objective information process but instead is rife remembering is far more of a biased reconstruction
with the operation of self-interest, self-enhancement, than an accurate record of actual events (Roediger,
and self-protective processes (Cacioppo & Buckner, & McDermott, 1999; McDonald & Hirt,
Berntson, 2004). For instance, because there is 1997). Mental simulation of alternative outcomes—
more information than can possibly be processed, made possible through the expansion of the frontal
people tend to search for and attend to evidence cortices—can dramatically influence our
that confirms what they already believe to be true evaluations and decisions, so much so that the
(Snyder & Swann, 1978). Information processing is counterfactual reasoning of silver medalists in the
biased in ways that protects the self from symbolic Olympic Games leave them, on average, less happy
as well as actual threats and promotes reproductive with their achievements than bronze medalists
success (e.g., Jones & Berglas, 1978). People (Medvec, Madey, & Gilovich, 1995). These biases in
believe they know how long things they choose will social cognition are automatic in the sense that they
make them feel good or bad, but in fact they grossly are spontaneous, their implementation does not
overestimate the duration of these feelings (Wilson require cognitive effort, and they are ubiquitous in
et al., 2000). Subtle reminders of their mortality can that they represent normative processing. They
push people to promote defensive information represent emergent properties from the operation of
processing, such as blaming the victim and the human brain, sculpted by adaptive and
enacting risky behaviors, as if to prove to reproductive successes and failures (Cacioppo &
themselves that the world is just and threats do not Berntson, 2004).
apply to them (Pyszczynski, Greenberg, & Solomon, During most of the past century, the nuances of
1999). In fact, people are not particularly good at social cognition and social processes, including
knowing the causes of their feelings or behavior unconscious processes, were plumbed through
(Nisbett & Wilson, 1977). They believe they know the clever experimental designs and measures of
that opposites attract, just as assuredly as they verbal reports, judgments, and reaction time.
know that its logical opposite is also true (i.e., birds These methodologies were limited in what they
of a feather flock together). People overestimate could reveal about social processes, however.
RT0996_P01.qxd 11/8/04 2:13 PM Page 3
Part 1: Volume Overview ■ 3
Social cognition and behavior are often affect- electrophysiological images of brain activity may
laden or habitual, and nuances deriving from these provide moment by moment information about
features proved difficult to capture fully using when and where social cognition is unfolding in
subjective measures and response latencies to the brain, and from this one can infer what is the
semantic (e.g., lexical decision) tasks (Zajonc, nature of the information processing operation
1980). In a recent Annual Review of being performed moment by moment in the brain.
Neurosciences, LeDoux observed: Parallel series of information processing
It is widely recognized that most cognitive operations, in theory, are as tractable as a single
processes occur unconsciously, with only the end
series of information processing operations—
products reaching awareness, and then only
sometimes. Emotion researchers, though, did not making it possible to examine the brain as it does
make this conceptual leap. They remained focused
on subjective emotional experience . . . The main two (or more) things at the same time.
lesson to be learned . . . is that emotion re- The field of social neuroscience stretches far
searchers need to figure out how to escape from
the shackles of subjectivity if emotion research is beyond social cognition, human studies, or brain
to thrive. (LeDoux, 2000, p.156)
imaging methodologies (see Cacioppo et al.,
The same is true in social psychology (Berntson 2002). Rather than canvassing the full scope of
& Cacioppo, 2000; Cacioppo & Berntson, 1992, social neuroscience, however, we selected the
2004). The study of automatic and implicit current readings to illustrate work within a limited
processes became popular in social psychology area that is shaping scientific thought about a
in the 1990s, but as important as these classic question in social psychology: Is there
developments were, the richness of the anything special about “social” cognition? We next
information provided remained quite limited. The review illustrative evidence bearing on this
purpose of this book is to introduce students to a question to provide a context for the readings that
new and developing approach to address these follow.
questions—an approach termed social
neuroscience (Cacioppo & Berntson, 1992). Social Processes and Behavior:
In social neuroscience, theory and methods in
Multipurpose or Specialized
Neural Mechanisms
the neurosciences constrain and inspire
hypotheses in the behavioral and social sciences, One of the fundamental questions in the field is
foster experimental tests of otherwise whether specific social constructs, processes, and
indistinguishable theoretical explanations, and representations have a definable neural locus. Is
increase the comprehensiveness and relevance of social information processed by specialized neural
social and behavioral theories. Whereas a measure circuitry? Two complementing strategies are
of reaction time may provide information about typically used to address this question: (1) one can
differences in the time it takes to perform specific examine the function or functions associated with a
mental operations, metabolic and particular neural locus or region to determine if it
RT0996_P01.qxd 11/8/04 2:13 PM Page 4
exclusively serves social information processing, or basal ganglia, hypothalamus, and periaqueductal
(2) one can place the focus on a specific social gray.
process or function to examine the different ways in Disputes have emerged, however, regarding
which this function might be achieved. whether certain nuclei (e.g., fusiform gyrus) function
In a seminal thesis based primarily on to process social information (e.g., faces; Kanwisher,
neurophysiological recordings in nonhuman 2000) or more generic forms of information (e.g.,
primates, Brothers (1990) proposed that the objects about which there is expertise; Gauthier
superior temporal sulcus (STS) is involved in et al., 2000; cf. Farah et al., 1998) and about the
integrative processing of conspecifics’ behavior, specific contribution to social cognition made by
and the amygdala and orbitofrontal cortex are various nuclei. These disputes depend in part on the
subsequently involved in specifying the conceptualization of brain localization and what it is
socioemotional relevance of social information. that is localized.
Kanwisher (e.g., Kanwisher, 2000; Kanwisher, The adaptive value of social recognition and
McDermott, & Chun, 1997), using functional communication contributed to the evolution of the
magnetic resonance imaging (fMRI) data, emphasized human brain’s preparedness for facial recognition
the role of the fusiform gyrus in face processing, and language acquisition, but these same
and Damasio and colleagues (e.g., Damasio, 1994), information-processing operations may be useful in
focusing primarily on data from humans with brain all kinds of information processing tasks confronting
lesions, have emphasized the role of the frontal people in contemporary societies. However, the
(ventromedial prefrontal, orbitofrontal) cortex, evolutionary function or functions that led to the rise
amygdala, and somatosensory cortex (insula, SI, SII) of a particular neural system in the human brain do
in social perception, cognition, and decision-making. not place strict constraints on the stimuli, tasks, or
Adolphs (2003, see Figure 1) has drawn from these functions that may be served today by the system.
literatures and brain lesion data to suggest that That is, a structure that already exists can take on a
social cognition draws upon neural mechanisms for new function, or an established process may be
perceiving, recognizing, and evaluating stimuli, co-opted for other purposes. This extended
which are then used to construct complex central adaptation of an existing structure or system is
representations of the social environment. These termed exaption and may contribute to the
central processes involve the fusiform gyrus and the appearance that there is nothing special in the brain
STS in the temporal region, as well as the amygdala, regarding social cognition.
orbitofrontal cortex, anterior and posterior cingulate Ideas about the anatomical basis of functional
cortices, and right somatosensory-related cortices. localization in the cortex have been debated for
Like nonsocial information processing, the central hundreds of years, until research on primary
processes of social cognition modulate effector sensory cortices (e.g., Munk, 1881; Tunturi, 1952)
systems including the motor and premotor cortices, and on somatosensory regions (e.g., Schaltenbrand
RT0996_P01.qxd 11/8/04 2:13 PM Page 5
Coarse perceptual Detailed perceptual

processing processing
Superior colliculus Fusiform gyrus
Superior temporal gyrus
Motivational evaluation Representation of

Amygdala perceived action Reappraisal
Orbitofrontal cortex Left frontal operculum
Ventral striatum Superior temporal gyrus
Self-
regulation
Emotional response Modulation of cognition
in body (memory, attention)
Visceral, autonomic, Cingulate cortex
endocrine changes Hippocampus
Basal forebrain
Representation of Social reasoning

emotional response Prefrontal cortex
Somatosensory-related
cortices
FIGURE 1 ■ Processes and brain structures that are involved in social cognition. It is
possible to assign sets of neural structures to various stages of information processing.
However, the flow of social information defies any simple scheme for at least two reasons:
It is multidimensional and it is recursive. A single process is implemented by a flexible set
of structures, and a single structure participates in several processes, often during distinct
windows of time. Processing routes differ in terms of their automaticity, cognitive penetra-
bility, detail of the representations they involve, and processing speed. The structures out-
lined in this figure share some core features of a social information processing system:
selectivity (they make distinctions between different kinds of information), categorization
and generalization, and the incorporation of past experience. Several of the components
of social cognition (inside the gray shaded area) contribute to social knowledge. Reap-
praisal and self-regulation are particular models of feedback modulation whereby evalua-
tion and emotional response to social stimuli can be volitionally influenced. (From Adolphs,
R. [2003]. Neuroscience Reviews, 4, 165–178. With permission.)
& Woolsey, 1964) refuted the hypothesis that the makes a particular action or when it observes
brain was a homogeneous tissue that depended on another individual making a similar action (Rizzolatti,
total mass to carry out functions. The more recent Fogassi, & Gallese, 2001). The same neurons also
discovery of mirror neurons in monkeys also respond on perceiving an object that affords specific
cautions against a premature assignment of function kinds of motor behaviors (Grezes & Decety, 2002;
to structures, and at the same time illustrates the Rizzolatti & Fadiga, 1998), but they do not otherwise
potential brain localization of social information tend to respond to the presentation of an object of
processing. Mirror neurons are a class of neurons in an action, or to the mimicking of an action in the
the ventral premotor cortex of monkeys (area F5, absence of the object. Kohler et al. (2002) recorded
among others) that become active when the monkey from individual neurons in the F5 area of monkeys
RT0996_P01.qxd 11/8/04 2:13 PM Page 6
homologous to Broca’s area in humans. They found amygdala undermined an animal’s capacity for
that individual neurons responded when the monkey effective social interactions.
performed specific motor behaviors, when the Using macaque monkeys, Amaral and colleagues
monkey observed other individuals performing the (Amaral et al., 2003; Prather et al., 2001) examined
same behavior, and when the monkey heard but the role of the amygdala in social cognition using a
could not see the same behavior being performed by more circumscribed lesion of the amygdala and
another. These results indicate that visual and quantitative measures of dyadic social interactions.
audiovisual mirror neurons code not the visual Amaral et al. (2003) observed the social behavior of
analysis of the action per se but the goal and monkeys with ibotenic acid lesions of the amygdala
meaning of the actions of both oneself and others, and controls matched for age, sex, and dominance.
as well as the perspective one takes on those actions Results revealed that the lesion of the amygdala
(Ruby & Decety, 2001). Studies employing fMRI appeared to have produced a socially uninhibited
suggest that similar functions may also be monkey. For instance, the lesioned monkeys did not
represented in the premotor cortex of humans go through the normal period of evaluation of the
(Grezes et al., 2003). other monkeys before engaging in social
Recall that the somatosensory cortices dominate interactions. As a result, the lesioned monkeys
most mammalian brains but in the human brain the initiated greater amounts of affiliative social
frontal and temporal lobules constitute over half the behavior than the control monkeys. Amaral et al.
cerebral cortex. The well-defined localization (2003) also found differences in how the control
of sensory and motor functions poses as a animals interacted with the lesioned animals. Rather
hypothesis but does not prove that more complex than shunning the lesioned monkeys because of
integrative processing by the brain is similarly their early and non-normative forwardness toward
compartmentalized. Brothers (1990), for instance, the control monkeys, the control monkeys
suggested that the amygdala was an essential generated more affiliative social behaviors toward
component of a set of nuclei involved in social the lesioned than the control monkeys. “The
cognition. The evidence that led to this suggestion is inevitable conclusion from this study is that in
illustrated by Dicks, Myers, and Kling (1969), who dyadic social interactions, monkeys with extensive
subjected a sample of free-roaming rhesus monkeys bilateral lesions of the amygdala can interpret and
to bilateral amygdalectomy before returning them to generate social gestures and initiate and receive
their social groups. Dicks et al. (1969) found that more affiliative social interactions than controls”
these monkeys invariably were ostracized and most (Amaral et al., 2003, p. 238). Amaral and
perished without the support of their troop. This led colleagues have posited that the amygdala
to the view that the amygdala was essential for the functions to suppress the engagement of objects
normal perception and production of expressive and conspecifics while an evaluation of the potential
displays and behaviors, and that damage to the threat is conducted. In the absence of a functioning
RT0996_P01.qxd 11/8/04 2:13 PM Page 7
amygdala, conspecifics are not regarded as the dramatic expansion of the human brain may
potentially dangerous or the social context is have been fueled by the complexities of social
regarded as safe, and, hence, social interactions interactions and communication, the central
are engaged. processes in the brain may not be specialized, or
Amaral et al. (2003) further reviewed evidence at least not specialized exclusively, for social
that the amygdala plays this role in threat appraisals information processing. Cortical and subcortical
generally, not only in the evaluation of potential areas may be active in a wide range of behavioral
dyadic partners. The latency to retrieve a food functions but across those functions the networks in
reward from in front of a stimulus object is slowed which they participate may be quite different. With
in normal animals when the stimulus has fear- these nuances in mind, knowing which brain
provoking qualities (e.g., rubber snake), but the structures or systems are involved, especially when
latency of lesioned monkeys was not altered by used in combination with a sophisticated
potential dangers in the environment. Prather et al. understanding of the roles or functions of these
(2001) evaluated whether the amygdala was structures and systems, fosters the construction of
essential not for the emission of social behaviors but crucial tests among competing hypotheses and
for the learning of social behaviors. Young monkeys leads to new hypotheses about the structure and
underwent the same bilateral lesion of the amygdala function of specific social processes,
at 2 weeks of age. Steps were taken to insure representations, and constructs. The readings in this
opportunities for normal social development and to book are designed to stimulate and to provoke just
avoid the behavioral differences associated with such thinking, and thereby, promote a deeper and
nursery rearing. Results reveal interactions with the more comprehensive understanding of social
mother that are similar for lesioned and control cognition, emotion, and behavior.
animals. The lesioned animals also showed little fear To simplify methodological variations, we have
of normally fear-provoking objects (e.g., rubber given preference to human research that employs
snake), although they did show more less social fMRI. When one first reads an article in which
interaction and more fear grimaces, screams in functional brain imaging measurements are
novel dyadic social interactions. Most social reported, one may be both impressed by the
behaviors, however, were indistinguishable between apparent ability of this new technology to image the
lesioned and control animals. inner workings of the normal human brain, and
In sum, the question is not whether, but where bewildered by the esoteric language of nuclear
and how activity in the brain serves a social process physics. To prepare readers for the articles that
that has the potential to inform theory in the social follow, we thought it would be useful to review, in
and brain sciences. The research on the essential nontechnical language, what fMRI is measuring,
role of the amygdala in social cognition is how and why the blood oxygen level dependent
illustrative of the general thesis that, even though (BOLD) response is assumed to be related to
RT0996_P01.qxd 11/8/04 2:13 PM Page 8
neuronal activity. In addition, we will offer the Sunday edition of The New York Times. This
understandable definitions for the abstruse neuromuscular action is accompanied by an
Ts—Tesla, T1, T2, TR, and TE—and why we should increased rate of oxygen utilization in the muscles
care what they are. in your arms. The initial muscular exertion results
Like other approaches, however, fMRI studies in reduction in the ratio of oxygenated to
are informative regarding questions within certain deoxygenated hemoglobin in your arm muscles,
contexts. Readers interested in some of the followed quickly by an increase in blood flow to the
limitations as well as potential of fMRI studies working muscles and an increase in the ratio of
might wish to see Cacioppo et al. (in press), oxygenated to deoxygenated hemoglobin.
Jezzard, Matthews, & Smith (2001), Sarter, Accompanying the increase in blood flow to the
Berntson, & Cacioppo (1996), and Zald & Curtis working muscles of your arms is a slight increase in
(in press). blood volume in these regions. This is why, after
putting down whatever heavy object you lifted, you
might have noticed that your arms felt bigger and
What Is fMRI Measuring?
the veins in your arms were more visible. Note that
Although the relationship is complex (Heeger & this change in blood flow increases the ratio of
Ress, 2002), there is a coupling between neuronal oxygenated to deoxygenated hemoglobin in this
activity and the local control of blood flow and region and does so when there is muscular
oxygenation in the brain. The current model of the exertion, whether or not work is actually performed
hemodynamic response posits that a transient (that is, whether the muscle exertion produced any
increase in neuronal activity within a region of the movement). The ratio of oxygenated to
brain begins consuming additional oxygen in the deoxygenated hemoglobin in the region increases
blood proximal to these cells but also causes local depending on exertion, metabolism, and
vasodilation. As a result, blood near a region of local vasodilation, not work per se.
neuronal activity soon has a higher ratio of For introductory purposes, the enhanced local
oxygenated to deoxygenated hemoglobin than blood blood flow and oxygenation that serves as the basis
in locally inactive areas. The BOLD fMRI provides a for brain imaging using fMRI can be viewed as
measure of these hemodynamic adjustments and— occurring in this fashion—an initial brief period of
by inference—the transient changes in neuronal deoxygenation upon activation of a brain region,
activity in the proximal brain tissue (cf. Buckner, followed by a slower increase in oxygenation to that
1998; Heeger & Ress, 2002; Liao et al., 2002; region that peaks approximately 4–12 seconds
Raichle, 2000). following the onset of the task-induced brain
If this is difficult to visualize, think of a time when activation. More specifically, the basis for the BOLD
you have picked up something heavy such as response is both the greater oxygen extraction and
dumbbells, a particularly heavy stack of books, or the greater blood flow to active brain tissue.1 At rest,
RT0996_P01.qxd 11/8/04 2:13 PM Page 9
oxygen extraction from blood in the brain is about a spinning top that essentially either spins right
40%. During activation, blood flow to the region side up or upside down and, like a top, tends to
increases about 30% and oxygen extraction wobble around that upward or downward
increases about 5%. Imagine a capillary in the brain alignment. Because much of the human brain is
at rest that contains 100 molecules of water, hydrogen nuclei are incredibly plentiful.
deoxyhemoglobin (dHB) and 100 molecules of How plentiful? There are more than 4 * 1019 water
oxyhemoglobin (HB). At rest, 40% of oxygen is protons/mm3 in the human brain (Jezzard et al.,
extracted from the capillary, which leaves 140 2001). If a hydrogen nucleus were equivalent to
molecules of dHB and 60 molecules of HB. The ratio 1 ft, the number of hydrogen nuclei within
of dHB/HB is 2.3. 1 cubic mm of brain tissue would be equivalent to
When the brain tissue served by this capillary is 304.24 billion trips around the earth’s equator. If a
activated, two additional molecules of HB are hydrogen nucleus were to be a single second, the
extracted (5% of the 40 HB molecules that were number of hydrogen nuclei within a cubic mm of
extracted at rest), and blood flow to the region brain tissue (a cube of tissue four-hundredths of
increases by 30%. This yields 172 molecules of dHB an inch high by four-hundredths of an inch wide by
(130 molecules of dHB that flowed into the region four-hundredths of an inch deep) would be the
plus the 42 molecules that were extracted from the equivalent of 1278 billion years. Suffice it to say
HB molecules) and 88 molecules of HB (130 HB that from the perspective of quantum physics,
molecules that flowed to the region less 42 that were 1 cubic mm of brain tissue is packed with hydrogen
extracted). The ratio of dHB/HB becomes 1.9— nuclei.
which means more oxygen has reached the brain An MRI instrument creates a strong magnetic field
region than is needed, and the proportion of (termed an applied magnetic field) that is
oxygenated hemoglobin is larger during activation homogeneous, which means all nuclei within the MRI
than at rest—a task-evoked blood oxygen level instrument are exposed to the same applied magnetic
dependent response. field—usually 1.5 to 3 Tesla in contemporary brain
Origins of the MRI signal. The basis for the fMRI imaging studies. A Tesla is a measure of magnetic flux
signal is the fact that certain nuclei (e.g., hydrogen) density, or in simpler terms, the strength of the
2
have an intrinsic magnetic dipole moment. A magnetic field created by the MRI scanner. The earth
hydrogen nucleus consists of a single proton that has a magnetic field but we hardly notice it in everyday
spins. Think of this hydrogen nucleus (proton) as life unless one is using a compass to navigate. A 3
Tesla magnetic field is about 60,000 times the strength
1
We thank Ana Solodkin from the University of Chicago for of the earth’s magnetic field (Bandettini et al., 2000).
these statistics and this example.
2
The term magnetic moment, or magnetic dipole moment, Placing an individual’s head into an MRI scanner
refers to the maximum torque that any given system of
for an fMRI study, to the physicist, is equivalent to
poles/charges can experience in a uniform magnetic/electric
field of unit strength. placing a very large number of hydrogen nuclei into
RT0996_P01.qxd 11/8/04 2:13 PM Page 10
a strong, homogeneous magnetic field produced by excited to relaxed state requires the release of
the magnet. When this is done, a small percentage energy. In fMRI imaging, the scanner injects a pulse
of these spinning hydrogen nuclei reorient their spin of RF energy to excite these nuclei and raise them
axis to align (parallel or antiparallel) to this magnetic out of their low energy states. When the RF pulse is
3
field. These nuclei precess (think of it as wobble) terminated, the hydrogen nuclei return to their lower
around this alignment much as a spinning top might energy states, and in so doing emit RF energy that is
wobble around its center of gravity. The parallel measured by the scanner.
alignment is a state of relatively low energy, so The RF energy absorbed by the hydrogen nuclei
slightly more spins reside at this than any other perturbs their spin and alters the wobble. The
orientation in a strong magnetic field. That is, a frequency of the wobbling (precession) of these
hydrogen nucleus whose spin is oriented parallel to hydrogen nuclei is a function of the strength of the
the applied magnetic field (think of it as a top magnetic field (higher magnetic field strengths
spinning upright) is said to be relaxed or in a low produce higher frequency wobbles). Any given
energy state, and a hydrogen nucleus whose spin is magnetic field will produce a signature precess
oriented against the magnetic field (think of it as a (wobble) frequency, which is called its Larmor
top spinning upside down) is said to be in an excited frequency. The Larmor frequency is also the
or high energy state. Importantly, the transition from resonant frequency for these nuclei. What this
a low to high energy state is accompanied by the means will become clear shortly. Of course, a human
absorption of energy in the radiofrequency (RF) brain contains more than hydrogen nuclei, and the
range, whereas a transition from a high to low applied magnetic field affects these other nuclei, as
energy state is accompanied by emission of energy well. But their frequency of precession differs from
4
in the RF range. This is actually quite intuitive; hydrogen nuclei, which means the MRI can be tuned
moving from a relaxed to excited state requires the to detect and quantify specifically the RF signals
input of energy, whereas the transition from an from hydrogen nuclei (Bandettini et al., 2000).5
Moreover, the relative abundance of hydrogen nuclei
3
Not all nuclei align to the magnetic field because their spins
in the brain and their high sensitivity for magnetic
become randomized by thermal motion. The number of resonance signal make hydrogen nuclei ideal
nuclei that align is about 3 per million per Tesla. Given there
are approximately 4 * 1019 nuclei in a cubic mm of brain candidates for brain imaging studies.
tissue, there is still a very large number of nuclei that change If a radio signal is now injected at the resonant
their alignment as a result of being placed in a magnetic field.
4
Electromagnetic energy can be described in terms of a (i.e., the Larmor) frequency for hydrogen nuclei, the
stream of photons, each traveling in a wave-like pattern,
moving at the speed of light and carrying some amount of
energy will be absorbed by some of these nuclei
energy. The difference between radio waves, visible light, (i.e., the nuclei will become excited), causing some
and gamma rays is the energy of the photons. RF waves have
photons with low energies and the longest wavelength.
Microwaves have a little more energy than radio waves,
5
infrared has still more, then visible, ultraviolet, X-rays, and The Larmor frequency in a 1.5 Tesla applied magnetic field is
gamma rays. 64 MHz, whereas in a 3 Tesla field it is 128 MHz.
RT0996_P01.qxd 11/8/04 2:13 PM Page 11
to align in the opposing orientation. Think of it as a protons to lower energy states), is the RF signal that
spinning top that is pulsed with an external source of is at the Larmor frequency for a particular hydrogen
energy and flips so that it is now spinning at a nucleus. (Recall from above that this signature
different orientation. frequency is a function of the applied magnetic
The hydrogen nuclei that absorbed the RF energy field.) Coils, or devices that can generate additional
from the brief pulse in an MRI scanner will dissipate small, local magnetic field gradients or differences,
this energy and return to their more stable (lower are used to slightly shift the Larmor frequency
energy) state (e.g., parallel to the applied magnetic across brain areas, and permit the scanner to
field) when the RF pulse terminates. As these nuclei recognize the location of the signal by its
transition from a high to low energy state, these frequency.7 It is simpler to envision this process in
hydrogen nuclei emit an RF signal at their signature one-dimensional than two-dimensional or three-
frequency (i.e., 64 MHz in a 1.5 Tesla magnetic field, dimensional space, but the notion of how one
128 MHz in a 3 Tesla field). The emission of this RF localizes the RF signal from relaxing hydrogen nuclei
signal is the MRI signal that is recorded in fMRI works generally the same in these higher
6
studies. Excited spins regain 66% of their dimensional spaces. Rather than thinking about
equilibrium magnetization over a period called T1 where the hydrogen nuclei might be within the three
and 95% of their equilibrium magnetization over dimensional brain, therefore, we will consider the
three T1 periods. (T1 refers to the time constant for simpler example of how to locate their position
the exponential decay of excitation of the nuclei.) along a single line.
Importantly, the T1 for a water molecule will depend A 3 Tesla MRI scanner produces a homogeneous,
on the local chemical environment (e.g., different constant 3 Tesla magnetic field. Therefore, all of the
parts of the brain). For instance, T1 is longer for hydrogen nuclei along any single line will have a
water in the cerebrospinal fluid than for water in resonant frequency of 128 MHz. If an RF pulse
tissue. excited some of these nuclei and their emissions
No detector could tell where such a signal is were recorded as they relaxed, detecting a 128 MHz
coming from if that is all that was involved. However, emission would not tell you where along the line the
if a range of frequencies were included in the radio emissions originated. If, however, a coil is
pulse, the only frequency that would be absorbed positioned at an angle to this line of hydrogen nuclei
(i.e., within limits, excite or move some of the and the coil applies a second but much smaller
protons to higher energy states) and then returned magnetic field, the local magnetic field for the
(i.e., emitted on dissipation or relaxation of the hydrogen nuclei along this line will differ slightly
6 7
In most brain imaging studies, the strong signal from protons Coils are used because when electricity flows through a coiled
in fat is ignored or edited from the image. This means that in conductor, a magnetic field is generated. Hence, one can de-
MRI and fMRI, it is the distribution of protons in tissue water sign coils whose magnetic flux density can be specified and
that is imaged (Jezzard et al., 2001). can be turned on and off quickly.
RT0996_P01.qxd 11/8/04 2:13 PM Page 12
Coil gradient
υ0 = 128 MHz
υ0 < 128 MHz υ0 > 128 MHz
2.99 Tesla field 3.01 Tesla field
3 Tesla applied field
FIGURE 2 ■ MRI and the localization of RF emissions

from hydrogen nuclei in water in the human brain, illus-
trated along one dimension. An MRI scanner applies a
large (e.g., 3 Tesla), homogeneous applied magnetic
field. An RF pulse is used to excite some of the
hydrogen nuclei, which absorb RF energy at a frequency
that is a specific function of the magnetic field applied
to the hydrogen nuclei. For instance, the resonance
frequency for hydrogen nuclei in a 3 Tesla magnetic
field is 128 MHz. A coil is also used to produce small
variations in the applied magnetic field along the line of
imaging. This produces slightly different applied mag-
netic fields along the line of hydrogen nuclei, and
causes their resonance frequency to vary slightly.
When the RF signals emitted by these hydrogen nuclei
are measured, a decomposition of this measured sig-
nal provides information on the strength and location of
each of the component signals—that is, where along
the line the signal originated and how much signal orig-
inated at each point along the line.
(see Figure 2). For instance, the applied magnetic frequency. When the RF pulse ends, the excited
field of the hydrogen nuclei that fall below the coil hydrogen nuclei transition from high to low energy
may exist in a magnetic field slightly higher than states (i.e., relax), as described earlier. In doing so,
3 Tesla, and those that fall above the coil may exist these nuclei emit RF frequencies at their resonant
in a field slightly lower than 3 Tesla. Because the frequencies, which differ depending where they are
frequency of the wobble of the hydrogen nuclei along our hypothetical line of hydrogen nuclei.
depends on the strength of the magnetic field, the These emitted RF frequencies are then quantified by
frequency at which these nuclei wobble will differ, the scanner.
with higher resonant frequencies at the end where The quantified RF signals will be the sum of many
the local magnetic field is higher, and lower resonant different RF signals, so this complex signal needs
frequencies at the end where the local magnetic field then to be decomposed into frequency components
is slightly lower. When an RF pulse of varying using mathematical procedures such as the fast
frequencies is applied briefly in the MRI scanner, the Fourier Transform. Which frequencies are present in
hydrogen nuclei along this line will absorb the RF this decomposed signal provides information about
energy that matches its resonant (Larmor) where along the line hydrogen nuclei had been
RT0996_P01.qxd 11/8/04 2:13 PM Page 13
excited and relaxed, and the power recorded at that at the atomic scale result in a loss of coherence in
RF would provide information about the strength of the phases of the resonance emissions of the
the signal. So that allows structural localization, but hydrogen nuclei, which produces an exponential
does it permit measurement of brain activity ? loss of intensity for the summed resonance signal
Functional MRI. Shortening the interpulse delay from all of the nuclei together. Consequently, the
(TR) in a pulse sequence means the same thing as measured radio-frequency signal decays more
increasing the rate at which radio-frequency pulses quickly when there are local inhomogeneities in the
are applied to the sample. As the TR shortens, the magnetic field because each hydrogen nucleus is
emitted RF signals from regions of the brain with a exposed to a slightly different magnetic field
shorter T1 (e.g., tissue) will increase relative to parts strength during excitation and their radio-frequency
of the brain with a longer T1. Within any specific emissions interfere with each other resulting in the
region of the brain, an excited, spinning, wobbling net RF signal diminishing more quickly than those in
resonating hydrogen nucleus, if isolated, would more homogeneous magnetic fields. The time
dissipate energy with a time constant of T1. In a real constant for the exponential function describing the
sample, however, one is measuring the period of dephasing is referred to as T2 (Jezzard
simultaneous emissions from a huge number of et al., 2001).
nuclei. When dealing with more than a trillion It follows from the above that T2 is an intrinsic
protons spinning per cubic mm across different property of nuclei in a particular chemical
8
regions of the brain, the nuclei are each environment. Increasing the delay before signal
continuously experiencing very small changes in detection in a pulse sequence is referred to as
local magnetic fields (i.e., “inhomogeneities”). These lengthening the TE. This has the effect of increasing
varying magnetic fields result in an exchange of the strength of the radio-frequency signals from
energy between the hydrogen nuclei, which results nuclei with a relatively long T2 (e.g., brain grey
in a dephasing of these nuclei and a reduction in the matter) relative to those with a short T2 (e.g., brain
intensity of the summoned resonance signal white matter). This is because the exponential decay
(emitted RF signals) from these nuclei. That is, some of the signal from the latter nuclei results in a more
of these nuclei will emit RF signals in phase, while precipitous drop in signal strength than does the
the emissions of others will fall out of phase (i.e., exponential decay over the same TE when the time
“dephase”) as a function of predictable factors. constant (T2) is relatively long (Bandettini et al.,
When the nuclei are all oscillating together, they are 2000; Jezzard et al., 2001).
said to have high coherence. Local inhomogeneities The rate at which the signal decays (i.e., T2)
depends upon physical and physiological factors.
8
If a hydrogen nucleus was the size of a top (about 3 cubic in.), Most important for investigators using fMRI for
fitting the same number of tops as hydrogen nuclei one finds in brain imaging, the signal decays more quickly in the
a single cubic mm of brain tissue would take approximately
2 quadrillion cubic mi. presence of deoxyhemoglobin than oxygenated
RT0996_P01.qxd 11/8/04 2:13 PM Page 14
hemoglobin (blood). This is because oxygenated within a single repetition time (i.e., TR; see Zald &
hemoglobin is less paramagnetic than Curtis, in press ). It is, however, sensitive to a
dexoygenated hemoglobin (recall that hemoglobin number of artifacts (e.g., movement). For this
contains iron), and produces a more homogeneous reason, the temporal resolution of the fMRI was
magnetic field, greater spin coherence (i.e., less initially limited by the need to block experimental
cancellation of the emitted radio-frequency signals), conditions. This limitation has been lessened by the
and a stronger MRI signal. BOLD fMRI techniques demonstrated feasibility of selective averaging
are designed to measure primarily changes in the techniques (Buckner et al., 1996). Ultimately, the
inhomogeneities of the magnetic field within each temporal resolution of the fMRI is limited by the fact
small volume of tissue resulting from these that the blood flow response typically lags the actual
variations in blood oxygenation and—by electrical signal by 1–2 sec, peaks in 4–12 sec, and
inference—neuronal activity (cf. Heeger & Ress, does not track activity on a msec-by-msec basis.
2002). As one might expect, the signal changes are That is, the blood flow response is influenced by
quite small – 0.5-5.0% at 3 Tesla, for instance, 9
activity levels over some time interval (a few
but techniques have been developed to produce hundred msec or more) and, thus, is less temporally
signal-to-noise ratios adequate to be detectable specific than the neuronal activity with which it is
(e.g., Cohen & Bookheimer, 1994). The essential associated. For many investigations, this temporal
point here is that an increase in the concentration of imprecision is of little or no importance. In studies in
deoxygenated hemoglobin causes a decrease in which higher temporal resolution is required, fMRI
image intensity, whereas a decrease in deoxygenated studies can be complemented by event-related brain
hemoglobin causes an increase in image intensity. potential (ERP) studies.
The information one gets from an fMRI scan Interpretation of fMRI data. A series of functional
depends upon how and when magnetic gradients scans are usually collected during a baseline and
and RF pulses (i.e., pulse sequences) are applied. during the performance of one or more tasks.
A common fMRI technique for studying the time Typically, the functional images for each participant
course of signal intensity changes in the brain is fast are realigned to correct for the participant’s
echo planar imaging (EPI), which makes it possible movement, and then coregistered with a structural
to acquire a complete two-dimensional image in as (T1 weighted) image (e.g., a high resolution MRI
little as 40 ms following a single excitation of the image of the individual’s brain). If required, the
spin system and multislice volumes of MR images images are spatially normalized to align brains
across participants, and statistical analyses are
9
performed to identify areas that have been activated
Higher strength magnetic fields (e.g., 3 Tesla MRIs) produce
greater excitation of the nuclei and stronger MRI signals but by the experimental manipulation. The results are
they can also be more sensitive to artifacts and, in some
then displayed on individual or average structural
instances, result in net loss in signal contrast, especially in
regions near air (e.g., orbitofrontal cortex). images. Averaging images across brains can also
RT0996_P01.qxd 11/8/04 2:13 PM Page 15
produce artifacts, so the presentation of images too, can be found on the internet (e.g.,
from individual brains, at least in supplementary medical-dictionary.com/). A few directional terms
materials, is often advisable (see Brett, Johnsrude, & are essential to understand when studying the
Owen, 2002). How one should perform statistical brain, however, so we cover them here. First is a
tests on such a large multivariate dataset, pair of terms tied to a spatial reference. Anterior
characterized by correlated dependent variables and refers to the front or in front of, whereas posterior
violations of sphericity assumptions, is important refers to the back or the back part of. There are also
but beyond the scope of this introduction. three dimensions (axes) that are used to specify
An important point to remember when reading topographical relations defined in terms of a body-
fMRI studies is that brain regions depicted as reference frame in bilaterally symmetrical animals
“active” refer to areas of the brain that showed task- (including humans): (a) rostrocaudal axis, which
related increases in deoxygenation and oxygenation, refers to the beak/tail dimension in nonhuman
an indicator of the activation of nearby brain tissue. animals and the human body, and to the dimension
This is a relative, not an absolute, measure. One ranging from the forebrain (rostral) to the spinal
can tell that the brain activation is task-related cord (caudal) in the human central nervous system;
either because the strength of the BOLD response (b) dorsoventral axis, which refers to the back
during a task is greater than resting baseline or (dorsal) and belly (ventral) dimension of the body,
because the strength of the BOLD response during and to the back (dorsal) and front (ventral)
a task differs from the BOLD response during dimension in the human brain and spinal cord; and
another task that differs from the first by, for (c) the mediolateral axis, which refers to the middle
instance, only one information processing or toward the middle (medial) and toward the sides
operation. Task minus baseline contrasts point to (lateral) in both the body and brain. This
the active regions of brain associated with the terminology can be confusing, because of the
performance of the task vs. resting, whereas alternative frames of reference, as the mapping
contrasts between two tasks point to the active between the spatial and body reference frames is
brain regions associated with the performance of different for quadrapeds and bipeds. For humans,
the information processing operation that differs however, anterior is generally considered to be
between the two tasks. synonymous with ventral and posterior with dorsal.
It will be helpful to have a brain atlas when There also are three orientations in which one might
reading fMRI studies to help visualize the brain slice or depict the brain: (a) an axial orientation
regions and their structural relationships to one means the brain is sliced top to bottom; (b) a
another. One such atlas that might be useful is the coronal orientation means the slices are made ear
whole brain atlas, available on the Internet at to ear or any plane parallel thereto; and (c) a
www.med.harvard.edu/AANLIB/home.html. sagittal orientation means the slices are made in the
A medical dictionary may also be helpful, and these, median plane (front to back in the direction an
RT0996_P01.qxd 11/8/04 2:13 PM Page 16
arrow would travel through the brain) or any plane Detection of cortical activation during averaged single trials
of a cognitive task using functional magnetic resonance
parallel thereto. imaging. Proceedings of the National Academy of Science,
Finally, remember the fact that there is brain 93(25):14878–14883.
Buckner, R. L. (1998). Event-related fMRI and the hemody-
activation associated with a psychological state or namic response. Human Brain Mapping, 6, 373–377.
Cacioppo, J. T., & Berntson, G. G. (1992). Social psychologi-
process is not particularly informative—we take as a cal contributions to the decade of the brain: Doctrine of mul-
given that the brain underlies psychological activity. tilevel analysis. American Psychologist, 47, 1019–1028.
Cacioppo, J. T., & Berntson, G. G. (2004). Social neuro-
Where and when brain activation occurs is science. In M. Gazzaniga (Ed.), The new cognitive neuro-
informative, however, because we are beginning to science (3rd edition). Cambridge, MA: MIT Press.
Cacioppo, J. T., Berntson, G. G., Adolphs, R., Carter, C. S.,
understand what kinds of information processing are Davidson, R. J., McClintock, M. K., McEwen, B. S.,
Meaney, M. J., Schacter, D. L., Sternberg, E. M., Suomi,
performed in various parts of the brain. By studying S. S., & Taylor, S. E. (2002). Foundations in social neuro-
the neural substrates of a social or psychological science. Cambridge, MA: MIT Press.
Cacioppo, J. T., Berntson, G. G., Lorig, T. S., Norris, C. J.,
process, we should be able to learn more about the Rickett, E., & Nusbaum, H. (2003). Just because you’re im-
component information processing operations and aging the brain doesn’t mean you can stop using your head:
A primer and set of first principles. Journal of Personality
sources of errors and biases in these operations and Social Psychology, 85, 650–661.
Cohen, M. S., & Bookheimer, S. Y. (1994). Localization of
(Cacioppo et al., in press). brain function using magnetic resonance imaging. Trends in
Neurosciences, 17, 268–277.
Damasio, A. R. (1994). Descartes’ error: Emotion, reason,
REFERENCES and the human brain. New York: Grosset/Putnam.
Dicks, D., Myers, R. E., & Kling, A. (1969). Uncus and amyg-
Adolphs, R. (2003). Cognitive neuroscience of human social
dalalesions: Effects on social behavior in the free ranging
behaviour. Neuroscience Reviews, 4, 165–178.
rhesus monkey. Science, 165, 69–71.
Amaral, D. G., Capitanio, J. P., Jourdain, M., Mason, W. A.,
Farah, M.J. (1994). Neuropsychological inferences with an
Mendoza, S. P., & Prather, M. (2003). The amygdala: Is it
interactive brain: A critique of the “locality” assumption.
an essential component of the neural network for social cog-
Behavioral and Brain Sciences, 22, 287–288.
nition? Neuropsychologica, 41, 235–240.
Gauthier, I., Skudlarski, P., Gore, J. C., & Anderson, A. W.
Aronson, E. (1968). Dissonance theory: Progress and prob-
(2000). Expertise for cars and birds recruits brain areas
lems. In R. P. Abelson, E. Aronson, W. J. McGuire, T. M.
involved in face recognition. Nature Neuroscience, 3,
Newcomb, M. J. Rosenberg, & P. H. Tannenbaum (Eds.),
191–197.
Theories of cognitive consistency: A sourcebook (pp. 5–27).
Grezes, J., Armony, J. L., Rowe, J., & Passingham, R. E.
Chicago: Rand McNally.
(2003). Activations related to “mirror” and “canonical” neu-
Bandettini, P. A., Birn, R. M., & Donahue, K. M. (2000).
rones in the human brain: An fMRI study. Neuroimage, 18,
Functional MRI: Background, methodology, limits, and im-
928–937.
plementation. In J. T. Cacioppo, L. G. Tassinary, & G. G.
Grezes, J., & Decety, J. (2002). Does visual perception of the
Berntson (Eds.), The handbook of psychophysiology (pp.
object afford action? Evidence from a neuroimaging study.
978–1014). New York: Cambridge University Press.
Neuropsychologia, 40, 212–222.
Berntson, G. G., Boysen, S. T., & Cacioppo, J. T. (1993). Neu-
Heeger, D. J., & Ress, D. (2002). What does fMRI tell us about
robehavioral organization and the cardinal principle of eval-
neuronal activity? Nature Reviews: Neuroscience, 3,
uative bivalence. Annals of the New York Academy of Sci-
142–151.
ences, 702, 75–102.
Jezzard, P., Matthews, P. M., & Smith, S. M. (2001). Func-
Berntson, G. G., & Cacioppo, J. T. (2000). Psychobiology and
tional MRI. Oxford: Oxford University Press.
social psychology: Past, present, and future. Personality
Jones, E. E., & Berglas, S. (1978). Control of attributions
and Social Psychology Review, 4, 3–15.
about the self through self-handicapping strategies: The
Brothers, L. (1990). The social brain: A project for integrating
appeal of alcohol and the role of under achievement.
primate behavior and neurophysiology in a new domain.
Personality and Social Psychology Bulletin, 4, 200–206.
Concepts in Neuroscience, 1, 27–51.
Kanwisher, N. (2000). Domain specificity in face perception.
Buckner R. L., Bandettini P. A., O’Craven K. M., Savoy R. L.,
Nature Neuroscience, 3, 759–763.
Petersen S. E., Raichle M. E., & Rosen B. R. (1996).
RT0996_P01.qxd 11/8/04 2:13 PM Page 17
Kanwisher, N., McDermott, K. B., & Chun, M. (1997). The Rizzolatti, G., Fogassi, L., & Gallese, V. (2001). Neurophysio-
fusiform face area: A module in human extrastriate cortex logical mechanisms underlying the understanding and imi-
specialized for face perception. Journal of Neuroscience. tation of action. Nature Reviews: Neuroscience, 2,
Kohler, E., Keysers, C., Umilta, M. A., Fogassi, L., Gallese, V., 661–670.
& Rizzolatti, G. (2002). Hearing sounds, understanding ac- Roediger, H. L., Buckner, R. L., & McDermott, K. B. (1999).
tions: Action representation in mirror neurons. Science, 297, Components of processing. In J. K. Foster & M. Jelicic
846–848. (Eds.), Memory: Systems, process, or function? (pp. 31–65).
LeDoux, J. (2000). Emotion circuits in the brain. Annual Re- New York: Oxford University Press.
view of Neuroscience, 23, 155–184. Ross, L., Greene, D., & House, P. (1977). The “false consen-
Liao, H., Worsley, K. J., Poline, J. B., Aston, J. D., Duncan, sus effect”: An egocentric bias in social perception and
G. H., & Evans, A. C. (2002). Estimating the delay of the attribution processes. Journal of Experimental Social
fMRI response. NeuroImage, 16, 593–606. Psychology, 13, 279–301.
McDonald, H. E., & Hirt, E. R. (1997). When expectancy Ross, M., & Sicoly, F. (1979). Egocentric biases in availability
meets desire: Motivational effects in reconstructive mem- and attribution. Journal of Personality and Social Psychol-
ory. Journal of Personality and Social Psychology, 72, ogy, 37, 322–336.
5–23. Ruby, P., & Decety, J. (2001). Effect of subjective perspective
McGuire, W. J. (1981). The probabilogical model of cognitive taking during simulation of action: A PET investigation of
structure and attitude change. In R. E. Petty, T. M. Ostrom, agency. Nature Neuroscience, 4, 546–550.
& T. C. Brock (Eds.), Cognitive responses in persuasion Sarter, M., Berntson, G. G., & Cacioppo, J. T. (1996). Brain
(pp. 291–308). Hillsdale, NJ: Erlbaum. imaging and cognitive neuroscience: Toward strong infer-
Medvec, V. H., Madey, S. F., & Gilovich, T. (1995). When less ence in attributing function to structure. American Psychol-
is more: Counterfactual thinking and satisfaction among ogist, 51, 13–21.
Olympic medalists. Journal of Personality & Social Psy- Schaltenbrand, G., & Woolsey, C. N. (1964). Cerebral localiza-
chology, 69, 603–610. tion and organization; proceedings of a symposium sponsored
Munk, H. (1881). Uber die funktionen der grosshirnrinde by the World Federation of Neurology, held at Lisbon, Portu-
[About the functions of the cortex]. Berlin: Hirschwald. gal, October, 1960. Madison: University of Wisconsin Press.
Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we Smith, N. K., Cacioppo, J. T., Larsen, J. T., & Chartrand, T. L.
can know: Verbal reports on mental processes. Psychologi- (2003). May I have your attention please: Electrocortical re-
cal Review, 84, 231–259. sponses to positive and negative stimuli. Neuropsychologia,
Pakkenberg, H. (1966). The number of nerve cells in the cere- 41, 171–183.
bral cortex of man. Journal of Comparative Neurology, 128, Snyder, M., & Swann, W. B., Jr. (1978). Hypothesis-testing
17–20. processes in social interaction. Journal of Personality and
Prather, M. D., Lavanex, P., Mauldin-Jourdain, M. L., Mason, Social Psychology, 35, 656–666.
W. A., Capitanio, J. P., Mendoza, S. P., et al. (2001). Increased Tunturi, A. R. (1952). A difference in the representation of audi-
social fear and decreased fear of objects in monkeys with tory signals for the left and right ears in the isofrequency con-
neonatal amygdala lesions. Neuroscience, 106, 653–658. tours of right ectosylvian auditory cortex of the dog. Journal
Pyszczynski, T., Greenberg, J., & Solomon, S. (1999). A of Comparative and Physiological Psychology, 168, 712–727.
dual-process model of defense against conscious and Wilson, T. D., Wheatley, T., Meyers, J. M., Gilbert, D. T., &
unconscious death-related thoughts: An extension of Axsom, D. (2000). Focalism: A source of durability bias in
terror management theory. Psychological Review, 106, affective forecasting. Journal of Personality and Social Psy-
835–845. chology, 78, 821–836.
Raichle, M. E. (2000). A brief history of human functional Zajonc, R. B. (1980). Feeling and thinking: Preferences need
brain mapping. In A. W. Toga & J. C. Mazziotta (Eds.), no inferences. American Psychologist, 35, 157–193.
Brain mapping: The systems (pp. 33–77). San Diego: Aca- Zald, D. H., & Curtis, C. (in press). Functional neuroimaging.
demic Press. In M. Eid & E. Diener (Eds.), Handbook of psychological
Rizzolatti, G., & Fadiga, L. (1998). Grasping objects and grasp- measurement: A multimethod perspective. Washington, DC:
ing action meanings: The dual role of monkey rostroventral American Psychological Association.
premotor cortex (area F5). Novartis Foundation Symposium,
218, 81–95.
RT0996_P01.qxd 11/8/04 2:13 PM Page 18
RT0996_P02.qxd 11/8/04 2:13 PM Page 19
PA R T 2
The Brain Determines Social Behavior:

The Story of Phineas Gage
•
There is an intuitive appeal to viewing a social psychological construct or

research enterprise as more legitimate, respectable, or “scientific” if the
social psychological measure, process, or representation is shown to
covary with some event in the brain. Nevertheless, investigations that
simply show there are changes in brain activation which correspond to
some aspect of social cognition, emotion, or behavior contribute little
because no scientific theory would predict otherwise. There is ample
evidence that the self and social behavior are maintained despite the loss of
various appendages, visceral organs, and personal relationships but are
terminated by the cessation of activity in the human brain and that this is
so even when the heart continues to beat.
This was not always evident, of course. The earliest localization
of mental faculties and bodily processes was based on blood, or
essences supposedly carried in the blood, rather than the brain. This
misconception is understandable, as the loss of consciousness and life
could clearly be demonstrated to follow dramatic losses of blood. By the
2nd century A.D., however, Galen had firmly planted the faculties of
reason within the brain. A few well-publicized clinical cases in the 19th
century called attention to the role of the brain, particularly the frontal
and temporal regions, in social cognition, affect, and behavior. Among
these was the case of Phineas Gage, a young American railroad construction
19
RT0996_P02.qxd 11/8/04 2:13 PM Page 20
supervisor, who in 1848 accidentally detonated a Both readings provide unequivocal evidence for a
dynamite blast, rocketing his tamping rod through principle articulated in the volume introduction—the
his cheek, eye, and skull and decimating the brain determines social processes and behavior, and
orbitofrontal and sections of the ventromedial the study of where and how the brain does this
cortex of his brain. The case of Phineas Gage is informs and expands our understanding of social
recounted in the first reading by Damasio et al. psychology. The cases covered in these readings also
(1994). illustrate two of three additional principles in social and
The second reading by Anderson et al. (1999) cognitive neuroscience: (1) the lesion of circumscribed
also concerns the cognitive, social, and behavioral areas of the brain could cause the loss of specific
effects of prefrontal cortex lesions. Whereas Phineas mental or nervous functions in humans; (2) activity in
Gage was 25 years of age when he sustained his circumscribed areas of the brain could reflect specific
brain injury, Anderson et al. (1999) examined the mental or nervous functions in humans; and (3) the
long-term consequences of damage to the prefrontal human brain is not a dispassionate information
cortex that occurred during infancy. The fascinating processing organ but a socioemotional processing
similarities and differences in the deficits that organ, as well. Brain lesion studies of the type
emerge from these readings illuminate the illustrated in this section speak to the first and third
importance and specific functions of the prefrontal principles, whereas brain imaging studies such as
cortex in social learning as well as social cognition, those in the subsequent sections of this reader speak
emotion, and behavior. to the second and third principles.
RT0996_C01.qxd 11/8/04 1:58 PM Page 21
R E A D I N G 1
The Return of Phineas Gage: Clues about the

Brain from the Skull of a Famous Patient
•
Hanna Damasio, Thomas Grabowski, Randall Frank,
Albert M. Galaburda, and Antonio R. Damasio*
When the landmark patient Phineas Gage died in 1861, no autopsy was performed, but his skull
was later recovered. The brain lesion that caused the profound personality changes for which his
case became famous has been presumed to have involved the left frontal region, but questions
have been raised about the involvement of other regions and about the exact placement of the
lesion within the vast frontal territory. Measurements from Gage’s skull and modern neuroimaging
techniques were used to reconstitute the accident and determine the probable location of the
lesion. The damage involved both left and right prefrontal cortices in a pattern that, as confirmed
by Gage’s modern counterparts, causes a defect in rational decision making and the processing
of emotion.
n 13 September 1848, Phineas P. Gage, a stone, partially filling the holes with explosive
O 25-year-old construction foreman for the Rut-
land and Burlington Railroad in New England,
powder, covering the powder with sand, and using
a fuse and a tamping iron to trigger an explosion
became a victim of a bizarre accident. In order to into the rock. On the fateful day, a momentary dis-
lay new rail tracks across Vermont, it was necessary traction let Gage begin tamping directly over the
to level the uneven terrain by controlled blasting. powder before his assistant had had a chance to
Among other tasks, Gage was in charge of the det- cover it with sand. The result was a powerful ex-
onations, which involved drilling holes in the plosion away from the rock and toward Gage. The
*H. Damasio and A. R. Damasio are in the Department of in the Department of Neurology, University of Iowa Hospitals
Neurology, University of Iowa Hospitals & Clinics, Iowa City, & Clinics, Iowa City, IA 52242, USA. A. M. Galaburda is in
IA 52242, and the Salk Institute for Biological Research, San the Department of Neurology, Harvard Medical School, Beth
Diego, CA 92186–5800, USA. T. Grabowski and R. Frank are Israel Hospital, Boston, MA 02215, USA.
21
RT0996_C01.qxd 11/8/04 1:58 PM Page 22
fine-pointed, 3-cm-thick, 109-cm-long tamping Twenty years after the accident, John Harlow,
iron was hurled, rocket-like, through his face, unaided by the tools of experimental neuropsy-
skull, brain, and then into the sky. Gage was mo- chology available today, perceptively correlated
mentarily stunned but regained full consciousness Gage’s cognitive and behavioral changes with a
immediately thereafter. He was able to talk and presumed area of focal damage in the frontal re-
even walk with the help of his men. The iron gion (1). Other cases of neurological damage
landed many yards away (1). were then revealing the brain’s foundation for lan-
Phineas Gage not only survived the momen- guage, motor function, and perception, and now
tous injury, in itself enough to earn him a place in Gage’s case indicated something even more sur-
the annals of medicine, but he survived as a dif- prising: Perhaps there were structures in the hu-
ferent man, and therein lies the greater signifi- man brain dedicated to the planning and execu-
cance of this case. Gage had been a responsible, tion of personally and socially suitable behavior,
intelligent, and socially well-adapted individual, to the aspect of reasoning known as rationality.
a favorite with peers and elders. He had made Given the power of this insight, Harlow’s ob-
progress and showed promise. The signs of a pro- servation should have made the scientific impact
found change in personality were already evident that the comparable suggestions based on the
during the convalescence under the care of his patients of Broca and Wernicke made (2). The
physician, John Harlow. But as the months suggestions, although surrounded by controversy,
passed it became apparent that the transformation became the foundation for the understanding of
was not only radical but difficult to comprehend. the neural basis of language and were pursued ac-
In some respects, Gage was fully recovered. He tively, while Harlow’s report on Gage did not in-
remained as able-bodied and appeared to be as spire a search for the neural basis of reasoning,
intelligent as before the accident; he had no im- decision-making, or social behavior. One factor
pairment of movement or speech; new learning likely to have contributed to the indifferent recep-
was intact, and neither memory nor intelligence tion accorded Harlow’s work was that the intel-
in the conventional sense had been affected. On lectual atmosphere of the time made it somewhat
the other hand, he had become irreverent and more acceptable that there was a neural basis for
capricious. His respect for the social conventions processes such as movement or even language
by which he once abided had vanished. His abun- rather than for moral reasoning and social behav-
dant profanity offended those around him. ior (3). But the principal explanation must rest
Perhaps most troubling, he had taken leave of his with the substance of Harlow’s report. Broca and
sense of responsibility. He could not be trusted to Wernicke had autopsy results, Harlow did not.
honor his commitments. His employers had Unsupported by anatomical evidence, Harlow’s
deemed him “the most efficient and capable” observation was the more easily dismissed. Be-
man in their “employ” but now had to dismiss cause the exact position of the lesion was not
him. In the words of his physician, “the equilib- known, some critics could claim that the damage
rium or balance, so to speak, between his intel- actually involved Broca’s so-called language
lectual faculty and animal propensities” had been “center,” and perhaps would also have involved
destroyed. In the words of his friends and ac- the nearby “motor centers.” And because the
quaintances, “Gage was no longer Gage” (1). patient showed neither paralysis nor aphasia,
Gage began a new life of wandering that ended a some critics reached the conclusion that there
dozen years later, in San Francisco, under the were no specialized regions at all (4). The British
custody of his family. Gage never returned to a physiologist David Ferrier was a rare dissenting
fully independent existence, never again held a voice. He thoughtfully ventured, in 1878, that the
job comparable to the one he once had. His lesion spared both motor and language centers,
accident had made headlines but his death went that it had damaged the left prefrontal cortex, and
unnoticed. No autopsy was obtained. that such damage probably explained Gage’s
RT0996_C01.qxd 11/8/04 1:58 PM Page 23
The Return of Phineas Gage ■ 23
behavioral defects, which he aptly described as a

“mental degradation” (5).
Harlow only learned of Gage’s death about
5 years after its occurrence. He proceeded to ask
Gage’s family to have the body exhumed so that
the skull could be recovered and kept as a medical
record. The strange request was granted, and
Phineas Gage was once again the protagonist of a
grim event. As a result, the skull and the tamping
iron, alongside which Gage had been buried, have
been part of the Warren Anatomical Medical
Museum at Harvard University.
As new cases of frontal damage were described
in this century, some of which did resemble that
of Gage, and as the enigmas of frontal lobe func-
tion continued to resist elucidation, Gage gradu-
ally acquired land-mark status. Our own interest
in the case grew out of the idea that Gage exem-
plified a particular type of cognitive and behav-
ioral defect caused by damage to ventral and me-
dial sectors of prefrontal cortex, rather than to the
left dorsolateral sector as implicit in the tradi-
tional view. It then occurred to us that some of the
image processing techniques now used to investi-
gate Gage’s counterparts could be used to test this
idea by going back in time, reconstituting the
accident, and determining the probable placement FIGURE 1.1 ■ Photographs of (A) several views of the
of his lesion. The following is the result of our skull of Phineas Gage and (B) the skull x-ray.
neuroanthropological effort.
We began by having one of us (A.M.G.) photo-
graph Gage’s skull inside and out and obtain a
skull x-ray (Figure 1.1) as well as a set of precise the rod’s trajectory as a straight line connecting
measurements (6) relative to bone landmarks. the center of the entry hole at orbital level to the
Using these measurements, we proceeded to de- center of the exit hole. This line was then carried
form linearly the three-dimensional reconstruc- downward to the level of the mandibular ramus.
tion of a standard human skull (7) so that its di- The skull anatomy allowed us to consider entry
mensions matched those of Phineas Gage’s skull. points within a 1.5-cm radius of this point (20
We also constructed Talairach’s stereotactic space points in all) (Figure 1.2).
for both this skull and Phineas Gage’s real skull Possible exit points were determined as fol-
(8). On the basis of the skull photographs, the di- lows: We decided to constrain the exit point to be
mensions of the entry and exit holes were scaled at least 1.5 cm (half the diameter of the rod) from
and mapped into the deformed standard skull. the lateral and posterior margins of the area of
Based on measurements of the iron rod and on the bone loss (Figure 1.3) because there were no dis-
recorded descriptions of the accident, we deter- ruptions of the outer table of the calvarium in
mined the range of likely trajectories of the rod. these directions (Figure 1.1 lower right panel).
Finally, we simulated those trajectories in three- However, we accepted that the rod might have
dimensional space using Brainvox (9). We modeled passed up to 1.5 cm anterior to the area of bone
RT0996_C01.qxd 11/8/04 1:58 PM Page 24
FIGURE 1.2 ■ (A color version of this figure follows page

146.) View of the entry-level area with the a priori most likely
first trajectory. (A) Skull with this first vector and the level (red)
at which entry points were marked. (B) View of a segment of
section 1. On the left is the mandibular ramus, and on the right
is the array of entry points. (C) Enlargement of the array of
entry points. One additional point was added (L20) to ensure
that every viable entry point was surrounded by nonviable
points. Nonviable vectors are shown in red and viable vectors
with labels identifying their exit points are shown in green.
Abbreviations: A, anterior; L, lateral; P, posterior; AM,
anteromesial; AL, anterolateral; PL, posterolateral; C, central.
FIGURE 1.3 ■ (A color version of this figure follows page

146.) (A) View from above the deformed skull with the exit
hole and the anterior bone flap traced in black. The blue circle
represents the first vector tested, and the gray surface repre-
sents the area where exit points were tested. (B) Schematic
enlargement of the exit hole and of the area tested for exit
points. The letter C marks the first tested vector (blue). The
numbers 1 through 15 mark the other exit points tested. Red
indicates nonviable vectors, green indicates viable vectors,
and the label identifies the entry point. Note that the a priori
best fit C was not viable.
RT0996_C01.qxd 11/8/04 1:58 PM Page 25
FIGURE 1.4 ■ (A color version of this figure follows page 146.) (A) Front and lateral skull views
with the projection of the five final vectors (V). The two red lines show the position of the two
sections seen in (B). (B) Skull sections 2 and 3: examples of two bottleneck levels at which the
viability of vectors was checked. Next to each section is an enlargement of the critical area.
Abbreviations: T, missing tooth; M, intact mandible; Z, intact zygoma with a chipped area (light blue).
loss because inspection of the bone in this region which, at each level, did not violate the following
revealed that it must have been separated com- rules: The vectors representing the trajectories
pletely from the rest of the calvarium (Figure 1.1). could not be closer than 1.5 cm from the mid-
Furthermore, the wound was described as an in- thickness of the zygomatic arch, 1 cm from the
verted funnel (1). We tested 16 points within the last superior molar, and 0.5 cm from the coronoid
rectangular-shaped exit area that we constructed process of the mandible (10). Only seven trajecto-
(Figure 1.3). ries satisfied these conditions (Figure 1.4). Two of
The trajectory connecting each of the entry and those seven invariably hit the anterior horn of the
exit points was tested at multiple anatomical lev- lateral ventricle and were therefore rejected as
els. The three-dimensional skull was resampled in anatomically improbable because they would not
planes perpendicular to the best a priori trajectory have been compatible with survival (the resulting
(C in Figs. 1.2 and 1.3). We were helped by sev- massive infection would not have been control-
eral important anatomical constraints. We knew lable in the preantibiotic era). When checked in
that the left mandible was intact; that the zygo- our collection of normal brains, one of the re-
matic arch was mostly intact but had a chipped maining five trajectories behaved better than any
area, at its medial and superior edge, that sug- other relative to the lower constraints and was
gested the rod had grazed it; and that the last su- thus chosen as the most likely trajectory. The final
perior molar socket was intact although the tooth step was to model the five acceptable trajectories
was missing. Acceptable trajectories were those of the iron rod in a three-dimensional reconstruction
RT0996_C01.qxd 11/8/04 1:58 PM Page 26
FIGURE 1.5 ■ (A color version of this figure follows page 146.)

Normal brain fitted with the five possible rods. The best rod is highlighted
in solid white [except for (B), where it is shown in red]. The areas
spared by the iron are highlighted in color: Broca, yellow; motor, red;
somatosensory, green; Wernicke, blue. (A) Lateral view of the brain.
Numbered black lines correspond to levels of the brain section shown in
(C). (D and E) Medical view of left and right hemispheres, respectively,
with the rod shown in white.
of a human brain that closely fit Phineas Gage’s However, the lesion did not involve the mesial as-
assumed brain dimensions (11). Talairach’s stereo- pect of field 6 [the supplementary motor area
tactic warpings were used for this final step. (SMA)]. The frontal operculum, which contains
The modeling yielded the results shown in Broca’s area and includes fields 44, 45, and 47,
Figure 1.5. In the left hemisphere, the lesion in- was also spared, both cortically and in the under-
volved the anterior half of the orbital frontal cortex lying white matter. In the right hemisphere, the le-
(Brodmann’s cytoarchitectonic fields 11 and 12), sion involved part of the anterior and mesial or-
the polar and anterior mesial frontal cortices bital region (field 12), the mesial and polar frontal
(fields 8 to 10 and 32), and the anterior-most cortices (fields 8 to 10 and 32), and the anterior
sector of the anterior cingulate gyrus (field 24). segment of the anterior cingulate gyrus (field 24).
RT0996_C01.qxd 11/8/04 1:58 PM Page 27
The SMA was spared. The white matter core of regardless of the domain (17). This assignment
the frontal lobes was more extensively damaged also agrees with the idea that in non-brain-dam-
in the left hemisphere than in the right. There was aged individuals the separate frontal regions are
no damage outside of the frontal lobes. interconnected and act cooperatively to support
Even allowing for error and taking into consid- reasoning and decision making. The mysteries of
eration that additional white matter damage likely frontal lobe function are slowly being solved, and
occurred in the surround of the iron’s trajectory, it is only fair to establish, on a more substantial
we can conclude that the lesion did not involve footing, the roles that Gage and Harlow played in
Broca’s area or the motor cortices and that it fa- the solution.
vored the ventromedial region of both frontal
lobes while sparing the dorsolateral. Thus, Ferrier
REFERENCES AND NOTES
was correct, and Gage fits a neuroanatomical pat-
tern that we have identified to date in 12 patients 1. J. M. Harlow, Pub. Mass. Med. Soc.. 2, 327 (1868).
2. P. Broca, Bull. Soc. Anthropol. 6, 337 (1865); C. Wemicke,
within a group of 28 individuals with frontal dam-
Der aphasische Symptomencomplex (Cohn und Weigert,
age (12). Their ability to make rational decisions Breslau, Poland, 1874). A remarkable number of basic
in personal and social matters is invariably com- insights on the functional specialization of the human
promised and so is their processing of emotion. brain, from motor function to sensory perception and to
On the contrary, their ability to tackle the logic of spoken and written language, came from the description
an abstract problem, to perform calculations, and of such cases mostly during the second half of the 19th
century. The cases usually acted as a springboard for
to call up appropriate knowledge and attend to it further research, but on occasion their significance was
remains intact. The establishment of such a pat- overlooked, as in the case of Gage. Another such exam-
tern has led to the hypothesis that emotion and its ple is the description of color perception impairment
underlying neural machinery participate in deci- (achromatopsia) caused by a ventral occipital lesion, by
sion making within the social domain and has D. Verrey [Arch. Ophthalmol. (Paris) 8, 289 (1888)].
His astonishing finding was first denied and then ignored
raised the possibility that the participation de- until the 1970s.
pends on the ventromedial frontal region (13). 3. Reasoning and social behavior were deemed inextricable
This region is reciprocally connected with subcor- from ethics and religion and not amenable to biological
tical nuclei that control basic biological regula- explanation.
tion, emotional processing, and social cognition 4. The reaction against claims for brain specialization was in
fact a reaction against phrenological doctrines, the curious
and behavior, for instance, in amygdala and hypo- and often unacknowledged inspiration for many of the
thalamus (14). Moreover, this region shows a high early case reports. The views of E. Dupuy exemplify the
concentration of serotonin S2 receptors in mon- attitude [Examen de Quelques Points de la Physiologie du
keys whose behavior is socially adapted as well as Cerveau (Delahaye, Paris, 1873); M. MacMillan, Brain
Cognit. 5, 67 (1986)].
a low concentration in aggressive, socially unco-
5. D. Ferrier, Br. Med. J. 1, 399 (1878).
operative animals (15). In contrast, structures in 6. The first measurements were those necessary to construct
the dorsolateral region are involved in other do- Gage’s Talairach stereotactic space and deform a three-
mains of cognition concerning extrapersonal dimensional, computerized tomography skull: the maxi-
space, objects, language, and arithmetic (16). mum length of the skull, the maximum height of the skull
above the inion-glabella line, the distance from this line to
These structures are largely intact in Gage-like
the floor of the middle fossa, the maximum width of the
patients, thus accounting for the patients’ normal skull, and the position of the section contour of Gage’s skull
performance in traditional neuropsychologic tests relative to the inion-glabella line. The second measure-
that are aimed at such domains. ments were those necessary to construct the entry and exit
The assignment of frontal regions to different areas: on the top external view, the measure of edges of the
triangular exit hole; on the internal view the distances from
cognitive domains is compatible with the idea
its three corners to the mid-sagittal line and to the nasion;
that frontal neurons in any of those regions may the distance from the borders of the hole to the fracture
be involved with attention, working memory, and lines seen anteriorly and posteriorly to this hole; and the
the categorization of contingent relationships dimensions of the entry hole at the level of the orbit.
RT0996_C01.qxd 11/8/04 1:58 PM Page 28
7. Thin-cut standard computerized tomography image of a ca- 125 (SD, 3.9). The seven brains were fitted with the possi-
daver head obtained at North Carolina Memorial Hospital. ble trajectories to determine which brain areas were in-
8. We introduced the following changes to the method volved. There were no significant differences in the areas
described by P. Fox, J. Perlmutter, and M. Raichle of damage. The modeling we present here was performed
[J. Comput. Assist. Tomogr. 9, 141 (1985)]. We calculated on subject 1600LL (length, 169 mm; height, 115.2 mm;
the mean distance from the anterior commissure (AC) to width, 125.6 mm).
the posterior commissure (PC) in a group of 27 normal 12. Data from the Lesion Registry of the University of Iowa’s
brains and used that distance for Gage (26.0 mm). We also Division of Cognitive Neuroscience as of 1993.
did not consider the AC-frontal pole and the PC-occipital 13. P. Eslinger and A. R. Damasio, Neurology 35, 1731
pole distances as equal because our group of normals had (1985); J. L. Saver and A. R. Damasio, Neuropsychologia
a mean difference of 5 mm between the two measures, and 29, 1241 (1991); A. R. Damasio, D. Tranel, H. Damasio,
Talairach himself did not give these two measurements as in Frontal Lobe Function and Dysfunction, H. S. Levin,
equal [J. Talairach and G. Szikla, Atlas d’Anatomie H. M. Eisenberg, A. L. Benton, Eds. (Oxford Univ. Press,
Stereotaxique du Telencephale (Masson, Paris, 1967); New York, 1991), pp. 217–229; S. Dehaene and J. P.
J. Talairach and P. Tournoux, Co-Planar Stereotaxic Atlas Changeux, Cereb. Cortex 1, 62 (1991).
of the Human Brain (Thieme, New York, 1988)]. We 14. P. S. Goldman-Rakic, in Handbook of Physiology; The
introduced an anterior shift of 3% to the center of the AC- Nervous System, F. Plum, Ed. (American Physiological
PC line and used that point as the center of the AC-PC Society, Bethesda, MD, 1987), vol. 5, pp. 373–401; D. N.
segment. This shift meant that the anterior sector of Pandya and E. H. Yeterian, in The Prefrontal Cortex: Its
Talairach’s space was 47% of the total length and that the Structure, Function and Pathology, H. B. M. Uylings, Ed.
posterior was 53%. We had no means of calculating the (Elsevier, Amsterdam, 1990); H. Barbas and D. N.
difference between the right and left width of Gage’s Pandya, J. Comp. Neurol. 286, 253 (1989).
brain; therefore, we assumed them to be equal. 15. M. J. Raleigh and G. L. Brammer, Soc. Neurosci. Abstr.
9. H. Damasio and R. Frank, Arch. Neurol. 49, 137 (1992). 19, 592 (1993).
10. There were two reasons to allow the vector this close to 16. M. Petrides and B. Milner, Neuropsychologia 20, 249
the mandible: (i) The zygomatic arch and the coronoid (1982); J. M. Fuster, The Prefrontal Cortex (Raven, New
process were never more than 2 cm apart; (ii) we assumed York, ed. 2, 1989); M. I. Posner and S. E. Petersen, Annu.
that, in reality, this distance might have been larger if the Rev. Neurosci. 13, 25 (1990).
mouth were open or if the mandible, a movable structure, 17. P. S. Goldman-Rakic, Sci. Am. 267, 110 (September
had been pushed by the impact of the iron rod. 1992); A. Bechara, A. R. Damasio, H. Damasio, S.
11. The final dimensions of Phineas Gage’s Talairach space Anderson, Cognition 50, 7 (1994); A. R. Damasio,
were as follows: total length, 171.6 mm; total height, Descartes’ Error: Emotion, Reason and the Human Brain
111.1 mm; and total width, 126.5 mm. Comparing these (Putnam, New York, in press).
dimensions to a group of 27 normal subjects, we found 18. We thank A. Paul of the Warren Anatomical Museum for
that in seven cases at least two of the dimensions were giving us access to Gage’s skull. Supported by National
close to those of Phineas Gage [mean length, 169.9 mm Institute of Neurological Diseases and Stroke grant PO1
(SD, 4.1); mean height, 113.6 (SD, 2.3), mean width, NS 19632 and by the Mathers Foundation.
RT0996_C02.qxd 11/8/04 1:59 PM Page 29
R E A D I N G 2
Impairment of Social and Moral Behavior Related

to Early Damage in Human Prefrontal Cortex
•
Steven W. Anderson, Antoine Bechara, Hanna Damasio, Daniel Tranel and
Antonio R. Damasio*
The long-term consequences of early prefrontal cortex lesions occurring before 16 months were
investigated in two adults. As is the case when such damage occurs in adulthood, the two early-
onset patients had severely impaired social behavior despite normal basic cognitive abilities, and
showed insensitivity to future consequences of decisions, defective autonomic responses to
punishment contingencies and failure to respond to behavioral interventions. Unlike adult-onset
patients, however, the two patients had defective social and moral reasoning, suggesting that the
acquisition of complex social conventions and moral rules had been impaired. Thus early-onset
prefrontal damage resulted in a syndrome resembling psychopathy.
Itotnormal
is well established that in adults who have had
development of social behavior, damage
certain sectors of prefrontal cortex produces a
neural and cognitive systems, namely in infancy,
because such cases are exceedingly rare. Informa-
tion about the early onset condition is vital to the
severe impairment of decision-making and dis- elucidation of how social and moral competencies
rupts social behavior, although the patients so af- develop from a neurobiological standpoint. A
fected preserve intellectual abilities and maintain number of questions have arisen in this regard.
factual knowledge of social conventions and First, would early-onset lesions lead to the ap-
moral rules.1–6 Little is known for certain, however, pearance of persistent defects comparable to those
about the consequences of comparable damage seen in adult-onset lesions, or would further de-
occurring before the maturation of the relevant velopment and brain plasticity reduce or cancel
* All of the Department of Neurology, Division of University of Iowa College of Medicine, Iowa City, Iowa
Behavioral Neurology and Cognitive Neuroscience, The 52242, USA.
29
RT0996_C02.qxd 11/8/04 1:59 PM Page 30
the effects of the lesions and prevent the appear- management in residential treatment facilities and
ance of the defects? Second, assuming early-onset the use of psychotropic medication were of no
lesions cause a comparable defect, would there be help. After repeatedly putting herself at physical
a dissociation between disrupted social behavior and financial risk, she became entirely dependent
and preserved factual social knowledge, as seen in on her parents and on social agencies for financial
the adult-onset condition, or would the acquisi- support and oversight of her personal affairs. She
tion of social knowledge at factual level be com- did not formulate any plans for her future and she
promised as well? We addressed these questions sought no employment. Whenever employment
by investigating two young adults who received was arranged, she was unable to hold the job due
focal nonprogressive prefrontal damage before to lack of dependability and gross infractions of
16 months of age. rules. Affect was labile and often poorly matched
to the situation, but superficial social behavior was
unremarkable. She never expressed guilt or re-
Results morse for her misbehavior. There was little or no
evidence that she experienced empathy, and her
The evidence presented here is based on detailed maternal behavior was marked by dangerous in-
histories obtained from medical and school sensitivity to the infant’s needs. She blamed her
records, as well as legal documents, extensive misdeeds and social difficulties on other people,
interviews with the patients’ parents, clinical and and she denied any difficulties with cognition or
experimental cognitive tasks and neuroimaging behavior.
studies. When first seen by us, the second patient (sub-
ject B) was 23 years old. He had undergone resec-
Clinical Evidence tion of a right frontal tumor at age three months.
The first patient (subject A) was 20 years old at the He had an excellent recovery and there were no
time of these studies and was ambidextrous. She signs of recurrence. Developmental milestones
had been run over by a vehicle at age 15 months. were normal and he was left handed. In early
At the time of the accident, she appeared to re- grade school, mild difficulties were noted with
cover fully within days. No behavioral abnormali- behavior control and peer interactions, but he was
ties were observed until the age of three years, not especially disruptive in school or at home. By
when she was first noted to be largely unrespon- age nine, however, he showed a general lack of
sive to verbal or physical punishment. Her behav- motivation, had limited social interactions, usu-
ior became progressively disruptive, so much so ally exhibited a neutral affect and suffered from
that, by age 14, she required placement in the first occasional brief and explosive outbursts of anger.
of several treatment facilities. Her teachers consid- His work habits were poor, and tutoring was rec-
ered her to be intelligent and academically capa- ommended. He was able to graduate from high
ble, but she routinely failed to complete assigned school, but perhaps because of the loss of struc-
tasks. Her adolescence was marked by disruptive ture for daily activities, his behavioral problems
behavior in school and at home (for example, fail- escalated after graduation. Left to himself, he lim-
ure to comply with rules, frequent loud confronta- ited his activities to viewing television and listen-
tions with peers and adults). She stole from her ing to music. His personal hygiene was poor and
family and from other children and shoplifted fre- his living quarters were filthy. He consumed large
quently, leading to multiple arrests. She was ver- quantities of foods with high fat and sugar con-
bally and physically abusive to others. She lied tent, and became progressively more obese. He
chronically. Her lack of friends was conspicuous. also displayed abnormal food choices, for in-
She ran away from home and from treatment facil- stance, eating uncooked frozen foods. Given his
ities. She exhibited early and risky sexual behavior frequent absences, tardiness and general lack of
leading to a pregnancy at age 18. Contingency dependability, he could not hold a job. He showed
RT0996_C02.qxd 11/8/04 1:59 PM Page 31
Impairment of Social and Moral Behavior Related to Early Damage in Human Prefrontal Cortex ■ 31
reckless financial behavior which resulted in large TABLE 2.1. Standardized Neuropsychological Test Data
debts, and engaged in poorly planned petty thiev- Subject A Subject B
ery. He frequently threatened others and occa-
sionally engaged in physical assault. He lied fre- WAIS-R
Information 37 63
quently, often without apparent motive. He had no Digit span 25 37
lasting friendships and displayed little empathy. Arithmetic 37 63
His sexual behavior was irresponsible. He fa- Similarities 37 25
Block design 75 75
thered a child in a casual relationship, and did not Digit symbol 25 25
fulfill his paternal obligations. He was dependent RAVLT
on his parents for financial support and legal Trial 5 78 11
guardianship. He showed no guilt or remorse for 30 min. recall 99 68
his behavior and could not formulate any realistic JLO 40 57
plans for his future. Complex figure test
Both patients were raised in stable, middle- Copy 21 39
30 min. recall 32 66
class homes by college-educated parents who de-
voted considerable time and resources to their WRAT-R
Reading 86 63
children. In neither case was there a family his- Spelling 81 63
tory of neurologic or psychiatric disease, and both Arithmetic 32 58
patients had socially well-adapted siblings whose COWA 43 15
behavior was normal. The neurological evaluation WCST
was normal in both patients, except for their be- Categories 16 16
havioral defects. Persev. errors 1* 88
TOH
Trial 1 7* 7*
Trial 2 1* 51
Neuropsychological Evidence Trial 3 1* 1*
Comprehensive neuropsychological evaluations Trial 4 1* 1*
Trial 5 1* 1*
(Table 2.1) revealed normal performances on
measures of intellectual ability (for example, fund WAIS-R, Wechsler Adult Intelligence Scale-Revised; RAVLT,
Rey Auditory Verbal Learning Test; JLO, Judgment of Line
of general information, ability to repeat and re- Orientation; WRAT-R, Wide Range Achievement Test-Revised;
verse random sequence of digits, mental arith- COWA, Controlled Oral Word Association; WCST, Wisconsin
Card Sorting Test; TOH, Tower of Hanoi. All tests were admin-
metic, verbal reasoning, nonverbal problem solv- istered according to standardized procedures.27,28,29 Test
ing, verbal and visual anterograde memory, performances are represented as percentile scores and
impairment is indicated by an asterisk.
speech and language, visuospatial perception,
visuomotor abilities and academic achievement).
As in the case of patients with adult-onset lesions, Sorting Test, Subject A; Tower of Hanoi, both
the behavioral inadequacy of the two patients with subjects). They also had significant impairments
early-onset lesions cannot be explained by a fail- of social-moral reasoning and verbal generation
ure in basic mental abilities. of responses to social situations (Figure 2.1).
The patients were asked to perform several Moral reasoning was conducted at a very early
cognitive tasks designed to assess their ability to (‘preconventional’) stage, in which moral dilem-
plan and execute multi-step procedures, use con- mas were approached largely from the egocentric
tingencies to guide behavior, reason through so- perspective of avoiding punishment.7 This stage
cial dilemmas and generate appropriate responses of moral reasoning is characteristic of 10-year-olds,
to social situations. Both patients had significant and is surpassed by most young adolescents. The
impairments on these tasks. They failed to show patients demonstrated limited consideration of the
normal learning of rules and strategies from re- social and emotional implications of decisions,
peated experience and feedback (Wisconsin Card failed to identify the primary issues involved in
RT0996_C02.qxd 11/8/04 1:59 PM Page 32
a) Level 3: Postconventional
Achieved by a
Stage 6: Personal commitment to universal moral minority of adults.
principles.
One of 6 adult-
Stage 5: Recognition that moral perspective may onset patients
conflict with law. Consider rights and welfare of all. at this level.
Level 2: Conventional Characteristic

of most adults
Stage 4: Recognition of obligations to society. and adolescents.
The individual is viewed within the system.
Five of 6 adult-
Stage 3: Reliance on the Golden Rule. Be a onset patients
good person in your own eyes and those of others. at this level.
Level 1: Preconventional Characteristic of

most children
Stage 2: Concrete reasoning that, to serve one's under age 9.
own needs, you must recognize other's rights.
Both early-
Stage 1: Egocentric perspective with decisions onset patients
based on avoidance of punishment. at this level.
b) 8 Subject A 1
Frequency of reference
Subject B
relevant responses
to consequences
AOFL
Mean number of
6 NC 0.75
4 0.5
2 0.25
0 0
OTT MEPS ACT
FIGURE 2.1 ■ Social and moral reasoning. (a) Kohlberg Moral Judgment
Task. (b) Social fluency; OTT, optional thinking test; MEPS, means-ends
problem solving; ACT, awareness of consequences.
social dilemmas and generated few response op- response options. They failed to choose options
tions for interpersonal conflicts. Their perform- with low immediate reward but positive long-term
ance was in stark contrast to that of patients with gains; rather, they persisted in choosing response
adult-onset prefrontal damage, who can access options which provided high immediate reward
the ‘facts’ of social knowledge in the format used but higher long-term loss (Figure 2.2).
in the laboratory (verbally packaged, outside of The electrodermal skin conductance response
real life and real time8). (SCR) was used as a dependent measure of
To explore the decision-making process fur- somatic-state activation, according to methods
ther, the patients participated in a computerized described elsewhere.11 After repeated trials, nor-
version of the Gambling Task.9,10 This task mal controls begin to generate anticipatory SCRs
simulates real-life decision-making in the way it when pondering the selection of a risky response
factors uncertainty of rewards and punishments (a response which may lead to long-term punish-
associated with various response options. Unlike ment). However, both patients failed to acquire
normal controls, but precisely as patients with these anticipatory SCRs, although they did show
adult-onset prefrontal lesions, both patients failed normal SCRs to a variety of unconditioned
to develop a preference for the advantageous stimuli. Again, these findings were similar to
RT0996_C02.qxd 11/8/04 1:59 PM Page 33
a) BEHAVIOR b) ANTICIPATORY SCRs c) SCRs TO LOUD SOUND
BAD DECKS BAD DECKS
GOOD DECKS
GOOD DECKS
Total # of cards selected from the bad
Average magnitude of anticipatory

80 5.0
SCRs before card selection (µS)

0.5
decks versus the good decks
Average magnitude of SCRs

70
4.0
to loud sound (µS)

0.4
60
50 3.0
0.3
40
0.2 2.0
30
20 1.0
0.1
10
0 0.0
0
Normals Subject A Subject B Normals Subject A Subject B
Normals Subject A Subject B
(n = 7) (n = 7)
(n = 7)
FIGURE 2.2 ■ Experimental decision-making and psychophysiology. (a) Responses on

the gambling task. (b) Anticipatory skin conductance responses (SCRs). (c) SCRs to an
unconditioned stimulus (sudden onset of 110-dB noise).
those from patients with adult-onset prefrontal and the sparing of at least one dorsolateral pre-
damage.11 frontal sector.
Neuroimaging Evidence
Discussion
The patients were studied with research-proto-
col magnetic resonance imaging, which permit- We begin by acknowledging that our sample was
ted reconstruction of their brains in three dimen- small, but our findings accord with the only two
sions using the Brainvox technique and other recorded instances of patients with early on-
subsequent analysis of their anatomical defects. set frontal lobe damage,13,14 both with life-long
Both patients had focal damage to prefrontal re- behavior dysfunction, although in neither case is
gions, and had no evidence of damage in other there precise neuroanatomical information. (One
brain areas (Figure 2.3). The lesion in subject A case, from 1947, predates modern neuropsycho-
was bilateral and involved the polar and ventro- logical and neuroimaging techniques, and lesions
medial prefrontal sectors. The lesion in subject of the other are not described satisfactorily and
B was unilateral, located in the right prefrontal may not be confined to the prefrontal region.) The
region, and involved the polar sector, both medi- sample is valuable, nonetheless, because of its
ally and dorsally. The lesions of both patients rarity, and the evidence is offered in the hope that
were located in sites whose damage in adults is it calls attention to other existing cases and facili-
known to produce the emotional and decision- tates their study and the extension of the prelimi-
making defects discussed above.2,3,12 Most fre- nary investigation noted here.
quently, these defects are caused by ventrome- In answer to the first question we posed, the
dial and bilateral lesions, but the condition also evidence presented above suggests that patients
has been noted with exclusively right, medial or with early-onset prefrontal lesions in bilateral ven-
lateral prefrontal lesions. The critical issue tromedial or right sectors resembled patients with
seems to be dysfunction in the medial prefrontal comparable adult-onset lesions in a number of
cortices (which can be caused either by direct ways. In early-onset patients, emotional responses
cortical damage or white matter undercutting) to social situations and behavior in situations that
RT0996_C02.qxd 11/8/04 1:59 PM Page 34
a)
subject A
b)
subject B
FIGURE 2.3 ■ Neuroanatomical analysis. (a) 3-D reconstructed

brain of patient I (subject A). There was a cystic formation occupy-
ing the polar region of both frontal lobes. This cyst displaced and
compressed prefrontal regions, especially in the anterior orbital
sector, more so on the left than on the right. Brodmann areas 11, 10
and 9 bilaterally, and 46 and 47 on the left, were involved.
Additionally, there was structural damage in the right mesial orbital
sector and the left polar cortices (Brodmann areas 11, 47 and 10).
(b) 3-D reconstructed brain of patient 2 (subject B). There was ex-
tensive damage to the right frontal lobe, encompassing prefrontal
cortices in mesial, polar and lateral sectors (Brodmann areas 10, 9,
46 and 8.) Both the lateral half of the orbital gyri and the anterior
sector of the cingulate gyrus were damaged. (Brodmann areas 12,
24 and 32.) The cortex of the inferior frontal gyrus was intact
(Brodmann areas 44, 45 and 47), but the underlying white matter
was damaged, especially in the anterior sector.
require knowledge of complex social conventions and into adulthood; second, those behavioral
and moral rules were inadequate. But whereas the defects were more severe in early-onset patients;
early-onset patients were comparable, at first third, the patients could not retrieve complex,
glance, to patients with adult-onset prefrontal socially relevant knowledge at the factual level.
lesions, a comprehensive analysis reveals several The greater severity of impairment in these two
distinctive features. First, the inadequate social subjects was especially notable. The adult-onset
behaviors were present throughout development prefrontal-lesion patients we studied (n 25)
RT0996_C02.qxd 11/8/04 1:59 PM Page 35
generally do not show the sort of antisocial behav- either does not occur or is insufficient to ensure
ior noted in the early-onset patients, for example, adequate social behavior in real-life and real-time
stealing, violence against persons or property. circumstances. Given that early-onset patients
Beyond the acute period, the disruptive behavior failed in both emotionally related and factual
of adult-onset patients tends to be more con- modes of retrieval, it is possible that they never
strained, although impulsiveness and susceptibil- acquired socially relevant knowledge, either in
ity to immediately present environmental cues emotional or factual modes, and that their pro-
leave them at risk of violating the rights of others. found behavioral inadequacy is explained by an
More often than not, the victims are the adult- absence of the diverse knowledge base necessary
onset patients themselves, not others, and their so- for social and moral behavior.
cial and moral ineptitude can hardly be described The cognitive and behavioral defects present in
as antisocial. these patients arose in the context of stable social
Patients with impairments of social behavior environments that led to normal and well-adapted
caused by adult-onset lesions of the pre-frontal social behavior in their siblings. In spite of exten-
cortex acquire varied aspects of socially relevant sive exposure to appropriate social behavior in
knowledge during normal development, and usu- their home and school environments, and in spite
ally have had decades of appropriate application of of the relevant instruction, the patients failed to
such knowledge to social situations before incur- acquire complex social knowledge during the reg-
ring brain damage. As shown here and previously, ular development period. Moreover, they failed to
following lesion onset in adulthood, they can con- respond to programs aimed at correcting their in-
tinue to access socially relevant knowledge at the appropriate behavior during adolescence and
level of declarative facts,5 and they can even solve young adulthood. This is an intriguing finding.
social problems when presented in a laboratory Although comparison of different complex func-
setting, that is, in a verbal format, outside of real tions should be cautious, it is noteworthy that pa-
time. This distinction might explain why the two tients with early damage to language cortices,
patients described here seemed to show less of a including those who undergo ablations of the en-
sense of guilt and remorse relative to their conduct tire left cerebral cortex at ages comparable to
than do adult-onset patients. Admittedly, however, those at which our patients acquired their lesions,
this is a clinical impression, and we have no con- emerge into adolescence and adulthood with lan-
trolled measurement yet to substantiate it. guage defects whose magnitude seems smaller
The mechanisms whereby adult-onset patients than the defects we encounter in the prefrontal pa-
fail in social behaviors are still under investiga- tients described here. That the magnitude of com-
tion, but we have suggested that an important pensation seemed smaller in our patients suggests
mechanism of the defect is the disruption of the that neural systems impaired by their lesions were
systems that hold covert, emotionally related critical for the acquisition of social knowledge, at
knowledge of social situations.2,9 Emotionally re- least in the manner in which that acquisition tradi-
lated knowledge is presumed to bias the reasoning tionally occurs. It is possible, for instance, that by
process covertly, namely, by enhancing attention destroying a critical cortical control for the pun-
and working memory related to options for action ishment and reward system, the acquisition of
and future consequences of choices, as well as to knowledge that depends on the coordinated con-
bias the process overtly, by qualifying options for tributions of punishment and reward situations
action or outcomes of actions in emotional terms. becomes severely compromised. Should this be
When emotionally related knowledge, covert or the case, it is possible that other neural systems
overt, is no longer available or cannot be re- might be recruited for the learning and processing
trieved, as shown in experiments involving failure of social knowledge, provided appropriate behav-
of anticipatory psychophysiological responses,10,11 ioral or pharmacological interventions could be
the declarative recall of socially relevant facts developed. For example, cognitive-behavioral
RT0996_C02.qxd 11/8/04 1:59 PM Page 36
strategies that rely on a different balance of pun- not only additional studies in humans, relying on
ishment and reward contributions might prove both lesions and functional neuroimaging, but
successful, and administration of neuromodula- also experimental studies in developing animals,
tors such as serotonin and dopamine might con- such as those demonstrating defects in social in-
ceivably help those interventions. teractions of neonate monkeys with lesions of the
The cognitive and behavioral profiles resulting amygdala and inferotemporal cortex.22
from early prefrontal damage resembled, in several
respects, the profiles resulting from adult-onset
damage. Unlike adult-onset patients, however, Methods
early-onset patients could not retrieve complex
social knowledge at the factual level, and may The behavioral histories were based on evidence
never have acquired such knowledge. Overall, the obtained from medical and school records and
profiles of early-onset patients bore considerable legal documents, as well as extensive interviews
similarity to those of patients with psychopathy or with the patients’ parents. Participants in this
sociopathy (‘Conduct Disorder’ or ‘Antisocial research provided informed consent in accord
Personality Disorder’, according to DSM-IV with the policies of the Institutional Review Board
nosology15), another early onset disorder charac- of the University of Iowa College of Medicine.
terized by a pervasive disregard for social and Neuroimaging analysis was conducted by an in-
moral standards, consistent irresponsibility and a vestigator blind to neuropsychological informa-
lack of remorse. Psychopathy may be associated tion, on the basis of thin-cut T1 weighted mag-
with dysfunction in prefrontal regions,16–18 espe- netic resonance (MR) images using Brainvox.23,24
cially in persons without predisposing psychoso- Comprehensive clinical neuropsychological
cial risk factors.18 Also of note, children with anti- evaluations were conducted according to stan-
social tendencies have deficiencies of moral dardized procedures (Table 2.1). Assessment of
reasoning relative to age-matched controls,19,20 social knowledge and moral reasoning was based
and abnormal psychophysiological arousal and on four measures, Standard Issue Moral Judge-
reactivity are found in adults with antisocial be- ment (SIMJ),7 the Optional Thinking Test
havior.21 The behavior of our patients differed (OTT),25 the Awareness of Consequences Test
from the typical profile of psychopathy in that our (ACT),25 and the Means-Ends Problem Solving
patients’ patterns of aggression seemed impulsive Procedure (MEPS).26 All of these procedures in-
rather than goal-directed, and also in the highly volve standardized verbal presentation to the sub-
transparent, almost child-like nature of their ject of moral dilemmas or social situations, and
transgressions and their attempts to cover them. require verbal responses.
In conclusion, early dysfunction in certain sec- In the SIMJ task, a subject is presented with a
tors of prefrontal cortex seems to cause abnormal conflict between two moral imperatives (a man
development of social and moral behavior, inde- must steal a drug in order to save his wife’s life).
pendently of social and psychological factors, The subject is asked to describe the protagonist’s
which do not seem to have played a role in the proper actions and their rationale through a series
condition of our subjects. This suggests that anti- of standard questions (for example, “Should he
social behavior may depend, at least in part, on steal the drug?”, “Is it right or wrong for him to
the abnormal operation of a multi-component steal it?” or “Why do you think that?”). Responses
neural system which includes, but is not limited were scored according to explicit criteria to allow
to, sectors of the prefrontal cortex. The causes of staging of specific levels of moral development.
that abnormal operation would range from prima- The OTT is designed to measure the ability to
rily biological (for instance, genetic, acting at the generate alternative solutions to hypothetical so-
molecular and cellular levels) to environmental. cial dilemmas (for instance, two people disagree
Further clarification of these questions requires on what TV channel to watch). A series of probes
RT0996_C02.qxd 11/8/04 1:59 PM Page 37
are used to elicit as many potential solutions as a loss of money. The magnitude of the yield of
the subject could produce. The number of discrete each deck and the magnitude and frequency of
relevant alternative solutions is scored. The ACT punishment associated with each deck are con-
is intended to sample a subject’s spontaneous controlled such that choosing from the decks with low
sideration of the consequences of actions. Hypo- initial reward turns out to be the most advanta-
thetical predicaments involving temptation to geous strategy over a long series of selections.9
transgress ordinary rules of social conduct are Subjects are required to make a series of 100 card
presented (for instance, receiving too much selections, but they are not told in advance how
money in a business transaction through a mis- many card selections they will be allowed to make.
take), and the subject must describe how the sce- Cards can be selected one at a time from any deck,
nario evolves, including the protagonist’s and subjects are free to switch from any deck to
thoughts prior to the action and the subsequent another at any time and as often as they wish. The
events. Scoring reflects the frequency with which decision to select from one deck or another is
the likely consequences of response options are largely influenced by schedules of rewards and
considered. The MEPS is intended to measure a punishment. These schedules are pre-programmed
subject’s ability to conceptualize effective means and known to the examiner, but not to the subject.
of achieving social goals. Scoring is based on the They are arranged in such a way that every time a
number of effective instrumental acts described as card is selected from deck A or B, the subject gets
methods of achieving goals in hypothetical sce- $100, and every time a card deck is selected from
narios (for example, how to meet people in a new C or D, the subject gets $50. However, in each of
neighborhood). the four decks, subjects encounter unpredictable
In the Gambling Task, subjects are presented money loss (punishment). The punishment is set to
with four decks of cards (named A, B, C and D) be higher in the high-paying decks, A and B, and
and instructed to select cards from the decks in a lower in the low-paying decks, C and D. In decks
manner to win as much play money as possible. A and B, the subject encounters a total loss of
After each card selection, they are awarded some $1,250 in every 10 cards. In decks C and D, the
money, but certain selections are also followed by subject encounters a total loss of $250 in every
FIGURE 2.4 ■ Control subjects with adult-onset prefrontal damage.

The overlap of lesions in the 6 patients with adult-onset lesions is
depicted on a normal reference brain. Lesions of individual sub-
jects were transferred onto the reference brain using MAP-3.24
Darker shade indicates a higher number of overlapping subjects.
The areas involved include all sectors damaged in the target
subjects.
RT0996_C02.qxd 11/8/04 1:59 PM Page 38
10 cards. In the longer term, decks A and B are dis- 6. Stuss, D. T. & Benson, D. F. The Frontal Lobes (Raven,
advantageous because they cost more (a loss of New York, 1986).
7. Colby, A. & Kohlberg, L. The Measurement of Moral
$250 in every 10 cards). Decks C and D are ad- Judgment (Cambridge Univ. Press, New York, 1987).
vantageous because they result in an overall gain 8. Saver, J. & Damasio, A. R. Preserved access and process-
in the end (a gain of $250 in every 10 cards6). ing of social knowledge in a patient with acquired
The methods for the psychophysiological sociopathy due to ventromedial frontal damage.
recordings (Figure 2.2) are described.11 Response Neuropsychologia 29, 1241–1249 (1991).
9. Bechara, A., Damasio, A. R., Damasio, H. & Anderson,
selection in the gambling task was temporally S. W. Insensitivity to future consequences following
linked by computer to ongoing SCR recordings, damage to human prefrontal cortex. Cognition 50, 7–15
and SCRs generated in the four seconds before (1994).
behavioral response selection were considered to 10. Bechara, A., Damasio, H., Tranel, D. & Damasio, A. R.
be anticipatory responses. The normal control sub- Deciding advantageously before knowing the advanta-
geous strategy. Science 275, 1293–1295 (1997).
jects (three male, four female) were matched to the 11. Bechara, A., Tranel, D., Damasio, H. & Damasio, A. R.
target subjects for age and education. The control Failure to respond autonomically to anticipated future
subjects with adult onset prefrontal damage (three outcomes following damage to prefrontal cortex. Cereb.
male, three female) were selected from our data- Cortex 6, 215–225 (1996).
base on the basis of lesion location, in order to 12. Damasio, A. R. & Anderson, S. W. in Clinical Neuropsy-
chology, 3rd edn. (eds. Heilman, K. M. & Valenstein, E.)
provide representation of adult-onset damage to 409–460 (Oxford Univ. Press, New York, 1993).
prefrontal areas including, and more extensive 13. Ackerly, S. S. & Benton, A. L. Report of a case of bilateral
than, the areas of damage in the early-onset cases frontal lobe defect. Assoc. Res. Nerv. Ment. Dis. 27,
(Figure 2.4). Lesions were due to a vascular event 479–504 (1947).
(n 3) or resection of a meningioma (n 3). Age 14. Price, B. H., Daffner, K. R., Stowe, R. M. & Mesulam,
M. M. The comportmental learning disabilities of early
of lesion onset ranged from 26 to 51 years, and sub- frontal lobe damage. Brain, 113, 1383–1393 (1990).
jects were studied at least one year following onset. 15. American Psychiatric Association. Diagnostic and
Statistical Manual of Mental Disorders 4th edn. (APA,
Washington, District of Columbia, 1994).
Acknowledgments 16. Deckel, A. W., Hesselbrock, V. & Bauer, L. Antisocial
personality disorder, childhood delinquency, and frontal
Supported by the National Institute of Neurological brain functioning: EEG and neuropsychological findings.
Diseases and Stroke Grant PO1 NS19632 and the J. Clin. Psychol. 52, 639–650 (1996).
17. Kuruoglu, A. C. et al. Single photon emission comput-
Mathers Foundation.
erised tomography in chronic alcoholism. Br. J. Psychiatry
169, 348–354 (1996).
18. Raine, A., Stoddard, J., Bihrle, S. & Buchsbaum,
REFERENCES
M. Prefrontal glucose deficits in murderes lacking psy-
1. Damasio, A. R., Tranel, D. & Damasio, H. in Frontal Lobe chosocial deprivation. Neuropsychiatry Neuropsychol.
Function and Dysfunction (eds. Levin, H. S., Eisenberg, Behav. Neurol. 11, 1–7 (1998).
H. M. & Benton, A. L.) 217–229 (Oxford Univ. Press, 19. Campagna, A. F. & Harter, S. Moral judgment in socio-
New York, 1991). pathic and normal children. J. Pers. Soc. Psychol. 31,
2. Damasio, A. R. Descartes’ Error. (Grosset/Putnam, New 199–205 (1975).
York, 1994). 20. Blair, R. J. R. Moral reasoning and the child with
3. Damasio, A. R. The somatic marker hypothesis and the psychopathic tendencies. Pers. Individ. Diff. 22, 731–739
possible functions of the prefrontal cortex. Philos. Trans. (1997).
R. Soc. Lond. B Biol. Sci. 351, 1413–1420 (1996). 21. Scarpa, A. & Raine, A. Psychophysiology of anger and
4. Grafman, J. in Structure and Functions of the Human violent behavior. Psychiatr. Clin. North Am. 20, 375–394
Prefrontal Cortex (eds. Grafman, J., Holyoak, K.J. & (1997).
Boller, F.) 337–368 (1995). 22. Newman, J. D. & Bachevalier, J. Neonatal ablations of the
5. Shallice T. & Burgess, P. W. Deficits in strategy applica- amygdala and inferior temporal cortex alter the vocal
tion following frontal lobe damage in man. Brain 114, response to social separation in rhesus macaques. Brain
727–741 (1991). Res. 758, 180–186 (1997).
RT0996_C02.qxd 11/8/04 1:59 PM Page 39
23. Damasio, H. & Frank, R. J. Three-dimensional in vivo 26. Platt, J. J. & Spivack, G. Manual for the Means-Ends
mapping of brain lesions in humans. Arch. Neurol. 49, Problem Solving Procedure (Widener University Institute
137–143 (1992). for Graduate Psychology, Chester, Pennsylvania, 1975).
24. Frank, R. J., Damasio, H. & Grabowski, T. J. Brainvox: 27. Lezak, M. Neuropsychological Assessment 3rd edn.
An interactive, multimodal visualization and analysis (Oxford Univ. Press, New York, 1995).
system for neuroanatomical imaging. Neuroimage 5, 28. Davis, H. P., Bajsjar, G. M. & Squire, L. R. Tower of Hanoi
13–30 (1997). Test—Colorado Neuropsychology Tests Version 2.0.
25. Platt, J. J. & Spivack, G. Measures of Interpersonal (Western Psychological Services, Los Angeles, 1995).
Problem-Solving for Adults and Adolescents (Department 29. Heaton, R. K. et al. Wisconsin Card Sorting Test Manual
of Mental Health Sciences, Hahnemann Medical College, (Psychological Assessment Resources, Odessa, Florida,
Philadelphia, 1977). 1993).
RT0996_C02.qxd 11/8/04 1:59 PM Page 40
RT0996_P03.qxd 11/8/04 2:13 PM Page 41
PA R T 3
Dissociable Systems for Attention,

Emotion, and Social Knowledge
•
The readings in the previous section suggested that the human brain is not
simply a dispassionate information processing organ but an organ
responsible for socioemotional information processing, as well. A premise
underlying work in cognitive and social neuroscience is that there is a
close, causal, relation between information processing operations (both
cognitive and affective) and the operation of specific nervous system
circuits. Consequently, lesions of circumscribed areas of the brain may
cause loss or alterations of specific mental or nervous functions in
humans, and activity in circumscribed areas of the brain may reflect
specific mental or nervous functions in humans. Although there are often
theoretical disputes regarding what precisely constitutes a mental function,
or the system responsible for a mental function, there is general agreement
regarding several broad domains of knowledge that can be distinguished
from one another (e.g., person knowledge, object knowledge). Recent
brain imaging studies have been applied to these domains of knowledge to
examine whether processing items from different domains of knowledge
are served by separable networks in the human brain.
We begin with a reading by Yamasaki, LaBar, and McCarthy (2002) on
the separability of prefrontal systems for attention and emotion. Their
essay illustrates how it is just as important to correctly specify the
psychological functions as it is to correctly measure the associated regions
41
RT0996_P03.qxd 11/8/04 2:13 PM Page 42
of brain activation. They report evidence not only superior temporal sulcus—areas discussed in Part 1,
suggesting that attention and emotion are governed the Volume Overview.
by different systems, but that attentional processing Humans are not only endowed with the capacities
involves areas in the dorsal prefrontal cortex, for attention, emotion, and social perception but
emotional processing involves areas in the ventral also with a sense of fairness and justice that
prefrontal cortex, and that these streams of saturates interpersonal interactions and civilized
information processing are integrated in the anterior societies. Moll et al. (2002) explore whether
cingulate gyrus. emotional moral reasoning and equally emotional
In the second reading, Mitchell, Heatherton, and but nonmoral reasoning involve dissociable neural
Macrae (2002) examine whether semantic networks. Their results accorded a special role in
knowledge about objects are represented or moral reasoning to structures involved in other
processed differently in the brain than semantic forms of social cognition, namely, the medial
knowledge about individuals. Mitchell and orbitofrontal cortex, temporal pole, and superior
colleagues found that person knowledge and object temporal sulcus (see Part 1, Volume Overview).
knowledge are subserved by distinct regions of the Together, the readings in this section raise the
brain and, further, that social knowledge involves possibility that there may be something special
regions including the medial prefrontal cortex and about social cognition.
RT0996_C03.qxd 11/8/04 2:00 PM Page 43
R E A D I N G 3
Dissociable Prefrontal Brain Systems for

Attention and Emotion
•
Hiroshi Yamasaki*†, Kevin S. LaBar‡, and Gregory McCarthy*§
The prefrontal cortex has been implicated in a variety of attentional, executive, and mnemonic
mental operations, yet its functional organization is still highly debated. The present study used
functional MRI to determine whether attentional and emotional functions are segregated into
dissociable prefrontal networks in the human brain. Subjects discriminated infrequent and
irregularly presented attentional targets (circles) from frequent standards (squares) while novel
distracting scenes, parametrically varied for emotional arousal, were intermittently presented.
Targets differentially activated middle frontal gyrus, posterior parietal cortex, and posterior
cingulate gyrus. Novel distracters activated inferior frontal gyrus, amygdala, and fusiform gyrus,
with significantly stronger activation evoked by the emotional scenes. The anterior cingulate gyrus
was the only brain region with equivalent responses to attentional and emotional stimuli. These
results show that attentional and emotional functions are segregated into parallel dorsal and ventral
streams that extend into prefrontal cortex and are integrated in the anterior cingulate. These
findings may have implications for understanding the neural dynamics underlying emotional
distractibility on attentional tasks in affective disorders.
Keywords: novelty, prefrontal cortex, amygdala, cingulate gyrus.
*Brain Imaging and Analysis Center, Duke University Medical 444–0806, Japan; ‡Center for Cognitive Neuroscience, Duke
Center, Durham, NC 27710; †Department of Integrative Phys- University, Durham, NC 27708; and §Department of Veterans
iology, National Institute for Physiological Sciences, Okazaki Affairs Medical Center, Durham, NC 27705.
43
RT0996_C03.qxd 11/8/04 2:00 PM Page 44
he prefrontal cortex (PFC) is a heterogeneous Methods

Tcontributes
brain region whose expansion in primates
to increased flexibility and control of Thirteen right-handed neurologically healthy
cognition and comportment. Whether the PFC is subjects participated in the study. All subjects
divided into domain-specific regions has come provided written informed consent for a protocol
under close scrutiny. A traditional approach to this approved by the Duke University Institutional
question has involved contrasting spatial versus Review Board. Before analysis, data from three
object processing to determine whether the PFC is subjects were discarded because of excessive
organized along a dorsal-ventral axis analogous to head movement. The remaining 10 subjects (four
posterior visual neocortex (1). However, electro- males) ranged in age from 20 to 22 yr.
physiological studies in monkeys (2, 3) and
neuroimaging studies in humans (4–7) have pro- Experimental Design
duced conflicting evidence for such a functional An imaging session consisted of 10 runs, each
parcellation. containing 132 stimuli presented singly at the
An alternative organization of PFC has been center of a back-projection screen with an onset-
proposed in recent neuroanatomical models. to-onset interval of 3,000 ms and a duration of
Mayberg (8) postulated that ventral regions of 2,000 ms. A fixation cross was presented in the
PFC are specialized for “vegetative-somatic” interval between stimuli. Standards consisted of
functions, whereas dorsal regions are specialized squares of varying sizes and colors and were pre-
for “attentional-cognitive” functions. This model sented on 84% of trials. Targets consisted of cir-
further posits that the rostral anterior cingulate cles of varying sizes and colors. Emotional dis-
gyrus acts as an interface between the two pro- tracters consisted of pictures selected primarily
cessing streams. Mood disorders are hypothe- from the International Affective Picture System
sized to reflect failure of coordinated interaction (IAPS; University of Florida, Gainesville, FL)
among these PFC compartments. Other anatomi- and included unpleasant themes of human vio-
cal models support the distinction between a dor- lence, mutilation, and disease. Neutral distracters
sal attentional control system and a ventral emo- consisted of pictures of ordinary activities and
tional arousal system that relay information from were equated to the emotional distracters with re-
posterior parietal cortex and amygdala into dor- spect to mean luminance, chromatic features, and
sal and ventral sectors of the PFC, respectively overall complexity of the scene. All distracters
(9, 10). contained human figures and were chosen on the
In the present study, functional MRI (fMRI) basis of 9-point arousal (1 low/9 high) and
was used to test whether attentional and emo- valence (1 negative/9 positive) scales pro-
tional functions are compartmentalized into vided in the IAPS norms and in a pilot group of
distinct prefrontal systems in the human brain. An undergraduate students. The range of arousal rat-
attention-demanding target detection task (“visual ings for the distracters was as follows: emotional,
oddball” paradigm) was modified from our previ- 5–8; neutral, 1–3. The range of valence ratings
ous studies in which subjects discriminated rare was as follows: emotional, 1–3; neutral, 4–6.
targets embedded in a stream of frequent standard Thus, the ratings for the chosen pictures did not
stimuli (11, 12). Responses to the attentional tar- overlap across the stimulus categories. No indi-
gets were segregated from responses to two cate- vidual distracter or target was repeated in an
gories of trial unique task-irrelevant distracters imaging session. Targets, emotional distracters,
presented intermittently and distinguished by and neutral distracters comprised ≈8, 4, and 4%,
their emotional salience. The distracter categories respectively, of the stimuli in a given list. Succes-
were equated for presentation frequency and other sive targets and distracters were pseudorandomly
aspects of stimulus novelty that could potentially distributed and separated by a 12- to 51-s interval
drive activation of PFC. (mean 18s). A total of 106 targets and 50 each of
RT0996_C03.qxd 11/8/04 2:00 PM Page 45
Dissociable Prefrontal Brain Systems for Attention and Emotion ■ 45
the emotional and neutral stimuli were presented realignment to the TR onset. Epochs synchro-
in a session. nized to the onset of targets, emotional distracters,
Subjects were required to press a button with and neutral distracters were extracted from the
the right index finger on detecting a circle (atten- continuous time series of image volumes follow-
tional target) and to press another button with the ing the method of Kirino et al. (12). Epochs
right middle finger for all other stimuli. Subjects containing two images preceding and five images
were also required to silently count the number of following each stimulus type were segregated and
targets presented during each list and to report averaged. The average MR signal values were
that count at the list’s conclusion. Stimuli were converted to percent signal change relative to the
projected on a 10-in-wide screen located within 6-s prestimulus baseline.
the open magnet bore directly behind the subject’s The primary analysis was based on anatomical
head. Subjects viewed the stimuli through mir- regions of interest (ROIs) drawn on each subject’s
rored glasses. Behavioral responses were ac- high-resolution coronal structural images. These
quired by using a fiber optic button box. Reaction ROIs included the superior frontal gyrus, middle
times and accuracy were measured by customized frontal gyrus, inferior frontal gyrus, cingulate
experimental control software. gyrus, superior temporal gyrus, amygdala, hip-
pocampus, intraparietal sulcus, supramarginal
MRI Acquisition gyrus, and fusiform gyrus in each hemisphere. The
group-averaged data showed no significant activa-
Images were acquired by using a 1.5-T General
tion in the superior frontal gyrus, superior temporal
Electric Signa NVi scanner equipped with
gyrus, or hippocampus, so these ROIs were not
41-mT/m gradients. The subject’s head was im-
considered further. Following the method of Jha
mobilized by using a vacuum cushion and tape.
and McCarthy (13) (see their figures 2 and 3), each
The anterior (AC) and posterior commissures
ROI was drawn on a slice-by-slice basis, and each
(PC) were identified in the midsagittal slice of a
slice was indexed relative to the AC so that the dis-
localizer series, and 34 contiguous slices were
tribution of activation within a ROI could be evalu-
prescribed parallel to the AC-PC plane for high-
ated and summarized across subjects. For example,
resolution T1-weighted structural images [repeti-
ROIs for the major gyri of the PFC were drawn on
tion time (TR) 450 ms; echo time (TE) 20 ms;
eight slices ranging from 7.50 to 33.75 mm anterior
field of view (FOV) 24 cm; matrix 2562;
to the AC. Mean signal change for all voxels within
slice thickness 3.75 mm] and gradient echo
each ROI was then computed for each time point
echoplanar images (TR 3s; TE 40 ms;
and plotted to visualize the hemodynamic response
FOV 24 cm; matrix 642; flip angle 90º;
profile for each ROI during each stimulus condi-
slice thickness 3.75 mm; resulting in 3.75-mm3
tion. The percent signal change at time points 3, 6,
isotropic voxels) sensitive to blood oxygenation-
9, 12, and 15 s poststimulus for each ROI was ana-
level-dependent contrast. An additional series of
lyzed by repeated-measures ANOVA followed by
oblique T1-weighted structural images perpendi-
post hoc analyses using the Student-Newman-
cular to the AC-PC were also acquired by using
Keuls test to further evaluate main effects due to
the parameters specified above.
stimulus category (target, emotional distracter,
neutral distracter). An α level of 0.05 was used to
fMRI Data Analysis determine significant activity in all contrasts.
Head motion was detected by center of mass In addition to the primary ROI analysis, a
measurements, and the data of three subjects were secondary voxel-based analysis was performed.
discarded because of head motion greater than After corrections for motion and temporal align-
3 mm. Compensation for the interleaved slice ment, each subject’s time series of whole-brain
acquisition was performed by using cubic spline volumes was coregistered to a standard echoplanar
interpolation of each voxel’s time course with template by using SPM99 (Wellcome Department
RT0996_C03.qxd 11/8/04 2:00 PM Page 46
of Neurology, London, U.K.). Epochs time- most anterior slices (22.50- to 33.75-mm anterior
locked to stimulus onsets were excised and aver- to the AC) with relatively little activation obtained
aged in the manner specified above, such that more posteriorly. There was a trend for targets to
each subject contributed a mean epoch of volumes evoke larger responses in the right hemisphere
for each of the target, emotional, and neutral cate- (P 0.09).
gories. As the volumes for each subject were in a In marked contrast to the results for the MFG,
common spatial coordinate system, t tests were the ROI analysis of the inferior frontal gyrus
then applied to compare the signal change for (IFG) revealed strong activation by emotional dis-
each voxel over a collapsed 6- to 9-s poststimulus tracters and lesser activation by neutral distracters
period. Contrasts were defined for targets versus at 6 s [F(2,18) 18.55, P 0.0001] and 9 s
emotional distracters and neutral versus emo- [F(2,18) 14.71, P 0.0002] (Figure 3.1C).
tional distracters. This secondary analysis was Post hoc tests showed that at the anterior IFG
performed both as a check on the ROI analysis emotional distracters evoked more robust activity
and to determine whether other brain regions not than neutral distracters that, in turn, evoked
measured in our primary analysis were differen- stronger activity than targets. Responses to emo-
tially influenced by the stimuli. tional distracters were largest in the segment of
the IFG located 18.75–22.50 mm anterior to the
AC, i.e., more posteriorly than the maximum
Results activity in MFG evoked by targets. At this more
Behavioral Performance posterior IFG locus, targets and neutral distracters
evoked little activity (Figure 3.1D). The double
A repeated-measures ANOVA revealed a main dissociation between the role of MFG and IFG
effect of stimulus type [F(3,27) 43.12, relative to attentional targets and emotional dis-
P 0.0001] on reaction time. Post hoc Student- tracters was confirmed as an interaction in a two-
Newman-Keuls analysis showed that reaction way within-subjects ANOVA [F(2,18) 87.60,
times to targets (691 146 ms), neutral dis- P 0.00001].
tracters (680 153 ms), and emotional dis- Qualitative inspection of the data from these
tracters (728 156 ms) were significantly longer PFC regions revealed a surprising feature. Namely,
than to standards (536 157 ms). Reaction times the MFG region activated by targets was deacti-
to emotional distracters were significantly longer vated by emotional distracters, and the anterior IFG
than for all other stimulus types. None of the region activated by distracters was relatively deac-
fMRI activation in our ROIs correlated with reac- tivated by targets (Figures 3.1B and C). Deactiva-
tion times across subjects. tions in these ROIs were confirmed by post hoc
t tests computed to test negative deviations from
fMRI Results: Prefrontal Cortex zero signal change. Bilateral signal suppression in
ROI analysis of the activation profile in the mid- MFG by emotional distracters was significant at 6 s
dle frontal gyrus (MFG) showed a main effect of [(t(9) 3.35, P 0.009]. Bilateral signal sup-
stimulus type at 3 s [F(2,18) 9.86, P 0.0002] pression in IFG by targets was marginally signifi-
and 6 s [F(2,18) 18.71, P 0.0001] (Figure 3.1B). cant at 9 s [t(9) 2.20, P 0.056], which was
Post hoc comparisons revealed that attentional predominantly driven by the left hemisphere
targets elicited a larger signal than either emo- [t(9) 2.76, P 0.02]. A similar trend was ob-
tional or neutral distracters, which did not differ served in left IFG at 6 s [t(9) 1.88, P 0.093].
significantly from each other. In 9 of 10
subjects, targets evoked greater activation in the
right hemisphere. Figure 3.1A shows the anterior- fMRI Results: Other Brain Regions
posterior distribution of MFG activation by targets. Strong and selective activation to targets was also
Targets produced the strongest response in the observed at 6 s poststimulus in posterior parietal
RT0996_C03.qxd 11/8/04 2:00 PM Page 47
a) MFG (target) A-P distribution b) anterior MFG e)

0.15 0.15
⽧ 33.75 䡲 Emotional ✽
fMRI signal change(%)
⽧ novel
● 30 ●
䡲 26.25 䡲䉱
Neutral
䉱 22.5 䉱 novel
⽧
〫 18.75 ● Target
䡩 15 ✽
●
▫ 11.25 〫䉱 ●
䡲 0 ●
䡲
䉭 7.5
⽧䉱 ●
䡲
䉱䡩▫ 䡲
● 䉱
䡲䡲
●
⽧䉭䉱䉱
䡲䉱
0 䉱
⽧
䡩䡲
䉭䉭
䡩
▫ 〫⽧
〫▫ 〫〫䡲䉱
䉱
䡲䡩
䉭䉱〫
䉭䉱
⽧ ▫ 䡩〫䉱
䡩
䡲䡲䡲
䉭
▫ 䉭
䡩
▫ ▫
–3 0 3 6 9 12 15 –3 0 3 6 9 12 15
Time since stimulus onset (s) Time since stimulus onset (s)
c) anterior IFG d) posterior IFG

0.2 Emotional # 0.2
䡲 ✽ Emotional ✽
fMRI signal change(%)
novel 䡲
novel
䡲䡲
䉱 Neutral ✽ 䉱
Neutral
novel novel ✽
● Target 䡲䡲
䉱䉱 ● Target
䉱䡲䉱
●
䡲䉱䡲● 䉱䡲
䉱
䡲 ●
䉱䡲
0 䉱
● 䉱 0 䡲
䉱 ●
䉱䉱
●
䡲 ● ● ● 䉱
● ● ●
䡲
● ●
–3 0 3 6 9 12 15 –3 0 3 6 9 12 15
Time since stimulus onset (s) Time since stimulus onset (s)
FIGURE 3.1 ■ (A color version of part e of this figure follows page 146.) Anterior-posterior (A-P) distribution of
prefrontal cortex activation. (a) MFG activation by attentional targets. Numbers in the box indicate relative distance from
the anterior commissure in mm. (b–d) Mean fMRI signal change (SEM) for the anterior MFG, anterior IFG, and
posterior IFG, respectively. In a–d, data from the right and left hemispheres are collapsed. Note change in vertical scale
across regions. Asterisks indicate time points where (b) targets evoked more activation than distracters, (c) distracters
evoked more activation than targets, and (d) emotional distracters evoked more activation than neutral distracters or
targets. The pound sign in c indicates the time point where emotional distracters evoked more activation than neutral
distracters, which in turn evoked more activation than targets. (e) Group-averaged t test results (P 0.001 uncor-
rected) for the contrast between emotional distracters (plotted in blue spectrum) and attentional targets (plotted in red
spectrum). Attentional target activity was observed in left MFG (BA 9/46; Talairach coordinates 36, 35, 30) and right
MFG (BA 9/46; 44, 35, 31). Emotional distracter activity was observed in left IFG (BA 45/47; 51, 33, 4) and right IFG
(BA 45/47; 55, 33, 0). The coronal section in e shows the single prefrontal slice where differential activation between
attentional targets and emotional distracters was most remarkable. However, peak activation to emotional distracters
was located 1 cm more posteriorly within IFG. R, right hemisphere; L, left hemisphere.
cortex, including the intraparietal sulcus [F(2,18) Emotional distracters also evoked more activation
38.47, P 0.0001] (Figure 3.2A) and supra- in the fusiform gyrus than did targets at 3 s
marginal gyrus [F(2,18) 19.16, P 0.0001] [F(2,18) 14.27, P 0.0002], 6 s [F(2,18)
(Figure 3.2B). The posterior cingulate gyrus was 56.74, P 0.0001] and 9 s [F(2,18) 19.43,
also strongly activated by targets (see Results) P 0.0001] (Figure 3.2E). Post hoc tests re-
(Figure 3.2C). None of these areas showed signif- vealed significantly more fusiform activity to
icant differences between emotional and neutral emotional than neutral distracters at 6 s. The ROI
distracters. analysis did not show any hemispheric asymme-
In contrast to these dorsal areas, ventral brain try effects in these structures.
regions did not respond to target stimuli but To test Mayberg’s (8) hypothesis regarding the
showed differential engagement to the distracters integrative role of the anterior cingulate gyrus,
as a function of their emotional content. Emotional cingulate ROIs were drawn by subdividing the
distracters evoked significant activity in the gyrus into four sectors according to horizontal
amygdala relative to neutral distracters and targets distance from the AC. Each cingulate region
at 6 s [F(2,18) 11.73, P 0.0006] (Figure 3.2D). included four slices. In each region at 6 s,
RT0996_C03.qxd 11/8/04 2:00 PM Page 48
a) IPS b) SMG c) posterior CG

0.3 0.15 0.15
Emotional
fMRI signal change (%)
䡲 ✽ ✽ ✽
novel
● ● ●
䉱
Neutral
novel
● Target ✽ ✽
● ✽ 䡲䡲
✽ 䡲 ● 䡲● ●
●
䡲䡲● 䉱䉱
0 ●
䉱䉱
䉱 0 䡲
●
䉱䉱
● 䉱䡲䡲● 䡲䉱
●
䡲䉱䡲●
䉱
䡲 ●
● 䉱䉱 ●
0 䡲
●
䉱
䉱
● 䡲䡲
䉱 ● 䡲
䉱
䡲䉱䉱䡲
䉱䡲
䉱
䡲
–3 0 3 6 9 12 15 –3 0 3 6 9 12 15 –3 0 3 6 9 12 15
d) AMG e) FFG f) anterior CG

0.3 0.7
✽ # 0.1 ✽
fMRI signal change (%)
䡲䡲
䡲
●
䉱
✽
䡲䡲
✽ 䡲
䉱䡲
䡲䉱
0 䡲䉱 ● 䡲
● 䡲
䡲䡲 ● 䉱
䉱䉱 ●
䉱
䡲 ●
䉱
䉱䉱 ●䡲 ● 䡲䉱 ●
0 ●䉱䡲
●䉱䉱䉱
䡲
䡲䡲 ● ● 0 ●䡲 ●
●䉱䉱䉱䉱 ● ●
● ● ● 䉱
–3 0 3 6 9 12 15 –3 0 3 6 9 12 15 –3 0 3 6 9 12 15
Time since stimulus onset (s) Time since stimulus onset (s) Time since stimulus onset (s)
FIGURE 3.2 ■ Mean signal change (SEM) in posterior brain regions. (Upper) Dorsal regions are presented in the (a)
intraparietal sulcus (IPS), (b) supramarginal gyrus (SMG), and (c) posterior cingulate (CG). (Lower) Ventral regions
are presented in the (d) amygdala (AMG), (e) fusiform gyrus (FFG), and (f) anterior CG (from 18.75 to 7.5 mm rostral
to the AC). Data from the right and left hemispheres are collapsed. Note change in vertical scale across regions.
Asterisks indicate time points where (a–c) targets evoked more activation than distracters, (d) emotional distracters
evoked more activation than neutral distracters or targets, (e) distracters evoked more activation than targets, and
(f) targets and emotional distracters evoked more activation than neutral distracters. The pound sign in e indicates
where emotional distracters elicited stronger responses than neutral distracters, which in turn elicited stronger
responses than targets.
repeated-measures ANOVAs with two independ- emotional or neutral distracters, similar to the
ent variables (stimulus condition and hemisphere) pattern seen in MFG and posterior parietal cortex.
were computed. In range from 18.75 to 7.5 mm
anterior to the AC, a main effect of stimulus type Discussion
was found [F(2,18) 6.66, P 0.007] (Figure 3.2F).
Post hoc comparisons indicated that both Behavioral studies have long shown that emo-
emotional distracters and targets evoked larger tional stimuli can modulate the allocation of
activation than neutral distracters. In range from attentional resources (14, 15). The neural systems
3.75 mm anterior to 7.5 mm posterior to the AC, mediating the interaction between emotional and
similar effects were observed [F(2,18) 14.45, attentional functions, though, have not been well
P 0.0002]. Of all brain regions we examined, characterized. Previous studies have compared
these portions of the cingulate gyrus [correspon- brain activation during attentional tasks to task-
ding to Brodmann’s area (BA) 24] were the only relevant stimuli with different levels of emotional
areas with equivalent responses to attentional meaning, as in the emotional Stroop interference
and emotional stimuli. In range from 11.25 to paradigm (16, 17). These studies have supported a
22.5 mm posterior to the AC, there were no sig- role for the rostral anterior cingulate when a pre-
nificant results. In range from 26.25 to 37.5 mm potent emotional reaction diverts processing re-
posterior to the AC, a main effect of stimulus type sources away from a simultaneous competing
was observed [F(2,18) 14.17, P 0.0003]. In task-appropriate response. The present study took
contrast to anterior regions of the cingulate gyrus, an alternate approach to this topic. Here, subjects
here targets generated larger responses than either performed an attentional target detection task
RT0996_C03.qxd 11/8/04 2:00 PM Page 49
while novel stimuli, parametrically varied for emotional states and motivated behaviors prefer-
emotional arousal levels, were intermittently pre- entially elicit ventromedial PFC, whereas exter-
sented. Making the emotional stimuli task irrele- nally triggered ones depend on lateral regions (21,
vant enabled a dissociation of attentional and 22). The results of the present experiment and our
emotional operations into their constituent net- prior study using auditory cues (23) are consistent
works, while simultaneously revealing where with the latter interpretation. One must keep in
those networks intersected in the brain. Our re- mind, though, that fMRI susceptibility artifacts
sults indicate that these faculties are segregated have precluded observation of a reliable signal in
into dissociable dorsal and ventral processing the medial circuit, and a direct test of these two
streams that extend into the PFC and integrate in accounts remains to be undertaken.
the anterior cingulate gyrus.
Defining the Role of PFC in Stimulus

A Ventrolateral PFC Interface for Novelty and Memory Encoding
Emotional Arousal A number of electrophysiological studies con-
Neuropsychological reports have revealed disso- ducted in normal subjects (24, 25), patients with
ciations across patient populations regarding the prefrontal lesions (26–28), and epilepsy patients
role of dorsal and ventral regions of PFC for with implanted electrodes (29) have implicated
cognitive and emotional functions, respectively the involvement of PFC in novelty detection.
(18, 19). Our findings provide evidence for this Little attention, however, has been paid to the
double dissociation in the healthy human brain. properties of novel stimuli that are critical for
However, in the present study, activation to emo- engaging PFC. Our findings suggest that IFG
tional distracters was strong in ventrolateral rather activation to novel distracters depends on their
than ventromedial PFC, an area that has been em- emotional salience particularly in more posterior
phasized in recent neuropsychological work. regions of the IFG (Figure 3.1D). Trial unique
Neuroanatomical studies of the limbic forebrain task-irrelevant novels that were neutral in emotional
have identified two parallel pathways by which content also activated the anterior IFG, but the sig-
emotionally arousing stimuli processed in the nal change was approximately 50% of that evoked
amygdala potentially interface with PFC (9). The by emotionally arousing novels (Figure 3.1C). The
first pathway is the canonical medial circuit link- neutral and emotional distracters were equated for
ing the basal amygdala with ventromedial at least four aspects of novelty: presentation his-
orbitofrontal cortex (BA 11), rostral insula, and tory (habituation or repetition effects), presenta-
subgenual portions of the anterior cingulated tion frequency (rarity of occurrence relative to
gyrus (BA 25). A second lateral pathway inter- other task events), stimulus complexity (including
connects inferotemporal cortex and basal amyg- presence of human figures), and lower-order per-
dala with ventrolateral PFC (BA 10/47) and ros- ceptual features (distinguishing colors, lumi-
tral anterior cingulate (BA 24/32). The distracters nance, size, etc.). Therefore, these stimulus prop-
in our task engaged the components of this latter erties could not account for the differential
circuit, with increasing levels of activation as a engagement of IFG across novel categories. The
function of stimulus arousal (Figures 3.1 and 3.2). anterior-posterior distribution of IFG activation to
Two accounts have been generated to explain novel scenes— ≈2 cm anterior to the AC—was
the differential engagement of medial versus lat- the same as that seen in our previous study using
eral sectors of ventral PFC during emotional alerting novel sounds (23). In combination, these
tasks. The first hypothesis is that negatively valent results argue for a multimodal representation of
stimuli engage medial sectors, whereas positively sensory cues with high emotional salience in IFG.
valent stimuli engage lateral sectors (20). An al- The foregoing discussion has implications for
ternative hypothesis is that internally generated understanding which features of novel sensory
RT0996_C03.qxd 11/8/04 2:00 PM Page 50
events make them particularly memorable. fMRI signal from the IFG region sensitive to
Several neuroimaging experiments using “subse- emotional distracters was suppressed in response
quent memory” paradigms have shown that stim- to attentional targets (Figure 3.1C). Analogous
uli engaging IFG during their initial encoding are deactivations in posterior ROIs were not ob-
selectively retained over time (30, 31). The quali- served. This pattern supports Drevets and
ties of the stimuli coded in IFG that facilitate Raichle’s (32) observation that neural activity is
memory retention are unknown. The region of reduced in some areas required for emotional pro-
IFG whose activity predicts subsequent recollec- cessing during higher cognitive processes and
tion overlaps with that observed to novel stimuli vice versa.
in the present report. Thus, the engagement of A reciprocal relationship between dorsal and
IFG may reflect an encoding mechanism that pro- ventral PFC may provide a neural substrate for
motes stimuli into long-term storage as a function cognitive—emotional interactions and their dys-
of their arousal value to the individual. However, regulation in mental illness. A hallmark of many
we note that the same posterior IFG region affective disorders is the inability to maintain at-
(Figure 3.1D) was also strongly activated by tentional focus on task-relevant operations in the
arousing environmental sounds (23), such as gun face of prepotent distracting stimuli. Some sub-
shots and breaking glass, suggesting that emo- jects in our sample showed delayed reaction times
tional arousal may be the critical factor in evoking to standards immediately after the emotional dis-
activity in this region. tracters, indicating a more protracted period of
distraction than that seen to the emotional stimuli
Role of Dorsolateral PFC in themselves. This pattern was less prominent to
Attention and Cognition standards after neutral distracters. Thus, both the
behavioral and neural effects of task-irrelevant
Attentional targets evoked significant activation
stimulation were modulated by the arousing qual-
in MFG, in consort with the parietal cortex and
ity of the distracting material rather than to dis-
posterior cingulate gyrus, but novel stimuli did
traction or novelty per se. Our results provide
not, consistent with our prior work (11, 12). The
support for Mayberg’s (8) dual-stream theory of
MFG activation was maximal in a region more
mood regulation, at least in healthy subjects. Fail-
than 3 cm anterior to the AC, similar to that
ure to coordinate the PFC compartments mediat-
observed in our auditory study (23). Thus, both
ing attention, emotion, and their interaction may
the IFG region activated by novels and the MFG
provide a neural substrate underlying emotional
region activated by targets appear to be multi-
distractibility in clinical populations.
modal in nature. The specific task-relevant com-
putations performed within MFG remain unclear.
We have shown activity in this area regardless of Acknowledgments
whether subjects mentally count the targets or re-
This work was supported by Department of
spond to them with a button-press response (12).
Veterans Affairs, National Institute of Mental
Thus, a specific task requirement to remember
Health Grants MH-05286 and MH-60451; the
particular stimuli is not a necessary prerequisite
National Alliance for Research on Schizophrenia
for engaging this area.
and Depression (K.S.L.); and the Japan Founda-
tion for Aging and Health (H.Y.)
A Clinical Model of Emotional Distraction
on Attention-Demanding Tasks
Abbreviations
An unexpected outcome of this study was that the
fMRI signal from the MFG region sensitive to at- PFC, prefrontal cortex; fMRI, functional
tentional targets was suppressed or deactivated in MRI; AC, anterior commissure; PC, posterior
response to emotional distracters. Similarly, the commissure; ROI, region of interest; MFG,
RT0996_C03.qxd 11/8/04 2:00 PM Page 51
middle frontal gyrus; IFG, inferior frontal gyrus; B., Herscovitch, P. & Post, R. M. (1994) Hum. Brain
BA, Brodmann’s area. Mapp. 1, 194–209.
17. Whalen, P. J., Rauch, S. L., Etcoff, N. L., McInerney, S. C.,
Lee, M. B. & Jenike, M. A. (1998) J. Neurosci. 18, 411–418.
REFERENCES 18. Bechara, A., Damasio, H., Tranel, D. & Anderson, S. W.
(1998) J. Neurosci. 18, 428–437.
1. Ungerleider, L. G. & Mishkin, M. (1982) in Analysis of 19. Stuss, D. T., Levine, B., Alexander, M. P., Hong,
Visual Behavior, eds. Ingle, D. J., Goodale, M. A. & J., Palumbo, C., Hamer, L., Murphy, K. J. & Izukawa,
Mansfield, R. J. W. (MIT Press, Cambridge, MA), D. (2000) Neuropsychologia 38, 388–402.
pp. 549–586. 20. Northoff, G., Richter, A., Gessner, M., Schlagenhauf, F.,
2. Wilson, F. A. & Rolls, E. T. (1993) Exp. Brain Res. 93, Fell, J., Baumgart, F., Kaulisch, T., Kötter, R., Stephan,
367–382. K. E., Leschinger, A., et al. (2000) Cereb. Cortex 10,
3. Rao, S. C., Rainer, G. & Miller, E. K. (1997) Science 276, 93–107.
821–824. 21. Chen, Y.-C., Thaler, D., Nixon, P. D., Stern, C. E. &
4. Smith, E. E., Jonides, J., Koeppe, R. A., Awh, E., Passingham, R. E. (1995) Exp. Brain Res. 102, 461–473.
Schumacher, E. H. & Minoshima, S. (1995) J. Cognit. 22. Lane, R. D., Reiman, E. M., Axelrod, B., Yun, L.-S.,
Neurosci. 7, 337–356. Holmes, A. & Schwartz, G. E. (1998) J. Cognit. Neurosci.
5. McCarthy, G., Puce, A., Constable, R. T., Krystal, J. H., 10, 525–535.
Gore, J. C. & Goldman-Rakic, P. (1996) Cereb. Cortex 6, 23. Hinton, S. C., MacFall, J. R. & McCarthy, G. (1999)
600–611. Neuroimage 9, S793.
6. Owen, A. M., Stern, C. E., Look, R. B., Tracey, I., Rosen, 24. Daffner, K. R., Mesulam, M. M., Scinto, L. F. M., Cohen,
B. R. & Petrides, M. (1998) Proc. Natl. Acad. Sci. USA L. G., Kennedy, B. P., West, W. C. & Holcomb, P. J.
95, 7721–7726. (1998) NeuroReport 9, 787–791.
7. D’Esposito, M., Postle, B. R., Ballard, D. & Lease, 25. Opitz, B., Mecklinger, A., Friederici, A. D. & von
J. (1999) Brain Cognit. 41, 66–86. Cramon, D. Y. (1999) Cereb. Cortex 9, 379–391.
8. Mayberg, H. S. (1997) J. Neuropsychiatr. 9, 471–481. 26. Knight, R. T. (1984) Electroencephalogr. Clin. Neuro-
9. Mega, M. S., Cummings, J. L., Salloway, S. & Malloy, physiol. 59, 9–20.
P. (1997) J. Neuropsychiatr. Clin. Neurosci. 9, 315–330. 27. Yamaguchi, S. & Knight, R. T. (1991) Electroen-
10. Mesulam, M.-M., ed. (2000) Principles of Behavioral and cephalogr. Clin. Neurophysiol. 78, 50–55.
Cognitive Neurology (Oxford Univ. Press, New York), 28. Daffner, K. R., Mesulam, M. M., Holcomb, P. J., Calvo,
pp. 1–120. V., Acar, D., Chabrerie, A., Kikinis, R., Jolesz, F. A.,
11. McCarthy, G., Luby, M., Gore, J. & Goldman-Rakic, Rentz, D. M. & Scinto, L. F. M. (2000) J. Neurol. Neuro-
P. (1997) J. Neurophysiol. 77, 1630–1634. surg. Psychiatr. 68, 18–24.
12. Kirino, E., Belger, A., Goldman-Rakic, P. & McCarthy, 29. Baudena, P., Halgren, E., Heit, G. & Clarke, J. M. (1995)
G. (2000) J. Neurosci. 20, 6612–6618. Electroencephalogr. Clin. Neurophysiol. 94, 251–264.
13. Jha, A. P. & McCarthy, G. (2000) J. Cognit. Neurosci. 12, 30. Brewer, J. B., Zhao, Z., Desmond, J. E., Glover, G. H. &
1–16. Gabrieli, J. D. E. (1998) Science 281, 1185–1187.
14. Easterbrook, J. A. (1959) Psychol. Rev. 66, 183–201. 31. Wagner, A. D., Schacter, D. L., Rotte, M., Koutstaal, W.,
15. LaBar, K. S., Mesulam, M.-M., Gitelman, D. R. & Maril, A., Dale, A. M., Rosen, B. R. & Buckner, R. L.
Weintraub, S. (2000) Neuropsychologia 38, 1734–1740. (1998) Science 281, 1188–1191.
16. George, M. S., Ketter, T. A., Parekh, P. I., Rosinsky, 32. Drevets, W. C. & Raichle, M. E. (1998) Cognit. Emot. 12,
N., Ring, H., Casey, B. J., Trimble, M. R., Horwitz, 353–385.
RT0996_C03.qxd 11/8/04 2:00 PM Page 52
RT0996_C04.qxd 11/8/04 2:00 PM Page 53
R E A D I N G 4
Distinct Neural Systems Subserve

Person and Object Knowledge
•
Jason P. Mitchell*, Todd F. Heatherton‡, and C. Neil Macrae‡
Studies using functional neuroimaging and patient populations have demonstrated that distinct
brain regions subserve semantic knowledge for different classes of inanimate objects (e.g., tools,
musical instruments, and houses). What this work has yet to consider, however, is how conceptual
knowledge about people may be organized in the brain. In particular, is there a distinct functional
neuroanatomy associated with person knowledge? By using event-related functional magnetic
resonance imaging (fMRI), we measured neural activity while participants made semantic
judgments about people or objects. A unique pattern of brain activity was associated with person
judgments and included brain regions previously implicated in other aspects of social-cognitive
functioning: medial prefrontal cortex, superior temporal cortex, intraparietal sulcus, and fusiform
gyrus. These regions were generally marked by relatively little change from baseline brain activity
for person judgments along with significant deactivations for object judgments. Together, these
findings support the notion that person knowledge may be functionally dissociable from other
classes of semantic knowledge within the brain.
mong the most intriguing findings in cognitive research have converged on the observation that
A neuroscience is that different categories or
classes of objects are often associated with
perception of, and semantic knowledge about,
particular classes of inanimate stimuli (e.g., tools,
distinct neuroanatomical regions. Both neuro- musical instruments, and houses) are subserved
psychological and functional neuroimaging by distinct areas of the human brain (1–8).
*Department of Psychology, Harvard University, William James and Brain Sciences, Center for Cognitive Neuroscience,
Hall, Cambridge, MA 02138; and ‡Department of Psychological Dartmouth College, Moore Hall, Hanover, NH 03755.
53
RT0996_C04.qxd 11/8/04 2:00 PM Page 54
Although the exact basis of this neuroanatomical ways from semantic knowledge about inanimate
localization remains open to debate, most re- objects, we expect the representation of person
searchers concur that the brain contains some knowledge in the human brain to conform to the
kind of category-specific neural architecture. In- category-specific neural organization observed in
deed, this observation has prompted some theo- object semantics. To this end, we used event-
rists to suggest that the mind may have evolved related functional MRI (fMRI) to test the predic-
dedicated neural circuits to deal with knowledge tion that the brain represents person knowledge in
pertaining to certain categories of objects; specif- a distinct manner from knowledge about inani-
ically, objects that have biological relevance or mate objects. Adopting a paradigm from related
significance to people (e.g., conspecifics, tools, research on the organization of semantic memory,
and plants) (4). The benefits of such a modular the current experiment compared the brain activ-
system reside in the rapid and relatively error-free ity associated with semantic judgments about
manner in which semantic knowledge can be people with that associated with comparable
selected and deployed. Were distinct classes of judgments about inanimate objects.
information to share a similar neuroanatomical
location, interitem competition might compro-
mise (e.g., slow down) the selection process. Materials and Methods
In the current experiment, we examined the
neural substrates of a class of semantic knowl- Fourteen paid volunteers from the Dartmouth
edge that earlier work on category specificity has College community (7 male and 7 female; age
largely ignored, namely, other people. Although range, 18–27) participated in this experiment. All
neural regions that subserve the perception of per- participants were right-handed, native English
sons (e.g., body parts and faces) have been char- speakers with no history of neurological prob-
acterized (9–16), research has yet to investigate lems. All gave informed consent according to the
how the brain represents general knowledge about procedures approved by the Committee for the
the internal, unobservable attributes of social Protection of Human Subjects at Dartmouth
agents. Person knowledge differs from knowledge College. Data from one female participant were
about inanimate objects in a number of potentially discarded because of problems with the acquisi-
important respects. Most obviously, the attributes tion of images during the functional scans.
used to describe persons differ substantially from
those used to describe inanimate objects. Whereas Imaging Procedure
a person may be described as anxious or devious, Imaging was conducted by using a 1.5-tesla GE
inanimate objects rarely engender such a descrip- Signa scanner. An Apple Powerbook G3 com-
tion. One basic feature of person knowledge is puter running PSYSCOPE V.1.2.5 (17) controlled
that it frequently refers to the mental states of oth- stimulus presentation and recorded participants’
ers, states that cannot be directly observed but behavioral responses by means of a keypress
may instead require generalization from one’s interfaced with a PSYSCOPE button box (Carnegie
own internal psychological properties (i.e., theory Mellon University, Pittsburgh). Stimuli were pro-
of mind). Finally, the application of person jected onto a screen at the end of the magnet bore
knowledge demands a flexibility that is typically that participants viewed by way of a mirror
unnecessary for most classes of object knowledge mounted on the head coil. A pillow and foam
(e.g., people must frequently track interactions cushions were placed within the head coil to min-
among independent agents acting in complex imize head movements.
social settings). We first collected a high-resolution T1-
To the extent that: (i) conspecifics are arguably weighted structural scan (SPGR) followed by four
the most important stimulus class to humans; and functional runs of 250 axial scans (20 slices;
(ii) person knowledge differs in several important 5 mm thick; 1 mm skip). Functional images were
RT0996_C04.qxd 11/8/04 2:00 PM Page 55
Distinct Neural Systems Subserve Person and Object Knowledge ■ 55
collected by using a gradient spin-echo echo- (Wellcome Department of Cognitive Neurology).

planar pulse sequence (repetition time 2,000 To allow the magnetic field to reach equilibrium,
ms; echo time 35 ms; flip angle 90º, 3.75 the first four time points (8 s) of each functional
3.75 in-plane resolution). The duration of each run were discarded. Preprocessing included slice
functional run was 8 min and 20 s. timing and motion correction, normalization to
the MN1305 stereotactic space (interpolating to
Behavioral Procedure 3-mm cubic voxels), and spatial smoothing with
an 8-mm Gaussian kernel. An automated segmen-
Participants responded to visually presented noun-
tation algorithm (Stanford University) identified
adjective pairs (4,000 ms duration) by pressing one
gray matter voxels from each participant’s
of two response buttons if the adjective could ever
T1-weighted anatomical scan, and subsequent
be true of the noun (left forefinger) or another but-
statistical modeling was restricted to these voxels.
ton if it could not (right forefinger). Nouns were the
Statistical analyses were performed by using the
name of a person (e.g., David, Emily) or an object
general linear model in which the event-related
from the categories clothing (e.g., glove, shirt) and
design was modeled by using a canonical hemo-
fruit (e.g., grape, mango). Half of the adjectives
dynamic response function and its temporal deriv-
could appropriately describe a person (e.g.,
ative. Comparisons of interest were implemented
assertive, energetic, fickle, nervous) but not any of
as linear contrasts. This analysis was performed
the objects, whereas the remaining half of the ad-
individually for each participant, and contrast im-
jectives could describe one class of objects, but not
ages for each participant were used in a second-
persons or the other class of objects (e.g., clothing:
level analysis treating participants as a random ef-
patched, threadbare; fruit: sundried, seedless). To
fect. Peak coordinates were identified by using a
ensure that participants made use of general seman-
statistical criterion of at least 19 contiguous voxels
tic knowledge about different classes of targets,
exceeding a voxel-wise threshold of P 0.001. A
they were further instructed to decide the appropri-
Monte Carlo simulation (www.wjh.harvard.edu/
ateness of the adjective for hypothetical exemplars
slotnick/scripts.htm) of our brain volume
of the noun (e.g., for a hypothetical person named
demonstrated that this cluster extent cutoff pro-
David, not an individual they might know with that
vided an experimentwise threshold of P 0.05,
name). Each trial began with a fixation cross pre-
corrected for multiple comparisons.
sented for 250 ms. A target noun was then presented
Regions of interest (ROIs) were defined from
alone for 1,000 ms (36-point New York font), after
clusters that survived these thresholding criteria,
which an adjective was also presented (36-point
and peristimulus hemodynamic time courses were
Helvetica font). The noun-adjective pair remained
extracted for each of these ROIs on a participant-
onscreen for an additional 2,750 ms, during which
by-participant basis (representing percent signal
the participant’s behavioral response was recorded.
change in each condition relative to the fixation
Each fMRI run consisted of 50 trials in which the
baseline). One sample, two-tailed t tests (random
adjective was appropriate to the noun and 50 trials
effects, threshold of P 0.01) were used to test
in which it was not. A pseudorandom order of
whether signal change at the time point corre-
trial types and a variable interstimulus interval
sponding to the peak response differed signifi-
(250–6,000 ms) was used to optimize estimation of
cantly from baseline in each region.
the event-related fMRI response (18). During inter-
stimulus intervals, participants passively viewed a
fixation crosshair, which defined the baseline.
Results
MRI Data Analysis Analysis of the reaction time data showed that par-
Preprocessing and statistical analysis of the fMRI ticipants made semantic judgments about Persons
data were performed by using SPM99 software significantly faster than comparable judgments
RT0996_C04.qxd 11/8/04 2:00 PM Page 56
about Objects [means, Ms: 900 ms vs. 1,019 ms,

t(12) 3.92, P 0.002, r 0.49]. In addition,
faster responses were returned on “yes” than
“no” trials [Ms: 870 ms vs. 980 ms, t(12) 6.29,
P 0.0001, r 0.67]; however, because this re-
sponse type factor did not impact on the fMRI
data, imaging analyses were collapsed across
“yes” and “no” trials.
To examine whether judgments about persons
and objects were associated with different pat-
terns of neural activity, we compared the event-
related BOLD (blood oxygen level-dependent)
signal associated with Person trials to that associ-
ated with Object trials. This comparison yielded
distinct patterns of brain activity for each type of
target. Object Person comparisons (Table 4.1
and Figure 4.1) demonstrated greater activity in
left inferior frontal gyrus (LIFG), left inferotem-
poral (IT) cortex, left posterior parietal cortex, left
superior frontal gyrus, and bilateral insula cortex.
LIFG modulation was observed in multiple re-
gions extending the entire extent of the inferior
frontal gyrus. No significant activation differ-
ences were observed between fruits and items of
clothing.
TABLE 4.1. Significant Peak Locations in

Object Person
t value FIGURE 4.1 ■ (A color version of this figure follows
page 146.) Activation maps show brain areas to be more
Anatomic label x y z Object Person active during Object trials than during Person trials.
Regions of modulation included the left inferior prefrontal
R. insula 36 23 6 5.70 †
3.20* cortex and the left IT cortex (a), as well as the left poste-
L. insula 33 24 6 6.42† 4.40†
rior parietal and the left insula cortex (b). See Table 4.1
L. inf. frontal gyrus 53 30 12 3.67* 1.34
50 19 27 4.80† 3.22* for the Talairach and Tournoux (49) atlas coordinates.
50 41 2 3.65* 1.40
50 7 22 5.26† 3.15*
50 24 4 4.34† 1.64 In sharp contrast, Person Object compar-
48 8 36 4.11* 2.56 isons (Table 4.2 and Figure 4.2) were associated
59 8 38 4.29* 3.41* with modulation in a very different set of brain
45 1 33 4.48† 3.04
L. inf. temporal 53 59 5 2.98 0.31 areas that included dorsal and ventral aspects of
50 53 10 4.53† 2.18 the medial prefrontal cortex (MPFC), right intra-
L. post. parietal 30 67 50 5.00† 3.21* parietal sulcus (IPS), right fusiform gyrus (FuG),
L. sup. frontal gyrus 9 17 43 3.96* 2.88
left superior temporal (ST) and medial temporal
Coordinates are from the Talairach and Tournoux atlas (49).
Object and Person columns display the t value associated with
(MT) cortex, left motor cortex, and regions of the
the area’s peak hemodynamic response relative to passive occipital cortex bilaterally.
baseline for Object and Person trials, respectively; *, P 0.01;
†, P 0.001; R, right; L, left; inf, inferior; post, posterior; sup,
To further investigate the neural response in
superior. regions modulated by Object and Person judgments,
RT0996_C04.qxd 11/8/04 2:00 PM Page 57
TABLE 4.2. Significant Peak Locations in Finally, because Person trials were associated
Person Object with significantly faster reaction times than Ob-
t value ject trials, we conducted a secondary analysis to
rule out the possibility that time course differ-
Anatomic label x y z Object Person
ences were spuriously produced by differences in
Dorsal MPFC 0 54 21 5.40 †
1.36 the relative difficulty of the Person and Object
Ventral MPFC 3 39 0 5.55† 3.41* judgments. Specifically, for each functional scan
12 36 0 4.39† 2.81
R. fusiform 30 51 3 2.08 0.92 we selected 20 Object trials and then selected a
R. intraparietal sulcus 63 33 33 3.19* 0.89 subset of 20 Person trials matched to the Object
60 33 21 3.67* 0.46 trials for reaction time. The resulting average re-
R. occipital 48 63 12 3.70* 1.64
L. sup. temporal 60 6 3 0.65 1.61 action time for Object trials (1,051 ms) was nearly
60 12 12 0.45 1.37 identical to that for Person trials (1,047 ms). We
L. med. temporal 66 24 12 1.55 0.25 then reanalyzed the fMRI data by using these tri-
66 18 15 0.78 0.39
L. motor 45 27 63 1.57 2.22 als and subsequently compared the peak signal
30 39 60 0.41 1.49 change for time courses associated with Person
33 36 69 1.12 1.71 and Object trials in each of the regions obtained in
30 27 69 1.46 2.29
L. occipital 51 75 21 3.07* 0.80 the primary analysis. Of the 17 peak activations
15 99 21 2.10 0.96 observed in Person Object comparisons, only a
Coordinates are from the Talairach and Tournoux atlas (49). single region did not continue to demonstrate a
Object and Person columns display the t value associated with significant difference (P 0.05) after matching
the area’s peak hemodynamic response relative to a passive
baseline for Object and Person trials, respectively. Negative t for time on task. This was the left MT region cen-
values represent deactivations relative to baseline; *, P 0.01; tered at 66, 18, 15, which was only margin-
†, P 0.001; L, left; R, right; MPFC, medial prefrontal cortex;
sup, superior; Ed, medial. ally significant, P 0.11. Of the 14 peak activa-
tions observed in Object Person comparisons,
we examined the hemodynamic time course in each all regions continued to demonstrate a significant
of the regions described above (Figure 4.3). Beyond difference after matching for time on task. Differ-
the observed neuroanatomical differences between ences were particularly stable in MPFC (for
Object and Person judgments, these two sets of brain Person Object) and LIFG (for Object
regions were associated with qualitatively different Person), where we obtained a significance level of
hemodynamic responses. Whereas regions identified P 0.01 for all comparisons after matching for
from Object Person comparisons uniformly pro- time on task.
duced signal changes above baseline (Table 4.1 and
Figures 4.3a and b), regions identified from Person
Object comparisons were generally marked by non- Discussion
significant or modest changes from baseline in re-
sponse to Person targets, along with significant deac- The results of this experiment demonstrate a qual-
tivations in response to Objects (Table 4.2 and itative dissociation between the brain areas sub-
Figures 4.3c and d). Indeed, the brain regions associ- serving semantic judgments about people and
ated with activations above baseline for Person trials inanimate objects. This dissociation was evi-
were almost entirely a subset of those demonstrating denced both by the neuroanatomical localization
activations for Object trials, except for some activa- of different brain areas modulated by Object and
tions unique to Person trials in bilateral basal ganglia, Person judgments (Figures 4.1 and 4.2) as well as
bilateral occipital lobe, and left cerebellum. In all by qualitative differences in the nature of the he-
cases, these unique Person trial activations were in modynamic time courses underlying these modu-
voxels adjacent to Object trial activations and ap- lations (Figure 4.3).
peared generally to be serving the motoric and per- Neuroanatomically, Object Person compar-
ceptual demands of the experimental task. isons yielded regions (LIFG, IT, and posterior
RT0996_C04.qxd 11/8/04 2:00 PM Page 58
FIGURE 4.2 ■ (A color version of this figure follows page 146.) Activation maps show brain areas to
be more active during Person trials than during Object trials. Regions of modulation included the left
temporal sulcus (a), the dorsal and ventral MPFC (b), the right FuG (c), and the right parietal temporal-
occipital junction (d). See Table 4.2 for the Talairach and Tournoux (49) atlas coordinates.
parietal cortex) that were highly consistent with parietal cortex during the maintenance of object
earlier neuroimaging research on semantic mem- information before a behavioral response (23) and
ory and object recognition. LIFG activation has during tactile object recognition tasks (24).
consistently been observed across a range of tasks In the same way, Person Object comparisons
that require use of semantic knowledge, including identified regions (dorsal and ventral MPFC, the
object naming and categorization, exemplar gen- IPS, the ST region, and FuG) that converge with
eration, abstract/concrete word decisions, and earlier work suggesting that these areas participate
tests of factual knowledge (19–22). In addition, in a range of social-cognitive tasks (25). For exam-
activation in the IT region was almost identical to ple, a number of studies have observed modulation
that observed in a previous study in which partic- in dorsal MPFC in tasks that require self-monitoring
ipants were required to make semantic judgments or the attribution of mental states to others
about tools versus animals (8), and similar activa- (26–28), as well as during retrieval of personally
tions throughout IT have been associated with a relevant memories (29, 30). In addition, patient
range of object perception/naming tasks (5–7). studies have linked ventral MPFC to activities that
Finally, earlier work has also implicated posterior involve the rapid, flexible use of social knowledge,
RT0996_C04.qxd 11/8/04 2:00 PM Page 59
a) 0.4 b) 0.1
0.35 0.05
䡲
䡲䡲䉭䡲
0.3 0 䉭
䉭䉭
䡲䉭
䡲
Percent signal change

0.25 –0.05
䉭䉭
䉭
0.2 –0.1
0.15 䡲 –0.15 䡲
䉭
0.1 –0.2 䡲
䡲䉭䡲
0.05 䉭䉭䡲 –0.25
䉭䉭䡲
0
䡲䡲
䉭䉭䉭 –0.3
䡲
䡲
–0.05 –0.35
–0.1 –0.4
c) 0.4 d) 0.1
0.35 0.05
䡲
0.3 0 䉭
䡲䉭
䡲䉭䉭
䉭䡲
䉭䡲䡲
0.25 –0.05
䉭
0.2
䡲 –0.1
䡲
0.15 –0.15
䡲䉭
0.1 –0.2
䡲
0.05 –0.25
䉭䡲
䡲䡲䉭䉭
0
䉭䉭䉭
䡲䉭
䡲 –0.3
䉭䡲
–0.05 䉭 –0.35 䡲
–0.1 –0.4
FIGURE 4.3 ■ The hemodynamic time courses in the left inferior prefrontal (a) and the left inferior temporal cor-
tex (c) were characterized by activations above baseline for Object trials (filled squares) and either modest or
nonsignificant activations for Person trials (open triangles). In contrast, the dorsal medial prefrontal (b) and right
lateral parietal cortices (d) were characterized by significant deactivations for Object trials, along with no significant
modulations for Person trials, a pattern typical for areas identified in Person Object comparisons. Time
courses were calculated by collapsing across multiple clusters within a neuroanatomical region (except for the
dorsal medial prefrontal cortex, for which only one cluster was identified). The scale is seconds (one second per
hash mark). Error bars display the standard error of the mean.
such as gender stereotyping (31) and the integra- eye gaze, body parts, and emotional expression
tion of emotion with thoughts and behavior (9–12). Finally, the modulation we observed in
(32–35). Similarly, the IPS, ST, and FuG have con- FuG corresponds closely to a region, dubbed the
sistently been linked to the perception of socially fusiform face area (FFA), that previous research
relevant stimuli, such as eye gaze, biological mo- suggests is selectively responsive to perception of,
tion, body parts, and faces. For instance, several re- imagery for, and identification of faces (13–16,
cent studies have observed IPS modulation when 38). We note that, although participants never re-
people assume the physical perspective of another ported engaging in visual imagery for the person
individual (36) or perceive targets displaying direct trials, these IPS, ST, and FuG modulations may
eye gaze (15, 37), tasks that have been linked to nevertheless have been associated with partici-
theory of mind and its associated cognitive func- pants’ spontaneous generation of mental images of
tions. Likewise, the ST region (the superior tempo- body parts and faces.
ral sulcus and adjacent areas of the superior and in- Importantly, Object and Person trials were also
ferior temporal gyri) has been shown to play an associated with qualitatively different hemody-
important role in the perception of social stimuli, namic time courses. Whereas Object Person
including head and mouth movements, changes in modulations took the form of activations above
RT0996_C04.qxd 11/8/04 2:00 PM Page 60
the baseline, modulations associated with Person consistently been observed relative to some other
Object comparisons were typically deactiva- active task, thereby obscuring whether any
tions. More specifically, these Person Object changes occur relative to a resting baseline (see
modulations were produced by consistent deacti- References 30, 40, and 47 for exceptions in
vations for Object trials along with nonsignificant MPFC). That social thought and perception ap-
or modest modulations for Person trials. As such, pear to be subserved by areas with high resting
these data raise a question about how to interpret metabolic rates suggests that social-cognitive
the observation that Person judgments consis- processes constitute an important component of
tently produced little change from a passive base- the brain’s resting state of activity.
line condition. Because regions with high metabolic rates
One framework in which to consider these are actively engaged during resting states (such as
results originates in the observation that, when at the passive fixation conditions that often define
rest, some brain regions are characterized by rela- the fMRI baseline), they consistently deactivate
tively high rates of metabolic activity (29, 39, 40). across a range of active cognitive tasks (39).
That is, the resting brain consistently assumes a When obliged to perform an active task, the brain
preferred configuration of metabolic activity, with typically suspends baseline processes, producing
some regions routinely displaying higher levels of deactivations in the regions subserving those
activity than others. Interestingly, of the four processes. However, because researchers have fo-
neuroanatomical regions with notably high rest- cused almost exclusively on the functional signif-
ing metabolic rates, three were observed in the icance of activations above baseline, relatively lit-
current study for Person Object comparisons: tle is known about the exact nature of the default
dorsal MPFC, ventral MPFC, and lateral parietal baseline processes taking place during rest. The
areas that include the IPS (medial parietal/ limits imposed by restricting interpretation to ac-
precuneus regions constitute the fourth high- tivations above baseline suggests that a different
metabolism region, which we did not observe analytic approach is required when investigating a
across comparisons in this experiment). region associated with high resting metabolic rate
Recently, Raichle and colleagues (28, 40, 41) and consistent deactivations across tasks. Because
have argued that such tonically high metabolic such regions are tonically engaged in the
rates may reflect high levels of spontaneous men- processes they subserve, they are unlikely to pro-
tal processing that take place during rest. To the duce activations above a resting baseline; one
extent that metabolic activity in a brain region may imagine a neural ceiling effect, of sorts.
corresponds to the engagement of mental opera- Instead, if a region is known to deactivate during
tions subserved by that region, high metabolic many active tasks, then identifying conditions that
rates suggest that some brain regions engage in produce no change from baseline can help to
continuous, active processing during resting characterize the kind of processing operations
states. Intriguingly, three of the four highest rest- that may occur spontaneously during rest.
ing metabolic rates are found in brain regions, More specifically, if one experimental condition
dorsal and ventral MPFC and lateral parietal cor- produces the kind of deactivation typical for a
tex (including the IPS), that have consistently particular region whereas a second condition pro-
been linked to social-cognitive processes such as duces little or no deviation from baseline, one
the simulation of other minds, the flexible use of may tentatively conclude that the processes
social and moral knowledge, self-referent mem- uniquely engaged by the second task overlap (at
ory, emotion regulation, and the perception of so- least in part) with the ongoing, default processes
cially relevant stimuli (15, 25–27, 30–37, 40, of that region.
43–48). Whereas these studies have typically The results of the current study demonstrate
reported MPFC and/or lateral parietal increases, exactly such a pattern in dorsal MPFC, ventral
it is important to note that these increases have MPFC, and lateral parietal regions. That is, Object
RT0996_C04.qxd 11/8/04 2:00 PM Page 61
trials produced significant deactivations across all perception and identification of socially relevant
three of these regions, replicating the deactiva- stimuli in the environment (as IT areas do for ob-
tions that are consistently observed in these reject perception), whereas MPFC areas may repre-
gions. In contrast, Person trials did not signifi- sent more elaborative semantic information about
cantly modulate activity in either dorsal MPFC or the descriptive characteristics or internal mental
lateral parietal cortex/IPS and only modestly states of social agents (akin to the role of LIFG in
modulated activity in ventral MPFC. Taken to- nonsocial semantics). Person knowledge may also
gether, these two characteristics of the observed have a unique contribution from the lateral pari-
MPFC and lateral parietal modulations suggest etal/IPS region, which may partly support theory
that the processes subserving semantic judgments of mind representations (15, 37). While this initial
about social targets overlap considerably with de- study points out the functional neuroanatomical
fault processes engaged during resting baseline. dissociation between the systems supporting ob-
By yielding a significant deactivation below base- ject and person knowledge, continuing research
line, ventral MPFC only partly conformed to the guided by these hypotheses will certainly be
idealized pattern of results. However, the observa- needed to characterize the nature of these systems
tion that Person judgments did not overlap con- more fully.
siderably with baseline ventral MPFC activity
may not be terribly surprising. Whereas ventral
MPFC has been linked to the use of social knowl- Acknowledgments
edge in real time, its role in representing abstract
We thank Lila Davachi, Souheil Inati, Orville
social knowledge seems to be minimal. Indeed,
Jackson, Bill Kelley, Tammy Laroche, Anat Maril,
patients with ventral MPFC lesions can usually
Rebecca Saxe, Scott Slotnick, Anthony Wagner,
articulate social traits and the norms that govern
and Carrie Wyland for their advice and assistance.
social life, although they appear impaired at mak-
This work was supported by National Science
ing appropriate use of such explicit information in
Foundation Grant BCS 0072861 and predoctoral
their everyday life (31).
National Research Service Award F31 NH65053.
In summary, the results of the present investi-
gation suggest that distinct networks of brain re-
gions subserve the representation of semantic Abbreviations
knowledge about people and objects. Although
the areas participating in the representation of ob- fMRI, functional MRI; LIFG, left inferior frontal
ject knowledge have been characterized exten- gyrus; MPFC, medial prefrontal cortex; ST, supe-
sively in both neuropsychological and neuroimag- rior temporal; MT cortex, medial temporal cortex;
ing work, the current study is, to the best of our IT cortex, inferotemporal cortex; FuG, fusiform
knowledge, the first to identify a network of re- gyrus; IPS, intraparietal sulcus.
gions that specifically subserve person knowl-
edge. One interesting future direction will be to REFERENCES
disentangle whether person-sensitive regions re-
1. Caramazza, A. & Shelton, J. R. (1998) J. Cognit.
spond simply to the presence of socially relevant
Neurosci. 10, 1–34.
stimuli or whether they subserve some specialized 2. Martin, A., Wiggs, C. L., Ungerleider, L. G. & Haxby,
cognitive operations brought to bear when thinking J. V. (1996) Nature 379, 649–652.
about people (e.g., representing the internal states 3. Martin, A. (2001) in Handbook of Functional Neuroimag-
of other intentional agents). We speculate that the ing of Cognition, eds. Cabeza, R. & Kingstone, A. (MIT
Press, Cambridge, MA), pp. 153–186.
semantic system for person knowledge may prove
4. Shelton, J. R. & Caramazza, A. (2001) in The Handbook
to rely on a neural architecture structurally similar of Cognitive Neuropsychology: What Deficits Reveal
to that underlying object knowledge: temporal ar- About the Human Mind, ed. Rapp, B. (Psychology
eas such as the ST region and FuG may subserve Press/Taylor & Francis, Philadelphia), pp. 423–443.
RT0996_C04.qxd 11/8/04 2:00 PM Page 62
5. Haxby, J. V., Ishai, A., Chao, L., Ungerleider, L. G. & 28. Gusnard, D. A. & Raichle, M. E. (2001) Nat. Rev. Neu-
Martin, A. (2000) Trends Cognit. Sci. 4, 3–4. rosci. 2, 685–694.
6. Haxby, J. V., Gobbini, M. I., Furey, M. L., Ishai, A., 29. Maguire, E. A. & Mummery, C. J. (1999) Hippocampus 9,
Schouten, J. L. & Pietrini, P. (2001) Science 293, 54–61.
2425–2430. 30. Kelley, W. M., Macrae, C. N., Wyland, C. L., Caglar,
7. Ishai, A., Ungerleider, L. G., Martin, A., Schouten, J. L. & S., Inati, S. & Heatherton, T. F. (2002) J. Cognit. Neu-
Haxby, J. V. (1999) Proc. Natl. Acad. Sci. USA 96, rosci. 14, 785–794.
9379–9384. 31. Milne, E. & Grafman, J. (2001) J. Neurosci. 21, 1–6.
8. Chao, L. L., Haxby, J. V. & Martin, A. (1999) Nat. Neu- 32. Damasio, A. R., Tranel, D. & Damasio, H. C. (1991) in
rosci. 2, 913–919. Frontal Lobe Function and Dysfunction, eds. Levin, H. S.,
9. Allison, T., Puce, A. & McCarthy, G. (2000) Trends Cog- Eisenberg, H. M. & Benton, A. L. (Oxford Univ. Press,
nit. Sci. 4, 267–278. New York), pp. 217–229.
10. Perrett, D. I., Harries, M. H., Mistlin, A. J., Hietanen, 33. Damasio, A. R. (1994) Descartes’ Error (Grosset/Putnam,
J. K., Benson, P. J., Bevan, R., Thomas, S., Oram, M. W., New York).
Ortega, J. & Brierley, K. (1990) Int. J. Comp. Psychol. 4, 34. Anderson, S. W., Bechara, A., Damasio, H., Tranel, D. &
25–55. Damasio, A. R. (1999) Nat. Neurosci. 2, 1032–1037.
11. Puce, A., Allison, T., Bentin, S., Gore, J. C. & McCarthy, 35. Bechara, A., Dolan, S., Denburg, N., Hindes, A., Ander-
G. (1998) J. Neurosci. 18, 2188–2199. son, S. W. & Nathan, P. E. (2001) Neuropsychologia 39,
12. Narumoto, J., Okada, T., Sadato, N., Fukui, K. & 376–380.
Yonekura, Y. (2001) Brain Res. Cognit. Brain Res. 12, 36. Zacks, J., Rypma, B., Gabrieli, J. D. E., Tversky, B. &
225–231. Glover, G. H. (1999) Neuropsychologia 37, 1029–1040.
13. Kanwisher, N., McDermott, J. & Chun, M. M. (1997) 37. Calder, A. J., Lawrence, A. D., Keane, J., Scott, S. K.,
J. Neurosci. 17, 4302–4311. Owen, A. M., Christoffels, I. & Young, A. W. (2002) Neu-
14. McCarthy, G., Puce, A., Gore, J. C. & Allison, T. (1997) ropsychologia 40, 1129–1138.
J. Cognit. Neurosci. 9, 605–610. 38. O’Craven, K. & Kanwisher, N. (2000) J. Cognit. Neu-
15. Hoffman, E. A. & Haxby, J. V. (2000) Nat. Neurosci. 3, rosci. 12, 1013–1023.
80–84. 39. Shulman, G. L., Fiez, J. A., Corbetta, M., Buckner, R. L.,
16. Halgren, E., Dale, A. M., Sereno, M. I., Tootell, R. B., Miezen, F. M., Raichle, M. E. & Petersen, S. E. (1997) J.
Marinkovic, K. & Rosen, B. R. (1999) Hum. Brain Mapp. Cognit. Neurosci. 9, 648–663.
7, 29–37. 40. Gusnard, D. A., Akbudak, E., Shulman, G. L. & Raichle,
17. Cohen, J., MacWhinney, B., Flatt, M. & Provost, J. (1993) M. E. (2001) Proc. Natl. Acad. Sci. USA 98, 4259–4264.
Behavioral Research Methods, Instruments, and Comput- 41. Raichle, M. E., MacLeod, A. M., Snyder, A. Z., Powers,
ers 25, 257–271. W. J., Gusnard, D. A. & Shulman, G. L. (2001) Proc. Natl.
18. Dale, A. M. (1999) Hum. Brain Mapp. 8, 109–114. Acad. Sci. USA 98, 676–682.
19. Cabeza, R. & Nyberg, L. (2000) J. Cognit. Neurosci. 12, 42. Fletcher, P. C., Happé, F., Frith, U., Baker, S. C., Dolan,
1–47. R. J., Frackowiak, R. S. J. & Fritch, C. D. (1995) Cogni-
20. Maril, A., Wagner, A. D. & Schacter, D. L. (2001) Neuron tion 57, 109–128.
31, 653–660. 43. Gallagher, H. L., Happé, F., Brunswick, N., Fletcher, P. C.,
21. Wagner, A. D., Schacter, D. L., Rotte, M., Koustaal, W., Frith, U. & Frith, C. D. (2000) Neuropsychologia 38, 11–21.
Maril, A., Dale, A. M., Rosen, B. R. & Buckner, R. L. 44. George, M. S., Ketter, T. A., Parekh, P. I., Herscovitch,
(1998) Science 281, 1188–1191. P. & Post, R. M. (1996) Biol. Psychiatry 40, 859–871.
22. Gabrieli, J. D. E., Poldrack, R. A. & Desmond, J. E. 45. Goel, V., Grafman, J., Sadato, N. & Hallet, M. (1995)
(1998) Proc. Natl. Acad. Sci. USA 95, 906–913. NeuroReport 6, 1741–1746.
23. Mecklinger, A., Bosch, V., Gruenewald, C., Bentin, S. & 46. Moll, J., de Oliveira-Souza, R., Eslinger, P. J., Bramati,
von Cramon, D. Y. (2000) Hum. Brain Mapp. 11, i.e., Mourao-Miranda, J., Andreiuolo, P. A. & Pessoa, L.
146–161. (2002) J. Neurosci. 22, 2730–2736.
24. Deibert, E., Kraut, M., Kremen, S. & Hart, J. (1999) Neu- 47. Pardo, J. V., Pardo, P. J. & Raichle, M. E. (1993) Am. J.
rology 52, 1413–1417. Psychiatry 150, 713–719.
25. Adolphs, R. (2001) Curr. Opin. Neurobiol. 11, 48. Reiman, E. M., Lane, R. D., Ahern, G. L., Schwartz,
231–239. G. E., Davidson, R. J., Friston, K. J., Yun, L.-S. & Chen,
26. Castelli, F., Happé, F., Frith, U. & Frith, C. (2000) Neu- K. (1997) Am. J. Psychiatry 154, 918–925.
roImage 12, 314–325. 49. Talairach, J. & Tournoux, P. (1988) Coplanar Stereotaxic
27. Frith, C. D. & Frith, U. (1999) Science 286, 1692–1695. Atlas of the Human Brain (Thieme, New York).
RT0996_C05.qxd 11/8/04 2:01 PM Page 63
R E A D I N G 5
Functional Networks in Emotional Moral

and Nonmoral Social Judgments
•
Jorge Moll,* Ricardo de Oliveira-Souza,* Ivanei E. Bramati,* and
Jordan Grafman†
Reading daily newspaper articles often evokes opinions and social judgments about the characters
and stories. Social and moral judgments rely on the proper functioning of neural circuits concerned
with complex cognitive and emotional processes. To examine whether dissociable neural systems
mediate emotionally charged moral and nonmoral social judgments, we used a visual sentence
verification task in conjunction with functional magnetic resonance imaging (fMRI). We found that
a network comprising the medial orbitofrontal cortex, the temporal pole and the superior temporal
sulcus of the left hemisphere was specifically activated by moral judgments. In contrast, judgment
of emotionally evocative, but non-moral statements activated the left amygdala, lingual gyri, and
the lateral orbital gyrus. These findings provide new evidence that the orbitofrontal cortex has
dedicated subregions specialized in processing specific forms of social behavior.
Keywords: moral judgment; fMRI; orbitofrontal; acquired sociopathy; frontal lobes.
Introduction several possible outcomes of that intention (Colby

et al., 1990; Haidt, in press). Such complex social
Humans routinely judge the social behavior of oth- judgments rely on the proper functioning of brain
ers (Brothers, 1997). These judgments can be networks dedicated to processing stimuli endowed
made on the basis of moral beliefs or can be just with social and emotional significance (Eisenberg
an interpretation of an actor’s intention and the et al., 1995; Damasio et al., 2000). The integration
*Neuroimaging and Behavioral Neurology Group (GNNC), †Cognitive Neuroscience Section, National Institute of Neuro-
Hospitals D’Or and LABS, Rio de Janeiro, RJ, Brazil; and logical Disorders and Stroke, Bethesda, Maryland.
63
RT0996_C05.qxd 11/8/04 2:01 PM Page 64
of cognitive evaluation with emotional bias allows encompasses the polar and mediobasal divisions of
humans to more confidently judge the social con- the frontal lobes (Kandel and Freed, 1989), the
sequences of other people’s actions (Fletcher et al., temporal poles (Miller et al., 1999) and several
1995; Frith and Frith, 1999). Damage to these sys- basal forebrain structures that are interconnected
tems may lead to distinct social behavior abnor- by the medial forebrain bundle and extend from the
malities (Eslinger and Damasio, 1985; Adolphs, ventromedial hypothalamus caudally to the amyg-
1999). The neural systems underlying these social dala and septal area rostrally (Adolphs et al., 1998;
abilities include the orbitofrontal cortex (OFC), the Flynn et al., 1988; Gorman and Cummings, 1992).
temporal neocortex and the amygdala (Brothers, In previous reports (Oliveira-Souza and Moll,
1997; Adolphs et al., 1998; Adolphs, 1999). While 2000; Moll et al., 2001), we addressed the brain
numerous functional imaging studies in humans networks involved in the processing of sentences
have demonstrated cortical and limbic activation with or without moral content using fMRI in nor-
when subjects visually process social (see mal subjects. A selective network of brain regions
Adolphs, 1999, for a review) or emotional stimuli was more active when subjects judged moral as
(Reiman et al., 1997; Lane et al., 1999), only a few opposed to factual statements. These regions in-
have examined complex social cognition (Hoffman cluded the ventral-anterior and medial sectors of
and Haxby, 2000; Golby et al., 2001). Investigations the prefrontal cortex (frontopolar and medial
of patients with autism (Frith and Frith, 1999) and frontal gyri), right anterior temporal cortex, left
psychopaths (Blair, 1995) suggest that different angular gyrus, and globus pallidus. Moreover,
aspects of social cognition can be selectively while the emotional valence of stimuli seemed to
impaired. For example, although autistic indi- be directly related to the activations in right ante-
viduals have generally impaired social behavior rior temporal lobe and subcortical nuclei, it
(Frith and Frith, 1999), psychopaths are impaired played only an ancillary role in the prefrontal cor-
on a subset of social behaviors that require moral tex activation. Despite such compelling evidence,
appraisals (Blair, 1995). since our previous study design did not include
The neurological and neuropsychiatric litera- emotional stimuli as an independent, nonmoral,
ture is also informative in regard to the organiza- experimental condition, it is difficult to conclude
tion of the human social brain. There is increasing which brain regions were distinctively recruited
evidence that primary psychopaths comprise a by emotion processing as opposed to moral judg-
particularly severe subgroup of antisocial individ- ment. Another recent fMRI study (Greene et al.,
uals whose deviant behaviors are related to struc- 2001) reported similar activation of the anterior
tural and functional brain abnormalities (Raine prefrontal cortex in response to complex moral
et al., 2000; Kiehl et al., 2001). Acquired brain judgments. This study did not include a nonmoral
damage in previously normal individuals can also emotional condition either and, thus, the reported
lead to a set of related clinical syndromes remi- activations could have been induced by emotional
niscent of primary psychopathy (Tranel, 1994). processing. Besides, the increased attentional and
The antisocial behavior of these patients may re- decision-making demands of the moral-emotional
sult from sheer impulsiveness and goal neglect to condition might be associated with slower reac-
recurrent flagrant criminal and evil actions that tion times, which could by themselves have led to
represent a bizarre change from their premorbid increased prefrontal activation (de Zubicaray et al.,
personality styles (Brower and Price, 2002). Like 2001). In our previous reports (Oliveira-Souza
primary psychopaths, such “acquired sociopaths” and Moll, 2000; Moll et al., 2001), we employed a
often retain the ability to tell right from wrong and measure of judgment difficulty and found equiva-
to articulate sound statements on morality and lent judgment difficulty for moral and nonmoral
social appropriateness, that stands in sharp con- stimuli, favoring the view that cognitive effort
trast to their behavior in real life. The damage in was not decisively involved in the brain activation
such cases falls within an extended area that patterns. Notwithstanding these observations, the
RT0996_C05.qxd 11/8/04 2:01 PM Page 65
Functional Networks in Emotional Moral and Nonmoral Social Judgments ■ 65
issue of which brain regions are specifically en- of education and gender distribution did not differ
gaged when normal subjects judge moral and from the fMRI subjects (age: t 1.76, P 0.11;
nonmoral, emotionally charged situations re- education: t 1.13, P 0.28; gender: x2 1.40,
mains unsettled. P 0.56), was recruited for a supplemental be-
Here we employed functional MRI to address havioral study. All subjects gave written informed
the role of emotional valence and moral content in consent and did not receive financial compensa-
social judgments. For this purpose, normal sub- tion. The study was conducted in the Hospital
jects were requested to judge statements pertain- Barra D’Or and approved by the hospital’s Institu-
ing to three main conditions: emotionally un- tional Review Board and Ethics Committee.
pleasant statements without moral connotations,
emotionally unpleasant statements with moral Stimuli and Task
connotations, and emotionally neutral statements. Subjects were asked to read short statements that
Scrambled statements were included as a baseline were visually presented through LCD goggles
condition. By including an experimental condi- (Resonance Technologies, Inc., CA) and to
tion evocative of emotions devoid of moral con- covertly judge them as being either right or
notations, we intended to explore the differential wrong. We chose the words “certo” and “errado,”
effect of emotional valence and moral judgment the Portuguese equivalents of “right” and
on brain activation. An additional feature of the “wrong” because, as in English, they allow moral,
present study was that statements pertaining to all factual, and structural connotations. Subjects un-
experimental conditions described human actions derstood that they were intended to use right or
unfolding in social scenarios. Based on the evi- wrong attributions in a broadly defined manner.
dence that different kinds of social abilities and By limiting the output of their cognitive opera-
emotional processing may be dissociated in cases tions to only two categories (“right” or “wrong”),
of brain damage (Eslinger and Damasio, 1985; we expected to force each participant to make
Blair, 1995), we hypothesized that the judgment standard decisions regardless of the specific con-
of unpleasant-moral and unpleasant-nonmoral so- tent of each sentence.
cial situations would lead to distinct patterns of
corticolimbic activation. More specifically, we
Rationale behind Moral and
expected that the anterior sectors of the prefrontal
Nonmoral Judgments
cortex, as shown by our previous study (Moll
et al., 2001), would be preferentially activated by The distinction between moral and nonmoral
moral judgments when compared with judgments judgments relies on psychological constructs that
of emotional situations without moral connota- can be objectively assessed and have been exten-
tions. We also predicted that the judgment of un- sively validated across different cultures (Snarey,
pleasant scenarios devoid of moral content would 1985; Colby et al., 1990). A further distinction
activate brain regions that mediate basic unpleas- between nonmoral social norm violations and
ant emotions, such as the amygdala. moral-social violations has also been supported by
empirical evidence. This distinction forms the basis
of the concept of conventional and moral trans-
gressions, whose psychological bases have been
Materials and Methods
worked out in normal individuals, psychopaths
Subjects and in patients with autism, in both adults and
Seven healthy right-handed adults (three males, children (Blair, 1995, 1996; Fisher and Blair, 1998).
mean age of 30.3 4.7 years, 14.9 2.0 years of
education), with no history of neuropsychiatric Experimental Conditions
disorders, participated in the fMRI study. An inde- Three main conditions included structurally similar
pendent group of seven subjects whose age, level neutral (NTR), moral (M), and nonmoral (NM)
RT0996_C05.qxd 11/8/04 2:01 PM Page 66
TABLE 5.1. Sample Statements ments were scored by two judges who used a pre-
Nonmoral neutral (NTR) defined lexicon of words that expressed basic or
He never uses the seat belt. moral emotions. The identification of basic in con-
The elderly sleep more at night. trast to moral emotions in these descriptions was
Fat children should make a diet.
The painter used his hand as a paintbrush. based on concepts firmly grounded in a large body
Judges use white uniforms. of empirical research (Haidt, in press). This proce-
Nonmoral unpleasant (NM) dure helped us ascertain that the statements em-
He licked the dirty toilet.
The elderly are used to eating living toads. ployed in the moral condition suited our purposes
Pregnant women often throw up. as moral-eliciting stimuli.
People don’t have tatoos inside their eyeballs. Each condition was composed of a set of 24
Judges often eat rotten food.
Moral (M) statements that were presented in blocks of three
He shot the victim to death. at a time. Stimuli were displayed for 5 s and were
The elderly are useless. separated by 5 s of a blank screen. A pilot study
Criminals should go to jail.
The father never treated his son as a slave. showed that this presentation rate allowed sub-
The judge condemned the innocent man. jects to comfortably read and judge each sentence.
Scrambled (SCR) The eight blocks of each condition followed a
Sons push use eat work.
Kick like poor rain old have. fixed pseudorandomized order. A fixation cross-
Life turn of shoes drink was brother. hair was displayed for 15 s between each block to
Your drive building must meat. allow a complete return of the BOLD signal to
Day daughter less ground parents loose.
baseline. Scrambled sentences were generated by
randomly reordering words sampled from the
other conditions, being semantically meaningless
statements (see Table 5.1 for a sample of the state- and grammatically incorrect (“nonsense”). Subjects
ments). NTR statements described relatively un- were not informed beforehand about the content
emotional situations, NM statements described of the sentences, yet they were asked to “try and
emotionally aversive scenarios without moral con- get their meaning and to judge all of them, re-
tent, while M statements described emotional situ- gardless of how weird they might sound.” This
ations designed to evoke moral attitudes and feel- was meant to encourage subjects to stick to the
ings. All statements described scenarios and people scrambled sentences and prevent their attention
acting in different settings, and, thus, each of the from wandering. To minimize the effects of plan-
NTR, NM, and M statements deployed explicit so- ning and to prevent the subjects from reasoning
cial contexts. An additional condition made up of about the future, the situations depicted in all
scrambled statements (SCR) was added to serve as statements referred to the present or past only.
a low level baseline. Subjects covertly judged all
the sentences in a categorical fashion as right or Behavioral Measures
wrong. After fMRI scanning, the sentences were To obtain a measure of task difficulty, seven sub-
presented again, this time in a randomized, un- jects not participating in the fMRI experiment
blocked fashion, and subjects were asked to overtly judged the blocked 24 NTR, M, NM, and SCR
judge them as right or wrong by referring to their statements. These blocks were administered in a
impressions while in the scanner. Subjects also randomized order across subjects. The time to
rated the degree of moral content and of the posi- judge each block was recorded as a response time
tive or negative emotional impact of each statement (RT) and taken as a proxy for task difficulty.
on 4-point Likert scales. They were instructed that Although collecting responses during fMRI data
moral content meant issues of values, rights, justice, acquisition would be more desirable, MR compat-
responsibilities and principles regarding peace and ible response devices were not available in our in-
care for others (Colby et al., 1990; Gilligan, 1993). stitution at the time this study was completed.
Subjects were encouraged to make a short verbal Although we could have obtained RTs from the
commentary about each statement. These com- same subjects by repeating the task after fMRI
RT0996_C05.qxd 11/8/04 2:01 PM Page 67
scanning, we pondered that subjects’ attitudes to removal and spatial normalization (3-D Gaussian
repeated stimuli would not necessarily reflect their kernel 4 mm) were performed. Datasets were
naïve responses. For this reason, a different group co-registered and Talairach transformed
of subjects matched for age, gender, education and (Talairach and Tournoux, 1998). Activation maps
cultural background was employed. This approach were analyzed with statistical parametric meth-
has been successfully adopted in previous func- ods (Friston et al., 1995) contained in Brain
tional imaging studies (Bottini et al., 1994). Voyager v. 3.9 (Brain Innovation, Maastricht,
Ratings for moral content, emotional impact, and The Netherlands). Regressors representing the
mean response times were assessed with analysis of experimental conditions were modeled with a he-
variance. Pairwise comparisons of means were modynamic response filter entered in a multiple
evaluated post hoc with the least significant differ- regression analysis, using a fixed-effects model.
ence test. The association between emotional impact Significance was assessed using a threshold cor-
and moral content in the M condition was assessed responding to P 0.001 (uncorrected) at the
with the Phi coefficient. Computations were per- voxel level and a three-dimensional cluster extent
formed using Statistica, v. 5.5 (StatSoft, 1999). threshold of 100 mm3 to protect against Type I er-
rors associated with multiple statistical compar-
fMRI Procedures and Data Analysis isons (Forman et al., 1995). Smaller activated
Anatomic (3D-GRE T1-weighted images, clusters (50–100 mm3) were reported when they
1.25 mm) and functional data (BOLD-EPI imag- fell in a region where an a priori hypothesis
ing, TR/TE 4980/66 ms, 128 128, FOV predicted activation (OFC and medial prefrontal
256 mm, thickness/gap 5/0.25 mm, 16 slices) cortex, amygdala and anterior temporal cortex).
were obtained with a 1.5 MRI scanner (Siemens Levels of statistical significance for each acti-
Vision, Germany). Data acquisition was synchro- vated cluster are reported in Table 5.2. Results
nized with stimulus presentation. Functional were overlaid on averaged anatomical data
datasets were 3D motion-corrected. Slice time from all subjects and on partially inflated three-
correction, temporal smoothing, linear trend dimensional templates of a sample subject.
TABLE 5.2. Anatomical Locations and Coordinates of Activations
Center Talairach coordinates

Cluster Cluster
Brain region, Brodmann areaa X Y Z P value volumeb
M vs NTRc
L Medial OFC, 10/11d –10 46 –12 0.0002 171
L Temporal pole, 38 –33 19 –23 0.001 50e
L STS, 21/22d –47 –40 –2 0.0004 148
NM vs NTRc
R Fusiform G, 19f 31 –66 –20 0.0004 139
R Inf Occ G, 19 28 –88 –22 0.001 100
R Lingual G, 17/18f 5 –72 0 0.000001 1750
Bilat Calcarine S, 17 –11 –88 2 0.000004 247
L Amygdalaf –20 –12 –6 0.000001 253
L Lingual G, 17/18f –23 –64 –15 0.000001 1068
L Lateral OFC, 11/47f –26 32 –1 0.001 108
L Fusiform G, 19f –34 –62 –11 0.00003 206
a
OFC, orbitofrontal cortex; STS, superior temporal sulcus; G, gyrus; S, sulcus; Inf, inferior; Occ, occipital; L, Left; R, right; Bilat,
bilateral.
b
Cluster volumes of at least 100 mm3.
c
M, NTR, and NM correspond to moral, neutral, and nonmoral social conditions, respectively.
d
Also activated in the M vs NM comparison.
e
Cluster volume below 100 mm3.
f
Also activated in the NM vs M comparison.
RT0996_C05.qxd 11/8/04 2:01 PM Page 68
Results
Behavioral
Moral content was rated significantly higher in
the M (2.12 1.41) in comparison to the NTR
(0.07 0.37) and NM (0.27 0.82) conditions
[F(2,499) 228.9, P 0.00002]. The M and NM
statements were designed so that they had nega-
tive emotional valence. This was validated by
subject ratings of those same statements at de-
briefing. The emotional valence of the NTR con-
dition approached zero (0.05 0.98), whereas
the net emotional valence for both the M (2.02
1.26) and NM (0.88 1.20) conditions was
negative. The NM and M conditions differed sig-
nificantly from the NTR condition in degree of
emotional impact [F(2,497) 138.23, P 0.0001].
M sentences were rated as having a higher emo-
tional impact than NM sentences (P 0.0001).
There was a significant relationship between
moral content and emotionality of statements
(r 0.44, p 0.0001), suggesting that moral
judgment interacts with the perceived emotional-
ity of the stimulus. Judgments in the M condition
were described more frequently with moral terms
(e.g., “pity” and “indignation”) than with basic
emotion words (123 53) in contrast to judg-
ments in the NM condition, in which primary
emotion words (e.g., “disgust”) were used more
often (136 1) (Phi 0.70, P 0.0001). This
result indicated that the M sentences were effec-
tive in eliciting moral concerns. There was an
overall difference in judgment times across trials
([F(3,18) 3.42, P 0.04]. Post hoc analyses
showed that this difference was due to slower RT
in the SCR condition (92.3 33.5 s, P 0.03).
FIGURE 5.1 ■ (A color version of this figure follows
Mean RT did not statistically differ for the NTR page 146.) Brain regions activated by emotionally
(75.3 20.7 s), M (73.7 24.3 s), and NM (76.7 evocative moral (M) and nonmoral (NM) judgments
25.3 s) conditions, suggesting that judging compared to neutral (NTR) ones. Activations were over-
them recruited equivalent degrees of effort. laid on sections through an averaged brain from all sub-
jects with inverted grayscale and on 3-D renderings of a
reference brain. (a) M vs NTR condition. Activated re-
gions were in the left orbitofrontal cortex (OFC) and in
Brain Activation the superior temporal sulcus and the left temporal pole.
Compared with the NTR condition, the M and (b) NM vs NTR condition. Activated regions were in the
NM conditions evoked distinct brain activation left amygdala and lateral OFC, and bilaterally in the
patterns (Figures 5.1a and 5.1b and Table 5.2). visual cortex.
The NM condition induced activation of the left
RT0996_C05.qxd 11/8/04 2:01 PM Page 69
amygdala and left lateral OFC as well as of sev- sector of the STS that has been previously impli-
eral regions of the ventral visual cortex (lingual, cated in processing visual social cues (e.g., Hoffman
inferior occipital and fusiform gyri). The M con- and Haxby, 2000), other sectors of the temporal
dition induced activation of the left medial OFC lobe were not activated when M and NM condi-
(gyrus rectus and medial orbital gyrus), left tem- tions were contrasted to each other. This is in
poral pole, and the cortex of the superior temporal agreement with the role of temporal lobe struc-
sulcus (STS), close to the angular gyrus. tures in semantic comprehension (Zahn et al.,
When the M condition was compared to the 2000), and indicates that semantic processing was
NM condition, the same activations in the medial well matched in these conditions. Mean signal
OFC (101 mm3, P 0.001) and STS (1063 mm3, changes and standard errors from the medial and
P 106) were seen. The left temporal pole acti- lateral OFC, amygdala, primary visual cortex and
vation was no longer observed. In the opposite STS cortex in each experimental condition, aver-
NM vs M comparison, the left amygdala aged across subjects, are displayed in Figure 5.2.
(242 mm3, P 4 106) and the lateral orbital In order to investigate the effects of right vs
gyrus (207 mm3, P 3 105) remained active, wrong categorical judgments on brain activation,
along with the visual cortex (lingual and fusiform we computed Kendall’s correlation coefficient be-
gyri). Notably, with the exception of a limited tween the percentage of BOLD signal increase in
1 2 3
4 5
1 - left amygdala
2 - left lateral OFC
3 - V1
4 - left medial OFC
5 - left STS
FIGURE 5.2 ■ (A color version of this figure follows page 146.) Mean signal changes of the left medial and lateral
OFC, left amygdala, primary visual cortex (V1), and the cortex of the left superior temporal sulcus (STS), obtained from
averaged MR signal from all subjects. Curve colors correspond to experimental conditions as follows: yellow, unpleasant
condition; light blue, moral condition; green, neutral condition; black, scrambled condition.
RT0996_C05.qxd 11/8/04 2:01 PM Page 70
whereas nonmoral social judgments associated

with unpleasant emotions induced lateral OFC
and amygdala activation. The amygdala has a ma-
jor role in processing emotionally arousing stim-
uli, both pleasant and aversive, and it has been
suggested that it may help allocate resources to
processing different kinds of biologically salient
stimuli (Adolphs et al., 1998; Golby et al., 2001).
Activation of the left amygdala by a cognitive-
emotional elicitation procedure (e.g., through lan-
guage) implies that top-down mechanisms may
activate this brain region under these conditions,
FIGURE 5.3 ■ (A color version of this figure follows in accord with recent functional imaging studies
page 146.) Brain regions activated by the neutral (NTR) employing threatening words (Isenberg et al.,
as compared to scrambled (SCR) condition (both temporal 1999) and the cognitive representation of fear
lobes and frontal opercula, supplementary motor area,
(Phelps et al., 2001). This finding is also consistent
anterior cingulate, basal ganglia, and thalamus).
with the suggestion that the left amygdala, rather
than the right, is more closely related to linguistic
the left amygdala and medial OFC, and the num-
affective processes (Markowitsch, 1998; Phelps et
ber of statements judged as right vs wrong in the
al., 2001). Contrary to our expectations, the amyg-
NM and M conditions, respectively. There was no
dala was not activated in the moral judgment con-
relationship between the number of right vs
dition, even though those statements were rated as
wrong judgments and the magnitude of the hemo-
most emotionally evocative. A possible explana-
dynamic response of the amygdala (T 0.20,
tion is that the medial OFC, which is intimately
P 0.62) or medial OFC (T 0.09, P 0.74),
linked to the processing of social rules and emo-
suggesting that activation in these brain regions
tions related to moral processing, down-regulates
was independent of the frequency of outcome of
the activity of the amygdala in certain circum-
the categorical right vs wrong judgments.
stances (Baxter et al., 2000). The amygdala is
When the NTR condition was contrasted to the
densely interconnected with the visual cortex,
SCR condition, a pattern of brain activation simi-
which is strongly activated by aversive pictures or
lar to that described in studies of sentence pro-
words (Reiman et al., 1997; Lane et al., 1999).
cessing was observed (Bottini et al., 1994). Acti-
Thus, the activation of the visual cortex in the NM
vated regions included the middle and posterior
condition is not surprising, and could have re-
portions of the superior temporal gyrus and sulcus
sulted from modulatory effects from the amyg-
bilaterally (more extensively in the left hemi-
dala (Morris et al., 1998).
sphere), the frontal opercula, the anterior cingu-
The amygdala is also massively interconnected
late and adjacent supplementary motor cortex, the
with the OFC, especially with the caudal sector of
thalamus and putamen, and additional foci in the
its lateral subdivision (Baxter et al., 2000; Öngur
anterior temporal lobes, fusiform gyrus, and
and Price, 2000), which was activated in the NM
cuneus (Figure 5.3).
condition. This region is activated in abstract
reward/punishment acquisition in humans
Discussion (O’Doherty et al., 2001) and, when damaged, is
thought to impair social behavior (Anderson et al.,
Our results provide new evidence that distinct 1999). The medial OFC activation in the M condi-
neural networks are activated by different kinds of tion is compatible with evidence showing that hu-
social judgment. In particular, moral judgments mans sustaining lesions in this region frequently
associated with unpleasant emotions induced acti- present with social disinhibition, lack of empathy
vation in the anterior aspect of the medial OFC, and increased levels of aggression (Grafman et al.,
RT0996_C05.qxd 11/8/04 2:01 PM Page 71
1996; Anderson et al., 1999; Pietrini et al., 2000; REFERENCES

Raine et al., 2000). One explanation for these im- Adolphs, R. 1999. Social cognition and the human brain.
pairments is that patients are deficient in their abil- Trends Cogn. Sci. 3: 469–479.
ity to choose among behavioral alternatives based Adolphs, R., Tranel, D., and Damasio, A. R. 1998. The human
amygdala in social judgment. Nature 393: 470–474.
on inferences about positive or negative outcomes
Anderson, S. W., Bechara, A., Damasio, H., Tranel, D., and
and changing reinforcement contingencies (Rolls Damasio, A. R. 1999. Impairment of social and moral be-
et al., 1994; Bechara et al., 2000). The present havior related to early damage in human prefrontal cortex.
findings suggest that the medial OFC may be even Nature Neurosci. 2: 1032–1037.
more critical for the integration of moral knowl- Baxter, M. G., Parker, A., Lindner, C. C. C., Izquierdo, A. D.,
and Murray, A. E. 2000. Control of response selection by re-
edge with the emotions that determine the rein-
inforcer value requires interaction of amygdala and orbital
forcing value of specific behavioral actions. It is prefrontal cortex. J. Neurosci. 20: 4311–4319.
also likely that the medial OFC, which receives Bechara, A., Tranel, D., and Damasio, H. 2000. Characteriza-
projections from the STS region, integrates cues tion of the decision-making effect of patients with ventro-
about the intentional and emotional states of oth- medial prefrontal cortex lesions. Brain 123: 2189–2202.
Blair, R. J. R. 1995. A cognitive developmental approach to
ers via signals from surface features of stimuli
morality: Investigating the psychopath. Cognition 57: 1–29.
such as facial expression, body posture, and voice Blair, R. J. R. 1996. Brief report: Morality in the autistic child.
inflexions into decision-making (Adolphs, 1999; J. Autism Dev. Disord. 26: 571–579.
Hoffman and Haxby, 2000). The finding that the Bottini, G., Corcoran, R., Sterzi, R., Paulesu, E., Schenone, P.,
temporal pole was activated in the M vs NTR con- Scarpa, P., Frackowiak, R. S., and Frith, C. D. 1994. The
role of the right hemisphere in the interpretation of figura-
trast, but not in the M vs NM or NM vs M com-
tive aspects of language. A positron emission tomography
parisons, favors the view that it participates non- activation study. Brain 117: 1241–1253.
specifically in both moral and nonmoral emotional Brothers, L. 1997. Friday’s Footprint. How Society Shapes
processing. This result is consistent with the The Human Mind. Oxford, New York.
weaker activation of the right anterior temporal Brower, M. C., and Price, B. H. 2002. Neuropsychiatry of
frontal lobe dysfunction in violent criminal behavior: A crit-
cortex when emotional impact was covaried in the
ical review. J. Neurol. Neurosurg. Psychiatry 71: 720–726.
design matrix in a previous study (Moll et al., 2001). Colby, A., Kohlberg, L., Speicher, B., Hewer, A., Candee, D.,
As reported above, different sectors of the OFC and Gibss, J. 1990. The Measurement of Moral Judgment.
were activated when moral or nonmoral social Cambridge, New York.
judgments were being contemplated. Since RTs Damasio, A. R., Grabowski, T. J., Bechara, A., Damasio, H.,
Ponto, L. L. B., Parvizi, J., and Hichwa, R. D. 2000. Sub-
were equivalent for the M and NM conditions, we
cortical and cortical brain activity during the feeling of self-
believe that these different patterns of activation generated emotions. Nature Neurosci. 3: 1049–1056.
are not attributable to task difficulty or effort. The de Zubicaray, G. I., Wilson, S. J., McMahon, K. L., and
dissociable networks we have identified probably Muthiah, S. 2001. The semantic interference effect in the
work in an integrated fashion during many social picture-word paradigm: An event-related fMRI study em-
ploying overt responses. Hum. Brain Mapp. 14: 218–227.
interactions that combine moral, social, and emo-
Eisenberg, L. 1995. The social construction of the human
tional demands. One limitation of our findings re- brain. Am. J. Psychiatry 152: 1563–1575.
gards agency. The judgments employed in our Eslinger, P. J., and Damasio, A. R. 1985. Severe disturbance of
study did not require reference to the subjects’ higher cognition after bilateral frontal lobe ablation: Patient
own behavior. On evolutionary grounds, it is likely EVR. Neurology 35: 1731–1741.
Fisher L., and Blair R. J. 1998. Cognitive impairment and its
that agency, or role-taking (Ruby and Decety,
relationship to psychopathic tendencies in children with
2001), critically influences social and emotional- emotional and behavioral difficulties. J. Abnorm. Child
based reasoning and behavior and can have a di- Psychol. 26: 511–519.
rect impact on the outcomes of goal-directed be- Fletcher, P. C., Happé, F., Frith, U., Baker, S. C., Dolan, R. J.,
havior. Examining the brain activation patterns Frackowiak, R. S., and Frith, C. D. 1995. Other minds in the
brain: A functional imaging study of “theory of mind” in
related to the differential effects of agency and
story comprehension. Cognition 57: 109–128.
emotion on moral and factual judgments is a logi- Flynn, F., Cummings, J. L., and Tomiyasu, U. 1988. Altered be-
cal next step for researchers interested in the brain havior associated with damage to the ventromedial hypothal-
representation of morality-influenced behavior. amus: A distinctive syndrome. Behavioral Neurol. 1: 49–58.
RT0996_C05.qxd 11/8/04 2:01 PM Page 72
Forman, S. D., Cohen, J. D., Fitzgerald, M., Eddy, W. F., Moll, J., Eslinger, P. J., and Oliveira-Souza, R. 2001. Fron-
Mintun, M. A., and Noll, D. C. 1995. Improved assessment topolar and anterior temporal cortex activation in a moral
of significant activation in functional magnetic resonance judgment task: Preliminary functional MRI results in nor-
imaging (fMRI): Use of a cluster-sized threshold. Magn. mal subjects. Arq. Neuropsiquiatr. 59: 657–664.
Reson. Med. 33: 636–647. Morris, J. S., Friston, K. J., Buchel, C., Frith, C. D., Young A. W.,
Friston, K. J., Holmes, A. P., Poline, J.-P., Grasby, P. J., Williams, Calder A. J., and Dolan R. J. 1998. A neuromodulatory role
S. C., Frackowiak, R. S., and Turner, R. 1995. Analysis of for the human amygdala in processing emotional facial
fMRI time-series revisited. Neuroimage 2: 45–53. expressions. Brain 121: 47–57.
Frith, C. D., and Frith, U. 1999. Interacting minds—A bio- O’Doherty, J., Kringelbach, M. L., Rolls, E. T., Hornak, J., and
logical basis. Science 286: 1692–1695. Andrews, C. 2001. Abstract reward and punishment repre-
Gilligan, C. 1993. In a Different Voice, 3rd ed. Harvard, Cam- sentations in the human orbitofrontal cortex. Nature Neu-
bridge. rosci. 4: 95–102.
Golby, A. J., Gabrieli, J. D. L., Chiao, J. Y., and Eberhardt, J. L. Oliveira-Souza, R., and Moll, J. 2000. The moral brain: Func-
2001. Differential responses in the fusiform region to same- tional MRI correlates of moral judgment in normal adults.
race and other-race faces. Nature Neurosci. 4: 845–850. Neurology 54 (Suppl. 3): 252.
Gorman, D. G., and Cummings, J. L. 1992. Hypersexuality Öngür, D., and Price, J. L. 2000. The organization of networks
following septal injury. Arch. Neurol. 49: 308–310. within the orbital and medial prefrontal cortex of rats, mon-
Grafman, J., Schwab, K., Warden, D., Pridgen, A., Brown, keys and humans. Cereb. Cortex 10: 206–219.
H. R., and Salazar, A. M. 1996. Frontal lobe injuries, vio- Phelps, E. A., O’Connor, K. J., Gatenby, J. C., Gore, J. C.,
lence, and aggression: A report of the Vietnam Head Injury Grillon, C., and Davis, M. 2001. Activation of the left
Study. Neurology 46: 1231–1738. amygdala to a cognitive representation of fear. Nature Neu-
Greene, J. D., Sommerville, R. B., Nystrom, L. E., Darley, J. M., rosci. 4: 437–441.
and Cohen, J. D. 2001. An fMRI investigation of emotional Pietrini, P., Guazzelli, M., Basso, G., Jaffe, K., and Grafman,
engagement in moral judgment. Science 293: 2105–2108. J. 2000. Neural correlates of imaginal aggressive behavior
Haidt, J. The moral emotions. (in press) In Handbook of Affec- assessed by positron emission tomography in healthy sub-
tive Sciences (R. J. Davidson, K. Scherer, H. H. Goldsmith, jects. Am. J. Psychiatry 157: 1772–1781.
Eds.) Oxford Univ. Press. Raine, A., Lencz, T., Bihrle, S., LaCasse, L., and Colletti, P.
Hoffman, E. A., and Haxby, J. V. 2000. Distinct representa- 2000. Reduced prefrontal gray matter volume and reduced
tions of eye gaze and identity in the distributed human neu- autonomic activity in antisocial personality disorder. Arch.
ral system for face perception. Nature Neurosci. 3: 80–84. Gen. Psychiatry 57: 119–127.
Isenberg, N., Silbersweig, D., Engelien, A., Emmerich, S., Reiman, E. M., Lane, R. D., Ahern, G. L., Schwartz, G. E.,
Malavade, K., Beattie, B., Leon, A. C., and Stern, E. 1999. Davidson, R. J., Friston, K. J., Yun, L. S., and Chen, K.
Linguistic threat activates the human amygdala. Proc. Natl. 1997. Neuroanatomical correlates of externally and inter-
Acad. Sci. USA 96: 10456–10459. nally generated human emotion. Am. J. Psychiatry 154:
Kandel, E., and Freed, D. 1989. Frontal-lobe dysfunction and 918–925.
anti-social behavior: a review. J. Clin. Psychol. 45: 404–413. Rolls, E. T., Hornak, J., Wade, D., and McGrath, J. 1994. Emo-
Kiehl, K. A., Smith, A. M., Hare, R. D., Mendrek, A., Forster, tion-related learning in patients with social and emotional
B. B., Brink, J., and Liddle, P. F. 2001. Limbic abnormali- changes associated with frontal lobe damage. J. Neurol.
ties in affective processing by criminal psychopaths as re- Neurosurg. Psychiatry 57: 1518–1524.
vealed by functional magnetic resonance imaging. Biol. Ruby, P., and Decety, J. 2001. Effect of subjective perspective
Psychiatry 50: 677–684. taking during simulation of action: A PET investigation of
Lane, R., Chua, P., and Dolan, R. 1999. Common effects of agency. Nature Neurosci. 4: 546–550.
emotional valence, arousal and attention on neural activa- Snarey, J. R. 1985. Cross-cultural universality of social-moral
tion during visual processing of pictures. Neuropsychologia development: A critical review of Kohlbergian research.
37: 989–997. Psychol. Bull. 97: 202–232
Markowitsch, H. J. 1998. Differential contribution of right and Talairach, J., and Tournoux, P. 1998. Co-Planar Stereotaxic
left amygdala to affective information processing. Behav. Atlas of the Human Brain. Thieme Medical, New York.
Neurol. 11: 233–244. Zahn, R., Huber, W., Drews, E., Erberich, S., Krings, T.,
Miller, B. L., Hou, C., Goldberg, M., and Mena, I. 1999. Ante- Willmes, K., and Schwarz, M. 2000. Hemispheric lateral-
rior temporal lobes: Social brain. In The Human Frontal ization at different levels of human auditory word process-
Lobes: Functions and Disorders. (B. L. Miller and J. L. ing: A functional magnetic resonance imaging study. Neu-
Cummings, Eds.) Guilford Press. rosci. Lett. 287: 195–198.
RT0996_P04.qxd 11/8/04 2:14 PM Page 73
PA R T 4
Dissociable Systems for Face

and Object Processing
•
Newborn babies prefer to look at pictures of faces compared to pictures of

objects, and by childhood children are experts at recognizing faces. In
contrast, individuals who suffer damage from a stroke in the ventral
temporal cortex region, an approximately 20 cm square area of tissue on
the bottom of the brain near the ears, are left with an inability to recognize
familiar faces. Although these patients show no apparent difficulty
differentiating among various objects, they cannot distinguish between their
spouses and other individuals even though they show larger electrodermal
(sweat) responses to their spouses! Are faces and objects processed
differently by the brain, or does the region of the brain that appears
important for processing faces also important for making detailed
discriminations within a category of objects?
Early research suggested that the fusiform gyrus, which falls in the
ventral temporal cortex, is selectively activated when viewing faces, and a
nearby area of cortex typically medial to the face-sensitive area is activated
when viewing places. This led to the suggestion that the fusiform gyrus was
a face-specific processing module. However, subsequent research
demonstrated that experts making fine-grain discriminations among objects
in their expertise (e.g., automobiles) showed increased activity in the face-
sensitive fusiform gyrus, as well.
Liu et al. (2002) employ a technique called magnetoencephalography
(MEG) to explore the serial information processing stages that are involved
73
RT0996_P04.qxd 11/8/04 2:14 PM Page 74
when processing faces. Measurements using MEG responses whether viewing houses or faces, but
provide high temporal and spatial resolution, although evidence is growing to suggest that these regions of
the sensitivity of MEG is limited to neural activity on the brain may be involved in the visual processing of
or near the surface of the brain. Liu et al. (2002) nonface objects that elicit the same set of
found a stage of face processing earlier than information processing operations. Haxby et al.
previously thought possible—a response as early as (2001) address this latter possibility, providing
100 msec in the ventral temporal region that is compelling evidence that face and object
associated with the successful categorization of a processing both involve much more distributed
stimulus as a face. They additionally found a information processing operations within the
previously observed information processing stage ventral temporal cortex than would be suggested
that emerged around 170 msec (the N170 in the by study of the brain region where the greatest
ventral temporal region, which tends to be larger in activation is observed (e.g., fusiform gyrus). Work
response to faces than other objects), which they of the type illustrated by Haxby et al. (2001) is
found to be associated with the successful beginning to suggest that even if the evolutionary
recognition of individual faces. Both MEG responses advantage of recognizing faces contributed to the
were more sensitive to face processing than object emergence of a specialized neural substrate for
(house) processing. Finally, the authors found face processing over evolutionary time, once
evidence that face processing involves a series of evolved the information processing operations
qualitatively different information processing performed by this specialized neural substrate are
operations. The neural response at 100 msec, for exapted for use in nonsocial information
instance, was particularly sensitive to parts of the face processing, as well, and represent but one aspect
(elemental information processing), whereas the of a representational system that is widely
neural response at 170 msec was especially sensitive distributed and overlapping in the ventral temporal
to the configuration of a face (holistic information cortex. Alternatively, the readings in this section
processing). may reflect that as advanced general processing
The Liu et al. (2002) study does not address substrates emerge in evolution, their functions may
whether participants who were as expert in be partially coopted for specialized processing
recognizing houses as they were in recognizing (such as faces) that have particular adaptive
faces would have displayed comparable MEG significance for the species.
RT0996_C06.qxd 11/8/04 2:02 PM Page 75
R E A D I N G 6
Stages of Processing in Face Perception:

An MEG Study
•
Jia Liu1, Alison Harris1 and Nancy Kanwisher1,2,3
Here we used magnetoencephalography (MEG) to investigate stages of processing in face perception

in humans. We found a face-selective MEG response occurring only 100 ms after stimulus onset (the
‘M100’), 70 ms earlier than previously reported. Further, the amplitude of this M100 response was
correlated with successful categorization of stimuli as faces, but not with successful recognition of
individual faces, whereas the previously-described face-selective ‘M170’ response was correlated with
both processes. These data suggest that face processing proceeds through two stages: an initial
stage of face categorization, and a later stage at which the identity of the individual face is extracted.
ace recognition is one of the most important sparrow)1,2. Evidence that a similar sequence may
F problems the human visual system must solve.
Here we used MEG to characterize the sequence
occur in face perception comes from single-unit
recordings in macaques showing that the initial tran-
of cognitive and neural processes underlying this sient response of face-selective neurons in infer-
remarkable ability. otemporal cortex reflects a rough categorization of
Two candidate stages of face processing are the face versus nonface, whereas subsequent firing of
categorization of a stimulus as a face, and the identi- the same neural population represents finer informa-
fication of a specific individual. Several studies of tion such as facial expression or individual identity3.
object recognition suggest that objects are first cate- It has been argued, however, that visual expertise in
gorized at a ‘basic level’ (dog, bird) before a finer discriminating exemplars of a specific visual
‘subordinate level’ identification is achieved (poodle, category may shift the point of initial contact
1 3
Department of Brain and Cognitive Sciences, NE20–443, MGH/MIT/HMS Athinoula A. Martinos Center for Biomedical
Massachusetts Institute of Technology, Cambridge, Massachu- Imaging, Charlestown, Massachusetts 02129, USA.
setts 02139, USA.
2
The McGovern Institute for Brain Research, Massachusetts
Institute of Technology, Cambridge, Massachusetts 02139, USA.
75
RT0996_C06.qxd 11/8/04 2:02 PM Page 76
with memory representations from the basic level to typical subject (experiment 1) shows the previously-
the subordinate level1,4. Given that we are all experts described10 face-selective M170 response occurring
at face recognition, this hypothesis predicts that peo- at a latency of about 160 ms (Figure 6.1a). This
ple should be able to identify an individual face as response may correspond approximately to the
fast as they can determine that it is a face at all. Al- N170 component in ERP studies7 and/or the N200 in
though some behavioral evidence is consistent with intracranial ERP studies21, as discussed later.
this hypothesis5, other evidence is not6. In addition to the M170, we found a smaller
MEG is an ideal technique for addressing these response peaking at a mean latency of 105 ms
questions, as its high temporal resolution enables us (the ‘M100’; range 84.5–130.5 ms, s.d. 16.1) that
to tease apart processing stages that may occur was significantly higher for faces than for houses.
within tens of milliseconds of each other. Prior work This result was seen with the same polarity in 13 of
with MEG and event-related potentials (ERPs) has 15 subjects. The scalp distribution of the face-selec-
characterized a response component called the N170 tive M100 response was slightly posterior to that of
(or M170 in MEG) that occurs around 170 ms after the M170, but the sensors showing the strongest
stimulus onset, and is about twice as large for face face-selectivity for the M100 largely overlapped
stimuli as for a variety of control nonface stimuli with those showing the strongest face-selectivity for
such as hands, houses or animals7–10. This response the M170. The MEG response of a representative
is thought to reflect the structural encoding of a subject at a typical overlapping face-selective sen-
face7,11,12, that is, the extraction of a perceptual rep- sor in the right hemisphere is shown in Figure 6.1b.
resentation of the face. Although several reports of For a stronger test of face selectivity, we meas-
even earlier category-selective responses have been ured the magnitude of the M100 response to a
published13–18, they are open to explanations invok- variety of control stimuli (experiment 2). Subjects
ing nonspecific repetition effects19 or differences in were asked to press a button whenever two con-
the low-level features of the stimuli20. secutive images were identical, obligating them to
When do the first truly face-selective responses attend to all stimuli regardless of inherent interest.
occur? We recorded MEG responses while sub- Target trials containing such repeated stimuli
jects passively viewed a sequence of photographs were excluded from the analysis. Accuracy on this
of faces and a variety of control stimuli (experi- one-back matching task was high for all cate-
ments 1 and 2). These experiments found a new gories ( 90% correct) except for hands (76%),
face-selective response occurring only 100 ms af- which are visually very similar to each other. The
ter stimulus onset (the M100), generated from ex- MEG data from experiment 1 were first used to
trastriate cortex (experiment 3). Further, we tested define sensors of interest (SOIs) in each subject—
whether the M100 and M170 amplitudes are cor- those sensors that produced significantly face-
related with success in face categorization and/or selective responses for both the M100 and the
face identification (experiment 4). Finally, we M170 (Methods). Both the amplitudes and latencies
tested for further qualitative differences in the of peak responses to the new stimulus conditions
processes underlying the M100 and the M170 by in experiment 2 in these same SOIs were then
measuring the responses of each component to quantified in the same subject in the same session.
face configurations and face parts (experiment 5). The M100 response to photographs of faces was
greater (Figure 6.2) than that to photographs of
animals (F1,12 10.2, P 0.01), human hands
Results (F1,12 9.0, P 0.02), houses (F1,12 8.1,
P 0.02) and nonface objects (F1,12 10.3,
A Face-Selective Response at a P 0.01). Therefore, the M100 is not generally
Latency of 100 ms selective for anything animate, or for any human
An interpolated map of the t-value comparing body part; instead, it seems to be selective for faces.
the MEG response to faces versus houses for a However, both the magnitude and selectivity of
RT0996_C06.qxd 11/8/04 2:02 PM Page 77
Stages of Processing in Face Perception ■ 77
a)
b)
FIGURE 6.1 ■ (A color version of this figure follows page 146.) MEG data
from a typical subject. (a) Pairwise t-tests between the responses at each sensor
reveal early (M100) and late (M170) significant differences in the MEG response
to faces versus houses over occipitotemporal cortex. (b) The MEG waveforms
are averaged across all face and house trials at a typical sensor of interest in the
right hemisphere. Red, faces; blue, houses; black, t-values. The left vertical scale
indicates the amplitude of the MEG response (1013 tesla) whereas the right one
shows the t-value. A value t 1.99 (horizontal green line) corresponds to
P 0.05 (uncorrected for comparisons at multiple time points).
the M100 response were weaker than those for the affect responses in visual areas V1, V2, V3, VP,
M170 response. V3A and V4v in humans22. We found no differ-
The earlier latency and somewhat more poste- ence in the amplitude of the M100 in each hemi-
rior scalp distribution of the M100 compared to sphere for contralaterally versus ipsilaterally
the M170 suggest that the two components may presented faces (F1,10 1), indicating that the
not originate from the same anatomical source. To source of this component must be beyond retino-
test whether the M100 might originate in retinotopic cortex.
topically-organized visual cortex, we separately
measured the amplitude of the M100 in each Decoupling Categorization
hemisphere to faces presented in the contralateral and Identification
versus ipsilateral visual field (2.8º off fixation) in The results described so far implicate both the
experiment 3. This manipulation is known to M100 and M170 in face processing, but do not
RT0996_C06.qxd 11/8/04 2:02 PM Page 78
technique similar to the recently proposed RISE

technique23,24 (Methods), each subject was then
run on a psychophysical staircase procedure in
which the percentage of phase coherence of each
test stimulus was gradually reduced until the
subject reached threshold performance on the
matching task (75% correct, 20 steps by QUEST
staircase25). In this way, five threshold face stim-
uli and five threshold house stimuli were con-
structed for each subject for each of the two tasks
(categorization and identification). Across all
stimuli and subjects, the resulting threshold face
FIGURE 6.2 ■ Amplitudes of the peak M100 response, and house stimuli had a mean percent phase co-
averaged across subjects, to faces and a variety of herence of 14% (face) and 18% (house) for the
nonface objects at predefined sensors of interest. The categorization task and 38% (face) and 51%
error bars show the standard deviation across subjects
of the difference of the M100 amplitudes between faces
(house) for the identification task, indicating that
and each category of nonface object. more stimulus information was necessary for the
identification task than for the categorization
task, as expected.
indicate what aspect of face processing each Next, each subject performed the same match-
component reflects. In experiment 4, we asked ing task (different category, different individual, or
whether each component is involved in the cate- same individual) in the MEG scanner, using face
gorization of a stimulus as a face, in the extraction and house stimuli with phase coherence varied
of the individual identity of a face, or both. Subjects across four levels: 0%, 90%, and the two previously-
were instructed to make two judgments about derived sets of thresholds for that subject—one for
each stimulus, determining both its category (face the categorization task, and the other for the iden-
or house) and its individual identity. tification task (Figure 6.3b). In addition, the origi-
In this experiment, ten subjects matched front- nal version of each image with unmodified spatial
view test images of faces and houses to profile frequencies was included to localize face-selective
views (faces) or three-quarter views (houses) of SOIs. By measuring both categorization and iden-
the same stimulus set presented earlier in the tification performance on each trial, the task al-
same trial (Figure 6.3a). There were three possi- lowed us to decouple the MEG correlates of suc-
ble responses: ‘different category’ if the sample cessful categorization from those of successful
was a face and the test was a house or vice versa, identification. To obtain the MEG correlates of
‘different individual but same category’ and successful categorization, we compared the aver-
‘same individual’. Correct categorization required age MEG response to the same test image when
discrimination between ‘different category’ trials the subject correctly categorized but failed to iden-
and either ‘different individual’ or ‘same indi- tify it, versus when they categorized it incorrectly.
vidual’ trials; correct identification required For identification, we compared the response to
distinguishing between ‘different individual’ and the same test image when the subject correctly
‘same individual’ trials. identified it versus when they incorrectly identi-
A set of face and house stimuli (five exemplars fied it but categorized it successfully.
each) were constructed, each of which had iden- MEG waveforms averaged across each subject’s
tical spatial frequency, luminance and contrast face-selective SOIs from the face categorization
spectra. Subjects were first trained to match each and identification tasks are shown in Figure 6.4a.
face and house with its profile or three-quarter The magnitudes of both the M100 (F1,9 9.5,
view, respectively, at 100% accuracy. Using a P 0.02) and the M170 (F1,9 5.8, P 0.05)
RT0996_C06.qxd 11/8/04 2:02 PM Page 79
a)
b)
FIGURE 6.3 ■ Stimulus and task for experiment 4. (a) In each trial, a sample stimulus was followed (after a de-
lay) by a test stimulus. The sample images (5 exemplars for each category) were profile-view faces or three-
quarter-view houses. (b) Test stimuli were frontal views of the sample stimuli. The phase coherence of the test
stimuli was varied from 0% (visual noise) to 90% in four levels; original images with 100% coherence were also
included. Here we show the data for the stimuli presented at categorization and identification thresholds only.
were larger for successful than for unsuccessful Accuracy on categorization and identification
categorization of faces (Figure 6.4b, top left). How- tasks at the two levels of phase coherence (derived
ever, only the M170 (F1,9 43.3, P 0.001), but from the previous psychophysical measurements)
not the M100 (F1,9 1), was higher for correct is shown in Table 6.1. Pairwise t-tests revealed no
than for incorrect identification of faces (interac- significant difference (P 0.2 in all cases) in
tion, F1,9 8.7, P 0.02) (Figure 6.4b, bottom accuracy as a function of stimulus category (faces
left). For house stimuli, neither the M100 nor the versus houses) for either task (categorization versus
M170 differed for correct versus incorrect trials in identification). Therefore, any difference between
either task (F 1 in all cases; Figure 6.4b, top right faces and houses seen in MEG responses cannot be
and bottom right). The finding that the M170 is
specific for face identification (not house identifi- TABLE 6.1. Accuracy as a Function of Task and Stimulus
cation) is further supported by a significant three- Category (Experiment 4, Guessing Corrected)
way interaction (F1,9 6.73, P 0.03) of face ver- Categorization Identification
sus house identification, success versus failure, and task task
M100 versus M170 (Figure 6.4b, bottom left and Face House Face House
right). In addition, neither the main effect of hemi- Categorization
sphere nor the interaction of task by hemisphere threshold 74% 72% 26% 19%
Identification threshold 95% 95% 73% 65%
was significant (F 1 in both cases).
RT0996_C06.qxd 11/8/04 2:02 PM Page 80
FIGURE 6.4 ■ Categorization versus identification. (a) MEG waveforms from the
face categorization (left) and identification (right) tasks. Dark, success; light, failure.
The waveforms were generated by averaging the selected raw data (Methods)
from independently defined SOIs in ten subjects. (b) The amplitudes of the M100
and M170 at SOIs averaged across subjects. Successful categorization of faces
elicited higher amplitudes at both M100 and M170 (top left), but no significant dif-
ference was found between successfully and unsuccessfully categorized houses
at the predefined SOIs (top right). Correctly identified (compared to incorrectly
identified) faces produced a significantly larger amplitude of the M170, but not
of the M100 (bottom left). The amplitude elicited by houses was not affected by
success or failure in the identification task (bottom right).
explained in terms of differences in behavioral per- between the two processes. One possible account
formance. Note that even when the stimuli were of this dissociation is that the selectivity of the
degraded to ‘categorization threshold level’, the underlying neural population may continuously
subjects’performances in the identification task was sharpen over time, permitting crude discrimina-
above chance (P 0.01 in both cases). tions (for example, between a face and a house)
In sum, both the M100 and M170 are correlated earlier, and more fine-grained discriminations
with successful face categorization, but only the (for example, between two different faces) later.
later M170 component is correlated with success- Indeed, the ratio of the response to faces versus
ful face identification, indicating a dissociation houses was lower for the M100 (1.6) than for the
RT0996_C06.qxd 11/8/04 2:02 PM Page 81
M170 (1.8, interaction P 0.03), showing that se- The responses to configuration and part stimuli
lectivity does indeed sharpen over time. However, recorded at independently defined face-selective
this fact alone does not indicate whether the sensors, averaged across subjects, are shown in
selectivity changes only in degree, or whether Figure 6.5b. Critically, we found a significant
qualitatively different representations underlie the two-way interaction of M100 versus M170
M100 and the M170. This question was addressed by configuration versus parts (F1,13 13.4, P
in experiment 5, in which we measured the M100 0.005). This interaction reflects the fact that the
and M170 responses to information about face amplitude of the M100 was significantly larger
configurations and face parts. for parts stimuli than for configuration stimuli
In experiment 5, two face-like stimulus cate- (F1,13 11.5, P 0.005), whereas a trend in
gories were constructed from veridical human the opposite direction was found for the M170
faces by orthogonally eliminating or disrupting (F1,13 3.35, P 0.09). Thus it is not merely the
either face configuration or face parts (eyes, nose degree of selectivity, but the qualitative nature of
and mouth; Figure 6.5a). Specifically, for ‘config- the selectivity, that differs between the M100 and
uration’ stimuli, face parts in each stimulus were the M170. Again, neither the main effect of hemi-
replaced by solid black ovals in their correspon- sphere nor the interaction of stimulus type by
ding locations, preserving the face configuration hemisphere was significant (F 1 in both cases).
but removing the contribution of face parts
(Figure 6.5a, left). Conversely, for ‘parts’ stimuli, Discussion
the face parts were kept intact but were rearranged
into a novel nonface configuration (Figure 6.5a In experiments 1–3, we report an MEG response
right). We measured MEG responses in 14 subjects component, occurring over occipitotemporal
who, while fixating, passively viewed these two cortex and peaking at a latency of ~100 ms, that is
sets of face-like stimuli (50 exemplars each) significantly larger for faces than for a variety of
presented in a random order. nonface objects. This result indicates that the
categorization of a stimulus as a face begins
within 100 ms after stimulus onset, substantially
earlier than previously thought7,20,26.
a) Unlike prior reports of very early category-
selective ERP responses13–18, the M100 reported
here cannot be explained in terms of differences
in the low-level features present in the stimuli.
The M100 response was stronger when the same
face stimulus was correctly perceived as a face than
b) when it was wrongly categorized as a nonface. This
result shows that the M100 reflects the subject’s
percept, not simply the low-level properties of the
stimulus.
It is possible that a correlate of the face-selective
M100 could be obtained with ERPs. However,
because MEG is sensitive to only a subset of the
neural activity that can be detected with scalp
ERPs27, there is no simple correspondence between
MEG responses and scalp ERP responses, and se-
FIGURE 6.5 ■ Face configuration versus face parts. (a)
Example stimuli. (b) Amplitudes of the M100 and the
lectivities that are clear in the MEG response may
M170 response, averaged across subjects, to configura- be diluted with ERPs. Similarly, the M170 response
tion and parts stimuli at predefined sensors of interest. measured with MEG probably corresponds to only
RT0996_C06.qxd 11/8/04 2:02 PM Page 82
one of the two sources hypothesized to underlie the identification tasks (experiment 4). Furthermore,
N170 response7,28. On the other hand, direct corre- the difference in the response for successful versus
spondences may exist between the M100 and M170 unsuccessful trials on face stimuli cannot be ex-
and the more focal intracranial P150 and N200 plained by general processes such as attention or
ERPs21, respectively, assuming that the later laten- association, because neither the M100 nor the
cies in the intracranial case arise from medication M170 amplitude differed for correct versus incor-
and/or histories of epilepsy typical of that subject rect trials on house stimuli. Thus, our data argue
population. Unfortunately, the limitations in current strongly that the categorization of a stimulus as a
source localization techniques leave these corre- face begins substantially earlier than the identifica-
spondences only tentative at present. tion of the particular face.
The latency of the M100 response is not directly Are these two stages—categorization and
comparable to the category-selective response that identification—simply different points on a contin-
occurs at a latency of 100 ms in inferotemporal uous spectrum of discrimination, with cruder dis-
(IT) neurons in macaques29,30 because all cortical criminations occurring at earlier latencies and finer
response latencies are shorter in macaques than discriminations occurring later, perhaps as initially
humans. For example, V1 responses occur 40–60 ms coarse neural population codes get sharpened over
after stimulus presentation in macaques31—about time34,35? Consistent with this hypothesis, the
20 ms earlier than they do in humans26,32. M170 shows stronger face selectivity than the
Given that at least 60–80 ms are thought to be M100. However, this hypothesis predicts that
necessary for visual information to reach primary the rank ordering of preferred stimuli must be the
visual cortex in humans32, this leaves only an same for the M100 and the M170. Contrary to this
additional 20–40 ms for the first face-selective prediction, the M100 showed a stronger response
responses to be generated in cortex. Such latencies to stimuli depicting face parts than face configura-
are hard to reconcile with models of visual catego- tions, whereas the M170 showed the opposite
rization that rely heavily on iterative feedback response profile (experiment 5). If neural popula-
loops and/or recurrent processing, and strengthen tions simply sharpened the selectivity of their
the claim that initial stimulus categorization is ac- response over time, this preference reversal would
complished by largely feedforward mechanisms33. not be seen. Instead, our data suggest that qualita-
The second major finding of this study is that tively different information is extracted from faces
both the M100 and the M170 are correlated with at 100 ms versus 170 ms after stimulus onset. Fi-
successful face categorization, but only the later nally, the observed change in response profile can-
M170 component is correlated with successful not be easily explained in terms of a progression
face identification. This finding indicates that from coarse/global information to fine/local infor-
processes critical for the identification of a face mation or in terms of a progression from less to
begin at a substantially later latency than more clear face features. Instead, information
processes critical for the categorization of a stim- about relatively local face parts is more important
ulus as a face. Evidently, our expertise with faces in determining the M100 response, whereas infor-
has not led us to be able to identify individual mation about relatively global face configurations
faces as early as we can tell they are faces at all (as is more important in the later M170 response. Thus
argued in ref. 5). the most natural account of our data is that the
The dissociation we report here between the M100 and the M170 reflect qualitatively distinct
processes underlying face categorization and those stages of face perception: an earlier stage that is
underlying face identification do not simply reflect critical for categorizing a stimulus as a face, which
the greater difficulty of identification compared to relies more on information about face parts, and a
categorization, because our results were obtained later stage that is critical for identifying individual
even when the stimuli were adjusted to produce faces, which relies more on information about face
identical performance in the categorization and configurations.
RT0996_C06.qxd 11/8/04 2:02 PM Page 83
Will the findings reported here hold for stimu- For each subject in experiment 1, t-tests were
lus classes other than faces? Given the numerous conducted between the MEG responses to faces
sources of evidence that faces are ‘special’36, we and houses at each time point (from 100 to 400
cannot simply assume that they will. Unfortu- ms; 500 time points) and each sensor (64 chan-
nately, we cannot run experiments comparable to nels) separately. SOIs were defined as those sen-
those reported here on other stimulus categories, sors where the magnetic fields evoked by faces
because we have not found MEG markers selec- were significantly larger than those by houses
tive for other categories. However, recent behav- (P 0.05) for at least five consecutive time points
ioral experiments suggest that the stages of pro- both within the time window centered at the
cessing reported here for face recognition will latency of the M100 and within that of the M170.
generalize to the visual recognition of nonface P-values for these SOI-defining statistics were not
objects as well6. corrected for multiple sensors or multiple time
point comparisons. All critical claims in this pa-
per are based on analyses of the average responses
Methods over these sensors in independent data sets, and
thus require no correction for multiple spatial
MEG recordings for experiments 1–3 were
hypotheses.
made using a 64-channel whole-head system
The peak amplitude of the M100 (maximum
with SQUID-based first-order gradiometer sen-
deflection) was determined for each stimulus type
sors (Kanazawa Institute of Technology MEG
in each hemisphere within a specified time win-
system at the KIT/MIT MEG Joint Research Lab
dow (width 40 ms) for each subject individu-
at MIT); experiments 4 and 5 were run after the
ally. Because there was no main effect of hemi-
system was upgraded to 96 channels. Magnetic
sphere (P 0.05) and no interaction of condition
brain activity was digitized continuously at a
by hemisphere (F 1), in subsequent analyses
sampling rate of 1,000 Hz (500 Hz for experi-
the data from the left and right hemisphere were
ment 4) and was filtered with 1-Hz high pass and
averaged within each subject (after flipping the
200-Hz low-pass cutoff and a 60-Hz notch.
sign of the data from the right hemisphere to
Informed consent was obtained from all subjects,
match the polarities).
and the study was approved by the MIT Committee
on the Use of Humans as Experimental Subjects Experiment 4: Categorization
(COUHES). Versus Identification
Ten subjects (age range 22–32) participated in ex-
Experiments 1-3: The Face-Selective periment 4. The MEG recordings were preceded
M100 Response by a training session ( 10 min) and then a
Fifteen subjects (age range 18–40) passively psychophysical staircase adjustment session
viewed 100 intermixed trials of faces and houses conducted in the MEG room. MEG data from two
(50 exemplars each) in experiment 1; two addi- additional subjects were discarded, one because
tional subjects’ data were discarded due to self- of performance that was more than two standard
reported sleepiness. The thirteen subjects who deviations below the mean, the other because of
showed the early face preference over occipi- polarities of both M100 and M170 that were re-
totemporal cortex also performed a one-back task versed compared to all other subjects (although
on faces and a variety of nonface objects (50 including the data from this subject did not
exemplars each) in experiment 2. Each image change the pattern or significance of the results).
subtended 5.7 5.7º of visual angle and was pre- Noise images were generated by inverse
sented at the center of gaze for 200 ms, followed Fourier transformation of the mean amplitude
by an ISI of 800 ms. The design for experiment 3 spectra with randomized phase spectra23,24,37.
is described in Results. Intermediate images containing x% phase spectra
RT0996_C06.qxd 11/8/04 2:02 PM Page 84
of original images and (100 – x)% random phase arrangements of nonface configurations varied
spectra were generated using linear interpolation across all exemplars of parts stimuli. Pho-
(phase spectra levels of 0% and 90% along with tographs of these stimuli were presented at the
categorization and identification thresholds). This center of gaze for 200 ms with an 800-ms ISI.
procedure ensured that all images were equated Fourteen subjects (age range 18–41) passively
for spatial frequency, luminance and contrast. viewed 100 intermixed trials of each stimulus
Analyses were done on only the subset of data category (50 exemplars each); three additional
for which subjects responded both correctly and subjects’ data were discarded due to lack of a
incorrectly to an identical stimulus. That is, for face-selective MEG response in the independent
each stimulus, equal numbers of successful and localizer scan.
unsuccessful trials were chosen, and the extra
trials were omitted from the analysis from
whichever condition had more trials. In particular, Acknowledgments
because there were more correct than incorrect We thank J. Sadr and P. Sinha for helpful dis-
trials, each incorrect trial was paired with the tem- cussions on their RISE technique, M. Eimer,
porally closest correct trial. This analysis was S. Hillyard, A. Marantz, M. Valdes-Sosa, P. Down-
conducted for each stimulus, guaranteeing that ing, W. Freiwald, K. Grill-Spector, Y. Jiang and the
average MEG responses on successful and unsuc- rest of the Kanwisher Lab for comments on the
cessful trials were derived from identical stimuli. manuscript. Supported by the Reed Fund and
Finally, the MEG recordings were averaged National Eye Institute Grant (EY13455) to N.K.
across stimuli separately for successful and
unsuccessful trials. Note that the trials used to
generate the waveform for face categorization REFERENCES
were only selected from the MEG responses to
1. Rosch, E., Mervis, C. B., Gray, W. D., Johnson, D. M. &
those stimuli degraded to ‘categorization thresh- Boyes-Braem, P. Basic objects in natural categories.
olds’, and the trials used to generate the waveform Cognit. Psychol. 8, 382–349 (1976).
for face identification were only selected from the 2. Jolicoeur, P., Gluck, M. A. & Kosslyn, S. M. Pictures and
MEG responses to those stimuli degraded to names: making the connection. Cognit. Psychol. 16,
243–275 (1984).
‘identification thresholds’. The same held for
3. Sugase, Y., Yamane, S., Ueno, S. & Kawano, K. Global
house categorization and identification. The exact and fine information coded by single neurons in the tem-
number of success and failure trials for each task poral visual cortex. Nature 400, 869–873 (1999).
varied across subjects, but ranged from 20 to 30 4. Tanaka, J. W. & Taylor, M. Object categories and expertise:
trials each for successful and unsuccessful catego- is the basic level in the eye of the beholder? Cognit.
Psychol. 23, 457–482 (1991).
rization and from 15 to 20 trials for successful and
5. Tanaka, J. W. The entry point of face recognition:
unsuccessful identification. evidence for face expertise. J. Exp. Psychol. Gen. 130,
534–543 (2001).
Experiment 5: Face Configuration 6. Grill-Spector, K. & Kanwisher, N. Common cortical
Versus Face Parts mechanisms for different components of visual object
recognition: a combined behavioral and fMRI study.
Two stimulus categories were constructed from
J. Vision 1, 474a (2001).
veridical faces. In ‘configuration’ stimuli, face 7. Bentin, S., Allison, T., Puce, A., Perez, E. & McCarthy,
parts in each veridical face were replaced by G. Electrophysiological studies of face perceptions in
solid black ovals in their corresponding loca- humans. J. Cogn. Neurosci. 8, 551–565 (1996).
tions, whereas for ‘parts’ stimuli, the face parts 8. Jeffreys, D. A. Evoked potential studies of face and object
processing. Vis. Cogn. 3, 1–38 (1996).
were kept intact but were rearranged into a novel
9. Sams, M., Hietanen, J. K., Hari, R., Ilmoniemi, R. J. &
nonface configuration. The size of the black ovals Lounasmaa, O. V. Face-specific responses from the hu-
in the configuration stimuli was matched to the man inferior occipito-temporal cortex. Neuroscience 77,
actual size of corresponding face parts, and the 49–55 (1997).
RT0996_C06.qxd 11/8/04 2:02 PM Page 85
10. Liu, J., Higuchi, M., Marantz, A. & Kanwisher, N. The 23. Sadr, J. & Sinha, P. Exploring object perception with
selectivity of the occipitotemporal M170 for faces. Random Image Structure Evolution. MIT Artif. Int. Lab.
Neuroreport 11, 337–341 (2000). Memo, 2001–6 (2001).
11. Bruce, V. & Young, A. Understanding face recognition. 24. Sadr. J. & Sinha, P. Object recognition and random image
Br. J. Psychol. 77 (Pt 3), 305–327 (1986). structure evolution. Cognit. Sci. (in press).
12. Eimer, M. The face-specific N170 component reflects late 25. Watson, A. B. & Pelli, D. G. Quest: a Bayesian adaptive
stages in the structural encoding of faces. Neuroreport 11, psychometric method. Percept. Psychophys. 33, 113–120
2319–2324 (2000). (1983).
13. Seeck, M. et al. Evidence for rapid face recognition from 26. Thorpe, S., Fize, D. & Marlot, C. Speed of processing in
human scalp and intracranial electrodes. Neuroreport 8, the human visual system. Nature 381, 520–522 (1996).
2749–2754 (1997). 27. Hämäläinen, M., Hari, R., Ilmoniemi, R. J., Knuutila, J. &
14. Schendan, H. E., Ganis, G. & Kutas, M. Neurophysiolog- Lounasmaa, O. V. Magnetoencephalography: theory,
ical evidence for visual perceptual categorization of instrumentation and applications to noninvasive studies of
words and faces within 150 ms. Psychophysiology 35, the working human brain. Rev. Mod. Phys. 65, 413–497
240–251 (1998). (1993).
15. Mouchetant-Rostaing, Y., Giard, M. H., Bentin, S., 28. Bentin, S. in Encyclopedia of Cognitive Science (ed.
Aguera, P. E. & Pernier, J. Neurophysiological correlates Nadel, L.) Neural basis of face perception. (Macmillan,
of face gender processing in humans. Eur. J. Neurosci. 12, London, in press).
303–310 (2000). 29. Oram, M. W. & Perrett, D. I. Time course of neural
16. Kawasaki, H. et al. Single-neuron responses to emotional responses discriminating different views of the face and
visual stimuli recorded in human ventral prefrontal cortex. head. J. Neurophysiol. 68, 70–84 (1992).
Nat. Neurosci. 4, 15–16 (2001). 30. Rolls, E. T. Neurons in the cortex of the temporal lobe and
17. Braeutigam, S., Bailey, A. J. & Swithenby, S. J. Task- in the amygdala of the monkey with responses selective
dependent early latency (30–60 ms) visual processing for faces. Hum. Neurobiol. 3, 209–222 (1984).
of human faces and other objects. Neuroreport 12, 31. Thorpe, S. J. & Fabre-Thorpe, M. Seeking categories in
1531–1536 (2001). the brain. Science 291, 260–263 (2001).
18. Streit, M., Wolwer, W., Brinkmeyer, J., Ihl, R. & Gaebel, 32. Gomez Gonzalez, C. M., Clark, V. P., Fan, S., Luck, S. J.
W. Electrophysiological correlates of emotional and & Hillyard, S. A. Sources of attention-sensitive visual
structural face processing in humans. Neurosci. Lett. 278, event-related potentials. Brain Topogr. 7, 41–51 (1994).
13–16 (2000). 33. Thorpe, S. & Imbert, M. Connectionism in Perspective
19. George, N., Jemel, B., Fiori, N. & Renault, B. Face and (eds. Pfeifer, R., Schreter, Z., Fogelman-Soulie, F. &
shape repetition effects in humans: a spatiotemporal ERP Steels, L.) 63–92 (Elsevier, Amsterdam, 1989).
study. Neuroreport 8, 1417–1423 (1997). 34. Kovacs, G., Vogels, R. & Orban, G. A. Cortical correlate
20. VanRullen, R. & Thorpe, S. J. The time course of visual of pattern backward masking. Proc. Natl. Acad. Sci. USA
processing: from early perception to decision-making. 92, 5587–5591 (1995).
J. Cogn. Neurosci. 13, 454–461 (2001). 35. Keysers, C., Xiao, D. K., Foldiak, P. & Perrett, D. I. The
21. Allison, T., Puce, A., Spencer, D. D. & McCarthy, G. speed of sight. J. Cogn. Neurosci. 13, 90–101 (2001).
Electrophysiological studies of human face perception. I: 36. Farah, M. J. Visual Cognition (eds. Kosslyn, S. M. &
Potentials generated in occipitotemporal cortex by face Osherson, D. N.) 101–119 (MIT Press, Cambridge,
and non-face stimuli. Cereb. Cortex 9, 415–430 (1999). Massachusetts, 1995).
22. Tootell, R. B., Mendola, J. D., Hadjikhani, N. K., Liu, A. 37. Rainer, G., Augath, M., Trinath, T. & Logothetis, N. K.
K. & Dale, A. M. The representation of the ipsilateral Nonmonotonic noise tuning of BOLD fMRI signal to
visual field in human cerebral cortex. Proc. Natl. Acad. natural images in the visual cortex of the anesthetized
Sci. USA 95, 818–824 (1998). monkey. Curr. Biol. 11, 846–854 (2001).
RT0996_C06.qxd 11/8/04 2:02 PM Page 86
RT0996_C07.qxd 11/8/04 2:03 PM Page 87
R E A D I N G 7
Distributed and Overlapping Representations of

Faces and Objects in Ventral Temporal Cortex
•
James V. Haxby,1 M. Ida Gobbini,1,2 Maura L. Furey,1,2 Alumit Ishai,1
Jennifer L. Schouten,1 and Pietro Pietrini3
The functional architecture of the object vision pathway in the human brain was investigated using
functional magnetic resonance imaging to measure patterns of response in ventral temporal cortex
while subjects viewed faces, cats, five categories of man-made objects, and nonsense pictures.
A distinct pattern of response was found for each stimulus category. The distinctiveness of the
response to a given category was not due simply to the regions that responded maximally to that
category, because the category being viewed also could be identified on the basis of the pattern of
response when those regions were excluded from the analysis. Patterns of response that
discriminated among all categories were found even within cortical regions that responded
maximally to only one category. These results indicate that the representations of faces and objects
in ventral temporal cortex are widely distributed and overlapping.
he ventral object vision pathway in the human differential tuning of individual neurons in tempo-
Tresentations
brain has the capacity to generate distinct rep-
for a virtually unlimited variety of
ral cortex to faces, whole objects, and complex
object form features (1–3). Although columns
individual faces and objects, but the functional containing cells that respond selectively to faces or
architecture that embodies this capacity is a matter similar features tend to cluster together, these stud-
of intense debate. Single-cell recording studies ies have not revealed any consistent larger scale
in the nonhuman primate have demonstrated organization for object representation. Numerous
1
Laboratory of Brain and Cognition, National Institute of Sciences and 3Laboratory of Clinical Biochemistry, Department
Mental Health, National Institutes of Health, Bethesda, MD of Experimental Pathology, University of Pisa, 1-56126 Pisa,
20892, USA. 2Department of Human and Environmental Italy.
87
RT0996_C07.qxd 11/8/04 2:03 PM Page 88
computational models for object recognition have According to this model, which we have named
been developed (4), but the correspondence between “object form topography,” ventral temporal cortex
these models and the neural architecture of the has a topographically organized representation of
ventral object vision pathway is uncertain. attributes of form that underlie face and object
Unlike single-cell studies, functional brain recognition. The representation of a face or object
imaging has revealed a large-scale spatial organi- is reflected by a distinct pattern of response across
zation for specialization within the ventral object a wide expanse of cortex in which both large- and
vision pathway, as demonstrated by differential small-amplitude responses carry information
patterns of response, i.e., increases in neural ac- about object appearance. Unlike the other models,
tivity indicated by localized increases in blood object form topography predicts how all cate-
oxygenation, to faces and other categories of gories might evoke distinct patterns of response in
objects in ventral temporal cortex (5–17). Models ventral temporal cortex and, thereby, provides an
for this functional architecture fall into three explicit account for how this cortex can produce
classes. One model proposes that ventral temporal unique representations for a virtually unlimited
cortex contains a limited number of areas that are number of categories.
specialized for representing specific categories of We tested our model by investigating the pat-
stimuli (5–8). Thus far, two specialized areas have terns of response evoked in ventral temporal cortex
been described: the fusiform face area (FFA) and by faces and multiple categories of objects. Our
the parahippocampal place area (PPA) (Figure 7.1). model predicts that each category elicits a distinct
A second model proposes that different areas in pattern of response in ventral temporal cortex that
ventral temporal cortex are specialized for different is also evident in the cortex that responds maxi-
types of perceptual processes (9–11). In particu- mally to other categories.
lar, this model proposes that the FFA is special-
ized for expert visual recognition of individual
exemplars from any object category, not just Analysis of Patterns of Neural Response
faces. The third model proposes that the represen- to Object Categories
tations of faces and different categories of objects
are widely distributed and overlapping (12–15). Patterns of response were measured with func-
tional magnetic resonance imaging (fMRI) in six
subjects while they viewed pictures of faces, cats,
five categories of man-made objects (houses,
chairs, scissors, shoes, and bottles), and control,
nonsense images (18) (Figure 7.2). The data were
analyzed to determine whether each stimulus cat-
egory evoked a pattern of response in the ventral
object vision pathway that could be distinguished
from the patterns of response evoked by all other
individual categories. Patterns of response were
examined in ventral temporal object-selective
cortex, defined as those voxels with responses that
differed significantly by category. The data for
each subject were split into two sets, namely even
FIGURE 7.1 ■ Schematic diagram illustrating the locations and odd runs. We then determined whether the
of the fusiform face area (FFA), which also has been stimulus category that a subject was viewing
implicated in expert visual recognition, and the parahip-
pocampal place area (PPA) on the ventral surface of
could be identified by examining the similarity
the right temporal lobe. In most brains, these areas are between the patterns of response evoked by each
bilateral. category on even and odd runs (19).
RT0996_C07.qxd 11/8/04 2:03 PM Page 89
Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex ■ 89
Distinct Patterns of Neural Response for

Multiple Categories of Objects
The pattern of response in object-selective ventral
temporal cortex correctly identified the category
being viewed in 96% of pairwise comparisons
(20). The pattern of response indicated when sub-
jects were viewing faces, houses, and scrambled
pictures with no errors (Table 7.1). Identification
accuracy for the small man-made objects (bottles,
scissors, shoes, and chairs) was significantly better
than chance for each category (21).
Category Identification Based on

Patterns of Nonmaximal Responses
Although these results suggest that category-
specific patterns of response are distributed and
overlapping, higher within-category correlations
could be due simply to the regions that reliably
respond maximally to each category, with no in-
formation about a specific category carried by the
pattern of response in cortex that responded max-
imally to other categories. To test whether the
patterns of non-maximal responses carry
category-related information, we analyzed
whether each stimulus category evoked a distinct
pattern of response in cortex that responded max-
imally to other categories. For each comparison
between patterns of response evoked by two cate-
gories, all of the voxels that responded maximally
to either category in either half of the data were
FIGURE 7.2 ■ Examples of stimuli. Subjects performed a excluded from the calculation of correlations
one-back repetition detection task in which repetitions of
meaningful pictures were different views of the same
(22). The specificity of the pattern of response to
face or object. each category was barely diminished by thus re-
stricting the analysis (Figure 7.4), with a mean ac-
Correlations between patterns of response served curacy of 94% for identifying the category being
as indices of similarity (Figure 7.3). For example, to viewed (Table 7.1) (23).
determine whether the pattern of response to one
category, such as chairs, could be distinguished
from the pattern of response to a different category, Patterns of Response within Cortical
such as shoes, the correlation between the pattern of Regions That Respond Maximally to
response to chairs on even runs and the response to One Category
chairs on odd runs (within-category correlation)
was compared with the correlation between the re- These results indicate that the category specificity
sponse to chairs on even runs and the response to of responses in ventral temporal cortex is not re-
shoes on odd runs (between-category correlation). stricted solely to regions that respond maximally
RT0996_C07.qxd 11/8/04 2:03 PM Page 90

RT0996_C07.qxd 11/8/04 2:03 PM Page 91
TABLE 7.1. Accuracy of Identification of the Category being Viewed Based on the Patterns of Response Evoked
in Ventral Temporal Cortex. Accuracies are the Percentage of Comparisons Between Two Categories that
Correctly Identified which Category was being Viewed.
Identification accuracy (%)
Volume
(cm3 SE)
Region Faces Houses Cats Bottles Scissors Shoes Chairs Scrambled
All ventral 22.9 2.8 100*** 100*** 98 2*** 90 6*** 92 6*** 92 7*** 96 2*** 100***
temporal
object-
selective
cortex
Minus regions 15.4 1.8 100*** 100*** 95 2*** 89 6*** 85 9** 90 8** 98 1*** 100***
that were
maximally
responsive to
categories
being
compared
Regions
maximally
responsive to:
Faces 3.1 0.9 94 7*** 99 1*** 76 13* 81 14* 77 9* 70 16 77 11* 92 7***
Houses 9.6 1.8 100*** 100*** 88 5*** 85 10** 81 6** 96 2*** 94 3*** 100***
Cats 2.6 0.4 96 4*** 96 2*** 82 8** 65 11 69 5** 76 9* 95 4*** 100***
Small objects 6.9 1.1 100*** 100*** 95 3*** 83 7** 92 8** 94 6*** 90 6*** 96 4***
Differs from chance (50%):*, P 0.05;**, P 0.01;***, P 0.001.
to certain stimuli, thus raising the question of viewed with 93% accuracy. Within only the cortex
whether the representation of faces and objects in that responded maximally to small, man-made
this cortex has a topographic organization that ex- objects, the pattern of response identified the
ists with a finer spatial resolution than that defined category being viewed with 94% accuracy. Even
by such regions. To investigate whether the cate- within the much smaller region that responded
gory specificity of response exists at this finer maximally to faces, the pattern of response iden-
spatial resolution, we examined the patterns of re- tified the category being viewed with 83% ac-
sponse within regions that responded maximally curacy, and accuracies were significantly better
to a single category or a small set of categories than chance for all categories except shoes. Simi-
(22) (Table 7.1). Within only the cortex that larly, the pattern of response within the region
responded maximally to houses, the pattern of that responded maximally to cats identified the
response correctly identified the category being category being viewed with 85% accuracy, with
FIGURE 7.3 ■ (A color version of this figure follows page 146.) The category specificity of patterns of response was
analyzed with pairwise contrasts between within-category and between-category correlations. The pattern of response
to each category was measured separately from data obtained on even-numbered and odd-numbered runs in each in-
dividual subject. These patterns were normalized to a mean of zero in each voxel across categories by subtracting the
mean response across all categories. Brain images shown here are the normalized patterns of response in two axial
slices in a single subject. The left side of the brain is on the left side of each image. Responses in all object-selective
voxels in ventral temporal cortex are shown. For each pairwise comparison, the within-category correlation is compared
with one between-category correlation. (A) Comparisons between the patterns of response to faces and houses in one
subject. The within-category correlations for faces (r 0.81) and houses (r 0.87) are both markedly larger than the
between-category correlations, yielding correct identifications of the category being viewed. (B) Comparisons between
the patterns of response to chairs and shoes in the same subject. The category being viewed was identified correctly
for all comparisons. (C) Mean response across all categories relative to a resting baseline.
RT0996_C07.qxd 11/8/04 2:03 PM Page 92
accuracies that were better than chance for all

categories except bottles.
These results demonstrate that the pattern of
response in ventral temporal cortex carries infor-
mation about the type of object being viewed,
even in cortex that responds maximally to other
categories, but the nature of this information is
unknown. To examine whether this information
concerns only low-level features of gray-scale
photographs that are shared by a category, such as
mean luminance, mean contrast, and spatial fre-
quencies, we reanalyzed data from a previous
study in which subjects viewed photographs and
line drawings of three categories (faces, house,
and chairs) (13). We examined whether the pat-
tern of response to a category of line drawings can
be identified on the basis of its similarity to re-
sponses to photographs of the same and different
categories and, conversely, whether the pattern of
response to a category of photographs can be
identified on the basis of its similarity to re-
sponses to line drawings. The results of this
reanalysis showed that similarities between pat-
terns of response to photographs and line draw-
ings of the same category correctly identified the
category being viewed, even when the analysis
was restricted to cortex that did not respond
maximally to either of the categories being dis-
criminated (96% correct pairwise discriminations)
[for detailed results, see supplemental material
(24)]. This result shows that patterns of nonmaxi-
mal responses do not represent low-level features
that are specific to the type of stimuli, such as
photographs, but, rather, appear to reflect infor-
mation that is more definitive of object category.
FIGURE 7.4 ■ Mean within-category and between-

category correlations (SE) between patterns of re- Discussion
sponse across all subjects for all ventral temporal object-
selective cortex and for ventral temporal cortex excluding These findings demonstrate distinct patterns of
the cortex that responded maximally to either of two cat- response in ventral temporal cortex for multiple cat-
egories being compared. The SE of within-category egories of objects, including different types of small
correlations after excluding maximally responsive cortex man-made objects. This specificity is not restricted
was based on the mean correlation across 14 pairwise
to categories for which dedicated systems might
comparisons for each subject.
have evolved because of their biological signifi-
cance. The specificity of the pattern of response for
each category was a property of a much greater
RT0996_C07.qxd 11/8/04 2:03 PM Page 93
extent of object-selective cortex in the ventral tem- We have proposed the term object form topogra-
poral lobe than the sector that responded maximally phy for the topographic organization of the
to that category. The category being viewed still distributed representation of faces and objects in
could be identified when the cortex that responded ventral temporal cortex. Our results demonstrate a
maximally to that category was excluded from the spatially organized functional architecture within
analysis. This result indicates that the representa- subregions of ventral temporal cortex that are de-
tions of faces and objects in ventral temporal cortex fined by a maximal response to a single object
are widely distributed and overlapping and that category. This architecture may be analogous to that
small or sub-maximal responses are an integral part found within early visual areas, such as V1, which
of these representations. When the analysis was fur- contain spatially organized maps of simpler visual
ther restricted to regions that responded maximally features, such as retinotopic location, edge orienta-
to a single category (houses, faces, or cats) or a tion, and color. Object form topography presumably
small number of categories (i.e., man-made reflects how the more complex attributes of visual
objects—bottles, scissors, shoes, and chairs), the appearance that underlie object and face recognition
patterns of response to other categories within these are related visually, structurally, or semantically.
regions were still significantly distinct. This result
suggests that regions such as the “parahippocampal
place area” or the “fusiform face area” are not ded- Population Encoding of
icated to representing only spatial arrangements or Visual Appearance
human faces but, rather, are part of a more extended
representation for all objects (25). The representation of a face or object involves the
concerted neural activity in a widely distributed cor-
tical space. Our analysis shows that the pattern of
Object Form Topography large and small responses, not just the location of
large responses, carries category-related informa-
We have shown in previous studies that the maxi- tion, suggesting that small responses are an integral
mally responsive regions for several of these part of the representation. Population responses in
categories—faces, houses, chairs, animals, and simpler systems, such as color vision, similarly
tools—have a consistent topography across in- rely on both large and small responses to determine
dividuals (12–14). Here, we show that the topo- the quality of the integrated percept. In color vision,
graphic arrangement of the full pattern of response the perceptual quality of a hue that evokes a maxi-
is consistent within subjects, but we are not able to mal red response in red-green neurons is also de-
perform a similar correlational analysis across sub- pendent on small responses in yellow-blue neurons
jects because current methods for warping individ- that determine whether that hue is perceived as be-
ual brains to a common shape are inadequate at this ing more orange or violet (26–28).
level of detail. The spatial resolution of this topog- What attributes of visual appearance could
raphy is smaller than that defined by category- underlie a population encoding of objects and
selective areas (1 cm), because category-related faces in which both large and small responses
patterns can be discerned within these areas, and determine the quality of the integrated percept?
greater than that of randomaly arranged single Others have suggested that these attributes may be
columns or small clusters of columns (1 mm), two-dimensional, view-dependent (2) or three-
because of the spatial resolution of the fMRI im- dimensional, structural (29) primitives that make
ages in this study (3.5 mm). Single-unit record- up an alphabet for shape recognition. In a repre-
ing studies in the monkey have suggested the exis- sentation based on primitive features, however, a
tence of a columnar orgnization for representations face or object would be specified by the strong
of complex features of form but have not revealed responses that indicate the features that are pres-
any larger scale topographic arrangement (2). ent in the stimulus, not by a combination of large
RT0996_C07.qxd 11/8/04 2:03 PM Page 94
and small responses. Another possibility is sug- all object categories differ (39). By contrast, our
gested by models of face appearance and perception results indicate how ventral extrastriate cortex can
that describe a face on the basis of a small number produce unique representations for all object
of dimensions—operationalized as principal or categories. Fortuitously, these representations
independent components—that capture how con- have a consistent topographic arrangement that
figurations of features typically covary across may provide a key for decoding the information
faces (30–32). In a representation based on con- that underlies face and object recognition.
tinuous dimensions, small and intermediate re-
sponses would be as important as large responses
for specifying the location of a vector in feature REFERENCES AND NOTES
space that best describes the appearance of a 1. C. G. Gross, C. E. Rocha-Miranda, D. B. Bender,
perceived face. Psychophysical evidence (33, 34) J. Neurophysiol. 35, 96 (1972).
supports the proposal that the neural representa- 2. K. Tanaka, Annu. Rev. Neurosci. 19, 109 (1996).
3. N. K. Logothetis, D. L. Scheinberg, Annu. Rev. Neurosci.
tion of faces may be based on such dimensions,
19, 577 (1996).
represented as opponent processes referenced to 4. S. Edelman, Trends Cogn. Sci. 1, 296 (1997).
the population mean. This opponent process 5. N. Kanwisher, J. McDermott, M. Chun, J. Neurosci. 17,
model demonstrates how a limited number of 4302 (1997).
channels can represent complex variations of 6. G. McCarthy, A. Puce, J. C. Gore, T. Allison, J. Cognit.
form, such as those that distinguish one face from Neurosci. 9, 605 (1997).
7. R. Epstein, N. Kanwisher, Nature 392, 598 (1999).
another, and suggests, further, how a limited 8. G. K. Aguirre, E. Zarahn, M. D’Esposito, Neuron 21, 373
cortical space could represent an unlimited variety (1998).
of faces. 9. I. Gauthier, M. J. Tarr, A. W. Anderson, P. Skudlarski, J. C.
Our analysis did not reveal any sectors of Gore, Nature Neurosci. 2, 568 (1999).
ventral temporal cortex that did not convey in- 10. I. Gauthier, Trends Cognit. Sci. 4, 1 (2000).
11. ——, P. Skudlarski, J. C. Gore, A. W. Anderson, Nature
formation about discriminations among several Neurosci. 3, 191 (2000).
stimulus categories, which leaves open the question 12. J. V. Haxby et al., Neuron 22, 189 (1999).
of how lesions can cause selective impairments 13. A. Ishai, L. G. Ungerleider, A. Martin, J. L. Schouten,
for recognizing individual faces (prosopagnosia) J. V. Haxby, Proc. Natl. Acad. Sci. U.S.A. 96, 9379 (1999).
or discriminating between members of a single 14. L. L. Chao, J. V. Haxby, A. Martin, Nature Neurosci. 2,
913 (1999).
category of objects (35–37). Our results do not 15. J. V. Haxby, A. Ishai, L. L. Chao, L. G. Ungerleider,
address the distribution or spatial scale of patterns A. Martin, Trends Cognit. Sci. 4, 3 (2000).
of response that discriminate between exemplars 16. S. Edelman, K. Grill-Spector, T. Kushnir, R. Malach,
within a category. Moreover, it is unclear whether Psychobiology 26, 309 (1998).
17. K. Grill-Spector et al., Neuron 24, 187 (1999).
any of these syndromes can be caused by a
18. Neural responses, as reflected in hemodynamic changes,
restricted lesion in a ventral temporal region that were measured in six subjects (five female and one male)
responds maximally to one category (38). with gradient echo echoplanar imaging on a GE 3T scan-
A population encoding based on the pattern of ner (General Electric, Milwaukee, WI) [repetition time
large and small responses in a wide expanse of (TR) 2500 ms, 40 3.5-mm-thick sagittal images, field
of view (FOV) 24 cm, echo time (TE) 30 ms, flip an-
cortex has the capacity to produce unique repre-
gle 90] while they performed a one-back repetition de-
sentations of a virtually unlimited number of tection task. High-resolution T1-weighted spoiled gradi-
object categories. Models of the functional archi- ent recall (SPGR) images were obtained for each subject
tecture of ventral extrastriate cortex that analyze to provide detailed anatomy (124 1.2-mm-thick sagittal
only mean responses in regions that are putatively images, FOV 24 cm). Stimuli were gray-scale images
of faces, houses, cats, bottles, scissors, shoes, chairs, and
specialized for restricted categories of stimuli
nonsense patterns. The categories were chosen so that all
(faces and places) (5, 7) or specific perceptual stimuli from a given category would have the same base
processes (visual expertise) (9–11) provide no level name. The specific categories were selected to allow
explicit account for how neural representations of comparison with our previous studies (faces, houses,
RT0996_C07.qxd 11/8/04 2:03 PM Page 95
chairs, animals, and tools) or ongoing studies (shoes and such comparisons (within-category A versus category A on
bottles). Control nonsense patterns were phase-scrambled odd runs with category B on even runs, within-category A
images of the intact objects. Twelve time series were ob- versus category A on even runs with category B on odd runs,
tained in each subject. Each time series began and ended and similar comparisons involving the within-category cor-
with 12 s of rest and contained eight stimulus blocks of relation for category B). The probability that the accuracy of
24-s duration, one for each category, separated by 12-s identifying each individual category exceeded chance was
intervals of rest. Stimuli were presented for 500 ms with determined with a simple test: The accuracy of identifying
an interstimulus interval of 1500 ms. Repetitions of mean- that category was determined for each subject as the propor-
ingful stimuli were pictures of the same face or object tion of pairwise comparisons that yielded correct identifica-
photographed from different angles. Stimuli for each tions, and a t-test (df 5) was used to test whether the mean
meaningful category were four images each of 12 differ- accuracy across subjects exceeded chance (50%).
ent exemplars. Image data were analyzed with multiple 20. The category being viewed was correctly identified
regression with no spatial smoothing (40). To identify against all other categories in 83% of cases (within-cate-
object-selective cortex, we used an eight-regressor model. gory correlation for a given category was the greatest of
The first regressor was the contrast between stimulus all correlations between the response to that category in
blocks and rest. The remaining seven regressors modeled one half of the data and the responses to that and other cat-
the response to each meaningful category. The omnibus egories in the other half of the data; chance accuracy
effect of these seven regressors was used as a test of the would be 12.5%). Accuracies for identifying patterns for
significance of differences among the responses to stimu- individual categories as compared with all other cate-
lus categories. To determine the patterns of response to gories ranged from 100% (faces, houses, and scrambled
each category on even-numbered and odd-numbered runs, pictures) to 67% (bottles and scissors). Detailed results
we used a 16-regressor model—eight regressors to model are published as supplemental material (24).
the response to each category relative to rest on even runs 21. Identification of the category being viewed when that
and eight regressors to model the response to each cate- category was one of the four, small, man-made categories
gory on odd runs with no regressor that contrasted all (bottles, scissors, shoes, and chairs) was significantly bet-
stimulus blocks to rest. The weight for each regressor ter than chance even when only the comparisons among
was used as an estimate of the strength of response rela- these categories were considered, both for all object-
tive to rest. Volumes of interest (VOI) were drawn on the selective ventral temporal cortex (bottles: mean accuracy
high-resolution structural images to identify ventral tem- across subjects was 83% for pairwise comparisons among
poral, lateral temporal, and ventrolateral occipital cortex. small man-made objects, differs from 50%, P 0.02;
The VOI for ventral temporal cortex extended from 70 to scissors: 86%, P 0.01; shoes: 86%, P 0.02; chairs:
20 mm posterior to the anterior commissure in Talairach 97%, P 0.001) and for only cortex that responded max-
brain atlas coordinates (41) and consisted of the lingual, imally to other categories (bottles: 86%, P 0.01; scis-
parahippocampal, fusiform, and inferior temporal gyri. sors: 83%, P 0.01; shoes: 86%, P 0.02; chairs:
The VOI for lateral temporal cortex also extended from 70 to 100%). The analysis concentrated on patterns of response
20 mm posterior to the anterior commissure and consisted of in ventral temporal cortex, where the category being
the middle temporal gyrus and both banks of the superior viewed could be identified with greatest accuracy, but
temporal sulcus. The VOI for ventrolateral occipital cortex identification accuracies were nearly as great for lateral
extended from the occipital pole to 70 mm posterior to the temporal (94%) and ventrolateral occipital cortices
anterior commissure and consisted of the lingual, fusiform, (92%). The ability to identify the category being viewed
inferior occipital, and middle occipital gyri. Voxels within on the basis of the patterns of response within several sub-
these VOIs that were significantly object-selective (P regions of ventral temporal cortex as well as in ventral oc-
106, uncorrected) were used for the analysis of within- cipital and lateral temporal cortex suggests the existence
category and between-category correlations. of multiple representations that encode different types of
19. Analysis of the accuracy with which the category being information about categories, such as visual form, typical
viewed could be identified focused on comparisons between patterns of motion, and internal spatial arrangements. We
patterns of response for pairs of categories, as illustrated in have suggested, for example, that the lateral temporal
Figure 7.3. Mean response in each voxel across categories cortex represents different patterns of motion that are
was subtracted from the response to each individual cate- associated with faces and different categories of objects
gory in each half of the data before calculating correlations. (42, 43).
If the within-category correlation (for example, response to 22. For correlation analyses of subsets of object-selective cor-
category A on even and odd runs) was larger than the be- tex, the category that elicited the maximal response in
tween-category correlation (correlation of the response to each voxel was determined for even runs, odd runs, and all
category A on even runs with the response to category B on runs. To examine whether the pattern of response to a
odd runs), that comparison was counted as a correct identifi- category could be discerned in cortex that responded max-
cation. For each pair of categories, therefore, there were four imally to other categories, we restricted comparisons of
RT0996_C07.qxd 11/8/04 2:03 PM Page 96
responses to pairs of categories to voxels that did not re- 35. H. Hecaen, R. Angelergues, Arch. Neurol. 7, 24 (1962).
spond maximally to either category on either even or odd 36. A. Damasio et al., Neurology 32, 331 (1982).
runs. This was the most exacting test of this prediction 37. E. K. Warrington, T. Shallice, Brain 107, 829 (1984).
that we could devise. To examine the response in regions 38. Of the category-selective agnosias, only prosopagnosia is
that responded maximally to only a single category (faces, a purely visual agnosia, and controversy still exists over
houses, or cats) or to only small man-made objects, we whether a pure prosopagnosia exists that has no effect on
included only those voxels that had maximal responses other aspects of visual object perception (45). Other category-
averaged across all runs. Thus, there was no overlap selective agnosias also involve loss of nonvisual knowl-
between these regions. edge about the affected category and are associated with
23. The image data were not smoothed before analysis; lesions in cortices other than those of the ventral temporal
nonetheless, it is possible that voxels outside of the re- lobe (37). The literature on lesions that cause prosopag-
gions showing maximal responses to a given category nosia is uninformative about what part of ventral temporal
could still be influenced by the maximally responsive cortex is critical for face recognition. The lesions that
region because of spatial smoothness due to imaging tech- cause prosopagnosia tend to be large (37). It has never
niques and partial volume effects. To address this issue, been demonstrated that the critical part of lesions that
we also analyzed our data after excluding all voxels that cause prosopagnosia involves damage to the small region
responded maximally to the two categories being com- that responds maximally to faces. For a lesion to cause
pared as well as all voxels that were adjacent to these re- prosopagnosia, it may require damage in regions that
gions. On average, this analysis excluded 58% of voxels show the most modulation of response to different indi-
from the calculation of correlations. Nonetheless, overall vidual faces, which may not be coextensive with the re-
accuracy for identifying the category being viewed was gion that responds maximally to faces, or damaged con-
92%, demonstrating that the results are not attributable to nections to cortices in other parts of the brain that are
the effect of maximally responsive regions on adjacent critical for face recognition, such as the superior temporal
voxels. sulcus or the anterior temporal cortex.
24. Supplemental data are available on Science online at 39. Models of the functional architecture of ventral extrastri-
www.sciencemag.org/cgi/content/full/293/5539/2425/DC1. ate cortex that focus analysis on mean responses in
25. Within the region that responds maximally to faces, sites regions that are putatively specialized for stimulus
may exist that respond exclusively to faces that are category (5, 7) or perceptual process (9–11) are not in-
interdigitated with sites that respond maximally, but not consistent with a coexisting functional architecture that
exclusively, to faces, as suggested by studies of evoked embodies the distinct representations for all categories.
potentials recorded with electrodes on the cortical surface Unlike the object form topography model proposed here,
(44). It is important to note, however, that most face- however, these specialized region models do not provide
selective recording sites in these studies do show some an explicit account for how such a coexisting functional
response to other objects and even the sites that demon- architecture is organized or how the representations for an
strate an N200 response exclusively to faces appear to unlimited variety of categories could differ from each
respond to other objects also but with different latencies. other within this architecture.
26. E. Hering, Outlines of a Theory of the Light Sense 40. J. V. Haxby, J. M. Maisog, S. M. Courtney, in Mapping
(Harvard Univ. Press, Cambridge, MA, 1964). and Modeling the Human Brain, P. Fox, J. Lancaster,
27. L. M. Hurvich, D. Jameson, Psychol. Rev. 64, 384 (1957). K. Friston, Eds. (Wiley, New York, in press).
28. R. L. De Valois, Cold Spring Harbor Symp. Quant. Biol. 41. J. Talairach, P. Tournoux, Co-Planar Stereotoxic Atlas of
30, 567 (1965). the Human Brain (Thieme, New York, 1988).
29. I. Biederman, Psychol. Bull. 94, 115 (1987). 42. J. V. Haxby, E. A. Hoffman, M. I. Gobbini, Trends Cognit.
30. A. J. O’Toole, H. Abdi, K. A. Deffenbacher, D. Valentin, Sci. 4, 223 (2000).
J. Opt. Soc. Am. A 10, 405 (1993). 43. A. Martin, L. G. Ungerleider, J. V. Haxby, in The New
31. P. J. B. Hancock, A. M. Burton, V. Bruce, Mem. Cognit. Cognitive Neurosciences, M. S. Gazzaniga, Ed. (MIT
24, 26 (1996). Press, Cambridge, MA, 1999), pp. 1023–1036.
32. A. Lanitis, C. J. Taylor, T. F. Cootes, IEEE Trans. Pattern 44. T. Allison, A. Puce, D. D. Spencer, G. McCarthy, Cereb.
Anal. Mach. Int. 19, 743 (1997). Cortex 9, 415 (1999).
33. V. Blanz, A. J. O’Toole, T. Vetter, H. A. Wild, Perception 45. I. Gauthier et al., J. Cogn. Neurosci. 12, 495 (2000).
29, 885 (2000). 46. We would like to thank R. Desimone, A. Martin,
34. D. A. Leopold, A. J. O’Toole, T. Vetter, V. Blanz, Nature L. Pessoa, G. Ronca, and L. Ungerleider for discussion
Neurosci. 4, 89 (2001). and comments on earlier versions of this reading.
RT0996_P05.qxd 11/8/04 2:14 PM Page 97
PA R T 5
Dissociable Systems for the Perception

of Biological Movement
•
The readings in the previous section, and again in this section, focus on
early perceptual and attentional stages in social information processing.
One might reasonably think that the influences of the social context would
be reserved for later information processing stages—such as cultural
prescriptions for what emotions are appropriate to display in different
social contexts. However, there is compelling behavioral evidence that
social factors (e.g., identifying and feeling cohesive with a group) affects
perceptual processes. When a person views a circle surrounded by larger
circles, it appears smaller than the identical circle surrounded by even larger
circles. This visual illusion is called the Ebbinghaus effect. If the circles are
replaced by faces, the illusion remains: when a person views a face
surrounded by larger faces, it appears smaller than the identical face
surrounded by even larger faces. According to social identity theory, members
of highly entitative (e.g., cohesive) groups should represent natural
comparison targets for each other, such that perceivers automatically and
nonconsciously engage in greater intragroup comparison for members of
highly entitative groups than members of nonentitative (e.g., noncohesive)
groups. To test this hypothesis, sets of Ebbinghaus configurations were
developed with faces used as the objects in the configurations. When these
faces were given the label of a highly entitative group (sorority or fraternity
members), the magnitude of the illusion was greater (indicating greater
97
RT0996_P05.qxd 11/8/04 2:14 PM Page 98
implicit social comparison) than when the faces that which is involved in the perception of
were given the label of a nonentitative group nonbiological motion.
(people born in the month of May). This work Puce and Perrett (2003) focus more specifically
illustrates top-down influences of a social factor on on social perception. They discuss a variety of
perceptual processing. evidence from recordings of single-cell activity in the
Our perceptual world is not constituted solely by nonhuman primate brain to positron emission
stationary objects but by a continuous flow of tomography (PET) and functional magnetic
movement and transitions. A fundamental perceptual resonance imaging (fMRI) evidence in humans.
quality of other animals (including humans) in our Their research also points to the involvement of the
habitat is the distinctive fashion in which these superior temporal sulcus in the processing of
animals move. People have no difficulty moving faces and bodies. Interestingly, they report
distinguishing between the movement of an that movements of the mouth indicative of speaking
automobile and a pedestrian, and there appear to be and movements of the eyes indicative of changes in
a greater similarity in the movement of a walking gaze—both of which carry information about social
person and a walking dog than there is between a attention and communication—produce especially
walking person and a person in a moving strong brain activation in the superior temporal
automobile. In this section, we consider whether sulcus. Like Grossman and Blake, Puce and Perrett
there are perceptual mechanisms specialized for conclude that the superior temporal sulcus is part of
detecting biological movement. a more widely distributed system of neural
In the first of these readings, Grossman and Blake processes that underlie social perception and
(2002) examine evidence that the visual perception cognition.
of form and motion involve separable neural Parsimony refers to a preference for the simplest
mechanisms. The readings thus far have pointed to assumption in the formulation of theory or in the
the temporal lobe as being an important region for interpretation of data. It is simple to think of social
social perception. Grossman and Blake’s research on perception and cognition as unfolding along a single
the perception of biological motion again implicates serial set of information processing stages, with
this region, and the posterior part of the superior each stage forming an isomorphic (one-to-one)
temporal sulcus in particular. Importantly, they link association with a familiar psychological construct.
activity in this region with activity in brain regions Parsimony, however, does not justify ignoring
found in the readings in the preceding section to be empirical anomalies that are inconsistent with
involved in face perception: the occipital and simplistic formulations. The readings are challenging
fusiform face areas. We again see that even very the simplicity of isomorphic thinking of this form.
early features of social perception (in this case, Evidence is accruing that even very early aspects of
biological motion) are subserved by a network of social information processing involve more
distributed neural processes distinguishable from distributed and overlapping information processing
RT0996_P05.qxd 11/8/04 2:14 PM Page 99
Part 5: Dissociable Systems for the Perception of Biological Movement ■ 99
operations in the brain than suggested by simple, structures in social psychology grow in scope by
linear stage models of social perception and drawing evidence from and providing new views of
cognition, and by studies that focus exclusively on related domains, these theoretical structures acquire
single loci based on where the greatest neural the property of consilience—the explanation of
response is observed. Such challenges to simplistic phenomena from different levels of organization that
theorizing are a sign of health. As the theoretical are connected and proven consistent with one another.
RT0996_P05.qxd 11/8/04 2:14 PM Page 100
RT0996_C08.qxd 11/8/04 2:04 PM Page 101
R E A D I N G 8
Brain Areas Active during Visual Perception

of Biological Motion
•
Emily D. Grossman and Randolph Blake*
Theories of vision posit that form and motion are represented by neural mechanisms segregated
into functionally and anatomically distinct pathways. Using point-light animations of biological
motion, we examine the extent to which form and motion pathways are mutually involved in
perceiving figures depicted by the spatio-temporal integration of local motion components.
Previous work discloses that viewing biological motion selectively activates a region on the
posterior superior temporal sulcus (STSp). Here we report that the occipital and fusiform face
areas (OFA and FFA) also contain neural signals capable of differentiating biological from
nonbiological motion. EBA and LOC, although involved in perception of human form, do not
contain neural signals selective for biological motion. Our results suggest that a network of
distributed neural areas in the form and motion pathways underlie the perception of biological
motion.
Introduction versions emphasizing distinctions between “sus-

tained” and “transient” aspects of vision (Breit-
It is widely believed that primate vision comprises meyer and Ganz, 1976; Kulikowski and Tolhurst,
multiple visual areas organized into hierarchical 1973), others distinguishing “color” and “broad-
pathways specialized for registering information band” channels (Schiller et al., 1990), and still oth-
about particular aspects of the visual scene (Felleman ers focusing on distinctions between perceiving
and Van Essen, 1991). Over the years, this overar- objects and acting upon objects (Goodale and
ching model has taken different forms, with some Humphrey, 1998). One currently popular version
*
Both of Vanderbilt Vision Research Center/Department of
Psychology, Vanderbilt University, Nashville, Tennessee 37203.
101
RT0996_C08.qxd 11/8/04 2:04 PM Page 102
of this theory posits a so-called “motion” pathway the perception of a human form engaged in a read-
extending into more dorsal aspects of extrastriate ily identified activity (Cutting et al., 1978;
and posterior parietal cortex, specialized for regis- Ahlström et al., 1997; Neri et al., 1998; Mather
tering information about the locations of objects et al., 1992). The compelling sense of human form
and their movements within the visual scene, and created by the spatio-temporal integration of these
an “object” stream pathway in ventral cortex in- local dot motions would seem to imply that “ob-
volved in specifying information about the shapes ject” and “motion” pathways are together creating
and identities of visual objects (Ungerleider and perception of an active person.
Mishkin, 1982; Livingstone and Hubel, 1987). Brain imaging studies in humans have pin-
This particular version of the multiple pathway pointed a region on the posterior superior tempo-
model has sparked a wealth of research aimed at ral sulcus (STSp) that is active when observers
testing the notion of object-grounded and motion- perceive biological motion in point-light anima-
grounded neural systems (Haxby et al., 1991; tions (Bonda et al., 1996; Howard et al., 1996;
Tanaka, 1996; Bradley et al., 1998; Kourtzi and Grossman et al., 2000; Vaina et al., 2001). It seems
Kanwisher, 2000). reasonable to place STSp within the motion path-
While not disputing the notion of object-based way, based on its proximity to motion-responsive
and motion-based processing streams, several re- areas MT and MST (Suneart et al., 1999); more-
cent neural imaging studies have sought to deter- over, STSp is far removed from ventral temporal
mine the extent to which neural representations of cortex and, by implication, brain areas involved in
objects are distributed throughout visual cortex. form perception. Nonetheless, point-light anima-
Sereno et al. (2002) found evidence for cue- tions portraying biological motion create com-
invariant, 3D shape representations in multiple pelling impressions of a class of recognizable
brain areas spanning object and motion pathways objects, namely humans, so it is natural to sup-
in anesthetized monkeys. Haxby et al. (2001) pose that this unique type of structure from mo-
found that the distribution of brain activity associ- tion also activates “object-selective” ventral stream
ated with viewing faces and objects was wide- mechanisms. Thus while STSp may be selectively
spread within ventral temporal cortex, leading activated when viewing biological motion percep-
these authors to downplay the importance of tion, the entire network of brain areas involved in
highly specialized neural areas in object recogni- registering all aspects of these salient animations
tion. We, too, have recently become interested in may extend to the form pathway.
the question of distributed neural representations, We have examined this supposition by isolating
in our case, representations associated with a par- brain areas generally believed to be involved in the
ticularly salient class of motion-defined forms, perception of objects, including human body parts,
i.e., biological motion. In this paper, we report re- and then measuring BOLD signals in those areas
sults from a brain imaging study that examines produced by viewing biological motion se-
patterns of neural activity within multiple brain quences. These areas are: (1) the lateral occipital
areas implicated in visual perception of bodies complex (LOC), which has been implicated in the
and body parts. perception of forms regardless of the visual cues
In these experiments, we have capitalized on a used to define those forms (Grill-Spector et al.,
vivid, remarkable example of motion-defined 1999); (2) the extrastriate body area (EBA), which
shape: Johansson’s “point-light” animation se- has recently been implicated in the perception of
quences (Johansson, 1973). These animations images of bodies and body parts, but not faces
convey complex human activities using just a (Downing et al., 2001); and (3) the occipital (OFA)
handful of dots placed on the joints of the human and fusiform face areas (FFA), both of which have
body. Single static frames resemble a meaningless been implicated in the perception of faces
cluster of dots portraying no hint of an object, hu- (Kanwisher et al., 1997), as well as in the percep-
man or otherwise, but when shown in rapid suc- tion of other highly familiar objects (Gauthier
cession, these animated dots are grouped to create et al., 1999). We used widely accepted stimuli and
RT0996_C08.qxd 11/8/04 2:04 PM Page 103
Brain Areas Active during Visual Perception of Biological Motion ■ 103
subtraction conditions to isolate these object-re- and the inferior temporal sulcus (ITS). This brain
sponsive brain areas, including pictures of head- area responds more strongly to point-light anima-
less bodies to activate the EBA, pictures of objects tions portraying biological motion than it does to
to activate the LOC, and pictures of faces to acti- “scrambled” animations, created with the same
vate the FFA and OFA (Figure 8.1). motion vectors but whose starting positions are
We also measured neural activity in STSp, the spatially randomized (Grossman and Blake,
brain area implicated in perception of biological 2001). STSp does not respond well to coherent,
motion. In localizing STSp, observers viewed translational motion, nor does it respond to the
point-light actors performing a variety of human presence of kinetic boundaries (Grossman et al.,
activities. However, these animations, composed 2000). In other words, this region is functionally
of only 12 dots, are visually sparse in comparison and anatomically distinct from other motion-
to the pictures of headless bodies, faces, and ob- responsive areas in the human brain, including
jects. To evaluate the consequence of this sparse- human MT (Watson et al., 1993; Tootell et al.,
ness on BOLD signal levels, we included anima- 1995; Orban et al., 1995) and the kinetic occipital
tions in which the entire figure of the actor was region (KO, or LO, as it is sometimes called; Van
visible. Also, to determine if neural responses Oostende et al., 1997; Malach et al., 1995).
were due to the presence of an object, or specifi- In the present study, observers viewed blocks
cally because the object is biological, we included of 1 s animations of point-light biological motion
animations of a nonbiological object (a rotating, interleaved with blocks of 1 s animations of
3D globe) defined solely by dot motions. scrambled motion. In nine of the ten observers, a
bilateral region on the posterior end of the STS, at
the junction with the ITS, was more active during
Results the biological epochs than during the scrambled
epochs (p .001). In the tenth observer, this re-
Our strategy entailed several steps: (1) use stan- gion was only found in the left hemisphere
dard subtraction conditions to localize STSp, (Table 8.1). Based on the anatomical location and
EBA, LOC, OFA, and FFA (Figure 8.2a), (2) as- the functional response during the localizer con-
sess patterns of activation across all of these areas dition, these regions became the STSp region of
by comparing evoked BOLD signals to a common interest (ROI) for each observer.
baseline of fixation, and (3) determine whether Within these ROIs, we compared BOLD sig-
neural signals within those regions are capable of nals during the four stimulus conditions depicting
discriminating between biological and scrambled biological events (Figure 8.2b). Two of these
point-light animations. To control for attention, conditions—whole body and point-light biologi-
which is known to modulate BOLD signals in cal—were animated sequences depicting human
many of the neural areas included in this study activities, while the other two conditions—head-
(Corbetta et al., 1991; Wojciulik et al., 1998), ob- less bodies and bodiless heads—were stationary
servers performed a 1-back task on the individual images. BOLD activity levels were highest during
stimuli within each block. the epochs of whole bodies, point-light bodies,
Results from our measurements are discussed and faces, and lowest during the epochs of head-
brain region by brain region in the following sec- less bodies. Pairwise comparisons revealed that
tions and are shown in Figures 8.2b–8.2e. Using only the difference between whole bodies and
some of the same data from Figure 8.2, Figures headless bodies was significant (p .001). It is
8.3 and 8.4 highlight contrasts important for eval- noteworthy that bodies defined by only twelve
uating differences in selectivity among the areas. points of light were as effective as whole body an-
imations in activating STSp. This is the only brain
The Posterior Superior Temporal Sulcus area in which these two dynamic biological mo-
STSp is located at the posterior end of the supe- tion animations resulted in equivalent neural re-
rior temporal sulcus, near the junction of the STS sponses. This is testimony to the vividness of
RT0996_C08.qxd 11/8/04 2:04 PM Page 104
a) Animated Stimuli
Time
3D Rotating
Point-Light Scrambled Whole
Globe
Biological Motion Bodies
b) Stationary Stimuli
Headless Bodies
Faces
Objects
FIGURE 8.1 ■ Schematics of the stimuli. Dynamic human activity was portrayed
by the motion twelve dots located on the joints and head of an actor performing
various activities. Occlusions of some dots naturally occur as limbs pass be-
hind the body. These occlusions help convey normal, three-dimensional body
movement, and so were retained in the animations (i.e., not all dots were visi-
ble at all times). The scrambled animations contained the same motion vectors
as the biological ones, but the initial starting positions of the dots are random-
ized within a region approximating the size of the body. The whole-body ani-
mations depicted an actor performing the same activities as the point-light an-
imations. The rotating, structured globe was created by moving 200 dots
sinusoidally within a circular aperture. The variable speed of the dots conveyed
three-dimensional structure, but the direction of the rotation was ambiguous.
The wire frame pictured above to denote three-dimensional structure was not
visible in the experiment. The stationary, headless body condition consisted of
images of bodies standing or sitting, with the heads erased. Observers also
viewed images of faces and common household objects.
RT0996_C08.qxd 11/8/04 2:04 PM Page 105
a) b)
c)
d)
e)
FIGURE 8.2 ■ (A color version of this figure follows page 146.) Summary of regions of interest (ROIs) and BOLD re-
sponses in biological related stimulus conditions. (a) The ROIs in the right hemisphere of observer D.L. are displayed on
the lateral and ventral surfaces of the gray matter. A cut, as indicated by the green plane, was made and the posterior
end of the cortex flattened. We examined BOLD signals in four regions of interest: STSp (red), EBA (purple), LOCd
(blue), FFA and OFA (orange). (b–e) The average BOLD activity levels for these ROIs (with FFA and OFA averaged)
during the stimulus conditions depicting some kind of biological object, or the scrambled biological motion vectors.
These stimulus conditions included animations of point-light biological motion (pink), point-light scrambled motion (yel-
low), whole-bodies (dark purple), pictures of faces (magenta), and stationary images of headless bodies (green). The
percent change activation levels are relative to a fixation baseline. Error bars indicate 1 standard error.
perception produced from these simple, sparsely globe implies that STSp is not simply registering
sampled animations and to the importance of structure from motion but, instead, is specialized
STSp in the perception of dynamically defined for the kinematics portraying biological motion.
complex activity. At the same time, motion is not absolutely crucial
In contrast to BOLD signals evoked by biolog- for activating STSp, for pictures of faces also pro-
ical motion, the rigidly rotating globe defined by duced reliable responses from this area. Face-
moving dots evoked trivially small BOLD signals responsive regions on the STS have been previ-
that were no different from those found during the ously reported in the literature (Chao et al., 1999;
scrambled motion epochs (Figure 8.3a). The Puce et al., 1998; Hoffman and Haxby, 2000), and
weak, nonspecific neural response to the kinetic it is possible that the face-responsive STS region
RT0996_C08.qxd 11/8/04 2:04 PM Page 106
a) STSp b) LOCd Point-Light

1 1 Scrambled
Globe
% Change from Fixation

0.8
* 0.8

0.6 0.6
0.4
0.2 0.2
0 0
Stimulus Condition Stimulus Condition
FIGURE 8.3 ■ Average percent change in the STSp and LOCd ROIs for the animated
dot displays. Point-light biological, scrambled, and the rotating globe were the visually
sparsest of all the stimuli used in these experiments. The biological and object identi-
ties were constructed by the movements of dots only, and could not be ascertained
from a single, stationary frame. Even when animated, the scrambled animation looks
like an incoherent cloud of dots. Asterisk indicates a significant difference for the bio-
logical motion condition over both the scrambled and rotating globe. All other con-
trasts were nonsignificant.
(STS-FA) and STSp overlap, resulting in strong was most active during stimulus conditions in
BOLD responses to the images of faces. Using the which pictures of the human body were shown
FFA localizer (faces-objects), we attempted to de- (p .05), which included the epochs of stationary
termine the extent of overlap between STSp and headless bodies and animations of whole bodies
STS-FA. However, we were able to localize STS- (including the heads). We found that faces alone
FA in only one observer, and there was no overlap were slightly less effective (though not signifi-
between the two ROIs. Further testing is needed cantly different) in activating this region. Body
to determine conclusively the relationship be- shapes depicted by the point-light sequences were
tween the face- and body-responsive regions on also slightly less effective than the explicit body
the STS. images in activating the EBA.
Both STSp and EBA have been implicated in the
The Extrastriate Body Area perception of bodies, albeit using very different vi-
Following the lead of Downing et al. (2001), we lo- sual stimuli. How are these two areas different? To
calized the EBA by subtracting activation evoked answer this question, it is interesting to compare
by pictures of stationary headless bodies from ac- the responses of EBA and STSp during the two lo-
tivity evoked by pictures of stationary household calizers used to identify these regions (Figure 8.4).
objects. Using this subtraction, we were able to lo- BOLD signal levels in STSp during point-light bi-
calize a region in the anterior extent of occipital ological animations were significantly higher than
cortex, dorsal to the inferior occipital sulcus. This during the scrambled intervals (p .01), but this
region was bilateral in five of the six observers was not true for EBA (p .41). Conversely, during
tested, and only in the left hemisphere of the sixth the EBA localizer, the images of headless bodies
(Talairach coordinates: left hemisphere: 39.3, produced significantly more activation in EBA than
70.1, 13.5; right hemisphere: 40.6, 65.7, 10.6). did images of objects (p .01), but there was no
Results for EBA are summarized in Figure significant difference between the two in STSp
8.2c. consistent with previous reports, this region (p .53). It is unlikely that the poor discrimination
RT0996_C08.qxd 11/8/04 2:04 PM Page 107
a) STSp Localizer images of faces. Instead, we believe it is the ab-

1 Point-Light
sence of dynamic signals, not the absence of a face,
Scrambled
0.8
*
that is responsible for STSp’s weak response to im-
0.6
ages of headless bodies.
0.4
The Lateral Occipital Complex (Dorsal)
0.2
The lateral occipital complex (LOC) is a large
0 area of cortex starting most dorsally at the poste-
Posterior STS EBA
rior end of the lateral occipital sulcus and extend-
ROI
ing through the ventral temporal cortex. We local-
b) EBA Localizer Headless Bodies
ized this region using the technique of Kourtzi
1.2
Objects and Kanwisher (2000), in which neural activity
1 * during periods of viewing texture patterns is sub-

0.8
tracted from activity associated with viewing
0.6
faces and objects. In our study, this subtraction re-
0.4 vealed a large region of activation, the most pos-
0.2 terior and dorsal extent of which we will refer to
0 as LOCd (Talairach coordinates: left hemisphere:
Posterior STS EBA
28.4, 77.8, 2.5; right hemisphere: 33.9,
FIGURE 8.4 ■ Average percent BOLD signal change dur- 74.7, 1.0). The more anterior extent of the ac-
ing the STSp and EBA localizer scans. (a) Subtracting
BOLD signal levels during the point-light scrambled (light
tivated region lies on the ventral surface of the
bars) from point-light biological (dark bars) motion intervals temporal lobe and encompasses other object- and
results in a significant contrast in STSp, but not in EBA. (b) face-responsive brain areas on the posterior
However, subtracting activity levels elicited by pictures of fusiform gyrus, including the occipital face area
household objects (light bars) from that elicited by pictures (OFA). Based on previous work showing clear
of stationary headless bodies (dark bars) results in higher
percent signal change and more contrast in EBA.
functional differences between anterior and poste-
rior regions of LOC (Grill-Spector et al., 1999;
Bar et al. 2001), we felt it important to analyze the
between headless bodies and objects in STSp is BOLD signals separately for the LOCd (dis-
due to the absence of a face in the headless bodies cussed in this section) and the OFA (discussed in
images. Quite the opposite, this region responded the following section).
equally well to animated displays with no facial in- BOLD signal levels in the LOCd were signifi-
formation (i.e., the point-light sequences) and to cantly higher when observers viewed faces and
TABLE 8.1. Frequency of Activated Regions during the Point-Light Biological Motion Localizer
Talaraich Coordinates
Num Observed Hemispheres Left Right

ROI Left Right X Y Z X Y Z
STSp 10/10 9/10 41.3 52.8 11.8 46.1 48.5 12.4

ITS 6/10 5/10 43.1 58.5 11.2 48.8 53.3 10.8
STSa 2/10 6/10 46.9 41.9 6.6 50.0 33.0 4.3
FFA 3/10 4/10 33.3 40.5 13.9 37.1 36.0 12.2
A total of ten observers viewed the biological and scrambled point-light animations. Simulations of false positive rates were used to
determine the threshold for the activation maps. Table indicates the number of observers and Talairach coordinates for each
hemisphere of the activated clusters. Abbreviations: STSp superior temporal sulcus, the posterior end, ITS inferior temporal
sulcus, specifically the small region of the ITS between the STS and MT, STSa anterior region on the superior temporal sulcus,
FFA fusiform face area.
RT0996_C08.qxd 11/8/04 2:04 PM Page 108
objects than when they viewed images of texture Of all the stimulus conditions tested, the im-
patterns created from scrambling the face and ob- ages of faces alone produced the highest re-
ject pictures (p .001). BOLD signals in LOCd sponses in the fusiform areas (Figure 8.2e). This
were highest during the conditions showing “face-only” activation was significantly higher
whole-body animations and face images (no sig- than that produced when observers viewed pic-
nificant difference between the two conditions, tures of objects (p .001), headless bodies
p .29), almost twice that for conditions when (p .01), or whole bodies with the faces intact
observers viewed stationary pictures of headless (p .05).
bodies or the point-light animations depicting bi- Although faces also activated the fusiform face
ological motion (Figure 8.2d; p .0001). There areas significantly more than the animations of
were no significant differences between BOLD point-light biological motion (p .001), the con-
signal levels while observers viewed point-light trast between point-light biological and point-
biological motion, scrambled motion, or the light scrambled was also significant (p .001).
structured rotating globe animation (Figure 8.3b). Critically, although activation levels are overall
lower, the biological organization of the point-
The Occipital and Fusiform Face Areas light animations is sufficient to activate this re-
Situated more anterior than LOC on the ventral gion, as evidenced by the contrast with scrambled
surface of the temporal lobe is a cluster of foci re- animations. Incidentally, our failure to identify re-
lated to object and face recognition. The most liable FFA activation in an earlier study (Grossman
widely studied of these regions is the fusiform et al., 2000) may be attributable, at least in part, to
face area (FFA), located on the ventral medial as- nonoptimal slice placement for capturing the ven-
pect of the temporal lobe abutting the cerebellum. tral temporal cortex in all observers. It is also
The FFA is functionally identified by its greater noteworthy that in the present study, unlike the
response to images of faces than to common earlier one, FFA was localized using the standard
household objects (Kanwisher et al., 1997). There stimulus contrast (faces minus objects).
is also another, more posterior region on the
fusiform gyrus that responds strongly to faces and
objects, and may lie within or just anterior to the Discussion
LOC. This occipital face area (OFA), like the
FFA, is often found by contrasting activation People watch other people all the time, trying to
when observers view pictures of faces versus pic- deduce intentions and moods based on dynamic
tures of objects. We were able to localize the FFA visual information. Indeed, recognizing what oth-
in both hemispheres of all five observers tested ers are up to is one of the most important percep-
using this contrast (Talairach coordinates: left tual activities we engage in. It is not surprising,
hemisphere: 34.0, 39.7, 15.3; right hemi- therefore, that people are remarkably adept at per-
sphere: 37.6, 38.9, 14.2); the OFA was local- ceiving intentions and affect even when those per-
ized in both hemispheres of three of these five ob- sonal characteristics are portrayed in point-light
servers, and only the left hemisphere of a fourth animations devoid of static form cues. Befitting
(Talairach coordinates: left hemisphere: 35.0, such a crucial perceptual skill, neural representa-
55.0, 11.7, right hemisphere: 37.5, 54.2, tions of biological activity are widely distributed
9.9). Across the stimulus conditions tested, we throughout visual cortex. Based on converging
found no differences in the patterns of activity lines of evidence, Allison et al. (2000) propose
levels between the FFA and OFA, with the excep- that a major component in this distributed repre-
tion of overall slightly higher percent signals sentation is a large expanse of cortex spread
changes in the OFA. The results from these two across the STS.
regions, therefore, were averaged and are pre- The results from our study, while confirming the
sented together. importance of the STS in perception of biological
RT0996_C08.qxd 11/8/04 2:04 PM Page 109
activity, reveal that neural areas in the ventral 2002). Of course, it is possible that FFA itself
stream are also activated when one views point- contains neural machinery for registering the mo-
light biological motion animations. The fusiform tion signals necessary for constructing the body
and occipital face areas in ventral temporal cortex representation in point-light animations. From
contain neural signals capable of discriminating our results, we cannot distinguish between these
between point-light animations that organize into two hypotheses, and to do so may require using
biological motion and those that organize into analytic techniques that reveal the strength of
scrambled motion. Our results also demonstrate functional connectivity among areas (Friston et
that other neural areas previously found to be se- al., 1995).
lectively involved in the perception of bodies and In contrast to FFA and OFA, LOC has been
objects, specifically LOC and EBA, can not dis- characterized as a general object recognition area,
criminate between biological and scrambled responding invariantly to images that can be inter-
point-light animations. To determine the discrim- preted as shapes regardless of the visual cues cre-
inability of the neural signals within these five re- ating the shapes’ contours (Grill-Spector et al.,
gions—STSp, FFA, OFA, EBA, and LOC—we 2001). Our results, however, place some limits on
probed each with a variety of biological and non- the generality of that characterization. In particu-
biological visual stimuli. What conclusions can we lar, point-light animations, although readily or-
draw from the patterns of activations in these dis- ganized into a visible human form, produced no
tributed areas? To answer this question, we need to greater activation in LOCd than did scrambled an-
reconsider the response selectivity of these neural imations which resembled a disorganized cloud of
regions. dots. Similarly, the rotating globe, also easily or-
In the ventral stream, point-light bodies cer- ganized into a structured shape, did not increase
tainly produce weaker responses than those brain activity beyond levels found during viewing
evoked by faces or, for that matter, by animated of scrambled motion. Evidently, the cue invari-
whole bodies in FFA and OFA. However, when ance of LOC does not extend to all forms defined
observers view point-light bodies, the fusiform by motion.
activations are significantly stronger than those Finally, biological and scrambled motion pro-
resulting from the same point-lights scrambled to duced equivalent BOLD responses in EBA, im-
destroy the impression of a human body. In prin- plying that this brain region does not carry signals
ciple, then, neural activity in the fusiform region capable of registering the presence of human
could form part of a distributed representation of forms as depicted in point-light animations. This
human bodies, including bodies defined by mo- is not to say that EBA does not register the pres-
tion. But how does this “object” pathway re- ence of a human form—indeed, this area is func-
gion—the FFA—acquire its selectivity for motion- tionally defined by its stronger response to images
defined bodies? After all, the human form in of headless bodies than to nonhuman objects
point-light animation sequences is portrayed ex- (Downing et al., 2001). Nor do our results imply
clusively by motion signals associated with hier- that EBA is not involved in viewing bodies in mo-
archical, pendular motions of the limbs: perception, as evidenced by the equivalent BOLD re-
tion of a human form emerges only from the sponses to whole bodies in motion and to images
integration of motion signals over space and time. of stationary, headless bodies. Instead, our results
To the extent that the motion analyses underlying indicate that the BOLD signals in EBA are con-
point-light animations are performed exclusively tingent upon the shape of the body being explic-
by neural mechanisms within the motion path- itly represented in the image.
way, our results would imply that the outputs of It is interesting to note that STSp behaves in a
those mechanisms project to the FFA; this idea, fashion complementary to EBA. Unlike in EBA,
incidentally, is one component of a recently pub- the BOLD responses in STSp are selectively
lished model of face recognition (O’Toole et al., driven by the dynamics of the human form. This is
RT0996_C08.qxd 11/8/04 2:04 PM Page 110
evidenced by (1) the equivalent BOLD responses 1997). Point-light biological motion sequences
produced by stationary images of bodies and by were created by videotaping an actor performing
nonhuman objects, and (2) the stronger BOLD re- various activities, including running, jumping,
sponses produced by dynamic human bodies than throwing, and kicking. The segments were digi-
by a moving, nonhuman object. Together, these tized, and the joint positions in each frame were
results imply that STSp is specialized for a partic- encoded as motion vectors with initial starting po-
ular class of dynamic events, namely moving hu- sitions. Scrambled biological motion animations
man bodies. Also, unlike in EBA, the BOLD re- were created by randomizing the starting posi-
sponses in STSp do not depend on the explicit tions of each joint within a region approximating
representation of the human body, for STSp re- that covered by the biological sequences. The mo-
sponded just as strongly to point-light animation tion vectors were left intact so that the joints
sequences as to animations showing whole bodies moved naturally as they would in the biological
in motion. animations. This manipulation ensured the indi-
To end on a speculative note, it is reasonable to vidual motion components were identical in both
wonder why the human brain would contain mul- the biological and scrambled animations; only the
tiple areas (e.g., EBA and STSp) dedicated to the hierarchical relations among the dots in the
perception of human bodies. Perhaps identifica- scrambled displays were destroyed. For both
tion of an individual constitutes a different per- kinds of animations, the joints were displayed as
ceptual task than perceiving an individual’s mood small, black dots subtending approximately 9 arc
or intentions. Intentions can be judged based on min of visual angle against a gray background.
gestures, actions, and expressions independent of Each biological activity sequence consisted of 20
identity. Accurate identification, on the other frames displayed in a 1 s interval (33 ms inter-
hand, must generalize across gesture, mood, and frame interval, 60 Hz). At this display rate, the bi-
activity. Given these divergent demands, extrac- ological animations generated smooth apparent
tion of the visual information subserving identifi- motion, depicting natural body movement.
cation and perception of intention may require Whole body animations depicted the same ac-
different neural operations, perhaps most effi- tivities as the point-light sequences, with the en-
ciently embodied in separate neural architectures tire body of the actor being visible. These anima-
(EBA and STSp, respectively). tions sequences were recorded with a digital
video camera, edited into 1 s clips that were then
converted to Quicktime files displayed with
Experimental Procedures Matlab QT routines.
Random dot cinematograms depicting a struc-
Participants
tured “globe” and coherent planar motion were
Ten individuals (6 men, 4 women) with normal or created with 200 dots horizontally displaced
corrected to normal vision participated in this within a circular aperture. Approximately half the
study. All observers had experience viewing dots moved leftwards within the aperture while
point-light animations and easily recognized all the other half moved rightwards. To create the
the biological motion sequences as human activi- structured globe, the speed of the dots was sinu-
ties. The observers gave informed consent as ap- soidally modulated such that the dots moved
proved by Vanderbilt University Institutional Re- fastest on the outer edges of the aperture and
view Board. slowest in the center. These animations create the
vivid impression of a 3D transparent globe rotat-
Stimuli ing about its vertical axis (Doner et al., 1984).
Visual stimuli were displayed using Matlab Two coherent planes of transparent motion were
(Mathworks, Inc.) together with routines from the created by moving half the dots leftward and half
Psychophysics Toolbox (Brainard, 1997; Pelli, rightward, with the constant speed throughout the
RT0996_C08.qxd 11/8/04 2:04 PM Page 111
animation. The average speed of the dots was 3.75 3.75 mm inplane, 5 mm, no gap), and T1
4.8º/s (48 ms interframe interval, 60Hz), and the high resolution images of the slice positions.
aperture subtended 7.2º 7.2º of visual angle. Slices positions were chosen to cover the entire
Some observers also viewed grayscale images occipital pole, the ventral surface of the temporal
of faces, headless bodies, and common household lobe, and the posterior extent of the superior tem-
objects. These images were static, not animated, poral sulcus. Because the high resolution slice
and they subtended 7.5º 7.5º of visual angle. anatomy images were collected in alignment with
Texture patterns were created by breaking the im- the T2 functional images, the high and low reso-
ages into small pieces that were then spatially lution images were naturally co-registered with
shuffled to yield scrambled images lacking coher- each other. The slice anatomy images were manu-
ent spatial structure. ally aligned with the T1 whole-brain images, sub-
All individuals participated in the STSp local- sequently allowing us to register the functional
izer experiment in which point-light biological images into the whole-brain coordinate space. Us-
and scrambled motion animations were presented ing this alignment method, we were able to have
alternately in a block design. The resulting images some individuals return to the scanner on different
from this scan were used to determine biological days, align the data into common space, and treat
motion responsive ROI. All individuals also the images as if they had been acquired on the
viewed alternating blocks of biological, scram- same day.
bled, and whole-body motion, and in a separate Functional scans lasted 172.5 s, the initial 6 s (4
scan the structured rotating globe and planar volumes) of which were discarded prior to analy-
transparent motion. For five individuals, epochs sis to allow for MR stabilization. The images and
of point-light biological motion were included animations were blocked into 10.5 s epochs con-
within the structured globe and planar motion sisting of seven stimuli (1 s each, 500 ms inter-
scan to eliminate any effect of neural adaptation stimulus interval). The exemplars within the
to the coherent dots and still maintain the atten- block were chosen randomly on each trial, but
tion of the observers. We found no difference in with a 50% chance of repeating on successive tri-
BOLD signal levels for the structured globe or als. Observers were instructed to monitor the
planar motion in the two condition or three condi- stimuli carefully, and indicate sequential repeti-
tion scans. Six of the ten observers (D.L., D.R., tions with a button press (1-back task), and the
F.R., J.L., K.B., M.R.) participated in additional task was made more taxing by requiring observers
scans designed to localize the EBA (alternating to respond within the short 500 ms interval be-
blocks of stationary headless bodies and objects). tween the animations. This kind of challenging
Five observers (D.L., D.R., F.R., K.B., M.R.) task maintains the observer’s level of attention
were also scanned to localize LOC and the FFA through a block of trials, an important considera-
(alternating blocks of faces, objects, and texture tion in view of evidence that attention can modu-
patterns). late BOLD signal in early and extrastriate visual
areas (Somers et al., 1999; Watanabe et al., 1998).
Imaging During the intervals involving the structured
All brain images were collected on a 3 Tesla GE globe and planar motion, the near and far surfaces
Signa scanner located within Vanderbilt Univer- of the flowfield were inherently ambiguous, re-
sity Medical School. Observers participated in sulting in spontaneous reversals of depth order-
scanning sessions that lasted approximately 1.5 ing. Consequently, a 1-back task would have had
hr. During this time, we acquired high resolution no objective measure of correctness. To promote
T1 anatomical images of the observer’s head (124 attention in these epochs, observers were in-
slices, 1.4 1.4 9375 mm), T2 functional im- structed to monitor the motion direction of the
ages in the axial plane (single-shot EPI, TR dots in the nearest plane of depth. For all condi-
1500 ms, TE 25 ms, flip 90º, 21 axial slices, tions, the order of the blocks was counterbalanced
RT0996_C08.qxd 11/8/04 2:04 PM Page 112
across subjects. Following each stimulus block However, to compare the activity levels during a va-
was a 3 s (2 volume) interval of fixation during riety of visual tasks, the raw MR signal from each
which only a small cross in the center of the scan was converted into percent change of the mean
screen was visible. BOLD signal activation level during the fixation in-
Visual displays were back-projected with a DLP tervals in the scan. Further, we found that because of
projector onto a screen located at the observer’s the short time between volume acquisitions (1.5),
feet. A periscope mirror attached to the birdcage and the relatively short block duration (10.5 s), the
headcoil was adjusted prior to the onset of the MR signal barely reached saturation before the end
scans to maximize viewing angle of the screen. of the epoch, and was more appropriately fit by a si-
nusoidal model (i.e., Boynton et al., 1996) than by an
Analysis on-off boxcar function. Thus in calculating percent
Images were corrected for in- and out-of-plane change values, we calculated the peak-to-peak dif-
motion using AIR 3.08 (Woods et al., 1998). All ferences between the stimulus and fixation intervals.
subsequent analyses were done using Brain Voy-
ager (Brain Innovations, Inc.) and Matlab. The re-
Acknowledgments
aligned images were corrected for linear trend
over time then spatially filtered with a 5 mm This work was supported by a Vanderbilt University
FWHM Gaussian filter. The filtered and unfiltered Discovery Grant, NSF BCS0079579 and NSF
images were averaged together to create “multifil- BCS0121962. We thank Nancy Kanwisher and
tered” images, as described by Skudlarski et al. Paul Downing for their headless bodies and object
(1999). Multifiltering minimizes sites of single stimuli, and we thank Marvin Chun, David Lyon,
voxel false positive activations while maximizing and Duje Tadin for comments on an earlier ver-
signal change that may be lost by spatial smooth- sion of the manuscript.
ing alone.
All observers viewed alternating blocks of
point-light biological and scrambled motion, al- REFERENCES
lowing us to localize STSp as described in our Ahlström, V., Blake, R., and Ahlström, U. (1997). Perception
earlier work (Grossman et al., 2000). Because of of biological motion. Perception 26, 1539–1548.
Allison, T., Puce, A., and McCarthy, G. (2000). Social percep-
spatial and temporal correlations naturally occur-
tion from visual cues: Role of the STS region. Trends Cogn.
ring in the data, the actual r-cutoff value for deter- Sci. 4, 267–278.
mining regions of interest (ROIs) was empirically Bar, M., Tootell, R.B.H., Schacter, D.L., Greve, D.N., Fischl,
derived through repeated simulations of false pos- B., Mendola, J.D., Rosen, B.R., and Dale, A.M. (2001).
itive rates occurring in the voxels within the brain. Cortical mechanisms specific to explicit visual object
recognition. Neuron 29, 529–535.
This was done in the following manner: (1) the
Bonda, E., Petrides, M., Ostry, D., and Evans, A. (1996). Spe-
BOLD signal values within each voxel were ran- cific involvement of human parietal systems and the amyg-
domly shuffled in time, (2) the correlation dala in the perception of biological motion. J. Neurosci. 16,
between the randomized time series and the local- 3737–3744.
izer boxcar was computed for each voxel, (3) the Boynton, G.M., Engel, S.A., Glover, G.H., and Heeger, D.J.
(1996). Linear systems analysis of functional magnetic res-
value corresponding to the upper .01% cutoff of
onance imaging in human V1. J. Neurosci. 16, 4207–4221.
the distribution of r values across the brain (corre- Bradley, D.C., Chang, G.C., and Andersen, R.A. (1998). En-
sponds to a 1% Type I error rate) was selected. coding of three-dimensional structure-from-motion by pri-
These steps were repeated 1000 times, and the fi- mate area MT neurons. Nature 392, 714–717.
nal r value threshold for determining the activa- Brainard, D.H. (1997). The psychophysics toolbox. Spat. Vis.
10, 443–446.
tion maps was taken as the mean value of the dis-
Breitmeyer, B.G., and Ganz, L. (1976). Implications of sus-
tribution of cutoff values. tained and transient channels for theories of visual pattern
In localizing the ROIs, we used previously pub- matching, saccadic suppression, and information process-
lished and widely accepted “subtraction” conditions. ing. Psychol. Rev. 8, 1–36.
RT0996_C08.qxd 11/8/04 2:04 PM Page 113
Chao, L.L., Martin, A., and Haxby, J.V. (1999). Are face-re- Howard, R.J., Brammer, M., Wright, I., Woodruff, P.W., Bull-
sponsive regions selective only for faces? Neuroreport 10, more, E.T., and Zeki, S. (1996). A direct demonstration of
2945–2950. functional specialization within motion-related visual and
Corbetta, M., Miezin, F.M., Dobmeyer, S., Shulman, G.L., and auditory cortex of the human brain. Curr. Biol. 6,
Petersen, S.E. (1991). Selective and divided attention dur- 1015–1019.
ing visual discriminations of shape, color, and speed: func- Johansson, G. (1973). Visual perception of biological motion
tional anatomy by positron emission tomography. J. Neu- and a model for its analysis. Percept. Psychophys. 14,
rosci. 11, 2383–2402. 201–211.
Cutting, J.E., Proffitt, D.R., and Kozlowski, L.T. (1978). A Kanwisher, N., McDermott, J., and Chun, M.M. (1997). The
biomechanical invariant for gait perception. J. Exp. Psy- fusiform face area: a module in human extrastriate visual
chol. Hum. Percept. Perform. 4, 357–372. cortex specialized for face perception. J. Neurosci. 17,
Doner, J., Lappin, J.S., and Perfetto, G. (1984). Detection of 4302–4311.
three-dimensional structure in moving optical patterns. J. Kourtzi, Z., and Kanwisher, N. (2000). Cortical regions in-
Exp. Psychol. Hum. Percept. Perform. 10, 1–11. volved in processing object shape. J. Neurosci. 20,
Downing, P., Jiang, Y., Shuman, M., and Kanwisher, N. 3310–3318.
(2001). A cortical area selective for visual processing of the Kulikowski, J.J., and Tolhurst, D.J. (1973). Psychophysical
human body. Science 293, 2470–2473. evidence for sustained and transient detectors in human vi-
Felleman, D.J., and Van Essen, D.C. (1991). Distributed hier- sion. J. Physiol. 232, 149–162.
archical processing in the primate cerebral cortex. Cereb. Livingstone, M.S., and Hubel, D.H. (1987). Segregation of
Cortex 1, 1–47. form, color, movement, and depth: Anatomy, physiology,
Friston, K.J., Ungerleider, L.G., Jezzard, P., and Turner, R. and perception. Science 240, 740–749.
(1995). Characterizing modulatory interactions between ar- Malach, R., Reppas, J.B., Benson, R.R., Kwong, K.K.,
eas V1 and V2 in human cortex: A new treatment of func- Jiang, H., Kennedy, W.A., Ledden, P.J., Brady, T.J.,
tional MRI data. Hum. Brain Mapp. 2, 211–224. Rosen, B.R., and Tootell, R.B. (1995). Object-related ac-
Gauthier, I., Skudlarski, P., Gore, J.C., and Anderson, A.W. tivity revealed by functional magnetic resonance imaging
(1999). Activation of the middle fusiform ‘face area’ in- in the human occipital cortex. Proc. Natl. Acad. Sci. USA
creases with expertise in recognizing novel objects. Nat. 92, 8135–8139.
Neurosci. 2, 568–573. Mather, G., Radford, K., and West, S. (1992). Low-level visual
Goodale, M.A., and Humphrey, G.K. (1998). The objects of processing of biological motion. Proc. R. Soc. Lond. B Biol.
action and perception. Cognition 67, 181–207. Sci. 249, 149–155.
Grill-Spector, K., Kushnir, T., Edelman, S., Avidan, G., Neri, P., Morrone, M.C., and Burr, D.C. (1998). Seeing bio-
Itzchak, Y., and Malach, R. (1999). Differential processing logical motion. Nature 395, 894–896.
of objects under various viewing conditions in the human Orban, G.A., Dupont, P., De Bruyn, B., Vogels, R., Vanden-
lateral occipital complex. Neuron 24, 187–203. berghe, R., and Mortelmans, L. (1995). A motion area in
Grill-Spector, K., Kourtzi, Z., and Kanwisher, N. (2001). The human visual cortex. Proc. Natl. Acad. Sci. USA 92,
lateral occipital complex and its role in object recognition. 993–997.
Vision Res. 41, 1409–1422. O’Toole, A.J., Roark, D.A., and Abdi, H. (2002). Recognizing
Grossman, E.D., and Blake, R. (2001). Brain activity evoked moving faces: A psychological and neural synthesis. Trends
by inverted and imagined biological motion. Vis. Res. 41, Cogn. Sci. 6, 261–266.
1475–1482. Pelli, D.G. (1997). The Video Toolbox software for visual psy-
Grossman, E., Donnelly, M., Price, R., Pickens, D., Morgan, chophysics: Transforming numbers into movies. Spat. Vis.
V., Neighbor, G., and Blake, R. (2000). Brain areas involved 10, 437–442.
in perception of biological motion. J. Cogn. Neurosci. 12, Puce, A., Allison, T., Bentin, S., Gore, J.C., and McCarthy, G.
711–720. (1998). Temporal cortex activations in viewing eye and
Haxby, J.V., Grady, C.L., Horwitz, B., Ungerleider, L.G., mouth movements. J. Neurosci. 18, 2188–2199.
Mishkin, M., Carson, R.E., Herscovitch, P., Schapiro, M.B., Schiller, P.H., Logothetis, N.K., and Charles, E.R. (1990).
and Rapoport, S.I. (1991). Dissociation of object and spatial Functions of the colour-opponent and broad-band channels
visual processing pathways in human extrastriate cortex. of the visual system. Nature 343, 68–70.
Proc. Natl. Acad. Sci. USA 88, 1621–1625. Sereno, M.E., Trinath, T., Augath, M., and Logothethis, N.K.
Haxby, J.V., Gobbini, M.I., Furey, M.L., Ishai, A., Schouten, (2002). Three-dimensional shape representation in monkey
J.L., and Pietrini, P. (2001). Distributed and overlapping cortex. Neuron 33, 635–652.
representations of faces and objects in ventral temporal cor- Skudlarski, P., Constable, R.T., and Gore, J.C. (1999). ROC
tex. Science 293, 2425–2430. analysis of statistical methods used in functional MRI: Indi-
Hoffman, E., and Haxby, J. (2000). Distinct representations of vidual subjects. Neuroimage 9, 311–329.
eye gaze and identity in the distributed human neural sys- Somers, D.C., Dale, A.M., Seiffert, A.E., and Tootell, R.B.
tem for face perception. Nat. Neurosci. 2, 574–580. (1999). Functional MRI reveals spatially specific attentional
RT0996_C08.qxd 11/8/04 2:04 PM Page 114
modulation in human primary visual cortex. Proc. Natl. Van Oostende, S., Sunaert, S., Van Hecke, P., Marchal, G., and
Acad. Sci. USA 96, 1663–1668. Orban, G.A. (1997). The kinetic occipital (KO) region in
Suneart, S., Van Hecke, P., Marchal, G., and Orban, G.A. man: An fMRI study. Cereb. Cortex 7, 690–701.
(1999). Motion responsive regions of the human brain. Exp. Watanabe, T., Hamer, A.M., Miyauchi, S., Sasaki, Y., Nielsen,
Brain Res. 127, 355–370. M., Palomo, D., and Mukai, I. (1998). Task-dependent in-
Tanaka, K. (1996). Inferotemporal cortex and object vision. fluences of attention on the activation of human primary vi-
Annu. Rev. Neurosci. 19, 109–139. sual cortex. Proc. Natl. Acad. Sci. USA 95, 11489–11492.
Tootell, R.B.H., Reppas, J.B., Kwong, K.K., Malach, R., Watson, J.D.G., Myers, R., Frackowiak, R.S., Hajnal, J.V.,
Born, R.T., Brady, T.J., Rosen, B.R., and Belliveau, J.W. Woods, R.P., Mazziotta, J.C., Shipp, S., and Zeki, S. (1993).
(1995). Functional analysis of human MT and related visual Area V5 of the human brain: Evidence from a combined
cortical areas using magnetic resonance imaging. J. Neu- study using positron emission tomography and magnetic
rosci. 15, 3215–3230. resonance imaging. Cereb. Cortex 3, 79–94.
Ungerleider, L., and Mishkin, M. (1982). Two cortical visual Wojciulik, E., Kanwisher, N., and Driver, J. (1998). Covert
systems. In Analysis of Visual Behavior, D. Ingle, M. visual attention modulates face-specific activity in the
Goodale, and R. Mansfield, eds. (Cambridge, MA: MIT human fusiform gyrus: fMRI study. J. Neurophys. 79,
Press), pp. 549–586. 1574–1578.
Vaina, L.M., Solomon, J., Chowdhury, S., Sinha, P., and Bel- Woods, R.P., Grafton, S.T., Watson, J.D., Sicotte, N.L., and
liveau, J.W. (2001). Functional neuroanatomy of biological Mazziotta, J.C. (1998). Automated image registration: II.
motion perception in humans. Proc. Natl. Acad. Sci. USA Intersubject validation of linear and nonlinear models. J.
98, 11656–11661. Comput. Assist. Tomogr. 22, 155–165.
RT0996_C09.qxd 11/8/04 2:04 PM Page 115
R E A D I N G 9
Electrophysiology and Brain Imaging of

Biological Motion
•
Aina Puce1 and David Perrett2
The movements of the faces and bodies of other conspecifics provide stimuli of considerable
interest to the social primate. Studies of single cells, field potential recordings and functional
neuroimaging data indicate that specialized visual mechanisms exist in the superior temporal
sulcus (STS) of both human and non-human primates that produce selective neural responses to
moving natural images of faces and bodies. STS mechanisms also process simplified displays of
biological motion involving point lights marking the limb articulations of animate bodies and
geometrical shapes whose motion simulates purposeful behaviour. Facial movements such as
deviations in eye gaze, important for gauging an individual’s social attention, and mouth
movements, indicative of potential utterances, generate particularly robust neural responses that
differentiate between movement types. Collectively such visual processing can enable the decoding
of complex social signals and through its outputs to limbic, frontal and parietal systems the STS
may play a part in enabling appropriate affective responses and social behaviour.
Keywords: biological motion; event related potentials; functional magnetic resonance imaging;
humans; single-unit electrophysiology; animals.
1. Introduction integrate effectively within their social living

structure. At a non-social level, successful preda-
Primates, being social animals, continually ob- tor evasion also necessitates being able to ‘read’
serve one another’s behaviour so as to be able to the actions of other species in one’s vicinity. The
1 2
Centre for Advanced Imaging, Department of Radiology, School of Psychology, University of St Andrews, St Andrews,
West Virginia University, PO Box 9236, Morgantown, WV Fife KY16 9JU, UK.
26506–9236, USA.
115
RT0996_C09.qxd 11/8/04 2:04 PM Page 116
ability to interpret the motion and action of others

in human primates goes beyond basic survival and
successful interactions with important con-
specifics. Many of our recreational and cultural
pursuits would not be possible without this ability.
Excellent symphony orchestras exist not only ow-
ing to the exceptional musicians, but also their
ability to interpret their conductors’ non-verbal
instructions. Conductors convey unambiguously
not only the technical way that the orchestra
should execute the piece of music, but modulate
the mood and emotional tone of the music meas-
ure by measure. The motion picture industry owes
much of its success today to its silent movie pio-
neers, who could entertain with their non-verbal
antics. The world’s elite athletes rely on the inter-
pretation of other’s movements to achieve their
team’s goals successfully and foil opponents.
2. Human Behavioural Studies of

Biological Motion Perception
FIGURE 9.1 ■ An example of a biological motion stimu-
The perception of moving biological forms can lus. (Adapted from Johansson (1973), with permission
rely on the ability to integrate form and motion from Percept. Psychophys.)
but it can also rely on the ability to define form
from motion (Oram & Perrett 1994, 1996). The et al. 1990a; Thornton et al. 1998). Fourth, the
latter is evident in the ingenious work of Johans- gender of the walker (and even the identity of spe-
son who filmed actors dressed in black with white cific individuals) can be recognized from pattern
dots attached to their joints on a completely black of gait and idiosyncratic body movements in these
set (Johansson 1973). With these moving dots hu- impoverished displays (Cutting & Kozlowski
man observers could reliably identify the walking 1977; Kozlowski & Cutting 1977). Fifth, there is
or running motions, for example, of another hu- a bias to perceive forward locomotion, at the
man or an animal (Figure 9.1). This type of stim- expense of misinterpreting the underlying form in
ulus is known as a Johansson, point light or bio- time-reversed biological motion films (Pavlova
logical motion display. et al. 2002). Finally, observers can discern various
A number of important observations have emotional expressions from viewing Johansson
emerged from the human behavioural biological faces (Bassili 1978).
motion perception literature. First, the perceptual In very low light conditions many animals are
effect of observing an individual walking or run- efficient at catching prey or evading predators. In
ning is severely compromised when the display is such conditions the patterns of articulation (typi-
inverted (Dittrich 1993; Pavlova & Sokolov cal of biological motion) may be more discernible
2000). Second, while biological motion repre- than the form of stationary animals. Indeed, in be-
senting locomotory movements is recognized the havioural experiments it is evident that point light
most efficiently, social and instrumental actions displays are sufficient for cats to discriminate the
can also be recognized from these impoverished pattern of locomotion of conspecifics (Blake
displays (Dittrich 1993). Third, biological motion 1993). In an ingenious behavioural study in cats, a
can be perceived even within masks of dots (Perrett forced choice task where selection of a biological
RT0996_C09.qxd 11/8/04 2:04 PM Page 117
Electrophysiology and Brain Imaging of Biological Motion ■ 117
motion display (of a cat walking or running) was for the sight of faces were characterized in this re-
rewarded with food resulted in the animals per- gion in monkeys (Perrett et al. 1982; Desimone
forming significantly above chance. A series of 1991). This STS brain region is known to be a
foil stimuli showing dots changing their spatial convergence point for the dorsal and ventral vi-
location provided a set of tight controls in this ex- sual streams. The STP area derives its input from
periment (Blake 1993). the MST area in the dorsal pathway and the ante-
Evidence for the existence of specialized brain rior inferior-temporal area in the ventral pathway
systems that analyse biological motion (and the (Boussaoud et al. 1990; Felleman & Van Essen
motion of humans and non-humans) comes from 1991). The cortex of the STS has connections
neuropsychological lesion studies. Dissociations with the amygdala (Aggleton et al. 1980) and also
between the ability to perceive biological motion with the orbitofrontal cortex (Barbas 1988), re-
and other types of motion have been demon- gions implicated in the processing of stimuli of
strated. Several patients who are to all intents and social and emotional significance in both human
purposes ‘motion blind’ can discriminate biologi- and non-human primates (reviewed in Baron-
cal motion stimuli (Vaina et al. 1990; McLeod et Cohen 1995; Brothers 1997; Adolphs 1999).
al. 1996). The opposite pattern, i.e., an inability to In addition to having face-specific cells, the
perceive biological motion yet have relatively cortex of the STS has other complex response
normal motion perception in general, has also properties. It has emerged that visual information
been reported (Schenk & Zihl 1997). about the shape and posture of the fingers, hands,
arms, legs and torso all impact on STS cell tuning
in addition to facial details such as the shape of
3. Biological Motion Perception in the mouth and direction of gaze (Desimone et al.
Non-Humans 1984; Wachsmuth et al. 1994; Perrett et al. 1984,
1985a; Jellema et al. 2000). Motion information
One brain region known as the STP area in the presumed to arrive from the dorsal stream projec-
cortex surrounding the STS has been the subject tions arrives in the STS some 20 ms ahead of form
of considerable scrutiny ever since cells selective information from the ventral stream (Figure 9.2a);
a) b)
70
response (spikes s–1)
130
response latency (ms)
60
120
50
110 40
SA
100 30
90
20
10
80 0
form + motion form
motion body
view
motion
< < > < < > direction
FIGURE 9.2 ■ Some response properties of primate STP area neurons elicited by biological motion
stimuli. (Adapted from Oram & Perrett (1994, 1996), with permission.) (a) Average response latencies
for neurons with different response properties. (b) An example of a neuron that does not differentiate be-
tween real human motion and biological motion. Also, the strongest response is in the motion direction
compatible with direction of the body.
RT0996_C09.qxd 11/8/04 2:04 PM Page 118
but despite this asynchrony, STS processing over- object move appropriately but are spatially sepa-
comes the ‘binding problem’ and only form and rated. This selectivity ensures that the cells are
motion arising from the same biological object more responsive in situations where the agent’s
are integrated within 100 ms of the moving form motion is causally related to the object’s motion.
becoming visible (Oram & Perrett 1996). Indeed, The STS cell populations coding body and hand
STS cell integration of form and motion is wide- actions appear to be exclusively visual, although
spread and there are numerous cell types special- information from the motor system does affect
izing in the processing of different types of face, other STS cell populations (Hietanen & Perrett
limb and whole body motion (Perrett et al. 1985b; 1996) and modulates STS activity in humans
Carey et al. 1997; Jellema et al. 2000, 2002; (Iacoboni et al. 2001; Nishitani & Hari 2001).
Jellema & Perrett 2002). Information defined by the visual characteriza-
While most STS cells derive sensitivity to body tion of actions in the STS appears to be relayed
movement by combining signals about the net via parietal systems (Gallese et al. 2002) to
translation or rotation of the body with the face frontal motor planning systems. In frontal and
and body form visible at any moment in time, a parietal areas a neural system has recently been
smaller proportion (20%) of cells are able to re- found to respond selectively both during the exe-
spond selectively to the form of the body defined cution of hand actions, and (like STS cells) during
through patterns of articulation in point light dis- the observation of corresponding actions per-
plays (Perrett et al. 1990a,b; Oram & Perrett formed by others. The frontal region of primate
1994, 1996; Figure 9.1). These cells tuned to bio- cortex had long been known to be somatotopically
logical motion are selective for the sight of the organized for the representation and control of
same action visible in full light and when depicted movements of the mouth and arm (Rizzolatti et al.
in point light displays. 1988). Neurons within area F5 of the monkey pre-
Cells responding to whole body motion exhibit motor cortex have now been labelled ‘mirror’ neu-
selectivity for direction of motion and view of the rons, because they discharge when monkeys per-
body: most respond preferentially to compatible form or observe the same hand actions (di
motion with the body moving forward in the di- Pellegrino et al. 1992; Rizzolatti et al. 1996a,b;
rection it faces, though some are tuned to back- Gallese et al. 1996). An F5 cell selective for the
ward locomotion with the body moving in the op- action of grasping would respond for example
posite direction to the way it faces (Perrett et al. when the monkey grasps an object in sight or in
1985b, 1989; Oram & Perrett 1996; Figure 9.2b). the dark (thereby demonstrating motoric proper-
This cellular tuning bias for forward locomotion ties). The visual properties of such an F5 cell are
may underlie the forward bias found in perceptual strikingly similar to those described in the STS:
interpretation of locomotion depicted in point both F5 and STS cells will respond when the
light displays (Pavlova et al. 2002). monkey observes the experimenter reaching and
Responses to purposeful hand object actions grasping an object, but not to the sight of the ex-
such as reaching for, picking, tearing and manipu- perimenter’s hand motion alone or the sight of the
lating objects have also been characterized in the object alone. These conjoint properties have led
STS (Perrett et al. 1989, 1990c; Jellema et al. Rizzolatti et al. (1996a,b) and Gallese et al.
2000). These STS cells are sensitive to the form of (1996) to postulate that the F5 neurons form a sys-
the hand performing the action, and are unrespon- tem for matching observation and executing ac-
sive to the sight of tools manipulating objects in tions for the grasping, manipulation and place-
the same manner as hands. Furthermore, the cells ment of objects. Because the cells additionally
code the spatio-temporal interaction between the respond selectively to the sound of actions
agent performing the action and the object of the (Kohler et al. 2002), the mirror system may pro-
action. For example, cells tuned to hands manipu- vide a supra-modal conceptual representation of
lating an object cease to respond if the hands and actions and their consequences in the world.
RT0996_C09.qxd 11/8/04 2:04 PM Page 119
Crucially the properties of the frontal mirror sys- 4. Human Neuroimaging and
tem indicate that we may understand actions per- Electrophysiological Studies of
formed by others because we can match the Biological Motion Perception
actions we sense through vision (and audition) to
our ability to produce the same actions ourselves. The first suggestion that humans may possess spe-
The actions of others are not always fully visi- cialized biological motion perception mecha-
ble, for example someone may become hidden nisms came from a point light display depicting a
from our sight as they move behind a tree, or their moving body designed to investigate the response
hands may not remain fully in view as they reach properties of medial temporal/V5, a region of oc-
to retrieve an object. The similarity of STS and F5 cipito-temporal cortex known to respond to mo-
systems in processing of actions has become more tion. In this fMRI study activation was observed
apparent in experiments investigating the nature in MT/V5 as well as areas of superior temporal
of processing during these moments when actions cortex. This was regarded at that time as surpris-
are partially or totally occluded from sight. Within ing, as the activation appeared to lie in brain
the STS it is now apparent that specific cell popu- regions traditionally regarded as participating in
lations are activated when the presence of a hid- auditory speech processing (Howard et al. 1996).
den person can be inferred from the preceding vi- Localization of primary auditory cortex was not
sual events (i.e., they were witnessed passing out performed in this visual stimulation study. In a
of sight behind a screen and have not yet been wit- PET study published in the same year Johansson
nessed re-emerging into sight, so they are likely to displays of body motion (depicting a person danc-
remain behind the screen; Baker et al. 2001). In ing), hand motion (depicting a hand reaching for a
an analogous manner, F5 cells may respond to the glass and bringing it to a mouth), object motion
sight of the experimenter reaching to grasp an ob- (depicting a three-dimensional structure rotating
ject. The same cells are active when the experi- and pitching) and control conditions, consisting
menter places an object behind a screen and then of either random dot motion or a static display of
reaches as if to grasp it (even though the object randomly placed dots, were shown to a group of
and hand are hidden from view (Umilta et al. healthy subjects (Bonda et al. 1996). The human
2001)). The sight of equivalent reaching when motion conditions selectively activated the infe-
there is no reason to believe an object is hidden rior parietal region and the STS. Specifically, the
from sight fails to activate the F5 cells. Thus F5 body motion condition selectively activated the
and STS cells code the sight of actions on the ba- right posterior STS, whereas the hand motion
sis of what is currently visible and on the basis of condition activated the left intraparietal sulcus
the recent perceptual history (Jellema & Perrett and the posterior STS (Bonda et al. 1996). In a
2002; Jellema et al. 2002). more recent fMRI study, a Johansson display
The manner in which temporal STS and frontal depicting a walker was used and the activation
F5 systems interact is not fully clear, but appears contrasted to control conditions that included a
to involve intermediate processing steps mediated dot display with non-random motion and a gender
by parietal areas (Nishitani & Hari 2000, 2001; discrimination task with real images of faces
Gallese et al. 2002). While STS and F5 cells have (Vaina et al. 2001). Biological motion differen-
similar visual properties they may subserve dis- tially activated a large number of dorsal and ven-
tinct functions; the frontal system perhaps serves tral regions, most notably the lateral occipital
to control the behaviour of the self particularly in complex, but the STS was not preferentially acti-
dealing with objects (Rizzolatti et al. 1996a,b), vated in this study.
whereas the STS system is specialized for the de- Grossman and colleagues found that biological
tection and recognition of the behaviour of others motion stimuli depicting jumping, kicking, running
(Perrett et al. 1990c; Mistlin & Perrett 1990; Hi- and throwing movements produced more right STS
etanen & Perrett 1996). activation than control motion irrespective of the
RT0996_C09.qxd 11/8/04 2:04 PM Page 120
visual field in which the biological motion display preferentially process certain attributes of these
was presented. Conversely, the control motion, high-level visual displays (reviewed by Allison et
including scrambled biological motion displays, al. 2000; Blakemore & Decety 2001). Figure 9.3
activated MT/MST areas and the lateral-occipital displays activation observed in these studies, ly-
complex (Grossman et al. 2000). Moreover, the ing along the posterior extent of the STS and its
STS region could also be activated by imagining ascending limb in inferior parietal cortex in re-
Johansson stimuli, although the size of the activa- sponse to observing movements of the body,
tion was small (Grossman et al. 2000). While the hands, eye and mouth. Activation in these regions
most robust STS activation was elicited by viewing can also be elicited to imagining the motion of
upright Johansson displays, a smaller STS activa- others (Grossman et al. 2000), and additionally to
tion signal was also seen to viewing inverted Jo- viewing static images of implied motion (Kourtzi
hansson displays. & Kanwisher 2000).
While biological motion clearly activates the Interestingly, differences in activation patterns
STS region in humans, the function of the region can occur when subjects view compatible versus
may be more general in performing a visual incompatible motion of the head or body (Thompson
analysis of bodies based on either the characteris- et al. 2002a). Specifically, the bilateral posterior
tic patterns of articulation that comprise biologi- lateral temporal cortex is active when viewing
cal motion or information about bodies that can compatible motion. By contrast, viewing incom-
be derived from static images (Downing et al. patible motion activates the right posterior lateral
2001); hence the term ‘extrastriate body area’ has temporal cortex, left anterior temporal cortex, left
been applied to one cortical region within the STS temporoparietal junction and left precentral
complex. gyrus. This extended network of activation might
be due to the novelty or salience of the incongru-
ent body and head motion stimuli (Downar et al.
5. Biological Motion Perception Versus 2002). The differential experience with compati-
Human Motion Perception ble and incompatible motion may explain STS
cell sensitivity to the compatibility of motion
As in non-human primates, responsiveness to direction and body view during the locomotion
Johansson-like displays of facial motion is pres- described above.
ent in STS regions that also respond to real im- What is unique about the motion of animate be-
ages of facial motion, e.g., non-linguistic mouth ings? Animals and humans possess articulated
movements (Puce et al. 2001), although the per- joints, enabling the movement of body parts with-
cent magnetic resonance signal change to the out having to maintain a constant spatial relation-
Johansson-like face was smaller than that ob- ship in space relative to each other. This results in
served to the natural facial images. In parallel to the ability to produce a limitless set of movements.
the neuroimaging data, direct measures of neural Man-made objects, such as utensils and tools, in
activity in humans, in the form of scalp ERPs, are general do not have this capability. Beauchamp
elicited to Johansson-like and real images of faces et al. (2002) investigated the differences in brain
(Thompson et al. 2002b), with a prominent nega- activation to these different types of high-level
tivity occurring at ca. 170 ms post-motion onset motion stimuli. Interestingly, observing human
(N170) over the bilateral temporal scalp. This ac- motion stimuli activated the STS and observing
tivity is significantly greater than that seen to mo- the motion of tools/utensils activated cortex ven-
tion controls. tral to the STS, on the MTG. In another fMRI ex-
Over the latter half of the 1990s, a series of periment in this same study, stimuli depicting ar-
PET and fMRI studies examining activation to ticulated and non-articulated human motion were
viewing the motion and actions of others have presented. The STS responded to the articulated
pointed to the existence of cortical networks that human motion and the MTG to non-articulated
RT0996_C09.qxd 11/8/04 2:04 PM Page 121
mouth
Calvert et al. lip reading (STG) eyes
Calvert et al. lip reading (AG) Puce et al. eye gaze
Puce et al. mouth movement Wicker et al. eye gaze
Puce & Allison mouth movement Hoffman & Haxby eye gaze
body hand
Howard et al. body movement Neville et al. ASL
Bonda et al. body movement Bonda et al. hand action
Senior et al. body movement Grezes et al. hand action
Kourtzi & Kanwisher Grezes et al. hand movement
body movement Grafton et al. hand grasp
Grossman et al. body movement Rizzolatti et al. hand grasp
FIGURE 9.3 ■ (A color version of this figure follows page 146.) Centers of
activation to viewing the face, hand and body movements of others obtained
from a series of PET and fMRI studies. (Adapted from Allison et al. (2000),
with permission.)
motion, indicating that these high-order process- motion. Specifically, they observed an anterior-
ing mechanisms process selectively the higher-or- posterior gradient of activation in the STS
der motion type (Beauchamp et al. 2002). regions, with non-rigid motion producing the
Grezes et al. (2001) also reported activation most anterior activation. Additionally, they ob-
differences between observing rigid and non-rigid served activation in left intraparietal cortex to
RT0996_C09.qxd 11/8/04 2:04 PM Page 122
non-rigid biological motion (Grezes et al. 2001). B. Gaze Perception

The magnitude of the activation in the STS to bi- Neuroimaging studies involving gaze perception
ological motion, and indeed in other cortical re- indicate that there is an active cortical network in-
gions, can be coloured by the task requirements volving occipito-temporal cortex (fusiform gyrus,
and the attention that the observer places on the inferior temporal gyrus, parietal lobule and bilat-
‘human’ quality of the motion (Vaina et al. 2001). eral middle temporal gyri) when subjects pas-
Additionally, attention to the displayed emotion sively view gaze aversion movements (Wicker
enhances fMRI activation in the STS, whereas in- et al. 1998). One prominently active region to
creased activation to facial attributes per se, such viewing eye movements (gaze aversion and also
as identity or isolated features, increased activa- eyes looking at the observer) is the cortex around
tion in all known face-sensitive cortical regions the STS, particularly in the right hemisphere, and
(Narumoto et al. 2001). this same region is active also to viewing opening
A. Social Cognition and closing movements of the mouth (Puce et al.
1998). Thus, as is evident from the single cell
The limbic system, in conjunction with the or- responses, the STS region contains neural popula-
bitofrontal cortex and the STS, is thought to form tions representing multiple aspects of the appear-
a network that is involved in social cognition ance of the face (including gaze) and body and
(Baron-Cohen 1995; Brothers 1997; Adolphs their motion; the STS should not be considered
1999). One important aspect of social cognition is exclusively an ‘eye detector’ or ‘eye processor’.
the identification of the direction of another’s The STS is more activated during judgements of
attention from their direction of gaze or head view gaze direction than during judgements of identity,
(Perrett et al. 1985a, 1992; Kleinke 1986; Allison whereas the fusiform and inferior occipito-tempo-
et al. 2000; Emery 2000). Indeed, the existence of ral activation is stronger during judgements of
an eye direction detector has been postulated in identity than gaze direction (Hoffman & Haxby
this hierarchical system of social cognition, which 2000). Intracranial ERP recordings from these
at its top level allows us to ‘mind-read’ and infer structures indicate that the STS responds to facial
the intentions of others (Baron-Cohen 1995; motion, whereas the ventral-temporal cortex re-
Baron-Cohen et al. 1997). While there is evidence sponds more strongly to static facial images (Puce
for cell populations coding for eye and attention & Allison 1999). This is not surprising if one con-
direction within STS (Perrett et al. 1985a, 1992), siders that eye gaze direction changes are tran-
the populations are not anatomically grouped in sient and their detection might require motion
such a way that scalp evoked potentials are neces- processing systems, whereas identity judgements
sarily linked to a given eye direction (Bentin et al. can be made independently of facial movements.
1996; Eimer 1998; Taylor et al. 2001). Our atten- Indeed, the processing of dynamic information
tion and behaviour can be modified when con- about facial expression and the processing of
fronted with a face with averted gaze. A periph- static information about facial identity appear
eral target stimulus is detected by normal subjects neuropsychologically dissociable (Campbell 1992;
more efficiently when it lies in the direction of gaze Humphreys et al. 1993).
of a central stimulus face (Friesen & Kingstone
1998; Driver et al. 1999; Hietanen 1999, 2002;
Langton & Bruce 2000). Moreover, patients with C. Lip Reading
unilateral neglect are less likely to extinguish a Lip reading, an important function for both hear-
contralesional target stimulus when it lies in the ing and deaf individuals, can be neuropsychologi-
gaze path of a stimulus face (Vuilleumier 2002). cally dissociated from face recognition (Campbell
Following the attention direction of someone’s et al. 1986), in a somewhat similar manner to gaze
gaze may be such an over-learned response that it perception. Normal lip reading uses cortex of the
needs little conscious awareness. STG in addition to other brain regions such as the
RT0996_C09.qxd 11/8/04 2:04 PM Page 123
angular gyrus, posterior cingulate, medial frontal The neuroimaging data mesh well with re-
cortex and frontal pole (Calvert et al. 1997). The ported disturbances in executing grasping move-
STG and surrounding cortex activate bilaterally ments in the neuropsychological lesion literature.
when subjects view face actions that could be in- For example, Jeannerod and colleagues have
terpreted as speech (Puce et al. 1998; Campbell et reported a case with bilateral posterior parietal le-
al. 2001), while some regions of the posterior sions of vascular origin where there was no diffi-
right STS activate for the sight of speech and non- culty in reaching toward the location of the object;
speech mouth movements (Campbell et al. 2001). however, a profound deficit in executing the antic-
Centres of activation to visual speech appear to ipatory grasping movement with the fingers
overlap those associated with hearing speech occurred to nondescript objects (cylindrical dow-
(Calvert et al. 1997), indicating that these regions els). Interestingly, there was no deficit in grasping
receive multimodal inputs during speech analysis behaviour when well-known recognizable objects
(Kawashima et al. 1999; Calvert et al. 2000). Fur- were used in the same test (Jeannerod et al. 1994).
ther evidence for this multimodal integration is a Mental imagery of hand and finger movements
phenomenon known as the McGurk effect was found to be impaired in patients with unilat-
(McGurk & MacDonald 1976), where what ob- eral parietal lesions, who had difficulties in pro-
servers hear when listening to speech sounds is al- ducing movements with their hands and fingers
tered by simultaneously viewing mouth move- (Sirigu et al. 1996). It has been reported that pa-
ments appropriate to a different speech utterance. tients with unilateral parietal lesions have more
Indeed, magnetoencephalographic recordings of difficulty in imitating gestures involving their
neural activity to speech stimuli show sensitivity own bodies relative to movements involving ex-
to auditory-visual mismatch (Sams et al. 1991) ternal objects, particularly if the lesion is in the
with activity 200 ms poststimulus augmented left hemisphere (Halsband et al. 2001).
when the visual speech does not correspond to the The human STS in its posterior extent has been
accompanying auditory speech. found to be active not only to the hand and body
movements of others (see Figure 9.3; Allison
D. The Mirror Neuron System and Action et al. 2000), but also to faces (Puce et al. 1998).
Observation/Execution Interestingly, ERP recordings indicate that neural
The existence of a mirror neuron system in hu- activity can differentiate between types of facial
mans has been investigated during the manipu- movements (Puce et al. 2000). Viewing mouth
lation of objects (Rizzolatti et al. 1996a,b; opening movements produces larger N170 re-
Binkofski et al. 1999a,b). The activation in sponses relative to viewing mouth closing move-
fronto-central regions, seen when subjects ob- ments. A similar N170 response gradient is seen
serve and/or execute grasping behaviours, is ac- for observing eyes averting their gaze away from
companied by activity in the parietal cortex and the observer relative to eyes focusing their gaze
STS (Jeannerod et al. 1995; Iacoboni et al. on the observer. Augmented neural responses to
1999, 2001; Rizzolatti et al. 2001; Gallese et al. eye aversion movements may be a powerful signal
2002), paralleling the mirror neuron system in that the observer is no longer the focus of an-
non-human primates. other’s attention. Similarly, larger N170s to
Additionally, the secondary somatosensory mouth opening movements might be important
cortex, SII, located in the temporal operculum is for recognizing the beginning of an utterance
postulated to analyse the intrinsic properties of (Puce et al. 2000). With recording electrodes sited
the graspable object while activation observed in in the STS of epilepsy surgery patients, selective
the cortex in the intraparietal sulcus was thought responses to mouth opening have been elicited
to be related to kineasthetic processes (Binkofski (see Allison et al. 2000, box 1). No responses
et al. 1999b), although strictly speaking it is not were observed to mouth closing movements or
part of the mirror neuron system. eye deviations, indicating that these regions might
RT0996_C09.qxd 11/8/04 2:04 PM Page 124
be responsive during lip reading (or the sight of Alternatively, if the central face looked away
gestures and emotional expressions in which the from the observer in the opposite direction to the
mouth opens, e.g., during eating and surprise). other two faces, a mutual gaze exchange
The Talairach coordinates of these electrode posi- between the central face and one of the flankers
tions are comparable to sites of fMRI activation in became apparent (‘mutual gaze exchange’).
lip reading (Calvert et al. 1997). Finally, the central face could look away from
If eye aversion movements are given a con- the observer and the other two flanker faces by
text, late ERPs that differ as a function of the so- looking up (‘control’). An N170 ERP to the gaze
cial significance of the aversion movement can aversion of the central face was elicited, and its
be elicited (Figure 9.4; A. Cooper and A. Puce, characteristics did not change as a function of
unpublished data). This was demonstrated in a condition (see also Puce et al. 2000). A later pos-
visual task where two permanently gaze-averted itive ERP, elicited between 300 and 500 ms post-
flanker faces were presented with a central face motion onset (P400) was seen to differentiate in
that changed its gaze direction. The central face latency as a function of viewing condition: group
could look in the same direction as both flanker attention produced the shortest latency response,
faces, setting up an apparently common focus followed by the mutual gaze exchange condition
of attention off to the side (‘group attention’). and then the control condition.
group attention
a) b)
8 P400
6 µV
amplitude (µV)
4
mutual gaze exchange 3
4 2
1
0
2
_1
_2
_3
_4
0
_2
control
_4 N170
_100 0 100 200 300 400 500 600 700 800 900
time (ms)
FIGURE 9.4 ■ (A color version of part b of this figure follows page 146.) ERPs elicited to a social attention task. (a)
ERP waveforms elicited to three conditions: solid line, group attention; dashed line, mutual gaze exchange; dotted line,
control. The arrows indicate a late peak of ERP activity that follows the N170 ERP (P400), which changes its latency
as a function of viewing condition. (b) Voltage maps for the three viewing conditions generated at the peak of P400
activity for the group attention condition (black arrow in (a)). The group attention condition shows fronto-temporal pos-
itivity, whereas the other two conditions show small posterior positivities.
RT0996_C09.qxd 11/8/04 2:04 PM Page 125
Our non-verbal and verbal facial movements activity can be altered as a function of affect type
usually do occur in an affective context, and pre- (Wheaton et al. 2002b). If gesture-affect combina-
liminary ERP data indicate that our brains are very tions are incongruous, as shown by increased reac-
sensitive to these gesture-affect blends. If facial tion time to classify affect in behavioural data, late
movements (either non-verbal or verbal) are com- ERP activity from 300 to 975 ms post-motion on-
bined with different types of affect, temporal scalp set is modulated as a function of not only affect or
N170 peak latency and the amplitude of later ERP gesture but also their combination (Wheaton et al.
a) b)
(i)
posterior
temporal fronto-central
N170
> P130
>
> P270
>
> (ii )
> posterior
temporal
>
N170
> >
FIGURE 9.5 ■ Schematic summary of ERP waveforms elicited in response to observing human
motion. (a) Posterior temporal N170 (solid line) to conditions listed in the left column is larger rel-
ative to N170 (dashed line) elicited to conditions listed in the right column. b(i) Frontocentral
ERPs show larger P130 and P270 components across body and hand motion conditions shown
in the left and right columns (solid versus dashed line). b(ii) Posterior temporal N170 (solid line)
is larger to hand closure relative to hand opening (dashed line).
RT0996_C09.qxd 11/8/04 2:04 PM Page 126
2002a). These preliminary data indicate that the inability to form or maintain social relationships.
processing of inconsistencies in others’ behaviour This can be difficult if the sufferer cannot process
can be detected physiologically. incoming social messages communicated by the
ERPs, in the form of N170 negativities occur- bodily and facial actions of others, or sends inap-
ring over bilateral temporal scalp regions, have propriate social reactions to such signals (e.g.,
been elicited not only to facial movements but Williams et al. 2001). Further neuroimaging and
also to hand and body movements (Wheaton et al. neurophysiological studies of healthy subjects
2001). The N170 activity was larger for observing and those with impairments of human motion
hand clenching movements relative to hand open- processing may shed light on the interactions be-
ing movements. In addition, ERP activity was tween the various components of these high-level
also observed to hand and body motion over the biological motion processing systems.
central scalp. Interestingly, ERP activity was
larger to observing a body stepping forward than
to a body stepping back (paralleling the cellular Acknowledgments
bias for forward or compatible direction of loco- A.P.’s research has been supported by the Na-
motion; Perrett et al. 1985b; Oram & Perrett tional Health and Medical Research Council
1994). Taken together, the ERP differentiation in (Australia) and the Australia Research Council.
the hand and body movements might indicate a
stronger neural signal for potentially threatening
movements (Wheaton et al. 2001). When fMRI REFERENCES
activation to these movement types is compared, Adolphs, R. 1999 Social cognition and the human brain.
there is a robust signal within the temporoparietal Trends Cogn. Sci. 3, 469–479.
cortex to all of these motion types (Wheaton et al. Aggleton, J.P., Burton, M.J. & Passingham, R.E. 1980 Cortical
2002c). Figure 9.5 summarizes the main findings and subcortical afferents to the amygdala of the rhesus mon-
key (Macaca mulatta). Brain Res. 190, 347–368.
from the ERP studies (Puce et al. 2000; Wheaton Allison, T., Puce, A. & McCarthy, G. 2000 Social perception
et al. 2001; Thompson et al. 2002b), and indicates from visual cues: role of the STS region. Trends Cogn. Sci.
that processing between movement types begins 4, 267–278.
before 200 ms postmotion onset not only in the Baker, C.I., Keysers, C., Jellema, T., Wicker, B. & Perrett, D.I.
posterior temporal cortex but also in the fronto- 2001 Neuronal representation of disappearing and hidden
objects in temporal cortex of the macaque. Exp. Brain Res.
central regions, which would be expected from 140, 375–381.
the distribution of action processing evident in Barbas, H. 1988 Anatomic organization of basoventral and
fMRI and cell recording. mediodorsal visual recipient prefrontal regions in the rhesus
monkey. J. Comp. Neurol. 276, 313–342.
E. Gesture and Action Processing: Baron-Cohen, S. 1995 Mindblindness: an essay on autism and
Implications for Disorders of Social theory of mind. Cambridge, MA: MIT Press.
Baron-Cohen, S., Wheelwright, S. & Joliffe, T. 1997 Is there a
Communication ‘language of the eyes’? Evidence from normal adults, and
The processing of non-verbally presented mes- adults with autism or Asperger syndrome Vis. Cogn 4,
sages, in the form of face and hand gestures, is 311–331.
crucial for social primates to be able to interact Bassili, J. N. 1978 Facial motion in the perception of faces and
of emotional expression. J. Exp. Psychol. Hum. Percept.
with one another—and there are considerable Perf. 4, 373–379.
similarities in the high-level biological motion Beauchamp, M. S., Lee, K. E., Haxby, J. V. & Martin, A. 2002
processing systems in human and non-human Parallel visual motion processing streams for manipulable
primates. The importance of comprehending ac- objects and human movements. Neuron 34, 149–159.
tions of others may also be evident when such Bentin, S., Allison, T., Puce, A., Perez, A. & McCarthy,
G. 1996 Electrophysiological studies of face perception in
comprehension is impaired in clinical conditions. humans. J. Cogn. Neurosci. 8, 551–565.
Disorders such as autism, Asperger syndrome, Binkofski, F., Buccino, G., Posse, S., Seitz, R. J., Rizzolatti,
and schizophrenia are characterized by the G. & Freund, H. J. 1999a A fronto-parietal circuit for object
RT0996_C09.qxd 11/8/04 2:04 PM Page 127
manipulation in man: evidence from an fMRI study. Eur. J. Downar, J., Crawley, A. P., Mikulis, D. J. & Davis, K. D. 2002
Neurosci. 11, 3276–3286. A cortical network sensitive to stimulus salience in a neutral
Binkofski, F., Buccino, G., Stephan, K. M., Rizzolatti, behavioral context across multiple sensory modalities.
G., Seitz, R. J. & Freund, H. J. 1999b A parieto-premotor J. Neurophysiol. 87, 615–620.
network for object manipulation: evidence from neuroimag- Downing, P. E., Jiang, Y. H., Shuman, M. & Kanwisher,
ing. Exp. Brain Res. 128, 210–213. N. 2001 A cortical area selective for visual processing of the
Blake, R. 1993 Cats perceive biological motion. Psychol. Sci. human body. Science 293, 2470–2473.
4, 54–57. Driver, J., Davis, G., Ricciardelli, P., Kidd, P., Maxwell, E. &
Blakemore, S.-J. & Decety, J. 2001 From the perception of ac- Baron-Cohen, S. 1999 Gaze perception triggers reflexive
tion to the understanding of intention. Nature Rev. Neurosci. visuospatial orienting. Vis. Cogn. 6, 509–540.
2, 561–567. Eimer, M. 1998 Does the face-specific N170 component
Bonda, E., Petrides, M., Ostry, D. & Evans, A. 1996 Specific reflect the activity of a specialized eye processor? Neurore-
involvement of human parietal systems and the amygdala in port 9, 2945–2948.
the perception of biological motion. J. Neurosci. 16, Emery, N. J. 2000 The eyes have it: the neuroethology, func-
3737–3744. tion and evolution of social gaze. Neurosci. Biobehav. Rev.
Boussaoud, D., Ungerleider, L. G. & Desimone, R. 1990 Path- 24, 581–604.
ways for motion analysis: cortical connections of the medial Felleman, D. J. & Van Essen, D. C. 1991 Distributed hierar-
superior temporal and fundus of the superior temporal vi- chical processing in the primate cerebral cortex. Cerebr.
sual areas in the macaque. J. Comp. Neurol. 296, 462–495. Cortex 1, 1–47.
Brothers, L. 1997 Friday’s footprint: how society shapes the Friesen, C. K. & Kingstone, A. 1998 The eyes have it! Reflex-
human mind. New York: Oxford University Press. ive orienting is triggered by nonpredictive gaze. Psychol.
Calvert, G. A., Bullmore, E. T., Brammer, M. J., Campbell, Bull. Rev. 5, 490–495.
R., Williams, S. C., McGuire, P. K., Woodruff, P. W., Iversen, Gallese, V., Fadiga, L., Fogassi, L. & Rizzolatti, G. 1996
S. D. & David, A. S. 1997 Activation of auditory cortex dur- Action recognition in the premotor cortex. Brain 119,
ing silent lipreading. Science 276, 593–595. 593–609.
Calvert, G. A., Campbell, R. & Brammer, M. J. 2000 Evidence Gallese, V., Fadiga, L., Fogassi, L. & Rizzolatti, G. 2002 Ac-
from functional magnetic resonance imaging of crossmodal tion representation and the inferior parietal lobule. Attention
binding in the human heteromodal cortex. Curr. Biol. 10, Perform. 19, 247–266.
649–657. Grezes, J., Fonlupt, P., Bertenthal, B., Delon-Martin, C., Sege-
Campbell, R. 1992 The neuropsychology of lipreading. Phil. barth, C. & Decety, J. 2001 Does perception of biological mo-
Trans. R. Soc. Lond. B 335, 39–45. tion rely on specific brain regions? Neuroimage 13, 775–785.
Campbell, R., Landis, T. & Regard, M. 1986 Face recognition Grossman, E., Donnelly, M., Price, R., Pickens, D., Morgan,
and lipreading. Brain 109, 509–521. V., Neighbor, G. & Blake, R. 2000 Brain areas involved in
Campbell, R., MacSweeney, M., Surguladze, S., Calvert, G., perception of biological motion. J. Cogn. Neurosci. 12,
McGuire, P., Suckling, J., Brammer, M. J. & David, A. S. 711–720.
2001 Cortical substrates for the perception of face actions: Halsband, U., Schmitt, J., Weyers, M., Binkofski, F., Grützner,
an fMRI study of the specificity of activation for seen G. & Freund, H. J. 2001 Recognition and imitation of pan-
speech and for meaningless lower-face acts (gurning). Brain tomimed motor acts after unilateral parietal and premotor
Res. Cogn. Brain Res. 12, 233–243. lesions: a perspective on apraxia. Neuropsychologia 39,
Carey, D. P., Perrett, D. I. & Oram, M. W. 1997 Recognizing, 200–216.
understanding and reproducing action. In Handbook of neu- Hietanen, J. K. 1999 Does your gaze direction and head
ropsychology, vol. 11. Action and cognition (ed. M. Jean- orientation shift my visual attention? Neuroreport 10,
nerod), pp. 111–129. Amsterdam: Elsevier. 3443–3447.
Cutting, J. E. & Kozlowski, L. T. 1977 Recognizing friends by Hietanen, J. K. 2002 Social attention orienting integrates vi-
their walk: gait perception without familiarity cues. Bull. sual information from head and body orientation. Psychol.
Psychonomic. Soc. 9, 353–356. Res. 66, 174–179.
Desimone, R. 1991 Face-selective cells in the temporal cortex Hietanen, J. K. & Perrett, D. I. 1996 Motion sensitive cells in
of monkeys. J. Cogn. Neurosci. 3, 1–8. the macaque superior temporal polysensory area: response
Desimone, R., Albright, T. D., Gross, C. G. & Bruce, C. 1984 discrimination between self- and externally generated pat-
Stimulus-selective properties of inferior temporal neurons tern motion. Behav. Brain Res. 76, 155–167.
in the macaque. J. Neurosci. 4, 2051–2062. Hoffman, E. A. & Haxby, J. V. 2000 Distinct representations of
di Pellegrino, G., Fadiga, L., Fogassi, V., Gallese, V. & Rizzo- eye gaze and identity in the distributed human neural sys-
latti, G. 1992 Understanding motor events: a neurophysio- tem for face perception. Nature Neurosci. 3, 80–84.
logical study. Exp. Brain Res. 91, 176–180. Howard, R. J., Brammer, M., Wright, I., Woodruff, P. W.,
Dittrich, W. H. 1993 Action categories and the perception of Bullmore, E. T. & Zeki, S. 1996 A direct demonstration of
biological motion. Perception 22, 15–22. functional specialization within motion-related visual and
RT0996_C09.qxd 11/8/04 2:04 PM Page 128
auditory cortex of the human brain. Curr. Biol. 6, Mistlin, A. J. & Perrett, D. I. 1990 Visual and somatosensory
1015–1019. processing in the macaque temporal cortex: the role of ‘ex-
Humphreys, G. W., Donnelly, N. & Riddoch, M. J. 1993 Expres- pectation’. Exp. Brain Res. 82, 437–450.
sion is computed separately from facial identity, and it is com- Narumoto, J., Okada, T., Sadato, N., Fukui, K. & Yonekura,
puted separately for moving and static faces: neuropsycholog- Y. 2001 Attention to emotion modulates fMRI activity in
ical evidence. Neuropsychologia 31, 173–181. human right superior temporal sulcus. Cogn. Brain Res. 12,
Iacoboni, M., Woods, R. P., Brass, M., Bekkering, H., Mazziotta, 225–231.
J. C. & Rizzolatti, G. 1999 Cortical mechanisms of human Nishitani, N. & Hari, R. 2000 Temporal dynamics of cortical
imitation. Science 286, 2526–2528. representation for action. Proc. Natl Acad. Sci. USA 97,
Iacoboni, M., Koski, L. M., Brass, M., Bekkering, H., Woods, 913–918.
R. P., Dubeau, M. C., Mazziotta, J. C. & Rizzolatti, G. 2001 Nishitani, N. & Hari, R. 2001 Sign language and mirror neu-
Reafferent copies of imitated actions in the right superior ron system. Neuroimage 12(6), S452.
temporal cortex. Proc. Natl Acad. Sci. USA 98, 13 995– Oram, M. W. & Perrett, D. I. 1994 Responses of anterior supe-
13 999. rior temporal polysensory (STPa) neurons to ‘biological
Jeannerod, M., Decety, J. & Michel, F. 1994 Impairment of motion’ stimuli. J. Cogn. Neurosci. 6, 99–116.
grasping movements following a bilateral posterior parietal Oram, M. W. & Perrett, D. I. 1996 Integration of form and mo-
lesion. Neuropsychologia 32, 369–380. tion in the anterior superior temporal polysensory area
Jeannerod, M., Arbib, M. A., Rizzolatti, G. & Sakata, H. 1995 (STPa) of the macaque monkey. J. Neurophysiol. 76,
Grasping objects: the cortical mechanisms of visuomotor 109–129.
transformation. Trends Neurosci. 18, 314–320. Pavlova, M. & Sokolov, A. 2000 Orientation specificity in bi-
Jellema, T. & Perrett, D. I. 2002 Coding of visible and hidden ological motion perception. Percept. Psychophys. 62,
actions. Attention Perform. 19, 356–380. 889–899.
Jellema, T., Baker, C. I., Wicker, B. & Perrett, D. I. 2000 Pavlova, M., Krägeloh-Mann, I., Birbaumer, N. & Sokolov,
Neural representation for the perception of the intentional- A. 2002 Biological motion shown backwards: the apparent-
ity of hand actions. Brain Cogn. 44, 280–302. facing effect. Perception 31, 435–443.
Jellema, T., Oram, M. W., Baker, C. I. & Perrett, D. I. 2002 Perrett, D. I., Rolls, E. T. & Caan, W. 1982 Visual neurons re-
Cell populations in the banks of the superior temporal sul- sponsive to faces in the monkey temporal cortex. Exp. Brain
cus of the macaque and imitation. In The imitative mind: de- Res. 47, 329–342.
velopment, evolution, and brain bases (ed. A. Meltzoff & Perrett, D. I., Smith, P. A. J., Potter, D. D., Mistlin, A. J., Head,
W. Prinz), pp. 267–290. Cambridge University Press. A. S., Milner, A. D. & Jeeves, M. A. 1984 Neurones respon-
Johansson, G. 1973 Visual perception of biological motion and sive to faces in the temporal cortex: studies of functional or-
a model of its analysis. Percept. Psychophys. 14, 202–211. ganization, sensitivity to identity and relation to perception.
Kawashima, R., Imaizumi, S., Mori, K., Okada, K., Goto, Hum. Neurobiol. 3, 197–208.
R., Kiritani, S., Ogawa, A. & Fukuda, H. 1999 Selective Perrett, D. I., Smith, P. A. J., Potter, D. D., Mistlin, A. J., Head,
visual and auditory attention toward utterances: a PET A. S., Milner, A. D. & Jeeves, M. A. 1985a Visual cells in
study. Neuroimage 10, 209–215. the temporal cortex sensitive to face view and gaze direc-
Kleinke, C. L. 1986 Gaze and eye contact: a research review. tion. Proc. R. Soc. Lond. B 223, 293–317.
Psychol. Bull. 100, 78–100. Perrett, D. I., Smith, P. A. J., Mistlin, A. J., Chitty, A. J., Head,
Kohler, E., Keysers, C., Umilta, M. A., Fogassi, L., Gallese, A. S., Potter, D. D., Broennimann, R., Milner, A. D. &
V. & Rizzolatti, G. 2002 Hearing sounds, understanding ac- Jeeves, M. A. 1985b Visual analysis of body movements by
tions: action representation in mirror neurons. Science 297, neurones in the temporal cortex of the macaque monkey: a
846–848. preliminary report. Behav. Brain Res. 16, 153–170.
Kourtzi, Z. & Kanwisher, N. 2000 Activation in human Perrett, D. I., Harries, M. H., Bevan, R., Thomas, S., Benson,
MT/MST by static images with implied motion. J. Cogn. P. J., Mistlin, A. J., Chitty, A. J., Hietanen, J. K. & Ortega,
Neurosci. 12, 48–55. J. E. 1989 Frameworks of analysis for the neural represen-
Kozlowski, L. T. & Cutting, J. E. 1977 Recognizing the sex of tation of animate objects and actions. J. Exp. Biol. 146,
a walker from a dynamic point-light display. Percept. Psy- 87–113.
chophys. 21, 575–580. Perrett, D. I., Harries, M. H., Benson, P. J., Chitty, A. J. &
Langton, S. R. H. & Bruce, V. 2000 You must see the point: au- Mistlin, A. J. 1990a Retrieval of structure from rigid and bi-
tomatic processing of cues to the direction of social atten- ological motion; an analysis of the visual response of neu-
tion. J. Exp. Psychol. Hum. Percep. Perf. 26, 747–757. rons in the macaque temporal cortex. In AI and the eye (ed.
McGurk, H. & MacDonald, J. 1976 Hearing lips and seeing T. Troscianko & A. Blake), pp. 181–201. Chichester, UK:
voices. Nature 264, 746–748. Wiley.
McLeod, P., Dittrich, W., Driver, J., Perrett, D. I. & Zihl, Perrett, D. I., Harries, M., Chitty, A. J. & Mistlin, A. J. 1990b
J. 1996 Preserved and impaired detection of structure from Three stages in the classification of body movements by
motion in a ‘motion-blind’ patient. Vis. Cogn. 3, 363–391. visual neurones. In Images and understanding (ed. H. B.
RT0996_C09.qxd 11/8/04 2:04 PM Page 129
Barlow, C. Blakemore & M. Weston-Smith), pp. 94–108. general. Abstract no. 14221. Academic Press OHBM
Cambridge University Press. Annual Scientific Meeting 2002.
Perrett, D. I., Mistlin, A. J., Harries, M. H. & Chitty, A. J. Thornton, I. M., Pinto, J. & Shiffrar, M. 1998 The visual per-
1990c Understanding the visual appearance and conse- ception of human locomotion. Cogn. Neuropsychol. 15,
quence of hand actions. In Vision and action: the control of 535–552.
grasping (ed. M. A. Goodale), pp. 163–180. Norwood, NJ: Umilta, M. A., Kohler, E., Gallese, V., Fogassi, L., Fadiga,
Ablex Publishing. L., Keysers, C. & Rizzolatti, G. 2001 I know what you are
Perrett, D. I., Hietanen, J. K., Oram, M. W. & Benson, P. J. doing: a neurophysiological study. Neuron 31, 155–165.
1992 Organization and functions of cells responsive to faces Vaina, L. M., LeMay, M., Bienfang, D. C., Choi, A. Y. &
in the temporal cortex. Phil. Trans. R. Soc. Lond. B 335, Nakayama, K. 1990 Intact ‘biological motion’ and ‘struc-
23–30. ture from motion’ perception in a patient with impaired mo-
Puce, A. & Allison, T. 1999 Differential processing of mobile tion mechanisms: a case study. Vis. Neurosci. 5, 353–369.
and static faces by temporal cortex. Neuroimage 9(6), S801. Vaina, L. M., Solomon, J., Chowdhury, S., Sinha, P. &
Puce, A., Allison, T., Bentin, S., Gore, J. C. & McCarthy, Belliveau, J. W. 2001 Functional neuroanatomy of biologi-
G. 1998 Temporal cortex activation in humans viewing eye cal motion perception in humans. Proc. Natl Acad. Sci. USA
and mouth movements. J. Neurosci. 18, 2188–2199. 98, 11 656–11 661.
Puce, A., Smith, A. & Allison, T. 2000 ERPs evoked by view- Vuilleumier, P. 2002 Perceived gaze direction in faces and spa-
ing moving eyes and mouths. Cogn. Neuropsychol. 17, tial attention: a study in patients with parietal damage and
221–239. unilateral neglect. Neuropsychologia 40, 1013–1026.
Puce, A., Castiello, U., Syngeniotis, A. & Abbott, D. 2001 The Wachsmuth, E., Oram, M. W. & Perrett, D. I. 1994 Recogni-
human STS region integrates form and motion. Neuroimage tion of objects and their component parts: responses of sin-
13(6), S931. gle units in the temporal cortex of the macaque. Cerebr.
Rizzolatti, G., Camarda, R., Fogassi, L., Gentilucci, M., Luppino, Cortex 4, 509–522.
G. & Matelli, M. 1988 Functional organization of inferior Wheaton, K. J., Pipingas, A., Silberstein, R. & Puce, A. 2001
area 6 in the macaque monkey. II. Area F5 and the control of Neuronal responses elicited to viewing the actions of others.
distal movements. Exp. Brain Res. 71, 491–507. Vis. Neurosci. 18, 401–406.
Rizzolatti, G., Fadiga, L., Gallese, V. & Fogassi, L. 1996a Pre- Wheaton, K. J., Aranda, G. & Puce, A. 2002a ERPs elicited
motor cortex and the recognition of motor actions. Brain to combined emotional and gestural movements of the
Res. Cogn. Brain Res. 3, 131–141. face as a function of congruency. Abstract no. 14186. Aca-
Rizzolatti, G., Fadiga, L., Matelli, M., Bettinardi, V., Paulesu, demic Press OHBM Annual Scientific Meeting 2002.
E., Perani, D. & Fazio, F. 1996b Localization of grasp rep- Wheaton K. J., Aranda, G. & Puce, A. 2002b Affective modu-
resentations in humans by PET. 1. Observation versus exe- lation of gestural and visual speech stimuli: an ERP study.
cution. Exp. Brain Res. 111, 246–252. Abstract no. 14215. Academic Press OHBM Annual Scien-
Rizzolatti, G., Fogassi, L. & Gallese, V. 2001 Neurophysio- tific Meeting 2002.
logical mechanisms underlying the understanding and imi- Wheaton, K. J., Thompson, J. C., Berkovic, S. F., Jackson,
tation of action. Nature Rev. Neurosci. 2, 661–670. G. & Puce, A. 2002c Brain regions responsive to the per-
Sams, M., Aulanko, R., Hämäläinen, M., Hari, R., Lounasmaa, ception of human motion. The fMRI Experience IV Proc.
O. V., Lu, S. T. & Simola, J. 1991 Seeing speech: visual in- NIH, Maryland 2002, p. 103.
formation from lip movements modifies activity in the hu- Wicker, B., Michel, F., Henaff, M.-A. & Decety, J. 1998 Brain
man auditory cortex. Neurosci. Lett. 127, 141–145. regions involved in the perception of gaze: a PET study.
Schenk, T. & Zihl, J. 1997 Visual motion perception after brain Neuroimage 8, 221–227.
damage: II. Deficits in form-from-motion perception. Neu- Williams, J. H., Whiten, A., Suddendorf, T. & Perrett, D. I., 2001
ropsychologia 35, 1299–1310. Imitation, mirror neurons and autism. Neurosci. Biobehav.
Sirigu, A., Duhamel, J. R., Cohen, L., Pillon, B., Dubois, B. & Rev. 25, 287–295.
Agid, Y. 1996 The mental representation of hand movements
after parietal cortex damage. Science 273, 1564–1568. GLOSSARY
Taylor, M. J., Edmonds, G. E., McCarthy, G. & Allison,
T. 2001 Eyes first! Eye processing develops before face pro- ERP: event-related potential
cessing in children. Neuroreport 12, 1671–1676. fMRI: functional magnetic resonance imaging
Thompson, J. C., Wheaton, K., Berkovic, S. F., Jackson, G. & MST: medial superior temporal
Puce, A. 2002a Hemodynamic responses in humans to the MTG: mid-temporal gyrus
perception of compatible and incompatible body motion. In PET: positron emission tomography
The fMRI Experience IV Proc. NIH, Maryland, 2002, 93. STG: superior temporal gyrus
Thompson, J. C., Wheaton, K., Castiello, U. & Puce, A. 2002b STP: superior temporal polysensory
ERPs differentiate between facial motion and motion in STS: superior temporal sulcus
RT0996_C09.qxd 11/8/04 2:04 PM Page 130
RT0996_P06.qxd 11/8/04 2:14 PM Page 131
PA R T 6
Biological Movement: From Perception

to Imitation to Emotion
•
In 1909, one of the fathers of psychology, Edward Tichner, argued that

people could never know what another felt by reasoning, that they could
only know by feeling themselves into the other’s feelings. Since that time,
a voluminous literature in human and nonhuman animals has accrued
demonstrating mimicry and contagion effects—typically without any
conscious awareness or control by the individuals involved. In the
preceding section, we learned that the perception of biological movement
involved a distributed neural system separable from the system involved in
the perception of nonbiological movement, and we saw that the perception
of particularly powerful social cues in biological movement (e.g., changes in
gaze direction, articulatory movements of the mouth) produced especially
robust activation of related brain regions. In this section, evidence is
reviewed that illuminates the neural substrates of imitative and emotional
contagion effects and raises the notion of a mirror system—a system of
neurons that subserve an individual’s capacity to recognize actions made by
others and in so doing to mirror the observed actions.
The human premotor cortex, which is involved in voluntary movements
of the body, is organized somatotopically. Using functional magnetic
resonance imaging (fMRI), Buccino et al. (2001) localized areas of the brain
that were active during the observation of movement by another individual.
They found that regions of the premotor cortex were activated when
131
RT0996_P06.qxd 11/8/04 2:14 PM Page 132
individuals observed the actions of another and, The regions of the brain important for imitation
more specifically, that the areas activated in the are not the regions important in emotion, but instead
premotor cortex corresponded to the regions that there is substantial overlap in the brain regions
would be active were the individual to have executed important in imitation and observation. By
the observed actions. The findings of Buccino et al. mimicking the observed action, however, individuals
(2001) are in accord with the hypothesis that there are in a better position to know by feeling
is a brain circuit that extracts and neurologically themselves what another person is feeling. This
represents the motor commands of another reasoning implies a mechanism through which
individual’s observed actions—the so-called direct imitation produces emotional contagion. Evidence
matching hypothesis. Imitative actions of the sort for this reasoning is provided by Carr et al. (2003) in
investigated by Buccino et al. do not require the second reading in this section. Using fMRI, Carr
voluntary control, however. Evidence from other and colleagues found that the brain regions
researchers indicates that imitative reactions are important for action representation and imitation,
faster than simple visual reaction times, and that such as the superior temporal sulcus, are connected
people’s awareness of their own imitative reactions to the insula and amygdala—regions in the limbic
occur significantly later than their imitative reactions. lobe that are involved in emotions.
RT0996_C10.qxd 11/8/04 2:05 PM Page 133
R E A D I N G 1 0
Action Observation Activates Premotor and

Parietal Areas in a Somatotopic Manner:
An fMRI Study
•
G. Buccino, F. Binkofski,1 G. R. Fink,1,2 L. Fadiga, L. Fogassi,
V. Gallese, R. J. Seitz,1, K. Zilles,2 G. Rizzolatti and H.-J. Freund1
Functional magnetic resonance imaging (fMRI) was used to localize brain areas that were active
during the observation of actions made by another individual. Object- and non-object-related
actions made with different effectors (mouth, hand and foot) were presented. Observation of both
object- and non-object-related actions determined a somatotopically organized activation of
premotor cortex. The somatotopic pattern was similar to that of the classical motor cortex
homunculus. During the observation of object-related actions, an activation, also somatotopically
organized, was additionally found in the posterior parietal lobe. Thus, when individuals observe an
action, an internal replica of that action is automatically generated in their premotor cortex. In the
case of object-related actions, a further object-related analysis is performed in the parietal lobe,
as if the subjects were indeed using those objects. These results bring the previous concept of an
action observation/execution matching system (mirror system) into a broader perspective: this
system is not restricted to the ventral premotor cortex, but involves several somatotopically
organized motor circuits.
Keywords: action observation, humans, mirror system, parietal lobe, premotor cortex.
2
Istituto di Fisiologia Umana, Università di Parma, Via Institute of Medicine, Research Center Juelich GmbH,
Volturno 39, I-43100 Parma, Italy. Germany.
1
Department of Neurology, Heinrich Heine University of
Duesseldorf, 5 Moorenstrasse, D-40225, Duesseldorf, Germany.
133
RT0996_C10.qxd 11/8/04 2:05 PM Page 134
Introduction out the hypothesis that the activation of Broca’s

area, reported during hand action observation,
In the monkey premotor cortex (area F5) there are was due to verbalization. If the verbalization
neurons that discharge both when the monkey hypothesis were true, Broca’s area should be the
performs specific hand actions (e.g., grasping an major activation focus during action observation,
object) and when it observes another individual regardless of the effector used.
performing the same or a similar action (Gallese The second aim was to determine to what extent
et al., 1996; Rizzolatti et al., 1996a). The hypothe- the presence of an object influences the analysis of
sis was forwarded that these neurons, called ‘mir- an observed action. When an individual acts on an
ror neurons’, subserve the capacity of individuals object, a specific, pragmatic analysis of the object
to recognise actions made by others. is carried out in the parietal lobe. This analysis is
There is growing evidence that a ‘mirror’ sys- distinct from the semantic processing performed
tem, similar to that described in the monkey, also in the temporal lobe (Jeannerod, 1994; Milner &
exists in humans. Electrophysiological studies Goodale, 1995). Would the observation of object-
(Hari et al., 1998; Cochin et al., 1999) showed related actions evoke this pragmatic analysis? An
that when a human subject observes hand actions activation of pragmatic representations would be
there is a desynchronization of the motor cortex evidence, that during action observation, individu-
similar, although weaker, to that occurring during als internally ‘re-act’ the observed action in terms
active movements. In agreement with these find- of both action and the object acted upon.
ings, transcranial magnetic stimulation (TMS)
experiments showed that motor-evoked potentials
recorded from hand muscles increase during the Materials and Methods
observation of hand movements (Fadiga et al.,
Subjects
1995; Strafella & Paus, 2000).
Because the motor cortex of primates does not Twelve healthy, right-handed subjects, aged 25–
receive a significant visual input, its activation, dur- 38 years old, took part in the experiment. All sub-
ing observation of actions made by others, ought to jects (except two) were naive as to the purpose of
be mediated by the premotor areas that are con- the experiment. They all gave their written
nected with it. This conclusion has been supported consent to the experimental procedure. The study
by brain imaging studies showing that during ob- was approved by the Ethical Committee of the
servation of hand/arm actions there is an activation Heinrich Heine University, Duesseldorf.
of the ventral premotor cortex centred to the
Broca’s region (Grafton et al., 1996; Rizzolatti et al., MRI Scanner and Scanning Sequences
1996b; Decety et al., 1997; Grezes et al., 1998; Functional magnetic resonance imaging (fMRI)
Iacoboni et al., 1999). Considering, however, that measurements were performed on a 1.5 Tesla
Broca’s area is the cortical motor speech centre, the Siemens Vision scanning system using standard
possibility cannot be excluded that Broca’s area echo-planar imaging (EPI) and a standard radio
activation, during action observation, were due to frequency head coil for signal transmission and
an internal verbalization of the observed actions reception. Thirty consecutive slices orientated
rather than to a ‘mirror’ mechanism. parallel to the anterior-posterior commissure
The main aim of the present study was to assess plane and covering the whole brain were ac-
whether the observation of actions made with quired. The following EPI sequences were used:
different effectors would activate specific parts of repetition time, 5 s; signal-gathering time (echo-
the premotor cortex in accord with the somato- time), 66 ms; , 90º; voxel size, 334 mm.
topic motor organization of the region. This acti-
vation specificity, if proved, would show on one Experimental Protocol
side, that the mirror system is not limited to hand While being scanned, subjects were asked to
action and on the other, would allow one to rule carefully observe different videotaped object- and
RT0996_C10.qxd 11/8/04 2:05 PM Page 135
Action Observation Activates Premotor and Parietal Areas in a Somatotopic Manner ■ 135
non-object-related actions, performed by another significantly activated if they passed the highest
individual with different effectors (mouth, threshold of Z-score (3.09) and belonged to a
arm/hand and foot). These videotaped actions were cluster of at least 10 activated pixels (P 0.05,
presented on a screen, situated outside the scanner. corrected for multiple comparisons). The acti-
Subjects could see them through a mirror (1015 cm) vated pixels surviving this procedure were super-
which was positioned in the scanner in front of imposed on high-resolution magnetic resonance
them. Videotaped actions were presented in se- (MR) scans of a standard brain (Montreal Neuro-
quences 25 s long. During each sequence the same logical Institute, MNI). Clusters of activated foci
action was presented 3–4 times. Each sequence was were assigned to the regions of interest according
presented twice during the experimental session. to their centres of mass activity with the aid of
The observed actions were: biting an apple and Talairach coordinates (1988) and prominent sulcal
chewing (mouth actions); reaching and grasping a landmarks. Furthermore, as far as Broca’s region is
ball or a little cup with the hand and mimicking concerned, Talairach coordinates were also com-
these actions without the object (hand actions) or pared with the coordinates of cytoarchitectonically
kicking a ball or pushing a brake and mimicking defined probability maps (Amunts et al., 1999).
these actions without the object (foot actions). Ob-
servation of both object- and non-object-related
mouth, hand and foot actions (active condition) was Results
contrasted with the observation of a static face, a
static hand and a static foot, respectively, as a con- The results of the experiment are shown in Fig-
trol condition. Static stimuli were presented for 25 s ures 10.1–10.3. Frontal and parietal activations
continuously. At the end of the experimental ses- related to action observation are presented in
sion, subjects had to report the actions they were colour. Other activations (mostly occipital) are
presented with. All subjects reported them correctly. shown in grey. These latter activations (probably
due to stronger activation of visual areas with
moving stimuli) will not be discussed here.
Image Analysis Activations during mouth action observation
Image analysis was performed on a SPARC II are shown in Figure 10.1. During the observation
workstation (Sun Microsystems) using MATLAB of non-object-related mouth actions (chewing, a),
(Mathworks Inc., Natick, MA, USA) and statisti- activation foci were present in areas 6 and 44 on
cal parametric mapping package SPM97d (Fris- both sides and in area 45 in the right hemisphere.
ton et al., 1995, 1997). First, functional images of Right hemisphere activation was larger and
each condition were realigned to the tenth image stronger than left hemisphere activation. During
to correct for head movements between scans. the observation of object-related mouth actions
Then the images were coregistered and trans- (biting an apple, b), the pattern of premotor acti-
formed into a standard stereotactic space, using vation was similar, although weaker, to that found
the intercommissural line as the reference plane during non-object-related actions. In addition,
for transformation (Friston et al., 1997). Active two activation foci were present in the parietal
and control conditions were modelled using a lobe. These foci were larger in the left than in the
delayed box-car reference vector, accounting for right hemisphere. The rostral focus was located
the delayed cerebral blood flow change after stim- in area 40 (area PF of von Economo, 1929), the
ulus presentation. Significantly activated pixels caudal focus in area 39 (area PG).
were searched for by using the general linear Figure 10.2 shows activation foci relative to
model approach for time series proposed by observation of arm/hand actions. During the ob-
Friston et al. (1995). Group activation maps were servation of non-object-related hand actions
calculated by pooling the data for each condition (mimicking reaching to grasp, a) there was a bi-
across all subjects. Pixels were identified as lateral activation of area 6 that was located dorsal
RT0996_C10.qxd 11/8/04 2:05 PM Page 136
FIGURE 10.1 ■ (A color version of this figure follows page 146.) Observation
of mouth actions. Projections of the activation foci on the lateral surface of a stan-
dard brain [Montreal Neurological Institute (MNI)] during the observation of non-
object-related (chewing: a) and object-related (biting an apple: b) mouth actions.
FIGURE 10.2 ■ (A color version of this figure follows page 146.) Observation of
hand actions. Projections of the activation foci on the lateral surface of a standard brain
(MNI) during the observation of non-object-related (mimicking grasping of a cup or a
ball, without object: a) and object-related (grasping a cup or a ball: b) hand actions.
RT0996_C10.qxd 11/8/04 2:05 PM Page 137
FIGURE 10.3 ■ (A color version of this figure follows page 146.) Observation of
foot actions. Projections of the activation foci on the lateral surface of a standard
brain (MNI) during the observation of non-object-related (mimicking kicking a ball or
pushing a brake, without the object: a) and during the observation of object-related
foot actions (kicking a ball or pushing a brake: b) foot actions.
to that found during mouth movement observa- activation of the frontal lobe (rostrally located).
tion. During the observation of object-related Because we have no explanation for this activa-
arm/hand actions (reaching-to-grasp movements, b) tion (found only in this condition), we will not
there was a bilateral activation of premotor cortex comment on it further. During the observation of
plus an activation site in area 44. Most interest- object-related foot actions (kicking a ball or push-
ingly, as in the case of the observation of mouth ing a brake, b), there was, as in the previous con-
movements, two activation foci were present in dition, an activation of a dorsal sector of area 6.
the parietal lobe. The rostral one was located in- In addition, there was an activation of the posterior
side the intraparietal sulcus, in an area caudal and part of the parietal lobe. The parietal activation
dorsal to that found in the mouth movement ob- was in part located in Brodmann’s area 7 [(PE)],
servation condition. This area probably corre- in part it overlapped the activation seen during
sponds to the anterior intraparietal area of the mouth and hand actions (Brodmann’s area
monkey. The caudal focus was in area 39 (area 39/PG).
PG). This last focus considerably overlapped that Figure 10.4 gives a global picture of the activa-
found during mouth movement observation. tions found during observation of mouth, hand
Figure 10.3 shows activation foci elicited by ob- and foot actions. It is evident that both the premo-
servation of foot actions. During the observation tor cortex and the parietal lobe activation foci are
of non-object-related foot actions (mimicking ball somatotopically organized. The premotor somato-
kicking or brake pushing, a), there was an activa- topy follows a pattern similar to that of the classi-
tion of a dorsal sector of area 6. There also was an cal motor homunculus (Penfield & Rasmussen,
RT0996_C10.qxd 11/8/04 2:05 PM Page 138
FIGURE 10.4 ■ (A color version of this figure follows page 146.) Somatotopy of
premotor and parietal cortices as revealed by action observation. (a) Observation
of non-object-related actions. (b) Observation of object-related actions. Activation
foci, shown in detail in the three previous figures, are projected on the lateral sur-
face of a standard brain (MNI). Red, activation during the observation of mouth
movements; green, activation during the observation of hand movements; blue,
activation during the observation of foot movements. Overlap of colours indicates
activation foci present during observation of actions made by different effectors.
1952). In the parietal lobe, the mouth is repre- 6 plus a dorsal sector of area 44 are recruited in
sented rostrally while the foot is located caudally. both hemispheres. Finally, the observation of foot
Table 10.1 shows the Talairach coordinates and actions elicits an activation of a dorsal sector of
Z scores of the activated foci during the observa- area 6, bilaterally. There is, therefore, a clear
tion of object- and non-object-related mouth, topographic shift in the premotor cortex activation
hand, and foot actions. from ventral to dorsal when the effector used in
the observed action moves from mouth to
Discussion arm/hand and to foot, respectively. This shift is
congruent with the classical motor organization of
The results of the present experiment show that the region (see Penfield & Rasmussen, 1952).
when an individual observes actions (made by These results are important for two reasons. First,
another individual) performed with different ef- the effector related somatotopic activation pattern in
fectors, different sectors of the premotor cortex the premotor cortex during the mere observation of
are activated. During mouth actions, there is a actions proves that, in humans, the mirror system is
bilateral activation of ventral area 6 and area not restricted to hand actions, but includes a rich
44 plus an activation of the right area 45. During repertoire of body actions. It therefore constitutes
hand actions, a more dorsal part of ventral area the neural substrate for a matching mechanism
RT0996_C10.qxd 11/8/04 2:05 PM Page 139
TABLE 10.1. Talairach Coordinates and Z-Scores of the Activated Foci During Observation of Object- and
Non-Object-Related Mouth, Hand, and Foot Actions
Activated foci during the observation of
Object-related actions Non-object-related actions
Actions/Brain area x y z Z-score x y z Z-score
Mouth actions
Brodmann’s area 6
R 48 0 32 4.38 52 0 32 3.28
L 56 0 36 4.18 52 4 44 3.55
Brodmann’s area 44
R 60 8 24 3.76 56 12 16 3.79
L 64 12 20 3.01 60 16 16 3.07
R 60 16 20 4.13 60 28 20 4.31
Inferior parietal lobule
R 52 24 20 3.31
52 32 44 3.39
L 36 52 56 5.16
60 24 36 4.21
Arm/hand actions
Brodmann’s area 6
R 48 0 44 4.66 52 0 48 3.64
L 56 4 44 5.84 60 4 40 3.72
R 56 12 12 3.01
L 64 4 24 3.72
Anterior intraparietal area
R 40 40 52 4.55
L 36 40 52 4.63
Foot actions
Brodmann’s area 6
R 40 4 60 3.38 44 4 56 3.93
L 32 8 64 3.30 40 4 60 4.05
Superior parietal lobule
R 24 60 68 5.69
L 32 64 60 5.05
R,right hemisphere; L, left hemisphere; x, y, z, Talairach coordinates.
mapping the observed actions on the observer’s of the rostral part of the inferior parietal lobule
motor representations. Second, these results defin- (area 40). During the observation of hand actions,
itively rule out the interpretation that cortical a more posterior sector of area 40, inside the in-
activation during action observation is due to traparietal sulcus, becomes active. This sector
verbalization. closely corresponds to that shown to be active
A further important result of the present exper- during object manipulation (Binkofski et al.,
iment is the demonstration of a marked difference 1999). It has been suggested that this sector is the
between the activation during the observation of human homologue of monkey anterior intrapari-
object-related and non-object-related actions. etal area. The observation of foot actions activates
Any time an object is the target of an action, the predominantly the posterior part of the superior
parietal lobe is strongly activated. This object- parietal lobule. Finally, in all conditions there is
related activation is also somatotopically organized activation of area 39 (area PG).
and depends on the effector used. During the ob- Although the motor organization of the parietal
servation of mouth actions, there is an activation lobe is not fully established, an organization
RT0996_C10.qxd 11/8/04 2:05 PM Page 140
similar to that described here for action observa- Abbreviations

tion in humans can be recognized for active move-
ments in nonhuman primates. In the monkey, EPI, echo planar imaging; fMRI, functional
mouth movements are represented in the rostral magnetic resonance imaging; MNI, Montreal
part of PF (Leinonen & Nyman, 1979; Fogassi Neurological Institute; TE, signal- (echo-) gather-
et al., 1998), distal hand movements in the ante- ing time; TR, sequence repetition time.
rior intraparietal area (Sakata et al., 1995) and
arm reaching movements, posteriorly on the me- REFERENCES
dial bank of the intraparietal sulcus (Colby &
Amunts, K., Schleicher, A., Buergel, U., Mohlberg,
Duhamel, 1991; Snyder et al., 1997). Furthermore, H., Uylings, H.B.M. & Zilles, K. (1999) Broca’s region
clinical and brain imaging studies strongly re-visited: cytoarchitecture and intersubject variability.
suggest that a segregated pattern of effector repre- J. Comp. Neurol., 412, 319–341.
sentations in the parietal lobe is also present in Binkofski, F., Buccino, G., Posse, S., Seitz, R.J., Rizzolatti,
humans (De Renzi, 1982; Jeannerod, 1986; Pause G. & Freund, H.-J. (1999) A fronto-parietal circuit for
object manipulation in man: evidence from an fMRI study.
et al., 1989; Seitz et al., 1991). Eur. J. Neurosci., 11, 3276–3286.
It is generally accepted that a fundamental role of Cochin, S., Barthelemy, C., Roux, S. & Martineau, J. (1999)
the parietal lobe is to describe objects for action Observation and execution of movement: similarities
(Jeannerod, 1994; Jeannerod et al., 1995; Milner & demonstrated by quantified electroencephalography. Eur.
Goodale, 1995). This ‘pragmatic’, action-orientated J. Neurosci., 11, 1839–1842.
Colby, C.L. & Duhamel, J.R. (1991) Heterogeneity of extras-
object description has been contrasted with the ‘se- triate visual areas and multiple parietal areas in the macaque
mantic’ description coded in the infero-temporal monkeys. Neuropsychologia, 29, 517–537.
lobe (Milner & Goodale, 1995; Jeannerod et al., De Renzi, E. (1982) Disorders of Space Exploration and
1995). The results of the present experiment indi- Cognition. Wiley, New York.
cate that a ‘pragmatic’ analysis is also carried out Decety, J., Grezes, J., Costes, N., Perani, D., Jeannerod,
M., Procyk, E., Grassi, F. & Fazio, F. (1997) Brain activity
when an individual observes an object-directed during observation of actions. Influence of action content
action made by another individual. If action under- and subject’s strategy. Brain, 120, 1763–1777.
standing were based on higher cognitive functions, von Economo, C., (1929) The Cytoarchitectonics of the
this parietal analysis would be unnecessary. Human Cerebral Cortex. Oxford University Press, London.
Taken together, the results of the present exper- Fadiga, L., Fogassi, L., Pavesi, G. & Rizzolatti, G. (1995)
Motor facilitation during action observation: a magnetic
iment strongly support the view that during action stimulation study. J. Neurophysiol., 73, 2608–2611.
observation there is a recruitment of the same Fogassi, L., Gallese, V., Fadiga, L. & Rizzolatti, G. (1998)
neural structures which would be normally Neurons responding to the sight of goal-directed hand/arm
involved in the actual execution of the observed actions in the parietal area PF (7b) of the macaque monkey.
action. When individuals observe an action, they Soc. Neurosci. Abstr., 24, 154.
Friston, K.J., Ashburner, J., Poline, J.B., Firth, C.D., Heather,
code that action in terms of the related voluntary J.D. & Frackowiak, R.S.J. (1997) Spatial realignment and
movements. The ‘seen’ actions are mapped onto normalization of images. Hum. Brain Mapp., 2, 165–189.
the corresponding motor representations of the Friston, K.J., Holmes, A.P., Poline, J.B., Grasby, P.J., Williams,
frontal lobe and, in the case of object-related S.C.R., Frackowiak, R.S.J. & Turner, R. (1995) Analysis of
actions, the ‘seen’ objects on the effector-related, fMRI time-series revisited. Neuroimage, 2, 5–53.
Gallese, V., Fadiga, L., Fogassi, L. & Rizzolatti, G. (1996)
pragmatic representations, in the parietal lobe. Action recognition in the premotor cortex. Brain, 119,
593–609.
Grafton, S.T., Arbib, M.A., Fadiga, L. & Rizzolatti, G. (1996)
Localization of grasp representations in humans by PET: 2.
Acknowledgments Observation compared with imagination. Exp. Brain Res.,
112, 103–111.
This study was supported by the Deutsche Grezes, J., Costes, N. & Decety, J. (1998) Top-down effect of
Forschungsgemeinschaft, MURST and by Human strategy on the perception of human biological motion: a
Frontier Science Program. PET investigation. Cogn. Neuropsychol., 15, 553–582.
RT0996_C10.qxd 11/8/04 2:05 PM Page 141
Hari, R., Forss, N., Avikainen, S., Kirveskari, E., Salenius, Penfield, W. & Rasmussen, T. (1952) The Cerebral Cortex of
S. & Rizzolatti, G. (1998) Activation of human primary mo- Man. MacMillan. New York.
tor cortex during action observation: a neuromagnetic study. Rizzolatti, G., Fadiga, L., Gallese, V. & Fogassi, L. (1996a)
Proc. Natl Acad. Sci. USA, 95, 15061–15065. Premotor cortex and the recognition of motor actions.
Iacoboni, M., Woods, R.P., Brass, M., Bekkering, H., Mazziotta, Cognitive Brain Res., 3, 131–141.
J.C. & Rizzolatti, G. (1999) Cortical mechanisms of human Rizzolatti, G., Fadiga, L., Matelli, M., Bettinardi, V., Paulesu,
imitation. Science, 286, 2526–2528. E., Perani, D. & Fazio, F. (1996b) Localization of grasp
Jeannerod, M. (1986) The formation of finger grip during representations in humans by PET: 1. Observation versus
prehension. A cortically mediated visuomotor pattern. execution. Exp. Brain Res., 111, 246–252.
Behav. Brain Res., 19, 99–116. Sakata, H., Taira, M., Murata, A. & Mine, S. (1995) Neural
Jeannerod, M. (1994) The representing brain: neural correlates mechanisms of visual guidance of hand action in the
of motor intention and imagery. Behav. Brain Sci., 17, parietal cortex of the monkey. Cerebral Cortex, 5, 429–438.
187–245. Seitz, R.J., Roland, P.E., Bohm, C., Greitz, T. & Stone-
Jeannerod, M., Arbib, M.A., Rizzolatti, G. & Sakata, H. (1995) Elander, S. (1991) Somatosensory discrimination of shape:
Grasping objects: the cortical mechanisms of visuomotor tactile exploration and cerebral activation. Eur. J. Neurosci.,
transformation. Trends Neurosci., 18, 314–320. 3, 481–492.
Leinonen, L. & Nyman, G. (1979) Functional properties of Snyder, L.H., Batista, A.P. & Andersen, R.A. (1997) Coding of
cells in anterolateral part of area 7 associative face area of intention in the posterior parietal cortex. Nature, 386,
awake monkeys. Exp. Brain Res., 34, 321–333. 167–169.
Milner, A.D. & Goodale, M.A. (1995) The Visual Brain in Strafella, A.P. & Paus, T. (2000) Modulation of cortical
Action. Oxford University Press, Oxford. excitability during action observation: a transcranial magnetic
Pause, M., Kunesch, E., Binkofski, F. & Freund, H.-J. (1989) stimulation study. Exp. Brain Res., 11, 2289–2292.
Sensorimotor disturbances in patients with lesions of the Talairach, J. & Tournoux, P. (1988) Co-Planar Stereotactic Atlas
parietal cortex. Brain, 112, 1599–1625. of the Human Brain. Thieme Medical Publishers, New York.
RT0996_C10.qxd 11/8/04 2:05 PM Page 142
RT0996_C11.qxd 11/9/04 4:12 PM Page 143
R E A D I N G 1 1
Neural Mechanisms of Empathy in Humans:

A Relay from Neural Systems for Imitation to
Limbic Areas
•
Laurie Carr†, Marco Iacoboni†‡§, Marie-Charlotte Dubeau†,
John C. Mazziotta†§ **††, and Gian Luigi Lenzi‡‡
How do we empathize with others? A mechanism according to which action representation

modulates emotional activity may provide an essential functional architecture for empathy. The
superior temporal and inferior frontal cortices are critical areas for action representation and are
connected to the limbic system via the insula. Thus, the insula may be a critical relay from action
representation to emotion. We used functional MRI while subjects were either imitating or simply
observing emotional facial expressions. Imitation and observation of emotions activated a largely
similar network of brain areas. Within this network, there was greater activity during imitation,
compared with observation of emotions, in premotor areas including the inferior frontal cortex, as
well as in the superior temporal cortex, insula, and amygdala. We understand what others feel by a
mechanism of action representation that allows empathy and modulates our emotional content.
The insula plays a fundamental role in this mechanism.
Egoalsmpathy plays a fundamental social role,

allowing the sharing of experiences, needs, and
across individuals. Its functional aspects
and corresponding neural mechanisms, however,
are poorly understood. When Theodore Lipps (as
cited in ref. 1) introduced the concept of empathy
†Ahmanson-Lovelace Brain Mapping Center, Neuropsychiatric of Medicine, University of California, Los Angeles, CA 90095;
Institute, Departments of ‡Psychiatry and Biobehavioral and ‡‡Department of Neurological Sciences, University
Sciences, Neurology, **Pharmacology, and ††Radiological “La Sapienza,” Rome, Italy 00185.
Sciences, and §Brain Research Institute, David Geffen School
143
RT0996_C11.qxd 11/9/04 4:12 PM Page 144
(Einfühlung), he theorized the critical role of action and the predicted sensory consequences of
inner imitation of the actions of others in generat- the planned imitative action can occur. (v) Once
ing empathy. In keeping with this concept, em- the visual description of the observed action and
pathic individuals exhibit nonconscious mimicry the predicted sensory consequences of the planned
of the postures, mannerisms, and facial expres- imitative action are matched, imitation can be
sions of others (the chameleon effect) to a greater initiated.
extent than nonempathic individuals (2). Thus, em- How is this moderately recursive circuit con-
pathy may occur via a mechanism of action repre- nected to the limbic system? Anatomical data
sentation that modulates and shapes emotional suggest that a sector of the insular lobe, the
contents. dysgranular field, is connected with the limbic
In the primate brain, relatively well-defined and system as well as with posterior parietal, inferior
separate neural systems are associated with frontal, and superior temporal cortex (23). This
emotions (3) and action representation (4–7). The connectivity pattern makes the insula a plausible
limbic system is critical for emotional processing candidate for relaying action representation in-
and behavior, and the circuit of frontoparietal formation to limbic areas processing emotional
networks interacting with the superior temporal content.
cortex is critical for action representation. This lat- To test this model, we used functional MRI
ter circuit is composed of inferior frontal and pos- (fMRI) while subjects were either observing or
terior parietal neurons that discharge during the imitating emotional facial expressions. The pre-
execution and also the observation of an action dictions were straightforward: If action represen-
(mirror neurons; ref. 7), and of superior temporal tation mediation is critical to empathy and the
neurons that discharge only during the observation understanding of the emotions of others, then even
of an action (6, 8, 9). Anatomical and neurophysi- the mere observation of emotional facial expres-
ological data in the nonhuman primate brain (see sion should activate the same brain regions of
review in ref. 7) and imaging human data (10–13) motor significance that are activated during the
suggest that this circuit is critical for imitation and imitation of the emotional face expressions. More-
that within this circuit, information processing over, a modulation of the action representation cir-
would flow as follows. (i) The superior temporal cuit onto limbic areas via the insula predicts
cortex codes an early visual description of the ac- greater activity during imitation, compared with
tion (6, 8, 9) and sends this information to poste- observation of emotion, throughout the whole net-
rior parietal mirror neurons (this privileged flow of work outlined above. In fact, mirror areas would
information from superior temporal to posterior be more active during imitation than observation
parietal is supported by the robust anatomical con- because of the simultaneous encoding of sensory
nections between superior temporal and posterior input and planning of motor output (13). Within
parietal cortex) (14). (ii) The posterior parietal cor- mirror areas, the inferior frontal cortex seems par-
tex codes the precise kinesthetic aspect of the ticularly important here, given that understanding
movement (15–18) and sends this information goals is an important component of empathy. The
to inferior frontal mirror neurons (anatomical superior temporal cortex would be more active
connections between these two regions are well during imitation than observation, as it receives
documented in the monkey) (19). (iii) The inferior efferent copies of motor commands from mirror
frontal cortex codes the goal of the action [both neu- areas (12). The insula would be more active during
rophysiological (5, 20, 21) and imaging data (22) imitation because its relay role would become
support this role for inferior frontal mirror neurons]. more important during imitation, compared with
(iv) Efferent copies of motor plans are sent from mere observation. Finally, limbic areas would also
parietal and frontal mirror areas back to the superior increase their activity because of the modulatory
temporal cortex (12), such that a matching mecha- role of motor areas with increased activity. Thus,
nism between the visual description of the observed observation and imitation of emotions should
RT0996_C11.qxd 11/9/04 4:12 PM Page 145
Neural Mechanisms of Empathy in Humans ■ 145
yield substantially similar patterns of activated mouth, but not during imitation of eye emotional
brain areas, with greater activity during imitation expressions. However, our imaging data (see
in premotor areas, in inferior frontal cortex, in below) do not support such dissociation.
superior temporal cortex, insula, and limbic areas.
Behavioral Tasks
Subjects were presented three runs of stimuli.
Materials and Methods One run consisted of six blocks of 24 s each. Each
block contained six pictures (of the six emotion
Subjects types), and each picture was presented for 4 s.
Eleven healthy, right-handed subjects participated Blocks were homogenous for stimulus type
in the experiment (seven males and four females). (i.e., all faces, or all eyes, or all mouths). Subjects
The mean age of the subject group was 29, and were asked to imitate and internally generate the
ranged from 21 to 39 years. All subjects were eval- target emotion on the computer screen, or to sim-
uated with a brief neurological examination and ply observe. Imitate/observe conditions were
questionnaire to screen for any medical/behavioral counterbalanced across runs, and task blocks
disorders. Handedness was evaluated by using a were separated by seven rest periods of 24 s
modified version of the Edinburgh Handedness (blank screen). The first rest period actually lasted
Inventory (24). The study was approved by the 36 s, the additional 12 s being related to the first
University of California at Los Angeles Institu- three brain volumes, which were discarded from
tional Review Board and was performed in ac- the analysis due to signal instabilities.
cordance with the ethical standards laid down
in the 1964 Declaration of Helsinki. Written in- Imaging
formed consent was obtained from all subjects Structural and fMRI measurements were per-
before inclusion in the study. formed on a General Electric 3.0 Tesla MRI
scanner with advanced nuclear magnetic reso-
Stimuli nance echo-planar imaging (EPI) up-grade
Stimuli were presented to subjects through mag- located in the Ahmanson-Lovelace Brain Mapping
net-compatible goggles. Using as stimulus a Center. Structural and functional scanning se-
widely known set of facial expressions (25), three quences performed in each subject included: one
stimulus picture sets were assembled, each con- structural scan (coplanar high-resolution EPI
taining randomly ordered depictions of six emo- volumes: repetition time (TR) 4,000 ms; echo
tions (happy, sad, angry, surprise, disgust, and time (TE) 54 ms; flip angle 90º; 128 128
afraid). Of the three stimulus sets, one contained matrix; 26 axial slices; 3.125-mm in-plane resolu-
whole faces, and the other two sets contained only tion; 4-mm thickness; skip 1 mm) for anatomical
eyes or only mouths, which were cropped from data, and three functional scans (echo planar T2*-
the same set of whole faces. All pictures, whether weighted gradient echo sequence; TR = 4,000 ms;
whole face, only mouth, or only eyes, consisted of TE 25 ms; flip angle 90º; 64 64 matrix;
different individuals, with males and females in 26 axial slices; 3.125-mm in-plane resolution;
equal proportion. The rationale for showing only 4-mm thickness; skip 1 mm). Each of the func-
parts of faces was suggested by the cortical repre- tional acquisitions covered the whole brain.
sentation of body parts in inferior frontal cortex, Individual subjects’ functional images were
where the mouth is represented but the eyes are aligned and registered to their respective coplanar
not (26). In principle, if eye emotional expres- structural images by using a rigid body linear
sions can be dissociated from the emotional ex- registration algorithm (27). Intersubject image
pression of the rest of the face, one might see the registration was performed with fifth-order
predicted pattern of activity in inferior frontal cor- polynomial nonlinear warping (28) of each sub-
tex during imitation of the whole face or of the ject’s images into a Talairach-compatible brain
RT0996_C11.qxd 11/9/04 4:12 PM Page 146
magnetic resonance atlas (29). Data were with previously published peaks in meta-analyses
smoothed by using an in-plane Gaussian filter for or individual studies in regions relevant to the
a final image resolution of 8.7 8.7 8.6 mm. hypothesis tested in this study.
Figures 11.1 and 11.2 show, respectively, the
Image Statistics location and time-series of the right primary mo-
Image statistics was performed with analyses tor face area and of the premotor face area. The
of variance (ANOVAs), allowing to factor out peaks of these activations correspond well with
run-to-run variability within subjects as well as published data, as discussed below. Task-related
intersubject overall signal variability (12, 13, activity is seen not only during imitation, but also
30–33). Given the time-course of the blood during observation. This observation-related ac-
oxygen level-dependent (BOLD) fMRI re- tivity is very clear in premotor cortex but also vis-
sponse, which takes several seconds to return to ible in primary motor cortex (although not reach-
baseline (34), contiguous brain volumes cannot ing significance in primary motor cortex).
be considered independent observations (35, Figure 11.3 shows the activations in inferior
36). Thus, the sum of signal intensity at each frontal cortex and anterior insula, with their corre-
voxel throughout each task was used as the de- sponding time-series. The activity of these three
pendent variable (12, 13, 22). Significance level regions is evidently correlated.
was set at P 0.001 uncorrected at each voxel. Figure 11.4 shows the significantly increased
To avoid false positives, only clusters bigger activity in the right amygdala during imitation,
than 20 significantly activated voxels were con- compared with observation of emotional facial
sidered (37). Factors included in the ANOVAs expressions
were subjects (n 11), functional scans (n
3), state (n 2, task/rest), task (n 2, imita- Discussion
tion/observation), and stimuli (n 3, whole
face, eyes, and mouth). The results of this study support our hypothesis
on the role of action representation for under-
standing the emotions of others. Largely overlap-
Results ping networks were activated by both observation
and imitation of facial emotional expressions.
Preliminary ANOVAs revealed no differences in Moreover, the observation of emotional expres-
activation among the three imitation tasks, and sions robustly activated premotor areas. Further,
no differences in activations among the three ob- fronto-temporal areas relevant to action represen-
servation tasks. Thus, main effects of imitation, tation, the amygdala, and the anterior insula had
observation, and imitation minus observation are significant signal increase during imitation com-
reported here. As Table 11.1 shows, there was a pared with observation of facial emotional
substantially similar network of activated areas expression.
for both imitation and observation of emotion. The peak of activation reported here in primary
Among the areas commonly activated by imita- motor cortex during imitation of facial emotional
tion and observation of facial emotional expres- expressions corresponds well with the location of
sions, the premotor face area, the dorsal sector the primary motor mouth area as determined by a
of pars opercularis of the inferior frontal gyrus, meta-analysis of published positron-emission to-
the superior temporal sulcus, the insula, and the mography (PET) studies, by a meta-analysis of
amygdala had greater activity during imitation original data in 30 subjects studied with PET, and
than observation of emotion. To give a sense of by a consensus probabilistic description of the lo-
the good overlap between the network described cation of the primary motor mouth area obtained
in this study and previously reported peaks of ac- merging the results of the two previously de-
tivation, Table 11.2 compares peak of activations scribed meta-analyses (38). This convergence
in the right hemisphere observed in this study confirms the robustness and reliability of the
RT0996_C11.qxd 11/9/04 4:12 PM Page 147
TABLE 11.1. Peaks of Activation in Talairach Coordinates

Talairach coordinates t values
Hemisphere Region BA x y z Imitation Observation Imi-obs
L M1 4 52 4 32 9.63 NS 7.44

R M1 4 44 10 36 10.93 NS 7.3
L S1 2 40 38 40 4.42 NS NS
R S1 3 56 22 36 4.14 NS 4.74
L PPC 7 24 60 40 4.42 5.77 NS
L PPC 40 40 46 50 4.51 3.72 NS
R PPC 39 30 54 38 4.98 5.3 NS
L PMC 6 30 2 50 6.14 NS 4.93
L PMC 6 40 2 32 10.98 3.91 4.84
L PMC 6 52 10 26 10.28 3.81 5.63
R PMC 6 48 8 28 11.4 6.14 6.7
R PMC 6 40 6 30 10.23 5.86 4.84
L Pre-SMA 6 8 6 58 8.84 NS 8.05
R Pre-SMA 6 0 4 52 9.26 NS 8.42
L RCZp 32 4 14 44 7.72 NS 6.74
L RCZa 32 8 30 26 4.98 NS 3.54
R ACC 32 8 16 52 NS 4.28 NS
R MPFC 9 6 54 34 NS 4.7 NS
L LPFC 10 36 50 12 11.86 NS 11.26
R LPFC 10 44 38 4 7.81 4.65 6.19
R LPFC 10 34 42 6 8.61 NS 6.37
L IFG 44 40 14 24 7.58 6.56 3.53
R IFG 44 50 14 16 9.16 4.74 6.51
L IFG 44 50 12 2 7.02 NS 9.26
R IFG 44 50 12 2 8.98 NS 9.91
L IFG 45 46 36 12 NS 5.67 NS
R IFG 45 46 26 8 8.14 3.4 5.4
L Insula 45 36 18 4 2.6 NS 4.65
R Insula 45 36 30 6 7.91 3.02 6.09
L STS 22 46 48 12 4.33 2.37 3.53
R STS 22 46 44 12 NS 3.72 NS
R FFA 19 39 64 14 11.86 11.86 NS
L Temp. pole 38 24 20 32 4.74 NS 5.49
R Temp. pole 38 36 26 28 8.47 NS 8.09
L Temp. pole 38 42 20 20 NS 4.28 NS
R Temp. pole 38 26 8 26 NS 3.95 NS
R Striatum 24 8 6 4.88 NS 3.67
L Amygdala 22 0 16 NS 3.91 NS
R Amygdala 22 0 10 4.7 3.16 3.77
In bold are peaks that follow the pattern predicted by the hypothesis of action representation route to empathy (i.e., activation during
imitation and observation, and greater activity during imitation compared to observation). The majority of these peaks were predicted
a priori, the only exception being the right dorsolateral prefrontal cortex, BA10. In italics are statistical levels approaching significance
in predicted areas. NS, not significant.
TABLE 11.2. Comparison of Observed Peaks of Activation

in Predicted Regions with Previously Reported Peaks of findings, despite the presence of facial motion
Activation in Imaging Meta-Analyses and Individual during imitation. In fact, residual motion artifacts
Studies of Action Observation, Imitation, and Emotion that were still present at individual level after mo-
Region x y z Ref. x y z tion correction were eliminated by the group
analysis. This result is likely due to the fact that
M1 44 10 36 38 48 9 35 each subject had different kinds of motion arti-
PMC 48 8 28 40 48 0 32
IFG 50 14 16 22 57 14 12 facts and, when all of the data were considered,
Insula 36 30 6 60 35 31 9 only common patterns of activity emerged.
STS 46 48 12 50 49 50 9 The data also clearly show peaks of activity in
Amygdala 22 0 10 59 24 2 22
the presupplementary motor area (pre-SMA) face
RT0996_C11.qxd 11/9/04 4:12 PM Page 148
FIGURE 11.1 ■ (A color version of this figure follows page 146.) Peaks of activations in the right central (labeled M1)
and precentral (labeled PMC) sulcus. The peak labeled M1 (x 44, y 10, z 36) corresponds entirely (considering
spatial resolution and variability factors) with meta-analytic PET data (x = 48 5.2, y 9 5.6, z 35 5.5) for
the mouth region of human primary motor cortex. The peak labeled PMC (x 48, y 8, z 28) corresponds well with
previously reported premotor mouth (x 48, y 0, z 32) peaks.
a) b)
FIGURE 11.2 ■ Time-series of peaks of activity in right central (M1) and precentral (PMC) sulcus shown in Figure 11.1.
Task-related activity is observable not only during imitation but also during observation of emotional facial expressions,
especially in PMC.
RT0996_C11.qxd 11/9/04 4:12 PM Page 149
1.6
1.2
0.8
0.4
–0.4
–0.8
Imitation Observation
FIGURE 11.3 ■ (A color version of this figure follows page 146.) Activations in the right insula (green) and right
(blue) and left (red) inferior frontal cortex. Relative time-series are coded with the corresponding colors. The time-series
have been normalized to the overall activity of each region. The activity profile of these three regions is extremely sim-
ilar throughout the whole series of tasks.
area and the face area of the posterior portion of

the rostral cingulate zone (RCZp) that correspond
well with the pre-SMA and RCZp face locations
as determined by a separate meta-analysis of
PET studies focusing on motor areas in the medial
wall of the frontal lobe (39). Thus, our dataset
represents an fMRI demonstration of human pri-
mary motor and rostral cingulate face area. With
regard to premotor regions, the peaks that we ob-
serve correspond well with premotor mouth peaks
described by action observation studies (40). As
Figure 11.2 shows, robust pre-motor responses
during observation of facial emotional expres-
sions were observed, in line with the hypothesis
that action representation mediates the recog-
nition of emotions in others even during simple
observation.
The activity in pars opercularis shows two
separate foci during imitation, a ventral and a
FIGURE 11.4 ■ (A color version of this figure follows
page 146.) Significantly increased activity in the right
dorsal peak. Only the dorsal peak remained acti-
amygdala during imitation of emotional facial expres- vated, although at significantly lower intensity,
sions compared with simple observation. during observation of emotion (Table 11.1).
RT0996_C11.qxd 11/9/04 4:12 PM Page 150
This pattern, with very similar peaks of activa- representation. This finding confirms a strong
tion, was also observed in a recent fMRI meta- input onto the anterior insular sector from areas
analysis comprising more than 50 subjects per- of motor significance.
forming hand action imitation and observation The increased activity in the amygdala during
in our lab.* Pars opercularis maps probabilisti- imitation compared with observation of emo-
cally onto Brodmann area 44 (41, 42), which is tional facial expression (Figure 11.4) reflects the
considered the human homologue of monkey modulation of the action representation circuit
area F5 (43–46) in which mirror neurons were onto limbic activity. It has been long hypoth-
described. In the monkey, F5 neurons coding esized (dating back to Darwin; refs. 53–55)
arm and mouth movements are not spatially seg- that facial muscular activity influences people’s
regated, and the human imaging data are consis- affective responses. We demonstrate here that
tent with this observation. The imaging data activity in the amygdala, a critical structure in
suggest that the dorsal sector represents the emotional behaviors and in the recognition of
mirror sector of pars opercularis, whereas the facial emotional expressions of others (56–59),
ventral sector may be simply a premotor area for increases while subjects imitate the facial emo-
hand and face movements. tional expressions of others, compared with mere
The superior temporal sulcus (STS) area shows observation.
greater activity for imitation than for observation Previous and current literature on observing
of emotional facial expressions, as predicted by and processing facial emotional expression pro-
the action representation mediation to empathy vides a rich context in which to consider the
hypothesis. This area also corresponds anatomi- nature of the empathic resonance induced by
cally well with an STS area specifically respond- our imitation paradigm. In general, our findings
ing to the observation of mouth movements fit well with previously published imaging data
observed in different studies from different labs on observation of facial expressions that report
(47–50). activation in both amygdala and anterior insula
The anterior sector of the insula was active for emotional facial expressions (for a review,
during both imitation and observation of see ref. 57 and references therein). A study on
emotion, but more so during imitation (Fig- conscious and unconscious processing of emo-
ure 11.3), fulfilling one of the predictions of tional facial expression (58) has suggested that
our hypothesis that action representation is a the left but not the right amygdala is associated
cognitive step toward empathy. This finding is with explicit representational content of the ob-
in line with two kinds of evidence available on served emotion. Our data, showing a right later-
this sector of the insular lobe. First, the anterior alized activation of the amygdala during imita-
insula receives slow-conducting unmyelinated tion of facial emotional expression, suggest that
fibers that respond to light, caress-like touch the type of empathic resonance induced by
and may be important for emotional and affil- imitation does not require explicit representa-
iative behavior between individuals (51). tional content and may be a form of “mirroring”
Second, imaging data suggest that the anterior that grounds empathy via an experiential
insular sector is important for the monitoring of mechanism.
agency (52), that is, the sense of ownership of In this study, we treated emotion as a single,
actions, which is a fundamental aspect of action unified entity. Recent literature has clearly shown
that different emotions seem related to different
neural systems. For instance, disgust seems to ac-
* Molnar-Szakacs, I., Iacoboni, M., Koski, L., Maeda, tivate preferentially the anterior insula (60),
F., Dubeau, M. C., Aziz-Zadeh, L. & Mazziotta, J. C. (2002) whereas fear seems to activate preferentially the
J. Cogn. Neurosci., Suppl. S, F118 (abstr.). amygdala (56, 57). We adopted this approach
RT0996_C11.qxd 11/9/04 4:12 PM Page 151
because our main goal was to investigate the rela- REFERENCES

tionships between action representation and
1. Gallese, V. (2001) J. Conscious. Stud. 8, 33–50.
emotion via an imitation paradigm. Future studies
2. Chartrand, T. L. & Bargh, J. A. (1999) J. Pers. Soc.
may successfully employ imitative paradigm to Psychol. 76, 893–910.
further explore the differential neural correlates of 3. LeDoux, J. E. (2000) Annu. Rev. Neurosci. 23, 155–184.
emotions. 4. diPellegrino, G., Fadiga, L., Fogassi, L., Gallese, V. &
Taken together, these data suggest that we un- Rizzolatti, G. (1992) Exp. Brain Res. 91, 176–180.
5. Gallese, V., Fadiga, L., Fogassi, L. & Rizzolatti, G. (1996)
derstand the feelings of others via a mechanism of
Brain 119, 593–609.
action representation shaping emotional content, 6. Perrett, D. I., Harries, M. H., Bevan, R., Thomas, S.,
such that we ground our empathic resonance in Benson, P. J., Mistlin, A. J., Chitty, A. J., Hietanen, J. K. &
the experience of our acting body and the emo- Ortega, J. E. (1989) J. Exp. Biol. 146, 87–113.
tions associated with specific movements. As 7. Rizzolatti, G., Fogassi, L. & Gallese, V. (2001) Nat. Rev.
Neurosci. 2, 661–670.
Lipps noted, “When I observe a circus performer
8. Perrett, D. I. & Emery, N. J. (1994) Curr. Psychol. Cogn.
on a hanging wire, I feel I am inside him” (1). To 13, 683–694.
empathize, we need to invoke the representation 9. Perrett, D. I., Harries, M. H., Mistlin, A. J., Hietanen, J. K.,
of the actions associated with the emotions we are Benson, P. J., Bevan, R., Thomas, S., Oram, M. W., Ortega,
witnessing. In the human brain, this empathic res- J. & Brierly, K. (1990) Int. J. Comp. Psychol. 4, 25–55.
10. Decety, J., Grezes, J., Costes, N., Perani, D., Jeannerod,
onance occurs via communication between action
M., Procyk, E., Grassi, F. & Fazio, F. (1997) Brain 120,
representation networks and limbic areas provided 1763–1777.
by the insula. Lesions in this circuit may determine 11. Grezes, J. & Decety, J. (2001) Hum. Brain Mapp. 12, 1–19.
an impairment in understanding the emotions of 12. Iacoboni, M., Koski, L. M., Brass, M., Bekkering, H.,
others and the inability to “empathize” with Woods, R. P., Dubeau, M. C., Mazziotta, J. C. & Rizzolatti,
G. (2001) Proc. Natl. Acad. Sci. USA 98, 13995–13999.
them.**
13. Iacoboni, M., Woods, R. P., Brass, M., Bekkering,
H., Mazziotta, J. C. & Rizzolatti, G. (1999) Science 286,
2526–2528.
Acknowledgments 14. Seltzer, B. & Pandya, D. N. (1994) J. Comp. Neurol. 343,
445–463.
This work was supported, in part, by the Brain 15. Kalaska, J. F., Caminiti, R. & Georgopoulos, A. P. (1983)
Mapping Medical Research Organization, the Exp. Brain Res. 51, 247–260.
16. Lacquaniti, F., Guigon, E., Bianchi, L., Ferraina, S. &
Brain Mapping Support Foundation, the Pierson-
Caminiti, R. (1995) Cereb. Cortex. 5, 391–409.
Lovelace Foundation, the Ahmanson Foundation, 17. Mountcastle, V., Lynch, J., Georgopoulos, A., Sakata,
the Tamkin Foundation, the Jennifer Jones- H. & Acuna, C. (1975) J. Neurophysiol. 38, 871–908.
Simon Foundation, the Capital Group Companies 18. Sakata, H., Takaoka, Y., Kawarasaki, A. & Shibutani,
Charitable Foundation, the Robson Family, the H. (1973) Brain Res. 64, 85–102.
19. Rizzolatti, G. & Luppino, G. (2001) Neuron 31, 889–901.
Northstar Fund, and grants from the National
20. Kohler, E., Keysers, C., Umilta, M. A., Fogassi, L., Gallese,
Center for Research Resources (RR12169 and V. & Rizzolatti, G. (2002) Science 297, 846–848.
RR08655). 21. Umilta, M. A., Kohler, E., Gallese, V., Fogassi, L., Fadiga,
L., Keysers, C. & Rizzolatti, G. (2001) Neuron 31, 155–165.
22. Koski, L., Wohlschlager, A., Bekkering, H., Woods, R. P.,
Dubeau, M. C., Mazziotta, J. C. & Iacoboni, M. (2002)
Abbreviations Cereb. Cortex 12, 847–855.
23. Augustine, J. R. (1996) Brain Res. Rev. 2, 229–294.
fMRI, functional MRI; PET, positron-emission
24. Oldfield, R. C. (1971) Neuropsychologia 9, 97–113.
tomography. 25. Ekman, P. & Friesen, W. V. (1976) Pictures of Facial
Affect (Consulting Psychologist Press, Palo Alto, CA).
** Vicenzini, E., Bodini, B., Gasparini, M., Di Piero, V., 26. Gentilucci, M., Fogassi, L., Luppino, G., Matelli, M.,
Mazziotta, J. C., lacoboni, M. & Lenzi, G. L. (2002) Camarda, R. & Rizzolatti, G. (1988) Exp. Brain Res. 71,
Cerebrovasc. Dis. 13, Suppl. 3, 52 (abstr.). 475–490.
RT0996_C11.qxd 11/9/04 4:12 PM Page 152
27. Woods, R. P., Grafton, S. T., Holmes, C. J., Cherry, S. R. 43. Geyer, S., Matelli, M., Luppino, G. & Zilles, K. (2000)
& Mazziotta, J. C. (1998) J. Comput. Assist. Tomogr. 22, Anat. Embryol. 202, 443–474.
139–152. 44. Petrides, M. & Pandya, D. N. (1994) in Handbook of
28. Woods, R. P., Grafton, S. T., Watson, J. D. G., Sicotte, N.L. Neuropsychology, eds. Boller, F. & Grafman, J. (Elsevier,
& Mazziotta, J. C. (1998) J. Comput. Assist. Tomogr. 22, Amsterdam), Vol. 9, pp. 17–58.
153–165. 45. Rizzolatti, G. & Arbib, M. (1998) Trends Neurosci. 21,
29. Woods, R. P., Dapretto, M., Sicotte, N. L., Toga, A. W. & 188–194.
Mazziotta, J. C. (1999) Hum. Brain Mapp. 8, 73–79. 46. vonBonin, G. & Bailey, P. (1947) The Neocortex of
30. Iacoboni, M., Woods, R. P. & Mazziotta, J. C. (1996) Macaca Mulatta (Univ. of Illinois Press, Urbana).
J. Neurophysiol. 76, 321–331. 47. Campbell, R., MacSweeney, M., Surguladze, S., Calvert,
31. Iacoboni, M., Woods, R. P., Lenzi, G. L. & Mazziotta, G., McGuire, P., Suckling, J., Brammer, M. J. & David,
J. C. (1997) Brain 120, 1635–1645. A. S. (2001) Brain Res. Cogn. Brain Res. 12, 233–243.
32. Iacoboni, M., Woods, R. P. & Mazziotta, J. C. (1998) 48. Puce, A., Allison, T., Bentin, S., Gore, J. C. & McCarthy,
Brain 121, 2135–2143. G. (1998) J. Neurosci. 18, 2188–2199.
33. Woods, R. P., Iacoboni, M., Grafton, S. T. & Mazziotta, 49. Allison, T., Puce, A. & McCarthy, G. (2000) Trends Cogn.
J. C. (1996) in Quantification of Brain Function Using Sci. 4, 267–278.
PET, eds. Myers, R., Cunningham, V., Bailey, D. & Jones, 50. Calvert, G. A., Campbell, R. & Brammer, M. J. (2000)
T. (Academic, San Diego), pp. 353–358. Curr. Biol. 10, 649–657.
34. Aguirre, G. K., Zarahn, E. & D’Esposito, M. (1998) 51. Olausson, H., Lamarre, Y., Backlund, H., Morin, C.,
Neuroimage 8, 360–369. Wallin, B. G., Starck, G., Ekholm, S., Strigo, I., Worsley,
35. Zarahn, E., Aguirre, G. K. & D’Esposito, M. (1997) K., Vallbo, A. B. & Bushnell, M. C. (2002) Nat. Neurosci.
Neuroimage 5, 179–197. 5, 900–904.
36. Aguirre, G. K., Zarahn, E. & D’Esposito, M. (1997) 52. Farrer, C. & Frith, C. D. (2002) Neuroimage 15, 596–603.
Neuroimage 5, 199–212. 53. Buck, R. (1980) J. Pers. Soc. Psychol. 38, 811–824.
37. Forman, S. D., Cohen, J. D., Fitzgerald, M., Eddy, W. F., 54. Ekman, P. (1973) Darwin and Facial Expression: A
Mintun, M. A. & Noll, D. C. (1995) Magn. Reson. Med. Century of Research in Review (Academic, New York).
33, 636–647. 55. Ekman, P. (1999) Handbook of Cognition and Emotion
38. Fox, P. T., Huang, A., Parsons, L. M., Xiong, J. H., (Wiley, Chichester, England), pp. 301–320.
Zamarippa, F., Rainey, L. & Lancaster, J. L. (2001) 56. Phillips, M. L., Young, A. W., Scott, S. K., Calder, A. J.,
Neuroimage 13, 196–209. Andrew, C., Giampietro, V., Williams, S. C., Bullmore,
39. Picard, N. & Strick, P. L. (1996) Cereb. Cortex 6, E. T., Brammer, M. & Gray, J. A. (1998) Proc. R. Soc.
342–353. London Ser. B Biol. Sci. 265, 1809–1817.
40. Buccino, G., Binkofski, F., Fink, G. R., Fadiga, L., 57. Phan, K. L., Wager, T., Taylor, S. F. & Liberzon, I. (2002)
Fogassi, L., Gallese, V., Seitz, R. J., Zilles, K., Rizzolatti, Neuroimage 16, 331–348.
G. & Freund, H. J. (2001) Eur. J. Neurosci. 13, 400–404. 58. Morris, J. S., Ohman, A. & Dolan, R. J. (1998) Nature
41. Mazziotta, J., Toga, A., Evans, A., Fox, P., Lancaster, J., 393, 467–470.
Zilles, K., Woods, R., Paus, T., Simpson, G., Pike, B., 59. Hariri, A. R., Bookheimer, S. Y. & Mazziotta, J. C. (2000)
et al. (2001) Philos. Trans. R. Soc. London B Biol. Sci. NeuroReport 11, 43–48.
356, 1293–1322. 60. Phillips, M. L., Young, A. W., Senior, C., Brammer, M.,
42. Mazziotta, J., Toga, A., Evans, A., Fox, P., Lancaster, J., Andrew, C., Calder, A. J., Bullmore, E. T., Perrett, D. I.,
Zilles, K., Woods, R., Paus, T., Simpson, G., Pike, B., et al. Rowland, D., Williams, S. C., et al. (1997) Nature 389,
(2001) J. Am. Med. Inform. Assoc. 8, 401–430. 495–498.
RT0996_P07.qxd 11/8/04 2:14 PM Page 153
PA R T 7
Animacy, Causality, and Theory of Mind

•
More than a half century ago, social psychologist Fritz Heider and
Mary-Ann Simmel constructed a simple film animation of three geometric
shapes—a small yellow circle, a small blue triangle, and a larger gray
triangle—set in motion. The relative distances among these three shapes,
and their positions in and about a large rectangle, varied during the film.
Heider and Simmel simply asked observers to describe what they saw in
the film. Most observers told elaborate stories, such as the circle and the
little triangle being in love, the gray triangle trying to steal away the circle,
the blue triangle fighting back, the circle and little triangle escaping into the
house, and once inside safely embracing. So prevalent is this tendency that
it may seem unremarkable that people imbue simple moving geometric
shapes with individual identities, motives, and emotions. Some individuals
missing a limbic structure called the amygdala, however, simply see
geometric objects moving about the screen.
You enter a new car showroom, intent on purchasing the automobile you
long dreamed of owning. A salesperson greets you politely, shows you the
car, answers your questions patiently, and even gives you the keys to test-
drive the car. When you return, the salesperson invites you into a private
office and asks what you would like to pay for the car. Despite the fact that
the salesperson has shown you extraordinary courtesy to this point, you
have little trust in the individual, and you guard your words at each point in
the negotiation that unfolds. You watch the salesperson intently, trying to
glean a hint regarding the lowest price the salesperson is willing to accept.
In so doing, you realize that the salesperson has knowledge, beliefs, and
153
RT0996_P07.qxd 11/8/04 2:14 PM Page 154
motives behind the behaviors you are allowed to people’s theory of mind. Functional magnetic
see, and you develop a theory of these aspects of resonance imaging (fMRI) was performed on
the salesperson’s mind, which you use to predict the individuals as they read short stories. In their first
salesperson’s actions. Indeed, you make ample use study, participants were instructed to read each
of your theory of mind in almost all your social story silently and to make sure they understood
interactions. You may not recall, but you were not what was happening. They then read stories from
born with this capacity; in most children a theory of each of four categories: false belief, mechanical
mind does not develop until about age 5. inference, human action, and nonhuman
In the preceding sections, we learned that people descriptions. Only the stories that featured false
often subtly mimic the behaviors they observe and beliefs invoked reasoning about others’ beliefs,
empathize with the emotions they see displayed. The reasoning, and feelings—that is, theory of mind
readings in this section build on these observations reasoning. In their second study, participants read
to address how the brain is a social information another series of studies that varied in the extent to
processing organ. Castelli et al. (2000) used positron which theory of mind was required to answer
emission tomography (PET) to image the brain of six questions about the story. The results of this
individuals while they watched short animated films research revealed greater bilateral activation in the
similar to the one constructed by Fritz and Simmel temporo-parietal junction (a region near the
six decades ago. The attribution of mental states superior temporal sulcus) in tasks that required
(beliefs, motives, emotions) was associated with reasoning about another person’s mental state.
increased activity in the medial prefrontal cortex, Importantly, the difficulty of the task was unrelated
superior temporal sulcus, fusiform gyrus, temporal to activity in this region, suggesting it may be
poles near the amygdala, and occipital gyrus. By selectively involved in tasks that require reasoning
now, many of these are becoming familiar areas to about another person’s mental states. Together, the
readers, for they have been implicated in other two readings in this section suggest that the human
studies in self-referent processing and in the brain is not just an isolated information processing
perception of faces and biological motion. machine, but a social brain, for with little
In the second reading, Saxe and Kanwisher encouragement or information, it appears capable
(2003) explore the possible neural substrates of of calculating the mental states of others.
RT0996_C12.qxd 11/8/04 2:07 PM Page 155
R E A D I N G 1 2
Movement and Mind: A Functional Imaging Study

of Perception and Interpretation of Complex
Intentional Movement Patterns
•
Fulvia Castelli,* Francesca Happé,† Uta Frith,* and Chris Frith‡
We report a functional neuroimaging study with positron emission tomography (PET) in which six
healthy adult volunteers were scanned while watching silent computer-presented animations. The
characters in the animations were simple geometrical shapes whose movement patterns selectively
evoked mental state attribution or simple action description. Results showed increased activation in
association with mental state attribution in four main regions: medial prefrontal cortex,
temporoparietal junction (superior temporal sulcus), basal temporal regions (fusiform gyrus and
temporal poles adjacent to the amygdala), and extrastriate cortex (occipital gyrus). Previous
imaging studies have implicated these regions in self-monitoring, in the perception of biological
motion, and in the attribution of mental states using verbal stimuli or visual depictions of the
human form. We suggest that these regions form a network for processing information about
intentions, and speculate that the ability to make inferences about other people’s mental states
evolved from the ability to make inferences about other creatures’ actions.
Keywords: brain imaging; theory of mind; mentalizing; biological motion; autism.
*Institute of Cognitive Neuroscience, University College College London; and ‡Wellcome Department of Cognitive
London, Alexandra House, 17 Queen Square, London WCIN Neurology, Institute of Neurology, University College
3AR, United Kingdom; †Institute of Psychiatry, Kings London.
155
RT0996_C12.qxd 11/8/04 2:07 PM Page 156
Introduction areas of activity was close to those previously re-

ported by Fletcher et al. (1995) and by Goel et al.
Recent interest in the evolution, development, and (1998), and relates to the paracingulate sulcus.
breakdown of social cognition has focused on Activity was also observed in the temporoparietal
Theory of Mind, the ability to attribute independ- junction bilaterally.
ent mental states to self and others in order to Previous brain imaging studies of mental state
explain and predict behaviour (Carruthers and attribution have tended to use high-level verbal
Smith, 1996; Baron-Cohen et al., 2000). Fodor stimuli (Baron-Cohen et al., 1994; Fletcher et al.,
(1992), Leslie and Thaiss (1992), Scholl and 1995; Happé et al., 1996; Goel et al., 1998) or
Leslie (1999), and others have suggested that our visual depictions of humans (Gallagher et al.,
ability to attribute mental states, or “mentalize,” 2000; Baron-Cohen et al., 1999). Mentalizing,
results from a dedicated, domain-specific, and however, involves processes at a number of levels,
possibly modular cognitive mechanism. This pro- from perceptual to conceptual. If mentalizing re-
posal gains particular support from studies of lies on a dedicated cognitive mechanism, or mod-
autism, a biologically based developmental disor- ule, then one interesting question concerns the na-
der, which appears to be characterized by a selec- ture of its obligatory triggering inputs. The aim of
tive impairment in Theory of Mind (Baron-Cohen the present study was to examine brain activation
et al., 1985; Frith et al., 1991; Happé and Frith, during exposure to simple, nonverbal stimuli de-
1996). Interest in the brain basis of normal Theory signed to evoke mental state attribution by their
of Mind is fired by the hope of a better under- kinetic properties alone. Inspiration for appropri-
standing of the neural systems affected in people ate stimuli came from the classic work of Heider
with autism, who are (in the main) unable to and Simmel (1944), who demonstrated that even
“mind-read” (Baron-Cohen, 1995). simple geometric shapes could elicit by their pat-
There is a growing number of published reports tern of contingent movement the attribution of
of functional brain imaging studies of Theory of complex internal states, such as intentions and be-
Mind (ToM). Most of these studies implicate acti- liefs. Subsequent work by Berry and colleagues
vation in medial frontal and temporoparietal (Berry et al., 1992; Berry and Springer, 1993) has
regions. For example, Fletcher et al. (1995), in a shown that properties of the movement, rather
PET study, scanned volunteers reading and an- than of the stimuli/characters, are fundamental to
swering questions about stories involving com- the complex attributions made by adults and chil-
plex mental states (ToM stories) and those involv- dren from the late preschool years onwards.
ing inferences of physical cause and effect To explore brain activation during such move-
(“physical” stories). Comparison of activation ment-provoked mental state attribution we used
during ToM versus “physical” stories revealed silent animations of three types. In our Theory of
increased activation in the medial frontal gyrus on Mind (ToM) condition, interaction between two
the left (BA 8/9), as well as in the posterior cingu- shapes (big and small triangle) was scripted to im-
late cortex and the right inferior parietal cortex ply complex mental states, such as intention to
(BA 40) at the temporoparietal junction. Gallagher deceive. Thus, in these animations one character’s
et al. (2000) used the same set of stories in an actions were readily seen as determined by what
fMRI study with normal volunteers. In addition to the other character “thought.” In the second ani-
the written stories, subjects were shown figurative mation type, “Goal-directed” (GD), the characters
drawings (humorous cartoons) which also interacted on a simple, purposeful level. That is,
prompted attribution of mental states. With the in these animations, one character’s actions were
greater resolution of fMRI it was possible to dis- seen as determined by what the other character
tinguish a number of peaks in Brodmann areas 8/9 “did.” In the third animation type, “Random ac-
and the border of 10 and 32, associated with both tion” (R), the two characters did not interact, and
ToM cartoons and stories. The location of these their behavior was not contingent—in effect they
RT0996_C12.qxd 11/8/04 2:07 PM Page 157
Movement and Mind ■ 157
were merely floating or bouncing around. Stimuli Method

in all three animation types moved in a self-
propelled fashion. Our stimuli could therefore be Subjects
graded in terms of complexity of predominantly Six right-handed male volunteers (aged 20 to 31,
evoked descriptions, from random movements, to mean 24.5 years) took part in this study. All subjects
goal-directed actions, and finally to complex in- were healthy, with no history of significant medical,
tentional states. Conversely, people’s descriptions psychiatric, or neurological illness. All gave writ-
could be graded in terms of their degree of men- ten, informed consent to take part in the study,
talizing regardless of the animation sequences which was approved by the ethics committee of the
they were describing. National Hospital for Neurology and Neurosurgery
The present animations were first used in a be- and the Administration of Radioactive Substances
havioral study that collected descriptions from Advisory Committee (ARSAC), UK.
children with autism, children with developmental
delays, normally developing children, and adults Data Acquisition
(Abell et al., 2000). Results from this study sup- All subjects underwent both PET and MRI scan-
ported the validity of the three types of animations. ning on the same day. A Siemens VISION
Children with autism proved to be less accurate in (Siemens, Erlangen) operating at 2.0T was used
their descriptions of ToM animations than children to acquire axial T1 weighted structural MRI im-
without autism. The 14 adults taking part in this ages for anatomical coregistration. PET scans
study attributed precise mental states, matching were performed with an ECAT EXACT HR+
the underlying script in 89% of their responses to scanning system (CTI Siemens, Knoxville, TN) in
the ToM animations, with descriptions of purpose- high sensitivity 3-D mode with septa retracted
ful movements for the remaining responses. They (Townsend et al., 1991). A venous cannula to ad-
attributed precise purposeful interactions in 93% minister the tracer was inserted in the antecubital
of their responses to the Goal-directed animations fossa vein. Approximately 350 Mbq of H2 15O in
with the remaining responses all involving mental 3 ml of normal saline were loaded into intra-
state attribution. Description of simple movement venous tubing and flushed into subjects over 20 s
without a purposeful component were given in at a rate of 10 ml/min by an automatic pump. Af-
64% of responses to the Random sequences while ter a delay of approximately 35 s, a rise in counts
purposeful movement was described for the re- could be detected in the head that peaked 30–40 s
mainder. Even though the vast majority of descrip- later (depending on individual circulation time).
tions of the three types of animations fell into an The interval between successive administrations
orderly pattern, the animations were ambiguous was 8 min. The data were acquired in one 90 s
enough for interpretations to occur that were either frame, beginning 5 s before the rising phase of the
simpler or richer than intended by the designers. head curve. After correcting for background activ-
Our prediction for the present study was that the ity, the true counts accumulated during this period
ToM animations, but not the Random animations, were taken as an index of cerebral blood flow
would evoke mental state attributions and show (Fox and Mintun, 1989). Images were recon-
activation patterns similar to those found in previ- structed by filtered back projection (Hanning
ous functional imaging studies of Theory of Mind filter, cut off frequency 0.5 cycles per pixel) into
(Goel et al., 1998; Fletcher et al., 1995; Baron- 63 image planes (separation 2.4 mm) and into a
Cohen et al., 1999; Gallagher et al., 2000). We ex- 128 128 pixel image matrix (pixel size 2.1
pected the GD animations to have an intermediate mm). Twelve scans were acquired per subject.
status. Going one step further, we predicted activa-
tion in ToM-related areas for all sequences that Statistical Analysis
provoked mental state interpretations, regardless Functional imaging analysis used the technique of
of the animation condition. Statistical Parametric Mapping implemented in
RT0996_C12.qxd 11/8/04 2:07 PM Page 158
SPM97 (Wellcome Department of Cognitive between 34 and 45 s, and the three types of anima-
Neurology, London, UK) (http://www.fil.ion.ucl. tions were matched for length. The “scripts” for
ac.uk/spm). For each subject, a set of 12 PET the ToM sequences involved the two triangles per-
scans was automatically realigned and then suading, bluffing, mocking, and surprising one an-
stereotactically normalized (Friston et al., 1995b) other. The Goal-directed “scripts” involved the
into the space of Talairach and Tournoux (1988). two triangles dancing together, one chasing, one
The scans were then smoothed using a Gaussian imitating, and one leading the other. The Random
kernel of 12 mm full-width half maximum. movement showed the two triangles bouncing off
The analysis of functional imaging data entails the walls resembling the movement of billiard
the creation of statistical parametric maps that balls, or merely drifting about. While the type of
represent a statistical assessment of condition- movement was by definition different between the
specific effects hypothesised by the experimenter three conditions, the basic visual characteristics in
(Friston et al., 1995a). The effects of global terms of shape, overall speed, and orientation
changes in blood flow were modelled as a con- changes were as similar as possible.
found using a subject-specific ANCOVA (Friston
et al., 1990). Areas of significant change in brain Procedure
activity were specified by appropriately weighted
Subjects were instructed before scanning (see
linear contrasts of the condition-specific effects
Appendix 1) and were given practice examples of
and determined using the t-statistic on a voxel by
the three types of animations. The animations
voxel basis. We created the relevant SPM [t] for
were presented on the screen of a Power Macin-
each comparison of conditions, which was then
tosh computer suspended on an adjustable cradle
transformed into an SPM [Z] and thresholded at a
at a suitable distance for each subject. Prior to
Z score of 3.09 (P 0.001 uncorrected). Clusters
scanning, it was ascertained that the subject could
of activated voxels were characterized in terms of
watch the animations comfortably.
their peak height and spatial extent conjointly.
There was a cued and an uncued condition. Be-
fore the cued condition subjects were told either
Design that they were going to see an animation showing
A 3 2 repeated measures within subjects design “an interaction with feelings and thoughts”
was used. Four different examples of each of three (ToM), or “a random movement” (R), or “a sim-
types of animation. ToM, Goal-directed, and Ran- ple interaction” (GD). Before the uncued condi-
dom were displayed over the course of 12 scans, tion, subjects were simply told that they were
divided into two consecutive counterbalanced about to see the next animation. Order of cued and
blocks: cued animation and uncued animation. In uncued blocks was counterbalanced across sub-
a previous study (Fletcher et al., 1995) subjects jects, so that half the subjects obtained cued ani-
were cued before the scan. They were told in ad- mations first, and half uncued animations first.
vance which kind of stimuli they were going to After each scan subjects were asked to tell the
see (see Appendix 1). In the present study we experimenter what they thought the triangles were
counterbalanced cued with uncued animations in doing. The experimenter always asked the same
order to control for the effect of prior knowledge. neutral question: “What was happening in this
animation?” Answers were recorded for later scor-
Animation Materials ing. On no occasion was feedback given, but sub-
jects were generally praised for their descriptions.
Twelve animations were used during the scanning,
and an additional three were shown for practice. All
the animations featured two characters, a big red tri- Scoring
angle and a small blue triangle, moving about on a The verbal descriptions given after each presentation
framed white background. Each sequence lasted (in between scans) were coded along four different
RT0996_C12.qxd 11/8/04 2:07 PM Page 159
dimensions. The aim of the scores was to distinguish The length of the descriptions differed signifi-
in each answer (1) the implied “intentionality,” that cantly (F(2,10) 19.49, P 0.001), with the
is, the degree of appreciation of mental states, (2) ToM animations eliciting longer explanations than
their appropriateness, that is, how well the underlying Goal-directed (t 5.48, P 0.003) or Random
script was captured, (3) the certainty of the explana- animations (t 5.11, P 0.004), which did not
tion, and (4) the length of each answer. The Inten- differ from each other (t 1.05, P 0.341).
tionality score reflected the use of mental state terms, There was no difference, however, in the “appro-
with scores ranging from 0 (nondeliberate action), to priateness” or “certainty” of the explanations given
5 (deliberate action aimed at affecting another’s men- to the three animation types (F(2,10) 0.49, P
tal state; see Appendix 2 for examples). Two raters 0.628 and F(2, 10) 3.33, P 0.078, respec-
gave the identical score 65% of the time, and had an tively). Absence or presence of cueing revealed
average discrepancy of only 1.4 points in the remain- neither a main effect nor interaction with the ani-
ing 35% of the cases. The Appropriateness score mation type in any of the four scores.
measured the understanding of the event depicted in
the animations, as intended by the designers (0 to 3).
Regional Cerebral Blood Flow:
The Certainty score graded the degree of hesitation
Subtraction Analysis
present in the verbal description (0 to 3). The Length
score classified the number of clauses in each answer There were no significant differences between
(0 to 4). Details of scoring are given in Appendix 2. cued and uncued presentations, nor were there
any order effects, or any significant interactions.
Data for cued and uncued sequences were there-
Results fore combined. There were significant differ-
ences between the three types of animation. ToM
Behavioral Data animations elicited more activity than Random
Table 12.1 shows the ratings of the descriptions of animations in four regions: temporalparietal
each type of animation. As can be seen, the three an- junction (at the end of the superior temporal sul-
imation types differed significantly in the degree of cus), basal temporal region (fusiform gyrus and
mental state attribution they evoked (F(2, 10) temporal poles, immediately adjacent to the
154.75, P 0.001). As expected on the basis of amygdala), extrastriate cortex (occipital gyrus),
Abell et al.’s results (2000) subjects attributed more and medial prefrontal cortex (see Table 12.2). All
intentionality to the characters’ behavior during these differences were observed in both hemi-
ToM animations than during GD (t value 5.89, spheres, but were more significant in the right
P 0.002) and R animations (t value 16.04, P hemisphere, except for the medial prefrontal
0.001). Random animations evoked significantly cortex. For all these regions differences occurred
fewer mental state attributions than Goal-directed between the ToM and the Random condition, with
animations (t value 17.43, P 0.001). the Goal-directed condition showing intermediate
TABLE 12.1. Verbal Descriptions Given by the Six Subjects for ToM, Goal-Directed, and Random Animations
Rated on Four Dimensions
Total score ToM Goal-Directed Random
maximum mean (s.d.) mean (s.d.) mean (s.d.)
Intentionality 20 15.8 (1.5) 9.7 (1.5) 0.7 (1.2)

Appropriateness 12 11.2 (1.6) 10.5 (1.4) 11.2 (1.2)
Certainty 12 10.7 (0.8) 10 (1.9) 11.3 (1.2)
Length 16 12.5 (3) 8.5 (3) 7.7 (3.6)
Note: Differences between the three conditions were significant at P 0.01 for Intentionality. Differences between ToM animations vs
GD and R animations were significant at P 0.01 for Length. All other differences were not significant.
RT0996_C12.qxd 11/8/04 2:07 PM Page 160
TABLE 12.2. Subtraction Analysis: Regions Where ToM Animations Elicited More Activity Than Random Animations
Coordinates
Foci of activation BA Left (x, y, z) Z score P Right (x, y, z) Z score P
Temporal-parietal junction
STS 22/39 58, 48, 4 4.3 0.06 60, 56, 12 6.2 0.001
Basal temporal
FuG 37 38, 44, 22 3.8 36, 56, 20 5.1 0.01
TmP/Am 38 38, 4, 32 3.2 34, 6, 26 4.0 0.05
Occipital lobe
OcG 19/18 30, 94, 12 4.6 0.02 38, 96, 10 5.0 0.01
OcG 19/18 32, 82, 24 4.1
Medial prefrontal
SFG 9 4, 60, 32 4.1
Note: The coordinates are given in the stereotactic space of Talairach and Tournoux (1988). Numbers in bold type indicate regions
where differences in activity were significant when corrected for multiple comparisons. Numbers in plain type indicate regions where
differences in activity were significant at P 0.0001, uncorrected. Brain regions are identified by name and by putative Brodmann
Area (BA) on the basis of the atlas of H. M. Duvernoy (1999) The Human Brain: Surface, Three-Dimensional Sectional Anatomy with
MRL and Blood Supply. Springer, Wien, New York. STS, superior temporal sulcus; TmP/Am, temporal pole adjacent to amygdala;
FuG, fusiform gyrus; OcG, occipital gyrus; SFG, superior frontal gyrus.
activity (see Figure 12.1). Direct comparison of medial occipital cortex (2x, 94y, 14z). The
ToM with GD confirmed that the differences locations of the activations are shown superim-
apparent in Figure 12.1 were significant in the posed on a standard brain in Figure 12.2.
case of temporoparietal regions and the temporal
pole at a level of P 0.0001 uncorrected and for Regional Cerebral Blood Flow:
occipital gyrus and fusiform gyrus at P 0.01. Correlational Analysis
Random movement when compared to ToM A further analysis was performed in which Inten-
movement, elicited more activity in one region of tionality scores, regardless of condition, were
CASTELLI ET AL.
a) b) c) d) e)
rCBF rCBF rCBF rCBF rCBF
52 52 52
52 52
51 51 51
51 51
50 50 50
50 50
49 49 49 49 49
48 48 48 48 48
ToM GD R ToM GD R ToM GD R ToM GD R ToM GD R

superior frontal gyrus superior temporal sulcus inferior temporal sulcus lateral occipital gyrus medial occipital gyrus
FIGURE 12.1 ■ Blood flow as a function of condition (ToM, Goal-directed, and Random) in key regions.
RT0996_C12.qxd 11/8/04 2:07 PM Page 161
a) more active as in the comparison of the three condi-

tions (see Table 12.3). These results were not af-
fected when the length of the descriptions given by
the subjects was entered as a confounding covariate.
Discussion
The present experiment took as its starting point
the pervasive tendency to perceive intentions in
b)
complex movement patterns even when no human
forms are depicted. We showed that different
types of silent animations selectively evoked de-
scriptions of what the characters were thinking or
descriptions of what the characters were doing.
The different types of descriptions occurred spon-
taneously, since alerting subjects in advance to the
nature of a particular sequence had no effect.
The main aim of this study was to locate a
brain system associated with the attribution of
c) mental states evoked by kinetic stimulus proper-
ties. At the same time we wished to relate the
findings to earlier studies of mentalizing with
different kinds of stimuli. Subtraction analysis
(contrasting the ToM sequences with Random
or Goal-directed sequences) gave the same pic-
ture as correlational analysis (correlating blood
flow with degree of mental state description
across all animations). The results showed in-
creased activation in four main areas bilaterally.
FIGURE 12.2 ■ Regions of significant cerebral blood flow
(rCBF) change associated with the perception of ToM
These areas include medial prefrontal cortex,
animations vs Random animations. (a) Saggital view of temporoparietal junction (superior temporal sul-
activation in superior frontal gyrus. (b) Coronal view of cus), basal temporal region (fusiform gyrus and
activation in superior temporal sulcus and fusiform temporal poles adjacent to the amygdala), and
gyrus. (c) Saggital view of activation in temporal pole ad- occipital cortex. All of these areas have been
jacent to the amygdala, fusiform gyrus, and occipital
gyrus.
implicated in prior studies of mentalizing. This
suggests that a system can be delineated which
correlated with blood flow response. This analysis is to some extent independent of the mode of
was carried out within subjects, thus avoiding the stimulus input, visual or verbal.
assumption that different individuals use the same The results of this study do not enable us to iden-
range of descriptions. An assumption inherent in tify the functions of these four regions, but clues to
this analysis is a linear relationship between in- their significance can be gained by considering pre-
tentionality scores and blood flow response. How- vious studies involving different paradigms.
ever, further analysis which allowed for a nonlin-
ear relationship did not produce a significant The Medial Prefrontal Region
increase in variance accounted for. An as yet unpublished fMRI study using Heider
The results of the correlational analysis were and Simmel type silent animations has recently
clearcut. The same four areas were identified as been summarized by Klin et al. (2000). The results
RT0996_C12.qxd 11/8/04 2:07 PM Page 162
TABLE 12.3. Correlation Analysis: Regions with Significant Correlations between Blood Flow Response and
Intentionality Score
Coordinates
Foci of activation BA Left (x, y, z) Z score P Right (x, y, z) Z score P
Temporal-parietal junction
MTG 21/37 60, 48, 4 4.6 0.02
STS 39 62, 58, 12 6.6 0.001
Basal temporal
FuG 37 36, 42,22 3.7 38, 54, 22 4.9 0.01
TmP/Am 34/38 30, 4, 24 2.9
22, 0, 16 3.05 0.08
Occipital lobe
OcG 18 40, 96, 10 4.8 0.01
OcG 17 16, 100, 8 4.2 0.05
Medial prefrontal
SFG 8/9 6, 58, 32 3.0
Note: Conventions as in Table 12.2. MTG, middle temporal gyrus; STS, superior temporal sulcus; FuG, inferior temporal gyrus;
TmP/Am, temporal pole adjacent to amygdala; OcG, occipital gyrus; SFG, superior frontal gyrus.
appear to be highly consistent with our findings. mental states. In a task of metaphor comprehen-
In particular, these authors mention strong activa- sion, which, according to some theorists (Sperber
tion in medial prefrontal cortex. The medial pre- and Wilson, 1986; Happé, 1993), requires recog-
frontal region activated during the attribution of nition of the speaker’s intentions, Bottini et al.
mental states to animated triangles has also been (1994) found activation in several loci, including
shown to be activated by other stimuli evoking left rostral anterior cingulate cortex, very close to
attribution of beliefs and intentions. These studies the area implicated in the studies of mentalizing
are summarized in Table 12.4. mentioned above. It appears, therefore, that a
For example, Goel et al. (1998) found left me- number of very different mentalizing tasks across
dial prefrontal gyrus activation associated with several modalities (e.g., verbal, nonverbal) and
reasoning about other people’s thoughts regarding with differing stimulus qualities (e.g., static, mov-
a novel object. Fletcher et al.’s (1995) story coming), activate regions of medial frontal cortex (see
prehension task, requiring inferences about a Table 12.4 for coordinates). Prefrontal cortex is
character’s intentions, showed peak activation in a implicated in a number of cognitive processes that
dorsal region of medial frontal cortex. This region might have a role in mentalizing tasks such as
was not activated in individuals with Asperger working memory and retrieval from episodic
syndrome, who show delays and deficits in memory. However, these processes usually en-
Theory of Mind (Happé et al., 1996). Gallagher gage lateral regions of prefrontal cortex rather
et al. (2000) have compared the same story task than medial regions as observed in the present
with a nonverbal comprehension task, using static experiment (Grady, 1999).
single frame cartoons. They found a convergence Studies of self-monitoring have also shown in-
between activations in response to verbal and vi- creased activity in areas including medial pre-
sual stimuli that prompt mental state attribution frontal and cingulate cortex. This suggests that
(reading a text and viewing a cartoon, respec- when subjects have to reflect on their own mental
tively), with bilateral activation in a ventral area states, they may use neural pathways similar to
of the medial prefrontal cortex. those underlying attribution of mental states to
In addition, medial prefrontal areas have been others. For example, subjects required to monitor
shown to be activated during a rather different their intended speech, in order to judge whether
task that may, nonetheless, require attribution of distorted feedback was their own or another
RT0996_C12.qxd 11/8/04 2:07 PM Page 163
TABLE 12.4. Coordinates for Activation of Medial Frontal Regions in Present and Related Studies
Coordinates
Task Cognitive process Study Left (x, y, z) Right (x, y, z)
Observing complex Mental state attribution Present study 4, 60, 32

intentional movement
(vs random movement)
Judge others’ knowledge Mental state attribution Goel et al. (1998) 12, 38, 32
Story comprehension Mental state attribution Fletcher et al. (1995) 12, 42, 40
Story comprehension Mental state attribution Happé et al. (1996) 12, 36, 36
(Asperger syndromes)
Story and cartoon Mental state attribution Gallagher et al. (2000) 10, 48, 12 8, 22, 46
comprehension
Metaphor comprehension Attribution of speaker’s Bottini et al. (1994) 2, 42, 8
communicative intention
Intended speech Monitoring own mental McGuire et al. (1996a) 2, 36, 36 2, 52, 4
monitoring states 10, 32, 24
Self-generated Monitoring own mental McGuire et al. (1996b) 8, 38, 24
thoughts states 0, 38, 36
Perceiving pain Monitoring own mental Rainville et al. (1997) 3, 20, 30
states
Perceiving tickle Monitoring own mental Blakemore et al. (1998) 2, 42, 6
states
Reporting emotions Monitoring own mental Lane et al. (1997) 0, 50, 16
states
Intended response Monitoring own mental Carter et al. (1998) 4, 25, 43
monitoring states
Observing human Perception of biological Bonda et al. (1996) 7, 58, 26
body movement motion
person’s voice (McGuire et al., 1996a), showed A more complex self-monitoring task elicited ac-
activation of bilateral medial frontal cortex and tivity in anterior cingulate cortex when subjects
anterior cingulate gyrus/medial prefrontal cortex were required to choose between competing re-
as well as temporoparietal junction bilaterally. sponses (Carter et al., 1998). Taken together, these
The prefrontal region was also activated in a results seem to indicate that online monitoring of
study where subjects reported self-generated inner states—own or others’—may engage the
thoughts independent from stimuli in the imme- anterior cingulate cortex and neighbouring medial
diate environment (McGuire et al., 1996b). A frontal regions, regardless of the specific source of
quite different type of self-monitoring task inves- information.
tigated the neural substrates of perceived pain The medial frontal region activated by our ToM
(Rainville et al., 1997). The anterior cingulate animations also overlaps with regions activated by
cortex showed increased activity when subjects point-light displays of biological motion. Bonda
perceived (under hypnosis) the increasing un- et al. (1996) used two biological movement condi-
pleasantness of hot water on their hand. Blake- tions, a dancing figure (human body movement) and
more et al. (1998) found anterior cingulate activity a grasping hand simulating the act of reaching out
associated with reporting a tickling sensation from for a glass and bringing it to the mouth (goal-
self-produced tactile stimulation. Activity in ante- directed action). The comparison of activation dur-
rior cingulate, extending into the medial prefrontal ing the two conditions showed that perception of a
region, was also observed when subjects reported dancing figure versus a grasping hand elicited a net-
their own emotional responses to pleasant, un- work of activation, including left medial prefrontal
pleasant and neutral pictures (Lane et al., 1997). cortex, close to that activated by our ToM animations.
RT0996_C12.qxd 11/8/04 2:07 PM Page 164
Grady (1999) provides an exhaustive list of ac- temporal gyrus during silent lip-reading of num-
tivations observed in prefrontal cortex classified bers versus still lips, and Grezes et al. (1998) re-
in terms of putative Brodmann areas. The vast ported activation of the superior/middle temporal
majority of these are lateral. However, some me- region during viewing of meaningful hand ges-
dial activations have been observed in the vicinity tures with tools and objects compared to stationary
of the area reported in the present study. The only hands. Taken together these studies implicate the
study observing a relevant activation in Brodmann superior temporal sulcus and adjacent cortex in the
area 10 was that of Bottini et al. (1994) on perception of a variety of human body movements.
metaphor comprehension. Activation in relevant This region is anterior and superior to the visual
regions of Brodmann area 9 have been observed motion area MT/V5 (Puce et al., 1998), indicating
in motor learning tasks and working memory that these activations are not attributable to move-
tasks, but the majority of the activations observed ment per se. It is notable, too, that all our anima-
during such tasks are more lateral and more pos- tions (including Random) displayed self-propelled
terior. Activations are also reported in medial movement as might be expected of animate
Brodmann area 8 for some language tasks and for agents. Our triangles, when described as moving
some object processing tasks, but here again all purposefully and intentionally, activated the key
the activations are more posterior than the one brain regions that have been activated by viewing
observed in the present study. biological motion. Human-like face or body char-
In previous studies of mentalizing, the activity acteristics thus do not appear to be necessary to
in medial frontal cortex lies at the border of ante- trigger the attribution of mental states. Future in-
rior cingulate cortex and medial frontal cortex in vestigations are needed to clarify what particular
the paracingulate sulcus (Gallagher et al., 2000). properties of biological motion are functionally
In an exhaustive examination of studies that have associated with temporoparietal activation, and
activated anterior cingulate cortex, Paus et al. whether distinct regions respond preferentially to
(1998) conclude that this region has distinct func- specific visual attributes of biological stimuli.
tions. The posterior part of ACC is primarily
engaged by motor tasks while the more anterior Basal Temporal Cortex
portions are particularly engaged when emotions The ToM animations also elicited bilateral activa-
are involved. The areas associated with mentaliz- tion in the basal temporal region, with peak com-
ing are clearly anterior to the motor region of an- ponents in the caudal part of the fusiform gyrus
terior cingulate cortex. and in the temporal poles adjacent to the amyg-
dala. Baron-Cohen et al. (1999) reported in-
Temporoparietal Region creased activation in the amygdala region during a
Increased activation in the junction between pari- mentalizing task involving judgement of a per-
etal and temporal lobes has been observed using a son’s eyes, as well as activation in medial
story comprehension task and static cartoons prefrontal cortex and the temporoparietal region.
(Gallagher et al., 2000). Again this area was highly Connections between these areas are known to be
active in response to stimuli that share properties strong (Amaral et al., 1992). Temporal pole acti-
of biological motion. Bonda et al. (1996), for ex- vation has previously been associated with narra-
ample, reported activity in the left caudal-most tives (Mazoyer et al., 1993; Fletcher et al., 1995)
part of the superior temporal sulcus when viewing and this fits with the idea that subjects inferred the
grasping hand movement compared to random scripts underlying ToM animations. These anima-
movement. Puce et al. (1998) found increased su- tions had certainly more narrative content com-
perior temporal sulcus activation when viewing pared to the other sequences.
faces in which eye gaze repeatedly changed direc- The studies of biological movement perception
tion, and faces in which the mouth opened and discussed above, also reported peak activations in
closed. Similarly Calvert et al. (1997) observed left fusiform gyrus and left temporal pole in re-
increased activation in a region of the superior sponse to observing meaningful hand gestures
RT0996_C12.qxd 11/8/04 2:07 PM Page 165
compared to stationary hands (Grezes et al., 1998). objects rather than letters (Fink et al., 1997b): local
Left fusiform gyrus activation was found during processing elicited increased lateral activation, while
observation of a dancing human figure compared global processing elicited increased medial activa-
to random movement (Bonda et al., 1996). The tion. It is notable that the comparisons between our
ventral temporal area has also been implicated in ToM and Random animations showed similar differ-
visual processing of static stimuli: while reading ential activations: lateral during ToM stimuli, and
words and naming pictures (e.g., Vandenberghe et medial during Random stimuli. An important differ-
al., 1996) and while reading Braille words, versus ence between the method used in the present study
letter-strings (Buchel et al., 1998). Several imag- and in Fink et al.’s studies is that the latter reported
ing studies have reported specific regions of the activations associated with global and local process-
fusiform gyrus to be more active during face ing resulting from a “top-down” (endogenous)
viewing compared to assorted pictures, hands, process. Subjects were specifically instructed to at-
scrambled faces, and houses (e.g., Kanwisher et al., tend to the stimuli at either the global or the local
1997), and more active during face than letter- level, whereas in our study subjects were not
string and texture perception (e.g., Puce et al., instructed how to view the stimuli. It makes some
1996). Gorno-Tempini et al. (1998) reported intuitive sense, however, that participants may have
increased activity in bilateral temporal poles asso- attended to global patterns of movement in the, ef-
ciated with famous and non-famous face and fectively meaningless, Random condition (float-
proper name processing. Activity in bilateral ing, bouncing), and paid more attention to the
fusiform gyri was increased while processing specifics of movement, interaction, and character
faces relative to names and scrambled faces. Thus details (e.g., which way a triangle is pointing) in the
different areas of the fusiform gyrus appear to be ToM scenarios. Taken together, these studies suggest
specialised for recognition of different kinds of that occipital sites may be implicated in the percep-
objects, including animate agents. tion of movement patterns that engage attention at
different (local-global) levels relevant to the attribu-
Occipital Cortex tion of animacy and intention. Although this specu-
In the present study, the ToM animations (relative to lation is unsupported with regard to the present ani-
Random) elicited increased bilateral occipital activa- mations, it is amenable to empirical testing.
tion in a lateral area, as was also found in Gallagher Of necessity, the movements in the ToM anima-
et al.’s (2000) study using a mentalizing task involv- tions were more complex in terms of greater varia-
ing static cartoons. In contrast, the reverse compari- tion of speed and direction of movement. It may be
son (Random versus ToM) activated a medial region this greater complexity that results in increased ac-
of occipital cortex. This result indicates a task spe- tivity in extrastriate regions. Thus, it remains possi-
cific effect, not found in other studies of mentalizing, ble that the pattern of activation we attribute to
that deserves further exploration. These areas were mentalising reflects in part extraneous tasks differ-
implicated in recent studies of global and local ences in, for example, psychophysical properties of
processing of complex visual stimuli (Fink et al., the stimuli or resulting eye-movement differences.
1997). In the Fink et al. study (1997a) subjects were Future tests in which psychophysical properties are
presented with large letters made out of small letters, systematically varied, are clearly needed.
and required to switch attention between global and In conclusion, the present study has shown that
local perceptual levels. Attentional modulation be- abstract movement patterns activate regions previ-
tween local and global processing was associated ously associated with mentalizing in stories and
with differential activity in prestriate cortex along static pictures. Our ToM animations revealed
the mediolateral axis. Local processing elicited in- increased activation in a network of brain regions,
creased left lateral activation, whereas global pro- including the medial prefrontal cortex, the tempo-
cessing elicited increased right medial activation. ral pole adjacent to the amygdala region and the
This distinction between lateral and medial occipital temporoparietal junction. All these regions have
regions was replicated in a second study using been repeatedly implicated in previous studies of
RT0996_C12.qxd 11/8/04 2:07 PM Page 166
mental state attribution and might reflect different and thoughts. By just watching them you will
components of this process. Two particularly probably imagine they are interacting, for exam-
important components, paracingulate sulcus and ple, courting each other.
temporoparietal junction, show overlap with previ- In this experiment there is no “right” or “wrong”
ous mentalizing studies as well as studies of self- answer. Sometimes I will tell you in advance what
monitoring and perception of biological motion. kind of animation you are going to see, for exam-
We tentatively suggest that the ability to make in- ple, a random movement, a simple interaction or an
ferences about other people’s mental states evolved interaction involving thoughts and feelings. While
from the ability to make inferences about other you are watching the animations, be relaxed, and
creatures’ actions and movements. This fits with … enjoy them! After each cartoon is over, I will ask
the observation that we commonly infer intentions you what you think the triangles were doing,
on the basis of observed action outcomes. The ac- whether they were randomly moving about, or
tivity of the prefrontal cortex and temporoparietal whether they were doing something more specific.
junction in our study is combined with activity in a
ventral visual pathway, from the extrastriate cortex
to the inferior and middle temporal gyri. Thus the Appendix 2
regions activated by viewing artfully animated tri-
Scoring Criteria and Examples for Verbal
angles appear to reveal a network for processing vi-
Descriptions of Animations
sual-kinetic information about intention in action.
Score (0–5) for Intentionality:
Appendix 1 0 action, nondeliberate
(e.g., “Bouncing,” “Moving around,” “Rotating”)
Instructions Given to Participants 1 deliberate action with no other
The aim of this experiment is to understand which (e.g., “Ice-skating”)
parts of your brain are active while watching a 2 deliberate action with another
short animated film sequence. (e.g., “Blue and red are fighting,” “Parent is
All you have to do is relax, and watch the ani- followed by child”)
mations shown on the monitor in front of you. 3 deliberate action in response to other’s action
Each animation lasts approximately 40 seconds. (e.g., “Big is chasing little,” “Red is allowing
The sequences are similar to one another (two the Blue to get close to him,” “Big is guarding
triangular shapes moving about) but different in little who was trying to escape”)
their content. The triangles act as characters per- 4 deliberate action in response to other’s
forming different movements, for example, danc- mental state
ing, drifting or courting each other. (e.g., “The little one is mocking the big one,”
There are different types of content: In some “Two people are arguing,” “A parent is encour-
animations the behaviour of both triangles will aging a child to go outside”)
appear disconnected from each other. They just 5 deliberate action with goal of affecting
move about, with random movement. other’s mental state
By contrast, other animations will show the two (e.g., “The blue triangle wanted to surprise the
triangles moving about doing something together, red one,” “Child pretending not to be doing
interacting. Their actions are somehow connected anything”)
to each other, for example, they are imitating each
other, or one is feeding the other. Score (0–3) for Appropriateness:
Still other animations show the two triangles 0 no answer, “I don’t know”
doing something more complex together, as if they 1 inappropriate answer: reference to the
are taking into account their reciprocal feelings wrong type of interaction between triangles
RT0996_C12.qxd 11/8/04 2:07 PM Page 167
2 partially correct answer: reference to cor- Score (0–4) for Length:

rect type of interaction but confused overall 0 no response
description 1 one clause
3 appropriate, clear answer 2 two clauses
3 four clauses
Score (0–3) for Certainty (Based on 4 more than four clauses
Voice Tone):
0 long hesitation or silence
1 hesitation, few words, sentences unfin- Appendix 3
ished, need to be prompted to say more
2 hesitation between words, alternative The stills in Figure 12.3 illustrate a “Theory of
answers Mind” animation. The animation was designed fol-
3 no hesitation at all, quick answer, descrip- lowing a script in which Big Triangle is coaxing the
tion correctly reflects the script underlying the reluctant Little Triangle to come out of an enclo-
animation sure. Subjects were presented with the animations
a) b)
Mother shows the child the Child doesn't want to go out

way out
c) d)
Mother persuades child to go out Child explores the outside
e)
Mother and child play

together happily
FIGURE 12.3
RT0996_C12.qxd 11/8/04 2:07 PM Page 168
without any suggestion relative to a story or char- figurative aspects of language. A positron emission tomog-
acters’ roles. The captions have been added here for raphy activation study. Brain 117:1241–1253.
Buchel, C., Price, C., and Friston, K. 1998. A multimodal lan-
clarification. guage region in the central visual pathway. Nature
394:274–277.
Calvert, G. A., Bullmore, E. T., Brammer, M. J., Campbell, R.,
Acknowledgments Williams, S. C., McGuire, P. K., Woodruff, P. W., Iversen,
S. D., and David, A. S. 1997. Activation of auditory cortex
Support from The Wellcome Trust and the MRC during silent lipreading. Science 276:593–596.
is gratefully acknowledged. F. Castelli is sup- Carruthers, P., and Smith, P. K. (Eds.) 1996. Theories of Theo-
ported by the European TMR Marie Curie Re- ries of Mind. Cambridge Univ. Press, Cambridge.
Carter, C. S., Braver, T. S., Barch, D. M., Botvinick, M. M.,
search Training Grant ERBFMBICT972667.
Noll, D., and Cohen, J. D. 1998. Anterior cingulate cortex,
error detection, and the online monitoring of performance.
REFERENCES Science 280:747–749.
Duvernoy, H. M. 1999. The Human Brain: Surface, Three-
Abell, F., Happé, F., and Frith, U. 2000. Do triangles play Dimensional Sectional Anatomy with MRI, and Blood Sup-
tricks? Attribution of mental states to animated shapes in ply. Springer-Wien, New York.
normal and abnormal development. J. Cogn. Dev., 15:1–20. Fink, G. R., Halligan, P. W., Marshall, J. C., Frith, C. D.,
Amaral, D., Price, J. L., Pitkanen, A., and Carmichael, S. T. Frackowiak, R. S., and Dolan, R. J. 1997. Neural mecha-
1992. Anatomical organisation of the primate amygdaloid nisms involved in the processing of global and local as-
cortex. In The Amygdala: Neurobiological Aspects of Emo- pects of hierarchically organized visual stimuli. Brain
tion, Memory and Mental Dysfunction (J. Aggleton, Ed.), 120:1779–1791.
pp. 1–66. Wiley, NY. Fletcher, P. C., Happé, F., Frith, U., Baker, S. C., Dolan, R. J.,
Baron-Cohen, S., Leslie, A. M., and Frith, U. 1985. Does the Frackowiak, R. S. J., and Frith, C. D. 1995. Other minds in
autistic child have a “theory of mind?” Cognition 21:37–46. the brain: A functional imaging study of “theory of mind” in
Baron-Cohen, S. 1995. Mindblindness: An Essay on Autism story comprehension. Cognition 57:109–128.
and Theory of Mind. MIT Press, Cambridge, MA. Fodor, J. A. 1992. A theory of the child’s theory of mind. Cog-
Baron-Cohen, S., Ring, H., Moriarty, J., and Schmitz, B. 1994. nition 44:283–296.
Recognition of mental state terms: Clinical findings in chil- Fox, P. T., and Mintun, M. A. 1989. Non-invasive functional brain
dren with autism and a functional neuro-imaging study of mapping by change-distribution analysis of averaged PET im-
normal adults. Br. J. Psychiatry 165:640–649. ages of H2 15O issue activity. J. Nuclear Med. 30:141–149.
Baron-Cohen, S., Ring, H., Williams, S., Weelwright, S., Bull- Friston, K. J., Worsley, K., Frackowiak, R. S. J., Mazziotta,
more, E., Brammer, M., and Andrew, C. 1999. Social intel- J. C., and Evans, A. C. 1994. Assessing the significance of
ligence in the normal and autistic brain: An fMRI study. focal activations using their spatial extent. Hum. Brain
Eur. J. Psychiatry 11:1891–1898. Mapp. 1:214–220.
Baron-Cohen, S., Tager-Flusberg, H., and Cohen, D.J. (Eds.) Friston, K. J., Ashburner, J., Frith, C. D., Poline, J. B., Heather,
2000. Understanding Other Minds: Perspectives from J. D., and Frackowiak, R. S. J. 1996. Spatial registration and
Autism, 2nd ed. Oxford Univ. Press, Oxford. normalization of images. Hum. Brain Mapp. 2:165–189.
Berry, D. S., Misovich, S. J., Kean, K. J., and Baron, R. M. Friston, K. J., Frith, C. D., Liddle, P. F., Dolan, R. J.,
1992. Effects of disruption of structure and motion on per- Lammertsma, A. A., and Frackowiak, R. S. J. 1990. The re-
ceptions of social causality. Personal. Soc. Psychol. Bull. lationship between global and local changes in PET scans.
18:237–244. J. Cereb. Blood Flow Metab. 10:458–466.
Berry, D. S., and Springer, K. 1993. Structure, motion, and Friston, K. J., Holmes, A. P., Worsley, K. J., Poline, J. B., Frith,
pre-schoolers’ perceptions of social causality. Ecol. Psy- C. D., and Frackowiak, R. S. J. 1995. Statistical parametric
chol. 5:273–283. maps in functional imaging: A General Linear approach.
Blakemore, S. J., Wolpert, D. M., and Frith, C. D. 1998. Cen- Hum. Brain Mapp. 2:189–210.
tral cancellation of self-produced tickle sensation. Nat. Neu- Frith, U., Morton, J., and Leslie, A. M. 1991. The cognitive ba-
rosci. 1:635–639. sis of a biological disorder: autism. Trends Neurosci.
Bonda, E., Petrides, M., Ostry, D., and Evans, A. 1996. Spe- 14:433–438.
cific involvement of human parietal systems and the amyg- Gallagher, H. L., Happé, F., Brunswick, N., Fletcher, P. C., Frith,
dala in the perception of biological motion. J. Neurosci. U., and Frith, C. D. 2000. Reading the mind in cartoons and
16:3737–3744. stories: An fMRI study of Theory of Mind in verbal and non-
Bottini, G., Corcoran, R., Sterzi, R., Paulesu, E., Schenone, verbal tasks. Neuropsychologia 38:11–21.
P., Scarpa, P., Frackowiak, R. S., and Frith, C. D. 1994. Goel, V., Grafman, J., Tajik, J., Gana, S., and Danto, D. 1998.
The role of the right hemisphere in the interpretation of Modelling other minds. Brain 120:1805–1822.
RT0996_C12.qxd 11/8/04 2:07 PM Page 169
Gorno-Tempini, M. L., Price, C. J., Josephs, O., Vanden- McGuire, P. K., Silbersweig, D. A., and Frith, C. D. 1996a.
berghe, R., Cappa, S. F., Kapur, N., and Frackowiak, R. S. J. Functional neuroanatomy of verbal self-monitoring. Brain
1998. The neural systems sustaining face and proper-name 119:907–917.
processing. Brain 121: 2103–2118. McGuire, P. K., Paulesu, E., Frackowiak, R. S., and Frith,
Grady, C. L. 1999. Neuroimaging and activation of the frontal C. D. 1996b. Brain activity during stimulus independent
lobes. In The Human Frontal Lobes: Functions and Disor- thought. Neuroreport 7:2095–2099.
ders (B. L. Miller and J. L. Cummings, Eds.), pp. 196–230. Nobre, A. C., Allison, T., and McCarthy, G. 1994. Word recog-
Guilford Press, NY. nition in the human inferior temporal lobe. Nature
Grezes, J., Costes, N., and Decety, J. 1998. Top-down effect of 372:260–263.
strategy on the perception of human biological motion: Paus, T., Koski, L., Caramanos, Z., and Westbury, C. 1998. Re-
A PET investigation. Cogn. Neuropsychol. 15:553–582. gional differences in the effects of task difficulty and motor
Happé, F. 1993. Communicative competence and theory of output on blood flow response in the human anterior cingu-
mind in autism: A test of Relevance theory. Cognition late cortex. NeuroReport 9:35–45.
48:101–119. Price, C. J., Moore, C. J., Humphreys, G. W., and Wise, R. J. S.
Happé, F., Ehlers, S., Fletcher, P., Frith, U., Johansson, M., Gill- 1997. Segregating semantic from phonological processes
berg, C., Dolan, R., Frackowiak, R., and Frith, C. D. 1996. during reading. J. Cogn. Neurosci. 9:727–733.
Theory of Mind in the brain: Evidence from a PET scan study Puce, A., Allison, T., Asgari, M., Gore, J. C., and McCarthy,
of Asperger syndrome. NeuroReport 8:197–201. G. 1996. Differential sensitivity of human visual cortex to
Happé, F., and Frith, U. 1996. The neuropsychology of autism. faces, letter strings, and textures: A functional magnetic res-
Brain 119:1377–1400. onance imaging study. J. Neurosci. 16:5205–5215.
Heider, F., and Simmel, M. 1944. An experimental study of Puce, A., Allison, T., Bentin, S., Gore, J. C., and McCarthy,
apparent behavior. Am. J. Psychol. 57:243–259. G. 1998. Temporal cortex activation in humans viewing eye
Kanwisher, N., McDermott, J., and Chun, M. M. 1997. The and mouth movements. J. Neurosci. 18:2188–2199.
fusiform face area: A module in human extrastriate cortex Rainville, P., Duncan, G. H., Price, D. D., Carrier, B., and
specialized for face perception. J. Neurosci. 17:4302–4311. Bushnell, M. C. 1997. Pain affect encoded in human anterior
Klin, A., Schultz, R., and Cohen, D. J. 2000. Theory of Mind in cingulate but not somatosensory cortex. Science 277:
action: Developmental perspectives on social neuroscience. In 968–971.
Understanding Other Minds. Perspectives from Developmen- Scholl, B. J., and Leslie, A. M. 1999. Modularity, development
tal Cognitive Neuroscience (S. Baron-Cohen, H. Tager-Flus- and “theory of mind.” Mind Lang. 14:131–153.
berg, and D. J. Cohen, Eds.), 2nd ed., pp. 357–388. Oxford Sperber, D., and Wilson, D. 1986. Relevance: Communication
Univ. Press, Oxford. and Cognition. Blackwell, Oxford.
Lane, R. D., Fink, G. R., Chau, P. M., and Dolan, R. J. 1997. Talairach, J., and Tournoux, P. 1988. A Co-planar Stereotaxic
Neural activation during selective attention to subjective Atlas of a Human Brain. Thieme-Verlag, Stuttgart.
emotional responses. Neuroreport 8:3969–3972. Townsend, D. W., Geissbuller, A., Defrise, M., Hoffman, E. J.,
Leslie, A. M., and Thaiss, L. 1992. Domain specificity in con- Spinks, T. J., Bailey, D. L. et al. 1991. Fully three-dimen-
ceptual development: Neuropsychological evidence from sional reconstruction for a PET camera with retractable
autism. Cognition 43:225–251. septa. IEEE Trans. Med. Eng. 10:505–512.
Mazoyer, B. M., Tzourio, N., Frak, V., Syrota, A., Murayama, Vandenberghe, R., Price, C., Wise R., Josephs, O., and Frack-
N., Levrier, O., Salamon, G., Dehaene, S., Cohen, L., and owiak, R. S. J. 1996. Functional anatomy of a common
Mehler, J. 1993. The cortical representation of speech. semantic system for words and pictures. Nature 383:
J. Cogn. Neurosci. 5:467–479. 254–256.
RT0996_C12.qxd 11/8/04 2:07 PM Page 170
RT0996_C13.qxd 11/8/04 2:07 PM Page 171
R E A D I N G 1 3
People Thinking about Thinking People: The Role of

the Temporo-Parietal Junction in “Theory of Mind”
•
R. Saxea and N. Kanwishera,b
Humans powerfully and flexibly interpret the behaviour of other people based on an understanding of
their minds: that is, we use a “theory of mind.” In this study we distinguish theory of mind, which
represents another person’s mental states, from a representation of the simple presence of another
person per se. The studies reported here establish for the first time that a region in the human temporo-
parietal junction (here called the TPJ-M) is involved specifically in reasoning about the contents of
another person’s mind. First, the TPJ-M was doubly dissociated from the nearby extrastriate body area
(EBA; Downing et al., 2001). Second, the TPJ-M does not respond to false representations in non-
social control stories. Third, the BOLD response in the TPJ-M bilaterally was higher when subjects read
stories about a character’s mental states, compared with stories that described people in physical detail,
which did not differ from stories about nonhuman objects. Thus, the role of the TPJ-M in
understanding other people appears to be specific to reasoning about the content of mental states.
Keywords: fMRI; Social cognitive neuroscience; False belief; Mentalising; Superior temporal
sulcus; EBA.
Tsonhenition
remarkable human facility with social cog-
depends on a fundamental ability to rea-
about other people. Specifically, we predict
and interpret the behaviour of people based on an
understanding of their minds: that is, we use a
“theory of mind.”1 In this study we show that a
a
Department of Brain and Cognitive Sciences, Massachusetts 1985; Wellman and Gelman, 1992). For discussions about
Institute of Technology, Cambridge, MA, USA the so-called theory-theory, see Carruthers and Smith, 1996,
b
McGovern Institute for Brain Research, Massachusetts Insti- and Malle et al., 2001. In this study, we use the term theory
tute of Technology, Cambridge, MA, USA of mind in a broader sense, to refer to any reasoning about
1
The term “theory of mind” has a more restricted sense, re- another person’s representational mental states (also called
ferring to the suggestion that the structure of knowledge in “belief-desire psychology,” e.g., Bartsch and Wellman,
the mind is analogous to a scientific theory (e.g., Carey, 1995).
171
RT0996_C13.qxd 11/8/04 2:07 PM Page 172
region of human temporo-parietal junction is se- provide a useful behavioural test of a ToM, be-
lectively involved in reasoning about the contents cause when the belief is false, the action predicted
of other people’s minds. by the belief is different from the action that would
Brain regions near the temporo-parietal junc- be predicted by the true state of affairs (Dennett,
tion (TPJ) have been implicated in a broad range 1978). Note, though, that everyday reasoning
of social cognition tasks (Allison et al., 2000; about other minds, by adults and children, depends
Gallagher and Frith, 2003; Greene and Haidt, on attributions of mostly true beliefs (e.g., Dennett,
2003). Regions near the TPJ have preferential 1996; Bartsch and Wellman, 1995).
responses to human faces (e.g., Hoffman and Previous investigations of the neural correlates
Haxby, 2000), bodies (e.g., Downing et al., 2001) of ToM (Fletcher et al., 1995; Gallagher et al.,
and biological motion (e.g., Grossman et al., 2000) have compared false belief (“theory of
2000). There is also some evidence that regions mind”) stories with two control conditions: “non-
within human TPJ are involved in theory of mind theory of mind stories,” which describe actions
(ToM). A number of studies have reported in- based on the character’s true beliefs, and “con-
creased responses in the TPJ when subjects read trol” stories, consisting of unrelated sentences.
verbal stories or see pictorial cartoons that re- These authors found that the TPJ response was
quire inferences about a character’s (false) be- high during theory of mind stories, but was also
liefs, compared with physical control stimuli high during non-theory of mind stories. They
(Fletcher et al. 1995; Brunet et al. 2000; Gallagher concluded (see also Gallagher and Frith, 2003)
et al. 2000; Castelli et al. 2000; Vogeley et al. that the TPJ is not selectively involved in ToM.
2001. A number of other brain regions have also This conclusion does not follow. Because the
been implicated in theory of mind; see reviews by non-theory of mind stories invite inferences
Gallagher and Frith, 2003, and Greene and Haidt, about the character’s (true) beliefs, a region in-
2003). volved in reasoning about other minds should
What is the role of the TPJ in these tasks? ToM show a high response to these stories, as well as
reasoning depends upon at least two kinds of rep- to the so-called theory of mind stories. (For an
resentation: a representation of another person argument against the use of unrelated sentences
per se and a representation of that other person’s as the baseline condition, see Ferstl and von
mental states (see Leslie, 1999). While a repre- Cramon, 2002.)
sentation of a person per se is a likely prerequisite We propose two basic tests for a region selec-
for ToM, achieving a representation of others’ tively involved in ToM reasoning. First, it must
mental states is the core responsibility of a ToM. show increased response to tasks/stimuli that
Some authors suggest that the TPJ is involved invite ToM reasoning (about true or false beliefs)
only in the preliminary stages of social cognition compared with logically similar non-social con-
that “aid” ToM, not in ToM reasoning itself (e.g., trols. Second, the region must respond not just
Gallagher and Frith, 2003). We provide here when a person is present in the stimulus, but
evidence against this suggestion, and argue on the specifically when subjects reason about the
contrary that a region of the TPJ is selectively in- person’s mental states. Below, we provide evi-
volved in representation of other peoples’ mental dence that a subregion of the TPJ, here called the
states. TPJ-M, passes both these criteria for a selective
Neuroimaging studies have followed develop- role in ToM.
mental psychology in using “false belief” stories
as the prototypical problem for ToM reasoning Experiment 1
(Fletcher et al., 1995; Gallagher et al., 2000; see
also Vogeley et al., 2001). In these scenarios, a We devised a new version of the false belief sto-
character’s action is based on the character’s false ries task (Fletcher et al., 1995) to compare reason-
belief (Wimmer and Perner, 1983). False beliefs ing about true and false beliefs to reasoning about
RT0996_C13.qxd 11/8/04 2:07 PM Page 173
People Thinking about Thinking People ■ 173
non-social control situations. ToM stories de- Stories were constructed to fit four categories:
scribed a character’s action caused by his/her false belief, mechanical inference, human action,
false belief. Descriptions of human actions re- and nonhuman descriptions (Appendix 1). Each
quired analysis of mental causes, in the absence of story was presented for 9500 ms, followed by a
false beliefs. We compared these conditions to 500-ms interstimulus interval. Each scan lasted
two non-social control conditions, (1) mechanical 260 s: four 40-s epochs, each containing four sto-
inference control stories, which required the sub- ries (one from each condition), and 20 s of fixa-
ject to infer a hidden physical (as opposed to mention between epochs. The order of conditions was
tal) process, such as melting or rusting (for exam- counterbalanced within and across runs. Subjects
ples, see Appendix 1), and (2) descriptions of were asked to press a button to indicate when they
nonhuman objects. had finished reading each story. Subjects read a
Unlike previous studies, we did not cue or in- total of 8 (4 subjects) or 12 (21 subjects) stories
struct subjects to attend specifically to mental per condition.
states. With this design we were able to look for Fourteen of the subjects from Experiment 1
regions of cortex in individual subjects that are se- (7 women) were also scanned on an EBA local-
lectively and spontaneously involved in under- izer in the same scan session, all at 3.0 T. Stimuli
standing the mental (as opposed to physical) consisted of 20 grayscale photographs of whole
causes of events. human bodies (including faces) in a range of pos-
To test whether the response to ToM stories was tures, standing and sitting, and 20 photographs of
a response to the presence of a person in the stim- easily recognizable inanimate objects (e.g., car,
ulus, we presented still photographs of people, drum, tulip). (Two other conditions, cropped faces
and nonhuman objects. Downing et al. (2001) and scrambled objects, were included in the scan
reported a bilateral region near the posterior supe- but were not analyzed here.)
rior temporal cortex that responds preferentially Image presentation followed the blocked de-
to the visual appearance of human bodies, com- sign described in Tong et al. (2000; Experiment 1)
pared with a range of control objects (the extras- except that images were presented at a rate of one
triate body area, EBA). We tested directly the every 800 ms (stimulus duration 500 ms, inter-
functional and anatomical relationship between stimulus interval 300 ms), and each scan lasted
the EBA and the (proposed) TPJ-M. 336 s. Subjects performed a one-back matching
task (Tong et al., 2000).
Methods MRI data were analyzed using SPM 99, FS-
Twenty-five healthy right-handed adults (12 women) fast, and in-house software.
volunteered or participated for payment. All subjects
had normal or corrected-to-normal vision and gave Results
informed consent to participate in the study. Average reading times for theory of mind and me-
Subjects were scanned in the Siemens 1.5- chanical inference stories did not differ signifi-
(9 subjects) and 3.0-T (16 subjects) scanners at cantly (ToM 6.4 s, MI 6.5 s, P 0.2).
the MGH-NMR center in Charlestown, MA, us- Random effects analyses of 25 subjects re-
ing a head coil. Standard echoplanar imaging pro- vealed five loci of greater activation during the
cedures were used [TR 2 s, TE 40 (3 T) or 30 theory of mind compared with mechanical infer-
(1.5 T) ms, flip angle 90º]. Twenty 5-mm-thick ence stories (P 0.05 corrected for multiple spa-
near-coronal slices (parallel to the brainstem) tial hypotheses): left and right TPJ-M, left and
covered the occipital lobe and the posterior por- right anterior superior temporal sulcus (aSTS),
tion of the temporal and parietal lobes. and precuneus (Table 13.1, Figure 13.1). [Consis-
Stimuli consisted of short center-justified sto- tent with many previous studies (e.g., Gusnard
ries, presented in 24-point white text on a black and Raichle, 2001; Raichle et al., 2000) the pre-
background (average number of words 36). cuneus was deactivated (BOLD signal less than
RT0996_C13.qxd 11/8/04 2:07 PM Page 174
TABLE 13.1. Experiment 1a The same pattern of results was apparent in in-
No. of voxels
dividual subjects (fixed effects P 0.0001, un-
MNI coordinate (P 0.05, corrected for all results reported here). Voxels
Region (max voxel) Z corrected) more responsive during ToM than mechanical in-
LTPJ-M [54 60 21] 5.88 63
ference stories were observed at the TPJ in 22 of
LaSTS [57 27 12] 5.40 55 25 subjects (bilaterally in 14, left in 5, and right in
Prec [9 51 33] 5.20 41 3 subjects). The aSTS activation was significant at
R TPJ-M [51 54 27] 5.10 10
R aSTS [66 18 15] 4.91 2
this level in 10 of 25 subjects. Because the TPJ-M
a
was most consistent across subjects and was the
Five regions showed increased signal during theory of mind,
compared with mechanical inference, stories (random effects, focus of our prior hypotheses, we concentrated on
n 25, P 0.05): left and right temporo-parietal junction this region in the subsequent analyses.
(TPJ-M), left and right anterior superior temporal sulcus (aSTS),
and precuneus (Prec). All coordinates are according to the We defined TPJ-M regions of interest (ROI) in
Montreal Neurological Institute standard brain. the left and right TPJ in each individual subject as
contiguous voxels in each hemisphere that were
fixation baseline) during all of our story condi- more active (P 0.0001) during false belief than
tions. The ToM stories deactivated the precuneus mechanical inference stories. The TPJ-M bilater-
less than mechanical inference stories. It was ally generalized beyond false beliefs, responding
therefore unclear whether this effect should be significantly more to human action (HA) stories
considered a response to ToM or to mechanical than to nonhuman descriptions [N-H D; paired
inference stories, and the precuneus response was samples t tests, right: HA average percent signal
not analyzed further.] change from fixation (PSC): 0.22, N-H D average
FIGURE 13.1. ■ (A color version of this figure follows page 146.) Experi-
ment 1. Random effects analysis, P 0.05, corrected, n 25. Theory of
mind mechanical inference stories. Crosshair marks the most significant
voxel in the left TPJ (1). Also visible are activations in right TPJ (2), left aSTS
(3), and precuneus (4). TPJ, temporo-parietal junction; aSTS, anterior supe-
rior temporal sulcus.
RT0996_C13.qxd 11/8/04 2:07 PM Page 175
PSC: 0.02, P 0.0001; left: HA average PSC: Downing et al., 2001). Both right and left EBA
0.35, N-H D average PSC: 0.10, P 0.0001]. ROIs failed to discriminate between any story
In the 14 subjects who also had an EBA local- conditions (paired samples t tests, all P 0.4, all
izer scan, EBA ROIs were defined as the cluster of story PSCs below 0.01; Figure 13.2).
contiguous voxels in extrastriate cortex (bilater- TPJ-M response to photographs was lateral-
ally in 13 subjects and right-only in 1 subject) that ized. The left TPJ-M did not discriminate between
was more active (P 0.0001) during pictures of photographs of people (PSC: 0.04) and of ob-
human bodies than during pictures of nonhuman jects (PSC: 0.09, paired samples t test, P 0.4).
objects in each individual subject (following The right TPJ-M showed a trend toward a greater
a)
0.9
0.8
0.7
% signal change
0.6 ToM
0.5 MI
0.4
0.3
0.2 Body
0.1 Obj
0
–0.1
–0.2
Left TPJ-M Right TPJ-M
b)
1.4
1.2
% signal change
1 ToM
MI
0.8
0.6
Body
0.4 Obj
0.2
0
Left EBA Right EBA
FIGURE 13.2 ■ Experiment 1. Average percent signal change from

fixation in (a) left and right TPJ-M and (b) left and right EBA, defined
in individual subjects (n 14). The EBA consisted of contiguous
voxels in bilateral extrastriate cortex that responded significantly
more to pictures of human bodies than pictures of nonhuman objects
(P 0.0001, uncorrected). The TPJ-M consisted of contiguous
voxels near the temporo-parietal junction that responded significantly
more to theory of mind (ToM) stories than to mechanical inference
(MI) stories (P 0.0001, uncorrected). (Response magnitudes for
the conditions that were used to define the regions of interest are
illustrative only.) The EBA did not respond to story stimuli. The right
TPJ-M differentiated between pictures of bodies and of objects
(P 0.05, paired samples t test), but the left TPJ-M did not.
ToM theory of mind (false belief) stories; MI mechanical infer-
ence stories, Body photographs of human bodies, Obj photo-
graphs of nonhuman objects; EBA, extrastriate body area.
RT0996_C13.qxd 11/8/04 2:07 PM Page 176
response to photographs of people (PSC: 0.24) nor mental states. In Experiment 2, we asked
than of objects (PSC: 0.10, paired samples t test, which of these two components was responsible
P 0.10). A repeated-measures ANOVA of con- for the observed activation. We directly compared
tent (person versus object) by stimulus modality the response of the TPJ-M to stories about people
(stories versus photograph) by hemisphere (right that did (desires) or did not (physical people) re-
versus left) revealed a main effect of person ob- quire inferences based on mental states.
ject (P 0.001) and of stories photographs Also, while they were controlled for difficulty
(P 0.05) modulated by an interaction between and causal structure, the logical structure of the
stimulus modality and hemisphere (response to ToM stories used in Experiment 1 (and previous
photographs only on the right, P 0.005) and a studies) differed systematically from the control
trend toward a three-way interaction (the right stories: only the false belief stories require the no-
TPJ-M response distinguishes photographs of tion of a false representation, in this case a false
bodies and objects more than the left TPJ-M, belief. This confounding factor was perceived by
P 0.1; Figure 13.2). developmental psychologists, who invented its
solution: “false photograph” stories (Zaitchik,
Discussion 1990), which require subjects to represent the
Experiment 1 thus shows an increased BOLD re- (false) content of a physical representation such
sponse in a region of the TPJ bilaterally, here as a photograph or map.
called the TPJ-M, during ToM compared with For Experiment 2, we therefore created five
mechanical inference stories. This activation is new sets of stories (for examples, see Appendix 2):
robust and reliable across individual subjects. (1) false belief stories, (2) false photograph sto-
This finding replicates the earlier reports with a ries, (3) desires, (4) inanimate descriptions, and
new set of stimuli, a less biased task (no cues), (5) physical people. Desire stories described a
and with more stringent statistical tests (both indi- character’s goals or intentions and thus rely on
vidual subject analyses and random effects group ToM. Nonhuman description stories consisted of
analyses). Our results confirm that the TPJ-M re- short descriptions of nonhuman objects such as
sponse to verbal descriptions generalizes to hu- plants, cars, or planets. Physical people stories
man actions based on true beliefs. were short descriptions of people from a purely
Importantly, we distinguished the TPJ-M from physical perspective: clothing, hair colour, facial
its neighbour, the EBA, which did not respond to markings, and so on.
any verbal story conditions. However, the TPJ-M We predicted that regions specifically involved
response to nonverbal social stimuli appeared to in ToM should have an equally low response in the
be lateralized. The left TPJ-M response was selec- nonhuman description and (critically) physical
tive for verbal descriptions, while the right TPJ-M people conditions, and a higher BOLD response
activation may generalize to nonverbal stimuli, in the desire condition. By contrast, regions in-
such as photographs. volved in processing any other representation of
other people would show a high BOLD signal for
the physical people condition.
Experiment 2 Methods
The results of Experiment 1 established that bilat- Twenty-one naive right-handed subjects (11 women)
eral regions near the TPJ show a greater increase were scanned at 1.5 T, using twenty 5-mm-thick
in BOLD signal when subjects reason about oth- axial slices that covered the whole brain. An addi-
ers’ mental states, than when they reason about tional 7 subjects from Experiment 1 (4 women)
nonhuman objects. However, in Experiment 1, also participated in part of Experiment 2. All were
stories involving people and mental states were scanned at 3.0 T using twenty 5-mm-thick near-
compared with stories that involve neither people coronal slices (parallel to the brainstem) covering
RT0996_C13.qxd 11/8/04 2:07 PM Page 177
most of the occipital lobe and the posterior portion false belief compared with false photograph sto-
of the temporal and parietal lobes. ries (P 0.0001, uncorrected) at the TPJ bilater-
Story stimuli consisted of 70 stories (12 each ally [right: (54 51 18), left: (48 63 33)], pre-
of false belief, false photograph, desire, physi- cuneus/posterior cingulate [(3 54 30)], right
cal description, and nonhuman description, av- anterior superior temporal sulcus [(54 18 15)],
erage number of words 32; see Appendix 2). and in medial superior frontal gyrus [(6 57 18)] in
After each story a two-alternative forced choice the frontal pole (Figure 13.2). Medial prefrontal
“fill-in-the-blank” question was presented for cortex has repeatedly been implicated in ToM pro-
4 s. The question consisted of a single sentence cessing, both in neuroimaging and in lesion stud-
with a word missing, presented above two alter- ies (e.g., Rowe et al., 2001; Stuss et al., 2001).
native completions on the left and right side of For the 7 subjects who were scanned in both
the screen. Subjects pressed the left-hand re- Experiments 1 and 2, two additional analyses
sponse button if the word on the left completed were conducted to confirm that the TPJ-M was
the sentence to fit the story, and the right-hand consistent across experiments. First, in all 7 sub-
button to choose the word on the right. Fifty jects the TPJ-M defined in Experiment 1 over-
percent of the false belief, false photograph, and lapped strikingly with TPJ-M defined by the con-
desire story questions probed the character’s trast of false belief (FB) versus false photograph
mental states; the other 50% probed the actual (FP) stories in Experiment 2. Figure 13.3a shows
outcome, to prevent formulaic response prepa- the overlap in a typical individual subject of the
ration. Subjects were given three practice trials TPJ-M defined by these two tasks. Second, this
before going into the scanner: two false belief overlap was confirmed with a functional ROI
trials, and one false photograph trial. analysis. Voxels near the TPJ are more active dur-
Fourteen subjects (including the 7 from Exper- ing ToM than mechanical inference stories in
iment 1) were tested on only false belief and false these individual subjects in Experiment 1 (P
photograph stories. For these subjects, each run 0.0001, uncorrected) were probed for their re-
lasted 204 s and consisted of six blocks [each con- sponse during Experiment 2. This independent
taining 1 story (10 s) and 1 question (4 s)], alter- ROI showed a much greater response to false be-
nating between the two conditions; there were lief than false photograph stories in Experiment 2
three blocks per condition per run. The remaining (mean FB PSC 1.6, mean FP PSC 0.7, t test
14 subjects were tested on all five conditions. P 0.02; Figure 13.3b). The reliability of the
Each run lasted 272 and consisted of 10 blocks TPJ-M across experiments makes it unlikely that
[each containing 1 story (10 s) and 1 question the results of Experiment 1 were the result of
(4 s)]. There were two blocks per condition per run. stimulus confounds or logical differences be-
Fixations of 12 s were interleaved between tween conditions.
blocks. The order of conditions was counterbal- For the 14 subjects who saw all five conditions,
anced across runs. Behavioural data were col- the fMRI data were further analyzed within indi-
lected during the scan. vidually defined functional regions of interest
(ROI) that included all voxels that met two crite-
Results ria, i.e., they were significantly more active in at
Subjects were slower when responding to ques- least half of the individual subjects during false
tions about false photograph than false belief sto- belief than false photograph stories (P 0.0001,
ries (FB: 2.6 vs. FP: 2.8 s, P 0.01), making it uncorrected), and they fell within a sphere of
unlikely that false belief inferences were simply 15-mm radius centered on the most significant
more difficult. voxel of clusters identified in the random effects
As predicted, a random effects analysis on the group analysis (P 0.0001, uncorrected) of the
21 subjects who underwent whole brain scanning same contrast. Using these criteria, we identified
revealed regions of increased BOLD signal to ROIs in the TPJ-M bilaterally and right aSTS.
RT0996_C13.qxd 11/8/04 2:07 PM Page 178
a)
b)
FIGURE 13.3 ■ (A color version of this figure follows page 146.)

(a) Experiments 1 and 2. Activation overlap within an individual sub-
ject showing bilateral temporo-parietal junction (bilateral TPJ) and
precuneus regions (fixed effects, P 0.001). Red theory of mind
mechanical inference (Exp. 1). Blue false belief false photo
(Exp. 2). Green both. (b) Single subject time course of response
during Experiment 2 to false belief (dark gray) and false photograph
(white) stories in the same subject’s TPJ-M, independently defined
by a greater response to theory of mind than to mechanical infer-
ence stories in Experiment 1; P 0.0001, uncorrected. Medium
gray indicates fixation. Time course averaged over four runs.
In the TPJ-M and the right aSTS, the BOLD medial prefrontal (3 57 39), and right lateral
signal change during desire stories was signifi- frontal cortex (39 15 54)].
cantly greater than during either physical people
or nonhuman description stories (both paired Discussion
samples t tests P 0.05), which did not differ The results of Experiment 2 confirm that the TPJ-
from each other (Figure 13.4). The left and right M shows an increased response to stimuli that in-
TPJ-M did not differ. Thus, these regions are not vite ToM reasoning compared with logically sim-
involved in the detection of any person in verbal ilar nonsocial controls (false photograph stories).
stories, but respond selectively to stories in which Second, the TPJ-M does not show an increased re-
describe (or imply) characters’ mental states. Did sponse to the mere presence of a person in the
any regions show the predicted profile of a re- stimulus (physical people stories). The right and
sponse to a person per se? At a lower threshold, a left TPJ-M responses to physical people stories
separate whole brain analysis (P 0.001, uncor- did not differ, thus resolving the ambiguity of the
rected) of physical people nonhuman descrip- apparently lateralized response to photographs of
tions revealed regions of frontal cortex [dorsal bodies in Experiment 1.
RT0996_C13.qxd 11/8/04 2:07 PM Page 179
0.40 False Belief

False Photo
0.30 Desire
Non-human Description
% signal change 0.20
Physical People
0.10
0.00
–0.10
–0.20
–0.30
Left TPJ-M Right TPJ-M
FIGURE 13.4 ■ Experiment 2. Average percent signal change in left and right
TPJ-M, defined in individual subjects (n 14) as voxels that respond signifi-
cantly more to false belief (FB) than to false photo (FP) stories (P 0.0001,
uncorrected. Response magnitude for these two conditions is illustrative only,
since these data were used to determine the region of interest). In the TPJ-M
bilaterally the BOLD response to physical people stories was significantly
lower than to desire stories (P 0.05), and not significantly different from
nonhuman description stories (P 0.1, repeated-analysis of variance).
Response decreases are commonly observed in the TPJ vicinity during de-
manding nonsocial tasks (Shulman et al., 1997; Gusnard and Raichle, 2001).
General Discussion appearance of human bodies. Previous studies

(Fletcher et al., 1995; Gallagher et al., 2000) have
In two experiments we found greater BOLD re- included “physical” stories describing acting peo-
sponse in a region within the TPJ bilaterally (here ple, which produced greater activation in the TPJ
called TPJ-M) while subjects read stories that de- than a scrambled sentence control. Our data show
scribe or imply a character’s goals and beliefs that the TPJ-M response was no greater to stories
than during stories about nonhuman objects. This that described other people in physical detail than
pattern is robust across subjects, tasks, and stim- that to stories describing the physical details of
uli, and is not merely an effect of the difficulty or non-human objects—and was significantly lower
logical structure of false belief stories, since the than to stories that did invite a mental state inter-
TPJ-M did not respond to the more difficult and pretation (desire stories).
logically similar false photograph stories. Could the TPJ-M activation reflect mental im-
We asked whether the TPJ-M represents the agery of the biological motion or goal-directed ac-
simple presence of another person (possibly via tion described in the false belief, human action, and
detecting a human body and/or biological motion) desire stories? We think this is unlikely. Saxe, R.,
or is involved specifically in ToM. We found that Xiao, D.K., Kovacs, G., Perrett, D., and Kanwisher,
the TPJ-M was anatomically and functionally dis- N. (unpublished data) found that the TPJ-M re-
tinct from the nearby EBA (Downing et al., 2001), sponse to a movie of a walking person was much
which responded preferentially to the visual ap- lower than its response to false belief stories. If the
pearance of human bodies, suggesting the pres- response of the TPJ-M to verbal stories was merely
ence of at least two distinct regions involved in a consequence of subjects’ imagining biological
social information processing. motion, we would predict the opposite. Also the
A key innovation of this study over previous TPJ-M was doubly dissociated from its neighbour,
studies was the inclusion in Experiment 2 of phys- the pSTS-VA (visual analysis of action), which
ical people stories, which described the physical responded more to the movies than to verbal stories.
RT0996_C13.qxd 11/8/04 2:07 PM Page 180
In all, our results show that a region of the TPJ2 even strips. Then he goes out to buy flour. His
is involved in reasoning about other minds, not mother comes home and throws all the newspaper
just in understanding stories involving people per strips away.
se (Gallagher and Frith, 2003; p 80). But criti-
cally, neighbouring subregions of cortex have dif- Mechanical inference (MI) sample story
ferent functional profiles, highlighting the neces- A pot of water was left on low heat yesterday in
sity of careful within-subject comparisons. The case anybody wanted tea. The pot stayed on the
TPJ-M, identified here by responses to (false) be- heat all night. Nobody did drink tea, but this
lief stories, may play a broad role in social and morning, the water was gone.
even moral cognition (Moll et al., 2002; Greene
and Haidt, 2003). Human action sample story
Jane is walking to work this morning through a
Acknowledgments very industrial area. In one place the crane is tak-
ing up the whole sidewalk. To get to her building,
This work was funded by grants NEI 13455 and she has to take a detour.
NIHM 66696. Our thanks especially to Yuhong
Jiang for comments and conversation, and to Ben Experiment 2
Balas, Robb Rutledge, Miles Shuman, and Amal
Instructions: “Please read each story carefully.
Dorai for help with data collection and analysis.
After each story, you will be given one fill-in-the-
blanks question about the story. Underneath will
Appendix be two words that could fill in the blank. Choose
the correct word (to make the sentence true in
Experiment 1 the story) by pressing the left button to choose the
Instructions: “Read each story silently to yourself. left-hand word, and the right button to choose the
Please make sure you understand what is happen- right-hand word.”
ing; it is more important that you understand the
story, than that you go as fast as possible. When False belief (FB) sample story
you are done reading the story, press the button.” John told Emily that he had a Porsche. Actually,
his car is a Ford. Emily doesn’t know anything
Theory of mind (ToM) sample story about cars though, so she believed John.
A boy is making a paper mache project for his art —
class. He spends hours ripping newspaper into When Emily sees John’s car she thinks it is a
Porsche Ford
2
What is the relationship between the TPJ-M and attention?
False photograph (FP) sample story
Selective attention leads to increases in regions of the TPJ dur-
ing social perception tasks (e.g., Narumoto et al., 2001; A photograph was taken of an apple hanging on a
Winston et al., 2002), and to decreases in regions of the TPJ tree branch. The film took half an hour to develop.
during visual attention tasks (Shulman et al., 1997; Gusnard In the meantime, a strong wind blew the apple to
and Raichle, 2001; Jiang Y., Kanwisher, N., unpublished data).
Downar et al. (2001) proposed “a role for the TPJ in detecting
the ground.
behaviourally relevant events in the sensory environment”
—
(p. 1256) that is interfered with by demanding visual attention. The developed photograph shows the apple on the
One possibility is that the mental states of other people consti- ground branch
tute a particular category of such “behaviourally relevant”
stimuli. Alternatively, these results may reflect functionally Desire sample story
and anatomically distinct subregions within the TPJ. Direct
testing of the relationship between the TPJ-M and selective at- For Susie’s birthday, her parents decided to have a
tention is an important avenue for future work. picnic in the park. They wanted ponies and games
RT0996_C13.qxd 11/8/04 2:07 PM Page 181
on the lawn. If it rained, the children would have auditory stimuli: an event-related fMRI study. Neuroimage
to play inside. 14, 1256–1267.
— Downing, P.E., Jiang, Y., et al., 2001. A cortical area selective
for visual processing of the human body. Science 293,
Susie’s parents wanted to have her birthday 2470–2473.
inside outside Ferstl, E.C., von Cramon, D.Y., 2002. What does the fronto-
median cortex contribute to language processing: coherence
Physical people sample story or theory of mind? Neuroimage 17, 1599–1612.
Fletcher, P.C., Happe, F., et al., 1995. Other minds in the brain:
Emily was always the tallest kid in her class. In a functional imaging study of “theory of mind” in story
kindergarten she was already over 4 feet tall. Now comprehension. Cognition 57, 109–128.
that she is in college she is 6´4.” She is a head Frith, C.D., Frith, U., 1999. Interacting minds—a biological
taller than the others. basis. Science 286, 1692–1695.
— Gallagher, H.L., Frith, C.D., 2003. Functional imaging of
In kindergarten Emily was over “theory of mind.” Trends Cogn. Sci. 7, 77–83.
Gallagher, H.L., Happe, F., et al., 2000. Reading the mind in
4 ft 6ft cartoons and stories: an fMRI study of “theory of mind” in
… tall. verbal and nonverbal tasks. Neuropsychologia 38, 11–21.
Greene, J., Haidt, J., 2003. How (and where) does moral
Non-human description sample story judgement work? Trends Cogn. Sci. 6, 517–523.
Grossman, E., Donnelly, M., et al., 2000. Brain areas involved
Nine planets and their moons, plus various lumps in perception of biological motion. J. Cogn. Neurosci 12,
of debris called asteroids and comets, make up the 711–720.
sun’s solar system. The earth is one of four rocky Gusnard, D.A., Raichle, M.E., 2001. Searching for a baseline:
planets in the inner solar system. functional imaging and the resting human brain. Nat. Rey.
— Neurosci. 2, 685–694.
Hoffman, E.A., Haxby, J.V., 2000. Distinct representations of
The solar system has
eye gaze and identity in the distributed human neural sys-
four nine tem for face perception. Nat. Neurosci. 3, 80–84.
… planets. Leslie, A., 1999. “Theory of mind” as a mechanism of selec-
tive attention, in: Gazzanigal, M. (Ed.), The New Cognitive
Neurosciences. MIT Press, Cambridge, MA.
Malle, B.F., Moses, L.J., Baldwin, D.A. (Eds.), 2001. Inten-
REFERENCES
tions and Intentionality: foundations of Social Cognition.
Allison, T., Puce, A., et al. 2000. Social perception from MIT Press, Cambridge, MA.
visual cues: role of the STS region. Trends Cogn. Sci. 4, Moll, J., et al., 2002. The neural correlates of moral sensitivity:
267–278. a functional magnetic resonance imaging investigation of
Bartsch, K., Wellman, H., 1995. Children Talk about the Mind. basic and moral emotions. J. Neurosci 22, 2730–2736.
Oxford University Press, New York. Narumoto, J., Okada, T., et al., 2001. Attention to emotion
Brunet, E., Sarfati, Y., et al., 2000. A PET investigation of the modulates fMRI activity in human right superior temporal
attribution of intentions with a nonverbal task. Neuroimage sulcus. Brain Res. Cogn. Brain Res. 12, 225–231.
11, 157–166. Rowe, A.D., Bullock, P.R., et al., 2001. “Theory of mind”
Carey, S., 1985. Conceptual Change in Childhood. MIT Press, impairments and their relationship to executive function-
Cambridge, MA. ing following frontal lobe excisions. Brain 124 (Pt 3),
Carruthers, P., Smith, P., 1996. Theories of Theories of Mind. 600–616.
Cambridge University Press, New York. Saxe, R., Xiao, D.K., et al., 2003. Distinct representations of
Castelli, F., Happe, F., et al., 2000. Movement and mind: a bodies, actions and thoughts in posterior superior temporal
functional imaging study of perception and interpretation of sulcus, submitted.
complex intentional movement patterns. Neuroimage 12, Shulman, G.L., Corbetta, M., et al., 1997. Top-down modula-
314–325. tion of early sensory cortex. Cereb Cortex 7, 193–206.
Dennett, D., 1978. Beliefs about beliefs. Behav. Brain Sci. 1, Stuss, D.T., Gallup Jr., G.G., et al., 2001. The frontal lobes
568–570. are necessary for “theory of mind.” Brain 124 (Pt 2),
Dennett, D., 1996. Kinds of Minds: Toward an Understanding 279–286.
of Consciousness. Basic Books, New York, NY. Tong, F., Nakayama, N., et al., 2000. Response properties of
Downar, J., Crawley, A.P., et al., 2001. The effect of task rele- the human fusiform face area. Cogn. Neuropsychol. 17,
vance on the cortical response to changes in visual and 257–279.
RT0996_C13.qxd 11/8/04 2:07 PM Page 182
Vogeley, K., Bussfeld, P., et al., 2001. Mind reading: neural children’s understanding of deception. Cognition 13,
mechanisms of theory of mind and self-perspective. Neu- 103–128.
roimage 14 (1 Pt 1), 170–181. Winston, J.S., Strange, B.A., et al., 2002. Automatic and in-
Wellman, H.M., Gelman, S., 1992. Cognitive development: tentional brain responses during evaluation of trustworthi-
foundational theories of core domains. Annu. Rev. Psychol. ness of faces. Nat. Neurosci. 5, 277–283.
43, 337–375. Zaitchik, D., 1990. When representations conflict with reality:
Wimmer, H., Perner, J., 1983. Beliefs about beliefs: represen- the pre-schooler’s problem with false beliefs and “false”
tation and constraining function of wrong beliefs in young photographs. Cognition 35, 41–68.
RT0996_P08.qxd 11/8/04 2:14 PM Page 183
PA R T 8
Social Perception and Cognition:

Multiple Routes
•
During most of the past century, the nuances of social cognition and social
processes, including unconscious processes, were plumbed through the
clever experimental designs and measures of verbal reports, judgments,
and reaction time. It has also long been recognized that self-reports and
overt behavior provide incomplete information on the human mind.
Although chronotropic measures provided a means of exploring possible
differences in the components of a series of cognitive operations as they
unfold over time, they, too, could be insensitive to the extent of parallel
processing that might be ongoing. When one adds a desire for parsimony,
or an economy of explanation, it is simple to see how theories in social
psychology tended to take the form of a series of information processing
steps or stages.
The 19th century neurologist John Hughlings Jackson, who cared more
about understanding his patients’ symptomatologies than parsimony, came
to appreciate the redundancy and complexity of the brain and mind. He
further suggested that the central nervous system was structured
hierarchically such that there was a rerepresentation of functions at
multiple levels within this neural hierarchy. Primitive protective responses
to aversive stimuli are organized at the level of the spinal cord, as is
apparent in flexor (pain) withdrawal reflexes that can be seen even after
spinal transection. These primitive protective reactions are expanded and
183
RT0996_P08.qxd 11/8/04 2:14 PM Page 184
embellished at higher levels of the nervous system. under conditions of reduced attention, the amygdala
The evolutionary development of higher neural responded to expressions of fear and disgust in a
systems, such as the limbic system, endowed fashion suggesting coarse automatic processing of
organisms with an expanded behavioral repertoire, social signals which may communicate a threat,
including escape reactions, aggressive responses, but under conditions of normal attention amygdala
and even the ability to anticipate and avoid aversive responds specifically to expressions of fear.
encounters. Evolution not only endowed humans The findings of Anderson et al. (2003) are
with primitive, lower-level adaptive reactions, but consistent with the suggestion that there are two
also sculpted the remarkable information parallel pathways to the amygdala—one that
processing capacities of the highest levels of the provides fast but coarse processing and is
brain. The implication is that neurobehavioral subcortically mediated, and one that provides
mechanisms are not localized to a single level of slower, more refined processing and is cortically
organization within the brain, but rather are mediated. It further follows from this model that the
represented at multiple levels of the nervous system. amygdala should be involved in processing the
Indeed, at progressively higher (more rostral) levels potential threat whether social judgments are
of organization (spinal, brain stem, limbic, cortical performed explicitly or implicitly, whereas cortical
regions) there is a general expansion in the range streams of social information processing should be
and relational complexity of contextual controls and especially likely to emerge when social judgments
in the breadth and flexibility of discriminative and are performed explicitly. The study by Winston et al.
adaptive responses. (2002) speaks directly to this question. Participants
In the first reading in this section, Anderson et al. were told to judge either the trustworthiness of faces
(2003) revisit the neural correlates of face they saw (explicit judgment) or the age of the faces
processing, but show that facial displays (neutral, they saw. Results of the fMRI scans during the task
fear, disgust) communicated different emotional indicated that activity in the amygdala increased to
circumstances. Displays indicating threats in untrustworthy facial displays whether participants
particular have been suggested to trigger increased were judging trustworthiness or age. Brain activation
vigilance and preparatory responses in the observer in, for instance, the superior temporal sulcus, by
largely independent of the observer’s awareness. contrast, was greater when participants made
Anderson et al. (2003) employed functional explicit rather than implicit trustworthiness
magnetic resonance imaging (fMRI) to examine judgments, and this activation was unrelated to the
whether displays of disgust as well as fear would level of trustworthiness of the facial display.
trigger the automatic processing of threat and to Together these studies suggest that there are
examine in more detail the role of the amygdala in multiple processes that may be invoked when
the processing of threat. Their results indicated that processing social signals.
RT0996_C14.qxd 11/8/04 2:08 PM Page 185
R E A D I N G 1 4
Neural Correlates of the Automatic

Processing of Threat Facial Signals
•
Adam K. Anderson,1 Kalina Christoff,1 David Panitz,1
Eve De Rosa,2 and John D.E. Gabrieli1,3
The present study examined whether automaticity, defined here as independence from
attentional modulation, is a fundamental principle of the neural systems specialized for
processing social signals of environmental threat. Attention was focused on either scenes or
faces presented in a single overlapping display. Facial expressions were neutral, fearful, or
disgusted. Amygdala responses to facial expressions of fear, a signifier of potential physical
attack, were not reduced with reduced attention to faces. In contrast, anterior insular responses
to facial expressions of disgust, a signifier of potential physical contamination, were reduced
with reduced attention. However, reduced attention enhanced the amygdala response to disgust
expressions; this enhanced amygdala response to disgust correlated with the magnitude of
attentional reduction in the anterior insular response to disgust. These results suggest that
automaticity is not fundamental to the processing of all facial signals of threat, but is unique to
amygdala processing of fear. Furthermore, amygdala processing of fear was not entirely
automatic, coming at the expense of specificity of response. Amygdala processing is thus
specific to fear only during attended processing, when cortical processing is undiminished, and
more broadly tuned to threat during unattended processing, when cortical processing is
diminished.
Keywords: amygdala; insula; fear; disgust; attention; emotion; faces; fMRI.
Departments of Psychology,1 Psychiatry,2 and Neuroscience,3

Stanford University, Stanford, California 94305.
185
RT0996_C14.qxd 11/8/04 2:08 PM Page 186
Introduction fear responses draw on two distinct pathways to

the amygdala: one pathway cortically and another
Facial expressions serve as important social signals subcortically mediated (LeDoux, 1996; Morris
of imminent environmental conditions. It is now et al., 1999, 2001). By circumventing the cortex, the
known that distinct expressions signaling environ- subcortical pathway may be more rapid and auto-
mental threat draw on distinct neural substrates matic, but should be at the expense of a more
specialized for their evaluation. Patient and neu- detailed cortical analysis of the stimulus (Jarrell
roimaging studies suggest that the amygdala is et al., 1987; LeDoux, 1995). Thus, amygdala auto-
critical for evaluating fearful facial expressions matic processing may be qualitatively distinct from
(Adolphs et al., 1994; Breiter et al., 1996; Morris processing under conditions of full awareness,
et al., 1996; Whalen et al., 1998). Similar evidence occurring at the expense of its specificity for fear.
indicates that the anterior insula, a region of primary To address these issues, the present study used
gustatory cortex substantially connected with the event-related functional magnetic resonance im-
amygdala (Mesulam and Mufson, 1982), is spe- aging (fMRI) to examine how attention influences
cialized for evaluating facial expressions of disgust amygdala and anterior insular processing of fear
(Phillips et al., 1997, 1998; Calder et al., 2000). and disgust. Manipulations of visual attention
The evidence that expressions of fear, a form of result in a pronounced modulation of extrastriate
threat related to physical attack (Gray, 1987), and responses (Corbetta et al., 1990; Haxby et al., 1994;
expressions of disgust, a form of threat related to Wojciulik et al., 1998; O’Craven et al., 1999). If
physical contamination and disease (Rozin and automaticity, defined here as the lack of reduction
Fallon, 1987), draw on specialized brain sub- in activation with reduced attention, is a funda-
strates is one measure of the special informational mental principle of the neural processing of social
status the human brain places on social signals of signals of environmental threat, then lack of atten-
potential environmental threats. Another measure tional modulation should extend to both amygdala
of the special status of social signals of threat is the processing of fear and anterior insular processing
proposal that their processing occurs automatically, of disgust. Furthermore, if automatic processing
proceeding largely independently of attention is qualitatively similar to processing taking place
(Ohman et al., 2001) and awareness (Esteves et al., during full attention, then reduced attention
1994). Evidence for such automaticity has been should not influence the response specificity in
shown by how the amygdala responds to fearful the amygdala and/or anterior insula.
faces during diminished attention (Vuilleumier
et al., 2001, 2002) and awareness (Whalen et al.,
1998). Amygdala activation to fearful faces has Materials and Methods
also been shown in patients with visual neglect
Participants
(Vuilleumier et al., 2002) and in patients with
cortical blindness (Morris et al., 2001). Informed consent to take part in a study approved
However, it is unknown whether automaticity by the Stanford University Panel on Human Sub-
is unique to amygdala fear processing or whether jects in Medical Research was obtained from each
it is a fundamental principle of neural systems subject (three men, nine women; mean age, 22.1
dedicated to threat signals. There is little, if any, years; range, 18–29).
evidence about the attentional properties of the
neural processing of disgust, or any facial expression Stimuli
other than fear. Furthermore, recent challenges to Stimuli consisted of photographs either of fearful,
the preattentive nature of amygdala processing disgusted, or neutral faces superimposed on pictures
(Pessoa et al., 2002a,b) suggest that the precise of places (see Figure 14.1a). For the purposes of
nature of automatic processing in the amygdala is decreasing stimulus repetition, which is thought
unknown. For instance, it has been proposed that to relate to pronounced amygdala habituation
RT0996_C14.qxd 11/8/04 2:08 PM Page 187
Neural Correlates of the Automatic Processing of Threat Facial Signals ■ 187
FIGURE 14.1 ■ (A color version of this figure follows page 146.) Face-place object selection
attention task, a, Example stimulus. Observers were presented with color-coded superim-
posed faces (disgusted, fearful, and neutral expressions in red) and places (inside and out-
side of buildings in green). Before each test stimulus, observers were presented with a color-
coded prompt indicating which task they were to perform on that trial: indicate the gender of
the face (attend trials) or indicate the location of the place (unattend trials). b, A representa-
tive subject demonstrated a greater response when attending to places (in green) in a bilat-
eral region along the collateral sulcus, consistent with the PPA and a greater response when
attending to faces (in red) in the right middle fusiform gyrus, consistent with the FFA.
(Breiter et al., 1996), increasing the number of presented using a magnet-compatible back-projector
unique facial exemplars was emphasized. Facial (Resonance Technology, Van Nuys, CA).
expression stimuli were taken from the Facial
Affect Series and supplemented by additional ap- Task Design
propriately normed exemplars, resulting in three We used an intermixed trial event-related design.
facial expression types for 18 distinct individuals On each trial, participants were first presented
(9 male, 9 female). Place stimuli consisted of pho- with central fixation (1 sec), which was replaced
tos of 18 interiors and 18 exteriors of buildings. by a color-coded prompt (750 msec) that indicated
Superimposition was achieved by rendering each whether to make a male/female judgment (attend
of the faces and places semi-transparent. All stimuli to the face) or an inside/outside judgment (attend
were standardized for luminosity, contrast, and to the place) of a subsequently presented stimulus.
transparency. All background place stimuli were After 250 msec, the superimposed face/place
300 300 pixels in size (at 72 dpi) with faces stimulus was presented for 750 msec. Participants
presented in an oval aperture 200 250 in size, were asked to indicate, as quickly and as accurately
which occluded gender stereotypic features such as possible, either the gender of the face (attend
as hair and facial shape. Stimuli were created such condition) or to indicate whether the place was the
that face gender, expression, and underlying place inside or outside of a building (unattend condition).
(interior and exterior) were completely crossed, We opted to use such an object attentional selection
yielding 108 independent stimuli. Across the course task to limit the role of eye movements, which
of scanning, each of these stimuli were presented would be a larger concern in spatial-selection tasks.
once during attended and once during unattended To ensure appropriate averaging of the overlapping
conditions for a total of 216 trials. Stimuli were hemodynamic responses from distinct trial types,
RT0996_C14.qxd 11/8/04 2:08 PM Page 188
trials were presented in a fixed randomized order 6 mm) were performed. During normalization,
that counterbalanced trial type history. voxels were resampled to 2 2 4 mm. The
After the test scans, participants took part in a presentation of each face/place stimulus was
fusiform face area (FFA) and parahippocampal modeled by a canonical hemodynamic response
place area (PPA) localizer scan. On each trial, function (hrf). For each individual, contrast images
subjects were first presented with central fixation were calculated by applying appropriate weights
(1 sec) and then an image of a neutral face (12 male, to the parameter estimates for the regressor of
12 female) or a building (1 sec). Images were pre- each event type. Group analysis for identification
sented in a fixed random intermixed order and of the amygdala and anterior insular regions of
participants were asked to simply indicate whether interest (ROIs) was performed on these contrast
a picture of a face or a place was presented. All images, which were submitted to a one-sample t test
stimuli were distinct from those used in the previous across the 12 subjects, with subjects entered as a
test scans. random effect. Group contrast images were over-
laid onto the SPM99 high-resolution T1 individual
Image Acquisition and Analysis template image for viewing. Coordinates of acti-
Participants were scanned with a 3 tesla Signa vation were converted from Montreal Neurological
(General Electric, Milwaukee, WI) scanner with a Institute to Talairach space.
prototype head coil. Foam padding placed around
the head was used to minimize movement. Every ROI Delineation
second we acquired seventeen 4 mm slices rang- The first phase of analysis was to replicate four
ing from the body of the corpus callosum to the separate findings for purposes of ROI delineation:
ventral surface of the anterior temporal lobe using (1) amygdala activation to fear faces, (2) anterior
a T2*-weighted spiral pulse sequence (in-plane insular activation to disgust faces, (3) FFA activa-
resolution, 3.755 mm; repetition time, 1000 msec; tion to faces, and (4) PPA activation to places.
echo time, 30 msec; 60º flip angle, 24 cm field of Accordingly, the ROIs were localized with rela-
view; 64 64 matrix acquisition). The intertrial tively liberal uncorrected criterion (p 0.01);
interval (ITI) was 8 sec. Four separate scans subsequently, signals from these regions were
collected 1728 frames (288 per condition), with submitted for examination of the main hypothe-
36 repetitions for each of the six trial types. Two ses. The amygdala ROI was defined by the con-
dummy trials were added at the beginning of each trast of fear relative to neutral faces when subjects
session to avoid scanner equilibration effects. The were attending to faces (cluster extent threshold,
same slice prescription and scanning parameters 10 voxels). The anterior insular ROI was defined
were used in the subsequent localizer scan, with by disgust relative to neutral faces when subjects
the exception of an increased ITI of 20 sec. One were attending to faces (extent threshold, 10 voxels).
session collected 960 frames (480 place, 480 face), Post-test localizer data were used to identify the
resulting in 24 repetitions of each trial type. FFA and PPA for each subject (extent threshold,
T1-weighted spin echo images were acquired for 5 voxels). Each subject’s FFA and PPA were de-
all slices that received functional scans as well as fined by a combination of functional and struc-
an additional T1-weighted whole-brain anatomy tural features. Right-hemisphere voxels confined
for the purposes of normalization of functional to the middle fusi-form gyrus that were more ac-
data into common stereotactic space. tive while viewing faces compared with places
Statistical analysis was performed using statis- were considered to be the FFA. In addition, voxels
tical parametric mapping software (SPM99; Well- lateral to the occipital temporal sulcus and con-
come Department of Cognitive Neurology, London, fined to the inferior and middle temporal gyri that
UK). After image reconstruction, motion estimation, were more active while viewing faces compared
realignment, slice-time correction, normalization, with places were considered to be face-responsive
and spatial smoothing (full width at half-maximum, regions within the lateral occipital complex, referred
RT0996_C14.qxd 11/8/04 2:08 PM Page 189
to here as the LOCf. Bilateral voxels confined to performance differences did not account for dif-
the parahippocampal gyrus that were more active ferences in BOLD responses on unattended trials.
while viewing places compared with faces were
considered to be the PPA. Effect of Attention on
Extrastriate Responses
ROI Signal Analyses Confirming previous results, decreased attention
For each subject, signal change indexed by the fit resulted in a substantial reduction in cortical acti-
of canonical hrf was extracted for each of the vations to both faces and places. A region func-
eight data frames for each of the six trial types tionally defined as more responsive to faces than
averaged across 36 trial repetitions and then sub- places in the right middle fusiform gyrus, consis-
mitted to statistical analysis. The α value for tent with the FFA, demonstrated a greater response
analysis of ROI signal was set at p 0.01. (average, 4–7 sec from stimulus onset) when sub-
jects were attending to faces and not places
Results (F(1,154) 386.69; p 0.0001) (Figure 14.1b).
Conversely, a bilateral region functionally defined
Behavioral Performance as more responsive to places than faces along the
Observers were less accurate in making gender collateral sulcus, consistent with the PPA, demon-
judgments on faces than location judgments on strated a greater response when subjects were at-
places (87.3 1.1 vs 79.3 1.4%; F(1,11) 14.07; tending to places and not to faces (F(1,154) 74.76;
p 0.003). Gender judgments were influenced by p 0.0001) (Figure 14.1b).
emotional expression (neutral, 75.7 1.7%; dis-
gust, 81.9 2.7%; fear, 80.3 2.3%; F(2,22) 3.62; Effect of Attention on
p 0.05). Accuracy in making place judgments Amygdala Response
was not influenced by the stimulus content of the When subjects were attending to faces, a com-
to-be-ignored faces (neutral, 87.5 2.2%; disgust, parison of fear relative to neutral faces resulted
86.6 1.6%; fear, 87.7 2.2%; F(2,22) 0.24; in a discrete activation in the right amygdala
p 0.79). Analysis of response latency revealed no (43 voxels, at a peak height x, 22; y, 1; z, 28; in
significant difference in the times taken to make Talairach coordinates, F(1,11) 20.52; p 0.0001)
face and place judgments (820 49 vs 754 46 (Figure 14.2a). During attended conditions, the peak
msec; F(1,11) 2.58; p 0.14). The response response in this functionally defined amygdala ROI
latency for gender judgments was influenced by was greater for fearful expressions than either dis-
emotional expression (neutral, 786 81 msec; gusted (fear vs disgust, F(1,154) 27.93; p 0.0001)
disgust, 755 69 msec; fear, 811 89 msec; or neutral expressions (fear vs neutral, F(1,154)
F(2,22) 4.46; p 0.03). The response latency for 40.22; p 0.0001); disgusted and neutral expres-
making place judgments was not influenced by sions did not differ (F(1,154) 1.12; p 0.28)
the stimulus content of the to-be-ignored faces (Figure 14.2b). Thus, the amygdala response was
(neutral, 732 81 msec; disgust, 727 77 msec; specific to fear and did not generalize to disgust.
fear, 753 86; F(2,22) 1.60; p 0.23). The magnitude of the amygdala response to fearful
Although facial expression did influence gen- faces was not significantly modulated by attention
der judgment accuracy and latency, and may have (F(1,154) 0.24; p 0.62), remaining greater than
contributed to the magnitude of blood—oxygen neutral expressions (fear vs neutral, F(1,154) 25.58;
level-dependent (BOLD) response when faces p 0.0001) during inattention.
were attended, critically, performance on place However, the amygdala demonstrated a surpris-
judgments (when subjects were instructed to ignore ing increase in response to expressions of disgust
faces) did not differ between face types. This sug- during unattended relative to attended conditions
gests that attention was equally divided for unat- (F(1,154) 48.67; p 0.0001) (Figure 14.3). Because
tended neutral, disgust, and fear face trials, so that of this increased response to disgust, when faces
RT0996_C14.qxd 11/8/04 2:08 PM Page 190
FIGURE 14.2 ■ (A color version of this figure follows page 146.) Attentional dependence of
amygdala and anterior insular responses to facial expressions. a, The amygdala was func-
tionally defined by the group level contrast of fear relative to neutral trials when faces were at-
tended. This resulted in a prominent activation in the right amygdala (at a peak height x, 22;
y,1; z,28; F(1,11) 20.52; p 0.0001). b, Effect of stimulus and attention on amygdala re-
sponse. Peak amygdala response is displayed for each facial stimulus type during attended
(red) and unattended (green) conditions. Attention did not significantly reduce the magnitude
of amygdala response to fear, but the enhanced response to disgust during reduced attention
suggests attention influenced the specificity of amygdala response. c, The insula was func-
tionally defined by contrasting activation on disgust trials compared with neutral trials when
faces were attended. This resulted in a prominent activation in the right anterior insula (at a
peak height x,44; y,5; z, 14; F(1,11) 32.72, p 0.0001). d, Effect of stimulus and attention
on anterior insular response. Peak anterior insular response is displayed for each facial stim-
ulus type during attended (red) and unattended (green) conditions. Reduced attention signif-
icantly reduced the magnitude of anterior insular response to disgust.
RT0996_C14.qxd 11/8/04 2:08 PM Page 191
were unattended, the amygdala response magnitude 54.99; p 0.0001) resulted in greater responses rel-
to fear was no longer significantly greater than that ative to neutral expressions (Figure 14.2b). Thus,
to disgust, with a tendency for a greater response to inattention did not significantly reduce the amyg-
disgust (F(1,154) 4.75; p 0.031). Both fear dala response to fear faces, but did significantly
(F(1,154) 31.89; p 0.0001) and disgust (F(1,154) enhance the amygdala response to disgust faces.
FIGURE 14.3 ■ (A color version of this figure follows page 146.) Response to dis-
gust faces when unattended. a, Amygdala response to disgust relative to neutral faces
when observers were attending to faces. No significant activation was found when
faces were attended. b, Amygdala response to disgust relative to neutral faces when
observers were attending to places. Activation was present when disgust faces were
unattended. c, Time course of the disgust response difference score (unattended mi-
nus attended). A negative deflection of time course represents a decreased response
when faces were attended. A positive deflection represents an increased response
when faces were unattended. An inverse effect of attention on anterior insula and
amygdala response to disgust faces peaked 6 sec after the stimulus onset.
RT0996_C14.qxd 11/8/04 2:08 PM Page 192
Effect of Attention on with diminished attention) is consistent with the

Anterior Insular Response notion that cortical processing can influence the
When subjects were attending to faces, a compari- breadth/narrowness of the amygdala response
son of disgust relative to neutral faces resulted in tuning. To examine this hypothesis more closely,
activation in the right anterior insula (22 voxels, we assayed the relation between the amygdala
at peak height x, 44; y, 6; z, 16; F(1,11) 32.72; response and how attention influences cortical
p 0.0001) (Figure 14.2c). During attended con- responsiveness to facial signals of threat in three
ditions, the peak response in this insular ROI was functionally defined face responsive regions: (1)
greater for disgusted expressions than for neutral within the FFA, a region specialized for face pro-
expressions (disgust vs neutral, F(1,154) 117.13; cessing; (2) within the LOCf, an area lateral to the
p 0.0001). Fear responses were also greater than FFA, purportedly within the lateral occipital com-
neutral in this region (fear vs neutral, F(1,154) 29.04; plex (Grill-Spector et al., 2001), a region specialized
p 0.0001), but there remained a greater response for shape processing; and (3) the anterior insula, a
to disgusted than to fearful faces (F(1,154) 29.53; region specialized for disgust face processing.
p 0.0001). The magnitude of insular response to During attended conditions, responses were
disgust (F(1,154) 61.88; p 0.0001) and fear greater for fearful relative to neutral faces in the
(F(1,154) 8.64; p 0.004) were both significantly FFA (F(1,154) 82.94; p 0.0001) and LOCf
reduced during unattended compared with attended (F(1,154) 48.37; p 0.0001). These greater re-
conditions. During inattention, the magnitude of sponses to fear were significantly diminished
insular response was no longer greater to disgust under unattended relative to attended conditions
than to fear (F(1,154) 0.26; p 0.6), but did remain in the FFA (F(1,154) 168.17; p 0.0001) and LOCf
greater to disgust than neutral (F(1,154) 10.58; (F(1,154) 61.47; p 0.0001), with response mag-
p 0.002) (Figure 14.2d). These results indicate nitudes in the FFA (F(1,154) 3.56; p 0.06) and
that both the magnitude and the specificity of LOCf (F(1,154) 1.67; p 0.19) no longer greater
the insular response to disgust were significantly for fearful versus neutral expressions. During
reduced with diminished attention. attended conditions, responses were also greater
In addition to the insula, patient and neuroimag- for disgusted relative to neutral faces in the FFA
ing studies suggest a role of the striatum in evalu- (F(1,154) 32.96; p 0.0001) and LOCf (F(1,154)
ating disgust expressions (Sprengelmeyer et al., 32.96; p 0.0001). These greater responses
1996; Phillips et al., 1997, 1998). When we reduced to disgust were also diminished during unat-
our statistical and extent thresholds (p 0.05 tended relative to attended conditions in the
and 5 voxels), activation in a contiguous bilateral FFA (F(1,154) 82.94; p 0.0001) and LOCf
ventral striatal region was greater for disgust than (F(1,154) 182.00; p 0.0001), with FFA and
for neutral expressions during attended conditions LOCf responses to disgust being numerically
(peak height on the right at x, 16; y, 18; z, 12; smaller than that of neutral expressions during un-
F(1,11) 25.91; p 0.0001; peak height on the left attended conditions. Thus, like the anterior insula,
at x, 2; y, 10; z, 8; F(1,11) 32.15; p 0.0001). FFA, and LOCf responses to fear and disgust were
Like the anterior insular response, inattention re- significantly reduced with diminished attention
sulted in a substantially reduced striatal response (Figure 14.4).
to disgust (F(1,154) 13.41; p 0.0003). This inverse effect of attention on cortical
(FFA, LOCf, and anterior insula) and amygdala
responses to disgust suggests that the loss of fear
Effect of Attention on specificity in the amygdala is related to diminished
Cortico-Amygdala Interactions cortical processing of disgust during inattention. To
The inverse relationship between cortical response examine such putative cortico-amygdala interac-
to disgust (diminished with diminished attention) tions, we assessed individual differences in the
and the amygdala response to disgust (enhanced magnitude of attentional modulation (attended vs
RT0996_C14.qxd 11/8/04 2:08 PM Page 193
0.5 0.4
a) b)
0.4
0.3
0.3
0.2
fMRI signal (±sem)
fMRI signal (±sem)

0.2
0.1 0.1
0 0
–0.1
–0.1
–0.2
–0.2
–0.3
–0.4 –0.3
–0.5 –0.4
c) 0.7 d) 0.5
0.6
0.4
0.5
0.4 0.3
fMRI signal (±sem)
0.3
fMRI signal (±sem)
0.2
0.2
0.1 0.1
0 0
–0.1
–0.1
–0.2
–0.3 –0.2
–0.4
–0.3
–0.5
–0.6 –0.4
–0.7 –0.5
FIGURE 14.4 ■ Effect of inattention on FFA, LOCf, anterior insula (INS), and amygdala (AMYG)
ROI responses to facial expressions. Bars represent the difference score between attended
and unattended conditions (unattend minus attend) for each fear, disgust, and neutral face.
The predominant effect of inattention was to reduce cortical responsiveness in the FFA, LOCf,
and insula. In contrast, the amygdala demonstrated a marked increased response to disgust.
unattended) of the amygdala response to disgust (standardized coefficient 0.471; F(1,93)

and its correlation with magnitude of attentional 19.58; p 0.0001), and neither FFA ( 0.17;
modulation of FFA, LOCf, and the anterior insular F(1,93) 2.22; p 0.13) nor LOCf ( 0.08;
responses to disgust. Although all three cortical F(1,93) 0.41; p 0.52), was significantly nega-
regions demonstrated substantial attentional mod- tively associated with enhanced amygdala response
ulation of disgust responses, multiple regression to disgust. That is, subjects who demonstrated the
analysis revealed that only the anterior insula largest attention-related decrease in anterior insular
RT0996_C14.qxd 11/8/04 2:08 PM Page 194
response tended to be the same as those who showed fear was not entirely automatic, coming at the
the largest increase in the amygdala response to expense of specificity of response. During inat-
disgust. This association between attentional mod- tention, the amygdala demonstrated a markedly
ulation of amygdala and insular responses was enhanced response to disgust. This finding suggests
stimulus dependent. Consistent with the attention- there are important limitations on what precise
independent amygdala response to fear, amygdala affective features the amygdala encodes automat-
responses were not significantly correlated with ically. Thus, amygdala automatic processing is
attention-dependent anterior insular responses to not specific to fearful faces, but rather, may be
fear (r 0.10; F(1,95) 0.95; p 0.33). confined to more coarse affective properties of
Responses to disgust in the FFA, LOCf, and faces, such as their valence or arousal/intensity.
anterior insula all demonstrated a pronounced In contrast to the present and previous studies,
reduction in response with reduced attention. In Pessoa et al. (2002a) have shown abolished corti-
addition, this association suggests that anterior cal and amygdala responses to fear faces under
insular responses to disgust may be critically de- conditions of extreme attentional load. Such con-
pendent on extrastriate face processing. To exam- tradictory results can be reconciled if we consider
ine this possibility further, we assayed the relation that different levels of attentional load will result
between individual differences in the magnitude in the modulation of activity at different levels
of attentional modulation of FFA, LOCf, and an- of the nervous system. Indeed, given similar at-
terior insular responses to disgust. A multiple re- tentional load, there are more pronounced modu-
gression analysis revealed that the LOCf ( lations in later visual cortical processing stages
0.53; F(1,93) 26.23; p 0.0001), but not the FFA [e.g., middle temporal (MT)] relative to earlier
( 0.17; F(1,93) 2.34; p 0.12), was signif- stages (e.g., VI) (Kastner et al., 1998, 2001). Severe
icantly positively associated with a reduced anterior attentional depletion may then result in modula-
insular response to disgust. tions very early in processing, before cortical
processing (O’Connor et al., 2002) such as in the
Discussion thalamic relays to the amygdala, functionally cut-
ting off the sensory inputs of the amygdala. This
Consistent with the notion that the amygdala would be consistent with demonstrations of pre-
processes fear automatically, the magnitude of the served amygdala fear responses in patients with
amygdala response to facial signals of fear was striate cortex lesions (Morris et al., 2001). In the
not significantly reduced with reduced attention, context of the present results, the automaticity of
despite reduced responses to fear in multiple cor- amygdala processing of fear is not all-or-none,
tical regions. However, such automaticity did not but a matter of degree. Relative to PPA processing
extend to all forms of facial threat processing: the of places, FFA processing of faces, and anterior
magnitude of anterior insular response to facial insula processing of disgust, the magnitude of the
signals of disgust was substantially reduced with amygdala response to fear demonstrates substantial
reduced attention. That automatic processing did attentional independence.
not extend to both amygdala processing of fear and The pronounced reduction in extrastriate response
anterior insula processing of disgust demonstrates during inattention contrasted with the amygdala
that automaticity is not a fundamental principle maintenance of response to fear. This is consistent
of neural systems dedicated to the processing of with fear processing in the amygdala occurring
facial expressions more generally, and facial ex- independently of extrastriate face processing
pressions related to threat in particular. Automatic- (Morris et al., 2001; Vuilleumier et al., 2001). Al-
ity appears unique to amygdala processing of though the magnitude of the amygdala response
social signals of fear. However, amygdala atten- to fear takes place independently of extra-striate
tional independence may not be complete. The face processing, the specificity of the amygdala
present study found that amygdala processing of response to fear may remain critically dependent
RT0996_C14.qxd 11/8/04 2:08 PM Page 195
on cortical processing. The enhanced amygdala limit the ability for the amygdala to resolve stimuli
response to expressions of disgust during de- of specific types.
creased attention is exceptional with respect to an Individual differences in the degree of atten-
extensive body of evidence showing reductions of tional degradation of processing in the anterior in-
brain response with reduced attention (Corbetta sular cortex were particularly tied to the amygdala
et al., 1990; Haxby et al., 1994; Wojciulik et al., loss of fear specificity, being directly related to
1998; O’Craven et al., 1999). This enhanced the magnitude of the enhanced amygdala re-
response to disgust may be a reflection of dimin- sponse to disgust. With its substantial projections
ished cortical influences on the amygdala. Although to the amygdala (Mesulam and Mufson, 1982),
previous studies have suggested that there are the anterior insula has been thought to convey
significant amygdala modulatory influences on cortical processing of affective stimulus content
cortical perceptual processing (Anderson and to the amygdala (Shi and Davis, 1999; Phelps et al.,
Phelps, 2001; Morris et al., 1998a, 1998b), the 2001). The anterior insular cortex may provide
present findings suggest that cortical processing the amygdala with more detailed information re-
can significantly modulate the amygdala response garding stimulus affective properties when atten-
(Phelps et al., 2001; Ochsner et al., 2002). tion is focused on the stimulus source. The result
To our knowledge, this interaction between the of diminished affective cortical processing is that
automaticity and the specificity of the amygdala the amygdala may respond more liberally to po-
response provides the first human evidence support- tentially significant stimulus events. In signal de-
ing an important proposed functional consequence tection terminology, with diminished cortical inputs
of having two parallel pathways to the amygdala: the amygdala may weigh more heavily “hits” and
one subcortically mediated and one cortically “misses” than “correct rejections” and “false
mediated (LeDoux, 1996). As proposed by LeDoux alarms.” That is, recognizing (hits) or failing to
(1996), by circumventing the cortex, a shorter recognize (misses) an environmental threat (e.g.,
thalamo-amygdala pathway processes information responding, or not, to a dangerous snake) should
in a more rapid and automatic manner. By engag- be more critical for amygdala processing than rec-
ing the cortex, a longer thalamo-cortico-amygdala ognizing (correct rejection) or failing to recognize
pathway allows for more detailed processing of (false alarms) an event as not threatening (e.g., re-
the stimulus, but in a less rapid and, as we propose sponding, or not, to snake-like objects, such as a
here, an attention-limited manner. An important curvy stick). This bias toward potentially impor-
consequence of bypassing cortical processing is tant events is not a reflection of a loss of sensitiv-
that the automaticity of the subcortical pathway ity to discriminate between potentially significant
should hypothetically come with a cost, at the and neutral events. During inattention, amygdala
expense of more fine-grained cortical analysis activation discriminated both fear and disgust from
(Thompson, 1962). Indeed, studies in monkeys neutral expressions. Thus, in healthy individuals
have shown altered amygdala discrimination of the amygdala does not “cry wolf” to all stimuli,
visual stimuli after reversible cooling of the infer- losing its predictive usefulness. Rather, under con-
otemporal cortex (Fukuda et al., 1987). Rabbits ditions of reduced stimulus analysis, the amyg-
with lesions of the auditory cortex have demon- dala appears to extend its response to a broader
strated impaired stimulus discrimination during range of potential threats, ensuring that potentially
auditory fear conditioning (Jarrell et al., 1987). In significant events will not be overlooked.
addition to reduced cortical processing, decreased However, this adaptive form of automaticity
attention is associated with decreased stimulus may not hold in clinical populations in which
discriminability (Yeshurun and Carrasco, 1998). there is substantial behavioral evidence of over-
Accordingly, reduced cortical responses during generalization of automatic processing to norma-
inattention can be interpreted as reflecting dimin- tively more neutral events (Williams et al., 1996).
ished cortical stimulus analysis that may ultimately This overgeneralization in clinical populations
RT0996_C14.qxd 11/8/04 2:08 PM Page 196
has been shown with respect to amygdala pro- activity (Rozin and Fallon, 1987; Levenson, 1992).
cessing as well. For instance, relative to nonsocial Commensurate with the prerequisite rapidity of
phobic individuals, patients with social phobia attack-related threat evaluations, the analysis of
demonstrate more pronounced amygdala response fear content from faces may occur early on, with
to neutral faces (Birbaumer et al., 1998). Broadening relative independence from higher-order attention-
of the amygdala response to other facial expres- limited processes. In contrast, disgust content
sions beyond fear has also been shown in patients from faces may have the luxury of occurring later,
with major depression, with this overgeneraliza- being dependent on more elaborative and
tion found to be reversible with treatment (Sheline attention-demanding processes. Thus, although
et al., 2001). In the context of the present results, selective pressures have promoted the development
the broadening of amygdala responsiveness in of specialized neural systems for the processing
clinical populations, and its reversibility, may of social signals of both fear and disgust, selection
reflect altered cortical modulatory influences on for automaticity may extend only to fear.
the amygdala response. Evidence of gender dif-
ferences in the amygdala response also under-
scores the variable nature of amygdala processing
Acknowledgments
(Cahill et al., 2001; Canli et al., 2002). The subjects This work was supported by National Institute of
in the present study were mostly women, so future Mental Health Grant MH12829–01 and by
studies with larger and gender-balanced samples McDonnell-Pew Program in Cognitive Neuro-
will be needed to examine whether the present science Grant 20002024.
findings apply equally to men and women.
Why is automaticity unique to the amygdala REFERENCES
processing of social signals of fear and does it not
extend to other brain regions specialized for so- Adolphs R, Tranel D, Damasio H, Damasio A (1994) Impaired
recognition of emotion in facial expressions following bilat-
cial signals of threat? Facial signals of fear, as eral damage to the human amygdala. Nature 372:669–672.
well as disgust, may serve as important cues for Anderson AK, Phelps EA (2001) Lesions of the human amyg-
searching one’s environment for the source of a dala impair enhanced perception of emotionally salient
potential threat (Whalen, 1998). Fear expressions events. Nature 411:305–309.
signal impending attack (e.g., response to a dan- Birbaumer N, Grodd W, Diedrich O, Klose U, Erb M, Lotze
M, Schneider F, Weiss U, Flor H (1998) fMRI reveals
gerous animal) (Gray, 1987). Disgust expressions amygdala activation to human faces in social phobics.
signal potential contamination or poisoning (e.g., NeuroReport 9:1223–1226.
rejection of harmful food) (Rozin and Fallon, Breiter HC, Etcoff NL, Whalen PJ, Kennedy WA, Rauch
1987). The character of the stimulus and response SL, Buckner RL, Strauss MM, Hyman SE, Rosen BR (1996)
Response and habituation of the human amygdala during
for attack-related threat (stimulus: moving target;
visual processing of facial expression. Neuron 17: 875–887.
response: immediate freezing or flight) versus Cahill L, Haier RJ, White NS, Fallon J, Kilpatrick
contamination-related threat (stimulus: stationary L, Lawrence C, Potkin SG, Alkire MT (2001) Sex-related
target; response: further inspection and passive difference in amygdala activity during emotionally influ-
avoidance) differ significantly (Sawchuk et al., enced memory storage. Neurobiol Learn Mem 75:1–9.
Calder AJ, Keane J, Manes F, Antoun N, Young AW (2000)
2002). Consistent with this division, predatory
Impaired recognition and experience of disgust following
animals tend to evoke fear, whereas disgust is brain injury. Nat Neurosci 3:1077–1078.
associated with animals that do not present signif- Canli T, Desmond JE, Zhao Z, Gabrieli JD (2002) Sex differ-
icant harm (e.g., spiders, slugs) (Ware et al., 1994). ences in the neural basis of emotional memories. Proc Natl
Similarly, fear and disgust demonstrate distinct Acad Sci USA 99:10789–10794.
Corbetta M, Miezin FM, Dobmeyer S, Shulman GL, Petersen SE
physiological signatures and action tendencies
(1990) Attentional modulation of neural processing of shape,
(Ekman, 1992; Levenson, 1992), with fear associ- color, and velocity in humans. Science 248:1556–1559.
ated with increased sympathetic activity (Ekman Ekman P (1992) An argument for basic emotions. Cognition
et al., 1983) and disgust more with parasympathetic Emotion 6:169–200.
RT0996_C14.qxd 11/8/04 2:08 PM Page 197
Ekman P, Levenson RW, Friesen WV (1983) Autonomic nerv- presentation of emotional faces in a cortically blind field.
ous system activity distinguishes among emotions. Science Brain 124:1241–1252.
221:1208–1210. O’Connor DH, Fukui MM, Pinsk MA, Kastner S (2002) At-
Esteves F, Dimberg U, Ohman A (1994) Automatically elicited tention modulates responses in the human lateral geniculate
fear: conditioned skin conductance responses to masked nucleus. Nat Neurosci 5:1203–1209.
facial expressions. Cognition Emotion 8:393–413. O’Craven KM, Downing PE, Kanwisher N (1999) fMRI evi-
Fukuda M, Ono T, Nakamura K (1987) Functional relations dence for objects as the units of attentional selection. Nature
among inferotemporal cortex, amygdala, and lateral hypothal- 401:584–587.
amus in monkey operant feeding behavior. J Neurophysiol Ochsner KN, Bunge SA, Gross JJ, Gabrieli JD (2002) Re-
57:1060–1077. thinking feelings: an fMRI study of the cognitive regulation
Gray JA (1987) The psychology of fear and stress, Ed 2. New of emotion. J Cogn Neurosci 14:1215–1219.
York: Cambridge UP. Ohman A, Lundqvist D, Esteves F (2001) The face in the
Grill-Spector K, Kourtzi Z, Kanwisher N (2001) The lateral crowd revisited: a threat advantage with schematic stimuli.
occipital complex and its role in object recognition. Vision J Pers Soc Psychol 80:381–396.
Res 41:1409–1422. Pessoa L, Kastner S, Ungerleider LG (2002a) Attentional con-
Haxby JV, Horwitz B, Ungerleider LG, Maisog JM, Pietrini P, trol of the processing of neutral and emotional stimuli.
Grady CL (1994) The functional organization of human ex- Brain Res Cognit Brain Res 15:31–45.
trastriate cortex: a PET-rCBF study of selective attention to Pessoa L, McKenna M, Gutierrez E, Ungerleider LG (2002b)
faces and locations. J Neurosci 14:6336–6353. Neural processing of emotional faces requires attention.
Jarrell TW, Gentile CG, Romanski LM, McCabe PM, Schnei- Proc Natl Acad Sci USA 99:11458–11463.
derman N (1987) Involvement of cortical and thalamic Phelps EA, O’Connor KJ, Gatenby JC, Gore JC, Grillon C,
auditory regions in retention of differential bradycardiac Davis M (2001) Activation of the left amygdala to a cogni-
conditioning to acoustic conditioned stimuli in rabbits. tive representation of fear. Nat Neurosci 4:437–441.
Brain Res 412:285–294. Phillips ML, Young AW, Senior C, Brammer M, Andrew C,
Kastner S, De Weerd P, Desimone R, Ungerleider LG (1998) Calder AJ, Bullmore ET, Perrett DI, Rowland D, Williams
Mechanisms of directed attention in the human extrastriate SC, Gray JA, David AS (1997) A specific neural substrate
cortex as revealed by functional MRI. Science 282:108–111. for perceiving facial expressions of disgust. Nature
Kastner S, De Weerd P, Pinsk MA, Elizondo MI, Desimone R, 389:495–498.
Ungerleider LG (2001) Modulation of sensory suppression: Phillips ML, Young AW, Scott SK, Calder AJ, Andrew C,
implications for receptive field sizes in the human visual Giampietro V, Williams SC, Bullmore ET, Brammer M,
cortex. J Neurophysiol 86:1398–1411. Gray JA (1998) Neural responses to facial and vocal expres-
LeDoux JE (1995) Emotion: clues from the brain. Annu Rev sions of fear and disgust. Proc R Soc Lond B Biol Sci 265:
Psychol 46:209–235. 1809–1817.
LeDoux JE (1996) The emotional brain: the mysterious under- Rozin P, Fallon AE (1987) A perspective on disgust. Psychol
pinnings of emotional life. New York: Simon & Schuster. Rev 94:23–41.
Levenson RW (1992) Autonomic nervous system differences Sawchuk CN, Meunier SA, Lohr JM, Westendorf DH (2002)
among emotions. Psychol Sci 3:23–27. Fear, disgust, and information processing in specific phobia:
Mesulam MM, Mufson EJ (1982) Insula of the old world mon- the application of signal detection theory. J Anxiety Disord
key. III. Efferent cortical output and comments on function. 16:495–510.
J Comp Neurol 212:38–52. Sheline YI, Barch DM, Donnelly JM, Ollinger JM, Snyder AZ,
Morris JS, Frith CD, Perrett DI, Rowland D, Young AW, Mintun MA (2001) Increased amygdala response to masked
Calder AJ, Dolan RJ (1996) A differential neural response in emotional faces in depressed subjects resolves with anti-
the human amygdala to fearful and happy facial expressions. depressant treatment: an fMRI study. Biol Psychiatry 50:
Nature 383:812–815. 651–658.
Morris JS, Friston KJ, Buchel C, Frith CD, Young AW, Calder Shi C, Davis M (1999) Pain pathways involved in fear condi-
AJ, Dolan RJ (1998a) A neuromodulatory role for the hu- tioning measured with fear-potentiated startle: lesion studies.
man amygdala in processing emotional facial expressions. J Neurosci 19:420–430.
Brain 121:47–57. Sprengelmeyer R, Young AW, Calder AJ, Karnat A, Lange H,
Morris JS, Friston KJ, Dolan RJ (1998b) Experience-dependent Homberg V, Perrett DI, Rowland D (1996) Loss of disgust:
modulation of tonotopic neural responses in human auditory perception of faces and emotions in Huntington’s disease.
cortex. Proc R Soc Lond B Biol Sci 265:649–657. Brain 119:1647–1665.
Morris JS, Ohman A, Dolan RJ (1999) A subcortical pathway Thompson RF (1962) The role of the cerebral cortex in stimu-
to the right amygdala mediating “unseen” fear. Proc Natl lus generalization. J Comp Physiol Psychol 55:279–287.
Acad Sci USA 96:1680–1685. Vuilleumier P, Armony JL, Driver J, Dolan RJ (2001) Effects
Morris JS, DeGelder B, Weiskrantz L, Dolan RJ (2001) Dif- of attention and emotion on face processing in the human
ferential extra-geniculostriate and amygdala responses to brain: an event-related fMRI study. Neuron 30:829–841.
RT0996_C14.qxd 11/8/04 2:08 PM Page 198
Vuilleumier P, Armony J, Clarke K, Husain M, Driver J, Dolan R expressions modulate amygdala activity without explicit
(2002) Neural response to emotional faces with and without knowledge. J Neurosci 18:411–418.
awareness: event-related fMRI in a parietal patient with visual Williams JMG, Mathews A, MacLeod C (1996) The emo-
extinction and spatial neglect. Neuropsychologia 40:2156. tional Stroop task and psychopathology. Psychol Bull
Ware J, Jain K, Burgess I, Davey GC (1994) Disease-avoidance 120:3–24.
model: factor analysis of common animal fears. Behav Res Wojciulik E, Kanwisher N, Driver J (1998) Covert visual
Ther 32:57–63. attention modulates face-specific activity in the human
Whalen PJ (1998) Fear, vigilance, and ambiguity: initial fusiform gyrus: fMRI study. J Neurophysiol 79: 1574–
neuroimaging studies of the human amygdala. Curr Direct 1578.
Psychol Sci 7:177–188. Yeshurun Y, Carrasco M (1998) Attention improves or impairs
Whalen PJ, Rauch SL, Etcoff NL, McInerney SC, Lee MB, visual performance by enhancing spatial resolution. Nature
Jenike MA (1998) Masked presentations of emotional facial 396:72–75.
RT0996_C15.qxd 11/9/04 4:11 PM Page 199
R E A D I N G 1 5
Automatic and Intentional Brain Responses during

Evaluation of Trustworthiness of Faces
•
J.S. Winston1, B.A. Strange2, J. O’Doherty1 and R.J. Dolan1,3
Successful social interaction partly depends on appraisal of others from their facial appearance.
A critical aspect of this appraisal relates to whether we consider others to be trustworthy. We
determined the neural basis for such trustworthiness judgments using event-related functional
magnetic resonance imaging. Subjects viewed faces and assessed either trustworthiness or age.
In a parametric factorial design, trustworthiness ratings were correlated with BOLD signal change
to reveal task-independent increased activity in bilateral amygdala and right insula in response to
faces judged untrustworthy. Right superior temporal sulcus (STS) showed enhanced signal change
during explicit trustworthiness judgments alone. The findings extend a proposed model of social
cognition by highlighting a functional dissociation between automatic engagement of amygdala
versus intentional engagement of STS in social judgment.
Iandttoisthat,
conjectured that human survival has depended
a large extent on accurate social judgments
as an evolutionary consequence, modular
However, facial emotional expression is only one
aspect of social judgment made about others. In
many situations, individuals must also decide
cognitive processes are devoted to such functions1. whether another person is someone to approach or
Neuropsychological studies and human functional avoid, trust or distrust. Preliminary evidence re-
imaging provide partial support for this idea of a garding the neural underpinnings of this sort of
dedicated ‘social intelligence’, particularly stud- evaluative judgment comes from studies in which
ies that address perception of facial expression2–7. patients with bilateral amygdala lesions make
1 3
Wellcome Department of Imaging Neuroscience, 12 Queen Royal Free and University College Medical School, Roland
Square, London WC1N 3BG, UK. Hill Street, London NW3 2PF, UK.
2
Institute of Cognitive Neuroscience, 17 Queen Square, London
WC1N 3AR, UK.
199
RT0996_C15.qxd 11/9/04 4:11 PM Page 200
abnormal social judgments about others based on faces involve such a network and that this net-
facial appearance8. These abnormalities are most work is differentially modulated by implicit
pronounced in relation to faces that received the and explicit evaluations.
most negative ratings by control subjects. Notably,
such deficits are not apparent in subjects with uni-
lateral amygdala lesions8. Patients with damage to Results
ventromedial prefrontal cortex also have difficulties
Behavioral
with trustworthiness decisions9,10.
The most influential neurobiological model of After scanning, on average subjects labeled more
social cognition11, based on inferences largely than half of the 120 faces as having ‘neutral’ emo-
from neurophysiological recordings in non-human tional expressions (mean, 65). Labeled emotional
primates, postulates that the superior temporal expression interacted significantly with trustwor-
sulcus acts as association cortex for processing thiness score across the group of subjects (Kruskal-
conspecifics’ behavior and that socially relevant in- Wallis test, p 0.001). Mann–Whitney U tests
formation is subsequently labeled by the emotional showed that the trustworthiness scores (from 1,
systems, such as amygdala and orbitofrontal cortex. least trustworthy, to 7) did not differ significantly
More recent models of human social cognition also between ‘disgusted,’ ‘fearful’ and ‘surprised’ faces
include sensory regions such as the face-processing and ‘neutral’ faces (p 0.05 in all cases). ‘Happy’
area in fusiform gyrus and somatosensory cortex faces (mean trustworthiness rating, 4.0) were rated
(including insula, SI and SII)12–14. as significantly more trustworthy than ‘neutral’faces
Here we used event-related functional mag- (mean rating, 3.9), and ‘angry’ (mean rating, 2.7)
netic resonance imaging (fMRI) to ascertain the and ‘sad’ (mean rating, 3.6) faces as significantly
neural substrates mediating evaluative social less trustworthy (p 0.01 in all cases). Mean
judgment. Processing of facial emotion can be trustworthiness scores (Figure 15.1a) were signif-
implicit, occurring when subjects make judg- icantly correlated with mean scores for anger,
ments about facial attributes unrelated to emo- happiness and sadness from the second group of
tion (for example, refs. 5–7, 15, 16). To estab- subjects (see Methods) (p 0.01 for each, two-
lish whether trustworthiness judgments might tailed; Figure 15.1b–e).
be similarly processed, we used a task in which
subjects viewed faces while making either ex- Neuroimaging
plicit judgments whether an individual was Linear contrasts were performed to produce sta-
trustworthy or an unrelated age assessment. To tistical parametric maps (SPMs) of the main
account for individual differences in trustwor- effect of task (explicit or implicit processing
thiness judgment, we acquired ratings of trust- of trustworthiness), the main effect of trustworthi-
worthiness for each stimulus from each subject ness and the interaction between these two fac-
after scanning and used these ratings as para- tors. An additional model in which the effects
metric covariates in our subsequent analysis. of facial emotion of the stimuli were included
Based on the models of social cognition out- as covariates of no interest was used to generate
lined above11–14, along with the neuropsycho- an SPM related to the main effect of trustworthi-
logical findings 8, we predicted that discrete ness independent of effects of facial emotional
brain regions, the amygdala, orbitofrontal cor- expression.
tex, fusiform gyrus and superior temporal sul- A significant activation in the explicit com-
cus, would be implicated in trustworthiness as- pared to implicit task, independent of trustworthi-
sessments. Consequently, these areas formed ness, was found in the right posterior superior
regions of interest in our statistical analysis. temporal sulcus (x, y, z coordinates, 56, 44, 4;
Our data indicate that social judgments about Z 4.27; p 0.05, corrected for multiple
RT0996_C15.qxd 11/9/04 4:11 PM Page 201
Automatic and Intentional Brain Responses during Evaluation of Trustworthiness of Faces ■ 201
b) c)
a)
d) e)
FIGURE 15.1 ■ Trustworthiness and emotion ratings for stimuli. (a) Means and standard deviations of trustworthiness
scores of stimuli, rank-ordered by trustworthiness score. (b–e) Mean emotion scores (from second cohort of sixteen
subjects) and mean trustworthiness scores (from cohort of subjects scanned with fMRI) for anger (b), fear (c), happi-
ness (d) or sadness (e). Lines of best fit are derived by linear regression. Both rating scales ranged from 1 (low degree
of emotion or highly untrustworthy) to 7 (highly emotional or highly trustworthy).
comparisons across a small volume of interest; Further areas showing increased response to
Figure 15.2; Table 15.1). Additionally, primary untrustworthy faces included left superior tempo-
visual cortex was significantly activated in this ral sulcus (50, 58, 10; Z 4.15) and a region
contrast. Attentional and emotional manipulations of the right superior middle insula (42, 4, 12;
are known to alter neural responses in early visual Z 3.48; Figure 15.3a and b). Additionally, bilat-
cortex17, and we propose that similar processes eral activation in the fusiform gyrus was evident
engendered by the explicit task account for this in this contrast (right, 44, 46, 24, Z 3.58;
latter activation. left, 48, 48, 24; Z 3.60; both p 0.05,
As predicted, significant bilateral amygdala corrected for multiple comparisons across a small
activation was evident in the contrast of volume of interest; Figure 15.4). Table 15.2 pres-
untrustworthy to trustworthy faces (right, 18, ents regions highlighted by this contrast as well as
0, 24; Z 4.29; left, 16, 4, 20; regions highlighted as more responsive to faces
Z 3.92; both p 0.05, corrected for multiple rated as trustworthy.
comparisons across a small volume of interest; To ensure that the main effect of untrustworthi-
Figure 15.3a). This examination of parametric ness was not driven by a highly significant activa-
data based on each subject’s ratings of faces in- tion in just one of the tasks alone, a masked con-
dicates that more untrustworthy faces evoke junction of simple effects of trustworthiness
greater BOLD responses in the amygdala under implicit and explicit task conditions was
(Figure 15.3c and d). carried out (Methods). This analysis confirmed
RT0996_C15.qxd 11/9/04 4:11 PM Page 202
a)
b)
FIGURE 15.2 ■ (A color version of part a of this figure follows page 146.) Main
effect of explicit social judgments. (a) Random-effects SPM overlaid on a normal-
ized structural scan from a single subject showing activation in right superior
temporal sulcus region (x, y, z 56, 44, 4; Z 4.27; p 0.05 small volume
corrected) when making judgments about trustworthiness compared to age. For
illustration, using threshold p 0.001 uncorrected, extent threshold of 5 voxels.
(b) BOLD signal measure categorized by task and trustworthiness of faces. ‘Low’,
‘med’ and ‘high’ refer to the least trustworthy third, median third and most trustwor-
thy third of faces calculated in the second model described in Methods. The y-axis
represents mean (across subjects) percentage signal change relative to whole
brain mean over scanning session for each event type. There is no clear pattern of
response to the faces according to trustworthiness. Note that statistical inference
is drawn only from the parametric model described in Methods, and not from the il-
lustrative model in (b).
TABLE 15.1. Cerebral Foci of Activation in Main Effect of Task

Coordinates of peak activation (mm)
Brain region x y z Z score
Explicit versus implicit

Primary visual cortex 2 98 8 4.49
Right posterior STS* 56 44 4 4.27
Right superior frontal gyrus 10 14 70 3.99
Left premotor cortex 48 2 26 3.84
Left extrastriate cortex 24 76 32 3.75
Right cuneus 12 40 56 3.62
Left primary sensory cortex 30 28 72 3.62
Supramarginal gyrus 62 40 34 3.50
Right anterior insula 48 34 6 3.37
Left superior frontal sulcus 42 12 40 3.34
Left pre-SMA 6 10 54 3.34
Implicit versus explicit
Left fusiform gyrus 36 36 18 3.59
Right cuneus 4 64 12 3.49
All values, p 0.001 uncorrected. *p 0.05 corrected for multiple comparisons across a small volume of interest.
RT0996_C15.qxd 11/9/04 4:11 PM Page 203
a) b)
c) d)
FIGURE 15.3 ■ (A color version of part a of this figure follows page 146.) Main effect
of trustworthiness in amygdala and insula. (a) Significant increases in BOLD signal to un-
trustworthy faces in the right and left amygdalae and right insula (right amygdala, 18,
0, 24; Z 4.29; p 0.01 corrected; left amygdala, 16, 4, 20; Z 3.92; p 0.025
corrected; right insula, 42, 4, 12; Z 3.48; p 0.001 uncorrected). (b–d) Responses
to faces as a function of degree of individually rated trustworthiness for right insula (b),
left amygdala (c) and right amygdala (d). Note greater responses to less trustworthy
faces across all these regions. The y-axis is as in Figure 15.2.
that bilateral amygdala, fusiform gyrus and right hypothesis were revealed in this contrast or in the
insula showed significant responses to untrust- reverse interaction term.
worthy faces independent of task. Notably, left Using an additional model that partialed out
STS activation was not observed in this contrast, effects from facial expression of basic emotions in
and a post-hoc test revealed that the effects in this the stimulus set, we performed a random effects
region were driven principally by trustworthiness analysis across the 14 subjects. Even under these
judgments under the explicit task. stringent criteria, right amygdala activation was
The contrast pertaining to the interaction of task still evident in this model at both uncorrected
and trustworthiness demonstrated an area in the (p 0.001) and small-volume corrected (p 0.05
lateral orbitofrontal cortex (28, 42, 10; Z 3.73, corrected for multiple comparisons across bilat-
p 0.0001, uncorrected) responsive to untrust- eral amygdala volume) thresholds (Figure 15.5).
worthy faces in the implicit task and to trustworthy This activation (peak at 22, 2, 18; Z 4.06)
faces in the explicit task. However, this activation overlapped with that reported in our primary
failed to survive correction for multiple compar- model. At lower thresholds (p 0.005, uncor-
isons across the entire volume of orbitofrontal rected), there was additional activation in left
cortex. No other areas about which we had a prior amygdala.
RT0996_C15.qxd 11/9/04 4:11 PM Page 204
a)
b) c)
FIGURE 15.4 ■ (A color version of part a of this figure follows page 146.) Main effect of
trustworthiness in fusiform gyrus. (a) Significant increases in BOLD signal to untrustworthy faces
in the fusiform gyrus bilaterally (right, 44, 46, 22; Z 3.58; p 0.05, small volume corrected;
left, 48, 48, 24; Z 3.60; p 0.05, small volume corrected). This activation is independent
of task in both the left (b) and right (c) fusiform gyrus. The y-axis is as in Figure 15.2.
TABLE 15.2. Cerebral Foci of Activation in Differential Effects of Trustworthiness of Faces

Coordinates of peak activation (mm)
Brain region x y z Z score
Untrustworthy versus trustworthy

Right amygdala* 18 0 24 4.29
Left superior temporal sulcus 50 58 10 4.15
Right intraparietal sulcus 22 54 48 4.00
Left extrastriate cortex 34 90 24 3.94
Left amygdala* 16 4 20 3.92
Right pre-SMA 8 8 62 3.83
Left parahippocampal gyrus 18 30 18 3.81
Right auditory cortex 66 18 4 3.75
Left inferior temporal gyrus 60 14 30 3.64
Left fusiform gyrus* 48 48 24 3.60
Right fusiform gyrus* 44 46 22 3.58
Thalamus 4 12 14 3.52
Right insula 42 4 12 3.48
Left superior temporal gyrus 58 32 14 3.42
Trustworthy versus untrustworthy
Left insula 36 4 4 3.65
Right dorsolateral prefrontal/frontopolar
cortex 34 52 6 3.23
All values, p 0.001 uncorrected. *p 0.05 corrected for multiple comparisons across a small volume of interest.
RT0996_C15.qxd 11/9/04 4:11 PM Page 205
of amygdala response during complex social

judgments.
In addition to the amygdala, the right insula
was also activated by faces that subjects consid-
ered untrustworthy regardless of task. The insula
is activated in a wide variety of functional imag-
ing studies of emotion (for example, refs. 22–25).
One suggested role for the insula is the mapping
of autonomic changes as they affect the body where
such mappings form the basis of ‘gut feelings’
about emotive stimuli26,27. Thus, a possible expla-
nation for the insula activation that we observed is
FIGURE 15.5 ■ (A color version of this figure follows that a consequence of amygdala activation is the
page 146.) Main effect of trustworthiness in amygdala generation of autonomically mediated changes
independent of facial emotion. Significant increases in in bodily states, which are then re-mapped to the
BOLD signal in response to untrustworthy faces in right insula.
amygdala even when scores for four basic facial emo-
Differential activation in face-responsive regions
tions are additionally used as parametric covariates in
the analysis. This activation is significant at p 0.05, of the fusiform gyrus was observed in relation to
corrected for multiple comparisons across the volume of trustworthiness in face stimuli. Increased activity
bilateral amygdala. Activation peak at 18, 2, 22 (Z is found in modality-specific cortical areas in re-
4.06), but overlaps with right amygdala activation focus sponse to stimuli with emotional content relative
shown in Figure 15.2. At lower threshold of p 0.005 un-
to non-emotional stimuli (for example, refs. 20,
corrected, activation is evident in left amygdala.
21, 28–31). Enhanced extrastriate activation in
response to emotional stimuli has been attributed
Discussion to modulatory influences from the amygdala32,
possibly mediated by anatomical back-projec-
The question addressed in this study was whether tions33. Indeed, a human lesion study highlights a
the dimension of trustworthiness in faces and the possible role for the amygdala in enhancing per-
process of making social judgments are associ- ceptual processing of threat stimuli34. We suggest
ated with distinct patterns of brain activation. By that such processes extend to faces representing
implication, the study is an explicit test of a pro- potential threat at the social level and that a neural
posed neurobiological model11. The principal consequence is enhanced fusiform activation.
findings of activation in amygdala, orbitofrontal The right STS showed task-related activation in
cortex and STS are highly consistent with this the explicit judgment condition but no differential
model. We also extend previous lesion data8 by activity according to trustworthiness. In other
showing amygdala activity in response to untrust- words, the right STS was activated when subjects
worthy faces regardless of whether subjects were made explicit judgments about trustworthiness. In
explicitly making trustworthiness judgments. this regard, the STS showed activity when subjects
This finding echoes earlier studies of obligatory were required to make inferences concerning the
threat-related processing in the amygdala18–21. In likely intentionality of others. This region has
contrast to many imaging studies of facial emo- been implicated in functional imaging studies on
tion (for example, ref. 5), the amygdala response biological motion35 and biological-like motion36.
to untrustworthy faces was bilateral, supporting More critically, activity in posterior STS and adja-
neuropsychological evidence that patients with cent regions at the temporo-parietal junction is
unilateral amygdala lesions can successfully make observed when subjects make theory of mind infer-
trustworthiness judgments8. To our knowledge, no ences37–39. This region is suggested to be involved
previous study has demonstrated an automaticity in intention detection40,41, rather than biological
RT0996_C15.qxd 11/9/04 4:11 PM Page 206
motion processing per se. Intention detection is a analysis accounted for the variance attributed to
critical component in determining whether or not to facial emotion. These results suggest that facial
trust an individual, which may explain the activity expressions of emotion provide a constituent ele-
in this region in our study. ment in making trustworthiness judgments but
Evidence from human patients with discrete that amygdala responses also were independent of
lesions of orbitofrontal cortex indicate that this these effects. Notably, patients with bilateral amyg-
region is critical for complex social judgment9,10. dala lesions show deficits in making social judg-
Unlike the amygdala, this region showed task- ments in the context of maintained ability to use
dependent activation. When subjects made explicit information about emotional expression of face
judgments of trustworthiness, this region responded stimuli8.
more strongly to faces deemed trustworthy. By It is interesting to speculate how the results of
contrast, when judging age, this region showed this study might generalize to social judgments
greater responses to untrustworthy individuals. about stimuli in other modalities. Patients with bi-
Other studies have reported similar task-dependent lateral amygdala lesions are able to make accurate
responses in lateral orbitofrontal cortex. For exam- trustworthiness judgments based on verbal reports8.
ple, responses in a region of lateral orbitofrontal It is plausible therefore that amygdala involve-
cortex vary between preference and recognition ment in trustworthiness decisions may be modal-
judgment tasks on the same stimuli42. A dissocia- ity-specific. This hypothesis could be tested in
tion between implicit/automatic social judgment follow-up experiments involving trustworthiness
and explicit (laboratory-tested) social judgment has judgments about vocal stimuli, or scenarios about
also been reported in a patient with orbitofrontal individuals based on written descriptions. Our
cortex damage9. This patient remained able to prediction would be that superior temporal sulcus
evaluate social situations under explicit task in- activation would remain in these other contexts
structions but was impaired in day-to-day (“auto- and that fusiform modulation would be substi-
matic”9) social judgments. Note that activation in tuted by modality-specific cortical responses, for
this region did not survive correction for multiple example, auditory cortex in the case of vocal
comparisons in our study, and we emphasize ef- stimuli.
fects in this region based on its known involvement In conclusion, we present functional brain im-
in social judgments9,10. aging evidence for a neural substrate of social
Several regions of interest in this study (amyg- cognition that conforms to a previously proposed
dala, orbitofrontal cortex and insula) are activated neurobiological model11. Our data extends this
in processing specific facial expressions. Facial model by highlighting a dissociation between au-
expressions of fear consistently activate the amyg- tomatic and intentional engagement within this
dala5,6,43, whereas facial expressions of disgust proposed circuitry. Thus, social judgments about
activate the anterior insula6,7. Additionally, we faces reflect a combination of brain responses that
demonstrate correlations between the trustworthi- are stimulus driven, in the case of the amygdala,
ness scores and scores for facial emotions attributed and driven by processes relating to inferences
to our stimulus set (Figure 15.1b-e). Consequently, concerning the intentionality of others, in the case
one possibility is that differential patterns of acti- of STS.
vation seen in this study reflect influences from
one or more emotional expressions alone. We as-
sessed this possibility by analyzing the fMRI data Methods
with additional nuisance covariates pertaining to
the degree of emotional expression of each of four Subjects
basic emotions (anger, fear, happiness, sadness). Informed consent to partake in a study approved
A significant right amygdala response to untrust- by the Joint National Hospital for Neurology
worthy faces persisted, even after this secondary and Neurosurgery/Institute of Neurology Ethics
RT0996_C15.qxd 11/9/04 4:11 PM Page 207
Committee was obtained from 16 right-handed informing the subject of the task requirement
Caucasian volunteers (8 male, 8 female; age range (“School/Uni” or “Trustworthiness”).
18–30 years; mean age 23.3 years). Two subjects Stimuli were presented on a gray background
(both females) were excluded from the analysis; once each in random order, randomly interspersed
one revealed psychiatric history after scanning with 60 null events. Each stimulus was presented for
and another provided extreme trustworthiness rat- 1 s with an inter-trial interval of 2 s. Between faces,
ings. (Spearman’s rho of correlation of ratings a fixation cross was presented. Null events were of
with mean of all other subjects, 0.445; for all 3 s duration, during which time a fixation cross
remaining subjects, Spearman’s rho values were remained on screen. Stimuli subtended visual angles
over 0.3.) All remaining subjects were free from of approximately 10º vertically and 5º horizontally.
psychiatric or neurological history. All subjects
except one had completed more than two years of Image Acquisition
post-16 education, and mean length of post-16
Subjects were scanned during task performance us-
formal education was 4.8 years.
ing a Siemens VISION system (Erlangen, Germany)
at 2 Tesla to acquire gradient-echo, echoplanar
Stimuli T2*-weighted images with BOLD (blood oxy-
Grayscale frontal images of 120 Caucasian male genation level dependent) contrast. Each volume
faces were selected from a larger selection of im- comprised 33 2.2 mm axial scans with 3-mm
ages following a pilot study outside the scanner. in-plane resolution, and volumes were continu-
The images were selected to cover a range of ously acquired every 2.5 s. Subjects were placed
trustworthiness scores rated by the subjects in the in light head restraint within the scanner to limit
pilot study (n 30; 13 females, 17 males, ages head movement during acquisition. Each run be-
17–32, mean age 23.5), but to score as low as gan with 5 ‘dummy’ volumes (subsequently dis-
possible on ratings of ‘happiness’ and ‘anger’. carded) to allow for T1 equilibration effects.
Gaze direction of all stimuli was directly forward. Additionally, a T1-weighted structural image was
Stimuli were adjusted to be of approximately acquired in each subject.
equal size and luminance and manipulated such All functional volumes were realigned44 and
that each face was centered on a gray background slice timing corrected (R. Henson et al., Neuroimage
in a 400 400 pixel image. Of the 120 stimuli 9, 125, 1999), normalized into a standard space45
used in the imaging study, 60 were high school to allow group analysis, and smoothed with an
student photographs and 60 photographs of uni- 8-mm FWHM Gaussian kernel to account for
versity students. There was no significant differ- residual intersubject differences.
ence in average trustworthiness score between the
two groups (Mann–Whitney U test, p 0.90). Debriefing
After scanning, participants undertook a self-paced
Psychological Task task in which they rated all the faces on a scale of
The scanning session for each participant was trustworthiness from 1 (highly untrustworthy) to
divided into two parts. In one half of the session, 7 (highly trustworthy). When all 120 faces had
60 faces were presented sequentially, and partici- been rated, a second task was performed, in which
pants made a judgment, indicating with a push- participants named emotions that they perceived
button response, whether the face was a high in the faces by means of a seven-way forced
school or university student. In the other half of choice procedure (neutral, happy, sad, angry, dis-
the session, they judged whether the face was gust, fear, surprise). To assist subjects with this
trustworthy or untrustworthy. The order of tasks task, we gave them a printed sheet with photo-
was counterbalanced between participants. At graphs of one face from the Ekman and Friesen
the start of each task, a word appeared on screen series46 expressing each of these seven emotions.
RT0996_C15.qxd 11/9/04 4:11 PM Page 208
Emotion Ratings for Stimuli mask was approximately 10 cm3, volume of the
An additional set of 16 subjects (10 males, 6 orbitofrontal mask approximately 50 cm3, and
females; age range 19–34 years; mean age 23.7 volume of the STS mask approximately 20 cm3.
years) undertook a task in which they rated the de- In the case of the fusiform gyri, small-volume
gree of emotional expression within each face on correction was based upon a sphere of 10 mm
each of four basic emotions (anger, fear, happiness, radius centered on coordinates derived from a pre-
sadness) in turn. Ratings were from 1 (neutral for vious study21. We report descriptively activations
this particular emotion) to 7 (highest degree of outside regions of interest surviving a threshold of
this particular emotion). p 0.001 uncorrected with an extent threshold
of 5 contiguous voxels.
To ensure that the main effect of trustworthiness
Data Analysis did not arise from a highly significant activation
Imaging data were analysed with SPM99 using an in just one of the simple effects (i.e., that activation
event-related model47. The experimental design was task independent), we created a mask from
allowed a parametric factorial analysis whereby random-effects SPMs for the simple effect of un-
trustworthiness was a parametric regressor and trustworthiness under both tasks (each thresholded
the task (age or trustworthiness judgment) the at p 0.05, uncorrected). This was used to mask
second factor. the main effect of trustworthiness. Activations
The presentation of each face was modeled by surviving this masking procedure reflect responses
convolving a delta function at each event onset during both implicit and explicit judgments.
with a canonical hemodynamic response function For the purposes of illustration, a second model
(HRF) and its temporal derivative to create re- was constructed by dividing the events for each
gressors of interest. These regressors were then subject into three groups by rank score for indi-
parametrically modulated to model subject-specific vidual stimuli (that is, the least trustworthy third
trustworthiness judgments: that is, the height of of faces as one event type, the median third as a
the HRF for stimuli was modulated as a function second, and the most trustworthy third as a third).
of the trustworthiness score assigned to that stim- This model is used in Figures 15.1–15.3 to
ulus by the subject. Subject-specific parameter demonstrate the direction of BOLD signal change
estimates pertaining to each regressor were calcu- with respect to trustworthiness score. Note that sta-
lated for each voxel48. Contrast images were cal- tistical inferences are drawn solely from the para-
culated by applying appropriate linear contrasts to metric model described above.
the parameter estimates for the parametric regres- The mean ratings of facial emotion derived
sor of each event. These contrast images were from a second set of 16 age-matched subjects (see
then entered into a one-sample t-test across the above) were used to construct another model for
14 subjects (that is, a random effects analysis). In the data. In this model, subject-specific ratings
regions about which we had a prior hypothesis, for trustworthiness were entered as parametric
we applied a correction for multiple comparisons covariates, as before. Additionally, mean ratings for
across a small volume of interest to the p-values each of the four emotions (anger, fear, happiness,
in this region49. We report predicted regions sur- sadness) were entered as nuisance parametric co-
viving this correction at p 0.05. Volumes of variates. The parameter estimates for trustworthi-
interest for amygdala, orbitofrontal cortex and ness are therefore rendered independent of the
STS were defined by drawing a mask around the effects of the four facial expressions, and variance
regions bilaterally on a normalized T1 structural better explained by the effects of a given facial
image with reference to an atlas of human neu- expression will be attributed to the regressor mod-
roanatomy50 using the software package MRIcro eling that facial expression. Contrast images for
(http://www.psychology.nottingham.ac.uk/staff/ trustworthiness derived from this model were
cr1/mricro.html). Total volume of the amygdala then entered into a random-effects analysis.
RT0996_C15.qxd 11/9/04 4:11 PM Page 209
Acknowledgments 17. Lane, R. D., Chau, P. M. & Dolan, R. J. Common effects of

emotional valence, arousal and attention on neural activation
This work was supported by a programme grant to during visual processing of pictures. Neuropsychologia
37, 989–997 (1999).
R.J.D. from the Wellcome Trust. 18. Morris, J. S., Ohman, A. & Dolan, R. J. Conscious and
unconscious emotional learning in the human amygdala.
Nature 393, 467–470 (1998).
REFERENCES
19. Whalen, P. J. et al. Masked presentations of emotional
1. Humphrey, N. Consciousness Regained: Chapters in the facial expressions modulate amygdala activity without
Development of Mind (Oxford Univ. Press, New York, 1983). explicit knowledge. J. Neurosci. 18, 411–418 (1998).
2. Adolphs, R., Tranel, D., Damasio, H. & Damasio, A. 20. Strange, B. A., Henson, R. N., Friston, K. J. & Dolan, R. J.
Impaired recognition of emotion in facial expressions fol- Brain mechanisms for detecting perceptual, semantic, and
lowing bilateral damage to the human amygdala. Nature emotional deviance. Neuroimage 12, 425–433 (2000).
372, 669–672 (1994). 21. Vuilleumier, P., Armony, J. L., Driver, J. & Dolan, R. J.
3. Hornak, J., Rolls, E. T. & Wade, D. Face and voice Effects of attention and emotion on face processing in the
expression identification in patients with emotional and human brain. An event-related fMRI study. Neuron 30,
behavioural changes following ventral frontal lobe dam- 829–841 (2001).
age. Neuropsychologia 34, 247–261 (1996). 22. Buechel, C., Morris, J., Dolan, R. J. & Friston, K. J. Brain
4. Adolphs, R., Damasio, H., Tranel, D., Cooper, G. & Damasio, systems mediating aversive conditioning: an event-related
A. R. A role for somatosensory cortices in the visual fMRI study. Neuron 20, 947–957 (1998).
recognition of emotion as revealed by three-dimensional 23. Buechel, C., Dolan, R. J., Armony, J. L. & Friston, K. J.
lesion mapping. J. Neurosci. 20, 2683–2690 (2000). Amygdala-hippocampal involvement in human aversive
5. Morris, J. S. et al. A differential neural response in the trace conditioning revealed through event-related functional
human amygdala to fearful and happy facial expressions. magnetic resonance imaging. J. Neurosci. 19, 10869–10876
Nature 383, 812–815 (1996). (1999).
6. Phillips, M. L. et al. A specific neural substrate for per- 24. Casey, K. L. Forebrain mechanisms of nociception and
ceiving facial expressions of disgust. Nature 389, 495–498 pain: analysis through imaging. Proc. Natl. Acad. Sci.
(1997). USA 96, 7668–7674 (1999).
7. Sprengelmeyer, R., Rausch, M., Eysel, U. T. & Przuntek, 25. Critchley, H. D., Mathias, C. J. & Dolan, R. J. Neural
H. Neural structures associated with recognition of facial activity in the human brain relating to uncertainty and
expressions of basic emotions. Proc. R. Soc. Lond. B Biol. arousal during anticipation. Neuron 29, 537–545 (2001).
Sci. 265, 1927–1931 (1998). 26. Damasio, A. The Feeling of What Happens: Body and
8. Adolphs, R., Tranel, D. & Damasio, A. R. The human Emotion in the Making of Consciousness (Harcourt Brace,
amygdala in social judgment. Nature 393, 470–474 New York, 1999).
(1998). 27. Critchley, H. D., Mathias, C. J. & Dolan, R. J. Neu-
9. Eslinger, P. J. & Damasio, A. R. Severe disturbance of roanatomical basis for first- and second-order representa-
higher cognition after bilateral frontal lobe ablation: patient tions of bodily states. Nat. Neurosci. 4, 207–212 (2001).
EVR. Neurology 35, 1731–1741 (1985). 28. Breiter, H. C. et al. Response and habituation of the human
10. Damasio, A. R. Descartes’ Error: Emotion, Reason and amygdala during visual processing of facial expression.
the Human Brain (Putnam, New York, 1994). Neuron 17, 875–887 (1996).
11. Brothers, L. The social brain: a project for integrating pri- 29. Isenberg, N. et al. Linguistic threat activates the human
mate behavior and neurophysiology in a new domain. amygdala. Proc. Natl. Acad. Sci. USA 96, 10456–10459
Concepts Neurosci. 1, 27–51 (1990). (1999).
12. Adolphs, R. Social cognition and the human brain. 30. Morris, J. S., Buchel, C. & Dolan, R. J. Parallel neural re-
Trends. Cogn. Sci. 3, 469–479 (1999). sponses in amygdala subregions and sensory cortex during
13. Adolphs, R. The neurobiology of social cognition. Curr. implicit fear conditioning. Neuroimage 13, 1044–1052
Opin. Neurobiol. 11, 231–239 (2001). (2001).
14. Allison, T., Puce, A. & McCarthy, G. Social perception 31. Dolan, R. J., Morris, J. S. & de Gelder, B. Crossmodal
from visual cues: role of the STS region. Trends Cogn. binding of fear in voice and face. Proc. Natl. Acad. Sci.
Sci. 4, 267–278 (2000). USA 98, 10006–10010 (2001).
15. Dolan, R. J. et al. Neural activation during covert process- 32. Morris, J. S. et al. A neuromodulatory role for the human
ing of positive emotional facial expressions. Neuroimage amygdala in processing emotional facial expressions.
4, 194–200 (1996). Brain 121, 47–57 (1998).
16. Blair, R. J., Morris, J. S., Frith, C. D., Perrett, D. I. & 33. Amaral, D. G. & Price, J. L. Amygdalo-cortical projections
Dolan, R. J. Dissociable neural responses to facial expres- in the monkey (Macaca fascicularis). J. Comp. Neurol.
sions of sadness and anger. Brain 122, 883–893 (1999). 230, 465–496 (1984).
RT0996_C15.qxd 11/9/04 4:11 PM Page 210
34. Anderson, A. K. & Phelps, E. A. Lesions of the human (eds. Baron-Cohen, S., Tager-Flusberg, H. & Cohen,
amygdala impair enhanced perception of emotionally D. J.) 334–356 (Oxford Univ. Press, New York, 2000).
salient events. Nature 411, 305–309 (2001). 42. Elliott, R. & Dolan, R. J. Neural response during prefer-
35. Bonda, E., Petrides, M., Ostry, D. & Evans, A. Specific ence and memory judgments for subliminally presented
involvement of human parietal systems and the amygdala stimuli: a functional neuroimaging study. J. Neurosci. 18,
in the perception of biological motion. J. Neurosci. 16, 4697–4704 (1998).
3737–3744 (1996). 43. Phillips, M. L. et al. Neural responses to facial and vocal
36. Castelli, F., Happe, F., Frith, U. & Frith, C. Movement expressions of fear and disgust. Proc. R. Soc. Land. B
and mind: a functional imaging study of perception and Biol. Sci. 265, 1809–1817 (1998).
interpretation of complex intentional movement patterns. 44. Friston, K. et al. Spatial registration and normalization of
Neuroimage 12, 314–325 (2000). images. Hum. Brain Mapp. 2, 165–189 (1995).
37. Fletcher, P. C. et al. Other minds in the brain: a functional 45. Talairach, J. & Tournoux, P. Co-planar Stereotaxic Atlas
imaging study of ‘theory of mind’ in story comprehension. of the Human Brain (Theime, Stuttgart, Germany, 1988).
Cognition 57, 109–128 (1995). 46. Ekman, P. & Friesen, W. V. Pictures of Facial Affect (Con-
38. Gallagher, H. L. et al. Reading the mind in cartoons sulting Psychologists Press, Palo Alto, California, 1975).
and stories: an fMRI study of ‘theory of mind’ in verbal 47. Josephs, O., Turner, R. & Friston, K. Event-related fMRI.
and nonverbal tasks. Neuropsychologia 38, 11–21 (2000). Hum. Brain Mapp. 5, 243–248 (1997).
39. Brunet, E., Sarfati, Y., Hardy-Bayle, M. C. & Decety, J. A 48. Friston, K. et al. Statistical parametric maps in functional
PET investigation of the attribution of intentions with a imaging: a general linear approach. Hum. Brain Mapp. 2,
nonverbal task. Neuroimage 11, 157–166 (2000). 189–210 (1995).
40. Frith, C. D. & Frith, U. Interacting minds—a biological 49. Worsley, K. et al. A unified statistical approach for deter-
basis. Science 286, 1692–1695 (1999). mining significant signals in images of cerebral activation.
41. Frith, C. & Frith, U. in Understanding Other Minds: Per- Hum. Brain Mapp. 4, 58–73 (1996).
spectives From Developmental Cognitive Neuroscience 50. Duvernoy, H. M. The Human Brain (Springer, Vienna, 1999).
RT0996_P09.qxd 11/8/04 2:14 PM Page 211
PA R T 9
Decision Making
•
The processes underlying choice were once thought to be straightforward:

individuals were conceptualized as rational decision makers who intuitively
calculated the expected value of alternatives and selected the option with
the highest expected value. To accomplish this feat, individuals were
assumed to know the utility of one good relative to an alternative, follow the
laws of probability, exhibit planning and forethought, avoid temptation, and
guess fairly accurately what others might do. As seen through the lens of
psychology and the neurosciences, this characterization of choice
processes is overly simplistic and misleading. Human decisions are subject
to a variety of internal biases and context effects from the availability
heuristic and confirmatory bias to groupthink. Psychological research has
demonstrated how alternative choices, previous experiences, associations,
culture, and ways of thinking about a situation all affect decisions in ways
that are not predicted by simple Bayesian models and Rational choices. For
instance, mental simulation of alternative outcomes through counterfactual
reasoning can dramatically influence our evaluations and decisions—so
much so that silver medalists in the Olympic Games are generally less
happy with their achievements than are bronze medalists.
A study of decision making we conducted with chimpanzees is illustrative
of the operation of both cognitive and emotional influences. The
chimpanzees in the study had been trained extensively in simple arithmetic
operations (counting, addition, subtraction) using Arabic numerals. During
testing, the chimps could see two reinforcement pans, each of which was
baited with different quantities of candies on each trial. A reverse
211
RT0996_P09.qxd 11/8/04 2:14 PM Page 212
reinforcement contingency was implemented such player rejected an offer, both players were left with
that the chimps received the candies from the pan to nothing. The goal of the game was to make as much
which it did not point. Thus, it was in the chimps money as possible. Such a task contingency meant it
best interest to select the smaller of the two candy was rational to accept inequitable as well as equitable
arrays in order to obtain the remaining, larger offers (because a small amount of money is better
quantity. Even after hundreds of training trials across than no money), and a rational player who anticipated
dozens of sessions, the chimpanzees’ performances this would offer inequitable divisions to the other
were significantly below chance. Moreover, their player. Neither of these rational choice predictions
performance worsened at higher reward ratios— were found, however, as most of the offers were
where they stood to benefit the most. This was true around $5 and around half of the inequitable offers
for all chimps in the study. In another condition, the were rejected. Sanfey and colleagues also used
chimps performed the same task but, rather than functional brain imaging (fMRI) to explore the regions
using candy arrays as stimuli, placards with of brain activation during the task. They found
corresponding Arabic numerals were substituted, inequitable relative to equitable offers were associated
and the chimps received the number of candies that with increased activity in the dorsolateral prefrontal
corresponded to the nonselected numeral. This cortex, bilateral anterior insula, and anterior
change led to an immediate above-chance cingulate—a pattern of activation suggesting that
performance. When the candies were again used as inequitable offers evoked more thought, conflict, and
experimental stimuli, performance fell immediately negative affect. Furthermore, the greater the activation
to below-chance levels, and when the Arabic of the anterior insula—an area involved in negative
numerals were used performance rose immediately emotions—the more likely the individual was to reject
to above chance. These results suggested that the an inequitable offer. Sanfrey et al.’s study, therefore,
chimps had acquired the rules of the task but this suggests that the choice processes are influenced not
knowledge, or at least its effect on behavior, was only by the goals and opportunities that exist in the
obscured by a potent competing emotional game or task, but also the attributions made to and
disposition arising from the intrinsic incentive emotions aroused by others.
properties of the candy arrays. Bar-On et al. (2003) build on this theme in their
The examination of the neural circuitry that is study of the details of decision making and social
activated during decision making by Sanfey et al. reasoning in two groups of individuals—those with
(2003) suggests that emotion and cognition are lesions in the ventromedial prefrontal cortex and
potent influences in human primates, as well. In their either the amygdala or insula cortices, and those
study, a player could propose a division of $10 to with lesions in other regions of the brain. Several
another player, who could either accept or reject the interesting results were obtained. First, the group
offer. Offers around $5 were equitable, whereas offers with lesions within the circuit comprised of the
less than $5 were inequitable splits. If the second ventromedial prefrontal cortex, amygdala, and insula
RT0996_P09.qxd 11/8/04 2:14 PM Page 213
Part 9: Decision Making ■ 213
performed more poorly on measures of social participants performed comparably on measures of

intelligence and decision making than the control cognitive intelligence and psychopathology, evidence
group. The competencies most affected by lesions in that cognitive and social intelligence are separable
this circuit were accurate self-awareness, self- constructs, with the latter critical for adaptive
expression, and self-regulation (e.g., changing decision making in a changing environment.
behavior when contingencies change), which Bar-On Together, these studies contribute to the
suggests may mean that the deficits in decision accumulating evidence that the brain is not simply
making may be caused in large part by the deficits in an analytic machine but a social information
social intelligence. Furthermore, the two groups of processing mechanism.
RT0996_P09.qxd 11/8/04 2:14 PM Page 214
RT0996_C16.qxd 11/8/04 2:10 PM Page 215
R E A D I N G 1 6
The Neural Basis of Economic Decision-Making

in the Ultimatum Game
•
Alan G. Sanfey,1,3 James K. Rilling,1* Jessica A. Aronson,2 Leigh E.
Nystrom,1,2 Jonathan D. Cohen1,2,4
The nascent field of neuroeconomics seeks to ground economic decision-making in the biological
substrate of the brain. We used functional magnetic resonance imaging of Ultimatum Game players
to investigate neural substrates of cognitive and emotional processes involved in economic
decision-making. In this game, two players split a sum of money; one player proposes a division
and the other can accept or reject this. We scanned players as they responded to fair and unfair
proposals. Unfair offers elicited activity in brain areas related to both emotion (anterior insula) and
cognition (dorsolateral prefrontal cortex). Further, significantly heightened activity in anterior insula
for rejected unfair offers suggests an important role for emotions in decision-making.
tandard economic models of human decision- neuroimaging to examine behavior in economic

S making (such as utility theory) have typically
minimized or ignored the influence of emotions on
games (3). This study applies functional neu-
roimaging techniques to investigate the relative
people’s decision-making behavior, idealizing the contributions of cognitive and emotional processes
decision-maker as a perfectly rational cognitive to human social decision-making.
machine. However, in recent years this assumption The limitations of the standard economic
has been challenged by behavioral economists, model are effectively illustrated by empirical find-
who have identified additional psychological and ings from a simple game known as the Ultimatum
emotional factors that influence decision-making Game. In the Ultimatum Game, two players are
(1, 2), and recently researchers have begun using given the opportunity to split a sum of money.
1
Center for the Study of Brain, Mind and Behavior, 2Depart- of Psychiatry, University of Pittsburgh, Pittsburgh, PA 15260,
ment of Psychology, 3Center for Health and Well-Being, USA.
Princeton University, Princeton, NJ 08544, USA. 4Department
215
RT0996_C16.qxd 11/8/04 2:10 PM Page 216
One player is deemed the proposer and the other, in the Ultimatum Game. We were interested in
the responder. The proposer makes an offer as to neural and behavioral reactions to offers which
how this money should be split between the two. were fair (the money is split 50:50) or unfair (the
The second player (the responder) can either proposer offered an unequal split to his or her
accept or reject this offer. If it is accepted, the advantage). In particular, we hypothesized that
money is split as proposed, but if the responder unfair offers would engage neural structures in-
rejects the offer, then neither player receives volved in both emotional and cognitive process-
anything. In either event, the game is over. ing, and that the magnitude of activation in these
The standard economic solution to the Ultima- structures might explain variance in the subse-
tum Game is for the proposer to offer the smallest quent decision to accept or reject these offers.
sum of money possible to the responder and for the Before scanning, each participant was intro-
responder to accept this offer, on the reasonable duced to 10 people they were told would partner
grounds that any monetary amount is preferable to with them in the games to follow. They were told
none. However, considerable behavioral research in that they would play a single iteration of the game
industrialized cultures indicates that, irrespective of with each partner and that their decisions with
the monetary sum, modal offers are typically each partner would not be revealed to the other
around 50% of the total amount. Low offers (around partners and, therefore, could not affect subse-
20% of the total) have about a 50% chance of being quent offers. The participants were then placed
rejected (4–8). This latter, quite robust, experimen- inside the MRI scanner and began playing the
tal finding is particularly intriguing, demonstrating Ultimatum Game with their partners via a com-
that circumstances exist in which people are moti- puter interface (Figure 16.1A) (13). They com-
vated to actively turn down monetary reward. pleted 30 rounds in all, 10 playing the game with
Why do people do this? The game is so simple a human partner (once with each of the 10 part-
that it is improbable that these rejections are due ners), 10 with a computer partner, and a further
to a failure to understand the rules of the game, or 10 control rounds in which they simply received
an inability to conceptualize a single-shot interac- money for a button press. The rounds were pre-
tion with a partner (9). On the basis of participant sented randomly, and all involved splitting $10.
reports, it appears that low offers are often re- Offers made by human partners in fact adhered to
jected after an angry reaction to an offer perceived a predetermined algorithm, which ensured that all
as unfair (10). Objecting to unfairness has been participants saw the same set (and a full range) of
proposed as a fundamental adaptive mechanism offers (14, 15). Half of these offers were fair, that
by which we assert and maintain a social reputa- is, a proposal to split the $10 evenly ($5:$5), with
tion (11), and the negative emotions provoked the remaining half proposing unequal splits (two
by unfair treatment in the Ultimatum Game can offers of $9:$1, two offers of $8:$2, and one
lead people to sacrifice sometimes considerable offer of $7:$3). The 10 offers from the computer
financial gain in order to punish their partner for partner were identical to those from the human
the slight. Unfair offers in the Ultimatum Game partners (half fair, half unfair). The 10 control
induce conflict in the responder between cogni- trials were designed to control for the response to
tive (“accept”) and emotional (“reject”) motives, monetary reinforcement, independent of the social
motives that we might expect to see represented in interaction. The distribution of offers generally
brain areas implicated in cognitive and emotional mimics the range of offers typically made in un-
modes of thought, with additional regions possi- controlled versions of the game (i.e., involving
bly mediating these competing goals (12). freely acting human partners).
To shed light on the neural and psychological Behavioral results were very similar to those
processes mediating such behaviors, we scanned typically found in Ultimatum Game experiments
19 participants using functional magnetic resonance (Figure 16.1B) (16). Participants accepted all fair
imaging (fMRI), each in the role of the responder offers, with decreasing acceptance rates as the
RT0996_C16.qxd 11/8/04 2:10 PM Page 217
The Neural Basis of Economic Decision-Making in the Ultimatum Game ■ 217
FIGURE 16.1 ■ (A) Time line for a single round of the Ultimatum Game. Each round lasted 36 s. Each round began with
a 12-s preparation interval. The participant then saw the photograph and name of their partner in that trial for 6 sec-
onds. A picture of a computer was shown if it was a computer trial, or a roulette wheel if it was a control trial. Next,
participants saw the offer proposed by the partner for a further 6 s, after which they indicated whether they accepted
or rejected the offer by pressing one of two buttons on a button box. (B) Behavioral results from the Ultimatum Game.
These are the offer acceptance rates averaged over all trials. Each of 19 participants saw five $5:$5 offers, one $7:$3
offer, two $8:$2 offers, and two $9:$1 offers from both human and computer partners (20 offers in total).
offers became less fair. Unfair offers of $2 and $1 significantly greater activation for a $9:$1 offer than
made by human partners were rejected at a signif- an $8:$2 offer from a human partner (Figure 16.2e)
icantly higher rate than those offers made by a (left insula, P 0.001; right insula, P 0.01), in
computer ($9:$1 offer: 2 5.28, 1 df, P 0.02; addition to the aforementioned greater activation
$8:$2 offer: 2 8.77, 1 df, P 0.003), suggest- for unfair offers than fair ($5:$5) offers.
ing that participants had a stronger emotional Activation of bilateral anterior insula to unfair
reaction to unfair offers from humans than to the offers from human partners is particularly interest-
same offers from a computer (17). ing in light of this region’s oft-noted association
Among the areas showing greater activation for with negative emotional states. Anterior insula
unfair compared with fair offers from human part- activation is consistently seen in neuroimaging
ners (Figure 16.2a and b; Table S1) were bilateral studies of pain and distress (18–20), hunger and
anterior insula, dorsolateral prefrontal cortex thirst (21, 22), and autonomic arousal (23). This
(DLPFC), and anterior cingulate cortex (ACC). region has also been implicated in studies of
The magnitude of activation was also significantly emotion, in particular involvement in the evaluation
greater for unfair offers from human partners as and representation of specific negative emotional
compared to both unfair offers from computer states (24). Chief amongst these are anger and
partners (left insula: t 2.52, P 0.02; right disgust, both of which have been found to engage
insula: t 2.2, P < 0.03) and low control offers a distinct region of the anterior insula activated by
(left insula: t 3.46, P 0.001; right insula: an unfair offer in the present study (25, 26).
t 2.83, P 0.05). This suggests that these acti- Though studies of disgust have largely focused on
vations were not solely a function of the amount physical sensations of taste and odor (27), it has
of money offered to the participant but rather been suggested that emotion-based disgust (as
were also uniquely sensitive to the context, perhaps induced by an insultingly unfair offer)
namely perceived unfair treatment from another may be conceptually similar. The recruitment of
human (Figure 16.2c and d). Further, regions of similar neural structures, namely the anterior
bilateral anterior insula demonstrated sensitivity insula, in both physical and moral disgust gives
to the degree of unfairness of an offer, exhibiting some credence to this notion.
RT0996_C16.qxd 11/8/04 2:10 PM Page 218
a) b) c)
d) e)
FIGURE 16.2 ■ (A color version of parts a and b of this figure follows page 146.) Activation related to the
presentation of an unfair offer. (a) Map of the t statistic for the contrast [unfair human offer – fair human offer]
showing activation of bilateral anterior insula and anterior cingulate cortex. Areas in orange showed greater
activation following unfair as compared with fair offers (P 0.001). (b) Map of the t statistic for the contrast
[unfair human offer – fair human offer] showing activation of right dorsolateral prefrontal cortex. (c) Event-related
plot for unfair and fair offers in right anterior insula. The offer was revealed at t O on the x axis. (d) Event-related
plot for unfair and fair offers in left anterior insula. (e) Event-related plot for different human unfair and fair offers
in subset of left anterior insula.
If the activation in the anterior insula is a reflec- r 0.39, P 0.05, one-tailed) (Figure 16.3a).
tion of the responders’ negative emotional response Of particular interest is whether these differences
to an unfair offer, we might expect activity in this in anterior insular activation extend to the trial
region to correlate with the subsequent decision to level. Looking across participants, an examination
accept or reject the offer. Because all fair offers and of individual trials also revealed a relation between
the vast majority of $7:$3 offers were accepted, we right anterior insular activity and the decision to ac-
focused on the $8:$2 and $9:$1 offers from a cept or reject (Figure 16.3b). Activation in this area
human partner for the analysis of whether neural was significantly greater in response to unfair
activity was related to the decision made in the offers that were later rejected (P 0.028, one-
game. Indeed, looking at the participant level, tailed). These results provide additional support
those participants with stronger anterior insula for the hypothesis that neural representations of
activation to unfair offers rejected a higher propor- emotional states guide human decision-making.
tion of these offers (right insula: correlation coeffi- In contrast to the insula, DLPFC usually has
cient r 0.45, P 0.025, one-tailed; left insula: been linked to cognitive processes such as goal
RT0996_C16.qxd 11/8/04 2:10 PM Page 219
a) the decision, we examined the balance between

activation in anterior insula and DLPFC for unfair
offers. Unfair offers that are subsequently rejected
have greater anterior insula than DLPFC activation,
whereas accepted offers exhibit greater DLPFC
than anterior insula (Figure 16.3b). The contrast
in activation between these two areas is signifi-
cantly different for accepted and rejected offers
(P 0.033, one-tailed), consistent with the
hypothesis that competition between these two
b) regions influences behavior. DLPFC activity
remains relatively constant across unfair offers,
perhaps reflecting the steady task representation
of money maximization, with anterior insula
scaling monotonically to the degree of unfairness,
reflecting the emotional response to the offer.
Caution is needed when comparing the magnitude
of the fMRI signal across brain regions. However,
it is interesting to note that the outcome of the
decision may reflect the relative engagement of
these regions, with greater anterior insula activa-
tion biasing toward rejection and greater DLPFC
biasing toward acceptance. Finally, unfair offers
were also associated with increased activity in
FIGURE 16.3 ■ (a) Acceptance rates of unfair offers ACC. ACC has been implicated in detection of
plotted against right anterior insula activation for each cognitive conflict (30, 31), and activation here
participant. (b) Right anterior insula and right DLPFC may reflect the conflict between cognitive and
activation for all unfair offer trials, categorized by subse-
quent acceptance or rejection.
emotional motivations in the Ultimatum Game.
This study sought to identify the neural corre-
lates of fairness and unfairness, and in particular
maintenance and executive control (28, 29). Thus, the relative contributions of cognitive and emo-
the DLPFC activation we observed in response to tional processes to human decision-making. A
unfair offers may relate to the representation and basic sense of fairness and unfairness is essential
active maintenance of the cognitive demands of to many aspects of societal and personal decision-
the task, namely the goal of accumulating as making and underlies notions as diverse as ethics,
much money as possible. An unfair offer is more social policy, legal practice, and personal moral-
difficult to accept, as indicated by the higher ity. Our results are consistent with the idea that
rejection rates of these offers, and hence higher the areas of anterior insula and DLPFC represent
cognitive demands may be placed on the partici- the twin demands of the Ultimatum Game task, the
pant in order to overcome the strong emotional emotional goal of resisting unfairness and the
tendency to reject the offer. Although DLPFC cognitive goal of accumulating money, respec-
activated to unfair offers, this activation did not tively. Further, our finding that activity in a region
correlate with acceptance rates (r = 0.04, well known for its involvement in negative emo-
P 0.05), suggesting that activation of this region tion is predictive of subsequent behavior supports
alone is not sufficient to predict behavior. However, the importance of emotional influences in human
motivated by the hypothesis that this region may decision-making. We believe that these findings,
be competing with emotional areas in influencing and work that proceeds from them, will provide a
RT0996_C16.qxd 11/8/04 2:10 PM Page 220
more detailed characterization of specific emo- unfair offers). The term “cognitive” is perhaps also prob-
tional responses, their neural substrates, and the so- lematic, for similar reasons. Terms such as “proximal”
and “distal” may be more accurate, respectively indicating
cial circumstances under which they are elicited. the immediate and longer-term sources of gain associated
Therefore, not only do our results provide direct with the behavior. However, until the field converges on a
empirical support for economic models that ac- new set of accepted terms for designating these classes of
knowledge the influence of emotional factors on motivation, we use the terms cognitive and emotional as
decision-making behavior, but they also provide intuitively accessible, if not technically accurate.
13. Materials and methods are available as supporting material
the first step toward the development of quantita- on Science Online.
tive measures that may be useful in constraining 14. This methodology deviates somewhat from the standards
the social utility function in economic models of experimental economics, a field that generally pro-
(32, 33). Models of decision-making cannot afford scribes the use of deception [see (34) for a summary of the
to ignore emotion as a vital and dynamic compo- issues, though there are some exceptions (35)]. We chose
to use a limited amount of deception in the current study
nent of our decisions and choices in the real world. primarily because of the heavy logistic demands of an
fMRI study, requiring a full distribution of offers in a
constrained number of participants. Practical issues
Supporting Online Material notwithstanding, we believe the use of deception had little
if any impact on our results, and any effect was not likely to
www.sciencemag.org/cgi/content/full/300/5626/ confound their interpretation. During the post-experiment
1755/DC1 debriefing, no subject gave any suggestion that they had
been suspicious of the offers they received. Further, the be-
Materials and Methods havioral results in the human partner condition replicate
those found in versions of the game using no deception,
Table S1 with approximately half of offers of 20% or less of the total
being rejected (9). Perhaps most importantly, if subjects
suspected deception, this should have diminished emo-
REFERENCES AND NOTES
tional responses (i.e., if subjects suspected the offers to be
1. C. Camerer, G. Loewenstein, in Advances in Behavioral fictitious, their emotional reactions to these offers, particu-
Economics, C. Camerer, G. Loewenstein, M. Rabin, Eds. larly unfair offers, should have been muted). The fact that
(Princeton Univ. Press, Princeton, NJ), in press. we observed significant effects consistent with emotional
2. G. Loewenstein, J. Lerner, in The Handbook of Affective responses suggests, once again, that the effects of deception
Science, R. J. Davidson, H. H. Goldsmith, K. R. Scherer, were minimal and, if they were present, have simply caused
Eds. (Oxford Univ. Press, Oxford, 2003). an underestimate of the observed effects. Although we are
3. K. McCabe, D. Houser, L. Ryan, V. Smith, T. Trouard, sensitive to the issue of deception, we believe that the
Proc. Natl. Acad. Sci. U.S.A. 98, 11832 (2001). methodological constraints of fMRI justified our practice
4. W. Guth, R. Schmittberger, B. Schwarze, J. Econ. Behav. and that the findings do not appear to be tainted by subjects’
Organ. 3, 376 (1982). possible perceptions of the deception used.
5. R. H. Thaler, J. Econ. Perspect, 2, 195 (1988). 15. A common concern regarding the use of deception in-
6. G. E. Bolton, R. Zwick, Game Econ. Behav. 10, 95 (1995). volves possible contamination of the participant pool. As
7. A. E. Roth, in Handbook of Experimental Economics, mentioned previously, rejection rates in the current study
J. H. Kagel, A. E. Roth, Eds. (Princeton Univ. Press, replicate those typically reported from uncontrolled
Princeton, NJ, 1995). Ultimatum Game studies; therefore, we do not believe we
8. See J. Henrich et al. [Am. Econ. Rev. 91, 73 (2001)] for suffered unduly from this. Furthermore, a comparison of
Ultimatum Game research in simple societies. rejection rates over the course of the experiment (i.e.,
9. C. Camerer, R. H. Thaler, J. Econ. Perspect. 9, 209 longitudinally over participants) indicates no systematic
(1995). trends in these rates (mean rejection rate of offers for first
10. M. M. Pillutla, J. K. Murnighan, Organ. Behav. Hum. six participants was 32%; mean rate for last six partici-
Dec. Proc. 68, 208 (1996). pants was 35%).
11. M. A. Nowak, K. M. Page, K. Sigmund, Science 289, 16. After the conclusion of the Ultimatum Game with all part-
1773 (2000). ners, subjects then played a single round of the Prisoner’s
12. We use the term “cognitive” here, in place of the term Dilemma (PD) game with each of the partners. This raises
“rational” (as commonly used in the traditional economic the possibility that subjects did not treat the Ultimatum
literature), in recognition of the fact that emotional Game as a true single-shot game. We do not believe
responses may also have a rational basis (e.g., to punish playing the PD game affected their play in the Ultimatum
RT0996_C16.qxd 11/8/04 2:10 PM Page 221
Game in this study for several reasons. First, our behav- 22. P. A. Tataranni et al. Proc. Natl. Acad. Sci. U.S.A. 96, 4569
ioral results support the notion that the Ultimatum Game (1999).
was played as a single-shot game. As noted above, the 23. H. D. Critchley, R. Elliott, C. J. Mathias, R. J. Dolan,
proportion of rejected offers in our study matches propor- J. Neurosci, 20, 3033 (2000).
tions reported in the experimental economic literature 24. A. J. Calder, A. D. Lawrence, A. W. Young, Nature Rev.
when the game is strictly controlled as single-shot. We Neurosci, 2, 352 (2001).
would have expected much higher rejection rates in an it- 25. A. R. Damasio et al. Nature Neurosci. 3, 1049 (2000).
erated Ultimatum Game. Second, unpublished data of 26. M. L. Phillips et al. Nature 389, 495 (1997).
ours using a single-shot Ultimatum Game (with no subse- 27. P. Rozin, A. E. Fallon, Psychol. Rev. 94, 23 (1987).
quent task) produced rejection rates of unfair offers that 28. E. K. Miller, J. D. Cohen, Annu. Rev. Neurosci 24, 167
are virtually identical to those reported here ($8:$2 split, (2001).
47% versus 49%; $9:$1 split, 61% versus 60%). We be- 29. A. D. Wagner, A. Maril, R. A. Bjork, D. L. Schacter,
lieve this evidence strongly suggests that subjects treated NeuroImage 14, 1337 (2001).
the Ultimatum Game as a single-shot game, as instructed. 30. A. W. MacDonald III, J. D. Cohen, V. A. Stenger, C. S.
17. We asked our participants as part of the debriefing process Carter, Science 288, 1835 (2000).
what they considered a “fair” offer to be irrespective of 31. M. Botvinick, L. E. Nystrom, K. Fissell, C. S. Carter, J. D.
their decision to accept or reject, thus providing an indica- Cohen, Nature 402, 179 (1999).
tion of their standards of fairness. Of our participants, 32. E. Fehr, K. M. Schmidt, Q. J. Econ. 114, 817 (1999).
58% considered any offer less than $5:$5 as unfair, with 33. G. E. Bolton, A. Ockenfels, Am Econ. Rev. 90, 166 (2000).
the remaining 42% deeming anything less than $7:$3 to 34. R. Hertwig, A. Ortmann, Behav. Brain Sci. 24, 383 (2001).
be an unfair division. 35. S. Bonetti, J. Econ. Psychol. 19, 377 (1998).
18. S. W. Derbyshire et al. Pain 73, 431 (1997). 36. We would like to thank D. Kahneman, A. Scheres, and
19. M. J. Iadarola et al. Brain 121, 931 (1998). three anonymous reviewers for their helpful comments.
20. K. C. Evans et al. J. Neurophysiol 88, 1500 (2002). This work was supported in part by grants from the Seaver
21. D. Denton et al. Proc. Natl. Acad. Sci. U.S.A. 96, 5304 (1999). Institute and the Mind, Brain, Body, and Health Initiative.
RT0996_C16.qxd 11/8/04 2:10 PM Page 222
RT0996_C17.qxd 11/8/04 2:10 PM Page 223
R E A D I N G 1 7
Exploring the Neurological Substrate of

Emotional and Social Intelligence
•
Reuven Bar-On,1, Daniel Tranel,2 Natalie L. Denburg2 and
Antoine Bechara2
The somatic marker hypothesis posits that deficits in emotional signalling (somatic states) lead to
poor judgment in decision-making, especially in the personal and social realms. Similar to this
hypothesis is the concept of emotional intelligence, which has been defined as an array of
emotional and social abilities, competencies and skills that enable individuals to cope with daily
demands and be more effective in their personal and social life. Patients with lesions to the
ventromedial (VM) prefrontal cortex have defective somatic markers and tend to exercise poor
judgment in decision-making, which is especially manifested in the disadvantageous choices they
typically make in their personal lives and in the ways in which they relate with others. Furthermore,
lesions to the amygdala or insular cortices, especially on the right side, also compromise somatic
state activation and decision-making. This suggests that the VM, amygdala and insular regions are
part of a neural system involved in somatic state activation and decision-making. We hypothesized
that the severe impairment of these patients in real-life decision-making and an inability to cope
effectively with environmental and social demands would be reflected in an abnormal level of
emotional and social intelligence. Twelve patients with focal, stable bilateral lesions of the VM
cortex or with right unilateral lesions of the amygdala or the right insular cortices, were tested on
the Emotional Quotient Inventory (EQ-i), a standardized psychometric measure of various aspects
of emotional and social intelligence. We also examined these patients with various other
procedures designed to measure decision-making (the Gambling Task), social functioning, as well
1
Emotional Intelligence Research Laboratory, Trent University, and Cognitive Neuroscience, University of Iowa College of
Peterborough, Canada, and 2Division of Behavioral Neurology Medicine, Iowa City, IA, USA.
223
RT0996_C17.qxd 11/8/04 2:10 PM Page 224
as personality changes and psychopathology; emotions (Damasio, 1994). In essence, they are
standardized neuropsychological tests were applied cognitively intelligent but typically behave in an
unintelligent manner with respect to exercising
to assess their cognitive intelligence, executive judgment in making decisions of a personal and
functioning, perception and memory as well. Their interpersonal nature. Based on these clinical obser-
results were compared with those of 11 patients vations, the somatic marker hypothesis has been
proposed to explain this specific type of impairment
with focal, stable lesions in structures outside the (Damasio, 1994). The somatic marker hypothesis
neural circuitry thought to mediate somatic state outlines a neurological explanation for decision-
activation and decision-making. Only patients with making impairment. This hypothesis suggests that
decision-making is a process that depends on emo-
lesions in the somatic marker circuitry revealed tional signals, which are defined as bio-regulatory
significantly low emotional intelligence and poor responses aimed at maintaining homeostasis and
judgment in decision-making as well as disturbances ensuring survival. The roots of this concept, as well
as those of emotional intelligence, can be traced
in social functioning, in spite of normal levels of back to Charles Darwin’s early research that culmi-
cognitive intelligence (IQ) and the absence of nated in the publication of the first scientific work
psychopathology based on DSM-IV criteria. The on the topic titled The Expression of the Emotions in
Man and Animals (Darwin, 1872).
findings provide preliminary evidence suggesting The somatic marker hypothesis specifies a num-
that emotional and social intelligence is different ber of structures and operations required for the
from cognitive intelligence. We suggest, moreover, normal function of decision-making. In brief, the
hypothesis posits that the amygdala is a critical
that the neural systems supporting somatic state substrate in the system that triggers somatic states
activation and personal judgment in decision-making activated by primary inducers (Bechara et al.,
may overlap with critical components of a neural 1999). Primary inducers are unconditioned stim-
uli that are innately set as pleasurable or aversive,
circuitry subserving emotional and social or conditioned stimuli, which when they are
intelligence, independent of the neural system present in the immediate environment, automati-
supporting cognitive intelligence. cally and obligatorily elicit a somatic response.
Secondary inducers are entities generated by
Keywords: social cognition; decision-making; recall or by thought, and they elicit a somatic
response when brought to memory (Damasio,
emotion; somatic markers; emotional intelligence. 1995). Once somatic states from primary inducers
are induced, signals from them are relayed to the
Introduction brain. Representations of these signals can remain
covert at the level of the brainstem, or can reach
Patients with lesions to the ventromedial (VM) the parietal cortices (insular/SI, SII) and posterior
cortex are subject to impaired personal judgment in cingulate cortices and be perceived as a feeling
decision-making, which is frequently manifested in (Maddock, 1999; Damasio et al., 2000). When we
the manner in which they relate with others. This process a secondary inducer, i.e., recall an event
specific type of impairment is observed even in associated with a feeling, we may re-enact the
cases in which cognitive capacity (IQ) falls within somatic state characteristic of the feeling. The
the normal or even above-normal range. In spite VM cortex is a trigger structure for somatic states
of normal intellectual capacity, moreover, these from secondary inducers (Bechara et al., 1999).
patients also reveal a compromised ability to Decision-making is a complex process that some-
experience, understand, express and effectively use times involves a conflict between a primary
RT0996_C17.qxd 11/8/04 2:10 PM Page 225
Exploring the Neurological Substrate of Emotional and Social Intelligence ■ 225
inducer (which may be positive) and a secondary achieve personal goals. The concept of emotional
inducer (which may be negative). Sometimes, the intelligence is closely related to the concept of
conflict could be between only secondary induc- social intelligence. This conceptual proximity is
ers (e.g., a positive thought versus a negative evident in the way in which social intelligence
thought). Regardless of how they are triggered, was first defined by Thorndike in 1920
once somatic states induced by primary and/or (Thorndike, 1920)—the ability to perceive one’s
secondary inducers are enacted in the body, all own and others’ internal states, motives and
these somatic states, which can be either positive behaviours, and to act toward them optimally on
or negative, are summed into one overall somatic the basis of that information. Because of the
state (figuratively and literally speaking). This similarity between both concepts, some psycholo-
overall somatic state then provides signals to the gists have suggested that they may relate to differ-
brain that participate in at least two functions. (i) ent aspects of the same construct and could
In one function, it provides a substrate for feeling actually be referred to as ‘emotional and social
the overall emotional state, possibly via the in- intelligence’ (Bar-On, 2000, 2001). With respect to
sular/somatosensory cortices (SI, SII). (ii) In the the present study, moreover, it is important to note
other function, it provides a substrate for biasing that both concepts and closely related constructs
the decision to select a response (Damasio, 1999). are thought by some to be based on a cognitive
This biasing effect may occur covertly at the level schemata (Mayer and Salovey, 1997; Taylor et al.,
of the striatum, in which case the person acts with- 1997; Lane, 2000; Zirkel, 2000). This means that
out a conscious decision to do so. The biasing what is perceptually scanned in one’s immediate
effect may also occur overtly at the level of the environment, together with one’s emotional reac-
lateral orbitofrontal cortex when the person favours tion to that which is perceived, may then be
a plan of action, and at the anterior cingulate when processed, evaluated and understood in order to
the person executes a plan of action that is under effectively guide interpersonal behaviour. In the
volitional control. somatic marker hypothesis, it is important to note
The concept of the somatic marker hypothesis that there is also a covert route that mediates be-
is thought to overlap with the concept of emo- haviour, which does not require information to be
tional intelligence regarding the use of emotions processed, evaluated and understood. However, it
to guide human behaviour (Bechara et al., 2000a). is not yet certain if such a covert route applies to the
The conceptural nexus between the somatic emotional and social intelligence construct as well.
marker hypothesis and emotional intelligence can Together with cognitive intelligence, emotional
be seen in the way the latter has been defined by and social intelligence form important components
some researchers (Bar-On, 2000, 2001): A multi- of general intelligence. One of the major differences
factorial array of interrelated emotional, personal between the two is that the former is thought to re-
and social competencies that influence our ability late primarily to higher order mental processes like
to actively and effectively cope with daily de- reasoning, while the latter focuses more on perceiv-
mands. Most conceptualizations of this construct ing, immediate processing and applying emotional
address one or more of the following basic com- and social content, information and knowledge. It
ponents: (i) the ability to be aware of and express has also been suggested that another fundamental
emotions; (ii) the ability to be aware of others’ difference between the two may be that cognitive
feelings and to establish interpersonal relation- intelligence is more cortically strategic in nature,
ships; (iii) the ability to manage and regulate while emotional and social intelligence is more lim-
emotions; (iv) the ability to realistically and flexi- bically tactical for immediate behaviour suited
bly cope with the immediate situation and solve more for survival and adaptation (Goleman, 1995;
problems of a personal and interpersonal nature as Bar-On, 1997a; Stein and Book, 2000). However,
they arise; and (v) the ability to generate positive thus far these theories are supported more by sup-
affect in order to be sufficiently self-motivated to position than by empirical findings.
RT0996_C17.qxd 11/8/04 2:10 PM Page 226
One of the primary purposes of this study was brain injury along the circuitry thought to mediate
to provide empirical evidence in support of the somatic state activation and decision-making.
hypothesis that emotional and social intelligence Because of the relatively small sample sizes
is different from cognitive intelligence, in that involved in the present study, non-parametric
these two major components of general intel- statistics were applied; the Mann–Whitney U test
ligence are supported by separate neural sub- was employed to examine the groups being com-
strates. Furthermore, we hypothesized that the pared for significant differences (Siegal, 1956).
neural systems that support emotional and social in- In tandem with this conservative approach, a two-
telligence overlap with neural systems subserving tailed evaluation was applied in interpreting the
somatic state activation and personal judgment in probability levels of the results. It was decided,
decision-making, which are separate from the moreover, to discuss significant differences on
neural systems supporting cognitive intelligence. Emotional Quotient Inventory (EQ-i) subscale
We predicted that patients with lesions in critical scores (e.g. emotional self-awareness) only if the
components of the somatic marker circuitry parent composite scale score (i.e. intrapersonal EQ)
(i.e. amygdala, VM prefrontal and insular/ proved to be significantly different when comparing
somatosensory cortices), who demonstrate severe the independent groups being examined.
impairments in real-life decision-making and an
inability to cope effectively with environmental Subjects
and social demands, would also demonstrate an The experimental and control groups were
abnormally low level of emotional and social in- matched with respect to gender, age, level of
telligence. Specifically, we predicted that these education and handedness. As can be seen in
patients would present significantly lower scores Table 17.1 the subjects were predominantly right-
and ratings than the control group (patients with handed, in their mid-forties and had 14 to 15 years
lesions outside the neural circuitry involved with the of education with no significant difference between
somatic state activation) on measures of emotional the groups regarding age or education. The ex-
intelligence, decision-making as well as overall perimental group included seven males and five
social functioning, despite maintaining normal females, and the control group comprised four
levels of cognitive intelligence. males and seven females.
Subjects with brain lesions were selected from
the Patient Registry of the University of Iowa’s
Methods Division of Cognitive Neuroscience. All brain-
damaged subjects had undergone basic neuropsy-
Twenty-three neurological patients were selected chological and neuroanatomical characterization
and divided into an experimental group and a con- according to the standard protocols of the Benton
trol group based on the presence or absence of Neuropsychology Laboratory (Tranel, 1996) and
TABLE 17.1. Demographic Data of Participating Subjects

Demographic data Control group Experimental group Z P-level (2-tailed)
Total number of participants 11 12 – –

Gender (male/female) 4/7 7/5 –/– –/–
Age (years) 47.1 43.5 0.46 0.644
Age range (years) 24–74 21–63 – –
Education (years) 14.6 13.7 0.85 0.398
Handedness (right/left) 10/1 12/0 –/– –/–
The Mann–Whitney U test was applied to compare the average age and years of education between the control group and the
experimental group for significant differences.
RT0996_C17.qxd 11/8/04 2:10 PM Page 227
the Laboratory of Neuroimaging and Human relative to the left. In all subjects, the dorsolateral
Neuroanatomy (Damasio and Damasio, 1989; sectors of the prefrontal cortex were intact.
Damasio and Frank, 1992; Damasio, 1995). All Amygdala lesions. Three subjects had lesions
subjects provided informed consent, which was involving the amygdala. Patients with bilateral
approved by the appropriate human subject amygdala damage are extremely rare. Therefore,
committees at the University of Iowa. Based on we decided to select patients with unilateral
clinical interviews, none of the subjects in this study amygdala damage. Because the somatic marker
had a history of mental retardation, psychiatric hypothesis emphasizes the role of the right
disorder, substance abuse, learning disability or hemisphere as opposed to the left in somatic state
systemic disease that might affect the CNS. activation and in light of recent support of this
The selection of subjects with brain lesions view in relation to patients with prefrontal cortex
conformed to the following criteria: (i) a stable damage (Tranel et al., 2000; Manes et al., 2002),
and chronic lesion at least 3 months post onset; we selected patients with unilateral damage to the
and (ii) involvement of a brain region that either right amygdala.
included (the experimental group) or excluded (the In all three subjects, the aetiology was a right
control group) structures of the somatic marker temporal lobectomy in order to treat an intractable
circuitry as shown below. The experimental, seizure disorder. In all three subjects, the amygdala
neuropsychological and neuroanatomical studies was completely removed. The entorhinal cortex
were all conducted when the subjects were in the overlaying the amygdaloid nucleus was damaged.
chronic phase of recovery (i.e. 3 months post However, there was minimal damage to the sur-
lesion onset). Data collection for the various rounding anterior sector of the hippocampal
studies was contemporaneous for each subject formation.
(i.e. experimental, neuropsychological and neu- Insular/somatosensory (SI, SII) lesions. Three
roanatomical data for a given subject were subjects had unilateral lesions involving the right
collected at the same time or at a very close point insular/somatosensory (SI, SII) cortices. In all
in time). subjects, the aetiology was a right middle cerebral
artery stroke. And in all three subjects, the insular
The experimental group cortex was damaged. There was also extensive
This group included patients with lesions in either damage to the superior and inferior parietal
the VM prefrontal cortex, the right insular/ lobules, which include the somatosensory (SI,
somatosensory cortex or the right amygdala. SII) cortices.
VM lesions. Six subjects had lesions involving In two of the subjects, the damage extended to
the VM cortex bilaterally. In four subjects, the the right dorsolateral prefrontal cortex and in-
damage was caused by a stroke. In the other two cluded the right pre-central gyrus, but the cortex
subjects, the damage was caused by a frontal lobe anterior to it was spared. There was also damage
meningioma that was surgically resected and to the superior temporal gyrus. In one subject, the
removed. In all six subjects, the damage was lesion included the right insular cortex and in-
bilateral and involved the anterior and posterior ferior parietal lobule, but did not extend into the
sectors of the VM cortex. In two of the subjects, prefrontal cortex or the temporal lobe.
the damage extended far posteriorly and most
likely included the basal forebrain. In one subject, The control group
the lesion extended superiorly above the genu of This group included patients with lesions outside
the corpus callosum in both hemispheres. In three the neural circuitry thought to mediate somatic
of the subjects, the lesions extended anteriorly state activation and decision-making. In all
and involved the frontopolar region. In two of 11 subjects, the aetiology of the lesion was a
the subjects, the lesions were asymmetrical in that stroke. In four subjects, the damage included any
the damage was more extensive on the right of the following regions: the superior and/or
RT0996_C17.qxd 11/8/04 2:10 PM Page 228
middle frontal gyri in the right prefrontal cortex, of one’s underlying emotional and social intelli-
the right precentral gyrus, the right paracentral gence. A more detailed discussion of the psycho-
lobule without damage below the body of the cor- metric properties of this instrument and how it was
pus callosum or extension to the frontal pole. In developed is found in the EQ-i technical manual
two subjects, the lesion involved similar territo- (Bar-On, 1997b) and elsewhere (Plake and Impara,
ries in the left hemisphere. In three subjects, the 1999). In brief, the EQ-i comprises 133 items and
lesions involved the posterior sector of the supe- employs a five-point Likert scale with a textual
rior and/or middle temporal gyrus on the right. In response format ranging from ‘very seldom or not
one subject, the lesion involved the posterior sec- true of me’ to ‘very often true of me or true of me’.
tor of the middle temporal gyrus on the left. In one A list of the inventory’s items is included in the
subject, the damage involved the right occipital EQ-i technical manual (Bar-On, 1997b). The sub-
cortex but spared the somatosensory and insular ject’s responses render a total emotional quotient
cortices. (EQ) score and the following five composite scale
scores give 15 subscale scores in all:
Measures (i) Intrapersonal EQ (comprising self-regard,
The vagary of subject availability, being neurologi- emotional self-awareness, assertiveness,
cally ill, not easily accessible, etc. produced some independence and self-actualization);
limitations on data collection but without interfer- (ii) Interpersonal EQ (comprising empathy, social
ing systemically with the outcome. Every subject responsibility and interpersonal relationship);
completed the EQ-i and the Gambling Task (see (iii) Stress management EQ (comprising stress
below) in the present study. Almost all the subjects tolerance and impulse control);
in the experimental group completed the social (iv) Adaptability EQ (comprising reality-testing,
functioning, cognitive intelligence and executive flexibility and problem-solving); and
functioning tests. However, a small number of sub- (v) General mood EQ (comprising optimism and
jects in the control group did not complete all these happiness).
tests. Quite often, this was because the clinician
could not justify administering these specific tests A brief description of the emotional and social
for these particular subjects. This was due to the intelligence competencies measured by the
nature of each subject’s lesion, in which there was 15 subscales is given in the Appendix. The EQ-i
no suspicion that the subject would have problems has a built-in correction factor which automatically
in the domains measured by the clinical tests in- adjusts the scale scores based on scores obtained
volved. Although not everyone was available to be from its two validity indices (the positive impres-
tested with all of the measures described below, sion and negative impression scales); this is an
data collection for the critical group, the experi- important psychometric factor for self-report
mental group, was nearly complete. The missing measures in that it reduces the distorting effects
data from some subjects, who were mostly those in of response bias, thereby, increasing the accuracy
the control group, do not systematically affect the of the results obtained. Also, this correction fac-
outcome of the results because nonparametric sta- tor is of particular importance in the current ap-
tistics were applied which are designed specifically plication of the EQ-i because some of the brain-
for examining small samples (Siegal, 1956). damaged subjects’ self-awareness of their
acquired deficits is limited (i.e. anosognosia).
Raw scores are computer-tabulated and automat-
Emotional intelligence ically converted into standard scores based on a
The Bar-On EQ-i was used to assess emotional mean of 100 and standard deviations of 15. It is
intelligence (Bar-On, 1997a). The EQ-i is a important to stress that the EQ-i is acknowl-
self-report measure of emotionally and socially edged as a valid measure of emotional intelli-
intelligent behaviour, which provides an estimate gence based on independent review (Plake and
RT0996_C17.qxd 11/8/04 2:10 PM Page 229
Impara, 1999). Moreover, the EQ-i is signifi- were evaluated with a series of semi-structured
cantly correlated with other measures that tap interviews and rating scales described in detail
various aspects of this construct, for example by Tranel and colleagues (Tranel et al., 2002).
with the Mayer– Salovey–Caruso Emotional Briefly, these entailed a comprehensive assess-
Intelligence Test (MSCEIT) (0.46), Trait Meta- ment by a clinical neuropsychologist (who was
Mood Scale (TMMS) (0.58), Emotional Intelli- unaware of the objectives and design of the cur-
gence Questionnaire (EIQ) (0.63) and the rent study) of each patient’s post-morbid employ-
20-item Toronto Alexithymia Scale (TAS-20) ment status, social functioning, interpersonal rela-
(0.72) (Bar-On, 2000). This means that the tionships and social standing. For each of these
EQ-i is tapping—relatively well—what these domains, the extent of social change or impair-
other measures are tapping (i.e. various aspects ment for each patient was rated on a three-point
of emotional intelligence). scale on which one corresponded to ‘no change
All participants completed the EQ-i (n 23). or impairment’; two corresponded to ‘moderate
change or impairment’; and three corresponded to
Personal judgment in decision-making ‘severe change or impairment.’ For each patient, a
The ability to exercise personal judgment in Total Social Change score was then calculated by
decision-making was assessed by the Gambling summing the four scores from each of the do-
Task. A detailed account of this measure is given mains. Higher overall scores are indicative of
elsewhere (Bechara et al., 1994, 2000b). In brief, greater levels of change (impairment). Sixteen
subjects are required to select a total of 100 cards participants (10 experimental and six control
from four packs labelled A, B, C and D. The subjects) were assessed for post-morbid social
subject’s decision to select from one pack versus functioning.
another is largely influenced by various schedules
Cognitive intelligence, perception,
of immediate reward and future punishment.
memory and executive functioning
These schedules are pre-programmed and known
only to the examiner and entail a number of basic Subjects were assessed according to standard
game rules that must be adhered to by the subject. protocols of the Benton Neuropsychology Labo-
First, every time the subject selects a card from ratory (Tranel, 1996). This included standardized
pack A or B, the subject receives $100; and every measurement of cognitive intelligence, percep-
time the subject selects a card from pack C or D, tion, memory and executive functioning.
the subject receives $50. However, in each of the Cognitive intelligence. This was measured with
four packs, subjects encounter unpredictable pun- the Wechsler Adult Intelligence Scale (WAIS-III);
ishments (money loss). The punishment is more 21 subjects (12 experimental and nine control)
severe in the high-paying packs A and B, and less were administered the WAIS.
severe in the low-paying packs C and D. Overall, Perception. This was measured with the Benton
selections from packs A and B are disadvanta- Facial Discrimination Test and the Benton
geous because they cost more in punishments in Judgment of Line Orientation Test; 18 subjects
the long run, i.e. one loses $250 every 10 cards. (10 experimental and eight control) completed
Packs C and D are advantageous because they these tests.
result in an overall gain in the long run, i.e. one Memory. This was measured with the Rey
wins $250 every 10 cards. Auditory Verbal Learning Test (RAVLT), the
All participants completed the Gambling Task Benton Visual Retention Test-Revised (BVRT),
(n 23). and the Complex Figure Test; 20 subjects (10
experimental and 10 control) were assessed with
Social functioning these instruments.
Post-morbid employment status, social function- Executive functioning. This was measured with
ing, interpersonal relationships and social standing the Wisconsin Card Sorting Test (WCST), the
RT0996_C17.qxd 11/8/04 2:10 PM Page 230
Trail-making Test (TMT) and the Controlled Oral was reversed for the control group who began to
Word Association Test (COWA); 16 subjects make more advantageous than disadvantageous
(10 experimental and six control) completed the choices (18.83 versus 08.36, respectively;
WCST and the TMT, and 20 (10 experimental and Z 3.67, P 0.000). This increase in exercising
10 control) were tested with the COWA. better judgment in decision-making among sub-
jects in the control group appears to follow a
Personality and psychopathology typical learning curve that would be expected;
Personality and psychopathology was assessed however, it is important to note that this normal
with the Minnesota Multiphasic Personality learning process did not occur among the subjects
Inventory (MMPI) (Butcher et al., 1989). Ten in the experimental group whose personal
subjects (seven experimental and three control) judgment tends to get worse as can be seen in
completed the MMPI. Figure 17.1. In addition to indicating the exis-
tence of a neurological substrate mediating the so-
matic marker hypothesis, this suggests that it
Results takes 40 attempts of trial and error before peo-
ple, who are not damaged along the targeted so-
Differences in Cognitive Intelligence
matic marker circuitry, to catch on to the rules of
Between the Control and the game and to do what is more advantageous for
Experimental Groups them. Based on these findings and in an attempt to
There are no significant differences between the obtain the most accurate evaluation of differences
control and experimental groups with respect to between the groups that were examined, it was
cognitive intelligence, executive functioning, decided to focus only on the last 60 attempts in
perception, memory or signs of psychopathology the Gambling Task when comparing personal
(see Table 17.2). In addition to a lack of signifi- judgment in decision-making with emotional in-
cant difference between the control group and the telligence and overall social functioning in the
experimental group regarding the level of their present study.
cognitive intelligence as can be seen in Table 17.2 Post-morbid social functioning was also found to
(Z 1.17, P 0.241), it is also important to point be significantly worse for the experimental group
out that no significant correlation was found compared with the control group (see Table 17.3).
between cognitive intelligence and emotional Table 17.4 reveals that subjects in the experi-
intelligence (r 0.08, P 0.740) for the clinical mental group have significantly lower emotional
sample examined in the present study. intelligence than those in the control group (82.3
versus 101.1, respectively; Z 3.33, P 0.001).
Differences in Decision-Making, Social A review of these results suggest that the key
Functioning and Emotional Intelligence emotional intelligence competencies involved
Between the Control and appear to be the ability to be aware of oneself
Experimental Groups (self-regard), to express oneself (assertiveness), to
A comparison of advantageous () to disadvanta- manage and control emotions (stress tolerance
geous () choices made in the Gambling Task for and impulse control), to adapt flexibly to change
the first 40 cards selected did not reveal signifi- (flexibility) and to solve problems of a personal
cant differences between the experimental and nature as they arise (problem-solving), as well
control groups (10.67 versus 04.73, respec- as the ability to motivate oneself and mobilize
tively; Z 1.52, P 0.128). During the next and positive affect (self-actualization, optimism and
final 60 selections, however, significant differ- happiness).
ences began to appear between the two groups A precursory examination of the differences
with the experimental group making more disad- between the control group and all of the three sub-
vantageous than advantageous choices; this ratio groups forming the experimental group reveals
RT0996_C17.qxd 11/8/04 2:11 PM Page 231
TABLE 17.2. Neuropsychological Test Scores for the Control and Experimental Groups
Neuropsychological tests Control group (n 11) Experimental group (n 12) Z P-level (2-tailed)
Cognitive intelligence
WAIS-III:
Full IQ 97.7 105.3 1.17 0.241
Verbal IQ 99.2 107.9 1.32 0.186
Performance IQ 95.7 102.8 1.42 0.155
Perception
Benton faces 44.4 43.9 0.50 0.620
Benton lines 25.4 24.6 0.81 0.420
Memory
WAIS:
Digit span 11.0 10.8 0.16 0.869
BVRT:
Correct 7.4 7.4 0.12 0.908
Errors 3.8 4.0 0.58 0.565
RAVLT:
Trial 1 to 5 10.1 12.0 1.53 0.127
30 minute recall 8.1 9.3 0.46 0.648
Recognition 28.9 28.8 0.91 0.361
Complex figure (Rey-O):
Copy 32.2 30.9 0.55 0.585
Delay 20.7 19.5 0.22 0.827
Executive functioning
Trails-making test A 34.8 39.6 0.06 0.957
Trails-making test B 86.8 79.9 0.33 0.745
WCST:
Perseverative errors 10.2 17.7 0.44 0.662
Categories 5.8 5.1 0.71 0.476
COWA 37.0 40.4 0.53 0.596
Personality/psychopathology
MMPI:
Scale 1 (Hs) 57.3 57.6 0.34 0.732
Scale 2 (D) 53.0 63.0 1.26 0.209
Scale 3 (Hy) 64.3 57.6 0.69 0.493
Scale 4 (Pd) 58.7 61.9 1.03 0.304
Scale 5 (Mf) 52.7 55.0 0.57 0.568
Scale 6 (Pa) 50.3 57.1 0.92 0.359
Scale 7 (Pt) 49.3 61.3 1.38 0.168
Scale 8 (Sc) 52.0 63.3 1.48 0.139
Scale 9 (Ma) 54.3 53.9 0.23 0.818
Scale 0 (Si) 49.7 52.9 0.34 0.731
The Mann–Whitney U test was applied to compare the scores for significant differences between the groups.
significant differences with respect to emotional subgroups is not justified because of the very
and social intelligence (those with damage to small sample sizes involved.
the VM prefrontal lobe (101 versus 92; Z 2.22,
P 0.026), the right amygdala (101 versus 80; Discussion
Z 2.43, P 0.015) and, especially, to the right
insular/somatosensory cortices (101 versus 65; Only those subjects with injury to the somatic
Z 2.58, P 0.010)). However, a closer exami- marker circuitry (the experimental group) re-
nation of the specific differences between these vealed significantly low emotional intelligence
RT0996_C17.qxd 11/8/04 2:11 PM Page 232
0 Control
1-20 21-40 41-60 61-80 81-100 Experimental
–2 cards cards cards cards cards
–4
–6
–8
FIGURE 17.1 ■ The ratio of advantageous () to disadvantageous ()

decisions made in the Gambling Task by the control group (n 11)
and the experimental group (n 12) progressing from the first 20 to
the last 20 cards selected.
and poor judgment in decision-making as well as Thus, the differences between the experimental
disturbances in social functioning, in spite of and control groups with respect to emotional and
normal levels of cognitive intelligence (IQ) and in social intelligence (EQ) exist in spite of the fact
the absence of psychopathology. Specifically, the that the two groups were well matched on demo-
experimental group made significantly more graphic and cognitive grounds. It was also
disadvantageous decisions (on the Gambling demonstrated that there was no significant corre-
Task) than the control group and their personal lation between IQ and EQ in the clinical sample
judgment got worse rather than better as time studied in the present study. In addition to recon-
went on, i.e. they failed to make advantageous firming the neurological substrate that mediates
choices and to learn from experience. In addition somatic state activation and decision-making,
to obtaining significantly lower scores on the these findings support our hypothesis that emo-
Gambling Task, the experimental group demon- tional and social intelligence is fundamentally
strated many disturbances in social functioning, different from cognitive intelligence. Moreover,
and they received significantly lower scores on the the neural systems that support emotional and so-
EQ-i indicating impaired emotional and social in- cial intelligence appear to overlap with the neural
telligence. Yet, there were no differences between systems subserving somatic state activation and
the experimental and control groups with respect personal judgment in decision-making, which
to cognitive capacity (IQ), executive functioning, are apparently separate from the neural systems
perception, memory or signs of psychopathology. supporting cognitive intelligence.
Similarly, there were no differences between the The key emotional intelligence competencies
two groups with respect to demographic factors. affected by damage to the neural circuitry of
TABLE 17.3. Differences in Post-Morbid Social Functioning Between the Control and Experimental Groups
Post-morbid changes in: Control group (n 6) Experimental group (n 10) Z P-level (2-tailed)
Employment 1.17 2.70 3.11 0.002

Social functioning 1.00 2.30 2.84 0.005
Interpersonal relationships 1.00 2.40 2.87 0.004
Social status 1.00 1.90 3.40 0.001
Total social change 4.17 9.30 3.00 0.003
The Mann–Whitney U test was applied to compare scores for significant differences.
RT0996_C17.qxd 11/8/04 2:11 PM Page 233
TABLE 17.4. Differences in Emotional Intelligence Between the Control and Experimental Groups
EQ-i scales (emotional intelligence) Control group (n 11) Experimental group (n 12) Z P-level (2-tailed)
Total EQ 101.1 82.3 3.33 0.001

Intrapersonal EQ 100.0 81.8 3.23 0.001
Self-regard 99.1 83.8 2.40 0.016
Emotional self-awareness 100.9 90.1 1.48 0.139
Assertiveness 103.6 82.6 3.21 0.001
Independence 97.7 87.3 1.58 0.115
Self-actualization 99.4 86.8 2.25 0.024
Interpersonal EQ 99.6 91.6 1.36 0.175
Empathy 98.6 89.8 1.24 0.216
Social responsibility 101.5 95.3 1.14 0.254
Interpersonal relationship 98.8 92.8 0.83 0.406
Stress management EQ 104.8 89.1 2.62 0.009
Stress tolerance 100.1 83.2 2.56 0.011
Impulse control 108.3 96.9 2.13 0.033
Adaptability EQ 100.0 86.3 2.28 0.023
Reality testing 99.8 91.0 1.08 0.280
Flexibility 100.3 86.8 2.38 0.017
Problem-solving 100.6 88.3 2.16 0.031
General mood EQ 99.9 83.3 3.27 0.001
Optimism 99.0 83.5 3.02 0.003
Happiness 100.9 85.8 2.71 0.007
Emotional intelligence was assessed by EQ-i and the Mann–Whitney U test was applied to compare scores for significant differences.
somatic markers appear to be the ability to be measures fail to detect acquired impairments in
aware of oneself and one’s emotions, to express tested individuals. This situation often occurs in
oneself and one’s feelings, to manage and control the case of VM patients as was previously noted,
emotions, to adapt flexibly to change and solve in which case collateral information is usually
problems of a personal nature, as well as the abil- needed in order to document changes in the per-
ity to motivate oneself and mobilize positive af- sonality and social behaviour of impaired patients
fect. Indeed, self-regard (accurate self-awareness) (Barrash et al., 2000). However, in the case of the
and, especially assertiveness (self-expression), EQ-i, the instrument proved to be successful in
stand out as those competencies that are affected detecting abnormal levels of emotional and social
most by brain injury to the neural circuitry being intelligence in the target subjects. This suggests
examined in the present study. This is understand- that the instrument has adequate construct validity
able in that these are two of the most important when used with individuals whom otherwise may
emotional intelligence competencies. be unaware of the limitations of their own emo-
It is important to note that the specific type of tional and social abilities. Indeed, low scores—
brain injury sustained by subjects belonging to the particularly on the two scales of self-regard
experimental group usually produces a certain (accurate self-awareness) and assertiveness (self-
degree of anosognosia (i.e. lack of self-awareness expression)—most likely mean that these sub-
of acquired impairments). Given the reliance of jects possess low self-awareness consistent with
the EQ-i on self-report, the question arises as to their symptomatology; their scores on these two
whether this instrument can provide a valid meas- scales would have been even lower without the
ure of emotional and social intelligence when EQ-i’s correction factor that automatically adjusts
used with this particular group of clinical subjects. scale scores to compensate for various types of in-
The issue of self-awareness of acquired im- accuracies in responding that occur for one reason
pairments is critical in situations where self-report or another.
RT0996_C17.qxd 11/8/04 2:11 PM Page 234
The findings of the present study indicate that been combined into one meta construct by
poor personal judgment in decision-making is re- some theorists (Gardner, 1983; Goleman, 1995;
lated to deficiencies in emotional intelligence, in Bar-On, 2001).
spite of average or above average levels of cogni- One of the implications of the significant sta-
tive intelligence. Relative to patients with brain tistical and neurological connections between
damage in areas outside of the neural circuitry the somatic marker hypothesis and the concept
studied in the present study (the control group), of emotional intelligence, based on the present
those who exercise poor judgment in decision- findings, is that emotional and social intelligence
making linked to impaired somatic state activa- is a valid construct which is neurally distinct
tion (the experimental group) are less emotionally from cognitive intelligence. The major differ-
intelligent based on lower EQ-i scores. Further- ences between these two important components
more, poor decision-making appears to be related of intelligence may be that cognitive intelligence
to an inadequate knowledge of who one is (accu- is more dependent on cortical structures that
rate self-awareness), what one wants and how to support logical reasoning, whereas emotional
convey this effectively and constructively (self- and social intelligence is more dependent on
expression). Equally important is that subjects limbic and related neural systems that support
who often fail to make the right decision are also the processing of emotions and feelings. In this
less effective in controlling their emotions, in study, we have examined clinical subjects who
maintaining a positive and optimistic attitude, and possess average or above average cognitive in-
in generating and selecting potentially effective telligence and significantly below average emo-
solutions. tional intelligence. However, we do not know the
Emotional and social intelligence provides a impact of below average cognitive intelligence
valuable approach to understanding why some (IQ) on emotional and social intelligence. The
people behave more intelligently than others, neural systems supporting emotional and cogni-
which is often revealed in making the right versus tive intelligence may be completely independ-
wrong decisions in one’s personal life and interac- ent, i.e. impaired cognitive intelligence does not
tions with others. There are a number of other compromise emotional intelligence. Alterna-
closely related concepts that are based on the way tively, the dissociation may only be partial
emotions are perceived, understood and used to (i.e. impaired emotional intelligence does not
guide effective human behaviour like ‘emotional compromise cognitive intelligence) as we have
awareness’ (Lane and Schwartz, 1987), ‘empathy’ shown in this study. However, impaired cogni-
(Brothers, 1990; Preston and de Waal, 2002), tive intelligence may compromise emotional in-
‘psychological mindedness’ (McCallum, 1989), telligence. Such asymmetrical relationship be-
‘theory of mind’ (Gopnik, 1993; Gallup, 1998; tween neural systems has been demonstrated
Blair, 1999; Frith and Frith, 1999), ‘practical before in relation to two functions of the pre-
intelligence’ (Sternberg, 1985), ‘successful in- frontal cortex: decision-making and working
telligence’ (Sternberg, 1997), etc. Most of these memory. Bachara and colleagues found that VM
concepts can be considered different components of lesions that impaired decision-making did not
emotional and social intelligence rather than sepa- compromise working memory, but dorsolateral
rate constructs. For example, psychological mind- lesions that impaired working memory did com-
edness is the salutogenic (i.e. non-pathological) promise decision-making (Bechara et al., 1998).
end of alexithymia and emotional awareness is a Thus, it would be intriguing to study patients
major component of this continuum. While this with Down’s syndrome or William’s syndrome,
continuum represents the essence of the intraper- for example, who have significantly limited cog-
sonal aspect of emotional intelligence, empathy nitive intelligence but are known to be relatively
and theory of mind represent the essence of social effective in social interactions. Another ap-
intelligence; both types of intelligence have proach is to examine the level of emotional and
RT0996_C17.qxd 11/8/04 2:11 PM Page 235
social intelligence in neurological patients with itself in low levels of emotional intelligence,
impaired cognitive intelligence. In this study, we which comprises a wide array of emotional and
used the WAIS-III to measure cognitive intelli- social competencies, which can have an ill effect
gence, whereas Duncan (2001), for instance, has on one’s ability to effectively cope with daily
used more sensitive tasks to measure various as- demands. Such impairment may include a de-
pects of cognitive intelligence, attributing them crease in one’s ability to: (i) be aware of and ex-
to the lateral orbitofrontal/dorsolateral prefrontal press oneself; (ii) function interpersonally; (iii)
cortices. Therefore, using these more sensitive manage and control emotions; (iv) generate posi-
measures of cognitive intelligence in future tive affect required in achieving personal goals;
studies will yield additional important information and (v) cope flexibly with the immediate situa-
about the relationships between the neural sys- tion, make decisions and solve problems of a per-
tems that support cognitive versus emotional sonal and interpersonal nature.
intelligence. Finally, the findings that emotional intelligence
One of the most important implications of the is significantly related to the ability to exercise
current findings is that the complex cognitive personal judgment in decision-making help ex-
processes that subserve social competence, which plain why this concept is highly connected with
appears to constitute a distinct form of intelli- human performance (Bar-On et al., 2003). To per-
gence dedicated to behaviour suited more for form well and be successful in one’s professional
survival and adaptation (Goleman, 1995; Bar-On, and personal life apparently requires the ability
1997b; Stein and Book, 2000), does not draw to make emotionally and socially intelligent
upon neural processes specialized for social in- decisions more than just having a high IQ.
formation. Rather, these processes depend on
known brain mechanisms related to emotion and
decision-making. Indeed, we have argued that the Acknowledgment
process of judgment and decision-making de-
This study was supported by NIH Program Project
pends on systems for: (i) memory, which is sup-
Grant PO1 NS19632.
ported by high-order association cortices as well
as the dorsolateral sector of the prefrontal cortex;
(ii) emotion, which is mediated by subcortical Abbreviations
limbic structures that trigger the emotional re-
sponse; and (iii) feelings which are supported by BVRT Benton Visual Retention Test-Revised;
limbic as well as closely associated regions such COWA Controlled Oral Word Association Test;
as the insula, surrounding parietal cortices and the EQ emotional quotient; EQ-i Emotional
cingulate cortex (Damasio, 1994, 1995, 1999; Quotient Inventory; MMPI Minnesota Multi-
Bechara et al., 2000a). Therefore, damage to the phasic Personality Inventory; RAVLT Rey
systems that impact emotion, feeling and/or mem- Auditory Verbal Learning Test; TMT Trail-
ory usually compromise the ability to make making Test; VM ventromedial; WAIS
advantageous decisions (Bechara et al., 2000a). Wechsler Adult Intelligence Scale; WCST Wis-
The VM prefrontal cortex links these systems to- consin Card Sorting Test.
gether; when damaged, there are a number of
manifestations that occur including alterations of
REFERENCES
emotional experience and social functioning
(Bechara et al., 2000a). The findings of the pres- Bar-On R. The Bar-On Emotional Quotient Inventory (EQ-i):
a test of emotional intelligence. Toronto (Canada): Multi-
ent study suggest that emotional and social intel-
Health Systems; 1997a.
ligence has neural roots, which may be associated Bar-On R. The Bar-On Emotional Quotient Inventory (EQ-i):
with these known basic mechanisms of the brain. technical manual. Toronto (Canada): Multi-Health Systems;
Impairment of these mechanisms may manifest 1997b.
RT0996_C17.qxd 11/8/04 2:11 PM Page 236
Bar-On R. Emotional and social intelligence: insights from the Damasio H, Frank R. Three-dimensional in vivo mapping
Emotional Quotient Inventory. In: Bar-On R, Parker JDA, of brain lesions in humans. Arch Neurol 1992; 49:
editors. The handbook of emotional intelligence. San 137–43.
Francisco: Jossey-Bass; 2000. p. 363–88. Darwin C. The expression of the emotions in man and animals.
Bar-On R. Emotional intelligence and self-actualization. London: Murray; 1872.
In: Ciarrochi J, Forgas JP, Mayer JD, editors. Emotional Duncan J. An adaptive coding model of neural function in pre-
intelligence in everyday life: A scientific inquiry. Philadelphia: frontal cortex. Nat Rev Neurosci 2001; 2: 820–9.
Psychology Press; 2001. p. 82–97. Frith CD, Frith U. Interacting minds: a biological basis.
Bar-On R, Fund S, Handley R. The impact of emotional Science 1999; 286: 1692–5.
intelligence on performance. In: Druskat V, Sala F, Mount Gallup GGJ. Self-awareness and the evolution of social intel-
G, editors. Emotional intelligence and performance at ligence. Behav Process 1998; 42: 239–47.
work. San Francisco: Jossey-Bass. In press 2003. Gardner H. Frames of mind. New York: Basic Books; 1983.
Barrash J, Tranel D, Anderson SW. Acquired personality Goleman D. Emotional intelligence. New York: Bantam
disturbances associated with bilateral damage to the ventro- Books; 1995.
medial prefrontal region. Dev Neuropsychol 2000; 18: Gopnik A. How we know our minds: the illusion of first-per-
355–81. son knowledge of intentionality. Behav Brain Sci 1993; 16:
Bechara A, Damasio AR, Damasio H, Anderson SW. Insensi- 1–14, 29–113.
tivity to future consequences following damage to human Lane RD. Levels of emotional awareness: Neurological, psy-
prefrontal cortex. Cognition 1994; 50: 7–15. chological, and social perspectives. In: Bar-On R, Parker
Bechara A, Damasio H, Tranel D, Anderson SW. Dissociation JDA, editors. The handbook of emotional intelligence. San
of working memory from decision making within the Francisco: Jossey-Bass; 2000; p. 171–91.
human prefrontal cortex. J Neurosci 1998; 18: 428–37. Lane RD, Schwartz GE. Levels of emotional awareness: a
Bechara A, Damasio H, Damasio AR, Lee GP. Different cognitive developmental theory and its application to psy-
contributions of the human amygdala and ventromedial chopathology. Am J Psychiatry 1987; 144: 133–43.
prefrontal cortex to decision-making. J Neurosci 1999; 19: Maddock RJ. The retrosplenial cortex and emotion: new in-
5473–81. sights from functional neuroimaging of the human brain.
Bechara A, Tranel D, Damasio AR. Poor judgment in spite of Trends Neurosci 1999; 22: 310–6.
high intellect: neurological evidence for emotional intelli- Manes F, Sahakian B, Clark L, Rogers R, Antoun N, Aitken M,
gence. In: Bar-On R, Parker JDA, editors. The handbook of et al. Decision-making processes following damage to the
emotional intelligence. San Francisco: Jossey-Bass; 2000a. prefrontal cortex. Brain 2002; 125: 624–39.
p. 192–214. Mayer JD, Salovey P. What is emotional intelligence: In:
Bechara A, Tranel D, Damasio H. Characterization of the Salovey P, Sluyter DJ, editors. Emotional development and
decision-making deficit of patients with ventromedial pre- emotional intelligence: educational implications. New
frontal cortex lesions. Brain 2000b; 123: 2189–202. York: Basic Books; 1997. p. 3–34.
Blair RJR. Psychophysiological responsiveness to the distress McCallum M. A controlled study of effectiveness and patient
of others in children with autism. Personality Individ Diff suitability for short-term group psychotherapy (doctoral
1999; 26: 477–85. dissertation). Montreal (Canada): McGill University;
Brothers L. The neural basis of primate social communication. 1989.
Motivation Emotion 1990; 14: 81–91. Plake BS, Impara JC, editors. The BarOn Emotional Quotient
Butcher JN, Dahlstrom WG, Graham JR, Tellegen A, Kaemmer Inventory (EQ-i). Supplement to the 13th mental measure-
B. Minnesota Multiphasic Personality Inventory-2 (MMPI- ment yearbook. Lincoln (NE): Buros Institute for Mental
2) manual for administration and scoring. Minneapolis Measurement; 1999.
(MN): University of Minnesota Press; 1989. Preston S, de Waal F. Empathy: Its ultimate and proximate
Damasio AR. Descartes’ error: emotion, reason, and the bases. Behav Brain Sci 2002; 25: 1–71.
human brain. New York: G. P. Putnam; 1994. Siegal S. Nonparametric statistics for the behavioral sciences.
Damasio AR. Toward a neurobiology of emotion and feeling: New York: McGraw-Hill; 1956.
operational concepts and hypotheses. Neuroscientist 1995; Stein SJ, Book HE. The EQ edge: emotional intelligence and
1: 19–25. your success. Toronto (Canada): Stoddart Publishing; 2000.
Damasio AR. The feeling of what happens: body and emotion in Sternberg RJ. Beyond IQ: a triarchic theory of human intelli-
the making of consciousness. New York: Harcourt Brace; 1999. gence. New York: Cambridge University Press; 1985.
Damasio AR. Grabowski TG, Bechara A, Damasio H, Ponto Sternberg RJ. Successful intelligence. New York: Plume; 1997.
LLB, Parvizi J, et al. Subcortical and cortical brain activity Taylor GJ, Bagby RM, Parker JDA. Disorders of affect regula-
during the feeling of self-generated emotions. Nat Neurosci tion: alexithymia in medical and psychiatric illness.
2000; 3: 1049–56. Cambridge: Cambridge University Press; 1997.
Damasio H, Damasio AR. Lesion analysis in neuropsychology. Thorndike EL. Intelligence and its uses. Harper’s Magazine
New York: Oxford University Press; 1989. 1920; 140: 227–35.
RT0996_C17.qxd 11/8/04 2:11 PM Page 237
Tranel D. The Iowa-Benton school of neuropsychological as- Tranel D, Bechara A, Denburg N L. Asymmetric functional
sessment. In: Grant I, Adams KM, editors. Neuropsycho- roles of right and left ventromedial prefrontal cortices in so-
logical assessment of neuropsychiatric disorders, Vol. 1. cial conduct, decision-making, and emotional processing.
2nd ed. New York: Oxford University Press; 1996. Cortex 2002; 38: 589–612.
p. 81–101. Zirkel S. Social intelligence: the development and mainte-
Tranel D, Bechara A, Damasio H, Damasio AR. Decision- nance of purposive behavior. In: Bar-On R, Parker JDA,
making in patients with unilateral ventromedial prefrontal editors. The handbook of emotional intelligence. San
cortex lesions (abstract). Soc Neurosci Abstr 2000; 26: 549. Francisco: Jossey-Bass; 2000. p. 3–27.
Appendix I The EQ-i Scales and What They Assess

EQ-i scales The EI competency assessed by each scale
Intrapersonal
Self-regard To accurately perceive, understand and accept oneself.
Emotional self-awareness To be aware of and understand one’s emotions.
Assertiveness To effectively and constructively express one’s emotions and oneself.
Independence To be self-reliant and free of emotional dependency on others.
Self-actualization To strive to achieve personal goals and actualize one’s potential.
Interpersonal
Empathy To be aware of and understand how others feel.
Social responsibility To identify with one’s social group and cooperate with others.
Interpersonal relationship To establish mutually satisfying relationships and relate well with others.
Stress management
Stress tolerance To effectively and constructively manage emotions.
Impulse control To effectively and constructively control emotions.
Adaptability
Reality-testing To objectively validate one’s feelings and thinking with external reality.
Flexibility To adapt and adjust one’s feelings and thinking to new situations.
Problem-solving To effectively solve problems of a personal and interpersonal nature.
General mood
Optimism To be positive and look at the brighter side of life.
Happiness To feel content with oneself, others and life in general.
RT0996_C17.qxd 11/8/04 2:11 PM Page 238
RT0996_P10.qxd 11/8/04 2:15 PM Page 239
PA R T 1 0
Biological Does Not Mean Predetermined:

Reciprocal Influences of Social and
Biological Processes
•
The readings in this book have illustrated that social and biological
approaches are complementary rather than antagonistic. Together, these
perspectives are helping to illuminate questions ranging from the social
sciences to the neurosciences by examining how organismic processes
are shaped, modulated, and modified by social factors and vice versa.
Rather than viewing social psychology and biological psychology as
generating inevitably oppositional forces, the readings in this book illustrate
some of the synergisms that result from a social neuroscientific approach.
The readings in the final section demonstrate that biological does not
mean fixed or predetermined regardless of environmental influences, and
indeed that sociocultural and biological processes even have reciprocal
influences. In some ways this is so patently obvious as to be trivial. The
culture in which we live influences what we deem to be valuable and
beautiful, and these learned evaluations in turn modulate activity, for
instance, in the reward circuitry in the brain. Genetic factors (genotype)
may place broad constraints on an individual’s development or functioning,
but genetic expression (phenotype) typically can be powerfully modulated
by environmental influences. For instance numerous genes, each with
small effects, contribute to the expression and outcome of many of the
239
RT0996_P10.qxd 11/8/04 2:15 PM Page 240
most costly human diseases we endure (e.g., animals nor from dominance hierarchies alone but,
cardiovascular disease, hypertension, inflammatory rather, they are apparent only after a dominance
arthritis). Genetic influences are documented but hierarchy emerged in socially housed groups. A
they do not tell the whole story. Environmental factors strictly physiological (or social) analysis, regardless
also determine much if not most of the expression of the sophistication of the measurement
and outcome of these diseases. technology, may not have revealed the orderly
The first reading in this section by Morgan and relationship that exists because the receptors
colleagues (2002) illustrates how environmental expressed by the brain were modulated by the social
factors can modulate genetic expression in the brain. context of the animal.
Dopamine is a powerful neurotransmitter, which acts There is a large body of evidence showing that
on dopamine receptors in the brain. Morgan and negative affect is associated with activation in a
colleagues studied the dopamine D2 family of neural circuit that includes the amygdala. Inhibitory
dopamine binding sites, which is related to cocaine’s connections between the prefrontal cortex and the
reinforcing and addictive effects. Morgan et al. amygdala have also been identified and are thought
(2002) examined the D2 receptors in dominant and to be involved in self-regulation and self-control.
subordinate male cynomolgus monkeys when they Evidence for the involvement of this circuit during
were individually housed and, later, when they were the exercise of voluntary control over responses to
socially housed. No differences were found in D2 aversive events is provided in the final reading by
family receptor binding potential in the monkeys when Ochsner et al. (2002). Individuals were exposed to
they were individually housed. After only 3 months neutral and aversive photographs under two
of being socially housed, however, a dominance instructional sets. In one task, participants were
hierarchy emerged, and those at the top of the shown neutral or negative pictures and were
dominance hierarchy showed a significant increase instructed to attend to the picture and to be aware of
in D2 family receptor binding potential, whereas the the feelings it aroused but to not attempt to alter
submissive animals showed no change in these feelings. In another task, participants were
dopaminergic characteristics. More interestingly, shown negative photographs and were instructed to
behavioral testing further showed that dominant reinterpret the pictures so that they no longer
animals self-administered cocaine at levels elicited a negative response. Participants performed
comparable to saline—that is, the dominant animals both tasks while functional magnetic resonance
acted as if cocaine had minimal reinforcing value. images (fMRI) were constructed. Behavioral
Subordinate monkeys, in contrast, reliably self- responses to the pictures confirmed that participants
administered cocaine at higher doses than saline. were able to voluntarily reduce the emotional impact
These results suggest that the vulnerability to the of the aversive pictures when instructed to do so.
abuse-related effects of cocaine are not predictable Moreover, the fMRI data further revealed increased
from dopaminergic characteristics of individual activity in the lateral and medial prefrontal regions of
RT0996_P10.qxd 11/8/04 2:15 PM Page 241
Part 10: Biological Does Not Mean Predetermined ■ 241
the brain and decreased activity in the amygdala and satisfactory explanation for complex behaviors, or
medial orbitofrontal cortex when the participants that molecular forms of representation provide the
deliberately thought about the aversive pictures in a only or best level of analysis for understanding
way that reduced their aversion to the pictures rather human behavior. Molar constructs such as those
than simply viewing these pictures. Together these developed by social psychologists provide a means
readings indicate that just because palpable of understanding highly complex activity without
biological processes underlie social behavior, it does needing to specify each individual action of the
not mean that social behavior is determined entirely simplest components, thereby providing an efficient
by one’s biological inheritance. An implication of this means of describing the behavior of a complex
line of reasoning is that a personally or socially system. Social and biological approaches to human
damaging behavior may have biological roots, but it behavior have traditionally been contrasted, as if the
does not necessarily follow that the person has no two were antagonistic or mutually exclusive. The
personal responsibility for having enacted this readings in this book demonstrate the fallacy of this
behavior. reasoning and suggest that the mechanisms underlying
In sum, all human behavior, at some level, is mind and behavior may not be fully explicable by a
biological, but this is not to say that biological biological or a social approach alone but, rather, that
representation yields a simple, singular, or a multilevel integrative analysis may be required.
RT0996_P10.qxd 11/8/04 2:15 PM Page 242
RT0996_C18.qxd 11/8/04 2:11 PM Page 243
R E A D I N G 1 8
Social Dominance in Monkeys: Dopamine D2

Receptors and Cocaine Self-Administration
•
Drake Morgan1, Kathleen A. Grant1, H. Donald Gage2,
Robert H. Mach1,2, Jay R. Kaplan3, Osric Prioleau1, Susan H. Nader1,
Nancy Buchheimer2, Richard L. Ehrenkaufer2 and Michael A. Nader1,2
Disruption of the dopaminergic system has been implicated in the etiology of many pathological
conditions, including drug addiction. Here we used positron emission tomography (PET) imaging
to study brain dopaminergic function in individually housed and in socially housed cynomolgus
macaques (n 20). Whereas the monkeys did not differ during individual housing, social housing
increased the amount or availability of dopamine D2 receptors in dominant monkeys and produced
no change in subordinate monkeys. These neurobiological changes had an important behavioral
influence as demonstrated by the finding that cocaine functioned as a reinforcer in subordinate but
not dominant monkeys. These data demonstrate that alterations in an organism’s environment can
produce profound biological changes that have important behavioral associations, including
vulnerability to cocaine addiction.
rug abuse is a significant public health problem. Administration, 2000). Although some of these new
D With regard to cocaine use in the United States,
the annual number of new users rose between 1994
cocaine users will eventually become addicted,
not all will. Unfortunately, little is known about
and 1998 from 500,000 to over 900,000 (“1999 the biological substrates and environmental influ-
National Household Survey on Drug Abuse,” ences underlying this differential vulnerability1,
Substance Abuse and Mental Health Services and ethical considerations make it impossible to
1
Department of Physiology and Pharmacology, 2Department Medicine, Medical Center Boulevard, Winston-Salem, North
of Radiology, 3Departments of Pathology (Comparative Med- Carolina 27157, USA.
icine) and Anthropology, Wake Forest University School of
243
RT0996_C18.qxd 11/8/04 2:11 PM Page 244
study in humans. One purpose of the present study measures (such as locomotor activity) were ob-
was to develop an animal model of vulnerability tained14, as well as initial PET scans. Monkeys
to drug abuse that incorporates the assessment of were subsequently assigned to social groups of
social behavior, the noninvasive brain imaging four. Consistent with previous research, the dom-
technique PET, and cocaine self-administration in inant monkeys engaged in significantly more
socially housed monkeys. Such a model could aggressive behaviors (1.9 versus 0 episodes/h)
identify variables that would aid in the characteri- and were submitted to more often than were
zation and treatment of individuals vulnerable to the subordinate monkeys (3.0 compared to
drug abuse. 0.1 episodes/h). Conversely, subordinate monkeys
Certain environmental conditions, such as avail- received aggression and submitted more often
ability of alternative reinforcers2 or living in an (3.6 and 3.5 episodes/h) than did dominant
‘enriched’ environment3, alter the reinforcing monkeys (0.1 and 0 episodes/h). Percentage of
effects of drugs, particularly cocaine4. Many of these time engaged in various affiliative behaviors
stimuli also produce changes in dopaminergic also depended on social rank. For example,
function5,6, a neurotransmitter system whose dys- dominant monkeys were groomed more often
regulation is linked to cocaine’s abuse potential7. (12.1% versus 4.9% of the time), whereas the
In particular, the dopamine D2 family of receptors subordinate monkeys spent more time alone
is related to cocaine’s effects8. Thus, there is a (27.8% versus 14.8% of the time; for details,
clear potential for interaction between particular see ref. 14).
environmental events, the actions of dopamine,
and the reinforcing effects of cocaine.
A profound environmental influence on dopamin- Social Rank and Dopaminergic
ergic function is the particular housing conditions Functioning
of animals and the dominance rank among socially Monkeys were first scanned using PET imaging
housed animals6,9–12. It is unclear whether these while individually housed and again after they
observed differences reflect a neurobiological were placed in social groups and a stable social
predisposition that determines dominance rank hierarchy was established. During all PET scans,
(a trait marker), or a neurobiological alteration there was a high level of uptake of [18F]fluoro-
induced by the attainment of dominance rank (a state clebopride (FCP) and a linear rate of washout
marker). Our hypothesis that D2 receptor binding from the region of interest (ROI), the basal gan-
potential is related to environmental influences and glia (Figure 18.1). In the reference region, the
associated with vulnerability to the abuse-related cerebellum, there was a low level of [18F]FCP
effects of cocaine13 was confirmed by the findings uptake and high rate of washout. After three
that there were differential changes in binding months of social housing, dominant monkeys
potential across social ranks, and that the domi- (rank 1) had significantly higher D2 receptor dis-
nant monkeys were less vulnerable to the rein- tribution volume ratios (DVRs) compared to
forcing effects of cocaine compared to subordinate subordinate (rank 4) monkeys (Table 18.1). These
animals. data replicate previous findings12 showing a
greater than 20% higher DVR in dominant mon-
keys compared to subordinate monkeys. Impor-
Results tantly, these findings extended the previous data
from female to male monkeys, and demonstrate
Behavioral Profiles of Socially that this difference is apparent after only three
Housed Monkeys months of social housing compared to three years
For the first 1.5 years of the study, the monkeys were in the previous study. In addition, the longitudi-
individually housed and various hormonal meas- nal design of the present experiment allowed
ures (such as cortisol, testosterone) and behavioral the comparison of the same ROIs obtained while
RT0996_C18.qxd 11/8/04 2:11 PM Page 245
Social Dominance in Monkeys ■ 245
0.10
0.09
Percent injected dose/cc

0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01
0 20 40 60 80 100 120 140 160 180
Time (min)
18
FIGURE 18.1 ■ [ F]FCP has high uptake and linear rate of washout in
the basal ganglia (BG; black symbols) relative to the cerebellum (Cb;
white symbols). Each point is the mean ( 1 s.d.) value determined
from 20 individually housed monkeys.
individually and socially housed, and revealed a This measure of dopaminergic function sug-
significant change in [18F]FCP binding only in gests that, rather than a predisposing trait, the
the eventual dominant monkeys (Table 18.1). That changes were a consequence of becoming the
is, the significant interaction between housing dominant monkey in a social group (Figure 18.3).
condition and social rank (F2,39 8.88, p 0.002) Previously examined neuroendocrine markers, such
demonstrates that the DVR of dominant mon- as cortisol and testosterone levels, were similarly not
keys increased from a mean of 2.49 to a mean of predictive of eventual social rank14. In contrast, high
3.04, an increase of approximately 22%, whereas levels of locomotor activity in individually housed
the mean DVR of subordinate monkeys was not monkeys were associated with eventual social
altered (means of 2.40 and 2.49) as a function subordination14. Age or weight of the monkeys did
of housing condition (Figure 18.2). The latter not correlate with D2 binding (all p values 0.05)
findings are similar to previous results showing during either housing condition, indicating no
low between-study variability with this PET relationship between these characteristics and
ligand15. dopaminergic function.
TABLE 18.1. Dopaminergic Characteristics of Monkeys

[18F]FCP distribution volume ratios
Social ranka Individually housed Socially housed Percent change
1 2.49 0.08 3.04 0.23b,c 22.0 8.8

2 2.58 0.13 2.99 0.13 16.7 6.0
3 2.58 0.13 2.88 0.30 13.4 15.3
4 2.40 0.06 2.49 0.10 3.9 5.3
Mean s.e.m. [18F]FCP DVR as determined with PET imaging in male cynomolgus monkeys as a function of social rank while indi-
vidually and socially housed. aFor individually housed scans, these numbers represent eventual social rank. bSignificantly higher than
individually housed ‘dominants.’ cSignificantly higher than socially housed subordinates.
RT0996_C18.qxd 11/8/04 2:11 PM Page 246
Dominant Subordinate Social Rank and Cocaine

Self-Administration
Rank 1 Rank 4
4.0 Subordinate monkeys reliably self-administered
cocaine across several doses, with the entire
3.5
dose–effect curve characterized as an inverted
U-shaped function (Figure 18.4 left, black sym-
Dopamine D2 receptor DVR
3.0
bols). There was a significant interaction between
2.5
dose and social rank (F4,134 3.54, p 0.01), and
2.0 further analyses showed that subordinate mon-
IND SOC IND SOC
keys self-administered more 0.01 and 0.03 mg/kg
cocaine injections than saline. In contrast, in the
Rank 2 Rank 3
4.0 dominant monkeys, cocaine failed to maintain
responding higher than saline, and therefore did
3.5 not function as a reinforcer (Figure 18.4 left,
3.0
white symbols). During these sessions, subordinate
monkeys had significantly higher intakes compared
2.5 to dominant monkeys (Figure 18.4 right; signifi-
cant interaction between dose and social rank,
2.0
F3,107 10.59, p 0.0001). A dose-dependent
IND SOC IND SOC
increase in cocaine intake occurred for the domi-
Housing Condition nant monkeys, suggesting that these animals were
18
FIGURE 18.2 ■ [ F]FCP binding potential changes as a
not avoiding cocaine altogether, but their total
function of social rank. Panels show the mean and indi- intakes were significantly below those of subordi-
vidual [18F]FCP DVR values for monkeys with different nate monkeys. Correlation analyses indicated that
social ranks, while they were individually (IND) and social rank was inversely related to the number
socially (SOC) housed. of reinforcers obtained per session (r2 0.62,
FIGURE 18.3 ■ (A color version of this figure follows page 146.) [18F]FCP
binding potential increases in dominant monkeys. Normalized, co-registered
PET images (percent injected dose per ml) of [18F]FCP binding in the basal
ganglia of a dominant and a subordinate monkey, while individually housed
and socially housed.
RT0996_C18.qxd 11/8/04 2:11 PM Page 247
a) REINFORCERS b) INTAKE
Whereas the present study represents the first
(per session)
50 2.0
(mg/kg/session) examination of intravenous drug self-administration
in socially housed monkeys, there have been sev-
40 *
1.5 eral studies examining other behavioral effects of
30
* psychomotor stimulants in similarly housed mon-
*
20
1.0 * keys16–19. These experiments have demonstrated
0.5
differential effects of cocaine and d-amphetamine
10 as a function of social rank19. Several rodent stud-
0 0.0 ies have examined the influence of housing condi-
S.003 .01 .03 .1
.003 .01 .03 .1 tions (individual versus social) on cocaine and
amphetamine self-administration, although the
COCAINE (mg/kg/inj)
results have been contradictory3,4,20–22. Differences
FIGURE 18.4 ■ Reinforcing effects of cocaine are greater in intravenous self-administration of cocaine across
in subordinate monkeys compared to dominant animals.
social ranks of rats or monkeys had not been pre-
Left, mean number of intravenous injections (either saline
or various doses of cocaine) per session for 5 dominant viously explored. The present study provides
(rank 1 and 2, white symbols) and 4 subordinate (rank 3 experimental evidence in monkeys that individual
and 4, black symbols) monkeys. Right, mean intake per differences in susceptibility to cocaine abuse
session for dominant (white symbols) and subordinate within a population may be mediated by social
(black symbols) monkeys. Each dose was available for at
dominance rank.
least 7 sessions and until responding was stable. Data
represent the mean of the last 3 days of availability for In the present study, the higher levels of
each animal. Asterisk indicates a statistically significant [18F]FCP binding in dominant monkeys could have
difference (p 0.05) from dominant monkeys at that resulted from increased levels of D2 receptors
particular dose, and from the appropriate saline point. and/or decreases in the basal levels of synaptic
dopamine. Previous rodent studies have examined
the neurochemical changes within the dopamine
system as a consequence of housing conditions6,9–11.
p 0.01) and to cocaine intake (r2 0.55,
For example, rats reared in isolation or in ‘environ-
p 0.02). These data demonstrate a resistance to
mentally impoverished’ conditions have increased
the reinforcing effects of cocaine in the dominant
synaptic levels of dopamine and/or increased
monkeys, and, conversely, show an increased
efflux of dopamine in response to particular envi-
vulnerability in the subordinate monkeys.
ronmental events (for example, after exposure to
stressful stimuli) when compared to socially
Discussion housed rodents or those living in ‘enriched’ envi-
ronments. Concurrent decreases in D2 responsiv-
The major findings from these studies were that ity and D2 receptor levels have been observed in
environmental conditions produced a relatively individually housed rats. Taken together with the
quick alteration in the dopaminergic system of present findings, these results suggest that indi-
socially housed dominant but not subordinate vidually housed monkeys and socially subordinate
monkeys and that this difference was associated animals have relatively high levels of synaptic
with a differential vulnerability to the reinforcing dopamine (that is, dopaminergic hyperactivity). We
effects of cocaine. These findings emerged from a would suggest that after social housing, the domi-
design using a combination of several experimen- nant animals return to a ‘normal’ state of dopamine
tal methods including repeated PET imaging in function, presumably as the result of being in con-
individually and socially housed monkeys to trol of environmental events such as social contact,
study changes in brain DA function, the evalua- mobility through the environment, and food and
tion of social interactions in monkeys, and the sexual resources. The ability to control resources
study of intravenous cocaine self-administration may induce neurochemical changes that are
in socially housed animals. reflected in the over 20% increase in D2 receptor
RT0996_C18.qxd 11/8/04 2:11 PM Page 248
DVR and a decrease in vulnerability to the rein- This behavioral and biological phenotype is similar
forcing effects of cocaine in the dominant ani- to the one defined in the present study, and suggests
mals. This latter finding is even more striking that environmental events (such as becoming a
when one considers that in non-human primate particular social rank) may be as important as
models, cocaine functions as a reinforcer in biological predispositions in determining vulnera-
nearly all individually housed subjects. bility to drug abuse. It is not clear, however, whether
The dopaminergic system is centrally involved changes or differences in extracellular dopamine
in the neurochemical pathology of various psychi- or the D2 receptors themselves are contributing to
atric and neurodegenerative diseases involving the differences in cocaine reinforcement.
the basal ganglia13,23–25. The link between The present study demonstrates that social
dopaminergic functioning and behavioral context can have profound effects on brain dopamin-
processes has been extensively studied in the field ergic function in adult, non-human primates. Fur-
of drug abuse26–28. Importantly, individual differ- thermore, these neurobiological alterations in D2
ences in dopaminergic function can result in vary- receptor levels or availability produced by an
ing degrees of susceptibility to drug abuse29,30. environmental change can lead to qualitatively
The most well studied animals in this respect are different behavioral phenotypes. In the present
the high- and low-responders (HR and LR, re- study, this difference manifested as dominant
spectively) to novelty as measured by locomotion monkeys being resistant to cocaine’s effects, while
in an open field. LR rats are less likely to acquire subordinate monkeys were shown to be susceptible
amphetamine self-administration compared to to the reinforcing effects of cocaine. The promise
HR rats31, similar to our findings that low loco- of this model, specifically for vulnerability to
motor levels in monkeys predicted ‘resistance’ to drug abuse but more generally to any behavioral
the reinforcing effects of cocaine (also see ref. phenotype, lies in the ability to use brain-imaging
14). Neurobiologically, HR rats have higher basal, techniques to identify individual neurobiological
cocaine-stimulated and stress-induced dopamine characteristics associated with a particular social
levels, and have lower D2 receptor levels in the or environmental state, so that the course of re-
nucleus accumbens32. Using extracellular single- lated diseases and treatment can be followed and
neuron recordings, HR rats have higher basal ultimately altered.
firing rates of dopamine neurons in the ventral
tegmental area and decreased sensitivity to D2
receptor stimulation compared to LR rats; these Methods
differences in firing rates are associated with dif-
ferences in sensitivity to cocaine’s reinforcing Subjects
effects33. These data suggest that a hyperactive Twenty experimentally naive adult male cynomol-
dopamine system is characteristic of a drug-prone gus monkeys (Macaca fascicularis) were purchased
phenotype. from a commercial vendor (Primate Products,
There is an extensive literature describing the Miami, Florida, or Biomedical Resources Foun-
involvement of D2-like receptors in modulating dation, Houston, Texas). The monkeys were bred
cocaine’s reinforcing effects using direct-acting in captivity and, after weaning, were primarily
agonists and antagonists in several animal models individually housed. At the start of the study,
of cocaine abuse34–36. Consistent with these animal monkeys lived in cages (Allentown Caging, Al-
studies, PET data obtained in non-drug abusing lentown, New Jersey) equipped with removable
humans suggest that levels of D2 receptors predict metal partitions, providing four compartments
the subjective effects of stimulants37. In particular, each containing 11.5 cubic feet of space and a
individuals with low D2 receptor DVRs reported water spigot. Throughout the experiment, monkeys
that methylphenidate was more pleasurable and less were weighed weekly and maintained at approxi-
aversive than did individuals with high D2 DVRs. mately 95% of their free-feeding body weight by
RT0996_C18.qxd 11/8/04 2:11 PM Page 249
limited access to Purina Monkey Chow (100–120 order of scanning as a function of social rank was
g/day). In addition, monkeys received a multiple randomly determined. Details regarding the PET
vitamin tablet and fresh fruit approximately 2–3 data acquisition protocol, blood sampling pro-
times per week. At the time of the first PET scans, cedure, and metabolite analysis have been fully
monkeys weighed an average of 5.0 kg (range, described15,38–39. Briefly, monkeys were initially
3.8–7.6 kg) and were approximately 4.8 years old anesthetized with 8 mg/kg ketamine, maintained
(range, 3.4–6.4 years old). Approximately on isoflurane anesthesia (1.5%), administered a
8 months (range, 5–12 months) after the first PET paralytic (0.07 mg/kg vecuronium bromide), and
scan and after at least 3 months of social housing, prepared with percutaneous arterial and venous
the second PET scan was conducted, at which catheters. Although it is possible that ketamine-
point the monkeys weighed 5.2 kg (range, induced changes in dopamine levels can alter tracer
4.4–7.8 kg). The experimental manipulations kinetics when injected during a PET study40, we
described in this manuscript were conducted in previously demonstrated that under the present
accordance with the Guide for the Care and Use conditions, this dose of ketamine does not alter
of Laboratory Animals as adopted and promul- the DVR values obtained with [18F]FCP15.
gated by the NIH, and with the approval of the PET scans were obtained with a Siemens/CTI
Institutional Animal Care and Use Committee. ECAT 951/31 PET scanner, which has been de-
scribed elsewhere15,38,39. The effective resolution
Behavioral Profiles of (full width at half maximum) of the PET scanner
Socially Housed Monkeys was 9 mm in all axes for the reconstructed image
after filtering (Hanning filter with a 0.4-cycle/
Monkeys were individually housed for approxi-
pixel cutoff; resolution determined experimentally
mately 10 months before the initial PET scans.
using a line source).
The monkeys were placed into social groups of
ROI determinations were made from frame 13
four after initial PET scans were completed.
(25–30 minutes post-injection acquisition frame)
Approximately 20 behavioral observation sessions,
of the 26 dynamically acquired emission frames,
to measure various aggressive, submissive and
as this frame occurs at the time that the basal gan-
affiliative behaviors, were conducted per pen at
glia uptake reaches a plateau, providing good
regular intervals over the next 3 months14. Domi-
whole brain anatomic definition and sufficient
nance rank was determined by the outcomes of
contrast of the basal ganglia versus the rest of the
dyadic agonistic encounters. That is, the monkey
brain. ROIs for both basal ganglia and cerebellar
that aggressed toward and typically elicited sub-
regions were determined using the isocontour
mission from all others was designated as the
mode. For the basal ganglia, an isocontour thresh-
first-ranking monkey. The monkey that aggressed
old of 95% was set. For the cerebellum, a reference
toward all except the first-ranking monkey was
region with low D2 receptor density, an isodensity
designated the second-ranking monkey, and so on.
contour of 65–85% was routinely used. The use
In each pen, a linear hierarchy formed where the
of set isocontour levels for ROI determination
relationships between monkeys were transitive.
insures that the ROIs are easily and reproducibly
defined. The narrow range of 95% was used for
Social Rank and Dopaminergic Function the basal ganglia, as it selects the highest 5% of
Monkeys were first scanned with the D2 radioligand the pixel activities in the slice, thus minimizing
[18F]FCP while individually housed and again the errors due to partial volume effect. In contrast,
after they were placed in social groups and a sta- using an isocontour level of 65–85% in the cere-
ble social hierarchy was established. The average bellum generates a fairly large region of nearly
time between scans was 8 months (range, 4.5– uniform tracer concentration. For all studies, the
12.1 months). All animals in a particular pen were first 4 frames (0–4 min) were used for registration.
scanned before beginning the next pen and the The individual housing scan from each monkey
RT0996_C18.qxd 11/8/04 2:11 PM Page 250
was registered to the social housing scan and the Cocaine Self-Administration
ROI from the social housing scan placed onto the Monkeys were first trained to respond with food
individual housing scan, ensuring that identical as the reinforcer, and subsequently, saline and
ROIs were used41. This method of defining ROIs increasing doses of cocaine were made available
resulted in the same overall effects as using other for self-administration. Each morning (5–7 days/
methods (e.g. separate ROIs for each scan or week), monkeys from each pen were individually
ROIs from the summed scans). Time–activity housed by partitioning the cage into four quad-
curves were generated by the ECAT software by rants. Next, each monkey was seated in a primate
placing the isocontour regions determined from chair (Primate Products) and the chair was
frame 13 over the same region in each of the wheeled to the operant chamber (Med Associates,
26 frames. The ROI value is the average of all pix- East Fairfield, Vermont). The back of the animal
els contained within the ROI, for each time frame. was cleaned with 95% ethyl alcohol and betadine,
In the basal ganglia, right and left sides were and a 20 gauge Huber Point Needle (Access Tech-
determined separately, then averaged. The distri- nologies) was inserted into the port, connecting
bution volumes (DV) for these regions were the infusion pump to the catheter. The pump was
determined using the linear portion of the Logan operated for approximately 3 s, filling the port and
plot42, which in all cases included the last 80 min catheter with the dose of cocaine available during
of the PET scan. DVR was used as a metric of the experimental session. During the session, a
specific binding. fixed number of responses on the response lever
At the start of the scan, approximately 4 mCi of (for example, a fixed ratio of 30) resulted in
[18F]FCP was injected, followed by 3 ml of he- presentation of a banana pellet or activation of the
parinized saline. Tracer doses of FCP were injected infusion pump for 10 seconds. Each condition
(2.39 0.35 µg), and the mass of FCP was not (saline or each dose of cocaine availability) was
different across groups. There was no relationship available for at least 7 consecutive sessions and
between injected mass and binding potential. until responding was stable (15% of the mean
for the last 3 sessions). There was a return to food-
Surgery reinforced responding after evaluating a cocaine
dose. In some social groups, the hierarchy changed
After the second PET scan, each monkey was sur- while the cocaine dose-response curves were being
gically prepared under sterile conditions with a determined. Because the effects of changes in
vascular access port (Model GPV, Access Tech- rank, as well as cocaine exposure, on D2 binding
nologies, Skokie, Illinois), implanted under keta- potential were not assessed, only self-administration
mine (15 mg/kg) and butorphanol (0.025 mg/kg) data obtained during the time in which we were
anesthesia. An incision was made near the femoral confident that the monkey’s rank was not different
or jugular vein, and the catheter was inserted into from when the PET scans were conducted was in-
the vein for a distance calculated to terminate in the cluded in the analysis. Thus, data from 9 monkeys
vena cava. The distal end of the catheter was that were evaluated with 4 doses of cocaine and
threaded subcutaneously to an incision made saline, whose rank did not change across the period
slightly off the midline of the back. The vascular of the study, were evaluated.
access port was placed within a pocket formed by
blunt dissection near the incision. The port and
catheter were then flushed with a 50% dextrose/ Statistical Analysis
saline solution containing heparin (500 Units/ml). For purposes of PET image analysis, animals with
Antibiotic (25 mg/kg kefzol, BID; Cefazolin a rank of 1 or 4 were considered dominant or sub-
sodium, Marsam Pharmaceuticals, Cherry Hill, ordinate, respectively, whereas those ranking 2 or
New Jersey) was administered prophylactically 3 were considered intermediate, and their data
for 7–10 days beginning on the day of the surgery. were combined (n 5 for each rank). Data were
RT0996_C18.qxd 11/8/04 2:11 PM Page 251
analyzed with a 3 (rank: dominant, intermediate locomotor activity, dopamine synthesis and dopamine release.
and subordinate) by 2 (housing: individual and Neuropharmacology 32, 885–893 (1993).
7. Hurd, Y. L., Svensson, P. & Ponten, M. The role of
social) repeated measures ANOVA using com- dopamine, dynorphin, and CART systems in the ventral
mercially available software. Because of the striatum and amygdala in cocaine abuse. Ann. NY Acad.
smaller number of animals tested for acquisition Sci. 877, 499–506 (1999).
of cocaine self-administration (n 9), animals 8. Volkow, N. D., Fowler, J. S. & Wang, G. J. Imaging stud-
were considered either dominant (rank of 1 or 2, ies on the role of dopamine in cocaine reinforcement and
addiction in humans. J. Psychopharmacol. 13, 337–345
n 5) or subordinate (rank of 3 or 4, n 4). A (1999).
repeated-measures ANOVA with dose (5 levels 9. Blanc, G. et al. Response to stress of mesocortico-frontal
including saline) as the within-subject variable dopaminergic neurones in rats after long-term isolation.
and social rank (that is, dominant and subordinate) Nature 284, 265–267 (1980).
as the between-subject variable was conducted. 10. Rilke, O., May, T., Oehler, J. & Wolffgramm, J. Influences
of housing conditions and ethanol intake on binding char-
Pairwise comparisons for all analyses were made acteristics of D2, 5-HT1A, and benzodiazepine receptors
using the student Neuman-Keuls test. Correlation of rats. Pharmacol. Biochem. Behav. 52, 23–28 (1995).
coefficients were computed using the same software 11. Hall, F. S. et al. Isolation rearing in rats: pre- and postsy-
package. Differences were considered statistically naptic changes in striatal dopaminergic systems. Pharmacol.
significant when p 0.05. Biochem. Behav. 59, 859–872 (1998).
12. Grant, K. A. et al. Effect of social status on striatal dopamine
D2 receptor binding characteristics in cynomolgus mon-
keys assessed with positron emission tomography. Synapse
Acknowledgments 29, 80–83 (1998).
13. Blum, K., Cull, J. G., Braverman, E. R. & Comings, D. E.
We would like to thank C.S. Carter for comments Reward deficiency syndrome. Am. Scientist 84, 132–145
on the manuscript; C. Hubbard, T. Moore and (1996).
J. Lile for assistance with handling the monkeys; 14. Morgan, D. et al. Predictors of social status in cynomolgus
R. Kuhner for conducting the PET scans; and monkeys (Macaca fascicularis) after group formation.
E. Nicks and P. Warren for assistance analyzing Am. J. Primatol. 52, 115–131 (2000).
15. Nader, M. A. et al. PET imaging of dopamine D2 receptors
the social behavior data. This research was sup- with [18F] fluoroclebopride in monkeys: effects of isoflurane-
ported by the National Institute on Drug Abuse and ketamine-induced anesthesia. Neuropsychopharma-
Grant DA-10584. cology 21, 589–596 (1999).
16. Crowley, T. J., Stynes, A. J., Hydinger, M. & Kaufman,
REFERENCES I. C. Ethanol, methamphetamine, pentobarbital, mor-
phine, and monkey social behavior. Arch. Gen. Psychiatry
1. Leshner, A. I. Vulnerability to addiction: new research 31, 829–838 (1974).
opportunities. Am. J. Med. Genet. 96, 590–591 (2000). 17. Crowley, T. J. et al. Cocaine, social behavior, and alcohol-
2. Nader, M. A. & Woolverton, W. L. Effects of increasing solution drinking in monkeys. Drug Alcohol Depend. 29,
the magnitude of an alternative reinforcer on drug choice 205–223 (1992).
in a discrete-trials choice procedure. Psychopharmacology 18. Martin, S. P., Smith, E. O. & Byrd, L. D. Effects of domi-
105, 169–174 (1991). nance rank on damphetamine-induced increases in aggres-
3. Schenk, S., Lacelle, G., Gorman, K. & Amit, Z. Cocaine sion. Pharmacol. Biochem. Behav. 37, 493–496 (1990).
self-administration in rats influenced by environmental 19. Miczek, K. A. & Gold, L. H. d-Amphetamine in squirrel
conditions: implications for the etiology of drug abuse. monkeys of different social status: effects on social and
Neurosci. Lett. 81, 227–231 (1987). agonistic behavior, locomotion, and stereotypes. Psy-
4. LeSage, M. G., Stafford, D. & Glowa, J. R. Preclinical chopharmacology 81, 183–190 (1983).
research on cocaine self-administration: environmental 20. Bozarth, M. A., Murray, A. & Wise, R. A. Influence of
determinants and their interaction with pharmacological housing conditions on the acquisition of intravenous
treatment. Neurosci. Biobehav. Rev. 23, 717–741 (1999). heroin and cocaine self-administration in rats. Pharmacol.
5. Schultz, W., Tremblay, L. & Hollerman, J. R. Reward pro- Biochem. Behav. 33, 903–907 (1989).
cessing in primate orbitofrontal cortex and basal ganglia. 21. Phillips, G. D. et al. Isolation rearing enhances the loco-
Cereb. Cortex 10, 272–284 (2000). motor response to cocaine and a novel environment, but
6. Bowling, S. L., Rowlett, J. K. & Bardo, M. T. The effect impairs the intravenous self-administration of cocaine.
of environmental enrichment on amphetamine-stimulated Psychopharmacology 115, 407–418 (1994).
RT0996_C18.qxd 11/8/04 2:11 PM Page 252
22. Boyle, A. E., Gill, K. J., Smith, B. R. & Amit, Z. Differential impulse activity of midbrain dopamine neurons. J. Neurosci.
effects of an early housing manipulation on cocaine- 20, 8876–8885 (2000).
induced activity and self-administration in laboratory rats. 34. Nader, M. A., Green, K. L., Luedtke, R. R. & Mach, R. H.
Pharmacol. Biochem. Behav. 39, 269–274 (1991). The effects of benzamide analogues on cocaine self-
23. Verhoeff, N. P. Radiotracer imaging of dopaminergic administration in rhesus monkeys. Psychopharmacology
transmission in neuropsychiatric disorders. Psychophar- 147, 143–152 (1999).
macology 147, 217–249 (1999). 35. Caine, S. B., Negus, S. S. & Mello, N. K. Effects of
24. Calabresi, P., De Murtas, M. & Bernardi, G. The neostria- dopamine D(1-like) and D(2-like) agonists on cocaine self-
tum beyond the motor function: experimental and clinical administration in rhesus monkeys: rapid assessment of
evidence. Neuroscience 78, 39–60 (1997). cocaine dose-effect functions. Psychopharmacology 148,
25. Wang, G. J. et al. Brain dopamine and obesity. Lancet 41–51 (2000).
357, 354–357 (2001). 36. Khroyan, T. V., Barrett-Larimore, R. L., Rowlett, J. K. &
26. Piazza, P. V. & Le Moal, M. The role of stress in drug self- Spealman, R.D. Dopamine D1- and D2-like receptor
administration. Trends Pharmacol. Sci. 19, 67–74 (1998). mechanisms in relapse to cocaine-seeking behavior:
27. Volkow, N. D. et al. Effects of chronic cocaine abuse on effects of selective antagonists and agonists. J. Pharmacol.
postsynaptic dopamine receptors. Am. J. Psychiatry 147, Exp. Ther. 294, 680–687 (2000).
719–724 (1990). 37. Volkow, N. D. et al. Prediction of reinforcing responses
28. Volkow, N. D. et al. Decreased dopamine D2 receptor to psychostimulants in humans by brain dopamine D2
availability is associated with reduced frontal metabolism receptor levels. Am. J. Psychiatry 156, 1440–1443 (1999).
in cocaine abusers. Synapse 14, 169–177 (1993). 38. Mach, R. H. et al. Comparison of two fluorine-18 labeled
29. Piazza, P. V. & LeMoal, M. Pathophysiological basis of benzamide derivatives that bind reversibly to dopamine
vulnerability to drug abuse: role of an interaction between D2 receptor: in vitro binding studies and positron emission
stress, glucocorticoids, and dopaminergic neurons. Annu. tomography. Synapse 24, 322–333 (1996).
Rev. Pharmacol. Toxicol. 36, 359–378 (1996). 39. Mach, R. H. et al. Use of positron emission tomography to
30. Lucas, L. R. et al. Neurochemical characterization of study the dynamics of psychostimulant-induced dopamine
individual vulnerability to addictive drugs in rats. Eur. J. release. Pharmacol. Biochem. Behav. 57, 477–486 (1997).
Neurosci. 10, 3153–3163 (1998). 40. Laruelle, M. Imaging synaptic neurotransmission with
31. Piazza, P. V., Deminiere, J.-M., Le Moal, M. & Simon, H. in vivo binding competition techniques: a critical review.
Factors that predict individual vulnerability to amphetamine J. Cereb. Blood Flow Metab. 20, 423–451 (2000).
self-administration. Science 245, 1511–1513 (1989). 41. Woods, R. P. et al. Automated image registration: I. General
32. Piazza, P. V., Deroche-Gamonent, V., Rouge-Pont, F. & methods and intrasubject, intramodality validation.
Le Moal, M. Vertical shifts in self-administration dose- J. Comput. Assist. Tomogr. 22, 141–152 (1998).
response functions predict a drug-vulnerable phenotype pre- 42. Logan, J. et al. Graphical analysis of reversible radioli-
disposed to addiction. J. Neurosci. 20, 4226–4232 (2000). gand binding from time-activity measurements applied to
33. Marinelli, M. & White, F. J. Enhanced vulnerability to [N-11C-methyl]-cocaine PET studies in human subjects.
cocaine self-administration is associated with elevated J. Cereb. Blood Flow Metab. 10, 740–747 (1990).
RT0996_C19.qxd 11/9/04 4:11 PM Page 253
R E A D I N G 1 9
Rethinking Feelings: An fMRI Study of

the Cognitive Regulation of Emotion
•
Kevin N. Ochsner1, Silvia A. Bunge2, James J. Gross1, and John D. E.
Gabrieli1
The ability to cognitively regulate emotional responses to aversive events is important for mental and
physical health. Little is known, however, about neural bases of the cognitive control of emotion. The
present study employed functional magnetic resonance imaging to examine the neural systems used to
reappraise highly negative scenes in unemotional terms. Reappraisal of highly negative scenes reduced
subjective experience of negative affect. Neural correlates of reappraisal were increased activation of
the lateral and medial prefrontal regions and decreased activation of the amygdala and medial orbito-
frontal cortex. These findings support the hypothesis that prefrontal cortex is involved in constructing
reappraisal strategies that can modulate activity in multiple emotion-processing systems.
Introduction lessening the emotional consequences of an other-

wise distressing experience.
We humans are extraordinarily adaptable creatures. The cognitive transformation of emotional
Drawing upon a vast array of coping skills, we can experience has been termed “reappraisal.” In both
successfully manage adversity in even the most try- experimental and individual-difference studies,
ing of circumstances. One of the most remarkable reappraising an aversive event in unemotional
of these skills was described by Shakespeare’s terms reduces negative affect with few of the
(1998/1623, p. 216) Hamlet, who observed, “there physiological, cognitive, or social costs associated
is nothing either good or bad, but thinking makes it with other emotion-regulatory strategies, such as
so.” Although Hamlet himself failed to capitalize on the suppression of emotion-expressive behavior
this insight, his message is clear: We can change the (Jackson, Malmstadt, Larson, & Davidson, 2000;
way we feel by changing the way we think, thereby Richards & Gross, 2000; Gross, 1998, 2002; Gross
1
Stanford University, 2Massachusetts Institute of Technology.
253
RT0996_C19.qxd 11/9/04 4:11 PM Page 254
& John, in press). The mechanisms that mediate affective response, signaling the need for reap-
such reappraisals, however, are not yet under- praisal to continue. In a variety of tasks involving
stood. The goal of the present study was to use response conflict (Barch et al., 2001; van Veen, Co-
functional magnetic resonance imaging (fMRI) to hen, Botvinick, Stenger, & Carter, 2001; Phelps,
elucidate the neural bases of reappraisal. Hyder, Blamire, & Shulman, 1997) or overriding
Although little prior work has directly exam- prepotent response tendencies (Carter et al., 2000;
ined the neural systems involved in the cognitive Peterson et al., 1999), these functions have been as-
control of emotion, we expected that it would in- sociated with the dorsal anterior cingulate cortex
volve processing dynamics similar to those impli- (for reviews, see Botvinick, Braver, Barch, Carter,
cated in other well-studied forms of cognitive & Cohen, 2001; Bush, Luu, & Posner, 2000). The
control. In general, cognitive control is thought to third process involves reevaluating the relationship
involve interactions between regions of lateral between internal (experiential or physiological)
(LPFC) and medial prefrontal cortex (MPFC) that states and external stimuli, which may be used to
implement control processes and subcortical and monitor changes in one’s emotional state during
posterior cortical regions that encode and repre- reappraisal. The dorsal regions of the MPFC are
sent specific kinds of information (Miller & Co- activated when making attributions about one’s
hen, 2001; Knight, Staines, Swick, & Chao, 1999; own (Paradiso et al., 1999; Lane, Fink, Chau, &
Smith & Jonides, 1999). By increasing or de- Dolan 1997) or another person’s (Gallagher et al.,
creasing activation of particular representations, 2000; Happe et al., 1996) emotional state as well as
prefrontal regions enable one to selectively during viewing of emotional films (Beauregard
attend to and maintain goal-relevant information et al., 1998; Lane, Reiman, Ahern, Schwartz, &
in mind and resist interference (Miller & Cohen, Davidson, 1997; Reiman et al., 1997) or photos
2001; Knight et al., 1999; Smith & Jonides, 1999). (Lane, Reiman, Bradley, et al., 1997). Importantly,
Based on these cognitive neuroscience models, activation of the medial frontal cortex when antici-
as well as process models of emotion and emo- pating painful shock (Chua, Krams, Toni, Passing-
tion regulation (Gross, 2002; Ochsner & Feldman ham, & Dolan, 1999; Hsieh, Stone-Elander, & Ing-
Barrett, 2001), we hypothesized that comparable var, 1999; Ploghaus et al., 1999) may be inversely
interactions between cognitive control and emotion- correlated with the experience of anxiety (Simpson
processing systems would underlie reappraisal. et al., 2000), suggesting its importance for regula-
With respect to cognitive-processing systems, tory control (cf. Morgan & LeDoux, 1995).
we hypothesized that reappraisal would involve With respect to emotion-processing systems,
three processes implemented by lateral and medial we hypothesized that reappraisal would modulate
frontal cortices. The first is the active generation of the processes involved in evaluating a stimulus as
a strategy for cognitively reframing an emotional affectively significant. Many theories of emotion
event in unemotional terms, and keeping that strat- posit that at least two types of evaluative processing
egy in mind as long as the eliciting conditions en- are involved in emotion generation (Lazarus, 1991;
dure. In neuropsychological (Barcelo & Knight, for a review, see Scherer, Schorr, & Johnstone,
2002; Stuss, Eskes, & Foster, 1994), functional im- 2001). One type is important for determining
aging (Cabeza & Nyberg, 2000; Smith & Jonides, whether a stimulus is affectively relevant and may
1999; Barch et al., 1997), and electrophysiological be relatively automatic, whereas a second type is
(Barcelo, Suwazono, & Knight, 2000; Nielsen- important for evaluating contextual meaning and
Bohlman & Knight, 1999) studies, these functions the appropriateness of possible responses (Scherer
have been associated with working memory et al., 2001; Lazarus, 1991). Evidence suggests
processes localized in the LPFC. The second that two highly interconnected brain structures
process may monitor interference between top- (Cavada, Company, Tejedor, Cruz-Rizzolo, &
down reappraisals that neutralize affect and bot- Reinoso-Suarez, 2000), the amygdala and medial
tom–up evaluations that continue to generate an orbital frontal cortex (MOFC), are associated
RT0996_C19.qxd 11/9/04 4:11 PM Page 255
Rethinking Feelings: An fMRI Study of the Cognitive Regulation of Emotion ■ 255
with these two types of emotion processing involves both attention to and awareness of one’s
(Ochsner & Feldman Barrett, 2001; LeDoux, emotional state, as well as regulatory processes
2000; Bechara, Damasio, Damasio, & Lee, 1999). directed towards altering it (Gross, 1998), we
On one hand, the amygdala is important for the employed two conditions: On “Attend trials,” par-
preattentive detection and recognition of affectively ticipants were asked to let themselves respond
salient stimuli (Anderson & Phelps, 2001; Morris, emotionally to each photo by being aware of their
Ohman, & Dolan, 1999; Whalen et al., 1998), feelings without trying to alter them. On “Reap-
learning and generating physiological and behav- praise trials,” participants were asked to interpret
ioral responses to them (LeDoux, 2000; Bechara photos so that they no longer felt negative in
et al., 1999), and modulating their consolidation response to them. Because both Attend and Reap-
into declarative memory (Hamann, Ely, Grafton, praise trials involve attention to emotion, regions
& Kilts, 1999; Cahill, Babinsky, Markowitsch, & more active when reappraising than attending
McGaugh, 1995). On the other hand, the MOFC were thought to reflect processes used to exert
is important for representing the pleasant or un- cognitive control. In contrast, regions more active
pleasant affective value of a stimulus (Kawasaki for Attend than Reappraise trials were thought to
et al., 2001; O’Doherty, Kringelbach, Rolls, be important for emotion processing that would
Hornak, & Andrews, 2001; Davidson & Irwin, be deactivated by reappraisal.
1999; Rolls, 1999; Elliott, Frith, & Dolan, 1997) Each trial began with a 4-sec presentation of a
in a flexible format that is sensitive to momentary negative or neutral photo, during which partici-
changes in social and motivational context pants were instructed simply to view the stimulus
(Ochsner & Feldman Barrett, 2001; Bechara, on the screen. This interval was intended to pro-
Damasio, & Damasio, 2000; Rolls, 1999). To- vide time for participants to apprehend complex
gether, the amygdala and the MOFC differentially scenes and to allow an emotional response to be
encode and represent the affective properties of generated that participants then would be asked to
stimuli (Bechara et al., 1999; Rolls, 1999), and we regulate. The word Attend (for negative or neutral
sought to determine whether reappraisal modulates photos) or Reappraise (negative photos only) then
activity in the MOFC, amygdala, or both. appeared beneath the photo and participants fol-
To test these hypotheses, we adapted an exper- lowed this instruction for 4 sec, at which time the
imental procedure used to study regulation of the photo disappeared from the screen. Because we
fear-potentiated startle eyeblink response (Jackson were interested in the processes used to actively
et al., 2000). In that study, participants viewed reappraise an affective event as it unfolds, we
aversive photos and were instructed either to in- focused our analyses on this portion of each trial.
crease, maintain, or decrease (postexperimental During this portion of the trial, we predicted that
debriefing suggested that participants reappraised) (a) Reappraise trials would result in greater lateral
their emotional reactions. Startle eyeblink magni- prefrontal activation than Attend trials and (b) that
tude (which was used as an indicator of the rela- Attend trials would show greater activation of the
tive strength of an emotional reaction across trial MOFC and the amygdala than would Reappraise
types) increased, remained constant, or decreased trials. For approximately three more seconds, par-
according to the regulatory strategy being em- ticipants could continue attending to or reapprais-
ployed. This result suggests that participants can ing any feelings that lingered after presentation of
successfully regulate their emotional responses the photo. A rating scale then appeared, which
on a trial-by-trial basis. To isolate the processes participants used to rate the strength of their cur-
related to the cognitive control of emotion, we rent negative affect and which we used to verify
needed to compare reappraisal to another condition that that reappraisal had successfully reduced
that draws on processes invoked by reappraisal, negative feelings as compared to Attend trials.
but are not related to the regulation of affect Finally, participants were instructed to relax for 4
per se. Because we hypothesized that reappraisal sec before the next trial began.
RT0996_C19.qxd 11/9/04 4:11 PM Page 256
Results the affective response they had to each photo

when it was viewed during the test session. Results
The experimental procedure included a measure indicated that pre- and post-test ratings of nega-
that allowed us to segregate and analyze separately tive affect were highly correlated for all trial types
the trials on which participants experienced their and all levels of negative photos (i.e, most, mod-
strongest emotional responses without biasing the erately, or least negative, all p .01), which sug-
initial perception of photos during scanning. In a gests that post-test ratings can provide a reliable in-
post-scanning session, participants viewed each neg- dex of one’s initial affective response to a photo.
ative photo a second time and rated the strength of Success of Reappraisal. Ratings made during
their initial affective response (i.e., when they first scanning showed that overall, moderate, and least
viewed it, before they had attended or reappraised). negative photos elicited less negative affect than
These ratings were used to identify the third of the the most negative photos (both p .01), which is
negative photos of each trial type that were rated consistent with post-test reports of initial affective
most negative by each participant. As has been found response. Significantly, reappraisal was success-
in prior studies (Canli, Zhao, Brewer, Gabrieli, & ful for the most negative photos: The average ratings
Cahill, 2000), preliminary analyses indicated that re- of the strength of negative affect were high on Attend
liable activation of emotion-processing systems was trials (M 3.48) and were significantly lower on
observed only for these most negative images. Given Reappraise trials1 [M 1.90, t(14) 17.41, p .01]
that the goal of this study was to examine the mod- (Figure 19.1). Reappraisal was potent enough that af-
ulation of these systems by reappraisal, the analyses fect was diminished to that experienced when attend-
reported here focused only on these trial types. ing to the least negative photos (p .5). Although
negative affect was not as great as that reported for
Subjective Reports of Negative Affect most negative photos, ratings for both the moderately
Segregating Most Negative Photos. Post-scan rat- negative and least negative photos showed a similar
ings of affective response indicated that the third
of photos rated most negative (M 3.77) elicited Strong 4
significantly greater negative affect than the
moderately negative (M 2.99) or least negative 3.5
(M 1.93) third of photos (all differ p .01).
negative affect
Strength of
3
Affect ratings for negative photos selected from
Attend as compared to Reappraise trials did not 2.5
differ significantly at any level of affect (all p .16).
2
The fact that retrospective ratings of negative
affect were equivalent on Attend and Reappraise 1.5
trials suggests that in-scan reappraisals did not Weak 1
bias post-scan emotional responses to photos.
However, because these ratings were provided
retrospectively, it is important to provide inde-
pendent evidence that post-test ratings can be
Attend Reapp Attend
correlated with a participant’s initial affective
response to a stimulus (Ochsner & Schacter, in Negative Neutral
press). A separate pilot behavioral study was con- FIGURE 19.1 ■ Average negative ratings made during
ducted to provide such evidence. Eight female scanning for the most negative photos (the third of photos
participants rated their affective response to pho- that elicited the most negative affect for a given participant).
Negative affect was strong on Attend trials and decreased
tos an average of 3 days before completing a test
significantly on Reappraise (Reapp) trials (p .01). When
session that procedurally was equivalent to the participants attended to their feelings towards neutral pho-
scanning session used in the present study. They tos, the negative affect elicited was significantly weaker
then completed post-test retrospective ratings of than for all other trial types (both p at least.01).
RT0996_C19.qxd 11/9/04 4:11 PM Page 257
pattern of successful reappraisal [moderate: Attend for each photo whether they had reinterpreted the
M 2.75, Reappraise M 1.52, t(14) 7.66, p photo (as instructed) or had used some other type
.01; least: Attend M 2.05, Reappraise M 1.43, of reappraisal strategy. Compliance with instructions
t(14) 4.27, p .01]. However, comparisons with was very high: On less than 4% of trials with
affect reported on Attend-neutral trials (M 1.08) highly negative photos did participants report
showed that in no case did reappraisal entirely elim- using another type of strategy.
inate negative affect (p .01 for all comparisons).
Manipulation Check. In a pre-scan training ses- Brain Imaging Results
sion, participants were instructed to reappraise Activation by Reappraisal. Reappraisal-sensitive
photos during scanning by generating an interpre- regions were identified by greater activation in re-
tation of, or story about, each photo that would sponse to the most negative photos on Reappraise
explain apparently negative events in a less negative than on Attend trials. Consistent with predictions,
way. To verify that participants had, in fact, reap- significantly activated regions included the dorsal
praised in this manner, during the post-scan rating and ventral regions of the left LPFC, as well as the
session participants also were asked to indicate dorsal MPFC (Figure 19.2, Table 19.1). Additional
Right Left
LPFC
MOFC
Bottom Front
LPFC
MOFC
Modulation Activation by
by reappraisal reappraisal
FIGURE 19.2 ■ (A color version of this figure follows page 146.) Group-aver-
aged brain activations when reappraising or attending to feelings in response
to the most negative photos. Two contrasts are shown: The Attend Reap-
praise (shown in red) contrast shows regions important for emotion processing
that are significantly modulated by reappraisal and the Reappraise Attend
(shown in green) contrast shows regions significantly activated when exerting
cognitive control over emotion activated by reappraisal. Top and bottom brain
images on the right show regions of the left dorsal and ventral LPFC associ-
ated with cognitive control that were activated by reappraisal. Right side and
bottom left brain images show reappraisal-related modulation of a region of left
MOFC associated with representing the affective properties of stimuli.
RT0996_C19.qxd 11/9/04 4:11 PM Page 258
TABLE 19.1. Group Activations for Reappraise Attend Contrast

Coordinates
Region of Activation Brodmann’s Area x y z Z Score Volume (mm3)
Group Contrast
Superior frontal gyrus L6 36 14 58 3.90 2736
Superior frontal gyrus L6/8 24 6 64 3.71 (L)
Middle frontal gyrus L6/8 24 10 56 3.68 (L)
Middle frontal gyrus L6/8 40 2 60 3.60 416
Inferior frontal gyrus L46 54 42 12 3.79 736
L44/10 48 46 4 3.31 (L)
Dorsomedial prefrontal cortex 8 12 18 54 3.47 1040
8 4 20 54 3.39
Dorsomedial prefrontal cortex 8/32 8 28 40 3.88 224
Temporal pole 28 22 4 26 4.21 128
Lateral occipital cortex 19 38 74 40 4.23 240
Supramarginal gyrus R39/40 54 70 30 4.23 688
Positive Correlation between Activation and Drop in Negative Affect then Reappraising
Anterior cingulate R24 6 14 32 4.67 88
Supramarginal gyrus R40 54 48 34 4.05 40
Clusters of 5 or more contiguous voxels whose global maxima meet a t threshold of 3.09, p .001 uncorrected, are reported. Local
maxima for these clusters are denoted with (L). Coordinates are in MNI space.
reappraisal-related activations were observed in left response increase when reappraising as compared
temporal pole, right supramarginal gyrus, and left to attending were correlated with reappraisal suc-
lateral occipital cortex (Figure 19.2, Table 19.1). cess. These correlations were highly significant both
Contrary to expectations, activation of the cingulate for the foci in the anterior cingulate (R2 .649,
cortex was not observed. r .805, p .0001) and the supramarginal gyrus
Cingulate involvement in reappraisal was re- (R2 .570, r .755, p .0006).
vealed, however, in an SPM99 regression analysis Modulation by Reappraisal. Emotion-sensitive
used to identify regions for which level of brain regions modulated by reappraisal were identified
activation across participants correlated signifi- by greater activation in response to the most neg-
cantly with reappraisal success. An index of reap- ative photos on Attend than on Reappraise trials.
praisal success was computed for each participant Activation was observed in a region of left MOFC
by subtracting the mean level of negative affect (Figure 19.2, Table 19.2). Additional regions of
reported on Reappraise trials from that reported activation were found in the left posterior insula,
on Attend trials when highly negative images right medial occipital cortex, and right inferior
were shown. Larger difference scores thus corre- parietal cortex (Figure 19.2, Table 19.2). The amyg-
sponded to a greater decrease in negative affect, dala, an a priori region-of-interest (ROI), was not
which is indicative of a more effective reappraisal. significantly activated at a map-wise statistical
At a threshold of p .001 (uncorrected), activity in threshold of p .001. However, significant activa-
no brain regions was negatively correlated, and in tion was observed in the right amygdala at a more
only two regions was positively correlated, with liberal threshold (p .005) (Figure 19.4). This
reappraisal success such that greater activation finding was confirmed by the results of a planned
predicted greater decreases in negative affect. These ROI analysis in which parameter estimates that
two regions were located in the right anterior cin- model the amplitude of the fMRI response were
gulate and right supramarginal gyrus (Figure 19.3, extracted from structurally defined ROIs (see, e.g.,
Table 19.1). To more precisely characterize these Figure 19.4). For the right amygdala, this analy-
correlations, mean parameter estimates of the sis revealed a significantly greater amplitude of
RT0996_C19.qxd 11/9/04 4:11 PM Page 259
a) b)
2.00 R2 = .645, r = .805, p = .0001
Parameter estimate of Cingulate

activation when Reappraising
4
1.50
3 1.00
.50
2
.00
1 -.50
.80 1.0 1.2 1.4 1.6 1.8 2.0 2.2

0
Z-value Drop in negative affect due to Reappraisal
FIGURE 19.3 ■ Region of right anterior cingulate cortex (MNI coordinates: 6, 14, 32) identified in a regression analysis
as showing a significant correlation between increasing activation and decreasing negative affect on Reappraise as
compared to Attend trials with negative photos. Activation is shown on SPM99 canonical T1 image.
response on Attend than Reappraise trials (p .025, tive photos on Attend than on Reappraise trials,
one-tailed for planned comparison). The response whereas activated regions of the LPFC exhibited
to most negative photos on Reappraise trials was precisely the opposite pattern (p .025, one-
not significantly different from the response to tailed for planned comparisons). For one of the
neutral photos on Attend trials [t(14) 1, p .5] activated prefrontal regions—the ventral LPFC—
(Figure 19.5). No significant differences in response the reappraisal-related increase in brain activation
were shown across trial types for the left amyg- was correlated across participants with the reap-
dala ROI [all t(14) 1, p .5]. praisal-related decrease in amygdala activation
To further characterize the relationships be- (r .677, p .004; for all other regions,
tween reappraisal-related increases and decreases p .19). Ventral LPFC activation also was nega-
in brain activation, mean parameter estimates tively correlated with MOFC activation, albeit to
across trial types were contrasted for a set of func- a lesser extent (r .494, p .06) (Figure 19.6).
tionally defined ROIs: The first was the region of Activation on Attend-neutral trials was not signif-
MOFC identified by the Attend Reappraise icantly different from either activation on At-
contrast, and the others were regions of the LPFC tend–most negative trials in this ventral prefrontal
activated by the Reappraise Attend contrast region [t(14) 1.2, p .24] or from activity on
(Table 19.1). These analyses indicated that Reappraise–most negative trials in MOFC and
MOFC exhibited greater activation to most nega- amygdala [both t(14) 1, p .5] (Figure 19.5).
TABLE 19.2. Group Activations for Attend Reappraise Contrast

Coordinates
Region of Activation Brodmann’s Area x y z Z Score Volume (mm3)
Medial orbito-frontal cortex L11 6 44 22 4.17 112

Posterior insula L13 44 16 2 3.84 240
Inferior parietal cortex R39/19 38 64 34 3.86 640
Medial occipital cortex R19 22 76 40 3.46 240
Amygdala R 16 12 20 2.88* 112
Clusters of 5 or more contiguous voxels whose global maxima meet a t threshold of 3.09, p .001 uncorrected, are reported. Local
maxima for these clusters are denoted with (L). Coordinates are in MNI space.
*T 2.98, p .005.
RT0996_C19.qxd 11/9/04 4:11 PM Page 260
2000; LeDoux, 2000; Bechara et al., 1999; Rolls,

6
1999).
5 Cognitive Processes
Supporting Reappraisal
4
The particular regions of the LPFC and MPFC ac-
3 tivated by reappraisal are similar to the regions
commonly activated across working memory and
2 response-selection tasks that involve maintaining
1
information in awareness and resisting interfer-
ence from competing inputs (Cabeza & Nyberg,
2000; Smith & Jonides, 1999; Courtney, Petit,
Z-value
Maisog, Ungerleider, & Haxby, 1998; Petit,
FIGURE 19.4 ■ (A color version of this figure follows Courtney, Ungerleider, & Haxby, 1998; Alexan-
page 146.) Coronal image showing the group-averaged der, Delong, & Strick, 1996). These similarities
cluster of activation in right amygdala for the Attend Reap- suggest that an overlapping set of prefrontal re-
praise contrast for trials with the most negative photos
(p .005). The focus is centered on MNI coordinates (16,
gions support the cognitive regulation of feelings
12, 20). Activation is shown on group-averaged and thoughts (Miller & Cohen, 2001; Ochsner &
anatomy. Feldman Barrett, 2001; Davidson & Irwin, 1999;
Knight et al., 1999; Smith & Jonides, 1999).
Discussion The finding that activation of the ventral LPFC
was inversely correlated with the activation of the
This study is one of the first to use functional amygdala and the MOFC suggests that this region
imaging to draw inferences about the neural bases may play a direct part in modulating emotion pro-
of the cognitive control of emotion. Behaviorally, cessing, perhaps related to the role of ventral
reappraisal of negative photos successfully dimin- frontal regions in interference control and behav-
ished negative affect. Neural correlates of effec- ioral inhibition more generally (Miller & Cohen,
tive reappraisal were (1) activation in the regions 2001; Smith & Jonides, 1999). It is notable, how-
of the LPFC and MPFC essential for working ever, that other prefrontal regions were even more
memory, cognitive control (Miller & Cohen, strongly activated by reappraisal, although these
2001; Knight et al., 1999; Smith & Jonides, 1999), activations did not correlate significantly with ac-
and self-monitoring (Gusnard, Akbudak, Shulman, tivity in emotion-processing regions. Although
& Raichle, 2001) and (2) decreased activation this initial study was not designed to determine
in two regions involved in emotion processing, the precise contributions to the reappraisal
the MOFC and the amygdala (Adolphs, 2001; process made by each region, they may mediate
Ochsner & Feldman Barrett, 2001; Bechara et al., processes necessary for, but not directly related
1999; Davidson & Irwin, 1999; Rolls, 1999). In to, successful reappraisal. For example, dorsome-
addition, the magnitude of ventral LPFC activa- dial prefrontal cortex was the region most
tion during reappraisal was inversely correlated strongly activated by reappraisal. This region has
with activation in both emotion-processing re- been associated with emotional awareness (Lane,
gions. Taken together, these findings provide 2000; Lane et al., 1997), drawing inferences about
the first evidence that reappraisal may modu- one’s own (Paradiso et al., 1999) or others’ (Gal-
late emotion processes implemented in the lagher et al., 2000; Happe et al., 1996) emotional
amygdala and MOFC that are involved in states, and self-related processing (Gusnard et al.,
evaluating the affective salience and contextual 2001) more generally. The need to monitor and
relevance of a stimulus (Ochsner & Feldman evaluate the self-relevance of emotional stimuli
Barrett, 2001; Phelps et al., 2001; Bechara et al., could be important whenever one reappraises
RT0996_C19.qxd 11/9/04 4:11 PM Page 261
Left
a) ventral LPFC
b)
.25
.20
.15
.10
.05
0
-.05
-.10
-.15
-.20
Parameter estimate of activation

Left
.10 MOFC
.075
.05
.025
0
-.025
-.05
Right
.15 Amygdala
.10
.05
-.05
-.10
Attend- Reapp- Attend-
Most-Neg Most-Neg Neutral
FIGURE 19.5 ■ ROI analyses. (a) Functionally or structurally defined ROIs and (b) group mean
parameter estimates (M SEM) for each ROI on Attend and Reappraise (Reapp) trials using
negative photos, and Attend trials using neutral photos. Note that the negative images con-
tributing to this analysis were identified as the third of negative images that were given the most
negative rating by each participant (see Methods and Results). Top row: A functionally defined
ROI within the left ventral LPFC (BA 46/10) activated by Reappraise Attend contrast, cen-
tered on MNI coordinates (54, 42, 12) and shown on group-averaged anatomy. This is the only
prefrontal region whose activation during reappraisal was inversely correlated with activation in
emotion-processing regions (shown in Figure 19.6). Middle row: A functionally defined ROI
within left MOFC (BA 11) identified by the Attend Reappraise contrast, centered on MNI
coordinates (6, 46, 20) and shown on group-averaged anatomy. Bottom row: Sample struc-
turally defined ROI for the right amygdala from a single subject centered on MNI coordinates
(24, 7, 15). Ventral LPFC activation on Reappraise–most negative trials was significantly
greater than on Attend–most negative trials (p .025, one-tailed), whereas the MOFC and
amygdala showed the opposite pattern (both p .025, one-tailed).
RT0996_C19.qxd 11/9/04 4:11 PM Page 262
a) Although cingulate activation was not observed in

the group contrast of Reappraise and Attend trials,
.08 r = -.494, p < .061 there was, across participants, a positive correla-
Parameter estimate of MOFC
.06
tion between cingulate activation and effective
.04
.02
reappraisal. The anterior cingulate cortex is
0 thought to be important for monitoring ongoing
-.02 processing and evaluating the need for cognitive
-.04 control (e.g., Botvinick et al., 2001), and it might
-.06
-.08
be expected that successful reappraisal would de-
-.1 pend upon the use of this process to monitor for
-.12 conflicts between initial emotional appraisals and
-.2 -.1 0 .1 .2 .3 .4 .5 .6 .7 .8 .9 cognitively restructured reappraisals. In the pres-
Parameter estimate of left ventral
LPFC activation when Reappraising
ent study, cingulate activation may therefore
reflect active monitoring that enhanced the cogni-
b) tive transformation of an aversive experience.
.2 r = -.677, p < .004 Emotion Processes Modulated

Parameter estimate of Amygdala
by Reappraisal
.1
The observation that reappraisal can influence
0 brain systems implicated in emotion processing
-.1 may have significance for contemporary appraisal
theories of emotion (for a review, see Scherer et al.,
-.2
2001). It has been clear that reappraisal dimin-
-.3 ishes negative emotion experience and negative
-.4 emotion-expressive behavior (Gross, 1998), but
-.2 -.1 0 .1 .2 .3 .4 .5 .6 .7 .8 .9 theorists have not specified the types of emotion
Parameter estimate of left ventral processing that might be influenced by reap-
LPFC activation when Reappraising praisal. Although the present study was not de-
FIGURE 19.6 ■ Correlations between reappraisal-induced signed to determine which specific emotion-
changes in parameter estimates of activation for the processing functions attributable to the amygdala
functional and structural ROIs shown in Figure 19.5.
and MOFC are modulated by reappraisal, modu-
During reappraisal, increases in response in the ventral
LPFC (a) correlated with decreases in response in the lation of these two brain structures is consistent
amygdala and (b) to a lesser degree correlated with de- with the idea that reappraisal can influence
creases in response in the MOFC. processes involved in evaluating the affective
salience of a stimulus (Anderson & Phelps, 2001;
Morris et al., 1999; Whalen et al., 1998), as well as
(Scherer et al., 2001; Lazarus, 1991) and may be those important for evaluating the salience of that
used to regulate anxiety in anticipation of aversive stimulus in the context of current situational or
events (Simpson et al., 2000). Similarly, the supe- personal goals (Kawasaki et al., 2001; Ochsner &
rior prefrontal regions have been associated with Feldman Barrett, 2001; O’Doherty et al., 2001;
spatial working memory and control of eye move- Bechara et al., 2000; Davidson & Irwin, 1999;
ments (Smith & Jonides, 1999; Courtney et al., Rolls, 1999; Elliott et al., 1997).
1998), both of which could be needed for analyz- Further work will be needed to determine
ing and reinterpreting perceptual inputs during which specific aspects of amygdala and OFC
reappraisal. functioning can be modulated by reappraisal. On
We also hypothesized that the anterior cingu- one hand, the appraisal function of the amygdala
late cortex would be involved in reappraisal. often is characterized as automatic (e.g., LeDoux,
RT0996_C19.qxd 11/9/04 4:11 PM Page 263
2000; Morris et al., 1999). In the present study, it amygdala activity (which could be due, in part, to
is likely that reappraisal did not modulate this early signal loss in MOFC). A second possible route in-
amygdala response, which is thought to depend volves prefrontal modulation of posterior percep-
upon subcortical inputs from the senses. Instead, tual and semantic inputs to the amygdala from the
reappraisal may have influenced a more sustained occipital and parietal regions (de Fockert, Rees,
response that may depend (as discussed below) on Frith, & Lavine, 2001; Miller & Cohen, 2001;
cortical inputs and is more amenable to control by Knight et al., 1999; Smith & Jonides, 1999). Reap-
cognitive processes. It will be important to deter- praising the affective significance of images in
mine whether and how reappraisal could influ- working memory may reorganize these inputs so
ence the early automatic response as well. On the that the amygdala and the MOFC no longer regis-
other hand, the MOFC often is characterized as ter the presence of an aversive stimulus. This view
serving a regulatory function,2 as evidenced, for is supported by the fact that reappraisal modu-
example, by its roles in decision-making involv- lated activation in the lateral occipital cortex, a re-
ing risky choices (e.g., Bechara et al., 2000) and gion associated with visual object processing, and
extinction of conditioned fear responses (e.g., the supramarginal gyrus, an inferior parietal re-
Morgan et al., 1995; see, however, footnote 2). On gion associated with attentional selection and
our view, the cognitive processes supporting reap- storage of information held in working memory
praisal, as well as the emotional processes sup- (Cabeza & Nyberg, 2000; Culham & Kanwisher,
porting context-sensitive evaluation, may both ex- 2001; Smith & Jonides, 1999). Future research
ert regulatory effects, albeit in different ways. may help determine which account is correct.
Whereas the evaluation processes supported by
OFC may support the selection of appropriate, The Nature of Emotion Regulation
and the transient suppression of inappropriate, af- The present study provides insight into the
fective responses, the reappraisal processes sup- processes supporting reappraisal. However, a
ported by lateral and medial prefrontal regions number of additional steps will be necessary to
may be important for modulating these evaluation develop a more complete framework for under-
processes themselves. By down-regulating multi- standing the cognitive and affective mechanisms
ple types of evaluation processes, reappraisal may of emotion and emotion regulation more generally
shift from an emotional to an unemotional mode (Ochsner & Feldman Barrett, 2001). In particular,
of stimulus analysis. the present research raises at least two important
Further work will also be needed to determine questions about the nature of emotion regulation.
exactly how prefrontal regions modulate the The first question concerns the way in which
amygdala and MOFC during reappraisal. In the emotion processing might be modulated differently
present study, we observed an inverse correlation by cognitive reappraisal as compared to other
between lateral prefrontal and amygdala activa- forms of emotion regulation. A pair of studies
tion during reappraisal, although these two struc- have examined attentional influences on emotion
tures share few direct connections. One route by processing and found that enhanced amygdala
which the LPFC could influence the amygdala is responses to fearful faces did not change as atten-
via the MOFC, which has reciprocal connections tion to a fear stimulus decreased (Anderson et al.,
with both regions (Cavada et al., 2000). By di- 2001; Vuilleumier, Armony, Driver, & Dolen, 2001).
rectly modulating representations of the affective These results contrast with the present finding that
significance of a stimulus in the MOFC, activa- attempts to cognitively transform feelings can
tion in the LPFC could blunt processing in the modulate amygdala activity. A handful of other
amygdala indirectly. This seems somewhat un- studies have examined the influence on amygdala
likely, however, because the correlation between processing of cognitive judgments that involve
the LPFC and the MOFC activity was not as explicit evaluation of the emotional properties
strong as the correlation between LPFC and of faces as compared to evaluation of stimulus
RT0996_C19.qxd 11/9/04 4:11 PM Page 264
dimensions unrelated to emotion, such as gender strategies that do not share this interpretive verbal
or age. Results have been mixed, with some studies component (e.g., those involving attentional de-
finding that evaluative judgments diminish amyg- ployment, as discussed above, or the suppression
dala activation (Hariri, Bookheimer, & Mazziotta, of expressive emotional behavior; Gross, 1998), it
2000; Liberzon et al., 2000) and others finding the is possible that activation of the right prefrontal
opposite (Winston, Strange, O’Doherty, & Dolan, systems would be observed. This hypothesis is
2002; Critchley et al., 2000). Although the precise supported by a study showing that regulating
relevance of these judgments to emotion regula- responses to a sexually arousing film clip by
tion is not clear, some of the discrepant findings viewing them from a detached third-person per-
could be attributable to a differential dependence spective activated right PFC and deactivated
of some evaluative judgments on processes in- structures related to sexual arousal, including the
volved in reappraisal. More generally, the present hypothalamus and the amygdala (Beauregard,
results may be difficult to directly relate to these Levesque, & Bourgouin, 2001). In this context,
studies because of the differences in the stimuli the deactivation of the right amygdala and the left
employed and the responses they evoke. Whereas MOFC may reflect stimulus- (as compared to
the present study used stimuli that elicit relatively verbally) driven processing of affective informa-
strong responses that induce changes in emotional tion by the amygdala (Phelps et al., 2001; Morris
experience, the words and faces employed in et al., 1999), and the observation that the right
other studies elicit weaker responses overall and amygdala and the left orbitofrontal cortex activity
only rarely alter experience (Ochsner & Feldman may be coupled during sensory processing of
Barrett, 2001; Davidson & Irwin, 1999). Future aversive stimuli (Zald & Pardo, 1997).
work may serve to clarify the precise ways in which A second possible explanation for lateralized
different types of regulation modulate different activations relates to findings associating negative
aspects of emotion (Ochsner & Feldman Barrett, affect with the right hemisphere and positive affect
2001; cf. Gray, in press). with the left hemisphere (e.g., Canli, Desmond,
A second question concerns the lateralization Zhao, Glover, & Gabrieli, 1998; see Davidson &
of activations and deactivations related to reap- Irwin, 1999, for a review). Thus, right amygdala
praisal and emotion processing, respectively. One deactivation could reflect down-regulation of sys-
possibility is that these findings are related to tems that generate negative appraisals whereas
properties of the particular regulatory processes left PFC activation could reflect engagement of
involved in the present study. Left lateralization of systems supporting positivizing reappraisals. This
reappraisal-related prefrontal activations may, for interpretation is consistent with the finding that
example, reflect a common verbal component of relatively greater resting activation of the left than
reappraisal strategies employed by participants, the right PFC is correlated with resistance to
who typically reported mentally talking them- depression, which may in turn reflect baseline dif-
selves through their reappraisals.3 Left prefrontal ferences in the ability to represent cognitive
regions have been implicated in interference tasks control strategies used to down-regulate emotion
involving verbal stimuli (e.g., Bunge, Ochsner, processing (Davidson, Putnam, & Larson, 2000).
Desmond, Glover, & Gabrieli, 2001; Macdonald, Consistent with this view, studies of resting brain
Cohen, Stenger, & Carter, 2000; D’Esposito, Postle, metabolism in individuals with depression or
Jonides, & Smith, 1999; Jonides, Smith, Marshuetz, obsessive–compulsive disorder—who may be
Koeppe, & Reuter-Lorenz, 1998), which suggests unable to effectively represent these strategies—
that this region may represent verbal reappraisal have shown hypoactivation of the prefrontal regions
strategies and help resolve interference with com- coupled with hyperactivation of the amygdala
peting negative evaluations generated by the percep- and/or the orbito-frontal cortex that normalizes
tion of aversive stimuli. Were one to examine other with effective treatment (Davidson et al., 2000;
types of reappraisal or other emotion-regulation Brody et al., 1999; Saxena et al., 1999).
RT0996_C19.qxd 11/9/04 4:11 PM Page 265
Conclusions screen for an additional 4 sec with an instruction

The aim of the present study was to use informa- either to ATTEND or REAPPRAISE replacing
tion about brain function to draw inferences about the instruction to VIEW. On Attend trials, either a
the mechanisms supporting one type of cognitive negative or a neutral photo was shown and partic-
control of emotion. As such, it represents one ipants were instructed to attend to and be aware
example of a growing trend towards using neuro- of, but not to try to alter, any feelings elicited by it.
science methods to address questions that tradi- On Reappraise trials, a negative photo was shown
tionally have been of interest to social and person- and participants were instructed to reinterpret the
ality psychologists (Ochsner & Lieberman, photo so that it no longer elicited a negative
2001). Although findings suggest that cognitive response. The 4-sec epoch during which partici-
reappraisal can modulate multiple types of emotion pants were attending or reappraising negative
processing, questions remain about the functional photos is the subject of the functional imaging
significance of observed frontal and amygdala analyses reported in the present study. The photo
activations, their relation to other forms of regula- then disappeared and, for 3.1 sec, participants
tion, and their relevance to clinical populations. could continue attending to, or reappraising, any
As future work addresses these questions, we may feelings that lingered after its presentation. A
be able to better connect Hamlet’s timeless obser- four-point scale (1 “weak” to 4 “strong”) for
vation that thinking can make things good or bad rating the strength of current negative affect then
with an increased understanding of how the brain was presented for 3 sec, and participants indicated
makes this possible. how they felt currently. Finally, an instruction to
RELAX appeared in the center of the screen for
5 sec. A 900-msec interval separated each trial.
Methods
Testing Procedure
Participants
One to three days before scanning, participants
Fifteen healthy right-handed female volunteers4 received extensive instruction in reappraisal. Pilot
recruited from Stanford University and the sur- testing suggested reappraisal was commonly ac-
rounding community (ages 18–30, M 21.9) complished by generating an interpretation of, or
gave informed consent and were paid $50 for their a story about, each photo that would explain
participation. apparently negative events in a less negative way
(e.g., women depicted crying outside of a church
Task could be described as attending a wedding instead
On the basis of normative ratings, two sets of of a funeral). No single type of reinterpretation
38 negative color photos and one set of 38 neutral was universally applicable to all photos, which
color photos were selected from the International was expected given that individuals must generate
Affective Picture System (Lang, Greenwald, context-appropriate reappraisals in everyday life.
Bradley, & Hamm, 1993). The task design was To strike a balance between generalizability and
adapted from Jackson et al., (2000). At the begin- experimental control, we instructed participants
ning of each trial, a photo (subtending approxi- to select the reinterpretation that was most effec-
mately 20 20º of visual angle) appeared in the tive for each photo. Training began by asking par-
center of a black screen for 4 sec with the instruc- ticipants to spontaneously generate reappraisals
tion VIEW printed in white underneath. Many of sample photos. After appropriate coaching and
photos depicted complex scenes, and during this shaping by the researcher to ensure that partici-
viewing period participants were instructed to pants could reinterpret photos quickly and effec-
view the photo, understand its content, and allow tively, the training ended with the completion of
themselves to experience/feel any emotional re- 18 practice trials. It was stressed that when asked
sponse it might elicit. The photo remained on the to reappraise, participants should neither look
RT0996_C19.qxd 11/9/04 4:11 PM Page 266
away (unless necessary; no subjects reported that half maximum). Low-frequency noise and differ-
it was) nor distract themselves with irrelevant ences in global signal between participants were
and/or positive thoughts. removed. Single participants’ data were analyzed
During scanning, participants completed one with a fixed-effects model (Friston, Jezzard, &
hundred and fourteen 20-sec trials over six sepa- Turner, 1994) and group data were analyzed using
rate scans. Each scan included approximately a random-effects model (Holmes & Friston,
equal numbers of each trial type, and trial order 1998). Effects were modeled using a box-car con-
was counterbalanced across scans so that every volved with a canonical hemodynamic response
trial type followed every other with equal proba- function for the 4-sec trial epoch during which
bility. Assignment of photos to trial types and participants reappraised or attended while a photo
scans was counterbalanced across participants. was on the screen. An anatomically defined gray
Psyscope was used to control stimulus presenta- matter mask was created and explicitly specified
tion and response collection. Upon completion of during analysis. This ensured that statistical
scanning, participants viewed all the negative analysis was performed in all brain regions, in-
photos they had seen in the scanner and indicated cluding those where signal may be low due to
the strength of their initial negative reaction to susceptibility artifacts. For the group analysis,
each one (i.e., during the viewing period, before functional images were averaged to create a single
attending, or reappraising). These ratings were image of mean activation per trial type and partic-
used to identify the trials that involved the photos ipant. To identify regions recruited across partici-
rated most negatively by each participant. To ver- pants that wee activated or relatively deactivated
ify that participants had, in fact, reappraised, for by reappraisal, one-sample t tests were performed
each photo they were asked to indicate whether on these average images to create a series of
they had generated an alternative interpretation or SPM{Z} maps depicting differences in brain acti-
whether they had used some other types of reap- vation between trial types. To identify regions for
praisal strategy. which the level of reappraisal-related activation
across participants was correlated with the reap-
Data Acquisition praisal-related decreases in negative affect, a sim-
Whole-brain imaging data were acquired on a 3-T ple regression analysis was performed on the
MRI Signa LX Horizon Echospeed scanner average images for the Reappraise Attend con-
(GE Medical Systems, 8.3_m4 systems revision). trast. Except as noted below, for group contrasts
T2-weighted flow-compensated spin-echo anatom- and regression analysis, a voxel-level threshold of
ical images (TR, 2000 msec; TE, 85 msec) were p .001 uncorrected for multiple comparisons
acquired in 16 contiguous 7-mm axial slices. (t 3.09) was used. An extent threshold of five
Functional images were acquired with the same contiguous voxels was applied to activated clusters
slice prescription using a T2*-sensitive gradient- meeting the voxel-level threshold. Maxima are
echo spiral pulse sequence (Glover & Lai, 1998) reported in MNI305 coordinates, as in SPM99.
(TE, 30 msec; TR, 1000 msec; two interleaves; To determine whether reappraisal modulated
flip angle 60º, field of view, 24 cm; 64 64 data the amygdala’s response to negative photos, struc-
acquisition matrix). turally defined ROIs were drawn around each par-
ticipant’s amygdalae on their in-plane anatomical
Data Analysis images (Desmond & Lim, 1997). Parameter esti-
Functional images were motion-corrected and mates (that model the amplitude of the fMRI
normalized to a standard template brain using response) averaged across all voxels for each ROI
SPM99 (Wellcome Department of Cognitive were then extracted for Reappraise and Attend tri-
Neurology). Normalized images were interpo- als on which the most negative photos had been
lated to 2 2 4-mm voxels and spatially presented. For comparison, parameter estimates
smoothed with a Gaussian filter (6 mm full width on Attend neutral trials also were extracted.
RT0996_C19.qxd 11/9/04 4:11 PM Page 267
Planned t tests (α .05, one-tailed) were used to MOFC. In some circumstances, when a UCS no longer fol-
lows a CS, these appraisal processes might be used to try
compare amygdala activation across these three
and figure out what action to take next. This decision
trial types. This analysis also was performed for process might transiently suppress a CR, which research
functionally defined ROIs in regions of a priori has shown can spontaneously reemerge at a later point. Sec-
interest in the LPFC and MOFC. ond, prior work has shown that OFC is involved in modify-
ing associations established in the amygdala (e.g., Rolls,
1999). This modification does not always involve down-
regulation of amygdala activity, however, and in many
Acknowledgments cases may involve coactivation of the two structures during
emotional learning and evaluation (e.g., Schoenbaum,
This research was supported by NSF grant BCD- Chiba, & Gallagher, 1998). In the case of extinction, OFC
0084496, McDonnel-Pew grant 98–23, and grant activity could also reflect the use of appraisal processes to
5 F32 MH11990–03 from the National Institute of encode new properties of the stimulus–context relationship.
Health. The authors thank Adam Anderson and We would suggest, therefore, that these data, along with the
results of the present study, are consistent with the conclu-
Kalina Christoff for discussion of relevant issues
sion that reappraisal may down-regulate two processing
and comments on earlier drafts of this reading. components of a system important for analyzing different
The data reported in this experiment have been kinds of emotion features. Under certain circumstances,
deposited in The fMRI Data Center (http:// these two emotion-related processors may themselves con-
www.fmridc.org). The accession number is figure to regulate responses. In the present circumstances,
regulatory effects are achieved by shutting them both down.
2–2002–1137G.
Ultimately, brain systems might best be thought of as per-
forming task non-specific computations that may play a
part in different types of behavior, whether or not that be-
NOTES havior is best described as regulatory.
1. To provide an independent check for bias in participants’ 3. In this regard, it is worth noting that variability introduced
subjective reports, and the affect ratings in particular, in a by the fact that we did not constrain participants to reap-
pre-scan session participants were asked to complete the praise photos in exactly the same way (they were free to
Marlowe–Crowne social desirability scale, which is com- implement a common cognitive reframing strategy in a
monly used as a measure of the tendency to provide re- whatever manner was appropriate for each photo) could
sponses that would be demanded in an experiment. Overall, provide a conservative estimate of reappraisal-related pre-
scores on this scale were low (M 13.125) and were uncor- frontal activations.
related (r .099, p .80) with reappraisal success (as in- 4. Only women were studied because women often exhibit
dexed by the drop in negative affect on reappraise as com- stronger emotional responses than men (Kring & Gordon,
pared to attend trials). Furthermore, if affect ratings reflected 1998), and prior research from our laboratory and others
compliance with experimental demand, then ratings might suggested that women respond more strongly and more re-
have been expected to drop on reappraise trials to the level of liably to the emotional stimuli used in this study (Ito, Ca-
affect reported on aware trials with neutral photos. This did cioppo, & Lang, 1998).
not occur. In fact, ratings on reappraise trials remained sensi-
tive to differences in intensity across photos: ratings on reap-
REFERENCES
praise trials with most negative photos were greater than on
reappraise trials for least negative images, which in turn, Adolphs, R. (2001). The neurobiology of social cognition.
were still greater than rating on aware neutral trials. Current Opinion in Neurobiology, 11, 231–239.
2. Two points concerning the MOFC’s role in regulation are Alexander, G. E., Delong, M., & Strick, P. L. (1996). Parallel
relevant here. First, recent studies suggest that the relation organization of functionally segregated circuits linking
of MOFC to extinction is not yet clear. Some animal lesion basal ganglia and cortex. Annual Review of Neuroscience, 9,
studies indicate that medial lesions impair extinction (Mor- 357–381.
gan et al., 1995), others suggest that they do not (Gewirtz, Anderson, A. K., Panitz, D. A., Ochsner, K. N., Bunge, S. A.,
Falls, & Davis, 1997), whereas others suggest that the im- Christoff, K., & Gabrieli, J. D. E. (2001). Examination of at-
pairments may be observed only during some phases of ex- tentional modulation of neural responses in affective and
tinction (Quirk, Russo, Barron, & Lebron, 2000). Although neural object processing domains. Society for Neuroscience
the reasons for these discrepancies await clarification, one Abstracts, 120.2.
possible contributing factor may be that subtle differences Anderson, A. K., & Phelps, E. A. (2001). Lesions of the hu-
in learning contexts lead to differential dependence on man amygdala impair enhanced perception of emotionally
context-sensitive appraisal processes associated with salient events. Nature, 411, 305–309.
RT0996_C19.qxd 11/9/04 4:11 PM Page 268
Barcelo, F., & Knight, R. T. (2002). Both random and perse- sociates with later memory for individual emotional experi-
verative errors underlie WCST deficits in prefrontal pa- ence. Journal of Neuroscience, 20, RC99.
tients. Neuropsychologia, 40, 349–356. Carter, C. S., Macdonald, A. M., Botvinick, M., Ross, L. L.,
Barcelo, F., Suwazono, S., & Knight, R. T. (2000). Prefrontal Stenger, V. A., Noll, D., & Cohen, J. D. (2000). Parsing ex-
modulation of visual processing in humans. Nature Neuro- ecutive processes: Strategic vs. evaluative functions of the
science, 3, 399–403. anterior cingulate cortex. Proceedings of the National Acad-
Barch, D. M., Braver, T. S., Akbudak, E., Conturo, T., emy of Sciences, U.S.A., 97, 1944–1948.
Ollinger, J., & Snyder, A. (2001). Anterior cingulate cortex Cavada, C., Company, T., Tejedor, J., Cruz-Rizzolo, R. J., &
and response conflict: Effects of response modality and pro- Reinoso-Suarez, F. (2000). The anatomical connections of
cessing domain. Cerebral Cortex, 11, 837–848. the macaque monkey orbitofrontal cortex. A review. Cere-
Barch, D. M., Braver, T. S., Nystrom, L. E., Forman, S. D., bral Cortex, 10, 220–242.
Noll, D. C., & Cohen, J. D. (1997). Dissociating working Chua, P., Krams, M., Toni, I., Passingham, R., & Dolan,
memory from task difficulty in human prefrontal cortex. R. (1999). A functional anatomy of anticipatory anxiety.
Neuropsychologia, 35, 1373–1380. Neuroimage, 9, 563–571.
Beauregard, M., Leroux, J. M., Bergman, S., Arzoumanian, Courtney, S. M., Petit, L., Maisog, J. M., Ungerleider, L. G., &
Y., Beaudoin, G., Bourgouin, P., & Stip, E. (1998). The Haxby, J. V. (1998). An area specialized for spatial working
functional neuroanatomy of major depression: An fMRI memory in human frontal cortex. Science, 279, 1347–1351.
study using an emotional activation paradigm. NeuroRe- Critchley, H., Daly, E., Phillips, M., Brammer, M., Bullmore,
port, 9, 3253–3258. E., Williams, S., Van Amelsvoort, T., Robertson, D., David,
Beauregard, M., Levesque, J., & Bourgouin, P. (2001). Neural A., & Murphy, D. (2000). Explicit and implicit neural
correlates of conscious self-regulation of emotion. Journal mechanisms for processing of social information from fa-
of Neuroscience, 21, RC165. cial expressions: A functional magnetic resonance imaging
Bechara, A., Damasio, H., & Damasio, A. R. (2000). Emotion, study. Human Brain Mapping, 9, 93–105.
decision making and the orbitofrontal cortex. Cerebral Cor- Culham, J. C., & Kanwisher, N. G. (2001). Neuroimaging of
tex, 10, 295–307. cognitive functions in human parietal cortex. Current Opin-
Bechara, A., Damasio, H., Damasio, A. R., & Lee, G. P. ion in Neurobiology, 11, 157–163.
(1999). Different contributions of the human amygdala and Davidson, R. J., & Irwin, W. (1999). The functional neu-
ventromedial prefrontal cortex to decision-making. Journal roanatomy of emotion and affective style. Trends in Cogni-
of Neuroscience, 19, 5473–5481. tive Sciences, 3, 11–21.
Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S., & Davidson, R. J., Putnam, K. M., & Larson, C. L. (2000). Dys-
Cohen, J. D. (2001). Conflict monitoring and cognitive con- function in the neural circuitry of emotion regulation—a
trol. Psychology Review, 108, 624–652. possible prelude to violence. Science, 289, 591–594.
Brody, A. L., Saxena, S., Silverman, D. H., Alborzian, S., Fair- de Fockert, J. W., Rees, G., Frith, C. D., & Lavie, N. (2001).
banks, L. A., Phelps, M. E., Huang, S. C., Wu, H. M., Maid- The role of working memory in visual selective attention.
ment, K., & Baxter, L. R., Jr. (1999). Brain metabolic Science, 291, 1803–1806.
changes in major depressive disorder from pre- to post- Desmond, J. E., & Lim, K. O. (1997). On- and offline Ta-
treatment with paroxetine. Psychiatry Research, 91, lairach registration for structural and functional MRI stud-
127–139. ies. Human Brain Mapping, 5, 58–73.
Bunge, S. A., Ochsner, K. N., Desmond, J. E., Glover, G. H., D’Esposito, M., Postle, B. R., Jonides, J., & Smith, E. E.
& Gabrieli, J. D. E. (2001). Prefrontal regions involved in (1999). The neural substrate and temporal dynamics of in-
keeping information in and out of mind. Brain, 124, terference effects in working memory as revealed by event-
2074–2086. related functional MRI. Proceedings of the National Acad-
Bush, G., Luu, P., & Posner, M. I. (2000). Cognitive and emo- emy of Sciences, U.S.A., 96, 7514–7519.
tional influences in anterior cingulate cortex. Trends in Cog- Elliott, R., Frith, C. D., & Dolan, R. J. (1997). Differential
nitive Sciences, 4, 215–222. neural response to positive and negative feedback in plan-
Cabeza, R., & Nyberg, L. (2000). Imaging cognition: II. An ning and guessing tasks. Neuropsychologia, 35.
empirical review of 275 PET and fMRI studies. Journal of Friston, K. J., Jezzard, P., & Turner, R. (1994). Analysis of
Cognitive Neuroscience, 12, 1–47. functional MRI time-series. Human Brain Mapping, 1,
Cahill, L., Babinsky, R., Markowitsch, H. J., & McGaugh, 153–171.
J. L. (1995). The amygdala and emotional memory. Nature, Gallagher, H. L., Happe, F., Brunswick, N., Fletcher, P. C.,
377, 295–296. Frith, U., & Frith, C. D. (2000). Reading the mind in car-
Canli, T., Desmond, J. E., Zhao, Z., Glover, G., & Gabrieli, toons and stories: An fMRI study of ‘theory of mind’ in ver-
J. D. (1998). Hemispheric asymmetry for emotional stimuli bal and nonverbal tasks. Neuropsychologia, 38, 11–21.
detected with fMRI. NeuroReport, 9, 3233–3239. Gewirtz, J. C., Falls, W. A., & Davis, M. (1997). Normal con-
Canli, T., Zhao, Z., Brewer, J., Gabrieli, J. D. E., & Cahill, L. ditioned inhibition and extinction of freezing and fear-
(2000). Event-related activation in the human amygdala as- potentiated startle following electrolytic lesions of medial
RT0996_C19.qxd 11/9/04 4:11 PM Page 269
prefrontal cortex in rats. Behavioral Neuroscience, 111, Knight, R. T., Staines, W. R., Swick, D., & Chao, L. L. (1999).
712–726. Prefrontal cortex regulates inhibition and excitation in dis-
Glover, G. H., & Lai, S. (1998). Self-navigated spiral fMRI: tributed neural networks. Acta Psychologica, 101, 159–178.
Interleaved versus single-shot. Magnetic Resonance Imag- Kring, A. M., & Gordon, A. H. (1998). Sex differences in emo-
ing, 39, 361–368. tion: Expression, experience, and physiology. Journal of
Gross, J. J. (1998). Antecedent- and response-focused emotion Personality and Social Psychology, 74, 686–703.
regulation: Divergent consequences for experience, expres- Lane, R. (2000). Neural correlates of emotional experience. In
sion, and physiology. Journal of Personality and Social Psy- R. D. Lane & L. Nadel (Eds.), Cognitive Neuroscience of
chology, 74, 224–237. Emotion. New York: Oxford University Press.
Gross, J. J. (2002). Emotion regulation: Affective, cognitive, Lane, R. D., Fink, G. R., Chau, P. M., & Dolan, R. J. (1997).
and social consequences. Psychophysiology, 9, 281–291. Neural activation during selective attention to subjective
Gross, J. J., & John, O. P. (in press). Wise emotion regulation. emotional responses. NeuroReport, 8, 3969–3972.
In L. Feldman Barrett & P. Salovey (Eds.), The wisdom of Lane, R. D., Reiman, E. M., Ahern, G. L., Schwartz, G. E., &
feelings: Psychological processes in emotional intelligence. Davidson, R. J. (1997). Neuroanatomical correlates of hap-
New York: Guilford. piness, sadness, and disgust. American Journal of Psychia-
Gusnard, D. A., Akbudak, E., Shulman, G. L., & Raichle, try, 154, 926–933.
M. E. (2001). Medial prefrontal cortex and self-referential Lane, R. D., Reiman, E. M., Bradley, M. M., Lang, P. J., Ah-
mental activity: Relation to a default mode of brain func- ern, G. L., Davidson, R. J., & Schwartz, G. E. (1997). Neu-
tion. Proceedings of the National Academy of Sciences, roanatomical correlates of pleasant and unpleasant emotion.
U.S.A., 98, 4259–4264. Neuropsychologia, 35, 1437–1444.
Hamann, S. B., Ely, T. D., Grafton, S. T., & Kilts, C. D. (1999). Lang, P. J., Greenwald, M. K., Bradley, M. M., & Hamm,
Amygdala activity related to enhanced memory for pleasant A. O. (1993). Looking at pictures: Affective, visceral, and
and aversive stimuli. Nature Neuroscience, 2, 289–293. behavioral reactions. Psychophysiology, 30, 261–273.
Happe, F., Ehlers, S., Fletcher, P., Frith, U., Johansson, M., Lazarus, R. S. (1991). Emotion and adaptation. New York:
Gillberg, C., Dolan, R., Frackowiak, R., & Frith, C. Oxford University Press.
(1996). ‘Theory of mind’ in the brain. Evidence from a LeDoux, J. E. (2000). Emotion circuits in the brain. Annual
PET scan study of Asperger syndrome. NeuroReport, 8, Review of Neuroscience, 23, 155–184.
197–201. Liberzon I., Taylor, S. F., Fig, L. M., Decker, L. R., Koeppe, R. A.,
Hariri, A. R., Bookheimer, S. Y., & Mazziotta, J. C. (2000). & Minoshima, S. (2000). Limbic activation and psychophysio-
Modulating emotional responses: Effects of a neocortical logic responses to aversive visual stimuli. Interaction with cog-
network on the limbic system. NeuroReport, 11, 43–48. nitive task. Neuropsychopharmacology, 23, 508–516.
Holmes, A. P., & Friston, K. J. (1998). Generalisability, ran- MacDonald, A. W., Cohen, J. D., Stenger, V. A., & Carter,
dom effects and population inference. Neuroimage: Ab- C. S. (2000). Dissociating the role of the dorsolateral pre-
stracts of the Fourth International Conference on Func- frontal and anterior cingulate cortex in cognitive control.
tional Mapping of the Human Brain, 7, S754. Science, 288, 1835–1838.
Hsieh, J. C., Stone-Elander, S., & Ingvar, M. (1999). Anticipa- Miller, E. K., & Cohen, J. D. (2001). An integrative theory of
tory coping of pain expressed in the human anterior cingu- prefrontal cortex function. Annual Review of Neuroscience,
late cortex: A positron emission tomography study. Neuro- 24, 67–202.
science Letters, 262, 61–64. Morgan, M. A., & LeDoux, J. E. (1995). Differential contribu-
Ito, T. A., Cacioppo, J. T., & Lang, P. J. (1998). Eliciting affect tion of dorsal and ventral medial prefrontal cortex to the ac-
using the International Affective Picture System: Trajecto- quisition and extinction of conditioned fear in rats. Behav-
ries through evaluative space. Personality and Social Psy- ioral Neuroscience, 109, 681–688.
chology Bulletin, 24, 855–879. Morris, J. S., Ohman, A., & Dolan, R. J. (1999). A subcortical
Jackson, D. C., Malmstadt, J. R., Larson, C. L., & Davidson, pathway to the right amygdala mediating “unseen” fear.
R. J. (2000). Suppression and enhancement of emotional re- Proceedings of the National Academy of Sciences, U.S.A.,
sponses to unpleasant pictures. Psychophysiology, 37, 96, 1680–1685.
515–522. Nielsen-Bohlman, L., & Knight, R. T. (1999). Prefrontal corti-
Jonides, J., Smith, E. E., Marshuetz, C., Koeppe, R. A., & cal involvement in visual working memory. Brain Research.
Reuter-Lorenz, P. A. (1998). Inhibition in verbal working Cognitive Brain Research, 8, 299–310.
memory revealed by brain activation. Proceedings of the Ochsner, K. N., & Feldman Barrett, L. (2001). A multi-process
National Academy of Sciences, U.S.A., 95, 8410–8413. perspective on the neuroscience of emotion. In G. Bonnano
Kawasaki, H., Kaufman, O., Damasio, H., Damasio, A. R., & T. J. Mayne (Eds.), Emotion: Current issues and future
Granner, M., Bakken, H., Hori, T., Howard, M. A., III, & directions (pp. 38–81). New York: Guilford.
Adolphs, R. (2001). Single-neuron responses to emotional Ochsner, K. N., & Lieberman, M. D. (2001). The emergence
visual stimuli recorded in human ventral prefrontal cortex. of social cognitive neuroscience. American Psychologist,
Nature Neuroscience, 4, 15–16. 56, 717–734.
RT0996_C19.qxd 11/9/04 4:11 PM Page 270
Ochsner, K. N., & Schacter, D. L. (in press). Remembering Saxena, S., Brody, A. L., Maidment, K. M., Dunkin, J. J.,
emotional events: A social cognitive neuroscience ap- Colgan, M., Alborzian, S., Phelps, M. E., & Baxter, L. R., Jr.
proach. In R. J. Davidson, H. Goldsmith, & K. R. Scherer (1999). Localized orbitofrontal and subcortical metabolic
(Eds.), Handbook of the affective sciences. Oxford Univer- changes and predictors of response to paroxetine treatment
sity Press: New York. in obsessive–compulsive disorder. Neuropsycho-pharma-
O’Doherty, J. O., Kringelbach, M. L., Rolls, E. T., Hornak, J., cology, 21, 683–693.
& Andrews, C. (2001). Abstract reward and punishment Scherer, K. R., Schorr, A., & Johnstone, T. (2001). Ap-
representations in the human orbitofrontal cortex. Nature praisal processes in emotion. New York: Oxford Univer-
Neuroscience, 4, 95–102. sity Press.
Paradiso, S., Jhonson, D. L., Andreasen, N. C., O’Leary, D. S., Schoenbaum, G., Chiba, A. A., & Gallagher, M. (1998). Or-
Watkins, G. L., Ponto, L. L., & Hichwa, R. D. (1999). Cere- bitofrontal cortex and basolateral amygdala encode ex-
bral blood flow changes associated with attribution of emo- pected outcomes during learning. Nature Neuroscience, 1,
tional valence to pleasant, unpleasant, and neutral visual 155–159.
stimuli in a PET study of normal subjects. American Jour- Shakespeare, W. (1998/1623). The Oxford Shakespeare: Ham-
nal of Psychiatry, 156, 1618–1629. let. New York: Oxford University Press.
Peterson, B. S., Skudlarski, P., Gatenby, J. C., Zhang, H., Simpson, J. R., Ongur, D., Akbudak, E., Conturo, T. E.,
Anderson, A. W., & Gore, J. C. (1999). An fMRI study of Ollinger, J. M., Snyder, A. Z., Gusnard, D. A., & Raichle,
Stroop word–color interference: Evidence for cingulate M. E. (2000). The emotional modulation of cognitive pro-
subregions subserving multiple distributed attentional sys- cessing: An fMRI study. Journal of Cognitive Neuro-
tems. Biological Psychiatry, 45, 1237–1258. science, 12, 157–170.
Petit, L., Courtney, S. M., Ungerleider, L. G., & Haxby, J. V. Smith, E., & Jonides, J. (1999). Storage and executive
(1998). Sustained activity in the medial wall during working processes in the frontal lobes. Science, 283, 657–660.
memory delays. Journal of Neuroscience, 18, 9429–9437. Stuss, D. T., Eskes, G. A., & Foster, J. K. (1994). Experimen-
Phelps, E. A., Hyder, F., Blamire, A. M., & Shulman, R. G. tal neuropsychological studies of frontal lobe functions. In
(1997). FMRI of the prefrontal cortex during overt verbal F. Boller & J. Grafman (Eds.), Handbook of neuropsychol-
fluency. NeuroReport, 8, 561–565. ogy. Amsterdam: Elsevier.
Phelps, E. A., O’Connor, K. J., Gatenby, J. C., Gore, J. C., van Veen, V., Cohen, J. D., Botvinick, M. M., Stenger, V. A., &
Grillon, C., & Davis, M. (2001). Activation of the left Carter, C. S. (2001). Anterior cingulate cortex, conflict
amygdala to a cognitive representation of fear. Nature Neu- monitoring, and levels of processing. Neuroimage, 14,
roscience, 4, 437–441. 1302–1308.
Ploghaus, A., Tracey, I., Gati, J. S., Clare, S., Menon, R. S., Vuilleumier, P., Armony, J. L., Driver, J., & Dolan, R. J.
Matthews, P. M., & Rawlins, J. N. (1999). Dissociating pain (2001). Effects of attention and emotion on face processing
from its anticipation in the human brain. Science, 284, in the human brain: An event-related fMRI study. Neuron,
1979–1981. 30, 829–841.
Quirk, G. J., Russo, G. K., Barron, J. L., & Lebron, K. (2000). Whalen, P. J., Rauch, S. L., Etcoff, N. L., McInerney, S. C.,
The role of ventromedial prefrontal cortex in the recovery of Lee, M. B., & Jenike, M. A. (1998). Masked presentations
extinguished fear. Journal of Neuroscience, 20, 6225–6231. of emotional facial expressions modulate amygdala activity
Reiman, E. M., Lane, R. D., Ahern, G. L., Schwartz, G. E., without explicit knowledge. Journal of Neuroscience, 181,
Davidson, R. J., Friston, K. J., Yun, L. S., & Chen, K. 411–418.
(1997). Neuroanatomical correlates of externally and inter- Winston, J. S., Strange, B. A., O’Doherty, J., & Dolan, R. J.
nally generated human emotion. American Journal of Psy- (2002). Automatic and intentional brain responses during
chiatry, 154, 918–925. evaluation of trustworthiness of faces. Nature Neuro-
Richards, J. M., & Gross, J. J. (2000). Emotion regulation and science, 5, 277–283.
memory: The cognitive costs of keeping one’s cool. Journal Zald, D. H., & Pardo, J. V. (1997). Emotion, olfaction, and the
of Personality and Social Psychology, 79, 410–424. human amygdala: Amygdala activation during aversive ol-
Rolls, E. T. (1999). The brain and emotion. New York: Oxford factory stimulation. Proceedings of the National Academy
University Press. of Sciences, U.S.A., 94, 4119–4124.
RT0996_A00.qxd 11/8/04 1:56 PM Page 271
A P P E N D I X
How to Read a Journal Article in

Social Psychology
•
Christian H. Jordan and Mark P. Zanna • University of Waterloo
When approaching a journal article for the first time, and often on subsequent occasions, most people
try to digest it as they would any piece of prose. They start at the beginning and read word for word,
until eventually they arrive at the end, perhaps a little bewildered, but with a vague sense of relief. This
is not an altogether terrible strategy; journal articles do have a logical structure that lends itself to this
sort of reading. There are, however, more efficient approaches—approaches that enable you, a student
of social psychology, to cut through peripheral details, avoid sophisticated statistics with which you
may not be familiar, and focus on the central ideas in an article. Arming yourself with a little
foreknowledge of what is contained in journal articles, as well as some practical advice on how to read
them, should help you read journal articles more effectively. If this sounds tempting, read on.
Jmentournal articles offer a window into the inner

workings of social psychology. They docu-
how social psychologists formulate hy-
its results have been shared with others, col-
leagues and students alike. Journal articles are a
primary means of communicating research find-
potheses, design empirical studies, analyze the ings. As such, they can be genuinely exciting
observations they collect, and interpret their re- and interesting to read.
sults. Journal articles also serve an invaluable That last claim may have caught you off guard.
archival function: they contain the full store of For beginning readers, journal articles may seem
common and cumulative knowledge of social anything but interesting and exciting. They may,
psychology. The documentation of past research on the contrary, appear daunting and esoteric,
allows researchers to build on past findings and laden with jargon and obscured by menacing sta-
advance our understanding of social behavior, tistics. Recognizing this fact, we hope to arm you,
without pursuing avenues of investigation that through this essay, with the basic information you
have already been explored. Perhaps most im- will need to read journal articles with a greater
portant, a research study is never complete until sense of comfort and perspective.
271
RT0996_A00.qxd 11/8/04 1:56 PM Page 272
Social psychologists study many fascinating readers. The format of review and theoretical arti-
topics, ranging from prejudice and discrimina- cles is less standardized, and more like that of
tion, to culture, persuasion, liking and love, con- textbooks and other scholarly writings, with
formity and obedience, aggression, and the self. which most readers are familiar. This is not to
In our daily lives, these are issues we often strug- suggest that such articles are easier to read and
gle to understand. Social psychologists present comprehend than research reports; they can be
systematic observations of, as well as a wealth of quite challenging indeed. It is simply the case
ideas about, such issues in journal articles. It that, because more rules apply to the writing of
would be a shame if the fascination and intrigue research reports, more guidelines can be offered
these topics have were lost in their translation into on how to read them.
journal publications. We don’t think they are, and
by the end of this essay we hope you won’t either.
Journal articles come in a variety of forms, in- The Anatomy of Research Reports
cluding research reports, review articles, and
theoretical articles. Put briefly, a research report is Most research reports in social psychology, and in
a formal presentation of an original research study, psychology in general, are written in a standard
or series of studies. A review article is an evalua- format prescribed by the American Psychological
tive survey of previously published work, usually Association (1994). This is a great boon to both
organized by a guiding theory or point of view. readers and writers. It allows writers to present
The author of a review article summarizes previ- their ideas and findings in a clear, systematic
ous investigations of a circumscribed problem, manner. Consequently, as a reader, once you
comments on what progress has been made toward understand this format, you will not be on com-
its resolution, and suggests areas of the problem pletely foreign ground when you approach a new
that require further study. A theoretical article also research report—regardless of its specific content.
evaluates past research but focuses on the develop- You will know where in the paper particular
ment of theories used to explain empirical find- information is found, making it easier to locate.
ings. Here, the author may present a new theory to No matter what your reasons for reading a re-
explain a set of findings, or may compare and con- search report, a firm understanding of the format
trast a set of competing theories, suggesting why in which it is written will ease your task. We dis-
cuss the format of research reports next, with
one theory might be the superior one.
some practical suggestions on how to read them.
This essay focuses primarily on how to read re-
Later, we discuss how this format reflects the
search reports, for several reasons. First, the bulk
process of scientific investigation, illustrating
of published literature in social psychology con-
how research reports have a coherent narrative
sists of research reports. Second, the summaries
structure.
presented in review articles, and the ideas set forth
in theoretical articles, are built on findings pre-
sented in research reports. To get a deep under- Title and Abstract
standing of how research is done in social Although you can’t judge a book by its cover, you
psychology, fluency in reading original research can learn a lot about a research report simply by
reports is essential. Moreover, theoretical articles reading its title. The title presents a concise state-
frequently report new studies that pit one theory ment of the theoretical issues investigated, and/or
against another, or test a novel prediction derived the variables that were studied. For example, the
from a new theory. In order to appraise the valid- following title was taken almost at random from a
ity of such theoretical contentions, a grounded prestigious journal in social psychology: “Sad and
understanding of basic findings is invaluable. guilty? Affective influences on the explanation of
Finally, most research reports are written in a conflict in close relationships” (Forgas, 1994,
standard format that is likely unfamiliar to new p. 56). Just by reading the title, it can be inferred
RT0996_A00.qxd 11/8/04 1:56 PM Page 273
Appendix ■ 273
that the study investigated how emotional states (e.g., why they chose an experiment or a correla-
change the way people explain conflict in close re- tional study).
lationships. It also suggests that when feeling sad, The introduction generally begins with a broad
people accept more personal blame for such con- consideration of the problem investigated. Here,
flicts (i.e., feel more guilty). the researchers want to illustrate that the problem
The abstract is also an invaluable source of they studied is a real problem about which people
information. It is a brief synopsis of the study, and should care. If the researchers are studying prej-
packs a lot of information into 150 words or less. udice, they may cite statistics that suggest
The abstract contains information about the prob- discrimination is prevalent, or describe specific
lem that was investigated, how it was investigated, cases of discrimination. Such information helps
the major findings of the study, and hints at the illustrate why the research is both practically and
theoretical and practical implications of the find- theoretically meaningful, and why you should
ings. Thus, the abstract is a useful summary of the bother reading about it. Such discussions are
research that provides the gist of the investigation. often quite interesting and useful. They can help
Reading this outline first can be very helpful, be- you decide for yourself if the research has merit.
cause it tells you where the report is going, and But they may not be essential for understanding
gives you a useful framework for organizing the study at hand. Read the introduction carefully,
information contained in the article. but choose judiciously what to focus on and
The title and abstract of a research report are remember. To understand a study, what you really
like a movie preview. A movie preview highlights need to understand is what the researchers’
the important aspects of a movie’s plot, and pro- hypotheses were, and how they were derived from
vides just enough information for one to decide theory, informal observation, or intuition. Other
whether to watch the whole movie. Just so with background information may be intriguing, but
titles and abstracts; they highlight the key features may not be critical to understand what the re-
of a research report to allow you to decide if you searchers did and why they did it.
want to read the whole paper. And just as with When reading the introduction, try answering
movie previews, they do not give the whole story. these questions: what problem was studied, and
Reading just the title and abstract is never enough why? How does this study relate to, and go
to fully understand a research report. beyond, past investigations of the problem? How
did the researchers derive their hypotheses? What
Introduction questions do the researchers hope to answer with
A research report has four main sections: introduc- this study?
tion, method, results, and discussion. Although it
is not explicitly labeled, the introduction begins Method
the main body of a research report. Here, the re- In the method section, the researchers translate
searchers set the stage for the study. They present their hypotheses into a set of specific, testable
the problem under investigation, and state why it questions. Here, the researchers introduce the
was important to study. By providing a brief remain characters of the study—the subjects or
view of past research and theory relevant to the participants—describing their characteristics
central issue of investigation, the researchers (gender, age, etc.) and how many of them were
place the study in an historical context and sug- involved. Then, they describe the materials (or
gest how the study advances knowledge of the apparatus), such as any questionnaires or special
problem. Beginning with broad theoretical and equipment, used in the study. Finally, they de-
practical considerations, the researchers delineate scribe chronologically the procedures of the
the rationale that led them to the specific set of study; that is, how the study was conducted.
hypotheses tested in the study. They also describe Often, an overview of the research design will
how they decided on their research strategy begin the method section. This overview provides
RT0996_A00.qxd 11/8/04 1:56 PM Page 274
a broad outline of the design, alerting you to what reading the method section, try putting yourself in
you should attend. the shoes of a participant in the study, and ask
The method is presented in great detail so that yourself if the instructions given to participants
other researchers can recreate the study to confirm seem sensible, realistic, and engaging. Imagining
(or question) its results. This degree of detail is nor- what it was like to be in the study will also help
mally not necessary to understand a study, so don’t you remember the study’s procedure, and aid you
get bogged down trying to memorize the particu- in interpreting the study’s results.
lars of the procedures. Focus on how the independ- When reading the method section, try answer-
ent variables were manipulated (or measured) and ing these questions: How were the hypotheses
how the dependent variables were measured. translated into testable questions? How were the
Measuring variables adequately is not always variables of interest manipulated and/or meas-
an easy matter. Many of the variables psycholo- ured? Did the measures used adequately reflect
gists are interested in cannot be directly observed, the variables of interest? For example, is self-re-
so they must be inferred from participants’ behav- ported income an adequate measure of social
ior. Happiness, for example, cannot be directly class? Why or why not?
observed. Thus, researchers interested in how be-
ing happy influences people’s judgments must in-
fer happiness (or its absence) from their behav- Results
ior—perhaps by asking people how happy they The results section describes how the observa-
are, and judging their degree of happiness from tions collected were analyzed to determine
their responses; perhaps by studying people’s fa- whether the original hypotheses were supported.
cial expressions for signs of happiness, such as Here, the data (observations of behavior) are de-
smiling. Think about the measures researchers scribed, and statistical tests are presented. Be-
use while reading the method section. Do they ad- cause of this, the results section is often intimidat-
equately reflect or capture the concepts they are ing to readers who have little or no training in
meant to measure? If a measure seems odd, con- statistics. Wading through complex and unfamil-
sider carefully how the researchers justify its use. iar statistical analyses is understandably confus-
Oftentimes in social psychology, getting there ing and frustrating. As a result, many students are
is half the fun. In other words, how a result is ob- tempted to skip over reading this section. We ad-
tained can be just as interesting as the result itself. vise you not to do so. Empirical findings are the
Social psychologists often strive to have partici- foundation of any science, and results sections are
pants behave in a natural, spontaneous manner, where such findings are presented.
while controlling enough of their environment to Take heart. Even the most prestigious re-
pinpoint the causes of their behavior. Sometimes, searchers were once in your shoes and sympathize
the major contribution of a research report is its with you. Though space in psychology journals is
presentation of a novel method of investigation. limited, researchers try to strike a balance be-
When this is the case, the method will be dis- tween the need to be clear and the need to be brief
cussed in some detail in the introduction. in describing their results. In an influential paper
Participants in social psychology studies are in- on how to write good research reports, Bem
telligent and inquisitive people who are respon- (1987) offered this advice to researchers:
sive to what happens around them. Because of
No matter how technical or abstruse your article is
this, they are not always initially told the true pur- in its particulars, intelligent nonpsychologists with
pose of a study. If they were told, they might not no expertise in statistics or experimental design
act naturally. Thus, researchers frequently need to should be able to comprehend the broad outlines
be creative, presenting a credible rationale for of what you did and why. They should understand
complying with procedures, without revealing the in general terms what was learned. (p. 74)
study’s purpose. This rationale is known as a Generally speaking, social psychologists try to
cover story, and is often an elaborate scenario. When practice this advice.
RT0996_A00.qxd 11/8/04 1:56 PM Page 275
Appendix ■ 275
Most statistical analyses presented in research re- evidence that any independent variable manipula-
ports test specific hypotheses. Often, each analysis tions were effective? For example, if testing for
presented is preceded by a reminder of the hypothe- behavioral differences between happy and sad
sis it is meant to test. After an analysis is presented, participants, did the researchers demonstrate that
researchers usually provide a narrative description of one group was in fact happier than the other?
the result in plain English. When the hypothesis What were the major findings of the study? Were
tested by a statistical analysis is not explicitly stated, the researchers’ original hypotheses supported by
you can usually determine the hypothesis that was their observations? If not, look in the discussion
tested by reading this narrative description of the resection for how the researchers explain the find-
sult, and referring back to the introduction to locate ings that were obtained.
an hypothesis that corresponds to that result. After
even the most complex statistical analysis, there will Discussion
be a written description of what the result means The discussion section frequently opens with a
conceptually. Turn your attention to these descrip- summary of what the study found, and an evalua-
tions. Focus on the conceptual meaning of research tion of whether the findings supported the original
findings, not on the mechanics of how they were ob- hypotheses. Here, the researchers evaluate the
tained (unless you’re comfortable with statistics). theoretical and practical implications of their re-
Aside from statistical tests and narrative sults. This can be particularly interesting when the
descriptions of results, results sections also results did not work out exactly as the researchers
frequently contain tables and graphs. These are anticipated. When such is the case, consider the
efficient summaries of data. Even if you are not researchers’ explanations carefully, and see if they
familiar with statistics, look closely at tables and seem plausible to you. Often, researchers will also
graphs, and pay attention to the means or correla- report any aspects of their study that limit their in-
tions presented in them. Researchers always interpretation of its results, and suggest further re-
clude written descriptions of the pertinent aspects search that could overcome these limitations to
of tables and graphs. When reading these descrip- provide a better understanding of the problem un-
tions, check the tables and graphs to make sure der investigation.
what the researchers say accurately reflects their Some readers find it useful to read the first few
data. If they say there was a difference between paragraphs of the discussion section before read-
two groups on a particular dependent measure, ing any other part of a research report. Like the
look at the means in the table that correspond to abstract, these few paragraphs usually contain all
those two groups, and see if the means do differ as of the main ideas of a research report: what the
described. Occasionally, results seem to become hypotheses were, the major findings and whether
stronger in their narrative description than an they supported the original hypotheses, and how
examination of the data would warrant. the findings relate to past research and theory.
Statistics can be misused. When they are, re- Having this information before reading a research
sults are difficult to interpret. Having said this, a report can guide your reading, allowing you to fo-
lack of statistical knowledge should not make you cus on the specific details you need to complete
overly cautious while reading results sections. Al- your understanding of a study. The description of
though not a perfect antidote, journal articles un- the results, for example, will alert you to the
dergo extensive review by professional researchers major variables that were studied. If they are
before publication. Thus, most misapplications of unfamiliar to you, you can pay special attention to
statistics are caught and corrected before an article how they are defined in the introduction, and how
is published. So, if you are unfamiliar with statis- they are operationalized in the method section.
tics, you can be reasonably confident that findings After you have finished reading an article, it also
are accurately reported. can be helpful to reread the first few paragraphs of
When reading the results section, try answering the discussion and the abstract. As noted, these two
these questions: did the researchers provide passages present highly distilled summaries of the
RT0996_A00.qxd 11/8/04 1:56 PM Page 276
major ideas in a research report. Just as they can a research report, is similar in essence to the intro-
help guide your reading of a report, they can also duction of a single study report. In both cases, the re-
help you consolidate your understanding of a searchers describe the problem investigated and its
report once you have finished reading it. They pro- practical and theoretical significance. They also
vide a check on whether you have understood the demonstrate how they derived their hypotheses, and
main points of a report, and offer a succinct digest explain how their research relates to past investiga-
of the research in the authors’ own words. tions of the problem. In contrast, the separate intro-
When reading the discussion section, try an- ductions to each individual study in reports of multi-
swering these questions: What conclusions can be ple studies are usually quite brief, and focus more
drawn from the study? What new information specifically on the logic and rationale of each partic-
does the study provide about the problem under ular study presented. Such introductions generally
investigation? Does the study help resolve the describe the methods used in the particular study,
problem? What are the practical and theoretical outlining how they answer questions that have not
implications of the study’s findings? Did the re- been adequately addressed by past research, includ-
sults contradict past research findings? If so, how ing studies reported earlier in the same article.
do the researchers explain this discrepancy? General discussion sections parallel discus-
sions of single studies, except on a somewhat
grander scale. They present all of the information
Some Notes on Reports of contained in discussions of single studies, but
Multiple Studies consider the implications of all the studies pre-
sented together. A general discussion section
Up to this point, we have implicitly assumed that brings the main ideas of a research program into
a research report describes just one study. It is bold relief. It typically begins with a concise sum-
also quite common, however, for a research report mary of a research program’s main findings, their
to describe a series of studies of the same problem relation to the original hypotheses, and their prac-
in a single article. When such is the case, each tical and theoretical implications. Thus, the sum-
study reported will have the same basic structure maries that begin general discussion sections are
(introduction, method, results, and discussion sec- counterparts of the summaries that begin discus-
tions) that we have outlined, with the notable ex- sion sections of single study reports. Each pres-
ception that sometimes the results and discussion ents a digest of the research presented in an article
section for each study are combined. Combined that can serve as both an organizing framework
“results and discussion” sections contain the same (when read first), and as a check on how well you
information that separate results and discussion have understood the main points of an article
sections normally contain. Sometimes, the au- (when read last).
thors present all their results first, and only then
discuss the implications of these results, just as Research Reporting as Story Telling
they would in separate results and discussion sec-
tions. Other times, however, the authors alternate A research report tells the story of how a re-
between describing results and discussing their searcher or group of researchers investigated a
implications, as each result is presented. In either specific problem. Thus, a research report has a
case, you should be on the lookout for the same linear, narrative structure with a beginning, mid-
information, as outlined earlier in our considera- dle, and end. In his paper on writing research
tion of separate results and discussion sections. reports, Bem noted that a research report:
Reports including multiple studies also differ
. . . is shaped like an hourglass. It begins with broad
from single study reports in that they include more general statements, progressively narrows down to
general introduction and discussion sections. The the specifics of [the] study, and then broadens out
general introduction, which begins the main body of again to more general considerations. (1987, p. 175)
RT0996_A00.qxd 11/8/04 1:56 PM Page 277
Appendix ■ 277
This format roughly mirrors the process of sci- The Rest of the Story
entific investigation, wherein researchers do the
following: (1) start with a broad idea from which There is really no such thing as research. There is
only search, more search, keep on searching.
they formulate a narrower set of hypotheses, in- (Bowering, 1988, p. 95)
formed by past empirical findings (introduction);
(2) design a specific set of concrete operations to Once you have read through a research report, and
test these hypotheses (method); (3) analyze the understand the researchers’ findings and their ex-
observations collected in this way, and decide if planations of them, the story does not end there.
they support the original hypotheses (results); and There is more than one interpretation for any set
(4) explore the broader theoretical and practical of findings. Different researchers often explain
implications of the findings, and consider how the same set of facts in different ways.
they contribute to an understanding of the prob- Let’s take a moment to dispel a nasty rumor. The
lem under investigation (discussion). Though rumor is this: researchers present their studies in a
these stages are somewhat arbitrary distinc- dispassionate manner, intending only to inform
tions—research actually proceeds in a number of readers of their findings and their interpretation of
different ways—they help elucidate the inner those findings. In truth, researchers aim not only to
logic of research reports. inform readers but also to persuade them (Stern-
When reading a research report, keep this linear berg, 1995). Researchers want to convince you
structure in mind. Although it is difficult to their ideas are right. There is never only one expla-
remember a series of seemingly disjointed facts, nation for a set of findings. Certainly, some expla-
when these facts are joined together in a logical, nations are better than others; some fit the available
narrative structure, they become easier to compre- data better, are more parsimonious, or require
hend and recall. Thus, always remember that a fewer questionable assumptions. The point here is
research report tells a story. It will help you to that researchers are very passionate about their
organize the information you read, and remember ideas, and want you to believe them. It’s up to you
it later. to decide if you want to buy their ideas or not.
Describing research reports as stories is not just Let’s compare social psychologists to sales-
a convenient metaphor. Research reports are clerks. Both social psychologists and salesclerks
stories. Stories can be said to consist of two com- want to sell you something: either their ideas, or
ponents: a telling of what happened, and an expla- their wares. You need to decide if you want to buy
nation of why it happened. It is tempting to view what they’re selling or not—and there are poten-
science as an endeavor that simply catalogues tially negative consequences for either decision. If
facts, but nothing is further from the truth. The you let a salesclerk dazzle you with a sales pitch,
goal of science, social psychology included, is to without thinking about it carefully, you might end
explain facts, to explain why what happened up buying a substandard product that you don’t
happened. Social psychology is built on the really need. After having done this a few times,
dynamic interplay of discovery and justification, people tend to become cynical, steeling themselves
the dialogue between systematic observation against any and all sales pitches. This, too, is dan-
of relations and their theoretical explanation. gerous. If you are overly critical of sales pitches,
Although research reports do present novel facts you could end up foregoing genuinely useful prod-
based on systematic observation, these facts are ucts. Thus, by analogy, when you are too critical in
presented in the service of ideas. Facts in isolation your reading of research reports, you might dis-
are trivia. Facts tied together by an explanatory miss, out of hand, some genuinely useful ideas—
theory are science. Therein lies the story. To really ideas that can help shed light on why people be-
understand what researchers have to say, you need have the way they do.
to consider how their explanations relate to their This discussion raises the important question
findings. of how critical one should be while reading a
RT0996_A00.qxd 11/8/04 1:56 PM Page 278
research report. In part, this will depend on why conduct a better test of the hypotheses, trying to
one is reading the report. If you are reading it sim- rule out alternative explanations of the findings;
ply to learn what the researchers have to say about (2) it can lead one to explore the limits of the
a particular issue, for example, then there is usu- findings, to see how widely applicable they are,
ally no need to be overly critical. If you want to perhaps exploring situations to which they do not
use the research as a basis for planning a new apply; (3) it can lead one to test the implications of
study, then you should be more critical. As you the findings, furthering scientific investigation of
develop an understanding of psychological theory the phenomenon; (4) it can inspire one to apply the
and research methods, you will also develop an findings, or a novel methodology, to a different
ability to criticize research on many different lev- area of investigation; and (5) it can provoke one to
els. And any piece of research can be criticized at test the findings in the context of a specific real
some level. As Jacob Cohen put it, “A successful world problem, to see if they can shed light on it.
piece of research doesn’t conclusively settle an is- All of these are excellent extensions of the original
sue, it just makes some theoretical proposition to research, and there are, undoubtedly, other ways
some degree more likely” (1990, p. 1311). Thus, that research findings can spur new investigations.
as a consumer of research reports, you have to The problem with being too critical, too soon,
strike a delicate balance between being overly when reading research reports is that the only
critical and overly accepting. further research one may be willing to attempt is
When reading a research report, at least research of the first type: redoing a study better.
initially, try to suspend your disbelief. Try to Sometimes this is desirable, particularly in the
understand the researchers’ story; that is, try to early stages of investigating a particular issue,
understand the facts—the findings and how they when the findings are novel and perhaps unex-
were obtained—and the suggested explanation of pected. But redoing a reasonably compelling
those facts—the researchers’ interpretation of the study, without extending it in any way, does little
findings and what they mean. Take the research to to advance our understanding of human behavior.
task only after you feel you understand what the Although the new study might be “better,” it will
authors are trying to say. not be “perfect,” so it would have to be run again,
Research reports serve not only an important and again, likely never reaching a stage where it is
archival function, documenting research and its beyond criticism. At some point, researchers have
findings, but also an invaluable stimulus function. to decide that the evidence is compelling enough
They can excite other researchers to join the in- to warrant investigation of the last four types. It is
vestigation of a particular issue, or to apply new these types of studies that most advance our
methods or theory to a different, perhaps novel, knowledge of social behavior. As you read more
issue. It is this stimulus function that Elliot research reports, you will become more comfort-
Aronson, an eminent social psychologist, referred able deciding when a study is “good enough” to
to when he admitted that, in publishing a study, he move beyond it. This is a somewhat subjective
hopes his colleagues will “look at it, be stimulated judgment, and should be made carefully.
by it, be provoked by it, annoyed by it, and then When social psychologists write up a research
go ahead and do it better. . . . That’s the exciting report for publication, it is because they believe
thing about science; it progresses by people they have something new and exciting to commu-
taking off on one another’s work” (1995, p. 5). nicate about social behavior. Most research reports
Science is indeed a cumulative enterprise, and that are submitted for publication are rejected.
each new study builds on what has (or, some- Thus, the reports that are eventually published are
times, has not) gone before it. In this way, re- deemed pertinent not only by the researchers who
search articles keep social psychology vibrant. wrote them, but also by the reviewers and editors
A study can inspire new research in a number of the journals in which they are published. These
of different ways, such as: (1) it can lead one to people, at least, believe the research reports they
RT0996_A00.qxd 11/8/04 1:56 PM Page 279
Appendix ■ 279
write and publish have something important and REFERENCES

interesting to say. Sometimes, you’ll disagree; not American Psychological Association. (1994). Publication
all journal articles are created equal, after all. But manual (4th ed.). Washington, DC.
we recommend that you, at least initially, give Aronson, E. (1995). Research in social psychology as a leap
these well-meaning social psychologists the bene- of faith. In E. Aronson (Ed.), Readings about the social
animal (7th ed., pp. 3–9). New York: W. H. Freeman and
fit of the doubt. Look for what they’re excited
Company.
about. Try to understand the authors’ story, and see Bem, D. J. (1987). Writing the empirical journal article. In M. P.
where it leads you. Zanna & J. M. Darley (Eds.), The compleat academic: A prac-
tical guide for the beginning social scientist (pp. 171-201).
NOTE New York: Random House.
Bowering, G. (1988). Errata. Red Deer, Alberta.: Red Deer
Preparation of this essay was facilitated by a Natural Sciences and College Press.
Engineering Research Council of Canada doctoral fellowship to Cohen, J. (1990). Things I have learned (so far). American
Christian H. Jordan. Thanks to Roy Baumeister, Arie Kruglanski, Psychologist, 45, 1304-1312.
Ziva Kunda, John Levine, Geoff MacDonald, Richard Moreland, Forgas, J. P. (1994). Sad and guilty? Affective influences on
Ian Newby-Clark, Steve Spencer, and Adam Zanna for their in- the explanation of conflict in close relationships. Journal of
sightful comments on, and appraisals of, various drafts of this pa- Personality and Social Psychology, 66, 56-68.
per. Thanks also to Arie Kruglanski and four anonymous editors Sternberg, R. J. (1995). The psychologist’s companion: A guide
of volumes in the series, Key Readings in Social Psychology, for to scientific writing for students and researchers (3rd ed.).
their helpful critiques of an initial outline of this paper. Cambridge: Cambridge University Press.
RT0996_A00.qxd 11/8/04 1:56 PM Page 280
RT0996_Author_Index.qxd 11/8/04 2:12 PM Page 281
Author Index*
•
A Bar-On R., 225, 228, 229, 234, 235

Barrash J., 233
Abell, F., 157
Bassili, J. N., 116
Ackerly, S. S., 33 (13)
Baudena, P., 49 (29)
Adolphs, R., 4, 5, 64, 70, 71, 117, 122, 186, 199, 200, 205,
Baxter, M. G., 70
206, 260
Beauchamp, M. S., 120, 121
Aggleton, J. P., 117
Beauregard, M., 254, 264
Aguirre, G. K., 88, 94, 146
Bechara, A., 27 (17), 32 (8 9 10 11), 33 (11), 35 (9 10 11), 35
Ahlström, V., 102
(11), 37 (9), 38 (11), 49 (18), 59 (35), 71, 224, 225, 229,
Alexander, G. E., 260
234, 235, 255, 260, 262, 263
Allison, T., 76, 82, 96n25, 108, 120, 122, 123, 150,
Benson, D. F., 29 (6), 38 (6)
171, 200
Bentin, S., 76, 81, 82, 122
Amaral, D. G., 6, 7, 164, 205
Benton, A. L., 33 (13)
Amunts, K., 135
Berglas, S., 2
Anderson, A. K., 195, 205, 255, 262, 263
Berntson, G. G., 1, 2, 3
Anderson, S. W., 33 (12), 59 (34), 70, 71
Berry, D. S., 156
Angelergues, R., 94
Biederman, I., 93
Arbib, M., 150
Binkofski, F., 123, 139
Aronson, E., 2
Birbaumer N, 196
Augustine, J. R., 144
Blair, R. J., 36 (20), 64, 65, 200, 234
Blake, R., 102, 116, 117
B Blakemore, S. J., 120, 163
Bachevalier, J., 36 (22) Blanc, G., 244, 247
Bailey, P., 150 Blanz, V., 94
Baker, C. I., 119 Blum, K., 244, 248
Bandettini, P. A., 9, 10, 13 Bolton, G. E., 216, 220
Bar, M., 107 Bonda, E., 102, 119, 163, 164, 165, 205
Barbas, H., 27 (14), 117 Bonetti, S., 220n14
Barcelo, F., 254 Book, H. E., 225, 235
Barch, D. M., 254 Bookheimer, S. Y., 14
Bargh, J. A., 144 Bottini, G., 67, 70, 164
Baron-Cohen, S., 117, 122, 156, 157, 164 Botvinick, M. M., 219, 254, 262
*
Numbers in parentheses indicate reference number.
281
Bowling, S. L., 244, 247 Crowley, T. J., 247

Boyle, A. E., 247 Culham, J. C., 263
Boynton, G. M., 112 Cummings, J. L., 64
Bozarth, M. A., 247 Curtis, C., 8, 14
Braeutigam, S., 76, 81 Cutting, J. E., 102, 116
Brainard, D. H., 110
Brammer, G. L., 27 (15) D
Breiter, H. C., 186, 187, 205
Breitmeyer, B. G., 101 D’Esposito, M., 44 (7), 264
Broca, P., 22 (2) Daffner, K. R., 49 (24), 49 (28)
Brothers, L., 4, 6, 63, 64, 117, 122, 200, 205, 206, 234 Dale, A. M., 55 (18)
Brower, M. C., 64 Damasio, A. R., 4, 27 (13), 29 (1), 32 (8), 33 (2), 33 (3), 33
Bruce, V., 76, 122 (12), 35 (2), 59 (32), 59 (33), 63, 64, 65, 94, 200, 205, 206,
Brunet, E., 172, 205 217, 224, 225, 227, 235
Buccino, G., 149 Damasio, H., 25 (9), 227
Buchel, C., 165 Darwin C., 224
Buck, R., 150 Davidson, R. J., 255, 260, 262, 264
Buckner, R. L., 8, 14 Davis M, 195
Buechel, C., 205 Decety, J., 5, 6, 71, 120, 134, 144
Bunge, S. A., 264 Deckel, A. W., 36 (16)
Burgess, P. W., 29 (5), 35 (5) de Fockert, J. W., 263
Bush, G., 254 Dehaene, S., 27 (13)
Deibert, E., 58 (24)
C Dennett, D., 172
Denton, D., 217
Cabeza, R., 58 (19), 254, 260, 263 Derbyshire, S. W., 217
Cacioppo, J. T., 2, 3, 8 De Renzi, E., 140
Cahill, L., 196, 255 Desimone, R., 117
Caine, S. B., 248 Desmond, J. E., 264
Calabresi, P., 248 De Valois, R. L., 93
Calder, A. J., 57 (37), 60 (37), 196, 217 de Waal, F., 234
Calvert, G. A., 123, 124, 150, 164 de Zubicaray, G. I., 64
Camerer, C., 215, 216, 220n14 Dicks, D., 6
Campagna, A. F., 36 (19) di Pellegrino, G., 118, 144
Campbell, R., 122, 123, 150 Dittrich, W. H., 116
Canli, T., 196, 256, 264 Dolan, R. J., 200, 205, 206
Caramazza, A., 53 (1), 53 (4), 54 (4) Doner, J., 110
Carey, D. P., 118 Downar, J., 120, 180n2
Carrasco M, 195 Downing, P., 102, 105, 107, 120, 172, 173
Carruthers, P., 156 Drevets, W. C., 50 (32)
Carter, C. S., 163, 254 Driver, J., 122
Casey, K. L., 205 Duhamel, J. R., 140
Castelli, F., 58 (26), 172, 205 Duncan J., 235
Cavada, C., 254, 263 Dupuy, E., 22 (4)
Changeux, J. P., 27 (13) Duvernoy, H. M., 208
Chao, L. L., 59 (8), 59 (9), 88, 93, 105
Chartrand, T. L., 144
Chen, Y.-C., 49 (21) E
Chua, P., 254 Easterbrook, J. A., 45 (14)
Cochin, S., 134 Edelman, S., 88
Cohen, J., 54 (17), 219, 254, 260, 263 Eimer, M., 76, 122
Cohen, M. S., 14 Ekman, P., 145, 150, 186, 196, 207
Colby, A., 36 (7), 63, 65, 66 Elliott, R., 206, 255, 262
Colby, C. L., 140 Emery, N. J., 144
Corbetta, M., 103, 186, 195 Epstein, R., 88, 94, 96n39
Courtney, S. M., 260, 262 Eslinger, P., 27 (13), 64, 65, 200, 206
Critchley, H., 205, 217, 264 Evans, K. C., 217
Author Index ■ 283
F Grafton, S. T., 134

Grant, K. A., 244
Fabre-Thorpe, M., 82
Gray, J. A., 186, 196
Fadiga, L., 5, 134
Greene, J., 64, 172, 180
Fallon, A. E., 186, 196, 217
Grezes, J., 5, 6, 134, 144, 165
Farah, M. J., 6, 83
Grill-Spector, K., 76, 83, 88, 102, 107, 109
Farrer, C., 150
Gross, C. G., 87
Fehr, E., 220
Gross, J. J., 253, 254, 255, 262, 264
Feldman Barrett, L., 254, 255, 260, 262, 263, 264
Grossman, E., 102, 103, 108, 112, 120, 172
Felleman, D. J., 101
Gusnard, D. A., 58 (28), 60 (28), 60 (40), 173,
Ferrier, D., 23 (5)
180n2, 260
Ferstl, E. C., 172
Guth, W., 216
Fink, G. R., 165
Fisher L., 65
Fletcher, P. C., 64, 156, 158, 162, 164, 172, 179, 205 H
Flynn, F., 64 Haidt, J., 172, 180
Fogassi, L., 140 Halgren, E., 59 (16)
Forman, S. D., 146 Hall, F. S., 244, 247
Fox, P., 23 (8), 146, 157 Halsband, U., 123
Frank, R. J., 8, 25 (9), 37 (24), 227 Hämäläinen, M., 81
Freed, D., 64 Hamann, S. B., 255
Friesen, C. K., 122 Hancock, P. J. B., 94
Friesen, W. V., 145, 207 Happe, F., 156, 162, 254, 260
Friston, K., 67, 135, 158, 207, 208, 264, 265 Hari, R., 118, 119, 134
Frith, C. D., 58 (27), 64, 150, 172, 179, 180, 205, 234 Hariri, A. R., 150, 264
Frith, U., 58 (27), 64, 156, 205, 234 Harlow, M., 22 (1), 25 (1)
Fukuda M, 195 Harter, S., 36 (19)
Haxby, J. V., 59 (5), 59 (6), 64, 69, 71, 88, 93, 95n18, 95n21,
G 102, 105, 122, 172, 186, 195
Hecaen, H., 94
Gabrieli, J. D. E., 58 (22) Heeger, D. J., 8, 14
Gallagher, H. L., 60 (43), 156, 157, 162, 164, 165, 172, 179, Heider, F., 156
180, 205, 260, 264 Henrich, J., 216
Gallese, V., 118, 119, 134, 144, 156 Hering, E., 93
Gallup, G. G. J., 234 Hertwig, R., 220n14
Ganz, L., 101 Hietanen, J. K., 118, 119, 122
Gardner H., 234 Hinton, S. C., 49 (23), 50 (23)
Gauthier, I., 4, 88, 94, 96n38, 96n39, 102, 101 Hirt, E. R., 2
Gentilucci, M., 145 Hoffman, E., 59 (15), 60 (15), 64, 69, 71, 105, 122, 172
George, M. S., 48 (16), 60 (44) Holmes, A. P., 265
George, N., 76 Hornak, J., 199
Gewirtz, J. C., 267n2 Howard, R. J., 102, 119
Geyer, S., 150 Hsieh, J. C., 254
Gilligan, C., 66 Hubel, D. H., 102
Goel, V., 60 (45), 156, 162 Humphrey, N., 199
Golby, A. J., 64, 70 Humphreys, G. W., 122
Gold, L. H., 247 Hurd, Y. L., 244
Goldman-Rakic, P. S., 27 (14), 27 (17) Hurvich, L. M., 93
Goleman, D., 225, 234, 235
Gomez Gonzalez, C. M., 82
Goodale, M. A., 101, 134, 140 I
Gopnik A., 234 Iacoboni, M., 118, 134, 144, 146
Gordon, A. H., 267n4 Iadarola, M. J., 217
Gorman, D. G., 64 Irwin, W., 260, 262, 264
Gorno-Tempini, M. L., 165 Isenberg, N., 70, 205
Grady, C. L., 162, 164 Ishai, A., 59 (7), 88, 92, 93
Grafman, J., 29 (4), 59 (31), 60 (31) Ito, T. A., 267n4
J Lerner, J., 215

LeSage, M. G., 244, 247
Jackson, D. C., 253, 255, 265
Leshner, A. I., 243
Jameson, D., 93
Levenson, R. W., 196
Jarrell TW, 186, 195
Levin, H. S., 27 (13)
Jeannerod, M., 123, 134, 140
Liao, H., 8
Jeffreys, D. A., 76
Liberzon, I., 264
Jellema, T., 117, 118, 119
Lieberman, M. D., 265
Jezzard, P., 8, 9, 13
Lim, K. O., 264
Jha, A. P., 45 (13)
Liu, J., 76
Johansson, G., 102, 116
Livingstone, M. S., 102
John, O. P., 253
Loewenstein, G., 215
Jolicoeur, P., 75
Logan, J., 250
Jones, E. E., 2
Logothetis, N. K., 87
Jonides, J., 254, 260, 262, 263
Lucas, L. R., 248
Josephs, O., 208
Luppino, G., 144
K M
Kalaska, J. F., 144 MacDonald, A. W., 264
Kandel, E., 64 MacDonald, J., 123
Kanwisher, N., 4, 59 (13), 59 (38), 76, 83, 88, 94, 96n39, MacDonald III, A. W., 219
102, 107, 108, 120, 165, 263 Mach, R. H., 249
Kastner, S., 194 MacMillan, M., 22 (4)
Kawasaki, H., 76, 81, 255, 262 Maddock, R. J., 224
Kelley, W. M., 58 (30), 60 (30) Maguire, E. A., 58 (29), 60 (29)
Keysers, C., 82 Malach, R., 103
Khroyan, T. V., 248 Manes F, 227
Kiehl, K. A., 64 Maril, A., 58 (20)
Kingstone, A., 122 Marinelli, M., 248
Kirino, E., 44 (12), 45 (12), 50 (12) Markowitsch, H. J., 70
Kleinke, C. L., 122 Martin, A., 53 (2), 53 (3), 95n21
Klin, A., 161 Martin, S. P., 247
Knight, R. T., 49 (26), 49 (27), 254, 260, 263 Mather, G., 102
Kohlberg, L., 36 (7) Mayberg, H. S., 44 (8), 47 (9), 50 (9)
Kohler, E., 5, 118, 144 Mayer, J. D., 225
Koski, L., 144, 146 Mazoyer, B. M., 164
Kourtzi, Z., 102, 107, 120 Mazziotta, J., 150
Kovacs, G., 82 McCabe, K., 215
Kozlowski, L. T., 116 McCallum M., 234
Kring, A. M., 267n4 McCarthy, G., 44 (5), 44 (11), 45 (13), 50 (11), 59 (14), 88,
Kulikowski, J. J., 101 94, 96n39
Kuruoglu, A. C., 36 (17) McDonald, H. E., 2
McGuire, P. K., 163
L McGuire, W. J., 2
LaBar, K. S., 45 (15) McGurk, H., 123
Lacquaniti, F., 144 McLeod, P., 117
Lane, R., 49 (22), 64, 70, 163, 201, 225, 234, 254, 260 Mecklinger, A., 58 (23)
Lang, P. J., 265 Medvec, V, H., 2
Langton, S. R. H., 122 Mega, M. S., 44 (9), 49 (9)
Lanitis, A., 94 Mesulam, M.-M., 44 (10)
Lazarus, R. S., 254, 261 Mesulam M. M., 186, 195
LeDoux, J. E., 3, 144, 186, 195, 254, 255, 260, 262, 263, Miczek, K. A., 247
267n2 Miller, B. L., 64
Leinonen, L., 140 Miller, E. K., 219, 254, 260, 263
Le Moal, M., 248 Milne, E., 59 (31), 60 (31)
Leopold, D. A., 94 Milner, A.D., 134, 140
Milner, B., 27 (16) Paus, T., 134, 164

Mintun, M. A., 157 Pause, M., 140
Mishkin, M., 44 (1), 102 Pavlova, M., 116, 118
Mistlin, A. J., 119 Pelli, D. G., 78, 110
Moll, J., 60 (46), 64, 71, 180 Penfield, W., 137, 138
Morgan, D., 244, 245, 248 Perner, J., 172
Morgan, M. A., 254, 263, 267n2 Perrett, D. I., 59 (10), 82, 116, 117, 118, 119, 122,
Morris, J. S., 2, 70, 150, 194, 195, 199, 200, 205, 206, 255, 126, 144
262, 263, 264 Pessoa L, 186, 194
Mouchetant-Rostaing, Y., 76, 81 Petersen, S. E., 27 (16)
Mountcastle, V., 144 Peterson, B. S., 254
Mufson, E. J., 186, 195 Petit, L., 260
Mummery, C. J., 58 (29), 60 (29) Petrides, M., 27 (16), 150
Munk, H., 4 Phan, K. L., 150
Murnighan, J. K., 216 Phelps, E. A., 70, 195, 205, 254, 255, 260, 262,
263, 264
N Phillips, G. D., 247
Phillips, M. L., 150, 186, 199, 200,
Nader, M. A., 244, 245, 247, 248, 249
206, 217
Narumoto, J., 59 (12), 122, 180n2
Piazza, P. V., 248
Neri, P., 102
Picard, N., 149
Newman, J. D., 36 (22)
Pietrini, P., 71
Nielsen-Bohlman, L., 254
Pillutia, M. M., 216
Nisbett, R. E., 2
Platt, J. J., 36 (25), 36 (26)
Nishitani, N., 118, 119
Ploghaus, A., 254
Northoff, G., 49 (20)
Posner, M. I., 27 (16)
Nowak, M. A., 216
Prather, M. D., 6, 7
Nyberg, L., 58 (19), 254, 260, 263
Preston, S., 234
Nyman, G., 140
Price, B. H., 33 (14), 64
Price, J. L., 70, 205
O
Puce, A., 54 (11), 59 (11), 105, 120, 122, 123, 126, 150,
O’Connor, D. H., 194 164, 165
O’Craven, K., 59 (38) Pyszczynski, T., 2
O’Craven KM, 186, 195
O’Doherty, J., 70 Q
O’Doherty, J. O., 255, 262
O’Toole, A. J., 94, 109 Quirk, G. J., 267n2
Ochsner, K. N., 195, 254, 255, 256, 260, 262, 263,
264, 265 R
Ockenfels, A., 220
Raichle, M. E., 8, 50 (32), 58 (28), 60 (28), 60 (41),
Ohman A, 186
173, 180n2
Olausson, H., 150
Raine, A., 36 (18), 36 (21), 64, 71
Oldfield, R. C., 145
Rainer, G., 84
Oliveira-Souza, R., 64
Rainville, P., 163
Öngür, D., 70
Raleigh, M. J., 27 (15)
Opitz, B., 49 (25)
Rao, S. C., 44 (3)
Oram, M. W., 82, 116, 118, 126
Rasmussen, T., 137, 138
Orban, G. A., 103
Reiman, E. M., 60 (48), 64, 70, 254
Ortmann, A., 220n14
Ress, D., 8, 14
Owen, A. M., 44 (6)
Richards, J. M., 253
Rilke, O., 244, 247
P
Rizzolatti, G., 5, 118, 119, 123, 134, 144, 150
Pakkenberg, H., 1 Roediger, H. L., 2
Pandya, D. N., 27 (14), 144, 150 Rolls, E. T., 44 (2), 70, 82, 255, 260, 262
Paradiso, S., 254, 260 Rosch, E., 75, 76
Pardo, J. V., 60 (47), 264 Ross, L., 2
Roth, A. E., 216 Stuss, D. T., 29 (6), 38 (6), 49 (19), 254

Rozin, P., 186, 196, 217 Sugase, Y., 75
Ruby, P., 6, 71 Suneart, S., 102
Szikla, G., 23 (8)
S
Sadr, J., 78, 84 T
Sakata, H., 140, 144 Talairach, J., 23 (8), 67, 95n18, 135, 158
Salovey, P., 225 Tanaka, J. W., 76
Sams, M., 76, 123 Tanaka, K., 87, 93, 102
Sarter, M., 8 Tataranni, P. A., 217
Saver, J., 27 (13), 32 (8) Taylor, G. J., 225
Sawchuk, C. N., 196 Taylor, M. J., 76, 122
Scarpa, A., 36 (21) Thaler, R. H., 216, 220n14
Schacter, D. L., 256 Thompson, J. C., 120, 126
Schaltenbrand, G., 5 Thompson, R. F., 195
Scheinberg, D. L., 87 Thorndike, E. L., 225
Schendan, H. E., 76, 81 Thornton, I. M., 116
Schenk, S., 244, 247 Thorpe, S., 76, 81, 82
Schenk, T., 117 Tolhurst, D. J., 101
Scherer, K. R., 254, 262 Tong, F., 173
Schiller, P. H., 101 Tootell, R. B., 77, 103
Schmidt, K. M., 220 Tournoux, P., 23 (8), 67, 95n18, 135, 158
Schoenbaum, G., 267n2 Townsend, D. W., 157
Schultz, W., 244 Tranel, D., 226, 227, 229
Schwartz, G. E., 234 Tunturi, A. R, 4
Seeck, M., 76, 81
Seitz, R. J., 140 U
Seltzer, B., 144 Umilta, M. A., 119, 144
Shakespeare, W., 253 Ungerleider, L., 44 (1), 102
Shallice T., 29 (5), 35 (5)
Sheline, Y. I., 196 V
Shelton, J. R., 53 (1), 53 (4), 54 (4)
Shi, C., 195 Vaina, L. M., 102, 117, 119, 122
Shulman, G. L., 60 (39), 180n2 Vandenberghe, R., 165
Sicoly, F., 2 Van Oostende, S., 103
Simmel, M., 156 VanRullen, R., 76, 81
Simpson, J. R., 254, 261, 262 van Veen, V., 254
Sinha, P., 78, 84 Verhoeff, N. P., 248
Skudlarski, P., 112 Vogeley, K., 172
Smith, E., 44 (4), 254, 260, 262, 263 Volkow, N. D., 244, 248
Smith, N. K., 1 vonBonin, G., 150
Smith, P. K., 156 von Cramon, D. Y., 172
Snarey, J. R., 65 Vuilleumier, P., 122, 186, 194, 205,
Snyder, L. H., 140 208, 263
Sokolov, A., 116
W
Somers, D. C., 111
Sperber, D., 162 Wachsmuth, E., 117
Spivak, G., 36 (25), 36 (26) Wagner, A. D., 58 (21), 219
Sprengelmeyer, R., 199, 200, 206 Wang, G. J., 248
Springer, K., 156 Ware, J., 196
Stein, S. J., 225, 235 Warrington, E. K., 94
Sternberg, R. J., 234 Watanabe, T., 111
Strafella, A. P., 134 Watson, A. B., 78
Strange, B. A., 205 Watson, J. D. G., 103
Streit, M., 76, 81 Whalen, P. J., 205, 255, 262, 196-186
Strick, P. L., 149 Wheaton, K. J., 125, 126
White, F. J., 248 Y

Wicker, B., 122
Yeshurun, Y., 195
Williams, J. H., 126
Yeterian, E. H., 27 (14)
Williams, J. M. G., 195
Young, A., 76
Wilson, D., 162
Wilson, F. A., 44 (2)
Z
Wilson, T. D., 2
Wimmer, H., 172 Zahn, R., 69
Winston, J. S., 180n2, 264 Zajonc, R. B., 3
Wojciulik, E., 103, 186, 195 Zald, D. H., 8, 14, 264
Woods, R. P., 112, 145, 146, 250 Zarahn, E., 146
Woolsey, C. N., 5 Zihl, J., 117
Woolverton, W. L., 244, 247 Zirkel S., 225
Worsley, K., 208 Zwick, R., 216
RT0996_Subject_Index.qxd 11/8/04 2:13 PM Page 289
Subject Index
•
A Animals, see Monkeys

ACC (anterior cingulate cortex, decision making, Animations
217–219 examples, 167
Acquired sociopathy, 64 Heider, Fritz, 153
ACT (Awareness of Consequences Test), 32, 36, 37 movement patterns, perception, 158
Action observation, premotor and parietal areas, Simmel, Mary-Ann, 153
133–141 ANOVA (Analysis of variance)
discussion, 138–140 attention and emotion, 45, 46
introduction, 134 empathy, neural mechanisms, 146
materials and methods, 134–135 moral judgments, 67
results, 135–138 social dominance, 251
Adult-onset lesions, vs. early-onset, 29–31, 33–36 Anterior cingulate gyrus
Agency, 71 integration role, 47–48
Amygdala lesions, 26
attention and emotion, 45, 189–191 Anterior mesial frontal cortices, lesions, 26
automaticity, 186, 194, 195, 205 Anterior orbital sectors, lesions, 34
disgust, 193 Antisocial Personality Disorder, 36
dissociable systems, 43 Assertiveness, 233
emotion, 253 Attention
emotional and social intelligence, 226, 227 amygdala responses, 189–191
empathy, neural mechanisms, 146, 149, 150 anterior insular response, 192
facial signals, threatening, 185, 186, 190–196 cognition, 50
fear, 194 cortico-amygdala interactions, 192–194
lesions, 199–200 extrastriate responses, 189
moral judgments, 69–70 PFC (prefrontal cortex), 34
movement patterns, perception, 159, 160, 161, Attentional dependence, facial signals, 190–191
162, 164 Attentional modulation, threat, 185, 186
perception, cognition, decision making, 4 Attention and emotion, dissociable systems, 43–51
reappraisal, 254–255, 258–263 discussion, 48–50
responses, attention, 189–191 fMRI (Functional Magnetic Resonance Imaging), 45–46
social cognition, 6–7 methods, 44–46
socioemotional relevance, 4 results, 46–48
somatic marker hypothesis, 224 Attention-demanding target detection task, 44
ToM (“Theory of Mind”), 153 Attention-demanding tasks, 50
trustworthiness, 200–201, 203–206 Autism, ToM (“Theory of Mind”), 156, 157
289
Automaticity BOLD (Blood oxygen level dependent) signal measure,

defined, 185 trustworthiness, 202
fear, 186, 194, 196 BOLD (Blood oxygen level dependent) signals
judgments, 205 biological motion, viewing, 105–106
parallel pathways, 195 person vs. object knowledge, 56
ToM (“Theory of Mind”), 173–174, 177
B trustworthiness, 203, 205
Brain activation, moral judgments, 68–70
Basal temporal cortex, movement patterns, 164–165
Brain dimensions, 28n11
Behavior
Brain lesions, see Lesions
changes, 22 (see also Lesions and behavior)
Brain model, Phineas Gage, 25–26
neural mechanisms, 3–8
Brain structures, social cognition, 5
social, 3–8, 241
Brainvox technique, 33–34, 36
Behavioral data, ToM (“Theory of Mind”), 159–161
Brodmann’s cytoarchitectonic fields
Behavioral measures, moral judgments, 66–67
behavior, 34
Behavioral performance, attention and emotion, 46
medial prefrontal areas, 164
Behavioral procedure, person vs. object knowledge, 55
Phineas Gage, 26
Behavioral profiles, monkeys, 244, 249
BVRT (Benton Visual Retention Test),
Behavioral results
229, 231
decision making, economic, 216–217
moral judgments, 68
C
movement patterns, perception, 159
trustworthiness, 200 Canonical hemodynamic response function, 55
Behavioral tasks, empathy, 145 Categorization and identification
“Belief-desire psychology,” 172n1 faces, 77–81
Benton Facial Discrimination Test, 229, 231 MEG (magnetoencephalography) , 83–84
Bias, regulation of emotion, 256, 267n1 Category identification, faces and objects, 89
Biological and social processes, 239–241 Cerebral cortex, 1, 21
Biological motion, visual perception, 31, 101–114 Choice process, 211–213
discussion, 108–110 Cingulate cortex, reappraisal, 258–259
experimental procedures, 110–112 Cingulate gyrus
introduction, 101–103 attention and emotion, 45, 47–48
results, 103–108 dissociable systems, 43
Biological motion, electrophysiology and imaging, lesions, 34
115–129 Cocaine self-administration, 246–247, 250
behavioral studies, perception, 116–117 Cognition
introduction, 115–116 information processing, 183
perception, non-humans, 117–119 social, 2–3, 5, 122, 200, 206
perception, electrophysiology and imaging, 119–120 social and moral, 180
perception vs. human motion perception, 120–123 social perception, 183–184
Biological motion vs. human motion, 120–123 Cognitive, definition, 216n12
Biological movement, perception, 97–99 Cognitive regulation of emotion, 253–270
Biological movement, perception, imitation, emotion, conclusions, 265
131–132 discussion, 260–265
BOLD (Blood oxygen level dependent) response introduction, 253–255
biological motion, viewing, 102–103, 107–108, 110, 111 methods, 265–267
definition, 7–8 reappraisal, 260–262
empathy, neural mechanisms, 146 results, 256–259
facial signals, threatening, 189 subjects, 265, 267n4
fMRI (Functional Magnetic Resonance Imaging), 8–9 Cognitive systems, maturation, 4
neuronal activity, 14 Cohesive groups, 97–98
task-related, 15 Coils, MRI, 11–12
ToM (“Theory of Mind”), 176, 179 Communication disorders, 126
trustworthiness, 201 Comparison targets, biological movement, 97
BOLD (Blood oxygen level dependent) signal change, 199 Conduct Disorder, 36
ToM (“Theory of Mind”), 178 Consilience, biological movement, 99
trustworthiness, 208 Contagion effects, 131
Subject Index ■ 291
“Control,” mirror system, 124 prefrontal cortex, 34

Cortex, functional localization, 4–5 regulation, 263–265
Cortical region responses, faces and objects, 89–92 Emotional and social intelligence, 223–237
Cortico-amygdala interactions, 192–194 discussion, 231–235
COWA (Controlled Oral Word Association), 31, 230 introduction, 224–226
methods, 226–230
D results, 230–231
Emotional arousal, 49
Darwin, C., 224 Emotional contagion, 132
Decision making Emotional distraction, 50
emotional and social intelligence, 223–225 Emotional intelligence, 31, 223, 224–225
IQ (cognitive intelligence), 224 Emotional valence, 64–65, 68
lesions, brain, 21–28 Emotion and attention, see Attention and emotion, dissociable
overview, 211–213 systems
results, 230–231 Emotions, decision making, 215
ventromedial frontal region, 27 Empathy, neural mechanisms, 143–152
Decision making, economic, 215–221 discussion, 146–151
discussion, 217–220 introduction, 143–145
introduction, 215–216 materials and methods, 145–146
methodology, 216, 220n14 results, 146
results, 216–217 Encoding, population, 93–94
Decision making, psychophysiology, 32–33 Entitative groups, 97–98
“Direct matching hypothesis,” 132 EQ-i (Emotional Quotient Inventory), 223
Disgust EQ-i (Emotional Quotient Inventory) , 226, 228,
amygdala, 191–192, 193 232–233, 237
facial signals, threatening, 185 ERP (event-related potential)
insula, 186–187, 193–196 face selective response, 76, 81–82
regions of interest (ROI), 188–189 facial movements, 123
Disorders of social communication, 126 gaze aversion, 124
Dissociable systems, 41–42; see also Attention and emotion, gaze perception, 122
dissociable systems; Person and object knowledge, ERPs (event-related potentials)
neural systems biological motion, 115
Distractors, 44, 47 human motion, 125, 126
DLPFC (Dorsolateral prefrontal cortex), 215, 217–219 social attention task, 124, 126
Dopamine, 36 Evolution
Dopaminergic functioning faces and objects, 74
positron emission tomography, 249–250 frontal region expansion, 1
social rank, 244–246 neural systems, 184
Drug abuse, 243–344 social cognition, 156
Exaption, definition, 4
E Experimental conditions, moral judgments, 65–66
Experimental design, attention and emotion, 44–45
Early-onset lesions vs. adult-onset, 29–31, 33–36
Expression of the emotions in man and animals” (Darwin), 224
EBA (Extrastriate body area)
Extrastriate responses, attention, 189
biological motion, 102–103, 106–107, 109–110
ToM (“Theory of Mind”), 175, 176, 179
F
Ebbinghaus effect, 97
Echo planar imaging (EPI), 134 Face and object processing, 73–74
Economics, see Decision making, economic Face configuration vs. face parts, 84
Effector systems, 4 Face perception, processing stages, 75–84
Electrophysiology, see Biological motion, electrophysiology discussion, 81–83
and imaging magnetoencephalography methods, 83–84
Emotion; see also Cognitive regulation of emotion methods, 83–84
bias, experimental, 256, 267n1 results, 76–81
facial, 208 Faces, judgments of trustworthiness, 199–210
lesions, 21–28 discussion, 205–206
modulation by reappraisal, 262–264 introduction, 199–200
Faces, judgments of trustworthiness (continued) Foot actions, 137, 138–139

methods, 206–208 Frontal cortex
results, 200–205 lesion, Gage, Phineas, 26
Faces and objects, ventral temporal cortex, 87–96 perception, cognition, decision making, 4
category identification, 89 Frontal lobes, 64
cortical region responses, 89–92 Frontal regions, 1
discussion, 92–94, 96n25 Functions, attentional and emotional, 43–51
neural response, object categories, 88–89 Fusiform gyrus
object form topography, 93 face processing, 4
population encoding, visual appearance, 93–94 faces and objects, ventral temporal cortex,
Face-selective response, 76–77 73, 74
Facial Affect Series, 187 movement patterns, perception, 159, 160, 161,
Facial expressions, 186 162, 164
Facial recognition, 4 trustworthiness, 200–202, 205
Facial signals, threatening, 185–198
discussion, 194–196 G
introduction, 186
Gage, Phineas, 19–20, 21–28
materials and methods, 186–189
methodology, 23–27
results, 189–194
research, early, 22–23
Fairness, neural mechanisms, 219
results, 27
False beliefs
skull, 23–25
stories task, 172–173
Gambling Task
ToM (“Theory of Mind”), 177, 179
decision making, 32–33
Family histories, lesions, 30–31
emotional and social intelligence, 223–224, 232
Fear
lesions, 37, 38
amygdala, 186, 186–187, 194
Gaze perception, 122
facial signals, threatening, 185
Genetics, 239–240
insula, 195
“Group attention,” mirror system, 124
regions of interest (ROI), 188–189
Groups, cohesive, 97–98
responses, 192
Feelings, see Cognitive regulation of emotion H
Female subjects, 265, 267n4
Ferrier, David, 22–23 Hand actions, 136, 138–139
FFA (Fusiform face area) Harlow, John, 22, 23
biological motion, viewing, 102–103, 108–110 Heider, Fritz, 153
faces and objects, 88, 93 Hemodynamic time courses, 57, 59
facial signals, 188 Homunculus, motor, 137
fMRI (Functional Magnetic Resonance Imaging), 13–16 Humans
action observation, 135 biological motion, 115
attention and emotion, 45–46 mirror neuron system, 134
biological motion, 115, 119–121 motion vs. biological motion, 120–123
biological motion, viewing, 111–112
data, reappraisal, 266–267 I
data interpretation, 14–16 IAPS (International Affective Picture System), 44
empathy, neural mechanisms, 145–146 Identification and categorization, faces, 77–81,
faces, judgments of trustworthiness, 200–205, 207 83–84
faces and objects, ventral temporal cortex, 93 Image analysis, action observation, 135
facial signals, threatening, 188–189 Imaging, see fMRI (Functional Magnetic Resonance
foot actions, 137 Imaging)
hand actions, 136 Imitation, emotional contagion, 132
moral judgments, 67 Inferior frontal cortex, empathy, 146, 147, 149
mouth actions, 136 Information processing
person vs. object knowledge, 54, 55 capacity for, 2–3
reappraisal, 254, 257–259 cognition, 183
technology, 9–16 social, 97, 184, 213
ToM (“Theory of Mind”), 173, 176–177, 177–178 ToM (“Theory of Mind”), 154
Insula LOC (lateral occipital complex), biological motion, 102–103,

decision making, economic, 215, 217–219 107–108, 109–110
emotional and social intelligence, 226, 227, 235 LPFC (lateral prefrontal cortex), reappraisal, 254, 257, 259–263
empathy, neural mechanisms, 146, 147, 149, 150
faces, judgments of trustworthiness, 199 M
facial signals, 188 Magnetoencephalography methods, 83–84
facial signals, threatening, 186, 190–196, 199 MAP-3, 37
response, attention, 192 Maturation, neural systems, 4
social intelligence, 199 MEG (magnetoencephalography), 73–74
trustworthiness, 200–206 categorization and identification, 77–81, 83–84
Intelligence, social, 199 face-selective response, 76–77
Intelligence, social and emotional, see Emotional and social identification and categorization, 77–81
intelligence Memory encoding, 49–50
IQ (cognitive intelligence) Mentalizing
controls, 230 movement patterns, 155–157, 161, 164–166
decision making, 224 PFC (prefrontal cortex), 162
emotional and social, 225–226, 229–232, 234–235 Mental states, (ToM), 176, (180n2)
MEPS (Means-Ends Problem Solving Procedure), 32, 36, 37
J Mesial orbital sectors, lesions, 34
Jackson, John Hughlings, 183–184 Mimicry, 131, 135, 136–137
JLO (Judgment of Line Orientation), 31, 229, 231 Mind and movement
Johansson’s “point light” animation, 102 Mirror neuron system
Judgment; see also Faces, judgments of trustworthiness; action observation, 131, 133, 138–139
Moral judgments perception, motion, 123–124
emotional intelligence, 235 processes, 5
social, 199 MMPI (Minnesota Multiphasic Personality Inventory),
230, 231
K Models, brain, 25–26
Modulation by reappraisal, 258–259
Kohlberg Moral Judgment Task, 32
MOFC (medial orbitofrontal cortex)
emotion, 253
L
reappraisal, 254–255, 259–263
Lamor frequency, 4–5 regulation, 263, 267n2
Language acquisition, 4 Monkeys; see also Social dominance
Lateralization, reappraisal, 264, 267n3 behavioral profiles, 249
Left polar cortices, lesions, 34 biological motion, perception, 117–119
Lesions; see also Gage, Phineas; Lesions and mirror neurons, 134
behavior premotor cortex, 134
adult onset vs. infancy, 29–30 Monte Carlo simulation, 55
amygdala, 199–200, 227 Moral judgments, 31, 63–71
behavior, 22, 27 discussion, 70–71
clinical evidence, 30–31 fMRI (Functional Magnetic Resonance Imaging), 64–65, 66
decision making, 21 introduction, 63–65
early-onset vs. adult-onset, 29–30, 33–36 materials and methods, 65–67
emotion, 21, 27 results, 68–70
insula, 227 subjects, 65
overlap, 37 Moral reasoning, 31, 32
subjects, 226–227 Moral reasoning assessment, 36
ventromedial (VM) cortex, 224, 227 Motion, see Biological motion, visual perception; Biological
Lesions and behavior, 29–39 movement, perception
discussion, 33–36 Motion, human vs. biological, 120–123
methods, 36–38 Motor commands, premotor cortex, 131–132
neuropsychological evidence, 31–33 Motor homunculus, 137
overlap, 37 Mouth actions, 136, 138–139
results, 30–33 Movement, see Biological motion, visual perception; Biological
Lip reading, perception, motion, 122–123 movement, perception
Movement patterns, perception, 155–169 Orbitofrontal cortex

discussion, 161–166 moral judgments, 67, 68–71
introduction, 156–157 sociomotional relevance, 4
methods, 157–159 Organization, functional, PFC (prefrontal cortex), 43–44
results, 159–161 OTT (optional thinking test), 32, 36–37
MPFC (Medial prefrontal cortex), lesions, 33 Overlap
MPFC(medial prefrontal cortex), reappraisal, 254, 257, lesions, 37
259–263 neurodynamic responses, 187–188
MRI (Magnetic Resonance Imaging), 4
P
attention and emotion, 45
fMRI (Functional Magnetic Resonance Imaging), 13–16 Parietal areas, see Action observation, premotor and parietal
RF emissions, 12 areas
signal, 9–13 Parietal lobe, action observation, 139–140
technology, 9–16 Parsimony, definition, 98
“Mutual gaze exchange,” mirror system, 124 Patient cases, lesions, 30–31
Patterns of response, faces and objects, 89–92
N Perception, 229; see also Biological motion, visual
perception; Social perception
Negative affect, reappraisal, 256–257
Performance, behavioral, 46
Neocortex, 1
Person and object knowledge, neural systems, 53–61
Neural hierarchy, Jackson, 183–184
behavior, 55
Neural mechanisms, behavior, 3–8
discussion, 57–62
Neural response, object categories, 88–89
evolution, 184
Neural responses, object categories, 89, 95n22
image data, 89, 96n25
MRI data analysis, 55
patterns, 88, 94n18, 95n19
results, 55–57
Neural systems, person and object knowledge, 53–61
PET (positron emission tomography), 115, 120–121
behavior, 55
biological movement, perception, 98
discussion, 57–62
emotion expressions, 146–147
evolution, 184
monkeys, dominance, 244–248
PFC (prefrontal cortex); see also Attention and emotion,
MRI data analysis, 55
dissociable systems
results, 55–57
attention and emotion, 46, 50
Neuroanatomical analysis, lesions, 34
damage, 29–30
Neuroanatomical models, 44
emotion, 253
Neurobiological model, 206
fMRI (Functional Magnetic Resonance Imaging), 47
Neuroimaging evidence, lesions, 33–34
functional organization, 43–44
Neuromodulators, 36
Gage, Phineas, 23
Neuropsychological evidence, 31–33
movement patterns, perception, 159, 160, 161, 162, 164
Novelty, 43, 49–50
organization, functional, 43–44
person vs. object knowledge, 60–61
O PFC (prefrontal cortex) and behavior, 29–39
Object form topography, 88, 93 clinical evidence, 30–31
Object knowledge, see Person and object knowledge, neural discussion, 33–36
systems fMRI (Functional Magnetic Resonance Imaging), 46
Objects, see Faces and Objects, ventral temporal cortex methods, 36–38
“Object-selective,” ventral stream mechanisms, 102 neuropsychological evidence, 31–33
Occipital areas, biological motion, 102–103 results, 30–33, 47
Occipital cortex, movement patterns, 165–166 “Point light” animation, 102
Occipital gyrus, movement patterns, 159, 160, 161, 162, 164 Population encoding, visual appearance, 93–94
OFA (occipital face areas), biological motion, 108–110 PPA (parahippocampal place areas)
Online resources faces and objects, 88, 93
faces and objects, (96n24) facial signals, 188
fMRI (Functional Magnetic Resonance Imaging), 14 object categories, 88
Orbital gyri, lesions, 34 Precession (wobbling), 4
Orbital region, lesions, 26 Predetermination, 239–241
Prefrontal brain systems, 43 Skulls

Premotor areas, see Action observation, premotor and measurements, 23, 27n6
parietal areas Phineas Gage, 23–25
Premotor cortex photographs, 24
action observation, 138–139 trajectories, 25, 28n10
mirror neuron system, 134 SMA (supplementary motor area), 26–27
motor commands, 131–132 Social and biological processes, 239–241
somatotopy, 137 Social behavior, 241
Processing of gestures and actions, 126 reasoning, 22, 27n3, 29
Prosopagnosia, 94, 96n38 Social cognition, 2–3
PSTS (Posterior superior temporal sulcus), 103, brain structures, 5
105–106 face processing, 200
Psychopathy, 64 neurobiological model, 206
Psychophysiological responses, 35, 38 perception, motion, 122
Psychophysiology, decision making, 32–33 Social cognitive neuroscience, 171
Pulse sequence, 13 Social dominance, monkeys, 243–252
Punishment and reward, 35–36 discussion, 247–248
introduction, 243–244
R methods, 248–251
results, 244–247
Rank, social
Social fluency, 32
cocaine self-administration, 246–247
Social functioning, 232
dopaminergic functioning, 244–246, 249–250
Social identity theory, 97
RAVLT (Rey Auditory Verbal Learning Test), 31, 229
Social information processing, 213
Reappraisal
Social intelligence, 199; see also Emotional and social
definition, 253–254
intelligence
lateralization, 264, 267n2
Social judgment, 199
LPFC (lateral prefrontal cortex), 254, 257, 259–263
Social knowledge assessment, 36
modulation, emotion processes, 262–263
Social perception
success of, 256–257
biological movement, 98
Reasoning
cognition, 183–184
moral, 29
Social processes, 2–3, 3–8
social behavior, 22, 27n3
Social rank
Reasoning, social and moral, 32
cocaine self-administration, 246–247
Regional cerebral blood flow, 159–161
dopaminergic functioning, 244–246, 249–250
Research, Phineas Gage, 22–23
Social reasoning, 32
Resonant frequency, 4–5
Somatic marker hypothesis, 223, 224–225, 231–232
Response patterns, faces and objects, 88–92
Somatic responses, 224
Responses, attention, 189–192
Somatosensory cortex, 4
Reward and punishment, 35–36
Somatotopism, 139–140
RF (radiofrequency), 10
Somatotopy, 137
emissions, MRI, 12
Spatial references, 14
energy, 4
Stimuli
Role-taking, 71
biological motion, viewing, 104, 110–111
empathy, neural mechanisms, 145
S faces, judgments of trustworthiness, 207, 208
SCR (electrodermal skin conductance response), faces and objects, ventral temporal cortex, 89
32–33, 38 Facial Affect Series, 187
Selective attention, 180n2 facial signals, threatening, 186–187
Self-regard, 233 Stimuli and task, moral judgments, 65
Semantic knowledge, 53–54 Stimulus novelty, 43, 48–50
Sensorimotor cortices, 1 Stories, ToM (“Theory of Mind”), 172–173, 179,
Serotonin, 27, 36 180–181
SIMJ (Standard Issue Moral Judgment), 36 Stroop interference paradigm, 48
Simmel, Mary-Ann, 153 Structures, brain, 5
Single-unit electrophysiology, 115 Subjects, female, 265, 267n4
Subjects, moral judgments, 65 U

Superior temporal sulcus (STS)
Ultimate Game; see also Decision making, economic
biological motion, perception, 117–119, 120–122
behavioral results, 216–217
biological motion, viewing, 103, 104–107, 108–110
cognitive thought, 216, 220n12
biological movement, perception, 98
deception, 220n14, 220n15
empathy, neural mechanisms, 146, 147–148, 150
decision making, 215–217
movement patterns, perception, 159, 160, 161, 162, 164
neuroimaging, 217–219
processing behavior, 4
Prisoner’s Dilemma game, 220n16
ToM (“Theory of Mind”), 171, 177, 178, 179
trustworthiness, 200–202, 204–205
V
Surgery, monkeys, 250
Ventral extrastriate cortex, objects, 94, 96n39
T Ventral object vision pathway, 87, 88
T1, 11, 13, 14, 36 Ventral stream mechanisms, “object-selective,” 102
T2, 13 Ventral temporal cortex, faces, 73, 74
Talairach and Tournoux atlas coordinates, 56, 57, 58 Ventral temporal cortex, faces and objects , 87–96
Talairach’s stereotactic space, 23 category identification, 89
Talairach’s stereotactic warpings, 26, 28n11 cortical region responses, 89–92
TE, 13 discussion, 92–94
Temporoparietal region, movement patterns, 164 neural response, object categories, 88–89
Threatening facial signals, see Facial signals, threatening object form topography, 93
Tichner, Edward, 131 population encoding, visual appearance, 93–94
TOH (Tower of Hanoi), 31 Ventromedial (VM) cortex, lesions, 224
ToM (“Theory of Mind”), 153–154, 171–181 Ventromedial (VM) prefrontal cortex
definition, 172n1 damage results, 235
discussion, 176, 178, 179–180 emotional and social intelligence, 223
emotional and social intelligence, 234 Ventromedial frontal region, decision making, 27
introduction, 171–172 “Visual oddball” paradigm, 44
methods, 173, 176–177 Visual perception, see Biological motion, visual perception
results, 173–176, 177–178 Voxel-based analysis, 45
selective attention, 180n2
Topography, object form, 93 W
TPJ (temporo-parietal junction) WAIS-III Wechsler Adult Intelligence Scale, 229
selective attention, 179–180, 180n2 WAIS-R (Weschler Adult Intelligence Scale-Revised, 31, 231
ToM (“Theory of Mind”), 171–173, 176 WCST (Wisconsin Card Sorting Test), 31, 229, 231
TR (Interpulse delay), 13 Wobbling (precession), 4
Trustworthiness; see also Faces, judgments of Women subjects, emotion, 265, 267n4
trustworthiness WRAT-R (Wide Range Achievement Test-Revised), 31
RT0996_Color Plates.qxd 11/8/04 2:16 PM Page 281
Color Plates
•
FIGURE 1.2 ■ View of the entry-level area with the a priori most likely first trajectory. (A) Skull
with this first vector and the level (red) at which entry points were marked. (B) View of a seg-
ment of section 1. On the left is the mandibular ramus, and on the right is the array of entry
points. (C) Enlargement of the array of entry points. One additional point was added (L20) to
ensure that every viable entry point was surrounded by nonviable points. Nonviable vectors
are shown in red, and viable vectors with labels identifying their exit points are shown in
green. Abbreviations: A, anterior; L, lateral; P, posterior; AM, anteromesial; AL, anterolateral;
PL, posterolateral; C, central.
■ Key Readings
FIGURE 1.3 ■ (A) View from above the deformed skull with
the exit hole and the anterior bone flap traced in black. The
blue circle represents the first vector tested, and the gray sur-
face represents the area where exit points were tested. (B)
Schematic enlargement of the exit hole and of the area tested
for exit points. The letter C marks the first tested vector (blue).
The numbers 1 through 15 mark the other exit points tested.
Red indicates nonviable vectors, green indicates viable vec-
tors, and the label identifies the entry point. Note that the a
priori best fit C was not viable.
FIGURE 1.4 ■ (A) Front and lateral skull views with the projection of the five final vectors (V). The
two red lines show the position of the two sections seen in (B). (B) Skull sections 2 and 3: ex-
amples of two bottleneck levels at which the viability of vectors was checked. Next to each sec-
tion is an enlargement of the critical area. Abbreviations: T, missing tooth; M, intact mandible; Z,
intact zygoma with a chipped area (light blue).
Color Plates ■
FIGURE 1.5 ■ Normal brain fitted with the five possible rods. The best rod is highlighted in solid white [except
for (B), where it is shown in red]. The areas spared by the iron are highlighted in color: Broca, yellow; motor,
red; somatosensory, green; Wernicke, blue. (A) Lateral view of the brain. Numbered black lines correspond
to levels of the brain section shown in (C). (D and E) Medical view of left and right hemispheres, respectively,
with the rod shown in white.
■ Key Readings
FIGURE 3.1 ■ Anterior-posterior (A-P) distribution of pre-

frontal cortex activation. (e) Group-averaged t test results
(P 0.001 uncorrected) for the contrast between emo-
tional distracters (plotted in blue spectrum) and attentional
targets (plotted in red spectrum). Attentional target activity
was observed in left MFG (BA 9/46; Talairach coordinates
-36, 35, 30) and right MFG (BA 9/46; 44, 35, 31). Emotional
distracter activity was observed in left IFG (BA 45/47; -51,
33, 4) and right IFG (BA 45/47; 55, 33, 0). The coronal sec-
tion in E shows the single prefrontal slice where differential
activation between attentional targets and emotional dis-
tracters was most remarkable. However, peak activation to
emotional distracters was located ≈1 cm more posteriorly
within IFG. R, right hemisphere; L, left hemisphere.
FIGURE 4.1 ■ Activation maps show brain areas to be more active during Object trials than during Person trials. Re-
gions of modulation included the left inferior prefrontal cortex and the left IT cortex (a), as well as the left posterior pari-
etal and the left insula cortex (b). See Table 4.1 for the Talairach and Tournoux (49) atlas coordinates.
Color Plates ■
FIGURE 4.2 ■ Activation maps show brain areas to be more active during Person trials than during Object trials.
Regions of modulation included the left temporal sulcus (a), the dorsal and ventral MPFC (b), the right FuG (c), and the
right parietal temporal-occipital junction (d). See Table 4.2 for the Talairach and Tournoux (49) atlas coordinates.
■ Key Readings
FIGURE 5.1 ■ Brain regions activated by emotionally evoca-

tive moral (M) and nonmoral (NM) judgments compared to
neutral (NTR) ones. Activations were overlaid on sections
through an averaged brain from all subjects with inverted
grayscale and on 3-D renderings of a reference brain. (a) M
vs NTR condition. Activated regions were in the left or-
bitofrontal cortex (OFC) and in the superior temporal sulcus
and the left temporal pole. (b) NM vs NTR condition. Acti-
vated regions were in the left amygdala and lateral OFC,
and bilaterally in the visual cortex.
Color Plates ■
1 2 3
4 5
1 - left amygdala
2 - left lateral OFC
3 - V1
4 - left medial OFC
5 - left STS
FIGURE 5.2 ■ Mean signal changes of the left medial and lateral OFC, left amygdala, primary visual cortex (V1), and
the cortex of the left superior temporal sulcus (STS), obtained from averaged MR signal from all subjects. Curve colors
correspond to experimental conditions as follows: yellow, unpleasant condition; light blue, moral condition; green, neu-
tral condition; black, scrambled condition.
FIGURE 5.3 ■ Brain regions activated by the neutral (NTR) as com-

pared to scrambled (SCR) condition (both temporal lobes and frontal
opercula, supplementary motor area, anterior cingulate, basal gan-
glia, and thalamus).
■ Key Readings
a)
b)
FIGURE 6.1 ■ MEG data from a typical subject. (a) Pairwise t-tests
between the responses at each sensor reveal early (M100) and late
(M170) significant differences in the MEG response to faces versus
houses over occipitotemporal cortex. (b) The MEG waveforms are
averaged across all face and house trials at a typical sensor of inter-
est in the right hemisphere. Red, faces; blue, houses; black, t-values.
The left vertical scale indicates the amplitude of the MEG response
(1013 tesla) whereas the right one shows the t-value. A value t
1.99 (horizontal green line) corresponds to p 0.05 (uncorrected for
comparisons at multiple time points).
FIGURE 7.3 ■ The category specificity of patterns of response was analyzed with pairwise contrasts between within-
category and between-category correlations. The pattern of response to each category was measured separately from
data obtained on even-numbered and odd-numbered runs in each individual subject. These patterns were normalized
to a mean of zero in each voxel across categories by subtracting the mean response across all categories. Brain im-
ages shown here are the normalized patterns of response in two axial slices in a single subject. The left side of the brain
is on the left side of each image. Responses in all object-selective voxels in ventral temporal cortex are shown. For each
pairwise comparison, the within-category correlation is compared with one between-category correlation. (A) Compar-
isons between the patterns of response to faces and houses in one subject. The within-category correlations for faces
(r 0.81) and houses (r 0.87) are both markedly larger than the between-category correlations, yielding correct
identifications of the category being viewed. (B) Comparisons between the patterns of response to chairs and shoes in
the same subject. The category being viewed was identified correctly for all comparisons. (C) Mean response across
all categories relative to a resting baseline.
Color Plates ■
■ Key Readings
FIGURE 8.2 ■ Summary of regions of interest (ROIs) and BOLD responses in biological related stimulus conditions.
(A) The ROIs in the right hemisphere of observer D.L. are displayed on the lateral and ventral surfaces of the gray
matter. A cut, as indicated by the green plane, was made and the posterior end of the cortex flattened. We examined
BOLD signals in four regions of interest: STSp (red), EBA (purple), LOCd (blue), FFA and OFA (orange). (B–E) The
average BOLD activity levels for these ROIs (with FFA and OFA averaged) during the stimulus conditions depicting
some kind of biological object, or the scrambled biological motion vectors. These stimulus conditions included anima-
tions of point-light biological motion (pink), point-light scrambled motion (yellow), whole-bodies (dark purple), pictures
of faces (magenta), and stationary images of headless bodies (green). The percent change activation levels are relative
to a fixation baseline. Error bars indicate 1 standard error.
Color Plates ■
mouth
Calvert et al. lip reading (STG)eyes

Calvert et al. lip reading (AG) Puce et al. eye gaze
Puce et al. mouth movement Wicker et al. eye gaze
Puce & Allison mouth movement Hoffman & Haxby eye gaze
body hand
Howard et al. body movement Neville et al. ASL
Bonda et al. body movement Bonda et al. hand action
Senior et al. body movement Grezes et al. hand action
Kourtzi & Kanwisher Grezes et al. hand movement
body movement Grafton et al. hand grasp
Grossman et al. body movement Rizzolatti et al. hand grasp
FIGURE 9.3 ■ Centers of activation to viewing the face, hand and body movements of
others obtained from a series of PET and fMRI studies. (Adapted from Allison et al. (2000),
with permission.)
■ Key Readings
FIGURE 9.4 ■ ERPs elicited to a social attention task. (b) Voltage maps for the three viewing conditions
generated at the peak of P400 activity for the group attention condition (black arrow in (a)). The group
attention condition shows fronto-temporal positivity, whereas the other two conditions show small pos-
terior positivities.
Color Plates ■
FIGURE 10.1 ■ Observation of mouth actions. Projections of the acti-

vation foci on the lateral surface of a standard brain [Montreal Neuro-
logical Institute (MNI)] during the observation of non-object-related
(chewing: a) and object-related (biting an apple: b) mouth actions.
FIGURE 10.2 ■ Observation of hand actions. Projections of the activa-

tion foci on the lateral surface of a standard brain (MNI) during the ob-
servation of non-object-related (mimicking grasping of a cup or a ball,
without object: a) and object-related (grasping a cup or a ball: b) hand
actions.
■ Key Readings
FIGURE 10.3 ■ Observation of foot actions. Projections of the activa-

tion foci on the lateral surface of a standard brain (MNI) during the ob-
servation of non-object-related (mimicking kicking a ball or pushing a
brake, without the object: a) and during the observation of object-
related foot actions (kicking a ball or pushing a brake: b) foot actions.
FIGURE 10.4 ■ Somatotopy of premotor and parietal cortices as re-

vealed by action observation. (a) Observation of non-object-related
actions. (b) Observation of object-related actions. Activation foci,
shown in detail in the three previous figures, are projected on the lat-
eral surface of a standard brain (MNI). Red, activation during the ob-
servation of mouth movements; green, activation during the observa-
tion of hand movements; blue, activation during the observation of
foot movements. Overlap of colours indicates activation foci present
during observation of actions made by different effectors.
Color Plates ■
FIGURE 11.1 ■ Peaks of activations in the right central (labeled M1) and precentral (labeled
PMC) sulcus. The peak labeled M1 (x 44, y 10, z 36) corresponds entirely (considering
spatial resolution and variability factors) with meta-analytic PET data (x = 48 5.2, y 9
5.6, z 35 5.5) for the mouth region of human primary motor cortex. The peak labeled PMC
(x 48, y 8, z 28) corresponds well with previously reported premotor mouth (x 48, y
0, z 32) peaks.
FIGURE 11.3 ■ Activations in the right insula (green) and right (blue) and left (red) inferior frontal
cortex. Relative time-series are coded with the corresponding colors. The time-series have been
normalized to the overall activity of each region. The activity profile of these three regions is ex-
tremely similar throughout the whole series of tasks.
■ Key Readings
FIGURE 11.4 ■ Significantly increased activity in the right

amygdala during imitation of emotional facial expressions
compared with simple observation.
FIGURE 13.1 ■ Experiment 1. Random effects analysis, P 0.05,

corrected, n 25. Theory of mind mechanical inference stories.
Crosshair marks the most significant voxel in the left TPJ (1). Also visible
are activations in right TPJ (2), left aSTS (3), and precuneus (4). TPJ,
temporo-parietal junction; aSTS, anterior superior temporal sulcus.
Color Plates ■
a)
b)
FIGURE 13.3 ■ (a) Experiments 1 and 2. Activation overlap within an individual subject showing bilateral temporo-
parietal junction (bilateral TPJ) and precuneus regions (fixed effects, P 0.001). Red theory of mind mechanical
inference (Exp. 1). Blue false belief false photo (Exp. 2). Green both. (b) Single subject time course of response
during Experiment 2 to false belief (dark gray) and false photograph (white) stories in the same subject’s TPJ-M,
independently defined by a greater response to theory of mind than to mechanical inference stories in Experiment 1;
P 0.0001, uncorrected. Medium gray indicates fixation. Time course averaged over four runs.
■ Key Readings
FIGURE 14.1 ■ Face-place object selection attention task, a, Example stimulus. Observers were presented with color-
coded superimposed faces (disgusted, fearful, and neutral expressions in red) and places (inside and outside of build-
ings in green). Before each test stimulus, observers were presented with a color-coded prompt indicating which task
they were to perform on that trial: indicate the gender of the face (attend trials) or indicate the location of the place (un-
attend trials). b, A representative subject demonstrated a greater response when attending to places (in green) in a bi-
lateral region along the collateral sulcus, consistent with the PPA and a greater response when attending to faces (in
red) in the right middle fusiform gyrus, consistent with the FFA.
Color Plates ■
FIGURE 14.2 ■ Attentional dependence of amygdala and anterior insular responses

to facial expressions. a, The amygdala was functionally defined by the group level
contrast of fear relative to neutral trials when faces were attended. This resulted in
a prominent activation in the right amygdala (at a peak height x,22;y,1;z,28;F(1,11)
20.52;p 0.0001). b, Effect of stimulus and attention on amygdala response. Peak
amygdala response is displayed for each facial stimulus type during attended (red)
and unattended (green) conditions. Attention did not significantly reduce the mag-
nitude of amygdala response to fear, but the enhanced response to disgust during
reduced attention suggests attention influenced the specificity of amygdala re-
sponse. c, The insula was functionally defined by contrasting activation on disgust
trials compared with neutral trials when faces were attended. This resulted in a
prominent activation in the right anterior insula (at a peak height x,44;y,5;z,
14;F(1,11) 32.72,p 0.0001). d, Effect of stimulus and attention on anterior in-
sular response. Peak anterior insular response is displayed for each facial stimulus
type during attended (red) and unattended (green) conditions. Reduced attention
significantly reduced the magnitude of anterior insular response to disgust.
■ Key Readings
FIGURE 14.3 ■ Response to disgust faces when unattended. a, Amygdala response to

disgust relative to neutral faces when observers were attending to faces. No significant
activation was found when faces were attended. b, Amygdala response to disgust rel-
ative to neutral faces when observers were attending to places. Activation was pres-
ent when disgust faces were unattended. c, Time course of the disgust response dif-
ference score (unattended minus attended). A negative deflection of time course
represents a decreased response when faces were attended. A positive deflection
represents an increased response when faces were unattended. An inverse effect of
attention on anterior insula and amygdala response to disgust faces peaked ~6 sec af-
ter the stimulus onset.
Color Plates ■
FIGURE 15.3 ■ Main effect of trustworthiness in

amygdala and insula. (a) Significant increases in
BOLD signal to untrustworthy faces in the right
and left amygdalae and right insula (right amyg-
dala, 18, 0, 24; Z 4.29; p 0.01 corrected;
left amygdala, 16, 4, 20; Z 3.92; p
0.025 corrected; right insula, 42, 4, 12; Z
3.48; p 0.001 uncorrected).
FIGURE 15.2 ■ Main effect of explicit
social judgments. (a) Random-effects
SPM overlaid on a normalized structural
scan from a single subject showing acti-
vation in right superior temporal sulcus
region (x, y, z 56, 44, 4; Z 4.27;
p 0.05 small volume corrected) when
making judgments about trustworthi-
ness compared to age. For illustration,
using threshold p 0.001 uncorrected,
extent threshold of 5 voxels.
FIGURE 15.5 ■ Main effect of trustworthiness in

amygdala independent of facial emotion. Signifi-
cant increases in BOLD signal in response to un-
trustworthy faces in right amygdala even when
scores for four basic facial emotions are addi-
tionally used as parametric covariates in the
analysis. This activation is significant at p
FIGURE 15.4 ■ Main effect of trustworthiness in 0.05, corrected for multiple comparisons across
fusiform gyrus. (a) Significant increases in BOLD the volume of bilateral amygdala. Activation
signal to untrustworthy faces in the fusiform gyrus peak at 18, 2, 22 (Z 4.06), but overlaps with
bilaterally (right, 44, 46, 22; Z 3.58; p right amygdala activation focus shown in Figure
0.05, small volume corrected; left, 48, 48, 15.2. At lower threshold of p 0.005 uncor-
24; Z 3.60; p 0.05, small volume corrected). rected, activation is evident in left amygdala.
■ Key Readings
FIGURE 16.2 ■ Activation related to the presentation of an unfair offer. (a) Map of the t statistic
for the contrast [unfair human offer – fair human offer] showing activation of bilateral anterior in-
sula and anterior cingulate cortex. Areas in orange showed greater activation following unfair
as compared with fair offers (p 0.001). (b) Map of the t statistic for the contrast [unfair human
offer – fair human offer] showing activation of right dorsolateral prefrontal cortex.
FIGURE 18.3 ■ [18F]FCP binding potential increases in dominant monkeys. Normalized, co-reg-
istered PET images (percent injected dose per ml) of [18F]FCP binding in the basal ganglia of
a dominant and a subordinate monkey, while individually housed and socially housed.
Color Plates ■
Right Left
LPFC
MOFC
Bottom Front
LPFC
MOFC
Modulation Activation by
by reappraisal reappraisal
FIGURE 19.2 ■ Group-averaged brain activations when reappraising or attending to feelings in response to the most
negative photos. Two contrasts are shown: The Attend > Reappraise (shown in red) contrast shows regions important
for emotion processing that are significantly modulated by reappraisal and the Reappraise > Attend (shown in green)
contrast shows regions significantly activated when exerting cognitive control over emotion activated by reappraisal.
Top and bottom brain images on the right show regions of the left dorsal and ventral LPFC associated with cognitive
control that were activated by reappraisal. Right side and bottom left brain images show reappraisal-related modulation
of a region of left MOFC associated with representing the affective properties of stimuli.
■ Key Readings
Z-value
FIGURE 19.4 ■ Coronal image showing the group-averaged cluster of activation in right amygdala for the Attend
Reappraise contrast for trials with the most negative photos (p .005). The focus is centered on MNI coordinates (16,
12, 20). Activation is shown on group-averaged anatomy.

Social Neuroscience Key Readings

Uploaded by

Copyright:

Available Formats

Social Neuroscience Key Readings

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Social Neuroscience Key Readings

Uploaded by

Copyright:

Available Formats

What is the aim of the Key Readings in Social Psychology series?

What is the aim of the Key Readings in Social Psychology series?

What structure will each volume in the series have?

What structure will each volume in the series have?

SOCIAL NEUROSCIENCE

Key Readings in Social Psychology

General Editor: ARIE W. KRUGLANSKI, University of Maryland at College Park

• an overview chapter, as well as introduction to sections and articles

Published in Great Britain by

Copyright © 2005 by Taylor & Francis Books, Inc.

This edition published in the Taylor & Francis e-Library, 2005.

Psychology Press, is an imprint of the Taylor & Francis Group.

Library of Congress Cataloging-in-Publication Data

Social neuroscience: key readings/edited by John T. Cacioppo and Gary G. Berntson.

ISBN 0-203-49619-1 Master e-book ISBN

ISBN 0-203-59512-2 (Adobe eReader Format)

About the Editors ix

Appendix: How to Read a Journal Article in Social

Author Index 281

About the Editors

John T. Cacioppo is the Tiffany and Margaret Blake Distinguished

Gary G. Berntson is a professor of psychology, psychiatry, and

400–404. Copyright © 2001 by Blackwell Pub- Trustworthiness of Faces. Nature Neuroscience,

Among the major evolutionary advances in humans is the striking

2 ■ Social Neuroscience: Key Readings

Part 1: Volume Overview ■ 3

4 ■ Social Neuroscience: Key Readings

Part 1: Volume Overview ■ 5

Coarse perceptual Detailed perceptual

Motivational evaluation Representation of

Representation of Social reasoning

6 ■ Social Neuroscience: Key Readings

Part 1: Volume Overview ■ 7

8 ■ Social Neuroscience: Key Readings

Part 1: Volume Overview ■ 9

10 ■ Social Neuroscience: Key Readings

Part 1: Volume Overview ■ 11

12 ■ Social Neuroscience: Key Readings

2.99 Tesla field 3.01 Tesla field

3 Tesla applied field

FIGURE 2 ■ MRI and the localization of RF emissions

Part 1: Volume Overview ■ 13

14 ■ Social Neuroscience: Key Readings

Part 1: Volume Overview ■ 15

16 ■ Social Neuroscience: Key Readings

Part 1: Volume Overview ■ 17

The Brain Determines Social Behavior:

There is an intuitive appeal to viewing a social psychological construct or

20 ■ Social Neuroscience: Key Readings

The Return of Phineas Gage: Clues about the

22 ■ Social Neuroscience: Key Readings

The Return of Phineas Gage ■ 23

behavioral defects, which he aptly described as a

24 ■ Social Neuroscience: Key Readings

FIGURE 1.2 ■ (A color version of this figure follows page

FIGURE 1.3 ■ (A color version of this figure follows page

The Return of Phineas Gage ■ 25

26 ■ Social Neuroscience: Key Readings

FIGURE 1.5 ■ (A color version of this figure follows page 146.)