DeJaeger Etal 2016
DeJaeger Etal 2016
DeJaeger Etal 2016
& 2016 The Author(s) Published by the Royal Society. All rights reserved.
Downloaded from http://rstb.royalsocietypublishing.org/ on August 4, 2017
rstb.royalsocietypublishing.org
tactile
feedback
Figure 1. Set-up for perceptual crossing experiment. Both participants are isolated, each controlling the position of a sensor along a shared virtual one-dimensional
line using a computer mouse. The squares on each side of the line represent the objects that can be sensed by each participant, respectively. Objects are identical in
size. When the sensor touches an object the participant gets a tactile feedback on the finger (green circle). Each participant can sense only three objects, a static one
(black square), the sensor of the other participant (red square) and a ‘shadow’ object that copies exactly the movement of the other’s sensor at a fixed distance (blue
square). (Copyright & 2010 De Jaegher et al. [2]. Licenced under Creative Commons Attribution 3.0 Unported, http://creativecommons.org/licences/by/3.0).
operation of brain mechanisms involved in social cognition, brain, in isolation, make a determining contribution. It is
whether the person is engaged in an interactive situation or not’ ([1], the relations between such external events, the body, and
p. 2). We use here the terminology introduced in [2]: an enabling the brain that matter, or rather, it is only within these
factor is causally necessary for a phenomenon to occur, while a relations, which are not merely contextual, that we can
constitutive factor is part of what makes the phenomenon make sense of brain function during most social cognition.
what it is. While a hypothesis rather than a scientific claim, the In such cases, there is no factoring out of such extra-brain
IBH stimulates a novel perspective on how social neuroscientists elements without removing at the same time something
should construe information processing that generates social be- essential to social cognition as such.
haviour. The unit of analysis is no longer delimited to the brain, What are those instances of social cognition where we
but broadened to include aspects of the social environment with expect extra-neural relational patterns to play a constitutive
which the brain interacts or has interacted. role? Certainly, those involving direct interaction with
There is a range of hypotheses about the role that embodi- others. But also those instances involving the presence of
ment and social context play in social cognition. The weakest others to various degrees ( physical, virtual, etc.) and which
claim is that social interaction is methodologically useful in predispose us to engage interactively, even if we do not or
social neuroscience, as it provides ecological validity and cannot actualize such dispositions.
engages research participants. Another weak claim is that Consider an example where, we argue, social interaction
social interaction needs to be considered as providing important plays a constitutive role in the performance of a social task.
contextual modulation of the social brain. To our knowledge, This is the perceptual crossing experiment by Auvray et al. [4]
nobody disagrees with these claims. They are now actively (figure 1). Two blindfolded participants are told to freely
pursued in social neuroscience, as evidenced in the other move a sensor (a computer mouse) along a shared virtual line.
contributions in this issue; we do not further treat them here. In this virtual ‘world’, each participant can encounter two differ-
A stronger claim is that social interaction facilitates particu- ent kinds of objects, one fixed in space and two moving objects.
lar kinds of brain processes: that is, there is a strong enabling One of the moving objects corresponds to the scanning sensor of
role for social interaction [2]. For instance, it is well known the other participant, and the other is attached to this sensor at a
that development in the absence of social interaction (severe fixed distance, like a ‘shadow’. In terms of their trajectories, the
social deprivation, in humans or other mammals) results in a moving objects are indistinguishable. Whenever her sensor
highly abnormal brain with highly abnormal cognition [3]. encounters an object on this line, the participant receives an
This fact also suggests important constraints on the design of on/off tactile stimulus—a tap on the finger, which is the same
artificial cognitive systems. To our knowledge, this claim is for each kind of object. This situation is symmetrical for both
also uncontentious. We do not discuss this enabling aspect of participants. Note that when a participant’s sensor encounters
social cognition here either. the shadow of the other participant, only the first participant
The hypothesis we discuss here concerns an occurrent will receive a tactile stimulus. When the two sensors meet,
instance of social cognition. We claim that a normal adult both participants receive a stimulus simultaneously.
human brain in isolation is insufficient for a typical instance of Participants are instructed to click the mouse button when-
social cognition. Of course, we acknowledge that the brain ever they judge they are in contact with the other participant.
plays a large, and probably major, role in social cognition.1 As a result, statistically, mouse clicks tend to concentrate on
But processes occurring in that brain at the time of an each other’s sensors (65.9% of clicks) and not on the identically
instance of social cognition are, in the typical case, not fully moving shadow objects (23%). This means that participants
constitutive of social cognition: additional events are also can find each other by ‘perceptually crossing’ their scanning
required. Those additional events involve relations between activities. However, the authors find that the probability of
the brain and ( parts of ) the rest of the world. It is important clicking following a stimulus is approximately the same
to note that we are not claiming that events external to the whether this stimulation comes from the other’s sensor or its
Downloaded from http://rstb.royalsocietypublishing.org/ on August 4, 2017
shadow. Auvray et al. explain this result in terms of the collec- there is a more or less continuous stream of causal interaction 3
tive dynamics of the interactive configuration. Each participant between brain and the world, and, in the case of the social
rstb.royalsocietypublishing.org
attempts to re-scan many times an object that is apparently world, ‘outputs’ from a brain influence ‘inputs’ (i.e. one
moving, by making back and forth mouse movements. When person influences another) in such a tightly coupled way that
this scanning involves an encounter between both sensors, it becomes impossible to distinguish input from output.
the participants tend to continue this activity for a long time. Instead, say advocates of the IBH, one should treat the whole
By contrast, when one of them is scanning the other’s system (two or more people interacting) as a single, dynami-
shadow, which is moving independently of this scanning cally coupled system. Cognition is constituted by the events
movement, this ‘encounter’ is short-lived. This means that in my brain, the events in the other person’s brain, and the
the dyadic system is organized such that sensor–sensor causal relations between them: the whole system matters.
encounters are more frequent than sensor–shadow encounters, I am sympathetic with what motivates the IBH, and of
which explains why participants can ‘find each other’ in spite course I agree that the classical computer metaphor is
extraordinarily dense and complex set of causal interactions that might ultimately ground what it is that the brain’s 4
that are part of an extensive processing architecture. What- representations are about; see the concluding section).
rstb.royalsocietypublishing.org
ever exactly one’s view on what cognition is, it is far more It may well be that ‘causal density’ as I have described it
than a reflex, far more than a fixed action pattern, and instead above is just a proxy for another property that is more fundamen-
is a highly inferential, context-dependent and flexible form of tal to cognition. Perhaps it is computational complexity. Perhaps
information processing. So far as we know, only the brains it is something that requires much more clarification, like ‘own-
of certain animals can generate examples of it. The causal ership’ for a person. Perhaps it is something like ‘manipulability’
interactions between those brains and the rest of the world that could ground our concept of causation; one could imagine
are simply too ‘thin’. manipulability as experimental manipulability by us, or as bio-
This brings me to my core objection against the IBH as a logical manipulability in terms of what is accessible to
hypothesis about cognition: in widening the causal base of evolution or development. Much more debate will be needed
cognition, it negates a distinction that is critical to understand to develop arguments for any of these, but for present purposes
between participants (and their surroundings) can be a consti- distinct brains at frequencies more than one order of magnitude 5
tutive part of the processes of social cognition as they are faster than the interactive movements?
rstb.royalsocietypublishing.org
enacted by the individual participants involved [1,2]. The Leaving aside the question of what role (if any) might be
strong version of the IBH simply hypothesizes that this played by such cross-scale synchronization, the evidence
possibility is, in general, a widespread plausibility. This can suggests that interaction patterns produce an entanglement
be criticized in two ways: the extension from possibility to between the brains of the participants. Internally, the wave
plausibility is not empirically warranted, or the very claim of influence across various temporal and spatial scales may
of possibility is wrong. The criticisms of the previous section travel from low to high frequencies via variations in neuronal
are centred on the second option. If this possibility claim is excitability [22 –25]. These top-down effects, evidenced also
wrong, then the constitutive version of IBH falls with it and in arrhythmic cross-frequency couplings [26], have been
only the developmental version remains. associated with different cognitive phenomena, notably
We discuss three aspects in support of the constitutive with the control of visual attention [27–29]. From here it is
participants. We do not think that this is an example of ‘just properties of either agent on its own, or to a linear aggregation 6
behaviour’, if by this is meant that no sense-making is of these. The task is transformed from type II to type I—it is
rstb.royalsocietypublishing.org
involved. It is a powerful exemplar that, thanks to its simpli- solved—by the interaction process. There is no need for the par-
city, can help us think differently about individual and ticipants’ brains to represent the distinction between sensor
interactive processes in more complex cases. and shadow at all to solve the task. The participants reap the
The perceptual crossing task is anything but simple. benefits and deal with quasi-disambiguated, type I stimuli:
It is only after the performance has been explained that it ‘if it moves but stays nearby (repeated crossings), then click’.
appears so. In fact, described in strict computational terms, If a process instantiates the solution to a cognitive problem
it is a highly ambiguous, type II problem [41], i.e. a problem it constitutes an instance of cognition. This is what social
where stimuli must be actively discriminated spatially and interaction does in perceptual crossing.
qualitatively using only temporal and proprioceptive cues Further empirical confirmation that social interaction can
(all ‘objects’ found in the virtual space produce the exact play constitutive roles in social cognition is provided by a vari-
the remainder of determination is given by the relational one that is more amenable to scientific observation and 7
dynamics of the interactive encounter. We call this process experimental manipulation.
rstb.royalsocietypublishing.org
participatory sense-making [43].
Consider escalation as a simple example of what we
mean by irreducibility in the case of interactive phenomena.
Typically, escalation involves an antagonistic pattern of 5. Response (R.A.)
interaction, sustained in time, increasing in intensity, and My co-authors are correct in taking me to reject the possi-
potentially spiralling out of control. Past conflictive inter- bility, not just the plausibility, of the IHB in what I wrote in
actions can predispose the onset of escalation even when §3. However, this depends on three concepts, whose relations
the interaction partners do not individually intend to were argued for only in the vaguest terms; let me say a bit
engage in an antagonistic exchange (see for instance, [44]). more about them here in responding to the arguments of
Sometimes escalation arises spontaneously as a result of §4. The three concepts are cognition, causation and explana-
brain. The only other good metric that comes to mind is itself can show emergent dynamical patterns that do not 8
something like ‘evolvability’ or ‘manipulability by evolution’. reduce to the activity of the mice. This conclusion is conso-
rstb.royalsocietypublishing.org
That is, there is a strong intuition that evolution can direct nant with Cosmelli & Thompson’s suggestions about the
changes in cognition through changes in the brain, but not brain-in-a-vat thought experiment [18]. The upshot is that
changes in the physical environment. Unfortunately, this we may be able to describe some aspects of cognition reason-
intuition only works for the non-social environment. For ably well as input –output transformations by the brain,
the social environment, there is instead a strong plausibility whereas others cannot be so described. Social cognition
that brains co-evolved, and so cognition indeed could typically may be of the latter kind.
evolve through changes across multiple brains. This leads us to consider the practical issue of which
To summarize: my current concept of cognition, however are the criteria for delineating the systems under study.
ill-defined, is squarely centred on the brain. But I have not One possible reading of the IBH could be that everything
made a serious attempt to revise this, and it is possible that matters, and so there is no right decomposition into causal
rstb.royalsocietypublishing.org
not just in the context of meaningful situations, but also in the cient for understanding cognition. From an enactive perspective,
context of evolution and development. Echoing the well- what matters is the embodied subject in relation to her world or
known ethological refrain that ‘nothing makes sense in meaningful environment. Interestingly, the same kind of argument
biology except in the light of evolution’ ([56], p. 125), we in favour of the IBH presented here in the context of social neuro-
science could be made in the context of many approaches to
would urge that social neuroscience should incorporate at
embodied cognition that remain methodologically individualistic.
least comparative neuroscience, developmental neuroscience, For these, the body is crucial for the mind, meaning the individual
and input from sciences that study social interaction into its body and not its engagement in social interactions. From the enactive
domain of study (see e.g. [57]). perspective, by contrast, both body and social engagement are
primordial [1,2].
Authors’ contributions. E.D.P. and H.D.J. are the primary authors of §§2 2
The situation is not unlike other debates in biology. Arguments for
References
1. Di Paolo EA, De Jaegher H. 2012 The interactive 13. Clark A, Chalmers DJ. 1998 The extended mind. 25. Le Van Quyen M. 2011 The brainweb of cross-scale
brain hypothesis. Front. Hum. Neurosci. 6, 1–16. Analysis 58, 7–19. (doi:10.1093/analys/58.1.7) interactions. New Ideas Psychol. 9, 57 –63. (doi:10.
(doi:10.3389/fnhum.2012.00163) 14. Rowlands M. 2003 Externalism: putting mind and 1016/j.newideapsych.2010.11.001)
2. De Jaegher H, Di Paolo EA, Gallagher S. 2010 Can social world back together again. Georgetown, DC: 26. He BJ, Zempel JM, Snyder AZ, Raichle ME. 2010 The
interaction constitute social cognition? Trends Cogn. Sci. Acumen/McGill-Queens University Press. temporal structures and functional significance of
14, 441–447. (doi:10.1016/j.tics.2010.06.009) 15. Byrge L, Sporns O, Smith LB. 2014 Developmental scale-free brain activity. Neuron 66, 353–369.
3. Harlow HF, Harlow MK. 1962 Social deprivation in process emerges from extended brain –body– (doi:10.1016/j.neuron.2010.04.020)
monkeys. Sci. Am. 207, 136–146. (doi:10.1038/ behavior networks. Trends Cogn. Sci. 18, 395–403. 27. Buschman TJ, Miller EK. 2007 Top-down versus
scientificamerican1162-136) (doi:10.1016/j.tics.2014.04.010) bottom-up control of attention in the prefrontal and
4. Auvray M, Lenay C, Stewart J. 2009 Perceptual 16. Simon HA. 1962 The architecture of complexity. posterior parietal cortices. Science 315, 1860–1862.
interactions in a minimalist virtual environment. Proc. Am. Philos. Soc. 106, 467–482. (doi:10.1126/science.1138071)
New Ideas Psychol. 27, 32 –47. (doi:10.1016/j. 17. Anderson ML, Richardson MJ, Chemero A. 2012 28. Saalmann YBP, Pigarev IN, Vidyasagar TR. 2007
newideapsych.2007.12.002) Eroding the boundaries of cognition: implications of Neural mechanisms of visual attention: how top-
5. Clark A. 1997 Being there: putting brain, body and embodiment. Top. Cogn. Sci. 4, 717 –730. (doi:10. down feedback highlights relevant locations. Science
world together again. Cambridge, MA: MIT Press. 1111/j.1756-8765.2012.01211.x) 316, 1612– 1615. (doi:10.1126/science.1139140)
6. Kelso JAS. 1995 Dynamic patterns: the self- 18. Cosmelli D, Thompson E. 2011 Brain in a vat or 29. Lakatos P, Karmos G, Mehta AD, Ulbert I, Schroeder
organization of brain and behaviour. Cambridge, body in a world? Brainbound versus enactive views CE. 2008 Entrainment of neuronal oscillations as a
MA: MIT Press. of experience. Philos. Top. 39, 163– 180. (doi:10. mechanism of attentional selection. Science 320,
7. Noë A. 2004 Action in perception. Cambridge, MA: 5840/philtopics201139119) 110–113. (doi:10.1126/science.1154735)
MIT Press. 19. Babiloni F, Astolfi L. 2014 Social neuroscience and 30. Astolfi L, Toppi J, de Vico Fallani F, Vecchiato G,
8. Findlay JM, Gilchrist ID. 2003 Active vision: the hyperscanning techniques: past, present and future. Salinari S, Mattia D, Cincotti F, Babiloni F. 2010
psychology of looking and seeing. Oxford, UK: Oxford Neurosci. Biobehav. Rev. 44, 76 –93. (doi:10.1016/j. Neuroelectrical hyperscanning measures
University Press. neubiorev.2012.07.006) simultaneous brain activity in humans. Brain
9. Przyrembel M, Smallwood J, Pauen M, Singer T. 20. Simon HA. 1969/1996 The sciences of the artificial. Topogr. 23, 243–256. (doi:10.1007/s10548-010-
2012 Illuminating the dark matter of social Cambridge, MA: MIT Press. 0147-9)
neuroscience: considering the problem of social 21. Dumas G, Nadel J, Soussignan R, Martinerie J, 31. Astolfi L et al. 2011 Imaging the social brain by
interaction from philosophical, psychological, and Garnero L. 2010 Inter-brain synchronization during simultaneous hyperscanning during subject
neuroscientific perspectives. Front. Hum. Neurosci. 6, social interaction. PLoS ONE 5, e12166. (doi:10. interaction. IEEE Intell. Syst. 26, 38 –45. (doi:10.
190. (doi:10.3389/fnhum.2012.00190) 1371/journal.pone.0012166) 1109/MIS.2011.61)
10. Koster-Hale J, Saxe R. 2013 Theory of mind: a 22. Canolty RT et al. 2006 High gamma power is phase- 32. Dumas G, Kelso JAS, Nadel J. 2014 Tackling the
neural prediction problem. Neuron 79, 836. (doi:10. locked to theta oscillations in human neocortex. social cognition paradox through multi-scale
1016/j.neuron.2013.08.020) Science 313, 1626–1629. (doi:10.1126/science. approaches. Frontiers in Psychology 5, 1 –4. (doi:10.
11. Clark A. 2013 Whatever next? Predictive brains, 1128115) 3389/fpsyg.2014.00882)
situated agents, and the future of cognitive science. 23. Womelsdorf T et al. 2007 Modulation of neuronal 33. Hari R, Kujala MV. 2009 Brain basis of human social
Behav. Brain Sci. 36, 181–204. (doi:10.1017/ interactions through neuronal synchronization. Science interaction: From concepts to brain imaging. Physiol.
S0140525X12000477) 316, 1609–1612. (doi:10.1126/science.1139597) Rev. 89, 453 –479. (doi:10.1152/physrev.00041.
12. Noë A. 2010 Out of our heads: why you are not your 24. Buzsáki G, Draguhn A. 2004 Neuronal oscillations in 2007)
brain, and other lessons from the biology of cortical networks. Science 304, 1926– 1929. (doi:10. 34. Kelso JAS, Dumas G, Tognoli E. 2013 Outline of a
consciousness. London, UK: Macmillan Press. 1126/science.1099745) general theory of behavior and brain coordination.
Downloaded from http://rstb.royalsocietypublishing.org/ on August 4, 2017
Neural Netw. 37, 120–131. (doi:10.1016/j.neunet. 42. Froese T, Iizuka H, Ikegami T. 2014 Embodied social 49. Fogel A. 1993 Developing through relationships: 10
2012.09.003) interaction constitutes social cognition in pairs of origins of communication, self and culture. London,
rstb.royalsocietypublishing.org
35. Van Orden GC, Holden JG, Turvey MT. 2003 Self- humans: a minimalist virtual reality experiment. Sci. UK: Harvester Wheatsheaf.
organization of cognitive performance. J. Exp. Rep. 4, 1–10. (doi:10.1038/srep03672) 50. Searle J. 1990 Consciousness, explanatory inversion,
Psychol. 132, 331–350. (doi:10.1037/0096-3445. 43. De Jaegher H, Di Paolo E. 2007 Participatory sense- and cognitive science. Behav. Brain Sci. 13, 585 –
132.3.331) making: an enactive approach to social cognition. 642. (doi:10.1017/S0140525X 00080304)
36. Dotov DG, Nie L, Chemero A. 2010 A demonstration Phenomenol. Cogn. Sci. 6, 485 –507. (doi:10.1007/ 51. Ramirez S et al. 2013 Creating a false memory in
of the transition from ready-to-hand to unready-to- s11097-007-9076-9) the hippocampus. Science 341, 387–391. (doi:10.
hand. PLoS ONE 5, e9433. (doi:10.1371/journal. 44. Granic I. 2000 The self-organization of parent-child 1126/science.1239073)
pone.0009433) relations: beyond bidirectional models. In Emotion, 52. Oyama S. 1985 The ontogeny of information:
37. Shockley K, Santana M-V, Fowler CA. 2003 Mutual Development, and self-organization. dynamic systems developmental systems and evolution. Cambridge,
interpersonal postural constraints are involved in approaches to emotional development (eds MD UK: Cambridge University Press.