99e9 PDF

BEHAVIORAL AND BRAIN SCIENCES (2001) 24, 195–261
Printed in the United States of America
On specification and the senses

Thomas A. Stoffregen
Division of Kinesiology, University of Minnesota, Minneapolis, MN 55455
[email protected]
www.homepages.uc.edu/~stoffrta/psi/index.html
Benoît G. Bardy
Division of Sport Sciences (STAPS), Université Paris Sud-XI, 91405 Orsay
Cedex France
[email protected]
Abstract: In this target article we question the assumption that perception is divided into separate domains of vision, hearing, touch,
taste, and smell. We review implications of this assumption for theories of perception and for our understanding of ambient energy ar-
rays (e.g., the optic and acoustic arrays) that are available to perceptual systems. We analyze three hypotheses about relations between
ambient arrays and physical reality: (1) that there is an ambiguous relation between ambient energy arrays and physical reality, (2) that
there is a unique relation between individual energy arrays and physical reality, and (3) that there is a redundant but unambiguous rela-
tion, within or across arrays, between energy arrays and physical reality. This is followed by a review of the physics of motion, focusing
on the existence and status of referents for physical motion. Our review indicates that it is not possible, in principle, for there to be a
unique relation between physical motion and the structure of individual energy arrays. We argue that physical motion relative to differ-
ent referents is specified only in the global array, which consists of higher-order relations across different forms of energy. The existence
of specificity in the global array is consistent with the idea of direct perception, and so poses a challenge to traditional, inference-based
theories of perception and cognition. However, it also presents a challenge to much of the ecological approach to perception and action,
which has accepted the assumption of separate senses.
Keywords: epistemology; information; intersensory; perception; perceptual learning; sensory neurophysiology; sensory systems; speci-
fication
1. Introduction teraction is specified in the spatiotemporal structure of am-

bient arrays. Specification refers to a lawful, 1:1 relation be-
One of the fundamental questions of perceptual theory is
tween patterns in ambient arrays and the aspects of the an-
whether the structured energy fields that are available to
imal environment interaction that give rise to them (Shaw
perceptual systems are sufficient, in and of themselves, for
et al. 1982). The ecological approach to perception and ac-
accurate perception. If potential sensory stimulation1 is not
tion is an established theory with broad empirical support,
sufficient, then accurate perception must depend upon op-
and for this reason we do not review it at length here (for
erations carried out by the animal, such as inferential pro-
cessing. Thus, the assumption that potential sensory stimu-
lation is insufficient for accurate perception leads to the
hypothesis that perception is indirect (i.e., accurate per- Thomas A. Stoffregen, Associate Professor in the
ception requires the addition, presumably mental, of infor- Division of Kinesiology at the University of Minnesota,
mation that is not available in sensory stimulation) which, is the author of over 40 publications in the area of per-
ception and action, including affordances, the percep-
in turn, leads theorists to focus on internal processing as the tion and control of spatial orientation, motion sickness,
locus of the most important issues in perception. On the virtual reality, audition, and perceptual-motor develop-
other hand, if potential sensory stimulation is sufficient for ment. He is a recipient of the Faculty Achievement
accurate perception, then perception can be direct, that is, Award from the University of Cincinnati, and his re-
accurate without the addition of information beyond what search has been funded by the National Science Foun-
is available in sensory stimulation. The latter view is central dation and the U.S. Navy. He is a Consulting Editor for
to the ecological approach to perception and action (Gib- Ecological Psychology, and serves on the Advisory Board
son 1979/1986). Proponents of the ecological approach for The Handbook of Virtual Environments Technology.
stress that ambient arrays are structured by the animal-en-
vironment interaction (that is, by the position and motion Benoît G. Bardy, Professor of Sport Sciences, is the
author of over 40 publications in the area of Human per-
of the animal relative to its environment), and that this ception and action, including coordination dynamics,
structuring is governed by physical law (i.e., laws of the the perceptual regulation of posture and locomotion,
propagation, reflection, and absorption of energy) in such a and the control of spatial orientation. He is the director
way that any given physical reality gives rise to a unique of the Center for Research in Sport Science at the Uni-
structure or pattern in ambient energy. This leads to the hy- versity of Paris Sud XI, and serves as an Associate Edi-
pothesis that potential sensory stimulation is sufficient for tor for Ecological Psychology.
accurate perception because the animal-environment in-
© 2001 Cambridge University Press 0140-525X/00 $12.50 195

Stoffregen & Bardy: On specification and the senses
general presentations of the ecological approach see Gib- while in other cases he suggested that information exists in
son 1966; 1986; Goldfield 1995; Michaels & Carello 1981; relations across forms of energy (sect. 6.1). We will argue
Turvey et al. 1981). that these positions are mutually exclusive. After present-
Specification is an hypothesis about the nature and sta- ing our view we discuss its relation to Gibson’s (sect. 6.2.2).
tus of ambient arrays before the stimulation of sensory re- [See also Ullman: “Against Direct Perception” BBS 3(3)
ceptors. Thus, the debate about specificity is not a psycho- 1980.]
logical debate. It is a debate about relations between states Our view of perception resembles contemporary dy-
of the world and the energy patterns to which those states namical theories of action, for which action consists of co-
give rise, prior to and independent of sensory stimulation ordination between distinct units, and should be defined at
or any psychological process (Gibson 1986; Kugler & Tur- the level of macroscopic variables, or order parameters
vey 1987; Reed 1996; Runeson & Frykholm 1983;). (e.g., Haken 1983; Kelso 1995; Thelen & Smith 1994; Tur-
In this target article we question existing approaches to vey et al. 1978). In dynamical theories of behavior, a given
the concept of specification, which are based on the as- action cannot be understood as the motion of a single mo-
sumption that specification exists (or does not exist) in in- tor “unit,” or as the additive contributions of the motions of
dividual forms of energy, such as the optic and acoustic ar- multiple units (Reed 1982). Similarly, for perception, we
rays. We present a novel argument for the existence of propose that there exist macroscopic variables, consisting of
specification. Because we question existing views that as- relations between different forms of ambient energy, that
sume the existence of specification, our analysis presents a these provide information about the animal-environment
challenge to theories based on these views, such as the eco- interaction; and that information exists only in these macro-
logical approach to perception and action. At the same scopic variables, that is, that it does not exist in the struc-
time, our argument for a new form of specification presents ture of individual forms of energy. In the present article we
a challenge to theories that assume that potential sensory do not claim that these informational macroscopic variables
stimulation bears an ambiguous relation to reality. Thus, are order parameters per se, exhibiting properties such as cir-
our analysis has consequences for theories of perception cular causality, enslaving, or time-scale conventions. Rather,
that are based on inferential processing. Our analysis im- we argue that with respect to specification the whole is not
plies that all theories of perception derived from existing only greater than, but qualitatively different from, the sum
views of specification are compromised by fundamental of the parts.
errors. We begin with a discussion of the assumption that there
Discussions of specification, both pro and con, have fo- exists a set of distinct perceptual systems that operate more
cused on the structure of single forms of energy, such as or less independently of one another, which we call the as-
light. However, behavior produces simultaneous changes in sumption of separate senses. We suggest that this assump-
the structure of multiple forms of ambient energy. For ex- tion may not be justified. This suggests the possibility of al-
ample, locomotion produces changes in the stimulation of ternative views of the senses. In section 3 we argue that the
(at least) the visual, vestibular, and somatosensory systems. assumption of separate senses leads to problems for exist-
Even the most elementary and pervasive acts, such as ing views of specification. These problems arise from the
breathing and controlling posture, produce changes in the assumption that specification exists in individual ambient
stimulation of multiple perceptual systems. This basic fact arrays, that is, in structures that may be sampled by sepa-
has had little influence on general theories of perception, rate senses. In section 4 we show that these problems ex-
and it has received little attention in discussions of specifi- tend to the level of physics. In section 5 we conclude that
cation. the concept of specification is incompatible with the as-
We suggest that the multisensory consequences of be- sumption of separate senses, and we discuss some general
havior may have fundamental implications for the nature of consequences of this for the interpretation of subjective
perception. We propose that perceptual systems do not judgments about motion. In section 6 we present an alter-
function independently, and that any attempt to understand native view of specification, which requires the rejection of
them independently must be fundamentally incomplete. the assumption of separate senses. Our decision to begin
Such a position has occasionally been argued (e.g., Sher- with the assumption of separate senses is for purposes of ex-
rington 1906; Welch & Warren 1986); however, our argu- plication, not from logical necessity. The argument could be
ment differs from others in important ways. We will at- presented in the reverse order, that is, our alternate view of
tempt to redefine perception, not as a process of picking up specification could be used to motivate a reconsideration of
information through a group of disparate “channels,” and the assumption of separate senses.
not as a set of interactions among discrete modalities, but
as the pick-up of information that exists in irreducible pat-
terns across different forms of energy. Consistent with the 2. The assumption of separate senses
ecological approach to perception and action (Gibson 1986)
we assume that behaviorally relevant aspects of reality are Throughout history, theories of perception have embodied
specified. However, we will argue that specification exists an assumption that perception is achieved via several sen-
only in patterns that extend across different forms of ambi- sory modalities. The assumption of separate senses under-
ent energy. lies virtually all theory and research on perception. It is as-
Our position is inspired in part by James Gibson’s (1966) sumed that there are multiple perceptual systems (the
theory of perceptual systems. However, we believe that number typically is five, but this is of secondary impor-
with respect to relations between the senses there are some tance). The senses are thought of as being “separate and in-
ambiguities in Gibson’s presentation. In some instances, teracting modalities” (Smith 1994, p. xi; cf. Bekesy 1959),
Gibson argued that information available to different per- such that the function of individual perceptual systems
ceptual systems is redundant (we discuss this in sect. 3.3.2), “provides basic information” that is needed in “under-
196 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

standing the interaction between or among these modali- from those that are peculiar to any one sense. However, this
ties” (Welch & Warren 1986, p. 3; emphasis added). Boring begs the question. Both his query and his argument assume
(1950, p. 182) referred to the division of perception by the prior existence of separate senses. Only under this as-
senses as one of psychology’s “primary principles of classi- sumption does it make sense to ask why we have more than
fication.” one.
The assumption of separate senses may seem to be so Later philosophers have not addressed the issue at all.
self-evident as to be atheoretical (i.e., free of implications Locke (1689/1975), in discussing the origin of ideas, ap-
for theories of perception). We will argue that the assump- peared to take for granted the existence of distinct senses:
tion carries profound theoretical implications. In section 6 Our senses, conversant about particular sensible objects, do
we will present an alternative view, in which perception is convey into the mind, several distinct perceptions of things, ac-
not divided into distinct perceptual systems. If there are cording to those various ways, wherein the objects do affect
credible alternatives to the assumption of separate senses, them: And thus we come by those ideas, we have of yellow,
then some rationale must be offered to motivate its reten- white, heat, cold, soft, hard, bitter, sweet, and all those which
tion. we call sensible qualities, which when I say the senses convey
into the mind, I mean, they from external objects convey into
the mind what produces there those perceptions. (p. 105)
2.1. A pervasive assumption Berkeley’s position was similar:
The assumption of separate senses is so basic that it is im- Sitting in my study I hear a coach drive along the street; I look
plicit even at the introductory level. Undergraduate text- through the casement and see it; I walk out and enter it. Thus,
books on psychology are organized in terms of individual common speech would incline one to think I heard, saw, and
senses, with chapters on vision, hearing, touch, and so on touched the same thing, to wit, the coach. It is nevertheless cer-
(e.g., Matlin & Foley 1992). No justification for this parsing tain the ideas intromitted by each sense are widely different
and distinct from each other. (1709/1975 p. 22)
is offered. The assumption of separate senses is reflected in
the existence of sense-specific journals (e.g., Vision Re- What is the source of this basic assumption? One source
search, The Journal of Auditory Research, The Journal of is the existence of anatomically distinct receptor systems.
Vestibular Research) and in treatises attempting to account Another is the existence of different forms of stimulus en-
for perception within a single modality (e.g., Cutting 1986; ergy. A third is neurophysiological differences between the
Gibson 1986; Handel 1989). It is implicit in theory and re- senses. We consider these in turn.
search in areas of cognition such as learning, attention,
memory, and imagery, each of which is commonly consid-
2.3. Anatomy
ered in the context of individual senses (e.g., “visual cogni-
tion,” Pinker 1985; “auditory imagery,” Reisberg 1992). We Sensory receptors have different anatomy and different
have been unable to locate an explicit justification of the as- anatomical locations (e.g., eyes, ears, tongue, nasal cavity,
sumption of separate senses in the philosophical, behav- skin, muscles, joints). However, classification on the basis
ioral, or neurophysiological literatures (e.g., Ackerman of anatomy depends upon the prior acceptance of the as-
1990; Gibson 1966; Heil 1983; Milne & Milne 1962; Pieron sumption that the senses exist and operate either exclu-
1952). sively or primarily as independent units. The anatomical
differences do not, by themselves, mandate this assump-
tion.
2.2. Historical origins
Consider binaural sound localization. Sound often ar-
The assumption of separate senses has been handed down rives at one ear before it arrives at the other. The time de-
to modern science from protoscientific natural philoso- lay between arrival at the two ears constitutes an irreducible
phers who received it, in turn, from pre-scientific episte- relation that is caused by the location of the sound source
mologists. Boring (1950) attributed it to Aristotle, Locke, relative to the head (Gibson 1966). The two ears function
and Berkeley. Yet none of these scholars approached the is- as a single, indivisible unit in picking up this informative re-
sue directly. lation. Similarly, consider binocular stereopsis. The optic
In the de Anima, Aristotle (1931, p. 425b) enumerated array differs at any two points of observation, such that re-
the senses, and asked “why we have more senses than one.” lations between simultaneous samples of the optic array
He answered in terms of special objects and common sensi- taken at two locations are influenced by the shape of objects
bles. Each sense has its special object, “that which cannot and their spatial layout. The result is an irreducible relation
be perceived by any other sense than that one in respect of between the two array samples: “The two eyes are not sep-
which no error is possible; in this sense color is the special arate sense channels for which signals must be compared;
object of sight, sound of hearing, flavor of taste,” (1931, p. rather they constitute a single binocular system” (Jones &
418a). This is in contrast with common sensibles, which are Lee 1981, p. 39).
“perceptible by any and all of the senses” (1931, p. 418a). In these examples, anatomically distinct structures func-
Among the common sensibles are movement, rest, number, tion in a unitary manner. In general, it is possible for
figure, and magnitude (Marks 1978; cf. Gibson 1983). For anatomically distinct structures to work together to achieve
Aristotle, multiple senses were required “to prevent a fail- irreducible, coordinated end products. By irreducible we
ure to apprehend the common sensibles . . . The fact that mean that the activity in question ceases to exist, or is qual-
the common sensibles are given in the objects of more than itatively altered, if not performed through the integrated
one sense reveals their distinction from each and all of the action of anatomically distinct units.2 The same may be
special sensibles” (1931, p. 425b). Thus, Aristotle’s view is true of stimulation of anatomically distinct structures in dif-
that multiple perceptual systems are required in order for ferent perceptual systems (several examples of this are
us to distinguish percepts that are general across senses given in sect. 6). Thus, the anatomical differences between
BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 197

the two ears, the two eyes, and so on, are not a sufficient entation when a line is presented visually, but also when an
reason for parsing perception into distinct senses (cf. Gib- invisible line is felt with the hand (Maunsell et al. 1989).
son 1966, p. 42).3 Similarly, the so-called vestibular nucleus is known to re-
spond to activity in the visual system (Stein & Meredith
1993). Thus, our current knowledge of neurophysiology
2.4. Energy cannot be used as an a priori justification for the assump-
A second possibility is that we might distinguish among the tion of separate senses.
senses on the basis of stimulus energy:
Seeing involves the activity of extracting information from light 2.6. Summary
radiation; hearing occurs when a creature gains information
from pressure waves of certain sorts; smell and taste involve the The previous discussion raises questions about the exis-
extraction of information from chemical features of the envi- tence of separate perceptual systems. We use this uncer-
ronment . . . touch incorporates the capacity to obtain infor- tainty to reconsider existing views of perception that are
mation about things via mechanical contact of some sort. (Heil based on the assumption of separate senses. In section 3,
1983, p. 8) we examine contemporary concepts of specification.
However, this is problematic, also. One cannot generate a list
of stimulus energies without prior knowledge of perceptual
3. Multiple senses and specification
systems. For example, defining vision as the pickup of infor-
mation from light requires a definition of “light”. The elec- In this section we focus on relations between the assumed
tromagnetic spectrum is a continuum that has no inherent existence of separate perceptual systems and the concept of
partitions. Only a narrow band of the spectrum is associated specification. We discuss three approaches to relations be-
with vision and thus called “light.” Therefore, defining vision tween the senses, which we refer to as the no specification,
in terms of electromagnetic energy requires an appeal to vis- modal specification, and independent specification views.
ible light, at which point the definition becomes circular. In many respects, these three views differ from one another
Similarly, animals are sensitive to only a limited range of profoundly. However, we will argue that in the context of
acoustic frequencies, and the range of audible frequencies intersensory relations the three views share much of the
differs across species. This means that defining hearing in same logic. We begin with a brief review of some conse-
terms of acoustic energy requires an appeal to audible vibra- quences of the hypothesis that there is no specification. We
tions, at which point this definition also becomes circular. then argue that these consequences are problematic for ex-
In addition, a given form of energy may stimulate ana- isting views that assume the existence of specification.
tomically distinct receptor systems. For example, in many
species infrared radiation is perceived as warmth, but in
some species of snakes it is used to “see”; this is dependent 3.1. No specification
on receptors that are different and separate from both the The oldest view of specification is that there is none. This
eyes and the skin (Hartline et al. 1978). Similarly, certain tradition is founded on the epistemological assumption that
forms of mechanical energy are involved in touch, while there is an ambiguous relation between sensory stimulation
others, differing only in frequency, are involved in hearing and physical reality (e.g., von Helmholtz 1962; Hochberg
(e.g., the concussion of fireworks can be felt as well as 1964; cf. Shaw et al. 1982): A given physical reality can give
heard). What is considered to be vision for one organism is rise to multiple patterns in ambient arrays, and a given sen-
considered to be touch for another, and perhaps audition sory pattern can be caused by multiple physical realities; the
for yet another. Thus, it seems unlikely that we can develop mapping between physical reality and patterns in sensory
an a priori argument for the existence of separate senses stimulation is many:many (Fig. 1A). In this section we pre-
from the existence of different types of ambient energy. sent some general corollaries of the assumption of separate
senses which we discuss here because they are most widely
2.5. Neurophysiology acknowledged and discussed within the no specification
view.
It might be argued that there is a neurophysiological basis
for the existence of distinct perceptual systems. However, 3.1.1. Independent percepts. An implication of the assump-
the nervous system does not appear to be organized in a tion of separate senses is that each perceptual system gener-
sense-specific fashion (cf. Alexandrov & Jarvilehto 1993). ates an independent indication of reality. There are “visual
Many structures in the nervous system respond to activity percepts” of an event, “auditory percepts,” and so on. A per-
originating in more than one sense modality (e.g., Fishman cept generated by one system is simultaneous with but inde-
& Michael 1973). This is true even for “seemingly dedi- pendent of a percept generated by a different system (e.g.,
cated unimodal regions” (Stein & Meredith 1993, p. xi; cf. Clifton et al. 1994; Marks 1987). One example would be per-
Weingarten & Spinelli 1966): ception of a person speaking, where it is assumed that there
Convergence of sensory inputs [has been found] in unicellular is a visual percept of the moving lips that is independent of
organisms, comparatively simple multicellular organisms such an auditory percept of the speech. By independent we mean
as flatworms, in the higher primates, and at all intervening lev- that speech can be heard without being seen (i.e., with the
els of complexity. In fact, we know of no animal with a nervous eyes closed), or seen without being heard (i.e., when viewing
system in which the different sensory representations are orga- a television with the sound turned off). Interactions between
nized so that they maintain exclusivity from one another. (Stein such independent percepts could occur (e. g., in the McGurk
& Meredith 1993, p. xii) effect, in which “seen” activity of a speaker influences per-
For example, neurons in higher levels of the so-called visual ception of “heard” speech; McGurk & MacDonald 1976),
cortex (i.e., area V4) respond selectively to a preferred ori- but these are logically posterior to the existence of a “visual

Figure 1. Some of the possible mappings between physical reality and the structure of ambient arrays. A: The no specification view as-
sumes that the mapping is many:many, so that the structure in ambient arrays is ambiguous with respect to reality. B: The modal speci-
fication view posits 1:1 mappings, but assumes that these exist within individual ambient arrays. C: The multiple specification version of
the independent specification view suggests that a given aspect of reality redundantly structures several parameters within a given am-
bient array. D: The amodal specification version of the independent specification view assumes that a given aspect of reality redundantly
structures parameters within different ambient arrays.
percept” and an “auditory percept” of speech. Another ex- speaker. Kuhl and Meltzoff (1984; Meltzoff & Kuhl 1994)
ample would be perception of one’s own walking, in which discussed two mechanisms that might explain the McGurk
there is stimulation of the visual, vestibular, and somatosen- effect. Each of these (identity matching and supramodal
sory systems, each of which is thought to be stimulated in- representations) is based on the assumption that integra-
dependent of the others (e.g., Lackner & DiZio 1988). “The tion of vision and hearing occurs within the observer. Eco-
multisensory nature of self-motion permits investigators to logically based accounts of the McGurk effect stress that
de-couple the relationship among sensory modalities in or- the speech event naturally structures both the optic and
der to assess each one’s contribution to perception” (Car- acoustic arrays, but continue to accept the assumption that
penter-Smith et al. 1995, p. 36). Interactions between the the resulting patterns are detected via “integration of the
senses occur in the nervous system (i.e., “sensory conver- information from the two modalities” (Fowler & Dekle
gence”), and are not properties of sensory stimulation. 1991, p. 822; Rosenblum & Saldana 1996). These examples
In the McGurk effect, the influence of sight upon what from divergent research domains illustrate the general im-
is heard is believed to result from an internal, inferential plication of the assumption of separate senses, that initially
comparison or matching of the visually perceived activity of (at least) each perceptual system generates an independent
the face and the auditorily perceived activity of the vocal indication of reality.
tract. McGurk and MacDonald (1976, p. 747) argued that
the visual-auditory interaction was produced by integration 3.1.2. Frames of reference. Perception can be understood
of “information from the two modalities.” Green et al. as a form of measurement and, like any system of measure-
(1991) argued that the presumed audio-visual integration ment, it must be conducted relative to some referent or
occurs relatively late in phonetic processing, after the ex- metric. Ideally, the referents for perception should be
traction of information about the voice characteristics of the physical; the surface of the earth, another person, and so on.

However, if there is no specification, then there can be no One form of antagonistic interaction is sensory suppression,
direct access to physical referents (Oman 1982); any knowl- in which perceivers resolve conflict “by responding selec-
edge of physical referents must be a product of internal pro- tively to input in one modality and ignoring inputs in other
cessing. This inferential processing, or unconscious infer- modalities” (Lewkowicz 1994, p. 166). For example, when
ence, requires its own referents. For example, Lakatos there is conflict between vision and touch, it often is be-
(1993, p. 143) argued that each sense relies upon its “prin- lieved that vision dominates touch in determining a single
cipal sensory dimension as the basis for interpretation.” For percept (e.g., Harris 1965; Marks 1978; Warren & Rossano
this reason, within traditional views it is widely hypothe- 1991). Alternately, the antagonistic interaction may consist
sized that the referents for perception are properties of sen- of a calculation in which different weights are assigned to
sory stimulation, or sensory reference frames (e.g., Soecht- inputs from each perceptual system (e.g., Oman 1982;
ing & Flanders 1992; cf. Feldman & Levin 1995). Parker & Poston 1984).
Different frames of reference are believed to exist for
each perceptual system; acoustic structure for the auditory 3.1.5. Illusory percepts. If the dominant input is not cor-
system, gravito-inertial force for the vestibule, anisotropic rect or if there are errors in the weighting scheme, then the
optical structure for the visual system (luminous lines, car- final percept will be inaccurate. For this reason, differences
pentered environments), and patterns of pressure within in stimulation across perceptual systems can lead to per-
and at the surface of the body for somatosensory systems. ceptual errors, or illusions (e.g., Dichgans & Brandt 1978;
Because they are based on qualitatively different forms of Warren & Rossano 1991; Witkin & Asch 1948). A large por-
energy, these frames of reference are mutually exclusive; tion of perceptual research relies on subjective reports that
optical structure is qualitatively unrelated to acoustic or are believed to reflect erroneous or illusory percepts (e.g.,
gravito-inertial structure, for instance. Thus, object motion Dichgans & Brandt 1978; Leibowitz et al. 1986; Wertheim
(e.g., the movement of the vocal tract during speech) would 1994).
be perceived by the visual system relative to an optical The no specification view leads to theories of perception
frame of reference, while it would be perceived by the au- in which sensory stimulation is ambiguous with respect to
ditory system relative to an acoustic frame of reference. reality, so that accurate perception depends upon internal
Similarly, walking would give rise to changes in optical stim- processing to resolve the ambiguity. Given that virtually all
ulation that differ qualitatively from changes in stimulation behavior gives rise to multimodal stimulation and that
of the vestibular and somatosensory systems. much of this stimulation is nonredundant, a pervasive role
of this processing must be the resolution of input conflict
3.1.3. Input conflict. When a single event influences multi- (Oman 1982). Proponents of the no specification view have
ple ambient arrays, it will often be the case that separate ar- directly addressed many of the resulting problems, and
rays suggest different and incompatible (i.e., mutually ex- have proposed a variety of internal processes to deal with
clusive) realities. In such cases, the patterns in different them. In the remainder of this section we argue that these
arrays are said to conflict with one another (e.g., Bushnell problems apply equally to existing views that assume the ex-
& Weinberger 1987; Harris 1965; McGurk & MacDonald istence of specification.
1976; Nashner et al. 1982); we refer to this as input conflict
(Stoffregen & Riccio 1991).4 Consider driving a car at con-
3.2. Modal specification
stant velocity. Mechanical properties of the vehicle and the
road produce low-amplitude vibration, which is transmitted The ecological approach to perception and action consti-
to the body and propagates upward through the body to the tutes a fundamental contrast to traditional theories. Part of
head. Due to the nonrigidity of the body, there are differ- the contrast exists at the level of epistemology. The ecolog-
ences in the phase and amplitude of vibration of the torso ical approach rejects the assumption that the relation be-
and head. Somatosensory stimulation (i.e., patterns of pres- tween potential sensory stimulation and physical reality is
sure on the torso) suggests vibration, but is ambiguous ambiguous. Within the ecological approach it is assumed
with respect to translation. Vestibular stimulation (patterns that there is a lawful 1:1 relation, or mapping, between po-
of gravito-inertial force at the head) also suggests vibra- tential sensory stimulation and reality such that properties
tion (and is ambiguous with respect to translation), but the in ambient arrays specify the underlying physical reality
vibration is not the same as that suggested by the soma- (e.g., Shaw et al. 1982). Specification would make it possi-
tosensory system. Visual stimulation (i.e., optical flow) sug- ble for perception to be direct, that is, for veridical infor-
gests linear translation coupled with low-amplitude vibra- mation about reality to be picked up without mediation by
tion. Vision will conflict with the other two systems because unconscious inference. We noted in section 3.1 that sensory
only vision suggests translation. In addition, there will be in- reference frames are required in the no specification view.
put conflict between patterns available to the somatosen- When specification is assumed to exist, sensory reference
sory and vestibular systems, since these suggest different vi- frames are unnecessary: If perception is direct, then per-
bratory motions. ceivables can be measured relative to physical referents.
The concept of specification has been applied in a vari-
3.1.4. Antagonistic interaction. When there is input con- ety of ways. The most common interpretation is what we re-
flict, the organism must make a choice (usually construed fer to as the hypothesis of modal specification. The modal
as unconscious inference) about the actual state of reality. specification hypothesis asserts that specificity exists in in-
The choice must be internal because the structure of the dividual energy arrays (e.g., Fowler 1986; Kugler & Turvey
ambient arrays is assumed to be ambiguous with respect to 1987, p. 9; see Fig. 1B). The argument is that for reasons of
reality. In most theories the determination of a single per- natural law each ambient array bears a specificational rela-
cept (i.e., the resolution of conflict) is believed to rely on tion to the underlying physical reality. Considerable effort
antagonistic interactions between the perceptual systems. has been directed to identifying the quantifiable parame-

ters of ambient arrays that may have the essential 1:1 rela- heading (Warren et al. 1991). The most general develop-
tion with aspects of physical reality. Examples include ment of this view has been presented by Cutting, who ar-
global optical flow (Gibson 1966), which is created by self- gued that observers “select among multiple sources of in-
motion through an illuminated environment, patterns in formation” (Cutting 1986, p. 241), each of which “equally
the haptic array produced by the inertia tensor (Solomon & specifies the physics of a situation for an object or event”
Turvey 1988), which is a property of handheld objects; and (Cutting 1991, p. 29). The result is that “all sources equally
t and its time-derivatives (e.g., Lee 1980), which can be in- specify the object or event perceived” (Cutting 1986, p. 248).
fluenced by temporal properties of impending collision,
such as time-to-contact (Tc). Presentations of the modal 3.3.2. Amodal specification. James Gibson (1966; see also
specification hypothesis generally have not addressed (i.e., Gibson 1969; 1983) argued that information can be amodal,
have neither accepted nor rejected) the possibility that in- or available redundantly to more than one perceptual sys-
formation may exist in patterns that extend across different tem (as we noted earlier, James Gibson also endorsed a dif-
energy arrays (e.g., Fowler 1986; Kugler & Turvey 1987; ferent position; this is discussed in sect. 6). Thus, the
Lee 1980; cf. Smith 1994). amodal specification view posits a 1:many mapping, with
The modal specification view does not posit relations be- properties of reality being specified by patterns existing re-
tween patterns in different ambient arrays. What are the dundantly in different ambient arrays (Fig. 1D). James Gib-
implications of this view for situations involving simultane- son presented this as a general assertion:
ous stimulation of multiple perceptual systems? By framing Different stimulus energies – acoustical, chemical, radiant –
its premises in terms of distinct ambient arrays, the modal can all carry the same stimulus information . . . patterns in the
specification view embraces the assumption that separate flux of sound, touch, and light from the environment may be
senses exist. This causes problems for intersensory rela- equivalent to one another by invariant laws of nature. (Gibson
tions, which can be illustrated by again considering driving 1966, p. 55; cf. Gibson 1986, p. 115)
at constant velocity. The discrepancy (i.e., nonredundancy) The concept of amodal information has a strong appeal
between stimulation of the visual, vestibular, and somato- for many ecological psychologists, who argue that it “does
sensory systems would constitute ambiguity concerning away completely with the paradox of detecting cross-modal
physical reality, that is, input conflict. If a given reality gives correspondences when the patterns of energy formally have
rise to different structures in two or more senses, then at nothing in common. If a person is detecting information
least one of the structures must be wrong, in other words, and not stimulus cues, the same information is available in
not specific to reality. various forms” (Pick 1986, p. 235). Within the ecological ap-
This problem has not been addressed in discussions of proach, research addressing relations between the senses
modal specification. For example, in the literature on opti- has concentrated almost exclusively on the concept of
cal flow it is common to suppose that the optical specifica- amodal specification (e.g., Bahrick 1988; Fitzpatrick et al.
tion of self-motion is independent of (Warren 1995) or will 1994; Lee 1990; Rosenblum & Saldana 1996; cf. Smith
dominate (Lee & Lishman 1975) information about self- 1994).
motion that is picked up by other sensory modalities, but
there has been little discussion of the input conflict that this 3.3.3. Problems with independent specification. For both
implies or its consequences for the concept of specification. independent specification views, the interpretation of driv-
ing at constant velocity appears to be similar to the inter-
pretation of the no specification and modal specification
3.3. Independent specification views. The multiple specification view does not posit any
We have seen that in the no specification view the mapping comparison between modalities, but if such a comparison
between reality and potential sensory stimulation is as- took place it would reveal a discrepancy or ambiguity con-
sumed to be many:many, while in the modal specification cerning reality in the arrays available to the visual, so-
view the mapping is assumed to be 1:1. The third view, matosensory, and vestibular systems, in other words, input
which we refer to as the independent specification hypoth- conflict (cf. Cutting & Vishton 1995, p. 98). If stimulus pa-
esis (Stoffregen & Pittenger 1995), proposes that the map- rameters can vary independently of one another, then at
ping is 1:many. In this view, each aspect of physical reality least one of them will vary independently of the relevant re-
gives rise to multiple, independent structures or patterns in ality; hence, that variable (at least) does not bear a lawful
one or more ambient arrays, yet each pattern is individually relation to reality. This is implicit in the statement that “one
specific to the underlying reality. In the behavioral science can in principle vary one without varying another” (Cutting
literature this proposal has taken two forms, one positing 1991, p. 29). Cutting’s assertion refers to variations that
1:many mappings within a given ambient array (multiple might be caused by an experimenter. However, the concept
specification), and the other positing 1:many mappings be- of specification is based on the physics of energy propaga-
tween arrays (amodal specification). We will argue that tion, which applies both within as well as beyond the labo-
these two views are logically identical. ratory. If such variations are possible in the laboratory, then
they must be possible in general. This problem is reflected
3.3.1. Multiple specification. In this view, a given aspect of in empirical evaluations of the multiple specification hy-
reality is believed to influence the structure of multiple as- pothesis. For example, Laurent et al. (1996) described op-
pects of a given ambient array (Fig. 1C). These different as- tical variables that, they argued, are independently specific
pects constitute “multiple sources of information” about re- to Tc. One of these was the rate of change of expansion of
ality, and it is hypothesized that perception can be achieved an optical contour, and another was the rate change of oc-
with equal fidelity on the basis of any one (Cutting & Vish- ular convergence required to maintain binocular fixation on
ton 1995). Optical examples include multiple specification an approaching object. Laurent et al. (1996) proposed eval-
of Tc (Laurent et al. 1996), and multiple specification of uating the relative effectiveness of these variables by using

an experimental manipulation that altered the relation of from the concept of input conflict. There has been no sim-
one optical variable to Tc while not influencing the other ilar movement among supporters of the modal specification
variable. By its nature, this manipulation violates the defi- and independent specification views. We have argued that
nition of specification: If it is possible to alter the relation by accepting the assumption of separate senses, the modal
between a stimulus variable and reality, then the stimulus specification and independent specification views implic-
variable does not have a unique, determinate relation to itly accept the existence of input conflict, which is incom-
that reality. Thus, the multiple specification view appears to patible with the concept of specification.
imply a lack of specificity and so implies a need for infer-
ential processing to resolve conflict between inputs.
The concept of amodal specification is commonly used to 4. Physical referents for physical motion
explain crossmodal influences on the perception of objects
or events that naturally structure more than one form of In section 3, our analysis focused on the possibility of a law-
ambient energy. Examples include objects that influence ful relation between ambient arrays and reality. We argued
the structure of the optic array and, when touched, of pres- that the assumption of separate senses leads to problems for
sure on the skin (e.g., Gibson & Walker 1984); support sur- theories of perception that assume the existence of specifi-
faces, which influence the structure of the optic and me- cation in ambient arrays. In this section we pursue this ar-
chanical arrays (e.g., Fitzpatrick et al. 1994); and speech, gument at a more fundamental level. We will argue that
which structures the optic and acoustic arrays (e.g., Rosen- concepts of specification that are based on individual forms
blum & Saldana 1996). In such situations the concept im- of ambient energy are problematic at a level of physics that
plies a comparison between information obtained via dif- is logically prior to the structuring of ambient energy. That
ferent perceptual systems. The postulation of such a is, we will argue that even if there were lawful relations be-
crossmodal comparison requires a prior assumption that tween some aspects of reality and the structure of individ-
the senses work separately in such a way that their outputs ual forms of ambient energy, this would not imply that these
can be compared. Thus, there must be, for example, a vi- structures bear a 1:1 relation to reality, in general. While
sual perception of an event that is then compared with an our focus is on the concept of specification, our analysis has
independently generated haptic perception of the same general implications for the interpretation (by behavioral
event (e.g., Gibson & Walker 1984). Fowler and Dekle scientists) of physical motion.
(1991, p. 822) proposed that speech perception is based on If specification exists, then it should be possible for per-
the “joint specification” of speech in the optic and acoustic ceivables to be measured relative to physical referents (sect.
arrays, while Rosenblum and Saldana (1996, p. 328) dis- 3.2). What are these physical referents? 5 In this section we
cussed speech in the context of “modality neutral kinematic discuss referents that are used by physicists. Each of the
patterns.” referents that we discuss could serve as a referent for a for-
Among proponents of amodal specification, there has mal (physical) analysis of motion, independent of any psy-
been little discussion of what happens when structures in chology.
different ambient arrays are discrepant rather than redun-
dant. Cases where discrepancies exist, such as driving,
4.1. All motion is relative
would appear to constitute input conflict, with its attendant
implication of the lack of specificity and need for inferen- Behavior consists of motion. Perception involves motion
tial processing. When patterns in different arrays are of receptor systems (often including the whole body), and
caused by different events (as in studies of the McGurk ef- action involves motion of effectors (often including the
fect), then the patterns must also be discrepant or conflict- whole body). Thus, the perception and control of behavior
ing (e.g., Rosenblum & Saldana 1996). The resolution of is largely equivalent to the perception and control of mo-
this conflict has not been discussed (e.g., Fowler & Dekle tion. This raises questions about the physics of motion.
1991; Rosenblum & Saldana 1996), but it would appear to How do we define motion? A common concept within
imply an internal process. Thus, with respect to specifica- the behavioral sciences is the idea of absolute motion. Be-
tion, the hypothesis of amodality appears to be logically havioral scientists often refer to “absolute motion” (Wert-
identical to the hypothesis of multiple specification within heim 1994, p. 302) or “objective motion” (Held & Leib-
a modality. owitz 1994, p. 451). Yet in physics absolute motion is not a
The independent specification hypothesis can be true meaningful concept. In general relativity, the concept of ab-
only if each candidate stimulus parameter always varies solute motion has no meaning (Becker 1954; Einstein & In-
uniquely with the corresponding physical event. Only then feld 1938). Motion can be defined only relative to some ref-
would specification be preserved, and input conflict erent: for example, motion relative to the earth, motion
avoided. Given that this is not true, it is not clear whether relative to the sun, motion relative to an object (Wade &
either form of the independent specification hypothesis can Swanston 1991, pp. 96–97).
be correct. While these physics are well understood, their conse-
quences for analyses of behavior have not been fully ad-
dressed. We know that the earth moves relative to the sun
3.4. Summary
and has a different motion relative to the galaxy, but a pow-
Each view of specification is confronted with problems that erful and widespread intuition remains that at the level of
arise from the existence of discrepancies among the pat- behavior the earth and its gravitational field constitute an
terns of energy available to different perceptual systems. absolute referent for motion. For example, Wertheim
Supporters of the no specification view have not provided (1994, p. 302) defined absolute motion as “motion relative
a justification for the assumption of separate senses, but to external space (i.e., 3-D ‘Newtonian’ space, as defined by
they have moved to address many of the issues that arise the horizontal surface of the earth and its gravitational

field).” Similarly, Dichgans and Brandt (1978, p. 758), altitude and velocity). The same effect would obtain for a
equated “orientation with respect to . . . gravity” with “po- restrained person in an automobile traveling at fixed speed
sition of objects and the observer on the earth’s surface.” In along a straight, flat road, or on a sled gliding across ice. In
both cases, the concept of absolute motion assumes that these cases the person would be in motion relative to the
motion relative to earth gravity is equivalent to motion rel- earth, but stationary relative to gravity. These examples
ative to the earth. Since the earth is a source of gravity, and show that the magnitude of velocity is irrelevant: A person
since gravity does not move relative to the earth, this as- can be stationary relative to gravity whether he is traveling
sumption is unquestioned. Nevertheless, it is incorrect. It at walking speed or at hundreds of kilometers per hour.
is possible to be in motion relative to the earth and station- This analysis illustrates a fundamental error in any at-
ary relative to earth gravity at the same time. tempt to equate motion relative to the earth with motion
relative to earth’s gravity. The intuitively comfortable idea
of absolute motion makes sense only if motion relative to
4.2. The earth and its gravity
gravity and motion relative to the earth’s surface are identi-
Earth gravity is an accelerative force that tends to move cal. Given that they are not, which of these should be con-
masses toward the earth’s center of mass. At any point on or sidered absolute, and why? This dilemma illustrates the
above the earth, gravity points toward the center of the meaninglessness of the concept of absolute motion (Becker
earth (Fig. 2). A consequence of this is that a person aligned 1954) and shows that any psychological construct of ab-
with gravity in Cincinnati will be at an angle relative to a solute motion, or of an absolute referent for motion, can
person aligned with gravity in Paris. This is true for points have no basis in physical reality.
arbitrarily close together: Upright people on opposite sides
of town are at a slight angle relative to one another. In ad-
4.3. Motion relative to the direction of balance
dition, a person traveling along a gravitational equipotential
(an arc at constant distance from the earth’s center of mass) Gravity is widely considered to be a fundamental constraint
will have a constant alignment relative to gravity; his in- on the control of orientation and motion (e.g., Schone
stantaneous alignment relative to the instantaneous (i.e., lo- 1984). However, this assumption is incorrect. In general,
cal) direction of gravity will always be the same. If the per- the orientation of physical bodies (both animate and inani-
son moves along the equipotential at constant velocity, mate) is not influenced directly by the direction of gravity.
there will be no acceleration relative to gravity, and so the Orientation is influenced directly by the direction of bal-
body will be gravito-inertially identical to one at rest rela- ance (Riccio & Stoffregen 1990).7 In general, an object will
tive to the earth (Goldstein 1980). Under these conditions, remain balanced only when aligned with the direction of
the person is moving relative to the earth, but stationary rel- balance. If a pencil standing on its eraser is aligned with the
ative to the earth’s gravitational field. direction of balance it will not fall over, even if the direction
The requirement for constant velocity motion along a of balance differs from the direction of gravity.
gravitational equipotential is restrictive in terms of real be- For animals, changes in the direction of balance are most
havior. For example, it excludes many forms of biological commonly created by their own behavior. These changes
motion or locomotion, because these are characterized by result from inertial forces that they apply to the support sur-
changes in velocity. However, the requirements can be met, face or medium in controlling orientation and locomotion.
sometimes with surprising ease. A person would need to be Animals often align their bodies relative to the direction of
restrained to prevent small accelerations due to nonrigid balance (Riccio 1995). For example, in curvlinear locomo-
body motion.6 Such a person could then be placed in an air- tion, the direction of balance is shifted out of alignment
craft during straight and level flight (i.e., flight at constant with gravity by the inertial forces generated by the animal
(or vehicle) in creating the turn. When runners, cyclists,
and motorcyclists lean into turns they do not fall over de-
spite the fact that their bodies are out of alignment with
gravity. This is because they remain aligned with the direc-
tion of balance as the latter rotates, just as tassels hung from
a car’s rearview mirror also “lean” during turns. The rota-
tion of the direction of balance in turns results from the in-
ertial forces that sustain the turn.
It might be argued that accelerations generated by ani-
mate behavior are of such low magnitude that they can be
ignored. However, the magnitude of acceleration gener-
ated by living things commonly exceeds magnitudes that
can be generated by most vehicles (Vogel 1988). Biologi-
cally generated accelerations are often brief, but this does
not imply that they can be (or are) ignored. Linear acceler-
ation changes the direction of balance (Riccio 1995), with
the change being proportional to the magnitude of the ac-
Figure 2. At any location on or above the earth’s surface, earth
celeration. One common example concerns sprinters who
gravity points toward the earth’s center of mass. A person aligned
with gravity in Cincinnati (DGC) is at an angle relative to a per- prepare to begin a race by setting their body at an angle,
son aligned with gravity in Paris (DGP). A person traveling from with the torso partially supported by the hands. When the
Cincinnati to Paris at constant velocity (relative to the earth) along starting gun goes off they apply maximum acceleration.
a gravitational equipotential is moving relative to earth but is sta- During a brief accelerative phase they can easily be ob-
tionary relative to gravity. served (e.g., on slow-motion film) to be tilted forward (rel-

ative to the ground) while running. As runners approach erence to the object, rather than by defining the object’s po-
top speed, they straighten up. The initial lean prepares sition and motion relative to other referents (Garrett et al.
them to be aligned with the shift in the direction of balance 1998; Zaal et al. 1998).
that will result from their own efforts (imagine what would
happen if they applied their full acceleration while stand-
ing erect). Horizontal acceleration decreases rapidly as they 4.5. Multiple, task-specific referents
approach top speed. The decrease in acceleration causes The existence of logically independent referents does not
the direction of balance to rotate toward the direction of imply that only one referent is relevant to behavior at any
gravity; as the runners straighten up, they should “track” given time. It may be that animals control different aspects
this rotation. of their behavior relative to different referents (Riccio
The direction of balance has no fixed relation to the earth’s 1995; cf. Fouque et al. 1999). A person who is in motion rel-
surface, or to earth gravity. A consequence of this is that ative to one referent and in stasis relative to another, for ex-
an animal can have one alignment or motion relative to the ample, may simultaneously control their orientation and
earth, another relative to gravity, and a third relative to the motion relative to both. Consider driving at constant veloc-
direction of balance. Equally important is the fact that ity on a flat highway. During turns, as the direction of bal-
the direction of balance is highly localized; it can differ for ance changes relative to the surface of the earth, the torso
adjacent animals (depending of what they are doing), and it remains aligned with the direction of balance (that is, it ro-
can change rapidly over time (e.g., brief changes brought tates as the direction of balance rotates), but the head and
about by the transient accelerations that characterize ani- eyes may maintain their orientation relative to the road
mate locomotion). This illustrates, at the level of behavior, (Fig. 3). A similar effect occurs in flight. During turns, the
the vacuity of any concept of absolute motion. It is to be pilot must control the orientation (and position) of the air-
stressed, again, that these referents (earth gravity, the di- craft relative to the surface of the earth (e.g., for naviga-
rection of balance, and the earth’s surface) are logically and tion), while at the same time aircraft orientation must be
physically distinct. maintained relative to the direction of balance (i.e., to main-
tain aerodynamic stability). Perception and control relative
4.4. No privileged referent to multiple, simultaneous referents will be adaptive in most
situations (Riccio 1995).
The discussion in section 4.3 might seem to suggest that the Our analysis reveals a general requirement for increased
local, instantaneous direction of balance is a fundamental care in characterizations of motion. The existence of an un-
or absolute referent for the control of behavior. We believe limited number of independent physical referents means
that this is not true. The reason is that the direction of bal- that a description of motion is meaningful if and only if it
ance is not always relevant to behavior. Two examples will names the referent (or referents) relative to which the mo-
make this point. First, orientation, and behavior in general, tion takes place. In turn, this means that across situations
continue to be controlled effectively when there is no di- action may be perceived and controlled relative to different
rection of balance (e.g., when the gravito-inertial force vec- referents. The selection of referents should have a func-
tor has a magnitude and direction of zero). This is true in tional basis (Riccio 1995); that is, it should depend on the
orbital spaceflight, where gravity has a direction and goals of action. One aspect of learning to perform new tasks
nonzero magnitude, but the gravito-inertial force vector has will be the determination of which referents are relevant.
a magnitude of zero and no direction. After a period of ad-
justment, humans and other animals control their orienta-
tion and motion very capably under such conditions. A re- 4.6. Summary
lated example occurs with water immersion. Under water, We have reviewed the fact that in physics there is no mean-
the gravito-inertial force vector has magnitude and direc- ingful concept of absolute motion. We have shown this to
tion, but for a person in a state of neutral buoyancy there is be true in the case of motion relative to the earth, which
no direction of balance (i.e., at neutral buoyancy the grav-
ito-inertial force vector imposes no directional constraints
on orientation or behavior). People who are at neutral
buoyancy have only a poor ability to perceive or control ori-
entation and locomotion relative to the gravito-inertial
force vector, but they have a good ability to perceive and
control orientation and locomotion relative to other refer-
ents, such as objects or surfaces (Stoffregen & Riccio 1988).
This indicates that neither the direction of balance nor the
gravito-inertial force vector is a general or exclusive refer-
ent for the control of behavior.
Sometimes behavior is not controlled relative to the di-
rection of balance even when the direction of balance is
present; we sacrifice alignment with the direction of bal-
Figure 3. Multiple, simultaneous referents. The driver main-
ance to some other goal. Examples include a soccer goal- tains his body in alignment with the direction of balance, while si-
keeper diving to catch a shot, and a baseball player diving multaneously maintaining his head and eyes in alignment with the
to catch a fly ball. In such cases, once the player has left the illuminated environment (the road). During straight driving (A)
ground, the ball may be the sole referent for both percep- the head and body are parallel, but in turns (B), when the direc-
tion and control. Recent research on prehension suggests tion of balance is not perpendicular to the road, the head and body
that reaching and grasping are organized directly with ref- remain aligned with their separate referents.

differs from motion relative to the earth’s gravitational tion and motion relative to the illuminated environment,
field. The absence of a single, fundamental referent for and the optic array does not specify real changes in position
motion led us to consider the existence of a variety of ref- and motion relative to the gravito-inertial environment.
erents, all independent and of equal reality: None can be Similarly, the structure of the gravito-inertial array is not
regarded as primary or basic to any other. Finally, this specific to motion relative to the surface of the earth; this is
leads to the idea that motion can be analyzed (and per- true, also, of structure in the optic array, which can vary in-
ceived and controlled) relative to multiple simultaneous dependent of motion relative to the earth’s surface. Com-
physical referents. In the next section we will consider the pare a situation in which a person flies along a gravitational
implications of these facts of physics for the concept of equipotential with a situation in which a person in a fixed-
specification. base simulator is exposed to a simulation of flight along a
gravitational equipotential. Although these situations differ
dramatically in their consequences for behavior (e.g., an
5. Physics and specification aircraft crash can be fatal, but a simulator crash is harmless),
it would not be possible to differentiate them on the basis
In this section we discuss some of the implications of our of structure in the optic array, or in the gravito-inertial ar-
analysis of physical referents for theories of perception. The ray. This is because the structure of each array is identical
first implication applies only to theories that posit the exis- in these two situations (leaving aside temporary limitations
tence of specificity between reality and the structure of am- in the technology of optical simulation). In general, there
bient energy arrays. The second implication is more gen- will not be a unique mapping of physical referents onto
eral. forms of ambient energy; some referents structure only one
form of energy, while others structure multiple forms. This
means that there is no single form of ambient energy whose
5.1. Ambiguity in single-energy arrays
structure is specific to position or motion relative to all
Our discussion of physical referents revealed that there are physical referents. For this reason, no single form of ambi-
an unlimited number of possible referents, all of which are ent energy (i.e., no single-energy array) can have a specifi-
mutually independent at the level of physics. Motion rela- cational relation to physical motion.
tive to any one of these is equally real. How does this relate Our analysis brings into question the assumption that
to the hypothesis that the structure of ambient energy structure in light, sound, and other forms of ambient energy
arrays is specific to reality? The specificity hypothesis is are (individually) specific to physical motion and, hence, to
not about the nature of reality, and so is not about the exis- reality in general. Two possible interpretations may be
tence of, or relations between physical referents. Rather, drawn from this. One is that specificity does not exist, that
specificity is about the mapping between the physics of the sensory stimulation bears an ambiguous relation to reality,
animal-environment interaction and the energy fields that and, therefore, that any approach based on specification
are available to perceptual systems. Accordingly, this map- cannot explain perception and action. The other possible
ping is logically posterior to the existence, number, and in- conclusion is that specificity exists in something other than
dependence of physical referents. This raises questions arrays of a single form of energy. The latter possibility is ad-
about relations between physical referents and the struc- dressed in section 6.
turing of ambient arrays. Does motion relative to a given
physical referent impart unique structure to a particular
5.2. Reinterpreting subjective reports
form of ambient energy? Can motion relative to a given
physical referent impart unique structure to more than one Our analysis of physical referents has important conse-
form of energy? quences for the perception of physical motion, and for in-
While all physical referents are equally real, they do not terpretation (by researchers) of subjective reports of phys-
have equivalent relations to various forms of ambient en- ical motion. These implications apply equally to studies that
ergy. For example, a given event may entail motion relative assume or reject the existence of specification. For exam-
to the surfaces and media that generate, reflect, or prop- ple, motion (or stasis) of the self relative to the illuminated
agate light (the illuminated environment), yet may entail environment is neither more nor less real than motion (or
simultaneous stasis or motion relative to the direction of stasis) of the self relative to the gravito-inertial environ-
balance (the gravito-inertial environment). Changes in po- ment, relative to the surface of the earth, or relative to any
sition or motion relative to the illuminated environment other physical referent. A person who is stationary relative
will not necessarily produce changes in position or motion to the surface of the earth (e.g., a person standing inside a
relative to the direction of balance, and vice versa. This “moving room,” Lishman & Lee 1973, or seated inside a ro-
means that a given pattern of optical structure may corre- tating drum, Dichgans & Brandt 1978) can be in motion rel-
spond to a variety of patterns of gravito-inertial structure, ative to the illuminated environment. The person’s motion
and vice versa. Consider an animal moving along a gravi- relative to the room or drum is real, just as his stasis rela-
tational equipotential (see sect. 4). The animal would be tive to the earth is real.
in motion relative to the illuminated environment (produc- This understanding of physical reference frames moti-
ing spatiotemporal changes in optical structure) but station- vates a substantial reinterpretation of many widely ac-
ary relative to the direction of balance (producing spatio- cepted concepts of perceptual error (i.e., illusion). It is
temporal stasis in what we might call the gravito-inertial widely believed that there is erroneous or illusory percep-
array).8 Consider also motion in an elevator, where there tion of self-motion. To underscore the importance of the
are changes in gravito-inertial structure (as the elevator ac- concept of perceptual error, Dichgans and Brandt (1978, p.
celerates), but stasis in optical structure. In these cases, the 755) began their chapter with these words: “The sensation
gravito-inertial array does not specify real changes in posi- of self motion is a common visual illusion”; this interpreta-

tion has been accepted generally (e.g., Howard 1982; Lee cation is that percepts and behaviors that are not congruent
& Lishman 1975; Nashner et al. 1982; Wertheim 1994). In with gravity or with the earth’s surface are not necessarily
these cases the error may be with the experimenter’s as- erroneous or illusory. This should motivate a substantial
sumption of an “absolute” referent for motion, and not with reinterpretation of many existing data reports and changes
participants’ percepts (cf. van Ingen Schenau 1980). in the way in which participants are asked to report their ex-
The statements “I am moving” and “I feel like I’m mov- periences of self-motion.
ing” previously thought to be unambiguous, are now seen We have argued that the physics of motion are logically
to be ambiguous. Consider the case of a person at rest rel- prior to the issue of the specification of motion in ambient
ative to the earth’s surface who is exposed to a display of op- energy arrays, and that motion relative to different physical
tical flow, which occurs in the cinema, in fixed-base flight referents will structure some ambient arrays but not others.
simulators, and in many laboratory experiments (e.g., Dich- A major implication of this is that specificity does not exist
gans & Brandt 1978; Lishman & Lee 1973). In such exper- in the structure of individual forms of ambient energy. This
iments participants are often asked to state whether they is not a problem for the no specification view, but it poses a
feel themselves to be moving. If they say that they are mov- fundamental problem for views, such as the ecological ap-
ing, they are correct (this is because they are in motion, rel- proach to perception and action, which assume that speci-
ative to the illuminated environment), but if they say they fication exists. If the principle of specification is to be sus-
are stationary, they are also correct (this is because they are tained, then it must take some form other than modal
stationary relative to the earth). The fact that both re- specification or independent specification. This is ad-
sponses can be correct illustrates the ambiguous nature of dressed in the section 6.
questions such as “Do you feel that you are in motion?” A
better question, reflecting the physics of motion, would be
“Do you feel that you are in motion relative to anything, and 6. Specificity in the global array
if so, relative to what?”
This reasoning applies as well to the control of self- If specification cannot exist in single-energy arrays, then it
motion. Dichgans and Brandt (1978, p. 787) noted that “il- may not be possible to sustain a theory of direct perception
lusions” of self-motion can affect the control of orientation within the assumption of separate senses. In this section, we
(posture): “deceptive visual motion impressions cause a dis- offer a novel hypothesis about specification, which does not
placement of . . . postural vertical . . . and body sway.” In require the assumption of separate senses. We do not at-
other words, the body sway is an error based on a mistaken tempt a formal proof of our hypothesis. Rather, we argue
percept. The error interpretation has been retained, either that it is possible and that it deserves to be tested.
explicitly or implicitly, in dynamical analyses of the percep-
tion and control of self-motion (e.g., Dijkstra et al. 1994),
6.1. The global array
and is common in neurophysiological research (Stein &
Meredith 1993). As an example, consider the experiments The concept of ambient arrays was developed in the con-
of Lishman and Lee (1973), in which standing participants text of single forms of energy, and there is wide acceptance
were exposed to optical flow created by a moving room. In of the existence of (at least) the optic array and the acoustic
a variety of conditions, subjects were asked to “report what array. We now draw attention to the existence of an ambi-
was happening when the apparatus was in motion” (p. 290). ent array, which has previously received little consideration.
Lishman and Lee divided these reports into those consis- This array, which we call the global array, consists of spa-
tent with “visual information,” those consistent with “me- tiotemporal structures that extend across multiple forms of
chanical information,” and “other” (“A report was classified ambient energy. These patterns are higher-order in the
as ‘other’ when it did not wholly correspond to either the vi- sense that they are superordinate to (and qualitatively dif-
sual or mechanical kinaesthetic information” [p. 290]). A ferent from) the patterns that exist within single-energy ar-
report that “I am moving” would be classed as “visual,” rays. Thus, in principle, information in the global array may
while a report that “the room is moving” would be classed be detected without prior or concurrent sensitivity to struc-
as “mechanical.” Our analysis has shown that each of these ture in single-energy arrays (cf. Gibson 1986, p. 141). Like
statements is ambiguous. Reports in the “other” category other ambient arrays, the global array is not an hypothesis,
were not analyzed. However, this category would include but a fact. The question is not whether it exists but, rather,
reports that were factually correct, such as “I am in motion whether it contains information, and whether that infor-
relative to the room, but stationary relative to the floor.” mation is detected and used by animals.
Lishman and Lee assumed that motion would be perceived The global array can be represented as an n-dimen-
relative to only a single referent. This assumption may have sional space. The number of dimensions is the sum of di-
lead them to exclude correct reports from their analysis, mensions of the different forms of energy, minus those di-
something that occurs routinely in studies of perceived self- mensions that are common across all forms of energy. The
motion (e.g., Dichgans & Brandt 1978; Graybiel 1952; dimensions of space (i.e., position) and time are common
Wertheim 1994). This suggests that errors in the experience across all forms of energy, while other dimensions are pe-
of self-motion may be less common than generally is sup- culiar to individual forms of energy. Considering these
posed.9 jointly yields a global array with several dimensions. For
purposes of illustration we reduce this to three (Fig. 4).
The structure of the global array (e.g. Fig. 4) is influ-
5.3. Summary
enced by all events, objects, and surfaces that influence the
The physics of motion, and of referents for motion, have im- structure of single-energy arrays. In addition, the global ar-
portant implications for the perception of motion and the ray is influenced by events that do not structure single-
interpretation of subjective reports of motion. One impli- energy arrays; among these are motion relative to some of

Figure 4. The global array, represented by a n-dimensional

space (three in this example) of energy structures
the referents discussed in section 4. Information about

these relative motions is essential for many common be-
haviors, and so animals have a strong motivation for being
Figure 5. Evolution over time of (A) single energy arrays and (B)
sensitive to information in the global array. In the absence the related global array during braking a vehicle to a stop. (A.U.
of such sensitivity, animals would be forced to obtain this means arbitrary units.)
information through inferential processing, that is, through
internal comparisons of the patterns in single-energy arrays
(i.e., those picked up by individual perceptual systems).
The following examples illustrate the existence of informa- directly. An animal that was sensitive only to structure in
tion in patterns that extend across forms of energy. These single-energy arrays would need to relate these structures
examples focus on patterns that extend across two or three through internal processing.10
kinds of stimulus energy. However, each is also a structure This example can be expanded to include patterns that
in the global array. The events in question may vary the extend across three or more forms of ambient energy.
structures in several forms of energy. Our discussion in Consider a car moving at constant velocity in congested
terms of a limited number of forms of ambient energy is for traffic, where some of the other cars are beyond the field of
clarity of presentation only. view but may still be heard as drivers honk their horns.
Consider a situation in which an automobile cruising at a Some of these other cars may be moving, while others may
constant velocity slows to a stop (Fig. 5); for simplicity we have stopped. The acoustic array, taken in isolation, does
consider only motion relative to the illuminated environ- not differentiate between motion of the subject car (or
ment (which influences optical structure) and relative to other cars) relative to the gravito-inertial force environment
the gravito-inertial force environment. Figure 5A shows the and motion of cars relative to each other. Similarly, the op-
consequences of this motion for structure in the optic array tical-gravito-inertial pattern discussed here provides no in-
and in the gravito-inertial array. Optical structure is am- formation about motion relative to any audible referents.
biguous with respect to motion relative to the gravito-iner- However, patterns that extend across optics, acoustics, and
tial environment: The same optical patterns could be gravito-inertial force will provide information about the si-
caused by deceleration of the body relative to the ground, multaneous motion or stasis of the subject car relative to the
or by deceleration of an illuminated enclosure (e.g., a mov- gravito-inertial force environment and relative to the audi-
ing room; Lee & Lishman 1975) relative to a gravito-iner- ble and visible environments.
tially stationary observer. At the same time, gravito-inertial The original example of information in the global array was
structure is ambiguous with respect to the nature of the mo- given by James Gibson (1966), who discussed an animal rest-
tion: The same patterns of acceleration could be caused by ing on a support surface. When the substratum is horizontal
deceleration to a stop, or by acceleration (in the opposite relative to the direction of balance, the reactive force that the
direction) to a constant nonzero velocity. Figure 5B shows surface generates against the dorsal surface of the animal is
the higher-order relation that exists in the global array be- parallel to the direction of balance. If the animal is standing,
tween optics and gravito-inertial force. This “optical-gravi- the ankle angle that corresponds to a state of balance will be
toinertial pattern” does not have the ambiguities of the pat- 908. However, when the substratum is at an angle relative to
terns in single-energy arrays; it specifies that the observer the direction of balance (e.g., on sloping ground) the reactive
is undergoing gravito-inertial deceleration relative to the il- force of the substratum is not parallel to the direction of bal-
luminated environment. An animal that was sensitive to this ance. The angular difference between the direction of bal-
higher-order pattern would be able to perceive its motion ance and the reactive force is the slope of the substratum rel-

ative to the direction of balance. This difference will give rise D the distance of the target, and A the amplitude of head
to nonredundant stimulation of the vestibular and so- movement.
matosensory systems. The nonredundancy will constitute a Peper et al. (1994) analyzed the perception of the location
structure in the global array, specifying the slope of the and timing of catching. In catching an object the catcher
ground relative to the direction of balance (Gibson 1966, p. needs information not only about when the object will ar-
63). Gibson (1966, pp. 271–72) noted that “the combination rive, but also about where. Peper et al. identified a param-
is invariant and constitutes a stimulus of a higher-order.” This eter in the global array that provides information about the
means that for animals that are sensitive to the global array, velocity at which the hand must move in order to be at the
perception of ground slope can be direct. right place at the right time to catch the object:
A similar analysis applies to a support surface that moves
Vh 5 (Xb 2 Xh)/t, (2)
independent of the direction of balance. Consider stance
on a platform that can rotate around an axis parallel to the where Vh is the hand velocity necessary to intercept a mov-
ankle joint (e.g., Nashner et al. 1982). When the platform ing object, Xb is the instantaneous sideward position of the
is stationary relative to the direction of balance, ankle rota- object, Xh is the current position of the hand, and t is the
tion will have a fixed relation to head displacement, so that Tc of the object with the fronto-parallel plane of the body
somatosensory and vestibular stimulation will be redun- (assuming constant velocity object motion). Optical struc-
dant. When the platform rotates, ankle rotation occurs in- ture is influenced by Xb and t, while patterns of mechani-
dependent of head displacement. This is often interpreted cal pressure are influenced by Xh. Some additional formal-
as a situation in which there is conflict between structure izations of structures in the global array have been
available to the somatosensory and vestibular systems (e.g., presented by Stoffregen and Riccio (1988, Eqs. 4–6).
Nashner et al. 1982). An alternative interpretation is that In each of these cases, animals that are sensitive to pat-
the discrepancy between mechanical and gravito-inertial terns in the global array can detect the relevant parameters
structures constitutes a pattern in the global array that pro- directly, whereas animals that are sensitive only to structure
vides information for the fact that the person is standing on in single-energy arrays can recover the necessary informa-
a rotating surface. Mechanical structure (available to the so- tion only through internal computation. Researchers have
matosensory system) taken alone is ambiguous with respect typically assumed that accurate perceptions of relative mo-
to the difference between body sway and rotation of the tion in these situations are derived from sense-specific sen-
ground surface. Similarly, gravito-inertial structure (avail- sitivity to structures in optics, force, acoustics, and so on.
able to the vestibular system) taken alone is ambiguous with That is, researchers have assumed that the patterns in the
respect to this difference. Only the superordinate relation global array are not sensed directly, but are broken down at
(that is, the structure in the global array) is uniquely related receptor surfaces, and then reconstructed inside the ani-
to each situation. mal. Our argument is that patterns in the global array might
One of the major challenges for our view will be the for- be sensed directly, without reduction to structures in indi-
malization of structure in the global array. In this target ar- vidual forms of energy.
ticle, we do not attempt such a formalization. However, sev-
eral recent analyses have formalized informative patterns
that exist as higher-order relations between forms of ambi- 6.2. Implications of the global array
ent energy, and so may be examples of structure in the The implications of the possible existence of specificity in
global array. These formalizations are intended to illustrate the global array are numerous. In this section we discuss
the mathematically rigorous basis of structures in the global several of these.
array; they need not be schemes for the weighting of sen-
sory inputs within the perceiver. Rather than internally ex- 6.2.1. No specification? If there is specificity in the global
ecuting the calculations on the right-hand side of each array, then the no specification view is not correct (cf. Gib-
equation, the left-hand side might be detected directly (this son 1966; Shaw et al. 1982), and theories that use the no
is the heart of our position). specification view to motivate hypotheses about internal
Bingham and Stassen (1994) analyzed the structuring of processing of ambiguous sensory inputs lose this motiva-
ambient arrays that results when the head moves relative to tion. We have already noted (sect. 3.4) that proponents of
illuminated objects. The purpose of their analysis was to the no specification view have not offered a justification for
identify information about the distance of illuminated ob- the assumption of separate senses. If none can be provided,
jects from the observer. The optical parameter t (i.e., the and if specification exists in the global array, this would sig-
inverse of the relative rate of dilation of a contour in the op- nificantly undermine the general motivation for this view.
tic array) is influenced by the physical Tc of the head with It might be argued that specification exists but is not de-
the distal object or surface. However, Bingham and Stassen tected by animals (e.g., Proffitt & Gilden 1989). To evalu-
noted that optical flow created by oscillatory head motion ate this argument it would be necessary to identify infor-
is ambiguous with respect to distance unless there is inde- mative structures in the global array and conduct new
pendent information about the velocity of head motion. research to determine whether these are detected. Existing
Head movement structures gravito-inertial patterns that studies of sensitivity to structures in single-energy arrays
are available to the vestibular system. This means that the may not be relevant.
higher-order relation between head velocity and optical
flow is unambiguously related to object distance: 6.2.2. Independent specification? The existence of the
global array is a problem for the independent specification
tpv /T 5 (1/2p)(D/A) (1)
view (which comprises amodal specification and mulitple
where tpv is the value of the optical parameter t at the peak specification; see sect. 3.3), since higher-order structures
velocity of head motion, T is the period of head oscillation, tend to undermine the idea of the independence of lower

order parameters. In addition, we have argued that patterns and so are not preferred when there is a choice. We believe
in single-energy arrays are not specific to reality (sect. 5.1). this accounts for behavioral changes that are observed with
If specificity exists solely in the global array, then neither blindness, such as restricted walking. Note that a lack of
version of the independent specification view can be cor- sensitivity to information that includes structure in light is
rect. not necessarily a “sensory deficit.” Species that have no
There can be redundancy across different forms of am- eyes (e.g., worms and moles) cannot detect patterns in the
bient energy, as postulated by the amodal specification global array that extend across the optic array and, like
view. In the amodal view, redundancy can be detected only blind individuals, cannot control behavior in all of the ways
by an internal comparison of patterns detected by different that are available to the sighted. Given the behavioral suc-
perceptual systems (sect. 3.3.3). In our view, redundancy is cess enjoyed by these species, it would appear to be inap-
a higher-order relation in the global array that can be de- propriate to refer to their perceptual experience as being
tected directly. The global array pattern that is created by deficient.
redundancy across individual forms of energy is a limiting
case of structure in the global array, but, like any other 6.2.4. Implications for research in neuroscience. Rather
structure in the global array, it differs qualitatively from re- than investigating the activity of neural units, the ecological
lated structures in single-energy arrays. approach to perception and action focuses on the interac-
We noted earlier (sect. 1) that James Gibson endorsed tion between the animal and the environment and on how
two positions with respect to the information available to this interaction structures ambient energy arrays. However,
different perceptual systems. Our position is not compati- the hypothesis that specification exists, and that it exists
ble with Gibson’s endorsement of amodal specification (see solely in the global array, could have substantial implica-
sect. 3.3.2). However, our position is compatible with Gib- tions for research on the nervous system. One implication
son’s claim that information exists in higher-order patterns is that neuroscientists might search for cells or nuclei that
that extend across different forms of ambient energy. respond selectively to structures in the global array. In neu-
rophysiological terms, this would mean searching for neural
6.2.3. Information, energy, and sensory loss. It might be units that respond to patterns of activity that extend across
argued that the global array does not always exist because different kinds of receptors, such as the retina and the
some forms of energy are not always present, such as in the vestibule. As we noted in section 2.5, many sites in the ner-
dark, or when there is total silence. Such an argument is vous system are known to be influenced by stimulation of
problematic because it relies on a confusion between en- different sensory organs (Stein & Meredith 1993). These
ergy and information. A lack of energy does not constitute sites are commonly interpreted as loci for inferential (e.g.,
a lack of information. Rather, the absence of a form of en- associative) integration of inputs from different receptors,
ergy is information (at a minimum, it is information of the but they could be reinterpreted as loci of information
absence of energy, e.g., for the fact that it is dark). James picked up from the global array. Rather than forming asso-
Gibson made this argument in the context of single-energy ciations between qualitatively different forms of stimulus
arrays (e.g., Gibson 1966; for additional treatments, see, energy, these sites may exhibit direct (i.e., nonassociative)
e.g., Michaels & Carello 1981; Turvey et al. 1981). He ar- sensitivity to qualitatively unitary patterns in the global ar-
gued that information for perception exists in patterns in ray. With such a reinterpretation, the existing knowledge
energy, not in energy per se. We believe that this argument base could be built upon in studying neural sensitivity to
holds in the context of the global array. If so, then the ab- structures in the global array. Research of this kind would
sence of any given form of energy would not imply the ab- be greatly facilitated by prior identification of such struc-
sence of the global array. tures, so that they could be presented and manipulated in
This has consequences for the loss of receptor systems, a controlled manner during recording of neural activity (re-
as occurs in blindness or deafness. In our view, all per- search on the global array could also be conducted in clin-
ceivers detect patterns in the global array. Individuals who ical populations; cf. Lee et al. 1984). As an example, Equa-
have suffered perceptual loss have lost their sensitivity to a tions 1 and 2 might be used to generate stimuli that could
particular class of these patterns, but they remain sensitive be used in studies of the nervous system.
to the remaining classes of patterns. Most people can de-
tect patterns in the global array that extend across optics, 6.2.5. Perception of unimodal stimuli by stationary ob-
acoustics, mechanical pressure, gravito-inertial force, and servers? We have argued that the global array is the sole
chemical energy. By contrast, blind people can detect only source of information sufficient for veridical perception,
those global array patterns that extend across acoustics, me- and that it contains information that is essential for the per-
chanical pressure, gravito-inertial force, and chemical en- ception and control of motion relative to different physical
ergy, while deaf people can detect only those patterns that referents. How does this argument relate to the perception
extend across optics, mechanical pressure, gravito-inertial of objects and events that influence the structure of indi-
force, and chemical energy. Loss of sensitivity to certain vidual forms of stimulus energy? Similarly, how does it re-
patterns in the global array should have implications for late to situations in which the perceiver appears to be a sta-
performatory action. Consider walking. Normal walking is tionary observer?
guided relative to the illuminated environment, and so de- There are extensive literatures on the perception of ob-
pends upon sensitivity to patterns that extend across optics. jects (e.g., Henderson 1992), events (e.g., Bingham et al.
When these patterns are not available (due to complete 1995), the location of sound sources (e.g., Guski 1990), and
darkness or to blindness), walking is still possible but must so on. In all these cases, experimenters ask observers to per-
be controlled on the basis of other patterns. These other ceive things that structure individual forms of ambient en-
patterns are available to normal animals (e.g., we can walk ergy, such as light or sound. Observers typically are station-
in complete darkness), but do not permit optimal control, ary in the sense that they are asked to remain still, giving

reports of their perceptual experience, rather than having briefly in a semicircle in front of the objects that they were
physical interactions with the objects or events. fixating) could more easily differentiate form than babies
While the experimenter’s interest may focus on a single who were stationary. Motion of the babies’ bodies (relative
modality, sensory stimulation is continuously multimodal. to the gravito-inertial force environment and relative to the
Changes in the structure of a given array occur in the con- illuminated environment) altered the structure of the optic
text of ongoing, simultaneous patterns in other ambient ar- array (e.g., through motion parallax) of somatosensory stim-
rays and, consequently, in the global array. In these situa- ulation and of vestibular stimulation (through variations
tions, as in any other, the global array exists and is available in the direction of balance). Note that stimulation of the
to perceivers. The fact that the experimenter is concerned vestibular and somatosensory systems differed because the
only with how perception is influenced by structure in one head is not rigidly attached to the torso (cf. Riccio 1995). In
form of energy does not imply that observers have an addition, body motion altered relations between patterns
equally narrow focus. We believe that even when asked to in these forms of energy, that is, structures in the global
make perceptual judgments about so-called unimodal stim- array.
uli, observers are motivated to sample (and, in fact, rely Finally, Mark (1987; Mark et al. 1990) has shown that re-
upon), information that is available in the global array. It is striction of postural sway can inhibit learning of changes in
almost certainly the case that observers perceive more than affordances. Standing observers looked at a chair whose
the experimental stimulus. Experimenters may present “an seatpan was height-adjustable and made judgments about
object or event,” and observers may comply with instruc- the maximum seatpan height on which they could sit. In
tions to give reports about “an object or event.” However, some conditions, observers’ shoes were fitted with blocks
rather than perceiving “an object or event,” observers may that increased their height and so their maximum sitting
perceive “an object or event that I am looking at (or listen- height. Immediately after donning the blocks, judgments of
ing to, etc.), in this place.” That is, observers may perceive maximum sitting height were inaccurate, but over a series
objects and events in relation to themselves and their sur- of trials judgments gravitated toward the correct (new)
roundings. Information about these relations is found only value despite the fact that observers were not permitted to
in the global array. Perceptual reports may not include practice sitting (Mark 1987). In later experiments, Mark et
these relational facts because the experimenter has not real. (1990) found that this spontaneous learning did not oc-
quested reports of this kind. cur in the absence of postural sway (when observers were
Relational information in the global array may be critical required to stand with their body and head pressed against
for perceptual reports that are commonly assumed to re- a wall).
flect unimodal perception. This is because perceivers are On the basis of these studies we conclude that even “sta-
active rather than passive; the success of perception re- tionary, passive” observation depends upon successful con-
quires adequate control of this act. In order to achieve and trol of movements of receptor systems and often of the
maintain visual fixation, for example, we must stabilize the whole body. We have argued that information for the con-
eyes, the head, and the body. In order to explore objects or trol of motion relative to physical referents exists only in the
events, we must be able to organize and control stable move- global array. Accordingly, we conclude that information in
ments of all of these. This suggests that observers are nei- the global array is required even for perception of objects
ther stationary nor passive (Gibson 1986). The actions and events that structure only a single form of ambient en-
needed for perception require information in the global ar- ergy.
ray.
The need for action in perception has been documented 6.2.6. Sensory interaction: The McGurk effect. As a final
at a variety of levels. Here we concentrate on subtle rela- example of the application of our theory to existing re-
tions between perception and postural motion; these mo- search, we reconsider the McGurk effect (see sect. 3.1.1),
tions can provide information to “stationary” observers. which is widely interpreted as reflecting general principles
Empirical research suggests that there is an intimate rela- of intersensory interaction (e.g., Kuhl & Meltzoff 1988;
tion between body motion and perception in contexts that Welch & Warren 1986). In studies of this effect, the visual
typically are analyzed without reference to motion of the portion of a videotape shows a speaker saying one syllable,
observer. while on the audio track a different syllable is presented.
As a first example, it is known that observational activi- Observers are instructed to report the syllable on the audio
ties as simple as visual fixation of stationary objects are in- track, and perceptual reports are strongly influenced by the
fluenced by controllable variations in postural motion. Stof- nominally ignored visible speaker. One of the most consis-
fregen et al. (1999a; 1999b; 2000b) instructed participants tent and dramatic findings is that perceptual reports fre-
to fixate a distant target or a nearby target while standing. quently are not consistent with either the visible or the au-
Reliable variations in parameters of postural sway were dible event. Rather, observers often report “a syllable that
elicited by changes in the fixation task. These differences has not been presented to either modality and that repre-
were observed across changes in the visual target (a blank sents a combination of both” (Green et al. 1991, p. 524).
target versus a block of printed text), in the nature of the vi- This presents a challenge to inference-based theories of
sual task (simple fixation versus search for target letters), speech perception (Green et al. 1991; McGurk & Mac-
and changes in target distance (near vs. far). Variations in Donald 1976); the sustained interest in the McGurk effect
posture were functionally related to constraints imposed by arises in part from the need to explain how it is that the fi-
the visual tasks, that is, modulations of postural sway facili- nal percept differs qualitatively from the patterns in the op-
tated visual performance. tic and acoustic arrays.
Similarly, Kellman and Short (1987) investigated the role In experiments on the McGurk effect, participants that
of body motion in the development of perception of three- are exposed to multimodal stimulation are asked to give
dimensional form. Babies who were moved (oscillated perceptual reports that are unimodal, that is, they are asked

to report only what they hear (e.g., McGurk & MacDonald The possibility that specificity exists solely in the global
1976). Numerous studies have documented the fact that array provides the possibility of direct perception, but only
reports are not consistent with structure in the acoustic ar- if the senses function as a single unit. To accept this possi-
ray .11 This is consistent with our general premise that per- bility requires rejection of the assumption of separate
ceptual systems do not function independently, but work in senses. A view emerges in which perception consists not of
a cooperative manner to pick up higher-order patterns in a group of systems working in parallel (and often in con-
the global array. In fact, we would predict just such an out- flict), but of a single system whose parts operate as a unit to
come. If speech perception is based on information in the pick up information that is available only to the unit.
global array, then it must be unnatural (or at least uncom- Our analysis has broad implications for research on per-
mon) for observers who can both see and hear the speaker ception and action. A few of these are briefly discussed
to be asked to report only what is audible; the global array here. In studies of perception researchers may need to take
provides information about what is being said, rather than into account the global information that is always available.
about what is visible or what is audible. Our position is sim- Researchers commonly present to subjects only a single
ilar to that of Fowler and Dekle (1991) in that we stress the form of energy. It is assumed that the application of stimu-
fact that multiple perceptual systems are stimulated simul- lus energy to only one sense is grounds to ignore, method-
taneously and that the stimulation has a single source (i.e., ologically and analytically, the energy available to other
a speaker). Our position differs in that we do not assume senses. However, when an experimenter stimulates a single
that observers are separately sensitive to structures in the modality, there is an influence on structure in the global ar-
optic and acoustic arrays but, rather, propose that observers ray (e.g., Figs. 4 and 5). Research is needed to determine
are directly sensitive to patterns that extend across these ar- whether animals are directly sensitive to structure in the
rays, that is, to patterns in the global array. Because such global array.
patterns are external to the perceiver, perception of speech Such research will require novel experimental method-
via information in the global array does not entail an inter- ologies and novel analyses. Can we manipulate structure in
nal process of integration. In research on the McGurk ef- the global array independent of structure in single-energy
fect, the discrepancy between the visible and audible con- arrays? One promising strategy is to use the method of pair-
sequences of speech is commonly interpreted as a conflict wise comparisons (Fouque et al. 1999). This method con-
between the two modalities, but it could also be interpreted sists of fixing the structure in one or more single-energy ar-
as creating information in the global array that specifies the rays while systematically varying the structure in the global
experimental manipulation, that is, the global array may array (across experimental conditions) in situations that
specify that what is seen and what is heard arise from two have consequences for behavior (Fouque et al. 1999). A re-
different speech acts. This leaves the question of why ob- ciprocal option is to fix parts of the structure of the global
servers often do not detect the manipulation. We regard array while varying the structure of individual forms of en-
this as an issue of perception (i.e., information pickup), ergy. This might be achieved if variations in different forms
rather than an issue of specification (i.e., the existence of in- of energy were appropriately coordinated. The possibility
formation). This is addressed in the next section. of additional methods should also be pursued.
It is essential that experimenters understand the physics
underlying experimental work and the physical reference
7. Conclusion frames relevant to the task. For instance, when a judgment
task involves the detection of stasis or motion, instructions
We have reconsidered traditional concepts of the senses. given to the subjects should be very specific with respect to
We have argued that there is no clear basis for the assump- the referents that are to be used for the perception of mo-
tion that perception is accomplished by a set of distinct per- tion. When the proper physics are employed, errors, illu-
ceptual systems. This led us to reconsider concepts of po- sions, and variability should decrease. In addition, behav-
tential sensory stimulation. We reviewed existing concepts ioral measures should be primary to phenomenal measures
of specification, that is, of the possibility that relations be- for those studies that investigate perception and control of
tween reality and patterns in ambient energy are unique. movement (Fouque et al. 1999; Riccio 1995).
We considered the hypothesis that specification does not Errors occur in both perception and performance, but
exist, the hypothesis that specification exists within individ- the existence of errors does not imply a lack of specificity.
ual forms of ambient energy, and the hypothesis that spec- Rather, errors may imply a need for perceptual-motor
ification exists redundantly within or across forms of energy. differentiation (learning) of those structures in the global
We argued that the assumption of separate senses creates array that are relevant to a given behavior. Errors can be
problems for any theory that assumes the existence of spec- expected when animals are prevented from exploring task-
ification. We then reviewed the physics of motion in an ef- relevant dynamics of the animal-environment system (e.g.,
fort to determine whether the structure within a single form Mark et al. 1990). This may account for the common ob-
of ambient energy can have a specificational relation to servation that observers in studies of the McGurk effect do
physical motion. We concluded that this is not possible; that not detect the experimental manipulation (i.e., the fact that
is, we concluded that specification cannot exist within the the audible and visible events are different syllables). In the
assumption of separate senses. In section 6 we proposed learning of perceptual-motor skills, one problem is to dis-
that specification exists. We proposed that structure exists cover and exploit different structures in the global array.
in the global array, that this superordinate structure carries Learning a somersault, for example, appears to depend
information that does not exist in any of the individual ar- upon the discovery and control of higher-order relations be-
rays, and that it is essential for accurate perception and con- tween vestibular, mechanical, and optical patterns of en-
trol of behavior with respect to the multitude of real refer- ergy (e.g., Bardy & Laurent 1998). To our knowledge, the
ents. literature on motor control and sport has not addressed the

existence of the global array, the evolution of its structure Stoffregen & Riccio 1988; 1991). The concepts of conflict and re-
during behavior, or the process of learning to pick up struc- dundancy across perceptual systems are important across a broad
tures within it that are relevant to particular behaviors. We range of research, including social psychology (e.g., Pennebaker
believe that a focus on the global array in the context of skill & Roberts 1992) and neurophysiology (e.g., Maunsell et al. 1989;
learning can reveal regularities and changes that may aid Stein & Meredith 1993). For a discussion of the relation between
input conflict and other concepts of intersensory conflict, see Stof-
our understanding of the learning process, ultimately lead- fregen and Riccio (1991).
ing to enhanced performance. In general, the existence of 5. Our discussion of physical referents might suggest that we
the global array poses new challenges for the study of per- believe that these referents are perceived. We do not argue that
ceptual-motor learning and development. Are infants (and physical referents are perceived as such. Our argument is that be-
novices, in general) sensitive to patterns in the global array, havior is perceived and controlled with reference to physical ref-
and if so, how does this sensitivity develop with experience? erents. In the ecological approach to perception and action, it is
It might be supposed that for infants and novices, initial the relation between the animal and the referent that is perceived
sensitivity is to structure in single-energy arrays, with expe- and controlled (that is, the affordances for behavior relative to the
rience leading to the pickup of structure in the global array. referent). Perception of “behavior relative to a physical referent”
One problem with this is that it requires the assumption of does not require prior or independent perception of the referent,
per se, just as perception of a triangle does not require prior or in-
separate senses. Another is that it would require that dependent perception of the lines that make up the triangle. For
novices begin with sensitivity to nonspecific structures, further discussion of this issue, see Gibson (1986) or Stoffregen
which is contrary to and unnecessary in ecological theory. and Riccio (1988).
In this target article we have questioned some of the most 6. Even small changes in velocity, such as those caused by head
fundamental assumptions that underlie theories of percep- movements, can have profound effects on the perception of self-
tion. Our rejection of the assumption of separate senses and motion. This accounts for the fact that the experience of vection
our analysis of physical referents for perception and control is more easily induced in restrained subjects (e.g., Dichgans &
pose challenges for any theory of perception. Our rejection Brandt 1978; Wertheim 1994).
of the hypothesis of specification in single-energy arrays 7. We refer to the direction of balance rather than to the di-
poses challenges for the ecological approach to perception rection of gravity (or, more properly, the direction of the gravito-
inertial force vector). In most situations, the direction of balance
and action. Finally, our presentation of the global array of- is contraparallel to the gravito-inertial force vector. However, the
fers the possibility of a theory of the perception and control two entities differ qualitatively (the direction of balance is defined
of behavior that is based on the lawful specification of prop- in terms of kinematics, the gravito-inertial force vector in terms of
erties of the animal-environment system. kinetics), and recent research has shown that both the perception
and control of body orientation are influenced more strongly by
ACKNOWLEDGMENTS the direction of balance than by the gravito-inertial force vector
Preparation of this article was supported by the National Science (Riccio et al. 1992).
Foundation (SBR-9601351, INT-9603315) and by the Centre Na- 8. The fact that we can present optical flow in the absence of
tional de la Recherche Scientifique (CNRS/NSF-3899), with ad- motion relative to the earth or relative to the gravito-inertial envi-
ditional support from the French Ministère de l’Education Na- ronment (e.g., in the cinema, in visual flight simulators, and in
tionale, de la Recherche et de la Technologie. “virtual environments”) shows that optical flow is not uniquely re-
We extend our grateful thanks to Gary Riccio for general ideas lated to, and so provides no information about, motion relative to
and discussions beginning in 1982, Karen Adolph and Stavros these referents (cf. Smets 1995, pp. 199–200).
Valenti for helpful discussions of errors in perception-action, 9. This raises the question of why people sway in response to
Steven B. Flynn for help with concepts of receptor anatomy, John imposed optical flow in moving rooms. That is, if there is no per-
Pittenger for help with the independent specification hypothesis, ceptual error, then why do they sway? It may be that they have
Patricia E. Murtha and Peter Suranyi for help with physics, and chosen to stabilize the head and eyes relative to the illuminated
Michael Turvey for helpful conversations on law and specifica- environment (the room), and that they use body sway to maintain
tion. We are also grateful for illuminating discussions with Lor- this stabilization.
raine Bahrik, Gregory Burton, Arlene Walker-Andrews, Florent 10. The optical and gravito-inertial patterns depicted in Figure
Fouque, Masato Sasaki, and students in Thomas A. Stoffregen’s 5a are not identical or redundant. In traditional approaches to
1994 graduate seminar on intermodal perception. visual-vestibular interaction, this type of nonredundancy is inter-
preted as intersensory conflict (see sect. 3.1.3). The global array
NOTES structure depicted in Figure 5b implies that the nonidentity of op-
1. In this target article we distinguish between potential sen- tical and gravito-inertial structure does not need to be interpreted
sory stimulation and actual sensory stimulation. The former con- as conflict (Stoffregen & Riccio 1991). Nonredundancies exist in
sists of patterns in ambient energy fields which exist outside the the stimulation of different perceptual systems, but the interpre-
head and can be analyzed without reference to any psychological tation of these in terms of intersensory conflict is not obligatory.
process. We refer to potential sensory stimulation in terms of ar- Conflict is an interpretation rather than a fact. If specificity exists
rays of ambient energy, such as the optic array. Actual sensory in the global array, then sensory conflict may not exist. Intersen-
stimulation consists of portions of ambient arrays that are sampled sory conflict is widely believed to cause motion sickness, but if it
by perceptual systems. does not exist then this cannot be true. This is part of the motiva-
2. This is distinct from ways in which scientists might analyze tion for an alternative theory of motion sickness (Riccio & Stof-
a complex system. It is possible to study one hand of a violinist, but fregen 1991), for which there is empirical support (Stoffregen et
there are no one-handed violinists. al., in press; Stoffregen & Smart 1998).
3. It might be argued that we have separate senses because the 11. In studies of the McGurk effect, responses that are not
receptor systems have distinct evolutionary histories. However, consistent with the audio track are routinely classified as errors or
this argument is circular because it begins with the assumption illusions. No a priori basis for this classification has been offered;
that there are separate systems. for example, McGurk and MacDonald (1976, p. 746) offered no
4. Many researchers argue that input conflict is uncommon or justification for the error classification other than that it was done
rare (e.g.. Welch & Warren 1986), but in ordinary behavior input “for the purpose of analysis.” As an alternative, observers’ re-
conflict must be the rule rather than the exception (Oman 1982; sponses could be classified into different groups without the de-

Commentary/Stoffregen & Bardy: On specification and the senses
scription of any group as being either correct or incorrect, illusory perception-action system with different relevant control variables
or veridical. The common interpretation is credible only if it is as- that could presumably correspond to different structures in the
sumed that “correct” responses are defined solely in terms of the global array. For example, sitting, crawling, and walking postures
soundtrack, that is, only if it is assumed that the experimenter’s in- involve different regions of permissible sway for different key piv-
terpretation of the situation is shared by the observers. If, as stu- ots around which the body rotates. In addition, each posture in-
dents of the McGurk effect allege, speech perception is inherently volves different muscle groups for executing movements and for
cross-modal, then it could be argued that the natural definition of generating compensatory sway; different vantage points for view-
“correct” would be in terms of percepts that reflect the influence ing the ground; different patterns of optic flow; different interac-
of both modalities. This, in turn, suggests that a more appropriate tions between visual, somatosensory, and vestibular information;
method for studying speech perception would be to ask observers and so on. Thus, a protracted period of learning is required for in-
to report “what you perceive” or “what was said,” rather than fants to identify the relevant control variables for each posture in
“what you heard.” development and to master their on-line calibration.
As predicted by this account, recent experiments showed that
infants must learn to calibrate actions to the changeable region of
permissible sway, but that learning does not transfer from sitting
to crawling (Adolph, in press) or from crawling to walking (Adolph
1997). When infants were tested at the edge of an adjustable gap
Open Peer Commentary (0–90 cm) in an experienced sitting posture, they leaned forward
over safe gaps and carefully avoided leaning over risky ones. But,
when the same infants were tested on the same gaps in an unfa-
Commentary submitted by the qualified professional readership of this miliar crawling posture, they fell into impossibly large gaps on
journal will be considered for publication in a later issue as Continuing multiple trials. Similarly, when infants were tested at the brink of
Commentary on this article. Integrative overviews and syntheses are es- an adjustable slope (0–368) in an experienced crawling posture,
pecially encouraged. they crawled down safe slopes but avoided risky ones. When the
same infants were tested on the same slopes in an unfamiliar walk-
ing posture, they fell down impossibly steep slopes on multiple tri-
als. In both gaps and slopes tasks, the pattern of errors was con-
Learning and exploration: sistent with a differentiation learning process, as proposed by
S&B. Errors decreased at the most extreme increments of gap and
Lessons from infants slope, where information for adaptive motor control was presum-
Karen E. Adolph, Ludovic M. Marin, and Frederic F. Fraisse ably most clear; errors increased at the middle range of incre-
ments, where information would be most difficult to differentiate.
Department of Psychology, New York University, New York, NY 10003.
{adolph; martin}@psych.nyu.edu [email protected]
Apparently, experience with each new postural milestone provides
www.psych.nyu.edu infants with the opportunity to differentiate the relevant informa-
tion in the new global array.
Abstract: Based on studies with infants, we expand on Stoffregen & Bardy’s Now, we turn to the second source of errors mentioned by S&B:
explanation of perceptual motor errors, given the global array. Information Inadequate exploration. Information in the global array does not
pick-up from the global array is not sufficient without adequate ex- come for free. As Gibson (1979) pointed out, actions are embed-
ploratory movements and learning to support perceptually guided activity. ded in a continuous perception-action loop. Exploratory move-
ments generate information that, in turn, elicits further explo-
The central point of Stoffregen & Bardy’s (S&B’s) intriguing arti- ration. Despite dozens of experimental manipulations of visual,
cle concerns the problem of specification (the information avail- vestibular, and somatosensory information (e.g., Bardy et al.
able in ambient arrays of energy). On S&B’s account, information 1999), surprisingly little is known about multimodal exploration in
in the global array is essential for accurate perception and adap- the service of guiding stance and locomotion. Which exploratory
tive motor control. They claim that perceptual-motor errors result movements give rise to what sorts of information and what infor-
from inadequate information pick-up. This unhappy situation oc- mation elicits further exploration?
curs when observers cannot differentiate the relevant structures We propose that exploratory behaviors generate the necessary
in the global array (lack of perceptual-motor learning) or when information for detecting upcoming threats to balance in a tem-
task constraints prevent adequate exploration to pick-up the in- poral and spatial sequence (Adolph & Eppler 1998). Exploration
formation (as is often the case in laboratory experiments). from a distance must occur first, yielding concurrent visual, vest-
Our commentary expands on the too brief explanation the au- ibular, and somatosensory information from ongoing body sway
thors provide for perceptual-motor errors. It is important to pro- (Mark et al. 1990; Riley et al. 1997). Information derived from di-
vide a clear explanation of how such errors might occur if, as the rect contact with a surface provides a closer simulation of the rele-
authors propose, information gleaned from the global array is suf- vant forces. But, exploring via direct contact must be elicited by
ficient to support perceptually guided activity. Our commentary prior long distance cues. Thus, errors should result when long dis-
aims to redress this deficiency. Like the authors, we draw our ex- tance cues about surface properties are unreliable. We argue that
amples from studies of stance and locomotion. We show that re- 1ong distance cues for depth (slopes, cliffs, gaps, etc.) are reliable,
sults from studies of infants can inform the debate about spec- but long distance cues for surface friction and rigidity are unreli-
ification. We begin with learning to differentiate the relevant able. Slippery surfaces can be shiny or matte. Squishy surfaces can
structures in the global array. be bumpy or smooth. Unfortunately for walkers, friction and rigid-
Typically, motor development appears as a sequence of postural ity are ubiquitous properties. Our account explains why variations
milestones – sitting, crawling, and walking. To maintain balance in in surface friction – not depth – are the leading cause of accidents
each posture, infants must keep their bodies within a region of from falling in adults (Lin et al. 1995). Like stepping into quick-
permissible postural sway (McCollum & Leen 1989; Riccio & sand, often walkers do not realize that a surface is slippery until
Stoffregen 1988). The extent of this region changes depending on they have already stepped onto it. Similarly, infants do not respond
variations in the ground surface, functional changes in body char- adaptively to information for surface friction or rigidity without di-
acteristics, changes in the location of the center of mass due to rect contact (Eppler et al. 2000; Gibson et al. 1987; Stoffregen et
body movements, and so on. Thus, perceptual control of balance al. 1997). Without adequate exploratory movements, the global ar-
is paramount. Adolph and Eppler (1998; in press) proposed that ray is impoverished and insufficient for guiding activity.
each postural milestone in development constitutes a different In summary, empirical findings from studies of infants’ stance

and locomotion are consistent with S&B’s specification hypothe- pletely different forms of energy are not separate senses, then
sis about the information available in the global array. Adaptive there may be no justification in differentiating any part of the hu-
control of action requires adequate exploratory movements to man body from another. Their example of two eyes being neces-
generate the requisite information and a protracted period of sary for binocular depth perception is not persuasive precisely
learning to differentiate the relevant information structures. In because it is a perception, not sensory transduction. Monocular
other words, using the global array for perception and action re- deprivation in early development prevents the proper connec-
quires exploration and learning. tions in the cerebral cortex from forming that would allow the
perception of depth due to optical disparity, despite the other-
wise normal functioning of both eyes (Blake & Hirsch 1975). Fur-
thermore, child behavioral studies show that the ability to use
Physiological convergence of sensory binocular disparity for depth perception follows, but does not co-
incide with, the ability to control vergence of the eyes, implicat-
signals as a prelude to perception ing an internal experience dependent process (Yonas & Granrud
1985). While there may be only one perceiver, there are certainly
Kurt F. Ahrens
many sensors, and their relations to each other must develop with
Department of Physics, University of California, San Diego, La Jolla, CA
experience, that is, they must be learned.
92093-0319. [email protected] www.physics.ucsd.edu/~kurt
S&B suggest that the GA is more than the sum of its parts, but
provide no supporting evidence. It seems implausible that single
Abstract: The global array may be a useful concept in studying behavior
in a complex environment, especially in the context of dynamical systems
energy arrays interact in any significant physical way. Sound and
theory. However, Stoffregen & Bardy’s arguments are weakened by the light emanating from an event are merely so much sound and
conflation of sensation and perception, and by the lack of evidence for syn- light; no new form of energy is created. One possible explanation
ergy between stimulus energy arrays; strong evidence places the conver- for their viewpoint is that different energy forms from one event,
gence of sensory stimuli inside the head. taken together, imply that event whereas taken singly they are am-
biguous. This logic is predicated upon knowledge of the regular-
Stoffregen & Bardy (S&B) postulate that people are able to di- ity of the environment (natural laws), about which individuals
rectly sample the global array (GA) of potential sensory stimuli learn early in development and throughout life (Thelen & Smith
and thereby perceive higher order patterns that single sensory sys- 1994). Without a learned framework that allows the classification
tems are unable to discern. These superordinate patterns in the of multisensory input patterns, inputs could come from one of a
GA are claimed to specify or characterize the event being per- hundred sources and only be so much noise. The aspect of the GA
ceived in an unequivocal manner while single energy arrays (e.g., that is different from the sum of its parts is the association of stim-
optic or acoustic) may be ambiguous. They conclude that future ulus patterns that the individual has built internally over a lifetime.
research in perception should pay special attention to the GA that Therefore, without a synergetic interaction between the energy
the subject experiences, as the synergetic combination of signals arrays, there is no need to postulate special sense organs respon-
in the GA may produce perceptions that single modality stimuli sive to higher order aspects of the GA; the ordinary senses are
cannot evoke. Furthermore, since body motions and other subtle enough.
behaviors that enable sampling of the GA may be difficult to con- S&B mention several examples of neurophysiological results in-
trol in experiments, they should be recognized and incorporated dicating convergence of sensory input. There are certainly many
into experimental designs rather than unsuccessfully controlled more, but the limbic system is one whose importance for making
and ignored. S&B’s recommendation to broaden the scope of per- associations between disparate stimuli has been repeatedly de-
ceptual and behavioral research is timely in light of the recent monstrated (Agglet et al. 1997; Buckner et al. 1995; Wood et al.
growth of whole brain functional imaging and high resolution 1999). That damage to the limbic system can produce deficits in
MEG and EEG methods. Furthermore, their proposal to focus ef- one’s ability to learn associations supports the idea that the im-
forts on characterization of the GA may be an appropriate adjunct portant relations between elements of the GA are laid down in
to the development of a dynamical systems theory formulation of neural circuitry by a physiological mechanism, but the question
agent-world interactions (Beer 1995); the agent (or individual) is remains how sensation of multiple sensory inputs triggers per-
a complex system, within the greater world-system, for whom the ception of a single event. Studies of the binding problem in visual
GA constitutes the total flux of input from world to agent. Never- perception have implicated correlated neuronal activity in the
theless, elements of their discourse are problematic: they appear perception of separate stimuli as a related group (Gray et al. 1989).
to conflate sensation and perception; they provide no evidence for Other studies have shown widespread patterns of activity corre-
the synergy between energy arrays they claim is the hallmark of lated with the inferred meaning of stimuli (Barrie et al. 1996;
the GA, nor do they provide evidence for novel sensors of higher Smart et al. 1997), and in conjunction with behavioral tasks
order patterns; and they discount the behavioral and neurophysi- (Bressler et al. 1993). These results are all consistent with a dy-
ological evidence that sensory integration occurs within the brain. namical systems interpretation of brain activity, in which the mul-
Sensation is the transduction of ambient energy (whether elec- tifaceted global input array nudges and cajoles the activity of this
tromagnetic, kinetic, or chemical potential) into a spatiotempo- highly interconnected system from one basin of attraction to an-
ral pattern of neural activity. No meaning is conveyed by this other in the elaboration of behavior.
process (Freeman 1995); it is entirely mechanistic. However, the
result of sensory transduction is a perturbation of the state of the
nervous system, which may lead to perception of the event pro-
ducing the stimulus energy. This occurs because the sensory
evoked pattern of neural activity interacts with a perceiver that
has learned from experience and is therefore able to construct
meaning from a familiar input pattern. In dynamical systems ter-
minology, the input drives the brain-state into a basin of attrac-
tion (one that is shaped by experience and prior state). S&B as-
sert that perception is undivided. This seems very likely true
and resonates with Freeman’s description of the unity of an
intentional mind (Freeman 1995). The problem arises when the
authors attempt to show that the senses are not separate. If
completely different anatomical structures that respond to com-

We are better off without perfect perception senses. However, attempting to gain access to perfect information
has a price: time. For interacting with the environment, timing can
Eli Brenner and Jeroen B. J. Smeets be much more important than precision. There is no point know-
Vakgroep Fysiologie, Erasmus Universiteit Rotterdam, 3000 DR Rotterdam, ing exactly when a ball will hit you if you only gain access to this
The Netherlands. [email protected] information once it is too late to react to it.
We have shown that it takes 200 msec to react to a change in the
Abstract: Stoffregen & Bardy’s target article is based on the assumption speed of a target that one is trying to hit (Brenner et al. 1998), but
that our senses’ ultimate purpose is to provide us with perfect information only 110 msec to react to a change in its position (Brenner &
about the outside world. We argue that it is often more important that in- Smeets 1997). Since these kinds of movements take only a few
formation be available quickly than that it be perfect. Consequently our hundred milliseconds, this difference in timing is not negligible.
nervous system processes different aspects of information about our sur-
rounding as separately as possible. The separation is not between the
When hitting moving targets the direction in which subjects move
senses, but between separate aspects of our surrounding. This results in their hand does not appear to depend on the target’s velocity
inconsistencies between judgments: sometimes because different frames (Brenner & Smeets 1996; Smeets & Brenner 1995), presumably
of reference are used. Such inconsistencies are fundamental to the way the because the disadvantages of waiting an additional 90 msec out-
information is picked up, however, and hence cannot be avoided with weigh the advantages of having access to reliable velocity infor-
clearer instructions to the subjects. mation.
In our opinion, the main task of our senses is to select the most
Since the Stoffregen & Bardy (S&B) target article deals with hu- suitable information for the task at hand, and to do so fast. From the
man interactions with the environment, it is impossible to ignore moment the information reaches our senses separate aspects of the
the physiology involved. Once one considers the physiology, it be- information are selected and analyzed for specific tasks, or parts of
comes evident that in practice there can be no “specification” of tasks. The selection starts even earlier if one considers the move-
the kind described in the target article. For instance, in color vi- ments we make to obtain the information. Separate independent
sion it is well known that various combinations of wavelengths of processing for different judgments can result in substantial con-
light can stimulate the three kinds of cones in exactly the same f1icts between them (Abrams & Landgraf 1990; Brenner et al. 1996;
manner, so that we are unable to distinguish between them. Sim- Glennerster et al. 1996; Mack et al. 1985). Nevertheless, the sepa-
ilarly, various combinations of ego-motion and motion of the en- ration seems be so complete that we even fail to notice conflicts
vironment can give rise to the same global optical flow (sect. 5.1). between attributes when the conflicts themselves could give us
These are examples of what S&B would call many-to-one map- valuable additional information (Brenner & Damme 1999; Brenner
pings. Unless all information from all the senses is considered for & Landy 1999). The main reason for our judgments normally being
every judgment, many-to-one mappings will give rise to conflict- approximately consistent is presumably the consistency in the world
ing judgments. S&B examine ways to avoid such conflicts. We “outside,” in what S&B call the “global array.”
question whether such conflicts have to be avoided. Abandoning the need to avoid conflicts allows the nervous sys-
Hidden behind worries about conflicting judgments is the as- tem to rely on different information for each judgment. We as-
sumption that our senses’ ultimate purpose is to provide us with sume that each judgment is based on the most reliable informa-
perfect information about the world “outside.” To obtain such per- tion for that particular judgment. Thus, relative positions are
fect information it makes sense to combine input from all the judged from retinal information alone, but egocentric localization
Figure 1 (Brenner & Smeets). Target velocity during simulated ego-motion that matched the simulated velocity in a preceding inter-
val without simulated ego-motion. All velocities are relative to the visual surrounding. Each thick line connects the centres of the ranges
of acceptable target velocities for the seven simulated ego-velocities for one of the five subjects. The thin diagonal line represents a con-
stant velocity of 0.2 m/sec of the target relative to the observer, ignoring the visually simulated ego-motion. The thin dashed line repre-
sents a constant velocity of 0.2 m/sec of the target relative to the visual surrounding. The two panels show the same subjects’ perfor-
mance with two different instructions.

needs extra-retinal information about the orientation of our eyes it presupposes the distinction it is supposed to demonstrate. One
as well (Brenner & Cornelissen 2000) In this example the differ- could use this strategy to disqualify any distinction whatsoever.
ence in information is associated with a difference in the referent Stronger arguments: anatomical distinctions will not work because
that is involved (see sect. 4.5). Our view implies that the referent there are other anatomical distinctions that definitely do not indi-
is fixed for any given judgment, rather than being something sub- cate a functional difference. Physical media and most brain centers
jects can choose as S&B suggest in section 5. are not generally dedicated to only one of the classic senses. Though
We examined subjects’ freedom in choosing a referent by ask- these arguments seem solid, section 2 neglects some stronger jus-
ing them to compare the initial and final velocities of an ap- tifications, like qualia-based distinctions (audition and vision are
proaching target (simulated with both monocular and binocular experienced differently), deficit based distinctions (people can be
cues on a large screen). The methods were very similar to those blind but not deaf), and evolutionary distinctions (electric fish have
used in a similar study on lateral motion (Brenner 1991) The tar- electric sense but humans do not). Two of these last are treated in
get initially approached at 0.2 m/sec while the background was other sections but their implications are more serious than S&B re-
static. During the presentation the visual background started mov- alize, as I will discuss later.
ing in depth so that the optic flow was consistent with forward or To support the second point of the analysis, that senses rarely
backward ego-motion of the subject. At the same time the target work in isolation, S&B give examples of cooperation of the classic
could change its velocity. We determined how fast the target had senses and present an important argument in section 6.2.5 about
to move during the simulated ego-motion for it to appear to con- implicit cooperation when senses seem to work in isolation. How-
tinue to move at the same speed. ever, evidence that sensory cooperation exists does not prove that
The subjects were initially instructed to judge whether the tar- the senses “operate as a unit” (sect. 7), nor does this cooperation
gets’ velocity changed, without explaining what we meant by “the necessarily follow from the arbitrary distinction argument. Fur-
velocity.” Subjects had no difficulty with this task, and all five sub- thermore, perceptual researchers are justified in believing that
jects spontaneously judged the target’s velocity relative to them- they have learned much from studies in which a particular sense
selves, ignoring the visually simulated ego-motion (left panel in or subsense is isolated; research that discovered pheremones
Fig. 1). These results are consistent with previous work on lateral useful in pest control might serve as one particularly practical ex-
motion, in which velocity judgments also appeared to be related ample.
to oneself (Brenner 1991; Brenner & van den Berg 1996). The third part of S&B’s argument, that the global array concept
We then showed the subjects their data and explained to them is the only way to save perceptual specification, seems more like a
that we were simulating ego-motion and that we wanted them to wish. Their implication is that a qualitative difference exists be-
judge the target’s velocity relative to the surrounding. Subjects tween perception through a single sensory channel (if that is even
found this much more difficult. The results are shown in the right possible) and by means of the global array. For S&B, perception
panel of Figure 1. It is clear from the variability that none of the through multiple integrated sensory channels provides a tighter
subjects was really able to do this task. Perhaps they would have and more trustworthy contact with the world than perception
been able to do so if they had actually experienced vection, or if through a single channel.
the simulation was not only visual, but apparently they were not With such implications, one cannot help but wonder how many
able to select the visual surrounding as a referent. systems are necessary to have a global array; one hopes the answer
is not “all of them.” Clearly, it is more than just two. The authors
use the example of airplane simulators that include visual and in-
ertial information. This environment that looks and feels like an
airplane is not one, so optic and inertial information in isolation or
How many systems make a global array? even working as a set do not specify aircraft flight. Since S&B think
the global array provides specificity in the simulator, some other
Gregory A. Burton
system (which they do not name) must disambiguate the simula-
Department of Psychology, Seton Hall University, South Orange, NJ 07079.
tor from the real thing. So in this example at least three systems
[email protected] www.pirate.shu.edu/~burtongr
are necessary. Do all perceivables require three systems or more?
Consider perception of cold. If I need to put on a coat, I do not
Abstract: Stoffregen & Bardy suggest that the global array provides the
specification that is lacking when senses are considered in isolation. This
think that haptic, olfactory or visual information should change my
seems to beg the question of the minimum number of senses in a global intention. If some perceivables require multiple systems but oth-
array. Individuals with sensory loss manage with fewer senses, and humans ers can “get by” on one or two, will an animal or person actually
manage with fewer than electric fish; so specification, if it exists, cannot know in any given situation whether a particular thing was per-
require all possible senses. ceived or whether they were “forced to obtain this information
through inferential processing” (sect. 6.1)? If I perceive some-
Stoffregen & Bardy’s (S&B’s) major theme is well worth endors- thing critical but cannot distinguish whether it was specified or in-
ing, in principle. Nonetheless, I remain unconvinced that evi- ferred, does the specification notion have any meaning at all?
dence against the assumption of separate senses is strong enough Perhaps S&B would contend that specification always requires
to demand changes in traditional perceptual experiments, or that all systems, the whole global array, even in cases that superficially
the utility of their global array construct can survive the problem- seem to require just one. A person under great stress may ignore
atic question of the minimum number of “separate” senses that the cold, so perhaps my perception of cold necessarily implies that
make a global array. I do not, through manifold other channels, perceive some more
Here are three interesting contentions of S&B’s. First, there is urgent source of stress. Aside from leaning toward tautology, this
no airtight way to logically distinguish any two classic senses more strategy seems contradicted by the authors’ discussion of other
definitively than any other two pieces of sensory equipment (like distinctions in sensory resources, sensory deficits, and cross-
the two ears). Second, the “separate senses” construct is illogical, species differences.
as is the prototypical experiment focusing on single senses, be- In section 6.2.3, it is suggested that blind people enjoy a global
cause distinct senses never work in isolation. Third, the concept array that still includes numerous sources of information, even if
of perceptual specification can only be saved by adherence to the one has been lost. Furthermore, the success of eyeless creatures
global array construct. is a demonstration (not to mention, I would add, poor electric-
S&B state the arbitrary-distinction argument most forcefully in sense deprived humanity), that the global array of any particular
section 2, analyzing whether the classic senses can be distinguished individual of a particular species does not have to include any par-
reliably by one of several means. I am afraid that I was unpersuaded ticular sense. If humans can enjoy specification without electric
by the repeated claim that a particular distinction is invalid because sense, then logically some perceivables in the Umwelt of the elec-

tric fish can be specified without electric sense, that is, without the about WS. The second mapping is from AS to perception-action
full global array. Therefore, specification is possible with some- (PA); its primary endpoint is control of adaptive (survival-promot-
thing less than all available sensory systems. How many systems ing) behavior. As S&B suggest, behavioral control is sometimes
does it take? imperfectly adaptive. Every perceptual theory must somehow ac-
The idea that there is a countable minimum number of systems count for PA error.
to make a global array not only seems to violate the spirit of the The probability of adaptive control of behavior depends on the
global array construct but also to demand the very kind of hard collective probability of WS-AS mapping sufficiently specific to af-
distinctions among senses that S&B decry in the first part of their ford adaptive control, of sensitivity to AS affording adaptive be-
target article. For example, if two systems were the minimum for havior, and of adequate performatory capability. Individual sur-
specification rather than inference, then the question of whether vival probabilities depend on the probability of adaptive control of
olfaction and gustation are two senses collaborating (say, on the individual behavior. Species survival requires only that enough in-
perception of food) or one flavor system would have serious im- dividuals survive to reproduce, not that every individual survive.
plications. Some PA error is tolerable.
In essence, opposing theories parse PA error in different ways.
Absolutists assert that the probability associated with WS-AS map-
ping is perfect; they deny any possibility that PA error arises from
Absolutist array specification and species imperfect (non-absolute) WS-AS mapping. Alternatively, the WS-
survival: An ecological perspective AS mapping itself might be imperfect, non-absolute: Multiple
on ecological perception WSs may occasion the same AS; the same WS might yield multi-
ple different ASs; or the WS-AS connection may be only partially
reliable. Thus, some portion of PA error could result from non-
Patrick A. Cabe
unique WS-AS mapping.
Department of Psychology, University of North Carolina at Pembroke,
Species survival ultimately requires only that WS map into AS
Pembroke, NC 28372-1510. [email protected]
sufficiently well to afford adaptive control of behavior. That map-
ping need not be absolute, so long as it is good enough to support
Abstract: Stoffregen & Bardy propose an absolutist transmodal array
structure model, intended to displace models of specification in all exist-
individual survival well enough to promote survival at the species
ing perceptual theories. Absolute specification of world structure in array level (where it counts most ecologically). Ecological considera-
structures, either unimodal or transmodal, may not be provable, but might tions, then, do not necessarily rule out non-absolute WS-AS map-
be falsifiable. Absolute specification, moreover, may not be a necessary ping. Evolution is opportunistic: Whatever works to support
postulate in an ecological approach to understanding perception-action. species survival tends to be preserved across generations. In prin-
ciple, if non-absolute WS-AS mapping can serve this survival end,
Stoffregen & Bardy (S&B) boldly propose an absolutist model of di- organisms may actually have evolved to use non-absolute map-
rect perception via transmodal structure in a “global array.” That au- ping.
dacious venture underscores several critical issues for perception. The ecological perception community faces an enormous chal-
S&B discredit all existing perceptual theories, because each al- lenge here. The nature of WS-AS mapping – unimodal or trans-
lows input conflict (input ambiguity) and therefore implies cogni- modal, absolute or non-absolute – cannot be simply a matter of
tive (indirect) processes. The syllogism is: If array structures (AS) faith or fiat; it must rest squarely on solid physical-mathematical
conflict, then perceivers must choose between alternative world analysis. Absolute mapping demands a completeness proof that
structure (WS) interpretations; choice implies cognitive process- has never been offered, and may not even be possible because it
ing; therefore, input conflict implies cognitive processing. The would require showing one infinite (WS) set to be commensurate
main focus is intersensory conflict, but any AS ambiguity yields the with another infinite (AS) set. Yet the absolutist position appears
same conclusion. Because all existing perceptual theories entail open in principle to disconfirmation (i.e., by showing that some as-
such ambiguities, they all entail cognitive processes. The target ar- pect of WS does not map to any AS).
ticle is silent about why cognitive processing is a fatal flaw, how- Even disconfirmation is problematic, however, because AS it-
ever. self is only loosely defined. The trend in the literature seems to be
But perception is inherently a matter of choice, of differentia- to demonstrate only post hoc the positive existence and utility of
tion of information most relevant for the organism (Gibson 1969). some AS for perception. Gibson (1979) claimed that “inex-
Organisms that cannot choose cannot be said to perceive. One haustible” (p. 243) mathematically describable AS invariants exist.
might grant that WS maps more completely to transmodal AS Such descriptions are as yet relatively sparse and not easily pre-
without conceding that the transmodal array mapping is absolute; dictable a priori on any principled basis (Cutting 1993). Appro-
the transmodal array, too, may map ambiguously to reality. With- priate heuristics to guide array invariant searches remain to be de-
out physical-mathematical analysis to confirm absolute WS-AS veloped (cf. Mace 1977; Neisser 1977), both for the current
mapping, the same criticism that S&B level against existing the- unimodal perspectives that S&B reject and surely for the more
ory may – by the same syllogism – force us to reject their trans- complicated transmodal view they defend.
modal model. Theories succeed when they state clearly under what circum-
Although the target article’s central theme is specification, what stances they do and do not work. Lacking clarity on the key issue of
that term connotes is not completely clear. S&B offer an absolutist specification, those boundaries cannot be drawn for ecological per-
definition: “Only if each stimulus parameter always [their em- ception. The ecological approach, with its reliance on “higher order”
phasis] varies uniquely with the corresponding event . . . would (how high?) AS variables seems relatively immune to falsification.
specification be preserved and input conflict avoided.” Theories The unimodal direct perceptual theory S&B dismiss has been
they reject, however, assert non-absolutist specification. Is speci- partially successful, but remains a perplexing undertaking. The
fication absolute or non-absolute; that is, can WS-AS mapping be transmodal model advanced in the target article exacerbates that
more or less specific? If so, to what degree can species tolerate perplexity. At its present stage of development, S&B’s proposal
non-unique WS-AS mappings? Possibly the absolutist position is seems as open as other perceptual theories to many of the same
not absolutely required. criticisms that these authors outline. The major benefit of this tar-
Perceptual theories all propose two mappings. The first is from get article may be simply to sharpen the focus on key issues that
WS (objects, surfaces, and events) to patterned energy flows (op- remain unresolved in perceptual theory.
tic, acoustic, mechanical, and chemical AS). If that mapping is
unique (at least 1:1 and probably mathematically “onto”; see Sol-
omon et al. 1989), then it is absolute, and absolutely informative

The patterns of energy used for action ception or stimulus discrimination that would include action pa-
are task-dependent rameters is lacking, although some attempts to link behavioral
changes with Skinner’s principle of operant conditioning have
Yann Coelloa and Yves Rossettib been made in the past (e.g., Ingvalsden & Whiting 1997). Unlike
aUnité de Recherche sur l’Evolution des Comportements et l’Apprentissage, S&B (sect. 4. 1), who assume that behavior consists of motions, we
UER Psychology, Université Charles de Gaulle, 59653 Villeneuve d’Ascq, would argue that behavior consists of stable coupling between
France; bEspace et Action, INSERM Unité 534, 69676 Bron, France. perception and action achieved through experience in an adaptive
[email protected] [email protected] context.
www.univ-lille3.fr/www/research/ureca/coello With regard to the last point, it is apparent that adaptive be-
www.lyon151.inserm.fr/unites/534_rossetti.html havior involves discerning a source of information (or invariant,
Gibson 1979) that is relevant according to the goal and the spatio-
Abstract: Is there any ecological purpose in assuming that perception for temporal constraints of the task. Thus, an ecological psychologist
action exists only through a global array of energy? Unlike Stoffregen & would be interested in finding out not only what information is
Bardy, who assume that behavior consists of movements, we would ar-
gue that behavior consists of a stable coupling between perception and ac-
perceived, but also how information and action are inherently re-
tion achieved through experience in an adaptive context. Determining lated. As emphasized by Grush (2000), “egocentric space exists in
target position in an aiming manual task and temporal control of impact virtue of the mastery of a battery of sensorimotor skills” (p. 64).
movement illustrate that patterns of energy used for action are task- This is obviously in line with Gibson’s idea that “the world is spec-
dependent. ified in the structure of light that reaches us, but it is entirely up
to us to perceive it” (1979, p. 63).
Beyond unusual postures and personal rendering, what charac- Behavior consists of actions, and may be initiated without any
terizes a skilled pianist playing a new score is the fact that his gaze prior motion. With regard to the global array hypothesis, S&B
is alternately oriented toward the vertical music stand and the hor- (sect. 6.1) suggest that perception is influenced by events that do
izontal piano keyboard where the hands are performing. A care- not structure single-energy arrays. However a stationary person
ful observation emphasizes that the arms fall under visual control desiring to grasp an object in a static visual environment needs to
mainly when a significant lateral displacement of one hand is re- determine the relative position of that object with respect to a
quired to reach distant keys. What does this tell us about the way point of observation, or a hand-starting position. An aiming man-
sensory signals are used for action? Obviously movements of short ual movement requires the perception of physical properties that
amplitude (say, one octave in the case of a piano keyboard) can be allow a distant object to be located egocentrically, with no need to
controlled through a coupling between visual (a set of music move the head back and forth to generate optic flow (Bingham &
notes), and kinesthetic or somesthetic (changes in configuration Pagano, 1998). Furthermore, directing one hand towards a distant
or in position of hands) signals processed with respect to a non- object usually requires continuous adjustment of hand displace-
visual referent. Conversely, movement of significant amplitude ment as distance is, in general, misperceived when vision condi-
cannot be accurately controlled on the basis of similar informa- tions are limited (Coello & Grealy 1997; Coello & Magnet 2000;
tion. Vision of body segments is needed for accurate distant reach- Foley & Held 1972; Goodale et al. 1986), or when the task is very
ing, the goal of movement being achieved by estimating the hand stringent in terms of spatio-temporal accuracy (Pisella et al. 2000;
to target visual gap including, presumably, a visual physical refer- Proteau 1992). In visuo-manual tasks, no energy patterns except
ent. the visual array would fulfil such constraints since the spatial tar-
Beyond a prosaic illustration, the relevant question suggested get is inevitably specified in the visual ambient array (reaching
by the pianist is how the spatio-temporal constraints of the task movements in darkness towards a sonorous target are usually
impose the selection of a specific source of information (or phys- broadly inaccurate, Berthier et al. 1996). Two examples illustrate
ical referent) which serves as the basis for smooth and accurate this, one relating to action initiation and the other to continuous
movement execution. According to Stroffegen & Bardy (S&B) action guiding.
(sect. 1), perception consists of picking up information that exists From a given point of view, multiple energy arrays may con-
in irreducible patterns across different forms of energy. Physical tribute to the perception of a distant target. Though considerable
aspects of the world are specified in energy patterns reaching the effort has been expended on demonstrating the participation of
sensory systems, but direct perception of behaviorally relevant extraretinal signal in distance perception (e.g., state of vergence,
aspects of reality involves macroscopic patterns (variables) that Treisilian et al. 1999), studies have recently shown that enriching
extend across different forms of ambient energy. Some experi- the visual environment reduces perceptual underestimation of
mental data seem to favor this perspective, for example, the cross- distance as observed otherwise (Coello & Grealy 1997; Coello &
modal perception of the location of a visible and noisy stimulus Magne 2000), even in monocular vision (Coello et al. 2000b).
(e.g., in speech perception as suggested by the McGurk effect). However, the most striking effect was that enriching the visual ar-
However, perception and action cannot be dissociated, even in the ray had a positive effect on action accuracy only when elements
case of speech perception (see Desjardin et al. 1997). This raises structuring the visual scene spread out in the action space (Coello
the question of whether there is any ecological purpose in assum- & Magne 2000). This was interpreted as denoting a dynamic as-
ing that perception for action exists only through the global array pect of space perception underlying the functional use of retinal
of energy. input in the relative coding of target distance. In the context of a
The task for a perceptual psychologist is to uncover the infor- reaching movement, space perception accordingly seems to re-
mation that specifies what we perceive (Gibson 2000). For the quire probing the optical array to determine target distance, which
ecological psychologist, people perceive affordances, that is, prop- cannot be accurately achieved otherwise (Treisilian et al. 1999).
erties of objects that have a value in terms of behavior (Gibson Again, in line with Gibson’s ideas: “the reality underlying the di-
1979; Sanders 1997; Stoffregen 2000). Hence, from an ontologi- mension of space is the adjacent order of objects or surface parts”
cal viewpoint, an object affords a class of actions that have an adap- (1979, p. 101). On the other side of the hand-to-target action, the
tive function for the perceiver. However, taking this assumption encoding of hand position does benefit from various sensory in-
for granted, three issues remain to be addressed: (1) What are the puts, with respective contributions strongly dependent on the ex-
properties of the actor-environment interaction that would con- perimental conditions. Vision may provide a stronger (e.g., passive
tribute to the emergence of affordances? (2) What makes an af- observation) or weaker input (e.g., prior to initiate an action) than
fordance effective at a particular time? and (3) How do afford- proprioception (Rossetti et al. 1995), and, in the dark, the propri-
ances constrain action in such a way that behavior emanates from oceptive contribution can be strongly altered by the cognitive
a continuous coupling between perception and action? With re- knowledge about experimental variables (Imanaka & Abernethy
gard to the first two points, it seems obvious that a theory of per- 2000).

Experimental data seem to support the further notion that initi- should be conceived “as interrelated rather than mutually exclu-
ating an action also includes the capacity to prospectively tune the sive” (Gibson 1966, p. 47), was eclipsed by his final and more
perceptual system to a specific source of information, so as to al- widely cited text, An ecological approach to visual perception
low a continuous and accurate guiding of the motor act. A particu- (Gibson 1979), a work that came dangerously close to celebrating
lar example of such perception-action coupling concerns the con- the self-sufficiency of vision. So we welcome S&B’s examination
trol of impact movement. Some activities, such as golf-putting, of the interrelatedness of the senses. Our commentary begins with
require a fine tuning of effector (putter) velocity at impact in order a plea for terminological consistency in the light of past usage
to make the object (ball) travel the expected path. In general, a con- (S&B are not the first people since Aristotle, Locke, and Berkeley
trolled approach of distant objects in anticipation-coincidence to discuss this issue), and then a plea for common sense. As we see
tasks is considered to depend upon estimating the time remaining it, their article conflates two distinct, if partially overlapping, ar-
before contact with the destination point. A first-order approxima- guments – one that takes things just a bit too far, and the other
tion of time to contact can be achieved by extracting from the op- hardly far enough.
tic flow the inverse of the relative rate of expansion of the contour Terminology. Long ago, Michotte and Metelli wrote about
of the approaching object (Lee 1976), or the relative rate of con- amodal perception or completion to characterize the experience
striction of the gap separating effector and target in the case of an of the continued existence of an object optically occluded by an-
aiming movement directed towards a stationary object (Lee 1997). other, such as the concealment of an object by another object, or
Applying this theory to impact movement (time to gap-closure, its disappearance behind a screen. They coined the term “amodal”
Craig et al. 2000) leads one to admit that absolute movement time because, as they argued, although we do not experience the con-
is known prior to action onset, which needs further examination. cealed object as having gone “out of existence,” it is nevertheless
Another way to understand the control of impact movement is to “out of sight” (Kanizsa & Gerbino 1982; Metelli 1940; Thines et
consider that the optical control variable is contingent on the task al. 1991). In addition, there is the case of what we might call trans-
constraints. Analyzing temporal control of golf-putting has shown modal perception, where a quality proper to one sensory modal-
that the acceleration of arms during the downswing is indeed con- ity is experienced by means of another: seeing the coolness of the
tinuously coupled with the relative rate of change of the visual gap shade of a tree, or the softness of an object. Then there is what von
between the starting and actual position of the putter, with weak Hornbostel (1925) and Werner (1934) called supramodal percep-
involvement of ball position (the time from departure control tion, where the same quality appears to be equally available to a
strategy, Coello et al. 2000a). This distinct way of using visual in- whole range of different sensory modalities. Finally, in our very
formation for controlled approach or impact with a stationary ob- brief and preliminary analysis, there is intermodal perception, the
ject indicates that achieving adaptive behavior involves a continu- simultaneous stimulation of the different senses (for this and re-
ous coupling between action and a specific, task-dependent lated issues, see Marks 1978).
source of information. This seems consistent with other observa- As S&B make clear, their article is not primarily concerned with
tions relating to skill learning. For instance, in gymnastics, despite perception but with the relation between ambient energy array
the high velocity reached, performing a somersault was found to and the environment. And they are concerned with intermodal ar-
rely primarily on visual information. As stated by Bardy and Lau- rays. Now, since Michotte and Metelli already have a just claim to
rent (1998), “because the body is in free fall during the jump, the the term “amodal,” what specific term might we apply to S&B’s
vestibular and the somato-sensory systems are unable to furnish proposed invariants? Since their argument is ambiguous, we will
information about (changing) body orientation relative to the offer two alternatives: multimodal or panmodal invariants.
ground, and thus vision must play a crucial role” (p. 975). Their Specification, representation, and reality. Given their commit-
study suggested indeed that experts’ somersaults are continuously ment to holistic thinking, it is not suprising that the Gestalt psy-
regulated with respect to a prospective visual variable (the first or- chologists questioned the possibility that any isolated energy pat-
der time to contact relating to the ground surface). tern could be specific to its source:
Coming back to the prosaic illustration brought forward in the
. . . just as ambiguous words become more specifically defined when
introduction, one may conclude that for a pianist changes in hand
they are placed in sentences that give them a contextual setting, the am-
posture are specified from visual chords as perceived on the score.
biguity of mediation events is reduced when the stimuli or manifesta-
However, achieving an accurate rendering would depend on the
tions referring to the distal stimulus are embedded in the total situation.
ability to identify the source of information that would enable the
(Heider 1958, p. 35 et seq.)
pianist to reach the correct keys smoothly and accurately with re-
spect to the spatio-temporal contraints prescribed by the music Although von Ehrenfel’s (1980) classic statement of the Gestalt
precepts. principle acknowledged the existence of intermodal Gestalten,
much of the subsequent Gestaltist literature remained curiously
limited to discussions of unimodal Gestalten, usually visual Gestal-
ten. S&B therefore could be seen as pursuing the holistic logic
of Gestaltism in their denial that any unimodal pattern of energy
Getting real about invariants could constitute information in Gibson’s sense. However, as we
see it, they conflate two distinct arguments. The first argument is
Alan Costall,a Giulia Parovel,b and Michele Sinicob that no unimodal structure could be specific to any environmen-
a
Department of Psychology, University of Portsmouth, Portsmouth P01 2DY, tal property or event, an argument they base on a convincing but
United Kingdom; bDepartment of General Psychology, University of Padua, very limited example, that of proprioception: as they convincingly
Padua 35100, Italy. [email protected]
establish, neither optical nor mechanical stimulation would be
{parovel; sinico}@psy.unipd.it
www.port.ac.uk/department/psychology/staff/alan.html
sufficient in this particular case. Rather, multimodal stimulation
(optical-vestibular) is necessary. But they also invoke a second and
Abstract: Stoffregen & Bardy argue that unimodal invariants do not exist,
quite different argument: that no energy pattern disconnected
and that only “amodal” invariants are possible. But they confuse two sep- from all possible additional energy patterns could specify the pres-
arate issues. Amodal invariants, we argue, do indeed exist to specify fea- ence or reality of an environmental property or event, as in their
tures of the environment, but not even an amodal invariant, in isolation, example of an approaching boulder. And this calls for a panmodal
could specify their “presence” or “reality.” array of a limitless kind.
We are not convinced that S&B have established their general
If the textbooks are to be believed, “perception” is largely re- claim that unimodal structures cannot ever be specific to environ-
stricted to vision. Even the important message of James Gibson’s mental properties and events. A pheromone may be completely
The senses considered as perceptual systems, that the senses specific to the female silk-moth, or a fingerprint to a particular

person. It is true that these structures do not, in themselves, spec-
ify the presence of a silk-moth or a person (the whole point of us-
ing fingerprints in forensic inquiry is that the culprit is no longer
at the scene of the crime). Nevertheless, these structures are not
cues in the traditional sense: they are not probabilistically related
to their reference. On the other hand, the specification of pres-
ence calls for more than even the amodal (multimodal or pan-
modal?) invariants identified by S&B. As James Gibson put it, the
fact that one is dealing with a real situation is specified by the very
inexhaustibility of the information available (Gibson 1979, p. 256).
As we see it, the distinction between the specification of an en-
vironmental property and of its presence can help us deal with two
otherwise awkward problems. The first is that the demonstration,
in the laboratory, of the “functional validity” of an invariant – even Figure 1 (Feldman & Lestienne). Tonic activity ( f ) of spindle
an amodal invariant – would seem to destroy what it seeks to af- afferents is related to muscle length, x. (A) The effect of gamma-
firm, the specific relation to the actual environmental event. But, efferent influences on the output signal of spindle afferents is
as Gibson emphasized, “The experimenter . . . is not trying to sim- measured by the shift of the solid line (arrow). As a result, the
ulate reality. [One] could not create the illusion of looking around same afferent activity may be associated with different muscle
and walking through the countryside in any case, for [one] would lengths (e.g., for the points on a vertical line) and the same mus-
have to create the countryside” (Gibson 1979, p. 305; emphasis cle length with different afferent outputs (for the points on a hor-
added). izontal line). The ambiguity disappears (B) if the muscle length is
The second problem is this. Despite their argument against the measured based on the changes in the afferent activity ( f ) relative
very possibility of unimodal invariants, in the Web version of their to that ( fr ) elicited by gamma influences at a referent muscle
article S&B not only chose to include photographs of themselves, length (xr) for which f2fr 5 0.
but photographs that look remarkably like them – and (as far as
we know) nobody else. Which is not to say that we think they are
there on the page. Plenty about them is, of course, missing from movements actively produced by subjects, spindle afferents may
their images, and plenty about their images informs us that it is fire at the same frequency despite the changes in the arm position
their images rather than themselves that are there (Polanyi 1970; (Hulliger et al. 1982). Despite the ambiguity of the relationship
Thines et al. 1991, pp. 117 et seq.). between the spindle output and the position of arm segments, the
arm is correctly perceived as being motionless in isometric condi-
tions although the activity of spindle afferent changes. In isotonic
conditions, the arm is perceived as moving although the spindle
With either separate or integrated arrays activity remains the same.
of senses, perception may not be direct How is this remarkable ability achieved? Stoffregen & Bardy
(S&B) could suggest that adequate position perception (kinesthe-
Anatol G. Feldmana and Francis G. Lestienneb sia) results from some integration (a “global array”) of spindle sig-
aDepartment of Physiology, Neurological Science Research Center, nals with those of other sensory systems. This would be consistent
University of Montreal, Institute for Rehabilitation of Montreal, Montreal, with the evidence that perception of position is based on signals
Quebec, Canada H3S 2J4; bCentre de Recherche en Activités Physiques et from muscle, joint, and skin afferents (Gandevia & Burke 1992)
Sportives Université de Caen Basse-Normandie, UPRES EA 2131, UFR, and that these signals are integrated at the level of neurons of as-
STAPS, 14000 Caen, Cedex, France. [email protected] cending tracts (Arshavsky et al. 1985; Lundberg 1975; McCrea
[email protected] www.crosswinds.net/~afeldman 1992). However, it has been shown that neurons integrating these
afferent signals also receive independent signals from central pat-
Abstract: The information required for perception may be available in the tern generators (e.g., Arshavsky et al. 1985). Due to these inde-
energy arrays that stimulate sensory organs but in a form not directly suit- pendent signals, the ambiguity of the relationship between the
able for the planning and execution of the organism’s actions in the envi- arm position and sensory output is likely not reduced but rather
ronment. The requisite form of information is obtained, with no loss of ad-
equate perception, by representation of sensory stimuli in frames of
induced by the nervous system at any level of sensory integration,
reference determined by internal control signals producing actions. This contrary to the basic idea of the target article.
process seems evolutionarily advantageous but makes perception essen- An answer to the question of how, in principle, adequate posi-
tially non-direct, regardless of the degree of intra- or inter-modal sensory tion sense can be achieved despite the ambiguity has been given
integration. in the framework of the lambda-model for motor control (Feld-
man & Levin 1995). According to the model, the central nervous
To plan and control movements of an artificial arm, robotic engi- system organizes spatial frames of reference for the sensorimotor
neers use “sensors” that directly, and usually separately, measure apparatus and produces active movements by shifting the frames
kinematic variables (the position, velocity, and acceleration of arm in terms of spatial coordinates. The hypothesis has been sup-
segments). As if on purpose, the nervous system avoids direct ported in several recent studies (Gribble et al. 1998; Feldman et
measurement of kinematic variables. For example, muscle spinal. 1998; Lestienne et al. 2000). The spatial frames of reference
dle receptors are sensitive to changes in muscle length and, as used for actions can also be used in perception (Feldman & Latash
such, they play a major role in the perception of position of body 1982; Feldman & Levin 1995). In particular, kinesthetic percep-
segments (Matthews 1981). However, in spindle afferents, the po- tion is likely based not on sensory signals per se but on these sig-
sitional signals are mixed with those related to the rate of change nals represented in a frame of reference determined by central
in muscle length. Moreover, afferent output is influenced by in- control signals underlying the motor output. For example, gamma-
dependent efferent signals from gamma-motoneurons, making motoneurons not only influence the sensory output of spindle af-
the relationship between the activity of spindle afferents and po- ferents but also the muscle motor output, via spindle afferent
sition ambiguous, even in static conditions (Fig. 1A). feedback to motoneurons.
For example, during tonic isometric contractions, the activity of To perceive position, some neural systems presumably measure
muscle spindle afferents increases with increasing torque under spindle afferent activity relative to its background level (“efferent
the influence of gamma-efferents (Valbo 1974) although arm po- copy”) specified by gamma motoneurons (Fig. 1B). During iso-
sition remains unchanged. On the other hand, during slow isotonic metric contractions, the efferent and afferent activity increases in

parallel so that the relative changes in the spindle activity are The global array: Not new
about zero, thus signalling that the arm is motionless. On the other to infant researchers
hand, during isotonic movements, the decrease in the spindle af-
ferent activity normally resulting from muscle shortening is likely Ross A. Flom and Lorraine E. Bahrick
compensated by the increasing activity of gamma efferents so that Department of Psychology, Florida International University, Miami, FL 33199.
the spindle activity may remain unchanged despite the changes in {flomr; bahrick}@fiu.edu www.fiu.edu/{~flom; ~bahrick}.html
the arm position. However, due to changes in the referent signals
coming from gamma motoneurons, the arm is perceived as mov- Abstract: We find Stoffregen & Bardy’s argument that the senses are
ing. Afferent signals from tendon organs, joint and skin receptors, united and that specificity exists within the global array compelling. How-
also measured relative to motor control signals, could contribute ever, this view is not entirely new and research on the development and
to kinesthesia in consonance with muscle spindle afferents (Feld- the origins of perception in infancy, inspired by Gibson’s ecological per-
man & Latash 1982). spective, also supports their claims. The inclusion of this developmental
research will strengthen and challenge some of Stoffregen & Bardy’s
The hypothesis that position sense involves action-based refer- views.
ents was also used to explain kinesthetic illusions arising during
muscle vibration (Feldman & Latash 1982) as well as the phantom Stoffregen & Bardy (S&B) tackle several of the more thorny issues
limb phenomenon in which a limb continues to be perceived even in the study of perception. In particular, they examine whether the
after it has been amputated (Feldman & Levin 1995; Melzack senses operate independently or collectively as a unitary system in
1989). The hypothesis that perception is based on sensory signals the pick-up of information. They also examine the nature of in-
represented in an action-based frame of reference seems applic- formation available for perception and how information can di-
able to other senses. For example, in all cases of postural control rectly specify an organism’s opportunities for action. S&B present
described in the target article, the requirement of mechanical bal- a compelling and richly elaborated argument, inspired largely by
ance only constrains the set of possible body configurations. Sub- Gibson’s theory and contemporary dynamical systems perspec-
jects can still choose a desired posture from this set in a way con- tives, for the claim that perception is based on the detection of
sistent with task demands. Based on experimental data (Feldman higher-order relations that exist across different forms of energy
et al. 1998; Lestienne et al. 2000), it has been suggested that taken in a global (spatiotemporal) array. S&B also argue that this view is
together, action-generating control signals can determine a refer- new and is incompatible with any view of perception that proposes
ent body configuration (“referent body image”) so that external that the senses work separately, or entails a process of internal
forces, including gravity, deflect the system from this configura- comparison across the senses to detect intersensory redundancies.
tion until balance between the muscle and external forces is re- We are in agreement with S&B on many points. For instance
gained at some actual body configuration. we agree with their assertion that information available in the
The same referent configuration could be used in the percep- global array is the fundamental basis for perception and action and
tion of the actual postural configuration to decide whether or not that it is not impoverished; rather information specifying the po-
the latter meets task demands. The ability to specify appropriate tential for action can be directly perceived. We also agree that
referent body configurations may be a basic mechanism underly- there is no clear basis for the assumption that perception is ac-
ing action and perception, which is preserved even in extreme complished by separate senses. Finally, we agree with their asser-
conditions, such as weightlessness when subjects retain the basic tion that perception is never truly unimodal, given that one always
set of body configurations characteristic of terrestrial conditions, perceives the environment in relation to the self. However, we be-
despite substantial changes in the tonic activity of muscles (Les- lieve that S&B have not fully captured the contribution of current
tienne & Gurfinkel 1988). developmental research with respect to amodal specification and
Action-based referent signals also seem necessary for visual infants’ perception of the global array. Their claim that most views
processing. For example, while walking in a room or tilting the of amodal perception entail a process of internal comparison is not
head, we do not perceive that the room is moving despite the mo- completely accurate. Finally, S&B have omitted a convincing ac-
tion of the projection of the visual scene on the retina. This im- count of how modality-specific properties and differences across
plies that the visual constancy is actively supported by referencing the senses are perceived within their perspective.
the visual stimuli to control signals producing motor actions, a Concerning S&B’s discussion of amodal specification and the
process reminiscent of the sense of a constant position during iso- global array, we point out that their view of amodal specification
metric contractions. The suggestion that visual and motor systems is not new. As highlighted by E. J. Gibson:
are interdependent is well supported by experimental observa-
tions of different sensorimotor illusions in humans (Lackner & More accurately, amodal specification refers to the fact that informa-
Dizio 1988; Lestienne et al. 1977) and by neurophysiological stud- tion is not specific to one sensory modality. I mean the term to suggest
ies in animals (Deliagina & Fagerstedt 2000). that there is information in stimulation, which is not tied to specific sen-
In conclusion, neither individual nor global array of sensory sations but is rather invariant over them. An amodal property is not an
stimuli is directly suitable for the planning and execution of ac- intermodal relationship strictly speaking. I mean by it a higher order re-
tions of the organism in the environment. Since actions are most lational stimulation which is not specific to a modality. Intensive di-
essential for the survival of organisms, an action-based represen- mensions might be an example. (Gibson 1969, p. 219)
tation of sensory stimuli has been naturally selected, which makes These assumptions have shaped much of the infant work con-
perception essentially non-direct, regardless of the degree of in- ducted from the ecological perspective (e.g., Bahrick 1988; 1992;
tra or inter-modal sensory integration. 1994; Bahrick & Lickliter 2000; Bahrick & Pickens 1994; Eppler
1995; Rochat 1995; Schmuckler 1995; Walker-Andrews 1988;
1997). These developmental researchers have similarly described
the global array as a spatio-temporal array, and have also empha-
sized that amodal specification is abstract, higher order, and en-
tails detection of relational information with a unified perceptual
system. These conceptualizations of amodal specification do not
involve a process of internal comparison in order to detect redun-
dancies across the senses (e.g., see Bahrick & Lickliter 2000). Fur-
ther, the “intensity hypothesis” (see Turkewitz et al. 1983) also em-
bodies S&B’s view of amodal specification and describes early
infancy as a period when all stimulation is experienced along an
undifferentiated dimension of intensity.

Given that the study of infants provides a view of the organism Goal directed meaning connects
in a more basic state, a greater emphasis on infant research, espe- perception and specification
cially that conducted to discover the origins and general principles
of perception, would be appropriate for S&B’s discussion of the
bases of perception. For example, infant research has generated Patrick Foo and J. A. S. Kelso
the following insights regarding the nature of perceptual devel- Center for Complex Systems and Brain Sciences, Florida Atlantic University,
Innovation Center II, Boca Raton, FL 33431-09910.
opment, relevant to and consistent with S&B’s views regarding
{foo; kelso}@walt.ccs.fau.edu www.ccs.fau.edu
amodal specification: First, infants are adept perceivers of amodal
relations, including temporal synchrony, tempo of action, rhythm,
intensity, and spatial location (see Bahrick & Pickens 1994). In Abstract: We believe that the task goal in voluntary movements provides
fact, temporal synchrony may be the most “global” amodal rela- meaning to existing information sources in the environment and deter-
mines, in a dynamic way, the use and relative importance of these differ-
tion, as it is inherently relational and cannot be perceived in any ent sources. This task-centered meaning bridges the apparent controversy
single modality alone. Second, infants detect intersensory rela- between what information is available in principle (i.e., specification), and
tions in order of increasing specificity. That is, global, amodal re- what information is perceived.
lations (e.g., synchrony) are detected developmentally prior to
nested amodal relations (e.g., temporal information specifying ob- Stoffregen & Bardy (S&B) analyze three hypotheses about rela-
ject composition, tempo, or rhythm), and amodal relations are de- tions between ambient arrays and physical reality, arguing that
tected developmentally prior to arbitrary, modality-specific rela- that it is not possible, in principle, for there to be a unique rela-
tions (e.g., the red object makes a squeaky sound; the word pen tion between physical motion and the structure of individual en-
goes with the thin object; Bahrick, 1992; 1994; in press). In fact, ergy arrays. S&B argue that physical motion relative to different
there is general agreement that infants detect temporal synchrony referents is specified only in the global array. This perspective
early on, if not at birth (Bahrick, in press; Lewkowicz 2000; Slater poses a challenge to traditional, inference-based theories of per-
et al. 1999). Third, the detection of arbitrary, modality-specific re- ception and cognition, as well as work within the ecological ap-
lations is guided and constrained by detection of amodal relations. proach to perception and action.
Thus infants learn arbitrary, modality-specific relations in the While this target article elucidates the differences between
presence, but not in the absence of amodal relations such as tem- specification (i.e., the existence of information), and perception
poral synchrony uniting the visual and acoustic stimulation (i.e., information pick-up), and the implications for theories of
(Gogate & Bahrick 1998; Hernandez-Reif & Bahrick, in press; perception, we would argue that what is missing is the acknowl-
Slater et al. 1999). Fourth, intersensory redundancy facilitates edgment of the meaningfulness of information, namely the pri-
perceptual differentiation and leads to more effective processing macy of the task-centered goal in these voluntary movements.
than the same information presented unimodally (Bahrick & Lick- S&B state,
liter 2000). Thus, the answer to S&B’s question as to whether in-
fants are sensitive to information in the global array, is a clear sometimes behavior is not controlled relative to the direction of balance
“yes,” and there is already much evidence on this topic. We are ea- even when the direction of balance is present; we sacrifice alignment
ger to find out how S&B integrate the general principles of de- with the direction of balance to some other goal. Examples include a
velopment outlined above into their discussion of perception. soccer goal-keeper diving to catch a shot, and a baseball player diving
Finally, we are in agreement that observers must also detect dif- to catch a fly ball. In such cases, once the player has left the ground, the
ferences in stimulation that emanate from different energy ball may be the sole referent for both perception and control.
sources. Detection of amodal invariants requires detecting what is What we would argue is that even before leaving the ground the
constant across different kinds of stimulation. In fact, redundancy ball is the most meaningful referent because the goal of the task
of this sort is a powerful garner of infant attention and infants de- is to catch the ball!
tect amodal properties (e.g., synchrony, rhythm, tempo) develop- We believe that the task goal determines the dynamic use of dif-
mentally prior to other properties. Further, according to the in- ferent sources of information and the saliency of each informa-
tersensory redundancy hypothesis (Bahrick & Lickliter 2000), tional source during the completion of a task. Recent experiments
infants are better able to discriminate an amodal property when it from our laboratory have shown how the introduction of different
is presented bimodally than unimodaly. Bahrick and Lickliter types of local sensory information (e.g., haptic or auditory) can
(2000) demonstrated that 5-month-olds detected a rhythm when both locally and globally stabilize a subject’s coordination during
it was presented visually and acoustically (a hammer banging in conditions where it would normally become unstable (Fink et al.
synchrony with its impact sounds), but not when it was presented 2000; Jirsa et al. 2000; Kelso et al. 2000). In addition, it has also
either visually or acoustically alone. This intersensory facilitation been shown that the same information (e.g., relative phase) can be
entails detection of the convergence of stimulation (rhythm) important for disparate tasks, such as movement pattern recogni-
across different modalities (a higher order temporal relation) and tion and production (Haken et al. 1990). Furthermore, when a bi-
therefore also implies an appreciation of the differences in stimu- ological system functionally stabilizes a system that is inherently
lation across modalities. Pick up of differences, too, must be ac- unstable (i.e., an inverted pendulum), the visually specified time-
complished with a unified perceptual system. In fact, we would to-balance drives the corrective actions necessary to avoid a cata-
suggest that the greater the differences in stimulation, the more strophic fall (Foo et al. 2000). However, during non-critical mo-
the redundancies across these differences stand out as higher or- tions of the pole, the same time-to-balance information does not
der invariants. One need not conclude that we have separate appear to be important.
senses in order to register the differences in stimulation emanat- One extension of this view that the goal determines the mean-
ing from different energy sources. Our perceptual system is flexi- ingfulness of the available information is that learning is a process
ble, unified across the senses, and attention to these different di- by which the organism discovers just what information is impor-
mensions is a matter of attentional selectivity. tant to the successful completion of the task. From the authors’
Briefly, S&B have done a wonderful job of making explicit and own example, “learning a somersault, for example, appears to de-
questioning the basic assumptions underlying current research pend upon the discovery and control of higher-order relations
and theory in perception. This article serves as a call to other re- between vestibular, mechanical, and optical patterns of energy
searchers to examine these issues, make explicit the assumptions (Bardy & Laurent 1998).” This process of discovering the relevant
underlying their research programs, and evaluate their internal perception/action relationships may also be seen in motor devel-
consistency. opment (e.g., prehension in infants, see Thelen 1990)
In short, the linkage between specification and perception is
driven by meaningful (i.e., task specific) informational variables

and their dynamics, the origins of which likely lie in the co-evolved energy is information” (emphasis in original). How does one un-
relationship between the organism and its environment. What derstand the situation confronting a congenitally blind person, for
could be more meaningful and specific to the goal directed func- whom there has never been this variety of stimulation? It seems
tioning of a system than information that specifies the coordina- perverse to suggest this person detects the absence of the optic ar-
tion among its parts? We are not only creatures of perception and ray; he never had access to it, and hence even having the knowl-
action; we are goal-directed creatures. As Sherrington (1906) re- edge that he lacks vision would not constitute information. Instead
marked, a dog not only walks; it walks to greet its master. he has access to a global array that has one dimension fewer than
that available to his sighted sister. Can this array (maximally global
for him, but less global than that of his sister) specify reality?
If the compressed global array cannot specify reality (as seems
Specification, information, and the loss to be suggested by S&B’s reference to “restricted walking” in the
blind) one would expect that it would not contain crucial infor-
of receptor systems mation carried by the optical structures and that, as a result, his
performance would be less timely and/or accurate. Since mobil-
Barry Hughes
ity in the blind is indeed often tentative, slow and restricted, it may
Department of Psychology, University of Auckland, Auckland, New Zealand.
be assumed that this is indeed the situation that arises. But it
[email protected]
www.psych.auckland.ac.nz/psych/staff/academic.html
hardly seems guaranteed in advance. If the information available
to the blind person can specify reality, then S&B ‘s insistence that
Abstract: Claims made in the target article that information-that-speci-
only the global array is specific cannot be sustained because two
fies is contained in relations among structured energy arrays, considered quite different global arrays (with n21 and n dimensions) would
as wholes, may well be true, but claims that information-that-specifies is both specify the same reality and one would have to conclude that
only contained in irreducible relations among structured energy arrays, the extra dimension contributes neither additional nor qualita-
considered as wholes, fail to persuade, at least in the case of the blind. tively different information. Is it conceivable that both brother
and sister could perceive certain features (or Gibsonian affor-
Stoffregen & Bardy (S&B) find almost all existing theoretical ac- dances) of the world with equivalent accuracy, in situations (such
counts of perception wanting, including both constructivism and as locomotion) that ordinarily depend upon “sensitivity” to pat-
direct realism, and their claims are bound to be controversial in a terns that extend across optics? If there were sufficient informa-
new way. I expect that their arguments will be unacceptable to tion in the blind person’s n21 dimensions to match the perfor-
constructivists for most of the same, noncontroversial reasons that mance of the sighted person’s n dimensions, this would seem to
direct realism has always been unacceptable (see Ullman 1980). imply the complete global array was no more specific than an in-
More controversial are the claims by S&B that the constructivists’ complete one. Either the contents of the optic array contain no
insistence on the ambiguity of stimulation at the level of both in- measurably greater amount of information (on its own or when in-
dividual sensory modalities and Gibson’s (1966; 1979) higher-level corporated into an existing global array) than are contained in the
perceptual systems were correct after all. The specificity that Gib- other arrays, or the (n21)-dimensional global array of the blind is
son argued for and sought experimentally, they argue, lies only in expanded without increasing its dimensionality.
the higher-order structures that extend across perceptual systems. Stoffregen has done as much as anyone to specify the informa-
S&B’s claims appear to constitute a simultaneous advocacy of an tional basis for some remarkable echolocational abilities of hu-
anti-reductionist holism (the information in the global array is not mans (Stoffregen & Pittenger 1995), including the discrimination
only more than but is “qualitatively different from” that in any sin- of the presence and shape of small object shapes based on self-
gle energy array) and an anti-psychological reductionism: that generated sounds that are heard directly and as echoes returning
(only) the physics of (only) ambient global arrays explains speci- from structured layouts, often within a time frame so that the
ficity and information for perception completely and without echoes are available before the original sound has been completed
residue. (e.g., Kellogg 1962; Rice 1967). Recently, I have found evidence
Their arguments for the existence of precise information in the that naive persons deprived of optical structures can make imme-
spatiotemporal structure of multiple sensory arrays (taken as a col- diate use of broadband continuous echoes to make decisions about
lective) may be more specific than that contained in such arrays the passability of gaps between walls when they retain active con-
considered additively, or within any single sensory array, and that trol of the direction in which the (head-mounted) cone of emitted
such information ought to be explicitly studied, are entirely rea- ultrasound is being directed and are free to explore the layout by
sonable. For the analogous reason that it is not possible to account any head movements that they choose (Hughes 2000; cf. Warren
for less variance by adding another factor in multiple regression, & Whang 1987). Plainly, the echoes heard are not the only infor-
the collective information in n energy arrays will never be less than mation available during such activities; the haptic, inertial, and
that contained in n21 energy arrays. However, whether it is qual- mechanical arrays are not obliterated. But what becomes of the
itatively different information that is contained in the so-called global array with the introduction of a new acoustic structure and
global array and whether there is information-that-specifies only the removal of optical ones? Does such modal substitution bear at
in such global arrays, however, are more contentious issues. In the all on the notions of information and specificity? I think so. In-
target article, specificity appears to be identified with information, stances of functionally equivalent perceptual descriptions being
such that only a perfect correspondence of the structure of the secured by different perceptual systems, suggest various hy-
global array with reality constitutes information (“The question is potheses, including that of amodal specification (Epstein 1985).
not whether [the global array] exists but, rather whether it con- Such manipulations may not be specific in the sense in which S&B
tains information”). While I cannot agree with this, my focus here prefer it, but it nonetheless remains theoretically relevant and rich
will be on a somewhat different issue. in practical implications for populations such as the blind.
S&B write as if there is only a single global array, one indepen-
dent of people or perceptual systems. I am puzzled by this. In sec- ACKNOWLEDGMENT
tion 6.2.3, S&B explicitly address the nature of stimulation and in- Grant support by the M & P Paykel Trust (Biomedical) and the University
formation in those with complete loss of “receptor systems.” The of Auckland Research Committee is gratefully acknowledged, as is the
notion of the global array’s status as the sole specifier of informa- hospitality of the staff of Escuela de Psicología, Universidad Católica de
tion, as S&B acknowledge, may be questioned because there are Chile during a sabbatical leave.
situations where the global array is not complete, as in the case of
complete darkness. They respond that the lack of energy need not
constitute a lack of information; in fact, “the absence of a form of

Brain science: A more direct way et al. 1988; see also Frost & Metin 1985 for redirected retinal
of understanding our senses projections to the somatosensory cortex). These visually driven au-
ditory-cortex neurons have typical properties of neurons in the vi-
Teija Kujala sual cortex, such as orientation and direction selectivity. Moreover,
Cognitive Brain Research Unit, Department of Psychology, University of these visual-auditory connections function in a meaningful man-
Helsinki, Helsinki Fin-00014, Finland. [email protected] ner (von Melchner et al. 2000). The ferrets could discriminate
www.psych.helsinki.fi/cbhru gratings presented to the part of the retina from which the pro-
jections had been redirected to the primary auditory cortex. Al-
Abstract: Stoffregen & Bardy suggest that the senses are not separable. though visual acuity was lower than normal for the input in the
However, they have a philosophical approach rather than using direct ev- part of retina that was projected to the auditory cortex, the find-
idence that the nervous system analyzes sensory information in a highly ing is nonetheless critical to the issue discussed in the target arti-
flexible manner. cle: it indicates that the initial analysis of visual information can
take place in the primary auditory cortex.
The target article tackles the popular question of how we perceive These findings on sensory representations and their plasticity in
information; concluding that the existence of separate sensory sys- the cortex support S&B’s main idea that sensory systems are not
tems is in doubt. As evidence, Stoffregen & Bardy (S&B) primar- as separate as previously thought. However, the claim that “there
ily use analogies from physics and examples of convergence and is no clear basis for the assumption that perception is accom-
interactions of different senses, but they largely ignore the exten- plished by a set of distinct perceptual systems” is far too strong.
sive and relevant data on human physiology, sensory systems, and Although there are interactions and synesthesia between the
brain processes in analyzing sensory information. senses, and even though the brain can be very adaptive in how it
In this commentary I present some empirical evidence con- processes sensory information, the senses are also in many re-
cerning the representations of sensory systems in the brain and spects distinct. For example, some sensory functions appear to be
their cross modal changes. Scientists have been enthusiastic about represented in the modality-specific areas of the brain in a rigid
cross-modal brain plasticity since the 1970s but it has been stud- manner. Zangaladze et al. (1999), for example, found that the vi-
ied perhaps even more intensively during recent years. By now sual cortex is involved in analyzing orientations received tactually
there is ample evidence that a brain area which normally processes but it does not seem to be processing somatosensory texture analy-
information primarily from one sensory modality can become re- sis.
sponsive to other modalities. For example, in early-onset deafness The research on sensory representation in the brain clearly pro-
caused by damage to the peripheral nervous system, brain areas vides some support for S&B’s doubt about the traditional view of
that normally process auditory input become responsive to visual separate senses, but S&B seem to take it too far. The flexibility in
input (Neville et al. 1983; Rebillard et al. 1977). Similarly, in early- the processing of sensory input between the modalities is clearly
onset blindess, areas that normally analyze visual information are greater than has been thought previously. However, this does not
taken over by auditory and somatosensory functions (see, for ex- warrant totally abandoning the notion of separate senses, particu-
ample, Cohen et al. 1997; Kujala et al. 1995a; 1995b; 1997; Rau- larly in light of evidence of limits in cross-modal representations.
sohecker & Korte 1993; Sadato et al. 1996). However, not just any
function spreads to brain areas deprived of their normal input. For ACKNOWLEDGMENT
example, the occipital cortex of the blind seems to be activated by This work was supported by the Academy of Finland
attended but not by unattended auditory or somatosensory stim-
uli (Kujala et al. 1995a; 1995b; 2000; Liotti et al. 1998).
These cross-modal activation patterns have a functional role in
perception. If the neural functioning of the occipital cortex of an Expanding the theory: Nonverbal
early blind individual is temporarily disrupted, simultaneous
Braille reading is hampered (Cohen et al. 1997). This shows that
determination of referents in a joystick task
traditionally “visua1” areas of the brain in individuals who have al- Katherine A. Leighty, Sarah E. Cummins-Sebree,
ways been totally blind actually process tactile information. Cross-
and Dorothy M. Fragaszy
modal reorganization of brain functions may take place even after
Department of Psychology, University of Georgia, Athens, GA 30602-3013.
childhood: Visual brain areas are activated by tactile (Buechel et
{kleighty; doree}@arches.uga.edu [email protected]
al. 1998) and auditory (Kujala et al. 1997) tasks in blind subjects www.teach.psy.uga.edu/dept/programsj/biopsych/biopsych.html
who had had normal vision until puberty or adulthood. Whether
this activity corresponds functionally to actual perception or in- Abstract: The arguments of Stoffregen & Bardy for studying perception
formation processing remains to be clarified. based on the global array are intriguing. This theory can be examined in
The neural substrate for these cross-modal changes might be nonhuman species using nonverbal tasks. We examine how monkeys mas-
neurons driven by sensory stimulation other than what is mainly ter a skill that incorporates a two-dimensional/three-dimensional interface.
received by a primary sensory brain area. Murata et al. (1965) We feel this provides excellent support for Stoffregen & Bardy’s theory.
found that, in the cat visual cortex, 47% of neurons respond to tac-
tile stimulation and 38% to auditory stimulation. The human oc- Stoffregen & Bardy (S & B) present situational examples that dem-
cipital cortex, which is traditionally thought to primarily analyze onstrate previous theories’ shortcomings in dealing with percep-
features of visual input, has an important role in tactile orientation tual illusions, specifically that of motion perception (sect. 6.1), and
analysis (Zangaladze et al. 1999): The disruption of occipital func- they propose a new perspective that utilizes specification in the
tion in human subjects interfered with tactile discrimination of global array. We believe that S&B’s ideas on the use of the global
grating orientation, but not with tactile discrimination of grating array in motion perception, specifically situations involving visual,
texture or the detection of electrical stimuli applied to the finger- kinesthetic, and vestibular stimulation, are useful in addressing
pad. skill acquisition in humans and nonhuman animals. We present
The flexibility of the nervous system in sensory processing has findings from our laboratory that support their theory that indi-
also been demonstrated by lesions to different brain areas and viduals work to detect the unique global array while mastering mo-
by various manipulations of the pathways that mediate sensory tor skills.
information. For example, if retinal projections are redirected to We study how capuchin monkeys (Cebus apella) acquire skill in
the auditory thalamus in neonatal ferrets, these projections not a task incorporating visual, kinesthetic, proprioceptive, and ves-
only activate parts of primary auditory cortex via thalamo-cortical tibular stimulation. Our capuchins (8 adult males) have mastered
connections but also form a retinotopic map (Roe et al. 1990; Sur using a joystick to move a cursor in a controlled manner on a com-

puter monitor. The system provides a unique learning situation in relevant patterns of the global array, and thus they are incorpo-
that while their movements are made in three dimensions, the rated into exploratory activity early in the learning process. In our
monkeys gain visual knowledge of results in two dimensions. To a view, and in accord with S&B’s theory, improved detection of the
capuchin, working in two-dimensional space is a novel experience. global array contributes to the identification and use of referents
In addition to providing artificial relations between visual feed- in performance of required actions.
back and manual action, this testing system enables us to study S&B note the paucity of research in the area of development of
mastery of a skill using different kinesthetic feedback than is avail- athletic and other motor skills in relation to the global array (sect.
able when action occurs directly upon the target object. The joy- 7). We believe that systematic investigations in humans and non-
stick and cursor are spatially separated and not physically con- humans, such as we have conducted with capuchins using non-
nected. Thus, unlike striking a ball with a bat, there is no direct verbal indices of perceptual processes, can help understand skill
kinesthetic feedback about the effect of moving the joystick (i.e., development. Similar work will be essential in evaluating S&B’s
no vibration of the joystick from contacting the cursor). This situ- theory of the specification of the global array. Comparative and
ation affords an interesting experimental context in which to study non-language-based testing of this theory will generate wider ac-
how perceivers learn about the two-dimensional/three-dimen- ceptance within perceptual psychology, along with better inter-
sional interface in “virtual” displays. facing with other fields (e.g., sports and developmental psychol-
S&B suggest that description of motion perception is meaning- ogy, behavioral ecology).
ful only if it is made in terms of referents (sect. 4.5). In the exam- Although this theory works well with tasks utilizing visual, kines-
ples presented, identification of these referents is elicited using thetic, and vestibular stimulation (e.g., joystick tasks), and also with
questions asked of the participants (sect. 5.2). However, percep- visual and auditory stimulation (e.g., McGurk effect, sect. 6.2.6),
tion in skill acquisition can be investigated in nonhuman animals we are curious to see how S&B and other investigators would ap-
as well. To evaluate perceptual processes in nonverbal individuals, ply the assertion that the senses work as a single system (sect. 7)
other ways of examining perception must be found. We have used to gustatory and olfactory stimulation. Expanding the theory to
visual tracking of the cursor and an unexpected phenomenon we incorporate all forms of stimulation would strengthen S&B’s ar-
call “body-tilting” to examine perceptual contributions to learning guments.
a perceptuomotor skill in capuchin monkeys. Both of these vari-
ables change in a patterned way with increasing mastery of the joy-
stick. We confine our comments here to the body-tilting phe-
nomenon. Situating situated multimodal perception:
We noticed that monkeys skilled at using the joystick demon-
strated a pronounced tilt of the torso, shoulders, and head in the The relevance of global arrays
direction of joystick movement and resultant displacement of to development
the cursor when they moved the cursor to the left or the right on
the computer monitor (Filion & Fragaszy 1997). To determine David J. Lewkowicza and Christian Scheierb
a
whether visual stimulation due to the cursor displacement or Institute for Basic Research, Staten Island, NY 10314; bCalifornia Institute
movement-associated stimulation related to manipulating the joy- of Technology, Pasadena, CA 91125. [email protected]
www.scholar.library.csi.cuny.edu/users/Lewkowicz
stick was the relevant referent in this task, we assigned four naive
[email protected]
capuchins to different learning situations. Two subjects learned to
use the joystick in the normal condition in which manipulation of
Abstract: Stoffregen & Bardy reject the likelihood that infants are sensi-
the joystick results in isomorphic movement of the cursor (e.g., a tive to the global array, implying that intersensory integration is not possi-
push to the left moves the cursor to the left). The other two ani- ble in early development. We argue that infants are sensitive to unimodal
mals experienced an inverted relationship in which movement of arrays and are able to integrate them through the active participation of
the joystick and cursor displacement was 1808 degrees opposed. their nervous system and that the observed developmental changes are
We found that animals in both the isomorphic and inverted due to experience and brain development.
groups tilted in the direction of the cursor displacement, whether
or not it was the same direction as joystick movement (Leighty & Stoffregen & Bardy (S&B) propose a bold new view that is a di-
Fragaszy 2000). Individuals in the inverted group were tilting in rect descendant of Gibson’s ecological psychology. Based on the
the direction opposite of their arm movement. After achieving underlying assumption that perception is thoroughly multimodal,
mastery in the inverted condition, these subjects experienced a re- they propose that perception and action are linked by a percep-
versal in the joystick/cursor relationship. They continued to tilt in tual system that is responsive to global, modality-independent ar-
the direction of cursor displacement under these new conditions. rays. The global arrays are detected directly and thus integration
This suggests that the relevant referent in mastering this skill is vi- of modality-specific information by internal (i.e., neuronal) pro-
sual (i.e., the direction of cursor displacement) and not the kines- cesses is unnecessary. In this way, they reject Gibson’s notion that
thetic properties of moving the joystick. perception and action are guided by distinct, single-modality ar-
Why would these animals tilt when using a joystick? As Stoffre- rays. Here we address some of the questions that the concept of
gen & Bardy note, a naive individual must identify and utilize dif- global array raises for those interested in the development of per-
ferent structures in the global array to acquire new perceptual- ception.
motor skills (sect. 7). It may be that tilting alters the structure of Based on their objection to the notion of separate senses, S&B
the global array in such a way that the animal is better able to de- explicitly reject the possibility that infants and novices are initially
tect the relationships embodied in this perceptually novel situa- sensitive to unimodal arrays and, thus, by implication endorse the
tion (sect. 6.2.5). It is interesting to highlight the instances of this idea that infants can perceive global arrays. S&B also reject the
form of behavior outside of the laboratory in humans. One can see possibility that experience might enable the pickup of global ar-
tilting being performed that seems to have no direct impact on the rays, and thus leave the developmentalist with the only obvious
outcome of the prior goal-directed action. For example, when option – the pick-up of global arrays either appears magically or
bowling or golfing, individuals often tilt in the direction of desired is there from birth and, therefore, is innate. The problem with ei-
outcome after releasing or striking the ball. One also sees tilting ther view is that it does not explain the processes underlying the
in children (and adults) playing video games while using a button development of perception. In addition, the overwhelming evi-
controller or joystick. It may be postulated that this form of be- dence is that perception does change over development. How,
havior is merely a superstitious act (according to operant defini- then, can we explain such changes, particularly with regard to in-
tions) that ultimately serves little function in performance. We be- tersensory functions? First, the different sensory systems have
lieve, however, that these behaviors increase one’s ability to detect their functional onset at different times during early development

and follow different developmental courses (Gottlieb 1971). As a tion that the perception of global arrays is innate. Even more trou-
result, the various sensory and perceptual abilities that would be bling for the ecological direct perception view is the finding that
essential for the detection of global arrays each emerge at differ- when the association cortex is inactivated the collicular cells cease
ent times. For example, basic perceptual competencies such as to perform intersensory integration. This, in turn, results in a fail-
spatial and temporal acuity, depth perception, object perception, ure to produce appropriate behavioral responses. In other words,
sound localization, texture perception, and shape perception, to perception is also in the head, not just “out there”!
name a few, each develop at different times and at different rates
(Keliman & Arterberry 1998). Moreover, motor abilities, which in
S&B’s view are critical to the detection of global arrays, lag con-
siderably behind. Thus, whereas by six months of age a number of Amodal specifying information:
basic sensory/perceptual abilities are already fairly well devel-
oped, crawling does not emerge until around eight months and
Where is occlusion?
walking not until even later. William M. Mace
Given that (1) self-generated perception-action coupling is es-
Department of Psychology, Trinity College, Hartford, CT O61O6.
sential to the perception of global arrays, (2) different perceptual [email protected] [email protected]
capacities develop asynchronously in the first months of life, and
(3) perceptual and motor development is dissociated, serious Abstract: James Gibson’s theory of information, as specific to (but not
doubts about pre-motor infants’ ability to perceive global arrays necessarily “like”) its sources, is especially indebted to the study of occlu-
can be raised. Therefore, what is needed to substantiate S&B’s hy- sion for its core examples. In occlusion, one is “amodally” aware of hidden
pothesis in human development is either a demonstration that ba- surfaces. Is this still too related to one modality to count as a good case for
bies can, in fact, perceive global arrays, or a suggestion as to how Stoffregen & Bardy?
they would develop this capability other than from integrating
modality-specific information. Moreover, if infants do not respond Stoffregen & Bardy (S&B) argue that researchers should examine
to global arrays (and thus must somehow learn to do so), the ques- information defined across energy fields but not defined within
tion is what exactly do they perceive? any one. I do not think that it is clear how to flesh out their sketch
Interesting to note, evidence shows that infants can perceive with relevant data and more detailed theory. To make their posi-
various types of intersensory relations (Lewkowicz & Lickliter tion clear, the authors will need to review a much larger body of
1994). If this evidence does not signify detection of global arrays data, some of which surely exists and much that will have to be
then what does it signify? Our view is that infants are, in fact, sen- generated. I take it that a major goal of their target article is to
sitive to unimodal arrays and that they can integrate them based stimulate work on both of these. It seems clear that their paradigm
on equivalent and redundant features through the active partici- case is the specification of ground slope that they mention in sec-
pation of the nervous system. The various developmental changes tion 6.1, drawn from Gibson 1966, pp. 62–63. I hope that S&B’s
observed in intersensory abilities are, in turn, likely to be due to response will recommend a variety of avenues of research but one
experience, as well as brain growth, development, and reorgani- set that I think would be especially helpful concerns occlusion.
zation (Lewkowicz 2000; von Melchner et al. 2000; Wallace & Occlusion emerges as a topic in the term “amodal.” The position
Stein 1997). S&B would likely reject the specifics of our view by that S&B attribute to James Gibson with the label “amodal” is a po-
stating, as they do, that it is not consistent with the ecological ap- sition Gibson did endorse in 1966, but he did not call it “amodal.”
proach; unfortunately, they offer no alternatives. For those inter- Moreover, he recognized the problems the authors warn about. “If
ested in development, however, the principal challenge is to de- stimulus information is equivalent across sense modalities, as I ar-
termine how a dynamic system that is constantly organizing and gue, then a new problem arises of the consistency and discrepancy
reorganizing (Gottlieb 1991; Thelen & Smith 1994) is capable of of information, either within a perceptual system or between per-
perceiving intersensory relations despite the fact that it may not ceptual systems (cross-modal, supra-intermodal, amodal, the ‘co-
be able to perceive global arrays. operation’ of the senses, ‘unity’ of the senses, etc. (Gibson 1966).”
Our view is that the nervous system (i.e., internal processes) Eleanor Gibson, as cited by S&B, used “amodal” in the authors’
plays an essential and critical role in the development of intersen- sense, but James Gibson did not. Taken by itself, this is a minor
sory perception. Indeed, one could argue that this is completely point, but considering it leads naturally to core concepts of eco-
consistent with ecological thinking. Specifically, if we extend the logical psychology, specification, and information. These words
evolutionary argument that Gibson used to motivate his ecological are used frequently in the target article, but without elaboration
psychology (and one that S&B would likely not find objectionable) (allowing S&B to stick closely to their main points). Because
that organisms evolved to respond directly to perceptual structure “specification” and “information” are common enough words in
then it is equally likely that nervous systems also evolved to do so. English, there is little in the target article that would mark them
Given that no behavior is possible without the brain, why ascribe as carefully chosen theoretical terms, yet what they indicate are
all behavioral control to external input and reject internal pro- precisely the qualities of the ecological position that make the au-
cesses? How can one speak of organism-environment mutuality, thors’ argument plausible. Therefore I think it is worthwhile un-
and not acknowledge the organism’s internal machinery and its at- derscoring their meaning for James Gibson.
tendant internal processes? Recent neurophysiological findings The usage of the word “amodal” that James Gibson cited was
show that, at least in the case of spatial multimodal cues, the ner- from Michotte et al. (1964). In what some have called the “rabbit
vous system does not automatically pick up the relations across hole effect,” which forms half of the well-known “tunnel effect,”
modalities but, instead, actively synthesizes them. Stein and his col- Michotte showed that changes of shape of a circle caused by pro-
leagues (Stein 1998) have shown that the deep layers of the mam- gressively removing segments, when shown as an animated film or
malian superior colliculus contain multimodal cells. These cells ex- its equivalent, did not look like something changing shape at all,
hibit a marked enhancement in activity when near-threshold but rather looked like a constant shaped circular form being hid-
auditory and visual stimuli are presented. In other words, there is den. Michotte referred to the awareness of the hidden parts as
a strong dissociation between the information in the global array, amodal because there was no visual sensation coming from them,
the corresponding neuronal activity, and behavioral responsive- what Gibson (1966, p. 205) liked to call a “sensationless percep-
ness. Moreover, these cells are not present at birth but appear later tion.” This usage of “amodal” is common in the work of Kanizsa
in development and when they do they still do not exhibit mature and Gerbino (1982) and the framing of the issue is similar to that
functional properties. While we do not know how these cells de- coming from the Grossberg group under the heading “filling in”
velop in human infants, the absence of multimodal responses in the (Pessoa et al. 1998).
developing colliculus of the cat casts serious doubt on the assump- Gibson argued that what was crucial here was the specifying

power of the optical change itself. He maintained that the optical tional procedures (Mark 1987). However, research can identify
changes caused by something’s going out of sight were different constraints on the nature of such computations so as to render im-
from the changes caused by something’s going out of existence. He plausible approaches based on cognitive computations.
reasoned that if going out of sight (with no change in existence) One strategy is to develop tasks in which the resulting behavior
and going out of existence are optically distinct transitions (for or perception appears to be grounded in the detection complex
starters, the first is reversible, the second is not), and an animal relationships among components of single arrays, so that the com-
can see that distinction, then there is a basis for perceiving the plex relationship is detected independently of one or more of its
continued existence of what is temporarily out of sight (Gibson et elementary components. Todd’s (1981) study of visual information
al. 1969). There is a real sense in which one can be said to be per- about moving objects showed that observers of a depiction of a
ceiving the hidden, and for Gibson that is based on the detection moving object were able to detect information about time to
of information (the specific type of change). arrival or where a free falling object would contact the ground.
Studying the details of something going out of sight is, of course, Todd’s analysis of the relationships available in these depictions in-
the all important (for Gibson) topic of occlusion. The optical con- dicated that observers were detecting a relationship among vari-
ditions for occlusion are specific to going out of sight as compared ables that included the object’s acceleration. Yet Todd demon-
to going out of existence. Both are more specific than referring to strated that observers were relatively poor in judging whether an
them as equal cases of “disappearance” of texture. Gibson counted object was accelerating or decelerating.
the optical changes specific to each as information, that is, the Apparently, Todd’s observers were detecting the complex rela-
changes would be informative about some state of the environment tionship independent of their perception of individual compo-
(its permanence or impermanence) relative to an observer/actor. nents. Whereas Todd’s analysis was restricted to the visual array,
What is important to emphasize here is that the language of speci- Mark’s (1987) examination of the perceived maximum surface
ficity was adopted as a contrast to terms that would presuppose height on which an observer could either climb on bipedally or sit
some kind of similarity or resemblance. It was not a paradox for on may well have involved relationships in the global array. When
Gibson to assert that something (occluded surfaces) could be per- observers’ capabilities were manipulated by having them wear 10-
ceived without being “present to the senses (Gibson 1986, p. 189).” cm high blocks on their feet, judgments of their action capabili-
He delighted in formulations like “the perception of persistence is ties initially overestimated their stair climbing capabilities and un-
not based on the persistence of the percept.” derestimated their maximum sitting capabilities. However, when
No doubt, S&B understand these points full well. However, be- given the opportunity to engage in exploratory movements (Mark
cause so much that is central to Gibson’s ideas about information et al. 1990), observers were able to discover their new capabilities
as specifying its sources is packed into the study of occlusion and in the absence of any opportunity to practice the relevant goal-
its kind of “amodality,” I would welcome their making an explicit directed action. Throughout the experiment, Mark (1987) also
connection between their ideas and occlusion. I would add that asked observers to estimate the height of the blocks on which they
occlusion need not be studied only visually. One can imagine that were standing. Participants consistently overestimated the actual
the sound heard suddenly by someone in the street from a door or height of the blocks by an amount greater than the error in their
window opening on a noisy party would not sound like the begin- judgments of their sitting or stair climbing capabilities.
ning of the party, but a “window” on to the sound of an ongoing This finding places a severe constraint on any computational
party. The sense of the party’s existence would be “amodal” in the procedure for determining a prospective actor’s sitting or stair
Michotte sense. How would that compare with turning a radio on climbing capabilities – a viable algorithm cannot involve per-
and off? In touch, one rarely touches all the connected surfaces at ceived block height in recalibrating the actor’s capabilities. Alter-
once, but can discover what is connected to what. Is occlusion, ei- nately, prospective actors may be detecting affordances (complex
ther from vision or this more generic sense, related to informative relationships between the environment and the prospective actor
cases in the Stoffregen & Bardy scheme? Is there multimodal oc- that have implications for action) directly. The analyses devised by
clusion? Or is this “amodality” beside the point? Ashby and Townsend (1986) for distinguishing perceptual inde-
pendence of individual perceptions go to the heart of this matter.
Amazeen (1999) has used this tool effectively to demonstrate that
haptically perceived heaviness is independent of haptically per-
Toward a strategy for demonstrating ceived size. The experimental goal, then, for implementing S&B’s
approach, is to devise experiments that show complex relation-
the perceptual independence of the global ships in the global array are perceived independently of proper-
array from individual sensory arrays ties of individual arrays.
Leonard S. Mark
Miami University, Oxford, OH 45056. [email protected]
www.miavs1.muohio.edu/~psycwis/mark.html
Functional separation of the senses is a
Abstract: This commentary discusses a strategy by which investigators can requirement of perception/action research
examine whether observers perceive properties of the global array inde-
pendently of properties in individual sensory arrays. Research showing Kipp McMichael and Geoffrey Bingham
that perception of complex relationships appears to be independent of the
Department of Psychology, Indiana University, Bloomington, IN 47408.
perception of individual components is considered. Ashby and Townsend’s
{kimcmich;gbingham}@indiana.edu www.indiana.edu/~psych
(1986) methods for identifying perceptual independence are important
tools for studying the global array.
Abstract: Stoffregen & Bardy’s arguments against separation of the senses
fail to consider the functional differences between the kinds of informa-
Stoffregen & Bardy (S&B) argue that prospective actors detect
tion potentially available in the structured energy arrays that correspond
properties of the global array independently of properties in indi- to the traditional senses. Since most perception/action research pursues a
vidual sensory arrays. One challenge is to develop research strate- strategy of information perturbation presupposing differential contribu-
gies and tools that could demonstrate whether prospective actors tions from the various ambient arrays, the global array hypothesis can only
are able to detect this information directly, that is, without having be extended and tested by analyses that consider the functional aspects
to compute global properties from the detection of independent along which the senses can, in fact, be separated.
properties contained in each source of energy. I am skeptical
about the possibility of proving that a computational approach will Stoffregen & Bardy’s (S&B’s) argument concerning the global ar-
not work because one could resort to a different set of computa- ray is intriguing hut potentially disturbing because it entails a re-

jection of the traditional separation of the senses. A bold claim in sensory-deficient conditions indicates that sufficient specification
its own right, this rejection ultimately calls into question the real- does exist in the optic array alone.
ity of separable ambient energy arrays and consequently denies S&B reject analysis of sources of information specific to the in-
that specification can exist within such separate arrays. We argue, dividual senses, but such analysis is required for any perturbation
however, that analysis of the informative character of the various study of perception/action. This is extremely significant because,
modalities is essential to understand the informative stability of as discussed by Bingham and Pagano (1998), most investigations
the hypothesized global array under conditions that perturb the study perception via perturbation methods wherein the informa-
functioning of the various senses. S&B present the global array as tion in particular sensory arrays is removed or attenuated. Bing-
a logically necessary conclusion from their rejection of separation ham and Pagano suggested that investigations should proceed by
of the senses. However, the global array hypothesis can stand in- observing the changes in performance when information is re-
dependent of this rejection and must do so if it is to stand at all. moved as compared to an unperturbed performance standard.
If we accept that specification is only possible within the global This requires an analysis of the information provided via particu-
array then perception/action must then rely exclusively on the lar senses to predict perturbation effects.
global array. The authors, however, did not describe many partic- The need to evaluate the effect of perturbation is important be-
ulars of specification in the global array. We note that there must cause the apparatus used to control and manipulate information
be multiple ways for the global array to be configured or for in- is itself perturbing aside from the intended experimental manip-
formation to get into the global array from the various modalities. ulation. Virtual reality (VR), for instance, has excited much inter-
This follows, for instance, from the successful visually guided lo- est because it allows optical information to be controlled via com-
comotion of John Hale (Cole 1995), who lacked proprioceptive in- puter graphics while retaining an active observer. But VR requires
formation below the neck or from the capable activity exhibited a user to look at a computer graphics display which entails per-
by blind and deaf individuals like Helen Keller. In these cases, a turbing visual accommodation and vergence. These perturbation
functionally adequate global array must be realizable via a subset effects must be evaluated if VR is to be used to study perception/
of the normally functional sensory organs. Clearly, however, not action. In our own VR research, we have referred to results from
just any subset of the sensory organs will do. Were we to combine previous studies in which either accommodation or vergence had
the sensory deficits of both Hale and Keller in the same unfortu- been isolated and manipulated to test their role in distance per-
nate perceiver, it is unclear how the olfactory, taste, and vestibu- ception. We used these results to predict the potential effect of the
lar systems could acquire information about distal layout or limb VR displays on performance in reaching tasks. We found that VR
position. produced the predicted effects (Bingham 2000a; Bingham et al.,
S&B have neglected to include an account of differential effects submitted). This research required analysis and study of the visual
of specific sensory deficits on the structure of the global array pre- system both independent of and as part of a system for the coor-
sumably because they reject the notion of separable senses. Nev- dination and control of reaching.
ertheless, testable predictions concerning the effects of specific Because we must ultimately probe the nature of perception/ac-
sensory attenuation or absence on the structure of the global ar- tion systems by means of perturbations like these, functional
ray would constitute an important empirical aspect of the author’s analysis of the senses is essential. Because Stoffregen & Bardy
hypothesis, particularly because temporary loss of certain modal- failed to address the role of such perturbations in the study of per-
ities is a regular aspect of our natural ecology. Ultimately, we must ception/action, it is difficult to access the theoretical or explana-
inquire how the global array evolves over time and changes when, tory potential of the global array and its requisite unity of the
for example, we turn off the lights before bed. Do we lose alto- senses. Discussion of methodology is essential to the enterprise.
gether the specification of evolving limb posture and body posi-
tion in the surrounding layout when one or more modalities are
attenuated? If specification persists, then how are we to under-
stand it without independent analysis of the contributions of the
modality (that is, the sense) that has been temporarily lost?
A different way to combine direct perception
S&B are critical of efforts to delineate separate sensory systems with intersensory interaction
using any anatomical or neurophysiological basis – but a lack of
differentiation between anatomically defined senses is not the Thomas Mergnera and Wolfgang Beckerb
aNeurologishche
same as a lack of differentiation between sensory systems. Gibson Klinik, Neurozentrum, Freiburg D-79106, Germany;
b
Sektion Neurophysiologie, Universitaet Ulm, Ulm D-89081, Germany.
(1966) focused on the role of perception in behavior such as main-
[email protected] [email protected]
taining orientation with respect to the surroundings. His func-
tional, rather than anatomical, approach to an analysis of the
Abstract: There is a discrepancy between Stoffregen & Bardy’s concept
senses did yield systems that spanned the traditional anatomically with experimental work on human self-motion perception. We suggest an
defined senses. Nevertheless, Gibson described vision and audi- alternative: (1) higher brain centers are informed by a given sensory cue
tion as among these perceptual systems and in his analysis, vision in a direct and rapid way (direct perception), and (2) this information is
could be used to achieve different goals in different tasks, for ex- then used to prime and shape a more complex mechanism that usually in-
ample to look at pictures or to guide locomotion. Thus, for Gib- volves several cues and processing steps (inferential).
son, the notion of separate senses was not simply an obsolete Aris-
totelian relic but a reality of the functional requirements of We welcome the Stoffregen & Bardy’s (S&B’s) attempt to broaden
perceptual behavior. the basis for ecological psychology or, more specifically, the direct
To understand how vision can be applied to such disparate tasks perception concept. As sensory physiologists dealing with infer-
requires analysis of the kinds of information available via the struc- ential explanations of human perception of self-motion, we are re-
tured arrays corresponding to the traditional senses. The optic ar- peatedly confronted with the limitations of our “bottom-up” ap-
ray cannot, by itself, provide information about absolute spatial proach and therefore take notice with great interest of the
scale because it consists of only angular and temporal patterns. “top-down” views offered by psychologists. We think that the is-
Nevertheless, as described by Gibson, optical structure alone does sue of directly perceiving a Gestalt-like “global stimulus array”
provide information about the ordinal structure of surrounding versus an intersensory interaction in terms of sensor fusion
layout and the observer’s spatio-temporal relation to it – enough (Mergner & Rosemeier 1998) may revive and deepen the inter-
information to allow the control of steering, for instance, in re- disciplinary discussion. We agree with the authors that perception
mote teleoperation. Obviously the human perception/action sys- mostly builds upon more than one sensory cue; however, based on
tem did not evolve to guide remote locomotion via a video display, experimental evidence concerning perceptual control of dynamic
but the ability of humans to succeed at this task under such behavior in space, we feel that multisensory processing mainly

takes place in the inferential elaboration of perception, whereas luminated funicular cabin, which is swaying in darkness). Like-
direct mechanisms (not necessarily multisensory ones) mostly wise, during vestibular-somatosensory interaction self-motion can
would have priming functions. In particular, we shall argue that: be viewed as being primarily based on somatosensory afferents
(1) an “inferential” concept of perception can largely be in line and being referenced to the body support, while the vestibular in-
with the ecological view, that (2) there may be a coexistens of a di- formation would be used to monitor and evaluate the kinematic
rect and an inferential information pick-up in one and the same state of the support (Mergner et al. 1997).
perception, and (3) that the former actually may prime or shape Direct and inferential perception may coexist. Investigations
the latter. into the conditions favoring the occurrence of CV lead us to be-
Concepts of ecological psychology as part of an inferential lieve that direct and inferential perceptions can coexist and actu-
theory. Referring to the example of the “global array” resolving ally may collaborate. Point of departure was the observation that
the ambiguity of an isolated consideration of the optokinetic sig- CV is facilitated in conditions with a brightly illuminated OKS, as
nal (sect. 6.1), we note that a simple neural network with one compared to an OKS that is so dimly illuminated that the visual
vestibular and one optokinetic neuron in the input layer, and one field shrinks to the point where the observer can no longer see his
“self” and one “pattern” neuron in the output layer could, in prin- orbital rim boundaries (Mergner et al. 2000a). These observations
ciple, determine whether the observer or the pattern was stopped. led us to investigate CV in conditions with an artificial orbital rim
Inasmuch as the activities of the two input neurons are viewed as that could be rotated independently of the eyes (which fixated a
a unique pattern propagating through the network, one may talk stationary or moving target) and of the OKS (stationary of mov-
of a global, direct perception. However, one can as well interpret ing). The stimuli were applied at very low frequency (0.05 Hz)
the network as an inferential processor which, using appropriate where, in case of a real self-rotation, vestibular afferents would no
input-to-output coupling weights, derives the correct answer from longer contribute much to its perception. The results of these ex-
two different afferents; yet, in spite of this inferential approach we periments led to a describing model which, among other facts, ex-
would still classify the answer as a holistic Gestalt perception. This plains why background motions across the retina do not elicit a CV
is to say that there is often no sharp division between the concepts if they are caused by eye movements. The model postulates two
of direct and inferential perception. independent internal notions of head motion relative to OKS. One
From our own work on the role of visual-vestibular interac- would be based on the visual afferents signaling the relative mo-
tion for human self-motion perception (Mergner et al. 2000b) – tion of the orbital rims (and hence of the head) with respect to the
experiments in which observers and their visual surround were OKS. The second would be a visual signal obtained by the sum-
rotated independently of each other using various combinations – mation of OKS retinal slip with an eye movement related effer-
we conclude that a rule-based ecological view is largely compati- ence copy (assuming a stationary head). Noticeably, each of these
ble with the inferential approach of biocybernetical models. In two cues alone is able to evoke CV; however, under normal cir-
fact, the 1atter reflect the evolutionary experience that the visual cumstances (i.e., in the presence of a structured background)
world as a whole is fixed in space and, hence, self-motion percep- where they tend to arise in combination, they neutralize each other.
tion is foremost dependent on the visual cue. The vestibular cue The purely visually derived signal can be viewed as providing a di-
would be, for one thing, a “technical embellishment” that takes rect perception, while the coordinate transformation giving rise to
over when the dynamical limits of the visual system are being ex- the second signal is an example of inferential processing. Thus, di-
ceeded (at frequencies >0.8 Hz). A different rule is invoked in sit- rect and inferential cues appear to coexist and to cooperate.
uations where large parts of our visual fields are covered by co- Could direct perception prime and shape inferential percep-
herently moving stimuli (a situation which usually occurs only tion? We hold that the direct perception concept refers mainly to
transiently). Self-motion perception then must rely on the vestibu- everyday situations in which individuals interact with their envi-
lar cue in order to avoid or to minimize visual self-motion illusions ronment and receive a wealth of mostly congruent sensory infor-
(vection, e.g., circular vection, CV. Contrary to S&B, we stick to mation. If one conceives of the brain as a learning neuronal net-
the notion of an “illusion” because, from an ecological point of work (ignoring its anatomical and developmental “prewiring”),
view, “true” motion is primarily a way to get from one point on the one would assume that it is overtrained with regard to everyday
earth to another). Finally, because low frequency horizontal self- situations and may have learned to handle these mostly on the ba-
rotations only give rise to optokinetic signals (the vestibular sys- sis of direct perception. Yet, also in these situations, inferential
tem in this plane has high-pass characteristics), a third rule spec- perception still may be required to adequately direct motor out-
ifies that low frequency optokinetic signals are to be interpreted put to the various joints of the body. Indeed, inferential models
as indicating self-motion (with the risk of creating an illusion). such as our model of vestibulo-somatosensory interaction (Merg-
These rules can be seen as features of the “global array” that de- ner et al. 1997) specify the kinematic state of each part of the body
termines the Gestalt of the resulting perception. (head, torso, hips, etc.). By directing an observer’s attention to
Interesting to note, with sinusoidal rotations of 0.4 Hz (at this these states, these notions become conscious perceptions and can
frequency both the visual and the vestibular cues provide reliable be probed experimentally. Using appropriate transformation laws,
motion information), perceived self-motion perception was ob- they can be processed to elaborate a behaviorally adequate motor
served to rather faithfully reflect the actual body rotation, inde- innervation. In contrast, a logical extension of the “global array”
pendent of the rotation of the optokinetic stimulus (OKS). This view to motor reactions would seem to require the combination of
observation apparently contradicts the first rule and would seem an already huge sensory parameter space with a similarly large
to indicate that subjects relied solely on the vestibular cue for their space encompassing all possible configurations of the observer’s
self-motion perception. However, the analysis of a model which own states, a notion that is difficult to accept, even in view of the
successfully simulated this and other observations (Mergner immense information storage capabilities of the brain. However,
et al. 2000b) suggests that during rotation in a stationary visual en- we nonetheless see a role here for direct perception (not neces-
vironment, self-motion perception is determined by a visual con- sarily for a global one, though): The operations required for in-
tribution; this contribution would originate from a “direct path- ferential processing and for the transformation into motor output
way” representing “head-to-visual reference” motion. The role of are likely to be quite complex because they have to take into
vestibular signal in this scheme is to define the kinematic state of account the rules of spherical kinematics (unlike in most labora-
the visual reference (after fusion with a processed version of the tory experiments, natural motions of the body and its parts are not
visual cue). According to this view one tends to perceive self- restricted to coplanar rotations). We speculate, therefore, that in
motion primarily with respect to a visual reference frame. This standard situations direct perception could provide higher brain
frame, in turn, is experienced as moving if its movements with re- centers with a first and rapid information on external events,
spect to the gravito-inertial reference frame exceed a vestibularty which would help these centers to rapidly select the correct cal-
defined threshold (example: movement experienced inside an il- culation path for the detailed inferential analysis of the given sit-

uation; in other words, a combination of direct and inferential pro- tal plane, we asked people to attempt to run and catch luminous
cesses might yield an optimal trade-off between speed and relia- balls in the pitch dark and to do so monocularly (Oudejans et al.
bility. However, in complex and rare situations (e.g., observer and 1999). We found that people were successful. Given that to-be-
visual background or body support moving simultaneously and in- caught balls are followed with pursuit eye movements (Oudejans
dependently of each other), direct perception is unlikely to pro- et al. 1999, Expt. 1), one would expect that the image of the ball
vide us with an adequate “first guess” of the ongoing external to be more or less stationary on the retina. The retina could not,
events so that, for the sake of reliability, the brain would have to therefore, be registering optical variables necessary to guide
rely foremost on its inferential capabilities. movement (e.g., Chapman’s, 1968, optical acceleration). We con-
There may be a relation between the hypothesized priming cluded that ultimately this unmistakably optical information was
function of direct perception and the role of cognition in percep- registered by non-retinal mechanisms. That is, the eye and head
tual processes. Cognition can, within wide limits, modulate and movements created by tracking transformed an optical pattern
predetermine the perceptual interpretation of a given set of sen- into eye or eye-and-head rotations, which would be picked up by
sory signals. For example, observers who expect a constant veloc- “vestibular and proprioceptive” mechanisms. Thus, while light is
ity rotation frequently will perceive such a rotation even if they necessarily detected by rods and cones, the information in light is
are passively turned in darkness although their only source of in- not necessarily detected by a retina. This counters S&B’s implicit
formation, the vestibular system, will indicate an exponentially assumption that once information is described that the nature of
decaying velocity. Likewise, the priming function of direct per- its detection is obvious.
ception may be one of setting the stage for an inferential inter- Point 2: What is specified? By entering into the debate of
pretation in agreement with the initial impression conveyed by the whether “physical reality” is or is not specified, S&B ignore the in-
direct perception. Conceivably this occurs by the intermediate of timacy of information and action. A discussion of “specificity”
a cognitive level where first the sensory Gestalt of the current sit- quickly gets spooky in the absence of a serious consideration of
uation is recognized. Stoffregen & Bardy do not address the role what is specified. We do not believe that S&B’s answer, physical
of cognition for perception, continuing a tradition of classical sen- reality, is sufficient. A key idea of the ecological approach is that
sory physiology which virtually ignored cognition. In our view, affordances can be specified by information and more generally
cognition is inextricably interwoven with perception, and instead that information can provide a basis for controlling movement.
of trying to eliminate its role by artificial experimental paradigms, Recently we have argued that variables of stimulation can be
it can be used to elucidate the mechanisms underlying inferential specific to action without their being specific to some environ-
processing (see Mergner et al. 2000b). mental fact or characteristic. This means that one should not al-
ways expect a 1:1 relation between an information source and a
property of the environment. We present two examples. First, in our
study of the timing of elbow extensions in the punching of falling
balls (Michaels et al., in press), we found that the onset and rate
Energy, information, detection, and action of elbow flexion were controlled by optical expansion – the rate of
Claire F. Michaels and Raoul R. D. Oudejans change of optical size of a ball as it approaches. Expansion rate,
however, does not specify properties of the approach of the ball;
Faculty of Human Movement Sciences, Vrije Universiteit, 1081 BT
Amsterdam, The Netherlands.
it varies with ball size, approach velocity, and distance. Neverthe-
{c_f_michaels; r_r_d_oudejans}@fbw.vu.nl less, it appears that, over the course of practice in the task, ex-
pansion rate comes to specify when and how fast to flex the elbow.
Abstracts: Before one can talk about global arrays and multimodal de- As a second example, we recently demonstrated (Oudejans et al.
tection, one must be clear about the concept of information: How is it dif- 2000) that errors in judging offside in soccer appear to be the re-
ferent from energy and how is it detected? And can it come to specify a sult of assistant referees’ using an optical variable that does not
needed movement? We consider these issues in our commentary. specify who is closer to the goal (attacker or defender). The off-
side judgment is based on the optical angle between defender and
Stoffregen & Bardy (S & B) present the concept of a global array attacker. This angle would specify who is closer to the defender’s
– invariants that span more than one individual energy type (and goal line only if the line judge were always on the offside line.
its associated sensory system). What distinguishes such invariants Oudejans et al. (2000) showed, however, that the line judge is fre-
from the usual variables studied by ecological psychologists is that quently off that line when judging offside. As a result, the line
they comprise variants at the level of the energy type or sensory judge sometimes puts the flag up even though the attacker is not
system; variants are, by definition, not specific to their sources. We offside, or keeps the flag down even though the attacker is offside
agree with S&B that such invariants exist and that it is a worth- (depending on the side at which the attacker passes the defender).
while enterprise to consider their possible role in perceiving and As in the ball-punching case, flagging appears to be lawfully re-
in acting. We disagree with many of the rest of their claims – par- lated to an optical variable that does not specify the environmen-
ticularly, the impossibility of specification in a single-energy array, tal facts. Instead, the arrow of specificity points toward the un-
their underestimation of Gibson’s (1966) claim that the senses be folding movement. One should look both ways, therefore, before
considered as perceptual systems, and their inattention to the declaring that arrays are non-specifying.
problem of what is specified, especially regarding action. In our The settings in which animals act have characteristics that are
commentary we make two points; one concerns the relations be- important for actions and may be useful in their control. There is
tween energy, information, and detection and the second claims a sea of structure and changing structure available that is poten-
that sometimes actions, rather than environmental facts, are spec- tial information. Some of these structures and transformations
ified by information. must “get into muscles,” to borrow Turvey’s phrase. For potential
Point 1: Energy is not information and information is not information to be actual, a variety of media must be transparent
picked up by receptors. Electromagnetic radiation is structured by to it; these can include energy distributions, chemical distribu-
refraction, diffraction, and reflection. The patterns so created can tions, skin deformations, and neural firing patterns, to name a few.
specify some of the properties of the media and surfaces. If these To make sweeping arguments about inherent limitations of single
are specific (ecologically, rather than mathematically or logically; types of energy, single organs, and which organs are detecting
see the commentary of Runeson et al. in this issue), living things what energy diverts attention from action and information and re-
might evolve, develop, or build devices that can pick them up. It vives the energy-based distinctions that Gibson had purged in
is not light, however, that is detected, but information. 1966, or so we thought.
In the course of trying to track down the optical variable(s) that
fielders use to guide their locomotion to catch balls hit in the sagit-

Fragmentation, coherence, and the dependence on the goals of action as the functional basis of the in-
perception/action divide teraction with the global array (e.g., end of sect. 4.5, also sect.
6.2.5), one sees how the theory also contains the ingredients to
Erik Myin cross the last gap created by fragmentation: the gap between per-
Department of Philosophy and Artificial Intelligence Laboratory , Free ception and action. A consequence of this is that the distinction
University of Brussels (VUB), B1050 Brussels, Belgium. [email protected] S&B make between perception and action can only be heuristic.
http://homepages.vub.ac.be/~emyin/ Just as they characterise their view as one in which perception is
not seen as the parallel action of a group of systems, but as the uni-
Abstract: I discuss Stoffregen & Bardy’s theory from the perspective of tary action of a system with parts, they, it seems to me, would have
the complementary aspect of input conflict, namely, input coherence – the to apply this to action and perception and see these as aspects of
unity of perception. In a classical approach this leads to the famous “bind-
ing problem.” The conceptual framework the authors construct leaves no
one global thing: the organism in its interaction with the environ-
space for a binding problem to arise. A remaining problem of perceptual ment (as I think they would certainly be willing to do).
conflict, arising in cases of inversion of the visual field can be handled by More interesting, bridging the perception/action gap in this
the theory the authors propose. way allows for a real form of fragmentation: erroneous or dis-
united perception when either the organism’s perceptual situation
Sensory conflict plays an important role in the target article. In this is problematic when viewing conditions are suboptimal or the per-
commentary, I will look at the paper with a complementary theme ceptual apparatus is damaged (cf. sect. 6.2.3) or when its action
in mind: sensory coherence, or the unity of perceptual conscious- possibilities are suboptimal. An interesting situation occurs when
ness. both are suboptimal, as in the well known case of people wearing
According to the classical view, perceptual input information is goggles that invert the retinal image.
not only ambiguous, therefore requiring inference, but it is also It has been well documented that once behavioral adaptation to
fragmented. Fragmentation applies at a variety of levels. First, the the goggles occurs, and once people learn to act in the “normal”
input information is shattered to pieces as it is spread out over thou- way again, their perception returns to normal. However, percep-
sands or even millions of receptor cells. At a higher level, fragmen- tual adaptation does not occur in an “all at once,” but in a frag-
tation occurs because of the separateness of the senses: separate mented way. Perceptual adaptation seems to depend on which be-
modules in the various modalities deal with the shattered pieces of havioral capacities have adapted, and they do not all adapt at the
input to build modality-specific higher-level representations. These same pace (for discussion, see Hurley 1998, p. 347–48, and O’Re-
higher level representations are supposed at least partially to solve gan & Noë, in press). What this suggests is that just like percep-
both the ambiguity problem and the fragmentation problem. Am- tual unity, perceptual fragmentation is not an internal affair.
biguity has been solved because knowledge not present (such as Rather it is a matter of behavioral “disunity.” At last, we have a real
Marr’s rigidity constraint, 1982, pp. 209–10) in the stimulus itself case of fragmentation, but one that perfectly fits within the frame-
has been brought to bear upon it. Fragmentation is partially undone work Stoffregen & Bardy sketch.
because from the receptor mosaic emerge sense-specific represen-
tations, perhaps in a “canonical” format. Yet fragmentation remains ACKNOWLEDGMENT
a problem. Now how do all these outputs from these separate mod- Thanks to the Flemish Community and the Free University of Brussels
ules get glued together? The situation is aggravated by the fact that ((VUB), project GOA 2) for financial support.
it is widely assumed that there is intermediate level fragmentation
within the modules: within the visual module there are submodules
for color, form, movement, and so on (Zeki 1993). So the traditional
view creates for itself the famous “binding problem”: How do all Act globally, think locally
these separate representations get together so as to lead to the uni-
Michael F. Neelon and Rick L. Jenison
fied perceptual consciousness we normally enjoy? The favourite so-
Department of Psychology, University of Wisconsin, Madison, WI 53706.
lution within the tradition is well-known: it is neural synchronisa-
[email protected] [email protected]
tion that is doing the job (Crick & Koch 1990). www.wavelet.psych.wisc.edu/jenison. html
Whatever the merits of the synchronisation idea, it is still un-
able to deal with still higher levels of fragmentation. What, for ex- Abstract: The authors attempt to prove that single energy arrays cannot
ample, binds lower level sensory representations (as outputs from specify reality. We offer contrary evidence that motion structures the
perceptual modules) to their conceptual representations? Even acoustic array to specify fundamental attributes of the source. Against di-
after the color of the bucket, its shape and its cold feel have been rect detection in general, we cite evidence that humans weight acoustic in-
bound, it still needs to be bound to how I conceive it: as the bucket puts differentially when making perceptual judgments of auditory motion.
I used to scare the attacking dog on aunt Margaret’s farm, for ex-
ample. And there is yet a further level of fragmentation; how does Stoffregen & Bardy (S&B) attempt to prove the inadequacy of per-
the passively received input representation, even when fully ceiving reality via single energy arrays through the following syllo-
bound with each other and with conceptual representations, be- gism: perception in general is largely the perception of motion; mo-
comes something I can act upon? How is the perception-action dition is relative and often indicated divergently among energy
vide crossed? Does it need to be bound with plans, or with goal- arrays; hence, reliable perception of the environment is not possi-
representations? Clearly, we’re on the verge of a combinatorial ble via single arrays. We agree that motion is vital to successful per-
explosion, or worse, conceptual impossibility (cf. Shanon 1993). ception. But in addition, dynamics can impose sufficient con-
One of the many nice aspects of this paper is that it shows constraints that allow even single sensory arrays to specify other distal
clusively that these problems of fragmentation and binding, as the properties relevant to an animal’s behavior beyond motion per se.
side coin of problems of input conflict, arise not only within the For example, analytical demonstrations exist showing that audi-
traditional view, but also within all forms of Gibsonian theories tory motion can structure the dynamic acoustic array to specify fun-
that assume separate senses. With hindsight, one sees that it could damental attributes of the sound source, such as its position, ve-
not be otherwise: fragmentation at separate senses (accepted by locity, and time-to-contact (TTC) (Jenison 1997). These higher
many Gibsonians) is just a higher-level version of fragmentation at order variables are inversely determined from the forward equa-
sensory receptors (criticised by all Gibsonians). It is only with tions describing the physical mapping of intensity, frequency, and
S&B’s proposal of unifying the senses ab initio, that the concep- interaural time delay (ITD) from source to observer. Such inverse
tual space closes so as not even to leave room for the problem of solutions are evidence that single energy arrays can specify reality
perceptual unity to emerge. provided that sufficient input dimensions within the modality exist.
From the various remarks in the target article concerning the We believe there is a more general inquiry into S&B’s hypoth-

esis concerning the adequacy of individual arrays for percep- They discovered that as task and source parameters changed, so
tion. Namely, how many sources of sensory input does it take to did the weighting schemes.
specify a distal property? In our view, this is a simple question of To conclude, we agree in principle with S&B’s arguments that
algebra. Specification of a higher order property implies the multiple, dynamic inputs should provide better specification of
existence of an inverse solution from the proximal inputs to the the external world than single or static inputs. However, they of-
distal variable. If there are not enough “knowns” (input) to “un- fer no evidence that global energy arrays are necessary for identi-
knowns” (distal variable), then the property may not be identifi- fying every environmental property; on the contrary, we cite analy-
able. Hence, increasing the number of inputs should generally ses which show the sufficiency of the dynamic acoustic array to
increase specification. This is the root of the S&B’s argument specify several attributes of moving sound sources. The degree to
for the necessity of multimodal (“global”) information. Again, we which the world is inversely determined by sensory inputs (i.e.,
would generalize this position by saying what is needed to specify perception) is a product of the lawful relationship of energy prop-
the world is simply enough proper input information, indepen- agation between them and the statistical reliability of the neural
dent of whether that information is conveyed within (“multiple representations of that input.
specification”) or across modalities (“amodal specification”). By
“proper” we mean that there is a lawful physical relationship be-
tween the distal property and the input.
While this approach explicates the physical journey from envi-
ronment to receptor, we also believe in a corresponding journey Input-driven behavior: One extreme of the
from receptor to percept. The achievement of perception is multisensory perceptual continuum
“harder” than just saying a variable has or has not been detected,
as most ecological theorists seem satisfied to claim (Marr 1982). Kelvin S. Oie and John J. Jeka
Typical in such arguments, there is no description by the authors Program in Neuroscience and Cognitive Science and the Department of
of how higher order information is in fact directly detected by the Kinesiology, University of Maryland, College Park, MD 20742.
animal (e.g., are there transducers for global variables [Fodor & [email protected] [email protected]
www.inform.umd.edu/EdRes/Colleges/HLHP/KNES/faculty/jeka
Pylyshyn 1981])? Rather than appealing to direct detection, we
www.glue.umd.edu/~kso3713/Welcome.html
formally question: (1) how are input sources or physical dimen-
sions combined when detecting properties specified by multiple
Abstract: The propositions that the senses are separate and that the
inputs? and (2) how may perception be achieved through a form global array may be sufficient for adequate perception are questioned.
of statistical estimation, given the stochastic nature of neural There is evidence that certain tasks may be primarily “input-driven,” but
transduction and transmission? these are a special case along the behavioral continuum. Many tasks in-
To investigate these questions, we have simulated the estima- volve sensory information that is ambiguous, and other sources of infor-
tion of higher order auditory variables (e.g., auditory TTC) using mation may be required for adequate perception.
two models of input fusion. The first employs a Kalman filter
which can successfully estimate higher order terms using as input Despite the long scientific tradition of separating perception into
the noise-corrupted observed intensity, frequency and ITD of a separate sensory systems, it has long been recognized that differ-
simulated moving sound source ( Jenison 1996). In theory, one ent sensory systems interact in ways that remain poorly under-
could extend the Kalman filter to integrate information from dif- stood. Our present understanding of multisensory interactions sits
ferent modalities, obviating any debate over a qualitative versus at the level of phenomena such as the McGurk effect, rather than
quantitative difference between uni- and multimodal input. What with organizational principles of how senses interact.
is required are representations of the forward equations mapping Stoffregen & Bardy (S&B) address the problem of multisensory
distal states to physical input dimensions across modalities. These interaction from the perspective that separate senses do not exist.
equations allow the filter to dynamically adapt the fusion of input Unfortunately, their arguments against separate senses are not
information in order to improve its estimate of source states. compelling. S&B reject an anatomical basis for separate senses due
It may be the case that the Kalman algorithm does not strictly to interactions of structures only within a sense (two eyes/two ears
represent neurophysiological mechanisms. However, it has been that work together), not between. There is little doubt that anatom-
proven equivalent to a form of dynamic Bayesian estimation ical substructures have developed to be sensitive to particular
(Meinhold & Singpurwalla 1983), a process which we believe bet- forms of energy. With eyes closed, how well could one determine
ter describes how neural systems may extract information from the intensity of a light source directed at the pinnae? Moreover, the
sensory input. As a model embodying fewer assumptions, we have fact that animals with different receptors interpret the same form
also trained a recurrent neural network to estimate TTC from the of energy in different ways does not argue against separate senses,
same acoustic dimensions (Neelon & Jenison 2000). Performance but against the unique meaning of a stimulus to a perceiver.
of both models degrades as a function of reducing either the num- More problematic is the question of whether structured energy
ber of inputs or their signal-to-noise ratio (e.g., increase sensor fields provide “sufficient/insufficient” information for accurate
noise or virtual distance of the source). This again illustrates that perception. The problem may be in the proposition itself. The
perceptual estimation is likely to be a continuous (though non- ecological view suggests that information for behavior is specified
linear) function of the quantity and fidelity of lower order inputs. uniquely in the ambient array, individual or global, and any non-
Ecological theorists may remain unconvinced that real ob- 1:1 mapping negates specification. But, why is specification an ei-
servers combine sensory inputs to perceive higher order informa- ther/or concept? Why not view the specification of behaviorally
tion. There is evidence, however, that human subjects differ- relevant information along a continuum? Under certain task con-
entially weight acoustic inputs when making judgments about ditions, the stimulus array dominates the response, which one
moving sources. First, we have tested listeners in tasks requiring might call “input-driven” behavior or perception. Time-to-contact
them to discriminate between the intrinsic frequency and TTC of (t) (Lee 1981) is a classical example of such input-driven percep-
two moving sound sources (Neelon & Jenison 1997; 2000). Sub- tion. Change the parameters of the task conditions, however, and
jects show a pattern of results similar to that of model simulations the same sensory information may now be ambiguous, requiring
as sensory input is degraded. Further evidence is provided by other processes and sources of information to be recruited for ad-
Lutfi & Wang (1999), who indirectly measured the weights human equate perception (e.g., memory). This view is more in line with
observers place on the acoustic inputs of level, frequency and ITD current thinking about cognition, which stresses the dynamic na-
when discriminating between moving sources. They correlated ture of processing inputs from multiple sources (cf. Beer 2000).
listener response with experimentally controlled input variation to In the search for specification, S&B appeal to the concept of the
estimate how much influence each input had on performance. global array; essentially a higher-dimensional version of the ambi-

ent sensory array originally introduced by Gibson (1979). Their about the nonexistence of separate senses. Their article clarifies
formulation is vague and the arbitrary three-dimensional plots in- the functional role of sensory information. Today, the separate-
tended as abstract representations of the global array are not par- senses view still remains in researchers’ heads especially because
ticularly informative. Examples of higher-dimensional variables of the computer analogy and semantic segregation of object prop-
are provided, however, that lend credence to the existence of the erties characteristic in humans. Many studies dealing with the
global array. S&B misstep, as discussed above, when the global ar- dominance of one sensory channel during a sensory conflict can
ray is viewed as eliminating the need for internal processing. still be found in the literature. But the results of these studies are
There may be situations in which the global array resolves ambi- highly contradictory. Indeed, the dominance of one sensory chan-
guities provided by single-sensory information. It does not follow nel depends on age, learning (Misceo et al. 1999), task or form of
that internal processes are no longer necessary. Again, the solution cognitive type. It would be interesting to reinterpret these differ-
may be task specific, with “input-driven” responses representing ences in sensitivity to one form of sensory information amodally.
one end of a multidimensional continuum. This view is consistent Our contribution, but also our objection, deals with the place of
with the engineering literature, which considers multisensory in- the brain in the specification of the animal-environment (A-E) re-
tegration as a subproblem of “data fusion,” in which any number lation. According to S&B the A-E relation is specified by infor-
of inputs, not just sensory, are combined to form a percept of an mation picked-up in the global array, but “before the stimulation
environment object or event (Hall & Llinas 1997). of sensory receptors.” The global array concept linked to the phys-
There is no argument here that structured energy arrays exist ical world and to its physical energies can be transposed to the
and that they may specify behaviorally relevant information under brain as both a physical object and a dynamical system. The phys-
certain task conditions, in the ecological sense. The question is ically different external energies (air vibration for audition, light
whether such behavior represents all perceptual processing or for vision) activate different sensory cortices with energy ex-
merely one extreme of the perceptual continuum. Responses to changes of a similar nature (chemoelectrical current is the com-
other task conditions may “weight” internal processes such as mon energy propagated along the axon and though the synapses).
memory/experience/instruction more heavily to form an ade- Moreover, neurons are structurally interconnected and work in
quate percept. Recent studies support the existence of internal parallel. By distinguishing a physical exterior and a biological in-
models in sensorimotor integration (Kawato 1999) and multisen- terior, the brain is excluded from the physical world. However, ex-
sory integration (Merfeld et al. 1999). There is growing support periments have shown that during ontogeny, brain structure is
that the extraction of information is a process of estimation, which specified through the A-E interaction, that is, by epigenetic fac-
may be based upon the statistical properties of multiple sources of tors. The studies of Von Melchner et al. (2000) and Sharma et al.
input over time (e.g., Oie et al. 2000; van der Kooij 1999). (2000) in ferrets and the studies of Kujala et al. (2000) in humans
A more precise formulation of the global array than that offered asked the following question: Are neural connections genetically
by the target article raises interesting questions of how to distin- pre-cabled? In this case, each cortical sensory area would be acti-
guish processes of direct perception from processes considered vated only by one specific physical energy. Or, in the opposite way,
more internal. For example, autonomous robots that process sen- is the cerebral architecture specified by the activation of the sen-
sory information from multiple sources for obstacle avoidance dy- sory pathways? The studies in ferrets have demonstrated that
namically display what is deemed cognitive behavior in the form when a modality-specific brain area such as the primary auditory
of decision-making (Schöner et al. 1995). The general scheme in- cortex is totally deprived of its normal sensory input (the stimu-
volves fusing two sensory sources that are spatially nearby to erect lating effect of air vibration is suppressed), it becomes responsive
a repellor, which the robot travels around, or to erect two separate to stimulation from other modalities such as the visual input. Thus,
repellors when sensory sources are spatially distant, which the ro- the rewired auditory cortex has the same characteristics as the vi-
bot may travel between. The decision to go around or between two sual cortex. Sharma et al. (2000) showed that in ferrets in which
obstacles arises from the fusion of sensory sources that vary para- retinal projections are routed into auditory pathways, visually re-
metrically. That this behavioral solution may be construed as a sponsive neurons in the rewired primary auditory cortex are also
classically internal process, decision-making, without any obvious organized into pinwheels, as observed in V1. In humans, some
internal representation indicates that dynamic aspects of multi- studies have shown that in the blind the occipital cortex are acti-
sensory processing may redefine what is considered internal. vated by sound changes (see Kujala et al. 2000, for a review). The
Our view is that Stoffregen & Bardy’s stance may contribute to neurophysiology of plasticity (i.e., the structural and functional
understanding the problem of multisensory perception if it is con- modifications of the nervous system by the properties of its
ceived as a special solution at one extreme of the perceptual con- activation) informs us about the amodal functioning of the brain.
tinuum. The dichotomous nature of the direct perception/inter- The specification of an auditory area by visual input (or the re-
nal processing discussion may follow the evolution of the nature/ verse) shows that a given part of the brain is not genetically pre-
nurture debate. It is not a question of one or the other, but of how destined to process a given type of information.
each are blended to solve the task at hand. Just as a change in one environmental property modifies the
global pattern of external energy, a change in one environmental
property modifies the neural connections between different sen-
sory cortices (primary or associative). There are contamination
phenomena in all the sensory cortices when one input is modified
Is the brain specified? (Pailhous et al. 1990; Schöner et al. 1998). Our opinion is that this
unique combination of neural activation, specified univocally by
Jean Pailhous, Elodie Varraine, and Mireille Bonnard
the properties of physical reality, modifies amodal perception. So,
Mouvement et Perception, Université de la Méditerranée, CNRS, Faculté des
the processing of the global array is not localized in associative
Sciences du Sport, 13288 Marseille, Cedex 9, France.
{pailhous; varreine; bonnard}@laps.univ-mrs.fr www.laps.univ-mrs.fr
areas as thought by the authors, but rather achieved through a
unique spatio-tcmporal activation pattern of the brain: perception
Abstract: How to conceive the place of the brain in the specification of
is more univocal than direct. The mysterious concept of direct
the animal environment relation? Reality is a continuum between exter- perception is involved in this context. Speaking about a neuro-
nal physical energies and brain energy. The global array concept linked to physiology of direct perception makes sense only if the univocal
the physical world and its physical energies could be transposed to the characteristic of network activation is taken into account. Then,
brain as a physical object and a dynamical system. we are in agreement with the authors that the A-E relation is glob-
ally specified by the physical world. But our opinion is that the
We are largely in agreement with Stoffregen & Bardy (S&B), par- physical world includes the brain and more generally the whole
ticularly with the epistemological and theoretical point of view organism (in another commentary, we would say that we cannot

understand the functioning of the nervous system isolated from its world, that most environmental stimuli have a multimodal com-
more immediate environment – the rest of the body – we cannot ponent, and that animals have neurons that are sensitive to multi-
do neuroscience without doing physiology!). The perception of re- modal stimuli, then this is not only an obvious point but has a great
ality is not mediated by the brain because the brain is included in deal of experimental support, and does not require proposing at
reality. It is clear that “direct perception” does not signify “with- this point. Gibson and others, including Charles Darwin, sug-
out working of the nervous system.” Reality is a continuum be- gested long ago that sensory systems are shaped by evolution to
tween external physical energics and physical (because chemo- decipher those signals that are of greatest importance to the or-
electrical) brain energy. By excluding the brain and its energy from ganism (Gibson 1979), and such signals often contain energy
the physical world in order not to separate the senses, inanimate across different modalities.
matter is separated from the living matter. If biology is separated Or perhaps by “directly sensitive” they mean that there are
from physics, there is no longer continuity between brain and en- modules or arrays that are constructed developmentally and/or
vironment, just coupling. Though we are totally in agreement with evolutionarily that are inherently capable of interpreting aspects
S&B on the non-separation of senses and on the concept of global of the global array relevant to survival. The authors are right in that
array, their demonstration would nevertheless, be easier if it inte- it is probably the case that such modules have not been properly
grated the brain. Indeed, there are neither channels nor boxes in searched for in many cases, given that most physiologists restrict
the brain, only networks, massively interconnected and working in themselves to one sense. But this notion of “combination-sensitive”
parallel. Even when the stimulation is unimodal, its treatment is neurons is a very old one (e.g., Lettvin et al. 1959) and has been
always global, as rightly underscored by the authors. applied to multimodal inputs by numerous investigators (e.g.,
With regard to this continuity between the physical world and Stein & Wallace 1996).
the brain (which does not exclude its singularities), how is it pos- Work by sensory physiologists over the past few decades has
sible to have a discontinuity between environment and behavior made it clear that the brain contains a series of tuned input chan-
at a macroscopic level? We will not be surprised that this discon- nels (sensory organs) that are sensitive to a particular range or type
tinuity is marked by the laws of adaptation: “the behavior of an an- of energy with minimal overlap, and that these classes are segre-
imal is adapted to its environment.” And why is the behavior gated by modality in the early stages of processing. Indeed, this
adapted ? Because the species which are not adapted to the envi- parallel organization is essential for optimum processing efficiency,
ronment do not survive (Darwin 1958). The perception-action as different neurons and pathways are specialized for processing
coupling is the result of an adaptive change of our central nervous certain aspects of a stimulus. Examples of this are the channels
system to environmental properties. In separating the nervous sys- coding form and motion in the mammalian visual pathway (Liv-
tem and its adaptive properties out of the physical world, the per- ingstone & Hubel 1988), the time and intensity channels of the
ception-action coupling is masked. That is the reason the neuro- auditory pathway (Feldman & Knudsen 1997), or the amplitude
physiology of plasticity has to have its place in the ecological and phase channels in the electrosensory pathway (Kawasaki &
approach. Guo 1998). In these cases, the neurons and synapses in each path-
way are designed optimally for the task at hand. Another crucial
advantage of parallel organization is that the separate lines can be
brought together in multiple different ways; if they were joined at
the outset this combination and recombination process would not
Cortical specification makes sense be possible. In that sense, arguing that perception is not organized
via different channels, while perhaps heuristically useful, seems
Sarah L. Pallas
out of tune (pun intended) with the evidence. Researchers have
Department of Biology, Georgia State University, Atlanta, GA 30303.
not “assumed that the patterns in the global array are not sensed
[email protected] http://www.gsu.edu/~bioslp
directly,” rather there is excellent evidence that sensory informa-
tion is decomposed at the receptor surface and then recombined
Abstract: Overwhelming evidence points to the existence of separate sen-
sory channels in the nervous system. The power of this type of parallel or-
in an internal, multimodal representation of the environment,
ganization is that information is first processed in neurons specialized to with attention to the position of the eyes, head and body through
code it most efficiently. However, sensory pathways are convergent and di- reafference or efference copy.
vergent at each level as well, as is necessary to interpret multimodal and How does an animal define one sensation as auditory and an-
conflicting information. other as visual, rather than as combined auditory/visual stimuli?
The evidence suggests that the assignment is historical; visual cor-
The prospect of commenting on a review that extends from the tex becomes visual during its development in large part because it
theory of relativity to cognitive psychology is somewhat daunting is usually hooked up to the photoreceptors. If one hooks up visual
to a non-psychologist, non-physicist, and thus I will attempt to cortex to auditory receptors, or vice versa, the perceptual assign-
provide a perspective from a sensory neurophysiologist’s and neu- ment of the tissue is transformed (Gao & Pallas 1999; Heil et al.
roanatomist’s point of view, filtered through our work on brain de- 1991; Pallas, in press, for review; Pallas et al. 1999; von Melchner
velopment and evolution. From that perspective, it seems Stoff- et a1. 2000). Anima1s with early visual inputs to the auditory path-
regen & Bardy (S&B) do not rigorously incorporate existing way, when asked to categorize a wide variety of visual and auditory
knowledge about the anatomy and physiology of both unimodal stimuli, will define visual stimulation of the auditory pathway as
and multimodal sensory pathways into their theory. However, visual (von Melchner et al. 2000) In other words, auditory cortex
their target article is certainly thought-provoking, and many of can be taught to process and perceive visual stimuli through ex-
their ideas and examples are of considerable value in clarifying perience.
possible future approaches to the questions they delineate. The biggest drawback of S&B’s theory is that the authors do not
S&B’s conceptualization of an animal being “directly sensitive” report or propose any concrete experiments to test their theory or
to the “global array” could be interpreted in several ways. It is dif- to refute the well-established current view. Their argument, that
ficult to see how a parameter such as the hand velocity necessary multinodal patterns of information are sensed directly and that
to intercept a moving object could be “detected directly” since senses function as a single unit, is of limited value if not backed up
multiple information sources are necessary to calculate it. If their by experimental evidence. Hopefully it will at least serve as an im-
argument is that information does not access the brain through petus to greater care in experimental design by the various disci-
unimodal channels to be later combined in multimodal areas, the plines engaged in the study of perception.
evidence does not support them.
If, on the other hand, their argument is that multimodal areas
are of critical importance to an animal’s ability to understand its

Direct perception of global invariants stand within S&B’s framework, whereas this phenomenon is read-
is not a fruitful notion ily explained from the perspective of multiple modal sources of in-
formation.
C. (Lieke) E. Peper and Peter J. Beek Integration of sensory signals. If perception is based on mul-
Institute for Fundamental and Clinical Human Movement Sciences, Faculty tiple modal sources of information, the process of integration may
of Human Movement Sciences, Vrije Universiteit, 1081 BT Amsterdam, The be viewed as the result of computations and weightings performed
Netherlands. {c_e_peper; p_j_beek} @fbw.vu.nl by some intelligent homunculus. However, recent findings re-
www.marlow.fbw.vu.nl garding MST (medial superior temporal) cells in monkeys may il-
lustrate a less spooky mechanism of integration of relevant “infor-
Abstract: The epistemological premises and scientific viability of Stoffre- mation.” Many of these neurons respond to both optic flow and
gen & Bardy’s ecological perspective are evaluated by analyzing the con-
cept of direct perception of global invariants vis-à-vis (1) behavioral evi-
vestibular information (with some cells showing enhanced activity
dence that perception is based on the integration of modal sources of when the two are congruent and others when they are incongru-
information and (2) neurophysiological aspects of the integration of sen- ent), while other neurons are sensitive to only one of the two
sory signals. modalities (Bremmer et al. 1999; Duffy 1998). Thus, MST cells
play a role in the detection of relative motion and self-motion.
Stoffregen & Bardy (S&B) aim at internal consistency within the Whether such sensitivity to modality-transcending information is
theoretical framework of ecological psychology. Starting from the to be interpreted as integration over different modalities or as di-
epistemological premises of (unique) specification and direct per- rect perception is a theoretical rather than empirical question.
ception, it is concluded that the senses cannot be separated. This Conclusion. S&B attempted to carry the epistemological
radical conclusion calls for an evaluation of the premises and via- premises of ecological psychology (specification and direct per-
bility of S&B’s ecological perspective. In doing so, we concentrate ception) to their logical extremes. However, many observations
on considerations and behavioral findings regarding illusions, in- and considerations cast doubts on the fruitfulness of the proposed
formational conflicts, flexibility, and learning, as well as neuro- direct pick up of global invariants. Although one option is to aban-
physiological aspects of the integration of sensory signals. don the concept of specification altogether (e.g., nonspecifying
Illusions and informational conflicts. It follows from S&B’s quantities that correlate highly with relevant properties may be
analysis that an event is only fully specified in the global array. used instead, cf. Michaels & De Vries 1998), an epistemology
However, this is not to say that modal arrays do not specify any- based on modality-specific specification may still provide an ade-
thing. Optic flow, for instance, specifies relative motion with re- quate framework for examining how animals know (their relation
gard to the visual environment. Experiments involving informa- to) the environment. Because the ecological approach rests heav-
tional incongruencies, resulting in illusory or real self-motion, ily on its epistemological premises, the question whether such a
indicate that such modal sources of information are used and that view qualifies as “ecological” depends on whether integration of
their relative importance may differ. This is underscored by modal sources of information is considered as “direct perception.”
graded degrees of illusion that scale with presentational aspects of Whatever one’s predilection in this regard, the road proposed by
the manipulated information. For instance, the visually-induced S&B does not seem to be the one to follow.
illusion of self-tilt increases with the field of view (Allison et al.
1999). It is unclear how such a graded effect can be understood in ACKNOWLEDGMENTS
terms of the direct perception of a global invariant. Moreover, the We thank Bart Krekelberg for his input regarding MST cells. LP is sup-
correspondence between postural sway patterns and modality- ported by The Netherlands Royal Academy of Sciences.
specific information (generated by, e.g., an oscillating visual scene
[Dijkstra et al. 1994] or touch bar [ Jeka et al. 1997]) seems more
readily understandable on the basis of modality-specific informa-
tion sources rather than nondecomposable global invariants. How important is specificity?
Only if perception is based on multiple sources of information
John Pickering
can the occurrence of informational conflicts be explained. A con-
Psychology Department, Warwick University, Coventry, CV4 7AL, United King-
vincing demonstration of such a conflict involves the observation
dom [email protected] www.warwick.ac.uk/staff/J.Pickering
that a deafferented patient could easily draw a five-point star while
watching her hand in a mirror, whereas this was rather difficult for
Abstract: There is good neuropsychological evidence for an amodal, re-
normal control subjects. As the relation between movement di- lational basis for perception and action. Using this idea, it may be possible
rection and visual scene (mirror image) was manipulated in both to define more accurately what is meant by specificity, in the Gibsonian
groups, the most logical explanation is that the control subjects ex- sense of the term. However, for complex organisms, and most especially
perienced a conflict between visual and proprioceptive informa- for creative, open-ended perceivers and actors such as human beings,
tion, whereas the patient did not because propriocepsis was un- specification may not be relevant.
available (Lajoie et al. 1992).
Flexibility and learning. The qualitative differences between Approaches to perception and action might be said to have direct
global arrays of different dimensions imply that global invariants and indirect poles. The direct approach claims that the informa-
used in an array with dimensions X and Y are of limited use in an tion available to active perceivers, or at least some portion of it,
XYZ-array. As such, the system is incapable of flexible adaptation is both necessary and sufficient to enable action towards the real
to situations that involve global arrays of different dimensions: A world, without error or distortion. Indirect or constructive ap-
global invariant is specific for a particular array and does not trans- proaches hold that sensory information is necessary, but not suffi-
fer to other (e.g., higher or lower dimensional) arrays. A system cient. Memory, inference, and other cognitive operations must oc-
combining multiple (modal) sources of information would be cur before we can make sense of sensation and hence use it to
more flexible in this regard. guide our actions.
Similar considerations pertain to the attunement to global in- The former pole we might identify with the ethos of physics,
variants during perceptual-motor learning. How can extensive with its search for law-like generalisations. The latter pole we
practice involving full vision result in the ability to juggle with eyes might identify with psychology, an altogether more pluralistic dis-
closed? After all, practice in a particular global array does not al- cipline, within which law-like generalisations are merely a part of
low for simultaneous attunement to an invariant in a qualitatively a far more heterogeneous and negotiable body of findings.
different (lower dimensional) array that is not actually present. While at different points in their article the authors deal with
Likewise, shifts in dependence on particular modalities during a both these poles, the underlying concern seems to be strongly with
learning process (Fleishman & Rich 1963) are difficult to under- the former, physics-like project. For example, in the introduction

they note that investigating how sensory arrays might specify the momentarily. In which case, a multi-modal array of relations might
properties of reality is a problem for physicists, not psychologists. have the role of maintaining the spatio-temporal consistency of ac-
They nonetheless pursue this issue and conclude that the arrays tion.
of ambient energy reaching any particular sensory system are ac- It seems a rather limited use of this powerful idea, merely to
tually ambiguous with respect to the dynamics of the physical pursue an old agenda of formally accounting for specification.
world from which they come. The arrays cannot, they claim, un- Sensory systems, and the amodal patterns of neural activity to
ambiguously specify real-world properties. which they contribute, are tuned by on-going activity. For all that
Prima facie, this seems like a significant challenge to the direct S&B acknowledge the importance of action, the emphasis re-
approach as it presently stands. That approach, based on the work mains, in true Gibsonian style, on perception, on what is there to
of James Gibson, has always rested on the assumption that the dy- be perceived. They note Berkeley’s scepticism about how the dif-
namic structure of an ambient array is rich enough to be the basis ferent types of sensory experience could ever cohere into unified
for adaptive, accurate action. If it can be shown that it is not, then percepts. In response, they propose higher order relations.
this seriously weakens the whole approach. But Berkeley is more easily defeated. Perception and action are
But Stoffregen & Bardy (S&B) do not give up on specification. unified by an effort after meaning. Sensory systems exist and are
In fact, they offer a new conceptual basis for it and one which, be- used in order to discern objects and situations fit for action. For
ing broader, may actually be more stable and general. The novel simpler organisms, this may well be achieved primarily on the ba-
aspect of their proposal is that specification does indeed exist, but sis of a specific, and hence specifiable, set of relations between the
within a global, that is, multi-sensory array of higher-order rela- world and the information reaching the senses. For more complex
tions between arrays of energy picked up by the different senses. ones, and most especially for human beings, the open, creative,
This proposal has a great deal to commend it. The notion that and adventitious nature of their actions makes specification, at
specification was tied to a particular sensory system seems unlikely whatever level, a secondary matter. It is not a central aspect of per-
a priori. Animals, after all, use their senses together, not sepa- ception and hence need not have such a primary role in psycho-
rately, and seldom rely on a single sense to coordinate complex logical inquiry.
patterns of activity. Moreover, there is a great deal of evidence that
points the other way. That there is cross-talk between sensory
pathways is well known, but what recent neuropsychological find-
ings indicate is that sometimes there may also be cross talk be- Three consequences of believing that
tween the sensory and motor pathways (e.g., Rizzolatti & Arbib
1998). Recent moves towards a more embodied, enactive view of
information lies in global arrays and that
perception and action likewise support the idea that we should perceptual systems use this information
treat the sensory-motor pathways as mutually constraining or
defining (Hurley 1998). John B. Pittenger
However, S&B are still pursuing a conventional ecological line, Department of Psychology, University of Arkansas at Little Rock, Little Rock,
AR 72204. [email protected]
namely, to demonstrate specificity as a challenge to inferential
theories. The task of ecological psychology is to show that specifi-
cation exists and rests in discoverable, law-like relations. But since Abstract: Stoffregen & Bardy provide grounds to suppose that specifica-
tion requires global arrays and that this information is used by perceptual
these relations are now treated in a rather more complex way than systems. Three conclusions follow from this supposition; (1) global speci-
before, the challenge is perhaps not as strong or distinctive as it fication will be taken seriously only if additional examples are discovered;
was. The notion of “higher-order” relations is a significant step (2) research into single-sense information must take global information
away from the simplicity and clarity of the original Gibsonian pro- into account, and (3) ecological psychologists must account for perceptions
gramme. based upon non-specific information.
Perhaps a more radical challenge, to both direct and indirect
approaches, would be to contest the notion of specification itself. Stoffregen & Bardy (S&B) make many claims about information
This is not to reject it. Clearly, under some circumstances and for in what they call “global arrays.” Rather than attempt to evaluate
some perceivers, especially simpler ones, specification is impor- all those claims, I will assume that the authors are at least partly
tant. For example, organisms like reptiles, who, while active and right and examine the consequences of that assumption. Specifi-
skillful are nonetheless bound to rather fixed patterns of action cally, I will suppose that the following “limited global hypothesis”
within relatively limited niches, probably rely on specific patterns is true: (1) some aspects of the environment are completely spec-
of energy which are not too difficult to discover. Such organisms, ified only by information in a global array, and (2) when such
in Popper’s terms, are in closed rather than open ecological and global array information is available, it is detected and used by per-
evolutionary niches. What they need to know in order to do what ceptual systems on some, but perhaps not all, occasions.
they need to do may indeed be quite specific and hence specifi- Even if the limited hypothesis becomes widely accepted, I will
able. be very surprised if many perceptual researchers go on to make
But for organisms living in more open niches, and especially for major changes in their theoretical or empirical work. Two classes
creative perceivers like human beings, specification may be irrel- of demonstrations must be developed before global arrays will
evant. Their actions are more adventitious and unpredictable; have a broad impact on research practice. First, more analyses of
they perceive and act under uncertainty. Here, precise specifica- information are needed, ones which clearly document the exis-
tion may not be a realistic objective for psychological research. tence of information that is available in the global array but not in
Complex organisms, especially human beings, are not like that. any “single sense” array. Second, researchers must be convinced
The world is imperfectly known and actions are seldom perfectly that perceivers make use of global information. In other words, we
adapted to circumstances. More often they are provisional, ex- need empirical demonstrations showing that perceptual perfor-
ploratory, and, being subject to inaccuracy, are continually recali- mance based on global information is measurably different (faster,
brated. more accurate, less variable) than that based on single-sense in-
The proposal that there exists a higher-order, multi-modal, re- formation. Until a reasonable number of both types of demon-
lational basis for perception and action is an exciting one. The strations are available, global information can be treated as a rare
challenge, or opportunity, is to use this idea to investigate the flex- special case rather than as a central fact about perception. In sum-
ibility, not the fixity, of higher-order perception and action. Thc mary, while S&B provide a first glimpse of what may eventually
sensory systems of higher organisms are tolerant. That is, they are constitute a major challenge to perceptual theory and research, a
able to operate under uncertainty and intermittent interruption. good deal of work remains to be done to make that challenge fully
Indeed, under most circumstances, specification may only exist credible.

My second group of comments concerns the appropriate con- Movement dynamics and the environment
duct of research. The ultimate goal of most perceptual researchers to be perceived
is, or at least should be, to provide an account for perception as it
occurs in everyday life. Researchers who both have this goal and Gary E. Riccio,a Richard E. A. van Emmerik,b
accept the limited global array hypothesis must include consider- and Brian T. Petersb
ation of global array information in their work. First, it would be a
Exponent, Inc., Natick, MA 01760; bDepartment of Exercise Science,
risky to limit analyses of information to single-sense arrays: If the University of Massachusetts, Amherst, MA 01003. [email protected]
information needed to perceive the environment is sometimes {rvanemmerik; bpeters}@excsci.umass.edu
available only in a global array, then the researcher cannot know www.umass.edu/SPHHS/exsci
in advance whether or not the information relevant to the aspect
of the environment he or she is studying lies in a local or a global Abstract: In perception science, an alternative to focusing on individual
array. Second, when global array information is known to be avail- sensory systems is to describe the environment to be perceived. We focus
able for the phenomenon under study, it is surely important to on the emergent dynamics of human-environment interactions as an im-
know whether or not that information is used by perceivers. While portant category of the environment to be perceived. We argue that in-
the researcher might well demonstrate that local information does formation about such dynamics is available in subtle patterns of movement
influence perception, we would still not know whether or not variability that, of necessity, stimulate multiple sensory systems.
global information is utilized in everyday perception.1 In sum-
mary, the target article presents a challenge to all perceptual re- Stoffregen & Bardy (S&B) argue that specification exists only in
searchers: We can no longer be certain that our research into per- patterns that extend across different forms of ambient energy.
ception based on single-sense information will contribute to the They emphasize that this is a fundamental problem for the vast
understanding of perception as it occurs in everyday life. majority of experiments that seek to arbitrate between theories of
Finally, the target article has important implications for follow- direct and indirect perception by manipulating “single-energy ar-
ers, including myself, of the ecological approach to psychology. rays.” Their arguments suggest further that such experiments are,
Ecologists believe that it is rarely useful to study perception using at best, misleading with respect to the behavior of real animals in
displays from which the researcher has removed some of the in- real environments.
formation available in naturally-occurring arrays. The logic behind We are in basic agreement with S&B about the centrality of the
this belief is straightforward: Since perception in everyday life is “global array” in understanding animate behavior. Our research,
based upon the detection of information which specifies the envi- however, is not concerned with comparisons between theories of
ronment, studies using arrays which do not specify the environ- direct and indirect perception. Instead, direct perception is a first
ment will not tell us how the perceptual system operates in every- principle of ecological psychology that guides our research on the
day life. Ecologists believe that the results of research in which the interdependence of perception and movement. Direct perception
experimenter has, by artifice, removed information from arrays motivates an interest in mappings between the world and the stim-
(e.g., by requiring monocular viewing, preventing head motion, ulation of an animal’s sensory systems. We are interested in how
showing displays for fractions of a second, etc.) have no clear in- such mappings allow one to observe and, thus, to control one’s
terpretation. However, if the limited hypothesis is correct, then movement in an environment. At the same time, we examine how
ecological psychologists will need to pay more attention to per- movement instantiates such mappings and how a tacit under-
ception in circumstances where the environment is not uniquely standing of our own movement dynamics reveals momentary task-
specified. specific domains over which 1:1 mappings exist (Riccio 1995; Ric-
It seems to me clear that circumstances arise both naturally and cio & McDonald 1998b).
frequently which force perceivers to interact with the environ- Our approach to understanding perception in the context of
ment with less-than-optimal information available. Moreover, real interactions between an individual and an environment is
such interactions often achieve the perceiver’s goals. Note, for ex- based on Gibson (1979/1986). Although this last work of James
ample, how effectively the blind walk and how much the sighted Gibson focused on visual perception, we believe it provides the
can do in the dark, when they are looking in the “wrong direction,” best blueprint for studying specificity in the “global array.” The
and so on. Similarly, the deaf function well in everyday life, as do reason is that Gibson made a clear scientific distinction between
the hearing when noise masks informative sounds. A systematic “the environment to be perceived” and “information in stimula-
account of perceptually-guided actions which are frequent, suc- tion,” and he gave logical precedence to the former. Put simply, a
cessful, and occur in natural situations must be included in any scientist must understand what can and should be perceived be-
theory of perception. fore questions about specificity can be addressed (Riccio 1993;
The target article rightly criticizes researchers for assuming, on 1995; Riccio & McDonald 1998b). This starting point would not
ill-examined grounds, that an adequate theory of perception can necessarily lead to mappings between the environment to be per-
be based on information carried only in single types of energy and ceived and patterns of stimulation (i.e., information in stimula-
detected only by the traditional senses. The question of whether tion) of a single sensory modality. In fact, as S&B point out, we
or not global information actually exists and is used by perceptual cannot determine how this could ever be the case when human
systems matters very much to both theory and research. The tar- movement is involved (Riccio 1995; Riccio & McDonald 1988a).
get article does not answer this question. It does, however, show Furthermore, we are sympathetic to their argument that it may be
us that we need to stop assuming that the answer is self-evident impossible or meaningless to study perception only in the absence
and start doing the difficult work that will provide the facts which of movement. The scientific study of perception must be broader
will lead to the answer. than what it has been throughout most of twentieth century.
Most of our work on human posture and movement has focused
NOTE on the environment to be perceived. A fundamental assumption
1. It is worth noting that we do not know very much about the accuracy
of such “ecological physics” is that the environment cannot be de-
of perception in everyday life. Most of our research concerns the effect of
some stimulus variable on perception: that variable is manipulated in var- scribed independently of the animal and the animal cannot be de-
ious ways and the perceptual consequences tracked. We rarely establish scribed independently of the environment (Gibson 1979/1986;
the accuracy of perception in everyday life (i.e., when full information is Riccio 1993; 1995; Riccio & McDonald 1998b; Riccio & Stoffre-
available and the perceiver’s actions are unconstrained) and then ask how gen 1991; Stoffregen & Riccio 1980; 1991). Any biomechanical
much of the variance in everyday perception can be accounted for by our model of posture or movement, for example, makes specific as-
pet stimulus variable. sumptions about the surroundings of the body (e.g., the surface of
support) even if they are not explicitly included as parameters of
the model. Descriptions of the human body and its surroundings

can be unified with respect to conventional mechanical properties in stimulation. Recent work from our laboratory suggests that the
or dynamical constraints that are due to momentary couplings be- coordination and timing of the eyes, head, and trunk when shift-
tween perception and movement (Riccio 1995; Riccio & McDon- ing gaze is influenced by gait cycle dynamics (Peters et al. 2000).
ald 1998b). We have been working to characterize such emergent The proposition here is that increased rigidity or decreased vari-
dynamics, which impose constraints on human posture and move- ability in segmental couplings could limit the degrees of freedom
ment (Riccio 1993; Riccio & McDonald 1998a). in this eye-head-trunk coupling and possibly obscure movement
Our working hypothesis is that the dynamics of the animal- dynamics that are important for perception.
environment interaction can be perceived and that this capacity is
essential to human adaptability. This requires that information ACKNOWLEDGMENT
about such dynamics is available in the stimulation of sensory sys- The writing of this paper was supported by a grant from the Whitaker
tems. We have argued that pick-up of dynamical information is fa- Foundation awarded to Richard van Emmerik.
cilitated by obtaining stimulation through the exploratory activity
of perceptual systems (Riccio 1993; Riccio & McDonald 1998a).
Perceptual systems include sensory systems and all the movement
systems of the body that act essentially as accessory structures for Inadequate information
sensory systems (Gibson 1979/1986) . We have provided evidence
that the dynamics of animal-environment interactions are revealed
and deficient perception
in subtle, albeit ubiquitous, patterns of movement variability that, Michael A. Riley
of necessity, stimulate multiple sensory systems (Riccio 1993; Ric-
Department of Psychology, University of Cincinnati, Cincinnati, OH 45221.
cio & McDonald 1998a). An important implication of our findings michael.riley@uc. www.oz.uc.edu/~rileym/pmdl/RileyLab
is that perception and movement will be compromised by condi-
tions that suppress or obscure the informative patterns of move- Abstract: Stoffregen & Bardy’s primary motivation for rejecting current
ment variability. We have argued that body restraint is one method views on specification in favor of the global array is that current forms of
of suppressing such patterns (Riccio et al. 1992; Stoffregen & Ric- specification in single-energy arrays allow the ambiguous or inadequate
cio 1988; 1991). We also have argued that pathological movement specification of reality. I show that this motivation is not justified, and that
may result from obscured patterns of movement variability (i.e., the global array concept still falls prey to inadequate specification.
impaired observability), rather than from impaired musculoskele-
tal dynamics per se (Riccio & Stoffregen 1991). In certain situations that Stoffregen & Bardy (S&B) describe,
Our research on movement coordination problems in Parkin- single-energy stimulus arrays are discrepant, resulting in “input
son’s disease has focused on the consequences of reduced vari- conflict, with its attendant lack of specificity, and . . . the need for
ability for movement coordination and perception. The classic inferential processing” (sect. 3.3.3). I do not question the exis-
Parkinsonian symptoms are tremor, rigidity, and slowness or ab- tence or formal structure of the global array, or even the possibility
sence of movement. Functional implications are that individuals that animals might use the global array to guide behavior. Instead,
with Parkinson’s disease experience increased instability during I question the assumption that insufficiencies of single-energy ar-
postural and movement tasks, an associated higher incidence of rays require an appeal to a global stimulus array. A theory of direct
falls, and reduced ability to make transition to another movement perception does not require that reality always be adequately
pattern. A common assumption in many studies on movement dis- specified in stimulus arrays, because perceptual errors may often
orders is that increased variability (e.g., stride length or stride fre- be traced to inadequacies of information. Furthermore, it is illog-
quency) is associated with instability during locomotion. However, ical to dismiss current formulations of specification in favor of the
based on theoretical considerations and empirical work from a dy- global array on the basis that information in single energy arrays
namical systems perspective (e.g., Diedrich & Warren 1990; does not adequately specify reality, because the global array may
Haken et al. 1985; van Emmerik et al. 1999), variability in coordi- also inadequately specify reality.
nation dynamics has been shown to be essential in pattern change. When reality structures two or more arrays differently, then ac-
In addition relatively high frequency variability can play a role in cording to S&B (sect. 3.2), “at least one of the structures must be
exploratory behavior (Riccio 1993; Riccio & McDonald 1998a). In wrong, in other words, not specific to reality.” From their per-
addition, noise added to a weak signal can improve detectability spective, any position that allows for structure in energy arrays to
in muscle spindle or cutaneous receptors (“aperiodic stochastic inadequately or incorrectly specify reality (i.e., that allows for in-
resonance”; Collins et al. 1996). put conflict) is fundamentally flawed. The motivation for their po-
In our work on gait in Parkinson’s disease we have shown a sys- sition seems to revolve around one implicit assumption: The only
tematic reduction in variability of relative phase between pelvic satisfactory account of specification is one that eliminates the po-
and thoracic rotations compared to age-matched control subjects. tential for inadequate information. S&B seem to imply, further-
This reduced variability is associated with a reduced capacity or more, that because direct perception (Gibson 1966; 1979/1986)
inability to change movement patterns (van Emmerik et al. 1999). depends upon specification, then in order for perception to be di-
The relative phase variability is considered a functional measure rect, stimulus information cannot be inadequate or deficient.
of rigidity in Parkinson’s disease. In a similar fashion, the classical Neither of those positions is necessary for a direct account of
4 – 6 Hz tremor in Parkinson’s disease has been shown to affect the perception. There are numerous instances where the information
coordination dynamics during gait and manual movements (van contained in energy that reaches a perceptual system is somehow
Emmerik & Wagenaar 1996). insufficient (i.e., does not specify reality). When this occurs, per-
The inability to change coordination dynamics in the trunk is ceptual error may result. Consider the classic “bent stick illusion”
linked to reduced variability in couplings and has expected conse- – a straight stick partially submerged in water appears to be bent,
quences for gait stability in patients with Parkinson’s disease. It is because the light that reaches the eyes contains inadequate infor-
our contention, however, that the observed reduction in variabil- mation (due to refraction). The information that specifies a bent
ity of segment couplings as well as the steady 4 – 6 Hz tremor not stick is inadequate because it specifies a false state of affairs (Gib-
only have consequences for stability, but also affect the patient’s son 1966). From a direct perception perspective, the illusory per-
capacity to detect relevant dynamics for the task at hand. In this ception is not problematic, because perception can only be as ac-
regard, we claim that the observed symptoms of rigidity and curate as the information that is detected (cf. Kennedy et al. 1992).
tremor in Parkinson’s disease patients have consequences for the S&B argue that a theory of specification is unsatisfactory if it al-
pick up or availability of information by perceptual systems. We lows inadequate specification. Because of their interest in inter-
are currently investigating the consequences of head, neck, and modal perception, the example of inadequate specification that they
trunk rigidity in these patients for the availability of information focus upon is input conflict. However, input conflict is but one ex-

ample of inadequate information; many other forms exist (Gibson Kudos to Stoffregen & Bardy (S&B) for addressing the problem
1966). The concept of the global array avoids input conflict, but it of multi-modal integration from a specificational perspective. Many
does not avoid other forms of inadequate information. For instance, Gibsonian-based explanations of multi-modal effects – including
there is nothing about the global array that would predict an ob- our own (e.g., Rosenbium & Saldaña 1996; Rosenblum et al. 2000)
server to perceive a partially submerged stick as being straight – have provided only a rough sketch of how specification might be
rather than bent. The global array is not immune to some forms of instantiated cross-modally. S&B’s thoughtful examination of these
inadequate specification, and therefore is potentially ambiguous issues provides an invaluable step toward solving the problem. Re-
with respect to reality. By their own criterion – the elimination of garding their solution, we suspect that the global array does exist
ambiguity with respect to reality – S&B’s global array is just as un- and that it can provide specificational information to guide be-
satisfactory as other specificational accounts. In my opinion, the havior. However, we are doubtful that specificity exists only at the
possibility of inadequate specification is not a problem with the level of the global array, and take issue with S&B’s arguments for
global array (for the same reasons it is not a problem for single ar- this claim. We suspect that specificity for behaviorally-relevant en-
rays), but it demonstrates faulty logic in S&B’s motivation for re- vironmental properties exists in energy arrays that can be appre-
jecting current accounts of specification in favor of the global array. hended by single or multiple perceptual systems.
S&B state that perceptual errors might imply a need for per- S&B build a strong case for the ubiquity of multi-modal per-
ceptual learning rather than a lack of specificity. While this may be ception in even the most seemingly unimodal tasks (e.g., change
true in many circumstances, it does not, in general, rule out the pos- in visual fixation). We are also generally supportive of the global
sibility that some perceptual errors are due to a lack of specificity. array concept, and believe it might prove a useful tool for the spec-
For instance, it is not clear how any amount of improvement in the ificational perspective in explaining some cross-modal effects. We
ability to discriminate structure in a stimulus array would prevent suspect that there are properties of the environment that are only
the bent stick illusion. Surely, one may come to know that the stick specified in higher-order relations across energy arrays. However,
looks bent because of refraction, but no amount of perceptual it is the proposed universality of these properties where we begin
learning could cause the stick to appear straight, rather than bent, to take issue with S&B’s proposal.
because the light that reaches the eyes specifies a bent stick. S&B argue that the global array is the only level at which spec-
An additional issue related to inadequate information and per- ification can exist. They motivate this argument with examples
ceptual learning deserves brief mention. It has been empirically that ostensibly show conflictual cross-modal specificity. However,
demonstrated that over the course of perceptual learning, people we are not likely the only readers to note that many of S&B’s ex-
may base their initial responses on nonspecific stimulus variables, amples are based on laboratory contrivances (moving rooms, ro-
but eventually shift to basing their responses on variables that tating drums, flight simulators, McGurk effects), or recent human
specify the object or event being perceived ( Jacobs et al. 1999; invention (cars, airplanes, elevators, weightless environments). It
Michaels & de Vries 1998). S&B (sect. 7) concluded that such a is unlikely that the evolution of human perceptual systems could
pattern of responses would be “contrary to and unnecessary in have anticipated these scenarios. While theories should ultimately
ecological theory,” presumably because it would involve a reliance be able to explain how perception occurs in these artifactual set-
on inadequate (nonspecific) information. I disagree, and instead tings, it would seem erroneous to build a theory of specification
suggest that this result supports the ecological perspective by around these examples.
demonstrating that the acquisition of accurate perceptions is In other examples, S&B enlist global array properties in lieu of
closely tied to the discovery and utilization of relevant, macro- thorough descriptions of single-energy arrays. Consider the traf-
scopic stimulus variables. When responses are not based on these fic scenario in which a thorough description of the acoustic array
variables, they are typically inaccurate and/or unreliable. (including ambient and reflected sound), could reveal structure
In sum, I question the logic of S&B’s motivation for and devel- that is specific to the (relatively) stationary environment as well as
opment of the global array concept, rather than the concept itself. to moving cars. With this fuller description, the acoustic array
Sensitivity to a global array is a plausible idea and the construct could be considered sufficient for specification. Similar criticisms
might have explanatory power. One suspects that ultimately the could be applied to S&B’s global array explanations of (kinesthet-
matter will become an empirical one. With respect to that possi- ically) detecting upright stance on an inclined board, and (visually)
bility, I raise a final concern. The novel methodologies that S&B detecting Tau for both target distance and head movements.
propose in order to test for sensitivity to the global array may not Finally, while not all aspects of an event scenario might be avail-
be compatible with the study of phenomena that have driven a able unimodally (e.g., concurrently perceiving self-movement and
substantial portion of research on intermodal perception – adap- having the awareness that it is driven by a simulated display), in-
tation to transformed stimulus arrays and intermodal discrepan- formational properties most relevant to any single action might be
cies (e.g., prism adaptation). Adaptation and global array method- available to a single sensory system. Consider the case of driving
ologies seem to place incompatible demands on experimental a car and apprehending one’s position relative to both the car and
protocols. Can empirical and theoretical connections between the outside world. This example highlights the importance of con-
global array and adaptation to intermodal discrepancies be made? struing events, and their perception, as nested. We suspect that
for many situations, each nested event is specified unimodally,
while the nesting relationship itself might be specified in the
global array (thought it need not). Construing events as nested
could relieve the problem of conflictual specification occurring in
The generality of specificity: Some lessons more natural settings.
from audiovisual speech From this analysis we can summarize our own perspective, a
perspective similar to Gibson’s. Information itself is modality-
Lawrence D. Rosenblum and Michael S. Gordon neutral. It can be instantiated in specificational structure that is
Department of Psychology, University of California, Riverside, Riverside, CA available to a single sense, available to multiple senses simultane-
92521. [email protected]
ously (and redundantly), or available only across senses (non-
www.psych.ucr.edu/faculty/rosenblum/larhome.html
redundantly). Fortunately for animals, the second case is the most
common: redundant information across modalities allows for a
Abstract: The global array might prove to be an important and even nec-
essary concept for explaining some multi-modal phenomena from the
graceful degradation from sensory impairment and diminished
specificational perspective. However, we suspect that specification exists ambient arrays (fog, noisy rooms). The latter case – global array
in energy arrays detectable by single or multiple sensory systems. We ar- specification – is more rare, less relied-upon, and may be idiosyn-
gue for a more general modality-neutral perspective and review results cratic to particularly higher-order stimulus properties (e.g., aes-
from recent research on audiovisual speech perception. thetic) and relations between tasks (e.g., in artifactual situations).

A modality-neutral1 perspective could account for multimodal Specificity is always contingent on
perception without requiring internal mediation across modali- constraints: Global versus individual
ties. A type of information (e.g., inverse rate of change informa- arrays is not the issue
tion for time-to-arrival, time-varying kinematic pattern informa-
tion for speech articulation) can be instantiated as structure in
Sverker Runeson,a David M. Jacobs,b Isabell E. K.
multiple arrays (visual; auditory), but perceiving is concerned with
the information, not the energy array in which it is available. In Andersson,a and Kairi Kreegipuuc
aDepartment of Psychology, Uppsala University, Uppsala SE 75142,
this sense, “cross-modal” integration is not something that occurs
Sweden; bHuman Movement Sciences, Vrije Universiteit, Amsterdam 1081
in the animal, but occur in – and as a property of – the informa-
BT, The Netherlands; cDepartment of Psychology, University of Tartu, Tartu
tion itself. This would be true whether the specifying structure ex- 50410, Estonia. {sverker.runeson; isabell.andersson}@psyk.uu.se
ists within a range of energy detectable by a single sensory system, www.psyk.uu.se [email protected] www.psyk.vu.nl
or across a range of energy detectable only by multiple sensory sys- [email protected]
tems. Specificity is general.
We feel that much of the neurophysiological and behavioral ev- Abstract: Stoffregen & Bardy’s proposal that perceptual systems can use
idence cited by S&B are also supportive of a modality-neutral ac- information defined across two or more sensory domains is valuable and
count (e.g., Fitzpatrick et al. 1994; Stein & Meredith 1993). The urgent in its own right. However, their claim of exclusive validity for global-
same seems true of recent neuropsychological and behavioral array information is superfluous and perpetuated for incorrect reasons.
findings on audiovisual speech perception. Recent brain imaging The seeming ambiguities of individual arrays emanate from failures to
research shows that visual speech can change auditory cortex ac- consider relevant ecological constraints and higher-order variables.
tivity during audiovisual integration and even silent lipreading
(Calvert et al. 1997; Sams et al. 1991). Also, mounting perceptual James Gibson’s (1950; 1966; 1979/1986) demonstrations of speci-
research suggests that the audiovisual streams are integrated very ficity and the associated refutation of the ambiguity dogma were
early, possibly at the level of information extraction (see Green made possible by two brave intellectual advances: Gibson recog-
1998, for a review). The sensitivity to modality-neutral informa- nized that useful information could reside in higher-order prop-
tion also seems to occur at a very young age. Infants detect audio- erties of the ambient arrays (a breakaway from elementarism) and,
visual correspondences in phonemic properties (Kuhl & Meltzoff furthermore, that by restricting analyses to natural conditions a
1984), and portray McGurk-effect-like behavior (Rosenblum et al. wealth of meaningful and reliable information could be brought
1997). These latter findings should also be encouraging to S&B: in evidence. The latter amounts to an invocation of ecological con-
they suggest that sensitivity to the specificity existent across straints, in addition to the laws of nature.
modalities is a fundamental perceptual ability. Stoffregen & Bardy’s (S&B’s) proposal that relevant information
But what of the McGurk effect itself? Would not the global ar- can reside in a global array extending across energy array borders
ray concept explain how a /d/ can be perceived from the concur- falls well within Gibson’s approach in the first respect. If useful
rent visual specification of /g/ and auditory specification of /b/? specifying variables can consist of relational properties across a
In fact, here we must tip our hats to S&B: it is likely that the re- single array and over time, it follows naturally that they could also
sultant perceived /g/ is specified in the structure existent across extend over two or more types of energy arrays. Thus, S&B’s pro-
optic and acoustic arrays. For an ecological explanation of the posal of a search for global invariants is laudable as such, as is their
McGurk effect, the global array could save the day. However, con- claim that properties specified by such invariants could be directly
trary to S&B, we do not think that the global array can provide in- perceived. This enterprise gains urgency from the recognition that
formation about the experimental manipulation itself. It is unclear the senses operate simultaneously and that information from dif-
what type of perceptual exploration might reveal the audiovisual ferent arrays is often necessary for the control of action.
discrepancy, short of seeing the audiovisual dubbing procedure. However, S&B not only proffer global invariants, they also
Furthermore, research shows that it makes little difference make extensive claims that specificity occurs exclusively with such
whether subjects are asked to report “what was said” or “what you invariants. We disagree, and argue that S&B’s failure to find spec-
heard”: audiovisual influences still occur (e.g., Massaro 1987). We ification in individual arrays follows from their failure to apply
find the inability of the global array to specify the McGurk ma- Gibson’s insights concerning ecological constraints and non-
nipulation encouraging: Perceptual encapsulation evidences the elementarism. The groundbreaking contributions by Gibson and
lawful nature of specificational information. others concerning information in individual arrays should remain
In conclusion, we think S&B have posed an important challenge a valid basis for perception research.
to specificational approaches. A specificational account must ex- Ecological constraints. All specificity is contingent on con-
plain how multiple sensory systems simultaneously extract struc- straints (“constraints as grantors of information,” Runeson 1988;
ture from energy. However, we will put our money on modality 1989; cf. Barwise & Perry 1983). Moreover, the specificity they en-
neutral information existent in all forms of specificity: whether de- gender pertains to properties, not the structures or media as such
tectable by single or multiple sensory systems. (cf. Bingham 2000b; Turvey 1992). Potentially, constraints can be
any lawfulness or regularity, ranging from laws of nature, via eco-
ACKNOWLEDGMENT logical universals such as textured environmental surfaces to, say,
Preparation of this comment was supported by NSF grant SBR-9617047 local conventions. Thus, relations of informative specificity vary in
to the first author. degree of universality depending on the scope and stability of the
NOTE constraints that grant them.
1. We prefer the term modality-neutral over “amodal”: while percep- It follows that specificity can not be meaningfully investigated
tual information is not modality-specific, it cannot exist without instantia- without acknowledging the questions: specification of what prop-
tion in some energy array (excepting ESP). erties, under what conditions? Without restricting analyses to par-
ticular properties and conditions, specificity will be hard to find.
We question the relevance of universal, unconstrained delibera-
tions on specificity for the understanding of perception and action.
Apparently, S&B do not consider the role of constraints in their
discussion of specificity. Thus, we are not impressed when they
take the hypothetical existence of perfect virtual-reality devices to
prove optic array ambiguity. To be consistent, they should also
have considered the possibility of slightly more complicated de-
vices as proof against specificity in the global array. What if light

would not propagate linearly? What if an evil genius were distort- component. When related to the perpendicular component it specifies
ing arrays? If one allows such arguments there will always be rea- slant of the surface of support without recourse to gravito-inertial infor-
sons to reject specificity (cf. Dretske 1981). mation.
An ecological stance does not proffer specificity because injec-
tion of yet higher order variables can outweigh such arguments,
but because it dismisses non-ecological circumstances as irrele-
vant. Constraints at the ecological level grant specificity to infor- Perceptual systems: Five1, one, or many?
mative variables, whether single- or multi-array. It is biologically
irrelevant whether a variable that specifies a useful property of the Nigel J.T. Thomas
organism-environment system is granted by ecological constraints Philosophy Department, California State University, Los Angeles, CA 90032.
[email protected]
or by physical laws alone.
www.members.leeds.ac.uk/n.j.thomas70/
Culture and technology add and break constraints. Traffic lights
add specification of safe street crossing. Conversely, virtual-reality
Abstract: The target article’s value lies not in its defence of specification,
gear and swinging rooms deliberately break a basic ecological con- or the “global array” concept, but in its challenge to the paradigm of 51
straint by decoupling the visible environment from the earth and senses, and its examples of multiple receptor types cooperatively partici-
thus makes the optic array lose some of its specificity. Inadvertent pating in specific pick-up tasks. Rather than analysing our perceptual en-
cases also occur, as in elevators. This presents new possibilities and dowment into 51 senses, it is more revealing to type perceptual systems
challenges to perceivers. Will they be able to educate their atten- according to task.
tion to detect other information – in the global array – that spec-
ifies the new situation? Interestingly, the success of virtual-reality Stoffregen & Bardy (S&B) can and should not hope to persuade
technology will rely on people’s failure to do so. us that the traditional five senses (plus, presumably, a few more,
Non-elementarism. S&B’s failure take advantage of Gibson’s like vestibular sense, proprioception, etc.) just do not exist. Clearly
nonelementaristic approach is evident in their dismissal of so- for many purposes it is valid and useful to think of the senses in
matosensory specificity concerning body sway versus tilting of the this way. The circularities they point to, arising from defining the
surface of support: Ankle rotation can be due to either of them. senses in terms of receptor types or energies transduced, do not
However, in many other somatosensory variables, the two events seem to be vicious.
have distinct effects. For instance, standing on a slanted board However, their argument opens up a very important pragmatic
produces force components parallel to the surface with corre- and heuristic question that has received almost no previous atten-
sponding shearing skin deformations, varying in proportion to the tion: Is thinking of our sensory endowment as consisting of 51 in-
angle of tilt. Conversely, sway produces alternating shifts between dependent perceptual systems the most useful and perspicuous
heels and toes in the perpendicular skin pressure component. Am- way to view the mechanisms of ecological perception? Here S&B
plitudes and directions of ankle joint load forces will also differ dis- make a good case for a negative answer, challenging deeply en-
tinctly. Somatosensory ambiguity has certainly not been demon- trenched and, up to now, virtually unexamined assumptions. They
strated.1 show that certain important, real-world perceptual tasks require
S&B’s claim that the acoustic array can only specify relative mo- the coordinated deployment of more than one type of receptor. If
tion between other cars and one’s own is similarly vacant. One can we think of the senses as 51 channels, our attention is diverted
hear quite a bit about how one’s own car is moving, in particular from such cases (which may well be the rule rather than the ex-
whether it is moving or not. The acoustic array is structured by re- ception), and even if we do notice them, the separate senses
flection and occlusion of sounds among cars on the road and framework leads us to posit unnecessarily complex and conceptu-
among terrain features. These multitudinous effects are a poten- ally suspect inferential or computational theoretical accounts of
tial source of informative variables in the acoustic structure, which them.
S&B have dismissed offhand. But if 51 senses is not the most useful picture of things, does it
We do not claim that we have shown or can show that specificity follow that the heuristic alternative is to think of the perceptual
exists in those cases. Our point is that S&B have not provided any environment as a single global array, presumably to be perceived
proof for the claimed ambiguities. In principle, ambiguity in nat- by a single global perceptual system? S&B apparently think that if
ural arrays is not the kind of thing that can be proved, because the ambient energies available to our perceptual system do not un-
there is no way to ensure that all possible higher-order variables ambiguously specify what is really out there then we must be
have been tried and all relevant constraints have been considered doomed to perceive the world only “indirectly,” our experience
for their information-granting potential (Runeson 1988). Ambi- mediated through representations and inferential processes. They
guity could become a tenable claim only after long and hard at- are thus led to the notion of the global array in the hope of find-
tempts to prove specificity have failed. ing an information source sufficiently rich to ensure specification.
Finally, we notice that S&B’s reasoning is remarkably congru- But unless we understand “direct perception’’ to mean “invariably
ous with that of the classical, no-specification tradition. The pos- veridical perception” (in which case perception certainly is not di-
sibility of specification is brushed very lightly, often by letting a rect) it simply does not follow that directness requires unambigu-
single variable (mis-)represent the informative potential of a ous specification. In fact, we do not ultimately rely on mere per-
whole energy field. With non-specificity seemingly proved, a case ception to tell us what is really out there, we rely on science, which
is made for the standard remedy: invocation of something more. certainly involves inferential processes. Specification is a red her-
Traditionally, the recourse has been to memory and constructive ring, and the theory of the global array is a (probably inadequate)
inference – S&B instead bring in additional sensory domains as solution to a non-problem.
remedial necessities. Of course, the global array undoubtedly exists, and our sensory
endowment as a whole undoubtedly exists too. Thus, (granting the
ACKNOWLEDGMENTS general framework of Gibsonian direct perception theory) it will
The writing of this commentary was supported by grants from the Swedish inevitably be true to describe any perceptual episode as the pick-
Council for Research in the Humanities and Social Sciences (HSFR), from up of an invariant of the global array by the global perceptual sys-
the Netherlands Organization for Scientific Research (NOW, grant no. tem. But this is not to say very much. In fact, it is surely the case
575-12-070), and from the Estonian Ministry of Education. that many instances of perceptual information pick up do make
use of only one receptor type, and even the examples given by
NOTE S&B each involve only a small subset of the receptor types we
1. It would be tempting to call this reasoning of S&B’s “pre-Gibsonian,” have. To insist on treating ambient arrays and perceptual systems
however, also Gibson (1966, pp. 62f ) failed to consider the shearing force only as “global” wholes would be to obscure this point, and threat-

ens to be just as misleading as the paradigm of 51 independent suasive. Perception and action also tend to be studied and de-
channels. scribed separately, although most scholars would not want to claim
A more revealing analysis might be to type perceptual systems that either could function independently of the other.
in terms of the specific sorts of environmental information that Drawing on random examples from the history of psychology,
they gather. The target article’s examples suggest that, instead of we will first show that there has been more explicit awareness that
saying “this is an instance of vision, this of audition,” and so on, we the senses do not function separately than S&B give credit for,
might do better to say things like “this is an instance of perceiving whether the umbrella be amodal, cross-modal, or multimodal
that your conveyance has come to a stop,” “this is an instance of functioning. Subsequently, we will show that there is ample evi-
perceiving that a surface affords sitting,” and so on. In this vein, dence that cross- or multimodal functioning is primitive in per-
we can think of our sensory endowment as comprised of a num- ceptual development in infants, and that modern theories of brain
ber – probably quite a large number – of perceptual instruments, development build heavily on multimodal sensory input to the
each specialized for the pick up of particular sorts of environ- nervous system as the foundation for the establishment of func-
mental information, and actively deployed as and when that tional neuronal groups.
information is needed for the guidance of behavior. A perceptual Historical antecedents of multimodal functioning. That S&B’s
instrument (alternatively a “smart perceptual mechanism” [Rune- position – and their postulation of a global array – may not be par-
son 1977] , or “smart sensor” [Burt 1988] ) is a complex of anatom- ticularly novel is attested to by Titchener (1901) in his classic text
ical and cognitive structures that is capable of actively testing for on experimental psychology, in which he takes an even more rad-
the presence or amplitude of some specific type of environmental ical view. To his mind, “perception is not simply an aggregate or
property. It consists not only of receptors, but also of efferent sys- group of sensations; it is an aggregate or group of sensations put
tems that “tune” them, the musculature that orients them and together under certain conditions, arranged or harmonised upon
moves them so as to sample the ambient energy arrays appropri- certain patterns. The conditions are found in the physical world
ately, and the neural structures and algorithms that control these about us and the arranger or harmoniser is Nature herself”
“tunings” and movements and orchestrate appropriate responses (p. 128). Sensations, he argued, are joined together by our physi-
to the receptor outputs (Thomas 1999). I take it that by switching cal environment, not by “anything psychological [that] has inter-
neural algorithms, and thus the way in which receptors are de- vened between the sensation and the perception” (p. 129). He fur-
ployed, our fairly limited array of receptor types can be recruited ther suggested that the psychologies, “by the mere fact that they
to do a large number of different perceptual jobs, or putting it an- treat of perceptions one by one, in separate paragraphs” (p. 127),
other way, to form parts of a large number of perceptual instru- have contributed to the popular misbelief of separate senses.
ments (cf. Ballard 1991 on “sensor fission”). We do not so much Earlier, Stumpf (1890), in his theory of tonal fusion, went even
have 51 general purpose senses as a large array of anatomically further both in criticising existing positions and in attempting to
overlapping, specialized perceptual instruments, a capacious “box provide neural justification for his ideas. He introduced the notion
of tricks” (Ramachandran 1990). of synergies of the cerebral cortex – “determinate modes of co-
From the entrenched standpoint of the orthodox paradigm of operation of two nervous structures having its ground in the struc-
51 senses, this theoretical perspective must seem strange and ture of the brain, of such a kind that whenever the two structures
counterintuitive, but S&B throw that paradigm into deep ques- give rise to their corresponding sensations there arises at the same
tion. Furthermore, they direct our attention toward the signi- time a determinate degree of fusion of these sensations” (p. 214).
ficant but previously under-explored possibility that many per- He discusses the way in which such synergies might have been
ceptual instruments may cooperatively employ more than one re- built up over evolution so that what were once separate sensations
ceptor type. The considerable value of the target article lies, I became, over time, synergies – an issue to which we will return
think, in these challenges to entrenched orthodoxy, rather than in below in a development context.
the unnecessary and heuristically rather unhelpful notion of the Sensory functioning in early development. S&B argue that the
“global array.” concept of amodal specification “implies a comparison between
information obtained via different perceptual systems” that must
have been “independently generated,” and that “the postulation
of such a cross-modal comparison requires a prior assumption that
the senses work separately in such a way that their outputs can be
“The assumption of separate senses”: compared” (sect. 3.3.3). This is a misrepresentation both of the de-
Pervasive? Perhaps – Persuasive? Hardly! velopmental work to which they refer and of other work in this do-
main that makes no explicit assumption that the senses function
Beatrix Vereijkena and H.T. A (John) Whitingb separately. On the contrary, E.J. Gibson has argued against such
a
Department of Psychology, Norwegian University of Science and breaking down of the world and then inventing “processing mech-
Technology, Trondheim 7034, Norway; bDepartment of Psychology, anisms” to put the world together again (Gibson 1977). Her work
University of York, Heslington, York, United Kingdom on perceptual development emphasises time and again the multi-
[email protected] www.svt.ntnu.no/psy/eatrix.Vereijken
modal character of perceptual events, the multisensory conse-
[email protected]
quences of behaviour, and that there exists no learning or devel-
opment that is strictly within modality (Gibson 1988; 1992; Gibson
Abstract: We show that Stoffregen & Bardy’s arguments against the as-
sumption of separately functioning senses have more historical antecedents
& Walker 1984).
than they give credit for, and that multimodal functioning – primitive in Reviewing empirical work on perceptual development provides
perceptual and brain development – does not require this assumption. compelling evidence that multimodal functioning is primitive in
What is needed is evidence that biological organisms are indeed detecting infancy, illustrating “a kind of unity of the senses in the newborn”
and acting upon information in a multimodal (or global) array. (Thelen & Smith 1994, p. 191). Finding evidence of cross-modal
functioning at such young ages argues against the necessity – or
The fact that the various senses have, since the turn of the cen- even likelihood – of inferential cognitive processing in order
tury, been described and investigated separately hardly reflects a for cross-modal matching to occur, as S&B would have it. By way
“pervasive” underlying assumption that the senses also function of example, newborns match visual with auditory information
independently as Stoffregen & Bardy (S&B) suggest. Further- (Spelke1976), they orient visually towards a heard sound
more, S&B’s attempt to overcome the difficulty by redefining per- (Mendelson & Haith 1976), and both visual and auditory stimula-
ception in a manner that fits their proposition – that is, “as the tion summate to produce neonate arousal (Lewkowicz & Turke-
pick-up of information that exists in irreducible patterns across witz 1980). Furthermore, newborns match oral and visual infor-
different forms of energy” (sect. 1) – is both circular and unper- mation with respect to textured pacifiers (Meltzoff & Borton

1979) and with respect to rigid versus elastic cylinders (Gibson & Abolition of the senses
Walker1984). In contrast, there is evidence that modality-specific
attributes of objects begin to be differentiated only later in infancy Nicholas J. Wade
(e.g., Walker-Andrews & Gibson 1986). The results of these stud- Department of Psychology, University of Dundee, Dundee DD1 4HN,
ies shed light on an issue that S&B raise in the last section of their Scotland. [email protected]
article: Are infants initially sensitive to structure in single-energy www.dundee.ac.uk/psychology/njwade/
arrays, with experience leading to the pick-up of structure in the
global array? Although the developmental work cited here uses Abstract: In advocating an extreme form of specification requiring the
the concepts of multimodal, cross-modal, and intermodal func- abolition of separate senses, Stoffregen & Bardy run the risk of diverting
tioning rather interchangeably, they do attest to development pro- attention from the multisensory integration of perception and action they
wish to champion.
ceeding in the opposite direction, that is from multimodal to
modality-specific functioning. Ontogeny, from Stumpf ’s (1890) Science progresses by building on and then making breaks with
perspective, thus seems to proceed in the opposite direction to the past. The emphasis is placed on the latter in the target article
phylogeny. Following S&B’s own argument, the direction of this by Stoffregen & Bardy (S&B). They stake a bold claim for a single
developmental trend leads to a conclusion opposite to theirs, perceptual system that utilizes global arrays of energy. In order to
namely that “the assumption of separate senses” (sect. 7) is not support their position, they need to dismantle the sensory edifice
necessary for cross-modal functioning to occur. built up over more than two thousand years. This is a tall order:
Multimodal input as the basis for brain development. S&B you have to be very sure of your ground if you are to argue that
make a convincing case that the assumption of separate senses is everybody has got it wrong until now. If the theoretical position
incompatible with physical reality and the notions of specification advocated is considered wanting in regard to singular sensory
and direct perception. However, as argued above, the postulation specification, then the force of the theoretical integration that fol-
of cross- or multimodal functioning does not rest on this assump- lows is irremedially flawed.
tion. Furthermore, with their definition of the global array, the au- The arguments against the separation of the senses address
thors provide an example of how multimodal functioning can ex- three aspects of function – the physics of the stimulus (energy),
ist without requiring inferential processing that would make the structure of the receptor system (anatomy), and the responses
perception mediated rather than direct. Support for the latter no- to stimulation (neurophysiology). S&B state that they have been
tion has been provided by Edelman (1987; 1992) in his concept of unable to find an explicit justification of the assumption of sepa-
reentrant mapping – the anatomical interrelating of several si- rate senses, and then they cite Aristotle’s deliberations, which are
multaneous perceptual and motor representations – which pro- directed specifically to this point. S&B seem to be presenting a
vides a neural mechanism for brain development that has an ex- theory of Aristotle’s “common sensibles” rather than of perception
plicit foundation in multimodal input to the neural network in generally. They do not refute Aristotle’s statements about the spe-
order for further development of the brain and its functions to oc- cial objects of sense.
cur. It is surprising that the authors did not cite the work of the
In Edelman’s theory of neuronal group selection, perceptual “common sense” philosopher, Thomas Reid (1710–1796). He
categorisation is the most fundamental psychological task of de- made the distinction that S&B are trying to sustain – that per-
velopment that forms the basis for further development of human ception is distinct from sensation, and that the former does not in-
cognition and action. As all perceptual events have a multimodal volve cognition. However, Reid does maintain that there are rep-
character and all behaviour has multisensory consequences (cf. resentational stages in perception:
Gibson 1988; 1992), the nervous system is continuously bom-
barded with multisensory stimulation that is temporally corre- Although there is no reasoning in perception, yet there are certain
lated. This time-locked nature of multimodal input, together with means and instruments, which, by the appointment of nature, must in-
the reciprocal activation of neuronal groups by reentrant mapping tervene between the object and our perception of it; and by these our
of motor activity along with sensory information from many perceptions are limited and regulated. First, if the object is not in con-
modalities, lies at the heart of category formation (see also Thelen tact with the organ of sense, there must be some medium which passes
& Smith 1994). In early development of the brain, movement and between them. Thus, in vision, the rays of light; in hearing, the vibra-
sensory signals are completely coupled and act together to form tions of elastic air; in smelling, the effluvia of the body smelt, must pass
the global maps that are the basis of further development. Neu- from the object to the organ; otherwise we have no perception. Second,
ronal groups thus get strengthened through their association in there must be some action or impression upon the organ of sense, ei-
the real-world, which forms the basis for experience-driven per- ther by the immediate application of the object, or by the medium that
ceptual categorisation. In other words, the perfect temporal asso- goes between them. Third, the nerves which go from the brain to the
ciation of multimodal information is the primary link between the organ, must receive some impression by means of that which was made
mind and the world, thereby providing a neural mechanism for upon the organ; and probably, by means of the nerves, some impres-
(the development of ) specification and direct perception. sion must be made upon the brain. Fourth, the impression made upon
In conclusion, in their rejection of the assumption of separate the organ, nerves, and brain, is followed by a sensation. And, last of all,
senses, S&B have, historically, a number of bedfellows; their the- this sensation is followed by the perception of the object. (Reid 1764,
sis has more empirical support than they are prepared to give pp. 424 –25)
credit for, particularly in the field of perceptual development. Advocating the unity of perception and action need not involve the
What remains to be demonstrated, as the authors are fully abolition of the senses.
aware, is the tenability of the second main idea in the target arti- It is difficult, in an historical sense, to imagine originating a clas-
cle, namely, that biological systems are directly sensitive to, and sification of the senses that did not depend on anatomical and per-
make use of, structure in the global array. ceptual distinctions. Such a classification would have preceded
others based on energy because the characteristics of perception
were described long before there was an adequate understanding
of energy sources in the environment. These categories were later
reinforced by evidence from neuroanatomy and neurophysiology:
specialized receptors respond to features of the stimulus and these
are analysed in discrete regions of the brain. Subsequent cortical
and subcortical analysis certainly integrates signals from different
sources, but this does not call for denying the specificity of the
senses. The examples of binaural localization and binocular stere-

opsis functioning “in a unitary manner” are not surprising, but The target article is the paper I almost wish I had written. S&B
they are unlikely to be treated as evidence against separation of present a logical, well-supported argument for specification. They
the senses. In the context of neurophysiology, concern should be point out how many of us have been unable to escape the as-
given to receptor systems and their cortical projections rather than sumption that the senses are separate channels even as we claimed
higher cortical functions which integrate signals from the sensory we were rejecting that accepted wisdom. They clarify for me why
projection areas. The arguments for unitary perception are more it has been so difficult to maintain precision with such terms as
forceful than those for a unitary sense. “intermodal,” “crossmodal,” “amodal,” and “multimodal.” I have
Most detailed examples of perception given in support of S&B’s tried, for example, to reserve “crossmodal” for situations in which
approach understandably derive from their own research, alan observer viewed something and subsequently heard or felt it, a
though they are from a narrow range of visual-vestibular interac- situation that may require the kind of inference that most theories
tions. Moreover, many are artificial in a real sense since they assume.
mostly involve vehicular motion. Any theory that is based on the S&B make additional points that speak to the common fallacy
global array and its use should restrict the phenomena to those that infants will be unable to “process” information when they en-
that occur in the natural environment rather than incorporate di- counter in the lab a stimulus event such as a moving, computer-
mensions that are unique to one species. Since an evolutionary generated disk punctuated by a beep at the lowest point in its tra-
perspective is implied by the authors, they should confine their jectory, flanked by another disk that is not. The usual assumption
analyses of perception and action to those which do not incorpo- is that infants must compare information obtained via vision to
rate artificial devices. that obtained via audition to determine which icon is consistent
The vestibular system provides a telling example of the manner with the sounds, and that this will tax their abilities (Bahrick 1992;
in which our understanding of perception has been advanced Lewkowicz 1993). Infants at a specific age may indeed fail a spe-
by the specification of sense. It was the first increment in two mil- cific intermodal task, but not because they cannot deal with si-
lennia to Aristotle’s five senses. The anatomy of the vestibular multaneous presentations of separate optic and acoustic arrays.
labyrinth was described long before its function was appreciated; Sensitivity to a higher-order pattern is required, not internal com-
between these two events the visual consequences of vestibular parisons of information derived from single-energy arrays.
stimulation were subjected to observation and experiment (see Two aspects of S&B’s paper could be improved. First, although
Wade 2000). Rotating the body to induce vertigo resulted in post- they acknowledge that James Gibson (1966) provided the original
rotary nystagmus and apparent visual motion, the directions of example for information in the global array, they do not describe
which were dependent upon head orientation during rotation. the scope of his contribution. This may represent misinterpreta-
These responses to body rotation could be interpreted when the tion, ambiguities in the theory, or evidence for the growth of
hydrodynamic theory of semicircular canal function was advanced Gibson’s own thinking. Congruent with the latter, Gibson (1979)
in the 1870s. Would such understanding have been achieved with- himself described the theory of information pick-up as in an “un-
out the specification of a separate vestibular sense? It is the case developed state.” But he went on to say: “Information is not
that Stoffregen and Riccio (1988) have denied the link between specific to the banks of photoreceptors, mechanoreceptors, and
vestibular stimulation and perceived orientation, but there are chemoreceptors that lie within the sense organs. Sensations are
those inclined to dispute their claims (see Curthoys & Wade 1990). specific to receptors and thus, normally, to the kinds of stimulus
energy that touch them off. But information is not energy-
specific” (p. 243). He asserted that we directly perceive the qual-
ities of things in the world, especially their affordances. S&B take
up these ideas and propel them much farther along the path Gib-
Infants, too, are global perceivers son had begun to clear.
Second, S&B little attend to developmental research. Consider
Arlene Walker-Andrews
results from Walker-Andrews and Lennon (1985) and Pickens
Department of Psychology, Rutgers University, New Brunswick, NJ 08903.
(1994). In the earlier study, 5-month-olds observed videotapes of
[email protected] www.rci.rutgers.edu/~arlenewa
a Volkswagen (VW) moving toward or away from them accompa-
nied by a noise that increased or decreased in amplitude. Infants
Abstract: Infants are global perceivers. They detect patterns in stimula-
tion that allow detection of many affordances of the environment. Pick-up
looked preferentially at the videotape consistent with the engine
of structural patterns across forms of ambient energy awaits maturation of noise – the approaching VW when the noise grew louder, the re-
sensory systems and improvements in motor skill, but development pro- ceding VW when it softened. Note that the rate of change in am-
ceeds rapidly during the first year. Researchers in perceptual development plitude was correlated with movements of both vehicles, but in-
must devise and refine existing tools to examine infants’ abilities. fants responded to directional information as well. Pickens (1994)
introduced critical refinements – a condition in which a toy train
When I first examined infants’ emotion perception using an in- changed in size only (rather than moving in depth), another in
termodal task (Walker 1982), I was asked why I compared an in- which each film’s brightness varied, and one in which the trains
fant’s looking time to a happy facial expression projected alongside moved up and down. Infants looked appropriately for motion in
a sad facial expression and accompanied by a happy vocal expres- depth and, less so, for size changes. They did not show preferences
sion to that shown in the opposite setup. That is, the comparison related to brightness changes or vertical movement. Five-month-
was with the infant’s looking time to the happy expression when it olds were sensitive to ecological relations specifying approach ver-
was projected alongside the sad expression, but accompanied by sus retreat and did not generalize to intensity or metaphorical re-
a sad vocal expression. The expectation instead had been that I lations.
should use, as baseline, looking at a facial expression when it was Although S&B fail to capitalize on the wealth of infant data, they
presented as one of a silent pair. I have struggled to answer this simultaneously present a challenge to developmental researchers.
question, never able to furnish an argument that convinced al- The authors proffer but dismiss the possibility that infants are ini-
though I was certain of my choice. I have argued that (1) two fa- tially sensitive to structure in single-energy arrays and develop
cial expressions and one soundtrack and (2) two facial expressions sensitivity to the global array because this assertion demands the
– comprise only two very different events. Stoffregen & Bardy corollary that there are separate senses. In fact, infants appear to
(S&B) provide the rationale I have been seeking: the whole is not be sensitive to the global array. They detect some invariant pat-
only greater than but is qualitatively different from the sum of its terns in the first few weeks of life (e.g., Gibson & Walker 1984;
parts. The integrated action of seeing and hearing leads to the per- Meltzoff & Borton 1979). Months later they can detect arbitrary
ception of an “irreducible” product, in the present example an relations such as those contrived between the color and taste of a
emotional expression that affords opportunities for action. substance (Reardon & Bushnell 1988) or a label and a moving ob-

ject (Gogate & Bahrick 1998). There is continued debate about ample of this illusion occurs when an earth-stationary observer is
the role, if any, of modality-specific information during infancy seated inside a rotating optokinetic drum. S&B claim that this is
(c.f. Bahrick & Lickliter 2000) as well as the acknowledgment that not an illusion, because the relative motion between the drum and
although all sensory systems are potentially functional prenatally, the observer is correctly perceived. It is, but that is not the illusion
they mature at different rates. Taking a lead from the present pa- which concerns a different percept, namely, perceiving self-mo-
per, however, those of us who study infants should consider more tion relative to the earth’s surface. S&B seem to believe that this
carefully how we present perceptual problems. It may no longer is not really perceived inside the drum, although experimenters
suffice to construct multimodal displays and test whether infants believe it is. They claim that there is no illusion; only a misunder-
appreciate invariant relations. S&B suggest some alternatives, but standing between observer and experimenter as to the frame of
designing such studies given infants’ limited behavioral reper- reference relevant to the observer’s percept, a misunderstanding
toires will be difficult. which should disappear when the frame of reference is explicitly
In closing, S&B have contributed an important essay in the de- stated in the perceiver’s verbal report.
bate about the nature and origins of perception. They make a con- However, whether S&B like it or not, earth-relative self-motion
vincing case for specification and for an active perceiver, one not really is experienced by observers inside an optokinetic drum: they
hobbled by the imposition of energy-specific stimulation but free believe that they are moving relative to the floor of the experi-
to sample the global array and detect affordances. mental room in which the drum is located (and perceive the drum
as stationary relative to that floor). Since this is not physically the
case, the term illusion is correct.
Contrary to what S&B suggest, researchers in the field of visual-
vestibular interactions and self-motion (including myself ) are al-
Motion, frames of reference, dead horses, ways careful to correctly ascertain the frame of reference in which
and metaphysics subjects report self-motion percepts. They either specifically ask
about it, or use non-verbal methods (e.g., by asking the subjects to
A. H. Wertheim continuously keep a joystick pointed toward where they believe
TNO Human Factors, 3769-ZG, Soesterberg, The Netherlands. the door of the experimental room is located). In fact, these re-
[email protected] searchers were the first to recognize the dangers of verbal ambi-
guities about frames of reference; terms such as “exocentric” and
Abstract: Various annoyingly incorrect statements of Stoffregen & Bardy “egocentric” originated from their work. S&B’s accusation that re-
are corrected, for example, that perception researchers commonly use the searchers “routinely exclude correct responses from their analysis
term “absolute motion” to denote motion without any frame of reference,
confuse earth-relative and gravity-relative motion, err with respect to the
. . . because of verbal ambiguities in their subjects’ reports,” re-
frame of reference implied by their subject is motion responses, believe in veals a shocking lack of knowledge. This is not even kicking a dead
sense specific motion percepts, and do not investigate sensory interactions horse, but kicking a nonexistent one.
at neurophysiological levels. In addition, much of the target article seems Another problem is S&B’s claim that my analysis of percepts of
to concern metaphysics rather than empirical science. “absolute motion” is sense-specific (see also Stoffregen 1994), that
is, requires only one sensory system. This is incorrect. In my
Stoffregen & Bardy (S&B) state that “a common concept . . . model (Wertheim 1994) the retinal coordinates of image motion
among many researchers, including myself, is . . . the idea of ab- are recalibrated into the 3D coordinates of the frame of reference
solute motion” not defined in terms of any frame of reference defined by the earth’s surface and gravity. This is brought about
(S&B use the term referent). But neither I nor anyone else that I with what I termed “reference signals.” These are compound sig-
know has ever embraced such a peculiar idea. The relativity of mo- nals constructed from sensory afferents generated by various sen-
tion has always been my explicit point of departure (Wertheim sory systems (somatosensory, vestibular, and visual). In addition,
1981) and I have argued that a theory must he flawed if it yields a retinal and reference signals themselves have no perceptual
concept of motion that is not definable in terms of a frame of ref- meaning. It is their interaction which yields percepts of motion.
erence (see e.g., my discussion of the “hidden reciprocity as- Consequently, on the perceptual level, one cannot speak of sep-
sumption” in Wertheim 1994, sect. R2). S&B call on researchers arate senses. This is also implied by other inferential theories,
always to mention the particular frame of reference in terms of which use the theoretical forerunners of reference signals (“extra-
which they define motion. But this is what everybody has been do- retinal signals,” “corollary discharges,” “efference copy signals”).
ing all along (see e.g., Swanston & Wade 1988, Wertheim 1994, Hence, no inferential theorist assumes that motion perception is
and many of its accompanying BBS commentaries, also sects. R5 sense-specific (see Wertheim 1999, for a more detailed analysis
and R6). In the literature (e.g., Kinchla 1971; Wertheim 1994, p. of this issue). Nor do vestibular researchers – who, for decades
302) the term “absolute motion” denotes motion defined in terms now, are trying to unravel the way how retinal, somatosensory,
of the three-dimensional (3D) frame of reference dimensioned by and vestibular afferents interact to bring about particular percepts
the earth’s surface and the direction of gravity. Other names might of self-motion – assume that motion perception (of any kind) is
have served just as well: for example, “motion relative to absolute sensory-specific (see e.g., Sauvan 1999). On the contrary, these
space,” “Newtonian motion,” “exocentric motion,” or “earth-rela- sensory interactions are their core business, both on the percep-
tive motion.” This is perfectly in line with Einstein’s claim that the tual and on the neurophysiological level.
idea of frameless motion has no meaning. When S&B defend Ein- Hence, it is not at all surprising that S&B have been unable to
stein’s views vis-à-vis those of perception researchers (see also locate an explicit justification of the assumption of separate senses
Stoffregen 1994), they beat a dead horse. in the philosophical, behavioral, or neurophysiological literatures.
The same can be said about S&B’s elaborate argument that Who would want to justify a false assumption? S&B’s call to search
there can be motion relative to the earth without it being relative “for neural units that respond to patterns of activity that extend
to the direction of gravity (i.e., when perpendicular to the direc- across different kinds of receptors, such as the retinae and the
tion of gravity). To my knowledge nobody has ever equated earth- vestibule,” again is out of touch with the literature. Here too the
relative motion with motion relative to the earth’s gravity. authors beat a dead horse.
S&B also criticize many researchers, including me, for stating Finally, it is difficult to make sense of S&B’s discussion of vari-
that the perception of visually induced self-motion is often illu- ous possible relations between an energy array and (aspects of) re-
sory. What those authors mean is that in the presence of a large ality – a relation which is “prior to and independent of . . . psy-
optic flow field, one often experiences a perception of self-motion chological processes.” The point is that, reality per se is unknown.
relative to the earth’s surface, while, physically speaking, one re- This is metaphysics, not empirical science. Reality can only be as-
mains stationary relative to that surface. The most common ex- sumed: we assume that what we perceive is reality. For all practi-

Response/Stoffregen & Bardy: On specification and the senses
cal purposes, the array itself is reality (or as close as one can get to Commentators who appeal to human perception or to
it). What else can S&B’s mean when they say that the array is “not the nervous system do not appear to be moving toward a
a hypothesis but a fact”? For empirical ecological psychology, an theory of perception and action that is general across
array’s relation to reality (whatever this means) is irrelevant. species (contra Wade, we seek a theory of perception and
Instead, it takes the concept of an energy array as its point of action that is general across species). For example, Ahrens
departure, and then tries to investigate what sort of information
(e.g., motions, frames of reference, etc.) can be recognized in its
defines sensation as the “transduction of ambient energy
structure and how it might be picked up by a perceiver. For in- into a spatiotemporal pattern of neural activity,” which
ferential theorists, who do not shun cognitive evaluations of in- seems to imply that sensation is limited to species that have
coming sensory signals, the issue is different: How do percepts of neurons. This would seem to require that we develop sep-
the various kinds of object- or self-motion arise from the neuro- arate theories to explain perception in species that have
physiological and cognitive processes that operate in association neurons and those that do not. This is problematic in the
with our sensory systems? As I have shown (Wertheim 1994), the simple sense of not being general. It also raises the ques-
answers offered by these two approaches are not mutually exclu- tion of the evolutionary development of perception, since
sive, for they refer to different percepts: relative motion between species with nervous systems evolved from predecessors
objects may be perceived from retinal information only, but per- that did not have nervous systems (Reed 1996). Similarly,
cepts of “absolute motion” (as defined above) require non-retinal
information as well. To describe these kinds of information in
Lewkowicz & Scheier assert that “no behavior is possible
terms of many energy arrays or a global multidimensional one is without the brain.” While this may be true for humans and
merely an exercise in semantics. other chordates, it is not true for behavior in general (Berg
2000; Pittenger & Dent 1988). The development of a cen-
tral nervous system is one way in which living things have
achieved adaptive control over the animal-environment in-
teraction, but it is not the only way and, most importantly,
it is not a sine qua non for the adaptive success of species
Authors’ Response (Reed 1996). Adaptive behavior requires perception and
control of the animal-environment interaction. Given this,
and given the fact that many successful species do not have
central nervous systems, a theory of perception and action
Specification in the global array that is general must be based on principles that are not lim-
ited to the properties or activity of any particular neural ap-
Thomas A. Stoffregena and Benoît G. Bardyb paratus (Gibson 1966; Reed 1996).
aDepartment of Psychology, University of Cincinnati, Cincinnati, OH 45221; It is important also to consider the fact that the nervous
bDivisionof Sport Sciences (STAPS), Université Paris Sud-XI, Orsay Cedex, system is an effect of behavior, at least as much as it is a
France. [email protected] [email protected] cause (Berthoz 1997; Pailhous et al.). This is true at the
Abstract: We discuss issues raised by the commentators, such as ontogenetic level: As Reed (1966, p.69) argued, “it is not the
specification in single-energy arrays, task-specific pickup of infor- animal’s brain that organized its world, but the evolutionary
mation, general principles of the ecological approach to percep- ecology of the animal that organized its brain.” This is also
tion and action, and how specification may be constrained by the true at the phylogenetic level, that is, in terms of individu-
facts of physical relativity. While the commentaries raise many im- als, as is illustrated in the examples provided by Kujala and
portant issues we conclude that they do not undermine our argu- Pailhous et al.1 We do not suggest that the nervous system
ment that specification exists solely in the global array. is wholly plastic; that this is not true is again shown by Ku-
jala. However, together with Pailhous et al., we reject the
very widespread assumption that the nervous system is pri-
R1. A general theory of perception and action mary. Perception and action (including adaptive learning)
occur in species that have very different nervous systems
We seek a theory of perception and action that is internally from ours and, indeed, in species that have no nervous sys-
consistent and general. General, in the sense that it applies tem at all (Berg 2000; Pittenger & Dent 1988; Reed 1966).
to all perception and action, across situations, across indi- For this reason, a general theory of perception and action
viduals, and across species. Internally consistent, in the cannot be rooted in the nervous system.
sense that concepts used to explain one aspect of percep- Several commentators (e.g., Kujala; Vereijken & Whit-
tion and action must be compatible with concepts used to ing) suggest that existing neurophysiological research sup-
explain other aspects (e.g., Shaw et al. 1982). ports our arguments about the global array. We agree that
Although many of the commentators work toward gen- such research may be consistent with the pickup of infor-
eral theories of behavior, some seek to understand percep- mation from the global array. However, existing neuro-
tion and action primarily in humans (e.g., Ahrens; Pallas; physiological research is not likely to be suitable for theory
Wade), while others focus their explanations of perception testing with respect to the global array. Much of the exist-
in “natural” circumstances (Runeson et al.). Perception ing neurophysiological research is consistent with a wide
and action are known to be adaptive in a great variety of variety of divergent theoretical positions (as pointed out by
species that exhibit an astonishing diversity of perceptual, Mergner & Becker). An example is the research on blind
motor, and neural anatomy, and in a great variety of situa- people cited by Kujala. While this work impressively illus-
tions, such as teleoperation (McMichael & Bingham) and trates the flexibility of the nervous system and the role of
orbital flight, which do not seem to fit any credible defini- experience in CNS organization (Pailhous et al.; Ver-
tion of natural. It is our belief (cf. Gibson 1966; Shaw et al. eijken & Whiting), it provides no information about the
1982), that a single theory of perception and action can and parameters of sensory stimulation to which the nervous sys-
should apply to all of these. tem is sensitive. There is, however, existing neurophysio-

logical and neuroethological research that supports the compatible with either position (e.g., Bardy et al. 1999; Jeka
general premises of the ecological approach to perception et al. 1998; Kelso 1995).
and action, adopting an a priori decision to focus on natural Researchers have sometimes suggested that it may be
(i.e., nonlaboratory) perception-action in response to natural possible to integrate ecological and Helmholtzian theories
(i.e., nonreductionist) stimuli. This research has demon- of perception (e.g., Mergner & Becker; Wertheim 1994;
strated the existence of neurons (for example) that react to cf. Oie & Jeka; Pickering). We disagree, and believe,
high-order, low-dimensional patterns in ambient energy along with James Gibson (1966), that the basic premises of
(e.g., Camhi 1984; Saito et al. 1986; Tanaka & Saito 1989). the two theories are not reconcilable. The ecological ap-
This research has been conducted in the context of single- proach to perception and action is incompatible with in-
energy arrays. As stated briefly in the target article (sect. ferential (i.e., Helmholtzian) approaches. A major reason
6.2.4) and in several commentaries (Wade; Kujala, and for this is that specification and non-specification motivate
others), new research of this kind is needed that is specifi- separate and incommensurate theories of perception. If
cally designed to permit empirical contrasts between neu- specification exists, then there is not a clear theoretical mo-
rophysiological sensitivity to structures in the global array, tivation for the existence of inferential processes (e.g., as-
as opposed to sensitivity to structures in single-energy ar- sociation) in mentation (Myin). Students of neurophysiol-
rays. ogy often assume that perception begins when energy is
transduced at receptors. That this assumption is not only
controversial but flatly unnecessary has been argued by
R2. Ecological psychology: General issues ecological psychologists for decades (e.g., E. J. Gibson 1969;
J. J. Gibson 1966; Michaels & Carello 1981; Reed 1996).
Several commentators raise questions that are relevant to The division of information pickup into sensation and
the ecological approach to perception and action, in general, perception, which is often taken for granted (e.g., Ahrens),
rather than to our analysis of the global array, in particular is in fact a controversial assumption (Gibson 1966; Reed
(e.g., Ahrens; Feldman & Lestienne; Oie & Jeka; Pal- 1996). The fact that stimulation of receptor surfaces gives
las). As noted in section 1 of the target article, the ecologi- rise to neural activity does not necessarily mean that the ac-
cal approach is an established theory with a well-developed tivity is “entirely mechanistic,” or that “no meaning is con-
set of principles (see e.g., Goldfield 1995; Michaels & Carello veyed” (Ahrens). The existence and operation of the ner-
1981). These often differ in fundamental ways from the vous system does not imply that neurological function and
principles that guide more traditional theories of percep- mentation are mechanistic. Similarly, the ecological ap-
tion. proach to perception and action explicitly rejects the as-
A common misconception is that the ecological approach sumption that knowledge about reality is calculated (Pal-
rejects the possibility that perception may be influenced by las).
or dependent upon mental activity. Internal, experience- Neelon & Jenison review a common argument that the
dependent influences on perception and action are an es- existence of specification does not explain “the achieve-
sential fact of life, even in single-celled animals (Pittenger ment of perception,” and we agree. Specification (in the
& Dent 1988), and any theory of perception and action global array or elsewhere) is not a theory of perception but,
must take these into account at a fundamental level (contra rather, a fact that has implications for theories of perception
Pickering). The ecological approach does this, mainly (e.g., Runeson & Frykholm 1983). As we noted in section
through the theory of differentiation (Gibson 1969; 1988). 6.2, if specification exists, then there is no theoretical moti-
The question is not whether mental activity exists, or whe- vation for invoking “statistical estimation” in order to per-
ther mental activity is essential (Oie & Jeka). The question ceive the world. Similarly, if specification exists in the global
is about the nature of mental activity, and this is intimately array, then there is no theoretical motivation for combining
related to the issue of specificity. If specificity does not ex- “multiple inputs.”
ist, that is, if potential sensory stimulation is ambiguous with In the target article, we considered the possibilities that
respect to reality, then mental activity must be inferential specification exists always, or never. Oie & Jeka and Pick-
(e.g., associative). However, if specification exists then ering (see also Michaels & Oudejans; Runeson et al.)
mental activity need not be inferential, that is, it may not be take a contrasting position, suggesting that specification
necessary to “process” information in order that perception may exist in some circumstances, but that it is not a gen-
be accurate. This is the essential distinction. We agree with eral phenomenon (what Cabe refers to as “non-absolutist”
Oie & Jeka that behavior is not always “dominated” by per- specification). This may seem tempting, but it is not with-
ceptual information, but this is not directly relevant to the out problems. As we noted in the target article (sect. 1), the
issue of whether reality is specified; the existence of speci- question of specification in ambient arrays is not a psycho-
fication is not a psychological issue (sect. 1), and cannot be logical question but, rather, a question of ecological physics.
confirmed or rejected on the basis of behavioral experi- Ecological physics is a branch of physics that deals with
ments. Similarly, specification cannot exist or be created in phenomena on the scale of, and with reference to, actual or
neural activity (Mergner & Becker; Peper & Beek); this potential perceiver-actors. With regard to specification, the
would be equivalent to indirect perception on the basis of question is whether there is a unique relation between as-
ambiguous stimuli. pects of reality and the structure of ambient arrays, as a con-
Dynamical systems theory is useful for describing per- sequence of the physical laws that govern the generation,
ception-action phenomena, but it is not helpful in resolving propagation, and reflection of energy. Because ambient ar-
the theoretical debate between direct and indirect per- rays are part of physics, the hypothesis that ambient arrays
ception (cf. Ahrens; Oie & Jeka). This is because dynam- are “partially,” or “occasionally” specific to reality is equiv-
ical systems theory is neutral with respect to the directness alent to a claim that the laws of the generation, propagation,
or indirectness of perception; dynamical descriptions are and reflection of ambient energy are sometimes in effect,

but at other times not in effect. More generally, the hy- feature of films is an instantaneous cut between images
pothesis of partial or occasional specification requires some recorded by cameras in different positions, or by a single
principle that can predict when specification exists, and camera at different times (Anderson 1996). A cut produces
when it does not. We know of no principled basis for such an instantaneous change in the optic array. This type of
a claim; Pickering and Oie & Jeka offer none, and do not change is physically impossible outside the cinema, due to
cite any. Note that our position does not require that every- the fact that living things have inertia, and so cannot achieve
thing be specified at every instant, or at every point of ob- an instantaneous shift from one point of observation to an-
servation. The consequences of this for perception are dis- other (Stoffregen 1997). In an evolutionary sense, cuts in
cussed in section R6. film must be grossly unnatural. Yet for millions of contem-
porary humans, optical shifts of this kind are a common-
place of daily life, and have been essentially from birth, in
R3. Specification in the global array film, television, home video, computer graphics, and so on.
For individuals, cuts in film can be classified as unnatural
With respect to specification, the thread of our argument only if natural can exclude typical, or ordinary. We know of
has been nicely summarized (with one exception) by Cabe: no basis for such an exclusion. More generally, Runeson et
Stoffregen & Bardy discredit all existing perceptual theories, al.’s argument is viable only if they can develop a rigorous
because each allows input conflict (input ambiguity) and there- definition of natural.2
fore implies cognitive (indirect) processes. The syllogism is: If Runeson et al. (see also Costall et al.; Michaels & Beck
array structures (AS) conflict, then perceivers must choose be-
1995, p. 274), suggest that virtual reality devices, moving
tween alternative world structure (WS) interpretations; choice
implies cognitive processing; therefore, input conflict implies rooms, rotating drums, and other such devices cause the
cognitive processing. The main focus is intersensory conflict, optic array to “lose some of its specificity,”3 and suggest that
but any AS ambiguity yields the same conclusion. Because all in such devices perception is illusory or erroneous (this ar-
existing perceptual theories entail such ambiguities, they all en- gument appears to apply to the cinema, as well). Each of
tail cognitive processes. these assertions can be questioned. We have argued that in
We endorse this summary, with the exception of Cabe’s these devices reality is specified (i.e., the nature of the de-
reference to cognitive processes. As we will argue below, vices as simulators, and the content of the simulation) and
the ecological approach to perception and action does not that this specification exists in the global array (Stoffregen
reject the reality of cognitive activity. Rather, the ecological et al. 2000b). We have also argued that this information is
approach rejects the assumption that cognitive activity picked up; this would account for the ability of users of so-
must be associative, inferential, or computational. This as- phisticated flight simulators (for example) to differentiate
sumption is based, both logically and historically, on the the simulation from the actual vehicle (Stoffregen et al.
prior assumption that in Cabe’s terms, there is ambiguity 2000a). We would also argue that success at teleoperation
between world structures and array structures. If, as we ar- (McMichael & Bingham) is possible in part because the
gued (sect. 6.2.1), the animal-environment system is speci- fact of teleoperation is specified in the global array. Differ-
fied by structures in the global array, then there would be entiation of the fact of teleoperation (cf. sect. R6) would
no theoretical motivation for postulating that mental activ- allow the user simultaneously to control different parts of
ity is inferential, associative, or computational. the body relative to the distal teleoperation environment
and the local gravito-inertial environment (cf. Patterson et
al. 1997; Smith et al. 1997). Are people in moving rooms,
R4. Constraints on specification patrons of the cinema, and users of virtual reality devices
really fooled? Can they distinguish motion relative to the il-
Runeson et al. claim that specification may exist in single- luminated environment from motion relative to the gravito-
energy arrays, basing their argument on the idea that spec- inertial force environment? In many cases the relevant data
ification occurs under the influence of different types of have not yet been collected (Stoffregen 1997; Stoffregen et
constraints. We endorse the general concept of constraints, al. 2000a).
and agree fully that they are critical to specification. How-
ever, their argument ignores our discussion of the indepen-
dence of physical referents (sect. 4). The independence of R5. Constraints on the detection of information
different physical referents is a profound constraint on
specification. Other types of constraints must be posterior Several commentators (e.g., Brenner & Smeets; Coello
to this, as we argued in section 5. We assume that any evil & Rossetti; Foo & Kelso; Runeson et al.), point out that
genius must obey the laws of physics; if not, then the “laws” perception is selective, and that ambient arrays are sampled
are not laws, and we must either re-open the search for the for information that is relevant to particular perceptual-
actual laws of physics, or abandon the idea that physics is motor goals. We cannot agree more and, as Myin notes, we
lawful. Runeson et al. ask about nonlinear propagation of made several references to this in the target article. Our
light, as if this were hypothetical, when it is known as a fact: own research has emphasized the task or goal-dependent
The bending of light by strong gravitational fields is a ma- nature of perception and action (e.g., Bardy et al. 1999;
jor confirmation of Einstein’s theory of relativity. Never- Marin et al. 1999; Stoffregen et al. 1999; 2000b). However,
theless, light, like an evil genius, obeys the laws of physics. the pickup of information is posterior to the existence of in-
Runeson et al. (cf. Shaw et al. 1982, p. 218) place strong formation. As noted in the target article (sect. 1), our argu-
emphasis on “natural” constraints. We regard this as a prob- ment about the global array is an argument about the in-
lem, because of the difficulty of defining natural. As one ex- formation that is available for perception. It is not an
ample, consider cinema. In the movies, recorded samples argument about the psychology of perception (e.g., when
of the optic array are reproduced on a screen. One common information is picked up by perceivers, or the basis for the

selective pickup of information). The ecological approach time, rather that at any given instant or moment (Adolph
of perception and action has already provided, in varying et al.). Instantaneous or arbitrarily brief structures in am-
degrees of detail, explanations and theories of information bient arrays may be ambiguous with respect to particular
pickup, including the fact that perception is selective and aspects of the animal-environment interaction (Riccio et
that selection is based on task-specific criteria (e.g., Gibson al.; cf. Runeson 1988). Similarly, not everything is specified
1988). These explanations can be applied to the pickup of to any given point of observation. One reason that percep-
information from the global array. tion is active is that activity on the part of the perceiver is
Classically in the ecological approach, what is specified often necessary in order to generate the information that
and what is picked up are two different questions, and the specifies task-relevant aspects of the animal-environment
second cannot be asked before the first is answered. Simi- interaction (e.g., Schmuckler & Tsang-Tong 2000). Second,
larly, what is picked up and how it is picked up are different information pickup takes place in spacetime, and both time
questions, but again, the first should be asked before the and movement are required (e.g., Gibson et al. 1987; Oude-
second. Our contribution deals primarily with the “what”: jans et al. 1996; Pittenger & Dent 1988; Riccio 1993; Riley
what is specified, and thus, what is picked up. et al, 1997; cf. Mark et al. 1990). Thus, the shape of the back
We also agree with Coello & Rossetti that perception of an object may not be specified in the global array that is
and action are inherently related, and that people (and available to an observer standing in front of it. Similarly, the
other animals) perceive meaningful properties of the animal- dynamics of an automobile may not be specified when it is
environment system (i.e., affordances, Stoffregen 2000a; not in operation, regardless of the point of observation
2000b, cf. Michaels & Oudejans; Walker-Andrews). adopted by an observer (this is why potential purchasers in-
The failure to perceive accurately arbitrarily chosen prop- sist on test-driving a car, rather than merely examining it in
erties of the physical world, such as velocity (Runeson the showroom). Perceptual-motor error may result from a
1974), or brightness (Oie & Jeka), may result not from any failure to pick up accurate information that is available, or
absence of specification per se, but from the experimenter’s from a failure (or inability) to engage in exploratory actions
assumption that these are proper objects of perception (cf. that will generate or reveal the relevant information (Ric-
Shaw et al. 1982). cio et al.). These examples are consistent with the well-
This brings us to the issue of what is specified. We can documented fact that the accuracy of perception is greatly
detect only the information that is available, that is, the in- improved when observers are permitted to engage in ex-
formation that exists. Thus, ecological theory predicts that ploratory activity. For example, circular vection (the expe-
we can perceive only those things that are specified. Those rience of earth-relative egorotation when presented with a
things will be specified which structure ambient energy ar- rotating optical display) is a frail phenomenon that is de-
rays in lawful ways. It may not be the case that this is true pendent upon restraint of the head (cf. sect. R7). Similarly,
of all possible states of the universe (e.g., Runeson 1974; illusory experiences associated with the Ames distorted
Shaw et al. 1982). This point leads to predictions for per- room depend almost entirely upon severe restrictions of
ception. It should be impossible to perceive directly things perceptual exploration (Runeson 1988).
that are not specified. We argued that the global array is The most general examples are provided by Adolph et
structured by aspects of the animal-environment system al., who describe relations between perceptual-motor error
(sect. 6.1). This statement was deliberately vague, in part and the growth of differentiation. Their examples are gen-
because we regard the issue as being beyond the scope of eral in at least two senses. First, the situations, percep-
the target article, and in part because it has been addressed tions, and actions involved are all characteristic of daily life
elsewhere (e.g., Shaw et al. 1982; Stoffregen 2000a; 2000b). outside the laboratory. Second, every able-bodied person
We agree, however, that the issue of what is specified is learns to perceive and control locomotion across a variety
of fundamental importance (McMichaels & Oudejans; of surfaces (as do the young of many other species). Adolph
Riccio et al.). et al. also underline the importance of exploratory behavior
(see also the classic study of Held & Hein 1963). How
would the perception and control of locomotion develop if
R6. Learning, exploration, and infants were restrained?
perceptual-motor errors Perception and action need not be perfect, as noted by
Cabe, and by Brenner & Smeets (cf. Pickering and Ri-
It is noted by Cabe (see also Pickering and Riley), that ley). In a Darwinian context, it is necessary only that per-
some error in perception and action is tolerable. Errors in ception and action be accurate enough to ensure survival.
perception and action exist at all developmental levels We agree with this, but it is not directly relevant to the is-
(Adolph et al.). But the existence of error does not imply sue of specification. Specification is part of the environ-
a lack of specificity in ambient arrays (contra Pickering, ment in which life evolved. Life came into existence in the
Riley, and Thomas, inadequate detection does not imply context of certain pre-existing aspects of reality, such as
inadequate specificity), and it does not imply what Cabe rocks, water, and the laws of physics that govern the gener-
refers to as “non-absolutist specification.” As we noted in ation, propagation, and absorption of ambient energy. The
section 7, errors in perception and action can (and do) arise existence of specification does not require that perception
from failures in the pickup of information. An essential and action be flawless. Specification guarantees the avail-
requirement in perceptual motor learning is the differ- ability of accurate information, but does not guarantee that
entiation of information that is task-relevant from informa- the information will be picked up or that, if picked up, it will
tion that is irrelevant to the task at hand (e.g., Foo & Kelso; be used accurately. The example of the “bent stick illusion”
Leighty et al.). For two reasons, this differentiation takes proposed by Riley is appropriate. In contradiction with his
time. interpretation of the illusion, we do not believe that the
First, and most important, specification exists in space- light reaching the eye contains “inadequate” information.

This is because information, as we defined it (i.e., as an ob- these facts of physics, and by his confidence that “no one
jective property of the animal-environment system) cannot has ever equated earth relative motion with motion relative
be inadequate. As acknowledged by Riley, the light reach- to the earth’s gravity.” However, he did not respond to our
ing the observer’s eye contains information about a stick be- quotations from behavioral scientists that appear to reflect
ing in water. In this context, what is inadequate is not the just this confusion. Dichgans and Brandt (1978, p.758)
information, but the detection of the information (e.g., equated “orientation with respect to . . . gravity,” with “po-
Michaels & Carello 1981). sition of object and the observer on the earth surface,” and
Wertheim himself defined absolute motion as “motion rel-
ative to external space [i.e., 3D, Newtonian space, as de-
R7. Relative motion fined by the horizontal surface of the earth and its gravita-
tional field” (Wertheim 1994, p. 302)]. Moreover, in his
A large portion of the target article concerns relative mo- commentary he defines a reference frame “dimensioned by
tion; indeed, it is the fulcrum of our argument against the the earth’s surface and the direction of gravity.” We are at a
existence of specification in single-energy arrays (sects. 4 loss as to how Wertheim’s statements can be reconciled.
and 5). For this reason, we were surprised that motion rel- As a matter of empirical practice, experimenters rarely
ative to different physical referents was not addressed by define for the experimental subject the frame of reference
commentators who argue that specification exists in single- to be used in making judgments about motion (e.g., Brandt
energy arrays (e.g., Michaels & Oudejans; Peper & et al. 1973; Ohmi et al. 1987; Wong & Frost 1978). For ex-
Beek; Rosenblum & Gordon; Runeson et al.). Thus, ample, in Wertheim (1981, p. 102), subjects were asked to
our argument for the absence of specification in single- rotate a potentiometer to indicate that they perceived a
energy arrays remains unanswered. The formulation of a stimulus to be moving or stationary, but there is no indica-
response to our arguments about relative motion is a seri- tion that subjects were given any instructions as to the ref-
ous challenge for scientists who argue that specification ex- erents for these judgments (i.e., moving and stationary rel-
ists in single-energy arrays. In particular, it would be nec- ative to what?).
essary to indicate how structures in, for example, the optic Wertheim points out that when exposed to optical flow
array, could provide information sufficient for the control rotating around an axis parallel to earth gravity, experimen-
of action relative to referents that do not structure that ar- tal participants experience an illusion that they are rotating
ray (e.g., motion relative to the acoustic environment, or relative to the earth. We acknowledge the existence of this
relative to the direction of balance). illusion. However, our interpretation of it differs from
Among all the commentaries only one is concerned with Wertheim’s. He treats it as a basic perceptual phenomenon,
issues relating to relative motion, and for this reason and used it as a building block for a general theory of the
Wertheim’s contribution is especially welcome. We ac- perception of object motion (Wertheim 1994). In our view
knowledge that many researchers, including Wertheim, try (cf. sect. 7), this illusion is a superficial experience that re-
to take into account some of the facts of relativity sults from the participant’s inability to engage in normal per-
(Wertheim cites some of the same ones that we cited in the ceptual-motor exploration (that is, to engage in movements
target article). However, in terms of the information avail- that will create the relevant specificational structures in the
able for perception, these researchers generally consider global array). The illusion is possible because of a highly con-
motion relative to a single class of referents, the hypothet- ditional ambiguity in potential sensory stimulation (cf. sect.
ical sensory reference frames (e.g., Soechting & Flanders R6). As we indicated in the target article (sect. 4; cf. Dich-
1992; Wade & Swanston 1991; Wertheim 1994). For exam- gans & Brandt 1978; Stoffregen & Riccio 1988), potential
ple, Wertheim (1981, p. 106) concludes that motion is sensory stimulation is identical during constant velocity mo-
perceived “relative to the magnitude of . . . extraretinal sig- tion of the self relative to the gravito-inertial force environ-
nals,” and he distinguishes between the “perception of moment and relative to the illuminated environment. For this
tion” relative to sensory reference frames and the “physical reason, these situations are fundamentally indistinguish-
determination of motion of an object” (p. 107). Wertheim able. However, potential sensory stimulation during motion
(1981; 1994), does not discuss or consider the possibility relative to these two referents is not the same when there is
that motion might be perceived directly relative to physical acceleration. This explains the fact that the illusion of bod-
referents. To be sure, motion can be measured relative to ily rotation relative to the earth is robust only when acceler-
the retina, the eardrum, and so on. However, as we noted ation is prevented, for example, when the head is subjected
in the target article, if specification exists, then sensory ref- to passive restraint (e.g., Brandt et al. 1973; Ohmi et al. 1987;
erence frames can be irrelevant to perception of motion rel- Wertheim 1987). The illusion is eliminated by head move-
ative to physical referents. This is because specification ments, and is often suppressed even by eye movements (e.g.,
would allow motion relative to physical referents to be per- DiZio & Lackner 1986). Similarly, it is largely because sub-
ceived directly, so that sensory reference frames would be jects in moving rooms are not restrained that we suggested
unnecessary. This reprises James Gibson’s (1966) argument that their perceptions may be veridical (sect. 5.2).
that sensations, while real, are irrelevant to perception (see In the target article we argued for the independence of
Mace, for an historical treatment). different physical referents for motion, and we proposed
In the target article (sect. 4.2) we pointed out that the that different aspects of behavior might be perceived and
surface of the earth and the earth’s gravitational field are controlled simultaneously, relative to different physical ref-
distinct physical referents for motion and that, conse- erents. Recent research has demonstrated simultaneous
quently, it is possible to move relative to one while being differential control of the head and body with respect to in-
stationary relative to the other. For this reason, we argued, dependent, referents (Patterson et al. 1997; Smith et al.
motion relative to these two referents cannot be equated. 1997). During simulated visual flight maneuvers (i.e., turns
We are encouraged that Wertheim appears to appreciate when the aircraft was controlled by looking out the win-

dows), pilots tilted their heads (relative to the torso), so that vide information which is not available in the structure of the
the head remained aligned with the horizon, while simulta- constituent single-energy arrays. One of these (Eq. 1) was
neously they controlled the torso so that it remained aligned taken from the work of Peper et al. (1994), while another
with the aircraft as the latter rotated relative to the horizon. (Eq. 2) was taken from the work of Bingham and Stassen
In each case, the referent chosen for the control of orien- (1994). In arguing for the existence of specification in sin-
tation was task-specific: Aligning the head relative to the gle-energy arrays, neither Peper & Beek nor McMichael
horizon made it easier to monitor aircraft altitude relative & Bingham refer to these examples, which nevertheless ap-
to the ground, while aligning the body relative to the air- pear to support our position.
craft minimized the effort required to control bodily orien-
tation. Changes in the task yielded changes in the referents
for control (cf. Coello & Rossetti; Foo & Kelso; Rune- R9. Falsifiability of specification and ambiguity
son et al.): Under instrument flight conditions, pilots
maintained head alignment with the vertical axis of the air- Cabe (see also Runeson et al.) raises the question of
craft (i.e., with the vertical axis of the instruments that they whether specification is a falsifiable concept. At the level of
were reading). individual parameters the concept of specification is not fal-
sifiable; it is always possible to argue that some other,
unidentified parameter has the specificational relation.
R8. Specification in single-energy arrays Runeson (1988) made essentially this argument. Similarly,
any claim of specification can be voided by a single excep-
Several commentators (mainly, students of the ecological tion (for instance, the claim of Burton, and Costall et al.,
approach to perception and action) argue for the existence that pheromones are specific to particular insects is voided
of specification in single-energy arrays (e.g., McMichael by the fact that pheromones can be presented in the ab-
& Bingham; Michaels & Oudejans; Peper & Beek; sence of the insects, as in pesticides or perfumes). This ar-
Rosenblum & Gordon; Runeson et al.). James Gibson gument applies to any version of the concept of specifica-
(1966) endorsed two positions, as noted in our sections tion; it is not peculiar to the possibility of specification in
3.3.2 and 6.1. On the one hand, he argued for amodal spec- the global array. However, the disproving of an instance
ification, while on the other, he described a structure in the does not disprove the general concept, because a thing that
global array. We have argued that these positions are mu- is not specified by one parameter might be specified by
tually exclusive. Ecological theory must either accept our some other one. When one candidate fails, it may be in-
conclusion, or explain how the two positions presented by cumbent for researchers to look for another candidate,
Gibson can be reconciled. The ecologically-oriented com- rather than to conclude that specification does not exist.
mentators did not directly address this (though Runeson The problem with this is that no matter how many candi-
el al. appear to reject Gibson’s claim that somatosensory date parameters are falsified, it is always possible that some
stimulation, taken in isolation, is ambiguous with respect to other, as yet unknown candidate will do the job (Runeson
body sway). 1988). This is similar to the problem, in inferential theories,
In addition, these commentators (together with Thomas) with the claim that perception and/or cognition are based
present arguments that do not acknowledge or take into ac- on mental calculations or any other inferential process.
count the fact of the constant, simultaneous stimulation of Whenever it is demonstrated that a given calculation or in-
multiple perceptual systems (sect. 1). Much of this multi- ference is either unnecessary or counterproductive (e.g.,
modal stimulation is critical for the perception and control Mark), it is always possible to claim that, rather than being
of the body and its parts (e.g., head, hand). This stimulation direct, some other inferential calculation is being executed.
is not ignored, that is, it is easy to demonstrate that it influ- We acknowledge that, at this level, this is a problem for the
ences the control of the body (e.g., Berthoz 1993; 1997). concept of specification and, thus, for any theory of direct
One implication of this is that any given parameter existing perception. However, scientists who do not accept direct
within a single type of ambient energy is never either the perception theories should be equally rigorous in acknowl-
sole stimulus, or the sole information that is picked up. If edging that the same problem applies to the falsifyability of
specification did exist in single-energy arrays (a point which theories of indirect perception.
we continue to dispute), this would still leave open the is- While specification and ambiguity appear to be unfalsifi-
sue of how the animal deals with the fact of constant, si- able at the level of individual parameters of ambient energy,
multaneous stimulation of (and pickup of information we believe that the issue of falsifiability extends beyond this
from) multiple perceptual systems. We have argued that level. The concept of specification may be falsifiable at the
this can be accommodated in an ecological theory only larger level of general principles. If the laws of physics are
within the concept of the global array (i.e., cooperative general and are in fact laws, then it may be possible to ar-
pickup of global structures); anything else implies internal, gue that specification, which is a consequence of these laws
inferential “integration,” or comparison (e.g., Feldman & is as real and valid as the laws themselves. Specification,
Lestienne). Defenders of specification in single-energy ar- then, would be falsifiable in the same sense that laws of
rays did not address this issue. physics are falsifiable. The concepts of inferential calcula-
Generally, commentators who defend the hypothesis of tion, association, or other processing are wholly psycholog-
specification within single-energy arrays do not address our ical, making no appeal to physical reality and thus may be
arguments about the assumption of separate senses (e.g., fundamentally resistant to falsification.
Runeson et al.; Neelon & Jenison), that is, they offered Several commentators (Brenner & Smeets; Michaels
no argument why this assumption should be retained & Oudejans; Runeson et al.; Riley; see also Michaels &
(cf. Thomas). In the target article, we provided mathe- Beek 1995) suggest that some parameters of potential sen-
matical examples of structures in the global array that pro- sory stimulation are partially or totally nonspecific to real-

ity. With respect to the global array, we disagree. We accept An analogy can be made to the difference between geo-
the claim that a parameter may not specify the particular as- metric figures and the lines that they comprise. A triangle,
pects of reality that concern the authors, but we point out for example, comprises three lines, but a triangle is more
that the same parameters may specify something else, that than three lines; it is three lines forming a closed figure. In
is, some other aspects of reality. If this is the case, then it our analogy, each line can represent the stimulation avail-
may be that all parameters of the global array are specific able to an individual perceptual system. If specifications ex-
to some aspects of reality. While any given parameter A may ists only in single-energy arrays, then perception of the tri-
not be specific to any particular aspect of reality X, it may angle, as such, would require the separate perception of
well be specific to some other aspect of reality, Y (Michaels each line, followed by some form of internal process that
& Oudejans). This leads us to predict that subjects will would “recover” the fact of triangularity (that is, permit the
perceive Y. This could be a problem if the experimenter as- perceiver to differentiate “three lines forming a triangle,”
sesses only the perception of X; when X is not perceived, from “three parallel lines”). Amodal redundancy would oc-
the experimenter may (incorrectly) conclude that percep- cur when the three lines were of equal length (the result-
tion was erroneous and, hence, that A did not specify any- ing figure would be an equilateral triangle). By contrast, the
thing. This possibility reinforces our recommendation that pickup of information from the global array would be akin
experimenters be cautious in evaluating perceptual reports to perception of the triangle as such, that is, perception of
as being “right,” or “wrong” (sects. 5.2 and 6.2.6). a three-sided closed figure. Perception of this kind would
not require that triangularity be “built up” through the in-
tegration of separately perceived lines. Moreover, equilat-
R10. Amodal specification? eral triangles are not “special” in the sense that amodal in-
formation is thought to be special. If triangularity is
Mace, together with Costall et al. and Vereijken & perceived as such, then equilateral triangles need not have
Whiting, describe the history of the term amodal. We ac- any privileged status (as is often attributed to amodal re-
cept these historical accounts. Our discussion of the hy- dundancy); they are just another type of triangle. Similarly,
pothesis of amodal specification, however (sect. 3.3.2), is if perception is based on the pickup of information in the
drawn from the contemporary literature, which may di- global array, then patterns that correspond to redundancy
verge from the historical sources. at the level of individual perceptual systems would have no
The assumption that stimulation is redundant across per- special status, and would be neither easier nor harder to de-
ceptual systems is widespread, extending beyond the con- tect than any other patterns in the global array.
cept of amodal specification. For example, redundancy is Walker-Andrews understands that the global array dif-
commonly assumed by students of intersensory interaction fers qualitatively from putative amodal specification. The
(e.g., Mergner & Becker), by researchers who study per- global array makes it possible, in principle, for perception
ception and action in simulators and virtual environments to be accurate without any comparison of the activity of dis-
(e.g., Kennedy et al. 1990), and in the literature on motion tinct perceptual systems. As Walker-Andrews notes, per-
sickness (e.g., Oman 1990). A serious problem with the con- ception based on the global array depends upon sensitivity
cept of amodal specification is that redundancy across any to higher-order patterns, and not upon comparisons be-
two single-energy arrays is rare (contra Flom & Bahrick; tween modalities.
Rosenblum & Gordon, and Mergner & Becker; for an
extensive discussion and list of examples, see Stoffregen &
Riccio 1991). We noted that the amodal specification view R11. Dimensionality of the global array
has not addressed the consequences of nonredundant rela-
tions among single-energy arrays (sect. 3.3.3). Commenta- Hughes and others (e.g., Burton; Costall et al.; McMi-
tors who support the concept of amodal redundancy (e.g., chael & Bingham; Peper & Beek; Pittenger) raise im-
Flom & Bahrick; Lewkowicz & Scheier; Rosenblum & portant questions about the dimensionality of the global
Gordon; Vereijken & Whiting), did not respond to this array. They argue that there are a variety of different global
problem. Focusing exclusively on cases that appear to ex- arrays, each having a different number of dimensions, that
hibit amodal redundancy is not likely to lead to a theory of is, each including a different constellation of types of ambi-
perception that is general. ent energy. This argument is based on the facts of sensory
Our view of amodality resembles James Gibson’s view of loss. Hughes argues that blind people, for example, are
static perception. “Static” perception is not opposed to “dy- exposed to a different global array than sighted people, with
namic” perception. It is a specific case (i.e., the limiting one global array including optics while the other does not.
case) of perception. Similarly, we believe that redundant re- We disagree. The arguments of Hughes, Burton, McMi-
lations among single-energy arrays are limiting cases of chael & Bingham, and Peper & Beek suggest a confu-
nonredundant relations among single-energy arrays. They sion between the existence of information, on the one hand,
exist, that is, there are a few situations in which the pattern and the sampling or pickup of information, on the other.
in one single-energy array is isomorphic with the pattern in This is clear in Burton’s question about how many percep-
another single-energy array (e.g., Flom & Bahrick).4 But tual systems are necessary to have a global array, and in
these redundancies may be irrelevant to perception, if per- McMichael & Bingham’s suggestion that sensory attenu-
ceivers are directly sensitive to the global array. Amodal re- ation or absence can influence the structure of the global
dundancy in single-energy arrays corresponds to a pattern array. The global array is not made up of perceptual sys-
in the global array which may be detected. This would dif- tems; it is made up of patterns of ambient energy that may
fer qualitatively from picking up separate patterns in differ- be sampled by perceptual systems. There is only one global
ent single-energy arrays, and then (internally) determining array. It includes relations among all forms of ambient
that they were (or were not) redundant. energy.

Different perceivers sample different parts of the global As noted in the target article (sect. 7; cf. sect. R1), new
array, depending on their needs, interests, and abilities. This research (and perhaps new research methods) will be re-
is well understood in the context of single-energy arrays. To quired in order to conduct direct tests of the hypothesis that
maintain the context of sensory loss, consider persons who perceivers detect structures in the global array. Mark’s sug-
are color blind. They are not sensitive to the full range of fre- gestion to use techniques for establishing perceptual inde-
quencies sampled by the normal human eye, yet no one pendence (Ashby & Townsend 1986) is welcome in this
would suggest that color blind people are exposed to an op- context, and should be pursued.
tic array that is any different from the one sampled by the It is also the case that existing methodologies can be
color sighted. There is only one optic array, which is sampled adapted to be relevant. Research should begin with an a pri-
differently by the color blind and the color sighted. Varia- ori derivation of information, either qualitative or, prefer-
tions in frequency of light are related to the colors of things ably, quantitative, that exists in the global array. For our ex-
in the world. Color blind people cannot pick up this partic- ample we will use the global array parameter identified by
ular type of information, and for this reason often do a poor Peper et al. (1994), and discussed in section 6.1 of the tar-
job of controlling their behavior with respect to the color of get article. Equation 1 relates to a situation in which a per-
things. Similarly, there is only one global array. It is sampled son must extend the hand laterally (relative to the torso) in
differently by people (and species) having different percep- order to catch a ball. The relation between optics and hap-
tual capabilities. Animals without functioning visual systems tics expressed in Equation 1 is specific to the affordance for
(e.g., moles, earthworms, and blind people), will be exposed interception (that is, to the hand velocity needed to move
to the global array, just like other animals. These animals the hand to the right place at the right time to make the
cannot detect or control behavior relative to the illuminated catch). Neither optics alone nor haptics alone provides the
environment, but that does not mean that they are unable to needed specificity.
sample the global array. The example of moles and earth- There is an analogy between the relation of ball move-
worms is important because it illustrates the close functional ment and hand movement, in this situation, and the relation
relation that exists between the capabilities of action and between distance and velocity in time-to-contact. Time-to-
perception systems. As noted in the target article (sect. contact is a consequence of the relation between distance
6.2.3), the ability of moles and worms to sample the global and velocity, but it is not necessary to perceive either dis-
array is entirely adequate for them to achieve adaptive be- tance or velocity in order to perceive time-to-contact. It is
havior. That is, these animals can perceive their behavior sufficient to perceive the higher-order relation between the
and control it relative to the same subset of physical refer- two.5 McLeod and Ross (1983) and Schiff and Detwiler
ents. In general, each species and each individual can detect (1979) tested the hypothesis that knowledge about time-to-
certain portions of the global array, which enable it to per- contact might be derived from mental calculations based on
ceive and control its actions relative to a certain set of phys- perceived distance and velocity, as opposed to being based
ical referents. This explains why blind people walk differ- on direct sensitivity to time-to-contact. Subjects viewed
ently from sighted people: The blind cannot perceive (or films depicting impending collision with the viewer. The
control) motion relative to the illuminated environment. films ended before collision, the subjects’ task was to indi-
cate when the collision would have occurred. The results in
each study were consistent with direct perception of time-
R12. Research methodology to-contact, and inconsistent with calculation of time-to-con-
tact from independent percepts of distance and velocity.6 If
Our theory will succeed to the extent that it can inspire use- it is possible to perceive time-to-contact without perceiving
ful research (as noted by Pallas; cf. Leighty et al.; Pit- either distance or velocity, then it may be possible to per-
tenger; Riley). In discussing the conduct of research we ceive the required velocity of hand displacement without
focus on three issues. perceiving, either time-to-contact, or current hand position.
McMichael & Bingham assert that analysis of struc- That is, in terms of Equation 1, it may be possible to per-
tures in single-energy arrays is a prerequisite for use of the ceive the left-hand side of the equation without having dis-
perturbation paradigm in research on perception, but they tinct sensitivity to the separate terms of the right-hand side.
offer no argument as to why this should be so. As we noted Experiments can address this question empirically. Here,
in the target article (sect. 7; cf. Fouque et al. 1999), the per- we propose one such experiment. As in the experiments of
turbation paradigm can be and has been used in research Peper et al. (1994), subjects would be instructed to catch
contrasting the pickup of information from single-energy balls that approach at an angle to the line of sight (i.e., so
arrays and from the global array. More generally, perturba- that the arm must be extended laterally, relative to the
tion of any single-energy array will simultaneously perturb torso, in order for catching to occur). Catching would re-
the global array; this cannot be prevented. The theoretical quire the subject to adapt Vh, the lateral velocity of the
and empirical question is which of these perturbations is hand, to the ratio between the instantaneous sideward ball-
detected by perceivers. This has general consequences for hand distance and t, the optical parameter related to time-
research on perception. When an experimenter employs a to-contact.
manipulation of structure in one or more single-energy ar- The essential manipulation would be to vary the values of
rays, that manipulation will also alter (or perturb) the struc- the optical and haptic parameters in Equation 1, but to do
ture of the global array. Thus, effects resulting from such so simultaneously in such a way that the required velocity of
manipulations could be caused by structure in the single- the hand would not change. The t parameter could be ma-
energy array(s), or by structure in the global array. This is nipulated using the deflating ball paradigm (e.g., Savels-
why existing research generally cannot be used to evaluate bergh et al. 1991), and Xh could be manipulated by chang-
perceptual sensitivity to the global array or, we would argue, ing the perceived direction or location of the arm through
to single-energy arrays. the manipulation of eigenvectors of its inertia tensors (e.g.,

References/Stoffregen & Bardy: On specification and the senses
Garrett et al. 1998; Pagano & Turvey 1995; Riley & Turvey, NOTES
in press; Turvey 1996). The crucial manipulation would be 1. Pailhous et al. believe that our position is that pickup of in-
to combine a later (perceived) arrival of the ball (by deflat- formation from the global array is “localized in associative areas.”
ing the ball during its approach) with a closer (perceived) lo- This is not our position.
cation of the hand. These manipulations could be combined 2. The same problem applies to Wade, who discounts exam-
ples that we drew from vehicular travel as being “artificial.” Hu-
so that they co-varied, that is, so that the value of Equation mans (together with their domesticated animals) have routinely
1 remained constant across the variation in its constituent traveled using vehicles, such as ships, for thousands of years. To
parameters. When this was the case, we would predict that refer to vehicular travel as artificial begs the definition of the term.
catching (and, by implication, hand velocity) would be ac- Note that we do not claim that everything is natural; rather, we
curate (that is, that subjects would respond on the basis of stress that the distinction between natural and unnatural or artifi-
the constant value of the global array parameter), despite cial is meaningful only if it can be defined.
the fact that t would not be specific to time-to-contact and 3. This suggests that Runeson et al. may endorse the concept
Xh would not be specific to hand position. That is, the ma- of partial or occasional specificity (sect. R2).
nipulation would produce an overestimation of the place of 4. Amodal redundancy may be less common than is supposed
contact if subjects relied on the haptic information only, and by supporters of the amodality view. Consider speech, which is of-
ten thought to give rise to redundant patterns in optics and
an overestimation of time-to-contact if they relied on opti- acoustics (e.g., Rosenblum & Gordon; Walker-Andrews). The
cal information only. Specific predictions (both quantitative acoustic waveform that reaches the ears is structured by the ac-
and qualitative) could be made in situations like this to vali- tivity of several anatomical structures, including the lips, tongue,
date the use of global array in ball catching, and, more gen- jaw, and larynx. The tongue is often not visible, or only partially
erally, in the perceptual guidance of action. visible, and the larynx is never seen. This means that the visible
part of speech (e.g., movements of the lips and jaw) corresponds
to only a portion of the acoustic waveform. In other works, the
R13. Concluding remarks acoustic and optical patterns are not identical; they are not
amodally redundant.
Is the animal-environment interaction specified in patterns 5. For similar reasons, the hypothesis raised by Brenner &
of ambient energy? As emphasized by many of the com- Smeets that detecting information in the global array should take
more time than in single-energy arrays may not be correct. It may
mentators, the answer to this question is central to any the- well be the case that the detection of higher-order relations
ory of perception and action. We have argued that specifi- (within- or between-energy arrays) take less time than the detec-
cation does not exist, and cannot exist, in patterns that are tion and combination of their constituent parts.
confined to any single form of ambient energy. This is due 6. Estimates of collision time were strongly correlated with col-
to the facts of motion in a relativistic universe. As reinforced lision time as depicted in the films, but tended toward underesti-
in our response, behavior is controlled relative to many mation of collision time. This might suggest that perception of
physical referents that are independent of one another, so time-to-contact was inaccurate and, therefore, not based on the
that motion relative to one referent may be independent of (nominally accurate) information available in the stimulus films.
motion relative to another. Crucially, motion relative to any An alternative interpretation of the underestimates is that they are
given referent often does not create or alter structure in all artifactual results of the use of a paradigm in which subjects made
judgments, rather than using perception to control action. This
forms of ambient energy. This means that, as a matter of is supported by the extraordinarily precise levels of timing that
physics, it is not possible for the structure of pattern in any are regularly observed in research on the control of interceptive
given form of energy to be uniquely related to an animal’s action (e.g., Bardy & Laurent 1998; Bootsma & van Wieringer
motion relative to the physical environment. If scientists as- 1990).
sume that each form of ambient energy constitutes a qual-
itatively distinct “input” to perceptual systems, then the
absence of specification in single-energy arrays would pre-
clude any theory of direct perception, and would require
that theories of perception include some mental mecha- References
nism that could “recover” information about reality from
impoverished or nonspecific stimulation. Letters “a” and “r” appearing before authors’ initials refer to target article
We have presented a very different view of the stimula- and response references, respectively.
tion available to perceptual systems. We have pointed out Abrams, R. A. & Landgraf, J. Z. (1990) Differential use of distance and location
that forms of ambient energy, while they differ qualitatively, information for spatial localization. Perception and Psychophysics 47:349 – 59.
do not exist independently. Just as there is structure within [EB]
any given form of energy, there also is structure across or Ackerman, D. (1990) A natural history of the senses. Vintage. [aTAS]
Adolph, K. E. (1997) Learning in the development of infant locomotion.
between forms of ambient energy. We have argued that Monographs of the Society for Research in Child Development 56:3, Serial No.
these multi-energy patterns, which make up what we call 251. [KEA]
the global array, are uniquely structured by the animal- (in press) Specificity of learning: Why infants fall over a veritable cliff.
environment interaction. If this is true then specification Psychological Science. [KEA]
exists, and it is possible, in principle, for perception to ex- Adolph, K. E. & Eppler, M. A. (1998) Development of visually guided locomotion.
Ecological Psychology 10:303–12. [KEA]
ist, to be direct, and to be used (through exploration, differ- (in press) Flexibility and specificity in infant motor skill acquisition. In: Progress
entiation, and learning) in the guidance of adaptive action. in infancy research, ed. J. Fagen. Ablex. [KEA]
Aggleton, J. P., Keen, S., Warburton, E. C. & Bussey, T. J. (1997) Extensive
cytotoxic lesions involving both the rhinal cortices and area TE impair
ACKNOWLEDGMENTS recognition but spare spatial alternation in the rat. Brain Research Bulletin
We thank all the commentators for their valuable contributions. 43:279–87. [KFA]
We hope that the current exchange will stimulate further discus- Alexandrov, Y. & Jarvilehto, T. (1993) Activity and reactivity in psychology and
sion (and research) on the concept of specification. neurophysiology. Ecological Psychology 5:85–103. [aTAS]

Allison, R. S., Howard, I. P. & Zacher, J. E. (1999) Effect of field size, head motion, Bingham, G. P. & Stassen, M. G. (1994) Monocular egocentric distance
and rotational velocity on roll vection and illusory self-tilt in a tumbling room. information generated by head movement. Ecological Psychology 6:219 – 38.
Perception 28:299 – 309. [CEP] [arTAS]
Amazeen, E. (1999) Perceptual independence of size and weight by dynamic Blake, R. & Hirsch, H. V. B. (1975) Deficits in binocular depth perception in
touch. Journal of Experimental Psychology: Human Perception and cats after alternating monocular deprivation. Science 190(4219):1114 –16.
Performance 25:102–19. [LSM] [KFA]
Anderson, J. D. (1996) The reality of illusion. Southern Illinois University Press. Bootsma, R. J. & Van Wieringen, P. C. W. (1990) Timing an attacking forehand
[rTAS] drive in table tennis. Journal of Experimental Psychology: Human Perception
Aristotle (1931) De anima. In: The works of Aristotle translated into English, ed. and Performance 16:21–29. [rTAS]
W. D. Ross. Clarendon Press. [aTAS] Boring, E. G. (1950) A history of experimental psychology, second edition.
Arshavsky, Y. I., Gelfand, I. M. & Orlovsky, G. N. (1985) The cerebellum and Appleton- Century-Crofts. [aTAS]
control of rhythmical movements. In: The motor system in neurobiology, ed. Brandt, T., Dichgans, J. & Koenig, E. (1973) Differential effects of central versus
E. V. Evarts, S. P. Wise & D. Bousfield. Elsevier. [AGF] peripheral vision on egocentric and exocentric motion perception.
Ashby, G. & Townsend, J. T. (1986) Varieties of perceptual independence. Experimental Brain Research 16:476–91. [rTAS]
Psychological Review 93:154–79. [LSM, rTAS] Bremmer, F., Kubishik, M., Pekel, M., Lappe, M. & Hoffman, K. P. (1999) Linear
Bahrick, L. E. (1988) Intermodal learning in infancy: Learning on the basis of two vestibular self-motion signals in monkey medial superior temporal area.
kinds of intermodal relations in audible and visible events. Child Development Annals of the New York Academy of Sciences 871:272–82. [CEP]
59:197–209. [RAF, aTAS] Brenner, E. (1991) Judging object motion during smooth pursuit eye movements:
(1992) Infants’ perceptual differentiation of amodal and modality-specific audio- The role of optic flow. Vision Research 31:1893–902. [EB]
visual relations. Journal of Experimental Child Psychology 53:180–99. [RAF, Brenner, E. & Cornelissen, E. W. (2000) Separate simultaneous processing of
AW-A] egocentric and relative positions. Vision Research 40:2557–63. [EB]
(1994) The development of infants’ sensitivity to arbitrary intermodal relations. Brenner, E. & Landy, M. S. (1999) Interaction between the perceived shape of two
Ecological Psychology 6:111–23. [RAF] objects. Vision Research 39:3834–48. [EB]
(in press) Increasing specificity on perceptual development: Infants’ detection of Brenner, E. & Smeets, J. B. J. (1996) Hitting moving targets: Co-operative control
nested levels of multimodal stimulation. Journal of Experimental Child of ‘when’ and ‘where’. Human Movement Science 15:39–53. [EB]
Psychology. [RAF] (1997) Fast responses of the human hand to changes in target position. Journal
Bahrick, L. E. & Lickliter, R. (2000) Intersensory redundancy guides attentional of Motor Behavior 29:297–310. [EB]
selectivity and perceptual learning in infancy. Developmental Psychology Brenner, E., Smeets, J. B. J. & de Lussanet, M. H. E. (1998) Hitting moving
36:190 –201. [RAF, AW-A] targets: Continuous control of the acceleration of the hand on the basis of the
Bahrick, L. E. & Pickens, J. N. (1994) Amodal relations: The basis for intermodal target’s velocity. Experimental Brain Research 122:467–74. [EB]
perception and learning. In: The development of intersensory perception: Brenner, E. & van Damme, W. J. M. (1999) Perceived distance, shape and size.
Comparative perspectives, ed. D. Lewkowicz & R. Lickliter. Erlbaum. Vision Research 39:975–86. [EB]
[RAF] Brenner, E. & van den Berg, A. V. (1996) The special role of distant structures in
Ballard, D. H. (1991) Animate vision. Artificial Intelligence 48:57–86. [NJTT] perceived object velocity. Vision Research 36:3805–14. [EB]
Bardy, B. G. & Laurent, M. (1998) How is body orientation controlled during Brenner, E., van den Berg, A. V. & van Damme, W. J. M. (1996) Perceived motion
somersaulting? Journal of Experimental Psychology: Human Perception and in depth. Vision Research 36:699–706. [EB]
Performance 24:963 –77. [YC, PF, arTAS] Bressler, S. L., Coppola, R. & Nakamura, R. (1993) Episodic multiregional cortical
Bardy, B. G., Marin, L., Stoffregen, T. A. & Bootsma, R. J. (1999) Postural coherence at multiple frequencies during visual task performance. Nature
coordination modes considered as emergent phenomena. Journal of 366:153- 56. [KFA]
Experimental Psychology: Human Perception and Performance 25:1284–301. Buckner, R. L., Petersen, S. E., Ojemann, J. G., Miezin, F. M., Squire, L. R. &
[KEA, rTAS] Raichle, M. E. (1995) Functional anatomical studies of explicit and implicit
Barrie, J. M., Freeman, W. J. & Lenhart, M. D. (1996) Spatiotemporal analysis of memory retrieval tasks. Journal of Neuroscience 15(1):12–29. [KFA]
pre-pyriform, visual, auditory, and somesthetic surface EEGs in trained Buechel, C., Price, K., Frackowiak, R. S. J. & Friston, K. (1998) Different
rabbits. Journal of Neurophysiology 76(1):520–39. [KFA] activation patterns in the visual cortex of late and congenitally blind subjects.
Barwise, J. & Perry, J. (1983) Situations and attitudes. MIT Press. [SR] Brain 121:409- 19. [TK]
Becker, R. A. (1954) Introduction to theoretical mechanics. McGraw-Hill. [aTAS] Burt, P. J. (1988) “Smart sensing” in machine vision. In: Machine vision, ed. H.
Beer, R. D. (1995) A dynamical systems perspective on agent-environment Freeman. Academic Press. [NJTT]
interaction. Artificial Intelligence 72:173–215. [KFA] Bushnell, E. W. & Weinberger, N. (1987) Infants’ detection of visual-tactual
(2000) Dynamical approaches to cognitive science. Trends in Cognitive Sciences discrepancies: Asymmetries that indicate a directive role of visual information.
4(3):91– 99. [KSO] Journal of Experimental Psychology: Human Perception and Performance
Berg, H. C. (2000) Motile behavior of bacteria. Physics Today 53:24–29. [rTAS] 13:601–608. [aTAS]
Berkeley, G. (1709/1975) Philosophical works, ed. M. R. Ayers. Orion. (Original Calvert, G. A., Bullmore, E. T., Brammer, M. J., Campbell, R., Williams, S. C. R.,
work published in 1709.) [aTAS] McGuire, P. K., Woodruff, P. W. R., Iversen, S. D. & David, A. S. (1997)
Berthier, N. E., Clifton, R. K., Gullapalli, V., McCall, D. D. & Robin, D. (1996) Activation of auditory cortex during silent lipreading. Science 276:593 – 96.
Visual information and object size in the control of reaching. Journal of Motor [LDR]
Behavior 28:187– 97. [YC] Camhi, J. M. (1984) Neuroethology: Nerve cells and the natural behavior of
Berthoz, A. (1997) Le sens du mouvement [The sense of movement]. Odile Jacob. animals. Sinauer. [rTAS]
[rTAS] Carpenter-Smith, T. R., Futamura, R. G. & Parker, D. E. (1995) Inertial
Berthoz, A., ed. (1993) Multisensory control of movement. Oxford University Press. acceleration as a measure of linear vection: An alternative to magnitude
[rTAS] estimation. Perception and Psychophysics 57:35–42. [aTAS]
Bingham, G. P. (2000a) Comparing reaching in virtual and actual environments: An Chapman, S. (1968) Catching a baseball. American Journal of Physics 36:868 –70.
effect of accomodation? Paper presented at a meeting of the Association for [CFM]
Research in Vision and Ophthalmology, Fl. Lauderdale, Florida, May 1st, Clifton, R. K., Rochat, P., Robin, D. J. & Berthier, N. E. (1994) Multimodal
2000. [KM] perception in the control of infant reaching. Journal of Experimental
(2000b) Events (like objects) are things, can have affordance properties, and can Psychology: Human Perception and Performance 20:876–86. [aTAS]
be perceived. Ecological Psychology 12:29–36. [SR] Coello, Y., Delay, D., Nougier, V. & Orliaguet, J. P. (2000a) Temporal control of
Bingham, G. P., Bradley, A., Bailey, M. & Vinner, R. (submitted) Accomodation, impact movement: The time from departure control hypothesis in golf
occlusion and disparity matching are used to guide reaching: A comparison of putting. International Journal of Sport Psychology 31:24–46. [YC]
actual versus virtual environments. Journal of Experimental Psychology: Coello, Y. & Grealy, M. A. (1997) Effect of size and frame of visual field on the
Human Perception and Performance. [KM] accuracy of an aiming movement. Perception 26:287–300. [YC]
Bingham, G. P. & Pagano, C. C. (1998) The necessity of a perception-action Coello, Y. & Magne, P. (2000) Determination of target position in a structured
approach to definite distance perception: Monocular distance perception to environment: Selection of information for action. European Journal of
guide reaching. Journal of Experimental Psychology: Human Perception and Cognitive Psychology. 12:489–519. [YC]
Performance 24:145 – 68. [YC, KM] Coello, Y., Magne, P. & Plenacoste, P. (2000b) The contribution of retinal signal to
Bingham, G. P., Schmidt, R. C. & Rosenblum, L. D. (1995) Dynamics and the the specification of target distance in a visuo-manual task. Current Psychology
orientation of kinematic forms in visual event recognition. Journal of Letters. 3:75–89. [YC]
Experimental Psychology: Human Perception and Performance 21:1473–93. Cohen, L. G., Celnik, P., Pascual-Leone, A., Corwell, B., Faiz, L., Dambrosia, J.,
[aTAS] Honda, M., Sadato, N., Gerloff, C., Catala, D. & Hallett, M. (1997)

Functional relevance of cross-modal plasticity in blind humans. Nature Fishman, M. & Michael, C. (1973) Integration of auditory information in the cat’s
389:180 – 83. [TK] visual cortex. Vision Research 13:1415–19. [aTAS]
Cole, J. (1995) Pride and the daily marathon. MIT Press. [KM] Fitzpatrick, P., Carello, C., Schmidt, R. C. & Corey, D. (1994) Haptic and visual
Collins, J. J., Imhoff, T. T. & Grigg, P. (1996) Noise-enhanced tactile sensation. perception of an affordance for upright posture. Ecological Psychology 6:265 –
Nature 383:770. [GER] 88. [LDR, aTAS]
Craig, C. M., Delay, D., Grealy, M. A. & Lee, D. N. (2000) Guiding the swing in Fleishman, E. A. & Rich, S. (1963) Role of kinesthetic and spatial visual abilities in
golf putting. Nature 405:295–96. [YC] perceptual-motor learning. Journal of Experimental Psychology 66:215 –26.
Crick, F. & Koch, C. (1990) Toward a neurobiological theory of consciousness. [CEP]
Seminars in Neurosciences 2:263–75. [EM] Fodor, J. A. & Pylyshyn, Z. W. (1981) How direct is visual perception? Some re-
Curthoys, I. S. & Wade, N. J. (1990) A balanced view of otolith function: Comment flections on Gibson’s “ecological approach.” Cognition 9(2):139 – 96. [MFN]
on Stoffregen and Riccio (1988). Psychological Review 97:132–34. [NJW] Foley, J. M. & Held, R. (1972) Visually directed pointing as a function of target
Cutting, J. E. (1986) Perception with an eye for motion. MIT Press. [aTAS] distance, direction, and available cues. Perception and Psychophysics 12:263 –
(1991) Four ways to reject directed perception. Ecological Psychology 3:25–34. 68. [YC]
[aTAS] Foo, P., Kelso, J. A. S. & deGuzman, G. C. (2000) Functional stabilization of
(1993) Perceptual artifacts and phenomena: Gibson’s role in the 20th century. unstable fixed points: Human pole balancing using “time-to-balance”
In: Foundations of perceptual theory, ed. S. C. Masin. Elsevier. [PAC] information. Journal of Experimental Psychology: Human Perception and
Cutting, J. E. & Vishton, P. M. (1995) Perceiving layout and knowing distances: Performance 4:1281–97. [PF]
The integration, relative potency, and contextual use of different information Fouque, F., Bardy, B. G., Stoffregen, T. A. & Boostsma, R. J. (1999) Action and
about depth. In: Perception of space and motion, ed. W. Epstein & S. Rogers. intermodal information influence the perception of orientation. Ecological
Academic Press. [aTAS] Psychology 11:1–43. [arTAS]
Darwin, C. (1859/1958) On the origin of species by means of natural selection. Fowler, C. A. (1986) An event approach to the study of speech perception from a
New American Library. (Original work published in 1859.) [JPa] direct-realist perspective. Journal of Phonetics 14:3–28. [aTAS]
Deliagina, T. G. & Fagerstedt, P. (2000) Responses of reticulospinal neurons in Fowler, C. A. & Dekle, D. J. (1991) Listening with the hand and eye: Cross-modal
intact lamprey to vestibular and visual inputs. Journal of Neurophysiology contributions to speech perception. Journal of Experimental Psychology:
83:864 –78. [AGF] Human Perception and Performance 17:816–28. [aTAS]
Desjardin, R. N., Rogers, J. & Werker, J. F. (1997) An exploration of why Freeman, W. J. (1995) Societies of brains: A study in the neuroscience of love and
preschoolers perform differently than do adults in audiovisual speech hate. Erlbaum. [KFA]
perception tasks. Journal of Experimental Child Psychology 23:85–110. Frost, D. O. & Metin, C. (1985) Induction of functional retinal projections to the
[YC] somatosensory system. Nature 317:162–64. [TK]
Dichgans, J. & Brandt, T. (1978) Visual-vestibular interaction: Effects on self- Gandevia, S. C. & Burke, D. (1992) Does the nervous system depend on
motion perception and postural control. In: Handbook of sensory physiology, kinesthetic information to control natural limb movements? Behavioral and
vol. VIII: Perception, ed. R. Held, H. Leibowitz & H. Teuber. Springer- Brain Sciences 15:614–32. [AGF]
Verlag. [arTAS] Gao, W.-J. & Pallas, S. L. (1999) Cross-modal reorganization of horizontal
Diedrich, F. J. & Warren, W. H. (1995) Why change gaits? Dynamics of the walk- connectivity in auditory cortex without altering thalamocortical projections.
run transition. Journal of Experimental Psychology: Human Perception and Journal of Neuroscience 19:7940–50. [SLP]
Performance 21:183–202. [GER] Garrett, S. R., Pagano, C., Austin, G. & Turvey, M. T. (1998) Spatial and physical
Dijkstra, T. M. H., Schöner, G. & Gielen, C. C. A. M. (1994) Temporal stability of frames of reference in positioning a limb. Perception and Psychophysics
the perception-action cycle for postural control in a moving visual 60:1206–15. [arTAS]
environment. Experimental Brain Research 97:477–86. [aTAS] Gibson, E. J. (1969) Principles of perceptual learning and development. Appleton-
Dijkstra, T. M. H., Schöner, G., Giese, M. A. & Gielen, C. C. A. M. (1994) Century-Crofts. [PAC, RAF, arTAS]
Frequency dependence of the action-perception cycle for postural control in a (1977) How perception really develops: A view from outside the network. In:
moving visual environment: Relative phase dynamics. Biological Cybernetics Basic processes in reading: Perception and comprehension, ed. D. Laberge &
71:489 – 501. [CEP] S. J. Samuels. Erlbaum. [BV]
DiZio, P. A. & Lackner, J. R. (1986) Perceived orientation, motion, and (1983) Development of knowledge about intermodal unity: Two views. In: Piaget
configuration of the body during viewing of an off-vertical, rotating surface. and the foundations of knowledge, ed. L. S. Liben. Erlbaum. [aTAS]
Perception and Psychophysics 39:39–46. [rTAS] (1988) Exploratory behavior in the development of perceiving, acting, and the
Dretske, F. I. (1981) Knowledge and the flow of information. MIT Press. [SR] acquiring of knowledge. Annual Review of Psychology 39:1– 41. [rTAS, BV]
Duffy, C. J. (1998) MST neurons respond to optic flow and translational (1992) How to think about perceptual learning: Twenty-five years later. In:
movement. Journal of Neurophysiology 80:1816–72. [CEP] Cognition: Conceptual and methodological issues, ed. H. L. Pick, P. van den
Edelman, G. M. (1987) Neural Darwinism. Basic Books. [BV] Brock & D. C. Knill. American Psychological Association. [BV]
(1992) Bright air, brilliant fire: On the matter of mind. Basic Books. [BV] (2000) Where is the information for affordances? Ecological Psychology 12:53 –
Einstein, A. & Infeld, L. (1938) The evolution of physics. Simon and Schuster. 56. [YC]
[aTAS] Gibson, E. J., Riccio, G. E., Schmuckler, M. A. Stoffregen, T. A., Rosenberg, D. &
Eppler, M. A. (1995) Development of manipulatory skills and the deployment of Taormina, J. (1987) Detection of the traversability of surfaces by crawling and
attention. Infant Behavior and Development 18:391–405. [RAF] walking infants. Journal of Experimental Psychology: Human Perception and
Eppler, M. A., Adolph, K. E., Weise, I., Marin, L. M. & Wechsler-Clearfield, M. Performance 13:533–44. [KEA, rTAS]
(2000) Exploration in the service of mobility (under review). [KEA] Gibson, E. J. & Walker, A. S. (1984) Development of knowledge of visual and
Epstein, W. (1985) Amodal information and transmodal perception. In: Electronic tactual affordances of substances. Child Development 55:453 – 60. [aTAS,
spatial sensing for the blind, ed. D. Warren & E. Strelow. Martinus-Nijhoff. BV, AW-A]
[BH] Gibson, J. J. (1950) The perception of the visual world. Houghton-Mifflin. [SR]
Feldman, A. G. & Latash, M. L. (1982) Interaction of afferent and efferent signals (1966) The senses considered as perceptual systems. Houghton-Mifflin. [AC,
underlying joint position sense: Empirical and theoretical approaches. Journal BH, WMM, KM, CFM, MAR, SR, arTAS]
of Motor Behavior 14:174–93. [AGF] (1968) Consistency vs. discrepancy of stimulus information. Unpublished notes.
Feldman, A. G. & Levin, M. F. (1995) The origin and use of positional frames of Available at: http://www.trincoll.edu/depts/ecopsyc/perils/folder4/
reference in motor control. Behavioral and Brain Sciences 18:723–806. consistency.html [WMM]
[AGF, aTAS] (1979/1986) The ecological approach to visual perception. Original 1979 edition,
Feldman, A. G., Levin, M. F., Mitnitski, A. & Archambault, P. (1998) Multi-muscle Houghton-Mifflin; 1986 edition, Erlbaum. [KEA, PAC, YC, AC, BH,
control in human movements. Keynote lecture. Journal of Electromyography WMM, KSO, SLP, GER, MAR, SR, aTAS]
and Kinesiology 8:383–90. [AGF] Gibson, J. J., Kaplan, G., Reynolds, H. & Wheeler, K. (1969) The change from
Feldman, D. E. & Knudson, E. I. (1997) An anatomical basis for visual calibration visible to invisible: A study of optical transitions. Perception and
of the auditory space map in the barn owl’s midbrain. Journal of Neuroscience Psychophysics 5:113–16. [WMM]
17:6820 –37. [SLP] Glennerster, A., Rogers, B. J. & Bradshaw, M. F. (1996) Stereoscopic depth
Filion, C. & Fragaszy, D. M. (1997) The psychomotor skills of tufted capuchins constancy depends on the subject’s task. Vision Research 36:3441– 56. [EB]
(Cebus apella). Paper presented at the meeting of the Southern Society for Gogate, L. J. & Bahrick, L. E. (1998) Intersensory redundancy facilitates learning
Philosophy and Psychology, Atlanta, GA, March 1997. [KAL] of arbitrary relations between vowel sounds and objects in seven-month-old
Fink, P., Jirsa, V. K., Foo, P. & Kelso, J. A. S. (2000) Local and global stabilization infants. Journal of Experimental Child Psychology 69:133–49. [RAF, AW-A]
of coordination by sensory information. Experimental Brain Research 134:9– Goldfield, E. C. (1995) Emergent forms: Origins and early development of human
20. [PF] action and perception. Oxford University Press. [arTAS]

Goldstein, H. (1980) Classical mechanics. Addison Wesley. [aTAS] Jeka, J. J., Oie, K., Schöner, G., Dijkstra, T. & Henson, E. (1998) Position and
Goodale, M. A., Pélisson, D. & Prablanc, C. (1986) Large adjustments in visually velocity coupling of postural sway to somatosensory drive. Journal of
guided reaching do not depend on vision of the hand or perception of target Neurophysiology 79:1661–74. [rTAS]
displacement. Nature 320:748–50. [YC] Jeka, J. J., Schöner, G., Dijkstra, T., Ribeiro, P. & Lackner, J. R. (1997) Coupling of
Gottlieb, G. (1971) Ontogenesis of sensory function in birds and mammals. In: The fingertip somatosensory information to head and body sway. Experimental
biopsychology of development, ed. E. Tobach, L. R. Aronson & E. Shaw. Brain Research 113:457–83. [CEP]
Academic Press. [DJL] Jenison, R. L. (1996) Estimation of time-to-arrival from acoustic information.
(1991) Experiential canalization of behavioral development: Theory. Paper presented at the Forum Acousticum, 1st meeting of the European
Developmental Psychology 27:35–39. [DJL] Acoustics Association, Antwerp, Belgium. [MFN]
Gray, C. M., Konig, P., Engel, A. K. & Singer, W. (1989) Oscillatory responses in (1997) On acoustic information for auditory motion. Ecological Psychology
cat visual cortex exhibit inter-columnar synchronization which reflects global 9(2):131–51. [MFN]
stimulus properties. Nature 338:334–37. [KFA] Jirsa, V. K., Fink, P., Foo, P. & Kelso, J. A. S. (2000) Parametric stabilization of
Graybiel, A. (1952) Oculogravic illusion. Archives of Opthalmology 48:605. [aTAS] biological coordination: A theoretical model. Journal of Biological Physics
Green, K. P. (1998) The use of auditory and visual information during phonetic 26:86–112. [PF]
processing: Implications for theories of speech perception. In: Hearing by eye Jones, R. K. & Lee, D. N. (1981) Why two eyes are better than one: The two views
II: Advances in the psychology of speechreading and auditory-visual speech, of binocular vision. Journal of Experimental Psychology: Human Perception
ed. R. Campbell, B. Dodd & D. Burnham. Psychology Press. [LDR] and Performance 7:30–40. [aTAS]
Green, K. P., Kuhl, P. K., Meltzoff, A. N. & Stevens, E. B. (1991) Integrating Kanizsa, G. & Gerbino, W. (1982) Amodal completion: Seeing or thinking? In:
speech information across talkers, gender, and sensory modality: Female faces Organization and representation in perception, ed. J. Beck. Erlbaum. [AC,
and male voices in the McGurk effect. Perception and Psychophysics 50:524– WMM]
36. [aTAS] Kawasaki, M. & Guo, Y.-X. (1998) Parallel projection of amplitude and phase
Gribble, P. L., Ostry, D. J., Sanguineti, V. & Laboissière, R. (1998) Are complex information from the hindbrain to the midbrain of the African electric fish,
control signals required for human arm movements? Journal of Gymnarchus niloticus. Journal of Neuroscience 18:7599–611. [SLP]
Neurophysiology 79:1409–24. [AGF] Kawato, M. (1999) Internal models for motor control and trajectory planning.
Grush, R. (2000) Self, world and space: The meaning and mechanisms of ego and Current Opinion in Neurobiology 9:718–27. [KSO]
allocentric spatial representation. Brain and Mind 1:59–92. [YC] Kellman, P. J. & Arterberry, M. E. (1998) The cradle of knowledge: Development of
Guski, R. (1990) Auditory localization: Effects of reflecting surfaces. Perception perception in infancy. MIT Press. [DJL]
19:819 –30. [aTAS] Kellman, P. J. & Short, K. R. (1987) Development of three-dimensional form
Haken, H. (1983) Synergetics: An introduction (3rd edition). Springer Verlag. perception. Journal of Experimental Psychology: Human Perception and
[aTAS] Performance 13:545–57. [aTAS]
Haken, H., Kelso, J. A. S. & Bunz, H. (1985) A theoretical model of phase Kellog, W. N. (1962) Sonar system of the blind. Science 137:399–404. [BH]
transitions in human hand movements. Biological Cybernetics 51:347–56. Kelso, J. A. S. (1995) Dynamic patterns. MIT Press. [arTAS]
[GER] Kelso, J. A. S., Fink, P. & DeLaplain, C. R. (submitted) Haptic information
Haken, H., Kelso, J. A. S., Fuchs, A. & Pandya, A. S. (1990) Dynamic pattern stabilizes and destabilizes coordinated movement: A reply to Carson et al.
recognition of coordinated biological motion. Neural Networks 3:395–401. (1999). Proceedings of the Royal Society of London. [PF]
[PF] Kennedy, J. M., Green, C. D., Nicholls, A. & Liu, C. H. (1992) Illusions and
Hall, D. L. & Llinas, J. (1997) An introduction to multisensor data fusion. knowing what is real. Ecological Psychology 4:153–72. [MAR]
Proceedings of the IEEE 85:6–23. [KSO] Kennedy, R. S., Hettinger, L. J. & Lilienthal, M. G. (1990) Simulator sickness. In:
Handel, S. (1989) Listening: An introduction to the perception of auditory events. Motion and space sickness, ed. G. H. Crampton. CRC Press. [rTAS]
MIT Press. [aTAS] Kinchla, R. A. (1971) Visual movement perception: A comparison between
Harris, C. S. (1965) Perceptual adaptation to inverted, reversed, and displaced absolute and relative movement discrimination. Perception and Psychophysics
vision. Psychological Review 72:419–44. [aTAS] 9(2A):165–71. [AHW]
Hartline, P. H., Kass, L. & Loop, M. S. (1978) Merging of the modalities in the Kugler, P. N. & Turvey, M. T. (1987) Information, natural law, and the self-
optic tectum: Infrared and visual integration in rattlesnakes. Science assembly of rhythmic movement. Erlbaum. [aTAS]
199:1225 –29. [aTAS] Kuhl, P. K. & Meltzoff, A. N. (1984) The intermodal representation of speech in
Heider, F. (1958) The psychology of interpersonal relations. Wiley. [AC] infants. Infant Behavior and Development 7:361–81. [LDR, aTAS]
Heil, J. (1983) Perception and cognition. University of California Press. [aTAS] (1988) Speech as an intermodal object of perception. In: Perceptual
Heil, P., Bronchti, G., Wollberg, Z. & Scheich, H. (1991) Invasion of visual cortex development in infancy: The Minnesota symposium on child psychology, ed.
by the auditory system in the naturally blind mole rat. NeuroReport 2:735–38. A. Yonas. Erlbaum. [aTAS]
[SLP] Kujala, T., Alho, K., Huotilainen, M., Ilmoniemi, R. J., Lehtokoski, A., Leinonen,
Held, R. & Hein, A. (1963) Movement-produced stimulation in the development A., Rinne, T., Salonen, O., Sinkkonen, J., Standertskjöld-Nordenstam, C.-G. &
of visually guided behavior. Journal of Comparative and Physiological Näätänen, R. (1997) Electrophysiological evidence for cross-modal plasticity
Psychology 56:872–76. [rTAS] in humans with early- and late-onset blindness. Psychophysiology 34:213 –16.
Held, R. & Leibowitz, H. (1994) The significance of vection. In: Perceiving events [TK]
and objects, ed. G. Jansson, S. S. Bergstrom & W. Epstein. Erlbaum. [aTAS] Kujala, T., Alho, K., Kekoni, J., Hämäläinen, H., Reinikainen, K., Salonen, O.,
Henderson, J. M. (1992) Identifying objects across saccades: Effects of extrafoveal Standertskjöld-Nordenstam, C.-G. & Näätänen, R. (1995a) Auditory and
preview and flanker object context. Journal of Experimental Psychology: somatosensory event-related brain potentials in early blind humans.
Learning, Memory, and Cognition 18:521–30. [aTAS] Experimental Brain Research 104:519–26. [TK]
Hernandez-Reif, M. & Bahrick, L. E. (in press) The development of visual-tactual Kujala, T., Alho, K. & Näätänen, R. (2000) Cross-modal reorganization of human
perception of objects: Amodal relations provide the basis for learning arbitrary cortical functions. Trends in Neuroscience 23(3):115–20. [TK, JP]
relations. Infancy. [RAF] Kujala, T., Huotilainen, M., Sinkkonen, J., Ahonen, A. I., Alho, K., Hämäläinen, M.
Hochberg, J. E. (1964) Perception. Prentice-Hall. [aTAS] S., Ilmoniemi, R. J., Kajola, M., Knuutila, J. E. T., Lavikainen, J., Salonen, O.,
Howard, I. P. (1982) Human visual orientation. Wiley. [aTAS] Simola, J., Standertskjöld-Nordenstam, C.-G., Tiitinen, H., Tissari, S. O. &
Hughes, B. (submitted) Active artificial echolocation: Human perception of clear Näätänen, R. (1995b) Auditory and somatosensory event-related brain
paths via sonar. [BH] potentials in early blind humans. Experimental Brain Research 104:519 –26.
Hulliger, M., Nordh, E. & Vallbo, D. B. (1982) The absence of position response in [TK]
spindle afferent units from human finger muscles during accurate position Lackner, J. R. & DiZio, P. (1988) Visual stimulation affects the perception of
holding. Journal of Physiology (London) 322:167–79. [AGF] voluntary leg movements during walking. Perception 17:71–80. [AGF,
Hurley, S. (1998) Consciousness in action. Harvard University Press. [EM, JPi] aTAS]
Imanaka, Y. & Abernethy, B. (2000) Distance-location interference in movement Lajoie, Y., Paillard, J., Teasdale, N., Bard, C., Fleury, M., Forget, R. & Lamarre, Y.
reproduction: An interaction between conscious and unconscious processing? (1992) Mirror drawing in a deafferented patient and normal subjects:
In: Beyond dissociation: Interaction between dissociated implicit and explicit Visuoproprioceptive conflict. Neurology 42:1104–106. [CEP]
processing, ed. Y. Rossetti & A. Revonsuo. Benjamins. [YC] Lakatos, S. (1993) Temporal contraints on apparent motion in auditory space.
Ingvalsden, R. P. & Whiting, H. T. A. (1997) Moderns views on motor skill learning Perception and Psychophysics 54:139–44. [aTAS]
are not representative. Human Movement Science 16:705–32. [YC] Laurent, M., Montagne, G. & Durey, A. (1996) Binocular invariants in interceptive
Jacobs, D. M., Runeson, S. & Michaels, C. F. (1999) Fostering information tasks: A directed perception approach. Perception 25:1437–50. [aTAS]
detection in the visual perception of relative mass. In: Studies in perception Lee, D. N. (1976) A theory of visual control of braking based on information about
and action V, ed. M. A. Grealy & J. A. Thomson. Erlbaum. [MAR] time-to-collision. Perception 5:437–59. [YC]

(1980) The optic flow field: The foundation of vision. Philosophical Transactions Marks, L. E. (1978) The unity of the senses: Interrelations among the modalities.
of the Royal Society of London 290:169–79. [aTAS] Academic Press. [AC, aTAS]
(1981) Plummeting gannets: A paradigm of ecological optics. Nature 293:293– (1987) On cross-modal similarity: Auditory-visual interactions in speeded
94. [KSO] discrimination. Journal of Experimental Psychology: Human Perception and
(1990) Getting around with light or sound. In: Perception and control of self- Performance 13:384–94. [aTAS]
motion, ed. R. Warren & A. H. Wertheim. Erlbaum. [aTAS] Marr, D. (1982) Vision. Freeman. [EM, MFN]
(1997) Elements of general tau theory. In: Proceedings from the Ninth Massaro, D. W. (1987) Speech perception by ear and eye: A paradigm for
International Conference on Perception and Action, ed. G. L. Torenvliet & K. psychological inquiry. Erlbaum. [LDR]
J. Vicente. University of Toronto Press. [YC] Matlin, M. W. & Foley, H. J. (1992) Sensation and perception. Allyn and Bacon.
Lee, D. N. & Lishman, J. R. (1975) Visual proprioceptive control of stance. Journal [aTAS]
of Human Movement Studies 1:87–95. [aTAS] Matthews, P. B. C. (1981) Muscle spindles: Their messages and their fusiomotor
Lee, D. N., Lough, F. & Lough, S. (1984) Activating the perceptuo-motor system supply. In: Handbook of physiology: I. The nervous system. American
in hemiparesis. Journal of Physiology 349:28. [aTAS] Physiological Society. [AGF]
Leibowitz, H. W., Post, R. B. & Sheehy, J. B. (1986) Efference, perceived Maunsell, J. H. R., Nealy, T. A., Sclar, G. & DePriest, D. D. (1989) Representation
movement, and illusory displacement. Acta Psychologica 63:23–34. [aTAS] of extraretinal information in monkey visual cortex. In: Neural mechanisms of
Leighty, K. A. & Fragaszy, D. M. (2000) The monkey and the joystick: Rates of visual perception, ed. D. M.-K. Lam & C. D. Gilbert. Gulf Publishing.
motor skill acquisition in tufted capuchins (Cebus apella). Paper presented at [aTAS]
the regional meeting of Psi Chi, Athens, GA, April 2000. [KAL] McCollum, G. & Leen, T. K. (1989) Form and exploration of mechanical stability
Lestienne, F. G. & Gurfinkel, V. S. (1988) Postural control in weightlessness: A limits in erect stance. Journal of Motor Behavior 21:225–44. [KEA]
dual process underlying adaptation to an unusual environment. Trends in McCrea, D. A. (1992) Can senses be made of spinal interneuron circuits?
Neuroscience 11:359–63. [AGF] Behavioral and Brain Sciences 15:633–43. [AGF]
Lestienne, F. G., Soechting, J. & Berthoz, A. (1977) Postural readjustments indiced McGurk, H. & MacDonald, J. W. (1976) Hearing lips and seeing voices. Nature
by linear motion of visual scenes. Experimental Brain Research 28:363–84. 264:746–48. [aTAS]
[AGF] McLeod, R. W. & Ross, H. E. (1983) Optic-flow and cognitive factors in time-to-
Lestienne, F. G., Thullier, F., Archambault, P., Levin, M. F. & Feldman, A. G. collision estimates. Perception 12:417–23. [rTAS]
(2000) Multi-muscle control of head movements in monkeys: The referent Meinhold, R. J. & Singpurwalla, N. D. (1983) Understanding the Kalman Filter.
configuration hypothesis. Neuroscience Letters 283:65–68. [AGF] The American Statistician 37(2):123–27. [MFN]
Lettvin, J. Y., Maturana, H. R., McCulloch, W. S. & Pitts, W. H. (1959) What the Meltzoff, A. N. & Borton, R. W. (1979) Intermodal matching by human neonates.
frog’s eye tells the frog’s brain. Proceedings of the Institute of Radio Engineers Nature 282:403–404. [BV, AW-A]
47:1940- 51. [SLP] Meltzoff, A. N. & Kuhl, P. K. (1994) Faces and speech: Intermodal processing of
Lewkowicz, D. J. (1993) Le role des indices specifiques de modalite dans la biologically relevant signals in infants and adults. In: The development of
perception des equivalences intermodales chez les bebes. Psychologie intersensory perception: Comparative perspectives, ed. D. J. Lewkowicz & R.
Francaise 38:33–40. [AW-A] Lickliter. Erlbaum. [aTAS]
(1994) Development of intersensory perception in human infants. In: The Melzack, R. (1989) Phantom limbs, the self and the brain: The D. O. Hebb
development of intersensory perception: Comparative perspectives, ed. D. J. Memorial Lecture. Canadian Psychology 30:1–16. [AGF]
Lewkowicz & R. Lickliter. Erlbaum. [aTAS] Mendelson, M. J. & Haith, M. M. (1976) The relation between audition and vision
(2000) The development of intersensory temporal perception: An epigenetic in the human newborn. Monographs of the Society for Research in Child
systems/limitations view. Psychological Bulletin 126(2):281– 308. [RAF, Development 41, Serial No. 167. [BV]
DJL] Merfeld, D. M., Zupan, L. & Peterka, R. J. (1999) Humans use internal models to
Lewkowicz, D. J. & Lickliter, R. (1994) The development of intersensory estimate gravity and linear acceleration. Nature 398:615–18. [KSO]
perception: Comparative perspectives. Erlbaum. [DJL] Mergner, T., Huber, W. & Becker, W. (1997) Vestibular-neck interaction and
Lewkowicz, D. J. & Turkewitz, G. (1980) Cross-modal equivalence in early infancy: transformations of sensory coordinates. Journal of Vestibular Research 7:119 –
Auditory-visual intensity matching. Developmental Psychology 16:597–607. 35. [TM]
[BV] Mergner, T. & Rosemeier, T. (1998) Interaction of vestibular, somatosensory and
Lin, L. J., Chiou, F. T. & Cohen, H. H. (1995) Slip and fall prevention: A review of visual signals for posture control and motion perception under terrestrial and
research, practice, and regulations. Journal of Safety Research 26:203–12. microgravity conditions. Brain Research Reviews 28:118–35. [TM]
[KEA] Mergner, T., Schweigart, G., Müller, M., Hlavacka, F. & Becker, W. (2000b) Visual
Liotti, M., Ryder, K. & Woldorff, M. G. (1998) Auditory attention in the contributions to human self-motion perception during horizontal body
congenitally blind: Where, when and what gets reorganized? NeuroReport rotation. Archive Italiennes de Biologie 138:1139–67. [TM]
9:1007–12. [TK] Mergner, T., Wertheim, A. & Rumberger, A. (2000a) Which retinal and extra-
Lishman, J. R. & Lee, D. N. (1973) The autonomy of visual kinaesthesis. retinal information is crucial for circular vection? Archive Italiennes de
Perception 2:287–94. [aTAS] Biologie 138:123–38. [TM]
Livingstone, M. & Hubel, D. H. (1988) Segregation of form, color, movement, Metelli, F. (1940) Ricerche sperimentali sulla percezione del movimento. Rivista di
and depth: Anatomy, physiology, and perception. Science 240:740–49. Psicologia 36:319–70. [AC]
[SLP] Michaels, C. F. & Beek, P. (1995) The state of ecological psychology. Ecological
Locke, J. (1689/1975) An essay concerning human understanding, ed. P. H. Psychology 7:259–78. [rTAS]
Nidditch. Clarendon Press. (Original work published in 1689.) [aTAS] Michaels, C. F. & Carello, C. (1981) Direct perception. Prentice-Hall. [arTAS]
Lundberg, A. (1975) Control of spinal mechanisms from the brain. In: The nervous Michaels, C. F. & de Vries, M. M. (1998) Higher-order and lower-order variables
system, vol. 2, ed. D. B. Tower. Raven Press. [AGF] in the visual perception of relative pulling force. Journal of Experimental
Lutfi, R. A. & Wang, W. (1999) Correlational analysis of acoustic cues for the Psychology: Human Perception and Performance 24:526–46. [CEP, MAR]
discrimination of auditory motion. Journal of the Acoustical Society of Michaels, C. F., Zeinstra, E. & Oudejans, R. R. D. (2001) Information and action
America 106(2):919–28. [MFN] in timing the punch of a falling ball. Quarterly Journal of Experimental
Mace, W. M. (1977) James J. Gibson’s strategy for perceiving: Ask not what’s inside Psychology. [CFM]
your head, but what your head is inside of. In: Perceiving, acting, and Michotte, Thinès, G. & Crabbé, G. (1964) Les compléments amodaux des
knowing: Toward an ecological psychology, ed. R. E. Shaw & J. Bransford. structure perceptives. In: Studia Psychologia. Publications Université de
Erlbaum. [PAC] Louvain. Translation in: Thinès, G., Costall, A. & Butterworth, G. (1991)
Mack, A., Heuer, F., Villardi, K. & Chambers, D. (1985) The dissociation of Michotte’s experimental phenomenology of perception. Erlbaum. [WMM]
position and extent in Müller-Lyer figures. Perception and Psychophysics Milne, L. J. & Milne, M. (1962) Senses of animals and men. Atheneum. [aTAS]
37:335 – 44. [EB] Misceo, G. F., Hershberger, W. A. & Mancini, R. L. (1999) Haptic estimates of
Marin, L., Bardy, B. G., Baumberger, B., Flückiger, M. & Stoffregen, T. A. (1999) discordant visual-haptic size vary developmentally. Perception and
Interaction between task demands and support surface in the control of goal- Psychophysics 61(4):608–14. [JP]
oriented stance. Human Movement Science 18:31–47. [rTAS] Murata, K., Cramer, H. & Bach-y-Rita, P. (1965) Neuronal convergence of noxious
Mark, L. S. (1987) Eyeheight-scaled information about affordances: A study of acoustic, and visual stimuli in the visual cortex of the cat. Journal of
sitting and stair climbing. Journal of Experimental Psychology: Human Neurophysiology 28:1223–39. [TK]
Perception and Performance 13:361–70. [LSM, aTAS] Nashner, L. M., Black, F. O. & Wall, C. (1982) Adaptation to altered support
Mark, L. S., Balliet, J. A., Craver, K. D., Douglas, S. D. & Fox, T. (1990) What an surface and visual conditions during stance: Patients with vestibular conflicts.
actor must do in order to perceive the affordance for sitting. Ecological Journal of Neuroscience 2:536–44. [aTAS]
Psychology 2:325–66. [KEA, LSM, arTAS] Neelon, M. F. & Jenison, R. L. (1997) Estimating the spectrum of a moving sound

source. Paper presented at the 134th Meeting of the Acoustical Society of Polanyi, M. (1970) What is a painting? British Journal of Aesthetics 10:225 – 36.
America, San Diego, California, December 1–5, 1997. [MFN] [AC]
(2000) Recursive neural network estimation of auditory time-to-contact. Paper Proffitt, D. R. & Gilden, D. L. (1989) Understanding natural dynamics. Journal of
presented at the 23rd Midwinter Research Meeting of the Association for Experimental Psychology: Human Perception and Performance 15:384 – 93.
Research in Otolaryngology, St. Petersburg Beach, Florida, February 20–24, [aTAS]
2000. [MFN] Proteau, L. (1992) On the specificity of learning and the role of visual information
Neisser, U. (1977) Gibson’s ecological optics: Consequences of a different stimulus for movement control. In: Vision and motor control, ed. L. Proteau & D.
description. Journal for the Theory of Social Behavior 7:17–28. [PAC] Elliott. Elsevier. [YC]
Neville, H. J., Schmidt, A. & Kutas, M. (1983) Altered visual-evoked potentials in Ramachandran, V. S. (1990) Visual perception in people and machines. In: AI and
congenitally deaf adults. Brain Research 266:127–32. [TK] the eye, ed. A. Blake & T. Troscianko. Wiley. [NJTT]
Ohmi, M., Howard, I. P. & Landolt, J. P. (1987) Circular vection as a function of Rauschecker, J. P. & Korte, M. (1993) Auditory compensation for early blindness in
foreground-background relationships. Perception 16:17–22. [rTAS] cat cerebral cortex. Journal of Neuroscience 13:4538–48. [TK]
Oie, K. S., Kiemel, T. & Jeka, J. J. (2000) Multisensory processing for spatial Reardon, P. & Bushnell, E. W. (1988) Infants’ sensitivity to arbitrary pairings of
orientation. Journal of Cognitive Neuroscience 39. [KSO] color and taste. Infant Behavior and Development 11:245–50. [AW-A]
Oman, C. M. (1982) A heuristic mathematical model for the dynamics of sensory Rebillard, G., Carlier, E., Rebillard, M., Pujol, R. (1977) Enhancement of visual
conflict and motion sickness. Acta Otolaryngologica 44:Supplement 392. responses on the primary auditory cortex of the cat after an early destruction
[aTAS] of cochlear receptors. Brain Research 129:162– 64. [TK]
(1990) Motion sickness: A synthesis and evaluation of the sensory conflict theory. Reed, E. S. (1982) An outline of a theory of action systems. Journal of Motor
Canadian Journal of Physiology and Pharmacology 68:294–303. [rTAS] Behavior 14:98–134. [aTAS]
O’Regan, J. K. & Noë, A. (in press) A sensorimotor account of vision and visual (1996) Encountering the world. Oxford University Press. [arTAS]
consciousness. Behavioral and Brain Sciences 24(5). [EM] Reid, T. (1764) An inquiry into the human mind, on the principles of common
Oudejans, R. R. D., Michaels, C. F., Bakker, F. C. & Davids, K. (1999) Shedding sense. Millar, Kincaid and Bell. [NJW]
some light on catching in the dark: Perceptual mechanisms for catching fly Reisberg, D. (1992) Auditory imagery. Erlbaum. [aTAS]
balls. Journal of Experimental Psychology: Human Perception and Riccio, G. E. (1993) Information in movement variability: About the qualitative
Performance 25:531– 42. [CFM] dynamics of posture and orientation. In: Variability and motor control, ed. K.
Oudejans, R. R. D., Michaels, C. F., Bakker, F. C. & Dolne, M. A. (1996) The M. Newell & D. M. Corcos. Human Kinetics. [GER, rTAS]
relevance of action in perceiving affordances: Perception of catchableness of (1995) Coordination of postural control and vehicular control: Implications for
fly balls. Journal of Experimental Psychology: Human Perception and multimodal perception and simulation of self-motion. In: Local applications of
Performance 22:879 – 91. [rTAS] the ecological approach to human-machine systems, ed. P. Hancock, J. Flach,
Oudejans, R. R. D., Verheijen, R., Bakker, F. C., Gerrits, J. C., Steinbrückner, M. J. Caird & K. Vicente. Erlbaum. [GER, aTAS]
& Beek, P. J. (2000) Errors in judging “offside” in football. Nature 404:33. Riccio, G. E., Martin, E. J. & Stoffregen, T. A. (1992) The role of balance dynamics
[CFM] in the active perception of orientation. Journal of Experimental Psychology:
Pagano, C. C. & Turvey, M. T. (1995) The inertia tensor as a basis for the Human Perception and Performance 18:624–44. [GER, aTAS]
perception of limb orientation. Journal of Experimental Psychology: Human Riccio, G. E. & McDonald, P. V. (1998a) Characteristics of EVA mass handling
Perception and Performance 21:1070–87. [rTAS] skill. Society of Automotive Engineers Paper No. 981625 (also NASA Technical
Pailhous, J., Ferrandez, A. M., Flückiger, M. & Baumberger, B. (1990) Paper 3684, Lyndon B. Johnson Space Center, Houston, TX). [GER]
Unintentional modulations of human gait by optical flow. Behavioural Brain (1998b) Multimodal perception and multicriterion control of nested systems: I.
Research 38:275 –81. [JPa] Coordination of postural control and vehicular control. NASA Technical Paper
Pallas, S. L. (in press) Cross-modal plasticity as a tool for understanding ontogeny 3703. Lyndon B. Johnson Space Center, Houston, TX. [GER]
and phylogeny of cerebral cortex. In: Cortical areas: Unity and diversity, vol. Riccio, G. E. & Stoffregen, T. A. (1988) Affordances as constraints on the control
1, ed. A. Shüz & R. Miller. Harwood. [SLP] of stance. Human Movement Science 7:265–300. [KEA]
Pallas, S. L., Littman, T. & Moore, D. R. (1999) Cross-modal reorganization of (1990) Gravitoinertial force versus the direction of balance in the perception and
callosal connectivity in auditory cortex without altering thalamocortical control of orientation. Psychological Review 97:135–37. [aTAS]
projections. Proceedings of the National Academy of Sciences USA 96:8751– (1991) An ecological theory of motion sickness and postural instability.
56. [SLP] Ecological Psychology 3:195–240. [GER]
Parker, D. E. & Poston, R. L. (1984) Tilt from head-inverted position produces Rice, C. E. (1967) Human echo perception. Science 155:656–64. [BH]
displacement of visual subjective vertical in the opposite direction. Perception Riley, M. A., Mitra, S., Stoffregen, T. A. & Turvey, M. T. (1997) Influences of body
and Psychophysics 36:461–65. [aTAS] lean and vision on unperturbed postural sway. Motor Control 1:229– 46.
Patterson, F. R., Cacioppo, A. J., Gallimore, J. J., Hinman, G. E. & Nalepka, J. P. [KEA, rTAS]
(1997) Aviation spatial orientation in relationship to head position and attitude Riley, M. A. & Turvey, M. T. (in press) Inertial constraints on limb proprioception
interpretation. Aviation, Space and Environmental Medicine 68:463–71. are independent of visual calibration. Journal of Experimental Psychology:
[rTAS] Human Perception and Performance. [rTAS]
Pennebaker, J. W. & Roberts, T. (1992) Toward a his and hers theory of emotion: Rizzolatti, G. & Arbib, M. (1998) Language within our grasp. Trends in
Gender differences in visceral perception. Journal of Social and Clinical Neuroscience 21(5):188–94. [JPi]
Psychology 11:199 –212. [aTAS] Rochat, P. (1995) The early objectification of the self. In: The self in infancy:
Peper, L., Bootsma, R. J., Mestre, D. R. & Bakker, F. C. (1994) Catching balls: Theory and research, ed. P. Rochat. Elsevier. [RAF]
How to get the hand to the right place at the right time. Journal of Roe, A. W., Pallas, S. L., Hahm, J.-O. & Sur, M. (1990) A map of visual space
Experimental Psychology: Human Perception and Performance 20:591–612. induced in primary auditory cortex. Science 250:818–20. [TK]
[arTAS] Rosenblum, L. D. & Saldaña, H. M. (1996) An audiovisual test of kinematic
Pessoa, L., Thompson, E. & Noe, A. (1998) Finding out about filling in: A guide to primitives for visual speech perception. Journal of Experimental Psychology:
perceptual completion for visual science and the philosophy of perception. Human Perception and Performance 22(2):318–31. [LDR]
Behavioral and Brain Sciences 21(6):723–48. [WMM] Rosenblum, L. D., Schmuckler, M. A. & Johnson, J. A. (1997) The McGurk effect
Peters, B. T., van Emmerik, R. E. A. & Bloomberg, J. J. (2000) The effects of gait- in infants. Perception and Psychophysics 59(3):347{endash}57. [LDR]
cycle events on head movements made to acquire visual targets in the Rosenblum, L. D., Yakel, D. A. & Greene, K. G. (2000) Face and mouth inversion
horizontal periphery. Society for Neuroscience Abstracts 26:459. [GER] affects on visual and audiovisual speech perception. Journal of Experimental
Pick, H. L., Jr. (1986) Reflections on the data and theory of cross-modal infancy Psychology: Human Perception and Performance 26(3):806–19. [LDR]
research. In: Advances in infancy research, ed. L. P. Lipsitt & C. Rovee- Rossetti, Y., Desmurget, M. & Prablanc, C. (1995) Vectorial coding of movement:
Collier. Ablex. [aTAS] Vision, proprioception, or both? Journal of Neurophysiology 74:457– 63.
Pickens, J. (1994) Perception of auditory-visual distance relations by 5-month-old [YC]
infants. Developmental Psychology 30:537–44. [AW-A] Runeson, S. (1974) Constant velocity – not perceived as such. Psychological
Pieron, H. (1952) The sensations. Yale University Press. [aTAS] Research 37:3–23. [rTAS]
Pinker, S. (1985) Visual cognition. MIT Press. [aTAS] (1977) On the possibility of “smart” perceptual mechanisms. Scandinavian
Pisella, L., Gréa, H., Tiliket, C., Vighetto, A., Desmurget, M., Rode, G., Boisson, Journal of Psychology 18:172–79. [NJTT]
D. & Rossetti, Y. (2000) An automatic pilot for the hand in the human (1988) The distorted room illusion, equivalent configurations, and the specificity
posterior parietal cortex: Toward a reinterpretation of optic ataxia. Nature of static optic arrays. Journal of Experimental Psychology: Human Perception
Neuroscience 3:729 – 36. [YC] and Performance 14:295–304. [SR, rTAS]
Pittenger, J. B. & Dent, C. H. (1988) A mechanism for the direct perception of (1989) A note on the utility of ecologically incomplete invariants. International
change: The example of bacterial chemotaxis. Perception 17:119–33. [rTAS] Society for Ecological Psychology Newsletter 4(1):6–9. [SR]

Runeson, S. & Frykholm, G. (1983) Kinematic specification of dynamics as an Spelke, E. S. (1976) Infants intermodal perception of events. Cognitive Psychology
informational basis for person and action perception: Expectation, gender 8:533–60. [BV]
recognition, and deceptive intention. Journal of Experimental Psychology: Stein, B. E. (1998) Neural mechanisms for synthesizing sensory information and
General 112:585–615. [arTAS] producing adaptive behaviors. Experimental Brain Research 123:124 – 35.
Rosenblum, L. D. & Saldana, H. M. (1996) An audiovisual test of kinematic [DJL]
primitives for visual speech perception. Journal of Experimental Psychology: Stein, B. E. & Meredith, M. A. (1993) The merging of the senses. MIT Press.
Human Perception and Performance 22:318–31. [aTAS] [LDR, aTAS]
Sadato, N., Pascual-Leone, A., Grafman, J., Ibanez, V., Deiber, M.-P., Dold, G. & Stein, B. E. & Wallace, M. T. (1996) Comparisons of cross-modal integration in
Hallett, M. (1996) Activation of the primary visual cortex by Braille reading in midbrain and cortex. Progress in Brain Research 112:289–99. [SLP]
blind subjects. Nature 380:526–28. [TK] Stoffregen, T. A. (1994) “Sensory” reference frames and the information for self-
Saito, H., Yukie, M., Tanaka, K., Hikosaka, K., Fukuda, Y. & Iwai, E. (1986) motion versus object motion. Behavioral and Brain Sciences 17(2):332– 33.
Integration of direction signals of image motion in the superior temporal [AHW]
sulcus of the macaque monkey. The Journal of Neuroscience 6:145–57. (1997) Filming the world: An essay review of Anderson’s The reality of illusion.
[rTAS] Ecological Psychology 9:161–77. [rTAS]
Sams, M., Aulanko, R., Hamalainen, M., Hari, R., Lounasmaa, O. V., Lu, S.-T. & (2000) Affordances and events. Ecological Psychology 12:1–28. [YC]
Simola, J. (1991) Seeing speech: Visual information from lip movements Stoffregen, T. A., Adolph, K. E., Thelen, E., Gorday, K. M. & Sheng, Y. Y. (1997)
modifies activity in the human auditory cortex. Neuroscience Letters 127:141– Toddler’s postural adaptations to different support surfaces. Motor Control
45. [LDR] 1:119–37. [KEA]
Sanders, J. T. (1997) An ontology of affordances. Ecological Psychology 9:97–112. Stoffregen, T. A., Bardy, B. G., Smart, L. J. & Pagulayan, R. J. (2000a) On the
[YC] nature and evaluation of fidelity in virtual environments. In: Psychological
Sauvan, X. M. (1999) Are sense specific reference frames so mutually exclusive? issues in the design and use of virtual environments, ed. L. J. Hettinger & M.
Behavioral and Brain Sciences 22(2):337–38. [AHW] W. Haas. Erlbaum (in press). [rTAS]
Savelsbergh, G. J. P., Whiting, H. T. A. & Bootsma, R. J. (1991) Grasping tau. Stoffregen, T. A., Hettinger, L. R., Haas, M., Roe, M. & Smart, L. J. (1999a)
Journal of Experimental Psychology: Human Perception and Performance Postural instability and motion sickness in a fixed flight simulator. Human
17:315 –22. [rTAS] Factors 42: 458–69. [aTAS]
Schiff, W. & Detwiler, M. L. (1979) Information used in judging impending Stoffregen, T. A., Pagulayan, R. J., Bardy, B. G. & Hettinger, L. J. (2000b)
collision. Perception 8:647–58. [rTAS] Modulating postural control to facilitate visual performance. Human
Schmuckler, M. A. (1995) Self-knowledge of body position: Integration of Movement Science 19:203–20. [arTAS]
perceptual and action system information. In: The self in infancy: Theory and Stoffregen, T. A. & Pittenger, J. B. (1995) Human echolocation as a basic form of
research, ed. P. Rochat. Elsevier. [RAF] perception and action. Ecological Psychology 7:181–216. [BH, aTAS]
Schmuckler, M. A. & Tsang-Tong, H. Y. (2000) The role of visual and body Stoffregen, T. A. & Riccio, G. E. (1988) An ecological theory of orientation and the
movement information in infant research. Developmental Psychology 36:499– vestibular system. Psychological Review 95:3–14. [GER, arTAS, NJW]
510. [rTAS] (1991) An ecological critique of the sensory conflict theory of motion sickness.
Schone, H. (1984) Spatial orientation: The spatial control of behavior in animals Ecological Psychology 3:159–94. [GER, arTAS]
and man, trans. C. Strausfeld. Princeton University Press. [aTAS] Stoffregen, T. A. & Smart, L. J. (1998) Postural instability precedes motion
Schöner, G. S., Dijkstra, T. M. H. & Jeka, J. J. (1998) Action-perception patterns sickness. Brain Research Bulletin 47:437–48. [aTAS]
emerge from coupling and adaptation. Ecological Psychology 10(3–4):323– Stoffregen, T. A., Smart, L. J., Bardy, B. G. & Pagulayan, R. J. (1999b) Postural
46. [ JP] stabilization of looking. Journal of Experimental Psychology: Human
Schöner, G. S., Dose, M. & Engels, C. (1995) Dynamics of behavior: Theory and Perception and Performance 25(6):1641–58. [arTAS]
applications for autonomous robot architectures. Robotics and Autonomous Stumpf, C. (1890) Tonpsychologie (Tone psychology). Hirzel. [BV]
Systems 16:213 –45. [KSO] Sur, M., Garraughty, P. E. & Roe, A. W. (1988) Experimentally induced visual
Shanon, B. (1993) The representational and the presentational. Harvester projections into auditory thalamus and cortex. Science 242:1437– 41. [TK]
Wheatsheaf. [EM] Swanston, M. T. & Wade, N. J. (1988) The perception of visual motion during
Sharma, J., Angelucci, A. & Sur, M. (2000) Induction of visual orientation modules movements of the eyes and of the head. Perception and Psychophysics
in auditory cortex. Nature 404(6780):841–47. [JP] 43(6):559–66. [AHW]
Shaw, R., Turvey, M. T. & Mace, W. (1982) Ecological psychology: The Tanaka, K. & Saito, H. (1989) Analysis of motion of the visual field by direction,
consequence of a commitment to realism. In: Cognition and the symbolic expansion/contraction, and rotation cells clustered in the dorsal part of the
process, ed. W. Weimerd & D. Palermo. Erlbaum. [arTAS] medial superior temporal area of the macaque monkey. Journal of
Sherrington, C. S. (1906) The integrative action of the nervous system. Yale Neurophysiology 62:626–41. [rTAS]
University Press. [PF, aTAS] Thelen, E. (1990) Coupling perception and action in the development of skill: A
Slater, A., Quinn, P. C., Brown, E. & Hayes, R. (1999) Intermodal perception at dynamic approach. In: Sensory-motor organization and development in
birth: Intersensory redundancy guides newborn infants’ learning of arbitrary infancy and early childhood, ed. H. Bloch & B. I. Berthenthal. Kluwer. [PF]
auditory-visual pairings. Developmental Science 2:333–38. [RAF] Thelen, E. & Smith, L. B. (1994) A dynamic systems approach to the development
Smart, A., German, P., Oshtory, S., Gaal, G., Barrie, J. M. & Freeman, W. J. (1997) of cognition and action. MIT Press. [KFA, DJL, aTAS, BV]
Spatio-temporal analysis of multi-electrode cortical EEG of awake rabbit. Thinès, G., Costall, A. & Butterworth, G. (1981) Michotte’s experimental
Society for Neuroscience Abstracts 23:488. [KFA] phenomenology of perception. Erlbaum. [AC]
Smeets, J. B. J. & Brenner, E. (1995) Perception and action based on the same Thomas, N. J. T. (1999) Are theories of imagery theories of imagination? An active
visual information: Distinction between position and velocity. Journal of perception approach to conscious mental content. Cognitive Science 23:207–
Experimental Psychology: Human Perception and Performance 21:19–31. 45. [NJTT]
[EB] Titchener, E. B. (1901) Experimental psychology: A manual of laboratory practice:
Smets, G. J. F. (1995) Designing for telepresence: The Delft virtual window Vol. 1. Macmillan. [BV]
system. In: Local applications of the ecological approach to human-machine Todd, J. T. (1981) Visual information about moving objects. Journal of Experi-
systems, ed. P. Hancock, J. Flach, J. Caird & K. Vicente. Erlbaum. [aTAS] mental Psychology: Human Perception and Performance 7:795 – 810. [LSM]
Smith, D. R., Cacioppo, A. J. & Hinman, G. E. (1997) Aviation spatial orientation Treisilian, J. R., Mon-Williams, M. & Kelly, B. M. (1999) Increasing confidence in
in relationship to head position, attitude interpretation, and control. Aviation, vergence as a cue to distance. Proceedings of the Royal Society of London
Space, and Environmental Medicine 68:472–78. [rTAS] 266:39–44. [YC]
Smith, L. B. (1994) Foreward. In: The development of intersensory perception: Turkewitz, G., Lewkowicz, D. J. & Gardner, J. M. (1983) Determinants of infant
Comparative perspectives, ed. D. J. Lewkowicz & R. Lickliter. Erlbaum. perception. In: Advances in the study of behavior, vol. 13, ed. J. S. Rosenblatt,
[aTAS] R. A. Hinde, C. Beer & M. C. Busnel. Academic Press. [RAF]
Soechting, J. F. & Flanders, M. (1992) Moving in three-dimensional space: Frames Turvey, M. T. (1992) Affordances and prospective control: An outline of the
of reference, vectors, and coordinate systems. Annual Review of Neuroscience ontology. Ecological Psychology 4:173–87. [SR]
15:167– 91. [arTAS] (1996) Dynamic touch. American Psychologist 51:1134– 52. [rTAS]
Solomon, H. Y. & Turvey, M. T. (1988) Haptically perceiving the distances Turvey, M. T., Shaw, R. E. & Mace, W. (1978) Issues in the theory of action:
reachable with hand-held objects. Journal of Experimental Psychology: Degrees of freedom, coordinative structures and coalitions. In: Attention and
Human Perception and Performance 14:404–27. [aTAS] performance VII, ed. J. Requin. Erlbaum. [aTAS]
Solomon, H. Y., Turvey, M. T. & Burton, G. (1989) Gravitational and muscular Ullman, S. (1980) Against direct perception. Behavioral and Brain Sciences 3:373–
variables in perceiving rod extent by wielding. Ecological Psychology 1:265– 415.
300. [PAC] Vallbo, D. B. (1974) Human muscle spindle discharge during isometric voluntary

contractions: Amplitude relations between spindle frequency and torque. Acta Warren, D. H. & Rossano, M. J. (1991) Intermodality relations: Vision and touch.
Physiological Scandinavica 90:310–36. [AGF] In: The psychology of touch, ed. M. A. Heller & W. Schiff. Erlbaum. [aTAS]
Van der Kooij, H., Jacobs, R., Koopman, B. & Grootenboer, H. (1999) A Warren, W. H. (1995) Self-motion: Visual perception and visual control. In:
multisensory integration model of human stance control. Biological Handbook of perception and cognition: Perception of space and motion,
Cybernetics 80:299 – 308. [KSO] ed. W. Epstein & S. J. Rogers. Academic Press. [aTAS]
Van Emmerik, R. E. A. & Wagennar, R. C. (1996) Dynamics of movement Warren, W. H., Blackwell, A. W., Kurtz, K. J. & Hatsopoulus, N. G. (1991) On the
coordination and tremor during gait in Parkinson’s disease. Human Movement sufficiency of the velocity field for perception of heading. Biological
Science 15:203 – 35. [GER] Cybernetics 65:311–20. [aTAS]
Van Emmerik, R. E. A., Wagenaar, R. C., Winogrodzka, A. & Wolters, E. C. (1999) Warren, W. H. & Whang, S. (1987) Visual guidance of walking through apertures:
Axial rigidity in Parkinson’s disease. Archives of Physical Medicine and Body-scaled information for affordances. Journal of Experimental Psychology:
Rehabilitation 80:186 – 91. [GER] Human Perception and Performance 13:371–83. [BH]
Van Ingen Schenau, G. J. (1980) Some fundamental aspects of biomechanics of Werner, H. (1934) Unité des sens. Journal de Psychologie Normale et Pathologique
overground versus treadmill locomotion. Medicine and Science in Sports and 31:190–205. [AC]
Excercise 12:257– 61. [aTAS] Weingarten, M. & Spinelli, D. (1966) Retinal receptive field changes produced by
Vogel, S. (1988) Life’s devices. Princeton University Press. [aTAS] auditory and somatic stimulation. Experimental Neurology 15:363–73.
Von Bekesy, G. (1959) Similarities between hearing and skin sensations. [aTAS]
Psychological Review 66:1–22. [aTAS] Welch, R. B. & Warren, D. H. (1986) Intersensory interactions. In: Handbook of
Von Ehrenfels, C. (1890) Über Gestaltqualitäten. Vierteljahrsschrift für human perception and performance, ed. K. R. Boff, L. Kaufman & J. P.
wissenschaftliche Philosophie 14:249–92. [AC] Thomas. Wiley. [aTAS]
Von Helmholtz, H. (1962) Treatise on physiological optics, trans. J. P. Southall. Wertheim, A. H. (1981) On the relativity of perceived motion. Acta Psychologica
Dover. (Original edition published in 1924.) [aTAS] 48:97–110. [rTAS, AHW]
Von Hornbostel, E. M. (1925) Die Einheit der Sinne. Melos, Zeitschrift für Musik (1987) Retinal and extraretinal information in movement perception: How to
4:290 –97. (Also published in English as: The unity of the senses. In: A source invert the Filehne illusion. Perception 16:299–308. [rTAS]
book of Gestalt psychology, ed. W. D. Ellis. Routledge & Kegan Paul, 1938). (1994) Motion perception during self-motion: The direct versus inferential
[AC] controversy revisited. Behavioral and Brain Sciences 17(2):293–355.
Von Melchner, L., Pallas, S. L. & Sur, M. (2000) Visual behaviour mediated by [arTAS, AHW]
retinal projections directed to the auditory pathway. Nature 404(6780):871– (1999) Motion percepts: “Sense specific,” “kinematic,” or. . . .? Behavioral and
76. [TK, DJL, JP, SLP] Brain Sciences 22(2):338–40. [AHW]
Wade, N. J. (2000) William Charles Wells (1957–1817) and vestibular research Witkin, H. A. & Asch, S. E. (1948) Studies in space orientation: IV. Further
before Purkinje and Flourens. Journal of Vestibular Research 10:127–13. experiments on perception of the upright with displaced visual fields. Journal
[NJW] of Experimental Psychology 38:762–82. [aTAS]
Wade, N. J. & Swanston, M. (1991) Visual perception: An introduction. Routledge. Wong, S. C. & Frost, B. J. (1978) Subjective motion and acceleration induced by
[arTAS] the movement of the observer’s entire visual field. Perception and
Walker, A. S. (1982) Intermodal perception of expressive behaviors by human Psychophysics 24:115–20. [rTAS]
infants. Journal of Experimental Child Psychology 33:514–35. [AW-A] Wood, E. R., Dudchenko, P. A. & Eichenbaum, H. (1999) The global record of
Walker-Andrews, A. S. (1988) Infants’ perception of the affordances of expressive memory in hippocampal neuronal activity. Nature 397(6720):613–16.
behaviors. In: Advances in infancy research, ed. C. K. Rovee-Collier. Ablex. [KFA]
[RAF] Yonas, A. & Granrud, C. E. (1985) The development of sensitivity to kinetic,
(1997) Infants’ perception of expressive behaviors: Differentiation of multimodal binocular, and pictorial depth information in human infants. In: Brain
information. Psychological Bulletin 121(3):437–56. [RAF] mechanisms and spatial vision, ed. D. Ingle, M. Jeannerod & D. Lee. Nijhoff.
Walker-Andrews, A. S. & Gibson, E. J. (1986) What develops in bimodal [KFA]
development? In: Advances in infancy research, ed. L. P. Lipsitt & C. Rovee- Zaal, F. T. J. M., Bootsma, R. J. & van Wieringen, P. C. W. (1998) Dynamics of
Collier. Ablex. [BV] reaching for stationary and moving objects: Data and model. Journal of
Walker-Andrews, A. S. & Lennon, E. (1985) Auditory-visual perception of Experimental Psychology: Human Perception and Performance (in press).
changing distance by human infants. Child Development 56:544– [aTAS]
48. [AW-A] Zangaladze, A., Epstein, C. M., Grafton, S. T. & Sathian, K. (1999) Involvement of
Wallace, M. T. & Stein, B. E. (1997) Development of multisensory neurons and visual cortex in tactile discrimination of orientation. Nature 401:587– 90.
multisensory integration in cat superior colliculus. Journal of Neuroscience [TK]
17(7):2429 – 44. [DJL] Zeki, S. (1993) A vision of the brain. Blackwell. [EM]

99e9 PDF

Uploaded by

Copyright:

Available Formats

99e9 PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

99e9 PDF

Uploaded by

Copyright:

Available Formats

BEHAVIORAL AND BRAIN SCIENCES (2001) 24, 195–261

Printed in the United States of America

On specification and the senses

1. Introduction teraction is specified in the spatiotemporal structure of am-

© 2001 Cambridge University Press 0140-525X/00 $12.50 195

196 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 197

198 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 199

200 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 201

202 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 203

204 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 205

206 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

Figure 4. The global array, represented by a n-dimensional

the referents discussed in section 4. Information about

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 207

208 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 209

210 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 211

212 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 213

214 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 215

216 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 217

218 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 219

220 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 221

222 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 223

224 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 225

226 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 227

228 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 229

230 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 231

232 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 233

234 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 235

236 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 237

238 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 239

240 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 241

242 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 243

244 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 245

246 BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2

BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 247