C. Casanova and M. Ptito (Eds.

)
Progress in Brain Research, Vol. 134
 2001 Elsevier Science B.V. All rights reserved

CHAPTER 27

The metamodal organization of the brain

Alvaro Pascual-Leone Ł and Roy Hamilton

Behavioral Neurology Unit, Department of Neurology, Beth Israel Deaconess Medical Center, Harvard Medical School, 330 Brookline
Avenue, Boston, MA 02215, USA

Introduction Such intersensory perceptual phenomena (Stein and

Meredith, 1993) are many and there is no question
Confronted with the question of how we perceive that the experience in one sensory modality influ-
the world, we are often taught that we have a se- ences the experience in another. Think, for example,
ries of distributed systems structured according to of listening to music over the radio or while watch-
the sensory modalities that they process. We talk ing an orchestra perform. The melody played by the
about a visual system, a somatosensory or tactile violins seems to ‘sound’ much clearer when we see
system, an auditory system, and so forth. Certainly, the instrumentalists and watch their fingering and
the existence of specialized detectors or receptors for bowing than when we do not. The ‘ventriloquism
different sensory modalities grants us the opportu- effect’ broadly refers to this phenomenon on which
nity to process different forms of energy and hence much of our knowledge of the world and an entire
capture different ‘views’ of the world in parallel. entertainment industry is based (Howard and Tem-
Some experiences are uniquely unimodal. Hue can pleton, 1966; Stein and Meredith, 1993). Multimodal
only be experienced by sight, tickle can only be felt association areas that contain multisensory cells neu-
by touch, and pitch can only be differentiated by rons are thought to provide a neural substrate for
audition. Nevertheless, our perceptual experience of integrating (‘merging’) sensory experiences, modu-
the world is richly multimodal as eloquently elabo- lating the saliency of stimuli, assigning experiential
rated by Barry Stein and Alex Meredith (Stein and and affective relevance, and providing the substrate
Meredith, 1993). We are able to extract informa- for the true perceptual experience of our rich world
tion derived from one sensory modality and use it (Stein and Meredith, 1993).
in another; we can, for example, know a shape by However, studies on early blind subjects and visu-
touch and identify it correctly by sight. This is re- ally deprived sighted subjects raise questions about
minds us of Molyneaux’s question, and raises the the organization of the brain in parallel unimodal
broad issue of internal versus experiential influences sensory systems that are eventually integrated in
in the organization of the brain. Furthermore, we multimodal association cortical regions. One may
are able to integrate into a richer percept the im- argue that early blindness represents a pathological
pressions generated by different sensory modalities. state resulting in substantial cross-modal brain plas-
ticity (Rauschecker, 1995; Hamilton and Pascual-
Ł
Corresponding author: A. Pascual-Leone, Behavioral
Leone, 1998; Kujala et al., 2000), but the findings
Neurology Unit, Department of Neurology, Beth Is- in visually deprived sighted subjects are difficult to
rael Deaconess Medical Center, 330 Brookline Avenue, account for with such an argument. After providing
Boston, MA 02215, USA. Tel.: C1-617-667-0203; Fax: a historical backdrop for the topic and reviewing
C1-617-975-5322; the experimental data, we will hypothesize that the
E-mail: [email protected] brain might actually represent a metamodal structure

CICERO/GALAYAA B.V. / CASANOVA 27: pp. 1-19

2

organized as operators that execute a given function experiment would not be able to equate his tac-
or computation regardless of sensory input modality. tile experience to visual percepts. Locke considered
Such operators might have a predilection for a given Molyneux’s query in his 1690 Essay Concerning Hu-
sensory input based on its relative suitability for man Understanding, and stated that “the blind man,
the assigned computation. Such predilection might at first, would not be able with certainty to say which
lead to operator-specific selective reinforcement of was the globe, which the cube : : : ”
certain sensory inputs, eventually generating the im- Molyneux was certainly not the first to ponder the
pression of a brain structured in parallel, segregated relationship between vision and touch in the mind.
systems processing different sensory signals. In this In a famous passage in his Dioptrics (1637), Rene
view, the ‘visual cortex’ is only ‘visual’ because we Descartes considers how a blind man might build up
have sight and because the assigned computation of a perceptual world by tapping objects around him
the striate cortex is best accomplished using retinal, with a stick. He first considers a sighted person using
visual information. Similarly, the ‘auditory cortex’ a stick in darkness, and says:
is only auditory in hearing individuals and only be-
cause the computation performed by the temporal, : : : without long practice this kind of sensation is
rather confused and dim; but if you take men born
perisylvian cortex is best implemented on cochlear,
blind, who have made use of such sensations all
auditory signals. However, in the face of visual de- their life, you will find they feel things with perfect
privation, the ‘striate cortex operator’ will unmask its exactness that one might almost say that they see
tactile and auditory inputs to implement its assigned with their hands : : :
computation using the available sensory information.
Descartes goes on to argue that normal vision
Molyneaux’s question resembles a blind man exploring and building up
his sense by successive probes with his stick. In
In 1688, the Irish philosopher William Molyneux 1709, Bishop George Berkeley agreed that there was
posed a question in a letter to John Locke that no necessary connection between a tactile world
would capture the attention of cognitive psycholo- and a sight world, that a connection between them
gists and philosophers of mind for more than three could only be established through experience. Over
centuries (Degenaar, 1996). Probably motivated to a century and a half later, William James argued
some degree by the fact that his own wife was blind, similarly that perceptions of space across different
Molyneaux posed the following question to his En- modalities have intrinsically different properties that
glish contemporary: make it impossible for cross-modal information to be
relayed between sight and touch without experience
Suppose a man born blind, and now adult, and (James, 1890).
then taught by his touch to distinguish between Not all great thinkers agreed that Molyneux’s
a cube and a sphere of the same metal, and the question can be answered in the negative. The
same bigness, so as to tell, when he felt one and respected and versatile scholar Gottfried Wilhelm
the other, which is the cube, which is the sphere.
Liebniz wrote a critical commentary on Locke’s
Suppose then, the cube and the sphere placed on a
table, and the blind man to be made to see. Query,
empirical approach to visual and tactile perception.
whether by sight, before he touched them, he could Leibniz argued that while visual and tactile sensa-
distinguish, and tell, which is the globe, which is tions of forms are different, the accompanying con-
the cube? cepts underlying their perceptions are the same or
share something critical in common. Leibniz makes
Molyneux’s question has served as the touchstone an explicit distinction between images and ideas,
for over three centuries of debate over the role of and argues that a person blind from birth would be
experience in the development of perception and the capable of distinguishing a sphere from a cube us-
existence of innate or acquired cross-modal relation- ing his understanding and the knowledge acquired
ships between the senses. Molyneux himself came to through the sense of touch with the aid of exact ideas
the conclusion that the blind subject of his thought of the forms of both objects. Francis Hutcheson,

CICERO/GALAYAA B.V. / CASANOVA 27: pp. 1-19

 3

another Irish philosopher, gave a similar answer to which was the Cat, and which was the Dog, he
Molyneux’s question. According to Hutcheson, we was ashamed to ask; but catching the Cat (which
can, by virtue of our ideas about form, judge by he knew by feeling) he was observed to look at
touch what the visual extension of a touched object her steadfastly, and then setting her down, said, So
will be when we open our eyes, although we cannot Puss! I shall know you another Time.
tell by touch which color we will see.
It has been similar with other patients in the 250
Surgical answers to a philosophical debate years since Cheselden’s operation: nearly all had
experienced the most profound, Lockean confusion
Molyneux’s question was approached experimen- and bewilderment. The best known and most influ-
tally in 1728, when William Cheselden, an English ential account of these patients is provided by Von
surgeon, removed the cataracts from the eyes of a Senden (1932). Reporting on 60 cases, Von Senden
13-year-old boy born blind. Cheselden’s celebrated attempted to investigate whether the tactile impres-
case was, by no means the first successful oper- sions of blind people provide them with any spatial
ation of its kind: the earliest reported dates from awareness and he also sought to discover how spatial
A.D. 1020, of a 30-year-old man operated upon in awareness develops in blind people who have been
Arabia, and several other cases were reported be- surgically treated.
tween that first operation and the one performed by Von Senden came to the conclusion that imme-
Cheselden. Chelselden’s commentary on the case, diately after the operation the patients were able to
however, stands out for its consideration of the distinguish objects one from the other and could lo-
changes in percept experienced post-surgically by calize them at a distance, but they could not identify
the patient. any objects. When it appeared that they could iden-
tify objects, he attributed it to the fact that they were
When he first saw, he was so far from making any using other senses or had prior knowledge of the
Judgement about Distances, that he thought all Ob- situation. Psychologists Richard Gregory and Jean
jects whatever touched his Eyes, (as he expressed Wallace dispute Von Senden’s conclusions on the
it) as what he felt, did his Skin; and thought no
grounds of their own study of a blind person who
Objects so agreeable as those which were smooth
and regular, though he could form no Judgment of
had been operated on (Gregory and Wallace, 1963)
their Shape, or guess what it was in any Object that and could recognized day-to-day objects, such as ta-
was pleasing to him: bles and chairs, immediately after the operation. Ex-
perimental approaches to Molyneux’s problem have
Despite his high intelligence and youth, the boy included visual deprivation in animals and the use of
encountered profound difficulties with the simplest ‘sensory substitution systems’ which represent visual
visual perceptions. He had no idea of space or size, scenes and objects tactually (Morgan, 1977). How-
and found it remarkably difficult to recognize and ever, we still lack unequivocal evidence with which
remember the visual form of objects, and had to to answer Molyneux’s 300-year-old conundrum.
constantly relate them to his previously acquired It seems clear from the experimental evidence as
tactile impressions. well as from a careful philosophical dissection of
the question itself, that there are indeed two different
He knew not the Shape of any Thing, nor any problems posed: “ : : : whether by sight, before he
one thing from another, however different in Shape, touched them, he could distinguish, and tell, which
or Magnitude; but upon being told what Things
is the globe, which is the cube”. In order to ‘dis-
were, whose Form he before knew from feeling,
he would carefully observe, that he might know
tinguish’ Molyneaux’s hypothetical subject would
them again; but having too many Objects to learn merely need to differentiate the two stimuli on the
at once, he forgot many of them; and (as he said) basis of sight. In order to ‘tell’ the subject would
at first he learned to know, and again forgot a have to learn by matching the newly experienced
thousand Things a Day. One particular only (though visual sensation to the old concept formed by touch
it might appear trifling) I will relate; Having forgot alone. It is likely the subject would only be able

CICERO/GALAYAA B.V. / CASANOVA 27: pp. 1-19

4

to ‘distinguish’, just as Chelsenden’s patient could meets in the pineal gland, and that information from
differentiate between his cat and his dog by looking one sensory modality interacts with information pro-
at them but needed to touch them to know (to ‘tell’) cessing from another modality (Descartes, 1976).
which one was actually the cat and which the dog. Unimodal models of sensory processing did not
Hence, we may conclude that senses are merged emerge until it had become clear that different
perhaps at different levels. Knowledge (‘telling’) re- brain regions were responsible for mediating dif-
quires the high-order integration of what is perceived ferent functions. Localization of cognitive processes
with what is recalled. Therefore, the patients of to particular regions of the brain began in early
Von Senden and others actually experienced a vi- 19th century with Francis Gall and J.C. Spurzheim,
sual agnosia, the inability of using information in who postulated that there were “about thirty-five af-
one sensory modality (vision) to access metamodal fective and intellectual faculties” in the brain (Gall
knowledge. Obviously, not knowing how to use spe- and Spurzheim, 1835). In one of the best known
cific sensory information for knowledge does not misapplications of an essentially good idea, Gall
imply that merging of the senses does not occur. and Spurzheim subsequently founded the study of
phrenology. The central ideas of their phrenological
How merged are the senses? system were that the brain was an elaborately wired
machine for producing behavior, thought, and emo-
The notion of information from different sensory tion, and that the cerebral cortex was a set of organs
modalities being merged and even being processed with different functions. They postulated that the fac-
together in the same brain areas is far from new1 1 . ulties of the brain were localized in specific organs
The medieval model of the mind contained within of the cerebral cortex and that the development or
it the concept of a ‘sensus communis’, a place in prominence of these faculties was a function of their
the brain where information from all modalities was activity, such that the amount of activity would be
summed into an integrated whole that could be uti- reflected in the size of the cortical organ. They thus
lized by the higher cognitive faculties of reason and reasoned that the size of each cortical organ, and
memory. Since medieval ‘neuroscientists’ could not hence the prominence of mental faculties would be
believe something as grand as consciousness could reflected by the prominence of cranial bumps on the
emerge from the mere physical tissues of the brain, overlying skull (Gall and Spurzheim, 1835).
this so-called ‘common sense’ was thought to be lo- Phrenology had wide appeal, especially in Eng-
cated in the lateral ventricles, which early anatomists land and the United States, and among many leading
collapsed into a single ventricle. The sensations intellectuals, but met considerable opposition from
merged in the sensus communis yielded images, the religious, political, and scientific establishments
and thus, fantasy and imagination were often located of the day. The most important and influential sci-
here as well. The second or middle ventricular space entific critique of Gall came from Pierre Fluorens,
was the site of cognitive process: reasoning, judg- later professor of natural history at the Sorbonne in
ment, and thought; the third was the site of memory Paris. Starting in the 1820s and continuing for over
(Gross, 1998). With the exception of some minor al- 20 years, he carried out a series of experiments on
terations, the multimodal view of sensory processing the behavioral effects of brain lesions, particularly in
remained popular among intellectuals throughout the pigeons. Fluorens reported that lesions of the cere-
Middle Ages and Renaissance. For example, in his bral hemisphere had devastating effects on willing,
Treatise on Man (1662), Renee Descartes, proposed judging, remembering, and perceiving but that the
that sensory information from different modalities site of lesions seemed irrelevant: all regions of the
hemispheres contributed to these functions. The only
exception was vision, in that unilateral, focal lesions
1
Much of the information discussed in the following produced only contralateral blindness (Clarke and
section is derived from Charles Gross’ superb book on the Jacyna, 1987; Gross, 1998). Despite the fact that
history of the Neurosciences. The readers are encouraged these holistic results tended to eclipse Gall’s ideas
to read this source (Gross, 1998). of discreet localization, the overall impact of Fluo-

CICERO/GALAYAA B.V. / CASANOVA 27: pp. 1-19

 5

rens’ work was to support Gall’s broader notion that of specific nerve energies. Under the influence of
cognitive function could be localized to the cere- this doctrine, in the later part of the 19th century,
bral cortex. By the third quarter of the 19th century, neural pathways were traced from the sense organs
Gall’s general ideas had been confirmed by Broca’s into the brain to find the specific regions in which
demonstration of an association between damage of they ended. The cortex was divided up into sepa-
the frontal lobes and aphasia, and again by Fritsch rate centers or organs on the basis of the pattern
and Hitzig’s experiments on stimulation of the mo- of its structure, thereby yielding the techniques of
tor cortex (Gross, 1998). Thus while the practice cytoarchitectonics and myeloarchitectonics. Cortical
of phrenology is no longer held in high esteem as lesions were made in animals to find the sensory
an index of mental faculties, the notion of cognitive centers, and in close parallel, attempts were made
function being subdivided and routed to different to correlate sensory losses in humans with sites of
regions of the brain has remained a central idea in cortical damage. Nowadays, for each of the sensory
neural science and a fundamental principle in the modalities, we assume that there exists a hierarchi-
clinical practice of neurology. cally organized system that begins with specialized
The last quarter of the 19th Century saw an receptors that ‘feed’ unimodal, primary cortical. A
intense search for localization of the sensory cen- series of secondary areas unimodally integrate differ-
ters in the cortex. Interestingly, a notable shift oc- ent aspects of the processed information. Eventually,
curred from the concept of function-specific centers multimodal association areas integrate the processed
to modality-specific brain regions. A major spur in signals with information derived from other sensory
the search for modality-specific cortical regions was modalities. Certainly, top-down influence from mul-
Johannes Müller’s doctrine of specific nerve ener- timodal association areas can change the function of
gies. This doctrine had three elements. The first and ‘hierarchically lower’ unimodal cortical regions. Fur-
most important was the revolutionary notion that per- thermore, in a modern-age experimental testing and
sons are aware of the states of their sensory nerves, refinement of von Helmholtz’s and Du Bois-Rey-
not of the state of the outside world itself. Secondly, mond’s hypotheses, Sur and colleagues have shown
when a given nerve type or nerve energy was ex- the enormous dependence of cortical development
cited, the same type of experience is produced no on external influences (Sur et al., 1990; Merzenich,
matter what the stimulus. The third element of this 2000). Sur and colleagues rewired the brain of young
doctrine was that the same physical stimulus ap- ferrets so that the retinal nerves were directed to
plied to different sense organs gives rise to different grow into the auditory thalamus, which was deprived
sensations. Hence, a similar blow to the eye and of its normal auditory inputs. This intervention re-
to the ear produce visual and auditory sensations, sults in the emergence of a functional primary visual
respectively (Müller, 1965). Subsequently Hermann cortex in the brain area destined to develop into the
von Helmholtz, the renowned physician, physicist, primary auditory cortex (Sur et al., 1990; Roe et al.,
and psychologist, and a student of Müller, argued 1990; Sharma et al., 2000). The ‘new’ visual cortex
that the specificity of nerve energies could be at- is topographically organized and its neurons are se-
tributed to the specific brain regions upon which lective for differently oriented visual stimuli and are
they terminate. He compared nerves to wires through arranged in visual orientation columns, much like it
which impulses between modalities are essentially would be expected in the ‘normal’ primary visual
identical, but take on different meaning based on the cortex. Furthermore, this ‘new visual cortex’ shows
portion of the brain to which they are routed. Emil behavioral functionality (Melchner et al., 2000). Us-
Du Bois-Reymond, another of Müller’s students, and ing a variant of this original paradigm, Melcher
the discoverer of the action potential, went one step and colleagues rewired cortical circuits on only one
further and claimed that if it were possible to cross- side of ferrets’ brains, using inputs to the unaffected
connect the auditory and optic nerves, we would hemisphere as a control. They then trained ferrets
see with our ears and hear with our eyes (Gross, on a task in which different responses elicited dif-
1998). Müller’s doctrine of specific nerve energies ferent rewards depending on whether the stimulus
thus became directed toward the cortex as the locus was visual or auditory. They found that when light

CICERO/GALAYAA B.V. / CASANOVA 27: pp. 1-19

6

stimuli were presented in the portion of the visual systems to adjust to blindness in a society that re-
field that is mediated only by rewired projection, lies heavily on vision. Blind subjects need to extract
the ferrets responded as though they perceived the crucial spatial information from touch and hearing.
stimuli to be visual rather than auditory. Literally, Perhaps the most instructive approach to the expe-
the rewired ferrets ‘see’ with the brain area that was rience of blindness is reading the several excellent
meant to be the primary auditory cortex. Importantly, accounts of the experiences of adults who became
these animals do not become confused nor display blind and had to adapt to their disability. One of
behavior that might suggest synaesthetic merging of the most compelling and lucid accounts of blindness,
the senses. Rather, they appear to have developed is John Hull’s autobiography ‘Touching the Rock’.
a fully functional visual cortex with its appropriate, Several passages should be underscored. For exam-
source-specific perceptual qualities. ple, Mr. Hull relates how he ‘knows’ his youngest
This functional ‘redefinition’ of the ‘auditory’ son, born after Hull became blind due to diabetes
cortex into a new ‘visual’ cortex requires rewiring differently than he ‘knows’ his other children whom
of the retinal, visual inputs and depriving the medial he saw being born and growing up. Another remark-
geniculate nucleus in the thalamus of its cochlear, able passage is the description of how he enjoys
auditory inputs. Furthermore, the experiments by Sur stepping outside in the rain. It is the sound that the
and colleagues are successful in very young ferrets rain makes on the different objects and planes that
that are born very early in their neural development allows him to create a mental image of the space that
Sur et al., 1990). Could it be, that in fact, in normal surrounds him. Indeed, blind people often do not de-
development, multimodal sensory inputs feed into scribe space, but rather describe specific objects and
all cortical regions and that external influences and routes that can guide a listener around the room or
predefined functional superiority of given cortical from one place to another: “As you enter my house
areas shape the brain? Could it be the case that the you will be able to feel the light switches on the
brain is actually metamodal, and that the impres- left, by the door frame. If you then turn to the left
sion of hierarchically structured unimodal systems and take a couple of short steps there will be a coat
is only the consequence of selection of functions by rack and next to it, on the right, you will feel the
groups of neural networks that compete with each door opening to the kitchen : : : ”. There is a lack of
other in order to acquire specific processes? One more global space perception, and substitution with
might envision a cortical region with a functional a percept that inches along, guided by touch.
role in spatial discrimination, that might therefore be Braille reading provides blind subjects with the
predisposed to perform the kinds of processes that opportunity to read and write communication, thus
vision requires. In this setting, from early develop- greatly expanding their integration into society and
ment onward, sight would be progressively selected their opportunities for employment. Braille reading
as the input signal for such an ‘operator’. Eventually requires the processing of tactile information into
such an operator might appear to be ‘visual cortex’ meaningful shapes. Subjects have to discriminate
by virtue of its dominant input, when in fact, under small patterns of raised dots with the pad of their
certain conditions, the presence of metamodal inputs index finger and extract spatial information through
could be unmasked. In the present article we advance touch rather than vision. In order to accomplish this
such a hypothesis, based on a series of experiments task, subjects move their index finger from side to
in early blind subjects and visually deprived sighted side at a controlled speed so that the sensory skin
volunteers. receptors are maximally activated by the raised dots
and as much information as possible is obtained.
Functional brain organization in the early and Therefore, learning to read Braille poses a great de-
congenitally blind mand on sensory and motor representations from a
rather small part of the body, the distant pad of the
Loss of vision due to injury to the eyes results in index finger, and stresses the spatial coding capa-
deafferentation of very large areas of the human bilities of tactile exploration. Blind, Braille readers
cortex and poses striking demands on other sensory do not have a lower peripheral sensory threshold

CICERO/GALAYAA B.V. / CASANOVA 27: pp. 1-19

 7

in the pad of the reading index finger than blind, Braille symbols evoked by TMS. In sighted subjects,
non-Braille reading controls or sighted volunteers rTMS to the visual cortex does not interfere with the
(Pascual-Leone and Torres, 1993). However, the sen- ability to detect or discriminate embossed roman let-
sory and motor representations of the reading index ters by touch. Therefore, it would appear that Braille
finger in the brain are significantly larger for Braille reading in the blind is an example of true ‘cross-
readers than for blind or sighted non-Braille readers modal sensory plasticity’ by which the deafferented,
(Pascual-Leone and Torres, 1993; Pascual-Leone et formally visual cortex, is recruited for demanding
al., 1993). This enlarged sensorimotor representation tactile tasks, thus enhancing the sensory discrimina-
in blind Braille readers develops slowly, over the tion abilities of the blind subjects and making the
course of months (Pascual-Leone et al., 1993, 1996) acquisition of tactile Braille reading skill possible.
and is modulated by the preceding activity (Pascual- Further support for the role of striate cortex in
Leone et al., 1995). Therefore, changes in represen- the perception of complex tactile stimuli comes from
tation of the reading finger in the brain cortex seem the findings in a most unfortunate patient (Hamilton
to play a critical role in the acquisition of the Braille et al., 2000). She had been blind since birth due
reading skill (Hamilton and Pascual-Leone, 1998). to retinopathy of prematurity. She was a highly
Peripherally blind subjects also have very large proficient Braille reader who used Braille at her
areas of their brain cortex deafferented from input work. She was a proof reader for the newsletter for
and hence parts of the former visual cortex are avail- the Spanish Organization for the Blind, and thus read
able to be recruited for the processing of tactile and Braille for 4–6 hours per day. At age 62, she had
auditory information. Indeed, proficient Braille read- an acute neurological event that resulted in coma
ing by blind subjects activates the dorsal and ventral from which she recovered but was left unable to
portion of the occipital cortex (Fig. 1A) (Sadato et read Braille. She had no difficulties discriminating
al., 1996; Sadato et al., 1998). Furthermore, there is textures or identifying everyday objects by touch, but
suppression of parietal operculum and activation of she was greatly impaired in tactile discriminations
the ventral portion of the occipital cortex in blind that required complex spatial decoding. She made
subjects during tactile discrimination tasks, oppo- frequent errors trying to identify even single Braille
site to the pattern of activation observed in sighted letters and she was completely unable to read Braille
subjects (Sadato et al., 1996, 1998). Studies us- words. She described being able to ‘feel’ the Braille
ing event-related potentials, cerebral blood flow, and dots, but she could not ‘make sense’ of what she
magnetoencephalography also suggest occipital cor- was touching. She complained that it felt “as if I
tex activation by tactile stimuli activation in early had never learned Braille at all.” Her neurological
blind humans (Hamilton and Pascual-Leone, 1998; examination was otherwise unremarkable, and her
Kujala et al., 2000). Participation of the striate ‘vi- MRI showed a bilateral occipital stroke (Fig. 2).
sual’ cortex in a tactile task seems related to the Therefore, the findings in this case, emphasize the
difficulty of the tactile discrimination regardless of role that the occipital cortex plays in Braille reading
whether there is a lexical component to it or not in early blind subjects.
(Sadato et al., 1996). These findings suggest that Cohen et al. argue for a critical period for this
the pathway for tactile discrimination changes with plasticity (Cohen et al., 1997), such that beyond
blindness. Indeed, in the congenitally blind, interfer- age 14, the striate cortex is no longer recruited for
ence with the function of the occipital cortex during the processing of tactile information. They base this
Braille reading using repetitive transcranial mag- argument on the study of late-blind subjects who
netic stimulation (rTMS) results in disruption of the became blind after age 15, in whom tactile stimula-
Braille reading skill (Fig. 1B) (Cohen et al., 1997). tion failed to reveal activation of the striate cortex
Subjects were aware that they had felt Braille charac- on PET, and in whom rTMS to the striate cortex
ters, but during rTMS to their occipital cortex were did not interfere with reading of embossed Roman
unable to discriminate them. In fact, some of the characters. Nevertheless there are a number of other
subjects reported phantom tactile sensations, feeling potential confounders. For example, late blind in-
Braille dots that were not there or distortions of the dividuals are often less skillful Braille readers, and

CICERO/GALAYAA B.V. / CASANOVA 27: pp. 1-19

8

Fig. 1. (A) Modified from (Sadato et al., 1996, 1998). Activation of the occipital cortex in congenitally and early blind subjects during
tactile Braille reading as demonstrated by positron emission tomography (PET). (B) Modified from (Cohen et al., 1997). Effects of
repetitive transcranial magnetic stimulation (TMS) to the occipital or somatosensory cortex on tactile Braille reading ability in early blind
subjects and on tactile perception of embossed Roman letter in sighted controls.

among the five subject studied, four had residual a lower mean threshold of 1.04 mm (SD D 0.19)
vision that could well affect the plastic changes that compared to 1.46 mm (SD D 0.46) for the sighted
can take place in the brain. group .F D 10:78, df D 1,28, P D 0:003). In-
A relationship between tactile discrimination abil- terestingly, there was a significant Group ð Finger
ity and the amount of activation of the striate cortex interaction .F D 3:03; df D 3,84; P D 0:034) indi-
in the blind is suggested by recent results in early cating that the extent of the difference in thresholds
and congenitally blind subjects. Behaviorally, early between the two groups was not the same across fin-
blind subjects show significant superiority to age- gertips. Post hoc comparisons revealed that among
matched sighted controls in a gratings-orientation the blind subjects, there were significant differences
discrimination task (Fig. 3) (Van Boven et al., 2000). in threshold for the different fingers, such that their
A gratings-orientation threshold was determined for preferred Braille reading finger has the lowest thresh-
the tip of several fingers using the Johnson, Van old (Fig. 3). Sighted controls showed no threshold
Boven, Phillips (JVP) Domes. Blind subjects had differences across fingers.

CICERO/GALAYAA B.V. / CASANOVA 27: pp. 1-19

 9

Fig. 2. Modified from (Hamilton et al., 2000). T2-weighted brain magnetic resonance imaging (MRI) of early blind woman with sudden
onset of alexia for Braille. Note the large, bilateral occipital stroke in distribution of bilateral posterior cerebral arteries.

In these same 15 early blind subjects, we found encountered by touch. There are, however, a few
a significant correlation between the amount of ac- pieces of evidence that suggest that visual imagery is
tivation in striate and peri-striate cortex to tactile not the phenomenon underlying activity across sen-
stimulation of the finger pads as measured by func- sory modalities. In the absence of further evidence,
tional magnetic resonance imaging (fMRI) and the it seems unlikely that early blind subjects, who in
gratings-orientation thresholds for their different fin- many cases have never had any vision whatsoever,
gers (Kiriakopoulos et al., 1999). The blood-oxygen are receiving top-down projections of visual percepts
level dependent (BOLD) activation in occipital cor- that then allow them to construct visual impressions
tex was significantly greater during stimulation of of tactile stimuli. Furthermore, studies with late blind
the preferred reading finger than during stimulation subjects, who often do remember the visual appear-
of adjacent fingers in the same hand or of the homol- ance of objects, suggest that these individuals may
ogous finger in the other hand (Fig. 3). not activate occipital cortical networks during tasks, ?#1
There are certainly alternative explanations as to although this finding is still a topic of debate (Büchel
how, in the blind, primary cortical areas that nor- et al., 1998).
mally mediate vision can become responsive during More interestingly, it may be the case that oc-
tactile tasks. One possibility is that occipital activa- cipital activation in the blind for tactile tasks does
tion in these tasks reflects mental imagery. Kosslyn not represent visual imagery, but instead represents
and colleagues have repeatedly shown that visual what can be thought of as ‘tactile imagery’. Accord-
imagery in normal subjects activates primary occip- ing to this hypothesis, top-down processes rather
ital cortical regions (Kosslyn, 1994). An argument than afferent inputs may be driving occipital corti-
could therefore be made that subjects who activate cal response. Although such a model postulates a
occipital networks during tactile tasks may, to some different ‘direction of flow’ of information from the
extent, be building visual representations of stimuli bottom-up architecture being hypothesized, this view

CICERO/GALAYAA B.V. / CASANOVA 27: pp. 1-19

10

Fig. 3. (A) JVP Domes used for grating orientation discrimination testing. (B) Histogram displaying the thresholds for gratings orientation
discrimination in early blind and sighted control subjects. Note the marked superiority (lower thresholds) of the blind subjects in general
and in particular of their dominant Braille reading finger. Modified from (Van Boven et al., 2000). (C) Modified from (Kiriakopoulos et
al., 1999). Representative example in one early blind subject of the differential activation of somatosensory and occipital cortex during
tactile stimulation of the Braille reading finger and of the homologous finger in the other hand. Note the relative hypoactivity of the
somatosensory cortex and the activation of the occipital cortex during stimulation of the reading finger.

CICERO/GALAYAA B.V. / CASANOVA 27: pp. 1-19

 11

is, nonetheless, entirely consistent with a metamodal few felt that they had become able to detect objects
model of the brain. The localization of tactile im- or furniture in their paths by the “echos of sounds”.
agery to the occipital region would imply that com- Similarly, most of them noted an improved ability to
plex reconstruction of aspects of a stimulus gained differentiate surfaces and identify objects by touch
via touch are being represented in the occipital cor- during the 7 blindfolded days.
tex, possibly because expert networks of neurons in Interestingly, at the end of the week, removal
that region of cortex have acquired, through compe- of the blindfold required a period of several hours
tition, the ability to perform that operation. for readjustment to a visual world, during which
Therefore, it seems that the occipital cortex in time these persons experienced difficulties in visual
early blind and congenitally blind subjects is en- spatial information decoding. Frequently subjects ex-
gaged in spatial tactile processing: it is both required perienced the greatest difficulties when attempting to
for it and its activation is correlated with perfor- cross the street after the blindfold had been removed:
mance on behavioral measures. Is this the case in “it was difficult to judge the height of the curb and it
only the blind? Is seems tempting to consider the was very anxiety provoking not to be confident about
possibility that such a role of the occipital cortex the time it would take for an approaching car to close
is present in all of us, sighted or blind, but that in. I would question whether I had the time to cross
visual input masks detectable evidence of processing the street safely because I could not judge the speed
of information from other sensory modalities. If so, of the cars or how long it would take them to get to
visual deprivation by blindfolding might unmask the me”.
metamodal role of the ‘visual cortex’.
The blindfold experiment
The subjective experience of instructors for the
blind in Spain In order to follow-up on these anecdotal reports by
instructors for the blind in Spain in a systematic
For a number of years, aspiring instructors for the and controlled fashion, we are conducting a study
blind in Spain were required to undergo a period of in which normal sighted volunteers are visually de-
complete blindfolding “to get a first hand experience prived for 5 days (Maguire et al., 1999; Schlaug
of what being blind really is like.” These future et al., 1999; Kauffman et al., 2000; Schlaug et al.,
instructors of the blind lived at a boarding school 2001). During this time they are kept at the General
and spent 1 week blindfolded. The blindfold was Clinical Research Center (GCRC) at Beth Israel Dea-
put on before getting up from bed in the morning coness Medical Center and are intensively trained in
and was worn all day until retiring for the night. tactile and auditory spatial discrimination tasks. In
The students were required to leave the blindfold addition to behavioral testing, subjects undergo se-
on until they were in bed and the lights had been rial fMRI studies to tactile and acoustic stimuli. Sub-
switched off. Most likely all of them cheated some, jects are fitted with a specially designed blindfold
tampered with the blindfold and sneaked a peak into that completely prevents all light perception. Poten-
the light occasionally. However, this was likely held tial tampering with the blindfold by the subjects is
to a minimum since bright light bothered their eyes, controlled with the use of a piece of photographic
which had adapted to darkness. paper attached to the inside of the blindfold and
Most of these trainees report noticing improved analyzed for possible exposure to light at the end
abilities to orient to sounds and judge distance by of the experiment. Non-blindfolded control subjects
sound by the end of the blindfolded week. For ex- are blindfolded temporarily for all serial MRI and
ample, several describe becoming able to identify behavioral experiments.
people quickly and accurately as they started talking MR Imaging is performed using a whole body 1.5
or even as they simply walked by due to the cadence T Siemens Vision EPI system. A 3-D T1-weighted
of their steps. Several learned to differentiate cars MR sequence (1:0 ð 1:0 ð 1:0 mm) is acquired for
by the sounds of their motors, and one described the co-localization with functional images. A set of 18
“joy of telling motorcycles apart by their sound.” A functional MR images (2:5 ð 2:5 ð 4 mm) parallel

CICERO/GALAYAA B.V. / CASANOVA 27: pp. 1-19

12

to the ACPC plane is acquired every 5 s for a total hours after blindfolding and persisted for several
of 40 s=epoch. For the auditory fMRI studies, vol- hours after the blindfold was removed at the end of
unteers perform a tone matching task and a motor the study week. Hallucinations at first included flash-
control task in counterbalanced order. Each task is ing lights, mirrors, lamps, trees, and full landscapes.
repeated 8 times. The matching task consisted of a By the conclusion of the second day of blindfolding,
series of tones and subjects have to compare each the images grew in such complexity that the subject
tone with the previous one, stating whether it is the experienced difficulty walking due to the ‘obstacles’
same or different using a button press. In the mo- he saw. While taking a walk outside, he reported
tor control task, subjects are instructed to alternate seeing “a ground of dirt rows, mounds of pebbles,
button presses between the two responses. For the or little small stones that were running from upper
tactile stimulation task, pairs of Braille symbols are left to lower right field of view and between them
presented to the pad of the right index finger and was running a small stream of water.” Furthermore,
the subject has to report whether they are the same the images became a constant presence to the sub-
or different. A motor control task of alternating but- ject, and by the end of the study week developed
ton presses is again used as control. Image analysis into ornate buildings of whitish green marble and
is done using the AFNI software package (2.2) us- cartoon-like figures. All of the hallucinations were
ing ‘3dvolreg’ for motion correction. All images are recognized as such, and could consciously be cleared
spatially standardized into the Talairach space. A by the subject, if desired.
two-factorial 3D ANOVA analysis is performed on Another subject, a 20-year-old female, experi-
each of the two groups, i.e. blind-folded and control enced an array of hallucinations that first appeared
group, with the first factor being ‘condition’ (tone, approximately 12 h after blindfolding and lasted
rest) and the second factor being ‘scan’ (before, throughout the course of the study week until mo-
during, after blindfolding). The F-tests and planned ments before the blindfold was removed. The hallu-
contrasts are corrected for multiple comparisons. cinations seemed to appear suddenly and transform
Behaviorally, subjects have been tolerating the into a series of different images much like in a dream
blindfolded period well, though all of them have “a butterfly becomes a sunset, that becomes an otter,
developed visual hallucinations that generally met that becomes a flower, that becomes several different
criteria for Charles–Bonnet syndrome (Maguire et types of animals, and it all is kind of this big stream.”
al., 1999). All subjects lacked a history of psy- Hallucinations varied in type from landscapes, such
chopathology, sensory disorders, or ocular pathol- as cities, skies and sunsets, to objects that, as in the
ogy, and all were readily cognizant of their complex, example transcribed above, frequently morphed into
often colorful, hallucinations. Visual hallucinations one another. The subject often described the visions
began shortly after individuals were blindfolded. as if looking at a kaleidoscope, with bright colors
Subjects were instructed to keep a daily journal of and shapes that moved in a continual rhythm: “if
their thoughts, dreams, and perceptions throughout there is a sunset or a sunrise I couldn’t look at the
the blindfolded period. Hallucinations were reported sun — because it was too bright : : : It would seem
as incidences of intrusive visual hallucinations that like all of this light would just collect where the sun
varied in onset, duration, and content. Additionally, was and I just couldn’t look there : : : would get
images also varied in complexity, ranging from Lil- like a spot and it would turn into something else.”
liputian figures to landscapes and cartoon figures. The subject also stressed that the hallucinations were
Hallucinations most often occurred during the sub- beautiful (“it is like art, sometimes they are much
ject’s ambulatory movements, such as when walking prettier I think than anything I’ve ever seen : : : .
to and from the rest room, and also between testing I really wish I could paint.”) and that motion was
sites within the research unit. No obvious memory always involved.
sources were associated with the hallucinations, nor However, the most exciting aspect of these re-
did they have a tendency to repeat themselves. sults is that the serial fMRI studies of these subjects
One subject, a 24-year-old male, experienced a showed activation of the visual cortex during tactile
broader range of images, which commenced a few and auditory stimulation of the fingers. These data

CICERO/GALAYAA B.V. / CASANOVA 27: pp. 1-19

 13

Fig. 4. Representative example of the changes in BOLD activation on functional brain MRI over the course of 5 days of complete
visual deafferentation through blindfolding. Note the increasing activation of occipital cortex, including striate cortex by the fifth day of
blindfolding and the dramatic return to baseline following removal of the blindfold for a single day (despite testing with the blindfold
on).

suggest that the appearance of visual hallucinations Strikingly, the occipital activation with tactile or
in these subjects may be the result of modifications auditory stimulation disappears 12–24 h following
made by the visual cortex in processing both tactile removal of the blindfold (Fig. 4), even though for the
and other non-visual sensory information. As illus- purpose of the fMRI the subjects wear the blindfold
trated in Fig. 4 over the course of the blindfolded again. In other words, removing the blindfold and
period, the striate and peri-striate cortex becomes being exposed to the seeing world for 12–24 h is
increasingly activated during the tactile stimulation. sufficient to revert all changes induced by the 5 days
On the first day of the blindfolding period there is of blindfolding.
activation in the contralateral somatosensory cortex, A TMS experiment (Kauffman et al., 2000) serves
but none in the occipital cortex. On the second and to explore the behavioral significance of the findings
particularly on the fifth day of blindfolding the acti- (Fig. 5). At the last day of blindfolding, rTMS to
vation in the somatosensory cortex is less, but there the occipital cortex of the volunteers significantly
is increasing BOLD activation in occipital, ‘visual’, disrupts their newly acquired tactile Braille symbol
regions. Similarly, activation of striate cortex dur- discrimination ability. The findings are similar to
ing auditory stimulation is seen at the end of the those found in early blind subjects. The sighted,
blindfold period and was absent prior to the blind- non-blindfolded control subjects do not show such
folding or in the control subjects (Schlaug et al., results. Interestingly, and in accordance with the
1999, 2001). fMRI results, 1 day after removal of the blindfold,

CICERO/GALAYAA B.V. / CASANOVA 27: pp. 1-19

14

Fig. 5. Modified from (Kauffman et al., 2000). Representative example of the effects of repetitive TMS to the occipital cortex onto the
tactile Braille symbol discrimination ability in a subject after 5 days of complete visual deafferentation through blindfolding. Note the
comparison with a subject undergoing the same experimental procedures but never blindfolded and with a congenitally blind control.

disruption of the occipital cortex by rTMS no longer choice of the animal models and anesthetic agent.
impairs tactile Braille reading ability. Most importantly, however, the competitive selec-
These results suggest that the occipital cortex had tion among expert systems that may determine the
become recruited for processing tactile and auditory functional specificity of cortical modules would be
information when it was deafferented of visual input. expected to powerfully suppress less suitable inputs.
However, as soon as visual input became available, Hence, experiments in selectively sensory deprived
even transiently, the induced changes rapidly re- animals might be needed.
versed to baseline. The speed of these changes is There is some evidence available that has indi-
such that establishment of new connections is not cated that neurons in striate cortex have auditory and
possible. Therefore, it must be assumed that tactile tactile in addition to visual inputs. However, these
and auditory input into ‘visual cortex’ is present results have been debated and gone largely without
in all of us and can be unmasked if behaviorally reliable reproduction by other investigators. Frank
desirable. Morell (Morell, 1972) studied visual and acoustic
responses of neurons in parastriate (18 and 19) areas
Anatomical support in adult cats using microelectrode recording tech-
niques. Seventy of 169 neurons recorded (41.4%)
Such a hypothesis requires some anatomical support. were responsive to visual and acoustic stimulation.
One might expect that multimodal sensory responses Interestingly, none of these neurons demonstrated a
of neurons in primary ‘visual’ cortex ought to be frequency tuning curve analogous to those of the
demonstrable and that tracing studies would have ‘primary auditory cortex’ to acoustic stimuli of 5–50
already revealed pathways relating information from kHz. However, the responsiveness of ‘visual cells’
the medial lemniscus, somatosensory thalamus, me- to acoustic stimuli depended on the location of the
dial geniculate, or other structures to the striate sound source in space. Fishman and Michael (Fish-
cortex. In fact, the evidence is rather scarce. man and Michael, 1973) studied striate cortex (area
A number of possible explanations can be dis- 17) also using microelectrode recordings in unanes-
cussed to account for the paucity of supporting evi- thetized cats, and similarly found that 38% of the
dence from animal models, including, for example, recorded cells responded to both auditory and visual

CICERO/GALAYAA B.V. / CASANOVA 27: pp. 1-19

 15

stimuli. As in the study by Morell (1972), bimodal function correspondences in the brain. Because the
cells had coincident acoustic and visual receptive structural properties of different brain regions differ
fields and were distributed in functionally segre- (e.g. different patterns of neural connectivity), dif-
gated, anatomically distinct clusters. Several previ- ferent brain regions are best at performing particular
ous studies had yielded similar findings (Lomo and types of functions. The second notion is that brain
Mollica, 1962; Jung et al., 1963; Murata et al., 1965; regions compete for the ability to perform a set of
Spinelli et al., 1968). Therefore, these findings argue tasks. This competition leads to functional special-
that neurons in striate and peristriate cortex might be ization of brain regions but also in the progressive
engaged in the spatial localization of stimuli regard- segregation of the best suited inputs. This competi-
less of sensory modality and might indeed be tuned tion is not an altogether fair fight. It is biased by
to specific topography along orientation columns. the structure–function correspondences; each region
The anatomical pathways by which acoustic or tac- tends to win the competition for those functions for
tile information reaches striate cortex are unclear. which its structure makes it particularly well-suited.
Thalamo-cortical or cortico-cortical pathways can be ME architecture incorporates the two ideas of
hypothesized but neither has been demonstrated to competition and structure–function correspondences
date. in a model that consists of two types of networks:
expert networks and a gating network. The system
Metamodal brain is designed to try to learn training patterns, which
consist of an input along with a desired output.
Based on the data presented, the visual cortex seems The expert networks compete with each other to
to be a metamodal structure that receives not only learn training patterns, while the gating network
visual, but also auditory and tactile stimuli. These mediates this competition. The output of the entire
inputs can be unmasked and functionally demon- architecture, denoted y, is the linear combination of
strated as they gain functional relevance during vi- the experts’ outputs:
sual deafferentation. Throughout these changes the X
n
yi1 D gi y I
‘visual cortex’ appears to subserve spatial discrim-
ination tasks regardless of the sensory input pro- where y I denotes the output of the i-th expert net-
cessed. Based on this example of the visual cortex, work and gi is the gating network output corre-
we propose that the brain, in a revised version of sponding to the i-th expert. The connection strengths
the views of the early 19th Century, is made up of the expert and gating networks are adjusted si-
of metamodal operators. These operators are local multaneously during training. Each expert network’s
neural networks defined by a given computation that output is compared with the target output at each
is applied regardless of the sensory input received. time step. The expert whose output most closely
This does not mean that there are not preferred matches the target is called the winner; the others
sensory modalities for specific computations (and are losers. The winning expert wins information and
hence operators). Indeed, this is the case and the thus learns a lot about the current training pattern;
reason that the cortex gives the illusion of being built the losers receive little or no information, and thus
around sensory modalities rather than operators. We learn little about the current training pattern. The gat-
shall propose a model by which such a cortical ing network receives information about the relative
architecture built around operators, rather than sen- performances of the experts on the current pattern.
sory modalities might be generated. This proposal is It adjusts the connection strengths so that when the
based on Robert Jacobs’ ‘mixture of experts’ (ME) current input (or similar input) recurs in the future
architecture (Jacobs, 1999). the activation of its output unit corresponding to the
winning expert will be larger and the activation of its
Mixtures of experts architecture remaining output units will be smaller. This learning
process has a positive feedback effect that ensures
The ME architecture implicitly contains two impor- that an expert system that won the competition for a
tant notions. The first is that there are structure– particular learning pattern will be most likely to win

CICERO/GALAYAA B.V. / CASANOVA 27: pp. 1-19

16

the same pattern in the future. This positive feedback computationally efficient model for achieving this
loop also forces different expert networks to learn specialization.
different tasks, since no expert network aside from For the purposes of applying ME architecture to
the one that won before will ever be more likely to biological systems, it is important to note that there
win in the future. Conversely, as an expert network is no need to determine which neural structure corre-
receives and incorporates the information it receives sponds to the gating network. This is because the ME
from the gating network in order to become special- architecture is computationally and thus functionally
ized for a particular task, it will become less likely to equivalent to another architecture that does not con-
compete well on patterns from unrelated tasks. tain gating networks, but instead contains inhibitory
Does ME architecture actually work, and can connections between expert networks that suppress
networks configured with ME architecture learn to each other’s output. The strengths of the inhibitory
make the kinds of distinctions that functionally dis- connections depend on the value of the current input
tinct brain modules seem to be able to make? Nat- pattern, such that, depending on the task at hand,
urally, the only way to find out would be to attempt different expert networks would be more or less ca-
to train these networks on tasks that mimic the kinds pable of winning the competition for that task. At the
of processes we believe occur in brains. Jacobs and end of training, the expert that won the competition
Kosslyn (1994) speculated that different subsystems in the context of the current input, or closely related
in the brain are responsible for making categorical inputs, strongly suppresses the outputs of other ex-
and coordinate visual judgements. In other words, perts. Experts that are the losers in the competition
they hypothesized that different neural agencies (or for a particular task suppress the outputs of other
‘operators’ in our terminology) using different kinds experts weakly or not at all.
of information are responsible for classifying a stim- Given what is currently known about how ME
ulus (e.g. object A is a building) versus identify- architecture works, it is not difficult to theorize how
ing a stimulus as a particular exemplar (e.g. object particular regions of the cortex might have acquired
A is the Leaning Tower of Pisa). They hypothe- functional specificity using a network that approxi-
sized that systems that make categorical judgements mates this architecture. One must first assume that
would be more efficient if they monitored visual there exist initial differences between competing
neurons with small non-overlapping receptive fields neural modules. The occipital cortex, for example,
(and thus with low-resolution representation of vi- might obtain the domain of visual processing be-
sual objects), whereas systems that make coordinate cause it is best suited for computations that may
visual judgements should be more efficient if they benefit from the information best supplied by vi-
monitor neurons with large, overlapping receptive sion. The data transformations at which the occipital
fields (high-resolution representation). Jacobs and cortex may initially excel might not be vision, per
Kosslyn used computer simulations to support this se. Instead, the occipital cortex may initially be a
theoretical structure–function relationship, demon- region of the brain with structural and functional
strating that neural networks with small receptive qualities that enable it to excel at tasks which re-
learn category tasks more quickly, while networks quire high-acuity processing of spatial information.
with large receptive fields learn coordinate tasks If this were the case, one would anticipate that the
more quickly. They then trained competing networks occipital cortical regions would constitute the expert
using small or large receptive field sizes and ME network that would win the competition for vision, a
architecture and found that networks with small re- sensory modality that provides tremendous amounts
ceptive fields tended to win the competition for cate- of distance and spatial information (as compared, for
gory tasks while networks with large receptive fields example, with sound or touch). As the occipital cor-
won the competition for coordinate tasks. This set tex continues to win the competition between expert
of simulations demonstrates that structure–function networks for the input of vision, it is able to perform
relationships and competition between networks can its task (spatial decoding) more and more accurately
lead to functional specialization of processing, and and hence adapts to become better and better at it,
suggests that ME architecture is a plausible and reinforcing visual input while increasingly suppress-

CICERO/GALAYAA B.V. / CASANOVA 27: pp. 1-19

 17

ing inputs from other sensory modalities. Eventually, the kinds of input received are radically altered.
the occipital cortex may become so specialized for Rather, if cortical modules compete with each other
vision that it seems as though it is a ‘visual’ cortex, to perform particular processing tasks, they should
designed for the specific task of further subdividing be thought of as being ‘metamodal’ brain centers that
and processing visual information. As evidenced in perform particular computational operations without
the studies of Sur and colleagues discussed earlier specific reference to type of sensory input. It is these
(Sur et al., 1990; Merzenich, 2000), the input to a processes, and hence the cortical modules perform-
brain region is a powerful bias that helps to deter- ing them, that are subsequently exploited and shaped
mine the nature of the processing that the module by the demands of the sensory modalities that re-
ultimately performs. If ME architecture bears any quire them. Thus the many studies that demonstrate
resemblance to the development of functional brain processing of sensory information outside of the
modules, it may not be the visual cortex that deter- classically recognized cortical boundaries for that
mines what the act of seeing will be like. Rather, it is modality may not be examples of the brain being
the task of seeing that instructs the functional mod- redundant or ‘getting its wires crossed,’ but instead
ules of the developing brain regarding what vision may represent the workings of an efficient meta-
will be like, and how it must be processed. modal brain, which uses inputs to those cortical
If the development of functional specificity of regions (operators) that seem best suited to execute
cortical modules is the result of competition between their computations successfully.
expert networks that have distinct structural and
functional properties, it becomes possible to suggest Acknowledgements
a mechanism by which apparent ‘crossmodal neu-
roplasticity’ might occur. In the case of early blind We wish to thank Alfonso Carammazza, Eduardo
subjects, for example, the brain loses a major source Fernandez, Jordan Grafman, Stephen Kosslyn, Mar-
of sensory input, which normally constitutes a com- garet O’Connor, Daniel Press, and Robert Stick-
plex task, or set of tasks, that is normally ‘won’ old for insightful comments. Our research was
by occipital networks. Without training from visual supported in part by the National Eye Institute
input, occipital cortical areas may not become spe- (RO1EY12091), the National Institute for Men-
cific for the particular functions involved in seeing. tal Health (RO1MH57980, RO1MH60734), and the
Furthermore, since occipital cortical networks may Harvard-Thorndike General Clinical Research Cen-
be initially structurally and functionally predisposed ter (NCRR MO1 RR01032).
to excel at tasks that require high-acuity and high-
fidelity processing of spatial information, they may References
begin to win the competition for other tasks that
make these kinds of processing demands. Tactile dis- Büchel, C., Price, C., Frackowiak, R.S.J. and Friston, K. (1998)
crimination or auditory localization may represent Different activation patterns in visual cortex of late and con-
genitally blind subjects. Brain, 121: 409–419.
just such kinds of tasks. As a result, the occipital Clarke, E. and Jacyna, L.S. (1987) Nineteenth-Century Origins
cortex may begin to subserve the functions of touch of Neuroscientific Concepts. University of California Press,
and hearing, which it would not have been able to do Berkeley, CA.
nearly as well if the demanding and specific needs of Cohen, L.G., Celnik, P., Pascual-Leone, A., Corwell, B., Falz,
visual processing still had to met. L., Dambrosia, J., Honda, M., Sadato, N., Gerloff, C., Catala,
M.D. and Hallett, M. (1997) Functional relevance of cross-
In light of the aforementioned model of modu- modal plasticity in blind humans. Nature, 389: 180–183.
lar specificity and the hypothesis of plasticity across Cohen, L.G., Weeks, R.A., Sadato, N., Celnik, P., Ishii, K. and
sensory modalities that emerges from it, it seems Hallett, M. (1999) Period of susceptibility for cross-modal
that referring to the kinds of cortical changes seen plasticity in the blind. Ann. Neurol., 45: 451–460.
in blind Braille readers and other subjects as cross- Degenaar, M. (1996) Molyneux’s Problem: Three Centuries of
Discussion on the Perception of Forms. Kluwer Academic,
modal plasticity is something of a misnomer. Func- Dordrecht.
tion-specific brain modules do not appear to have Descartes, R. (1976=1662) Treatise on Man. Harvard University
preordained roles that simply get ‘switched’ when Press, Cambridge.

CICERO/GALAYAA B.V. / CASANOVA 27: pp. 1-19

 18

Fishman, M.C. and Michael, C.R. (1973) Integration of auditory Melchner, L.v., Pallas, S.L. and Sur, M. (2000) Visual behaviour
information in the cat’s visual cortex. Vision Res., 13: 1415– medicated by retinal projections directed to the auditory path-
1419. way. Nature, 404: 871–876.
Gall, F.J. and Spurzheim, J.C. (1835) On the Function of the Merzenich, M. (2000) Seeing in the sound zone. Nature, 404:
Brain and Each of its Parts: with Observations on the Possi- 820–821.
bility of Determining the Instincts, Propensities and Talents, or Morell, F. (1972) Visual system’s view of acoustic space. Nature,
the Moral and Intellectual Dispositions of Men and Animals, 238: 44–46.
by the Configuration of the Brain and Head. Marsh, Capen, Morgan, M.J. (1977) Molyneux’s Question: Vision, Touch and
and Lyon, Boston. the Philosophy of Perception. Cambridge University Press,
Gregory, R.L. and Wallace, J. (1963) Recovery From Early Cambridge.
Blindness: A Case Study, Heffer and Sons. Müller, J.M. (1965=1838) On the specific energies of nerves. In:
Gross, C.G. (1998) In: C.G. Gross (Ed.), Brain, Vision, Memory: R. Herrnstein and E. Boring (Eds.), Sourcebook of the History
Tales in the History of Neuroscience. MIT Press, Cambridge, of Psychology. Harvard University Press, Cambridge, MA.
MA. Murata, K., Cramer, H. and Bach-y-Rita, P. (1965) Neuronal
Hamilton, R.H. and Pascual-Leone, A. (1998) Cortical plasticity convergence of noxious, acoustic and visual stimuli in the
associated with Braille learning. Trends Cogn. Sci., 2: 168– visual cortex of the cat. J. Neurophysiol., 28: 1223–1240.
174. Pascual-Leone, A. and Torres, F. (1993) Plasticity of the senso-
Hamilton, R., Keenan, J.P., Catala, M.D. and Pascual-Leone, A. rimotor cortex representation of the reading finger in Braille
(2000) Alexia for Braille following bilateral occipital stroke in readers. Brain, 116: 39–52.
an early blind woman. NeuroReport, 11: 237–240. Pascual-Leone, A., Cammarota, A., Wassermann, E.M., Brasil-
Howard, I.P. and Templeton, W.B. (1966) Human Spatial Orien- Neto, J.P., Cohen, L.G. and Hallett, M. (1993) Modulation of
tation, Wiley, London. motor cortical outputs to the reading hand of Braille readers.
Jacobs, R.A. and Kosslyn, S.M. (1994) Encoding shape and Ann. Neurol., 34: 33–37.
spatial relations: the role of receptive field size in coordinating
Pascual-Leone, A., Wassermann, E.M., Sadato, N. and Hallett,
complementary representations. Cogn. Sci., 18: 361–386.
M. (1995) The role of reading activity on the modulation of
Jacobs, R. (1999) Computational studies of the development of
motor cortical outputs to the reading hand in Braille readers.
functionally specialized neural modules. Trends Cogn. Sci., 3:
Ann. Neurol., 38: 910–915.
31–38.
Pascual-Leone, A., Tarazona, F. and Catala, M.D. (1996) Mod-
James, W. (1890) The Principles of Psychology. Henry Holt,
ulation of motor cortical output maps associated with the
New York.
acquisition of the Braille reading skill. Neuroimage, 3: S554.
Jung, R., Kornhuber, H.H. and Da Fonseca, J.S. (1963) Multi-
Rauschecker, J.P. (1995) Compensatory plasticity and sensory
sensory convergence on cortical neurons: neuronal effects of
substitution in the cerebral cortex. Tends Neurosci., 18: 36–43.
visual, acoustic and vestibular stimuli in the superior convolu-
Roe, A.W., Pallas, S.L., Hahm, J.-O. and Sur, M. (1990) A map
tions of the cat’s cortex. In: G. Moruzzi, A. Fessard and H.H.
Jasper (Eds.), Progress in Brain Research. Elsevier, Amster- of visual world induced in primary auditory cortex. Science,
?#2 dam, pp. 207–240. 250: 818–820.
Kauffman, T., Hamilton, R., Warde, A., Keenan, J.P. and Pascual- Sadato, N., Pascual-Leone, A., Grafman, J., Ibañez, V., Deiber,
Leone, A. (2000) The role of visual cortex in tactile Braille M.-P., Dold, G. and Hallett, M. (1996) Activation of the
reading: the early blind, the sighted, and the blindfolded. Ann. primary visual cortex by Braille reading in blind subjects.
Neurol., in press. Nature, 380: 526–528.
Kiriakopoulos, E., Baker, J., Hamilton, R. and Pascual-Leone, Sadato, N., Pascual-Leone, A., Grafman, J., Deiber, M.-P.,
A. (1999) Relationship between tactile spatial acuity and brain Ibanez, V. and Hallett, M. (1998) Neural networks for Braille
activation on brain functional magnetic resonance imaging. reading by the blind. Brain, 121: 1213–1229.
Neurology, 52(Suppl. 2): A307. Schlaug, G., Halpern, A.R., Press, D.Z., Baker, J., Edelman, R.R.
Kosslyn, S.M. (1994) Image and the Brain: The Resolution of and Pascual-Leone, A. (1999) Changes in cortical auditory
the Imagery Debate. MIT Press, Cambridge. processing after blindfolding. Soc. Neurosci. Abstr., 25: 1628.
Kujala, T., Alho, K. and Näätänen, R. (2000) Cross-modal reor- Schlaug, G., Chen, C., Press, D.Z., Warde, A., Chen, Q. and
ganization of human cortical functions. Trends Neurosci., 23: Pascual-Leone, A. (2001) Hearing with the mind’s eye. Hum.
115–120. Brain Mapping, in press.
Lomo, T. and Mollica, A. (1962) Activity of single units in Sharma, J., Angelucci, A. and Sur, M. (2000) Induction of visual
the primary optic cortex in the unanesthetized rabbit during orientation modules in auditory cortex. Nature, 404: 841–847.
visual, acoustic, olfactory and painful stimulation. Arch. Ital. Spinelli, D.W., Starr, A. and Barrett, T.W. (1968) Auditory speci-
Biol., 100: 86–120. ficity in unit recordings from cat’s visual cortex. Exp. Neurol.,
Maguire, D.P., Stickgold, R., Warde, A., Kiriakopoulos, E. and 22: 75–84.
Pascual-Leone, A. (1999) Appearance of Charles Bonnet syn- Stein, B.E. and Meredith, M.A. (1993) The Merging of the
drome (CBS) following prolonged blindfolding and Braille Senses. MIT Press, Cambridge, MA.
training. Soc. Neurosci. Abstr., 25: 1315. Sur, M., Pallas, S.L. and Roe, A.W. (1990) Cross-modal plastic-

CICERO/GALAYAA B.V. / CASANOVA 27: pp. 1-19

 19

ity in cortical development: differentiation and specification of Von Senden, M. (1932) Space and Sight: The Perception of
sensory neocortex. Trends Neurosci., 13: 227–233. Space and Shape in the Congenitally Blind Before and After
Van Boven, R., Hamilton, R., Kaufman, T., Keenan, J.P. and Operation. J.A. Barth.
Pascual-Leone, A. (2000) Tactile spatial resolution in blind
Braille readers. Neurology, in press.

QUERIES:
?#1: Please note that ‘[4]’ has been removed. (page 9)
?#2: Please supply volume no. for Jung et al., 1963 (page 18)

CICERO/GALAYAA B.V. / CASANOVA 27: pp. 1-19