Holcomb 1993

Language and Cognitive Processes
ISSN: 0169-0965 (Print) 1464-0732 (Online) Journal homepage: http://www.tandfonline.com/loi/plcp20
Cross-modal semantic priming: A time-course

analysis using event-related brain potentials
Phillip J. Holcomb & Jane E. Anderson
To cite this article: Phillip J. Holcomb & Jane E. Anderson (1993) Cross-modal semantic priming:
A time-course analysis using event-related brain potentials, Language and Cognitive Processes,
8:4, 379-411, DOI: 10.1080/01690969308407583
To link to this article: http://dx.doi.org/10.1080/01690969308407583
Published online: 13 Dec 2007.
Submit your article to this journal
Article views: 118
View related articles
Citing articles: 54 View citing articles
Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=plcp20
Download by: [University of Exeter] Date: 05 June 2016, At: 16:35

LANGUAGE AND COGNITIVE PROCESSES. 1993,8 (4) 379-411
Cross-modal Semantic Priming: A Time-course

Analysis Using Event-related Brain Potentials
Phillip J. Holcomb and Jane E. Anderson
Department of Psychology, TUBUniversity, Medford, Massachusetts,
USA
Downloaded by [University of Exeter] at 16:35 05 June 2016
In two experiments, the timecourse of behavioural and electrophysiological

measures of semantic priming were examined between the visual and audi-
tory modalities. In Experiment 1, auditory targets (words and pseudowords)
were paired with visual prime words that onset either simultaneously with
the target (0 msec stimulus onset asynchrony, SOA) or at one of two delays
(200 and 800msec SOA). Subjects made speeded lexical decisions to the
auditory targets. Large priming effects were found across the three SOAs
for reaction time and the N400 effect (the difference between related and
unrelated target words). In Experiment 2, auditory primes were paired with
visual targets. Here signscant behavioural priming was found across the
SOAs (larger for 0 and 800 msec), but the N W effect was significant only
for the 200 and 800 msec conditions. It is suggested that the data are most
consistent with an amodal semantic system that is tapped by separate
modality specific encoding processes.
INTRODUCTION
Over the past several decades, psycholinguists and cognitive psychologists
have shown substantial interest in the processes and representations under-
lying language comprehension during reading and listening. However,
comparatively little emphasis has been placed on determining the locus or
extent of common language processes across the modalities (but see
Bradley & Forster, 1987; Ellis & Young, 1988; Radeau et al., 1992;
Shallice, 1988). On the one hand, it seems obvious that written and spoken
words place unique demands on the readerllistener and that each modality
must therefore enlist a set of its own “modality specific” processes during
comprehension. However, it is equally clear that at some point after initial
Requests for reprints should be addressed to Phillip J. Holcomb, Department of Psycho-

logy, Tufts University, Medford, Massachusetts 02155. USA.
This research was supported by NIH Grant HD25889 to P.J.H.
0 1993 Lawrence Erlbaum AsJociates Ltd and VSP Publications

380 HOLCOMB AND ANDERSON
encoding operations that a shared “common” set of processes and/or

representations must be employed during reading and listening. The issue
we wish to begin to address here is the locus of such common processed
representations. The results from two experiments will be reported that
directly examined the time-course of semantic priming between the mod-
alities using event-related brain potentials (ERPs).
Although there is some question about the exact locus of semantic
priming effects during word processing (e.g. see Neely, 1991,for a recent
review), in the case of ERPs, there is at least partial consensus that the
N400 component (a negative-going EFW that peaks at approximately 400
msec post-stimulus onset) is most sensitive to a relatively late post-
recognition “integrative” process (e.g. Brown & Hagoort, 1993;Holcomb,
1993; Rugg, 1990). In a host of studies (see Osterhout and Holcomb, in

press, for a review) it has been shown that the N400 is larger in amplitude
to target words that are more difficult to integrate semantically with a prior
context. In these studies, context is typically manipulated by the presence
or absence of a prior semantically related word or sentence fragment that
predicts or fits with the target word.
In one such study, Holcomb and Neville (1990)reported a similar, but
non-identical, set of semantic priming effects for word pairs presented
separately in either the visual or auditory modalities. Their study, like the
experiments to be reported here, examined the effects of a single word con-
text (prime) on the N400 component of a target item presented 1150 msec
later. In both modalities, they found a robust difference in the amplitude
of the N400 when a target word was preceded by a semantically related
prime word compared to when the same word was preceded by an
unrelated word. However, the time-course and scalp distribution of this
“N400 effect”’ was different for the two modalities. In particular, the
effect onset earlier (200vs 300 msec) and lasted longer (800vs 600 msec)
for spoken than for written words. Also, written words, as in several
previous studies (e.g. Kutas & Hillyard, 1980;1984), tended to produce a
slightly larger effect over the right hemisphere while spoken words resulted
in a more bilaterally symmetrical N400. Holcomb and Neville interpreted
the earlier onset of the N400 effect for spoken words, which was less than
the duration of even the shortest words presented, as supporting the
hypothesis that spoken word recognition can occuf on-line, prior to the
arrival of all of a word’s acoustic information. They pointed out that this
conclusion is consistent with the cohort model of Marslen-Wilson and
colleagues (e.g. Marslen-Wilson, 1987; Marslen-Wilson & Tyler, 1980;
Marslen-Wilson & Welsh, 1978).
‘In a semantic priming task, the “ N W effect” refers to the difference in the N400
components elicited by unprimed VJ primed words-unprimed words elicit a more negative-
going N400 compared to words that are primed.
CROSSMODAL SEMANTIC PRIMING AND ERPs 381
In a second study, Holcomb and Neville (1991) demonstrated that the

onset of the N400 can occur even earlier for spoken words in connected
natural speech. In this experiment, the N400 effect between sentence final
words that fit with the context of the sentence and final words that did not
fit started as early as 50 msec post-word-onset (average final word duration
was 561 msec). Holcomb and Neville attributed this even earlier auditory
N400 effect to the influence of prosodic and co-articulatory cues that are
usually present in connected natural speech. It should be noted that the
50msec onset of the spoken sentence N400 effects contrasts with the
findings from visually presented sentences where N400 effects do not
usually start before 200 msec (e.g. Kutas & Hillyard, 1980).
The results discussed so far support the hypothesis that somewhat
different mechanisms underlie semantic priming for spoken and written

words and are consistent with the prediction that N400 priming berween
the modalities should either be absent or, at least, reveal a different
pattern from within-modality priming. However, by at least one account,
this prediction might not prove correct. Even if it is assumed that the
modalities usually use different word recognition processes, they might,
nevertheless, reveal robust between-modality priming due to some process
that mediates between the modalities. There are at least three possibilities
for such a process. The first two, collectively referred to here as the
conversion hypotheses, argue that words are converted from one modality
to the other during reading and possibly during listening as well. 10 the
“recoding” version of this hypothesis, conversion presumably occurs
early, prior to word recognition (e.g. Coltheart, 1978). In the “translation”
version, conversion occurs only once a word has been recognised by a
modality specific system. According to both of these views, between- and
within-modality priming result from a somewhat different sequence of
events. Within-modality priming results from a modality specific process
(e.g. spreading activation within a unimodal lexicon andor a post-lexical
checking process that relies on semantic information from a modality
specific system; see Neely, 1991, for a review of both types of priming
mechanisms), and between-modality priming results from conversion of
the prime into the target modality and the subsequent operation of a
modality specific process. Therefore, it follows that between-modality
priming should produce a different pattern of effects than within-modality
priming, particularly if the prime and target words occu close together in
time, that is, when the prime has not had time to be converted prior to
the arrival of the target.2
Wote that it is doubtful that the N400 could be w d to differentiate between the recoding
and rraodation hypotheses because of it being sensitive to a relatively late post-lexical priming
process (Brown & Hagoort, 1993; Holcomb, 1993; Rugg. 1990).
A second possibility, the common semantic system hypothesis, suggests

that while written and spoken words are processed in their own respective
perceptual and lexical systems, they activate meanings in a common
semantic or conceptual system. Although they did not discuss written and
spoken words, Kroll and Potter (1984) have argued for such an amodal
system in the case of written word and picture processing. According to
this view, cross-modal priming takes place in an amodal semantic/
conceptual memory system (e.g. due to a process like spreading activation)
that is shared by the modalities. This view also has the parsimonious
advantage of using the same mechanism for accounting for within-modality
priming and therefore predicts that a similar pattern of effects should be
obtained for between- and within-modality conditions.
A number of studies have found evidence of between-modality priming

using reaction time measures. These measures have been reported in both
word pair tasks (Marslen-Wilson, 1987; Swinney, 1979) and in sentence
tasks (Onifer & Swinney, 1981; Seidenberg,Tanenhaus, Leiman, & Bienk-
owski, 1982; Swinney, Onifer, Prather, & Hirshkowitz, 1979; Zwitserlood,
1989) where the sentence is usually presented in the auditory modality and
a target word is presented visually. However, these findings are consistent
with both of the above hypotheses about the source of between-modality
priming, because in all of these reports the interval between the prime
(context) and target items was relatively long, thus allowing time for a
putative conversion process to intervene and possibly go unnoticed.
Another problem with most of the prior cross-modal studies is that none
of them used a truly on-line dependent measure, that is, one that can
monitor a process ai it occurs.
Recently, Anderson and Holcomb (submitted) compared the N400
effect in the visual and auditory modalities while manipulating the interval
between the onset of the prime and target words (the stimulus onset
asynchrony or SOA). They reasoned that changes in the N400 produced
by varying the primeharget interval would reveal important information
about the timecourse of the processes underlying semantic priming. When
both the prime and target words were presented in the visual modality (the
prime just to the left of a fixation point and the target to the right), there
were N400 priming effects present across the three SOAs (0, 200 and 800
msec), and when both the prime and target were presented in the auditory
modality (the prime to the right ear and the target to the left), priming
effects were found for the 0 and 800msec conditions (they were not
significant at 200msec). These results demonstrate that in the case of
simultaneously presented words (0 SOA), there is substantial temporal
overlap in the processing of semantic information and suggests that the
prime and target were being dealt with in parallel.
The primary purpose of the current experiments was to extend the
CROSSMODAL SEMANTIC PRIMING AND ERPs 383
within-modality findings of Holcomb and Neville (1990; 1991) and Ander-

son and Holcomb (submitted) to the case where the prime and target
words were in diferent modalities. Specifically, it was predicted that if
between-modality priming (as measured by the N400) results from a
conversion process, then there should be evidence of an N400 effect
between modalities at relatively long primdtarget intervals, but little if any
evidence of such an effect at short intervals (i.e. prior to conversion).
However, if between-modality priming results from activity in a common
semantic system, then there should be evidence of an N400 effect between
modalities at both short and long intervals (as was the case in the within-
modality conditions of Anderson and Holcomb).
Below we report the results of two experiments examining the time-
course of between-modality priming. In the first experiment, the prime

word was presented in the visual modality and the target item was pre-
sented auditorily (visuallauditory). In Experiment 2, the conditions were
reversed and the prime was auditory and the target visual (auditory/visual).
A lexical decision task in which one-third of the target items were pseudo-
words (e.g. doctor/teble) was used in both experiments. Among the word
targets, half were semantically related to the prior prime word (doctor/
nurse) and half were unrelated (shoe/nurse). In such tasks, semantic
priming effects are usually manifested by quicker and more accurate
button presses to related target words compared to unrelated target words,
and by smaller N400 amplitudes for related compared to unrelated targets.
The interval between prime and target was also manipulated across three
SOAs: long (800msec), short (200 msec) and simultaneousonset (0 msec).
The relatedness and SOA variables were mixed within a single experimen-
tal session so that the subjects did not know from trial to trial what
level of each independent variable was coming next. Also,while they were
told they were only to respond to the target items in both experiments,
they were also instructed to read covertly (listen to) the prime word as
well.
One unavoidable difference between the experiments was due to the
very different temporal parameters of spoken and written words. In
Experiment 1, even in the 0 SOA condition, the entire visual prime
stimulus was available from the onset point of the auditory target, while
in Experiment 2 only a small portion of even the shortest duration spoken
prime was available at the same time as the entire visual target. However,
we reasoned that this difference might not be as important as it appears
because of the evidence for the onset of spoken word priming being earlier
(e.g. Holcomb & Neville, 1990). Even though spoken words might have
an average duration greater than 500msec, it is clear that they access
semantic information far earlier than this point (in fact, before written
words!). As will be seen, this rationale did not prove to be totally correct.
EXPERIMENT 1: VISUAL/AUDlTORY
The purpose of the first experiment was to determine if the same mechan-
isms supporting semantic priming within the auditory modality also medi-
ate priming between visual and auditory words. It was predicted that if the
same processes are involved, then there should be evidence of an N400
effect between modalities at the shortest SOAs. However, if between-
modality priming is mediated via a conversion process, then only the
longest prime-target interval (800 SOA) should show significant N400
effects.
Method
Subjects
Twelve right-..anded Tufts University undergraduates (7 females, 5
males) with a mean age of 19.42 years (SD = 1.68 years) received partial
course credit or $10.0 for their participation. All of them were native
speakers of English with normal visual and auditory acuity.
Stimuli 8nd Procedure

Stimuli were presented on a 20-inch monitor (NEC model 5D)or binau-
rally through headphones (Sony MDR S30) under the control of an IBM-
PC compatible computer. The subject sat in a comfortable chair in a sound
attenuating chamber. On each trial, the subject was presented with two
stimuli. One (the prime) was a legal English word presented visually; the
other (the target) was either a legal word or a pronounceable nonword
(pseudoword) which was presented through headphones. Each subject was
presented with a total of 360 visual-auditory pairs, comprised of equal
proportions of semantically related, semantically unrelated and word
pseudoword pairs. This will be referred to as the urger type variable.
Examples of pairs for each of the three target type conditions are: SALT-
PEPPER, MORE-TRUCK and NICKEL-PLONE. Unrelated pairs were
formed by rearranging the related pairs so that the primes and targets did
not have any semantic relationship. Pseudowords were constructed from
legal words by altering one letter (phoneme) in such a way that it did not
violate the orthographical or phonological rules of English. None of the
pseudowords were pseudo-homophones. All visual stimuli were two to
seven letters in length and all auditory stimuli were of one or two syllables.
Related and unrelated word pairs were selected from six similarly
constructed lists of 40 related word pairs (see Appendix). The pairs of
words and pseudowordswere selected from three similarly constructed lists
CROSS-MODAL SEMANTIC PRIMING AND ERPs 385
of 40 word-pseudoword pairs. The word pairs were counterbalanced so

that, across subjects, target words appeared in both the related and
unrelated conditions and in each of the three SOA conditions (see below).
However, within subjects, each List and therefore each stimulus was
presented once.
A second within-subject variable was the stimulus onset synchrony
between items in each pair. Forty stimulus pairs in each of the three target
type conditions (related, unrelated and pseudoword) were presented with
an SOA of Omsec, 40 others were presented with an SOA of 200msec,
and the remaining 40 were presented with an SOA of W m s e c . To
summarise, each subject was presented with a total of 360 pairs of words
(in a pseudorandom order) which were either related, unrelated or word-
pseudoword pairs, and had an SOA of either 0,200 or 800 msec, resulting
in a total of 40 stimulus pairs in each of nine conditions (3 SOAs x 3 target
types)-
The visual stimuli were displayed as black lower-case letters on a white
background. Each word subtended from 0.5” to 1.8”of horizontal and 0.4”
of vertical visual angle. The auditory stimuli were spoken by a female
member of our research team and were digitised (16 kHz, 24 pole 7.9 kHz
Butterworth filter) by a Data Translations analogue-to-digital converter
(12 bits, model DT2821). Each stimulus was edited using software that
allowed us to listen to the stimulus while visually inspecting its waveform
in order to store it from the time of onset. This was done so that the precise
time of its onset could be time-locked with EEG digitisation. At the time
of the experiment, the stimuli were output through a digital-to-analogue
converter, then filtered (7.9IrHz) and Sent to the subject’s headphones.
The average duration of auditory targets was 568msec (range 300-862
msec) .
Each trial began with a warning stimulus (a red “X”)in the middle of
the screen. Then, 500 msec later, the prime replaced the warning stimulus
and remained on the screen for 200 msec. For the 0 msec SOA condition,
the target onset was simultaneous with the onset of the prime; for the other
two SOAs, the target onset was either 200 or 800msec after the onset of
the prime. Next, 1500msec after the onset of the target, a green “ X ’
appeared in the middle of the screen, indicating to the subject that it was
alright to blink. Finally, after a 1250msec inter-trial interval, the green
“X”changed to a red “X”and the next trial began.
The subjects were instructed to respond as quickly and accurately as
possible by pressing a button labelled “YES” with one thumb if the target
was a real word, or a button labelled “NO”with their other thumb if it
was not a real word. They were told to try to pay attention to the visual
prime but not to make an overt response. The hand used for each response
was counterbalanced across subjects. The subjects were told not to blink
or move their eyes while the stimuli were being presented. The experiment
lasted about 35 min, including short breaks about every 60 trials. A
practice block of eight trials preceded the experiment.
EEG Procedure
Tin electrodes were held in place on the scalp with an elastic cap
(Electrode-Cap International). The scalp locations included standard
International 10-20 system locations over the left and right hemispheres at
the frontal (ITand F8) and occipital sites ( 0 1 and 0 2 ) and three locations
on the midline: frontal (Fz), central (Cz) and parietal (Pz). In addition,
six electrodes were placed at the following non-standard locations pre-
viously found to be sensitive to language manipulations (cg. Holcomb,

Coffey, & Neville, 1992; Holcomb & Neville, 1990; 1991): left and right
temporal-parietal (Wernicke’s area and its right hemisphere homologue,
WL and WR: 30% of the interaural distance lateral to a point 13% of the
nasion-inion distance posterior to Cz); left and right temporal (TL and
TR: 33% of the interaural distance lateral to Cz); and left and right
anterior-temporal (ATL and ATR: 50% of the distance between T3/4and
F7/8).To monitor for eye blinks, one electrode was placed below the left
eye, and to monitor for horizontal eye movement, an electrode was placed
lateral to the right eye. All the electrodes were referenced to the left
mastoid, and the right mastoid was recorded from actively in order to
determine if there were different experimental contributions to these two
presumably neutral sites.
The electroencephalogram (EEG) was amplified by a Grass Model 12
amplifier system using a bandpass of 0.01 to 100Hz (3 dB cut-off). The
EEG was sampled continuously throughout the experiment (200Hz),and
off-line, separate ERPs were averaged (using a pre-target baseline of 100
msec) for each subject at each electrode site for the three target types
(related, unrelated and pseudoword) at each of the three SOAs. Only
correct response trials that were free of eye and muscle artifact were
included. In addition, difference waves were formed by subtracting the
ERPs of the related condition from the E m s of the unrelated condition.
Data Anel p i s
The mean reaction times for correct responses between 200 and 2000
msec and the percentage of errors were calculated for each subject. EFWs
to targets were quantified by measuring the mean amplitude in three
latency windows: 300-550, 550-800 and 800-1150msec. To examine the
time-course of priming effects more closely, the mean amplitude measures
of 100msec epochs were also taken starting 100msec post-target and
extending to 1100 msec. The use of multiple windows carries a greater risk
of type 1error; however, we utilise these analyses only as a supplementary
measure to examine timecourse effects.
Repeated measures analyses of variance (ANOVAs) were performed on
the above dependent measures. A 2 X 3 ANOVA was done with target
type (related vs unrelated: note that the pseudoword condition was not
included in any of the analyses to be reported here) and SOA (0 vs 200 vs
800msec) as the factors. For the ERP analyses, the midline and lateral
sites were analysed separately. In addition to target type and SOA, for the
midline analyses there was an electrode site factor [frontal (Fz) vs central
(Cz) vs parietal (Pz)], and at the lateral sites there was an electrode site
factor (frontal vs anterior-temporal vs temporal vs Wernicke’s vs occipital)
and a hemisphere factor (left vs right). Significant target type x SOA

interactions were followed up with simple effects tests to help elucidate
the source of the interaction. This involved analysing the effects of target
type separately for each SOA. The Geisser-Greenhouse (1959) correction
was applied to all analyses with more than one degree of freedom in the
numerator.
ResuIts
Beha vioural Data

Across the SOA conditions, related targets were responded to more
quickly than unrelated targets [main effect of target type: F(1,ll)= 74.89,
P < 0.001; see Table 11. Reaction times for the 0 msec SOA were slightly
longer than for the other SOAs; however, this difference only approached
sigdicance [main effect of SOA: F(2,22) = 3.38,P < 0.061. The interac-
tion between SOA and target type also approached significance [F(2,22)
= 3.80, P < O.OSS], indicating that the priming effect tended to become
slightly larger as the SOA became longer. The subjects made more errors
to the unrelated than to the related targets [main effect of target type:
F(1,ll) = 42.19, P < 0.0011, and they made more errors to targets at the
longer than at the shorter SOAs [main effect of SOA: F(2,22) = 7.23, P
c 0.009].
Electrophysiological Data
The grand mean target ERPs are plotted in Fig. 1. Notice that these
waveforms appear somewhat different at the three SOAs. This is due, in
part, to the differential overlap of the visual prime and auditory target
ERPs, particularly in the 0 and 200 msec SOA conditions. In the case of
the 200 msec condition, the early P2 component from the prime-elicited
TABLE 1
Mean ( i S D ) Reaction Times (rnmc) and Percent Errors
SOA Related Words Unrelated Words Pseudowords
Exprimem I : Visdaudirory
0 RT 822f91 889floo 985f98
PE 1.9f1.9 4.0k2.2 8.6f6.7
u)o RT 797ff2 mi81 !mi89

PE 2.3f2.7 S.0f3.2 10.9f9.5
800 RT 779282 892f74 954f77

PE 2.3f3.1 8.6f4.3 7.7f6.1
Experiment 2: Audirorylvitual
0 RT 7Mf104 735f 115 834f127
PE 1.2f1.7 2.1f4.3 4.8fS.9
200 RT 670f 119 Mi137 813f141

PE 1.7f3.6 1.9f3.0 4.2351
800 RT 630+117 670f118 7762144

PE 1.0f2.2 2.3f2.9 4.8353
ERP occurred just as the auditory target stimulus onset. Thus the resulting
target ERP is a summation of the activity generated by the target and the
ongoing activity generated by the prime. Nevertheless, a large negativity,
which peaked at approximately 100 msec (N100 or Nl), can be seen at all
three SOAS. The N1 was present at all but the most posterior sites (01,
02) and was largest from the fronto-central electrodes. Following the N1
there was an equally large positive-going wave which peaked at approxi-
mately 200 msec (€200or €2). The €2had a similar scalp distribution to
the N1.
The P2 component was followed by a broad negative-going wave peak-
ing between 400 and 500 msec. This negativity, which overlaps the window
usually associated with the N400, was largest (i.e. was most negative) at
the more anterior sites. The broad negativity was followed at the more
posterior sites by a late peaking positivity (P3), which continued through
the end of the recording epoch (1180 msec).
Target Type Effects

300-55Omec. In this epoch, the unrelated targets elicited a signifi-
cantly more negative-going ERP than the related targets [midline: F(1,ll)
= 18.34, P < 0.001; lateral: F(1,ll) = 17.6, P < 0.0021 and this “priming”
or N400 effect did not interact with the SOA variable. The interaction
between priming and electrode site approached significance at the lateral
sites (F(4,44) = 3.79, P < 0.0621, indicating that target type differences
were largest at the Wernicke’s and temporal sites. Finally, there was no
evidence for a lateral asymmetry in this epoch.
550-800 m c c . Unrelated targets continued to be more negative-going

than related targets in this epoch [main effect of target type, midline:
F(1,ll) = 16.66, P < 0.002; lateral: F(1,ll) = 24.39, P < 0.001]. Also,
the interaction between target type and electrode site was significant at the
lateral sites [F(4,44) = 6.37, P < 0.0081, indicating that priming was
greatest at the Wernicke’s and temporal sites. As in the previous epoch,
there was no evidence of a significant difference in priming at the three

SOAs (midline: P > 0.15; lateral: P > 0.39), nor was there a significant
asymmetry in the priming effect across the hemispheres.
800-1150mec. In the final epoch, the unrelated targets continued to

produce more negative-going ERPs than related targets [main effect of
target type,midline: F(1.11) = 10.39, P < 0.008; lateral: F(1,ll) = 9.87,
P < 0.0091 and this effect did not differ across the three SOAs.
Time-course Analyses (100 m e c Epochs). To examine the timecourse
of the priming effect, the waveform was divided into 100msec epochs
beginning with 100msec and extending to ll00msec. The differences in
mean amplitude during each epoch are listed in Table 2. As can be seen
from Table 2, the first reliable effects of relatedness occur starting in the
300-400msec window and continue through to the end of the recording
epoch. The only indication of a difference in priming between the SOAs
is in the 200-300msec window, where a significant SOA x target type
interaction indicates that only the 0 SOA produced a priming effect.
Discussion
In Experiment 1, we examined semantic priming at different SOAs with
the prime presented in the visual modality and the target in the auditory
modality. Large effects, both behavioural and electrophysiological, were
found across SOA conditions. The reaction time differences tended to be
slightly larger as the SOA became longer (indicated by an interaction that
approached significance). However, the magnitude of the ERP priming
effect did not differ between the SOAs. This N400 effect began around
300 msec and continued to the end of the measuring epoch.
The fact that the visual primes were able to prime the auditory targets
at the simultaneous (0 SOA) and 200 SOA conditions is most consistent
-+ a1
-. ..
..........
..
......
?7
-Pa
+ I ......
...--..-...
n
--+$-...
WL.
............
+
- T
01 +
L Lxx) *I*! - . o*%
........................ 600900
+ + -
+ +
-T GT -1 ......
01
+
I . .
300 6 0 0 - p d o '
.
O2 y r ............
390
..... -
$A.L
*.....
.-...
-p.,x
: .
.......’.. 5..-
....
P)
...
.,A. Fa
....
Fz
* + ......-.-.....
..........._ +
.C.
ATL
.: .. ATR
._..
CZ
\ ... +J.
+
.-:
........
..
n
,(woo
I
......
m
...
+ +
01 02
+ +
(C) - Rebed torteis ..... U n n & d taqers
FIG. 1 Grand mean ERPs (n = 12) to related and unrelated auditory target words in
Experiment I: (a) 0 SOA condition; (b) 200 SOA condition; (c) 800 SOA condition. The
ERPs in the left-hand column arc from electrodes placed over the left hemisphere sites, the
middle column is from the midline sites and the right-hand column is from the right
hemisphere sites. T i e is measured in msec, each tic mark representing 100 msec. Stimulus
onset is the vertical calibration bar.
with the hypothesis that the two modalities share a common semantic
process. If priming had been due to a conversion process, there should
have been evidence of cross-modal priming effects only when there was
sufficient time for such a process, that is, in the 800 SOA condition.
The pattern of results obtained in Experiment 1 was similar to that of
Anderson and Holcomb (submitted). They found ERP evidence of short
interval (0 SOA) priming when both the prime and target were auditory,
although their effects were not as large or consistent as those in the present
study. The similar early timecourse of priming between these two studies
is consistent with the hypothesis that within- and between-modality seman-
tic priming (as measured here) rely on a common amodal semantic system.
The early onset of the N400 effect is also consistent with previous ERP
findings (Holcomb & Neville, 1990; 1991) that auditory words can be
recognised prior to their completion (Marslen-Wilson, 1987). The average
length of the targets was 568msec, with the shortest duration being 300
msec. The N400 effect began in the 300-400 msec window and approached
TABLE 2
The Size of the Semantic Priming Effect (Unrelated-Related) in pV at Each of the 100
msec Epoch8 for Each SOA in Experiment 1 (Visual Prime, Auditory Target)
SOA P-Vdues
Epoch 0 200 800 77- mXSOA
1oo-mmSCc
midline -1.19 1.10 -0.42 NS NS
lateral -0.32 0.35 -0.09 NS NS
2m-300 m x c
midline -1.5V 1.03 1.00 NS 0.0267
lateral -0.46 0.42 -0.48 NS NS
300-400llWC
midline -2.w -1.14 -2.sob 0.0043 NS

lateral -1.2T -0.64 -1.19. 0.0073 NS
400-500msec
midline -4.11' -2.w -3.w 0.0014 NS
lateral -2.w -1.306 -1.83' 0.0018 NS
500-600mSeC
midline -5.5T -4.14' -2.976 0.0005 NS
lateral -3.03d -2.14' -1.79b O.ooo2 NS
600-700mSec
midline -4.86' -2.69b -3.096 0.0024 NS
lateral -2.75' -1.81b -2.17c O.ooo4 NS
700-800~c
midline -4.w -2.01 -2.736 O.OO40 0.0914
lateral -2.7ff -1.47. -2.w 0.0015 NS
800-900msec
midline -4.w -2.33' -2.486 0.0038 NS
lateral -2.w -1.W -1.896 0.0031 NS
900-1000mscc
midline -3.61' -2.346 -1.67 0.0153 NS
lateral -2.11' -1.696 -1.806 0.0088 NS
1OOO-1100 msec
midline -3.36' -3.05' -0.38 0.0128 0.0401
lateral -1.826 - 1.65' -0.81 0.0255 NS
Note: Superscripts indicate significance of separate analps at each SOA. TT. target type.
'P < 0.1; b~ < 0.05; ' P < 0.01; 'f < 0.001; ' P - v ~ ufor
~ main effect of target type.
significance in the 200-300msec window of the 0 SOA condition. This
occurred even though in an auditory lexical decision task the subject
should wait until the end of the word to make a behavioural response,
since they cannot know for sure that an item is a real word until all of the
acoustic information in that stimulus has been heard. This latter observa-
tion may account for the long duration of the N400 observed in this and
other studies (e.g. Anderson & Holcomb, submitted; Holcomb & Neville,
1990). The scalp distribution of the N400 effect was also similar to
Holcomb and Neville’s (1990) auditory lexical decision study. The N400
effect was largest from more posterior lateral sites (temporal and Wer-
nicke’s) and was symmetrical across the hemispheres. Rugg et al. (this
issue) report auditory repetition priming effects which began h e r than
visual effects. In addition to task differences (repetition vs semantic

priming), this may be due to the fact that their auditory stimuli had a
longer average duration (about 659msec) and lower frequency (see the
General Discussion for a more thorough treatment of Rugg and co-
workers’ study).
EXPERIMENT 2: AUDITORYNISUAL
The purpose of this experiment was to determine if the findings from
Experiment 1 could be generalised to the case where the prime and target
modalities are reversed (i.e. auditory/visual). Given the tentative conclu-
sions outlined above for Experiment 1, it seems reasonable to predict a
similar pattern of results for visual targets primed by auditory words-
even at short prime-target SOAS. That is, when an auditory prime onsets
either simultaneously or shortly before a visual target, the processing of
the initial sounds from the prime should influence the ongoing processing
of the visual target. This prediction is based on the presence of semantic
effects which onset as early as 200msec in spoken word pairs (Holcomb
& Neville, 1990) and in writtedspoken pairs (Experiment 1). From these
findings it was reasoned that if spoken words can be primed at this latency
and if between-modality priming relies on the same mechanism as within-
modality priming, then a spoken word should also be able to prime a visual
word at short intervals. It is noteworthy that the 0 SOA conditions in the
two experiments are very similar, since in both cases a visual and an
auditory stimulus are presented simultaneously. The major difference is in
the instructions to the subject concerning which stimulus they should
respond to.
Zwitserlood (1989) used a cross-modal procedure with auditorily pre-
sented sentence contexts and a sentence final word which was or was not
semantically related to a visual target word. The visual probes were
presented at the offset of auditory final word fragments which varied in
length. She found evidence of facilitation at short intervals (mean = 130

and 199msec) of probes related to both the actual word to be presented
and to a close “competitor” in the initial cohort, independent of the degree
of biasing context. At longer intervals (mean = 278 and 410msec), the
pattern of activation changed, presumably reflecting the influence of
contextual constraints and additional acoustic information on word selec-
tion. Zwitserlood’s design, however, differed from that used in the current
experiment in that the auditory prime word was terminated at the end of
the SOA interval, so that in the shortest SOA condition the subjects heard,
on average, only 130 mSec of the prime word. In the current experiment,
all prime words were fully presented. Another difference was that sent-
ences were used in the study by Zwitserlood and word pairs were used in
the present study.

Following the common semantic system hypothesis, it was predicted that
a pattern of priming effects similar to those found in Experiment 1 would
also be obtained in Experiment 2. That is, significant behavioural and ERP
differences should be observed between the related and unrelated target
types at all three SOAs. However, the conversion hypothesis predicts
significant N400 effects only at longer intervals because of the additional
time required for converting the prime or target to the other modality.
Method
Subjects
Twelve right-handed Tufts University undergraduates (7 females, 5
males) with a mean age of 18.5 k 1.0 years received partial course credit
for their participation. They were all native speakers of English with
normal visual and auditory acuity. None of the subjects had participated
in the first experiment.
Stimuli and Procedure

The lists of stimuli and the procedure were the same as those used in
Experiment 1. The only differencebetween the experiments was that the
modality of the prime and target were reversed. In Experiment 2, primes
were presented in the auditory modality and targets in the visual modality.
The average duration of the primes was 562 msec (range 375-812 msec).
Data Analysis
The data from Experiment 2 were analysed using the same procedures
employed in Experiment 1.
CROSS-MODAL SEMANTlC PRIMING AND ERPs 395
Results
Behavioural Data
Across the SOA conditions, related targets were responded to more
quickly than unrelated targets [main effect of target type: F(1,ll) = 31.86,
P < 0.001; see Table 11. There was also a main effect of SOA [F(2,22) =
34.91, P < 0.0011, indicating that the RTs became shorter as the SOA
became longer. However, there was no significant interaction between
target type and SOA (P > 0.25), indicating that priming was not signifi-
cantly different in each of the SOA conditions.
The responses to the unrelated targets were only marginally less accurate
than the related targets [F(l,ll) = 3.59, P < 0.0851 and there was no
difference across SOAs (P> 0.68).
Electrophysiological Data
The grand mean ERPs are plotted in Fig. 2. As in Experiment 1, the
differential overlap of visual and auditory stimuli made the early compo-
nents of the ERPs appear slightly different for each SOA. In the 0 SOA
condition, where auditory and visual stimuli onset at the same time, the
early N1 and P2 components are similar to those seen in Experiment 1.
Note that from a purely physical standpoint, this condition is very similar
to the 0 SOA condition in Experiment 1 in that they both have a
simultaneous visual and auditory stimulus. However, in the 200 and 800
SOA conditions, where the onset of the visual target is not simultaneous
with the onset of the auditory prime, the distributions of the N1 and F 2
are quite different from Experiment 1. At the anterior sites the N1 is small,
and at the posterior sites (01, WL,Pt, WR and 0 2 ) it peaks later (200
msec). Following the N1, there was a positivity (P2)around 200-250 msec,
which was anteriorly distributed in the 200 SOA condition, but was more
widely distributed in the 800msec condition. From this point on, the
waveforms were more positive relative to the baseline than they were in
Experiment 1 (auditory target).
As in Experiment 1, the P2 in this experiment was followed by a
prominent negative-going component (N400), peaking between 350 and
400msec. This negativity was widely distributed and at the anterior sites
was slightly larger over the left hemisphere. Following the negativity was
a large positive wave which peaked between 500 and 600 msec (P3) at the
posterior sites (note that the late positivity at the lateral anterior sites
peaks closer to 900msec). The P3 tended to be larger over the right
hemisphere at anterior sites for all SOAs and at posterior sites for the 800
SOA.
396
- T -T
-T \A.
" w.xm
+T3\1'-,
+ .. r:
-......... - T ..... . -1
01
Q -R # b r . d W
FIG. 2 Grand mean ERPs (n = 12) to related and unxelatcd visual target words in
Experiment 2: (a) 0 SOA condition; (b) 200 SOA condition; (c) 800 SOA condition. For
further information, see legend to Fig. I.
Target Type Effects

300-550 msec. The unrelated targets elicited a more negative-going
wave than did related targets over the midline sites [maineffect of target
type: F(1,ll) = 10.48, P < 0.0081, but only the more posterior sites
produced a similar effect at the lateral sites [target type x electrode site
interaction: F(4,44)= 9.27,P c 0.0071.There was also a difference in the
priming effect across SOAs [target type x SOA interaction, midline:
F(2,22) = 6.69,P c 0.006; lateral: F(2.22) = 3.09,P c 0.0771.Separate
follow-up analyses at each SOA indicated that at the 0 SOA there was no
significant difference between unrelated and related targets (see Fig. 2;Ps
> 0.7); at the 200 SOA the difference was significant at the midline
[F(l,ll)= 5.57, P < 0.038;lateral: P > 0.141; and at the 800 SOA the
difference was significant in both analyses [midline: F(1.11) = 18.04,P <
0.001;lateral: F(1,ll) = 8.68,P < 0.0131.
550-800 msec. There was no significant difference between related and

unrelated targets in this epoch (midline: P > 0.3;lateral: P > 0.85). nor
were there any interactions of target type and SOA. At the lateral sites,
TAW3
The Sire of the Semantic Priving Effects (Unreiaed-Related) In pV at Each of the 100
m= Epochs for Each SQA in Experiment 2 (Auditory Prime, Visual Target)
...
SOA P-Vdw
loo-mmsec
midline 0.81 -0.63 -0.47 NS NS
lateral 0.49 -0.27 0.00 NS NS
u)0-300~
midline 1.2~ -0.47 -1.m NS 0.0746
lateral 0.67 -0.23 -0.27 NS NS
3ak4almxc
midline 0.35 -0.82 -3.5' 0.0103 0.0053

lateral 0.23 -0.49 -l.DC 0.1043 0.0803
400-500mSec
midline -0.10 -2.05' -3.45' 0.0030 0.0092
lateral 0.12 -1 .03' -1 .37' 0.0509 0.0408
Mo-6oomscc
midliae -0.41 -0.63 -0.61 NS NS
lateral 0.13 -0.08 0.21 NS NS
600-700 mSCc
midline -0.67 -0.75 -0.08 NS NS
lateral 0.15 -0.41 0.63 NS NS
700-800-
d&C -1.17 -1.35 -0.80 NS NS
Iamd -0.36 -0.86 0.11 NS NS
800-900mxf
midline -0.93 -1.15 -1.38 NS NS
lateral -0.36 -l.W -0.51 NS NS
900-1ooo msec
midline -1.01 -1.16 -1.63' 0.0638 NS
lateral -0.31 -0.90 -0.70 NS NS
1am-1100mSec
midline -1.e -1.796 -1.82' 0.0077 NS
lateral -0.27 -1.256 -0.86 0.0750 NS
NOW:Supencripa indicate dgaificana of separate analyses at each SOA. IT,target type.
'P < 0.1;*P < 0.05; 'P < 0.01;'P < 0.001;'P-value for main effect of target type.
there was an interaction between target t y p and electrode site [F(4,44)=

4.03, P < 0.051, indicating that the unrelated targets were more negative-
going posteriorly, but anteriorly the related targets were more negative-
going.
8OeIIJOmsec. There was a main effect of target type at the midline

sites across SOAs [F(l,ll) = 6.75,P < 0.0251, and at the lateral sites there
was a significant interaction of target type,SOA and electrode site [F(8,88)
= 3.55, P < 0.0361; for the 200 SOA the effect was larger anteriorly, but
for the 800 SOA the effect was larger posteriorly (what Little difference
there was for the 0 SOA was largest posteriorly).
Time-course Anlllysu (IoOmec Epochs). Table 3 reports the results

of the 100msec timecourse analyses for Experiment 2. As can be seen,
the effects of target type are much more restricted in time than in
Experiment 1 and are significant only for the 200 and 8oomsec SOAs
during the traditional latency window for the N400.However, later (1OOO-
1100msec), the 0 SOA did show a difference that approached significance
in the predicted direction.
Discussion
When the prime was auditory and the target visual, there was a significant
RT priming effect across the SOAs. However, ERF' priming (i.e. the N400
effect) was significant in the 200 (midline only) and 800 SOA conditions,
but there was no hint of an N400 in the 0 SOA condition. Moreover, the
effect was larger and began earlier in the 800 than the 200 SOA condition.
This pattern of effects is similar, but not identical, to that obtained by
Anderson and Holcomb (submitted) for stimuli within the visual modality.
While they found evidence of EFW priming in the 200 and 800 SOA
conditions, they also obtained a significant N400 effect in the 0 SOA
condition which onset between 300 and 400msec. Therefore, the failure
of the 0 SOA auditory-visual condition to show a similar effect suggests
that cross-modality priming may not rely on the exact same processes as
within-modality priming and calls into question the veracity of the common
semantic system hypotheses offered at the end of Experiment 1. Further
discussion of these findings and their implications will be presented in the
General Discussion.
BETWEEN-EXPERIMENT COMPARISONS
Behavioural Data
In order to test for the existence of any effects due to the modality of
presentation, analyses were done with Experiment as a between-subjects
variable. Visual targets (Experiment 2) were responded to faster (161
msec) than auditory targets (Experiment 1) [main effect of experiment:
F(1,22) = 15.82,P < 0.001].Across experiments, targets were responded
to faster as the SOA became longer [main effect of SOA: F(2,44) = 28.88,
P < 0.0011, but when the target was visual, this decrease had a steeper
decline [experiment x SOA interaction: F(2,44) = 8.15,P < 0.0021. For
both auditory and visual targets, the unrelated targets were responded to
more slowly [main effect of target type: F(1,22) = 105.66,P < 0.001],but
this effect was greater when the target was auditory [experiment x target
type interaction: F(l,22) = 25.97, P < 0.001]. Across experiments, the
priming effect was greatest in the 800 SOA [target type x SOA interaction:
F(2,44)= 3.9,P < 0.0341.
More errors were made in Experiment 1 (auditory target) than in
Experiment 2 (visual target) [main effect of experiment: F(1.22) = 12.22,
P < 0.002].Across experiments, the subjects made more errors to unre-
lated targets than to related targets [main effect of target type: F(1,22) =
40.85,P < 0.001],and this effect was greater when the target was auditory
[target type x experiment interaction: F(1,22) = 17.59,P < 0.0011.
Difference Waves
In order to facilitate ERP comparisons between the experiments, differ-
ence waves are formed by subtracting the related from the unrelated
waveforms. This procedure tends to remove waveform features that the
two conditions of interest have in common (e.g. modality-specific “exoge-
nous” activity such as the N1 and F’2 component^)^ and permits an analysis
of pure ERP priming differences between the experiments. The mean
amplitude from 200 to 700 was calculated for each difference wave and the
resulting measures from both experiments were entered into a mixed-
design ANOVA with one between-subject factor (experiment) and two
(SOA and electrode site) or three (SOA, electrode site and hemisphere)
within-subject factors (note that the difference wave technique collapses
’Another advantage of this procedure is that it reduces the contribution of differential

prime-target ERP overlap for the three SOAS. As cao be seen in Figs 1 and 2, the early part
of the target ERR in the 200 SOA condition were “contaminated” by the €9and N400-like
negativity produced by the prime. By subtracting the related from the unrelated ERR, the
prime P2s and N40(k, which are similar for these two conditions, cancel each other out.
CROSS-MODAL SEMANTlC PRIMING AND ERPs 401
JI
+ + A
wL*
+ A
- OSOA ..... 100 SOA - WOSOA

FIG. 3 Difference waves calculated by subtracting related from unrelated target ERPs at
each of the three SOAS: (a) Experiment 1; (b) Experiment 2.
the target type factor). These difference wave analyses are intended to
serve as supplementary analyses to the more conventional ones reported
above.
As Fig. 3 illustrates, in Experiment 1 (visual prime and auditory target)
the difference between unrelated and related targets ( N W effect) was
large, peaked around 500-600msec and extended through much of the
measuring epoch in all three SOA conditions. However, in Experiment 2
(auditory prime and visual target), the difference was visible in only the
200 and 800 SOA conditions, peaked around 400-500msec and lasted only
until about 600msec.
A single latency window (200-700msec) was used to quantify the
difference waves. ANOVAs on this measure indicated that the N400 effect
was significantly greater in Experiment 1 than in Experiment 2 [main effect
of experiment, midline: F(1,22)= 5.61,P < 0.027; lateral: F(1,U) = 8.38,
P < 0.008]. The distribution of the effect was not significantly different
between the two experiments (midline: P > 0.90; lateral: P > 0.12),
although across experiments there was a difference in the distribution
[midline: F(2,44) = 3.95,P < 0.047; lateral: F(4,88) = 9.94,P < 0.0011,
indicating that the N400 effect was greater at the more posterior sites.
The interaction of SOA and experiment was significant at the midline
[F(2,44)= 4.01,P < 0.033) and approached significance at the lateral sites
[F(2,44)= 2.65,P < 0.0881.This interaction reflects the finding reported
earlier of statistically similar N400 effects of Experiment 1 (visual prime/
auditory target) but differential effects across SOA for Experiment 2
(auditory prime/visuaI target).
Since previous studies (e.g. Holcomb et al., 1992; Kutas, Van Petten,
& Besson, 1988)have found a difference in the laterality of the visual N400
effect at the temporal-parietal locations (right hemisphere usually greater

than left), separate analyses were done at Wernicke’s sites. Unlike many
previous studies using visual targets, there was no evidence of a significant
difference in the size of the effect between the hemispheres (P> 0.63).
GENERAL DISCUSSION
Summary of Findings
The main purpose of this study was to examine the interaction of word
processes between the visual and auditory modalities. This was done by
comparing semantic priming effects in two cross-modal experiments in
which the interval between the prime and target words was manipulated.
It was predicted that if a common semantic system is shared in the
processing of visual and auditory words, relatively consistent N400 effects
would be seen across SOAs. However, if cross-modal priming is due to a
word-level recoding process or to a post-recognition translation process,
priming would be delayed (due to the extra time required for conversion)
and should only occur for the 800 SOA condition.
When the prime was visual and the target auditory (Experiment l), there
were large behavioural and ERP priming effects observed at all three
SOAs, and the ERP effects onset at a relatively early point after the onset
of the target (between 200 and 400 msec in the 0 SOA condition). These
findings are most consistent with the common semantic system hypothesis
and would seem to contradict the alternative position that cross-modal
priming results from the addition of a word-level recoding or post-
recognition translation process. Further evidence in favour of the common
semantic system hypothesis comes from two sources. First, while the
auditory target effects (Experiment 1) tended to be larger than the visual
ones (Experiment 2), there were no reliable N400 differences across the
scalp between the modalities. That is, the N400 had the same posterior-
maximum, bilaterally symmetrical distribution regardless of whether the
target was visual or auditory. This suggests that for cross-modality presen-
tations, the N400 effect reflects little if any modality-specific pr~cessing.~
‘However, this finding would seem to be at odds with that of Domalski, Smith and Halgren
(1991), who reported a different scalp distribution for auditory-auditory and auditory-visual
words. In their study, auditory targets elicited a relatively larger frontal N400 effect and visual
targets a larger posterior effect. The most likely explanation for the discrepancies between
the studies is differences in methodologies. Dolmalski et al. (1991) w d repetition priming
(as opposed to semantic priming), a long primdrarget interval and an oldhew judgement
task (as opposed to a lexical decision task). Perhaps more important. their comparisons were
not symmetrical, with auditory targets being a withimmodality comparison and visual targets
a beween-modality comparison.
Second, the pattern of effects across SOAs in Experiment 1 was similar to
the within-modality studies of Anderson and Holcomb (submitted). This
suggests that the same mechanism was operating for between- and within-
modality semantic priming. This is inconsistent with the conversion
hypothesis, which argues that somewhat different mechanisms should
mediate priming within and between modalities.
However, it would appear that the above conclusions should be tem-
pered by the findings from Experiment 2, where the prime was auditory
and the target visual. Although reaction time differences were significant
in the 0 SOA condition, there was no semantic priming effect on the N400.
In the 200 SOA condition there were small N400 effects, while in the 800
SOA condition there were quite robust effects. These results are quite
different from Anderson and Holcomb’s (submitted) findings of clear N400

effects in simultaneous auditory-auditory and visual-visual presentations.
Taken together, the findings of short interval priming within modalities
and its absence between modalities (Experiment 2) would Seem to offer
some support for the conversion hypotheses.
Additional support for this position comes from another cross-modal
study by Rugg et al. (this issue). They found differences between the
modalities using a repetition priming task in which the first presentation
was in either the visual or auditory modality and the second presentation
(six items later) was in the same or different modality. The ERP priming
effects for auditory repetitions were very similar whether the prime was
auditory or visual. However, the repetition effect for the visual targets
onset later when the prime was auditory compared to when it was visual.
They interpreted this as evidence that visual words are automatically
converted to an auditory code, but that auditory words are not automati-
cally converted to a visual code. This latter conclusion assumes that the
delayed onset of the auditory/visual priming effect was due to the addition-
al time required to convert the visual target word into an auditory
representation (i.e. the conversion hypothesis). The failure to find evi-
dence of N400 priming in the 0 SOA auditory/visual condition of Experi-
ment 2 would appear to agree with Rugg and co-workers’ conclusions.
That is, in the 0 SOA condition, the completion of “visual” processing of
the target might have preceded completion of auditory recoding of the
target and therefore priming, as reflected by the N400,might not have had
time to take place. However, this interpretation is at a loss in explaining
the robust N400 effect in the 0 SOA condition of Experiment 1 (visual/
auditory). If the visual word has to be recoded to prime or be primed by
the auditory word, it is difficult to see how this could have happened in
time to support an N400 effect between 200 and 400 msec in Experiment
1, but not in Experiment 2.
However, there are at least two additional possibilities, both of which
are consistent with the common semantic system hypothesis. The first
explanation focuses on the temporal dynamics of spoken and written

words. While auditory words unfold over time, visual words are fully
available at stimulus onset. Even if spoken word processing is “on-line”,
it still may be that, at short SOAs, insufficient acoustic information has
reached the subject prior to processing of the visual target. What disting-
uishes this explanation from Rugg and co-workers’ account is that it does
not need to assume a conversion operation, but instead it suggests that
visual and auditory word processing precede along distinct and separate
routes, and then after recognition feed into a common semantic system
(where semantic priming presumably takes place). According to this view,
weak or non-existent priming could OCCUT if there was an absence of
temporal overlap in the processing of the prime and target by the semantic
system. In the case of the 0 SOA auditory/visualcondition, this could have

happened if the semantic system received and processed target word
information from the modality-specificvisual system prior to the arrival of
sufficient prime word information from the auditory system. This Seems
particularly plausible given the different temporal dynamics of spoken and
written words.
A final possibility for the discrepant 0 SOA findings focuses on the role
of attentional mechanisms. Even though it has been found that limitations
of divided attention are reduced when stimuli are presented cross-modally
rather than within modalities (e.g. Treisman & Davies, 1973), it is possible
that in each of the two experiments reported here there were differences
in the competition for attention resources. In support of this argument, it
has been shown that in some situations there is an attentional bias for
visual stimuli over auditory stimuli (Colavita, 1974;Colavita & Weisberg,
1979; Posner, Nissen, & Klein, 1976). For example, Colavita (1974)
required subjects to respond rapidly to a tone with one hand and to a light
with the other hand. He found that when there was an occasional simulta-
neously presented tone and tight (of equal perceived intensity), the sub
jects tended to respond to the visual stimulus and often reported being
unaware of the auditory stimulus. Similarly, Posner et al. (1W6)reported
a series of studies suggesting that visual stimuli tend to be less automati-
cally alerting, resulting in a need for attention to be more “actively”
directed towards them. According to this account, when there is competi-
tion between inputs from the two modalities, there is a reduction in the
amount of attention allocated to the auditory modality. This explanation
would be especially relevant in the 0 SOA condition. When both stimuli
were presented simultaneously, the visual stimulus (as either a prime or a
target) may have received preferential attention over the auditory stimu-
lus. This would have resulted in greater attentional resources being allo-
cated to the visual prime in Experiment 1 (visuaYauditory) and to the
visual target in Experiment 2 (auditory/visual). Since in Experiment 1 the
task was to respond to the auditory word, which was being presented over
time, there may have been less of a cost associated with the visual attention
bias, since there was still time to process the auditory target. Alternatively,
making the auditory stimulus the target might have demanded enough
attentional resources to offset the visual bias. However, in Experiment 2,
a response was also required to the visual target, which could have added
to the visual attentional bias resulting in less processing of the auditory
prime. This general notion of the N400 priming effect being affected by
attentional variables is consistent with the findings of Holcomb (1988), who
demonstrated that when attention was diverted from a prime word, the
target N400 was attenuated.
BehaviouralERP Differences
There was also evidence of differences in auditory and visual processing
in the comparisons between the behavioural and ERP data. For example,
although there were no ERP priming effects at the 0 SOA in Experiment
2, there was a significant (33msec) reaction time priming effect. One
possibility for these different findings is that the RT and N400 effects might
reflect the operation of a somewhat different set of processes (Holcomb,
1993; Kounios & Holcomb, 1992). For example, it may be that RT is more
sensitive to certain post-lexical decision processes such as Neely and
Keefe’s (1989) retrospective matching strategy. According to this account,
some part of RT semantic priming in the lexical decision task is due to the
relatedness of the prime and target being used to help make the appropri-
ate wordnonword decision. In the case of related primes and targets, when
a high degree of relatedness is detected, a rapid “word” decision can be
made, but in the case of an unrelated prime and target, there is an initial
tendency to decide “nonword” which delays the correct “word” response.
Evidence that subjects use this strategy comes from the higher error rates
for unrelated targets, which presumably result from subjects occasionally
acting on their initial semantically based impulse to respond “nonword”.
All of the unrelated conditions in both experiments produced higher error
rates than the related condition, suggesting that the subjects were using a
retrospective semantic-matching strategy. Unfortunately, there has not
been a study designed to look at the effect of the retrospective matching
strategy on the N400.However, Kounios and Holcomb (1992)have shown
that certain other late decision processes that influence RT in a sentence
verification task do not seem to have much impact on the N400.
Conclusions
Most of the data from the current experiments seem to favour the interpre-
tation that cross-modality semantic priming, as measured by the N400, is
mediated by a common semantic system that receives input from separate
modality-specific recognition systems. The data do not fit as well with
either of the alternative .interpretations (recoding or post-recognition
translation) because neither of these can account for the presence of
equivalent priming across SOAs in the visual/auditory experiment (Experi-
ment 1). The seemingly most damaging finding for the common semantic
system hypothesis was the absence of a robust ERP priming effect for the
0 SOA auditorylvisual condition. However, neither of the most plausible
explanations for this finding-namely, that visual target processing cap-
tured most of the available attentional resources leaving little for auditory
prime processing andor that visual target processing was fast enough to
not benefit from the partial auditory information available at this short
interval-are inconsistent with the common semantic system position.
One line of future studies will need to extend this research to other
domains. For example, while it makes sense that language would maintain
a common semantic system for spoken and written words, it is less clear
why image-based processes important for picture recognition would neces-
sarily tap the same semantic system. By examining the time-course of
wordpicture and picture/word priming, it should be possible to determine
if there is a similar or different pattern of priming between words and
pictures.
Manuscript received December 1991
Revised manuscript received April 1993
REFERENCES
Anderson, J.E.,& Holcornb, P.J. (submitted). Auditory and visual semantic priming using
different stimulus onset asynchronies: An event-related brain potential study.
Bradley, D.C.,& Fonter. K.I.(1987). A reader's view of listening. Cognirion,25,103-134.
Brown, C.. & Hagoort, P. (1993). The processing nature of the N400: Evidence from
masked priming. Journal of Cognitive Neuroscience, 5. 34-44.
Colavita. F.B. (1974). Human sensory dominance. Perception and Prychophysics, 16,
409-412.
Colavita. F.B., & Weisberg, D. (1979). A further investigation of visual dominance. Percep-
rion and Psychophysiu, 25,345-347.
Coltheart, M. (1978). Lexical aaxss in simple reading tasks. In G. Underwood (Ed.),
Strategies of inform'on processing. London: Academic Press.
Domalski, P., Smith, M.E.,& Halgren, E. (1991). Cross-modal repetition effects on the
N4. Psychological Science. 2 , 173-178.
Ellis. A.W., & Young, A.W. (1988). Human cognitive neuropsychology. Hove: Lawrence
Erlbaum Associates Ltd.
Geisscr, S.,& Greenhouse, S. (1959). On methods in the analysis of profile data. Psycho-
mem'ka, 24, 95-112.
Holcomb. P.J. (1988). Automatic and attcntional procews: An event-related brain poten-
tial analysis of semantic priming. Broin and Lunguge, 35,685.
Holcomb, P.J. (1993). Semantic priming and stimulus degradation: Implications for the role
of the N400 in language p-ing. P ~ ~ h o ~ h ~ s i o 30,
l o g47-61.
y,
Holcomb, P.J., & Neville, H.J. (1990). Semantic priming in visual and auditory lexical
decision: A between modality comparison. h g u a g e and Cognitive Procases, 5,281-312.
Holcomb, P.J., & Neville, H.J. (1991). The electrophysiology of spoken sentence proces-
sing. Psychobiology, 19, 286300.
Holcomb, P.J., Coffey, S., & Neville. H. (1992). The effects of context on visual and
auditory sentena processing: A developmental analysis using event-related brain poten-
tials. Developmend Newopsychology, 8, 203-241.
Kounios, J., & Holcomb, P.J. (1992). Structure and process in semantic memory: Evidence
from event-related braio potentials and reaction times. Journal of E X p c M Psy-
chology: General. 121,459-479.

Kroll, J.F., & Potter, M.C. (1984). Recognizing words, pictures and concepts: A compari-
son of lexical, object and reality decisions. Journal of Verbal h o m i n g and Verbal
Behavior. 23.39-66.
Kutas, M.,& Hillyard, S.A. (1980). Reading senseless sentences: Brain potentials reflect
semantic incongruity. Scuncc, 207, 203-205.
Kutas, M., & Hillyard, S.A. (1984). Brain potentials during reading reflect word expectancy
and semantic association. Nanue, 307. 161-163.
Kutas, M.,Van Pctten, C..& Besson, M. (1988). Event-related potential asymmetries
during reading of sentences. Electroencephalography and Clinical Newophysiology, 69,
218-233.
Marslen-Wilson, W. (1987). Functional parallelism in spoken word-recognition. Cognition,
25.71-102.
Mmlen-Wilson, W.D., & Tyler, L.K. (1980). The temporal structure of spoken language
understanding. Cognition. 8, 1-71.
Marslen-Wilson, W.D.. & Welsh, A. (1978). Processing interactions and lexical access
during word recognition in continuous speech. Cognitive Psychology, 10, 29-63.
Neely, J.H. (1991). Semantic priming effects in visual word recognition: A selective review
of current finding and theories. In D.Besner & G.W. Humphrcys (Eds), B a s i c p r o c ~ u
in reading: Visual word recogninbn, pp. 264-336. Hillsdale, NJ: Lawtence Erlbaum
Associates Inc.
Neely, J.H.. & Keefe, D.E. (1989). Semantic context effects on visual word processing: A
hybrid prospcctivdretrospective proccsing theory. In G.H. Bower (Ed.), The psychology
of learning and motivation, pp. 207-248. Advances in Research and Theory,Vol. 24. New
York: Academic Press.
Onifer, W.,& Swinney, D.A. (1981). Accessing lexical ambiguities during sentence compre-
hension: Effects of frequency of meaning and contextual bias. Memory and Cognition, 9.
225-236.
Osterhout, L., & Holcomb, P.J. (in press). Event-relatcd potentials and language compre-
hension. In M.D. Rugg & M.G.H. Coles (Eds), Elrctrophysbloglcal smdies of human
cognirion. Oxford: Oxford University Press.
Posner, M.I., Nissen, M.J., & Kleh, R.M. (1976). Visual dominance: An information-
processing account of its origins and significance. Psychological Review, 83,157-171.
Radeau, M., Morais, J., Mousty, P., Saereos, M.. & Bertelson, P. (1992). A listener's
investigation of printed word processing. J o v d of ExperimtOr Psychology: H m n
Perception and Performance, 18, 861-871.
Rugg, M. (1990).Event-related brain potentials dissociate repetition effects of high- and

low-frequency words. Memory and Cognition, 18,367-379.
Scidenberg, M.S.,Tanenhaus, M.K., Leiman, J.M.,& Bienkowski, M. (1982). Automatic
access of the meanings of ambiguous words in context: Some limitations of knowledge-
b u d processing. CO@~VCP@OlOgY, 14, 489-537.
Shallice, T. (1988). From nruropsychology ro menfal structure. Cambridge: Cambridge
University Press.
Swinney, D.A. (1979). Lexical access during Sentence comprehension: (Re)consideration of
context effects. J o m d of Verbal Learning and Verbal Behavior, 14, 645-660.
Swinney, D.A.. Onifer, W., Rather, P.,& Hirshkowitz, M. (1979). Semantic facilitation
across sensory modalities in the p r e s s i n g of individual words and sentences. Memory
and Cognition. 7, 159-165.
Treisman, A.M., k Davies, A. (1973). Divided attention to ear and eye. In S. Komblum
(Ed.), Anem’on ondperformonce N,pp. 101-117. New York: Academic Pres.
Zwitserlood. P. (1989). The IOCUS of the effects of sententid-semantic context in spoken-

word processing. Cognirion, 3 2 , 2 5 4 .
APPENDIX
Related Pairs
Lisr I List 2 List 3
bottom top help assist aunt uncle

wag tail jog run UP down
color red ocean sea bacon eggs
Skirt dress win loSe sweep broom
cat kitten snake Serpent dirty clean
Play time clock Yes no
brass bronze dead alive blanket sheet
sing song start stop sky blue
sleep dream doctor nu= april may
sick well spend Save rock stone
mildew mold wagon cart dog PUPPY
one two hug kiss Pots Pano
gr- green CUP saucer curtain drape
mow rain shoe sock rich Poor
tavern bar IUnCh dinner comb brush
Push Pull ship boat near far
jelly jam open close leap jump
War pea= beer wine lost found
heaven hell robin bird vapor stem
pork beef mix stir home how
thirst hunger read write salad SOUP
old new skunk Sti& stroll walk
CROSS-MODAL SEMANTIC PRIMING AND ERPS 4O9
Lirr 1 (Cont.) List 2 (Cont.) Lirr 3 (Cont.)

teach lean! scissor cut woman man
moon star lamb sheep frown smile
chair table half whole jail prison
stab knife king queen hands feet
salt pepper square circle stress tension
key lock ink pen Boat sink
bug insect Plus minus Past future
fresh stale hide seek flame fire
platter way shovel spade son father
shirt Pants due stick hrst last
river stream lease rent peach plum
gun shoot sad w nose
lake bad
pond P”Y god good

PW fail dim bright sit stand
calf COW g- duck church temple
silk satin wheat grain W m hot
scratch itch hop skip sting hurt
best worst fast slow step stair
Lkt 4 List 5 List 6
narrow wide her him elbow knee

inch foot kind mean plow field
monkey ape tUdC shell gamble bet
circus clown mob crowd blind deaf
angry mad cigar Pipe begin end
month Year BrSP hold breeze wind
baby child straw hay follow lead
leave stay lung heart nail hammer
rest relax chew swallow promise pledge
bps teeth long short earth world
borrow lend coward hero beetle roach
rough smooth wet dry punch hit
town city street road auto car
scarf neck more less iP tear
big large bible book true false
fox wolf bus truck fog haze
deep shallow rug Earpet gin vodka
rat mouse cash check farm barn
small tiny north south many few
pie cake taste smell love hate
almond nut Lime lemon greek latin
east west cough sneeze spice herb
hear listen labor work gem jewel
oil gas yours mine toy doll
stove cook wasp hornet dwarf midget
bag sack storm cloud curse swear
kill murder sugar Sweet couch sofa
me YOU scream yell bull steer
Litr 4 (Cont.) Lisr 5 (Cont.) List 6 (Cont.)
swindle cheat receive send drunk sober

out in mom dad sleet hail
nine ten sloppy neat hatchet ax
some none gate fence plate dish
floor Wall knit sew he she
pine Oak doe deer dusk dawn
fish swim on Off hard soft
weak strong dirt mud dull sharp
OWCe pound toll tax arm leg
sand beach eat drink grow shrink
talk SpcA steel iron dumb smart
milk Cream fork spoon i a cold
Word-Pseudoword Pain
Lirt I List 2 List 3
nickel plone finger meP razor fut

jet Pod ale stip early made
job dack raw smill bank fest
honey tuste hair papper skate wes
them teP repair kess odd wenter
delay feke coffee smike flower eags
eagle moY money thomb pea whote
hill crume juia shap needle drenk
give tarm funny tham happy grim
hieh broath bed cheld black ent
brave spoaker quiet nug once jonk
mintlc plant wutch gold het
Wife 0- head wab glove af
age dath jar nanc steak samrner
reach slem shrib tar cless
OW tosk raLC lettle bear soath
rabbit crob thin clawn ball wade
colt dask clay mew tire bist
bashful claf Pool namber dive misic
fat bradge bell tib donkey 4%
friend hant owl Pmce tree dad
spider nid touch dilliU orange boak
stock sheve spring moke dark hote
born nuck pain stond frog luds
caw plin candy lote bOY drap
line polace chalk rkall string ged
food im low nane Pig brish
wish nannal quick selid 80 navcr
price borth mint fulse front scster
drive erdcr Part COVil bat dine r
mouth simmit fact bruve Size lanch
=P queet paint lutter jaw bebblc
CROSS-MODAL SEMANTIC PRIMING AND ERPs 41 1
Lisr 1 (Cont.) Lisr 2 (Cont.) Lirf 3 (Coot.)
club drave MSt gome tape druft

show blick finish lig night xipe
frame lund art pem monster tabc
race dreve phone ClaK dime teble
valley Pag type eld train vosit
goal mim Pay seft soap tant
block aven space gerl watch plitter
switch froak move fober drum gutes

Holcomb 1993

Uploaded by

Copyright:

Available Formats

Holcomb 1993

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Holcomb 1993

Uploaded by

Copyright:

Available Formats

Language and Cognitive Processes

ISSN: 0169-0965 (Print) 1464-0732 (Online) Journal homepage: http://www.tandfonline.com/loi/plcp20

Cross-modal semantic priming: A time-course

Phillip J. Holcomb & Jane E. Anderson

To link to this article: http://dx.doi.org/10.1080/01690969308407583

Published online: 13 Dec 2007.

Submit your article to this journal

Article views: 118

View related articles

Citing articles: 54 View citing articles

Full Terms & Conditions of access and use can be found at

Download by: [University of Exeter] Date: 05 June 2016, At: 16:35

Cross-modal Semantic Priming: A Time-course

In two experiments, the timecourse of behavioural and electrophysiological

Requests for reprints should be addressed to Phillip J. Holcomb, Department of Psycho-

0 1993 Lawrence Erlbaum AsJociates Ltd and VSP Publications

encoding operations that a shared “common” set of processes and/or

1993; Rugg, 1990). In a host of studies (see Osterhout and Holcomb, in

In a second study, Holcomb and Neville (1991) demonstrated that the

different mechanisms underlie semantic priming for spoken and written

A second possibility, the common semantic system hypothesis, suggests

A number of studies have found evidence of between-modality priming

within-modality findings of Holcomb and Neville (1990; 1991) and Ander-

course of between-modality priming. In the first experiment, the prime

Stimuli 8nd Procedure

of 40 word-pseudoword pairs. The word pairs were counterbalanced so

viously found to be sensitive to language manipulations (cg. Holcomb,

and a hemisphere factor (left vs right). Significant target type x SOA

Beha vioural Data

SOA Related Words Unrelated Words Pseudowords

u)o RT 797ff2 mi81 !mi89

800 RT 779282 892f74 954f77

200 RT 670f 119 Mi137 813f141

800 RT 630+117 670f118 7762144

Target Type Effects

550-800 m c c . Unrelated targets continued to be more negative-going

there was no evidence of a significant difference in priming at the three

800-1150mec. In the final epoch, the unrelated targets continued to

midline -2.w -1.14 -2.sob 0.0043 NS

visual effects. In addition to task differences (repetition vs semantic

length. She found evidence of facilitation at short intervals (mean = 130

the present study.

Stimuli and Procedure

difference across SOAs (P> 0.68).

Target Type Effects

550-800 msec. There was no significant difference between related and

midline 0.35 -0.82 -3.5' 0.0103 0.0053

there was an interaction between target t y p and electrode site [F(4,44)=

8OeIIJOmsec. There was a main effect of target type at the midline

Time-course Anlllysu (IoOmec Epochs). Table 3 reports the results

’Another advantage of this procedure is that it reduces the contribution of differential

- OSOA ..... 100 SOA - WOSOA

effect at the temporal-parietal locations (right hemisphere usually greater

different from Anderson and Holcomb’s (submitted) findings of clear N400

explanation focuses on the temporal dynamics of spoken and written

system. In the case of the 0 SOA auditory/visualcondition, this could have

chology: General. 121,459-479.

Rugg, M. (1990).Event-related brain potentials dissociate repetition effects of high- and

Zwitserlood. P. (1989). The IOCUS of the effects of sententid-semantic context in spoken-

Lisr I List 2 List 3