Models of the neurobiology of language suggest that a small number of anatomically fixed brain re... more Models of the neurobiology of language suggest that a small number of anatomically fixed brain regions are responsible for language functioning. This observation derives from centuries of examining brain injury causing aphasia and is supported by decades of neuroimaging studies. The latter rely on thresholded measures of central tendency applied to activity patterns resulting from heterogeneous stimuli. We hypothesised that these methods obscure the whole brain distribution of regions supporting language. Specifically, cortical ‘language regions’ and the corresponding ‘language network’ consist of input regions and connectivity hubs. The latter primarily coordinate peripheral regions whose activity is variable, making them likely to be averaged out following thresholding. We tested these hypotheses in two studies using neuroimaging meta-analyses and functional magnetic resonance imaging during film watching. Both converged to suggest that averaging over heterogeneous words is locali...
Most research on the neurobiology of language ignores consciousness and vice versa. Here, languag... more Most research on the neurobiology of language ignores consciousness and vice versa. Here, language and inner speech are proposed to cause and sustain self-awareness and meta-self-awareness, i.e., extended consciousness. Converging empirical evidence supporting this proposal is reviewed and explanatory mechanisms are described. The latter are embedded in a ‘HOLISTIC’ model of the supporting neurobiology that involves a ‘core’ set of inner speech production regions that initiate conscious ‘overhearing’ of words. These have an affective quality deriving from activation of associated sensory, motor, and emotional representations, involving a largely unconscious dynamic ‘periphery’, distributed throughout the whole brain. This is the basis for inner conversations, involving ‘default mode’ activation and prefrontal and thalamic/brainstem selection of contextually relevant responses. Entrenched patterns of connectivity in these networks form the basis of diminished wellbeing and mental hea...
Rodent and human studies have implicated an amygdala-prefrontal circuit during threat processing.... more Rodent and human studies have implicated an amygdala-prefrontal circuit during threat processing. One possibility is that while amygdala activity underlies core features of anxiety (e.g. detection of salient information), prefrontal cortices (i.e. dorsomedial prefrontal/anterior cingulate cortex) entrain its responsiveness. To date, this has been established in tightly controlled paradigms (predominantly using static face perception tasks) but has not been extended to more naturalistic settings. Consequently, using ‘movie fMRI’—in which participants watch ecologically-richer movies rather than constrained cognitive tasks—we sought to test whether individual differences in anxiety correlate with the degree of face-dependent amygdala-prefrontal coupling in two independent samples. Analyses suggested increased face-dependent superior parietal activation and decreased speech-dependent auditory cortex activation as a function of anxiety. However, we failed to find evidence for anxiety-de...
Neuroimaging has advanced our understanding of human psychology using reductionist stimuli that o... more Neuroimaging has advanced our understanding of human psychology using reductionist stimuli that often do not resemble information the brain naturally encounters. It has improved our understanding of the network organization of the brain mostly through analyses of ‘resting-state’ data for which the functions of networks cannot be verifiably labelled. We make a ‘Naturalistic Neuroimaging Database’ (NNDb v1.0) publically available to allow for a more complete understanding of the brain under more ecological conditions during which networks can be labelled. Eighty-six participants underwent behavioural testing and watched one of 10 full-length movies while functional magnetic resonance imaging was acquired. Resulting timeseries data are shown to be of high quality, with good signal-to-noise ratio, few outliers and low movement. Data-driven functional analyses provide further evidence of data quality. They also demonstrate accurate timeseries/movie alignment and how movie annotations mig...
Proceedings of the National Academy of Sciences, 2020
SignificanceSpeech perception is improved when we are able to see the person who is speaking, but... more SignificanceSpeech perception is improved when we are able to see the person who is speaking, but how visual speech cues are used to improve speech perception is currently unclear. Brain imaging has revealed that regions responsible for motor control are active during the perception of speech, opening up the possibility that visual cues are mapped onto an internal representation of the vocal tract. Here, we show that there is sufficient information in the configuration of the face to recover the vocal tract configuration and that the key areas responsible for driving the correspondence vary in accordance with the articulation required to form the acoustic signal at the appropriate point in a sentence.
The role of the cerebellum in speech perception remains a mystery. Given its uniform architecture... more The role of the cerebellum in speech perception remains a mystery. Given its uniform architecture, we tested the hypothesis that it implements a domain-general mechanism whose role in speech is determined by connectivity. We collated all neuroimaging studies reporting cerebellar activity in the Neurosynth database (n = 8,206). From this set, we found all studies involving passive speech and sound perception (n = 72, 64% speech, 12.5% sounds, 12.5% music, and 11% tones) and speech production and articulation (n = 175). Standard and coactivation neuroimaging meta-analyses were used to compare cerebellar and associated cortical activations between passive perception and production. We found distinct regions of perception-and production-related activity in the cerebellum and regions of perception-production overlap. Each of these regions had distinct patterns of cortico-cerebellar connectivity. To test for domain generality versus specificity, we identified all psychological and task-re...
There is a widespread assumption that there are a static set of ‘language regions’ in the brain. ... more There is a widespread assumption that there are a static set of ‘language regions’ in the brain. Yet, people still regularly produce familiar ‘formulaic’ expressions when those regions are severely damaged. This suggests that the neurobiology of language varies with the extent of word sequence learning and might not be fixed. We test the hypothesis that perceiving sentences is mostly supported by sensorimotor regions involved in speech production and not ‘language regions’ after overlearning. Twelve participants underwent two sessions of behavioural testing and functional magnetic resonance imaging (fMRI), separated by 15 days. During this period, they repeated two sentences 30 times each, twice a day. In both fMRI sessions, participants ‘passively’ listened to those two sentences and novel sentences. Lastly, they spoke novel sentences. Behavioural results confirm that participants overlearned sentences. Correspondingly, there was an increase or recruitment of sensorimotor regions i...
Communication naturally occurs in dynamic face-to-face environments where spoken words are embedd... more Communication naturally occurs in dynamic face-to-face environments where spoken words are embedded in linguistic discourse and accompanied by multimodal cues. Existing research supports predictive brain models where prior discourse but also prosody, mouth movements, and hand-gestures individually contribute to comprehension. In electroencephalography (EEG) studies, more predictable words show reduced negativity, peaking at approximately 400ms after onset of the word (N400). Meaningful gestures and prosody also reduce the N400, while the effects of rhythmic gestures and mouth movements remain unclear. However, these studies have only focused on individual cues while in the real world, communicative cues co-occur and potentially interact. We measured EEG elicited by words while participants watched videos of a speaker producing short naturalistic passages. For each word, we quantified the information carried by prior linguistic discourse (surprisal), prosody (mean pitch), mouth infor...
How is speech understood despite the lack of a deterministic relationship between the sounds reac... more How is speech understood despite the lack of a deterministic relationship between the sounds reaching auditory cortex and what we perceive? One possibility is that unheard words that are unconsciously activated in association with listening context are used to constrain interpretation. We hypothesized that a mechanism for doing so involves reusing the ability of the brain to predict the sensory effects of speaking associated words. Predictions are then compared to signals arriving in auditory cortex, resulting in reduced processing demands when accurate. Indeed, we show that sensorimotor brain regions are more active prior to words predictable from listening context. This activity resembles lexical and speech production related processes and, specifically, subsequent but still unpresented words. When those words occur, auditory cortex activity is reduced, through feedback connectivity. In less predictive contexts, activity patterns and connectivity for the same words are markedly di...
Functional magnetic resonance imaging (fMRI) studies of speech sound categorization often compare... more Functional magnetic resonance imaging (fMRI) studies of speech sound categorization often compare conditions in which a stimulus is presented repeatedly to conditions in which multiple stimuli are presented. This approach has established that a set of superior temporal and inferior parietal regions respond more strongly to conditions containing stimulus change. Here, we examine whether this contrast is driven by habituation to a repeating condition or by selective responding to change. Experiment 1 directly tests this by comparing the observed response to long trains of stimuli against a constructed hemodynamic response modeling the hypothesis that no habituation occurs. The results are consistent with the view that enhanced response to conditions involving phonemic variability reflect change detection. In a second experiment, the specificity of these responses to linguistically relevant stimulus variability was studied by including a condition in which the talker, rather than phone...
Quarterly Journal of Experimental Psychology, 2011
During a conversation, we hear the sound of the talker as well as the intended message. Tradition... more During a conversation, we hear the sound of the talker as well as the intended message. Traditional models of speech perception posit that acoustic details of a talker's voice are not encoded with the message whereas more recent models propose that talker identity is automatically encoded. When shadowing speech, listeners often fail to detect a change in talker identity. The present study was designed to investigate whether talker changes would be detected when listeners are actively engaged in a normal conversation, and visual information about the speaker is absent. Participants were called on the phone, and during the conversation the experimenter was surreptitiously replaced by another talker. Participants rarely noticed the change. However, when explicitly monitoring for a change, detection increased. Voice memory tests suggested that participants remembered only coarse information about both voices, rather than fine details. This suggests that although listeners are capabl...
Models of the neurobiology of language suggest that a small number of anatomically fixed brain re... more Models of the neurobiology of language suggest that a small number of anatomically fixed brain regions are responsible for language functioning. This observation derives from centuries of examining brain injury causing aphasia and is supported by decades of neuroimaging studies. The latter rely on thresholded measures of central tendency applied to activity patterns resulting from heterogeneous stimuli. We hypothesised that these methods obscure the whole brain distribution of regions supporting language. Specifically, cortical ‘language regions’ and the corresponding ‘language network’ consist of input regions and connectivity hubs. The latter primarily coordinate peripheral regions whose activity is variable, making them likely to be averaged out following thresholding. We tested these hypotheses in two studies using neuroimaging meta-analyses and functional magnetic resonance imaging during film watching. Both converged to suggest that averaging over heterogeneous words is locali...
Most research on the neurobiology of language ignores consciousness and vice versa. Here, languag... more Most research on the neurobiology of language ignores consciousness and vice versa. Here, language and inner speech are proposed to cause and sustain self-awareness and meta-self-awareness, i.e., extended consciousness. Converging empirical evidence supporting this proposal is reviewed and explanatory mechanisms are described. The latter are embedded in a ‘HOLISTIC’ model of the supporting neurobiology that involves a ‘core’ set of inner speech production regions that initiate conscious ‘overhearing’ of words. These have an affective quality deriving from activation of associated sensory, motor, and emotional representations, involving a largely unconscious dynamic ‘periphery’, distributed throughout the whole brain. This is the basis for inner conversations, involving ‘default mode’ activation and prefrontal and thalamic/brainstem selection of contextually relevant responses. Entrenched patterns of connectivity in these networks form the basis of diminished wellbeing and mental hea...
Rodent and human studies have implicated an amygdala-prefrontal circuit during threat processing.... more Rodent and human studies have implicated an amygdala-prefrontal circuit during threat processing. One possibility is that while amygdala activity underlies core features of anxiety (e.g. detection of salient information), prefrontal cortices (i.e. dorsomedial prefrontal/anterior cingulate cortex) entrain its responsiveness. To date, this has been established in tightly controlled paradigms (predominantly using static face perception tasks) but has not been extended to more naturalistic settings. Consequently, using ‘movie fMRI’—in which participants watch ecologically-richer movies rather than constrained cognitive tasks—we sought to test whether individual differences in anxiety correlate with the degree of face-dependent amygdala-prefrontal coupling in two independent samples. Analyses suggested increased face-dependent superior parietal activation and decreased speech-dependent auditory cortex activation as a function of anxiety. However, we failed to find evidence for anxiety-de...
Neuroimaging has advanced our understanding of human psychology using reductionist stimuli that o... more Neuroimaging has advanced our understanding of human psychology using reductionist stimuli that often do not resemble information the brain naturally encounters. It has improved our understanding of the network organization of the brain mostly through analyses of ‘resting-state’ data for which the functions of networks cannot be verifiably labelled. We make a ‘Naturalistic Neuroimaging Database’ (NNDb v1.0) publically available to allow for a more complete understanding of the brain under more ecological conditions during which networks can be labelled. Eighty-six participants underwent behavioural testing and watched one of 10 full-length movies while functional magnetic resonance imaging was acquired. Resulting timeseries data are shown to be of high quality, with good signal-to-noise ratio, few outliers and low movement. Data-driven functional analyses provide further evidence of data quality. They also demonstrate accurate timeseries/movie alignment and how movie annotations mig...
Proceedings of the National Academy of Sciences, 2020
SignificanceSpeech perception is improved when we are able to see the person who is speaking, but... more SignificanceSpeech perception is improved when we are able to see the person who is speaking, but how visual speech cues are used to improve speech perception is currently unclear. Brain imaging has revealed that regions responsible for motor control are active during the perception of speech, opening up the possibility that visual cues are mapped onto an internal representation of the vocal tract. Here, we show that there is sufficient information in the configuration of the face to recover the vocal tract configuration and that the key areas responsible for driving the correspondence vary in accordance with the articulation required to form the acoustic signal at the appropriate point in a sentence.
The role of the cerebellum in speech perception remains a mystery. Given its uniform architecture... more The role of the cerebellum in speech perception remains a mystery. Given its uniform architecture, we tested the hypothesis that it implements a domain-general mechanism whose role in speech is determined by connectivity. We collated all neuroimaging studies reporting cerebellar activity in the Neurosynth database (n = 8,206). From this set, we found all studies involving passive speech and sound perception (n = 72, 64% speech, 12.5% sounds, 12.5% music, and 11% tones) and speech production and articulation (n = 175). Standard and coactivation neuroimaging meta-analyses were used to compare cerebellar and associated cortical activations between passive perception and production. We found distinct regions of perception-and production-related activity in the cerebellum and regions of perception-production overlap. Each of these regions had distinct patterns of cortico-cerebellar connectivity. To test for domain generality versus specificity, we identified all psychological and task-re...
There is a widespread assumption that there are a static set of ‘language regions’ in the brain. ... more There is a widespread assumption that there are a static set of ‘language regions’ in the brain. Yet, people still regularly produce familiar ‘formulaic’ expressions when those regions are severely damaged. This suggests that the neurobiology of language varies with the extent of word sequence learning and might not be fixed. We test the hypothesis that perceiving sentences is mostly supported by sensorimotor regions involved in speech production and not ‘language regions’ after overlearning. Twelve participants underwent two sessions of behavioural testing and functional magnetic resonance imaging (fMRI), separated by 15 days. During this period, they repeated two sentences 30 times each, twice a day. In both fMRI sessions, participants ‘passively’ listened to those two sentences and novel sentences. Lastly, they spoke novel sentences. Behavioural results confirm that participants overlearned sentences. Correspondingly, there was an increase or recruitment of sensorimotor regions i...
Communication naturally occurs in dynamic face-to-face environments where spoken words are embedd... more Communication naturally occurs in dynamic face-to-face environments where spoken words are embedded in linguistic discourse and accompanied by multimodal cues. Existing research supports predictive brain models where prior discourse but also prosody, mouth movements, and hand-gestures individually contribute to comprehension. In electroencephalography (EEG) studies, more predictable words show reduced negativity, peaking at approximately 400ms after onset of the word (N400). Meaningful gestures and prosody also reduce the N400, while the effects of rhythmic gestures and mouth movements remain unclear. However, these studies have only focused on individual cues while in the real world, communicative cues co-occur and potentially interact. We measured EEG elicited by words while participants watched videos of a speaker producing short naturalistic passages. For each word, we quantified the information carried by prior linguistic discourse (surprisal), prosody (mean pitch), mouth infor...
How is speech understood despite the lack of a deterministic relationship between the sounds reac... more How is speech understood despite the lack of a deterministic relationship between the sounds reaching auditory cortex and what we perceive? One possibility is that unheard words that are unconsciously activated in association with listening context are used to constrain interpretation. We hypothesized that a mechanism for doing so involves reusing the ability of the brain to predict the sensory effects of speaking associated words. Predictions are then compared to signals arriving in auditory cortex, resulting in reduced processing demands when accurate. Indeed, we show that sensorimotor brain regions are more active prior to words predictable from listening context. This activity resembles lexical and speech production related processes and, specifically, subsequent but still unpresented words. When those words occur, auditory cortex activity is reduced, through feedback connectivity. In less predictive contexts, activity patterns and connectivity for the same words are markedly di...
Functional magnetic resonance imaging (fMRI) studies of speech sound categorization often compare... more Functional magnetic resonance imaging (fMRI) studies of speech sound categorization often compare conditions in which a stimulus is presented repeatedly to conditions in which multiple stimuli are presented. This approach has established that a set of superior temporal and inferior parietal regions respond more strongly to conditions containing stimulus change. Here, we examine whether this contrast is driven by habituation to a repeating condition or by selective responding to change. Experiment 1 directly tests this by comparing the observed response to long trains of stimuli against a constructed hemodynamic response modeling the hypothesis that no habituation occurs. The results are consistent with the view that enhanced response to conditions involving phonemic variability reflect change detection. In a second experiment, the specificity of these responses to linguistically relevant stimulus variability was studied by including a condition in which the talker, rather than phone...
Quarterly Journal of Experimental Psychology, 2011
During a conversation, we hear the sound of the talker as well as the intended message. Tradition... more During a conversation, we hear the sound of the talker as well as the intended message. Traditional models of speech perception posit that acoustic details of a talker's voice are not encoded with the message whereas more recent models propose that talker identity is automatically encoded. When shadowing speech, listeners often fail to detect a change in talker identity. The present study was designed to investigate whether talker changes would be detected when listeners are actively engaged in a normal conversation, and visual information about the speaker is absent. Participants were called on the phone, and during the conversation the experimenter was surreptitiously replaced by another talker. Participants rarely noticed the change. However, when explicitly monitoring for a change, detection increased. Voice memory tests suggested that participants remembered only coarse information about both voices, rather than fine details. This suggests that although listeners are capabl...
Uploads
Papers by Jeremy Skipper