Contesti, pratiche e risorse della comunicazione multimodale, 2024
The aim of this paper is to present a learner corpus collected in a volunteer organization in Nap... more The aim of this paper is to present a learner corpus collected in a volunteer organization in Naples as part of a still ongoing project on plurilingual and intercultural education and translanguaging. So far, we have collected the linguistic portraits and oral autobiographies in L2 Italian of 171 adult immigrants, including 81 women, mean age 32, representing 40 nationalities, 28 mother tongues and different levels of second language proficiency and educational backgrounds. The article proposes a multidimensional analysis of the acquired data, based on graphic, textual and prosodic aspects, with the aim of observing affective schemes and attitudes in the multimodal representation of plural linguistic repertoires. Some significant examples from the corpus are provided and described in detail.
Parkinson's Disease dysarthria affects the speech motor control, causing alterations at the supra... more Parkinson's Disease dysarthria affects the speech motor control, causing alterations at the suprasegmental level of speech. In previous researches, vowel percentage (%V) and the mean interval between two consecutive vowel onset points (VtoV) were effectively used in the synchronic description of the rhythmic variations of Italian PD speech, compared to healthy speech, even at a very early stage of the disease. This study aims at verifying the early alteration of PD speech rhythm using a diachronic approach. To reach this goal, a corpus of read speech produced by a single PD subject (female, 66 years old) has been collected, consisting of 15 radiophonic speech samples (about 100 s each) on the same topic, recorded between 2001 and 2021. The speech samples were manually segmented in consonantal and vocalic intervals by means of Praat, allowing the calculation of %V and VtoV. The results show an alteration of %V values since 2018, two years before the diagnosis and the insurgence of motor symptoms. Moreover, first results of the application of the automatic segmentation performed by SPPAS on a selection of PD speech samples will also be presented.
The development of dialogue systems benefits from the study of the communication strategies used ... more The development of dialogue systems benefits from the study of the communication strategies used by human speakers. In the context of recommendation dialogue systems some researchers have investigated the sociable recommendation strategies employed by the Recommenders in natural settings to make successful and persuasive recommendations (Hayati et al., 2020, INSPIRED corpus). However, the Seeker’s contribution, as well as the Recommender’s, shapes the development of the communicative exchange, in that the Seekers may use specific strategies to disclose their preferences and reach their goal. So, modelling the Seeker’s communicative strategies along with the ones used by the Recommender may improve the efficiency of recommendation dialogue systems. In this work, we provide a reliable tagset for the Seekers utterances present in the Inspired dataset, defining a set of communicative strategies coherent with the already existing one for the Recommenders.
This contribution is meant to give an insight into the topic of L2 acquisition and teaching in th... more This contribution is meant to give an insight into the topic of L2 acquisition and teaching in the case of low-literate adult learners. To the aim, two applied researches will be presented, both focusing on L2 Italian: one investigates the oral skills of Senegalese learners with different educational backgrounds in the country of origin; the other concerns the assessment of L2 writing skills in a multilingual group of refugees and asylum seekers. The results of both studies contribute to look at this peculiar target of learners from an unusual perspective, unveiling skills that can often be “invisible” to L2 literacy teachers
Questo lavoro propone l’analisi di un caso particolare di parlato filmico doppiato. Nello specifi... more Questo lavoro propone l’analisi di un caso particolare di parlato filmico doppiato. Nello specifico, intende osservare in che modo il processo di apprendimento spontaneo di una lingua seconda da parte di un adulto sia stato rappresentato nel doppiaggio in lingua italiana del film The Terminal, diretto da S. Spielberg. Attraverso un’analisi formale delle trascrizioni di tutti i turni di parola del protagonista, si è tentato di rintracciare gli stadi di sviluppo della sua interlingua, con particolare riferimento all’espressione della temporalità/aspettualità/modalità verbale e alla struttura dell’accordo di genere e numero. I dati raccolti dimostrano un parziale tentativo di riprodurre in maniera realistica le sequenze di apprendimento, con esiti differenziati a seconda della struttura morfosintattica osservata. A differenza di quanto accade nella versione originale del film in lingua inglese, la resa italiana di The Terminal sembra non riuscire a rendere in maniera scientificamente r...
The purpose of this study is to investigate the role of voice, text and body language in message ... more The purpose of this study is to investigate the role of voice, text and body language in message decoding by native Italian speakers. To the purpose, a script based on a real life situation - the discovery of a partner’s affair - was written. Non professional actors were videotaped while reciting the story in a neutral environment. No props, no costumes were indeed used to indicate environments and characters. The collected corpus of speech was segmented into syllables, then the syllables of the original dialogue were replaced with number sequences. The actors read the new dialogue based on numbers without phrasing, inflection, and intonation, but pausing according to the silences marked in the text. The numeric dialogue was segmented into syllables and then, through the transplantation technique, the prosodic features of the original dialogue were transferred to the dialogue based on number. After that, three tests were administered to three groups of native Italian subjects. One g...
A training experience that takes advantage of technologies for acoustic analysis in order to tayl... more A training experience that takes advantage of technologies for acoustic analysis in order to taylor the learning experience on learners abilities and learning style.
The present study is intended to compare two approaches of labeling expressive corpora: auto-anno... more The present study is intended to compare two approaches of labeling expressive corpora: auto-annotation and annotation by external lay listeners. These two methods have been applied to the semi-spontaneous emotional speech produced by Chinese learners of L2 Italian, involved in the CardTask, a moodinduction procedure that allows us to control the context of interaction, preserving the spontaneity of reactions. The emotional responses to the stimuli presented in the task were the object of an auto-annotation session. The same samples were then administered only in the auditory mode to 20 Italian and 20 Chinese lay listeners. The results of perceptual tests have underlined some similarities and differences between both autoand external annotation, and between the ratings given by external Italian and Chinese listeners. The labels chosen by native Italians were similar to those selected in the auto-annotation session, particularly in the case of anxiety, fear and disgust. The correspon...
The study of vocal emotions poses different methodological problems, especially regarding the sty... more The study of vocal emotions poses different methodological problems, especially regarding the style of speech analyzed. If spontaneous speech on the one hand guarantees data authenticity, on the other hand does not allow to control the variables affecting spoken communication. Recited speech allows to control the verbal content of emotional productions and obtain easily comparable data. Dubbed speech represents another feasible alternative. The task of a dubber is to use a different language to vehiculate the same emotional state expressed in the original version, to sound as natural as possible. The present study has twofold objective: firstly it aims to describe the prosodic characteristics of dubbed speech, to evaluate its effectiveness for the cross-linguistic analysis of vocal emotions; secondly it is intended to verify if romance and non-romance languages present similar prosodic patterns in the expression of different emotional states. To reach these goals, sequences taken fr...
Patients with Parkinson's disease (PD) usually reveal speech disorders and, among other sympt... more Patients with Parkinson's disease (PD) usually reveal speech disorders and, among other symptoms, the alteration of speech rhythm. The purpose of this study is twofold: (1) to test the validity of two acoustic parameters—%V, vowel percentage and VtoV, the mean interval between two consecutive vowel onset points—for the identification of rhythm variation in early-stage PD speech and (2) to analyze the effect of PD on speech rhythm in two different speaking tasks: reading passage and monolog. A group of 20 patients with early-stage PD was involved in this study and compared with 20 age- and sex-matched healthy controls (HCs). The results of the acoustic analysis confirmed that %V is a useful cue for early-stage PD speech characterization, having significantly higher values in the production of patients with PD than the values in HC speech. A simple speaking task, such as the reading task, was found to be more effective than spontaneous speech in the detection of rhythmic variations.
Parkinson’s Disease (PD) is a neurological illness which also has effects on speech production, r... more Parkinson’s Disease (PD) is a neurological illness which also has effects on speech production, resulting in segmental and suprasegmental abnormalities. The aim of the current study is to test the validity of two acoustic parameters %V and VtoV for the detection of rhythmical variation in early-stage PD speech, in comparison to healthy speech. 40 Italian native speakers were enrolled in the research, 20 early-stage PD subjects and 20 neurologically healthy and matched controls, and a corpus of read speech was collected. The results of voice analysis confirmed an alteration of vocalic duration and %V in PD productions. In particular, %V could be a reliable cue for PD speech characterization, even at the very early onset of the disease.
The present study is intended to figure out the extent to which prosody and intonation affect lis... more The present study is intended to figure out the extent to which prosody and intonation affect listeners’ ability to estimate the speaker’s age. The performance of a 40-year old anchorman and another by the same speaker at the age of 80 were spectroacoustically analyzed in order to identify the prosodic features of a “young” and an “old” voice. The results of the analysis have shown significant differences between the two voices on a suprasegmental level. To test the effects of these differences on a perceptual level, through the prosodic transplantation technique, the F0 values and the durations of segments and silences were transferred from the “young” to the “old” voice and viceversa. Two age recognition tests, based on original and transplanted voices, were administered to Italian listeners. The results of perceptual tests have confirmed the strict relationship between some rhythmic and prosodic features and the speaker’s age and have demonstrated the effectiveness of the transpl...
Parkinson’s Disease (PD) is a neurological illness which also has effects on speech production, r... more Parkinson’s Disease (PD) is a neurological illness which also has effects on speech production, resulting in segmental and suprasegmental abnormalities. The aim of the current study is to test the validity of two acoustic parameters - %V and VtoV - for the detection of rhythmical variation in early-stage PD speech, in comparison to healthy speech. 40 Italian native speakers were enrolled in the research, 20 early-stage PD subjects and 20 neurologically healthy and matched controls, and a corpus of read speech was collected. The results of voice analysis confirmed an alteration of vocalic duration and %V in PD productions. In particular, %V could be a reliable cue for PD speech characterization, even at the very early onset of the disease.
The influence of L1 literacy on the acquisition of L2 oracy has recently become one of the intere... more The influence of L1 literacy on the acquisition of L2 oracy has recently become one of the interests of SLA research. However, what is still missing is an exploration of the development of L2 prosodic competence by low-literate adults learners. To fill this gap, in the present study Senegalese learners of L2 Italian were involved. In Senegal, where literacy acquisition is usually absent in the L1 (mostly Wolof), it is possible to acquire it in French or Arabic, albeit the educational models of the French and Koranic schools are quite different. In French schools writing and reading practices are favored, whereas in Koranic schools speaking and listening abilities are mostly practiced by reciting the Koran. In order to understand the relationship between L1 literacy and L2 oracy in the case of low-literate Senegalese learners of L2 Italian and to verify if the different educational models proposed in Senegal influence L2 prosodic competence even of low-educated learners, perceptual a...
The present study is intended to figure out the extent to which prosody and intonation affect lis... more The present study is intended to figure out the extent to which prosody and intonation affect listeners’ ability to estimate the speaker’s age. The performance of a 40-year old anchorman and another by the same speaker at the age of 80 were spectroacoustically analyzed in order to identify the prosodic features of a “young” and an “old” voice. The results of the analysis have shown significant differences between the two voices on a suprasegmental level. To test the effects of these differences on a perceptual level, through the prosodic transplantation technique, the F0 values and the durations of segments and silences were transferred from the “young” to the “old” voice and viceversa. Two age recognition tests, based on original and transplanted voices, were administered to Italian listeners. The results of perceptual tests have confirmed the strict relationship between some rhythmic and prosodic features and the speaker’s age and have demonstrated the effectiveness of the transplantation technique. With advancing age, articulation rate and speech rate slow down, voice register rises and tonal range widens. Moreover, the “old” voice is also characterized by a higher percentage of vocalic portion which determines a shift of the Italian rhythm towards the isomoraic pattern.
Contesti, pratiche e risorse della comunicazione multimodale, 2024
The aim of this paper is to present a learner corpus collected in a volunteer organization in Nap... more The aim of this paper is to present a learner corpus collected in a volunteer organization in Naples as part of a still ongoing project on plurilingual and intercultural education and translanguaging. So far, we have collected the linguistic portraits and oral autobiographies in L2 Italian of 171 adult immigrants, including 81 women, mean age 32, representing 40 nationalities, 28 mother tongues and different levels of second language proficiency and educational backgrounds. The article proposes a multidimensional analysis of the acquired data, based on graphic, textual and prosodic aspects, with the aim of observing affective schemes and attitudes in the multimodal representation of plural linguistic repertoires. Some significant examples from the corpus are provided and described in detail.
Parkinson's Disease dysarthria affects the speech motor control, causing alterations at the supra... more Parkinson's Disease dysarthria affects the speech motor control, causing alterations at the suprasegmental level of speech. In previous researches, vowel percentage (%V) and the mean interval between two consecutive vowel onset points (VtoV) were effectively used in the synchronic description of the rhythmic variations of Italian PD speech, compared to healthy speech, even at a very early stage of the disease. This study aims at verifying the early alteration of PD speech rhythm using a diachronic approach. To reach this goal, a corpus of read speech produced by a single PD subject (female, 66 years old) has been collected, consisting of 15 radiophonic speech samples (about 100 s each) on the same topic, recorded between 2001 and 2021. The speech samples were manually segmented in consonantal and vocalic intervals by means of Praat, allowing the calculation of %V and VtoV. The results show an alteration of %V values since 2018, two years before the diagnosis and the insurgence of motor symptoms. Moreover, first results of the application of the automatic segmentation performed by SPPAS on a selection of PD speech samples will also be presented.
The development of dialogue systems benefits from the study of the communication strategies used ... more The development of dialogue systems benefits from the study of the communication strategies used by human speakers. In the context of recommendation dialogue systems some researchers have investigated the sociable recommendation strategies employed by the Recommenders in natural settings to make successful and persuasive recommendations (Hayati et al., 2020, INSPIRED corpus). However, the Seeker’s contribution, as well as the Recommender’s, shapes the development of the communicative exchange, in that the Seekers may use specific strategies to disclose their preferences and reach their goal. So, modelling the Seeker’s communicative strategies along with the ones used by the Recommender may improve the efficiency of recommendation dialogue systems. In this work, we provide a reliable tagset for the Seekers utterances present in the Inspired dataset, defining a set of communicative strategies coherent with the already existing one for the Recommenders.
This contribution is meant to give an insight into the topic of L2 acquisition and teaching in th... more This contribution is meant to give an insight into the topic of L2 acquisition and teaching in the case of low-literate adult learners. To the aim, two applied researches will be presented, both focusing on L2 Italian: one investigates the oral skills of Senegalese learners with different educational backgrounds in the country of origin; the other concerns the assessment of L2 writing skills in a multilingual group of refugees and asylum seekers. The results of both studies contribute to look at this peculiar target of learners from an unusual perspective, unveiling skills that can often be “invisible” to L2 literacy teachers
Questo lavoro propone l’analisi di un caso particolare di parlato filmico doppiato. Nello specifi... more Questo lavoro propone l’analisi di un caso particolare di parlato filmico doppiato. Nello specifico, intende osservare in che modo il processo di apprendimento spontaneo di una lingua seconda da parte di un adulto sia stato rappresentato nel doppiaggio in lingua italiana del film The Terminal, diretto da S. Spielberg. Attraverso un’analisi formale delle trascrizioni di tutti i turni di parola del protagonista, si è tentato di rintracciare gli stadi di sviluppo della sua interlingua, con particolare riferimento all’espressione della temporalità/aspettualità/modalità verbale e alla struttura dell’accordo di genere e numero. I dati raccolti dimostrano un parziale tentativo di riprodurre in maniera realistica le sequenze di apprendimento, con esiti differenziati a seconda della struttura morfosintattica osservata. A differenza di quanto accade nella versione originale del film in lingua inglese, la resa italiana di The Terminal sembra non riuscire a rendere in maniera scientificamente r...
The purpose of this study is to investigate the role of voice, text and body language in message ... more The purpose of this study is to investigate the role of voice, text and body language in message decoding by native Italian speakers. To the purpose, a script based on a real life situation - the discovery of a partner’s affair - was written. Non professional actors were videotaped while reciting the story in a neutral environment. No props, no costumes were indeed used to indicate environments and characters. The collected corpus of speech was segmented into syllables, then the syllables of the original dialogue were replaced with number sequences. The actors read the new dialogue based on numbers without phrasing, inflection, and intonation, but pausing according to the silences marked in the text. The numeric dialogue was segmented into syllables and then, through the transplantation technique, the prosodic features of the original dialogue were transferred to the dialogue based on number. After that, three tests were administered to three groups of native Italian subjects. One g...
A training experience that takes advantage of technologies for acoustic analysis in order to tayl... more A training experience that takes advantage of technologies for acoustic analysis in order to taylor the learning experience on learners abilities and learning style.
The present study is intended to compare two approaches of labeling expressive corpora: auto-anno... more The present study is intended to compare two approaches of labeling expressive corpora: auto-annotation and annotation by external lay listeners. These two methods have been applied to the semi-spontaneous emotional speech produced by Chinese learners of L2 Italian, involved in the CardTask, a moodinduction procedure that allows us to control the context of interaction, preserving the spontaneity of reactions. The emotional responses to the stimuli presented in the task were the object of an auto-annotation session. The same samples were then administered only in the auditory mode to 20 Italian and 20 Chinese lay listeners. The results of perceptual tests have underlined some similarities and differences between both autoand external annotation, and between the ratings given by external Italian and Chinese listeners. The labels chosen by native Italians were similar to those selected in the auto-annotation session, particularly in the case of anxiety, fear and disgust. The correspon...
The study of vocal emotions poses different methodological problems, especially regarding the sty... more The study of vocal emotions poses different methodological problems, especially regarding the style of speech analyzed. If spontaneous speech on the one hand guarantees data authenticity, on the other hand does not allow to control the variables affecting spoken communication. Recited speech allows to control the verbal content of emotional productions and obtain easily comparable data. Dubbed speech represents another feasible alternative. The task of a dubber is to use a different language to vehiculate the same emotional state expressed in the original version, to sound as natural as possible. The present study has twofold objective: firstly it aims to describe the prosodic characteristics of dubbed speech, to evaluate its effectiveness for the cross-linguistic analysis of vocal emotions; secondly it is intended to verify if romance and non-romance languages present similar prosodic patterns in the expression of different emotional states. To reach these goals, sequences taken fr...
Patients with Parkinson's disease (PD) usually reveal speech disorders and, among other sympt... more Patients with Parkinson's disease (PD) usually reveal speech disorders and, among other symptoms, the alteration of speech rhythm. The purpose of this study is twofold: (1) to test the validity of two acoustic parameters—%V, vowel percentage and VtoV, the mean interval between two consecutive vowel onset points—for the identification of rhythm variation in early-stage PD speech and (2) to analyze the effect of PD on speech rhythm in two different speaking tasks: reading passage and monolog. A group of 20 patients with early-stage PD was involved in this study and compared with 20 age- and sex-matched healthy controls (HCs). The results of the acoustic analysis confirmed that %V is a useful cue for early-stage PD speech characterization, having significantly higher values in the production of patients with PD than the values in HC speech. A simple speaking task, such as the reading task, was found to be more effective than spontaneous speech in the detection of rhythmic variations.
Parkinson’s Disease (PD) is a neurological illness which also has effects on speech production, r... more Parkinson’s Disease (PD) is a neurological illness which also has effects on speech production, resulting in segmental and suprasegmental abnormalities. The aim of the current study is to test the validity of two acoustic parameters %V and VtoV for the detection of rhythmical variation in early-stage PD speech, in comparison to healthy speech. 40 Italian native speakers were enrolled in the research, 20 early-stage PD subjects and 20 neurologically healthy and matched controls, and a corpus of read speech was collected. The results of voice analysis confirmed an alteration of vocalic duration and %V in PD productions. In particular, %V could be a reliable cue for PD speech characterization, even at the very early onset of the disease.
The present study is intended to figure out the extent to which prosody and intonation affect lis... more The present study is intended to figure out the extent to which prosody and intonation affect listeners’ ability to estimate the speaker’s age. The performance of a 40-year old anchorman and another by the same speaker at the age of 80 were spectroacoustically analyzed in order to identify the prosodic features of a “young” and an “old” voice. The results of the analysis have shown significant differences between the two voices on a suprasegmental level. To test the effects of these differences on a perceptual level, through the prosodic transplantation technique, the F0 values and the durations of segments and silences were transferred from the “young” to the “old” voice and viceversa. Two age recognition tests, based on original and transplanted voices, were administered to Italian listeners. The results of perceptual tests have confirmed the strict relationship between some rhythmic and prosodic features and the speaker’s age and have demonstrated the effectiveness of the transpl...
Parkinson’s Disease (PD) is a neurological illness which also has effects on speech production, r... more Parkinson’s Disease (PD) is a neurological illness which also has effects on speech production, resulting in segmental and suprasegmental abnormalities. The aim of the current study is to test the validity of two acoustic parameters - %V and VtoV - for the detection of rhythmical variation in early-stage PD speech, in comparison to healthy speech. 40 Italian native speakers were enrolled in the research, 20 early-stage PD subjects and 20 neurologically healthy and matched controls, and a corpus of read speech was collected. The results of voice analysis confirmed an alteration of vocalic duration and %V in PD productions. In particular, %V could be a reliable cue for PD speech characterization, even at the very early onset of the disease.
The influence of L1 literacy on the acquisition of L2 oracy has recently become one of the intere... more The influence of L1 literacy on the acquisition of L2 oracy has recently become one of the interests of SLA research. However, what is still missing is an exploration of the development of L2 prosodic competence by low-literate adults learners. To fill this gap, in the present study Senegalese learners of L2 Italian were involved. In Senegal, where literacy acquisition is usually absent in the L1 (mostly Wolof), it is possible to acquire it in French or Arabic, albeit the educational models of the French and Koranic schools are quite different. In French schools writing and reading practices are favored, whereas in Koranic schools speaking and listening abilities are mostly practiced by reciting the Koran. In order to understand the relationship between L1 literacy and L2 oracy in the case of low-literate Senegalese learners of L2 Italian and to verify if the different educational models proposed in Senegal influence L2 prosodic competence even of low-educated learners, perceptual a...
The present study is intended to figure out the extent to which prosody and intonation affect lis... more The present study is intended to figure out the extent to which prosody and intonation affect listeners’ ability to estimate the speaker’s age. The performance of a 40-year old anchorman and another by the same speaker at the age of 80 were spectroacoustically analyzed in order to identify the prosodic features of a “young” and an “old” voice. The results of the analysis have shown significant differences between the two voices on a suprasegmental level. To test the effects of these differences on a perceptual level, through the prosodic transplantation technique, the F0 values and the durations of segments and silences were transferred from the “young” to the “old” voice and viceversa. Two age recognition tests, based on original and transplanted voices, were administered to Italian listeners. The results of perceptual tests have confirmed the strict relationship between some rhythmic and prosodic features and the speaker’s age and have demonstrated the effectiveness of the transplantation technique. With advancing age, articulation rate and speech rate slow down, voice register rises and tonal range widens. Moreover, the “old” voice is also characterized by a higher percentage of vocalic portion which determines a shift of the Italian rhythm towards the isomoraic pattern.
Questo libro è rilasciato con licenza Creative Commons " Attribuzione -Non Commerciale -Non Opere... more Questo libro è rilasciato con licenza Creative Commons " Attribuzione -Non Commerciale -Non Opere Derivate 2.0" consultabile in rete all'indirizzo http://creativecommons. org. Pertanto questo libro è libero e può essere riprodotto e distribuito con ogni mezzo fisico, meccanico o elettronico, a condizione che la riproduzione del testo avvenga integralmente e senza modifiche, ad uso privato e a fini non commerciali.
Questo libro è rilasciato con licenza Creative Commons "Attribuzione -Non Commerciale -Non Opere ... more Questo libro è rilasciato con licenza Creative Commons "Attribuzione -Non Commerciale -Non Opere Derivate 2.0", consultabile in rete all'indirizzo http://creativecommons.org. Pertanto questo libro è libero e può essere riprodotto e distribuito con ogni mezzo fisico, meccanico o elettronico, a condizione che la riproduzione del testo avvenga integralmente e senza modifiche, ad uso privato e a fini non commerciali.
Uploads
Papers by Marta Maffia