In the current study we use electroencephalography (EEG) to detect heard music from the brain sig... more In the current study we use electroencephalography (EEG) to detect heard music from the brain signal, hypothesizing that the time structure in music makes it especially suitable for decoding perception from EEG signals. While excluding music with vocals, we classified the perception of seven different musical fragments of about three seconds, both individually and cross-participants, using only time domain information (the event-related potential, ERP). The best individual results are 70% correct in a seven-class problem while using single trials, and when using multiple trials we achieve 100% correct after six presentations of the stimulus. When classifying across participants, a maximum rate of 53% was reached, supporting a general representation of each musical fragment over participants. While for some music stimuli the amplitude envelope correlated well with the ERP, this was not true for all stimuli. Aspects of the stimulus that may contribute to the differences between the EEG responses to the pieces of music are discussed.
An experiment was conducted to determine whether decoding auditory evoked potentials during passi... more An experiment was conducted to determine whether decoding auditory evoked potentials during passive listening and providing the classifier output as a neurofeedback signal leads to the enhancement of auditory perceptual discrimination and/or brain responses related to auditory perception. Results indicate an enhancement of both behavioral discrimination and brain responses to frequency stimuli across four days of measurements.
In this paper, we show how acoustic variance within lexical tones in disyllabic Mandarin Chinese ... more In this paper, we show how acoustic variance within lexical tones in disyllabic Mandarin Chinese pseudowords affects discrimination abilities in both native and non-native speakers of Mandarin Chinese. Within-category acoustic variance did not hinder native speakers in discriminating between lexical tones, whereas it precludes Dutch native speakers from reaching native level performance. Furthermore, the influence of acoustic variance was not uniform but asymmetric, dependent on the presentation order of the lexical tones to be discriminated. An exploratory analysis using an active adaptive oddball paradigm was used to quantify the extent of the perceptual asymmetry. We discuss two possible mechanisms underlying this asymmetry and propose possible paradigms to investigate these mechanisms.
Evidence is presented that expressive timing in music is not relationally invariant with global t... more Evidence is presented that expressive timing in music is not relationally invariant with global tempo. Our results stem from an analysis of repeated performances of Beethoven's variations on a Paisiello theme. Recordings were made of two pianists playing the pieces at three tempi. In contrast with the relational-invariance hypothesis (see Repp, 1994), between-tempo correlations were in general lower than within-tempo correlations. Analyses of variance of log-transformed inter-onset intervals (IOIs) showed significant interactions between tempo and IOI, i.e., evidence against a proportional relation between timing and tempo. Complex, but consistent, nonproportional patterns were shown in the analyses of the timing of the grace notes in the piece. The analysis suggests that timing aspects of music are closely linked to the musical structure and can be studied and manipulated only respecting this relation-not as a global timing pattern or tempo curve. Finally, it is shown that methodological issues of data collection and analysis had a significant influence on the results. variation, or even simply changing the key from major to minor without modifying the tempo curve-produced results judged unmusical, and we argued against their use on this basis. Instead, we suggested that the timing aspects of music are linked to its structural aspects and should therefore be studied and manipulated respecting this relation and as a global transformation based on a tempo curve. Repp (1994), prompted by our work, explored the scaling of expressive aspects of music systematically with regard to global tempo, for performances of Schumann's "Tr~iumerei." In his work, which will be referred to as the "Trgumerei" study, there was relational invariance: expressive timing scaled "approximately" proportionally with tempo. Consequently, a tempo curve for a different tempo can be obtained if the original is plotted on a log-duration y axis and shifted vertically. To gain further understanding on this issue, we present a new analysis. We follow the lines as set out in the "Trfiumerei" study to allow easy comparison. However, we shall restrict ourselves to the examination of the patterns of onset timing.
Two experiments on categorical rhythm perception are reported, the object of which was to investi... more Two experiments on categorical rhythm perception are reported, the object of which was to investigate how listeners perceive discrete rhythmic categories while listening to rhythms performed on a continuous time scale. This is studied by considering the space of all temporal patterns (all possible rhythms made up of three intervals) and how they, in perception, are partitioned into categories, ie where the boundaries of these categories are located. This process of categorisation is formalised as the mapping from the continuous space of a series of time intervals to a discrete, symbolic domain of integer-ratio sequences. The methodological framework uses concepts from mathematics and psychology (eg convexity and entropy) that allow precise characterisations of the empirical results. In the first experiment, twenty-nine participants performed an identification task with 66 rhythmic stimuli (a systematic sampling of the performance space). The results show that listeners do not just p...
The central purpose of this paper is to elaborate on the methods for computational modeling and t... more The central purpose of this paper is to elaborate on the methods for computational modeling and to show how, although computational modeling should in principle be instrumental in the understanding of the structure of musical knowledge and the processes involved in music cognition, in practice it too often degenerates into a loose "if you want to understand the theory, here, look in my program" approach. We will show how many issues cannot be decided by inspecting computer programs as they are written nowadays, and we will indicate a possible solution, giving examples from the field of music cognition research.
In the literature of musicology, computer music research and the psychology of music, timing or t... more In the literature of musicology, computer music research and the psychology of music, timing or tempo measurements are mostly presented in the form of continuous curves. The notion of these tempo curves is dangerous, despite its widespread use, because it lulls its u s e r s i n t o t h e false impression that a continuous concept of temporal flow has an independent existence, a musical or psychological reality, and that time can be perceived independent of events carrying it. But if one bases a transformation or manipulation of timing on the implied characteristics of such a notion, one is doomed to fail. KEY WORDS representation of time, tempo curves, expressive timing In which we decided to have a good time, invited an expert, and had our first disappointment. Not so long ago we decided to spend a Christmas holiday studying music and its performance. One of us is an a m a t e u r m a t h e m a t i c i a n (M) and the o t h e r one likes to delve into old psychology textbooks (P), and because we enjoy impressing each o t h e r with n e w facts and insights, we o f t e n find ourselves in v e h e m e n t discussions. T h e r e f o r e we t h o u g h t we might have a pleasant and peaceful time by putting our beloved hobby horses aside and e m b a r k upon a subject about which neither of us k n e w much: the timing aspects of music. We became interested in this field because we had noticed, while playing with the c o m p u t e r , our favourite toy, that adding just a bit of r a n d o m timing noise to a p r o g r a m that played a score in an o t h e r w i s e metronomically perfect way, made the music much more pleasant to listen to. It seemed as if we could make m o r e sense of it. But we suspected that there was m o r e to timing and expressive p e r f o r m a n c e t h a n adding bits of noise, so we invited a mutual friend w h o is a retired professional pianist to spend Christmas in o u r small but well equipped laboratory. O u r friend has a great love for the piano and its music, but is completely i g n o r a n t of the advances of m o d e r n technology. To d e m o n s t r a t e to him o u r latest s e q u e n c e r p r o g r a m we asked him to play the t h e m e f r o m the six variations composed by Ludwig van Beethoven on the duet Nel cor pih non mi sento, the score of which we had lying a r o u n d (see Figure 1). Even t h o u g h he was s o m e w h a t disturbed by the touch and harpsichord-like sound of the electronic piano, he was quite fascinated with the possibility of recording and playing back on the same i n s t r u m e n t. Enthusiastically we told him that this system was more than just a m o d e r n version of the pianola: 'You can examine and change e v e r y detail you want; for instance, inspect the timing, accurately to the millisecond, add and r e m o v e notes, make notes longer or 123 See http://www.hum.uva.nl/mmm/papers/dh-93-f.html for sound examples.
Automatic Music Transcription is the extraction of an acceptable notation from performed music. O... more Automatic Music Transcription is the extraction of an acceptable notation from performed music. One important task in this problem is rhythm quantization which refers to categorization of note durations. Although quantization of a pure mechanical performance is rather straightforward, the task becomes increasingly difficult in presence of musical expression, i.e. systematic variations in timing of notes and in tempo. For transcription of natural performances, we employ a framework based on Bayesian statistics. Expressive deviations are modelled by a probabilistic performance model from which the corresponding optimal quantizer is derived by Bayes theorem. We demonstrate that many different quantization schemata can be derived in this framework by proposing suitable prior and likelihood distributions. The derived quantizer operates on short groups of onsets and is thus flexible both in capturing the structure of timing deviations and in controlling the complexity of resulting notations. The model is trained on data resulting from a psychoacoustical experiment and thus can mimic the behaviour of a human transcriber on this task.
In the current study we use electroencephalography (EEG) to detect heard music from the brain sig... more In the current study we use electroencephalography (EEG) to detect heard music from the brain signal, hypothesizing that the time structure in music makes it especially suitable for decoding perception from EEG signals. While excluding music with vocals, we classified the perception of seven different musical fragments of about three seconds, both individually and cross-participants, using only time domain information (the event-related potential, ERP). The best individual results are 70% correct in a seven-class problem while using single trials, and when using multiple trials we achieve 100% correct after six presentations of the stimulus. When classifying across participants, a maximum rate of 53% was reached, supporting a general representation of each musical fragment over participants. While for some music stimuli the amplitude envelope correlated well with the ERP, this was not true for all stimuli. Aspects of the stimulus that may contribute to the differences between the EEG responses to the pieces of music are discussed.
An experiment was conducted to determine whether decoding auditory evoked potentials during passi... more An experiment was conducted to determine whether decoding auditory evoked potentials during passive listening and providing the classifier output as a neurofeedback signal leads to the enhancement of auditory perceptual discrimination and/or brain responses related to auditory perception. Results indicate an enhancement of both behavioral discrimination and brain responses to frequency stimuli across four days of measurements.
In this paper, we show how acoustic variance within lexical tones in disyllabic Mandarin Chinese ... more In this paper, we show how acoustic variance within lexical tones in disyllabic Mandarin Chinese pseudowords affects discrimination abilities in both native and non-native speakers of Mandarin Chinese. Within-category acoustic variance did not hinder native speakers in discriminating between lexical tones, whereas it precludes Dutch native speakers from reaching native level performance. Furthermore, the influence of acoustic variance was not uniform but asymmetric, dependent on the presentation order of the lexical tones to be discriminated. An exploratory analysis using an active adaptive oddball paradigm was used to quantify the extent of the perceptual asymmetry. We discuss two possible mechanisms underlying this asymmetry and propose possible paradigms to investigate these mechanisms.
Evidence is presented that expressive timing in music is not relationally invariant with global t... more Evidence is presented that expressive timing in music is not relationally invariant with global tempo. Our results stem from an analysis of repeated performances of Beethoven's variations on a Paisiello theme. Recordings were made of two pianists playing the pieces at three tempi. In contrast with the relational-invariance hypothesis (see Repp, 1994), between-tempo correlations were in general lower than within-tempo correlations. Analyses of variance of log-transformed inter-onset intervals (IOIs) showed significant interactions between tempo and IOI, i.e., evidence against a proportional relation between timing and tempo. Complex, but consistent, nonproportional patterns were shown in the analyses of the timing of the grace notes in the piece. The analysis suggests that timing aspects of music are closely linked to the musical structure and can be studied and manipulated only respecting this relation-not as a global timing pattern or tempo curve. Finally, it is shown that methodological issues of data collection and analysis had a significant influence on the results. variation, or even simply changing the key from major to minor without modifying the tempo curve-produced results judged unmusical, and we argued against their use on this basis. Instead, we suggested that the timing aspects of music are linked to its structural aspects and should therefore be studied and manipulated respecting this relation and as a global transformation based on a tempo curve. Repp (1994), prompted by our work, explored the scaling of expressive aspects of music systematically with regard to global tempo, for performances of Schumann's "Tr~iumerei." In his work, which will be referred to as the "Trgumerei" study, there was relational invariance: expressive timing scaled "approximately" proportionally with tempo. Consequently, a tempo curve for a different tempo can be obtained if the original is plotted on a log-duration y axis and shifted vertically. To gain further understanding on this issue, we present a new analysis. We follow the lines as set out in the "Trfiumerei" study to allow easy comparison. However, we shall restrict ourselves to the examination of the patterns of onset timing.
Two experiments on categorical rhythm perception are reported, the object of which was to investi... more Two experiments on categorical rhythm perception are reported, the object of which was to investigate how listeners perceive discrete rhythmic categories while listening to rhythms performed on a continuous time scale. This is studied by considering the space of all temporal patterns (all possible rhythms made up of three intervals) and how they, in perception, are partitioned into categories, ie where the boundaries of these categories are located. This process of categorisation is formalised as the mapping from the continuous space of a series of time intervals to a discrete, symbolic domain of integer-ratio sequences. The methodological framework uses concepts from mathematics and psychology (eg convexity and entropy) that allow precise characterisations of the empirical results. In the first experiment, twenty-nine participants performed an identification task with 66 rhythmic stimuli (a systematic sampling of the performance space). The results show that listeners do not just p...
The central purpose of this paper is to elaborate on the methods for computational modeling and t... more The central purpose of this paper is to elaborate on the methods for computational modeling and to show how, although computational modeling should in principle be instrumental in the understanding of the structure of musical knowledge and the processes involved in music cognition, in practice it too often degenerates into a loose "if you want to understand the theory, here, look in my program" approach. We will show how many issues cannot be decided by inspecting computer programs as they are written nowadays, and we will indicate a possible solution, giving examples from the field of music cognition research.
In the literature of musicology, computer music research and the psychology of music, timing or t... more In the literature of musicology, computer music research and the psychology of music, timing or tempo measurements are mostly presented in the form of continuous curves. The notion of these tempo curves is dangerous, despite its widespread use, because it lulls its u s e r s i n t o t h e false impression that a continuous concept of temporal flow has an independent existence, a musical or psychological reality, and that time can be perceived independent of events carrying it. But if one bases a transformation or manipulation of timing on the implied characteristics of such a notion, one is doomed to fail. KEY WORDS representation of time, tempo curves, expressive timing In which we decided to have a good time, invited an expert, and had our first disappointment. Not so long ago we decided to spend a Christmas holiday studying music and its performance. One of us is an a m a t e u r m a t h e m a t i c i a n (M) and the o t h e r one likes to delve into old psychology textbooks (P), and because we enjoy impressing each o t h e r with n e w facts and insights, we o f t e n find ourselves in v e h e m e n t discussions. T h e r e f o r e we t h o u g h t we might have a pleasant and peaceful time by putting our beloved hobby horses aside and e m b a r k upon a subject about which neither of us k n e w much: the timing aspects of music. We became interested in this field because we had noticed, while playing with the c o m p u t e r , our favourite toy, that adding just a bit of r a n d o m timing noise to a p r o g r a m that played a score in an o t h e r w i s e metronomically perfect way, made the music much more pleasant to listen to. It seemed as if we could make m o r e sense of it. But we suspected that there was m o r e to timing and expressive p e r f o r m a n c e t h a n adding bits of noise, so we invited a mutual friend w h o is a retired professional pianist to spend Christmas in o u r small but well equipped laboratory. O u r friend has a great love for the piano and its music, but is completely i g n o r a n t of the advances of m o d e r n technology. To d e m o n s t r a t e to him o u r latest s e q u e n c e r p r o g r a m we asked him to play the t h e m e f r o m the six variations composed by Ludwig van Beethoven on the duet Nel cor pih non mi sento, the score of which we had lying a r o u n d (see Figure 1). Even t h o u g h he was s o m e w h a t disturbed by the touch and harpsichord-like sound of the electronic piano, he was quite fascinated with the possibility of recording and playing back on the same i n s t r u m e n t. Enthusiastically we told him that this system was more than just a m o d e r n version of the pianola: 'You can examine and change e v e r y detail you want; for instance, inspect the timing, accurately to the millisecond, add and r e m o v e notes, make notes longer or 123 See http://www.hum.uva.nl/mmm/papers/dh-93-f.html for sound examples.
Automatic Music Transcription is the extraction of an acceptable notation from performed music. O... more Automatic Music Transcription is the extraction of an acceptable notation from performed music. One important task in this problem is rhythm quantization which refers to categorization of note durations. Although quantization of a pure mechanical performance is rather straightforward, the task becomes increasingly difficult in presence of musical expression, i.e. systematic variations in timing of notes and in tempo. For transcription of natural performances, we employ a framework based on Bayesian statistics. Expressive deviations are modelled by a probabilistic performance model from which the corresponding optimal quantizer is derived by Bayes theorem. We demonstrate that many different quantization schemata can be derived in this framework by proposing suitable prior and likelihood distributions. The derived quantizer operates on short groups of onsets and is thus flexible both in capturing the structure of timing deviations and in controlling the complexity of resulting notations. The model is trained on data resulting from a psychoacoustical experiment and thus can mimic the behaviour of a human transcriber on this task.
This primer, designed for those interested in the perception of rhythm, provides readers with bot... more This primer, designed for those interested in the perception of rhythm, provides readers with both an overview of recent research in the introductions to each section, and a broad selection of chapters dealing with more detailed studies. It consists of contributions by some of the most respected investigators in the fields of motor behaviour, timing control, music cognition and psychology and arose out of the 7th Workshop on Rhythm Perception and Production, held in Wassenaar in 1998.;The book begins with a discussion of more generic studies of timing and synchronisation, focusing on the two main methods of modelling timing behaviour (the modern complex dynamics approach and the older methods of analysis of covariance) and some progress is made towards an integration of these two historically conflicting approaches. It then moves on to consider rhythm perception and production in a wide variety of contexts, with a prticular focus on music and language. The individual contributions range from attempts to model the processes involved in tracking or synchronising to an external pulse to detailed studies of the ways in which the rhythmic complexities of real musical and linguistic behaviour (such as polyrhythmic drumming) are executed and perceived.
Uploads
Papers by Peter Desain