SCORE-INFORMED ANALYSIS OF INTONATION AND PITCH

MODULATION IN JAZZ SOLOS

Jakob Abeer1,2 Estefana Cano2 Klaus Frieler1

Martin Pfleiderer 1
Wolf-Georg Zaddach1
1
Jazzomat Research Project, University of Music Franz Liszt, Weimar, Germany
2
Semantic Music Technologies Group, Fraunhofer IDMT, Ilmenau, Germany
[email protected]

ABSTRACT representation does not reveal information about timbre,

intonation, and pitch modulation. Therefore, audio-level
The paper presents new approaches for analyzing the char- analysis of recorded improvisations is necessary to charac-
acteristics of intonation and pitch modulation of woodwind terize those non-syntactical, expressive dimensions in or-
and brass solos in jazz recordings. To this end, we use der to get a comprehensive and exhaustive description of a
score-informed analysis techniques for source separation personal style.
and fundamental frequency tracking. After splitting the
audio into a solo and a backing track, a reference tuning
frequency is estimated from the backing track. Next, we 2. GOALS
compute the fundamental frequency contour for each tone Polyphonic music recordings exhibit strong spectral and
in the solo and a set of features describing its temporal temporal overlaps between harmonic components of dif-
shape. Based on this data, we first investigate, whether ferent instrument sources. Hence, the transcription and
the tuning frequencies of jazz recordings changed over the analysis of the individual sound sources remain one of the
decades of the last century. Second, we analyze whether most challenging tasks in Music Information Retrieval
the intonation is artist-specific. Finally, we examine how (MIR). We approach this task by using high-quality melody
the modulation frequency of vibrato tones depends on con- transcriptions provided by music experts as foundation for
textual parameters such as pitch, duration, and tempo as a score-informed audio analysis. In particular, we use score
well as the performing artist. information for the source separation of the solo instru-
ment from the audio mixture and for the frame-wise track-
1. INTRODUCTION ing of the fundamental frequency of each tone. Our main
goal is to investigate, which intonation and modulation stra-
The personal styles of improvising jazz musicians can be
tegies are applied by woodwind and brass instrument play-
described from various musical perspectives. There are
ers in jazz solos.
several structural or syntactical aspects of the improvised
melodic lines, which could be idiosyncratic for a certain
musician, e.g., preferred pitches, intervals, scales, melodic 3. RELATED WORK
contours, rhythms or typical patterns, licks, and formu-
Various MIR publications investigate the intonation and
las. These dimensions can be best explored using a sym-
tuning of music recordings, ranging from historic solo harp-
bolic representation, e.g., Western staff notation or MIDI.
sichord recordings [5], over classical music recordings [11],
However, there are other important aspects, which define
to Non-Western music styles such as Carnatic and Hindus-
personal style and make it recognizable: timbre (sound
tani music [17]. The tuning frequency of audio record-
characteristics such as roughness or breathiness), micro-
ings is commonly estimated based on pitch frequencies [6],
timing (systematic deviations from the underlying metric
high-resolution interval histograms [17], or adjustable fil-
structure), dynamics (the changes in intensity of tones or
terbanks [11]. Just intonation and equal temperament are
phrases), intonation (the pitch accuracy with respect to a
generally used as reference tunings for the analysis of mu-
given tone system), articulation (e.g., legato or staccato
sic performances. Lerch [11] points out that observed tun-
playing) and pitch modulation (the variation of the funda-
ing deviations can have different reasons ranging from de-
mental frequency within the duration of a tone). Symbolic
viation of harmonic frequencies from the equal tempered
scale to deviations due to non-equal temperament.
c Jakob Abeer, Estefana Cano, Klaus Frieler, Martin Most automatic music transcription algorithm aim at
Pfleiderer, Wolf-Georg Zaddach. Licensed under a Creative Commons a symbolic representation of tone events, which are de-
Attribution 4.0 International License (CC BY 4.0). Attribution: Jakob
Abeer, Estefana Cano, Klaus Frieler, Martin Pfleiderer, Wolf-Georg
scribed by distinct onset times, durations, and constant
Zaddach. Score-informed Analysis of Intonation and Pitch Modulation pitches [15]. Some automatic melody extraction algorithms
in Jazz Solos, 16th International Society for Music Information Retrieval such as proposed in [16] and [6] include an estimation of
Conference, 2015. the tone-wise contours of the fundamental frequency (f0 )

823
824 Proceedings of the 16th ISMIR Conference, Malaga, Spain, October 26-30, 2015

as well, which is an essential pre-processing step for ana- all musicians in the dataset with their instrument, the num-
lyzing the applied frequency modulation techniques. There ber of solos NS , and the total number of tones and f0 con-
are many studies on vibrato detection in audio recordings tours NN , respectively. The solos were manually anno-
[14], particularly for singing voice [8,9,12]. Other publica- tated by musicology and jazz students based on excerpts
tions deal with analyzing the deviation of f0 contours from from commercial audio recordings. The annotations in-
the target pitch [8] as well as with segmenting f0 contours clude score-level melody transcription (MIDI pitch, tone
based on modulations such as vibrato and pitch glides [12] onset, and duration) as well as additional annotation layers
or bendings [10]. To the best knowledge of the authors, with respect to melody phrases, metric structure, chords,
no publication so far analyzes intonation and modulation and modulation techniques. So far, the tone-wise anno-
techniques in recorded jazz solos. tations of modulation techniques are incomplete and only
represent the most clear examples within the solos. Figure
4. METHOD 2 gives an overview over the number of annotated tones per
artist. In total, 87643 tones and f0 contours are included in
Figure 1 gives an overview over our analysis approach, the dataset.
all processing steps are detailed in the following sections.
Section 4.1 describes the dataset of jazz solo audio excerpts Woody Shaw
Wayne Shorter Falloff
Steve Lacy Slide
and transcriptions. Two separate score-informed analysis Steve Coleman
Stan Getz
Sonny Stitt
Vibrato

techniques are involved. At first, a source separation algo- Sonny Rollins

Sidney Bechet
Roy Eldridge
Paul Desmond
rithm is performed (see Section 4.2), which separates the Ornette Coleman
Miles Davis
Michael Brecker
original audio recording into a solo track containing the Louis Armstrong
Lester Young
Lee Konitz

improvising solo instrument and a backing track contain-

Kenny Dorham
Joshua Redman
John Coltrane
Joe Lovano
ing the accompanying band, i.e., the rhythm section (most Joe Henderson
J.J. Johnson
Freddie Hubbard

often piano, double bass, and drums). The backing track is

Fats Navarro
Eric Dolphy
Don Byas
Dizzy Gillespie
used to estimate the reference tuning frequency (see Sec- Dexter Gordon
David Murray
David Liebman
tion 4.4). The second step is the tracking of frame-wise f0 Coleman Hawkins
Clifford Brown
Chet Baker

contours for each note played by the solo instrument (see

Charlie Parker
Cannonball Adderley
Bob Berg
Bix Beiderbecke
Section 4.3). Based on the extracted f0 contours, we com- Benny Goodman
Benny Carter
Ben Webster

pute several contour features (see Section 4.5) to describe

Art Pepper
0 100 200 300

their temporal shape. In the experiments reported in Sec-

Notes

tion 5, we analyze how these features depend on contextual Figure 2: Number of tones of each artist which are anno-
parameters such as tone duration and pitch and whether tated with fall-off, slide, and vibrato.
these might be specific for the personal style.

4.2 Score-informed Source Separation

To separate the solo/lead instrument from the backing track,
the method for pitch-informed solo and accompaniment
separation proposed in [4] was used. For this study, the
automatic pitch detection stage in the separation algorithm
was bypassed, and the manual melody transcriptions were
used as prior information. The separation method is based
on an iterative modeling of the solo instrument in the spec-
tral domain. The model is constructed taking into account
characteristics of musical instruments such as common am-
plitude modulation, inharmonicity, and enforcing magni-
tude and frequency smoothness constraints in the estima-
tion. The separation method has proven to be robust in
the extraction of a great variety of solo instruments, as
well as being particularly efficient, with computation times
that allow real-time processing. The complete dataset was
Figure 1: Proposed algorithm for score-informed analysis processed and independent signals for the solo instruments
of tuning and modulation in improvised jazz solos. and the backing tracks were extracted.

4.3 Score-informed f0 tracking

4.1 Dataset & Melody Annotations
The original audio recordings are processed at a sampling
The dataset used in this publication is a subset of 207 jazz rate of 22.05 kHz. In order to track the f0 contour of each
solos taken from the Weimar Jazz Database 1 . Table 1 lists tone, the signal is analyzed between the annotated note on-
1 http://jazzomat.hfm-weimar.de (last accessed Juli 10, set and offset time, for which a reassigned magnitude spec-
2015) trogram M 2 RKN + with K frequency bins and N frames
Proceedings of the 16th ISMIR Conference, Malaga, Spain, October 26-30, 2015 825

Performer Inst. NS NN Performer Inst. NS NN Performer Inst. NS NN Performer Inst. NS NN

Art Pepper cl/as 4 2134 David Liebman ss/ts 4 3286 John Coltrane ts/ss 11 8969 Sidney Bechet ss 2 489
Ben Webster ts 4 1497 David Murray ts 4 2295 Joshua Redman ts 5 2429 Sonny Rollins ts 10 4639
Benny Carter as 3 1153 Dexter Gordon ts 4 3056 Kenny Dorham tp 6 2149 Sonny Stitt ts 4 1284
Benny Goodman cl 7 1154 Dizzy Gillespie tp 4 967 Lee Konitz as 4 1839 Stan Getz ts 6 3253
Bix Beiderbecke tp 4 519 Don Byas ts 7 2022 Lester Young ts 4 887 Steve Coleman as 3 1353
Bob Berg ts 5 3275 Eric Dolphy as 2 1109 Louis Arm- tp 4 634 Steve Lacy ss 4 1437
strong
Cannonball as 5 2623 Fats Navarro tp 4 937 Michael ts 4 2605 Wayne Shorter ts 9 3013
Adderley Brecker
Charlie Parker as 6 1688 Freddie Hub- tp 6 2266 Miles Davis tp 7 2080 Woody Shaw tp 5 1822
bard
Chet Baker tp 6 1100 J.J. Johnson tb 2 754 Ornette Cole- as 3 1782
man
Clifford Brown tp 4 1676 Joe Henderson ts 6 3830 Paul Desmond as 8 2142
Coleman ts 6 2613 Joe Lovano ts/ts- 2 1787 Roy Eldridge tp 6 1744
Hawkins c

Table 1: Overview over all artists in the dataset. For each artist, the number of solos NS , the total number of notes NN , as
well as the instrument is given (ts: tenor saxophone, ss: soprano saxophone, as: alto saxophone, cl: clarinet, tp: trumpet,
cor: cornet, tb: trombone, ts-c: C melody tenor saxophone).

is computed. We use a logarithmic frequency axis with a ing frequency hypotheses, a final tuning frequency esti-
high resolution of 50 bins/semitone and a frequency range mate fref is derived. We modified the originally proposed
of 2 semitones around the annotated pitch. Based on an search range for fref to 440 Hz0.5 semitone (correspond-
initial short-time Fourier transform (STFT) with a block- ing MIDI pitch range: 69 0.5) and the stepsize to 0.1
size of 1024, a hopsize of 64, and a zero-padding factor of cents. As will be shown in Section 5.1, the influence of
16, the magnitude values are mapped (reassigned) towards source separation artifacts on the estimation accuracy of
the frequency bins that correspond to their instantaneous the reference tuning frequency can be neglected.
frequency values at the original frequency bins computed
using the method proposed by Abe in [1]. Two steps are 4.5 Feature Extraction
performed for each tone to estimate its f0 contour. First,
Based on the estimated contour f0 (n) of each tone, we first
we estimate a suitable starting frame within the tones du-
perform a smoothing using a two-element moving aver-
ration with a prominent peak close to the annotated pitch.
age filter in order to compensate for local irregularities and
Second, we perform a contour tracking both forwards and
possible estimation errors. The extracted audio features
backwards in time. Further details are provided in [3].
describe the gradient of the f0 contour as well as its tem-
poral modulation. Table 2 lists all computed audio features
4.4 Tuning Frequency Estimation and their dimensionality.
The oldest recordings in our dataset date back to the year
1924, two years before the American music industry rec- Category Feature Label Dim.

ommended 440 Hz for A4 as standard tuning, and 12 years Gradient Linear slope 1
Gradient Median gradients (first half, second half, over- 3
before the American Standards Association officially all)
adopted it. Hence, we can not rely on the assumption of Gradient Ratio of ascending frames 1
Gradient Ratio of ascending / descending / constant 3
a constant and fixed overall tuning. Moreover, the techni- segments
cal level of recording studios were rather low at this time, Gradient Median gradient of longest segments 1
Gradient Relative duration of longest segments 1
which might result in tuning deviations by speed fluctu- Gradient Pitch progression 1
ations of recording machines as well as from instruments Modulation Modulation frequency [Hz] 1
Modulation Modulation dominance 1
tuned to another reference pitch such as studio or live venue Modulation Modulation range [cent] 1
Modulation Number of modulation periods 1
pianos. Hence, we estimate a reference tuning frequency Modulation Average relative / absolute f0 deviation 2
fref prior to the intonation analysis of the solo instrument Modulation f0 deviation inter-quartile-range 1
from the backing track of the rhythm section, which we ob-
tain from the source separation process explained in Sec-
tion 4.2. The reference tuning frequency corresponds to Table 2: Summary of audio features to descript the f0 con-
the fundamental frequency of the pitch A4 in the backing tours.
track.
In the Chroma Toolbox [13], a triangular filterbank is
4.5.1 Gradient features
generated based on a given tuning frequency in such way
that its center frequencies are aligned to the chromatic scale Based on the gradient f0 (n) = f0 (n+1) f0 (n), we first
within the full piano pitch range. For a given audio signal, determine frames and segments of adjacent frames with as-
the magnitude spectrogram is averaged over the full signal cending ( f0 (n) > 0), descending ( f0 (n) < 0), and
duration and processed using the filterbank. By constant frequency. We use the relative duration (with re-
maximizing the filterbank output energy over different tun- spect to the note duration) of each segment class as fea-
826 Proceedings of the 16th ISMIR Conference, Malaga, Spain, October 26-30, 2015

tures. Also, we compute median gradients in the first and as on the backing track obtained from the source separation
second halves, over the whole note, as well as over the of the solo part (compare Section 4.2) to get two estimates
Backing
longest segment. Overall pitch progression is measured by NoSolo
fref and fref of the reference tuning frequency.
the difference of average f0 values in the end and begin- The results show a very high sample correlation of r =
ning of each tone. Furthermore, we use linear regression 0.97 (p < 0.001) and a small root mean squared error of
to estimate the linear slope of the f0 contour. RMSE = 1.05 Hz between both estimates. These results
indicate that the influence of source separation artifacts is
4.5.2 Modulation features
negligible for the tuning estimation process. Therefore, we
We analyze the modulation of the f0 contour by comput- will use fref = fref
Backing
as an estimate of the reference
ing the autocorrelation over f0 (n). Fletcher [7] reported tuning frequency throughout the paper.
for woodwind instruments that a vibrato frequency range
between 5 and 8 Hz is comfortable for listeners and com- 5.2 Relationship between the Reference Tuning and
mon for players. We add a safety margin of 2 Hz and the Recording Year / Decade
search for the lag position max of the highest local maxi-
mum within the lag range that corresponds to fundamental How did the tuning frequency fref of commercial jazz
frequency values of fmod 2 [3, 10] Hz and estimate the recordings change during the 20th century? Figure 3 shows
modulation frequency as fmod = 1/max . The difference the distribution of solos in the dataset over the from the
between the maximum and median magnitude within this 1920s to the 2000s. Moreover, the inserted boxplots illus-
frequency band is used as dominance measure for the mod- trate the deviation f = 1200 log2 f440ref
between the tuning
ulation. Other applied features are the number of modula- frequency fref and 440 Hz in cent.
tion periods and the frequency modulation range in cent. Absolute tuning deviation | f | and recording year of
each solo are weakly negatively correlated (r = 0.33,
4.6 Analysis of Intonation and Modulation p < 0.001). Hence, the absolute deviation from the tun-
Techniques ing frequency from 440 Hz decreased over the the course
of the 20th century, reflecting the spread of the 440 Hz
We distinguish three modulation techniques fall-off, slide,
standard (1955 adopted by the International Standards Or-
and vibrato. Table 3 provides a description of the charac-
ganization), as well as the progress of studio technology.
teristic f0 contour shape for each technique. The number
of tones in our dataset annotated with each technique is
given in Table 3. 80

60
Technique Description Notes
f [cent] | Number of solos

40
Fall-off Drop of the f0 contour in the end of the tone after a 146
stationary part. 20
Slide Rise or drop of the f0 in the beginning of the tone 708
towards a stationary part. 0
Vibrato Periodic modulation of the f0 contour during the sta- 1380
tionary part of the tone. 20
None No discernible modulation of the f0 contour / No 83587
modulation technique annotated. 40

60
19201930

19301940

19401950

19501960

19601970

19701980

19801990

19902000

Table 3: Frequency modulation techniques considered and

number of annotated notes in the dataset for each tech-
Recording Year Decade
nique.
Figure 3: The box plot shows the reference tuning de-
viation from 440 Hz in cent for different recording year
5. RESULTS decades. The bars show the number of solos in the dataset
for each decade.
5.1 Influence of Source Separation on the Reference
Tuning Estimation
After the application of source separation algorithms, parts 5.3 Dependency of Intonation from Artist and
of the isolated solo instrument often remain audible in the Instrument
backing track due to artifacts or interference. We first in-
vestigated the influence of the source separation step de- The distribution of the absolute deviation of tone-wise fun-
scribed in Section 4.2 on the reference tuning estimation. damental frequency values from the estimated tuning fre-
A subset of 13 solos was randomly selected from the dataset, quency as well as modulation range are shown for all mu-
covering various recording decades and solo instruments. sicians in Figure 4, and for all instruments in Figure 5.
For each solo, we took a 20s segment from the original According to Figure 4, the overall pitch intonation of
recording, where only the rhythm section and no solo in- jazz musicians is astonishingly accurate. Some woodwind
strument is playing. We used the tuning estimation method and brass players tend to play a bit sharp, few a bit flat
described in Section 4.4 on both this 20s segment as well but throughout in a range of less than 25 cent. There are
Proceedings of the 16th ISMIR Conference, Malaga, Spain, October 26-30, 2015 827

few exceptions: Sidney Bechet, a traditional soprano saxo- 5.4 Context-dependency of the Modulation Frequency
phonist, has very high values; however, presumably this is of Vibrato
caused not by a sharp intonation but by the high percentage
Does the modulation frequency of vibrato depend on pitch,
of pitch slides played by him (almost 15 % of the tones, cf.
or duration of the vibrato tones, or on the tempo of the
Figure 2).
piece? For the 1380 tones with vibrato notes (cf. Table
For most players, the range of frequency modulation, 3), we found no significant correlations between modula-
i.e., the size of vibrato, is around 25 cent. There are some tion frequency and pitch (r = 0.02, p = 0.42), duration
bigger modulation ranges from 35 to 50 cent, predomi- (r = 0.02, p = 0.5), nor tempo (r = 0.0, p = 0.83).
nantly used by tenor saxophone players associated with The small effect size of the correlation indicates that de-
swing style (Ben Webster, Coleman Hawkins, Don Byas, spite the high variety of tempo values in the dataset (mean
and Lester Young), but also by postbop tenor saxophonist tempo 154.52 bpm, standard deviation 68.16 bpm), the
Joe Lovano, and, again, by Sidney Bechet, showing the modulation frequency only slightly increases with increas-
largest variance of modulation ranges. Therefore, there are ing tempo.
some slight personal and stylistic peculiarities in the use Furthermore, we investigated, whether and how the mo-
of vibrato size. However, there are no obvious trends of dulation frequency of vibrato is connected to the under-
intonation according to different instruments (cf. Figure lying metrical structure of a solo. We computed the ra-
5), since for each instrument there seem to be players who tio r = Tmod /Tsolo between the modulation tempo and
play a bit sharp as well as players who play a bit low; note the average tempo of the solo. The modulation tempo is
that for trombone and c-melody sax there is only one mu- computed as Tmod = 60fmod . Figure 6 shows the ratio
sician (J.J. Johnson resp. Joe Lovano) in our sample. Like- r against the average tempo of the solo. There is no ev-
wise, there is no evidence for general trends of modulation idence in our data for a strategy to adapt the modulation
ranges with respect to instrument. frequency of vibrato to integer multiples of the tempo of
the piece, e.g., to use a vibrato speed according to simple
subdivision of the beat (e.g. eighth notes or eighth triplets).
100

75
As Figure 6 shows, for medium and fast tempos (100 to
AvF0Dev [cent]

50

25 350 bpm) the vibrato frequency varies only between the

0 beat and the 16th note level. For slower tempos, the vi-
25
brato tempo could be up to six or seven times as fast as the
50
beatbut rarely much faster.
BenWeb

WooSha
WaySho
BenGoo

CanAdd

CheBak

DonBya

PauDes
ColHaw

DavMur

LouArm
DexGor

JoeHen

LeeKon
BenCar

ChaPar

JosRed
KenDor

LesYou
BobBer

FreHub

SonRol
FatNav

JoeLov

RoyEld
OrnCol

SidBec
DavLie

JohCol

MicBre

StaGet

SteLac
ArtPep

MilDav

SteCol
SonSti
J.JJoh
BixBei

CliBro

EriDol
DizGil

12

100 10

8
Tmod / Tsolo

75
ModRange [cent]

6
50
4
25
2

0
0
50 100 150 200 250 300 350
BenWeb

WooSha
WaySho
BenGoo

CanAdd

CheBak

DonBya

PauDes
ColHaw

DavMur

LouArm
DexGor

JoeHen

LeeKon
BenCar

ChaPar

JosRed
KenDor

LesYou
BobBer

FreHub

SonRol

Tsolo
FatNav

JoeLov

RoyEld
OrnCol

SidBec
DavLie

JohCol

MicBre

StaGet

SteLac
ArtPep

MilDav

SteCol
SonSti
J.JJoh
BixBei

CliBro

EriDol
DizGil

Figure 6: Ratio between the modulation frequency of vi-

Figure 4: Absolute deviation of tone-wise fundamental
brato tones and the average tempo of a piece vs. the aver-
frequency values from the estimated tuning frequency in
age tempo.
cent and modulation range in cent for all musicians (for
their full names see Table 1).
5.5 Artist-dependency of the Modulation Frequency
of Vibrato
100 100
75 Although there is no obvious correlation between modu-
ModRange [cent]

75
AvF0Dev [cent]

50 lation frequency and pitch, duration, or tempo, there are

25 50 some peculiarities of musicians according to vibrato modu-
0
25 lation speed. In Figure 7 only those musicians are included
25
for which more than twenty annotated vibrato tones could
50 0
be found in our data set. All in all, there seems to be no
tsc

tsc
cor

cor

clear correlation between vibrato speed and jazz style or

as

as
ss

ss
tb
tp

tb
tp
ts

ts
cl

cl

instrument, which indicates that modulation technique is

Figure 5: Absolute deviation of note-wise fundamental
mostly an idiosyncratic part of personal styles. Strikingly,
frequency values from the estimated tuning frequency in
several trumpet players can be found there (Louis Arm-
cent and modulation range in cent for all instruments.
strong, Kenny Dorham, Roy Eldridge) using vibrato to an
considerable amount and size. This is in sharp contrast to
828 Proceedings of the 16th ISMIR Conference, Malaga, Spain, October 26-30, 2015

Combining transcriptions with state-of-art MIR algorithms

Modulation frequency [Hz]

10
significantly enhances the methodical and analytical tool
8
box of jazz research (as well as other subfields of mu-
6
sicology and performance studies). In turn, this kind of
4 fine-structured analysis might be useful in guiding auto-
2 matic transcription algorithms by providing relevant back-
BenWeb
ground information on tone characteristics. Moreover, in
BenGoo

CheBak

DonBya

PauDes
DavMur

LouArm
LeeKon
BenCar

ChaPar

KenDor
FreHub

RoyEld
SidBec

StaGet
ArtPep

SonSti
BixBei
this study we demonstrated exemplarily that our method
can be readily applied for a range of different research
Figure 7: Modulation frequency in Hz in vibrato notes for questions, from historical analysis of reference tuning in
different performers. Only performers with more than 20 20th century jazz recordings to more general questions such
vibrato notes are shown. as intonation accuracy or differences in f0 modulations
with respect to tempo, instrument class, stylistic trends, or
personal style.
playing standards for brass instruments in classical music, As a case study, we investigated whether some these ex-
where it is custom to play without any vibrato [7]. pressive aspects, i.e., intonation, slides, vibrato speed and
vibrato range, are correlated with structural features of the
5.6 Automatic Classification of Frequency solos (absolute pitch, tone duration, overall tempo, meter)
Modulation Techniques and whether those aspects are characteristic for an instru-
Using the set of features discussed in Section 4.5, we ex- ment, a jazz style or the personal style of a musician. While
tracted an 18-dimensional feature vector for each tone, there is little evidence for a general correlation between in-
which was used to automatically classify tones with re- tonation and pitch modulation (slide, vibrato) on the one
spect to their modulation class. To this end, we only con- hand, and structural features on the other hand, the issue
sidered tones annotated with fall-off, slide, and vibrato of how intonation and pitch modulation contributes to the
since all remaining tones were not explicitly annotated. We formation of a jazz style and personal style needs further
used a Support Vector Machine (SVM) classifier with a lin- examination with more data and including listening tests
ear kernel function as classification algorithm and perform for style discrimination.
a 10-fold cross-validation. Due to the imbalanced class For the future, we plan to complete and refine the f0 -
sizes (cf. Table 3), we repeatedly re-sampled from the ex- modulation annotations for the dataset, with the overall
isting class items such that all classes have the same num- goal of the design of an automated f0 -modulation anno-
ber of items as the largest class from the original dataset. tation algorithm. Finally, we aim at a complete description
The confusion matrix is shown in Table 4. The highest of personal timbre characteristics, the so-called sound of
accuracy of 92.25 % was achieved for vibrato tones. The a player, which is an important dimension of jazz music,
classes fall-off and slide show lower accuracy values of and not yet fully addressed. Dynamics [2], intonation, ar-
48.04 % and 67.32 %, respectively. One might assume, ticulation, and f0 -modulation are part of this sound, but
that the similar f0 contour shapes of fall-offs and the slide- other aspects such as breathiness, roughness and general
downs causes part of the confusions between both classes. spectral characteristics (and their classification) are still to
be explored.
Correct Classified
Fall-off Slide Vibrato 7. ACKNOWLEDGEMENTS
Fall-off 48.04 37.46 14.49
Slide 23.55 67.32 9.13 The Jazzomat research project is supported by a grant DFG-
Vibrato 4.06 3.7 92.25
PF 669/7-1 (Melodisch-rhythmische Gestaltung von Jazz-
improvisationen. Rechnerbasierte Musikanalyse einstim-
Table 4: Confusion matrix for the automatic classification miger Jazzsoli) by the Deutsche Forschungsgemeinschaft
of frequency modulation techniques. All values are given (DFG). The authors would like to thank all jazz and mu-
in percent. sicology students participating in the transcription and an-
notation process.

6. CONCLUSIONS
In this exploratory study, we proposed a
score-informed algorithm for the extraction of
non-syntactical features in jazz solos played with wind and
brass instruments. This method allows for an analysis of
performative and expressive aspects of jazz improvisation
which are not captured by the traditional approaches of
jazz research such as transcriptions (even though some rudi-
mentary notation for f0 -modulations are used sometimes).
Proceedings of the 16th ISMIR Conference, Malaga, Spain, October 26-30, 2015 829

8. REFERENCES [11] Alexander Lerch. On the requirement of automatic tun-

ing frequency estimation. In Proceedings of the 7th In-
[1] Toshihiko Abe, Takao Kobayashi, and Satoshi Imai.
ternations Conference on Music Information Retrieval
Harmonics tracking and pitch extraction based on in-
(ISMIR), Victoria, Canada, 2006.
stantaneous frequency. In Proceedings of the 1995
IEEE International Conference on Acoustics Speech [12] Sai Sumanth Miryala, Kalika Bali, Ranjita Bhagwan,
and Signal Processing (ICASSP), pages 756759, De- and Monojit Choudhury. Automatically identifying vo-
troit, USA, 1995. cal expressions for music transcription. In Proceedings
of the 14th International Society for Music Information
[2] Jakob Abeer, Estefana Cano, Klaus Frieler, and Mar-
Retrieval Conference (ISMIR), Curitiba, Brazil, 2013.
tin Pfleiderer. Dynamics in jazz improvisation - score-
informed estimation and contextual analysis of tone in- [13] Meinard Mller and Sebastian Ewert. Chroma tool-
tensities in trumpet and saxophone solos. In Proceed- box: Matlab implementations for extracting variants
ings of the 9th Conference on Interdisciplinary Musi- of chroma-based audio features. In Proceedings of the
cology (CIM), Berlin, Germany, 2014. 12th International Society for Music Information Re-
[3] Jakob Abeer, Martin Pfleiderer, Klaus Frieler, and trieval Conference (ISMIR), pages 215220, Miami,
Wolf-Georg Zaddach. Score-informed tracking and USA, 2011.
contextual analysis of fundamental frequency contours [14] T. zaslan, Serra X., and Arcos J. L. Characteriza-
in trumpet and saxophone jazz solos. In Proceedings tion of embellishments in ney performances of makam
of the 17th International Conference on Digital Audio music in turkey. In Proceedings of the 13th Interna-
Effects (DAFx-14), Erlangen, Germany, 2014. tional Society for Music Information Retrieval Confer-
[4] Estefana Cano, Gerald Schuller, and Christian ence (ISMIR), pages 1318, 2012.
Dittmar. Pitch-informed solo and accompaniment sep- [15] Matti P. Ryynnen and Anssi Klapuri. Automatic tran-
aration: towards its use in music education applica- scription of melody, bass line, and chords in poly-
tions. EURASIP Journal on Advances in Signal Pro- phonic music. Computer Music Journal, 32:7286,
cessing, 23:119, 2014. 2008.
[5] Simon Dixon, Dan Tidhar, and Emmanouil Benetos. [16] Justin Salamon, Geoffroy Peeters, and Axel Rbel. Sta-
The temperament police: the truth, the ground truth, tistical characterization of melodic pitch contours and
and nothing but the truth. In Proceedings of the 12th its application for melody extraction. In Proceedings
International Society for Music Information Retrieval of the 13th International Society for Music Information
Conference (ISMIR), pages 281286, Miami, USA, Retrieval Conference (ISMIR), Porto, Portugal, 2012.
2011.
[17] Joan Serr, Gopala K. Koduri, Marius Miron, and
[6] Karin Dressler. Pitch estimation by the pair-wise evalu- Xavier Serra. Assessing the tuning of sung indian clas-
ation of spectral peaks. In Proceedings of the 42nd AES sical music. In Proceedings of the 12th International
International Conference on Semantic Audio, pages 1 Society for Music Information Retrieval Conference
10, Ilmenau, Germany, 2011. (ISMIR), pages 157162, Miami, USA, 2011.
[7] N. H. Fletcher. Vibrato in music physics and psy-
chophysics. In Proceedings of the International Sym-
posium on Music Acoustics, pages 14, 2010.
[8] David Gerhard. Pitch track target deviation in natural
singing. In Proceedings of the 6th International Con-
ference on Music Information Retrieval (ISMIR), pages
514519, London, UK, 2005.
[9] Chao-Ling Hsu and Jyh-Shing Roger Jang. Singing
pitch extraction by voice vibrato/tremolo estimation
and instrument partial deletion. In Proceedings of the
11th International Society for Music Information Re-
trieval Conference (ISMIR), pages 525530, Utrecht,
Netherlands, 2010.
[10] Christian Kehling, Jakob Abeer, Christian Dittmar,
and Gerald Schuller. Automatic tablature transcription
of electric guitar recordings by estimation of score- and
instrument-related parameters. In Proceedings of the
17th International Conference on Digital Audio Effects
(DAFx-14), Erlangen, Germany, 2014.