Evolutionary Jazz Improvisation - Jazz Harmony and Solo Improvisation Created by Means of Evolutionary Algorithms With Automatic Fitness
Evolutionary Jazz Improvisation - Jazz Harmony and Solo Improvisation Created by Means of Evolutionary Algorithms With Automatic Fitness
Evolutionary Jazz Improvisation - Jazz Harmony and Solo Improvisation Created by Means of Evolutionary Algorithms With Automatic Fitness
org
DOI: 10.13189/saj.2013.010302
Abstract This paper describes the Evolutionary Jazz b13 etc. and exchanged some chord progressions by
Improvisation (EJI) system for creation of jazz harmony and inserting an extra subdominant parallel, e.g.
jazz solo improvisations based on evolutionary algorithms G7 – C was replaced by Dm7 – G7 – C
with automatic evaluation. The evaluation function is However, the focus was still on major/minor and fifth
implemented using score calculation based on specific progressions. The main harmonic contribution from cool
events recognized in the chord progression and solo line, and jazz and hardbop during the 50’s was further advanced chord
alignment to different optimal intensity increase and colourizations. A few new-thinking musicians, like Ornette
relaxation curves. The evaluation function is based on the Coleman, Cecil Taylor, Don Cherry and others, began at the
author’s solo analysis of 73 great masters’ solo end of the 50’s to split up the harmonic foundation prevailing
characteristics and their techniques used for build-up of jazz until then, and this development continued during the
solos. The results have been evaluated by our live jazz group subsequent decades under stylistic classification into “modal
and used in jam sessions, some of which have been recorded jazz”, “avant-garde”, “free form” etc. Current jazz musicians
and made available for listening at the links given later. have to some extent adopted this break-up tendency.
Experiments have also been made during the 60’s and 70’s
Keywords Jazz, Evolution, Jazz Harmony, Artificial by e.g. Herbie Hancock, Miles Davies and fusion musicians
Intelligence Brecker Brothers. Not to mention all experiments in the
classical music domain during the entire 20th century from
Schoenberg and onwards.
However, mostly you can in the “modern” jazz styles trace
1. Aims and Context some remainders of the functional harmony principles and
the fifth circle basis. When progressive or avant-garde
1.1. The EJI Harmony System musicians create compositions with new harmony, there still
is a risk to get stuck in conventions dictated by routine and
Jazz harmony has during the jazz history mainly been learnt behaviour, idiomatic properties of the instrument and
functionally based on principles of tonality derived from the musician’s physical and muscular restrictions. The
classical and romantic periods of the 18th and 19th centuries. computer has no such restrictions but creates harmonies
Since the birth of jazz during the first two decades of the 20th controlled by the programmed algorithms. The aim is to be
century, the jazz harmony has been systematically organized able to free oneself from traditional thinking and find new
around a tonal centre by fifth progressions, which means that ways of thinking about jazz harmony.
each chord has been related to a base note and classified as By means of computer based evolutionary principles we
minor or major, and optionally also enriched with introduce a function-less harmony system that somewhat
colourization, such as: changes the musical feeling in jazz compositions by
Cm, Eb7, G13b9, A7#11 excluding the relation to any base note and the major/minor
Blues and ragtime harmony mainly used simple concept.
major/minor triads at the distance of fifths. Swing music In addition to alternate types of harmonies, the EJI
enriched the chords with sixths and ninths but the chord harmony system also forms the basis for scale construction
progressions were mainly the same. Bebop further enriched used by the jazz musician when performing a solo.
the chords with further colourizations such as b9, #9, #11, 13, Dahlstedt [7, 8], Dean [11], Levine [12], Manning [13],
68 Evolutionary Jazz Improvisation - Jazz Harmony and Solo Improvisation Created by
Means of Evolutionary Algorithms with Automatic Fitness
Rowe [15] and Thywissen [17] have made valuable 20’th century. In GenJam the function of listening to motives
contributions in the same area and have been sources of from fellow musicians has been solved by means of an
inspiration in the harmony area of the EJI system. Analog-To-MIDI converter device. However, GenJam does
not have the grand format principle of building up a solo with
1.2. Jazz Solo Improvisation focus on intensity fluctuations, which is the long-term aim of
our research.
The most important feature of a good jazz musician is to Francois Pachet [14] has developed a system where the
be able to keep an entire solo together as an entity, i.e. to user plays a melodic material, from which the system builds
build up the solo phrase by phrase in collaboration with the its solo according to Markov chain probability calculation.
other musicians, where each phrase is a natural continuation The user can at any time introduce new melodic material to
of the previous phrase and leads further to a climax of which the system responds.
intensity. After the climax the solo should be rounded off. A Dahlstedt [7][8] uses small melody fragments and
longer solo might contain several climaxes, but they should combines them using operators in a recursively generative
then be organized in a musically meaningful way. A good tree structure. Thywissen [17] uses generative music
improviser is not allowed to drop the focus and give way to grammars in his GeNotator project. Dahlstedt and
meaningless cascades of notes or producing routine phrases Thywissen apply their theories to classical music
for lack of artistic ideas. The challenge is to be able to plan composition. In our project we try to apply similar theories to
the structure of the solo already from start, and then stick to improvised jazz music. Also Manning’s [13] exploration of
the plan during the entire solo. There are some excellent MIDI technologies and Dean’s [11] work on
examples in the history of jazz with this ability, such as John hyperimprovisation have been valuable.
Coltrane, Miles Davis, Keith Jarrett, Bill Evans and Johnny Robert Rowe has in his two volumes, Interactive Music
Griffin, just to mention a few. Systems and Machine Musicianship [15] some interesting
An improviser often uses standard phrases and motives features such as scales connected to certain chord types,
trained during a long time of practicing and concerting. He which have been valuable for the development of this
relies on routines built up through repeated usage of similar system.
muscular movements well accommodated to the physical
design of the instrument.
The main purpose of the EJI project is not to create a jazz 2. Methodology
improvising computer of its own, but merely to open your
mind to new thinking, free oneself from old habitual paces of 2.1. Artificial Evolution
playing and enrich the improvisation style with new kinds of
musical material. The evolutionary algorithm process starts, from a basic set
The EJI project aims at making the computer build up a of parameters (genome), by creating a first random
solo based on these principles. This is done using population of pictures, melodies, chord progressions or
evolutionary principles on a genome structure consisting of a whatever. The fitness evaluation then takes place by
raw melody line split up into small melody fragments (delta examining the bred material (children) and selecting the best,
phrases) and a hierarchical structure of operators applied to optionally by giving each a score. The children with the
the delta phrases. Initially, the raw melody line is built up highest score have the highest probability to become parents
according to a “rubber band” principle, where each pitch for the next generation. The breeding is done by combining
interval is constructed using energy constraints much like the the genome of two parents by crossover, optionally by
tension of a stretched rubber band. After application of the applying a mutation somewhere in the genome. The
operators, the delta phrases will be somewhat different, mutation might imply a shift between two parameter values,
however hopefully preserving some kind of musical idea. or a slight modification of a genome value.
The aim is to ensure logical development of a consistent The principle of using evolutionary algorithms to develop
material during the solo and thus reflect the feature of a new artistic productions was first started on a broader scale
well-planned solo. in the digital graphics area, such as forerunner Karl Sims
The technical aspects and details are described in a [16]. The evolutionary algorithms principle is well
specific paper [2] by the author. accommodated to that area because when using interactive
There are others working with similar concepts. Al Biles evaluation of a created generation, as described by Dawkins
[1] has developed a system, GenJam, which uses phrases [9, 10], you can swiftly scan over a great number of pictures
played by the “master” soloist as basis for the computer and select the best according to your personal preference.
played solo. GenJam has three types of improvisation: whole With audio material, however, the selection procedure is
chorus, chase improvising and collective improvising. The much slower since you will have to listen through each bred
two last types are typical for older jazz forms like New melody in a generation, one at a time. The first experiments
Orleans jazz and the Swing era, while the first type is more in the music area have been made by Collin Johnson and
relevant to modern jazz forms from the second half of the Palle Dahlstedt [7, 8].
Sport and Art 1(3): 67-74, 2013 69
The fitness selection and breeding is repeated generation a more homogenous chord sequence.
by generation until you arrive at a genome good enough to be - Maximum number of half-note steps to be allowed when a
used for reproduction of artworks (pictures / melodies etc.). voice moves from chord to chord. Also in this case, a greater
See [2] for details. tolerance gives more abrupt chord changes.
This process is much the same as the genetic process of A genome consists of the absolute MIDI pitches for the
creating a new species generation in nature, only that it must initial chord. The pitches are randomly created within a
be sped up considerably to have a chance to be completed in specific pitch range around middle C. For each chord change
proper time, which is accomplished by the support of the the genome holds the number of half-note steps per note (fig.
computer. 1).
On breeding, the crossover is performed by combining The creation of the raw material uses a similar technique
different sections of two parents’ genomes similar to the as the one used in the mid-point displacement algorithm for
process of combining DNA for species. This means that the landscape generation. It is created by originating from start
figures (e.g. -1, 0, 1, 0) are taken from one of the parents and end pitches, then dividing the interval recursively. The
from the beginning of the genome, up to the randomly middle pitch is stored for each interval division. It is
selected break point, from where the remainder is taken from represented as a deviation from the mean between the start
the other parent. Then, a mutation is made by amending a and end pitch. Thus, the representation is a binary tree of
few values one step up or down, so -1 might be -2 or 0, etc. deviation figures from the mean line between two interval
When a child has been created in this way, it is evaluated end points.
as described above. If the child's score exceeds that of the The raw material is created as follows. First we calculate a
worst individual, it will replace that individual, which is random start pitch and an end pitch (fig. 3).
discarded. If the created child is worse than the worst
individual of the population, the child will be discarded.
Thus, according to the elitism principle, a created child, if
kept, will always improve the quality of the entire
population.
Figure 3. The start and end points of the rubber band.
In this experiment 10000 iterations are used in each run.
Maybe a larger number of iterations could result in a still Then we generate a random pitch in the middle of the time
better solution, but by experimentation we have found 10000 span, which is allowed to deviate from the mean pitch line by
iterations enough. a maximum pitch span (fig. 4) specified by a control
Further details are found in [4] and [5] by the author. parameter.
That said about harmony evolution, let us turn to solo
evolution.
adjusted to achieve sudden burst outs of short notes. 2.7. Operator Tree
The note lengths are not quantized to the standard
rhythmical values of whole notes, half notes, quarter notes, The purpose of organizing operators hierarchically into an
eighths, triplets etc. The lengths can have any MIDI ticks operator tree is to allow each delta phrase to be processed
value. The reason for this is to not being tied up to traditional hierarchically by a series of operators. Fig. 7 shows the
musical thinking concerning rhythm, but to concentrate on structure of an operator tree.
melody shapes and intensity fluctuations. This will however
provide a free-rhythmic feeling separated from the beat. As
an option, when applying the melodies to a jam session
situation, we have created a function for accommodation of
the rhythms to standard values of quarter notes, eighths,
sixteenths, triplets etc.
For dynamics, the rubber band principle is applied
similarly; a start note volume and an end note volume are
generated. The middle note volume of the melody interval is
selected with a deviation from the mean volume. The Figure 7. Structure of an operator tree.
deviation must be within the allowed volume span, which The operator at the top level is applied to all delta phrases.
also is reduced by a certain factor each time the interval is The operators at level 2 are applied to half of the delta
divided. By modifying the volume span factor you can phrases each. The 4 operators at level 3 are applied to ¼ of
achieve more or less smooth volume shapes. the delta phrases. The division by 2 for each level is
By combining the rubber band principles for pitch, length continued until there is one single operator for each delta
and volume we achieve pitch shapes, length shapes and phrase. The effect of this is that each delta phrase is
volume shapes operating independently of each other. processed by a series of operators from top to bottom of the
The technical representation of the raw material is MIDI operator tree, one operator per level. Two adjacent delta
pitch, length and volume for the start note and end note. The phrases will be processed by the same series of operators,
contour is represented as a tree structure of relative values of except for the last operator, which is different. Two distant
pitch, duration and volume (relative to the mean of the end delta phrases will be processed by two different sets of
points of the current interval), which, when applied operators. The effect will be a process introducing
recursively, will recreate the exact contour. conformity over the whole solo, and as the recursive process
Thus, the genome representation for the raw material branches out, variation is introduced between sections.
rubber band melody is: Each operator modifies a delta phrase in one particular
- Maximum allowed span from the beginning for pitch, way. The operator type options are given in table 3.
length and volume
- Factors for reduction of the allowed span for pitch, length Table 3. Operator options.
and volume
Unity, no change.
- Start note: pitch, length, volume
- End note: pitch, length, volume Transposition of a random single note by a random number of half notes
- For each interval division (2, 4, 8, 16, etc.): Middle note – Transposition of the entire delta phrase by a random number of half
deviation from the mean: pitch, length, volume, stored as a notes
factor of the maximum allowed deviation span for the Addition of a note at a random position of the delta phrase, random
interval in question. within the allowed span
Further details are found in [2 by the author. Removal of a randomly selected note in the middle of the delta phrase
The composition of operators in the operator tree is Figure 9. Calculated points per bar during the solo, with 50%
randomly created based on probability percents (control accumulation from the previous bar.
parameters) per each type of operator in table 3. Since the intensity provided by a melody fragment tends
To incur more operator processing to a delta phrase than to stay in the listener’s ear for some time, the score values per
accomplished by one series of operators from top to bottom bar will be preserved to some extent; 50% of the score value
of the operator tree, a delta phrase is allowed with a certain for one bar is added to the score value of the next bar. The
probability of recursion. If an operator is selected for score values per bar will have a graphic representation
recursion, all operators in the corresponding sub-tree will get something like in fig. 9.
the same number of recursions (fig. 8). The frequency of Having calculated the score value per bar, the score value
doing this is controlled by a parameter. pattern is compared to an optimal curve. The idea is that the
solo might contain several climaxes and several relaxation
points without any predestined position. We classify a point
above a specific limit as a climax (max point). The limit is
specified as a percentage (control parameter) of the overall
maximum point. We have experimented with 90%, but other
values could be used. A relaxation point is a point below a
certain limit, specified in the same way as a percentage of the
overall maximum point. We have experimented with 10%.
There might be several max points in sequence, and several
min points in sequence. The highest of the climaxes in
Figure 8. Recursion of a sub-tree.
sequence is classified as the max point, and the lowest of the
To allow for a richer variation, each node of the operator relaxations is classified as the min point. We measure the
tree can be equipped with up to 10 operators, which means optimal gradient between each max point and the subsequent
that several operator options from table 1 can be performed min point, and the optimal gradient between each min point
to the corresponding set of delta phrases. If recursion is in and the subsequent max point. An example is shown in
effect for the node, all additional operator options will be figure 10.
performed the specified number of times.
The genome representation of an operator is:
- Operator type (table 3)
- No of steps to transpose
- Note number for transposition, rhythm adjustment, addition,
removal
- New deviation factors for addition (pitch, length, volume)
- Additional operators (a list of up to 10 additional operators
per node) Figure 10. Flexible optimum method.
- Recursion (number of times to repeat the operator sub-tree)
- Insertion of rests (length) The system rewards great differences between max and
- Repetition (first, middle or last part) min points. The greater differences, the higher score. The
score value to be assigned is calculated as the sum of the
2.8. The Solo Evaluation Process differences between each max and min point, which then is
divided by the number of max/min points with the aim of
The evaluation of an improvised solo is carried out by a avoiding too many max/min points.
number of analysis functions, which contribute with a score The deviations from the optimal gradient per bar are
value stored per note of the melody. When all analysis subtracted. The aim is to get as little total deviation as
functions have contributed with their score values per note, possible from the optimal gradient.
the points are aggregated per bar. At the end of the evaluation, To summarize, the greatest differences between climax
the aggregated points per bar reflect the intensity fluctuations and relaxation, and the closest connection to the optimal
of the melody. gradients, will be the best result.
selected as parents for breeding [2]. The selection of two Chord progressions created by the evolutionary algorithm
parents for a single child is stochastically made, based on described above provide the feeling of a continuous progress
their evaluation score, implying that the best parents will towards new heights without arriving at any rest points,
create the largest number of children. There is a lower score which is the case with traditional functional harmony, where
limit that must be exceeded by any parent to allow for some chords have a striving character to dissolve into tonics.
participation in the stochastic selection. The limit is Our conclusion is that this new kind of harmony has an
controlled by a parameter. The limit is also increased along on-going forward-striving feature not prevalent in
with the evolution process, since the entire population will mainstream jazz harmony.
grow better and better. The limit is at each generation set to Compared to manual evaluation, where you have to listen
90% of the best score. to each generated individual, one at a time, the automatic
After breeding by crossover, followed by some mutations evaluation takes full advantage of the evolution theory by
on a probability basis [4], the child is evaluated and, if it has utilizing large populations and a huge number of generations.
a better score than the worst one, it will replace the worst Furthermore, the evaluation criteria are kept strictly constant
individual, which is discarded from the population. i.e. we do not change focus on the objectives of the evolution
The fitness selection and breeding is repeated a number of process. Of course there are also drawbacks with automatic
times, specified by a control parameter. We have evaluation. It is difficult, if at all possible, to make the
experimented with 10000, but any number can be used. computer evaluate abstract concepts such as musicality,
However, 10000 has been regarded enough, since very little tension, expectation, climax, relaxation etc. Anyway, with
score improvement has been measured after 10000 iterations. the automatic evaluation we obtain results that might not
It is also possible to perform the repeated selection and otherwise have been discovered.
breeding until the total score value has reached a limit value When jamming with a jazz group on tunes with this new
(a control parameter). type of harmony, it has the effect on the soloist of
The analysis functions contributing with the detailed score continuously proceeding towards a climax never completely
values are described in detail in other documents written by reached. The soloist is compelled to go on and on and on.
the author [3, 4, 5, 6]. They correspond to the categorization The listener will be involved in this forward-striving feeling
of the techniques resulting from the author’s solo analysis of of wanting more all the time.
73 great masters. Furthermore, the score given by each When I experimented with these ideas in a live jazz group,
function corresponds to the level of utilization. E.g. if a it turned out that the musicians had apparent difficulties in
technique, like repetition, say, is used by many musicians, it keeping chords and scales in their minds during their solos,
will give a high score when encountered in the solo. Below is since they had to learn completely new chords and scales.
a summary of the analysis functions: The harmony was of a kind that they could not apply their
●Density current knowledge and personal routine and not trust old
●Rhythm learnt patterns of behaviour. Clever and experienced
●Motif repetition musicians appeared to be relative beginners, at least during
●Sequence the first rehearsals. Difficulties became obvious especially
●Chromatics when playing tunes with an odd periodicity where a chord
●Melodig heights could last for 3 bars and the next chord for 2 ½ bar, etc. So
●Rests the time required for rehearsal tended to grow remarkably.
●Polyphony For example, the bassist, who normally bases his walking
●Counterpoint bass paths on a base note accentuated at the first beat of each
●Ornamentation bar and scale walking at the remaining beats, got into
●Cascades of rapid notes problems when there was no specific base note. Learning to
●Mix of extremely high and low notes play this new kind of music is a laborious task that requires a
●Stubborn twisting new way of thinking and a lot of practice and patience.
●Rhythmic effects Furthermore, to find the most adequate way of playing, a
●Motives from other tunes lot of time was spent on discussion and reflection in the
●Double tempo acoustic live jazz group. For instance, a great deal of
●Rubato cooperative work was used for accommodation of the
●Hard swing bassist’s notes and the piano chord layout to each other.
●Polyrhythmic effects
Refer to [6] for a closer description of the analysis functions. 3.2. Solo Improvisation
Comparing the manual solo evaluation to the automatic
3. Results and Future Work evaluation, the manual method has obvious advantages by
being able to select melodies of personal preference.
3.1. Harmony However, by being able to use a huge population and a great
74 Evolutionary Jazz Improvisation - Jazz Harmony and Solo Improvisation Created by
Means of Evolutionary Algorithms with Automatic Fitness
number of generations, like in the harmony evolution, the [2] Anonymous (2008) A Generative Representation for the
drawback of artificial automated score calculation is by far Evolution of Jazz Solos. In Proceedings of EvoWorkshop
Conference 2008, Napoli.
overcome.
As previously mentioned, the overall main purpose of this [3] Anonymous (2008) Evolutionary Jazz Harmony, In
project is not to create a jazz improvising computer program Proceedings of BIOMA Conference 2008, Ljubljana
of its own, but merely to open your mind to new thinking, [4] Anonymous (2008) Automatic Fitness in Evolutionary Jazz
free yourself from old habitual paces of playing and enrich Improvisation, In Proceedings of ICMC Conference 2008,
your improvisation style with new kinds of musical material. Belfast.
The automatic fitness process can be considered fulfilling
[5] Anonymous (2008) Automatic Fitness in Evolutionary Jazz
this aim. Improvisation, In Proceedings of ICMC Conference 2008,
The sounding output of the generated music examples has Belfast.
obvious intensity fluctuations indicating some kind of solo
grand layout often used by professional jazz musicians. The [6] Anonymous (2010) Analysis Jazz Solo Build-up Techniques,
http://oden.ei.hv.se/kjelIn Proceedings of ICMC Conference
music examples definitely have an interesting feature of 2008, Belfast.
intensity build-up and relaxation.
A set of unaccompanied sound examples can be heard at [7] Dahlstedt, P. (2001) Sounds Unheard of – Evolutionary
algorithms as creative tools for the contemporary composer.
this link in General MIDI format:
In Proceedings of ICMC Conference 2001, La Habana, Cuba.
http://oden.ei.hv.se/kjell/autofitness .
A link to an example with computer generated drums and [8] Dahlstedt, P. (2001) Creating and Exploring Huge Parameter
bass accompaniment is also available: Spaces: Interactive Evolution as a Tool for Sound Generation
In Proceedings of ICMC Conference 2001, La Habana, Cuba.
http://oden.ei.hv.se/kjell/autofitness/midi_evolv1.mid
An example of an orchestrated tune with virtual [9] Dawkins, R. (1986) The Blind Watchmaker. New York W. W.
instruments and my own acoustic piano playing is available Norton & Company, Inc. New York, USA.
at this link: [10] Dawkins, R. (1976) The Selfish Gene. Oxford University
http://oden.ei.hv.se/kjell/autofitness/gate.mp3 Press Inc., New York , USA.
The same tune has been recorded by our live jazz group:
http://oden.ei.hv.se/kjell/CD/Gate.mp3 [11] Dean, T. (2003) Hyperimprovisation: Computer-Interactive
Sound Improvisation. A-R Editions Inc.,Middleton,
Wisconsin.
3.3 Future Work
[12] Levine, M. (1995) The Jazz Theory Book. SHER MUSIC CO.
The study of jazz improvisation solos has mainly been Petaluma, CA, USA.
carried out from individual perspectives, i.e. how the [13] Manning, P. (2004) Electronic and Computer Music. Oxford
individual soloist shapes his performance. But it has become University Press, New York, USA.
more and more apparent during the listening phase that, the
[14] Pachet, F. (2002) Interacting with a Musical Learning System:
communicative features are ultimate to the intensity build-up. The Continuator. SONY-CSL, Paris, France.
Therefore we will examine the memetics theory introduced http://www.csl.sony.fr/¨pachet (Accessed 2 March 2006).
by Richard Dawkins at the end of his book The Selfish Gene
[10]. [15] Rowe, R. (1993) Interactive Music Systems. The MIT Press,
Cambridge, Massachusetts, USA.
[16] Sims, K. (1991) Artificial Evolution for Computer Graphics,
ACM SIGGRAPH '91 Conference Proceedings, Las Vegas,
Nevada, July 1991.
REFERENCES [17] Thywissen, K. (1996) GeNotator: An environment for
[1] Biles, J.A. (1994) GenJam: a genetic algorithm for generating investigating the application of generic algorithms in
jazz solos. Proceedings of the 1994 International Computer computer assisted composition. In Proceedings of
Music Conference. ICMA, San Fransisco, pp. 131-137. International Computer Music Conference 1996 (ICMC96),
pp. 274-277, Hong Kong.