Algorithmic Music

This thesis focuses on the analysis and simulation of the behaviour of cognitive systems – both biological and non-biological – which are able to process musical information (i.e., creative music agents). The research questions of this work are: How can we formally define a creative system? How can we formalise the behaviour of creative music agents? How can we simulate the behaviour of creative music agents computationally? Neither mathematical nor computational systems sophisticated enough exist that attempt to describe/simulate how music agents behave and form musical societies.

To respond to the research questions a double approach based both on theoretical investigation and computer simulation is employed. Two formalised frameworks are proposed, i.e., General Creativity (GC) and Music Systems Theory (MST). GC attempts to extend the notion of creativity to non-biological systems. In doing so, GC provides a new definition of a creative system as well as of an iterative creative process. These notions are at the basis of MST, a theory which aims to describe
how agents (both biological and non-biological) involved in musical activities interact with each other and create musical societies. By using systems theory, MST attempts to describe how single agents and musical societies evolve over time.

The core concepts of GC and MST are used as a basis to design S(S(S)), an AI multi-agent system which has the double goal of creating convincing rock songs and of simulating a society of virtual songwriters which create songs, listen to the songs composed by their peers, and change their compositional style based on what they hear. S(S(S)) aims to provide an indirect proof of the validity of GC and MST. An exhaustive evaluation of S(S(S)) was carried out with expert rock musicians which demonstrated that the system shows an overall high level of creativity and that the single agents of the system are indeed capable of modifying their compositional
style.

This thesis provides several contributions. GC provides a formalise definition of creative systems which is valid for human and non-human systems, and which can be used as a basis for designing computational creative systems. MST is the first mathematical theory that attempts to formalise the behaviour of musical societies based on systems theory. S(S(S)) is one of the first systems that is able to show a degree of transformational creativity. The thesis also shows by example that the
simultaneous use of theoretical and computational modelling is an effective tool for tackling complex musical issues, and it is hoped that it will form the theoretical and methodological basis for future research in this rapidly advancing field.

Les L-systèmes constituent un outil original de représentation des formes musicales à disposition
des compositeurs, notamment dans le domaine de la musique générative. Peu normatifs, ils
permettent de stimuler la créativité en pensant la forme par croissance organique, une approche
compatible aussi bien avec la tradition ancienne du développement continu qu'avec celle plus
récente des organisations par processus.

When composing music for an interactive medium the composer must overcome different obstacles from more linear entertainment such as film.  The non-linear nature of games often conflicts with the idea of player autonomy and thus new methods of working must be implemented.

This paper looks at using a "generative" approach to musical composition in games as a possible solution to some of the challenges found in composing for an interactive medium. Various forms of generative and algorithmic music will be discussed and a proposed strategy for use in a game situation will be given.

Computer code is a form of notational language. It prescribes actions to be carried out by the computer, often by systems called interpreters. When code is used to write music, we are therefore operating with programming language as a relatively new form of musical notation. Music is a time-based art form and the traditional musical score is a linear chronograph with instructions for an interpreter. Here code and traditional notation are somewhat at odds, since code is written as text, without any representational timeline. This can pose problems, for example for a composer who is working on a section in the middle of a long piece, but has to repeatedly run the code from the beginning or make temporary arrangements to solve this difficulty in the compositional process. In short: code does not come with a timeline but is rather the material used for building timelines. This article explores the context of creating linear ‘code scores’ in the area of musical notation. It presents the Threnoscope as an example of a system that implements both representational notation and a prescriptive code score.

Il live coding è una forma di improvvisazione che consiste nella scrittura di algoritmi in tempo reale.
In sostanziale continuità con la tradizione della ricerca nel campo della musica algoritmica, la pratica del live coding è emersa negli ultimi venti anni grazie al lavoro di programmatori informatici, compositori e performer. La disponibilità in forma gratuita e libera degli strumenti di programmazione per il live coding ha permesso lo sviluppo di una sottocultura in grado di generare un contesto socio-culturale specifico.
Benché il live coding sia da qualche anno oggetto di studi accademici, le ricerche sugli aspetti musicologici e tecnologici sono ancora all’inizio. A partire da quanto sinora esplorato dalla letteratura, questi tesi ha lo scopo di indagare le implicazioni compositive e performative proprie del live coding, tentando di tracciare un possibile percorso di sviluppo di un ambiente per l’improvvisazione algoritmica. All’analisi storica e musicologica è affiancato dunque uno studio delle attuali architetture informatiche, al quale segue la costruzione di un sistema specifico in grado di applicare tecniche compositive tradizionali e processamenti elettroacustici in tempo reale.
Il primo capitolo si occupa di inquadrare il contesto storico-sociologico che ha portato allo sviluppo delle attuali forme di improvvisazione algoritmica, dapprima descrivendo la nascita della cultura degli Algorave (sez. 1.1), dunque ricostruendo le influenze ideative provenienti dai precursori della computer music (sez. 1.2); a partire da tali prospettive si è delineata una breve storia del live coding (sez. 1.3), sia sul piano tecnologico, sia su quello sociologico, sino a giungere all’attuale conformazione associazionistica e non-centralizzata, i cui aspetti etnografici e discografici sono discussi nella sez. 1.4. È stato proficuo indagare quali fossero i principi politici e ideologici propri della comunità dei live coders (sez. 1.5), con particolare enfasi sulle modalità di divulgazione e condivisione della conoscenza proprie dell’Open Source. L’indagine sulle proprietà musicologiche intrinseche dell’improvvisazione algoritmica è oggetto di studio nel secondo capitolo. In particolare sono state prese in esame le implicazioni derivanti dalle specificità tecniche di questa
1
pratica sul piano temporale di una performance (sez. 2.1) e le relazioni tra scrittura musicale e scrittura algoritmica (sez. 2.2); conseguentemente sono state analizzate le potenzialità inscritte nella codifica del linguaggio musicale mediante i linguaggi di programmazione (sez. 2.3). Una riflessione sulla multimedialità del live coding e i suoi effetti percettivi sul pubblico è presente nella sez. 2.4.
Il terzo capitolo ha lo scopo di comparare gli attuali linguaggi di programmazione musicale in tempo reale (sez. 3.1) e riflettere sui parametri determinanti nella scelta di un design specifico (sez. 3.2). In luce di quanto rilevato, si è proposta una possibile architettura composta da molteplici linguaggi utilizzati in sinergia (TidalCycles, Haskell, SuperCollider, Max/ MSP), il cui funzionamento è descritto dalla sez. 3.3 alla 3.6. A una spiegazione teoretica sono affiancati esempi scritti in tali linguaggi.
Una dimostrazione delle possibilità inscritte nel sistema così definito costituisce il quarto e ultimo capitolo. La sez. 4.1 raccoglie una serie di algoritmi di sintesi e processamento scritti in linguaggio SuperCollider negli ultimi tre anni, mentre una messa in pratica delle tecniche stocastiche è illustrata nella sez. 4.2. La tesi si conclude con l’implementazione di alcune tecniche ritmico-contrappuntistiche in linguaggio Haskell (sez. 4.3). Quanto mostrato nell’ultimo capitolo rappresenta un esempio di personalizzazione di un ambiente di live coding, le cui funzionalità possono essere definite dal compositore elettroacustico. Piuttosto che una trattazione esaustiva del campo di possibilità del live coding, si tratta dunque di fornire degli strumenti, affinché ciascun compositore possa intraprendere il live coding con la prospettiva di poter costruire un ambiente specifico e direzionato verso le proprie esigenze espressive.
Tutti gli algoritmi mostrati nell’ultimo capitolo sono presenti sul mio profilo GitHub (Olbos, 2020) e possono essere utilizzati e modificati liberamente.

The field of interactive music systems (IMSs), beginning in the 1980s, is still relatively young and fast moving. The field of music theory-analysis, during the same period (since 1980), has undergone a major transformation in terms of technological innovations, flexibility, and breadth. The two fields have not really caught up with each other. It will be interesting to see what arises as they do — especially as both fields have become more concerned with the role of the body and embodied cognition. Framed in terms of contrasting epistemological orientations, this essay considers some relevant developments in IMSs, music cognition, and music theory and analysis, leading up to the present.  The most popular approaches to music cognition and to IMS design are rationalist (Ashby 2010), exploiting the ‘correct’ embodiments of music (Godøy 2004, Leman 2007, Paine 2009) based on affordances (Gibson 1977, Kelso 1998). This essay, however, advocates a pragmatist (Ashby 2010) approach inspired by music analysis and exploiting the potential of kinesthetic learning (associative learning). Prompted by a progressive approach to music analysis, theory, perception, and cognition (Dubiel 1999, Mailman 2007), interactive music technology can also be constructive, flexible, and progressive, by exploiting kinesthetic learning from immersion in new and unusual motion-to-sound mappings derived from dynamic formal processes in analysed music. In this way, immersive interactive systems offer an opportunity systematically to learn new associations based on principles theorised in response to analysis. Experience of these systems essentially ‘rewires the brain’, thereby exemplifying what Korsyn (2004) has attributed to Lewin’s (1986, 1987) approach to music perception: the ironist approach, as formulated by Rorty (1989). Thus a pragmatist ironist experimental (PIE) approach is articulated. Rather than committing to any particular ways music is already embodied, this approach acknowledges the flexible nature of embodied musical experience. It forges and uses interactive music technologies continually to redescribe and therefore reform how music is embodied, thereby disembodying to re-embody music, expanding how music is heard, contemplated and experienced.

In the last years an increasing interest can be observed for developments in game engine technologies as a versatile creative tool. In particular, the possibility to visualize and simulate real-time complex physical behaviors facilitates the design and implementation of 3D virtual music instruments and the exploration of sound gesture as a result of their kinematic and spatial properties. This paper describes two case examples in the form of linear compositions based on non-conventional instrumental designs where audio is procedurally generated using custom-built APIs in the game engine scripting language (Unity3D-Javascript/C#). Sound events are also organized as a sequence of flexible code instructions, resulting in a quasi-fixed piece duration with subtle timbral variations over multiple playbacks. In both cases, the model presented shows inherit spatial characteristics, which are useful in order to build spatialization patterns in a multichannel loudspeakers configuration and emphasize the strong causal connection between the visual and sonic aspects of these works.

A lot of attention in music system design is being given to data entry and score storage formats, timbral synthesis techniques, and to user interface refinements. Such considerations often say more about the problems of computer implementation of music than they do about problems or structures inherent in musical acitivy itself.

The rhizome concept explored by Deleuze and Guatarri has had an important influence on formal thinking in music and new media. This paper explores the development of rhizomatic musical scores that are arranged cartographically with nodal points allowing for alternate pathways to be traversed. The challenges of pre-digital exemplars of rhizomatic structure are discussed. It follows the development of concepts and technology used in the creation of five works by the author Ubahn c. 1985: the Rosenberg Variations [2012], The Last Years [2012], Sacrificial Zones [2014], detritus [2015] and trash vortex [2015]. The paper discusses the potential for the evolution of novel formal structure using a rhizomatic approach.

In the last years an increasing interest can be observed for developments in game engine technologies as a versatile creative tool. In particular, the possibility to visualize and simulate real-time complex physical behaviors facilitates the design and implementation of 3D virtual music instruments and the exploration of sound gesture as a result of their kinematic and spatial properties. This paper describes two case examples in the form of linear compositions based on non-conventional instrumental designs where audio is procedurally generated using custom-built APIs in the game engine scripting language (Unity3D-Javascript/C#). Sound events are also organized as a sequence of flexible code instructions, resulting in a quasi-fixed piece duration with subtle timbral variations over multiple playbacks. In both cases, the model presented shows inherit spatial characteristics, which are useful in order to build spatialization patterns in a multichannel loudspeakers configuration and em...

Chapter from 'Parametric Composition' (Morgan & Legard, 2017).

Throughout this book, we have often presented code by way of small, illustrative script snippets alongside occasional longer pieces, such as the 12-tone invention in chapter 10. Longer pieces, such as the aforementioned invention, often demonstrate a fairly linear flow of list processing as a way of demonstrating the technique and thinking behind scripted composition.
However, there will be many compositional situations in which the making of the final piece will not necessarily follow such a linear 'waterfall' coding approach, and may need to make use of loops, logic and the preparation of bespoke functions (we encountered some of these in chapters 7, 12, 16 and 21). This chapter explores the use of logical mechanisms and mathematical operators.

This paper concerns the hypersampler implemented for my piece "Il grifo nelle perle nere" for piano and hypersampler, composed in 2008 and premiered during the 54th Venice Biennale’s International Festival of Contemporary Music, in the framework of XVII CIM – Colloquium on Music Informatics.
The hypersampler involves a real-time synthesis engine based on processes of feature extraction as an alternative to hyperinstruments’ physical control paradigm.
Features are derived from the performance of a traditional musician on an acoustic instrument – a piano
– and are used as a control for the mapping between the instantaneous power spectrum of the acoustic instrument’s sound output (the musical dynamics performed by the pianist) and real-time synthesis engine’s parameters.

Esta presentación se analizan diversos métodos y técnicas vinculados a las teorías de la complejidad y el caos y su impacto en la musicología en general (y la antropología de la música en particular). La premisa fundamental establece una concordancia entre "clases de universalidad" que atraviesan las formulaciones del trabajo científico, independientemente de la materialidad de los objetos disciplinarios. En particular se examinan recursos conceptuales y herramientas de trabajo relacionados con la geometría fractal, los sistemas complejos adaptativos, los algoritmos genéticos y la dinámica no lineal. Los mismos involucran principios de independencia de escala, distribución 1/f, dimensión fractal, auto-organización que permiten comprender la problemática de la música en la cultura desde una óptica inusual.

—The new work, Morphons and Bions (2011), is a real-time computer music work generated in Pd. All sounds in the work are synthesized as opposed to recorded. The sound sources rely fundamentally on white noise and digital noise mediated by classical synthesis techniques and random processes. Since the work is built on a substrate entirely made of noise, the piece is situated within certain philosophical and aesthetic issues surrounding noise, its use, and its definition. This paper seeks to address these issues.

Aunque el tema se ha retorcido, simplificado y exagerado hasta el paroxismo, la música posee innegablemente características algorítmicas.  En ella se manifiestan una combinatoria, una red de relaciones, una pauta generativa, un conjunto de formas posibles y una lógica de series temporales cultural y cognitivamente modulada. Es por eso que se la puede analizar y sintetizar ricamente y desde puntos de vista novedosos mediante herramientas ligadas a las teorías de la complejidad y el caos.

Por añadidura -y por razones que aun no se comprenden bien- es más probable que la algorítmica compleja produzca música y no que engendre mero ruido.  Todavía falta bastante para el advenimiento de la máquina sensible que nos prometiera Ray Kurzweil; pero si se consideran válidos los experimentos que admiten géneros musicales de vanguardia, se podrá comprobar al cabo de los ejercicios que se proponen en esta página (y por poco empeño que se ponga en su ejecución ) que los algoritmos sonoros pueden superar con holgura una versión no trivial de la prueba de Turing. Aun en el terreno de las formas clásicas y convencionales de composición, los avances que han habido en la segunda década del siglo XXI en materia de inteligencia computacional compleja van mucho más allá de lo que en la época del heideggeriano Hubert Dreyfus [1929-2017] se habría estimado verosímil.

En esta presentación (preparada originalmente para los Cursos de Contexto de la Universidad Central de Bogotá, Colombia, dictados en el año 2010) se examinan algunos de esos instrumentos virtuales, basados en formalismos tales como los gráficos de recurrencia, la dimensión fractal, la geometría fractal propiamente dicha, los modelos basados en agentes, los autómatas celulares, la teoría de grafos y el algoritmo genético. Algunas variantes de complejidad desorganizada también integran estas estrategias: redes neuronales, sistemas expertos, métodos estocásticos, camino al azar, cadenas de Márkov. Algunas veces (como en GenJam) se combinan varias especies algorítmicas.

No puede asegurarse sin más que los recursos algorítmicos de la complejidad sean mejores en su desempeño compositivo que los que implementaban los métodos aleatorios o combinatorios más convencionales. Lo que sí es seguro es que las músicas generadas mediante algoritmos complejos se sienten casi invariablemente más “humanas” o “naturales” que las que surgen de métodos numéricos estándar, los cuales con frecuencia tienden ya sea hacia la rigidez o hacia el sinsentido.

Correspondientemente, la premisa dominante en la línea de investigación que aquí se adopta sugiere que si un método algorítmico o principio formal no puede sintetizar música entonces es dudoso que se trate de una genuina herramienta de complejidad. Más allá de que esa presunción resulte cierta o no, la creación musical puede constituir un recurso pedagógico excelente para introducir a los estudiosos de todas las ciencias en el tema de la complejidad y en las complicaciones de las técnicas que la integran. Para no hablar, por supuesto, de demixtificar  y tornar menos ominosas las matemáticas implicadas. Aun cuando los factores cuantitativos que aquí se despliegan son mínimos (y quizá por eso mismo) en pocos campos del conocimiento las matemáticas se sienten tan cercanas a la creación artística.

La presentación sobre Música y Complejidad (que se puede bajar, reproducir y modificar a total voluntad) puede verse en el documento aquí expuesto.

This paper studies the use of Gumowski-Mira maps for sonic arts. Gumowski-Mira maps are a set of chaotic systems that produce many organic orbits that resemble cells, flowers and other life forms. This has prompted mathematicians and eventually artists to study them. These maps carry a potential for use in the sonic arts, but until now such use is non-existent. The paper describes two ways of using Gumowski-Mira maps: for synthesis and spatializa-tion. The synthesis approach, which runs in real-time, takes the dynamical system output as the real and imaginary input to an inverse Fourier transformation, thus directly sonifying the algorithm. The spatialization approach uses the shapes of Gumowski-Mira maps as shapes across the acoustic space, using the first 128 iterations of each map as audio particles. The shapes can change based on the maps' initial parameters. The maps are explored in live performance using Leap Motion and Cycling '74's MIRA for iPad as control interfaces of audio processing in SuperCollider. Examples are given in two works, Cells #1 and #2.

"Dalla Sua Orbita" is a piece for hyperviolin written in 2007 and revised in 2011 in which the hyperinstrument system produces modifications of sound and structure starting from incoming data. The musical data is encoded through processes of feature extraction and preprocessed by a limiter, then routed to algorithms for sound transformation/generation, which produce a musical result which varies, according to process intensity, from morphologically independent sound material (e.g. sound materials generated by the granulators [Patch 5] and pitch transposers [Patch 3]) to a reinforcement of the starting instrumental materials (spatialisation), timbre modifications (ring modulation with distortion of negative sideband output signal [Patch 2], live-sampling subsequent reading of fragments varying rate [Patch 4]).
The instrumental performer must be aware of how the hyperinstrument system interprets his actions instant by instant, so to provide significance to his technical and interpretational choices, in connection with the performance of the live-electronics interpreter.
The mapping is directly applied to the traditional musical instrument, implemented by specific hardware designed by professor Matteo Ricchetti and realized by InfoMus- Lab of Genoa.

This paper studies the use of Gumowski-­Mira maps for sonic arts. Gumowski-­Mira maps are a set of chaotic systems that produce many organic orbits that resemble cells, flowers and other life forms. This has prompted mathema-ticians and eventually artists to study them. These maps carry a potential for use in the sonic arts, but until now such use is non-­existent. The paper describes two ways of using Gumowski-­Mira maps: for synthesis and spatialization. The synthesis approach, which runs in real-­time, takes the dynamical system output as the real and imaginary input to an inverse Fourier transformation, thus directly sonifying the algorithm. The spatialization approach uses the shapes of Gumowski-­Mira maps as shapes across the acoustic space, using the first 128 iterations of each map as audio particles. The shapes can change based on the maps' initial parameters. The maps are explored in live performance using Leap Motion and Cycling '74's MIRA for iPad as control interfaces of audio processing in SuperCollider. Examples are given in two works, Cells #1 and #2.

Morphons and Bions is a real-time, computer-generated work that uses white noise as the fundamental synthesis component combined with classical synthesis techniques. Random and stochastic processes control the synthesis parameters. This approach creates sounds evocative of living mechanisms. The sonic mechanisms can metamorphose between noisy, engine-like sounds to pitched, chatter-like sounds. This paper details the concepts behind the sound synthesis, as well as showing the random processes controlling the synthesis. A brief discussion follows of the philosophy of the work.