MUSIC ANALYSIS THROUGH VISUALIZATION

Jia Li

[email protected]
CCRMA, Stanford University
Shanghai Conservatory of Music

ABSTRACT
In this paper analytic visualizations are used to
selectively highlight salient musical features in four
modern compositions, focusing on micro or macro Early in the twentieth century, graphic techniques were
structures: from motivic pitch contour to large-scale form. used to illustrate analytic aspects of musical scores. It can
At a glance these visualizations allow a quick grasp of the be dated back at least to Alfred O. Lorenz (1924), who
structure and assist listeners to make connections between used graphs to show the modulatory scheme for Wagner's
local features and global trends. Textures obscured by Ring Cycle [3]. He developed a method of exploring
musical notation become more apparent when displayed Wagner’s musical texture as large, closed totalities. With
in a graphical format, such as broad registral shifts, the increasing popularity of computers in the 1970s,
polyphonic streaming, as well as interplay between music visualization becomes more accessible; either from
instruments. Pitch, timbre and voicing are plotted against pitch and rhythm extracted from traditional scores or
time to show large-scale patterns that would otherwise be sound-based visualizations such as waveforms and
difficult to recognize in a musical score or compare sonograms.
between different works. Music analysis through While sound-based visualizations are useful for
compositional data visualization not only makes sense to displaying performance information, this paper examines
musicians but also to non-musicians, facilitating compositional structures that are best visualized from
collaboration and exchange with artists and technicians in data. While the musical notation is optimal for conveying
other media. instructions to a performer, it is not easy to quickly read
larger structures from a score, which may otherwise be
obscured by crossing many pages or extra-musical
directions. Pitch, rhythm and timbre are the clearest
features that can be extracted from scores, so these
features are used in the following visualizations1. The
1. INTRODUCTION axes for the visualizations display pitch against time in a
At the turn of the nineteenth century, technological manner similar to piano rolls or MIDI sequencing
change occurred rapidly. Many inventions such as software, with time on the horizontal axis and pitch on
photography, cinematography, sound recording, the vertical axis. Color and shape are used in some of the
telephones and aviation were heralding the dawn of a new visualizations to highlight timbral or voice information.
age. Artists sought new ways to reflect the modernity of
the era, such as subjective perspective in Expressionism
and multiple perspectives in cubism. Modern
compositional tendencies reflected those of the visual arts,
with music becoming less tied to traditional tonality and
2. A MOTIF AND ITS DEVELOPMENT
musical notation. Music became more conceptual and The motif, a device widely used in compositions, always
experimental, with a unique form for every work. Some has been one of the basic musical elements. Impressive
composers even abandon traditional music notation, examples include the four-note motivic cell in
instead experimenting with graphical ones, such as John Beethoven’s Symphony No.5 Movement I, and the atonal
Cage's Concerto for Piano. This kind of breakdown in thematic motif of Aaron Copland’s Piano Variations.
genre gives rise to a need for a tangible form of the Development of a motif is a common method of
structure to assist in understanding. composition. Therefore, visualizing a piece’s motif and
subsequent development is the best way to show the
benefit of score visualization.
He Xuntian’s Scent Dance I (2009) for solo clarinet is
Copyright: © 2016 Jia Li et al. This is an open-access article dis-
constructed using a short motivic pattern that is varied
tributed under the terms of the Creative Commons Attribution License
and developed throughout the piece. Although the
3.0 Unported, which permits unrestricted use, distribution, and
reproduction in any medium, provided the original author and source
1
are credited. A good example of this sort of visualization can be seen in those
of the Music Animation Machine. (http://www.musanim.com).
composition is written for a single instrument, the measure 4 forms a mirror point for symmetry in the ten
monophonic line perceptually splits into polyphonic rhythms of the motif. The offset between pitch and
voices as the pitch range expands, bifurcating into two rhythm shows up in the graphical version as a distorted
interlocking forms of the motif. The piece's construction symmetry.
is that of a fractal, with the motive pattern-forming event
the large-scale double-arched structure of the entire
piece. The following visualizations are used to highlight
the motivic structures that He uses to construct
Scent Dance I.

2.1 The Motivic Pattern

Figure 1 The motivic pattern of Scent Dance I (Bars: 1-7)
Scent Dance I starts with a ten-note motif that is used to
generate structure throughout the piece. This section
illustrates how graphical notation can be used to analyze
and explain the construction of the piece more clearly
than can be done by looking at the traditional score.
Figure 1 shows the opening motif in traditional notation,
and a more graphical interpretation is shown in Figure 2,
which plots pitch against time. The individual notes are
shown as dots in the graphical form, with dashed lines Figure 2 The visualization of the motivic pattern (Bars: 1-
enhancing the motivic shape of a double arch, or a capital 7)
"M". Although the contour of the motif can be seen in the
original score, other notational features such as barlines,
dynamic markings, and slurs obscure the essence of the 2.2 The First Part with Motivic Development
pattern. Thus, the graphical display in Figure 2 better
highlights the importance of pitch contour. Exact pitches Based on these elements of pitch, rhythm and contour, the
are more difficult to read from the graphical version of motif is used to build the following phrase (Fig.3). Using
the pattern, but this helps to emphasize the contour rather the compositional data, we can plot a graph (Fig. 4) in
than the pitches. Besides this, it also helps to find out which each pattern is clearly visible, showing 14 patterns
how many notes and how many times of these notes have including the opening motif. A quick examination of the
been used in the motif, even in the whole piece. This graphical patterns shows that they are similar, but not
motif contains four pitch classes, each repeated a exactly the same. Some contain wider intervals, while
different number of times: 1 A, 2 Fs, 3 C-sharps and 4 Ds. others have longer durations inside each motivic variation.
At a glance, the motif seems to be symmetric, but in The ending note of a motivic cell is not only the end of
fact it is slightly asymmetric. The motif consists of two one cell, but also the beginning of the next. All of the
slightly offset symmetries between pitch and rhythm. The patterns connect to each other, constructing the first part
C-sharp at the start of measure 4 is a point of symmetry of the composition.
for the first nine notes of the motif, while the middle of

Figure 3 The first part with development on the motif (Bars: 1-34)

Figure 4 The visualization of the first part (Bars: 1-101)

2.3 The Second Part with Development but rather a complex perceptual stream. One stream is
outlined in Figure 6 with a blue line following the lowest
In second part of Scent Dance I, the development of the notes of the pattern. This lower outline forms the same
motif is more complex. Figure 5 gives the notation for the contour as the original motif (Figs. 1 & 2). Another
modified three-measure motif. On the graph in Figure 6, stream is outlines by the red line following the contour of
eight repetitions of this motif can be seen. The patterns in the upper pitches. This top stream can be viewed as an
this part seem difference from the original motif. inversion of the lower one, a bit like a “W”, or it can be
However, by carefully observing the shape of the new
viewed as an offset version of the “M” contour. These
motif, we can identify patterns that use the “M” shape
interlocking patterns and their trails are not otherwise
contour through the graph. The pattern changes into two
easy to recognize in the score due to the rhythmic activity
interlocking forms of the original motif, each in a
and linear presentation of the notation.
different register. It is no longer a monophonic melody,

Figure 5 The second part with development on the motif (Bars: 123-25)

Figure 6 The visualization of the second part (Bars: 123-142)

For the two layers of the pattern, we can see that the
2.4 The Fourth Part with Development basic shape of the original motif has been kept in the
lowest layer. The upper layer has more activity. At the
The technique of multi-layering in the second part end of the part, some notes change their register to the
continues within the fourth part, as the motif is further higher octave, giving the music more tension than before
overlaid and streamed. It is more complex than ever and leading to the climax of the piece.
before (Figs. 7 & 8). Besides the two layers, a mid-
ranged pedal note and grace note has been added.

Figure 7 The fourth part with development on the motif (Bars: 280-287)
 Figure 8 The visualization of the fourth part (Bars: 247-287)

active but also change register: the pitch ranges being

2.5 The Whole Structure
a3–f4, a#4–f5, and b4–g5 / b3–g4. The interval in these
In the visualization of the entire composition seen in three parts is about a sixth. In second part, the range is
Figure 9, we can clearly perceive the composers’ E3–Bb4; the fourth part has wider pitch range that is
thinking—the motif’s transformation and recurrence in E3–Bb6; in the coda, the range is C4–C6. From these
the original form, but in different registers, gives alternating narrow and wide pitch ranges, the whole
structure to the whole composition. The form of this structure of the piece has the same shape as a capital
piece is ABACA, plus a coda. In the whole graph of the “M”, mimicking the opening motif.
piece, we can see the first, third and fifth parts are more

Figure 9 He Xuntian: Scent Dance I

huge contrast of pitch range within a composition. This

3. VISUALIZING OTHER FEATURES piece is for harpsichord by György Ligeti, written in 1968.
The composer wanted to compose a piece “that would be
Data visualization not only shows the trail from the motif a paradoxically continuous sound, something
to the whole piece, but also shows many other aspects of like Atmosphères, but that would have to consist of
a work. innumerable thin slices of salami”. Ligeti’s earlier works
“A good graphic representation enables us to perceive used a technique known as micropolyphony. Rather than
many aspects of the musical ‘shape’ of a composition” a using large orchestra, this piece uses extremely rapid
[4]. Even though the graphs only show pitch and time, activity by a dense, rich stack of pitches with a solo
other higher-level features such as pitch range and texture instrument to create the impression of continuous sound
can be seen from micro to macro forms. and a monolithic image.
The graph also shows an attraction and repulsion of the
3.1 Pitch Range and Tendency two hands. In the outer parts of the composition the two
hands are entangled in the same registers, but in the
Viewing pitch usage in modern music on a graph can
middle part they drift away from each other.
amaze an audience. The visual patterns are more obvious
than in classical music. For example, Figure 10 shows a
 Figure 10 Ligeti: Continuum for Harpsichord
pitch, rhythm, register, timbre, and melodic contour is
3.2 Texture and Voice Webern’s innovation. His eagerness to redefine imitative
contrapuntal techniques, such as canon and fugue, can be
The texture generated by the two dimension of pitch and seen clearly using this graph.
time represent the inter-stream correlations of a piece. Different colors represent each voice in the score. Some
Through an overview of this texture, broad categories of imitative patterns can be seen counterpoint relationships,
monophony, homophony, polyphony, and heterophony such as the pink and green points, or the yellow and
can be read from the graphs. purple diamonds. These pairing have inversional
The following graph displays Webern’s Symphony, symmetry.
movement 1, part 1 (Fig. 11). Schematic organization of

Figure 11 Anton Webern: Symphony (Op. 21) Mv.1 Part1.

two pianos and two clarinets doubling both bass clarinet

and flute as well as piccolo.
4. MACRO STRUCTURE Figure 12 shows that the piece is organized into five
sections. We can see that the two pianos (shown in
In musical compositions, texture also delineates structural orange and light green) build a dense background, using
aspects. Data visualization is an ideal format for syncopated ostinatos. In first, third and fifth parts they
recognizing such structures. Let us examine the complete cover a wide register, while in second and fourth parts
musical formal structure of a piece, which can be they move to narrower higher section of the register. This
recognized in a glance from a visualization, showing both gives space to the cello (purple), viola (blue) and bass
the dissimilarity and similarity in the macro structure. clarinet (red) for their sustained lower tones. It also
The following example is Steve Reich's minimalist work shows that the division between sections is very smooth
"Octet" composed in 1979, later rescored as "Eight with some overlapping between the sections. These
Lines" in 1983. The ensemble consists of string quartet, characteristics show the compositional technology, such
as repetitive figures, slow harmonic rhythm and canons.
 Figure 12 Steve Reich: Eight Lines

6. REFERENCES
5. CONCLUSIONS [1] Brian Evans, “The graphic design of musical
structure: Scores for listeners: Incantation and
Above all, visualization graphs help in the recognition of mortuos plango, vivos voco.” In EMS-05:
motifs and their development through pitch contours, Electroacoustic Music Studies Conference 2005,
register, texture and the structure in both the pitch and 2005.
time domains. Graphs can make the audience more
aware of the overall structure in a composition rather than [2] Tobias Kunze and Heinrich Taube, “See–a
focusing on individual notes or phrases. They can find structured event editor: Visualizing compositional
the details on graph to enhance their listening experience. data in common music.” In ICMC’96: Proceedings
Performers can also discover new relationships between of the International Computer Music Conference
the surface features of notes and the deeper structures of 1996, pages 63–66, San Francisco, CA, USA, 1996.
form by using such graphs; otherwise, the link between ICMA.
notes and a composers’ overall intentions are difficult for [3] Alexander R. Brinkman and Martha R. Mesiti:
individual band or orchestra members to intuit from the Computer-Graphic Tools for Music Analysis, in
parts. Proceedings of the 1991 ICMC, pp. 53-56
(Montreal, Canada: McGill University, 1991)
We have seen that visualization can produce a beautiful [4] Waters, Simon and Tamas Ungvary, “The
picture for a piece. Listening to the music while viewing Sonogram: A Tool for Visual Documentation of
the graphs, a listener will become a bit like the conductor Musical Structure.” Proceeding of the 1990 ICMC,
who sees the whole score. People can foresee what will 159-63. San Francisco: ICMA.
happen in the next passage and remember the music [5] Pope, S.T., “Music Notations and the Representation
already passed through on the graph. Mapping time into of Musical Structure and Knowledge.” Perspectives
space also allows non-musician to understand better the of New Music 24(2): pp. 156-189. 1986
temporal aspects of the music. The artists and technicians [6] Huron, David. “Characterizing Musical Texture.”
can also use it as a sketch to interrelate with other media, Proceeding of the 1989 ICMC, pp. 131-34. San
such as dance, drama and animation. Francisco: ICMA.
For more example graphs, view the GraphMusic
channel on YouTube at [7] He Xuntian. Scent Dance I. Schott Music, Mainz,
www.youtube.com/channel/UCnF-gtWPS520-C4-bu6WvQg . Germany; 2009. ISMN: 979-0-001-19082-4