On Listening as Analysis: The Selfie Orchestra Project

Owen Chapman, Peter Sinclair

Canadian Theatre Review, Volume 184, Fall 2020, pp. 27-32 (Article)

Published by University of Toronto Press

For additional information about this article

[ Access provided at 8 Jan 2023 00:08 GMT from Carleton University Library ]
 On Listening as Analysis | FEATURES

On Listening as Analysis:
The Selfie Orchestra Project
by Owen Chapman and Peter Sinclair

Listening is both a way of navigating the world and a means of This method suggests that listening provides a way to organize
making sense out of what we encounter. These reflections stem from knowledge by situating it in embodied experience. The question that
our ongoing development of an interactive sound art piece entitled emerges for Selfie Orchestra is “Whose bodies are more significant
Selfie Orchestra.1 We begin with a brief description of this project, in such meaning making: the artist’s or the audience’s?” From this,
highlighting our attempts to provoke expanded forms of listening, we move to a brief reflection on John Cage’s 4’33” (1952) and Max
leveraging new technologies of data mapping, visualization, and Neuhaus’s Listen (1966) as precedent artworks that make space for
sonification in order to reveal hidden rhythms and relationships attentive listening. We then contrast the open-ended interactivity
among bodies in built environments. promoted by these works with the more constrained forms we have
We then turn to a theoretical examination of listening as a been developing with Selfie Orchestra.
method for reflection and research in the social sciences, with specific While ‘analysis’ is most often understood as establishing the
reference to Henri Lefebvre’s evocative concept of ‘rhythmanalysis.’ structure of something by breaking it down to its constitutive

Participants inside the installation.

doi:10.3138/ctr.184.005 ctr 184 fall 2020 27

parts, we conclude with some thoughts on how listening suggests is produced with a technique known as photogrammetry, where
a different conception, one that is closer to the ancient Greek root thousands of photos are taken of the surrounding environment and
of the term that means a form of ‘loosening.’ then rendered into a 3D model. The result of this rendering was, in
our case, both realistic and warped due to the many glitches that
Description of the Project occurred because of the sheer size of the area we wanted to map.
The Selfie Orchestra involves an immersive visual and auditory La Friche la Belle de Mai in Marseille, where the piece was created,
environment that is constructed, in part, by the spectating public. is a very large repurposed tobacco processing centre from the
Participants begin by downloading our custom mobile application 1860s with many levels, staircases, hidden corners, and concrete
onto their cell phones and recording a 10-second selfie video pillars. Though smaller spaces can be rendered quite realistically
(speaking their name, singing a short musical phrase, describing with photogrammetry, with such a vast, open-air space our efforts
the environment around them, etc.) and then closing the record produced a distinctly dystopian rendition—where cars melt into
function, thereby uploading their selfie video to our database the ground beneath them, for instance—as our cameras were not
before heading out for a short walk (approximately 15 minutes) able to ‘see’ their undersides from our upper-level vantage points.
around the installation location. The app records the time and While it is possible to imagine a complete Borgesian mapping of
GPS (latitude and longitude) coordinates of this walk and uploads La Friche, where each three-dimensional detail and angle has been
this information into our database. Upon their return, participants captured in digital high resolution, this is certainly not what we did,
enter the installation itself, which is an immersive video projection and the technical complexity of achieving such a rendering keeps
ringed by audio speakers in a surround sound configuration. this goal in the realm of speculative fiction (for the time being).
Inside the installation, participants are invited to find their own The photogrammetry was combined with a digital architectural
selfie ‘avatars,’ which look like giant marbles. There can be many model that has a significant effect on the aesthetics and enables a
such marbles rolling through the simulation simultaneously, wireframe inner structure of the vast buildings (pillars, levels, etc.).
depending on the number of participants. The surface of each The sound of the installation is created through a form of
sphere features an animated GIF video corresponding to an granular synthesis applied to the original selfie recordings. Gran­
individual participant’s upload, complete with sound. The avatars ular synthesis involves layering randomized snippets (or grains)
move along separate paths in the 3D simulation that correspond to of sound, eliminating the original form while maintaining and
each participant’s walk, looping back to their starting points once enriching its timbre. For example, one could start with the first
finished. Using a software called Unity 3d, the visual simulation phoneme of a recorded word, and then randomly extract and play

Simulation of La Friche children’s park with two selfie avatars.

28 ctr 184 fall 2020 doi:10.3138/ctr.184.005

back even smaller sonic particles from this selection, scrolling or levels as the avatars rolled through the vast area of La Friche. How-
‘scrubbing’ through different zones of the original sound. This ever, certain participants chose to remain close to the installation
maintains the sound’s ‘spectral signature’ or frequency content, perimeter during their walks and therefore ended up (unwittingly)
while undoing its relationship to linear time or amplitude dominating the sound field in the installation. Moreover, mobile
envelopes. As each new upload is entered into the system, the GPS tracking is heavily affected by user-controlled settings on in-
sound of the video becomes available for processing through the dividual phones, as well as different levels of hardware sensitivity
granular synthesis engine. The resulting signals can then be further to materials like concrete and other signal blockers. This meant
manipulated through amplitude control and the application of that some participants’ avatars appeared many kilometres away
other effects like EQ filters, reverbs, or delays before being mixed from La Friche in our simulation, rendering them virtually invis-
into the larger whole. This is achieved through a connected series ible and silent. In the end, we moved away from this mixing choice
of digital interfaces that allow us to react spontaneously to what the (amplitude = proximity) and switched to a model we will describe
system provides in any given moment. In this, we (Chapman and later, which allowed participants to manipulate virtual camera and
Sinclair) operate somewhat like DJs or live electronic musicians, microphone positions in the simulation through videogame con-
modulating and mixing diverse sound sources in real-time. For trollers. For us as performers, embracing chance and adapting to
certain elements we tried to develop automated responses based spontaneous shifts in musical flow produced an experience similar
on the data generated by the original walks, with mixed results. to that of a DJ mixing a collection of records they don’t know.
In addition, we used convolution reverb modelling techniques to Though exciting serendipitous moments occur, sonic ‘train wrecks’
map acoustic resonances within different areas of La Friche and were also all too common, with rhythms, volumes, frequencies, vo-
included these zones in the simulation. This allowed our mixes to cal elements, etc., mixed into accidental combinations. While we
feature actual reverb and echo parameters that mimicked the local as artists are drawn to noise and sonic intensity, we also hoped to
built environment but was, for the most part, only noticed by feature quieter, more subtle audio relationships in the work, and
audience members when it was drawn to their attention. this proved tricky. At times, it felt like the audience, after their
One relationship we were able to develop fairly easily was am- walks, were invited to simply bear witness to our attempts to make
plitude versus proximity. Initially, we coded the work to decrease the experimental electronic music out of their contributions.2
volume of each selfie avatar according to their relative distance from In order to enhance the interactivity in the work, we opted
the installation location—the further away, the softer they would to offer participants videogame controllers inside the installation.
be. We assumed this would cause natural ebbs and flows in sound This allowed them to interact with the visualization, zooming in

Simulation of La Friche skate park with selfie avatar in right upper corner.
Photo by Pierre Gondard

doi:10.3138/ctr.184.005 ctr 184 fall 2020 29

and out of different areas of the on-screen projections and thereby our bodies or eardrums, at least not consciously? What of the
controlling, to some degree, the relative amplitude and audio- kinaesthetic rhythms, the micro-timed and polyphonic inputs and
mix position of the different components. Imagine a first-person outputs, the choreographic intricacies of moving through built
videogame where moving away from a location causes the sounds environments that we perform simply by walking in the city? These
of that location to become more distant, or where turning from can be measured, but can they be mapped? How does one render
left to right results in a corresponding shift in the sonic field. This them ‘listenable,’ and to what end? How much artistic ‘authorship’
enhanced participant involvement in the work, but also served to can or should there be in this process? To put it another way, how
further undermine the indexicality of relationships between what does one present ‘listening to the world’ in the embodied way that
happened outside the installation and what happened within. Lefebvre wants? Is there a continuum between (social) scientific
and artistic methods for such research? We all want to avoid the
Listening as Analysis vexing predicament of the positivist, namely “rendering oneself
Lefebvre’s theory of rhythmanalysis understands human subjects passive, forgetting one’s knowledge, in order to re-present it in its
as nodal points within complex networks that define them, but entirety in the interpretation” (Lefebvre 19). But how, precisely,
which they also help to shape. In his portrait of the progressive are we to get past ourselves and our own limited capacities, our
researcher who treats urban space as a social production, he limited perspectives of analysis?
outlines an expanded conception of listening. As an alternative to
the psychoanalyst, the “rhythmanalyst,” he states,
What of the kinaesthetic rhythms, the
will be attentive, but not only to the words or pieces of
information, the confessions and confidences of a partner or micro-timed and polyphonic inputs and
client. He will listen to the world, and above all to what are outputs, the choreographic intricacies of
disdainfully called noises, which are said without meaning, and
to murmurs [rumeurs], full of meaning—and finally he will moving through built environments that
listen to silences….The psychoanalyst encounters difficulties
when he listens out. How is he to orientate his knowledge,
we perform simply by walking in the city?
forget his past, make himself anew and passive, and not
interpret prematurely? The rhythmanalyst will not have Our aim with the Selfie Orchestra project is to enable such
these methodological obligations: rendering oneself passive, engagement—to allow participants to exceed their own perspectives
forgetting one’s knowledge, in order to re-present it in its
in order to learn new things about their everyday sonic world and
entirety in the interpretation. He listens—and first to his body;
the rhythmic mechanisms they use to navigate it. Many of these
he learns rhythm from it, in order consequently to appreciate
external rhythms. His body serves him as a metronome. (19) rhythms cannot be listened to with the naked ear, so to speak, and
can only be revealed or ‘unconcealed’ (Heidegger) through different
Starting with the personal body as the locus of interpretive forms of technology. This is achieved through a complex system
experience, Lefebvre’s method of rhythmanalysis provides a means that records and tracks these rhythms, these urban choreographic
to identify and engage with contingent, yet recurrent loops that improvisations, rendering them audible through different abstract
manifest in the everyday life of the researcher, and about which processes of sonification (Sinclair). Selfie Orchestra uses data drawn
one can speak with a certain amount of subjective authority. An from participants’ déambulations (wanderings) in a particular site
expanded conception of listening that considers it as more than a or space and transforms these streams into different individual
sense for ‘receiving’ is key to this move, and it provides a vital way soundtracks that are mixed into an improvised, interactive, multi-
to ‘situate’ one’s knowledge (Haraway) about an object, subject, channel composition. It’s exciting to witness, fun to contribute to—
network, etc. Attentive listening, it is suggested, provides a way loud, brightly coloured, and volatile—but is it rhythmanalysis?
to access deeper knowledge about the world around us—how it is ‘Listening’ as a category of human activity provides a potent
ordered and how we order it and ourselves in the process. metaphor for many different types of knowing and sensing, and the
concept has been the focus of many scholarly/artistic investigations
(Bull; Chion; Crawford; Lefebvre; McCartney; Oliveros; Oram;
Rhythmanalysis provides a means to Robinson; Sterne; Westerkamp; among others). It is beyond the
identify and engage with contingent, scope of this article to parse the details of this rich body of literature.
Instead, we focus on the specific notion, common to most of these
yet recurrent loops that manifest in perspectives, that listening is not a passive or neutral activity: how
the everyday life of the researcher, we focus our attention, the agency we may or may not have in
and about which one can speak with a choosing that focus, and what we focus on are important factors
that shape our experience of the world. In fact, one could argue that
certain amount of subjective authority. listening as a concept is as much about this focusing of attention as
it is about receiving or sensing energy through vibration.
But how does one do this? How do we listen to our body John Cage’s composition 4’33” (1952) involves a performance
and its intersections with the other rhythms that make up modern in three movements where nothing is actually played by the
urban life? Soundscape composer Hildegard Westerkamp suggests interpreting musicians. Through the radical act of “composing”
we start by listening to our own breath, and then move outwards through silence, it calls attention to a simple fact: that listening,
(Westerkamp). But what of the rhythms that do not vibrate in every moment, is an active engagement with the world. Our

30 ctr 184 fall 2020 doi:10.3138/ctr.184.005

ears are always on, as is the body’s capacity to feel vibrations. In loosely guided or released (in the sense of unleashed). This is
negotiating our immediate, “lived temporality” (Lefebvre 21), we less about emphasizing the intervention of the producer, or
tune to different elements of the manifold aural impulses around analyst, and more about encouraging new or deeper forms of
us (and within) on an “as-needed” basis. attending to the world for one’s audience through listening. The
Max Neuhaus took inspiration from Cage’s work when he idea is to provoke personal, yet shared experiences. Listening is
began his series of sound art pieces Listen in 1966. He notes that partial, based in the dynamics of attention (foregrounding and
the problem with a work like 4’33” was that most “members of backgrounding), and this is perhaps its most intriguing quality
the audience seemed more impressed with the scandal of ‘ordin- and greatest strength as a form of analysis. Holding to such a
ary’ sounds placed in a ‘sacred’ place [i.e., the concert hall] than method promotes an undeniably different paradigm from visual
with the sounds themselves, and few were able to carry the experi- notions like ‘looking closely’ at something. An aural paradigm
ence over to a new perspective on the sounds of their daily lives” treats analysis less as an atomizing tool for interpretation based in
(Neuhaus, “Walks”).3 To remedy this, Neuhaus’s work brought breaking things down into their constitutive parts, and more in its
audience members outside: rubber stamping the word “Listen” ancient Greek conception as a type of loosening that can resolve a
onto their hands before taking them to different locations in the problem. The Oxford English Dictionary outlines the etymology
city for improvised listening experiences mixed with percussion of ‘analysis’ as passing from the ancient Greek term ἀνάλυσις or
performances he would provide. As with Listen, the audience for the “action of loosing or releasing” into its more contemporary
Selfie Orchestra is mobile. The initial selfie video acts in a way that definition related to methods that employ “deductive reasoning to
is similar to the rubber stamp—prompting new forms of listen- establish the nature, structure, and essential features of something,
ing-based interaction with the urban environment. starting from its constituent parts” (Oxford English Dictionary).
Aaron Smuts asserts that something is interactive for an indi- However, this resonance, this echo—the way that ‘analysis’ can
vidual if it responds in a way that is neither (1) radically random also mean a loosening up of interpretation, an openness towards
nor (2) completely controllable (54). Smuts claims that 4’33” is alternative conceptions, a relativizing of artistic intent by filtering
not an interactive work, since what it produces is radically random it through ‘lived temporalities’ via the meaningful interaction
given that it is composed of whatever sounds happen to occur dur- of one’s audience—this is the line of open-ended thinking that
ing the performance’s 4 minute and 33 second duration. However,
Cage’s work only stands as non-interactive if one considers listen-
ing as ‘passive.’ If one thinks of the active, multilayered and unique
listener experiences that a performance of 4’33” opens up for every
member of an audience (given that we won’t all attend to the same
elements while listening, even if many are shared), then one recog-
nizes that what the work provides is not radically random nor com-
pletely controllable. It is both, and neither. The audience deter-
mines their own content through the situation the artist has set up.
However, in Cage’s work, the audience is immobile, which restricts
their listening experience. With Selfie Orchestra, the experience is
both physically and aurally active (as with Listen). Nevertheless,
we had hoped to achieve a greater level of audience autonomy in
the  production of indexical sonic events—introducing signals
in the installation that were the direct result of the unique rhythms
that people encountered and generated while walking.
The tools at our disposal—i.e., state-of-the-art mobile software
and hardware for developing networked algorithmic relationships
between data input and sonic output—required very active forms
of artistic intervention in order to function together in a cohesive
whole. Directing the final outcome with a much higher level of
‘Wizard of Oz’ control than we had originally intended, the question
emerged—to what degree can a remixing of urban pedestrian
rhythms produce a readable mapping for participants? Our aim is
to highlight sound’s relationality, and yet putting the work into a
performance context troubles this effort and is prey to the power
structures and social dynamics that operate in such arenas. Our
work reveals problems with the dream of rhythmanalysis—too
much emphasis is placed on the expertise of the analyst, which in
our case is linked to technological wrangling. The artists are in clear
control of the final outcome, and it is difficult for participants to
perceive how their contributions affect the whole.
What is reinforced in this discussion, however, is that while Owen Chapman and Peter Sinclair mixing audio.
listening as analysis cannot easily be made objective, it can be Photo by Pierre Gondard

doi:10.3138/ctr.184.005 ctr 184 fall 2020 31

we will be carrying forward into our next phase of research and Crawford, Kate. “Following You: Disciplines of Listening in Social
creation with the work. Learning from our experiences so far, our Media.” Continuum: Journal of Media & Cultural Studies, vol. 23,
next step will be to develop a workshop approach to the piece, no. 4, 2009, pp. 525–535. doi.org/10.1080/10304310903003270.
where participants are given more time and autonomy in making Haraway, Donna. “Situated Knowledges: The Science Question in
Feminism and the Privilege of Partial Perspective.” Feminist Studies,
contributions to the system and generating their components of
vol. 14, no. 3, 1988, pp. 575–599. doi.org/10.2307/3178066.
the analysis. Our hope is that a loosening of artistic control will Heidegger, Martin. The Question Concerning Technology and Other
bring us closer to our original goals of promoting expanded forms Essays. Translated by William Lovitt, Harper and Row, 1977.
of listening that carry over into the everyday lives of our audience. Lefebvre, Henri. Rhythmanalysis: Space, Time and Everyday Life.
Translated by Gerald Moore and Stuart Elden, Bloomsbury
Academic, 2013.
Notes McCartney, Andra. “‘How Am I to Listen to You’: Soundwalking,
1 The project was most recently showcased during the “Les Musiques” Intimacy and Improvised Listening.” Negotiated Moments:
festival in Marseille May 2019, presented by the gmem—Centre Improvisation, Sound, and Subjectivity. Edited by Ellen Waterman
National de Création Musicale—Marseille and in partnership with and Gillian Siddall, Duke UP, 2016, pp. 37–54.
La Friche la Belle de Mai (Marseille, FR) and co-presented by Sec- Neuhaus, Max. “Walks.” web.archive.org/web/20090218020122/http://
onde Nature (Marseille). Artistic co-creators: Owen Chapman and max-neuhaus.info/soundworks/vectors/walks/.
Peter Sinclair, Unity 3d development: Jonathan Tanant, Produc- Oliveros, Pauline. “Quantum Improvisation: The Cybernetic Presence.”
tion Assistant: Mélissa Mathieu. Co-produced by Seconde Nature Sounding the Margins: Collected Writings 1992–2009. Edited by
(Marseille, FR), La Chambre Blanche (Québec, CAN), gmem- Lawton Hall, Deep Listening Publications, 2010.
CNCM-marseille, thecamp (Aix-en-Provence). This participative Oram, Daphne. An Individual Note of Music, Sound and Electronics.
creation, as part of the European Future DiverCities project, is sup- Galliard, 1972.
ported by the European Commission’s Creative Europe program, Oxford English Dictionary. “Analysis, n.” OED Online. Oxford UP Online.
the Provence-Alpes-Côte-d’Azur Regional Council, the Ministry of Robinson, Dylan. Hungry Listening: Resonant Theory for Indigenous
Foreign Affairs (FR), the Ministry of Culture (FR), the Bouches-du- Sound Studies. U of Minnesota P, 2020.
Rhône Departmental Council, the Québec Arts Council, the Can- Sinclair, Peter. “Inside Zeno’s Arrow: Mobile Captation and
ada Council for the Arts, the City of Aix-en-Provence, the City of Sonification.” Wi: Journal of Mobile Media, vol. 9, no. 2, 2015.
Québec and Aix-Marseille-Provence Métropole. Sterne, Jonathan. MP3: The Meaning of a Format. Duke UP, 2012.
Westerkamp, Hildegard. “Soundwalking.” Autumn Leaves, Sound
2 Please refer to nujus.net/~petesinc/wiki/?page=SelfieOrchestra for and the Environment in Artistic Practice. Edited by Angus Carlyle,
audio, image, and video examples. Double Entendre, 2007, p. 49.
3 Neuhaus passed away in 2009. At the time of writing, Neuhaus’s
personal website, max-neuhaus.info, has gone offline. To our
knowledge the cited source has not been published elsewhere. About the Authors
Owen Chapman is an Associate Professor in Sound Production and
Scholarship in the department of Communication Studies at Concordia
Works Cited University. His research-creation involves work on the place of sound in
Borges, Jorge Luis. A Universal History of Infamy. Translated by everyday life. His audio productions have been featured internationally in
Norman Thomas di Giovanni, Penguin Books, 1970. media workshops, site-specific installations, and video soundtracks as well
Bull, Michael. Sound Moves: IPod Culture and Urban Experience. as solo and group performances.
Routledge. 2007. Peter Sinclair is a Sound Artist, researcher in sound art practice at
Chion, Michel. “The Three Listening Modes.” Audio-Vision: Sound on UMR PRISM CNRS and professor of Art at ESA-Aix. He is internation-
Screen. Edited by Claudia Gorbman, Columbia UP, 1994, pp. 25–34. ally renowned for his sound installations as well as for his work on collab-
orative and participative environments. His artistic experimentations use
networked games, mobile media and data sonification.

32 ctr 184 fall 2020 doi:10.3138/ctr.184.005