Skip to main content

Fabio Morreale

The University of Auckland, Creative arts and industries, Faculty Member

Followers

173

Following

114

Co-authors

4

Public Views

I am a Lecturer and Coordinator of Music Technology at the School of Music of the University of Auckland. Prior to that, I worked as a Postdoctoral Researcher at the Augmented Instruments Laboratory at Queen Mary University of London and at the Interaction Lab of the University of Trento. I completed a doctoral degree in Human-Computer Interaction (University of Trento) and a MCompSc and a BCompSc (University of Verona). The focus of my research is the critical analysis and innovation of music technologies, and particularly in the design of digital musical instruments (DMIs). I have put my theoretical research into practice by developing numerous musical instruments, interfaces, and installations .

less

Vrije Universiteit Brussel

Graduate Center of the City University of New York

Universidade Católica Portuguesa

Théodora Psychoyou

Université Paris-Sorbonne (Paris IV)

Alexander R Jensenius

University of Oslo

Kwantlen Polytechnic University

Kansas State University

Elisa Giaccardi

Politecnico di Milano

Arizona State University

KTH Royal Institute of Technology

InterestsView All (14)

Uploads

Papers by Fabio Morreale

Building a Maker Community Around an Open Hardware Platform

Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

My Library Has Just Been Obliterated": Producing New Norms of Use Via Software Update

Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

In-the-moment and Beyond: Combining Post-hoc and Real-Time Data for the Study of Audience Perception of Electronic Music Performance

Human-Computer Interaction - INTERACT 2017

This paper presents a methodology for the study of audience perception of live performances, usin... more This paper presents a methodology for the study of audience perception of live performances, using a combined approach of post-hoc and real-time data. We conducted a study that queried audience enjoyment and their perception of error in digital musical instrument (DMI) performance. We collected quantitative and qualitative data from the participants via paper survey after each performance and at the end of the concert, and during the performances spectators were invited to indicate moments of enjoyment and incidences of error using a two-button mobile app interface. This produced 58 paired post-hoc and real-time data sets for analysis. We demonstrate that real-time indication of error does not translate to reported non-enjoyment and post-hoc and real-time data sets are not necessarily consistent for each participant. In conclusion we make the case for a combined approach to audience studies in live performance contexts.

Musical Instruments for Novices: Comparing NIME, HCI and Crowdfunding Approaches

New Directions in Music and Human-Computer Interaction

Designing musical instruments to make performance accessible to novice musicians is a goal which ... more Designing musical instruments to make performance accessible to novice musicians is a goal which long predates digital technology. However, just in the space of the past 6 years, dozens of instrument designs have been introduced in various academic venues and in commercial crowdfunding campaigns. In this paper, we draw comparisons in design, evaluation and marketing across four domains: crowdfunding campaigns on Kickstarter and Indiegogo; the New Interfaces for Musical Expression (NIME) conference; conferences in human-computer interaction (HCI); and researchers creating accessible instruments for children and adults with disabilities. We observe striking differences in approach between commercial and academic projects, with less pronounced differences between each of the academic communities. The paper concludes with general reflections on the identity and purpose of instruments for novice musicians, with suggestions for future exploration.

Proceedings of the 9th International Conference on Digital and Interactive Arts

Musical Instruments for Novices: Comparing NIME, HCI and Crowdfunding Approaches

Springer Series on Cultural Computing, 2019

Designing musical instruments to make performance accessible to novice musicians is a goal which ... more Designing musical instruments to make performance accessible to novice musicians is a goal which long predates digital technology. However, just in the space of the past 6 years, dozens of instrument designs have been introduced in various academic venues and in commercial crowdfunding campaigns. In this paper, we draw comparisons in design, evaluation and marketing across four domains: crowdfunding campaigns on Kickstarter and Indiegogo; the New Interfaces for Musical Expression (NIME) conference; conferences in human-computer interaction (HCI); and researchers creating accessible instruments for children and adults with disabilities. We observe striking differences in approach between commercial and academic projects, with less pronounced differences between each of the academic communities. The paper concludes with general reflections on the identity and purpose of instruments for novice musicians, with suggestions for future exploration.

Designing a Musical Playground in the Kindergarten

Divesting Control to the Machine in Digital Art

Effect of Instrument Structure Alterations on Violin Performance

Frontiers in Psychology

Extensive training with a musical instrument results in the automatization of the bodily operatio... more Extensive training with a musical instrument results in the automatization of the bodily operations needed to manipulate the instrument: the performer no longer has to consciously think about the instrument while playing. The ability of the performer to automate operations on the instrument is due to sensorimotor mechanisms that can predict changes in the state of the body and the instrument in response to motor commands. But how strong are these mechanisms? To what extent can we alter the structure of the instrument before they disappear? We performed an exploratory study to understand whether and how sensorimotor predictions survive instrument modification. We asked seven professional violinists to perform repertoire pieces and sight-reading exercises on four different violins: their own, a cheap violin, a small violin, and a violin whose strings had been put on in reverse order. We performed a series of quantitative investigations on performance intonation and duration, and on bowing gestures and errors. The analysis revealed that participants struggled adapting to the altered instruments, suggesting that prediction mechanisms are a function of instrument configuration. In particular, the analysis of bowing errors, intonation, and of performance duration suggested that the performance with the reverse violin was much less fluent and precise than the performer's own instrument; the performance with the small violin was also substandard though to a lesser extent. We also observed that violinists were differently affected by instrument modifications, suggesting that the capability to adapt to a new instrument is highly personal.

Observations on visitors' behaviour in The Music Room

In this paper, we reflect on the evaluation of The Music Room, an interactive installation that w... more In this paper, we reflect on the evaluation of The Music Room, an interactive installation that was designed with the aim of allowing everybody to experience musical creativity. Couples entering The Music Room can compose original piano music by moving throughout the space. The metaphor of emotion mediates the interaction between people's behaviour and an algorithmic composer. Specifically, the distance between people and their average speed map the emotionality of music: distance influences the pleasantness of the music, while speed influences the intensity. This paper focuses on the off-line analysis of the player experience exploiting log-data and video analysis, as well as on subjective evaluations collected through interviews and questionnaires collected in two public exhibitions of the installation. We elaborate on this data to understand how players appropriated the original design idea and which factors contributed to fun and engagement. Reconsidering our design assumptions against behavioural data, we notice a number of unexpected behaviours. Elaborating on them, we contribute some reflection on the design of musical interfaces and the understanding of visitors' engagement.

Robin: An Algorithmic Composer for Interactive Scenarios

The purpose of this paper is to present Robin, an algorithmic composer specifically designed for ... more The purpose of this paper is to present Robin, an algorithmic composer specifically designed for interactive situations. Users can interact in real time with the algorithmic composition by means of control strategies based on emotions. This study aims at providing a system for automatic music generation to be applied to interactive systems for music creation targeted at non-musicians. Robin adopts a rulebased approach to compose original tonal music in classical piano style. The first practical application of Robin is The Music Room, an interactive installation which enables people to compose tonal music in pairs by communicating emotion expressed by moving throughout a room.

Observations on visitors’ behaviour in The Music Room

In this paper, we reflect on the evaluation of The Music Room, an interactive installation that w... more In this paper, we reflect on the evaluation of The Music Room, an interactive installation that was designed with the aim of allowing everybody to experience musical creativity. Couples entering The Music Room can compose original piano music by moving throughout the space. The metaphor of emotion mediates the interaction between people's behaviour and an algorithmic composer. Specifically, the distance between people and their average speed map the emotionality of music: distance influences the pleasantness of the music, while speed influences the intensity. This paper focuses on the off-line analysis of the player experience exploiting log-data and video analysis, as well as on subjective evaluations collected through interviews and questionnaires collected in two public exhibitions of the installation. We elaborate on this data to understand how players appropriated the original design idea and which factors contributed to fun and engagement. Reconsidering our design assumptions against behavioural data, we notice a number of unexpected behaviours. Elaborating on them, we contribute some reflection on the design of musical interfaces and the understanding of visitors' engagement.

The effect of expertise in evaluating emotions in music

This study investigates the role of expertise in the listener judgment of emotion in music. Previ... more This study investigates the role of expertise in the listener judgment of emotion in music. Previous studies suggest that the most important factors are mode and tempo, respectively influencing valence and arousal. The effect is stronger when the two parameters converge (major mode combined with fast tempo and vice versa), whereas tempo predominates when they do not converge. An open question is whether and how these judgments vary with the expertise of the listener. Our hypothesis is that non-experts will base their evaluation mainly on tempo, disregarding mode, which is more complex to be aware of. On the other hand, experts will take advantage of both sources of information. The experiment involved 40 participants. Experts were students who attended at least five years at music school. Non-experts had no formal musical training. Valence (positive vs. negative) and arousal (high vs. low) were manipulated independently in a 2*2 within-subjects design. Seven short piano pieces were composed and manipulated using the four conditions, for a total of 28 snippets. For each snippet, participants were asked to rate the values of the experienced valence and arousal on a seven-point scale. Results confirmed that for both types of listeners arousal was determined exclusively by tempo. Valence was primarily influenced by mode. Non-experts were also influenced by tempo for valence evaluation, while experts were not. As regards valence, mode is predominant in cases of divergent conditions but only for experts. Implications of these findings for the design of computing systems to allow nonmusicians to create music are discussed.

Robin: an algorithmic composer for interactive scenarios

The purpose of this paper is to present Robin, an algorithmic composer specifically designed for ... more The purpose of this paper is to present Robin, an algorithmic composer specifically designed for interactive situations. Users can interact in real time with the algorithmic composition by means of control strategies based on emotions. This study aims at providing a system for automatic music generation to be applied to interactive systems for music creation targeted at non-musicians. Robin adopts a rulebased approach to compose original tonal music in classical piano style. The first practical application of Robin is The Music Room, an interactive installation which enables people to compose tonal music in pairs by communicating emotion expressed by moving throughout a room.

CHI '13 Extended Abstracts on Human Factors in Computing Systems on - CHI EA '13, 2013

This paper presents The Music Room, an interactive installation where couples compose original mu... more This paper presents The Music Room, an interactive installation where couples compose original music. The music is generated by Robin, an automatic composition system, according to relative distance between the users and the speed of their own movements. Proximity maps the pleasantness of music, while speed maps its intensity. The Music Room was exhibited during the EU Researchers' Night in Trento, where it met with a strong interest by visitors.

Interfaces for Musical Creativity for Non-Experts

Current systems aiming at facilitating and extending musical expression typically rely on concept... more Current systems aiming at facilitating and extending musical expression typically rely on concepts that nonmusicians can hardly comprehend. To attract such users, different interaction metaphors laying outside of the musical domain need to be considered. This study is a step towards the definition of a new class of interfaces for musical expression that targets non-musicians and is centered on experience and performance of the users; we analyze the design requirements and present the current development state of our ...

The Influence of Coauthorship in the Interpretation of Multimodal Interfaces

This paper presents a model to codesign interpretively flexible artefacts. We present the case st... more This paper presents a model to codesign interpretively flexible artefacts. We present the case study of Beatfield, a multimodal system that allows users to control audiovisual material by means of tangible interaction. The design of the system was coauthored by individuals with different background and interests to encourage a range of difference interpretations. The capability of Beatfield to foster multiple interpretations was evaluated in a qualitative study with 21 participants. Elaborating on the outcome of this study, we present a new design model that can be used to stimulate heterogeneous interpretations of interactive artefacts.

Multisensory experiences for singers

Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing Adjunct - UbiComp '16, 2016

Supporting Singers with Tangible and Visual Feedback

Proceedings of the International Working Conference on Advanced Visual Interfaces - AVI '16, 2016

Building a Maker Community Around an Open Hardware Platform

Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

My Library Has Just Been Obliterated": Producing New Norms of Use Via Software Update

Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

In-the-moment and Beyond: Combining Post-hoc and Real-Time Data for the Study of Audience Perception of Electronic Music Performance

Human-Computer Interaction - INTERACT 2017

This paper presents a methodology for the study of audience perception of live performances, usin... more This paper presents a methodology for the study of audience perception of live performances, using a combined approach of post-hoc and real-time data. We conducted a study that queried audience enjoyment and their perception of error in digital musical instrument (DMI) performance. We collected quantitative and qualitative data from the participants via paper survey after each performance and at the end of the concert, and during the performances spectators were invited to indicate moments of enjoyment and incidences of error using a two-button mobile app interface. This produced 58 paired post-hoc and real-time data sets for analysis. We demonstrate that real-time indication of error does not translate to reported non-enjoyment and post-hoc and real-time data sets are not necessarily consistent for each participant. In conclusion we make the case for a combined approach to audience studies in live performance contexts.

Musical Instruments for Novices: Comparing NIME, HCI and Crowdfunding Approaches

New Directions in Music and Human-Computer Interaction

Designing musical instruments to make performance accessible to novice musicians is a goal which ... more Designing musical instruments to make performance accessible to novice musicians is a goal which long predates digital technology. However, just in the space of the past 6 years, dozens of instrument designs have been introduced in various academic venues and in commercial crowdfunding campaigns. In this paper, we draw comparisons in design, evaluation and marketing across four domains: crowdfunding campaigns on Kickstarter and Indiegogo; the New Interfaces for Musical Expression (NIME) conference; conferences in human-computer interaction (HCI); and researchers creating accessible instruments for children and adults with disabilities. We observe striking differences in approach between commercial and academic projects, with less pronounced differences between each of the academic communities. The paper concludes with general reflections on the identity and purpose of instruments for novice musicians, with suggestions for future exploration.

Proceedings of the 9th International Conference on Digital and Interactive Arts

Musical Instruments for Novices: Comparing NIME, HCI and Crowdfunding Approaches

Springer Series on Cultural Computing, 2019

Designing musical instruments to make performance accessible to novice musicians is a goal which ... more Designing musical instruments to make performance accessible to novice musicians is a goal which long predates digital technology. However, just in the space of the past 6 years, dozens of instrument designs have been introduced in various academic venues and in commercial crowdfunding campaigns. In this paper, we draw comparisons in design, evaluation and marketing across four domains: crowdfunding campaigns on Kickstarter and Indiegogo; the New Interfaces for Musical Expression (NIME) conference; conferences in human-computer interaction (HCI); and researchers creating accessible instruments for children and adults with disabilities. We observe striking differences in approach between commercial and academic projects, with less pronounced differences between each of the academic communities. The paper concludes with general reflections on the identity and purpose of instruments for novice musicians, with suggestions for future exploration.

Designing a Musical Playground in the Kindergarten

Divesting Control to the Machine in Digital Art

Effect of Instrument Structure Alterations on Violin Performance

Frontiers in Psychology

Extensive training with a musical instrument results in the automatization of the bodily operatio... more Extensive training with a musical instrument results in the automatization of the bodily operations needed to manipulate the instrument: the performer no longer has to consciously think about the instrument while playing. The ability of the performer to automate operations on the instrument is due to sensorimotor mechanisms that can predict changes in the state of the body and the instrument in response to motor commands. But how strong are these mechanisms? To what extent can we alter the structure of the instrument before they disappear? We performed an exploratory study to understand whether and how sensorimotor predictions survive instrument modification. We asked seven professional violinists to perform repertoire pieces and sight-reading exercises on four different violins: their own, a cheap violin, a small violin, and a violin whose strings had been put on in reverse order. We performed a series of quantitative investigations on performance intonation and duration, and on bowing gestures and errors. The analysis revealed that participants struggled adapting to the altered instruments, suggesting that prediction mechanisms are a function of instrument configuration. In particular, the analysis of bowing errors, intonation, and of performance duration suggested that the performance with the reverse violin was much less fluent and precise than the performer's own instrument; the performance with the small violin was also substandard though to a lesser extent. We also observed that violinists were differently affected by instrument modifications, suggesting that the capability to adapt to a new instrument is highly personal.

Observations on visitors' behaviour in The Music Room

In this paper, we reflect on the evaluation of The Music Room, an interactive installation that w... more In this paper, we reflect on the evaluation of The Music Room, an interactive installation that was designed with the aim of allowing everybody to experience musical creativity. Couples entering The Music Room can compose original piano music by moving throughout the space. The metaphor of emotion mediates the interaction between people's behaviour and an algorithmic composer. Specifically, the distance between people and their average speed map the emotionality of music: distance influences the pleasantness of the music, while speed influences the intensity. This paper focuses on the off-line analysis of the player experience exploiting log-data and video analysis, as well as on subjective evaluations collected through interviews and questionnaires collected in two public exhibitions of the installation. We elaborate on this data to understand how players appropriated the original design idea and which factors contributed to fun and engagement. Reconsidering our design assumptions against behavioural data, we notice a number of unexpected behaviours. Elaborating on them, we contribute some reflection on the design of musical interfaces and the understanding of visitors' engagement.

Robin: An Algorithmic Composer for Interactive Scenarios

The purpose of this paper is to present Robin, an algorithmic composer specifically designed for ... more The purpose of this paper is to present Robin, an algorithmic composer specifically designed for interactive situations. Users can interact in real time with the algorithmic composition by means of control strategies based on emotions. This study aims at providing a system for automatic music generation to be applied to interactive systems for music creation targeted at non-musicians. Robin adopts a rulebased approach to compose original tonal music in classical piano style. The first practical application of Robin is The Music Room, an interactive installation which enables people to compose tonal music in pairs by communicating emotion expressed by moving throughout a room.

Observations on visitors’ behaviour in The Music Room

In this paper, we reflect on the evaluation of The Music Room, an interactive installation that w... more In this paper, we reflect on the evaluation of The Music Room, an interactive installation that was designed with the aim of allowing everybody to experience musical creativity. Couples entering The Music Room can compose original piano music by moving throughout the space. The metaphor of emotion mediates the interaction between people's behaviour and an algorithmic composer. Specifically, the distance between people and their average speed map the emotionality of music: distance influences the pleasantness of the music, while speed influences the intensity. This paper focuses on the off-line analysis of the player experience exploiting log-data and video analysis, as well as on subjective evaluations collected through interviews and questionnaires collected in two public exhibitions of the installation. We elaborate on this data to understand how players appropriated the original design idea and which factors contributed to fun and engagement. Reconsidering our design assumptions against behavioural data, we notice a number of unexpected behaviours. Elaborating on them, we contribute some reflection on the design of musical interfaces and the understanding of visitors' engagement.

The effect of expertise in evaluating emotions in music

This study investigates the role of expertise in the listener judgment of emotion in music. Previ... more This study investigates the role of expertise in the listener judgment of emotion in music. Previous studies suggest that the most important factors are mode and tempo, respectively influencing valence and arousal. The effect is stronger when the two parameters converge (major mode combined with fast tempo and vice versa), whereas tempo predominates when they do not converge. An open question is whether and how these judgments vary with the expertise of the listener. Our hypothesis is that non-experts will base their evaluation mainly on tempo, disregarding mode, which is more complex to be aware of. On the other hand, experts will take advantage of both sources of information. The experiment involved 40 participants. Experts were students who attended at least five years at music school. Non-experts had no formal musical training. Valence (positive vs. negative) and arousal (high vs. low) were manipulated independently in a 2*2 within-subjects design. Seven short piano pieces were composed and manipulated using the four conditions, for a total of 28 snippets. For each snippet, participants were asked to rate the values of the experienced valence and arousal on a seven-point scale. Results confirmed that for both types of listeners arousal was determined exclusively by tempo. Valence was primarily influenced by mode. Non-experts were also influenced by tempo for valence evaluation, while experts were not. As regards valence, mode is predominant in cases of divergent conditions but only for experts. Implications of these findings for the design of computing systems to allow nonmusicians to create music are discussed.

Robin: an algorithmic composer for interactive scenarios

The purpose of this paper is to present Robin, an algorithmic composer specifically designed for ... more The purpose of this paper is to present Robin, an algorithmic composer specifically designed for interactive situations. Users can interact in real time with the algorithmic composition by means of control strategies based on emotions. This study aims at providing a system for automatic music generation to be applied to interactive systems for music creation targeted at non-musicians. Robin adopts a rulebased approach to compose original tonal music in classical piano style. The first practical application of Robin is The Music Room, an interactive installation which enables people to compose tonal music in pairs by communicating emotion expressed by moving throughout a room.

CHI '13 Extended Abstracts on Human Factors in Computing Systems on - CHI EA '13, 2013

This paper presents The Music Room, an interactive installation where couples compose original mu... more This paper presents The Music Room, an interactive installation where couples compose original music. The music is generated by Robin, an automatic composition system, according to relative distance between the users and the speed of their own movements. Proximity maps the pleasantness of music, while speed maps its intensity. The Music Room was exhibited during the EU Researchers' Night in Trento, where it met with a strong interest by visitors.

Interfaces for Musical Creativity for Non-Experts

Current systems aiming at facilitating and extending musical expression typically rely on concept... more Current systems aiming at facilitating and extending musical expression typically rely on concepts that nonmusicians can hardly comprehend. To attract such users, different interaction metaphors laying outside of the musical domain need to be considered. This study is a step towards the definition of a new class of interfaces for musical expression that targets non-musicians and is centered on experience and performance of the users; we analyze the design requirements and present the current development state of our ...

The Influence of Coauthorship in the Interpretation of Multimodal Interfaces

This paper presents a model to codesign interpretively flexible artefacts. We present the case st... more This paper presents a model to codesign interpretively flexible artefacts. We present the case study of Beatfield, a multimodal system that allows users to control audiovisual material by means of tangible interaction. The design of the system was coauthored by individuals with different background and interests to encourage a range of difference interpretations. The capability of Beatfield to foster multiple interpretations was evaluated in a qualitative study with 21 participants. Elaborating on the outcome of this study, we present a new design model that can be used to stimulate heterogeneous interpretations of interactive artefacts.

Multisensory experiences for singers

Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing Adjunct - UbiComp '16, 2016

Supporting Singers with Tangible and Visual Feedback

Proceedings of the International Working Conference on Advanced Visual Interfaces - AVI '16, 2016

"The finer the musician, the smaller the details" : NIMEcraft under the microscope

by Jack Armitage and Fabio Morreale

Many digital musical instrument design frameworks have been proposed that are well suited for ana... more Many digital musical instrument design frameworks have been proposed that are well suited for analysis and comparison. However, not all provide applicable design suggestions, especially where subtle, important details are concerned. Using traditional lutherie as a model, we conducted a series of interviews to explore how violin makers " go beyond the obvious " , and how players perceive and describe subtle details of instrumental quality. We find that lutherie frameworks provide clear design methods, but are not enough to make a fine violin. Success comes after acquiring sufficient tacit knowledge, which enables detailed craft through subjective , empirical methods. Testing instruments for subtle qualities was suggested to be a different skill to playing. Whilst players are able to identify some specific details about instrumental quality by comparison, these are often not actionable, and important aspects of " sound and feeling " are much more difficult to describe. In the DMI domain, we introduce the term NIMEcraft to describe subtle differences between otherwise identical instruments and their underlying design processes, and consider how to improve the dissemination of NIMEcraft.