Sonic interaction design: sound, information and experience

CHI 2008 Proceedings · Workshops April 5-10, 2008 · Florence, Italy Sonic Interaction Design: Sound, Information and Experience Davide Rocchesso Gerhard Eckel Abstract IUAV University of Venice IEM - Graz, Austria Venezia, Italy [email protected] Sonic Interaction Design (SID) is an emerging field that is positioned at the intersection of auditory display, ubiquitous computing, interaction design, and interactive arts. SID can be used to describe practice and inquiry into any of various roles that sound may play in the interaction loop between users and artifacts, services, or environments, in applications that range from the critical functionality of an alarm, to the artistic significance of a musical creation. This field is devoted to the privileged role the auditory channel can assume in exploiting the convergence of computing, communication, and interactive technologies. An overemphasis on visual displays has constrained the development of interactive systems that are capable of making more appropriate use of the auditory modality. Today the ubiquity of computing and communication resources allows us to think about sounds in a proactive way. This workshop puts a spotlight on such issues in the context of the emerging domain of SID. [email protected] Karmen Franinovic Stefania Serafin ZHDK - Zürich, Switzerland Aalborg University Copenhagen [email protected] Ballerup, Denmark [email protected] Thomas Hermann University of Bielefeld Frauke Behrendt Bielefeld, Germany University of Sussex [email protected] Brighton, UK [email protected] Sandra Pauletto University of York, UK Nicola Bernardini [email protected] Conservatorio “C. Pollini” Padova, Italy Patrick Susini [email protected] IRCAM Paris, France Roberto Bresin [email protected] Keywords KTH Stockholm, Sweden Yon Visell [email protected] McGill University Montreal, Canada Sound Design, Auditory Display, Multimodal Interaction, Product Design, Interactive Arts and Music, Sound Perception and Cognition, Sound Modelling [email protected] ACM Classification Keywords Copyright is held by the authors. CHI 2008, April 5 –10, 2008, Florence, Italy H.5.5. Sound and Music Computing H.5.2. User Interfaces ACM 978-1-60558-012-8/08/04. 3969 CHI 2008 Proceedings · Workshops April 5-10, 2008 · Florence, Italy Introduction Goals of the Workshop Thinking back to Edison's phonograph or, more recently, to the mp3 craze, one may well say that music has been a driving factor for technology [1]. But, can the same be said for sound in general? To date, nonmusical sounds have been accepted as by-products of technologies rather than being exploited for their intrinsic value. As a result, an acoustically polluted world is what most people have been experiencing since the industrial revolution. Times are mature to think about sound as one of the main design dimensions of the environments in which we live and work. That means overcoming the sound-as-noise cultural barrier and promoting a sound-as-information attitude. This tendency is already visible in the market where new products (e.g., the Nintendo Wii) exploit the tight coupling between sound and gesture in interaction. The same trends have manifested in the performing arts for many years, and have been recently exposed as main issues in the New Interfaces for Musical Expression (www.nime.org) series of conferences, which are also a spin-off of a CHI workshop. Designing the sonic appearance of interactive systems is thus becoming a competitive issue as well as a lively playground, which nevertheless depends on knowledge that has grown in several interrelated fields of research and practice. The relevant areas of research have grown in significance, both as a result of the design needs associated to improving the sonic jungles we are increasingly confronted with, and as economies of scale and miniaturization have contributed to a widening array of interactive artifacts and systems that are embedded with ever more sophisticated computing, sensing and actuating capabilities. The SID workshop intends to foster discussions around and promote the integration of scattered research efforts in sound modeling, design, art, perception and cognition into a coherent movement capable of improving the interactive acoustic appearance of future environments. It aims to achieve this by providing a forum to present and discuss relevant current work within and spanning these areas, organized around several key themes. Perceptual, Cognitive, and Emotional Study of Sonic Interactions This strand of the workshop showcases current research surrounding the ways humans perceive, understand, and interact through sound – knowledge that is basic for the design of new systems that utilize it as a primary modality. From low-level psychoacoustic features to more complex attributes motivated by auditory modeling, work in this area has positively impacted numerous application areas. This has included the identification of sound features salient to the function of an artifact or system, as in the design of alerting or signaling sounds [2], to perceptual features of importance for the design of musical sounds or control spaces, or those that are matched to more holistic subjective quality descriptors for a product’s identity, interaction and task context. Knowledge about certain classes of functional sounds is of particular interest for our everyday life. Perception and classification of everyday sounds is also an emerging theme which is featured in one of the contributions to the workshop. Among other desirable outcomes of research in this area, one may identify the development of new sound assessment and predictive tools that may be useful for sound designers, and the creation of 3970 CHI 2008 Proceedings · Workshops psychologically founded frameworks that are useful for the evaluation of the quality of a particular sound design [2]. Product Sound Design Sound design already plays a significant role in many areas of product design, especially those which create products with high functional densities, strong design identities, or which address demanding markets such as car design. Prominent industries that have benefited from it include the automobile and cosmetics industries, but lower profile applications have arisen in diverse other areas, from kitchen appliances to toys and office equipment. This strand of the workshop presents the state-of the art in product design and discusses future opportunities. The interactive use of sound should be seen as distinct from applications that serve primarily to signal or alert a user to an event, process, or state. The workshop showcases ways in which sound can take on a more prominent role [2]: Sound can create or reveal new functionalities in a product, when these functionalities may not be apparent through other modalities, such as the physical form or other visible indicators. Sound design is also crucial in mobile devices, as shown in one of the contributions to the workshop. Sound can also shape the sonic appearance of an artifact, where appearance is thought of as primarily referring to the qualities of the sound that is produced through interaction. Sound can also be used to provide feedback to aid users' performance with or control over an interface, tool, device, or physical activity. [4] Interactive Art and Music A major strand of the workshop deals with recent work in the interactive and performing arts that exploits the April 5-10, 2008 · Florence, Italy role of enactive engagement with sound–augmented interactive objects. The work of such artists is increasingly being recognized as a driving factor for technology and sometimes for science as well. New fields such as that of “aesthetic computing” [5] are emerging from the joint efforts and mutual interests of scientists and artists. Artists and designers have been engaged for some time in questioning the ways that sound is used in everyday life and in creatively exploring future possibilities for interactive products and systems, including those with embedded sound [6] or in a mobile context [7]. This workshop strand critically explores the present use of sound as integral to such artworks, and reveals related future opportunities and trends for interactive sound artefacts through presentations from artist-researchers in the sonic and performing arts. Some associated goals of this area of contact are to inspire SID research in complementary disciplines, to reveal interactive product opportunities, and expose social and ethical issues. Auditory Display and Sonification Auditory display research to date has gravitated around a number of topics such as the sonification of datasets (interactive or otherwise), auditory feedback in computing displays, auditory icons, earcons, signaling, and mobile communication and computing applications. Some of this work has been motivated by an increase in awareness of the need for multimodal human computer interfaces, as the contexts in which computing takes place today include many in which an additional visual display may not be effective, or may not be an option. Sonification is beginning to impact many fields such as information engineering, data mining, biomedicine, and rehabilitation. By contrast with visualization, sonification inherently develops in time, is critically 3971 CHI 2008 Proceedings · Workshops dependent on interaction, and exploits the fastest of human senses. Research in interactive sonification has as a key objective that of empowering the designers of such data sonifications [8]. The role of interactivity is crucial in applications exploiting the auditory channel in the exploration of complex information spaces. Research is required to devise and test new interaction methods and new mapping strategies for sonic exploration in order to improve the effectiveness of information foraging, browsing, and surveying with sound. Sonic Interaction Design These key themes provide a solid framework for the workshop's aim of advancing the discussion surrounding the varied roles of sound in interactive objects, systems and environments. The themes are well represented in the accepted contributions in the form of theoretical and aesthetical aspects of SID, novel tangible musical instruments, algorithms for synthesis of everyday sounds and sonification. Transversal to the four themes, other areas of discussion emerge from the submitted contributions, such as the need for a new pedagogy of product sound design, or the role of sound in enhancing presence in virtual environments. Sonic Interaction Design is emerging as a discipline where all these areas contribute valuable elements for closing the design loop in sound-based interaction. Acknowledgements This workshop is sponsored by the European COST Action IC0601 on Sonic Interaction Design [9]. April 5-10, 2008 · Florence, Italy References [1] The S2S² Consortium. A Roadmap for Sound and Music Computing, 2007, www.soundandmusiccomputing.org/roadmap [2] C. Suied, L. Pruvost, P. Susini, N. Misdariis, S. Langlois, S. McAdams, "Speeded detection of sound signals based on temporal differences", J. Ac. Soc. Am. 119(5):3333, 2006. [3] P. Susini, N. Misdariis, G. Lemaitre, D. Rocchesso, P. Polotti, K. Franinovic, Y. Visell, K. Obermayer, "Closing the Loop of Sound Evaluation and Design,” Proc. of the 2nd ISCA/DEGA Workshop on Perceptual Quality of Systems, 2006. [4] D.Rocchesso, R. Bresin, M. Fernstrom, “Sounding Objects”, IEEE Multimedia, 10(2):42-52, 2003. [5] P. A. Fishwick, editor. Aesthetic Computing. MIT Press, Cambridge, Massachusetts, 2006. [6] K. Franinovic, D. Hug, Y. Visell, “Sound Embodied: Explorations of sonic interaction design for everyday objects in a workshop setting”, Proc. of the Intl. Conf. on Auditory Display (ICAD), 2007. [7] L. Gaye, L. Holmquist, F. Behrendt, A. Tanaka. “Mobile Music Technology. Report on an Emerging Community”. Proc. of the Intl. Conf. on New Interfaces for Musical Expression (NIME), 2006. [8] T. Hermann and A. Hunt. An introduction to interactive sonification. IEEE Multimedia, 12(2):20–24, 2005. Editorial, special issue on interactive sonification. [9] Informational website of the COST IC0601 Action on Sonic Interaction Design: http://www.costsid.org/ 3972