Media Arts & Technology

Inhabitat is a mixed-reality artwork in which participants become part of an imaginary ecology through three simultaneous perspectives of scale and agency; three distinct ways to see with other eyes. This imaginary world was exhibited at a children's science museum for five months, using an interactive projection-augmented sculpture, a large screen and speaker array, and a virtual reality head-mounted display. This paper documents the work's motivations and design contributions, along with accounts of visitors' playful engagements and reflections within the complex interconnectivity of an artificial nature.

We document techniques and insights gained through the creation of interactive visualizations of biologically-inspired complex systems that have been exhibited as mixed-reality art installations since 2007. A binding theme is the importance of endogenous accounts: that all perceivable forms have dynamic ontological capacities within the world; that the simulated world is able to autonomously originate; that as a result interaction can lead to exploratory discovery; and that visitors become part of the ecosystem, both through immersive display and through interactions that induce presence. Details of how each of these components have been applied in the visualization, sonification, and interaction design are given with specific examples of prototypes and exhibited installations.

Artificial Nature is a trans-disciplinary research project drawing upon bio-inspired system theories in the production of engaging immersive worlds as art installations. Embodied world making and immersion are identified as key components in an exploration of creative ecosystems toward art-as-it-could-be. A detailed account of the design of a successfully exhibited creative ecosystem is given in these terms, and open questions are outlined.

A mixed reality artwork in which participants become part of an imaginary ecology through three simultaneous perspectives of scale and agency; three distinct ways to see with other eyes. Shown at MOXI, The Wolf Museum of Exploration + Innovation, Santa Barbara US

The Interim DynaPiano (IDP) is an integrated computer hardware/software configuration for music composition, production, and performance based on a Sun Microsystems Inc. SPARCstation computer and the Musical Object Development Environment (MODE) software. The IDP SPARCstation is a powerful hardware-accelerated color graphics RISC-(reduced instruction set computer) based workstation computer running the UNIX operating system. It is augmented by large RAM and disk memories and coprocessors and interfaces for real-time sampled sound and MIDI I/O. The MODE is a large hierarchy of object-oriented software components for music written in the Smalltalk-80 language and programming system. MODE software applications in IDP support flexible structured music composition, sampled sound recording and processing, and real-time music performance using MIDI or sampled sounds.

The UCSB Allosphere is a 3-story-high spherical instrument in which virtual environments and performances can be experienced in a fully immersive fashion. It is made of a perforated aluminum sphere, ten meters in diameter, suspended inside a quasi-anechoic cube. The space is now being equipped with high-resolution active stereo projectors, a 3D sound system with hundreds of speakers, and with tracking and interaction mechanisms. The Allosphere allows for the exploration of large-scale data sets in an environment that is at the same time multimodal, multimedia, multi-user, immersive, and interactive. This novel and unique instrument will be used for research into scientific visualization/auralization and data exploration, and as a research environment for behavioral and cognitive scientists. It will also serve as a research and performance space for artists exploring new forms of art. In particular, the Allosphere has been carefully designed to allow for immersive music applications. In this paper, we give an overview of the instrument, focusing on the audio subsystem. We present first results and our experiences in developing and using the Allosphere in several prototype projects.

Persistent storage and access of sound/music meta-data is an increasingly relevant topic to the developers of multimedia software. This paper focuses on the design of music signal analysis tools and database formats for modern applications. It is partly tutorial in nature, and partly a discussion of design issues. We begin with a high-level overview of the dimensions of music database (MDB) software, and then walk through the common g feature extraction techniques. A requirements analysis of several application categories will allow us to carefully determine which features might be most useful for them. This leads us to suggest concrete architectural and design criteria, and to close by introducing several of our recent implemented systems. The authors believe that much current MDB software suffers due to ad-hoc design of analysis systems and feature vectors, which often incorporate only low-level features and are not tuned for the application at hand. Our goal is to advance the state of the art of music meta-data extraction and database design by fostering a better engineering practice in the construction of high-level feature vectors and analysis engines for music software.

This informal and subjective presentation will introduce and compare several software systems written by the myself and others for computer music composition and performance based on higher‐level abstractions of musical data structures. I will then evaluate a few of the issues in real‐time interaction with structural descriptions of musical data.The premise is that very interesting live‐performance software environments could be based in existing technology for structural music description, but that much of the current real‐time performance‐oriented software for music is rather limited in that it supports only very low‐level notions of musical structures. The examples will demonstrate various systems for graphical interaction with procedural, knowledge‐based, hierarchical and/or stochastic music description systems that could be used for live performance.

The Siren system is a general-purpose music composition and production framework integrated with Squeak Smalltalk (1); it is a re-implementation of the Musical Object Development Environment (MODE) (5), the software component of the on-going "Interim DynaPiano" project. Siren is a Smalltalk class library (about 200 classes) for building musical applications; it runs on a variety of platforms with support for MIDI and audio I/O. Siren's source code is available for free on the Internet; see the Siren home page at the

The addition of spatially localized sound to an existing graphics oriented synthetic environment (virtual reality system) is investigated. To build 3-D audio systems that are robust, listener-independent, real-time, multi-source, and able to give stable sound localization is beyond the current state-of-the-art. The "auralizer" or "aural renderer" described is built as a test-bed for experimenting with the known techniques for generating sound localization cues based on the geometrical models available in a synthetic 3-D world. The psychoacoustical background of sound localization is introduced, and the design and usage of the Distributed Interactive Virtual Environment (DIVE) auralizer is described. The system's implementation and performance are evaluated

This panel discussion will review the standards that the computer music community has produced and how these standards were created, followed by a guided interactive group discussion about future directions for our community in terms of old and new standards.

This paper describes several generations of computer music systems and the music they have enabled. It will introduce the software tools used in some of my music compositions realized in the years 1979-94 at a variety of studios using various software and hardware systems and programming languages. These tools use a wide range of compositional methods, including (among others): high-level graphical notations, limited stochastic selection, Markov transition tables, forward-chaining expert systems, non-deterministic Petri networks, and hierarchical rule-based knowledge systems. The paper begins by defining several of the terms that are frequently used in the computer music literature with respect to computer-aided composition and realization, and introduces several of the categories of modern models of music composition. A series of indepth examples are then drawn from my works of the last 15 years, giving descriptions of the models, the software tools, and demonstrating the resulting music.

The task of building a description language for audio synthesis and processing consists of balancing a variety of conflicting demands and constraints such as easy learning curve, usability, flexibility, extensibility, and run-time performance. There are many alternatives as to what a modern language for describing signal processing patches should look like. This paper describes the object-oriented models and design patterns used in version 4 of the CREATE Signal Library (CSL), a full rewrite that included an effort to use concepts from the "4MS" metamodel for multimedia systems, and to integrate a set of design patterns for signal processing. We refer the reader to other publications for an introduction to CSL, and will concentrate on design and implementation choices in CSL4 that simplify the kernel classes, improve their performance, and ease their extension while using best-practice software engineering techniques.

The UCSB Allosphere is a 3-story-high spherical instrument in which virtual environments and performances can be experienced in full immersion. The space is now being equipped with high-resolution active stereo projectors, a 3D sound system with several hundred speakers, and with tracking and interaction mechanisms. The Allosphere is at the same time multimodal, multimedia, multi-user, immersive, and interactive. This novel and unique instrument will be used for research into scientific visualization/auralization and data exploration, and as a research environment for behavioral and cognitive scientists. It will also serve as a research and performance space for artists exploring new forms of art. In particular, the Allosphere has been carefully designed to allow for immersive music and aural applications. In this paper, we give an overview of the instrument, focusing on the audio subsystem. We give the rationale behind some of the design decisions and explain the different techniques employed in making the Allosphere a truly general-purpose immersive audiovisual lab and stage. Finally, we present first results and our experiences in developing and using the Allosphere in several prototype projects.