Hydraulikos: Ice, Water, and Steam as User-Interfaces
Steve Mann and Ryan Janzen
University of Toronto
Faculty of Applied Science and Engineering, Faculty of Arts and Sciences, and Faculty of Forestry
http://hydraulikos.org
ABSTRACT
In 2001 the term "Natural User Interface" (NUI) was coined to denote the use of wearable computing or of physical matter
(solids, liquids, and gases) as direct user interfaces for metaphor-free computing ["Intelligent Image Processing", S. Mann,
John Wiley & Sons, Inc., 2001]. An example of NUI is the idioscope, a highly expressive musical instrument based on
continuous (“undigital”) scratch input ["Natural Interfaces for Musical Expression...", S. Mann, in Proc. NIME 2007, Jun610, New York, NY, USA.].
Human beings are "cyborgs" in the sense that we usually experience nature indirectly, through technologies like shoes,
clothing, or smartphones. In fact we're often forbidden from interacting directly with the world around us, e.g. simply
removing our shoes to feel the earth beneath our feet is likely to have us stopped by police or security guards.
Natural User-Interfaces challenge this layer of indirection, and use direct physical contact with multisensory primordial input
devices such as solids, liquids, and gases.
H2O (dihydrogen monoxide) is the only chemical substance that we commonly and directly experience in all three of these
states-of-matter. Thus H2O is a natural choice for a natural user-interface.
H2O is not the same thing as water: it is more general than water in the sense that it can also exist as ice or steam. We
explore ice and steam as primordial natural user interfaces.
Our ultimate goal is the creation of a centre for Cyborg-Environment Interaction (CEI) as a research trajectory exploring
the relationship between nature and technology. Presently, we will celebrate the solid and gaseous states of H2O through
ice mallets and steam pipes, in a performance entitled “Sublime Sublimation”.
BIOGRAPHIES
STEVE MANN
Steve Mann has been described by the media as "the world's first cyborg" and "the father of wearable computing" for his
invention of Mediated Reality (predecessor of Augmented Reality), and also invented HDR and panoramics (U.S. Pat.s
5828793+5706416) now implemented in most cameras including Apple iPhone. He also invented the neckworn sensor
camera now manufactured by Microsoft.
Mann creates interventions+inventions to combine art+science+technology, with emphasis on interplay between technology
and nature. Mann is the inventor of the hydraulophone, awarded numerous patents, the world's first musical instrument to
make sound from vibrations in liquid (other instruments make sound from vibrations in solids or gases), won first place in the
Coram International Sustainable Design Award, and is the recipient of the 2004 Leonardo Award for Excellence.
Mann has written more than 200 publications+books+patents, and his work and inventions have been shown at the
Smithsonian Institute, National Museum of American History, The Science Museum (Wellcome Wing, opening with Her
Majesty The Queen June 2000), MoMA (New York), Stedelijk Museum (Amsterdam), Triennale di Milano, Austin Museum
of Art, and San Francisco Art Institute.
Copyright is held by the author / owner(s).
TEI 2012, Kingston, Ontario, Canada, February 19 – 22, 2012.
ISBN 978-1-4503-1174-8/12/0002 $10.00
27
He has been featured by AP News, New York Times, LA-Times, Time, Newsweek, Fortune, WiReD, NBC, ABC, CNN,
David Letterman (#6 on Letterman's Top Ten), CBC-TV, CBS, Scientific American, Scientific American Frontiers,
Discovery Channel, Byte, Reuters, New Scientist, Rolling Stone, and BBC. Thousands of articles about him have appeared
worldwide in many languages.
He received his PhD degree from MIT in 1997, and is a tenured professor at University of Toronto, where he teaches and
does research in the Faculty of Applied Science and Engineering, the Faculty of Arts and Sciences, and the Faculty of
Forestry.
Mann's award winning documentary cyborglog ShootingBack, and ideas from his recent book "CYBORG... in the Age of the
Wearable Computer" (Randomhouse Doubleday, 2001) inspired a 35mm feature length motion picture film about his life
(http://wearcam.org/cyberman.htm), said, by P.O.V., to be "Canada's most important film of the year".
Together with his students James Fung and Chris Aimone, and neurologist Ariel Garten, Mann founded InteraXon, a
Canadian company, that is commercializing the cyborg technology developed by Mann, Fung, and Aimone. InteraXon
created a large-scale public art installation for the Vancouver Olympics (the flagship project of the Ontario Pavillion),
running Feb. 12-28, 2010. The installation bridged the gap between cyberspace (cyborgspace) and physical space, allowing
participants to use their brainwaves to control the lights on major architectural landmarks (the CN Tower in Toronto, the
Parliament Buildings in Ottawa, and the lights on Niagara Falls).
RYAN JANZEN
Ryan Janzen's compositions have been performed internationally in New York, Las Vegas, San Francisco, Copenhagen and
Shanghai. Featured on CBC, Danish radio, and at the World Expo, Ryan Janzen's compositions have been sought after for art
music, film, and esoteric concerts. As co-organizer of 14 live concerts and producer of 3 albums, he has collaborated
extensively with musicians, orchestras, and interdisciplinary researchers.
Janzen's work embodies a fusion between art and science that comes naturally: Janzen's scientific research has led to
advances in acoustics, aerospace engineering, and electric vehicle propulsion, and was awarded a federal grant from the
Natural Sciences and Engineering Research Council. His music research is published in eight international music
publications. After founding a technology consulting company in elementary school, Janzen worked on the award-winning
film Commedia Fantasia (winner of the 2002 Houston International Film Festival Gold Medal Award in performing arts), and
has since produced an array of art music, film music, orchestral and choral compositions, performance art, and high-tech
performances that push the limits of art and science.
Janzen in 2006 was the world's first composer to create music for hydraulophone, an exotic instrument which is played by
touching jets of water. Janzen developed a new expressive performance style and (along with inventor Steve Mann) a new
musical notation which gives the performer access to a never-before-seen ability: polyphonic embouchure, brought into
existence by hydraulophones.
His film scores include "Ritual", an eerie film+concert performance in which audience and performers are left in pitch dark.
Janzen's compositions have been commissioned recently for a grand-opening of the Canadian National Institute for the Blind
in Alberta, and for the Marshall Fels Elliott short film "After Hours". His compositions have been performed at the Music
Gallery in Toronto, NIME in New York City, and the Vandkulturhuset in Denmark.
Janzen's multimedia work "Stray" was featured in a Florence, Italy presentation. This piece of music was written in the C
programming language, for pipe organ and andantephone (a musical instrument designed with Steve Mann which transforms
seismic waves from footsteps).
Fusion between music and science is revealed in Janzen's talks as an invited lecturer internationally. He is featured in 22
newspaper, magazine and TV news programs on art, science and technology.
Samples of his work can be heard/seen at: ryanjanzen.ca
28