Perspectives on Aesthetic Computing

Author(s): Paul Fishwick, Stephan Diehl, Jane Prophet and Jonas Löwgren
Source: Leonardo, Vol. 38, No. 2 (2005), pp. 133-141
Published by: The MIT Press
Stable URL: https://www.jstor.org/stable/1577793
Accessed: 04-11-2018 23:20 UTC

REFERENCES
Linked references are available on JSTOR for this article:
https://www.jstor.org/stable/1577793?seq=1&cid=pdf-reference#references_tab_contents
You may need to log in to JSTOR to access the linked references.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide
range of content in a trusted digital archive. We use information technology and tools to increase productivity and
facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected].

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at
https://about.jstor.org/terms

The MIT Press is collaborating with JSTOR to digitize, preserve and extend access to
Leonardo

This content downloaded from 161.116.100.134 on Sun, 04 Nov 2018 23:20:22 UTC
All use subject to https://about.jstor.org/terms
 GENERAL ARTICLE

Perspectives on
Aesthetic Computing
ABSTRACT

The authors present an int

duction to the new inter-
disciplinary area of aestheti
computing and proceed to
Paul Fishwick, define this area with examp
from each of their own disc

Stephan Diehl, Jane ProphetWhile

plines, practices and resea
several decades of

andJonas Lowgren
publication and work have
resulted in significant adva
ments in art as implemente
through technology, less em
sis has been placed on stud
the converse issue of art's
effect on computing, or "ae
thetic computing." The auth
nrncant thL ir inriixII
K III iinil u Ainr
In collaboration with Leonardo, we participated
terpreted more generally, Knuth
this area is tfollow
and then
th
J ,tisI
wi
in the Aesthetic Computing workshop in the hills of south- expressing the idea of aesthetics er'sas brief c
western Germany inJuly 2002. The workshop [1], organized being more than the purely cogni- work t
by Paul Fishwick, Roger Malina and Christa Sommerer,tive, was even within computing, which y persp
s-
one week in duration and was attended by over 30 represen- grew out of mathematics. A textual approa
thetic computing
nner, in this ma
tatives of the following disciplines: art, design, computersection
sci- of a computer program )duc- will the p
ence and mathematics. The purpose of the workshop was to
have both denotative as well as con-
ysis tion to
define this area and to try to bring to light key aspects ofnotative
the signifiers, and it is easy to of t
field from a variety of perspectives. An aesthetic computingimagine that the program might
M
manifesto was recently published in Leonardo [2]. Aesthetic align itself with the goals of art by
computing is the application of art practice and theory to using
com- the types and variety of rep-
puting. Given this brief definition, several questions come to
resentations employed in the art world, th
mind; we shall address these questions and then proceed traditional
with boundaries of what may be
the purpose of this paper, which is to cover sample projects in
computer program representation.
the area and the different ways in which the authors proceed The second question, regarding the com
to accomplish their singular crafts. with art, is addressed by first noting that
One question regarding the name "aesthetic computing" digital art, the idea of aesthetic computin
may well be whether we can justify its existence. After all, is
affecting-and reflecting-some aspect o
not all computing within the realm of aesthetics, and is there
results in an agenda quite different from
not already a significant number of projects that capture some
other fields of digital art.
combination of computing and art? To acknowledge our interest in aesthet
Let us begin with the first question. We will define aesthet-
to expand human-computer interaction
ics broadly as a combination of cognitive and sensory modes
to reach into numerous areas of comput
of experience, according to its scope in philosophy [3]. primary
Evi- operating systems interface to th
dence of the employment of the purely cognitive aspect ofcomputer
aes- scientists to create programs and
thetics can in fact be found within mathematics [4] and ate models of geometry and dynamics. T
computing: One speaks of an elegant proof of a theorem, or
observations that partially justify the m
a beautiful representation. With such qualifiers, the mathe-
matician is usually referring to cognitively grounded aesthet-
ics. Our intent is to explore aesthetics by means of the bridge
that separates the cognitive and sensory perspectives-it is by S1
crossing this bridge that we may enrich both design and com-
puting. As Knuth points out in his discussion of Metafont, Fig. 1. Paul Fishwick, a
2D static snapshot of an
which underlies his TeX typesetting system, "Type design can
interactive diagrammatic
be hazardous to your other interests. Once you get hooked, FSM interface. (? Paul
you will develop intense feelings about letterforms" [5]. In- Fishwick)

Paul Fishwick (educator), Department of Computer & Information Science & Engineer-
ing, University of Florida, Gainesville, FL 32611, U.S.A. E-mail: <[email protected]>.

Stephan Diehl (educator), Computer Science, Catholic University Eichstatt, Ostenstr. 14,
85072 Eichstatt, Germany. E-mail: <[email protected]>.

Jane Prophet (artist, researcher), Centre for Arts Research, Technology and Education,
University of Westminster, 70 Great Portland Street, London, U.K. E-mail:
<[email protected]>.

Jonas Lowgren (designer, researcher), School of Arts and Communication, Malmo

University, SE-205 06 Malmo, Sweden. E-mail: <[email protected]>.
S3 S2

?2005 ISAST LEONARDO, Vol. 38,

This content downloaded from 161.116.100.134 on Sun, 04 Nov 2018 23:20:22 UTC
All use subject to https://about.jstor.org/terms
 ing areas of information visualization [7] tion. In XML parlance, content refers to
and software visualization [8,9] represent an abstract specification defined as a doc-
approaches toward this goal, and yet their ument tree, and presentation refers to
representations of information and soft- the way the tree is presented to the user-
ware could benefit from greater empha- the way it looks and sounds. Thus, using
sis on a wider range of artistic expression RUBE and guided by the XML philoso-
without sacrificing utility. So it is not that phy, one may specify an equation and
the area of design does not presently con- choose to present the equation as either
cern itself with incorporating aesthetics linear text, a network or a 3D structure.
but that the current level and degree of The choice of which presentation to em-
this incorporation need to be expanded ploy can be determined by XML style
beyond those of the typical user interface sheets and their associated transforma-
at the operating systems level. tions.

Fig. 2. Paul Fishwick, a 3D agent-based We now have a working idea of aes- RUBE's architecture is based on open-
presentation of the FSM using the Virtual thetic computing, and we need to ex- source software and begins with author-
Reality Modeling Language (VRML). plore different approaches. There follows ing toolkits: SodiPodi for 2D vector
(? Paul Fishwick)
a statement on aesthetic computing by drawing and Blender for 3D modeling.
each author, followed by a discussion of Let us consider the 3D pipeline begin-
one another's views and statements. ning with Blender. The user creates a 3D
computing: (1) aesthetics in computing model in Blender and then uses a Python
are broader than the purely cognitive di- scripting interface, which allows attri-
PAUL FISHWICK: FORMAL
mension; and (2) the art-science conflu- butions to be made regarding seman-
REPRESENTATIONS
ence embedded within the discipline of tics. For example, one might point to
interaction design is broader than the Scope an object and designate it as a state or
primary "desktop" interface. In applying aesthetics to computing, we a function. After the semantic assign-
The first observation is one of both on- need to confine ourselves to some aspect ment, an X3D (eXtensible 3D) file is cre-
tology and epistemology-we might
of computing, or one of its subfields such ated for the presentation and a special
leverage existing aesthetic principles to-
as human-computer interaction, visuali- XML file is created for specifying the for-
ward the sensory. Consider the case zation
of or discrete structures, to name a mal model. After some XML transfor-
"software patterns" [6] as one example.few. At the University of Florida, we have mations, this XML file is translated into
One can surely conclude that these pat-constructed a software system called Javascript orJava, whereby it can be rein-
terns reflect abstractions of software RUBE [10-13], in which we have fo- corporated into the VRML (Virtual Re-
structures and that they are foundcused lurk-primarily on representations in ality Modeling Language) file, resulting
ing in numerous software applications. mathematics and computing notation, in an interactive VRML world. The 2D
from the notation of algebraic and dif-
There are two ways of looking at patterns, transformations are similar, except that
ferential equations to that of program
reflecting two ways of defining aesthetics: Scalable Vector Graphics (SVG) is used
cognitive and material. What if the andpat-
data structures, informed, however, for presentation.
terns revealed themselves in a way that an artistic sensibility. Our basic Let us begin with a formal definition
through
was more attuned to material embodi- idea has been to build a system that al- of a Finite State Machine (FSM) M [14].
ment? This would allow us to build upon
lows a multiplicity of different notationsThese machines have states that are in-

to be constructed so that one may see

the existing pattern literature and extend terconnected through transitions that go
it, as well as to extend the pattern andrepre-
hear the same underlying formalism into effect when an input to the machine
sentations. The "factory" method es-
in numerous ways. Not only do differ- is of a particular value. Here is a formal
poused by Gamma et al. could be ent thepeople and cultural entities enjoy definition for M:
basis for a two-dimensional or three-di- working with a formalism using different M= <Q I, O, , A>
mensional scene that looks and operates metaphors, but a single person or groupQ= {S1, S2, S31, 8: Q x I- Q
like a factory. The look and feel ofcan thealso benefit from exposure to diverse 6 (SI, 0)=S1; 6 (S2,0)=S2; 6 (S3,0)=S3;
factory would improve with artisticpresentations.
in- 3 (S, 1)=S2; 6 (S2,1)=S3; 8 (S3,1)=S2
fluence and guidance and would provide I=/0, 11, k: Q- 0
strong metaphorical cues. This sensory Implementation Even though this text might seem to be
dimension seems lacking in the pattern RUBE therefore allows for different rep-the formal specification for M, it is actu-
literature; one generally finds that resentations
rep- to be applied to a select
ally one of many types of presentation of
resentation is limited to rectangles number and of formal dynamic modelthe underlying formalism that is encoded
arcs. And yet it is not only in the patternspecifications. Using RUBE, it is possible
in XML. In general, all presentations re-
literature that it is lacking, given that to change the way formal models look quire additional natural-language se-
the visual minimalism of program struc- and sound. By formal models, I am re- mantics if we are to make sense of them.
tures and the mathematical structures ferring to a large class of models used to
Qis the state set for M, I the input set, O
underlying them is fairly common in systems that incorporate time for
specify the output set, 6 the transition function
computing. analysis and simulation, such as finite from one state to another and k the out-
The second observation implies that state machines, Petri networks, Markovput function.
interaction design needs to concern itself models, queuing models and System Dy- Figure 1 illustrates our second presen-
with all questions of interaction and pre- namics graphs, as well as ordinary and tation of the FSM. It has iconic proper-
sentation found in computing, including ties such that when the machine is in state
partial differential equations. RUBE uses
that of how to represent data and pro- XML (eXtensible Markup Language), S2, the presentation of a circle for S2
gram structures, for instance. The emerg- which separates content from presenta- encodes the concept of a boundary and

134 Fishwick et al., Perspectives on Aesthetic Computing

This content downloaded from 161.116.100.134 on Sun, 04 Nov 2018 23:20:22 UTC
All use subject to https://about.jstor.org/terms
that which is inside it. That is, the graph-
ical depiction of S2 is consistent with the
underlying metaphors of set theory, Fig. 3. Stephan Diehl, program
structure with analysis results.
whereas the purely textual presentation (? Stephan Diehl)
does not capture these metaphors. More-
over, Fig. 1 is incomplete on a non-inter-
active medium, such as paper, since the
additional information encoded but not

visible in the text representation is ob-

tained through minimal human-com-
puter interaction (e.g. pressing a button
to yield the text layer). Similarly, the ar-
rows capture the notion of transition .....:: :.............................

from one state to another, since anyone

who has seen an arrow fly knows that it is
aimed toward a target. The metaphors of
the figure dramatically improve visuali-
zation of the semantics of the machine,
the visual presentations are strikingly dif-program text is written in an artificial lan-
and so one is led to wonder whether em-ferent. guage with a strict syntax and more or
ploying presentations with alternative The expression of an abstract state inless well-defined semantics. Trying to un-
aesthetics might further improve the an im-FSM, for example, need not requirederstand a real software system by read-
pact of the metaphor. The underlyingthat as- the state be presented visually in aing its millions of lines of program text
minimalist fashion. The key task is tois a vain task. As a consequence, many
sumption is that material aspects of levels
of representation are based largely strengthen
on the metaphor of what ittools have been developed to support
means
what is available for a society, as well as to be a state and the correspon- software understanding. These tools rely
what is affordable and materially effi-ding elements of boundary that go along on analysis and visualization techniques.
with it.
cient. Consider Fig. 2 as a representation In a famous paper, Turing Award recip-
that has only recently become possible The second question, about cultural ient Fredrick Brooks even stated that
barriers, may lie at the heart of the aes- "software is invisible and unvisualizable"
through computer graphics and the abil-
ity to employ 3D components. The meta-thetic computing challenge. Computer because each kind of visualization only
scientists have been educated with mini-
phor of the circle as a boundary has been addresses "one dimension of the intri-

malist figures and text, and so it may

replaced by a small gazebo-like structure. cately interlocked software elephant"
The arrow in Fig. 1 is now shown as come
a as a shock to realize that our rep-[ 15]. These dimensions include the static
resentations of formal objects are notstructure
woman walking from one state to another as of the software, its dynamics
along a lamp-lit walkway. While Fig. 2
constrained as we may have thought. and
Un- its evolution. To put it in other
til the era of computer graphics and words,
could be seen as introducing gratuitous fast different kinds of visualizations
imagery, it can alternatively be seencomputers,
as show how software is encoded, how it
we had little need to inquire
a structure that reinforces the spatio- behaves and how it is developed. In the
about what initially appeared to be exotic
temporal metaphors that Fig. 1 onlyways
be- to encode formal knowledge. How- following I present and discuss exam-
gins to reveal. The idea here is thatever, ples of visualizations for each of these
an this is a challenge not only for com-
dimensions.
puter scientists but also for artists, since
FSM, by its formal definition, is rich with
In Fig. 3 a graphical representation of
artists should be encouraged to consider
metaphor and that Fig. 2 exploits the
metaphor by surfacing it through visual- the computer and computing practices a program, its control-flow graph, is
ization, interaction and familiar icons. shown. In addition the graph contains
as subject material in addition to raw ma-
Movement from one state to another some information computed by a pro-
terial. This suggestion of formal struc-
looks like movement, and the thing mov-
tures gram analysis. With the help of this visu-
as raw or subject material may strike
ing is more familiar to us than a some
mental alization, developers can detect certain
artists as a modernist-era agenda;
construct would be. however, the computer and its mathe- kinds of errors, so-called stack overflows,
A host of philosophical issues comes matical foundations enable the creation in their programs [16].
into play here. Is there not a need to of en-significantly higher complexity as InaFig. 4, a snapshot of the animated
force visual minimalism within this sort execution of a sorting algorithm is
tool, or as a subject, than paint, palette
of structure? What are the cultural bar- knife or chisel ever could. shown. The window contains several rep-
riers that might prevent the adoption of resentations or views of the data sorted.

models like Fig. 2 in science and engi- Algorithm animations are typically used
STEPHAN DIEHL: in education. Most of the animation
neering?
SOFTWARE VISUALIZATION
With respect to the first issue, we techniques do not scale for real software
should note that it is quite possible to Software is neither matter nor energy.systems.
It
maintain abstraction without requiring is just a kind of information. Matter and Finally, Fig. 5 is a pixel map. In this ex-
visual minimalism. Within an artistic con- energy are media that carry information ample the color of the pixel at position
text, this can be seen when comparing and thus software. For new software to be (x,y) represents the number of times files
and contrasting the genres of Abstractdeveloped or existing software to be un- fx and fy have been changed together rel-
Expressionism and Surrealism. Both derstood,of ative to the total number of times file fx
it has to be projected into a hu-
these genres contain a wide variety manly of readable form-the program text. has been changed. From this figure the
works that employ symbolism, iconog- developer can see how strongly different
Note that the program text is not the soft-
raphy and rich semiotics even though ware, just a representation thereof. The files are coupled. We call this kind of cou-

Fishwick et al., Perspectives on Aesthetic Computing 135

This content downloaded from 161.116.100.134 on Sun, 04 Nov 2018 23:20:22 UTC
All use subject to https://about.jstor.org/terms
pling evolutionary [17], because it is sualizations, the current research on soft-
pothesis-driven ethos of the medical re-
based on the change histories of files, to ware visualization is rich in the different
search lab to reflexive practice in the art
distinguish it from the logical coupling properties of software that have been studio
vi- to the empirically driven environ-
usually used in software engineering. As sualized but poor in the spectrum of ment
vi- of mathematics. The aesthetic con-
the files are sorted by their containing di- sual metaphors used: boxes, circles, lines
text of each discipline is equally diverse.
rectories, the pixels form blocks. These and color. Visualization in the laboratory differs
blocks indicate that files within a direc- from visualization in the art studio. In the
tory are coupled, that is, often changed medical laboratory, representation is usu-
JANE
together. Software developers are mainly
PROPHET: ally taken literally, as scientific illustra-
INTERDISCIPLINARITY AND
interested in the outliers: those pixels tion. In the art studio, "representation"
AESTHETIC DIFFERENCE
representing couplings between files in is a term and process framed by cultural
different directories, such as those la- I consider aesthetic computing throughand art historical theories of represen-
beled "Patches" in Fig. 5. Outliers can be a discussion of the project Cell. Cell is
tation
an (for example, an image, sound or
a sign of a bad system architecture. interdisciplinary collaboration between
object can signify something without ac-
We have now seen three very different an artist (myself), a liver pathologist tually sounding or looking anything like
kinds of visualizations as they are used for (Neil Theise), a mathematician (Mark it). In Cell, we document, develop and
understanding software and its develop-d'Inverno), a computer scientist (Rob evaluate the interdisciplinary collabora-
ment process. Saunders) and a curator (Peter Ride). tive process itself, including discussion of
Now what about the aesthetics of these Cell explores new approaches to the our different aesthetic values.
visualizations? It may come as a surpriserepresentation of cell behavior, using Through studio and laboratory visits,
mathematics to bridge the gap between Theise and I identified significant aes-
to the artistic reader, but there actually
were some aesthetic criteria involved in scientific theory and computer visualiza- thetic differences. In cell biology the
the computing of these visualizations: tion. Related literature is found in Good- "photographs" of tissue slides have a
In Fig. 3 the number of edge crossings sell's research [18]. Results include solely
truth-status and are accepted as "proof'
and bends has been reduced, and di- authored and co-authored papers of in experiments within papers. The
rected edges are mostly drawn down- peer-reviewed medical, mathematical beauty of these representations is also val-
wards. In Fig. 4 color is used consistently
modeling and simulation journals; math-
ued. Figure 6 shows one of Theise's im-
in the different views. While the algo-
ematical models of a new paradigm ages,
of representing skin tissue from a
stem cell behavior; dynamic simulations
rithm actually performs discrete changes, female mouse who received a bone mar-
the animation of these changes is exe-of the mathematical model; art installa- row transplant from a male mouse. The
tions; and illustrations of cells and their
cuted smoothly. In Fig. 5, color coding tissue has been dyed, as is typical in such
based on the heat metaphor was origi- behavior generated using artificial life experiments, and blue nuclei of hair fol-
nally used: The color red showed the(Alife) techniques. Here my focus is licle on lining cells surround the orange,
highest coupling, blue low coupling and the graphic outputs from Celrs agent- autofluorescent hair shaft (large arrow).
white no coupling at all. In other words, based Alife simulations. Two of these nuclei contain fluorescently
hot files are those that are often changed labeled Y-chromosomes (small arrows)
together. Context indicating that they derive from the do-
The sole purpose of the above-men- Celrs practical research context ranges nated male bone marrow, not from the
tioned aesthetic criteria is to produce vi- female's own original cells. Thus, bone
from Theise's medical laboratory to d'In-
sualizations that convey the information verno's and Saunders's mathematical and marrow stem cells have given rise to skin-
as clearly and effectively as possible. As computer science labs and to my type art lining cells.
a consequence of concentration on the studio. Each setting has particular em- As is typical of contemporary artists, I
automatic generation and usability of vi- bedded methodologies, from the was hy-educated to resist beautifying arti-
facts and taught that photographic rep-
resentations have no truth-status and that

their meanings are subjective rather than

objective. Alife systems that have a graph-
ical output pose further challenges to no-
>....- Fig. 4. Stephan Diehl,
animation of the
tions of representation. Such outputs are
!I . _ HeapSort algorithm. at least semi-autonomous; images can
iew . iB| - -- (? Stephan Diehl) endlessly change (some might say evolve)
as they are produced in real time to rep-
resent the software running beneath
them, which is constantly changing as a
complex system of interactions between
agents takes place.

Converging: Focusing
on Behavior across Time
Theise's research examines the plasticity
of adult stem cells and their function, us-
ing processes including the analysis of
specimens of cell tissue. Because the tis-
sue is dead at the time it is analyzed, it
represents a frozen moment in time,

136 Fishwick et al., Perspectives on Aesthetic Computing

This content downloaded from 161.116.100.134 on Sun, 04 Nov 2018 23:20:22 UTC
All use subject to https://about.jstor.org/terms
 screen as a framing device; stem cells are
"born" at the center of the screen, mov-
. . . ? '
Fig. 5. Stephan Diehl, ing to the edges as they age, divide and
number of simultaneously
: , . . : t
.. r
':
. die. The line and single tone for each cell
changed files. (? Stephan
Diehl) is deliberately abstracted and looks inor-
i
1-- %:E
?* ISnnl.
. .H.
ganic. This is more aesthetically pleasing
to me-it makes no attempt to suspend
t

+
s
? isf
:?
?:
71:
?: i
the viewer's disbelief. It is obvious that
t^* **.
* ~~~~~~~~~~~F,i
this work is computer generated.
*.ODD e
SOJiW:|i
J ~~~~~:08
.e . k.~~~~~~~~~~~~~~L: By contrast, the 3D version has been
influenced by the aesthetics of medical
.}, illustration and its goal of explaining via
* 9 ] wW EA USO precise observation of the appearance of
? .... .. .. ........,

things; this is at odds with my emphasis

on the behavior of things (in this case,
stem cells). This version (using the same
underlying Alife simulator but with a dif-
I~' :
........_ . r .. -. .. .
ferent graphic output) is characterized
by its photorealism: detailed surface ren-
dering, depth and transparency (Fig. 7).
This was pleasing to Theise, as it was fa-
miliar to peers used to seeing video and
from which researchers hope to under- stills of actual cells recorded via micro-
stand another aspect of stem cell behav- scopes. However, Theise might deter-
ior and extrapolate further hypotheses mine system-level changes in behavior
to test. My experience as an artist work- that could influence wet laboratory work
ing with time-based media and Alife sug- more easily via the abstract 2D version.
gested a different approach to assessing Anecdotal evidence suggests that ab-
stem cell behavior. We therefore de- straction prompts the viewer to look for
veloped an Alife engine to enable the patterns of behavior rather than focusing
scientist to study simulated stem cell be- on the activity of individual cells.
havior as it happens within the complex
system of a wider community of cell types
value without the intentions of the work"
JONAS LOWGREN:
and enzymes. Stills from the real-time 3D
INTERACTION DESIGN
graphics driven by our simulation
[20].(Fig.
If conceptual art has an aesthetic,
7) show images that draw on theit aes-
is the dematerialization of the art- AND AESTHETICS
thetics of medical illustration. The user
object; the object only has value as aInteraction
ma- design is concerned with
terialization of the idea, not in and of
clicks on a moving cell to see a data read- shaping the use qualities of digital arti-
out. Figure 7a shows photorealistic semi-
itself. Mathematics and computing sci-facts. Another way of putting it would be
transparent cells moving, dividing ence and can both operate without materi-to say that interaction design is design of
dying in 3D space. Figure 7b illustrates ality and can describe the immaterial,the digital materials, where the word de-
information available when a user clicks which is one reason why there may be asign is used in the strong sense of ex-
on a cell or activates a global command mutual attraction between computer sci-ploring all aspects of a possible future:
that makes the data for all cells visible. entists, mathematicians and artists work-aesthetic and ethical aspects as well as
The initial Cellcollaboration expanded ing conceptually using digital media. structural and functional.
to include d'Inverno, who determined It may seem odd, and in fact it is odd,
the mathematical rules that describe Aesthetics and Visualization to talk about aesthetic computing as if it
stem cell behavior as proposed by ToTheise
date, the real-time graphic represen-were something new and hitherto unex-
and modified by me (that stem cells tationcanarising from Cells complex adap-plored. All computing is aesthetic in the
evolve into mature cells of other or- tive system (Fig. 7 and Color Plate A sense
No. that all use of digital artifacts en-
gans such as skeletal muscles, bones 2) isand
not what I would consider a piece of aesthetic reactions. To be sure, many
tails
brains, with unexpected plasticity). fine art in its own right. This is scien-
Saun- contemporary digital artifacts tend to
ders interpreted the mathematicaltific model elicit aesthetic reactions along the lines
visualization, or graphics, informed
producing real-time graphical displays by my aesthetic framework. I wantof thefrustration, indifference or boredom,
with me. The aesthetic (designed by look and feel to reflect thebut
graphic un-these are aesthetic reactions never-
Saunders and Prophet) is influenced by software, to draw attention
derlying to
theless, and as designers, we are free to
medical and scientific illustration. the essence of the idea or concept. From aim for other kinds of reactions if we like.
this standpoint the 2D Java version
Of course, there are historical reasons
Aesthetics and Conceptual Art
(Color Plate A No. 2) is a more satisfying
for the existence of such blind spots. The
The notion of "an" aesthetic is chal-
outcome than the 3D version. Here,academic
my roots of interaction design stem
colorart,
lenged in Cell. As in much conceptual from the field of human-computer in-
palette has white stem cells. Cells
the idea behind Cell (namely, modeling
in each lineage tree are a particular teraction,
hue, where the main focus has al-
the behavior of stem cells) and the means darker as they differentiate
becoming ways been on the task-oriented use of
through
of producing it (via interdisciplinary col- division. Lines signify genes
digital artifacts and in particular on its
laboration) are more important thanwithin circles defining cells. I use efficiency
the the and absence of errors. It is not

Fishwick et al., Perspectives on Aesthetic Computing 137

This content downloaded from 161.116.100.134 on Sun, 04 Nov 2018 23:20:22 UTC
All use subject to https://about.jstor.org/terms
 icala "window" fills again with metaphori-
tion is pliable to the user if it feels like
responsive material, a matter for inquiry cal condensation. If Granny happens to
that can be manipulated and experi- see the signs before they are gone, she
enced in a tactile sense. Pliability con-
can reply in the same way. A sign on the
tributes to a highly involved process display
of disappears completely in a few
exploration where the loop between hours.

senses, thought and action is rapid and The strength of the Hazed Windows
physical rather than elaborate and concept
ab- is not in its focus on lightweight,
emotional communication, which has
stract. I make a small, quick move-the
material shapes and responds-I noticebeen realized thousands of times before,
something new-I make another move- but rather in the elegant questioning of
and so on. Ahlberg and Shneiderman the hidden core assumptions of com-
puter-mediated communication. When
were the first to articulate this quality,
under the label of "tight coupling" we deal with e-mail, we devote our full at-
[21].
In the interpretation above, pliability tention to it. We expect messages to per-
concerns the micro-qualities of interac- sist until we choose to file or delete them.

tion, the qualities of the surface. Exam- In short, we approach the communica-
ples of interaction design aiming at tion situation as a binary task. The Hazed
pliability include the influential concept Window hints at the possibility of a new
of dynamic queries [22] as well as the middle ground, a new approach to com-
Fig. 6. Hair follicle with lining cells derived
from bone marrow. (? Neil Theise)
more recent interaction technique Sens- puter-mediated communication in text
A-Patch [23]. Another aspect of pliabil- and pictures that exhibits a much greater
ity has to do with the user's capacity to degree of fluency.
act freely and shape the material accord- Seductivity refers to the captivating
surprising that the image of computer ing to the larger situation at hand, such qualities of a digital artifact. Following
use is sometimes simplified to the extent as annotating the margins of a paper the seminal analysis by Khaslavsky and
that useful, efficient and error-free use form to communicate something outside Shedroff [26], seduction is described an-
is seen as the whole picture. The disre- the rigid boundaries of the form itself. alytically as a process of enticement, re-
garded parts of the picture, including aes- As Henderson and Harris [24] point out, lationship and fulfillment. Enticement
thetic qualities of the use, will then this kind of deep pliability is often un- concerns grabbing attention and making
appear as new when they are brought necessarily lost in the transition from pa- an emotional promise. The subsequent
into consideration. per to computer systems in, for example, relationship is based on making progress
How, then, should aesthetic use quali- administrative work. It can be straight- with small fulfillments and more prom-
ties be dealt with in interaction design? forwardly observed that many existing ises, possibly lasting for a long time. The
A common fallacy is to equate aesthetics administrative systems could be extended fulfillment, or ending, involves making
with pleasing visual design. To be sure, to accommodate free-form annotation good on the final promises and ending
there are design situations in which the
and the equivalent of sticky notes. the experience in a positive and memo-
immediate visual impression is an im- Fluency as an aesthetic quality ofrable digi-way.
portant factor in determining the tal out-artifacts is brought to the fore inIt re- should be clear from the description
come of the interaction. A good example above that seductivity, in the sense used
lation to the increasingly pervasive digital
is a web shop with one-time customers infrastructure. Use is not necessarilyhere, a pri-is a quality that crucially depends
who ideally should spend money on their mary activity at the focus of attention; on theitdynamic gestalt, the temporal
first and only visit. The visual design qualities of the interaction. It has noth-
of a binary variable in which a digital
is not
the web shop's front page is crucial in artifact
es- is either used or not. Withing to do with sexually explicit pictures
ubiq-
tablishing the right combination ofuitous de- and mobile computing, it becomes on the front page of a web site. The ex-
sire and credibility to actually make more of a dance among multiple repre- ample offered by Khaslavsky and Shedroff
customers enter, shop and pay. sentations and mediations. Streams of is the Visual Thesaurus [27] by Plumb
In nearly every other case, however, information
we flow between center Design.
and pe- It is a web application that du-
need to realize that a digital artifact is
riphery as we move through the shiftingplicates the contents of a traditional the-
constituted primarily not by its static environments
vi- of everyday life andsauruswork. but takes on entirely new qualities
sual design but by its dynamic gestalt Transitions need to be graceful by and
virtue of its interactive properties. The
-the character of the interaction it al- nondisruptive. user explores words, their synonyms and
lows over time. Digital design materials As a simple but conceptually powerful eventually the transient nature of lan-
are temporal in this respect, at least example
as of fluency, consider theguage Hazed
itself by navigating a beautifully an-
much as they are spatial. Below are Windows three concept by interaction imatedde-network of words and their
examples of aesthetic qualities in the signusestudents Trine Freiesleben,interrelations.
Miska
of digital artifacts. They serve as illustra- Knapek and Henrik Moberg at Malm6 Khaslavsky and Shedroff argue that the
tions of how the aesthetics of interaction University [25]. The idea is simplyVisual to of-Thesaurus is seductive in the sense

design can be approached and articu- fer a more lightweight and transient that it offers surprising novelty, goes be-
lated for further debate as well as for communication channel, for instance yond be-obvious needs and expectations
practical application in concrete design tween a little girl and her granny who and creates
live an emotional response due
situations. in different cities. The girl draws ortowrites its visual and interactional beauty (en-
Pliability is the quality of plasticity, of her finger on a display in herticement).
with home. It connects to personal goals
malleability of the digital artifact under Her signs are redrawn at Granny's through display the fascination of words and
the hands of the user. A set of informa- but gradually fade away, as the metaphor- concepts and promises to fulfill those

138 Fishzick et al., Perspectives on Aesthetic Computing

This content downloaded from 161.116.100.134 on Sun, 04 Nov 2018 23:20:22 UTC
All use subject to https://about.jstor.org/terms
goals (relationship). The casual viewer disciplinarity leads to new collabora-representations without sacrificing ab-
may discover deeper meanings in look- tions, influencing the computing straction. Mathematics and software are
ing up a word by apprehending the mul- process. Applying aesthetics to com- concerned not with visual abstraction

tidimensional and dynamic relationships puting is a matter of process and hu- but with conceptual abstraction. For ex-
between concepts (fulfillment). man integration, not just a static ample, a stack is a common abstract
result from the application. data structure used in computing. That
An Analysis of Perspectives * Lowgren broadens the traditional a stack can be represented in 3D using
As suggested in the title of this paper, the view of usability to include the aes-an aesthetic form that is meaningful to
reader is meant to come away with the thetic qualities of the digital designa group of users (that is, a culture) does
differing views of four authors. What are material. As a community, computing not defeat the conceptual abstraction.
the connections, and is there a synthesis professionals must balance cognitive Rather, the 3D concretization of the
among the varying views? What was and sensory qualities in design. stack brings to the surface the inherent
learned during the workshop? Starting For the most part, there is generalmetaphor that led to 2D diagrammatic
with the second question, we learned that agreement among the authors that each stack representations in the first place;
the terms aesthetics and computing are section corresponds to a separate axisthe concept of a stack is rife with meta-
complex and that aesthetic computing is upon which aesthetics can be seen to ap- phor-using artistic styles and famil-
not merely the straightforward applica- ply to computing; however, there is one iar objects brings these metaphors to
tion of one field to another but also a self- significant issue raised by Fishwick and life within the same abstract conceptual
reflective activity. By juxtaposing two Diehl that deserves explanation. Diehlsphere of knowledge.
words in a new way, we suggest that the discusses the state of the art in software The key conflict is cultural-scientists
art, design and computing communities visualization and emphasizes the current have been trained with visually abstract
must reevaluate the very meaning of the way in which software designers generally representations for the sake of material
words aesthetics and computing and not view aesthetics-as optimality considera- and labor economy. With the advent of
only what it means for them to be com- tions inherent within the visual design. new display technologies, it is no longer
bined as a phrase. For example, an aesthetic can be defined inconvenient to form novel 3D repre-
To summarize the key perspectives al- to minimize the number of crossingssentations in while simultaneously preserv-
ready presented: a 2D graph, or to ensure that all lines ing abstraction. There is a need for
* Fishwick stresses the use of aesthetic leading into a node are equidistant. Diehl computer scientists to embrace a wider
style and genre within traditional also quotes Brooks's idea that "software definition of aesthetics [28], one influ-
notations for computing artifacts. is invisible." Diehl's views on characteris- enced substantially by the arts. This cul-
Aesthetics in both computing tics and of design accurately reflect how the tural transition may be difficult but can
mathematics can be pushed in the vastdi-majority of computing professionals be gradually facilitated by the kind of
rection of artistic styles to augment view aesthetics and computing artifacts evolutionary collaboration described by
cognitive aesthetics. -as primarily mental constructs and,Prophet, if in which artists and scientists
* Diehl discusses the use of visualiza- not mental, focused on formal qualities work closely together. In such a fashion,
tion and a specific type of aesthetic ("formal" is used here as in the artist's all parties become accustomed to a
for software design and execution lexicon).
di- broader definition of aesthetics. The

agrams. There is already a great Fishwick

deal takes issue with confining need
the for this broader definition is re-
role of aesthetics to such a narrow inter-
of research on applying visualization, inforced by L6wgren's discussion of in-
and one particular sort of aesthetic
pretation and suggests that future repre- teraction qualities, as well as recent
(optimality), to software design.sentations for software models should movements in the design community
* Prophet defines a new type ofcaptureteam more personalized and stylis- [29,30], where emotion, style and visual
collaboration, a kind of "aesthetic tic forms, given that new technologies appeal are found to be equally as impor-
have increased our ability to afford
computing in action," in which trans- tant 3D
as function. There are additional re-

Fig. 7. Jane Prophet, Cell 3D Alife representations (a, left; b, right). (? Jane Prophet. Rendering design by Rob Saunders.)
_~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~)~: -_ ::"'. ?

Fishwick et al., Perspectives on Aesthetic Computing 139

This content downloaded from 161.116.100.134 on Sun, 04 Nov 2018 23:20:22 UTC
All use subject to https://about.jstor.org/terms
suits in the foundational cognitive role professionals will tend to shy away from
less our current state of progress. An il-
of metaphor [31]. If richer designs foster it. However, there is a line that stretches
lustrative example is our introduction of
a metaphorical understanding, then such from "no use" to "full use," if there the
candimension of utility as one of the ways
designs ought to be encouraged and in- be such unambiguously defined things. in which the fundamental values of the

vestigated. We further suggest that whatTo see this, we need to step back todisciplines
the of art and computing differ.
is relevant for product design is also rel-
definition of aesthetic computing: the ap- The most advanced integrative stage,
evant for the design of more abstract plication of art practice and theory to
according to Jantsch, is interdisciplinar-
computer programs, data structures and computing. ity. It involves direct cooperation in both
models. Both types of designs reflect a There is no reason why this application directions where the outcomes typically
necessary mixture and balance of form must be targeted on artifacts of high gen- could not be achieved entirely without
and function. eral utility. By exploring the boundaries any of the disciplines involved. It also en-
Certainly, optimal designs for achiev-and interstices of the "use range," we tails the formation of new concepts, prac-
think that we can enhance both art and
ing a goal or satisfying a particular pur- tices and values that transgress the
computing. Artists will become more fa-
pose are still useful when considering the traditional boundaries of these disci-
problem of design from the perspective miliar with elements of computing, such plines. It is our firm conviction that the
of performance. However, this functionalas data structures, programs and archi- encounter between art and computing
tectures, and even the core mathemati-holds the potential for interdisciplinarity
bias of design must be carefully balanced
with what one learns by exploring aes- cal structures upon which computinginisthis strong sense, and our ongoing di-
thetics in the arts: Design is also about
founded. For example, what artists pro- alogue might represent a step in that di-
user choice, culture and the freedom to duce [33,34] reflects a computing rection. es-
incorporate arbitrary interactive 2D sence,
or whether or not the result is of
3D icons to represent formal structure. immediately obvious utility. To the extent
Acknowledgments
Asking, for example, whether it is more that a piece makes one reflect, it is useful
The authors are indebted to Dagstuhl, Germany,
appropriate to use a square versus a cir-
in exploration, creativity and education,
where many of the ideas in this paper flourished in
cle in a design presupposes that there and
ex- so even the term "use" or "usability"a synergistic weeklong session. We also would like to
ists an optimal design condition, whenbecomes
in suspect. Thus, aesthetic com-acknowledge the Leonardo organization for their co-
sponsorship of the event. Paul Fishwick would like to
many instances, the design conditionputingis for the artist can range from soft-thank his graduate students in the RUBE Project
strongly oriented toward user preferenceware art (a fairly new movement defined(Minho Park,Jinho Lee and Hyunju Shim), the Na-
and not performance. Designs mustby beartists producing their own programstional Science Foundation, working via grant EIA-
0119532, and the Air Force Research Laboratory,
or languages) to representational art inworking via grant F30602-01-1-05920119532. Jane
useful, yet this does not imply universal
usability for an objective purpose, but which
in- the computing element becomesProphet's contribution is part of the Cell collabora-
tion and has been written following discussion with
stead utility for a segment of the popu-the subject of the art, rather than the ma-Mark d'Inverno, Peter Ride, Rob Saunders, Neil
lation. This type of free-form designterial
is for the art as in software art. It is Theise and KatrinaJungnickel. Cellresearch has been
common in automobiles, architecture conducted with awards from the Wellcome Trust
this ability to weave through the webs of
sciart, Shinkansen Future Physical "BioTech" and the
and kitchen products; they all must bothutility and computing that makes aes-Quintin Hogg Trust.
be usable and contain aesthetic qualitiesthetic computing a unique enterprise.
for whoever purchases them. We contend For the computer scientist, what the
References
that software and formal structures used artists produce will be fertile ground for
in computing are no different in this re-representation, interaction design and1. Aesthetic Computing Dagstuhl Seminar 02291,
15-19 July 2002, <http://www.dagstuhl.de/02291>.
gard. Representation, in general, has
human-computer interfaces in general.
always been driven by the state of tech-
While through software art some artists2. The Dagstuhl Aesthetic Computing Workshop,
"Aesthetic Computing Manifesto," Leonardo 36, No.
nology and the economy. When all one
are becoming computer scientists of a4 (2003) p. 255.
sort, we can also observe the converse sit-
has is clay, then clay marks must be used
3. M. Kelley, ed., Encyclopedia of Aesthetics, 4 vols. (Ox-
uation as some computer scientists be-
for representation. As our display and in- ford, U.K.: Oxford Univ. Press, 1998).
come artists.
teraction technologies evolve, our rep-
resentations can also advance, not to On a more general level, the en-4.J. Hadamard, The Psychology of Invention in the Math-
ematicalField (Princeton, NJ: Princeton Univ. Press,
replace older representations, but to aug-
counter between art and computing may 1945).
ment them in the same way that new me- be studied in terms of disciplinary rela-
5. D. Knuth, The METAFont Book (Reading, MA: Ad-
dia have augmented, or remediated [32],
tions.Jantsch [35], the originator of such
dison-Wesley, 1986).
old media. concepts as multi- and interdisciplinarity,
6. E. Gamma et al., Design Patterns: Elements of Reusable
views disciplinary integration as an evo- Object-Oriented Software (Reading, MA: Addison-
Wesley, 1995).
lutionary hierarchy. If a traditional disci-
CONCLUSIONS
plinary approach is specialization in
7. S. Card, J. Mackinlay and B. Shneiderman, eds.,
One of the key tensions in our discussion
isolation, then multidisciplinarity sim-Readings in Information Visualization: Using Vision to
Think (San Francisco, CA: Morgan Kaufmann, 1999).
is the interplay between that which artists
ply refers to adding different disciplines
produce versus that which computerwithout
sci- any direct cooperation betweenS. Diehl, ed., Software Visualization (New York:
8.
them. Springer Verlag, 2001).
entists produce. Artists have an agenda
based on a wide variety of styles and aes-The next step up the evolutionary lad- 9. J. Stasko et al., eds., Software Visualization: Pro-
gramming as a Multimedia Experience (Cambridge, MA:
thetics, from formalism and culturalder ex-is crossdisciplinarity, where one dis- MIT Press, 1998).
ploration to capturing social, political or supports the other within the
cipline
10. T. Kim,J. Lee and P. Fishwick, "A Two-Stage Mod-
economic aspects of phenomena. Com- other's own discipline. Our discussion eling and Simulation Process for Web-Based Model-
above, and the identification of fields
puter scientists primarily are after high- ing and Simulation," ACM Transactions on Modeling
such as software art and aesthetic com-
utility artifacts; if something is useless, and Simulation 12, No. 3, 230-248 (July 2002).

most mathematicians and computing puting, might suggest that this is more orFishwick, "Aesthetic Programming: Crafting
11. P.

140 Fishwick et al., Perspectives on Aesthetic Computing

This content downloaded from 161.116.100.134 on Sun, 04 Nov 2018 23:20:22 UTC
All use subject to https://about.jstor.org/terms
Personalized Software," Leonardo 35, No. 4, 383-390 puting Systems (CHI '94 Proceedings) (New York: 31. G. Lakoff and M.Johnson, Philosophy in the Flesh:
(2002). ACM Press, 1994) pp. 313-317. TheEmbodied Mind and Its Challenge to Western Thought
(NewYork: Basic Books, 1999).
12. J. Hopkins and P. Fishwick, "Synthetic Human 22. C. Ahlberg, C. Williamson and B. Shneiderman,
Agents for Modeling and Simulation," Proceedings of Dynamic Queries for Information Exploration: An Imple- 32.J. Bolter and R. Grusin, Remediation: Understand-
the IEEE 89, No. 2, 131-147 (February 2001). mentation and Evaluation. Human Factors in Comput- ing New Media (Cambridge, MA: MIT Press, 1999).
ing Systems (CHI '92 Proceedings) (New York: ACM
13.J. Hopkins and P. Fishwick, "Exploiting the Agent- 33. J. Maeda, Design by Numbers (Cambridge, MA:
Press, 1992) pp. 619-626.
Based Metaphor in Software Visualization using the MIT Press, 1999).
RUBE System," Journal of Visual Languages and Com- 23.J. L6wgren, "Sens-A-Patch: Interactive Visualiza-
puting 14, No. 1, 97-117 (February 2003). tion of Label Spaces," in E. Banissi et al., eds., Pro- 34. Processing Language: <http://www.processing.
ceedings of the Fifth International Conference on org>.
14. P. Fishwick, Simulation Model Design and Execution:
Information Visualization (IV2001) (Los Alamitos, CA:
Building Digital Worlds (Englewood Cliffs, NJ: Pren- 35. E. Jantsch, Technological Planning and Social Fu-
tice-Hall, 1995).
IEEE Computer Society, 2001) pp. 7-12. Web:
tures (London: Associated Business Programmes Ltd,
<http://webzone.k3.mah.se/k3jolo/Sens-A-Patch>.
1972).
15. F.P. Brooks, "No Silver Bullet," IEEE Computer20,
24. A. Henderson andJ. Harris, "Beyond Formalism:
No. 4,10-19 (April 1987).
The Art and Science of Designing Pliant Systems," in
16. aiCall StackAnalyzer, AbsInt GmbH <http://www. K. Kaasgard, ed., SoftwareDesign and Usability (Copen- Manuscript received 15 October 2003.
aicall.de>. hagen, Denmark: CBS Press, 2000).

25.His-
17. T. Zimmermann, S. Diehl and A. Zeller, "How T. Freiesleben, M. Knapek and H. Moberg, Corresponding author Paul Fishwick is with
Hazed
toryJustifies System Architecture," Proceedings of In-Windows, 2001 <http://webzone.k3.mah.se/ the Computer and Information Science De-
kid01016/projekt_forskningstema.htm>.
ternational Workshop on Principles of Software Evolution partment at the University of Florida.
IWPSE 2003, Helsinki, Finland, 1-2 September 2003
(IEEE Computer Society Press, 2003). 26. J. Khaslavsky and N. Shedroff, "Understanding
the Seductive Experience," Communications of the Stephan Diehl is with the Computer Science
18. D. Goodsell, <http://www.scripps.edu/pub/
ACM42, No. 5, 45-49 (1999). Department at the Catholic University Eich-
goodsell/>. statt, Germany.
27. Visual Thesaurus: <http://www.plumbdesign.
19. Sol LeWitt, "Paragraphs on Conceptual Art," Art- com/thesaurus>.
forum 5, No. 10, 79-83 (June 1967). Jane Prophet is Director of the Centre for Arts
28. D. Norman, Emotional Design: Why We Love (or
20.J. Kosuth, in G. Guercio, ed., Art after Philosophy Hate) Everyday Things (NewYork: Basic Books, 2003).
Research Technology and Education
and After: Collected Writing 1966-1990 (Cambridge, (CARTE), University of Westminster, London.
MA: MIT Press, 1991). 29. Norman [28].

21. C. Ahlberg and B. Shneiderman, "Visual Infor- 30. P.Jordan, DesigningPleasurable Products: An Intro-
Jonas L6wgren is with the School of Arts and
mation Seeking: Tight Coupling of Dynamic Query duction to the New Human Factors (Boca Raton, FL: Communication, Malmd University, Malmi,
Filters with Starfield Displays," Human Factors in Com- CRC Press, 2000). Sweden.

I ,wjwl !g i! i:i.iw ...... 1

Live Art and Science on the Internet

The Internet has become a venue and medium for art as a means to broadcast ideas to a worldwide audi-
ence. Leonardo and Guest Editor Martha Wilson seek texts on the subject of "Live Art and Science on the
Internet" for a series of special sections in the international journal Leonardo, both in print and online.

As artists and others produce live art on the Internet, liveness, presence, mediatization, online activism,
surveillance and identity/gender, among other issues, are being explored. We seek texts documenting such
work, as well as texts on the history of this field of practice and on the vocabulary being used to describe it.
We also seek texts from scientists who have used the Internet to conduct science investigations live on-line.

Guest Editor Martha Wilson and her peer review committee seek Statements (500 words plus one image
describing one work), Notes (2,500 words plus 6 images describing a body of work), Galleries (750-word
curator's introduction plus up to 10 images by individual artists, each with a 200-word caption) and
Articles (5,000 words plus 12 images). Texts describing the work of a living artist or scientist must be
written by the artist or scientist him/herself, with a co-author if necessary.

This call for papers is open for 2004, 2005 and 2006.

Please send an initial statement of interest with a brief explanation of your project to Martha Wilson:
<[email protected]>. For author guidelines, follow the link "Info for Authors" on Leonardo
On-Line <www.leonardo.info>.

Fishwick et al., Perspectives on Aesthetic Computing 141

This content downloaded from 161.116.100.134 on Sun, 04 Nov 2018 23:20:22 UTC
All use subject to https://about.jstor.org/terms