Design and semantics of form and movement (DeSForM 2006)

Keynote speakers

Abstract

This presentation focuses on the development of a model used to conceptualise and communicate the way in which consumers respond to product appearance. This work was initially inspired by the observation that despite a wide array of literature from many sources addressing aspects of product appearance, there was little to integrate this body of knowledge. We aimed to provide an initial point of reference to integrate well known concepts and raise awareness of lesser known texts. We also wanted to present a generalised framework, which would enable better explanation of complex concepts, for both a design and a non-design audience.

When a new technology -in an abstract sense -is inserted into an existing socio-cultural fabric, we have to negotiate how this entry can be made. In this process, technology be-comes tangible and has a new meaning assigned to it. We are quite keen to be involved in projects which embody this process as there is a lot of new ground to break. Sometimes these projects are more experimental, as in our interactive installations, other times they are more pragmatic, such as kiosk interfaces.

The essence of interaction design is about control, in multiple senses. It has to predict and control the process and outcome of the interaction between the user (or viewer, participant) and the artifact. The user, driven by his/her interest and prompted by the cue from the artifact, does something, then the artifact has to provide a good response, so the user gets a strong sense of control, the user feels s/he is in the driver's seat. In order to achieve this, interaction design should have a clear logic.

To what extent this logic should be made obvious, depends on the context. In practical applications there shouldn't be any ambiguity, as it will lead to frustration and failure of the intended goal, such as, for example, a quick transaction.

In the context of art and entertainment, the process of discovery can be part of the reason of existence of a project, thus the underlying logic doesn't have to be made as explicit. Nevertheless, experience has taught us that people have a very limited attention span. Demanding too much commitment from an audience to reap a "reward" may result in loosing the audience before it ever gets any reward. Therefore, we have come to like interaction design that's obvious, simple and delivers delight.

Our approach to interactivity is strongly informed by our background in product design. Using a product or moving through an environment is inherently a process of reciprocal actions. We see the application of digital interactive technology as an extension of this reciprocal nature of products and environments. So we apply the same thinking. We design for and around the users. We try to predict their behavior and design to guide them through a series of prescribed actions, which trigger desired events.

Context is always the starting point of our design. Each of our solutions is the result of a thorough study into the unique context of the project. What unifies the various projects is our focus on human behavior and our pursuit of designs that achieve both simplicity and emotional resonance. We strongly believe that to create a coherent user experience a inte-grated approach to hardware and software design is crucial.

The phrase " reflection to reflex action" was coined by Paul Virilio to describe the nature of interactive media, which tries to generate a reflex action from the user/viewer as opposed to reflection as in the case of traditional media.

The framework: how objects are read

The framework is based around a simple communications model, which suggests that designers have intentions about how the products that they create will be perceived.

This intent is embodied in the form of the artefacts that the designer creates, which will then be interpreted either immediately, or more likely at a later date, in a different context by potential users and consumers. The model is primarily based around the visual senses.

As the transmitter of the designer's 'message', the artefact is essentially nothing more than a collection of surfaces, interfaces, images, textures, transitions, colours and materials. For convenience, the consumer's interpretation of these has been divided into sensing, cognition, affect and behaviour, whilst recognising that in practice these tend to happen simultaneously. Attention has been focused on the complexities of the consumer's cognitive response, or the judgements that the consumer makes about the product based on the information perceived by the sense. These judgements can be grouped into three classifications:

• Aesthetic impression: or the sensation that results from the perception of attractiveness (or unattractiveness) as a result of viewing a product.

• Semantic interpretation: what a product is seen to say about its function, mode-of-use and qualities.

• Symbolic association: the perception of what a product says about its owner or user, the personal and social significance attached to the design.

In this presentation each of these will be described, with illustrative examples.

The future of product semantics …

As a relatively new body of knowledge, there remains much to be done in understanding and classifying the various elements of consumer response. Work in consumer response remains largely unconnected to research in the design domain. Of particular interest is the evolving interplay between aesthetic impression, semantic interpretation and symbolic association.

Arguably, the 0th Century has been dominated by a Modernist perspective on Semantic Interpretation; which emphasises understanding of form over exuberance.

However, growth of mega-brands and increasing brand awareness has raised the importance of the social significance that may be associated with artefacts. Finally, over the last 5-10 years, there has been a steady return to decoration, and the application of pattern. This is in part a reflection of the availability of new production methods and may yet be a short term fashion. It could also perhaps indicate a longer term progression from the modernist style towards an emphasis on the emotional over the rational in product design.

Seeing things: consumer response to product appearance concerned with how an interface feels, how it behaves and the experience it offers.

Physical computing is one area of interaction design that explores the possibility of making intangible data tangible -as a way to inform the user of its functionality. Up until recently, digital devices tended to indicate activity through a set of pulsing LED's and information via icons on a screen. Physical computing, on the other hand, looks for new ways to make devices express their functionality via physical elements that move, adapt or morph depending on the functionality being conveyed. Embedded microprocessors and sensors mean that devices can now sense the environment, the user and the presence of other devices -and respond accordingly. They can now adapt to their environment, and therefore deliver only the appropriate functionality -at the appropriate time.

Keynote speakers Heather Martin, Copenhagen Institute of Interaction Design, Denmark

Her focus is on creating new forms of tangible interfaces for products, services and environments & on tangible interfaces for objects and spaces. She has worked on creating new interfaces for various clients including Sony (2004) and for the Italian lighting manufacturer Artemide (2005). Before joining IDII she worked at IDEO in London as a senior interaction designer and project manager. From 2000 to 2001 she was a project manager and creative lead for IDEO on the Prada New York Epicentre done in collaboration with Rem Koolhaas and OMA/AMO, where she helped to design and develop the RFID enabled interactive dressing rooms. From 2002 to 2003 she led a worldwide team at IDEO on the creation of a wireless LAN in-flight entertainment device for use on-board private VIP airplanes by Lufthansa Technik. Recently she co-founded the Copenhagen Institute of Interaction Design (CIID).

Reflection to Reflex Action

The design of interactive systems in digitally enhanced products and environments Introduction Design, once codified in physical infrastructure is difficult to reverse, and it is rarely up for public debate.

Victor Margolin

The fish trap exists because of the fish. In essence, form has no meaning; it is an invitation, a window to possibility. Meaning resides, and is latent within us, in the relationships we perceive and cultivate in our minds and through what we negotiate with others. This is the paradox that makes the stable attribution of meaning to anything so difficult, whether it is form, color or motion.

People see more, they see differently, and thus, they challenge existing limitations. Meaning is emergent, and so forms trigger multiple shifting associative meanings.

However, we subvert meaning by designing it into form. this "language" has migrated far beyond the borders of California. It has been appropriated by clubbers, break dancers and YouTubers all over the world, but has lost its linguistic qualities in these realms, the very reason why it exists. If one were to relocate a Toronto club goer to the streets of L.A. right now, after the proliferation of its public popularity, the dance would convey gibberish to those who are native to the language, and the clubber would be met with anything but applause. Time Matters. Through the reshaping of one form, prisoners can satisfy different levels of human needs. The toothbrush can be multiple things at once. At a given time it can be a shiv, a tool for cleaning teeth, or a carved piece of art. One form can participate in events that contribute to fulfilling one's physiological needs, esteem needs in social settings, or the form itself can be a medium of self-actualization.

It is our motivations that curate the relationships we engage in to trigger revelations of what can be, and what meanings those relationships bring. We perceive context and conditions that we are participants within, and we forge our own meanings; in turn we can contribute this meaning in a shared dialogue to negotiate normative ones.

What can we learn from this perceptive ability of people and the potentiality of form?

The field of ethnography poses an interesting point in this regard. Ethnography is the study of how "alternative cultures do the same things differently." [Haviland] Cultures develop arrays of tools and networks of infrastructure that support their survival. We all have the same physiological needs, yet we develop many differing tools and support systems with various permutations to achieve them. Forms are driven and shaped by goals and intentions of the mind. Artificial forms are the facilitators of our daily lives, they dictate how easily and fluidly we can go about our day: because they are, we can.

How can objects be sensitive to our goals? Can they be literate to our actions? How can we better equip them to help us achieve?

Will a language of form and movement inevitably produce a Universal Homogenous dialect, or alternatively will a heterogeneous explosion of dialects evolve?

Can contextual mapping and meaning mapping over time be achieved to form a living document capturing a language unfolding in real time.

Form Collapses Possibility

Forms alone do not collapse possibility, but the essentialist framing of forms that associate a sole meaning to a form neuter ones perception to imagine what the form can do or be. Forms will be a platform of dialogue: they will be simultaneously the substrate of a question or goal, the expression of a possible answer, and a passport to exploration of possibility.

What else can this be? How can it help me achieve my goals right now, how can I leverage this? I think therefore it is.

Kevin Kelly of Ideo has been developing a blog "Street

Use" [Kelly]

Steffan Larsson

The idea of forms having multiple meanings can be substantiated by the fact that meaning is emergent. This is why we cannot design semantics in a systematic and scientific way, for they are individually variable, as well as culturally and socially formed. The failure of Esperanto and the success of Creole and Pidgin languages [Diamond] illustrate this vividly.

Have we considered how this inherent human characteristic of emergence can be harnessed, socially and individually through forms? "Real Time Graz"[Ratti] is a project where GPS mapping of cell phone trajectories is used to illustrate the nomadic portrait of people in the city; this information describes in a rather low resolution how people use space. These traces of flow are then used to inform the emerging shape of the city to complement how space is actually used.

"The BCN Formula Game" [Hubers] has taken this further by developing software that generates building proposals for Barcelona in real time through modeling software that creates structures in response to flows of people, traffic and commercial activities. Although both of these platforms embrace emergence they are crude examples given the oncoming ability and impact of responsive and adaptive technologies that have been developed to date, not to mention the potential developments that could support such changes in the future.

How can our traces influence the language of form and movement to best-fit context and particular actors in real time?

What if an object's form or gesture was a word and its definition was based on how people see, use and understand it in relation to its current context? and sharing. When the clouds lower and darken, the wind picks up, and the leaves dance, we don't think, "what is happening?" We run for cover or pull out our umbrella.

Meanings Will Develop

It will take time to learn to "read" newly emerging responsive architectures and objects as innately as we respond to natural events. Only after we have interfaced with them on an ongoing basis will we be able to effortlessly understand them. For example, being able to understand the state of the internal social dynamics of a building by observing its shape from down the street, or recognizing the state of the contents of a container by its visual attitude from across the room at a glance. This engagement creates a concept of meaning which influences behavior according to personal motivations.

We engage in the act of the social negotiation of meaning with others, we establish a normative meaning that is harvested from group consensus. These behaviors, if beneficial, are codified into forms such as artifacts, spaces, systems and media, so as to extend its benefit to people, and become once again percepts to feed this ongoing cycle. and clothing. We continue to make observations and build upon the constructs that we have built before, and in doing so have created forms and meanings that are further and further from survival, but which carry no less meaning. This process is automated in the former example of the "BNC Formula Game" [Hubers] where forms are generated according to what is happening.

This process is cyclical in nature, from percept to percept, our observations and behaviors are transformed into things that can be seen, heard and felt.

Emerging Perceptions-A New Literacy

DeBeule, Bergen

The search for truth is in one way hard and another way easy-for it is evident in that no one of us can master it fully, nor miss it wholly. Each one of us adds a little to our knowledge of nature, and from all the facts assembled arises a certain grandeur.

Awareness and Memory

We leave traces of ourselves behind as every moment what make us who we are. Our inquiry will be visible, our expressed knowledge will be legible, and our mastery will be explicit to objects and people. The sensory perception capability of things is directly proportional to its ability to read not only the physical but also the invisible relationships that sculpt our environment.

Most of today's objects and spaces are blind, deaf and mute, however, scores of new capabilities are migrating to the shores and guts of products, and are being woven into the very fabric of architecture. This will result in more than just a perceptual extension of us. This distributed sensory organ will read, store, reflect and respond expressively to us in a way that is beneficial and valuable.

It's not about the objects or spaces themselves, but of them as images, products of what happens in-between.

They will increasingly become a form of socio-cultural barometers that aid in the flows in ways that they have How can we enable a co-evolution and interplay between our personal traces, and objects and spaces?

How will our traces (a form of Social Capital), our digital shadows, interact with each other? How will our pasts converse to enhance our present physical experience?

How will our present physical, situational and emotional context contribute to what is displaying?

Expression And Literacy

Blogs such as Information Aesthetics and We Make Money What if the projects, the Objects themselves circulate and share these developments?

What can we learn from heraldry, patina and natural expression, such as a ripening fruit, as a pre-existing historical or natural form of communication?

Communication between objects will, in the beginning, work in chorus to combine sensor capabilities, processing power, storage, expressions, behaviors, and share memories, to understand and render responses to meaning. Our meanings will be derived from our behaviors in context, read by objects, which become a responsive porous vessel, a new form of media. They are becoming dynamic content.

This language of form and movement has been around forever. Throughout the course of a day we may employ about a thousand words, however, we express 0,000 facial expressions, and 5,000 hand and bodily gestures [Danesi]. Most of our communication happens through our visual interpretation of form and movement. We can "read" clouds, water and the landscapes of rock and of flesh. Undoubtedly, if we were to propose a semantic framework for form and movement, we should employ the wealth of experience in our literacy of displayed behavior.

Expressive capabilities are already migrating from screens to the surfaces and movements of objects for public use.

Ambient Devices [], a Massachusetts based startup,

pioneered by graduates of the MIT media lab. Their products enable data from the internet to be translated into personally programmed color and movements in forms such as The Orb, which can display stock information, weather or any other information from the Ambient Information Network. The Inflating USB key [Komissarov] alters its size in relation to how much data is stored on it. Email erosion [Ham; Muilenberg] is a more art centric piece that aims to display relationships that are usually invisible. Erosion is a project that uses a water-soluble block as a display; incoming emails cause flows of water to alter the form of the block in proportion to the flows of email. Alternatively, the broadband cord project [Weiser; Seely Brown] displays data packet flows in physical space through servos that shake the cord in accordance to Internet activity. These projects are examples of how this language is alive and its evolution underway.

What can we learn from these isolated developments when viewed as examples of a larger language?

Can we allow these solutions to contribute to and fuel negotiations of the evolution of a sensory language?

It is also important to ask: Who shapes the message, who shapes the medium? How does the behavior of form affect and influence the behavior of people? Can we develop a notation system that formalizes and documents form and motion?

Open Ending

Due to the emergent properties of meaning we cannot design semantics in a systematic and scientific way, for they are individually variable, culturally and socially formed.

Instead, we should provide the conditions and tools that create a platform for semantics to unfold naturally, because people already shape form and produce meaning around them continually. If an enhanced dialogue with the environment is to evolve through a language of form and movement, then it will be sensitive and responsive to context and identities of people in the transformative process of meaning making.

How can we begin to bring into motion a platform to allow the public to assist in the creation and evolution of this language?

How can an open source, open content community be shaped to best monitor, review and engage in the developments of this language?

Enabling characteristics that thrive off of the morphology of meaning, rather than being threatened by it, turns a very difficult problem into an opportunity. This approach includes and complements the ongoing development of forms that communicate. The future of design is not in form or movement, color or texture, but in the temporal curation of relationships within changing landscape of social practice.

Introduction

Traditionally, man makes tools to extend their abilities.

The main disadvantage of the resulting tools is the lack of flexibility, due to the inability to react to changes in the environment, which has to be controlled and kept constant as much as possible. Integrating intelligence into the product should make it able to perceive the environment and to act accordingly, in a benevolent way. This is the way a product should behave, adapting to the variety of users in the diversity of contexts in order to support their social well-being.

Contrary to all expectations, this idea is not new. We can find such a principle in the initial development of the Karakuri Ningyo, the Japanese mechanical puppets, started in the Edo Period. The famous ChaHakobi, a tea server puppet, was developed to seduce the guest, by bringing tea in an abstractive way, through the stylisation of the movements of the face, deepness of communication between the guest and the host. This is the first example of a product adapting to the users behaviour and conveying benevolence stance. We believe that, since ancient times, people aspire to transformations and self-achievement. If any product could support this aspiration, how could it bring its contribution?

The theory of the golden ratio has been a cornerstone of the geometric approach to aesthetics in architecture,

Method

Designing a new product can be seen as an opportunity to investigate the consumer's needs and expectations as well as the behaviours in particular contexts of the daily life.

In the process of designing an automatic food dispenser, factors that have been found to influence the perception of the delivery modes. In this step, we evaluated the existing delivery modes for both populations, thus taking into account the semantic attributes found in section 1.

In the third step, we selected the most relevant food delivery modes for each population and tried to measure the influence of each of the 5 factors independently and correlatedly.

Needs and expectations regarding the food vending machine

The first market study on this topic was done on 0 participants, 0 in Japan and 0 in France. The results we gathered showed that the main drawbacks of the vending machines are their connotations; being seen as cold and not user-friendly by both people in France and Japan, especially among the female participants. In fact, vending machines are perceived as not creating any positive experience for the user. It only completes the function of "food delivery". Vending machines delivery modes involve for instance throwing the goods purchased in the retrieval zone of the machine, which may be inaccessible to some customers. This may be the case because the machines have been designed that way to prevent from vandalism and robbery. This is conveying as well as inconvenience a feeling of rudeness that does not offer a pleasurable moment to the consumer. A customer that wants to buy a product that is selling on the top shelf in the vending machine, puts his money in the coin slot and then the vending machine drops the product from the top shelf all the way down to the very bottom of the machine to deliver the food. The main expectations of the participants are to feel carefulness, simplicity, elegance, respect and benevolence during interaction with the food vending machine.

New innovations like the "soft drop", which is a lift that conveys goods more carefully in the retrieval zone of the machines, are being made recently in Japan and France.

Also, some Japanese makers, through a universal design approach, are now proposing a more user-friendly food vending machine that catered for the needs of elderly consumers (numerous in Japan) and women by making the product retrieval easily accessible. But still few have been done to adapt the delivery modes to the diversity of users and major aspirations.

Existing food delivery modes and semantics of movement

The following section presents a selection of existing food delivery modes that can be observed in food markets, fast-food and none fast-food restaurants. Each mode is a schematised drawing associated with photography.

Physical, observable similarities and differences between the delivery modes contribute to determine the semantic attributes associated with each mode or group of modes. The pictures of the gestures were presented to participants who were asked to classify them by preference and ascertain whether they have a positive or negative impression as well as the meaning they convey. The main advantages of vending machines are obviously to get goods immediately and to avoid waiting queues.

However, taking into account fast service and respectful handling of the goods it can be interesting to exploit the speed variations (acceleration and deceleration) to evoke a meaningful gesture to adapt to people's expectations and contexts.

Outputs of the preferences and semantic analysis 3.1 Protocol

In the previous sections, 0 participants were involved through a questionnaire method; 0 Japanese ( females/ 1 males, employed/1 students), and 0 French (10 females/10 males, 1 employed/7 students).

As shown in the figures below, participants have been asked to express their preferences, their positive impressions and to express the connotative values conveyed by each presented gesture.

Results

To experiment with videos was very difficult and cannot be considered as being relevant to precisely determine what the impressions of people would be in a real context.

The participants had some difficulties transferring the action in a real situation of interaction with a vending machine. However, this short experiment allowed to confirm the cultural preferences previously described. Japanese participants (% ) had a better impression again on the A gesture and French participants (67%) had a better impression again on the H gesture. However the impression was varying according to the speed factor.

It would have been interesting to experiment using acceleration or deceleration of speeds of movement.

For this experiment the speed of the movement was maintained constant. We proposed three speed variations : V1 = slow; V = average speed; V = fast, to evaluate a potential impact of the speed factor.

Regarding the influence of speed, we observed that speed has an important impact. In fact the delivery modes that were previously very appreciated by Japanese and French people were badly perceived after a drastic augmentation of the speed (V). On the other hand, slow speed (V1), was not perceived as negatively by participants.

We also observed that a minority of people selected as positive the "G gesture" previously perceived as looking efficient associate with a fast speed, V. According to their verbalizations, they appreciated to have the fastest service as possible as it is a vending machine.

In this section we describe how the designers enacted their design sketches using gestures to make them behave.

Often they incorporated these enactments within acts of speech, but, as we will see, some enactments had no signifying word or verbal counterpart during the sessions.

The enactments enhanced what the designers wanted to communicate, much in the same way that sketches provide simplified visualizations of a complex design proposal. In this section we present examples of how such communicative enactments were performed.

Hypothesis 1. The first hypothesis H0 cannot be rejected, because there is no significant difference between the satisfaction of the designers of their own sculptures made through sketching or through gesturing, see Table 7.

Preferences

From the first question, major preferences observed in Japan and France seem not to converge at all. Indeed, while gestures A and B are by far more preferred by Japanese people, French participants timidly announce their preference quite equally for the modes E, L and H. The modes selected in the first place by the Japanese participants are gestures that were previously classified as "Respectful" and "Friendly" (see figure 1). On the other hand, the gestures chosen by the French participants were classified as rather "Stylish" and "Friendly". Apparently, Japanese participants' preferences are focusing on two-hand gestures, as they convey a meaning of respect, particularly important in Japan, while French preferences are mostly found among "Stylish" gestures using "one-hand".

Regarding the mode E which is the "bell jar service" that was a direct metaphor of a food delivery mode, it has been appreciated by both groups of participants, also less strongly by the Japanese. In this case, people could identify more rapidly the meaning conveyed by this metaphor that brought a positive impression to every participant as it was clearly understood. B and C gestures have been preferred firstly by these participants; The "B" movement direction has been largely chosen by the French participants while the "C" movement direction has been preferred by the Japanese participants. We presume that they have been chosen firstly, as the final position of the device is the same and convey and impression of easiness as the good is coming directly in the hands of the person.

We also assume that they have been selected as they both allow to have an upper view of the goods purchased before it comes to hands and thus allow to visually check the aspect of the food.

Semantic values

Regarding the semantic evaluation of the delivery modes, In order to insure if there is a gap between the French and the Japanese individual perception of respectful gestures, considering the gap found in the use of language, we wanted to know if the signs were understood in the same way by the French and Japanese participants regarding the two keywords chosen : Respectful and Friendly. We can advance that if we want to express a "Respectful gesture" to people in France and Japan, we will not use the same physical signs to convey this connotation in both countries as they can be understood differently by each population. However we can use the "bell jar" metaphor to convey a positive impression to all people. In this part, however three modes have been selected among them, two are the most appreciated by French and Japanese, they are A (very appreciated by Japanese), H(very appreciated by French) and one that was less appreciated by both populations but that was considered as very efficient, G (less appreciated by Japanese and French people). Participants were asked to give their impressions while watching the videos (negative -, neutral = , positive +).

Proposals

Taking into account the results of parts . and ., we propose two different delivery gestures for Japanese people (see figure 17) and for French people (see figure 1) The A mode has been perceived as being the most benevolent delivery mode for Japanese people. As previously shown, thanks to the market survey, Japanese people perceived the vending machines as being very "cold", and the A gesture has been selected as the most friendly gesture by Japanese participants (see figure 1). The "A gesture" is also the favourite gesture in the preferences of Japanese participants.

As for French participants, the benevolent gesture selected is the "E gesture", the bell jar gesture. In fact, according to the market survey, vending machines are seen as "boring" by many French people and the "E gesture" has been selected as the most surprising and friendly gesture (see figure 1) by French people. "E gesture" is a good balance between the "Surprise" and "Friendliness" for French people. The "E gesture" is also one of the favourite gestures in the preferences of the French participants.

Conclusion

As products are dedicated to people, and people are invited to act because a design fits their physical measures and skills, it is to the responsibility of the designers to propose intuitive and context adapted systems for people.

In the present study done within the early stage on a food vending machine design, three questions have been treated.

The first one concerned the expectations of the users.

The semantic attributes "respectful, stylish, friendly, efficient and surprising" are representing the expectations of both Japanese and French participants regarding food vending machines. Secondly, each of the existing food delivery mode (manual as well as automatic modes) has been evaluated through a semantic differential by both Japanese and French population. The outputs bring different results for Japanese and French people. While Japanese participants put in a very high position two respectful delivery modes, both involving a two-hand gesture, French participants focus on more stylish and surprising modes, involving one-hand gestures.

The preferences analyses led to 5 factors, influencing delivery mode semantics. Beside the one-or two-hand system, the position of the food and the surface contact with food seem to be discriminative as well. The grasping mode (Referenced as K) is perceived as being the less attractive for both populations. Finally, the speed and the direction of the movement have been found as factors influencing the perception of the benevolence of the delivery modes. These 5 factors impact the perception of quality and influence the acceptability of the final product by the final user. We will use these factors on real mock-ups that are under process, in order to experiment more objectively the variation of the speed and direction of movement for each gesture, and the construction of experience for the user, in each mode. We also try to seek the relationship between factors and perception of action for the user. It should help to seek how to support the user to anticipate what to do and how to do each task requested for the vending machine. We tried proposing to contribute, within the design field, giving keys for making vending machine systems more attractive and benevolent to the users. considers the interaction people-objects ontologically as a conversation, as a reflection of emotion through movement, and discusses how behaviour is connected to emotion in design.

Tools for the observation of emotion and behaviour were compared. The tools include Conversation Analysis, Laban Notation, Thinking Out Loud, and the designer's note taking. A new tool was suggested, Feeling Out Loud, for the specific recognition of emotion in users. All tools were used to notate the same interactions between people and objects in controlled experiments. The findings suggest that a combination of these tools can be very useful for studying users' emotional responses, and their effect on behaviour in the interaction with products. In the cases in which behaviour was believed to be connected with emotion, Laban Notation and discussion with the choreographer were very useful for revealing a connection between the emotional response obtained from Feeling Out Loud, with the resulting behaviour.

Further research will include both techniques in order to assess more thoroughly the connection between emotion and behaviour.

The complexity of behaviour renders it impossible to find one tool that offers a thorough analysis from different perspectives. This study suggests that tools can be used for different purposes and that a combination of them offers a more complete analysis.

Note taking by designers was not considered as useful as expected. Feedback from other disciplines and from the users proved much more revealing. Re-watching the behaviour on tape proved to be a relevant research tool for design practice. In the cases in which behaviour was believed to be connected with emotion, Laban Notation and discussion with the choreographer were very useful for connecting the emotional response obtained from FOL, with the resulting behaviour. Further research could use both techniques in order to assess more thoroughly the connection between emotion and behaviour.

These observations and comparisons fall short from offering a detailed description of behaviour. However, they are an initial step in organising the tools and identifying the areas in which they can be used in design. For instance, one of the designers expressed his serious interest on learning Laban Notation, because he would find it very useful for studying movement in his product designs.

Finally, the tools can be very useful for design research and practice, but there remains an important need for researchers to develop a sensitivity to people's needs, and desires, as well as the hidden beauty in behaviour.

Since interaction is a fast growing design specialization, more and more design issues come forward that are considered to be essential in interaction focused design. In this paper we highlighted (1) an example of product design as the creation of an event, with () the potential of movement and human body language, as this brings forward a very rich vocabulary for interactive design.

We literally moved through the project we presented in this paper. By doing so we iteratively developed insights and integrated discoveries. This congregated into a design that, again literally, incorporated many different interaction design issues in a 'moving' way.

We believe, based on our experience, that the human body incorporates a great source of undiscovered opportunities for design, especially when it comes to the design of interactive products. Our body language contains a vocabulary that links more directly to design than we might think, which can be read in many possible meanings of the words. Fig. 3 The concept, a screen which motivates discovering each other by touch and meeting in a totally new way They can be divided into three groups, depending on the determining factor in this interaction. Historically first comes the understanding that perception precedes and may provoke an action; a different view suggests 1 As an agent is understood any object or subject that can express own behaviour, e.g. human, animal, or animat.

that actions determine our perceptions; recently more evidence suggests the perception-action unity.

The Information-processing view on the perception-action interplay postulates that perception precedes action.

Even though it has been established more than a century ago by Donders [5] it remains to be a widely accepted

We began our research with an inappropriate presumption;

that human personality could be defined and summarised to help us apply human qualities to 'productality' designs.

We soon realised that it was not possible to define or apply human qualities to products because inanimate objects cannot have a personality but they can reflect a particular personality characteristic back to the user. To help us consider these quali-ties in product design terms, new methods tools and techniques have to be created and implemented. We have undertaken this task with the introduction of a new descrip-tive term; 'productality' as opposed to product personality (see section .). . In essence, designers need to understand more of the complexity of 'self' [7] and the narrative of 'flow' [] to be able to design meaningful and rewarding 'productality' for others.

The method, tools and techniques we have developed represent the real value from this research and we believe they will begin to help bridge the design practitioner's transition from conventional industrial design practice, to a position of understanding about designing with behaviour. We hope to have shown that despite the eclectic process of researching this complex subject, it can be broken down into meaningful chunks without losing too much coherence, sophistication and elegance.

Presently we are half way through our yearlong brief to better understand 'produc-tality'. The research is at a very early stage in creating methods and examples of ways to understand the field of practice where products can act as both mirrors and beacons of personality characteristics.

This project is not just quirky styling. No one knows the potential benefits that 'productality' can have when applied We know from previous research sponsored by Philips involving the CfDR that movement is extremely powerful in gaining attention and communicating various types of information to the user through one-way interaction.

We anticipated that the two-way interaction process is far more complex and our experience suggests that it is not just twice as complex rather more exponential.

As a result of the research conducted so far it has been established that understanding how to design with behaviour has much to offer in enhancing relationships between users and products. In order to achieve the goal of designing 'productality' a broad range of topics has been studied and our research has opened up many possible avenues of exploration, which often demanded an eclectic response on our part. In a relatively open field of enquiry such as this, where much is to be done [], our decisions concerning the direction of the project have been made on the basis of opportunism, resource availability and the pragmatics of promoting the role of the design practitioner. We therefore see the benefit of our work to date as an exploration, which will hopefully create a bridge-head for further study.

As you can see in the video, the children play on a variety of ways with the tiles. They play for a long time, together, and are physically very active. These are promising observations, but they can't be supported yet, when looking at the long term use. ; it is not a fixed dictionary that we can use for translations. This knowledge is organic, dynamic and developed over time. As such, it is obviously different from one culture to another and is projected to objects that we see for the first time: we try to put them in already existing category. This is the space where ontology and culture resist the design of narrative propositions. In this respect, how can we design for more interpretative appropriation5?

The encounter with tangible design invites the user to experience and develop skills through interaction;

Naturally, we are more familiar with physical objects and qualities of materials, then with software behaviors and abstract systems. Tangible design has the familiar physical qualities, which invite the user to explore directly the unfamiliar function. Design should shape the object, but also consider the exploratory interactions of discovering it.

Therefore, the mission of design is to expose through tangible design the logic of the innovative functions, but also to design the process of discovering through physical interaction.

Background

Design Research has often highlighted the importance of developing a better understanding of human behaviour in order to improve user experience (Frascara, 00; Jordan, 000; Laurel, 00). Some research has focused on observing product users and suggesting ways in which their experiences may be improved. A number of tools have been developed to assist in these observations (Don & Petrick, 00; Ireland, 00; Laurel, 00; Plowman, 00; Purpura, 00).

In order to study behaviour, it is necessary to understand how it is motivated. One way is to relate it to emotion.

Psychologists agree that the main role of emotion is to motivate and control behaviour (Cornelius,16;Frijda,16; Lattal & Chase, 00).

Currently, there is a lack of a coherent structure or framework that addresses the way in which behaviour is affected by the emotions products elicit. Moreover, Edgar R. Rodríguez Ramírez [email protected] there is no consistent way in which behaviour may be reported in order to share information inside and across organisations. Often, researchers are required to verbally explain their observations, and as 'digested' accounts of observed events are no longer objective descriptions.

Furthermore, reports of behaviour are limited in that they focus on only some aspects of behaviour, and because the complexity of behaviour renders it virtually impossible to explain every single detail.

This indicates two main challenges. Firstly, to offer a more structured description or framework of the connection between emotion and behaviour in design. Secondly, and in order to work on the latter point, to develop a consistent way to report observed behaviour that, even if it presents a version of behaviour, it is also open to interpretation and is closer to reality. The first point requires discussion broader than this paper would allow. However, this paper attempts to offer an introduction to the subject.

These points bring up further challenges. Is it possible to encompass a report about behaviour that is closer to reality? Reality according to whom? Even if behaviour is reported through media such as video, there are factors that are not included, such as people's thoughts and feelings; even the angle of the camera influences what we see and how we see it. Furthermore, what behaviour we see depends on how we look at it. A study of behaviour could start by studying its relationship to emotion, and how emotion affects it. This approach would encompass only this side of the equation, and although limited to the connection emotion-behaviour, it would cover an important part of the process.

Observation has been stressed as a necessary way to discover information about users. The inclusion of other tools can be helpful. Interviews can offer insightful information about users. However, many researchers agree that observation expands the gathering of information by making manifest details that users are many times not aware of. However, observation by itself is not sufficient input for developing a relevant understanding of the complexity of behaviour.

If seen under a causal perspective, emotion may be considered as a determinant of behaviour. However, there seems to be a closer relationship between emotion and behaviour, and not a linear continuation. It has been suggested that behaviour is a part of the expression of emotion, and not a simple reaction (Sheets-Johnstone, 1). While sometimes it is possible to observe traces of emotion in users behaviour; for instance through facial expressions, many times this is not so evident, and may be misinterpreted. In such cases, it is necessary to obtain feedback directly from the user in order to have more thorough information.

Some tools, such as Thinking Out Loud (TOL), offer an insight about what the user is thinking while performing a task. TOL can also focus on what the user is feeling and thus offer an insight of the emotions that she or he is experiencing. However, this tool may be too intrusive for the user. Some users admit, and even complain, that TOL was too uncomfortable and that the task they performed was influenced by having to say what they were doing. It is also difficult for many people to verbalise their feelings, which could diminish the effectiveness of TOL for this purpose.

Nevertheless, some research on CHI has proven that there are some techniques to make users feel comfortable in TOL sessions (Gould,Marcus,& Chavan,006).

Furthermore, TOL has also been proven as effective as 'retrospective reporting', in which users watch a recording of the performed task and talk about the problems they faced, and so forth (Van Den Haak, De Jong, & Schellens, 00).

The assessment of people's emotion to objects has taken place in different ways. Desmet developed the tool Premo for assessing emotional responses to the appearance of products (Desmet, 00). People need to be watching a screen where animations of a cartoon expressing different emotions are being played. However, this tool has limitations when used for assessing emotional response to interaction with products. This research suggests Feeling Out Loud (FOL) as a new method to assess people's emotion during the interaction with objects.

These are just a few of the complexities that research needs to face in order to develop a process for the study of behaviour. Behaviour can be defined as what people do. As broad as it is, this definition of behaviour is quite simple. This broadness contributes to the complexity of studying it. This research starts by comparing tools for observing people interact with products, and attempts to highlight advantages from each tool for the recording of behaviour and the recognition of emotion related to behaviour.

This research addresses the following questions:

a How can design researchers trace and report behaviour in a clear way that can be reinterpreted by other researchers and by designers? b How can these reports offer an account of the performed behaviour, and the emotions users were experiencing? c How can this be used to address the relationships between the emotions a product elicits, and the resulting user behaviour?

In order to address a) tools for recording and reporting behaviour from other disciplines were compared. The tools include Thinking Out Loud, Conversation Analysis (CA), and Laban Notation. Feeling Out Loud is suggested in this research as a new tool, and was also tested.

Each tool offers a focus on a specific part of behaviour.

For instance, Laban Notation is a tool to transcribe movement; while CA would normally focus on the speech between two or more people. These techniques are further explained in the Methodology.

In order to address b), three designers assessed the notations extracted from a), including their ease of understanding, flexibility to be reinterpreted, and value for design practice. The discussion section deals with incise c).

Each tool offers a focus to a particular part of behaviour, be it movement (Laban Notation), emotion (FOL), interaction (CA), or motivation (TOL). This, in Aristotle's words, would be the substratum. According to Aristotle, a particular substance is a combination of matter and form. The matter of the substance is its substratum, or the elements that give potential to its form; for instance, the bricks of a house, the movements of a dancer, or the movements of a person using her mobile phone. The form of the substance is what gives it its differentia. The actual house is the form, or the dance performed as a whole, or the task performed on the mobile phone, with particular formal characteristics that differentiate them from any other house and any other dance and any other interaction with a phone. The formula that gives the components is the account of the matter, while the formula that gives the differentia is the account of the form. Emotion not only motivates behaviour, and therefore movement, it is rather intrinsically connected to it.

Phenomenological theory argues that the human body is a unity that reacts according to stimuli from the environment. An emotion is not processed only in the brain; it is a complex reaction that is connected with movement. In other words, both movement and emotion form the matter and the form.

From an ontological perspective, the behaviour a person engages in when interacting with a product may be seen as a conversation with the object. If there is knowledge embedded in the object, the person can read such knowledge and interpret it. The person can then react and engage in an interaction in which there is information passed from the person to the object, and back. If the interaction is seen this way, then it can be analysed with Conversation Analysis tools. Although some authors suggest the knowledge embedded in the object as affordances (Norman,10), it has been suggested that a conversation implies more than what an object affords for the user; a conversation allows a more complex relationship between people and objects that is flexible and can grow to different levels than the initial affordance offered (Rodríguez Ramírez, 006).

Methodology

Step 1

Five people were video recorded performing specific tasks. The tasks were intended to be very simple and short in duration, in order to decrease the complexity of behaviour, and to isolate specific behaviour connected with specific emotions. The tasks were differentiated by the expected level of emotion they would elicit (low, medium, and high). Three different designs of products were used for each task. The tasks were:

• To use a USB memory key to back-up two files (Fig. 1).

Step

Participants were asked to give feedback about the tools and this was included in the final discussion.

Results and Discussion

Designers were asked to write and draw notes from the observations in order to have a 'control group'. This allowed the researcher to compare how useful designers found the other tools, compared to their own notes ( Table 1).

First of all, designers found useful to watch the video and the response increased from 1/5 from their original notes (5 being the highest possible), to 16/5 after watching the video. The lowest score was given to Laban Notation. This is not surprising given that designers had been given an introduction to the technique, but were not proficient with the system, which is difficult to comprehend and comprable to encountering a new alphabet. This response suffered a dramatic change when the choreographer explained her notations and findings to the designers. Laban Notation with explanation by the professional received the highest score with a 1./5.

FOL was second with 17./5. These findings do not attempt to be statistically valid. Their main objective is to highlight the advantages and potential of these tools for design research in observing and reporting behaviour. In spite of this, it is necessary to mention that the success of Laban Notation might have been due to the interaction with the professional choreographer. In this case, designers were openly excited to hear the interpretations of the beauty of movement given by the choreographer. This suggests that regardless of the tool used, the main value of the Laban Notation test was to observe behaviour from a dance perspective.

Designers might not need nor desire to learn the Laban

Notation technique, but agreed that developing such artistic sensitivity to movement would be extremely beneficial for the design of interactions with products. (Figure 1). In an independent observation, the choreographer notated this frustration and the behaviour that accompanied it. The transcription symbols read as: "B leaned to the side, with her torso to the front, attempting to put object 1 (the key) into object (laptop), repeating movement three times, leaves objects and goes back to original position" (Figure ). A combination of FOL, Labanotation and CA showed in this case useful for the study of behaviour.

Problem statement

The theory of the golden ratio has been a cornerstone of the geometric approach to aesthetics in architecture, design and painting. The basic idea is that in many wellproportioned compositions there are certain dimensions (lengths) that relate according to the ratio 1: Phi where the number Phi = 1,61... . This number can also be found as the solution of a simple quadratic equation; roughly speaking it is the ratio of a rectangle such that when a square is cut-off, the remaining rectangle has the same proportion again (more details in the next section).

This ratio also plays a role in nature: many patterns that emerge are governed by the same ratio 1 .

In three dimensions, there is another number, called the plastic number, first discovered by Dom Hans van der Laan (See Figure 1), which lies in the heart of a theory of proportions in architecture. In the same way as for Phi, there is no simple recipe for beauty. Like the golden ratio, the plastic number can be derived in an algebraic way. It is the solution of a cubic equation. is a response to the three-dimensionality of our world.

It is truly aesthetic in the original Greek sense, i.e., its concern is not 'beauty' but clarity of perception. (end quote) Extrapolating these developments, it makes sense to assume, at least for the sake of the discussion, that some day not too far from now, we will be in a position and have the complete freedom to let any object make the movements that we, as designers, want it to make. Another Braun example, the T1000 is shown in Fig. 6.

Again, squares are the dominant geometric theme. Geometry was a natural choice, because it is wellaccepted that there is considerable beauty in the patterns and proportions stemming from geometrical If we do that, we see that we get the same pattern again (assuming that it streches to infinity at both ends, which is no problem in mathematics). Of course if we shift it over approximately At first sight this has not much to do with geometry, let alone symmetry, but we will explain this useful connection now. If we take a rectangle whose sides are in golden ratio, in other words, the long side equals 1.61 times the short side, then we can perform a transformation consisting of three steps:

1. cut off a square from the rectangle;

. take the remaining smaller rectangle and turn it by 0°;

. scale it up until the short side fits the short side of the original rectangle.

Then we find the same rectangle, in other words, the long side of the transformed rectangle equals the long side of the original rectangle as well. For the formal mathematical treatment we refer to Section 6.

The golden ratio and the plastic number

Nowe will derive the quadratic equation for the golden ratio. Readers who do no like math can skip this section.

Consider the rectangle ABCD of Figure 10. Figure 10. There are two values of x for which the equation holds. The largest of these solutions is Ф.

In the same way, the plastic number can be derived as the solution of the equation x -x -1 = 0. The math is in Figure 11. The golden ratio appears frequently in nature together with the so called Fibonacci numbers. 6 The golden ratio is also used in the a-posteriori analysis The same problem appeared in the theory of relativity.

The solution was found Lorentz and Einstein. They In this article we explore another possible solution by studying the equations and the constants that define the "natural" movements of objects. Then we assume then they are related to the beauty and naturalness of the perceived movement. More specifically, we refer to the concepts of eigen-value and eigen-function that appear in many physical problems. The insights that may come out of this investigation can be combined with another important notion, human movement (anthropomorphism as a source of beauty). Again, symmetry can be considered as well. It will also be interesting to consider the possibility that the "natural" speed sets a kind of standard, by which the actual movements are judged. For example, in a value-system where control, thoughtfulness, peace of mind are highly valued, it seems plausible that a slow movement is appreciated over a fast movement and that "slow" means: slower than the "natural" speed. 7 . If x is the object's position, then "x dot" denotes its speed and "x double dot" the acceleration, the rate of change of the speed itself. This is how the mathematicians write it:

• x is the position: where the object is, • x is the object's speed: how fast it changes where it is,

• x is its accelaration: how fast the object changes its speed. The physicists and the mathematicians have a most interesting terminology to speak about the numbers and the functions involved. They call them "eigen-values" and "eigen-functions". The word "Eigen" comes from the German, where it means "own"or "peculiar". It is almost the same in Dutch. This terminology is particularly fruitful because it captures the intuition that there is some special frequency and some special behaviour built-in to the mechanical construction, quite independent of how the construction is used or triggered. But at the same time the terminology has a very precise mathematical underpinning which is in the heart of a variety of physical phenomena. What is this essence of eigen-functions? The idea is that some operation takes place which works on behaviours and that transforms a given behaviour into more of the same behaviour. In the pendulum, where the position

x behaves as a sinus function then the effect of the rope-and-gravity construction is to sustain that sinus movement. Mathematically this can be seen in the formula:

The operation is "minus double dot" and when this is applied to x the outcome is x again, except for a number, here g/L. Please note the analogy with the D symmetries, the effect is that the pattern goes on for ever .

Toward products

At present, many products which move do so because of functional reasons: the product's main function is to move matter or transform matter. The function dictates the movement. Often the product is adapted to the human form or the human scale, but in certain cases this is because the user is part of the task; a typical example is shown in Figure 1, where the user acts as the motor, the energy source for the stapler. A very different example is in Figure 1. This product has to be folded and unfolded by hand, but for a different purpose: the purpose is to adapt the light to the human scale, either to carry it (folded) or to provide the user with light (when unfolded). In both these examples the user still is the motor, so rather than observing a movement, he/she creates the movement him/herself while feeling counter-forces and other haptic effects 11 .

But in future both types of objects can be changed to move autonomously once the technology discussed in Section is available. Rather being the human motor, working for the machine, we will design and use machines who adapt themselves to the human scale. When not in

In the example of the pendulum, these would be g/L and perhaps m.

10 But once there are many examples of systems which deny these assumptions, there is no more ground for the assumptions. They will gradually be replaced by new models, mental models, computerised models, which then will be part of human culture.

11 There may be a certain aesthetic quality in this combined action/feeling; this leads to the important question how to design the aesthetics of interaction, which however is considered outside the scope of the present paper.

use, they better be small, when needed, they adopt the best position to perform their function, for example a media function (note that light is a medium too). Folding and unfolding are important but not the only examples.

In both cases the artificial movement can be engineered in many ways, with less constraints than present-day solutions. It is for such products that we consider the theory of this article to be relevant.

Concluding remarks

We could not answer all questions raised in the present Hence, the overarching question for this paper is how interaction designers work on the dynamic aspects of their design object.

The study points out the importance of designers seeing each other, as well as seeing the sketches and hearing verbal descriptions of the design. Indeed, as we will show, some design ideas are never presented verbally, but only through gestures and common understanding.

The Role of Making Models in Design

When designers perform acts on their models, including sketches, scenarios, storyboards, diagrams, physical models and computer prototypes, we say that they act in the action context; that is, here and now in the workplace activity [1], [1]. The models, however, are also representations of what will happen in the target context, in the virtual world of hypothetical user activity where a future design solution will be used [7]

Models in Interaction Design

Turning to the area of interaction design, the models and design representations employed there typically include lists of tasks and functions, user personas and scenarios, diagrams of structures and interactions, user interface sketches, paper prototypes, and computer prototypes.

Sketching on paper is particularly important during early design explorations [5], [17], [1], [1].

Expressing Interaction

Interaction Bluetooth. Finally, the iLounge contains high-quality audio and video equipment that can be used for videoconferences, or during user studies.

Procedure

Four master's students in interaction design, two male and two female, were invited to the iLounge. They all knew each other well, having taken the same courses for four years. The two female students were given a design brief asking them to design an interactive space to be used for studio classes. The two male students were given a brief asking them to design a drawing tool for an interactive digital whiteboard. The briefs thus pointed towards design solutions in the direction of the iLounge they were to visit and experience. Our idea was that they were to seriously consider how they would like such an environment to be structured, and thus come up with ideas about how iLounge could be improved. We recorded all sessions using both audio and video from multiple cameras. This video material formed the empirical material for this study. No interventions were made during the sessions, except during the evaluation, which was facilitated.

Analysis

After we gathered the data, we analyzed it together.

The focus of our analysis was on the gestures and dramatizations (i.e. enactments) of their design proposals.

During the analysis we interpreted the enactments and their performatives. We also traced our interpretations of events in the synthesis sessions to events in the presentation sessions. All verbal utterances and gestures were transcribed into a protocol in our native language (Swedish). We then analyzed the transcriptions further as we engaged with them theoretically using previous research, and only then did we translate them into English.

Without a Word for the Design

The two women, whom we will call Anna and Barbara, only in the enactment that the two designers shared.

Interaction Walkthroughs

The two men, whom we will call Christian and Daniel, They both pointed to the object-centred character of the digital whiteboard (that the user works with drawn objects rather than with pen strokes). In excerpt , we see how Daniel goes to the smartboard and presented his idea about the differences ( Fig. ). Their preliminary and quite spontaneous analysis of the differences between traditional and digital whiteboards was clearly connected to their enactment of interaction.

Improvised Role Play

In the following excerpt we exemplify how the two male designers explored the different uses of the actual smartboards in order to design them to be used for This episode of enactment is interesting in that both designers cooperated in the role play; Christian followed Daniel, playing along with his initiatives. In the earlier excerpts, we also saw that the female designers were playing along, but they mirrored each other's enactments rather than one taking a distinct lead. In this session the two designers cooperate and play along, using both gestures and voice in taking on the roles of users. These short role playing sessions evolve into a discussion of what target context the user would be in. They explore the concept of cooperative sketching by role playing.

In our material, the participants performed enactments using gestures that mimic the actions of users in an interaction walkthrough. The interaction walkthrough is a gesture-driven enactment and it helps experiencing and figuring out the behaviour of the artefact-to-be-designed.

Christian and Daniel took on the role of two users and imagined themselves in a certain situation of use. This improvised role play is a scenario-driven enactment, and is used as a designer's think aloud exploration which contextualises the design solution in an imagined target situation.

Without these forms of expression, it would be difficult to express interaction and also the basic design concept behind their design solutions.

Discussion

We began this paper with an overarching question:

Interaction Walkthroughs and

Future Research

The research presented here is conducted in workshops with interaction design master students.

Conclusions

In an analysis of gestures in interaction design we have On the left, the tooth-brushing schema is elaborated. Sequences of actions that could be learned via complementarity are marked with numbers (1-9). Obligatory sequence transitions are indicated with black lines.

Decision junctures with multiple options, where value or priority is assigned, are marked with black dots.

Grey arrows connect actions that restore the ecology of objects to equilibrium to the actions they 'undo'.

Note that some action descriptions, e.g. 'brush' can be broken down into sequences of actions at a finer scale. Synaesthesia is a familiar concept in the design world, although it is generally not as profound as in the abovementioned examples. We believe that synaesthesia is not only powerful as a guideline to design rich products, but it is also a powerful tool for the designer during the early phases of the design process, especially using the relation between the designer's bodily movements when exploring and visualizing his ideas and the expression of the final product, including appearance and interaction.

One can see this relation when looking for example at design sketches from Ron Arad and Luigi Collani, see Once that expression appears detectable, we can start looking for the salient elements of these movements in order to formulate rules for computer implementation. for Yoshinaga Prince Company which was produced since 1984 in

Japan [6]. The material of the designs, mostly synthetic, fits the entire expression.

We start by showing the potential and scope of a design tool that exploits kinaesthesia in synaesthesia. Thereupon we explain the experiment and our findings. This support can be given in many ways. However, if a computer wants to make sense of a designer expressing his ideas with expressive movements, it desperately needs guidelines to interpret and capture the essence of these movements. In order to discover these guidelines, we conducted explorative research. In this paper we will focus on one of our studies on the expressive character of gestures. Other studies on e.g., general approaches to gestural design, and the visualisation of geometry, can be found in separate publications [, , 10].

The detection and formalisation of patterns in expressive gestures is extremely complex, therefore, we started this study with two central questions:

1. Is gestural sketching suited, or even more suited than traditional sketching on paper to develop and visualise expressive ideas?

. Is an outsider, more specifically a designer, able to recognise the intense and extensive expression of the gestures used during the design process?

The first question can confirm our premise that gestural sketching is a useful addition to the existing design tools.

The second question needs to be answered affirmatively to be able to formulate any rule. If one cannot recognise it, it becomes very hard to formalise it.

To successfully integrate behavioural concerns into design practice, and to make this process repeatable, appropriate information and tools must be developed and incorporated into the design process. The findings of this design study have provided an insight into the type of information required by designers to consider these issues and appropriate formats for conveying this information.

A further outcome of this study has been to inform the authors of potential ways in which design-led approaches may effectively be configured to maximise their effectiveness in addressing user behaviour.

What occurred in the 10s with the theory of product semantics was a fresh initiative at an old idea, revamped and neatly labelled but insufficiently refined for the practicable purposes of use in the day-to-day design process. This inability to answer designers' demands for a new and reliable methodology led to frustration with the concept and eventually its rejection in many quarters. Indeed Krippendorff (Capitello 11) blamed an 'untheoretical design profession' for the rejection of the concept of product semantics and the field study indicated that generally designers do not value academic theories as particularly useful for everyday design practice. Indeed, one of the more surprising findings was that even those designers who adopted a product semantic approach were not particularly aware of the subject specific literature.

However, calls for more expressive and communicative products in the market place still persist. Such admonitions suggest that the attempts in the 10s to formulate a theory of product semantics were worthwhile, since these are precisely the issues that it tried to address.

However, it also confirms the findings of the study that the ideas central to the theory were not properly understood or disseminated by a large enough section of the design world.

The set-up

To answer the aforementioned questions, we needed to find a medium to make expressivity measurable and comparable, without losing the subtleties necessary for

design. The answer is provided by synaesthesia.

Previous research from Smets & Overbeeke [11] indicated not only that design students are able to convert patterns from one sense organ to another as the example in Figure 1 shows, but also that the results between students are often related. In one of the exercises they asked the design students to create a sculpture expressing one of nine

The set-up of the design part

Six types of stimuli were created by the design students (see Table 1 Therefore, we extended the design exercise with experts in movements, four dance students. They were asked to express every scent in a two-minute dance (GeDa).

Secondly, the gestural condition did not produce an image of the design. Therefore, the designers sketched their virtual design on paper after every gestural session (SkGe),

including the movements of the sculpture if present.

Both designers and dancers were asked after every session to indicate their satisfaction with the expression of their own creations in relation to the scent on a scale from 1-5 (very dissatisfied -very satisfied).

To The subjects received the drawings by Roselien, placed in random order. They were offered as much time as necessary to make the couples. Afterwards, they were asked to give two to three catchwords to characterise the match. The second part consisted of four short sessions. In every session, the four dances by one student had to be matched with the four drawings Roselien made when watching this dancer performing. This matching part resulted in four times four couples. The subjects received the dancers in random order. Within a session, the drawings were randomly offered and the tapes were randomly distributed across the monitors. Again, the subjects were offered as much time as necessary.

The hypotheses

The two central questions were tested with five hypotheses, which are discussed separately. Table 4 Hypothesis 3: An independent interpreter can detect the expression of gestures and produce a sketch which is considered to be similar to the original sketch by the designer.

Matching drawings with films

We use the same argument for the matching of the films by the dancers with the sketches by Roselien. Table 6 Hypothesis 5: An independent interpreter can detect the expression of gestures and produce a sketch which is considered to be similar to the original dance.

Designs created Designs created with gestures with sketches

Mean: 3.7 3.8

Standard Deviation: 0.849 0.533 Table 7 The results for hypothesis 1: the satisfaction of the designers about their design.

Hypothesis . Table 8 Number of correct matches per subject. Table 9 Number of subjects that made a correct match for a couple 'drawing -drawing'

Hypothesis 5. Table 10 shows that only for the last series by dancer 0, H0 'the number of subjects that match a session correctly ≤ ' is rejected. When we look at the correct matches of the three competent subjects together series (nos. 01, 0 and 0), then every series is scored at least once correctly. although her drawings of the first three dancers were less convincing than those of the last dancer, or three out of four dancers were less distinct in their expression.

We would like to finish this paper with a beautiful example of the expressive wealth of gestures. The three stimuli depicted show the expressivity of body movements and the synaesthetic relationship between shape, colour and movements (see Figure ). When we saw the films of dancer (Sjoukje Philip) and designer 17

(Wing-Ken Cheung), we were struck by the resemblance between the expressivity of their movements. It was even more striking to see the sketches made as a result of these movements. All three sculptures were highly similar in terms of shape and colour. This is all the more striking, It holds many practical implications in the field of design in general and to thesis -Talking to The Hand in particular.

Natural, silent language

Gibson's approach and the theory of direct realism suggest that for humans the appearances of the physical world are deeply familiar. It might be said that we read the environment through a language; objects are not only placed in a location and carry an arbitrary appearance, but they are related to a scheme that in turn is related to another scheme, and together they form a greater scheme that represents the environment in total. A scheme is interpreted as a sum of operating forces and constrains all conveyed by visual signs and symbols.

Morphological features, textures, and projected physical vectors serve as those signs. We see then that a physical environment is embedded with layers of meaningful information, and that appearances in the physical world are not at all superficial, but rather derive from substances, properties and inner mechanisms.

An acute design consideration

The relevancy of the idea of affordance to the field of interaction design and to my thesis in particular, is great.

In his book The Design of Everyday Things, Donald Through the use of the 'natural' language of affordance, these objects will form a familiar exchange of information with a spectator or a user.

Bodily clues

The concept of affordance explains how an appearance is meaningful, and how might an object 'talks to the hand' in an iconic manner, like a sign or a symbol that carry It is crucial to identify a possible mechanism with which the hands could generate this kind of perception.

'I am my body'

In his aphorism 'I am my body', Maurice Merleau-Ponty, the existential French philosopher, pointed to the fact 5 Wikipedia. Perception.

Perception and reality.

Peripheral awareness

The periphery in our awareness, describe events and occurrences we are aware of with out putting them in the highest priority of attention or the center. immediately, showing that we were attuned to the noise in the periphery, and could come quickly to attend to it.' 16 11 The Internet Encyclopedia 15 Marks, Haptic Visuality:

Touching with the Eyes. 16 Weiser & Brown.

Designing Calm

Technology. The Periphery.

The periphery is only a hierarchy of priorities, which might change when circumstances change. It does not suggest that the things in the periphery are less important, since what was before a background noise or at the periphery can become the center of attention. 17

Technology. The Periphery. 1 Weiser & Brown.

Technology. The Periphery.

An integrative experience

The experience of the physical environment is comprised of a stream of parallel simultaneous experiences that take place in the entire scope of our awareness, center to peripheral. We often describe that stream as a 'natural' experience, as it provides a sense of integration with the environment. The next sections will report the iterative prototyping part of the exploration set to verify these proposals.

It will describe the morphological and semantic issues that appeared in the process of developing shape-change designs, illustrate the decisions taken along that process and summarize the findings of each iteration session.

The various prototyping sessions where performed under three themes drawn from a preliminary mapping of shapechange properties. Section focuses on the visual aspects of shape-change. Section 5 looks into the tactile realm of shape-change and section 6 with the adaptive quality of shape-change.

Section 7 describes the working demos that were developed at the end of the process as a summery.

'Body language' for objects

In every system, at any given time, there exist multiple statuses in which the system might rest. Only in transparent systems that consist of only their visible structure, are those statuses obvious to the observer.

In order for a spectator to realize the actual system status, a clear indication of this status needs to be displayed; however, a straightforward design to meet this requirement is not always possible, as often the status is hidden, or not always self-explanatory. For example, the 17 Weiser & Brown.

Prototype 3: Synchronization

The idea of an object getting in or out of sync builds on the convention and expectation that man made objects should to be aligned to something in the space. Everything refers to some sort of origin, the gravity, the hand or head

Findings

Through the implementation of the 'Morphing cube' prototypes, I found answers to most of my initial questions that were not categorical, and seemed to be valid with in circumstances of the specific design.

The importance of a reference point in which a change could be measured was crucial. In order for a spectator to predict the physical potential of a certain entity, the design must hint at its entire capacity. I conclude that a shapechange can only be made measured and mapped through the existence of an internal skeleton, which the change could be referenced to. interfaces could be used.

The Tacto

'Blind tests'

In our everyday experience of the surrounding, touch plays an important role. 1 We view this interface as an alternative, or rather complementary, to visual based interaction. Thematically it seemed to fit the concept of peripheral awareness mentioned earlier, and also allowed us to experiment with idea of 'haptic visuality'.

User observation (1)

The set up for the observation was a box into which In the first stage of this iteration the participant was requested to read:

• Single basic shapes.

• Compositions of various basic shapes with similar sizes

• Compositions of various basic shapes with varying sizes Looking into the specific of hand motoric, the finger work, we saw that participants 'previewed' the terrain quickly and were akin to boundaries more then morphology. This was made obvious through both the lack of overall boundaries that define the perimeter of 'feeling' and the actual reading of a shape by following its edges.

However, due to the granular nature of the pin surface raster, those edges were interpreted ambiguously. We discovered that smooth transitions having less of an edgy quality, were more suited to be displayed on the pin surface.

On another level, the basic demand to call a shape by its name seemed to be a methodological misconception.

It was made clear through comparing participant's response to the engraved composition that could be generally described as 'Paths', as oppose to the embossed compositions generally described as 'shapes'. Participants could easily draw the outlines of a single path or a composition of paths, while exhibiting great difficulty when doing the same for compositions of shapes.

User observation (2)

The second iteration was focused more on identifying design opportunities for the interface mentioned earlier.

It used the same 'box' setup as in the previous iteration.

On The second task, of identifying and distinguishing between behaviors and approaches of the animated transition, also proved highly potential as all participants were able to identify, easily, quickly and accurately, all five exemplars they were introduced with.

We concluded that the real opportunity for a valid interface based on shape-change surface, was animation.

The Tacto-phone

The Tacto-phone is a detailed video scenario of a cell phone interface that allows for interaction with locationbased information. It does so in an 'on the go' and 'in the pocket' fashion. By that I mean not having to stop walking, or moving through space, in order to interact with the device and through it, the relevant information. And also, being able of performing initial interaction, like preview, with out taking the device out of the pocket.

The Tacto-phone uses a touch sensitive morphing surface to display and interact with animated shapes.

It is suggested that the surface would be placed on the backside of a cellular phone, complimenting its screen. The surface together with an audio channel provide a preview and initial interaction with information placed in a specific geographic location, without using sight. [Fig. 1] Tactons can be used for example, to distinguish between information that refers to people, from one that refers to content, or one that refers to events. As the animation is dynamic, the display area is mapped to a certain physical size that represents relative distance from the actual geographic position of the information, in reference to the spatial context. For example in a city, the mapping would be linear and related to a street. It provides basic navigation functionality as well as sensitivity required to filter all available information. The bigger the distance threshold is, the more information will be displayed.

'Habitual interaction'

Following the digital environment adaptation analysis an adaptable interface reflects an ever-changing structure of information by adapting its appearance to the new circumstances. I hypothesized that by doing so, hidden qualities of the structure of this information (the database), would be exposed. I also assumed that manifesting this adaptation through physical action, the exchange between the user and the interface would enhance the user 'habituality' of with the virtual environment in which the information is situated, making the user more familiar with this information.

InfoTerrain web interface

InfoTerrain is a prototype with morphing interface that follows the approach proposed above. The three-dimensional output depends on the 'rating' of Popularity -a map that shows how a link is ranked within a certain reference group. For example del.ic.ious link sharing. [Fig. 1] On-screen those links, which were mapped to the pin surface, were dynamically highlighted with a different color while the mouse pointer was moved around the screen.

[ Fig. 0] The prototype brought about several interesting insights:

• Disorientating triphonic input, eye and two hands were causing the user to loose focus Interesting issues came up in the design process, concerning the proportions of the plates and the distance between them.

One consideration was that the visual output of disposition is optimized when the plates are closer together, while the affordance of the required twisting action was optimal when the plates were further apart.

'Morphing mouse'

The 'Morphing mouse' refers to the previous 'Habitual The "button" that appears allows clicking on the screen object/ link, while exposing its qualities, through shape & behavior.

For example, a "button's" height can represent the size of a folder, or the score of a link in google. While a rhythmic up N' down movement might indicate the current beat playing in an Internet radio station the link is pointing to.

Design considerations

The realization of this prototype drew many efforts. The

Feedback

It was certainly an easy task to identify on a demonstration that users quickly catch the InSync interaction. This was also a simple explanation since the animation and what's behind it are strongly connected through the twisting metaphor.

The 'Morphing mouse' however lack the action needed to complete the experience, when a "button" appeared the loop wasn't closed through clicking on the bump. This selfexplanatory action was evidently missing when observing users response.

Summery

Talking to The Hand builds upon three external ideas, affordance, habituality and 'the periphery', all of which support, although not categorically, three design proposals.

For example the 'Morphing cube' prototypes might fall under affordance as well as 'the periphery'.

The first proposal is the idea of a 'body language' for objects based on the concept of affordance. It envisions digital artifacts that produce communication by adopting With the proliferation of mobile phones these physical and architectural signifiers have become largely obsolete.

For the most part, the obligation is placed on the user to use their phone appropriately [10]. Users negotiate their own rules of engagement [11] and it through this process of appropriation and assimilation that the impact of the product is enacted to a greater or lesser degree [1].

Designing for Behavioural Change

Findings of a prior literature review identified the following approaches for designing behavioural change;

eco-feedback, behaviour steering [scripts, affordances and constraints] and intelligence.

Eco-Feedback

Eco-Feedback, grounded in Feedback Intervention Eco-feedback is located towards the user end as this approach provides the user with information to make informed decisions autonomously; intelligent products however retain a greater degree of influence and control. For the most part evidence of how the research findings related to conceptual ideas generated was not explicitly recorded in the logbooks yet it was, in some cases, articulated in the verbal presentation. An exception to this was Student 5 who co-located his design ideas next to the research that inspired them, clearly demonstrating the links between the research undertaken and the design solutions generated.

Application of Design-led Approach[es] for Behavioural Change

The value of using a combined approach to analyse the students design process became apparent when reviewing their logbooks. In most cases it was difficult to assess the student's level of understanding of these approaches from the logbooks. It was also unclear which approach was chosen and how it was applied. In terms of applying these approaches, most of the students advocated a 'mixed' approach, combining two or more approaches. Few students used the approaches as a starting point, preferring instead to apply one or more approach as part of their idea generation process to provide "a direction to think about the problems"

Design Outcomes

Analysis of all ideas generated in sketch form and those detailed as final solutions revealed a range of approaches to solving the various social issues identified, however some commonalities could be observed in the type of solution In the final design the phone displays its embarrassment by emitting a red light which disables functionality for a fixed period enabling the phone to calm down, figure 11. and through combining different elements together generated novel solutions to the problems identified.

Comparison with existing design concepts

The design concepts discussed in section . were how and when they applied design-led approaches and their perceptions of the effectiveness of these approaches in changing user behaviour.

Reflection on design-led approaches

It is interesting to compare the student's perceptions with those of the authors. The student's debate concerning the moral and ethical issues inherent in designing for behavioural change reflects those of the authors in a previous publication [7]. In both discussions concerns were raised regarding the level of control or influence which designers or manufacturers should ethically integrate into the product design. This debate was coupled with discussion relating to the effectiveness of design-led approaches in changing user behaviour.

Eco-feedback was understood by most students and was arguably the easiest approach to apply within the product design. However, the students questioned its effectiveness in changing ingrained user behaviours due to the potential for the user to ignore the feedback provided.

Eco-feedback approaches provide users with information to enable them to change their behaviour, however, information does not necessarily lead to action as the consumer must be able to link their behaviour with the long and short terms consequences. Behaviour steering was the least understood approach, yet this may be due in some respects to the lack of tangible examples of how this approach could be applied within product design.

Intelligent products were seen as having the greatest potential for effecting change. However, some students felt that if the product continuously regulated behaviour it would not encourage people to learn from their 'mistakes' and could result in the user feeling controlled or restricted by the product.

Most students appeared to favour a combined approach which is a particularly interesting development especially as the brief originally stipulated that only one approach be used.

Type and format of information

The findings of this study indicated that designers need to be made aware of potential problems caused by user behaviour through introducing the concept of designing for behavioural change as a design challenge. In terms of the effectiveness of these approaches in aiding the design process, it is interesting to note that rather than using these approaches as a prescriptive method or tool, the design students viewed them as inspiration for concepts or as a means to develop or evaluate the effectiveness of design ideas. It has, therefore become clear that any outcome of this research should focus on providing potential directions or approaches, rather than prescriptive methodologies to follow. The provision of existing design case studies not only acted as a stimulus for approaching the problems identified but also helped to clarify the design-led approaches introduced. This finding is supported by those of Lofthouse [] who identified "that ecodesign information should be presented visually using case studies and examples" "to encourage, inspire and educate" designers and "support idea generation" . Taking into account these findings, the next stage of the research will be to create an informative resource for designers explaining why they should design for behavioural change;

approaches they could apply and inspirational case studies.

This will be tested with a group of designers.

A combined, tailored approach

Drawing together a series of findings led the authors to consider two key issues; the potential ways in which design-led approaches may effectively be configured or a 'design theory' in the years that followed, leading to a notable decline in debate about the idea, apparent in the design press, in the early 10s. The study therefore sought: to establish the reasons for the emergence of the theory of product semantics in the 10s; to examine the various theoretical principles upon which the theory is based; and, to analyse the way in which these theoretical ideas were put into practice. As well as a review of the available literature, a field study was also undertaken to: establish the extent to which design practitioners in the UK were aware of the concept; establish the extent to which it had been adopted by design practitioners in the design of industrial products; identify any viable, consistently efficacious methods that allowed the design of easy to operate and symbolically meaningful products; and the extent to which it was valued as a design theory.

Ultimately, the aim of the study was to assess the validity of the theory of product semantics and by providing an It is argued, therefore, that product semantics can be seen as a valid approach that does, when utilised in an appropriate manner, provide a useful theoretical framework for certain aspects of the design process. Some fifteen years after its emergence, the study suggested that product semantics remained as an 'umbrella term' for a loose collection of associated ideas and not as a single, coherent theory, which was undoubtedly a major cause of its original dissipation. However, the Lisa Krohn's 'Phonebook' 1987: operational self-evide

Figure 3

The 'wiggles and waves' syndrome of product semantics in 1980s a minority view and greater awareness of the concept is needed if product semantics is to play a more significant part in the design process in the future, as many of those who use it clearly think it will.

All of us have had tedious encounters with products,

in which we had to spend a lot of effort to find out what a product can do for us, and how we bring it to doing so. It can take a lot of time, trying out, asking advice, decyphering manuals, and plain random pushing of buttons, to get to know the product. Since Norman popularized the Gibsonian notion of affordances, it has become an acknowledged duty of the designer to express to the intended user what the product was intended to afford him or her, and how he or she can attain the state of fulfillment. Often this comes down to telling the user 'which button to push'.

Although there is a growing skill and knowledge base that helps in educating new generations of designers, there is still a way to go. Many products still need 'fixes', such as the handwritten signs and small notes fixed to doors, water taps, that we encounter in daily life, especially in public spaces and 'dirty' industrial work situations. In our homes, and often in office work situations, we tend to eventually learn how we can get our VCR to turn the TV on and off, how to switch channels, and which buttons to ignore (Fig. 1). Life mainly becomes difficult when we accidentally hit one of these buttons, end up lost in some menu dialogue, and have to fight our way out of it, sometimes by performing a hard power-off reset. But in these situations, we establish a repertoire of routines, and notice the aggravation less. It becomes more noticeable in situations where we encounter a product for the first time. In public spaces, such as trains, and boats, and planes, and in hotel rooms (especially abroad), we have a multitude of first encounters of the affordance kind, which we fail to enjoy. It is here that miscommunication, lack of expressiveness, is at its most obvious.

In this study, we collected a number of these encounters in public spaces, and use these examples to distinguish and discuss some relevant issues. Apart from collecting a few amusing anecdotes, we hope to illustrate some ways of approaching the expression of affordances that can help design students and practitioners better develop their designs.

What theory there is

The need for products to express their purpose and . Especially in the emerging field of (computerhuman) interaction design, educators, researchers, and practitioners paid explicit attention to this, driven by a need for finding expressiveness in an application area where increasingly fewer physical constraints could express mechanism and functionality. In part this led to a confusion in the usage of the term 'affordance', where its original meaning 'possible future events/actions that the product offers to a specific user' got mixed with 'expressing those possible actions'. For some, this led to a view that putting an icon on a button was equal to giving an affordance to an application. Norman clarified these issues in his Interactions paper (Norman,1).

In this paper, we stick to the original meaning: affordances are events and action opportunities, and the designer has to express these through the design of the product.

Gaver (11) distinguished affordance and information, and explored the combinations of these two notions, each of which can be true/false for an action opportunity. A successful expressive design both has the right affordance and shows it, resulting in a perceived affordance; if the product is clear that it doesn't perform the function the user desires, this results in a correct rejection.

Mismatches between affordance and information yield hidden affordances (the product can perform the function, but it is difficult to find out how to get it to work) and false affordances, or posing (the product suggests it can do something which it cannot). An interesting limitation in current theory is that it puts all the responsibility, and all the blame, at the designer's footsteps, whereas the examples we present below show that the designer is often a few steps away from the user, and cannot control how the product reaches the user. We see therefore, that some of the errors are due not to the product, but to the way it is incorporated in the environment. And that many of the patches were probably invented by janitors and maintenance people in close contact with the users, rather than by the design Fig. 1. All alike. Left: We typically use a few buttons of our remote controls, ignore most, and learn to live with them. When using somebody else's toys, we're back to square one. Middle:

One of these knobs can be turned to open the door to the lecture theatre, the other can only be pulled. Which is which? Right: A patch? ATM's often use rows of buttons whose function is indicated on-screen, which works for a flat screen, but creates problems of parallax for a receded screen. To reduce these problems, small bars were added later to more clearly connect the button and the screen. The smudges on the screen are also a sign that many users expected it to be a touchscreen.

departments hidden in corporate agencies. This offers food for thought on the roles of all those involved in the design and application process.

Collecting confusion

Aim of this study was to generate a collection of examples of confusion that is large enough to be structured into categories. The examples were gathered by visiting public places, observing and recording confusion. Also, we were liberal in our method, in asking people for situations and products they had found confusing, and including some of our own experiences, especially from international travel, and from the literature. There was no lack of volunteers. Admitted, this is explorative rather than disciplined evaluative research. The method cannot find the worst designs (which are not noticed at all, so can't be remembered) and is likely to be biased toward the most annoying (and therefore best remembered) rather than the most confusing examples. Public toilets show evolutionary stages due to the introduction of body proximity sensors. Some sensors look like buttons or ornaments, and don't express their sensing activity. After these sensors were introduced, signs were needed to tell people they didn't have to flush. Recently, we see the opposite: in some places where there are buttons, these look so similar to sensors, that signs are added that users had to 'please flush manually'. Fig. ). Other favorite locations were public walkways (10) and especially trains (7). In one instance, we observed an elderly lady waiting patiently in front of a door, which carried a sign 'automatic'. The lady didn't understand the door, waiting for it to open. The door didn't understand the lady either, it only closed automatically, after a fixed time (See Fig. ).

Results and discussion

Another criterion for classifying the examples came from considering the interaction between user and product as a scenario over time, rather than a momentary activation. Table 1 illustrates to achieve a unified and aesthetically pleasing whole". But also, we see that many of these patches are not made by the designers during conceptualization, but by the janitors, facility managers, shopkeepers, and friendly neighbors who operate in the situation where the product is applied.

Step Frequency Comments During some interactions, the product can offer degrees of freedom, e.g., temperature of water, or direction of dry-blower for drying hands and hair (Fig. 5). This must be recognized.

completing (stop)

3 Some interactions need to be concluded, e.g., most people want to shut doors or flush the toilet behind them (Fig. ). Seen as part of the interaction, these actions are completion. Many people live now in a world where they can reverse the time (Ctrl-Z), instantiate objects (Copy/Paste) or manipulate simultaneously different items (Select, Dragn-Drop). These properties, like many other are described, evaluated and generated through interaction models such as "direct manipulation"1, or more recently "tangible interfaces"10 or "instrumental interaction" .

In the last few years, a new class of physical objects has appeared: they can change their shape, being animated or controlled by motion, gestures or physical interactions.

They are now very similar to computational objects like icons or windows. Can we apply interaction models coming from computational systems to them? Can we transfer structures and practices from computer to these dynamic embodiments? In order to explore this issue we introduce the model of Instrumental Interaction created by Beaudouin-Lafon . Then we present two dynamic objects implementing some of its properties.

Instrumental Interaction

In The participant empathizes with Blendie and in this new approach to a domestic appliance, a conscious and personally meaningful relationship is facilitated.

Instrumental interaction in the physical world

As an example of transfer from the world of computer to the physical one, we present here two projects that try to reify the concept of sliding into a physical object. As shown in figure one, the scrollbar is basically an instrument that allows modifying a value or a quantity by navigating through it. It could be actuated by its little arrows on the extremity or by grasping directly its handle.

What could be a tangible scrollbar, how this instrument should be shaped and actuated in the real world? Would it be as meaningful for people that use it? As an exploration of these issues we introduce two objects: Tangicam , a video camera for children and Telebeads an electronic jewelry system that can capture motion and interact with domestic appliances. They respond to a very contemporary consumers' habit,

DACIA Main Outcomes

As shown by the design cases illustrated in the next section, Bionics has proven to be a very good approach to convey to students the concepts of ecological sustainability, formal and typological innovation.

During a design process, every time that designers need to verify the optimisation of the various aspects in a product, they discover that nature has already found the best solution, for example, requiring the minimum quantity of material or using the most appropriate section in relation to the efforts that the structure have to resist.

The concept of best performance with minimal effort

illustrates to students what is ecological sustainability in products. Buckley [10] and Hjelm [11] We are now at a point where we can ask: in how far was our set-up of the exercises successful in enabling students -and us -to reflect in-depth on interactions? This question is explored here. Previously, I have held some of our efforts up against all four rationales that were briefly reviewed above [1].

Examples of role playing techniques applied

We first adopted In all examples presented below, the role playing exercises took place at the project stage where user research had been carried out in the form of interviews or observations. The step to be taken next, aided by the role playing exercise, was design idea generation.

Role play example 1: the Story Walk

The first role playing exercise in the MSc project "Exploring Interactions" was a combination of improvisation theatre techniques adapted by me for the course, and the Storyboardwalk developed by Saakes and van de Lelie, see also [1]. The combined workshop was called the Story Walk.

Twenty students in four groups took part in the workshop of one afternoon, in a studio.

Taking a situation from their prior research, they were asked to break it down into dramatic elements, improvise with the elements, then settle on a story, act it out, photograph that, print it out right away and lay it down in a storyboard collage.

Evaluation

The evaluation with the students showed that none of the four groups had actually experimented with an emerging story. All groups had taken a story they had previously sketched following their research, and played it through. They went away from the afternoon with a visual record of an interaction. This they said they valued. But on balance, it can be concluded that the students were mainly focused on dramatizing and presenting their story, rather than on the experience of being in that story, or on experimenting with unexpected interactions arising from the situation.

. Role play example : Project "Organizing Things".

The third example of role play comes from an ongoing design research project of our own, in which we look at the integration of user research in the design process: the project "Organizing Things". Preliminary findings from that project are described in Boess et al [17]. We seek insight into our combined design and research activity as 'investigative designers'. In team meetings, we recalled to our attention actions that had been described and shown by participants of our prior research, and we role played some of those actions (Figure ).

While these last two workshops (the second one was actually three parallel workshops with 0 students each) are very recent and have not been fully evaluated yet, some initial notes can be made. Both times, the students set about the workshop tasks with great earnest and application. Following the scenes being acted, they held discussions in which in-depth themes about the firsthand experience of interacting with things emerged. The students worked almost self-sufficiently throughout the entire workshop. In the time that the students had been given to prepare, tutors could take the time for some brief coaching and explanation for each group, so that the students came back to the forum with some confidence about their role.

Role play example 2:

Quality of Interaction Workshop

A year later, a shorter workshop was held, for a larger group of about 0 students, in groups of three to five. In a short idea finding phase, the students were asked to use their prior user research to give a desired interaction a two-part name describing its quality. The technique hails from Hekkert and van Dijk's ViP approach [16]. Each student then asked their group mates to act out this interaction. Each student's design idea would be 'played back' to him or her as problem owner, with opportunity for adaption and discussion. In a sense, it was a mini forum theatre exercise [1]. And there may be a need to prepare students for the acting itself, to let them practice it and get a feel for it.

Conclusions and Further Work

Our examples have shown us that if we want to access the innovation through experience potential of role playing that was laid out by Boal and Johnstone, we need to be attentive to its set-up. While the first three attempts did not enable us fully to get to work on that, the fourth and fifth showed that more preparation for the acting In this approach a Choreography of Interaction is created, which involves the design of a product that motivates this

Interaction Choreography [,].

In this paper we show how this project resulted in a product that was designed for, and was part of, a meeting event at the festival 'a camping flight to Lowlands paradise'.

Lowlands is a three-day festival, attracting a wide variety of people, most of them relatively high educated. The visitors sleep at a camping adjacent to the festival area. The festival

has a very open minded and tolerant atmosphere [5,6].

We illustrate how product design, when approached with a choreographic perspective and a focus on body language between people, can be seen as the design and motivation of an event In the next part we introduce the designed event, the meeting that was choreographed as the reason and starting point for the design of the product. In the second part we present the product that was designed to motivate the choreographed meeting event, and we will explain how its characteristics are meant to do so. These two parts overlap, since the product design evolved from, and was part of, the creation of the meeting event.

Designing a meeting event for Lowlands

The During the development of this concept, first associations to product ideas started to rise: The closer you get, the more contact area you create with the unknown other side. It all feels very challenging but intimate, yet you are separated by the screen.

Another possibility to imagine is to break through the spatial separation between another person and yourself.

When having certain surfaces on your body that match and react to surfaces on other people's bodies, spatial shapes can be composed, which for example could be translated in the composition of music.

Designing the challenger, mediator and motor of the meeting

These first associations were the starting point of the product design. This product design phase started as a natural reaction to the design of the event; in search for products that would fit and elicit the meeting event (see fig. ).

Work in progress

Now that the product had resulted in a first prototype, the project was at an end. The design however is not complete.

The Eventually, this continuation should develop into both useful findings for design-research at our faculty, a good learning process for the students and a good time for the visitors of the Lowlands festival.

Ambient Technology and Product Personality

The miniaturisation of technology now means that a product's form no longer needs to follow its function.

Philips have long since recognised that design of the interaction between a user and the product is becoming at different times and this should infer that the design of a system that can recognise such differences and react accordingly. Kozlov believes that; '…the mani-festations, or enactments, of these I-positions may be more strongly evident in some contexts than in others. Contextual triggers may be spatial (different 'heres') tempo-ral (various 'nows'), interactions with people (real or imaginary), or social institutions (rules, roles, rituals, etc.)' [16].

Whilst personalities can be seen as complex to consider, our first impressions of people can also be misleading because they might infer behaviour to the onlooker.

An unshaven person may well give the impression that their behaviour is to not take pride in their appearance.

Likewise, the appearance of an object can be misleading.

The train designed by Raymond Loewy that looked really fast was, in fact, no faster than any other train [17].

Behaviour is shaped by innateness and experience [1].

At the moment, most products have closed behaviours, rigid and not easily modified by experience. Open behaviour products have an innate (inbuilt) way of reacting, but are flexible and easily altered through learning. The Sony Aibo dog is one of very few products that have this inbuilt and adaptive way of reacting within a product. Aibo is given 'a puppy-like personality that develops as it is played with.' [1]. Interaction between a user and an object might also be classed as intelligent or unintelligent. The difference between these two is not in the intelligence of the object, but that of the interaction.

'We dis-cover intelligence not in things (be they machines or animals) but in our interaction with them.' [0].

Previous work at the Centre for Design Research (CfDR)

The direction of the project described in this paper has been influenced by previous research in the CfDR to understand ways in which the characteristics and qualities of movement could be designed into products to promote greater empathy with users. The hypothesis of that work stated that 'introducing designed physical movements will enrich product behaviour' [1].

Understanding Human Personality

At this stage of our work we did not see a logical order in which to study brand per-sonality and product personality or the nature of their relationship to the user. To un-derstand product personality, the components that make up human personality must be made explicit. In the first instance, our project team, brainstormed the differences between emotion and personality and soon realised that emotions were just one of the many factors built into personality. We investigated various definitions of personality that use methods of differentiation and modelling and rely on natural inclination of personality types to behave and think in certain ways, e.g.: Myers

Briggs [5], Ey-senck [6] and the Big 5 [7].

However, as explained in 1., the complexity of human personality is such that we needed to simplify the scope of our study in its application to product personality to make it practicable. By researching different definitions that describe the components of human personality, we limited the study to key factors of; emotion, behaviour, appearance, habits and temperament [].

Identifying Philips Brand Personality Traits

In order to begin to correlate human personality Later on in our research we revised our thinking, on the basis that consumers prefer products that are similar to that of their own selfconcept (see section 1.) the project team decided that 'productality' should be designed around the user rather than a company's brand personality traits. This does not mean that a company will not be able to develop or reinforce a brand character, simply that the starting premise should be based around the user. Naturally, the manner in which this is done ought to consider and reinforce the brand character, where possible.

Human & Product Personality Comparison -Notes on Language

In order to advance the study of product personality, we could not presume that it holds the same components that build up human personality. To remove preconceptions that might arise from the term; product personality, we decided that we needed to rename it as 'productality', which is how it is referred to throughout this paper.

'Productality' is the conjoined term of product and personality. Its conceptual defini-tion is that it should encompass all the factors in a product that provide it with character and behaviour including visual appearance and physical interactions. Further dis-tinctions were made between the language associated with people and products using less pejorative terms. Rather than human appearance we would use 'Product Aesthet-ics'.

Personality trait could be referred to in product language as 'Productality Attrib-ute' and human behaviour extremes as 'Product Behaviour Limits'.

Ontology of User-Product Interaction

In the same way that a product cannot experience an emotion, a product cannot develop habits or portray temperaments; rather they are implemented through the innate behaviour designed into that product to be perceived by people as appropriate. We decided to refer the components of 'productality' therefore to behaviour, where behaviour is 'the action or reaction of [or to] something' [1]. We need to understand how product behaviour might enhance an experience and how this experience might determine the relationship between user and product. Personality might be defined as the constant character that is formed over time [] and we must therefore look at 'productality' in the same way as use over time. Take an old car for example. The owner might be the only person who knows its 'perceived temperament' on ignition hence a certain bond is formed.

Reviewing & Evolving Previous Work

The CfDR's previous work noted that industrial design practice uses many tools and techniques that facilitate

Qualities of Movement

Reviewing the previous work helped us to familiarise ourselves with the issues in-volved when deliberately designing a movement rather than movement arising as a by-product of the design process. We decided to take a step on from that body of work, by investigating and analysing the qualities that can make a movement appear to have character. We drew up a matrix of existing products that contain movement and analysed our perceptions of them in terms of the function, compared to their tac-tility, sound, visual character and emotional cues. We noticed that one particular CD player looked like it was giving the CD gracefully rather than ejecting it. We noted through our observations that a 'gesturelike' movement suggested more character than a purely mechanical-like movement.

We associate some gestures with certain meanings. For example, an up and down movement at the top of a form could, if designed correctly, look as though it is nodding. We filmed ourselves product miming, where we performed a certain gesture associated with a certain product, such as using a computer mouse but without the mouse. From this we learned that we associate certain gestures with certain products. The gesture is perceived as intrinsic to its use. This realisation raised the question; could a motion, which contains a gesture, represent the Philips brand or can it only as-sociate with a certain type of product?

Motion Mood Boards

Animation Methods

We developed a case study using animation methods that focus on how the story is created rather than the finished product. When a sketch or model is animated it can appear to have a personality. Animators get into the world of their characters, such as in a 'Bugs Life' [6], where they literally crouched amongst the plants to get a feel for the

Ceremony as a Way to Enrich Interaction

We then decided to explore ways in which we might increase the empathy of users with the product/brand.

We conjectured that if we could create a product that would behave in different ways according to the scenario, perhaps the user's relationship with the product would improve, hence their recollection of the experience of it.

Enhancing Interactions Using Associations

Using associations as a method of design can help to bridge the gap between new ex-periences and known experiences. With the project undertaken by our team to design an iron, the association of creating fire using two sticks was used as a method for de-signing the interaction of heating up (see figure ). In the same way that the previous work in the CfDR relied on associations; so that 'if a person understands the form and it is not 'alien' then they tend to be able to move on and understand a motion' [] an association might provide known interactions.

Our study concluded that too many unknowns might inhibit the user's enjoyable experience of interaction with a product. A gesture might use associations in a way to convey a required meaning. This might be taken forward as a recognised gesture within Philips products when a product re-quires that it be heated up, however, there are many contextual factors that can affect understanding of the connotation of a gesture. As with any product, there will be certain core functions and others that are peripheral. For example: a functional requirement might be that the iron needs to provide steam. The non-functional requirement might be how this steam is released by a button that is perhaps spherical, square or by some other interface.

We found it necessary to define the functional requirements clearly from the start in order to organise, prepare and focus our creative abilities on how each function might behave using our analysis of the nonfunctional. In this respect, the functional might be described as 'what' is being designed. Each design project is then made dis-tinctive through the use of the non-functional requirements, or 'how' the design is implemented. We have looked at the 'how' and understood the 'what' as a given.

Beyond the Visual

Aesthetic, semantic and symbolic meanings in design are complex in their relation to the senses, the context and the consumer's responses [5]. First impressions form a large part in the associated character that we give to 'things' and the impression is largely based on visual data.

However, product personality is not just about visual capabilities that the user gets to know, then 'productality' might be instigated (see figure ). The requirement to sup-port sustainable business for an interactive product manufacturer is that the modifying impression becomes catalytic to a virtuous cycle not a regressive one in terms of user experience. with the object. Judgements are formed over time as the user gets to know how the product will respond to their interaction and behaviour. If this process is followed faithfully it is expected that it will enhance the user's experience and the perceived 'productality' of the object. Reflecting and portraying these findings in order to determine what is of value and worthy of taking forward is part of the challenge of the project. We anticipate that as we learn more from our research, the nature of the new tools and techniques we are deriving will be changed or refined.

Initial Design Process

Initially we planned to directly apply the Philips brand traits or 'Productality Attrib-utes' to each designed feature (see figure 5). For example, with the iron when we tried to design a way of filling the water, we might have designed it in a caring, un-derstanding or playful way and the most effective concept would then be taken for-ward. However, we soon realised that this method of designing, which we referred to as 'designing by numbers' constrained our creative abilities because it was too linear and would therefore not appeal to design practitioners.

Evolution of the Behaviour and Interaction

Diagram

We realised that it is important to be aware of the limits of the products' behaviour, in order to ascertain the level of response that an object is giving within these limits at any one time. For example, it is rare that we use a car to its extremes, for example driving as fast as it can go.

However, if we impel that car to its maximum limit, we are made fully aware of its capabilities and therefore able to gauge the level at which we are pushing the car at any one time. A similar theory developed by the previous CfDR team was known as 'freedoms of movement'. This theory was used to provide 'an understanding of what range of movements an object is capable of performing' [6].

We started out with the idea of applying levels of response to each functional requirement. For example, with the iron, there were different levels of 'thirst' to indi-cate the amount of water in the tank. By looking at both the input and output, we could start to see a pattern developing between user interaction and product response. Figures 6 and 7 show the evolution of the diagrams over the two project stages to date. Within the behaviour and interaction diagram there are various definitions that need to be detailed. The four inner circles represent the user input and the user feedback. Each of these is split into emotional and physical. These will be explained using the example of switching a product on or off:

User Interaction: The physical interaction is the literal 'pushing of the button'; the emotional interaction might be 'the manner in which that button is pushed'. This is because our belief is that the emotional state manifests itself in a physical manner. For example the emotional state of feeling angry might be manifested in a hard push of the button.

User Feedback: The physical feedback from the product to the user will be that the device is on or off and if applicable on or off in a certain manner. So to use the same example, the product is on in an angry way.

The emotional feedback is a reflection on the physical feedback. If the user recognises that the product is on in a certain way (physical feedback) this will prompt a certain emotion. So in this instance, the user might be prompted to try to relax as they recognise that they have treated the product in an angry manner.

The products are represented in the circles surrounding the user. The faded circles with diamond and square inners indicate the 'related product' that might interact with the main product under enquiry. To look at the main product, a breakdown is re-quired to define product response and product action:

• Product Response -The product might react to a direct user interaction. The response to being switched on would be that the product is on, which if done in a certain manner, will be indicated through this response.

• Product Action: The result of either an indirect interaction or the need to present a message providing an immediate physical feedback to the user. A product action might be the indication of a battery running low, which may then start the interac-tion with the user.

Timeline of use

The timeline of use outlines all of the procedures associated with the product in mind and could be described as a specification of the functional requirements.

Deciphering the process through which a product is used is of utmost importance. A thorough in-vestigation into every ascertainable interaction with the product is carried out. With the iron: the iron, ironing board and garment to be ironed were all included in the timeline in order not to overlook any of the important stages of use (see figure ). We have also devised a standard sketch sheet (see figure 10, right) where the designer can self-analyse and guide their concept in relation to context/scenario, limits, ceremony, behaviour, and interaction, ease of use and personality traits.

Example of Use of the Behaviour and Interaction Diagram

Each behavioural feature can be applied as a spoke to the main behaviour and interaction diagram to show how everything links together. Figure 11, the design of the beard trimmer represented on the diagram, shows how the levels of behaviour or number of interactions might vary depending on the design of each feature. The number of spokes reveals the complexity of interaction that has been developed.

Findings from the Iron & Beard Trimmer

Through the iron project, we refined our methods and found that there were certain elements to reflect upon for application to the method used for the beard trimmer project. We found three levels of behaviour to be the optimum for recognising a transi-tion between levels. We concluded that the start and finish of the interaction should link together. How the user last used the product should be reflected in the product response on next use. Enabling a reward to be given for good use at the consequent use. However the be-havioural response should be helpful, not annoying. A balance is required, that allows the user to benefit from a more pleasurable experience. For example, with the beard trimmer, a more pleasurable way of charging could only be used if the trimmer had been treated well on the previous use.

The method of using levels to design behaviour into products has the potential to create interactions that might, in certain contexts, be deemed annoying. It is important to be aware of this, as well as try to evaluate the advantages versus disadvantages. A product's performance should not decline below a tolerable average of acceptance.

Further Research

At the moment we are using our understanding of interaction and behaviour based on our explorative action research, including the evolving methods, tools and techniques for designing with behaviour. The next project cycle will attempt to develop form and dual personality conjointly using two user personas as inspiration in the design of a DVD HD Recorder. The intention is to gain further insight into the elements of be-haviour that enhance the experience and user product relationships while at the same time, seeing if one form can provide two behavioural identities suitable to two differ-ent personas.

This raises various questions: Can the perceived behaviour of a product vary in the same form and therefore can different products take on the same behav-ioural identity in different forms? Can this reinforce company brand traits? Can there be such a thing as product impersonation and if so; will this give rise to new concerns about product behavioural affectation, fidelity and integrity, in the way traditional in-dustrial design debates on truth to materials raged a generation ago [50]. Whilst it is not expected that all of these questions can be addressed in the timeframe remaining for this research, a fourth and final project cycle will be carried out feeding the in-sights gained to date into a more context related design.

We also intend to accumulate user interpretations of the product outcomes in response to the above project cycles.

These should help us to gain further insight into how the emotional feedback that we trigger will be described for the benefit of other designers wishing to apply our methods. We propose to ascertain how or whether the manner in which a product is used can be reflected in the responding actions (behaviour/productality) of the product and therefore inform the user of their own behaviour, whilst betokening a memorable experience of the product manufacturers brand identity.

Freedom

An important aspect when designing for this user group was freedom. This, combined with the energy, creativity and the child's natural urge to discover, was the starting point for the project. As designer you only have to

give the child something to discover, something that can be discovered and isn't bound to a big set of rules.

The interactive tiles are a good step into this direction.

Moreover, positive aspects of playing can be found back in the interaction with the tiles. Let me shortly elaborate on some:

Social Interaction

Since children from this age are still very individual, it won't be a good idea to force them to play with others.

The basic goal is to let them have fun, play. With the interactive tiles, it is possible to play alone, but playing together has a clear added value. The children see that they can do more when playing together.

Motor Skills

While the children play with the tiles, they are physically very active. The stamp, jump on the tiles, move and throw the tiles and walk and run over them. During these activities, the child's motor skills are trained (e.g. when jumping from one tile to another or balancing on a tower).

Build & Use

The tiles are robust and have about 0 cm square.

Children can quickly build simple things and use them immediately (like a path or floor to walk over, or a tower to stand on). Because they can build things, use it, and break it down again, they can enjoy themselves for a long time with it…

Embedded

The tile is built in such a way that it can respond flexibly. Every tile has its own microcontroller. This microcontroller can easily be programmed to respond differently. This way, different types of behavior can be given to the tiles. During this project I haven't experimented with the behavior.

Demo

The demo will exist out of a set of working tiles (that can be tried out by people) and a video of the tiles being used by children. The video shows clearly how the tiles can be used by the children, and how the children respond to the concept.

Design explorations

In this chapter I present three design approaches. Each unfolds a different way to transfer an idea of interaction into a physical object. All of them share the goal of communicating the functionality to users and all invite a certain behavior -an interaction that engages the understanding of the objects and its use. These approaches are neither formal nor contradictory -they can therefore be combined in one single design.

Form follows function: materialize a concept and follow its new physical logic

One way to express a concept in physical form is to define its rationale then use basic shapes or compositions, which express the same meaning. This way, the design is in fact a D conceptual model, translating an idea into material, mostly using abstract shapes. In this approach the function defines the design. Table: An by users, not according to their function, but because of their shape and location in relation to the blanket).

The Message

Form suggests interaction: 'Packing' new functionality in a familiar icon

In this approach the design is inspired by a parallel model, which holds similar qualities of interaction. Borrowing a familiar shape (a cultural icon), can inform the user regarding the function of the object and hint to the opportunities of its experience.

Ambra

performing the USB Memory Key test, FOL clearly expressed her frustration and deceit when it was difficult to insert the key to the laptop