Human Agent Interaction

This paper describes Gamebots, a multi-agent system infrastructure derived from an Internet-based multi-player video game called Unreal Tournament. The game allows characters to be controlled over client-server network connections by feeding sensory information to client players (humans and agents). Unlike other standard test-beds, the Gamebots domain allows both human players and agents, or bots, to play simultaneously; thus providing the opportunity to study human team behavior and to construct agents that play collaboratively with humans. The Gamebots system provides a built-in scripting language giving interested researchers the ability to create their own multi-agent tasks and environments for the simulation.

We give a preliminary perspective on the basic principles and pitfalls of adjustable autonomy and humancentered teamwork. We then summarize the interim results of our study on the problem of work practice modeling and human-agent collaboration in space applications, the development of a broad model of human-agent teamwork grounded in practice, and the integration of the Brahms, KAoS, and NOMADS agent frameworks We hope our work will benefit those who plan and participate in work activities in a wide variety of space applications, as well as those who are interested in design and execution tools for teams of robots that can function as effective assistants to humans.

Increasingly, social robots are being used in business contexts. A key area of research interest is whether people find these machines as trustworthy and credible. Our initial findings show that while the robot gets lower ratings on its expertise levels - yet, people find it attractive and trustworthy.

This paper describes Gamebots, a multi-agent system infrastructure derived from an Internet-based multi-player video game called Unreal Tournament. The game allows characters to be controlled over client-server network connections by feeding sensory information to client players (humans and agents). Unlike other standard test-beds, the Gamebots domain allows both human players and agents, or bots, to play simultaneously; thus providing the opportunity to study human team behavior and to construct agents that play collaboratively with humans. The Gamebots system provides a built-in scripting language giving interested researchers the ability to create their own multi-agent tasks and environments for the simulation.

The present study investigated the role of age in the perception of emotional nonverbal behaviors of a virtual assistant in a 2 (seniors vs young participants) x 2 (happy vs sad situations) x 4 (three emotional nonverbal behaviors related to each situation vs neutral behavior) mixed factorial design. In the study, a virtual agent acted as an assistant to review the imaginary monthly schedule of the participant. After uttering each schedule, the agent showed different emotional nonverbal behaviors to express empathy. The stimulus materials were presented to 60 participants (30 elderly people and 30 younger adults) in 25 videos. All participants had to rate the agent's friendliness, intelligence, empathy, trustworthiness, and helpfulness immediately after watching each video. The results indicated that in the presence of the emotional nonverbal behaviors the elderly rated the agent as more empathic and trustworthy compared to the younger adults. The data also revealed that there were differences with respect to rating specific emotional nonverbal behaviors as more empathic than others. Elderly people perceived Dropping the Arms plus Sad Face, Head down, Sad Face, Head Nod plus Smile, and Smile as more empathic, and the nonverbal behaviors perceived as most empathic for the younger adults were Dropping the Arms plus Sad Face, and Head down. CCS CONCEPTS • Human-centered computing → Human computer interaction (HCI) → HCI design and evaluation methods KEYWORDS Human-agent interaction; empathy; nonverbal-behavior Permission to make digital

In this paper I am presenting a behavioural model of the student as the main actor of a virtual learning environment. The model is part of a larger project -DANTE -Socio-Cultural Models implemented through multi-agent architecture for e-learning -that has as main objective the development of a model for the virtual education system, student centred, that facilitates the learning through collaboration as a form of social interaction. The project presumes the combination of the artificial intelligence (multi agent) system with elements of the socio-cultural theory of learning by collaboration and human actors. The model requires its own universe in which the human agents interact with the artificial ones (software agents). Student behaviour is modelled as a function of beliefs, affects, learning styles, intention and motivation, taking into account the context influences. The model is tributary to traditional behaviour and learning styles models (Rogers, Jung, Piaget, Fishbein, Kolb and followers). The model has been validated on a sample of 450 students at the university (18-22 years old, computer science and informatics specialities).

As we find greater potential for agent instructors, we must be aware of how their language use can affect the user and interaction as a whole. This study investigates the use of intentionally imprecise or vague language as a communicative strategy to mitigate the impact of instructions. We look at the effects it has on improving the perception of agents and user performance. A series of assembly tasks were ran in which users constructed Lego models with the spoken instructions of vague and non-vague agents. Results show that though the non-vague agent was seen as more direct and authoritative, responses to other attributes and performance were much more varied. Findings suggest there is potential for vague language human-agent interaction, though there are several obstacles in agent design to overcome first.

In this paper we introduce a group-messaging interface that allows humans to efficiently interact with a group of agents through a hierarchical and customizable text protocol. Our approach is presented in the context of the MAST mobile agent-based framework for security and administration of large scale computer networks. The MAST framework is primarily human-centric and directly supports human-agent interaction that enables customized agents to notify administrators and react to abnormal environmental conditions. The proposed IRC-like interface was developed and tested in the context of MAST. In this paper we present the group-manager interface in contrast with other agent interfaces currently available in the MAST framework.

New Media enthusiasts have long extolled the digital medium as opening up new forms of storytelling. Interactive stories are said to enable the user to enact the narrative interpretations and decisions previously internal in other media. 'Interactivity' is meant to allow user to control and manipulate the story in many ways. This paper investigates this presumption by exploring current theories -from human-computer interaction, human-agent interaction, human-robot interaction and reader response -and analysing current applications.

Because ever more powerful intelligent agents will interact with people in increasingly sophisticated and important ways, greater attention must be given to the technical and social aspects of how to make agents acceptable to people [16.72]. From a technical perspective, we want to help ensure the protection of agent states, the viability of agent communities, and the reliability of the resources on which they depend. To accomplish this, we must guarantee, insofar as is possible, that the autonomy of agents can always be bounded by an explicit enforceable policy that can be continually adjusted to maximize the agents' effectiveness and safety for both human beings and computational environments. From a social perspective, we want agents to be designed to fit well with how people actually work together. Explicit policies governing human-agent interaction, based on careful observation of work practice and an understanding of current research in the social sciences and cognitive engineering, can help assure that effective and natural coordination, appropriate levels and modalities of feedback, and adequate predictability and responsiveness to human control are maintained. These factors are key to providing the reassurance and trust that are the prerequisites to the widespread acceptance of agent technology for non-trivial applications.

The study of agent helpers using linguistic strategies such as vague language and politeness has often come across obstacles. One of these is the quality of the agent's voice and its lack of appropriate fit for using these strategies. The first approach of this article compares human vs. synthesised voices in agents using vague language. This approach analyses the 60,000-word text corpus of participant interviews to investigate the differences of user attitudes towards the agents, their voices and their use of vague language. It discovers that while the acceptance of vague language is still met with resistance in agent instructors, using a human voice yields more positive results than the synthesised alternatives. The second approach in this article discusses the development of a novel multimodal corpus of video and text data to create multiple analyses of human-agent interaction in agent-instructed assembly tasks. The second approach analyses user spontaneous facial actions and...

This paper presents an investigation of the self-serving biases of interface agent users. An experiment that involved 202 MS Office users demonstrated that, in contrast to the self-serving hypothesis in attribution theory, people do not always attribute the successful outcomes of human-agent interaction to themselves and negative results to interface agents. At the same time, it was found that as the degree of autonomy of MS Office interface agents increases, users tend to assign more negative attributions to agents under the condition of failure and more positive attributions under the condition of success. Overall, this research attempts to understand the behavior of interface agent users and presents several conclusions that may be of interest to human-computer interaction researchers and software designers working on the incorporation of interface agents in end-user systems.

The study of agent helpers using linguistic strategies such as vague language and politeness has often come across obstacles. One of these is the quality of the agent's voice and its lack of appropriate fit for using these strategies. The first approach of this article compares human vs. synthesised voices in agents using vague language. This approach analyses the 60,000-word text corpus of participant interviews to investigate the differences of user attitudes towards the agents, their voices and their use of vague language. It discovers that while the acceptance of vague language is still met with resistance in agent instructors, using a human voice yields more positive results than the synthesised alternatives. The second approach in this article discusses the development of a novel multimodal corpus of video and text data to create multiple analyses of human-agent interaction in agent-instructed assembly tasks. The second approach analyses user spontaneous facial actions and gestures during their interaction in the tasks. It found that agents are able to elicit these facial actions and gestures and posits that further analysis of this nonverbal feedback may help to create a more adaptive agent. Finally, the approaches used in this article suggest these can contribute to furthering the understanding of what it means to interact with software agents. ACM Reference Format: Leigh Clark, Abdulmalik Ofemile, Svenja Adolphs, and Tom Rodden. 2016. A multimodal approach to assessing user experiences with agent helpers.

A two-phase procedure, based on biosignal recordings, is applied in an attempt to classify the emotion valence content in human-agent interactions. In the first phase, participants are exposed to a sample of pictures with known valence values (taken from IAPS) and classifiers are trained on the physiological data recorded. During the second phase, biosignals are recorded for each participant while interacting with an embodied conversational agent (ECA) and the classifiers trained in the first phase are applied. The results from the procedure are promising and are discussed in the paper together with the problems encountered and the suggestions for possible future improvement.

Accurately interpreting and expressing affect is fundamental to empathetic relationships. A platform for sensing and interpreting several aspects of users' nonverbal affective information and responding through an expressive agent has been developed. The platform includes integration of multi-modal affective sensors with a real time inference engine, a behavior engine, and a 3d scriptable expressive humanoid agent within a graphical virtual environment. Currently the sensors include a pressure-sensitive mouse, a BlueTooth wireless skin conductivity sensor, a TekScan pressure sensor on a chair, and a stereo head tracking system as well as an IBM Blue Eyes infrared-sensitive camera. These sensors feed into custom algorithms for analysis of individual channels of information, such as postural and facial expressions, which in turn are combined with additional channels of information to make an inference about the user's affective state. The system further synchronizes this sensor data with the agent behaviors and with video of the user and his or her on-screen activity. This platform is seen as a generalpurpose tool applicable to research in several areas, including how to design an affective learning companion, and how to further basic understanding of empathy and emotion contagion in human-agent interaction.

Linking intelligent-agent technologies with human teams requires a sophisticated understanding of concepts cutting across the psychological and computational sciences. This type of human-agent teamwork requires the blending of a complex set of attitudinal, behavioral, and cognitive components for coordination to proceed effectively. Similarly, Situation Awareness is a complex phenomenon that requires the integration of multiple individual and team level processes. While situation awareness has been studied extensively, comparatively little research has looked at the relation between socio-cognitive factors and situation awareness. In this paper, we present a representative framework for understanding how socio-cognitive factors can be used to augment our understanding of certain components of situation awareness. We then present a set of research propositions that can be explored in mixed reality simulation-based environments. This line of research explores the factors that drive the human component of human-agent interaction in order to understand how it is that performance in such teams can be simulated to improve coordinative processes.

The context of interaction is ever changing but to some extent, investigating these contexts can often be confined to Inter-human interaction for example, understanding how people from different cultures create better super diverse communities in multicultural cities and actualize participatory design of systems as culturally diverse teams. However, increasing pervasive computing is extending the boundaries of interaction to contexts such as automated checkout systems in supermarkets, satellite navigation systems giving directions to drivers and workers sharing work spaces with intelligent robots. This creates the third space of hybridity or in-betweens where human interactional culture meets agent interactional culture thus creating a new culture. This paper reports a scoping study that uses interaction strategies to understand the patterns of nonverbal listenership behaviours of speakers of English as a second language (L2) taking instructions from English as first language (L1)-mode devices during assembly tasks. Ten participants (L2 speakers and student teachers of English) were tasked with assembling two Lego models using vague verbal instructions from a computer interface (L1 agent instructor) and one human instructor (L2 speaker of English) using his voice within a 15-minute time limit per iteration. The interface used three voices of which two are synthesised and one is non-synthesised human recording by a voice actor. A 5-hour long multimodal corpus was built and analysed from these interactions. The results indicate that although there are differences in the way humans interact with agent and other humans, it is possible for humans to treat agents the same way that they treat other humans during interaction. The results imply that English Language Teaching (ELT) theories have to understand new functions that emerge from interaction between humans and agents, attention has to be paid accommodation strategies required to enhance communication between them and ELT materials design should be user-driven.

Because ever more powerful intelligent agents will interact with people in increasingly sophisticated and important ways, greater attention must be given to the technical and social aspects of how to make agents acceptable to people [4], p. 51]. The technical challenge is to devise a computational structure that guarantees that from the technical standpoint all is under control. We want

We propose a new approach to human-robot interaction (HRI) in which a common target object is anthropomorphized using attachable humanoid parts. With this approach, the user perceives that the target has an intention and senses what the imaginary body of the target looks like through the attached body parts. We experimented how users accepted the intentions and imaginary body image of a common target object using the humanoid parts. We also compared the resulting HRI with that in the case of the general communication robot Robovie to demonstrate the possibilities our proposed method offers. The results indicated that an anthropomorphized target object can interact with users through this approach. Furthermore, in comparison to interaction with an independent communication robot, with this approach, users remembered the functions of the anthropomorphized target to a greater extent and were more intimate with it.

We propose a new approach to human-robot interaction (HRI) in which a common target object is anthropomorphized using attachable humanoid parts. With this approach, the user perceives that the target has an intention and senses what the imaginary body of the target looks like through the attached body parts. We experimented how users accepted the intentions and imaginary body image of a common target object using the humanoid parts. We also compared the resulting HRI with that in the case of the general communication robot Robovie to demonstrate the possibilities our proposed method offers. The results indicated that an anthropomorphized target object can interact with users through this approach. Furthermore, in comparison to interaction with an independent communication robot, with this approach, users remembered the functions of the anthropomorphized target to a greater extent and were more intimate with it.

The context of interaction is ever changing but to some extent, investigating these contexts can often be confined to Inter-human interaction for example, understanding how people from different cultures create better superdiverse communities in multicultural cities and actualise participatory design of systems as culturally diverse teams. However, increasing pervasive computing is extending the boundaries of interaction to contexts such as automated checkout systems in supermarkets, satellite navigation systems giving directions to drivers and workers sharing work spaces with intelligent robots. This creates the third space of hybridity or inbetweeness where human interactional culture meets agent interactional culture thus creating a new culture. This paper reports a scoping study that uses interaction strategies to understand the patterns of nonverbal listenership behaviours of speakers of English as a second language (L2) taking instructions from English as first language (L1)-mode devices during assembly tasks. Ten participants (L2 speakers and student teachers of English) were tasked with assembling two Lego models using vague verbal instructions from a computer interface (L1 agent instructor) and one human instructor (L2 speaker of English) using his voice within a 15-minute time limit per iteration. The interface used three voices of which two are synthesised and one is non-synthesised human recording by a voice actor. A 5-hour long multimodal corpus was built and analysed from these interactions. The results indicate that although there are differences in the way humans interact with agent and other humans, it is possible for humans to treat agents the same way that they treat other humans during interaction. The results imply that English Language Teaching (ELT) theories have to understand new functions that emerge from interaction between humans and agents, attention has to be paid accommodation strategies required to enhance communication between them and ELT materials design should be user-driven.

In this paper, we outline an approach to policy-based coordination in joint human-agent activity. The approach is grounded in a theory of joint activity originally developed in the context of discourse, and now applied to the broader realm of human-agent interaction. We have been gradually implementing selected aspects of policy-based coordination within the KAoS services framework and have been developing a body of examples that will guide additional testing of these ideas through detailed studies of work practice.

As we find greater potential for agent instructors, we must be aware of how their language use can affect the user and interaction as a whole. This study investigates the use of intentionally imprecise or vague language as a communicative strategy to mitigate the impact of instructions. We look at the effects it has on improving the perception of agents and user performance. A series of assembly tasks were ran in which users constructed Lego models with the spoken instructions of vague and non-vague agents. Results show that though the non-vague agent was seen as more direct and authoritative, responses to other attributes and performance were much more varied. Findings suggest there is potential for vague language human-agent interaction, though there are several obstacles in agent design to overcome first.

This paper summarizes our efforts to address some of the technical and social aspects of agent design for increased human acceptability. From a technical perspective, we want to be able to ensure the protection of agent state, the viability of agent communities, and the reliability of the resources on which they depend. To accomplish this, we must guarantee insofar as possible that the autonomy of agents can always be bounded by explicit enforceable policy that can be continually adjusted to maximize their effectiveness ...

This paper summarizes our efforts to bring together and extend the best in current theory and technologies for teamwork-centered autonomy for space applications. Traditional planning technologies at the foundation of intelligent robotic systems typically take an autonomycentered approach, with representations, mechanisms, and algorithms that have been designed to ingest a set of goals and output a complete plan in the most efficient and sound fashion possible. A teamwork-centered autonomy approach, on the other hand, takes as a beginning premise ...

Participation in natural, real-time dialogue calls for behaviors sup- ported by perception-action cycles from around 100 msec and up. Generating certain kinds of such behaviors, namely envelope feedback, has been possible since the early 90s. Real-time backchannel feedback related to the content of a dialogue has been more difficult to achieve. In this paper we describe our progress in allowing

This paper outlines a preliminary perspective on teamwork and adjustable autonomy in groups involving a mix of humans and autonomous agents. Unlike other approaches to agent teamwork, a humancentered perspective to human-agent interaction is used. The paper describes how we are integrating the Brahms and KAoS agent frameworks in order to model and simulate realistic work situations in space and to support the design of robotic and software agents for human-agent teamwork.

Social modifi cation using implementation of partial agency toward objects ability of various robotics technologies, such as actuators and sensors, means that researchers in informatics have many methods to access the real world. Moreover, the rapid development of computers has enabled mechanical engineers to develop more sophisticated control methods, and much effort is aimed at developing ubiquitous and humanrobot interaction interfaces.

In this paper we introduce a group-messaging interface that allows humans to efficiently interact with a group of agents through a hierarchical and customizable text protocol. Our approach is presented in the context of the MAST mobile agent-based framework for security and administration of large scale computer networks. The MAST framework is primarily human-centric and directly supports human-agent interaction that enables customized agents to notify administrators and react to abnormal environmental conditions. The proposed IRC-like interface was developed and tested in the context of MAST. In this paper we present the group-manager interface in contrast with other agent interfaces currently available in the MAST framework.

From virtual assistants to social robots, people are increasingly interacting with intelligent and highly communicative technologies throughout their daily lives. This shift from communicating with people to communicating with people and machines challenges how scholars have theorized and studied communication. Human-Machine Communication: Rethinking Communication, Technology, and Ourselves addresses this transition in how people communicate and who, or what, they communicate with and the implications of this evolution for communication research. Geared toward scholars interested in people’s interactions with technology, this book serves as an introduction to human-machine communication (HMC) as a specific area of study within communication (encompassing human-computer interaction, human-robot interaction, and human-agent interaction) and to the research possibilities of HMC. This collection includes papers presented as part of a scholarly conference on HMC, along with invited works from noted researchers. Topics include defining HMC, theoretical approaches to HMC, applications of HMC, and the larger implications of HMC for self and society. The research presented here focuses on people’s interactions with multiple technologies (artificial intelligence, algorithms, and robots) used within different contexts (home, workplace, education, journalism, and healthcare) from a variety of epistemological and methodological approaches (empirical, rhetorical, and critical/cultural). Overall, Human-Machine Communication provides readers with an understanding of HMC in a way that supports and promotes further scholarly inquiry in a growing area of communication research

The htmlButler project aims at enhancing the human-agent interaction via the visual wrapper technology while preserving versatility. htmlButler will allow to an untrained user who has only the most basic Web knowledge to visually specify simple but useful wrappers and to a more techsavvy user to visually or otherwise specify more complex wrappers. htmlButler was started 2005/2 and is based on visual wrapping technology research carried out in the Lixto project since 2000. What is new in htmlButler is (a) that the application is entirely server based, the user accessing it through his or her standard browser, (b) that, because of the centralized wrapper configuration and processing, the knowledge about popular wrappers can be leveraged to facilitate the specification of wrappers for new users, and (c) that users can contribute narrow and precise ontologies that help the system in recognizing potential meaning in Web pages, thereby alleviating the complexity of future wrapper configurations

In this work, we propose a design for a user attention model featuring three core components. Our system components can work in real-time, offering indications of user attention from different sensory inputs (both visual and neurophysiological). Intention of the current work is to keep the equipment as unintrusive as possible, while keeping the confidence of the inputs as high as possible. We discuss potential applications of such a system, particularly with respect to evaluating user attentive behaviour during human-agent interactions and as a more natural interface for interacting with agents.

This paper summarizes our efforts to address some of the technical and social aspects of agent design for increased human acceptability. From a technical perspective, we want to be able to ensure the protection of agent state, the viability of agent communities, and the reliability of the resources on which they depend. To accomplish this, we must guarantee insofar as possible that the autonomy of agents can always be bounded by explicit enforceable policy that can be continually adjusted to maximize their effectiveness ...

Because ever more powerful intelligent agents will interact with people in increasingly sophisticated and important ways, greater attention must be given to the technical and social aspects of how to make agents acceptable to people [4], p. 51]. The technical challenge is to devise a computational structure that guarantees that from the technical standpoint all is under control. We want

Cet article présente une architecture générale vers une approche plus générique pour les agents conversationnels. Notre architecture contient des modules génériques (dans le sens d'indépendant de l'application) de traitement des langues naturelles (LN) basés sur les ontologies pour l'interprétation de commandes. Nous nous intéresserons particulièrement à la présentation du générateur d'évènements et du gestionnaire de dialogue, le tout reposant sur une approche ascendante pour l'appariement des commandes de l'utilisateur avec l'ensemble des actions possibles.

RÉSUMÉ. Cet article présente une architecture générale vers une approche plus générique pour les agents conversationnels. Notre architecture contient des modules génériques (dans le sens d'indépendant de l'application) de traitement des langues naturelles (LN) basés sur les on- tologies pour l'interprétation de commandes. Nous nous intéresserons particulièrement à la présentation du générateur d'évènements et du gestionnaire de dialogue, le tout reposant sur une approche ascendante pour l'appariement des commandes de l'utilisateur avec l'ensemble des actions possibles. ABSTRACT. This paper presents a general architecture towards a more generic approach to con- versational agents. Our architecture contains generic (in sense of application independent) natural language (NL) modules that are based on ontologies for command interpretation. We focus on the presentation of the event generator and dialogue manager modules which rely on a bottom-up approac...