Digital Voice Assistants (DVA’s)

Expert panel organized at IARIA CENTRIC (Fourteenth International Conference on Advances in Human-oriented and Personalized Mechanisms, Technologies, and Services) 2021, showcasing MITRE and other work on this topic.

Agenda:
Chair
• Jeff Stanley, Lead Human-Centered Engineer, MITRE, USA
Panellists
• Charles Rinker, PRSONAS, USA
• Jennifer Strickland, MITRE, USA
• Natalie Friedman, Cornell Tech, USA
• Corey Miller, MITRE / Georgetown University, USA
• Sara Beth Elson, MITRE, USA
• Kelly Horinek, MITRE, USA

Sketches in Voice User Interface explores the conversational and evocative aspects of peoples’ interactions with no-screen embodied voice user interfaces (VUIs) in domestic spaces. The project uses an annotated research through design methodology to create a series of Sketches in Voice User Interface for relational conversations with users. The research involves an autoethnographic study of existing voice-based virtual personal assistants (VPAs). Informed by these precedents Sketches in VUI are designed through iterative prototyping to explore ways in which VUIs can go beyond the existing virtual personal assistant in our everyday conversations. Unlike the conventional voice-based VPAs (Siri, Alexa, Google Assistant) operating on the commands of the user, the Sketches in VUI drive conversations and take an agentive role in human-computer conversations.
Using the design research approach, this project serves as a bridge between two key contextual voices in the domain of conversational technologies. On one hand, is the tech industry’s case for usability that VUI is ‘the most natural interface.’ On the other hand, is the social sciences case critically calling VUI ‘an artificial nature’ and questioning if conversations with a machine are conversations at all.
The project concludes with an ‘experience study’ to enquire into the experience of participants as they converse with the designed Sketches. The study observes how participants react to the Sketches (behavioural response) and how they feel (emotional response), comparing them to their experience of existing voice-based VPAs, captured via videography and qualitative interviews. The study findings along with the designed Sketches form an annotated portfolio of generated knowledge about relational conversations with embodied voice user interfaces in our intimate spaces.

AI voice assistants are based on software that enters in dialogue with users through speech in order to provide replies to the users’ queries or execute tasks such as sending emails, searching on the Web, or turning on a lamp. Each assistant is represented as an individual character or persona (e.g. “Siri” or “Alexa”) that despite being non-human can be imagined and interacted as such. Focusing on the cases of Alexa, Siri and Google Assistant, this working paper argues that AI voice assistants activate an ambivalent relationship with users, giving them the illusions of control in their interactions with the assistant while at the same time withdrawing them from actual control over the computing systems that lie behind the interface. The paper illustrates how this is made possible at the interface level by mechanisms of projection that expect users to contribute to the construction of the assistant as a persona, and how this construction ultimately conceals the networked computing systems administered by the corporations who developed these tools.

This working paper is an early draft of chapter 6, Simone Natale, Deceitful Media: Artificial Intelligence and Social Life after the Turing Test, forthcoming with Oxford University Press, 2021.

The purpose of this memo is to identify the security needs for the Intelligent Voice Assistant (IVA) Sonya project prior to release. Security cannot hold up the project, because the security phase must be agile so that it follows the projected release date. Also, this memo describes the use of IVA Sonya. Therefore, security must fast track the analysis and assessment.

Voice assistants such as Siri, Alexa, and Google Assistant have recently been the subject of lively debates in regard to issues such as artificial intelligence, surveillance, gender stereotypes, and privacy. Less attention, however, has been given to the fact that voice assistants are also web interfaces that might impact on how the web is accessed, understood and employed by users. This article aims to advance work in this context by identifying a range of issues that should spark additional reflections and discussions within communication and media studies and related fields. In particular, the article focuses on three key issues that have to do with long-standing discussions about the social and political impact of the internet: the role of web platforms in shaping information access, the relationship between production and consumption online, and the role of affect in informing engagement with web resources. Considering these issues in regard to voice assistants not only helps contextualize these technologies within existing debates in communication and media studies, but also highlights that voice assistants pose novel questions to internet research, challenging assumptions of what the web looks like as speech becomes one of the key ways to access resources and information online.

Artificial Intelligence is a technology that plays a major role in education. Nowadays AI is a part of our daily life. Many educational organizations work with the government to implement better AI facilities in education. In this paper, I argue that the human teacher is better than the robot teacher. Lack of human contact is the main issue with robot teachers. Robots are advanced but can't replace teachers instead of this if we can use a voice assistant as a teacher assistant. Voice assistants like Alexa, Siri, Amazon Echo can be used by teachers. Voice assistant able to record audio, can make calls, send messages, give GPS information. The main approach of this paper is to help teachers in their routine work, so teachers can more concentrate on students' progress.

This paper presents a Healthcare system by providing virtual personal assistance using the Amazon Alexa. Amazon Alexa is an application program that understands voice commands and completes that commands for the user. Amazon Echo is not only a smart speaker but operates as an intelligent personal virtual assistant. Alexa is Hands-free that can allow easier access to the medical information. Alexa has been programmed with the information about the patients, doctors, nurses, pharmacy details, events and general information about the Healthcare. Users can interact with the Alexa to find out the scheduled appointments, availability of the doctor, availability of the rooms, etc.

Morana Alač, Yelena Gluzman, Tiffany Aflatoun, Adil Bari, Buhang Jing, and German Mozqueda. "Talking to a Toaster: How Everyday Interactions with Digital Voice Assistants Resist a Return to the Individual." Evental Aesthetics, Vol. 9 No. 1 (2020), 3-53.

Discussions of the problematic relationship between AI and society have recently only heightened. These discussions, nevertheless, remain partial until they take into account how we live with AI technologies in the unremarkable circumstances of our everyday affairs. In arguing for the importance of such a “noticing,” this article centers on the internet of things and associated digital voice assistants (DVAs). These commercial social robots, designed as conversation-oriented devices, manifest their incompleteness in their need for other voices. Paying attention to that relationality at an embodied scale of analysis brings up our involvement in situated interactional production, while also manifesting its reciprocal character. This not only puts into question the conviction of DVA designers that these gadgets will generate effects of presence in relation to an intentional mind, but also gives us the resources to resist a parallel return to the individual that more often transpires in the discussions of the problematic relationship between AI and society. We practice this resistance by evoking efforts in distributed cognition and the extended mind hypothesis, but we also go beyond the instrumentalist reasoning that primarily recognizes the world as carved into convenient tools that can extend our cognition. To do so we focus on the achieved quality of bodies and environments—two constitutive elements of DVA technology—thereby pointing out how the self in the context of the voiced AI importantly derives from the openness between humans and machines in the interactional scenes of which they are a part.

This research examined the effect of the voice assistant's female or male gender and affiliative or assertive style on its likeability, perceived intelligence, trustworthiness, persuasiveness, confidence and the perception of its gender in a binary axis. To gather the data, a random sample of 93 adults were recruited online and participants interacted with the four voice assistants through a webpage. The study involved four assistants with female or male voices; and with affiliative or assertive style of language. The assistants performed a service task. Although there was no significant effect, the findings expand our understanding of the links between human-voice assistant interaction by elucidating the effects and trends on a critical and contemporary subject.

Virtuelle bzw. intelligente persönliche Assistenten (IPAs), die mit Schnittstellen zur Ein-und Ausgabe gesprochener Sprache ausgestattet sind, bevölkern – u. a. in Form smarter Lautsprecher – zunehmend die häusliche Umgebung. Versteht man die Smartness der Geräte als eine auf akustische Daten bezogene Künstliche Intelligenz, treten zwei unterschiedliche Aspekte zutage, die zwar selten zusammen betrachtet werden, aber dennoch zumeist koexistent sind und einander sogar zu einem gewissen Grad bedingen: Zum einen zeigen wir anhand der Interaktion mit Sprachassistenten, dass akustische Intelligenz als maschinelle Intelligibilität auf die Zuarbeit der menschlichen Nutzer*innen angewiesen ist, um kommunizieren zu können. Sie erweist sich diesbezüglich als eine verteilte Form von Intelligenz. Zum anderen erscheint akustische Intelligenz als anbieterseitige Strategie der Beschaffung, Auswertung und ökonomischen Verwertung der von den Nutzer*innen produzierten akustischen Daten, die wiederum auf den technischen Bedingungen maschineller Lernverfahren und Cloud-Infrastrukturen basiert und sich als Datafizierung häuslicher Sprachkommunikation manifestiert.

The emergence of smart speakers and voice-activated personal assistants (VAPAs) calls for updated scrutiny and theorization of auditory surveillance. This paper introduces the neologism and concept of "eavesmining" (eavesdropping + data mining) to characterize a mode of surveillance that operates on the edge of acoustic space and digital infrastructure. In contributing to a sonic epistemology of surveillance, I explain how eavesmining platforms and processes burrow the voice as a medium between sound and data and articulate the acoustic excavation of smart environments. The paper discusses eavesmining in relation to theories of dataveillance, the sensor society, and surveillance capitalism before outlining the potential contributions offered by a theoretical alignment with sound studies literature. The paper centers on an empirical case study of the Amazon Echo and Alexa conditions of use. By conducting a discourse analysis of Amazon's End User Agreements (EUAs), I provide evidence in support of growing privacy and surveillance concerns produced by Amazon's eavesmining platform that are obfuscated by the illegibility of the documents.

This paper summarizes six presentations, including
two paper submissions, in the special track “Design and
Synthesis of Personalities”. This special track has shed light on
different ways MITRE has incorporated elements of
personality into human-machine interaction research, and it
has showcased perspectives from other organizations as well.
The paper explores major open questions for personality
synthesis – such as how to decide on the “right” personality,
how to express that personality, and what guidance is needed
for personality development – as approached by these six
presenters.

Conversational user interfaces (CUIs) such as chatbots and voice assistants are increasingly used to deliver services not just in industry but in government. Therefore, it is increasingly important for CUIs to provide good experiences for constituents with diverse backgrounds and abilities. Existing research on CUI personality focuses on engaging typical target users. Synthesizing existing literature on CUI personalities with principles for inclusive design, we discuss how to design CUI personalities that provide good experiences for diverse users. Key considerations are to consider the user’s unique situation, their expectations and preferences toward technology, and their purpose in using the technology. Our intent is to identify challenges for future research and to move towards a set of guidelines for inclusive CUI design.

This short essay on digital assistants and accent regionalisation was written for the UQ Art Museum's online exhibition 'Conflict in My Outlook.' The exhibition and provocation essays are available to access online: https://www.conflictinmyoutlook.online/provocations

Bringing positive experiences to users is one of the key goals when designing conversational agents (CAs). Yet we still lack an understanding of users' underlying needs to achieve positive experiences and how to support them in design. This research first applies Self-Determination Theory in an interview study to explore how users' needs of competence, autonomy and relatedness could be supported or undermined in CA experiences. Ten guidelines are then derived from the interview findings. The key findings demonstrate that: competence is affected by users' knowledge of the CA capabilities and effectiveness of the conversation; autonomy is influenced by flexibility of the conversation, personalisation of the experiences, and control over user data; regarding relatedness, users still have concerns over integrating social features into CAs. The guidelines recommend how to inform users about the system capabilities, design effective and socially appropriate conversations, and support increased system intelligence, customisation, and data transparency. CCS CONCEPTS • Human-centered computing~Human computer interaction (HCI)

This paper reports on a preliminary comparative study of Alexa, Siri, and Google Assistant voice assistants (VA) that explores the origins of answers provided on each platform in an attempt to determine the extent that these origins influence responses. Questions were selected from Text Recognition (TREC) 2017 Live Question Answering (QA) Track Data, a collection of pre-assessed questions that are part of the National Institute of Standards and Technology (NIST) TREC QA track. Responses were collected as voice memos and screen captures, then analyzed to determine the origins of each answer or set of answers provided. Results indicate that the origins of answers are different search engines, and that algorithm-centered processes in each voice assistant result in vast differences in answers to questions. Because the online search results provided as answers by voice assistants are influenced by content and structured data, technical communicators and UX practitioners can help ensure that voice assistants are able to provide accurate, complete, and ethical responses to users’ questions.

Computer voice is experiencing a renaissance through the growing popularity of voice-based interfaces, agents, and environments. Yet, how to measure the user experience (UX) of voice-based systems remains an open and urgent question, especially given that their form factors and interaction styles tend to be non-visual, intangible, and often considered disembodied or “body-less.” As a first step, we surveyed the ACM and IEEE literatures to determine which quantitative measures and measurements have been deemed important for voice UX. Our findings show that there is little consensus, even with similar situations and systems, as well as an overreliance on lab work and unvalidated scales. In response, we offer two high-level descriptive frameworks for guiding future research, developing standardized instruments, and informing ongoing review work. Our work highlights the current strengths and weaknesses of voice UX research and charts a path towards measuring voice UX in a more comprehensive way.

This paper points to the longstanding and significant role that the sound interface plays in shaping the ways we attend to media by considering its phatic function. Employing a reverse engineering approach, the paper consists of an analysis of historical transformations in the regimen of attention produced by sound media to date, followed by discourse analyses of scientific and industry communities of digital sound interface design. Introducing the term "phatic alignment" to describe how media and humans are arranged in space and adjusted to communicate with one another, this paper points to the increasing hold of the digital sound interface over the user's attention and identifies the premises affording this trend. The paper argues that in abstracting the human ear as an automated "phatic threshold" that regulates the user's attention, digital sound interface design situates the user in constant attentiveness to media and sets the stage for next-generation communication technologies.

We introduce J.A.N.E. – a proof-of-concept voice-based travel assistant. It is an attempt to show how to handle increasingly complex user queries against the web while balancing between an intuitive user interface and a proper knowledge quality level. As the use case, the search for travel directions based on user preferences regarding cuisine, art and activities was chosen. The system integrates knowledge from several sources, including Wikidata, LinkedGeoData and OpenWeatherMap. The voice interaction with the user is built on the Amazon Alexa platform. A system architecture description is supplemented by the discussion about the motivation and requirements for such complex assistants.

The present research investigates the use of voice as a means of amplifying digital media communication and experience. Coming from the question "How could a designer benefit from the use of voice in digital media projects?", aspects related to the most prominent voice interaction have been investigated aiming at the establishment of parameters that can help designers project voice interfaces in a suitable and efficient way. The theoretical framework of this study on voice, which is relevant to the field of Design of conversation, embodies researches on the process of complex patterns of human conversation and speech, aspects of interactions human/machine and voice user interface design, amongst others. In addition, I shed light on the impact of technological innovations in this field and the intelligent technologies which are amplified through artificial intelligence and biosensors as well as the current soundscape of the "talking objects". As a result, a set of guiding principles to voice interface design for hearables is proposed. Finally, applications and predictions for possible outcomes in regards to the use of such principles and methods are suggested.

This paper presents a Speculative Fiction method for uncovering socio-technical narratives embedded into technology-specifically, within the digital home assistant, Amazon Alexa. The methodology draws from a model intended for the analysis of science fiction and examines tensions between estrangement and cognition. Here, we adapt a literary analysis to create a method for analyzing an emerging technology. As a test case, we apply the method toward an analysis of Amazon's Alexa to explore the role of assurance in its design. For this analysis, Alexa is considered as an artifact of its own techno-social narrative. The paper shows that Speculative Fiction frameworks can inform a deep reading of Alexa's narratives, and generate insights for future research into human-machine interactions.

Technology is developing every day in the world and as a result, many software and hardware innovations are emerging that facilitate human life. One of the most obvious of these innovations is the voice assistants working with voice commands. These assistants are a new and fast alternative to touch-based interaction technology that has been in use for years. Voice assistants, which are watched in many fields, are almost not encountered in the field of architecture. Therefore, the subject has been found worthy of study as an original and new research area for architecture. The aim of this study is to investigate the potential use of voice assistants in architecture and to make suggestions in order to become widespread in the field of architecture. In this context, the general usage areas of voice commands are first introduced and the experiences of the user are examined. Afterwards, architecture is divided into five sub branches and voice command technology is examined in terms of these sub branches. Finally, various suggestions are presented to draw attention to the potential contributions that voice assistants can make to the field of architecture. It is anticipated that this study will draw attention to the voice assistant technology and bring a new perspective to the world of architecture that will contribute to all processes of architecture.

Dünyada teknoloji her gün gelişmekte ve bunun sonucu olarak insan yaşamını kolaylaştırıcı birçok yazılımsal ve donanımsal yenilikler ortaya çıkmaktadır. Bu yeniliklerin en belirginlerinden biri sesli komutlar ile çalışan sesli asistanlardır. Bu asistanlar yıllardır kullanılmakta olan dokunmaya dayalı etkileşim teknolojisine yeni ve hızlı bir alternatif oluşturmaktadır. Bu çalışmanın amacı, pek çok alanda kullanılan sesli asistanların mimarlığın çeşitli dallarında mevcut olan az sayıdaki kullanımlarına değinmek ve bu kullanımın genişletilebilmesi adına öneriler ortaya koymaktır. Bu bağlamda önce sesli komutların genel kullanım alanları ve kullanıcılara yaşatılan deneyimler ortaya koyulmuştur. Daha sonra sesli komutların mimarlığın farklı alanlarındaki kullanımını daha iyi incelemek adına mimarlık, beş alt dalı olacak şekilde şematize edilmiş ve sesli komutların bu alt dallardaki kullanımlarındaki örneklere değinilmiştir. Son olarak sesli asistanların mimarlık alanındaki kullanımlarının daha çeşitli ve verimli hale getirilebilmesi adına birtakım öneriler sunulmuştur. Bu çalışmanın, sesli asistan teknolojisine dikkat çekerek mimarlığın tüm süreçlerine katkı sağlayacak yeni bir bakış açısını mimarlık dünyasına kazandıracağı öngörülmektedir.

Smart speakers such as the Google Home and the Amazon Echo have recently become popular internet-connected consumer devices. They have the seemingly magical capacity to hear user 'invocations' and provide intelligent responses in natural language. They answer users' calls with facts, calculations, recipes, news, music, scripted chit chat and even turn the lights on. This paper analyses and categorises popular smart speaker commands and extends Austin (1962) and Searle's (1976) analysis and classifications of speech acts to develop a typology of invocationary acts. Analysing this array of acts helps to evaluate the significance and implications of this new media form.