Papers by Nikos Moustakas
Journal of the Audio Engineering Society, 2021
In this work we present an adaptive audio mixing technique to be implemented in the design of Aug... more In this work we present an adaptive audio mixing technique to be implemented in the design of Augmented Reality Audio (ARA) systems. The content of such systems is delivered entirely through the acoustic channel: the real acoustic environment is mixed with a virtual soundscape and returns to the listener as "pseudoacoustic" environment. We argue that the proposed adaptive mixing technique enhances user immersion in the augmented space in terms of the localization of sound objects. The need to optimise our ARA mixing engine emerged from our previous research, and more specifically from the analysis of the experimental results regarding the development of the Augmented Reality Audio Game (ARAG) "Audio Legends" that was tested on the field. The purpose of our new design was to aid sound localization, which is a crucial and demanding factor for delivering an immersive acoustic experience. We describe in depth the adaptive mixing along with the experimental test-bed. ...
Multimodal Technologies and Interaction, 2019
Augmented Reality Audio Games (ARAG) enrich the physical world with virtual sounds to express the... more Augmented Reality Audio Games (ARAG) enrich the physical world with virtual sounds to express their content and mechanics. Existing ARAG implementations have focused on exploring the surroundings and navigating to virtual sound sources as the main mode of interaction. This paper suggests that gestural activity with a handheld device can realize complex modes of sonic interaction in the augmented environment, resulting in an enhanced immersive game experience. The ARAG “Audio Legends” was designed and tested to evaluate the usability and immersion of a system featuring an exploration phase based on auditory navigation, as well as an action phase, in which players aim at virtual sonic targets and wave the device to hit them or hold the device to block them. The results of the experiment provide evidence that players are easily accustomed to auditory navigation and that gestural sonic interaction is perceived as difficult, yet this does not affect negatively the system’s usability and ...
Applied Sciences, 2021
At the core of augmented reality audio (ARA) technology lies the ARA mix, a process responsible f... more At the core of augmented reality audio (ARA) technology lies the ARA mix, a process responsible for the assignment of a virtual environment to a real one. Legacy ARA mix models have focused on the natural reproduction of the real environment, whereas the virtual environment is simply mixed through fixed gain methods. This study presents a novel approach of a dynamic ARA mix that facilitates a smooth adaptation of the virtual environment to the real one, as well as dynamic control of the virtual audio engine, by taking into account the inherent characteristics of both ARA technology and binaural auditory perception. A prototype feature extraction technique of auditory perception characteristics through a real-time binaural loudness prediction method was used to upgrade the legacy ARA mix model into a dynamic model, which was evaluated through benchmarks and subjective tests and showed encouraging results in terms of functionality and acceptance.
Journal of The Audio Engineering Society, 2011
Audio-games represent a game alternative based on audible feedback rather than on visual. They ma... more Audio-games represent a game alternative based on audible feedback rather than on visual. They may benefit from parametric sound synthesis and advanced audio technologies (ie augmented reality audio), in order to effectively realize complex scenarios. In this work, a multiplayer ...
Journal of The Audio Engineering Society, 2019
Audio-only games represent an alternative type of gaming genres that continuously evolves followi... more Audio-only games represent an alternative type of gaming genres that continuously evolves following the technolog- ical trends that boost the video-games market. Since augmented reality is now a widespread approach for producing new kind of immersive applications, including games, it is expected that audio-only games will be influenced by this approach. This work represents the beginning of an attempt to investigate the process of delivering augmented audio-only games, focusing on specific technical factors that can improve the user interaction and the overall game-play experience. In particularly, it focuses on a new augmented reality audio mixing process that is optimized for variable acoustic environments, allowing the development of new attractive titles of audio-only games.
The Electronic Journal of e-Learning, 2019
Citizen Science aims to engage people in research activities on important issues related to their... more Citizen Science aims to engage people in research activities on important issues related to their well-being. Smart Cities aim to provide them with services that improve the quality of their life. Both concepts have seen significant growth in the last years and can be further enhanced by combining their purposes with Internet of Things technologies that allow for dynamic and large-scale communication and interaction. However, exciting and retaining the interest of participants is a key factor for such initiatives. In this paper we suggest that engagement in Citizen Science projects applied on Smart Cities infrastructure can be enhanced through contextual and structural game elements realized through augmented audio interactive mechanisms. Our inter-disciplinary framework is described through the paradigm of a collaborative bird call recognition game, in which users collect and submit audio data that are then classified and used for augmenting physical space. We discuss the Playful Learning, Internet of Audio Things, and Bird Monitoring principles that shaped the design of our paradigm and analyze the design issues of its potential technical implementation.
Audio Engineering Society …, Jan 1, 2009
Augmented reality audio represents a new trend in digital technology that enriches the real acous... more Augmented reality audio represents a new trend in digital technology that enriches the real acoustic environment with synthesized sound produced by virtual sound objects. On the other hand, an audio-game is based only on audible feedback rather than on visual. In this work, an audio-game prototype is presented which takes advantage of the characteristics of augmented reality audio for realizing more complex audio-game scenarios. The prototype was realized as an audiovisual interactive installation, allowing the further involvement of the physical game space as the secondary component for constructing the game ambient environment. A sequence of tests has shown that the proposed prototype can efficiently support complex game scenarios provided that the necessary advanced interaction paths are available.
Proceedings of the AES …
Audio-games represent a game alternative based on audible feedback rather than on visual. They ma... more Audio-games represent a game alternative based on audible feedback rather than on visual. They may benefit from parametric sound synthesis and advanced audio technologies (i.e. augmented reality audio), in order to effectively realize complex scenarios. In this work, a multiplayer game prototype is introduced which employs the concept of controlled mixed reality in order to augment the sound environment of each player. The prototype is realized as multiple user audiovisual installations, which are interconnected in order to communicate the status of the selected control parameters in real-time. The prototype reveals significant relevance to the-well known on-line multiplayer games, with its novelty originating from the fact that user interaction is realized in augmented reality audio environments.
ionio.gr
“dots” is an interactive sound installation that takes into account the spatial position of an ar... more “dots” is an interactive sound installation that takes into account the spatial position of an arbitrary number of participants in order to algorithmically synthesize an audio stream in real-time. The installation core is a software application developed during this work, which employs advanced video and audio processing techniques in order to detect the exact participants’ positions and to weight-mix short audio granules. Audio mixing is performed using a virtual spatial gridding of the installation space in two dimensions. The synthesized audio stream reproduction is combined with a number of appropriately designed visual effects, which aim to enhance the participants’ comprehension and render the “dots” installation a high-quality interactive audiovisual platform.
International Journal Papers by Nikos Moustakas
MDPI - APPLIED SCIENCE, 2021
At the core of augmented reality audio (ARA) technology lies the ARA mix, a process responsible f... more At the core of augmented reality audio (ARA) technology lies the ARA mix, a process responsible for the assignment of a virtual environment to a real one. Legacy ARA mix models have focused on the natural reproduction of the real environment, whereas the virtual environment is simply mixed through fixed gain methods. This study presents a novel approach of a dynamic ARA mix that facilitates a smooth adaptation of the virtual environment to the real one, as well as dynamic control of the virtual audio engine, by taking into account the inherent characteristics of both ARA technology and binaural auditory perception. A prototype feature extraction technique of auditory perception characteristics through a real-time binaural loudness prediction method was used to upgrade the legacy ARA mix model into a dynamic model, which was evaluated through benchmarks and subjective tests and showed encouraging results in terms of functionality and acceptance.
Multimodal Technologies Interact, 2019
Augmented Reality Audio Games (ARAG) enrich the physical world with virtual sounds to express the... more Augmented Reality Audio Games (ARAG) enrich the physical world with virtual sounds to express their content and mechanics. Existing ARAG implementations have focused on exploring the surroundings and navigating to virtual sound sources as the main mode of interaction. This paper suggests that gestural activity with a handheld device can realize complex modes of sonic interaction in the augmented environment, resulting in an enhanced immersive game experience. The ARAG "Audio Legends" was designed and tested to evaluate the usability and immersion of a system featuring an exploration phase based on auditory navigation, as well as an action phase, in which players aim at virtual sonic targets and wave the device to hit them or hold the device to block them. The results of the experiment provide evidence that players are easily accustomed to auditory navigation and that gestural sonic interaction is perceived as difficult, yet this does not affect negatively the system's usability and players' immersion. Findings also include indications that elements, such as sound design, the synchronization of sound and gesture, the fidelity of audio augmentation, and environmental conditions, also affect significantly the game experience, whereas background factors, such as age, sex, and game or music experience, do not have any critical impact.
The Electronic Journal of e-Learning, 2019
Educational practices are constantly adjusting to technological advances, in order to improve the... more Educational practices are constantly adjusting to technological advances, in order to improve their effectiveness in delivering knowledge and preparing students for the challenges of modern digital society. Electronic games and augmented reality environments are two such media that can shape powerful modes of interactive and immersive experiences. Audio games in particular, i.e. electronic games that utilise data sonification and audio interaction techniques to express all narrative and gameplay content, have been proven to enhance players' skills, such as concentration, memory, fantasy, emotional response, perception, data management and cooperation. However, despite the promising research findings reported in literature, there has been no systematic approach in integrating systems based on audio interaction into formal education. This article suggests that audio games can be designed and implemented for the delivery of targeted curricula through an engaging learning experience, whereas fusing audio game mechanics into new interactive technologies, such as augmented reality environments, will further enhance the students' immersiveness in the learning process. Towards establishing the theoretical framework for the design of educational audio games, as well as educational augmented reality audio games, the authors review existing approaches and argue that the inherent features of both audio interaction and augmented reality systems agree with the official educational goals formulated by the Greek Ministry of Education and the International Baccalaureate Institution. Then, they discuss four original projects in terms of sound and mechanics design to demonstrate that audio games can address a wide range of subjects, adopt cross-curricular strategies, facilitate complex modes of interaction, and realise creative, collaborative and inclusive learning activities.
Panhellenic Conference Papers by Nikos Moustakas
1ο Πανελλήνιο Συνέδριο «Το εκπαιδευτικό παιχνίδι στην τυπική και μη τυπική μάθηση», 2021
Το πεδίο της Επαυξημένης Πραγματικότητας αφορά τεχνολογίες, οι οποίες εμπλουτίζουν τον φυσικό κόσ... more Το πεδίο της Επαυξημένης Πραγματικότητας αφορά τεχνολογίες, οι οποίες εμπλουτίζουν τον φυσικό κόσμο με εικονικές πληροφορίες και παρέχουν στους χρήστες δυνατότητες αλληλεπίδρασης με το περιβάλλον τους που δεν θα ήταν εφικτές μέσω της άμεσης αισθητηριακής αντίληψης. Εξαιτίας των εγγενών αυτών χαρακτηριστικών, η Επαυξημένη Πραγματικότητα χρησιμοποιείται όλο και περισσότερο σε τομείς, όπως η εκπαίδευση, η μουσειακή και γενικότερα πολιτιστική περιήγηση, η προσβασιμότητα, η θεραπεία, και η ψυχαγωγία, γεγονός που τονίζει την ανάγκη εξοικείωσης μαθητών και σπουδαστών με το νέο αυτό μέσο, ώστε να αντεπεξέλθουν στις σύγχρονες και μελλοντικές προκλήσεις σε κοινωνικό και εργασιακό επίπεδο. Η αξιοποίηση ηχητικών ερεθισμάτων ως στοιχείων επαύξησης και η ως εκ τούτου πραγμάτωση της επαυξημένης αλληλεπίδρασης μέσω του ακουστικού καναλιού συνιστούν το πεδίο της Ακουστικώς Επαυξημένης Πραγματικότητας. Το Εργαστήριο Επεξεργασίας Οπτικοακουστικών Σημάτων (ΕΠ.ΟΑ.ΣΗ) του Τμήματος Τεχνών ‘Ήχου και Εικόνας του Ιονίου Πανεπιστημίου δραστηριοποιείται τα τελευταία χρόνια στον σχεδιασμό και την ανάπτυξη ακουστικώς επαυξημένων ηχητικών παιχνιδιών με σκοπό αφενός τη διερεύνηση των τρόπων αλληλεπίδρασης και εμβύθισης σε ένα ακουστικώς επαυξημένο παιγνιακό περιβάλλον και αφετέρου την εφαρμογή ακουστικώς επαυξημένων τεχνικών για την επιμόρφωση και ευαισθητοποίηση των χρηστών σχετικά με γνωστικά αντικείμενα που άπτονται της εκπαίδευσης και του πολιτισμού. Η παρούσα εισήγηση επιχειρεί να αναδείξει τις δυνατότητες αξιοποίησης ακουστικώς επαυξημένων ηχητικών παιχνιδιών στην εκπαίδευση μέσα από το πρίσμα των πεδίων της Ακουστικής Επαύξησης και της Πολιτιστικής Περιήγησης. Έπειτα, παρουσιάζει το ακουστικώς επαυξημένο ηχητικό παιχνίδι «Audio Legends», περιγράφοντας τον επιμορφωτικό σχετικά με την πολιτιστική κληρονομιά της Κέρκυρας χαρακτήρα του παιχνιδιού, τη σύνθεση του σεναρίου και των εκπορευόμενων παιγνιακών μηχανικών, τις τεχνικές σχεδιαστικές προδιαγραφές, την πειραματική δοκιμή του και τα συμπεράσματα που αντλήθηκαν σχετικά με την αποδοχή και εμβυθιστικότητα του νέου αυτού μέσου.
Πρακτικά του 9ου Πανελλήνιου Συνεδρίου του Ελληνικού Ινστιτούτου Ακουστικής (ΕΛΙΝΑ) «Ακουστική 2018», 2018
Στην παρούσα εισήγηση επιχειρείται μια πρώτη προσέγγιση της σύγκλισης των πεδίων των ηχητικών παι... more Στην παρούσα εισήγηση επιχειρείται μια πρώτη προσέγγιση της σύγκλισης των πεδίων των ηχητικών παιχνιδιών και της επαυξημένης πραγματικότητας με στόχο τη δημιουργία ηχητικά επαυξημένων παιγνιακών συστημάτων που θα αξιοποιούν τα εγγενή χαρακτηριστικά των δύο πεδίων βελτιστοποιώντας τις συνθήκες διάδρασης και εμβύθισης της παιγνιακής εμπειρίας. Αρχικά περιγράφονται οι συνηθέστερες τεχνικές ηχητικής αναπαράστασης, με έμφαση σε αυτές που αφορούν στη χωρική τοποθέτηση και κίνηση των ηχητικών συμβάντων, καθώς και τα βασικά δομικά στοιχεία της ηχητικής παιγνιακής μηχανικής. Έπειτα, αποτυπώνονται οι σύγχρονες κατευθύνσεις εφαρμογής ηχητικά επαυξημένων συστηματων, και τέλος προτείνεται η ανάπτυξη ενός προηγμένου μείκτη ηχητικής επαυξημένης πραγματικότητας με βάση την αμφιωτική ακουστότητα.
Uploads
Papers by Nikos Moustakas
International Journal Papers by Nikos Moustakas
Panhellenic Conference Papers by Nikos Moustakas