Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services
Figure 1: Illustrative examples using autostereograms. a) Password input. b) Wearable e-mail noti... more Figure 1: Illustrative examples using autostereograms. a) Password input. b) Wearable e-mail notification. c) Private space in collaborative conditions. d) 3D video game. e) Bar gamified special menu. Black elements represent the hidden 3D scene content.
This study demonstrates how playing a well-designed multitasking motion video game in a virtual r... more This study demonstrates how playing a well-designed multitasking motion video game in a virtual reality (VR) environment can positively impact the cognitive and physical health of older players. We developed a video game that combines cognitive and physical training in a VR environment. The impact of playing the game was measured through a four-week longitudinal experiment. Twenty healthy older adults were randomly assigned to either an intervention group (i.e., game training) or a control group (i.e., no contact). Participants played three 45-min sessions per week completing cognitive tests for attention, working memory, reasoning and a test for physical balance before and after the intervention. Results showed that compared to the control group, the game group showed significant improvements in working memory and a potential for enhancing reasoning and balance ability. Furthermore, while the older adults enjoyed playing the video game, ability enhancements were associated with the...
We proposed a new model (SH-Model) to observe the human effects in pointing tasks. The SH-Model i... more We proposed a new model (SH-Model) to observe the human effects in pointing tasks. The SH-Model is approved as a reliable evaluation model. We also show how the SH-Model can be applied as an evaluation tool to observe the human effects. We evaluated four input devices, a mouse, a pen with a big tablet, a trackball and a pen with a small tablet. The comprehensive analysis including the SH-Model, ANOVA analysis and questionnaire can offer a clear comparison of the four input devices. The coefficients of the human factor in the SH-Model show the features of different human performance effects when using different devices. According to our analysis, the mouse is the best for the pointing task designed for our experiment, whereas the trackball is the worst. This study not only verifies the application of the SH-Model as a valid evaluation tool for the various devices, but also helps us to observe the human effects separately from the system effects.
People are accustomed to capture important events and ideas by sketching notes on papers. Pen and... more People are accustomed to capture important events and ideas by sketching notes on papers. Pen and paper offer people great freedom and naturalness to perform these activities. During note sketching, people generally don’t pay much attention to the organization of notes, instead they use implicit spatial relationships and concise informal organizing symbols to structuralize notes. We conducted a study to discover how people sketch and organize their notes. The structuralized notes are easy to be maintained, manipulated and reused. Based on our observations, we designed algorithms to structuralize freeform notes into a consistent hierarchical structure. Both implicit spatial parsing and gesture-based structuralizing are enabled. With the supports of these algorithms, we built a system SketchPoint notebook as a tool that allows a user to sketch various notes in a freeform manner. It conducts lazy note structuralizing implicitly. The structuralized notes of SketchPoint enable structure-...
IEICE Transactions on Information and Systems, 2001
SUMMARY Although research into multimodal interfaces has been around for a long time, we believe ... more SUMMARY Although research into multimodal interfaces has been around for a long time, we believe that some basic issues have not been studied yet, e.g. the choice of modalities and their combinations is usually made without any quantitative evaluation. This study seeks to identify the best combinations of modalities through usability testing. How do users choose different interaction modes when they work on a particular application? Two experimental evaluations were conducted to compare interaction modes on a CAD system and a map system respectively. For the CAD system, the results show that, in terms of total manipulation time (drawing and modification time) and subjective preferences, the “pen + speech + mouse” combination was the best of the seven interaction modes tested. On the map system, the results show that the “pen + speech” combination mode is the best of fourteen interaction modes tested. The experiments also provide information on how users adapt to each interaction mod...
This position paper is in two parts. It addresses some areas involved in human-computer interacti... more This position paper is in two parts. It addresses some areas involved in human-computer interaction in Japan and China. Part I provides an overview of human-computer interaction activities in some major professional societies and reviews their current status in Japan. It uses a concrete example from a special-interest group in Japan. It also reviews the current state of human-computer interaction publication and education in Japanese universities. Part II gives information on education and research activities related to human-computer interaction in China.
Typing Japanese into computers consists of typing Roman alphabet, displaying the kana character, ... more Typing Japanese into computers consists of typing Roman alphabet, displaying the kana character, converting kana to kanji, and selecting the intended kanji character from a list of homophonic candidates. This paper presents a study of four candidate display styles, three commonly used in commercial products ("vertical," "horizontal," and "compact-horizontal") and one novel ("matrix"), together with various manual selection methods (mouse, numeric key, spacebar, cursor key, numeric keypad and unrestricted input method). The results show that (1) when typing a single kanji character, about 70% of total operation time is spent on choice selection; and (2) the "compact-horizontal" and "matrix" display styles are superior to the other display styles.
CCF Transactions on Pervasive Computing and Interaction
The paper presents a qualitative study to explore the use of fitness trackers and their social fu... more The paper presents a qualitative study to explore the use of fitness trackers and their social functions in intergenerational settings. The study covered three phases of semi-structured interviews with older and younger adults during individual and intergenerational use of the fitness trackers. The study revealed comparability as common fitness practice for older adults. The findings show that intergenerational fitness tracking practices can increase in-person meetings and daily discourses and thus enhance family social bonds. An unexpected benefit of this practice is its ability to help older adults overcome technology barriers related to the use of fitness trackers. Overall speaking, families whose intergenerational members already enjoy a strong relationship are likely to gain the most from such practices. Many challenges remain especially concerning the motivation and involvement of younger partners and the user experience design aspect of such digital programs. For this purpose...
Proceedings of the 2018 ACM International Conference on Interactive Surfaces and Spaces
Although the HCI community inevitably contributes to engagement via beauty according to the atten... more Although the HCI community inevitably contributes to engagement via beauty according to the attention paid to known and yet to be discovered principles of aesthetics for digital interface design, it is lacking an epistemological corpus which should include the notion, human factors and the quantification of aesthetic aspects. The aim of the proposed workshop is to discuss these issues in order to strengthen aesthetic studies specifically for HCI and related fields. We want to create a forum for discussing, drafting and promoting the foundations for disciplined aesthetics design within the HCI community. We thus welcome contributions such as theories, methodologies, evaluation methods, and potential applications regarding effective aesthetics for HCI and related fields. Concretely, we aim to (i) map the present state-of-art of aesthetic research in HCI, (ii) build a multidisciplinary community of experts, and (iii) raise the profile of this aesthetics research area within HCI community.
The 34th Annual ACM Symposium on User Interface Software and Technology
Steering law reveals a linear relationship between the movement time (MT) and the index of diffic... more Steering law reveals a linear relationship between the movement time (MT) and the index of difficulty (ID) in trajectory-based steering tasks. However, it does not relate the variance or distribution of MT to I D. In this paper, we propose and evaluate models that predict the variance and distribution of MT based on ID for steering tasks. We first propose a quadratic variance model which reveals that the variance of MT is quadratically related to ID with the linear coefficient being 0. Empirical evaluation on a new and a previously collected dataset show that the quadratic variance model accounts for between 78% and 97% of variance of observed MT variances; it outperforms other model candidates such as linear and constant models; adding the linear coefficient leads to no improvement on the model fitness. The variance model enables predicting the distribution of MT given ID: we can use the variance model to predict the variance (or scale) parameter and Steering law to predict the mean (or location) parameter of a distribution. We have evaluated six types of distributions for predicting the distribution of MT. Our investigation also shows that positively skewed distribution such as Gamma, Lognormal, Exponentially Modified Gaussian (ExGaussian), and Extreme value distributions outperformed the symmetric distribution such as Gaussian and truncated Gaussian distribution in predicting the MT distribution, and Gamma distribution performed slightly better than other positively skewed distributions. Overall, our research advances the MT prediction of steering tasks from a point estimate to variance and distribution estimates, which provides a more complete understanding of steering behavior and quantifies the uncertainty of MT prediction.
Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems
Artificial Intelligence (AI) can refer to the machine learning algorithms and the automation appl... more Artificial Intelligence (AI) can refer to the machine learning algorithms and the automation applications built on top of these algorithms. Human-computer interaction (HCI) researchers have studied these AI applications and suggested various Human-Centered AI (HCAI) principles for an explainable, safe, reliable, and trustworthy interaction experience. While some designers believe that computers should be supertools and active appliances, others believe that these latest AI systems can be collaborators. With today’s AI algorithm breakthroughs, in this panel we ask whether the supertool or the collaboration metaphors best support work and play? How can we design AI systems to work best with people or for people? What does it take to get there? This panel will bring together panelists with diverse backgrounds to engage the audience through the discussion of their shared or diverging visions on the future of human-AI interaction design.
Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems
We are experiencing two revolutions: ubiquitous digital technology and worldwide population aging... more We are experiencing two revolutions: ubiquitous digital technology and worldwide population aging: digital devices are becoming ubiquitous; and older people are becoming the largest demographic group. However, despite the recent increase in related CHI publication, older adults continue to be underrepresented in HCI research as well as commercially, further widening the digital divide they face and hampering their social participation. Therefore, the overarching aim of this workshop is to increase the momentum for such research within CHI and related fields such as gerontechnology. For this, we plan to create a space for discussing and sharing principles and strategies to design interactions and evaluate user interfaces for the aging population. We thus welcome contributions to empirical studies, theories, design and evaluation of user interfaces for older adults. Concretely, we aim to: map the state-of-art of senior-centred interaction research, build a multidisciplinary community of experts, and raise the profile of this research within SIGCHI.
Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
Relatively little is known about eye and finger movement in typing with mobile devices. Most prio... more Relatively little is known about eye and finger movement in typing with mobile devices. Most prior studies of mobile typing rely on log data, while data on finger and eye movements in typing come from studies with physical keyboards. This paper presents new findings from a transcription task with mobile touchscreen devices. Movement strategies were found to emerge in response to sharing of visual attention: attention is needed for guiding finger movements and detecting typing errors. In contrast to typing on physical keyboards, visual attention is kept mostly on the virtual keyboard, and glances at the text display are associated with performance. When typing with two fingers, although users make more errors, they manage to detect and correct them more quickly. This explains part of the known superiority of two-thumb typing over one-finger typing. We release the extensive dataset on everyday typing on smartphones.
Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems
We are concurrently witnessing two significant shifts: digital devices are becoming ubiquitous, a... more We are concurrently witnessing two significant shifts: digital devices are becoming ubiquitous, and older people are becoming a very large demographic group. However, despite the recent increase in related CHI publications, older adults continue to be underrepresented in HCI research as well as commercially. Therefore, the overarching aim of this workshop is to increase the momentum for such research within CHI and related fields such as gerontechnology. For this, we plan to create a space for discussing and sharing principles and strategies to design interactions and evaluate user interfaces (UI) for the ageing population. We thus welcome contributions of empirical studies, theories, design and evaluation of UIs for older adults. Building on the success of last two year's workshops, we aim to grow the community of CHI researchers across borders interested in this topic by fostering a space to exchange results, methods, approaches, and ideas from research on interactive applications in support of older adults that are reflective of international diversity that is representative of CHI.
Beside decreasing the abilities, individual difference prevails among older adults, as some indiv... more Beside decreasing the abilities, individual difference prevails among older adults, as some individuals are completely healthy at the age 90 while some are not at even 60. In context of touchscreen interface design, it is critical to understand the design space as a function of abilities. In this work, we articulate a better understanding of the effects of ageing and examine their HCI task performing capabilities in terms of interfaces design. We design a text-entry interface in particular, as ageing users often achieve slow text entry performance, thus proving to be a bottleneck for technology use. Our developed text entry interface is tuned based on the parameter values for Elderly having finger tremor. We present initial study results showing the improvement of the accuracy of touch typing in smartphone over the baseline Qwerty keyboard. By carefully considering other sensorimotor abilities, we believe that the current smartphone text-entry interface designs will become more usable to the ageing populations.
Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services
Figure 1: Illustrative examples using autostereograms. a) Password input. b) Wearable e-mail noti... more Figure 1: Illustrative examples using autostereograms. a) Password input. b) Wearable e-mail notification. c) Private space in collaborative conditions. d) 3D video game. e) Bar gamified special menu. Black elements represent the hidden 3D scene content.
This study demonstrates how playing a well-designed multitasking motion video game in a virtual r... more This study demonstrates how playing a well-designed multitasking motion video game in a virtual reality (VR) environment can positively impact the cognitive and physical health of older players. We developed a video game that combines cognitive and physical training in a VR environment. The impact of playing the game was measured through a four-week longitudinal experiment. Twenty healthy older adults were randomly assigned to either an intervention group (i.e., game training) or a control group (i.e., no contact). Participants played three 45-min sessions per week completing cognitive tests for attention, working memory, reasoning and a test for physical balance before and after the intervention. Results showed that compared to the control group, the game group showed significant improvements in working memory and a potential for enhancing reasoning and balance ability. Furthermore, while the older adults enjoyed playing the video game, ability enhancements were associated with the...
We proposed a new model (SH-Model) to observe the human effects in pointing tasks. The SH-Model i... more We proposed a new model (SH-Model) to observe the human effects in pointing tasks. The SH-Model is approved as a reliable evaluation model. We also show how the SH-Model can be applied as an evaluation tool to observe the human effects. We evaluated four input devices, a mouse, a pen with a big tablet, a trackball and a pen with a small tablet. The comprehensive analysis including the SH-Model, ANOVA analysis and questionnaire can offer a clear comparison of the four input devices. The coefficients of the human factor in the SH-Model show the features of different human performance effects when using different devices. According to our analysis, the mouse is the best for the pointing task designed for our experiment, whereas the trackball is the worst. This study not only verifies the application of the SH-Model as a valid evaluation tool for the various devices, but also helps us to observe the human effects separately from the system effects.
People are accustomed to capture important events and ideas by sketching notes on papers. Pen and... more People are accustomed to capture important events and ideas by sketching notes on papers. Pen and paper offer people great freedom and naturalness to perform these activities. During note sketching, people generally don’t pay much attention to the organization of notes, instead they use implicit spatial relationships and concise informal organizing symbols to structuralize notes. We conducted a study to discover how people sketch and organize their notes. The structuralized notes are easy to be maintained, manipulated and reused. Based on our observations, we designed algorithms to structuralize freeform notes into a consistent hierarchical structure. Both implicit spatial parsing and gesture-based structuralizing are enabled. With the supports of these algorithms, we built a system SketchPoint notebook as a tool that allows a user to sketch various notes in a freeform manner. It conducts lazy note structuralizing implicitly. The structuralized notes of SketchPoint enable structure-...
IEICE Transactions on Information and Systems, 2001
SUMMARY Although research into multimodal interfaces has been around for a long time, we believe ... more SUMMARY Although research into multimodal interfaces has been around for a long time, we believe that some basic issues have not been studied yet, e.g. the choice of modalities and their combinations is usually made without any quantitative evaluation. This study seeks to identify the best combinations of modalities through usability testing. How do users choose different interaction modes when they work on a particular application? Two experimental evaluations were conducted to compare interaction modes on a CAD system and a map system respectively. For the CAD system, the results show that, in terms of total manipulation time (drawing and modification time) and subjective preferences, the “pen + speech + mouse” combination was the best of the seven interaction modes tested. On the map system, the results show that the “pen + speech” combination mode is the best of fourteen interaction modes tested. The experiments also provide information on how users adapt to each interaction mod...
This position paper is in two parts. It addresses some areas involved in human-computer interacti... more This position paper is in two parts. It addresses some areas involved in human-computer interaction in Japan and China. Part I provides an overview of human-computer interaction activities in some major professional societies and reviews their current status in Japan. It uses a concrete example from a special-interest group in Japan. It also reviews the current state of human-computer interaction publication and education in Japanese universities. Part II gives information on education and research activities related to human-computer interaction in China.
Typing Japanese into computers consists of typing Roman alphabet, displaying the kana character, ... more Typing Japanese into computers consists of typing Roman alphabet, displaying the kana character, converting kana to kanji, and selecting the intended kanji character from a list of homophonic candidates. This paper presents a study of four candidate display styles, three commonly used in commercial products ("vertical," "horizontal," and "compact-horizontal") and one novel ("matrix"), together with various manual selection methods (mouse, numeric key, spacebar, cursor key, numeric keypad and unrestricted input method). The results show that (1) when typing a single kanji character, about 70% of total operation time is spent on choice selection; and (2) the "compact-horizontal" and "matrix" display styles are superior to the other display styles.
CCF Transactions on Pervasive Computing and Interaction
The paper presents a qualitative study to explore the use of fitness trackers and their social fu... more The paper presents a qualitative study to explore the use of fitness trackers and their social functions in intergenerational settings. The study covered three phases of semi-structured interviews with older and younger adults during individual and intergenerational use of the fitness trackers. The study revealed comparability as common fitness practice for older adults. The findings show that intergenerational fitness tracking practices can increase in-person meetings and daily discourses and thus enhance family social bonds. An unexpected benefit of this practice is its ability to help older adults overcome technology barriers related to the use of fitness trackers. Overall speaking, families whose intergenerational members already enjoy a strong relationship are likely to gain the most from such practices. Many challenges remain especially concerning the motivation and involvement of younger partners and the user experience design aspect of such digital programs. For this purpose...
Proceedings of the 2018 ACM International Conference on Interactive Surfaces and Spaces
Although the HCI community inevitably contributes to engagement via beauty according to the atten... more Although the HCI community inevitably contributes to engagement via beauty according to the attention paid to known and yet to be discovered principles of aesthetics for digital interface design, it is lacking an epistemological corpus which should include the notion, human factors and the quantification of aesthetic aspects. The aim of the proposed workshop is to discuss these issues in order to strengthen aesthetic studies specifically for HCI and related fields. We want to create a forum for discussing, drafting and promoting the foundations for disciplined aesthetics design within the HCI community. We thus welcome contributions such as theories, methodologies, evaluation methods, and potential applications regarding effective aesthetics for HCI and related fields. Concretely, we aim to (i) map the present state-of-art of aesthetic research in HCI, (ii) build a multidisciplinary community of experts, and (iii) raise the profile of this aesthetics research area within HCI community.
The 34th Annual ACM Symposium on User Interface Software and Technology
Steering law reveals a linear relationship between the movement time (MT) and the index of diffic... more Steering law reveals a linear relationship between the movement time (MT) and the index of difficulty (ID) in trajectory-based steering tasks. However, it does not relate the variance or distribution of MT to I D. In this paper, we propose and evaluate models that predict the variance and distribution of MT based on ID for steering tasks. We first propose a quadratic variance model which reveals that the variance of MT is quadratically related to ID with the linear coefficient being 0. Empirical evaluation on a new and a previously collected dataset show that the quadratic variance model accounts for between 78% and 97% of variance of observed MT variances; it outperforms other model candidates such as linear and constant models; adding the linear coefficient leads to no improvement on the model fitness. The variance model enables predicting the distribution of MT given ID: we can use the variance model to predict the variance (or scale) parameter and Steering law to predict the mean (or location) parameter of a distribution. We have evaluated six types of distributions for predicting the distribution of MT. Our investigation also shows that positively skewed distribution such as Gamma, Lognormal, Exponentially Modified Gaussian (ExGaussian), and Extreme value distributions outperformed the symmetric distribution such as Gaussian and truncated Gaussian distribution in predicting the MT distribution, and Gamma distribution performed slightly better than other positively skewed distributions. Overall, our research advances the MT prediction of steering tasks from a point estimate to variance and distribution estimates, which provides a more complete understanding of steering behavior and quantifies the uncertainty of MT prediction.
Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems
Artificial Intelligence (AI) can refer to the machine learning algorithms and the automation appl... more Artificial Intelligence (AI) can refer to the machine learning algorithms and the automation applications built on top of these algorithms. Human-computer interaction (HCI) researchers have studied these AI applications and suggested various Human-Centered AI (HCAI) principles for an explainable, safe, reliable, and trustworthy interaction experience. While some designers believe that computers should be supertools and active appliances, others believe that these latest AI systems can be collaborators. With today’s AI algorithm breakthroughs, in this panel we ask whether the supertool or the collaboration metaphors best support work and play? How can we design AI systems to work best with people or for people? What does it take to get there? This panel will bring together panelists with diverse backgrounds to engage the audience through the discussion of their shared or diverging visions on the future of human-AI interaction design.
Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems
We are experiencing two revolutions: ubiquitous digital technology and worldwide population aging... more We are experiencing two revolutions: ubiquitous digital technology and worldwide population aging: digital devices are becoming ubiquitous; and older people are becoming the largest demographic group. However, despite the recent increase in related CHI publication, older adults continue to be underrepresented in HCI research as well as commercially, further widening the digital divide they face and hampering their social participation. Therefore, the overarching aim of this workshop is to increase the momentum for such research within CHI and related fields such as gerontechnology. For this, we plan to create a space for discussing and sharing principles and strategies to design interactions and evaluate user interfaces for the aging population. We thus welcome contributions to empirical studies, theories, design and evaluation of user interfaces for older adults. Concretely, we aim to: map the state-of-art of senior-centred interaction research, build a multidisciplinary community of experts, and raise the profile of this research within SIGCHI.
Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
Relatively little is known about eye and finger movement in typing with mobile devices. Most prio... more Relatively little is known about eye and finger movement in typing with mobile devices. Most prior studies of mobile typing rely on log data, while data on finger and eye movements in typing come from studies with physical keyboards. This paper presents new findings from a transcription task with mobile touchscreen devices. Movement strategies were found to emerge in response to sharing of visual attention: attention is needed for guiding finger movements and detecting typing errors. In contrast to typing on physical keyboards, visual attention is kept mostly on the virtual keyboard, and glances at the text display are associated with performance. When typing with two fingers, although users make more errors, they manage to detect and correct them more quickly. This explains part of the known superiority of two-thumb typing over one-finger typing. We release the extensive dataset on everyday typing on smartphones.
Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems
We are concurrently witnessing two significant shifts: digital devices are becoming ubiquitous, a... more We are concurrently witnessing two significant shifts: digital devices are becoming ubiquitous, and older people are becoming a very large demographic group. However, despite the recent increase in related CHI publications, older adults continue to be underrepresented in HCI research as well as commercially. Therefore, the overarching aim of this workshop is to increase the momentum for such research within CHI and related fields such as gerontechnology. For this, we plan to create a space for discussing and sharing principles and strategies to design interactions and evaluate user interfaces (UI) for the ageing population. We thus welcome contributions of empirical studies, theories, design and evaluation of UIs for older adults. Building on the success of last two year's workshops, we aim to grow the community of CHI researchers across borders interested in this topic by fostering a space to exchange results, methods, approaches, and ideas from research on interactive applications in support of older adults that are reflective of international diversity that is representative of CHI.
Beside decreasing the abilities, individual difference prevails among older adults, as some indiv... more Beside decreasing the abilities, individual difference prevails among older adults, as some individuals are completely healthy at the age 90 while some are not at even 60. In context of touchscreen interface design, it is critical to understand the design space as a function of abilities. In this work, we articulate a better understanding of the effects of ageing and examine their HCI task performing capabilities in terms of interfaces design. We design a text-entry interface in particular, as ageing users often achieve slow text entry performance, thus proving to be a bottleneck for technology use. Our developed text entry interface is tuned based on the parameter values for Elderly having finger tremor. We present initial study results showing the improvement of the accuracy of touch typing in smartphone over the baseline Qwerty keyboard. By carefully considering other sensorimotor abilities, we believe that the current smartphone text-entry interface designs will become more usable to the ageing populations.
Uploads
Papers by Xiangshi Ren