Journal of Computing and Information Science in Engineering
We present an experimental investigation of spatial audio feedback using smartphones to support d... more We present an experimental investigation of spatial audio feedback using smartphones to support direction localization in pointing tasks for people with visual impairments (PVIs). We do this using a mobile game based on a bow-and-arrow metaphor. Our game provides a combination of spatial and non-spatial (sound beacon) audio to help the user locate the direction of the target. Our experiments with sighted, sighted-blindfolded, and visually impaired users show that (a) the efficacy of spatial audio is relatively higher for PVIs than for blindfolded sighted users during the initial reaction time for direction localization, (b) the general behavior between PVIs and blindfolded individuals is statistically similar, and (c) the lack of spatial audio significantly reduces the localization performance even in sighted-blindfolded users. Based on our findings, we discuss the system and interaction design implications for making future mobile-based spatial interactions accessible to PVIs.
Technology solutions for accessibility have long been created using a narrow utilitarian lens, es... more Technology solutions for accessibility have long been created using a narrow utilitarian lens, especially in the Global South, due to multi-dimensional challenges and resource constraints: an emphasis on purely functional outcomes supported by sterile cost–benefit analysis that ignores the fact that people with disability are people first, with their own aspirations for leisure and enjoyment in addition to skills and employment. We propose an alternative design methodology called the ludic design for accessibility (LDA) that puts play and playfulness at the center of all assistive technology design and use. We then describe a seven-step framework for designers to apply this methodology to create impactful solutions. Though LDA is universally applicable, we highlight the factors that make it especially relevant in the context of accessibility in the Global South.
X Latin American Conference on Human Computer Interaction, 2021
Over 80% of the 1 billion people with disabilities in the world live in the Global South, yet, mo... more Over 80% of the 1 billion people with disabilities in the world live in the Global South, yet, most of the research at the intersection of disability and technology published across HCI venues is based in the Global North. With limited published research, and a large range of socio-cultural, practical and environmental differences across the Global South, HCI researchers and practitioners can encounter unexpected challenges when planning and conducting accessibility studies. The aim of this panel is to bring together three experienced academics who have conducted, managed and published impactful HCI research working with people with disabilities in various regions of the Global South from Latin America to East Africa and South Asia. Panelists will give an initial overview of their experiences conducting research with participants who had different disabilities and other relevant stakeholders, dedicating particular attention to the practical aspects commonly involved in research projects. CCS CONCEPTS • Social and professional topics → People with disabilities • HCI Design and evaluation methods → Field Studies
Camera based assistive technologies can provide people with visual impairments (PVIs) visually de... more Camera based assistive technologies can provide people with visual impairments (PVIs) visually derived information about people in their vicinity. Furthermore, the advent of smart glasses offers the possibility of not only analyzing visual information in front of the wearer, but also behind them through an extended field of view. Although such ‘visually available’ information can enhance one’s social interactions, the privacy and ethical implications for automated judgments about bystanders especially from the perspective of PVIs remains underexplored. To study the concerns of both bystanders and PVIs with such technologies, we conducted two online surveys with visually impaired participants as wearers (N=128) and sighted participants as bystanders (N=136). Although PVIs found some types of information to be improper or impolite (such as someone’s weight), our overarching finding is the shared ethical concern between PVIs and bystanders related to the fallibility of AI, where bystan...
Proceedings of the 16th ACM Conference on Embedded Networked Sensor Systems, 2018
The proliferation of Internet of Things (IoT) devices has led to the deployment of various types ... more The proliferation of Internet of Things (IoT) devices has led to the deployment of various types of sensors in the homes, offices, buildings, lawns, cities, and even in agricultural farms. Due to the diverse nature of IoT deployments and the likelihood of sensor failures in-the-wild, a key challenge in the design of IoT systems is ensuring the integrity, accuracy, and fidelity of sensor data. We present a system based on the Fall-curve primitive -- a sensor's voltage response when the power is turned off -- to characterize the sensor. A sensor's Fall-curve constitutes a unique signature using which it is possible to identify the sensor and also detect whether it is operating correctly or not. In this demo, we show Fall-curve in action to accurately detect and identify the sensors connected to an IoT device. Furthermore, we also show that Fall-curves can reliably detect various transient and permanent sensor faults in an IoT device.
Proceedings of the 17th Conference on Embedded Networked Sensor Systems, 2019
Drive-by sensing has emerged as a popular way to achieve fine-grained sensing of physical phenome... more Drive-by sensing has emerged as a popular way to achieve fine-grained sensing of physical phenomena. However, for it to be effective at a city-scale, there is a need to optimally select a subset of vehicles from a larger available fleet. These chosen vehicles must maximize coverage of the entire city. Simultaneously, they must fulfill other deployment requirements specific to the sensing application such as reference-monitor colocation instances for gas sensors. In this paper, we describe a system to evaluate the coverage offered by different subsets of vehicles for sensor deployment based on historical vehicle mobility data. Our system allows evaluation of different vehicle selection algorithms, and also provides two in-built baselines --- i) Random-MP, and ii) MaxPoints --- for comparison. Finally, we provide visualizations showing coverage to gauge the efficacy of different vehicle selections.
We highlight the critical need to include voices from disability studies in the Global south in a... more We highlight the critical need to include voices from disability studies in the Global south in any discourse on Assistive technology design and development. We present the following reasons for this importance: 1) The majority of the global population of people with disabilities live in the global south. 2) The lived realities of people with disabilities (PwDs)in the global south are very different from those of the people with disabilities in the global north, including the capacity of the PwDs to absorb any available benefits. 3) Assistive technology work in the global north works on the premise that independence and access that is equal to the mainstream population is the desired end goal of the people with disabilities. We believe that the design of assistive technology solutions for these populations have to be rethought from the bottom up, taking the social context and the existing networks of support around PwDs.
We present a device for visually impaired persons (VIPs) that delivers contextual audio assistanc... more We present a device for visually impaired persons (VIPs) that delivers contextual audio assistance for physical objects and tasks. In initial observations, we found ubiquitous use of audio-assistance technologies by VIPs for interacting with computing devices, such as Android TalkBack. However, we also saw that devices without screens frequently lack accessibility features. Our solution allows a VIP to obtain audio assistance in the presence of an arbitrary physical interface or object through a chest-mounted device. On-board are camera sensors that point towards the user's personal front-facing grasping region. Upon detecting certain gestures such as picking up an object, the device provides helpful contextual audio information to the user. Textual interfaces can be read aloud by sliding a finger over the surface of the object, allowing the user to hear a document or receive audio guidance for non-assistively-enabled electronic devices. The user may provide questions verbally i...
Proceedings of the International Conference on Internet-of-Things Design and Implementation, 2021
Sensor data quality plays a fundamental role in increasing the adoption of IoT devices for enviro... more Sensor data quality plays a fundamental role in increasing the adoption of IoT devices for environmental data collection. Due to the nature of the deployment, i.e., in-the-wild and in harsh environments, coupled with limitations of low-cost components, sensors are prone to failures. A significant fraction of faults result from drift and catastrophic faults in sensors' sensing components leading to serious data inaccuracies. However, it is challenging to detect faults by analyzing just the sensor data as a faulty sensor data can mimic non-faulty data and an anomalous sensor reading need not represent a faulty data. Existing data-centric approaches rely on additional contextual information or sensor redundancy to detect such faults. This paper presents a systematic approach to detect faults and drifts, by devising a novel sensor fingerprint called Cur-rentSense. CurrentSense captures the electrical characteristics of the hardware components in a sensor, with working, drifted, and faulty sensors having distinct fingerprints. This fingerprint is used to determine the sensors' health, and compensate for drift or diagnose catastrophic faults without any contextual information. The CurrentSense approach is non-intrusive, and can be applied to a wide variety of environmental sensors. We show the working of the proposed approach with the help of air pollution sensors. We perform an extensive evaluation in both controlled setup and real-world deployments with 51 sensors across multiple cities for 8 months period. Our approach outperforms existing anomaly detectors and can detect and isolate faults with an 1 score of 98% and compensate for sensor drift errors by 86%.
Proceedings of the 18th Conference on Embedded Networked Sensor Systems, 2020
Fine-grained air pollution monitoring is a fundamental step towards curbing pollution levels. Thi... more Fine-grained air pollution monitoring is a fundamental step towards curbing pollution levels. This is sought to be achieved by the large-scale deployment of low-cost sensors at high spatio-temporal resolution. Due to the nature of these deployments, in-the-wild and in harsh environments, sensors are prone to failures and hence ensuring data reliability is challenging. Furthermore, detecting a fault by analyzing the sensor data using existing data-centric approaches is non-trivial. This demonstration presents a sensor fault diagnosis system that employs the current signature of the sensor to address data reliability issues. The current signature captures the electrical characteristics of the hardware components enabling accurate detection and isolation of faults in low-cost pollution sensors.
Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 2021
Most children who are blind live in low-resource settings and attend schools that have poor techn... more Most children who are blind live in low-resource settings and attend schools that have poor technical infrastructure, overburdened teachers, and outdated curriculum. Our work explores the role digital games can play to develop digital skills of such children in the early grades. Recognizing the critical role of teachers in introducing children to technology, we conducted a mixed-methods study to examine which attributes of digital games teachers find useful for children and what challenges they perceive in integrating digital games in schools for the blind. Our findings indicate that teachers prefer games that align well with curriculum objectives, promote learning, improve soft skills, and increase engagement with computers. Despite being overburdened and lacking technological support, teachers expressed strong enthusiasm to integrate these games in school curriculum and schedule. We conclude by discussing design implications for designers of accessible games in low-resource settings. CCS CONCEPTS • Human-centered computing → Accessibility; Empirical studies in HCI .
Proceedings of the Conference on Computing & Sustainable Societies - COMPASS 19, 2019
Recent work in networked systems has shown that using aerial imagery for farm monitoring can enab... more Recent work in networked systems has shown that using aerial imagery for farm monitoring can enable precision agriculture by lowering the cost and reducing the overhead of large scale sensor deployment. However, acquiring aerial imagery requires a drone, which has high capital and operational costs, often beyond the reach of farmers in the developing world. In this paper, we present TYE (Tethered eYE), an inexpensive platform for aerial imagery. It consists of a tethered helium balloon with a custom mount that can hold a smartphone (or a camera) with a battery pack. The balloon can be carried using a tether by a person or a vehicle. We incorporate various techniques to increase the operational time of the system, and to provide actionable insights even with unstable imagery. We develop path-planning algorithms *The work was done when the authors were at Microsoft.
Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, 2021
Augmented, virtual and mixed reality technologies offer new ways of interacting with digital medi... more Augmented, virtual and mixed reality technologies offer new ways of interacting with digital media. However, such technologies are not well explored for people with different ranges of abilities beyond a few specific navigation and gaming applications. While new standardization activities are investigating accessibility issues with existing AR/VR systems, commercial systems are still confined to specialized hardware and software limiting their widespread adoption among people with disabilities as well as seniors. This proposal takes a novel approach by exploring the application of user model-based personalization for AR/VR systems to improve accessibility. The workshop will be organized by experienced researchers in the field of human computer interaction, robotics control, assistive technology, and AR/VR systems, and will consist of peer reviewed papers and hands-on demonstrations. Keynote speeches and demonstrations will cover latest accessibility research at Microsoft, Google, Verizon and leading universities.
Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 2021
People with vision impairments access smartphones with the help of screen reader apps such as Tal... more People with vision impairments access smartphones with the help of screen reader apps such as TalkBack for Android and VoiceOver for iPhone. Prior research has mostly focused on understanding touchscreen phone adoption and typing performance of novice blind users by logging their real-world smartphone usage. Understanding smartphone usage pattern and practices of expert users can help in developing tools and tutorials for transitioning novice and intermediate users to expert users. In this work, we logged smartphone usage data of eight expert Android smartphone users with visual impairments for four weeks, and then interviewed them. This paper presents a detailed analysis that uncovered novel usage patterns, such as extensive usage of directional gestures, reliance on voice and external keyboard for text input, and repurposed explore by touch for single-tap. We conclude with design recommendations to inform the future of mobile accessibility, including hardware guidelines and rethinking accessible software design.
Journal of Computing and Information Science in Engineering
We present an experimental investigation of spatial audio feedback using smartphones to support d... more We present an experimental investigation of spatial audio feedback using smartphones to support direction localization in pointing tasks for people with visual impairments (PVIs). We do this using a mobile game based on a bow-and-arrow metaphor. Our game provides a combination of spatial and non-spatial (sound beacon) audio to help the user locate the direction of the target. Our experiments with sighted, sighted-blindfolded, and visually impaired users show that (a) the efficacy of spatial audio is relatively higher for PVIs than for blindfolded sighted users during the initial reaction time for direction localization, (b) the general behavior between PVIs and blindfolded individuals is statistically similar, and (c) the lack of spatial audio significantly reduces the localization performance even in sighted-blindfolded users. Based on our findings, we discuss the system and interaction design implications for making future mobile-based spatial interactions accessible to PVIs.
Technology solutions for accessibility have long been created using a narrow utilitarian lens, es... more Technology solutions for accessibility have long been created using a narrow utilitarian lens, especially in the Global South, due to multi-dimensional challenges and resource constraints: an emphasis on purely functional outcomes supported by sterile cost–benefit analysis that ignores the fact that people with disability are people first, with their own aspirations for leisure and enjoyment in addition to skills and employment. We propose an alternative design methodology called the ludic design for accessibility (LDA) that puts play and playfulness at the center of all assistive technology design and use. We then describe a seven-step framework for designers to apply this methodology to create impactful solutions. Though LDA is universally applicable, we highlight the factors that make it especially relevant in the context of accessibility in the Global South.
X Latin American Conference on Human Computer Interaction, 2021
Over 80% of the 1 billion people with disabilities in the world live in the Global South, yet, mo... more Over 80% of the 1 billion people with disabilities in the world live in the Global South, yet, most of the research at the intersection of disability and technology published across HCI venues is based in the Global North. With limited published research, and a large range of socio-cultural, practical and environmental differences across the Global South, HCI researchers and practitioners can encounter unexpected challenges when planning and conducting accessibility studies. The aim of this panel is to bring together three experienced academics who have conducted, managed and published impactful HCI research working with people with disabilities in various regions of the Global South from Latin America to East Africa and South Asia. Panelists will give an initial overview of their experiences conducting research with participants who had different disabilities and other relevant stakeholders, dedicating particular attention to the practical aspects commonly involved in research projects. CCS CONCEPTS • Social and professional topics → People with disabilities • HCI Design and evaluation methods → Field Studies
Camera based assistive technologies can provide people with visual impairments (PVIs) visually de... more Camera based assistive technologies can provide people with visual impairments (PVIs) visually derived information about people in their vicinity. Furthermore, the advent of smart glasses offers the possibility of not only analyzing visual information in front of the wearer, but also behind them through an extended field of view. Although such ‘visually available’ information can enhance one’s social interactions, the privacy and ethical implications for automated judgments about bystanders especially from the perspective of PVIs remains underexplored. To study the concerns of both bystanders and PVIs with such technologies, we conducted two online surveys with visually impaired participants as wearers (N=128) and sighted participants as bystanders (N=136). Although PVIs found some types of information to be improper or impolite (such as someone’s weight), our overarching finding is the shared ethical concern between PVIs and bystanders related to the fallibility of AI, where bystan...
Proceedings of the 16th ACM Conference on Embedded Networked Sensor Systems, 2018
The proliferation of Internet of Things (IoT) devices has led to the deployment of various types ... more The proliferation of Internet of Things (IoT) devices has led to the deployment of various types of sensors in the homes, offices, buildings, lawns, cities, and even in agricultural farms. Due to the diverse nature of IoT deployments and the likelihood of sensor failures in-the-wild, a key challenge in the design of IoT systems is ensuring the integrity, accuracy, and fidelity of sensor data. We present a system based on the Fall-curve primitive -- a sensor's voltage response when the power is turned off -- to characterize the sensor. A sensor's Fall-curve constitutes a unique signature using which it is possible to identify the sensor and also detect whether it is operating correctly or not. In this demo, we show Fall-curve in action to accurately detect and identify the sensors connected to an IoT device. Furthermore, we also show that Fall-curves can reliably detect various transient and permanent sensor faults in an IoT device.
Proceedings of the 17th Conference on Embedded Networked Sensor Systems, 2019
Drive-by sensing has emerged as a popular way to achieve fine-grained sensing of physical phenome... more Drive-by sensing has emerged as a popular way to achieve fine-grained sensing of physical phenomena. However, for it to be effective at a city-scale, there is a need to optimally select a subset of vehicles from a larger available fleet. These chosen vehicles must maximize coverage of the entire city. Simultaneously, they must fulfill other deployment requirements specific to the sensing application such as reference-monitor colocation instances for gas sensors. In this paper, we describe a system to evaluate the coverage offered by different subsets of vehicles for sensor deployment based on historical vehicle mobility data. Our system allows evaluation of different vehicle selection algorithms, and also provides two in-built baselines --- i) Random-MP, and ii) MaxPoints --- for comparison. Finally, we provide visualizations showing coverage to gauge the efficacy of different vehicle selections.
We highlight the critical need to include voices from disability studies in the Global south in a... more We highlight the critical need to include voices from disability studies in the Global south in any discourse on Assistive technology design and development. We present the following reasons for this importance: 1) The majority of the global population of people with disabilities live in the global south. 2) The lived realities of people with disabilities (PwDs)in the global south are very different from those of the people with disabilities in the global north, including the capacity of the PwDs to absorb any available benefits. 3) Assistive technology work in the global north works on the premise that independence and access that is equal to the mainstream population is the desired end goal of the people with disabilities. We believe that the design of assistive technology solutions for these populations have to be rethought from the bottom up, taking the social context and the existing networks of support around PwDs.
We present a device for visually impaired persons (VIPs) that delivers contextual audio assistanc... more We present a device for visually impaired persons (VIPs) that delivers contextual audio assistance for physical objects and tasks. In initial observations, we found ubiquitous use of audio-assistance technologies by VIPs for interacting with computing devices, such as Android TalkBack. However, we also saw that devices without screens frequently lack accessibility features. Our solution allows a VIP to obtain audio assistance in the presence of an arbitrary physical interface or object through a chest-mounted device. On-board are camera sensors that point towards the user's personal front-facing grasping region. Upon detecting certain gestures such as picking up an object, the device provides helpful contextual audio information to the user. Textual interfaces can be read aloud by sliding a finger over the surface of the object, allowing the user to hear a document or receive audio guidance for non-assistively-enabled electronic devices. The user may provide questions verbally i...
Proceedings of the International Conference on Internet-of-Things Design and Implementation, 2021
Sensor data quality plays a fundamental role in increasing the adoption of IoT devices for enviro... more Sensor data quality plays a fundamental role in increasing the adoption of IoT devices for environmental data collection. Due to the nature of the deployment, i.e., in-the-wild and in harsh environments, coupled with limitations of low-cost components, sensors are prone to failures. A significant fraction of faults result from drift and catastrophic faults in sensors' sensing components leading to serious data inaccuracies. However, it is challenging to detect faults by analyzing just the sensor data as a faulty sensor data can mimic non-faulty data and an anomalous sensor reading need not represent a faulty data. Existing data-centric approaches rely on additional contextual information or sensor redundancy to detect such faults. This paper presents a systematic approach to detect faults and drifts, by devising a novel sensor fingerprint called Cur-rentSense. CurrentSense captures the electrical characteristics of the hardware components in a sensor, with working, drifted, and faulty sensors having distinct fingerprints. This fingerprint is used to determine the sensors' health, and compensate for drift or diagnose catastrophic faults without any contextual information. The CurrentSense approach is non-intrusive, and can be applied to a wide variety of environmental sensors. We show the working of the proposed approach with the help of air pollution sensors. We perform an extensive evaluation in both controlled setup and real-world deployments with 51 sensors across multiple cities for 8 months period. Our approach outperforms existing anomaly detectors and can detect and isolate faults with an 1 score of 98% and compensate for sensor drift errors by 86%.
Proceedings of the 18th Conference on Embedded Networked Sensor Systems, 2020
Fine-grained air pollution monitoring is a fundamental step towards curbing pollution levels. Thi... more Fine-grained air pollution monitoring is a fundamental step towards curbing pollution levels. This is sought to be achieved by the large-scale deployment of low-cost sensors at high spatio-temporal resolution. Due to the nature of these deployments, in-the-wild and in harsh environments, sensors are prone to failures and hence ensuring data reliability is challenging. Furthermore, detecting a fault by analyzing the sensor data using existing data-centric approaches is non-trivial. This demonstration presents a sensor fault diagnosis system that employs the current signature of the sensor to address data reliability issues. The current signature captures the electrical characteristics of the hardware components enabling accurate detection and isolation of faults in low-cost pollution sensors.
Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 2021
Most children who are blind live in low-resource settings and attend schools that have poor techn... more Most children who are blind live in low-resource settings and attend schools that have poor technical infrastructure, overburdened teachers, and outdated curriculum. Our work explores the role digital games can play to develop digital skills of such children in the early grades. Recognizing the critical role of teachers in introducing children to technology, we conducted a mixed-methods study to examine which attributes of digital games teachers find useful for children and what challenges they perceive in integrating digital games in schools for the blind. Our findings indicate that teachers prefer games that align well with curriculum objectives, promote learning, improve soft skills, and increase engagement with computers. Despite being overburdened and lacking technological support, teachers expressed strong enthusiasm to integrate these games in school curriculum and schedule. We conclude by discussing design implications for designers of accessible games in low-resource settings. CCS CONCEPTS • Human-centered computing → Accessibility; Empirical studies in HCI .
Proceedings of the Conference on Computing & Sustainable Societies - COMPASS 19, 2019
Recent work in networked systems has shown that using aerial imagery for farm monitoring can enab... more Recent work in networked systems has shown that using aerial imagery for farm monitoring can enable precision agriculture by lowering the cost and reducing the overhead of large scale sensor deployment. However, acquiring aerial imagery requires a drone, which has high capital and operational costs, often beyond the reach of farmers in the developing world. In this paper, we present TYE (Tethered eYE), an inexpensive platform for aerial imagery. It consists of a tethered helium balloon with a custom mount that can hold a smartphone (or a camera) with a battery pack. The balloon can be carried using a tether by a person or a vehicle. We incorporate various techniques to increase the operational time of the system, and to provide actionable insights even with unstable imagery. We develop path-planning algorithms *The work was done when the authors were at Microsoft.
Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, 2021
Augmented, virtual and mixed reality technologies offer new ways of interacting with digital medi... more Augmented, virtual and mixed reality technologies offer new ways of interacting with digital media. However, such technologies are not well explored for people with different ranges of abilities beyond a few specific navigation and gaming applications. While new standardization activities are investigating accessibility issues with existing AR/VR systems, commercial systems are still confined to specialized hardware and software limiting their widespread adoption among people with disabilities as well as seniors. This proposal takes a novel approach by exploring the application of user model-based personalization for AR/VR systems to improve accessibility. The workshop will be organized by experienced researchers in the field of human computer interaction, robotics control, assistive technology, and AR/VR systems, and will consist of peer reviewed papers and hands-on demonstrations. Keynote speeches and demonstrations will cover latest accessibility research at Microsoft, Google, Verizon and leading universities.
Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 2021
People with vision impairments access smartphones with the help of screen reader apps such as Tal... more People with vision impairments access smartphones with the help of screen reader apps such as TalkBack for Android and VoiceOver for iPhone. Prior research has mostly focused on understanding touchscreen phone adoption and typing performance of novice blind users by logging their real-world smartphone usage. Understanding smartphone usage pattern and practices of expert users can help in developing tools and tutorials for transitioning novice and intermediate users to expert users. In this work, we logged smartphone usage data of eight expert Android smartphone users with visual impairments for four weeks, and then interviewed them. This paper presents a detailed analysis that uncovered novel usage patterns, such as extensive usage of directional gestures, reliance on voice and external keyboard for text input, and repurposed explore by touch for single-tap. We conclude with design recommendations to inform the future of mobile accessibility, including hardware guidelines and rethinking accessible software design.
Uploads
Papers by Swami Manohar