2022 27th Asia and South Pacific Design Automation Conference (ASP-DAC), Jan 17, 2022
Perceiving the environment for better and more efficient situational awareness is essential in ap... more Perceiving the environment for better and more efficient situational awareness is essential in applications such as wildlife surveillance, wildfire detection, crop irrigation, and building management. Energy-harvesting, intermittently-powered sensors have emerged as a zero maintenance solution for long-term environmental perception. However, these devices suffer from intermittent and varying energy supply, which presents three major challenges for executing perceptual tasks: (1) intelligently scaling computation in light of constrained resources and dynamic energy availability, (2) planning communication and sensing tasks, (3) and coordinating sensor nodes to increase the total perceptual range of the network. We propose an adaptive framework, AdaSens, which adapts the operations of intermittently-powered sensor nodes in a coordinated manner to cover as much as possible of the targeted scene, both spatially and temporally, under interruptions and constrained resources. We evaluate AdaSens on a real-world surveillance video dataset, VideoWeb, and show at least 16% improvement on the coverage of the important frames compared with other methods.
This paper presents a decentralized control frame-work for lattice-based Modular Self-Reconfigura... more This paper presents a decentralized control frame-work for lattice-based Modular Self-Reconfigurable Robots (MSRR) which utilizes a novel magnetic fiducial system to facilitate neighbor identification and to enable algorithms which promise scalable functionality for systems with many modules. In this system individual modules autonomously follow simple behaviors while periodically accepting input from a centralized controller. This system is demonstrated with three initial behaviors: (1) Path following: modules follow a three dimensional path based on magnetic fiducial tags embedded in their neighbors, (2) Line formation: modules transform from a 3D structure into a line following a partially decentralized control algorithm, and (3) Light gradient aggregation: the formation of a group of modules guided by a global stimulus (i.e. visible light). This paper provides details of the neighbor identification system, introduces the three behaviors and presents the results of physical experiments performed with a system of twelve 3D M-Block robotic modules.
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Whenever a user interacts with a device, mechanical work is performed to actuate the user interfa... more Whenever a user interacts with a device, mechanical work is performed to actuate the user interface elements; the resulting energy is typically wasted, dissipated as sound and heat. Previous work has shown that many devices can be powered entirely from this otherwise wasted user interface energy. For these devices, wires and batteries, along with the related hassles of replacement and charging, become unnecessary and onerous. So far, these works have been restricted to proof-of-concept demonstrations; a specific bespoke harvesting and sensing circuit is constructed for the application at hand. The challenge of harvesting energy while simultaneously sensing fine-grained input signals from diverse modalities makes prototyping new devices difficult. To fill this gap, we present a hardware toolkit which provides a common electrical interface for harvesting energy from user interface elements. This facilitates exploring the composability, utility, and breadth of enabled applications of i...
2022 27th Asia and South Pacific Design Automation Conference (ASP-DAC), 2022
Perceiving the environment for better and more efficient situational awareness is essential in ap... more Perceiving the environment for better and more efficient situational awareness is essential in applications such as wildlife surveillance, wildfire detection, crop irrigation, and building management. Energy-harvesting, intermittently-powered sensors have emerged as a zero maintenance solution for long-term environmental perception. However, these devices suffer from intermittent and varying energy supply, which presents three major challenges for executing perceptual tasks: (1) intelligently scaling computation in light of constrained resources and dynamic energy availability, (2) planning communication and sensing tasks, (3) and coordinating sensor nodes to increase the total perceptual range of the network. We propose an adaptive framework, AdaSens, which adapts the operations of intermittently-powered sensor nodes in a coordinated manner to cover as much as possible of the targeted scene, both spatially and temporally, under interruptions and constrained resources. We evaluate AdaSens on a real-world surveillance video dataset, VideoWeb, and show at least 16% improvement on the coverage of the important frames compared with other methods.
2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019
This paper presents a decentralized control frame-work for lattice-based Modular Self-Reconfigura... more This paper presents a decentralized control frame-work for lattice-based Modular Self-Reconfigurable Robots (MSRR) which utilizes a novel magnetic fiducial system to facilitate neighbor identification and to enable algorithms which promise scalable functionality for systems with many modules. In this system individual modules autonomously follow simple behaviors while periodically accepting input from a centralized controller. This system is demonstrated with three initial behaviors: (1) Path following: modules follow a three dimensional path based on magnetic fiducial tags embedded in their neighbors, (2) Line formation: modules transform from a 3D structure into a line following a partially decentralized control algorithm, and (3) Light gradient aggregation: the formation of a group of modules guided by a global stimulus (i.e. visible light). This paper provides details of the neighbor identification system, introduces the three behaviors and presents the results of physical experiments performed with a system of twelve 3D M-Block robotic modules.
Accessible machine learning algorithms, software, and diagnostic tools for energy-efficient devic... more Accessible machine learning algorithms, software, and diagnostic tools for energy-efficient devices and systems are extremely valuable across a broad range of application domains. In scientific domains, real-time near-sensor processing can drastically improve experimental design and accelerate scientific discoveries. We have developed hls4ml, an open-source software-hardware co-design workflow to interpret and translate machine learning algorithms for implementation in FPGAs and ASICs specifically to support domain scientists. In this paper, we describe the essential features of the hls4ml workflow including network optimization techniques— such as pruning and quantization-aware training—which can be incorporated naturally into the device implementations. We expand on previous hls4ml work by extending capabilities and techniques towards low-power implementations and increased usability: new PythonAPIs, quantization-aware pruning, end-to-end FPGAworkflows, long pipeline kernels for l...
2022 27th Asia and South Pacific Design Automation Conference (ASP-DAC), Jan 17, 2022
Perceiving the environment for better and more efficient situational awareness is essential in ap... more Perceiving the environment for better and more efficient situational awareness is essential in applications such as wildlife surveillance, wildfire detection, crop irrigation, and building management. Energy-harvesting, intermittently-powered sensors have emerged as a zero maintenance solution for long-term environmental perception. However, these devices suffer from intermittent and varying energy supply, which presents three major challenges for executing perceptual tasks: (1) intelligently scaling computation in light of constrained resources and dynamic energy availability, (2) planning communication and sensing tasks, (3) and coordinating sensor nodes to increase the total perceptual range of the network. We propose an adaptive framework, AdaSens, which adapts the operations of intermittently-powered sensor nodes in a coordinated manner to cover as much as possible of the targeted scene, both spatially and temporally, under interruptions and constrained resources. We evaluate AdaSens on a real-world surveillance video dataset, VideoWeb, and show at least 16% improvement on the coverage of the important frames compared with other methods.
This paper presents a decentralized control frame-work for lattice-based Modular Self-Reconfigura... more This paper presents a decentralized control frame-work for lattice-based Modular Self-Reconfigurable Robots (MSRR) which utilizes a novel magnetic fiducial system to facilitate neighbor identification and to enable algorithms which promise scalable functionality for systems with many modules. In this system individual modules autonomously follow simple behaviors while periodically accepting input from a centralized controller. This system is demonstrated with three initial behaviors: (1) Path following: modules follow a three dimensional path based on magnetic fiducial tags embedded in their neighbors, (2) Line formation: modules transform from a 3D structure into a line following a partially decentralized control algorithm, and (3) Light gradient aggregation: the formation of a group of modules guided by a global stimulus (i.e. visible light). This paper provides details of the neighbor identification system, introduces the three behaviors and presents the results of physical experiments performed with a system of twelve 3D M-Block robotic modules.
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Whenever a user interacts with a device, mechanical work is performed to actuate the user interfa... more Whenever a user interacts with a device, mechanical work is performed to actuate the user interface elements; the resulting energy is typically wasted, dissipated as sound and heat. Previous work has shown that many devices can be powered entirely from this otherwise wasted user interface energy. For these devices, wires and batteries, along with the related hassles of replacement and charging, become unnecessary and onerous. So far, these works have been restricted to proof-of-concept demonstrations; a specific bespoke harvesting and sensing circuit is constructed for the application at hand. The challenge of harvesting energy while simultaneously sensing fine-grained input signals from diverse modalities makes prototyping new devices difficult. To fill this gap, we present a hardware toolkit which provides a common electrical interface for harvesting energy from user interface elements. This facilitates exploring the composability, utility, and breadth of enabled applications of i...
2022 27th Asia and South Pacific Design Automation Conference (ASP-DAC), 2022
Perceiving the environment for better and more efficient situational awareness is essential in ap... more Perceiving the environment for better and more efficient situational awareness is essential in applications such as wildlife surveillance, wildfire detection, crop irrigation, and building management. Energy-harvesting, intermittently-powered sensors have emerged as a zero maintenance solution for long-term environmental perception. However, these devices suffer from intermittent and varying energy supply, which presents three major challenges for executing perceptual tasks: (1) intelligently scaling computation in light of constrained resources and dynamic energy availability, (2) planning communication and sensing tasks, (3) and coordinating sensor nodes to increase the total perceptual range of the network. We propose an adaptive framework, AdaSens, which adapts the operations of intermittently-powered sensor nodes in a coordinated manner to cover as much as possible of the targeted scene, both spatially and temporally, under interruptions and constrained resources. We evaluate AdaSens on a real-world surveillance video dataset, VideoWeb, and show at least 16% improvement on the coverage of the important frames compared with other methods.
2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019
This paper presents a decentralized control frame-work for lattice-based Modular Self-Reconfigura... more This paper presents a decentralized control frame-work for lattice-based Modular Self-Reconfigurable Robots (MSRR) which utilizes a novel magnetic fiducial system to facilitate neighbor identification and to enable algorithms which promise scalable functionality for systems with many modules. In this system individual modules autonomously follow simple behaviors while periodically accepting input from a centralized controller. This system is demonstrated with three initial behaviors: (1) Path following: modules follow a three dimensional path based on magnetic fiducial tags embedded in their neighbors, (2) Line formation: modules transform from a 3D structure into a line following a partially decentralized control algorithm, and (3) Light gradient aggregation: the formation of a group of modules guided by a global stimulus (i.e. visible light). This paper provides details of the neighbor identification system, introduces the three behaviors and presents the results of physical experiments performed with a system of twelve 3D M-Block robotic modules.
Accessible machine learning algorithms, software, and diagnostic tools for energy-efficient devic... more Accessible machine learning algorithms, software, and diagnostic tools for energy-efficient devices and systems are extremely valuable across a broad range of application domains. In scientific domains, real-time near-sensor processing can drastically improve experimental design and accelerate scientific discoveries. We have developed hls4ml, an open-source software-hardware co-design workflow to interpret and translate machine learning algorithms for implementation in FPGAs and ASICs specifically to support domain scientists. In this paper, we describe the essential features of the hls4ml workflow including network optimization techniques— such as pruning and quantization-aware training—which can be incorporated naturally into the device implementations. We expand on previous hls4ml work by extending capabilities and techniques towards low-power implementations and increased usability: new PythonAPIs, quantization-aware pruning, end-to-end FPGAworkflows, long pipeline kernels for l...
Uploads
Papers by John Mamish