2018 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), 2018
The success of the exascale supercomputer is largely debated to remain dependent on novel breakth... more The success of the exascale supercomputer is largely debated to remain dependent on novel breakthroughs in technology that effectively reduce the power consumption and thermal dissipation requirements. In this work, we consider the integration of co-processors in high-performance computing (HPC) to enable low-power, seamless computation offloading of certain operations. In particular, we explore the so-called Vision Processing Unit (VPU), a highly-parallel vector processor with a power envelope of less than 1W. We evaluate this chip during inference using a pre-trained GoogLeNet convolutional network model and a large image dataset from the ImageNet ILSVRC challenge. Preliminary results indicate that a multi-VPU configuration provides similar performance compared to reference CPU and GPU implementations, while reducing the thermal-design power (TDP) up to 8× in comparison.
Proceedings of the 4th International Conference on Geographical Information Systems Theory, Applications and Management, 2018
The Volumetric Accelerator (VOLA) format is a compact data structure that unifies computer vision... more The Volumetric Accelerator (VOLA) format is a compact data structure that unifies computer vision and 3D rendering and allows for the rapid calculation of connected components, per-voxel census/accounting, Deep Learning and Convolutional Neural Network (CNN) inference, path planning and obstacle avoidance. Using a hierarchical bit array format allows it to run efficiently on embedded systems and maximize the level of data compression for network transmission. The proposed format allows massive scale volumetric data to be used in embedded applications where it would be inconceivable to utilize point-clouds due to memory constraints. Furthermore, geographical and qualitative data is embedded in the file structure to allow it to be used in place of standard point cloud formats. This work examines the reduction in file size when encoding 3D data using the VOLA format. Four real world Light Detection and Ranging (LiDAR) datasets are converted and produced data an order of magnitude smaller than the current binary standard for point cloud data. Additionally, a new metric based on a neighborhood lookup is developed that measures an accurate resolution for a point cloud dataset.
•Volumetric applications can run efficiently on embedded platforms •Optimal data-structures can a... more •Volumetric applications can run efficiently on embedded platforms •Optimal data-structures can allow 128× reduction in RAM requirements •Bit-per-voxel Octree allows compact interchangeable format for M2M •Two bit's per voxel allows colour and other information to be stored per sub-volume
Proceedings of the 4th International Conference on Geographical Information Systems Theory, Applications and Management, 2018
Traditional hashing methods used to store 3D volumetric data utilise large prime numbers. The int... more Traditional hashing methods used to store 3D volumetric data utilise large prime numbers. The intention of this is to achieve well-distributed hash addresses to minimise addressing collisions. Hashing is an attractive method to store 3D volumetric data, as it provides simple method to store, index and retrieve data. However, implementations fail to provide theoretical support as to why they utilise large primes which act to create a hash address through randomising key values. Neither is it specified what a "large" prime is. 3D data is inherently well-distributed as each coordinate in 3D space is already unique. It is thus investigated in this paper whether this randomisation through the use of large primes is necessary. The history of the use of primes for hashing 3D data is also investigated, as is whether their use has persisted due to habit rather than due to methodical investigation.
Proceedings of the 4th International Conference on Geographical Information Systems Theory, Applications and Management, 2018
LiDAR provides highly accurate 3D point cloud data for a number of tasks such as forest surveying... more LiDAR provides highly accurate 3D point cloud data for a number of tasks such as forest surveying and urban planning. Automatic classification of this data, however, is challenging since the dataset can be extremely large and manual annotation is labour intensive if not impossible. We provide a method of automatically annotating airborne LiDAR data for individual trees or tree regions by filtering out the ground measurements and then using the number of returns embedded in the dataset. The method is validated on a manually annotated dataset for Dublin city with promising results.
While several approaches to face emotion recognition task are proposed in literature, none of the... more While several approaches to face emotion recognition task are proposed in literature, none of them reports on power consumption nor inference time required to run the system in an embedded environment. Without adequate knowledge about these factors it is not clear whether we are actually able to provide accurate face emotion recognition in the embedded environment or not, and if not, how far we are from making it feasible and what are the biggest bottlenecks we face. The main goal of this paper is to answer these questions and to convey the message that instead of reporting only detection accuracy also power consumption and inference time should be reported as real usability of the proposed systems and their adoption in human computer interaction strongly depends on it. In this paper, we identify the state-of-the art face emotion recognition methods that are potentially suitable for embedded environment and the most frequently used datasets for this task. Our study shows that most o...
In this paper, we study deep learning approaches for monocular visual odometry (VO). Deep learnin... more In this paper, we study deep learning approaches for monocular visual odometry (VO). Deep learning solutions have shown to be effective in VO applications, replacing the need for highly engineered steps, such as feature extraction and outlier rejection in a traditional pipeline. We propose a new architecture combining ego-motion estimation and sequence-based learning using deep neural networks. We estimate camera motion from optical flow using Convolutional Neural Networks (CNNs) and model the motion dynamics using Recurrent Neural Networks (RNNs). The network outputs the relative 6-DOF camera poses for a sequence, and implicitly learns the absolute scale without the need for camera intrinsics. The entire trajectory is then integrated without any post-calibration. We evaluate the proposed method on the KITTI dataset and compare it with traditional and other deep learning approaches in the literature.
IEE Irish Signals and Systems Conference 2005, 2005
An accelerator for linear algebra kernel functions is described, in particular the sparse-matrix ... more An accelerator for linear algebra kernel functions is described, in particular the sparse-matrix by vector multiplication (SMVM) computation which is at the heart of all finite element calculations. The proposed architecture seeks to accelerate the key performance-limiting SMVM operation at the heart of these applications through a combination of a dedicated datapath optimized for these applications, in particular by incorporating direct support for symmetric matrices, and SMVM (sparse-matrix by vector multiplication) to dot-product (DDOT) chaining. System simulations performed using a cycle-accurate C++ architectural model and a database of over 100 large symmetric matrices demonstrate that a 20% average FLOPs/cycle performance improvement can be achieved for the symmetric case.
The required precision for attitude determination in spacecraft is increasing, providing a need f... more The required precision for attitude determination in spacecraft is increasing, providing a need for more accurate attitude determination sensors. The star sensor or star tracker provides unmatched arc-second precision and with the rise of micro satellites these sensors are becoming smaller, faster and more efficient. The most critical component in the star sensor system is the lost-in-space star identification algorithm which identifies stars in a scene without a priori attitude information. In this paper, we present an efficient lost-in-space star identification algorithm using a neural network and a robust and novel feature extraction method. Since a neural network implicitly stores the patterns associated with a guide star, a database lookup is eliminated from the matching process. The search time is therefore not influenced by the number of patterns stored in the network, making it constant (O(1)). This search time is unrivalled by other star identification algorithms. The prese...
The lost-in-space star identification algorithm is able to identify stars without a priori attitu... more The lost-in-space star identification algorithm is able to identify stars without a priori attitude information and is arguably the most critical component of a star sensor system. In this paper, the 2009 survey by Spratling and Mortari is extended and recent lost-in-space star identification algorithms are surveyed. The covered literature is a qualitative representation of the current research in the field. A taxonomy of these algorithms based on their feature extraction method is defined. Furthermore, we show that in current literature the comparison of these algorithms can produce inconsistent conclusions. In order to mitigate these inconsistencies, this paper lists the considerations related to the relative performance evaluation of these algorithms using simulation.
Computer vision and deep learning are clearly demonstrating a capability to create engaging cogni... more Computer vision and deep learning are clearly demonstrating a capability to create engaging cognitive applications and services. However, these applications have been mostly confined to powerful Graphic Processing Units (GPUs) or the cloud due to their demanding computational requirements. Cloud processing has obvious bandwidth, energy consumption and privacy issues. The Eyes of Things (EoT) is a powerful and versatile embedded computer vision platform which allows the user to develop artificial vision and deep learning applications that analyse images locally. In this article, we use the deep learning capabilities of an EoT device for a real-life facial informatics application: a doll capable of recognizing emotions, using deep learning techniques, and acting accordingly. The main impact and significance of the presented application is in showing that a toy can now do advanced processing locally, without the need of further computation in the cloud, thus reducing latency and removi...
2016 International Conference on Image, Vision and Computing (ICIVC), 2016
Commonly, HoG/SVM classifier uses rectangular images for HoG feature descriptor extraction and tr... more Commonly, HoG/SVM classifier uses rectangular images for HoG feature descriptor extraction and training. This means that significant additional work has to be done to process irrelevant pixels belonging to the background surrounding the object of interest. Moreover, some areas of the foreground also can be eliminated from the processing to improve the algorithm speed and memory wise. In Boundary-Bitmap HoG approach proposed in this paper, the boundary of irregular shape of the object is represented by a bitmap to avoid processing of extra background and (partially) foreground pixels. Bitmap, derived from the training dataset, encodes those portions of an image to be used to train a classifier. Experimental results show that not only the proposed algorithm decreases the workload associated with HoG/SVM classifiers by 92.5% compared to the state-of-the-art, but also it shows an average increase about 6% in recall and a decrease about 3% in precision in comparison with standard HoG.
Presents a conference poster that addresses precision refinement for media processors for system-... more Presents a conference poster that addresses precision refinement for media processors for system-on-chip systems.
This article consists of a collection of slides from the author's conference presentation on ... more This article consists of a collection of slides from the author's conference presentation on the special features, system design, processing capabilities, and targeted markets for Movidius's 1TOPS/W, a software programmable media processor.
2016 First International Workshop on Sensing, Processing and Learning for Intelligent Machines (SPLINE), 2016
Following the success of Convolutional Neural Networks (CNNs) on object recognition using 2D imag... more Following the success of Convolutional Neural Networks (CNNs) on object recognition using 2D images, they are extended in this paper to process 3D data. Nearly most of current systems require huge amount of computation for dealing with large amount of data. In this paper, an efficient 3D volumetric object representation, Volumetric Accelerator (VOLA), is presented which requires much less memory than the normal volumetric representations. Based on this, a few 3D digit datasets using 2D MNIST and 2D digit fonts with different rotations along the x, y, and z axis are introduced. Finally, we introduce a combination of multiple CNN models based on the famous LeNet model. The trained CNN models based on the generated dataset have achieved the average accuracy of 90.30% and 81.85% for 3D-MNIST and 3D-Fonts datasets, respectively. Furthermore, experimental results show that VOLA-based CNNs perform 1.5x faster than the original LeNet.
2018 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), 2018
The success of the exascale supercomputer is largely debated to remain dependent on novel breakth... more The success of the exascale supercomputer is largely debated to remain dependent on novel breakthroughs in technology that effectively reduce the power consumption and thermal dissipation requirements. In this work, we consider the integration of co-processors in high-performance computing (HPC) to enable low-power, seamless computation offloading of certain operations. In particular, we explore the so-called Vision Processing Unit (VPU), a highly-parallel vector processor with a power envelope of less than 1W. We evaluate this chip during inference using a pre-trained GoogLeNet convolutional network model and a large image dataset from the ImageNet ILSVRC challenge. Preliminary results indicate that a multi-VPU configuration provides similar performance compared to reference CPU and GPU implementations, while reducing the thermal-design power (TDP) up to 8× in comparison.
Proceedings of the 4th International Conference on Geographical Information Systems Theory, Applications and Management, 2018
The Volumetric Accelerator (VOLA) format is a compact data structure that unifies computer vision... more The Volumetric Accelerator (VOLA) format is a compact data structure that unifies computer vision and 3D rendering and allows for the rapid calculation of connected components, per-voxel census/accounting, Deep Learning and Convolutional Neural Network (CNN) inference, path planning and obstacle avoidance. Using a hierarchical bit array format allows it to run efficiently on embedded systems and maximize the level of data compression for network transmission. The proposed format allows massive scale volumetric data to be used in embedded applications where it would be inconceivable to utilize point-clouds due to memory constraints. Furthermore, geographical and qualitative data is embedded in the file structure to allow it to be used in place of standard point cloud formats. This work examines the reduction in file size when encoding 3D data using the VOLA format. Four real world Light Detection and Ranging (LiDAR) datasets are converted and produced data an order of magnitude smaller than the current binary standard for point cloud data. Additionally, a new metric based on a neighborhood lookup is developed that measures an accurate resolution for a point cloud dataset.
•Volumetric applications can run efficiently on embedded platforms •Optimal data-structures can a... more •Volumetric applications can run efficiently on embedded platforms •Optimal data-structures can allow 128× reduction in RAM requirements •Bit-per-voxel Octree allows compact interchangeable format for M2M •Two bit's per voxel allows colour and other information to be stored per sub-volume
Proceedings of the 4th International Conference on Geographical Information Systems Theory, Applications and Management, 2018
Traditional hashing methods used to store 3D volumetric data utilise large prime numbers. The int... more Traditional hashing methods used to store 3D volumetric data utilise large prime numbers. The intention of this is to achieve well-distributed hash addresses to minimise addressing collisions. Hashing is an attractive method to store 3D volumetric data, as it provides simple method to store, index and retrieve data. However, implementations fail to provide theoretical support as to why they utilise large primes which act to create a hash address through randomising key values. Neither is it specified what a "large" prime is. 3D data is inherently well-distributed as each coordinate in 3D space is already unique. It is thus investigated in this paper whether this randomisation through the use of large primes is necessary. The history of the use of primes for hashing 3D data is also investigated, as is whether their use has persisted due to habit rather than due to methodical investigation.
Proceedings of the 4th International Conference on Geographical Information Systems Theory, Applications and Management, 2018
LiDAR provides highly accurate 3D point cloud data for a number of tasks such as forest surveying... more LiDAR provides highly accurate 3D point cloud data for a number of tasks such as forest surveying and urban planning. Automatic classification of this data, however, is challenging since the dataset can be extremely large and manual annotation is labour intensive if not impossible. We provide a method of automatically annotating airborne LiDAR data for individual trees or tree regions by filtering out the ground measurements and then using the number of returns embedded in the dataset. The method is validated on a manually annotated dataset for Dublin city with promising results.
While several approaches to face emotion recognition task are proposed in literature, none of the... more While several approaches to face emotion recognition task are proposed in literature, none of them reports on power consumption nor inference time required to run the system in an embedded environment. Without adequate knowledge about these factors it is not clear whether we are actually able to provide accurate face emotion recognition in the embedded environment or not, and if not, how far we are from making it feasible and what are the biggest bottlenecks we face. The main goal of this paper is to answer these questions and to convey the message that instead of reporting only detection accuracy also power consumption and inference time should be reported as real usability of the proposed systems and their adoption in human computer interaction strongly depends on it. In this paper, we identify the state-of-the art face emotion recognition methods that are potentially suitable for embedded environment and the most frequently used datasets for this task. Our study shows that most o...
In this paper, we study deep learning approaches for monocular visual odometry (VO). Deep learnin... more In this paper, we study deep learning approaches for monocular visual odometry (VO). Deep learning solutions have shown to be effective in VO applications, replacing the need for highly engineered steps, such as feature extraction and outlier rejection in a traditional pipeline. We propose a new architecture combining ego-motion estimation and sequence-based learning using deep neural networks. We estimate camera motion from optical flow using Convolutional Neural Networks (CNNs) and model the motion dynamics using Recurrent Neural Networks (RNNs). The network outputs the relative 6-DOF camera poses for a sequence, and implicitly learns the absolute scale without the need for camera intrinsics. The entire trajectory is then integrated without any post-calibration. We evaluate the proposed method on the KITTI dataset and compare it with traditional and other deep learning approaches in the literature.
IEE Irish Signals and Systems Conference 2005, 2005
An accelerator for linear algebra kernel functions is described, in particular the sparse-matrix ... more An accelerator for linear algebra kernel functions is described, in particular the sparse-matrix by vector multiplication (SMVM) computation which is at the heart of all finite element calculations. The proposed architecture seeks to accelerate the key performance-limiting SMVM operation at the heart of these applications through a combination of a dedicated datapath optimized for these applications, in particular by incorporating direct support for symmetric matrices, and SMVM (sparse-matrix by vector multiplication) to dot-product (DDOT) chaining. System simulations performed using a cycle-accurate C++ architectural model and a database of over 100 large symmetric matrices demonstrate that a 20% average FLOPs/cycle performance improvement can be achieved for the symmetric case.
The required precision for attitude determination in spacecraft is increasing, providing a need f... more The required precision for attitude determination in spacecraft is increasing, providing a need for more accurate attitude determination sensors. The star sensor or star tracker provides unmatched arc-second precision and with the rise of micro satellites these sensors are becoming smaller, faster and more efficient. The most critical component in the star sensor system is the lost-in-space star identification algorithm which identifies stars in a scene without a priori attitude information. In this paper, we present an efficient lost-in-space star identification algorithm using a neural network and a robust and novel feature extraction method. Since a neural network implicitly stores the patterns associated with a guide star, a database lookup is eliminated from the matching process. The search time is therefore not influenced by the number of patterns stored in the network, making it constant (O(1)). This search time is unrivalled by other star identification algorithms. The prese...
The lost-in-space star identification algorithm is able to identify stars without a priori attitu... more The lost-in-space star identification algorithm is able to identify stars without a priori attitude information and is arguably the most critical component of a star sensor system. In this paper, the 2009 survey by Spratling and Mortari is extended and recent lost-in-space star identification algorithms are surveyed. The covered literature is a qualitative representation of the current research in the field. A taxonomy of these algorithms based on their feature extraction method is defined. Furthermore, we show that in current literature the comparison of these algorithms can produce inconsistent conclusions. In order to mitigate these inconsistencies, this paper lists the considerations related to the relative performance evaluation of these algorithms using simulation.
Computer vision and deep learning are clearly demonstrating a capability to create engaging cogni... more Computer vision and deep learning are clearly demonstrating a capability to create engaging cognitive applications and services. However, these applications have been mostly confined to powerful Graphic Processing Units (GPUs) or the cloud due to their demanding computational requirements. Cloud processing has obvious bandwidth, energy consumption and privacy issues. The Eyes of Things (EoT) is a powerful and versatile embedded computer vision platform which allows the user to develop artificial vision and deep learning applications that analyse images locally. In this article, we use the deep learning capabilities of an EoT device for a real-life facial informatics application: a doll capable of recognizing emotions, using deep learning techniques, and acting accordingly. The main impact and significance of the presented application is in showing that a toy can now do advanced processing locally, without the need of further computation in the cloud, thus reducing latency and removi...
2016 International Conference on Image, Vision and Computing (ICIVC), 2016
Commonly, HoG/SVM classifier uses rectangular images for HoG feature descriptor extraction and tr... more Commonly, HoG/SVM classifier uses rectangular images for HoG feature descriptor extraction and training. This means that significant additional work has to be done to process irrelevant pixels belonging to the background surrounding the object of interest. Moreover, some areas of the foreground also can be eliminated from the processing to improve the algorithm speed and memory wise. In Boundary-Bitmap HoG approach proposed in this paper, the boundary of irregular shape of the object is represented by a bitmap to avoid processing of extra background and (partially) foreground pixels. Bitmap, derived from the training dataset, encodes those portions of an image to be used to train a classifier. Experimental results show that not only the proposed algorithm decreases the workload associated with HoG/SVM classifiers by 92.5% compared to the state-of-the-art, but also it shows an average increase about 6% in recall and a decrease about 3% in precision in comparison with standard HoG.
Presents a conference poster that addresses precision refinement for media processors for system-... more Presents a conference poster that addresses precision refinement for media processors for system-on-chip systems.
This article consists of a collection of slides from the author's conference presentation on ... more This article consists of a collection of slides from the author's conference presentation on the special features, system design, processing capabilities, and targeted markets for Movidius's 1TOPS/W, a software programmable media processor.
2016 First International Workshop on Sensing, Processing and Learning for Intelligent Machines (SPLINE), 2016
Following the success of Convolutional Neural Networks (CNNs) on object recognition using 2D imag... more Following the success of Convolutional Neural Networks (CNNs) on object recognition using 2D images, they are extended in this paper to process 3D data. Nearly most of current systems require huge amount of computation for dealing with large amount of data. In this paper, an efficient 3D volumetric object representation, Volumetric Accelerator (VOLA), is presented which requires much less memory than the normal volumetric representations. Based on this, a few 3D digit datasets using 2D MNIST and 2D digit fonts with different rotations along the x, y, and z axis are introduced. Finally, we introduce a combination of multiple CNN models based on the famous LeNet model. The trained CNN models based on the generated dataset have achieved the average accuracy of 90.30% and 81.85% for 3D-MNIST and 3D-Fonts datasets, respectively. Furthermore, experimental results show that VOLA-based CNNs perform 1.5x faster than the original LeNet.
Uploads
Papers by david moloney