3D Image Processing

Autonomous robots have long been a goal of scientists and engineers everywhere and as more advances are being made in the field it becomes clear that a major part of that autonomy will be the vision which the robots can employ to discover their environment. In the field of robotic vision one of the most important operations is segmentation, i.e the separation of parts of the images which represent objects from the overall image, in our case point clouds, these images can be then classified or be used in a number of other 3D vision applications the robot needs to apply to it as to function autonomously. This paper presents one possible implementation of the segmentation process, using the Robot Operating System (ROS) software platform for integration with the robot and the Point Cloud Library (PCL) for the image processing algorithm.

The problem of recursively approximating motion resulting from the Optical Flow (OF) in video thru Total Least Squares (TLS) techniques is addressed. TLS method solves an inconsistent system Gu=z , with G and z in error due to temporal/spatial derivatives, and nonlinearity, while the Ordinary Least Squares (OLS) model has noise only in z. Sources of difficulty involve the non-stationarity of the field, the ill-posedness, and the existence of noise in the data. Three ways of applying the TLS with different noise conjectures to the end problem are observed. First, the classical TLS (cTLS) is introduced, where the entries of the error matrices of each row of the augmented matrix [G;z] have zero mean and the same standard deviation. Next, the Generalized Total Least Squares (GTLS) is defined to provide a more stable solution, but it still has some problems. The Generalized Scaled TLS (GSTLS) has G and z tainted by different sources of additive zero-mean Gaussian noise and scaling [G;z] by nonsingular D and E, that is, D[G;z]E makes the errors iid with zero mean and a diagonal covariance matrix. The scaling is computed from some knowledge on the error distribution to improve the GTLS estimate. For moderate levels of additive noise, GSTLS outperforms the OLS, and the GTLS approaches. Although any TLS variant requires more computations than the OLS, it is still applicable with proper scaling of the data matrix.

Facial expression image analysis can either be in the form of static image analysis or dynamic temporal 3D image or video analysis. The former involves static images taken on an individual at a specific point in time and is in 2-dimensional format. The latter involves dynamic textures extraction of video sequences extended in a temporal domain. Dynamic texture analysis involves short term facial expression movements in 3D in a temporal or spatial domain. Two feature extraction algorithms are used in 3D facial expression analysis namely holistic and local algorithms. Holistic algorithms analyze the whole face whilst the local algorithms analyze a facial image in small components namely nose, mouth, cheek and forehead. The paper uses a popular local feature extraction algorithm called LBP-TOP, dynamic image features based on video sequences in a temporal domain. Volume Local Binary Patterns combine texture, motion and appearance. VLBP and LBP-TOP out-performed other approaches by including local facial feature extraction algorithms which are resistant to gray-scale modifications and computation. It is also crucial to note that these emotions being natural reactions , recognition of feature selection and edge detection from the video sequences can increase accuracy and reduce the error rate. This can be achieved by removing unimportant information from the facial images. The results showed better percentage recognition rate by using local facial extraction algorithms like local binary patterns and local directional patterns than holistic algorithms like GLCM and Linear Discriminant Analysis. The study proposes local binary pattern variant LBP-TOP, local directional patterns and support vector machines aided by genetic algorithms for feature selection. The study was based on Facial Expressions and Emotions (FEED) and CK+ image sources.

Based on my experience with 3d laser scanning technology and IMAX 3d movie production, I was asked to give the keynote speech to this conference outlining my thoughts on the impact of these impressive new technologies on the traditional museum goer.

This paper focuses on manners of filling the gaps in existing standards
that are used in tridimensionalWeb technologies.We proposed the way of encoding huge 3D data sets in lossless PNG format and use of programmable rendering pipeline to decoding PNG file. It allows to reduce significantly the file with 3D data, time of transmission via Web and time needed to decode the file.

IET indexes its books and journal in SCOPUS and IEEE Xplore.
Computer Vision (CV) and Sensors play a decisive role in the operation of Unmanned Aerial Vehicle (UAV), but there exists a void when it comes to analysing the extent of their impact on the entire UAV system. In general, the fact that a UAV is a Cyber-Physical System (CPS) is not taken into account. In this proposal, we propose to expand on earlier books covering the use of CV and sensing in UAVs. Among other things, an entirely autonomous UAV can help to (i) obtain information about the environment, (ii) work for an extended period of time without human interference, (iii) move either all or part of itself all over its operating location devoid of human help and (iv) stay away from dangerous situations for people and their possessions. A Vision System (VS) entails the way CV data will be utilized, the appropriate architecture for total avionics integration, the control interfaces, and the UAV operation. Since the VS core is its sensors and cameras, multi-sensor fusion, navigation, hazard detection, and ground correlation in real time are important operational aspects that can benefit from CV knowledge and technology. This book will aim to collect and shed some light on the existing information on CV software and hardware for UAVs as well as pinpoint aspects that need additional thinking. It will list standards and a set of prerequisites (or lack of them thereof) when it comes to CV deployment in UAVs. The issue of data fusion takes a centre place when the book explores ways to deal with sensor data and images as well as their integration and display. The best practices to fuse image and sensor information to enhance UAV performance by means of CV can greatly improve all aspects of the corresponding CPS. The CPS viewpoint can improve the way UAVs interact with the Internet of Things (IoT), use cloud computing, meet communications requirements, implement hardware/software paradigms necessary to handle video streaming, incorporate satellite data, and combine CV with Virtual/Augmented Realities.

VOLUME 2-DEPLOYMENT AND APPLICATIONS: This tome introduces procedures, standards, and prerequisites for the deployment of Computer Vision (CV) in UAVs from their application point of view. It discusses existing/desirable open source software tools, image banks, benchmarks, Quality of Experience (QoE), Quality of Service (QoS) and how CV can benefit from a Robot Operating System (ROS) in surveillance, remote sensing, inspection, maintenance and repair among other usages, while offering an assessment of current bottlenecks and trends. It will pave the road towards better studies on the necessity and viability of implementing collaborative environments for visualization, knowledge management and teleoperation of UAVs. This is planned to be the companion volume of Estrela, Hemanth, Saotome (Eds) / Imaging and Sensing for Unmanned Aerial Vehicles: Volume 1-Control and Performance.

Editor(s):
Dr. Vania V. Estrela, https://www.linkedin.com/in/vania-v-estrela-96b9bb29/
Universidade Federal Fluminense (UFF), RJ, Brazil
[email protected]

Dr. Jude Hemanth,https://www.karunya.edu/ece/drjude.html
Karunya University, Coimbatore, India
[email protected]

Dr. Osamu Saotome, https://www.linkedin.com/in/osamu-saotome-83935818
Instituto Tecnológico de Aeronáutica, CTA-ITA-IEEA, São José dos Campos, SP, Brazil
[email protected]

CONTENTS:
1. Image Acquisition and Restoration in UAVs
2. Image Fusion in UAVs
3. Super-Resolution Imaging in UAVs
4. 2D/3D/4D Imaging in UAVs
5. Multi-view Image and ToF Sensor Fusion in UAVs
6. Range Imaging in UAVs
7. Multispectral and Hyperspectral Imaging in UAVs
8. Imaging Standards and UAVs
9. Virtual/Augmented Reality in UAVs
10. Collaborative Environments in UAVs
11. Archiving, Storage, and Compression in UAVs
12. Analysis, Indexing, Retrieval in UAVs
13. Multicast/Broadcast/Streaming in UAVs
14. Modelling, Simulation and UAVs
15.  Image-Oriented Estimation and Identification in UAVs
16. Open Source Software in UAVs
17. Image Banks and Benchmarks in UAVs
18. Quality of Experience (QoE) and Quality of Service (QoS) in UAVs
19. Robot Operating System (ROS) in UAVs
20. Cloud Computing in UAVs

Specification and Schedule

July 1st, 2017: Call for Chapter Abstracts
September 1, 2017: One-Page Chapter Abstract (up to 1000 words) Submission Deadline. Free style.
          A proposal must outline one of topics fromthe list above (mention its number, for instance 1. and reference PBCE120B).
November 30, 2017: Last Day for Notification of Acceptance
Jan 30, 2018: Full Chapter Submissions
March 30 2018: Review Chapter Submissions and send comments to authors
May 31, 2018: Receive revised Chapter Submissions
June 30, 2018: Notification of Final Acceptance
July 31, 2018: Gather all material, figure files and copyrights permission forms
Aug 30, 2018: Book editors to finalize introduction and conclusion chapters
Sept 15th, 2018, Delivery of full manuscript to the IET
Scheduled publication: Feb/March 2019

Readership: Graduate students and Researchers in the fields of Electrical and Computer Engineering, Computer Science, Mechanical Engineering, Civil Engineering, Humanitarian Engineering, Control Systems, Geoscience and Remote Sensing, Instrumentation and Measurement, Intelligent Transportation Systems, Oceanic Engineering, Safety Engineering, Reliability, Robotics and Automation, Signal Processing, Technology and Engineering Management, Environmental Engineering, Public Health Management, Non-Invasive Testing/Monitoring and Vehicular Technology.

Additional Information: Dr. Vania V. Estrela,    [email protected]
                                          Dr. Jude Hemanth,      [email protected]

This paper focuses on manners of filling the gaps in existing standards that are used in tridimensional Web technologies. We proposed the way of encoding huge 3D data sets in lossless PNG format and use of programmable rendering pipeline to decoding PNG file. It allows to reduce significantly the file with 3D data, time of transmission via Web and time needed to decode the file.

Abstract/// These last three decades, the integration of digital technology in the field of architectural practice, and particularly in the study of building heritage, generated major upheavals. Through a historical perspective of architectural survey, this research discusses the cognitive dimension intrinsic to its practice and to the technological challenges it faces. On that basis, it appears that the digitizations coming from the application of the latest acquisition methods (photogrammetry & lasergrammetry) express high completeness (visual and metric) with the artifacts surveyed, but also produce significant data volumes. This "information overload" does not reinforces the architectural representation in its role of knowledge vehicle. Facing this lack of intelligibility, the purpose of this research is to provide an epistemological answer considering the need of intelligibility without loosing the value of digitizations. This research offers an original low level (non-interpretative) approach, where the meaning of elements ("semantic" and "geometric" structure) comes from the morphological similarities observed on a corpus of digitized shapes. The statistical analysis and the use of different morphological descriptors help to formalize characteristic signatures of the corpus; they correspond to the formal expression of a determined number of geometric features. This approach provides a methodological solution for "big data" processing. Based on accumulation of data, it offers a complementary analytical response to "traditional" high-level approaches (interpretative) where the shape is characterized with pre-structured knowledge specific to the architectural field. If one of the challenges is to apply geometric analytical methods on "big data", the other one is to confront, through the comparative study, high and low level observations. The purpose is to refine the understanding of the stylistic propagations in time and in space. To experiment this approach, 31 columns of the cloister of the Abbey of Saint-Michel-de-Cuxa have undergone a thorough morphological analysis.
Keywords: Architectural representation, survey, digital, architectural heritage, shape analysis, semantic description.
///
Résumé/// L'arrivée du numérique dans le champ des pratiques architecturales et particulièrement dans celui de l'étude et de la documentation du fait bâti a engendré ces trois dernières décennies d'importants bouleversements. A travers une mise en perspective historique et une définition étendue du relevé d'architecture, cette recherche vise à discuter et mettre en tension la dimension cognitive propre à sa pratique avec les enjeux technologiques auxquels elle fait face. Sur cette base, un constat s'impose : les empreintes numériques issues de l'application des méthodes d'acquisition les plus récentes (photogrammétrie & lasergrammétrie) expriment une forte exhaustivité (visuelle et métrique) avec les artefacts relevés mais produisent également d'importants volumes de données. Cette "surcharge d'informations" ne permet pas ou peu de renforcer la représentation architecturale dans son rôle de véhicule de connaissances. Face à ce déficit d'intelligibilité, l'ambition de cette thèse est de fournir une réponse épistémologique permettant de considérer ce besoin d'intelligibilité sans tourner le dos à la richesse des numérisations produites. Nous proposons ici une approche originale de bas niveau (non-interprétative), où la signification des éléments (structure "sémantique" et "géométrique") provient de l'étude des similarités morphologiques observées au sein d'un corpus de formes. L'analyse statistique et le recours à différents descripteurs morphologiques ont permis de formaliser des signatures caractéristiques du corpus étudié ; il s'agit ni plus ni moins de l'expression formelle d'un nombre déterminé d'attributs géométriques. De manière plus profonde, notre démarche fournit une solution méthodologique pour le traitement de données massives. En s'appuyant sur le cumul de données, ce travail offre une réponse analytique complémentaire aux approches "traditionnelles" de haut niveau (interprétatives) où la caractérisation de la forme s'appuie sur les connaissances pré-structurées du domaine. Si l'un des enjeux consiste à appliquer les méthodes d'analyse géométrique aux données massives, l'autre est de confronter, par l'étude comparative, les observations de haut et bas niveaux. A travers cette recherche, notre souhait est d'affiner la compréhension des propagations stylistiques dans le temps et dans l'espace. Afin d'expérimenter cette approche, 31 colonnes du cloître de l'Abbaye de Saint-Michel-de-Cuxa ont fait l'objet d'une analyse morphologique approfondie. Mots-clés : Représentation architecturale, relevé, numérique, patrimoine architectural, analyse de formes, description sémantique.

A low-cost depth camera recently introduced is synchronized with a specially devised low-cost motorized turntable. This results in a low-cost motorized depth sensor, able to provide a large number of registered side views, and which is exploited here for the quantitative characterization of the shoots of entire plants. A set of four new shape de-scriptors of the shoots is proposed, which are constructed from the depth images on multiple side views of the shoots of plants. The four descriptors quantify effective volume, multiscale organization, spatial symmetries, and lacunarity of the plants. The four descriptors are here defined, validated on synthetic scenes with known properties, and then applied on nine different-looking real plants to illustrate their abilities for quantitative characterization and comparison. The resulting motorized depth sensor and associated image processing open new perspectives to various plant science applications including plant growth and architecture monitoring , plant response to stresses or the assessment of aesthetic parameters for ornamental plants.

Major secondary structure elements such as α helices and β sheets can be computationally detected from cryoelectron microscopy (cryo-EM) density maps with medium resolutions of 5–10 Å. However, a critical piece of information for modeling atomic structures is missing, because there are no tools to detect β strands from cryo-EM maps at medium resolutions. We propose a method, StrandTwister, to detect the traces of β strands through the analysis of twist, an intrinsic nature of a β sheet. StrandTwister has been tested using 100 β sheets simulated at 10 Å resolution and 39 β sheets computationally detected from cryo-EM density maps at 4.4–7.4 Å resolutions. Although experimentally derived cryo-EM maps contain errors, StrandTwister’s best detections over 39 cases were able to detect 81.87% of the β strands, with an overall 1.66 Å two-way distance between the detected and observed β traces. StrandTwister appears to detect the traces of β strands on major β sheets quite accurately, particularly at the central area of a β sheet.

A new algorithm for deriving skeletons and segmentations from point cloud data in
O
(
n
) time is explained in this publication.
This skeleton is represented as a graph, which can be embedded into the point cloud. The CAMPINO method, (C)ollapsing (A)nd
(M)erging (P)rocedures (IN) (O)ctree-graphs, is based on cycle elimination in a graph as derived from an octree based space
division procedure. The algorithm is able to extract the skeleton from point clouds generated from either one or multiple
viewpoints. The correspondence between the vertices of the graph and the original points of the point cloud is used to derive an
initial segmentation of these points. The principle of the algorithm is demonstrated on a synthetic point cloud consisting of 3
connected tori. Initially this algorithm was developed to obtain skeletons from point clouds representing natural trees, measured
with the terrestrial laser scanner IMAGER 5003 of Zoller+Fröhlich. The results show that CAMPINO is able to automatically
derive realistic skeletons that fit the original point cloud well and are suited as a basis for e.g. further automatic feature extraction or
skeleton-based registration.

Low cost stereoscopic virtual reality hardware interfacing with nearly any computer and stereoscopic software running on any PC is described. Both are user configurable for serial or parallel ports. Stereo modeling, rendering, and interaction via gloves or 6D mice are provided. Low cost LCD Visors and external interfaces represent a breakthrough in convenience and price/performance.

Soil cores from a field growing barley and barley mutants without root hairs under conventional and minimum tillage were sampled. They were X-ray scanned to produce a 3D image and then the roots were washed out and weight and length were determined by conventional means. Root volume and surface area were then calculated from the 3D images using state of the art software and methodology, and the measured and calculated measures were correlated. The only strong and significant correlation was between measured weight and calculated volume for mutants without root hairs. It is concluded that the software cannot segment out very small roots, but segmentation accuracy also depends on root structure in some unknown way. Any study using X-ray computed tomography to quantify roots as they grow in situ should start with a calibration for the conditions in question.

Recent advances in electron cryo-tomography (cryoET) allows thin samples such as macromolecular complexes and small bacterial cells to be imaged. In order to reconstruct a three-dimensional image, computation is needed to combine 2-dimensional images taken from a number of different viewing orientations. In this paper, we focus on a special situation in which the biological interest is in a local region of an object that has a protrusion. We propose an idea to cut computationally in the region of interest at the protrusion. We propose a localized crop-cuboid model whose location is predictable in each 2D projection image once the viewing orientation of the projection is known. A weighting procedure was implemented in the 3D reconstruction based on the protruding feature and the projection orientation. Our simulation demonstrates that the localized cropping and reconstruction can resolve the protrusion features similarly to the global reconstruction, in spite of less information is used in the reconstruction.