Figure 1: Jigsaw is a big data management software developed by Double Negative. It allows artist... more Figure 1: Jigsaw is a big data management software developed by Double Negative. It allows artists to efficiently organize and process the vast amount of heterogeneous data captured on a movie set for digital post production.
Accurate online estimation of the structure of the environment together with the pose of the robo... more Accurate online estimation of the structure of the environment together with the pose of the robot is an important component to enable autonomous robotic applications. This paper analyses the different parameterisations used in structure from motion (SFM) problem in the context of accuracy and efficiency of the on-line solutions. Three point parameterisations are compared: Euclidean, inverse depth and inverse distance. At the same time two representations , global and local point coordinates are tested. Different metrics are used to compare the results, camera localisation errors, re-projection errors, execution time as well as a complete analysis on how different parameter-isations affect the convergence, system's condition number and the incremental solving are provided. The paper shows that, with the correct parameterisation, efficient globally consistent SFM is possible, which under the assumption of small, bounded number of correspondences performs in constant time in open loop.
In this paper we present a novel data structure, the Bayes tree, which exploits the connections b... more In this paper we present a novel data structure, the Bayes tree, which exploits the connections between graphical model inference and sparse linear algebra. The proposed data structure provides a new perspective on an entire class of simultaneous localization and mapping (SLAM) algorithms. Similar to a junction tree, a Bayes tree encodes a factored probability density, but unlike the junction tree it is directed and maps more naturally to the square root information matrix of the SLAM problem. This makes it eminently suited to encode the sparse nature of the problem, especially in a smoothing and mapping (SAM) context. The inherent sparsity of SAM has already been exploited in the literature to produce efficient solutions in both batch and online mapping. The graphical model perspective allows us to develop a novel incremental algorithm that seamlessly incorporates reordering and relinearization. This obviates the need for expensive periodic batch operations from previous approaches, which negatively affect the performance and detract from the intended online nature of the algorithm. The new method is evaluated using simulated and real-world datasets in both landmark and pose SLAM settings.
This paper shows results on outdoor vision-based loop closing for Simultaneous Localization and M... more This paper shows results on outdoor vision-based loop closing for Simultaneous Localization and Maping. Our experiments show that for loops of over 50m, the pose estimates maintained with a Delayed-State Extended Information Filter are consistent enough to guarantee assertion of visionbased pose constraints for loop closure, provided no necessary information links are added to the estimator. The technique computes relative pose constraints via a robust least squares minimisation of 3D point correspondences, which are in turn obtained from the matching of SIFT features over candidate image pairs. We propose a loop closure test that checks both for closeness of means and for highly informative updates at the same time.
The objective of this paper is to present a downsized prototype of an Autonomous Underwater Vehic... more The objective of this paper is to present a downsized prototype of an Autonomous Underwater Vehicle (AUV) designed with the idea of dealing with cooperative tasks involving multiple agents. The entire system has been designed for marine and underwater environmental surveillance and protection. The agents provide an interesting point in the surveillance of shallow coastal waters and shallow waters up
This paper presents an efficient approach to outdoor visually augmented odometry. The technique c... more This paper presents an efficient approach to outdoor visually augmented odometry. The technique computes relative pose constraints via a robust least squares minimisation of 3D point correspondences, which are in turn obtained from the matching of SIFT features over two consecutive image pairs. Pose constraints are then used to build a history of pose estimates with and incremental delayed-state information filter. The efficiency of the approach resides on the exact sparseness of the delayed-state information form used.
The purpose of this work is to analyse what happens to the surface texture information when seen ... more The purpose of this work is to analyse what happens to the surface texture information when seen from different distances. 4-source Colour Photometric Stereo, which provides the surface shape and colour information, is used to predict how a surface looks like when changing the distance of the camera. Fifteen real sets of images captured at three different distances A, B,
2007 Ieee Rsj International Conference on Intelligent Robots and Systems, Oct 1, 2007
This paper shows results on outdoor vision-based loop closing for Simultaneous Localization and M... more This paper shows results on outdoor vision-based loop closing for Simultaneous Localization and Mapping. Our experiments show that for loops of over 50m, the pose estimates maintained with a Delayed-State Extended Information Filter are consistent enough to guarantee assertion of visionbased pose constraints for loop closure, provided no necessary information links are added to the estimator. The technique computes relative pose constraints via a robust least squares minimization of 3D point correspondences, which are in turn obtained from the matching of SIFT features over candidate image pairs. We propose a loop closure test that checks both for closeness of means and for highly informative updates at the same time.
Proceedings of the 2009 Ieee Rsj International Conference on Intelligent Robots and Systems, Oct 10, 2009
This paper introduces an approach that reduces the size of the state and maximizes the sparsity o... more This paper introduces an approach that reduces the size of the state and maximizes the sparsity of the information matrix in exactly sparse delayed-state SLAM. We propose constant time procedures to measure the distance between a given pair of poses, the mutual information gain for a given candidate link, and the joint marginals required for both measures. Using these measures, we can readily identify non redundant poses and highly informative links and use only those to augment and to update the state, respectively. The result is a delayed-state SLAM system that reduces both the use of memory and the execution time and that delays filter inconsistency by reducing the number of linearization introduced when adding new loop closure links. We evaluate the advantage of the proposed approach using simulations and data sets collected with real robots.
—Efficiently solving nonlinear least squares (NLS) problems is crucial for many applications in r... more —Efficiently solving nonlinear least squares (NLS) problems is crucial for many applications in robotics. In online applications, solving the associated nolinear systems every step may become very expensive. This paper introduces online, incremental solutions, which take full advantage of the sparse-block structure of the problems in robotics. In general, the solution of the nonlinear system is approximated by incrementally solving a series of linearized problems. The most computationally demanding part is to assemble and solve the linearized system at each iteration. In our solution, this is mitigated by incrementally updating the factorized form of the linear system and changing the linearization point only if needed. The incremental updates are done using a resumed factorization only on the parts affected by the new information added to the system at every step. The sparsity of the factorized form directly affects the efficiency. In order to obtain an incremental factorization with persistent reduced fill-in, a new incremental ordering scheme is proposed. Furthermore, the implementation exploits the block structure of the problems and offers efficient solutions to manipulate block matrices, including a highly efficient Cholesky factorization on sparse block matrices. In this work, we focus our efforts on testing the method on SLAM applications, but the applicability of the technique remains general. The experimental results show that our implementation outperforms the state of the art SLAM implementations on all the tested datasets.
The computational bottleneck in all informationbased algorithms for SLAM is the recovery of the s... more The computational bottleneck in all informationbased algorithms for SLAM is the recovery of the state mean and covariance. The mean is needed to evaluate model Jacobians and the covariance is needed to generate data association hypotheses. Recovering the state mean and covariance requires the inversion of a matrix of the size of the state. Current state recovery methods use sparse linear algebra tools that have quadratic cost, either in memory or in time. In this paper, we present an approach to state estimation that is worst case linear both in execution time and in memory footprint at loop closure, and constant otherwise. The approach relies on a state representation that combines the Kalman and the information-based state representations. The strategy is valid for any SLAM system that maintains constraints between robot poses at different time slices. This includes both Pose SLAM, the variant of SLAM where only the robot trajectory is estimated, and hierarchical techniques in whi...
Realistic 3D models of the environment are beneficial in many fields, from natural or man-made st... more Realistic 3D models of the environment are beneficial in many fields, from natural or man-made structure inspection and volumetric analysis, to movie-making, in particular, special effects integration to natural scenes. Spherical cameras are becoming popular in environment modelling because they capture the full surrounding scene visible from the camera location as a consistent seamless image at once. In this paper, we propose a novel pipeline to obtain fast and accurate 3D reconstructions from spherical images. In order to have a better estimation of the structure, the system integrates a joint camera pose and structure refinement step. This strategy proves to be much faster, yet equally accurate, when compared to the conventional method, registration of a dense point cloud via iterative closest point (ICP). Both methods require an initial estimate for successful convergence. The initial positions of the 3D points are obtained from stereo processing of pair of spherical images with known baseline. The initial positions of the cameras are obtained from a robust wide-baseline matching procedure. The performance and accuracy of the 3D reconstruction pipeline is analysed through extensive tests on several indoor and outdoor datasets.
— Many estimation problems in robotics rely on efficiently solving nonlinear least squares (NLS).... more — Many estimation problems in robotics rely on efficiently solving nonlinear least squares (NLS). For example, it is well known that the simultaneous localisation and mapping (SLAM) problem can be formulated as a maximum likelihood estimation (MLE) and solved using NLS, yielding a mean state vector. However, for many applications recovering only the mean vector is not enough. Data association, active decisions, next best view, are only few of the applications that require fast state covariance recovery. The problem is not simple since, in general, the covariance is obtained by inverting the system matrix and the result is dense. The main contribution of this paper is a novel algorithm for fast incremental covariance update, complemented by a highly efficient implementation of the covariance recovery. This combination yields to two orders of magnitude reduction in computation time, compared to the other state of the art solutions. The proposed algorithm is applicable to any NLS solver implementation, and does not depend on incremental strategies described in our previous papers, which are not a subject of this paper.
Figure 1: Jigsaw is a big data management software developed by Double Negative. It allows artist... more Figure 1: Jigsaw is a big data management software developed by Double Negative. It allows artists to efficiently organize and process the vast amount of heterogeneous data captured on a movie set for digital post production.
Accurate online estimation of the structure of the environment together with the pose of the robo... more Accurate online estimation of the structure of the environment together with the pose of the robot is an important component to enable autonomous robotic applications. This paper analyses the different parameterisations used in structure from motion (SFM) problem in the context of accuracy and efficiency of the on-line solutions. Three point parameterisations are compared: Euclidean, inverse depth and inverse distance. At the same time two representations , global and local point coordinates are tested. Different metrics are used to compare the results, camera localisation errors, re-projection errors, execution time as well as a complete analysis on how different parameter-isations affect the convergence, system's condition number and the incremental solving are provided. The paper shows that, with the correct parameterisation, efficient globally consistent SFM is possible, which under the assumption of small, bounded number of correspondences performs in constant time in open loop.
In this paper we present a novel data structure, the Bayes tree, which exploits the connections b... more In this paper we present a novel data structure, the Bayes tree, which exploits the connections between graphical model inference and sparse linear algebra. The proposed data structure provides a new perspective on an entire class of simultaneous localization and mapping (SLAM) algorithms. Similar to a junction tree, a Bayes tree encodes a factored probability density, but unlike the junction tree it is directed and maps more naturally to the square root information matrix of the SLAM problem. This makes it eminently suited to encode the sparse nature of the problem, especially in a smoothing and mapping (SAM) context. The inherent sparsity of SAM has already been exploited in the literature to produce efficient solutions in both batch and online mapping. The graphical model perspective allows us to develop a novel incremental algorithm that seamlessly incorporates reordering and relinearization. This obviates the need for expensive periodic batch operations from previous approaches, which negatively affect the performance and detract from the intended online nature of the algorithm. The new method is evaluated using simulated and real-world datasets in both landmark and pose SLAM settings.
This paper shows results on outdoor vision-based loop closing for Simultaneous Localization and M... more This paper shows results on outdoor vision-based loop closing for Simultaneous Localization and Maping. Our experiments show that for loops of over 50m, the pose estimates maintained with a Delayed-State Extended Information Filter are consistent enough to guarantee assertion of visionbased pose constraints for loop closure, provided no necessary information links are added to the estimator. The technique computes relative pose constraints via a robust least squares minimisation of 3D point correspondences, which are in turn obtained from the matching of SIFT features over candidate image pairs. We propose a loop closure test that checks both for closeness of means and for highly informative updates at the same time.
The objective of this paper is to present a downsized prototype of an Autonomous Underwater Vehic... more The objective of this paper is to present a downsized prototype of an Autonomous Underwater Vehicle (AUV) designed with the idea of dealing with cooperative tasks involving multiple agents. The entire system has been designed for marine and underwater environmental surveillance and protection. The agents provide an interesting point in the surveillance of shallow coastal waters and shallow waters up
This paper presents an efficient approach to outdoor visually augmented odometry. The technique c... more This paper presents an efficient approach to outdoor visually augmented odometry. The technique computes relative pose constraints via a robust least squares minimisation of 3D point correspondences, which are in turn obtained from the matching of SIFT features over two consecutive image pairs. Pose constraints are then used to build a history of pose estimates with and incremental delayed-state information filter. The efficiency of the approach resides on the exact sparseness of the delayed-state information form used.
The purpose of this work is to analyse what happens to the surface texture information when seen ... more The purpose of this work is to analyse what happens to the surface texture information when seen from different distances. 4-source Colour Photometric Stereo, which provides the surface shape and colour information, is used to predict how a surface looks like when changing the distance of the camera. Fifteen real sets of images captured at three different distances A, B,
2007 Ieee Rsj International Conference on Intelligent Robots and Systems, Oct 1, 2007
This paper shows results on outdoor vision-based loop closing for Simultaneous Localization and M... more This paper shows results on outdoor vision-based loop closing for Simultaneous Localization and Mapping. Our experiments show that for loops of over 50m, the pose estimates maintained with a Delayed-State Extended Information Filter are consistent enough to guarantee assertion of visionbased pose constraints for loop closure, provided no necessary information links are added to the estimator. The technique computes relative pose constraints via a robust least squares minimization of 3D point correspondences, which are in turn obtained from the matching of SIFT features over candidate image pairs. We propose a loop closure test that checks both for closeness of means and for highly informative updates at the same time.
Proceedings of the 2009 Ieee Rsj International Conference on Intelligent Robots and Systems, Oct 10, 2009
This paper introduces an approach that reduces the size of the state and maximizes the sparsity o... more This paper introduces an approach that reduces the size of the state and maximizes the sparsity of the information matrix in exactly sparse delayed-state SLAM. We propose constant time procedures to measure the distance between a given pair of poses, the mutual information gain for a given candidate link, and the joint marginals required for both measures. Using these measures, we can readily identify non redundant poses and highly informative links and use only those to augment and to update the state, respectively. The result is a delayed-state SLAM system that reduces both the use of memory and the execution time and that delays filter inconsistency by reducing the number of linearization introduced when adding new loop closure links. We evaluate the advantage of the proposed approach using simulations and data sets collected with real robots.
—Efficiently solving nonlinear least squares (NLS) problems is crucial for many applications in r... more —Efficiently solving nonlinear least squares (NLS) problems is crucial for many applications in robotics. In online applications, solving the associated nolinear systems every step may become very expensive. This paper introduces online, incremental solutions, which take full advantage of the sparse-block structure of the problems in robotics. In general, the solution of the nonlinear system is approximated by incrementally solving a series of linearized problems. The most computationally demanding part is to assemble and solve the linearized system at each iteration. In our solution, this is mitigated by incrementally updating the factorized form of the linear system and changing the linearization point only if needed. The incremental updates are done using a resumed factorization only on the parts affected by the new information added to the system at every step. The sparsity of the factorized form directly affects the efficiency. In order to obtain an incremental factorization with persistent reduced fill-in, a new incremental ordering scheme is proposed. Furthermore, the implementation exploits the block structure of the problems and offers efficient solutions to manipulate block matrices, including a highly efficient Cholesky factorization on sparse block matrices. In this work, we focus our efforts on testing the method on SLAM applications, but the applicability of the technique remains general. The experimental results show that our implementation outperforms the state of the art SLAM implementations on all the tested datasets.
The computational bottleneck in all informationbased algorithms for SLAM is the recovery of the s... more The computational bottleneck in all informationbased algorithms for SLAM is the recovery of the state mean and covariance. The mean is needed to evaluate model Jacobians and the covariance is needed to generate data association hypotheses. Recovering the state mean and covariance requires the inversion of a matrix of the size of the state. Current state recovery methods use sparse linear algebra tools that have quadratic cost, either in memory or in time. In this paper, we present an approach to state estimation that is worst case linear both in execution time and in memory footprint at loop closure, and constant otherwise. The approach relies on a state representation that combines the Kalman and the information-based state representations. The strategy is valid for any SLAM system that maintains constraints between robot poses at different time slices. This includes both Pose SLAM, the variant of SLAM where only the robot trajectory is estimated, and hierarchical techniques in whi...
Realistic 3D models of the environment are beneficial in many fields, from natural or man-made st... more Realistic 3D models of the environment are beneficial in many fields, from natural or man-made structure inspection and volumetric analysis, to movie-making, in particular, special effects integration to natural scenes. Spherical cameras are becoming popular in environment modelling because they capture the full surrounding scene visible from the camera location as a consistent seamless image at once. In this paper, we propose a novel pipeline to obtain fast and accurate 3D reconstructions from spherical images. In order to have a better estimation of the structure, the system integrates a joint camera pose and structure refinement step. This strategy proves to be much faster, yet equally accurate, when compared to the conventional method, registration of a dense point cloud via iterative closest point (ICP). Both methods require an initial estimate for successful convergence. The initial positions of the 3D points are obtained from stereo processing of pair of spherical images with known baseline. The initial positions of the cameras are obtained from a robust wide-baseline matching procedure. The performance and accuracy of the 3D reconstruction pipeline is analysed through extensive tests on several indoor and outdoor datasets.
— Many estimation problems in robotics rely on efficiently solving nonlinear least squares (NLS).... more — Many estimation problems in robotics rely on efficiently solving nonlinear least squares (NLS). For example, it is well known that the simultaneous localisation and mapping (SLAM) problem can be formulated as a maximum likelihood estimation (MLE) and solved using NLS, yielding a mean state vector. However, for many applications recovering only the mean vector is not enough. Data association, active decisions, next best view, are only few of the applications that require fast state covariance recovery. The problem is not simple since, in general, the covariance is obtained by inverting the system matrix and the result is dense. The main contribution of this paper is a novel algorithm for fast incremental covariance update, complemented by a highly efficient implementation of the covariance recovery. This combination yields to two orders of magnitude reduction in computation time, compared to the other state of the art solutions. The proposed algorithm is applicable to any NLS solver implementation, and does not depend on incremental strategies described in our previous papers, which are not a subject of this paper.
Uploads
Papers by Viorela Ila