Object discovery and modeling have been widely
studied in the computer vision and robotics commu... more Object discovery and modeling have been widely
studied in the computer vision and robotics communities.
SLAM approaches that make use of objects and higher level
features have also recently been proposed. Using higher level
features provides several benefits: these can be more discriminative,
which helps data association, and can serve to inform
service robotic tasks that require higher level information,
such as object models and poses. We propose an approach
for online object discovery and object modeling, and extend a
SLAM system to utilize these discovered and modeled objects as
landmarks to help localize the robot in an online manner. Such
landmarks are particularly useful for detecting loop closures
in larger maps. In addition to the map, our system outputs
a database of detected object models for use in future SLAM
or service robotic tasks. Experimental results are presented to
demonstrate the approach’s ability to detect and model objects,
as well as to improve SLAM results by detecting loop closures.
Learning spatial models from sensor data raises the challenging data association problem of relat... more Learning spatial models from sensor data raises the challenging data association problem of relating model parameters to individual measurements. This paper proposes an EM-based algorithm, which solves the model learning and the data association problem in parallel. The algorithm is developed in the context of the the structure from motion problem, which is the problem of estimating a 3D scene model from a collection of image data. To accommodate the spatial constraints in this domain, we compute virtual measurements as sufficient statistics to be used in the M-step. We develop an efficient Markov chain Monte Carlo sampling method called chain flipping, to calculate these statistics in the E-step. Experimental results show that we can solve hard data association problems when learning models of 3D scenes, and that we can do so efficiently. We conjecture that this approach can be applied to a broad range of model learning problems from sensor data, such as the robot mapping problem.
We present incremental smoothing and mapping (iSAM), a novel approach to the simultaneous localiz... more We present incremental smoothing and mapping (iSAM), a novel approach to the simultaneous localization and mapping problem that is based on fast incremental matrix factorization. iSAM provides an efficient and exact solution by updating a QR factorization of the naturally sparse smoothing information matrix, therefore recalculating only the matrix entries that actually change. iSAM is efficient even for robot trajectories with many loops as it avoids unnecessary fill-in in the factor matrix by periodic variable reordering. Also, to enable data association in real-time, we provide efficient algorithms to access the estimation uncertainties of interest based on the factored information matrix. We systematically evaluate the different components of iSAM as well as the overall algorithm using various simulated and real-world datasets for both landmark and pose-only settings.
Conference on intelligent transportation systems, Jan 1, 1997
We present a real-time model-based vision approach for detecting and tracking vehicles from a mov... more We present a real-time model-based vision approach for detecting and tracking vehicles from a moving platform. It was developed in the context of the CMU Navlab project and is intended to provide the Navlabs with situational awareness in mixed tra c. Tracking is done by combining a simple image processing technique with a 3D extended Kalman lter and a measurement equation that projects from the 3D model to image space. No ground plane assumption is made. The resulting system runs at frame rate or higher, and produces excellent estimates of road curvature, distance to and relative speed of a tracked vehicle. We h a ve complemented the tracker with a novel machine learning based algorithm for car detection, the CANSS algorithm, which serves to initialize tracking.
Development is an important, powerful and integral element of biological evolution. In this paper... more Development is an important, powerful and integral element of biological evolution. In this paper we present two models of development that can be used to evolve functional autonomous agents, complete with bodies and neural control systems. The first and most complex model is more biologically defensible in its details. It has been used to hand-design a genome for the development of complete agents capable of executing a simple avoidance task. These agents were then incrementally improved through evolution. The second model is simpler and uses a random Boolean network model for the genome and cell state that is somewhat more removed from the biological realm, making it easier to analyze and more amenable to artificial evolution. Using this model, we have successfully evolved complete agents from scratch that are capable of following curved lines.
Black's influential paper on EigenTracking, they were successfully applied in tracking. For noisy... more Black's influential paper on EigenTracking, they were successfully applied in tracking. For noisy targets, optimization-based algorithms (including EigenTracking) often fail catastrophically after losing track. Particle filters have recently emerged as a robust method for tracking in the presence of multi-modal distributions. To use subspace representations in a particle filter, the number of samples increases exponentially as the state vector includes the subspace coefficients. We introduce an efficient method for using subspace representations in a particle filter by applying Rao-Blackwellization to integrate out the subspace coefficients in the state vector. Fewer samples are needed since part of the posterior over the state vector is analytically calculated. We use probabilistic principal component analysis to obtain analytically tractable integrals. We show experimental results in a scenario in which we track a target in clutter.
This note represents my attempt at explaining the EM algorithm . This is just a slight variation ... more This note represents my attempt at explaining the EM algorithm . This is just a slight variation on Tom Minka?s tutorial , perhaps a little easier (or per haps not). It includes a graphical example to provide some intuition.
A method is presented to recover 3D scene structure and camera motion from multiple images withou... more A method is presented to recover 3D scene structure and camera motion from multiple images without the need for correspondence information. The problem is framed as finding the maximum likelihood structure and motion given only the 2D measurements, integrating over all possible assignments of 3D features to 2D measurements. This goal is achieved by means of an algorithm which iteratively refines a probability distribution over the set of all correspondence assignments. At each iteration a new structure from motion problem is solved, using as input a set of 'virtual measurements' derived from this probability distribution. The distribution needed can be efficiently obtained by Markov Chain Monte Carlo sampling. The approach is cast within the framework of Expectation-Maximization, which guarantees convergence to a local maximizer of the likelihood. The algorithm works well in practice, as will be demonstrated using results on several real image sequences.
Artificial Life IV, Proceedings of the Fourth …, Jan 1, 1994
We are interested in the synthesis of autonomous agents using evolutionary techniques. Most work ... more We are interested in the synthesis of autonomous agents using evolutionary techniques. Most work in this area utilizes a direct mapping from genotypic space to phenotypic space. In order to address some of the limitations of this approach, we present a simplified yet biologically defensible model of the developmental process. The design issues that arise when formulating this model at the molecular, cellular and organismal level are discussed, and for each of these issues we describe how they were resolved in our implementation. We present and analyze some of the morphologies that can be explored using this model, specifically one that has agent-like properties. In addition, we demonstrate that this developmental model can be evolved.
The International Journal of Robotics …, Jan 1, 2006
Solving the SLAM (simultaneous localization and mapping) problem is one way to enable a robot to ... more Solving the SLAM (simultaneous localization and mapping) problem is one way to enable a robot to explore, map, and navigate in a previously unknown environment. Smoothing approaches have been investigated as a viable alternative to extended Kalman filter (EKF)based solutions to the problem. In particular, approaches have been looked at that factorize either the associated information matrix or the measurement Jacobian into square root form. Such techniques have several significant advantages over the EKF: they are faster yet exact; they can be used in either batch or incremental mode; are better equipped to deal with non-linear process and measurement models; and yield the entire robot trajectory, at lower cost for a large class of SLAM problems. In addition, in an indirect but dramatic way, column ordering heuristics automatically exploit the locality inherent in the geographic nature of the SLAM problem. This paper presents the theory underlying these methods, along with an interpretation of factorization in terms of the graphical model associated with the SLAM problem. Both simulation results and actual SLAM experiments in large-scale environments are presented that underscore the potential of these methods as an alternative to EKF-based approaches.
We describe a Markov chain Monte Carlo based particle filter that effectively de als with interac... more We describe a Markov chain Monte Carlo based particle filter that effectively de als with interacting targets, i.e., targets that are influenced by the proximity and /or behav-
To navigate reliably in indoor environments, a mobile robot must know where it is. This includes ... more To navigate reliably in indoor environments, a mobile robot must know where it is. This includes both the ability of globally localizing the robot from scratch, as well as tracking the robot's position once its location is known. Vision has long been advertised as providing a solution to these problems, but we still lack efficient solutions in unmodified environments. Many existing approaches require modification of the environment to function properly, and those that work within unmodified environments seldomly address the problem of global localization.
We describe a particle filter that effectively deals with interacting targets -targets that are i... more We describe a particle filter that effectively deals with interacting targets -targets that are influenced by the proximity and/or behavior of other targets. The particle filter includes a Markov random field (MRF) motion prior that helps maintain the identity of targets throughout an interaction, significantly reducing tracker failures. We show that this MRF prior can be easily implemented by including an additional interaction factor in the importance weights of the particle filter. However, the computational requirements of the resulting multi-target filter render it unusable for large numbers of targets. Consequently, we replace the traditional importance sampling step in the particle filter with a novel Markov chain Monte Carlo (MCMC) sampling step to obtain a more efficient MCMC-based multi-target filter.
This paper explores several statistical pattern recognition techniques to classify utterances acc... more This paper explores several statistical pattern recognition techniques to classify utterances according to their emotional content. We have recorded a corpus containing emotional speech with over a 1000 utterances from different speakers. We present a new method of extracting prosodic features from speech, based on a smoothing spline approximation of the pitch contour. To make maximal use of the limited amount of training data available, we introduce a novel pattern recognition technique: majority voting of subspace specialists. Using this technique, we obtain classification performance that is close to human performance on the task.
This paper describes Minerva, an interactive tour-guide robot that was successfully deployed in a... more This paper describes Minerva, an interactive tour-guide robot that was successfully deployed in a Smithsonian museum. Minerva's software is pervasively probabilistic, relying on explicit representations of uncertainty in perception and control. During 2 weeks of operation, the robot interacted with thousands of people, both in the museum and through the Web, traversing more than 44 km at speeds of up to 163 cm/sec in the unmodified museum.
This paper describes an interactive tour-guide robot, which was successfully exhibited in a Smith... more This paper describes an interactive tour-guide robot, which was successfully exhibited in a Smithsonian museum. During its two weeks of operation, the robot interacted with thousands of people, traversing more than 44 km at speeds of up to 163 cm/sec. Our approach specifically addresses issues such as safe navigation in unmodified and dynamic environments, and short-term human-robot interaction. It uses learning pervasively at all levels of the software architecture.
This paper presents a new algorithm for mobile robot localization, called Monte Carlo Localizatio... more This paper presents a new algorithm for mobile robot localization, called Monte Carlo Localization (MCL). MCL is a version of Markov localization, a family of probabilistic approaches that have recently been applied with great practical success. However, previous approaches were either computationally cumbersome (such as grid-based approaches that represent the state space by high-resolution 3D grids), or had to resort to extremely coarse-grained resolutions. Our approach is computationally efficient while retaining the ability to represent (almost) arbitrary distributions. MCL applies sampling-based methods for approximating probability distributions, in a way that places computation "where needed. " The number of samples is adapted on-line, thereby invoking large sample sets only when necessary. Empirical results illustrate that MCL yields improved accuracy while requiring an order of magnitude less computation when compared to previous approaches. It is also much easier to implement.
Object discovery and modeling have been widely
studied in the computer vision and robotics commu... more Object discovery and modeling have been widely
studied in the computer vision and robotics communities.
SLAM approaches that make use of objects and higher level
features have also recently been proposed. Using higher level
features provides several benefits: these can be more discriminative,
which helps data association, and can serve to inform
service robotic tasks that require higher level information,
such as object models and poses. We propose an approach
for online object discovery and object modeling, and extend a
SLAM system to utilize these discovered and modeled objects as
landmarks to help localize the robot in an online manner. Such
landmarks are particularly useful for detecting loop closures
in larger maps. In addition to the map, our system outputs
a database of detected object models for use in future SLAM
or service robotic tasks. Experimental results are presented to
demonstrate the approach’s ability to detect and model objects,
as well as to improve SLAM results by detecting loop closures.
Learning spatial models from sensor data raises the challenging data association problem of relat... more Learning spatial models from sensor data raises the challenging data association problem of relating model parameters to individual measurements. This paper proposes an EM-based algorithm, which solves the model learning and the data association problem in parallel. The algorithm is developed in the context of the the structure from motion problem, which is the problem of estimating a 3D scene model from a collection of image data. To accommodate the spatial constraints in this domain, we compute virtual measurements as sufficient statistics to be used in the M-step. We develop an efficient Markov chain Monte Carlo sampling method called chain flipping, to calculate these statistics in the E-step. Experimental results show that we can solve hard data association problems when learning models of 3D scenes, and that we can do so efficiently. We conjecture that this approach can be applied to a broad range of model learning problems from sensor data, such as the robot mapping problem.
We present incremental smoothing and mapping (iSAM), a novel approach to the simultaneous localiz... more We present incremental smoothing and mapping (iSAM), a novel approach to the simultaneous localization and mapping problem that is based on fast incremental matrix factorization. iSAM provides an efficient and exact solution by updating a QR factorization of the naturally sparse smoothing information matrix, therefore recalculating only the matrix entries that actually change. iSAM is efficient even for robot trajectories with many loops as it avoids unnecessary fill-in in the factor matrix by periodic variable reordering. Also, to enable data association in real-time, we provide efficient algorithms to access the estimation uncertainties of interest based on the factored information matrix. We systematically evaluate the different components of iSAM as well as the overall algorithm using various simulated and real-world datasets for both landmark and pose-only settings.
Conference on intelligent transportation systems, Jan 1, 1997
We present a real-time model-based vision approach for detecting and tracking vehicles from a mov... more We present a real-time model-based vision approach for detecting and tracking vehicles from a moving platform. It was developed in the context of the CMU Navlab project and is intended to provide the Navlabs with situational awareness in mixed tra c. Tracking is done by combining a simple image processing technique with a 3D extended Kalman lter and a measurement equation that projects from the 3D model to image space. No ground plane assumption is made. The resulting system runs at frame rate or higher, and produces excellent estimates of road curvature, distance to and relative speed of a tracked vehicle. We h a ve complemented the tracker with a novel machine learning based algorithm for car detection, the CANSS algorithm, which serves to initialize tracking.
Development is an important, powerful and integral element of biological evolution. In this paper... more Development is an important, powerful and integral element of biological evolution. In this paper we present two models of development that can be used to evolve functional autonomous agents, complete with bodies and neural control systems. The first and most complex model is more biologically defensible in its details. It has been used to hand-design a genome for the development of complete agents capable of executing a simple avoidance task. These agents were then incrementally improved through evolution. The second model is simpler and uses a random Boolean network model for the genome and cell state that is somewhat more removed from the biological realm, making it easier to analyze and more amenable to artificial evolution. Using this model, we have successfully evolved complete agents from scratch that are capable of following curved lines.
Black's influential paper on EigenTracking, they were successfully applied in tracking. For noisy... more Black's influential paper on EigenTracking, they were successfully applied in tracking. For noisy targets, optimization-based algorithms (including EigenTracking) often fail catastrophically after losing track. Particle filters have recently emerged as a robust method for tracking in the presence of multi-modal distributions. To use subspace representations in a particle filter, the number of samples increases exponentially as the state vector includes the subspace coefficients. We introduce an efficient method for using subspace representations in a particle filter by applying Rao-Blackwellization to integrate out the subspace coefficients in the state vector. Fewer samples are needed since part of the posterior over the state vector is analytically calculated. We use probabilistic principal component analysis to obtain analytically tractable integrals. We show experimental results in a scenario in which we track a target in clutter.
This note represents my attempt at explaining the EM algorithm . This is just a slight variation ... more This note represents my attempt at explaining the EM algorithm . This is just a slight variation on Tom Minka?s tutorial , perhaps a little easier (or per haps not). It includes a graphical example to provide some intuition.
A method is presented to recover 3D scene structure and camera motion from multiple images withou... more A method is presented to recover 3D scene structure and camera motion from multiple images without the need for correspondence information. The problem is framed as finding the maximum likelihood structure and motion given only the 2D measurements, integrating over all possible assignments of 3D features to 2D measurements. This goal is achieved by means of an algorithm which iteratively refines a probability distribution over the set of all correspondence assignments. At each iteration a new structure from motion problem is solved, using as input a set of 'virtual measurements' derived from this probability distribution. The distribution needed can be efficiently obtained by Markov Chain Monte Carlo sampling. The approach is cast within the framework of Expectation-Maximization, which guarantees convergence to a local maximizer of the likelihood. The algorithm works well in practice, as will be demonstrated using results on several real image sequences.
Artificial Life IV, Proceedings of the Fourth …, Jan 1, 1994
We are interested in the synthesis of autonomous agents using evolutionary techniques. Most work ... more We are interested in the synthesis of autonomous agents using evolutionary techniques. Most work in this area utilizes a direct mapping from genotypic space to phenotypic space. In order to address some of the limitations of this approach, we present a simplified yet biologically defensible model of the developmental process. The design issues that arise when formulating this model at the molecular, cellular and organismal level are discussed, and for each of these issues we describe how they were resolved in our implementation. We present and analyze some of the morphologies that can be explored using this model, specifically one that has agent-like properties. In addition, we demonstrate that this developmental model can be evolved.
The International Journal of Robotics …, Jan 1, 2006
Solving the SLAM (simultaneous localization and mapping) problem is one way to enable a robot to ... more Solving the SLAM (simultaneous localization and mapping) problem is one way to enable a robot to explore, map, and navigate in a previously unknown environment. Smoothing approaches have been investigated as a viable alternative to extended Kalman filter (EKF)based solutions to the problem. In particular, approaches have been looked at that factorize either the associated information matrix or the measurement Jacobian into square root form. Such techniques have several significant advantages over the EKF: they are faster yet exact; they can be used in either batch or incremental mode; are better equipped to deal with non-linear process and measurement models; and yield the entire robot trajectory, at lower cost for a large class of SLAM problems. In addition, in an indirect but dramatic way, column ordering heuristics automatically exploit the locality inherent in the geographic nature of the SLAM problem. This paper presents the theory underlying these methods, along with an interpretation of factorization in terms of the graphical model associated with the SLAM problem. Both simulation results and actual SLAM experiments in large-scale environments are presented that underscore the potential of these methods as an alternative to EKF-based approaches.
We describe a Markov chain Monte Carlo based particle filter that effectively de als with interac... more We describe a Markov chain Monte Carlo based particle filter that effectively de als with interacting targets, i.e., targets that are influenced by the proximity and /or behav-
To navigate reliably in indoor environments, a mobile robot must know where it is. This includes ... more To navigate reliably in indoor environments, a mobile robot must know where it is. This includes both the ability of globally localizing the robot from scratch, as well as tracking the robot's position once its location is known. Vision has long been advertised as providing a solution to these problems, but we still lack efficient solutions in unmodified environments. Many existing approaches require modification of the environment to function properly, and those that work within unmodified environments seldomly address the problem of global localization.
We describe a particle filter that effectively deals with interacting targets -targets that are i... more We describe a particle filter that effectively deals with interacting targets -targets that are influenced by the proximity and/or behavior of other targets. The particle filter includes a Markov random field (MRF) motion prior that helps maintain the identity of targets throughout an interaction, significantly reducing tracker failures. We show that this MRF prior can be easily implemented by including an additional interaction factor in the importance weights of the particle filter. However, the computational requirements of the resulting multi-target filter render it unusable for large numbers of targets. Consequently, we replace the traditional importance sampling step in the particle filter with a novel Markov chain Monte Carlo (MCMC) sampling step to obtain a more efficient MCMC-based multi-target filter.
This paper explores several statistical pattern recognition techniques to classify utterances acc... more This paper explores several statistical pattern recognition techniques to classify utterances according to their emotional content. We have recorded a corpus containing emotional speech with over a 1000 utterances from different speakers. We present a new method of extracting prosodic features from speech, based on a smoothing spline approximation of the pitch contour. To make maximal use of the limited amount of training data available, we introduce a novel pattern recognition technique: majority voting of subspace specialists. Using this technique, we obtain classification performance that is close to human performance on the task.
This paper describes Minerva, an interactive tour-guide robot that was successfully deployed in a... more This paper describes Minerva, an interactive tour-guide robot that was successfully deployed in a Smithsonian museum. Minerva's software is pervasively probabilistic, relying on explicit representations of uncertainty in perception and control. During 2 weeks of operation, the robot interacted with thousands of people, both in the museum and through the Web, traversing more than 44 km at speeds of up to 163 cm/sec in the unmodified museum.
This paper describes an interactive tour-guide robot, which was successfully exhibited in a Smith... more This paper describes an interactive tour-guide robot, which was successfully exhibited in a Smithsonian museum. During its two weeks of operation, the robot interacted with thousands of people, traversing more than 44 km at speeds of up to 163 cm/sec. Our approach specifically addresses issues such as safe navigation in unmodified and dynamic environments, and short-term human-robot interaction. It uses learning pervasively at all levels of the software architecture.
This paper presents a new algorithm for mobile robot localization, called Monte Carlo Localizatio... more This paper presents a new algorithm for mobile robot localization, called Monte Carlo Localization (MCL). MCL is a version of Markov localization, a family of probabilistic approaches that have recently been applied with great practical success. However, previous approaches were either computationally cumbersome (such as grid-based approaches that represent the state space by high-resolution 3D grids), or had to resort to extremely coarse-grained resolutions. Our approach is computationally efficient while retaining the ability to represent (almost) arbitrary distributions. MCL applies sampling-based methods for approximating probability distributions, in a way that places computation "where needed. " The number of samples is adapted on-line, thereby invoking large sample sets only when necessary. Empirical results illustrate that MCL yields improved accuracy while requiring an order of magnitude less computation when compared to previous approaches. It is also much easier to implement.
Uploads
Papers by Frank Dellaert
studied in the computer vision and robotics communities.
SLAM approaches that make use of objects and higher level
features have also recently been proposed. Using higher level
features provides several benefits: these can be more discriminative,
which helps data association, and can serve to inform
service robotic tasks that require higher level information,
such as object models and poses. We propose an approach
for online object discovery and object modeling, and extend a
SLAM system to utilize these discovered and modeled objects as
landmarks to help localize the robot in an online manner. Such
landmarks are particularly useful for detecting loop closures
in larger maps. In addition to the map, our system outputs
a database of detected object models for use in future SLAM
or service robotic tasks. Experimental results are presented to
demonstrate the approach’s ability to detect and model objects,
as well as to improve SLAM results by detecting loop closures.
studied in the computer vision and robotics communities.
SLAM approaches that make use of objects and higher level
features have also recently been proposed. Using higher level
features provides several benefits: these can be more discriminative,
which helps data association, and can serve to inform
service robotic tasks that require higher level information,
such as object models and poses. We propose an approach
for online object discovery and object modeling, and extend a
SLAM system to utilize these discovered and modeled objects as
landmarks to help localize the robot in an online manner. Such
landmarks are particularly useful for detecting loop closures
in larger maps. In addition to the map, our system outputs
a database of detected object models for use in future SLAM
or service robotic tasks. Experimental results are presented to
demonstrate the approach’s ability to detect and model objects,
as well as to improve SLAM results by detecting loop closures.