We usually think of the physical space as being embedded in a three-dimensional Euclidean space w... more We usually think of the physical space as being embedded in a three-dimensional Euclidean space where measurements of lengths and angles do make sense. It turns out that for artificial systems, such as robots, this is not a mandatory viewpoint and that it is sometimes sufficient to think of the physical space as being embedded in an affine or even projective space. The question then arises of how to relate these geometric models to image measurements and to geometric properties of sets of cameras. We first consider that the world is modelled as a projective space and determine how projective invariant information can be recovered from the images and used in applications. Next we consider that the world is an affine space and determine how affine invariant information can be recovered from the images and used in applications. Finally, we do not move to the Euclidean layer because this is the layer where everybody else has been working with from the early days on, but rather to an intermediate level between the affine and Euclidean ones. For each of the three layers we explain various calibration procedures, from fully automatic ones to ones that use some a priori information. The calibration increases in difficulty from the projective to the Euclidean layer at the same time as the information that can be recovered from the images becomes more and more specific and detailed. The two main applications that we consider are the detection of obstacles and the navigation of a robot vehicle.
This paper discusses the problem of predicting image features in an image from image features in ... more This paper discusses the problem of predicting image features in an image from image features in two other images and the epipolar geometry between the three images. We adopt the most general camera model of perpective projection and show that a point can be predicted in the third image as a bilinear function of its images in the first two cameras, that the tangents to three corresponding curves are related by a trilinear function, and that the curvature of a curve in the third image is a linear function of the curvatures at the corresponding points in the other two images. We thus answer completely the following question: given two views of an object, what would a third view look like? We show that in the special case of orthographic projection our results for points reduce to those of Unman and Basri [19]. We demonstrate on synthetic as well as on real data the applicability of our theory.
Assuming that we only know the epipolar geometry of a pair of stereo images, encoded in the so-ca... more Assuming that we only know the epipolar geometry of a pair of stereo images, encoded in the so-called fundamental matrix, we show that some useful and intuitive three-dimensional information, such as relative positions of points and planes and 3D convex hulls, can be computed in the images without performing any three-dimensional reconstruction. We introduce the notion of visibility, which allows deriving those properties. Results on real data are shown.
In this paper we present two new approaches to planetary rover perception. One approach concerns ... more In this paper we present two new approaches to planetary rover perception. One approach concerns stereo driving without 3-D reconstruction. This approach begins with weakly calibrated stereo images, and evaluates the traversability of terrain using shape indicators such as relative slope and relative elevation. The approach then evaluates candidate paths based on the traversability analysis and generates the best path. The second approach involves estimating vehicle position by observing the Sun. At a given time, a measurement of the Sun's altitude constrains the observer to lie on a circle on the terrestrial surface called the circle of equal altitude. We determine the position of the observer by intersecting circles of equal altitude identified at different times. We are validating experimentally both approaches in unstructured, outdoor environments with several wheeled rovers, Future efforts will transfer the developed technology into Lunar Rover demonstration and flight prog...
Le cadre general de ce travail est le developpement de systemes de vision a plusieurs cameras per... more Le cadre general de ce travail est le developpement de systemes de vision a plusieurs cameras permettant a un robot mobile, par vision stereoscopique, de percevoir son environnement tridimensionnel. Suivant une approche unifiee fondee sur l'utilisation de la geometrie projective, nous abordons tour-a-tour plusieurs problemes intervenant a differents stades du processus de vision stereoscopique. Nous presentons tout d'abord une methode de calibration permettant de determiner, par minimisation d'un critere lie aux niveaux de gris de l'image, tous les parametres de projection d'une camera. Puis nous menons une etude geometrique des relations entre les objets ponctuels ou courbes de l'espace et leurs projections sur une, deux ou trois cameras lorsque celles-ci sont calibrees (calibration forte) ou que seule la geometrie epipolaire est connue (calibration faible). En particulier, nous montrons qu'a partir d'appariements de points dans deux images faiblement calibrees, il est possible d'extraire des informations tridimensionnelles importantes. L'utilisation des relations geometriques nous permet d'elaborer deux algorithmes de mise en correspondance. Le premier apparie des courbes representant les contours extraits de trois images faiblement calibrees. Le second reconstruit des surfaces tridimensionnelles a partir d'un ensemble d'au moins deux cameras fortement calibrees. Enfin, nous presentons des experiences realisees sur un systeme robotique mobile dote de vision stereoscopique. Les applications potentielles de ces algorithmes sont tres importantes, notamment par le fait que la calibration faible d'un systeme de cameras peut etre realisee a partir d'images de l'environnement (interieur ou exterieur), et constitue donc un pas important vers l'autonomie des systemes robotiques
Proceedings of 12th International Conference on Pattern Recognition
We usually think of the physical space as being embedded in a three-dimensional Euclidean space w... more We usually think of the physical space as being embedded in a three-dimensional Euclidean space where measurements of lengths and angles do make sense. It turns out that for artificial systems, such as robots, this is not a mandatory viewpoint and that it is sometimes sufficient to think of the physical space as being embedded in an affine or even projective space. The question then arises of how to relate these geometric models to image measurements and to geometric properties of sets of cameras. We first consider that the world is modelled as a projective space and determine how projective invariant information can be recovered from the images and used in applications. Next we consider that the world is an affine space and determine how affine invariant information can be recovered from the images and used in applications. Finally, we do not move to the Euclidean layer because this is the layer where everybody else has been working with from the early days on, but rather to an intermediate level between the affine and Euclidean ones. For each of the three layers we explain various calibration procedures, from fully automatic ones to ones that use some a priori information. The calibration increases in difficulty from the projective to the Euclidean layer at the same time as the information that can be recovered from the images becomes more and more specific and detailed. The two main applications that we consider are the detection of obstacles and the navigation of a robot vehicle.
In this paper we present two new approaches to plane- tary rover perception. One approach concern... more In this paper we present two new approaches to plane- tary rover perception. One approach concerns stereo driv- ing without 3-D reconstruction. This approach begins with weakly calibrated stereo images, and evaluates the traversability of terrain using shape indicators such as rel- ative slope and relative elevation. The approach then eval- uates candidate paths based on the traversability analysis and generates the best path.
Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition
An edge-based trinocular stereovision algorithm is presented. The primitives it works on are cubi... more An edge-based trinocular stereovision algorithm is presented. The primitives it works on are cubic B-spline approximations of the 2-D edges. This allows one to deal conveniently with curvature and to extend to some nonpolyhedral scenes to previous stereo algorithms. To build a matching primitive, the principle of the algorithm is, first, to find a triplet of corresponding points on three
We present a variational approach to dense stereo reconstruction which combines powerful tools su... more We present a variational approach to dense stereo reconstruction which combines powerful tools such as regularization and multi-scale processing to estimate directly depth from a number of stereo images, while preserving depth discontinuities. The problem is set as a regularization and minimization of a nonquadratic functional. The Tikhonov quadratic regularization term usually used to recover smooth solution is replaced by a function of the gradient depth specifically derived to allow depth discontinuities formation in the solution. Conditions to be fulfilled by this specific regularizing term to preserve discontinuities are also presented. To solve this problem in the discrete case, a PDE-based explicit scheme for moving iteratively towards the solution has been developed. This approach presents the additional advantages of not introducing any intermediate representation such as disparity or rectified images: depth is computed directly from the grey-level images and we can also deal with any number (greater than two) of cameras. Promising experimental results illustrate the capabilities of this approach.
Proceedings of IEEE International Conference on Computer Vision
Abstract In this paper we present a vision system for autonomous navigation based on stereo perce... more Abstract In this paper we present a vision system for autonomous navigation based on stereo perception without 3-D recon-struction. This approach uses weakly calibrated stereo im-ages, ie, images for which only the epipolar geometry is known. The vision system first ...
Automatic Extraction of Man-Made Objects from Aerial and Space Images, 1995
In this paper, we address the problem of the recovery of the Euclidean geometry of a scene from a... more In this paper, we address the problem of the recovery of the Euclidean geometry of a scene from a sequence of images without any prior knowledge either about the parameters of the cameras, or about the motion of the camera(s). We do not require any knowledge of the absolute coordinates of some control points in the scene to achieve this goal. Using various computer vision tools, we establish correspondences between images and recover the epipolar geometry of the set of images, from which we show how to compute the complete set of perspective projection matrices for each camera position. These being known, we proceed to reconstruct the scene. This reconstruction is defined up to an unknown projective transformation (i.e. is parameterized with 15 arbitrary parameters). Next we show how to go from this reconstruction to a more constrained class of reconstructions, defined up to an unknown affine transformation (i.e. parameterized with 12 arbitrary parameters) by exploiting known geometric relations between features in the scene such as parallelism. Finally, we show how to go from this reconstruction to another class, defined up to an unknown similitude (i.e. parameterized with 7 arbitrary parameters). This means that in an Euclidean frame attached to the scene or to one of the cameras, the reconstruction depends only upon one parameter, the global scale. This parameter is easily fixed as soon as one absolute length measurement is known. We see this vision system as a building block, a vision server, of a CAD system that is used by a human to model a scene for such applications as simulation, virtual or augmented reality. We believe that such a system can save a lot of tedious work to the human observer as well as play a leading role in keeping the geometric data base accurate and coherent.
In this paper, we investigate algorithms for evaluating surface orientation from pairs of stereo ... more In this paper, we investigate algorithms for evaluating surface orientation from pairs of stereo images using limited calibration information, and without reconstructing an explicit metric representation of the observed scene. We describe and compare two approaches based on the property that a given orientation at a point in space defines a homography between the two images. The first approach uses a parameterization of image deformation, and only requires knowledge of the epipolar geometry. The other one uses a more robust 3D parameterization, but requires approximate knowledge of some other calibration parameters. Finally, we introduce a probabilistic model that accounts for the degradation of the slope estimates as the points get further away from the cameras, and allows to compute the probability that the surface orientation at any given point is within some angular tolerance from the orientation of a reference plane. We show results on real images and investigate a potential'application to robot motion planning.
Proceedings of 12th International Conference on Pattern Recognition
Addresses the well-known problem of estimating the motion and structure of a plane, but in the ca... more Addresses the well-known problem of estimating the motion and structure of a plane, but in the case where the visual system is not calibrated and in an unbounded monocular image sequence. The authors first define plane collineations and analyse some of their properties when utilized to analyse the retinal motion in an uncalibrated image sequence. The authors show how to relate them to the Euclidian parameters of the scene. In particular, the authors discuss how to detect and estimate the collineation of the plane at infinity and use this quantity for auto-calibration
We usually think of the physical space as being embedded in a three-dimensional Euclidean space w... more We usually think of the physical space as being embedded in a three-dimensional Euclidean space where measurements of lengths and angles do make sense. It turns out that for artificial systems, such as robots, this is not a mandatory viewpoint and that it is sometimes sufficient to think of the physical space as being embedded in an affine or even projective space. The question then arises of how to relate these geometric models to image measurements and to geometric properties of sets of cameras. We first consider that the world is modelled as a projective space and determine how projective invariant information can be recovered from the images and used in applications. Next we consider that the world is an affine space and determine how affine invariant information can be recovered from the images and used in applications. Finally, we do not move to the Euclidean layer because this is the layer where everybody else has been working with from the early days on, but rather to an intermediate level between the affine and Euclidean ones. For each of the three layers we explain various calibration procedures, from fully automatic ones to ones that use some a priori information. The calibration increases in difficulty from the projective to the Euclidean layer at the same time as the information that can be recovered from the images becomes more and more specific and detailed. The two main applications that we consider are the detection of obstacles and the navigation of a robot vehicle.
This paper discusses the problem of predicting image features in an image from image features in ... more This paper discusses the problem of predicting image features in an image from image features in two other images and the epipolar geometry between the three images. We adopt the most general camera model of perpective projection and show that a point can be predicted in the third image as a bilinear function of its images in the first two cameras, that the tangents to three corresponding curves are related by a trilinear function, and that the curvature of a curve in the third image is a linear function of the curvatures at the corresponding points in the other two images. We thus answer completely the following question: given two views of an object, what would a third view look like? We show that in the special case of orthographic projection our results for points reduce to those of Unman and Basri [19]. We demonstrate on synthetic as well as on real data the applicability of our theory.
Assuming that we only know the epipolar geometry of a pair of stereo images, encoded in the so-ca... more Assuming that we only know the epipolar geometry of a pair of stereo images, encoded in the so-called fundamental matrix, we show that some useful and intuitive three-dimensional information, such as relative positions of points and planes and 3D convex hulls, can be computed in the images without performing any three-dimensional reconstruction. We introduce the notion of visibility, which allows deriving those properties. Results on real data are shown.
In this paper we present two new approaches to planetary rover perception. One approach concerns ... more In this paper we present two new approaches to planetary rover perception. One approach concerns stereo driving without 3-D reconstruction. This approach begins with weakly calibrated stereo images, and evaluates the traversability of terrain using shape indicators such as relative slope and relative elevation. The approach then evaluates candidate paths based on the traversability analysis and generates the best path. The second approach involves estimating vehicle position by observing the Sun. At a given time, a measurement of the Sun's altitude constrains the observer to lie on a circle on the terrestrial surface called the circle of equal altitude. We determine the position of the observer by intersecting circles of equal altitude identified at different times. We are validating experimentally both approaches in unstructured, outdoor environments with several wheeled rovers, Future efforts will transfer the developed technology into Lunar Rover demonstration and flight prog...
Le cadre general de ce travail est le developpement de systemes de vision a plusieurs cameras per... more Le cadre general de ce travail est le developpement de systemes de vision a plusieurs cameras permettant a un robot mobile, par vision stereoscopique, de percevoir son environnement tridimensionnel. Suivant une approche unifiee fondee sur l'utilisation de la geometrie projective, nous abordons tour-a-tour plusieurs problemes intervenant a differents stades du processus de vision stereoscopique. Nous presentons tout d'abord une methode de calibration permettant de determiner, par minimisation d'un critere lie aux niveaux de gris de l'image, tous les parametres de projection d'une camera. Puis nous menons une etude geometrique des relations entre les objets ponctuels ou courbes de l'espace et leurs projections sur une, deux ou trois cameras lorsque celles-ci sont calibrees (calibration forte) ou que seule la geometrie epipolaire est connue (calibration faible). En particulier, nous montrons qu'a partir d'appariements de points dans deux images faiblement calibrees, il est possible d'extraire des informations tridimensionnelles importantes. L'utilisation des relations geometriques nous permet d'elaborer deux algorithmes de mise en correspondance. Le premier apparie des courbes representant les contours extraits de trois images faiblement calibrees. Le second reconstruit des surfaces tridimensionnelles a partir d'un ensemble d'au moins deux cameras fortement calibrees. Enfin, nous presentons des experiences realisees sur un systeme robotique mobile dote de vision stereoscopique. Les applications potentielles de ces algorithmes sont tres importantes, notamment par le fait que la calibration faible d'un systeme de cameras peut etre realisee a partir d'images de l'environnement (interieur ou exterieur), et constitue donc un pas important vers l'autonomie des systemes robotiques
Proceedings of 12th International Conference on Pattern Recognition
We usually think of the physical space as being embedded in a three-dimensional Euclidean space w... more We usually think of the physical space as being embedded in a three-dimensional Euclidean space where measurements of lengths and angles do make sense. It turns out that for artificial systems, such as robots, this is not a mandatory viewpoint and that it is sometimes sufficient to think of the physical space as being embedded in an affine or even projective space. The question then arises of how to relate these geometric models to image measurements and to geometric properties of sets of cameras. We first consider that the world is modelled as a projective space and determine how projective invariant information can be recovered from the images and used in applications. Next we consider that the world is an affine space and determine how affine invariant information can be recovered from the images and used in applications. Finally, we do not move to the Euclidean layer because this is the layer where everybody else has been working with from the early days on, but rather to an intermediate level between the affine and Euclidean ones. For each of the three layers we explain various calibration procedures, from fully automatic ones to ones that use some a priori information. The calibration increases in difficulty from the projective to the Euclidean layer at the same time as the information that can be recovered from the images becomes more and more specific and detailed. The two main applications that we consider are the detection of obstacles and the navigation of a robot vehicle.
In this paper we present two new approaches to plane- tary rover perception. One approach concern... more In this paper we present two new approaches to plane- tary rover perception. One approach concerns stereo driv- ing without 3-D reconstruction. This approach begins with weakly calibrated stereo images, and evaluates the traversability of terrain using shape indicators such as rel- ative slope and relative elevation. The approach then eval- uates candidate paths based on the traversability analysis and generates the best path.
Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition
An edge-based trinocular stereovision algorithm is presented. The primitives it works on are cubi... more An edge-based trinocular stereovision algorithm is presented. The primitives it works on are cubic B-spline approximations of the 2-D edges. This allows one to deal conveniently with curvature and to extend to some nonpolyhedral scenes to previous stereo algorithms. To build a matching primitive, the principle of the algorithm is, first, to find a triplet of corresponding points on three
We present a variational approach to dense stereo reconstruction which combines powerful tools su... more We present a variational approach to dense stereo reconstruction which combines powerful tools such as regularization and multi-scale processing to estimate directly depth from a number of stereo images, while preserving depth discontinuities. The problem is set as a regularization and minimization of a nonquadratic functional. The Tikhonov quadratic regularization term usually used to recover smooth solution is replaced by a function of the gradient depth specifically derived to allow depth discontinuities formation in the solution. Conditions to be fulfilled by this specific regularizing term to preserve discontinuities are also presented. To solve this problem in the discrete case, a PDE-based explicit scheme for moving iteratively towards the solution has been developed. This approach presents the additional advantages of not introducing any intermediate representation such as disparity or rectified images: depth is computed directly from the grey-level images and we can also deal with any number (greater than two) of cameras. Promising experimental results illustrate the capabilities of this approach.
Proceedings of IEEE International Conference on Computer Vision
Abstract In this paper we present a vision system for autonomous navigation based on stereo perce... more Abstract In this paper we present a vision system for autonomous navigation based on stereo perception without 3-D recon-struction. This approach uses weakly calibrated stereo im-ages, ie, images for which only the epipolar geometry is known. The vision system first ...
Automatic Extraction of Man-Made Objects from Aerial and Space Images, 1995
In this paper, we address the problem of the recovery of the Euclidean geometry of a scene from a... more In this paper, we address the problem of the recovery of the Euclidean geometry of a scene from a sequence of images without any prior knowledge either about the parameters of the cameras, or about the motion of the camera(s). We do not require any knowledge of the absolute coordinates of some control points in the scene to achieve this goal. Using various computer vision tools, we establish correspondences between images and recover the epipolar geometry of the set of images, from which we show how to compute the complete set of perspective projection matrices for each camera position. These being known, we proceed to reconstruct the scene. This reconstruction is defined up to an unknown projective transformation (i.e. is parameterized with 15 arbitrary parameters). Next we show how to go from this reconstruction to a more constrained class of reconstructions, defined up to an unknown affine transformation (i.e. parameterized with 12 arbitrary parameters) by exploiting known geometric relations between features in the scene such as parallelism. Finally, we show how to go from this reconstruction to another class, defined up to an unknown similitude (i.e. parameterized with 7 arbitrary parameters). This means that in an Euclidean frame attached to the scene or to one of the cameras, the reconstruction depends only upon one parameter, the global scale. This parameter is easily fixed as soon as one absolute length measurement is known. We see this vision system as a building block, a vision server, of a CAD system that is used by a human to model a scene for such applications as simulation, virtual or augmented reality. We believe that such a system can save a lot of tedious work to the human observer as well as play a leading role in keeping the geometric data base accurate and coherent.
In this paper, we investigate algorithms for evaluating surface orientation from pairs of stereo ... more In this paper, we investigate algorithms for evaluating surface orientation from pairs of stereo images using limited calibration information, and without reconstructing an explicit metric representation of the observed scene. We describe and compare two approaches based on the property that a given orientation at a point in space defines a homography between the two images. The first approach uses a parameterization of image deformation, and only requires knowledge of the epipolar geometry. The other one uses a more robust 3D parameterization, but requires approximate knowledge of some other calibration parameters. Finally, we introduce a probabilistic model that accounts for the degradation of the slope estimates as the points get further away from the cameras, and allows to compute the probability that the surface orientation at any given point is within some angular tolerance from the orientation of a reference plane. We show results on real images and investigate a potential'application to robot motion planning.
Proceedings of 12th International Conference on Pattern Recognition
Addresses the well-known problem of estimating the motion and structure of a plane, but in the ca... more Addresses the well-known problem of estimating the motion and structure of a plane, but in the case where the visual system is not calibrated and in an unbounded monocular image sequence. The authors first define plane collineations and analyse some of their properties when utilized to analyse the retinal motion in an uncalibrated image sequence. The authors show how to relate them to the Euclidian parameters of the scene. In particular, the authors discuss how to detect and estimate the collineation of the plane at infinity and use this quantity for auto-calibration
Uploads
Papers by Luc Robert