Stereo Matching

In this paper, we present a general guideline to establish the relation between a distribution model and its corresponding similarity estimation. A rich set of distance metrics, such as harmonic distance and geometric distance, is derived according to Maximum Likelihood theory. These metrics can provide a more accurate feature model than the conventional Euclidean distance (SSD) and Manhattan distance (SAD). Because the feature elements are from heterogeneous sources and may have different influence on similarity estimation, the assumption of single isotropic distribution model is often inappropriate. We propose a novel boosted distance metric that not only finds the best distance metric that fits the distribution of the underlying elements but also selects the most important feature elements with respect to similarity. We experiment with different distance metrics for similarity estimation and compute the accuracy of different methods in two applications: stereo matching and motion tracking in video sequences. The boosted distance metric is tested on fifteen benchmark data sets from the UCI repository and two image retrieval applications. In all the experiments, robust results are obtained based on the proposed methods.

This work aims at defining a new method for matching correspondences in stereoscopic image analysis. A representation of occlusions drives the overall matching process. Based on the taxonomy proposed by Scharstein and Szelinsky (2002, IJCV, 47, 7-42), the dense stereo matching process is divided into three tasks: matching cost computation, aggregation of local evidence and computation of disparity values. Within the second and third phases new strategies are introduced in an attempt to improve the reliability of results. Aggregation is based on a new local matching measure, and neural techniques compute disparities adaptively. Two experimental studies were conducted to evaluate and compare the solutions proposed. The first uses a standard well-known dataset including data with true disparity maps; the second study was conducted on complex real images acquired by a scanning electron microscope (SEM).

Stereo vision is a well-known ranging method because it resembles the basic mechanism of the human eye. However, the computational complexity and large amount of data access make real-time processing of stereo vision challenging because of the inherent instruction cycle delay within conventional computers. In order to solve this problem, the past 20 years of research have focused on the use of dedicated hardware architecture for stereo vision. This paper proposes a fully pipelined stereo vision system providing a dense disparity image with additional sub-pixel accuracy in real-time. The entire stereo vision process, such as rectification, stereo matching, and post-processing, is realized using a single field programmable gate array (FPGA) without the necessity of any external devices. The hardware implementation is more than 230 times faster when compared to a software program operating on a conventional computer, and shows stronger performance over previous hardware-related studies. Index Terms-Field programmable gate arrays, integrated circuit design, stereo vision, video signal processing. I. Introduction S TEREO vision is a traditional method for acquiring 3-D information from a stereo image pair. Stereo vision has many advantages over other 3-D sensing methods in terms of safety, undetectable characteristics, cost, operating range, and reliability [1]. For these reasons, stereo vision is widely used in many application areas including intelligent robots, autonomous vehicles, human-computer interfaces, and security and defense applications [2]-[5].

We propose a new approach for stereo matching in Autonomous Mobile Robot applications. In this framework an accurate but slow reconstruction of the 3D scene is not needed; rather, it is more important to have a fast localization of the obstacles to avoid them. All the methods in the literature are based on a punctual correspondence, but they are inefficient

In this paper, a new window-based method for stereo matching is proposed. Differing with the existing local approaches, our algorithm divides the matching process into two steps, initial matching and disparity calibration. Initial disparity is first approximated by an adaptive support-weight and a rank transform method, and then a compact disparity calibration approach is designed to refine the initial disparity, so an accurate result can be acquired. The experimental results are evaluated on the Middlebury dataset, showing that our method is better than other local methods on standard stereo benchmarks.

A method for solving dense stereo matching problem is presented in this paper. First, a new generalized ground control points (GGCPs) scheme is introduced, where one or more disparity candidates for the true disparity of each pixel are assigned by local matching using the oriented spatial filters. By allowing "all" pixels to have multiple candidates for their true disparities, GGCPs not only guarantee to provide a sufficient number of starting pixels needed for guiding the subsequent matching process, but also remarkably reduce the risk of false match, improving the previous GCP-based approaches where the number of the selected control points tends to be inversely proportional to the reliability. Second, by employing a two-pass dynamic programming technique that performs optimization both along and across the scanlines, we solve the typical inter-scanline inconsistency problem. Moreover, combined with the GGCPs, the stability and efficiency of the optimization are improved significantly. Experimental results for the standard data sets show that the proposed algorithm achieves comparable results to the state-of-the-arts with much less computational cost.

Window-based correlation algorithms are widely used for stereo matching due to their computational efficiency as compared to global algorithms. In this paper, a multiple window correlation algorithm for stereo matching is presented which addresses the problems associated with a fixed window size. The developed algorithm differs from the previous multiple window algorithms by introducing a reliability test to select the most reliable window among multiple windows of increasing sizes. This ensures that at least one window is large enough to cover a region of adequate intensity variations while at the same time small enough to cover a constant depth region. A recursive computation procedure is also used to allow a computationally efficient implementation of the algorithm. The outcome obtained from a standard set of images with known disparity maps shows that the generated disparity maps are more accurate as compared to two popular stereo matching local algorithms. © 2007 SPIE and IS&T.

A snake is an energy-minimizing spline guided by external constraint forces and influenced by image forces that pull it toward features such as lines and edges. Snakes are active contour models: they lock onto nearby edges, localizing them accurately. Scale-space continuation can be used to enlarge the capture region surrounding a feature. Snakes provide a unified account of a number of visual problems, including detection of edges, lines, and subjective contours; motion tracking; and stereo matching. We have used snakes successfully for interactive interpretation, in which user-imposed constraint forces guide the snake near features of interest.

Technological advances and challenges ] M ultiview imaging (MVI) has attracted considerable attention recently due to its increasingly wide range of applications and the decreasing cost of digital cameras. This opens up many new and interesting research topics and applications, such as virtual view synthesis for three-dimensional (3-D) television (3DTV) and entertainment, high-performance imaging, video processing and analysis for surveillance, distance learning, industry inspection, etc.

Why do we have colour? What use is it to us? Some of the obvious answers are that we see colour so that we can recognise objects, spot objects more quickly, tell when fruit is ripe or rotten. These reasons make a lot of sense, but are there others? In this paper, we explore the things that colour makes easier for computational vision systems. In particular, we examine the role of colour in understanding specularities, processing interreflections, identifiying metals from plastics and wet surfaces from dry ones, choosing foveation points, disambiguating stereo matches, discriminating textures and identifying objects. Of course, what is easier for a computational vision system is not necessarily the same for the human visual system but it can perhaps help us create some hypotheses about the role of colour in human perception. We also consider the role of colour constancy in terms of whether or not it is required for colour to be useful to a computer vision system.

Le prospettive architettoniche dipinte negli affreschi delle Ville Venete sono in grado di rivelare informazioni finora inedite, anche sui loro autori e committenti. I saggi qui raccolti, circoscritti alla Riviera del Brenta in provincia di Venezia, mostrano come la scienza della rappresentazione possa contribuire in modo determinante a nuove scoperte sulla storia di questo prezioso patrimonio.

The painted architectural perspectives in the Venetian Villas reveal features, both of the frescoes and of their painters, that can only be discovered through a representational approach. The studies here collected, focused on the Riviera del Brenta in the province of Venice, are thus a starting point.

The stereo matching algorithms for binocular vision are very popular and widely used. Still, the algorithms may have lower matching quality or higher time complexity. Matching can be viewed as a process of finding the degree of correlation between two groups of data. This area belongs to the one of the most explored topics in stereo vision. The depth value can be found by the disparity estimat ion by taking advantage of the similarity in these two views, which can be captured at the same time in a depth generation algorithm. It is not easy to find the accurate depth value if there is high similarity in color sp aces around the area. Stereo matchi ng is a fundamental problem in computer vision that estimates depth of a 3D scene with a pair of images. This paper presents the fundamental steps for finding depth extraction and related terms

Stereo matching is one of the most active research areas in computer vision for decades. The task of stereo matching is to find the point correspondence between two images of the same scene taken from different viewpoints. This paper presents a segment-based stereo matching algorithm. Firstly, the reference image is segmented using hill-climbing algorithm and local stereo matching is performed Scale Invariant Feature Transform (SIFT) feature points with Sum of Absolute Differences (SAD) block matching. Secondly, a set of reliable pixels is constructed by comparing the matching cost and the mutual cross-checking consistent between the left and right initial disparity maps, which can lead to an actual disparity plane. Thirdly, a set of all possible disparity planes are extracted and then plane fitting and neighboring segment merging are performed. Finally, the disparity planes are set in each region using graph cuts to obtain final disparity map. The evaluation of proposed algorithm on the Middlebury data set result shows that the proposed algorithm is competitive with state-of-the-art stereo matching algorithms.

Active contour model has been widely used in image processing applications such as boundary delineation, image segmentation, stereo matching, shape recognition and object tracking. In this paper a novel particle swarm optimization scheme has been introduced to evolve snake over time in a way to reduce time complexity while improving quality of results. Traditional active contour models converge slowly and are prone to local minima due to their complex nature. Various evolutionary techniques including genetic algorithms, particle swarm optimization and predator prey optimization have been successfully employed to tackle this problem. Most of these methods are general problem solvers that, more or less, formulate the snake model equations as a minimization problem and try to optimize it. In contrary, our proposed approach integrates concepts from active contour model into particle swarm optimization so that each particle will represent a snaxel of the active contour. Canonical velocity update equation in particle swarm algorithm is modified to embrace the snake kinematics. This new model makes it possible to have advantages of swarm based searching strategies and active contour principles all together. Aptness of the proposed approach has been examined through several experiments on synthetic and real world images of CT and MRI images of brain and the results demonstrate its promising performance particularly in handling boundary concavities and snake initialization problems.

In this paper, image flow fields from parallel stereo cameras are analyzed to determine the relative 3-D translational motion of the camera platform with respect to objects in view and to establish stereo correspondence of features in the left and right images. A two-step procedure is suggested. In the first step, translational motion parameters are determined from linear equations whose coefficients consist of the sums of measured quantities in the two images. Separate equations are developed for cases when measurements of either the full optical flow or the normal flow are available. This computation does not require feature-to-feature correspondence. In addition, no assumption is made about the surfaces being viewed. In the second step of the calculation, with the knowledge of the estimated translational motion parameters, the binocular flow information is used to find features in one image that correspond to given features in the other image. Experimental results with synthetic and laboratory images indicate that the method gives accurate results even in the presence of noise.

Abstract—We describe a method that allows for accurate in-flight calibration of the interior orientation of any pushbroom camera and that in particular solves the problem of modeling the distortions induced by charge coupled device (CCD) misalign-ments. The distortion induced on the ...

Stereo vision, resulting in the knowledge of deep information in a scene, is of great importance in the field of machine vision, robotics and image analysis. In this article, an explicit analysis of the existing stereo matching methods, up to date, is presented. The presented algorithms are discussed in terms of speed, accuracy, coverage, time consumption, and disparity range. Towards the direction of real-time operation, the development of stereo matching algorithms, suitable for efficient hardware implementation is highly desirable. Implementations of stereo matching algorithms in hardware for real-time applications are also discussed in details.

In this paper, we propose a symmetric stereo model to handle occlusion in dense two-frame stereo. Our occlusion reasoning is directly based on the visibility constraint that is more general than both ordering and uniqueness constraints used in previous work. The visibility constraint requires occlusion in one image and disparity in the other to be consistent. We embed the visibility constraint within an energy minimization framework, resulting in a symmetric stereo model that treats left and right images equally. An iterative optimization algorithm is used to approximate the minimum of the energy using belief propagation. Our stereo model can also incorporate segmentation as a soft constraint. Experimental results on the Middlebury stereo images show that our algorithm is state-of-the-art.

We apply an AM-FM surface albedo model to analyze the projection of surface patterns viewed through a binocular camera system. This is used to support the use of modulation-based stereo matching where local image phase is used to compute stereo disparities. Local image phase is an advantagous feature for image matching, since the problem of computing disparities reduces to identifying local phase shifts between the stereoscopic image data. Local phase shifts, however, are problematic at high frequencies due to phase wrapping when disparities exceed 6. We meld powerful multichannel Gabor image demodulation techniques for multiscale (coarse-to-fine) computation of local image phase with a disparity channel model for depth computation. The resulting framework unifies phasebased matching approaches with AM-FM surface/image models. We demonstrate the concepts in a stereo algorithm that generates a dense, accurate disparity map without the problems associated with phase wrapping.

This paper presents a new local binocular stereo algorithm which takes into consideration plane fitting at the per-pixel level. Two disparity calculation passes are used. The first pass assumes that surfaces in the scene are fronto-parallel and generates an initial disparity map, from which the disparity plane orientations of all pixels are extracted and refined. In the second pass, the cost aggregation for each pixel is conducted along the estimated disparity plane orientations, rather than the fronto-parallel ones. Large window size with adaptive support weights is used to ensure the effectiveness of the slanted surface modeling. The disparity search space is also quantized at sub-pixel level to improve the accuracy of the disparity results. The experimental results demonstrate the validity of our presented approach.

Energy minimization is often the key point of solving problems in computer vision. For decades, many methods have been proposed (deterministic, stochastic,...). Some can only reach local minimum and others strong local minimum close to the optimal solution (global minimum). Since beginning of 21th century, minimization based on Graph theory have been generalized to find global minimum of multi-labeling problems. In this work, we study deterministic local minimization methods (Iterative Conditional Modes and Direct Descent Energy), and a stochastic global minimization with an improved Simulated Annealing algorithm. A new approach formulation to help local minimization to converge to a minimum closed to the global one is proposed. This method combines local and global energy constraints in an multiresolution way. We focus on stereo matching application. The improved Simulated Annealing proved to reach global minimum as good as Graph based minimization methods. Promising results of proposed local minimization methods are obtained on Middlebury Stereo database compare to global methods.

Stereo video techniques are effective for estimating the space-time wave dynamics over an area of the ocean. Indeed, a stereo camera view allows retrieval of both spatial and temporal data whose statistical content is richer than that of time series data retrieved from point wave probes. To prove this, we consider an application of the Wave Acquisition Stereo System (WASS) for the analysis of offshore video measurements of gravity waves in the Northern Adriatic Sea. In particular, we deployed WASS at the oceanographic platform Acqua Alta, off the Venice coast, Italy. Three experimental studies were performed, and the overlapping field of view of the acquired stereo images covered an area of approximately 1100 m 2 . Analysis of the WASS measurements show that the sea surface can be accurately estimated in space and time together, yielding associated directional spectra and wave statistics that agree well with theoretical models. From the observed wavenumber-frequency spectrum one can also predict the vertical profile of the current flow underneath the wave surface. Finally, future improvements of WASS and applications are discussed.

This paper presents a fast and reliable stereo matching algorithm which produces a dense disparity map by using fast cross correlation, rectangular subregioning (RSR) and 3D maximum-surface techniques in a coarse-to-fine scheme. Fast correlation is achieved by using the box-filtering technique whose speed is invariant to the size of the correlation window and by segmenting the stereo images into rectangular subimages at different levels of the pyramid. By working with rectangular subimages, not only can the speed of the correlation be further increased, the intermediate memory storage requirement can also be reduced. The disparity map for the stereo images is found in the 3D correlation coefficient volume by obtaining the global 3D maximum-surface rather than simply choosing the position that gives the local maximum correlation coefficient value for each pixel. The 3D maximum-surface is obtained using our new two-stage dynamic programming (TSDP) technique. There are two original contributions in this paper: (1) development of the RSR technique for fast similarity measure; and (2) development of the TSDP technique for efficiently obtaining 3D maximum-surface in a 3D volume. Typical running time of our algorithm implemented in the C language on a 512 × 512 image is in the order of a few seconds on a 500 MHz PC. A variety of synthetic and real images have been tested, and good results have been obtained.

Automated methods for reliable 3-D reconstruction of man-made objects are essential to many users and providers of 3-D city data. Manual 3-D processing of aerial images is time consuming and requires the expertise of qualified personnel. Therefore, the necessity to interpret, classify and quantitatively process aerial images in a semi- or fully automatic mode and to integra te the results into CAD- or spatial information systems is more urgent than ever. This paper describes current research activities at ETH Zurich in automated 3-D reconstruction of buildings. Two conceptually different approaches to building reconstruction are presented: TOBAGO and ARUBA. The first approach is semi-automatic and aims at the automatic structuring of manually measured 3-D point clouds to generate CAD models of complete buildings. It is geared towards a reliable and flexible procedure which has a large potential to be useful for professional practice. The second approach aims at fully automatic detec...

Indoor and outdoor urban environments posses many regularities which can be efficiently exploited and used for general image parsing tasks. We present a novel approach for detecting rectilinear structures and demonstrate their use for wide baseline stereo matching, planar 3D reconstruction, and computation of geometric context. Assuming a presence of dominant orthogonal vanishing directions, we proceed by formulating the detection of the rectilinear structures as a labeling problem on detected line segments. The line segment labels, respecting the proposed grammar rules, are established as the MAP assignment of the corresponding MRF. The proposed framework allows to detect both full as well as partial rectangles, rectangle-inrectangle structures, and rectangles sharing edges. The use of detected rectangles is demonstrated in the context of difficult wide baseline matching tasks in the presence of repetitive structures and large appearance changes.

This paper presents a novel application of particle swarm optimization (PSO) in combination with another computational intelligence (CI) technique, namely, proximal support vector machine (PSVM) for machinery fault detection. Both real-valued and binary PSO algorithms have been considered along with linear and nonlinear versions of PSVM. The time domain vibration signals of a rotating machine with normal and defective bearings are processed for feature extraction. The features extracted from original and preprocessed signals are used as inputs to the classifiers (PSVM) for detection of machine condition. Input features are selected using a PSO algorithm. The classifiers are trained with a subset of experimental data for known machine conditions and are tested using the remaining data. The procedure is illustrated using the experimental vibration data of a rotating machine. The influences of the number of features, PSO algorithms and type of classifiers (linear or nonlinear PSVM) on the detection success are investigated. Results are compared with a genetic algorithm (GA) and principal component analysis (PCA). The PSO based approach gave test classification success above 90% which were comparable with the GA and much better than PCA. The results show the effectiveness of the selected features and classifiers in detection of machine condition.

I would like to sincerely acknowledge my supervisor Professor John S. Zelek for all the guidance, kindness and support he has provided me throughout this study. I cannot imagine completing my thesis without his insightful supervision and continuous support. I would also like to express my appreciation to my thesis readers, Professor Paul Fieguth and Professor Otman Basir, for providing me with their valuable comments. In addition, I would like to thank the Defense Research and Development Canada (DRDC) for partially supporting this research.

A Digital Surface Model (DSM) generated by automatic stereo matching of convergent image networks includes a great number of 3D-points that come from diverse combinations of stereo-pairs. This points can have a very different accuracy and reliability, in addition to a great and nondesirable spatial redundancy. We analized different methods to generate several synthesis DSM whose points are the result

Applications of Moiré patterns have been reported in stereo matching (disparity measurement) and in developing a binary similarity measure. Matching based on Moiré patterns requires that the images be converted to pseudo random dot representation. Two classes of algorithms for converting images to random dots are presented: (i) high pass filtering, and (ii) bit slicing. Parameters which influence dot density and uniformity of distribution are investigated.

We have analysed the requirements for a robust stereo vision algorithm for use in typical industrial applications. For such applications the views obtained in the two cameras have large differences in visual appearance due to the orientation difference between the two cameras and the close proximity of illumination sources. We have concluded that for this category of problem, feature-based methods should be more robust than conventional, area-based approaches, and this cor,clusion appears to be borne out in the published titerature. However, correlation-based approaches are more suited to efficient implementation on available hardware. The technique which we have developed, called Stretch-Correlation, is based on the cross-correlation of warped image blocks which have been preprocessed to maximize the useful information content. Our new method models the severe warping effects encountered in difficult stereo problems and effectively relaxes the front-o-parallel constraint which is normally imposed in area-based disparity calculation. This algorithm imposes effectively most of the local constraints present in feature-based algorithms, and can be efficiently implemented on available hardware.

ABSTACT: The generation of a digital surface model (DSM) from remotely sensed images is an important task for various applications such as telecommunications, cell planning, the environment monitoring... In general, extracting DSMs consists of: camera modelling, stereo matching and interpolation. However, in the case of satellite imagery and because of the satellite characteristics, it is difficult to apply the algorithms and the model generally used with aerial images. In this paper, we adopted a model which uses all parameters of acquisition: it is the full model. The matching algorithm used takes into account geometry of the image. It is base on a hierarchical approach followed by a relaxation process. The full model of SPOT5 sensors is then used for the calculation of the 3D-co-ordinates on the ground. Results show good quality and accuracy of the obtained DSM compared to ground control points and DSM data of superior quality.