Scribd Logo
Upload

Welcome to Scribd!

  • 199 views

    Stereo Vision-Estimation of Disparity: Advanced Computer Vision Course MEC - 301/2

    Uploaded by

    Rekha Jayaprakash
    This document discusses stereo vision and the estimation of disparity through stereo matching. It provides an overview of basic stereo vision concepts like epipolar geometry and disparity maps. It then describes the steps taken in the presented approach, which includes feature extraction of corners, corner matching using PCA, fundamental matrix calculation, image rectification, and disparity map generation using sum of squared differences. The goal is to estimate pixel disparity from stereo image pairs in order to infer 3D scene structure.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as PPTX, PDF, TXT or read online from Scribd

    Stereo Vision-Estimation of Disparity: Advanced Computer Vision Course MEC - 301/2

    Uploaded by

    Rekha Jayaprakash
    199 views29 pages
    This document discusses stereo vision and the estimation of disparity through stereo matching. It provides an overview of basic stereo vision concepts like epipolar geometry and disparity maps. It then describes the steps taken in the presented approach, which includes feature extraction of corners, corner matching using PCA, fundamental matrix calculation, image rectification, and disparity map generation using sum of squared differences. The goal is to estimate pixel disparity from stereo image pairs in order to infer 3D scene structure.

    Original Title

    Stereo Vision

    Copyright

    © Attribution Non-Commercial (BY-NC)

    Available Formats

    PPTX, PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document discusses stereo vision and the estimation of disparity through stereo matching. It provides an overview of basic stereo vision concepts like epipolar geometry and disparity maps. It then describes the steps taken in the presented approach, which includes feature extraction of corners, corner matching using PCA, fundamental matrix calculation, image rectification, and disparity map generation using sum of squared differences. The goal is to estimate pixel disparity from stereo image pairs in order to infer 3D scene structure.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as PPTX, PDF, TXT or read online from Scribd
    Download as pptx, pdf, or txt
    199 views29 pages

    Stereo Vision-Estimation of Disparity: Advanced Computer Vision Course MEC - 301/2

    Uploaded by

    Rekha Jayaprakash
    This document discusses stereo vision and the estimation of disparity through stereo matching. It provides an overview of basic stereo vision concepts like epipolar geometry and disparity maps. It then describes the steps taken in the presented approach, which includes feature extraction of corners, corner matching using PCA, fundamental matrix calculation, image rectification, and disparity map generation using sum of squared differences. The goal is to estimate pixel disparity from stereo image pairs in order to infer 3D scene structure.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as PPTX, PDF, TXT or read online from Scribd
    Download as pptx, pdf, or txt
    of 29

    Stereo Vision- Estimation

    of Disparity
    Advanced Computer Vision
    Course MEC- 301/2

    Guided by,
    Presented by,
    Dr. S. Majumder, Lakshmi Prabha N S
    Head- SR Lab, Aman Arora
    CMERI- Durgapur. Rekha Jayaprakash
    PGRPE’09 Mechatronics
    Overview
    Basic concepts behind stereo vision
    Epipolar geometry and contraints
    Feature extraction
    Feature matching
    Rectification process
    Disparity map
    Experimental results
    Introduction to stereo vision
    Stereo vision refers to the ability to infer information on the
    3-D structure and distance of a scene from two or more
    images taken from different viewpoints.
    Terms in stereo vision:
     Depth of a pixel is the distance between the camera and the object
    containing the pixel.
     Disparity is the shifted distance between two matching pixels in the left and
    right images.
     Disparity Map is the output image of stereo vision. The colour of each pixel
    in an image corresponds to the disparity of the corresponding pixel in the
    input image. It is often a gray-scale image, using brighter colour for smaller
    disparity.
    Challenges
     The first problem is correspondence, consists in determining which item in

    the left image corresponds to which item in the right image.


     The second problem is reconstruction. Our vivid 3-D perception of the

    world is due to the interpretation that the brain gives of the computed
    difference in retinal position, named disparity, between corresponding items.
     The disparity of all the image points forms the so called disparity map,

    which can be displayed as an image. If the geometry of the stereo system is


    known, the disparity map can be converted into a 3-D map of the viewed
    scene (reconstruction).
    Constraints
    The images are taken of the same scene, with the
    cameras next to each other. There are some constraints
    on the stereo images.
    The camera image planes must be the same.
    The cameras must have the same focal length, f.
    The cameras must be vertically aligned.
    A simplified stereo
    imaging system
    Two cameras with their optical axes
    parallel and separated by a distance d.
    The line connecting the camera lens
    centers is called the baseline .
    The focal length of both cameras is f .
    Let the x axis of the 3-D world coordinate
    system be parallel to the baseline .
    Consider a point (x, y, z)in 3-D world
    coordinates on an object.
    Let the point (x, y, z)have image
    coordinates(x’l,y’l) and (x’r,y’r) in the left
    and right image planes of the respective
    cameras
    Our approach for estimating disparity
    map
    Steps:
     Image Acquisition
     Feature Extraction (Corner)
     Feature (Corner) Matching using PCA
     Fundamental Matrix Calculation
     Rectification of stereo pair images
     Generating Disparity Map using SSD Method
    Feature Extraction
    Extracting the features such as edge, corner, region etc.., from the stereo pair images and matching
    them for correspondence is the initial step to get the 3D information from the stereo pair images.
    Corner extraction using KLT corner detector:
     Local structure matrix (tensor)

    The derivatives of intensity function f(x,y) are first calculated at each point using Prewitt derivative
    filter.
     The entries of matrix (f2x ,fxfy, f2y) are obtained.

     Finally each of the entries is smoothened by Gaussian filter WG(r;σ).


    Feature Extraction
     Calculate eigenvalues for window size nxn in Cstr (λ1, λ2) centered at p.
    For a perfect uniform image: Cstr =0 and λ1= λ2=0.
    For a perfect step edge: λ1>0,λ2=0, where eigenvector associated with λ1
    is orthogonal to the edge.
    For a corner .
     If smaller eigen vaule (say λ2 ) is greater than λthr, save the p into a list L.
     Find prominent corner (greater λ2) among neighboring corners within a
    window.
     Eliminate prominent corners overlapping in window size DxD.
     Sort L in decreasing order of λ2, Scan the sorted list from top to bottom. For each
    current point p, delete all points appearing further in the list which belong to the
    neighbourhood of p.
    Original Left Image Original Right Image
    Detected corners (white dots) in left and
    right images respectively
    Corner matching using PCA
    Placing the detected corner points as centre and take a
    block of 50x50 or 100x100 of left image.
    Now this form the training set where size of the
    training set is equal to the number of theTraining
    cornersetpoints
    from left image
    which are not near to border of the left image.

    Image I : (Nx x Ny) pixels

    Training Set

    Mean Face
    1 M
       n
    M n1
    Mean subtracted n  n  
    image
    Difference Matrix

    Covariance MatrixM
    1
    C  n n
    M n1
     T
     AAT

    Y= AT. A size (M x M).


    U ⋅νi = μi ⋅νi
    Α ⋅ ΑT ⋅ Α ⋅νi = μi ⋅Α ⋅νi
    Χ ⋅Α ⋅νi = μi ⋅Α ⋅νi
    ui = Α ⋅νi
    Corner matching using PCA
    M'≤ M (remaining omitted because it contributes less variance)
    Eigenvectors can be considered as the vectors pointing in the direction
    of the maximum variance and the value of the variance the eigenvector
    represents is directly proportional to the value of the eigenvalue
    Projection ωk = ukT⋅Φ = ukT⋅(Γ − Ψ)
    Weight Matrix Ω = [ω1 ω2 ... ωM' ]T
    (Nx x Ny) in image space to (M‘ x 1) in vector space
    Corner matching using PCA
    Given a test image T, obtained by taking the corner
    points in right image as centre and taking 50x50 or
    100x100 block around it. Test set from right image
    Test image vector ΓT
    Mean subtracted image ΦT = ΓT − Ψ
    Projection ωk = ukT⋅(ΓT − Ψ)
    Weight Matrix ΩT = [ω1 ω2 ... ωM' ]T
    Corner matching using PCA
    Euclidian distance εk
    εk2 = ||( Ω - Ωk)||2
    Calculate εmin the minimum value of Euclidian distance.
    Compare εmin with threshold Θ,
    εmin < Θ  Taken as match for the corresponding test
    image.
    Calculation of fundamental matrix
    Using 8-point algorithm
     Let x is a point in left image and x’ is the
    corresponding point in right image, then
    Calculation of fundamental matrix
    Hence above equation is of the form AF=0
    Take SVD(A)
     f = last column of V matrix.
    Calculate F=SM2’ * f * SM1
     [U D V] = SVD(F)
    Make D(3,3) = 0.(enforcing the singularity)
     Calculate F = U * D * V’
    Rectification of images
    Epipolar rectification is an important stage in stereo matching.

    So that the epipolar lines are parallel and horizontal and

    corresponding points have the same vertical coordinates.


    .
    Steps for rectification

     Assume that intrinsic parameters are unknown and that a number of


    corresponding tie-points are available.
     Impose the collineations that make the tie-points satisfy the epipolar
    geometry of a rectified image pair.
     Calculate the width(number of columns) and height (number of rows) of the
    right image.
     Pass x, width, height, ml, mr as input arguments to one function which
    calculates kol, kor, F using the initial values of variables such as 5 rotation
    angles (Y-left, Z-left, X-right, Y-right, Z-right) and the focal length as zeros
    which is contained in x. Rotation around x-axis of left camera is set to zero
    because geometrically, this coincide with rotating a rectified pair around the
    baseline, which do not alter the rectification.
    Steps for rectification
     With the initial approximated
    values form,
     old intrinsic parameters kor, kol
    matrix and rotation matrix Rl &
    Rr
    Rl = eulR([0,yl,zl]);
    Rr = eulR([xr,yr,zr]);
     new intrinsics (arbitrary)
    Knl = (Kol+Kor)/2;
    Knr = Knl;
     compute collineations
    Hr = Knr*Rr*inv(Kor);
    Hl = Knl*Rl*inv(Kol);
    Steps for rectification
     Re-compute the knr and knl after centering the left and right image and making the
    vertical coordinate of the principal point (which is the center pixel in this case) as
    same.
     Again re-compute the value of Hr and Hl and name it as TR and TL.
     Find the smallest bounding box for both images, by taking the corners of left and
    right image and the rectifying transformation matrix TR and TL.
     Corner =

     Multiply TL (3x3) and TR with corners of the left and right image to get
    corner_x matrix. Find minx , maxx, miny, maxy of corner_x and assign it to bb1
    &bb2 (1x4).
     Take min of bb1 , bb2 and max of bb1 , bb2 and assign it to q1 & q2.
     Hence bounding box is given by bb =[q1(1:2),q2(3:4)].
    Steps for rectification
    Since the input image is colored one, copy each plane (R, G, B)
    from left image to I.
    Calculate I2 with interp2 function using the method as bilinear.
    With isnan function, calculate alpha which is a logical matrix,
    contains 1 where I2 is not equal to 0.
    Repeat the above steps for other two plane of left image and for
    right image and calculate JL, bbL, alphaL, JR, bbR, alphaR.
    Calculate mlx by multiplying TL with ml.
    Calculate mrx by multiplying TR with mr.
    Plot JL and JR which gives the rectified image of left and right
    image.
    Right image with epipolar lines through the
    Output of Rectification process
    matched corner points
    Disparity Map generation
    Rectification of the left and right image pair simplifies
    the correspondence problem. Now the search for the
    corresponding pixel in right image can be carried out in
    1-D instead of 2-D.
    Algorithm
     For any pixel in the left image, a window is
    considered with appropriate size centering that pixel.
     Since the images are rectified another window is taken
    in the right image centering any pixel in the same row
    as in left image.
    Disparity map
     The window is shifted horizontally by one pixel in the right image and Sum
    of Squared Difference (SSD) is calculated for each move.
     Minimum value of SSD gives the matching pixel and its position in the
    right images corresponding to left image pixel.
     The window is then shifted horizontally in the left image by one pixel and
    the corresponding pixel in right image is found as explained in steps above.
     The process is repeated for all pixels in the left and right image.
     The relative position of corresponding pixels in left and right image gives
    disparity. Disparity map is generated by plotting disparity of all pixels with
    different gray levels
    output
    Reference
    [1] Edwin Olson, Melissa Hao, “An Introduction to Stereo Vision and Disparity
    Computation”.
    [2] Bailu Ding, “A Survey: Stereo Vision via Belief Propagation”, December 27, 2009
    [3] Prof. V. M. Gadre “Tangible Images”, Digital Signal Processing Application Assignment
    Report,IIT Bombay
    [4] Konstantinos G. Derpanis , “The Harris Corner Detector” October 27, 2004.
    [5] Dmitrij Csetverikov, “Basic Algorithms for Digital image Analysis”, Eotvos Lorand
    University Budapest, Hungary.
    [6] M. Turk and A. Pentland, “Eigenfaces for recognition,” J. Cognitive Neuroscience, vol. 3,
    pp. 71-86, 1991.
    [7] A. Fusiello and L. Irsara, “Quasi-Euclidean Uncalibrated Epipolar Rectification,”.
    Books
    [8] Emanuele.T, Alessandro.V, “Introductory Techniques for 3D computer vision”.
     
     
    Thank you

    You might also like