Scribd Logo
Upload

Welcome to Scribd!

  • 8 views

    Harris InterestPoints Andfeatures 1

    Uploaded by

    Ankush Jain
    The document discusses aspects of classical computer vision including edges and features, tracking, classification, and shape from 3D. It then provides details on SIFT including motivation, the idea, overall procedure, scale-space interest points using difference of Gaussians, keypoint localization with orientation, and the SIFT descriptor.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Harris InterestPoints Andfeatures 1

    Uploaded by

    Ankush Jain
    8 views64 pages
    The document discusses aspects of classical computer vision including edges and features, tracking, classification, and shape from 3D. It then provides details on SIFT including motivation, the idea, overall procedure, scale-space interest points using difference of Gaussians, keypoint localization with orientation, and the SIFT descriptor.

    Original Title

    Harris_InterestPoints_andFeatures_1

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    The document discusses aspects of classical computer vision including edges and features, tracking, classification, and shape from 3D. It then provides details on SIFT including motivation, the idea, overall procedure, scale-space interest points using difference of Gaussians, keypoint localization with orientation, and the SIFT descriptor.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    8 views64 pages

    Harris InterestPoints Andfeatures 1

    Uploaded by

    Ankush Jain
    The document discusses aspects of classical computer vision including edges and features, tracking, classification, and shape from 3D. It then provides details on SIFT including motivation, the idea, overall procedure, scale-space interest points using difference of Gaussians, keypoint localization with orientation, and the SIFT descriptor.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 64

    Aspects of Classical Computer Vision

    Edges &
    Features

    Tracking Classification

    Shape From 3D

    Edges and Features are precursors to other functions such as tracking, recovering 3D shape from
    2D images, and object classification
    Uses of Features -
    •Automate object tracking
    •Point matching for computing disparity
    •Stereo calibration
    •Estimation of fundamental matrix
    •Motion based segmentation
    •Recognition
    •3D object reconstruction
    •Robot navigation
    •Image retrieval and indexing
    𝑤 𝑥, 𝑦
    window
    SIFT: Motivation
    The Harris operator is not invariant to scale and correlation
    is not invariant to rotation1.

    For better image matching, Lowe’s goal was to develop an


    interest operator that is invariant to scale and rotation.

    Also, Lowe aimed to create a descriptor that was robust to


    the variations corresponding to typical viewing conditions.
    The descriptor is the most-used part of SIFT.

    1But Schmid and Mohr developed a rotation invariant descriptor for it in 1997.

    4/28/2020 30
    Idea of SIFT
    Image content is transformed into local feature
    coordinates that are invariant to translation, rotation, scale,
    and other imaging parameters

    SIFT Features

    4/28/2020 31
    Overall Procedure at a High Level
    1. Scale-space extrema detection
    Search over multiple scales and image locations.

    2. Keypoint localization
    Fit a model to detrmine location and scale.
    Select keypoints based on a measure of stability.
    3. Orientation assignment
    Compute best orientation(s) for each keypoint region.

    4. Keypoint description
    Use local image gradients at selected scale and rotation
    to describe each keypoint region.

    4/28/2020 35
    A blob detector for different sizes
    Lowe’s Scale-space Interest Points
    Laplacian of Gaussian kernel
    ◦ Scale normalised (x by scale2)
    ◦ Proposed by Lindeberg
    Scale-space detection
    ◦ Find local maxima across scale/space
    ◦ A good “blob” detector

    [ T. Lindeberg IJCV 1998 ]


    4/28/2020 40
    Lowe’s Scale-space Interest Points:
    Difference of Gaussians

    Gaussian is an ad hoc solution of heat


    diffusion equation

    Hence

    k is not necessarily very small in


    practice

    4/28/2020 41
    Lowe’s Pyramid Scheme
    • Scale space is separated into octaves:
    • Octave 1 uses scale 
    • Octave 2 uses scale 2
    • etc.

    • In each octave, the initial image is repeatedly convolved


    with Gaussians to produce a set of scale space images.

    • Adjacent Gaussians are subtracted to produce the DOG

    • After each octave, the Gaussian image is down-sampled


    by a factor of 2 to produce an image ¼ the size to start
    the next level.

    4/28/2020 42
    Keypoint localization with orientation

    832
    233x189
    initial keypoints

    536
    729
    keypoints after keypoints after
    gradient threshold ratio threshold

    4/28/2020 54
    4. Scale Invariant Feature Transform (SIFT)
    1. Take a 16 x16 window around interest point (i.e., at
    the scale detected).
    2. Divide into a 4x4 grid of cells.
    3. Compute histogram of image gradient directions in
    each cell (8 bins each).
    16 histograms x 8 orientations
    = 128 features
    5. Feature matching
    For each feature in A, find nearest neighbor in B

    A B
    Example: Planar recognition
    Training images
    Example: Planar recognition
    • Reliably recognized
    at a rotation of 60°
    away from the
    camera
    • Affine fit is an
    approximation of
    perspective
    projection
    • Only 3 points are
    needed for
    recognition
    3D object recognition
    Training images
    3D object recognition

    Only 3 keys are needed for


    recognition, so extra keys
    provide robustness
    Affine model is no longer
    as accurate
    Recognition under occlusion
    Illumination invariance
    Robot Localization
    Credits and Sources
    ▪This lecture contains open source materials authored by
    ▪ Dr. Mubarak Shah
    ▪ Dr. Ajit Rajwade
    ▪ Dr. David Lowe (inventor of SIFT)
    ▪ Dr. George Bebis
    ▪ Dr. Cornelia Fermüller

    You might also like