Features

Features
All is Vanity, by C. Allan Gilbert, 1873-1929

Readings
• M. Brown et al. Multi-Image Matching using Multi-Scale Oriented Patches,
CVPR 2005
Today’s lecture
• Feature detection
• Feature matching
• Applications
Invariant local features
Find features that are invariant to transformations
• geometric invariance: translation, rotation, scale
• photometric invariance: brightness, exposure, …
Feature Descriptors
Advantages of local features
Locality
• features are local, so robust to occlusion and clutter
Distinctiveness:
• can differentiate a large database of objects
Quantity
• hundreds or thousands in a single image
Efficiency
• real-time performance achievable
Generality
• exploit different types of features in different situations
More motivation…
Feature points are used for:
• Image alignment (e.g., mosaics)
• 3D reconstruction
• Motion tracking
• Object recognition
• Indexing and database retrieval
• Robot navigation
• … other
What makes a good feature?
– gradients are different, large magnitudes

– large1, large 2
Feature detection
Want a “feature detection” function
• gives large values only for image patches that are good features
• How might you define f in terms of 1, 2 ?
The Harris operator
Want a “feature detection” function
• gives large values only for image patches that are good features
• How might you define f in terms of 1, 2 ?
(harmonic mean)
• Called the “Harris Corner Detector” or “Harris Operator”

• Lots of other detectors, this is one of the most popular
The Algorithm:
• Find points with large response (f > threshold)
• Take the points of local maxima of f
Input images
Compute f
Threshold (f > value)
Find local maxima of f
Harris features (in red)
Invariance
Suppose you rotate the image by some angle
• Will you still pick up the same features?
What if you translate the image instead?

Change in brightness?
Scale?
Scale invariant detection
Suppose you’re looking for corners
Key idea: find scale that gives local maximum of f

• f is a local maximum in both position and scale
Feature descriptors
We know how to detect good points
Next question: How to match them?
?
Lots of possibilities (this is a popular research area)
• simple: match square windows around the point
• state of the art (e.g.,): SIFT
– David Lowe, UBC http://www.cs.ubc.ca/~lowe/keypoints/
Rotation invariance for feature descriptors
Find dominant orientation of the image patch
• From the motion lecture, this is given by the eigenvector of
ATA corresponding to the larger eigenvalue
• Rotate the patch according to this angle
MOPS [Brown, Szeliski, Winder, CVPR’2005]

Detections at multiple scales
Matching features
What do we do about the “bad” matches?

RAndom SAmple Consensus
Select one match, count inliers

RAndom SAmple Consensus
Select one match, count inliers

Least squares fit
Find “average” translation vector

RANSAC (RANdom SAmpling Consensus)
Popular approach for robust model fitting with outliers
RANSAC loop:
1. Select K feature matches (at random)
2. Fit model (e.g., homography) based on these features
3. Count inliers:
— number of other features that fit the model to within some
specified threshold
4. The model with the largest number of inliers wins
5. Re-fit the model based on all of these inliers
More info:
http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/FISHER/RANSAC/
RANSAC
Lots of applications
Features are used for:
• Image alignment (e.g., mosaics)
• 3D reconstruction
• Motion tracking
• Object recognition
• Indexing and database retrieval
• Robot navigation
• … other
Autostitch (Brown and Lowe)
Fully automatic panorama generation
• Input: set of images
• Output: panorama(s)
Uses SIFT (Scale-Invariant Feature Transform) to
find/align images
http://www.cs.ubc.ca/~mbrown/autostitch/autostitch.html
Microsoft version
• part of the Digital Image Pro and Digital Image Suite
1. Solve for homography
2. Find connected sets of images
Object recognition (David Lowe)
Sony Aibo
SIFT usage:
Recognize
charging
station
Communicate
with visual
cards
Teach object
recognition
The office of the past (Kim et al.)
http://grail.cs.washington.edu/projects/office/
3D scene recovery

Features

Uploaded by

Copyright:

Available Formats

Features

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Features

Uploaded by

Copyright:

Available Formats

Features

All is Vanity, by C. Allan Gilbert, 1873-1929

– gradients are different, large magnitudes

• Called the “Harris Corner Detector” or “Harris Operator”

What if you translate the image instead?

Key idea: find scale that gives local maximum of f

MOPS [Brown, Szeliski, Winder, CVPR’2005]

What do we do about the “bad” matches?

Select one match, count inliers

Select one match, count inliers

Find “average” translation vector

You might also like