Computer Vision Introduction

What is “Image Processing and
Computer Vision”?
Image Computer
Processing Vision
manipulate image
process image data
data
generate symbolic
generate another
data
image
1
Computer Vision
• Reconstruction
– Recover 3D information from data
• Recognition
– Detect and identify objects
• Understanding
– What is happening in the scene?
Historical overview
• 1920s
– Coding images for transmission by telegraph (3 hours)
• 1960s
– Computers powerful enough to store images and process in
realistic times
– Space program
1960s - 1970s
• Applications
– Medical imaging
– Remote sensing
– Astronomy
Today
– DTV
– Image interpretation
– Biometry
– GIS
– Tele-surgery
System Overview
Captured data
Enhancement
Feature Extraction
Feature Recognition
Image Recognition
Labels
Why study Computer Vision?
• Images and movies are everywhere

• Fast-growing collection of useful applications
– building representations of the 3D world from pictures
– automated surveillance (who’s doing what)
– movie post-processing
– face finding
• Various deep and attractive scientific mysteries
– how does object recognition work?
• Greater understanding of human vision
Part I: The Physics of Imaging
• How images are formed

– Cameras
• What a camera does
• How to tell where the camera was
– Light
• How to measure light
• What light does at surfaces
• How the brightness values we see in cameras are determined
– Color
• The underlying mechanisms of color
• How to describe it and measure it
Part II: Early Vision in One Image
• Simple inferences from individual pixel values

• Representing small patches of image
– For three reasons
• We wish to establish correspondence between (say) points in
different images, so we need to describe the neighborhood of
the points
• Sharp changes are important in practice --- known as “edges”
• Representing texture by giving some statistics of the different
kinds of small patch present in the texture.
– Tigers have lots of bars, few spots
– Leopards are the other way
Representing an image patch
• Filter outputs
– essentially form a dot-product between a pattern and an image,
while shifting the pattern across the image
– strong response -> image locally looks like the pattern
– e.g. derivatives measured by filtering with a kernel that looks like a
big derivative (bright bar next to dark bar)
Convolve this image
To get this
With this kernel

Texture
• Many objects are distinguished by their texture

– Tigers, cheetahs, grass, trees
• We represent texture with statistics of filter outputs
– For tigers, bar filters at a coarse scale respond strongly
– For cheetahs, spots at the same scale
– For grass, long narrow bars
– For the leaves of trees, extended spots
• Objects with different textures can be segmented
• The variation in textures is a cue to shape
Computer Vision : CISC4/689
Shape from texture
Part III: Early Vision in Multiple Images
• The geometry of multiple views

– Where could it appear in camera 2 (3, etc.) given it was here in 1
(1 and 2, etc.)?
• Stereopsis
– What we know about the world from having 2 eyes
• Structure from motion
– What we know about the world from having many eyes
• or, more commonly, our eyes moving.
3D Reconstruction from multiple views
• Multiple views arise from

– stereo
– motion
• Strategy
– “triangulate” from distinct measurements of the same thing
• Issues
– Correspondence: which points in the images are projections of the
same 3D point?
– The representation: what do we report?
– Noise: how do we get stable, accurate reports
Part IV: Mid-Level Vision
Impose some order on groups of pixels to separate them from

each other and infer shape information
• Finding coherent structure so as to break the image or
movie into big units
– Segmentation:
• Breaking images and videos into useful pieces
• E.g. finding video sequences that correspond to one shot
• E.g. finding image components that are coherent in internal
appearance
– Tracking:
• Keeping track of a moving object through a long sequence of
views
Part V: High Level Vision (Geometry)
• The relations between object geometry and image

geometry
– Model based vision
• find the position and orientation of known objects
– Smooth surfaces and outlines
• how the outline of a curved object is formed, and what it looks
like
– Aspect graphs
• how the outline of a curved object moves around as you view it
from different directions
– Range data
Part VI: High Level Vision
(Probabilistic)
• Using classifiers and probability to recognize objects
– Templates and classifiers
• how to find objects that look the same from view to view with
a classifier
– Relations
• break up objects into big, simple parts, find the parts with a
classifier, and then reason about the relationships between the
parts to find the object.
– Geometric templates from spatial relations
• extend this trick so that templates are formed from relations
between much smaller parts
Part VII: Some Applications in Detail
• Finding images in large collections

– searching for pictures
– browsing collections of pictures
• Image based rendering
– often very difficult to produce models that look like real objects
• surface weathering, etc., create details that are hard to model
• Solution: make new pictures from old
Some applications of recognition
• Digital libraries
– Find me the pic of a certain posture from skating video
• Surveillance
– Warn me if there is a mugging in the grove
• HCI
– Do what I show you
• Military
– Shoot this, not that
What are the problems in recognition?
• Which bits of image should be recognised together?
– Segmentation.
• How can objects be recognised without focusing on detail?
– Abstraction.
• How can objects with many free parameters be
recognised?
– No popular name, but it’s a crucial problem anyhow.
• How do we structure very large modelbases?
– again, no popular name; abstraction and learning come into this
Segmentation
• Which image components “belong together”?

• Belong together=lie on the same object
• Cues
– similar colour
– similar texture
– not separated by contour
– form a suggestive shape when assembled
Image Segmentation
Image Segmentation
Matching templates
• Some objects are 2D patterns

– e.g. faces
• Build an explicit pattern matcher
– discount changes in illumination by using a parametric model
– changes in background are hard
– changes in pose are hard
http://www.ri.cmu.edu/projects/project_271.html
Relations between templates
• e.g. find faces by

– finding eyes, nose, mouth
– finding assembly of the three that has the “right” relations
http://www.ri.cmu.edu/projects/project_320.html
Tracking
• Use a model to predict next position and refine using next

image
• Model:
– simple dynamic models (second order dynamics)
– kinematic models
– etc.
• Face tracking and eye tracking now work rather well
Application results
• Rigid Motion
• Reconstruction:
2D 3D
2D 3D
2D+3D
Clouds: Interpolation
Clouds: Reconstruction
Tongue: Reconstruction
More..
• Tongue Tracking
• Face Tracking
• Stereo: Human Body

• Stereo: Ice (Hard)
• Bio-medical:
Tongue-head Tongue-skull
Few More..
• ACCESS
• Stereo-Face Tracker
Project on Image Guided Surgery:
A collaboration between the MIT AI Lab and Brigham
and Women's Surgical Planning Laboratory
• The Computer Vision Group of the MIT Artificial
Intelligence Lab has been collaborating closely for several
years with the Surgical Planning Laboratory of Brigham
and Women's Hospital. As part of the collaboration, tools
are being developed to support image guided surgery. Such
tools will enable surgeons to visualize internal structures
through an automated overlay of 3D reconstructions of
internal anatomy on top of live video views of a patient.
We are developing image analysis tools for leveraging the
detailed three-dimensional structure and relationships in
medical images. Sample applications are in preoperative
surgical planning, intraoperative surgical guidance,
navigation, and instrument tracking.
Figures by kind permission of Eric Grimson; further information can be
obtained from his web site http://www.ai.mit.edu/people/welg/welg.html.
Figures by kind permission of Eric Grimson; further information can be
obtained from his web site http://www.ai.mit.edu/people/welg/welg.html.
Some Results
• MRI data
• Rotate Model
• Peel

Computer Vision Introduction

Uploaded by

Copyright:

Available Formats

Computer Vision Introduction

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Computer Vision Introduction

Uploaded by

Copyright:

Available Formats

What is “Image Processing and

• Images and movies are everywhere

• How images are formed

• Simple inferences from individual pixel values

With this kernel

• Many objects are distinguished by their texture

• The geometry of multiple views

• Multiple views arise from

Impose some order on groups of pixels to separate them from

• The relations between object geometry and image

• Finding images in large collections

• Which image components “belong together”?

• Some objects are 2D patterns

• e.g. find faces by

• Use a model to predict next position and refine using next

• Stereo: Human Body

You might also like