Basic Mathematics of Projection

01/20/11
Projective Geometry and Camera Models

Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem
Administrative Stuff
Office hours
Derek: Wed 4-5pm + drop by Ian: Mon 3-4pm, Thurs 3:30-4:30pm
HW 1: out Monday
Prob1: Geometry, today and Tues Prob2: Lighting, next Thurs Prob3: Filters, following week
Next Thurs: Im out, David Forsyth will cover
Last class: intro

Overview of vision, examples of state of art Logistics
Next two classes: Single-view Geometry

How tall is this woman? How high is the camera? What is the camera rotation? What is the focal length of the camera?
Which ball is closer?
Todays class
Mapping between image and world coordinates
Pinhole camera model Projective geometry
Vanishing points and lines
Projection matrix
Image formation
Lets design a camera
Idea 1: put a piece of film in front of an object Do we get a reasonable image?
Slide source: Seitz
Pinhole camera
Idea 2: add a barrier to block off most of the rays

This reduces blurring The opening known as the aperture
Slide source: Seitz
Pinhole camera
f
f = focal length c = center of the camera

Figure from Forsyth
Camera obscura: the pre-camera

First idea: Mo-Ti, China (470BC to 390BC) First built: Alhacen, Iraq/Egypt (965 to 1039AD)
Illustration of Camera Obscura
Freestanding camera obscura at UNC Chapel Hill

Photo by Seth Ilys
Camera Obscura used for Tracing
Lens Based Camera Obscura, 1568
First Photograph
Oldest surviving photograph
Took 8 hours on pewter plate
Photograph of the first photograph
Joseph Niepce, 1826
Stored at UT Austin
Niepce later teamed up with Daguerre, who eventually created Daguerrotypes
Dimensionality Reduction Machine (3D to 2D)

3D world 2D image
Point of observation
Figures Stephen E. Palmer, 2002
Projection can be tricky
Slide source: Seitz
Projection can be tricky
Slide source: Seitz
Projective Geometry
What is lost?
Length
Who is taller?
Which is closer?
Length is not preserved
A C B
Figure by David Forsyth
Projective Geometry
What is lost?
Length Angles
Parallel?
Perpendicular?
Projective Geometry
What is preserved?
Straight lines are still straight

Parallel lines in the world intersect in the image at a vanishing point
Vanishing Point Vanishing Line
Vanishing Point

Vertical vanishing point (at infinity) Vanishing line
Vanishing point
Vanishing point
Slide from Efros, Photo from Criminisi
Photo from online Tate collection
Note on estimating vanishing points
Use multiple lines for better accuracy but lines will not intersect at exactly the same point in practice One solution: take mean of intersecting pairs bad idea! Instead, minimize angular differences
Vanishing objects
Projection: world coordinatesimage coordinates
Optical Center (u0, v0)
.
f
.
Camera Center (tx, ty, tz)
X P = Y Z
u p= v
Homogeneous coordinates
Conversion
Converting to homogeneous coordinates
homogeneous image coordinates
homogeneous scene coordinates
Converting from homogeneous coordinates
Homogeneous coordinates
Invariant to scaling
x kx kx x kw w k y = ky ky = y kw w w kw
Homogeneous Coordinates Cartesian Coordinates
Point in Cartesian is ray in Homogeneous
Basic geometry in homogeneous coordinates

ai Line equation: ax + by + c = 0 linei = b i ci Append 1 to pixel coordinate to get ui homogeneous coordinate pi = v i 1
Line given by cross product of two points
lineij = pi p j
Intersection of two lines given by cross product of the lines qij = linei line j
Another problem solved by homogeneous coordinates
Intersection of parallel lines

Cartesian: (Inf, Inf) Homogeneous: (1, 1, 0) Cartesian: (Inf, Inf) Homogeneous: (1, 2, 0)
Projection matrix
R,T
Slide Credit: Saverese
jw kw Ow iw
x = K[R t ] X
x: Image Coordinates: (u,v,1) K: Intrinsic Matrix (3x3) R: Rotation (3x3) t: Translation (3x1) X: World Coordinates: (X,Y,Z,1)
Interlude: when have I used this stuff?
When have I used this stuff?

Object Recognition (CVPR 2006)

Single-view reconstruction (SIGGRAPH 2005)

Getting spatial layout in indoor scenes (ICCV 2009)

Inserting photographed objects into images (SIGGRAPH 2007)
Original
Created

Inserting synthetic objects into images
Projection matrix
Intrinsic Assumptions Extrinsic Assumptions No rotation Unit aspect ratio

Optical center at (0,0) No skew Camera at (0,0,0)
K
x = K [I 0] X
u f wv = 0 1 0
0 f 0
x 0 0 y 0 0 z 1 0 1
Remove assumption: known optical center

Intrinsic Assumptions Extrinsic Assumptions No rotation Unit aspect ratio
No skew Camera at (0,0,0)
x = K [I 0] X
u f wv = 0 1 0
0 f 0
u0 v0 1
x 0 y 0 z 0 1
Remove assumption: square pixels

Intrinsic Assumptions Extrinsic Assumptions
No skew No rotation Camera at (0,0,0)
x = K [I 0] X
u wv = 0 1 0
u0 v0 1
x 0 y 0 z 0 1
Remove assumption: non-skewed pixels

No rotation Camera at (0,0,0)
x = K [I 0] X
u wv = 0 1 0
u0 v0 1
x 0 y 0 z 0 1
Note: different books use different notation for parameters
Oriented and Translated Camera

R
jw
t
Ow iw
kw
Allow camera translation

No rotation
x = K [I t ] X
u wv = 0 1 0
x u0 1 0 0 t x y v0 0 1 0 t y z 1 0 0 1 t z 1
3D Rotation of Points
Rotation around the coordinate axes, counter-clockwise:

0 1 Rx ( ) = 0 cos 0 sin sin cos 0 sin 1 0 0 cos 0 0 0 1
y
p
cos Ry ( ) = 0 sin cos sin Rz ( ) = sin cos 0 0
Allow camera rotation

x = K [R t ] X
u wv = 0 1 0 s u0 r11 v0 r21 1 r31 r12 r22 r32 r13 r23 r33 x tx y ty z tz 1
Degrees of freedom
x = K [R t ] X
5 6
u wv = 0 1 0
u0 r11 v0 r21 1 r31
r12 r22 r32
r13 r23 r33
x tx y ty z tz 1
Vanishing Point = Projection from Infinity

x xR x y p = K[R t ] p = KR y p = K y R z z zR 0
u f wv = 0 1 0 0 f 0 u0 xR v0 y R 1 zR
fxR u= + u0 zR fyR v= + v0 zR
Orthographic Projection
Special case of perspective projection
Distance from the COP to the image plane is infinite
Image World
Also called parallel projection Whats the projection matrix?
x u 1 0 0 0 y w v = 0 1 0 0 z 1 0 0 0 1 1
Slide by Steve Seitz
Scaled Orthographic Projection

Special case of perspective projection
Object dimensions are small compared to distance to camera
Image World
u f Also called weak perspective w v = 0 Whats the projection matrix? 1 0
0 f 0
x 0 0 y 0 0 z 0 s 1
Suppose we have two 3D cubes on the ground facing the viewer, one near, one far.
1. What would they look like in perspective? 2. What would they look like in weak perspective?
Photo credit: GazetteLive.co.uk
Beyond Pinholes: Radial Distortion
Corrected Barrel Distortion

Image from Martin Habbecke
Things to remember
Vanishing line
Vertical vanishing point (at infinity)
Vanishing points and vanishing lines Pinhole camera model and camera projection matrix Homogeneous coordinates
Vanishing point
Vanishing point
x = K [R t ] X
Next class
Applications of camera model and projective geometry
Recovering the camera intrinsic and extrinsic parameters from an image Recovering size in the world Projecting from one plane to another
Questions
What about focus, aperture, DOF, FOV, etc?
Adding a lens
circle of confusion
A lens focuses light onto the film

There is a specific distance at which objects are in focus
other points project to a circle of confusion in the image
Changing the shape of the lens changes this distance
Focal length, aperture, depth of field

F focal point optical center (Center Of Projection)
A lens focuses parallel rays onto a single focal point

focal point at a distance f beyond the plane of the lens Aperture of diameter D restricts the range of rays
Slide source: Seitz
The eye
The human eye is a camera

Whats the film?
Iris - colored annulus with radial muscles Pupil - the hole (aperture) whose size is controlled by the iris
photoreceptor cells (rods and cones) in the retina
Depth of field
Slide source: Seitz
f / 5.6
f / 32
Changing the aperture size or focal length affects depth of field

Flower images from Wikipedia
http://en.wikipedia.org/wiki/Depth_of_field
Varying the aperture
Slide from Efros
Large aperture = small DOF
Small aperture = large DOF
Shrinking the aperture
Why not make the aperture as small as possible?

Less light gets through Diffraction effects
Shrinking the aperture
Relation between field of view and focal length

Field of view (angle width)
d fov = tan 2f
1
Film/Sensor Width
Focal length
Dolly Zoom or Vertigo Effect

http://www.youtube.com/watch?v=Y48R6-iIYHs
How is this done?
Zoom in while moving away
http://en.wikipedia.org/wiki/Focal_length

Basic Mathematics of Projection

Uploaded by

Copyright:

Available Formats

Basic Mathematics of Projection

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Basic Mathematics of Projection

Uploaded by

Copyright:

Available Formats

01/20/11

Projective Geometry and Camera Models

Next Thurs: Im out, David Forsyth will cover

Last class: intro

Next two classes: Single-view Geometry

Which ball is closer?

Lets design a camera

Idea 1: put a piece of film in front of an object Do we get a reasonable image?

Slide source: Seitz

Idea 2: add a barrier to block off most of the rays

f = focal length c = center of the camera

Camera obscura: the pre-camera

Illustration of Camera Obscura

Freestanding camera obscura at UNC Chapel Hill

Camera Obscura used for Tracing

Lens Based Camera Obscura, 1568

Photograph of the first photograph

Joseph Niepce, 1826

Niepce later teamed up with Daguerre, who eventually created Daguerrotypes

Dimensionality Reduction Machine (3D to 2D)

Figures Stephen E. Palmer, 2002

Projection can be tricky

Slide source: Seitz

Projection can be tricky

Slide source: Seitz

Length is not preserved

Figure by David Forsyth

Vanishing points and lines

Vanishing points and lines

Vanishing Point Vanishing Line

Vanishing points and lines

Slide from Efros, Photo from Criminisi

Vanishing points and lines

Photo from online Tate collection

Note on estimating vanishing points

Projection: world coordinatesimage coordinates

Optical Center (u0, v0)

homogeneous image coordinates

homogeneous scene coordinates

Converting from homogeneous coordinates

Point in Cartesian is ray in Homogeneous

Basic geometry in homogeneous coordinates

Line given by cross product of two points

Another problem solved by homogeneous coordinates

Intersection of parallel lines

Slide Credit: Saverese

Interlude: when have I used this stuff?

When have I used this stuff?

When have I used this stuff?

When have I used this stuff?

When have I used this stuff?

When have I used this stuff?

Intrinsic Assumptions Extrinsic Assumptions No rotation Unit aspect ratio

Remove assumption: known optical center

Remove assumption: square pixels

Remove assumption: non-skewed pixels

Note: different books use different notation for parameters

Oriented and Translated Camera

Allow camera translation

Slide Credit: Saverese