Epipolar Geometry 2

Epipolar geometry
Three questions:
(i) Correspondence geometry: Given an image point x in the first
view, how does this constrain the position of the corresponding point
x in the second image?
(ii) Camera geometry (motion): Given a set of corresponding
image points {xi xi}, i=1,,n, what are the cameras P and
P for the two views? Or what is the geometric
transformation between the views?
(iii) Scene geometry (structure): Given corresponding image

points xi xi and cameras P, P, what is the position of the
point X in space?
The epipolar geometry
C,C,x,x and X are coplanar

All points on p project on l and l

Family of planes p and lines l and l

Intersection in e and e
epipoles e,e
= intersection of baseline with image plane
= projection of projection center in other image
= vanishing point of camera motion direction
an epipolar plane = plane containing baseline (1-D family)

an epipolar line = intersection of epipolar plane with image
(always come in corresponding pairs)
Example: converging cameras
Example: motion parallel with image plane
Example: forward motion
e
Matrix form of cross product
a2b3 a3b2 0 a3 a2
a b a3b1 a1b3 a3 0 a1 b a b
a1b2 a2b1 a2 a1 0
a ( a b) 0
b ( a b) 0
Geometric transformation
P' RP t
p MP with M [ I | 0]
p' M ' P' with M ' [ R | t ]
Calibrated Camera
p ' Ep 0
p (u, v,1)T
p'[t ( Rp )] 0 with
p ' (u ' , v ' ,1) T
Essential matrix p' Ep 0 with E t R SR

Uncalibrated Camera

p and p' points in pixel coordinate s correspond ing to p and p' in camera coordinate s
1 1
p M int p and p' M 'int p' p' F p 0
T

p' E p 0 with F M ' T
EM 1
int int
Fundamental matrix
Properties of fundamental and
essential matrix
Matrix is 3 x 3
Transpose : If F is essential matrix of cameras (P, P).
FT is essential matrix of camera (P,P)
Epipolar lines: Think of p and p as points in the projective

plane then F p is projective line in the right image.
That is l=F p l = FT p
Epipole: Since for any p the epipolar line l=F p contains

the epipole e. Thus (eT F) p=0 for a all p .
Thus eT F=0 and F e =0
Fundamental matrix
Encodes information of the intrinsic and
extrinisic parameters
F is of rank 2, since S has rank 2 (R and M
and M have full rank)
Has 7 degrees of freedom
There are 9 elements, but scaling is not
significant and det F = 0
Essential matrix
Encodes information of the extrinisic
parameters only
E is of rank 2, since S has rank 2 (and R has
full rank)
Its two nonzero singular values are equal
Has only 5 degrees of freedom, 3 for
rotation, 2 for translation
Scaling ambiguity
P' RP t
P RP t
p T p' T
z P z ( RP t )
Depth Z and Z and t can only be recovered up to a scale factor

Only the direction of translation can be obtained
Least square approach
n
Minimize i i
(p
i 1
' Fp ) 2
under the constraint | F |2 1
We have a homogeneous system A f =0

The least square solution is smallest singular value of A,
i.e. the last column of V in SVD of A = U D VT
Non-Linear Least Squares Approach
Minimize
n
(
i 1
d 2
( pi
'
Fpi ) d 2
( pi Fp'i ))
with respect to the coefficients of F

Using an appropriate rank 2 parameterization
Locating the epipoles
p'T Fe 0
Fe 0
e is the nullspace of F ;
e' is the nullspace of F T
SVD of F = UDVT.
Rectification
Image Reprojection
reproject image planes onto common
plane parallel to line between optical
centers
Rectification
Rotate the left camera so epipole goes to
infinity along the horizontal axis
Apply the same rotation to the right camera
Rotate the right camera by R
Adjust the scale
3D Reconstruction
Stereo: we know the viewing geometry (extrinsic
parameters) and the intrinsic parameters: Find
correspondences exploiting epipolar geometry, then
reconstruct
Structure from motion (with calibrated cameras): Find
correspondences, then estimate extrinsic parameters
(rotation and direction of translation), then reconstruct.
Uncalibrated cameras: Find correspondences,
Compute projection matrices (up to a projective
transformation), then reconstruct up to a projective
transformation.
Reconstruction by triangulation
P
If cameras are intrinsically and extrinsically calibrated, find

P as the midpoint of the common perpendicular to the two rays
in space.
Triangulation
ap ray through C and p,
bRp + T ray though C and p expressed in right coordinate
system
ap'bRp c( p'Rp ) T
R Rr RlT
R=?
T=? T Tr RTl
Point reconstruction
x MX x' M' X
Linear triangulation
x MX x' M' X
x MX 0 AX 0
x'M' X' 0
xm 3T m1T
Linear combination
x m 3 X m1 X 0
T T
ym 3 m 2
T T

y m X m X 0
of 2 other equations T T A T
3 2
x ' m ' T
m '
1
xm X y m X 0
3
T T
y ' m' T m' T
2 1 3 2
homogeneous X 1
AX 0 Homogenous system:
X is last column of V in the SVD of A= USVT
geometric error
d (x, x) 2 d (x' , x' ) 2 subject to x'T Fx 0

or equivalently subject to x MX and x' M' X
Geometric error
Reconstruct matches in projective frame

by minimizing the reprojection error
Non-iterative optimal solution

Reconstruction for intrinsically
calibrated cameras
Compute the essential matrix E using normalized points.
Select M=[I|0] M=[R|T] then E=[Tx]R

Find T and R using SVD of E
Decomposition of E
E [Tx ]R E can be computed up to scale factor
Ty2 Tz2 TxTy TxTz

EE T [Tx ]RR T [Tx ]T TxTy Tx2 Tz2 TyTz
TxTz T T T 2
T 2
y z x y
Tr ( EET ) 2||T| | T can be computed up to sign

(EET is quadratic)
Four solutions for the decomposition,
Correct one corresponds to positive depth values
SVD decomposition of E
E = USVT
[T ] UZU T R UWV T or R UW T V T
0 1 0 0 1 0
with W 1 0 0 and Z - 1 0 0

0 0 1 0 0 0
Reconstruction from uncalibrated
cameras
Reconstruction problem:
given xixi , compute M,M and Xi
x i MX i x i MXi for all i
without additional information possible

only up to projective ambiguity
Projective Reconstruction
Theorem
Assume we determine matching points xi
and xi. Then we can compute a unique
Fundamental matrix F.
The camera matrices M, M cannot be
recovered uniquely
Thus the reconstruction (Xi) is not unique
There exists a projective transformation H
such that
X 2,i HX 1,i , M 2 M1H 1 M ' 2 M '1H 1
Reconstruction ambiguity:
projective
x i MX i MH H X
-1
P P i
From Projective to Metric
Reconstruction
Compute homography H such that XEi=HXi
for 5 or more control points XEi with known
Euclidean position.
Then the metric reconstruction is
M M MH 1 M ' M M ' H 1 X M ,i HX i
Rectification using 5 points
Affine reconstructions
From affine to metric
Use constraints from scene orthogonal lines
Use constraints arising from having the
same camera in both images
Reconstruction from N Views
Projective or affine reconstruction from a
possible large set of images
Given a set of camera Mi,
For each camera Mi a set of image point xji
Find 3D points Xj and cameras Mi, such that
MiXj=xji
Bundle adjustment
Solve following minimization problem
Find Mi and Xj that minimize
d (
i, j
M i
X j , x i 2
j)
Levenberg Marquardt algorithm

Problems many parameters
11 per camera, 3 per 3d point
Useful as final adjustment step for bundles of
rays

Epipolar Geometry 2

Uploaded by

Copyright:

Available Formats

Epipolar Geometry 2

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Epipolar Geometry 2

Uploaded by

Copyright:

Available Formats

Epipolar geometry

(iii) Scene geometry (structure): Given corresponding image

C,C,x,x and X are coplanar

All points on p project on l and l

Family of planes p and lines l and l

an epipolar plane = plane containing baseline (1-D family)

Essential matrix p' Ep 0 with E t R SR

Epipolar lines: Think of p and p as points in the projective

Epipole: Since for any p the epipolar line l=F p contains

Depth Z and Z and t can only be recovered up to a scale factor

under the constraint | F |2 1

We have a homogeneous system A f =0

with respect to the coefficients of F

If cameras are intrinsically and extrinsically calibrated, find

d (x, x) 2 d (x' , x' ) 2 subject to x'T Fx 0

Reconstruct matches in projective frame

Non-iterative optimal solution

Select M=[I|0] M=[R|T] then E=[Tx]R

Ty2 Tz2 TxTy TxTz

Tr ( EET ) 2||T| | T can be computed up to sign

given xixi , compute M,M and Xi

x i MX i x i MXi for all i

without additional information possible

Levenberg Marquardt algorithm

You might also like