Second Edition (March 2011)

These rough notes are for the Vector Spaces part of CIE Further Mathematics Alevel, since
these topics are not covered in many books. I hope they are helpful. I have included proofs and
explanations for most things which can be understood by students, even if they are not on the
syllabus.
Dr. Zen Harper, CIE centre at Shanghai Experimental School
Email: [email protected]
VECTOR SPACES, SPAN, LINEAR INDEPENDENCE, BASES
Denition 1 A vector space V is a collection of objects with the following properties:
There is some special element 0 V (the zero vector) and some operation +, meaning:
for any u, v V , there is some u + v V . We have the following properties (for all
u, v, w V ):
u +v = v +u, (u +v) +w = u + (v +w), u +0 = u.
For every u V , there is some element u V such that u + (u) = 0.
For every number R and every u V , there is some u V . We have the following
properties (for all u, v V and all , R):
1u = u, 0u = 0, (u+v) = u+v, ( +)u = u+u, (u) = ()u.
With these properties, we can prove that u is UNIQUE, and (1)u = u. We write u v
instead of u + (v).
THE MOST OBVIOUS EXAMPLE: R
n
, the space of all column vectors of height n.
OTHER EXAMPLES: see Example 17 below. The set of all (realvalued) functions y sat-
isfying
d
2
y
dx
2

dy
dx
6y = 0 is a vector space. THIS IS ONE OF THE REASONS WHY VECTOR
SPACES ARE SO IMPORTANT! This is a subspace of the vector space of all continuous functions
f : R R.
(In this case, the space has dimension 2, spanned by the functions e
3x
, e
2x
; see later for the
denition of dimension).
Denition 2 Let V be a vector space and W V . We say that W is a vector subspace of V if
W is itself a vector space. We only need to check that a, b W = a + b W, a W
for every R.
Example 3 In R
2
, consider the sets A =
__
x
y
_
: x = 13y
_
, B =
__
x
y
_
: 14x + 3y = 0
_
,
C =
__
x
y
_
: x + y = 1
_
, D =
__
x
y
_
: x
2
+y
2
= 1
_
, E =
__
x
y
_
: x, y Z
_
.
Then A, B are vector subspaces of R
2
, e.g. let u =
_
u
1
u
2
_
, v =
_
v
1
v
2
_
A. Then u
1
= 13u
2
,
v
1
= 13v
2
, so u
1
+ v
1
= 13(u
2
+ v
2
); thus u + v =
_
u
1
+v
1
u
2
+v
2
_
A. Also u
1
= 13(u
2
), so
u A.
1
C, D are NOT subspaces because 0 C, D. E is not a subspace because a E does not
imply
1
2
a E.
If we dened a set F similarly by the equation xy = 0, then a F = a F, but
a, b F does not imply a +b F, so F is not a subspace.
_

_
_
_
x
y
z
_
_
R
3
:
2x y + 3z = 0,
x + y + 4z = 0,
x 4y + 2z = 0
_

_
is a vector subspace of R
3
.
Every vector subspace of R
3
is either: {0} (a point), a line through 0, a plane through 0, or R
3
itself.
Denition 4 Let u
1
, u
2
, u
3
, . . . , V . A linear combination of these vectors is

N
j=1

j
u
j
, for
some N and numbers
1
,
2
,
3
, . . . ,
N
R. Their span (or linear span) is
span{u
1
, u
2
, u
3
, . . .} = { all linear combinations of u
1
, u
2
, u
3
, . . . }
NOTE: the collection of vectors u
1
, u
2
, . . . could be innite; however, a linear combination
must be a sum of only nitely many vectors.
It is obvious that the span of every (nonempty) collection of vectors is a vector subspace.
Example 5 Let a =
_
1
1
_
, b =
_
17
0
_
in R
2
. Then span {a, b} = R
2
, because
_
x
y
_
=
(y)a +
1
17
(x +y) b for every x, y R.
Let c =
_
1
2
_
, d =
_
3
6
_
in R
2
. Then span{c, d} =
__
x
y
_
R
2
: y = 2x
_
=
span{c} = span{d}.
Denition 6 u
1
, u
2
, u
3
, . . . , u
N
V are called linearly independent if:
N

j=1

j
u
j
= 0 =
j
= 0 for all j.
For example: every plane in R
3
passing through (0, 0, 0) is the span of two linearly indepen-
dent vectors.
Example 7 a =
_
_
1
1
0
_
_
, b =
_
_
0
1
2
_
_
, c =
_
_
0
0
3
_
_
are linearly independent in R
3
.
Proof: Let a + b + c = 0. Then
_
_

+
2 3
_
_
= 0, so = 0, + = 0, 2 3 = 0.
Thus = = = 0 (check this calculation yourself!)
Example 8 a =
_
_
1
2
1
_
_
, b =
_
_
0
1
2
_
_
, c =
_
_
2
1
4
_
_
are NOT linearly independent in R
3
.
2
Proof: 2a + 3b = c, so 2a + 3b + (1)c = 0.
Also, any u
1
, u
2
, u
3
, . . . , u
N
with u
j
= 0 for some j is not linearly independent.
PROOF: 0u
1
+ 0u
2
+ + 1u
j
+ 0u
j+1
+ + 0u
N
= 0, but 0, 0, . . . , 1, 0, . . . , 0 are not
all zero.
Exercise 9 If a, b, c R
3
are orthogonal and nonzero, so that a b = a c = b c = 0 and
a, b, c = 0, then a, b, c are linearly independent.
Fact 10
_
_
_
_
_
x
11
x
21
.
.
.
x
n1
_
_
_
_
_
,
_
_
_
_
_
x
12
x
22
.
.
.
x
n2
_
_
_
_
_
, . . . ,
_
_
_
_
_
x
1n
x
2n
.
.
.
x
nn
_
_
_
_
_
are linearly
independent
in R
n
det
_
_
_
_
_
x
11
x
12
x
1n
x
21
x
22
x
2n
.
.
.
.
.
.
.
.
.
.
.
.
x
n1
x
n2
x
nn
_
_
_
_
_
= 0.
Denition 11 A basis of V is a linearly independent collection of vectors u
1
, u
2
, u
3
, . . . V
such that span{u
1
, u
2
, u
3
, . . .} = V .
Theorem 12 Let u
1
, u
2
, u
3
, . . . be a basis of V . Then every a V is a UNIQUE linear combi-
nation a =

j

j
u
j
.
Proof: By denition,
1
,
2
,
3
, . . . EXIST, because u
1
, u
2
, u
3
, . . . span V . If also a =

j

j
u
j
for some
1
,
2
, . . . then
0 = a a =

j
u
j

j
u
j
=

j
(
j

j
)u
j
,
so
j

j
= 0 for each j because u
1
, u
2
, u
3
, . . . are linearly independent. Therefore
j
=
j
for each j. Therefore
1
,
2
,
3
, . . . are UNIQUE.
Exercise 13 Let A be any n n invertible matrix, and let v
1
, v
2
, . . . , v
n
R
n
. Then:
v
1
, v
2
, . . . , v
n
is a basis of R
n
Av
1
, Av
2
, . . . , Av
n
is a basis of R
n
.
Denition 14 A vector space V is called nite dimensional if there is a basis of nitely many
vectors u
1
, u
2
, . . . , u
n
V .
Fact 15 If V is nite dimensional, then every basis of V has the same number n of vectors;
then n = dim(V ) is called the DIMENSION of V .
Example 16 dim(R
n
) = n. For n = 3, a basis is e
1
=
_
_
1
0
0
_
_
, e
2
=
_
_
0
1
0
_
_
, e
3
=
_
_
0
0
1
_
_
.
More generally, the standard basis for R
n
is e
1
, e
2
, . . . , e
n
, where (e
i
)
i
= 1 (the i
th
row), and
(e
i
)
j
= 0 if i = j.
In Example 3 above, dim(A) = dim(B) = 1. One possible basis for A is the single vector
_
13
1
_
. (But any nonzero vector in A will also give a basis). One possible basis for B is the
single vector
_
3/7
2
_
. In Example 5 above, {a, b} is a basis for R
2
.
3
Example 17 Let Y be the set of all functions y : R R which satisfy
d
2
y
dx
2

dy
dx
6y = 0.
Then Y is a nitedimensional vector space with dim(Y ) = 2.
Proof: From Differential Equations: let y
1
= e
3x
, y
2
= e
2x
. Then {y
1
, y
2
} is a basis of Y .
Fact 18 Let U, V be vector spaces with U V , and V nitedimensional. THEN U is nite
dimensional with dim(U) dim(V ). Also dim(U) = dim(V ) U = V .
Theorem 19 Let u
1
, u
2
, . . . , u
m
V be linearly independent. Then:
(i) u
1
, u
2
, . . . , u
m
is a basis for span{u
1
, u
2
, . . . , u
m
}.
(ii) If m = dim(V ) then u
1
, u
2
, . . . , u
m
is a basis for V .
(iii) Every collection of vectors a, b, c, . . . V containing more than dim(V ) vectors is not
linearly independent.
Proof: (i) by denition. (ii) by Fact 18 (since the span is a subspace of V with the same
dimension). In (iii), the vectors a, b, c, . . . cannot be linearly independent (because if they
were, then their span would be a subspace of V with higher dimension, contradicting Fact 18).
Denition 20 For any matrix A, the column space is the linear span of the columns of A. Thus,
it is a vector space of column vectors. The row space is the linear span of the rows of A, and
thus a vector space of row vectors
_
x
1
x
2
x
n
_
.
Example 21 The column space of
_
_
3 1 0 1
0 1 0 0
0 0 0 2
_
_
is R
3
, and the row space is
__
3 + 0 + 2
_
: , , R
_
=
__
x y 0 z
_
: x, y, z R
_
.
The same set of vectors can have many different descriptions, but notice that the second de-
scription of the row space using x, y, z is simpler than the rst description using , , .
Proof: (CHECK THESE CALCULATIONS!) The column space is the set of all vectors

_
_
3
0
0
_
_
+
_
_
1
1
0
_
_
+
_
_
0
0
0
_
_
+
_
_
1
0
2
_
_
=
_
_
3 +

2
_
_
over all , , , R. This is clearly a subset of R
3
. Conversely, given any vector
_
_
x
y
z
_
_
R
3
,
we can nd , , , such that x = 3 +, y = , z = 2. This is easy: let = y, =
1
2
y,
then let =
1
3
(x + ), and can be anything: the value of is unimportant.
For the row space, we have to show two things: rst, given any , , R, there are some
x, y, z R such that
_
3 + 0 + 2
_
=
_
x y 0 z
_
. This is obvious: let x = 3,
etc. Second, given any x, y, z R, there are some , , R such that it holds. This is also
fairly easy: let =
1
3
x, then = y , then nally =
1
2
(z +).
IMPORTANT: this row space is NOT a subspace of R
4
, because R
n
uses columns, not rows.
4
RANGE SPACE, RANK, NULL SPACE, NULLITY, LINEAR
TRANSFORMATIONS
Denition 22 Let V, W be vector spaces, and T : V W any function. Then T is called a
linear transformation if
T(a +b) = T(a) + T(b) for all a, b V , , R.
Theorem 23 Let T : R
n
R
m
be any linear transformation. There is a unique matrix A, with
m rows and n columns, such that T(v) = Av.
Conversely, if B is any matrix with m rows and n columns, then the function S : R
n
R
m
given by S(v) = Bv is a linear transformation.
Proof: Obvious. The column vectors of Aare exactly Te
1
, Te
2
, . . . , Te
n
, where e
1
, e
2
, . . . , e
n
is the standard basis of R
n
.
Denition 24 For any linear transformation T : V W, we have
Range space of T = {Tv : v V }, Rank of T = dim( Range space of T ),
Null space of T = {v V : Tv = 0}, Nullity of T = dim( Null space of T ).
Obviously the null space and range space are vector subspaces of V and W, respectively.
Theorem 25 Let T : R
n
R
m
be a linear transformation represented by the matrix A. Then:
Range space of T = Column space of A
Proof: Obvious, since Ae
j
is the j
th
column of A, with e
1
, e
2
, . . . , e
n
the standard basis of
R
n
. (I already said this in the proof of Theorem 23).
Fact (RankNullity Formula): dim(V ) = dim(Null space of T) + dim(Range space of T).
Proof: (***) From undergraduate Linear Algebra, V =
_
V/null(T)
_
null(T), so
dim(V ) = dim
_
V/null(T)
_
+ dim
_
null(T)
_
.
But T : V W induces an isomorphism

T : V/null(T) T(V ), given by

T(v + null(T)) =
Tv. Hence dim
_
V/null(T)
_
= dim(T(V )).
Example 26 (Invertibility can never work for nonsquare matrices) Let A, B, C be matri-
ces with AB = I (mm) and CA = I (n n). Then WE MUST HAVE n = m.
Proof: A, B, C must represent linear transformations A : R
n
R
m
and B, C : R
m
R
n
.
The range space of A is R
m
, because u = Iu = (AB)u = A(Bu) for every u R
m
.
The null space of Ais {0}, because if Av = 0 then v = Iv = (CA)v = C(Av) = C0 = 0.
Thus n = dim(R
n
) = dim(Null space of A) + dim(Range space of A)
= dim({0}) + dim(R
m
) = 0 +m = m.
Theorem 27 If A is a SQUARE matrix, then A is invertible the null space of A is {0}.
5
Proof: Let u be in the null space, so that Au = 0. If also A is invertible, then u = A
1
0 = 0.
Conversely, suppose that the null space of A is {0}. Let A have size nn, so that A denes
a function A : R
n
R
n
. Now
Au = Av Au Av = 0 A(u v) = 0
u v is in the null space of A u v = 0 u = v.
So A, as a function on R
n
, is onetoone: different vectors are sent to different vectors.
Secondly, the nullity of A is zero, so the rank of A is n 0 = n. Thus A sends R
n
onto R
n
.
Thus A is onetoone and onto, so denes an INVERTIBLE function on R
n
. Let the inverse
function be f : R
n
R
n
. Hence, f(Au) = u = Af(u) for all u R
n
.
It is obvious that f : R
n
R
n
is linear. Therefore, f can be represented by some
n n matrix B. Therefore u = f(A(u)) = f(Au) = B(Au) = (BA)u for all u R
n
,
so BA = I [because the matrix representing a linear transformation is unique]. Similarly,
u = A(f(u)) = A(Bu) = (AB)u, so AB = I. Therefore B = A
1
.
Theorem 28 If A, B are SQUARE matrices and AB = I, then A, B are invertible, B = A
1
.
The denition of A
1
says A
1
A = AA
1
= I. So this DOESNT FOLLOW just from the
denition of inverse! We DONT assume that BA = I, so we have to prove it!!
Proof: Since A, B are square, THEY ARE THE SAME SIZE (otherwise AB would not be
dened). Now:
EITHER: (RUBBISH PROOF!!) 1 = det(I) = det(AB) = det(A)det(B), so det(A) = 0, so
A is invertible; therefore A
1
= A
1
I = A
1
(AB) = (A
1
A)B = IB = B.
OR: (PROPER PROOF!) if AB = I and v is in the null space of B, then v = Iv = (AB)v =
A(Bv) = A0 = 0. Therefore, the null space of B is {0}. Therefore by Theorem 27, B is
invertible.
So A = AI = A(BB
1
) = (AB)(B
1
) = IB
1
= B
1
, so A = B
1
. It is always true that
B
1
is invertible with (B
1
)
1
= B [PROVE THIS!!!!], thus A
1
= B.
Warning WE MUST HAVE SQUARE MATRICES!!!!! e.g. if A =
_
1 0 0
1 0 1
_
and B =
_
_
1 0
0 0
1 1
_
_
, then AB =
_
1 0
0 1
_
, but A, B are not invertible (they couldnt possibly be, since
they are not square!) Notice that BA =
_
_
1 0 0
0 0 0
0 0 1
_
_
is not the identity matrix.
ROW OPERATIONS
Given any matrix, there are 3 kinds of row operations on the rows R
1
, R
2
, R
3
, . . .
1. Swap R
i
, R
j
, for some i = j;
2. Replace R
i
by R
i
, for some = 0;
3. Replace R
i
by R
i
+R
j
, where i = j.
6
e.g. A =
_
_
1 2 1
2 3 4
1 0 1
_
_
, B =
_
_
2 3 4
1 2 1
1 0 1
_
_
, C =
_
_
1 2 1
6 9 12
1 0 1
_
_
, D =
_
_
1 2 1
0 1 6
1 0 1
_
_
.
A B: swap R
1
, R
2
. A C: replace R
2
by 3R
2
. A D: replace R
2
by R
2
2R
1
.
Effect of the row operations on determinants: let M be a square n n matrix.
1. Changes det(M) into det(M).
2. Changes det(M) into det(M).
3. Does not change det(M).
IMPORTANT: if M is an n n matrix, then det(M) =
n
det(M), not det(M), because
we do operation 2 to each of the n rows.
IMPORTANT: we can do row operations on matrices of any size; however, for determinants,
we must have square matrices.
To invert a matrix A by row operations: Play a game: write A, I next to each other:
(AI). Use row operations to try to change the lefthand matrix A into I, the identity matrix.
Each time, do the same row operations to the righthand matrix. You win when you get (IB);
in this case, B = A
1
.
If you ever get a noninvertible matrix on the left or right, then the original matrix A was
also noninvertible.
If A is invertible then it is always possible to win the game (and nd A
1
by this method).
However, if A is invertible then there are many different possible moves leading to a win.
EXPLANATION: each row operation corresponds to multiplying on the left by an invertible
matrix M. For example:
_
_
0 1 0
1 0 0
0 0 1
_
_
_
_
a
1
a
2
a
3
a
4
b
1
b
2
b
3
b
4

_
_
=
_
_
b
1
b
2
b
3
b
4
a
1
a
2
a
3
a
4

_
_
,
_
_
1 0
0 1 0
0 0 1
_
_
_
_
a
1
a
2
a
3
a
4
b
1
b
2
b
3
b
4

_
_
=
_
_
a
1
+b
1
a
2
+b
2
a
3
+b
3
a
4
+ b
4
b
1
b
2
b
3
b
4

_
_
, . . .
Thus if M
1
, M
2
, M
3
, . . . M
n
are matrices representing our moves, then our game proceeds:
(AI) (M
1
AM
1
) (M
2
M
1
AM
2
M
1
)
(M
n
M
2
M
1
AM
n
M
2
M
1
) = (MAM),
so if ever we reach (IB) = (MAM), then we have I = MA and B = M, so that A
1
= B.
Example 29 Find A
1
, where A =
_
_
1 1 0
1 3 0
3 1 1
_
_
. One possible sequence of moves is:
_
_
1 1 0
1 3 0
3 1 1
_
_
_
_
_
_
1 0 0
0 1 0
0 0 1
_
_

_
_
1 1 0
0 4 0
3 1 1
_
_
_
_
_
_
1 0 0
1 1 0
0 0 1
_
_

_
_
1 1 0
0 1 0
3 1 1
_
_
_
_
_
_
1 0 0
1/4 1/4 0
0 0 1
_
_
7

_
_
1 0 0
0 1 0
3 1 1
_
_
_
_
_
_
3/4 1/4 0
1/4 1/4 0
0 0 1
_
_

_
_
1 0 0
0 1 0
3 0 1
_
_
_
_
_
_
3/4 1/4 0
1/4 1/4 0
1/4 1/4 1
_
_

_
_
1 0 0
0 1 0
0 0 1
_
_
_
_
_
_
3/4 1/4 0
1/4 1/4 0
10/4 2/4 1
_
_
, so nally A
1
=
_
_
3/4 1/4 0
1/4 1/4 0
5/2 1/2 1
_
_
=
1
4
_
_
3 1 0
1 1 0
10 2 4
_
_
.
ROW ECHELON FORM
Denition 30 A matrix is in row echelon form if the number of zero entries from the left in each
row increases (strictly) as we go down the rows (unless a whole row is zero, in which case all
lower rows remain zero also).
For example, the following matrices are in row echelon form:
B
1
=
_
_
1 0 2 1
0 3 0 0
0 0 0 4
_
_
, B
2
=
_
_
0 11 3 1
0 0 0 7
0 0 0 0
_
_
, B
3
=
_
0 2 17
0 0 0
_
, B
4
=
_
_
0 1
0 0
0 0
_
_
but the following matrices are NOT in row echelon form:
_
_
0 0 1
0 2 0
0 0 0
_
_
,
_
_
4 0 0
0 0 1
0 0 2
_
_
,
_
_
1 1 1 0
0 0 1 1
0 1 0 0
_
_
.
Fact 31 If A is any matrix, we can use row operations to convert it to another matrix B in row
echelon form. B is called a row echelon form reduction of A. We always have
Rank of A = The number of rows of B which are not all zero.
Explanation (***) It is not hard to show that row operations do not change the rank (because
the matrix representing any row operation is invertible). Although, note that the range space
DOES change; it is only its dimension which is xed. (Exercise: if V is a vector subspace of W
and T : W W is an invertible linear transformation, then dim(V ) = dim(T(V )).)
Thus Rank(A) = Rank(B). Now, the formula for Rank(B) becomes obvious if you think
long enough about how B acts on vectors, and use the Row Echelon Form property: its similar
to the proof that upper triangular matrices with nonzero diagonal entries are invertible by
back substitution starting from the lowest variables.
(Sorry this is just a messy bit of computation; I cant think of a really nice proof right now.
I think its possible to do something with induction and removing unnecessary columns, but it
still seems fairly messy).
Thus, the row echelon form is useful for calculating the rank of a matrix (and hence also the
nullity). In the example just above, B
1
, B
2
, B
3
, B
4
have ranks 3, 2, 1, 1, and nullities 1, 2, 2, 1
respectively (remember the RankNullity Formula above).
FINDING BASES FOR THE RANGE SPACE AND NULL SPACE
Let A be any matrix, B any row echelon form reduction of A.
Fact 32 If B has exactly k rows not identically zero, choose any linearly independent collection
of k columns from B. The corresponding k columns of A give a basis of the range space of A.
8
Proof: (***) Let b
n
1
, b
n
2
, . . . , b
n
k
be the column vectors of columns n
1
, n
2
, . . . , n
k
of B,
and a
n
1
, a
n
2
, . . . , a
n
k
the corresponding column vectors of A. We already know that k is the
dimension of the range space (by Fact 31); so we only need to prove linear independence.
a
n
j
= Ae
n
j
and b
n
j
= Be
n
j
for each j, where e
1
, e
2
, . . . are the standard basis vectors. Now
B = MA for some invertible M, so b
n
j
= Be
n
j
= MAe
n
j
= M(Ae
n
j
) = Ma
n
j
.
CLAIM: a
n
1
, a
n
2
, . . . , a
n
k
are linearly independent.
PROOF: let

j

j
a
n
j
= 0. Then 0 = M
_

j

j
a
n
j
_
=

j

j
Ma
n
j
=

j

j
b
n
j
. There-
fore
1
=
2
= =
k
= 0 because b
n
1
, b
n
2
, . . . , b
n
k
are linearly independent (by our
choice). Thus a
n
1
, a
n
2
, . . . , a
n
k
are linearly independent.
Example 33 let us reduce a matrix to row echelon form:
A =
_
_
_
_
1 1 2 3 4
1 2 0 3 2
0 0 0 0 0
0 0 0 1 2
_
_
_
_

_
_
_
_
1 1 2 3 4
0 1 2 0 6
0 0 0 0 0
0 0 0 1 2
_
_
_
_

_
_
_
_
1 1 2 3 4
0 1 2 0 6
0 0 0 1 2
0 0 0 0 0
_
_
_
_
= B,
by the row operations New R
2
= (Old R
2
) + R
1
, followed by Swap(R
3
, R
4
). Now B has 3
nonzero rows. Columns 1, 3, 4 of B are linearly independent; therefore, a basis for the range
space of A is
_
_
_
_
1
1
0
0
_
_
_
_
,
_
_
_
_
2
0
0
0
_
_
_
_
,
_
_
_
_
3
3
0
1
_
_
_
_
. Other choices are possible: columns 1, 2, 4, or columns
1, 2, 5, or columns 1, 3, 5 would also work (since they are obviously linearly independent.
NOTE: columns 2, 4, 5 or columns 3, 4, 5 are also linearly independent, so would also work;
but this is not obvious.
REMARK: Finding linearly independent columns of a reduced row echelon matrix is very
easy; just look at the lowest nonzero entries in each column. Its similar to showing that upper
triangular matrices with nonzero diagonal entries are always invertible by direct solution of lin-
ear equations. Try lots of examples and it becomes obvious! Its really a kind of algorithmic
understanding.
Theorem 34 Every basis for the null space of B is also a basis for the null space of A.
Proof: B = MA, so Au = 0 = Bu = MAu = M(0) = 0. Therefore (null space of A)
(null space of B). But A = M
1
B, so similarly (null space of B) (null space of A). Thus
A, B have the same null space. So any basis for one null space will also be a basis for the other.
This is useful because it is always easy to nd a basis for the null space of a matrix in row
echelon form.
Example: nd the null space of A from Example 33 above. We know the nullity is 5 3 = 2.
Let u =
_
_
_
_
_
_
1
0

_
_
_
_
_
_
, v =
_
_
_
_
_
_
0
1

_
_
_
_
_
_
(since it is obvious that these are linearly independent); now choose
9
the other numbers so that u, v lie in the null space of B (the reduced form of A):
0 = Bu = B
_
_
_
_
_
_
1
0
x
y
z
_
_
_
_
_
_
=
_
_
_
_
1 + 2x + 3y + 4z
2x + 6z
y + 2z
0
_
_
_
_
,
so that 2x +3y +4z +1 = 0, 2x +6z = 0, y +2z = 0. This is easy to solve: we get y = 2z,
x = 3z, (6 6 + 4)z = 1, so that z = 1/8, x = 3/8, y = 1/4.
Similarly, if 0 = B
_
_
_
_
_
_
0
1
p
q
r
_
_
_
_
_
_
, then p =
1
4
, q =
1
2
, r =
1
4
. Hence, the pair
_
_
_
_
_
_
1
0
3/8
1/4
1/8
_
_
_
_
_
_
,
_
_
_
_
_
_
0
1
1/4
1/2
1/4
_
_
_
_
_
_
is a basis for the null space of A. We can clear the fractions to get
_
_
_
_
_
_
8
0
3
2
1
_
_
_
_
_
_
,
_
_
_
_
_
_
0
4
1
2
1
_
_
_
_
_
_
.
EIGENVECTORS, EIGENVALUES, DIAGONALISATION
Denition 35 Let A be an nn matrix, so that A gives a linear transformation A : R
n
R
n
.
If Au = u for some 0 = u R
n
and R, then u is an eigenvector of A with eigenvalue .
IMPORTANT: WE MUST HAVE u = 0.
Exercise 36 If = 0, then u is an eigenvector of A u is an eigenvector of A.
Theorem 37 is an eigenvalue of A I A is singular det(I A) = 0.
Proof: If is an eigenvalue of A with u = 0 an eigenvector, then (I A)u = uu = 0.
Therefore, I A is not invertible (because if it were, we would have u = (I A)
1
(0) = 0,
a contradiction because u = 0.) Thus also det(I A) = 0.
Conversely, suppose that det(I A) = 0, so that I A is singular. Then, there must
be some u = 0 such that (I A)u = 0, by Theorem 27. Then u is an eigenvector with
eigenvalue .
So, YOU CAN FIND EIGENVALUES BY SOLVING det(I A) = 0. This is a polynomial
equation in , of degree n.
[*** Notice that if we use the complex numbers C, then eigenvalues always exist, because
polynomials in C always have solutions in C. But this is NOT ON THE SYLLABUS. ***]
Fact 38 Let
1
,
2
, . . . be DISTINCT eigenvalues of A. Then the corresponding eigenvectors
u
1
, u
2
, . . . are linearly independent.
Proof: (***) Assume that

j

j
u
j
= 0 for some
j
. Then p(A)u
j
= p(
j
)u
j
for every
polynomial p, so
0 = p(A)0 = p(A)
_

j
u
j
_
=

j
p(A)u
j
=

j
p(
j
)u
j
.
10
Now, for each xed k, there is some polynomial p
k
such that
p
k
(
j
) =
_
1 (j = k),
0 (j = k).
For example, take the Lagrange polynomial p
k
(z) =

j
(z
j
)

j=k
(
k

j
)
1
. Now with
p = p
k
above, we get 0 =
k
u
k
, so that
k
= 0 (because u
k
= 0 by denition of eigenvectors).
Therefore
k
= 0 for all k, so u
1
, u
2
, . . . are linearly independent as required.
Fact 39 Let Abe an nn matrix with DISTINCT eigenvalues
1
,
2
, . . . ,
n
. Let u
1
, u
2
, . . . , u
n
be eigenvectors (with
j
the eigenvalue for u
j
). Then A = QDQ
1
, where D is a diagonal
matrix with entries
1
,
2
, . . . ,
n
, and Q has column vectors u
1
, u
2
, . . . , u
n
.
*** [If some of the eigenvalues of A are repeated, so we do not have n distinct eigenvalues,
then it doesnt work; but this is NOT ON THE SYLLABUS!! Sometimes diagonalisation is not
possible; instead we have the Jordan Canonical Form.] ***
Proof: (***) By denition Q has column vectors {u
j
}, so u
j
= Qe
j
. Thus
j
u
j
= Au
j
=
AQe
j
. But also De
j
is the j
th
column of D, so De
j
=
j
e
j
. Thus
QDe
j
= Q(
j
e
j
) =
j
Qe
j
=
j
u
j
= Au
j
= AQe
j
.
Since this is true for every j (and {e
1
, . . . , e
n
} is a basis of R
n
), QD = AQ.
Now the columns of Q are u
1
, u
2
, . . . , u
n
, which are linearly independent (by Fact 38), and
hence a basis of R
n
. Therefore Q is invertible, by two methods:
EITHER: Fact 10 (RUBBISH!!), OR: the Range Space of Q is the column space of Q (Theo-
rem 25), which has dimension n; so the rank of Q is n. Now look at the proof of Theorem 27.
But QD = AQ. Therefore, A = QDQ
1
.
In the exam, you will only have to calculate eigenvectors and eigenvalues for 2 2 and 3 3
matrices, and the eigenvalues will always be distinct and in R (not C).
Main use of diagonalisation: if A = QDQ
1
then A
n
= QD
n
Q
1
, and D
n
is easy to nd
because D is DIAGONAL.
Example 40 Let A = QDQ
1
, where Q =
_
2 1
3 0
_
and D =
_
1 0
0 2
_
. Find A
n
.
We have Q
1
=
1
20(3)1
_
0 1
3 2
_
=
1
3
_
0 1
3 2
_
. Therefore,
A
n
=
1
3
_
0 1
3 2
__
1 0
0 2
n
__
2 1
3 0
_
=
1
3
_
0 2
n
3 2
n+1
__
2 1
3 0
_
=
_
2
n
0
2 2
n+1
1
_
USEFUL CHECK: when n = 0, we get A
0
=
_
1 0
2 2 1
_
=
_
1 0
0 1
_
. Hence A
0
= I.
Note also that the formula is true for n < 0, e.g. A
1
=
_
1/2 0
1 1
_
.
11