Eigenvalues and eigenvectors Why we care The characteristic polynomial

MATH 4A - Linear Algebra with Applications

Lecture 20: Eigenvalues and eigenvectors

17 May 2019

Reading: §5.1-5.4
Recommended problems from §5.1: 1, 3, 5, 7, 11, 15, 17, 19, 21,
22, 23, 31, 32
Recommended problems from §5.2: 1-21 odd, 22
Eigenvalues and eigenvectors Why we care The characteristic polynomial

Lecture plan

1 Eigenvalues and eigenvectors

2 Why we care

3 The characteristic polynomial

Eigenvalues and eigenvectors Why we care The characteristic polynomial

Motivation
Our goal for the next several lectures is to understand the structure
of a linear transformation T : V → V from a vector space V to
itself. This is a nice special case (instead of more generally
considering linear transformations where the domain and codomain
are different), because it easier to compare a vector v to its image
T (v) when these two vectors live in the same vector space.

To be more concrete, we will mostly think about linear

transformations from Rn to itself. Thus, we will be thinking about
n × n (or square) matrices.

The basic idea is that (often) a linear transformation T : Rn → Rn

has vectors on which it acts particularly simply. Moreover, the set
of such vectors (often) forms a basis of Rn , so if we express the
matrix for T in this basis (instead of the standard basis) it is
particularly simple.
Eigenvalues and eigenvectors Why we care The characteristic polynomial

Example

Consider the matrix  

3 −2
A=
1 0
and the vectors    
−1 2
u= v= .
1 1
Then    
−5 4
Au = Av = = 2v.
−1 2
So, A does something complicated to u, but when A acts on v, the
result is the same as scalar multiplying v by 2, which is pretty
simple.
Eigenvalues and eigenvectors Why we care The characteristic polynomial

Definition

An eigenvector of an n × n matrix A is a nonzero vector x such

that Ax = λx for some scalar λ. We call the scalar λ an eigenvalue
of A if there is a nontrivial solution x of Ax = λx; such an x is
called an eigenvector corresponding to λ.

Remarks:
1 We require an eigenvector x 6= 0 because otherwise 0 would be
an eigenvector for any choice of λ. We only want eigenvalues
that are eigenvalues for an interesting/nontrivial reason.
2 An eigenvalue is the same thing as a scalar such that the
homogeneous linear system (A − λIn )x = 0 has a nontrivial
solution. (See board.) Any nontrivial solution is an
eigenvector.
Eigenvalues and eigenvectors Why we care The characteristic polynomial

Eigenvector testing: iClicker 1

Testing whether or not a vector v is an eigenvector of the square

matrix A is easy: Just matrix multiply Av and determine if the
result is a multiple of v. For example, let:
     
1 6 6 3
A= u= v= .
5 2 −5 −2

Are the vectors u and v eigenvectors of A?

1 yes, both are
2 no, neither are
3 u is, but v is not
4 v is, but u is not
5 eigenwhat?
Eigenvalues and eigenvectors Why we care The characteristic polynomial

An aside: another characterization of invertibility

By the way: we don’t allow the zero vector 0 to be an eigenvector.

However, we do allow the zero scalar 0 to be an eigenvalue.

In fact, 0 is an eigenvalue of the square matrix A if and only if A is

not invertible.

(Proof: 0 is an eigenvalue if and only if Ax = 0 has nontrivial

solution if and only if A is not invertible.)
Eigenvalues and eigenvectors Why we care The characteristic polynomial

Eigenvalue testing

Testing whether a given scalar λ is an eigenvalue of A is not as

easy as testing if a vector is an eigenvector, but it’s still pretty easy:

Let A be a square matrix and let λ be a given scalar. To decide if

λ is an eigenvalue, we should decide if (A − λI )x = 0 has
nontrivial solutions. As usual, to do this, we should row reduce the
augmented matrix

(A − λI ) 0
and determine if there are any free variables (that is, if any of the
first n columns are not pivot columns).
Eigenvalues and eigenvectors Why we care The characteristic polynomial

iClicker 2

Let  
1 6
A= .
5 2
Is 7 an eigenvalue of A? (Hint:
 
−6 6
A − 7I = .)
5 −5

1 yes
2 no
3 eigenwhat?
Eigenvalues and eigenvectors Why we care The characteristic polynomial

iClicker 3

Let  
1 6
A= .
5 2
Is 1 an eigenvalue of A? (Hint:

A − 1I = · · · .)

1 yes
2 no
3 eigenwhat?
Eigenvalues and eigenvectors Why we care The characteristic polynomial

Given an eigenvalue, how do we find a corresponding

eigenvector?

Suppose you knew that λ is an eigenvalue for A, but you didn’t

know any eigenvectors corresponding to λ. How do we find one?

Well, just find a nontrivial solution of (A − λI )x = 0! To do so,

find the reduced echelon form of


(A − λI ) 0 .

Since λ is an eigenvalue, this is guaranteed to have free variables.

Using the reduced echelon form, you can write down the general
solution. Any nontrivial solution will be an eigenvector
corresponding to λ.
Eigenvalues and eigenvectors Why we care The characteristic polynomial

Example
We saw that λ = 7 is an eigenvalue of
 
1 6
A= .
5 2

Let’s find an eigenvector.

   
−6 6 augment + row operations. . . 1 −1 0
A − 7I = −−−−−−−−−−−−−−−−−→
5 −5 0 0 0

General solution:  
1
x2 .
1
   
1 −23
Thus, and are both examples of eigenvectors
1 −23
corresponding to the eigenvalue λ.
Eigenvalues and eigenvectors Why we care The characteristic polynomial

Definition

Given an eigenvalue λ of the n × n matrix A, the set of

eigenvectors of A corresponding to A, together with 0, is called the
eigenspace of A corresponding to λ.

Remarks:
1 the eigenspace of A corresponding to the eigenvalue λ is the
set of solutions to the homogeneous equation (A − λI )x = 0,
hence it is a subspace of Rn . This justifies the name
“eigenspace,” instead of just, say, “eigenset.”
2 the method we outlined on the previous two slides to find an
eigenvector for an eigenvalue can easily be extended to find a
basis of the eigenspace corresponding to λ: just find the
parametric vector form of the general solutions to the
homogeneous equation (A − λI )x = 0. In particular, the
dimension of the eigenspace is the number of free variables.
Eigenvalues and eigenvectors Why we care The characteristic polynomial

Caveat

While we use row reduction to find an eigenvector corresponding

to a given eigenvalue, in general there is no particularly useful
relation between the eigenvalues of row equivalent matrices.

In other words, the echelon form of A doesn’t really help us find

the eigenvalues of A. We’ll have to do something else. (This is
what the characteristic equation is for...)
Eigenvalues and eigenvectors Why we care The characteristic polynomial

Given a matrix A, it acts on its eigenspace corresponding

to λ by dilation by λ
For example, λ = 2 is an eigenvalue of the 3 × 3 matrix
 
4 −1 6
A = 2 1 6 .
2 −1 8
The eigenspace is spanned by
   
1 −3
u = 2 and v =  0  .
0 1
If x = c1 u + c2 v is any vector in the eigenspace, then

Ax = λx = 2x.

You should imagine the eigenspace as a 2-dimensional plane in R3 ,

and A simply stretches every vector in this plane by a factor of 2.
Eigenvalues and eigenvectors Why we care The characteristic polynomial

If we could find “enough” eigenvectors, we could use

linearity to understand how A acts in general

More precisely, suppose that we have a vector x that can be

written as a linear combination of eigenvectors v1 , v2 , . . . , vl :

x = c1 v1 + c2 v2 + · · · + cl vl

and suppose the eigenvectors v1 , v2 , . . . , vl correspond to

eigenvalues λ1 , λ2 , . . . , λl . . Then

Ax = A(c1 v1 + c2 v2 + · · · + cl vl )
= c1 Av1 + c2 Av2 + · · · + cl Avl
= c1 λ1 v1 + c2 λ2 v2 + · · · + cl λl vl .

Thus, with these assumptions, matrix multiplying is quite easy,

because we never actually have to matrix multiply—we just scalar
multiply instead!
Eigenvalues and eigenvectors Why we care The characteristic polynomial

Punchline (to be made more precise in coming days)

In other words, changing bases from the standard basis of Rn to an

“eigenbasis” of A (that is, a basis of Rn in which every vector is an
eigenvector of A) would make A act like a diagonal matrix!

Thus, if A has an eigenbasis, then the action of Rn can be

understood by dilating each eigenspace by the corresponding
eigenvalue, and figuring out what happens to other vectors by
using linearity of matrix multiplication. This is what we meant by
studying the “structure” of a linear transformation.
Eigenvalues and eigenvectors Why we care The characteristic polynomial

Preliminary result

An important step in making the previous slides more precise is the

following result:
Theorem
If v1 , . . . , vr are eigenvectors that correspond to distinct
eigenvalues λ1 , . . . , λr of an n × n matrix A, then the set
{v1 , . . . , vr } is linearly independent.

Proof: see board.

Eigenvalues and eigenvectors Why we care The characteristic polynomial

Now that we are motivated to care about eigenvalues and

eigenvectors of a square matrix A, how do we find them?

Contrary to everything we’ve learned so far in this class, row

reduction is not very helpful for this. In general, we will use
something called the characteristic polynomial of A. More on that
in a second. For now, we note that for certain special matrices, it
is sometimes very easy to find the eigenvalues and eigenvectors.
Theorem
The eigenvalues of a triangular matrix are the entries on its main
diagonal.
Eigenvalues and eigenvectors Why we care The characteristic polynomial

Example

The eigenvalues of  
−1 −5 −1
 0 1147 7 
0 0 34
are −1, 1147 and 34. The eigenvalues of
 
−25 0 0 0
 −6 −1 0 0
 
 0 −1 1 0
−3 1 −5 −1

are −25, −1 and 1.

Eigenvalues and eigenvectors Why we care The characteristic polynomial

Motivation

We have seen that testing if any given vector (scalar) is an

eigenvector (eigenvalue, resp.) of a n × n matrix A is easy. But
how can we actually find the eigenvalues and eigenvectors without
guesswork? In this case, using guesswork to search for a needle in a
haystack is not feasible, because there is an infinite amount of hay!

To rectify this, we will now associate a certain degree n polynomial

to A called the characteristic polynomial. The roots of this
polynomial will be the eigenvalues of A.
Eigenvalues and eigenvectors Why we care The characteristic polynomial

Warm-up example

Consider the 2 × 2 matrix

 
−1 4
A= .
8 −3

An eigenvalue of A is any scalar λ such that (A − λI )x = 0 has

nontrivial solutions. For any λ, this equation has nontrivial
solutions if and only if A − λI is not invertible. But a matrix is not
invertible if and only if it has determinant zero.

Therefore, λ is an eigenvalue of A if and only if det(A − λI ) = 0.

Eigenvalues and eigenvectors Why we care The characteristic polynomial

Note that in our example,

 
−1 − λ 4
det(A − λI ) = det
8 −3 − λ
= (−1 − λ)(−3 − λ) − (4)(8) = λ2 + 4λ − 29

is a degree 2 polynomial.

By the previous slide, the roots of this polynomial—that is, the

values of λ that make it 0—are exactly the eigenvalues of A. In
this example, we can use the quadratic equation to find that the
two eigenvalues of A are
√ √
−4 + 132 −4 − 132
and
2 2
Eigenvalues and eigenvectors Why we care The characteristic polynomial

Definition

Let A be an n × n matrix. The characteristic polynomial of A is

the degree n polynomial (with variable λ):

det(A − λIn ).

The characteristic equation of A is the equation

det(A − λIn ) = 0.

For the same reasons described in the previous example, the

eigenvalues of A are exactly the solutions of the characteristic
equation of A. In other words, the eigenvalues of A are the roots
of the characteristic polynomial.
The multiplicity of λ as a root of the characteristic polynomial of
A is called the multiplicity of the eigenvalue.
Eigenvalues and eigenvectors Why we care The characteristic polynomial

Example
The characteristic polynomial of
 
−1 1 −12 0 0
 0 1 −1 9 −37
 
A= 0 0
 0 0 −4 

0 0 0 −2 1 
0 0 0 0 −2
is
 
−1 − λ 1 −12 0 0
 0
 1 − λ −1 9 −37 
det(A − λI5 ) = det  0
 0 0−λ 0 −4  
 0 0 0 −2 − λ 1 
0 0 0 0 −2 − λ
= (−1 − λ)(1 − λ)(0 − λ)(−2 − λ)(−2 − λ)
= λ5 + 2λ4 − 3λ3 − 4λ2 + 4λ.
Eigenvalues and eigenvectors Why we care The characteristic polynomial

Remark

The characteristic polynomial allows us to convert the question of

finding eigenvalues (which, a priori, involves solving the non-linear
system of equations Ax = λx in n + 1 unknowns) to solving a single
polynomial (in particular, non-linear) equation in one unknown.

The nonlinearity is annoying and a source of serious difficulty, but

that’s still (in principle) quite an improvement.