Math 19B: linear algebra and probability Oliver Knill, Spring 2011

3/23-25/2011: Midterm exam

Your Name:
Please enter your name in the above box. You have to do this exam on your own.
Answer each question on the same page as the question is asked. If needed, use the back or
the next empty page for work. If you need additional paper, write your name on it.
Do not detach pages from this exam packet or unstaple the packet.
Please write neatly Answers which are illegible for the grader can not be given credit.
Unless stated, give computations, details reasons.
No calculators, computers, or other electronic aids are allowed. There is no hour time limit.
90 minutes should suce and 3 hours are reasonable. You have to chose one day from the
23-25th and do the exam during that day. Staple it together and bring it to class on Friday.
The exam is open book in the sense that you can use the class lecture notes and any
handwritten notes of yours but no other books or text resources.
1 10
2 10
3 10
4 10
5 10
6 10
7 10
8 10
9 10
10 10
Total: 100
Problem 1) TF questions (10 points)
Solution:
All the TF questions are true except, 1,7,9
1) T F
Two events A, B which are disjoint satisfy P[A B] = P[A] + P[B].
Explanation:
Solution:
It is the union, not the intersection.
2) T F
Two events A, B which are independent satisfy P[A B] = P[A] P[B].
Explanation:
Solution:
By denition
3) T F
The empty set is always an event in any probability space.
Explanation:
Solution:
By denition, the entire probability space is there and so also the complement.
4) T F
If T is a linear map which satises T(
_
1
1
_
) =
_
1
1
_
and T(
_
1
1
_
) =
_
1
1
_
, then T is orthogonal.
Explanation:
Solution:
Use the vectors as a basis. They form an orthogonal system.
5) T F
The vectors
_

_
1
1
1
_

_ ,
_

_
1
1
0
_

_ ,
_

_
1
0
0
_

_ are linearly independent.

Explanation:
Solution:
row reduce
6) T F
The expectation of the random variable X = (2, 3, 5) is 10/3.
Explanation:
Solution:
It is 2+3+5 divided by 3.
7) T F
The standard deviation of the random variable X = (5, 6, 7) is 1/3.
Explanation:
Solution:
The mean is 6 so that X E[X] = (1, 0, 1) which has length

2.
8) T F
Let A, B be arbitrary 2 2 matrices. If a vector x is in the kernel of A,
then it is in the kernel of BA.
Explanation:
Solution:
Ax = 0 implies BAx = 0. But ABx is not necessarily 0.
9) T F
Let A, B be arbitrary 2 2 matrices. If a vector x is in the kernel of B,
then it is in the kernel of BA.
Explanation:
Solution:
Ax = 0 implies BAx = 0. But ABx is not necessarily 0.
10) T F
The least square solution of the linear system of equations Ax = y is a real
unique solution of the system if A is invertible.
Solution:
One can see this from the formula for example. Since (A
T
A)
1
= A
1
A
T
, one has
(A
T
A)
1
A
T
= A
1
.
Problem 2) (10 points)
Match the matrices with their actions:
A-J domain codomain A-J domain codomain
A=
_
1 0
2 1
_
B=
_
1 0
1 2
_
C=
_
1 0
1 1
_
D=
_
0 1
1 0
_
E=
_
0 1
1 2
_
F=
_
0 1
2 1
_
G=
_
1 1
1 0
_
H=
_
1 1
0 1
_
I=
_
1 1
1 2
_
J=
_
1 1
2 1
_
Solution:
B, F, A, G, C, H, D, I, J, E
Problem 3) (10 points) Systems of linear equations
a) (6 points) Find the general solution of the following system of linear equations using row re-
duction.
x + y + z + u + v = 4
x y + z u + v = 0
x y + z = 2
b) (2 points) The solutions x =
_

_
x
y
z
u
v
_

_
can be written as x =

b +V , where

b is a particular solution
and V is a linear space. What is the dimension of V ?
c) (2 points) Which of the three cases did appear: exactly one solution, no solution or innitely
many solutions?
Solution:
Row reduce the matrix
A =
_

_
1 1 1 1 1 4
1 1 1 1 1 0
1 1 1 0 0 2
_

_ .
It becomes
rref(A) =
_

_
1 0 1 0 1 2
0 1 0 0 1 0
0 0 0 1 1 2
_

_ .
a) Write the system with the row reduced matrix and introduce free variables. The
solution can be written as [2, 0, 0, 2, 0]
T
+ s[1, 0, 1, 0, 0]
T
+ t[1, 1, 0, 1, 1]
T
, where s and
t are free variables.
b) The dimension of the solution space is 2 . This is the dimension of the kernel of the
matrix A which appears in the equation Ax = b.
c) The system is consistent. But we have innitely many solutions .
Problem 4) (10 points) Random variables, independence
Lets look at the following two vectors in R
4
:
X =
_

_
7
5
4
2
_

_
, Y =
_

_
1
1
2
2
_

_
.
a) (2 points) The two vectors span a linear space V . Write V it as an image of a matrix A.
b) (3 points) Find the space W of all vectors perpendicular to X, Y . Write W as the kernel of a
matrix B.
We can see these matrices also as random variables over the probability space {1, 2, 3, 4 }. For
example X(1) = 7, X(2) = 5, X(3) = 4, X(4) = 2.
c) (2 points) Check that the two random variables X, Y are uncorrelated. What does Pythagoras
tell about Var[X + Y ]?
d) (3 points) Are the two random variables X, Y independent random variables?
Solution:
a) A =
_

_
7 1
5 1
4 2
2 2
_

_
.
b) B =
_
7 5 4 2
1 1 2 2
_
row reduces to
_
1 0 7 6
0 1 9 8
_
. Its kernel is spanned by
[7, 9, 1, 0]
T
and [6, 8, 0, 1]
T
. c) V ar[X + Y ] = Var[X] + Var[Y ].
d) No, P[X = 7, Y = 1] = P[X = 7]P[Y = 1] is not true.
Problem 5) (10 points) Basis and Image
a) (5 points) Find a basis for the kernel of the following diamond matrix:
A =
_

_
0 0 0 0 8 0 0 0 0
0 0 0 8 8 8 0 0 0
0 0 8 8 8 8 8 0 0
0 8 8 8 8 8 8 8 0
8 8 8 8 8 8 8 8 8
0 8 8 8 8 8 8 8 0
0 8 8 8 8 8 8 8 0
0 0 0 8 8 8 0 0 0
0 0 0 0 8 0 0 0 0
_

_
.
b) (5 points) Find a basis for the image of the matrix A.
Solution:
Row reduce the matrix A to get
A =
_

_
1 0 0 0 0 0 0 0 1
0 1 0 0 0 0 0 1 0
0 0 1 0 0 0 1 0 0
0 0 0 1 0 1 0 0 0
0 0 0 0 1 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
_

_
.
We see that the rst 5 columns are pivot columns and that the last 4 columns are redun-
dant columns.
a) The 4 vectors
B
Ker
=
_

_
_

_
0
0
0
1
0
1
0
0
0
_

_
,
_

_
0
0
1
0
0
0
1
0
0
_

_
,
_

_
0
1
0
0
0
0
0
1
0
_

_
,
_

_
1
0
0
0
0
0
0
0
1
_

_
_

_
form a basis for the kernel.
b) The rst 5 column vectors of the matrix A form a basis for the image:
B
Ran
=
_

_
_

_
0
0
0
0
8
0
0
0
0
_

_
,
_

_
0
0
0
8
8
8
0
0
0
_

_
,
_

_
0
0
8
8
8
8
8
0
0
_

_
,
_

_
0
8
8
8
8
8
8
8
0
_

_
,
_

_
8
8
8
8
8
8
8
8
8
_

_
_

_
Problem 6) (10 points) Inverse and Coordinates
Let
S =
_
1 1
1 1
_
and
A =
_
1 1
1 1
_
.
a) (4 points) Find the inverse of S by row reducing the 2 4 matrix [S|I
2
].
b) (2 points) Assume the matrix A describes the transformation in the standard basis. Find the
matrix B in the basis given by the column vectors of S.
c) (2 points) Two of the three matrices A, B, S are similar. Which ones?
Solution:
a) S
1
=
_
1 1
1 1
_
/2.
b) B = S
1
AS = S because A = S.
c) All three matrices A, B, S are similar.
Problem 7) (10 points) Expectation, Variance, Covariance
The vectors X =
_

_
1
2
3
1
5
7
_

_
and Y =
_

_
1
1
1
1
1
2
_

_
can be seen as random variables over a probability space
with 6 elements. They encode two dierent data measurements, where 6 probes were taken.
a) (2 points) Find the expectations E[X], E[Y ].
b) (2 points) Find the variances Var[X] and Var[Y ].
c) (2 points) Find the standard deviations [X] and [Y ].
d) (2 points) What is the covariance Cov[X, Y ] of X and Y ?
e) (2 points) Find the correlation Corr[X, Y ] of X and Y .
Solution:
a) E[X] = 19/6, E[Y ] = 1/2.
b) Var[X] = E[X
2
] E[X
2
] = 173/36
Var[Y ] = E[Y
2
] = E[Y
2
] = 5/4
c) [X] =

173/6, [Y ] =

5/2.
d) Cov[X, Y ] = (E[XY ] E[X]E[Y ]) = 23/12.
e) Corr[X, Y ] = (E[XY ] E[X]E[Y ])/([X][Y ]) = (23/12)/Sqrt[173 5/(36 4)] =
23/

865.
Problem 8) (10 points) Combinatorics and Binomial Distribution
a) (2 points) We throw 20 coins. What is the probability that 5 heads show up?
b) (3 points) We throw 7 dice. What is the probability that 3 of the dice show the number 5?
c) (5 points) Assume that the random variable X counts the number of heads when throwing a
coin 20 times. What is the expectation of this random variable?
Solution:
a)
_
20
5
_
(1/2)
5
(1/2)
5
.
b)
_
7
3
_
(1/6)
3
(5/6)
4
.
c) 20 1/2 = 10.
Problem 9) (10 points) Bayes formula
a) (7 points) We throw 10 coins. What is the probability that the rst coin is head if we know
that 5 times heads comes up?
b) (3 points) Is the following argument correct? Whether your answer is yes or no, give a
short reason. The chance that an earth quake hit is P[A] = 1/1000. The chance that a tsunami
hits is P[B] = 1/1000. Therefore, the chance that an earth quake and a tsunami hit both is
P[A] P[B] = 1/1

000

000. The event that a quake and tsunami hit at the same time is therefore
a one in a million event.
Solution:
a) A = {X
1
= 1} B = {X = 5}. We use the Bayes formula
P[A|B] = P[B|A]P[A]/(P[B|A] + P[B|A
c
] .
P[B|A] = B(9, 4)(1/2)
4
(1/2)
5
. P[B|A
c
] = B(10, 5)(1/2)
5
(1/2)
4
. P[A] = 1/2. The result
is 1/2.
b) The two events are not necessarily independent. Actually they are often correlated
since Earthquakes and Tsunamis can occur together if the epicenter is in the ocean.
Problem 10) (10 points) Data tting
Fit the following data using functions of the form f(x) = ax + bx
3
.
x y
1 0
1 1
1 2
-1 1
Solution:
Write down the system of equations
a + b = 0
a + b = 1
a + b = 2
a b = 1
This is in matrix form Ax = b, where
A =
_

_
1 1
1 1
1 1
1 1
_

_
.
Now form A
T
A =
_
4 4
4 4
_
and A
T
b = {2, 2}. The matrix A
T
A is not invertible. Indeed,
we could have seen that the kernel of A is nontrivial. It is a rare case, where the tting
problem does not have a unique solution. The gure shows what is going on. There are
many cubic curves of this type which have the same least square property. It is the three
data points which are on top of each other which produced this ambiguity. Moving just
one data point a bit away, would have produced us a unique solution.