Homework 1 PDF
Homework 1 PDF
Homework 1 PDF
Rationale and learning expectations: This homework covers some of the very basic topics from linear algebra, which
are necessary for understanding of most concepts in machine learning. All students, regardless of their targeted grade
in this course, are expected to have complete mastery of the concepts being tested in this homework.
1. Let the matrix B ∈ R2×2 . Provide specific examples of B for the case when:
(a) (1 point) The matrix B is a diagonal matrix.
(b) (1 point) The matrix B is a symmetric matrix.
2. Provide ‘True’ or ‘False’ answers to the following statements:
(a) (1 point) Let x ∈ Rn and A ∈ Rn×n . Then Ax is a column vector and xT A is a row vector.
(b) (1 point) Let x ∈ Rn and A ∈ Rn×n . Then xT Ax is a column vector.
(c) (1 point) Let x ∈ Rn and A ∈ Rn×n . Then (Ax)T = AT xT .
(d) (1 point) Let A ∈ Rn×n and B ∈ Rn×n . Then AB = BA.
3 0 −1
3. Let A := and x := . Evaluate the following expressions:
0 4 2
(a) (2 points) xT x
(b) (2 points) xT A
(c) (2 points) AT x
(d) (2 points) xT Ax
(e) (2 points) trace(A)
(f) (2 points) 2I − A
4. (3 points) Consider the following system of equations:
2x + 3y − 4z = 3
−x − y + 5z = 2.
5. (4 points) Consider a matrix A ∈ R3×8 . Which of the following statements cannot be true? Provide a justification
for your answer in each case.
(a) The rank of A is four.
(b) A has five linearly independent columns.
(c) A has two linearly independent rows.
(d) The inverse matrix A−1 has dimensions 8 × 3.
−1 2
6. (4 points) Let A = . Provide answers to the following, along with justification for each answer.
2 −4
(a) What is the rank of A?
(b) What is the determinant of A?
(c) How many linearly independent columns does A have?
AB = BA = I.
What is B as a function of A?
9. Consider a matrix A ∈ Rn×m .
(a) (2 points) Let AT A = Im . What does this imply about the column vectors of A in terms of angles between
them?
(b) (2 points) Let AAT = In . What does this imply about the row vectors of A in terms of angles between
them?
10. (2 points) Let A ∈ R3×3 and let rank(A) = 3. What is the null space of A? Justify your answer.
11. (2 points) Let A ∈ R3×3 and let rank(A) = 2. How many basis vectors are needed to characterize the null space
of A? Justify your answer.
12. (4 points) Let x ∈ Rn and let {qi }ni=1 be a collection
Pn of ni vectors that form an orthonormal basis of Rn . Suppose
n T
x ∈ R be a vector that can be expressed as x = i=1 2 qi . Derive the value of qn x by showing all steps.
13. (2 points) Let (λ, x) be an eigenvalue–eigenvector pair for a matrix A, where λ ∈ R. Assuming λ = 3, what is
3xT AT ? Show all steps.
14. (3 points) Which of the following statements are true and which are false?
(a) Every matrix has an eigenvalue decomposition (EVD).
(b) Every matrix has a singular value decomposition (SVD).
(c) Eigenvalues are always real valued.
(d) Singular values are always real valued.
(e) Eigenvalues are always non-negative.
(f) Singular values are always non-negative.