1 Basic Geometric Intuition: For Example, See Theorems 6.3.8 and 6.3.9 in Lay's Linear Algebra Book On The Syllabus

Advanced Econometric Methods I
Exercises for Session 1

Vladislav Morozov
1 Basic Geometric Intuition

Consider the linear model from the lectures:
𝑦 = 𝑋𝛽 + 𝜀 (1)
Label the 𝑘th column of 𝑋 𝑥(𝑘) (don’t confuse it with the vector of covariates of unit 𝑖 𝑥𝑖 ).
a In what space does the vector 𝑦 lie? What is the dimension of column space of 𝑋? Draw
a picture for 𝑛 = 2, 𝑘 = 1 and for 𝑛 = 3, 𝑘 = 1, 2.
b Show that 𝑋𝑒 = 0 for the OLS vector of residuals 𝑒. State this is as an orthogonality condi-
tion involving 𝑒 and the regressors. Show that there are unique 𝑒 and 𝑧 ∈ span{𝑥(1) , . . . , 𝑥(𝑘) }
such that and 𝑒 is orthogonal to 𝑧 and ‖𝑒‖ is minimized1 .
c What happens if 𝑦 lies in the linear span of columns of 𝑋? Draw a picture for 𝑛 = 3, 𝑘 = 2
d What happens if 𝑋 is not full column rank? Draw a picture for 𝑛 = 3, 𝑘 = 2. The vector
𝑧 obtained in b is unique. How can this be reconciled with possible non-uniqueness of the
solution to 𝑋 ′ 𝑋𝑏 = 𝑋 ′ 𝑦?
2 Residuals and Linear Changes of Regressors

Consider the linear model
𝑦 = 𝑋𝛽 + 𝜀 (2)
where 𝑋 is 𝑛 × 𝑘. Suppose we have an alternative collection of regressors 𝑍 such that
𝑍 = 𝑋𝐶 for some full rank 𝑘 × 𝑘 matrix 𝐶. Consider the linear model 𝑦 = 𝑍𝛾 + 𝑢
1. If E(𝜀|𝑋) = 0, what can you say about E(𝜀|𝑍)? What relation should 𝛽 and 𝛾 satisfy
for E(𝑢|𝑍) = 0?
ˆ from the two models are equal. Show

2. Argue geometrically that the residuals 𝑒 and 𝑢
this algebraically.
3 Under IID Normality OLS Gives UMVUE

Consider the linear model
𝑦 = 𝑋𝛽 + 𝜀 (3)
1
For example, see theorems 6.3.8 and 6.3.9 in Lay’s linear algebra book on the syllabus
1
Let 𝜀 ∼ 𝑁 (0, 𝐼) and 𝑋 be a deterministic 𝑛 × 𝑘 matrix of rank 𝑘 (just so that we don’t have
to deal with statements conditional on 𝑋).
a Show that 𝑏 is a sufficient statistic for 𝛽. It is also complete (by theorem 6.2.25 in Casella
and Berger, but you don’t have to show this).
b Conclude that 𝑋𝑏 is the UMVUE of E(𝑦).
c Show this without using completeness and sufficiency of 𝑏 (Hint: 1.5.16 in Hayashi)
4 Nonlinear Conditional Expectation and Attempting To

Make It Linear
Suppose that √︀
E(𝑦𝑖 |𝑋) = 𝛽0 + 𝛽1 𝑥𝑖 (4)
The ’intuitive’ option is to estimate (𝛽0 , 𝛽1 ) by running the linear regression
𝑦𝑖2 = 𝛽0 + 𝛽1 𝑥𝑖 + 𝜀𝑖 (5)
a Show that if E(𝑦|𝑥), then 𝑦 = 𝑥 + 𝜀 with E(𝜀|𝑥) = 0.
b Suppose 𝑦𝑖 is not a deterministic function of 𝑥𝑖 . Argue using Jensen’s inequality why the
’intuitive’ suggestion will not lead to nice results.
√
c Confirm this by defining 𝑢𝑖 = 𝑦𝑖 − 𝛽0 + 𝛽1 𝑥𝑖 and computing E(𝑦𝑖2 |𝑋). What can and
cannot be estimated from this?
5 OLS As a Method Of Moments Estimator

Consider the linear model:
𝑦 = 𝑋𝛽 + 𝜀 (6)
under all the assumptions from the lectures including E(𝜀|𝑋) = 0. Propose a method of
moments estimator for 𝛽 and compute its covariance matrix conditional on 𝑋.
6 Recursive Updating for OLS

Algebraic, optional. In some applications data may arrive sequentially, one observation after
another, and the only thing of interest is the OLS estimate 𝑏𝑛 . This problem shows how to
recompute it when new data arrives without storing the complete data set. Suppose we have
𝑛 observations available, housed in (𝑦𝑛 , 𝑋𝑛 ), 𝑋𝑛 𝑛 × 𝑘, 𝑛 > 𝑘, 𝑋𝑛 maximal rank. Let
−1
𝑏𝑛 = (𝑋𝑛′ 𝑋𝑛 ) 𝑋𝑛′ 𝑦𝑛 (7)
2
be the estimate on the first 𝑛 observations. Suppose a new observation comes along (𝑦𝑛+1 , 𝑥𝑛+1 )
comes along. Establish a recursion which allows us to update 𝛽𝑛 using new information
without recomputing from scratch using the following steps.
1. Define the matrix

[︂(︂ )︂′ (︂ )︂]︂−1
𝑋𝑛−1 𝑋𝑛−1 [︀ ′ ′ −1
]︀
𝐴𝑛 = = 𝑋 𝑛−1 𝑋 𝑛−1 + 𝑥 𝑛 𝑥𝑛 (8)
𝑥′𝑛 𝑥′𝑛
so that 𝑏𝑛 = 𝐴𝑛 𝑋𝑛′ 𝑦𝑛 .
2. Establish a recursion to compute 𝐴𝑛 from 𝐴𝑛−1 and 𝑥𝑛 .

Hint: use the Sherman-Morrison lemma: let 𝐴 be invertible and 𝑢, 𝑣 be real-valued
column vectors. 𝐴 + 𝑢𝑣 ′ is invertible iff 1 + 𝑣 ′ 𝐴−1 𝑢 ̸= 0. If so, then
−1 𝐴−1 𝑢𝑣 ′ 𝐴−1
(𝐴 + 𝑢𝑣 ′ ) = 𝐴−1 − (9)
1 + 𝑣 ′ 𝐴−1 𝑢
3. Setting
1 (︀ ′
)︀−1
𝐾𝑛 = (︀ )︀−1 𝑋𝑛−1 𝑋𝑛−1 𝑥𝑛 (10)
′
1 + 𝑥′𝑛 𝑋𝑛−1 𝑋𝑛−1 𝑥𝑛
𝐴𝑛−1 𝑥𝑛
= (11)
1 + 𝑥′𝑛 𝐴𝑛−1 𝑥𝑛
show that
𝑏𝑛 = 𝑏𝑛−1 + 𝐾𝑛 (𝑦𝑛 − 𝑥𝑛 𝑏𝑛−1 ) (12)
(the term in the denominator of 𝐾𝑛 is essentially the influence of observation 𝑥𝑛 ,
showing how much attention should 𝑏𝑛 pay to it. See p. 21 in Hayashi for a bit more
on this)
4. How much memory does this approach need? (In simple words, how many numbers do
you have to store to do a step of this iteration?)

1 Basic Geometric Intuition: For Example, See Theorems 6.3.8 and 6.3.9 in Lay's Linear Algebra Book On The Syllabus

Uploaded by

Copyright:

Available Formats

1 Basic Geometric Intuition: For Example, See Theorems 6.3.8 and 6.3.9 in Lay's Linear Algebra Book On The Syllabus

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

1 Basic Geometric Intuition: For Example, See Theorems 6.3.8 and 6.3.9 in Lay's Linear Algebra Book On The Syllabus

Uploaded by

Copyright:

Available Formats

Advanced Econometric Methods I

Exercises for Session 1

1 Basic Geometric Intuition

2 Residuals and Linear Changes of Regressors

ˆ from the two models are equal. Show

3 Under IID Normality OLS Gives UMVUE

b Conclude that 𝑋𝑏 is the UMVUE of E(𝑦).

4 Nonlinear Conditional Expectation and Attempting To

a Show that if E(𝑦|𝑥), then 𝑦 = 𝑥 + 𝜀 with E(𝜀|𝑥) = 0.

5 OLS As a Method Of Moments Estimator

6 Recursive Updating for OLS

1. Define the matrix

2. Establish a recursion to compute 𝐴𝑛 from 𝐴𝑛−1 and 𝑥𝑛 .

You might also like