Autocorrelation

Chapter 12
The nature of the problem

 Defined as “correlation between members of series

of observations ordered in time (as in time series
data) or space (as in cross-sectional data)

E  ui u j   0 i j
 The CLRM assumes that autocorrelation does not
exist in the disturbance

E  ui u j   0 i j

1/6/2020 Prepared by Sri Yani K 2

The nature of the problem
There are several reason:
1. Inertia
2. Specification bias: excluded variables
3. Specification bias: incorrect functional form
4. Cobweb-phenomenon
5. Lags
6. “Manipulation” of data
7. Data transformation
8. Non-stationarity

1/6/2020 Prepared by Sri Yani K 3

OLS estimation in the presence of
autocorrelation
 The two-variable model: Yt  1   2 X t  ut
 We must assume the mechanism that generates ut
ut   ut 1   t 1    1
  rho  is the coefficient of correlation
 t is the stochastic disturbance term, that it satisfied
the OLS assumption
E  t   0
var   t    2
cov   t ,  t  s   0 s0

1/6/2020 Prepared by Sri Yani K 4

OLS estimation in the presence of
autocorrelation
ut   ut 1   t 1    1
known as Markov first-order autoregressive scheme.
Given the AR(1) scheme, it can be shown that:
 2
var  ut   E  ut2    2
1 
 2
cov  ut , ut  s   E  ut ut  s    s 
1  2
cor  ut , ut  s    s

1/6/2020 Prepared by Sri Yani K 5

OLS estimation in the presence of
autocorrelation
 The OLS estimator is

ˆ2 
 xy t t
and   x
var ˆ2 
2
x 2
t
2
t

Under AR(1) scheme, the variance of estimator is

 
var ˆ2 
2 
1  2 
 xx t t 1
 2 2 x x
t t 2
 ...  2 
n 1 x x
t n


AR 1   x  2
t x 2
t x 2
t x 2
t 
 2  1 r   1 r 
 
var ˆ2
AR 1
 2 
 xt  1  r  
ˆ
  var  2   
OLS 1  r 




r
x xt t 1
, r2 
x xt t 2
, and so on  Does not minimum variance
x 2
t x 2
t

1/6/2020 Prepared by Sri Yani K 6

The BLUE estimation in the presence
of autocorrelation
 The two-variable model and the AR(1) process, the
BLUE estimator is
n

  x   x  y
t t 1 t   yt 1 
ˆ2GLS  t 2
n
C
x  x 
2
t t 1
t 2

2

var ˆ2GLS   n
D
x  x 
2
t t 1
t 2

C and D are a correction factor that may be disregareded

1/6/2020 Prepared by Sri Yani K 7

Consequences of using OLS in the
presence of autocorrelation
 The OLS estimators remain unbiased, consistent, and
asymptotically normally distributed, they are no longer efficient.
As a consequences, the usual t, F, and 2 test cannot be
legitimate applied.
 OLS estimation allowing for autocorrelation: should use GLS

b2

H0: b2= 0

GLS 95% interval

OLS 95% interval

1/6/2020 Prepared by Sri Yani K 8

Consequences of using OLS in the
presence of autocorrelation
 OLS estimation disregarding autocorrelation:
errors will arise for the reasons
1. The residual variance is likely to underestimate the
true residual variance
2. Overestimate R2
3. Even if the residual variance is not underestimated,
the variance of coefficient may underestimate
4. The t and F tests of significance are no longer
valid, and if applied, are likely to give seriously
misleading conclusion about the statistical
significance of the estimated regression coefficients

1/6/2020 Prepared by Sri Yani K 9

Consequences of using OLS in the
presence of autocorrelation
 Under the classical assumption

 
ˆ 2 i
ˆ
u 2

an unbiased estimator of  2

 n  2
 
E ˆ 2   2
 But if there is autocorrelation, given by AR(1)

 2 n   2 1     2  r 
E ˆ  
2

n2
 
if  and r positive  E ˆ 2 <  2

1/6/2020 Prepared by Sri Yani K 10

Detecting Autocorrelation

1. Graphical method
2. The Runs test
3. Durbin-Watson (DW) d test
4. The Breusch-Godfrey (BG) test: a general
test of autocorrelation

1/6/2020 Prepared by Sri Yani K 11

The Runs test (Geary test)

 If there are too many runs  indicating

negative serial correlation
 If there are too few runs  indicating positive
serial correlation
 Now let
N = total number of observations (N1+N2)
N1 = number of (+) symbols (i.e., + residuals)
N2 = number of (-) symbols (i.e., - residuals)
R = number of runs

1/6/2020 Prepared by Sri Yani K 12

The Runs test (Geary test)

 H0: residuals are independent

 Assuming that N1>10 and N2>10, the number of
runs is asymptotically normally distributed with

2 N1 N 2
Mean E  R  1
N
2 N1 N 2  2 N1 N 2  N 
Variance R 
2

 N   N  1
2

1/6/2020 Prepared by Sri Yani K 13

The Runs test (Geary test)

 If the H0 of randomness is sustainable, following

the properties of the normal distribution, we
should expect that

Prob  E  R   1.96 R  R  E  R   1.96 R   0.95

 Decision Rule: do not reject the H0 with 95%
confidence if R lies in the preceding confidence
interval, and reject the H0 if R lies outside these
limits

1/6/2020 Prepared by Sri Yani K 14

Durbin-Watson (DW) d test
 Defined as
  uˆ  uˆ   uˆ   uˆ  2 uˆ uˆ
n 2 2 2
t 1 t 1 t t 1
d t 2

t t

 uˆ  uˆ
n 2 n 2
t 1 t t 1 t

  uˆ uˆ 
d  2 1   setting  uˆ   uˆ
t t 1 2 2
   t 1
 uˆ 
2
t
t

d  2 1  ˆ 
But since  1    1, implies that 0  d  4
ˆ  0, d  2  no autocorrelation
ˆ  1, d  0  perfect positive autocorrelation
ˆ  1, d  4  perfect negative autocorrelation
1/6/2020 Prepared by Sri Yani K 15
Durbin-Watson (DW) d test
 The assumptions underlying the d statistic
1. The regression model includes the intercept term
2. The explanatory variables are nonstochastic
3. The disturbances are generated by AR(1)
4. The error term is assumed to be normally
distributed
5. The regression model does not the lagged value
of dependent variable as one of the explanatory
variables
6. There are no missing observation in the data

1/6/2020 Prepared by Sri Yani K 16

Durbin-Watson (DW) d test
 The mechanism of the DW test are
1. Run the OLS regression and obtain the residuals
2. Compute d
3. Find out the critical dL and dU values
4. Decision Rule:

H0 Decision If
No positive auto correlation Reject 0 < d < dL
No positive auto correlation No decision dL  d  dU
No negative auto correlation Reject 4-dL  d  4
No negative auto correlation No decision 4-dU  d  4-dL
No auto correlation, positive or Do not reject dU  d  4-dU
negative

1/6/2020 Prepared by Sri Yani K 17

Durbin-Watson (DW) d test

Zone of indecision

Reject H01 Reject H01

Positive Do not reject H01 or Negative
autocorrela H02 or both autocorrela
tion tion

0 d
dL dU 2 4-dU 4-dL 4
H01: no positive autocorrelation
H02: no negative autocorrelation

1/6/2020 Prepared by Sri Yani K 18

The Breusch-Godfrey (BG) test

 This test allows for

1. Nonstochastic regressors such as the lagged values
of the regressand
2. Higher-order autoregressive scheme
3. Simple or higher-order moving average of white
noise error terms
 Also known as Lagrange Multiplier (LM) test,
Durbin’s M test (if the residuals follow the
AR(1) scheme )

1/6/2020 Prepared by Sri Yani K 19

The Breusch-Godfrey (BG) test

 The BG test involves the following steps

1. Estimate the model by OLS and obtain the residuals
2. Regress the residuals on the explanatory variables and
the lagged value of the residual
uˆt  1   2 X t  ˆ1uˆt 1  ˆ 2uˆt 2  ...  ˆ puˆt  p   t
3. H0: 1=2=…=p=0
4. If the sample size is large, BG have
n  p R 2 ~  p2
if n  p R 2  the critical  2  reject H0

1/6/2020 Prepared by Sri Yani K 20

Remedial measures
 If we find that there is autocorrelation. We have
four option:
1. Try to find out if the autocorrelation is pure
autocorrelation and not the result of mis-specification of
the model
2. If it is pure autocorrelation, one can use appropriate
transformation of the original model
3. In the large samples, we can use the Newey-West
method to obtain standard error of OLS estimators that
are corrected for autocorrelation
4. In some situations we can continue tu use the OLS
method

1/6/2020 Prepared by Sri Yani K 21

Model Mis-specification versus Pure
Autocorrelation
 In time series data, it is quite possible that the
variables exhibit trends
 Check the possibility of nonlinearity
relationship and the other sources of
autocorrelation
 Make sure that autocorrelation suffer from
pure autocorrelation and not necessarily from
specification bias.

1/6/2020 Prepared by Sri Yani K 22

Correcting for pure autocorrelation:
the GLS method
 The two-variable regression model

Yt  1  2 X t  ut
and assume that the error term follows the
AR(1) scheme
ut  ut 1   t 1    1
 We consider two cases:
1. When  is known
2. When  is not known

1/6/2020 Prepared by Sri Yani K 23

Correcting for pure autocorrelation:
the GLS method
 When  is known
Yt  1   2 X t  ut
Yt 1  1   2 X t 1   ut 1
Yt  Yt 1   1 1      2  X t   X t 1    ut   ut 1 
Yt   1   2 X t   t
Since the error term satisfies the usual OLS
assumption, we can apply OLS and obtain the
BLUE estimators.

1/6/2020 Prepared by Sri Yani K 24

Correcting for pure autocorrelation:
the GLS method
 When  is not known
  is rarely known in practice
 Several possibilities to estimate 
1. The first-difference method
2.  based on Durbin-Watson d statistic
3.  estimated from the residuals
4. Iterative methods of estimating 

1/6/2020 Prepared by Sri Yani K 25

The first-difference method
  lies between 0 and 1
 = 0 no autocorrelation
 = 1 perfect positive or negative autocorrelation
 If assume =1  the first difference equations
Yt  Yt 1   2  X t  X t 1   ut  ut 1 
Yt  2 X t   t
 Appropriate if DW d statistic is quite low or  is high.
Rule of thumb d < R2

1/6/2020 Prepared by Sri Yani K 26

The first-difference method
 The first-difference transformation is valid only if =1
 The Berenblutt-Webb test  to test the hypothesis that
=1. The test statistic is g-statistic


n 2
eˆ
g 2 t

 uˆ
n 2
1 t
 ut  the OLS residuals from the original regression
et  the OLS residuals from the first-difference
regression

1/6/2020 Prepared by Sri Yani K 27

The first-difference method
 To test the significance of the g-statistic,
assuming that the level form regression contains
the intercept term, use the DW tables

 H0: =1
H1: =0

if the g-statistic < the lower limit of d, do not

reject H0 that =1

1/6/2020 Prepared by Sri Yani K 28

 based on Durbin-Watson d statistic
 The relationship between  and d
1
ˆ  1  d
2
 In reasonably large samples, we can obtain 
and use it to transform the data.
 The relationship between  and d may not hold
in small samples

1/6/2020 Prepared by Sri Yani K 29

 estimated from the residuals

 If the AR(1) scheme in residuals is valid, a

simple way to estimate  is to regress the
residuals on the lagged values of residuals

uˆt  ˆ uˆt 1  t
where ut are the residuals obtained from the
original regression, and vt are the error term
of this regression

1/6/2020 Prepared by Sri Yani K 30

Iterative methods of estimating 

 All the methods of estimating  are a single

estimate of 
 The iterative methods  estimate 
iteratively
1. Cochrane-Orcutt iterative procedure
2. Cochrane-Orcutt two-step procedure
3. Durbin two-step procedure
4. Hildreth-Lu scanning or search procedure

1/6/2020 Prepared by Sri Yani K 31

General comments
1. Since the OLS estimators are consistent in large
samples, it makes little difference in 
2. The various methods are basically two-step methods
 feasible (estimated) GLS methods
3. Whenever we use an FGLS or EGLS method, the
estimated coefficients will not necessarily have the
usual optimum properties of the classical model, such
as BLUE, especially in small samples .
4. In using EGLS, if we do not include the first
observation, not only the numerical values but also the
efficiency of the estimators can be adversely affected.

1/6/2020 Prepared by Sri Yani K 32

The Newey-West Method of Correcting
the OLS Standard Errors
 The corrected standard errors are known as
HAC (heteroscedasticity- and autocorrelation-
consistent) standard errors or Newey-West
standard errors
 This procedure is strictly speaking valid in
large samples and may not be appropriate in
small samples

1/6/2020 Prepared by Sri Yani K 33

OLS vs FGLS and HAC

OLS Unbiased, consistent, and asymptotically

normally distributed, but not efficient.
The t, F, and 2 tests is no longer appropriate

FGLS & Efficient, but properties for small sample are

HAC not well documented.
Griliches & Rao (1969): if the sample
relatively small and  < 0.3, OLS is as good
or better than FGLS.

1/6/2020 Prepared by Sri Yani K 34

Forecasting with Autocorrelated Error
Terms
 Dynamic forecasting
Yt  1   2 X t  ut
ut   ut 1   t
Yt  1   2 X t   ut 1   t
If we want to forecast Yt 1 , we obtain
Yt 1  1   2 X t 1   ut   t 1
The estimated value is: Yˆt 1  ˆ1  ˆ2 X t 1  ˆ uˆt
Yˆ  ˆ  ˆ X  ˆ 2uˆ
t 2 1 2 t 2 t

and so on
1/6/2020 Prepared by Sri Yani K 35
Additional Aspect of Autocorrelation

 Dummy Variables and Autocorrelation

 ARCH and GARCH Models
 ARCH if the error variance is related to the squared
error term in the previous term
 GARCH if the error variance is related to the squared
error terms several periods in the past
 Coexistence of autocorrelation and
Heteroscedasticity

1/6/2020 Prepared by Sri Yani K 36