Handout for Sem II 2020 PD

Time Series
Data Generating Process
The process of realisation of a time series data is known as the data generating process
(DGP). The underlying factors determining the process are stochastic or random and thus
DGP is a stochastic process.
In statistical sense, a time series is a random variable ordered in time. Therefore, a time series
variable can be analysed in terms of its probability distribution.
The stochastic process of a time series {yt} is defined by the marginal distribution
function obtained from this joint distribution:
 
f t  yt    ....  f  y1 , y2 ,.... yT dy1dy2 ...dyt 1dyt 1....dyT
 

The marginal distribution function defined in the above equation is characterised by the first
and second order moments:
t  E yt 
 t2  E yt  t 2  Eyt2   t2
 t ,k  cov yt , yk   E yt  t  yk  k   E yt yk   t k

The DGP of a time series is of two types: stationary and non-stationary. Stationary and
nonstationary processes are different in their properties, and they require different inference
procedures.

A time series variable, yt, is generated by a stationary process and the variable will be stationary
if the marginal distribution of the variable is time invariant.
For a time series variable {yt} = (y1, y2, y3,…..yT), the T-dimensional distribution function is
defined by
Ft  y1 , y2 ,.... yT   PY1  y1 ,......YT  yT 
The DGP of {yt} is said to be first order stationary, if the one dimensional distribution function
of the series is time-invariant:
Ft  yi   Ft k  yi  )
The process will be second order stationary if

Ft yi , y j   Ft k yi , y j 
In this way, the process is said to be Tth order stationary if
Ft  y1, y2 ,..... yT   Ft k  y1, y2 ,.... yT 
A time series process is strictly stationary or strongly stationary if the first and second order
moments of the marginal distribution are time independent, and the series is Tth order
stationary:

1. E( yt )  E( yt 1 )  ..  
2. E[( yt   ) ]  E[( yt 1   ) ]  ..   y  Vary
2 2 2

3. E[( yt   )( yt  k   )]  E[( yt  j   )( yt  j  k   )]  ..   k  Cov( yt  j yt  j  k )

4. Ft  y1, y2 ,..... yT   Ft k  y1, y2 ,.... yT 

A time series is said to be weakly stationary or covariance stationary if its first and second
order moments are unaffected by a change in time origin. In other words, if conditions 1 to 3
are satisfied, then yt is called covariance stationary, or weakly stationary.

A time series variable is nonstationary if the marginal distribution of {yt} is time varying.
Nonstationary process is a state of statistical disequilibrium exhibiting trend. A stationary series
does not follow any trend. For nonstationary time series, either mean or variance or both are
time decaying:
1. E ( yt )   t 

2. E ( yt )   t 
2 2

3. E ( yt yt k )  Cov( yt  j yt  j k )   k t 

Stationary Time Series

We will discuss the type of DGP followed by the series and find out the restrictions under
which the series becomes stationary.
Autoregressive Process (AR)
If a time series variable is generated by its past values and random error, the DGP is called
autoregressive (AR) process.

The AR process of order p, AR (p), is specified as:

yt  0  1 yt 1  2 yt 2  ..... p yt  p   t
For, p =1, (2.3.1) reduces to AR(1)
yt  0  1 yt 1   t
For p =2, the series becomes AR(2)

yt  0  1 yt 1  2 yt 2   t
It is convenient to use a time-series operator called the lag operator to express equations in
compact form. The lag operator, L(.), is a mathematical operator or function in which the
argument is an element of a time series. By applying the lag operator to yt we get its predecessor
yt–1:
yt 1  Lyt ,
yt 2  Lyt 1  LLyt   L2 yt

Similarly, yt  p  L yt
p

AR(1) process

Suppose that a time series variable, yt, follows AR(1) process. The DGP of yt following AR(1)
is described as

yt  0  1 yt 1   t

By setting t = 1, 2, 3 and so on in this equation we have the following solution,

t 1 t 1
yt  0  1i  1t y0   1i t i
i 0 i 0
Here, yt could be interpreted as an aggregation of the entire history of innovations.

If time span is sufficiently large and 1  1 , the time path of yt will be

0 
yt    1i t i
1  1  i0
This process could not be useful for higher order autoregression.

By using lag operator, AR(1) equation could be expressed as

1  1Lyt  0   t
Or,
yt  0 1  1   1  1L  t
1 1
By expanding the polynomial function in right hand side we get the similar result under the
restriction that 1  1 :

0 
yt    1i  t i
1  1  i 0

AR(2) process
The AR(2) process of yt is specified as

yt  0  1 yt 1  2 yt 2   t
By using lag operator, the AR(2) process is expressed as

1   L   L y    
1 2
2
t 0 t

The lag polynomial function or, AR inverse characteristic polynomial for AR(2) is

 L  1  1 L  2 L2
And AR inverse characteristic equation is

 L  1  1L  2 L2  0
The corresponding AR characteristic equation is

 z   z 2  1 z  2  0
1
Here, z 
L

The roots of the quadratic characteristic equation are easily found to be

1  12  4 2
z1 , z2 =
2

We will discuss below that the stochastic behaviour of AR(2) process depends on the nature of
the roots.

Example 1

Consider the following AR(2) process

yt  0.75 yt 1  0.125 yt 2   t .
The lag polynomial for this process is 1  0.75L  0.125L2 and the characteristic equation is

z 2  0.75z  0.125  0 .

We can find that the characteristic roots are z1 = 0.50 and z2 = 0.25. Both roots are real and less
than one in absolute value, so this AR(2) process is stationary.

Example 2

yt  1.25 yt 1  0.25 yt 2   t

The lag polynomial is 1  1.25L  0.25L2 .

The corresponding characteristic equation is

z 2  1.25z  0.25  0 .

The characteristic roots are z1 = 1, and z2 = 0.25. In this case, one root is equal to unity and the
other is less than unity, and the AR(2) process is nonstationary containing 1 unit root.

Example 3

Let, yt  4.2  0.2 yt 1  0.35 yt 2   t

The AR characteristic equation is

z 2  0.2 z  0.35  0

The characteristic roots, z1 = 0.7 and z2 = -0.5, both are less than 1 in absolute value, and the
series will be stationary

Example 4

yt  1.6 yt 1  0.9 yt 2   t

Here, AR characteristic equation is

z 2  1.6 z  0.9  0

The characteristic roots will be imaginary and the homogeneous solution has the form

yth  A1r t cost  A2 

r 0.9  0.949

1.6
  cos 1 1
 0.567
20.92

Therefore,

yth  A1 (0.949)t cos0.567t  A2 

Therefore, the AR process is stationary.

Moving Average (MA) process

The MA process of order q, MA (q), is specified as

yt   t  1 t 1  2 t  2  .........  q t  q
The term moving average arises from the fact that yt is obtained by applying the weights 1, θ1,
…… θq to the innovations εt, εt-1, …….εt-q and then moving the weights forward to εt+1, εt,
…….εt-q+1 to get yt+1.

For q =1 in (2.4.1), the series will be MA(1):

yt   t  1 t 1

Or, by using lag operator it can be expressed as,

 t  1  1L1 yt  1  1L  12 L2  13 L3  ......yt


Or, yt    1  yt i   t
i

i 0

Therefore, MA(1) process converts into AR (∞) process provided that 1  1 .

For MA(1), the mean, variance and covariance of yt are

E(yt) = 0

V  yt   1  12  2 
Cov y t , y t  k   E  t  1 t 1  t  k  1 t  k 1 
 E  t  t  k   1 E  t  t  k 1   1 E  t 1 t  k   12 E  t 1 t  k 1 

  2 1  12 , for  k 0
 1  2 , for k 1
 0, for k 1

The mean, variance and covariance of MA(1) series are time independent. Therefore, yt
following MA(1) is stationary. Although, yt is a function of white noise variable, it will not be
white noise, because the covariance is not zero for k = 1.
The autocorrelation function for MA(1) process is 0 beyond lag length 1. This fact is important
for forecasting.

For MA(2) series,

yt   t  1 t 1   2 t 2
the mean, variance and covariance are calculated as follows

E  yt   0

V  yt   1  12   22  2 
Cov y t , y t  k   E  t  1 t 1   2  t  2  t  k  1 t  k 1   2  t  k 2 
 E  t  t  k   1 E  t  t  k 1    2 E  t  t  k  2   1 E  t 1 t  k   12 E  t 1 t  k 1 
 1 2 E  t 1 t  k  2    2 E  t 2  t  k    21 E  t 2  t  k 1    22 E  t 2  t  k  2 

  2 1  12   22 , for  k 0
 1 1   2 , for k  1
2

  2 2 , for k2
 0, for k  2

Therefore, mean, variance and covariance are time invariant, but covariance is not equal to zero
for k = 1 and k = 2.

For an MA(2) process,

1 1   2 
Corr  yt , yt 1   1 
1  12   22

2
Corr  yt , yt 2    2 
1  12   22
ρk = 0, for k > 2
Autoregressive Moving Average (ARMA) process
The ARMA (p, q) model is specified as

yt  0  1 yt 1  2 yt 2  ...   p yt  p   t  1 t 1  ..   q t q
In this equation, there are p autoregressive terms and q moving average terms. This type of
stochastic process is referred to as an autoregressive moving average process of order (p, q), or
an ARMA(p, q) process. If q = 0, the process is a pure autoregressive process of order p, AR(p).
Similarly, if p = 0, the process is a pure moving average of order q, MA(q).

By using lag operator, we can rewrite the ARMA(p, q) process as

 Lyt  0   L t
The left-hand side is the autoregressive part of the process which is the homogeneous
difference equation containing p lags and the right-hand side is the moving average part
containing q lags plus a drift that allows the mean of y to be non-zero.

Consider, first the ARMA (1,1) model:

yt  0  1 yt 1   t  1 t 1
We can expresse it as

0 
yt   y01t   1i 1 1  1  t i
1  1 i 1

Autocorrelation Function
Autocorrelation refers to the correlation of a time series variable with its own past and future
values.
Autocorrelation is defined as the ratio of autocovariance to variance of a time series variable:
cov yt , yt  k   k
k  
var  yt  0
The graphical representation of ACF is called correlogram.

The AR(1) form of the demeaned series is

zt  1 zt 1   t zt  yt  
Multiplying both sides of (10.6.6) by zt-k,

zt zt k  1 zt 1 zt k   t zt k
Taking expectations on both sides,

E( zt zt k )  1E( zt 1 zt k )  E( t zt k )
Or,  k  1 k 1
Here, εt, εt-1, ……εt-k-1 are independent of yt-k

Or,  k  1  0
k

k
Therefore,  k   1k
0

The autocorrelation function is converging as long as |ϕ1| < 1. Therefore, |ϕ1| < 1 is a necessary
and sufficient condition for covariance stationarity of the AR(1) process. The condition |ϕ1| <
1 also assures that the AR process is ergodic.

For stationary series, ACF converges to 0 as k increases. In AR(1), as |ϕ1| < 1

lim k  0
k 

If 1  0 , ρk converges monotonically, but if  1  0, it converges with dampened oscillatory

path around 0.

The AR(2) form of yt in mean deviation form is specified as

zt  1 zt 1  2 zt 2   t

where zt  yt  E yt 
Multiplying both sides by zt-k,

zt zt k  1 zt 1 zt k  2 zt 2 zt k   t zt k
After taking expectations on both sides,

 k  1 k 1  2 k 2
Or,  k  1 k 1  2  k 2
This equation is called the Yule-Walker equation.
It is valid for k ≥ 2
For k = 0, 1
1
1 
1  2
For k= 2,

 2  1 1  2
or ,
12  2  2 1  2 
2   2  1
1  2  1  2
We can solve the second order difference equation to find out the ACF function for AR(2)
which is similar to the time path of yt . The explicit solution depends critically on the
characteristic roots. For real and unequal roots, the solution will be

 k  A1h1k  A2 h2k
at k = 0, A1  A2  1
at k = 1, 1  A1h1  A2 h2
Therefore,
1  A1h1  1  A1 h2

1  h2
or, A1 
h1  h2
h1  1
or, A2 
h1  h2
1  h2 k h1  1 k
Therefore,  k  h1  h2
h1  h2 h1  h2

Or,  k 
1  h2 h1k  h1  1 h2k
h1  h2
Here,

1  12  42 1  12  42

h1  , h2 
2 2

1   12  42
1  h2   1
1  2 2

21  1  12  1  2  12  42 1 1  2   1  2  12  42

Or, 1  h2  
21  2  21  2 
 1  12  42 1
h1  1  
2 1  2

1  12  1  2  12  42  21 1  2  12  42  1 1  2 

Or, h1  1  
21  2  21  2 

and h1  h2  12  42 ]

Therefore,  k 
1  h h
1
2 k 1
2 
 1  h22 h1k 1 
h2  h1 1  h1h2 
If the roots are real and equal, 12  42  0 . In this case,
 k  A1h k  A2 khk

at k = 0, A1 = 1
at k =1, 1  h  A2 h
1
  h 1  2 2 1  2
A2  1  1  1 
h 1 1  2 1  2
2
k
Therefore,  k  1  1  2 k  1 
 
 1  2  2 

If the roots are complex, 12  42  0 and

sin k   
k  r k
sin 
where, r   2
 1 
  cos 1  
 2  
 2 

 1  2 
  tan 1  
 1  2 

Therefore, in the case of AR(2), the autocorrelation function can assume a wide variety of
shapes. In all cases, the magnitude of ρk dies out exponentially fast as the lag k increases if
stationarity restrictions are satisfied. In the case of complex roots, ρk displays a damped sine
wave behaviour with damping factor r, frequency θ, and phase Φ.

Autocorrelation function for MA(1)

We consider a moving average process of order 1, MA(1):
yt   t  1 t 1
Multiplying both sides of by yt-k and taking expectations,

E( yt yt k )  E( t yt k )  1E( t 1 yt k )
For k = 0,
 0   2  12 2  1  12  2

For k = 1,

 
E yt yt 1  E  t yt 1   1 E  t 1 yt 1 
or ,  1  1 2

Therefore,
1
1 
1  12

For k  2,  k  k  0
Therefore, the MA(1) process has no correlation beyond lag 1.

Autocorrelation function for MA(2)

Consider the moving average process of order two:

yt   t  1 t 1   2 t 2

Or, E( yt yt k )  E( t yt k )  1 E( t 1 yt k )   2 E( t 2 yt k )

For k=0,

E ( yt yt )  E ( t yt )  1 E ( t 1 yt )   2 E ( t 2 yt )
or ,  0   2  12 2   22 2  (1  12   22 ) 2

For k=1,

E ( yt yt 1 )  E ( t yt 1 )  1 E ( t 1 yt 1 )   2 E ( t 2 yt 1 )
or ,  1  1 2   21 2

For k=2

E ( yt yt 2 )  E ( t yt 2 )  1 E ( t 1 yt 2 )   2 E ( t 2 yt 2 )
or ,  2   2 2
For k ≥ 3

k  0
Therefore, for an MA(2) process,
1 1   2 
1 
1  12   22
2
2 
1  12   22

 k  0 , for k ≥ 3
Therefore, the MA(2) process has no correlation beyond lag 2.

Autocorrelation function for ARMA process

We can start with ARMA(1, 1) model:

zt  1 zt 1   t  1 t 1

Now, E zt  t   
2

Ezt  t 1   1  1  2
Multiplying both sides of (10.6.24) by zt-k, and taking expectations

E( zt zt k )  1E( zt 1 zt k )  E( t zt k )  1E( t 1 zt k )

Therefore,

 0  1 1   2 1  1 (1  1 )
 1  1 0  1 2
Substituting the value of γ1

 0  1 1 0  1 2    2 1  1 (1  1 )

 2 1  211  12 
0 
1  12

 k  1 k 1 , for k ≥ 1
By solving the simple recursion gives

k 
1  11 1  1   k 1
1  211  12
1

The autocorrelation function decays exponentially with the increase in lag length k.
Partial Autocorrelation Function (PACF)

The partial correlation between two variables measures the degree of relationship between
them which is not explained by the correlations of other variables. For example, if we regress
a variable y on other variables x1, x2, and x3, the partial correlation between y and x3 is the
correlation between y and x3 after eliminating the effects of x1 and x2 on y.
Formally, we can define partial correlation as
cov y, x3 | x1 , x2 
* 
V  y | x1 , x2 V x3 | x1 , x2 

For a time series yt, the partial autocorrelation between yt and yt-k is defined as the conditional
correlation between yt and yt-k. The condition imposed on yt-k+1, ... , yt-1, the set of observations
that lie between the time points t and t−k.

cov yt , yt k | yt 1 , yt 2 ,..... yt k 1 
 k* 
V  yt | yt 1 , yt 2 ....., yt k 1 V  yt k | yt 1 , yt 2 ,... yt k 1 

Alternatively,

 k*  corr  yt , yt k | yt 1 , yt 2 ,......, yt k 1 

Or,  k  corr  yt  E yt | yt 1 , yt 2 ,.... yt k 1 , yt k 

*


Or,  k  corr yt  k1 yt 1  k 2 yt 2  ......  k ,k 1 yt k 1 , yt k
*

Or,  k
*
 corr  k ,k yt k   t , yt k   kk
Therefore, the partial autocorrelation at lag k is equal to the estimated AR(k) coefficient in an
autoregressive model with k terms.

Partial autocorrelation for AR series

The convenient way to find the PACF is to first construct the demeaned series by subtracting
the mean of the series from each observation and use the mean corrected series to estimate the
model. After estimating the model the coefficient for yt-k will be the partial autocorrelation
coefficient of order k.
Since there is no intervening value in AR (1), both the autocorrelation and partial
autocorrelation between yt and yt-1 are the same:

For AR (1): yt  11 yt 1   t

PAC is
1*  11  1
For AR (2):
yt  21 yt 1  22 yt  2   t

The coefficient of yt-2 in the linear regression of yt on yt-1, and yt-2 is the partial correlation
coefficient of order 2:
 2*  corr yt , yt 2 | yt 1   corr yt  21 yt 1 , yt 2   corr 22 yt 2   t , yt 2   22

Partial autocorrelation for MA series

An invertible MA(q) process could be converted into AR(∞) process and the PACF will never
cut off as for the AR(p) with finite p.

It can be shown that PACF of MA(1) is

 1 k 1  12  , for k ≥ 1
 kk  
1  12k 1
Similarly, an invertible ARMA model has an infinite AR representation, hence the PACF will
not cut off.

The PACF of MA models behaves like ACF for AR models and PACF for AR models
behaves like ACF for MA models.

The shape of the ACF and PACF are described in short for different types of DGP of a time
series in the following Chart.

ACF PACF
WN rs  0, s  0 ss  0, s  0
AR(1) Exponential Decay: Spike at lag 1 (at p for AR(p))
rs   s
11  r1 ,ss  0 for s  2
  0  direct decay
  0  oscillating
MA(1) Positive (negative) spike Oscillating (geometric) decay
at lag 1 for   0 (   0 ) for 11  0 ( 11  0 )
rs  0 for
s2
ARMA(1,1) Exponential (oscillating) decay Oscillating (exponential) decay
at lag 1 if   0 (   0 ). at lag 1, 11  1
Decay at lag q for ARMA(p,q) Decay after lag p for ARMA(p,q)
Sample Autocorrelation Function

Sample autocorrelation is based on the sample observations, and is defined as

T

  y t  y  yt k  y 
rk  t  k 1
T

 y  y
2
t
t 1

for any integer k.

For large sample, rk is normally distributed with mean k.

The sample ACF plays an important role in identifying an appropriate stochastic process. For
systematic inference concerning ρk, we need the sampling distribution of the estimator rk.
It can be shown that, for iid random disturbance with finite variance, the sample autocorrelation
rk, is approximately identically independently normally distributed with mean 0 and variance
1/T for T large. Hence, approximately 95 percent of the sample autocorrelations should fall
between the bounds ±1.96/√T.

The statistic used to test autocorrelation is defined as

k
Q  T  ri 2
i 1

If {yt} has a finite variance, Q is approximately distributed as χ2, the sum of squares of the
independent N(0, 1) random variables, with k degrees of freedom.

A large value of Q suggests that the sample autocorrelations of the data are too large to be a
sample from an iid sequence. We, therefore, reject the iid hypothesis of yt at level α if Q > χ21−α
(k) where χ21−α (k).

Ljung and Box (1978) formulated a test in which Q is replaced by

ri 2 k
QLB  T (T  2)
i 1 T  i

The distribution of this statistic is better approximated by the χ2 distribution with k degrees of
freedom.
Nonstationarity with unit root
Many economic and financial time series like asset prices, exchange rates and GDP exhibit
trending behaviour or nonstationarity either in mean, or in variance, or in both.
The characteristic roots of the autoregressive or moving average polynomial of an ARMA
process is helpful to determine the nature of stochastic behaviour of a time series. The
characteristic roots of AR characteristic equation less than unity implies that the series is
stationary. When the characteristic roots lie on or near the unit circle we can say that the series
contains unit root. The presence of unit root in autoregressive process implies that the series is
nonstationary exhibiting stochastic trend with or without deterministic trend. A series with unit
roots follows random walk. If a series contains unit root, it has no tendency to return to a long
run deterministic path. The variance of the series with unit root is time dependent and increases
over time. This type of non-stationarity produces permanent effects from random shocks. A
series with no unit roots having constant mean and variance is stationary, and that the effects
of shocks on the series dissipate over time. This feature is crucial for economic forecasting.

To understand the concept of unit root and the behaviour of AR process, let we start with the
AR(1):

yt  1 yt 1   t ,
or, (1  1L) yt   t

The characteristic equation corresponding to the AR(1) process is

1
z  1  0 , where z 
L

The value of z is the characteristic root. If the characteristic root equals unity, then yt contains
unit root. Therefore, yt following AR(1) process will contain unit root when

z  1  1
In this case the AR(1) will be
yt  yt 1   t

or, yt   t , the change of a time series is purely stochastic

Here, the current value of the variable, yt, is its value one period before plus the current period
shock.
If y0 is a given initial condition, the general solution is
t 1
yt  y0    t i
i 0

In a random walk process, yt is the sum of its initial value and the accumulation of shocks. In
time series literature, the accumulation of shocks is known as stochastic trend.
In this case we have,

E  yt   y0
Therefore, the unconditional mean of yt does not exhibit any trend.

The variance and covariance of yt are:

2
 t 1 
V  yt   E   t  i   t 2
 i 0 

 t 1   t 1 
Cov yt , yt  k   E   t i    t  k  i   t  k  2
 i 0   i 0 
Trend in variance is the outcome of stochastic trend.

The variable yt generated through the random walk model is non-stationary exhibiting
stochastic trend. But after taking a difference of the series it becomes stationary:
y y   t

Therefore, the time series, yt, following purely a stochastic change over time exhibits stochastic
trend, and the stochastic process of the series is known as the difference stationary process
(DSP).

Consider now AR(2) series:

yt  0  1 yt 1  2 yt 2   t
the inverse characteristic equation is

1  1L  2 L2  0

1
By setting z  we get the characteristic equation:
L

z 2  1 z  2  0

1  12  4 2
The characteristic roots, z 
2

Sum of the roots, z1+z2 = ϕ1

Product of the roots, z1×z2 = - ϕ2

When at least one of the characteristic roots equal to unity, the AR (2) series will contain unit
root.

If z1 = z2 = 1, there are two unit roots. In this case, ϕ1=2, and ϕ2= -1

The stochastic behaviour of yt with 2 unit roots will be

yt  0  2 yt 1  yt 2   t

In the case of AR (2), maximum number of unit root = 2, and in this case the series to be
differenced twice to have stationary series and the series will be integrated of order 2.

If z1 = 1 and z2 < 1, or z2 = 1 and z1 < 1, the AR(2) series has one unit root, then we have to
take 1st difference to make it stationary, and in this case it will be integrated of order 1.

If z1 < 1 and z2 < 1, the AR(2) series has no unit root, the series will be stationary and it is
called integrated of order 0.

Non-stationarity without unit root

A nonstationary series without unit root exhibits only deterministic trend. Suppose that a series,
yt, always changes by the same fixed amount from one period to the next:
yt  

Or, yt  yt 1  
The general solution to this linear difference equation is
yt  y0  t
Here, yt is exhibiting deterministic linear time trend.
The actual time behaviour of yt is described by the following way:

yt    t   t ,  t ~ N 0,  2 ,  cov t ,  t k   0
The population regression function is

E  yt     t
This is the popular trend model used to estimate growth rate. The trend in mean value is
completely predictable and is known as the deterministic trend.
The variance of yt,
 
V  yt   E yt  E  yt   E  t2   2
2
The variance of yt is time invariant and dose not exhibit any trend.
Cov yt , yt  k   Eyt  E yt yt  k  E yt  k   E t t  k   0
While variance and covariance are time invariant, the mean of yt is time variant. Thus, yt is
nonstationary time series exhibiting deterministic trend. Deterministic trend is a systematic
change of the mean level of a series over time.

After detrending the series it becomes stationary:

zt = yt – E(yt) = εt
Therefore, yt is originally non-stationary, but after de trending it becomes stationary. The data
generating process (DGP) of yt follows in this case is known as the trend stationary process
(TSP).
The issue whether a time series is of DSP or TSP is important because the dynamic properties
of the two processes are different. While the TSP is predictable, the DSP is not completely
predictable. For TSP, the effects of external shocks are temporary, while for DSP any random
shock to the series has a permanent effect. A TSP has a trend in the mean but no trend in the
variance, but a DSP has a trend in the variance with or without trend in the mean1.

TSP DSP
1. Non stationary 1. Non stationary
2. Stationary by detrending 2. Stationary by taking difference
3. Trend in mean – deterministic trend 3. Trend in mean – deterministic trend,
4. No trend in variance or covariance – or no trend in mean (without
no stochastic trend deterministic trend)
5. Effect of shock is transitory, it dies 4. Trend in variance or covariance –
out shortly stochastic trend. The stochastic
trend incorporates all the random
shocks (ε1 to εt) that have permanent
effects on the level of yt.
5. Effect of shock is permanent, it
persists for long period

Unit Root Test

The existence of unit roots is often suspected by visual inspection of the autocorrelation
function (ACF) and data plots. As long as the ACF decays slowly, the time series should be
considered having at least one unit root and the operation of differencing the time series may
be performed repeatedly to obtain a stationary time series. Many statistical tests for unit roots
are based on autoregression tests of linear dependence.

1
A random walk without drift has no trend in the mean values of the variable.
Dickey-Fuller tests assumed that a series has at most one unit root. Dickey and Fuller (1979)
developed a procedure for testing whether a variable has a unit root. The null hypothesis is that
the variable contains a unit root, and the alternative is that the variable was generated by a
stationary process.
We can express AR(1) in three different alternative ways as

yt  1 yt 1   t

yt  0  1 yt 1   t

yt  0  1 yt 1  t   t
Autoregressive unit root tests are based on testing the null hypothesis that the root of the
autoregressive polynomial is unity, ϕ1 = 1, against the alternative hypothesis that ϕ1 < 1.

H 0 : 1  1
H 1 : 1  1

In the third specification, the null hypothesis is that yt follows a unit root with drift and a time
trend in the regression.
H 0 : 1  1,   0
H 1 : 1  1,   0

However, such a regression is likely to be affected by serial correlation. To control for that, the
Dickey–Fuller test estimates a model of the following form:
yt  yt 1   t

yt  0  yt 1   t

yt  0  yt 1  t   t
The hypothesis is
H0 :   0
H1 :   0

or
H0 :   0,   0
H1 :   0 ,   0

Augmented Dickey-Fuller (ADF) unit root test

The Dickey-Fuller test is associated with AR(1) process. If a time series follows autoregressive
process of higher order we need to incorporate some augmented terms to convert it into AR(1).
Consider that yt follows AR(2) process:
yt  0  1 yt 1  2 yt 2   t
We can modify the model as
yt  0  1 yt 1  2 yt 2   t
 0  (1  2 ) yt 1  2  yt 1  yt 2    t
 0  1  2  yt 1  1yt 1   t

Here, 1  2

yt  0  yt 1  1yt 1   t ,   1  2  1

In equation (3.4.16), 1yt 1 is called the augmented term and the model is augmented Dickey
Fuller (ADF) model. The unit root test by using ADF model is called ADF unit root test. If yt
follows AR(2), we need 1 augmented term to convert it into AR(1). Therefore, it is ADF of
order 1. The AR(1) series is ADF of order 0.

Cointegration
Classical econometric theory assumes that the random disturbance is white noise and this
assumption remains valid when the variables used in a regression model are stationary.
Consider a linear relationship between two time series variables:

x1t  1   2 x2t   t

If the series, x1t and x2t, are stationary, the random disturbance,  t  x1t  1   2 x2t , will be
stationary and we can consistently estimate the parameters by using OLS
Suppose that x1t denotes GDP and x2t denotes number of car accident over time. In this example,
there is no reason that x1t and x2t are causally related. But, this relationship may produce
empirical results in which the R2 is quite high, and the Durbin-Watson statistic is quite low.
This is the symptom for spurious regression because of non-stationary random walks behaviour
of the variables.

Investigating the relationship between nonstationary variables relates to the problem of

cointegration.
Cointegration implies co-movement of the two or more unit root variables with similar
stochastic trends. It provides long run equilibrium relationship between two or more unit root
variables. The long run relationships are closely linked to the concepts of equilibrium
relationships in economic theory and of persistent co-movements of economic time series in
econometrics.
The equilibrium relationship holds automatically when the series themselves are stationary, or
the error in equilibrium is stationary. The long run relationship is in equilibrium when the linear
combination of the unit root variables is stationary.

In a two variable framework, if x1t and x2t both are integrated of order 1 containing single unit
root and their linear combination, x1t  1   2 x2t   t is stationary, then x1t and x2t will be
cointegrated.

By following Engle and Granger (1987), we can define cointegration in the following way:

For two time series x1t and x2t which are both I(d), if there exists a vector of β = ( β1, β2) such
that  t  x1t  1   2 x2t is I(d-b), d  b > 0, then, by following Engle and Granger (1987), x1t
and x2t are defined as cointegrated of order (d, b).

Testing for Cointegration: Engle-Granger 2 Steps Method

Engle and Granger (1987) proposed for testing the null hypothesis of no cointegration between
a set of I (1) variables in a single equation method. In the first step, they estimate the
coefficients of a static relationship between a set of I (1) variables by ordinary least squares
and, in the second step, ADF unit root test is carried out to the residual. Rejection of the null
hypothesis provides an evidence in favour of cointegration.

For two I(1) variables, the cointegration equation between two variables is

x1t   0  1 x2t  ut

For cointegration, ut is I(0).

The Engle-Granger two-step procedure involves estimation of β0 and β1 by ordinary least

squares using the static regression and finding out of estimated residual in the first step; and
testing for unit root of the estimated residual in the second step. In this methodology we have
to follow the following steps:

1. Test whether x1t and x2t are I(1) by applying ADF.

2. Estimate the parameters of this equation when both x1t and x2t are I(1)
3. Test whether the least squares residual ût appears to be I(0) or not.

If ût has a unit root, then x1t and x2t are not cointegrated.

Therefore, to carry out the test we have to estimate by applying OLS, and unit root tests are
applied to the residual û t .

For this, consider the autoregression of the residuals:

uˆt  uˆt 1  vt
The intercept term has not been included because the residuals from a regression equation have
zero mean. If the null hypothesis is rejected at a given significance level, we can infer that the
x1t and x2t are cointegrated of order (1, 1), CI (1, 1).