BOX-JENKINS

MODELS
S T Q S 3 11 3 S TAT I S T I C A L M O D E L L I N G

DR. RAZIK RIDZUAN

INTRODUCTION
• The Box-Jenkins methodology refers to a set of procedures for identifying and
estimating time series models within the class of autoregressive integrated moving
average (ARIMA) models.
• ARIMA models are regression models that use lagged values of the dependent variable
or random disturbance term as explanatory variables.
• ARIMA models rely heavily on the autocorrelation pattern in the data
• This method applies to both non-seasonal and seasonal data.
• In this topic, we deal with non-seasonal data.

2
• Three basic ARIMA models for a stationary time series :
– Autoregressive model of order ,

The depends on its previous values.

– Moving Average model of order ,

The depends on its previous random error terms.

– Autoregresive-moving average model of order and ,

Autoregressive component Moving Average component

The depends on its previous values and previous random error terms.

3
• In an ARIMA model, the random disturbance term is typically known as a “white
noise”.
• It is identically and independently distributed with a mean of 0 and a common variance
across all observations.
• We write

4
A FOUR-STEP ITERATIVE PROCEDURE

Forecasting
Diagnostics
Checking
Parameter
Estimation

Tentative Identification
• Stationarity Checking and
Differencing
• Model Identification

5
STEP ONE: TENTATIVE IDENFICATION
A) STATIONARITY CHECKING

• “Stationarity” is a fundamental property underlying almost all time series statistical

models.
• A time series is said to be stationary if it satisfies the following conditions:

6
• The white noise series satisfies the stationarity condition because

7
EXAMPLES –STATIONARY TS

8
EXAMPLES –STATIONARY TS

9
NON-STATIONARITY
• How to identify?
– Time plot
– The ACF of stationary data drops to zero relatively quickly
– The ACF of non-stationary data decreases slowly.
– Unit root test shows positive for structural breaks. Structural breaks refers to
sudden and unexpected change in the mean of the data (drop down, go up).

10
EXAMPLES – NON-STATIONARY TS

11
EXAMPLES – NON-STATIONARY TS

12
DIFFERENCING

• Differencing continues until stationarity is achieved.

• The number of times that the original series must be differenced in order to achieve
stationarity is called the order of integration, denoted by .
• In practice, it is almost never necessary to go beyond second difference, because real
data generally involve only first or second level non-stationarity.

13
14
15
DIFFERENCING
• Backward shift operator, .

• operating on , has the effect of shifting the data back one period.
• Two applications of B on shifts the data back two periods:

• So, applications of on , shifts the data back periods

• The backward shift operator is convenient for describing the process of differencing.

• Therefore,
16
UNIT-ROOT TEST

• To determine more objectively whether differencing is required, we can use unit root
test.
• One of the test is Kwiatkowski-Phillips-Schmidt-Shin (KPSS) test.
Time series are stationary.
Time series are non-stationary.
• Small p-value suggests differencing is required.

17
EXAMPLE – GLOBAL TEMPERATURE DEVIATIONS
Time series are stationary.
Time series are non-stationary.

18
STATIONARITY SUMMARY

• Stationarity of data is a fundamental requirement for all time series analysis.

• MA processes are always stationary.
• AR and ARMA processes are generally not stationary unless appropriate restrictions
are imposed on the model parameters.

19
STEP ONE: TENTATIVE IDENFICATION
B) MODEL IDENTIFICATION

• When the time series are stationary, one may proceed to tentative identification of
models through visual inspection of both the sample autocorrelation (SAC) and partial
sample autocorrelation (PSAC) functions.

20
SAMPLE AUTOCORRELATION FUNCTION (SAC)
• For the stationary series , the SAC at lag is

where is the sample mean.

• measures the linear relationship between time series observations separated by a lag of
time units.
• The standard error of is

• The statistics is

21
THE BEHAVIOUR OF SAC

1. The SAC can cut off. A spike at lag exists in the SAC if is statistically large. If

then is considered to be statistically large. The SAC cuts off after lag if there are no
spikes at lags greater than in the SAC.

22
2. The SAC is said to die down if this function does not cut off but rather decreases in a
‘steady’ fashion. The SAC can die down in
i. A damped exponential fashion
ii. A damped sine-wave fashion
iii. Or a combination of both i. and ii.
• The SAC can die down fairly quickly or extremely slowly.
• The time series values should be considered stationary if the SAC of the time series
values either cuts off fairly quickly or dies down fairly quickly.
• However, if the SAC of the time series values dies down extremely slowly, and at
lag 1 is close to 1, then the time series values should be considered non-stationary.

23
SAMPLE PARTIAL AUTOCORRELATION
FUNCTION (SPAC)

• For the series , the SPAC at lag is

where

for .

24
• when ,

• when

and

25
• may intuitively be thought of as the sample autocorrelation of time series observations
separated by a lag k time units with the effects of the intervening observations
eliminated.
• This function plays an important role in data analysis aimed at identifying the extent of
the lag in an autoregressive model.
• The standard error of is

• The statistic is

26
• The behavior of SPAC similar to its of the SAC. The only difference is that is
considered to be statistically large if

for any .

27
SUMMARY ON THE BEHAVIOURS OF SAC AND
SPAC

28
29
30
31
COMMON DEPENDENCE STRUCTURE
Model SAC (ACF) SPAC (PACF)
Dies down in damped
MA(1) Cuts off after lag 1
exponential decay
Dies down
MA(2) Cuts off after lag 2 (damped exponential
decay & sine wave)
Dies down in damped
AR(1) Cuts off after lag 1
exponential decay
Dies down
AR(2) (damped exponential Cuts off after lag 2
decay & sine wave)
Dies down in damped Dies down in damped
ARMA(1,1)
exponential decay exponential decay
32
EXAMPLE - LUTEINIZING HORMONE IN BLOOD
SAMPLES

33
• Cuts at lag 1 • Cuts at lag 1

34
 AR(1)

• Should we combine
both and ? ARMA(1,1)
• or ARIMA(1,0,
1)

MA(1)

35