2010 3rd International Conference on Information Management, Innovation Management and Industrial Engineering

The Relative Performance of VAR and VECM Model

Jianfeng Zhang, Wenxiu Hu Xin Zhang

Department of Finance School of Business
Xi’an University of Technology Queen’s University
Xi`an, China Kingston, Canada
[email protected], [email protected] [email protected]

Abstract—Two models are examined in this study: autoregressive model with unit roots (PENM, 1997).
Vector Autoregressive Model (VAR) and Vector Error However more recently, VECM model has given an
Correction Model (VECM). Based on three indices: opportunity to develop more complex analyses in forex
S&P 500, Nikkei 225 (NIKKEI), and Morgan Stanley markets, because they can accommodate both long-term
EAFE (MSCIEAFE or MSCI EAFE) Index, we and dynamic processes. Although the two models have
implement VAR and VECM models, including the pre- some individual advantages, there is not a conclusion about
estimation diagnostics, model estimation and which works better between VAR and VECM model. For
interpretation and post-estimation tests, etc. By testing, parsimonious reasons, we will try to report only different
we find that while the VECM model consistently results in the two models.
outperforms the VAR model within sample, it is hard to The paper is structured in the following manner:
tell which works better out-of-sample. Our examination Section 2 describes the data; In section 3, we study the
return series and VAR; In section 4, we study the
suggests four conclusions: First, both models do the
cointegration and VECM; Section 5 discusses the horse
best job with MSCI EAFE. Second, VECM model
racing of models; and Section 6 presents the conclusions.
does a better job within sample. Third, both models
create less error out-of-sample than within sample.
Fourth, VECM model does a better job than VAR II. DATA
in S&P 500 and NIKKEI. The data used in this paper are for three indices, i.e.,
Keywords-Forecasting Performance; VAR Model; S&P 500, Nikkei 225 (NIKKEI), and Morgan Stanley
VECM Model; Cointegration Estimation EAFE (MSCIEAFE or MSCI EAFE) Index. All these
data are collected from Data Stream, a commercial
database provided by Thomson Financial.
I. INTRODUCTION To be able to work on both VAR and VECM models,
Various studies have been documented in the literature we need both stationary and non-stationary time series. I
seeking to forecast accuracy the model effect, individually use (natural) logged Price Index series of the three indices
(LITTERMAN, 1979,1986; BERNARDUS, 2005; YAN- as non-stationary series and the corresponding return series
LEUNG, 2008; INOUE, 2004; JOHANSEN, 1991; as stationary series (Stationary here is ex ante expectation,
GRANGER, POON, 2003, 2005). However, there are few the exact stationarity will be tested in the paper). Return
models, besides VAR and VECM that have satisfied series are defined as the log differences of the Price Index
forecasting result. Vector Autoregressive model is a series.
natural generalization of univariate autoregressive (AR) We use logged Price Indexes for two reasons. First, as
model. VAR methodology was proposed as an alternative all the Price Indexes are non-negative and highly skewed,
to simultaneous equation models and made significant using logged Price Indexes can at least partially remove the
advances in the 1980s (ENGLE; GRANGER 1987; skewness. More importantly, VECM models use first order
CAMPBELL; SHILLER, 1987). At the beginning of its difference of the series as the dependent variable. The first
development, Sims (1980) and Litterman (1979, 1986) order difference of logged Price Index is exactly the logged
approached this methodology as being more appropriate return of these series. Thus, by using logged Price Index,
for forecasting than simultaneous equation models. One of we essentially make the dependent variable in the VAR
the model advantages resides in the fact that in general, its and VECM models to be exactly the same, allowing direct
forecasts are considered superior to simultaneous equation comparison of the forecasting errors of these two models.
models (SIMS, 1980; MCNEES, 1986). A notable difference of the data in the current paper is
The VECM models as developed by Engle and that we will limit the sample period to calendar year 2007
Granger (1987) have as their aim the insertion of short- to 2008, with the first year data to estimate model and next
term adjustments due to the presence of cointegration. As year data to carry out (some of) post-estimation tests.
VECM takes into account the cointegrating relationships There are two reasons for shortening the sample period.
among the endogenous variables, In fact, Vector Error First, using too long a sample period may lead to the issue
Correction Model (VECM) is equivalent to a vector of structural change. We expect the relationship among

978-0-7695-4279-9/10 $26.00 © 2010 IEEE 132

DOI 10.1109/ICIII.2010.195
these series to change over time due to market integration Table 1: Time Trend Test
of the three capital markets, policy changes and so on. Coefficient P-Value
Second, using too large a sample is too demanding in S&P 500 Return 0.0000 0.561
computing power. NIKKEI Return 0.0000 0.606
III. RETURN SERIES AND VAR MSCIEAFE Return 0.0000 0.353
We then test the stationary of individual time series.
A. Pre-estimation Diagnostics All three tests significantly reject the null hypothesis of unit
Table 1 presents results from simple time trend root and we conclude that these series are stationary.
regressions. Coefficient reported in the table is the We also carry out a multivariate white noise test and
coefficient on the time trend variable and P-Value is the the p-value of 0.0000 strongly rejects the null hypothesis
corresponding p-value. Table 1 suggests that none of the of white noise.
return series shows significant time trend. We then run stata command varsoc to select the lag
for fitting VAR model. Table 2 reports the output from
By seasonality test, the null hypothesis is that the
varsoc. It seems that all information criterions and LR
series do not contain any seasonality. None of the three
test suggest VAR (1) to be the best choice.
series seems to show any seasonality.
Table 2: Lag selection for VAR
lag LL LR df p FPE AIC HQIC SBIC
0 2443.88 1.1e-12 -18.9952 -18.9785 -18.9538
1 2496.27 104.77* 9 8.1e-13*
0.000 -19.3328* -19.2662* -19.1671*
2 2504.47 16.406 9 0.0598.1e-13 -19.3266 -19.21 -19.0366
3 2508.21 7.4885 9 0.5868.5e-13 -19.2857 -19.1191 -18.8714
4 2513.72 11.01 9 0.2758.7e-13 -19.2585 -19.0419 -18.7199
out the IRF and forecast practices in the next subsection.
B. Model Estimation
It is natural to reason from this evidence that the
We fit a VAR (1) model and the estimations are U.S. equity market is still the most influential market
reported in the following equation, where*** means among the three. However, one should be cautious in
significant at 0.01, ** at 0.05 and * at 0.10, respectively. generalizing these results. For one thing, S&P 500 may
only represent the top and most successful firms in the
⎡S & P t ⎤ ⎡ 0.1023 ⎤ ⎡ S & P t -1 ⎤ U.S..The number of firms financial economists work with
⎢ ⎥ ⎢ ⎥ ⎢ ⎥
⎢ NIKKEI t ⎥ = ⎢ 0.0371 ⎥ + ⎢ NIKKEI t -1 ⎥ ⋅ for research in a given year is generally over 8,000. Thus,
⎢ ⎥ ⎢ ⎥ ⎢ ⎥ how representative S&P 500 is questionable. Nevertheless,
⎢⎣ EAFE t ⎥⎦ ⎢⎣ 0.0322 ⎥⎦ ⎢⎣ EAFE t -1 ⎥⎦ given current results, we are quite confident the S&P
index is more influential than the other two, at least
⎡ - 0.1874 ∗ ∗ ∗ 0.0100 0.0592 ⎤ ⎡ eS & P, t ⎤ statistically. We will provide further discussion in the
⎢ ⎥ ⎢ ⎥ next subsection too.
⎢ 0.4741 ∗ ∗ ∗ - 0.0442 0.1365 ⎥+ ⎢ eNIKKEI, t ⎥
⎢ ⎥ ⎢ ⎥ C. Post-Estimation
⎢⎣ 0.3843 ∗ ∗ ∗ 0.0524 - 0.1514 ∗ ⎥⎦ ⎢⎣ eEAFE, t ⎥⎦
We first test whether the VAR (1) has removed
A first observation from the estimations is that none of autocorrelation from the series. We first get three series of
the intercepts is statistically significantly different from 0. residuals from the model and carry out a multivariate white
Second, except for NIKKEI, all series show statistically noise test.
negative autocorrelation at lag 1. Actually, even for the For the residuals series, we cannot reject the null
NIKKEI series, the lag 1 autocorrelation is negative (albeit hypothesis of (multivariate) white noise. We also run the
statistically not significant). LM test for autocorrelation in residuals (varlmar) for 5
A more interesting finding is that it seems that the lags. None of the statistics reject the null hypothesis of no-
U.S. equity market has a strong impact on the other two autocorrelation.
markets. Others constant, a 10% increase in S&P 500 (log) However, a (unreported) multivariate normality test
return will lead to a 4% and 3% increase in NIKKEI 225 suggests that the hypothesis of number 3 dimensional
and MSCIEAFE index returns the next day, respectively. normality is rejected. It will not hurt the result except for
But the impact from NIKKEI and MSCI EAFE on S&P the efficiency of the estimates. This is not surprising either,
500 is rather limited, both in the sense of magnitude and given the large literature documenting skewness and
significance. A 10% increase in these two indices will lead kurtosis on stock returns. This suggests that a multivariate
to less than 1% in S&P 500, if any. Nor are these two GARCH model may be more appropriate, though this is
markets having any significant impact on each other. beyond the scope of the current paper. Testing VAR
Further discussion on this will be presented when we carry

133
Granger Causality, Results confirm my inference in the p −1
previous subsection. In general, S&P500 return Granger
causes the other two and not vice versa. Nor are the other
Δ y t = αβ y t−1 + ∑
i=1
ΓiΔ y t − i + ν + ε t

two series Granger causing each other. However, a 1% Notice that since first order differences of PI are used,
change in S&p 500 today will (statistically or granger) the constant vector v in the equation system represents a
causing almost 0.5% change of NIKKEI 255 and linear trend vector. Also, notice that the summation is till
MSCIEAFE in the same direction in the next day. P-1 lag in the above equation, the lag chosen here is
However, the most important observation is that none essentially the same as in the VAR model. For
of the aforementioned impacts survive after 3 trading days, parsimonious reason, we will only report the coefficients
which confirms the stability of the system. of interest.
While forecasting should be part of the post-estimation In particular, we first report the vector β. Actually,
diagnostics, we will postpone it to the last section where
β̂= (1, 0.4013, −1.5489).
we compare the various models.
That is to say, S&P + 0.4013 * NIKKEI − 1.5489
IV. COINTEGRATION AND VECM *MSCIFAFE = I (0). I get the S&P + 0.4013 * NIKKEI
While the previous section investigates the three − 1.5489 * MSCIFAFE series using Stata command
return series, we will investigate the Price Index series in predict. ADF test strongly reject (P-Value 0.0000) the null
this section. hypothesis of unit root and we infer that the series are
stationary. Thus, the VECM estimation seems work quite
A. Pre-estimation diagnostics well.
Time Trend Test suggests that all of the Price Index The Γ matrix gives very close interpretation to the one
series show significant time trend. in VAR model and is thus omitted.
At this point, it is attractive to detrend the Price Index
series. However, we use the undetrended series for two C. Post-estimation
reasons. On the one hand, the objective of this section is to We first test whether the VECM (2) has removed
practise with cointegration tests and VECM models, both autocorrelation from the series. Multivariate Ljung-Box
of which require all our series to contain unit root. We test test gives a test statistics of 38.3452 and P-Value of
the stationarity of detrend Price Index seires. Except for 0.7480. Thus, we cannot reject the null hypothesis of the
the (detrended) NIKKEI series, neither of the other two residuals being multivariate white noise. LM tests
series contain unit root. On the other hand, it makes thing (unreported) support the previous tests in that no-
more complicated when interpenetrating the results if we autocorrelations are found in the first 5 lags.
use detrended Price Index. Also, detrending the time series However, a (unreported) multivariate normality test
makes it much harder to compare the forecasting errors in suggests that the hypothesis of number 3 dimensional
the two models. Thus, in the remaining of the paper, we normality is rejected. Again, it will not hurt the result
will use (logged) Price Indexes. By doing Edwards except for the efficiency of the estimates.
seasonality test statistics and corresponding P-Value of The Granger causality tests is quite clear that only
seasonality tests for the three series, none of the three S&P 500 Granger causes the other two and no feedback is
series seems to show any seasonality. We then test the found. Not surprisingly, the results are consistent with
stationary of individual time series. By testing the ADF those reported in the VAR model.
and corresponding (Mackinnon) P-values, All the
V. HORSE RACING OF MODELS
(undetrended) Price Index series contain unit roots.
Unreported PP tests and KPSS tests support these While it is interesting comparing VAR and VECM
conclusions too. models with different lags, it is more interesting
We also carry out a multivariate white noise test and comparing the VAR and VECM model. As discussed in
the p-value of 0.0000 strongly rejects the null hypothesis Section 1, the design of the current study makes it
of white noise. possible for a fair compare of the two models using
Johansen test suggest that there is one cointegration forecasting errors. Table 3 reports the within sample and
out-of-sample RMSE and MAE for the VAR (1) model
relation among the three series.
and for the VECM (2) model. The reported values are
We then run stata command varsoc to select the lag for
100 times of the actual value so as to illustrate their
fitting VECM model. It seems that all information difference. The in-sample RMSE and MAE are calculated
criterions and LR test suggest VECM (2) to be the first based on data in 2007, the year of which these two
choice. models are estimated. Out-of-sample RMSE and MAE
B. Model estimation are calculated based on 2008 data. The AIC and BIC of
the two models are previously reported.
Considering the time trend test, we allow time trend in
the VECM too. Thus, the VECM can be written as:

134
 Table 3: Forecast Errors
Within Sample Out of Sample
MAE RMSE MAE RMSE
Panel A: VAR (1)
S&P 500 0.7978 1.0379 0.5293 0.6977
NIKKEI 0.9760 1.2788 0.7791 1.0086
MSCI EAFE 0.7152 0.9494 0.4226 0.5610
Panel B: VECM (2)
S&P 500 0.7948 1.0369 0.5318 0.6985
NIKKEI 0.9757 1.2788 0.7787 1.0080
MSCI EAFE 0.7034 0.9321 0.4446 0.5750
All these results suggest the followings: First, [2] Campbell, J, and J. Shiller. “Cointegration tests and present value
comparing RMSEs and MAEs for three equations within models.” Journal of Political Economy, 95(1987), 1062-1088.
each model, both VAR and VECM do the best job with [3] Hansen, Peter Reinhard. “Structural changes in the cointegrated
vector autoregressive model.” Journal of Econometrics, 114(2003),
MSCI EAFE, next S&P and NIKKEI. Second, VECM 261-295.
model does a better job within sample for all the three [4] Engle, R. and Granger, C. W. J. “Cointegration and Error Correction
equations. Third, quite astonishingly, both models create Representation, Estimation, and Testing.” Econometrica, 55(1987),
less error out-of-sample than within sample. While not 251-276.
reported, we use year 2006 data for out-of-sample error [5] Hendry, D. F. “Econometric Modelling with Cointegrated Variables:
calculation and get qualitatively very similar results. An Overview.” Oxford Bulletin of Economics and Statistics, 48
Fourth, the prediction errors out-of-sample vary across (1986), 201–212.
the three equations. For the 2008 data, VECM model [6] Phillips, P. C. B. and Perron, P. “Testing for a Unit Root in Time
does a better job than VAR in S&P 500 and NIKKEI, Series Regression.” Biometrica, 75 (1988), 335-346.
but worse in MSCI EAFE. [7] Wickens, M. R. “Interpreting Cointegrating Vectors and Common
Stochastic Trends.” Journal of Econometrics, 74 (1996), 255-271.
VI. CONCLUSION [8] Christoffersen, P.F. and Diebold, F.X. “Cointegration and long-
horizon forecasting.” Journal of Business and Economic Statistics,
In this paper, we estimate VAR and VECM model 16(1998), 450-458.
using essentially the same dataset. The design of the paper [9] Chou, R.Y. “Forecasting financial volatilities with extreme values:
allows me to compare directly the forecast errors of these the conditional autoregressive range (CARR) model.” Journal of
Money, Credit & Banking, 37(2005), 561-582.
two models.
[10] Clements, M.P., Hendry, D.F. “Forecasting with Difference and
Our examination of the relative performance of two Trend Stationary Models.” The Econometric Journal, 4(2001), 1-19.
alternative forecasting models suggests four conclusions: [11] Diebold, F.X. and Mariano, R.S. “Comparing predictive accuracy.”
First, Both VAR and VECM do the best job with MSCI Journal of Business & Economic Statistics, 13(1995), 253-263.
EAFE, next S&P and NIKKEI. Second, VECM model [12] Fernandes, M., de Sá Mota, B. and Rocha, G. “A multivariate
does a better job within sample for all the three equations. conditional autoregressive range model.” Economics Letters,
Third, both models create less error out-of-sample than 86(2005), 435-440.
within sample. Fourth, VECM model does a better job than [13] Inoue, A. and Kilian, L. “In-sample or out-of-sample tests of
predictability: which one should we use?” Econometric Reviews,
VAR in S&P 500 and NIKKEI, but worse in MSCI EAFE. 23(2004), 371-402.
We find that while the VECM model consistently [14] Johansen, S. “Estimation and hypothesis testing of cointegration
outperforms the VAR model within sample, it is hard to vectors in Gaussian vector autoregressive models.” Econometrica,
tell which works better out-of-sample. 59(1991), 1551-1580.
[15] SIMS, C. A. “Macroeconomics and Reality.” Econometrica,
ACKNOWLEDGMENT 48(1980), 1-48.
Financial support from the National Nature Science [16] Parkinson, M. “The extreme value method for estimating the variance
of the rate of return.” The Journal of Business, 53(1980), 61-65.
Foundation of China and Doctoral Fund of Ministry of
[17] Poon, S.H. and Granger, C.W.J. “Forecasting volatility in financial
Education of China (70973096, 20096118110010) are markets: A review. ”Journal of Economic Literature, 41(2003), 478-
gratefully acknowledged. 539.
[18] Poon, S.H. and Granger, C. “Pratical issues in forecasting volatility.”
REFERENCES Financial Analysts Journal, 61(2005), 45-56.
[1] Penm, J. H., Penm, J. H. W. and Terrell, R. D. “The Selection of [19] CAMPBELL J., SHILLER R. “Cointegration and Tests of Present
Zero-Non-Zero Patterned Cointegrating Vectors in Error-Correction Value Models.” Journal of Political Economy, 95(1987),1062-1088.
Modelling.” Econometric Review, 16 (1997), 281-304. 5-15.

135