Forecasting Crude Oil Prices Using Wavelet Neural Networks

Proc. 5th Student Conference of Management Science and Technology (ΦΣΔΕΤ). Athnes, Greece (2008). Forecasting Crude Oil Prices Using Wavelet Neural Networks A. Alexandridis 1, E. Livanis2 Department of Accounting and Finance, University of Macedonia of Economics and Social Studies, 156 Egnatia St., P.O. 54006, Thessaloniki, Greece. Abstract—According to International Energy Outlook 2007 the total world demand of energy is projected to increase through 2030 about 95% for the non-OECD region and 24% for OECD nations. Crude oil is one of the most critical energy commodities while with coal and natural gas are projected to provide roughly the 86% share of the total US primary energy supply in 2030. In this paper, we use wavelet neural networks to forecast monthly West Texas Intermediate (WTI) crude oil spot prices. As explanatory variables we consider price lags, the producer price index for petroleum and the world production of crude oil. The data are provided by the Energy Information Administration (EIA). The proposed model is used to forecast in-sample and out-of-sample. We forecast one, three and six month future prices of crude oil and we compare our estimates with the EIA’s STEO econometric forecasting model. Keywords— Crude Oil Pricing, Wavelet Neural Networks, Forecasting. 1. Introduction. According to International Energy Outlook 20073 the total world demand of energy is projected to increase through 2030 about 95% for the non-OECD region and 24% for OECD nations. The difference between OECD and non-OECD nations refers to the ongoing rates of economic development and population growth in non-OECD region. Especially, much of the growth in energy demand occurs in non-OECD Asia, mainly in China and India. Also, according to the Short-Term Energy Outlook (STEO) world oil consumption is expected to grow by 1.3 million bbl/d in both 2008 and 2009, in response to higher projected oil prices and increased risks of a global economic slowdown. Non-OECD countries are expected to account for 1.1 million bbl/d of world consumption growth in 2008, with gains concentrated in China, the Middle East oil-producing countries, India, and other Asian countries. Moreover, West Texas Intermediate (WTI) crude oil prices, which averaged $72.32 per barrel in 2007, are projected to average $94.11 and $85.92 per barrel, respectively, in 2008 and 2009. The projected higher costs for crude oil in 2008 are likely to be passed on to all petroleum products. However, STEO recognizes the possibility that prices would be higher if the economic slowdown is short-lived and consumption remains robust, or if oil production capacity expansion levels turn out to be lower than expected. 1 Corresponding Author, Phone: +30 2310 891631, e-mail: [email protected] Phone: +30 2310 891615, e-mail: [email protected] 3 Published by the Energy Information Administration, Office of Integrated Analysis and Forecasting, U.S. Department of Energy. 2 1 While crude oil is a central source of energy, governments and industrial sector companies’ activities are affected by the future oil price. Also rising oil prices affects consumer goods. Because of this the scientific community and market participants try to develop predictive models for crude oil prices. This has proved to be very difficult because of the complexity of the market. However, a better forecast of the expected crude oil price helps market participants to improve their plans and decisions. According to Kaboudan (2001) oil prices follow cyclical patterns over time. They tend to escalate for an extended period. Reverse direction then perhaps escalate again. Periodicity is not constant and variations within an escalating or a decreasing period are typical. Also, global demand for petroleum products is highly seasonal and is greatest during the winter months, when countries increase their use of distillate heating oil and residual fuels. Supply of crude oil, including both production and net imports, also shows a similar seasonal variation. In this paper, we try to investigate if a wavelet neural network estimator can provide some incremental value. The rest of the paper is organized as follows: In the next section we give a review of the relevant literature on forecasting crude oil prices. In section 3, we present our data. In section 4, we present our results. We give our forecasting framework and we explain how a wavelet neural network can be used for predicting future returns. Finally in section 5 we conclude. 2. Literature Review. The literature on crude oil market is enormous. This section provides background mainly in the area of development forecasting models for crude oil price and introduces the dynamics of the crude oil prices time series. Oil price forecast approaches can separated in two groups. The first group contains single-factor time series models where future predictions of oil price are produced based on the oil price time-series. The second group consists of multi-factor models where future predictions of oil price are produced based on correlated variables to the oil price such as consumption, supply, inventories or financial indexes. Early works use different models of the GARCH family. Morana (2001) used a semi parametric approach to forecast Brent ($/barrel). The approach was based on the GARCH properties of the oil price volatility, The results of the out-of-sample forecasting suggest that the forecasting approach can be used to obtain a performance measure for the forward price, in addition to compute interval forecasts via bootstrapping for the oil price. Moosa and AlLoughani (1994) uses a GARCH-M(1,1) model to find that the future prices are neither unbiased nor efficient forecasters of spot prices. Sadorsky (1999) using vector regression conclude that changes in oil prices impact economic activity but, changes in economic activity have little impact on oil prices. Sadorsky (2006) uses several univariate and multivariate statistical models such as GARCH, TGARCH, AR, BIGARCH, to estimate forecasts of daily volatility in petroleum futures price returns. Postali and Piccheti (2006) use a simple Geometric Brownian Motion and argue that their model can perform well as a proxy for the movement of oil prices. Peters (1994) argues about the use of such models. Most financial markets have sharper peaks and fat tails; hence models based on normality assumption must be avoided. Moreover the findings of Peters (1994) suggest long-memory. Dees et al (2007) proposed a model of the world oil market to forecast oil supply, demand, and real prices and to analyse risks with each. The model simulates oil demand with behavioural equations that relate demand to domestic economic activity and the real price of oil. Ye et al (2002) presented a short-term monthly forecasting model of West Texas Intermediate crude oil spot price using OECD petroleum inventory levels. Based on an understanding of petroleum market fundamentals and observed market behaviour during the post-Gulf War period, the model was developed with the objectives of being both simple and practical, with required data readily available. Adragni et al. (2001) point out that oil prices appear highly non-linear whiles the findings of Panas and Ninni (2000) suggest strong evidence of chaos in a number of oil products. Hence, 2 nonlinear models such as Neural Networks or Chaos Theory can be used. Gori et al (2007) examined the evolution of price and consumption of oil in the last decades to construct a relationship between them. Then they consider three possible scenarios of oil price: parabolic, linear and chaotic behaviour, to predict the evolution of price and consumption of oil up to December 2003. Rehrl and Friedrich (2006), using the LOPEX (Long-term Oil Price and EXtraction) model generated long-term scenarios of future world oil supply and forecast up to the year 2010. Although, the explanatory power is limited to the underlying idealising assumptions. Tang and Hammoudeh (2002) suggest that nonlinear approaches based on the Target Zone Theory can improve oil price forecasts. Kaboudan (2001) performs short-term monthly forecasts of crude oil prices and suggests that genetic programming outperforms random walk while neural networks forecast proved inferior. However, we argue about the input variables of used for the construction of the network. Kaboudan (2001) examines the explanatory power of the following variables: monthly world crude production, OECD consumption, world crude oil stocks, monthly change in known US stocks and lagged FOB crude oil price of US imports. All variables except the lagged prices were not helpful. As we show in the section our findings differ. Moreover using only lagged price variables as inputs for his models results to forecasts that “follow” the spot price of crude oil. Shambora and Rossiter (2005) used an artificial neural network model with moving average crossover inputs to predict price in the crude oil futures market. Kaboudan (2001) used genetic programming and ANN to forecast refineries acquisition cost of crude oil. Wang et al. (2004) used an integrating web-based text mining and rule-based expert system techniques, and econometrical techniques with intelligent forecasting techniques. According to their results the proposed methodology namely TEI@I seems to outperform the individual ANN and ARIMA models in terms of RMSE. Amin-Naseri & Gharacheh (2007) propose a hybrid artificial intelligence model for monthly crude oil price forecasting by means of feed-forward neural networks, genetic algorithm and k-means clustering. Amin-Naseri & Gharacheh (2007) model outperforms forecasts provided by the econometric model of the STEO but as well as forecasts provided by previous works. Finally Yousefi et al. (2005) introduce a wavelet-based prediction procedure and market data on crude oil is used to provide forecasts over different forecasting horizon. 4. Crude oil price forecasting using wavelet neural networks. In this section the wavelet neural network is explained and a forecast method is proposed Wavelet networks proposed by Zhang & Benveniste (1992) as an alternative to feedforward neural networks. Wavelet networks are one hidden layer networks that use a wavelet as an activation function instead of the classic sigmoid function. Works as Daubechies (1992) and Mallat (1999) give concise treatment of wavelet theory. The activation function can be a wavenet (orthogonal wavelets) or a wave frame (continuous wavelets). Wavelet networks are performing excellent in predicting nonlinear behaviors (Gao & Tsoukalas, 2001). Wavelets show local characteristics hence the hidden units of the wavelet network affect the prediction of the network only in a local range. (Postalcioglou & Becerikli, 2007). In contrast to sigmoid neural networks, wavelet networks allow constructive procedures that efficiently initialize the parameters of the network. Using wavelet decomposition a wavelet library can be constructed. Each wavelon can be constructed using the best wavelet of the wavelet library. These procedures allow the wavelet network to converge to a global minimum of the cost function. Also starting the network training very close to the solution leads to smaller training times. Finally, wavelet networks provide information of the participation of each wavelon to the approximation and the dynamics of the generating process. Wavelet networks have been used in a variety of applications so far. They first have been used in static and dynamic input-output modeling (Zhang & Benveniste, 1992; Postalcioglu & Becerikli, 2007) and proved that wavelet networks need less training iterations. Szu et al. 3 (1992) used for classification of phonemes and speaker recognition. Gao & Tsoukalas (2001) consider wavelet networks one of the most promising tools to solve electricity load prediction problems. Subasi et al. (2005) used wavelet networks for classification of electroencephalography (EEG) signals while Khayamian et al. (2005) used wavelet networks as a multivariate calibration method for simultaneous determination of test samples of copper, iron and aluminum. Figure 1 shows wavelet network with m inputs, xm, λ hidden units, a bias term and a single output, y (x) . Figure 2. A single layer wavelet neural network All data for this study were obtained from the US Department of Energy. The variable to forecast is West Texas Intermediate (WTI) crude oil price. The series used start January 1986 and taken at monthly closing price. The available data was until October 2007. The US Department of Energy offers a variety of data sets such as US and world crude oil consumption, supply and production. Additionally, petroleum indexes such as the AMEX and variables that affect the crude oil price such as natural gas and electricity price can be found. In bibliography variables such as supply, demand, stocks and financial indexes were proposed as explanatory factor for the crude oil prices. In this study we chose to examine the following variables for the West Texas Intermediate crude oil prices (WTP): the Consumer Price Index (CIC), the producer price index (WP), the industry production index (ZOT), S&P 500, the AMEX index, the USA petroleum stocks (SUSA), the USA and OECD petroleum consumption (CUSA and COECD), the USA crude oil production (PUSA) and the world crude oil production (PWORLD). First we calculate the correlation between each variable and WTP. From Table 1 we conclude that all variables are correlated with oil price as expected. We find that the oil prices are un-elastic to consumption. Although the oil price is increasing the last 5 years the consumption is moving around a constant mean. Also, we observer a negative correlation between USA production and price, -0.6719, while the correlation between world production and price is 0.7653. This is a result of the decreasing production policy adapted from US. From table 1 we conclude that the most significant variables are the WP index, the AMEX index, the world crude oil production and the consumer price index. As mentioned before there is long-memory in oil price time-series, hence lagged oil prices must be used. Testing different models we conclude to four input variables for our network: the WP index and the world crude oil production as well as crude oil prices with one and two lags. Although the AMEX index has a high correlation with oil prices is not useful in 4 forecasting since the AMEX index variations depend on the oil price changes and not the opposite. Finally, the inclusion of CIC index on our network led to inferior results. Variable CIC ZOT SUSA CUSA PWORLD Correlation 0.7540 0.6988 0.4900 0.6974 0.7653 Variable WP AMEX COECD PUSA S&P 500 Correlation 0.9874 0.9313 0.5594 -0.6719 0.6228 Table 1. Correlation between different variables and the crude oil prices The ν-fold cross-validation suggests that our network should be trained with one hidden unit. Although oil time-series have long memory, observing figures 1 and 2 one can conclude that the dynamics of the time-series have changed. Hence only recent available data should be used. In order to produce accurate forecasts we use the following procedure. Οne month ahead forecasts produced using a rolling window of 120 data points as training set for the network. In other words in order to forecast the monthly crude oil price for the next month information only from the last 10 years (120 months) is used. Following this procedure one month out of sample forecast from 1/1997 up to 10/2007 produced. Figure 2 shows the process of applying a sliding window of size 121 (120 lags and 1 target) on a time-series. The window is sliding until the training set includes the last date (10/2007). The procedure is repeated for forecasts of 3 and 6 months ahead i.e. using information of the last 10 years the network is trained to produce forecasts of the crude oil price for the next three and six months. Figure 2. Sliding window technique Next the proposed model is tested in two different data-sets. The first data set consists of forecasts from 1/2000 to 12/2002 while the second one consists of forecasts from 10/2005 to 12/2006. In order to evaluate our model we compare it against competing models such as the Wang et al.(2004) model which used artificial neural networks and web-based text mining (WANG) and Amin-Naseri & Gharcheh (2007) model which use a feed-forward neural networks, genetic algorithm and k-means clustering (AMIN) for the first data set and against the STEO model of Energy Information Administration in U.S. Department of Energy and Amin-Naseri & Gharcheh (2007) for the second data set. Table 2 provides performance criteria needed for the comparison. Six different criteria were used for the comparison. As for mean absolute error (MAE), mean absolute percentage error (MAPE), sum of square errors (SSE), mean square error (MSE), root mean square error (RMSE) and maximum absolute error (Max AE) the proposed model is shown to be superior to other models. It should be noted that Wang et al. (2004) did not report any information regarding to some criteria. Moreover the three month ahead and six month ahead forecasts provide better results than the one month ahead forecasts proposed by the AMIN and WANG models. Table 3 provides the performance criteria for the second data set. It is clear that our model does not perform well in this period. Probably this is happening because the training 5 set differs significantly from the forecast set. Finally we should mention that our network is significant smaller than the one used by AMIN. In contrast to Amin-Naseri & Gharcheh (2007) who used a three hidden layer network with 31 hidden units in each layer, we use a single hidden layer with only one hidden unit. Figure 3 shows the real and the fitted crude oil prices from 1/1986 to 12/1996 while figure 4 shows the real and the one month ahead forecasts from 1/1997 up to 10/2007. It is clear that the wavelet network performs well and it is able to produce accurate results. The MAE for the whole period (1/1997-10/2007) is 1.02 while the MSE is 2.05 and the Max AE is 7.36. Figure 3. Real oil prices and the fit produced by the wavelet network. Figure 4. Real oil prices and one month forecasts. 6 5. Conclusions. Crude oil price affects economy and governments therefore knowledge of its future movements can lead to better decisions in various managerial levels. However oil price forecasting is not a trivial procedure since oil price time series proved to have high volatility, non-linear and chaotic dynamics. Recent papers shows that researchers tend to recognize the limitations of GARCH models and start to use non-linear tools such as neural networks or chaos theory. In this study we propose a wavelet neural network in order to forecast crude oil prices. First we examine different explanatory variables and we select the world crude oil production, the Producer Price Index and two lags of oil prices as network input variables. We used our model to produce one, three and six month out-of-sample forecasts from 1/1997 up to 10/2007. Results show that wavelet networks can learn the oil price dynamics. To evaluate the performance of the proposed model, three models (STEO, AMIN and WANG) were chosen for comparison. The models were compared in two different data sets using six performance criteria. In the first data set the proposed model outperforms other models in all criteria. Moreover the three and six month forecasts are also superior to forecasts obtained from the other models. However the proposed model does not perform well in the second data set, probably because the training set differs significantly from the forecast set. In general our model performs well both in-sample and out of sample. However the results presented here can be improved further. First an adaptive method that selects the correct range of past data for the network training must be proposed. The dynamics of the oil price timeseries must be analyzed. Also, here we examine only a few parameters that affect the oil price. A rigorous analysis of explanatory variables must be done in order to build the best training set for the network. Finally, since risk managers are more interested in predicting price intervals for the future oil price movements than simply point estimates, one can provide a framework for estimating confidence and prediction intervals. WANG AMIN 1 Month 3 Months 6 Months x 1.61 1.38 1.46 1.46 x 5.90% 4.90% 5.34% 5.48% x 3.52 5.02 4.77 4.71 x 133.4 129.01 131.71 125.75 5.61 3.71 3.58 3.65 3.39 2.37 1.92 1.89 1.91 1.84 Table 2. Performance criteria for the 2models and the proposed model for 1, 3 and six month forecast for the first data set MAE MAPE Max AE SSE MSE RMSE MAE MAPE Max AE SSE MSE RMSE STEO 2.36 3.70% 7.2 138.7 9.25 3.04 AMIN 2.29 3.60% 5.97 136.5 9.1 3.02 1 Month 2.67 4.12% 6.15 164.46 10.9 3.31 3 Months 2.47 3.89% 6.48 147.99 9.8 3.14 6 Months 3.45 5.37% 8.24 282.33 18.82 4.33 Table 2. Performance criteria for the 2models and the proposed model for 1, 3 and six month forecast for the second data set. 7 References [1] B. Adragni, A. Chatrath, K.K. Dhanda and K. Raffie (2001). Chaos in oil prices? Evidence from futures markets. Energy Economics vol. 23, pp. 405-425. [2] M.R. Amin-Naseri and E.A. Gharacheh (2007). A hybrid artificial intelligence approach to monthly forecasting oil price time series. Proceedings of EANN. [3] I. Daubechies (1992). Ten lectures on wavelets. SIAM. [4] S. Dees, P. Karadeloglou, R.K. Kaufmann and M. Sanchez (2007). Modeling the world oil market: Assessment of a quarterly econometric model. Energy Policy vol. 35, pp. 178-191. [5] R. Gao and L.H. Tsoukalas (2001). Neural-wavelet methodology for load forecasting. Journal of Intelligent & Robotic Systems vol. 31, pp. 149-157. [6] F. Gori, D. Ludovisi and P.F. Cerritelli (2007). Forecast of oil price and consumption in the short term under three scenarios: Parabolic, linear and chaotic behavior. Energy vol. 32, pp. 1291-1296. [7] M.A. Kaboudan (2001). Compumetric forecasting of crude oil prices. Proceedings of the 2001 Congress on Evolutionary Computation vol. 1, pp. 283-287. [8] T. Khayamian, A.A. Ensafi, R. Tabaraki and M. Esteki (2005). Principal componentwavelet networks as a new multivariate calibration model. Analytical Letters vol. 38(9), pp. 1477-1489. [9] S.G. Mallat (1999). A wavelet tour of signal processing. Academic Press, San Diego. [10] I.A. Moosa and N.E. Al-Loughani (1994). Unbiasedness and time varying risk premia in the crude oil future markets. Energy Economics vol. 16(2), pp. 99-105. [11] C. Morana (2001). A semiparametric approach to short-term oil price forecasting. Energy Economics vol. 23, pp. 325-338. [12] E. Panas and V. Ninni (2000). Are oil markets chaotic? A non-linear dynamic analysis. Energy Economics 22, pp. 549-568. [13] E. Peters (1994). Fractal market hypothesis: Applying chaos theory to investment and economics.Wiley. [14]S. Postalcioglu and Y. Becerikli (2007). Wavelet networks for nonlinear system modeling. Neural Computing & Applications vol. 16, pp. 443-441. [15] F.A.S. Postali and P. Picchetti (2006). Geometric Brownian motion and structure breaks in oil prices: A quantitative analysis. Energy Economics vol. 28, pp. 506-522. [16] T. Rehrl and R. Friedrich (2006). Modeling long-term oil price and extraction with a Hubbert approach: The LOPEX model. Energy Policy vol. 34, pp. 2413-2428. [17] P. Sadorsky (1999). Oil price shocks and stock market activity. Energy Economics vol. 21(5), pp. 449-469. [18] P. Sadorsky (2006). Modeling and forecasting petroleum futures volatility. Energy Economics vol. 28, pp. 467-488. [19] W.E. Shambora and R. Rossiter (2007). Are there exploitable inefficiencies in the futures market for oil? Energy Economics vol. 29, pp. 18-27. [20] A. Subasi, A. Alkan, E. Koklukaya and M.K. Kiymik (2005). Wavelet neural network classification of EEG signals by using AR model with MLE pre-processing. Neural Networks vol. 18. pp. 985-997. [21] L. Tang and S. Hammoudeh (2002). An empirical exploration of the world oil price under the target zone model. Energy Economics vol. 24, pp. 557-596 [22] S. Wang, L. Yu and K.K. Lai (2004). A novel hybrid AI system framework for crude oil price forecasting. Lecture Note in Computer Science vol. 3327, pp. 203-242. [23] M. Ye, J. Zyren and J Shore (2002). Forecasting crude oil spot price using OECD petroleum inventory levels. International Advances in Economic Research vol. 8, pp. 324334. [24] S. Yousefi, I. Weinreich and D. Reinarz (2005). Wavelet-based prediction of oil prices. Chaos, Solitons and Fractals 25, pp. 265-275. [25] Q. Zhang and A. Benveniste (1992). Wavelet networks. IEEE Trans. Neural Networks vol. 3(6), pp 889-898. 8