MATEC Web of Conferences 68, 19001 (2016) DOI: 10.

1051/matecconf/20166819001
ICIEA 2016

Forex Market Prediction Using NARX Neural Network with Bagging

Nima Shahbazi1 , Masoud Memarzadeh2 and Jarek Gryz1

1
Department of Electrical Engineering and Computer Science, York University, Canada
2
Department of Mathematics, Tehran University, Iran

Abstract. We propose a new methodfor predicting movements in Forex market based on NARX neural network
withtime shifting bagging techniqueand financial indicators, such as relative strength index and stochastic indicators.
Neural networks have prominent learning ability but they often exhibit bad and unpredictable performance for noisy
data. When compared with the static neural networks, our method significantly reducesthe error rate of the
responseandimproves the performance of the prediction. We tested three different types ofarchitecture for predicting
the response and determined the best network approach. We applied our method to prediction the hourly foreign
exchange rates and found remarkable predictability in comprehensive experiments with 2 different foreign exchange
rates (GBPUSD and EURUSD).

1 Introduction Phase 1: We find the most relevant financial

indicators to be fed into the network by sensitivity
The Forex Market is the largest and most liquid market analysis regarding the response, the input/output
around the word with an estimated $2 trillion traded indicators, and the architecture.Then we train the network
every day.Modelling the system is difficult as currency with news data with static neural network (Section 0).
market is characterized by significant non-linearity and Phase 2: Using the same inputs and output,we use the
low signal to noise ratio. Also, foreign exchange rates are NARX neural Network architecture (Section 0). By
affected not only by economic news but alsopolitical and usingdynamic network both the error rates as well as the
psychological factors. Not surprisingly, predicting the directional success improve(Section 0).
movement of foreign exchange rates or finding a trend in Phase 3: In the last phase, time shifting bagging
the market is very difficult. method (Section 0)is added to the system and shown to
There are several neural network approaches [1] and improve the test error and directional success rate on the
[2] which ledto successful predictions. However, these test data.
approaches use moving averages for the input of their After presenting the experimental results, a discussion
networks and do not use any bagging which,as we will on future research concludes the paper.
showlater, improves the response prediction. It has been
shown in [3] that the relationship between neural
networks and conventional statistical approaches for time 2 Architecture and learning algorithm
series forecasting are complementary. It has also been
indicated in [4] that traditional statistical techniques for 2.1 NARX neural network as a forecasting tool
forecasting have reached theirlimit inapplications with for Forex
nonlinearities in the data set (such as stock
indices).Recently, Dynamic Neural Networks such as Neural Networks (NN) based prediction is a suitable tool
NARX have been used in many businessapplications for forecasting in non-linear time seriesapplications. With
especially when the problem domain involves predictions neural network approach to time series it is not necessary
[5] but again, there is no bagging or indicator selection. to know any information regarding the cause that
This paper proposes a new method for predictingforex generates the signal.
market movement by using NARX neural network with The architectural approach proposed here to deal with
bagging technique. Instead of using the next close price Forex market is based on Nonlinear Autoregressive
or moving averages, we uses RSI (Relative Strength models with eXogenous input (NARX model)[7]. All
Index) indicator for the output response. We feed data other dynamic networks have either been focused
aboutForex news to the networkasthere is strong networks, with the dynamics only at the input layer, or
correlation between the news piece and the stock price feed-forward networks. NARX is a recurrent dynamic
behavior[6]. network, with feedback connections enclosing several
Our contributions can be described as three phases: layers of the network. It is based on the linear ARX

© The Authors, published by EDP Sciences. This is an open access article distributed under the terms of the Creative Commons Attribution
License 4.0 (http://creativecommons.org/licenses/by/4.0/).
MATEC Web of Conferences 68, 19001 (2016) DOI: 10.1051/matecconf/20166819001
ICIEA 2016

model, which is commonly used in time-series input data, given the expected result as outputs. In this
modeling.This is a powerful class of models which has approach we use RMSE (Root Mean Square Error)
been demonstrated to be well suited for modeling aserror function:
nonlinear systems and especially time series. It has been
shown that in NARX networks learning is more effective 1 1

 =  ∑ 2 
=1( ) =  ∑=1( −  )
2 (2)
than in other neural networks and that these networks
converge much faster and generalize better than the other where  is the target and  is the predicted value. In this
networks [8]. paper, the training function that updates the weights and
The defining equation for the NARX model is: bias values is the Levenberg-Marquardt function. In
() = (( − 1), … ( − 2),  −  , ( − 1), (1) general, in function approximation problems, for
networks that contain up to a few hundred weights, the
( − 2), … , ( −  )) Levenberg-Marquardt algorithm will have the fastest
convergence. This advantage is especially noticeable if
where the next value of the dependent output signal () very accurate training is required. Also,it was shown that
is regressed on previous values of the output signal and the neural network training can be made more efficient if
previous values of an independent ()(exogenous) input certain preprocessing steps on the network inputs and
signal. One can implement the NARX model by using a targets are performed. The normalization of the input and
feed-forward neural network to approximate the function target values simplifies the problem of the outliers in the
F. network. The normalized inputs and targets that are
There is another important configuration for NARX returned will all fall in the interval[-1, 1].Another factor
model (useful in training) that we use in our approach. in the neural network isthe transfer function. The best
Let the output of the NARX network be an estimate of transfer function is chosen by the best performance seen
the output of some nonlinear dynamic system that we are on the test result. In this problem Tangent-Sigmoid
trying to model. The output is fed back to the input of the makes the learning phase faster and increases the neural
feed-forward neural network as part of the standard network accuracy.
NARX architecture. Since the true output is available
during the training of the network one could create a
series-parallel architecture [9] in which the true output is 3 Model construction and evaluation
used instead of feeding back the estimated output as
shown in Figure 1. This has two advantages. First, the
3.1 Dataset and input/output predictors
input to the feed-forward network is more accurate.
Second, the resulting network has purely feed-forward The prediction is done hourly. The dataset used here is
architecture and static back propagation can be used for pulled from Alpari-Forex.com[11]. For the news release
training as our dynamic network is now a feed-forward data, two sources on relevant currencies are used: (1)
neural network without any feedback. Fxstreet.com[12], economic calendar from 2001; and (2)
Bloomberg.com [13], economic calendar from 2001.
We define six hour window, three hours before the
related news release and three hours after the news
release, and gather the data (Open, Low, High and Close)
every minute in the six hour window for training network.
We divided the data into (70%, 15% and 15%) with 70%
use for training, 15% for validation,and the rest for
testing.
Figure 1. Serial/parallel architecture for NARX model. Once the raw data have been chosen by six hour
window, a set of indicators based on those values might
2.2 Learning algorithm be developed. In this work we used five indicators for
input/output of the neural network shown in Table 1. For
For learning, a dynamic back-propagation algorithm is all formulas below, the subscript t±1 indicates time t plus
needed to evaluate the gradients. This is more or minus one hour.
computationally than static back-propagation. In addition, Differential RSI (Relative Strength Index) as
the error can be larger for dynamic than for static Input/Output
networks. Training is more likely to fall into local 1. A technical momentum indicator that compares the
minima [10]. But asshownin Figure 1we can use instead a magnitude of recent gains to recent losses in an attempt to
series-parallel architecture.Thus, it is possible to use true determine overbought and oversold conditions of an asset.
output instead of the estimated output to train the network RSI ranges from 0 to 100. We use differential RSI as:
with the feedback connections.
The process of training a neural network involves  () =  − −1 (3)
iteratively presenting it with the input data so that it is
calibrated and can be used later as a forecasting tool. The The output of the network will be:
objective of the training is to minimize a defined error  ( + 1) = +1 −  (4)
function, which implies that the neural network fit the

2
MATEC Web of Conferences 68, 19001 (2016) DOI: 10.1051/matecconf/20166819001
ICIEA 2016

2. Differential EMA (Exponential Moving Average) 5. Differential PR (William %R) as Input

as Input Williams’s %R, or just %R, is a technical analysis
oscillator showing the current closing price in relation to
Table 1. Selected features and their formulae: C, closing price; the high and low of the past N values (for a given N). Its
H, high price; L, low price;  , lowest low price in the last n
purpose is to tell whether a stock or commodity market is
samples;  , highest high in the last n samples; Up, upward
price change; Dw, downward price change. trading near the high or the low, or somewhere in
between, of its recent trading range. The oscillator is on a
negative scale, from -100 (lowest) up to 0 (highest). We
Name of Formula use Differential PR as:
features
! () = !  − ! −1 (8)
−1
STOCH %  − −5 100 In this paper, the hourly period sample for each of the
 × indicator is given in Table 2.
D −5 − −5 
=0
Table 2. Period (in hours) samples used in this work.
PR( Willia  − 
× 100
m %R)  −  Name of features Period

100 − STOCH %D 5:sampling

RSI 3:smoothing
100
 ∑ −1 − ∑−1
=0 
(1 + ( =0 )/(1 + (
PR( William %R) 14


 = 
−1 +  × (!"#
− 
−1 ) RSI 14
EMA

EMA 14
MACD 
$%& − 
&'
MACD Fast:12 Slow=26

Exponential moving average (EMA), also known as

an exponentially weighted moving average (EWMA), is a 3.2 Construction phases and trading strategy
type of infinite impulse response filter that applies results
weighting factors that decrease exponentially. We use
Differential EMA as: In order to track the improvement of the method we
divided the problem into three phases.All networks were

() = 
 − 
−1 (5) trained between January 2001 and May 2014. For each
experiment, a range of different training parameters and
3. Differential STOCH (Stochastic Oscillator %D) as
numbers of hidden nodes were tried, and the results are
Input
listed here. For each experiment, error in predicting
In technical analysis of securities trading, the
training and test set outputs are reported, in terms of
stochastic oscillator is a momentum indicator that uses
RMSE, along with the directional success.
support and resistance levels. The term stochastic refers
While the accuracy of a model may be measured
to the location of the current price in relation to its price
using RMSE (the error in directional success) what is
range over a period of time. This method attempts to
ultimately needed is a measure of the effectiveness of the
predict price turning points by comparing the closing
model in relation to its use in driving decisions to buy/sell
price of a security to its price range. STOCH ranges from
0 to 100. We use Differential STOCH as: shares. As the output of the model is  ( + 1) =
+1 −  , the trading strategy will be:
*-() = *- − *-−1 (6)
$  ( + 1) >  3 4%'
. (9)
4. MACD (Moving Average Convergence/Divergence) $  ( + 1) <  '' 4%'
as Input
MACD makes use of moving averages of different Here  is non-negative tuning parameter. The value of
time frames to indicate momentum changes and swings in the tuning parameter is traded off between the number of
the mood of the crowd, to give buying and selling signals trades and directional success that can be evaluated from
that catches the big moves. The MACD indicator the back testing result on historical data (we ended up
measures the difference between two moving averages using 1.22 for  ). In all phases, a range of different
(EMA): parameter settings and configurations of hidden nodes
were evaluated and the most successful ones are

() =
 −
−1 (7) documented. The results given in tables are for both

3
MATEC Web of Conferences 68, 19001 (2016) DOI: 10.1051/matecconf/20166819001
ICIEA 2016

GBPUSD/EURUSD rates. The training parameters lay the result of the output for 12 ways. Prediction errors are
within the following ranges: shown in Table 5.
1. Learning rate, (0.001 to 0.3)
2. Momentum, (0.002 to 0.4) Table 5. Phase 3 train/test errors GBPUSD.

3.2.1 Phase 1 RMSE

In this phase we use static feed forward network for Prediction Error
training. The prediction errors on train/test sets are shown GBP USD
in Table 3.
Table 3. Phase 1 train/test errors. Training Set 3.53 3.23

Test Set 4.17 3.87

RMSE
Set The results in Table 5 shows that the error rates for
GBPUSD EURUSD train/test improve remarkably when the time shifting
average method is used.There is a significant change in
test error rate when bagging method is used in
Training Set 3.92 3.66
combination with NARX neural network. InTable 6the
directional success for all the phases is listed.
Test Set 4.72 4.42
Table 6. Directional success on all the phases.

3.2.2 Phase 2 Directional success rate on

In the second phase a series-parallel architecture as test date
Phase No.
shown in Figure 1is used, so that now NARX is used as
the network architecture.The improvements in RMSE GBPUSD EURUSD
from using this network are shown inTable 4.
Table 4. Phase 2 train/test errors. 1 54% 57%

RMSE 2 60% 63%

Set
GBPUSD EURUSD 3 68% 72%

Training Set 3.76 3.36

Test Set 4.32 4.02

3.2.3 Phase 3
In the last phase of the work we used the time shifting
bagging method. Time shifting bagging method works as
follows: we gather the data in a minute and produce the
hourly rates based on M1 (minutes) data. We can sample
the data in 60 different ways for hourly sampling. Thus,
we can start sampling in M1 data at time t and sample
repeatedly at times t-60, t-120, t-180, …. Therefore, the
sampling has 60 steps starting at time t. Or we can start
the sampling in M1 data at time t-1, and sample Figure 2. Directional success rate, test set and training set as a
repeatedly at times t-61, t-121, t-181, … . Clearly, we function of phase No. for each currency.
can do this kind of sampling in 60 different ways to
prepare the data for the input of the network. In this work
3.3 Discussion
we sample in 12 different ways starting at t, t-5, t-10…
and t-55 and compute each of the indicators based on The purpose of Phase 1 was to determine the
these datasets. Finally, we train the network and average performance of a neural network trained solely on the

4
MATEC Web of Conferences 68, 19001 (2016) DOI: 10.1051/matecconf/20166819001
ICIEA 2016

hourly time series data. This showed thedirectional 4. A. Refenes and G. F. A. Zapranis, "Stock
success of 54%(GBPUSD) and 57%(EURUSD). The performance modeling using neural networks: A
results indicate that this is useful; yet the success rate is comparative study with regression models," Neural
not good enough to attempt to use this method for profit Network, 5, 961-970 (1994)
in the real market. 5. L. Haizhon and K. Robert, "A Dynamic Neural
In Phase 2, the architecture was changed to NARX Network Method for Time Series Prediction Using
neural network and the directional success rate became the KIII Model," in Proceeding of the 2003
60%(GBPUSD) and 63%(EURUSD). By changing the International Joint Conference on Neural Networks,
method to time shifting bagging, as we did in Phase 3, we (2003)
improved the rate to 68%(GBPUSD) and 72%(EURUSD) 6. G. Fung, "The Predicting Power of Textual
which is a significant rate of success. Information on Financial Markets," IEEE Intelligent
Informatics Bulletin, 5, 1-10 (2005)
7. S. Haykin, Neural Networks and Learning Machines
4 Conclusions (3rd Edition), Pearson Education, (2008)
There is an enormous range of indicators that are not 8. T. Lin, C. L. Giles, B. G. Horne and S. Kung, "A
accounted for in this paper. The analysis presented here Delay Damage Model Selection Algorithm for
shows only the benefit of considering NARX neural NARX Neural Networks," IEEE Transactions on
network and time shifting bagging method with some Signal Processing, “Special Issue on Neural
external indicators when predicting Forex market Networks”, 45, 2719-2730 (1997)
movement. The use of other indicators or even different 9. K. S. Narendra and K. Parthasarathy, ""Learning
bagging methods should be explored further. Also, this Automata Approach to Hierarchical Multiobjective
study was on two pairs of currencies only and should be Analysis,"," IEEE Transactions on Systems, Man
repeated on other currency combinations. and Cybernetics, 20, 263-272 (1991)
10. K. S. Narendra and K. Parthasarathy, ""Learning
Automata Approach to Hierarchical Multiobjective
References Analysis,"," IEEE Transactions on Systems, Man
and Cybernetics, 20, 263-272 (1991)
1. J. Yao and C. Tan, "A case study on using neural 11. "Alpari," 1 09 2013. [Online]. Available:
networks to perform Technical Forecasting of www.alpari-forex.com.
Forex," Neurocomputing, 34, 79-98 (2000) 12. "FXstreet," 1 09 2013. [Online]. Available:
2. F. Ferna´ndez-Rodrı´guez, C. Gonza´lez-Martel and www.fxstreet.com.
S. Sosviall-Rivero, "On the profitability of technical 13. "Bloomberg," 01 09 2013. [Online]. Available:
trading rules based on artificial neural networks: www.bloomberg.com.
Evidence from the Madrid Stock Market,"
Economics Letters, 69, 89-94 (2000)
3. H. White, "Learning in artificial neural networks: A
statistical perspective," Neural Computing, 1, 425-
464 (1989)