Hydrological Sciences Journal

ISSN: 0262-6667 (Print) 2150-3435 (Online) Journal homepage: http://www.tandfonline.com/loi/thsj20

Comparative performance analysis of climate re-

analysis approaches in Angola

Sandra Pombo, Rodrigo Proença de Oliveira & André Mendes

To cite this article: Sandra Pombo, Rodrigo Proença de Oliveira & André Mendes (2017)
Comparative performance analysis of climate re-analysis approaches in Angola, Hydrological
Sciences Journal, 62:5, 698-714, DOI: 10.1080/02626667.2016.1257856

To link to this article: http://dx.doi.org/10.1080/02626667.2016.1257856

Accepted author version posted online: 09

Nov 2016.
Published online: 06 Dec 2016.

Submit your article to this journal

Article views: 31

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=thsj20

Download by: [95.93.232.159] Date: 04 April 2017, At: 04:47

HYDROLOGICAL SCIENCES JOURNAL – JOURNAL DES SCIENCES HYDROLOGIQUES, 2017
VOL. 62, NO. 5, 698–714
http://dx.doi.org/10.1080/02626667.2016.1257856

Comparative performance analysis of climate re-analysis approaches in Angola

Sandra Pombo, Rodrigo Proença de Oliveira and André Mendes
Department of Civil Engineering, Architecture and Georesources, Civil Engineering Research and Innovation for Sustainability (CERIS),
Instituto Superior Técnico, Universidade de Lisboa, Lisbon, Portugal

ABSTRACT ARTICLE HISTORY

This paper examines the performance of three climate re-analysis datasets in Angola, namely the Received 20 April 2016
Accepted 23 September 2016
ERA-40, NCEP-r1 and JRA-55, by comparing annual and seasonal estimates of precipitation,
surface air temperature and evaporation with ground observation measurements. The observa- EDITOR
tional dataset describes a region poorly covered by international climate databases and it is A. Castellarin
believed that most of its data have not been used in the data assimilation procedures of the
ASSOCIATE EDITOR
climate models. This paper therefore provides a fresh look at the performance of these climate re-
S. Kanae
analysis datasets in a vast area where distance and civil war have hindered ground monitoring
efforts. The re-analysis exercises offer better temperature estimates than precipitation estimates. KEYWORDS
When offered, the evaporation estimates from all three products are very poor. The three Re-analysis exercises;
products are able to describe the main features of the spatial distribution of average annual performance; Angola
precipitation and temperature, but struggle to reproduce the temporal changes of these vari-
ables. The results from a set of performance criteria show that the correlation between the
observed ground measurements and re-analysis estimates is poor overall and the NSE values
indicate that the average measured value at each location is usually a better estimate than the re-
analysis estimate.

1 Introduction
estimates with ground gauge records (Nicholson et al.
Water management requires quantitative and reliable 2003, Hughes 2006, Beighley et al. 2011). Pombo et al.
information on the spatial and temporal changes of (2015) evaluated the performance of four remote sen-
hydro-meteorological variables, which is usually sing products in Angola to estimate annual and
obtained by analysing large datasets of ground (or in monthly precipitation and concluded that TRMM
situ) monitoring records. The effort required to acquire 3B43 (Kummerow et al. 1998) provided the most con-
these datasets is very costly in large countries with sparse sistent and least biased estimations.
road networks, as in the case of Angola. Consequently, A possible alternative to remote monitoring is the use
the number of ground monitoring stations is being of datasets obtained from climate re-analysis exercises. A
reduced worldwide (Vörösmarty et al. 2000). In re-analysis exercise runs a numerical weather forecast
Angola, the mobility and cost problem was further model using climate data observations gathered from
aggravated by the civil war which, until recently, barred many different sources, which are continuously fed
travelling within the country and prevented proper into the model as control or frontier conditions. The
maintenance of most monitoring stations. model works as a sophisticated time and space interpo-
The use of remote sensors installed in satellites can lation mechanism and the end result is a decades-long
be very useful in overcoming this problem, and there estimate of the past record of different climate variables
are now a number of standard products that routinely at a grid-spaced set of points. These exercises are mainly
offer estimates of precipitation and surface air tem- performed to better understand all aspects of the Earth’s
perature at regularly spaced locations (grid) and at climate system but their results may be useful to com-
regular time intervals (few hours), which may be used plete or complement the climate variables monitoring
to improve the description of the spatial-temporal records (Trenberth et al. 2008).
changes of these variables (Sapiano and Arkin 2009, This article evaluates the performance of three re-
Maidment et al. 2012). Several studies have evaluated analysis exercises, namely NCEP-r1, ERA-40 and JRA-
the performance of these standard remote sensing pro- 55, with the goal of confirming whether readily avail-
ducts in Africa by comparing monthly and annual able standard re-analysis products are accurate enough

CONTACT Sandra Pombo [email protected]

© 2016 IAHS
 HYDROLOGICAL SCIENCES JOURNAL – JOURNAL DES SCIENCES HYDROLOGIQUES 699

and of interest for water management studies in areas have their origins in this central plateau, with water
where ground monitoring data are scarce. The paper courses flowing north towards the Zaire (or Congo),
also analyses the estimation errors of the re-analysis east towards the Zambezi, southeast towards the
exercises to verify whether it is possible to correct the Okavango and southwest towards the Cunene.
re-analysis estimates and obtain a better fit with the Angola’s climate is largely determined by (i) the
ground measurements in Angola. It is not our goal to presence of a high-pressure centre in the South
propose specific and concrete changes to the algorithm Atlantic and of a low-pressure centre in the continent;
used in the re-analysis exercises. Instead, we aim to (ii) the cold Benguela Current that flows west–east; and
verify whether there is a correction procedure to be (iii) the land topography, with the dominating central
applied by practitioners on the final results of the re- plateau with an altitude of more than 1000 m (Fig. 1).
analysis exercises. The climate varies considerably from the coast to the
The paper is organized as follows. Section 2 briefly central plateau and even between the north and south
presents the geography of Angola and the country’s coasts. The north, from Cabinda to north of Luanda,
hydrological monitoring network. Section 3 describes has a damp, tropical climate. South of Luanda and
the applied methodology and Section 4 presents the along the coast towards the Namib Desert in the south-
results obtained. Finally, Section 5 discusses the results west, the climate varies from dry tropical climate to a
and draws conclusions. hot desert climate, where rainfall is scarce (Pombo
et al. 2015). Namibe, in the southwest of the country,
registers a mean annual precipitation of 50 mm year−1.
2 Angola geography and hydrological
The northern, eastern and central regions have a
monitoring network
humid tropical climate, with average annual precipita-
The Republic of Angola is located on the western tion above 1250 mm year−1. There are two seasons
Atlantic coast of southern Africa between Namibia controlled by the movement of the Inter-Tropical
and the Democratic Republic of the Congo. It is the Conversion Zone (Lars 2004): a dry, cool season from
fifth largest African country, with an area of June to late September, and a rainy, hot season from
1 246 700 km2 and a western ocean front of 1650 km October to April or May with, on average, over 95% of
(Fig. 1). The centre of the country is a sequence of annual rainfall.
plateaus lying at an altitude of 1000–2000 m, and The oldest monitoring station in Angola is João
covering around 60% of the territory. The highest Capelo, installed in 1857 in the capital city of Luanda.
peak is Monte Môco, in Huambo province, which From 1940 onwards a significant effort was made to
reaches 2620 m. Tributaries of the main African rivers create a structured hydro-metrological monitoring

Figure 1. Angola geography, provincial capitals and main rivers.

700 S. POMBO ET AL.

Table 1. Number of monitoring stations with records collected Quality control and homogenization has to be
for the current research. undertaken prior to any data analyses in order to
No. of No. of No. of eliminate any erroneous values and non-climatic biases
No. of hydrological precipitation temperature evaporation
years with monitoring monitoring monitoring in the time series. Double mass plots and tests on
complete records stations stations stations partial means with a confidence interval of 95% were
>40 11 2 1 used to check the homogeneity of the annual data
>30 and ≤40 26 2 3
>20 and ≤30 107 20 24 series (Pombo et al. 2015).
>10 and ≤20 201 57 62
>5 and ≤10 119 42 45

3 Methodology
network, and in 1974 Angola had a structured precipita-
tion monitoring network with 464 gauges stations with 3.1 Climate re-analysis products
more than 5 years of complete data. The number of Standard climate re-analysis products offer annual and
stations measuring surface air temperature and evapora- seasonal estimates of precipitation, air temperature and
tion was considerably lower. Surface air temperature was evaporation, and their results are now readily and
measured with thermometers and evaporation with Piche easily available for use by water managers, without
evaporimeters, both placed in shelters. requiring any deep understanding of the numerical
During the civil war that followed the country’s climate simulation methodology that supports the esti-
independence in 1975, many of the stations were mation of these variables.
destroyed, and presently Angola has a sparse monitor- The first generation of atmospheric re-analysis exer-
ing network with only 16 stations located in the main cises was developed in the mid to late 1990s at the US
cities remaining in operation, clearly an insufficient National Oceanic and Atmospheric Administration’s
number for this vast country. The end of the civil war (NOAA) National Centers for Environmental Prediction
provided the opportunity to increase the investment in (NCEP/NCAR; NCEP-r1), at the European Centre for
monitoring activities. Medium-Range Weather Forecasts (ECMWF; ERA 15)
Table 1 presents the number of stations with and at the Japanese Meteorological Agency (JMA; JRA-
monthly precipitation and evaporation and monthly 25). A second-generation global re-analysis includes the
surface air temperature records with more than ERA-40 (ECMWF) and JRA 55 (JMA) exercises.
5 years of complete records but with varying record This article evaluates the performance of three re-
lengths and degrees of data completeness. A period of analysis exercises, namely NCEP-r1, ERA-40 and JRA-
15 water years, from October 1959 to September 1974, 55, using a common period of 15 years, from October
was selected as the reference, assuming a compromise 1960 to September 1974 (Table 2). All exercises assim-
between the number of available stations and their ilate observations in time intervals of 6 hours, with the
record lengths. All stations with less than 10% of miss- first two using a three-dimensional variational data
ing data in the reference period were considered. assimilation (3D-Var) analysis and the latter using a
Figure 2 presents the locations of the 130 precipitation 4D-Var analysis. Data assimilation is the process by
stations, 61 temperature stations and 55 evaporation which observations are incorporated into a numerical
stations used in the study (Pombo et al. 2015). forecast model of a real system. The process advances

Figure 2. Locations of (a) precipitation, (b) temperature and (c) evaporation monitoring stations used in the study. The vertical lines
show the transects used in Figures 4, 8 and 12.
 HYDROLOGICAL SCIENCES JOURNAL – JOURNAL DES SCIENCES HYDROLOGIQUES 701

Table 2. Summary of the global atmospheric re-analysis exercises used.

Assimilation
Global model scheme and Publicly available
Exercise Time range resolution model vintage Available time step(s) dataset resolution
NCEP r1 1948–present T62 L28 3D-Var /1995 Sub-daily, Daily, Monthly 2.5° × 2.5°
ECMWF ERA 40 1957–2002 T159 L60 3D-Var /2004 Sub-daily, Monthly 2.5° × 2.5°
JRA 55 1958–2012 T319 L60 4D-Var /2009 Sub-daily, Monthly 1.25° × 1.25°

by analysis cycles and, in each cycle, observations The short period of concurrent ground measure-
within a given assimilation window are combined ments and re-analysis exercise estimates from the dif-
with the model results to produce the “best” estimate ferent data sources adds another difficulty when
of the state of the system, balancing the uncertainty in comparing the datasets. The civil war in Angola that
the observational data and in the model forecast. The started in 1974 compromised the country’s meteorolo-
3D-Var analysis assumes a steady-state condition gical observation effort during the period when the re-
within the assimilation window, whereas 4D-Var ana- analysis exercises became available. Consequently, the
lysis uses the forecast model to predict the change in number of simultaneous estimates from ground sta-
time of the state of the system (Gauthier et al. 2007). tions and re-analysis exercise estimates is small for
The NCEP-r1 was the first re-analysis project of its such a large country as Angola.
kind for NOAA and used a wide variety of weather To overcome these challenges, several approaches were
observations from ships, planes, radiosonde observa- followed to compare the estimates from re-analysis exer-
tion (RAOBs), station data and satellite observations cise estimates with ground station measurements.
(Kalnay et al. 1996), covering the period from January Although subjective in nature, a simple visual comparison
1948 to the present. The resolution of the global re- of mean annual value maps is one way to compare the
analysis model is T62 (209 km). The dataset used in spatial distribution patterns of both datasets. To help the
this research was downloaded from http://www.esrl. comparison of each set of maps, north–south transects
noaa.gov/psd/data/gridded/data.ncep.reanalysis.html. along three longitudes were computed. The analysis was
The ERA 40 (Uppala et al. 2006) is the second re- performed for the common period of 15 years from 1959/
analysis project from ECMWF and is a global atmospheric 60 to 1973/74; the maps obtained from ground measure-
re-analysis exercise covering 45 years, from September ments at each monitoring station and from the re-analysis
1957 to August 2002. Many meteorological observation estimates at each grid cell were computed using co-kriging
sources were used, including radiosondes, balloons, air- interpolation, with the terrain elevation as the independent
craft, buoys, satellites and scatterometers. The resolution of variable (Deng 1986, ESRI 2003). The use of co-kriging for
the global re-analysis model is T159 (about 125 km). The spatial interpolation of rainfall or temperature over com-
dataset used in this research was downloaded from http:// plex terrain has been suggested as way to improve perfor-
data-portal.ecmwf.int/data/d/era40_daily/. mance when compared to inverse distance weighting
JRA-55 (Kobayashi et al. 2014) is the second re- (IDW) and ordinary kriging (Buytaert et al. 2006,
analysis project from JMA, covering 55 years starting Cedeño and Cornejo 2008). Although these maps and
from 1958, when regular radiosonde observations transects provide an overall view of the performance of
began on a global basis. The spatial resolution is each re-analysis exercise and of their ability to offer regio-
TL319 (about 55 km). The dataset used in this research nal estimates of the climate variables, the poor density of
was downloaded from http://rda.ucar.edu/#!lfd?nb= the monitoring stations in some parts of the country
y&b=proj&v=JMA Japanese 55-year Re-analysis. hinders any accurate validation of the products.
A second type of analysis uses scatterplots to compare
estimates of annual and seasonal precipitation, mean
3.2 Performance indicators temperature and annual evaporation obtained from re-
The quality of annual and seasonal estimates of rainfall, air analysis exercises with concurrent ground measurements
temperature and evaporation from the three re-analysis observed at specific meteorological stations. The re-ana-
datasets was evaluated by comparing them with measure- lysis exercise-based estimates at each station site were
ments from ground stations. When comparing estimates computed by interpolating the estimates of the four adja-
from re-analysis exercises against ground station measure- cent grid points, assuming that each value refers to the
ments, the different spatial scales of the two datasets have middle point of the cell. The comparison was made for
to be carefully considered, as re-analysis exercise dataset 130 ground stations in the case of precipitation, 61
estimates are offered in a grid format and ground measure- ground stations in the case of temperature and 55 ground
ment values exist only at specific locations. stations in the case of evaporation (Fig. 2).
702 S. POMBO ET AL.

From each scatterplot the following performance RMSE measure two distinct properties of the dataset,
indicators were computed: (a) slope of the linear as a re-analysis product can provide unbiased estimates
regression line relating the gauge observations and with a value of RMSE significantly different from zero.
the re-analysis exercise estimates, without intercept; The overall RMSE of a product considering all sites
(b) bias; (c) root mean square error (RMSE); (d) in the dataset, can be computed by:
Nash-Sutcliffe efficiency (NSE); (e) Pearson correlation vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
u
coefficient (r); and (f) the coefficient of determination u1 X S X T  2
(R2) associated with the previous linear regression. RMSE ¼ t Rs;t  Gs;t (6)
ST s¼1 t¼1
The slope of the linear regression is an indicator of
the re-analysis exercise’s systematic deviation from the The NSE for the estimates at a given site s is given by
observed values, i.e. its tendency to overestimate or (Nash and Sutcliffe 1970):
underestimate them. Values greater than 1 indicate a
T 
P 2
tendency to overestimate ground measurements, Rs;t  Gs;t
whereas values smaller than 1 depict the opposite trend.
NSEs ¼ 1  t¼1
T 
(7)
The bias is also an indicator of the estimation error P 2

s  Gs;t
G
trend at a given site s and can be computed by: t¼1

1X T   The NSE has no units and indicates the prediction skills

biass ¼ Rs;t  Gs;t (1)
T t¼1 of the estimate, relative to a reference, in this case the
gauge mean. It varies from −∞ to 1, 1 being the perfect
where Rs,t and and Gs,t are the re-analysis exercise skill. Negative values of the NSE mean that the observed
estimate and the ground measurement, respectively, at mean is a better predictor than the estimated value, 0
site s and time t. The bias measures the systematic means that the observed mean is as good as the estimate,
deviation of the estimates from the observed values and positive values show good prediction skill (Schaefli
and is expressed in mm or °C. Positive values of bias and Gupta 2007, Gupta et al. 2009).
indicate an overestimation trend, and negative values To estimate the prediction skills of a product, inde-
an opposite trend. pendently of each site, one can compute the NSE by:
The mean normalized bias (Nbias) is given by:
P T 
S P 2
T 
P  Rs;t  Gs;t
1
T Rs;t  Gs;t NSE ¼ 1  s¼1 t¼1
(8)
t¼1 PS PT  2
Nbiass ¼ 100
s (2)
s  Gs;t
G
G
s¼1 t¼1

where G
s is the mean of Gs;t for site s.

The Pearson correlation coefficient, r, measures the
When analysing the dataset as a whole, for all sites,
linear correlation between ground measurements, Gs,t,
the non-normalized bias can be computed by Equation
and satellite estimates, Rs,t, and ranges between −100%
(3), where S is the number of sites in the dataset:
and +100%.
1 XS X T   The coefficient of determination, R2, describes the
bias ¼ Rs;t  Gs;t (3) proportion of the variance of ground measurements,
ST s¼1 t¼1
Gs,t, explained by a linear function of re-analysis exer-
^ s;t . It can assume
cise estimates, with a null intercept, G
The RMSE and the mean normalized RMSE (NRMSE)
at a given site s can be computed by: negative values, meaning that the mean of the ground
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi measurements is a better fit than the linear model
u T
u1 X  2 estimates, and is computed by:
RMSEs ¼ t Rs;t  Gs;t (4)
T i¼1 P T 
S P 2
^ s;t
Gs;t  G
sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
PT  2 R2 ¼ 1  s¼1S t¼1T (9)
PP 
1
T Rs;t  Gs;t
2
Gs;t  G
i¼1 s¼1 t¼1
NRMSEs ¼ 100
s (5)
G
Higher values of r and R2 indicate that is possible to
The RMSE is an indicator of the mean absolute error establish a linear relationship between the re-analysis
for any given estimate and is expressed in mm or °C, exercise estimate and the ground measurement, which
while NRMSE is expressed as percentage. Bias and can be used to correct the estimate.
 HYDROLOGICAL SCIENCES JOURNAL – JOURNAL DES SCIENCES HYDROLOGIQUES 703

Together, these indicators allow overall evaluation if from ground measurements (Fig. 3), a possible benefit
the re-analysis exercise is able to consistently provide of using 4D-Var data assimilation.
accurate estimates of ground measurements. To facilitate the comparison between these maps,
A third type of analysis uses the Nbias, NRSME and Figure 4 presents the transect of each climate re-ana-
NSE indicators to verify whether the accuracy of the re- lysis exercise estimate of the annual precipitation along
analysis exercise’s seasonal precipitation and tempera- three north–south lines, at longitudes 13°50ʹE, 17°35ʹE
ture estimates varies significantly with altitude, a key and 20°20ʹE, and compares them with the transects
factor in the spatial distribution of these variables. obtained from observed measurements. Figure 2
shows the locations of the transects, together with the
location of the monitoring stations, and is clear that the
4 Analysis of results reduced number of monitoring stations in the interior
of the country constrains the accuracy of the in situ
4.1 Precipitation transects. Nevertheless, Figure 4 shows that, at long-
Figure 3 shows the mean annual precipitation maps for itudes 17°35ʹE and 20°20ʹE, the profiles obtained from
the period 1959/60–1973/74 obtained from in situ data climate re-analysis follow the in situ data profile,
and from the climate re-analysis datasets. The maps although always with lower values. At longitude 13°
show a similar spatial distribution in all datasets, with a 50ʹE, the transect closer to the ocean, the climate re-
clear decreasing trend from the northeast to the south- analysis transects follow the in situ profile from 7°50′S
west. All climate re-analysis exercises provide an esti- to 11°50′S, but diverge at latitudes further south than
mate of the mean areal annual rainfall in Angola that is 11°50′S. This area corresponds to the Lubango region,
lower than the estimates obtained with the in situ data which lies at an altitude above 1500 m, overlooking the
(1098 mm). JRA-55 provides the highest estimate Namib Desert by the coast, and registers a mean
(970 mm), whereas NCEP-r1 provides the lowest esti- annual precipitation close to 1000 mm year−1. The
mate (876 mm). The spatial distribution obtained from climate re-analysis exercises fail to predict the specific
JRA-55 estimates is the most similar to that obtained local condition of this area.

In Situ ERA-40

NCEP-r1 JRA-55

Figure 3. Mean annual precipitation maps for 1959/60–1973/74 obtained by co-kriging interpolation from ground observation
measurements and re-analysis estimates (values in mm).
704 S. POMBO ET AL.

Figure 4. North–south transect of mean annual precipitation of the 1959/60–1973/74 record obtained at longitudes 13°50ʹE, 17°
35ʹE and 20°20ʹE.

Figure 5 shows the annual precipitation and seaso- line and the negative bias values), whereas NCEP-r1
nal precipitation values obtained from the re-analysis shows higher regression coefficients. The NSE values
datasets against in situ records from 1959/60 to 1973/ computed with Equation (8) are all negative, showing
74, with each point representing a monitoring site and that the observed mean annual precipitation in each
one year or season. All products show a positive monitoring station is usually a better estimate of
Pearson correlation coefficient with the ground annual precipitation than the re-analysis estimate. If
records, although with a large dispersion around the the NSE statistic is computed using the mean annual
regression line. The large RMSE and low NSE values precipitation over the whole country as the reference
show the poor precipitation prediction ability of all benchmark (a less demanding benchmark), the statistic
products. The trend to underestimate annual precipita- values improve significantly and a positive value is
tion and wet season precipitation is clear, as well as the obtained for the annual precipitation estimates from
poor prediction power of the products for the dry ERA-40 and for the dry season precipitation estimates
season. For annual precipitation and wet season pre- from NCEP-r1, showing that the re-analysis exercises
cipitation, JRA-55 offers estimates with a lower under- are able to reproduce the spatial distribution of annual
estimation error (shown by the slope of the regression and seasonal precipitation.
 HYDROLOGICAL SCIENCES JOURNAL – JOURNAL DES SCIENCES HYDROLOGIQUES 705

ERA-40 NCEP-r1 JRA-55

ANNUAL PRECIPITATION

3000 3000 3000

r = 57.5% r= 59.0% r= 52.1%
bias = -181 mm bias = -291 mm bias = -105 mm y = 0.8562x
2500 2500 2500
RMSE = 427 mm y = 0.7861x RMSE = 476 mm RMSE = 471 mm R2 = 0.2019
NSE = -3.12 y = 0.6982x
NSE = -2.31 R2 = 0.2250 2000 R2 = 0.2981 2000
NSE = -3.03
2000

1500 1500 1500

1000 1000 1000

500 500 500

0 0 0
0 500 1000 1500 2000 2500 3000 0 500 1000 1500 2000 2500 3000 0 500 1000 1500 2000 2500 3000
In Situ In Situ In Situ

DRY SEASON PRECIPITATION

350
r = 58.2% 350 350
r = 71.2% r = 39.2%
300 bias = 27 mm
300 bias = -16 mm 300 bias = -21 mm
RMSE = 57 mm
250 y = 1.0304x RMSE = 39 mm RMSE = 54 mm
NSE = -3.47 250 NSE = -1.10 250
R2 = 0.0635 NSE = -3.02
200 200
200 y = 0.4256x
150 R2 = 0.4991 y = 0.3201x
150 150
R2 = 0.1534
100 100 100

50 50 50

0 0 0
0 50 100 150 200 250 300 350 0 50 100 150 200 250 300 350 0 50 100 150 200 250 300 350
In Situ In Situ In Situ

WET SEASON PRECIPITATION

3000 3000 3000
r = 55.2% r = 55.6% r = 48.3%
bias = -83 mm 2500 bias = -275 mm 2500 bias = -82 mm
2500 y = 0.8709x
RSME = 468 mm y = 0.7561x RSME = 463 mm RMSE = 468 mm
R2 = 0.1503
NSE = -3.15 R2 = 0.1838 NSE = -3.06 y = 0.7023x NSE = -3.15
2000 2000 2000
R2 = 0.2478
1500 1500 1500

1000 1000 1000

500 500 500

0 0 0
0 500 1000 1500 2000 2500 3000 0 500 1000 1500 2000 2500 3000 0 500 1000 1500 2000 2500 3000
In Situ In Situ In Situ

Figure 5. Annual precipitation and seasonal precipitation values obtained from the re-analysis datasets against in situ records for
1959/60–1973/74. Each point represents a monitoring site and one year or season (values in mm).

In addition, although the global NSE values com- data were available for the climate simulation models
puted for the ensemble of all gauge locations (Fig. 5) used in the re-analysis exercises.
are negative, some NSE values computed by Equation Figure 6 shows the mean normalized bias (Nbias, %),
(7) for specific gauges are positive, showing that for the mean normalized root mean square error (NRMSE, %)
some locations the re-analysis exercises are able to and the Nash-Sutcliffe efficiency (NSE) between the annual
provide estimates that are better than the observed precipitation estimates from each climate re-analysis exer-
mean precipitation in that location. The product with cise dataset and the observed annual precipitation records
a higher number of locations with positive NSE values in each monitoring station. These statistics are plotted
is the ERA-40 (13 out of 130 gauges) and most of these against the altitude of each station and show that there is
13 gauges are located northwest of Malange. The poor no clear and consistent relationship between the statistic
performance of the re-analysis exercises in the south- values and altitude, with the possible exception of Nbias.
eastern part of the country may be related to the poor All products seem to overestimate annual precipitation by
monitoring network in this area and to the fact that no the coast and underestimate it above 500 m. Precipitation
706 S. POMBO ET AL.

ERA-40 NCEP-r1 JRA-55

150% 150% 150%

100% 100% 100%

Normalized bias

50% 50% 50%

(Nbias)

0% 0% 0%

-50% -50% -50%

-100% -100% -100%

0 500 1000 1500 2000 0 500 1000 1500 2000 0 500 1000 1500 2000
Altitude (m) Altitude (m) Altitude (m)

200% 200% 200%

Normalized RMSE

150% 150% 150%

(NRMSE)

100% 100% 100%

50% 50% 50%

0% 0% 0%
0 500 1000 1500 2000 0 500 1000 1500 2000 0 500 1000 1500 2000
Altitude (m) Altitude (m) Altitude (m)

1 1
-1 -1 -1
-3 -3 -3
-5 -5 -5
NSE

-7 -7 -7
-9 -9 -9
-11 -11 -11
-13 -13 -13
-15 -15 -15
0 500 1000 1500 2000 0 500 1000 1500 2000 0 500 1000 1500 2000
Altitude (m) Altitude (m) Altitude (m)

Figure 6. Performance indicators of annual precipitation estimates from ERA-40, NCEP-r1 and JRA-55 re-analysis exercises using the
period 1959/60–1973/74. Each point represents a monitoring site.

Table 3. Performance indicators of annual precipitation and seasonal precipitation values obtained from the re-analysis datasets
and the ensemble means of the re-analysis against in situ records for the period 1959/60–1973/74.
Annual precipitation Dry season precipitation Wet season precipitation
ERA-40 NCEP-r1 JRA-55 AVG ERA-40 NCEP-r1 JRA-55 AVG ERA-40 NCEP-r1 JRA-55 AVG
r 57.5 59.0 52.1 61.5 58.2 71.2 39.2 67.1 55.2 55.6 48.3 58.4
Bias −181 −291 −105 −192 27 −16 −21 −4 −83 −275 −82 −182
RMSE 427 476 471 412 57 39 54 37 468 463 468 405
NSE −2.31 −3.12 −3.03 −2.09 −3.47 −1.10 −3.02 −0.87 −3.15 −3.06 −3.15 −2.10

estimation by the coast is challenging due to its land–ocean of the re-analysis exercises improves the NSE values, but
interface characteristics and because the scale adopted by these remain negative even after the correction, suggest-
the climatic models used in the re-analysis exercises to ing some randomness of the estimation error.
describe the terrain characteristics may not be sufficiently Yang and Wang (2012) verified a number of East Asia
accurate. The region above 500 m corresponds to the summer monsoon numerical simulations and concluded
interior of Angola where precipitation gauges are scarce. that the use of the ensemble means of the different regional
The precipitation estimates are poor most probably climate models reduced the estimates bias. In Angola, the
because no data were available for the climate simulation use of the ensemble means of the re-analysis exercises
models. improves the NSE and RMSE values, but the changes in r
The NRMSE values shown in Figure 6 are quite sig- and bias are not always toward better values (Table 3).
nificant, usually above 20%, and do not show any increas-
ing or decreasing trend with altitude. The NSE values are
all negative, showing that the observed mean annual 4.2 Temperature
precipitation at each monitoring station is a better esti- Figure 7 presents the mean surface air temperature maps
mate than the re-analysis value. The correction of the bias from 1959/60 to 1973/74 and shows that all re-analysis
 HYDROLOGICAL SCIENCES JOURNAL – JOURNAL DES SCIENCES HYDROLOGIQUES 707

In Situ ERA-40

NCEP-r1 JRA-55

Figure 7. Mean temperature map for 1959/60–1973/74 obtained by co-kriging interpolation from ground observation measure-
ments and re-analysis estimates.

datasets are able to capture the general spatial distribution higher than those obtained for precipitation. In addition,
pattern of the in situ observations. Figure 8 shows the if the NSE statistic is computed using the average annual
corresponding transects, illustrating that all products temperature over the whole country as the reference
struggle to predict annual mean temperature along long- benchmark, positive values, higher than 0.9, are
itude 13°50ʹE, the transect closest to the coast. The obtained by all products for both the annual and seaso-
NCEP-r1 follows the remaining two in situ transects nal estimates, showing that the re-analysis exercises per-
more closely, while JRA-55 presents a similar profile, form better when estimating temperature and are able to
although with larger values. The failure to predict annual reproduce the spatial distribution of annual and seasonal
mean temperature along longitude 13°50ʹE occurs mainly mean temperature. For four specific locations, the ERA-
in the coastal Sumbe region and in the high-altitude 40 product provides estimates that have a zero NSE
Lubango region, where in situ temperature drops signifi- value, when computed by Equation (7).
cantly. The climate re-analysis exercises seem to fail to The NCEP-r1 offers estimates with a lower bias, with a
predict the specific local conditions of these areas. slight trend to overestimate mean annual and seasonal
Figure 9 shows the mean annual temperature values temperatures, and JRA-55 with larger bias, always over-
obtained from the re-analysis datasets against the estimating temperature. The RMSE values for all products
ground records for the 1959/60–1973/74 period, with are similar, with no product showing a clearly better per-
each point representing a monitoring site and one year formance than the others. The set of outlier points seen in
or season. The NSE values computed with Equation (8) the upper left corner of each plot are from five monitoring
are all negative, showing that the observed mean annual stations located in the above-mentioned Lubango region,
temperature at each monitoring station is usually a where the temperature is much lower than that of the
better estimate than the re-analysis estimate. The NSE surrounding region. By removing these five monitoring
values obtained for temperature estimates are, however, stations from the dataset, the values of r, bias, RMSE and
708 S. POMBO ET AL.

Figure 8. North–south transect of average temperature of the 1959/60–1973/74 record obtained at longitudes 13°50ʹE, 17°35ʹE and
20°20ʹE.

NSE improve significantly, with r for the mean annual The ensemble means of the re-analysis exercises
temperature always higher than 73.1%, and the RMSE present a higher bias than the NCEP-r1 estimates, but
always smaller than 1.5°C. led to improved values of NSE and r for the mean
In Figure 10, the performance indicators are plotted annual surface air temperature and mean dry season
against the altitude of each station to evaluate the perfor- surface air temperature (Table 4).
mance of each climate re-analysis exercise. The plots show
an increasing Nbias with increasing altitude, demonstrat-
ing a trend of the re-analysis exercises to overestimate
4.3 Evaporation
mean annual temperature at altitudes higher than 1000 m
and to underestimate it at lower altitudes. The NRMSE Evaporation is amongst the most poorly measured
values present a similar trend. The high-altitude sites hydroclimatic variables, which forces many studies to
correspond to the stations in the Lubango region. estimate it as the residual of a water-balance equation.
 HYDROLOGICAL SCIENCES JOURNAL – JOURNAL DES SCIENCES HYDROLOGIQUES 709

ERA-40 NCEP-r1 JRA-55

MEAN ANNUAL SURFACE AIR TEMPERATURE

28 28 28
y = 0.9997x y = 0.9958x y = 1.0342x
26
R2 = -2.511
26 R2 = -0.861 26 R2 = -0.476
24 24 24

22 22 22

20 20 20

18 18 18
r = 42.9% r = 55.6% r = 70.5%
16 16 16 bias = 0.9 °C
bias = 0.2 °C bias = 0.1 °C
14 RMSE = 2.3 °C 14 RMSE = 2.1 °C 14 RMSE = 2.0 °C
NSE = -9.19 NSE = -5.90 NSE = -5.92
12 12 12
12 14 16 18 20 22 24 26 28 12 14 16 18 20 22 24 26 28 12 14 16 18 20 22 24 26 28
In Situ In Situ In Situ

MEAN DRY SEASON SURFACE AIR TEMPERATURE

28 28 28
y = 1.0172x y = 1.0054x y = 1.0618x
26 R2 = -2.890 26 R2 = -0.135 26 R2 = -0.432
24 24 24

22 22 22

20 20 20

18 18 18
r = 52.7% r = 64.9% r = 68.5%
16 16 16
bias = 0.6 °C bias = 0.3 °C bias = 1.4 °C
14 RMSE = 2.2 °C 14 RMSE = 1.9 °C 14 RMSE = 2.3 °C
NSE = -6.32 NSE = -3.99 NSE = -6.58
12 12 12
12 14 16 18 20 22 24 26 28 12 14 16 18 20 22 24 26 28 12 14 16 18 20 22 24 26 28
In Situ In Situ In Situ

MEAN WET SEASON SURFACE AIR TEMPERATURE

28 28 28
y = 0.9909x y = 0.9907x y = 1.0211x
26 R2 = -1.619 26 R2 = -0.849 26 R2 = -0.233
24 24 24
22 22 22

20 20 20

18 18 18
r = 44.0% r = 51.7% r = 72.0%
16 16 16
bias = 0.03 °C bias = 0.00 °C bias = 0.66 °C
14 RMSE = 2.4 °C 14 RMSE = 2.3 °C 14 RMSE = 2.0 °C
NSE = -9.19 NSE = -6.76 NSE = -4.83
12 12 12
12 14 16 18 20 22 24 26 28 12 14 16 18 20 22 24 26 28 12 14 16 18 20 22 24 26 28
In Situ In Situ In Situ

Figure 9. Mean temperature values obtained from the re-analysis datasets against in situ records for 1959/60–1973/74. Each point
represents a monitoring site and one year or season (values in °C).

Figure 11 presents maps of the mean annual evapora- obtained with the in situ data is 1945.0 mm year−1 and
tion for the period 1959/60–1973/74 for the in situ data the homologues obtained by the climate re-analysis
and the climate re-analysis exercises, and Figure 12 exercises are 891.0 and 685.5 mm year−1 for ERA-40
show the corresponding transects. It is clear that the and JRA-55, respectively.
climate re-analysis exercises are not able to reproduce Figure 13 shows the annual evaporation and seaso-
the observed mean annual evaporation distribution, nal evaporation values obtained from the re-analysis
but the small number of monitoring stations, particu- datasets against in situ records from the period 1959/60
larly in the south of the country, hinders any strong to 1973/74, with each point representing a monitoring
conclusion on this subject. In situ data show a clear site and one year or season. The plots show that either
increasing trend from the northeast to the southwest, the re-analysis exercises are not able to estimate eva-
while both climate re-analysis exercises show an oppo- poration values, or the errors associated with the in situ
site trend. Areal mean annual evaporation in Angola measurements are significant.
710 S. POMBO ET AL.

ERA-40 NCEP-r1 JRA-55

Normalized bias 60% 60% 60%
50% 50% 50%
40% 40% 40%
30% 30%
(Nbias)
30%
20% 20% 20%
10% 10% 10%
0% 0% 0%
-10% -10% -10%
-20% -20% -20%
0 500 1000 1500 2000 0 500 1000 1500 2000 0 500 1000 1500 2000
Altitude (m) Altitude (m) Altitude (m)

60% 60% 60%

Normalized RMSE

50% 50% 50%

(NRMSE)

40% 40% 40%

30% 30% 30%

20% 20% 20%

10% 10% 10%

0% 0% 0%
0 500 1000 1500 2000 0 500 1000 1500 2000 0 500 1000 1500 2000
Altitude (m) Altitude (m) Altitude (m)

20 20 20
Range with significance [-1 ; 1] Range with significance [-1 ; 1] Range with significance [-1 ; 1]
0 0 0

-20 -20 -20

NSE

-40 -40 -40

-60 -60 -60

-80 -80 -80

-100 -100 -100

0 500 1000 1500 2000 0 500 1000 1500 2000 0 500 1000 1500 2000
Altitude (m) Altitude (m) Altitude (m)

Figure 10. Performance indicators of mean temperature estimates from ERA-40, NCEP-r1 and JRA-55 re-analysis exercises for 1959/
60–1973/74. Each point represents a monitoring site.

Table 4. Performance indicators of mean surface air temperature values obtained from the re-analysis datasets and the ensemble
means of the re-analysis against in situ records for the period 1959/60–1973/74.
Mean annual surface air temperature Mean dry season surface air temperature Mean wet season surface air temperature
ERA-40 NCEP-r1 JRA-55 AVG ERA-40 NCEP-r1 JRA-55 AVG ERA-40 NCEP-r1 JRA-55 AVG
r 42.9 55.5 70.5 72.3 52.7 64.9 68.5 68.9 44.0 51.7 72.0 61.5
Bias 0.2 0.1 0.9 0.4 0.6 0.3 1.4 0.7 0.03 0.00 0.66 0.23
RMSE 2.3 2.1 2.0 2.0 2.2 1.9 2.3 2.1 2.4 2.3 2.0 2.1
NSE −9.2 −5.9 −5.9 −5.0 −5.3 −4.0 −6.6 −3.7 −9.2 −6.8 −4.8 −5.2

5 Discussion and conclusions precipitation. All products struggle to estimate precipi-

tation in the dry season. The statistics used to evaluate
The main objective of the present research was to
the products do not identify any specific product that
identify the standard climate re-analysis dataset that
outperforms the others, although ERA-40 and JRA-55
offers the best precipitation, evaporation and tempera-
precipitation estimates are marginally superior.
ture estimates for Angola, a region with limited records
Similar results have been published in the litera-
of in situ ground monitoring data. Three climate re-
ture. Ma et al. (2009) evaluated precipitation esti-
analysis products were examined by comparing annual
mates from re-analysis exercises with ground-based
and seasonal precipitation, evaporation and tempera-
measurements in China and found that ERA-40 was
ture estimates with ground observation measurements.
better aligned with the in situ observations than
All re-analysis products exhibit similar correlation
NCEP-r1. Although relatively small biases were cal-
values between precipitation estimates and in situ
culated for China as a whole, the precipitation esti-
records with an overall trend to underestimate
mates from some regions were associated with
 HYDROLOGICAL SCIENCES JOURNAL – JOURNAL DES SCIENCES HYDROLOGIQUES 711

In Situ

ERA-40 JRA-55

Figure 11. Mean annual evaporation map for 1959/60–1973/74 obtained by co-kriging interpolation from ground observation
measurements and re-analysis estimate (values in mm).

significant estimation errors. In Brazil, results estimates for Angola and the results generally outper-
showed that the ERA-40 performed better than formed the re-analysis estimates at the annual and
other models and that the validity of NCEP precipi- seasonal levels presented in this paper.
tation estimates was questionable (Getirana et al. As expected, the power of the re-analysis exercise to
2011). Voisin et al. (2008) evaluated precipitation estimate temperature is much better than for precipita-
products for global hydrological application and tion, as temperature is a much more stable variable in
expressed a preference for the ERA-40. The authors time and space. On average, NCEP-r1 provides the best
also concluded that the precipitation datasets agreed results for the annual and seasonal temperatures.
reasonably on the seasonality but less on monthly The results for evaporation are not encouraging and
anomalies, and that the largest differences in preci- display a significant divergence between re-analysis
pitation were observed in mountainous regions and estimates and in situ records. The re-analysis exercises
regions where in situ networks are sparse, such as substantially underestimate in situ values and show a
Africa. negative correlation with in situ records.
These results highlight the limitations of re-analysis The validation of precipitation, evaporation and
data to undertake precipitation variability studies over temperature exercises remains a challenge in large
Angola. Previous studies in areas with dense gauge and poorly monitored regions because of the irregular
networks or radar coverage suggested that re-analysis distribution of gauges in such complex and highly
exercises perform better in higher latitudes and cooler variable regions. The quality of the re-analysis esti-
regions, where precipitation arises from frontal sys- mates is dependent on the number and type of obser-
tems, and that convective precipitation, common in vations assimilated in each exercise, and the number of
lower and warmer latitudes such as in Angola, is better gauges located in Angola and used in the re-analysis
characterized by satellite precipitation (Ebert et al. exercises is very small. Uppala et al. (2006) showed
2007, Sapiano and Arkin 2009, Vila et al. 2009). some maps with the datasets used in each re-analysis
Pombo et al. (2015) studied satellite precipitation exercise, indicating that only gauges in the
712 S. POMBO ET AL.

Figure 12. North–south transect of mean annual evaporation of the 1959/60–1973/74 record obtained at longitudes 13°50ʹE, 17°
35ʹE and 20°20ʹE.

northwestern part of the country, near the capital city temporal variability and a strong link to topography.
of Luanda, were used, and the results of this study The measurement uncertainty and the representative-
show that the products perform better in that region. ness of each monitoring station may also affect the
Part of the disagreement between the re-analysis comparison results.
products and the in situ observations may also be due The results of this study show that re-analysis exer-
to the scale differences. The gridded re-analysis pro- cises are able to characterize the spatial distribution of
ducts offer average values of each variable over a grid precipitation and temperature and the major fluctua-
cell with a side length of 100–200 km, which are tions of these variables over time, but are less able to
compared with point observations at monitoring sta- reproduce the observed annual and inter-annual varia-
tions. Errors arise from this type of evaluation, parti- bility of these variables at a particular site. This sug-
cularly for precipitation, which has a high spatial and gests that re-analysis exercise estimates are not yet able
 HYDROLOGICAL SCIENCES JOURNAL – JOURNAL DES SCIENCES HYDROLOGIQUES 713

ERA 40 JRA 55
ANNUAL EVAPORATION
5000 5000
R2 = -2.442 R2 = -2.271
r = -51.7% r = -53.7%
4000 bias = -755 mm 4000 bias = -949 mm
RMSE = 1 111 mm RMSE = 1 230 mm
3000 NSE = -24.20 3000 NSE = -29.92

2000 2000

1000 1000

0
0
0 1000 2000 3000 4000 5000
0 1000 2000 3000 4000 5000
In Situ In Situ

DRY SEASON EVAPORATION

2000 2000
R2 = -1.749 R2 = -1.187
r = -45.6% r = -59.5%
1500
bias = -482 mm bias = -570 mm
1500
RMSE = 624 mm RMSE = 694 mm
NSE = -31.82 NSE = -39.51
1000 1000

500 500

0 0
0 500 1000 1500 2000 0 500 1000 1500 2000
In Situ In Situ

WET SEASON EVAPORATION

2500 2500
R2 = -2.819 R2 = -2.705
r = -55.8% r = -56.0%
2000 bias = -271 mm 2000
bias = -379 mm
RMSE = 573 mm RMSE = 614 mm
1500 NSE = -12.98 1500 NSE = -15.08

1000 1000

500 500

0 0
0 500 1000 1500 2000 2500 0 500 1000 1500 2000 2500
In Situ In Situ

Figure 13. Annual evaporation values and seasonal evaporation values obtained from the re-analysis datasets against in situ records
for 1959/60–1973/74. Each point represents a monitoring site and one year or season (values in mm).

to substitute for in situ monitoring records in water Acknowledgements

resources studies in Angola; but there may be advan-
The re-analysis datasets were obtained from the NOAA,
tages in the combined use of the two types of datasets. ECMWF and JMA websites. The authors also thank the
The adoption of simple or weighted ensemble means reviewers of this manuscript for their useful and insightful
of the re-analysis results as precipitation and surface air comments.
temperature estimates may be a promising avenue for
further research. While the results of this study have
shown that simple means do not always lead to better
Disclosure statement
results, the optimized weighted ensemble means may
achieve better estimates. No potential conflict of interest was reported by the authors.
714 S. POMBO ET AL.

Funding Kummerow, C., et al., 1998. The tropical rainfall measuring

mission (TRMM) sensor package. Journal of Atmospheric
This work was partially funded by project SFRH/BD/70290/ and Oceanic Technology, 15, 809–817. doi:10.1175/1520-
2010 from the Fundação para a Ciência e a Tecnologia. 0426(1998)015<0809:TTRMMT>2.0.CO;2
Lars, E.P., 2004. National strategy plan for rehabilitation of the
hydrometric network in Angola. Oslo: Norwegian Water
References Resources and Energy Directorate, 1–31, Report No. 8.
Ma, L., et al., 2009. Evaluation of precipitation from the
Beighley, R.E., et al., 2011. Comparing satellite derived pre- ERA-40, NCEP-r1 and CMAP-1, CMAP-2 and GPCP-2
cipitation datasets using the Hillslope River Routing with ground-based measurements in China. Journal of
(HRR) model in the Congo River Basin. Hydrological Geophysical Research, 114, D09105. doi:10.1029/
Processes, 25, 3216–3229. doi:10.1002/hyp.8045 2008JD011178
Buytaert, W., et al., 2006. Spatial and temporal rainfall varia- Maidment, R.I., et al., 2012. Evaluation of satellite-based and
bility in mountainous areas: A case study from the south model re-analysis rainfall estimates for Uganda.
Ecuadorian Andes. Journal of Hydrology, 329, 413–421. Meteorological Applications, 20 (3), 308–317. doi:10.1002/
doi:10.1016/j.jhydrol.2006.02.031 met.1283
Cedeño, J. and Cornejo, M.P., 2008. Evaluation of three Nash, J.E. and Sutcliffe, J.V., 1970. River flow forecasting
precipitation products on Ecuadorian coast [online]. through conceptual models part I — a discussion of prin-
Available from: http://wcrp.ipsl.jussieu.fr/Workshops/ ciples. Journal of Hydrology, 10 (3), 282–290. doi:10.1016/
Reanalysis2008/Documents/Posters/P3-25_ea.pdf 0022-1694(70)90255-6
[Accessed 19 September 2015]. Nicholson, S.E., et al., 2003. Validation of TRMM and other
Deng, E.D., 1986. Comparison of indicator co-kriging to dis- rainfall estimates with a high-density gauge dataset for West
junctive for earthquake parametric estimation (geostatistics, Africa. Part II: validation of TRMM rainfall products.
ground motion). Thesis (PhD). University of Missouri, USA. Journal of Applied Meteorology, 42 (10), 1355–1368.
Ebert, E.E., Janowiak, J.E., and Kidd, C., 2007. Comparison doi:10.1175/1520-0450(2003)042<1355:VOTAOR>2.0.CO;2
of near-real-time precipitation estimates from satellite Pombo, S., Oliveira, R.P., and Mendes, A., 2015. Validation
observations and numerical models. Bulletin of the of remote-sensing precipitation products for Angola.
American Meteorological Society, 88, 47–64. doi:10.1175/ Meteorological Applications, 22 (3), 395–409. doi:10.1002/
BAMS-88-1-47 met.1467
ESRI, 2003. ArcGIS 9. Using ArcGis geostatistical analyst. Sapiano, M.R.P. and Arkin, P.A., 2009. An intercomparison and
USA. Available from: www.esri.com validation of high-resolution satellite precipitation estimates
Gauthier, P., et al., 2007. Extension of 3DVAR to 4DVAR: with 3-hourly gauge data. Journal of Hydrometeorology, 10,
implementation of 4DVAR at the meteorological service 149–166. doi:10.1175/2008JHM1052.1
of Canada. Monthly Weather Review, 135 (6), 2339–2354. Schaefli, B. and Gupta, H.V., 2007. Do Nash values have
doi:10.1175/MWR3394.1 value? Hydrological Processes, 21 (15), 2075–2080.
Getirana, A., et al., 2011. Assessment of different precipita- doi:10.1002/hyp.6825
tion datasets and their impacts on the water balance of the Trenberth, K.E., Koike, T., and Onogi, K., 2008. Progress and
Negro River basin. Journal of Hydrology, 404, 304–322. prospects for reanalysis for weather and climate. Eos,
Gupta, H.V., et al., 2009. Decomposition of the mean Transactions American Geophysical Union, 89 (26), 234–
squared error and NSE performance criteria: implications 235. doi:10.1029/2008EO260002
for improving hydrological modelling. Journal of Uppala, S.M., et al., 2006. The ERA-40 re-analysis. Quarterly
Hydrology, 377, 80–91. doi:10.1016/j.jhydrol.2009.08.003 Journal of the Royal Meteorological Society, 131 (612),
Hughes, D.A., 2006. An evaluation of potential use of satellite 2961–3012. doi:10.1256/qj.04.176
rainfall data for input to water resource estimation models Vila, D.A., et al., 2009. Statistical evaluation of combined
in southern Africa. In: S. Demuth, eds., et al.. Climate daily gauge observations and rainfall satellite over conti-
variability and change—hydrological impacts (Proceedings nental South America. Journal of Hydrometeorology, 10
of the Fifth FRIEND World Conference held at Havana, (2), 533–543. doi:10.1175/2008JHM1048.1
Cuba), November 2006. Wallingford, UK: IAHS, IAHS Voisin, N., Wood, A.W., and Lettenmaier, D.P., 2008.
Publ, 308, 75–80. Available from: http://iahs.info/ Evaluation of precipitation products for global hydrologi-
uploads/dms/13639.17-75-80-66-308-Hughes.pdf cal prediction. Journal of Hydrometeorology, 9 (3), 388–
Kalnay, E., et al., 1996. The NCEP/NCAR 40-year reana- 407. doi:10.1175/2007JHM938.1
lysis project. Bulletin of the American Meteorological Vörösmarty, C.J., et al., 2000. Global water resources: vulner-
Society, 77, 437–471. doi:10.1175/1520-0477(1996) ability from climate change and population growth. Science,
077<0437:TNYRP>2.0.CO;2 289 (5477), 284–288. doi:10.1126/science.2895477.284
Kobayashi, S., et al., 2014. The JRA-55 reanalysis: general Yang, H. and Wang, B., 2012. Reduction of systematic biases
specifications and basic characteristics. Journal of the in regional climate downscaling through ensemble forcing.
Meteorological Society of Japan, 93, 5–48. doi:10.2151/ Climate Dynamics, 38 (3–4), 655–665. doi:10.1007/s00382-
jmsj.2015-001 011-1006-4