GEOPHYSICAL RESEARCH LETTERS, VOL. 36, L01701, doi:10.

1029/2008GL035561, 2009

Impact of initialization strategies and observations on seasonal

forecast skill
M. Balmaseda1 and D. Anderson1
Received 1 August 2008; revised 27 October 2008; accepted 4 November 2008; published 3 January 2009.
[1] Seasonal forecasting is a rapidly developing field with will be in closer balance at the start of the integrations. Luo et
considerable effort devoted to developing comprehensive al. [2005] show that their approach gives good forecast
coupled general circulation models. Initialising these models results but those of Keenlyside et al. [2005], using a similar
is also important, as the forecast skill depends strongly on the method, were not so encouraging. Comparison shows the
way the coupled model is initialised. Three commonly used skill of Luo et al. [2005] to be slightly lower than that of the
strategies are evaluated using the ECMWF System3, a state ECMWF system S3 which uses (i), but the coupled models
of the art coupled model. The most skillful scheme is that were very different and so no firm conclusions can be drawn
which makes most use of atmospheric and oceanic data even as to the relative merits of the different initialisation strate-
though this may generate initial imbalances in the coupled gies. Alves et al. [2003], using a previous version of the
state. At seasonal time scales, increasing the balance at the ECMWF seasonal forecasting system, showed that the sea-
cost of being further from the observations can degrade the sonal forecast of the SST in the Equatorial Pacific were better
forecasts. The relative importance of different components with method (i) than with method (ii). To date no comparison
of the ocean observing system, equatorial moorings, of the three methods using a state of the art CGCM has been
altimetery and Argo floats, on forecast skill is assessed. made. This paper evaluates the impact of the above initial-
Citation: Balmaseda, M., and D. Anderson (2009), Impact of ization strategies on the coupled model drift, amplitude of
initialization strategies and observations on seasonal forecast skill, interannual variability and skill of seasonal forecasts. An
Geophys. Res. Lett., 36, L01701, doi:10.1029/2008GL035561. additional series of Observing System Experiments (OSES)
is then conducted to assess the relative importance of differ-
1. Introduction ent components of the ocean observing systems.
[2] The potential for climate predictability at seasonal 2. Evaluation of Different Initialization Strategies
time scales resides in information provided by the initial
conditions, in particular the upper ocean density structure. 2.1. Methodology, Model and Experiments
We consider three initialisation strategies which differ in the [4] Testing the impact of ocean initial conditions on the
way data are used to constrain the initial conditions. Method seasonal forecast skill requires generation of a comprehen-
(i), the most common practice at operational centres such as sive set of forecasts from the different ocean states. The
ECMWF, is to initialise the atmosphere and ocean sepa- baseline experiment for this study is the ECMWF S3
rately, assimilating all available meteorological and oceanic seasonal forecasting system [Anderson et al., 2006; Molteni
observations. The coupled system thus starts close to the et al., 2007], where the ocean initial conditions are created
observed state but whether this leads to the most skillful using the ECMWF ocean reanalysis system ORA-S3
forecasts is less obvious as the method can have undesirable [Balmaseda et al., 2008]. All available observations of
initialization shocks. In analysing the ocean, the model is temperature, salinity and altimetric sea level anomalies are
forced by surface stress and surface fluxes of heat and fresh used. The assimilation of altimeter data needs the prescrip-
water obtained from atmospheric analyses and the quality of tion of the mean dynamic topography (MDT), which in
these fluxes has a strong influence on the quality of the ORA-S3 is derived from a previous assimilation run where
ocean analyses. subsurface temperature and salinity were used (MDT1). In
[3] An alternative initialisation method (ii) is to take the methods (i) and (ii) the atmospheric fluxes are from the
same atmospheric fluxes as in (i) and to force the ocean with ERA40 reanalysis for the period January 1959 to June 2002
them but not to assimilate ocean data. In this method the and NWP operational analyses thereafter (ERA/OPS). In all
coupled model may be in closer balance since there is no methods the ocean model SSTs are strongly relaxed to
ocean data to push the ocean away from being in balance analyzed daily SST maps from the OIv2 SST product
with the winds. Luo et al. [2005] used yet a different [Reynolds et al., 2002] from 1982 onwards. In methods ii
strategy (iii) in which the fluxes were obtained from the and iii, no other ocean observations are used. The ocean
atmospheric model by forcing it with observed SST. The model has a horizontal resolution of 1°  1° with equatorial
fluxes so obtained were then used to force the ocean during refinement. For further details see Balmaseda et al. [2008].
the ocean data assimilation. This strategy can reduce the [5] The same ocean model that is used for the analyses is
initialization shock, since the atmosphere and ocean models used for the coupled forecasts, coupled daily to the atmo-
spheric model, IFS cycle 31r1 at T159 horizontal resolution
with 62 levels in the vertical, extending up to 5 hPa. Table 1
1 gives a summary of the different experiments conducted.
ECMWF, Shinfield Park, Reading, UK.
The hindcast, sometimes called reforecast experiments con-
Copyright 2009 by the American Geophysical Union. sist of 80 different start times, spanning the period 1987–
0094-8276/09/2008GL035561 2006 and sampled every three months (Jan, Apr, Jul and

L01701 1 of 5
L01701 BALMASEDA AND ANDERSON: IMPACT OF INITIALIZATION L01701

Table 1. Description of Experiments (defined in Table 2) for the 3 experiments. The amplitude of
Experiment Information in the Ocean Initial Conditions the interannual variability of the coupled model as a
i ALL SST + Atmos obs + Ocean Obs function of lead time is shown in the lower panels. This
ii NO-OCOBS SST + Atmos obs latter is calculated as the ratio between the standard devi-
iii SST-ONLY SST ation of the interannual anomalies of the coupled model
NOMOOR ALL except moorings
NOALTI ALL except altimeter
(computed separately for each ensemble member) and that
NOARGO ALL except Argo of the observed SST.
MDT0 As ALL, but using MDT0 [8] Both the model bias and the amplitude of the inter-
annual variability are sensitive to the initial conditions. In
the Eastern Pacific (NINO3; Figures 1a and 1c), ALL shows
Oct). For each date, an ensemble of 5 coupled forecasts with the strongest warm bias for forecasts initialized in April,
perturbed initial conditions is integrated to 7-months ahead. July and October. The warm bias is symptomatic of the
[6] The three initialisation strategies can also be viewed as existence of initialization shock: the coupled model is not
OSE type experiments in which atmospheric and oceanic able to maintain the slope of the thermocline in the initial
data are withdrawn as can be seen in Table 1. Differences in conditions, and fast dynamic adjustment takes place through
forecast skill between ALL (method i) and NO-OCOBS a downwelling Kelvin wave which depresses the thermo-
(method ii) are indicative of the impact of ocean observa- cline in the Eastern Pacific, shutting down the upwelling
tions, between NO-OCOBS and SST-ONLY (method iii) of and producing surface warming. The bias is close to zero in
the impact of atmospheric data that went into the ERA40/ experiment NO-OCOBS, where the initial conditions have a
OPS. The combined impact of ocean and atmosphere infor- flatter thermocline, and consequently the dynamic Kelvin
mation can be gauged by the differences between ALL and wave adjustment is weaker. The cold bias in SST-ONLY,
SST-ONLY. All skill scores have been cross-validated. which develops especially fast for October starts is likely to
be related to the thermocline being too shallow in the initial
2.2. Assessment of Skill of Different conditions, leading to an overestimation of the cooling by
Initialisation Strategies upwelling and the development of a cold bias as soon as the
[7] The evolution of the SST bias in the coupled model is strong relaxation to SST used in the initialization process is
shown in Figures 1a and 1b for regions NINO3 and NINO4 turned off.

Figure 1. (top) Forecast drift as a function of forecast lead time for 4 start months in regions (a) NINO3 and (b) NINO4
for experiments ALL, NO-OCOBS and SST-ONLY. (bottom) Variance ratio as a function of lead time for the same
experiments averaged over all start months in (c) NINO3 and (d) NINO4.

2 of 5
L01701 BALMASEDA AND ANDERSON: IMPACT OF INITIALIZATION L01701

Table 2. Definition of Area Averaged Indices monthly mean absolute error (MAE) resulting from adding
Area Name Latitudes Longitudes information from the ocean and/or atmosphere observations
NINO3 5N – 5S 150W – 90W
for forecast range 1 – 7 months appears in Figure 3a. For
NINO4 5N – 5S 160E – 150W example, in the EQ3 region the impact of not using ocean
EQ3 5N – 5S 150E – 170W obs is to increase the MAE error by 12%, of using neither
EQPAC 5N – 5S 130E – 80W ocean nor atmospheric observations is about 28%. The
EQIND 5N – 5S 40E – 120E impact of not using atmospheric observations is close to
WTIO 10N – 10S 50E – 70E
SETIO 0N – 10S 90E – 110E 15%. With the exception of EQATL, the best scores are
EQATL 5N – 5S 70W – 30E achieved by experiment ALL. This means that for the
NSTRATL 28N – 5N 80W – 20E ECMWF system, which uses i, the benefits of the ocean
data assimilation and the use of fluxes from atmospheric
(re)analyses more than offset problems arising from initial-
ization shock. In the first 3 months of the forecast (not
[9] The amplitude of the interannual variability seems to
shown), the combined information of oceanic and atmo-
be related to the magnitude of the bias, the least activity
spheric observations reduces the error by more than 40% in
occurring in the presence of warm bias. This is to be
the different areas of the Equatorial Pacific (EQ3, NINO4,
expected if convective processes set an upper limit on
NINO3). Atmospheric observations are the main contributor
how large values of SST can be and could explain why
to the reduction of forecast error. The contribution of the
the amplitude of the interannual variability in ALL is low.
ocean observations is largest in the Central Western Pacific
However, it does not explain the low levels of activity in
(13% in EQ3), but is negative in EQATL.
NO-OCOBS and SST-ONLY, suggesting that the underes-
[14] The contribution of oceanic and atmospheric obser-
timation of the interannual variability in NINO3 is not only
vations seem to be cumulative in the reduction for MAE
related to the initial conditions, but stems from other sources
error at all lead times. The OC + AT bars measure the
of error in the coupled model.
difference in skill between (i) and (iii) confirming that the
[10] In the Central Western Pacific (NINO4; Figures 1b
assimilation of atmospheric and oceanic data is markedly
and 1d) the initial conditions also have a large impact on the
better than using just SST, suggesting that the Luo et al.
model bias and interannual variability. Here, ALL shows the
[2005] approach is not the best, at least not at the forecast
smallest bias, followed by NO-OCOBS. The cold bias for
ranges considered here.
SST-ONLY is the largest. The cold biases in NO-OCOBS
and SST-ONLY are especially large during the second half
of the year, consistent with the cold tongue penetrating too
far west. In this region the amplitude of the interannual
variability is related to the mean state and to the initializa-
tion procedure. For instance, overactive upwelling, charac-
teristic of a cold tongue regime in this area, will produce
an overestimation of the interannual variability, as happens
in experiment SST-ONLY. The amplitude is underestimated
in experiment ALL, even when the bias is small. The
underestimation of the interannual variability in NINO4
for experiment ALL, and in NINO3 for all the experiments,
suggests the existence of errors not corrected with the
initialization, such as the underestimation of the atmospheric
intraseasonal variability.
[11] Balmaseda et al. [2008] show that the assimilation of
ocean observations has two main effects on the ocean mean
state: it increases the heat content of the Equatorial Pacific
by deepening the thermocline and increases the slope of the
thermocline. Results shown in Figure 1 suggest that while
the first correction is maintained during the forecast, thus
avoiding the westward penetration of the cold tongue and
the cold bias in NINO4, the slope of the thermocline is
difficult to maintain, and is lost by rapid dynamical adjust-
ment leading to the warm bias in the Eastern Pacific
(NINO3).
[12] The impact of initialization strategies in the forecast
skill appears in Figure 2 as a function of lead time for region
NINO4. In this region the most skillful forecast at all lead
times is obtained by method (i), and the worst by method
(iii). There is a clear advantage from assimilating ocean
observations. The results hold for both RMS error Figure 2. Impact of initialization strategies in forecast skill
(Figure 2a) and anomaly correlation (Figure 2b). as a function of lead time in region NINO4, in terms of
[13] Figure 3 shows the impact on forecast skill for (a) RMS error and (b) anomaly correlation. The best skill is
various regions in Table 2. The relative reduction in the obtained by experiment ALL and the worst by SST-ONLY.

3 of 5
L01701 BALMASEDA AND ANDERSON: IMPACT OF INITIALIZATION L01701

indicative of errors in the ocean initial conditions, although

comparison with independent data shows that assimilating
ocean data does improve the quality of the ocean analysis
[Balmaseda et al., 2008]. It could also reflect spurious
variability produced by the non-stationarity of the ocean
observing system. A closer look at the impact of the
different ocean observing systems for more specific time
periods is given in the next section.

3. Relative Importance of Individual Ocean

Observing Systems
[16] In the previous section we showed that the most
skillful forecasts come from analyses which make most use
of ocean observations. A further set of experiments was
conducted to assess the contributions to forecast skill from
the three main components of the observing system. In
experiment NOMOOR the information from the moored
array TAO/TRITON/PIRATA was removed, and in experi-
ment NOALTI, the altimeter-derived sea level anomalies
were not assimilated. In experiment MDT0, the external
MDT used in the assimilation of altimeter comes from
experiment NO-OCOBS, i.e. it does not contain information
about ocean observations. The impact of altimeter, moor-
ings and MDT is measured by comparing experiments ALL,
NOMOOR, NOALTI and MDT0, for the period 1993–
2006 (56 initial dates), 1993 being the start of high quality
altimetry.
[17] Figure 3b shows the relative reduction in the 1 – 7
months forecast error by including information from the
altimeter, the moored arrays and the external MDT. The
information from the mooring array is the dominant factor
in improving the skill in the different regions of the
Equatorial Pacific and improves the skill in the Equatorial
Indian Ocean. This is likely a remote effect, since there are
few Indian Ocean moorings. The remote influence of the
moorings in the Indian Ocean can occur via the ocean initial
conditions (i.e. the remote effect of assimilating data in the
Pacific propagating into the Indian Ocean via the Indone-
Figure 3. Impact of initialization in forecast skill for sian Through-flow), or via an atmospheric bridge, whereby
different regions, as measured by the reduction in mean an improvement in forecast skill in the Pacific leads to an
absolute error for the forecast range 1 – 7 months. (a) improved atmospheric circulation which impacts forecasts
Comparison of initialization strategies for the period 1987– of the Indian Ocean. The impact of the external MDT is also
2006. OCOBS indicates the differences between strategy i and quite substantial in the Pacific, and to a lesser degree the
ii which differ in the use of ocean observations. ATOBS Equatorial Indian Ocean. The effect of altimeter data is
indicates differences between ii and iii, which differ in the use more noticeable in the NINO3 and NSTRATL. Moorings,
of atmospheric data, while OC + AT gives differences between MDT and Altimeter have also a positive impact on the
i and iii and represents the combined impact of atmospheric WTIO, although the individual contributions are small. The
and oceanic data. (b) Comparison of altimeter, moorings and Equatorial Atlantic again stands out as the only region
MDT for the period 1993–2006. ALTI indicates the difference where the different observational information consistently
in skill between NO-ALTI and ALL, and MOOR the has a detrimental effect, although only the altimeter contri-
difference between NO-MOOR and ALL. MEAN indicates bution is significant at the 70% confidence level of a one-
the differences from using the different MDTs. (c) Comparison tailed T-test.
between Argo, altimeter and moorings for the period 2001– [18] The positive impact of Argo on forecast skill has
2006. ARGO represents the difference between NO-ARGO already been reported by Balmaseda et al. [2007]. To
and ALL. Only differences exceeding the 70% significant measure the relative impact of Argo compared to the other
level of a one-tailed T-test are shown. ocean observing system, here we use exactly the same
experiment (NOARGO, where the information from the
Argo floats was withdrawn), and compare it with NOM-
[15] The degraded skill in the Equatorial Atlantic may be OOR and NOALTI. The comparison is only done for the
indicative of poor balance between the ocean initial con- period 2001– 2006 (28 dates in total), as ARGO really only
ditions and the coupled model fluxes, perhaps symptomatic started in 2001. Figure 3c shows the impact on forecast skill
of coupled model errors [Davey et al., 2001]. It can also be

4 of 5
L01701 BALMASEDA AND ANDERSON: IMPACT OF INITIALIZATION L01701

of Argo, moorings and altimeter. The statistics have been Subtropical Atlantic and in the Eastern Pacific, while Argo
calculated only for the (rather short) post-Argo period has a larger effect in the Central-Western Pacific, and in the
2001 – 2006 and so the impacts are best considered as Indian Ocean. The equatorial Atlantic is a region where the
indicative rather than definitive. Figure 3c shows the forecasts are not improved through the use of ocean
dominant impact of Argo in the NINO4 (18%), and EQIND observations, probably indicative of model error.
(12%) regions. Argo is the only observing system with a [21] More sophisticated assimilation methods that reduce
significant positive impact on the WTIO and SETIO regions the initialization shock but still make full use of the
(8%). The information from the moorings is still dominant observations are desirable, though not yet developed. It
in the whole Equatorial Pacific (16%), NINO3 (15%) and has been shown that in the ECMWF seasonal forecasting
EQ3(13%), and quite large in NINO4 (16%), although here System 3 the quality of the wind product used in the
it is less important than that from Argo. Meanwhile the generation of the ocean initial conditions is instrumental
altimeter has a significant positive impact in the Equatorial for increasing the skill of the seasonal forecasts, with the
Pacific, and is the only observing system with positive winds from atmospheric reanalysis being far superior to
impact in the North Subtropical Atlantic (8%). Again, for those obtained by an atmospheric model forced by observed
this period, all the observing systems have a negative SST. This result should be taken into account in the ongoing
impact on the EQATL region. Whether this is related to efforts to design coupled model initialization strategies.
the faulty sensors in the SOLO/FSI Argo floats needs to be
investigated further. [22] Acknowledgments. The authors thank Franco Molteni for his
useful comments in the preparation of this manuscript.

4. Summary and Conclusions References

[19] The most complex assimilation used operationally is Alves, O., M. Balmaseda, D. Anderson, and T. Stockdale (2003), Sensitiv-
ity of dynamical seasonal forecasts to ocean initial conditions, Q.J.R.
uncoupled initialisation in which atmospheric observations Meteorol. Soc., 130, 647 – 668.
are assimilated into an atmospheric GCM, and ocean Anderson, D. L. T., et al. (2006), Development of the ECMWF seasonal
observations are assimilated into an ocean model forced forecast System 3, ECMWF Tech. Memo. 503, Eur. Cent. for Medium
by fluxes from the atmospheric analysis. Complex assimi- Range Weather Forecasts, Reading, U.K.
Balmaseda, M., D. Anderson, and A. Vidard (2007), Impact of Argo on
lation methods are used in both media. Simpler methods analyses of the global ocean, Geophys. Res. Lett., 34, L16605,
without complex assimilation have also been advocated doi:10.1029/2007GL030452.
[e.g., Luo et al., 2005]. These are appealing as they do Balmaseda, M. A., A. Vidard, and D. Anderson (2008), The ECMWF
ORA-S3S ocean analysis system, Mon. Weather Rev., 136, 3018 – 3034.
not require large investment in assimilation strategies and Davey, M., et al. (2001), STOIC: A study of coupled model climatology
may have reduced initialisation shock, since the coupled and variability in tropical ocean regions, Clim. Dyn., 18, 403 – 420.
model should be in better balance at the start of the forecast. Keenlyside, N. S., M. Latif, M. Botzet, J. Jungclaus, and U. Schulzweida
(2005), A coupled method for initializing El Niño Southern Oscillation
However, the results here, using a controlled methodology forecasts using sea surface temperature, Tellus, Ser. A, 57, 340 – 356.
and a common coupled GCM, indicate that this is not so. Luo, J. J., S. Masson, S. Behera, S. Shingu, and T. Yamagata (2005),
The best forecasts come from the most comprehensive Seasonal climate predictability in a coupled OAGCM using a different
system. It is not always true that the model biases are approach for ensemble forecasts, J. Clim., 18, 4474 – 4497.
Molteni, F., et al. (2007), ECMWF Seasonal Forecast System 3, CLIVAR
smaller in the simpler systems as might have been thought. Exch., 43, 7 – 9.
[20] Observing system experiments have been used to Reynolds, R., N. Rayner, T. Smith, D. Stokes, and W. Wang (2002), An
gauge the contribution of different observation types to the improved in situ and satellite SST analysis for climate, J. Clim., 15,
1609 – 1625.
forecast skill. All the observing systems contribute to the
improvement of seasonal forecast skill almost everywhere.
The moorings having a dominant effect in the equatorial D. Anderson and M. Balmaseda, ECMWF, Reading RG2 9AX, UK.
Pacific, altimeter data have a significant effect in the North ([email protected])

5 of 5