ML Prediction of Global Ionospheric TEC Maps

15427390, 2022, 9, Downloaded from https://agupubs.onlinelibrary.wiley.com/doi/10.1029/2022SW003135 by National Medical Library The Director, Wiley Online Library on [18/10/2022].
See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
RESEARCH ARTICLE ML Prediction of Global Ionospheric TEC Maps
10.1029/2022SW003135
Lei Liu1 , Y. Jade Morton1 , and Yunxiang Liu1
Key Points: 1
Ann and H. J. Smead Aerospace Engineering Sciences Department, University of Colorado Boulder, Boulder, CO, USA
• F our convolutional long short-term
memory (convLSTM)-based models
are investigated to forecast global
ionospheric total electron content Abstract This paper applies the convolutional long short-term memory (convLSTM)-based machine
maps with up to 24 hr of lead time at a learning models to forecast global ionospheric total electron content (TEC) maps with up to 24 hr of lead time
1-hr interval at a 1-hr interval. Four convLSTM-based models were investigated, and the one that implements the L1 loss
• The one that implements the L1
loss function and residual strategy function and the residual prediction strategy demonstrates the best performance. The convLSTM models are
demonstrates the best performance trained and evaluated using Center for Orbit Determination in Europe (CODE) global TEC maps over a period
among four convLSTM-based models of nearly seven years from 19 October 2014 to 21 July 2021. Results show that the best convLSTM model
• This best performing convLSTM
model also shows more accurate outperforms the 1-day predicted global TEC products released by CODE analysis center (c1pg) and persistence
prediction compared to c1pg and models under various levels of solar and geomagnetic activities, except for a lead time beyond 8 hr during the
persistence models storm time where the c1pg has slightly better performance. The convLSTM forecasting performance degrades
as the lead time increases.
Supporting Information:
Supporting Information may be found in Plain Language Summary Reliable specification and prediction of ionospheric total electron
the online version of this article. content (TEC) are not only helpful for mitigating uncertainties in global navigation satellite system-based
position, navigation, and timing services, but also for timely warning of space weather activities. We apply
Correspondence to: convolutional long short-term memory (convLSTM)-based machine learning models to forecast global
L. Liu, ionospheric TEC maps with up to 24 hr of lead time at a 1-hr interval. Four convLSTM-based models were
[email protected]
investigated, and the one that implements the L1 loss function and residual prediction strategy demonstrates the
best performance. Moreover, our developed convLSTM model shows competitive performance when compared
Citation:
to two conventional models under various levels of solar and geomagnetic activities.
Liu, L., Morton, Y. J., & Liu, Y.
(2022). ML prediction of global
ionospheric TEC maps. Space Weather,
20, e2022SW003135. https://doi. 1. Introduction
org/10.1029/2022SW003135
Ionospheric total electron content (TEC) refers to the total number of electrons integrated along a radio wave
Received 23 APR 2022 propagation path within a unit cross section𝐴𝐴area (unit ∶ TECU, 1 TECU = 1016 electrons∕m2 ). Satellite navi-
Accepted 4 SEP 2022 gation and radio communication system performances are degraded when radio waves traverse the ionosphere
in the presence of the ionospheric TEC gradients and plasma density irregularities (Basu et al., 1990; Jacobsen
& Andalsvik, 2016; Liu, Zou, Yao, & Aa, 2020; Yang et al., 2020; Yeh & Liu, 1982). Reliable specification
and prediction of ionospheric TEC are not only helpful for mitigating uncertainties in global navigation satellite
system (GNSS)-based position, navigation, and timing (PNT) services, but also for timely warning of space
weather activities (Jakowski et al., 2012; Klobuchar, 1987; Morton et al., 2020).
Ionospheric TEC maps can be constructed using networks of dual-frequency GNSS receiver measurements
(Chen et al., 2021; Hernandez-Pajares et al., 2009; Schaer et al., 1999; Yao et al., 2018). A major challenge is
to forecast global TEC maps accurately, especially for relatively long lead times such as one day. This is due to
the ionospheric dynamic nature driven by solar-geomagnetic activities and multi-scale ionospheric processes
(Camporeale, 2019; Cowley, 2000; Liu, Zou, Yao, & Aa, 2020; Yao et al., 2016). Numerous approaches have
been developed to forecast global ionospheric TEC maps. For example, Schaer et al. (1999) presented the least
squares collocation method to extrapolate spherical harmonic (SH) coefficients that were used for construct-
ing TEC maps. The resulting global ionosphere maps (GIM) are one of the international GNSS service (IGS)
official ionospheric products released by the Center for Orbit Determination in Europe (CODE). García-Rigo
et al. (2011) developed a global TEC prediction model based on the discrete cosine transform. Wang et al. (2018)
© 2022. The Authors. developed an adaptive autoregressive (AR) model to predict the SH coefficients and reproduce the global TEC
This is an open access article under map by using the predicted SH coefficients.
the terms of the Creative Commons
Attribution License, which permits use, In recent years, machine learning (ML) techniques have become a promising and effective tool to predict global
distribution and reproduction in any
TEC maps. For example, Perez (2019) predicted global ionospheric TEC based on a fully connected neural
medium, provided the original work is
properly cited. network (NN). The model was evaluated by the position error in GNSS single frequency ionospheric delay
LIU ET AL. 1 of 13
15427390, 2022, 9, Downloaded from https://agupubs.onlinelibrary.wiley.com/doi/10.1029/2022SW003135 by National Medical Library The Director, Wiley Online Library on [18/10/2022]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
Space Weather 10.1029/2022SW003135
correction. However, the equatorial ionization anomalies (EIA) were not well captured by the predicted global
TEC maps. Lee et al., 2020 predicted daily global TEC maps with a 2-hr interval using an image-based condi-
tional generative adversarial network. The results demonstrated that the model had slightly better performance
compared to the CODE prediction products. Liu, Zou, Yao, and Wang (2020) applied the long short-term memory
(LSTM) network to forecast SH coefficients for the next two hours by incorporating SH coefficients, solar data,
and geomagnetic data. The model showed competitive prediction performance compared to traditional models,
while further study is still needed to extend its prediction lead time. Cesaroni et al. (2020) applied a nonlinear
AR NN with external input (NARX) to forecast 18 grid points from GIM 24 hr ahead and extended it to the
global scale by minimizing the difference between the NARX prediction and NeQuick-2 output. Results show
that the approach was able to reproduce typical ionospheric structures, especially under disturbed ionospheric
conditions. Chen et al. (2022) developed a multi-step auxiliary algorithm to forecast global TEC maps based on
the LSTM network. The results showed that the prediction model had a good generalization performance. Boulch
et al. (2018) presented convolutional recurrent neural networks for global ionospheric TEC prediction using
CODE TEC data from 2014 to 2016. Their results showed that the prediction performance was comparable with
that of the AR, autoregressive moving average, and the radial basis function NN.
This paper extends the work of Boulch et al. (2018) to implement four variations of an image-based convolutional
long short-term memory (convLSTM) ML algorithm to forecast global TEC maps with a lead time up to 24 hr
at a 1-hr interval. The CODE TEC maps over a 7-year span from 2014 to 2021 collected under various levels
of solar and geomagnetic activities are used in the implementations. The best performing model among the four
implementations is selected based on a set of skill scores, and is compared against two conventional prediction
models under various levels of solar and geomagnetic conditions to demonstrate the superior performance.
Main contributions of this work compared to that of Boulch et al. (2018) include the use of an extensive global
TEC data set spanning 7 years from October 2014 to July 2021, the improved time resolution from 2 hr to 1 hr,
different data segmentation for training, testing, and validation, the prediction of residual TEC maps between
consecutive days at the same hour, as well as the utilization of batch normalization (Ioffe & Szegedy, 2015) and
dropout (Wager et al., 2013), and comprehensive performance evaluations.
This paper is organized as follows. Section 2 describes the data used in the study, the specific ML algorithm
implementations, and evaluation metrics. Section 3 presents qualitative and statistical evaluation. Section 4
discusses the conclusion.
2. Data and Methodology

2.1. Data Description
Global ionospheric TEC data are publicly available from many Ionosphere Associate Analysis Centers, such as
CODE, University Politecnica Catalonia (UPC), European Space Agency (ESA), and Jet Propulsion Laboratory
(JPL). CODE's Global Ionosphere Maps (GIM) are gridded TEC maps that are represented by the SH function
derived from worldwide ground-based GNSS measurements (Schaer et al., 1999). The temporal resolution of the
CODE GIM has been updated from 2 hr to 1 hr since 19 October 2014. Moreover, studies have shown that CODE
GIM has better precision and reliability than UPC, ESA, and JPL (Hernandez-Pajares et al., 2009; Li et al., 2015;
Roma-Dollase et al., 2018). In this study, we aim at predicting global TEC maps at 1-hr interval, so CODE TEC
maps after 19 October 2014 are used. Figure 1 shows an example of the CODE TEC map at 12:00 UT on 19
October 2014. Locations of IGS GNSS receivers, from which measurements were used to produce this map, are
marked by white colored dots. The spatial resolution
𝐴𝐴 is 2.5◦ latitude
𝐴𝐴 by 5◦ longitude (71 in height by 73 in width).
To ease the ML training process, the dimension of the TEC map is resized𝐴𝐴from 71 ×𝐴𝐴73 to 73 × 73 by padding
zero values over north and south poles. The grid size remains
𝐴𝐴 2.5◦ × 5◦ .
The hourly CODE TEC data used in this study are from 19 October 2014 to 26 July 2021, which are divided into
training (60%), validation (20%), and testing (20%) sets. The training data set is used to fit the model by updating
its parameters via the gradient descent method. The validation set is applied to check and ensure that the model
is not overfitted or underfitted to the training set. The testing set is used to test the trained model's ability to
generalize to new and previous unseen data (Goodfellow et al., 2016). To ensure that all three sets of data contain
low, moderate, and strong solar activity levels and capture seasonal effects, we adopt the following novel data
segmentation strategy as illustrated in Figure 2. The data set is divided into multiple groups, where each group
LIU ET AL. 2 of 13
Figure 1. An example of the global total electron content (TEC) map from the centre for orbit determination in Europe
global ionosphere maps at 12:00 UT on 19 October 2014. White colored dots represent global navigation satellite system
receivers used to generate the TEC map. The dashed gray line represents the geomagnetic equator. The color scale indicates
the value in TECU.
contains 50 days of data. For each group (see the bottom panel of Figure 2), the data in the first 30 days (yellow
shaded area) are used for training, and the next 10 days (red shaded area) and last 10 days (cyan shaded area) are
used for validation and testing, respectively. Based on this criteria, 35,784, 11,760, and 11,760 data samples are
obtained for training, validation, and testing, respectively. We use the KP index as an indicator of the geomagnetic
storm level. A geomagnetic storm is declared if KP ≥ 4 in this study, otherwise it is defined as quiet time. As a
result, the quiet/storm ratio is around 85%/15% for training, validation, or testing sets (see Figure S1 in Support-
ing Information S1). This means that all of the three data sets cover similar proportion of quiet/storm data.
Figure 2. Top: Data ranging from 19 October 2014 to 21 July 2021 are used. Each group (50 days) in the data set is divided
into training (yellow shaded area), validation (red shaded area) and testing (cyan shaded area) subsets, respectively. Bottom:
A close-up view of one group of data starting on 19 October 2017. The F10.7 indicator (blue line) shows the daily solar
activity level.
LIU ET AL. 3 of 13
Figure 3. Illustration of machine learning architecture.𝐴𝐴Input: 𝐴𝐴1 , 𝑋𝑋2 , . . . , 𝑋𝑋23 , 𝑋𝑋24 are input total electron content (TEC)
maps from the past 24 hr. Output:
𝐴𝐴 𝑋𝑋̂ 25 , 𝑋𝑋̂ 26 , . . . , 𝑋𝑋̂ 47 , 𝑋𝑋̂ 48 are 24 predicted targets in the next 24 hr. The targets can be either
TEC maps or TEC map residuals, depending on the prediction strategy. Conv: convolutional layer. Convolutional long
short-term memory: convolutional long short-term memory layer. Deconv: deconvolutional (deconv) layer.
2.2. ConvLSTM ML Algorithm
The convLSTM architecture is capable of learning features from a spatiotemporal sequence. It has been success-
fully applied in many fields of multi-dimensional spatiotemporal predictions (Shi et al., 2015, 2017; Liu
et al., 2021). In this study, the convLSTM layer is used as the core module to predict global TEC maps. The
detailed structure of the convLSTM module can be found in Figure S2 in Supporting Information S1.
Figure 3 illustrates the convLSTM-based ML architecture, which consists of four parts: input, encoder, decoder,
and output. The dimensions of the input and output maps are the 𝐴𝐴 same (73 × 73 ). The inputs consist of 24 consec-
utive TEC 𝐴𝐴maps (𝐴𝐴1 , 𝑋𝑋2 , . . . , 𝑋𝑋23 , 𝑋𝑋24 ) with a 1-hr interval. These maps are fed into three encoder blocks. Each
encoder block includes a convolutional (conv) layer and a convLSTM layer. The conv layer and the convLSTM
layer are used for down-sampling and learning spatiotemporal features, respectively. Similarly, 3 decoder blocks
are composed of deconvolutional (deconv) and convLSTM layers. The hidden features generated by the encoder
blocks are used as the inputs for the decoder blocks. The encoder parts are then unfolded using the decoder blocks
to predict 24 future maps, which are elements of the output 𝐴𝐴 (𝑋𝑋̂ 25 , 𝑋𝑋̂ 26 , . . . , 𝑋𝑋̂ 47 , 𝑋𝑋̂ 48 ) shown in Figure 3. Detailed
descriptions of this architecture can be found in Shi et al. (2015) and Liu et al. (2021). Here, two prediction strat-
egies are implemented to predict global TEC maps.
Residual prediction: The output

𝐴𝐴 (𝑋𝑋̂ 25 , 𝑋𝑋̂ 26 , . . . , 𝑋𝑋̂ 47 , 𝑋𝑋̂ 48 ) is the TEC residual between the map at𝐴𝐴time 𝐴𝐴 and
𝐴𝐴 𝐴𝐴 − 24 , namely,
(1)
𝑋̂ 𝑡 = 𝑇𝐸𝐶(𝑡) − 𝑇𝐸𝐶(𝑡 − 24)
where
𝐴𝐴 𝐴𝐴 = 25, 26, . . . , 47, 48 . Once the TEC residual is produced by the ML model, the predicted TEC map is
obtained by adding the background TEC from the previous day to the predicted TEC residual to obtain the
predicted TEC.
Direct prediction: TEC maps are predicted directly by the ML model. This means the model output
𝐴𝐴 (𝑋𝑋̂ 25 , 𝑋𝑋̂ 26 , . . . , 𝑋𝑋̂ 47 , 𝑋𝑋̂ 48 ) is the hourly predicted TEC map for the next day.
The parameter configurations of this architecture, such as channel number, filter size, stride, and padding, as well
as the height and width of each output matrix after operations of conv, convLSTM and deconv layers, can be
found in Table 1. Definitions on these parameters can be found in Text S1 in Supporting Information S1. Total
number of parameters in the ML model is 915,282.
2.3. Evaluation Metrics
To quantify the prediction performance, the following metrics are utilized:
LIU ET AL. 4 of 13
Table 1
Detailed Parameter Configurations of the Model Architecture
Channel number Height and width
Network Module I O Filter size Stride Padding I O

Encoder conv 1 8 𝐴𝐴 3 × 3 𝐴𝐴 2 × 2 𝐴𝐴 1 × 1 𝐴𝐴 73 ×𝐴𝐴73 37 × 37
convLSTM 8 16 𝐴𝐴 5 × 5 𝐴𝐴 1 × 1 𝐴𝐴 2 × 2 𝐴𝐴 37
37 ×𝐴𝐴 37 × 37
conv 16 16 𝐴𝐴 3 × 3 𝐴𝐴 2 × 2 𝐴𝐴 1 × 1 𝐴𝐴 37 ×𝐴𝐴37 19 × 19
19 ×𝐴𝐴 19 × 19
conv 32 32 𝐴𝐴 3 × 3 𝐴𝐴 2 × 2 𝐴𝐴 1 × 1 𝐴𝐴 19
19 ×𝐴𝐴 10 × 10
10 ×𝐴𝐴 10 × 10
Decoder convLSTM 32 32 𝐴𝐴 5 × 5 𝐴𝐴 1 × 1 𝐴𝐴 2 × 2 𝐴𝐴 10
10 × 𝐴𝐴 10 × 1 0
deconv 32 32 𝐴𝐴 3 × 3 𝐴𝐴 2 × 2 𝐴𝐴 1 × 1 𝐴𝐴 10
10 × 𝐴𝐴 19 × 19
19 ×𝐴𝐴 19 × 19
deconv 32 32 𝐴𝐴 3 × 3 𝐴𝐴 2 × 2 𝐴𝐴 1 × 1 𝐴𝐴 19
19 × 𝐴𝐴 37 × 37
37 ×𝐴𝐴 37 × 37
deconv 16 8 𝐴𝐴 3 × 3 𝐴𝐴 2 × 2 𝐴𝐴 1 × 1 𝐴𝐴 37
37 × 𝐴𝐴 73 × 73
conv 8 1 𝐴𝐴 1 × 1 𝐴𝐴 1 × 1 𝐴𝐴 0 × 0 𝐴𝐴 73
73 ×𝐴𝐴 73 × 73
Note. I and O represent input and output of each module, respectively.
⎧ 1 ∑𝑁
⎪ ME = (𝑥̂ 𝑖 − 𝑥𝑖 )
⎪ 𝑁 √ 𝑖=1
⎪ 1 ∑𝑁
⎪RMSE = ̂ 𝑖 − 𝑥 𝑖 )2
𝑖=1 (𝑥
(2) 𝑁
⎨ ∑
⎪MAE = 1 𝑁 |𝑥̂ 𝑖 − 𝑥𝑖 |
⎪ 𝑁 𝑖=1
⎪CC = 𝑐𝑜𝑣 (𝑥̂ 𝑖 , 𝑥𝑖 )
⎪ 𝜎𝑥̂ 𝑖 ⋅ 𝜎𝑥𝑖
⎩
where ME is the mean error; RMSE is the root mean square error; MAE is mean absolute error; CC is the Pearson
correlation coefficient. 𝐴𝐴 𝐴𝐴 is the number of the TEC𝐴𝐴bins. 𝐴𝐴 𝐴𝐴𝑖𝑖 and 𝐴𝐴𝐴 𝑖𝑖 are the gridded TEC values of the ground truth
and prediction, respectively. 𝐴𝐴 𝐴𝐴𝐴𝐴𝐴𝐴 (𝑥𝑥̂ 𝑖𝑖 , 𝑥𝑥𝑖𝑖 ) is the covariance between
𝐴𝐴 𝐴𝐴𝐴 𝑖𝑖 and
𝐴𝐴 𝐴𝐴 𝐴𝐴𝑖𝑖 . 𝐴𝐴𝐴𝐴𝑥𝑥̂ 𝑖𝑖 and 𝐴𝐴𝑥𝑥𝑖𝑖 are the standard deviations
𝐴𝐴 of 𝐴𝐴𝐴 𝑖𝑖 and 𝐴𝐴𝑖𝑖 , respectively.
3. Results and Discussions

The models are implemented in Python with the PyTorch package and trained on a server with two 12 GB
NVIDIA TITAN GPUs, dual 10-core Intel(R) Xeon(R) Silver 4114 CPUs @ 2.20 GHz, and 251 GB of RAM.
In this study, the Adam optimizer is employed (Kingma & Ba, 2014). The Leaky rectified linear unit (ReLU)
activation function with a leaky value of 0.2 is used to avoid dead neurons in the networks caused by conventional
ReLU due to the “dying ReLU problem” (Xu et al., 2015). Batch normalization is implemented to accelerate
training by reducing internal covariate shift (Ioffe & Szegedy, 2015). Dropout regularization is used to reduce
overfitting and to improve the model generalization ability (Wager et al., 2013). The learning rate and batch size
are set to 5e−5 and 72 based on trial-and-error runs, respectively. Moreover, a learning rate schedule technique,
called ReduceLROnPlateau, is used in this study. It can monitor a quantity and decays the learning rate when the
quantity stops improving. It is set as following in this study: If no improvement is seen for 10 continuous epochs,
the learning rate will be reduced by a factor of 0.6.
Four convLSTM-based models with different combinations of loss functions

𝐴𝐴 𝐴𝐴1 and 𝐴𝐴2 . Details about these two
𝐴𝐴
loss functions can be found in Text S2 in Supporting Information S1 and prediction strategies (direct and residual
predictions, see details in Section 2.2) are implemented. These four models are shown in Table 2.
LIU ET AL. 5 of 13
Table 2 Table 3 shows performance comparison of the four models on training, vali-
Four Convolutional Long Short-Term Memory-Based Models Implemented dation and testing sets, respectively. Model C has the best performance with
in This Study by Combining Different Loss Functions and Prediction lower RMSE and MAE values and higher CC values compared to the other
Strategies three models on all data sets. This may be due to two reasons. First, 𝐴𝐴 the 𝐴𝐴1
Model Loss function Prediction strategy loss is capable of forcing the model to obtain accurate predictions that are far
away from the mean values (Dietterich, 2002; Wang et al., 2022), which is
A 𝐴𝐴 𝐴𝐴1 Direct prediction
especially helpful for predicting TEC values with large magnitudes, such as
B 𝐴𝐴 𝐴𝐴2 Direct prediction
the TEC peaks around the local noon sector and EIAcrest regions. Second, the
C 𝐴𝐴 𝐴𝐴1 Residual prediction utilization of residual prediction strategy removes the background ionosphere,
D 𝐴𝐴 𝐴𝐴2 Residual prediction forcing the model to only focus on the smaller range associated with deviations
of the ionosphere. Because of the better performance of model C, ML predic-
tion results and performances discussed in the remaining paper are all obtained
from model C. In the remaining paper, the convLSTM refers to model C.
3.1. Qualitative Case Studies Using the Test Data Set
In this case study, we compare the convLSTM prediction with that of c1pg during the main phase of a storm event
that is an example randomly selected from the testing data set. The c1pg is 1-day predicted global ionospheric
TEC map released by CODE analysis center, and it is produced based on the extrapolation of SH coefficients
(Schaer et al., 1999). Figure 4 shows the 1-hr interval TEC maps predicted by the convLSTM and c1pg models.
The codg refers to CODE's GIM, which is the ground truth. The plots show that the predicted TEC maps from
convLSTM and c1pg agree well with the truth. In particular, large-scale ionospheric patterns, such as the daytime
EIA crests, are well-preserved. We can also observe from Figure 4 that the predicted TEC maps from convLSTM
are closer to the truth than those from c1pg. For example, The TEC enhancement shown 𝐴𝐴 near 30◦ E − 80◦ E longi-
tude
𝐴𝐴 and 40 S − 60 S latitude of codg is well reproduced in the convLSTM model (see red circles in the middle
◦ ◦
column of Figure 4), while the c1pg fails to capture it. It is noted that the TEC enhancement shown in the codg is
confirmed by checking the Madrigal TEC map (see Figure S3 in Supporting Information S1).
Figure 5 shows the difference between the truth and predicted maps for the storm event shown in Figure 4. The
magnitude of the TEC difference maps from convLSTM is clearly smaller than that from c1pg. This indicates
the convLSTM ML model has a better prediction performance than c1pg. In addition, an example during the
geomagnetic quiet time also corroborates the same conclusion that the convLSTM has a better performance (see
Figure S4 in Supporting Information S1 and Movie S2).
Figure 6 compares the convLSTM model performance with the c1pg and persistence models for three additional
storm events from the testing data set. The persistence model assumes that the predicted TEC map at𝐴𝐴time 𝐴𝐴 is the
same as 𝐴𝐴
that at 𝐴𝐴 − 24 . Figure 6a shows the geomagnetic conditions using KP (left axis) and Dst (right axis) indi-
ces from April 12 to 19 in 2016. The Dst minima for these three storm events are −56, −61, and −58 nT, respec-
tively. They are marked by magenta dots in the right axis in Figure 6a. Figures 6b and 6c show time series of the
hourly ME and RMSE variations for these three models during these three storm events. In general, the convL-
STM model shows smaller ME and RMSE errors than the persistence model, indicating the convLSTM has better
prediction capability than the persistence model for the storm time. Moreover, the convLSTM model shows an
improved performance over the c1pg for the first few hours of each day during the storm time, though the convL-
Table 3
Quantitative Results of Four Convolutional Long Short-Term Memory-Based Machine Learning Models on Different Data
Sets
RMSE (TECU) MAE (TECU) CC
Model Train Validation Test Train Validation Test Train Validation Test
A 1. 90 1.97 1.94 1.22 1.25 1.24 0.97 0.97 0.96
B 2.0 2.03 2.01 1.34 1.35 1.33 0.96 0.96 0.96
C 1.72 1.90 1.88 1.12 1.20 1.20 0.98 0.97 0.97
D 2.06 2.04 2.03 1.35 1.33 1.33 0.96 0.97 0.96
LIU ET AL. 6 of 13
Figure 4. A testing example of the predicted global total electron content (TEC) maps at a 1-hr interval during the main
phase of one geomagnetic storm event (10:00–13:00 UT, 6 May 2015), where the KP varies from 3.7 to 5.3, and F10.7 is
138.6. The left column represents the TEC maps from codg, which are the ground truth. The middle and right columns
represent the predicted TEC maps from convolutional long short-term memory and c1pg, respectively. Full sets of the
predicted global TEC maps with lead times up to 24 hr are available in Movie S1.
STM shows a comparable or slightly degraded performance for the rest hours of each day when compared to the
c1pg. This sightly storm-time prediction performance degradation of the convLSTM model is also observed in
Figure 8b when compared to the c1pg. In future, more storm-time TEC data or external solar/geomagnetic data
will be involved in training the convLSTM model to improve the model storm-time TEC prediction capability.
3.2. Statistical Evaluation Using the Test Data Set

A statistical evaluation is also conducted under varying levels of solar and geomagnetic activities. The convL-
STM is compared to c1pg and a persistence model. F10.7 and KP parameters are used as indicators of solar and
geomagnetic levels, respectively. Table 4 shows the definitions of the high/low solar activity and geomagnetic
storm/quiet levels.
Figure 7 shows comparison results under low (a) and high (b) solar activities on the testing set as functions of the
lead time. Four metrics are plotted for each solar activity level. The following observations are obtained:
1. T he convLSTM model shows better prediction performance compared to c1pg and persistence models for all
lead times and with all metrics.
2. A slight performance degradation is observed for convLSTM under high solar activities compared to the ones
under low solar activities. For example, the RMSE increases from 1.78 to 3.31 TECU. Such levels of prediction
error difference are within the margin of the truth precision, which is around 2–8 TECU (Hernandez-Pajares
et al., 2009; Li et al., 2015; Roma-Dollase et al., 2018).
LIU ET AL. 7 of 13
Figure 5. Comparison of the total electron content difference maps between codg and convolutional long short-term memory
(convLSTM). Left column: codg minus convLSTM. Right column: codg minus c1pg.
3. T
he predictive capability of the convLSTM degrades with increasing lead time, while c1pg and persistence
models show almost no dependence on the lead time.
Figure 8 compares the performance using the testing data set collected under geomagnetic quiet (a) and storm (b)
conditions. We have the following observations:
1. T he convLSTM model shows better performance compared to other two models for all lead times tested under
geomagnetic quiet conditions.
2. During geomagnetic storms, the prediction performance of the convLSTM remains the best when the lead
time is smaller than 8 hr. The c1pg shows a slightly better performance compared to convLSTM when the
lead time is longer than 8 hr, and there is a notable performance improvement with increasing lead time under
geomagnetic storm conditions. Additional investigations may be required to study this behavior.
3. For the convLSTM model, the minimum RMSE errors for quiet/storm time are 0.81/1.28 TECU, and the
maximum RMSE errors for quiet/storm time are 2.15/3.26 TECU. This indicates that the RMSE degradation
of convLSTM is not significant under geomagnetic storm conditions. Similar observations can be found from
other metrics. This may be due to the fact that some features from geomagnetic activities have already been
contained in the 24 input TEC maps of the convLSTM model, thus leading to an insignificant performance
degradation under storm conditions.
LIU ET AL. 8 of 13
Figure 6. (a) KP (left axis) and Dst (right axis) variations during 12–19 April, 2016 where the minimum Dst for each storm
event is marked by the magenta point in the right axis. Hourly ME (b) and RMSE (c) errors from the convolutional long
short-term memory (red points), c1pg (blue points), and persistence (cyan points) models. The average ME and RMSE values
of each model for the covered time periods are shown in the legend.
Figure 9 plots the ME (top panel) and RMSE (bottom panel) errors of the convLSTM, c1pg and persistence
models on the test set as functions of latitude. The convLSTM has the smallest ME error. Moreover, the ME
errors of the convLSTM and persistence model stabilize around zero across all latitudes. This indicates that there
are significant less systematic biases in the TEC maps predicted by convLSTM and the persistence models as
compared to c1pg. However, c1pg underestimates/overestimates the predicted TEC over the mid-high latitude of
the southern/northern hemispheres. In terms of RMSE errors, the convLSTM shows an improved performance
over the other two models over various latitudes. In addition, the RMSE errors of three models show obvious
latitudinal dependence, and two peaks are located near geographic latitude of
𝐴𝐴 10◦𝐴𝐴S and 20◦ N , respectively. This is probably related to strong ionospheric
Table 4
The Threshold of Different Solar and Geomagnetic Activity Levels gradients around the daily EIA crest.
Indicators Threshold Defined levels

F10.7 𝐴𝐴 𝐴𝐴 10.7 ≥ 120 High solar activity
4. Conclusions
𝐴𝐴 𝐴𝐴 10.7 < 120 Low solar activity This paper discusses convLSTM-based ML algorithms implemented to
KP 𝐴𝐴 KP ≥ 4 Geomagnetic storm predict daily global TEC maps using CODE data collected from 19 October
𝐴𝐴 KP < 4 Geomagnetic quiet
2014 to 21 July 2021. Among four different implementations, the convLSTM
LIU ET AL. 9 of 13
Figure 7. Performance comparison among convolutional long short-term memory (convLSTM), c1pg, and the persistence
models under low (a) and high (b) solar activity levels on the test data set. Red, blue, and cyan curves in each panel refer
to metrics obtained from convLSTM, c1pg, and persistence models, respectively. Uncertainties (3 times standard error) for
different metrics are included in each panel. Mean metrics across all lead time are illustrated at the bottom of each panel.
model that combines𝐴𝐴 the 𝐴𝐴1 loss function and residual prediction strategy shows the best performance. This
best-performing convLSTM model also shows more accurate prediction compared to c1pg, which is a 1-day
predicted global TEC product released by CODE analysis center. Moreover, prominent structures, such as typical
EIA and TEC enhancement over the southern hemisphere, are successfully reproduced by the convLSTM model.
Our statistical evaluation shows that the convLSTM model significantly outperforms both the c1pg and persis-
tence models under all levels of solar activities for all lead times tested and at all latitudes. A slight performance
LIU ET AL. 10 of 13
Figure 8. Similar to Figure 7, but for geomagnetic quiet (a) and storm levels (b) on the testing set.
degradation is observed for convLSTM under high solar activities compared to the ones under low solar activi-
ties. When compared to c1pg and persistence models, the convLSTM model shows better performance for all lead
times tested under geomagnetic quiet conditions and for lead times below 8 hr of storm conditions. For lead times
beyond 8 hr during the storm times, the c1pg has slightly better performance. The convLSTM model shows an
insignificant performance degradation during the storm time compared to the quiet time. The convLSTM model
degrades with the increased lead time, while only a very weak time dependence is observed in results from the
c1pg and persistence models.
LIU ET AL. 11 of 13
Figure 9. Latitudinal ME (top panel) and RMSE (bottom panel) errors of three prediction models on the testing set. Red,
blue and cyan curves represent evaluations metrics from convolutional long short-term memory, c1pg and persistence models,
respectively. Uncertainties (3 times standard error) for different metrics are included in each panel. The average ME and
RMSE values of each model across all latitude are shown in the legend.
Data Availability Statement

The centre for orbit determination in Europe global ionosphere maps data are available from the Crustal Dynam-
ics Data Information System (https://cddis.nasa.gov/) via registration. Solar and Geomagnetic data are obtained
from the OMNI Goddard Flight Center (https://omniweb.gsfc.nasa.gov/).
Acknowledgments
This project is sponsored by DARPA References
DSO Space Environment Exploitation
Program grant #DI 9AC00009. The Basu, S., Basu, S., MacKenzie, E., Coley, W. R., Sharber, J. R., & Hoegy, W. R. (1990). Plasma structuring by the gradient drift instability at high
authors acknowledge the SWAMI server latitudes and comparison with velocity shear driven processes. Journal of Geophysical Research, 95(A6), 7799–7818. https://doi.org/10.1029/
at the Space Weather Technology, JA095iA06p07799
Research and Education Center, Univer- Boulch, A., Cherrier, N., & Castaings, T. (2018). Ionospheric activity prediction using convolutional recurrent neural networks. arXiv preprint
sity of Colorado, Boulder for providing arXiv:1810.13273.
high-performance computing that contrib- Camporeale, E. (2019). The challenge of machine learning in space weather: Nowcasting and forecasting. Space Weather, 17(8), 1166–1207.
uted to this work. https://doi.org/10.1029/2018SW002061
LIU ET AL. 12 of 13
Cesaroni, C., Spogli, L., Aragon-Angel, A., Fiocca, M., Dear, V., De Franceschi, G., & Romano, V. (2020). Neural network based model for global
total electron content forecasting. Journal of Space Weather and Space Climate, 10, 11. https://doi.org/10.1051/swsc/2020013
Chen, P., Liu, H., Schmidt, M., Yao, Y., & Yao, W. (2021). Near real-time global ionospheric modeling based on an adaptive kalman filter state
error covariance matrix determination method. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–12. https://doi.org/10.1109/
TGRS.2021.3091705
Chen, Z., Liao, W., Li, H., Wang, J., Deng, X., & Hong, S. (2022). Prediction of global ionospheric TEC based on deep learning. Space Weather,
20(4), e2021SW002854. https://doi.org/10.1029/2021SW002854
Cowley, S. W. H. (2000). Magnetosphere-ionosphere interactions: A tutorial review. In Magnetospheric current systems geophysical monograph
series (Vol. 118, pp. 91–106). AGU. https://doi.org/10.1029/GM118p0091
Dietterich, T. G. (2002). Machine learning for sequential data: A review. In Joint IAPR international workshops on statistical techniques in
pattern recognition (SPR) and structural and syntactic pattern recognition (SSPR) (pp. 15–30). Springer.
García-Rigo, A., Monte, E., Hernández-Pajares, M., Juan, J. M., Sanz, J., Aragón-Angel, A., & Salazar, D. (2011). Global prediction of the verti-
cal total electron content of the ionosphere based on GPS data. Radio Science, 46(06), 1–3. https://doi.org/10.1029/2010RS004643
Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT press.
Hernandez-Pajares, M., Juan, J. M., Sanz, J., Orus, R., Garcia-Rigo, A., Feltens, J., et al. (2009). The IGS VTEC maps: A reliable source of
ionospheric information since 1998. Journal of Geodesy, 83(3–4), 263–275. https://doi.org/10.1007/s00190-008-0266-1
Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International
conference on machine learning (pp. 448–456). PMLR.
Jacobsen, K. S., & Andalsvik, Y. L. (2016). Overview of the 2015 St. Patrick's Day storm and its consequences for RTK and PPP positioning in
Norway. Journal of Space Weather and Space Climate, 6(A9), 1–12. https://doi.org/10.1051/swsc/2016004
Jakowski, N., Béniguel, Y., De Franceschi, G., Pajares, M. H., Jacobsen, K. S., Stanislawska, I., et al. (2012). Monitoring, tracking and fore-
casting ionospheric perturbations using GNSS techniques. Journal of Space Weather and Space Climate, 2(A22). https://doi.org/10.1051/
swsc/2012022
Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
Klobuchar, J. A. (1987). Ionospheric time-delay algorithm for single-frequency GPS users. IEEE Transactions on Aerospace and Electronic
Systems(3), 325–331. https://doi.org/10.1109/TAES.1987.310829
Lee, S., Ji, E. Y., Moon, Y. J., & Park, E. (2020). One-day forecasting of global TEC using a novel deep learning model. Space Weather, 19(1),
2020SW002600. https://doi.org/10.1029/2020SW002600
Li, Z., Yuan, Y., Wang, N., Hernandez-Pajares, M., & Huo, X. (2015). Shpts: Towards a new method for generating precise global ionospheric
TEC map based on spherical harmonic and generalized trigonometric series functions. Journal of Geodesy, 89(4), 331–345. https://doi.
org/10.1007/s00190-014-0778-9
Liu, L., Morton, Y. J., & Liu, Y. (2021). Machine learning prediction of storm-time high-latitude ionospheric irregularities from GNSS-derived
ROTI maps. Geophysical Research Letters, 48(20), e2021GL095561. https://doi.org/10.1029/2021GL095561
Liu, L., Zou, S., Yao, Y., & Aa, E. (2020). Multi-scale ionosphere responses to the May 2017 magnetic storm over the Asian sector. GPS Solu-
tions, 24(1), 1–15. https://doi.org/10.1007/s10291-019-0940-1
Liu, L., Zou, S., Yao, Y., & Wang, Z. (2020b). Forecasting global ionospheric TEC using deep learning approach. Space Weather, 18(11),
e2020SW002501. https://doi.org/10.1029/2020SW002501
Morton, Y. J., Yang, Z., Breitsch, B., Bourne, H., & Rino, C. (2020). Ionospheric effects, monitoring, and mitigation techniques. Position, navi-
gation, and timing technologies in the 21st century: Integrated satellite navigation. Sensor Systems, and Civil Applications, 1, 879–937. https://
doi.org/10.1002/9781119458449.ch31
Perez, R. O. (2019). Using TensorFlow-based neural network to estimate GNSS single frequency ionospheric delay (IONONet). Advances in
Space Research, 63(5), 1607–1618. https://doi.org/10.1016/j.asr.2018.11.011
Roma-Dollase, D., Hernández-Pajares, M., Krankowski, A., Kotulak, K., Ghoddousi-Fard, R., Yuan, Y., et al. (2018). Consistency of seven
different GNSS global ionospheric mapping techniques during one solar cycle. Journal of Geodesy, 92(6), 691–706. https://doi.org/10.1007/
s00190-017-1088-9
Schaer, S. (1999). Mapping and predicting the Earths ionosphere using the global positioning system. Ph.D. thesis, Ph.D. dissertation. University
of Bern,
Shi, X., Chen, Z., Wang, H., Yeung, D. Y., Wong, W. K., & Woo, W. C. (2015). Convolutional LSTM network: A machine learning approach for
precipitation nowcasting. arXiv preprint arXiv:1506.04214.
Shi, X., Gao, Z., Lausen, L., Wang, H., Yeung, D. Y., Wong, W. K., & Woo, W. C. (2017). Deep learning for precipitation nowcasting: A bench-
mark and a new model. arXiv preprint arXiv:1706.03458.
Wager, S., Wang, S., & Liang, P. S. (2013). Dropout training as adaptive regularization. Advances in Neural Information Processing Systems, 26,
351–359. https://doi.org/10.48550/arXiv.1307.1493
Wang, C., Xin, S., Liu, X., Shi, C., & Fan, L. (2018). Prediction of global ionospheric VTEC maps using an adaptive autoregressive model. Earth
Planets and Space, 70(1), 18. https://doi.org/10.1186/s40623-017-0762-8
Wang, Q., Ma, Y., Zhao, K., & Tian, Y. (2022). A comprehensive survey of loss functions in machine learning. Annals of Data Science, 9(2),
187–212. https://doi.org/10.1007/s40745-020-00253-5
Xu, B., Wang, N., Chen, T., & Li, M. (2015). Empirical evaluation of rectified activations in convolutional network. arXiv preprint
arXiv:1505.00853.
Yang, Z., Morton, Y. J., Zakharenkova, I., Cherniak, I., Song, S., & Li, W. (2020). Global view of ionospheric disturbance impacts on kinematic
GPS positioning solutions during the 2015 St. Patrick's Day storm. Journal of Geophysical Research: Space Physics, 125(7), e2019JA027681.
https://doi.org/10.1029/2019ja027681
Yao, Y., Liu, L., Kong, J., & Zhai, C. (2016). Analysis of the global ionospheric disturbances of the March 2015 great storm. Journal of Geophys-
ical Research: Space Physics, 121(12), 12157–12170. https://doi.org/10.1002/2016JA023352
Yao, Y., Liu, L., Kong, J., & Zhai, C. (2018). Global ionospheric modeling based on multi-GNSS, satellite altimetry, and Formosat-3/COSMIC
data. GPS Solutions, 22(4), 104. https://doi.org/10.1007/s10291-018-0770-6
Yeh, K. C., & Liu, C. H. (1982). Radio wave scintillations in the ionosphere. Proceedings of the IEEE, 70(4), 324–360. https://doi.org/10.1109/
PROC.1982.12313
LIU ET AL. 13 of 13

ML Prediction of Global Ionospheric TEC Maps

Uploaded by

Copyright:

Available Formats

ML Prediction of Global Ionospheric TEC Maps

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ML Prediction of Global Ionospheric TEC Maps

Uploaded by

Copyright:

Available Formats

15427390, 2022, 9, Downloaded from https://agupubs.onlinelibrary.wiley.com/doi/10.1029/2022SW003135 by National Medical Library The Director, Wiley Online Library on [18/10/2022].

2. Data and Methodology

2.2. ConvLSTM ML Algorithm

Residual prediction: The output

2.3. Evaluation Metrics

To quantify the prediction performance, the following metrics are utilized:

Network Module I O Filter size Stride Padding I O

3. Results and Discussions

Four convLSTM-based models with different combinations of loss functions

3.1. Qualitative Case Studies Using the Test Data Set

3.2. Statistical Evaluation Using the Test Data Set

Indicators Threshold Defined levels

Data Availability Statement

You might also like