Advancing Plasma Simulations Through LSTM Integration in The Particle-in-Cell Method
Advancing Plasma Simulations Through LSTM Integration in The Particle-in-Cell Method
Advancing Plasma Simulations Through LSTM Integration in The Particle-in-Cell Method
Abstract—In this study, we present a new approach for im- prohibitive due to the involvement of numerous particles.
proving the accuracy of plasma simulations by integrating Long To tackle this challenge, hybrid methods such as Particle-in-
Short-Term Memory (LSTM) deep learning with the Particle- Cell/Monte Carlo Collision (PIC/MCC) have been proposed
in-Cell (PIC) method. Our goal is to enhance the prediction
of the electric field in a 1D plasma system by leveraging the [5]. By combining kinetic and fluid approaches, PIC/MCC
LSTM model, which was trained using position and velocity enables the simulation of systems with heavy particles (e.g.,
data. We employed an iterative training process and performed ions) while treating electrons as a fluid. This hybridization
parameter tuning, focusing on key performance metrics such as approach enhances computational efficiency, making it feasible
Mean Squared Error (MSE) and Mean Absolute Error (MAE). to tackle complex systems within the constraints of computer
Through extensive experimentation, we observed a significant
reduction in various loss types, indicating improved prediction resources.
accuracy. However, it is crucial to note that the model’s perfor- Despite the numerous advantages of PIC/MCC methods
mance eventually reached a plateau, and we observed signs of in plasma simulation, they are not without limitations and
potential overfitting after a certain number of training epochs. disadvantages, particularly when applied to complex systems.
Our findings highlight the efficiency of integrating LSTM with One significant drawback is their computational expense,
the PIC method, enabling more precise and efficient predictions
of the electric field in plasma simulations. especially when dealing with large-scale systems that involve
Index Terms—Plasma simulations, Long Short-Term Memory a high number of particles [6]. Consequently, simulation
(LSTM) ,Particle-in-Cell, Deep Learning, Neural Networks times tend to be lengthy, and the computational costs can be
substantial. Another disadvantage of the PIC/MCC method is
I. I NTRODUCTION the challenge of conserving momentum and total energy in the
Computational methods have emerged as vital tools for simulations [7], [8]. This arises from the hybrid nature of the
simulating plasma phenomena across diverse fields of study. approach, where ions are treated as particles and electrons as a
With plasma constituting 99.9% of the visible universe [1], its fluid. Such treatment can introduce inaccuracies in simulating
unique state of matter holds great significance in understanding momentum and energy exchange [9], [10].
numerous natural phenomena affecting our planet, includ- Furthermore, the calculation of the electric potential by
ing lightning, the Earth’s ionosphere, and SolarWinds [2]. solving the Poisson equation on the grid and subsequently
Moreover, plasma serves as a fundamental component in the using it to determine the electric field is a computationally
development of advanced technologies like fusion machines, demanding process [4], [5]. This iterative procedure can be-
hall thrusters, and semiconductors [3]. Among the array of come a significant bottleneck in the simulation, contributing
computational methods available for plasma simulations, the to computational expenses.
Particle-in-Cell (PIC) method has gained prominence for its Moreover, the accuracy of the PIC/MCC method can be
ability to accurately model plasma dynamics in a variety of constrained by the resolution of the grid employed for solving
applications. the Poisson equation and calculating the electric field. Conse-
The PIC simulation method, initially proposed by Hockney quently, errors may arise in the simulation results, particularly
and Eastwood in 1981 [4], has gained widespread usage in in regions characterized by high electric fields.
the field of plasma dynamics. By employing super-particles to To address these challenges, researchers are actively en-
represent a large number of real particles, PIC enables accurate gaged in developing more efficient algorithms and methods
quantitative analysis of plasma behavior. This approach proves [11]. In recent years, there has been an increasing explo-
particularly valuable as it allows simulations to handle a ration of artificial intelligence (AI) techniques to enhance
significant number of particles effectively. the PIC method, enabling improved accuracy and facilitating
The utilization of a computational grid, referred to as the simulation of highly complex systems [12]–[14]. These
”cells,” facilitates the calculation of electric and magnetic advancements in AI present promising avenues for overcoming
fields, and their interpolation to particle positions. Never- the limitations of traditional PIC approaches and expanding
theless, the computational cost of simulations can become their capabilities in plasma simulations.
II. P RIOR RESEARCH suggested advancements in diagnostics and modeling through
PIC simulations and deep learning and machine learning
By integrating machine learning techniques with particle-in- techniques [3].
cell (PIC) codes, researchers have the potential to revolutionize These prior research works collectively highlight the
plasma physics research and enhance our understanding of progress and potential of machine learning and deep learning
complex plasma phenomena. techniques in enhancing plasma simulations, optimizing pa-
In a recent study conducted (J.R. Smith et al 2020) op- rameters, and advancing our understanding of plasma physics.
timized laser-based ion acceleration using an evolutionary
algorithm (EA) and one-dimensional PIC simulations. The III. M ECHANISM OF THE BASIC PARTICLE - IN -C ELL (PIC)
researchers identified an ”optimal” target density profile that S IMULATION
outperformed conventional choices, resulting in a 20% in- The Particle-in-Cell method utilizes the Vlasov equation
crease in maximum ion energy and improved conversion effi- to describe the evolution of the particle distribution function,
ciency. This statistics-driven approach showcased the potential enabling the simulation of plasma dynamics and interactions
for enhancing laser-plasma simulations and experiments in the in a self-consistent manner.
field of laser-based ion acceleration [15].
A. Vlasov-Maxwell Equations
In the same vein (Andreas Dopp et al 2023), the application
of machine learning in laser-plasma physics, specifically laser- The Particle-in-Cell (PIC) method utilizes the Vlasov equa-
plasma. The study emphasized the increasing abundance of tion to describe the evolution of the particle distribution
data and the need for precise control this complex phe- function, enabling the simulation of plasma dynamics and
nomenon. The focus was on integrating machine learning interactions in a self-consistent manner.
techniques to optimize simulations [16]. ∂f q
+ v · ∇x f + (E + v × B) · ∇v f = 0 (1)
In 2021, Aguilar and Markidis proposed a novel approach ∂t m
to plasma simulations by combining deep learning (DL) tech- f (x, v, t) represents the particle distribution function, q and
niques with the PIC method. Their work involved developing m are the charge and mass of a particle, x and v denote position
a DL-based PIC method that utilized a Multilayer Percep- and velocity vectors, E represents the electric field, and B
tron (MLP) and a Convolutional Neural Network (CNN) to represents the magnetic field. Maxwell’s equations describe
calculate the electric field from the electron phase space. the electromagnetic fields
The study demonstrated the effectiveness of the DL-based ρ
method in reproducing the expected growth rate of the two- ∇·E=
ε0
stream instability. Although the DL-based PIC method did ∇·B=0
not conserve total energy and momentum, it showed stability (2)
∂B
against the cold-beam instability, a common issue in traditional ∇×E=−
PIC methods [17]. ∂t
∂E
A study titled ”Machine Learning-Based Models for Ad- ∇ × B = µ0 J + µ0 ε0
∂t
vanced Physics Extensions in Particle-in-Cell Codes” (Chiara
Where ρ represents the charge density, ϵ0 is the vacuum
Badiali et al 2022). proposed a methodology for implementing
permittivity, µ0 is the vacuum permeability, and J denotes the
ML-based models in PIC codes, focusing on Monte Carlo or
current density.
statistical extensions. The approach enabled the development
of neural networks in Python, utilizing advanced ML tools for B. Initialization Phase
efficient training and testing. A proof-of-concept implementa- The PIC simulation begins with the initialization phase,
tion using the OSIRIS PIC code was showcased, where a fully- where the initial conditions for the plasma system are estab-
connected neural network replaced a section of a Compton lished. This involves setting up the initial particle distribution
scattering module. The ML-based method reproduced con- function f (x, v, t=0), charge and current densities, and
ventional results with comparable computational performance, electric field (E) and magnetic field (B) values consistent
indicating the potential for improved performance in physics with the specified plasma conditions and satisfying Maxwell’s
extensions within the PIC algorithm [18]. equations. Various techniques, such as uniform spatial distri-
A comprehensive paper titled ”Roadmap for Low- bution and Gaussian velocity distribution, can be employed to
Temperature Plasma Science and Technology” published in initialize the particle population.
2022 by I. Adamovich et al. provided an updated overview
of the field. The paper highlighted various areas of interest C. Computational Cycle
such as plasma-enabled additive manufacturing, soft materials, 1) Charge and Current Density Calculation: The charge
electrification of chemical conversions, plasma propulsion, density ρ and current densities (J) are computed on the grid
extreme plasma regimes, plasmas in hypersonics, and data- cells. In the NGP scheme, the charge density is calculated by
driven plasma science. Challenges in diagnostics and model- summing the contributions from particles within each cell
ing, particularly in extreme conditions characterized by low X
ρ(x) = V qi S(x − xi ) (3)
temperature and high density, were addressed. The authors i
Here, qi represents the charge of particle i, V is the cell IV. I NTEGRATING THE LSTM- BASED P REDICTION IN THE
volume, and S(x − xi ) is the interpolation function. PARTICLE - IN -C ELL M ETHOD
2) Particle Movement: The particle mover or pusher stage
involves updating particle positions (xi ) and velocities (vi ) by traditional Particle-in-Cell (PIC) simulations involve com-
solving the Newton equation. A leap-frog scheme is commonly putationally intensive calculations to determine the electric
used in one dimension field at each time step based on particle positions and ve-
locities. To address this issue several deep learning models,
1 including CNNs, MLPs, and RNNs, can be integrated into
xi (t + ∆t) = xi (t) + vi (t + )∆t (4) the design of PIC simulations. However, while CNNs excel
2
at image recognition and MLPs are suitable for regressions,
1 1 mi qi RNNs are well-suited for handling sequential data, which
vi (t + ) = vi (t − ) + E(xi , t)∆t (5) aligns with the nature of PIC simulations. Nonetheless, RNNs
2 2 ∆t
face a significant challenge known as the vanishing gradient
3) Field Solving: The field solver stage involves solving problem.
Maxwell’s equations. In the electrostatic limit, the Poisson This issue arises due to the activation function and the chain
equation is solved for the electric potential ϕ using finite rule of differentiation employed during gradient computation.
difference methods As gradients are backpropagated through the network layers,
they can exponentially diminish, impeding the effective learn-
∇2 Φ(x) = −ϵ0 ρ(x) (6) ing and weight updates in earlier layers. Consequently, cap-
turing long-term dependencies and temporal patterns becomes
Once ϕ is known, the electric field E is obtained by taking difficult, particularly in the context of the iterative nature of
the gradient PIC simulations. These limitations in gradient propagation can
lead to cumulative errors in the phase-space calculation as
E(x) = −∇Φ(x) (7)
well as the total energy and momentum of the simulation One
common approach is the use of gated recurrent units (GRUs) or
D. Iteration and Time Advancement
long short-term memory (LSTM) units within the RNN archi-
The PIC simulation progresses through successive iterations, tecture. These mechanisms incorporate specialized gates that
advancing the system in time. Each iteration involves updating regulate the flow of information and gradients, enabling the
particle positions, calculating fields, and updating the charge network to better retain and propagate information over longer
density. The simulation continues until the desired time or sequences. By selectively updating and forgetting information,
condition is reached GRUs and LSTMs alleviate the vanishing gradient problem
and facilitate the learning of complex temporal dependencies
By leveraging LSTM-based prediction, the time-consuming
field calculation step can be replaced with a trained model
that quickly estimates the electric field. This reduces compu-
tational overhead, allowing for faster simulations and enabling
the study of larger-scale plasma systems. The LSTM-based
predictions enable faster simulations, Capturing Nonlinear and
Transient Effects and improve accuracy.
D. Data Augmentation
To increase the amount of data, we repeated each combi-
nation 10 times which results in slight changes to the original
copy, this helps in amplifying the dataset which translate to
better performance by our model.
E. Profiling Setup
We run the LSTM model on a Linux-based system with
intel core i7 8700k, 2 NVIDIA Titan XP 12GB, and a 64GB
of RAM. With a batch size of 64 and a bfloat16 precision in
PyTorch.
Fig. 6: STM
Fig. 8
Fig. 7
Fig. 5: STM
VI. CONCLUSION
In this study, we successfully integrated an LSTM model
with a Particle-in-Cell (PIC) simulation to improve the ac-
curacy and efficiency of plasma physics simulations. Specifi-
cally, we focused on the two-stream interaction, a fundamen-
tal phenomenon in plasma physics. By constructing a PIC
code and generating the phase space data for each particle,
we trained an LSTM model to predict the electric field,
replacing the conventional method within the PIC frame-
work. The LSTM model demonstrated excellent capability
in predicting the electric field, benefiting from its internal
architecture designed to capture temporal dependencies in
sequential data. By leveraging the iterative nature of the
Fig. 14: a PIC simulation and the inherent ability of the LSTM model
to learn from time-series data, we achieved accurate pre-
dictions that closely aligned with the results obtained from
the original PIC method. Moreover, the LSTM-PIC approach
significantly reduced computational time compared to tradi-
tional PIC simulations. Compared to other studies that utilize
CNN models, our LSTM-PIC approach primarily relies on
numerical data, eliminating the need for combining images
with data for training. The LSTM-PIC model provided results
that closely matched the trends and values obtained from the
original PIC, while significantly reducing computational time.
However, the model’s predictability was affected in certain
scenarios, particularly when using data beyond the training
range or during unstable regimes. These limitations can be
attributed to insufficient training coverage and complexity of
Fig. 15: b the data. Future enhancements for the LSTM-PIC system
could involve incorporating additional layers that integrate
Fig. 16: Grid of Images physically informed models, such as incorporating total energy
or momentum into the loss function. This would address
To assess the accuracy of the LSTM-PIC model from a issues that may arise during the simulation and ensure ac-
physical perspective, we evaluated the conservation of energy curate predictions. Additionally, exploring the coupling of the
and mass. In the context of a 1D simulation of two streams of LSTM-PIC model with other techniques, such as Generative
electrons, we examined the conservation equations governing Adversarial Networks (GANs), could enable automation and
energy and mass *** improve the system’s performance using phase-space images.
Further studies should also be conducted to investigate more
∂ρ complex systems involving magnetic fields, as well as extend
+ ∇ · (ρv) = 0 (14)
∂t the methodology to two and three-dimensional simulations
By comparing these conservation equations between the with diverse shapes and geometries. These advancements will
LSTM-PIC model and the conventional PIC simulation, we enhance our understanding and modeling capabilities, paving
the way for improved simulations and accurate predictions in
a wide range of plasma phenomena.
ACKNOWLEDGMENT
R EFERENCES
R EFERENCES
[1] N. Cosmology, B. Bang, and E. P. Universe, ”Electric Plasma Universe
Arrives,” September 2015.
[2] B. Leybourne and G. Gregori, ”Introduction to plasma tectonics &
electric geology: Solar wind coupling to planetary circuits lightning tells
the stellar transformer story,” IMCIC 2020 - 11th Int. Multi-Conference
Complexity, Informatics Cybern. Proc., vol. 2, June 2021, pp. 48-54.
[3] I. Adamovich et al., ”The 2022 Plasma Roadmap: low temperature
plasma science and technology,” J. Phys. D. Appl. Phys., vol. 55, no.
37, 2022, doi: 10.1088/1361-6463/ac5e1c.
[4] J. W. Hockney and R. W. Eastwood, ”Computer simulation using
particles,” Taylor & Francis, Inc., 1988.
[5] C. K. Birdsall and A. B. Langdon, ”Plasma physics via computer
simulation,” 2004.
[6] H. Matsumoto and T. Sato, ”Computer simulation of space plasmas,”
Comput. Simul. Sp. Plasmas, 1985.
[7] J. U. Brackbill, ”On energy and momentum conservation in particle-in-
cell plasma simulation,” J. Comput. Phys., vol. 317, pp. 405-427, 2016,
doi: 10.1016/j.jcp.2016.04.050.
[8] J. A. Heikkinen et al., ”Interpolation for momentum conservation
in 3D toroidal gyrokinetic particle simulation of plasmas,” Comput.
Phys. Commun., vol. 183, no. 8, pp. 1719-1727, Aug. 2012, doi:
10.1016/J.CPC.2012.03.013.
[9] C. Liu et al., ”Hybrid simulation of energetic particles interacting
with magnetohydrodynamics using a slow manifold algorithm and GPU
acceleration,” Comput. Phys. Commun., vol. 275, p. 108313, 2022, doi:
10.1016/j.cpc.2022.108313.
[10] S. Markidis and G. Lapenta, ”The energy conserving particle-in-cell
method,” J. Comput. Phys., vol. 230, no. 18, pp. 7037-7052, Aug. 2011,
doi: 10.1016/J.JCP.2011.05.033.
[11] J. Teunissen and U. Ebert, ”Simulating streamer discharges in 3D with
the parallel adaptive Afivo framework,” J. Phys. D. Appl. Phys., vol. 50,
no. 47, p. aa8faf, 2017, doi: 10.1088/1361-6463/aa8faf.
[12] M. Mörtl et al., ”Enhancing Hall Effect Thruster Simulations with
Deep Recurrent Networks,” September 2019, [Online]. Available:
http://www.netlib.org/lapack/.
[13] K. Ch
oudhary et al., ”Recent advances and applications of deep learning
methods in materials science,” doi: 10.1038/s41524-022-00734-6.
[14] K. L. van de Plassche et al., ”Fast modeling of turbulent transport in
fusion plasmas using neural networks,” Phys. Plasmas, vol. 27, no. 2,
p. 22310, Feb. 2020, doi: 10.1063/1.5134126.
[15] J. R. Smith, C. Orban, J. T. Morrison, and K. M. George, ”Optimiz-
ing laser–plasma interactions for ion acceleration using particle-in-cell
simulations and evolutionary algorithms,” 2020.
[16] D. Andreas, C. Eberle, S. Howard, F. Irshad, J. Lin, and M. Streeter,
”Data-driven Science and Machine Learning Methods in Laser-Plasma
Physics,” 2023.
[17] X. Aguilar and S. Markidis, ”A Deep Learning-Based Particle-in-Cell
Method for Plasma Simulations,” in Proc. - IEEE Int. Conf. Clust.
Comput. ICCC, vol. 2021-Septe, pp. 692–697, 2021.
[18] C. Badiali, P. J. Bilbao, F. Cruz, and L. O. Silva, ”Machine-learning-
based models in particle-in-cell codes for advanced physics extensions,”
J. Plasma Phys., vol. 88, no. 6, 2022.