Deep Learning For Procedural Content Generation

Noname manuscript No.
(will be inserted by the editor)
Deep Learning for Procedural Content Generation

Jialin Liu1 · Sam Snodgrass2 · Ahmed Khalifa3 · Sebastian Risi2,4 ·
Georgios N. Yannakakis2,5,6 · Julian Togelius2,3
arXiv:2010.04548v1 [cs.AI] 9 Oct 2020
Received: date / Accepted: date
Abstract Procedural content generation in video applied to generate game content directly or indirectly,
games has a long history. Existing procedural content discusses deep learning methods that could be used for
generation methods, such as search-based, solver-based, content generation purposes but are rarely used today,
rule-based and grammar-based methods have been ap- and envisages some limitations and potential future di-
plied to various content types such as levels, maps, char- rections of deep learning for procedural content gener-
acter models, and textures. A research field centered on ation.
content generation in games has existed for more than
Keywords Procedural content generation · Game
a decade. More recently, deep learning has powered a
design · Deep learning · Machine learning · Computa-
remarkable range of inventions in content production,
tional and artificial intelligence
which are applicable to games. While some cutting-edge
deep learning methods are applied on their own, oth-
ers are applied in combination with more traditional 1 Introduction
methods, or in an interactive setting. This article sur-
veys the various deep learning methods that have been Deep learning has powered a remarkable range of in-
ventions in content production in recent years, includ-
J. Liu ing new methods for generating audio, images, 3D ob-
E-mail: [email protected] jects, network layouts, and other content types across
S. Snodgrass a range of domains. It stands to reason that many of
E-mail: [email protected] these inventions would be applicable to games. In par-
A. Khalifa ticular, modern video games require large quantities of
E-mail: [email protected] high-definition media, which could potentially be gen-
S. Risi erated through deep learning approaches. For example,
E-mail: [email protected] promising recent methods for generating photo-realistic
G. N. Yannakakis faces could be used for character creation in games.
E-mail: [email protected] At the same time, video games have a long tradition
J. Togelius of procedural content generation (PCG) [132], where
E-mail: [email protected] some forms of game content have been generated algo-
1
rithmically for a long time; the history of digital PCG
Guangdong Provincial Key Laboratory of Brain-inspired
Intelligent Computation, Department of Computer Science
in games stretches back four decades. In the last decade
and Engineering, Southern University of Science and Tech- and a half, we have additionally seen a research commu-
nology, Shenzhen, China nity spring up around challenges posed by game content
2
Modl.ai, Copenhagen, Denmark generation [16, 93, 112, 129, 133, 134, 148]. This re-
3
New York University, New York, USA
4
IT University of Copenhagen, Copenhagen, Denmark
search community has applied methods from core com-
5
Institute of Digital Games, University of Malta, Msida, puter science, such as grammar expansion [22]; AI, such
Malta as constraint solving [115] and evolutionary computa-
6
Technical University of Crete, Chania, Greece tion [7, 133]; and graphics, such as fractal noise [24].
2 J. Liu, S. Snodgrass, A. Khalifa, S. Risi, G. N. Yannakakis and J. Togelius
But only in the last few years have we seen a real ef- Here, we delineate the scope of our article by compar-
fort to bring the tools of deep learning to game content ing it to existing books and surveys in Section 2.1 and
generation. Section 2.2. Section 2.3 describes our paper selection
Deep learning brings new opportunities and leads methodology.
to exciting advances in PCG, such as generative ad-
versarial networks (GANs) [32], deep variational au-
toencoders (VAEs) [63] and long short-term memory
(LSTM) [34, 45]. However, those methods for other gen-
erative or creative purposes are not always applicable
to games and need certain adaptations due to the func-
tionality criteria of different game content. Methods for 2.1 Related Work
generating images (e.g., generative networks) can be
used to generate image-like game content (e.g., level
A number of books and surveys of PCG with differ-
maps, landscapes, and sprites). However, the generated
ent focuses and aims have been published in the past
levels should be playable and require specific gameplay
two decade [16, 93, 112, 129, 133, 134, 148]. The two
skill-depth. The generated sprites should imply spe-
textbooks for PCG [112] and Game AI [148] cover the
cific character or emotion, as well as coherence within
search-based methods, solver-based methods, construc-
the game. Training reliable models requires a necessary
tive generation methods (such as cellular automata and
amount and quality of data, while the available data of
grammar-based methods), fractals, noise, and ad-hoc
content and playing experience for most games is lim-
methods for generating diverse game content. De Kegel
ited. Careful consideration and sophisticated design of
and Haahr [16] reviewed the PCG methods for eleven
adaptation techniques are requisites for applying deep
categories of puzzles, but few work based on deep learn-
learning methods to generate game content.
ing has been reported. The article by Togelius et al.
It is important to note that content generation has reviews the search-based PCG methods, defined as us-
uses outside of designing and developing games for hu- ing meta-heuristics to search in a predefined content
mans to experience. In addition to creating content in space, not necessarily represented by the same format
games meant for humans to play, content generation of the content itself, and automatically generate new
can also play a crucial role in creating generalizable content [133]. The search is led by a fitness or eval-
game-based and game-like benchmarks for reinforce- uation function which measures the quality or playa-
ment learning and other forms of AI [26, 136]. bility of the generated content. The experience-driven
This article surveys the various approaches that PCG framework [147] largely adopts a search-based ap-
have been taken to generate game content with deep proach and reviews ways in which algorithms can gen-
learning, and also discusses methods proposed from erate content for adjusting the player experience. Most
within deep learning research that could be used for of the reviewed search-based methods in both survey
PCG purposes. First, we give an overview of types of papers rely on evolutionary algorithms. In this article,
game content that could conceivably be generated by we also cover some search-based methods which coop-
deep learning, including the particular constraints and erated with deep learning methods for generating con-
affordances of each content type and examples of such tent. The most famous example may be latent variable
applications (if they exist), followed by an overview of evolution [5]. Risi and Togelius [93] focuses on PCG
applicable deep learning methods. for applications in Reinforcement Learning (RL), while
the work based on RL methods reviewed in this arti-
cle mainly used RL agents to play the generated levels,
2 Scope of The Review which indirectly served as content evaluators. Khalifa
et al. [62] models the level generation as an iterative
This article discusses the use of deep learning (DL) process that one needs to edit the levels to meet cer-
methods, here defined as neural networks with at least tain requirements or achieve some specific goals. RL
two layers and some nonlinearity [33], for game con- agents need to learn to generate levels through this it-
tent generation. We take an inclusive view of games as erative process. The study of Summerville et al. [129],
any games a human would conceivably play, including published in 2018, reviews the PCG via Machine Learn-
board games, card games, and any type of video games, ing (PCGML) methods, building on e.g. Markov chains
such as arcade games, role-playing games, first-person (e.g., [118, 119, 120, 131, 152]), n-grams (e.g., [14]), and
shooters, puzzle games, and many others. Several other Bayes nets (e.g.,[37]), whereas we will focus exclusively
surveys and overviews of PCG in games already exist. on deep learning in this article.
Deep Learning for Procedural Content Generation 3
2.2 Novelty of The Review 3 Content Types
The differences between the current article and the Generally, game content can be distinguished from the
PCGML survey [129] is that (i) our article focuses on content meant for non-interactive media by various
DL-based methods, defined at the beginning of Section forms of functionality constraints. Video, images, and
2 (although other techniques will be mentioned for con- music all require coherence, and in general that aes-
trast); (ii) our article surveys more types of game con- thetic suffers when the coherence fails. For example,
tent, such as narrative text and graphical textures; (iii) GANs can often create images that are locally convinc-
we also discuss applications of deep learning to support ing but globally incoherent, such as a side-view of a car
PCG, such as for content quality prediction; and (iv) where the front wheels have a different size and style
our survey is written more than three years after the to the back wheels. This may be annoying to the hu-
PCGML survey was first submitted and two years af- man viewer, but the image still unmistakably depicts
ter it was published, during which time an avalanche of a car; it doesn’t turn into a blur of random pixels just
new work in the field has appeared. because the wheels on the car don’t match. In contrast,
when generating a game level, if the final door has no
During the two years after the publication of [129],
matching key the level is unplayable; the level’s utility
PCG via deep learning has been growing quickly and
as content is not just slightly diminished, but essentially
a significant number of papers and articles have been
zero (unless manually repaired). Making a neural net-
published. The trend was mainly set by latent variable
work learn to produce only functional content is often
evolution [5] in 2018. A review of the state-of-the-art
a tall task, and is one of the core challenges of using
and the latest applications of deep learning to PCG is
deep learning for PCG. Not all types of game content
needed.
have the same extent of functional constraints however,
and some offer affordances that may make content gen-
eration relatively easier. Also, not all content is nec-
essary; depending on the game’s design, there might
be artifacts that are allowed to be broken, as the user
2.3 Paper Collection Methodology
can simply discard them and select others. Weapons in
Borderlands are a good example of optional content.
To collect the related papers published or online since
2018, till end of August 2020, we have searched with
Google Scholar and Web of Science using the following 3.1 Game Levels
search terms ( “game”) AND (“design”) and (“game”)
AND (“procedural content generation” OR “pcg”), sep- The most common type of content to generate in games
arately. We systematically went through the returned is levels. These are spaces in two or three dimensions
papers, most of which were publications in the IEEE that need to be traversed. Typically, these are necessary
Transactions on Computational Intelligence and AI in rather than optional, and have strong functional con-
Games (T-CIAIG), the IEEE Transactions on Games straints that require them to be playable. For example,
(ToG), in the proceedings of the IEEE Conference on there can not be impassable geometry (such as gaps or
Computational Intelligence and Games (CIG) series, walls) blocking traversal of the level, items needed to
the IEEE Conference on Games (CoG) series, the In- finish the levels must be present, and enemies cannot
ternational Conference on the Foundations of Digital be unbeatable. 2D, side-scrolling platform games is a
Games (FDG) series, the Artificial Intelligence for In- genre where procedural generation is particularly com-
teractive Digital Entertainment (AIIDE) Conference mon, both in entertainment-focused games (in particu-
series and their related workshops, as well as special ses- lar indie games) and in academic research. Among the
sions at other conferences, such as the IEEE Congress former, the standout game Spelunky has defined a way
on Evolutionary Computation (IEEE CEC). We also of building 2D platform games around PCG; among the
went through the papers that have been recently ac- latter, the Mario AI Framework [135], built around an
cepted in 2020 by the conferences mentioned above. open-source clone of Super Mario Bros, has been used
Only work that involve direct or indirect use of DL- in so many research projects that it could be called the
based methods for generating game content or evaluat- “drosophila of PCG research”. Another type of com-
ing content or content generators are reviewed in this monly attempted 2D level is the rogue-like or dungeon-
article, while the ones being returned due to citations crawler level, where the objectives and constraints are
with the search terms but are out of our scope are not similar to the platform game level, but which are viewed
included. from the top down so physics works differently. Related
to this are levels for first-person shooters. Another kind to be animatable, so that they can produce believable
of 2D level is the battle map, used in strategy games movements or facial expressions.
such as StarCraft or player-versus-player modes of first-
person shooters. While such maps also have “hard” con-
straints, such as sufficient room for the players’ bases, 3.4 Textures
there are also the softer constraints of balancing; many
features contribute to the quality of battle maps, but Textures are used in almost all 3D games, and is per-
balancing is paramount. haps the type of content that has the fewest function-
Levels for music games, such as as Guitar Hero or ality constraints. Procedural methods such as Perlin
Dance Dance revolution, can be seen as 2D levels as Noise [24, 89] have been used for texture generation
well. Here the player is automatically moved along the in games since the birth of commercial 3D games with
level, and has to carry out certain actions in time with DOOM. Deep learning methods for texture generation
the music, as prompted by level features. Some inter- could provide a viable alternative in this case.
esting work has been done on learning to create such
music game levels from existing music (e.g., [21, 139]).
3.5 Music and Sound
3.2 Text Most games feature a soundtrack, often composed of

both music and sound effects. The constraints on the
Almost all games include some form of text, and typi- soundtrack tend to be relatively soft compared to other
cally they use text to convey narrative. This text typ- types of content constraints; the sound effects should
ically has very strong constraints, as it needs to be be appropriate to the actions in the game at any given
truthful with regards to what happens in the game. For moment, and the music to the emotional tone of the
example, if the text says that the King lives in Stock- moment, but inappropriate sound does not necessarily
holm, this must actually be the case lest it misleads break the game. Quite a few games involve some kind
the player. Traditionally, generative text in games has of procedural soundtrack, and some research projects
not been very ambitious and used simple text substitu- have focused on music generation able to adapt to af-
tion or grammar-based approaches. Outside of games, fective shifts in real-time [106]. At the same time, deep
deep learning has made great strides with LSTM net- learning has made impressive strides in learning to gen-
works [34, 45] and, more recently, transformers able to erate music with some modes of controllability [18], but
generate coherent and stylistically relevant text. How- we have yet to see the use of deep learning methods for
ever, these methods are not easy to integrate into most sound generation in games.
games because of the lack of control over deep learning-
based text generators. However, games such as AI Dun-
geon 2 have managed to build gameplay on top of al- 4 Training Methods and Neural Architectures
most uncontrollable text generation. of DLPCG
Due to the different types and roles of content in games,

3.3 Character Models diverse deep learning methods have been adapted for
PCG. In this section we present different ways to apply
Faces and character models are examples where deep deep learning for PCG systems, the target content, and
learning has advanced content creation capabilities rad- their generality. The approaches are categorized by the
ically in recent years, but these methods have generally type of machine learning method used for training. Ad-
not made their way into games. Datasets of thousands ditionally, works combining evolutionary computation
of real human faces, such as the Celeb-A dataset [75], techniques to deep learning methods are also presented.
have become standard benchmark for developing new The works reviewed in this section are summarized in
GAN variations, leading to some impressive break- Fig. 1, categorized by the content types and deep learn-
throughs in face generation. While many games have ing methods.
a need for (human) faces in various roles, including for Generating different types of content often requires
freshly generated NPCs, the character design feature different types of neural architectures. In the use cases
of role-playing games is a standout application case for reviewed in Section 4.1 and Section 4.2, LSTMs are
controllable PCG, where machine learning-based meth- mostly used for time-dependent sequential data (e.g.,
ods have yet to make their mark. Depending on the action sequences, agent paths, charts for rhythm) and
features of the game, these faces or models might need language models, while convolutional neural networks
2D Map/Level 3D Map/Level Rhythm

Narrative Texture Music [107] [48] [55]
20 Face & Character Card & Deck Other
[25]
[151] [145] [31] [85] [143]

Number of papers
15 [130] [50]
[83] [69]
[27] [50]
[142],[2] [59] [71]
[21]
10
[38] [150] [20] [29] [3] [65]
[64] [113] [4] [20]
[114] [23]
[126] [69] [54] [96] [36] [84] [141] [77] [52] [103] [104]
[10]
[139] [67] [15] [99] [121] [103] [104] [137] [140] [84] [36]
5 [100] [97] [146] [49] [94] [44]
[58] [40] [39] [62] [41] [23]
[127] [57] [4] [17] [84]
0
SL USL RL AL EC
Fig. 1: This figure shows the distribution of research by methods and content types. We notice the disproportion-
ately large amount of work on 2D level and map generation compared to all other content types.
are often used for any type of image-like content. A very labels as input to generate new instances of those design
popular class of architecture for content generation are patterns.
GANs [32]. A GAN consists of two networks, a gen- Karavolos et al. [57] trained a CNN to predict the
erator and a discriminator that are trained iteratively outcomes of a simplified 3 versus 3 multiplayer death-
to allow the generator to create more realistic content, match shooter game to evaluate and determine if the
while the discriminator is getting better at distinguish levels, represented by maps and weapon parameters, are
generated content from real data. balanced or favor a team. Based on the outcome pre-
dictor from [57], Karavolos et al. [58] further designed
a DL surrogate model for pairing levels and character
classes for desired game outcomes.
4.1 Supervised Learning
Tsujino and Yamanishi [139] represented rhythm-
Supervised learning (SL) methods have been used in a based video game levels by charts and implemented
variety of ways for content generation. Often as a pre- Dance Dance Gradation (DDG), a system with LSTMs
dictor, SL models predict the gameplay outcomes of trained on levels of different degrees of difficulty to gen-
games with the generated content, either for evaluating erate new levels. DDG can tune the difficulty degree of
the quality of content, or for meeting specific prefer- generated charts by changing the fractions of easy or
ences (such as game style, image style and color) or hard charts used to compose the training dataset [139].
adapting the generated levels to desired skill-depth. Liang et al. [67] used C-BLSTM [105] to generate lev-
The study of Summerville et al. [127] extracted els of rhythm games, represented by actions and corre-
player paths in Mario from gameplay videos and sponding timing, of different difficulties, trained on the
used them to annotate training levels. Then, separate beatmaps collected from OSU!, a famous rhythm game.
LSTMs are trained on levels annotated with different Beside considering skill-depth required in game lev-
players’ paths in order to generate personalized levels els, the emotion sent by content has also been studied.
based on the players’ chosen paths [127]. Then, Guzdial Guzdial et al. [38] studied the emotion shown by the
et al. [40] trained a random forest on expert-labeled game visuals, such as abstract texture, color of game
design patterns from Mario levels (i.e., small sections maps and scene, including the visual effects, and trained
of levels given descriptive class labels) to classify level a CNN to generate textures for some given target emo-
structures and an autoencoder with level structures and tion.
Soares and Bulitko [123] trained a VAE [63] to clas- and later be able to generate level segments that follow
sify NPC behaviors to Leaders, Followers, and Random, a certain distribution/style. Davoodi et al. [15] trained
in a simple artificial life environment. Sirota et al. [114] an autoencoder to repair manually designed levels for
trained two RNNs, a speaker and a listener, by playing different games by re-iterating it over the decoder while
a referential game with concepts and human-generated using a trained discriminator from a GAN model to
annotations to design communication systems for NPCs determine the stopping criteria. Besides levels, autoen-
in games. coder has also been used to generate 3D shapes [151].
Moreover, USL methods for image generation have
4.2 Standard Unsupervised Learning also been applied to generating sprites and characters
in games. The recent work by Mordvintsev et al. [83]
Most unsupervised learning (USL) techniques in PCG learned cellular automata (CA) to imitate the develop-
focus on learning a representation of all the content and ment of organism and generate images, represented by
then sample new content from this representation. For 2D grids of cells. A cell is similar to the tile considered
example, using autoencoders to learn to replicate game in the MarioGan [140] (explained later in section 4.5).
levels. Another direction usually taken is transforming A cell contains a cell state (e.g., a discrete value or a
the data into a sequence and use unsupervised learning vector of RGB values), while a tile contains a discrete
to learn the relation between these elements similar to value which refers to an object type or part of it.
Markov Chains relations. For example, learning from a
Applications of USL methods to content generation
text corpus how to predict the next word based on the
for card games and text adventure games have also been
previous ones.
investigated. An example is [130]. Summerville and
Summerville and Mateas [126] trained LSTMs on
Mateas [130] trained encoding and decoding LSTMs
Mario levels annotated with agent paths by represent-
on Magic: The Gathering cards, represented as se-
ing the 2D levels as one dimensional strings of tiles.
quences of tokens corresponding to the important in-
Jain et al. [54] trained autoencoders on sliding-window
formation on the cards (e.g., mana cost, effect, power,
segments of Super Mario Bros levels, which were rep-
etc.). The LSTMs were trained on corrupted versions
resented by 2D arrays, to generate and repair levels.
of the cards, and encoded cards were used as input
Jain et al. [54] considered a tile as being empty or
to the decoder at generation time. Another example is
occupied, but has inspired many follow-up investiga-
the endless text adventure game AI Dungeon 2 1 (ear-
tions. Blending has lead to new and creative game lev-
lier version as AI Dungeon). AI Dungeon 2 is built on
els. Sarkar and Cooper [96] trained separate LSTMs
OpenAI’s GPT-2 model [92], a 1.5B parameter Trans-
on two different game domains (Mario and Kid Icarus),
former, and fine-tuned on some text adventures ob-
and generated new blended level sections with alternat-
tained from chooseyourstory.com, according to its
ing generators. Sarkar et al. [99] further explored gen-
developer Nick Walton [142]. In a game, a player can
erating blended levels by training variational autoen-
interact with the game by inputting text commands,
coders and GANs on Mario and Kid Icarus, and gen-
then the AI dungeon master will generate content of
erating new blended level sections that interpolate be-
the game environment (updates in the game story) ac-
tween the domains using the latent vectors. Snodgrass
cording to the commands and provide text feedback.
and Sarkar [121] also used VAEs to model and generate
By doing so, each player can build his/her own unique
platformer level structures which was finished by using
game story. Ammanabrolu et al. [2] focused on procedu-
a search-based approach to blend details from several
rally generating interactive fiction worlds and proposed
other games. Sarkar et al. [100] explored two variants of
AskBERT to construct knowledge graph. AskBERT ex-
VAEs (linear are GRU) for blending platforming game
tracts objective information in the game worlds, such as
levels and associated paths in those levels. Sarkar and
characters and objects, via question-answering model.
Cooper [97] trained VAEs to learn a sequential model
Ferreira et al. [27] proposed Bardo Composer, a sys-
of level segment generation and a random forest clas-
tem that automatically composes music for tabletop
sifier to determine the exact location of a newly gen-
role-playing games. In Bardo Composer, a BERT model
erated segment to the previous segment (an ancestor).
cooperates with a stochastic bi-objective beam search
The resulted levels are not only more coherent [97], but
model to identify music emotion, and then generate mu-
also more creative [98] because of the changing altitude
sic pieces that reflects the identified emotion.
of platformer and various possible heading directions.
Yang et al. [146] trained Gaussian Mixture VAE to
learn relation between game level segments from various
1
games (Super Mario Bros, Kid Icarus, and Megaman) https://github.com/AIDungeon/AIDungeon
tain goals (based on the current generation problem).

They proposed 3 main transformation: Narrow, Turtle,
and Wide. These transformation focus on the different
ways that the generator controls where it is modifying.
Fig. 2 shows examples of the generated levels over three
different problems using trained agents in the PCGRL
framework.
4.4 Adversarial Learning
Adversarial learning (AL) models are perfect for gener-

ating content represented by pixel-based images or 2D
(a) Binary (b) Zelda (c) Sokoban array of tiles, such as levels as a map, landscapes and
sprites. The most popular model among the reviewed
Fig. 2: Generated examples from three different prob- works would be GAN [32] and its variants.
lems using PCGRL envrionment introduced by Khalifa 2D levels of most arcade games can be simplified as
et al. [62]. 2D arrays of tiles, where each tile contains a type of
object or part of an object. Examples include the lev-
els designed using Video Game Description Language
4.3 Reinforcement Learning
(VGDL) [101] in the General Video Game AI plat-
Using reinforcement learning (RL) for PCG is a very re- form [87, 88], and the tile-based levels in the Mario
cent proposition which is just beginning to be explored. AI framework [110]. As shown in the top-left of Fig.
Here, the generation task is transformed into a Markov 4, each tile contains a type of object or part of it,
Decision Process (MDP), where a model is trained to such as ground, pipe, empty and enemy, represented
iteratively select the action that would maximize ex- either by a symbol or an integer. Kuang and Luo [64]
pected future content quality. This transformation is implemented an interactive map designing system us-
not an easy task and there is no standard way of han- ing different generative models to generate 2D maps,
dling it. which can be further extended to 3D scenes. Torrado
One of the early projects that uses RL is by Chen et al. [137] designed a new GAN architecture, Condi-
et al. [10]. They used a small network of one hidden tional Embedding Self-Attention GAN (CESAGAN),
layer to generate a hearthstone deck of cards that can to tackle the low quality and diversity issue of gener-
beat a specific other deck given a certain player. The ated 2D levels by traditional GANs, and increased the
agent can modify the current deck by substituting any amount of training data to CESAGAN with a boot-
of its cards with a different one. The goal is to maximize strapping technique. They applied their technique to
the win rate of the playing agent using the current deck Zelda, a dungeon crawler game from GVGAI [87].
against a predefined deck. To facilitate the input form for generative mod-
Earle [23] used RL to play the game of SimCity els, such as GANs, 3D landscapes are often converted
(Maxis, 1989). They used a fractal network (convo- to 2D height map. Wulff-Jensen et al. [145] trained
lutional network with structured skip connections) as a deep convolutional GAN (DCGAN) on digital ele-
their network architecture and optimized it towards in- vation maps sampled from the Alps dataset to gen-
creasing the city population. At each step, the agent can erate 2D height maps as input to Unity for creating
change any space on the map to any other type. This 3D landscapes for video games. Giacomello et al. [31]
project is a borderline example of PCG. The aim of converted each 3D DOOM level to several 2D images,
the project was to play the game of SimCity where the among which a HeightMap was used to indicate the
trained agent will learn to be a city planner/generator. 3D information and other were top-down images of the
corresponding level. In [31], two GANs were trained on
As we can see, most of the RL PCG requires an
human-designed levels, one of which took plain 2D im-
adaptation for the input to be able to be used dur-
ages as input and the other used both the images and
ing generation. Khalifa et al. [62] introduced a frame-
some of the extracted features. Park et al. [85] trained
work2 for 2D level generation using RL. The genera-
a multistep DCGAN, adapted from [140], to generate
tion process is framed as an iterative process where at
levels of an educational game, ENGAGE. The levels
every step the generator modifies the level toward cer-
were represented by a 2D array of tiles, from a top-
2
https://github.com/amidos2006/gym-pcgrl down view, during training and creation, and then con-
verted to 3D levels to be used in the game [85]. Volz

et al. [141] explored the use of GANs in the context
of match-3 levels, attempting to model the local and
global structures of those levels. Awiszus et al. [3] pro-
posed token-based oneshot arbitrary dimension genera-
tive adversarial network (TOAD-GAN), adapted from
SinGan [108], trained on a single sample level, to gener-
ate tile-based levels. In the work using GANs for level
generation that have been reviewed so far, game levels
are tackled as image only during training while the con-
straints for validating levels are not considered at all.
Recently, Di Liello et al. [20] presented constrained ad-
versarial networks (CANs) which encourages the gen-
erator to learn to generate valid levels by penalizing
Fig. 3: The key phases of DeLeNoX for the autonomous
it due to invalid structures generated during training.
generation of content [69]. DeLeNox adopts the prin-
But still, these methods generate individual segments
ciples of exploration (realized via constrained novelty
of platformer levels separately and then combine them
search), transformation (realized via deep denoising au-
together randomly or according to some increasing level
toencoders) and iterative refinement (realized through
difficulty [140]. Different from above work, Fontaine
the increasing complexity of NEAT architectures). Im-
et al. [29] proposed latent space illumination (LSI),
age reproduced with authors’ permission.
which uses quality diversity algorithms, such as Covari-
ance Matrix Adaptation MAP-Elites (CMA-ME) [28],
to search the latent space of trained generators, aiming
prove itself by playing the new generated levels, while
at increasing the diversity of generated levels. A recent
the generator will improve itself based on the agent per-
work by Kumaran et al. focused on generating levels
formance on its generated levels. In this work, RL is
in multiple distinct games. Instead of training several
used to play the generated content and not to generate
GANs for these games separately, a novel GAN archi-
the content; an RL agent interacted with the generative
tecture, composed of a branched generator and multiple
model to create levels adapted to the agent’s playing
parallel discriminators, was proposed [65].
strength.
Besides generating 2D and 3D levels represented
as pixel-based or tile-based images, texture [25] and
sprite generation [48] have also been investigated.
Hong et al. [48] generated 2D image sprites using a 4.5 Evolutionary Computation
multi-discriminator GAN, in which two encoders were
used for bone graph, shape and color, without shar- There is a long tradition of using evolutionary compu-
ing parameters. Additionally, two discriminators, one tation (EC) approaches for training (deep) neural net-
for shape and the other for color, were used in [48] works. While these are sometimes not regarded as DL,
to generate sprites’ skeletons and color, respectively. the standard definition of DL does in fact not reference
Another potential application is GAN-based charac- gradient descent. Most evolved networks are deep, and
ter generation [55] for video games, such as The Sims architectures created by evolutionary algorithms such
(Maxis, 2000). Wang and Kurabayashi [143] proposed as NEAT [124] often have multiple layers and recurrent
Sketch2Map to generate 3D terrains from sketches. components [102].
Sketch2Map used a conditional GAN (cGAN) to con- For example, Hoover et al. [51] represented game
vert a sketch into an elevation bitmap, which is inter- levels as functional scaffolding for musical composition
preted to generate the practical terrain asset by a de- voices [49]. Taking Mario as an example, each level is
terministic algorithm [143]. presented by a set of voices with the size of possible
More recently, Bontrager and Togelius [4] proposed tile types in a level. Each voice is a one dimensional
a new training method similar to GANs, where the array of same length of the level, in which each element
network consists of two parts: generator and agent. indicates the vertical position of the tile if it presents
The generator is trying to generate new playable levels on the corresponding column, otherwise 0. Neural net-
adapted to the agent’s strength, while the agent plays works were trained and evolved through neuroevolu-
the game and reports how playable it is and how hard tion of augmenting topologies (NEAT) [124] to suggest
it is to play. Similar to GANs, the agent will try to im- placements of tiles in Mario levels [51].
Real levels
GAN training process
(Phase 1)
Real
samples
Real? Fake?
Discriminator
Gaussian noise
Generated
Generator
samples
Generated levels
Latent vector
Trained
Generator
Fig. 5: Screenshot of interactive evolution interface in
[103], reproduced with authors’ permission.
CMA-ES
Simulations of game Evolution
Evaluation (Phase 2)
In the work of Volz et al. [140], a DCGAN [91] is

Fig. 4: Overview process of MarioGan [140], reproduced trained on a set of level segments of Super Mario Bros
with authors’ permission. represented by 2D array of tiles, and then latent vari-
able evolution (LVE) [5] is applied to search for levels
Hoover et al. [50] evolved CPPNs through NEAT that are more playable and encourage particular be-
for generating both audio and visual content in the haviors evaluated by the games simulated by an A*
game AudioInSpace. Risi et al. [94] evolved and trained agent. The overview process is illustrated in Fig. 4. The
CPPNs with NEAT to generate flower images for a resulted framework, MarioGAN [140], certainly identi-
flower-breeding video game Petalz 3 . The CPPNs of dif- fied a new and creative way of generating game con-
ferent flowers can be mated to generate new flowers. tent. However, two issues have been observed: (i) broken
pipes occur in some of the level segments generated by
Evolutionary Computation techniques have also
GANs, and (ii) the segments were connected directly in
been combined with unsupervised DL methods for gen-
an arbitrary order to build complete levels, while how
erating new content. A prominent example is the Deep
to combine segments to make the resulted levels more
Learning Novelty Explorer (DeLeNoX) [69]. DeLeNoX
structured and organized was not exploited. To tackle
alternates phases of content exploration and content
the former issue, Shu et al. [113] trained a MLP model
transformation for the generation of spaceships, de-
to learn the surrounding information of tiles and de-
picted as 2D black and white images (Fig. 3). In the ex-
tect wrong tiles in the generated segments (e.g., Fig.
ploration phase, constrained novelty search seeks max-
6). An evolutionary repairer is designed to search for
imally diverse artifacts and generates a training set. In
optimal replacement tiles for fixing the broken pipe
the transformation phase, a deep autoencoder learns
[113]. To tackle the latter issue, a graph grammar was
to compress the variation between the found artifacts
used to combine rooms of Zelda generated by a GAN
into a lower-dimensional space. The newly trained en-
into dungeons [36], and Schrum et al. [104] proposed
coder is then used as the basis for a new fitness function,
CPPN2GAN which used Compositional Pattern Pro-
transforming the search criteria for the next exploration
ducing Networks (CPPNs) to organize level segments
phase [69]. The process continues repeating exploration
generated by GANs into complete levels.
and transformation phases thereby iteratively refining
and complexifying the generated outcomes. Inspired by [140], Irfan et al. [52] applied LVE and
Arguably one of the most popular examples of EC trained DCGANs on randomly generated levels of 3
for DLPCG is the aforementioned Latent Variable ap- single player games from the GVGAI framework [87],
proach [5], which combines unsupervised learning in Freeway, Zelda and Colourescape. Based on the work
the form of a GAN/VAE with evolutionary compu- of [140], Mott et al. [84] designed a new fitness func-
tation to search for content in the learned space of a tion for CMA-ES as a weighted sum of the number of
GAN/VAE. Originating from synthesizing new finger- frames that an action is feasible, the fraction of agents
print [95], in the context of games this approach has that completed a level and the largest fraction to con-
been employed to generate Super Mario Bros and Zelda trol the difficulty of generated levels. The weights are
levels [104, 140]. evaluated and tuned via the human playing-tests per-
formed on the levels generated using the corresponding
3
https://www.facebook.com/Petalz-238904402867390/ fitness function [84].
Fig. 6: Top: A MLP model trained on human-designed levels labels wrong tiles (in red rectangle) and unsure tiles
(in blue rectangle) in a segment. Bottom: Segment fixed by an evolutionary repairer assisted by the trained MLP
model [113]. Images reproduced with authors’ permission.
Evolutionary methods for content generation can 5 Using Deep Learning to Evaluate Content
also be combined with user feedback, such as through and Content Generators
interactive evolutionary computation (IEC), in which
human evaluation is used instead of the fitness eval-
uation by a simulator. For example, Hastings et al. Evaluating content generators is not a trivial task.
[44] used CPPNs to represent weapons in a multi- Much of the ML and DL-based PCG work has focused
player video game Galactic Arms Race4 . The CPPNs their evaluations on the generated content, and used
are evolved during the game playing with the prefer- those evaluations as proxies for evaluating the genera-
ences abstracted from the past playing of players. IEC tor itself. However, the computational creativity com-
combined with LVE can allow users to breed their own munity has identified that in order to get a full picture
game levels, such as Zelda and Mario [104]. Based on of the generator (or creative program) the process by
[36, 140], a mixed-initiative tile-based level design tool which the output content is created should be evalu-
was implemented by Schrum et al. [103], which allows ated as well. Colton [11], Jordanous [56], Pease and
human to interact with the evolution and exploration Colton [86] each propose frameworks and methodolo-
within latent level-design space (interface illustrated in gies for evaluating the creativity of the process of a gen-
Fig. 5), and to play the generated levels in real-time. erator. Smith and Whitehead [116] (later expanded on
by Summerville [125]) proposed methods for holistically
EC methods can also collaborate with human to evaluating a content generation approach, by evaluat-
generate and evaluate or repair game content. Liapis ing large swaths of generated content to get a broader
et al. [71] presented Sentient World tool which allows understanding of the generative space of a content gen-
interactions with human designers and generates game erator and its biases within that generative space. Sum-
maps using Neuroevolution via novelty search. Sentient merville [125] focused on ML-based generators, and pro-
World can generate high resolution maps based on the posed approaches for highlighting the shortcomings and
rough terrain sketches drawn by designers, as well as strengths of a generator through methodically high-
the iterative refining via selection and editing options lighting generated artifacts (e.g., artifact most similar
opened to designers. to an artifact in the training set).
Karavolos et al. [59] generated levels of a first-person

In this section we survey uses of deep learning for
shooter (FPS) game with targeting gameplay outcomes,
content generation in an indirect fashion. In particular,
in which a genetic algorithm is used to generate levels of
we list studies (cf. Fig. 7) that consider deep learning
specific fitness values based on the predicted outcomes
for testing or evaluating game content through the anal-
by a CNN trained on simulated matches.
ysis of generated content (Section 5.1), construction
of human-like playing bots (Section 5.2), or the con-
struction of reliable models of player experience (Sec-
tion 5.3). We additionally highlight which of these ap-
proaches focus on evaluating the generator itself instead
4
http://gar.eecs.ucf.edu/ of only the content.
5.1 Analyzing Content to personalize games for different players according to

their actions.
Statistical measures on the generated content and sim- Karavolos et al. [57] trained a CNN to predict the
ilarity measures based on the content used in train- outcomes of a simplified 3 versus 3 multiplayer death-
ing set (e.g., [77]) can give insight into the generative match shooter game to evaluate and determine if the
space of a content generator and its biases within that levels, represented by maps and weapon parameters, are
space. Statistical measures can be used to compare the balanced or favoring a team. Based on the predictor for
distribution of generated content to the distribution of the same deathmatch shooter game, Karavolos et al.
the training set [125]. Similarity measures can also be [58] further designed a DL surrogate model for pairing
specifically designed for this task. For example, Lucas levels and character classes for desired game outcomes.
and Volz [77] compared occurrences of small structures Gudmundsson et al. [35] imitated the behavior of hu-
in the generated set to their presence in the training set man through SL and performed experimental study on
to measure similarity. non-deterministic puzzle games Candy Crush Saga and
Many similarity and statistical measures suffer from Candy Crush Soda Saga. A CNN was trained on human
the same drawback of only measuring what is quan- player data, and then used to predict the action that
tifiable. Recent approaches in deep learning can help human players most likely to select when playing levels
avoid this drawback by learning latent semantic fea- that were unseen during training [35]. This approach
tures of the content. Recent work has developed ap- can be used to measure metrics such as the diversity of
proaches to style transfer [61, 74] by traversing the actions to evaluate generated new levels. Notice, each of
learned latent space of the model, and others have an- these methods focuses on evaluating the generated arti-
alyzed the learned latent space of their models to find facts, but can be expanded to more broadly evaluating
semantic meaning in the features [1]. These advances the generator itself if the results of artifact evaluations
have led to the use of latent space-based distance and are used to stratify the generative space or further ex-
similarity measures [144]. Leveraging the latent space plore the biases and capabilities of the generator.
learned by a model to create similarity measures be-
tween pieces of content might allow us to develop more
semantically meaningful similarity measures in addition 5.3 Experiencing Content
to the statistical measures currently in use. As an in-
dicative example of such a research direction, Isaksen Human user trials and surveys can provide the most
et al. [53] categorized tile-based 2D game levels with se- useful insight into the less quantifiable (i.e. subjective)
mantic hashing based on autoencoders. The proposed features of the content and the generation process, such
approach [53] can be used to categorize the generated as the human-perceived quality of the generated con-
level segments or rooms and group the ones sharing sim- tent over time. A large volume of studies focus on the
ilar styles to build a complete game level or dungeon. use of deep learning for modeling aspects of player ex-
perience which can be used, in turn, to evaluate the con-
tent that is generated and experienced by the player.
5.2 Playing Content Player experience is usually provided as annotated la-
bels (ratings or ranks) or even continuous traces via
In this section we review methods based on ANNs and crowdsourcing. Running user evaluations and crowd-
DL for reliable playtesting which can be used, in turn, sourcing labels of subjective aspects such as experience,
to evaluate game content generators in an indirect fash- however, can be a laborious task which may not be fea-
ion. Simulated playtesting [46, 47, 140] of generated sible if what is desired is the quick iteration on the
content can give quick insights into the features of the generative system. One approach for further leveraging
content and the generative space of the content genera- the output of a user evaluation is to treat the user eval-
tor [116, 125]. Guzdial et al. [42] propose the use of deep uations as features to be learned. Larsson and Petri
reinforcement learning agents for simulated playtesting [66] trained neural networks using NEAT to predict
as a way of creating more human-like playtraces. Guz- the user rating of user-created StarCraft maps. This
dial et al. [42] specifically focus on deep RL agents for approach [66] can be extended to evaluate generated
Mario, where human-like control is simulated by giv- StarCraft maps.
ing the agent imprecise controls via stochastic effects Within the platformer genre, a series of studies by
on actions. Similarly, Min et al. [82] designed a goal Shaker et al. [109, 110, 111] investigate the use of
recognition framework based on stacked denoising au- DL models of player experience for the generation of
toencoders for open-ended games, which can be used experience-tailored Super Mario Bros levels. Camilleri
et al. [8] view a player’s believability as a content gener- Analyzing content Playing content
ation problem and used various forms of deep networks [1, 53, 61, 74] [35, 42, 57, 58, 82]
to infer the mapping between game content, gameplay
Experiencing content
and believability in a Super Mario Bros variant. The
[8, 42, 66, 109, 110, 111]
networks of that study predict the degree to which [9, 78, 79, 80, 81, 90, 128]
a combination of gameplay behavior and a generated
level can be considered believable. Guzdial et al. [42]
Fig. 7: Summary of the works that focused on analyz-
trained a CNN to predict rate of the difficulty, enjoy-
ing, playing or experiencing generated content.
ment and aesthetics of game levels and performed case
studies on Infinite Mario Bros, which was further en-
hanced by the features extracted from search history of 6 Discussion and Outlook
an A* agent. Similarly, Summerville et al. [128] used a
regression model on a large set of statistical measures The combination of deep learning and PCG in games
to find measures that predict those same human eval- is beneficial for both game research—as deep learning
uations of Mario levels. More recently, Pfau et al. [90] enhances our capacity to generate content—and deep
proposed deep player behavior modeling (DPBM) with learning research since games pose challenging prob-
a multi-layer perceptron (MLP) trained on behavioral lems for deep learning to solve. Deep learning opens
data and game observation to map game states to ac- new opportunities for the autonomous generation of
tion probabilities. All aforementioned approaches can content of any type and has a plethora of use cases
be used, for instance, to evaluate generated levels. within games. As we saw throughout this article, deep
learning may serve as a content generator, as a con-
The first application of CNNs for modeling player tent evaluator, as a gameplay outcome predictor, as a
experience is introduced by Martinez et al. [80]. CNNs driver of search, and as a pattern recognizer for repair
in that study consider and fuse the content of a 3D and style transfer. This section surveys the areas with a
maze prey-predator game and the in-game behavior of particular importance for the current and future use of
the player [79] and predict reported ranks of player ex- DLPCG in games with an emphasis on mixed-initiative
perience via use deep preference learning. Looking at generation, style transfer and breeding, underexplored
the challenge of player affect modeling by solely focus- content types, learning from small datasets, orchestrat-
ing on gameplay, Makantasis et al. [78] used various ing different content types within a game, and general-
CNN models to predict the level of arousal of survival izing generation across games.
shooter games directly from the pixels of gameplay in
a general player-agnostic fashion. Thus CNNs map be-
tween gameplay behavior and game content as repre- 6.1 Mixed-initiative DLPCG
sented by pixels—such as in-game play features and UI
elements. In principle, such surrogate models of arousal Autonomous PCG systems, including the cases where
can be used directly and evaluate video content of any the initiative of the human designer is limited to al-
game within the the survival shooter genre. In a similar gorithmic parameterizations [148], can hardly generate
recent study various types of neural networks have been content with target quality or features. Recently, more
trained to predict the continuous viewer engagement of and more work takes into account the preferences or in-
PUBG streamed games on Twitch [81]; the engagement put of designers or players in different ways while gen-
models obtained are highly accurate and general across erating content. Mixed-initiative PCG [149], formally
different streamers. Camilleri et al. [9] took player expe- defined as “the process that considers both the human
rience modeling to the next level and built models that and the computer proactively making content contri-
are general across many different games. The models butions to the game design task” [148], offers a more
are build on simple 1-hidden layer networks indicating controllable and practical design process that may in-
the potential of the methodology with larger DL repre- volve the use of DLPCG algorithms but their use is
sentations for the general evaluation of the experience limited so far.
of game content across games. Similar to the previous Level generation in games, as a popular application
section, each of these methods are predominantly used of mixed-initiative DLPCG, requires some initial spec-
to evaluate content. However, using these methods to ifications (i.e. the initiative) from the designer—e.g. in
evaluate large samples of content from a generator can the form of sketches [43]—to assist the design process.
enable a meta-analysis of the types of content a partic- A popular example of the mixed-initiative paradigm
ular generator tends towards creating. is the shallow neural network model presented in [70]
which generates game strategy maps based on the ter- age breeding, among which, the models for portraits
rain sketches drawn by designers. The map generation and anime-style faces, can be used to generate comic or
feature of Sentient Sketchbook features neuroevolution- video game characters and the one for landscapes can be
ary search which is driven by design objectives and the used to generate background images for games. Blend-
novelty of the map. Moving from level to image gen- ing levels from different games has recently gained more
eration, Serpa and Rodrigues [107] adapted the GAN- attention from the research community, with much re-
based Pix2Pix architecture to generate both gray and cent work focusing on blending platformer levels. Sarkar
color pixel art sprites from sketches using a single net- and Cooper [96] and [99] trained separate models on
work. two different games, and then blended new levels using
Taking platform games as the domain under investi- these trained models via interpolation or alternation.
gation, Guzdial et al. [39] developed a mixed-initiative Snodgrass and Sarkar [121] used VAEs to generate level
Super Mario Bros level design tool that leveraged sev- structures, and a search-based approach to blend de-
eral existing PCGML techniques, including Markov tails from various platformers, while Sarkar et al. [100]
chains [117], LSTM [126] and Bayes Net [37], to assist directly trained VAEs on levels from several platform-
the user in creating levels. Guzdial et al. [39] gathered ing games and interpolated the latent vectors between
data on how the users interacted with the models in domains for blending.
the tool, and trained a CNN on that collected data.
This CNN was then used to better predict and gener-
ate level sections along with the user. Later, Guzdial 6.3 Underexplored Content Types
et al. [41] used the trained CNN with active learning
based on the user current interaction to generate levels Most of the reviewed works focus on the design of con-
for Super Mario in a mixed-initiative fashion [68, 149]. tent that can be represented by 2D images of tiles or
Recently, Schrum et al. [103] allowed the designers to pixels, such as 2D levels, landscapes and sprites (cf. Sec-
change manually the latent vectors of the trained gen- tion 4). Only a few of them considered text and narra-
erative model or define the mutation strength of their tive generation, music and rhythm generation, weapons
evolutionary generator for tile-based 2D levels. Delarosa generation for FPS, etc.
et al. [17] presented RL Brush, a human-driven, AI- In the research we have surveyed, platformer and
augmented design tool also for tile-based 2D levels, in dungeon-like games (e.g., arcade games, FPS games and
which RL-based models have been used to enhance hu- adventure games) are clearly over-represented. In par-
man design with suggestions generated by PCG meth- ticular, Super Mario Bros and Zelda are usually used
ods. for testing the GAN-based level generation approaches.
However, the types of games are not limited to ar-
cade games and the generation of some commonly seen
6.2 Style Transfer, Breeding and Blending types of game content are rarely investigated. For in-
stance, the generation of characters (skills, actions, and
Most style transfer methods and generative models for images) for fighting games and multi-player online bat-
image, music and sound [6], can be applied to gener- tle games; the generation of cards and rules for strategy
ate game content. So far, only a few work focused on card games (e.g., Hearthstone); event generation (sto-
the style transfer for game content (e.g., [71, 107, 150]). ries and effects) (e.g., for The Sims); goal generation
[71] generated game maps based on the terrain sketches in all kinds of games. Several approaches from other
and [107] generated art sprites from sketches drawn by fields can be adapted to DLPCG, such as transfer learn-
human. However, a number of diverse input sketches to ing for image generation in games, story generation for
these two work can also be generated using deep learn- text-based adventure games and conversational NPCs.
ing approaches based on a single human sketch [43].
Moreover, algorithms and techniques designed for im-
6.4 Content Generation in Real-time - Personalized
age generation can often be adapted to the automatic
Game content
generation of faces and sprites in games. For instance,
[150] applied a neural styling algorithm [30] to change
Another less explored area is content generation in real-
artistic style of graphics in a strategy game Hedgewars 5 .
time, such as generating level segments during game-
Another example is ArtBreeder 6 , which contains sev-
play, according to the actual player’s playing skill-
eral generative models for creating new images by im-
depth, style and preferences. Taking Super Mario Bros
5
http://www.hedgewars.org/ as an example, several MarioGAN models can be
6
https://artbreeder.com/ trained offline using a variety of fitness functions with
different aims (e.g., encourage more jumps by putting to train on FPS levels from Quake, Halo and Call of
more pipes, put more coins for players to collect, adjust Duty to learn to generate new levels for Half-Life. It
the difficulty by controlling the number of enemies), should be even easier to train character models on exist-
and then be selected to generate new level segments ing human-designed characters from several open-world
during the game after determining the player’s prefer- games, as they share the same functionality constraints.
ences and performance according to the gameplay data The trained generator would likely be a conditional
during first segments. model, that takes some encoding of the characteristics
of a game as input. In all of these cases, the deep learn-
ing model would have to learn to represent the under-
6.5 Learning from Small Data lying similarities between content for the games it was
trained on, as well as the differences.
One of the main limitations for most forms of PCG
based on deep learning, or PCGML in general, is the ac-
cess to training data. Some games have a large amount 6.7 Orchestration for Game Generation
of existing content, either made by developers or by
users. However, for a game in development there may A key future research direction for any PCG framework
not be content to learn from, because the content may is the generation of more than one domain of compu-
not be made yet. In fact, not having to produce all of tational creativity within games. The six key compu-
that content may be a prime reason for wanting to train tational game creativity domains as defined by Liapis
a content generator in the first place. What would be et al. [72] include visuals, audio, narrative, levels, rules
desirable here would be a way of training a generator and gameplay. A process that considers the output of
based on only a few pieces of hand-designed content, two or more of these domain generators up to the gener-
such as items, levels, or characters. ation of a complete game is referred to as orchestration
One approach to doing this is bootstrapping, where [73]. In other words, orchestration can be defined as the
a generator is first trained on just a few examples, “harmonization of the game generation process” [73].
and whenever it produces new content that satisfies While orchestration is a core aim for the au-
the functionality constraints, this content gets added tonomous generation of complete games Liapis et al.
to the training set for continued training of the gener- [73] reported only a few game generation systems that
ator [137]. This approach requires a reliable test of the considered more than one generation domain. These in-
functionality constraints, for example the playability of clude Angelina [12, 13], Game-O-matic [138], Sonancia
a level can be tested with game-playing agents. [76], AudioInSpace [50] and the FPS generator by Kar-
Note that the amount of data required to train a re- avolos et al. [58, 60]. Among these case studies of orches-
liable model varies greatly depending on the complexity trated game generation only a few can be considered
of the model, the complexity of the data, and the train- early embryos of DLPCG-based game orchestration. In
ing procedures of the model. For example, the training particular, the work by Karavolos et al. [58, 60], So-
data limitation does not apply to PCG methods based nancia [76], and AudioInSpace [50] use various forms
on reinforcement learning. Further, MarioGAN [140] of shallow and deep neural networks—both as surro-
was trained on a single Mario level broken into many gate models (indirectly) and as generative functions
sections. Snodgrass et al. [122] explored the effects of (directly)—to generate content for multiple domains
the amount and diversity of training data on a simple within games. As deep learning is of particular impor-
Markov chain model and an LSTM, and found that the tance for fusing the generation process across content
benefits of additional data dropped off after several lev- representations of dissimilar resolutions and character-
els. Further studies exploring the data requirements of istics [148], we expect to witness an increase in DL re-
DLPCG models can help illuminate the usability and search work towards achieving game orchestration.
scalability of these approaches.
7 Conclusions
6.6 Generalization across Games
The work surveyed in this paper is the result of two
Another, and arguably better, approach to learning convergent trends from the last few years. One is the
generators for games for which you do not (yet) have increasing use of deep learning for generative tasks in
much content would be trained on content from other non-game contexts, such as GANs and VAEs used for
games. After all, games from a particular genre have generating pictures of faces and RNNs used for generat-
much in common, and it should arguably be possible ing voices and music. The other is the increasing use of
machine learning in PCG, something that was unheard JCYJ20190809121403553), the Shenzhen Science and Tech-
of until five years or so ago. Both of these trends build nology Program (Grant No. KQTD2016112514355531) and
the Program for University Key Laboratory of Guangdong
on the deep learning revolution itself, which has made Province (Grant No. 2017KSYS008). S. Risi was supported
machine learning effective on completely new classes of by a Google Faculty Research award and a Sapere Aude:DFF-
problems. Starting Grant. A. Khalifa and J. Togelius acknowledge the
As a result, interest in deep learning for PCG has financial support from National Science Foundation (NSF)
award number 1717324 - “RI: Small: General Intelligence
exploded. Examples abound, as our survey shows. It is through Algorithm Invention and Selection”. G. N. Yan-
very likely that we will see rapid progress in this re- nakakis was supported by European Union’s Horizon 2020
search direction in the near future. This survey paper AI4Media (951911) and TAMED (101003397) projects. This
attempts to contribute to this progress by surveying is a pre-print of an article published in Neural Computing
and Applications. The final authenticated version is available
and systematizing this work and implicitly and explic- online at: https://doi.org/10.1007/s00521-020-05383-8.
itly pointing out relevant and fertile research problems.
We believe that this is a very timely effort given the
exciting pace of this field. Conflict of interest
Deep learning methods have been applied alone or
in collaboration with other PCG methods to gener- S. Snodgrass, S. Risi, G. N. Yannakakis, and J. Togelius
ate game content and to analyze, play and experience declare that they a financial interest in modl.ai, which
content. Due to the characteristics of different types develops AI technologies for games.
of content, different types of deep neural architectures
have been used. Among the reviewed work, the widely References
used neural architectures include convolutional neural
networks for supervised learning tasks, varying from 1. Abdal R, Qin Y, Wonka P (2019) Im-
generating texture or music for target emotion to pre- age2StyleGAN: How to embed images into the
dicting game outcomes or difficulty rate; long short- StyleGAN latent space? In: Proceedings of the
term memory for generating sequential data like charts IEEE International Conference on Computer Vi-
for rhythm and narrative or for predicting action se- sion, pp 4432–4441
quences; deep variational autoencoders, mostly used 2. Ammanabrolu P, Cheung W, Tu D, Broniec W,
for generating level maps and sometimes for classifying Riedl MO (2020) Bringing stories alive: Generat-
NPCs’ or players’ behaviors; and generative adversar- ing interactive fiction worlds. In: Proceedings of
ial networks for creating image-like content (e.g., level the Sixteenth Annual AAAI Conference on Artifi-
maps, landscapes, faces and sprites). A part from the cial Intelligence and Interactive Digital Entertain-
direct use of deep learning methods or their alliance ment (AIIDE 2020)
with evolutionary computation to generate game con- 3. Awiszus M, Schubert F, Rosenhahn B (2020)
tent, they have also been used for evaluating content Toad-gan: Coherent style level generation from a
and content generators in an indirect manner. single example. In: Proceedings of the Sixteenth
Although a variety of game content (e.g., levels, Annual AAAI Conference on Artificial Intelli-
text, character models, textures, music and sound) have gence and Interactive Digital Entertainment (AI-
been investigated, the generation of content like event, IDE 2020)
goals or character features with skill-depth can be ex- 4. Bontrager P, Togelius J (2020) Fully differentiable
ploited more. As a future research, evolving or training procedural content generation through generative
game-playing agents and content generators in paral- playing networks. arXiv preprint arXiv:200205259
lel, such as in the recent work of Dharna et al. [19], 5. Bontrager P, Roy A, Togelius J, Memon N, Ross
is of great interest, as well as the generalization across A (2018) DeepMasterPrints: Generating master-
games. Besides those, online generation of game con- prints for dictionary attacks via latent variable
tent to adapt players’ skill and preferences in real-time evolution. In: 2018 IEEE 9th International Con-
will accelerate the realization of personalized games. ference on Biometrics Theory, Applications and
Systems (BTAS), IEEE, pp 1–9
Acknowledgements J. Liu was supported by the Na- 6. Briot JP, Hadjeres G, Pachet F (2019) Deep
tional Natural Science Foundation of China (Grant No. learning techniques for music generation, vol 10.
61906083), the Guangdong Provincial Key Laboratory (Grant Springer
No. 2020B121201001), the Program for Guangdong In-
7. Browne C, Maire F (2010) Evolutionary game de-
troducing Innovative and Entrepreneurial Teams (Grant
No. 2017ZT07X386), the Science and Technology Inno- sign. IEEE Transactions on Computational Intel-
vation Committee Foundation of Shenzhen (Grant No. ligence and AI in Games 2(1):1–16
8. Camilleri E, Yannakakis GN, Dingli A (2016) 20. Di Liello L, Ardino P, Gobbi J, Morettin P, Teso
Platformer level design for player believability. In: S, Passerini A (2020) Efficient generation of struc-
2016 IEEE Conference on Computational Intelli- tured objects with constrained adversarial net-
gence and Games (CIG), IEEE, pp 1–8 works. arXiv preprint arXiv:200713197
9. Camilleri E, Yannakakis GN, Liapis A (2017) To- 21. Donahue C, Lipton ZC, McAuley J (2017) Dance
wards general models of player affect. In: 2017 Sev- dance convolution. In: International Conference
enth International Conference on Affective Com- on Machine Learning, pp 1039–1048
puting and Intelligent Interaction (ACII), IEEE, 22. Dormans J (2010) Adventures in level design: gen-
pp 333–339 erating missions and spaces for action adventure
10. Chen Z, Amato C, Nguyen THD, Cooper S, Sun games. In: Proceedings of the 2010 workshop on
Y, El-Nasr MS (2018) Q-deckrec: A fast deck rec- procedural content generation in games, pp 1–8
ommendation system for collectible card games. 23. Earle S (2019) Using fractal neural networks to
In: 2018 IEEE Conference on Computational In- play SimCity 1 and Conway’s Game of Life at vari-
telligence and Games (CIG), pp 1–8, DOI 10.110 able scales. In: Proceedings of the Experimental
9/CIG.2018.8490446 AI in Games (EXAG) Workshop at AIIDE
11. Colton S (2008) Creativity versus the perception 24. Ebert DS, Musgrave FK, Peachey D, Perlin K,
of creativity in computational systems. In: AAAI Worley S (2003) Texturing & modeling: a proce-
spring symposium: creative intelligent systems, dural approach. Morgan Kaufmann
vol 8 25. Fadaeddini A, Majidi B, Eshghi M (2018) A case
12. Cook M, Colton S, Raad A, Gow J (2013) Me- study of generative adversarial networks for proce-
chanic miner: Reflection-driven game mechanic dural synthesis of original textures in video games.
discovery and level design. In: European Confer- In: 2018 2nd National and 1st International Dig-
ence on the Applications of Evolutionary Compu- ital Games Research Conference: Trends, Tech-
tation, Springer, pp 284–293 nologies, and Applications (DGRC), IEEE, pp
13. Cook M, Colton S, Gow J (2016) The angelina 118–122
videogame design system—part i. IEEE Trans- 26. Fang K, Zhu Y, Savarese S, Fei-Fei L (2020)
actions on Computational Intelligence and AI in Adaptive procedural task generation for
Games 9(2):192–203 hard-exploration problems. arXiv preprint
14. Dahlskog S, Togelius J, Nelson MJ (2014) Lin- arXiv:200700350
ear levels through n-grams. In: Proceedings of the 27. Ferreira LN, Lelis LH, Whitehead J (2020)
18th International Academic MindTrek Confer- Computer-generated music for tabletop role-
ence: Media Business, Management, Content & playing games. In: Proceedings of the Sixteenth
Services, pp 200–206 Annual AAAI Conference on Artificial Intelli-
15. Davoodi O, Ashtiani M, Rajabi M (2020) An ap- gence and Interactive Digital Entertainment (AI-
proach for the evaluation and correction of manu- IDE 2020)
ally designed video game levels using deep neural 28. Fontaine M, Togelius J, Nikolaidis S, Hoover AK
networks. The Computer Journal DOI 10.1093/co (2020) Covariance matrix adaptation for the rapid
mjnl/bxaa071 illumination of behavior space. In: Proceedings of
16. De Kegel B, Haahr M (2020) Procedural puz- the 2020 Genetic and Evolutionary Computation
zle generation: A survey. IEEE Transactions on Conference
Games 12(1):21–40 29. Fontaine MC, Liu R, Togelius J, Hoover AK, Niko-
17. Delarosa O, Dong H, Ruan M, Khalifa A, To- laidis S (2020) Illuminating Mario scenes in the
gelius J (2020) Mixed-initiative level design with latent space of a generative adversarial network.
RL brush. arXiv preprint arXiv:200802778 arXiv preprint arXiv:200705674
18. Dhariwal P, Jun H, Payne C, Kim JW, Radford 30. Gatys LA, Ecker AS, Bethge M (2015) A neu-
A, Sutskever I (2020) Jukebox: A generative model ral algorithm of artistic style. arXiv preprint
for music. arXiv preprint arXiv:200500341 arXiv:150806576
19. Dharna A, Togelius J, Soros L (2020) Coevolu- 31. Giacomello E, Lanzi PL, Loiacono D (2018) Doom
tion of game levels and game-playing agents. In: level generation using generative adversarial net-
Proceedings of the Sixteenth Annual AAAI Con- works. In: 2018 IEEE Games, Entertainment, Me-
ference on Artificial Intelligence and Interactive dia Conference (GEM), IEEE, pp 316–323
Digital Entertainment (AIIDE 2020) 32. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B,
Warde-Farley D, Ozair S, Courville A, Bengio Y
(2014) Generative adversarial nets. In: Ghahra- video game. IEEE Transactions on Computational
mani Z, Welling M, Cortes C, Lawrence ND, Wein- Intelligence and AI in Games 1(4):245–263
berger KQ (eds) Advances in Neural Information 45. Hochreiter S, Schmidhuber J (1997) Lstm can
Processing Systems 27, Curran Associates, Inc., solve hard long time lag problems. In: Advances
pp 2672–2680 in neural information processing systems, pp 473–
33. Goodfellow I, Bengio Y, Courville A (2016) Deep 479
Learning. MIT Press, http://www.deeplearning 46. Holmgård C, Liapis A, Togelius J, Yannakakis
book.org GN (2014) Evolving personas for player decision
34. Greff K, Srivastava RK, Koutnı́k J, Steunebrink modeling. In: 2014 IEEE Conference on Compu-
BR, Schmidhuber J (2017) LSTM: A search space tational Intelligence and Games, IEEE, pp 1–8
odyssey. IEEE Transactions on Neural Networks 47. Holmgård C, Green MC, Liapis A, Togelius J
and Learning Systems 28(10):2222–2232 (2019) Automated playtesting with procedural
35. Gudmundsson SF, Eisen P, Poromaa E, Nodet A, personas through mcts with evolved heuristics.
Purmonen S, Kozakowski B, Meurling R, Cao L IEEE Transactions on Games 11(4):352–362
(2018) Human-like playtesting with deep learning. 48. Hong S, Kim S, Kang S (2019) Game sprite gener-
In: 2018 IEEE Conference on Computational In- ator using a multi discriminator gan. KSII Trans-
telligence and Games (CIG), IEEE, pp 1–8 actions on Internet & Information Systems 13(8)
36. Gutierrez J, Schrum J (2020) Generative adver- 49. Hoover AK, Szerlip PA, Stanley KO (2014) Func-
sarial network rooms in generative graph grammar tional scaffolding for composing additional musi-
dungeons for the Legend of Zelda. In: 2020 IEEE cal voices. Computer Music Journal 38(4):80–99
Congress on Evolutionary Computation (CEC), 50. Hoover AK, Cachia W, Liapis A, Yannakakis GN
IEEE (2015) Audioinspace: Exploring the creative fusion
37. Guzdial M, Riedl M (2016) Game level genera- of generative audio, visuals and gameplay. In: In-
tion from gameplay videos. In: Twelfth Artificial ternational Conference on Evolutionary and Bi-
Intelligence and Interactive Digital Entertainment ologically Inspired Music and Art, Springer, pp
Conference 101–112
38. Guzdial M, Long D, Cassion C, Das A (2017) Vi- 51. Hoover AK, Togelius J, Yannakis GN (2015) Com-
sual procedural content generation with an artifi- posing video game levels with music metaphors
cial abstract artist. In: Proceedings of ICCC Com- through functional scaffolding. In: First Compu-
putational Creativity and Games Workshop tational Creativity and Games Workshop. ACC
39. Guzdial M, Liao N, Riedl M (2018) Co-creative 52. Irfan A, Zafar A, Hassan S (2019) Evolving lev-
level design via machine learning. Proceedings of els for general games using deep convolutional
the Experimental AI in Games (EXAG) Workshop generative adversarial networks. In: 2019 11th
at AIIDE Computer Science and Electronic Engineering
40. Guzdial M, Reno J, Chen J, Smith G, Riedl M (CEEC), IEEE, pp 96–101
(2018) Explainable pcgml via game design pat- 53. Isaksen A, Holmgård C, Togelius J (2017) Seman-
terns. Proceedings of the Experimental AI in tic hashing for video game levels. Game & Puzzle
Games (EXAG) Workshop at AIIDE Design 3(1):10–16
41. Guzdial M, Liao N, Chen J, Chen SY, Shah S, 54. Jain R, Isaksen A, Holmgård C, Togelius J (2016)
Shah V, Reno J, Smith G, Riedl MO (2019) Autoencoders for level generation, repair, and
Friend, collaborator, student, manager: How de- recognition. In: Proceedings of the ICCC Work-
sign of an ai-driven game level editor affects cre- shop on Computational Creativity and Games
ators. In: Proceedings of the 2019 CHI Conference 55. Jin Y, Zhang J, Li M, Tian Y, Zhu H, Fang
on Human Factors in Computing Systems, pp 1– Z (2017) Towards the automatic anime charac-
13 ters creation with generative adversarial networks.
42. Guzdial MJ, Sturtevant N, Li B (2016) Deep static CoRR abs/1708.05509, URL http://arxiv.org/
and dynamic level analysis: A study on infinite abs/1708.05509, 1708.05509
mario. In: Twelfth Artificial Intelligence and In- 56. Jordanous A (2012) A standardised procedure for
teractive Digital Entertainment Conference evaluating creative systems: Computational cre-
43. Ha D, Eck D (2017) A neural representation of ativity evaluation based on what it is to be cre-
sketch drawings. arXiv preprint arXiv:170403477 ative. Cognitive Computation 4(3):246–279
44. Hastings EJ, Guha RK, Stanley KO (2009) Auto- 57. Karavolos D, Liapis A, Yannakakis G (2017)
matic content generation in the galactic arms race Learning the patterns of balance in a multi-player
shooter game. In: Proceedings of the 12th Inter- 69. Liapis A, Martı́nez HP, Togelius J, Yannakakis
national Conference on the Foundations of Digital GN (2013) Transforming exploratory creativity
Games, pp 1–10 with delenox. In: International Conference on
58. Karavolos D, Liapis A, Yannakakis GN (2018) Computational Creativity
Pairing character classes in a deathmatch shooter 70. Liapis A, Yannakakis GN, Togelius J (2013) Sen-
game via a deep-learning surrogate model. In: Pro- tient sketchbook: Computer-aided game level au-
ceedings of the 13th International Conference on thoring. In: In Proceedings of The 2013 ACM Con-
the Foundations of Digital Games, pp 1–10 ference on Foundations of Digital Games
59. Karavolos D, Liapis A, Yannakakis GN (2018) Us- 71. Liapis A, Yannakakis GN, Togelius J (2013) Sen-
ing a surrogate model of gameplay for automated tient world: Human-based procedural cartogra-
level design. In: 2018 IEEE Conference on Com- phy. In: International Conference on Evolutionary
putational Intelligence and Games (CIG), IEEE, and Biologically Inspired Music and Art, Springer,
pp 1–8 pp 180–191
60. Karavolos D, Liapis A, Yannakakis GN (2019) 72. Liapis A, Yannakakis GN, Togelius J (2014) Com-
A multi-faceted surrogate model for search-based putational game creativity. In: ICCC
procedural content generation. IEEE Transactions 73. Liapis A, Yannakakis GN, Nelson MJ, Preuss M,
on Games Early Access Bidarra R (2018) Orchestrating game generation.
61. Karras T, Laine S, Aittala M, Hellsten J, Lehti- IEEE Transactions on Games 11(1):48–68
nen J, Aila T (2019) Analyzing and improv- 74. Liu MY, Breuel T, Kautz J (2017) Unsuper-
ing the image quality of stylegan. arXiv preprint vised image-to-image translation networks. In:
arXiv:191204958 Advances in neural information processing sys-
62. Khalifa A, Bontrager P, Earle S, Togelius tems, pp 700–708
J (2020) Pcgrl: Procedural content genera- 75. Liu Z, Luo P, Wang X, Tang X (2015) Deep learn-
tion via reinforcement learning. arXiv preprint ing face attributes in the wild. In: Proceedings
arXiv:200109212 URL https://arxiv.org/ab of International Conference on Computer Vision
s/2001.09212 (ICCV)
63. Kingma DP, Welling M (2013) Auto-encoding 76. Lopes P, Liapis A, Yannakakis GN (2015) Sonan-
variational bayes. arXiv preprint arXiv:13126114, cia: Sonification of procedurally generated game
64. Kuang P, Luo D (2020) Conditional convolu- levels. In: ICCC
tional generative adversarial networks based in- 77. Lucas SM, Volz V (2019) Tile pattern kl-
teractive procedural game map generation. In: Fu- divergence for analysing and evolving game levels.
ture of Information and Communication Confer- In: Proceedings of the Genetic and Evolutionary
ence, Springer, pp 400–419 Computation Conference, Association for Com-
65. Kumaran V, Mott BW, Lester JC (2020) Gener- puting Machinery, New York, NY, USA, GECCO
ating game levels for multiple distinct games with ’19, p 170–178, DOI 10.1145/3321707.3321781,
a common latent space. In: Proceedings of the Six- URL https://doi.org/10.1145/3321707.3321
teenth Annual AAAI Conference on Artificial In- 781
telligence and Interactive Digital Entertainment 78. Makantasis K, Liapis A, Yannakakis GN (2019)
(AIIDE 2020) From pixels to affect: A study on games and player
66. Larsson S, Petri O (2016) Content evaluation of experience. In: 2019 8th International Conference
starcraft maps using neuroevolution. URL http: on Affective Computing and Intelligent Interac-
//urn.kb.se/resolve?urn=urn:nbn:se:bth-11 tion (ACII), IEEE, pp 1–7
684 79. Martı́nez HP, Yannakakis GN (2014) Deep multi-
67. Liang Y, Li W, Ikeda K (2019) Procedural con- modal fusion: Combining discrete events and con-
tent generation of rhythm games using deep learn- tinuous signals. In: Proceedings of the 16th Inter-
ing methods. In: Joint International Conference national conference on multimodal interaction, pp
on Entertainment Computing and Serious Games, 34–41
Springer, pp 134–145 80. Martinez HP, Bengio Y, Yannakakis GN (2013)
68. Liapis A, Yannakis GN (2016) Boosting com- Learning deep physiological models of af-
putational creativity with human interaction in fect. IEEE Computational intelligence magazine
mixed-initiative co-creation tasks. In: Proceedings 8(2):20–33
of the ICCC Workshop on Computational Cre- 81. Melhart D, Gravina D, Yannakakis GN (2020)
ativity and Games Moment-to-moment Engagement Prediction
through the Eyes of the Observer: PUBG 94. Risi S, Lehman J, D’Ambrosio DB, Hall R, Stan-
Streaming on Twitch. In: Foundations of Digital ley KO (2015) Petalz: Search-based procedural
Games content generation for the casual gamer. IEEE
82. Min W, Ha EY, Rowe J, Mott B, Lester J (2014) Transactions on Computational Intelligence and
Deep learning-based goal recognition in open- AI in Games 8(3):244–255
ended digital games. In: Tenth Artificial Intelli- 95. Roy A, Memon N, Ross A (2017) Masterprint:
gence and Interactive Digital Entertainment Con- Exploring the vulnerability of partial fingerprint-
ference based authentication systems. IEEE Transactions
83. Mordvintsev A, Randazzo E, Niklasson E, on Information Forensics and Security 12(9):2013–
Levin M (2020) Growing neural cellular au- 2025
tomata. Distill DOI 1 0. 23 91 5/ di st il l. 00 023, 96. Sarkar A, Cooper S (2018) Blending levels from
https://distill.pub/2020/growing-ca different games using lstms. In: Proceedings of the
84. Mott J, Nandi S, Zeller L (2019) Controllable Experimental AI in Games (EXAG) Workshop at
and coherent level generation: A two-pronged ap- AIIDE
proach. In: Experimental AI in Games Workshop 97. Sarkar A, Cooper S (2020) Sequential segment-
85. Park K, Mott BW, Min W, Boyer KE, Wiebe EN, based level generation and blending using
Lester JC (2019) Generating educational game variational autoencoders. arXiv preprint
levels with multistep deep convolutional genera- arXiv:200708746
tive adversarial networks. In: 2019 IEEE Confer- 98. Sarkar A, Cooper S (2020) Towards game design
ence on Games (CoG), IEEE, pp 1–8 via creative machine learning (gdcml). In: Pro-
86. Pease A, Colton S (2011) On impact and evalua- ceedings of The 2020 IEEE Conference on Games
tion in computational creativity: A discussion of (CoG)
the turing test and an alternative proposal. In: 99. Sarkar A, Yang Z, Cooper S (2019) Controllable
Proceedings of the AISB symposium on AI and level blending between games using variational au-
Philosophy, vol 39 toencoders. In: Proceedings of the Experimental
87. Perez-Liebana D, Liu J, Khalifa A, Gaina RD, AI in Games (EXAG) Workshop at AIIDE
Togelius J, Lucas SM (2019) General video game 100. Sarkar A, Summerville A, Snodgrass S, Bentley G,
ai: A multitrack framework for evaluating agents, Osborn J (2020) Exploring level blending across
games, and content generation algorithms. IEEE platformers via paths and affordances. In: Six-
Transactions on Games 11(3):195–214 teenth Artificial Intelligence and Interactive Digi-
88. Perez-Liebana D, Lucas SM, Gaina RD, Togelius tal Entertainment Conference
J, Khalifa A, Liu J (2019) General Video Game 101. Schaul T (2013) A video game description lan-
Artificial Intelligence. Morgan & Claypool Pub- guage for model-based or interactive learning. In:
lishers, https://gaigresearch.github.io/gvg Proceedings of the IEEE Conference on Computa-
aibook/ tional Intelligence in Games, IEEE Press, Niagara
89. Perlin K (1985) An image synthesizer. ACM Sig- Falls
graph Computer Graphics 19(3):287–296 102. Schmidhuber J (2015) Deep learning in neural net-
90. Pfau J, Liapis A, Volkmar G, Yannakakis GN, works: An overview. Neural Networks 61:85 – 117,
Malaka R (2020) Dungeons & replicants: Auto- DOI https://doi.org/10.1016/j.neunet.2014.09.0
mated game balancing via deep player behavior 03
modeling. In: Proceedings of the 2020 IEEE Con- 103. Schrum J, Gutierrez J, Volz V, Liu J, Lucas SM,
ference on Games (CoG) Risi S (2020) Interactive evolution and exploration
91. Radford A, Metz L, Chintala S (2015) Unsu- within latent level-design space of generative ad-
pervised representation learning with deep con- versarial networks. In: Proceedings of the Genetic
volutional generative adversarial networks. arXiv and Evolutionary Computation Conference, ACM
preprint arXiv:151106434 104. Schrum J, Volz V, Risi S (2020) CPPN2GAN:
92. Radford A, Wu J, Child R, Luan D, Amodei D, Combining compositional pattern producing net-
Sutskever I (2019) Language models are unsuper- works and GANs for large-scale pattern genera-
vised multitask learners. OpenAI Blog tion. In: Proceedings of the Genetic and Evolu-
93. Risi S, Togelius J (2019) Increasing generality in tionary Computation Conference, ACM
machine learning through procedural content gen- 105. Schuster M, Paliwal KK (1997) Bidirectional re-
eration. arXiv preprint arXiv:191113071 current neural networks. IEEE transactions on
Signal Processing 45(11):2673–2681
106. Scirea M, Eklund P, Togelius J, Risi S (2018) 118. Snodgrass S, Ontanon S (2015) A hierarchical
Evolving in-game mood-expressive music with mdmc approach to 2d video game map generation.
metacompose. In: the Audio Mostly 2018 on In: Eleventh Artificial Intelligence and Interactive
Sound in Immersion and Emotion, pp 1–8 Digital Entertainment Conference
107. Serpa YR, Rodrigues MAF (2019) Towards 119. Snodgrass S, Ontanón S (2016) Controllable pro-
machine-learning assisted asset generation for cedural content generation via constrained multi-
games: A study on pixel art sprite sheets. In: 2019 dimensional markov chain sampling. In: IJCAI, pp
18th Brazilian Symposium on Computer Games 780–786
and Digital Entertainment (SBGames), IEEE, pp 120. Snodgrass S, Ontanón S (2016) Learning to gener-
182–191 ate video game maps using markov models. IEEE
108. Shaham TR, Dekel T, Michaeli T (2019) Singan: transactions on computational intelligence and AI
Learning a generative model from a single natural in games 9(4):410–422
image. In: Proceedings of the IEEE International 121. Snodgrass S, Sarkar A (2020) Multi-domain
Conference on Computer Vision, pp 4570–4580 level generation and blending with sketches via
109. Shaker N, Yannakakis G, Togelius J (2010) To- example-driven bsp and variational autoencoders.
wards automatic personalized content generation In: Proceedings of the 15th International Confer-
for platform games. In: Sixth Artificial Intelligence ence on the Foundations of Digital Games
and Interactive Digital Entertainment Conference 122. Snodgrass S, Summerville A, Ontañón S (2017)
110. Shaker N, Togelius J, Yannakakis GN, Weber B, Studying the effects of training data on machine
Shimizu T, Hashiyama T, Sorenson N, Pasquier P, learning-based procedural content generation. In:
Mawhorter P, Takahashi G, et al. (2011) The 2010 Thirteenth Artificial Intelligence and Interactive
Mario AI championship: Level generation track. Digital Entertainment Conference
IEEE Transactions on Computational Intelligence 123. Soares ES, Bulitko V (2019) Deep variational au-
and AI in Games 3(4):332–347 toencoders for npc behaviour classification. In:
111. Shaker N, Nicolau M, Yannakakis GN, Togelius J, 2019 IEEE Conference on Games (CoG), IEEE,
O’neill M (2012) Evolving levels for Super Mario pp 1–4
Bros using grammatical evolution. In: Computa- 124. Stanley KO, Miikkulainen R (2002) Evolving neu-
tional Intelligence and Games, IEEE, pp 304–311 ral networks through augmenting topologies. Evo-
112. Shaker N, Togelius J, Nelson MJ (2016) Procedu- lutionary Computation 10(2):99–127
ral Content Generation in Games. Springer 125. Summerville A (2018) Expanding expressive
113. Shu T, Wang Z, Liu J, Yao X (2020) A novel range: Evaluation methodologies for procedural
cnet-assisted evolutionary level repairer and its content generation. In: Fourteenth Artificial In-
applications to Super Mario Bros. In: 2020 IEEE telligence and Interactive Digital Entertainment
Congress on Evolutionary Computation (CEC), Conference
IEEE 126. Summerville A, Mateas M (2016) Super Mario as
114. Sirota J, Bulitko V, Brown MR, Hernandez SP a string: Platformer level generation via LSTMs.
(2019) Towards procedurally generated languages In: International Joint Conference of DiGRA and
for non-playable characters in video games. In: FDG
2019 IEEE Conference on Games (CoG), IEEE, 127. Summerville A, Guzdial M, Mateas M, Riedl MO
pp 1–4 (2016) Learning player tailored content from ob-
115. Smith AM, Mateas M (2011) Answer set program- servation: Platformer level generation from video
ming for procedural content generation: A design traces using lstms. In: Twelfth Artificial Intelli-
space approach. IEEE Transactions on Computa- gence and Interactive Digital Entertainment Con-
tional Intelligence and AI in Games 3(3):187–200 ference
116. Smith G, Whitehead J (2010) Analyzing the ex- 128. Summerville A, Mariño JR, Snodgrass S, Ontañón
pressive range of a level generator. In: Proceed- S, Lelis LH (2017) Understanding Mario: An eval-
ings of the 2010 Workshop on Procedural Content uation of design metrics for platformers. In: Pro-
Generation in Games, pp 1–7 ceedings of the 12th International Conference on
117. Snodgrass S, Ontañón S (2014) Experiments in the Foundations of Digital Games, pp 1–10
map generation using markov chains. In: Proceed- 129. Summerville A, Snodgrass S, Guzdial M,
ings of the 9th Conference on the Foundations of Holmgård C, Hoover AK, Isaksen A, Nealen
Digital Games A, Togelius J (2018) Procedural content gen-
eration via machine learning (PCGML). IEEE
Transactions on Games 10(3):257–270 141. Volz V, Justesen N, Snodgrass S, Asadi S, Pur-

130. Summerville AJ, Mateas M (2016) Mystical tutor: monen S, Holmgård C, Togelius J, Risi S (2020)
A magic: The gathering design assistant via de- Capturing local and global patterns in procedural
noising sequence-to-sequence learning. In: Twelfth content generation via machine learning. In: Pro-
artificial intelligence and interactive digital enter- ceedings of The 2020 IEEE Conference on Games
tainment conference (CoG)
131. Summerville AJ, Philip S, Mateas M (2015) 142. Walton N (2019) AI Dungeon 2: Creating in-
MCMCTS PCG 4 SMB: Monte carlo tree search finitely generated text adventures with deep learn-
to guide platformer level generation. In: Artificial ing language models. https://pcc.cs.byu.edu
Intelligence and Interactive Digital Entertainment /2019/11/21/ai-dungeon-2-creating-infi
132. Togelius J, Kastbjerg E, Schedl D, Yannakakis GN nitely-generated-text-adventures-with-
(2011) What is procedural content generation? deep-learning-language-models/, accessed:
mario on the borderline. In: Proceedings of the 2020-05-02
2nd International Workshop on Procedural Con- 143. Wang T, Kurabayashi S (2020) Sketch2map: A
tent Generation in Games, ACM, p 3 game map design support system allowing quick
133. Togelius J, Yannakakis GN, Stanley KO, Browne hand sketch prototyping. In: Proceedings of The
C (2011) Search-based procedural content genera- 2020 IEEE Conference on Games (CoG)
tion: A taxonomy and survey. IEEE Transactions 144. Wong A, Wang GH (2017) Image retrieval demo:
on Computational Intelligence and AI in Games a demo for image retrieval. https://github.com
3(3):172–186 /DoctorKey/image retrieval demo
134. Togelius J, Champandard AJ, Lanzi PL, Mateas 145. Wulff-Jensen A, Rant NN, Møller TN, Billeskov
M, Paiva A, Preuss M, Stanley KO (2013) Pro- JA (2017) Deep convolutional generative adver-
cedural content generation: Goals, challenges and sarial network for procedural 3d landscape gener-
actionable steps. In: Schloss Dagstuhl-Leibniz- ation based on dem. In: Interactivity, Game Cre-
Zentrum fuer Informatik ation, Design, Learning, and Innovation, Springer,
135. Togelius J, Shaker N, Karakovskiy S, Yannakakis pp 85–94
GN (2013) The Mario AI championship 2009- 146. Yang Z, Sarkar A, Cooper S (2020) Game level
2012. AI Magazine 34(3):89–92 clustering and generation using gaussian mixture
136. Torrado RR, Bontrager P, Togelius J, Liu J, Perez- vaes. In: Proceedings of the Sixteenth Annual
Liebana D (2018) Deep reinforcement learning for AAAI Conference on Artificial Intelligence and
general video game AI. In: Proceedings of The Interactive Digital Entertainment (AIIDE 2020),
2018 IEEE Conference on Computational Intel- AAAI
ligence and Games (CIG), IEEE, pp 1–8 147. Yannakakis GN, Togelius J (2011) Experience-
137. Torrado RR, Khalifa A, Green MC, Justesen N, driven procedural content generation. IEEE
Risi S, Togelius J (2019) Bootstrapping condi- Transactions on Affective Computing 2(3):147–
tional gans for video game level generation. arXiv 161
preprint arXiv:191001603 148. Yannakakis GN, Togelius J (2018) Artificial Intel-
138. Treanor M, Blackford B, Mateas M, Bogost ligence and Games. Springer, http://gameaibo
I (2012) Game-o-matic: Generating videogames ok.org
that represent ideas. In: Proceedings of the The 149. Yannakakis GN, Liapis A, Alexopoulos C (2014)
third workshop on Procedural Content Generation Mixed-initiative co-creativity. In: Proceedings of
in Games, pp 1–8 the 9th Conference on the Foundations of Digital
139. Tsujino Y, Yamanishi R (2018) Dance dance gra- Games
dation: A generation of fine-tuned dance charts. 150. Yoo B, Kim KJ (2016) Changing video game
In: International Conference on Entertainment graphic styles using neural algorithms. In: 2016
Computing, Springer, pp 175–187 IEEE Conference on Computational Intelligence
140. Volz V, Schrum J, Liu J, Lucas SM, Smith A, and Games (CIG), IEEE, pp 1–2
Risi S (2018) Evolving mario levels in the latent 151. Yumer ME, Asente P, Mech R, Kara LB (2015)
space of a deep convolutional generative adversar- Procedural modeling using autoencoder networks.
ial network. In: Proceedings of the Genetic and In: Proceedings of the 28th Annual ACM Sym-
Evolutionary Computation Conference, ACM, pp posium on User Interface Software & Technol-
221–228 ogy, Association for Computing Machinery, New
York, NY, USA, UIST ’15, p 109–118, DOI
10.1145/2807442.2807448 11th Computer Science and Electronic Engineer-

152. Zafar A, Irfan A, Sabir MZ (2019) Generat- ing (CEEC), IEEE, pp 134–138
ing general levels using markov chains. In: 2019

Deep Learning For Procedural Content Generation

Uploaded by

Copyright:

Available Formats

Deep Learning For Procedural Content Generation

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Deep Learning For Procedural Content Generation

Uploaded by

Copyright:

Available Formats

Noname manuscript No.

(will be inserted by the editor)

Deep Learning for Procedural Content Generation

Received: date / Accepted: date

2.2 Novelty of The Review 3 Content Types

3.2 Text Most games feature a soundtrack, often composed of

Due to the different types and roles of content in games,

2D Map/Level 3D Map/Level Rhythm

[151] [145] [31] [85] [143]

tain goals (based on the current generation problem).

4.4 Adversarial Learning

Adversarial learning (AL) models are perfect for gener-

verted to 3D levels to be used in the game [85]. Volz

In the work of Volz et al. [140], a DCGAN [91] is

Karavolos et al. [59] generated levels of a first-person

5.1 Analyzing Content to personalize games for different players according to

Transactions on Games 10(3):257–270 141. Volz V, Justesen N, Snodgrass S, Asadi S, Pur-

10.1145/2807442.2807448 11th Computer Science and Electronic Engineer-

You might also like