Entertainment Computing 5 (2014) 367377

Contents lists available at ScienceDirect

Entertainment Computing
journal homepage: ees.elsevier.com/entcom

Flow framework for analyzing the quality of educational games q

Kristian Kiili a,, Timo Lainema b, Sara de Freitas c, Sylvester Arnab c
a

Tampere University of Technology, Pohjoisranta 11 A, P.O. Box 300, 28101 Pori, Finland
Turku School of Economics, University of Turku, Rehtorinpellonkatu 3, FI-20500 Turku, Finland
c
Serious Games Institute, Coventry University, Coventry, United Kingdom
b

a r t i c l e

i n f o

Article history:
Received 4 June 2013
Revised 3 July 2014
Accepted 1 August 2014
Available online 22 August 2014
Keywords:
Game-based learning
Game analysis
Serious games
Technology-enhanced learning
Flow experience
User experience

a b s t r a c t
The challenge of educational game design is to develop solutions that appeal to as many players as possible, but are still educationally effective. One foundation for analyzing and designing educational
engagement is the ow theory. This article presents a ow framework that describes the dimensions
of ow experience that can be used to analyze the quality of educational games. The framework also provides design-support for producing good educational games, because it can be used to reveal ways to
optimize learning effects and user experience. However, the framework only works as a link between
educational theory and game design, which is useful for game analysis but does not provide the means
for a complete game design. To evaluate the elements included in the proposed framework, we analyzed
university students experiences in participating in a business simulation game. We found that the
students ow experience in the game was high and the ndings indicated that sense of control, clear
goals and challenge-skill dimensions of ow scored the highest. Overall, the results indicate that the ow
framework is a useful tool to aid the analysis of game-based learning experiences.
2014 Elsevier B.V. All rights reserved.

1. Introduction
The purpose of games is to create appealing and compelling
experiences to players. Thus, games can be seen as artefacts or a
cultural form that arouse meaningful immersive experiences
[1,2]. According to Dewey [3] experience is a result of interplay
between the present situation and our prior experiences. More
recently, neuroscientists such as Gerard Edelman have explained
learning as building upon existing mental maps [4]. Consequently,
players do not have identical playing experiences, but each players
experience is totally unique. Thus, the analysis of the subjective
playing experience is crucial part of the game design process. The
enjoyment level that an educational game offers is a key factor in
determining whether the player will be engaged in the gameplay
and achieve the objectives of the game. Thus, the ability to quantify
the playing experience is important goal for both industry and
academia.
In general, we need a reliable way to measure the overall
engagement level of games and to pinpoint specic areas of the
experience that should be improved. Several constructs have been
proposed to describe playing experience, but denitional agreement has not been achieved. The most common concepts that have
q

This paper has been recommended for acceptance by Francesco Bellotti.

Corresponding author.

http://dx.doi.org/10.1016/j.entcom.2014.08.002
1875-9521/ 2014 Elsevier B.V. All rights reserved.

been linked to playing experience are ow [5,6], immersion [7],

presence [8], involvement, and arousal, which have overlapping
but also distinctive characteristics. According to Procci et al. [9]
the concept of ow is one of the most popular constructs to
describe the playing experience. Flow describes a state of complete
absorption in an activity and refers to the optimal experience
[5,10]. During the optimal experience, a person is in a psychological state where he or she is so involved with the goal-driven activity that nothing else seems to matter. An activity that produces
such experiences is so pleasant that the person may be willing to
do something for its own sake, without being concerned with what
he will get out of his action. Csikszentmihalyis [5,10] ow theory
subsequently has been applied in several different domains including, for example sports, art, work, humancomputer interactions,
games and education. In fact, according to [11] preliminary
research suggests that game-playing experience is consistent with
the dimensions of the ow experience.
The basic elements that comprise every game are: mechanics,
story, aesthetics and technology. These are all essential and none
of the elements is more important than the others [1]. In educational games the learning objective is also involved, which makes
the game design more challenging. Educational games have to be
designed properly to incorporate engagement that integrates with
educational effectiveness. While work on existing learning theories
is well developed, in recent work, three areas of learning theory

368

K. Kiili et al. / Entertainment Computing 5 (2014) 367377

have been outlined for game-based learning: associative (more

task-centered approaches to learning), cognitive which rely upon
constructivist approaches to learning and situative (more
socially-based learning) [12]. These learning approaches create a
theoretical foundation for our current work.
The aim of this article is to propose a ow framework that facilitates the analysis educational games and provides design-support
for game developers. The design principles of engagement [6]
provide a starting point for this work. The paper starts with a background section that discusses the elements that constitute user
experience and the pedagogical theories that frame the desired
learning process and experience. The following section describes
the proposed ow framework. Finally, in order to evaluate the
usefulness and the relevance of the ow framework, its attributes
and potential to indicate success of a game design, the analysis of
students experience with an educational business simulation
game, RealGame, is reported.

2. Background
2.1. User experience
There have been some efforts in creating models of user experience [e.g. 1316]. In particular there is a need for designers of educational artefacts to understand how users interact with different
types of artefacts and how this interaction affects users educational experiences. While some work in simulation design [e.g.
17] has explored this, the need to consider this from an educational
gaming perspective is relatively under-theorized, which presents
problems for replicating good design and developing improving
standards of design.
The user experience is often paralleled with usability [e.g. 18],
although the user experience does not consider enough the deeper
principles of experience design or the emotional side of product
use. It is obvious that user experience approach extends usability
techniques [19] that aim more at the removal of obstacles from
technical perspective than at providing engaging and rewarding
experiences for users. In this paper usability or playability in a
game context is considered as being only one factor among others
that affects user experience. This view is in line with Forlizzi and
Battarbee [14] who have argued that user experience should be
considered also from physical, sensual, cognitive, emotional, and
aesthetic perspectives.
Fig. 1 shows the authors macro-level conception about user
experience. The aesthetical, emotional and sensual aspects are
not distinguished in macro-level. However, in micro-level they

Fig. 1. The macro-level elements of the user experience.

are seen as integral parts of game artifact that affect user

experience.
The user experience consists of three main elements: users, an
artefact and a task. The user experience emerges from the interplay
between these elements in a certain context of use. This context of
use is the actual condition under which a given artefact is normally
used. The characteristics of the users, such as emotions, values and
prior experience, determine how users perceive an artefact and the
task at hand. We want to note that we understand the task concept
broadly and thus it also refers to the goals of the user related to a
certain activity. The usability of an artefact is determined base on
the interaction between the users and the artefact. Usefulness
refers to the design of an artefact containing the right functions
required for users to perform their tasks efciently and to accomplish their goals easily and efciently [20].
If the task is engaging, the user is willing to use more effort in
accomplish the task. Skinner and Belmonts [21] denition of
engagement in the educational context can be applied to user
experience. According to them, engagement refers to the intensity
and emotional quality of a users involvement in initiating and carrying out activities. Engaged users show sustained behavioural and
cognitive involvement in activities accompanied by a positive
emotional tone. To summarize, good usability, a useful artefact
and an engaging task (challenges that the game provides) create
prerequisites for a good educational experience. However, we want
to emphasize that designers cannot design the subjective experience; only the context from which the experience arouses may
be designed.
2.2. Constructivism and cognitive load theory
Wu et al. [22] in their recent study found that until 2009 the
majority of published studies on game-based learning were not
based on any specic learning theory in their study only 91 of
567 studies based their investigations on a learning theory. They
also found that the development of learning theory orientations
has prompted more studies to focus on constructivism and humanism (i.e. experiential learning, which has had a central role in simulation game research; see Lainema [23]) than on behaviorism and
cognitivism. A look at the very recent research on game-based
learning (especially in research that takes place in the discipline
of education) reveals a plethora of game studies that base their
argumentation on constructivism. For example, constructivism
has recently been referred to when studying learning in virtual
worlds [24,25], business simulation games [23,26,27], primary
schools and elementary education [28,29], educational game
development [30], and debrieng of game learning [31].
In fact, for example Mayer [32] has argued that constructivism
has become the dominant view of how people learn. The underlying premise of constructivism is that learning is a process in which
learners are active sense makers who seek to construct coherent
and organized knowledge [3,32,33]. This means that in games
learning occurs when the players active exploration (i.e. exploring
the game world and testing discovered solutions to games problems) makes them develop a knowledge representation of their
experience or discover an inconsistency between their current
knowledge representation and their experience. Attributed to view
of social constructivism, learning usually occurs within a social
context in which interactions between other people will activate
collaborative exploration, articulation, reection, and hence assimilation or accommodation for improved knowledge representations
[34,35]. However, according to Kirschner et al. [36] constructivism
is too often implemented using minimal guidance approach that
wrongly assumes that people learn best in an unguided or minimally guided environment the recent instructional design
research has clearly shown that guidance support learning.

K. Kiili et al. / Entertainment Computing 5 (2014) 367377

In order to be able to understand how can we support active

knowledge construction process in games we have to consider
the structure of human cognitive architecture. Any instructional
process that ignores the human cognitive architecture is not likely
to be effective [36]. The cognitive load theory provides a foundation to consider learner characteristics from the cognitive perspective. In general, the purpose of the cognitive load theory is to
bridge the gap between information structures presented in the
learning materials and human cognitive architecture so that learners can use their working memory more efciently [37]. Human
cognitive architecture is based on models of human memory that
is usually divided to sensory memory, working memory and
long-term memory from which working memory and long-term
memory are most important for educational game designers. When
designing educational games we should consider the constraints of
human cognition and design the gameplay according to target
groups skills, characteristics and prior knowledge.
In terms of cognitive load theory free exploration of highly complex game environment and bad instructional design may generate
too heavy cognitive load that can hinder learning. Sweller et al.
[38] have identied three separate sources of cognitive load. Cognitive load may be affected by the intrinsic nature of the material
(intrinsic cognitive load), the manner in which the material is presented (extraneous cognitive load), or by the effort needed for the
construction of schemata (germane cognitive load).
Intrinsic cognitive load refers to the inherent nature of the task or
the subject matter of the learning content. If the learning content
consists of numerous elements that are related to one another, the
intrinsic cognitive load is high. In contrast, if the material is simple,
including only a few connections between elements, the intrinsic
cognitive load is low. According to the cognitive load theory,
instructional design cannot change the intrinsic cognitive load.
Therefore, the most important aspects of the cognitive load theory
for educational game designers are extraneous cognitive load and
germane cognitive load. From learning point of view extraneous
cognitive load is unnecessary cognitive load and is determined by
the instructional design. If the game is poorly designed, the extraneous cognitive load is high because learners have to engage in irrelevant cognitive processing. Mayer [39] has primarily examined
different presentation formats in order to reduce the extraneous
cognitive load of learning materials. However, the reduction of the
extraneous cognitive load by an ideal instructional format does
not guarantee that all free cognitive resources will be allocated to
a deeper knowledge construction process [40]. Unused working
memory capacity should be used by optimizing the germane cognitive load, by stimulating the player to process the provided content
more deeply. According to Kirschner [41] the approach of encouraging learners to engage in appropriate cognitive processing can only
work if the total cognitive load of instructional design is within
working memory limits. If a learners cognitive system is overloaded, it might impact negatively on learning and playing behavior.

3. Flow framework for educational games

This section describes the proposed ow framework for educational games (see Fig. 2). The elements of ow can be divided into
two groups: Flow antecedents (the grey star in Fig. 2) and the Flow
state. The ow antecedents (described in more in detail in Section
3.1) are factors that contribute to the ow state and thus should be
considered in educational game design. The dimensions of ow
state (described more in detail in Section 3.2) are more abstract
and they in a way describe the feelings of ow experience. The
white triangles that surround the star describe meaningful factors
that affect the design of the learning experience and game-based
learning artefacts. In the model the order of antecedents is not

369

Fig. 2. Flow framework for educational games.

relevant and they not purposely bind to factors surrounding them.

Finally on the planes of the pentagon are illustrated ve mind
lenses (described more in detail in Section 3.3). (1) The sensing
mind, (2) the processing mind, (3) the integrating mind, (4) the
relating mind, and (5) the transferring mind lenses provide means
to consider game elements systematically from the learning and
interaction perspectives and relates ow dimensions to learning
processes. Theoretically the mind lenses are founded on principles
of cognitive load theory [41], multimedia learning theory [32] and
constructivism [42]. The educational foundation of the framework
and focus points of the mind lenses are based on the constructivism and cognitive load theory as discussed in Section 2.2.

3.1. Description of ow antecedents

All the ow antecedents except playability see (Fig. 2) are consistent with the original ow dimensions [5]. Next the antecedents
are described and discussed in terms of educational game design.
When the players goals are clear he can more easily stay
focused on the learning tasks. It is good practice to provide a clear
main goal in the beginning of the game. Furthermore, the main
goal should be divided into sub-goals and provide them at an
appropriate pace in order to create feelings of success. If the goals
seem too challenging, the probability of experiencing ow is low.
Furthermore, the goals should be related to the learning objectives
of the game. If the learning objectives are discrete from gameplay,
the game may fail to produce educationally effective experiences.
The main purpose of the feedback is to inform the player about
his performance and progression toward the goals, to monitor progress of the learner by the tutor, and to create a feedback loop
between the game and the level achieved. In the proposed model,
the feedback dimension is divided into immediate feedback and
cognitive feedback [43]. The immediate feedback keeps the player
focused. If the player has to wait long before he can realize what
effect his action caused, he will become distracted and loose the
focus on the task. Additionally, the delayed feedback may create
interpretation problems and in the worst-case even lead to misconceptions and negative learning transfer. The cognitive feedback
relates to the cognitive problem solving it provides the account
for learning and cognitive immersion. The cognitive feedback aims
to stimulate the player to reect on his experiences and tested

370

K. Kiili et al. / Entertainment Computing 5 (2014) 367377

solutions in order to further develop his mental models [44] and

playing strategies. In other words, it focuses players attention on
information that is relevant for learning objectives. However, the
main issue within game-based experiences has been that feedback
models are often generalized rather than personalized even though
the technology would allow the modeling of user performance and
user characteristics.
Previous research has demonstrated how feedback can be used
in a more sophisticated way to personalize the game experience
and to create more user-centered design [45]. In their previous
work, the authors have proposed a new feedback model that
includes the type, content, format and frequency of feedback to
be given in-game and extra-game [45]. For example, feedback
can be given to the learner via scaffolded learning in the use of
an in-game avatar. For example, in AnimalClass games [46] a
players avatars gestures illustrate the certainty of its knowledge.
Based on the agents gestures, a player can gure out what his
agent knows and what he should do next. In Roma Nova a similar
approach is adopted where virtual agents present the learner with
information about ancient Rome and provide missions and quests.
Bellotti et al. introduce the Experience Engine [47], an ad-hoc articial intelligence engine designed to deliver tasks in order to optimize each players experience and meeting teachers dened
educational goals.
The playability antecedent is included to replace Csikszentmihalyis action-awareness merging dimension, which is problematic
in the learning game context. This was done, because according
to Csikszentmihalyi, all ow inducing activities become spontaneous and automatic, which is not desirable from a learning point of
view. In contrast, the principles of experiential and constructive
learning approaches give emphasis to the point that learning is
an active and conscious knowledge-construction process. It is
noteworthy that reection is not always a conscious action by a
player. However, only when a player consciously processes his
experiences can he make active and aware decisions about his
playing strategies and thereby form a constructive hypothesis to
test. Thus, a distinction between activities related to learning and
controlling the game should be made. This means that controlling
the game (input controls; e.g. controlling the character with
mouse) should be spontaneous and automatic, but the educational
content related to a players tasks should be consciously processed
and reected. However, this is not always the case; sometimes
learning to control a game can be a main task or use of challenging
user interfaces can positively affect on cognitive abilities (e.g.
attention, mastery). This is true for games that would require
devices or interfacing instruments to be utilized as part of the
gameplay (e.g. using a haptic instrument to play virtual snooker
or to control an airplane, to virtually touch virtual objects [48]
and using Wii control to emulate physiotherapy).
Generally, the aim of a learning game is to provide students
with challenges that are balanced with their skill level. Furthermore, challenges should be related to the main task so that the
ow experience is possible. When both the task and the use of
the artefact are complex, then the artefact and the task may detract
from the players attention. In fact, bad playability decreases the
likelihood of experiencing task-based ow because the player has
to sacrice attention and other cognitive resources to the inappropriate activity. Because the information processing capacity of
working memory is limited [49], all possible resources should be
available for relevant information processing (the main task)
rather than for the use of the game controls. Thus, the aim is to balance the complexity of the main task as well as the control needed
to complete the tasks. Generally, the aim of the user interface
design of games is to support the shift from cognitive interaction
to uent interaction, which in some cases will require come practice to acquire uency. In an ideal situation, the controls of the

game are transparent and allow the player to focus on higher order
cognition rather than solely upon tasks. However, designers should
ensure that they do not over simplify the gameplay in the way that
players can only supercially rush through the game without deeper processing of the game content.
The challenge dimension can be explained with the three-channel model of ow [5,10]. Challenges and skills that are theoretically
the most important dimensions of ow are represented on the axes
of the model (Fig. 3). The letter P represents a person playing, for
example, snooker. At the beginning (P1), the player has only little
knowledge about snooker and can only perform basic shots. However, the player enjoys the activity (is occasionally in ow) because
he feels that the difculty is just right for his rudimentary skills.
While training his basic shots, the players skills are bound to
improve, and he will feel bored (P2) performing such shots. Or
he might notice that playing against an opponent is still too hard
and he will realize that there are much greater challenges than
performing basic shots individually. His poor performance may
cause feelings of anxiety (P3).
Boredom and anxiety are negative experiences that motivate
the player to strive for the ow state. If the player is bored (P2),
he has to increase the challenge he is facing. The player can set a
more difcult goal that matches his skills. For example, he could
play against an appropriate opponent that he can barely win
against in order to get back to the ow state (P4). In contrast, if
the player feels anxiety (P3), he must increase his skills in order
to get back to the ow state (P4). The player could, for example,
develop his playing strategy and train to perform safety shots. In
general, it can be said that ow emerges in the space between anxiety and boredom. The ow channel can be extended by providing
some guidance to the player, or by providing the possibility of solving problems collaboratively. The need to adopt constructivist as
well as associative learning is reected in this need for cognitive
as well as task-centered approaches to learning in-game. Thus,
Vygotskys zone of proximal development [35] is added to the original model. The zone of proximal development refers to the
difference between what a learner can do without help and what
he or she can do with help. For example, in the snooker case, the
player could ask for help from more procient players to help
him to develop his cue technique and playing strategy. The
extended model also acknowledges the importance of situative
or social learning [51].

Fig. 3. The extended three-channel model of ow (Modied from [50]).

K. Kiili et al. / Entertainment Computing 5 (2014) 367377

The model shows that ow is a linear channel where both P1

and P4 represent situations where the player is in the ow state.
Although both situations are equally enjoyable, P4 is more complex because the challenges involved and skills required are
greater. Neither situations P1 or P4 are permanent states because
every now and then the player tends to either feel boredom or anxiety, which motivates him to strive for the ow state in order to
feel enjoyment again. In conclusion, this dynamic feature explains
why ow activities lead to growth and discovery. From the point of
view of learning activities, the three-channel model of ow has an
important role in that it represents how the process of ow might
develop through a single activity. The challenge of the game design
is to keep the player in a ow state by increasing the skill level of
the game while the skill level of the player increases in order to
maximize the impact of playing. The way how different people perceive challenges makes the balancing of challenge level difcult.
Some people seem to prefer very challenging task although they
may perform badly in a game. On the other hand some people prefer easy challenges and expect to perform well in a game. Thus, in
many studies the reliability of challenge-skill dimension is usually
quite low.
In many competitive games the behaviour of opponents affects
the challenge level of the game. In general, opponents can be either
human-controlled or computer-controlled. The construction of
human-like behaviour in games is challenging and requires methods far beyond scripted interactions [52,53]. The previous research
results indicate that the type of the opponent inuences signicantly the playing experiences. For example, [11] showed that
users who played against a human-controlled opponent reported
more experiences of enjoyment and ow. Thus, the challenge of
game design is to create believable human-like behaviour for
non-player characters that can adapt to players skill level and that
way facilitates ow experiences [54].
Sense of control clearly relates to the challenge-skill balance
dimension. Csikszentmihalyi [10] has stated that sense of control
refers to possibility rather than to actuality of the control. It can
be said that a person senses when he can develop sufcient skills
to reduce the margin of error close to zero, which makes the experience enjoyable. For example, a trainee snooker player can train
hard and dream about perfect skills. However, unconsciously he
knows that he cannot ever reach such a skill level, but still the illusion, a dream of it, lives and motivates the player to work hard
towards his goals and dreams.
3.2. Description of ow state
According to [50], whenever people reect on their ow experiences, they mention some, and often all, of the following characteristics: concentration, time distortion, rewarding experience and
loss of self-consciousness. During a ow experience, such as during
game-play, a person is totally focused on the activity and is able to
forget all unpleasant things. However, during important activities
such as learning, it is hard to stop thinking how others evaluate
us. When a player can ignore what others think of him or her,
the player has lost self-consciousness. The problem is that the criticism that the player may face turns his attention away from the
actual task and turns too much to self, which does not facilitate
the performance and playing experience. Self seems to disappear
from awareness during ow in ow there is no room for selfscrutiny [10] and thus the player is not worried about others opinions (e.g. players, teachers, etc.). Here the self refers to the selfesteem and thus loss of self-consciousness does not limit reective
thinking processes. According to Csikszentmihalyi [10] during the
ow experience the sense of time tends to bear little relation to
the passage of time as measured by the absolute convention of a
clock. Time seems to either pass really fast or the seconds may feel

371

like minutes. Rewarding experience refers to an activity that is

done, not with the expectation of some future benet, but simply
because the doing itself is interesting and fun.
For example, [55] have stated that sports can offer such rewarding experiences that one does it for no other reason than to be part
of it. Furthermore, they argue that a sport setting is structured to
enhance ow. Although winning is important in sports, ow does
not depend on the nal outcomes of an activity, and offers athletes
something more than just a successful outcome. The playing of
games is convergent with sports. In fact, an optimal experience
usually occurs when a persons body or mind is stretched to its limits in a voluntary effort to accomplish something difcult and
worthwhile [10]. Such experiences are not necessarily pleasant
when they occur, but they still produce enjoyment. However, no
matter whether the experience is pleasant or not, ow works as
a hook that engages players and gets them to play games again
and again.
3.3. Description of mind lenses
The essential processes for active learning are: selecting relevant material, processing selected material, integrating selected
material with prior knowledge, sharing own understanding with
others e.g. [39] and the ability to transfer the learned knowledge
and skills into new situations. When playing serious games learners are challenged to extract relevant information from a game
world, interpret provided feedback, select corresponding parts of
information and integrate all of these elements to coherent representation and at the same time track the state of the game, decide
right actions to carry out, and possibly communicate with other
players. This requires a lot from the player, because the game
world changes during playing, important information may be presented only a while, and thus it needs to be kept actively in working memory in order to integrate it to earlier presented
information and relate it to ones actions and prior knowledge.
Such demands may easily impose high cognitive load in learners
cognitive system and hinder effective learning and playing. Thus,
the ow framework distinguishes ve mind lenses that provide
means to systematically and reectively consider game elements
from the learning and interaction perspectives without neglecting
the ow principles.
3.3.1. The lens of the sensing mind
The lens of the sensing mind relates to sensory memory. It
emphasizes that humans cannot attend to all things at once, but
have to select areas of interest from the world. In general, learners
prior knowledge affects how learner perceives the game world and
what he or she selects to process consciously in working memory.
Attention is used to focus our mental capacities of the sensory
input so that the mind can successfully process the stimulus of
interest [56]. For the design of game based learning interactions
this means that we should ensure that the learner selects the most
important content and elements (things that support the adoption
of learning objectives) from the game for further processing in his
working memory. The game designers should consider the use of
audio-visual effects to grasp players attention in crucial moments.
3.3.2. The lens of the processing mind
The lens of the processing mind addresses the limitations of
human working memory that is responsible of processing selected
information and retrieving existing knowledge from long-term
memory. This lens emphasizes that the human working memory
is very limited in both capacity and duration. According to Miller
[49], we can deal with no more than seven elements of information
at a time without overloading the information processing capacity
and decreasing the effectiveness of processing. Thus, according to

372

K. Kiili et al. / Entertainment Computing 5 (2014) 367377

Sweller et al. [38] any interactions between elements held in the

working memory require working memory capacity, reducing the
number of elements that can be dealt with simultaneously. When
designing learning interactions we should remember that every
game element has a cognitive price and they should be wisely used
[57]. For example, Schrader and Bastiaens [58] found that the highimmersive gaming environment can lead to the strongest form of
virtual presence but on the other hand can also decrease learning.
This nding is consistent with multimedia learning theory arguing
that multimedia presentations are more effective when irrelevant
material is excluded e.g. [59]. Thus, the game designers should
decide whether the extraneous elements engage players so much
that it is reasonable to include them into the game or not. One
further issue, which complicates considering the lens of the processing mind is that games usually provide a learning environment
which is dynamic and complex (i.e. games create a process of activities and events linked to and dependent on each other) and in this
respect are different from most other forms of teaching and learning. This dynamicity is clearly a thing that creates a challenge for
the game designers as game design is not only about the content,
the game elements and the way they are presented, but also their
cause-effect relationships that stress the learners working memory. Furthermore, the representation of the game elements should
be optimized according to multimedia learning principles [39]. For
example, the modality effect states that working memory capacity
may be increased by the use of visual, auditory and haptic information processing channels simultaneously. In complex games haptic
feedback could provide new ways to reduce extraneous cognitive
load and that way release working memory resources to other
senses. Furthermore, haptic interaction supports immersion while
learners can experience the game more realistically and
pervasively.
3.3.3. The lens of the integrating mind
The lens of the integrating mind emphasizes the meaning of
long-term memory in human intellectual skills. According to the
cognitive load theory the human intellectual ability relies on
knowledge stored in long-term memory, because our working
memory is incapable of highly complex interactions involving
novel content. Thus, we should carefully consider the learners
knowledge level when designing learning interactions designs
that require learners to engage in complex reasoning processes
involving combinations of unfamiliar elements are likely to be decient [38]. Integrating of selected information with existing knowledge involves building connections between relevant portions of
prior knowledge and incoming information. The game designers
should consider how to support the activation of knowledge in
long-term memory in order to facilitate integration process. To
summarize, according to this lens it is important to be aware of
the characteristics of the target group and optimize the challenges
and game content accordingly.
3.3.4. The lens of the relating mind
The lens of the relating mind addresses the meaning of sharing
your learning experiences with others, which is central in the
sociocultural view of constructivism. Duffy and Cunningham [60]
claim that a primary way in which mental functions are altered
by the mediation of language signs is that knowledge, and thereby
learning, becomes a social, communicative, and discursive process,
inexorably grounded in talk. These views connect the game learning elements besides of being something to just interact with to
the social nature of game-based learning.
For a new player of any multiplayer game the process of starting
as a novice and gradually becoming a more skillful player is almost
exactly how Lave and Wenger [61] describe learning as legitimate
peripheral participation. Peripheral participation means that a

learner is always located in the social world, and changing locations and perspectives are part of actors learning trajectories,
developing identities, and forms of membership. Here peripherality is a positive term, suggesting an opening, a way of gaining access
to sources for understanding through growing involvement. The
learner is a newcomer who changes knowledge, skill, and discourse
and at some point becomes an old-timer. This is a process of developing identity and the learner transforms into a member of a community of practice. This process is motivated by the growing use
value of participation, and by the newcomers desires to become
more skilled and knowledgeable members of the community. The
challenge of the game design is to keep the player in a ow state
by increasing the skill level of the game while the skill level of
the player increases in order to maximize the impact of playing
and support the development of a player. This process is something
that can be quite clearly be seen in massive multiplayer games
where a novice player gradually learns to become a better player
and attains a better social status in the virtual game world. Social
status, leaderboards and different forms of character development
plays a crucial role in engaging users to playing and socializing.
What is said above does not mean that in the learning situation
the players should be left working, communicating and collaborating alone. According to Hmlinen and Oksanen [62] in authentic
learning contexts totally free collaboration does not necessarily
promote productive collaboration or high-level learning. Thus,
players need support for collaboration that can be embedded into
the game itself or provide teacher a possibility to facilitate players
collaboration. For example, collaboration scripts are used to support collaborative learning [63]. The main idea of collaboration
scripts is to improve collaboration through structuring interaction
processes among players [62].
Ketamo and Kiili [57] have shown that breaks in game playing
facilitate conceptual change in single player games. They argue
that informal discussions about the game during breaks triggered
players to reect on their playing behavior and consider the content more deeply, which led to changes in thinking. This nding
supports the idea of debrieng that has been found to be crucial
part of game based learning interventions. Both the in game and
external discussions relay on Vygotskys [35] principles arguing
that learning is best understood in light of others within an individuals world. In fact, the Zone of Proximal Development (ZPD)
dened by Vygotsky [35] stresses that the assistance and scaffolding cues provided by the more capable persons may lead to learners intellectual growth. To summarize, the purpose of relating
mind lens is to trigger game designers to implement game elements that facilitate purposeful collaboration, sharing of knowledge and provides teachers possibilities to participate in game
events.
3.3.5. The lens of the transferring mind
The purpose of transferring mind lens is to emphasize that
game designers should consider ways to facilitate the transfer of
learning in games in order to maximize the benets. Transfer of
learning (hereafter transfer) means the ability to apply knowledge
or procedures learned in one context to new contexts. Transfer is
usually divided into near and far transfers [64]. Near transfer refers
to ability to apply learned knowledge and procedures in closely
related settings. In contrast, far transfer refers both to the ability
to use what was learned in one setting to a clearly different one
as well as the ability to solve novel problems that share a common
structure with the knowledge initially learned.
The research has indicated that transfer of knowledge and problem-solving skills into novel situations is rare especially when far
transfer is considered [64]. For example, DeLeeuw and Mayer
[65] found that adding game-like features to a computer-based
learning activity caused students to pay attention to game details

K. Kiili et al. / Entertainment Computing 5 (2014) 367377

but did not motivate students particularly men to learn more

deeply. However, the study utilized only some game like features
such as competition and thus the results cannot be generalized
to all kind of games. One of the main reasons for the inability of
traditional teaching methods to facilitate the development of exible and useful knowledge and skills is the lack of contextualizing
or anchoring the content being learned [66]. In fact, the promise of
serious games lays on assumption that in games learning can take
place in authentic context [67]. An important question is in what
delity we should model the real world context in games so that
the transfer of learning is possible. Inadequate delity may result
in unfullled learning outcomes as learners struggle to address
the additional demands required to reect on virtual experiences
in the context of real-world events [6870]. When considering
the representation of the content we should also remember the
constraints that limited capacity of working memory causes.
The taxonomy proposed by Barnett and Ceci [64] breaks transfer into content and context factors. The content factor considers
what is transferred. It is important to consider learning objectives
of the game because design of transfer may be expected to differ
depending on whether one should learn a specic fact or more general principle. The aim of the context factor is to clarify the distinction between near and far transfer. The context factor is divided
into several dimensions including knowledge domain, physical
context, temporal context, functional context, social context, and
modality. The consideration of these dimensions is important
when trying to design an educational game that supports transfer.
According to Bereby-Meyer et al. [71] motivation plays an
important role in transfer. The literature of achievement motivation goals makes a distinction between performance goals and
mastery goals [72]. Performance goals refer to engagement in the
task with the purpose of demonstrating ones competence. People
directed by performance goals tend to avoid challenges because
they perceive failure as indicating insufcient ability. In contrast,
mastery goals refer to an engagement in the task with purpose of
mastering skills and learning. Mastery goals tend to foster adaptive
patterns of achievement characterized by challenge seeking, concentration on task and metacognitive activity as well as transfer
[73]. Thus, from ow perspective game designers should structure
the game in the way that it facilitates mastery goals. Especially,
goal and feedback dimensions of ow can be used to direct the
motivation to mastery goals. However, this does not mean that
mastery of the game is necessary to experience ow, but the ow
experience facilitates the achievement of intrinsically motivating
mastery goals.
4. Case study on RealGame
The objective of this case study is to consider the usefulness of
the ow framework (Fig. 2) in studying ow experience in educational games, which would link to whether the design facilitates
the achievement of high level of ow. A collaborative business simulation game called RealGame [74] was used as a test bed. RealGame was selected to this study, because we wanted to study
ow in a complex game environment that is not visually as attractive as common entertainment games. The analysis of ow experience using the proposed framework is a means to evaluate the
proposed attributes included in the framework.

373

part of the course Enterprise Systems, which is a course given by

the department of Information Systems Science. The participants
were mainly majoring in Accounting and Finance, Marketing,
Management, Logistics, and Information Systems.
4.2. Description of the test bed
In RealGame business simulation game (http://www.
realgame.) the problems and situations that the students face
are designed to be very similar to those of real-life working contexts of business organization. The students are supposed to apply
their schooled knowledge and skills in the gaming environment.
An important characteristic of the simulation game is its clock-driven nature that reects realistic time-dependent decision-making
in the business world. Such continuous processing presents
authentic tasks rather than abstract instructions. This means that
the game is not turn-based like most of the business games, but
it has an internal time, which proceeds at a certain dened speed.
The game operator can change the clock speed during the game if
he or she wants to. In terms of ow this means that the game operator can increase the game challenge and skills requirement
through increasing the game clock speed while the skill level of
the players increases and players are able to make decisions faster.
The game operator can make decisions about the game speed
adjustment according to players performance analytics that the
game provides. The game clock speed cannot be adjusted individually, which means that all players of the gaming session work
under same time pressure.
The purpose of the used game scenario was to set a team of
players in a position where they steer a manufacturing company
called Modern Bikes Ltd (the second simulation session of the
course). The imaginary Modern Bikes simulation company is situated in one of the Nordic countries and it produces Road bikes
and Mountain bikes for three different market areas. The aim of
using the simulation game was to give the participants a view of
the different functions in a manufacturing organization and to
illustrate how challenging it is to parameterize different automatic
enterprise information systems functions, like the re-order point in
the inventory, when the customer demand is not stable. Fig. 4
shows some of the decision-making areas and windows of the
game.
The Modern Bikes model was played in a competitive format:
the companies within each session competed against each other
(common raw material resources and common customer markets).
During the game teams made different kinds of decisions on different aspects dealing with the operational environment of the simulation company. This means that they manage the basic material
ow, follow market reports, and try to react to competitor market
actions, and so on. For example, teams can make decisions on
terms of delivery, sales prices, terms of payment, marketing investments, and product development. Playing the simulation game is
demanding as the teams also have to manage the whole supply
chain process from suppliers to customers and the monetary process of the company. In terms of the extended three-channel model
of ow, in RealGame the challenge level for the participants is
adjusted by increasing or decreasing the simulation internal clock
speed. This way the problem of anxiety or boredom is avoided, but
managing this requires that the game operator is constantly in the
picture.

4.1. Participants

4.3. Procedure

The participants of the study were students of Turku School of

Economics, Finland (N = 98). The majority of the participants were
younger than 25 and they participated this course on their 2nd or
3rd year of studies. The business simulation gaming sessions were

As 129 students enrolled the course, it was decided to have ve

exercise groups. Each of these ve groups played the simulation
game twice. Before the sessions, the students were given a simulation introduction document and a short pre-assignment. In the

374

K. Kiili et al. / Entertainment Computing 5 (2014) 367377

Fig. 4. Example view of RealGame business simulation game interface.

introductory rst simulation session, the simulation game was less

complex and the simulation clock ran more slowly than in the second session. The second session (Modern Bikes Ltd,) was organized
two weeks after the introduction session. The participants were
given basic information upon which to plan this new situation so
that the increased simulation speed would not become uncontrolled during the second playing session.
Each of the sessions lasted approximately four hours. In each
gaming session there were 68 companies competing against one
another. The companies were steered by groups of two to four participants (the most common number being 3 students in 27 out of
32 groups). During the sessions the game was occasionally stopped
and nancial reports were run. The participants were given time to
analyze the game process and to create plans for their future operations. Gradually during the sessions the clock speed was
increased. At the end of the day the gaming part of the session
was stopped and situation reports were run, and analysis and game
debrieng performed.
Research data was gathered from the second simulation session. After the simulation session players were asked to ll in the
questionnaires. 103 students participated the second gaming session and out of these, 98 returned a properly lled questionnaire.
4.4. Measures
The data related to ow was gathered with a 9-item questionnaire developed by the authors (see http://www.owfactory./
research/owscale.pdf). A 6-point Likert-type response format
was used. The items included were derived from the GameFlow
questionnaire [6]. The dimensions included were challenge, goal,
feedback, playability, concentration, time distortion, rewarding
experience, loss of self-consciousness, and sense of control. Each

dimension was measured with a scenario-based item in order to

avoid interpretation problems that have appeared in earlier studies. For example, the feedback dimension was operationalized as
follows: The game provided me such a feedback that I was aware
how I was performing. I could really perceive the consequences of
my actions. We also utilized the nancial and performance results
from the students managed companies (Turnover, Prot, average
production costs etc.), indicating the groups ability to manage
their decision-making environment.
4.5. Results
Table 1 shows that the ow level experienced by the players
was high (M = 4.60, SD = .63) and experiences were quite congruent. The reliability of the used ow questionnaire indicates that
the ow dimensions are internally quite consistent (a = .78). This
result supports the ndings of [10] who argued that whenever people reect on their ow experiences, they often tend to mention all

Table 1
Means and standard deviations of ow dimensions (N = 98).
Flow dimension

SD

Challenge skill balance

Clear goals
Feedback
Playability
Sense of control
Rewarding experience
Concentration
Loss of self-consciousness
Time distortion
Flow experience (construct)

4.81
4.95
4.40
4.18
5.14
4.43
4.46
4.44
4.57
4.60

.98
.90
1.13
1.27
.97
1.05
1.10
1.35
1.06
.62

K. Kiili et al. / Entertainment Computing 5 (2014) 367377

the nine ow dimensions. In general, high mean values of each

dimension indicate that the game was well-designed and provided
appropriate circumstances for experiencing ow. The feeling of
control, clear goals, and challenge-skill balance dimensions scored
the highest values.
The players had clear goals and they understood the aim of the
game (M = 4.95, SD = .90). In the game players could dene shortterm goals for their company, which facilitates achieving of ow.
Although the user interface of the game is complex and the
playability dimension of ow scored lowest (M = 4.18, SD = 1.27)
high sense of control was experienced (M = 5.14, SD = .97). This
indicates that players felt that by practicing they can learn to
master the simulation environment.
Although RealGame is not visually as rich as most commercial
video games, it still produces very high experiences of ow.
Plausible reasons for this are that the challenge in the simulation
game is a meaningful one (M = 4.81, SD = .98). The participants
showed high ownership on their achievements in their simulation
game company. The meaningfulness of the task was probably one
of the key characteristics in the simulations successful application.
Furthermore, the varying clock speed of the simulation functioned
as a tool with which to maintain good level of interest of the participants. Increment of the clock speed raised the challenge level
and while the players learned to run their companies better and
better, the challenge level was never too easy. In fact the players
reported that they really liked the time pressure that increased
clock speed created. This outcome shows clear support for the
three-channel model of ow.
The ow construct had clear relations with game performance.
The ow correlated with (1) Turnover (r = .29, p = .004; Turnover is
the sales of the simulation company, and calculated automatically
by the simulation application), (2) Prot (r = .33, p = .001;
Prot = Turnover different costs in the company; calculated automatically), and (3) Teams position within the game session
according to the Prot gures (r = .31, p = .002). Furthermore, the
analysis of user behaviour indicated that the ability to inuence
on game events contributes to the ow experience. For example,
the reward dimension was related with the number of changes
each team made in their sales offers (r = .32, p = .001) and the number of all team decisions and activities (like reports run and windows selected) made (r = .28, p = .007). These results seem to
indicate that games, which require continuous situation scanning
and decision-making, and include time-intensity, provide good
possibilities for experiencing ow. Overall, it can be said that the
ability to inuence on game events as well as on other players is
one of the major factors enhancing the ow experience.
In terms of mind lenses players seemed to struggle now and
then in nding all relevant information to back up their decisions.
However, this is actually a feature of the game that aims to simulate the complex decision making system. Thus, in this kind of
game it is not reasonable to provide all information directly to
players, but players have to know what information they need
and nd that information from the system. In fact, the game does
not include any extraneous information and graphics, but only
content that can be used in making decisions and controlling the
game. From processing mind perspective the time intensity of
the game can also be regarded as harmful for the learning, if the
time pressure becomes the actual purpose. Tin fact, the observations of the training sessions revealed that without breaks in the
simulation (clock stopped) and purposeful preceding planning
assignments the participants would not be able to effectively consider how they are performing in the simulation and how they
could boost their performance. The operator of the game have to
ensure that players have enough time to reectively process the
feedback that the game provides as well as time to discuss about
the game events and decisions as the relating mind lens suggests.

375

Like the players gradually learn how to master their game company, the teacher also learns how to run the game sessions and
facilitate learning outcomes. Without decent breaks the participants would satisfy in just playing the simulation game without
necessarily having very high motivation in achieving good nancial
results, reecting on their performance, and constructing new
knowledge. The observations also showed that almost without
exception the teams are extremely eagerly discussing their simulation companys challenges and decisions. This takes place under
the time pressure and our assumption is that the time intensity
of the environment leads to very high concentration in the decision-making and meaningfulness of the experience, which further
enhances the teamwork. Thus, time intensity together with joint
responsibility in the team facilitates ow and intensive peer learning. Finally, in this case we did not study the transfer of learning
and the gathered data cannot be used to consider the transfer of
learning lens of the ow framework.
5. Conclusions
In this paper we have presented the ow framework for analyzing and designing educational games. The framework describes the
dimensions of ow experience that can be used to analyze the
quality of educational games. The framework also provides
design-support for producing good educational games, because it
can be used to reveal ways to optimize learning effects and user
experience. In order to support the integration of learning objectives the framework distinguishes ve mind lenses that provide
means to consider game elements systematically from the learning
and interaction perspectives. However, the framework works only
as a link between user experience, educational theory and game
design and does not provide the means to a whole game design
project. Nevertheless, the ow framework can be used to scrutinize
game designs and reveal new ways to optimize learning effects and
user experience.
Based on the proposed ow framework we studied the playing
experiences of RealGame business simulation game. The results
showed that the framework can be used to analyze the overall quality of the playing experience, but it does not provide detailed information about the shortages or highlights of the game. If the aim is to
study reasons why the game fails to produce a good playing experience, the used ow scale needs to be extended with dimensions
related to game mechanics and audio-visual implementation or
complementary research methods has to be used. In general, the
proposed framework has the potential to inform the design of
new games as well as provide insights to redesign existing games
based on the analysis of the attributes included in the framework.
In the future we will concentrate on developing an extended
playing experience scale that takes also the game mechanics, user
interface solutions, audio-visual implementation, social aspects
and the mind lenses into account. Furthermore, one of our future
aims is to validate the framework in different game contexts.
Acknowledgements
This work has been co-funded by the EU under the FP7, in the
Games and Learning Alliance (GALA) Network of Excellence, Grant
Agreement nr. 258169.
References
[1] J. Schell, The Art of Game Design: a Book of Lenses, Morgan Kaufmann
Publishers, Burlington, MA, 2008.
[2] S. de Freitas, T. Neumann, The use of exploratory learning for supporting
immersive learning in virtual environments, Comput. Educ. 52 (2) (2009)
343352.

376

K. Kiili et al. / Entertainment Computing 5 (2014) 367377

[3] J. Dewey, Experience and Education, Simon and Schuster, New York, 1938/
1997.
[4] G. Edelman, Bright Air, Brilliant Fire: On the Matter of the Mind, Basic Books,
1994. pp. 316.
[5] M. Csikszentmihalyi, Beyond Boredom and Anxiety, Jossey-Bass, San Francisco,
1975.
[6] K. Kiili, T. Lainema, Foundation for measuring engagement in educational
games, J. Interact. Learn Res. 19 (3) (2008) 469488.
[7] C. Jennett, A.L. Cox, P. Cairns, S. Dhoparee, A. Epps, T. Tijs, A. Walton, Measuring
and dening the experience of immersion in games, Int. J. Hum. Comput. Stud.
66 (9) (2008) 641661.
[8] W. Wirth, T. Hartmann, S. Bocking, P. Vorderer, C. Klimmt, H. Schramm, T.
Saari, J. Laarni, N. Ravaja, F.R. Gouveia, F. Biocca, A. Sacau, L. Jncke, T.
Baumgartner, P. Jncke, A process model of the formation of spatial presence
experiences, Media Psychol. 9 (2007) 493525.
[9] K. Procci, A.R. Singer, K.R. Levy, C. Bowers, Measuring the ow experience of
gamers: an evaluation of the DFS-2, Computers in Human Behavior 2012
conference, 28(6), 2012, 23062312.
[10] M. Csikszentmihalyi, Flow: The Psychology of Optimal Experience, Harper
Perennial, New York, 1991.
[11] D. Weibel, B. Wissmath, S. Habegger, Y. Steiner, R. Groner, Playing online
games against computer- vs. human-controlled opponents: effects on
presence, ow, and enjoyment, Comput. Hum. Behav. 24 (2008) 22742291.
[12] S. de Freitas, J. Jameson, E-Learning Reader, Continuum Press, London & New
York, 2012.
[13] B. Buxton, Sketching User Experiences: Getting the Design Right and the Right
Design, Morgan Kaufmann Publishers, Redmond, Washington, 2007.
[14] J. Forlizzi, K. Battarbee, Understanding experience in interactive systems,
Proceedings of DIS 2004, Cambridge, MA, 2004, 261268.
[15] J. Forlizzi, S. Ford, The Building Blocks of Experience. An Early Framework for
Interaction Designers, in Proceedings of the DIS 2000 seminar,
Communications of the ACM, 2000, 419423.
[16] J.J. Garret, The Elements of User Experience. User-Centered Design for the Web,
New Riders Publishing, USA, 2003.
[17] K. Barton, P. McKellar, From Master to Games Master: Managing
Disequilibrium and Scaffolding in Simulation-Based Learning, in: S. de
Freitas, P. Maharg (Eds.), Digital Games and Learning, Continuum Press,
London, 2011, pp. 226251.
[18] J. Nielsen, Usability Engineering, Academic Press, San Diego, London & San
Francisco, 1993.
[19] P. Lew, L. Olsina, L. Zhang, Quality, quality in use, actual usability and user
experience as key drivers for web application evaluation, LNCS 6189 (2010)
218232.
[20] J. McGrenere, W. Ho, Affordances: clarifying and evolving a concept,
Proceedings of Graphics Interface, 2000, Montreal, 2000.
[21] E.A. Skinner, M.J. Belmont, Motivation in the classroom: reciprocal effects of
teacher behaviour and student engagement across the school year, J. Educ.
Psychol. 85 (1993) 571581.
[22] W-H. Wu, W-B. Chiou, H-Y. Kao, C-H.A. Hu, S-H. Huang, Re-exploring gameassisted learning research: the perspective of learning theoretical base,
Comput. Educ. 59 (4) (2012) 11531161.
[23] T. Lainema, Perspective Making Constructivism as a Meaning Structure for
Simulation Gaming. Simulation & Gamin, Interdiscip. J. Theory Pract. Re. 40 (1)
(2009) 4867.
[24] C. Girvan, T. Savage, Identifying an appropriate pedagogy for virtual worlds: A
Communal Constructivism case study, Comput. Educ. 55 (1) (2010) 342349.
[25] H.-M. Huang, U. Rauch, S.-S. Liaw, Investigating learners attitudes toward
virtual reality learning environments: Based on a constructivist approach,
Comput. Educ. 55 (3) (2010) 11711182.
[26] Y.-H. Tao, C.-J. Cheng, S.-Y. Sun, What inuences college students to continue
using business simulation games? The Taiwan experience, Comput. Educ. 53
(3) (2009) 929939.
[27] P. Thavikulwat, S. Pillutla, A constructivist approach to designing business
simulations for strategic management, Simul. Gaming 41 (2) (2010) 208
230.
_
G. Kzlkaya, The effects of
[28] H. Tzn, M. Ylmaz-Soylu, T. Karakus, Y. Inal,
computer games on primary school students achievement and motivation in
geography learning, Comput. Educ. 52 (1) (2009) 6877.
[29] N. Vos, H. van der Meijden, E. Denessen, Effects of constructing versus playing
an educational game on student motivation and deep learning strategy use,
Comput. Educ. 56 (1) (2011) 127137.
[30] Y-J. Lee, Empowering teachers to create educational software: a constructivist
approach utilizing Etoys, pair programming and cognitive apprenticeship,
Comput. Educ. 56 (2) (2011) 527538.
[31] W. Kriz, A systemic-constructivist approach to the facilitation and debrieng
of simulations and games, Simul. Gaming 41 (5) (2008) 663680.
[32] R. Mayer, Should there be a three-strikes rule against pure discovery learning?,
Am Psychol. 59 (2004) 1419.
[33] D.C. Phillips, How, why, what, when, and where: perspectives on
constructivism in psychology and education, Issues in Education, 3 1997,
151194.
[34] J. Piaget, Construction of Reality in the Child, Routledge & Kegan Paul, London,
1957.
[35] L.S. Vygotsky, in: B.S.M. Cole, V. John-Steiner, S. Scribner, E. Souberman (Eds.),
Mind in Society: The Development of Higher Psychological Processes, Harvard
University Press, Cambridge, MA, 1978.

[36] P.A. Kirschner, J. Sweller, R.E. Clark, Why minimal guidance during instruction
does not work: An analysis of the failure of constructivist, discovery, problembased, experiential, and inquiry-based teaching, Educ. Psychol. 41 (2) (2006)
7586.
[37] F.G.W.C. Paas, A. Renkl, J. Sweller, Cognitive load theory: instructional
implications of the interaction between information structure and cognitive
architecture, Instr. Sci. 32 (1) (2003) 18.
[38] J. Sweller, J. Van Merrienboer, F. Paas, Cognitive architecture and instructional
design, Educ. Psychol. Rev. 10 (1998) 251296.
[39] R.E. Mayer, Multimedia Learning, Cambridge University Press, New York, 2001.
[40] M. Bannert, Managing cognitive loadrecent trends in cognitive load theory,
Learn. Instr. 12 (2002) 139146.
[41] P. Kirschner, Cognitive load theory: implications of cognitive load theory on
the design of learning, Learn. Instr. 12 (2002) 110.
[42] D.H. Jonassen, S. Land, Preface, in: Jonassen, Land (Eds.), Theoretical
Foundations of Learning Environments, Lawrence Erlbaum Associates, 2002,
pp. iiiix.
[43] D.L. Butler, P.H. Winne, Feedback as self-regulated learning: A theoretical
synthesis, Rev. Educ. Res. 65 (1995) 245281.
[44] J.J.G. Merrienboer, P.A. Kirschner, Ten Steps to Complex Learning: A Systematic
Approach to Four-Component Instructional Design, Lawrence Erlbaum
Associates, London, 2007.
[45] I. Dunwell, S. de Freitas, S. Jarvis, Four-dimensional consideration of feedback
in serious games, in: S. de Freitas, P. Maharg (Eds.), Digital Games and
Learning, Continuum Publishing, 2011, pp. 4262.
[46] H. Ketamo, M. Suominen, Learning-by-Teaching in Educational Games: In
proceedings of Ed-Media 2008, 30.6,4.7.2008, Vienna, Austria, 29542963.
[47] F. Bellotti, R. Berta, A. De Gloria, L. Primavera, Adaptive Experience Engine for
Serious Games, IEEE Trans. Comput. Intell. Games 1 (4) (2009) 264280.
[48] S. Arnab, P. Petridis, I. Dunwell, S. de Freitas, Enhancing learning in distributed
virtual worlds through touch: a browser-based architecture for haptic
interaction, in: M. Ma, A. Oikonomou, L.C. Jain (Eds.), Serious Games and
Edutainment Applications, Springer-Verlag, 2011.
[49] G. Miller, The magical number seven, plus or minus two: some limits on our
capacity for processing information, Psychol. Rev. 63 (1956) 8197.
[50] K. Kiili, On Educational Game Design: Building Blocks of Flow Experience.
Publications of Tampere University of Technology, no. 571. Doctoral thesis,
Tampere University of Technology, Finland (2005).
[51] T. Mayes, S. de Freitas, Learning and e-Learning: The role of theory, in: H.
Beetham, R. Sharpe (Eds.), Rethinking Pedagogy in the Digital Age, Routledge,
London, 2007.
[52] H.B. Amor, J. Murray, O. Obst, Fast, Neat and Under Control: Arbitrating
Between Steering Behaviors, in: S. Rabin (Ed.), AI Game Programming Wisdom
3, Charles River Media Inc., Massachusetts, 2006, pp. 221232.
[53] H. Ketamo, C. Nurmi, K. Kallama, User Generated AI in Sports Education: In
proceedings of ECGBL 2011, October 2011, Athens, Greece, 2011, pp. 2021.
[54] D. Panzoli, C. Peters, I. Dunwell, S. Sanchez, P. Petridis, A. Protopsaltis, V. Scesa,
S. de Freitas, Levels of Interaction: A User-Guided Experience in Large-Scale
Virtual Environments. In: IEEE 2nd International Conference in Games and
Virtual Worlds for Serious Applications, Braga, Portugal, March 26-27, IEEE,
2010, pp. 8790.
[55] S. Jackson, M. Csikszentmihalyi, Flow in Sports: The Keys to Optimal
Experiences and Performances, Human Kinetics, Champaign, IL, 1999.
[56] A.T. Duchowski, Eye Tracking Methodology Theory and Practice, 2 ed.,
Springer, 2007.
[57] H. Ketamo, K. Kiili, Conceptual change takes time: Game based learning cannot
be only supplementary amusement, J. Educ. Multimedia Hypermedia 19 (4)
(2010) 399419.
[58] C. Schrader, T.J. Bastiaens, The inuence of virtual presence. Effects on
experienced cognitive load and learning outcomes in educational computer
games, Comput. Hum. Behav. 28 (2) (2012) 648658.
[59] R.E. Mayer, R. Moreno, Nine ways to reduce cognitive load in multimedia
learning, Educ. Psychol. 38 (2003) 4352.
[60] T.M. Duffy, D.J. Cunningham, Constructivism: Implications for the design and
delivery of instruction, in: D.H. Jonassen (Ed.), Handbook of Research for
Educational Communication and Technology, Simon & Schuster/MacMillan,
New York, 1996, pp. 170198.
[61] J. Lave, E. Wenger, Situated Learning, Cambridge University Press, New York,
1991.
[62] R. Hmlinen, K. Oksanen, Challenge of supporting vocational learning:
Empowering collaboration in a scripted 3D game How does teachers realtime orchestration make a difference?, Comput Educ. 59 (2) (2012) 281293.
[63] L. Kobbe, A. Weinberger, P. Dillenbourg, A. Harrer, R. Hmlinen, P. Hkkinen,
et al., Specifying computer-supported collaboration scripts, Int. J. Comput.
Supported Collaborative Learn. 2 (2/3) (2007) 211224.
[64] S.M. Barnett, S.J. Ceci, When and where do we apply what we have learn? A
taxonomy for far transfer, Psychol. Bull. 128 (2002) 612637.
[65] K.E. DeLeeuw, R.E. Mayer, Cognitive consequences of making computer-based
learning activities more game-like, Comput. Hum. Behav. 27 (5) (2011) 2011
2016.
[66] E. Lehtinen, Information and communication technology in education: Desires,
promises, and obstacles, in: M. Watson, T. Downes (Eds.), Communications
and Networking in Education: Learning in a Networked Society, Kluwer,
Boston, 2000, pp. 311328.
[67] T. Lainema, S. Nurmi, Applying an authentic, dynamic learning environment in
real world business, Comput. Educ. 47 (1) (2006) 94115.

K. Kiili et al. / Entertainment Computing 5 (2014) 367377

[68] S. Arnab, P. Petridis, I. Dunwell, S. de Freitas, Enhancing learning in distributed
virtual worlds through touch: a browser-based architecture for haptic
interaction, in: M. Ma, A. Oikonomou, L.C. Jain (Eds.), Serious Games and
Edutainment Applications, Springer-Verlag, 2011. ISBN: 978-1-4471-2160-2.
[69] S. Warburton, Second Life in higher education: Assessing the potential for and
the barriers to deploying virtual words in learning and teaching, Br. J. Educ.
Technol. 40 (3) (2009) 414426.
[70] B. Parker, F. Myrick, A critical examination of high-delity human patient
simulation within the context of nursing pedagogy, Nurse Educ. Today 29 (3)
(2009) 322329.

377

[71] Y. Bereby-Meyer, S. Moran, E. Unger-Aviram, When performance goals deter

performance. Transfer of skills in integrative negotiations, J. Organizational
Behav. Hum. Decis. Processes 93 (2004) 142154.
[72] C. Ames, Classrooms: Goals, structures, and student motivation, J. Educ.
Psychol. 84 (1992) 261271.
[73] Y. Bereby-Meyer, A. Kaplan, The effect of motivational goals and age on the
ability to transfer knowledge, J. Contemp. Educ. 30 (2005) 122.
[74] T. Lainema, Enhancing organizational business process perception
Experiences from constructing and applying a dynamic business simulation
game, Turku School Econ. Series A-5 (2003) 1274.