PureEdgeSim: A Simulation Toolkit for

Performance Evaluation of Cloud, Fog, and Pure

Edge Computing Environments
Charafeddine Mechalikh Hajer Taktak Faouzi Moussa
University of Tunis El Manar, Faculty University of Tunis El Manar, Faculty University of Tunis El Manar, Faculty
of Sciences of Tunis, of Sciences of Tunis, of Sciences of Tunis,
LIPAH-LR11ES14, 2092 LIPAH-LR11ES14, 2092 LIPAH-LR11ES14, 2092
Tunis, Tunisia Tunis, Tunisia Tunis, Tunisia
[email protected] [email protected] [email protected]

Abstract—Fog and Edge Computing are two computing adopted strategies and their performance before their
paradigms that aim to solve the Cloud limitations by bringing deployment. This was the idea behind CloudSim which is a
its applications close to IoT devices at the edge of the network. rich and a highly extensible simulation framework that
As a result, they decrease both the latency and the Cloud enables the simulation and the performance evaluation of
workload. However, despite the research advancement in this Cloud resources and services [3]. CloudSim allows its users
field, there is still a lack of simulation tools, especially in the case to implement only the lacking part and to extend it depending
of Pure Edge Computing where heterogeneous and constrained on their needs, rather than implementing all the different
devices cooperate and share their resources in a peer to peer aspects of the simulation which requires a lot of time and
manner [9]. In this paper, we present PureEdgeSim, a simulation
effort.
toolkit that enables the simulation of resource management
strategies and the performance evaluation of Cloud, Fog, and Nevertheless, Fog and -especially- Edge Computing did
Pure Edge Computing environments in terms of delays, energy not get much attention when it comes to simulation, with only
consumption, network congestion, resource utilization, and a few simulators for Fog Computing such as iFogSim. This
tasks success rate as well. We also propose a load balancing simulator is designed to simulate IoT-Fog environments [4].
algorithm that is based on Fuzzy Decision Tree. It leverages It allows adding various component types (i.e. sensors,
from reinforcement learning that allows it to adapt to the IoT actuators, gateways, and the Cloud) and linking them as a
environmental changes and enables the IoT self-capabilities
topology. However, this topology remains static during the
[15]. Finally, we describe a case study that simulates an IoT
whole simulation process making this simulator lack mobility
environment in order to demonstrate PureEdgeSim capabilities
and to evaluate the performance of those computing paradigms
support. Also, modeling thousands of devices will require
and the impact of the adopted load balancing algorithm. The adding them, defining their specifications, and linking them
simulation results show the effectiveness of PureEdgeSim in one by one which is not convenient and a very time-
simulating Cloud, Fog, and Edge Computing environments. The consuming task. On top of that, its network model is
results also highlight the advantages of adopting the Pure Edge oversimplified where a fixed delay is assigned to each link,
Computing in terms of scalability and delays as compared to the ignoring the effect of the network load on the transmission
commonly used architectures that use the Fog and the Cloud. delay. EdgeCloudSim [10] is another simulator for Fog
On the other hand, the proposed algorithm has reduced the Computing environments that addresses the iFogSim
tasks completion delay by nearly 25.4%, the energy deficiencies. It ensures more scalability by generating mobile
consumption by 57.2%, and the failure rate by 39%, as devices automatically. It also supports the mobility of these
compared to the state of the art algorithm. devices and brings a realistic network model where the
Keywords—Simulation, Modeling, Edge Computing, Fog, bandwidth allocated for each device depends on the number
Cloud, Load Balancing, Internet of Things. of devices sharing that network. However, it still lacks the
specifications of the Cloud data centers, hosts, and virtual
I. INTRODUCTION machines. Besides, it lacks modeling the energy consumption
of edge devices, their death (i.e. out of energy), and the
With the emergence of the Internet of Things and due to
possibility of executing tasks locally on these edge devices or
the centralized nature of Cloud Computing and its remote
to offload them to other edge devices in a peer-to-peer
location, several challenges have risen to the top. Mainly the
manner.
scalability issue as the Cloud Computing is hardly able to
support the rapid growth of IoT devices which will reach 34 In this paper, we introduce PureEdgeSim, a simulator that
billion by 2020 [1]. Moreover, Cloud Computing could not enables the simulation of resource management strategies and
meet the requirements of IoT applications such as low the performance evaluation of Cloud, Fog, and Pure Edge
latency, and mobility. This introduced the need for a new Computing environments. It grantees high scalability by
paradigm that not only provides all the features of Cloud enabling the simulation of thousands of devices. It also
Computing but also, solves those issues. supports the edge devices heterogeneity (i.e. devices
mobility: mobile or fixed device, energy source: battery or
Fog and Edge Computing are such computing paradigms.
power supply, different latency requirements, etc.) and
They bring Cloud applications closer to IoT devices. As a
provides an even more realistic network model. It also
result, the data generated from these devices can be processed
provides a tasks orchestrator module that manages the
directly at the edge of the network which reduces the Cloud
available resources, balances the workload, and enables the
workload and leads to a more scalable network. In such
simulation of multi-tiers scenarios where many computing
complex environments (i.e. Cloud, Fog, and Edge
paradigms can be used in conjunction with one another.
Computing), the simulation makes it possible to evaluate the

978-1-7281-4484-9/19/$31.00 ©2019 IEEE 700

Authorized licensed use limited to: ULAKBIM UASL - Lefke Avrupa Universitesi. Downloaded on November 17,2023 at 07:15:34 UTC from IEEE Xplore. Restrictions apply.
 In order to demonstrate PureEdgeSim capabilities, a
simulation scenario that simulates the IoT environment is High Cloud Cloud
proposed, by which we evaluate the performance of the latency
aforementioned paradigms and the impact of the adopted load Offload tasks
balancing algorithm. Moreover, to demonstrate the to the Cloud Fog node
possibilities offered by PureEdgeSim, we propose a load Offload to
balancing algorithm that is based on Fuzzy Decision Tree and edge
leverages from reinforcement learning. This algorithm is Offload to Fog device
designed to enable the self-capabilities of IoT by adapting to
the environmental changes and to provide better support for Low
the devices heterogeneity and meet their quality of service latency
requirements. The simulation results clearly show the (a) Cloud (b) Fog (c) Edge
capabilities of PureEdgeSim in simulating such complex Computing Computing Computing
scenarios and its possibilities. The results also highlight the
Fig. 1. Cloud, Fog, and Edge Computing architectures.
advantages of adopting the Pure Edge Computing, and the
effectiveness of the proposed algorithm that outperformed the Fog and Edge Computing use the same tasks offloading
state of the art solution [11] in every aspect of the approach. In [6], the authors have proposed a task offloading
comparison: It reduced the tasks completion delay by 25.4%, mechanism that offloads the tasks between mobile devices in
the energy consumption of edge devices by 57.2%, and the order to balance their energy consumption. As a result, the
tasks failure rate by 39%. network life has increased by 400%. A related system was
developed in [7] called Serendipity system. It enables mobile
This paper is organized as follows: Section II presents the devices to remotely access resources on other devices to
background of Fog and Edge Computing. Section III execute their applications which minimizes the local power
describes PureEdgeSim architecture. In section IV and V, a consumption while decreasing the overall computation
use case scenario is introduced, a fuzzy decision tree based completion time. This approach resulted in a tasks
load balancing algorithm is proposed, and the simulation completion time that is up to 6.6 times faster.
results are evaluated and discussed. Finally, section VI
concludes the paper and describes future directions. A. Mukherjee et al. [16] introduced a framework that uses
the smart devices available at the edge such as mobile phones,
II. BACKGROUND: FOG AND EDGE COMPUTING gateways, etc. as heterogeneous computing nodes to perform
. The definition of Edge Computing varies from paper to data analytics in IoT applications. To do so, capacity-based
another. Although the majority of papers has considered Edge data partitioning was introduced where data is partitioned
and Fog Computing as the same paradigm [8] [10] [12] [13], according to the capacities of the available edge devices.
some others have differentiated them. In [13], Edge Although the performance has decreased when using edge
Computing is defined by the exclusion of the Cloud, while devices, the Cloud workload has also decreased which can
Fog Computing works jointly with the Cloud. On the other solve the Cloud scalability challenge. In [11] an energy-
hand, according to [9], -which is the definition that we aware offloading algorithm was proposed. This algorithm
adopted- the main difference between Fog and the Pure Edge allows mobile devices to dynamically chose the Cloud or the
Computing is their locations. In the Pure Edge Computing Fog to offload their tasks depending on the delay tolerance
[9], edge nodes are deployed in the edge devices themselves and the energy consumption. The results show that this
following peer-to-peer architecture (Figure 1), unlike Fog approach performs better than Cloud-only or Fog-only
Computing, where the Fog nodes are deployed on servers, strategies. Authors in [17] proposed a platform that
mini-Clouds, etc. Table I summarizes the main differences orchestrates the workload between IoT gateways, Fog
between the Edge, Fog and Cloud Computing. servers, and the global Cloud based on the availability of
resources. In [18], V. Chamola et al. focused on reducing
TABLE I. A COMPARISON OF THE DIFFERENT COMPUTING latency in Mobile Edge Computing. By searching for the best
PARADIGMS [5] [9]. Fog node to execute the tasks when the nearest one is
overloaded. This algorithm achieved a very low latency as
Cloud Fog Edge
compared to the traditional scheme which uses only the
Latency High Low Very Low
nearest node.
Location Within the Close to the The ingress of the
internet network edge network Although, Pure Edge Computing gained a lot of interest
Network access Multi-hop Few hops Single-hop [6][7][11][16], there is always a lack of simulation tools. To
Resources Vertical Vertical and Horizontal the best of our knowledge, there is no simulation tool capable
scaling Horizontal
of modeling Pure Edge Computing environments (which
Network style Client-server Client-server Peer to peer
includes offloading the tasks between edge devices and
Architecture Centralized Distributed Distributed
balancing their load, simulating their heterogeneity, and
Number of Few (100- Large (dozens Very large
nodes 1000) of thousands) (billions) measuring their energy consumption and their resources
utilization). To address this, we present PureEdgeSim, a
In conventional Cloud Computing, the devices at the edge simulation toolkit that is designed to simulate Cloud, Fog,
of the network offload their tasks to the Cloud in order to be and Pure Edge Computing environments. Thus, enabling a
processed. This tasks offloading can be necessary for several realistic simulation of multitude of scenarios such as Mobile
reasons: some devices offload their tasks due to their low Devices Clouds [6][7], VANET clouds, Internet of Things
computational capabilities, devices with capacity-limited [16], Mobile Edge Computing [18], and multi-tiers scenarios
batteries need to offload their tasks to extend their lives, and where different computing paradigms are used (such as
so on.

701
Authorized licensed use limited to: ULAKBIM UASL - Lefke Avrupa Universitesi. Downloaded on November 17,2023 at 07:15:34 UTC from IEEE Xplore. Restrictions apply.
 Foggy [17]), just to name a few. Basically, any scenario that In order to ensure higher extensibility, wider
involves geographically distributed or mobile devices could applicability, and better code reusability, PureEdgeSim
be simulated using PureEdgeSim. provides a modular architecture where each module deals
with a specific aspect of the simulation of Cloud, Fog, and
III. PUREEDGESIM ARCHITECTURE Pure Edge Computing environments. It consists of seven
The simulation of Fog and Edge Computing modules (Figure 3):
environments allows evaluating the adopted resources a) Simulation Manager: This module represents the
management strategies before their actual deployment. core of the simulation. It is responsible for initiating and
However, the heterogeneity of possible scenarios complicates managing the simulation environment, scheduling events,
the task, especially when using a simulator such as Omnet++ and generating the output files. It represents the interface
or NS-3 where the user has to define all the aspects of the between CloudSim Plus and the other PureEdgeSim modules.
It contains two main classes: 1) the Simulation Manager class
simulation from the specification of devices resources,
which initializes the simulation, starts it, and schedules its
networking, energy, mobility, etc. which requires a lot of
end. It also schedules the tasks generation (i.e. when each task
time and effort. PureEdgeSim (Pure Edge Computing will be generated and sent to the Orchestrator), shows the
Simulator) takes advantages of CloudSim Plus features simulation progress, and prints the results. 2) The Simulation
including the native support for the discrete events simulation Logger class that is responsible for the simulation output. It
which is used during the communication between its computes the results, displays them at the end of every
components. This makes the simulation time complexity iteration, and saves them in a comma-separated value (CSV)
depend on the number of events created during the format in order to easily exploit them later using a
simulation. To reduce the simulation duration, PureEdgeSim spreadsheet editor (e.g. Microsoft Excel, OpenOffice Calc).
offers its users quick and full control of the simulation b) Data Centers Manager: Edge and Fog Computing
environment through its set of parameters, where the users simulators need to be easily extended with new types of
can trade-off between the realism of the simulation and its devices without modifying their internals [14]. Besides, each
duration. Thus, the realism will depend on the user application scenario (smart cities, connected vehicles, etc.)
parameters especially the intervals for the CPU utilization may require different types of devices (in terms of mobility,
update, the energy consumption update, and the network energy source, etc.). This is ensured by the Data Centers
update (Table III). The smaller are these intervals, the more Manager module. This module manages all the data centers
accurate and realistic will be the simulation, and the longer it and devices (i.e. Cloud, Fog or Edge). It consists of two main
will take. PureEdgeSim also leverages from CloudSim Plus classes: 1) The Edge Data Center class that is inherited from
native support for parallel simulations that makes use of all the Datacenter Simple class of CloudSim Plus. It contains the
the available CPU cores. It also takes advantages from its rich specific properties of edge devices and their corresponding
and very extensible library that covers all the aspects of Cloud methods such as the location, the mobility (i.e. whether a
Computing from resources (i.e. data centers, hosts, etc.) to device is mobile or not), the energy source (i.e. battery or
services (i.e. virtual machine allocation policies, CPU electrical outlet), and the battery capacity if the device is
battery-powered, which enables to model the heterogeneity
schedulers, etc.). Hence, only a few classes were added in
of devices. 2) The Server Manager that generates the
order to model and simulate the Fog and the Pure Edge different servers and devices, their hosts, and their virtual
Computing environments. Therefore, the CloudSim Plus machines according to the configuration files (i.e. input files
layer is responsible for providing key components that are that describe the data centers and edge devices
extended by PureEdgeSim (Figure 2) and it is also behind the characteristics).
interactions of PureEdgeSim components that are represented
in Figures 3. Main Application Scenario Location
Manager Manager
Edge Device Task generation Location Manager
Simulation
Tasks Orchestration
Manager
Data Tasks Server Energy CPU utilization
Center orchestration manager model model
Tasks Generator
Network Network Model Data Centers
Manager
Simulation Network Module
Simulation Scenario = <<uses>>
Core Logger
Manager Manager Fig. 3. The relations between PureEdgeSim modules.

PureEdgeSim c) Tasks Generator: This module is responsible for

generating the tasks. By default and depending on the
scenario, the Task Generator assigns an application such as
Cloud Resources Cloud Services e-health, smart-home, and augmented-reality, to each edge
device. Each application has its specific characteristics that
CloudSim Plus can be defined in the applications input file (i.e. task size,
Fig. 2. The PureEdgeSim layered architecture. CPU utilization, latency requirement, etc.). Then, according
to the assigned application type and the tasks generation rate

702
Authorized licensed use limited to: ULAKBIM UASL - Lefke Avrupa Universitesi. Downloaded on November 17,2023 at 07:15:34 UTC from IEEE Xplore. Restrictions apply.
 that is defined by the user, the Task Generator will generate IV. APPLICATION AND CASE STUDY
all the tasks that will be offloaded by each device during the To demonstrate the effectiveness of PureEdgeSim and its
simulation. However, depending on their needs,
capabilities in simulating and modeling complex scenarios,
PureEdgeSim users can easily extend the Task Generator
class and create a custom generator or a more realistic one. we propose a simulation scenario that imitates IoT
environments (Figure 4), where a huge number of
d) Location Manager: Thanks to their geographical heterogeneous devices run different applications with
distribution [5], Fog and Edge computing can serve mobile different quality of service requirements. The goal here is to
devices during their mobility while guaranteeing the achieve the highest rate of successfully executed tasks
minimum latency which gave birth to a new breed of latency-
sensitive applications such as the connected vehicles. This High latency
module enables that by assigning an initial location to each The cloud
Cloud
data
one of them and by managing their mobility. It contains two The orchestrator centers
main classes: The Location class which represents the X and
Y coordinate of the device, and the Mobility Manager class Fog server Fog
Fog server
that generates the mobility path of mobile devices. servers
e) The Network Module: Consists primarily of the Tablet Smartphone
Network Model. Unlike in Cloudsim plus (same for IoT gateway Edge
PC
CloudSim), where the bandwidth allocated to each virtual devices
Device metadata Offload the task
machine remains static, this network model takes into
account the network load at each instant of the simulation and IoT
sensors
changes the allocated bandwidth for each task being Low latency
transferred accordingly. Therefore, when transferring a task, Fig. 4. A simple representation of the simulation scenario.
at each instant (i.e. from the beginning until the end of the
transfer), its allocated bandwidth will vary according to the The different resources (i.e. Cloud, Fog, and Edge
total number of tasks being transferred using that network at resources) will be generated according to the proportions
that moment. This network model also takes into account the already mentioned at the end of Table I, with the edge devices
bandwidth limit caused by the WAN or LAN congestion. As having low computing capabilities as compared to the Cloud
a result, if multiple devices connect to the same LAN access and the Fog virtual machines. Therefore, the Cloud will be
point, the bandwidth allocated to each device decreases. If represented by one data center, while the Fog will have 8
this allocated bandwidth is less than that of the WAN, the geographically distributed data centers. Each data center will
transfer speed of the tasks offloaded to the Cloud will be have 4 virtual machines that have a computing capacity of
limited by the low bandwidth of the LAN and not by that of 2000 Mips. On the other hand, the number of edge devices
the WAN. will increase from 100 devices to 1000 devices in order to
a) The Tasks Orchestration Module: In order to allow its evaluate the scalability of each paradigm. To model their
wider applicability PureEdgeSim enables the implementation heterogeneity, we propose four types of edge devices as
of custom resource management strategies. This is done via described in Table II (where Type 4 refers to simple sensors
the Tasks Orchestration module. It consists of the that only generate data and have no computing capabilities).
Orchestrator which represents the decision maker. It decides
whether to offload the task or execute it locally and where to TABLE II. EDGE DEVICES TYPES
offload it, depending on the orchestration (i.e.load balancing) Edge devices types Type 1 Type 2 Type 3 Type 4
algorithm and the architecture that are used. Users can easily Mobility No Yes No No
implement their own orchestration policies by extending the Battery-powered Yes Yes No No
Orchestrator class, which reflects PureEdgeSim flexibility Percentage of devices 30 20 20 30
(%)
and high extensibility. Battery-capacity (Wh) 15.75 20 - -
b) Scenario Manager: To make implementing the Idle energy consumption 0.00055 0.00055 0.00055 0.00055
simulation scenarios easier and effortless, each of the rate (Wh/s)
previous modules provide a set of adjustable parameters Max energy 0.055 0.055 0.055 0.055
consumption rate (Wh/s)
(Table III). The Scenario Manager loads those parameters CPU (MIPS) 400 400 800 -
from the input files. It loads the applications characteristics Ram (MByte) 800 500 1500 -
that will be used by the Tasks Generator. It also loads the Storage (MByte) 10000 2000 10000 -
Cloud data centers, the Fog data centers, and the edge devices
characteristics that are needed by the Data Centers Manager We introduce four types of applications: 1) Smart home,
from their corresponding files. Finally, it loads the network which is neither a latency-sensitive application nor a
settings used by the Network Module, the architectures and compute-intensive task. 2) E-Health, which is a very latency-
the algorithms used by the Tasks Orchestration Module, and sensitive application. 3) Augmented reality, which is both a
other simulation parameters including simulation delay, log latency-sensitive and a compute-intensive task. 4) Finally, a
parameters, etc. which are required by the Simulation very compute-intensive application that is not sensitive to
Manager. It contains two important classes, the File Parser delay. The Tasks Generator will assign one of these
that loads the user scenario from the configuration files, and applications to each edge device. Then, each device will
the Simulation Parameters class which acts as a placeholder generate its tasks according to the assigned application type
for those parameters. and will offload them.

703
Authorized licensed use limited to: ULAKBIM UASL - Lefke Avrupa Universitesi. Downloaded on November 17,2023 at 07:15:34 UTC from IEEE Xplore. Restrictions apply.
 A. The Evaluated Architectures uncertainties and the imprecise values. Besides, this will
exponentially reduce the size of the decision tree by replacing
Currently, the Orchestrator will be deployed on the
the values belonging to the same interval by one linguistic
default location used by PureEdgeSim (on each wireless
access point). To orchestrate the tasks, it will use one of the term. So, instead of creating a decision rule for each value
following architectures: 1) Cloud-Only: The tasks will be (e.g. creating a rule for each battery percentage from 1% to
offloaded to the Cloud. 2) Fog-and-Cloud: The tasks will be 100%), the rules will consist of the aforementioned linguistic
offloaded to the Fog or to the Cloud. 3) All: The idea here is terms. The crisp values of the available resources and the
to leverage from nearby devices that have sufficient offloading request values will be all fuzzified during this
computing capacity (i.e. PC, smartphone, tablet, smart TV, phase as shown in Figure 5.
etc.) as shown in Figure 4. Even though these devices offer b) The classification phase: The Orchestrator will build
the lowest computing capacity, their power lies in their large the decision tree from a predefined set of rules and will
number that is expected to reach 10 billion devices by 2020 classify all the resources (i.e. the fuzzification results). This
(excluding IoT devices) [1]. When a device joins the network, classification will consider the following criteria: The latency
it communicates its meta-data (Figure 4) including the sensitivity of the task, the latency of the offloading
latency requirement of its application, its resources, and its destination (i.e. the resource), the mobility of both the device
remaining energy with the orchestrator. The Orchestrator offloading the task and the offloading destination, the
will then add that device to its list of resources, in order to
remaining energy of both of them, their computing capacity,
exploit it later when offloading a task. In this case, edge
and their resource utilization (Figure 5). It will result in three
devices can offload their tasks to the Cloud, Fog, other nearby
edge devices, or just execute them locally. Finally, 4) Edge- possible classes “high”, “medium”, and “low” that refers to
Only: The Fog and the Cloud are totally absent. Thus, the the offloading suitability. Therefore, the resources classified
devices can offload their tasks to other nearby devices or as “high” are more suitable for the offloaded task.
execute them locally. c) The offloading phase: After classifying the resources,
the Orchestrator will select the class with the highest
B. The Tasks Orchestration Algorithms suitability (i.e. the “high” class, or the “medium” one in case
The IoT network topology is subject to frequent changes. there is no “high” one, and so on). Then, it will choose the
It may change due to the growth in the number of devices, optimal offloading location among the resources of that class
their mobility, or due to the death of some devices when they (i.e. with minimum resource utilization and energy
run out of energy. The Orchestrator should consider this and consumption) and will offload the task.
adapt to these changes. To enable this, we propose a tasks Algorithm 1 Finding the offloading destination
orchestration algorithm (Algorithm 1) that is based on the
fuzzy decision tree. Fuzzy decision trees enable the learning 1. procedure findDestination(request, resources)
2. level← null
from feature-based data sets. They leverage from the high 3. minCost← -1
comprehensibility of decision trees and the ability of fuzzy 4. tuple, selectedTuple← null
representation to deal with uncertain or imprecise values that 5. decisionTree←initDecisionTree(dataSetFile)
are caused by noise or by errors in the measurements. This 6. fuzzificationResults ← Fuzzification(request, resources)
algorithm also uses reinforcement learning that enables it to 7. classes← decisionTree.classify(fuzzificationResults)
learn and generate its decision rules according to the scenario, 8. for each tuple ∈ getHighestClass(classes) do
9. newCost←getCost(getResourceID(tuple))
which enables it to continuously adapt to the environmental 10. if (min= -1 or newCost < minCost) //find the optimal
changes. It consists of four phases: location
11. minCost ← newCost
a) The fuzzification phase: After receiving a task 12. selectedTuple ← tuple
offloading request, before classifying the resources. The 13. end if
Orchestrator will launch the fuzzification phase, in which the 14. end for
crisp values (i.e. the energy, computing capacity, etc.) will be 15. addToHistory(getTaskID(request), selectedTuple)
represented by linguistic terms “high”, “medium”, or “low”. 16. offload(request, getResourceID(selectedTuple))
By doing so, the Orchestrator will be able to deal with 17. end
d) Reinforcement learning phase: In order to
continuously adapt to the unpredictable changes in the
resources status and their workloads, to meet the IoT need for
high degree of autonomy, and to provide better support the
heterogeneity of IoT devices, the decision tree needs to be
updated frequently. The decision tree that was generated for
10 devices may not satisfy the needs of 20 devices which
requires the Orchestrator to change it continuously. Once the
task is offloaded and executed, the device will reward the
Orchestrator if its task has been executed successfully or
punish it otherwise. The Orchestrator will then update the
decision tree in order to improve its classification, either by
increasing the suitability on the corresponding decision rule
or by decreasing it otherwise (i.e. to “high” if its reward-to-
punishment rate exceeded 98%). By doing so, it will avoid
Fig. 5. The fuzzification phase. the resources that are not suitable for that type of tasks and

704
Authorized licensed use limited to: ULAKBIM UASL - Lefke Avrupa Universitesi. Downloaded on November 17,2023 at 07:15:34 UTC from IEEE Xplore. Restrictions apply.
 devices (i.e. low success rate) which results in a higher architectures have given the lowest tasks execution delay.
adaptability and better quality of service. It will also add the Whereas, the average tasks execution delay has increased
decision rules that were missing. remarkably (more than 3 seconds) when using the Edge
Computing, even if it is used in conjunction with the Fog and
In order to evaluate each one of them, the following the Cloud (i.e. the All architecture), due to the very large
algorithms will be used by the Orchestrator: 1) Round-Robin, number of IoT devices and their relatively low computing
where it will run through the whole list of resources and capabilities. Hence, more tasks are executed on the edge
choose them one by one successively. This algorithm does devices which increases the average execution delay,
not prioritize any resource over the others. Thus, it will result especially when using a strategy that does not prioritize any
in the performance of the used paradigms rather than that of resource over the others (Round-Robin).
the algorithm, which we believe is the best way to compare
the different architectures. 2) FDT, that represents the However, when the number of devices increases the
proposed algorithm which is based on Fuzzy Decision Tree. situation changes. The tasks completion time rose sharply
Finally, 3) The state of the art solution SA, which is jointly starting from 400 devices in the Cloud-Only case and from
minimizing the energy, the latency, and resource utilization 800 devices when using the Fog with the Cloud. While in the
[11]. Table III resumes the simulation parameters. other two cases (Edge-Only and All architectures), it
remained steady and almost non-existent. This is not
TABLE III. SIMULATION PARAMETERS surprising as the growth in the number of devices means the
Parameter Value
increase in number of resources (i.e. the horizontal
Simulation duration 30 (min) scalability, Table I), unlike in the Cloud and Fog where the
Save log file true increased number of devices will contribute in increasing
CPU utilization and energy 0.1 (s) their workloads. Thus, making the network lack computing
consumption update interval resources and leading to high waiting delays.
Min number of edge devices 100
Max number of edge devices 1000 30 100

Average WAN usage (s)

Average CPU usage (%)

Edge devices counter size 100 25 80
Tasks generation rate 2 (Tasks/device/min) 20
Simulation Area 400x400 (meters) 60
15
Mobility interval [10 (s)-300 (s)] 40
Wlan bandwidth 200 (Mbits/s) 10
Wan bandwidth 30 (Mbits/s) 5 20
Wan propagation delay 0.1 (s) 0 0
Network update interval 0.1 (s) 100 300 500 700 900 100 300 500 700 900
Architectures Cloud-Only, Fog-and-Cloud, All, Edge devices count Edge devices count
Edge-Only. Cloud-Only Cloud-Only
Orchestrator deployment Default Fog-And-Cloud Fog-And-Cloud
Orchestration algorithms Round-Robin, SA, FDT. All All
Applications E-Health, Smart-Home, Augmented (a) Average WAN usage per (b) Average cloud VMs cpu
Reality, compute-intensive task usage
application. Fig. 7. The average Cloud resources usage (Round-Robin).
V. SIMULATION RESULTS AND DISCUSSION Figure 7 describes furthermore the scalability challenge
As proof of concept, in this section, we will evaluate the of the conventional Cloud Computing paradigm, in terms of
performance of the different computing paradigms based on network utilization and computation. When offloading all the
the obtained simulation results. tasks to the Cloud and by increasing the number of devices,
the average transfer delay has increased from an average of 5
In this simulation scenario, various computing paradigms seconds per task at 100 devices to 26 seconds at 1000 devices
have been used in different combinations with a growing which reflects the network congestion. By using the Fog in
number of edge devices ranging from 100 to 1000 devices for conjunction with the Cloud (Fog-and-Cloud architecture), at
the purpose of evaluating the scalability, the strengths, and first glance, it appears that the Fog has fixed the issue as the
the weak points of each one of those paradigms. However, we WAN usage remained almost stable with just a 0.8 seconds
will focus more on the role of Edge Computing which can be difference between the delay at 100 devices and the delay at
only evaluated by PureEdgeSim. 1000 devices. However, the curve is still rising, especially
when comparing it to that of the All architecture that uses
Edge Computing along with the Fog and the Cloud. In this
Average waiting delay (s)

1024
case, by leveraging from the huge number of devices and
256 their distributed nature, not only, the WAN utilization
64 remains almost constant and negligible in comparison with
the previous architectures for any number of devices, but
16 also, it decreased from an average of 0.12 seconds at 100
4 devices to 0.03 seconds at 1000 devices.
100 300 500 700 900
Edge devices count
Similarly, Figure 7.b exhibits the effect of the growth in
Cloud-Only Edge-Only Fog-And-Cloud All
IoT devices on the utilization of Cloud resources for the
aforementioned architectures. When using the Cloud only,
Fig. 6. The average tasks completion delay (Round-Robin). starting from 400 devices, the resources operated at about
In Figure 6, the average tasks completion delays for the 100% almost the whole time. This reflects its huge workload
different architectures are given. Due to their high computing and its lack for computation resources, and justifies the high
capabilities, the Cloud-Only and the Fog-and-Cloud tasks completion delay that suddenly increased at 400 devices

705
Authorized licensed use limited to: ULAKBIM UASL - Lefke Avrupa Universitesi. Downloaded on November 17,2023 at 07:15:34 UTC from IEEE Xplore. Restrictions apply.
 (Figure 6). Although it is not remarkable, the Cloud resources reference), two other algorithms were used in order to analyze
utilization increased by approximately 2% when both Fog their effect on the performance of the whole system: the
and Cloud are in use, while the utilization of Fog resources proposed FDT algorithm and SA which refers to the state of
rose by nearly 80%. This means that the Fog will start the art solution [11].
suffering from the same lack of computing resources at 1200
devices. To overcome this problem, it will need to offload the Figure 9.a shows the number of battery-powered devices
tasks surplus to the Cloud. Thus, increasing the Cloud that remained alive at each instant of the simulation. The
workload again. On the other hand, when using Edge Round-Robin performed the worst; the first device died after
Computing along with the Cloud and the Fog, the Cloud 738 seconds and an average remaining energy of 8.3% for the
resources remained almost idle with an average utilization 10 devices that remained alive at the end of the simulation.
that does not even exceed the 0.3% for any number devices, The SA algorithm performed better than Round-Robin as it
resulting in a more scalable network that supports up to checks the resource utilization before offloading. Therefore,
thousands times the current number of devices. due to the high computing capacity and the low latency of
Fog Computing, it is selected more often than the edge
100
devices (Figure 9.b). As a result, the first death was after 1133
Successfully executed tasks (%)

80 seconds, and an average remaining energy of 16.5% for the

305 devices that remained alive at the end of the simulation.
60
But, still, this death of devices caused the failure of 1882
40 tasks (3.13% of all tasks). On the other hand, no device died
20
when using the proposed FDT algorithm, with an average
remaining energy of 58% at the end of the simulation.
0
100 200 300 400 500 600 700 800 900 1000 Figure 9.b gives the average CPU usage for the different
Edge devices count paradigms using those three algorithms. Unlike Round-
Cloud-Only Edge-Only Fog-And-Cloud All Robin, the SA strategy relies more on Fog resources as they
meet better its criteria (i.e. low latency, energy, and
Fig. 8. Tasks successfully executed (Round-Robin). resources). This results in offloading more tasks to the Fog.
Figure 8, supports the obtained results and links between Then, if Fog resources are unavailable it searches for the
the aforementioned scalability problem and the tasks success optimal between the Cloud and the Edge. This solution has
rate. The latter dropped steeply from 58.9% at 400 devices to managed to increase the network lifetime by 53.5% (from 738
7.6% at 600 devices when using the Cloud-Only architecture to 1133 seconds). But still, it is not suitable as it does not
due to the high tasks transfer and completion delays (Figure consider the latency requirements of the tasks and the nature
6 and 7) that are caused by the network congestion and the of edge devices.
Cloud overload. The use of Fog along with the Cloud has only 6 100
reduced the drop rate from 78% at 100 devices down to

Energy consumed (%)

5
44.9% at 1000 devices, but it is still considered as a huge 80
4
drop. Whereas, the use of Edge Computing, whether
Delay (s)

60
standalone or with the Fog and the Cloud gives the best 3
40
success rate that remained stable (near 60%) for any number 2

of devices. 1 20

0 0
Although it outperformed the Fog and the Cloud, a lot of 100 300 500 700 900 100 300 500 700 900
tasks (around 40%) has failed either due to the high execution Edge devices count Edge devices count
delay or due to the death of devices after running out of FDT SA FDT SA
energy. This can be solved by using the appropriate load (b) Energy consumption of
(a) Task completion delay
balancing algorithm that takes into account the heterogeneity battery-powered devices
of devices and their quality of service requirements. Fig. 10. The average tasks completion delay and the average consumed
PureEdgeSim supports such scenarios, by allowing the users energy (Architecture =All).
to implement their own algorithms. To demonstrate that, in As opposed to the SA algorithm, the proposed FDT
Figure 9, along with Round-Robin (that is used as a algorithm provides better support for the devices
heterogeneity and their quality of service requirements.
500 70
Thanks to the implemented reinforcement learning approach,
Remaining edge devices

Average CPU usage (%)

60
400 this algorithm avoids as much as possible the resources with
50
300
a lower success rate (under 98%). It also avoids offloading
40
the tasks to the battery-powered and mobile devices, and only
30
200
offloads to them when there is no other choice. Edge
20
100 Computing does not support mobility as much as Fog
10
Computing do, as Fog servers are static and have a wider
0 0
0 300 600 900 1200 1500 1800 Average Average Average coverage area. The proposed algorithm takes that into
Time (s)
Cloud CPU Fog CPU Edge CPU account by offloading the tasks of mobile devices either to
Usage Usage Usage
SA the Cloud or to the Fog depending on the latency-sensitivity.
Round-Robin The Edge will only be selected if the resources of the Fog or
FDT SA FDT Round-Robin
the Cloud are not suitable (e.g. busy). But even in this case,
(a) The edge devices (b) The average CPU
remaining alive utilisation
mobile devices will be avoided as much as possible. Which
justifies why the Fog and the Cloud utilization are higher as
Fig. 9. The devices death time and the resources utilization (Architecture
=All, Edge Devices Count=1000). compared to the other two algorithms (Figure 9.b). However,

706
Authorized licensed use limited to: ULAKBIM UASL - Lefke Avrupa Universitesi. Downloaded on November 17,2023 at 07:15:34 UTC from IEEE Xplore. Restrictions apply.
 although it is higher, the proposed algorithm achieved a better Internet of Things and the development of novel resource
quality of service: the completion delay has been reduced by management strategies.
25.4%, the number of tasks that are failed due to high delays
has been decreased by 34%, the energy consumption of edge SOFTWARE AVAILABILITY
devices has been reduced by 57.2%, and the number of tasks The PureEdgeSim simulator is available for download at
failed due to mobility has been decreased by 44%, as https://github.com/CharafeddineMechalikh/PureEdgeSim/
compared to the State-of-the-Art algorithm (Figure 10 and
11). REFERENCES
1500 500 [1] Camhi, J. (2015). BI intelligence projects 34 billion devices will be
Number of failed tasks

connected by 2020. Business Insider, 6.

Number of failed tasks

1200 400
[2] Silva Filho, M. C., et al. (2017). CloudSim plus: a Cloud Computing
900 300 simulation framework pursuing software engineering principles for
improved modularity, extensibility and correctness. In Integrated
600 200
Network and Service Management (IM), 2017 IFIP/IEEE Symposium
300 100 on (pp. 400-406). IEEE.
[3] Calheiros, R. N., et al. (2011). CloudSim: a toolkit for modeling and
0 0 simulation of Cloud Computing environments and evaluation of
100 300 500 700 900 100 300 500 700 900
resource provisioning algorithms. Software: Practice and experience,
Edge devices count Edge devices count 41(1), 23-50.
FDT SA FDT SA
[4] Gupta, H., et al. (2017). iFogSim: A toolkit for modeling and
(a) Tasks failed due to high (b) Tasks failed due to devices simulation of resource management techniques in the Internet of
delay mobility Things, Edge and Fog Computing environments. Software: Practice
Fig. 11. The tasks failure rate (Architecture =All). and Experience, 47(9), 1275-1296.
[5] Bonomi, F., et al. (2012). Fog Computing and its role in the internet of
things. In Proceedings of the first edition of the MCC workshop on
VI. CONCLUSION Mobile Cloud Computing (pp. 13-16). ACM.
Fog and Edge Computing are two emerging computing [6] Mtibaa, A., et al. (2013). Towards resource sharing in mobile device
paradigms that bring resources and applications close to the Clouds: Power balancing across mobile devices. In ACM SIGCOMM
edge of the network. Thus, decreasing both the latency and the Computer Communication Review (Vol. 43, No. 4, pp. 51-56). ACM.
load of the Cloud, enabling IoT applications to meet their [7] Shi, C., et al.. (2012). Serendipity: enabling remote computing among
intermittently connected mobile devices. In Proceedings of the
quality of service requirements, and leading to a more scalable thirteenth ACM international symposium on Mobile Ad Hoc
network. In this paper, we introduced PureEdgeSim, a Networking and Computing (pp. 145-154). ACM.
simulation toolkit designed to simulate the Cloud, Fog, and [8] Marín-Tordera, et al. (2017). Do we all really know what a Fog node
Pure Edge Computing environments and their resources is? Current trends towards an open definition. Computer
management strategies, and to evaluate their performances. Communications, 109, 117-130.
[9] D’Angelo, M., et al. (2016). Pure Edge Computing Platform for the
To conclude, not only Fog Computing was delaying the Future Internet. In Federation of International Conferences on
appearance of the Cloud scalability problem rather than Software Technologies: Applications and Foundations (pp. 458-469).
solving it definitively, but also, it is suffering from that same Springer, Cham.
issue. This requires continuously scaling up its resources (i.e. [10] Sonmez, C., et al. (2017, May). EdgeCloudSim: An environment for
a cost-prohibitive solution) in order to catch up with the fast performance evaluation of Edge Computing systems. In Fog and
Mobile Edge Computing (FMEC), 2017 Second International
growth of IoT devices. On the other hand, even though it Conference on (pp. 39-44). IEEE.
seems to be underrated, Edge Computing represents a cost-
[11] Zhao, X., et al. (2016, July). An Energy Consumption Oriented
effective alternative that makes the growth in the number of Offloading Algorithm for Fog Computing. In International Conference
connected devices on its favor. It has outperformed both the on Heterogeneous Networking for Quality, Reliability, Security and
Cloud and the Fog, thanks to its horizontal scalability. With Robustness (pp. 293-301). Springer, Cham.
a good pricing model, the latter could easily help to overcome [12] Qaisar, S., et al. (2016). Fog Networking: An Enabler for Next
the Cloud limitations. Nevertheless, due to the heterogeneity Generation Internet of Things. In International Conference on
of IoT devices, conventional optimization techniques are not Computational Science and Its Applications (pp. 353-365). Springer,
Cham.
sufficient. We need to take into account other aspects such as
[13] Ai, Y., et al. (2018). Edge Computing technologies for Internet of
the devices mobility, their remaining energy, and the latency Things: a primer. Digital Communications and Networks, 4(2), 77-86.
sensitivity of its application. By considering them, the [14] Kecskemeti, G., et al. (2017). Modelling and simulation challenges in
proposed load balancing algorithm that is based on the Fuzzy internet of things. IEEE Cloud computing, 4(1), 62-69.
Decision Tree has outperformed the state of the art solution [15] Borgia, E. (2014). The Internet of Things vision: Key features,
on every aspect of the comparison. It has reduced the average applications and open issues. Computer Communications, 54, 1-31.
tasks completion delays by 25.4%. It has improved the [16] Mukherjee, A., et al. (2014). Angels for distributed analytics in iot. In
network lifetime. In addition, it has reduced the tasks failure Internet of Things (WF-IoT), 2014 IEEE World Forum on (pp. 565-
rate by 39%, and the energy consumption of edge devices by 570). IEEE.
57.2%, thanks to its set of criteria and to the adopted [17] Santoro, D., et al. (2017). Foggy: a platform for workload orchestration
reinforcement learning approach. As a future work, we plan in a Fog Computing environment. In Cloud Computing Technology and
Science (CloudCom), 2017 IEEE International Conference on (pp.
to add support for the registry and the virtual machines 231-234). IEEE.
migration. [18] Chamola, V., et al. (2017). Latency aware mobile task assignment and
By introducing this simulator, we hope that this modest load balancing for edge cloudlets. In Pervasive Computing and
Communications Workshops (PerCom Workshops), 2017 IEEE
work encourages the adoption of Edge Computing in the International Conference on (pp. 587-592). IEEE.

707
Authorized licensed use limited to: ULAKBIM UASL - Lefke Avrupa Universitesi. Downloaded on November 17,2023 at 07:15:34 UTC from IEEE Xplore. Restrictions apply.