2014 IEEE International Conference on Cloud Engineering

Cloud QoS Scaling by Fuzzy Logic

Stefan Frey, Claudia Lüthje, Christoph Reich Nathan Clarke

Furtwangen University of Applied Science Plymouth University
Cloud Research Lab Centre for Security, Communications and Network Research
Furtwangen, Germany Plymouth, United Kingdom
{stefan.frey, christoph.reich}@hs-furtwangen.de [email protected]

Abstract—One of the biggest advantages of cloud infras- weekly variability in usage. This is referred to as horizontal
tructures is the elasticity. Cloud services are monitored and scalability. In principle such a scaling service has clearly
based on the resource utilization and performance load, they the potential to guarantee the quality of a service. Key
get scaled up or down, by provision or deprovision of cloud
resources. The goal is to guarantee the customers an acceptable Performance Indicators (KPIs), such as the request response
performance with a minimum of resources. Such Quality of time, are typically used to state the quality of a service.
Service (QoS) characteristics are stated in a contract, called These build the Service Level Objectives (SLOs) in a Service
Service Level Agreement (SLA) negotiated between customer Level Agreement (SLA) and are usually not controlled
and provider. The approach of this paper shows that with directly. KPIs can only be controlled indirectly, by scaling
additional imprecise information (e.g. expected daytime/week-
time performance) modeled with fuzzy logic and used in a the service resources up or down.
behavior, load and performance prediction model, the up and SLA violations can happen especially due to sudden peak
down scaling mechanism of a cloud service can be optimized. demands, caused by several different kinds of reasons (e.g.
Evaluation results confirm, that using this approach, SLA product launches, political statements, service advertisement,
violation can be minimized. weather changes, etc.), or the scaling delay of the infrastruc-
Keywords-Cloud Computing, Elasticity, Scaling Service, Fuzzy ture (e.g. VM start time, Load Balancer reconfiguration). For
Logic, SLA, QoS example, Figure 1 illustrates a so-called ”flash crowd”, with
the traffic load of a server during the 1998 World Cup. One
I. I NTRODUCTION of the peak loads was 8th of July (see arrow) during the
After an initial hype, cloud computing has established semi final between Germany and Holland [3]. This could
itself as adequate means of providing resources on demand have been easily predicted by an football expert.
on a self-service basis and gives customers access to a large
pool of computational power and storage. Cloud computing
provides a practical alternative to locally hosted resources.
With cloud computing, customers do not have to manage
and maintain their own IT assets and are not bound to their
locally limited resources. Instead they are charged by cloud
providers based upon the amount of resources they utilize
or keep reserved. Through this “pay-per-use” model costs
and pre-investments can be reduced. At the same time it
allows customers to dynamically adapt to changing business
demands. However, one of the main problems, are Quality
of Service (QoS) guarantees for the user’s applications
by the cloud provider. At present most cloud computing
providers offer only rudimentary service guarantees, like
for example, guaranteeing 99,95% availability for their EC2 Figure 1: Daily Traffic Volume of the World Cup 1998
cloud service, stated in generic Service Level Agreements Website
(SLA) [1]. In most cases cloud providers do not give any
performance guarantees at all. However, adequate scaling is very important. If the scale
Scaling services, like Amazons Auto Scaling service [2] down happens too early SLA violations can occur. On the
adjust the capacity of virtual machines (VMs) up or down other hand if scale down happens too late the customers will
automatically according to e.g. CPU utilization. Such scaling pay for resources they do not utilize. In order to minimize
services regulate the number of VMs used to maintain the the number of SLA violations and to guarantee the QoS,
performance of a service that experiences hourly, daily, or a behavior, a load and performance prediction model is

978-1-4799-3766-0/14 $31.00 © 2014 IEEE 343

DOI 10.1109/IC2E.2014.30
needed. If one could predict the usage of a service, looking It addresses the issue of determining the best trade-off
ahead further than the infrastructure delay time, one could between profit and customer satisfaction. This is addressed
guarantee the QoS for that specific service. The approach using an analytical performance model to predict service
presented in this paper aims to improve scaling of the cloud performance. In spite of our approach the used utility
with help of fuzzy parameters. function and capacity planning method are not using non-
The remainder of the paper is organized as follows. In infrastructure metrics. A lot of recent work deals with
Section II the related research efforts are discussed. In computing resource allocation in clouds, focusing on spe-
Section III, the specific approach using fuzzy logic for con- cific management objectives, such as energy efficiency [9],
trolling the scalable cloud service is introduced. The proof fairness [10], economic fitness [11], and service differentia-
of concept is reported in Section IV. Finally a conclusion is tion [12]. All of the above works however deliver placement
drawn and future work is suggested in Section V. solutions. They do not consider the problem of controlling
a load balanced or a cloud scaling service. Related research
II. R ELATED W ORK can be found in the area of forecasting the load of elec-
Since 2009 many teams have been working on the prob- trical power grids, in [13] and [14], where they use social,
lem to improve QoS for cloud computing. Armstrong and economic, and weather condition factors. To achieve QoS
Djemame tried to transfer the technologies of QoS from grid guarantees this paper uses such additional factors.
to cloud computing [4]. The paper of Rochwerger et al. [5]
discussed the funded project RESERVOIR, in which pooled III. F UZZY M ODULE TO I MPROVE Q O S C ONTROL
resources handle peaks and slopes of resources. Most approaches consider infrastructure sensor data like
Another interesting approach is the Q-Clouds framework bandwidth, request/response time, CPU usage, memory us-
described by Nathuji et al. [6]. This framework for the age, etc. to control the scaling infrastructure as depicted in
management of cloud servers enables the possibility to apply figure 2 (dashed box). The approach of this paper is to
and control QoS. They introduce Q-states to provide the use additional, often imprecise information (e.g. weather)
possibility for users to define certain metric limits for SLOs, to improve the management to meet the QoS requirements
based on a cost model. The more the customer is ready to stated in SLAs. These imprecise factors (e.g. user wants
pay, the less likely a SLA violation will occur. The controller scaling aggressive/moderate, etc.), political factors (legal
component uses a MIMO (multi-input, multi-output) model changes, political summits, etc.), economic/market factors
for the calculation of VM resources. So an input vector (product advertising, product launch, etc.), other factors
is defined by the platform controller. Based on that the influencing the service usage (e.g. weather, gossip, etc.) can
output vector delivers the predicted QoS values. Unlike the not be modeled precisely.
approach in this paper they basically used infrastructure Fuzzy logic allows to model imprecise information by the
metrics (e.g. performance, memory, etc.) to control QoS. user in the form of non-numeric linguistic variables (e.g.
The next important steps in the QoS for cloud computing weather: bad/good/excellent). These fuzzy inputs are used
were developed by Ferretti/Ghini/Panerieri [7]. Their paper in the fuzzy control system, that uses expert knowledge to
presents an architecture, which provides cloud resources inference a fuzzy output. After defuzzifying this output to a
dynamically. The developed middleware tries to avoid SLA crisp value, then this states how big the up and down scale
violations with the same use case as presented in this paper. factor should be. For example, if a customer wants to have
Therefore they split the problem into three components: an aggressive scaling control the infrastructure will scale up
The host platform is a dynamic configuration, that should with e.g. 3 VMs otherwise with only one VM at a time.
guarantee the requirement of the SLA. The monitoring The scaling domain expertise is modeled in a knowledge
component is checking the host platform and its application base with fuzzy IF-THEN rules.
to display changes in the configuration and violation of the
QoS requirements. The third component is responsible for A. Fuzzy Controled Scaling Architecture
dispatching and load balancing. This component tries to keep Figure 2 shows the architecture for a load balanced
the required QoS. It has been implemented in an intelligent service by automatically scaling up/down the infrastructure
way, to distribute available resources. It is distinct to load by provisioning/deprovisioning VMs. It consists of two new
balancing for requests and sessions. Different to this paper modules compared to the traditional scaling infrastructure
is, that a load balancer takes over the monitoring of SLAs (blue box), the Data Collector and the Fuzzy Control Mod-
instead of a separate module. Furthermore the only metric ule.
that is used is the actual request response time. Compared to The Data Collector collects all information data, crisp
our approach, there can be considered mean value, derivation (e.g. cpu usage) and imprecise data (e.g. weather). The data
value, imprecise information and admin control information. is categorized in infrastructure data (e.g. req./resp. time),
Another SLA driven approach from Kouki and Ledoux [8] history data (e.g. req./resp. time 5 minutes ago), control
tries to optimize the capacity planning for cloud applications. action (e.g. aggressive up/down scale), environment data

344
 is higher.
Time-dependent sensor data contain parameters that can
influence the infrastructure management at a predefined
time. The knowledge of the typical weekly usage for an
service (see Fig. 3) can be modeled and therefore the
decision to scale up or down strongly or weakly depending
whether the change is high or not.

Figure 3: Example: Weekly Load of the HFU Learning

Management Platform

Figure 2: Fuzzy Controlled Scaling Architecture Service-dependent sensor data involve parameters that
influence the control infrastructure depending on the related
service. Political parameters, like new legal issues enforcing
(e.g. daytime), and other information that might influence more logging at the service side. Market events, like product
the load of the service. All collected data is input data to the launches, marketing events, new prices, etc. can influence
Fuzzy Control Module, where the data is fuzzified. Results the usage of services. Gossip, modeled as good news or bad
are propagated by the fuzzy inference engine and quantified news is influencing service usages. Importance of service
by defuzzification. The defuzzified value (Number of VM to might need a more aggressive management to make sure,
be started or stopped) is put into the Scale Control module, that the SLA violations can be minimized.
which generates XML-RPC calls to the Cloud Management C. Fuzzy Control Module
System.
The Fuzzy Control Module consists of four main fuzzy
B. Collected Information Factors for the Fuzzy Control control processes represented by the four sub-modules re-
spectively (see Fig. 2). The crisp and imprecise input data is
The relevant information to improve elasticity can be converted into fuzzy values for each input fuzzy set with the
categorized into the following monitoring data: infrastruc- Fuzzifying module. The decision making logic of the Fuzzy
ture, infrastructure history, time-dependent, and service- Inference module determines how the fuzzy logic operations
dependent sensor data. are performed (SUP-MIN inference), and together with the
Infrastructure sensor data includes factors that can mostly Knowledge Base module determine the outputs of each fuzzy
be monitored using sensors placed in various locations and IF-THEN rules. Those are combined and converted to crisp
layers of the cloud infrastructure. KPIs, like request response values with the Defuzzification module. The output crisp
time, which can easily be measured at the load balancer value can be calculated by the center of gravity or the
(LB). Cloud specific parameters, like start time of VMs, weighted average and are converted to the number of VM
can be aquisitioned at the cloud management system. The to started or stopped.
quality of the cloud infrastructure or service implementation
can be taken into account as well. The load balancing control
might be influenced by the basic robustness of the overall
infrastructure. The infrastructure robustness can be modeled
by an imprecise parameter e.g. strong, weak.
Infrastructure history sensor data are parameters that have
been previously collected into a history data base. The
purpose is to calculate values like, mean values, derivation
values, etc. These statistical data can be good indicators to
Figure 4: Input Fuzzy Set for Load Deviation
improve the LB management. Imprecise history parameters
can be of interest as well. Suppose a service depends on the
Fuzzification: Fuzzification is the process of decom-
weather condition (e.g. online shop for winter tires), then a
posing input data into fuzzy sets, with trapezoidal shaped
sudden change of the weather condition from dry to snowy
membership functions. Figure 4 shows a system of fuzzy
condition makes it more likely, that the load of such a service

345
sets for an input with trapezoidal membership functions. time from generating the request until the response arrives
Any particular input is interpreted from this fuzzy set and a at the load balancer is measured. Within the logic modules
degree of membership is interpreted. If the request-response- these determined request response times then are checked
time, for example, it set to about 100 request-per-seconds, and based on the stored fuzzy or conventional rules will
the fuzzy value loaddeviation is set to low. be decided to scale the service up, down or do nothing.
Fuzzy Inference: The fuzzy values gathered from the And finally the scaler, which connects to the actual cloud
input data are processed by the inference engine using the management system, which is responsible for the provision
expert domain knowledge modeled as fuzzy IF-THEN rules. and adding or removal of resources. The used conventional
The following fuzzy rules are examples how to state the rules are represented by a simple boundary system based
domain knowledge in the area of up and down scale control. with low and high thresholds. When the measured average
request response hits the upper boundary a VM is started
IF ReqRespTime_rising=high AND
or when the lower boundary is hit a VM is stopped. These
expected_ReqRespTime_rising=high AND
boundary systems are widely used and provide a common
product_launch=now AND
way of QoS management. The fuzzy set basically uses the
....
THEN
up_scale=very high
...
Defuzzification: After the fuzzy reasoning the resulting
linguistic output variable (e.g. scale up = high) needs to
be translated into a crisp value (e.g. number of VMs to be
started or stopped at time). Defuzzification maps the output
from the fuzzy domain back into the crisp domain. The most
common defuzzification methods is the Center-of-Area (C-
o-A) often referred to as Center-of-Gravity and is defined as
follows: 
μi (x)xdx
x∗ =  (1)
μi (x)dx
where x∗ is the defuzzified output, μi (x) is the aggregated
membership function and x is the output variable. The C-o- Figure 5: Simulator Module Diagram
A method calculates the area under the scaled membership
functions and within the range of the output variable and
afterwards calculates the geometric center of this area. same boundaries as the conventional rules, but as additional
decision factors, a prediction based on expert knowledge is
IV. P ROOF OF C ONCEPT BY S IMULATION used. This additional information allows an earlier response
The aim of this research was to verify if the extended to the changing loads and thus an better reaction to upcom-
fuzzy approach was able to ensure the QoS better than other ing demands. Based on historical data the approximate usage
conventional procedures. For this we evaluate and discuss and thus the load can be determined for a service. Figure
the results obtained by simulation. First the Simulation 3 show the historic data of the load for a service during
Environment will be introduced. a week. Based on this a simplified load prediction during
daytime can be derived.
A. Simulation Environment Based on such knowledge an expert specifies whether the
In order to test the feasibility of our approach, we created load will expectedly be increasing at a high, regular or low
a simulation environment to be capable of validating the rate. A predicted regular load will result in a rule set that
general fuzzy controlled scaling architecture proposed in this equals the conventional boundaries, therefore resulting in
paper. The simulator therefore consists out of four major essentially the same behavior. In case of a high prediction,
modules (see Fig. 5). Firstly, the request generator module, the scaler generally scales up faster, which means VMs are
which simulates the real life usage by generating requests started on a lower load and additionally up to two VMs can
to the cloud service. It is important to note, that in this be started at the same time based on the load. Simultaneously
simulation requests are generated with a static predefined the scale down boundary is set to a lower load to keep
workload trend. The second component is the load balancer a higher pool of available VMs. The low prediction, is in
module, which distributes the generated requests equally to principle a reversed high prediction, which will change the
the pooled Virtual Machines (VMs) by round-robin. Here behavior into generally scale up later and scale down faster
also the request response time is determined, therefore the and up to two machines at once.

346
 As an additional basis for decision, the slope of the last
measured points is used. The basis for this assumption is that
a strong growth of response time indicates an upcoming peak
load. The slope factor is therefore calculated of two response
time values with the following formula:

ΔY Y 2 − Y1
slopea = = (2)
ΔX X2 − X1
Where Y and X are the time and value of the measured
responsetime and slopea is the currently determined slope.
Since the course response time is unlikely linear, the slope Figure 6: Load Graph
must be determined more than once in order to reconstruct
the actual system behavior. Problematic are load bursts,
which the determined slope suddenly rises or falls down
extremely. In the worst case this could cause a VM to be
started up and immediately get shut down again or vice
versa. Therefore, the average of of the last n slopes are
used for the assessment. The simulator is based on a model
in which a generated request includes a static processing
time of 100ms. The KPI, is measured as the request/response
time, based on the average of the last 10 processed requests.
Thereby the time is counted from the generation of the
request, till arrival of the response after the processing at the Figure 7: Conventional Rule Set Results
load balancer. The QoS limit has been set to 2000ms in this
model and the conventional rule set regulates at an average
response time of 1500ms by scaling up and at 1000ms by Result with fuzzy rule set (high prediction): Compared to
scaling down one VM at a time. To eliminate the influences the output of the fuzzy rules with high prediction, as shown
of the test environment, like processor fluctuations the factor in figure 8, it becomes clear that the VMs are allowed much
of 10 was used to all above described values. earlier to be started, allowing to reduce the response time and
To determine the suitability of the procedure presented, remain inside the SLA margin. Although for the first peak
different scenarios have been created and tested with and load two VMs additional VMs are used, this resembles a
without the fuzzy control mechanism. relatively small expense compared to breaking the SLA. It
is shown later on in the simulation that better results are
B. Simulation Scenario delivered in the smaller peaks with the same number of
VMs.
In the presented scenario a complex course of load is
sent to the service. In the first three minutes there is an
peak demand, followed by reoccurring burst loads. This
scenario simulates a peak load that doesn’t flatten quickly.
This could be the case for services that depend heavily on
news (e.g. launch of intensive product advertising). If the
related service shows up in the news an initial large peak
is produced and is recreated as smaller recurring peaks by
spreading the word. Figure 6 shows load graph for this
scenario.
Result with conventional rule set: The conventional rule
set result graph shows that in this test the QoS limit of Figure 8: Fuzzy & ”High” Prediction Results
20,000ms is exceeded for the first initial peak load and is
nearly hit for the following rises. The scaling lags behind
Result with fuzzy rule set (high prediction & slope):
compared to the load increase. So it cannot cope with the
Comparing these results with those of the fuzzy rules with
increasing load fast enough by starting new VMs. This is
high prediction and slope (see Figure 9), the response time
clearly proven by the fact that the maximum number of VMs
has been reduced again, in spite of the very rapid scale up of
simultaneously used is only reached immediately before the
VMs. The response time is in the range of all other peaks
flattening of the load.
and is in this case is maintained very spacious. However,

347
in this case an additional VM is switched on compared to [3] M. Arlitt and T. Jin, “A workload characterization study of
the fuzzy rules with high prediction. The large differences the 1998 World Cup Web site,” Network, IEEE, vol. 14, no. 3,
in the results graphs are however not dependent on this pp. 30–37, 2000.
additional resource. Instead the upscaling of sevmultiplel [4] D. Armstrong and K. Djemame, “Towards quality of service
VMs simultaneously achieves the improvement. In this case in the cloud,” 2009, pp. 226 – 240.
up to 3 VMs are allocated at the same time.
[5] B. Rochwerger, A. Galis, E. Levy, J. Caceres, D. Breit-
gand, Y. Wolfsthal, I. Llorente, M. Wusthoff, R. Montero,
and E. Elmroth, “Reservoir: Management technologies and
requirements for next generation service oriented infrastruc-
tures,” in Integrated Network Management, 2009. IM ’09.
IFIP/IEEE International Symposium on, 2009, pp. 307 –310.

[6] R. Nathuji, A. Kansal, and A. Ghaffarkhah, “Q-clouds: man-

aging performance interference effects for qos-aware clouds,”
in Proceedings of the 5th European conference on Computer
systems, ser. EuroSys ’10, New York, NY, USA, 2010.

[7] S. Ferretti, V. Ghini, F. Panzieri, M. Pellegrini, and E. Turrini,

Figure 9: Fuzzy & ”High” Prediction & Slope Results “QoS-Aware Clouds,” in Proceedings of the 2010 IEEE 3rd
International Conference on Cloud Computing, ser. CLOUD
’10. Washington, DC, USA: IEEE Computer Society, 2010.
V. C ONCLUSION AND F UTURE W ORK [8] Y. Kouki and T. Ledoux, “Sla-driven capacity planning for
The aim of this paper was to show how a common cloud cloud applications,” in Cloud Computing Technology and Sci-
computing scaling service could be enabled to guarantee ence (CloudCom), 2012 IEEE 4th International Conference
on, 2012, pp. 135–140.
QoS parameters. Especially the KPI, request-response-time,
has been the focus. The presented extended QoS provi- [9] C.-T. Yang, K.-C. Wang, H.-Y. Cheng, C.-T. Kuo, and
sioning architecture with an fuzzy control module has been W. C. C. Chu, “Green power management with dynamic
delineated. As proof of concept several tests were performed resource allocation for cloud virtual machines,” in Proceed-
in order to prove the effectiveness of each approach. It was ings of the 2011 IEEE International Conference on High
Performance Computing and Communications, ser. HPCC
noted that additional input parameters can improve the fuzzy ’11. Washington, DC, USA: IEEE Computer Society, 2011.
rule set and that violation of SLAs could be avoided. It
can be concluded that with the presented approach it is [10] F. Wuhib, R. Stadler, and M. Spreitzer, “A gossip protocol for
clear that more resources are used which can lead to higher dynamic resource management in large cloud environments,”
costs for the customer. But the response time and the SLA vol. 9, no. 2, 2012, pp. 213–225.
thresholds are maintained better and less SLA violations [11] J. Xu, M. Zhao, J. Fortes, R. Carpenter, and M. Yousif, “Auto-
occur. Whether this is economical in each case must be nomic resource management in virtualized data centers using
decided by experts contemplating the SLA and contract data fuzzy logic-based approaches,” vol. 11, no. 3. Hingham, MA,
Future work is to proof these results within real cloud USA: Kluwer Academic Publishers, Sep. 2008, pp. 213–227.
environments and to develop an easy to use user interface.
[12] J. Rao, Y. Wei, J. Gong, and C.-Z. Xu, “DynaQoS: model-
This shall allow users to specify imprecise information input free self-tuning fuzzy control of virtualized resources for qos
and expert knowledge. Additionally, the expansion to other provisioning,” in Proceedings of the Nineteenth International
QoS parameters, and further fuzzy input data has to be Workshop on Quality of Service, ser. IWQoS ’11. Piscataway,
examined. NJ, USA: IEEE Press, 2011, pp. 31:1–31:9.

ACKNOWLEDGMENT [13] Y.-M. Wi, S.-K. Joo, and K.-B. Song, “Holiday load forecast-
ing using fuzzy polynomial regression with weather feature
This research is supported by the German Federal Min- selection and adjustment,” vol. 27, no. 2, may 2012.
istry of Education and Research (BMBF) through the re-
search grant number 03FH046PX2. [14] K.-B. Song, S.-K. Ha, J.-W. Park, D.-J. Kweon, and K.-
H. Kim, “Hybrid load forecasting method with analysis of
R EFERENCES temperature sensitivities,” vol. 21, no. 2, may 2006.
[1] Amazon ec2 service level agreement. Online. Retrived
24.07.2013. [Online]. Available: http://aws.amazon.com/ec2-
sla/

[2] Amazon auto scaling service. Online. Retrived 01.07.2013.

[Online]. Available: http://aws.amazon.com/autoscaling/

348