A Kubernetes-Based Monitoring Platform for

Dynamic Cloud Resource Provisioning

Chia-Chen Chang and Shun-Ren Yang En-Hau Yeh and Phone Lin Jeu-Yih Jeng
Dept. of Comp. Sci. Dept. of Comp. Sci. Billing Information Laboratory
National Tsing Hua University National Taiwan University Telecommunication Laboratories
Hsinchu, Taiwan, R.O.C. Taipei, Taiwan, R.O.C. Chunghwa Telecom Co., Ltd.,
[email protected] [email protected] Taipei, Taiwan, R.O.C.
[email protected]

Abstract—Recently, more and more network operators have One of the most popular solutions for container-based
deployed cloud environment to implement network operations virtualization is using Docker for container packaging with
centers that monitor the status of their large-scale mobile Kubernetes for multihost container management [1]. Kuber-
or wireline networks. Typically, the cloud environment adopts
container-based virtualization that uses Docker for container netes follows the masterslave model, which uses a master to
packaging with Kubernetes for multihost Docker container man- manage Docker containers across multiple Kubernetes nodes
agement. In such a container-based environment, it is important (which are physical or virtual machines). A master and its
that the Kubernetes can dynamically monitor the resource controlled nodes constitute a “Kubernetes cluster.” A developer
requirements and/or usage of the running applications, and can deploy an application in the docker containers via the
then adjust the resource provisioned to the managed containers
accordingly. Currently, Kubernetes provides a naive dynamic assistance of the Kubernetes master. Typically, an application
resource-provisioning mechanism which only considers CPU is divided into one or more tasks executed in one or more
utilization and thus is not effective. This paper aims at developing containers. Each container is generally restricted in terms of
a generic platform to facilitate dynamic resource-provisioning the maximum resource quantity that it can consume. That is to
based on Kubernetes. Our platform contains the following three say, the total amount of resource provisioned to an application
features. First, our platform includes a comprehensive monitoring
mechanism that integrates and provides the relatively complete is the sum of its executed containers’ resource capacity.
system resource utilization and application QoS metrics to the As mentioned above, Kubernetes is a tool which is used
resource-provisioning algorithm to make the better provisioning to manage multiple Docker containers across multiple ma-
strategy. Second, our platform modularizes the operation of chines. In such a container-based environment, it is important
dynamic resource-provisioning operation so that the users can that the Kubernetes can dynamically monitor the resource
easily deploy a newly designed algorithm to replace an existing
one in our platform. Third, the dynamic resource-provisioning requirements and/or usage of the running applications, and
operation in our platform is implemented as a control loop then adjust the resource provisioned to the managed containers
which can consequently be applied to all the running application accordingly, preventing from resource overprovisioning and
following a user-defined time interval without other manual underprovisioning. Typically, the dynamic resource provision-
configuration. ing procedure can be simply separated into four steps: (1) The
Index Terms—Container, Docker, dynamic resource provision-
ing, monitoring, Kubernetes. monitor step: Monitoring the resource usage and limitation of
the entire environment and all of the running applications; (2)
The analyze step: Integrating the monitored data as the input
I. I NTRODUCTION
data of the plan step; (3) The plan step: Using an resource
Recently, more and more network operators (e.g., provisioning algorithm to determine the resource-provisioning
Chunghwa Telcom, Taiwan) have deployed and operated large strategy based on the data generated by the analyze step; and
data centers to manage their businesses. Specifically, the oper- (4) The execute step: Adjusting the resource provisioning to
ators have adopted cloud technologies to implement network all of the running applications according to the strategy in the
operations centers that monitor the status of their large-scale plan step.
mobile or wireline networks, such as computing-resource In Kubernetes, a built-in mechanism has been provided
usage and network response time. for dynamic resource provisioning. In this mechanism, the
In data centers, container-based virtualization, also known users can first set a desired percentage of CPU utilization
as operating system (OS)-level virtualization, is a new virtu- to a specific group of containers that are executed for an
alization technology that divides a server into multiple virtual application. Then, the mechanism executes the four steps that
instances, called containers, by sharing the underlying OS implement dynamic resource provisioning. In particular, in the
kernel. Unlike traditional virtual machines (VMs), lightweight monitor and analyze steps, Kubernetes uses a user-deployed
containers can be started rapidly, which brings about a lower monitoring tool to retrieve the CPU utilization of each running
overhead of computing resources and time in deployment. container; in the plan step, Kubernetes uses its own algorithm

978-1-5090-5019-2/17/$31.00 ©2017 IEEE

to determine the number of containers to ensure that the transfer containers across machines in a simple way. There are
average CPU utilization approaches the user’s desired quantity. three main components of Docker, as follows:
However, this mechanism has some deficiencies. In the plan ∙ Docker images. Docker images are read-only templates
step, Kubernetes only allows users execute its own resource- which are the bases of creating Docker containers.;
provisioning algorithm; in other words, users cannot apply ∙ Docker registries. Docker registries are the storage that
another resource-provisioning algorithm by themselves. This holds a huge collection of Docker images.;
suggests that the mechanism is not flexible. Furthermore, the ∙ Docker containers. Docker containers are the virtual
algorithm only considers CPU utilization when determining instances in which the applications are running. Each
the resource provisioning strategy. This is not realistic enough Docker container contains the running application and
because there are many factors affecting the application per- all of its dependency files, such as code, system libraries,
formance. Lastly, this mechanism has to be invoked manually system tools, and so on.
for each application. Therefore, the more running applications,
Users (i.e., Docker clients) can send Docker commands to
the more redundant manual configurations that users have to
the Docker daemon, a process running on a host machine
set.
to manage and manipulate containers, using the terminal or
This paper develops a generic platform to facilitate dynamic
the remote application programming interface (API). After
resource provisioning over Kubernetes, while resolving the
receiving the commands, the Docker daemon will manipulate
existing mechanism’s deficiencies. Specifically, our platform
the containers and the images on the host machine accordingly,
has the following features:
such as by using the images to create containers, pulling
∙ A comprehensive monitoring mechanism. Unlike the ex-
images from the remote registry, pushing images to the remote
isting Kubernetes built-in mechanism that only considers registry, and so on.
CPU utilization, our platform takes both system resource
utilization and application QoS metrics into account. III. T HE C ONTAINER - BASED C LUSTER M ANAGEMENT
Specifically, our platform employs an open-source tool S YSTEM
to retrieve all the resource utilization metrics (such as
Many leading IT companies have integrated Docker into
memory and disk access) that the underlying system can
their products, including the Google Cloud Platform, IBM
provide. Moreover, we integrate the application QoS met-
Bluemix, Microsoft Azure, and so on. Although container-
rics (such as response time) monitoring into our platform.
based virtualization has brought about advancements in vir-
Thus, a rather complete set of monitored metrics can be
tualization technology, it is complicated to manipulate a vast
exposed to the resource-provisioning algorithm, for better
number of containers running on a cluster made up of servers.
provisioning decisions;
Thus, there is a need for container-based cluster management
∙ Deployment flexibility. In our platform, we modularize the
and orchestration tools. Kubernetes [3] is such an instrument,
four steps of dynamic resource-provisioning, so that the
representing an open source Docker container cluster manage-
users can easily deploy a newly designed algorithm to
ment tool that was originally developed by Google. So far, a
replace an existing one without affecting other modules;
number of IT companies, including IBM, Microsoft, Red Hat,
∙ Automatic operation. After the users deploy a desired
and so on, have supported and contributed to the Kubernetes
dynamic resource-provisioning algorithm, the resource-
project. Given its widespread use, Kubernetes seems to be the
provisioning operation will be applied automatically and
current trend in the IT industry.
periodically to all the running applications in our platform
To follow this trend, our work aims at developing a
following a user-defined time interval.
monitoring platform for dynamic resource provisioning on a
The remainder of the paper is organized as follows: Section Kubernetes-based environment. As shown in Figure 1, our
2 gives a brief introduction to Docker. Section 3 describes system architecture contains one master and multiple nodes,
the system architecture and components of our platform. where each node is either a physical or a virtual machine that
Section 4 illustrates the implementation of dynamic-resource provides the runtime environment for containers. The node
provisioning using our platform. Section 5 describes one components can be described as follows:
experiment performed to verify the operations of dynamic ∙ Pod. One or more containers can be grouped into a pod,
resource provisioning in our platform. Finally, the paper is which is a basic deployment unit that can be operated
concluded in Section 6. and managed in a Kubernetes cluster. Each pod can be
II. OVERVIEW OF D OCKER C ONTAINER - BASED seen as a virtual server and will be assigned a virtual IP
V IRTUALIZATION address by the master;
There have been a lot of implementations of container-based ∙ Kubelet. Kubelet is the Kubernetes node agent, which
virtualization, such as LXC, lmctfy, FreeBSD jail, and Docker. runs on each node of the system. It monitors the pod
In particular, Docker [2], an open source project that was specifications through the master and maintains these
initially released in 2013, has drawn great attention from the pods on the node. Furthermore, it is also responsible
IT industry. Docker is a software containerization platform that for reporting node events, pod status, and the resource
allows users to deploy their application inside a container and utilization of a node to the master;
 ApplicationClients ∙ Kubernetes services. A Kubernetes service is used to
identify which pods used to execute the same application
Node Node should be grouped together. When traffic arrives, it will
Kubernetes Pod(1) Pod(1) Pod(1) Pod(1)
... ...
be recognized as its corresponding service IP, and then
Master Container Container Container Container
... be redirected to one of its corresponding backet the term
etcd(4)
cAdvisor(3) Kubelet(2) cAdvisor(3) Kubelet(2)
“service” represents the “Kubernetes service” below);
LinuxOS LinuxOS
∙ Replication controllers. The Kubernetes replication con-
Scheduler(5) ... troller is used to control the number of pods that are
Node
executed for an application in the system; and
APIServer Node
(6) Pod(1) Pod(1) Pod(1) Pod(1) ∙ Labels and label selectors. Labels are key and value pairs
...
Container ... Container Container Container that are used to group each Kubernetes objects, such as
Controller ...
Manager(7) pods, services, and so on, into sets. Label selectors are
cAdvisor(3) Kubelet(2)
cAdvisor(3) Kubelet(2) used to determine the subset of the Kubernetes objects.
LinuxOS
LinuxOS
LinuxOS For example, suppose that a Kubernetes service has two
label selectors, key1/value1 and key2/value2; the label set
ServiceLayer
of any pod will then contain these two key and value pairs
PhysicalMachine VirtualMachine PublicCloud PrivateCloud belonging to this service.

Fig. 1: The system architecture. IV. I MPLEMENTATION OF THE DYNAMIC

R ESOURCE - PROVISIONING P ROCEDURE
To realize dynamic resource provisioning, our system imple-
∙ cAdvisor. cAdvisor is an open-source container resource ments the Monitor-Analyze-Plan-Execute (MAPE) model [4].
usage monitoring tool that has been integrated into Therefore, our system contains the four modules—Monitor,
Kubelet binary. It automatically gathers the information Data Aggregator, Resource Scheduler and Pod Scaler—which
on resource utilization of all the containers in the node. correspond to the four phases of the MAPE loop, respectively.
The master is the central controller of the system. It contains The implementation of these four modules are described in the
four Kubernetes master components—etcd, Scheduler, API following subsections.
Server, and Controller Manager—which are used to control
and manage the nodes and the containers in the system. Briefly, A. The Monitor Module
the functions of the four master components in our system are The monitor module is used to monitor the metrics of the
as follows: overall system status. These can be used as the input data
∙ etcd. As a storage component, etcd is used to store the of the resource-provisioning algorithm. We divide the monitor
state of the system, allowing the other master compo- module into two parts, namely system resource metrics moni-
nents to synchronize themselves to the desired state by toring and application performance metrics monitoring, which
watching etcd; are described below.
1) System resource metrics monitoring: To achieve system
∙ Scheduler. The scheduler is responsible for scheduling
resource metrics monitoring, we adopt three existing open
each pod on a node in the system;
source technologies in our system, as follows:
∙ API Server. The API server is responsible for receiving ∙ Heapster v.0.19.1. Heapster [5] is a container-based
commands and manipulating the data for Kubernetes cluster-monitoring tool for Kubernetes. It gathers the
objects (such as pods) accordingly in the system. Users resource utilization information of each node and then
can send commands to the API server by using the exports the gathered information into a storage backend.;
Kubernetes command line interface (CLI), kubectl; ∙ InfluxDB v.0.9.4.1. InfluxDB [6] is a database designed
∙ Controller Manager. The controller manager monitors to store time series data. In our system, InfluxDB is used
etcd and regulates the state of the entire system. In other to store the resource utilization information collected by
words, if the state of the system changes, the controller Heapster; and
manager will force the system into the desired state ∙ Grafana v.2.1.0. Grafana [7] is an application used for
using the Kubernetes API. time-series data visualization. It will query the time series
Because our system is based on Kubernetes, some terms data from the database and then display these data in
and mechanisms of Kubernetes need to be illustrated; these graphs.
are described as follows: The implementation of system resource metrics monitor-
∙ Kubernetes namespaces. The Kubernetes namespace is ing is as shown in Figure 2. Specifically, Heapster is the
used to partition the user-created Kubernetes objects into main monitoring component that interacts with InfluxDB and
a named group. Thus, each application can be run into Grafana for database access and visual presentation. Like other
different Kubernetes namespaces to prevent them from applications, Heapster, InfluxDB, and Grafana are all deployed
affecting each other; within pods. In our system, they are deployed separately in
 Master Monitor

SystemResourceMetrics Applicationperformance Kubernetes

Monitoring metricsmonitoring
(1) Master

Node Node etcd

Pod Pod Pod ... Pod
(3) Application
... InfluxDB Container ...... Container ServiceIP
Heapster Pod
Container
Container Pod
Container
Container collection responsetime Scheduler
Grafana Pod Pod (2)
(6)
cAdvisor Kubelet Apache (3) JMeter (1)
(2) APIServer
cAdvisor Kubelet Node
cAdvisor
cAdvisor Node
Kubelet
Kubelet JMeter Invoker
(4) (5) Controller
Predefined
testplan JMeter Manager
Fig. 2: System resource metrics monitoring. output
LinuxOS

Fig. 3: Application performance metrics monitoring.

three containers, where the Heapster container is placed in
one pod, while the other two containers are in another pod.
In such a deployment, however, for Heapster to handle the the current information about all the services and their
system resource metrics monitoring, we have to let Heapster corresponding virtual IP addresses and ports throughout
have the master IP address and the service IP address of the the system from the master. The master will return the
pod which contains the InfluxDB. data in JSON format;
After the deployment of Heapster, InfluxDB and Grafana, Step 2. JMeter provides some configuration elements that can
the system resource metrics monitoring uses the following be used to set up default values or variables. In our JMeter
process: Invoker program, we use CSV Data Set Config to make
Step 1. Heapster will ask the master for the list of nodes in JMeter send requests to the current service IPs in the
the system using the Kubernetes API; system.;
Step 2. In each node, Kubelet gathers the resource utilization Step 3. We have defined a JMeter test plan file that makes
information for the whole machine and the containers, JMeter send 100 requests to each deployed application.
which are deployed on that node, by using cAdvisor. We propose that the average interarrival time of these 100
Step 3. After receiving the resource-utilization information requests in the test plan is three seconds. JMeter will then
concerning each node, Heapster will expose these data send requests according to the predefined JMeter test plan
into the database, called “k8s” in the InfluxDB. The to the deployed applications using the service IPs, which
Grafana deployed in our system will send queries to are saved in the file generated in step 2;
retrieve the data stored in the “k8s” database and display Step 4. We make JMeter report the results of the requests in
these data in graphs. a designated file in CSV format. In this file, each line
2) Application performance metrics monitoring: Here, we shows the information, including the application URL,
consider the application response time as a representative response time, response code, and so on, of an individual
application performance metric. To monitor application re- request;
sponse time, we install Apache JMeter v.2.13 [8] in our Step 5. After JMeter finishes sending requests and generates
system. Apache JMeter is an open source application written the result file, our JMeter Invoker program will then load
in Java. It is designed for workload testing and performance the result from the JMeter result file and calculate the
measurement of multiple kinds of servers, such as web servers, average response time for all of the deployed applications;
ftp servers, database servers, mail servers, and so on. Users Step 6. Finally, the JMeter Invoker program outputs the av-
can describe the behavior of workload testing in a JMeter test erage response time of each deployed application into a
plan and save the test plan as a JMX file. By executing JMeter file in JSON format.
with the test plan file, JMeter will send the requests to the These six steps will be executed in an infinite loop. Thus,
designated application accordingly. Besides, JMeter can output each iteration will update the application response time in the
the information of each requests into a CSV file. Users can final output file.
employ this output file to analyze the performance of the tested
application. B. The Data Aggregator Module
To implement the application performance metrics moni- The data aggregator is used to integrate the system status
toring, we implement a program, JMeter Invoker, to invoke (including nodes and pods), system resource metrics, and
Apache JMeter and make its behavior in accordance with our application performance metrics. The operation of the data
expectations. The steps of application performance metrics aggregator is shown in Figure 4. The requisite steps are as
monitoring are shown in Figure 3; They are as follows: follows:
Step 1. We use the Kubernetes CLI, kubectl, to retrieve Step 1. Fetching information in our system using the
 KubernetesNodes
(2) Resource
Kubernetes
Data (1) Scheduler
Master
Pod Pod Pod
Aggregator Kubernetes Container Container Container
etcd
Master
label1=app1
 label1=app1
 label1=app1

(3) (4) Node (1)
label2=ver1 label2=ver1 label2=ver1
őŰť
Pod őŰť
Pod Scheduler
etcd
įįį InfluxDB
Heapster ńŰůŵŢŪůŦų (2)
Grafana Pod
Scaler APIServer
Application Scheduler (3) Service:app1 ReplicationController:app1
cAdvisor Kubelet Controller
responsetime Manager label1=app1 label1=app1
... label2=ver1 
label2=ver1
LinuxOS
APIServer LinuxOS

Controller
Fig. 5: Pod Scaler.
Dataaggregator Manager
Report
LinuxOS the application is considered in a light-load period. Therefore,
we decrease one pod for the application. Algorithm 1 can be
Fig. 4: The operation of data aggregator module.
replaced by a newly designed algorithm.

Algorithm 1 Default Resource Provisioning Algorithm

Kubenetes CLI, kubectl. The information concerning the
Input: The set of applications deployed in our system, 𝐴𝑛 ;
nodes, namespaces, pods, and services is retrieved from
Output: The number of pods of each application;
the overall system. For each namespace, we compare the
1: for each application 𝐴𝑖 deployed in our system do
label of each pod and each service and then record the
2: 𝑈𝑐𝑝𝑢 ← the average CPU utilization ratio of 𝐴𝑖
mapping of pods and services;
3: 𝑈𝑚𝑒𝑚 ← the average memory utilization ratio of 𝐴𝑖
Step 2. We use the InfluxDB HTTP API to send the In-
4: 𝑁𝑝𝑜𝑑 ← the number of pods of 𝐴𝑖
fluxDB Query Language (InfluxQL) to the k8s database
5: if 𝑈𝑐𝑝𝑢 ≥ 0.7 OR 𝑈𝑚𝑒𝑚 ≥ 0.7 then
to retrieve the stored metrics values. The format of the
6: 𝑁𝑝𝑜𝑑 = 𝑁𝑝𝑜𝑑 + 1
InfluxDB HTTP API is shown as follows:
7: else if 𝑈𝑐𝑝𝑢 ≤ 0.4 AND 𝑈𝑚𝑒𝑚 ≤ 0.4 then
curl -G ’http://<InfluxDB IP>:8086/query?pretty= 8: if 𝑁𝑝𝑜𝑑 >1 then
true’ --data-urlencode "db=k8s" --data-
urlencode "q=<InfluxQL query>"
9: 𝑁𝑝𝑜𝑑 = 𝑁𝑝𝑜𝑑 − 1
10: end if
We use the InfluxQL to list the metric values every 10 11: end if
seconds over the past 6 hours and retrieve the newest 12: POD S CALER(𝐴𝑖 , 𝑁𝑝𝑜𝑑 )
value. 13: end for
Step 3. Fetching the application performance metrics from
the output of the JMeter Invoker output file; and D. The Pod Scaler Module
Step 4. Integrating all of the information gathered in the
The pod scaler module is responsible for scaling the
previous steps into a result file in JSON format. The
applications according to the resource provisioning strategy
integrated output file is divided into two parts: The first
determined by the resource scheduler. Figure 5 shows the
part is the server information in the system, while the
operation of the pod scaler. The steps are further described
other is the application information in the system.
as follows:
C. The Resource Scheduler Module Step 1. Receiving the dynamic resource-provisioning strategy
from the resource scheduler module;
The resource scheduler is used to implement the resource- Step 2. Asking the master for all the replication controller
provisioning algorithm. We have implemented a simple information and then parsing this information to identify
resource-provisioning algorithm, algorithm 1, as the default the corresponding replication controller. We compare the
algorithm in our system. The CPU utilization ratio of a pod labels of each pod and each replication controller to find
is its current CPU usage divided by its CPU limit. 𝑈𝑐𝑝𝑢 out the corresponding pods and services of a replication
represents the arithmetic mean of the CPU utilization ratios controller; and
of all of the pods that are executed for a distinct applica- Step 3. Using the kubectl command to set a new size for the
tion. Similarly, 𝑈𝑚𝑒𝑚 represents the arithmetic mean of the replication controller of each application. This will gen-
memory-utilization ratios of all the pods that are executed for erate the resource provisioning according to the resource-
a distinct application. According to this default algorithm, if provisioning strategy.
the 𝑈𝑐𝑝𝑢 or 𝑈𝑚𝑒𝑚 of an application in the system is greater
than 70%, the application is considered in a heavy-load period. V. E XPERIMENTAL D EMONSTRATION
Then, we increase one pod for that application. If the 𝑈𝑐𝑝𝑢 and We used OpenShift Origin [9] as our experiment envi-
𝑈𝑚𝑒𝑚 of an application in the system are both less than 40%, ronment. This is an open source cloud application platform
 Software Version
the existing mechanism’s deficiencies. Specifically, our plat-
OpenShift Origin v1.2.0-rc1
Kubernetes v1.2.0-36-g4a3f9c5 form has the following features:
Docker 1.9.1 ∙ A comprehensive monitoring mechanism. Unlike the ex-
Operating System CentOS Linux release 7.1.1503 isting Kubernetes built-in mechanism that only considers
TABLE I: System environment. CPU utilization, our platform takes both system resource
utilization and application QoS metrics into account.
0.8 3 Thus, a rather complete set of monitored metrics can be
0.6
60000 2 exposed to the resource-provisioning algorithm, for better

(pods)
0.4
(requests/minute)

50000
1
40000 0.2 provisioning decisions;
30000 0 0
∙ Deployment flexibility. In our platform, we modularize the

11
21
31
41
51
61
71
81
91
1

101
111
20000
10000
0
(minutes) four steps of dynamic resource-provisioning, so that the
AverageCPUutilizationratio Averagememoryutilizationratio
users can easily deploy a newly designed algorithm to
11
21
31
41
51
61
71
81
91
1

101
111
121

currentnumofpod

(minutes)
(b) Resource utilization ratio replace an existing one without affecting other modules;
(a) Arrival rate. and number of pods ∙ Automatic operation. After the users deploy a desired
1.5 4
3
dynamic resource-provisioning algorithm, the resource-
1
(cores)

provisioning operation will be applied automatically and

(GiB)

2
0.5
1 periodically to all the running applications in our platform
0 0
following a user-defined time interval.
1
10
19
28
37
46
55
64
73
82
91
100
109
118

1
9
17
25
33
41
49
57
65
73
81
89
97
105
(minute) 113 (minutes)
TotalCPUusage TotalCPUlimit Totalmemoryusage Totalmemorylimit VII. ACKNOWLEDGMENT
(c) CPU. (d) Memory Shun-Ren Yang’s work was supported in part by Chunghwa
Fig. 6: Experiment: the ticket monster application. Telecom, and by Ministry of Science and Technology
(MOST), R.O.C., under grant numbers MOST 105-2221-
E-007-089, MOST 105-2218-E-009-029. Phone Lin’s work
proposed by Red Hat that was developed based on Docker was supported in part by Chunghwa Telecom, and by
(for container packaging) and Kubernetes (for cluster man- Ministry of Science and Technology (MOST), R.O.C.,
agement). Following the OpenShift Origin requirement, we under grant numbers MOST 106-2923-E-002-005-MY3,
installed CentOS as the OS onto each machine. OpenShift MOST 104-2923-E-002-005-MY3, MOST 103-2221-E-002-
Origin allows us to use the Kubernetes API directly to control 249-MY3, MOST 103-2221-E-002-152-MY3, MOST 105-
all of the Kubernetes components. In our experiment environ- 2627-E-002-001, MOST 105-2622-8-009-008, MOST 105-
ment, we had one master and one node that were both installed 2218-E-002-013, MOST 105-2218-E-009-029, MOST 105-
on the same machine. The software versions are provided 2511-S-002-006-MY2, MOST 106-2221-E-002-031, MOST
in Table I. We invoked the dynamic resource-provisioning 106-2218-E-002-029.
procedure every 1 minute.
R EFERENCES
Over this experiment environment, we deployed a JBoss
sample application, Ticket Monster [10], as the testing appli- [1] Cloud Foundry Foundation. Hope Versus Reality: Con-
cation. Ticket Monster is an online ticketing demo application tainers In 2016. Technical report, June 2016. URL: https:
that users can use to book tickets for concerts, sports games, //www.cloudfoundry.org/learn/2016-container-report/.
and so on. We wrapped the application in a Docker image. [2] Docker. [Online]. Available: https://www.docker.com/.
We then deployed the Ticket Monster into a pod using this [3] Kubernetes. [Online]. Available: http://kubernetes.io/.
image. We used JMeter on another machine to send the HTTP [4] E. Casalicchio and L. Silvestri. Architectures for au-
request to the Ticket Monster application. The duration of the tonomic service management in cloud-based systems.
experiment was 2 hours; during this time, we made the arrival Computers and Communications (ISCC), 2011 IEEE
rate of HTTP requests increase linearly. The Ticket Monster Symposium on, pages 161–166, June 2011.
received nearly 50,000 requests per minute at the end of the [5] Heapster. [Online]. Available: https : / / github . com /
experiment, as shown in Figure 6a. The comparison of the kubernetes/heapster.
CPU utilization ratio, memory utilization ratio, and number of [6] Influxdb. [Online]. Available: https : / / influxdata . com /
executing pods is shown in Figure 6b. Here, we can observe time-series-platform/influxdb/.
that when the memory utilization reached 70%, the system [7] Grafana. [Online]. Available: http://grafana.org/.
increased by one pod. Figures 6c and 6d show the relationship [8] Apache jmeter. [Online]. Available: http : / / jmeter .
of usage quantity and limit quantity of CPU and memory, apache.org/.
respectively. [9] Openshift origin. [Online]. Available: https : / / www .
openshift.org.
VI. C ONCLUSIONS [10] Ticket monster tutorial. [Online]. Available: http : / /
This paper developed a generic platform to facilitate dy- developers.redhat.com/ticket-monster/.
namic resource provisioning over Kubernetes, while resolving