Received April 12, 2019, accepted May 10, 2019, date of publication May 17, 2019, date of current

version May 30, 2019.

Digital Object Identifier 10.1109/ACCESS.2019.2917532

Semi-Supervised K-Means DDoS Detection Method

Using Hybrid Feature Selection Algorithm
YONGHAO GU , KAIYUE LI, ZHENYANG GUO, AND YONGFEI WANG
Beijing Key Laboratory of Intelligent Telecommunication Software and Multimedia, School of Computer Science, Beijing University of Posts and
Telecommunications, Beijing 100876, China
Corresponding author: Yonghao Gu ([email protected])
This work was supported in part by the National Natural Science Foundation of China under Grant U1536121, Grant 61370195, and
Grant 61873040, in part by the China Computer Federation-NSFOCUS (CCF-NSFOCUS) KunPeng Research Fund under Grant 2018004,
in part by the Fundamental Research Funds for the Central Universities, in part by the Communication Soft Science Foundation of Ministry
of Industry and Information under Grant 2015-R-29, in part by the State Grid Technology Project under Grant SGTYHT/15-JS-191, and in
part by the Key Lab of Information Network Security Foundation of Ministry of Public Security.

ABSTRACT Distributed denial of service (DDoS) attack is an attempt to make an online service unavailable
by overwhelming it with traffic from multiple sources. Therefore, it is necessary to propose an effective
method to detect DDoS attack from massive data traffics. However, the existing schemes have some
limitations, including that supervised learning methods, need large numbers of labeled data and unsupervised
learning algorithms have relatively low detection rate and high false positive rate. In order to tackle these
issues, this paper presents a semi-supervised weighted k-means detection method. Specifically, we firstly
present a Hadoop-based hybrid feature selection algorithm to find the most effective feature sets and propose
an improved density-based initial cluster centers selection algorithm to solve the problem of outliers and
local optimal. Then, we provide the Semi-supervised K-means algorithm using hybrid feature selection
(SKM-HFS) to detect attacks. Finally, we exploit DARPA DDoS dataset, CAIDA ‘‘DDoS attack 2007’’
dataset, CICIDS ‘‘DDoS attack 2017’’ dataset and real-world dataset to carry out the verification experiment.
The experiment results have demonstrated that the proposed method outperforms the benchmark in the
respect of detection performance and technique for order preference by similarity to an ideal solution
(TOPSIS) evaluation factor.

INDEX TERMS DDoS attack, semi-supervised k-means, Hadoop-based hybrid feature selection, ratio of
average sum of squared errors (SSE) to cluster distance (RSD), TOPSIS.

I. INTRODUCTION for ordinary customers to enter, disrupting trade. Nowadays

In computing, a denial-of-service attack (DoS attack) is a DDoS attacks are rising significantly and high-volume attack
cyber-attack in which the perpetrator seeks to make a machine events have occurred quite frequently, which have a lot to do
or network resource unavailable to its intended users by with the prevalence of Internet of Things (IoT) botnets, such
temporarily or indefinitely disrupting services of a host con- as Mirai botnet.
nected to Internet. DoS is typically accomplished by flooding
the targeted machine or resource with superfluous requests in
A. MOTIVATION
an attempt to overload systems and prevent some or all legiti-
To prevent against DDoS attacks, researchers have proposed
mate requests from being fulfilled. In a distributed denial-of-
and implemented various countermeasures, including detec-
service attack (DDoS attack), incoming traffic flooding the
tion, defense and traceback. Among all these countermea-
victim originates from many different sources. This effec-
sures, DDoS detection is the first and most important step
tively makes it impossible to stop attack simply by blocking
in fighting against DDoS attacks. There are two classes of
a single source. A DDoS attack is analogous to a group of
DDoS detection techniques: misuse detection and anomaly
people crowding the entry door of a shop, making it hard
detection. Misuse detection techniques try to detect attack
The associate editor coordinating the review of this manuscript and by comparing the current activity of destination network to
approving it for publication was Zesong Fei. a database of known attack signatures. But, these techniques
2169-3536
2019 IEEE. Translations and content mining are permitted for academic research only.
VOLUME 7, 2019 Personal use is also permitted, but republication/redistribution requires IEEE permission. 64351
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
 Y. Gu et al.: Semi-Supervised K-Means DDoS Detection Method Using Hybrid Feature Selection Algorithm

are difficult to detect new attacks. So, anomaly detection detection methods. Machine learning methods mainly
techniques are introduced to detect unknown attacks by com- include unsupervised learning and supervised learning.
paring the current activity of destination network to an estab- Unsupervised learning techniques deal with learning tasks
lished normal activity represented as a profile. The basic with unlabeled or untagged data, and clustering is the most
detection approach is to use machine learning to create a popular unsupervised learning technique. K-means algo-
model of trustworthy activity, and then compare new behavior rithm, as a clustering method, has been successfully used
against this model. Machine learning methods mainly include to detect anomalies [1] and DDoS [2], and some modified
supervised learning and unsupervised learning. Machine k-means methods [3], [4] are provided to improve detection
learning-based detection methods have the following limita- efficiency. Besides, there are some other unsupervised learn-
tions. Insufficient labeled data of supervised learning meth- ing methods to detect DDoS attacks [5].
ods lead to low detection rate, and unreasonable initialization Meanwhile, many supervised learning algorithms are used
of unsupervised learning parameters leads to local optimal for DDoS detection [6]. Nguyen et al. [7] uses k-NN clas-
or poor detection effect. In addition, too many features in sifier method and cosine formula based algorithm to detect
learning process cause ‘‘the curse of dimensionality’’, and DDoS attacks. Xiaoet al. [8] presents a CKNN (KNN with
unreasonable feature sets lead to poor detection performance. Correlation analysis) detection method, which exploits cor-
In order to overcome the above limitations, this paper pro- relation information of training data to improve classification
poses a semi-supervised clustering detection method using accuracy. Vijayasarathy et al. [9] uses multiple Bayesian
hybrid feature selection algorithm, and the provided method classifiers to detect DDoS attacks. However, naive Bayes is
uses only small amount of labeled data and relatively large based on a very strong independence assumption, which is
amount of unlabeled data to detect DDoS attack behavior. not always satisfied. Bouzida et al. [10] proposes a decision
tree-based detection method and exploits KDD99 dataset for
B. CONTRIBUTIONS model training and testing to obtain 93% detection rate. But,
The main contributions of this paper can be summarized as this model requires a large number of labeled data for effec-
follows: tive training. Li et al. [11] demonstrates a DDoS detection
1) It presents a hadoop-based hybrid feature selection system based on LVQ neural network to improve accuracy.
method combined with subsequent learning process to Cheng et al. [12] proposes a flow correlation degree feature
find the most effective feature set. (Section III) and applies a random forest detection model, which has a
2) It proposes a semi-supervised weighted k-means 98.57% detection rate and a 2.72% false positive rate. With
method using hybrid feature selection algorithm increase of labeled samples, detect accuracy is improved.
(SKM-HFS) to achieve better detection performance, Khundrakpam et al. [13] detects DDoS attack using a mul-
and this method requires fewer labeled data sets for tilayer perceptron (MLP) classification method with genetic
training. (Section IV) algorithm.
3) It provides an improved density-based initial clus- In summary, supervised learning method has better accu-
ter centers selection method to solve the problem racy. But, one limitation of it is the need of large-scale labeled
of outliers and local optimal of k-means clustering. data to train the classifier, which is not easy to get. Unsu-
(Section IV) pervised learning has the advantage of detecting new sam-
4) It exploits DARPA DDoS dataset, CAIDA ‘‘DDoS ples better than supervised learning. However, the manually
attack 2007’’ dataset, CICIDS ‘‘DDoS attack 2017’’ assignment of cluster numbers and other parameters could
dataset and real-world dataset to verify that the pro- result in relatively low accuracy.
posed method outperforms the benchmark in the In addition to the above machine learning methods, chaos
respect of detection performance and TOPSIS evalu- theory is used in DDoS detection in recent years [14]–[16]
ation factor. (Section V) and has a nice detection performance. This paper mainly deals
The remainder of this paper is organized as follows. with the limitations of supervised learning and unsupervised
In Section II, we review the related works about DDoS detec- learning detection methods, and also compares the detection
tion methods and feature selection methods in DDoS detec- performance between the proposed method and the chaos
tion. Section III provides the hybrid feature selection method theory based method.
and Section IV proposes the semi-supervised k-means DDoS
detection algorithm using hybrid feature selection method, B. FEATURE SELECTION METHODS
and Section V shows the experiment details and gives the The research on machine learning based DDoS detection
experiment results and analyses. Finally, conclusions and method not only focuses on detection models, but also
future work are provided in Section VI. includes the feature selection methods. Related papers of
feature selection methods in DDoS detection are shown as
II. RELATED WORKS follows.
A. DETECTION METHODS Yusof et al. [17] combines the consistency subset eval-
There are many DDoS attack detection methods, while uation (CSE) and DDoS characteristic features (DCF) for
this paper mainly focuses on machine learning based feature selection, which is superior to traditional features

64352 VOLUME 7, 2019

Y. Gu et al.: Semi-Supervised K-Means DDoS Detection Method Using Hybrid Feature Selection Algorithm

selection method such as Information Gain, Gain Ratio, packet type (H (PacType)), occurrence rate of TCP packet
Correlated features selection(CFS), but this paper does not (TCPRate), occurrence rate of UDP packet (UDPRate) and
tackle feature redundancy. Balkanli et al. [18] employs the occurrence rate of ICMP packet (ICMPRate), time interval
Chi-Square and Symmetrical Uncertainty with Decision Tree of packets (PckTimeInt). Many other features are variants
classifier to detect backscatter DDoS behaviors, which needs of the above 13 features, or derived from them. Therefore,
large number of labeled features. Zi et al. [19] provides the it is practical to compare the detection effects between these
linear correlation coefficient for feature ranking and Modified 13 features as candidate features and select the final feature
Global K-means algorithm(MGKM) to detect attacks. As the sets from them.
number of top-ranked features decreases, a point where the In order to verify the performance of each feature while
cluster function value drops heavily will be chosen and reducing the number of candidate features, we compare the
the final selected features will be identified. However, this changes of each feature value before and under DDoS attacks.
method could not capture correlations between features that The DARPA DDoS dataset [35] is used in comparison experi-
are not liner in nature. Jiang et al. [20] proposes the filter ments. The variation of each feature is shown in the following
algorithm GAIG for feature selection by combing genetic figures (Fig. 1 and Fig. 2). The first half (left of the dotted
algorithm as the search strategy and information gain as the line) of each figure shows the feature value fluctuation with-
evaluation function, which could reduce the noisy features. out attack, and the second half (right of the dotted line) shows
However, the provided genetic search strategy has a high time the feature value fluctuation under attack.
complexity. Osanaiye et al. [21] presents an Ensemble-based As we can see from these figures, the larger the relative
Multi-Filter Feature Selection (EMFFS) method combining difference of feature values between left part and right part
Information Gain, Gain Ratio, Chi-square and ReliefF, which is, the more obvious the distinction between normal traffic
has a high computational complexity. and attack traffic is, the more effective the feature is. Fig. 1
From the above analysis, it is necessary to provide a rea- shows the values of 9 features with better performance, and
sonable feature selection algorithm before model training to Fig. 2 shows the other 4 feature values with relatively poor
achieve effective attack detection. performance.
In addition, this paper focuses on feature selection method
III. HYBRID FEATURE SELECTION METHOD rather than extracted features. If other good features are
As we know, there are many features used for DDoS detec- found, they can be added to the corresponding candidate
tion. While, too many features in the learning process cause feature set to participate in the proposed feature selection
‘‘the curse of dimensionality’’, and unreasonable feature sets process. Therefore, this paper chooses these 9 features in
lead to poor detection performance. This paper proposes a Tab. 1 as the candidate feature set.
hybrid feature selection method for DDoS detection. The
TABLE 1. Main features used in DDoS detection methods.
method includes three steps, namely data normalization, fea-
ture ranking and feature subset searching. The input of this
method is the candidate feature set and the output is the
selected feature set for detection model.

A. CANDIDATE FEATURE SET

Through investigation of related works, we find that detec-
tion features mainly include: entropy [24]–[27], conditional
entropy [28], Renyi entropy [29], ϕ-entropy of source ip
(destination ip, protocol) [30], occurrence rate of TCP packet
(UDP packet, ICMP packet) [25], percent of packets with
the port number 80, variance of the numbers of packets
to each destination ip, average of payloads, probability of
occurrence of IP [31], mean time intervals (MTI), TTL,
time stamp, ACK value, SYN value [32], variation index of
source IPs [33], answer resource record, authority resource Why use entropy of traffic packets field as detection
record, average packet size [34] and etc. Among the above feature in many papers? Lakhina et al. [36] found that
38 features, the most widely used features are the following each kind of anomalies affects the distribution of certain traf-
13 ones: entropy of source ip (H (Sip)), entropy of destination fic features. In one case, some feature distributions become
ip (H (Dip)), entropy of source port (H (Sport)), entropy of more dispersed (e.g. source IP address in DDoS), while other
destination port (H (Dport)), conditional entropy of source ip feature distributions become concentrated (e.g. destination
given destination ip (H (Sip | Dip)), conditional entropy of IP address in DDoS) on a small set of values. We need
source ip given destination port (H (Sip | Dport)), conditional to find some statistic metrics to quantify the distribution
entropy of destination port given destination ip (H (Dport | of traffic features. Generally, entropy refers to disorder or
Dip)), One-Way Connection Density (OWCD), entropy of uncertainty, and the definition of entropy used in information

VOLUME 7, 2019 64353

 Y. Gu et al.: Semi-Supervised K-Means DDoS Detection Method Using Hybrid Feature Selection Algorithm

FIGURE 2. Four features with poor detection effect.

X means the variable of one network traffic feature, which has

n values xi (i = 1, . . . , n), and P(xi ) represents the probability
Pn
of each value, satisfying P(xi ) = 1. If we use entropy as
i=1
a detection feature, we can distinguish between normal and
abnormal behavior by getting all entropy values of the traffic
feature during a period of time. The calculation method of
OWCD can be seen in our previous work [22].

B. DATA NORMALIZATION
Before making feature selection, data normalization is an
essential process, which scales the value of each feature into
a well-proportioned range, so that the bias in favor of fea-
tures with greater values is eliminated from the dataset. The
normalization transforms each feature value linearly scaled
to the one in the range of [0], [1] using the equation (2):

xmj − min xmj

xmj ∈Xi
xmj = , (2)
max xmj − min xmj
xmj ∈Xi xmj ∈Xi

in which max xmj and min xmj respectively stands for the
xmj ∈Xi xmj ∈Xi
maximum and minimum value of the mth feature. The process
FIGURE 1. Nine features with better detection effect.
of data normalization is described as follows.
1) Collect the necessary metadata from the actual dataset,
theory is directly analogous to the definition used in statistical
including data record number, time, protocol type,
thermodynamics. Entropy could be used as such a metric to
source IP address, destination IP address, source port
detect DDoS attacks effectively, which represents the random
number, and destination port number, which are stored
feature of network traffic. It describes the degree of concen-
in the txt or csv file.
tration and dispersal characteristic of traffic. Entropy is the
2) The above file is processed by sliding window
measure of information and uncertainty of a random variable.
principle.
The entropy of variable X can be defined as:
3) Use the metadata of each sliding window unit in the
n
X above file to calculate the feature values in Tab. 1 and
H (X ) = − P(xi ) log2 (P(xi )) (1) form the value set for each feature.
i=1 4) The value of each feature is normalized by equation (2).

64354 VOLUME 7, 2019

Y. Gu et al.: Semi-Supervised K-Means DDoS Detection Method Using Hybrid Feature Selection Algorithm

k−1 X
k
C. FEATURE RANKING X
ci − cj 2 .

=
After feature normalization, all candidate features will be
i=1 j=i+1
ranked by filter models. Traditional filter methods analyze
features independent of the classifier and use ‘‘goodness’’ The larger the dinter−cluster is, the lower the inter-cluster
metric to decide which features should be kept. They sort similarity is.
each feature through specific evaluation index, and ‘‘poor’’ Since the RSD is proportional to the intra-cluster dis-
ranking index may not have ideal accuracy in attack detection. tance and inversely proportional to the inter-cluster distance,
In this paper, the filter process is combined with subsequent the RSD of each feature ft is computed as
learning algorithm, and a novel ranking index is proposed by
SSE
using the objective function SSE (Sum of Squared Errors) RSD (ft ) = .
k−1 k
of k-means method. The definition of the provided ranking P P 2
n· ci − cj
index is as follows: i=1 j=i+1
Definition 1: Ratio of average SSE to cluster
Distance (RSD) The smaller the RSD (ft ) is, the better the clustering result
The datasets X = {x1 , . . . , xn } are clustered by k-means is by using the feature ft , which means the feature is much
algorithm based on each feature ft (t = 1, 2, ..., l), and the better for detecting attacks.
center of each cluster Ci (i = 1, 2, · · · , k) is expressed as ci . So, the candidate feature set is sorted by RSD (ft ) and
So the ratio of average SSE to the sum of the distances features whose RSD values are above a given threshold are
between each pair of centers is obtained, which is shorted discarded. And finally, the initial feature subset is obtained
0
as RSD (ft ) as F = {f1 , f2 , · · · , fl 0 } from the candidate feature set F =
{f1 , f2 , · · · , fl }. The process of feature ranking is described as
SSE
RSD (ft ) = , (3) follows.
k−1 k 2
1) The normalized candidate feature set is clustered by
P P
n· ci − cj
i=1 j=i+1 k-means method.
and SSE is computed using the following equation (4) 2) According to the clustering results, k clustering centers
{ci (i = 1, 2, · · · , k)} of each feature value and SSE are
k X
X obtained.
SSE = kci − xk2 . (4) 3) Calculate the RSD values of each feature using
i=1 x∈Ci
equation (3).
Why we useRSDas the feature ranking index? 4) The RSD values are arranged in ascending order, and
The evaluation criterion of good clustering model is having the first l 0 features are selected as initial feature set or
both high intra-cluster similarity and low inter-cluster similar- the corresponding feature set whose RSD values are
ity. Intuitively, the proposed ranking index should be propor- less than a given threshold are selected as the initial
0
tional to the intra-cluster distance and inversely proportional feature set F = {f1 , f2 , · · · , fl 0 }.
to the inter-cluster distance.
For the intra-cluster similarity, SSE in k-means algorithm D. FEATURE SUBSET SEARCHING
0
characterizes the clustering degree of intra-cluster data. The After getting the initial feature set F , we need a proper search
dataset X is clustered by k-means algorithm, and the center strategy to find the best feature subset for detection model.
of each cluster Ci is expressed as ci . The mean intra-cluster In order to reduce the time complexity, we use the Sequential
distance of X in the k clusters is Forward Selection (SFS) algorithm. SFS is a search method
k that starts with an empty set of features and adds a single
1 1XX 0
dintra−cluster = SSE = kci − xk2 . feature from F in each iteration with the fitness function f (x)
n n monotonously increasing or decreasing. The SFS algorithm
i=1 x∈Ci
has following steps:
The smaller the dintra−cluster is, the higher the intra-cluster 00 00
1) Initialize the empty feature subset F : F = {∅};
similarity is. 0 00
2) Select a feature fi from F and add it to F , satisfying
For the inter-cluster similarity, the distance between any
fitness function
two clusters Ci and Cj is expressed by the distance between  00    00   00 
the centers of the two clusters as f Fk = min f Fk−1 , f1 , f Fk−1 , f2 , · · · ,
2
dinter−cluster (Ci , Cj ) = ci − cj .
 00 
f Fk−1 , fl 0 −k+1 ,
So, the inter-cluster similarity of k clusters is expressed by
00
the sum of all the cluster-cluster distances as in which Fk stands for the current selected feature
k−1 X
k subset after the k th iteration;
00 00
3) If f (Fk ) > f (Fk−1 ), the algorithm is stopped, and Fk−1
"
X
dinter−cluster = dinter−cluster (Ci , Cj ) 0
i=1 j=i+1 is the final selected feature subset; Else if k = l ,

VOLUME 7, 2019 64355

 Y. Gu et al.: Semi-Supervised K-Means DDoS Detection Method Using Hybrid Feature Selection Algorithm

the algorithm is stopped, and Fk" is the final selected Algorithm 1 Hadoop-Based Hybrid Feature Selection
feature subset; Else return 2. Algorithm.
Furthermore, we need to find a fitness function f (x) with Input: Threshold θ, set of data points X =
monotonous decreasing. As we know, DDoS detection effect {x1 , x2 , · · · , xn }, set of labeled data L ⊂ X , candidate
is mainly evaluated by two indexes: True Positive Rate (TPR) feature set F = {f1 , f2 , · · · , fl }, and each feature’s values
and False Positive Rate (FPR). TPR also called the Recall and labels are stored in each feature_file
00
Rate (RR) in some fields, measures the proportion of actual Output: Selected feature subset F = {f1 , f2 , · · · , fl 00 }
positives (DDoS attacks) that are correctly identified as Method:
such. FPR is calculated as the ratio between the numbers of 1. MapTask1:
negative events (normal traffics) wrongly categorized as pos- Context. write(key, value) //construct
itives and the total number of actual negative events. The fit- <key, value> pairs for the feature_file
ness function f (x) of feature selection in DDoS attacks should 2. ReduceTask1:
be related to these two evaluating indexes RR and FPR. (2.a) Normalize each data point in L and get dataset
0
So, we need to find a method or function to consider these L
two evaluation indexes simultaneously, which is a multi- (2.b) Calculate RSD(ft ) for each ft (t = 1,2,. . . ,l) in
0
criteria decision-making problem. TOPSIS [37] is exactly dataset L
a multi-criteria decision-making approach. This paper uses 3. Arrange RSD values in the ascending order
TOPSIS method, whose value is expressed by T(RR, FPR) 4. if RSD(ft ) < θ (for t = 1,2,. . . ,l)
0
as the evaluation factor. T(RR,FPR) is set to 1-RR+FPR to add ft to F
represent the fitness function of the SFS process. The smaller else
the T(RR,FPR) value is, the selected feature subset is much discard ft
better for detecting attacks. 5.Repeat
(5.a) MapTask 2:
E. HYBRID FEATURE SELECTION ALGORITHM Context. write(key, value) //construct <key,
This paper exploits parallelism to provide a hadoop-based value> pairs for the filter_file
feature selection method shown in Fig. 3, and the correspond- (5.b) ReduceTask2:
0 00
ing algorithm is shown as Algorithm 1. In this algorithm, Select a feature fi from F and add it to F ,
the parameter key in the step 1 and step 5.a represents the satisfying the value of T (RR,FPR)
label, whose values include 0 (labeled attack data), 1 (labeled is minimum for the set X
normal data), 2 (unlabeled data) and 3 (test data). The (5.c) if the value of T (RR,FPR)
parameter value represents the feature value corresponding is decreased after adding the new feature fi in 5.a
to each key.
update filter_file
return 5.a
else
end Repeat
00
return F

is depicted in Fig. 4. The detection framework consists

of four major phases: (1) data preparation, where labeled
and unlabeled training data and test data are prepared,
(2) data preprocessing, where training and test data are pre-
processed and important features are selected using pro-
posed hybrid feature selection method, (3) model training,
where the model is training using proposed SKM-HFS, and
(4) attack detection and evaluation, where the trained model
is used to detect DDoS attacks on test data and several
FIGURE 3. The flow chart of hadoop-based hybrid feature selection
method. indexes are used to evaluate detection performance. The
contributions of this paper are shown as red font in Fig. 4.
IV. SEMI-SUPERVISED K-MEANS DETECTION The proposed hybrid feature selection method is shown in
ALGORITHM USING HYBRID FEATURE Section III. This section will focus on the proposed detec-
SELECTION METHOD tion model, which is based on the semi-supervised weighted
By using the aforementioned hybrid feature selection method, k-means method using hybrid feature selection algorithm
the framework of our proposed DDoS detection prototype (SKM-HFS).

64356 VOLUME 7, 2019

Y. Gu et al.: Semi-Supervised K-Means DDoS Detection Method Using Hybrid Feature Selection Algorithm

FIGURE 4. The framework of the proposed DDoS detection prototype.

A. SEMI-SUPERVISED K-MEANS CLUSTERING B. PROPOSED INITIAL CLUSTER CENTERS

K-means algorithm is based on iterative relocation that parti- SELECTION METHOD
tion a dataset into k clusters, locally minimizing the average As we know, the final clusters of k-means are highly depen-
squared distance between the cluster data points and the dent on the initial centers they are fed, and the random
cluster centers. The objective function based on this distance selection of the initial cluster center is easy to cause the
has been shown as equation (4). However, k-means clustering local optimization of the algorithm. While, our previous study
has the following disadvantages: (1) the selection of k value called MF-CKM DDoS detection method [22] uses the mean
is very difficult to estimate; (2) the features of the cluster- value of the labeled dataset as the initial center, which solves
ing algorithm are equal weighting, which is inappropriate the local optimal problem to some extent, but it has a high
sometimes; (3) the initial cluster centers of the algorithm are sensitivity to the outlier data.
randomly selected, and the selection of the center has a great To solve the outlier problem, Wang and Liu [23] provides
influence on the clustering results. a point density-based method to select the initial cluster
As we know, k stands for the clustering number. This centers for intrusion detection. However, this method does
paper only needs to distinguish two traffics, normal and not consider the situation when there is more than one point
DDoS attacks. Besides, we use two categories of datasets with maximum density, such as Fig. 5. This method will
to train our model, which are Inside Sniffer-Phase 5 of randomly select one point with the maximum density as the
Lincoln Laboratory Scenarios (DDoS) 1.0 as attack dataset, initial cluster center, which will cause the initial center offset.
and Week 1 of 1999 training data (attack free) as normal
dataset. So, the clustering number k is set to two.
In order to solve the last two disadvantages of k-means
algorithm, we use semi-supervised weighted k-means algo-
rithm, which uses a small amount of labeled data to constrain
the selection of the initial center points, and improve the
classification accuracy of algorithm. With respect to feature
weighting, this paper uses the RSD value defined by equa-
tion (3) to assign the corresponding weight to each feature.
The smaller the RSD value is, the greater important the corre-
sponding feature is. So the weights of selected feature subset
F 00 = {f1 , f2 , . . . , fl 00 } are computed using equation (5).
(RSD (f1 ))−1
W = w1 , w2 , · · · , wf = PT

i=1 (RSD (ft ))
−1

(RSD (f2 ))−1

× PI ,··· ,
i=1 (RSD (ft ))
−1
FIGURE 5. A scene with more than one maximum density point in a
(RSD (fi ))−1 cluster.
× Pi } (5)
i=1 (RSD (fr ))
−1
To solve this problem of initial center offset, this
The object function SSE of equation (4) is changed as paper provides an improved density-based method, given in
equation (6). Algorithm 3, in which the calculation method of the proposed
algorithm parameter (Radius λ) is shown in Algorithm 2. The
00
k X X
X l
SSE = kwm · (cmi − xm )k2 (6) proposed method is based on the concept of point density as
i=1 x∈Ci m=1 Definition 2.
VOLUME 7, 2019 64357
 Y. Gu et al.: Semi-Supervised K-Means DDoS Detection Method Using Hybrid Feature Selection Algorithm

Algorithm 2 Selection Algorithm of Radius (Take Labeled C. SEMI-SUPERVISED K-MEANS ALGORITHM USING
Normal Dataset for Example). HYBRID FEATURE SELECTION METHOD
Input: Dataset N with Normal Label This paper provides a semi-supervised clustering detec-
Output: Radius λ tion algorithm, which is named as Semi-supervised
Method: K-Means algorithm using Hybrid Feature Selection method
1. xi is the randomly selected data point in N , (SKM-HFS). This method uses the small amount of labeled
and calculate the Euclidean distance between xi data to guide the selection of initial cluster centers, and use
and other data points in N , expressed other unlabeled data to train and form clusters.
as di1 , . . . , di(n−1) , adds ni to dataset P The detection system based on SKM-HFS algorithm is
2. Arrange distance sets from di1 to di(n−1) divided into three parts: feature selection, model training and
in ascending order and store in dataset Q model testing. In the feature selection phase, feature subset is
3. k = (n − 1) /h; //h >1 selected using the proposed hybrid method in Section III.
λ = dik ; // set the dik as the initial λ
Q.clear(); // empty the dataset Q Algorithm 4 Semi-Supervised K-Means Algorithm Using
4. Repeat Hybrid Feature Selection Method (SKM-HFS).
(4.a) xj is another randomly selected data Input: Set of data points X = {x1 , x2 , . . . , xn }, set of
point from N -P, and calculate the Euclidean labeled data L ⊂ X , number of clusters k, selected feature
distance between xj and other data points in N , 00
subset F = {f1 , f2 , · · · , fl 00 } using Algorithm 1, feature
expressed as dj1 , . . . , dj(n−1) , adds xj to dataset P weights W using equation (5).
(4.b) Arrange above distance set in ascending order Output: Disjoint k partitioning {Xi }ki=1 of X such that
and store in dataset Q SKM-HFS objective function is optimized
(4.c) k = (n − 1) /h; z = djk ; Q.clear(); Method:
if z < λ, λ = z; 1. Obtain k initial cluster centers {ci } using
(4.d) if P = N , Algorithm2 (i = 1, . . . , , k)
end Repeat; 2. Repeat until algorithm convergence
else (2.a) For xj ∈ L, if xj ∈ Li assign xj to the cluster i.
return 4.a / L, assign xj to the cluster i∗ ,
For xj ∈
return λ l 00
(xmj − cmi ) · wm 2 )

satisfying i∗ = arg min(
P
i m=1
Algorithm 3 Improved Density-based Initial Cluster Cen- (2.b) Update the center of cluster i :
ters Selection Algorithm (Take Labeled Normal Dataset for P
Example). xj
xj ∈Xj
Input: Radius λ, Dataset N with Normal Label ci =
Output: The Initial Center c1 of the Normal Dataset |Xi |
Method: return {Xi }ki=1
1. Calculate the density of each data point
xi with the radius λ, which is expressed
respectively as D (x1 , λ) , . . . , D (xn , λ). (xi ∈ N , n In the model training phase, the SKM-HFS algorithm is
is the number of data points in N ) provided as Algorithm 4. Using the small number of labeled
2. Arrange the point density set in the
data, the initial cluster centers are calculated by Algorithm 2.
descending order D (xi , λ) ≥ D xj , λ ≥ · · · ≥ The equation (6) is used to calculate the similarity between
D (xz , λ) (i, j,. . . ,z ∈[1,2, . . . ,n]) other unlabeled data and the initial cluster centers until
3. If the data point of the maximum density is not this algorithm converges (SKM-HFS objective function is
unique, the mean value of all corresponding points optimized).
with the maximum density is taken as c1 , or else the In the detection phase, by capturing new data packets and
data point xi with the maximum density D (xi , λ) extracting features, the distances between the feature values
is c1 of new data and each cluster center are calculated. Then,
return c1 the new data is assigned to the closest cluster, which is based
on the distance.

V. EXPERIMENTAL RESULTS AND ANALYSIS

Definition 2: Given any point xi within an n-dimensional A. DATASET AND DATA PREPROCESSING
ball of radius λ, the density of point xi is the number of data We use three public datasets and one real-world dataset
points within radius λ, denoted by D (xi , λ). shown in the following experiments. In the proposed

64358 VOLUME 7, 2019

Y. Gu et al.: Semi-Supervised K-Means DDoS Detection Method Using Hybrid Feature Selection Algorithm

SKM-HFS detection method, we make use of labeled data 2) REAL-WORLD DATASET

and unlabeled data in the training process. The labeled data In order to test our proposed method in real environment,
is used to initialize the centers for each cluster, and unlabeled we use a server as the victim host. Several benign users
data is used to form the initial clusters. The labeled data normally visit the server at first and then three attackers
includes labeled attack data and labeled normal data. We will launch DDoS attacks to the server using Hping3 tool during
explain how to select training data and test data for each several minutes shown in Fig. 6. Meanwhile, Wireshark tool
dataset in the following. is used to capture all the traffics at the victim server including
legitimate and attack packets, saved as pcap file. This cap-
1) PUBLIC DATASETS tured dataset is fully labeled. We firstly exploit 30,000 records
of data as labeled attack dataset and 30,000 labeled normal
The DARPA DDoS dataset [35] is stored as binary file and
dataset. Secondly, we make the mixture of 100,000 attack
we use tShark tool to convert binary file to txt file, which
data and normal data removing labels as unlabeled data.
contains numbers of vectors formed as (data record num-
Finally, we exploit another 100,000 removing labels’ data,
ber, time, source IP, destination IP, source port, destination
mixed as test data.
port, protocol type). We use Inside Sniffer – Phase 5 of
Lincoln Laboratory Scenarios (DDoS) 1.0 as labeled attack
data, Monday’s Tcpdump data of Week 1 of 1999 DAPRA
Intrusion Detection Evaluation Data Set as labeled normal
data, Tcpdump data of Four-Hour Subset of Training Data
(1998 DARPA Intrusion Detection Evaluation Data Set) as
unlabeled data, and Monday’s Tcpdump data (First Week of
Test Data) as test data.
The CAIDA ‘‘DDoS attack 2007’’ dataset [38] is a
sequence of anonymized traffic traces from a DDoS attack on
4 August 2007 (20:50:08 UTC to 21:56:16 UTC) containing
approximately one hour traffic. The CAIDA ‘‘DDoS attack
2007’’ dataset is stored as pcap file and we use Python library
scapy to convert pcap file to txt file, which includes the
following protocol fields such as source IP, destination IP, FIGURE 6. Test network architecture of real environment.
source port, destination port, protocol type and etc. CAIDA
dataset only contains DDoS attack data. We firstly exploit
30,000 records in CAIDA as labeled attack data, and use When processing the above data files, the feature values
Monday’s Tcpdump data of Week 1 of 1999 DARPA Intru- are obtained by using the sliding window principle shown as
sion Detection Evaluation Data Set as labeled normal data. follows:
Secondly, we exploit Tcpdump data of Four-Hour Subset 1) Suppose that M packets are in the same window unit,
of Training Data (1998 DARPA Intrusion Detection Eval- calculate feature values of this unit.
uation Data Set) as unlabeled data. Finally, we make the 2) Remove the first packet of the window, add a new
mixture of remaining CAIDA data not in training set and packet, and get the new sliding window unit.
Tuesday’s Tcpdump data of Week 2 of 1999 DARPA Intru- 3) Calculate feature values of the new window unit.
sion Detection Evaluation Data Set as test data, which include 4) Return to step 2) until the end of packets.
50,000 records. The experiments are carried out using Eclipse neon.2,
The CICIDS ‘‘DDoS attack 2017’’ dataset [39] is acquired PyCharm and Hadoop 2.7.6 cluster running on a PC with
by the Information Security Centre of Excellence (ISCX) of Intel(R) Core (TM) i5-3210M, 2.50GHz CPU and 4.0GB
New Brunswick University in 2017 after a week of traffic cap- RAM.
ture, and DDoS attacks are part of the dataset. The CICIDS
‘‘DDoS attack 2017’’ dataset is stored as csv file and we B. RESULTS AND PERFORMANCE ANALYSIS
convert it to txt file, which includes source IP, destination IP, 1) FEATURE SELECTION
source port, destination port, protocol ID and etc. CICIDS is The features extracted from the above datasets are normal-
fully labeled dataset, including attack data and normal data. ized, and then the candidate features are sorted by RSD
We firstly exploit 30,000 abnormal data as labeled attack value using equation (3). The RSD values of all candidate
data, and 30,000 normal data as labeled normal data. Sec- features on the above four datasets are ranked shown in
ondly, we make the mixture of 30,000 attack data and normal Fig. 7 to Fig. 10.
data removing labels as unlabeled data. Finally, we exploit How to determine the threshold θ in Algorithm 1? In the fil-
another 50,000 removing labels’ data, mixed as test ter process of feature selection, the purpose of setting thresh-
data. old θ is to filter out features that contribute little (the larger

VOLUME 7, 2019 64359

 Y. Gu et al.: Semi-Supervised K-Means DDoS Detection Method Using Hybrid Feature Selection Algorithm

FIGURE 7. RSD values of candidate features on DARPA. FIGURE 10. RSD values of candidate features on real-world dataset.

of other features. That is to say, the RSD values of feature

subset are arranged incrementally, and remove the features,
whose RSD values increased sharply. That is, the slope of the
RSD curve is significantly larger than that of the previous one.
As shown in Fig. 7, Fig. 8 and Fig. 10, the slope at the dotted
line increases sharply, and the RSD value corresponding to the
dotted line is the threshold θ. So, we retain the feature whose
RSD value is less than the threshold value. Of course, when
there are few candidate features and no features contributed
obviously little to attack detection, these features can be
sorted without discarding, and all of them enter the feature
subset searching process.
FIGURE 8. RSD values of candidate features on CAIDA. As we can see from Fig. 7, RSD value of the features
on DARPA dataset rises sharply after the feature H (Dip),
so the candidate features H (Dport|Dip) and H (Sip|Dport) are
discarded, and the initial feature subset is F 0 = {OWCD,
H (Sip|Dip), H (Sip), H (Sport), H (Dport), H (Dip)}. In the
same way, we can get the initial feature subsets of other three
datasets shown in Fig.8 to Fig. 10.
After we get the initial feature subset F 0 , F 0 will be input
to the feature subset searching stage. The result of this stage
is always evaluated by detection rate (RR) and false positive
rate (FPR) simultaneously. In our algorithm, the detection
effects of different feature subsets are evaluated by the fitness
function T (RR,FPR) provided in Section III. The smaller the
T (RR,FPR) value is, the selected feature subset is much better
for detecting attacks.
FIGURE 9. RSD values of candidate features on CICIDS. The T (RR,FPR) of different feature subsets are shown
in Fig. 11 to Fig. 14. Fig. 11 shows that the minimum
T (RR,FPR) value is obtained by the third feature subset, and
the RSD value is, the little the contribution of corresponding the final selected feature subset is F 00 = {H (Sip), OWCD,
feature is) to attack detection and reduce the time of following H (Dport)}. In the same way, we can get the final feature
feature subset searching. The setting of fixed threshold often subsets of other three datasets shown in Fig. 12 to Fig. 14.
requires expert experience and domain knowledge. How- The selected features of the proposed method used for all the
ever, in different scenarios or datasets (e.g., the four datasets four datasets are shown in Tab. 2.
mentioned in this paper), threshold is not always a fixed Do different feature selection methods affect the detec-
value, so it is necessary to provide some methods to assist tion effect? In this section, the proposed hybrid feature
threshold setting. In this paper, we propose to determine the selection algorithm is compared with other feature selection
threshold according to the way that the RSD value of one methods [17]–[19], [21] for DDoS detection on the above
feature increases significantly compared with the RSD value datasets. We respectively implement the five different feature

64360 VOLUME 7, 2019

Y. Gu et al.: Semi-Supervised K-Means DDoS Detection Method Using Hybrid Feature Selection Algorithm

FIGURE 11. T(RR,FPR) of different feature subsets on DARPA.

FIGURE 14. T(RR,FPR) of different feature subsets on real-world dataset.

The definitions for these metrics are

TP FP
RR = and FPR = ,
TP + FN FP + TN
in which TP, FN, FP, and TN indicate successfully classified
or misclassified samples. TP indicates successfully detected
malicious samples while TN indicates correctly detected
benign samples. FN indicates omitted malicious samples and
FP indicates wrongly alarmed benign samples. Then we
calculate the T(RR, FPR) using TOPSIS method for each
method.
The performance of different feature selection methods is
FIGURE 12. T(RR,FPR) of different feature subsets on CAIDA.
shown in Tab. 3. It can be seen from Tab. 3 that the proposed
hybrid feature selection algorithm has the minimum T (RR,
FPR) value on these four datasets, which demonstrates that
the proposed hybrid feature selection is mostly effective for
DDoS detection. All in all, the performance of the feature
selection method does affect the detection results, and the
proposed method is superior to other existing methods.

2) INITIAL CLUSTER CENTERS SELECTION

USING DIFFERENT METHODS
In this experiment, we want to verify the effect of dif-
ferent initial cluster center selection methods on detec-
tion performance. We compares four selection methods
mentioned in section IV, namely random selection method
(original k-means), average value-based selection method
FIGURE 13. T(RR,FPR) of different feature subsets on CICIDS. (MF-CKM) [22], density-based selection method [23] and
proposed improved density-based selection method. The
TABLE 2. Selected features for all the four datasets.
comparison experiments use the above four datasets in the
condition of having outliers or not respectively.
The results are shown in Fig. 15 to Fig. 18, and the four
numbers from 1 to 4 in the figures represent the four selection
methods mentioned above. As can be seen from these figures,
there is little difference of detection performance between
four methods in the condition of no outliers, no matter what
the datasets are. However, It can be clearly seen that TOPSIS
selection methods combined with the same detection method values of the first two methods increases significantly when
(k-means) in the same experimental environment, and calcu- outliers exist, which indicates that outliers seriously affect
late RR and FPR values for each method in different datasets. these two algorithms and reduce their detection effects.

VOLUME 7, 2019 64361

 Y. Gu et al.: Semi-Supervised K-Means DDoS Detection Method Using Hybrid Feature Selection Algorithm

TABLE 3. Performance comparison of Different Feature Selection Methods using different datasets.

FIGURE 15. TOPSIS values of four initial center selection methods on FIGURE 16. TOPSIS values of four initial center selection methods on
DARPA dataset. CAIDA dataset.

As for the third method, outliers have little influence on

it except for the fourth dataset. In contrast, our proposed
selection method always performs well when outliers exist.
Besides, if the condition of ‘‘more than one point with maxi-
mum density in the cluster’’ is satisfied, our proposed method
performs better than the third one, shown in Fig. 15 and
Fig. 18. If the condition is not satisfied, our method is not
worse than the third one, shown in Fig. 16 and Fig. 17.
In conclusion, the proposed initial center selection method
outperforms the existing methods when there are outliers,
especially when there is more than one point with maximum
density in the cluster.
FIGURE 17. TOPSIS values of four initial center selection methods on
3) PERFORMANCE COMPARISION OF DIFFERENT CICIDS dataset.
DETECTION METHODS ON DIFFERENT DATASETS
By using the above four datasets, this experiment compares
the proposed detection method (SKM-HFS) with existing dataset with our result, including RR and FPR. For CICIDS
works. Because DARPA and CAIDA datasets are com- datasets and our datasets, no papers have experimented with
monly used in DDoS detection experiments, we compare them and no ready-made results are available. Therefore,
the ready-made result in each paper using the corresponding we implement several detection methods and compare them

64362 VOLUME 7, 2019

Y. Gu et al.: Semi-Supervised K-Means DDoS Detection Method Using Hybrid Feature Selection Algorithm

TABLE 4. Performance of Different Detection Methods using different datasets.

TABLE 5. TRAINNING time between parallel and non-parallel algorithm.

TABLE 6. Detection time of different algorithms using real-world data.

FIGURE 18. TOPSIS values of four initial center selection methods on

real-world dataset.

complexity of the proposed method is O(k · n2 ), in which

n represents the number of training samples and k stands
with our methods by calculating RR and FPR. Furthermore, for feature dimension. By using hadoop-based parallel algo-
we use TOPSIS method to calculate T(RR, FPR) for each rithm, the training time is greatly reduced. The comparison
method using different datasets. The comparison result is of training time between parallel and non-parallel algorithm
shown in Tab. 4. Each algorithm performance in the table is using real-world data is shown in Tab. 5.
arranged in the descending order according to TOPSIS values In the detection phase, this paper compares the time con-
in each dataset. We can see from Tab. 4 that the proposed sumed by different methods using real-world data, which is
method has the smallest T (RR, FPR) value and outperforms shown in Tab. 6. As can be seen from the table, the proposed
all other methods in all of four datasets. method has the smallest detection delay.

4) TIME COMPLEXITY ANALYSIS VI. CONCLUSION AND FUTURE WORK

The time of the proposed algorithm mainly includes train- In order to tackle the issues of supervised and unsuper-
ing time and detection time. In the training phase, the time vised based DDoS detection methods, this paper presents a

VOLUME 7, 2019 64363

 Y. Gu et al.: Semi-Supervised K-Means DDoS Detection Method Using Hybrid Feature Selection Algorithm

semi-supervised weighted k-means detection method. Spe- [12] J. Cheng, M. Li, X. Tang, V. S. Sheng, Y. Liu, and W. Guo, ‘‘Flow
cially, we firstly provide a hadoop-based hybrid feature selec- correlation degree optimization driven random forest for detecting DDoS
attacks in cloud computing,’’ Secur. Commun. Netw., vol. 2018, Nov. 2018,
tion method to find the most effective feature set. Secondly, Art. no. 6459326.
we present an improved density-based initial cluster cen- [13] K. J. Singh, K. Thongam, and T. De, ‘‘Entropy-based application layer
ters selection method to solve the problem of outliers and DDoS attack detection using artificial neural networks,’’ Entropy, vol. 18,
no. 10, pp. 350–366, 2016.
local optimal of k-means clustering. Then, we propose a [14] A. Chonka, J. Singh, and W. Zhou, ‘‘Chaos theory based detection against
semi-supervised weighted k-means method using hybrid fea- network mimicking DDoS attacks,’’ IEEE Commun. Lett., vol. 13, no. 9,
ture selection algorithm (SKM-HFS) to achieve better detec- pp. 717–719, Sep. 2009.
[15] X. Wu and Y. Chen, ‘‘Validation of chaos hypothesis in NADA and
tion performance. Finally, we exploit DARPA DDoS dataset, improved DDoS detection algorithm,’’ IEEE Commun. Lett., vol. 17,
CAIDA ‘‘DDoS attack 2007’’ dataset, CICIDS ‘‘DDoS attack no. 12, pp. 2396–2399, Dec. 2013.
2017’’ dataset and real-world dataset to carry out the ver- [16] S. M. T. Nezhad, M. Nazari, and E. A. Gharavol, ‘‘A novel DoS and
DDoS attacks detection algorithm using ARIMA time series model and
ification experiments. Three conclusions are drawn from chaotic system in computer networks,’’ IEEE Commun. Lett., vol. 20, no. 4,
the experiment results. Firstly, the hybrid feature selection pp. 700–703, Apr. 2016.
method is much better than other feature selection methods [17] A. R. Yusof, N. I. Udzir, A. Selamat, H. Hamdan, and M. T. Abdullah,
‘‘Adaptive feature selection for denial of services (DoS) attack,’’ in Proc.
using TOPSIS as evaluation factor. Secondly, the improved IEEE Conf. Appl., Inf. Netw. Secur. (AINS), Miri, Malaysia, Nov. 2017,
density-based initial cluster centers selection algorithm is pp. 81–84.
the most effective in the presence of outliers and more than [18] E. Balkanli, A. N. Zincir-Heywood, and M. I. Heywood, ‘‘Feature selection
for robust backscatter DDoS detection,’’ in Proc. IEEE 40th Local Comput.
one maximum density point. Thirdly, the proposed detection Netw. Conf. Workshops (LCN Workshops), Clearwater Beach, FL, USA,
method outperforms the benchmark in the respect of detec- Oct. 2015, pp. 611–618.
tion performance and TOPSIS. [19] L. Zi, J. Yearwood, and X.-W. Wu, ‘‘Adaptive clustering with feature
ranking for DDoS attacks detection,’’ in Proc. 4th Int. Conf. Netw. Syst.
In the future, more and larger datasets will be used to verify Secur., Melbourne, VIC, Australia, Sep. 2010, pp. 281–286.
the advantages of the provided algorithm in terms of the [20] H. Jiang, S. Chen, H. Hu, and K. Qian, ‘‘Lightweight detection approach
generalization and robustness. In addition, the parallel ability of DDoS attacks based on GAIG algorithm for feature selection,’’ Appl.
Res. Comput., vol. 33, no. 2, pp. 502–506, Feb. 2016.
of the proposed method will be further improved. [21] O. Osanaiye, H. Cai, K. K. Choo, A. Dehghantanha, Z. Xu, and M. Dlodlo,
‘‘Ensemble-based multi-filter feature selection method for DDoS detec-
tion in cloud computing,’’ EURASIP J. Wireless Commun. Netw., vol. 1,
REFERENCES
pp. 130–139, May 2016.
[1] W. L. Al-Yaseen, Z. A. Othman, and M. Z. A. Nazri, ‘‘Multi-level hybrid [22] Y. Gu, Y. Wang, Z. Yang, F. Xiong, and Y. Gao, ‘‘Multiple-features-based
support vector machine and extreme learning machine based on modified semisupervised clustering DDoS detection method,’’ Math. Problems Eng.,
K-means for intrusion detection system,’’ Expert Syst. Appl., vol. 67, vol. 2017, Dec. 2017, Art. no. 5202836.
pp. 296–303, Jan. 2017. [23] Q. Wang and S. H. Liu, ‘‘Application research of improved K-means
[2] J. Yu, Z. Li, H. Chen, and X. Chen, ‘‘A detection and offense algorithm in intrusion detection,’’ Comput. Eng. Appl., vol. 51, no. 17,
mechanism to defend against application layer DDoS attacks,’’ in pp. 124–127, 2015.
Proc. Int. Conf. Netw. Services (ICNS), Athens, Greece, Jun. 2007, [24] N. Hoque, H. Kashyap, and D. K. Bhattacharyya, ‘‘Real-time DDoS
p. 54. attack detection using FPGA,’’ Comput. Commun., vol. 110, pp. 48–58,
[3] M. I. W. Praman, Y. Purwanto, and F. Y. Suratman, ‘‘DDoS detection Sep. 2017.
using modified K-means clustering with chain initialization over [25] X. Ma and Y. Chen, ‘‘DDoS detection method based on chaos analysis of
landmark window,’’ in Proc. Int. Conf. Control, Electron., Renew. network traffic entropy,’’ IEEE Commun. Lett., vol. 18, no. 1, pp. 114–117,
Energy Commun. (ICCEREC), Bandung, Indonesia, Aug. 2015, Jan. 2014.
pp. 7–11. [26] S. Behal and K. Kumar, ‘‘Detection of DDoS attacks and flash events
[4] X. Qin, T. Xu, and C. Wang, ‘‘DDoS attack detection using flow entropy using information theory metrics—An empirical investigation,’’ Comput.
and clustering technique,’’ in Proc. 11th Int. Conf. Comput. Intell. Secur. Commun., vol. 103, pp. 18–28, May 2017.
(CIS), Shenzhen, China, Dec. 2015, pp. 412–415. [27] M. Sachdeva, K. Kumar, and G. Singh, ‘‘A comprehensive approach to
[5] L. Guo, P. Li, X. Di, and L. Cong, ‘‘The research of application layer discriminate DDoS attacks from flash events,’’ J. Inf. Secur. Appl., vol. 26,
DDoS attack detection based the model of human access,’’ Comput. Secur., pp. 8–22, Feb. 2016.
vol. 6, pp. 11–14, Jun. 2014. [28] Y. Liu, J. Yin, J. Cheng, and B. Zhang, ‘‘Detecting DDoS attacks using
conditional entropy,’’ in Proc. Int. Conf. Comput. Appl. Syst. Modeling
[6] E. Balkanli, J. Alves, and A. N. Zincir-Heywood, ‘‘Supervised learning to
(ICCASM), Taiyuan, China, Oct. 2010, pp. 278–282.
detect DDoS attacks,’’ in Proc. IEEE Symp. Comput. Intell. Cyber Secur.
[29] M. Baskar, T. Gnanasekaran, and S. Saravanan, ‘‘Adaptive IP traceback
(CICS), Orlando, FL, USA, Dec. 2014, pp. 1–8.
mechanism for detecting low rate DDoS attacks,’’ in Proc. IEEE Int. Conf.
[7] H. V. Nguyen and Y. Choi, ‘‘Proactive detection of DDoS attacks utilizing
Emerg. Trends Comput., Commun. Nanotechnol. (ICECCN), Tirunelveli,
k-NN classifier in an anti-DDoS framework,’’ Int. J. Elect., Comput., Syst.
India, Mar. 2013, pp. 373–377.
Eng., vol. 4, no. 4, pp. 247–252, Feb. 2010.
[30] S. Behal and K. Kumar, ‘‘Detection of DDoS attacks and flash events using
[8] P. Xiao, W. Qu, H. Qi, and Z. Li, ‘‘Detecting DDoS attacks against data novel information theory metrics,’’ Comput. Netw., vol. 116, pp. 96–110,
center with correlation analysis,’’ Comput. Commun., vol. 67, pp. 66–74, Apr. 2017.
Aug. 2015. [31] N. Furutani, T. Ban, J. Nakazato, J. Shimamura, J. Kitazono, and S. Ozawa,
[9] R. Vijayasarathy, S. V. Raghavan, and B. Ravindran, ‘‘A system approach ‘‘Detection of DDoS backscatter based on traffic features of darknet
to network modeling for DDoS detection using a Naive Bayesian clas- TCP packets,’’ in Proc. 9th Asia Joint Conf. Inf. Secur., Wuhan, China,
sifier,’’ in Proc. 3rd Int. Conf. Commun. Syst. Netw., Bangalore, India, Sep. 2014, pp. 39–43.
Jan. 2011, pp. 1–10. [32] N. A. Singh, K. J. Singh, and T. De, ‘‘Distributed denial of service attack
[10] Y. Bouzida and F. Cuppens, ‘‘Detecting known and novel network intru- detection using naive Bayes classifier through info gain feature selection,’’
sions,’’ in Proc. IFIP Int. Inf. Secur. Conf., Karlstad, Sweden, 2006, in Proc. Int. Conf. Inform. Anal., Aug. 2016, p. 54.
pp. 258–270. [33] N. Hoque, D. K. Bhattacharyya, and J. K. Kalita, ‘‘Denial of service
[11] J. Li, Y. Liu, and L. Gu, ‘‘DDoS attack detection based on neural net- attack detection using multivariate correlation analysis,’’ in Proc. 2nd Int.
work,’’ in Proc. 2nd Int. Symp. Aware Comput., Tainan, China, Nov. 2010, Conf. Inf. Commun. Technol. Competitive Strategies, Hangzhou, China,
pp. 196–199. Mar. 2016, p. 100.

64364 VOLUME 7, 2019

Y. Gu et al.: Semi-Supervised K-Means DDoS Detection Method Using Hybrid Feature Selection Algorithm

[34] I. L. Meitei, K. J. Singh, and T. De, ‘‘Detection of DDoS DNS amplification KAIYUE LI received the B.S. degree from the
attack using classification algorithm,’’ in Proc. Int. Conf. Inform. Anal., Hebei University of Technology, China, in 2017.
Pondicherry, India, Aug. 2016, p. 81. She is currently pursuing the M.Eng. degree with
[35] Lincoln Laboratory Scenario (DDoS) 1.0 of DARPA Intrusion the Beijing Key Laboratory of Intelligent Telecom-
Detection Evaluation Data Sets. Accessed: 2000. [Online]. Available: munications Software and Multimedia, School
http://www.ll.mit.edu/ideval/data/2000/LLS_DDOS_1.0.html of Computer, Beijing University of Posts and
[36] A. Lakhina, M. Crovella, and C. Diot, ‘‘Mining anomalies using traffic Telecommunications, China. Her current research
feature distributions,’’ in Proc. ACM Conf. Appl., Technol., Archit., Proto-
interest includes network security.
cols Comput. Commun. (SIGCOMM), Philadelphia, PA, USA, Aug. 2005,
pp. 217–228.
[37] A. Keikha and H. M. Nehi, ‘‘A complex method based on TOPSIS and
Choquet integral to solve multi attribute group decision making problems
with interval type-2 fuzzy numbers,’’ in Proc. 4th Iranian Joint Congr.
Fuzzy Intell. Syst. (CFIS), Zahedan, Iran, Sep. 2015, pp. 1–5.
[38] The CAIDA UCSD DDoS Attack 2007 Dataset. Accessed: Aug.
2007. [Online]. Available: http://www.caida.org/data/passive/ddos-
20070804_dataset.xml
[39] The CICIDS DDoS Attack 2017 Dataset. Accessed: 2017. [Online]. Avail-
able: https://www.unb.ca/cic/datasets/ids-2017.html
[40] S. O. Al-Mamory and Z. M. Algelal, ‘‘A modified DBSCAN cluster-
ing algorithm for proactive detection of DDoS attacks,’’ in Proc. Annu.
Conf. New Trends Inf. Commun. Technol. Appl. (NTICT), Baghdad, Iraq,
ZHENYANG GUO received the B.S. degree in
Mar. 2017, pp. 304–309.
[41] W. Bhaya and M. Ebadymanaa, ‘‘DDoS attack detection approach using
software engineering from Heilongjiang Univer-
an efficient cluster analysis in large data scale,’’ in Proc. Annu. Conf. New sity, China, in 2017. He is currently pursuing the
Trends Inf. Commun. Technol. Appl. (NTICT), Baghdad, Iraq, Mar. 2017, M.Eng. degree with the Beijing Key Laboratory
pp. 168–173. of Intelligent Telecommunications Software and
[42] P. A. R. Kumar and S. Selvakumar, ‘‘Distributed denial of service attack Multimedia, School of Computer, Beijing Univer-
detection using an ensemble of neural classifier,’’ Comput. Commun., sity of Posts and Telecommunications, China. His
vol. 34, no. 11, pp. 1328–1341, 2011. current research interests include machine learn-
[43] H. Luo, Y. Lin, H. Zhang, and M. Zukerman, ‘‘Preventing DDoS attacks ing, network security, and detection on botnet.
by identifier/locator separation,’’ IEEE Netw., vol. 27, no. 6, pp. 60–65,
Nov./Dec. 2013.
[44] T. Andrysiak, Ł. Saganowski, and M. Choraś, ‘‘DDoS attacks detection by
means of greedy algorithms,’’ in Image Processing and Communications
Challenges 4. Berlin, Germany: Springer, 2013, pp. 303–310.
[45] V. Srihari and R. Anitha, ‘‘DDoS detection system using wavelet features
and semi-supervised learning,’’ in Security in Computing and Communi-
cations. Berlin, Germany: Springer, 2014, pp. 291–303.
[46] H. Liu, Y. Sun, V. C. Valgenti, and M. S. Kim, ‘‘TrustGuard: A flow-
level reputation-based DDoS defense system,’’ in Proc. IEEE Con-
sum. Commun. Netw. Conf. (CCNC), Las Vegas, NV, USA, Jan. 2011,
pp. 287–291.

YONGHAO GU received the Ph.D. degree from YONGFEI WANG received the B.S. degree from
the Beijing University of Posts and Telecommu- Hebei North University, China, in 2012. He is
nications, China, in 2007, where he is currently a currently pursuing M.Eng. the degree with the
Lecturer with the Beijing Key Laboratory of Intel- Beijing Key Laboratory of Intelligent Telecom-
ligent Telecommunications Software and Multi- munications Software and Multimedia, School
media, School of Computer. His current research of Computer, Beijing University of Posts and
interests include network security and privacy Telecommunications, China. His current research
preservation. interests include network security and privacy
preservation.

VOLUME 7, 2019 64365