Flowstalker: Comprehensive Traffic Flow Monitoring On The Data Plane Using P4
Flowstalker: Comprehensive Traffic Flow Monitoring On The Data Plane Using P4
Flowstalker: Comprehensive Traffic Flow Monitoring On The Data Plane Using P4
Abstract—Programmability has been extensively investigated to manner. Traditionally this is done by sampling packets [4],
enable a more flexible operation of computer networks, and which may result in low accuracy of the sampled data. How-
in this context the P4 language was designed entirely for ever, with the advent of the P4 language came the opportunity
programming the data plane. With programmable data planes
comes the possibility of revisiting many inefficient approaches to for revisiting this problem with a more decentralized approach
networking problems, for example how we use the data plane using a programmable data plane [5]. While P4 offers us
to create an understanding of the network state. Traditional the possibility of analyzing every packet of every flow, such
switches expose only a bare minimum of what happens in their method is inadequate for the data plane and does not scale
forwarding plane, forcing us to resort to inefficient methods, (as seen by Cisco’s NetFlow [6], that showed to be too
such as snapshotting, to acquire the state of the network. We
advocate that by combining the programmable hardware on cumbersome for high speed networking). Thus, we need to
switches with every switch specific view over its traffic, we filter out irrelevant flows to minimize the impact, both in terms
are able to accomplish the same tasks in a more efficient and of memory and CPU, and track only the necessary information
comprehensive manner. In this paper we present an efficient that can later be derived into more meaningful features.
monitoring mechanism using programmable data planes. Our
mechanism capitalizes on data plane programmability to perform In this work we present FlowStalker, a comprehensive moni-
tasks that are usually performed solely by the control plane (e.g. toring system that operates entirely on the data plane, relying
traffic monitoring and information gathering). Firstly, we present
a monitoring system based on a two-phase monitoring scheme on P4 switches to gather traffic data in a distributed manner.
that runs directly on the data plane. Secondly, we introduce a Firstly, we introduce an extensible and scalable monitor-
flexible method for network data gathering, enabling “control- ing approach that consists of two phases: (1) a proactive,
plane-free" consolidation of data from switches. Finally, we show lightweight, phase that detects target flows based on heavy
techniques for using both the monitor and the gathering system hitter detection strategies; and (2) a reactive, heavyweight,
to create constant, snapshot-free analysis of network traffic.
phase, which captures and stores specific meta-data about
these flows. Secondly, we propose a data gathering system that
I. I NTRODUCTION leverages data plane telemetry [5] to ascertain the in-locus state
of a network by gathering monitored data from the switches
Efforts to bring programmability to the data plane are be- and consolidating it at the controller. The core idea of our
coming a viable alternative to traditional switches. The P4 gathering system is to subdivide a network into clusters and
programming language [1] is a step forward in conciliating enable cluster data to be collected quickly should we need it.
the theory of programmable data planes with the needs of In order to do so, we created a special packet, which we call
the industry. These efforts however suffer from the fact that Crawler Packet (Cp), that enables the information gathering
application specific integrated circuits (ASICs) nowadays are to take place as a cluster specific event without the controller
nearly unbeatable in terms of speed. To make programmability needing to interact with all of the cluster members. As opposed
attractive we need a suitable trade-off between flexibility and to the usual method for doing this in SDN, which is to poll the
efficiency. This has led to some compromises which cause switches individually and wait for their response, our method
restrictions to programmability on the data plane. On the plus acts with very little intervention from the controller.
side of these compromises, data plane programmability gives
us a degree of control and proximity to the switches that was Our results in monitoring a reduced set of features show that
previously impossible. Even in Software Defined Networking we can achieve meaningful monitoring within the data plane
(SDN) our access to the inner workings of the switches was with relatively low impact on the throughput of monitored
limited to the their interface with the controller, meaning that flows. We also evaluated our strategy on different topologies
most of the passing meta-data is lost. The fact that we discard with different cluster sizes and it has shown to be very efficient
relevant meta-data at switches only means that we have to go in reducing message exchanging and connection overhead on
through much greater lengths to recover it when we need it the control path between the controller and the switches.
(e.g., for QoE prediction [2], anomaly detection [3]).
The rest of this paper is structured as follows. In Section II
Usually, non-programmable data planes lack the resources to we present an overview on programmable networks and the P4
implement an in-depth analysis of network state in a scalable language. Section III gives an in-depth analysis of FlowStalker.
Software-based paradigms for networking enable decoupling • per-flow: metrics are generated once and updated for
software solutions from the hardware in which they exe- each packet of the target flow during the reactive phase.
cute, making the management and operation of the network For instance, beginning of monitoring, packet and byte
infrastructure more flexible and adaptive. Software-Defined counts, moving average of RTT;
Networking (SDN) promotes the separation of the control • per-packet: a new instance of data is generated and
and data planes, transforming the switch into a bare packet stored for each packet pertaining to a target flow. These
forwarding device [7]. Emerging needs for both operating metric values are inherently larger than per-flow data.
and managing networks, however, can benefit from enhanced Hence, a more in-depth record of the flow is possible,
functionalities on the data plane, such as ease of deployment maintaining data, such as timestamps, from which many
for new services or protocol extensibility. This has motivated other interesting features can be later derived for use in
the idea of using P4 to create a programmable network core. controller applications such as classification algorithms
or QoS monitoring.
P4 [1] is a programming language to describe the behavior of
network switches. It builds its interface upon a set of features The reactive phase keeps a hash table in the switch to encode
common to many existing switch architectures. This ensures data, in which a flow (defined by its source and destination
flexibility for deployment and the high level of abstraction IP addresses1 ) is assumed to be the lookup key. Each entry
needed to make the language compatible with current designs. in the hash table is composed of an array of P4 registers.
In P4, a parser handles every incoming packet, mapping The rationale behind this abstract representation is to encode
packet headers into P4 data structures. Packets are sent to new values of a given metric of interest using a lightweight
a match+action table, which will match the packets against allocation procedure, which manages both the allocation of
control plane defined rules, and act upon them based on P4- entries to flows and the registers inside an entry. The allocation
programmed actions. These actions can manipulate headers procedure necessary to encode each type of metric varies, but
and store information on persistent registers. P4 programs can can generally be implemented through bitwise operations. We
be compiled to specific target switches, with each construct of advocate this offers a space-efficient data encoding abstraction,
the program being mapped into target-specific primitives and while also presenting low computational overhead (as shown
their architectural counterparts. in Section IV). This data structure can be used to store a
history of per-flow and per-packet metrics, such as packet-
III. F LOW S TALKER sizes and packet arrival timestamps. A small sample of raw
features that can be tracked and the corresponding derivable
In this section, we present a novel mechanism based on P4
metrics are presented in Table I.
to monitor stateful data on the data plane, called FlowStalker.
Next, we discuss the design of our solution, and highlight the
TABLE I
advantages it brings over traditional monitoring strategies. D ERIVATIONS OF R AW METRICS .
Cluster 0
Full CP
Stateful Data
Flows Per-
Packet Per Flow
Empty CP Reactive
IP A → IP B 0110 0110 ... 01100110 0110
Behavior 0110 0110 ... 0110
Control
Plane API IP V → IP W 0110 0110 ...... 0110 0110
0110 0110 0110 0110
Controller Injects CP
2
...
...
...
...
...
Configuration into Clusters
Gathering
IP X → IP Y 0110 0110 ...... 01100110 0110
System 0110 0110 0110
Thresholds
Clustering Empty CP
Monitoring
Append stateful data into
Module
3 the crawler packet
Intercept
Table FlowStalker Eth. S0
Crawler Packet
Full CP
S0
Eth. S0 S1
Eth. S0 S1 S2 S2 S1
Crawler Packet
Crawler Packet
5 Filled CP is forwarded 4
back to the controller Data Plane CP Routed internally
Cluster 1 by the nodes,
collecting data
Fig. 1. FlowStalker overview: monitored metrics are maintained at the switches. After a warning is sent to the controller (step 1), Crawler Packets are sent
to each cluster to collect stateful data from individual switches (steps 2-3). After a Cp traverses the cluster, it returns with gathered information (steps 4-5).
Base timestamp) and the next arriving packet timestamp. For B. Aggregate Data Plane Information Gathering
example, Figure 2 presents an overview of how it is possible
to record packet arrival timestamps. The representation of the In this section, we discuss our strategy for efficiently collecting
array reflects the implementation through a sequence of P4 the data that the SDN controller will need. As opposed to tra-
registers. At switch time 15, the first monitored packet arrives ditional polling strategies, in which the controller obtains data
and the per-flow metric Base is set. When the next packet snapshots from individual switches, we present a decentralized
arrives, at t1 , only its offset from Base will be stored in the gathering mechanism in the data plane. We perform network
register. The same applies to the packet arriving at timestamp telemetry to consolidate the stateful data stored on switches.
t2 = 24, in which the offset from the base timestamp is
calculated and bit 9 is marked; t3 follows the same pattern. Next, we discuss how we divide the network into clusters
and setup routes inside them to keep telemetry efficient and
less prone to bottlenecking at both the controller and the link
Arriving Packets Encoded
between a switch and its controller (control path). We then
Base
Timestamps ( ti ) Timestamps show how information is aggregated from the switches within
a specific cluster and sent back to the control plane.
t0 =15 offset =0 15 0000 0000 0000
Clustering Method: We divide the data plane switches into
t1=16 offset=1
15 0100 0000 0000
logical groups, called clusters, to modularize data collection.
This creates the neighborhoods in which data-collecting pack-
t2=24 offset=9 0100 0010 0000
ets (Crawler Packets, as described next) circulate. The process
15
of partitioning the network into clusters is accomplished by
t3=28 offset=11 using a Markov clusterization algorithm, described in [9], that
15 0100 0010 0001
analyzes the “closeness” of nodes (i.e., switches) according to
their connections (i.e., links between switches). The weights
Fig. 2. Tracking packet arriving timestamps.
of these connections should be set according to a metric that
produces the best results for our application, e.g., network
After extracting data from the flow, the reactive phase incre- latency. After clusters are formed, we use a DFS algorithm
ments the packet counter of this flow (previously maintained on each cluster to determine a single route that spans the
by the proactive phase) and compares it against the high whole cluster. This creates forwarding rules that match crawler
threshold, which once crossed will trigger a warning (Figure packets for every switch in a cluster. Since a switch never
1, step 1) to the controller about the violating flow, which in receives more than one crawler packet from the same neighbor,
turn should start the data gathering process (described next). the rules act by matching the source IP and forwarding the
Crawler Packet Write Area IV. S TRATEGY E VALUATION
L2 HDR CP-Info Switch 1 Switch 2 Switch 3 ... Switch N
To evaluate FlowStalker, we observed both the monitoring
InfoType FlowOffset Data Offset and gathering systems on the BMv2 P4 software switch2 . The
experiments were performed on a Linux virtual machine with
Fig. 3. Structure of a Crawler packet. 6 logical CPUs at 3.20 GHz and 8GB of RAM.
5 switches. 3 http://iperf.fr/
increasing the number of monitored items (and the number
400
of operations to compute and store them) did not cause any
Heavy
perceptible impact in terms of throughput. 350 Medium
Light
300
B. Gathering
End-to-End Delay
250
V. R ELATED W ORK
Light Medium Heavy
4 KB 60 KB 120 KB
BMv2 Thrift
3.5 KB
Crawler Packet Monitoring strategies vary, but more recent approaches tend
50 KB 100 KB
to have more sophisticated systems reliant on programmability
Bytes Exchanged Control Path
3 KB
40 KB 80 KB
of both control and data planes.
2.5 KB