SoftwareDefinedNetworking - A Comprehensive Survey (31-63)

Kreutz et al.
: Software-Defined Networking: A Comprehensive Survey
the necessary information from elements involved in the Debugging and troubleshooting in networking is at a
operation of a data center (infrastructure, operators, appli- very primitive stage. In traditional networks, engineers
cations), it is possible to continuously build signatures for and developers have to use tools such as ping,
applications by passively capturing control traffic. Then, traceroute, tcpdump, nmap, netflow, and
the signature history can be used to identify differences in SNMP statistics for debugging and troubleshooting.
behavior. Every time a difference is detected, operators Debugging a complex network with such primitive tools
can reactively or proactively take corrective measures. This is very hard. Even when one considers frameworks such as
can help to isolate abnormal components and avoid further XTrace [374], Netreplay [376], and NetCheck [377], which
damage to the infrastructure. improve debugging capabilities in networks, it is still diffi-
cult to troubleshoot networking infrastructures. For in-
8) Toward SDN App Stores: As can be observed in stance, these frameworks require a huge effort in terms of
Tables 9 and 10, most SDN applications rely on NOX and network instrumentation. The additional complexity intro-
OpenFlow. NOX was the first controller available for duced by different types of devices, technologies, and
general use, making it a natural choice for most use cases vendor-specific components and features makes matters
so far. As indicated by the sheer number of security-related worse. As a consequence, these solutions may find it hard to
applications, security is probably one of the killer appli- be widely implemented and deployed in current networks.
cations for SDNs. Curiously, while most use cases rely on SDN offers some hope in this respect. The hardware-
OpenFlow, new solutions such as SoftRAN are considering agnostic software-based control capabilities and the use of
different APIs, as is the case of the Femto API [253], [303]. open standards for control communication can potentially
This diversity of applications and APIs will most probably make debugging and troubleshooting easier. The flexibility
keep growing in SDN. and programmability introduced by SDN is indeed opening
There are other kinds of network applications that do new avenues for developing better tools to debug, trouble-
not easily fit in our taxonomy, such as Avior [365], OESS shoot, verify, and test networks [378]–[385].
[366], and SDN App Store [367], [368]. Avior and OESS Early debugging tools for OpenFlow-enabled networks,
are graphical interfaces and sets of software tools that such as ndb [378], OFRewind [379], and NetSight [386],
make it easier to configure and manage controllers (e.g., make it easier to discover the source of network problems
Floodlight) and OpenFlow-enabled switches, respectively. such as faulty device firmware [378], inconsistent or non-
By leveraging their graphical functions it is possible to existing flow rules [378], [379], lack of reachability [378],
program OpenFlow enabled devices without coding in a [379], and faulty routing [378], [379]. Similarly to the
particular programming language. well-known gdb software debugger, ndb provides basic
The SDN App Store [367], [368], owned by HP, is debugging actions such as breakpoint, watch, backtrace,
probably the first SDN application market store. Custo- single step, and continue. These primitives help applica-
mers using HP’s OpenFlow controller have access to the tion developers to debug networks in a similar way to
online SDN App Store and are able to select applications to traditional software. By using ndb’s postcards (i.e., a
be dynamically downloaded and installed in the controller. unique packet identifier composed of a truncated copy of
The idea is similar to the Android Market or the Apple the packet’s header, the matching flow entry, the switch,
Store, making it easier for developers to provide new and the output port), for instance, a programmer is able to
applications and for customers to obtain them. quickly identify and isolate a buggy OpenFlow switch with
hardware or software problems. If the switch is presenting
I. Cross-Layer Issues abnormal behavior such as corrupting parts of the packet
In this section, we look at cross-layer problems such as header, by analyzing the problematic flow sequences with
debugging and troubleshooting, testing, verification, simula- a debugging tool, one can find (in a matter of few seconds)
tion, and emulation. A summary of the existing tools for where the packets of a flow are being corrupted, and take
dealing with these cross-layer issues can be found on Table 11. the necessary actions to solve the problem.
The OFRewind [379] tool works differently. The idea is
1) Debugging and Troubleshooting: Debugging and trou- to record and replay network events, in particular control
bleshooting have been important subjects in computing messages. These usually account for less than 1% of the
infrastructures, parallel and distributed systems, embed- data traffic and are responsible for 95%–99% of the bugs
ded systems, and desktop applications [369]–[375]. The [385]. This tool allows operators to perform fine-grained
two predominant strategies applied to debug and trouble- tracing of network behavior, being able to decide which
shoot are runtime debugging (e.g., gdb-like tools) and subsets of the network will be recorded and, afterwards,
post-mortem analysis (e.g., tracing, replay, and visualiza- select specific parts of the traces to be replayed. These
tion). Despite the constant evolution and the emergence of replays provide valuable information to find the root cause
new techniques to improve debugging and troubleshoot- of the network misbehavior. Likewise, NetRevert [387]
ing, there are still several open avenues and research also records the state of OpenFlow networks. However,
questions [370]. the primary goal is not to reproduce network behavior, but
44 Proceedings of the IEEE | Vol. 103, No. 1, January 2015

Kreutz et al.: Software-Defined Networking: A Comprehensive Survey
Table 11 Debugging, Verification, and Simulation
Vol. 103, No. 1, January 2015 | Proceedings of the IEEE 45

rather to provide rollback recovery in case of failures, allow the development and execution of a rich set of tests
which is a common approach used in distributed systems on OpenFlow-enabled devices. Its ultimate goal is to mea-
for eliminating transient errors in nodes [388], [389]. sure the processing capacity and bottlenecks of control
Despite the availability of these debugging and verifi- applications and forwarding devices. With this tool, users
cation tools, it is still difficult to answer questions such as: are able to observe and evaluate forwarding table consis-
What is happening to my packets that are flowing from tency, flow setup latency, flow space granularity, packet
point A to point B? What path do they follow? What header modification types, and traffic monitoring capabilities
modifications do they undergo on the way? To answer (e.g., counters).
some of these questions one could recur to the history of FlowChecker [382], OFTEN [384], and VeriFlow [383]
the packets. A packet’s history corresponds to the paths it are three examples of tools to verify correctness properties
uses to traverse the network, and the header modifications violations on the system. While the former two are based
in each hop of the path. NetSight [386] is a platform whose on offline analysis, the latter is capable of online checking
primary goal is to allow applications that use the history of of network invariants. Verification constraints include
the packets to be built, in order to find out problems in a security and reachability issues, configuration updates on
network. This platform is composed of three essential ele- the network, loops, black holes, etc.
ments: 1) NetSight, with its dedicated servers that receive Other formal modeling techniques, such as Alloy, can
and process the postcards for building the packet history; be applied to SDNs to identify unexpected behavior [390].
2) the NetSigh-SwitchAssist, which can be used in For instance, a protocol specification can be weak when it
switches to reduce the processing burden on the dedicated under-specifies some aspects of the protocol or due to a
servers; and 3) the NetSight-HostAssist to generate and very specific sequence of events. In such situations, model
process postcards on end hosts (e.g., in the hypervisor on a checking techniques such as Alloy can help to find and
virtualized infrastructure). correct unexpected behaviors.
netwatch [386], netshark [386], and nprof Tools such as FLOWGUARD [396] are specifically de-
[386] are three examples of tools built over NetSight. The signed to detect and resolve security policy violations in
first one is a live network invariant monitor. For instance, OpenFlow-enabled networks. FLOWGUARD is able to
an alarm can be triggered every time a packet violates any examine on-the-fly network policy updates, check indirect
invariant (e.g., no loops). The second one, netshark, security violations (e.g., OpenFlow’s Set-Field actions
enables users to define and execute filters on the entire modification) and perform stateful monitoring. The frame-
history of packets. With this tool, a network operator can work uses five resolution strategies for real-time security
view a complete list of properties of packets at each hop, policy violation resolution, flow rejecting, dependency
such as input port, output port, and packet header values. breaking, update rejecting, flow removing, and packet
Finally, nprof can be used to profile sets of network links blocking [396]. These resolutions are applied over diverse
to provide data for analyzing traffic patterns and routing update situations in OpenFlow-enabled networks.
decisions that might be contributing to link load. More recently, tools such as VeriCon [397] have been
designed to verify the correctness of SDN applications in a
2) Testing and Verification: Verification and testing tools large range of network topologies and by analyzing a broad
can complement debugging and troubleshooting. Recent range of sequences of network events. In particular,
research [380], [382]–[385], [390], [391] has shown that VeriCon confirms, or not, the correct execution of the
verification techniques can be applied to detect and avoid SDN program.
problems in SDN, such as forwarding loops and black One of the challenges in testing and verification is to
holes. Verification can be done at different layers (at the verify forwarding tables in very large networks to find
controllers, network applications, or network devices). routing errors, which can cause traffic losses and security
Additionally, there are different network propertiesV breaches, as quickly as possible. In large-scale networks, it
mostly topology specificVthat can be formally verified, is not possible to assume that the network snapshot, at any
provided a network model is available. Examples of such point, is consistent, due to the frequent changes in routing
properties are connectivity, loop freedom, and access con- state. Therefore, solutions such as HSA [398], Anteater
trol [29]. A number of tools have also been proposed to [399], NetPlumber [400], Veri-Flow [383], and assertion
evaluate the performance of OpenFlow controllers by languages [401] are not suited for this kind of environ-
emulating the load of large-scale networks (e.g., Cbench ment. Another important issue is related on how fast the
[392], OFCBenchmark [393], PktBlaster [394]). Similarly, verification process is done, especially in modern data
benchmarking tools for OpenFlow switches are also centers that have very tight timing requirements. Libra
available (e.g., OFLOPS [381] and FLOPS-Turbo [395]). [391] represents one of the first attempts to address these
Tools such as NICE [380] generate sets of diverse particular challenges of large-scale networks. This tool
streams of packets to test as many events as possible, ex- provides the means for capturing stable and consistent
posing corner cases such as race conditions. Similarly, snapshots of large-scale network deployments, while also
OFLOPS [381] provides a set of features and functions that applying long prefix matching techniques to increase the

scalability of the system. By using MapReduce computa- V. ONGOING RESEARCH EFFORTS

tions, Libra is capable of verifying the correctness of a AND CHALLENGES
network with up to 10 000 nodes within one minute. The research developments we have surveyed so far seek to
Anteater [399] is a tool that analyzes the data plane state overcome the challenges of realizing the vision and fulfilling
of network devices by encoding switch configurations as the promises of SDN. While Section IV provided a per-
boolean satisfiability problem (SAT) instances, allowing to spective structured across the layers of the ‘‘SDN stack,’’ this
use a SAT solver to analyze the network state. The tool is section highlights research efforts we consider of particular
capable of verifying violations of invariants such as loop-free importance for unleashing the full potential of SDN, and that
forwarding, connectivity, and consistency. These invariants therefore deserves a specific coverage in this survey.
usually indicate a bug in the network, i.e., their detection
helps to increase the reliability of the network data plane.
A. Switch Designs
3) Simulation and Emulation: Simulation and emulation Currently available OpenFlow switches are very di-
software is of particular importance for fast prototyping and verse and exhibit notable differences in terms of feature
testing without the need for expensive physical devices. set (e.g., flow table size, optional actions), performance
Mininet [110] is the first system that provides a quick and (e.g., fast versus slow path, control channel latency/
easy way to prototype and evaluate SDN protocols and throughput), interpretation and adherence to the protocol
applications. One of the key properties of Mininet is its use specification (e.g., BARRIER command), and architecture
of software-based OpenFlow switches in virtualized con- (e.g., hardware versus software designs).
tainers, providing the exact same semantics of hardware-
based OpenFlow switches. This means that controllers or 1) Heterogeneous Implementations: Implementation
applications developed and tested in the emulated envi- choices have a fundamental impact on the behavior, accu-
ronment can be (in theory) deployed in an OpenFlow- racy, and performance of switches, ranging from differ-
enabled network without any modification. Users can easily ences in flow counter behavior [418] to a number of other
emulate an OpenFlow network with hundreds of nodes and performance metrics [381]. One approach to accommodate
dozens of switches by using a single personal computer. such heterogeneity is through NOSIX, a portable API that
Mininet-HiFi [402] is an evolution of Mininet that en- separates the application expectations from the switch
hances the container-based (lightweight) virtualization heterogeneity [246]. To do so, NOSIX provides a pipeline
with mechanisms to enforce performance isolation, re- of multiple virtual flow tables and switch drivers. Virtual
source provisioning, and accurate monitoring for perfor- flow tables are intended to meet the expectations of appli-
mance fidelity. One of the main goals of Mininet-HiFi is to cations and are ultimately translated by the drivers into
improve the reproducibility of networking research. actual switch flow tables. Toward taming the complexity of
Mininet CE [403] and SDN Cloud DC [404] are exten- multiple OpenFlow protocol versions with different sets of
sions to Mininet for enabling large-scale simulations. Mini- required and optional capabilities, a roadblock for SDN
net CE combines groups of Mininet instances into one practitioners, tinyNBI [419], has been proposed as a simple
cluster of simulator instances to model global-scale net- API providing a unifying set of core abstractions of five
works. SDN Cloud DC enhances Mininet and POX to OpenFlow protocol versions (from 1.0 to 1.4). Ongoing
emulate an SDN-based intra-DC network by implementing efforts to introduce a new HAL for non-OpenFlow capable
new software modules such as data center topology discovery devices [420] include the development of open source
and network traffic generation. Recent emulation platform artifacts like Revised OpenFlow Library (ROFL) and the
proposals that enable large-scale experiments following a eXtensible DataPath daemon (xDPd), a framework for
distributed approach include Max-iNet [405], DOT [406], creating new OpenFlow data path implementations based
and CityFlow [407]. The latter is a project with the main goal on a diverse set of hardware and software platforms. A
of building an emulated control plane for a city of one million related open source effort to develop a common library to
inhabitants. Such initiatives are a starting point to provide implement OpenFlow 1.0 and 1.3 protocol endpoints
experimental insights for large-scale SDN deployments. (switch agents and controllers) is libfluid [421], winner of
The capability of simulating OpenFlow devices has also the OpenFlow driver competition organized by the ONF.
been added to the popular ns-3 simulator [408]. Another Within the ONF, the Forwarding Abstraction Working
simulator is fs-sdn, which extends the fs simulation engine Group (FAWG) is pursuing another solution to the hetero-
[409] by incorporating a controller and switching compo- geneity problem, through table type patterns (TTPs) [121].
nents with OpenFlow support. Its main goal is to provide a A TTP is a standards-based and negotiated switch-level
more realistic and scalable simulation platform as com- behavioral abstraction. It consists of the relationships
pared to Mininet. Finally, STS [410] is a simulator de- between tables forming a graph structure, the types of
signed to allow developers to specify and apply a variety of tables in the graph, a set of the parameterized table pro-
test cases, while allowing them to interactively examine perties for each table in the graph, the legal flow-mod
the state of the network. and table-mod commands for each flow table, and the

metadata mask that can be passed between each table pair agent inside the switch [418]. Our current understanding
in the graph. indicates that an effective way forward is a native design of
SDN switches consistent with the evolution of the south-
2) Flow Table Capacity: Flow matching rules are stored bound API standardization activities [121], [435].
in flow tables inside network devices. One practical chal-
lenge is to provide switches with large and efficient flow 4) Evolving Switch Designs and Hardware Enhancements:
tables to store the rules [422]. TCAMs are a common choice As in any software/hardware innovation cycle, a number of
to hold flow tables. While flexible and efficient in terms of advancements are to be expected from the hardware per-
matching capabilities, TCAMs are costly and usually small spective to improve SDN capabilities and performance.
(from 4000 to 32 000 entries). Some TCAM chips today New SDN switch designs are appearing in a myriad of
integrate 18 million-bit (configured as 500 000 entries hardware combinations to efficiently work together with
36 bit per entry) into a single chip working at 133 MHz TCAMs, such as static random-access memory (SRAM),
[423], i.e., capable of 133 million lookups per second. dynamic random-access memory (DRAM), reduced-
However, these chips are expensive and have a high-power latency DRAM, graphics processing unit (GPU), field-
consumption [424], representing a major power drain in a programmable gate array (FPGA), network processors,
switching device [425]. These are some of the reasons why CPUs, among other specialized network processors
currently available OpenFlow devices have TCAMs with [436]–[441]. These early works suggest the need for
roughly 8000 entries, where the actual capacity in terms of additional efforts into new hardware architectures for
OpenFlow table size has a nontrivial relationship to the future SDN switching devices. For instance, some pro-
type of flow entries being used [426], [427]. OpenFlow posals target technologies such as GPUs that have demon-
version 1.1 introduced multiple tables, thereby adding strated 20 gigabits per second (Gb/s) with flow tables of
extra flexibility and scalability. Indeed, OpenFlow 1.0 up to 1 million exact match entries and up to 1000
implied state explosion due to its flat table model [121]. wildcard entries [438]. Alternatives to TCAM-based de-
However, supporting multiple tables in hardware is signs include new hardware architectures and com-
challenging and limitedVyet another motivation for the ponents, as well as new and more scalable forwarding
ongoing ONF FAWG work on TTPs [121]. planes, such as the one proposed by the Rain Man firm-
Some efforts focus on compression techniques to re- ware [442]. Other design solutions, such as parallel lookup
duce the number of flow entries in TCAMs [428]–[430]. models [443], can also be applied to SDN to reduce costs in
The Espresso heuristic [430] can be used to compress wild switching and routing devices. Recent proposals on cache-
cards of OpenFlow-based interdomain routing tables, re- like OpenFlow switch arrangements [444] shed some light
ducing the forwarding information base (FIB) by 17% and, on overcoming the practical limitations of flow table sizes
consequently, saving up to 40 000 flow table entries [428]. with clever switching designs. Additionally, counters re-
Shadow MACs [429] propose label switching for solving present another practical challenge in SDN hardware im-
two problems, consistent updates and rule space exhaus- plementations. Many counters already exist, and they
tion, by using opaque values (similar to MPLS labels) to could lead to significant control plane monitoring over-
encode fine-grained paths as labels. A major benefit of head [418]. Software-defined counters (SDCs) [434] have
fixed-size labels is relying on exact-math lookups which been proposed to provide both scalability and flexibility.
can be easily and cost-effectively implemented by simple Application-aware SDN architectures are being pro-
hardware tables instead of requiring rules to be encoded in posed to generalize the standard OpenFlow forwarding
expensive TCAM tables. abstractions by including stateful actions to allow process-
ing information from layers 4 to 7 [445]. To this end,
3) Performance: Today, the throughput of commercial application flow tables are proposed as data plane applica-
OpenFlow switches varies from 38 to 1000 flow-mod tion modules that require only local state, i.e., do not
per second, with most devices achieving a throughput depend on a global view of the network. Those tiny appli-
lower than 500 flow-mod per second [431], [432]. This cation modules run inside the forwarding devices (and can
is clearly a limiting factor that will be addressed in the be installed on demand), alleviating the overhead on the
switch design processVsupport of OpenFlow in existing control plane and augmenting the efficiency of certain
product lines has been more a retrofitting activity than a tasks, which can be kept in the data plane. Similarly, other
clean feature planning and implementation activity. De- initiatives propose solutions based on preinstalled state
ployment experiences [433] have pointed to a series of machines. Flow-level State Transition (FAST) [446] allows
challenges stemming from the limited embedded CPU controllers to proactively program state transitions in for-
power of current commercial OpenFlow switches. One warding devices, allowing switches to run dynamic actions
approach to handle the problem consists of adding more that require only local information.
powerful CPUs into the switches, as proposed in [434]. Other approaches toward evolving switch designs in-
Others have proposed to rethink the distribution of control clude CAching in Buckets (CAB), a reactive wildcard
actions between external controllers and the OpenFlow caching proposal that uses a geometric representation of

the rule set, which is divided into small logical structures overcoming some of the limitations of OpenFlow (e.g.,
(buckets) [447]. Through this technique, CAB is able to expressiveness, support of user-defined protocols, memory
solve the rule dependency problem and achieve efficient efficiency), through generic flow instruction sets. Open
usage of control plane resources, namely bandwidth, con- source prototypes are available [31] as well as evaluation
troller processing load, and flow setup latency. results showing the line-speed capabilities using a network
New programmable Ethernet switch chips, such as processing unit (NPU)-based [453] proof of concept imple-
XPliant Ethernet [448], are emerging into this new market mentation. In this line, we already mentioned OpenState
of programmable networks. Its main aim is enabling new [155], another initiative that aims to augment the capabi-
protocol support and the addition of new features through lity and flexibility of forwarding devices. By taking advan-
software updates, increasing flexibility. One example of tage of eXtended Finite State Machines (XFSMs) [454],
such flexibility is the support of GENEVE [39], a recent [455], OpenState proposes an abstractionVas a super set
effort toward generic network virtualization encapsulation of OpenFlow primitivesVto enable stateful handling of
protocols, and OpenFlow. The throughput of the first OpenFlow rules inside forwarding devices.
family of XPliant Ethernet chip varies from 880 Gb/s to In the same way as TTPs allow controllers to compile
3.2 Terabits per second (Tb/s), supporting up to 64 ports of the right set of low-lever instructions known to be sup-
40 GbE or 50 GbE, for instance. ported by the switches, a new breed of switch referred to as
Microchip companies like Intel are already shipping programmable, protocol-independent packet processor
processors with flexible SDN capabilities to the market (P4) [456] suggests an evolution path for OpenFlow,
[449]. Recent advances in general-purpose CPU technol- based on a high-level compiler. This proposal would allow
ogy include a data plane development kit (DPDK) [450] the functionality of programmable switches (i.e., pipeline,
that allows high-level programming of how data packets header parsing, field matching) to be not only specified by
will be processed directly within network interface cards. the controller but also changed in the field. In this model,
Prototype implementations of Intel DPDK accelerated programmers are able to decide how the forwarding plane
switch shows the potential to deliver high-performance processes packets without caring about implementation
SDN software switches [441]. This trend is likely to conti- details. It is then the compiler that transforms the impera-
nue since high-speed and specialized hardware is needed tive program into a control flow graph that can be mapped
to boost SDN performance and scalability for large, real- to different target switches.
world networks. Hardware-programmable technologies
such as FPGA are widely used to reduce time and costs B. Controller Platforms
of hardware-based feature implementations. NetFPGA, for In the SDN model, the controller platform is a critical
instance, has been a pioneering technology used to imple- pillar of the architecture, and, as such, efforts are being
ment OpenFlow 1.0 switches [437], providing a commod- devoted to turn SDN controllers into high-performance,
ity cost-effective prototyping solution. Another line of scalable, distributed, modular, and highly available
work on SDN data planes proposes to augment switches programmer-friendly software. Distributed controller plat-
with FPGA to (remotely) define the queue management forms, in particular, have to address a variety of challenges.
and scheduling behavior of packet switches [451]. Finally, Deserving special consideration are the latency between
recent developments have shown that state-of-the-art forwarding devices and controller instances, fault toler-
system-on-chip (SoC) platforms, such as the Xilinx Zynq ance, load balancing, consistency, and synchronization,
ZC706 board, can be used to implement OpenFlow devices among other issues [7], [457], [458]. Operators should
yielding 88 Gb/s throughput for 1000 flow supporting also be able to observe and understand how the combina-
dynamic updates [452]. tion of different functions and modules can impact their
network [459].
5) Native SDN Switch Designs: Most of the SDN switch As the SDN community learns from the development
(re)design efforts so far follow an evolutionary approach to and operational experiences with OpenFlow controllers
retrofit OpenFlow-specific programmable features into (e.g., Beacon [186]), further advancements are expected in
existing hardware layouts, following common wisdom on terms of raw performance of controller implementations,
switch/router designs and consolidated technologies (e.g., including the exploitation of hierarchical designs and
SRAM, TCAM, FPGA). One departure from this approach optimized buffer sizing [460]. One approach to increase
is the ongoing work on forwarding meta-morphosis [435], the performance of controllers is the IRIS IO engine [461],
a reconfigurable match table model inspired from RISC- enabling significant increases in the flow-setup rate of
like pipeline architecture applied to switching chips. This SDN controllers. Another way of reducing the control
work illustrates the feasibility of realizing a minimal set of plane overhead is by keeping a compressed copy of the
action primitives for flexible header processing in hard- flow tables in the controller’s memory [462].
ware, at almost no additional cost or power. Also in line
with the core SDN goals of highly flexible and program- 1) Modularity and Flexibility: A series of ongoing re-
mable (hardware-based) data planes, POF [120] aims at search efforts target the modular and flexible composition

of controllers. RAON [463] proposes a recursive abstrac- by automatically and transparently resolving conflicts. In
tion of OpenFlow controllers where each controller sees other words, Statesman allows a safe composition of unco-
the controllers below as OpenFlow switches. Open re- ordinated or conflicting application’s actions.
search issues include the definition of suitable interfaces Another recent approach to simplify network manage-
between the different layers in such a hierarchy of con- ment is the idea of compositional SDN hypervisors [181].
trollers. Other open issues to be further investigated in this Its main feature is allowing applications written in differ-
context are the east/westbound APIs, and their use in ent languages, or on different platforms, to work together
enabling suitable hierarchical designs to achieve scalabil- in processing the same traffic. The key integration compo-
ity, modularity, and security [218]. For instance, each level nent is a set of simple prioritized lists of OpenFlow rules,
of a hierarchy of controllers can offer different abstractions which can be generated by different programming lan-
and scopes for either intradata and interdata center rout- guages or applications.
ing, thus increasing scalability and modularity. Similarly,
from a security perspective, each hierarchical level may 3) High Availability: In production, SDN controllers
be a part of a different trust domain. Therefore, east/ need to sustain healthy operation under the pressure of
westbound interfaces between the different layers of con- different objectives from the applications they host. Many
trollers should be capable of enforcing both intradomain advances are called for in order to deal with potential risk
and interdomain security policies. vectors of controller-based solutions [359]. Certainly,
Another important observation is that, currently, the many solutions will leverage on results from the distrib-
lack of modularity in most SDN controllers forces devel- uted systems and security communities made over the last
opers to re-implement basic network services from scratch decade. For instance, recent efforts propose consistent,
in each new application [29]. fault-tolerant data stores for building reliable distributed
As in software engineering in general, lack of modu- controllers [198], [213], [458].
larity results in controller implementations that are Another possible approach toward building low laten-
hard to build, maintain, and extendVand ultimately be- cy, highly available SDN controllers is to exploit controller
come resistant to further innovations, resembling tradi- locality [457], [466]. Classical models of distributed sys-
tional ‘‘hardware-defined’’ networks. As surveyed in tems, such as LOCAL and CONGEST [467], can be ex-
Section IV-G, SDN programming abstractions (e.g., Pyretic plored to solve this problem. Those models can be used to
[225]) introduce modularity in SDN applications and develop coordination protocols that enable each controller
simplify their development altogether. Further research to take independent actions over events that take place in
efforts (e.g., Corybantic [464]) try to achieve modularity in its local neighborhood [457].
SDN control programs. Other contributions toward Another core challenge relates to the fundamental
achieving modular controllers can be expected from other tradeoffs between the consistency model of state distribu-
areas of computer science (e.g., principles from Operating tion in distributed SDN controllers, the consistency re-
System [196]) and best practices of modern cloud-scale quirements of control applications, and performance
software applications. [466]. To ease development, the application should ideally
not be aware of the vagaries of distributed state. This im-
2) Interoperability and Application Portability: Similarly to plies a strong consistency model, which can be achieved
forwarding device vendor agnosticism that stems from with distributed data stores as proposed recently [213].
standard southbound interfaces, it is important to foster However, keeping all control data in a consistent distri-
interoperability between controllers. Early initiatives to- buted data store is unfeasible due to the inherent perfor-
ward more interoperable control platforms include porta- mance penalties. Therefore, hybrid solutions are likely to
ble programming languages such as Pyretic [225] and east/ coexist requiring application developers to be aware of the
westbound interfaces among controllers, such as SDNi tradeoffs and penalties of using, or not, a strong consis-
[209], ForCES CE–CE interface [30], [211], and ForCES tency model, a tenet of the distributed Onix controller [7].
Intra-NE mechanisms [212]. However, these efforts are yet High availability can also be achieved through
far from fully realizing controller interoperability and ap- improved southbound APIs and controller placement heu-
plication portability. ristics and formal models [468]–[470]. These aim to maxi-
In contrast to Pyretic [248], PANE [197], Maple [263], mize resilience and scalability by allowing forwarding
and Corybantic [464], which are restricted to traffic engi- devices to connect to multiple controllers in a cost-effec-
neering applications and/or impose network state conflict tive and efficient way [469]. Early efforts in this direction
resolution at the application level (making application de- have already shown that forwarding devices connecting to
sign and testing more complicated), Statesman [465] pro- two or three controllers can typically achieve high availa-
poses a framework to enable a variety of loosely coupled bility (up to five nines) and robustness in terms of control
network applications to coexist on the same control plane plane connectivity [468], [470]. It has also been shown
without compromising network safety and performance. that the number of required controllers is more dependent
This framework makes application development simpler on the topology than on network size [468]. Another

finding worth mentioning is the fact that for most common comes a challenge when considering critical control plane
topologies and network sizes fewer than ten controllers functions such as those related to link failure detection or
seem to be enough [468]. fast reaction decisions. The resilience of an OpenFlow
network depends on fault tolerance in the data plane (as in
4) Delegation of Control: To increase operational effi- traditional networks) but also on the high availability of
ciency, SDN controllers can delegate control functions to the (logically) centralized control plane functions. Hence,
report state and attribute value changes, threshold crossing the resilience of SDN is challenging due to the multiple
alerts, hardware failures, and so forth. These notifications possible failures of the different pieces of the architecture.
typically follow a publish/subscribe model, i.e., controllers As noted in [477], there is a lack of sufficient research
and applications subscribe (on demand) to the particular and experience in building and operating fault-tolerant
class of notifications they are interested in. In addition, SDNs. Google B4 [8] may be one of the few examples that
these subsystems may provide resilience and trustworthi- have proven that SDN can be resilient at scale. A number
ness properties [471]. of related efforts [357], [262], [363], [478]–[483] have
Some reasons for delegating control to the data plane started to tackle the concerns around control plane split
include [218]: architectures. The distributed controller architectures sur-
• low latency response to a variety of network events; veyed in Section IV-D are examples of approaches toward
• the amount of traffic that must be processed in the resilient SDN controller platforms with different tradeoffs
data plane, in particular in large-scale networks in terms of consistency, durability, and scalability.
such as data centers; On a detailed discussion on whether the CAP theorem
• low-level functions such as those (byte or bit [484] applies to networks, Panda et al. [479] argue that the
oriented) required by repetitive synchronous di- tradeoffs in building consistent, available, and partition-
gital hierarchy (SDH) [472] multiplex section tolerant distributed databases (i.e., CAP theorem) are ap-
overhead; plicable to SDN. The CAP theorem demonstrates that it is
• functions well understood and standardized, such impossible for data store systems to simultaneously
as encryption, BIP [473], AIS [474] insertion, MAC achieve strong consistency, availability, and partition
learning, and codec control message (CCM) [475] tolerance. While availability and partition tolerance pro-
exchanges; blems are similar in both distributed databases and net-
• controller failure tolerance, i.e., essential network works, the problem of consistency in SDN relates to the
functions should be able to keep a basic network consistent application of policies.
operation even when controllers are down; Considering an OpenFlow network, when a switch
• basic low-level functions usually available in data detects a link failure (port-down event), a notification
plane silicon, such as protection switching state is sent to the controller, which then takes the required
machines, CCM counters, and timers; actions (reroute computation, precomputed backup path
• all those functions that do not add any value when lookup) and installs updated flow entries in the required
moved from the data to the control plane. switches to redirect the affected traffic. Such reactive
Strong candidates for execution in the forwarding strategies imply high restoration time due to the necessary
devices instead of being implemented in the control interaction with the controller and additional load on the
platforms thus include OAM, ICMP processing, MAC control channel. One experimental work on OpenFlow for
learning, neighbor discovery, defect recognition, and in- carrier-grade networks investigated the restoration process
tegration [218]. This would not only reduce the overhead and measured a restoration times in the order of 100 ms
(traffic and computing) of the control plane, but also im- [478]. The delay introduced by the controller may, in some
prove network efficiency by keeping basic networking cases, be prohibitive.
functions in the data plane. In order to meet carrier grade requirements (e.g.,
50 ms of recovery time), protection schemes are required
C. Resilience to mitigate the effects of a separate control plane. Suitable
Achieving resilient communication is a top purpose of protection mechanisms (e.g., installation of preestablished
networking. As such, SDNs are expected to yield the same backup paths in the forwarding devices) can be imple-
levels of availability as legacy and any new alternative mented by means of OpenFlow group table entries using
technology. Split control architectures as SDN are com- ‘‘fast-fail-over’’ actions. An OpenFlow fault management
monly questioned [476] about their actual capability of approach [357] similar to MPLS global path protection
being resilient to faults that may compromise the control- could also be a viable solution, provided that OpenFlow
to-data plane communications and thus result in ‘‘brain- switches are extended with end-to-end path monitoring
less’’ networks. Indeed, the malfunctioning of particular capabilities similarly to those specified by bidirectional
SDN elements should not result in the loss of availability. forwarding detection (BFD) [485]. Such protection
The relocation of SDN control plane functionality, from schemes are a critical design choice for larger scale net-
inside the boxes to remote, logically centralized loci, be- works and may also require considerable additional flow

space. By using primary and secondary path pairs prog- As a result, several efforts have been devoted to tackle
rammed as OpenFlow fast fail-over group table entries, a the SDN scaling concerns, including DevoFlow [418],
path restoration time of 3.3 ms has been reported [486] SDCs [434], DIFANE [489], Onix [7], HyperFlow [195],
using BFD sessions to quickly detect link failures. Kandoo [229], Maestro [188], NOX–MT [187], and Maple
On a related line of data plane resilience, SlickFlow [263]. Still related to scalability, the notion of elasticity in
[482] leverages the idea of using packet header space to SDN controllers is also being pursued [228], [363], [481].
carry alternative path information to implement resilient Elastic approaches include dynamically changing the
source routing in OpenFlow networks. Under the presence number of controllers and their locations under different
of failures along a primary path, packets can be rerouted to conditions [490].
alternative paths by the switches themselves without in- Most of the research efforts addressing scaling limi-
volving the controller. Another recent proposal that uses tations of SDN can be classified in three categories: data
in-packet information is INFLEX [483], an SDN-based plane, control plane, and hybrid. While targeting the data
architecture for cross-layer network resilience which pro- plane, proposals such as DevoFlow [418] and SDCs [434]
vides on-demand path fail-over by having endpoints tag actually reduce the overhead of the control plane by dele-
packets with virtual routing plane information that can be gating some work to the forwarding devices. For instance,
used by egress routers to reroute by changing tags upon instead of requesting a decision from the controller for
failure detection. every flow, switches can selectively identify the flows (e.g.,
Similarly to SlickFlow, OSP [280] proposes a protec- elephant flows) that may need higher level decisions from
tion approach for data plane resilience. It is based on the control plane applications. Another example is to in-
protecting individual segments of a path avoiding the introduce more powerful general purpose CPUs in the for-
tervention of the controller upon failure. The recovery warding devices to enable SDCs. A general purpose CPU
time depends on the failure detection time, i.e., a few tens and SDCs offer new possibilities for reducing the control
of milliseconds in the proposed scenarios. In the same plane overhead by allowing software-based implementa-
direction, other proposals are starting to appear for ena- tions of functions for data aggregation and compression,
bling fast-fail-over mechanisms for link protection and for instance.
restoration in OpenFlow-based networks [487]. Maestro [188], NOX–MT [187], Kandoo [229], Beacon
Language-based solutions to the data plane fault- [186], and Maple [263] are examples of the effort on
tolerance problem have also been proposed [262]. In this designing and deploying high-performance controllers,
work, the authors propose a language that compiles regular i.e., trying to increase the performance of the control
expressions into OpenFlow rules to express what network plane. These controllers mainly explore well-known tech-
paths packets may take and what degree of (link level) fault niques from networking, computer architectures, and
tolerance is required. Such abstractions around fault to- high-performance computing, such as buffering, pipelin-
lerance allow developers to build fault recovery capabilities ing, and parallelism, to increase the throughput of the
into applications without huge coding efforts. control platform.
The hybrid category is composed of solutions that try to
split the control logic functions between specialized data
D. Scalability plane devices and controllers. In this category, DIFANE
Scalability has been one of the major concerns of SDNs [489] proposes authoritative (intermediate) switches to
from the outset. This is a problem that needs to be keep all traffic in the data plane, targeting a more scalable
addressed in any systemVe.g., in traditional networksV and efficient control plane. Authoritative switches are re-
and is obviously also a matter of much discussion in the sponsible for installing rules on the remaining switches,
context of SDN [11]. Most of the scalability concerns in while the controller is still responsible for generating all
SDNs are related to the decoupling of the control and data the rules required by the logic of applications. By dividing
planes. Of particular relevance are reactive network con- the controller work with these special switches, the overall
figurations where the first packet of a new flow is sent by system scales better.
the first forwarding element to the controller. The addi- Table 12 provides a nonexhaustive list of proposals
tional control plane traffic increases network load and addressing scalability issues of SDN. We characterize these
makes the control plane a potential bottleneck. Addition- issues by application domain (control or data plane), their
ally, as the flow tables of switches are configured in real purpose, the throughput in terms of number of flows per
time by an outside entity, there is also the extra latency second (when the results of the experiments are reported),
introduced by the flow setup process. In large-scale net- and the strategies used. As can be observed, the vast ma-
works, controllers will need to be able to process millions jority are control plane solutions that try to increase scala-
of flows per second [488] without compromising the bility by using distributed and multicore architectures.
quality of its service. Therefore, these overheads on the Some figures are relatively impressive, with some solu-
control plane and on flow setup latency are (arguably) two tions achieving up to 20 million flows/s. However, we
of the major scaling concerns in SDN. should caution the reader that current evaluations

Table 12 Summary and Characterization of Scalability Proposals for SDNs
consider only simple applications and count basically the E. Performance Evaluation
number of packet-in and packet-out messages to As introduced in Section IV-A, there are already several
measure throughput. The actual performance of control- OpenFlow implementations from hardware and software
lers will be affected by other factors, such as the number vendors being deployed in different types of networks,
and complexity of the applications running on the from small enterprise to large-scale data centers. There-
controller and security mechanisms implemented. For fore, a growing number of experiments over SDN-enabled
example, a routing algorithm consumes more computing networks is expected in the near future. This will naturally
resources and needs more time to execute than a simple create new challenges, as questions regarding SDN perfor-
learning switch application. Also, current evaluations are mance and scalability have not yet been properly inves-
done using plain TCP connections. The performance is tigated. Understanding the performance and limitation of
very likely to change when basic security mechanisms are the SDN concept is a requirement for its implementation
put in place, such as TLS, or more advanced mechanisms in production networks. There are very few performance
to avoid eavesdropping, man-in-the-middle and DoS evaluation studies of OpenFlow and SDN architecture.
attacks on the control plane. Although simulation studies and experimentation are
Another important issue concerning scalability is data among the most widely used performance evaluation tech-
distribution among controller replicas in distributed archi- niques, analytical modeling has its own benefits as well. A
tectures. Distributed control platforms rely on data distri- closed-form description of a networking architecture paves
bution mechanisms to achieve their goals. For instance, the way for network designers to have a quick (and
controllers such as Onix, HyperFlow, and ONOS need approximate) estimate of the performance of their design,
mechanisms to keep a consistent state in the distributed without the need to spend considerable time for simula-
control platform. Recently, experimental evaluations have tion studies or expensive experimental setup [433].
shown that high-performance distributed and fault-toler- Some work has investigated ways to improve the per-
ant data stores can be used to tackle such challenges [213]. formance of switching capabilities in SDN. These mainly
Nevertheless, further work is necessary to properly under- consist of observing the performance of OpenFlow-
stand state distribution tradeoffs [466]. enabled networks regarding different aspects, such as

lookup performance [492], hardware acceleration [439], device performance and better decide which one is best
the influence of types of rules and packet sizes [493], per- suited for the target network infrastructure.
formance bottlenecks of current OpenFlow implementa- Surprisingly, despite being designed to evaluate the
tions [418], how reactive settings impact the performance performance of controllers, Cbench is currently a single-
on data center networks [494], and the impact of configu- threaded tool. Therefore, multiple instances have to be
ration on OpenFlow switches [392]. started to utilize multiple CPUs. It also only establishes
Design choices can have a significant impact on the one controller connection for all emulated switches. Un-
lookup performance of OpenFlow switching in Linux fortunately, this means little can be derived from the
operating system using standard commodity network results in terms of controller performance and behavior or
interface cards [492]. Just by using commodity network estimation of different bounds at the moment. For in-
hardware the packet switching throughput can be im- stance, aggregated statistics are gathered for all switches
proved by up to 25% when compared to one based on soft but not for each individual switch. As a result, it is not
OpenFlow switching [492]. Similarly, hardware accel- possible to identify whether all responses of the controller
eration based on network processors can also be applied are for a single switch, or whether the capacity of the
to perform OpenFlow switching. In such cases, early re- controller is actually shared among the switches. Flexible
ports indicate that performance, in terms of packet delay, OpenFlow controller benchmarks are available though.
can be improved by 20% when compared to conventional OFCBenchmark [393] is one of the recent developments.
designs [439]. It creates a set of message-generating virtual switches,
By utilizing Intel’s DPDK library [450], it has been which can be configured independently from each other to
shown that it is possible to provide flexible traffic steering emulate a specific scenario and to maintain their own
capability at the hypervisor level (e.g., KVM) without the statistics.
performance limitations imposed by traditional hardware Another interesting question to pose when evaluating
switching techniques [495], such as SR–IOV [496]. This is the performance of SDN architectures is what is the re-
particularly relevant since most of the current enterprise quired number of controllers for a given network topology
deployments of SDN are in virtualized data center infra- and where to place the controllers [469], [497]. By analyz-
structures, as in VMware’s NVP solution [112]. ing the performance of controllers in different network
Current OpenFlow switch implementations can lead to topologies, it is possible to conclude that one controller is
performance bottlenecks with respect to the CPU load often enough to keep the latency at a reasonable rate [497].
[418]. Yet, modifications on the protocol specification can Moreover, as observed in the same experiments, in the
help reduce the occurrence of these bottlenecks. Further general case adding k controllers to the network can re-
investigations provide measurements regarding the per- duce the latency by a factor of k. However, there are cases,
formance of the OpenFlow switch for different types of such as large-scale networks and WANs, where more con-
rules and packet sizes [493]. trollers should be deployed to achieve high reliability and
In data centers, a reactive setting of flow rules can lead low control plane latency.
to an unacceptable performance when only eight switches Recent studies also show that the SDN control plane
are handled by one OpenFlow controller [494]. This cannot be fully physically centralized due to responsive-
means that large-scale SDN deployments should probably ness, reliability, and scalability metrics [466], [469].
not rely on a purely reactive ‘‘modus operandi,’’ but rather Therefore, distributed controllers are the natural choice
on a combination of proactive and reactive flow setup. for creating a logically centralized control plane, while be-
To foster the evaluation of different performance as- ing capable of coping with the demands of large-scale
pects of OpenFlow devices, frameworks such as OFLOPS networks. However, distributed controllers bring addition-
[381], OFLOPS-Turbo [395], Cbench [187], and OFC- al challenges, such as the consistency of the global network
Benchmark [393] have been proposed. They provide a set view, which can significantly affect the performance of the
of tools to analyze the performance of OpenFlow switches network if not carefully engineered. Taking two applica-
and controllers. Cbench [187], [392] is a benchmark tool tions as examples, one that ignores inconsistencies and
developed to evaluate the performance of OpenFlow con- another that takes inconsistency into consideration, it is
trollers. By taking advantage of the Cbench, it is possible possible to observe that optimality is significantly affected
to identify performance improvements for OpenFlow con- when inconsistencies are not considered and that the
trollers based on different environment and system con- robustness of an application is increased when the con-
figurations, such as the number of forwarding devices, troller is aware of the network state distribution [466].
network topology, overall network workload, type of Most of these initiatives toward identifying the limita-
equipments, forwarding complexity, and overhead of the tions and bottlenecks of SDN architectures can take a lot of
applications being executed on top of controllers [187]. time and effort to produce consistent outputs due to the
Therefore, such tools can help system designers make practical development and experimentation requirements.
better decisions regarding the performance of devices and As mentioned before, analytic models can quickly pro-
the network, while also allowing end users to measure the vide performance indicators and potential scalability

bottlenecks for an OpenFlow switch-controller system be-

fore detailed data are available. While simulation can pro-
vide detailed insight into a certain configuration, the
analytical model greatly simplifies a conceptual deploy-
ment decision. For instance, a Network calculus-based
model can be used to evaluate the performance of an SDN
switch and the interaction of SDN switches and controllers
[498]. The proposed SDN switch model captured the
closed form of the packet delay and buffer length inside the
SDN switch according to the parameters of a cumulative
arrival process. Using recent measurements, the authors
have reproduced the packet processing delay of two va-
riants of OpenFlow switches and computed the buffer re-
quirements of an OpenFlow controller. Analytic models
based on queuing theory for the forwarding speed and
blocking probability of current OpenFlow switches can also Fig. 10. Main threat vectors of SDN architectures.
be used to estimate the performance of the network [492].
F. Security and Dependability

attacks on control plane communication and logically cen-
Cyber attacks against financial institutions, energy fa-
tralized controllers. It is worth mentioning that most
cilities, government units, and research institutions are
threats vectors are independent of the technology or the
becoming one of the top concerns of governments and
protocol (e.g., OpenFlow, POF, and ForCES), because they
agencies around the globe [499]–[504]. Different inci-
represent threats on conceptual and architectural layers of
dents, such as Stuxnet [503], have already shown the per-
SDN itself.
sistence of threat vectors [505]. Put another way, these
As shown in Fig. 10 and Table 13, there are at least
attacks are capable of damaging a nation’s wide infrastruc-
seven identified threats vector in SDN architectures. The
ture, which represent a significant and concerning issue.
first threat vector consists of forged or faked traffic flows in
As expected, one of the most common means of executing
the data plane, which can be used to attack forwarding
those attacks is through the network, either the Internet or
devices and controllers. The second allows an attacker to
the local area network. It can be used as a simple transport
exploit vulnerabilities of forwarding devices and conse-
infrastructure for the attack or as a potentialized weapon
quently wreak havoc with the network. Threat vectors
to amplify the impact of the attack. For instance, high
three, four, and five are the most dangerous ones, since
capacity networks can be used to launch large-scale at-
they can compromise the network operation. Attacks on
tacks, even though the attacker has only a low capacity
the control plane, controllers, and applications can easily
network connection at his premises.
grant an attacker the control of the network. For instance,
Due to the danger of cyber attacks and the current
a faulty or malicious controller or application could be
landscape of digital threats, security and dependability are
used to reprogram the entire network for data theft pur-
top priorities in SDN. While research and experimen-
poses, e.g., in a data center. The sixth threat vector is
tation on SDNs is being conducted by some commercial
linked to attacks on and vulnerabilities in administrative
players (e.g., Google, Yahoo!, Rackspace, Microsoft),
stations. A compromised critical computer, directly con-
commercial adoption is still in its early stage. Industry
nected to the control network will empower the attacker
experts believe that security and dependability are issues
that need to be addressed and further investigated in SDN Table 13 SDN Specific Versus Nonspecific Threats
[359], [506], [507].
Additionally, from the dependability perspective, avail-
ability of Internet routers is today a major concern with the
widespread of clouds and their strong expectations about
the network [508]. It is, therefore, crucial to achieve high
levels of availability on SDN control platforms if they
are to become the main pillars of networked applica-
tions [468].
Different threat vectors have already been identified in
SDN architectures [359], as well as several security issues
and weaknesses in OpenFlow-based networks [194], [201],
[509]–[514]. While some threat vectors are common to
existing networks, others are more specific to SDN, such as

Table 14 Attacks to OpenFlow Networks
with resources to launch more easily an attack to the con- estingly, all reported threats and attacks affect all versions
troller, for instance. Last, threat vector number seven re- (1.0 to 1.3.1) of the OpenFlow specification. It is also
presents the lack of trusted resources for forensics and worth emphasizing that some attacks, such as spoofing, are
remediation, which can compromise investigations (e.g., not specific to SDN. However, these attacks can have a
forensics analysis) and preclude fast and secure recovery larger impact in SDNs. For instance, by spoofing the ad-
modes for bringing the network back into a safe operation dress of the network controller, the attacker (using a fake
condition. controller) could take over the control of the entire net-
As can be observed in Table 13, threat vectors 3 to 5 are work. A smart attack could persist for only a few seconds,
specific to SDN as they stem from the separation of the i.e., just the time needed to install special rules on all
control and data planes and the consequent introduction of forwarding devices for its malicious purposes (e.g., traffic
a new entity in these networksVthe logically centralized cloning). Such attack could be very hard to detect.
controller. The other vectors were already present in tra- Taking counter falsification as another example, an
ditional networks. However, the impact of these threats attacker can try to guess installed flow rules and, subse-
could be larger than todayVor at least it may be expressed quently, forge packets to artificially increase the counter.
differentlyVand as a consequence it may need to be dealt Such attack would be specially critical for billing and load
with differently. balancing systems, for instance. A customer could be
OpenFlow networks are subject to a variety of security charged for more traffic than she, in fact used, while a load
and dependability problems such as spoofing [509], tam- balancing algorithm may take nonoptimal decisions due to
pering [509], repudiation [509], information disclosure forged counters.
[509], denial of service [509], [511], [512], elevation of Flow networks include the lack of strong security re-
privileges [509], and the assumption that all applications commendations for developers, the lack of TLS and access
are benign and will not affect SDN operation [194]. The control support on most switch and controller implemen-
lack of isolation, protection, access control, and stronger tations [510], the belief that TCP is enough because links
security recommendations [194], [201], [510]–[512] are are ‘‘physically secure’’ [510], [512], the fact that many
some of the reasons for these vulnerabilities. We will ex- switches have listener mode activated by default (allowing
plore these next. the establishment of malicious TCP connections, for in-
stance) [512] or that flow table verification capabilities are
1) OpenFlow Security Assessment: There is already a harder to implement when TLS is not in use [260], [510].
number of identified security issues in OpenFlow-enabled In addition, the high denial of service risk posed to cen-
networks. Starting from a STRIDE methodology [515], it is tralized controllers is worth mentioning [260], [511], as
possible to identify different attacks to OpenFlow-enabled well as the vulnerabilities in the controllers themselves
networks. Table 14 summarizes these attacks (based on [260], [359], bugs and vulnerabilities in applications [516],
[509]). For instance, information disclosure can be targeted flooding attacks [16], insecure northbound inter-
achieved through side channel attacks targeting the flow faces that can lead to security breaches [16], and the risk of
rule setup process. When reactive flow setup is in place, resource depletion attacks [511], [512]. For instance, it has
obtaining information about network operation is rela- been shown that an attacker can easily compromise control
tively easy. An attacker that measures the delay experi- plane communications through DoS attacks and launch a
enced by the first packet of a flow and the subsequent can resource depletion attack on control platforms by exploiting
easily infer that the target network is a reactive SDN, and a single application such as a learning switch [511], [512].
proceed with a specialized attack. This attackVknown as Another point of concern is the fact that current con-
fingerprinting [511]Vmay be the first step to launch a DoS trollers, such as Floodlight, OpenDaylight, POX, and
attack intended to exhaust the resources of the network, Beacon, have several security and resiliency issues [194].
for example. If the SDN is proactive, guessing its forward- Common application development problems (bugs), such
ing rule policies is harder, but still feasible [509]. Inter- as the sudden exit of an application or the continuous

Table 15 Countermeasures for Security Threats in OpenFlow Networks Rate limiting, packet dropping, shorter timeouts, and
flow aggregations are techniques that can be applied on
controllers and forwarding devices to mitigate different
types of attacks, such as denial of service and information
disclosure. For instance, reduced timeouts can be used to
mitigate the effect of an attack exploring the reactive ope-
ration mode of the network to make the controller install
rules that divert traffic to a malicious machine. With re-
duced timeouts, the attacker would be forced to constantly
generate a number of forged packets to avoid timeout
expiration, making the attack more likely to be detected.
Rate limiting and packet dropping can be applied to avoid
DoS attacks on the control plane or stop ongoing attacks
directly on the data plane by installing specific rules on the
devices where the attacks is being originated.
Forensics and remediation encompass mechanisms
such as secure logging, event correlation, and consistent
reporting. If anything wrong happens with the network,
operators should be able to safely figure out the root cause
of the problem and put the network to work on a secure
operation mode as fast as possible. Additionally, tech-
niques to tolerate faults and intrusions, such as state ma-
allocation of memory space, are enough to crash existing chine replication [517], proactive–reactive recovery [518],
controllers. On the security perspective, a simple mali- and diversity [210], can be added to control platforms for
cious action such as changing the value of a data structure increasing the robustness and security properties by auto-
in memory can also directly affect the operation and relia- matically masking and removing faults. Put differently,
bility of current controllers. These examples are illustra- SDN controllers should be able to resist against different
tive that, from a security and dependability perspective, types of events (e.g., power outages, network disruption,
there is still a long way to go. communication failures, network partitioning) and attacks
(e.g., DDoS, resource exhaustion) [213], [359]. One of the
2) Countermeasures for OpenFlow-Based SDNs: Several most traditional ways of achieving high availability is
countermeasures can be put in place to mitigate the secu- through replication. Yet, proactive–reactive recovery and
rity threats in SDNs. Table 15 summarizes a number of diversity are two examples of crucial techniques that add
countermeasures that can be applied to different elements value to the system for resisting against different kinds of
of an SDN/OpenFlow-enabled network. Some of these attacks and failures (e.g., those exploring common vul-
measures, namely rate limiting, event filtering, packet nerabilities or caused by software aging problems).
dropping, shorter timeouts, and flow aggregation, are al- Other countermeasures to address different threats and
ready recommended in the most recent versions of the issues of SDN include enhancing the security and depen-
OpenFlow specification (version 1.3.1 and later). How- dability of controllers, protection, and isolation of applica-
ever, most of them are not yet supported or implemented tions [194], [201], [359], [506], trust management
in SDN deployments. between controllers and forwarding devices [359], integ-
Traditional techniques such as access control, attack rity checks of controllers and applications [359], forensics
detection mechanisms, event filtering (e.g., controller de- and remediation [359], [506], verification frameworks
cides which asynchronous messages he is not going to [201], [519], [520], and resilient control planes [359],
accept), firewalls, and intrusion detection systems can be [506], [521], [520]. Protection and isolation mechanisms
used to mitigate the impact of or to avoid attacks. They can should be part of any controller. Applications should be
be implemented in different devices, such as controllers, isolated from each other and from the controller.
forwarding devices, middleboxes, and so forth. Middle- Different techniques such as security domains (e.g.,
boxes can be a good option for enforcing security policies kernel, security, and user level) and data access protection
in an enterprise because they are (in general) more robust mechanisms should be put in place in order to avoid
and special purpose (high-performance) devices. Such a security threats from network applications.
strategy also reduces the potential overhead cause by im- Implementing trust between controllers and forward-
plementing these countermeasures directly on controllers ing is another requirement for ensuring that malicious
or forwarding devices. However, middleboxes can add elements cannot harm the network without being
extra complexity to the network management, i.e., in- detected. An attacker can try to spoof the IP address of
crease the OPEX at the cost of better performance. the controller and make switches connect to its own

controller. This is currently the case since most controllers effect on the sales division of vendors that are required to
and switches only establish insecure TCP connections. adapt accordingly.
Complementary, integrity checks on controller and appli- Pioneering SDN operational deployments have been
cation software can help to ensure that safe code is being mainly greenfield scenarios and/or tightly controlled sin-
bootstrapped, which eliminates harmful software from be- gle administrative domains. Initial rollout strategies are
ing started once the system restarts. Besides integrity mainly based on virtual switch overlay models or Open-
checks, other things such as highly specialized malware Flow-only network-wide controls. However, a broader
detection systems should be developed for SDN. Third- adoption of SDN beyond data center silosVand between
party network applications should always be scanned for themselvesVrequires considering the interaction and
bad code and vulnerabilities because a malicious applica- integration with legacy control planes providing traditional
tion represents a significant security threat to the network. switching; routing; and operation, administration, and
It is worth mentioning that there are also other ap- management (OAM) functions. Certainly, rip-and-replace
proaches for mitigating security threats in SDN, such as is not a viable strategy for the broad adoption of new net-
declarative languages to eliminate network protocol vul- working technologies.
nerabilities [265]. This kind of descriptive languages can Hybrid networking in SDN should allow deploying
specify semantic constraints, structural constraints, and OpenFlow for a subset of all flows only, enable OpenFlow
safe access properties of OpenFlow messages. Then, a on a subset of devices and/or ports only, and provide op-
compiler can use these inputs to find programmers’ imple- tions to interact with existing OAM protocols, legacy de-
mentation mistakes on message operations. In other words, vices, and neighboring domains. As in any technology
such languages can help find and eliminate implementation transition period where forklift upgrades may not be a
vulnerabilities of southbound specifications. choice for many, migration paths are critical for adoption.
Proposals providing basic security properties such as Hybrid networking in SDN spans several levels. The
authentication [522] and access control [523] are starting Migration Working Group of the ONF is tackling the sce-
to appear. C–BAS [522] is a certificate-based authentica- nario where hybrid switch architectures and hybrid
tion, authorization, and accounting (AAA) architecture for (OpenFlow and non-OpenFlow) devices coexist. Hybrid
improving the security control on SDN experimental faci- switches can be configured to behave as a legacy switch or
lities. Solutions in the spirit of C–BAS can be made highly as an OpenFlow switch and, in some cases, as both simul-
secure and dependable through hybrid system architec- taneously. This can be achieved, for example, by partition-
tures, which combine different technologies and tech- ing the set of ports of a switch, where one subset is devoted
niques from distributed systems, security, and fault and to OpenFlow-controlled networks, and the other subset to
intrusion tolerance [524]–[526]. legacy networks. For these subsets to be active at the same
time, each one having its own data plane, multitable sup-
G. Migration and Hybrid Deployments port at the forwarding engine (e.g., via TCAM partition-
The promises by SDN to deliver easier design, opera- ing) is required. Besides port-based partitioning, it is also
tion, and management of computer networks are endan- possible to rely on VLAN-based (prior to entering the
gered by challenges regarding incremental deployability, OpenFlow pipeline) or flow-based partitioning using
robustness, and scalability. A prime SDN adoption chal- OpenFlow matching and the LOCAL and/or NORMAL ac-
lenge relates to organizational barriers that may arise due tions to redirect packets to the legacy pipeline or the
to the first (and second) order effects of SDN automation switch’s local networking stack and its management stack.
capabilities and ‘‘layer/domain blurring.’’ Some level of Flow-based partitioning is the most flexible option, as it
human resistance is to be expected and may affect the allows each packet entering a switch to be classified by an
decision and deployment processes of SDN, especially by OpenFlow flow description and treated by the appropriate
those that may regard the control refactorization of SDN as data plane (OpenFlow or legacy).
a risk to the current chain of control and command, or There are diverse controllers, such as OpenDaylight
even to their job security. This complex social challenge is [13], HP VAN SDN [184], and OpenContrail [183], that
similar (and potentially larger) to known issues between have been designed to integrate current non-SDN technol-
the transport and IP network divisions of service providers, ogies (e.g., SNMP, PCEP, BGP, and NETCONF) with SDN
or the system administrator, storage, networking, and sec- interfaces such as OpenFlow and OVSDB. Nonetheless,
urity teams of enterprise organizations. Such a challenge is controllers such as ClosedFlow [219] have been recently
observable on today’s virtualized data centers, through the proposed with the aim of introducing SDN-like program-
shift in role and decision power between the networking ming capabilities in traditional network infrastructures,
and server people. Similarly, the development and opera- making the integration of legacy and SDN-enabled
tions (DevOps) movement has caused a shift in the locus of networks a reality without side effects in terms of prog-
influence, not only on the network architecture but also on rammability and global network control. ClosedFlow is
purchasing, and this is an effect that SDN may exacerbate. designed to control legacy Ethernet devices (e.g., Cisco
These changes in role and power causes a second-order 3550 switches with a minimum IOS of 12.2 SE) in a similar

way as OpenFlow controller allows administrators to con- common centralized configuration interface to build vir-
trol OpenFlow-enabled devices. More importantly, Clo- tual networks using VLANs. An abstraction of the physical
sedFlow does not impose any change on the forwarding network topology is taken into account by a centralized
devices. It only takes advantage of the existing hardware controller that applies a path finder mechanism, in order
and firmware capabilities to mimic an SDN control over to calculate network paths and program the OpenFlow
the network, i.e., allow dynamic and flexible programma- switches via REST interfaces and legacy devices using
bility in the data plane. The next step could be the integ- NETCONF [44].
ration of controllers like ClosedFlow- and OpenFlow-based More recently, frameworks such as ESCAPE [532] and
controllers, promoting interoperability among controllers its extensions have been proposed to provide multilayer
and a smooth transition from legacy infrastructures to service orchestration in multidomains. Such frameworks
SDN-enabled infrastructure with nearly all the capabilities combine different tools and technologies such as Click
of a clean-slate SDN-enabled infrastructure. [533], POX [231], OpenDaylight [13], and NETCONF [44].
Furthermore, controllers may have to be separated into In other words, those frameworks integrate different SDN
distinct peer domains for different reasons, such as scala- solutions with traditional ones. Therefore, they might be
bility, technology, controllers from different vendors, con- useful tools on the process of integrating or migrating le-
trollers with different service functionality, and diversity gacy networking infrastructure to SDN.
of administrative domains [218]. Controllers from differ- Other hybrid solutions starting to emerge include open
ent domains, or with distinct purposes, are also required source hybrid IP/SDN (OSHI) [534]. OSHI combines
to be backward compatible either by retrofitting or ex- Quagga for open shortest path first routing and SDN
tending existing multidomain protocols (e.g., BGP) or by capable switching devices (e.g., Open vSwitch) on Linux to
proposing new SDN-to-SDN protocols (also known as provide backward compatibility for supporting incremen-
east/westbound APIs). tal SDN deployments, i.e., enabling interoperability with
Some efforts have been already devoted to the chal- non-OF forwarding devices in carrier-grade networks.
lenges of migration and hybrid SDNs. RouteFlow [527] While full SDN deployments are straightforward only
implements an IP level control plane on top of an Open- in some green field deployments such as data center net-
Flow network, allowing the underlying devices to act as IP works or by means of an overlay model approach, hybrid
routers under different possible arrangements. The SDN approaches represent a very likely deployment model
Cardigan project [50], [528] has deployed RouteFlow at that can be pursued by different means, including the
a live Internet eXchange now for over a year. LegacyFlow following [535].
[529] extends the OpenFlow-based controlled network to • Topology-based hybrid SDN: Based on a topological
embrace non-OpenFlow nodes. There are also some other separation of the nodes controlled by traditional
early use cases on integrating complex legacy system such and SDN paradigms. The network is partitioned
as DOCSIS [161], Gigabit Ethernet passive optical network, in different zones and each node belongs to only
and DWDM reconfigurable optical add/drop multiplexer one zone.
(ROADM) [157], [158]. The common grounds of these • Service-based hybrid SDN: Conventional networks
pieces of work are: 1) considering hybrid as the coexistence and SDN provide different services, where overlap-
of traditional environments of closed vendor’s routers and ping nodes, controlling a different portion of the
switches with new OpenFlow-enabled devices; 2) targeting FIB (or generalized flow table) of each node. Ex-
the interconnection of both control and data planes of le- amples include network-wide services like for-
gacy and new network elements; and 3) taking a controller- warding that can be based on legacy distributed
centric approach, drawing the hybrid line outside of any control, while SDN provides edge-to-edge services
device itself, but into the controller application space. such as enforcement of traffic engineering and ac-
Panopticon [530] defines an architecture and method- cess policies, or services requiring full traffic visi-
ology to consistently implement SDN inside enterprise bility (e.g., monitoring).
legacy networks through network orchestration under • Class-based hybrid SDN: Based on the partition
strict budget constraints. The proposed architecture in- of traffic in classes, some controlled by SDN and
cludes policy configurations, troubleshooting, and main- the remaining by legacy protocols. While each
tenance tasks establishing transitional networks (SDN and paradigm controls a disjoint set of node for-
legacy) in structures called solitary confinement trees warding entries, each paradigm is responsible for
(SCTs), where VLAN IDs are efficiently used by orches- all network services for the assigned traffic
tration algorithms to build paths in order to steer traffic classes.
through SDN switches. Defying the partial SDN imple- • Integrated hybrid SDN: A model where SDN is
mentation concept, they confirm that this could be a long- responsible for all the network services and uses
term operational strategy solution for enterprise networks. traditional protocols (e.g., BGP) as an interface to
HybNET [531] presents a network management frame- node FIBs. For example, it can control forwarding
work for hybrid OpenFlow-legacy networks. It provides a paths by injecting carefully selected routes into a

routing system or adjusting protocol settings (e.g., Carrier networks are using the SDN paradigm as the
IGP weights). Past efforts on RCPs [85] and the technology means for solving a number of long standing
ongoing efforts within ODL [13] can be considered problems. Some of these efforts include new architectures
examples of this hybrid model. for a smooth migration from the current mobile core in-
In general, benefits of hybrid approaches include ena- frastructure to SDN [222], and techno-economic models
bling flexibility (e.g., easy match on packet fields for mid- for virtualization of these networks [551], [552]; carrier-
dleboxing) and SDN-specific features (e.g., declarative grade OpenFlow virtualization schemes [112], [553], in-
management interface) while partially keeping the including virtualized broadband access infrastructures [554],
herited characteristics of conventional networking such as techniques that are allowing the offer of network-as-a-
robustness, scalability, technology maturity, and low de- service [555]; programmable GEPON and DWDM
ployment costs. On the negative side, the drawbacks of ROADM [157]–[160]; large-scale interautonomous sys-
hybridization include the need for ensuring profitable tems (ASs) SDN-enabled deployments [556]; flexible con-
interactions between the networking paradigms (SDN and trol of network resources [557], including offering MPLS
traditional) while dealing with the heterogeneity that services using an SDN approach [558]; and the investiga-
largely depends on the model. tion of novel network architectures, from proposals to se-
Initial tradeoff analyses [535] suggest that the combi- parate the network edge from the core [559], [560], with
nation of centralized and distributed paradigms may pro- the latter forming the fabric that transports packets as
vide mutual benefits. However, future work is required to defined by an intelligent edge, to software-defined Inter-
devise techniques and interaction mechanisms that maxinet exchange points [528], [561].
mize such benefits while limiting the added complexity of Use case analysis [562] of management functions re-
the paradigm coexistence. quired by carrier networks have identified a set of require-
ments and existing limitations in the SDN protocol toolbox.
H. Meeting Carrier-Grade and Cloud Requirements For instance, it has been pinpointed that OF-Config [54]
A number of carrier-grade infrastructure providers needs a few extensions in order to meet the carrier require-
(e.g., NTT, AT&T, Verizon, Deutsche Telekom) are at the ments, such as physical resource discovery, logical link
core of the SDN community with the ultimate goal of configuration, logical switch instantiation, and device and
solving their long standing networking problems. In the link OAM configuration [562]. Similarly, OpenFlow exten-
telecom world, NTT can be considered one of the forefront sions have also been proposed to realize packet-optical
runners in terms of investing in the adoption and deploy- integration with SDN [563]. In order to support SDN con-
ment of SDN in all kinds of network infrastructures, from cepts in large-scale wide area networks, different exten-
backbone, data center, to edge customers [269]. In 2013, sions and mechanisms are required, both technology
NTT launched an SDN-based, on-demand elastic provi- specific (e.g., MPLS BFD) and technology agnostic, such
sioning platform of network resources (e.g., bandwidth) as: resiliency mechanisms for surviving link failures [486],
for HD video broadcasters [536]. Similarly, as a global failures of controller or forwarding elements; solutions for
cloud provider with data centers spread across the globe integrating residential customer services in different forms
[537], the same company launched a similar service for its (i.e., support also current technologies); new energy-
cloud customers, who are now capable of taking advantage efficient networking approaches; QoS properties for packet
of dynamic networking provisioning intradata and inter- classification, metering, coloring, policing, shaping, and
data centers [538]. AT&T is another telecom company that scheduling; and multilayer aspects outlining different
is investing heavily in new services, such as user-defined stages of packet-optical integration [563]–[565].
network clouds, that take advantage of recent develop- SDN technology also brings new possibilities for cloud
ments in NFV and SDN [539]. As we mentioned before, providers. By taking advantage of the logically centralized
SDN and NFV are complementary technologies that can be control of network resources [8], [566], it is possible to
applicable to different types of networks, from local net- simplify and optimize network management of data cen-
works and data centers to transport networks [540]–[545]. ters and achieve: 1) efficient intradata-center networking,
Recently, several research initiatives have worked toward including fast recovery mechanisms for the data and con-
combining SDN and NFV through Intel’s DPDK, a set of trol planes [478], [567], [568], adaptive traffic engineering
libraries and drivers that facilitates the development of with minimal modifications to DC networks [278], simpli-
network-intensive applications and allows the implemen- fied fault-tolerant routing [569], performance isolation
tation of fine-grained network functions [546]. Early work [570], and easy and efficient resource migration (e.g., of
toward service chaining has been proposed by combining VMs and virtual networks) [478]; 2) improved interdata-
SDN and NFV technologies [27], [547]–[550], and studies center communication, including the ability to fully utilize
around the ForCES’s [30] applicability to SDN-enhanced the expensive high-bandwidth links without impairing
NFV have also come to light [540]. These are some of the quality of service [8], [571]; 3) higher levels of reliability
early examples of the opportunities SDNs seem to bring to (with novel fault management mechanisms, etc.) [478],
telecom and cloud providers. [486], [567], [569]; and 4) cost reduction by replacing

Table 16 Carrier-Grade and Cloud Provider Expectations and Challenges
complex, expensive hardware by simple and cheaper for- workload changes. Recent advances makes on-demand
warding devices [8], [572]. provisioning of resources possible, at nearly all infrastruc-
Table 16 summarizes some of the carrier-grade net- tural layers. The fully automated provisioning and orches-
work and cloud infrastructure providers’ requirements. In tration of IT infrastructures as been recently named
this table, we show the current challenges and what is to software-defined environments (SDEs) [171], [172], by
be expected with SDN. As we saw before, some of the IBM. This is a novel approach that is expected to have
expectations are already becoming a reality, but many are significant potential in simplifying IT management, opti-
still open issues. What seems to be clear is that SDN re- mizing the use of the infrastructure, reduce costs, and
presents an opportunity for telecom and cloud providers, reduce the time to market of new ideas and products. In
in providing flexibility, cost effectiveness, and easier man- an SDE, workloads can be easily and automatically assigned
agement of their networks. to the appropriate IT resources based on application
characteristics, security and service level policies, and the
I. SDN: The Missing Piece Toward best-available resources to deliver continuous, dynamic
Software-Defined Environments optimization and reconfiguration to address infrastructure
The convergence of different technologies is enabling issues in a rapid and responsive manner. Table 17 sum-
the emergence of fully programmable IT infrastructures. It marizes the traditional approaches and some of the key
is already possible to dynamically and automatically con- features being enabled by SDEs [577], [578].
figure or reconfigure the entire IT stack, from the network In an SDE, the workloads are managed independently
infrastructure up to the applications, to better respond to of the systems and underlying infrastructure, i.e., are not

Table 17 SDE Pushing IT to the Next Frontier
tied to a specific technology or vendor [171], [172]. Anoth- Fig. 11 gives a simplified overview of an SDE, by taking the
er characteristic of this new approach is to offer a prog- approach developed by IBM as its basis. The main idea of
rammatic access to the environment as a whole, selecting an SDE-based infrastructure is that the business needs that
the best available resources based on the current status of define the workloads trigger the reconfiguration of the
the infrastructure, and enforcing the policies defined. In global IT infrastructure (compute, storage, network). This
this sense, it shares much of the philosophy of SDN. is an important step toward a more customizable IT in-
Interestingly, one of the missing key pieces of an SDE was, frastructure that focuses on the business requirements
until now, SDN. rather than on the limitations of the infrastructure itself.
The four essential building blocks of an SDE [171],
[172], [578] are:
• SDNs [579], [580]; VI. CONCLUSION
• software-defined storage (SDS) [577]; Traditional networks are complex and hard to manage.
• software-defined compute (SDC) [171]; One of the reasons is that the control and data planes are
• software-defined management (SDM) [581]. vertically integrated and vendor specific. Another, con-
In the last decade, the advances in virtualization of curring reason, is that typical networking devices are also
compute and storage, together with the availability of so- tightly tied to line products and versions. In other words,
phisticated cloud orchestration tools have enabled SDS, each line of product may have its own particular config-
SDC, and SDM. These architectural components have uration and management interfaces, implying long cycles
been widely used by cloud providers and for building IT for producing product updates (e.g., new firmware) or
infrastructures in different enterprise environments. upgrades (e.g., new versions of the devices). All this has
However, the lack of programmable network control has given rise to vendor lock-in problems for network infra-
so far hindered the realization of a complete SDE. SDN is structure owners, as well as posing severe restrictions to
seen as the technology that may fill this gap, as attested by change and innovation.
the emergence of cloud-scale network virtualization plat- SDN created an opportunity for solving these long-
forms based on this new paradigm [112]. standing problems. Some of the key ideas of SDN are the
The IBM SmartCloud Orchestrator is one of the first introduction of dynamic programmability in forwarding
examples of an SDE [171], [172]. It integrates compute, devices through open southbound interfaces, the decoupl-
storage, management, and networking in a structured way. ing of the control and data plane, and the global view of the
Fig. 11. Overview of an IT infrastructure based on an SDE.

network by logical centralization of the ‘‘network brain.’’ SDN has successfully managed to pave the way toward
While data plane elements became dumb, but highly effi- a next-generation networking, spawning an innovative re-
cient and programmable packet forwarding devices, the search and development environment, promoting advances
control plane elements are now represented by a single in several areas: switch and controller platform design,
entity, the controller or NOS. Applications implementing evolution of scalability and performance of devices and
the network logic run on top of the controller and are architectures, promotion of security and dependability.
much easier to develop and deploy when compared to We will continue to witness extensive activity around
traditional networks. Given the global view, consistency of SDN in the near future. Emerging topics requiring further
policies is straightforward to enforce. SDN represents a research are, for example: the migration path to SDN, ex-
major paradigm shift in the development and evolution of tending SDN toward carrier transport networks, realization
networks, introducing a new pace of innovation in net- of the network-as-a-service cloud computing paradigm, or
working infrastructure. SDEs. As such, we would like to receive feedback from the
In spite of recent and interesting attempts to survey networking/SDN community as this novel paradigm evolves,
this new chapter in the history of networks [14]–[16], the to make this a ‘‘living document’’ that gets updated and
literature was still lacking, to the best of our knowledge, improved based on the community feedback. We have set up
a single extensive and comprehensive overview of the a github repository (https://github.com/SDN-Survey/latex/
building blocks, concepts, and challenges of SDNs. Try- wiki) for this purpose, and we invite our readers to join us in
ing to address this gap, this paper used a layered ap- this communal effort. Additionally, new releases of the
proach to methodically dissect the state of the art in survey will be available at http://arxiv.org/abs/1406.0440. h
terms of concepts, ideas, and components of SDN, cover-
ing a broad range of existing solutions, as well as future
directions. Acknowledgment
We started by comparing this new paradigm with tradi- The authors would like to thank the anonymous re-
tional networks and discussing how academy and industry viewers and a number of fellows that have contributed to
helped shape SDN. Following a bottom-up approach, we this work: J. Rexford for her feedback on an early version
provided an in-depth overview of what we consider the of this work and encouragement to get it finished;
eight fundamental facets of the SDN problem: 1) hardware S. Seetharaman for reviewing the draft and providing
infrastructure; 2) southbound interfaces; 3) network vir- inputs to alternative SDN views; D. Meyer for his thoughts
tualization (hypervisor layer between the forwarding de- on organizational challenges; T. Nadeau for his inputs on
vices and the NOSs); 4) NOSs (SDN controllers and control OpenDaylight; and L. M. Contreras Murillo for his con-
platforms); 5) northbound interfaces (common prog- tributions to SDN standardization. In addition, the
ramming abstractions offered to network applications); authors would also like to acknowledge the several contri-
6) virtualization using slicing techniques provided by spe- butions from the community, namely, from A. A. Lazar,
cial purpose libraries and/or programming languages and C. Cascone, G. Patra, H. Evangelos, J. Ancieta, J. Stringer,
compilers; 7) network programming languages; and finally, K. Pentikousis, L. de Paula, M. Canini, P. Wette, R. Fontes,
8) network applications. R. Rosa, R. Costa, R. de Freitas Gesuatto, and W. John.
REFERENCES defined networking,’’ IEEE Commun. Mag., [14] A. Lara, A. Kolasani, and B. Ramamurthy,
vol. 51, no. 2, pp. 114–119, Feb. 2013. ‘‘Network innovation using OpenFlow:
[1] T. Benson, A. Akella, and D. Maltz, A survey,’’ IEEE Commun. Surv. Tut., vol. 16,
‘‘Unraveling the complexity of network [7] T. Koponen et al., ‘‘Onix: A distributed
control platform for large-scale production no. 1, pp. 493–512, First Quart. 2014.
management,’’ in Proc. 6th USENIX Symp.
Networked Syst. Design Implement., 2009, networks,’’ in Proc. 9th USENIX Conf. Oper. [15] B. Nunes, M. Mendonca, X.-N. Nguyen,
pp. 335–348. Syst. Design Implement., 2010, pp. 1–6. K. Obraczka, and T. Turletti, ‘‘A survey
[8] S. Jain et al., ‘‘B4: Experience with a of software-defined networking: Past,
[2] B. Raghavan et al., ‘‘Software-defined present, future of programmable networks,’’
internet architecture: Decoupling globally-deployed software defined wan,’’ in
Proc. ACM SIGCOMM Conf., 2013, pp. 3–14. IEEE Commun. Surv. Tut., vol. 16, no. 3,
architecture from infrastructure,’’ in pp. 1617–1634, Third Quart. 2014.
Proc. 11th ACM Workshop Hot Topics [9] N. McKeown et al., ‘‘OpenFlow: Enabling
Netw., 2012, pp. 43–48. innovation in campus networks,’’ SIGCOMM [16] Y. Jarraya, T. Madi, and M. Debbabi,
Comput. Commun. Rev., vol. 38, no. 2, ‘‘A survey and a layered taxonomy of
[3] A. Ghodsi et al., ‘‘Intelligent design software-defined networking,’’ IEEE
enables architectural evolution,’’ in pp. 69–74, Mar. 2008.
Commun. Surv. Tut., vol. 16, no. 4,
Proc. 10th ACM Workshop Hot Topics [10] Open Networking Foundation (ONF),
pp. 1955–1980, Fourth Quart. 2014.
Netw., 2011, pp. 3:1–3:6. 2014. [Online]. Available: https://www.
opennetworking.org/ [17] N. Feamster, J. Rexford, and E. Zegura,
[4] N. Mckeown, ‘‘How SDN will shape ‘‘The road to SDN,’’ Queue, vol. 11, no. 12,
networking,’’ Oct. 2011. [Online]. Available: [11] S. Yeganeh, A. Tootoonchian, and Y. Ganjali,
pp. 20:20–20:40, Dec. 2013.
http://www.youtube.com/watch?v= ‘‘On scalability of software-defined
c9-K5O_qYgA. networking,’’ IEEE Commun. Mag., [18] R. Presuhn, ‘‘Version 2 of the protocol
vol. 51, no. 2, pp. 136–141, Feb. 2013. operations for the simple network
[5] S. Schenker, ‘‘The future of networking, management protocol (SNMP),’’ Internet
the past of protocols,’’ Oct. 2011. [Online]. [12] VMware, Inc., NSX Virtualization Platform,
Engineering Task Force, RFC 3416 (Internet
Available: http://www.youtube.com/ 2013. [Online]. Available: https://www.
Standard), Dec. 2002. [Online]. Available:
watch?v=YHeyuD89n1Y. vmware.com/products/nsx/.
http://www.ietf.org/rfc/rfc3416.txt.
[6] H. Kim and N. Feamster, ‘‘Improving [13] OpenDaylight, A Linux Foundation
[19] J. Pan, S. Paul, and R. Jain, ‘‘A survey of the
network management with software Collaborative Project, 2013. [Online].
research on future internet architectures,’’
Available: http://www.opendaylight.org.

IEEE Commun. Mag., vol. 49, no. 7, pp. 26–36, plane for network virtualization overlays,’’ [53] Open Networking Foundation (ONF),
Jul. 2011. Internet Engineering Task Force, Oct. 2013. ‘‘OpenFlow switch specification,’’ Tech.
[20] N. Feamster and H. Balakrishnan, [Online]. Available: http://tools.ietf.org/ Rep., Oct. 2013. [Online]. Available: https://
‘‘Detecting BGP configuration faults html/draft-maino-nvo3-lisp-cp-03. www.opennetworking.org/images/stories/
with static analysis,’’ in Proc. 2nd Conf. [38] Y. Hertoghs et al., ‘‘A unified LISP mapping downloads/sdn-resources/onf-specifications/
Symp. Netw. Syst. Design Implement., 2005, database for L2 and L3 network virtualization openflow/openflow-spec-v1.4.0.pdf.
vol. 2, pp. 43–56. overlays,’’ Internet Engineering Task Force, [54] Open Networking Foundation (ONF),
[21] R. Barrett, S. Haar, and R. Whitestone, Feb. 2014. [Online]. Available: http://tools. ‘‘OpenFlow management and configuration
‘‘Routing snafu causes internet outage,’’ ietf.org/html/draft-hertoghs-nvo3-lisp- protocol (OF-CONFIG) v1.2,’’ Tech. Rep.,
Interactive Week, vol. 25, 1997. controlplane-unified-01. 2014. [Online]. Available: https://www.
[39] J. Gross, T. Sridhar, P. Garg, C. Wright, and opennetworking.org/images/stories/
[22] K. Butler, T. Farley, P. McDaniel, and
I. Ganga, ‘‘Geneve: Generic network downloads/sdn-resources/onf-specifications/
J. Rexford, ‘‘A survey of BGP security
virtualization encapsulation,’’ Internet openflow-config/of-config-1.2.pdf.
issues and solutions,’’ Proc. IEEE, vol. 98,
no. 1, pp. 100–122, Jan. 2010. Engineering Task Force, Internet Draft, [55] Open Networking Foundation (ONF),
Feb. 2014. [Online]. Available: http://tools. ‘‘OpenFlow notifications framework
[23] J. Sherry and S. Ratnasamy, ‘‘A survey of
ietf.org/id/draft-gross-geneve-00.txt. OpenFlow management,’’ Oct. 2013.
enterprise middlebox deployments,’’ Electr.
[40] R. Jain and S. Paul, ‘‘Network virtualization [Online]. Available: https://www.
Eng. Comput. Sci. Dept., Univ. California,
and software defined networking for cloud opennetworking.org/images/stories/
Berkeley, Berkeley, CA, USA, Tech. Rep.
computing: A survey,’’ IEEE Commun. Mag., downloads/sdn-resources/onf-specifications/
UCB/EECS-2012-24, Feb. 2012.
vol. 51, no. 11, pp. 24–31, Nov. 2013. openflow-config/of-notifications-framework-
[24] K. Greene, ‘‘10 Breakthrough Technologies: 1.0.pdf.
Software-defined Networking MIT Technol. [41] E. Haleplidis et al., ‘‘SDN layers and
architecture terminology,’’ Internet [56] Open Networking Foundation (ONF),
Rev., 2009. [Online]. Available: http://
Engineering Task Force, Internet Draft, ‘‘OpenFlow table type patterns,’’ Tech.
www2.technologyreview.com/article/
Sep. 2014. [Online]. Available: http:// Rep., Aug. 2014. [Online]. Available: https://
412194/tr10-software-defined-networking/.
www.ietf.org/id/draft-irtf-sdnrg-layer- www.opennetworking.org/images/stories/
[25] P. Newman, G. Minshall, and T. L. Lyon, downloads/sdn-resources/onf-specifications/
terminology-02.txt.
‘‘IP switchingVATM under IP,’’ IEEE/ACM openflow/OpenFlow%20Table%20Type%
Trans. Netw., vol. 6, no. 2, pp. 117–129, [42] Y. Rekhter, T. Li, and S. Hares, ‘‘A border
20Patterns%20v1.0.pdf.
Apr. 1998. gateway protocol 4 (BGP-4),’’ Internet
Engineering Task Force, RFC 4271 (Draft [57] Open Networking Foundation (ONF),
[26] N. Gude et al., ‘‘NOX: Towards an operating ‘‘Optical transport use cases,’’ Tech. Rep.,
Standard), Jan. 2006. [Online]. Available:
system for networks,’’ Comput. Commun. 2014. [Online]. Available: https://www.
http://www.ietf.org/rfc/rfc4271.txt.
Rev., vol. 38, no. 3, pp. 105–110, 2008. opennetworking.org/images/stories/
[43] J. Vasseur and J. L. Roux, ‘‘Path computation
[27] H. Jamjoom, D. Williams, and U. Sharma, downloads/sdn-resources/technical-reports/
element (PCE) communication protocol
‘‘Don’t call them middle-boxes, call them optical-transport-use-cases.pdf.
(PCEP),’’ Internet Engineering Task Force,
middlepipes,’’ in Proc. 3rd Workshop Hot [58] Open Networking Foundation (ONF),
RFC 5440 (Proposed Standard), Mar. 2009.
Topics Softw. Defined Netw., 2014, pp. 19–24. ‘‘Requirements analysis for transport
[Online]. Available: http://www.ietf.org/rfc/
[28] H. Alkhatib et al., ‘‘IEEE CS 2022 Report rfc5440.txt. OpenFlow/SDN,’’ Tech. Rep., Aug. 2014.
(Draft),’’ IEEE Computer Society, Tech. [Online]. Available: https://www.
[44] R. Enns, M. Bjorklund, J. Schoenwaelder,
Rep., Feb. 2014. opennetworking.org/images/stories/
and A. Bierman ‘‘Network configuration
[29] M. Casado, N. Foster, and A. Guha, downloads/sdn-resources/technical-reports/
protocol (NETCONF),’’ Internet
‘‘Abstractions for software-defined onf2014.227\_Optical\_Transport\_
Engineering Task Force, RFC 6241
networks,’’ ACM Commun., vol. 57, no. 10, Requirements.pdf.
(Proposed Standard), Jun. 2011. [Online].
pp. 86–95, Sep. 2014. Available: http://www.ietf.org/rfc/ [59] Open Networking Foundation (ONF),
[30] A. Doria et al., ‘‘Forwarding and control rfc6241.txt. ‘‘Migration use cases and methods,‘‘
element separation (ForCES) protocol Migration Working Group, Tech. Rep.,
[45] A. Corradi, M. Fanelli, and L. Foschini,
specification,’’ Internet Engineering Task 2014. [Online]. Available: https://www.
‘‘VM consolidation: A real case based
Force, Mar. 2010. [Online]. Available: opennetworking.org/images/stories/
on openstack cloud,’’ Future Generat.
http://www.ietf.org/rfc/rfc5810.txt. downloads/sdn-resources/use-cases/
Comput. Syst., vol. 32, pp. 118–127, 2014.
Migration-WG-Use-Cases.pdf.
[31] H. Song, ‘‘Protocol-oblivious forwarding: [46] A. Shang, J. Liao, and L. Du, ‘‘Pica8
Unleash the power of SDN through a [60] Open Networking Foundation (ONF),
Xorplus, 2014. [Online]. Available:
future-proof forwarding plane,’’ in Proc. ‘‘Software-defined networking: The
http://sourceforge.net/projects/xorplus/.
2nd ACM SIGCOMM Workshop Hot Topics new norm for networks,’’ Tech. Rep.,
[47] P. Jakma and D. Lamparter, ‘‘Introduction Apr. 2012. [Online]. Available: https://
Softw. Defined Netw., 2013, pp. 127–132.
to the quagga routing suite,’’ IEEE Network, www.opennetworking.org/images/stories/
[32] T. D. Nadeau and K. Gray, SDN: Software vol. 28, no. 2, pp. 42–48, Mar. 2014. downloads/sdn-resources/white-papers/
Defined Networks, 1st ed. Sebastopol,
[48] NetFPGA, 2014. [Online]. Available: wp-sdn-newnorm.pdf.
CA, USA: O’Reilly Media, 2013.
http://netfpga.org/ [61] H. Xie, T. Tsou, D. Lopez, and H. Yin,
[33] N. M. K. Chowdhury and R. Boutaba,
[49] Linux Foundation, ‘‘Open platform for ‘‘Use cases for ALTO with software defined
‘‘A survey of network virtualization,’’
NFV,’’ Sep. 2014. [Online]. Available: networks,’’ Internet Engineering Task
Comput. Netw., vol. 54, no. 5, pp. 862–876,
https://www.opnfv.org. Force, Internet Draft, Jun. 2012. [Online].
2010.
[50] C. E. Rothenberg et al., ‘‘When open source Available: http://tools.ietf.org/html/
[34] B. Davie and J. Gross, ‘‘A stateless transport draft-xie-alto-sdn-use-cases-01.
meets network control planes,’’ IEEE
tunneling protocol for network virtualization
Computer, Special Issue on Software-Defined [62] A. Atlas, J. Halpern, S. Hares, D. Ward, and
(STT),’’ Internet Engineering Task Force,
Networking, vol. 47, no. 11, pp. 46–54, T. Nadeau, ‘‘An architecture for the
Apr. 2014. [Online]. Available: http://tools.
Nov. 2014. interface to the routing system,’’ Internet
ietf.org/html/draft-davie-stt-06.
[51] Open Networking Foundation (ONF), Engineering Task Force, Internet Draft,
[35] M. Mahalingam et al., ‘‘VXLAN: A Jul. 2014. [Online]. Available: https://tools.
‘‘SDN architecture,’’ Tech. Rep., Jun. 2014.
framework for overlaying virtualized layer 2 ietf.org/html/draft-ietf-i2rs-architecture-05.
[Online]. Available: https://www.
networks over layer 3 networks,’’ Internet
opennetworking.org/images/stories/ [63] R. Enns, ‘‘NETCONF configuration
Engineering Task Force, Internet Draft,
downloads/sdn-resources/technical-reports/ protocol,’’ Internet Engineering Task
Nov. 2013. [Online]. Available: http://www.
TR_SDN_ARCH_1.0_06062014.pdf. Force, Internet Draft, Dec. 2004. [Online].
ietf.org/id/draft-mahalingam-dutt-dcops-
[52] Open Networking Foundation (ONF), Available: http://tools.ietf.org/html/rfc4741.
vxlan-06.txt.
‘‘Conformance test specification for [64] L. Kreeger, D. Dutt, T. Narten, and D. Black,
[36] M. Sridharan et al., ‘‘NVGRE: Network
OpenFlow switch specification,’’ Tech. Rep., ‘‘Network virtualization NVE to NVA
virtualization using generic routing
Jun. 2013. [Online]. Available: https:// control protocol requirements,’’ Internet
encapsulation,’’ Internet Engineering Task
www.opennetworking.org/images/stories/ Engineering Task Force, Internet Draft,
Force, Internet Draft, Aug. 2013. [Online].
downloads/sdn-resources/onf-specifications/ Apr. 2014. [Online]. Available: http://
Available: http://tools.ietf.org/id/draft-
openflow-test/conformance-test-spec- tools.ietf.org/html/draft-ietf-nvo3-nve-
sridharan-virtualization-nvgre-03.txt.
openflow-1.0.1.pdf. nva-cp-req-02.
[37] F. Maino, V. Ermagan, Y. Hertoghs,
D. Farinacci, and M. Smith ‘‘LISP control

[65] D. King and A. Farrel, ‘‘A PCE-based SDN architectures,’’ Tech. Rep., Sep. 2013. [94] P. Newman et al., ‘‘Ipsilon’s general
architecture for application-based network [Online]. Available: http://www.oiforum. switch management protocol specification
operations,’’ Internet Engineering Task com/public/documents/OIF\_Carrier\_ version 1.1,’’ Internet Engineering Task
Force, Internet Draft, Aug. 2014. [Online]. WG\_Requirements\_on\_Transport\_ Force, RFC 1987 (Informational), Aug. 1996,
Available: https://tools.ietf.org/html/draft- Networks\_in\_SDN\_Architectures\_ updated by RFC 2297. [Online]. Available:
farrkingel-pce-abno-architecture-11. Sept2013.pdf. http://www.ietf.org/rfc/rfc1987.txt.
[66] D. Dhody, L. C. Y. Lee, O. Gonzalez, and [78] Open Data Center Alliance, [95] A. Doria and K. Sundell, ‘‘General
N. Ciulli, ‘‘Cross stratum optimization ‘‘Software-defined networking Rev. 2.0,’’ switch management protocol (GSMP)
enabled path computation,’’ Internet Tech. Rep., 2014. [Online]. Available: applicability,’’ Internet Engineering Task
Engineering Task Force, Internet Draft, http://www.opendatacenteralliance.org/ Force, RFC 3294 (Informational), Jun. 2002.
Jul. 2014. [Online]. Available: http://tools. docs/software\_defined\_networking\_ [Online]. Available: http://www.ietf.org/
ietf.org/html/draft-dhody-pce-cso-enabled- master\_usage\_model\_rev2.pdf. rfc/rfc3294.txt.
path-computation-06. [79] European Telecommunications Standards [96] J. Van der Merwe, S. Rooney, I. Leslie, and
[67] B. K. F. Hu and H. Cankaya, ‘‘SPRING Institute (ETSI), ‘‘Network functions S. Crosby, ‘‘The tempest-a practical
OpenFlow interworking requirements,’’ virtualization (NFV); architectural framework for network programmability,’’
Internet Engineering Task Force, Internet framework v1.1.1ETSI GS NFV 002,’’ Tech. IEEE Network, vol. 12, no. 3, pp. 20–28,
Draft, Sep. 2014. [Online]. Available: Rep., Oct. 2013. [Online]. Available: http:// May 1998.
https://tools.ietf.org/html/draft-khc-spring- www.etsi.org/deliver/etsi_gs/NFV/001_099/ [97] T. Lakshman, T. Nandagopal, R. Ramjee,
openflow-interworking-req-00. 002/01.01.01_60/gs_NFV002v010101p.pdf. K. Sabnani, and T. Woo, ‘‘The SoftRouter
[68] E. P. S. Kim, J. Park, and L. Contreras, [80] Automatic Terminal Information Service architecture,’’ in Proc. 3rd ACM Workshop
‘‘SPRING use cases for software-defined (ATIS), ‘‘Operational opportunities and Hot Topics Netw., San Diego, CA, USA,
networking,’’ Internet Engineering Task challenges of SDN/NFV programmable Nov. 2004, vol. 2004, pp. 1–6.
Force, Internet Draft, Jul. 2014. [Online]. infrastructure,’’ Tech. Rep. ATIS-I-0000044, [98] A. Greenberg et al., ‘‘A clean slate 4D
Available: http://tools.ietf.org/html/ Oct. 2014. [Online]. Available: https://www. approach to network control and
draft-kim-spring-use-cases-00. atis.org/docstore/product.aspx?id=28143. management,’’ SIGCOMM Comput.
[69] D. Ceccarelli, L. Fang, Y. Lee, and D. Lopez, [81] D. Tennenhouse, J. Smith, W. Sincoskie, Commun. Rev., vol. 35, no. 5, pp. 41–54,
‘‘Framework for abstraction and control of D. Wetherall, and G. Minden, ‘‘A survey Oct. 2005.
transport networks,’’ Internet Engineering of active network research,’’ IEEE Commun. [99] J. Van der Merwe et al., ‘‘Dynamic
Task Force, Internet Draft, Feb. 2014. Mag., vol. 35, no. 1, pp. 80–86, Jan. 1997. connectivity management with an
[Online]. Available: https://xml.resource. [82] A. Lazar, K.-S. Lim, and F. Marconcini, intelligent route service control point,’’ in
org/html/draft-ceccarelli-actn-framework-01. ‘‘Realizing a foundation for programmability Proc. SIGCOMM Workshop Internet Netw.
[70] M. Boucadair and C. Jacquenet, of ATM networks with the binding Manage., 2006, pp. 29–34.
‘‘Software-defined networking: A architecture,’’ IEEE J. Sel. Areas Commun., [100] M. Casado et al., ‘‘SANE: A protection
perspective from within a service provider vol. 14, no. 7, pp. 1214–1227, Sep. 1996. architecture for enterprise networks,’’ in
environment,’’ Internet Engineering Task [83] A. Lazar, ‘‘Programming telecommunication Proc. 15th Conf. USENIX Security Symp.,
Force, Internet Draft, Mar. 2014. [Online]. networks,’’ IEEE Network, vol. 11, no. 5, 2006, vol. 15, Article 10.
Available: https://tools.ietf.org/html/ pp. 8–18, Sep. 1997. [101] M. Casado et al., ‘‘Ethane: Taking control
rfc7149.
[84] D. Sheinbein and R. P. Weber, ‘‘800 service of the enterprise,’’ in Proc. Conf. Appl.
[71] E. Haleplidis et al., ‘‘SDN layers and using SPC network capability,’’ Bell Syst. Technol. Architect. Protocols Comput.
architecture terminology,’’ Internet Tech. J., vol. 61, no. 7, pp. 1737–1744, Commun., 2007, DOI: 10.1145/1282380.
Engineering Task Force, Internet Draft, Sep. 1982. 1282382.
Aug. 2014. [Online]. Available: http://
[85] M. Caesar et al., ‘‘Design and [102] M. Macedonia and D. Brutzman, ‘‘Mbone
tools.ietf.org/html/draft-haleplidis-sdnrg-
implementation of a routing control provides audio and video across the internet,’’
layer-terminology-07.
platform,’’ in Proc. 2nd Conf. Symp. Computer, vol. 27, no. 4, pp. 30–36, 1994.
[72] C. B. L. Contreras and D. Lopez, Netw. Syst. Design Implement., [103] R. Fink and R. Hinden, ‘‘6bone (IPv6 Testing
‘‘Cooperating layered architecture for SDN,’’ 2005, vol. 2, pp. 15–28. Address Allocation) Phaseout,’’ Internet
Internet Engineering Task Force, Internet
[86] J. Biswas et al., ‘‘The IEEE P1520 Engineering Task Force, RFC 3701
Draft, Aug. 2014. [Online]. Available:
standards initiative for programmable (Informational), Mar. 2004. [Online].
http://tools.ietf.org/html/draft-contreras-
network interfaces,’’ Commun. Mag., Available: http://www.ietf.org/rfc/
sdnrg-layered-sdn-00.
vol. 36, no. 10, pp. 64–70, Oct. 1998. rfc3701.txt.
[73] Y. Cheng and C. Zhou, ‘‘Framework of
[87] B. Schwartz et al., ‘‘Smart packets for [104] D. Andersen, H. Balakrishnan, F. Kaashoek,
signalling for SDNVWorking document,’’
active networks,’’ in Proc. IEEE 2nd and R. Morris, ‘‘Resilient overlay networks,’’
International Telecommunication Union
Conf. Open Architect. Netw. Programm., SIGOPS Oper. Syst. Rev., vol. 35, no. 5,
Telecommunication Standardization
Mar. 1999, pp. 90–97. pp. 131–145, Oct. 2001.
Sector (ITU-T), Tech. Rep., work item:
Q.Suppl.-SDN. [Online]. Available: [88] D. Wetherall, J. V. Guttag, and [105] B. Chun et al., ‘‘Planetlab: An overlay
http://www.itu.int D. Tennenhouse, ‘‘Ants: A toolkit for testbed for broad-coverage services,’’
building and dynamically deploying SIGCOMM Comput. Commun. Rev.,
[74] International Telecommunication Union
network protocols,’’ in Proc. IEEE vol. 33, no. 3, pp. 3–12, Jul. 2003.
Telecommunication Standardization
Conf. Open Architect. Netw. Programm., [106] T. Anderson, L. Peterson, S. Shenker, and
Sector (ITU-T), ‘‘Scenarios and signalling
Apr. 1998, pp. 117–129. J. Turner, ‘‘Overcoming the internet impasse
requirements for software-defined BAN
(SBAN)VWorking document,’’ ITU-T SG 11, [89] D. Alexander et al., ‘‘The switchware through virtualization,’’ Computer, vol. 38,
Tech. Rep., work item: Q.SBAN, Jul. 2014. active network architecture,’’ IEEE Network, no. 4, pp. 34–41, Apr. 2005.
[Online]. Available: http://www.itu.int. vol. 12, no. 3, pp. 29–36, May 1998. [107] L. Peterson et al., ‘‘Geni design principles,’’
[75] International Telecommunication Union [90] K. Calvert, S. Bhattacharjee, E. Zegura, and Computer, vol. 39, no. 9, pp. 102–105,
Telecommunication Standardization Sector J. Sterbenz, ‘‘Directions in active networks,’’ Sep. 2006.
(ITU-T), ‘‘Framework of software-defined IEEE Commun. Mag., vol. 36, no. 10, pp. 72–78, [108] A. Bavier, N. Feamster, M. Huang,
networking,’’ Tech. Rep., Recommendation Oct. 1998. L. Peterson, and J. Rexford, ‘‘In VINI
ITU-T Y.3300, Jun. 2014. [Online]. [91] T. Wolf and J. Turner, ‘‘Design issues for veritas: Realistic and controlled network
Available: http://www.itu.int/rec/T-REC-Y. high performance active routers,’’ in Proc. experimentation,’’ SIGCOMM Comput.
3300-201406-I/en. Int. Zurich Seminar Broadband Commun., Commun. Rev., vol. 36, no. 4, pp. 3–14,
[76] Broadband Forum, ‘‘High level 2000, pp. 199–205. Aug. 2006.
requirements and framework for SDN [92] S. da Silva, Y. Yemini, and D. Florissi, [109] B. Pfaff et al., ‘‘Extending networking into
in telecommunication broadband ‘‘The NetScript active network system,’’ the virtualization layer,’’ in Proc. Workshop
networksBroadband Forum SD-313,’’ IEEE J. Sel. Areas Commun., vol. 19, no. 3, Hot Topics Netw., 2009, pp. 1–6.
Tech. Rep., Sep. 2014. [Online]. Available: pp. 538–551, Mar. 2001. [110] B. Lantz, B. Heller, and N. McKeown,
http://www.broadband-forum.org/technical/ [93] D. L. Tennenhouse and D. J. Wetherall, ‘‘A network in a laptop: Rapid prototyping
technicalwip.php. ‘‘Towards an active network architecture,’’ for software-defined networks,’’ in Proc. 9th
[77] Optical Interworking Forum (OIF), SIGCOMM Comput. Commun. Rev., vol. 37, ACM SIGCOMM Workshop Hot Topics Netw.,
‘‘Requirements on transport networks in no. 5, pp. 81–94, Oct. 2007. 2010, pp. 19:1–19:6.

[111] R. Sherwood et al., ‘‘Can the production [129] Juniper Networks, ‘‘Ex9200 Ethernet [149] M. Casado, ‘‘OpenStack and network
network be the testbed?’’ in Proc. 9th USENIX switch,’’ 2013. [Online]. Available: virtualization,’’ Apr. 2013. [Online].
Conf. Oper. Syst. Design Implement., 2010, http://www.juniper.net/us/en/local/pdf/ Available: http://blogs.vmware.com/
pp. 1–6. datasheets/1000432-en.pdf. vmware/2013/04/openstack-and-network-
[112] T. Koponen et al., ‘‘Network virtualization [130] I. Yokneam, ‘‘EZchip announces virtualization.html.
in multi-tenant datacenters,’’ in Proc. 11th OpenFlow 1.1 implementations on its [150] Pica8 Open Networking, ‘‘Pica8’s OS for
USENIX Symp. Netw. Syst. Design Implement., NP-4 100-gigabit network processor,’’ 2011. Open Switches,’’ 2013. [Online]. Available:
Apr. 2014, pp. 203–216. [Online]. Available: http://www.ezchip. http://www.pica8.org/open-switching/
[113] V. Bollapragada, C. Murphy, and R. White, com/pr_110713.htm. open-switching-overview.php.
Inside Cisco IOS Software Architecture, 1st ed. [131] BROCADE, ‘‘MLX Series,’’ 2013. [Online]. [151] Open Network Install Environment (ONIE),
Indianapolis, IN, USA: Cisco Press, Jul. 2000. Available: http://www.brocade.com/ 2013. [Online]. Available: http://onie.org/
[114] Juniper Networks, ‘‘Junos OS architecture products/all/routers/product-details/ [152] T. Kato et al., ‘‘Case study of applying SPLE
overview,’’ 2012. [Online]. Available: netiron-mlx-series/system-options.page. to development of network switch products,’’
http://www.juniper.net/techpubs/en_US/ [132] IBM, ‘‘System networking RackSwitch in Proc. 17th Int. Softw. Product Line Conf.,
junos12.1/topics/concept/junos-software- G8264,’’ 2013. [Online]. Available: 2013, pp. 198–207.
architecture.html. http://www-03.ibm.com/systems/ [153] B. Pfaff and B. Davie, ‘‘The Open vSwitch
[115] Extreme Networks, ‘‘ExtremeXOS operating networking/switches/rack/g8264/. database management protocol,’’ Internet
system, version 15.4,’’ 2014. [Online]. [133] NEC, ‘‘ProgrammableFlow family of Engineering Task Force, RFC 7047
Available: http://learn.extremenetworks. products,’’ 2013. [Online]. Available: (Informational), Dec. 2013. [Online].
com/rs/extreme/images/EXOS-DS.pdf. http://www.necam.com/SDN/. Available: http://www.ietf.org/rfc/
[116] Alcatel-Lucent, ‘‘SR OS,’’ 2014. [Online]. [134] Pica8, ‘‘3920,’’ 2013. [Online]. Available: rfc7047.txt.
Available: http://www3. alcatel-lucent.com/ http://www.pica8.org/documents/pica8- [154] M. Smith et al., ‘‘OpFlex control protocol,’’
products/sros/. datasheet-64x10gbe-p3780-p3920.pdf. Internet Engineering Task Force,’’ Internet
[117] U. Krishnaswamy et al., ‘‘ONOS: An open [135] Plexxi, ‘‘Switch 1,’’ 2013. [Online]. Available: Draft, Apr. 2014. [Online]. Available: http://
source distributed SDN OS,’’ 2013. [Online]. http://www.plexxi.com/wp-content/ tools.ietf.org/html/draft-smith-opflex-00.
Available: http://www.slideshare.net/ themes/plexxi/assets/pdf/Plexxi_Switch_ [155] G. Bianchi, M. Bonola, A. Capone, and
umeshkrishnaswamy/open-network- 1_Datasheet_Dec_2012.pdf. C. Cascone, ‘‘OpenState: Programming
operating-system. [136] Centec Networks, ‘‘v330 OpenFlow switch platform-independent stateful OpenFlow
[118] A. T. Campbell, I. Katzela, K. Miki, and reference design,’’ 2013. [Online]. Available: applications inside the switch,’’ SIGCOMM
J. Vicente, ‘‘Open signaling for ATM, http://www.centecnetworks.com/en/ Comput. Commun. Rev., vol. 44, no. 2,
internet and mobile networks SolutionList.asp?ID=42. pp. 44–51, Apr. 2014.
(OPENSIG’98),’’ SIGCOMM Comput. [137] Cyan, Inc., ‘‘Z-Series,’’ 2013. [Online]. [156] M. Sune, V. Alvarez, T. Jungel, U. Toseef,
Commun. Rev., vol. 29, no. 1, pp. 97–108, Available: http://www.cyaninc.com/en/ and K. Pentikousis, ‘‘An OpenFlow
Jan. 1999. our-solutions/z-series/. implementation for network processors,’’ in
[119] R. Sherwood et al., ‘‘Carving research Proc. 3rd Eur. Workshop Softw. Defined
[138] Juniper Networks, Inc., ‘‘Contrail virtual
slices out of your production networks with Netw., 2014, 2 pp.
router,’’ 2013. [Online]. Available: https://
OpenFlow,’’ SIGCOMM Comput. Commun. github.com/Juniper/contrail-vrouter. [157] D. Parniewicz et al., ‘‘Design and
Rev., vol. 40, no. 1, pp. 129–130, Jan. 2010. implementation of an OpenFlow hardware
[139] FlowForwarding, ‘‘LINC-Switch,’’
[120] H. Song, J. Gong, J. Song, and J. Yu, abstraction layer,’’ in Proc. ACM SIGCOMM
2013. [Online]. Available: http://www.
‘‘Protocol oblivious forwarding (POF),’’ Workshop Distrib. Cloud Comput., 2014,
flowforwarding.org/.
2013. [Online]. Available: http://www. pp. 71–76.
[140] K. Rutka, K. Kaplita, S. Narayan, and
poforwarding.org/. [158] B. Belter et al., ‘‘Hardware abstraction
S. Bailey, ‘‘LINC switch,’’ 2013. [Online].
[121] Open Networking Foundation (ONF), layer as an SDN-enabler for non-OpenFlow
Available: http://www.opennetsummit.org/
‘‘Charter: Forwarding abstractions working network equipment,’’ in Proc. 3rd Eur.
pdf/2013/research_track/poster_papers/
group,’’ Apr. 2014. [Online]. Available: Workshop Softw. Defined Netw., 2014, 6 pp.
ons2013-final36.pdf.
https://www.opennetworking.org/images/ [159] B. Belter et al., ‘‘Programmable abstraction
[141] E. L. Fernandes and C. E. Rothenberg,
stories/downloads/working-groups/ of datapath,’’ in Proc. 3rd Eur. Workshop
‘‘OpenFlow 1.3 software switch,
charter-forwarding-abstractions.pdf. Softw. Defined Netw., 2014, pp. 7–12.
SBRC’2014,’’ 2014. [Online]. Available:
[122] Centec Networks, ‘‘V350VCentec open https://github.com/CPqD/ofsoftswitch13. [160] R. G. Clegg et al., ‘‘Pushing software defined
SDN platform,’’ 2013. [Online]. Available: networking to the access,’’ in Proc. 3rd Eur.
[142] Open vSwitch, 2013. [Online]. Available:
http://www.valleytalk.org/wp-content/ Workshop Softw. Defined Netw., 2014, pp. 1–6.
http://vswitch.org/
uploads/2013/04/Centec-Open-SDN- [161] V. Fuentes et al., ‘‘Integrating complex
Platform.pdf. [143] OpenFlow Community, ‘‘Switching reference
legacy systems under OpenFlow control:
system,’’ 2009. [Online]. Available: http://
[123] NECProgrammable, ‘‘Flow UNIVERGE The DOCSIS use case,’’ in Proc. 3rd Eur.
www.openflow.org/wp/downloads/.
PF5820,’’ 2013. [Online]. Available: Workshop Softw. Defined Netw., 2014, 6 pp.
http://www.nec.com/en/global/ [144] Y. Mundada, R. Sherwood, and N. Feamster,
[162] T. J. Bittman, G. J. Weiss, M. A. Margevicius,
prod/pflow/images_documents/ ‘‘An OpenFlow switch element for click,’’
and P. Dawson, ‘‘Magic quadrant for x86
ProgrammableFlow_Switch_PF5820.pdf. Proc. Symp. Click Modular Router, 2009, 1 p.
server virtualization infrastructure,’’
[Online]. Available: http://www.cc.gatech.
[124] NoviFlow, ‘‘NoviSwitch 1248 high Gartner, Tech. Rep., Jun. 2013.
edu/?yogeshm3/click_symposium2009.pdf.
performance OpenFlow switch,’’ 2013. [163] D. W. Cearley, D. Scott, J. Skorupa, and
[Online]. Available: http://205.236.122.20/ [145] Big Switch Networks, ‘‘Project floodlight,’’
T. J. Bittman, ‘‘Top 10 technology trends,
gestion/NoviSwitch1248Datasheet.pdf. 2013. [Online]. Available: http://www.
2013: Cloud computing and hybrid
projectfloodlight.org/.
[125] HP, ‘‘8200 ZL switch series,’’ 2013. [Online]. IT drive future IT models,’’ Feb. 2013.
Available: http://h17007.www1.hp.com/ [146] Y. Yiakoumis, J. Schulz-Zander, and J. Zhu, [Online]. Available: http://www.
us/en/networking/products/switches/ ‘‘Pantou: OpenFlow 1.0 for OpenWRT,’’ gartnersummit.com/Gartnertop_10_
HP_8200_zl_Switch_Series/. 2011. [Online]. Available: http://www. technology_trends_2012_37716.pdf.
openflow.org/wk/index.php/Open_
[126] Arista Networks, ‘‘7150 Series,’’ 2013. [164] C. Peng, M. Kim, Z. Zhang, and H. Lei,
Flow1.0_forOpenWRT.
[Online]. Available: http://www. ‘‘VDN: Virtual machine image distribution
aristanetworks.com/media/system/ [147] A. Weissberger, ‘‘VMware’s network network for cloud data centers,’’ in Proc.
pdf/Datasheets/7150S_Datasheet.pdf. virtualization poses huge threat to data IEEE INFOCOM, Mar. 2012, pp. 181–189.
center switch fabric vendors,’’ 2013.
[127] Extreme Networks, ‘‘Blackdiamond x8,’’ [165] Z. Zhang et al., ‘‘VMThunder: Fast
[Online]. Available: http://viodi.com/2013/
2013. [Online]. Available: http://www. provisioning of large-scale virtual machine
05/06/vmwares-network-virtualization-
extremenetworks.com/libraries/products/ clusters,’’ IEEE Trans. Parallel Distrib. Syst.,
poses-huge-threat-to-data-center-switch-
DSBDX_1832.pdf. vol. 25, no. 12, pp. 3328–3338, Dec. 2014.
fabric-vendors/.
[128] Huawei Technologies Co., Ltd., ‘‘Cx600 [166] R. Sherwood et al., ‘‘FlowVisor: A network
[148] S. Shenker, ‘‘Stanford
metro services platform,’’ 2013. [Online]. virtualization layer,’’ Deutsche Telekom
SeminarVSoftware-defined networking
Available: http://www.huawei.com/ucmf/ Inc. R&D Lab, Stanford, Nicira Networks,
at the crossroads,’’ Jun. 2013. [Online].
groups/public/documents/attachments/ Tech. Rep., 2009.
Available: http://www.youtube.com/
hw_132369.pdf.
watch?v=WabdXYzCAOU.

[167] S. Azodolmolky et al., ‘‘Optical FlowVisor: controllers,’’ Aug. 2013. [Online]. Available: [202] L. Richardson and S. Ruby, RESTful
An OpenFlow-based optical network http://arxiv.org/abs/1308.6138. Web Services. Sebastopol, CA, USA:
virtualization approach,’’ in Proc. Nat. [186] D. Erickson, ‘‘The Beacon OpenFlow O’Reilly Media, 2008.
Fiber Optic Eng. Conf., Mar. 2012. controller,’’ in Proc. 2nd ACM SIGCOMM [203] T. L. Hinrichs, N. S. Gude, M. Casado,
[168] D. A. Drutskoy, ‘‘Software-defined Workshop Hot Topics Softw. Defined Netw., J. C. Mitchell, and S. Shenker, ‘‘Practical
network virtualization with FlowN,’’ Ph.D. 2013, pp. 13–18. declarative network management,’’ in Proc.
dissertation, Dept. Comput. Sci., Princeton [187] A. Tootoonchian, S. Gorbunov, Y. Ganjali, 1st ACM Workshop Res. Enterprise Netw.,
Univ., Princeton, NJ, USA, Jun. 2012. M. Casado, and R. Sherwood, ‘‘On controller 2009, DOI: 10.1145/1592681.1592683.
[169] A. Al-Shabibi et al., ‘‘OpenVirteX: A Network performance in software-defined networks,’’ [204] N. Foster et al., ‘‘Frenetic: A network
Hypervisor,’’ 2014. [Online]. Available: in Proc. 2nd USENIX Conf. Hot Topics Manage. programming language,’’ SIGPLAN Notes,
http://ovx.onlab.us/wp-content/uploads/ Internet Cloud Enterprise Netw. Services, 2012, vol. 46, no. 9, pp. 279–291, 2011.
2014/04/ovx-ons14. pdf. p. 10. [205] C. Monsanto, N. Foster, R. Harrison, and
[170] A. Al-Shabibi et al., ‘‘OpenVirteX: Make your [188] Z. Cai, A. L. Cox, and T. S. E. Ng, ‘‘Maestro: D. Walker, ‘‘A compiler and run-time system
virtual SDNs programmable,’’ in Proc. 3rd A system for scalable OpenFlow control,’’ for network programming languages,’’
Workshop Hot Topics Softw. Defined Netw., Rice Univ., Houston, TX, USA, Tech. SIGPLAN Notes, vol. 47, no. 1, pp. 217–230,
2014, pp. 25–30. Rep., 2011. Jan. 2012.
[171] S. Racherla et al., Implementing IBM Software [189] Project Floodlight, ‘‘Floodlight,’’ 2012. [206] A. Singla and B. Rijsman, ‘‘Contrail
Defined Network for Virtual Environments. [Online]. Available: http://floodlight. architecture,’’ Juniper Networks, Tech.
Durham, NC, USA: IBM RedBooks, openflowhub.org/. Rep., 2013.
May 2014. [190] Y. Takamiya and N. Karanatsios, ‘‘Trema [207] Open Networking Foundation (ONF),
[172] C. Li et al., ‘‘Software defined environments: OpenFlow controller framework,’’ 2012. ‘‘OpenFlow management and configuration
An introduction,’’ IBM J. Res. Develop., [Online]. Available: https://github.com/ protocol (OF-Config 1.1.1),’’ Mar. 2014.
vol. 58, no. 2, pp. 1–11, Mar. 2014. trema/trema. [Online]. Available: https://www.
[173] A. Gudipati, L. E. Li, and S. Katti, [191] Nippon Telegraph and Telephone opennetworking.org/images/stories/
‘‘Radiovisor: A slicing plane for radio Corporation, ‘‘RYU network operating downloads/sdn-resources/onf-specifications/
access networks,’’ in Proc. 3rd Workshop system,’’ 2012. [Online]. Available: openflow-config/of-config-1-1-1.pdf.
Hot Topics Softw. Defined Netw., 2014, http://osrg.github.com/ryu/. [208] D. Harrington, R. Presuhn, and B. Wijnen,
pp. 237–238. [192] M. Banikazemi, D. Olshefski, A. Shaikh, ‘‘An architecture for describing simple
[174] H. Yamanaka, E. Kawai, S. Ishii, and J. Tracey, and G. Wang, ‘‘Meridian: network management protocol (SNMP)
S. Shimojo, ‘‘AutoVFlow: Autonomous An SDN platform for cloud network management frameworks,’’ Internet
virtualization for wide-area OpenFlow services,’’ IEEE Commun. Mag., vol. 51, no. 2, Engineering Task Force, Dec. 2002.
networks,’’ in Proc. 3rd Eur. Workshop pp. 120–127, Feb. 2013. [Online]. Available: http://www.ietf.org/
Softw. Defined Netw., 2014, 6 pp. rfc/rfc3411.txt.
[193] NEC, ‘‘Award-winning software-defined
[175] Berlin Institute for Software Defined networking NEC ProgrammableFlow [209] H. Yin et al., ‘‘SDNi: A message exchange
Networks (BISDN) GmbH, ‘‘The eXtensible networking suite,’’ Sep. 2013. [Online]. protocol for software defined networks
OpenFlow Datapath Daemon (xdpd) Available: http://www.necam.com/ (SDNS) across multiple domains,’’ Internet
bringing innovation into the fast path,’’ 2014. docs/?id=67c33426-0a2b-4b87-9a7a- Engineering Task Force, Internet Draft,
[Online]. Available: http://xdpd.org/. d3cecc14d26a. Jun. 2012. [Online]. Available: http://tools.
ietf.org/id/draft-yin-sdn-sdni-00.txt.
[176] R. Doriguzzi-Corin, E. Salvadori, [194] S. Shin et al., ‘‘Rosemary: A robust, secure,
M. Gerola, M. Sune, and H. Woesner, high-performance network operating [210] M. Garcia, A. Bessani, I. Gashi, N. Neves,
‘‘A datapath-centric virtualization system,’’ in Proc. 21st ACM Conf. Comput. and R. Obelheiro, ‘‘Analysis of operating
mechanism for OpenFlow networks,’’ in Commun. Security, Scottsdale, AZ, USA, system diversity for intrusion tolerance,’’
Proc. 3rd Eur. Workshop Softw. Defined Nov. 2014, pp. 78–89. Softw., Practice Experience, vol. 44, no. 6,
Netw., 2014, pp. 1–6. pp. 735–770, 2014.
[195] A. Tootoonchian and Y. Ganjali,
[177] T. Szyrkowiec et al., ‘‘Demonstration of ‘‘HyperFlow: A distributed control plane [211] Z. Wang, T. Tsou, J. Huang, X. Shi, and
SDN based optical network virtualization for OpenFlow,’’ in Proc. Internet Netw. X. Yin, ‘‘Analysis of comparisons between
and multidomain service orchestration,’’ in Manage. Conf. Res. Enterprise Netw., 2010, OpenFlow and ForCES,’’ Internet
Proc. 3rd Eur. Workshop Softw. Defined p. 3. Engineering Task Force, Internet Draft,
Netw., 2014, 2 pp. Dec. 2011. [Online]. Available: http://tools.
[196] M. Monaco, O. Michel, and E. Keller,
ietf.org/id/draft-wang-forces-compare-open-
[178] D. Depaoli, R. Doriguzzi-Corin, M. Gerola, ‘‘Applying operating system principles to
flow-forces-00.txt.
and E. Salvadori, ‘‘Demonstrating a SDN controller design,’’ in Proc. 12th ACM
distributed and version-agnostic OpenFlow Workshop Hot Topics Netw., College Park, [212] K. Ogawa, W. M. Wang, E. Haleplidis, and
slicing mechanism,’’ in Proc. 3rd Eur. MD, USA, Nov. 2013, DOI: 10.1145/ J. H. Salim, ‘‘ForCES Intra-NE high
Workshop Softw. Defined Netw., 2014, 2 pp. 2535771.2535789. availability,’’ Internet Engineering Task
Force, Internet Draft, Oct. 2013. [Online].
[179] Z. Bozakov and P. Papadimitriou, ‘‘AutoSlice: [197] A. D. Ferguson, A. Guha, C. Liang,
Available: http://tools.ietf.org/id/draft-ietf-
Automated and scalable slicing for R. Fonseca, and S. Krishnamurthi,
forces-ceha-08.txt.
software-defined networks,’’ in Proc. ACM ‘‘Participatory networking: An API for
Conf. CoNEXT Student Workshop, 2012, application control of SDNs,’’ in Proc. [213] F. A. Botelho, F. M. V. Ramos, D. Kreutz,
pp. 3–4. ACM SIGCOMM Conf., 2013, pp. 327–338. and A. N. Bessani, ‘‘On the feasibility of a
consistent and fault-tolerant data store for
[180] D. Drutskoy, E. Keller, and J. Rexford, [198] F. Botelho, A. Bessani, F. M. V. Ramos, and
SDNs,’’ in Proc. 2nd Eur. Workshop Softw.
‘‘Scalable network virtualization in P. Ferreira, ‘‘On the design of practical
Defined Netw., 2013, pp. 38–43.
software-defined networks,’’ IEEE fault-tolerant SDN controllers,’’ in Proc. 3rd
Internet Comput., vol. 17, no. 2, pp. 20–27, Eur. Workshop Softw. Defined Netw., 2014, [214] S. Vinoski, ‘‘Advanced message queuing
Mar./Apr. 2013. 6 pp. protocol,’’ IEEE Internet Comput., vol. 10,
no. 6, pp. 87–89, Nov. 2006.
[181] X. Jin, J. Rexford, and D. Walker, [199] S. Matsumoto, S. Hitz, and A. Perrig,
‘‘Incremental update for a compositional ‘‘Fleet: Defending SDNs from malicious [215] M. Canini, P. Kuznetsov, D. Levin, and
SDN hypervisor,’’ in Proc. 3rd Workshop administrators,’’ in Proc. 3rd Workshop S. Schmid, ‘‘Software transactional
Hot Topics Softw. Defined Netw., 2014, Hot Topics Softw. Defined Netw., 2014, networking: Concurrent and consistent
pp. 187–192. pp. 103–108. policy composition,’’ in Proc. 2nd ACM
SIGCOMM Workshop Hot Topics Softw.
[182] S. Ghorbani and B. Godfrey, ‘‘Towards [200] A. Bierman, M. Bjorklund, K. Watsen, and
Defined Netw., 2013, DOI: 10.1145/2491185.
correct network virtualization,’’ in Proc. R. Fernando, ‘‘RESTCONF protocol,’’
2491200.
3rd Workshop Hot Topics Softw. Defined Netw., Internet Engineering Task Force, Internet
2014, pp. 109–114. Draft, Jul. 2014. [Online]. Available: [216] A. Ghodsi, ‘‘Distributed k-ary system:
http://tools.ietf.org/html/draft-ietf- Algorithms for distributed Hash tables,’’
[183] Juniper Networks, ‘‘Opencontrail,’’ 2013.
netconf-restconf-01. Ph.D. dissertation, Dept. Electr., Comp.
[Online]. Available: http://opencontrail.org/.
Software Syst., Royal Inst. Technol. (KTH),
[184] HP, ‘‘SDN controller architecture,’’ [201] P. Porras et al., ‘‘A security enforcement
Stockholm, Sweden, Oct. 2006.
Tech. Rep., Sep. 2013. kernel for OpenFlow networks,’’ in Proc.
1st Workshop Hot Topics Softw. Defined [217] W. Stallings, ‘‘Software-defined networks
[185] K. Phemius, M. Bouet, and J. Leguay, and OpenFlow,’’ Internet Protocol J., vol. 16,
Netw., 2012, pp. 121–126.
‘‘DISCO: Distributed multi-domain SDN no. 1, pp. 1–6, 2013.

[218] Open Networking Foundation (ONF), network,’’ in Proc. 2nd Workshop Hot Topics [255] M. S. Farooq, S. A. Khan, F. Ahmad, S. Islam,
‘‘SDN architecture,’’ Jun. 2014. [Online]. Softw. Defined Netw., 2013, pp. 25–30. and A. Abid, ‘‘An evaluation framework
Available: https://www.opennetworking.org/ [237] J. Dix, ‘‘Clarifying the role of software-defined and comparative analysis of the widely used
images/stories/downloads/sdn-resources/ networking northbound APIs,’’ May 2013. first programming languages,’’ PLoS ONE,
technical-reports/TR_SDN_ARCH_1.0_ [Online]. Available: http://www. vol. 9, no. 2, 2014, DOI: 10.1371/journal.
06062014.pdf. networkworld.com/news/2013/050213- pone.0088941.
[219] R. Hand and E. Keller, ‘‘ClosedFlow: sherwood-269366.html. [256] A. D. Ferguson, A. Guha, C. Liang,
OpenFlow-like control over proprietary [238] I. Guis, ‘‘The SDN gold rush to the R. Fonseca, and S. Krishnamurthi,
devices,’’ in Proc. 3rd Workshop Hot Topics northbound API,’’ Nov. 2012. [Online]. ‘‘Hierarchical policies for software defined
Softw. Defined Netw., 2014, pp. 7–12. Available: http://www.sdncentral.com/ networks,’’ in Proc. 1st Workshop Hot Topics
[220] M. Jarschel, T. Zinner, T. HoQfeld, technology/the-sdn-gold-rush-to-the- Softw. Defined Netw., 2012, pp. 37–42.
P. Tran-Gia, and W. Kellerer, ‘‘Interfaces, northbound-api/2012/11/. [257] T. Nelson, A. D. Ferguson, M. J. Scheer, and
attributes, use cases: A compass for [239] B. Salisbury, ‘‘The northbound APIVA big S. Krishnamurthi, ‘‘Tierless programming
SDN,’’ IEEE Commun. Mag., vol. 52, no. 6, little problem,’’ 2012. and reasoning for software-defined
pp. 210–217, Jun. 2014. networks,’’ in Proc. 11th USENIX Symp.
[240] G. Ferro, ‘‘Northbound API, southbound
[221] E. Mannie, ‘‘Generalized multi-protocol Netw. Syst. Design Implement., Apr. 2014,
API, east/north LAN navigation in an
label switching (GMPLS) architecture,’’ pp. 519–531.
OpenFlow world and an SDN compass,’’
Internet Engineering Task Force, RFC 3945 Aug. 2012. [258] N. P. Katta, J. Rexford, and D. Walker,
(Proposed Standard), Oct. 2004, updated by ‘‘Logic programming for software-defined
[241] B. Casemore, ‘‘Northbound API: The
RFC 6002. [Online]. Available: http://www. networks,’’ in Proc. ACM SIGPLAN Workshop
standardization debate,’’ Sep. 2012. [Online].
ietf.org/rfc/rfc3945.txt. Cross-Model Lang. Design Implement., 2012,
Available: http://nerdtwilight.wordpress.
[222] K. Pentikousis, Y. Wang, and W. Hu, pp. 1–3.
com/2012/09/18/northbound-api-the-
‘‘MobileFlow: Toward software-defined standardization-debate/. [259] S. Shin et al., ‘‘FRESCO: Modular
mobile networks,’’ IEEE Commun. Mag., composable security services for
[242] I. Pepelnjak, ‘‘SDN controller northbound
vol. 51, no. 7, pp. 44–53, Jul. 2013. software-defined networks,’’ Internet
API is the crucial missing piece,’’ Sep. 2012.
[223] A. Voellmy and P. Hudak, ‘‘Nettle: Taking Society NDSS, Feb. 2013.
[Online]. Available: http://blog.ioshints.
the sting out of programming network info/2012/09/sdn-controller-northbound- [260] S. Son, S. Shin, V. Yegneswaran, P. Porras,
routers,’’ in Proc. 13th Int. Conf. Practical api-is.html. and G. Gu, ‘‘Model checking invariant
Aspects Declarative Lang., 2011, pp. 235–249. security properties in OpenFlow,’’ in Proc.
[243] S. Johnson, ‘‘A primer on northbound APIs:
[224] A. Voellmy, H. Kim, and N. Feamster, IEEE Int. Conf. Commun., Jun. 2013,
Their role in a software-defined network,’’
‘‘Procera: A language for high-level reactive pp. 1974–1979.
Dec. 2012. [Online]. Available: http://
network control,’’ in Proc. 1st Workshop Hot searchsdn.techtarget.com/feature/A-primer- [261] A. Tootoonchian, M. Ghobadi, and
Topics Softw. Defined Netw., 2012, pp. 43–48. on-northbound-APIs-Their-role-in-a- Y. Ganjali, ‘‘OpenTM: Traffic matrix
[225] C. Monsanto, J. Reich, N. Foster, software-defined-network. estimator for OpenFlow networks,’’ in Proc.
J. Rexford, and D. Walker, ‘‘Composing 11th Int. Conf. Passive Active Meas., 2010,
[244] R. G. Little, ‘‘ONF to standardize northbound
software-defined networks,’’ in Proc. 10th pp. 201–210.
API for SDN applications?’’ Oct. 2013.
USENIX Conf. Netw. Syst. Design Implement., [Online]. Available: http://searchsdn. [262] M. Reitblatt, M. Canini, A. Guha, and
2013, pp. 1–14. techtarget.com/news/2240206604/ONF- N. Foster, ‘‘FatTire: Declarative fault
[226] C. J. Anderson et al., ‘‘NetKAT: Semantic to-standardize-northbound-API-for-SDN- tolerance for software defined networks,’’ in
foundations for networks,’’ SIGPLAN Notes, applications. Proc. 2nd Workshop Hot Topics Softw. Defined
vol. 49, no. 1, pp. 113–126, Jan. 2014. Netw., 2013, pp. 109–114.
[245] Austin Common Standards Revision
[227] S. Narayana, J. Rexford, and D. Walker, Group, ‘‘POSIX,’’ 2014. [Online]. Available: [263] A. Voellmy, J. Wang, Y. R. Yang, B. Ford, and
‘‘Compiling path queries in software-defined http://standards.ieee.org/develop/wg/ P. Hudak, ‘‘Maple: Simplifying SDN
networks,’’ in Proc. 3rd Workshop Hot Topics POSIX.html. programming using algorithmic policies,’’ in
Softw. Defined Netw., 2014, pp. 181–186. Proc. ACM SIGCOMM Conf., 2013, pp. 87–98.
[246] M. Yu, A. Wundsam, and M. Raju,
[228] A. A. Dixit, F. Hao, S. Mukherjee, ‘‘NOSIX: A lightweight portability layer for [264] R. Soule, S. Basu, R. Kleinberg, E. G. Sirer,
T. Lakshman, and R. Kompella, ‘‘Elasticon: the SDN OS,’’ SIGCOMM Comput. Commun. and N. Foster, ‘‘Managing the network with
An elastic distributed SDN controller,’’ in Rev., vol. 44, no. 2, pp. 28–35, Apr. 2014. Merlin,’’ in Proc. 12th ACM Workshop Hot
Proc. 10th ACM/IEEE Symp. Architect. Netw. Topics Netw., Nov. 2013, DOI: 10.1145/
[247] R. Chua, ‘‘OpenFlow northbound API: A new
Commun. Syst., 2014, pp. 17–28. 2535771.2535792.
Olympic sport,’’ 2012. [Online]. Available:
[229] S. Hassas Yeganeh and Y. Ganjali, ‘‘Kandoo: http://www.sdncentral.com/sdn-blog/ [265] C. Jasson Casey, A. Sutton, G. Dos Reis, and
A framework for efficient and scalable openflow-northbound-api-olympics/ A. Sprintson, ‘‘Eliminating network protocol
offloading of control applications,’’ in Proc. 2012/07/. vulnerabilities through abstraction and
1st Workshop Hot Topics Softw. Defined Netw., systems language design,’’ Nov. 2013.
[248] J. Reich, C. Monsanto, N. Foster, J. Rexford,
2012, pp. 19–24. [Online]. Available: http://arxiv.org/abs/
and D. Walker, ‘‘Modular SDN programming
1311.3336.
[230] D. Saikia, ‘‘MuL OpenFlow Controller,’’ with pyretic,’’ USENIX Mag., vol. 38, no. 5,
2013. [Online]. Available: http:// Oct. 2013. [266] X. Wen et al., ‘‘Compiling minimum
sourceforge.net/projects/mul/. incremental update for modular SDN
[249] K.-K. Yap, T.-Y. Huang, B. Dodson,
languages,’’ in Proc. 3rd Workshop Hot Topics
[231] M. McCauley, ‘‘POX,’’ 2012. [Online]. M. S. Lam, and N. McKeown, ‘‘Towards
Available: http://www.noxrepo.org/. software-friendly networks,’’ in Proc. 1st ACM
Asia-Pacific Workshop Syst., 2010, pp. 49–54. [267] P. Pereini, M. Kuzniar, and D. Kostic,
[232] H. Shimonishi and S. Ishii, ‘‘Virtualized
‘‘OpenFlow needs you! A call for a discussion
network infrastructure using OpenFlow,’’ in [250] S. Gutz, A. Story, C. Schlesinger, and
about a cleaner OpenFlow API,’’ in Proc.
Proc. IEEE/IFIP Netw. Oper. Manage. Symp. N. Foster, ‘‘Splendid isolation: A slice
2nd Eur. Workshop Softw. Defined Netw.,
Workshops, 2010, pp. 74–79. abstraction for software-defined networks,’’
Oct. 2013, pp. 44–49.
[233] G. Appenzeller, ‘‘SNAC,’’ 2011. [Online]. in Proc. 1st Workshop Hot Topics Softw.
Defined Netw., 2012, pp. 79–84. [268] F. Facca et al., ‘‘NetIDE: First steps towards
Available: http://www. openflowhub.org/
an integrated development environment
display/Snac. [251] D. Turull, M. Hidell, and P. Sjödin,
for portable network apps,’’ in Proc. 2nd Eur.
[234] B. Casemore, ‘‘SDN controller ecosystems ‘‘Evaluating OpenFlow in libnetvirt,’’ in Proc.
Workshop Softw. Defined Netw., Oct. 2013,
critical to market success,’’ 2012. [Online]. 8th Swedish Nat. Comput. Netw. Workshop,
pp. 105–110.
Available: http://nerdtwilight.wordpress. Oct. 2012, pp. 1–5.
[269] E. Reinecke, ‘‘Mapping the future
com/2012/06/05/sdn-controller-ecosystems- [252] Quantum Community, ‘‘OpenStack
of software-defined networking,’’ 2014.
critical-to-market-success/. networking (‘Quantum’),’’ 2012.
[Online]. Available: http://goo.gl/fQCvRF.
[235] R. Kwan and C. Leung, ‘‘A survey of [253] Small Cell Forum, ‘‘Femto APIs,’’ 2013.
[270] M. Scharf et al., ‘‘Dynamic VPN optimization
scheduling and interference mitigation in [Online]. Available: http://www.
by ALTO guidance,’’ in Proc. 2nd Eur.
LTE,’’ J. Electr. Comput. Eng., vol. 2010, smallcellforum.org/developers/.
Workshop Softw. Defined Netw., Oct. 2013,
Jan. 2010, DOI: 10.1155/2010/273486. [254] M. Guzdial, ‘‘Education: Paving the way pp. 13–18.
[236] A. Gudipati, D. Perry, L. E. Li, and S. Katti, for computational thinking,’’ Commun.
[271] M. Stiemerling, S. Kiesel, S. Previdi, and
‘‘SoftRAN: Software defined radio access ACM, vol. 51, no. 8, pp. 25–27, Aug. 2008.
M. Scharf, ‘‘ALTO deployment

considerations,’’ Internet Engineering Task defined networking with generalized [305] S. Shirali-Shahreza and Y. Ganjali,
Force, Internet Draft, Feb. 2014. [Online]. OpenFlows,’’ in Proc. IEEE Netw. Oper. ‘‘FleXam: Flexible sampling extension for
Available: http://tools.ietf.org/html/draft- Manage. Symp., Apr. 2012, pp. 1167–1174. monitoring and security applications in
ietf-alto-deployments-09. [288] S. Sharma et al., ‘‘Implementing quality of OpenFlow,’’ in Proc. 2nd ACM SIGCOMM
[272] R. Alimi, R. Penno, and Y. Yang, ‘‘ALTO service for the software defined networking Workshop Hot Topics Softw. Defined Netw.,
protocol,’’ Internet Engineering Task enabled future internet,’’ in Proc. 3rd Eur. 2013, pp. 167–168.
Force, Internet Draft, Mar. 2014. [Online]. Workshop Softw. Defined Netw., 2014, [306] C. Yu et al., ‘‘FlowSense: Monitoring
Available: https://datatracker.ietf.org/doc/ pp. 49–54. network utilization with zero measurement
draft-ietf-alto-protocol/. [289] W. Kim et al., ‘‘Automated and scalable cost,’’ in Proc. 14th Int. Conf. Passive Active
[273] N. Handigol et al., ‘‘Aster*x: Load-balancing QoS control for network convergence,’’ in Meas., 2013, pp. 31–41.
web traffic over wide-area networks,’’ 2009. Proc. Internet Netw. Manage. Conf. Res. [307] L. Jose, M. Yu, and J. Rexford, ‘‘Online
[274] B. Heller et al., ‘‘ElasticTree: Saving energy Enterprise Netw., 2010, p. 1. measurement of large traffic aggregates on
in data center networks,’’ in Proc. 7th USENIX [290] A. Ishimori, F. Farias, E. Cerqueira, and commodity switches,’’ in Proc. 11th USENIX
Conf. Netw. Syst. Design Implement., 2010, A. Abelem, ‘‘Control of multiple packet Conf. Hot Topics Manage. Internet Cloud
p. 17. schedulers for improving QoS on OpenFlow/ Enterprise Netw. Services, 2011, p. 13.
[275] M. S. Seddiki et al., ‘‘FlowQoS: QoS for the SDN networking,’’ in Proc. 2nd Eur. [308] N. L. M. van Adrichem, C. Doerr, and
rest of us,’’ in Proc. 3rd Workshop Hot Topics Workshop Softw. Defined Netw., Oct. 2013, F. A. Kuipers, ‘‘OpenNetMon: Network
Softw. Defined Netw., 2014, pp. 207–208. pp. 81–86. monitoring in openflow software-defined
[291] D. Palma et al., ‘‘The QueuePusher: Enabling networks,’’ in Proc. IEEE Netw. Oper. Manage.
[276] M. Al-Fares, S. Radhakrishnan, B. Raghavan,
queue management in OpenFlow,’’ in Proc. Symp., 2014, DOI: 10.1109/NOMS.2014.
N. Huang, and A. Vahdat, ‘‘Hedera: Dynamic
3rd Eur. Workshop Softw. Defined Netw., 2014, 6838228.
flow scheduling for data center networks,’’
in Proc. 7th USENIX Conf. Netw. Syst. Design pp. 125–126. [309] J. Suh, T. Kwon, C. Dixon, W. Felter, and
Implement., 2010, p. 19. [292] Z. A. Qazi et al., ‘‘SIMPLE-fying middlebox J. Carter, ‘‘OpenSample: A low-latency,
policy enforcement using SDN,’’ in Proc. sampling-based measurement platform for
[277] C. Macapuna, C. Rothenberg, and
Conf. Appl. Technol. Architect. Protocols commodity SDN,’’ in Proc. IEEE 34th Int.
M. Magalhaes, ‘‘In-packet bloom filter based
Comput. Commun., 2013, pp. 27–38. Conf. Distrib. Comput. Syst., Jun. 2014,
data center networking with distributed
pp. 228–237.
OpenFlow controllers,’’ in Proc. IEEE [293] P. Skoldstrom and B. C. Sanchez, ‘‘Virtual
GLOBECOM Workshops, 2010, pp. 584–588. aggregation using SDN,’’ in Proc. 2nd Eur. [310] sFlow.org Forum, 2012. [Online]. Available:
Workshop Softw. Defined Netw., 2013, http://www. sflow.org/
[278] T. Benson, A. Anand, A. Akella, and
M. Zhang, ‘‘MicroTE: Fine grained traffic pp. 56–61. [311] M. Yu, L. Jose, and R. Miao, ‘‘Software
engineering for data centers,’’ in Proc. [294] J. Schulz-Zander, N. Sarrar, and S. Schmid, defined traffic measurement with
7th Conf. Emerging Netw. Exp. Technol., ‘‘AeroFlux: A near-sighted controller OpenSketch,’’ in Proc. 10th USENIX
2011, pp. 8:1–8:12. architecture for software-defined wireless Conf. Netw. Syst. Design Implement.,
networks,’’ presented at the Open Netw. 2013, pp. 29–42.
[279] H. Egilmez, S. Dane, K. Bagci, and A. Tekalp,
‘‘OpenQoS: An Open-Flow controller design Summit, Santa Clara, CA, USA, 2014. [312] C. Argyropoulos, D. Kalogeras,
for multimedia delivery with end-to-end [295] J. Schulz-Zander, N. Sarrar, and S. Schmid, G. Androulidakis, and V. Maglaris,
quality of service over software-defined ‘‘Towards a scalable and near-sighted control ‘‘PaFloMonVA slice aware passive flow
networks,’’ in Proc. Asia-Pacific Signal plane architecture for WiFi SDNs,’’ in monitoring framework for OpenFlow
Inf. Process. Assoc. Annu. Summit Conf., Proc. 3rd Workshop Hot Topics Softw. Defined enabled experimental facilities,’’ in Proc.
2012, pp. 1–8. Netw., 2014, pp. 217–218. Eur. Workshop Softw. Defined Netw., 2012,
pp. 97–102.
[280] A. Sgambelluri, A. Giorgetti, [296] H. Ali-Ahmad et al., ‘‘CROWD: An SDN
F. Cugini, F. Paolucci, and P. Castoldi, approach for densenets,’’ in Proc. 2nd Eur. [313] S. R. Chowdhury, M. F. Bari, R. Ahmed, and
‘‘OpenFlow-based segment protection Workshop Softw. Defined Netw., Oct. 2013, R. Boutaba, ‘‘PayLess: A low cost network
in Ethernet networks,’’ IEEE/OSA J. Opt. pp. 25–31. monitoring framework for software defined
Commun. Netw., vol. 5, no. 9, pp. 1066–1075, networks,’’ in Proc. 14th IEEE/IFIP Netw.
[297] J. Vestin et al., ‘‘CloudMAC: Towards
Sep. 2013. Oper. Manage. Symp., 2014, DOI: 10.1109/
software defined WLANs,’’ SIGMOBILE
NOMS.2014.6838227.
[281] M. Bari, S. Chowdhury, R. Ahmed, and Mob. Comput. Commun. Rev., vol. 16, no. 4,
R. Boutaba, ‘‘PolicyCop: An autonomic QoS pp. 42–45, Feb. 2013. [314] G. Wang, T. E. Ng, and A. Shaikh,
policy enforcement framework for software ‘‘Programming your network at run-time
[298] A. Dawson, M. K. Marina, and F. J. Garcia,
defined networks,’’ in Proc. IEEE SDN Future for big data applications,’’ in Proc. 1st
‘‘On the benefits of RAN virtualisation in
Netw. Services, Nov. 2013, DOI: 10.1109/ Workshop Hot Topics Softw. Defined Netw.,
C-RAN based mobile networks,’’ in Proc. 3rd
SDN4FNS.2013.6702548. 2012, pp. 103–108.
Eur. Workshop Softw. Defined Netw., 2014,
[282] K. Nagaraj and S. Katti, ‘‘ProCel: Smart 6 pp. [315] T. Benson, A. Akella, A. Shaikh, and S. Sahu,
traffic handling for a scalable software EPC,’’ ‘‘Cloudnaas: A cloud networking platform
[299] Y. Yamasaki, Y. Miyamoto, J. Yamato,
in Proc. 3rd Workshop Hot Topics Softw. for enterprise applications,’’ in Proc.
H. Goto, and H. Sone, ‘‘Flexible access
Defined Netw., 2014, pp. 43–48. 2nd ACM Symp. Cloud Comput., 2011,
management system for campus VLAN based
pp. 8:1–8:13.
[283] P. Xiong and H. Hacigümüs, ‘‘Pronto: A on OpenFlow,’’ in Proc. IEEE/IPSJ 11th Int.
software-defined networking based system Symp. Appl. Internet, 2011, pp. 347–351. [316] A. Das et al., ‘‘Transparent and flexible
for performance management of analytical network management for big data processing
[300] J. Schulz-Zander et al., ‘‘Programmatic
queries on distributed data stores,’’ PVLDB, in the cloud,’’ in Proc. 5th USENIX Conf.
orchestration of WiFi networks,’’ in Proc.
vol. 7, no. 13, pp. 1661–1664, 2014. Hot Topics Cloud Comput., San Jose, CA, USA,
USENIX Annu. Tech. Conf., Jun. 2014,
2013, pp. 1–6.
[284] P. Xiong, H. Hacigumus, and J. F. Naughton, pp. 347–358.
‘‘A software-defined networking based [317] A. Arefin, V. K. Singh, G. Jiang, Y. Zhang,
[301] M. Yang et al., ‘‘OpenRAN: A
approach for performance management of and C. Lumezanu, ‘‘Diagnosing data center
software-defined ran architecture via
analytical queries on distributed data stores,’’ behavior flow by flow,’’ in Proc. IEEE 33rd
virtualization,’’ in Proc. ACM SIGCOMM
in Proc. ACM SIGMOD Int. Conf. Manage. Int. Conf. Distrib. Comput. Syst., Jul. 2013,
Conf., 2013, pp. 549–550.
Data, 2014, pp. 955–966. DOI: 10.1109/ICDCS.2013.18.
[302] K.-K. Yap et al., ‘‘OpenRoads: Empowering
[285] N. Handigol, S. Seetharaman, M. Flajslik, [318] E. Keller, S. Ghorbani, M. Caesar, and
research in mobile networks,’’ SIGCOMM
N. McKeown, and R. Johari, ‘‘Plug-n-serve: J. Rexford, ‘‘Live migration of an entire
Comput. Commun. Rev., vol. 40, no. 1,
Load-balancing web traffic using OpenFlow,’’ network (and its hosts),’’ in Proc. 11th
pp. 125–126, Jan. 2010.
2009. ACM Workshop Hot Topics Netw., 2012,
[303] V. Chandrasekhar, J. Andrews, and pp. 109–114.
[286] M. Veiga Neves, C. De Rose, K. Katrinis, and A. Gatherer, ‘‘Femtocell networks:
H. Franke, ‘‘Pythia: Faster big data in motion [319] R. Raghavendra, J. Lobo, and K.-W. Lee,
A survey,’’ IEEE Commun. Mag., vol. 46,
through predictive software-defined network ‘‘Dynamic graph query primitives for
no. 9, pp. 59–67, Sep. 2008.
optimization at runtime,’’ in Proc. IEEE SDN-based cloudnetwork management,’’ in
[304] Y. Yu, C. Qian, and X. Li, ‘‘Distributed and Proc. 1st Workshop Hot Topics Softw. Defined
28th Int. Parallel Distrib. Process. Symp.,
collaborative traffic monitoring in software Netw., 2012, pp. 97–102.
May 2014, pp. 82–90.
defined networks,’’ in Proc. 3rd Workshop Hot
[287] K. Jeong, J. Kim, and Y.-T. Kim, ‘‘QoS-aware [320] M. Ghobadi, ‘‘TCP adaptation framework
Topics Softw. Defined Netw., 2014, pp. 85–90.
network operating system for software in data centers,’’ Ph.D. dissertation, Grad.

Dept. Comput. Sci., Univ. Toronto, Toronto, tree switching,’’ in Proc. IEEE/IPSJ 12th Int. Virtualized Infrastructure Syst. Architect.,
ON, Canada, 2013. Symp. Appl. Internet, 2012, pp. 60–67. 2010, pp. 25–32.
[321] R. Hand, M. Ton, and E. Keller, ‘‘Active [336] K. Giotis, G. Androulidakis, and V. Maglaris, [352] M. Bansal, J. Mehlman, S. Katti, and P. Levis,
security,’’ in Proc. 12th ACM Workshop Hot ‘‘Leveraging SDN for efficient anomaly ‘‘Openradio: A programmable wireless
Topics Netw., Nov. 2013, 17 pp. detection and mitigation on legacy dataplane,’’ in Proc. 1st Workshop Hot Topics
[322] S. Shin, V. Yegneswaran, P. Porras, and networks,’’ in Proc. 3rd Eur. Workshop Softw. Defined Netw., 2012, pp. 109–114.
G. Gu, ‘‘AVANT-GUARD: Scalable and Softw. Defined Netw., 2014, 6 pp. [353] S. Sundaresan et al., ‘‘Broadband internet
vigilant switch flow management in [337] G. Yao, J. Bi, and P. Xiao, ‘‘Source address performance: A view from the gateway,’’
software-defined networks,’’ in Proc. ACM validation solution with OpenFlow/NOX SIGCOMM Comput. Commun. Rev., vol. 41,
Conf. Comput. Commun. Security, 2013, architecture,’’ in Proc. 19th IEEE Int. Conf. no. 4, pp. 134–145, Aug. 2011.
pp. 413–424. Netw. Protocols, 2011, pp. 7–12. [354] S. A. Mehdi, J. Khalid, and S. A. Khayam,
[323] S. Shin and G. Gu, ‘‘CloudWatcher: Network [338] R. Wang, D. Butnariu, and J. Rexford, ‘‘Revisiting traffic anomaly detection using
security monitoring using OpenFlow in ‘‘OpenFlow-based server load balancing gone software defined networking,’’ in Proc.
dynamic cloud networks (or: How to provide wild,’’ in Proc. 11th USENIX Conf. Hot Topics 14th Int. Conf. Recent Adv. Intrusion Detection,
security monitoring as a service in clouds?),’’ Manage. Internet Cloud Enterprise Netw. 2011, pp. 161–180.
in Proc. 20th IEEE Int. Conf. Netw. Protocols, Services, 2011, p. 12. [355] P. Wette and H. Karl, ‘‘Which flows are
2012, DOI: 10.1109/ICNP.2012.6459946. [339] X.-N. Nguyen, D. Saucez, C. Barakat, and hiding behind my wildcard rule?: Adding
[324] E. Tantar, M. Palattella, T. Avanesov, T. Turletti, ‘‘Optimizing rules placement packet sampling to OpenFlow,’’ in Proc.
M. Kantor, and T. Engel, ‘‘Cognition: A in OpenFlow networks: Trading routing ACM SIGCOMM Conf., 2013, pp. 541–542.
tool for reinforcing security in software for better efficiency,’’ in Proc. 3rd Workshop [356] S. Tarkoma, C. E. Rothenberg, and
defined networks,’’ in EVOLVEVA Bridge Hot Topics Softw. Defined Netw., 2014, E. Lagerspetz, ‘‘Theory and practice of
between Probability, Set Oriented Numerics, pp. 127–132. bloom filters for distributed systems,’’
Evolutionary Computation V, vol. 288, [340] A. Schwabe and H. Karl, ‘‘Using MAC IEEE Commun. Surv. Tut., vol. 14, no. 1,
A.-A. Tantar et al., Ed. Berlin, Germany: addresses as efficient routing labels in data pp. 131–155, 2012.
Springer-Verlag, 2014, pp. 61–78. centers,’’ in Proc. 3rd Workshop Hot Topics [357] J. Kempf et al., ‘‘Scalable fault management
[325] R. Braga, E. Mota, and A. Passito, Softw. Defined Netw., 2014, pp. 115–120. for OpenFlow,’’ in Proc. IEEE Int. Conf.
‘‘Lightweight DDoS flooding attack detection [341] I. F. Akyildiz, A. Lee, P. Wang, M. Luo, and Commun., Jun. 2012, pp. 6606–6610.
using NOX/OpenFlow,’’ in Proc. IEEE 35th W. Chou, ‘‘A roadmap for traffic engineering [358] G. Bianchi, M. Bonola, G. Picierro,
Conf. Local Comput. Netw., Oct. 2010, in SDN-OpenFlow networks,’’ Comput. S. Pontarelli, and M. Monaci, ‘‘StreaMon:
pp. 408–415. Netw., vol. 71, pp. 1–30, Oct. 2014. A software-defined monitoring platform,’’
[326] G. Stabler, A. Rosen, S. Goasguen, and [342] H. Ballani, P. Francis, T. Cao, and J. Wang, in Proc. 26th ITC, Sep. 2014, pp. 1–6.
K.-C. Wang, ‘‘Elastic IP and security groups ‘‘Making routers last longer with Viaggre,’’ in [359] D. Kreutz, F. M. Ramos, and P. Verissimo,
implementation using OpenFlow,’’ in Proc. Proc. 6th USENIX Symp. Netw. Syst. Design ‘‘Towards secure and dependable
6th Int. Workshop Virtualization Technol. Implement., 2009, pp. 453–466. software-defined networks,’’ in Proc.
Distrib. Comput. Date, 2012, pp. 53–60.
[343] D. Meyer, L. Zhang, and K. Fall, ‘‘Report 2nd ACM SIGCOMM Workshop Hot Topics
[327] J. Matias, J. Garay, A. Mendiola, N. Toledo, from the IAB Workshop on Routing and Softw. Defined Netw., 2013, pp. 55–60.
and E. Jacob, ‘‘FlowNAC: Flow-based Addressing,’’ Internet Engineering Task [360] K. Kant, ‘‘Data center evolution: A
network access control,’’ in Proc. 3rd Eur. Force, RFC 4984 (Informational), Sep. 2007. tutorial on state of the art, issues,
Workshop Softw. Defined Netw., 2014, 6 pp. [Online]. Available: http://www.ietf.org/rfc/ challenges,’’ Comput. Netw., vol. 53, no. 17,
[328] K. Wang, Y. Qi, B. Yang, Y. Xue, and rfc4984.txt. pp. 2939–2965, 2009.
J. Li, ‘‘LiveSec: Towards effective security [344] M. Jarschel, F. Wamser, T. Hohn, [361] A. Greenberg, J. Hamilton, D. A. Maltz, and
management in large-scale production T. Zinner, and P. Tran-Gia, ‘‘SDN-based P. Patel, ‘‘The cost of a cloud: Research
networks,’’ in Proc. 32nd Int. Conf. application-aware networking on the problems in data center networks,’’
Distrib. Comput. Syst. Workshops, example of YouTube video streaming,’’ in SIGCOMM Comput. Commun. Rev., vol. 39,
Jun. 2012, pp. 451–460. Proc. 2nd Eur. Workshop Softw. Defined Netw., no. 1, pp. 68–73, Dec. 2008.
[329] A. Sapio et al., ‘‘MAPPER: A mobile Oct. 2013, pp. 87–92.
[362] M. Bari et al., ‘‘Data center network
application personal policy enforcement [345] T. G. Edwards and W. Belkin, ‘‘Using virtualization: A survey,’’ IEEE Commun.
router for enterprise networks,’’ in Proc. 3rd SDN to facilitate precisely timed actions Surv. Tut., vol. 15, no. 2, pp. 909–928, 2013.
Eur. Workshop Softw. Defined Netw., 2014, on real-time data streams,’’ in Proc. 3rd
2 pp. [363] A. Krishnamurthy, S. P. Chandrabose, and
Workshop Hot Topics Softw. Defined Netw.,
A. Gember-Jacobson, ‘‘Pratyaastha: An
[330] Y. Wang, Y. Zhang, V. Singh, C. Lumezanu, 2014, pp. 55–60.
efficient elastic distributed SDN control
and G. Jiang, ‘‘NetFuse: Short-circuiting [346] H. Kumar, H. H. Gharakheili, and plane,’’ in Proc. 3rd Workshop Hot Topics
traffic surges in the cloud,’’ in Proc. IEEE Int. V. Sivaraman, ‘‘User control of quality Softw. Defined Netw., 2014, pp. 133–138.
Conf. Commun., 2013, DOI: 10.1109/ICC. of experience in home networks using
2013.6655095. [364] P. Calyam et al., ‘‘Leveraging OpenFlow
SDN,’’ in Proc. IEEE Int. Conf. Adv. Netw.
for resource placement of virtual desktop
[331] J. H. Jafarian, E. Al-Shaer, and Q. Duan, Telecommun. Syst., 2013, DOI: 10.1109/
cloud applications,’’ in Proc. IFIP/IEEE
‘‘OpenFlow random host mutation: ANTS.2013.6802847.
Int. Symp. Integr. Netw. Manage., 2013,
Transparent moving target defense using [347] L. Li, Z. Mao, and J. Rexford, ‘‘Toward pp. 311–319.
software defined networking,’’ in Proc. 1st software-defined cellular networks,’’ in
Workshop Hot Topics Softw. Defined Netw., [365] J. Parraga, ‘‘Avior,’’ 2013. [Online]. Available:
Proc. Eur. Workshop Softw. Defined Netw.,
2012, pp. 127–132. http://openflow.marist. edu/avior.
2012, pp. 7–12.
[332] J. R. Ballard, I. Rae, and A. Akella, [366] GlobalNOC, ‘‘OESSVOpen Exchange
[348] X. Jin, L. Erran Li, L. Vanbever, and
‘‘Extensible and scalable network monitoring Software Suite,’’ 2013. [Online]. Available:
J. Rexford, ‘‘SoftCell: Scalable and flexible
using OpenSAFE,’’ in Proc. Internet Netw. http://globalnoc.iu.edu/sdn/oess.html.
cellular core network architecture,’’ in
Manage. Conf. Res. Enterprise Netw., 2010, Proc. 9th Int. Conf. Emerging Netw. Exp. [367] C. Duckett, ‘‘Software defined networking:
p. 8. Technol., 2013, pp. 163–174. HP has an App store for that,’’ 2013.
[333] A. Zaalouk, R. Khondoker, R. Marx, and [Online]. Available: http://www.zdnet.com/
[349] P. Dely, A. Kassler, and N. Bayer, ‘‘OpenFlow
K. Bayarou, ‘‘OrchSec: An orchestrator-based software-defined-networking-hp-has-an-
for wireless mesh networks,’’ in Proc. 20th
architecture for enhancing network-security app-store-for-that-7000021365/.
Int. Conf. Comput. Commun. Netw., 2011,
using network monitoring and SDN pp. 1–6. [368] Hewlett-Packard Company (HP), ‘‘SDN
control functions,’’ in Proc. IEEE Netw. app store,’’ 2013. [Online]. Available:
[350] M. J. Yang, S. Y. Lim, H. J. Park, and
Oper. Manage. Symp., May 2014, http://h17007.www1.hp.com/us/en/
N. H. Park, ‘‘Solving the data overload:
DOI: 10.1109/NOMS.2014.6838409. networking/solutions/technology/sdn/
Device-to-device bearer control architecture
[334] Flow-RT group, ‘‘sFlow-RT,’’ 2014. [Online]. devcenter/#sdnAppstore.
for cellular data offloading,’’ IEEE Veh.
Available: http://www. inmon.com/ Technol. Mag., vol. 8, no. 1, pp. 31–39, [369] B. H. Sigelman et al., ‘‘Dapper, a large-scale
products/sFlow-RT.php. Mar. 2013. distributed systems tracing infrastructure,’’
[335] D. Kotani, K. Suzuki, and H. Shimonishi, Google, Inc., Tech. Rep., 2010.
[351] K.-K. Yap et al., ‘‘Blueprint for introducing
‘‘A design and implementation of OpenFlow innovation into wireless mobile networks,’’ [370] L. Layman et al., ‘‘Debugging revisited:
controller handling ip multicast with fast in Proc. 2nd ACM SIGCOMM Workshop Toward understanding the debugging needs
of contemporary software developers,’’ in

Proc. ACM/IEEE Int. Symp. Empirical Softw. what your packet did last hop: Using packet [404] J. Teixeira et al., ‘‘Datacenter in a box:
Eng. Meas., Oct. 2013, pp. 383–392. histories to troubleshoot networks,’’ in Test your SDN cloud-datacenter controller
[371] U. Erlingsson, M. Peinado, S. Peter, Proc. 11th USENIX Symp. Netw. Syst. Design at home,’’ in Proc. 2nd Eur. Workshop Softw.
M. Budiu, and G. Mainar-Ruiz, ‘‘Fay: Implement., Apr. 2014, pp. 71–85. Defined Netw., Oct. 2013, pp. 99–104.
Extensible distributed tracing from kernels [387] Y. Zhang, N. Beheshti, and [405] P. Wette et al., ‘‘Maxinet: Distributed
to clusters,’’ ACM Trans. Comput. Syst., R. Manghirmalani, ‘‘NetRevert: Rollback emulation of software-defined networks,’’ in
vol. 30, no. 4, pp. 13:1–13:35, Nov. 2012. recovery in SDN,’’ in Proc. 3rd Workshop Proc. Netw. Conf., Jun. 2014, pp. 1–9.
[372] S. Tomaselli and O. Landsiedel, ‘‘Towards Hot Topics Softw. Defined Netw., 2014, [406] A. Roy, M. Bari, M. Zhani, R. Ahmed, and
lightweight logging and replay of embedded, pp. 231–232. R. Boutaba, ‘‘Design and management of
distributed systems,’’ in Proc. Workshop [388] J. S. Plank, M. Beck, G. Kingsley, and K. Li, DOT: A distributed OpenFlow testbed,’’ in
Architecting Safety Collaborative Mobile ‘‘Libckpt: Transparent checkpointing under Proc. Netw. Oper. Manage. Symp., May 2014,
Systems/32nd Int. Conf. Comput. Safety Reliab. unix,’’ in Proc. USENIX Tech. Conf., 1995, pp. 1–9.
Security, M. Roy, Ed., Toulouse, France, p. 18. [407] A. Carter et al., ‘‘CityFlow: OpenFlow city
Sep. 2013, 7 pp. [389] J. S. Plank, Y. Chen, K. Li, M. Beck, and experimentVLinking infrastructure and
[373] J. Tan, S. Kavulya, R. Gandhi, and G. Kingsley, ‘‘Memory exclusion: Optimizing applications,’’ in Proc. 3rd Eur. Workshop
P. Narasimhan, ‘‘Visual, log-based causal the performance of checkpointing Softw. Defined Netw., 2014, pp. 129–130.
tracing for performance debugging of map systems,’’ Softw., Practice Exp., vol. 29, no. 2, [408] ns-3 project, ‘‘OpenFlow switch support,’’
reduce systems,’’ in Proc. IEEE 30th Int. pp. 125–142, Feb. 1999. 2013. [Online]. Available: http://www.
Conf. Distrib. Comput. Syst., Jun. 2010, [390] N. Ruchansky and D. Proserpio, ‘‘A (not) nsnam.org/docs/release/3.13/models/html/
pp. 795–806. nice way to verify the OpenFlow switch openflow-switch.html.
[374] R. Fonseca, G. Porter, R. H. Katz, S. Shenker, specification: Formal modelling of the [409] J. Sommers et al., ‘‘Efficient network-wide
and I. Stoica, ‘‘X-trace: A pervasive network OpenFlow switch using alloy,’’ in Proc. flow record generation,’’ in Proc. IEEE
tracing framework,’’ in Proc. 4th USENIX ACM SIGCOMM Conf., 2013, pp. 527–528. INFOCOM, 2011, pp. 2363–2371.
Conf. Netw. Syst. Design Implement., 2007, [391] H. Zeng et al., ‘‘Libra: Divide and conquer to [410] ucb-sts, ‘‘STSVSDN troubleshooting
p. 20. verify forwarding tables in huge networks,’’ simulator,’’ 2013. [Online]. Available:
[375] V. Trivedi, ‘‘Software development: in Proc. 11th USENIX Symp. Netw. Syst. Design http://ucb-sts.github.io/sts/.
Debugging and testing,’’ in How to Speak Implement., Apr. 2014, pp. 87–99.
[411] H. Zhang et al., ‘‘Enabling layer 2 pathlet
Technology. New York, NY, USA: Apress, [392] R. Sherwood and K.-K. Yap, ‘‘Cbench tracing through context encoding in
2014, pp. 89–95. controller benchmarker,’’ 2011. [Online]. software-defined networking,’’ in Proc. 3rd
[376] A. Anand and A. Akella, ‘‘Netreplay: Available: http://www.openflow.org/wk/ Workshop Hot Topics Softw. Defined Netw.,
A new network primitive,’’ SIGMETRICS index.php/Oflops. 2014, pp. 169–174.
Perform. Eval. Rev., vol. 37, no. 3, pp. 14–19, [393] M. Jarschel, F. Lehrieder, Z. Magyari, and [412] K. Agarwal, E. Rozner, C. Dixon, and
Jan. 2010. R. Pries, ‘‘A flexible OpenFlow-controller J. Carter, ‘‘SDN traceroute: Tracing
[377] Y. Zhuang et al., ‘‘Netcheck: Network benchmark,’’ in Proc. Eur. Workshop SDN forwarding without changing
diagnoses from blackbox traces,’’ in Proc. Softw. Defined Netw., 2012, pp. 48–53. network behavior,’’ in Proc. 3rd Workshop
11th USENIX Symp. Netw. Syst. Design [394] Veryx Technologies, ‘‘PktBlaster SDN Hot Topics Softw. Defined Netw., 2014,
Implement., Apr. 2014, pp. 115–128. controller test.’’ [Online]. Available: pp. 145–150.
[378] N. Handigol, B. Heller, V. Jeyakumar, http://sdn.veryxtech.com/. [413] S. K. Fayaz and V. Sekar, ‘‘Testing stateful
D. Maziéres, and N. McKeown, [395] C. Rotsos, G. Antichi, M. Bruyere, and dynamic data planes with FlowTest,’’ in
‘‘Where is the debugger for my P. Owezarski, and A. W. Moore, ‘‘An Proc. 3rd Workshop Hot Topics Softw.
software-defined network?’’ in Proc. open testing framework for next-generation Defined Netw., 2014, pp. 79–84.
1st Workshop Hot Topics Softw. Defined OpenFlow switches,’’ in Proc. 3rd Eur. [414] M. Shahbaz et al., ‘‘Architecture for an open
Netw., 2012, pp. 55–60. Workshop Softw. Defined Netw., 2014, source network tester,’’ in Proc. ACM/IEEE
[379] A. Wundsam, D. Levin, S. Seetharaman, and pp. 127–128. Symp. Architect. Netw. Commun. Syst.,
A. Feldmann, ‘‘OFRewind: Enabling record [396] H. Hu, W. Han, G.-J. Ahn, and Z. Zhao, Oct. 2013, pp. 123–124.
and replay troubleshooting for networks,’’ in ‘‘FLOWGUARD: Building robust firewalls [415] N. Laurent, S. Vissicchio, and M. Canini,
Proc. USENIX Conf. USENIX Annu. Tech. for software-defined networks,’’ in Proc. 3rd ‘‘SDLoad: An extensible framework for SDN
Conf., 2011, p. 29. Workshop Hot Topics Softw. Defined Netw., workload generation,’’ in Proc. 3rd Workshop
[380] M. Canini, D. Venzano, P. Perešı́ni, 2014, pp. 97–102. Hot Topics Softw. Defined Netw., 2014,
D. Kostić, and J. Rexford, ‘‘A NICE way to [397] T. Ball et al., ‘‘Vericon: Towards verifying pp. 215–216.
test OpenFlow applications,’’ in Proc. 9th controller programs in software-defined [416] M. Gupta, J. Sommers, and P. Barford,
USENIX Conf. Netw. Syst. Design Implement., networks,’’ SIGPLAN Notes, vol. 49, no. 6, ‘‘Fast, accurate simulation for SDN
Apr. 2012, pp. 127–140. pp. 282–293, Jun. 2014. prototyping,’’ in Proc. 2nd ACM SIGCOMM
[381] C. Rotsos, N. Sarrar, S. Uhlig, R. Sherwood, [398] P. Kazemian, G. Varghese, and N. McKeown, Workshop Hot Topics Softw. Defined Netw.,
and A. W. Moore, ‘‘OFLOPS: An open ‘‘Header space analysis: Static checking 2013, pp. 31–36.
framework for OpenFlow switch evaluation,’’ for networks,’’ in Proc. 9th USENIX Conf. [417] R. R. Fontes, A. L. C. Oliveira, T. R. Pinheiro,
in Proc. 13th Int. Conf. Passive Active Meas., Netw. Syst. Design Implement., 2012, p. 9. P. N. Sampaio, and R. A. Figueira,
2012, pp. 85–95.
[399] H. Mai et al., ‘‘Debugging the data plane with ‘‘Authoring of OpenFlow networks with
[382] E. Al-Shaer and S. Al-Haj, ‘‘FlowChecker: anteater,’’ SIGCOMM Comput. Commun. Rev., visual network description (SDN version),’’
Configuration analysis and verification vol. 41, no. 4, pp. 290–301, Aug. 2011. in Proc. Summer Comput. Simul. Conf.,
of federated OpenFlow infrastructures,’’ in Monterey, CA, USA, 2014, pp. 22:1–22:6.
[400] P. Kazemian et al., ‘‘Real time network policy
Proc. 3rd ACM Workshop Assurable Usable
checking using header space analysis,’’ in [418] A. R. Curtis et al., ‘‘DevoFlow: Scaling
Security Config., 2010, pp. 37–44.
Proc. 10th USENIX Conf. Netw. Syst. Design flow management for high-performance
[383] A. Khurshid, W. Zhou, M. Caesar, and Implement., 2013, pp. 99–112. networks,’’ Comput. Commun. Rev., vol. 41,
P. B. Godfrey, ‘‘VeriFlow: Verifying no. 4, pp. 254–265, Aug. 2011.
[401] R. Beckett et al., ‘‘An assertion language for
network-wide invariants in real time,’’ in
debugging SDN applications,’’ in Proc. 3rd [419] C. J. Casey, A. Sutton, and A. Sprintson,
Proc. 1st Workshop Hot Topics Softw. Defined
Workshop Hot Topics Softw. Defined Netw., ‘‘tinyNBI: Distilling an API from essential
Netw., 2012, pp. 49–54.
2014, pp. 91–96. OpenFlow abstractions,’’ 2014. [Online].
[384] M. Kuzniar, M. Canini, and D. Kostic, Available: http://arxiv.org/abs/1403.6644.
[402] N. Handigol, B. Heller, V. Jeyakumar,
‘‘OFTEN testing OpenFlow networks,’’ in
B. Lantz, and N. McKeown, ‘‘Reproducible [420] L. Ogrodowczyk et al., ‘‘Hardware abstraction
Proc. 1st Eur. Workshop Softw. Defined Netw.,
network experiments using container-based layer for non-OpenFlow capable devices,’’ in
2012, pp. 54–60.
emulation,’’ in Proc. 8th Int. Conf. Emerging Proc. 30th Trans Eur. Res. Edu. Netw. Conf.,
[385] G. Altekar and I. Stoica, ‘‘Focus replay Netw. Exp. Technol., 2012, pp. 253–264. 2014, 8 pp.
debugging effort on the control plane,’’
[403] V. Antonenko and R. Smelyanskiy, [421] A. Vidal, C. E. Rothenberg, and
Electr. Eng. Comput. Sci., Univ. California
‘‘Global network modelling based on Mininet F. L. Verdi, ‘‘The libfluid OpenFlow
Berkeley, Berkeley, CA, USA, Tech. Rep.,
approach,’’ in Proc. 2nd ACM SIGCOMM driver implementation,’’ in Proc. 32nd Brazilian
May 2010.
Workshop Hot Topics Softw. Defined Netw., Symp. Comp. Netw. (SBRC), May 2014,
[386] N. Handigol, B. Heller, V. Jeyakumar, 2013, pp. 145–146. pp. 1029–1036.
D. Mazières, and N. McKeown, ‘‘I know

[422] M. Appelman and M. D. Boer, [438] G. Memon et al., ‘‘FlashFlow: A GPU-based [455] I. Tinnirello et al., ‘‘Wireless MAC
‘‘Performance analysis of open-flow fully programmable OpenFlow switch,’’ processors: Programming MAC protocols
hardware,’’ Univ. Amsterdam, Amsterdam, Univ. Oregon, Eugene, OR, USA, Tech. Rep., on commodity hardware,’’ in Proc. IEEE
The Netherlands, Tech. Rep., Feb. 2012. 2013. INFOCOM, Mar. 2012, pp. 1269–1277.
[423] K. Kannan and S. Banerjee, ‘‘Compact [439] Y. Luo, P. Cascon, E. Murray, and J. Ortega, [456] P. Bosshart et al., ‘‘Programming
TCAM: Flow entry compaction in TCAM ‘‘Accelerating Open-Flow switching with protocol-independent packet processors,’’
for power aware SDN,’’ in Distributed network processors,’’ in Proc. 5th ACM/IEEE 2013. [Online]. Available: http://arxiv.org/
Computing and Networking, vol. 7730, Symp. Architect. Netw. Commun. Syst., 2009, abs/1312.1719.
D. Frey, M. Raynal, S. Sarkar, pp. 70–71. [457] S. Schmid and J. Suomela, ‘‘Exploiting
R. Shyamasundar, and P. Sinha, Eds. [440] A. Rostami, T. Jungel, A. Koepsel, locality in distributed SDN control,’’ in Proc.
Berlin, Germany: Springer-Verlag, 2013, H. Woesner, and A. Wolisz, ‘‘Oran: 2nd ACM SIGCOMM Workshop Hot Topics
pp. 439–444. OpenFlow routers for academic networks,’’ Softw. Defined Netw., 2013, pp. 121–126.
[424] J. Liao, ‘‘SDN system performance,’’ in Proc. IEEE 13th Int. Conf. High Performance [458] P. Berde, ‘‘ONOS: Towards an open,
Jun. 2012. [Online]. Available: Switching Routing, 2012, pp. 216–222. distributed SDN OS,’’ in Proc. 3rd Workshop
http://pica8.org/blogs/?p=201. [441] G. Pongracz, L. Molnar, and Z. Kis, Hot Topics Softw. Defined Netw., 2014,
[425] B. Agrawal and T. Sherwood, ‘‘Modeling ‘‘Removing roadblocks from SDN: OpenFlow DOI: 10.1145/2620728.2620744.
TCAM power for next generation network software switch performance on Intel [459] D. M. Volpano, X. Sun, and G. G. Xie,
devices,’’ in Proc. IEEE Int. Symp. Performance DPDK,’’ in Proc. 2nd Eur. Workshop Softw. ‘‘Towards systematic detection and
Anal. Syst. Softw., 2006, pp. 120–129. Defined Netw., Oct. 2013, pp. 62–67. resolution of network control conflicts,’’ in
[426] B. Owens, ‘‘OpenFlow switching [442] B. Stephens, ‘‘Designing scalable networks Proc. 3rd Workshop Hot Topics Softw. Defined
performance: Not all TCAM is created for future large datacenters,’’ Ph.D. Netw., 2014, pp. 67–72.
equal,’’ Feb. 2013. [Online]. Available: dissertation, Dept. Comp., Rice Univ., [460] S. Azodolmolky, P. Wieder, and
http://packetpushers. net/openflow- Houston, TX, USA, May 2012. R. Yahyapour, ‘‘Performance evaluation
switching-performance-not-all-tcam-is- [443] Y. Li, D. Zhang, K. Huang, D. He, and of a scalable software-defined networking
created-equal/. W. Long, ‘‘A memory-efficient parallel deployment,’’ in Proc. 2nd Eur. Workshop
[427] B. Salisbury, ‘‘TCAMs and OpenFlowVWhat routing lookup model with fast updates,’’ Softw. Defined Netw., Oct. 2013, pp. 68–74.
every SDN practitioner must know,’’ Comput. Commun., vol. 38, pp. 60–71, [461] S. H. Park, B. Lee, J. Shin, and S. Yang,
Jul. 2012. [Online]. Available: http://www. Feb. 2014. ‘‘A high-performance IO engine for SDN
sdncentral.com/technology/sdn-openflow- [444] N. Katta, J. Rexford, and D. Walker, ‘‘Infinite controllers,’’ in Proc. 3rd Eur. Workshop
tcam-need-to-know/2012/07/. CacheFlow in software-defined networks,’’ Softw. Defined Netw., 2014, 2 pp.
[428] W. Braun and M. Menth, ‘‘Wildcard Princeton Schl. Eng. Appl. Sci., Princeton, [462] Y. Zhang, S. Natarajan, X. Huang,
compression of inter-domain routing NJ, USA, Tech. Rep., Oct. 2013. N. Beheshti, and R. Manghirmalani,
tables for OpenFlow-based software-defined [445] H. Mekky, F. Hao, S. Mukherjee, Z.-L. Zhang, ‘‘A compressive method for maintaining
networking,’’ in Proc. 3rd Eur. Workshop and T. Lakshman, ‘‘Application-aware data forwarding states in SDN controller,’’ in
Softw. Defined Netw., 2014, 6 pp. plane processing in SDN,’’ in Proc. 3rd Proc. 3rd Workshop Hot Topics Softw. Defined
[429] K. Agarwal, C. Dixon, E. Rozner, and Workshop Hot Topics Softw. Defined Netw., Netw., 2014, pp. 139–144.
J. Carter, ‘‘Shadow MACs: Scalable 2014, pp. 13–18. [463] S. H. Park et al., ‘‘RAON: Recursive
label-switching for commodity Ethernet,’’ in [446] M. Moshref, A. Bhargava, A. Gupta, M. Yu, abstraction of OpenFlow networks,’’ in
Proc. 3rd Workshop Hot Topics Softw. Defined and R. Govindan, ‘‘Flow-level state transition Proc. 3rd Eur. Workshop Softw. Defined Netw.,
Netw., 2014, pp. 157–162. as a new switch primitive for SDN,’’ in Proc. 2014, 2 pp.
[430] R. Rudell and A. Sangiovanni-Vincentelli, 3rd Workshop Hot Topics Softw. Defined Netw., [464] A. AuYoung et al., ‘‘Corybantic: Towards
‘‘Multiple-valued minimization for PLA 2014, pp. 61–66. the modular composition of SDN control
optimization,’’ IEEE Trans. Comput.-Aided [447] B. Yan, Y. Xu, H. Xing, K. Xi, and H. J. Chao, programs,’’ in Proc. 12th ACM Workshop Hot
Design Integr. Circuits Syst., vol. 6, no. 5, ‘‘CAB: A reactive wildcard rule caching Topics Netw., Nov. 2013, DOI: 10.1145/
pp. 727–750, Sep. 1987. system for software-defined networks,’’ in 2535771.2535795.
[431] R. Bifulco and M. Dusi, ‘‘Reactive logic Proc. 3rd Workshop Hot Topics Softw. Defined [465] P. Sun et al., ‘‘A network-state management
in software-defined networking: Accounting Netw., 2014, pp. 163–168. service,’’ in Proc. ACM Conf. SIGCOMM,
for the limitations of the switches,’’ in [448] S. McGillicuddy, ‘‘XPliant Ethernet chip sets 2014, pp. 563–574.
Proc. 3rd Eur. Workshop Softw. Defined Netw., new standard for programmability,’’ 2014. [466] D. Levin, A. Wundsam, B. Heller,
2014, 6 pp. [Online]. Available: http://goo.gl/xE8K9B. N. Handigol, and A. Feldmann, ‘‘Logically
[432] B. Stephens, A. Cox, W. Felter, C. Dixon, and [449] Intel Processors, ‘‘Software defined centralized? State distribution trade-offs
J. Carter, ‘‘Past: Scalable Ethernet for data networking and software-based services with in software defined networks,’’ in Proc. 1st
centers,’’ in Proc. 8th Int. Conf. Emerging Intel Processors,’’ 2012. [Online]. Available: Workshop Hot Topics Softw. Defined Netw.,
Netw. Exp. Technol., 2012, pp. 49–60. http://www.intel.com/content/dam/doc/ 2012, DOI: 10.1145/2342441.2342443.
[433] M. Kobayashi et al., ‘‘Maturing of OpenFlow white-paper/communications-ia-software- [467] D. Peleg, Distributed Computing: A
and software-defined networking through defined-networking-paper.pdf. Locality-Sensitive Approach. Philadelphia,
deployments,’’ Comput. Netw., vol. 61, Special [450] Intel Corporation, ‘‘Intel data plane PA, USA: SIAM, 2000.
Issue on Future Internet TestbedsVPart I, development kit,’’ 2014. [Online]. Available: [468] F. J. Ros and P. M. Ruiz, ‘‘Five nines of
pp. 151–175, 2014. http://www.intel.com/content/dam/www/ southbound reliability in software-defined
[434] J. C. Mogul and P. Congdon, ‘‘Hey, you public/us/en/documents/guides/intel-dpdk- networks,’’ in Proc. 3rd Workshop Hot Topics
darned counters! Get off my asic!’’ in Proc. 1st getting-started-guide.pdf. Softw. Defined Netw., 2014, pp. 31–36.
Workshop Hot Topics Softw. Defined Netw., [451] A. Sivaraman, K. Winstein, S. Subramanian, [469] V. Daniel Philip and Y. Gourhant,
2012, pp. 25–30. and H. Balakrishnan, ‘‘No silver bullet: ‘‘Cross-control: A scalable multi-topology
[435] P. Bosshart et al., ‘‘Forwarding Extending SDN to the data plane,’’ in fault restoration mechanism using logically
metamorphosis: Fast programmable Proc. 12th ACM Workshop Hot Topics Netw., centralized controllers,’’ in Proc. IEEE 15th
match-action processing in hardware Nov. 2013, DOI: 10.1145/2535771.2535796. Int. Conf. High Performance Switching Routing,
for SDN,’’ in Proc. ACM SIGCOMM Conf., [452] S. Zhou, W. Jiang, and V. Prasanna, Jul. 2014, pp. 57–63.
2013, pp. 99–110. ‘‘A programmable and scalable OpenFlow [470] M. Borokhovich, L. Schiff, and S. Schmid,
[436] O. Ferkouss et al., ‘‘A 100 gig network switch using heterogeneous SoC platforms,’’ ‘‘Provable data plane connectivity with
processor platform for openflow,’’ in Proc. in Proc. 3rd Workshop Hot Topics Softw. local fast failover: Introducing OpenFlow
7th Int. Conf. Netw. Service Manage., 2011, Defined Netw., 2014, pp. 239–240. graph algorithms,’’ in Proc. 3rd Workshop
pp. 1–4. [453] S. Hauger et al., ‘‘Packet processing at Hot Topics Softw. Defined Netw., 2014,
[437] J. Naous, D. Erickson, G. A. Covington, 100 Gbps and beyondVChallenges and pp. 121–126.
G. Appenzeller, and N. McKeown, perspectives,’’ in Proc. ITG Symp. Photon. [471] D. Kreutz, A. Casimiro, and M. Pasin,
‘‘Implementing an OpenFlow switch on the Netw., May 2009, pp. 1–10. ‘‘A trustworthy and resilient event broker
NetFPGA platform,’’ in Proc. 4th ACM/IEEE [454] G. Bianchi et al., ‘‘MAClets: Active MAC for monitoring cloud infrastructures,’’ in
Symp. Architect. Netw. Commun. Syst., protocols over hard-coded devices,’’ in Proc. Proc. 12th IFIP WG 6.1 DAIS, 2012,
2008, DOI: 10.1145/1477942.1477944. 8th Int. Conf. Emerging Netw. Exp. Technol., pp. 87–95.
2012, pp. 229–240.

[472] K. Tesink, ‘‘Definitions of managed objects Openflow based,’’ in Proc. 9th Int. Conf. barrage-of-cyberattacks-challenges-campus-
for the synchronous optical network/ Future Internet Technol., 2014, pp. 16:1–16:2. culture.html
synchronous digital hierarchy (SONET/ [488] T. Benson, A. Akella, and D. A. Maltz, [505] C. Tankard, ‘‘Advanced persistent threats
SDH) interface type,’’ Internet Engineering ‘‘Network traffic characteristics of data and how to monitor and deter them,’’ Netw.
Task Force, Internet Draft, Sep. 2003. centers in the wild,’’ in Proc. 10th ACM Security, vol. 2011, no. 8, pp. 16–19, 2011.
[Online]. Available: http://tools.ietf.org/ SIGCOMM Conf. Internet Meas., 2010, [506] S. Sorensen, ‘‘Security implications of
html/rfc3592. pp. 267–280. software-defined networks,’’ Fierce
[473] R. Prasanna, ‘‘BIP: Billing information [489] M. Yu, J. Rexford, M. J. Freedman, and Telecom, 2012. [Online]. Available: http://
protocol,’’ Internet Engineering Task J. Wang, ‘‘Scalable flow-based networking www.fiercetelecom.com/story/security-
Force, Internet Draft, Dec. 2002. [Online]. with difane,’’ SIGCOMM Comput. Commun. implications-software-defined-networks/
Available: http://tools.ietf.org/html/draft- Rev., vol. 40, no. 4, pp. 351–362, Aug. 2010. 2012-05-14.
prasanna-bip-00.
[490] M. F. Bari et al., ‘‘Dynamic controller [507] S. M. Kerner, ‘‘Is SDN secure?’’
[474] G. Swallow, A. Fulignoli, M. Vigoureux, provisioning in software defined networks,’’ Enterprise Networking Planet,
S. Boutros, and D. Ward, ‘‘MPLS fault in Proc. 9th Int. Conf. Netw. Service Manage., Mar. 2013. [Online]. Available: http://www.
management operations, administration, 2013, pp. 18–25. enterprisenetworkingplanet.com/netsecur/
maintenance (OAM),’’ Internet Engineering is-sdn-secure.html.
[491] V. Yazici, M. O. Sunay, and A. O. Ercan,
Task Force, RFC 6427 (Proposed Standard),
‘‘Controlling a software-defined network [508] A. Agapi et al., ‘‘Routers for the cloud: Can
Nov. 2011. [Online]. Available: http://
via distributed controllers,’’ in Proc. the internet achieve 5-nines availability?’’
www.ietf.org/rfc/rfc6427.txt.
Conf. Implement. Future Media Internet IEEE Internet Comput., vol. 15, no. 5,
[475] D. Whiting, R. Housley, and N. Ferguson, Towards New Horizons, Oct. 2012, pp. 16–22. pp. 72–77, Sep./Oct. 2011.
‘‘Counter with CBC-MAC (CCM),’’
[492] M. Jarschel et al., ‘‘Modeling and [509] R. Kloti, ‘‘OpenFlow: A security analysis,’’
Internet Engineering Task Force, RFC
performance evaluation of an OpenFlow M.S. thesis, Dept. Inf. Tech. Elec. Eng.,
3610 (Informational), Sep. 2003. [Online].
architecture,’’ in Proc. 23rd Int. Teletraffic Swiss Fed. Inst. Technol.
Available: http://www.ietf.org/rfc/
Congr., Sep. 2011, pp. 1–7. Zurich (ETH), Zurich, Switzerland, 2013.
rfc3610.txt.
[493] A. Bianco, R. Birke, L. Giraudo, and [510] M. Wasserman and S. Hartman, ‘‘Security
[476] M. Desai and T. Nandagopal, ‘‘Coping with
M. Palacin, ‘‘OpenFlow switching: Data analysis of the Open Networking Foundation
link failures in centralized control plane
plane performance,’’ in Proc. IEEE Int. (ONF) OpenFlow switch specification,’’
architectures,’’ in Proc. 2nd Int. Conf.
Conf. Commun., May 2010, DOI: 10.1109/ Internet Engineering Task Force, Apr. 2013.
Commun. Syst. Netw., 2010, DOI: 10.1109/
ICC.2010.5502016. [Online]. Available: https://datatracker.
COMSNETS.2010.5431977.
[494] R. Pries, M. Jarschel, and S. Goll, ‘‘On ietf.org/doc/draft-mrw-sdnsec-openflow-
[477] H. Kim et al., ‘‘Coronet: Fault tolerance for analysis/.
the usability of OpenFlow in data center
software defined networks,’’ in Proc. 20th
environments,’’ in Proc. IEEE Int. Conf. [511] S. Shin and G. Gu, ‘‘Attacking
IEEE Int. Conf. Network Protocols, Oct. 2012,
Commun., Jun. 2012, pp. 5533–5537. software-defined networks: A first feasibility
DOI: 10.1109/ICNP.2012.6459938.
[495] J. Hwang, K. K. Ramakrishnan, and T. Wood, study,’’ in Proc. 2nd Workshop Hot Topics
[478] S. Sharma, D. Staessens, D. Colle, Softw. Defined Netw., 2013, pp. 1–2.
‘‘NetVM: High performance and flexible
M. Pickavet, and P. Demeester,
networking using virtualization on [512] K. Benton, L. J. Camp, and C. Small,
‘‘OpenFlow: Meeting carrier-grade
commodity platforms,’’ in Proc. 11th ‘‘OpenFlow vulnerability assessment,’’ in
recovery requirements,’’ Comput. Commun.,
USENIX Symp. Netw. Syst. Design Implement., Proc. 2nd ACM SIGCOMM Workshop
vol. 36, no. 6, pp. 656–665, Mar. 2013.
Apr. 2014, pp. 445–458. Hot Topics Softw. Defined Netw., 2013,
[479] A. Panda, C. Scott, A. Ghodsi, T. Koponen, pp. 151–152.
[496] Y. Dong, Z. Yu, and G. Rose, ‘‘SR-IOV
and S. Shenker, ‘‘Cap for networks,’’ in Proc.
networking in Xen: Architecture, design [513] S. Scott-Hayward, G. O’Callaghan, and
2nd ACM SIGCOMM Workshop Hot Topics
and implementation,’’ in Proc. 1st Conf. S. Sezer, ‘‘SDN security: A survey,’’ in
I/O Virtualization, 2008, p. 10. Proc. IEEE SDN Future Netw. Services,
[480] M. Kuźniar, P. Perešı́ni, N. Vasić, M. Canini, Nov. 2013, DOI: 10.1109/SDN4FNS.2013.
[497] B. Heller, R. Sherwood, and N. McKeown,
and D. Kostić, ‘‘Automatic failure recovery 6702553.
‘‘The controller placement problem,’’ in
for software-defined networks,’’ in Proc. 2nd
Proc. 1st Workshop Hot Topics Softw. Defined [514] S. Sezer et al., ‘‘Are we ready for
ACM SIGCOMM Workshop Hot Topics Softw.
Netw., 2012, pp. 7–12. SDN? Implementation challenges for
Defined Netw., 2013, pp. 159–160.
[498] S. Azodolmolky et al., ‘‘An analytical model software-defined networks,’’ IEEE
[481] A. Dixit, F. Hao, S. Mukherjee, T. Lakshman, Commun. Mag., vol. 51, no. 7, pp. 36–43,
for software defined networking: A network
and R. Kompella, ‘‘Towards an elastic Jul. 2013.
calculus-based approach,’’ in Proc. IEEE
distributed SDN controller,’’ in Proc. 2nd
GlobeCom, Oct. 2013, DOI: 10.1109/ [515] S. Hernan, S. Lambert, T. Ostwald, and
ACM SIGCOMM Workshop Hot Topics Softw.
GLOCOM.2013.6831269. A. Shostack, ‘‘Uncover security design
Defined Netw., 2013, pp. 7–12.
[499] M. Marchetti, M. Colajanni, M. Messori, flaws using the STRIDE approach,’’ MSDN
[482] R. Ramos, M. Martinello, and Mag., Nov. 2006. [Online]. Available:
L. Aniello, and Y. Vigfusson, ‘‘Cyber attacks
C. Esteve Rothenberg, ‘‘SlickFlow: Resilient http://msdn.microsoft.com/en-us/magazine/
on financial critical infrastructures,’’ in
source routing in data center networks cc163519.aspx.
Collaborative Financial Infrastructure
unlocked by OpenFlow,’’ in Proc. IEEE 38th
Protection, R. Baldoni and G. Chockler, Eds. [516] B. Chandrasekaran and T. Benson,
Conf. Local Comput. Netw., Oct. 2013,
Berlin, Germany: Springer-Verlag, 2012, ‘‘Tolerating SDN application failures with
pp. 606–613.
pp. 53–82. LegoSDN,’’ in Proc. 3rd Workshop Hot Topics
[483] J. T. Araújo, R. Landa, R. G. Clegg, and Softw. Defined Netw., 2014, pp. 235–236.
[500] S. Amin and A. Giacomoni, ‘‘Smart grid,
G. Pavlou, ‘‘Software-defined network
safe grid,’’ IEEE Power Energy Mag., vol. 10, [517] W. J. Bolosky, D. Bradshaw,
support for transport resilience,’’ in Proc.
no. 1, pp. 33–40, Jan.-Feb. 2012. R. B. Haagens, N. P. Kusters, and P. Li,
IEEE Netw. Oper. Manage. Symp., 2014,
[501] A. Nicholson, S. Webber, S. Dyer, T. Patel, ‘‘Paxos replicated state machines as the basis
DOI: 10.1109/NOMS.2014.6838243.
and H. Janicke, ‘‘SCADA security in the of a high-performance data store,’’ in Proc.
[484] E. Brewer, ‘‘Pushing the cap: Strategies Symp. Netw. Syst. Design Implement., 2011,
light of cyber-warfare,’’ Comput. Security,
for consistency and availability,’’ Computer, pp. 141–154.
vol. 31, no. 4, pp. 418–436, 2012.
vol. 45, no. 2, pp. 23–29, Feb. 2012.
[502] K.-K. R. Choo, ‘‘The cyber threat landscape: [518] P. Sousa, A. Bessani, M. Correia, N. Neves,
[485] D. Katz and D. Ward, ‘‘Bidirectional and P. Verissimo, ‘‘Highly available
Challenges and future research directions,’’
forwarding detection (BFD),’’ Internet intrusion-tolerant services with
Comput. Security, vol. 30, no. 8, pp. 719–731,
Engineering Task Force, RFC 5880 proactive-reactive recovery,’’ IEEE
2011.
(Proposed Standard), Jun. 2010. [Online]. Trans. Parallel Distrib. Syst., vol. 21, no. 4,
Available: http://www.ietf.org/rfc/ [503] D. Kushner. (2013, Mar.). The real story
pp. 452–465, Apr. 2010.
rfc5880.txt. of Stuxnet. IEEE Spectrum. [Online]. 50(3),
pp. 48–53. Available: http://spectrum.ieee. [519] R. Chua, ‘‘SDN security: Oxymoron?
[486] N. L. M. van Adrichem, New interview with Phil Porras of SRI
org/telecom/security/the-real-story-of-stuxnet
B. J. van Asten, and F. A. Kuipers, International,’’ 2013. [Online]. Available:
‘‘Fast recovery in software-defined [504] R. Perez-Pena, ‘‘Universities face a rising
http://www.sdncentral. com/technology/
networks,’’ in Proc. 3rd Eur. Workshop barrage of cyberattacks,’’ New York Times,
sdn-security-oxymoron-phil-porras-sri/
Softw. Defined Netw., 2014, 6 pp. Jul. 2013. [Online]. Available: http://
2013/02/.
www.nytimes.com/2013/07/17/education/
[487] N. M. Sahri and K. Okamura, ‘‘Fast failover
mechanism for software defined networking:

[520] J. Korniak, ‘‘The GMPLS controlled optical Commun. Rev., vol. 44, no. 2, pp. 70–75, mobile network,’’ in Proc. Eur. Workshop
networks as industry communication Apr. 2014. Softw. Defined Netw., Oct. 2012, pp. 67–72.
platform,’’ IEEE Trans. Ind. Inf., vol. 7, no. 4, [536] P. Bernier, ‘‘NTT recognized with IBC [552] Open Networking Foundation (ONF),
pp. 671–678, Nov. 2011. award for SDN-based HDTV service,’’ ‘‘Operator network monetization
[521] P. Fonseca, R. Bennesby, E. Mota, and Sep. 2013. [Online]. Available: http:// through OpenFlow-enabled SDN,’’
A. Passito, ‘‘A replication component for www.sdnzone.com/topics/software- Apr. 2013. [Online]. Available: https://
resilient OpenFlow-based networking,’’ in defined-network/articles/353466-ntt- www.opennetworking.org/images/stories/
Proc. IEEE Netw. Oper. Manage. Symp., recognized-with-ibc-award-sdn-based- downloads/sdn-resources/solution-briefs/
Apr. 2012, pp. 933–939. hdtv.htm. sb-network-monetization.pdf.
[522] U. Toseef, A. Zaalouk, T. Rothe, [537] NTT DATA, ‘‘Infrastructure services,’’ 2014. [553] P. Skoldstrom and W. John,
M. Broadbent, and K. Pentikousis, [Online]. Available: http://www.nttdata. ‘‘Implementation and evaluation of a
‘‘C-BAS: Certificate-based AAA for SDN com/global/en/services/infrastructure/ carrier-grade OpenFlow virtualization
experimental facilities,’’ in Proc. 3rd Eur. solution.html. scheme,’’ in Proc. 2nd Eur. Workshop
Workshop Softw. Defined Netw., 2014, 6 pp. [538] M. Wagner, ‘‘NTT taps SDN to enhance Softw. Defined Netw., Oct. 2013, pp. 75–80.
[523] F. Klaedtke, G. O. Karame, R. Bifulco, and cloud flexibility,’’ Mar. 2014. [Online]. [554] H. H. Gharakheili and V. Sivaraman,
H. Cui, ‘‘Access control for SDN controllers,’’ Available: http://www.lightreading.com/ ‘‘Virtualizing national broad-band
in Proc. 3rd Workshop Hot Topics Softw. ntt-taps-sdn-to-enhance-cloud-flexibility/ access infrastructure,’’ in Proc. 9th Int.
Defined Netw., 2014, pp. 219–220. d/d-id/708133. Conf. Emerging Netw. Exp. Technol., 2013,
[524] D. Kreutz, A. Bessani, E. Feitosa, and [539] AT&T Inc., ‘‘AT&T introduces the pp. 27–30.
H. Cunha, ‘‘Towards secure and dependable ‘user-defined network cloud’: A vision [555] Pacnet Australia, ‘‘Pacnet offers first
authentication and authorization for the network of the future,’’ Feb. 2014. Pan-Asia network-as-a-service architecture,’’
infrastructures,’’ in Proc. Pacific Rim Int. [Online]. Available: http://www.att.com/ Nov. 2013. [Online]. Available: http://
Symp. Dependable Comput., Nov. 2014, gen/press-room?pid=25274&cdvn=news& www.cmo.com.au/mediareleases/17701/
pp. 43–52. newsarticleid=37439&mapcode=. pacnet-offers-first-pan-asia-network-
[525] D. Kreutz and E. Feitosa, ‘‘Identity [540] E. Haleplidis et al., ‘‘ForCES applicability as-a-service/.
providers-as-a-service built as to SDN-enhanced NFV,’’ in Proc. [556] R. Bennesby, P. Fonseca, E. Mota, and
cloud-of-clouds: Challenges and 3rd Eur. Workshop Softw. Defined Netw., A. Passito, ‘‘An inter-AS routing component
opportunities,’’ in Position Papers of the 2014, 6 pp. for software-defined networks,’’ in Proc. IEEE
2014 Federated Conference on Computer [541] Open Networking Foundation (ONF), Netw. Oper. Manage. Symp., Apr. 2012,
Science and Information Systems, vol. 3, ‘‘OpenFlow-enabled SDN and pp. 138–145.
M. P. M. Ganzha and L. Maciaszek, Eds. network functions virtualization,’’ [557] N. D. Corporation, ‘‘NTT DATA advance
Warsaw, Poland: PTI, 2014, pp. 101–108. Feb. 2014. [Online]. Available: https:// in SDN business provides highly-flexible
[526] P. Verssimo, N. Neves, and M. Correia, www.opennetworking.org/images/stories/ control of network by software,’’ Jun. 2012.
‘‘Intrusion-tolerant architectures: Concepts downloads/sdn-resources/solution-briefs/ [Online]. Available: http://www.nttdata.
and design,’’ in Architecting Dependable sb-sdn-nvf-solution.pdf. com/global/en/news-center/pressrelease/
Systems, vol. 2677, R. de Lemos, C. Gacek, [542] I. Cerrato et al., ‘‘User-specific network 2012/060801.html.
and A. Romanovsky, Eds. Berlin, Germany: service functions in an SDN-enabled [558] S. Das, A. Sharafat, G. Parulkar, and
Springer-Verlag, 2003, pp. 3–36. network node,’’ in Proc. 3rd Eur. Workshop N. McKeown, ‘‘MPLS with a simple OPEN
[527] C. E. Rothenberg et al., ‘‘Revisiting routing Softw. Defined Netw., 2014, 2 pp. control plane,’’ in Proc. Nat. Fiber Opt.
control platforms with the eyes and muscles [543] M. Xia, M. Shirazipour, Y. Zhang, H. Green, Eng. Conf. Opt. Fiber Commun. Conf. Expo.,
of software-defined networking,’’ in Proc. and A. Takacs, ‘‘SDN and optical flow 2011, pp. 1–3.
1st Workshop Hot Topics Softw. Defined Netw., steering for network function virtualization,’’ [559] M. Casado, T. Koponen, S. Shenker, and
2012, pp. 13–18. presented at the Open Networking Summit, A. Tootoonchian, ‘‘Fabric: A retrospective
[528] J. P. Stringer, Q. Fu, C. Lorier, R. Nelson, Santa Clara, CA, USA, 2014. on evolving SDN,’’ in Proc. 1st Workshop
and C. E. Rothenberg, ‘‘Cardigan: Deploying [544] E. Haleplidis, J. Hadi Salim, S. Denazis, and Hot Topics Softw. Defined Netw., 2012,
a distributed routing fabric,’’ in Proc. 2nd O. Koufopavlou, ‘‘Towards a network pp. 85–90.
ACM SIGCOMM Workshop Hot Topics Softw. abstraction model for SDN,’’ J. Netw. [560] M. Martinello, M. Ribeiro, R. de Oliveira,
Defined Netw., 2013, pp. 169–170. Syst. Manage., 2014, DOI: 10.1007/ and R. de Angelis Vitoi, ‘‘Keyflow: A
[529] C. E. Rothenberg et al., ‘‘Hybrid networking s10922-014-9319-3. prototype for evolving SDN toward core
towards a software defined era,’’ in Network [545] A. Gember-Jacobson et al., ‘‘OpenNF: network fabrics,’’ IEEE Network, vol. 28,
Innovation Through OpenFlow and SDN: Enabling innovation in network function no. 2, pp. 12–19, Mar. 2014.
Principles and Design Book. London, U.K.: control,’’ in Proc. ACM Conf. SIGCOMM, [561] N. Feamster et al., ‘‘SDX: A software-defined
Taylor & Francis/CRC Press, 2014. 2014, pp. 163–174. internet exchange,’’ in Proc. IETF, Orlando,
[530] D. Levin, M. Canini, S. Schmid, F. Schaffert, [546] I. Cerrato, M. Annarumma, and F. Risso, FL, USA, Mar. 2013. [Online]. Available:
and A. Feldmann, ‘‘Panopticon: Reaping ‘‘Supporting fine-grained network functions http://www.ietf.org/proceedings/86/slides/
the benefits of incremental SDN deployment through Intel DPDK,’’ in Proc. 3rd Eur. slides-86-sdnrg-6.
in enterprise networks,’’ in Proc. USENIX Workshop Softw. Defined Netw., 2014, 6 pp. [562] A. Devlic, W. John, and P. Skoldstrom,
Annu. Tech. Conf., Jun. 2014, pp. 333–345. ‘‘A use-case based analysis of network
[547] J. Ruckert, J. Blendin, N. Leymann,
[531] H. Lu et al., ‘‘Hybnet: Network manager G. Schyguda, and D. Hausheer, ‘‘Demo: management functions in the ONF SDN
for a hybrid network infrastructure,’’ in Software-defined network service chaining,’’ model,’’ in Proc. Eur. Workshop Softw.
Proc. Ind. Track 13th ACM/IFIP/USENIX in Proc. 3rd Eur. Workshop Softw. Defined Defined Netw., Oct. 2012, pp. 85–90.
Int. Middleware Conf., 2013, pp. 6:1–6:6. Netw., 2014, 2 pp. [563] M. Shirazipour, W. John, J. Kempf,
[532] A. Csoma et al., ‘‘Multi-layered service [548] J. Blendin, J. Ruckert, N. Leymann, H. Green, and M. Tatipamula, ‘‘Realizing
orches-tration in a multi-domain network G. Schyguda, and D. Hausheer, packet-optical integration with SDN and
environment,’’ in Proc. 3rd Eur. Workshop ‘‘Software-defined network service OpenFlow 1.1 extensions,’’ in Proc. IEEE
Softw. Defined Netw., 2014, 2 pp. chaining,’’ in Proc. 3rd Eur. Workshop Int. Conf. Commun., 2012, pp. 6633–6637.
[533] R. Morris, E. Kohler, J. Jannotti, and Softw. Defined Netw., 2014, 6 pp. [564] W. John et al., ‘‘SplitArchitecture: SDN for
M. F. Kaashoek, ‘‘The click modular [549] P. Skoldstrom et al., ‘‘Towards unified the carrier domain,’’ IEEE Commun. Mag.,
router,’’ SIGOPS Oper. Syst. Rev., vol. 33, programmability of cloud and carrier vol. 52, no. 10, pp. 146–152, Oct. 2014.
no. 5, pp. 217–231, Dec. 1999. infrastructure,’’ in Proc. 3rd Eur. Workshop [565] W. John et al., ‘‘Split architecture for
[534] S. Salsano et al., ‘‘OSHIVOpen source Softw. Defined Netw., 2014, 6 pp. large scale wide area networks,’’ Feb. 2014.
hybrid IP/SDN networking (and its [550] W. John et al., ‘‘Research directions in [566] C.-Y. Hong et al., ‘‘Achieving high utilization
emulation on Mininet and on distributed network service chaining,’’ in Proc. IEEE with software-driven WAN,’’ in Proc. ACM
SDN testbeds),’’ Apr. 2014. [Online]. SDN Future Netw. Services, Nov. 2013, SIGCOMM Conf., 2013, pp. 15–26.
Available: http://arxiv.org/abs/1404.4806. DOI: 10.1109/SDN4FNS.2013.6702549. [567] D. Staessens, S. Sharma, D. Colle,
[535] S. Vissicchio, L. Vanbever, and [551] B. Naudts et al., ‘‘Techno-economic M. Pickavet, and P. Demeester, ‘‘Software
O. Bonaventure, ‘‘Opportunities and analysis of software defined networking defined networking: Meeting carrier grade
research challenges of hybrid software as architecture for the virtualization of a requirements,’’ in Proc. 18th IEEE Workshop
defined networks,’’ SIGCOMM Comput.

Local Metropolitan Area Netw., Oct. 2011, http://www.telecomasia.net/content/ environments,’’ IBM J. Res. Develop., vol. 58,
DOI: 10.1109/LANMAN.2011.6076935. taking-sdn-transport-and-beyond. no. 2, pp. 1–12, Mar. 2014.
[568] S. Sharma, D. Staessens, D. Colle, [573] S. Elby, ‘‘Carrier vision of SDN, 2012. [578] W. Arnold et al., ‘‘Workload orchestration
M. Pickavet, and P. Demeester, ‘‘A [Online]. Available: http://www.brighttalk. and optimization for software defined
demonstration of automatic bootstrapping com/webcast/6985/58527. environments,’’ IBM J. Res. Develop.,
of resilient OpenFlow networks,’’ in Proc. [574] B. Anwer, T. Benson, N. Feamster, D. Levin, vol. 58, no. 2, pp. 1–12, Mar. 2014.
IFIP/IEEE Int. Symp. Integr. Netw. Manage., and J. Rexford, ‘‘A slick control plane for [579] C. Dixon et al., ‘‘Software defined
2013, pp. 1066–1067. network middleboxes,’’ in Proc. 2nd ACM networking to support the software
[569] R. Niranjan Mysore et al., ‘‘PortLand: A SIGCOMM Workshop Hot Topics Softw. defined environment,’’ IBM J. Res. Develop.,
scalable fault-tolerant layer 2 data center Defined Netw., 2013, pp. 147–148. vol. 58, no. 2, pp. 1–14, Mar. 2014.
network fabric,’’ SIGCOMM Comput. [575] C. Gerlach and H.-M. Foisel, ‘‘OIF carrier [580] IBM Systems and Technology Group,
Commun. Rev., vol. 39, no. 4, pp. 39–50, WG requirements on transport networks in ‘‘IBM software defined network for virtual
Aug. 2009. SDN architectures,’’ Optical Internetworking environments,’’ IBM Corporation, Tech.
[570] A. Greenberg et al., ‘‘VL2: A scalable and Forum (OIF), Tech. Rep., Sep. 2013. Rep., Jan. 2014.
flexible data center network,’’ in Proc. [576] L. Velasco, A. Asensio, J. Berral, A. Castro, [581] IBM Systems, ‘‘Manage all workloads with
ACM SIGCOMM Conf. Data Commun., and V. Lopez, ‘‘Towards a carrier SDN: an efficient, scalable software defined
2009, pp. 51–62. An example for elastic inter-datacenter environment (SDE),’’ 2014. [Online].
[571] A. Sadasivarao et al., ‘‘Bursting data between connectivity,’’ in Proc. 39th Eur. Conf. Exhibit. Available: http://www-03.ibm.com/systems/
data centers: Case for transport SDN,’’ in Opt. Commun., 2013, DOI: 10.1049/cp. infrastructure/us/en/software-defined-
Proc. IEEE 21st Annu. Symp. High-Performance 2013.1289. environment/.
Interconnects, 2013, pp. 87–90. [577] A. Alba et al., ‘‘Efficient and agile
[572] J. C. Tanner, ‘‘Taking SDN to transport storage management in software defined
and beyond,’’ 2013. [Online]. Available:
ABOUT THE AUTHORS

Diego Kreutz (Member, IEEE) received the Com- Institute of Engineering of Lisbon, Lisbon, Portugal; and at the University
puter Science degree, the M.Sc. degree in infor- of Aveiro, Aveiro, Portugal. Over the past 12 years he has taught 20+
matics, in 2009 and the M.Sc. degree in production courses: from physics (electromagnetism) to electrical engineering
engineering from the Federal University of Santa (digital electronics, electric circuits, telecommunication systems, and
Maria, Santa Maria, Brazil, in 2005. Currently, he is foundations) to computer science (operating and distributed systems,
working toward the Ph.D. degree in informatics computer networks, algorithms, programming languages). Periods
engineering at the Faculty of Sciences of Univer- outside academia include working as a Researcher at Portugal Telecom
sity of Lisbon, Lisbon, Portugal. and at Telefonica Research Barcelona. His current research interests
Over the past 11 years he has worked as an include software-defined networking, network virtualization, and cloud
Assistant Professor in the Lutheran University of computing, with security and dependability as an orthogonal concern.
Brazil, Canoas, RS, Brazil, and in the Federal University of Pampa,
Rio Grande do Sul, Brazil, and as a researcher member of the Software/
Hardware Integration Lab (LISHA), Federal University of Santa Catarina,
Florianópolis, Brazil. Out of the academia, he also has experience as an
independent technical consultant on network operations and manage-
ment for small and medium enterprises and government institutions. He
is involved in research projects related to intrusion tolerance, security,
and future networks including the TRONE and SecFuNet international
projects. His main research interests are in network control platforms,
software-defined networks, intrusion tolerance, system security and Paulo Esteves Verı́ssimo (Fellow, IEEE) is cur-
dependability, high-performance computing, and cloud computing. rently a Professor and FNR PEARL Chair at the
Faculty of Science, Technology and Communica-
tion (FSTC), University of Luxembourg (UL),
Luxembourg, Luxembourg; and head of the CritiX
research group at UL’s Interdisciplinary Centre
for Security, Reliability and Trust. He is currently
Fernando M. V. Ramos (Member, IEEE) received interested in secure and dependable distributed
the Ph.D. degree in computer science and engi- architectures, middleware and algorithms for:
neering from the University of Cambridge, resilience of large-scale systems and critical
Cambridge, U.K., in 2012. He received his Master infrastructures, privacy and integrity of highly sensitive data, and
of Science degree in telecommunications from adaptability and safety of real-time networked embedded systems. He is
Queen Mary University of London, London, U.K. the author of over 170 peer-refereed publications and coauthor of five
(with distinction, and best student award), in 2003, books.
and the ‘‘Licenciatura’’ degree in electronics and Prof. Verı́ssimo is a Fellow of the Association for Computing
telecommunications engineering (5 year under- Machinery (ACM). He is an Associate Editor of the International Journal
graduate course) from the University of Aveiro, on Critical Infrastructure Protection. He is Chair of the International
Aveiro, Portugal, in 2001. Federation for Information Processing (IFIP) Working Group (WG) 10.4 on
He is an Assistant Professor at the University of Lisbon, Lisbon, Dependable Computing and Fault-Tolerance and Vice-Chair of the
Portugal. His previous academic positions include those of Teaching Steering Committee of the IEEE/IFIP Dependable Systems and Networks
Assistant (Supervisor) at the University of Cambridge; at the Higher (DSN) Conference.

Christian Esteve Rothenberg (Member, IEEE) pursuing his Ph.D. degree. In August 2010, he joined the High Per-
received the Telecommunication Engineering de- formance Networks research group of the School of Computer Science
gree from the Universidad Politécnica de Madrid and Electronic Engineering (CSEE), University of Essex, Colchester, Essex,
(ETSIT-UPM), Madrid, Spain, the M.Sc. (Dipl. Ing.) U.K., as a Senior Research Officer. He has been the technical investigator
degree in electrical engineering and information of various national and European Union (EU)-funded projects. Software-
technology from the Darmstadt University of defined networking (SDN) has been one of his research interests since
Technology (TUD), Darmstadt, Germany, 2006, 2010, in which he has been investigating the extension of OpenFlow
and the Ph.D. degree in computer engineering toward its application in core transport (optical) networks. He has pub-
from the University of Campinas (UNICAMP), lished more than 50 scientific papers in international conferences, jour-
Campinas, Brazil, in 2010. nals, and books. One of his recent books is Software Defined Networking
He is an Assistant Professor in the Faculty of Electrical and Computer with OpenFlow (Birmingham, U.K.: Packt Publishing, 2013). Currently, he
Engineering, UNICAMP. From 2010 to 2013, he was a Senior Research is with Gesellschaft für Wissenschaftliche Datenverarbeitung mbH
Scientist in the areas of IP systems and networking at CPqD Research and Göttingen (GWDG), Göttigen, Germany, as a Senior Researcher and has
Development Center in Telecommunications, Campinas, Brazil, where he led SDN related activities since September 2012.
was technical lead of R&D activities in the field of OpenFlow/SDN such as Dr. Azodolmolky is a professional member of the Association for
the RouteFlow project, the OpenFlow 1.3 Ericsson/CPqD softswitch, or Computing Machinery (ACM).
the Open Networking Foundation (ONF) Driver competition. He holds two
international patents and has over 50 publications, including scientific
journals and top-tier networking conferences such as SIGCOMM and Steve Uhlig (Member, IEEE) received the Ph.D.
INFOCOM. Since April 2013, he has been an ONF Research Associate. degree in applied sciences from the University of
Louvain, Place de l’Université, Belgium, in 2004.
Siamak Azodolmolky (Senior Member, IEEE) From 2004 to 2006, he was a Postdoctoral
received the Computer Engineering degree from Fellow of the Belgian National Fund for Scientific
Tehran University, Tehran, Iran, in 1994, the M.Sc. Research (F.N.R.S.). His thesis won the annual IBM
degree in computer architecture from Azad Uni- Belgium/F.N.R.S. Computer Science Prize 2005.
versity in 1998, the M.Sc. degree, with distinction, Between 2004 and 2006, he was a Visiting
from Carnegie Mellon University, Pittsburgh, PA, Scientist at Intel Research, Cambridge, U.K., and
USA, in 2006, and the Ph.D. degree from the at the Applied Mathematics Department, Univer-
Universitat Politécnica de Catalunya (UPC), sity of Adelaide, Adelaide, S.A., Australia. Between 2006 and 2008, he
Barcelona, Spain, in 2011. was with Delft University of Technology, Delft, The Netherlands. He was a
He was employed by Data Processing Iran Co. Senior Research Scientist with the Technische Universität Berlin/
(IBM in Iran) as a Software Developer, Systems Engineer, and as a Senior Deutsche Telekom Laboratories, Berlin, Germany. Starting in January
R&D Engineer during 1992–2001. He joined Athens Information Technol- 2012, he became a Professor of Networks and Head of the Networks
ogy (AIT) as a Research Scientist and Software Developer in 2007, while Research group at the Queen Mary University of London, London, U.K.

SoftwareDefinedNetworking - A Comprehensive Survey (31-63)

Uploaded by

Copyright:

Available Formats

SoftwareDefinedNetworking - A Comprehensive Survey (31-63)

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

SoftwareDefinedNetworking - A Comprehensive Survey (31-63)

Uploaded by

Copyright:

Available Formats

Kreutz et al.

: Software-Defined Networking: A Comprehensive Survey

44 Proceedings of the IEEE | Vol. 103, No. 1, January 2015

Table 11 Debugging, Verification, and Simulation

Vol. 103, No. 1, January 2015 | Proceedings of the IEEE 45

46 Proceedings of the IEEE | Vol. 103, No. 1, January 2015

scalability of the system. By using MapReduce computa- V. ONGOING RESEARCH EFFORTS

Vol. 103, No. 1, January 2015 | Proceedings of the IEEE 47

48 Proceedings of the IEEE | Vol. 103, No. 1, January 2015

Vol. 103, No. 1, January 2015 | Proceedings of the IEEE 49

50 Proceedings of the IEEE | Vol. 103, No. 1, January 2015

Vol. 103, No. 1, January 2015 | Proceedings of the IEEE 51

52 Proceedings of the IEEE | Vol. 103, No. 1, January 2015

Table 12 Summary and Characterization of Scalability Proposals for SDNs

Vol. 103, No. 1, January 2015 | Proceedings of the IEEE 53

54 Proceedings of the IEEE | Vol. 103, No. 1, January 2015

bottlenecks for an OpenFlow switch-controller system be-

F. Security and Dependability

Vol. 103, No. 1, January 2015 | Proceedings of the IEEE 55

Table 14 Attacks to OpenFlow Networks

56 Proceedings of the IEEE | Vol. 103, No. 1, January 2015

Vol. 103, No. 1, January 2015 | Proceedings of the IEEE 57

58 Proceedings of the IEEE | Vol. 103, No. 1, January 2015

Vol. 103, No. 1, January 2015 | Proceedings of the IEEE 59

60 Proceedings of the IEEE | Vol. 103, No. 1, January 2015

Table 16 Carrier-Grade and Cloud Provider Expectations and Challenges

Vol. 103, No. 1, January 2015 | Proceedings of the IEEE 61

Table 17 SDE Pushing IT to the Next Frontier

Fig. 11. Overview of an IT infrastructure based on an SDE.

62 Proceedings of the IEEE | Vol. 103, No. 1, January 2015

Vol. 103, No. 1, January 2015 | Proceedings of the IEEE 63

64 Proceedings of the IEEE | Vol. 103, No. 1, January 2015

Vol. 103, No. 1, January 2015 | Proceedings of the IEEE 65

66 Proceedings of the IEEE | Vol. 103, No. 1, January 2015

Vol. 103, No. 1, January 2015 | Proceedings of the IEEE 67

68 Proceedings of the IEEE | Vol. 103, No. 1, January 2015

Vol. 103, No. 1, January 2015 | Proceedings of the IEEE 69

70 Proceedings of the IEEE | Vol. 103, No. 1, January 2015

Vol. 103, No. 1, January 2015 | Proceedings of the IEEE 71

72 Proceedings of the IEEE | Vol. 103, No. 1, January 2015

Vol. 103, No. 1, January 2015 | Proceedings of the IEEE 73

74 Proceedings of the IEEE | Vol. 103, No. 1, January 2015

ABOUT THE AUTHORS

Vol. 103, No. 1, January 2015 | Proceedings of the IEEE 75

76 Proceedings of the IEEE | Vol. 103, No. 1, January 2015

You might also like