Network Functions Virtualization: Challenges and

Opportunities for Innovations

Bo Han, Vijay Gopalakrishnan, Lusheng Ji and Seungjoon Lee
AT&T Labs Research
Bedminster, NJ 07921, USA

Typical Network Appliances NFV-Based Approach

AbstractNetwork Functions Virtualization (NFV) was re-
cently proposed to improve the flexibility of network service
provisioning and reduce the time to market of new services. Virtual
Appliances
By leveraging virtualization technologies and commercial off-the- Firewall CDN NAT
shelf programmable hardware, such as general purpose servers,
storage and switches, NFV decouples the software implementa- General Purpose
tion of network functions from the underlying hardware. As an DPI VPN IPTV Servers
emerging technology, NFV brings several challenges to network
operators, such as the guarantee of network performance for Standard Storage
virtual appliances, their dynamic instantiation and migration, Router PGW IMS and Switches
and their efficient placement. In this article, we provide a brief
overview of NFV, explain its requirements and architectural Fig. 1: From dedicated hardware-based appliances for network
framework, present several use cases and discuss the challenges
and future directions in this burgeoning research area. services, such as firewall, Content Delivery Network (CDN),
Network Address Translation (NAT), Deep Packet Inspection
Index TermsNetwork functions virtualization, virtual net- (DPI), Virtual Private Network (VPN), IPTV, router, Packet
work appliance, dynamic service provisioning, Cloud RAN, Cloud Data Network Gateway (PDN-GW or PGW) and IP Multime-
EPC.
dia Subsystem (IMS), to software-based NFV solutions.

I. I NTRODUCTION As an innovative step towards implementing a lower cost

agile network infrastructure, NFV can potentially bring sev-
It is well-known that bringing in new services into to-
eral benefits to network carriers, dramatically changing the
days networks is becoming increasingly difficult due to the
landscape of the telecommunications industry. It may reduce
proprietary nature of existing hardware appliances, the cost
capital investment and energy consumption by consolidating
of offering the space and energy for a variety of middle-
networking appliances [2], decrease the time to market of
boxes, and the lack of skilled professionals to integrate and
a new service by changing the typical innovation cycle of
maintain these services. Network Functions Virtualization was
network operators (e.g., through software-based service de-
recently proposed to alleviate these problems, along with other
ployment), and rapidly introduce targeted and tailored services
emerging technologies, such as Software Defined Networking
based on customer needs, just to list a few.
(SDN) and cloud computing.1
Along with the benefits of NFV, network operators also
NFV transforms how network operators architect their in- face several technical challenges when deploying virtual ap-
frastructure by leveraging the full-blown virtualization technol- pliances. A frequently raised issue about Virtualized Network
ogy to separate software instance from hardware platform and Functions (VNFs)2 is their network performance. Previous
by decoupling functionality from location for faster network- work has shown that virtualization may lead to abnormal
ing service provisioning [4]. Essentially, NFV implements latency variations and significant throughput instability even
network functions through software virtualization techniques when the underlying network is only lightly utilized [14].
and runs them on commodity hardware (i.e., industry standard Therefore, ensuring that network performance remains at least
servers, storage and switches), as shown in Figure 1. These as good as that of purpose-built hardware implementations will
virtual appliances can be instantiated on demand without the be one of the key challenges in realizing NFV. Besides the
installation of new equipment. For example, network operators network performance issue, another major problem network
may run an open-source software-based firewall in a Virtual carriers are confronted with is how to smoothly migrate from
Machine (VM) on an x86 platform. Recent trials have demon- the existing network infrastructure to NFV-based solutions,
strated that it is feasible to implement network functions on given the formers large scale and tight coupling among its
general purpose processor based platforms, for example, for components. Moreover, the separation of functionality from
physical layer signal processing [3]. location also casts the problem of how to efficiently place
1 We discuss the relationship between NFV, SDN and cloud computing in 2 A VNF is the software instance in NFV that consists of some number or
Section III. portion of VMs running different processes for a network function.
the virtual appliances and dynamically instantiate them on B. Manageability
demand. The NFV infrastructure should be able to instantiate VNFs
These facts all impose the need to investigate open research in the right locations at the right time, dynamically allo-
issues brought by NFV in order to ensure its successful cate and scale hardware resources for them and intercon-
adoption. However, there are very limited prior efforts in the nect them to achieve service chaining5 . This flexibility of
literature to offer an overview of aspects to be considered service provisioning poses new requirements to manage both
and issues to be addressed when adopting NFV. Our goal is virtual and legacy appliances. The manageability in NFV is
to bridge this gap by identifying critical research challenges quite different from that in data center networking where the
involved in the evolution towards NFV. hardware resources are almost equivalent, which makes their
In this article, we first present the key technical requirements coordination easier. However, the cost and value of resources
of NFV (Section II). We then introduce its architectural frame- may vary significantly between network points of presence and
work (Section III) and standardization activities (Section IV). customers premises. The management functionality should
We also describe several use cases of NFV, including the take the variations into account and optimize resource usage
virtualization of mobile base station, cellular core network and across the wide area.
home network (Section V). Finally, we discuss the open re- Since service unavailability is typically thought unaccept-
search issues and point out future directions for NFV, focusing able, network carriers usually over-provision their services [5]
on the network performance of virtualized appliances, their and thus the utilization of resources allocated to these ser-
efficient instantiation, placement and migration (Section VI). vices is normally low, due to the offered redundancy for
unexpected traffic increase or service element failure. If we
II. T ECHNICAL R EQUIREMENTS share cloud resources across multiple services and their failure
modes are independent, we can leverage the pool of spare
In this section, we summarize the technical requirements resources to provide the necessary redundancy across them
when implementing virtualized network functions, including and dynamically create VNFs to appropriately handle traffic
the network performance of VNFs, their manageability, relia- increase or failure. In addition, NFV can potentially improve
bility, security, and the coexistence with existing platforms. resource utilization through the elasticity feature of cloud
computing, for example, by consolidating the workload on a
small number of servers during overnight hours and turning
A. Performance the rest off (or using them for services such as online gaming).
When talking about software-based implementation of net- The management functionality should be able to support the
work functions through virtualization technologies on general sharing of spare resources and the elastic provisioning of
purpose servers, the first question we may ask is whether the network services effectively.
performance, such as throughput and latency, will be affected. Although NFV may make planned maintenance relatively
The per-instance capacity of a VNF may be less than the easy [15], it presents new requirements for service quality
corresponding physical version on dedicated hardware. management. Network operators should be able to obtain and
Although it is hard to completely avoid the performance process actionable information from various service impacting
degradation, we should keep it as small as possible while events, determine and correlate faults and recover from them,
not impacting the portability of VNFs on heterogeneous hard- by monitoring compute, storage and network resource usage
ware platforms. One possible solution is to leverage clustered during the life cycle of a VNF. Since VNFs can be dynam-
VNF instances and modern software technologies, such as ically created/migrated, it brings an additional dimension of
Linux New API (NAPI)3 and Intels Data Plane Development complexity in terms of keeping track of where a given VNF
Kit (DPDK)4 . When deploying VNF instances, we need to is running. Moreover, a VNF can behave erratically even if
design efficient algorithms to split network load across a the underlying infrastructure is running fine, which makes the
number of distributed and clustered VMs while keeping the detection of issues non-trivial.
latency requirement in mind. Moreover, the underlying NFV
C. Reliability and Stability
infrastructure should be able to gather network performance
information at different levels (e.g., hypervisor, virtual switch Reliability is an important requirement for network opera-
and network adapter). We discuss the research challenges tors when offering specific services (e.g., voice call and video
related to NFV performance in Section VI. on demand), no matter through physical or virtual network
A bottom line when designing NFV systems is that we appliances. Carriers need to guarantee that service reliability
should understand the maximum achievable performance of and service level agreement are not affected when evolving to
the underlying programmable hardware platforms. Based on NFV. Purpose-built network equipment can provide the tradi-
this information, we can make the proper design decisions. tional five-nines reliability in telecommunications industry. To
5 Service chaining describes a method for the delivery of network services
3 http://www.linuxfoundation.org/collaborate/workgroups/networking/napi
based on their function associations and enables the ordering and topological
4 http://dpdk.org/ independence of the network functions.
meet the same reliability requirement, NFV needs to build the Network operators should support a smooth migration path
resilience into software when moving to error-prone hardware from proprietary physical appliances to open standard based
platforms. Moreover, as we mentioned above, the elasticity virtual ones, since they may not be able to update all their
of service provisioning may require the consolidation and existing services and equipment to NFV-based solutions. The
migration of VNFs based on traffic load and user demand. developed NFV solutions need to be compatible with existing
All these operations create new points of failure that should Operation and Business Support Systems (OSS/BSS) and
be handled automatically. Element and Network Management Systems (EMS/NMS), and
In addition, ensuring service stability poses another chal- work in a hybrid environment with both physical and virtual
lenge to NFV, especially when reconfiguring or relocating a network functions.
large number of software-based virtual appliances from dif-
III. D ESIGN AND A RCHITECTURAL F RAMEWORK
ferent vendors and running on different hypervisors. Network
operators should be able to move VNF components from Virtualization provides us the opportunity for a flexible
one hardware platform onto a different platform while still software design. Existing networking services are supported
satisfying the service continuity requirement. They also need by diverse network functions that are connected in a static
to specify the values of several key performance indicators to way. NFV enables additional dynamic schemes to create
achieve service stability and continuity, including maximum and manage network functions. Its key concept is the VNF
non-intentional packet loss rate and call/session drop rate, forwarding graph which simplifies the service chain provi-
maximum per-flow delay and latency variation, and maximum sioning by quickly and inexpensively creating, modifying and
time to detect and recover from failures. removing service chains. On one hand, we can compose
several VNFs together to reduce management complexity, for
D. Security instance, by merging the serving gateway (SGW) and PGW
of a 4G core network into a single box. On the other hand,
When deploying virtualized network functions, operators we can decompose a VNF into smaller functional blocks for
need to make sure that the security features of their network reusability and faster response time. However, we note that
will not be affected. NFV may bring in new security concerns the actual carrier-grade deployment of VNF instances should
along with its benefits. The virtual appliances may run in be transparent to end-to-end services.
data centers that are not owned by network operators directly. Compared with the current practice, NFV introduces the
These virtualized network functions may even be outsourced following three major differences [12]:
to third parties [10]. The introduction of new elements, such as Separation of software from hardware: This separation
orchestrators and hypervisors, may generate additional security enables the software to evolve independently from the
vulnerabilities which increase the load of intrusion detection hardware, and vice versa.
systems. The underlying shared networking and storage can Flexible deployment of network functions: NFV can au-
also introduce new security threats, for example, when running tomatically deploy network-function software on a pool
a software router in a VM that shares the physical resources of hardware resources which may run different functions
with other network appliances. Moreover, these software-based at different times in different data centers.
components may be offered by different vendors, potentially Dynamic service provisioning: Network operators can
creating security holes due to integration complexity. All these scale the NFV performance dynamically and on a grow-
changes require us to rethink security issues when designing as-you-need basis with fine granularity control based on
and building NFV systems. the current network conditions.
NFV can also enhance the security level of a wide spectrum
We illustrate the high-level architectural framework of NFV
of networking services. The creation, management and adjust-
in Figure 2. Its four major functional blocks are the or-
ment of security zones become easier, since network operators
chestrator, VNF manager, virtualization layer and virtualized
can automate the placement of virtualized firewalls, create
infrastructure manager. The orchestrator is responsible for
dedicated software firewalls on-demand to protect specific
the management and orchestration of software resources and
network domains, and update the security rules of deployed
the virtualized hardware infrastructure to realize networking
firewalls remotely.
services. The VNF manager is in charge of the instantiation,
scaling, termination and update events during the lifecycle of a
E. Interoperability and Compatibility
VNF, and supports zero-touch automation. The virtualization
Another key issue for NFV is to design standard interfaces layer abstracts the physical resources and anchors the VNFs
between not only a range of virtual appliances but also these to the virtualized infrastructure. It ensures that the VNF life-
virtualized implementations and legacy equipment. As one of cycle is independent of the underlying hardware platforms by
the goals of NFV is to promote openness, network carriers may offering standardized interfaces. This type of functionality is
need to integrate and operate servers, hypervisors and virtual typically provided in the forms of VMs and their hypervisors.
appliances from different vendors in a multi-tenant NFV The virtualized infrastructure manager is used to virtualize
environment. Their seamless integration requires a unified and manage the configurable compute, network and storage
interface to facilitate the interoperability among them. resources and control their interaction with VNFs. It allocates
 Operation/Business Support Systems Orchestrator
to achieve the common architecture required to support virtu-
alized network functions through a consistent approach. This
ISG was initiated by several leading telecommunication carri-
VNF
VNF VNF VNF VNF
Manager ers, including AT&T, BT, China Mobile, Deutsche Telekom,
Orange, Telefonica and Verizon. It has quickly attracted broad
industry support and had over 150 members and participants
by the end of 2013, ranging from network operators to
Virtual Virtual Virtual
Computing Networking Storage
equipment vendors and IT vendors.
The ETSI NFV ISG currently has four working groups:
Virtualization Layer Virtualized Infrastructure Architecture, Management and Orchestration,
Infrastructure
Software Architecture and Reliability & Availability; and two
Manager
Compute Network Storage expert groups: Security and Performance & Portability. It
Shared Hardware Resources
has also developed a Proof of Concept (PoC) Framework to
coordinate multi-vendor PoCs and build the confidence that
Virtualized Infrastructure NFV is a viable technology. Although it is not a standards
development organization, it seeks to define the requirements
Fig. 2: NFV architectural framework [12]. that network operators may adopt and tailor for their com-
mercial deployment. Part of this article (e.g., the architectural
framework) is based on the NFV white paper [4] and several
VMs onto hypervisors and manages their network connectivity. related specifications [12], [13] published by this ISG.
It also analyzes the root cause of performance issues and
collects information about infrastructure fault and for capacity V. U SE C ASES
planning and optimization.
As we can see from this architectural framework, the two In this section, we describe several use cases of NFV,
major enablers of NFV are industry-standard servers and tech- including the virtualization of cellular base station, mobile
nologies developed for cloud computing. A common feature core network and home network. We focus on the problems of
of industry-standard servers is that their high volume makes existing architecture and the benefits of NFV-based solutions.
it easy to find interchangeable components inside them with NFV is applicable to both data plane processing and control
competitive price, compared with network appliances based plane function. We refer interested readers to the specification
on bespoke Application Specific Integrated Circuits (ASICs). of ETSI [13] for more use cases, such as the virtualization of
Using these general purpose servers can also reduce the num- CDN and fixed access network.
ber of different hardware architectures in operators networks
and prolong the lifecycle of hardware when technologies A. Virtualization of Cellular Base Station
evolve (e.g., running different software versions on the same The Radio Access Network (RAN) of traditional cellular
platform). Recent developments of cloud computing, such as networks is usually composed of stand-alone base stations
various hypervisors, OpenStack and Open vSwitch, also make which process and transmit wireless signal on behalf of
NFV achievable in reality. For example, the cloud management mobile phones and forward their data to the core network
and orchestration schemes enable the automatic instantiation through backhaul connections. This RAN architecture has
and migration of VMs running specific network services. several limitations. First, cellular operators provision their base
NFV is closely related to other emerging technologies, such stations to handle the maximum expected network load, but
as SDN. SDN is a networking technology that decouples the the traffic of a base station is fluctuating over time due to
control plane from the underlying data plane and consolidates usage pattern and user mobility. Therefore, the processing
the control functions into a logically centralized controller. power of base stations is usually not fully utilized. However,
NFV and SDN are mutually beneficial, highly complementary it is impossible to share the processing resources among them
to each other, and share the same feature of promoting because they are geographically dispersed. Second, given the
innovation, creativity, openness and competitiveness. These limited spectrum resource, base stations need to reuse radio
two solutions can be combined to create greater value. For frequency which makes the planning and optimization of base
example, SDN can support NFV to enhance its performance, station deployment hard, especially in urban areas. Third, base
facilitate its operation and simplify the compatibility with stations require their own backhaul transmission equipment,
legacy deployments. However, we emphasize that the virtu- environment surveillance system, cooling system and backup
alization and deployment of network functions do not rely on battery, which in turn need large space to host them.
SDN technologies, and vice versa. Over the years, the RAN architecture has evolved from
the all-in-one base stations to distributed base stations, which
IV. S TANDARDS R ELATED ACTIVITIES separate the radio function unit (a.k.a., Remote Radio Head
European Telecommunications Standards Institute (ETSI) or RRH) from the digital function unit (a.k.a., BaseBand Unit
has created an Industry Specification Group (ISG) for NFV or BBU). Baseband wireless signal is carried over fiber links
 BBU Pool PGW
PGW

IMS MME MME

IMS
SGW SGW

Legacy EPC
Optical Backhaul Network Cloud EPC

Cooperative RRHs

Fig. 4: Virtualization of EPC and its coexistence with legacy

EPC.

Fig. 3: The Cloud RAN architecture. B. Virtualization of Mobile Core Network

Todays mobile core networks suffer from a huge variety
of expensive and proprietary equipment, as well as from
between RRH and BBU which also makes their physical sepa- inflexible hard-state signaling protocols. When a specific
ration possible (e.g., by a few kilometers). In this architecture, function is not available, cellular operators have to replace
BBU implements the antenna array system functionality and an existing equipment even if it is still sufficient for most
the physical and MAC layers; while RRH obtains and converts purposes [7], which reveals the difficulty to scale up and down
the wireless signal and amplifies the power. offered services rapidly as required. Moreover, the mobile core
Distributed base stations have paved the way to further network leverages the tunneling mechanism over lower-layer
evolve the architecture to Cloud RAN by virtualizing BBUs in transport protocols to and from a few centralized gateways
data centers and thus enabling dynamic service provisioning. (PGWs in case of 4G EPC) for the delivery of user data traffic.
Cloud RAN leverages many advanced technologies, including The same is true for 2G and 3G networks. These long-distance
the common public radio interface in wireless communication, permanent tunnels are very expensive to control and maintain
the coarse/dense wavelength-division multiplexing in optical for cellular operators.
communication and the real-time virtualization in cloud com- Cloud EPC can potentially address these problems by
puting, as shown in Figure 3. The virtualization target is virtualizing the mobile core network to meet changing market
usually the BBU pool which typically runs in data centers. requirements. The virtualization targets of EPC include Mo-
Moreover, for the traditional RAN architecture, the virtualiza- bility Management Entity (MME), Home Subscriber Server
tion target can also be the evolved NodeB (eNodeB) in a 4G (HSS), SGW, PGW and Policy and Charging Rules Function
network, NodeB in a 3G network, or a legacy base station in (PCRF). To better support Voice over LTE (VoLTE), cellular
a 2G network (e.g., running a part of eNodeBs in VMs). operators can also virtualize the components of an IMS,
including various Call Session Control Functions (CSCFs)
The decoupling of baseband processing and radio units such as Proxy-CSCF, Serving-CSCF and Interrogating-CSCF,
can potentially bring various benefits. By centralizing and and Breakout and Media Gateway Control Functions. We
virtualizing the BBUs, Cloud RAN can significantly reduce the illustrate the virtualization of EPC for 4G LTE networks and its
operation, computing, energy and real-estate cost for cellular coexistence with the legacy EPC in Figure 4. The coexistence
carriers, thanks to easy software/firmware upgrades, fewer site is made possible through technologies such as MME pooling.
visits and lower site space leasing cost. For example, based on We note that it is possible to virtualize only part of the
the analysis of real-world data, Bhaumik et al. [2] reported that mobile core network, such as SGW and PGW, and use physical
Cloud RAN can reduce at least 22% computing resources by appliance for other components.
sharing the processing load among base stations and exploiting
By virtualizing the aforementioned network functions,
the load variations.
Cloud EPC allows us to move towards a more intelligent,
Cloud RAN can also enable advanced technologies such as resilient and scalable core architecture. It enables flexible
Coordinated MultiPoint (CoMP) in 4G Long Term Evolution distribution of hardware resources to eliminate performance
(LTE) networks and LTE-Advanced networks. CoMP dynam- bottlenecks and rapid launch of innovative services to generate
ically coordinates the transmission and reception between new revenue sources (e.g., machine-to-machine communica-
user equipment and multiple eNodeBs, and thus improves tions). The virtualization of EPC frees distributed network
reception performance, reduces interference levels and better resources from their geographic limitations to ensure service
utilizes network resources. It requires joint signal processing reliability and stability in the event of local resource failure and
for both uplink and downlink data and coordinated schedul- reduce the Total Cost of Ownership (TCO). It also makes the
ing/beamforming among base stations, which can be achieved flexible deployment of SGW and PGW possible, for example,
by the virtualized BBU pool. co-locating them with an eNodeB and thus eliminating the
 updating of the CPE devices and alleviating the call center
Internet and product return burdens. Second, it improves the quality
STB RG
of experience by offering near unlimited storage capacity and
enabling access to all services and shared content from differ-
ent locations and multiple devices, such as smartphones and
tablets. Third, it allows dynamic service quality management
and controlled sharing among user application streams which
helps content providers programmatically provision capacity
to end users via open APIs [11]. Finally, it introduces new
vSTB vRG
services more smoothly and less cumbersome by minimizing
the dependency on the CPE functions.
Fig. 5: Virtualization of home network. VI. R ESEARCH C HALLENGES AND F UTURE D IRECTIONS

long-distance tunnels. With Cloud EPC, cellular carriers can In this section, we discuss some of the research challenges
not only expand their current horizontal market business, but and future directions for NFV, including the network per-
also capitalize on previously untouched vertical markets. formance of virtualization, the placement, instantiation and
migration of virtual appliances and the outsourcing of VNFs.
C. Virtualization of Home Network
A. Network Performance of VNF
Network service providers offer home services through
dedicated Customer Premise Equipment (CPE) supported by The recent effort from the telecommunications industry has
network-located backend systems. Typical CPE devices in- been centered on the software virtualization framework (e.g,.
clude Residential Gateways (RGs) for Internet access and management and orchestration). However, it is challenging to
Set-Top Boxes (STBs) for multimedia services. Under this offer guaranteed network performance for virtual appliances.
architecture, the delivery of time-shifted IPTV services is Wang and Ng [14] measured the end-to-end networking per-
known to be complicated, due to the interactive stream control formance of the Amazon EC2 cloud service. They found that
functions (e.g., rewind and fast-forward) [1]. The emerging the sharing of processors may lead to very unstable TCP/UDP
NFV technology with the availability of high throughput last- throughput, fluctuating between zero and 1 Gbps at the tens of
mile access facilitates the virtualization of home network and milliseconds time granularity, and the delay variations among
brings down the complexity of IPTV services. Amazon EC2 instances can be 100 times larger than most
We depict the architecture of virtualized home networks propagation delays which are smaller than 0.2 ms, even when
in Figure 5. The virtualization targets are STBs and a range the network is not heavily loaded. The unstable networking
of components of RGs, such as firewall, DHCP server, VPN characteristics caused by virtualization can obviously affect
gateway and NAT router. By moving them to data centers, the performance and deployment of virtual appliances.
network and service operators need to provide only low As we mentioned in Section II, it may be possible to lever-
cost devices to customers for physical connectivity with low age Linux NAPI and Intels DPDK to improve the network
maintenance requirements, demonstrated by the three gray performance of VNFs. NAPI is a modification of the packet
boxes at the left hand bottom corner of Figure 5. These devices processing framework in Linux device drivers, aiming at im-
need to provide only the layer 2 functionality for Internet proving the performance of high-speed networking. It achieves
access, as the layer 3 and above functions of RGs are moved this goal by disabling some interrupts when the network traffic
into the operators network. We note that with this virtual load is high and switching to polling the devices instead, and
architecture, it is possible to share some functionalities of thus avoids frequent interruptions sharing the same message
RGs and STBs among customers. The concept of virtualizing that there are lots of packets to process. Another advantage
home network is not actually new. Multiple-System Operators of this polling-based approach is that when the kernel is
(MSOs) have been pushing the Cloud Digital Video Recorder overwhelmed, the packets that cannot be handled in time
(DVR) or Network DVR solutions for several years. Cloud are simply dropped in the device queues (i.e., overwritten in
DVR stores the recorded video programs at the MSOs central the incoming buffer). Intels DPDK is another software-based
location (e.g., the video hub office) instead of the consumers acceleration for high speed networking applications that also
home and relieves the storage requirement on STBs. 6 uses polling to avoid the overhead of interrupt processing.
This virtualized architecture presents numerous advantages Recent work by Hwang et al. [6] extends the DPDK libraries
to network operators and end users. First, it reduces the to provide low latency and high throughput networking in
operating expense by avoiding the constant maintenance and virtualized environments.

6 In terms of the legal considerations, a major MSO in the US won a B. Placement of Virtual Appliances
court battle against content providers regarding the technology of sharing
a stored program in the cloud among multiple users (http://en.wikipedia.org/ Ideally network operators should place VNFs where they
wiki/Cartoon Network, LP v. CSC Holdings, Inc.). will be used most effectively and least expensively. Although
the virtualization of certain network functions is straightfor- However, it is challenging to keep the packet forwarding
ward, there are a number of network functions that have strict uninterrupted and the migration disruptions minimized; while
delay requirements. For example, network functions offered at the same time guarantee the stringent throughput and latency
by middle-boxes usually depend on the network topology and requirements. Another interesting research topic is the design
these boxes are placed on the direct path between two end of a hypervisor [15] that splits the software of control plane
points. When virtualizing these functions and moving their from its state, such as routing information bases.
software implementations into data centers, data traffic may
go through indirect paths, causing a potential delay of packets. D. VNF Outsourcing
Therefore, the placement of VMs that carry VNFs is crucial The end-to-end principle of initial Internet architecture that
to the performance of offered services. For these services, does not modify packets on-the-fly is no longer valid in current
it would be advantageous and efficient to run some network networks with the deployment of a variety of middle-boxes.
functions at the edge of the network [8]. Based on a study of 57 enterprise networks with different sizes,
Using mobile core network as an example, we could place ranging from fewer than 1,000 hosts to more than 100,000
a PGW, which currently sits in the cellular core network, hosts, Sherry et al. [10] found that the number of middle-
right next to an eNodeB, and forward user traffic to the boxes in a typical enterprise is comparable to its number of
Internet as early as possible. However, the co-location of PGW hosted routers. In the last five years, surveyed large networks
and eNodeB will make the mobility management difficult, as had paid more than a million US dollars for their middle-
neighboring eNodeBs will no longer share the same PGW box equipment. Moreover, a network with about 100 middle-
as the anchor point. A possible solution would be to install boxes may need a management team of 100-500 personnel for
virtualized PGWs that handle traffic for a small geographical tasks such as configuration, upgrades, monitoring, diagnostics,
area at the Mobile Telephone Switching Office (MTSO) or training and vendor interaction [10].
some other network points of presence in the metro area. By advocating the split of network functions and their loca-
Future work regarding low latency operation should be based tions, NFV makes the outsourcing of middle-boxes to a third-
on the investigation of the redirection architecture and the party [10] easier, which may release network carriers from
carriers footprint of data centers. some of the cumbersome operation and maintenance tasks.
The placement of virtual appliances, such as VPN gateways, With the help of VNF Service Providers (e.g., cloud service
can also enhance the security features of networking services. providers or their partners), end users and small businesses
Todays VPN gateways are usually installed at locations very may also be able to enjoy more diverse networking services
deep into the core network. By moving virtualized VPN which are previously not affordable due to their associated
gateways to the network edge and closer to end users, we can complexity and costs. However, the charging rules and policy
better isolate VPN traffic from other Internet traffic and reduce interactions between carrier network infrastructure and out-
the complexity of core networks. Clearly this approach may sourced VNFs need to be carefully investigated before taking
lead to the support of more VPN gateways than the current actual actions. Another open question along this direction is to
practice. Thus, there is a need to optimize the number of identify what types of VNFs can be outsourced to third parties
instantiated virtual VPN gateways. and how to do it efficiently.
There are also several other open research issues for NFV.
C. Instantiation and Migration of Virtual Appliances For example, using dedicated hardware appliances, it is rel-
Network infrastructure will become more fluid when de- atively easy to identify which component is malfunctioning
ploying VNFs. To consolidate VNFs running in VMs based and isolate it when a failure occurs. When deploying network
on traffic demand, network operators need to instantiate and functions in software at different locations, troubleshooting
migrate virtual appliances dynamically and efficiently. The and fault isolation become harder. Moreover, as the creation
native solution of running VNFs in Linux or other commodity of VMs is easy, when the number of VNFs increases the so-
OS VMs has a slow instantiation time (around several seconds) called VM Sprawl could happen. There may be a large amount
and a relatively large memory footprint. The carrier-grade of VNFs sprawling across the network even if they are seldom
deployment of VNFs requires a lightweight VM implemen- used. As a result, the same management inefficiency problem
tation. For instance, Martins et al. [9] recently proposed that NFV was proposed to solve may recur. The efficient
ClickOS, a tiny Xen-based VM to facilitate NFV. ClickOS management and orchestration of VNFs, especially in the wide
can be instantiated within around 30 milliseconds and requires area, is another challenging issue.
about 5 MB memory when running. However, optimizing
the performance of this type of lightweight simplified VMs, VII. C ONCLUSION
especially during the wide-area migration, is still an open In this article, we presented an overview of the emerg-
research issue. ing network functions virtualization technology, illustrated
Take virtual routers as an example, by enabling their free its architectural framework, summarized several use cases
movement, carriers can separate the logical configurations and discussed some interesting future research directions.
(e.g., packet-forwarding functions) from physical routers, and NFV extracts the functionality in specialized appliances and
simplify management tasks, such as planned maintenance [15]. replicates it in the virtual form. It is envisioned that NFV,
along with cloud computing and SDN, will become a critical
enabling technology to radically revolutionize the way network
operators architect and monetize their infrastructure. NFV is
prospectively the unifying revolution among the three, offering
more revenue opportunities in the services value chain. We
are looking forward to more initiatives from the networking
research community to tackle various challenging issues intro-
duced by NFV and its widespread and successful adoption.
R EFERENCES
[1] V. Aggarwal, V. Gopalakrishnan, R. Jana, K. K. Ramakrishnan, and
V. A. Vaishampayan. Optimizing Cloud Resources for Delivering IPTV
Services Through Virtualization. IEEE Transactions on Multimedia,
15(4):789801, June 2013.
[2] S. Bhaumik, S. P. Chandrabose, M. K. Jataprolu, G. Kumar, A. Muralid-
har, P. Polakos, V. Srinivasan, and T. Woo. CloudIQ: A Framework for
Processing Base Stations in a Data Center. In Proceedings of MOBICOM
2012, pages 125136, Aug. 2012.
[3] China Mobile Research Institute. C-RAN The Road Towards Green
RAN. China Mobile White Paper, Oct. 2011.
[4] M. Chiosi et al. Network Functions Virtualisation: An Introduction,
Benefits, Enablers, Challenges & Call for Action. ETSI White Paper,
Oct. 2012.
[5] A. Greenberg, J. Hamilton, D. A. Maltz, and P. Patel. The Cost of a
Cloud: Research Problems in Data Center Networks. ACM SIGCOMM
Computer Communication Review, 39(1):6873, Jan. 2009.
[6] J. Hwang, K. K. Ramakrishnan, and T. Wood. NetVM: High Perfor-
mance and Flexible Networking Using Virtualization on Commodity
Platforms. In Proceedings of NSDI 2014, pages 445458, Apr. 2014.
[7] X. Jin, L. E. Li, L. Vanbever, and J. Rexford. SoftCell: Scalable
and Flexible Cellular Core Network Architecture. In Proceedings of
CoNEXT 2013, pages 163174, Dec. 2013.
[8] A. Manzalini, R. Minerva, F. Callegati, W. Cerroni, and A. Campi.
Clouds of Virtual Machines in Edge Networks. IEEE Communications
Magazine, 51(7):6370, July 2013.
[9] J. Martins, M. Ahmed, C. Raiciu, V. Olteanu, M. Honda, R. Bifulco,
and F. Huici. ClickOS and the Art of Network Function Virtualization.
In Proceedings of NSDI 2014, pages 459473, Apr. 2014.
[10] J. Sherry, S. Hasan, C. Scott, A. Krishnamurthy, S. Ratnasamy, and
V. Sekar. Making Middleboxes Someone Elses Problem: Network
Processing as a Cloud Service. In Proceedings of SIGCOMM 2012,
pages 1324, Aug. 2012.
[11] V. Sivaraman, T. Moors, H. H. Gharakheili, D. Ong, J. Matthews,
and C. Russell. Virtualizing the Access Network via Open APIs. In
Proceedings of CoNEXT 2013, pages 3142, Dec. 2013.
[12] The European Telecommunications Standards Institute. Network Func-
tions Virtualisation (NFV); Architectural Framework. GS NFV 002
(V1.1.1), Oct. 2013.
[13] The European Telecommunications Standards Institute. Network Func-
tions Virtualisation (NFV); Use Cases. GS NFV 001 (V1.1.1), Oct.
2013.
[14] G. Wang and T. S. E. Ng. The Impact of Virtualization on Network
Performance of Amazon EC2 Data Center. In Proceedings of INFOCOM
2010, pages 11631171, Mar. 2010.
[15] Y. Wang, E. Keller, B. Biskeborn, J. van der Merwe, and J. Rexford.
Virtual Routers on the Move: Live Router Migration as a Network-
Management Primitive. In Proceedings of SIGCOMM 2008, pages 231
242, Aug. 2008.