Overview On Network Orchestration

IEEE COMMUNICATIONS SURVEYS & TUTORIALS 1
Network Service Orchestration: A Survey

Nathan F. Saraiva de Sousa, Danny A. Lachos Perez, Raphael V. Rosa,
Mateus A. S. Santos, and Christian Esteve Rothenberg
Abstract—Business models of network service providers are need to model the end-to-end service and have the ability to
undergoing an evolving transformation fueled by vertical cus- abstract and automate the control of physical and virtual re-
tomer demands and technological advances such as 5G, Soft- sources delivering the service. The coordinated set of activities
ware Defined Networking (SDN), and Network Function Vir-
tualization (NFV). Emerging scenarios call for agile network behind such process is commonly referred to as orchestration.
services consuming network, storage, and compute resources In general, orchestration refers to the idea of automatically
across heterogeneous infrastructures and administrative domains. selecting and controlling multiple resources, services, and
arXiv:1803.06596v2 [cs.NI] 23 Mar 2018
Coordinating resource control and service creation across inter- systems to meet certain objectives (e.g. a customer requesting
connected domains and diverse technologies becomes a grand a specific network service). Altogether, the process shall be
challenge. Research and development efforts are being devoted
on enabling orchestration processes to automate, coordinate, and timely, consistent, secure, and lead to cost reduction due to
manage the deployment and operation of network services. In this automation and virtualization.
survey, we delve into the topic of Network Service Orchestration Multiple stakeholders are involved in the development and
(NSO) by reviewing the historical background, relevant research standardization of enabling technologies for network soft-
projects, enabling technologies, and standardization activities. We warization and their embodiment into next generation net-
define key concepts and propose a taxonomy of NSO approaches
and solutions to pave the way to the understanding of the works (e.g. 5G) based on SDN, NFV, and Orchestration build-
diverse ongoing efforts towards the realization of multiple NSO ing blocks and reference architectures. The ecosystem includes
application scenarios. Based on the analysis of the state of Standards Developing Organizations (SDOs), as well as indus-
affairs, we finalize by discussing a series of open challenges try groups, open source projects, foundations, diverse user-lead
and research opportunities, altogether contributing to a timely groups, and so on. Examples of these players include Euro-
and comprehensive survey on the vibrant and strategic topic of
network service orchestration. pean Telecommunications Standards Institute (ETSI), Metro
Ethernet Forum (MEF), Organization for the Advancement
Index Terms—Network Service Orchestration (NSO), SDN,
of Structured Information Standards (OASIS), Linux Foun-
NFV, multi-domain, orchestration, virtualization, lifecycle man-
agement. dation, and Open Networking Foundation (ONF). Similarly,
several (academic and industrial) research and (commercial)
development efforts in orchestration, SDN and NFV have been
I. I NTRODUCTION
going in recent years, concretized in a number of collaborative
T ELECOMMUNICATION infrastructures consist of a

myriad of technologies from specialized domains such as
radio, access, transport, core and (virtualized) data center net-
endeavors such as Open Source MANO (OSM) [5], Open-
Stack [6], Open Network Automation Platform (ONAP) [7],
5G-Exchange (5G-E X) [8], Central Office Re-architected as a
works. Designing, deploying and operating end-to-end services Datacenter (CORD) [9], etc.
are commonly manual and long processed performed via tradi- Unfortunately, broad understanding and practical definitions
tional Operation Support Systems (OSS) resulting in long lead of Network Service Orchestration (NSO) are still missing
times (weeks or months) until effective service delivery [1]. –not only across but also inside networking communities.
Moreover, the involved workflows are commonly hampered by The maturity of ongoing efforts varies largely with the
built-in hazards of infrastructures strongly coupled to physical overall technical approach being very much fragmented and
topologies and hardware-specific constraints. showing little consolidation around an overarching notion
Technological advances under the flags of Software Defined of network service orchestration. For instance, the work by
Networking (SDN) [2] and Network Function Virtualization Rotsos et al. [10] is the first notable attempt to survey the
(NFV) [3] bring new ways in which network operators can realm of network service orchestration. The authors provide an
create, deploy, and manage their services. SDN and NFV analysis of the diverse standardization activities around NSO
introduce new means for efficient and flexible utilization from an operator perspective. The article follows a top-down
of their infrastructures through a software-centric service approach, defining terminologies, requirements, and objectives
paradigm [4]. However, to realize this paradigm, there is a of a network service orchestrator.
This work is under submission for peer review. Until then, and even after In this survey, our main objectives are to provide a com-
an eventual publication, the authors are most welcome for any feedback to prehensive review of research, standardization and software
improve the work and turn the github and arxiv versions of this publication development efforts around the overcharged term of Network
a “living document” driven by community contributions as NSO evolves. Do
not hesitate to contact the authors and/or submit github pull requests or issues: Service Orchestration. We present an in-depth and up-to-date
https://github.com/intrig-unicamp/publications/tree/master/NSO-Survey. study on network service orchestration covering some history
N. Saraiva, D. Lachos, R. Rosa, and C. Rothenberg are with Department of and context, related enabling technologies, standardization ac-
Computer Engineering and Industrial Automation, Electrical and Computer
Engineering, University of Campinas (UNICAMP), Campinas, SP, Brazil. tivities, actual solutions, open challenges and research opportu-
M. Santos is with Ericsson Research, Indaiatuba, SP, Brazil. nities. In contrast to [10], we propose a view of NSO also from
Fig. 1. High-level reference model to illustrate the scope of Network Service Orchestration (NSO) in single-domain and multi-domain environment. The
NSO need to have an overview of entire environment to compose the service mainly if the service to use resources of different domains.
a customer perspective and propose a taxonomy of the main Survey Organization: We propose the best of our vision
characteristics and features of NSO approaches. We also cover on NSO in this work. The survey is organized as depicted in
how recent open source platforms and research projects map Figure 2. Section II presents essential background and key
to the primary characteristics and technical implementations technologies related to network service orchestration. Con-
to NSO realization. cepts, functions, scope, and a taxonomy of NSO are presented
Throughout the survey, we distinguish between two types of in Section III. Section IV focuses on the standardization efforts
domains. First, administrative domains, which map to different whereas Section V covers major research projects around
organizations and therefore may exist within a single service NSO. Section VI provides an overview of open source and
provider or cover a set of service providers. In one adminis- commercial solutions. Section VII presents some potential
trative domain, multiple technology domains can exist based scenarios to illustrate the NSO in practice. The discussion in
the type of technology in scope, for example, Cloud, SDN, Section VIII points to a series of open challenges and research
NFV, or Legacy. Broadly speaking, we refer to NSO as the opportunities. Finally, Section IX concludes the survey.
automated coordination of resources and services embracing
both single-domain and multi-domain contexts. II. BACKGROUND
Figure 1 presents a generic high-level reference model NSO foundations can be rooted back to three enabling
for multi-domain Network Service Orchestration, featuring a technologies, namely Cloud Computing, Software Defined
Multi-Domain Orchestrator (MDO) per administrative realm Networking, and Network Function Virtualization. This sec-
and including the notion of a Marketplace for business interaction presents a brief overview of these topics, as well as an
tions. MDOs coordinate resources and services in a multiple introduction to the historical background and definitions of
administrative domain scope covering multiple technology term “orchestration”.
domains [11]. The exchange of information, resources, and
services themselves are essential components of an end-to-
end network service delivery. The MDO exposes the available A. Cloud Computing
services to the marketplace allowing service providers to sell Cloud computing is a model for enabling ubiquitous, con-
network services directly to their customers or other providers venient, on-demand network access to a shared pool of con-
under various possible resources consumption models (e.g. figurable computing resources (e.g., networks, storage, and
trading resources from each other). The MDO can be seen services) that can be rapidly and automatically provisioned
as a single element with a possible split into two functional and released with minimal effort [12]. Thereby, the resources
components: Service Orchestrator (SO) and Resource Orches- are traded on demand, that is, the customer only pays what
trator (RO). The SO orchestrates high-level services while the to use. Cloud computing becomes one of relevant technology
RO is responsible for managing resource and orchestrating for the 5G networks mainly because it provides high data rate,
workflows across technology domains. The Domain Orchestra- high mobility, and centralized management [13].
tors (DOs) performs orchestration in each local domain acting The service models of cloud computing are generally cat-
on the underlying infrastructures and exposing resources and egorized into three classes: Software as a Service (SaaS),
network functions northbound to the MDO. Platform as a Service (PaaS), and Infrastructure as a Service
Fig. 2. Summarized view of this survey on NSO.
(IaaS). In a cloud IaaS, the infrastructure is offered as a SDN controller or Network Operating System (NOS). The
service to the customer. Each customer can have its virtual SDN controller creates an abstract network view while hiding
resources, such as compute, storage, and network. SaaS details of the underlying physical or virtual infrastructure.
includes applications such as Facebook, Google Apps, Twitter, Running on the top of the SDN controller, software network
and Microsot Office 365. applications can perform not only traditional functionalities
PaaS provides services according to a users applications such as routing, load balancing, classification [15], or In-
without installing or configuring the operating system. The trusion Detection Systems, but also propose novel use cases
customers can develop and deploy their applications in the such as service orchestration across multi-domain and multi-
same development environment. The PaaS model includes technology in 5G networks [16]. Those applications together
services such as Microsoft Azure, Google App Engine, RedHat with other industry and academy initiatives towards flexible
OpenShift, and Amazon Elastic Beanstalk. network services over programmable resources are among the
In SaaS, in turn, the customer is able to use the providers’ main drivers of SDN.
applications running on a cloud infrastructure [3]. The soft- The communication between the SDN controller and the
wares are maintained and managed by a cloud provider. IaaS forwarding devices is done through Southbound Interfaces
includes applications, for example, OpenStack, CloudStack, (SBI S), which allow decoupling the control and data plane
Amazon CloudFormation, and Google Compute Engine. via open communication protocols (i.e. well-defined APIs).
In a cloud environment, the notion of orchestration has also Different SDN SBI S can be considered (e.g., ForCES [17],
been used for integrating basic services [14]. The Orchestra- OVSDB [18], POF [19], etc.), with OpenFlow [20], [21] being
tion in the cloud involves dynamically deploying, managing the most widely accepted solution available in commercial and
and maintaining resource and services across multiple hetero- open source (hardware and software) devices.
geneous cloud platforms in order to meet the needs of clients. Service orchestrators, OSSes and other network applications
This procedure demands to automatize processes and create a can be developed on top of high-level Northbound Interfaces
workflow. However, this is not a simple task. (NBI S) offered by a SDN controller. Indeed, NBI S are crucial
components to control and monitor the network services
B. Software Defined Networking (SDN) orchestration. Unlike SBI S, where Openflow is a well-known
SDN [2] is an evolving networking paradigm that attempts SDN standard protocol, NBI S are still an open issue with
to resolve the strongly vertical integration of current network different controllers offering a variety of NBI S (e.g., RESTful
environments. To this end, SDN proposals decouple the con- APIs [22], NVP NBAPI [23], [24], SDMN API [25], etc.).
trol plane (i.e. control logic) from the data plane (i.e. data In addition, other type of high-level NBI S category are im-
forwarding equipments). With this new architecture, routers plemented as NOS management applications [10]. Examples
and switches become simple forwarding network elements of this category include Virtual Tenant Networks, ALTO, and
whose control logic is provided by an external entity called Intent-based networking (IBN).
The logically centralized SDN controller act in spirit of Within the scope of the ISG NFV [31], service chain is
computer operating systems that provide a high-level abstrac- defined as a graph of logical links connecting NFs towards
tion for the management of computer resources (e.g., hard describing traffic flow between these network functions. This
drive, CPU, memory) by playing the network operating system is equivalent to the Service Function Chaining (SFC) [32]
role for network management [26]. As such, it provides a set defined by Service Function Chaining Working Group (IETF
of services (base network services, management, orchestration) SFC WG) of the Internet Engineering Task Force (IETF). An
and common interfaces (North/South/East/West) to developers end-to-end network service may cover one or more Network
who can implement different control applications and improve Function Forwarding Graph (NF-FG) which interconnect NFs
manageability of networks. Moreover, such interfaces are and end points. Figure 3 describes two examples of end-to-end
used within the Management and Orchestration (MANO) network services. The first (green line) is composed of virtual
framework to deploy end-to-end connectivity. As today, the Customer Premises Equipment (V CPE) and virtual Firewall
most popular open source SDN controllers are Open Network (V FW) VNFs and two endpoints (A1 and A2). The second
Operating System (ONOS) [27] and OpendayLight [28]. (red line) is composed of V CPE and virtual Deep Packet
In SDN, the concept of orchestration is vital to automate Inspection (V DPI) VNFs and two endpoints (B1 and B2).
network operations properly. SDN network domains need Note that NFV allows sharing a multi-tenant VNFs between
single-domain or multi-domain orchestration systems to co- different network services.
ordinate end-to-end connectivity services through different ETSI has developed a reference architectural framework
network domains controlled by different SDN controller in- and specifications in support of NFV management and orches-
stances, which in turn must communicate directly with the tration. The framework focuses on the support VNF operation
physical network [29]. across different hypervisors and computing resources. It also
covers the orchestration and lifecycle management of physical
C. Network Function Virtualization (NFV) and virtual resources. According to [33], “the framework is
Traditionally, the telecommunication operators have based described at a functional level and it does not propose any
their networks on a built-in infrastructure strongly coupled specific implementation.” Figure 4 shows the ETSI NFV-
to physical topologies and proprietary devices, resulting in Management and Orchestration (MANO) architectural frame-
network services constrained to the network topology and the work with their main functional blocks [34]:
physical location of the network appliances. As a consequence,
• Operation/Business Support System (OSS/BSS): block
it becomes hard for providers to offer new services with lower
responsible for operation and business applications that
cost and more efficiency and agility [3]. Network Function
network service providers use to provision and operate
Virtualization has been proposed to solve these problems [30]
their network services. It is not tightly integrated into the
and change the mode networks are designed and operated by
NFV-MANO architecture.
taking a software-centric approached leveraging advances in
• Element Management (EM): component responsible for
virtualization technologies and general purpose processors.
the network management functions FCAPS (Fault, Con-
According to ETSI Industry Specification Group (ISG)
figuration, Accounting, Performance, and Security) of a
NFV [31], Network Function Virtualization is responsible
running VNF.
for separating network functions from the hardware and of-
• VNF: functional block representing the Virtualised Net-
fering them through virtualized services, decomposed into
work Function implemented on a physical server. For
Virtualized Network Function (VNF), on general purpose
instance, Router VNF, Switch VNF, Firewall etc.
servers. With the virtualization of the network functions,
• NFV Infrastructure (NFVI): representing all the hardware
NFV promises more flexible and faster network function
(compute, storage, and networking) and software compo-
deployment, as well as dynamic scaling of the VNFs towards
nents where VNFs are deployed, managed and executed.
providing finer settings. In NFV environment new services
• Network Function Virtualization Orchestrator (NFVO): it
do not require new hardware infrastructure, but simply the
is the primary component, in charge of the orchestration
software installation, i.e. to create VNFs.
of NFVI resources across multiple Virtualized Infras-
Moreover, the NFV can address Network Functions (NFs)
tructure Managers (VIMs) and lifecycle management of
in the most appropriate location, providing better user traffic
network services.
performance. The network service can be decomposed in one
• VNF Manager (VNFM): performs configuration and
or more VNFs, and each one can be constituted in one or
VNF lifecycle management (e.g., instantiation, update,
more Virtual Machines (VMs). Each VNF is described by
query, scaling, termination) on its domain.
a Virtualized Network Function Descriptor (VNFD) which
• VIM: block provides controlling and managing the NFVI
details the behavioral and deployment information of a VNF.
resources as well the interaction of a VNF with hardware
VNFs can be connected or combined as building blocks
resources. For example, OpenStack as cloud platform and
to offer a full-scale network communication service. This
OpenDaylight and ONOS as SDN controllers.
connection is known as service chain. Service chain provides
logical connectivity between the virtual devices of NFV ar- The NFV-MANO functional block performs all the
chitecture. It is worthwhile noting not only connectivity order virtualization-specific management, coordination, and automa-
importance, but also the logical environment interconnection tion tasks in the NFV architecture including the components
with physical networks. NFVO, VNFM, VIM, NFV Service, VNF Catalogue, NFV
in small part, after assigns to different musical instruments.

When, who, and what will be played, as well as the conducting
are essential parts towards achieving the desired effect.
One of the first works in the Information and Communi-
cation Technology (ICT) area that cite the term orchestration
[37] relate it with coordination and control of multiple media
traffics. It distinguishes the orchestration from synchronization
and defines an architecture where the orchestration acts in dif-
ferent layers. In the same scope, [38] relates the orchestration
with multimedia data. In this work, the orchestration is associ-
ated with multimedia presentation lifecycle management. The
lifecycle management involves the coordination of stages that
constitute all orchestration process.
The use of orchestration is also widely discussed in the
scope of web services. In this context, orchestration and
automation are considered separate processes. The work in
[39] defines orchestration like an executable process that
Fig. 3. Example of two end-to-end network services composed of two NFs
each. NFV enables the reuse of VNFs, e.g., V CPE. can interact both internal and external services and must be
dynamic, flexible, and adaptable the changes. It emphasizes
that orchestration describes how web services can act with
each other at the message level, including the business logic
and execution order of the activities.
More recently in 2009, [40] provides an overview of defini-
tion and design of Management and Service-aware Networking
Architectures (MANA) for the Future Internet (FI). One of the
pillars for the FI pointed by the article is Orchestration. In the
envisioned architecture, the orchestration function coordinates
the integrated behavior and operations to dynamically adapt
and optimize resources in response to changing context fol-
lowing business objectives and policies.
Orchestration is generally related to service automation in
cloud [41] and NFV environments. In spite of that, its concepts
are not clearly defined in the scope of NFV yet [42], [43].
Fig. 4. The NFV-MANO architectural framework. Adapted from [34]
E. Orchestration: Definitions
Instance, and NFVI Resource. Various communities differ with respect to the meaning,
The NFV-MANO reference architecture does not specify assumptions and scope of orchestration functions. Thus, it
anything about SDN in its architecture instead it assumes is helpful begin by reviewing the community understanding
that necessary transport infrastructure is already established to get the main concepts and significance. To this end, we
and ready to be used. However, in [35], ETSI identifies use overview the leading organizations and efforts defining the
cases and the most common options for using SDN in an term Orchestration in diverse contexts.
NFV architectural framework. It also points proof of concepts
A couple of years ago, the term orchestration was adopted
and recommendations to be fulfilled by the organizations that
by ETSI in the scope of NFV. In ETSI NFV, the meaning of
intend to perform such integration. [36] provides a recent in-
orchestration is implied, leading to a vague distinction between
depth survey on NFV state of affairs.
orchestration and management. Thus, its meaning may just
be inferred from the NFVO functions (both resources and
D. Orchestration: Historical Overview services layers). Similarly, the Internet Engineering Task Force
The academic community and industry generally require (IETF) comes up with an orchestration definition closely
some time to define the real meaning, reach and context of aligned with ETSI.
the concepts related to new technology trends as is the case National Institute of Standards and Technology (NIST) [44]
with the term Orchestration. The term orchestration is used was one of the first organizations to define the concept of
in many different areas, such as multimedia, music, service- Service Orchestration. According to NIST vision, orchestra-
oriented architecture, business processes, Cloud, SDN, and, tion is a process related to the arrangement, coordination, and
more recently, in NFV. management of virtualized infrastructure to provide different
From an end-user perspective, orchestration reminds a sym- cloud services to customers.
phony orchestra where a set of instruments play together The ONF [45] has formally defined orchestration as usage
according to an arrangement. The music is arranged and split and selection of resources by orchestrator for satisfying client
demands according to the service level. The meaning of the turn, is concerned with the execution of a workflow (processes)
orchestration is evident given a SDN Controller. ONF men- in a correct order. It controls the overall workflow process from
tioned that main functions of Orchestration are two-fold. First, starting the service until it ends. Its objective is to optimize
orchestration implies to split heavy-loaded service requests and automate the network service deployment.
into service components. Moreover, it distributes the afore- Figure 5 illustrates the relationship among the three terms
mentioned components among supported platforms, creating cited. There is a certain hierarchic between them. The orches-
an integrated end-to-end solution across multiple domains. tration is a high-level plane, below the management, and in
The ITU-T Recommendation Y.3300 [46] describes the the bottom the automation. In our vision, the orchestration de-
framework of software defined networking. This recommen- pends on tasks performed by management. Both management
dation defines that SDN functions are programming, orches- and orchestration are based on the use of automation in the
trating, controlling and managing network resources. Also, execution of their tasks. Nevertheless, some activities are only
it mentions that orchestration provides automated control performed by a certain function, optimization, for instance, can
and management of network resources. Nevertheless, ITU-T not be achieved by simple automation. The optimization is a
does not clarify the difference between SDN functions and responsibility of orchestration. There is a difference between
orchestration, what causes some confusion. them, but, if they work together in the execution of processes,
According to 3GPP Technical Specification 28.801 [47], the services deployments will succeed with further accuracy.
orchestration is responsible for interpreting and translating a
given service request into a configuration of resources (phys- III. N ETWORK S ERVICE O RCHESTRATION
ical and/or virtualized), as needed for service establishment.
The configuration of resources may use resource allocation A. Towards a Practical Definition
policies or actual available resources. We refer to the Network Service Orchestration (NSO) when
In the 5G white paper issued by NGMN [48], there is applied in the services deployment performed by telecommu-
an end-to-end management and orchestration entity which nication operators and service providers. We regard NSO not
composes the proposed architecture, and it is in charge of precisely as a unique technology but a concept to understand
translating the service request (business models) into infras- network services in detail, relying on multiple technologies
tructure resources, beyond managing tasks such as resource and paradigms to achieve such an overarching goal. In a nut-
scaling and network functions geographic distribution. It is shell, network service orchestration comprises the semantics
worthwhile noting this proposal is similar to the one presented of requested service, and thereby it coordinates specific actions
by ETSI NFVO. in order to fulfill the service requirements and to manage its
The MEF [49] proposes Lifecycle Service Orchestration end-to-end lifecycle.
(LSO) as a reference architecture for multi-domain orches- The entire orchestration process proposed by NSO involves
tration. LSO, based on network-as-a-service principles, ex- business and operations that go beyond the delivery of network
tends the NFV-MANO architecture and creates new capabil- services as defined by ETSI. ETSI NFV-MANO is a platform
ities. The orchestration of LSO refers to ”automated service for management and orchestration required to provisioning
management across multiple operator networks that include VNFs in an NFV domain. The MANO is agnostic and thus
fulfillment, control, performance, assurance, usage, security, has no insight of what is executed within a VNF, restricting
analytics, and policy capabilities.” its responsibility and capability to the VNF instantiation and
In addition to all the above-mentioned leading organizations, lifecycle management.
there are some works in the literature which also define Based on Figure 1, the MDO understands the operating
the concept of orchestration. According to [50], orchestration capabilities of the Network Service (NS) in a broad sense.
enables programmability for creating and deploying of end-to- When a customer demands an NS, firstly it requests the order
end network services and dynamic network control through a to a service provider or telecommunication operator through
single interface. Thyagaturu et al. [51] address orchestration as Business-to-Business (B2B) interface or a trading platform we
the coordination of network services and operations in a higher refer to as Marketplace. After that, the MDO interacts with
layer, abstracting the underlying physical infrastructure. The any MANO element or other elements (e.g., OSS/BSS, SDN
work in [52] makes a generic definition of orchestration as Controllers, Analytic Systems) to create the NS. Therefore,
automated management of complex systems and services. a given MANO does not know if the VNFs it is deploying
From the definitions of orchestration presented, we can is a load balancer, firewall, or gateway. Meanwhile, the DO
derive a clear relationship among orchestration, automation, just coordinates and manages the orchestration process at
and management. Although the three terms are lumped to- a given domain, connecting the involved elements such as
gether, it is necessary an understanding of the differences network systems, SDN controllers, management software, and
between them as they are not the same thing. Automation IT software platforms.
describes a simple and technical task without the human The NSO works at a higher level in the control and manage-
intervention, for example, launching a web server, stopping ment stack with interfaces to the OSS/BSS. During a network
a server. Management is responsible for maintaining and service creation, the orchestration process can exceed the
healthiness of infrastructure. Its role consists of activities such domains boundaries being necessary to use resources and/or
as alarms for event detection, monitoring, backups of critical services of other providers or operators. Such resources com-
systems, upgrades, and license management. Orchestration, in prised of physical and virtual components. Thus, the NSO is
Fig. 5. Relationship among orchestration, management, and automation. Both

orchestration and management use automation in their processes.
supposed to provide service delivery both within single and/or

multi-domain environments (more details in Section III-B).
In this sense, different organizations and telecommunication
enterprises have developed many open source projects, driving
orchestration evolution towards open standards that it will Fig. 6. Illustration of relationships between NSO, NFV, SDN, and Cloud.
permit the implementation of products with a large scale of
integration. Section VI addresses some of these projects.
In addition, the customers are demanding full information vide deployment of end-to-end service independently of
regarding a given hired network service such as detailed geographical location.
pricing, real-time analytics, and a certain control over the • Proper interaction with different MANO and non-MANO
service. NSO can offer more information to the customers and elements which leads to better executed workflows.
put more control into their hand. Its objective is to understand • New markets opportunities, offering enhanced services
the service profoundly and to enable that providers/operators and reducing OPEX.
attend customer demands.
From an operator and service provider viewpoint, NSO B. Single and Multi-Domain Orchestration
enables to set up new end-to-end services in minutes, keeping
Orchestration in the single and multi-domain environment is
those services working and ensuring acceptable performance
different. In a single domain, the orchestrator is in charge of all
levels. This process reduces OPEX and provides enhanced
services and resource availability within its domain as well as
services creating new market opportunities and raising the rev-
has total control over those resources. A domain orchestrator
enues. As well as, it opens up chances for different companies
manages the network service lifecycle and interacts with other
become service providers or provide virtual network functions.
components not only to control VNFs, but also computing,
NSO is in charge of all network service lifecycle and storage, and networking resources. Its scope is limited by
delivers an end-to-end connectivity service. To achieve so, or- administrative boundaries of the provider. As shown in Fig-
chestration is supported by advances in cloud computing, and ure 1, domain orchestrators can orchestrate heterogeneous
technologies such as SDN and NFV, which offer the ability technological domains such as SDN, NFV, Legacy, and Data
to reconfigure the network quickly as well as programming center. Under single domain environment, it is noticeable that
the forwarding and processing of the traffic. Figure 6 aims at the domain orchestrator works as described by ETSI in [34].
illustrating the relationships between NSO, NFV, SDN, and
However, in a multi-domain environment, local orches-
Cloud Computing.
trators do not know the resources and topologies used by
Each one of these paradigms has different functions: high other providers. So, multi-domain orchestration is more com-
level orchestration for NSO, function programming for NFV, plex, since it is supposed to provide end-to-end services,
networking programming for SDN, and resource virtualization which requires cross-domain information exchange features
for cloud computing. They can work in an integrated pattern (cf. [53]). Currently, there is not a standard for information
to offer advantages such as agility, cost reduction, automation, exchange process in multi-domain environments, either multi-
softwarization and end-to-end connectivity, to enable novel technology domains or multiple administrative domains. There
services and applications such as 5G networks. are some multi-domain orchestration candidates, e.g., T-NOVA
After this analysis, we can identify the main NSO features FP7 project [54], ONAP [7], Escape [55], and 5G-E X [16].
as follows: All of them will be discussed later in this survey.
• High-level vision of the NS that permit an overview of ETSI proposes some options regarding multi-domain or-
all involved domains, technological and administrative. chestration. Initially, ETSI NFV Release 2 presents two ar-
• Smart services deployment and provisioning. These are chitectures to address multi-domain scenarios [34]. In the first,
related to in-deep knowledge about the services, what the NFVO is split into Network Service Orchestrator, manages
enable better make decisions. the network service, and Resource Orchestrator, provides an
• Single and multi-domain environment support that pro- abstract resource present in the administrative domain. A use
case for this first option is illustrated in Figure 7(a). A Network tion [57]. Network service orchestration functions provided by
Operator offers resources to different departments within the the NFVO are listed below.
same operator, likewise to a different network operator. One • Management of Network Services templates and VNF
or more Data centers and VIMs represent an administrative Packages. This includes validation of templates and
domain and provide an abstracted view of its resources (virtual packages with the objective of verifying the artifacts’
and physical). The Service Orchestrator and VNF Manager authenticity and integrity. Besides, the software images
can or can not be part of another domain. In this use case, are cataloged in involved Points of Presence (P O Ps) using
a service can run on the infrastructure provided and managed the support of VIM.
by another Service Provider. • Network Service instantiation and management;
The second architecture does not split the NFVO, but • Management of the instantiation of VNFMs and VNFs
creates a new reference point between NFVOs (See Fig- (with support of VNFMs);
ure 7(b)) called Umbrella NFVO. This use case requires • Validation and authorization of NFVI resource requests
the composition of services towards deploying a high-level from VNF managers (resources that impact NS);
network service. Such service can include network services • Management of network service instances topology;
hosted and offered by different administrative domains. Each • Policy management related to affinity, scaling (auto or
domain is responsible for orchestrating its resources and manual), fault tolerance, performance, and topology.
network services. This approach has objectives similar to
ETSI NFVO functions regarding Resource Orchestration is
first, however, an administrative domain is also composed of
expressed as follows:
VNFMs (together with their related VNFs) and NFVO. The
• Validation and authorization of NFVI resource requests
NFVO provides standard NFVO functionalities, with a scope
limited to the network services, VNFs and resources that are from VNF Managers;
• NFVI resource management including compute, storage
part of its domain.
More recently, the ETSI NFV Release 3 presented others and network;
• Collect usage information of NFVI resources;
options to support network services across multiple admin-
istrative domains [56]. In particular, the use case entitled Related to NSO, the orchestrator, in turn, has a more
“Network Services provided using multiple administrative comprehensive function: to decouple the high-level service
domains” proposes a multi-domain architecture using NFV- layer (e.g., marketplace, network slicing) from underlying
MANO. Such architecture introduces the new reference point management and resources layer (e.g., VNFs, Controller,
named “Or-Or” between NFVOs to enable communication EMS), simplifying innovations and enhancing flexibility in
and interoperability. Differently of second option (Figure 7(b)), both contexts. The orchestrator allows complex functions to be
in this approach, there is a hierarchy between the domains. In implemented in underlying technologies and infrastructures.
the example shown in Figure 7(c), NFVO in Administrative For example, real-time analytics of network services can
Domain C is on-top, using network services offered by Ad- be deployed through the orchestrator. Another example, the
ministrative Domains A and B, as well as managing composite orchestrator can connect the traditional OSS/BSS to the virtu-
NS lifecycle. alized infrastructure. The Figure 8 represents the significance
In the scope of this paper, end-to-end network services are of the orchestrator in the context of network service.
composed of one or more network functions interconnected The orchestrator creates an abstraction unified point, en-
by forwarding graphs. Such services might span multiple abling to abstract physical and virtual resources, transparently
clouds and geographical locations. Given that, they require exposing them to service providers and other actors, including
complex workflow management, coordination, and synchro- marketplace and other orchestrators. It gives service providers
nization between multiple involved domains (infrastructure en- further control of their network services and enables develop-
tities), which are performed by one (or more) orchestrator(s). ers to create new services and functions.
Examples of end-to-end services are virtual extensible LAN To accomplish this, the orchestrator must be inserted in
(VxLAN), video service delivery, and virtual private network. each layer of telecommunication network stack, from the
application layer down to the data plane. Therefore, different
orchestrators can exist in each plane, not being limited to a
C. Orchestrator Functions single orchestrator [43]. Some of the existing orchestration
Section III-A identifies the various areas of term orches- solutions use an orchestrator logically centralized and consider
tration. Orchestration can be inserted in the context of cloud, only “softwarized” networks (see Section VI). However, this
NFV, management systems, web services and more recently is impracticable for large and heterogeneous networks.
in the deployment of end-to-end network service in large net- The orchestrator can be classified according to its function
works with multiple technologies and administrative domains. in: Service Orchestration (SO), Resource Orchestration (RO),
In this scope, the orchestrator is the component responsible and Lifecycle Orchestration (LO). Figure 9 illustrates the three
for automatic resource coordination and control, as well as primary network service orchestrator functions.
service provision to customers. The Service Orchestration is responsible for service com-
In the NFV context, ETSI NFV-MANO defines the or- position and decomposition. It can be taken as the upper
chestrator with two main functions including resources orches- layer, focused in the interaction with other components such
tration across multiple VIMs and network service orchestra- as Marketplace and OSS / Business Support Systems (BSS).
(a) (b) (c)
Fig. 7. ETSI approaches for multiple administrative domains: (a) approach in which the orchestrator is split into two components (NSO and RO), (b) approach
with multiple orchestrators and a new reference point: Umbrella NFVO, (c) approach that introduces hierarchy and the new reference point Or-Or. Adapted
from [34] and [56].
Fig. 8. Strategic role of the Orchestrator as the glue between the actual
services and the underlying management of resources. Fig. 9. Different orchestrator functions: Resource Orchestration, Service Or-
chestration, and Lifecycle Orchestration. There is a relationship of dependency
and continuity between the functions.
The Lifecycle Orchestration deals with the management of

workflows, processes, and dependencies across service compo- In order to achieve such state is necessary to execute four
nents. Besides, it maintains the services running according to tasks: create Virtual Deployment Unit (VDU)1, create VDU2,
the contracted Service Level Agreement. Finally, the Resource configure network and run the application. Therefore, the state
Orchestration is in charge of mapping service requests to only is finished when all those activities are completed.
resources, either virtual and/or physical. This mapping occurs Service lifecycle automation will allow that requested ser-
across elements such as NFVO, Element Management Sys- vice remains in a desired state of behavior during its life-
tems (EMS), and SDN controllers. time. With the automation, the system responds proactively
Lifecycle is used to manage a network service with various to changes network and service conditions without human
states (created, provisioned, stopped, etc.). When some action intervention, getting resilience and faults tolerance. These
is applied to a network service (e.g., provision a network functional aspects of an orchestrator to guarantee the state of
service), many activities may be needed to apply on the a network towards a service goal are also being referred to as
components of this network service. Hence, a workflow is Intent-based Networking (IBN), cf. [58].
used to execute a bunch of tasks in correct order. Each state
of lifecycle can generate one or more activities on workflows.
The Figure 10(a) depicts the relationship between lifecycle and D. Taxonomy
workflow of a Network Service. While many aspects of orchestration are under active de-
Figure 10(b) presents an example to improve the real velopment and commercial roll-outs, others are still in a pre-
definition of lifecycle and workflow in the context of network liminary maturity phase. This subsection enumerates central
service. One of the states in the service lifecycle is the Created. concepts and characteristics related to any NSO approach. It
(a) (b)
Fig. 10. Difference between Lifecycle and Workflow: (a) Lifecycle – sequence of states and workflow – activities in correct order and (b) example of network
service lifecycle.
becomes very challenging trying to summarize all concepts SDN and Network as a Service (NaaS) paradigms can be
related to orchestration in a single work, a challenge exacer- gathered to provide end-to-end service provisioning. While
bated by the fast evolving pace of so many moving pieces, SDN supply the orchestration of underlying network (switches,
from standards to enabling technologies. Figure 11 presents router, and links), the NaaS is responsible for private access
the proposed taxonomy as the result of extensive literature to the network and customer security [60].
research as well as practical experiences with a number of The NFV, in turn, can offer new services including NFVI
orchestration platforms and research projects. as a Service (NFVI), VNFaaS, S LaaS and Virtual Network
We identify seven key aspects to characterize network Platform as a Service (VNPaaS). The NFVIaaS provides
service orchestration: jointly IaaS and NaaS tailored for NFV. VNFaaS is a
service that implements virtualized Network Functions to the
1) Service Models. Relates to the type of services unlocked
Enterprises and/or end customers. VNPaaS is a platform
by the NSO, which may offer new business and relation-
available by service providers allowing customers to create
ships and opportunities (e.g., VNF as a Service (VNF),
their own network services. The S LaaS is a concept that the
Slice as a Service (S LaaS)).
slices are traded and used to build infrastructure services.
2) Software: Identifies major software-related characteris-
tics of the orchestration solutions, including specificities All these services can work in parallel to offer higher-level
of the management and standard interfaces. services. Each one acts in a specific area and offers features
3) Resource: Refers to the type of underlying resources to customers, enterprises, or other providers.
(e.g., network, compute, and storage) used for the net- 2) Software: There are many software artifacts related to
work service deployment. orchestration covering from a single cloud environment up
4) Technology: Points to target technologies for NSO (e.g., to more complex scenarios involving multi-domain orchestra-
Cloud, SDN, NFV, and Legacy). tion. These solutions are outcomes of open source initiatives,
5) Scope: Considers the application domain in terms of research projects or commercial vendors.
network segments embraced by the network service Open source approaches significantly accelerate consensus,
orchestration (i.e., from access network to data centers). delivering high performance, peer-reviewed code that forms
6) Architecture: Unfolds into three relevant architectural a basis for an ecosystem of solutions. Open source makes
dimensions with relate to single- and multi-domain it possible to create a single unified orchestration abstraction.
orchestration and functional organization. Thus, both research projects and commercial vendors leverage
7) SDO (Standards Development Organization): Relates to open source technologies to accelerate and improve their
standardization activities in scope of the NSO. solutions. Operators, such as Telefonica, China Mobile, AT&T,
Additional sub-areas contribute to an in-depth analysis in and NTT, appear committed to using open source as a way to
different contexts, which are further discussed in the following speed up their development of orchestration platforms [61].
sections. The Open Source Initiative (OSI)1 defines licenses un-
der Open Source Definition compliance, which allows code
1) Service Models: This aspect corresponds to the different
and software to be freely used, shared and modified. The
service models related to orchestration process. Each service
more popular open source licenses are Apache License 2.0,
is inserted in the context of cloud, SDN, and/or NFV. Cloud
Berkeley Software Distribution (BSD), GNU General Public
computing offers three categories of services such as IaaS,
License (GPL), Mozilla Public License 2.0, and Eclipse Public
PaaS and SaaS [59]. In IaaS, Cloud Service Provider (CSP)
License. Namely, the most important orchestration projects
renders a virtual infrastructure to the customers. In PaaS,
and frameworks (for instance, Aria, Cloudify, CORD, Gohan,
CSPs provide development environment as a service. Finally,
SaaS is a service that furnishes applications hosted and
managed in the cloud. 1 http://opensource.org
Fig. 11. NSO Taxonomy with seven approach: Service Model, Software, Resource, Technology, Scope of Application, Architecture, and Standards Standards
Developing Organization (SDO).
Open Baton, Tacker, ONAP, SONATA, and T-NOVA) present REST-based API. GUI, in turn, offers a graphic interface that
a widespread usage of Apache License 2.0. makes it easier its use.
Another topic related to open source is governance. In short, 3) Resource: During the creation of a network service,
governance defines the processes, structures, and organiza- the resource orchestration is responsible for orchestrating the
tions. It determines how power is exercised and distributed underlying infrastructure. Such infrastructure is composed of
and how decisions are taken. Commonly, a governing board heterogeneous hardware and software for hosting and con-
is responsible for the budget, trademark/legal, marketing, necting the network services. The resources include compute,
compliance, and overall direction, while a technical steering storage, and network [64]. In essence, there are three types of
committee is responsible for technical guidance. networks: packet, optical and wireless (e.g., Wi-Fi, wi-max,
An open source orchestration project may be organized and mobile network).
as a single community (e.g. vendor-lead) or can be hosted Resources are shared and abstracted making use of virtu-
(and eventually integrated with other projects) by a foundation alization techniques (e.g., para-virtualization [65], full virtu-
entity [62]. A remarkable example is the Linux Foundation, alization [66], and containers [67]), defining virtual infras-
which among multiple networking related projects is in charge tructures that can be used as physical ones. Sharing and
of ONAP, an open source platform aiming at the automation, management of resources are much more complex in multi-
design, orchestration, and management of SDN and NFV domain scenarios. The NSO needs to know all the available
services. Another noteworthy example of an orchestration open resources towards the efficient deployment of the NSs.
source project under the Linux Foundation flagship aiming at 4) Technology: NSO involves complex workflows and dif-
delivering a standard NFV/SDN platform for the industry is ferent technologies involved throughout orchestration process:
Open Platform for NFV (OPNFV) [63]. cloud computing, SDN, NFV, and legacy.
NSO solutions need to perform management tasks such as The cloud computing paradigm provides resource virtualiza-
remote device configuration, monitoring and fault manage- tion and improves resource availability and usage by means
ment. Moreover, they require defining interface of commu- of orchestration and management procedures. This includes
nication between various software components. For this, there automatic instantiation, migration and snapshot of VMs, High-
are diverse types of management and standard interfaces such Availability, and dynamic allocation of resources [30].
as Command Line Interface (CLI), Application Programming The SDN promotes control across network layers and
Interface (API), and Graphical User Interface (GUI). The CLI logical centralization of network infrastructure management.
just is used to execute commands directly in the software Its main functions is to connect the VNFs and the NFVI-
using remote access via SSH or Telnet. The API enables the P O Ps. In parallel, the NFV technology promotes the network
remote management and interconnection with other softwares functions programming in order to enable elasticity, automa-
through specifics commands. The majority of solutions use tion, and resilience in cloud environments [10]. As illustrated
in Figure 6, cloud computing, SDN and NFV are enabler request resource and services from each other on a peer-to-
technologies to the NSO. The NSO must also handle legacy peer fashion.
technologies such as MPLS, BGP, SONET / SDH, and WDM. Finally, functions,as discussed in Sec. III-C, refers to the
5) Scope: Resources of operators under an orchestration main tasks developed by network service orchestrator: service
application domain can be part of access networks, aggregation orchestration, resource orchestration, and lifecycle orchestra-
networks, core networks and data centers [11]. The access tion. These functions can be separated or together in the
network is the entry point which connects customers to their same component of an orchestration framework. This decision
service provider. It encompasses various technologies, i.e., depends on how the orchestrator was developed.
fixed access, wireless access (Wi-Fi, LTE, radio, WiMAX), 7) Standards Development Organization (SDO): Several
optical, and provide connectivity to heterogeneous services Standards Development Organizations, including ETSI, MEF,
such as mobile network and Internet of Things (I OT). The core IETF, and International Telecommunication Union (ITU), are
network is the central part of a telecommunications network actively working on a collection of standards in order to define
that connects local providers to each other. The aggregation reference architectures, protocols, and interfaces in scope of
network, in turn, connects the access network to core network. the orchestration domain. Besides, other organizations, aca-
The data center is the local where are localized the computing demic, vendors and industrial are working in parallel with
and storage resources. diverse goals. The main efforts within standardization bodies
The infrastructure is formed by heterogeneous technologies will be outlined next.
that may be owned by different infrastructure providers. The
network service orchestration in this environment is a chal- IV. NSO AND S TANDARDIZATION
lenging task. The NSO must have a view of resources and Interoperability and standardization constitute essential fac-
services since access network until the data center to deploy tors of the success of a network service orchestration solution.
end-to-end network services. Besides, it is also important to An important design goal for any new networking paradigm re-
provide consistent and continuous service, independent of the lates to openness of interfaces, especially in order to overcome
underlying infrastructure [11]. interoperability issues [10]. Several standardization efforts
6) Architecture: An NSO architecture can be divided into are delivering collections of norms and recommendations to
three sub-categories: (i) domain, (ii) organization, and (iii) define architectural guidelines and/or frameworks in addition
functions. The domain refers to coverage of the orchestration to standardized protocol extensions to enable NSO. This
process in one or more administrative domains: single-domain section presents the main standardization bodies at the NSO
and multi-domain. In each scenario, the orchestration has its scope. Table I presents a summary of the main SDOs and
peculiarities and challenges. organizations related to NSO standardization, as well as the
Single-domain orchestration studies focus on vertical main outcomes produced to date.
NFV/SDN orchestration within a same administrative domain.
In our definition, an administrative domain can have multiple A. ETSI
technological domains, such as SDN, NFV, and Legacy.
The taxonomy is aligned with ETSI NFV architecture that ETSI ISG NFV defines the MANO architectural frame-
addresses orchestration for NFV. The multi-domain orches- work to enable orchestration of VNFs on top of virtual-
tration involves the instantiation of network service among ized infrastructures. Since 2012, the group provides pre-
two or more administrative domains. It is composed of planes standardization and specification documents in different ar-
(or layers) with different functions and architecture topology. eas, including management and orchestration. NFVO takes a
The multi-domain interfaces are not present in original ETSI fundamental role in NFV-MANO functional components, as
NFV architecture defined in [57] realizing:
The organization refers to the different architectural ar- • the orchestration of infrastructure resources (including
rangements of a NSO solution. We identified three types multiple VIMs), fulfilling the Resource Orchestration
of organization: hierarchical, cascading and distributed. The functions
hierarchical approach assumes a high-level orchestrator that • and the management of Network Services, fulfilling the
has visibility of the entire other domains and capable of network service orchestration functions.
configuring services across different domains. The service Logically composing ETSI NFVO, NSO stipulates gen-
provider facing the customer as a single entry point will eral workflows on network services (e.g., scaling, topol-
maintain relationships with other providers to complete the ogy/performance management, automation), which conse-
requested service. According to [44], the hierarchical approach quently reach abstracted functionalities in other MANO com-
is impractical because of scalability and trust constraints. ponents – lifecycle management of VNFs in coordination with
Under the cascade model, the provider partially satisfies the VNFM and the consume of NFVI resources in accordance
service request but complements the service by using resources with VIM operational tasks.
from another provider. If this provider does not have all the Currently, ETSI matures NFV in different areas, such as
resources, it also can request for another and so on (e.g., a architecture, testing, evolution and ecosystem. Among ongoing
mobile network provider using a satellite provider). In the topics approached, network slicing report, multi-administrative
distributed model, there is not a central actor, and providers domain support [34], [56], and multi-site services(drafting
TABLE I
NSO S TANDARDIZATION O UTCOMES
SDO Working Group Scope Outcomes

Service Quality Metrics for NFV Orchestration [68]
NFV ISG (Initial) Management and Orchestration Framework [33]
Multiparty Administrative domains [69]
VNF Architecture and SDN in NFV Architecture [70]
Orchestration of virtualized resources [71]
NFV ISG (Release 2) Functional requirements for Orchestrator [71]
ETSI NFV
Lifecycle management of Network Services [71]
Network Service Templates Specification [72]
Policy management [73]
Report on architecture options to support multiple administrative
NFV ISG (Release 3)
domains [56]
Lifecycle Service Orchestration Vision [49]
MEF The Third Network NFV, LSO
LSO Reference Architecture and Framework [74]
SDN Architecture [75]
Architecture and
ONF SDN Mapping Orchestration Application to SDN [76]
Framework
Definition of Orchestration [77]
IETF SFC SFC, NFV SFC Architecture [32]
White Paper: Next Generation Networks [78]
NGMN Work Programme 5G Network
5G Network and Service Management and Orchestration [79]
ZOOM (Zero-touch Orchestration, Operations and Manage-
TM Forum Project SDN, NFV
ment) [80]
3GPP S5 5G Network (mobile) Management and orchestration for next generation network [47]
TOSCA version 1.0 [81]
Resource and Service
OASIS TOSCA TOSCA for NFV Version 1.0 [82]
Modeling
TOSCA in YAML Version 1.2 [83]
Report on Standards Gap Analysis in 5G Network [84]
Terms and definitions for 5G network [85]
5G Network (IMT-2020)
ITU-T SG 13 and network 5G Network management and orchestration requirements [86]
ITU softwarization 5G Network management and orchestration framework [87]
Standardization and open source activities related to network
softwarization [88]
ITU-R Mobile, radiodetermina- Framework and overall objectives of the 5G Network [89]
tion, amateur and related
satellite services
stage) highlight important aspects of evolving the NFV archi- fulfilling network service level specifications. Metro Ethernet
tectural framework, including possible new NSO functionali- Forum (MEF)’s LSO provides re-usable engineering specifi-
ties. In the upcoming years, ETSI is expected to keep playing cations to realize end-to-end automated and orchestrated con-
a driving role represents a path towards realization of concepts nectivity services through common information models, open
built upon the recommendations/reports, as attested bu open APIs, well-defined interface profiles, and attaining detailed
source projects such as OPNFV. business process flows. Therefore, in LSO Service Providers
orchestrate connectivity across all internal and external do-
mains from one or more network administrative domains.
B. MEF A detailed LSO reference architecture [74] presents com-
Metro Ethernet Forum (MEF)’s Third Network [49] ap- mon functional components and interfaces being exemplified
proaches NaaS comprising agility, assurance and orchestration in comparison with ETSI NFV framework and ONF SDN
as its main characteristics to broach LSO in their defined Car- architecture. Internally, a Service Orchestration Functionality
rier Ethernet 2.0. LSO, as a primer component, provides net- provides to LSO coordinated end-to-end management and
work service lifecycle management when approaching series control of Layer 2 and Layer 3 Connectivity Services realizing
of capabilities (e.g., control, performance, analytics) towards lifecycle management supporting different capabilities. Be-
sides, LSO defines APIs for essential functions such as service The list of protocols involved in NSO is by far not complete
ordering, configuration, fulfillment, assurance and billing. A and many new extensions to existing protocols and new ones
recent example of MEF’s use case conceptualization presents are expected due to the broadening needs of interoperable
an understanding of Software Defined Wide Area Network network service orchestration solutions.
(WAN) (WAN) managed services in face of LSO reference
architecture [90]. E. NGMN
Next Generation Mobile Networks (NGMN) in [79] pro-
C. ONF vides key requirements and high-level architecture principles
At ONF, the SDN architecture defines orchestration as TR- of Network and Service Management including Orchestration
521 [77] states: “In the sense of feedback control, orches- for 5G. Based on a series of user stories (e.g., slice creation,
tration is the defining characteristic of an SDN controller. real-time provisioning, 5G end-to-end service management),
Orchestration is the selection of resources to satisfy service the document establishes a common set of requirements.
demands in an optimal way, where the available resources, the Among them self-healing, scalability, testing and automation,
service demands, and the optimization criteria are all subject analysis, modeling, etc. Regarding orchestration functionali-
to change”. ties, the presented user stories introduce components (e.g.,
Logically, ONF perceives the SDN controller jointly over- SDN controllers and ETSI NFVO), which execute actions
seeing service and resource-oriented models to orchestrate to perform actors goals. For instance, slice creation would be
network services through intents on a client-server basis. From end-to-end service orchestration interpreting and translate ser-
top-to-bottom, a service-oriented perspective relates to invoca- vice definitions into a configuration of resources (virtualized
tion and management of a service-oriented API to establish or not) needed for service fulfillment.
one or more service contexts and to fulfill client’s requested As part of the initially envisioned 5G White Paper [78],
service attributes. Such requirements guide the SDN controller NGMN provided business models and use cases based on
in orchestrating and virtualizing underlying resources to build added values that 5G would bring for future mobile networks.
mappings that satisfy the network service abstraction and In general, SDN and NFV components are listed as enablers
realization. While in a bottom-up view, a resource-oriented for operational sustainability that will drive cost/energy effi-
model consists of SDN controller exposing underlying re- ciency, flexibility and scalability, operations awareness, among
source contexts so clients might query information and request other factors for simplified network deployment, operation,
services on top of them. In accordance, resource alterations and management. Such technology candidates highlight the
might imply in reallocation or exception handling of service importance of orchestration capabilities besides the evolution
behavior, which might be contained in policies specified by of radio access technologies towards 5G realization.
client’s specific attributes in a service request.
Recursively, stacks of SDN controllers might coordinate a F. TM Forum
hierarchy of network service requests into resource allocation
TeleManagement Forum (TM Forum) is a global association
according to their visibility and control of underlying techno-
for digital businesses (e.g., service providers, telecom opera-
logical and administrative network domains (e.g., Cross Stra-
tors, etc.) which provides industry best practices, standards and
tum Orchestration [76]). Thus, SDN controllers might have
proof-of-concepts for the operational management systems,
North-South and/or East/West relationships with each other. At
also known as Operations Support Systems (OSSs).
last, a common ground for orchestration concepts is published
One of the biggest TM Forum achievements is the definition
by ONF as “Orchestration: A More Holistic View” [75],
of a telecom business process (eTOM) and application (TAM)
elucidating considerations of its capabilities, among them,
maps, including all activities related to an operator, from
employing policy to guide decisions and resources feedback,
the services design to the runtime operation, considering
as well its analysis.
assurance, charging, and billing of the customer, among others.
In order to accommodate the SDN/NFV impacts, the TM
D. IETF Forum has created the Zero-Touch Orchestration, Operations
Different working and research groups at IETF address and Management (ZOOM) program, which intends to build
NSO from varying angles. Traffic Engineering Architecture more dynamic support systems, fostering service and business
and Signaling (TEAS) working group characterizes protocols, agility.
methods, interfaces and mechanisms for centralized (e.g., As a related research project, SELFNET is, on one side, ac-
PCE) and distributed path computation (e.g., MPLS, GMPLS) tively following and aligning its architecture definition with the
of traffic engineered paths/tunnels delivering specific network TM Forum ZOOM and FMO recommendations. Additionally,
metrics (e.g., throughput, latency). Application-based Network SELFNET, through one partner of the consortium that is an ac-
Operations (ABNO) proposes modular a modular control ar- tive member of TM Forum, is also going to actively contribute
chitecture, standardized by IETF aggregating already standard to the ZOOM working group with respect to the impact that the
components, such as Path Computation Element (PCE) to NFV/SDN paradigm has on the OSS information model (CFS
orchestrate connectivity services. SFC Working Group (WG) Customer Facing Service, RFS Resource Facing Services,
defines a distributed architecture to enable network elements LR Logical Resources, PR Physical Resources). Besides the
compute NF forwarding graphs realizing overlay paths. ZOOM working group, SELFNET will also contribute to the
FMO working group by participating in the next generation The ITU is working on the definition of the framework and
OSS architecture, which includes the autonomic management overall objectives of the future 5G systems, named as IMT-
capabilities to close the autonomic management loop: 1) 2020 (International Mobile Telecommunications for 2020) sys-
Supervision 2) Autonomic 3) Orchestration/Actuation. tems [89]. The documentation is detailed in Recommendation
ITU-R M.2083-0. It describes potential user and application
G. 3GPP trends, growth in traffic, technological trends and spectrum
Related to the ongoing specification “Study on management implications aiming to provide guidelines on the telecommu-
and orchestration architecture of next generation network nications for 2020 and beyond.
and service” [91], 3GPP analyzes its existing architectural Besides, the Study Group 13 of ITU-T is developing a
management mechanisms in contrast with next generation report on standards gap analysis [84] that describes the high-
networks and services in order to recommend enhancements, level view of the network architecture for IMT-2020 including
for instance, to support network operational features (e.g., real- requirements, gap analyses, and design principles of IMT-
time, on-demand, automation) as evolution from Long Term 2020. Its objective is to give directions for developing stan-
Evolution (LTE) management. Among the item sets contained dards on network architecture in IMT-2020. In this report also
in the scope, the specification addresses: the scenario in which includes the study areas: end-to-end quality of service (QoS)
the applications are hosted close to the access network; end- framework, emerging network technologies, mobile front haul
to-end user services; and vertical applications, such as critical and back haul, and network softwarization. The report is based
communications. on the related works in ITU-R and other SDOs.
Another ongoing specification, “Telecommunication man-
agement; Study on management and orchestration of network V. R ESEARCH P ROJECTS
slicing for next generation network” [47], presents com-
prehensive 3GPP views on network slicing associated with This section presents an overview of relevant NSO research
automation, sharing, isolation/separation and related aspects projects and positions our taxonomy accordingly as summa-
of ETSI NFV MANO. In both documents, use cases and rized in Table II, providing a single vision of their scope
requirements cover single and multi-operator services taking and status. The following subsections are identified by project
into consideration performance, fault tolerance and configura- name and its duration.
tion aspects.
A. T-NOVA (2014/01-2016/12)
H. OASIS
The focus of the FP7 T-NOVA project [54] is to design
OASIS standardizes Topology and Orchestration Specifica- and implement an integrated management architecture for the
tion for Cloud Applications (TOSCA) focused on “Enhancing automated provision, configuration, monitoring and optimiza-
the portability and operational management of cloud appli- tion of network connectivity and Network Functions as a Ser-
cations and services across their entire lifecycle”. TOSCA vice (NFaaS). Such architecture includes: (i) a micro-service
Simple Profile in YAML v1.0 was approved as standard in based NFV orchestration platform–called TeNOR [93], (ii) an
2016 in a rapidly growing ecosystem of open source communi- infrastructure visualization and management environment and
ties, vendors, researchers and cloud service providers. Looking (iii) an NFV Marketplace where a set of network services and
forward, TOSCA Technical Committee develops a Simple functions can be created and published by service providers
Profile for NFV based on ETSI NFV recommendations. and, subsequently, acquired and instantiated on-demand by
Logically, TOSCA allows the expressiveness of service costumers or others providers.
to resource mappings via flexible and compoundable data In the T-NOVA architecture, TeNOR is the highest-level in-
structures, also providing methods for specifying workflows frastructure management entity that supports multi-pop/multi-
and, therefore, enable lifecycle management tasks. In both administration domain, transport network (i.e.MPLS, Optical,
Simple and NFV Profiles, TOSCA models service behaviors Carrier Ethernet, etc.) management between POPs, and data
defining components containing capabilities and requirements, center cloud assets control. The TeNOR Orchestrator is split
and relationships among them. TOSCA realizes a compliant into two elements: (i) Network Service Orchestrator that
model of conformance and interoperability for NSOs, enhanc- manages the Network Service lifecycle, and (ii) Virtualized
ing the portability of network services. Resource Orchestrator that orchestrates the underlying com-
puting and network resources [94].
I. ITU T-NOVA leverages cloud management architectures for the
International Telecommunications Union (ITU) is the provision of resources (compute and storage) and extends SDN
United Nations specialized agency for information and com- for efficient management of the network infrastructure [95].
munication technologies (ICTs). It develops technical stan- Its architecture is based on concepts from ETSI NFV model
dards that ensure networks and technologies seamlessly in- and expands it with a marketplace layer and specific add-on
terconnected. The Study Groups of ITUs Telecommunica- features. All softwares produced in the project are available
tion Standardization Sector (ITU-T) develops international as open source at github2 .
standards known as ITU-T Recommendations which act as
defining elements in the global infrastructure of ICTs [92]. 2 https://github.com/T-NOVA
B. UNIFY (2013/11-2016/04) and interfaces. Among its main components, the project de-
3
The FP7 Unify project dedicated to approaching multiple fines modules for: topology abstraction; topology distribution;
technology domains to orchestrate joint network services con- resource repository; Service Level Agreement (SLA) man-
cerning compute, storage and networking. The primary focus ager; policy database; resource monitoring; service catalog;
set flexibility as its core concern, especially to bring methods and an inter-provider NFVO. 5GEx currently utilizes outcome
to automate and verify network services. resources mostly from projects Unify and T-NOVA, especially
The Unify architecture contains components in a hierar- joining their open source components into already prototyped
chical composition enabling recursiveness. At the bottom, demonstrations.
a set of Controller Adapters (CAs) interface technology-
specific domains (e.g., optical, radio, data center) to abstract D. SONATA (07/2015-12/2017)
southbound APIs for a typical model of information to
With 15 partners representing the telecommunication oper-
define software programmability over a network, compute
ators, service providers, academic institutes (among others),
and storage elements, such as virtualized container, SDN
the Service Programming and Orchestration for Virtualised
optical controller and OpenStack. Overseeing CAs, Resource
Software Networks (SONATA) project [96] targets to ad-
Orchestrators (ROs) define ways to manage the abstracted
dress two significant technological challenges envisioned for
components of technology-domains specifically. For instance,
5G networks: (i) flexible programmability and (ii) deploy-
while an RO for a SDN controller orchestrates network flows
ment optimization of software networks for complex ser-
(e.g., allocating bandwidth and latency), an RO for a cloud or-
vices/applications. To do so, SONATA provides an integrated
chestrator would be concerned more over orchestrate network
development and operational process for supporting network
jointly with compute and storage resources (e.g., allocating
function chaining and orchestration [97].
memory and disk). Moreover, managing one or more ROs,
a global orchestrator performs network service orchestration The major components of the SONATA architecture consist
in multiple technological domains, understanding the service of two parts: (i) the SONATA Software Development Kit (SDK)
decomposition and outsourcing specific tasks to ROs. that supports functionalities and tools for the development and
validation of VNFs and NS and (ii) the SONATA Service
Altogether, Unify presents a common model of information
Platform, which offers the functionalities to orchestrate and
to interconnect different technological domains, CAs, ROs and
manage network services during their lifecycles with a MANO
global orchestrator. Such YANG model was named Virtualizer,
framework and interact with the underlying virtual infrastruc-
and logically defined configurations following the NETCONF
ture through Virtual Infrastructure Managers (VIM) and WAN
protocol. Different demos showcasing joint orchestration of
Infrastructure Managers (WIM) [98].
computing and network resources were presented, using the
open source orchestrator ESCAPE,4 for instance, modeling The project describes the use cases envisioned for the
VNFs over data centers interconnected via an SDN enabled SONATA framework and the requirements extracted from
network domain. them. These use cases encompass a wide range of network
services including NFVIaaS, VNFaaS, vContent Delivery
Following the ONF SDN architecture, Unify demonstrated
Network (CDN), and personal security. One of the use cases
methods to apply recursiveness across its functional compo-
consists of hierarchical service providers simulating one multi-
nents in order to decompose network services to technological-
domain scenario. In this scenario, Service Programming and
specific domains.
Orchestration for Virtualised Software Networks (SONATA)
does not address the business aspects only the technical
C. 5GEx (03/2015-03/2018) approaches are in scope. SONATA intends to cover aspects
5GEx project aims agile exchange mechanisms for con- in the cloud, SDN and NFV domains [99].
tracting, invoking and settling for the wholesale consumption Moreover, the project proposes to interact and manage with
of resources and virtual network service across administrative not only VNFs also support legacy [100]. Besides, it describes
domains. Formed by a consortium of vendors, operators, and technical requirements for integrating network slicing in the
universities, 5GEx allows end-to-end network and service SONATA platform. The SONATA framework complies with
elements to mix in multi-vendor, heterogeneous technology the ETSI NFV-MANO reference architecture [100]. The
and resource environments. In such way, the project tar- results of the project are shared with the community through
gets business relationships among administrative domains, a public repository.
including possible external service providers without physical
infrastructure resources. E. 5G-Transformer (06/2017-11/2019)
Architecturally, 5GEx addresses business-to-business (B2B)
and business-to-customer (B2C) relationships across multi- The 5G-Transformer Project [101] consists of a group of
administrative domain orchestrator that might interface dif- 18 companies including mobile operators, vendors, and uni-
ferent technological domains. Basically, 5GEx extends ETSI versities. The objective of the project is to transform currents
NFV MANO architecture with new functional components mobile transport network into a Mobile Transport and Com-
puting Platform (MTP) based on SDN, NFV, orchestration,
3 http://www.fp7-unify.eu/ and analytics, which brings the Network Slicing paradigm into
4 https://github.com/hsnlab/escape mobile transport networks. The project will support a variety
of vertical industries use cases such as automotive, healthcare, is the main component orchestrating generalized virtualized
and media/entertainment. functions such as NFV and IoT. Giotis et al. [107] propose
Likewise, 5G-Transformer defines three new components to a modular architecture that enables policy-based management
the proposed architecture: (i) vertical slicer as a logical entry of Virtualized Network Functions. The proposed architecture
point to create network slices, (ii) Service Orchestrator for can handle the lifecycle of VNFs and instantiate applications
end-to-end service orchestration and computing resources, and as service chains. The work also offers an Information Model
(iii) Mobile Transport and Computing Platform for integrate towards map the VNF functions and capabilities.
fronthaul and backhaul networks. The Service Orchestrator is The work in [108] proposes a novel network slicing manage-
the main decision point of the system and interacts with others ment and orchestration framework. The proposed framework
SOs to the end-to-end service (de)composition of virtual automates service network design, deployment, configuration,
resources and orchestrate the resources even across multiple activation, and lifecycle management in a multi-domain envi-
administrative domains. Its function is similar to our definition ronment. It can manage resources of the same type such as
of NSO. The project architecture is still ongoing and is not NFV, SDN and Physical Network Function (PNF), belonging
clear its organization (hierarchical, cascade, or distributed). to different organizational domains and belonging to the same
The project is initial phase, so it has few outcomes. Parts network domain such as access, core, and transport.
of all the results produced in the project will be published
as open source. The proposed solutions are not aligned with VI. E NABLING T ECHNOLOGIES AND S OLUTIONS
standards yet, but they shall use concepts of the 3GPP and Some of the existing orchestrating solutions are just tied
MGMN [102]. to a specific networking environment, and moreover, some of
them can orchestrate an only limited number of services [42].
F. VITAL (02/2015-07/2017) In this section, an overview of main orchestration frameworks
The H2020 VITAL project [103] addresses the integration is presented, including open source, proposed and commer-
of Terrestrial and Satellite networks through the applicability cial solutions. The projects cover different technologies and
of two key technologies such as SDN and NFV. The main domains. The Table III summarizes the main characteristics
VITAL outcomes are (i) the virtualization and abstraction of of each open source projects with respect to leader entities,
satellite network functions and (ii) supporting Multi-domain resource domains, scope NFV-MANO, VNF definition, Man-
service/resource orchestration capabilities for a hybrid combi- agement Interface, and coverage area (single/multi-domain).
nation of satellite and terrestrial networks [104].
The VITAL overall architecture is in line with the main A. Open Source Solutions
directions established by ETSI ISG NFV [33], with additional
Open Source Foundations such as the Apache Foundation
concepts extended to the satellite communication domains and
and the Linux Foundation are increasingly becoming the
network service orchestration deployed across different ad-
hosting entities for large collaborative open source projects
ministrative domains. This architecture includes, among other,
in the area of networking. The most important projects are
functional entities (NFVO, VNFM, SO, Federation Layer) for
ONOS, CORD, Open Daylight, OPNFV and, recently, ONAP,
the provision and management of the NS lifecycle. In addition,
formed by the merger of Open-Orchestrator (O PEN -O) and
a physical network infrastructure block with virtualization
ECOMP. All the projects are important to create a well-defined
support includes SDN and non-SDN (legacy) based network
platform for service orchestration.
elements for flexible and scalable infrastructure management.
Note that to 5G network, standardization and open source
Implementing the relevant parts of the VITAL architecture,
are essential for fast innovation. Vendors, operators, and
X–MANO [105] is a cross-domain network service orches-
communities are betting on open source solutions. Even so,
tration framework. It supports different orchestration architec-
existing solutions are still not mature enough, and advanced
tures such as hierarchical, cascading and peer-to-peer. More-
network service orchestration platforms are missing [109].
over, it introduces an information model and a programmable
In early 2016, the Linux Foundation formed the O PEN -O
network service in order to enable confidentiality and network
Project to develop the first open source software framework
service lifecycle programmability, respectively.
and orchestrator for agile operations of SDN and NFV.
ONOS is also developing an orchestration platform for the
G. Other Research Efforts CORD project to provide Anything as a Service (XaaS)
Further architectural proposals and research contributions exploiting SDN, micro-services and disaggregation using open
can be found in the recent literature. source software and commodity hardware [43].
Recent research works have addressed the definition of Many open source initiatives towards network service or-
NFV/SDN architectures. Vilalta et al. [106] present and chestration are being deployed and this including operators,
NFV/SDN architecture for delivery of 5G services across VNF vendors and integrators. However, these are still in the
multi technological and administrative domains. The solution early stages. We describe next some of these initiatives.
is different from the NFV reference architecture. It con- 1) Open Source MANO: ETSI Open Source MANO [5]
sists of four main functional blocks: Virtualized Functions is an ETSI-hosted project to develop an Open Source NFV-
Orchestrator (VF-O), SDN IT and Network Orchestrator, MANO platform aligned with ETSI NFV Information Mod-
Cloud/Fog Orchestrator and SDN Orchestrator. The VF-O els and that meets the requirements of production NFV
TABLE II
S UMMARY OF RESEARCH PROJECTS RELATED TO NSO
Class Feature T-Nova Unify 5GEx SONATA VITAL 5G-T

IaaS/NVFIaaS # #
NaaS/NVFIaaS # #
Service SaaS/VNFaaS # #
Paaa/VNPaaS # # # # # #
SlaaS # # # #
Open Source G
# G
#
Packet
Resource/
Optical # # #
Network
Wireless #
Compute # # #
Resource
Storage # # #
Cloud
SDN
Technology
NFV
Legacy #
G # Ø
Access #
Aggregation
Scope
Core #
Data center #
Architecture / Single #
Domain Multiple G
# G
#
Hierarchical Ø
Architecture /
Cascade # # # # Ø
Organization
Distributed # # # # Ø
Service
#
Orchestration
Resource
G
#
Architecture / Orchestration
Functions Lifecycle
#
Orchestration
ETSI G
# G
# #
MEF # # # # # #
SDO 3GPP # # # # # G
#
MGMN # # # # # G
#
Others # G
# # # # #
# Outside the Scope, G
# Partial Scope, Within the Scope, Ø Undefined
networks. The project launched its third release [110] in Network Services and VNFs on a Cloud/NFV infrastructure
October 2017 and presented improvements in security, service platform such as OpenStack. It is based on ETSI MANO
assurance, resilience, and Interoperability. One of the main architectural framework and provides a functional stack to
goals of this project is to promote the integration between orchestrate end-to-end network services using VNFs.
standardization and open source initiatives. The NFVO is responsible for the high-level management
The OSM architecture has a clear split of orchestration of VNFs and managing resources in the VIM. The VNFM
function between Resource Orchestrator and Service Orches- manages components that belongs to the same VNF instance
trator. It integrates open source software initiatives such as controlling the VNF lifecycle. The Tacker also does mapping
Riftware as Network Service Orchestrator and GUI, Open- to SFC (Service Function Chain) and supports auto scaling
MANO as Resource Orchestrator (NFVO), and Juju 5 Server and TOSCA NFV Profile (using heat-translator).
as Configuration Manager (G-VNFM). The resource orches- The tacker components are directly integrated into Open-
trator supports both cloud and SDN environments. The service Stack and thus provides limited interoperability with others
orchestrator provides VNF and NS lifecycle management and VIMs. It combines the NFVO and VNFM into a single
consumes open Information/Data Models, such as YANG. Its element nevertheless, internally, the functionalities are divided.
architecture covers only single administrative domain. Another limitation is that it just works in single domain
2) Tacker: Tacker [111] is an official OpenStack project environments.
building a Generic VNFM and a NFVO to deploy and operate 3) Cloudify: Cloudify [112] is an orchestration-centric
framework for cloud orchestration focusing on optimization
5 https://www.ubuntu.com/cloud/juju NFV orchestration and management. It provides a NFVO and
Generic-VNFM in the context of the ETSI NFV, and can Standard. It allows it to be a vendor-independent platform
interact with different VIMs, containers, and non-virtualized (i.e., interoperable with different vendor solutions) and easily
devices and infrastructures. Cloudify is aligned with the extensible (at every level) for supporting new functionalities
MANO reference architecture but not fully compliant. and existing platforms.
Besides, Cloudify provides full end-to-end lifecycle of The current Open Baton release 3 includes many different
NFV orchestration through a simple TOSCA-based blueprint features and components for building a complete environment
following a model-driven and application-centric approach. fully compliant with the NFV specification. Among the most
It includes Agile Reference Implementation of Automation important are: (i) a NFVO (exposing TOSCA APIs) , (ii) a
(ARIA) as its core orchestration engine providing advanced generic VNFM and Juju VNFM, (iii) a marketplace integrated
management and ongoing automation. within the Open Baton dashboard, (iv) an Autoscaling and
In order to help contribute to open source NFV-MANO Fault Management System and (v) a powerful event engine
adoption, Cloudify engages in and sponsors diverse NFV for the dispatching of lifecycle events execution.
projects and standards organizations, such as TOSCA spec- Finally, Open Baton is included as a supporting project in
ification, ARIA and ONAP. the project named Orchestra6 . This OPNFV initiative seeks
4) ONAP: Under the Linux Foundation banner, Open Net- to integrate the Open Baton orchestration functionalities with
work Automation Platform (ONAP) [7] resulted from the existing OPNFV projects in order to execute testing scenarios
union of two open source MANO initiatives (OPEN-O [113] (and provide feedbacks) without requiring any modifications
and OpenECOMP [114]). The ONAP software platform de- in their projects.
ploys a unified architecture and implementation, with robust 7) ARIA TOSCA: Under the Apache Software Foundation,
capabilities for the design, creation, orchestration, monitoring Agile Reference Implementation of Automation (ARIA) [119]
and lifecycle management of physical and virtual network is a framework for building TOSCA-based orchestration so-
functions [115]. Also, the ONAP functionalities are expected lutions. It supports multi-cloud and multi-VIM environments
to address automated deployment and management and poli- while offering a Command Line Interface (CLI) to develop
cies optimization through an intelligent operation of network and execute TOSCA templates, and an easily consumable
resource using big data and Artificial Intelligent (AI) [116]. Software Development Kit (SDK) for building TOSCA en-
Two of the biggest challenges to merge two large sets abled software. By taking advantage of its programmable
of code are: (i) define a higher-level common information interface libraries, ARIA can be embedded into collaborative
model unifying the predominant data models used by OPEN-O projects that want to implement TOSCA-based orchestration.
(TOSCA) and OpenECOMP (YANG) and, (ii) create a stan- For example, Open-O [113] is using the ARIA TOSCA code-
dard process to the onboarding and lifecycle management of base to create its SDN & NFV orchestration tool [120].
VNFs so that end users can introduce these using an automated 8) XOS: Designed around the idea of Everything-as-a-
process (without requiring core developer teams) [117]. Service (XaaS), XOS [121] unifies SDN, NFV, and Cloud
5) X–MANO: X–MANO [105] is an orchestration frame- services (all running on commodity servers) under a single
work to coordinate end-to-end network service delivery across uniform programming environment. The XOS software struc-
different administrative domains. X–MANO introduces com- tures is organized around three layers: (i) a Data Model (im-
ponents and interfaces to address several challenges and plemented in Django7 ) which records the logically centralized
requirements for cross-domain network service orchestration state of the system, (ii) a set of Views (running on top of
such as (i) business aspects and architectural considerations, the Data Model) for customizing access to the XOS services
(ii) confidentiality, and (iii) life-cycle management. In the and (iii) a Controller Framework (from-scratch program) is
former case, X–MANO supports hierarchical, cascading and responsible for distributed state management.
peer-to-peer architectural solutions by introducing a flexible, XOS runs on the top of a mix of service controllers such
deployment-agnostic federation interface between different ad- as data center cloud management systems (e.g., OpenStack),
ministrative and technological domains. The confidentiality re- SDN-based network controllers (e.g., ONOS), network hy-
quirement is addressed by the introduction of a set of abstrac- pervisors (e.g., OpenVirtex), virtualized access services (e.g.,
tions (backed by a consistent information model) so that each CORD), etc. This collection of services controllers allows the
domain advertises capabilities, resources, and VNFs without mapping to XOS onto the ETSI NFV Architecture playing
exposing details of implementation to external entities. To the role of a VNFM. Using XOS as the VNFM facilitates
address the multi-domain life-cycle management requirement, unbundling the glsnfvo and enable to control both a set of
X–MANO introduces the concept of programmable network EMs and the VIM [122].
service based on a domain specific scripting language to allow 9) TeNOR: Developed by the T-NOVA project [54], the
network service developers to use a flexible programmable main focus of this Multitenant/Multi NFVI-PoP orchestration
Multi-Domain Network Service Descriptor (MDNS), so that platform is to manage the entire NS lifecycle service, opti-
network services are deployed and managed in a customized mizing the networking and IT resources usage. TeNOR [93]
way. presents an architecture based on a collection of loosely
6) Open Baton: Built by the Fraunhofer Fokus Institute coupled, collaborating services (also know as micro-service
and the Technical University of Berlin, Open Baton [118]
is an open source reference implementation of the NFVO 6 https://wiki.opnfv.org/display/PROJ/Orchestra
based on the ETSI NFV MANO specification and the TOSCA 7 https://www.djangoproject.com/
architecture) that ensure a modular operation of the system. Cisco offers a product named Network Services Orches-
Micro-services are responsible for managing, providing and trator enabled by Tail-f [128]. It is an orchestration platform
monitoring NS/VNFs, in addition to forcing SLA agreements that provides lifecycle service automation for hybrid networks
and determining required infrastructure resources to support a (i.e., multi-vendors). Cisco NSO enables to design and deliver
NS instance. services faster and proposes an end-to-end orchestration across
Its architecture is split into two main components: Network multiple domains. The platform deploys some management
Service Orchestrator, responsible for NS lifecycle and associ- and orchestration functions such as NSO, Telco cloud orches-
ated tasks, and Virtualized Resource Orchestrator, responsible tration, NFVO, and VNFM.
for the management of the underlying physical resources. To The Blue Planet SDN/NFV Orchestration platform [129] is
map the best available location in the infrastructure, TeNOR a Ciena’s solution that provides an integration of orchestration,
implements service mapping algorithms using NS and VNF management and analytics capabilities. It aims to automate and
descriptors. Both descriptors follow the TeNOR’s data model virtualize network service across physical and virtual domains.
specifications that are a derived and extended version of the The platform supports multiple use cases, including SD-
ETSI NSD and VNFD data model. WAN service orchestration, NFV-based service automation,
10) Gohan: NTT’s Gohan [123] is a MANO-related initia- and CORD orchestration.
tive for SDN and NFV orchestration. The Gohan architecture Another commercial solution is the HPE Service Director
is based on micro-services (just as the TeNOR implemen- of the Hewlett Packard Enterprise. The product is a service
tation) within a single unified process in order to keep the orchestration OSS solution that manages end-to-end service
system architecture and deployment model simple. A Gohan and provides analytics-based planning and closed-loop au-
service definition uses a JSON schema (both definition and tomation using declarations-based service model. It supports
configuration of resources). With this schema, Gohan delivers multi-vendor VNF, multi-VIM, various OpenStack flavors and
a called schema-driven service deployment, and it includes multiple SDN controllers.
REST-based API server, database backend, command line The Oracle Communications Network Service Orchestration
interface (CLI), and web user-interface (WebUI). Finally, a solution [130] orchestrates, automates, and optimizes VNF
couple of applicable use cases for the NTT’s Gohan include and network service lifecycle management by integrating
to use it (i) in the Service Catalog and Orchestration Layer with BSS/OSS, service portals, and orchestrators. It has two
on top of Cloud services and (ii) as a kind of NFV MANO environments to deploy the network services: one design-time
which manages both Cloud VIM and legacy network devices. environment to design, define and program the capabilities,
11) ESCAPE: Based on the architecture proposed by EU and a run-time execution environment to execute the logic
FP7 UNIFY project [124], ESCAPE (Extensible Service programmed and lifecycle management. In essence, it plays
ChAin Prototyping Environment) is a NFV proof of concept the roles of the NFVO, Telco cloud orchestration, and end-
framework which supports three main layers of the UNIFY to-end service.
architecture: (i) service layer, (ii) orchestrator layer and, (iii) Ericsson offers some solutions in the scope of the cloud,
infrastructure layer [125]. It can operate as a Multi-domain SDN and orchestration. One of them is the Ericsson Network
orchestrator for different technological domains, as well as Manager [131] that provides a unified multi-layer, multi-
different administrative domains. ESCAPE also supports re- domain (SDN, NFV, radio, transport and core) management
mote domain management (recursive orchestration), and it systems and plays various roles such as VNFM, network
operates on joint resource abstraction models (networks and slicing, and network analytics.
clouds) [126]. Many of the above-mentioned products are often exten-
In the current implementation of the process flow in ES- sions of proprietary platforms. There are few details publicly
CAPE, it receives a specific service request on its REST available, mostly marketing material. The list of commercial
API of the Service Layer. It then sends the requested Service solutions is not exhaustive and will certainly become outdated.
Function Chains to the Orchestration Layer to map the service However, the overview should serve as a glimpse on the
components to its global resource view. As a final step, the expected rise of commercial NSO solutions in the near future
calculated service parts are sent to the corresponding local as enabling open source technologies and standards mature.
orchestrators towards instantiating the service.
VII. A PPLICATION S CENARIOS
B. Commercial Solutions NSO is envisaged to support diverse use case scenarios.
This section aims at providing a brief practical view on a
The commercial orchestration solutions market is rising and
number of application domains and the main benefits provided
will be formed by diverse types of companies including new
by NSO in each scenario, delivering a sample of the expected
startups, service provider IT vendors, VNF vendors, and the
potential of NSO in operation.
traditional network equipment vendors [127].
Some software and hardware vendors already offer net-
work orchestration solutions. Below are presented the major A. Next Generation Mobile Telecommunication Networks
commercial products that we consider as mature and robust The fifth generation of mobile communication systems (5G)
solutions. All information about the products was got through is expected to meet diverse and stringent requirements that are
the vendor’s site and technical reports. currently not supported by current mobile telecommunication
TABLE III
S UMMARY OF O PEN S OURCE NSO I MPLEMENTATIONS
Resource Domain MANO Interface Management Domain

Solution Leader VNF Definition
Cloud SDN NFV Legacy NFVO VNFM VIM CLI API GUI Single Multiple
ARIA Apache Founda- TOSCA 3 3 3 3

TOSCA tion
Cloudify GigaSpace TOSCA 3 3 3 3 3 3 3 3
ESCAPE FP7 UNIFY Unify 3 3 3 3 3 3 3 3 3
Gohan NTT Data Own 3 3 3 3 3 3 3 3 3 3
ONAP Linux HOT, TOSCA, 3 3 3 3 3 3 3 3 3 3 3 3
Foundation YANG
Open Baton Fraunhofer / TU TOSCA, Own 3 3 3 3 3 3 3 3
Berlin
OSM ETSI YANG 3 3 3 3 3 3 3 3 3 3
Tacker OpenStack HOT, TOSCA 3 3 3 3 3 3 3 3
Foundation
TeNOR FP7 T-NOVA ETSI 3 3 3 3 3 3 3
X-MANO H2020 VITAL TOSCA 3 3 3 3 3
XOS ON.Lab - 3 3 3 3 3 3 3 3
networks, like ubiquitous connectivity (connectivity available perspective, logical services are implemented through central
anywhere), zero latency (lower than few milliseconds) and controllers as part of a NSO workflow.
high-speed connection (10 times higher than 4G). Optical transport of traffic across long-range areas, from
An efficient realization of 5G requires a flexible and pro- data centers to end customers as Fiber-to-the-X (e.g., houses
grammable infrastructure covering transport, radio, and cloud FTTH, curbs FTTC, Nodes FTTN), involve different interme-
resources [79]. SDN and NFV are considered key enabling diate elements requiring packet-optical conversions and vice-
technologies to provide the required flexibility in processing versa. An NSO envisioned in this scenario of packet-optical
and programmability, whereas end-to-end orchestration is re- integration can take advantage of the knowledge about topol-
garded fundamental to improve the mobile service creation ogy and equipments status, therefore optimizing traffic for-
and resource utilization across the all network segments, from warding according to optical and packet-oriented capabilities.
radio access to transport [132]. For instance, an NSO could optimize and aggregate Multi-
Furthermore, end-to-end orchestration should tackle a sig- Protocol Label Switching (MPLS) Label Switching Paths
nificant challenge in mobile telecommunication networks, (LSPs) inside optical transport networks as part of higher-
namely, the integration of different technologies, including level service lifecycle goals.
radio, SDN and NFV, so that network services may be Ongoing work at MEF aims to standardize SD-WAN [90]
dynamically created and adapted across the domains (wireless, as the means to flexibly achieve programmable micro-
aggregation and core). segmented paths – based on QoS, security and business
Finally, mobile management and orchestration solutions are policies – across sites (public or private clouds), using overlay
expected to enable (i) congestion handling per subscriber or tunnels over varied underlay technologies, such as broadband
traffic, (ii) dynamic allocation of resources according to traffic Internet and MPLS. A service orchestrator is needed to tailor
variation and/or service requirements, and (iii) load reduction and scale paths on-demand to assure application policies
on transport networks and central processing units [133]. by interfacing a controller that manages programmable edge
SD-WAN routers, spanning multiple provider sites. WAN
B. Transport Networks traffic can flow through non-trusting administrative domains in
heterogeneous wired/wireless underlay networks with varying
Optical networks evolved from statically assigned single
performance metrics.
and multi-mode fiber channels to highly flexible modula-
tion schemes using separate wavelengths. Nowadays, optical
equipments allow prompt wavelength conversion and flexible C. Data Centers
packet-to-optical setups. Given that agility increase, more Data Centers have long been upgraded with network vir-
programmability is being added to optical networks, for in- tualization for traffic forwarding and scaling L2 domains,
stance through PCE-based architectures for application-based such as VXLAN. Current technologies realize hypervisor
network operations (ABNO) [134]. tunneling for north-south and east-west traffic in data centers.
Under the flag of Software-Defined Optical Networks [135], More importantly, with the advent of operating system-level
such as those based on OpenFlow extensions, different use virtualization (a.k.a containers), even more flexible methods
cases target transport networks to deliver new approaches on of end-host network virtualization have been deployed in data
wavelength-based routing and virtualization of optical paths. centers – there are examples already available in commercial
Like PCE, different forms of SDN abstractions in optical products (e.g., VMWare NSX). In addition, computer virtual-
networks come with a logically centralized entity to program ization platforms also contain networking extensions/plugins
network elements encompassing optical paths. In a wider for dynamic networking between servers (e.g., Kubernetes and
OpenStack). Those logically programmable network fulfill- E. Intelligent Transport System

ments derive the properties that concern a NSO. The Intelligent Transport System (ITS) is composed of
The orchestration of cloud resources [136] has been a long- diverse components, including smart infrastructures, radio and
standing topic of research and actual commercial solutions. core networks, and connected vehicles/transports. The main
NSO programmability has been increasingly important to users are automotive and transport companies, governments,
keep isolation in-network and at servers for heterogeneous and vehicle users. The system consists of sensors embedded
customers that inhabit public clouds (e.g., Azure, AWS and in roads and cities to communicate with each other and/or
Google Cloud). For instance, Kubernetes, using kube-proxy, with smart vehicles and other networks. Such system focuses
defines networking in Google Cloud via a set of dynamic on massive communication among involved elements and
routes associations between service addresses and bridges’ ad- provides benefits of sustainability, security, and mobility in
dresses in PODs (servers) hosting containers; ideally, a service cities, roads, and railways [133].
is maintained independently of the associated containers host Smart vehicles, transport, and infrastructure are some of
location. Container-based orchestration is a production reality the fields where the network service orchestration can con-
but many challenges remain open [137], a number of them tribute largely. The main difficulty arises from the fact that
related to the seamless integration with network services inside components such as Evolved NodeB (E NB) [141], Roadside
the data centers and across data centers. Unit (RSU) [142], and core network need to operate towards
Similar concepts of NSO characteristics already exist to offering integrated services and with fine-tuning configurations
program paths optimizing traffic workloads, high throughput harmonically. Another problem is the dynamism of the net-
and low latency across data centers and to edge CDNs – best work traffic with the significant amount of data and constant
examples being Google B4 and Andromeda SDN projects. changes in the network.
Therefore, NSO already plays an essential role in data center The orchestration can handle a big amount of data, contexts,
networking as it became a pioneer in direct application of and interfaces under an automatic and agile way. It demands
SDN concepts. for an overview of all the infrastructure and connected devices
Lately, research topics in this domain concern integration of to enhance decision making process. With the adoption of
multiple cloud environments envisioning different guarantees NSO, the elements of the network architecture can be exposed
of SLA for distinct classes of traffic. As more mobile appli- as VNFs and new elements, e.g., telemetry and analytics, may
cations evolve towards accomplishing customers requirements be introduced. As a result, issues as scalability and location
for low latency and high throughput (e.g., virtual and aug- can be solved. Besides, NSO using SDN can handle the inter-
mented reality), NSO will play an important role addressing system handover that consists of a switch among different
issues originated from those requirements. networks (WCDMA, LTE, WI-FI) due to the fast movement
of the transports. Many challenges need to be overcome and
D. Network Slicing orchestration is regarded as a key factor to the success of ITS.
Future mobile/5G and fixed networks scenarios with diverse
service requirements represent a growing and more complex F. Internet of Things
challenge at the time of managing network resources. Network According to Gubbi et al. [143], I OT is a network of sensing
Slicing is being widely discussed in standard organizations as and actuating devices providing the ability to share information
a key mechanism to provide flexibility in the management through a unified platform. Such devices or ”things” may
of network resources [79]. Network Slicing enables operators transmit a big amount of data over a network without requiring
to create multiple network resources and (virtual) network human-to-human or human-to-computer interaction. Its appli-
functions isolated and customized over the same physical in- cation areas include homes, cities, industry, energy systems,
frastructure [138]. Such dedicated networks, built on a shared agriculture, and health. Due to the amount of generated-data
infrastructure can reduce the cost of the network deployment, and its dynamic and transient operational behavior, I OT will
speeds up the time to market and offer individual networks lead to scalability and management issues in the process of
customizations according to customer requirements so that transport, processing, and storage of the data in real time [3].
operators can introduce new market services [108]. Besides, the various entities involved need to be orchestrated
Increased flexibility introduces higher complexity in design to convert the data into actionable information [144].
and operation of network slices. Keys to avoid the CAPEX and NSO along with NFV and SDN allow network services
OPEX increase is to automate the full lifecycle phases of a to be automatically deployed and managed. In this scenario,
slice: (i) preparation phase, (ii) instantiation, configuration and SDN is responsible for establishing the network connections,
activation phase, (iii) runtime phase and (iv) decommissioning NFV provides the management of the network functions,
phase [139]. Besides the automation, other management and and NSO govern all deployment process of the end-to-end
orchestration use cases of network slicing are fault man- network service. Such paradigms can help to process and
agement, performance management, and policy management. manage significant amounts of IoT-generated data with better
It is also expected multi-operator coordination management network efficiency. The separation between resources and
in order to create end-to-end network slices across multiple services provided by such technologies enables the isolation
administrative domains and some level of management to be and lower impact risks of I OT on other infrastructures. Also,
exposed to the network slice tenant [140]. they can reduce the human intervention in the operation of the
network, feature that is essential to the achievement of Internet B. Security and Resiliency
of Things. Softwarized networks modify the way how services are
The authors in [145] propose an orchestrator for Internet deployed replacing the hardware-based network service com-
of Things that manages all planes of an I OT ecosystem. ponents with software-based solutions [98]. Through tech-
The orchestrator selects resources and deploys the services nologies such as SDN and NFV, such network can provide
according to security, reliability, and efficiency requirements. automation, programmability, and flexibility. Generally, it de-
This approach enables an overall view of the whole envi- pends on centralized control, which leads to risks to security
ronment, reducing costs and improving the user experience. and resiliency [149]. Thus, new protection capabilities need to
Thus, orchestration allows the creation of more flexible and be put in place, including advanced management capabilities
scalable services, reducing the probability of failure correlation such as authentication, access control, and fault management.
between application components. Security and resiliency must be considered both in design
and operation stages of network services. Typically, the ser-
VIII. C HALLENGES AND RESEARCH OPPORTUNITIES vices are deployed first, prior to any efforts regarding security
NSO promises to improve efficiency when instantiating development. However, security must be a mandatory issue,
(day 1) and operating (day 2) network services, but the mainly in a highly connected and virtualized environment.
path ahead is not without challenges. This section provides a Service instantiation involves automated processes that add
discussion on the main challenges and research opportunities and delete network elements and functions without human
for NSO, including scalability, security, resource modeling, intervention. A critical problem is the addition of a malicious
performance, and interoperability. node that can perform attacks, catch valuable information and
even the disruption of the entire services.
An essential requirement for a multi-domain orchestration
A. Scalability
platform is the capability to hide specific details of each
Some researches assume that 5G network might connect 50 domain. This ensures privacy and confidentially of the do-
billion devices until 2020 [146], [147]. This growth is due mains, preserving capabilities and resources to an external
to the emergence of vertical industries such as Internet of component [105].
Things, Smart Cities, and Sensor Networks. In this scenario, Resilience in main NSO components such as orchestrators,
orchestration process requires the ability to handle the growth controllers, and managers is also a critical problem because it
of networks and services to support the huge amount of can impact directly in overall service operation. Besides, open
connected nodes. interfaces that support network programmability and NSO
In addition, the network services can be deployed over components communication with other external elements such
different domains managed by third parties, infrastructure cov- as OSS and other orchestrators are an open issue and a hot
ering large geographical space and diverse type of resources topic in research [64], [149], [150]. In the same direction,
such as access, transport, and core networks. This environ- the 5G-PPP published a white paper [151] suggesting that the
ment demands high scalability of the components involved, orchestration platform must be secure, reliable and flexible.
including orchestrators, controllers, and managers.
Most current NSO use cases are just based on deploying a
network service in a controlled scenario. Just a use case is not C. Resource and Service Modeling
able to check the scalability of the solution. In a production Network services need to be efficiently modeled towards
environment, the orchestrator is responsible for orchestrating deploying resource requirements, configuration parameters,
millions of customers and services at the same time. Hence, management policies, and performance metrics. Service mod-
scalability is an important feature for NSO success. eling will enable abstraction of resources and capabilities of
Some orchestration solutions mainly focus on centralized underlying layers. It simplifies the understanding of functions
solutions, which pose scalability issues. The works [43] and provides a generic way to represent resource and service.
and [148] suggest different orchestrators involved in the or- However, it is a major challenge to translate higher-level
chestration process of end-to-end network services, not being policies, which are generated from the resource allocation and
limited to a single orchestrator. However, there are several optimization mechanisms, into a lower level configuration.
particularities on each layer that could be better explored with Templates and standards should be developed to guarantee
specific orchestrators, instead of adopting a global orchestrator automated and consistent translation [152]. Besides, the stan-
approach. In this way, we argue that the whole orchestration dardization can enable the interoperability and integration of
process can experience better results if split among different network services templates and addresses limitations arising
actors (or orchestrators). in the deployment of services in heterogeneous landscape.
A key challenge is therefore to develop an orchestration There are templates and data modeling languages for Net-
process that is massively scalable. This process could involve work Function Virtualization (NFV) and Network Service
one or more orchestrators, becoming open and flexible enough (NS) such as TOSCA, YANG, and HOT. In addition, some
to address future applications and enable the integration with organizations propose their own approaches to the definition
external components. The orchestration must avoid the con- of Network Services, e.g., Open Baton and Gohan.
gestions and bottlenecks in the management and orchestration ETSI NFV MANO proposes VNF and Network Service
plane to handle the requests for network services. descriptors as templates for the definition of functions and
services. According to ETSI, NS is defined as a set of VNFs NSO lifecycles and run-time MANO resource allocation and
and/or PNFs interconnected by Virtual Links (VLs) and one management decisions are potential techniques towards service
or more VNF Forwarding Graph. guarantees and SLA compliance.
On the other hand, ETSI NS specifies lowest level resources In addition, a better composition between the traffic for-
such as CPU, memory, and network, but it does not extend warding and NF placement is required towards optimizing the
the resource modeling and does not define a data model to NS deployment. The first steps to provide service performance
the descriptors [153]. Thus, its approach is driven to single guarantees are to avoid heavily loaded service nodes and to
domain environment [148]. identify bottleneck links. Algorithms and machine learning
On the other hand, the IETF SFC provides the ability to techniques can archive better results in this composition.
define an ordered list of network services, or service functions Thus, how to achieve high performance is an important
(e.g., firewalls, load balancers, DPI) connecting them in a problem in the research and development of NSO solutions.
virtual chain. However, SFC does not describe the underlying Projects within the 5G Infrastructure Public Private Partnership
resource, since its primary focus is service operation, apart (5G-PPP) [151] are targeting enhanced performance towards
from the forwarding topology. As opposed to ETSI, SFC scope better user experience.
covers multi-domain connections.
Resource and service modeling in softwarized networks E. Multi-tenancy and Interoperability
including multi-domain scenarios need further work. This
Typically, operators infrastructures are organized in several
evolution will enable interoperability of network services and
domains that differ in geographical locations, management
the correct mapping between the high-level configuration and
(e.g., legacy or SDN), administrative boundaries, and tech-
the underlying infrastructure. Currently, the interoperability
nologies. One of the challenges for service providers is to
among the diverse orchestration platforms does not exist.
create and to manage services across unique and proprietary
interfaces, making integration and startup difficult tasks to be
D. Performance and Service Assurance achieved, as well as increasing the operational costs.
In this scenario, interoperability is essential to enable
The changes that orchestration technology brings to the
the deployment of end-to-end network services. Few end-
telecommunication infrastructures make them increasingly vir-
to-end services will be confined within the boundaries of
tualized and software-based. So, performance is a constant
a single domain. They normally encompass a multi-domain
challenge in a highly dynamic environment of virtual functions
orchestration environment composed of providers and vendors
and services.
with different incentives and business models [109]. There is
This change reflects on enabling technologies. For instance,
no consensus about how would be the exchanging process
the NFV should meet performance requirements to support,
between the multiple actors in deployment end-to-end network
in a standard server, the packet processing, including high
services. In fact, ETSI MANO architecture does not bring any
I/O speed, fast transmission, and short delays [152]. The
provisioning for this kind of exchange [31].
VNFs must achieve a performance comparable to specialized
A number of orchestration solutions based on the ETSI
hardware. According to [3], some applications require specific
MANO architecture have emerged with the objective of
capabilities, but virtualization can degrade their performance.
proposing a complete orchestration framework. Table III
This generates a trade-off between performance and flexi-
shows notable solutions. Although the progress made by ETSI
bility. However, recent advances in CPU and virtualization
in defining architecture and interfaces, each solution uses
technologies are overcoming these challenges include Data
a particular implementation and data model, which makes
Plane Development Kit (DPDK) [154] – libraries and drivers
interoperability difficult to achieve (cf. [160]). As a result,
for fast packet processing, NetVM [155] – enabling high
chaining network functions leveraging different solutions for
bandwidth network functions to operate at near line speed,
a single network service deployment and operation is currently
and ClickOS [156] – minimalist operating that supports high
a very costly proposition in terms of development efforts and
throughput, low delay, and isolation. Likewise, the docu-
time-to-market.
ment [157] of the ETSI provides a set of recommendations on
Standardization is a path to enable interoperability of net-
the minimum requirements that the hardware and virtualized
work services between operators and address limitations that
layer should have to achieve high performance.
arise in the deployment of services, as explained in Section IV.
Another question is performance monitoring coupled with
Another parallel track towards interoperability is a broad
Network Services maintenance. Both require a global view
adoption of software components and broad agreements on
of the resources and a unified control and optimization pro-
APIs along data and information models fueled by re-usable
cess with various optimization policies running in it. The
open source artifacts.
monitoring is required to avoid the violation of SLAs in
the Service layer. In the order to keep NS performance, it
is demanded that the system equally performs in different F. Network Service Lifecycle Management
layers. In multi-domain scenarios, this becomes more complex Network service lifecycle consists in all process for de-
because it is necessary the exchange of information and ployment, execution, and termination of a network service.
resources between different organizations/domains [53]. VNF The Network Service Lifecycle Management is fundamental
benchmarking [158] and NS chain profiling [159] coupled to to ensure the correct operation of the service.
Nevertheless, the network services can have specific life- ACKNOWLEDGMENT

cycle management requirements. For example, an NS can
use specific resources as Single Root I/O Virtualization (SR- This research was supported by the Innovation Center, Er-
IOV) [161] and DPDK or need resources across various icsson S.A., Brazil, grant UNI.62. The authors would also like
domains. This type of requirements becomes harder the service to express their gratitude to review contributions from David
deployment. Moura, Lucian Beraldo, Nazrul Islam, and Suneet Singh (in
alphabetical order) funded by the EU-Brazil NECOS project
One possible solution is service lifecycle automation. It
under grant agreement no. 777067. The authors are thankful
enables lifecycle management without human intervention.
for any feedback to improve the work. Do not hesitate to
Automation can be obtained through heuristic algorithms
contact the authors and/or via github: https://github.com/intrig-
and machine learning techniques. ONAP is working on new
unicamp/publications/tree/master/NSO-Survey.
close control loops (e.g., CLAMP - Closed Loop Automa-
tion Management Platform)8 towards providing automation,
performance optimization and Service Lifecycle Management, R EFERENCES
eventually leveraging network analytics and machine learning
assisted decisions. Nevertheless, many aspects of run-time [1] C. Corporation, “Products — Multi-domain Service Orchestration,”
2017, accessed 2017-08-07. [Online]. Available: http://www.blueplanet.
(day 2) workflow modeling and implementation remain open, com/products/multi-domain-service-orchestration.html
with TOSCA extensions and BPMN/BPML approaches [162] [2] D. Kreutz, F. M. V. Ramos, P. Esteves Verissimo, C. Esteve Rothenberg,
undergoing improvements to meet the needs of NSO-based S. Azodolmolky, and S. Uhlig, “Software-Defined Networking: A
Comprehensive Survey,” Proceedings of the IEEE, vol. 103, no. 1,
lifecycle automation. pp. 14–76, 1 2015. [Online]. Available: http://ieeexplore.ieee.org/
document/6994333/
[3] R. Mijumbi, J. Serrat, J.-l. Gorricho, N. Bouten, F. De Turck,
IX. CONCLUSIONS and R. Boutaba, “Network Function Virtualization: State-of-the-
Art and Research Challenges,” IEEE Communications Surveys &
The traditional telecommunication industry is facing mul- Tutorials, vol. 18, no. 1, pp. 236–262, 2016. [Online]. Available:
tiple challenges to keep competitive and improve the mode http://ieeexplore.ieee.org/document/7243304/
[4] B. Sonkoly, R. Szabo, D. Jocha, J. Czentye, M. Kind, and F.-J.
network services are designed, deployed and managed. Archi- Westphal, “UNIFYing Cloud and Carrier Network Resources: An
tectures and enabling technologies such as Cloud Computing, Architectural View,” in 2015 IEEE Global Communications Conference
SDN and NFV, are providing new paths to overcome these (GLOBECOM). IEEE, 12 2014, pp. 1–7. [Online]. Available: http:
//ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=7417869
challenges in a software-driven approach. Network Service [5] ETSI, “Open Source MANO.” [Online]. Available: https://osm.etsi.org/
Orchestration (NSO) is a strategic element in this process [6] O. Foundation, “Heat - OpenStack.” [Online]. Available: https:
of evolution. NSO aims at converging various technologies //wiki.openstack.org/wiki/Heat
by providing a broader and comprehensible view of network [7] L. Foundation, “ONAP – Open Network Automation Platform,” 2017,
accessed 2017-09-05. [Online]. Available: https://www.onap.org/
services. [8] R. Guerzoni, I. Vaishnavi, D. Perez Caparros, A. Galis, F. Tusa,
In this comprehensive survey on network service orches- P. Monti, A. Sganbelluri, G. Biczók, B. Sonkoly, L. Toka, A. Ramos,
tration, we aim at highlighting its importance and trying to J. Melián, O. Dugeon, F. Cugini, B. Martini, P. Iovanna, G. Giuliani,
R. Figueiredo, L. M. Contreras-Murillo, C. J. Bernardos, C. Santana,
contribute to a common understanding of the concept and and R. Szabo, “Analysis of end-to-end multi-domain management
diverse approaches towards practical embodiments of NSO. and orchestration frameworks for software defined infrastructures: an
We present enabling technologies, clarify on the definition of architectural survey,” Transactions on Emerging Telecommunications
Technologies, vol. 28, no. 4, p. e3103, 4 2017. [Online]. Available:
term orchestration, review standardization advances, research http://doi.wiley.com/10.1002/ett.3103
projects, commercial solutions, and list a number of challenges [9] ON.LAB, “Open CORD.” [Online]. Available: https://opencord.org/
such as resource and service modeling, multi-tenancy and in- [10] C. Rotsos, D. King, A. Farshad, J. Bird, L. Fawcett, N. Georgalas,
M. Gunkel, K. Shiomoto, A. Wang, A. Mauthe, N. Race, and
teroperability, multi-domain orchestration, scalability, security D. Hutchison, “Network service orchestration standardization: A
and resiliency, performance, and lifecycle management. technology survey,” Computer Standards & Interfaces, vol. 54, pp.
The application of NSO in some scenarios was also pre- 203–215, 11 2017. [Online]. Available: http://www.sciencedirect.com/
science/article/pii/S0920548916302458
sented, where it is possible to sense its potential and un- [11] 5G PPP Architecture Working Group, “View on 5G Architecture,”
derstand the motivation behind so much ongoing work. We White paper, no. July, 2016.
also observe a growing trend towards the use of open source [12] P. Mell and T. Grance, “The NIST Definition of Cloud Computing Rec-
ommendations of the National Institute of Standards and Technology,”
components or solutions in orchestration platforms; however, Nist Special Publication, vol. 145, p. 7, 2011.
the platforms require to evolve until become suitable for [13] N. T. Le, M. A. Hossain, A. Islam, D.-y. Kim, Y.-J. Choi, and Y. M.
production. An important contribution of this work was the Jang, “Survey of Promising Technologies for 5G Networks,” Mobile
Information Systems, vol. 2016, pp. 1–25, 2016. [Online]. Available:
definition of a taxonomy that categorizes the leading charac- https://www.hindawi.com/journals/misy/2016/2676589/
teristics and features related to network service orchestration. [14] M. A. Vouk, “Cloud computing - Issues, research and implementa-
Despite the fast pacing issues of this vibrant topic, we tions,” in Proceedings of the International Conference on Information
Technology Interfaces, ITI, 2008, pp. 31–40.
expect this survey to serve as a guideline to researchers and [15] N. Saraiva, T. Falco, A. Macłdo, and A. Soares, “A proposed archi-
practitioners looking into an overview of network service tecture for choice of switching paradigm in hybrid optical networks
orchestration fundamentals, a reference to relevant related (ocs/obs),” in 2014 XL Latin American Computing Conference (CLEI),
Sept 2014, pp. 1–6.
work and pointers to open research questions. [16] C. J. Bernardos, O. Dugeon, A. Galis, D. Morris, and C. Simon, “5G
Exchange (5GEx) Multi-domain Orchestration for Software Defined
8 https://github.com/onap/clamp Infrastructures,” 2015.
[17] A. Doria, J. Salim, R. Haas, H. Khosravi, W. Wang, R. Gopal, [36] B. Yi, X. Wang, K. Li, S. K. Das, and M. Huang, “A
and J. Halpern, “RFC 5810: Forwarding and Control Element comprehensive survey of network function virtualization,” Computer
Separation (ForCES) Protocol Specification,” Tech. Rep., 2010. Networks, vol. 133, pp. 212–262, 2018. [Online]. Available:
[Online]. Available: https://tools.ietf.org/html/rfc5810 https://doi.org/10.1016/j.comnet.2018.01.021
[18] B. Davie, “RFC 7047: The Open vSwitch Database Management [37] A. Campbell, G. Coulson, F. Garcı́a, and D. Hutchison, “A continuous
Protocol,” Tech. Rep., 2013. [Online]. Available: https://www.ietf.org/ media transport and orchestration service,” in Conference proceedings
rfc/rfc7047.txt on Communications architectures & protocols - SIGCOMM ’92. New
[19] H. Song, “Protocol-oblivious forwarding,” in Proceedings of the York, NY, USA: ACM Press, 1992, pp. 99–110. [Online]. Available:
second ACM SIGCOMM workshop on Hot topics in software http://dl.acm.org/citation.cfm?id=144253%5Cnhttp://portal.acm.org/
defined networking - HotSDN ’13, ACM. New York, NY, citation.cfm?doid=144179.144253http://portal.acm.org/citation.cfm?
USA: ACM Press, 2013, p. 127. [Online]. Available: http: doid=144179.144253
//dl.acm.org/citation.cfm?doid=2491185.2491190 [38] W. Robbins, “Implementation and performance issues in an
[20] N. McKeown, T. Anderson, H. Balakrishnan, G. Parulkar, L. Peterson, object-oriented orchestration architecture,” in Proceedings of IEEE
J. Rexford, S. Shenker, and J. Turner, “OpenFlow,” ACM SIGCOMM International Conference on Multimedia Computing and Systems, no.
Computer Communication Review, vol. 38, no. 2, p. 69, 3 2008. figure 1. IEEE Comput. Soc, 1997, pp. 628–629. [Online]. Available:
[Online]. Available: http://portal.acm.org/citation.cfm?doid=1355734. http://ieeexplore.ieee.org/document/609789/
1355746 [39] C. Peltz, “Web services orchestration and choreography,” Computer,
[21] SDX Central, “What are SDN Southbound APIs?” 2014. [Online]. vol. 36, no. 10, pp. 46–52, 10 2003. [Online]. Available:
Available: https://www.sdxcentral.com/sdn/definitions/southbound- http://ieeexplore.ieee.org/document/1236471/
interface-api/ [40] A. Galis, H. Abramowicz, M. Brunner, D. Raz, P. Chemouil,
[22] L. Richardson and S. Ruby, RESTful web services. ” O’Reilly Media, J. Butler, C. Polychronopoulos, S. Clayman, H. de Meer, T. Coupaye,
Inc.”, 2008. A. Pras, K. Sabnani, P. Massonet, and S. Naqvi, “Management and
[23] T. Koponen, M. Casado, N. Gude, J. Stribling, L. Poutievski, M. Zhu, service-aware networking architectures (MANA) for future Internet,”
R. Ramanathan, Y. Iwata, H. Inoue, T. Hama, and S. Shenker, “Onix: A in 2009 Fourth International Conference on Communications and
Distributed Control Platform for Large-scale Production Networks.” in Networking in China. IEEE, 8 2009, pp. 1–13. [Online]. Available:
Operating Systems Design and Implementation - OSDI, vol. 10, 2010, http://ieeexplore.ieee.org/document/5339964/
pp. 1–6. [41] C. E. Abosi, R. Nejabati, and D. Simeonidou, “Service Oriented
[24] T. Koponen, K. Amidon, P. Balland, M. Casado, A. Chanda, B. Fulton, Resource Orchestration in Future Optical Networks,” in 2011
I. Ganichev, J. Gross, P. Ingram, E. J. Jackson, and others, “Network Proceedings of 20th International Conference on Computer
Virtualization in Multi-tenant Datacenters,” in NSDI, vol. 14, 2014, pp. Communications and Networks (ICCCN). IEEE, 7 2011, pp. 1–6.
203–216. [Online]. Available: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.
[25] K. Pentikousis, Y. Wang, and W. Hu, “Mobileflow: Toward htm?arnumber=6005932http://ieeexplore.ieee.org/document/6005932/
software-defined mobile networks,” IEEE Communications Magazine, [42] S. Kuklinski, K. T. Dinh, C. Destre, and I. G. Ben Yahia,
vol. 51, no. 7, pp. 44–53, 7 2013. [Online]. Available: http: “Design principles of generalized network orchestrators,” in 2016
//ieeexplore.ieee.org/document/6553677/ IEEE International Conference on Communications Workshops
[26] N. Gude, T. Koponen, J. Pettit, B. Pfaff, M. Casado, N. McKeown, (ICC). IEEE, 5 2016, pp. 430–435. [Online]. Available: http:
and S. Shenker, “Nox: Towards an operating system for networks,” //ieeexplore.ieee.org/document/7503825/
SIGCOMM Comput. Commun. Rev., vol. 38, no. 3, pp. 105–110, [43] R. Alvizu, “Advance Optical Routing Techniques in the Software
Jul. 2008. [Online]. Available: http://doi.acm.org/10.1145/1384609. Defined Era,” Ph.D. dissertation, Politecnico di Milano, 2016. [Online].
1384625 Available: http://hdl.handle.net/10589/131914%0A
[27] ON.LAB, “ONOS - A new carrier-grade SDN network operating [44] R. B. Bohn, J. Messina, F. Liu, J. Tong, and J. Mao, “NIST Cloud
system designed for high availability, performance, scale-out.” Computing Reference Architecture,” in 2011 IEEE World Congress
accessed 2017-08-08. [Online]. Available: http://onosproject.org/ on Services. IEEE, 7 2011, pp. 594–596. [Online]. Available:
[28] Linux Foundation, “OpenDaylight.” [Online]. Available: https://www. http://ieeexplore.ieee.org/document/6012797/
opendaylight.org/ [45] Open Networking Foundation, “Framework and Architecture for the
[29] Infonetics Research, “The Evolution of SDN and NFV Orchestration,” Application of SDN to Carrier networks,” Tech. Rep., 2016. [Online].
2015, accessed 2018-02-12. [Online]. Available: https://www.juniper. Available: www.opennetworking.org
net/assets/cn/zh/local/pdf/analyst-reports/2000604-en.pdf [46] International Telecommunication Union, “ITU-T Y.3300: Framework
[30] ETSI, “Network Functions Virtualisation: An Introduction, Benefits, of software-defined networking,” Tech. Rep., 2014.
Enablers, Challenges & Call for Action,” in SDN and OpenFlow [47] 3GPP, “TR 28.801: Study on management and orchestration of network
World Congress, 2012. [Online]. Available: https://portal.etsi.org/NFV/ slicing for next generation network,” Tech. Rep., 2017. [Online].
NFV White Paper.pdf Available: http://www.3gpp.org/ftp/Specs/html-info/28801.htm
[31] ETSI Industry Specification Group (ISG) NFV, “ETSI GS [48] NGMN Alliance, “NGMN 5G White Paper,” Tech. Rep., 2015.
NFV 003 V1.2.1: Network Functions Virtualisation (NFV); [Online]. Available: https://www.ngmn.org/uploads/media/NGMN
Terminology for Main Concepts in NFV,” 2014. [Online]. Available: 5G White Paper V1 0.pdf
http://www.etsi.org/deliver/etsi gs/NFV/001 099/003/01.02.01 60/gs [49] Metro Ethernet Forum, “The Third Network: Lifecycle Service
NFV003v010201p.pdf Orchestration Vision,” MEF, Tech. Rep., February 2015, accessed
[32] J. Halpern and C. Pignataro, “Service Function Chaining (SFC) 2017-10-01. [Online]. Available: https://www.mef.net/Assets/White
Architecture,” Tech. Rep., oct 2015. [Online]. Available: https: Papers/MEF Third Network LSO Vision FINAL.pdf
//datatracker.ietf.org/doc/html/rfc7665https://www.rfc-editor.org/info/ [50] A. Rostami, P. Öhlén, M. A. S. Santos, and A. Vidal, “Multi-Domain
rfc7665 Orchestration across RAN and Transport for 5G,” in Proceedings of the
[33] ETSI Industry Specification Group (ISG) NFV, “ETSI GS 2016 conference on ACM SIGCOMM 2016 Conference - SIGCOMM
NFV 002 V1.1.1: Network Functions Virtualisation (NFV); ’16. New York, NY, USA: ACM Press, 2016, pp. 613–614. [Online].
Architectural Framework,” Tech. Rep., 2013. [Online]. Available: Available: http://dl.acm.org/citation.cfm?doid=2934872.2959073
http://www.etsi.org/deliver/etsi gs/NFV/001 099/002/01.01.01 60/gs [51] A. S. Thyagaturu, A. Mercian, M. P. McGarry, M. Reisslein,
NFV002v010101p.pdf and W. Kellerer, “Software Defined Optical Networks (SDONs): A
[34] ——, “Network Functions Virtualisation (NFV); Management and Comprehensive Survey,” IEEE Communications Surveys & Tutorials,
Orchestration; Report on Architectural Options,” Tech. Rep., 2014. vol. 18, no. 4, pp. 2738–2786, 2016. [Online]. Available: http://arxiv.
[Online]. Available: http://www.etsi.org/deliver/etsi gs/NFV-IFA/001 org/abs/1511.04376http://ieeexplore.ieee.org/document/7503119/
099/009/01.01.01 60/gs NFV-IFA009v010101p.pdf [52] R. Guerzoni, D. P. Caparros, P. Monti, G. Giuliani, J. Melian,
[35] ——, “ETSI GS NFV-EVE 005 V1.1.1: Network Functions R. Figueiredo, A. Ramos, C. J. Bernardos, B. Sonkoly, F. Tusa,
Virtualisation (NFV); Ecosystem; Report on SDN Usage in NFV A. Galis, I. Vaishnavi, F. Ubaldi, A. Sgambelluri, C. Santana, and
Architectural Framework,” ETSI, Tech. Rep. [Online]. Available: R. Szabo, “Multi-domain Orchestration and Management of Software
http://www.etsi.org/deliver/etsi gs/NFV-EVE/001 099/005/01.01.01 Defined Infrastructures: a Bottom-Up Approach,” EUCNC - Europe
60/gs nfv-eve005v010101p.pdf Conference on Networks and Communications, pp. 3–8, 2016.
[53] R. V. Rosa, M. A. S. Santos, and C. E. Rothenberg, “Md2-nfv: The requirements specification,” Tech. Rep., 2017. [Online]. Available:
case for multi-domain distributed network functions virtualization,” in http://www.etsi.org/deliver/etsi gs/NFV- IFA/001 099/010/02.03.01
2015 International Conference and Workshops on Networked Systems 60/gs NFV-IFA010v020301p.pdf
(NetSys), March 2015, pp. 1–5. [72] ——, “GS NFV-IFA 014 - V2.3.1: Network Functions Virtualisation
[54] FP7 project T-NOVA, “T-NOVA Project, Network Functions as (NFV) Release 2; Management and Orchestration; Network Service
a Service over Virtualised Infrastructures.” [Online]. Available: Templates Specification,” Tech. Rep., 2017. [Online]. Available:
http://www.t-nova.eu/ http://www.etsi.org/standards-search
[55] B. Sonkoly, J. Czentye, R. Szabo, D. Jocha, J. Elek, S. Sahhaf, [73] ——, “GR NFV-IFA 023 - V3.1.1: Network Functions Virtualisation
W. Tavernier, and F. Risso, “Multi-Domain Service Orchestration Over (NFV); Management and Orchestration; Report on Policy Management
Networks and Clouds,” in Proceedings of the 2015 ACM Conference in MANO Release 3,” Tech. Rep., 2017. [Online]. Available:
on Special Interest Group on Data Communication - SIGCOMM ’15. http://www.etsi.org/standards-search
New York, New York, USA: ACM Press, 2015, pp. 377–378. [Online]. [74] Metro Ethernet Forum, “Lifecycle Service Orchestration (LSO):
Available: http://dl.acm.org/citation.cfm?doid=2785956.2790041 Reference Architecture and Framework,” Mar. 2016, accessed on
[56] ETSI Industry Specification Group (ISG) NFV, “GR NFV-IFA 2017-10-01. [Online]. Available: http://dev.mef.net/Assets/Technical
028 - V3.1.1: Network Functions Virtualisation (NFV) Release Specifications/PDF/MEF 55.pdf
3; Management and Orchestration; Report on architecture options [75] Open Networking Foundation, “TR-521: SDN architecture 1.1,”
to support multiple administrative domains,” Tech. Rep. [Online]. 2016, accessed on 2017-10-01. [Online]. Available: https:
Available: http://www.etsi.org/deliver/etsi gr/NFV-IFA/001 099/028/ //www.opennetworking.org/images/stories/downloads/sdn- resources/
03.01.01 60/gr NFV-IFA028v030101p.pdf technical-reports/TR-521 SDN Architecture issue 1.1.pdf
[57] ——, “GS NFV-MAN 001 - V1.1.1: Network Functions Virtualisation [76] ——, “TR-528: Mapping Cross Stratum Orchestration (CSO) to
(NFV); Management and Orchestration,” Tech. Rep., 2014. [Online]. the SDN architecture,” March 2016. [Online]. Available: https://
Available: http://www.etsi.org/deliver/etsi gs/NFV- MAN/001 099/ www.opennetworking.org/wp-content/uploads/2014/10/TR-528 CSO
001/01.01.01 60/gs NFV-MAN001v010101p.pdf Architecture.pdf
[58] E. Rojas, “From software-defined to human-defined networking: Chal- [77] ——, “TR-540: Orchestration: A More Holistic View,”
lenges and opportunities,” IEEE Network, vol. 32, no. 1, pp. 179–185, July 2017, accessed on 2017-10-01. [Online]. Available:
Jan 2018. https://www.opennetworking.org/wp- content/uploads/2014/10/TR-
[59] N. Leavitt, “Is Cloud Computing Really Ready for Prime Time?” 540 Orchestration- A More Holistic View 1.50.47 PM.pdf
Computer, vol. 42, no. 1, pp. 15–20, 1 2009. [Online]. Available: [78] NGMN Alliance, “5G White Paper,” Next Generation Mobile
http://ieeexplore.ieee.org/document/4755149/ Networks, White paper, pp. 1–125, 2015. [Online]. Available:
[60] M. Karakus and A. Durresi, “Quality of Service (QoS) in Software https://www.ngmn.org/uploads/media/NGMN 5G White Paper V1
Defined Networking (SDN): A survey,” Journal of Network and 0 01.pdf
Computer Applications, vol. 80, no. December 2016, pp. 200–218, [79] ——, “5G Network and Service Management including
2017. [Online]. Available: http://dx.doi.org/10.1016/j.jnca.2016.12.019 Orchestration,” March 2017. [Online]. Available: https://www.ngmn.
[61] Sdxcentral, “2016 Mega NFV Report Pt . 1 :,” 2016. org/uploads/media/170307 5G Network and Service Management
[Online]. Available: https://www.sdxcentral.com/wp-content/uploads/ including Orchestration 2.12.7.pdf
2016/04/SDxCentral- Mega- NFV- Report- Part- 1- MANO- and- NFVI- [80] TM Forum, “ZOOM Project.” [Online]. Available: https://www.
2016-B.pdf tmforum.org/collaboration/zoom-project/
[62] Opensource.com, “Four ways to organize as an open source [81] Organization for the Advancement of Structured Information Standards,
community — Opensource.com.” [Online]. Available: https: “Topology and Orchestration Specification for Cloud Applications
//opensource.com/business/13/6/four- types- organizational- structures- Version 1.0,” 2013. [Online]. Available: http://docs.oasis-open.org/
within-open-source-communities tosca/TOSCA/v1.0/TOSCA-v1.0.html
[63] Linux Foundation, “OPNFV - Open Platform for NFV,” accessed [82] ——, “TOSCA Simple Profile for Network Functions Virtualization
2018-01-13. [Online]. Available: https://www.opnfv.org/ (NFV)Version 1.0,” 2017. [Online]. Available: http://docs.oasis-
[64] J. Ordonez-Lucena, P. Ameigeiras, D. Lopez, J. J. Ramos- open.org/tosca/tosca-nfv/v1.0/tosca-nfv-v1.0.html
Munoz, J. Lorca, and J. Folgueira, “Network Slicing for 5G [83] ——, “TOSCA Simple Profile in YAML Version 1.2,” 2017.
with SDN/NFV: Concepts, Architectures and Challenges,” IEEE [Online]. Available: http://docs.oasis-open.org/tosca/TOSCA-Simple-
Communications Magazine, 3 2017. [Online]. Available: https: Profile-YAML/v1.2/TOSCA-Simple-Profile-YAML-v1.2.html
//arxiv.org/pdf/1703.04676.pdfhttp://arxiv.org/abs/1703.04676http://dx. [84] ITU Telecommunication Standardization, “FG IMT-2020: Report on
doi.org/10.1109/MCOM.2017.1600935 Standards Gap Analysis,” 2015. [Online]. Available: https://www.
[65] N. Fallenbeck, H. J. Picht, M. Smith, and B. Freisleben, “Xen and ietf.org/lib/dt/documents/LIAISON/liaison- 2016- 02- 26- itu- t- sg- 13-
the art of cluster scheduling,” in First International Workshop on ietf- ls- on- report- on- standard- gap- analysis- from- itu- t- focus- group-
Virtualization Technology in Distributed Computing (VTDC 2006), Nov on-imt-2020-and-on-extension-of-lifetime-of-focus-g-attachment-2.
2006, pp. 4–4. pdf
[66] J. P. Walters, V. Chaudhary, M. Cha, S. G. Jr., and S. Gallo, “A com- [85] ——, “Recommendation ITU-T Y.3100: Terms and definitions
parison of virtualization technologies for hpc,” in 22nd International for IMT-2020 network,” Tech. Rep., 2017. [Online]. Available:
Conference on Advanced Information Networking and Applications http://handle.itu.int/11.1002/1000/13349
(aina 2008), March 2008, pp. 861–868. [86] ——, “Recommendation ITU-T Y.3110: IMT-2020 network
[67] A. B. S., H. M. J., J. P. Martin, S. Cherian, and Y. Sastri, “System management and orchestration requirements,” ITU-T, Tech. Rep.,
performance evaluation of para virtualization, container virtualization, 2017. [Online]. Available: http://handle.itu.int/11.1002/1000/13350
and full virtualization using xen, openvz, and xenserver,” in 2014 [87] ——, “Recommendation ITU-T Y.3111: IMT-2020 network
Fourth International Conference on Advances in Computing and Com- management and orchestration framework,” Tech. Rep., 2017.
munications, Aug 2014, pp. 247–250. [Online]. Available: http://handle.itu.int/11.1002/1000/13351
[68] ETSI Industry Specification Group (ISG) NFV, “GS NFV-INF [88] I. T. Standardization, “ITU-T Y-series Recommendations Supplement
010 - V1.1.1: Network Functions Virtualisation (NFV); Service 44: Standardization and open source activities related to network
Quality Metrics,” Tech. Rep. [Online]. Available: http://www. softwarization of IMT-2020,” Tech. Rep., 2017.
etsi.org/deliver/etsi gs/NFV-INF/001 099/010/01.01.01 60/gs NFV- [89] ITU Radiocommunication, “Recommendation ITU-R M.2083-0: IMT
INF010v010101p.pdf Vision Framework and overall objectives of the future development of
[69] ——, “GR NFV-SEC 003 - V1.2.1: Network Functions Virtualisation IMT for 2020 and beyond,” 2015. [Online]. Available: https://www.itu.
(NFV); NFV Security; Security and Trust Guidance,” Tech. Rep., int/dms pubrec/itu-r/rec/m/R-REC-M.2083-0-201509-I!!PDF-E.pdf
2016. [Online]. Available: http://www.etsi.org/standards-search [90] Metro Ethernet Forum, “Understanding SD-WAN Managed Services,”
[70] ——, “GS NFV-SWA 001 - V1.1.1: Network Functions Virtualisation July 2017, accessed on 2017-10-01. [Online]. Available: https:
(NFV); Virtual Network Functions Architecture,” Tech. Rep., //www.mef.net/resources/download?id=45&fileid=file1
2014. [Online]. Available: http://www.etsi.org/deliver/etsi gs/NFV- [91] 3GPP, “TR 28.800 Release 14: Study on management and
SWA/001 099/001/01.01.01 60/gs NFV-SWA001v010101p.pdf orchestration architecture of next generation network and service,”
[71] ——, “GS NFV-IFA 010 - V2.3.1: Network Functions Virtualisation Tech. Rep., 2017, accessed 2017-11-28. [Online]. Available: http:
(NFV) Release 2; Management and Orchestration; Functional //www.3gpp.org/ftp/Specs/html-info/28800.htm
[92] International Telecommunication Union, “ITU: Committed to and Security (IUCC-CSS). IEEE, 12 2016, pp. 189–195. [Online].
connecting the world.” [Online]. Available: http://www.itu.int/home/ Available: http://ieeexplore.ieee.org/document/7828601/
[93] J. F. Riera, J. Batall, J. Bonnet, M. Das, M. McGrath, G. Petralia, [110] A. Israel, A. Hoban, A. Tierno, F. Salguero, G. Blas, K. Kashalkar,
F. Liberati, A. Giuseppi, A. Pietrabissa, A. Ceselli, A. Petrini, M. Tru- M. Ceppi, M. Shuttleworth, M. Harper, M. Marchetti, R. Velandy,
bian, P. Papadimitrou, D. Dietrich, A. Ramos, J. Melin, G. Xilouris, S. Almagia, and V. Little, “OSM Release Three: A Technical
A. Kourtis, T. Kourtis, and E. K. Markakis, “Tenor: Steps towards an Overview,” ETSI, Tech. Rep. October, 2017. [Online]. Available: https:
orchestration platform for multi-pop nfv deployment,” in 2016 IEEE //osm.etsi.org/images/OSM-Whitepaper-TechContent-ReleaseTHREE-
NetSoft Conference and Workshops (NetSoft), June 2016, pp. 243–250. FINAL.PDF
[94] M.-A. Kourtis, M. J. McGrath, G. Gardikis, G. Xilouris, V. Riccobene, [111] OpenStack Foundation, “Tacker - OpenStack,” 2016. [Online].
P. Papadimitriou, E. Trouva, F. Liberati, M. Trubian, J. Batalle, Available: https://wiki.openstack.org/wiki/Tacker
H. Koumaras, D. Dietrich, A. Ramos, J. Ferrer Riera, J. Bonnet, [112] GigaSpaces, “Cloudify,” 2015. [Online]. Available: http://cloudify.co/
A. Pietrabissa, A. Ceselli, and A. Petrini, “T-NOVA: An Open-Source [113] Linux Foundation, “Open Orchestrator.” [Online]. Available: https:
MANO Stack for NFV Infrastructures,” IEEE Transactions on Network //www.open-o.org/
and Service Management, vol. 14, no. 3, pp. 586–602, 9 2017. [114] AT&T, “ECOMP ( Enhanced Control , Orchestration , Management &
[Online]. Available: http://ieeexplore.ieee.org/document/7997799/ Policy ) Architecture White Paper,” 2016.
[95] FP7 project T-NOVA, “D2.1: System Use Cases and Requirements,” [115] L. Foundation, “ONAP Developer Wiki,” 2017, accessed 2017-09-05.
Tech. Rep. [Online]. Available: http://www.t-nova.eu/wp-content/ [Online]. Available: https://wiki.onap.org/
uploads/2014/11/TNOVA D2.1 Use Cases and Requirements.pdf [116] C. N. Digest, “The Open Network Automation Platform looks like
[96] S. Consortium, “SONATA NFV,” 2016, accessed 2017-10-04. [Online]. a turning point for telecom architecture,” 2017, accessed 2017-09-
Available: http://www.sonata-nfv.eu 05. [Online]. Available: http://www.convergedigest.com/2017/04/the-
[97] H. Karl, S. Drxler, M. Peuster, A. Galis, M. Bredel, A. Ramos, open-network-automation-platform.html
J. Martrat, M. S. Siddiqui, S. van Rossem, W. Tavernier, [117] L. Reading, “ONAP Makes Splashy ONS Debut,” 2017, accessed
and G. Xilouris, “Devops for network function virtualisation: an 2017-09-11. [Online]. Available: http://www.lightreading.com/nfv/nfv-
architectural approach,” Transactions on Emerging Telecommunications mano/onap-makes-splashy-ons-debut/d/d-id/731887
Technologies, vol. 27, no. 9, pp. 1206–1215, 2016, ett.3084. [Online]. [118] Fraunhofer and T. Berlin, “Open Baton: An open source reference
Available: http://dx.doi.org/10.1002/ett.3084 implementation of the ETSI Network Function Virtualization MANO
[98] S. Draxler, H. Karl, M. Peuster, H. R. Kouchaksaraei, M. Bredel, specification,” 2017, accessed 2017-09-11. [Online]. Available:
J. Lessmann, T. Soenen, W. Tavernier, S. Mendel-Brin, and http://openbaton.github.io/
G. Xilouris, “SONATA: Service programming and orchestration for [119] A. S. Foundation, “Apache ARIA TOSCA,” 2017, accessed 2017-09-
virtualized software networks,” in 2017 IEEE International Conference 21. [Online]. Available: http://ariatosca.incubator.apache.org/
on Communications Workshops (ICC Workshops). IEEE, 5 2017, [120] Cloudify, “A Primer on Project ARIA - Simple, Open Source TOSCA-
pp. 973–978. [Online]. Available: http://arxiv.org/abs/1605.05850http: Based Orchestration Engine,” 2016, accessed 2017-09-26. [Online].
//ieeexplore.ieee.org/document/7962785/ Available: http://cloudify.co/2016/03/15/cloud- open- networking-
[99] SONATA Consortium, “D2.2 Architecture Design,” Tech. Rep., 2015. summit-nfv-sdn-aria-tosca-application-orchestration-automation.html
[100] ——, “D2.3 Updated Requirements and Architecture Design,” Tech. [121] L. Peterson, S. Baker, M. De Leenheer, A. Bavier, S. Bhatia,
Rep., 2016. M. Wawrzoniak, J. Nelson, and J. Hartman, “Xos: An extensible
[101] H2020 5G-TRANSFORMER Project, “5G Mobile Transport Platform cloud operating system,” in Proceedings of the 2Nd International
for Verticals,” 2017. [Online]. Available: http://5g-transformer.eu/ Workshop on Software-Defined Ecosystems, ser. BigSystem ’15.
[102] ——, “Initial Communication, Dissemination, and Exploitation Plan New York, NY, USA: ACM, 2015, pp. 23–30. [Online]. Available:
(CoDEP) draft including Standardization roadmap,” Tech. Rep., 2017. http://doi.acm.org/10.1145/2756594.2756598
[Online]. Available: http://5g-transformer.eu/wp-content/uploads/2017/ [122] O. Project, “XOS: A Service Abstract Layer for CORD,” 2015,
12/Initial CoDEP draft including Standardization roadmap.pdf accessed 2017-09-26. [Online]. Available: http://xos.wpengine.com/
[103] H. V. Project, “VIrtualized hybrid satellite-TerrestriAl systems for wp-content/uploads/2015/06/Whitepaper-XOS.pdf
resilient and fLexible future networks,” 2016, accessed 2017-10-09. [123] NTT, “Gohan - REST-based api server to evolve your cloud
[Online]. Available: http://www.ict-vital.eu/ service very rapidly,” 2015, accessed 2017-09-14. [Online]. Available:
[104] ——, “D2.3 System Architecture: Final Report,” 2016, accessed http://gohan.cloudwan.io/
2017-12-05. [Online]. Available: https://drive.google.com/file/d/ [124] F. UNIFY, “UNIFY: Unifying Cloud and Carrier Networks,” 2013,
0B5yhgJbT3R8kam5DbUM1ZHNSVGs/view accessed 2017-12-01. [Online]. Available: http://www.fp7-unify.eu/
[105] A. Francescon, G. Baggio, R. Fedrizzi, R. Ferrusy, I. G. [125] A. Csoma, B. Sonkoly, L. Csikor, F. Németh, A. Gulyas, W. Tavernier,
Ben Yahiaz, and R. Riggio, “X-MANO: Cross-domain management and S. Sahhaf, “Escape: Extensible service chain prototyping
and orchestration of network services,” in 2017 IEEE Conference on environment using mininet, click, netconf and pox,” in Proceedings
Network Softwarization (NetSoft). IEEE, 7 2017, pp. 1–5. [Online]. of the 2014 ACM Conference on SIGCOMM, ser. SIGCOMM ’14.
Available: http://ieeexplore.ieee.org/document/8004223/ New York, NY, USA: ACM, 2014, pp. 125–126. [Online]. Available:
[106] R. Vilalta, A. Mayoral, R. Casellas, R. Martı́nez, and R. Muñoz, http://doi.acm.org/10.1145/2619239.2631448
“SDN / NFV Orchestration of Multi-technology and Multi-domain [126] B. Sonkoly, J. Czentye, R. Szabo, D. Jocha, J. Elek, S. Sahhaf,
Networks in Cloud / Fog Architectures for 5G Services,” 2016 W. Tavernier, and F. Risso, “Multi-domain service orchestration over
21st OptoElectronics and Communications Conference (OECC) networks and clouds: A unified approach,” ACM SIGCOMM Computer
held jointly with 2016 International Conference on Photonics Communication Review, vol. 45, no. 4, pp. 377–378, 2015.
in Switching (PS), vol. 1, pp. 1–3, 2016. [Online]. Available: [127] Sdxcentral, “Lifecycle Service Orchestration (LSO) Market Overview,”
http://ieeexplore.ieee.org/document/7718525/ Tech. Rep., 2016.
[107] K. Giotis, Y. Kryftis, and V. Maglaris, “Policy-based orchestration [128] Cisco Inc, “Network Services Orchestrator Data Sheet - Cisco.”
of NFV services in Software-Defined Networks,” in Proceedings [Online]. Available: https://www.cisco.com/c/en/us/products/collateral/
of the 2015 1st IEEE Conference on Network Softwarization cloud- systems- management/network- services- orchestrator/datasheet-
(NetSoft). IEEE, 4 2015, pp. 1–5. [Online]. Available: http: c78-734576.html
//ieeexplore.ieee.org/document/7116145/ [129] BluePlanet, “Blue Planet Software Suite,” 2017. [Online]. Available:
[108] A. Devlic, A. Hamidian, D. Liang, M. Eriksson, A. Consoli, http://media.ciena.com/documents/BP Blue Planet PB.pdf
and J. Lundstedt, “NESMO: Network slicing management and [130] Oracle Communications, “Oracle Communications Network Service
orchestration framework,” in 2017 IEEE International Conference on Orchestration Solution.” [Online]. Available: http://www.oracle.
Communications Workshops (ICC Workshops). IEEE, 5 2017, pp. com/us/industries/communications/network- service- orchestration- ds-
1202–1208. [Online]. Available: http://ieeexplore.ieee.org/document/ 2412291.pdf
7962822/ [131] Ericsson Inc., “Ericsson Network Manager.” [Online]. Available:
[109] K. Katsalis, N. Nikaein, and A. Edmonds, “Multi-Domain https://www.ericsson.com/ourportfolio/it-and-cloud-products/network-
Orchestration for NFV: Challenges and Research Directions,” in manager?nav=productcategory
2016 15th International Conference on Ubiquitous Computing and [132] A. Rostami, P. Ohlen, K. Wang, Z. Ghebretensae, B. Skubic, M. Santos,
Communications and 2016 International Symposium on Cyberspace and A. Vidal, “Orchestration of ran and transport networks for 5g: An
sdn approach,” IEEE Communications Magazine, vol. 55, no. 4, pp. World Congress. Barcelone, Spain: 5G Infrastructure association,
64–70, April 2017. Feb. 2017. [Online]. Available: https://hal.inria.fr/hal-01488208
[133] Ericsson Inc., “5g Use Cases,” Tech. Rep., 2015. [Online]. Available: [152] Yong Li and Min Chen, “Software-Defined Network Function
https://www.ericsson.com/assets/local/news/2015/7/5g-use-cases.pdf Virtualization: A Survey,” IEEE Access, vol. 3, pp. 2542–2553, 2015.
[134] D. King and A. Farrel, “A pce-based architecture for application-based [Online]. Available: http://ieeexplore.ieee.org/document/7350211/
network operations,” March 2015, rFC7491. [Online]. Available: [153] R. Mijumbi, J. Serrat, J.-l. Gorricho, S. Latre, M. Charalambides,
http://tools.ietf.org/rfc/rfc7491.txt and D. Lopez, “Management and orchestration challenges in
[135] A. S. Thyagaturu, A. Mercian, M. P. McGarry, M. Reisslein, and network functions virtualization,” IEEE Communications Magazine,
W. Kellerer, “Software defined optical networks (sdons): A comprehen- vol. 54, no. 1, pp. 98–105, 1 2016. [Online]. Available: http:
sive survey,” IEEE Communications Surveys Tutorials, vol. 18, no. 4, //ieeexplore.ieee.org/document/7378433/
pp. 2738–2786, Fourthquarter 2016. [154] Linux Foundation, “DPDK - Data Plane Development Kit.” [Online].
[136] C. Liu, Y. Mao, J. Van der Merwe, and M. Fernandez, “Cloud resource Available: https://dpdk.org/
orchestration: A data-centric approach,” in Proceedings of the biennial [155] J. Hwang, K. K. Ramakrishnan, and T. Wood, “Netvm: High per-
Conference on Innovative Data Systems Research (CIDR). Citeseer, formance and flexible networking using virtualization on commodity
2011, pp. 1–8. platforms,” IEEE Transactions on Network and Service Management,
[137] A. Tosatto, P. Ruiu, and A. Attanasio, “Container-based orchestration vol. 12, no. 1, pp. 34–47, March 2015.
in cloud: State of the art and challenges,” in 2015 Ninth International [156] J. Martins, M. Ahmed, C. Raiciu, V. Olteanu, M. Honda, R. Bifulco,
Conference on Complex, Intelligent, and Software Intensive Systems, and F. Huici, “Clickos and the art of network function virtualization,”
July 2015, pp. 70–75. in Proceedings of the 11th USENIX Conference on Networked
[138] A. Galis, “Perspectives on Network Slicing Towards the New “Bread Systems Design and Implementation, ser. NSDI’14. Berkeley, CA,
and Butter” of Networking and Servicing,” January 2018. [Online]. USA: USENIX Association, 2014, pp. 459–473. [Online]. Available:
Available: https://sdn.ieee.org/newsletter/january- 2018/perspectives- http://dl.acm.org/citation.cfm?id=2616448.2616491
on-network-slicing-towards-the-new-bread-and-butter-of-networking- [157] ETSI Industry Specification Group (ISG) NFV, “GS NFV-PER
and-servicing 001 - V1.1.1: Network Functions Virtualisation (NFV) NFV
[139] 3GPP TR 28.801 v2.0.1. (2017) Study on management and Performance & Portability Best Practises,” 2014. [Online].
orchestration of network slicing for next generation network. Accessed Available: http://www.etsi.org/deliver/etsi gs/NFV-PER/001 099/001/
on 2017-12-01. [Online]. Available: http://www.3gpp.org/ftp/Specs/ 01.01.01 60/gs nfv-per001v010101p.pdf
html-info/28801.htm [158] R. V. Rosa, C. E. Rothenberg, and R. Szabo, “Vbaas: Vnf benchmark-
[140] L. M. Contreras and D. R. Lopez, “A Network Service as-a-service,” in 2015 Fourth European Workshop on Software Defined
Provider Perspective on Network Slicing,” January 2018. [Online]. Networks, Sept 2015, pp. 79–84.
Available: https://sdn.ieee.org/newsletter/january- 2018/a- network- [159] M. Peuster and H. Karl, “Understand your chains: Towards per-
service-provider-perspective-on-network-slicing formance profile-based network service management,” in 2016 Fifth
[141] T. Wu, L. Rui, A. Xiong, and S. Guo, “An automation pci allocation European Workshop on Software-Defined Networks (EWSDN), Oct
method for enodeb and home enodeb cell,” in 2010 6th International 2016, pp. 7–12.
Conference on Wireless Communications Networking and Mobile Com- [160] L. C. Hoyos and C. E. Rothenberg, “Non: Network function virtualiza-
puting (WiCOM), Sept 2010, pp. 1–4. tion ontology towards semantic service implementation,” in 2016 8th
[142] R. Zhang, F. Yan, W. Xia, S. Xing, Y. Wu, and L. Shen, “An IEEE Latin-American Conference on Communications (LATINCOM),
optimal roadside unit placement method for vanet localization,” in Nov 2016, pp. 1–6.
GLOBECOM 2017 - 2017 IEEE Global Communications Conference, [161] Y. Dong, X. Yang, X. Li, J. Li, K. Tian, and H. Guan, “High perfor-
Dec 2017, pp. 1–6. mance network virtualization with sr-iov,” in HPCA - 16 2010 The
[143] J. Gubbi, R. Buyya, S. Marusic, and M. Palaniswami, “Internet of Sixteenth International Symposium on High-Performance Computer
Things (IoT): A vision, architectural elements, and future directions,” Architecture, Jan 2010, pp. 1–10.
Future Generation Computer Systems, vol. 29, no. 7, pp. 1645–1660, [162] D. Calcaterra, V. Cartelli, G. D. Modica, and O. Tomarchio,
2013. [Online]. Available: http://dx.doi.org/10.1016/j.future.2013.01. “Combining TOSCA and BPMN to enable automated cloud service
010 provisioning,” in CLOSER 2017 - Proceedings of the 7th International
[144] C. Consel and M. Kabac, “Internet of Things: From Small- to Large- Conference on Cloud Computing and Services Science, Porto,
Scale Orchestration,” in 2017 IEEE 37th International Conference Portugal, April 24-26, 2017., 2017, pp. 159–168. [Online]. Available:
on Distributed Computing Systems (ICDCS). IEEE, 6 2017, pp. https://doi.org/10.5220/0006304701590168
1748–1755. [Online]. Available: http://ieeexplore.ieee.org/document/
7980112/
[145] Z. Wen, R. Yang, P. Garraghan, T. Lin, J. Xu, and M. Rovatsos,
“Fog orchestration for Internet of Things services,” IEEE Internet
Computing, vol. 21, no. 2, pp. 16–24, 2017.
[146] N. Panwar, S. Sharma, and A. K. Singh, “A survey on 5G: The
next generation of mobile communication,” Physical Communication,
vol. 18, pp. 64–84, 3 2016. [Online]. Available: http://linkinghub.
elsevier.com/retrieve/pii/S1874490715000531
[147] D. Evans, “The Internet of Things - How the Next Evolution of the
Internet is Changing Everything,” CISCO white paper, no. April, pp. Nathan F. Saraiva de Sousa has a BSc and an
1–11, 2011. [Online]. Available: http://scholar.google.com/scholar? MSc in Computer Science from Federal University
hl=en&btnG=Search&q=intitle:The+Internet+of+Things+-+How+the+ of Piaui (UFPI), in 2005 and 2015, respectively. He
Next+Evolution+of+the+Internet+is+Changing+Everything#0 is currently a Ph.D. student at the Department of
[148] J. Garay, J. Matias, J. Unzilla, and E. Jacob, “Service description Computer Engineering and Industrial Automation of
in the NFV revolution: Trends, challenges and a way forward,” the School of Electrical and Computer Engineering
IEEE Communications Magazine, vol. 54, no. 3, pp. 68–74, 3 2016. UNICAMP. He works in a project funded by Eric-
[Online]. Available: http://ieeexplore.ieee.org/document/7432174/ sson Research Brazil. Since 2010 he is a Computer
[149] G. Arfaoui, J. M. S. Vilchez, and J.-p. Wary, “Security and Resilience Analyst at the Federal Institute of Piaui (IFPI). His
in 5G: Current Challenges and Future Directions,” in 2017 IEEE research interests span SDN, NFV, and multi-domain
Trustcom/BigDataSE/ICESS. IEEE, 8 2017, pp. 1010–1015. [Online]. orchestration.
Available: http://ieeexplore.ieee.org/document/8029548/
[150] B. Jaeger, “Security orchestrator: Introducing a security orchestrator
in the context of the etsi nfv reference architecture,” in 2015 IEEE
Trustcom/BigDataSE/ISPA, vol. 1, Aug 2015, pp. 1255–1260.
[151] S.-E. Elayoubi, J.-S. Bedo, M. Filippou, A. Gavras, D. Giustiniano,
P. Iovanna, A. Manzalini, O. Queseth, T. Rokkas, M. Surridge, and
T. Tjelta, “5G innovations for new business opportunities,” in Mobile
Danny Alex Lachos Perez is a Ph.D. student in

the Faculty of Electrical and Computer Engineer-
ing at University of Campinas (UNICAMP). He is
involved as a researcher in the 5G Multi-Domain
Orchestration project supported by the Innovation
Center of Ericsson Telecommunication S.A (Brazil).
His current research interests are 5G Networks,
Multi-domain Orchestration, SDN, NFV, Machine
Learning and Graph Database. He holds a Master
degree in Computer Engineering from University of
Campinas, Brazil, 2016 and he also received the
Computer Engineering degree from Pedro Ruiz Gallo University, Perú, 2004.
Raphael V. Rosa Currently pursues

his thesis on multi-domain distributed
NFV as a PhD student in University of
Campinas, Brazil. During the last two
years, he worked as a visiting researcher
in Ericsson Research Hungary, where he
contributed to EU-FP7 Unify project and
developed activities within H2020 5G
Exchange project. His main interests sit
on state-of-the-art SDN and NFV research
topics.
Mateus A. S. Santos received his Ph.D. from

University of Sao Paulo (USP) in 2014. From 2013
to 2014 he was a research scholar with the Inter-
Networking Research Group at UC Santa Cruz. He
was also a postdoctoral researcher with University
of Campinas (UNICAMP) from 2014 to 2016. His
research interests are in software-defined network-
ing, network functions virtualization and network
security. He is currently with Ericsson Research
Brazil.
Christian Esteve Rothenberg is an Assistant

Professor in the Faculty of Electrical & Computer
Engineering (FEEC) at University of Campinas
(UNICAMP), Brazil, where he received his Ph.D.
and currently leads the Information & Networking
Technologies Research & Innovation Group
(INTRIG). His research activities span all layers
of distributed systems and network architectures
and are often carried in collaboration with industry,
resulting in multiple open source projects in SDN
and NFV among other scientific results. Christian
has two international patents and over 100 publications, including scientific
journals and top-tier networking conferences such as SIGCOMM and
INFOCOM.

Overview On Network Orchestration

Uploaded by

Copyright:

Available Formats

Overview On Network Orchestration

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Overview On Network Orchestration

Uploaded by

Copyright:

Available Formats

IEEE COMMUNICATIONS SURVEYS & TUTORIALS 1

Network Service Orchestration: A Survey

T ELECOMMUNICATION infrastructures consist of a

Fig. 2. Summarized view of this survey on NSO.

in small part, after assigns to different musical instruments.

Fig. 5. Relationship among orchestration, management, and automation. Both

supposed to provide service delivery both within single and/or

(a) (b) (c)

The Lifecycle Orchestration deals with the management of

SDO Working Group Scope Outcomes

Class Feature T-Nova Unify 5GEx SONATA VITAL 5G-T

Resource Domain MANO Interface Management Domain

ARIA Apache Founda- TOSCA 3 3 3 3

OpenStack). Those logically programmable network fulfill- E. Intelligent Transport System

Nevertheless, the network services can have specific life- ACKNOWLEDGMENT

Danny Alex Lachos Perez is a Ph.D. student in

Raphael V. Rosa Currently pursues

Mateus A. S. Santos received his Ph.D. from

Christian Esteve Rothenberg is an Assistant

You might also like