Getting Started

Download as pdf or txt
Download as pdf or txt
You are on page 1of 56

vMX

Getting Started Guide

Release

14.1R6

Modified: 2015-10-29

Copyright 2015, Juniper Networks, Inc.

Juniper Networks, Inc.


1133 Innovation Way
Sunnyvale, California 94089
USA
408-745-2000
www.juniper.net
Copyright 2015, Juniper Networks, Inc. All rights reserved.
Juniper Networks, Junos, Steel-Belted Radius, NetScreen, and ScreenOS are registered trademarks of Juniper Networks, Inc. in the United
States and other countries. The Juniper Networks Logo, the Junos logo, and JunosE are trademarks of Juniper Networks, Inc. All other
trademarks, service marks, registered trademarks, or registered service marks are the property of their respective owners.
Juniper Networks assumes no responsibility for any inaccuracies in this document. Juniper Networks reserves the right to change, modify,
transfer, or otherwise revise this publication without notice.

vMX Getting Started Guide


14.1R6
Copyright 2015, Juniper Networks, Inc.
All rights reserved.
The information in this document is current as of the date on the title page.
YEAR 2000 NOTICE
Juniper Networks hardware and software products are Year 2000 compliant. Junos OS has no known time-related limitations through the
year 2038. However, the NTP application is known to have some difficulty in the year 2036.

END USER LICENSE AGREEMENT


The Juniper Networks product that is the subject of this technical documentation consists of (or is intended for use with) Juniper Networks
software. Use of such software is subject to the terms and conditions of the End User License Agreement (EULA) posted at
http://www.juniper.net/support/eula.html. By downloading, installing or using such software, you agree to the terms and conditions of
that EULA.

ii

Copyright 2015, Juniper Networks, Inc.

Table of Contents
About the Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Documentation and Release Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Documentation Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Documentation Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
Requesting Technical Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii
Self-Help Online Tools and Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . xii
Opening a Case with JTAC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

Chapter 1

vMX Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
vMX Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Benefits and Uses of vMX Routers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Architecture of a vMX Instance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Virtualization Techniques for the vMX VFP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Paravirtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
PCI Passthrough with SR-IOV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Automation for vMX Routers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Chapter 2

Setting Up vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Minimum Hardware and Software Requirements . . . . . . . . . . . . . . . . . . . . . . . . . 23
vMX Package Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Chapter 3

Installing vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Preparing the System to Install vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Upgrading the Kernel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Upgrading to libvirt 1.2.8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Installing vMX for Different Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Installing vMX for Lab Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Installing vMX for Low-Bandwidth Applications . . . . . . . . . . . . . . . . . . . . . . . 31
Installing vMX for High-Bandwidth Applications . . . . . . . . . . . . . . . . . . . . . . . 33
Configuring the vMX Instance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Configuring the Host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Configuring the VCP VM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Configuring the VFP VM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Configuring Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Deploying vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Connecting to VMs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Logging In to VCP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Logging In to VFP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

Copyright 2015, Juniper Networks, Inc.

iii

Getting Started Guide

Managing vMX Licenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39


Adding a License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Deleting a License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
Viewing the Chassis Serial ID for vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

Chapter 4

Managing vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
Managing vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
Controlling vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
Deploying vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
Managing vMX Deployments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Specifying the Temporary File Directory . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Specifying the Environment File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Configuring Logging Options for vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Connecting to Console Port for the VMs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Getting Help for the Script Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Binding virtio Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
Setting Up the Device Bindings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Creating Device Bindings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
Deleting Device Bindings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Verifying Device Bindings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

Chapter 5

Troubleshooting vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Viewing VFP Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Viewing VFP Log Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Troubleshooting VFP and VCP Connection Establishment . . . . . . . . . . . . . . . . . . 53
Verifying Whether VMs Are Running . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
Viewing CPU Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
Verifying BIOS Settings for SR-IOV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

iv

Copyright 2015, Juniper Networks, Inc.

List of Figures
Chapter 1

vMX Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Figure 1: Architecture of a vMX Instance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Chapter 4

Managing vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
Figure 2: Binding Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

Copyright 2015, Juniper Networks, Inc.

Getting Started Guide

vi

Copyright 2015, Juniper Networks, Inc.

List of Tables
About the Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Table 1: Notice Icons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x
Table 2: Text and Syntax Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x

Chapter 1

vMX Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Table 3: Considerations for Choosing a Virtualization Technique . . . . . . . . . . . . . . 18
Table 4: Application Packages for Licenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Chapter 2

Setting Up vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Table 5: Minimum Hardware Requirements for vMX . . . . . . . . . . . . . . . . . . . . . . . 23
Table 6: Minimum Software Requirements for vMX . . . . . . . . . . . . . . . . . . . . . . . . 24
Table 7: vMX Package Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Chapter 3

Installing vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Table 8: Sample Configurations for Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

Copyright 2015, Juniper Networks, Inc.

vii

Getting Started Guide

viii

Copyright 2015, Juniper Networks, Inc.

About the Documentation

Documentation and Release Notes on page ix

Documentation Conventions on page ix

Documentation Feedback on page xi

Requesting Technical Support on page xii

Documentation and Release Notes

To obtain the most current version of all Juniper Networks technical documentation,
see the product documentation page on the Juniper Networks website at
http://www.juniper.net/techpubs/.
If the information in the latest release notes differs from the information in the
documentation, follow the product Release Notes.
Juniper Networks Books publishes books by Juniper Networks engineers and subject
matter experts. These books go beyond the technical documentation to explore the
nuances of network architecture, deployment, and administration. The current list can
be viewed at http://www.juniper.net/books.

Documentation Conventions
Table 1 on page x defines notice icons used in this guide.

Copyright 2015, Juniper Networks, Inc.

ix

Getting Started Guide

Table 1: Notice Icons


Icon

Meaning

Description

Informational note

Indicates important features or instructions.

Caution

Indicates a situation that might result in loss of data or hardware damage.

Warning

Alerts you to the risk of personal injury or death.

Laser warning

Alerts you to the risk of personal injury from a laser.

Tip

Indicates helpful information.

Best practice

Alerts you to a recommended use or implementation.

Table 2 on page x defines the text and syntax conventions used in this guide.

Table 2: Text and Syntax Conventions


Convention

Description

Examples

Bold text like this

Represents text that you type.

To enter configuration mode, type the


configure command:
user@host> configure

Fixed-width text like this

Italic text like this

Italic text like this

Represents output that appears on the


terminal screen.

user@host> show chassis alarms

Introduces or emphasizes important


new terms.

Identifies guide names.

A policy term is a named structure


that defines match conditions and
actions.

Identifies RFC and Internet draft titles.

Junos OS CLI User Guide

RFC 1997, BGP Communities Attribute

Represents variables (options for which


you substitute a value) in commands or
configuration statements.

No alarms currently active

Configure the machines domain name:


[edit]
root@# set system domain-name
domain-name

Copyright 2015, Juniper Networks, Inc.

About the Documentation

Table 2: Text and Syntax Conventions (continued)


Convention

Description

Examples

Text like this

Represents names of configuration


statements, commands, files, and
directories; configuration hierarchy levels;
or labels on routing platform
components.

To configure a stub area, include the


stub statement at the [edit protocols
ospf area area-id] hierarchy level.

The console port is labeled CONSOLE.

< > (angle brackets)

Encloses optional keywords or variables.

stub <default-metric metric>;

| (pipe symbol)

Indicates a choice between the mutually


exclusive keywords or variables on either
side of the symbol. The set of choices is
often enclosed in parentheses for clarity.

broadcast | multicast

# (pound sign)

Indicates a comment specified on the


same line as the configuration statement
to which it applies.

rsvp { # Required for dynamic MPLS only

[ ] (square brackets)

Encloses a variable for which you can


substitute one or more values.

community name members [


community-ids ]

Indention and braces ( { } )

Identifies a level in the configuration


hierarchy.

; (semicolon)

Identifies a leaf statement at a


configuration hierarchy level.

(string1 | string2 | string3)

[edit]
routing-options {
static {
route default {
nexthop address;
retain;
}
}
}

GUI Conventions
Bold text like this

Represents graphical user interface (GUI)


items you click or select.

> (bold right angle bracket)

Separates levels in a hierarchy of menu


selections.

In the Logical Interfaces box, select


All Interfaces.

To cancel the configuration, click


Cancel.

In the configuration editor hierarchy,


select Protocols>Ospf.

Documentation Feedback
We encourage you to provide feedback, comments, and suggestions so that we can
improve the documentation. You can provide feedback by using either of the following
methods:

Online feedback rating systemOn any page at the Juniper Networks Technical
Documentation site at http://www.juniper.net/techpubs/index.html, simply click the
stars to rate the content, and use the pop-up form to provide us with information about
your experience. Alternately, you can use the online feedback form at
http://www.juniper.net/techpubs/feedback/.

Copyright 2015, Juniper Networks, Inc.

xi

Getting Started Guide

E-mailSend your comments to [email protected]. Include the document


or topic name, URL or page number, and software version (if applicable).

Requesting Technical Support


Technical product support is available through the Juniper Networks Technical Assistance
Center (JTAC). If you are a customer with an active J-Care or Partner Support Service
support contract, or are covered under warranty, and need post-sales technical support,
you can access our tools and resources online or open a case with JTAC.

JTAC policiesFor a complete understanding of our JTAC procedures and policies,


review the JTAC User Guide located at
http://www.juniper.net/us/en/local/pdf/resource-guides/7100059-en.pdf.

Product warrantiesFor product warranty information, visit


http://www.juniper.net/support/warranty/.

JTAC hours of operationThe JTAC centers have resources available 24 hours a day,
7 days a week, 365 days a year.

Self-Help Online Tools and Resources


For quick and easy problem resolution, Juniper Networks has designed an online
self-service portal called the Customer Support Center (CSC) that provides you with the
following features:

Find CSC offerings: http://www.juniper.net/customers/support/

Search for known bugs: http://www2.juniper.net/kb/

Find product documentation: http://www.juniper.net/techpubs/

Find solutions and answer questions using our Knowledge Base: http://kb.juniper.net/

Download the latest versions of software and review release notes:


http://www.juniper.net/customers/csc/software/

Search technical bulletins for relevant hardware and software notifications:


http://kb.juniper.net/InfoCenter/

Join and participate in the Juniper Networks Community Forum:


http://www.juniper.net/company/communities/

Open a case online in the CSC Case Management tool: http://www.juniper.net/cm/

To verify service entitlement by product serial number, use our Serial Number Entitlement
(SNE) Tool: https://tools.juniper.net/SerialNumberEntitlementSearch/

Opening a Case with JTAC


You can open a case with JTAC on the Web or by telephone.

xii

Use the Case Management tool in the CSC at http://www.juniper.net/cm/.

Call 1-888-314-JTAC (1-888-314-5822 toll-free in the USA, Canada, and Mexico).

Copyright 2015, Juniper Networks, Inc.

About the Documentation

For international or direct-dial options in countries without toll-free numbers, see


http://www.juniper.net/support/requesting-support.html.

Copyright 2015, Juniper Networks, Inc.

xiii

Getting Started Guide

xiv

Copyright 2015, Juniper Networks, Inc.

CHAPTER 1

vMX Overview

vMX Overview on page 15

Benefits and Uses of vMX Routers on page 16

Architecture of a vMX Instance on page 17

Virtualization Techniques for the vMX VFP on page 18

Automation for vMX Routers on page 20

Licensing on page 20

vMX Overview
The vMX router is a virtual version of the MX Series 3D Universal Edge Router. Like the
MX Series router, the vMX router runs the Junos Operating System (Junos OS) and
supports Junos OS packet handling and forwarding modeled after the Trio chipset.
Configuration and management of vMX routers are the same as for physical MX Series
routers, allowing you to add the vMX router to a network without having to update your
operations support systems (OSS).
You install vMX software components on an industry-standard x86 server running a Linux
operating system, applicable third-party software, and the Kernel-based Virtual Machine
(KVM) hypervisor. vMX software components come in one software package that you
install by running an orchestration script included with the package. The orchestration
script uses a configuration file that you customize for your vMX deployment. You can
install multiple vMX instances on one server.

Related
Documentation

Architecture of a vMX Instance on page 17

Minimum Hardware and Software Requirements on page 23

Copyright 2015, Juniper Networks, Inc.

15

Getting Started Guide

Benefits and Uses of vMX Routers


You can use virtual devices to lower your capital expenditure and operating costs,
sometimes through automating network operations. Even without automation, use of
the vMX application on standard x86 servers enables you to:

Quickly introduce new services

More easily deliver customized and personalized services to customers

Scale operations to push IP services closer to customers or to manage network growth


when growth forecasts are low or uncertain

Quickly expand service offerings into new sites

A well designed automation strategy decreases costs as well as increasing network


efficiency. By automating network tasks with the vMX router, you can:

Simplify network operations

Quickly deploy new vMX instances

Efficiently install a default Junos OS configuration on all or selected vMX instances

Quickly reconfigure existing vMX routers

You can deploy the vMX router to meet some specific network edge requirements, such
as:

Related
Documentation

16

Network simulation

Temporary deployment until a physical MX Series router is available

Automation for vMX Routers on page 20

Copyright 2015, Juniper Networks, Inc.

Chapter 1: vMX Overview

Architecture of a vMX Instance


Figure 1 on page 17 illustrates the architecture of a single vMX instance. Understanding
this architecture can help you plan your vMX configuration and choose configuration
values for the orchestration script. As with many networking architectures, the vMX
architecture is organized in layers:

The vMX router at the top

Linux, third-party software, and the KVM hypervisor in the middle layer

The x86 server in the physical layer at the bottom

Figure 1: Architecture of a vMX Instance

The server at the physical layer contains the physical NICs, CPUs, memory, and Ethernet
management port. The middle layer contains the Linux operating system, applicable
third-party software, and the hypervisor. The physical and middle layers form the host.
The vMX instance at the top layer contains two separate VMs for the virtual forwarding
plane (VFP) and the virtual control plane (VCP). The VFP VM runs the virtual Trio
forwarding plane software and the VCP VM runs Junos OS.
The hypervisor presents the physical NIC to the VFP VM as a virtual NIC. The orchestration
script maps each virtual NIC to a vMX interface that you specify in the configuration file.
After you run the orchestration script and the vMX instance is created, you use the Junos
OS CLI to configure these vMX interfaces in the VCP. The vMX router supports the
following types of interfaces:

Gigabit Ethernet (ge)

10-Gigabit Ethernet (xe)

100-Gigabit Ethernet (et)

Copyright 2015, Juniper Networks, Inc.

17

Getting Started Guide

The VCP VM and VFP VM require Layer 2 connectivity to communicate with each other.
The vMX orchestration script automatically configures an internal bridge for each vMX
instance to enable this communication. Because the internal bridge is local to the server,
you do not need to specify any configuration values, such as IP addresses.
The VCP VM and VFP VM also require Layer 2 connectivity to communicate with the
Ethernet management port on the server. You must specify virtual Ethernet interfaces
with unique IP addresses and MAC addresses for both the VFP and VCP in the
configuration file to enable the orchestration script to set up an external bridge for a vMX
instance. Ethernet management traffic for all vMX instances enters the server through
the Ethernet management port.
The way network traffic passes from the physical NIC to the virtual NIC depends on the
virtualization technique that you configure.
Related
Documentation

Virtualization Techniques for the vMX VFP on page 18

Virtualization Techniques for the vMX VFP


In a virtual environment, the hypervisor emulates the physical components in the host,
enabling the application in the VM to operate independently from the host. For a routing
application, full emulation of the underlying host is not optimal for packet throughput,
because the hypervisor must create at least two copies of a packet before the application
in the VM receives it. Two techniques are available to improve packet I/O performance
in a virtual environment:

ParavirtualizationVirtualization technique in which a software component similar to


the underlying hardware component resides in the VM and interacts with the hypervisor
to execute many operations. In contrast to full virtualization, this technique reduces
the overhead of virtualization in the VM.

PCI passthroughVirtualization technique in which a physical PCI device is directly


connected to a VM, bypassing the hypervisor. Drivers in the VM can directly access the
PCI device, resulting in a high rate of data transfer.

Choose a technique based on how you want to use the vMX router. SeeTable 3 on page 18.

Table 3: Considerations for Choosing a Virtualization Technique


Consideration
Use Cases

Paravirtualization Technique

PCI Passthrough Technique

Network simulation

Static vMX deployments

Low-throughput applications

High-throughput applications

VM Mobility

Moving vMX instance to a new server without


reconfiguration

Creating an identical vMX instance on a new


server

Host Requirements

No requirements specific to this technique

Physical NIC must support PCI passthrough

18

Copyright 2015, Juniper Networks, Inc.

Chapter 1: vMX Overview

Paravirtualization
In a paravirtualized router, the VM and the host work together to efficiently move packets
from the physical NIC to the application in the VM. You implement paravirtualization on
the vMX router by configuring virtio, a technique that the KVM hypervisor supports that
optimizes network and disk operations for the VM. Both the VFP VM and the host contain
virtio drivers that interact to move packets. You must provide the following information
in the configuration file for each vMX interface:

Junos OS name

Unique MAC address

If you want to move the VM from one server to another, you can do so without
reconfiguration, provided the names and MAC addresses of each interface remain the
same.

PCI Passthrough with SR-IOV


The vMX router supports PCI passthrough in combination with single root I/O virtualization
(SR-IOV). In the PCI passthrough technique, you directly assign a NICs memory space
to a VM, enabling packets to bypass the hypervisor. Bypassing the hypervisor increases
efficiency and results in high throughput of packets.
With SR-IOV, the hypervisor detects the physical NICs (known as a physical functions)
and creates multiple virtual NICs (known as virtual functions) in the VFP VM. In the vMX
implementation, the host dedicates a NIC to a single VM.
When you configure PCI passthrough with SR-IOV, you specify the following parameters
for each vMX interface:

Junos OS name

Unique MAC address

Name of the physical NIC

Because you create a direct connection between a virtual NIC and a physical NIC, you
cannot move a VM from one host to another. If you need to move a VM to another host,
you must install a new vMX instance on that host, and delete the vMX instance on the
original host.
Related
Documentation

Installing vMX for Different Use Cases on page 29

Configuring the vMX Instance on page 35

Minimum Hardware and Software Requirements on page 23

Copyright 2015, Juniper Networks, Inc.

19

Getting Started Guide

Automation for vMX Routers


Automating network tasks simplifies network configuration, provisioning, and
maintenance. Because the vMX software uses the same Junos OS software as MX Series
routers, and other Juniper Networks routing devices, vMX supports the same automation
tools as Junos OS. In addition, you can use standard automation tools to deploy the vMX,
as you do other virtualized software.

Related
Documentation

Benefits and Uses of vMX Routers on page 16

Licensing
Licenses are required for using vMX features. When you order licenses, this information
is bound to a customer ID. If you did not order the licenses, contact your account team
or Juniper Networks Customer Care for assistance. When you order a license, you receive
instructions for generating license activation keys on the Juniper Networks License
Management System.
The vMX licenses are based on application packages and processing capacity.
Table 4 on page 20 describes the features available with application packages.

Table 4: Application Packages for Licenses


Application Package

Features

BASE

IP routing with 32,000 routes in the forwarding table


Basic Layer 2 functionality, Layer 2 bridging and switching

ADVANCE

Features in the BASE application package


IP routing with routes up to platform scale in the forwarding table
IP and MPLS switching for unicast and multicast applications
Layer 2 features include Layer 2 VPN, VPLS, EVPN, and Layer 2 Circuit
VXLAN

PREMIUM

Features in the BASE and ADVANCE application packages


Layer 3 VPN for IP and multicast

An application package is associated with a bandwidth license. vMX provides full-duplex


bandwidth in the following capacities: 100 Mbps, 250 Mbps, 500 Mbps, 1 Gbps, 5 Gbps,
10 Gbps, and 40 Gbps. Bandwidth licenses that are not associated with a specific
application package apply to all application packages.

20

Copyright 2015, Juniper Networks, Inc.

Chapter 1: vMX Overview

You can download the vMX software BASE application package with 1 Mbps bandwidth
and evaluate it for 30 days without a license. To use additional features beyond the 30
days, you must order the appropriate license.
If you need to move your vMX installation to another host, you must remove vMX from
the current host before installing vMX and adding the license on the new host.
Related
Documentation

Managing vMX Licenses on page 39

Copyright 2015, Juniper Networks, Inc.

21

Getting Started Guide

22

Copyright 2015, Juniper Networks, Inc.

CHAPTER 2

Setting Up vMX

Minimum Hardware and Software Requirements on page 23

vMX Package Contents on page 24

Minimum Hardware and Software Requirements


Table 5 on page 23 lists the hardware requirements.

Table 5: Minimum Hardware Requirements for vMX


Description

Value

Sample system configuration

For lab simulation and low performance (less than 100 Mbps) use cases, any x86 processor
(Intel or AMD) with VT-d capability. Be sure to specify the vPFE_lite image in the vmx.conf
file.
For all other use cases, Intel Ivy Bridge processors or later are required.
Example of Ivy Bridge processor: Intel Xeon E5-2667 v2 @ 3.30 GHz 25 MB Cache
For single root I/O virtualization (SR-IOV) NIC type, use Intel 82599-based PCI-Express cards
(10 Gbps) and Ivy Bridge processors.

Number of cores

For lab simulation use case: 2 (1 for VCP and 1 for VFP)
For low-bandwidth applications: 4 (1 for VCP and 3 for VFP)
For high-bandwidth applications: 5 (1 for VCP and 4 for VFP)

Memory

Minimum: 8 GB (2 GB for VCP, 6 GB for VFP)


Additional 2 GB recommended for host OS

Storage

Local or NAS

Other requirements

Intel VT-d capability

Table 6 on page 24 lists the software requirements.

Copyright 2015, Juniper Networks, Inc.

23

Getting Started Guide

Table 6: Minimum Software Requirements for vMX


Description

Value

Operating system

Ubuntu 14.04 LTS (recommended host OS)


Linux 3.13.0-32-generic

Virtualization

QEMU-KVM 2.0.0+dfsg-2ubuntu1.11 or later

Required packages

bridge-utils qemu-kvm libvirt-bin python python-netifaces vnc4server


libyaml-dev python-yaml numactl libparted0-dev libpciaccess-dev
libnuma-dev libyajl-dev libxml2-dev libglib2.0-dev libnl-dev libnl-dev
python-pip python-dev libxml2-dev libxslt-dev

NOTE: Other additional packages might be


required to satisfy all dependencies.

NOTE: libvirt 1.2.8

NOTE: Use the apt-get install pkg name command to install a package.

Related
Documentation

Preparing the System to Install vMX on page 27

Installing vMX for Different Use Cases on page 29

vMX Package Contents


Table 7 on page 24 lists the contents of the vMX package.

Table 7: vMX Package Contents


Filename

Description

images/jinstall-vmx*.img

Software image file for VCP.

images/vmxhdd.img

Software image file for VCP file storage.

images/vPFE_*.img

Software image file for VFP (performance version).

images/vPFE_lite_*.img

Software image file for VFP (lite version). Use this image for lab simulation and
low performance (less than 100 Mbps) applications.

config/vmx.conf

Configuration file for defining vMX parameters. See Configuring the vMX Instance
on page 35 for more information.

config/vmx-junosdev.conf

Configuration file for binding devices (for virtio NICs).

env/

OS environment settings.

scripts/

Juniper Networks orchestration scripts.

24

Copyright 2015, Juniper Networks, Inc.

Chapter 2: Setting Up vMX

Table 7: vMX Package Contents (continued)


Filename

Description

vmx.sh

Main orchestration script. See Managing vMX on page 43 for more information
about command options.
NOTE: Only English locale is supported for using the vmx.sh script.

The vMX package consists of the following components:


build
config
vmx.conf
vmx-junosdev.conf
docs
drivers
ixgbe-3.19.1
env
images
jinstall64-vmx-14.1R5.2-domestic.img
jinstall64-vmx-14.1R5.2-domestic-signed.img
vmxhdd.img
vPFE_20150223.img
vPFE_lite_20150223.img
scripts
common
junosdev-bind
kvm
templates
vmx.sh

The image names might differ from the sample ones listed here.

NOTE: Modified IXGBE drivers are included in the package. Multicast


promiscuous mode for Virtual Functions is needed to receive control traffic
that comes with broadcast MAC addresses. The reference driver does not
come with this mode set, so the IXGBE drivers in this package contain certain
modifications to overcome this limitation.

Related
Documentation

Installing vMX for Different Use Cases on page 29

Configuring the vMX Instance on page 35

Managing vMX on page 43

Copyright 2015, Juniper Networks, Inc.

25

Getting Started Guide

26

Copyright 2015, Juniper Networks, Inc.

CHAPTER 3

Installing vMX

Preparing the System to Install vMX on page 27

Installing vMX for Different Use Cases on page 29

Configuring the vMX Instance on page 35

Deploying vMX on page 37

Connecting to VMs on page 38

Managing vMX Licenses on page 39

Viewing the Chassis Serial ID for vMX on page 40

Preparing the System to Install vMX


To prepare the host system for installing vMX:
1.

Meet the minimum software and OS requirements described in Table 6 on page 24.
See Upgrading the Kernel on page 28 and Upgrading to libvirt 1.2.8 on page 28.

2. Enable Intel VT-d in BIOS. (We recommend that you verify the process with the vendor

because different systems have different methods to enable VT-d.)


Refer to the procedure to enable VT-d available on the Intel Website.
3. If you are using SR-IOV, you must perform this step.

NOTE: You must remove any previous installation with an external bridge
in /etc/network/interfaces and revert to using the original management
interface. Make sure that the ifconfig -a command does not show external
bridges before you proceed with the installation.
To determine whether an external bridge is displayed, use the ifconfig
command to see the management interface. To confirm that this interface
is used for an external bridge group, use the brctl show command to see
whether the management interface is listed as an external bridge.

Enable SR-IOV capability by turning on intel_iommu=on in the /etc/default/grub


directory.
GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on"

Copyright 2015, Juniper Networks, Inc.

27

Getting Started Guide

Append the intel_iommu=on string to any existing text for the


GRUB_CMDLINE_LINUX_DEFAULT parameter.
Run the update-grub command followed by the reboot command.
4. Run the modprobe kvm-intel command before you install vMX.

To meet the minimum software and OS requirements, you might need to perform these
tasks:

Upgrading the Kernel on page 28

Upgrading to libvirt 1.2.8 on page 28

Upgrading the Kernel


To upgrade the kernel:
1.

Run the following commands:


apt-get install linux-firmware
apt-get install linux-image-3.13.0.32-generic
apt-get install linux-image-extra-3.13.0.32-generic
apt-get install linux-headers-3.13.0.32-generic

2. Restart the system.

NOTE: If you are using Ubuntu 14.04.1 LTS, which comes with
3.13.0-32-generic, you can skip this step. Ubuntu 14.04 comes with a lower
version of kernel (Linux 3.13.0-24-generic) than the recommended version
(Linux 3.13.0-32-generic).

Upgrading to libvirt 1.2.8


Ubuntu 14.04 supports libvirt 1.2.2 (which works for VFP lite version). If you are using the
VFP performance version, you must upgrade to libvirt 1.2.8.
To upgrade libvirt:
1.

Make sure that you install all the packages listed in Table 6 on page 24.

2. Navigate to the /tmp directory using the cd /tmp command.


3. Get the libvirt-1.2.8 source code by using the command

wget http://libvirt.org/sources/libvirt-1.2.8.tar.gz.
4. Uncompress and untar the file using the tar xzvf libvirt-1.2.8.tar.gz command.
5. Navigate to the libvirt-1.2.8 directory using the cd libvirt-1.2.8 command.
6. Stop libvirtd with the service libvirt-bin stop command.
7. Run the ./configure --prefix=/usr --localstatedir=/ --with-numactl command.
8. Run the make command.
9. Run the make install command.

28

Copyright 2015, Juniper Networks, Inc.

Chapter 3: Installing vMX

10. Make sure that the libvirtd daemon is running. (Use the service libvirt-bin start command

to start it again. If it does not start, use the /usr/sbin/libvirtd -d command.)


root@vmx-server:~# ps aux | grep libvirtd
root 1509 0.0 0.0 372564 16452 ?
Sl 10:25 0:00 /usr/sbin/libvirtd -d
11. Verify that the versions of libvirtd and virsh are 1.2.8.

root@vmx-server:~# /usr/sbin/libvirtd --version


libvirtd (libvirt) 1.2.8
root@ vmx-server:~# /usr/bin/virsh --version
1.2.8
root@ vmx-server:~#

The system displays the code compilation log.

NOTE: If you cannot deploy vMX after upgrading libvirt, bring down the virbr0
bridge with the ifconfig virbr0 down command and delete the bridge with the
brctl delbr virbr0 command.

Related
Documentation

Installing vMX for Different Use Cases on page 29

Installing vMX for Different Use Cases


Installing vMX is different for specific use cases. Table 8 on page 29 lists the sample
configuration requirements for some vMX use cases.

Table 8: Sample Configurations for Use Cases


Use Case

Minimum vCPUs

Minimum Memory

NIC Device Type

Lab simulation

4
1 for VCP
3 for VFP

8GB
2 GB for VCP
6 GB for VFP

virtio

4
1 for VCP
3 for VFP

8GB
2 GB for VCP
6 GB for VFP

virtio or SR-IOV

5
1 for VCP
4 for VFP

8GB
2 GB for VCP
6 GB for VFP

SR-IOV

Up to 100 Mbps performance


Low-bandwidth applications
Up to 3 Gbps performance
High-bandwidth applications
or performance testing
For 3 Gbps and beyond
performance (with minimum
of two 10Gb Ethernet ports)
Up to 80 Gbps of raw
performance

Copyright 2015, Juniper Networks, Inc.

29

Getting Started Guide

To install vMX for a particular use case, perform one of the following tasks:

Installing vMX for Lab Simulation on page 30

Installing vMX for Low-Bandwidth Applications on page 31

Installing vMX for High-Bandwidth Applications on page 33

Installing vMX for Lab Simulation


The use case for lab simulation uses the virtio NIC.
To install vMX for the lab simulation (less than 100 Mbps) application use case:
1.

Download the vMX software package as root and uncompress the package using tar
xzvf package-name command. For example:
tar xzvf vmx_20150625.0.tgz

2. Change directory to the location of the uncompressed vMX package. For example:

cd vmx_20150625.0
3. Edit the config/vmx.conf text file with a text editor to configure a single vMX instance.

Ensure the following parameters are set properly in the vMX configuration file:

device-type : virtio

forwarding-engine-image : vPFE_lite_* (lite version)

See Configuring the vMX Instance on page 35.


4. Run the ./vmx.sh -lv --install script to deploy the vMX instance specified by the

config/vmx.conf startup configuration file and provide verbose-level logging to a file.

See Deploying vMX on page 37.


Here is a sample vMX startup configuration file using the virtio device type for lab
simulation:
--#Configuration on the host side
HOST:
identifier
:
host-management-interface :
routing-engine-image
:
routing-engine-hdd
:
forwarding-engine-image
:
--#External bridge configuration
BRIDGES:
- type : external
name : br-ext

- management interface, VM images etc.


vmx1
# Maximum 4 characters
eth0
"/home/vmx/vmxlite/images/jinstall64-vmx.img"
"/home/vmx/vmxlite/images/vmxhdd.img"
"/home/vmx/vmxlite/images/vPFE_lite.img"

# Max 10 characters

--#vRE VM parameters
CONTROL_PLANE:
vcpus
: 1
memory-mb
: 1024
console_port: 8601

30

Copyright 2015, Juniper Networks, Inc.

Chapter 3: Installing vMX

interfaces :
- type
: static
ipaddr
: 10.102.144.94
macaddr
: "0A:00:DD:C0:DE:0E"
--#vPFE VM parameters
FORWARDING_PLANE:
memory-mb
: 6144
vcpus
: 3
console_port: 8602
device-type : virtio
interfaces :
- type
: static
ipaddr
: 10.102.144.98
macaddr
: "0A:00:DD:C0:DE:10"
--#Interfaces
JUNOS_DEVICES:
- interface
mac-address
description

: ge-0/0/0
: "02:06:0A:0E:FF:F0"
: "ge-0/0/0 interface"

- interface
mac-address
description

: ge-0/0/1
: "02:06:0A:0E:FF:F1"
: "ge-0/0/1 interface"

- interface
mac-address
description

: ge-0/0/2
: "02:06:0A:0E:FF:F2"
: "ge-0/0/2 interface"

- interface
mac-address
description

: ge-0/0/3
: "02:06:0A:0E:FF:F3"
: "ge-0/0/3 interface"

Installing vMX for Low-Bandwidth Applications


The use case for low-bandwidth applications uses virtio or SR-IOV NICs.
To install vMX for the low-bandwidth (up to 3 Gbps) application use case:
1.

Download the vMX software package as root and uncompress the package using tar
xzvf package-name command. For example:
tar xzvf vmx_20150625.0.tgz

2. Change directory to the location of the uncompressed vMX package. For example:

cd vmx_20150625.0
3. Edit the config/vmx.conf text file with a text editor to configure a single vMX instance.

Ensure the following parameters are set properly in the vMX configuration file:

device-type: virtio or device-type: sriov

forwarding-engine-image: vPFE_* (performance version)

Copyright 2015, Juniper Networks, Inc.

31

Getting Started Guide

See Configuring the vMX Instance on page 35.


4. Run the ./vmx.sh -lv --install script to deploy the vMX instance specified by the

config/vmx.conf startup configuration file and provide verbose-level logging to a file.

See Deploying vMX on page 37.


Here is a sample vMX startup configuration file using the virtio device type for
low-bandwidth applications:
--#Configuration on the host side
HOST:
identifier
:
host-management-interface :
routing-engine-image
:
routing-engine-hdd
:
forwarding-engine-image
:
--#External bridge configuration
BRIDGES:
- type : external
name : br-ext

- management interface, VM images etc.


vmx1
# Maximum 4 characters
eth0
"/home/vmx/vmx/images/jinstall64-vmx.img"
"/home/vmx/vmx/images/vmxhdd.img"
"/home/vmx/vmx/images/vPFE.img"

# Max 10 characters

--#vRE VM parameters
CONTROL_PLANE:
vcpus
: 1
memory-mb
: 1024
console_port: 8601
interfaces :
- type
: static
ipaddr
: 10.102.144.94
macaddr
: "0A:00:DD:C0:DE:0E"
--#vPFE VM parameters
FORWARDING_PLANE:
memory-mb
: 6144
vcpus
: 3
console_port: 8602
device-type : virtio
interfaces :
- type
: static
ipaddr
: 10.102.144.98
macaddr
: "0A:00:DD:C0:DE:10"

32

--#Interfaces
JUNOS_DEVICES:
- interface
mac-address
description

: ge-0/0/0
: "02:06:0A:0E:FF:F0"
: "ge-0/0/0 interface"

- interface
mac-address
description

: ge-0/0/1
: "02:06:0A:0E:FF:F1"
: "ge-0/0/1 interface"

Copyright 2015, Juniper Networks, Inc.

Chapter 3: Installing vMX

- interface
mac-address
description

: ge-0/0/2
: "02:06:0A:0E:FF:F2"
: "ge-0/0/2 interface"

- interface
mac-address
description

: ge-0/0/3
: "02:06:0A:0E:FF:F3"
: "ge-0/0/3 interface"

Installing vMX for High-Bandwidth Applications


The use case for high-bandwidth applications uses the SR-IOV NICs.
To install vMX for the high-bandwidth (above 3 Gbps) application use case:
1.

Download the vMX software package as root and uncompress the package using tar
xzvf package-name command. For example:
tar xzvf vmx_20150625.0.tgz

2. Change directory to the location of the uncompressed vMX package. For example:

cd vmx_20150625.0
3. Edit the config/vmx.conf text file with a text editor to configure a single vMX instance.

Ensure the following parameters are set properly in the vMX configuration file:

device-type: sriov

forwarding-engine-image: vPFE_* (performance version)

See Configuring the vMX Instance on page 35.


4. Run the ./vmx.sh -lv --install script to deploy the vMX instance specified by the

config/vmx.conf startup configuration file and provide verbose-level logging to a file.

See Deploying vMX on page 37.


Here is a sample vMX startup configuration file using the SR-IOV device type:
--#Configuration on the host side
HOST:
identifier
:
host-management-interface :
routing-engine-image
:
routing-engine-hdd
:
forwarding-engine-image
:
--#External bridge configuration
BRIDGES:
- type : external
name : br-ext

- management interface, VM images etc.


vmx1
# Maximum 4 characters
eth0
"/home/vmx/images/jinstall64-vmx.img"
"/home/vmx/images/vmxhdd.img"
"/home/vmx/images/vPFE.img"

# Max 10 characters

--#VCP VM parameters
CONTROL_PLANE:
vcpus
: 1
memory-mb
: 4096
console_port: 8601

Copyright 2015, Juniper Networks, Inc.

33

Getting Started Guide

interfaces :
- type
: static
ipaddr
: 10.102.144.94
macaddr
: "0A:00:DD:C0:DE:0E"
--#VFP VM parameters
FORWARDING_PLANE:
memory-mb
: 16384
vcpus
: 7
console_port: 8602
device-type : sriov
interfaces :
- type
: static
ipaddr
: 10.102.144.98
macaddr
: "0A:00:DD:C0:DE:10"

Related
Documentation

34

--#Interfaces
JUNOS_DEVICES:
- interface
port-speed-mbps
nic
mtu
virtual-function
mac-address
description

:
:
:
:
:
:
:

ge-0/0/0
10000
eth1
2000
# DO NOT EDIT
0
"02:06:0A:0E:FF:F0"
"ge-0/0/0 connects to eth1"

- interface
port-speed-mbps
nic
mtu
virtual-function
mac-address
description

:
:
:
:
:
:
:

ge-0/0/1
10000
eth2
2000
# DO NOT EDIT
0
"02:06:0A:0E:FF:F1"
"ge-0/0/1 connects to eth2"

- interface
port-speed-mbps
nic
mtu
virtual-function
mac-address
description

:
:
:
:
:
:
:

ge-0/0/2
10000
eth3
2000
# DO NOT EDIT
0
"02:06:0A:0E:FF:F2"
"ge-0/0/2 connects to eth3"

- interface
port-speed-mbps
nic
mtu
virtual-function
mac-address
description

:
:
:
:
:
:
:

ge-0/0/3
10000
eth4
2000
# DO NOT EDIT
0
"02:06:0A:0E:FF:F3"
"ge-0/0/3 connects to eth4"

Preparing the System to Install vMX on page 27

Configuring the vMX Instance on page 35

Deploying vMX on page 37

Copyright 2015, Juniper Networks, Inc.

Chapter 3: Installing vMX

Configuring the vMX Instance


The parameters required to configure vMX are defined in the startup configuration file.
The configuration file is in YAML format. The default file is config/vmx.conf. You can save
your configuration file to a different name for different instances.
To customize the configuration, perform these tasks:

Configuring the Host on page 35

Configuring the VCP VM on page 35

Configuring the VFP VM on page 36

Configuring Interfaces on page 36

Configuring the Host


To configure the host environment, you must change the identifier for each vMX instance
and you must provide the correct path for the images.
To configure the host, navigate to Host and specify the following parameters:

identifierName of the vMX instance, maximum of four alphanumeric characters.

host-management-interfaceName of the physical NIC on the host device that is used

for management access (eth0).

NOTE: The interfaces for HOST:host-management-interface,


CONTROL_PLANE, and FORWARDING_PLANE must be on the same subnet.

routing-engine-imageAbsolute path to the jinstall64-vmx.img file for launching VCP.

routing-engine-hddAbsolute path to the vmxhdd.img file for VCP storage.

forwarding-engine-imageAbsolute path to the vPFE.img file for launching VFP. The

image file can be the performance image or the lite image.

make-local-copy-of-images(Optional) Makes a local copy of the VCP and VFP images

and uses the local copy to launch vMX. Default value is yes.

make-local-copy-of-vmxhdd(Optional) Makes a local copy of the VCP storage image

and uses the local copy to launch vMX. Default value is yes.

Configuring the VCP VM


To configure the VCP VM, you must change the IP address and you must make sure the
console port is not being used by another vMX instance or another server.
To configure the VCP VM, navigate to CONTROL_PLANE and specify the following
parameters:

Copyright 2015, Juniper Networks, Inc.

35

Getting Started Guide

vcpusNumber of vCPUs for the VCP, default is 1.

memory-mbAmount of memory for the VCP, default is 2 GB.

console_portKVM TCP-based console port. It must be a unique number.

ipaddrManagement IP address for the VCP VM (fxp0). Navigate to interfaces > type
(static) > ipaddr to modify this parameter.

NOTE: The interfaces for HOST:host-management-interface,


CONTROL_PLANE, and FORWARDING_PLANE must be on the same subnet.

Configuring the VFP VM


Before you configure the VFP VM, consider the following:

You must make sure the console port is not being used by another vMX instance or
another server.

To disable network access to the VFP console, do not configure an IP address.

Based on your requirements, you might want to change the memory, number of vCPUs,
and the device type. See Table 8 on page 29 for some sample configuration
requirements.

To configure the VFP VM, navigate to FORWARDING_PLANE and specify the following
parameters:

memory-mbAmount of memory for the VFP, default is 6 GB.

vcpusNumber of vCPUs for the VFP, default is 3.

console_portKVM TCP-based console port. It must be a unique number.

device-typeNIC interface type, either sriov or virtio.

ipaddrManagement IP address for the VFP VM (eth0). Navigate to interfaces > type
(static) > ipaddr to modify this parameter.

NOTE: The interfaces for HOST:host-management-interface,


CONTROL_PLANE, and FORWARDING_PLANE must be on the same subnet.

Configuring Interfaces
The JUNOS_DEVICES interface names correspond to the Linux physical NIC names on
the host. Bring up the Linux physical NIC ports that are defined in this section before
proceeding. For example, use the ifconfig eth9 up command to bring up the NIC ports on
the eth9 interface.
To configure interfaces for virtio device types, you must specify the interface and the
MAC address.

36

Copyright 2015, Juniper Networks, Inc.

Chapter 3: Installing vMX

To configure interfaces for SR-IOV device types, you must specify the interface, the NIC,
and the MAC address.
To configure the routed interfaces, navigate to JUNOS_DEVICES and specify the following
parameters:

interfaceName of the interface on the vMX.

NOTE: The interface names that are defined in the vmx.conf file must be
contiguous starting from ge-0/0/0. The total number of interfaces
supported is 10, up to ge-0/0/9.

port-speed-mbps(SR-IOV only) Port speed for the physical NIC, default is 10000

Mbps.

nic(SR-IOV only) Name of the physical NIC.

NOTE: Depending on the version of udev, you can rename the classic Linux
standard ethXX names. See Predictable Network Interface Names for more
information.

mtu(SR-IOV only) MTU value, default is 2000 and maximum is 9500.

To change the MTU configuration for virtio device types, modify the mtu parameter in
the device binding file (vmx-junosdev.conf).

Related
Documentation

virtual-function(SR-IOV only) Child unit of the physical NIC, default is 0.

mac-addressMAC address for the physical NIC.

descriptionDescription of the mapping.

Preparing the System to Install vMX on page 27

Installing vMX for Different Use Cases on page 29

Deploying vMX
Running the ./vmx.sh --install script to deploy vMX sets up:

Host NICs

SR-IOV or virtio device types

External and internal bridges (br-ext and br-int)

NOTE: When br-ext is being created, access to the management port may
be frozen temporarily when it gets attached to the br-ext bridge.

Copyright 2015, Juniper Networks, Inc.

37

Getting Started Guide

Using the --install option also launches the VCP and VFP VMs.
We recommend you deploy the vMX by running the ./vmx.sh -lv --install script to provide
verbose-level logging to a file for the deployment of the vMX instance.

NOTE: Only English locale is supported for using the vmx.sh script.

Related
Documentation

Configuring the vMX Instance on page 35

Managing vMX on page 43

Connecting to VMs
Perform these tasks to connect to the virtual machines for first-time configuration, to
enable access by other means (like Telnet or SSH):

Logging In to VCP on page 38

Logging In to VFP on page 38

Logging In to VCP
You can access the serial console using the ./vmx.sh --console vcp vmx-id command,
where vmx-id is the vMX identifier specified in the startup configuration file, and log in
with the username root and no password.
To disconnect from the console, log out of the session and press Ctrl + ]. At the telnet>
prompt, type close and press Enter.

Logging In to VFP
You can access the serial console using the ./vmx.sh --console vfp vmx-id command,
where vmx-id is the vMX identifier specified in the startup configuration file, and log in
with the username root and password root.
You can connect to VFP using the SSH protocol. Use the IP address defined under
FORWARDING_PLANE in the vmx.conf file. For security reasons, you cannot connect to
VFP using the Telnet protocol.

NOTE: Configure the management IP address on eth0 and routing for


reachability by using the Linux commands ifconfig and route add on VFP VM.

To disconnect from the console, log out of the session and press Ctrl + ]. At the telnet>
prompt, type close and press Enter.
Related
Documentation

38

Deploying vMX on page 37

Managing vMX on page 43

Copyright 2015, Juniper Networks, Inc.

Chapter 3: Installing vMX

Managing vMX Licenses


You must add a license to use vMX features. The licensed features are enforced based
on the license you purchased.
If you need to move your vMX installation to another host, you must remove vMX from
the current host before installing vMX and adding the license on the new host.

Adding a License on page 39

Deleting a License on page 40

Adding a License
To add a license key to the vMX:
1.

Copy the license activation key file to the VCP and add the license key by specifying
the filename.
user@vmx> request system license add filename

Or, you can copy and paste the license activation key directly to add the license key.
For example:
user@vmx> request system license add terminal
E408408918 aeaqib qcsbja okbuqe rcmxnq vjocwf uxfsta
z5ufjb kdrmt6 57bimv 2f3ddp qttcdn 627q4a
jx4s5x hiri
2. Verify that the license is installed. VMX-BANDWIDTH indicates the licensed bandwidth

and VMX-SCALE indicates the application package (VMX-SCALE 1 is the BASE


package, VMX-SCALE 2 is the ADVANCE package, and VMX-SCALE 3 is the PREMIUM
package). This information is also listed as Features in the Licenses installed section.
For example, this output indicates that the 40G perpetual license for the PREMIUM
application package is installed.
user@vmx> show system license
License usage:
Feature name
scale-subscriber
scale-l2tp
scale-mobile-ip
VMX-BANDWIDTH
VMX-SCALE

Licenses
used
0
0
0
40000
3

Licenses
installed
1000
1000
1000
40000
3

Licenses
needed
0
0
0
0
0

Expiry
permanent
permanent
permanent
permanent
permanent

Licenses installed:
License identifier: JUNOS640113
License version: 4
Software Serial Number: 1012620150123J
Customer ID: vMX-Juniper
Features:
vmx-bandwidth-40g - vmx-bandwidth-40g
permanent
vmx-feature-premium - vmx-feature-premium
permanent

Copyright 2015, Juniper Networks, Inc.

39

Getting Started Guide

3. Verify the configured bandwidth for PFE traffic matches the licensed bandwidth

(VMX-BANDWIDTH). The current and average bandwidth are also displayed.


user@vmx> show pfe statistics traffic bandwidth
Configured Bandwidth
: 40000000000 bps
Bandwidth
: 0 bps
Average Bandwidth
: 0 bps

Deleting a License
To delete a vMX license:
1.

Display the installed licenses.


user@vmx> show system license installed
License identifier: JUNOS640113
License version: 4
Features:
vmx-bandwidth-40g - vmx-bandwidth-40g
permanent
vmx-feature-premium - vmx-feature-premium
permanent

2. Delete the license.

user@vmx> request system license delete license-identifier

For example:
user@vmx> request system license delete JUNOS640113
3. Verify that the license is deleted.
user@vmx> show system license
License usage:
Licenses
Feature name
used
scale-subscriber
0
scale-l2tp
0
scale-mobile-ip
0

Licenses
installed
1000
1000
1000

Licenses
needed
0
0
0

Expiry
permanent
permanent
permanent

Licenses installed: none

Related
Documentation

Licensing on page 20

Viewing the Chassis Serial ID for vMX


You can view the chassis serial ID by using the show chassis hardware command from
the CLI, or by using the sysctl hw.chassis.serialid command from shell.
user@vmx> show chassis hardware
Hardware inventory:
Item
Version Part number
Chassis
Midplane
Routing Engine
CB 0
FPC 0
CPU

40

Serial number
VM54599D128A

Description
VMX
RE-VMX
VMX SCB
Virtual FPC

Copyright 2015, Juniper Networks, Inc.

Chapter 3: Installing vMX

Copyright 2015, Juniper Networks, Inc.

41

Getting Started Guide

42

Copyright 2015, Juniper Networks, Inc.

CHAPTER 4

Managing vMX

Managing vMX on page 43

Binding virtio Devices on page 46

Managing vMX

NOTE: Only English locale is supported for using the vmx.sh script.

After you install and deploy vMX, you can use the vmx.sh script with different options to
perform these tasks:

Controlling vMX on page 43

Configuring Logging Options for vMX on page 45

Connecting to Console Port for the VMs on page 45

Getting Help for the Script Options on page 45

Controlling vMX
When you are controlling vMX with the vmx.sh script, you can perform these tasks:

Deploying vMX on page 43

Managing vMX Deployments on page 44

Specifying the Temporary File Directory on page 44

Specifying the Environment File on page 45

Deploying vMX

NOTE: You must be logged in as root to use the control options.

To deploy vMX, use these options with the vmx.sh script:


--cfg fileUse the specified vMX startup configuration file. The default file is
config/vmx.conf.

Copyright 2015, Juniper Networks, Inc.

43

Getting Started Guide

--installStart vMX by setting up the environment, driver dependencies, and memory

requirements and deploying the vMX. If you do not specify a startup configuration
file with the --cfg option, the default file is used.
This example deploys a new vMX instance specified by the my-vmx.cfg configuration file
and provides verbose-level logging to a file:
./vmx.sh -lv --install --cfg config/my-vmx.cfg

Managing vMX Deployments

NOTE: You must be logged in as root to use the control options.

Use these options with the vmx.sh script to stop, start, restart, verify, and clean up an
existing vMX:
--cfg fileUse the specified vMX startup configuration file. The default file is
config/vmx.conf.
--cleanupStop vMX and clean up relevant information about the vMX instance. It also

tears down the Linux bridges and other dependencies. If you do not specify a startup
configuration file with the --cfg option, the default file is used.
--restartStop and start a running vMX. This option is useful for redeploying a vMX that

has parameter changes in the startup configuration file. If you do not specify a startup
configuration file with the --cfg option, the default file is used.
--startStart the vMX that was running and stopped. If you do not specify a startup

configuration file with the --cfg option, the default file is used.
--statusVerify the status of a deployed vMX. If you do not specify a startup configuration

file with the --cfg option, the default file is used.


--stopStop vMX without cleaning up build files so that the vMX can be started quickly

without setup performed by the --install option.


This example tears down an existing vMX instance specified by the my-vmx.cfg
configuration file:
./vmx.sh --cleanup --cfg config/my-vmx.cfg

Specifying the Temporary File Directory

NOTE: You must be logged in as root to use the control options.

To specify the directory used for temporary files, run the ./vmx.sh build directory script.
The default directory is build/vmx-id, where vmx-id is the vMX identifier specified in the
startup configuration file.

44

Copyright 2015, Juniper Networks, Inc.

Chapter 4: Managing vMX

Specifying the Environment File

NOTE: You must be logged in as root to use the control options.

To specify the environment file (.env), run the ./vmx.sh env file script. The default file is
env/ubuntu_sriov.env.

Configuring Logging Options for vMX


You can enable logging options. It is especially useful when used with the control options,
such as --install.
Use these options to configure logging:
-lEnable logging to a file in the specified build directory. The default directory is
build/vmx-id/logs, where vmx-id is the vMX identifier specified in the startup

configuration file. By default, logging is disabled.


-lvEnable logging with verbose details.
-lvfEnable logging with verbose details to the foreground (standard output).

This example deploys a new vMX instance specified by the my-vmx.cfg configuration file
and provides verbose-level logging to a file:
./vmx.sh -lv --install --cfg config/my-vmx.cfg

Connecting to Console Port for the VMs


Use these options with the vmx.sh script to connect to the console of the VCP or VFP of
the specified vMX:
--console vcp [vmx-id]Connect to the console of the VCP for the specified vMX. The

vMX identifier is specified in the startup configuration file.


--console vfp [vmx-id]Connect to the console of the VFP for the specified vMX. The

vMX identifier is specified in the startup configuration file.


This example connects to the console of the VCP for the vMX instance specified by the
vmx1 identifier:
./vmx.sh --console vcp vmx1

Getting Help for the Script Options


To obtain on-line help for the vmx.sh script options, run the ./vmx.sh --help script.
Related
Documentation

Deploying vMX on page 37

Connecting to VMs on page 38

Copyright 2015, Juniper Networks, Inc.

45

Getting Started Guide

Binding virtio Devices


For configurations using virtio device types, you can bind multiple vMX instances together
on the same system if the host has enough CPU and memory to support the vMX
instances. You configure each vMX instance with a different startup configuration file.
If you are deploying multiple vMX instances, make sure:

The VM identifiers are unique across all instances.

The console ports of the VCP and the VFP are unique across all instances.

The external static IP address of the VCP and the VFP are unique across all instances.

The MAC addresses of the VCP and the VFP are unique across all instances, whenever
specified.

NOTE: All VMs share the same management domain. The physical
management interface (for example, eth0) is also part of this global external
bridge.

You can connect virtio NICs in the vMX to physical NICs or virtio NICs in another vMX by
binding these devices as shown in Figure 2 on page 46.

Figure 2: Binding Devices

To manage device bindings, perform these tasks:

46

Setting Up the Device Bindings on page 47

Creating Device Bindings on page 48

Deleting Device Bindings on page 49

Verifying Device Bindings on page 49

Copyright 2015, Juniper Networks, Inc.

Chapter 4: Managing vMX

Setting Up the Device Bindings


The parameters required to configure vMX to bind devices are defined in the device-binding
file. The device-binding file is in YAML format. The default file is config/vmx-junosdev.conf.
The device-binding file defines the endpoints of each link originating from the VFP of a
vMX. One endpoint must be a device using virtio NICs. The other endpoint can be a physical
NIC, a virtio NIC in another vMX instance, or a Linux bridge.
To bind the vMX instances together:
1.

Edit the config/vmx-junosdev.conf file to set up the communication between the vMX
instances.

2. Modify the link_name to the name of the Linux bridge (as shown by the brctl show

command). The link name can be 15 characters long. It must be unique for each bridge.
If more than two interfaces (virtual or physical) are connected by a Linux bridge, then
the bridge name is derived from the dev_name of the common endpoint for the
connected devices.
3. Specify the mtu to change the MTU value for virtio device types from the default of

1500. The maximum value is 9500.


To change the MTU configuration for SR-IOV device types, modify the mtu parameter
in the startup configuration file (vmx.conf).
4. Specify the endpoints for vMX devices (junos_dev type) by customizing these

parameters:

typeType of device is junos_dev.

vm-nameName of the vMX identifier specified in the startup configuration file for

that vMX instance.

dev-nameName of the interface on vMX as specified in the startup configuration

file.
5. Specify the endpoints for physical NICs (host_dev type) by customizing these

parameters:

typeType of device is host_dev.

dev-nameName of the physical NIC on the host.

6. Specify the endpoints for bridges (bridge_dev type) by customizing these parameters:

typeType of device is bridge_dev.

dev-nameName of the Linux bridge.

7. If you have multiple device-binding files, save them with different names.

Here is a sample vMX device-binding file:


interfaces :

- link_name

Copyright 2015, Juniper Networks, Inc.

: link_host

47

Getting Started Guide

mtu
: 1500
endpoint_1 :
- type
: junos_dev
vm_name
: vmx1
dev_name
: ge-0/0/0
endpoint_2 :
- type
: host_dev
dev_name
: int2
- link_name : link_vmx_12
mtu
: 1500
endpoint_1 :
- type
: junos_dev
vm_name
: vmx1
dev_name
: ge-0/0/1
endpoint_2 :
- type
: junos_dev
vm_name
: vmx2
dev_name
: ge-0/0/0
- link_name : bridge_vmx_123
endpoint_1 :
- type
: junos_dev
vm_name
: vmx1
dev_name
: ge-0/0/2
endpoint_2 :
- type
: bridge_dev
dev_name
: bridge1
- link_name : bridge_vmx_123
endpoint_1 :
- type
: junos_dev
vm_name
: vmx2
dev_name
: ge-0/0/1
endpoint_2 :
- type
: bridge_dev
dev_name
: bridge1
- link_name : bridge_vmx_123
endpoint_1 :
- type
: junos_dev
vm_name
: vmx3
dev_name
: ge-0/0/0
endpoint_2 :
- type
: bridge_dev
dev_name
: bridge1

Creating Device Bindings


NOTE: You must be logged in as root to bind devices.

To bind devices with virtio NICs to other devices, define your devices in the vMX
device-binding file and run the ./vmx.sh --bind-dev -cfg device-binding-file script to create
the device binding. If you do not specify a file, the default file is config/vmx-junosdev.conf.
This example creates device bindings with the specified device-binding file:

48

Copyright 2015, Juniper Networks, Inc.

Chapter 4: Managing vMX

./vmx.sh --bind-dev -cfg config/vmx1-junosdev.conf

Deleting Device Bindings


NOTE: You must be logged in as root to unbind devices.

To unbind devices, run the ./vmx.sh --unbind-dev -cfg device-binding-file script to delete
the device bindings created with the --bind-dev option. If you do not specify a file, the
default file is config/vmx-junosdev.conf.
This example deletes device bindings with the specified device-binding file:
./vmx.sh --unbind-dev -cfg config/vmx1-junosdev.conf

Verifying Device Bindings


NOTE: You must be logged in as root to bind devices.

To verify the status of device bindings created with the --bind-dev option, run the ./vmx.sh
--bind-check -cfg device-binding-file script. If you do not specify a file, the default file is
config/vmx-junosdev.conf.
This example verifies the status of the device bindings for the specified device-binding
file:
./vmx.sh --bind-check -cfg config/vmx1-junosdev.conf

Related
Documentation

Installing vMX for Different Use Cases on page 29

Configuring the vMX Instance on page 35

Copyright 2015, Juniper Networks, Inc.

49

Getting Started Guide

50

Copyright 2015, Juniper Networks, Inc.

CHAPTER 5

Troubleshooting vMX

Viewing VFP Statistics on page 51

Viewing VFP Log Files on page 52

Troubleshooting VFP and VCP Connection Establishment on page 53

Verifying Whether VMs Are Running on page 53

Viewing CPU Information on page 54

Verifying BIOS Settings for SR-IOV on page 54

Viewing VFP Statistics


You can view the VFP statistics from a Web browser. The statistics are the relative
counters from the start of the HTTP connection.
The RPIO Stats and Hostif Stats sections provide statistics about the internal
communication between the VCP and the VFP. The RPIO session uses port 3000 and
the HostIF session uses port 3200.
The Port Stats section provides statistics about the packets received and transmitted
to the NIC interfaces.

There is a receive (rx) and transmit (tx) line for each port. Port 0 maps to the ge-0/0/0
interface, port 1 maps to the ge-0/0/1 interface, and so forth. rx0 displays statistics for
packets received from port 0 and tx1 displays statistics for packets transmitted to port
1.

Errors are miscellaneous errors reported by the physical layer NIC.

The Ring Stats section provides statistics about packet processing.

There is an I/O thread (io) for packets received from a port.

There is a worker thread (wk) for each CPU core.

Copyright 2015, Juniper Networks, Inc.

51

Getting Started Guide

The host interface (host) sends protocol packets to the VCP.

The queue processes the packets. The columns provide this information about the
queues:

The Producer and Consumer columns display the source and destination that
generate packets for this queue. The values can be io, wk, tx, or host.

The Priority column displays the priority of the queue. The values can be Normal or
High (only for control packets).

The Free and Used columns display the queue occupancy. The queue has 1024
entries.

The Enqueues and Dequeues columns display the number of queue operations.

The Drops column indicates whether the queue is being drained fast enough.

To view the statistics:


1.

Navigate to http://vfp-mgmt-ip:8080/, where vfp-mgmt-ip is the management IP


address for the VFP VM. For example:
http://172.16.0.2:8080/

2. When prompted, enter pfe as the username and pfe as the password.
3. View the statistics displayed in the browser window.

Related
Documentation

Connecting to VMs on page 38

Viewing VFP Log Files


The VFP saves the following files:

VFP log files are saved in the /var/log directory.

VFP crash files are saved in the /var/crash directory.

To view the VFP log or crash files:


1.

Log in to the VFP console by using the ./vmx.sh --console vfp vmx-id command, where
vmx-id is the vMX identifier specified in the startup configuration file.

2. Navigate to the appropriate directory to determine whether there are any files to view.
# cd /var/crash
# ls -l
-rwxr-xr-x 1 root root 864678 Jan

4 02:14 core.riot.1420366466.8271.gz

3. (Optional) If necessary, unzip the file and view it using GDB.


# gunzip core.riot.1420366466.8271.gz
# gdb /build/app core.riot.1420366466.8271

Related
Documentation

52

Connecting to VMs on page 38

Copyright 2015, Juniper Networks, Inc.

Chapter 5: Troubleshooting vMX

Troubleshooting VFP and VCP Connection Establishment


Purpose

When the VCP and VFP connection is established, the show interfaces terse command
in the VCP CLI displays the ge-0/0/x interfaces and the following messages appear in
the VFP syslog file:
RPIO: Accepted connection from 172.16.0.1:50896 <-> vPFE:3000
RPIO: Accepted connection from 172.16.0.1:56098 <-> vPFE:3000
HOSTIF: Accepted connection

If the VCP cannot connect to the VFP, the VFP syslog file does not display the RPIO and
HOSTIF messages.
Action

Run the request chassis fpc slot 0 restart command from the VCP CLI. If an FPC is in
transition error message is displayed, then run restart chassis-control.
If these commands do not correct the problem, verify whether the VCP can ping the VFP
from the routing-instance __juniper_private1__. The three management interfaces (for the
host, VCP VM, and VFP VM) connected to the internal bridge should be able to reach
each other. For example:
root> ping 172.16.0.2 routing-instance __juniper_private1__
PING 172.16.0.2 (172.16.0.2): 56 data bytes
64 bytes from 172.16.0.2: icmp_seq=0 ttl=64 time=0.273 ms
64 bytes from 172.16.0.2: icmp_seq=1 ttl=64 time=0.606 ms

If the VCP cannot ping the VFP, perform these tasks:


1.

Use the brctl show command to verify the bridge configuration and connected
interfaces.

2. Verify that the startup configuration file is correct.


3. Verify that the VFP and the VCP VMs are up and the correct IP addresses are assigned.
4. Restart the FPC from the VCP VM.
5. Restart the chassis management process from VCP VM.
6. Stop and start the VFP VM.
7. Stop and start the VCP VM.
8. Restart the host.

If the problem persists, contact the Juniper Networks Technical Assistance Center (JTAC).

Related
Documentation

Connecting to VMs on page 38

Verifying Whether VMs Are Running


To verify that the VMs are running after vMX is installed, use the virsh list command. The
virsh list command displays the name and state of the VM. The state can be: running,
idle, paused, shutdown, crashed, or dying.

Copyright 2015, Juniper Networks, Inc.

53

Getting Started Guide

You can stop and start VMs with the following virsh commands.

Related
Documentation

virsh destroyForcefully stop a VM while leaving its resources intact.

virsh startStart an inactive VM that was defined previously.

Connecting to VMs on page 38

Viewing CPU Information


On the host server, use the lscpu command to display CPU information. The output
displays such information as the total number of CPUs, the number of cores per socket,
and the number of CPU sockets. For example:
root@vmx-host:~# lscpu
Architecture:
CPU op-mode(s):
Byte Order:
CPU(s):
On-line CPU(s) list:
Thread(s) per core:
Core(s) per socket:
Socket(s):
NUMA node(s):
Vendor ID:
CPU family:
Model:
Stepping:
CPU MHz:
BogoMIPS:
Virtualization:
L1d cache:
L1i cache:
L2 cache:
L3 cache:
NUMA node0 CPU(s):
NUMA node1 CPU(s):
NUMA node2 CPU(s):
NUMA node3 CPU(s):

Related
Documentation

x86_64
32-bit, 64-bit
Little Endian
40
0-39
1
10
4
4
GenuineIntel
6
62
7
3191.766
6385.87
VT-x
32K
32K
256K
38400K
0,4,8,12,16,20,24,28,32,36
1,5,9,13,17,21,25,29,33,37
2,6,10,14,18,22,26,30,34,38
3,7,11,15,19,23,27,31,35,39

Minimum Hardware and Software Requirements on page 23

Verifying BIOS Settings for SR-IOV


If you are having problems with the SR-IOV ports, make sure BIOS has the following
settings:

54

SR-IOV is enabled.

VT-d is enabled.

Hyperthreading is disabled.

Copyright 2015, Juniper Networks, Inc.

Chapter 5: Troubleshooting vMX

We recommend that you verify the process with the vendor because different systems
have different methods to access and change BIOS settings.
Related
Documentation

Preparing the System to Install vMX on page 27

Copyright 2015, Juniper Networks, Inc.

55

Getting Started Guide

56

Copyright 2015, Juniper Networks, Inc.

You might also like