01-03 Understanding VXLANs

CloudEngine 8800, 7800, 6800, and 5800 Series
Switches
Configuration Guide - VXLAN 3 Understanding VXLANs
3 Understanding VXLANs
3.1 Basic Concepts of VXLANs

3.2 VXLAN Packet Format
3.3 Combinations of Underlay and Overlay Networks
3.4 VXLAN Gateway Classification
3.5 EVPN Basic Principles
3.1 Basic Concepts of VXLANs

Virtual extensible local area network (VXLAN) is an NVO3 network virtualization
technology that encapsulates data packets sent from virtual machines (VMs) into
UDP packets and encapsulates IP and MAC addresses used on the physical
network in outer headers before sending the packets over an IP network. The
egress tunnel endpoint then decapsulates and sends the packets to the destination
VM.
Issue 07 (2021-03-10) Copyright © Huawei Technologies Co., Ltd. 8

Switches
Figure 3-1 VXLAN architecture
VBDIF
BD
NVE
VNI
VTEP UDP 4789
IP IP2
Network
VNI VNI
L3 5020 5030
Packet Device3 Gateway
NVE
VAP2 VAP3
VX
LAN
VLAN 20 Untag
l
Tu
ne
n
n
ne
Tu
l
E
NV
N
L2
LA
l Gateway
unne
VX
A N T
Device1 VXL Device2
NVE vSwitch
vSwitch VM1 VM2 ... VMm
VM1 ... VMm Untag
192.168.10.2/24
VLAN 10 VLAN 20
192.168.10.1/24 192.168.20.1/24
Server1 Server2
VXLAN allows a virtual network to provide access services to a large number of

tenants. Tenants are able to plan their own virtual networks, not limited by
physical network IP addresses or broadcast domains. This greatly simplifies
network management. Table 3-1 describes VXLAN concepts.
Table 3-1 VXLAN concepts

Concept Description
Underlay and VXLAN allows virtual Layer 2 or Layer 3 networks (overlay

overlay networks) to be built over existing physical networks (underlay
networks networks). Overlay networks use encapsulation technologies
to transmit tenant packets between sites over Layer 3
forwarding paths provided by underlay networks. Tenants are
aware of only overlay networks.

Switches
Concept Description
Network A network entity that is deployed at the network edge and

virtualization implements network virtualization functions.
edge (NVE) NOTE
vSwitches on devices and servers can function as NVEs.
There are three NVE deployment modes, which are used
according to the locations of NVE deployment:
● Hardware mode: All NVEs are deployed on NVE-capable
devices, which perform VXLAN encapsulation and
decapsulation.
● Software mode: All NVEs are deployed on vSwitches, which
perform VXLAN encapsulation and decapsulation.
● Hybrid mode: Some NVEs are deployed on vSwitches, and
others on NVE-capable devices. Both vSwitches and NVE-
capable devices may perform VXLAN encapsulation and
decapsulation.
VXLAN tunnel A VXLAN tunnel endpoint that encapsulates and decapsulates

endpoint VXLAN packets. It is represented by an NVE on the controller.
(VTEP) A VTEP connects to a physical network and is assigned a
physical network IP address, which is irrelevant to virtual
networks.
In VXLAN packets, the source IP address is the local node's
VTEP address, and the destination IP address is the remote
node's VTEP address. This pair of VTEP addresses corresponds
to a VXLAN tunnel.
VXLAN A VXLAN segment identifier similar to a VLAN ID. VMs on

network different VXLAN segments cannot communicate directly at
identifier (VNI) Layer 2.
A VNI identifies only one tenant. Even if multiple end users
belong to the same VNI, they are considered one tenant. A VNI
consists of 24 bits and supports a maximum of 16M tenants.
In distributed VXLAN gateway scenarios, a VNI can be a Layer
2 or Layer 3 VNI.
● A Layer 2 VNI is mapped to a BD in 1:1 mode for intra-
segment transmission of VXLAN packets.
● A Layer 3 VNI is bound to a VPN instance for inter-segment
transmission of VXLAN packets.
Bridge domain A Layer 2 broadcast domain through which VXLAN data

(BD) packets are forwarded.
VNIs identifying VNs must be mapped to BDs in 1:1 mode so
that the BDs can function as entities that transmit VXLAN
traffic on a VXLAN network.

Switches
Concept Description
VBDIF A Layer 3 logical interface created for a BD. Configuring IP

interface addresses for VBDIF interfaces allows communication between
VXLANs on different network segments and between VXLANs
and non-VXLANs and implements Layer 2 network access to a
Layer 3 network.
Virtual access A VXLAN service access point that can be a Layer 2 sub-
point (VAP) interface or VLAN.
● If a Layer 2 sub-interface is used as a service access point, it
can have different encapsulation types configured to
transmit various types of data packets. After a Layer 2 sub-
interface is added to a BD, the sub-interface can transmit
data packets through this BD.
● If a VLAN is used as a service access point, it can be bound
to a BD for data packets in the VLAN to be transmitted
through this BD.
Gateway A device that ensures communication between VXLANs

identified by different VNIs and between VXLANs and non-
VXLANs.
A VXLAN gateway can be a Layer 2 or Layer 3 gateway.
● Layer 2 gateway: allows tenants to access VXLANs and
intra-segment communication on a VXLAN.
● Layer 3 gateway: allows inter-segment VXLAN
communication and access to external networks.
Traffic Encapsulation Types

When a Layer 2 sub-interface is used as a service access point, different
encapsulation types can be configured for the sub-interface to transmit various
types of data packets. After a Layer 2 sub-interface is added to a BD, the sub-
interface can transmit data packets through this BD. Table 3-2 describes the
different encapsulation types.

Switches
Table 3-2 Traffic encapsulation types

Traffic Description
Encapsulation
Type
dot1q If a dot1q sub-interface receives a single-tagged VLAN

packet, the sub-interface forwards only the packet with a
specified VLAN ID. If a dot1q sub-interface receives a double-
tagged VLAN packet, the sub-interface forwards only the
packet with a specified outer VLAN ID.
● When performing VXLAN encapsulation on packets, a
dot1q Layer 2 sub-interface removes the outer tags of the
packets.
● When performing VXLAN decapsulation on packets, a
dot1q Layer 2 sub-interface adds specified VLAN tags to
the packets.
For the CE8860EI, CE8868EI, CE8861EI, CE8850EI, CE7850EI,
CE7855EI, CE6850EI, CE6850HI, CE6850U-HI, CE6851HI,
CE6855HI, CE6856HI, CE6857EI, CE6860EI, CE6865EI,
CE6870EI, CE6875EI, the VLAN ID of a dot1q Layer 2 sub-
interface can be a value range. In this case, the sub-interface
transparently transmits packets without removing the VLAN
tag.
When setting the encapsulation type to dot1q for a Layer 2
sub-interface, note the following:
● The VLAN IDs specified for the Layer 2 sub-interface
cannot be the same as either the VLAN IDs of packets
allowed to pass through the corresponding Layer 2
interfaces or the MUX VLAN IDs.
● Layer 2 and Layer 3 sub-interfaces cannot have the same
VLAN IDs specified.
● The VLAN ID ranges specified for different dot1q Layer 2
sub-interfaces of the same main interface cannot overlap.

Switches
Traffic Description
Encapsulation
Type
untag An untagged Layer 2 sub-interface receives only packets that

do not carry VLAN tags.
● When performing VXLAN encapsulation on packets, an
untagged Layer 2 sub-interface does not add any VLAN
tag to the packets.
● During VXLAN packet decapsulation on the CE6870EI and
CE6875EI, an untagged Layer 2 sub-interface removes the
VLAN tags of single-tagged inner packets or the outer
VLAN tags of double-tagged inner packets.
● During VXLAN packet decapsulation on the CE6870EI, an
untagged Layer 2 sub-interface removes the VLAN tags of
single-tagged inner packets or the outer VLAN tags of
double-tagged inner packets.
● During VXLAN packet decapsulation on devices except the
CE6870EI and CE6875EI, an untagged Layer 2 sub-
interface does not process the VLAN tags of inner packets.
When setting the encapsulation type to untag for a Layer 2
● Ensure that the corresponding physical interface of the
Layer 2 sub-interface does not have any configuration,
and is removed from the default VLAN.
● Untagged Layer 2 sub-interfaces can be configured only
for Layer 2 physical interfaces and Eth-Trunk interfaces.
● An interface can have only one untagged Layer 2 sub-
interface configured.

Switches
Traffic Description
Encapsulation
Type
qinq A QinQ sub-interface receives only tagged packets with

specified inner and outer VLAN tags.
● When performing VXLAN encapsulation on packets, a
QinQ sub-interface removes two VLAN tags from packets
if the action of the Layer 2 sub-interface is set to
removing two VLAN tags and maintains the VLAN tags of
packets if the action of the Layer 2 sub-interface is not set
to removing two VLAN tags.
● When performing VXLAN decapsulation on packets, a
QinQ sub-interface adds two specific VLAN tags to
packets if the action of the Layer 2 sub-interface is set to
removing two VLAN tags and maintain the VLAN tags of
packets if the action of the Layer 2 sub-interface is not set
to removing two VLAN tags.
When a Layer 2 sub-interface with the encapsulation type of
default, dot1q transparent transmission (with the rewrite
no-action command run), or QinQ transparent transmission
(without the rewrite pop double command) is bound to a
BD, this BD does not support IGMP snooping, DHCP
snooping, VBDIF interfaces, or ARP broadcast suppression.
NOTE
The traffic behavior for QinQ interfaces bound to the same BD must
be the same. In a VXLAN networking, the traffic behavior for
different devices' QinQ interfaces bound to the same BD must be the
same.
After a VLAN range is configured on a QinQ Layer 2 sub-interface,
the rewrite pop double command cannot be configured on the sub-
interface. Similarly, after the rewrite pop double command is
configured on the device, a VLAN range cannot be configured on a
QinQ Layer 2 sub-interface.
The outer VLAN encapsulated for a Layer 2 QinQ sub-
interface cannot be the same as the default VLAN and
allowed VLAN of the corresponding Layer 2 main interface.

Switches
Traffic Description
Encapsulation
Type
default A default Layer 2 sub-interface receives all packets,

irrespective of whether the packets carry VLAN tags.
When performing VXLAN encapsulation and decapsulation
on packets, a default Layer 2 sub-interface does not process
VLAN tags of the packets.
When setting the encapsulation type to default for a Layer 2
● Ensure that the interface for the Layer 2 sub-interface is
not added to any VLAN.
● Default Layer 2 sub-interfaces can be configured only for
Layer 2 physical interfaces and Eth-Trunk interfaces.
● If a default Layer 2 sub-interface is created for an
interface, the interface cannot have other types of Layer 2
sub-interfaces configured.
NOTE
When a sub-interface that is configured with dot1q and QinQ receives double-tagged VLAN
packets, the QinQ sub-interface preferentially processes the packets. For example, if a dot1q
and QinQ sub-interface carries the VLAN ID of 10 for dot1q and outer VLAN ID of 10 and
inner VLAN ID of 20 for QinQ and receives a packet with the outer VLAN ID of 10 and inner
VLAN ID of 20, the QinQ sub-interface preferentially processes the packet. If a dot1q and
QinQ sub-interface carries the VLAN ID of 10 for dot1q and outer VLAN ID of 10 and inner
VLAN ID of 20 for QinQ and receives a packet with the outer VLAN ID of 10 and inner
VLAN ID of non-20, the dot1q sub-interface preferentially processes the packet.
3.2 VXLAN Packet Format

VXLAN is a network virtualization technology that performs MAC-in-UDP
encapsulation by adding a UDP header and a VXLAN header before an original
Ethernet packet. Figure 3-2 shows the VXLAN packet format.

Switches
Figure 3-2 VXLAN packet format
MAC MAC 802.1Q Ethernet

DA SA Tag Type
…... Protocol …... IP SA IP DA
VXLAN encapsulation Original packet
Outer Outer Outer Inner Inner

VXLAN
Ethernet IP UDP Ethernet IP Payload
header
header header header header header
VXLAN Flags
Reserved VNI Reserved
(00001000)
8 bits 24 bits 24 bits 8 bits
Source DestPort UDP UDP

Port (VXLAN Port) Length Checksum
16 bits 16 bits 16 bits 16 bits
Table 3-3 Fields in the VXLAN packet format

Field Description
VXLAN header ● VXLAN Flags (8 bits): The value is 00001000.

● VNI (24 bits): VXLAN Segment ID or VXLAN Network
Identifier used to identify a VXLAN segment.
● Reserved fields (24 bits and 8 bits): must be set to 0.
Outer UDP header ● DestPort: destination port number, which is 4789 for
UDP.
● Source Port: source port number, which is calculated
by performing the hash operation on the inner
packet.
Outer IP header ● IP SA: source IP address, which is the IP address of

the local VTEP of a VXLAN tunnel.
● IP DA: destination IP address, which is the IP address
of the remote VTEP of a VXLAN tunnel.
If the underlay network is an IPv4 network, the IP
address of a VTEP is an IPv4 address. If the underlay
network is an IPv6 network, the IP address of a VTEP is
an IPv6 address.

Switches
Field Description
Outer Ethernet ● MAC DA: destination MAC address, which is the MAC
header address mapped to the next-hop IP address based on
the destination VTEP address in the routing table of
the VTEP on which the VM that sends packets
resides.
● MAC SA: source MAC address, which is the MAC
address of the VTEP on which the VM that sends
packet resides.
● 802.1Q Tag: VLAN tag carried in packets. This field is
optional.
● Ethernet Type: Ethernet packet type.
3.3 Combinations of Underlay and Overlay Networks

A basic network on which a VXLAN tunnel is established is called underlay
Network, whereas the service network carried by the VXLAN tunnel is called
overlay network. In VXLAN scenarios, the following combinations of underlay and
overlay networks are applicable.
Category Definition Example
IPv4 over IPv4 The overlay network and As shown in Figure 3-3,
underlay network are the server IP and VTEP IP
both IPv4 networks. addresses are all IPv4
addresses.
IPv6 over IPv4 The overlay network is As shown in Figure 3-3,

an IPv6 network, and the the server IP addresses
underlay network is an are IPv6 addresses, and
IPv4 network. the VTEP IP addresses
are IPv4 addresses.
IPv4 over IPv6 The overlay network is As shown in Figure 3-3,

an IPv4 network, and the the server IP addresses
underlay network is an are IPv4 addresses, and
IPv6 network. the VTEP IP addresses
are IPv6 addresses.
IPv6 over IPv6 The overlay network and As shown in Figure 3-3,
underlay network are the server IP and VTEP IP
both IPv6 networks. addresses are all IPv6
addresses.

Switches
Figure 3-3 Combinations of underlay and overlay networks
Device2
VTEP IP
el
VX
nn
LA
Tu
N
N
Tu
LA
VTEP IP VTEP IP
nn
VX
el
Device1 VXLAN Tunnel Device3
vSwitch vSwitch vSwitch

... ... ...
VM1 VMm VM1 VMm VM1 VMm
Server IP Server IP Server IP

Server1 Server2 Server3
NVE
NOTE
VXLAN implementation principles are similar in the preceding combinations of underlay and
overlay networks. To make the description concise and clear, an IPv4 over IPv4 network is taken
as an example for subsequent descriptions. For other types of network combination, only the
implementation differences are described.
3.4 VXLAN Gateway Classification

A VXLAN gateway is a device that ensures communication between VXLANs
identified by different VNIs and between VXLANs and non-VXLANs.
A VXLAN gateway can be a Layer 2 or Layer 3 gateway.
● Layer 2 gateway: allows tenants to access VXLANs and intra-segment
communication on a VXLAN.
● Layer 3 gateway: allows inter-segment VXLAN communication and access to
external networks.
Layer 3 VXLAN gateways can be deployed in centralized or distributed mode.

Switches
Centralized VXLAN Gateway Mode

In this mode, Layer 3 gateways are configured on one device. On the network
shown in Figure 3-4, traffic across network segments is forwarded through Layer
3 gateways to implement centralized traffic management.
Figure 3-4 Centralized VXLAN gateway networking

L3 Spine1 Spine2
Gateway
L2
Gateway
Leaf1 Leaf2
Server1 Server2 Server3

10.1.1.1/24 10.10.1.1/24 10.20.1.1/24
Inter-segment traffic
Centralized VXLAN gateway deployment has its advantages and disadvantages.

● Advantage: Inter-segment traffic can be centrally managed, and gateway
deployment and management is easy.
● Disadvantages:
– Forwarding paths are not optimal. Inter-segment Layer 3 traffic of data
centers connected to the same Layer 2 gateway must be transmitted to
the centralized Layer 3 gateway for forwarding.
– The ARP entry specification is a bottleneck. ARP entries must be
generated for tenants on the Layer 3 gateway. However, only a limited
number of ARP entries are allowed by the Layer 3 gateway, impeding
data center network expansion.
Distributed VXLAN Gateway Mode

● Background
Deploying distributed VXLAN gateways addresses problems that occur in
centralized VXLAN gateway networking. Distributed VXLAN gateways use the
spine-leaf network. In this networking, leaf nodes, which can function as
Layer 3 VXLAN gateways, are used as VTEPs to establish VXLAN tunnels.
Spine nodes are unaware of the VXLAN tunnels and only forward VXLAN
packets between different leaf nodes. On the network shown in Figure 3-5,
Server 1 and Server 2 on different network segments both connect to Leaf 1.

Switches
When Server 1 and Server 2 communicate, traffic is forwarded only through

Leaf 1, not through any spine node.
Figure 3-5 Distributed VXLAN gateway networking
Spine1 Spine2
L3
Gateway
Leaf1 Leaf2
L2
Gateway
Server1 Server2 Server3 Server4

10.1.1.1/24 10.10.1.1/24 10.20.1.1/24 10.10.1.2/24
Inter-segment traffic
A spine node supports high-speed IP forwarding capabilities.

A leaf node can:
– Function as a Layer 2 VXLAN gateway to connect to physical servers or
VMs and allow tenants to access VXLANs.
– Function as a Layer 3 VXLAN gateway to perform VXLAN encapsulation
and decapsulation to allow inter-segment VXLAN communication and
access to external networks.
● Characteristics of distributed VXLAN gateways
Distributed VXLAN gateway networking has the following characteristics:
– Flexible deployment. A leaf node can function as both Layer 2 and Layer
3 VXLAN gateways.
– Improved network expansion capabilities. A leaf node only needs to learn
the ARP entries of servers attached to it. A centralized Layer 3 gateway in
the same scenario, however, has to learn the ARP entries of all servers on
the network. Therefore, the ARP entry specification is no longer a
bottleneck on a distributed VXLAN gateway.
3.5 EVPN Basic Principles

Introduction
Ethernet virtual private network (EVPN) is a VPN technology used for Layer 2
internetworking. EVPN is similar to BGP/MPLS IP VPN. EVPN defines a new type of

Switches
BGP network layer reachability information (NLRI), called the EVPN NLRI. The
EVPN NLRI defines new BGP EVPN routes to implement MAC address learning and
advertisement between different sites on a Layer 2 network.
VXLAN does not provide the control plane, and VTEP discovery and host
information (IP and MAC addresses, VNIs, and gateway VTEP IP address) learning
are implemented by traffic flooding on the data plane, resulting in high traffic
volumes on DC networks. To address this problem, VXLAN uses EVPN as the
control plane. EVPN allows VTEPs to exchange BGP EVPN routes to implement
automatic VTEP discovery and host information advertisement, preventing
unnecessary traffic flooding.
In summary, EVPN introduces several new types of BGP EVPN routes through BGP
extension for advertising VTEP addresses and host information. In this way, EVPN
applied to VXLAN networks enables VTEP discovery and host information learning
on the control plane instead of on the data plane.
Related Concepts
To help you better understand how EVPN works on a VXLAN network, the
following fundamental concepts are provided:
● Ethernet segment (ES): a set of Ethernet links that connect a VM to more
than one gateway NVE interfaces in an EVPN VXLAN scenario
● Ethernet segment identifier (ESI): an identifier that represents a unique ES
across a network
● EVPN instance (EVI): an instance whose name is the ID of the BD to which
the EVPN instance is bound on an EVPN VXLAN network
● Integrated routing and bridging (IRB): a type of route that carries VNIs, MAC
addresses, and IP addresses for transmitting Layer 2 and Layer 3 routing
information
● Designated forwarder (DF): a designated gateway that is elected for
forwarding BUM traffic to a VM in a VM multi-homing scenario
BGP EVPN Routes

EVPN NLRI defines the following BGP EVPN route types applicable to the VXLAN
control plane:
NOTE
Only the CE6881, CE6881K, CE6863K, CE6881E, and CE6863 support type 1 and type 4
routes.
Type 1 route—Ethernet auto-discovery (A-D) route

Ethernet A-D routes can be per ES routes or per EVI routes. Figure 3-6 shows the
format of an Ethernet A-D route.

Switches
Figure 3-6 Format of an Ethernet A-D route
Route Distinguisher (8 bytes)

Ethernet Segment Identifier (10 bytes)
Ethernet Tag ID (4 bytes)
MPLS Label (3 bytes)
Table 3-4 describes the meaning of each field.
Table 3-4 Fields of an Ethernet A-D route

Field Ethernet A-D per ES Route Ethernet A-D per EVI
Route
Route A combination of a source RD value set in an EVI.

Distinguisher VTEP IP address on a VXLAN
gateway and 0 in the format of
X.X.X.X:0.
Ethernet Unique identifier for defining Unique identifier for

Segment the connections between defining the connections
Identifier VXLAN gateways and a VM. In between VXLAN gateways
a VM multi-homing scenario, and a VM. In a VM multi-
this field is used to determine homing scenario, this field is
which VXLAN gateways are used to determine which
connected to the same VM. VXLAN gateways are
connected to the same VM.
Ethernet Tag Must be all Fs. Identifier that represents a

ID particular broadcast domain
in an EVI. This field with all
0s indicates that the EVI has
only one broadcast domain.
MPLS Label Must be all 0s. VNI associated with the BD

bound to an EVI.
After gateways establish BGP EVPN peer relationships, they exchange Ethernet A-
D routes to advertise their reachability to the MAC addresses of their connected
sites. Specifically, in EVPN ESI all-active scenarios, Ethernet A-D per ES routes are
used for fast convergence and split horizon, whereas Ethernet A-D per EVI routes
are used for aliasing. For details, see 5.3 EVPN ESI All-Active Function.
Type 2 route—MAC/IP route
The following figure shows the format of MAC/IP routes.

Switches
Figure 3-7 Format of a MAC/IP route

MAC Address Length (1 byte)
MAC Address (6 bytes)
IP Address Length (1 byte)
IP Address (0, 4, or 16 bytes)
MPLS Label1 (3 bytes)
MPLS Label2 (0 or 3 bytes)
The following table describes the fields.
Field Description
Route RD value of an EVPN instance

Distingui
sher
Ethernet Unique ID for defining the connection between local and remote
Segment devices
Identifier
Ethernet VLAN ID configured on the device

Tag ID
MAC Length of the host MAC address carried in the route

Address
Length
MAC Host MAC address carried in the route

Address
IP Mask length of the host IP address carried in the route

Address
Length
IP Host IP address carried in the route

Address
MPLS Layer 2 VNI carried in the route

Label1

Label2
MAC/IP routes provide the following functions on the VXLAN control plane:
● MAC address advertisement

To implement Layer 2 communication between intra-subnet hosts, the source
and remote VTEPs must learn the MAC addresses of the hosts. The VTEPs

Switches
function as BGP EVPN peers to exchange MAC/IP routes so that they can
obtain the host MAC addresses. The MAC Address field identifies the MAC
address of a host.
● ARP advertisement
A MAC/IP route can carry both the MAC and IP addresses of a host, and
therefore can be used to advertise ARP entries between VTEPs. The MAC
Address field identifies the MAC address of the host, whereas the IP Address
field identifies the IP address of the host. This type of MAC/IP route is called
the ARP route. ARP advertisement applies to the following scenarios:
a. ARP broadcast suppression. After a Layer 3 gateway learns the ARP entry
of a host, it generates host information that contains the host IP and
MAC addresses, Layer 2 VNI, and gateway's VTEP IP address. The Layer 3
gateway then transmits an ARP route carrying the host information to a
Layer 2 gateway. When the Layer 2 gateway receives an ARP request, it
checks whether it has the host information corresponding to the
destination IP address of the packet. If such host information exists, the
Layer 2 gateway replaces the broadcast MAC address in the ARP request
with the destination unicast MAC address and unicasts the packet. This
implementation suppresses ARP broadcast packets.
b. VM migration in distributed gateway scenarios. After a VM migrates from
one gateway to another, the new gateway learns the ARP entry of the
VM (after the VM sends gratuitous ARP packets) and generates host
information that contains the host IP and MAC addresses, Layer 2 VNI,
and gateway's VTEP IP address. The new gateway then transmits an ARP
route carrying the host information to the original gateway. After the
original gateway receives the ARP route, it detects a VM location change
and triggers ARP probe. If ARP probe fails, the original gateway
withdraws the ARP and host routes of the VM.
● IP route advertisement
In distributed VXLAN gateway scenarios, to implement Layer 3
communication between inter-subnet hosts, the source and remote VTEPs
that function as Layer 3 gateways must learn the host IP routes. The VTEPs
function as BGP EVPN peers to exchange MAC/IP routes so that they can
obtain the host IP routes. The IP Address field identifies the destination
address of the IP route. In addition, the MPLS Label2 field must carry the
Layer 3 VNI. This type of MAC/IP route is called the integrated routing and
bridging (IRB) route.
NOTE
An ARP route carries host MAC and IP addresses and a Layer 2 VNI. An IRB route
carries host MAC and IP addresses, a Layer 2 VNI, and a Layer 3 VNI. Therefore, IRB
routes contain ARP routes and can be used to advertise IP routes as well as ARP
entries.
● ND entry flooding
A MAC/IP route can carry both the MAC address and IPv6 address of a host.
Therefore, this type of route can be used to transmit ND entries between
VTEPs, implementing ND entry advertisement. The MAC Address field carried
in a MAC/IP route indicates information about the host MAC address, and the
IP Address field identifies information about the host IPv6 address. The
MAC/IP route in this case is also called an ND route. ND entry flooding

Switches
applies to the following scenarios (For details, see 5.6 NS Multicast

Suppression on a VXLAN.):
– NS multicast suppression. After a VXLAN gateway collects information
about a local IPv6 host, it generates an ND entry or proxy ND entry and
floods the entry through a MAC/IP route. After receiving the MAC/IP
route, other VXLAN gateways (BGP EVPN peers) each generates a local
proxy ND entry. In this manner, when a VXLAN gateway receives an NS
message, it searches the local proxy ND table. If an entry is hit, the
VXLAN gateway directly performs proxy ND or multicast-to-unicast
processing to reduce or suppress NS message flooding.
– ND spoofing attack defense. In an ND spoofing attack, an attacker
associates its MAC address with the IPv6 address of a host so that all
traffic destined for the IPv6 address is sent to the attacker. After ND
flooding is enabled, VXLAN gateways can synchronize the proxy ND entry
of the same IPv6 host. After an attacker goes online, another proxy ND
entry is repeatedly generated for the same IPv6 host and flooded to other
VXLAN gateways. Through proxy ND entry conflict detection, an IPv6
address conflict alarm is triggered to remind users that an ND spoofing
attack may have occurred.
– IPv6 VM migration in a distributed gateway scenario. After an IPv6 VM is
migrated from one gateway to another, the VM sends a gratuitous NA
message. After receiving the message, the new gateway generates an ND
entry and floods it to the original gateway through a MAC/IP route. After
receiving the message, the original gateway detects that the location of
the IPv6 VM changes and triggers NUD. When the original gateway
cannot detect the IPv6 VM in the original location, it deletes its local ND
entry and uses an MAC/IP route to instruct the new gateway to delete
the old proxy ND entry for the IPv6 VM.
● Host IPv6 route advertisement
In a distributed gateway scenario, to implement Layer 3 communication
between hosts on different subnets, the VTEPs (functioning as Layer 3
gateways) must learn host IPv6 routes from each other. To achieve this, VTEPs
as EVPN peers exchange MAC/IP routes to advertise host IPv6 routes to each
other. The IP Address field carried in the MAC/IP routes indicates the
destination addresses of host IPv6 routes, and the MPLS Label2 field must
carry a Layer 3 VNI. MAC/IP routes in this case are also called IRBv6 routes.
NOTE
An ND route carries the following valid information: host MAC address, host IPv6
address, and Layer 2 VNI. An IRBv6 route carries the following valid information: host
MAC address, host IPv6 address, Layer 2 VNI, and Layer 3 VNI. It can be seen that an
IRBv6 route includes information about an ND route and therefore can be used to
advertise both a host IPv6 route and host ND entry.
Type 3 route—inclusive multicast route

An inclusive multicast route comprises a prefix and a PMSI attribute. The following
figure shows the format of inclusive multicast routes.

Switches
Figure 3-8 Format of an inclusive multicast route

Prefix
IP Address Length (1 byte)
Originating Router's IP Address (4 or 16 bytes)
PMSI attribute
Flags (1 byte)
Tunnel Type (1 byte)
Tunnel Identifier (variable)
Field Description
Route RD value of an EVPN instance

Distingui
sher
Ethernet VLAN ID on the device

Tag ID The value is all 0s in this type of route.
IP Mask length of the local VTEP's IP address carried in the route

Address
Length
Originati Local VTEP's IP address carried in the route

ng
Router's
IP
Address
Flags Flags indicating whether leaf node information is required for the
tunnel
This field is inapplicable in VXLAN scenarios.
Tunnel Tunnel type carried in the route

Type The value can only be 6, representing Ingress Replication in VXLAN
scenarios. It is used for BUM packet forwarding.

Label
Tunnel Tunnel identifier carried in the route

Identifier This field is the local VTEP's IP address in VXLAN scenarios.

Switches
This type of route is used on the VXLAN control plane for automatic VTEP
discovery and dynamic VXLAN tunnel establishment. VTEPs that function as BGP
EVPN peers exchange inclusive multicast routes to transfer Layer 2 VNIs and
VTEPs' IP addresses. The Originating Router's IP Address field identifies the local
VTEP's IP address; the MPLS Label field identifies a Layer 2 VNI. If the remote
VTEP's IP address is reachable at Layer 3, a VXLAN tunnel to the remote VTEP is
established. In addition, the local end creates a VNI-based ingress replication list
and adds the peer VTEP IP address to the list for subsequent BUM packet
forwarding.
Type 4 route—Ethernet segment route
Figure 3-9 shows the format of an Ethernet segment route.
Figure 3-9 Format of an Ethernet segment route

IP Address Length (1 bytes)
Originating Router's IP Address (4 or 16 bytes)
Table 3-5 describes the meaning of each field.
Table 3-5 Fields of an Ethernet segment route

Field Description
Route A combination of a source VTEP IP address on a VXLAN

Distinguisher gateway and 0 in the format of X.X.X.X:0.
Ethernet Unique identifier for defining the connections between

Segment VXLAN gateways and a VM. In a VM multi-homing scenario,
Identifier this field is used to determine which VXLAN gateways are
connected to the same VM.
IP Address Length of a source IP address. In VXLAN scenarios, it is the

Length length of a VTEP address.
Originating Source IP address. In VXLAN scenarios, it is a VTEP address.

Router's IP
Address
Ethernet segment routes carry ESI information, source IP address and RD (source
IP:0) on the local device. These routes are exchanged among gateways that are
connected to the same VM. This ensures that these gateways automatically
discover each other. Ethernet segment routes are mainly used in DF election. For
details, see 5.3 EVPN ESI All-Active Function.

Switches
Type 5 route—IP prefix route

The following figure shows the format of IP prefix routes.
Figure 3-10 Format of an IP prefix route

IP Prefix Length (1 byte)
IP Prefix (4 or 16 bytes)
GW IP Address (4 or 16 bytes)
Field Description
Route RD value of an VPN instance

Distingui
sher
Ethernet Unique ID for defining the connection between local and remote
Segment devices
Identifier
Ethernet VLAN ID configured on the device

Tag ID
IP Prefix Length of the IP prefix carried in the route

Length
IP Prefix IP prefix carried in the route
GW IP Default gateway address

Address

Label
The IP Prefix Length and IP Prefix fields in an IP prefix route can identify a host IP
address or network segment.
● If the IP Prefix Length and IP Prefix fields in an IP prefix route identify a host
IP address, the route is used for IP route advertisement in distributed VXLAN
gateway scenarios, which functions the same as an IRB route on the VXLAN
control plane.

Switches
● If the IP Prefix Length and IP Prefix fields in an IP prefix route identify a

network segment, the route allows hosts on a VXLAN to access external
networks.

01-03 Understanding VXLANs

Uploaded by

Copyright:

Available Formats

01-03 Understanding VXLANs

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

01-03 Understanding VXLANs

Uploaded by

Copyright:

Available Formats

CloudEngine 8800, 7800, 6800, and 5800 Series

3.1 Basic Concepts of VXLANs

3.1 Basic Concepts of VXLANs

Issue 07 (2021-03-10) Copyright © Huawei Technologies Co., Ltd. 8

Figure 3-1 VXLAN architecture

VXLAN allows a virtual network to provide access services to a large number of

Table 3-1 VXLAN concepts

Underlay and VXLAN allows virtual Layer 2 or Layer 3 networks (overlay

Issue 07 (2021-03-10) Copyright © Huawei Technologies Co., Ltd. 9

Network A network entity that is deployed at the network edge and

VXLAN tunnel A VXLAN tunnel endpoint that encapsulates and decapsulates

VXLAN A VXLAN segment identifier similar to a VLAN ID. VMs on

Bridge domain A Layer 2 broadcast domain through which VXLAN data

Issue 07 (2021-03-10) Copyright © Huawei Technologies Co., Ltd. 10

VBDIF A Layer 3 logical interface created for a BD. Configuring IP

Gateway A device that ensures communication between VXLANs

Traffic Encapsulation Types

Issue 07 (2021-03-10) Copyright © Huawei Technologies Co., Ltd. 11

Table 3-2 Traffic encapsulation types

dot1q If a dot1q sub-interface receives a single-tagged VLAN

Issue 07 (2021-03-10) Copyright © Huawei Technologies Co., Ltd. 12

untag An untagged Layer 2 sub-interface receives only packets that

Issue 07 (2021-03-10) Copyright © Huawei Technologies Co., Ltd. 13

qinq A QinQ sub-interface receives only tagged packets with

Issue 07 (2021-03-10) Copyright © Huawei Technologies Co., Ltd. 14

default A default Layer 2 sub-interface receives all packets,

3.2 VXLAN Packet Format

Issue 07 (2021-03-10) Copyright © Huawei Technologies Co., Ltd. 15

Figure 3-2 VXLAN packet format

MAC MAC 802.1Q Ethernet

…... Protocol …... IP SA IP DA

VXLAN encapsulation Original packet

Outer Outer Outer Inner Inner

Source DestPort UDP UDP

Table 3-3 Fields in the VXLAN packet format

VXLAN header ● VXLAN Flags (8 bits): The value is 00001000.

Outer IP header ● IP SA: source IP address, which is the IP address of

Issue 07 (2021-03-10) Copyright © Huawei Technologies Co., Ltd. 16

3.3 Combinations of Underlay and Overlay Networks

IPv6 over IPv4 The overlay network is As shown in Figure 3-3,

IPv4 over IPv6 The overlay network is As shown in Figure 3-3,

Issue 07 (2021-03-10) Copyright © Huawei Technologies Co., Ltd. 17

Figure 3-3 Combinations of underlay and overlay networks

vSwitch vSwitch vSwitch

Server IP Server IP Server IP

3.4 VXLAN Gateway Classification

Issue 07 (2021-03-10) Copyright © Huawei Technologies Co., Ltd. 18

Centralized VXLAN Gateway Mode

Figure 3-4 Centralized VXLAN gateway networking

Server1 Server2 Server3

Centralized VXLAN gateway deployment has its advantages and disadvantages.

Distributed VXLAN Gateway Mode

Issue 07 (2021-03-10) Copyright © Huawei Technologies Co., Ltd. 19

When Server 1 and Server 2 communicate, traffic is forwarded only through

Figure 3-5 Distributed VXLAN gateway networking

Server1 Server2 Server3 Server4

A spine node supports high-speed IP forwarding capabilities.

3.5 EVPN Basic Principles

Issue 07 (2021-03-10) Copyright © Huawei Technologies Co., Ltd. 20

BGP EVPN Routes