DGTL BRKDCN 1645
DGTL BRKDCN 1645
DGTL BRKDCN 1645
#CiscoLive
#CiscoLive
Agenda
• A short overview on Data Center Evolution
• Introduction to Overlays and VXLAN
• Understanding how MP-BGP is used as a control plane
• Packet Walk with VXLAN
• Design options and additional use cases
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 3
Data Center “Fabric” Journey
Layer-3 HSRP HSRP
Layer-2
Spanning-Tree
Baremet al Hypervisor Hypervisor Hypervisor Baremet al Hypervisor Baremet al Baremet al Hypervisor Hypervisor
ACI
VPC VPC
Layer 3 VTEP VTEP VTEP VTEP VTEP VTEP VTEP
Layer 2
Baremetal Hypervisor Hypervisor Hypervisor Baremetal Hypervisor Baremetal Baremetal Hypervisor Hypervisor
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 4
Why VXLAN Overlay
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 5
Overlay Taxonomy
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 6
VXLAN Packet
• VXLAN is point to multi-point tunneling mechanism to extend Layer 2 networks over an IP network
VXLAN Tunnel
Ethernet Frames
Host IP Network Host
NETWORK
OVERLAY
1 4
Host Host
2
Switch 1 Switch 2 5
Host Host
3 6
IP/UDP Packets
Host
7
Host
Switch 3 8
Host
9
Outer Outer Outer Outer Outer Outer VXLAN Inner Inner Optiona Original CRC
PLANE
DATA
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 7
Lets Build a VXLAN Fabric
VXLAN Fabric – Creating the underlay network
IP routed Network
• Flexible topologies
• Recommend a network with redundant paths using ECMP for load sharing
• Support any routing protocols --- OSFP, IS-IS, BGP, etc.
• All proven best practices for IP routing network apply
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 9
Two Modes of VXLAN
Flood-and-Learn VXLAN: VXLAN EVPN:
• No control plane • EVPN as control plane
• Data driven flood and learning • VTEPs exchange L2/L3 host and subnet
→ Ethernet in the overlay network reachability through EVPN control plane
→ Routing protocol for both L2 and L3
forwarding
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 10
VXLAN BUM Traffic Handling
• BUM Traffic --- Multi-destination traffic
• Broadcast
• Unknown Layer-2 Unicast
• Multicast
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 11
VXLAN with BGP EVPN
Control Plane
EVPN Primer --- MP-BGP Review
Virtual Routing and Forwarding (VRF)
Layer-3 segmentation for tenants’ routing space
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 13
What is VXLAN/EVPN?
• Standards based Overlay (VXLAN) with Standards based Control-Plane (BGP)
• Layer-2 MAC and Layer-3 IP information distribution by Control-Plane (BGP)
• Forwarding decision based on Control-Plane (minimizes flooding)
• Integrated Routing/Bridging (IRB) for Optimized Forwarding in the Overlay
Control-
EVPN MP-BGP - RFC 7432
Plane
➢ EVPN over NVO Tunnels (VXLAN, NVGRE, MPLSoE) for Data Center Fabric encapsulations
➢ Provides Layer-2 and Layer-3 Overlays over simple IP Networks
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 14
RR EVPN Route Reflector
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 17
Configuration Snippet
Vlan 10
vn-segment 5010
Vlan 20 Layer 2 VNI Spine
vn-segment 5020
Vlan 1000
!Layer 3 VNI Layer 3 VNI VRF VRF
vn-segment 9999
Vlan 2000
!Layer 3 VNI VTEP VTEP VTEP VTEP Leaf
vn-segment 9998
interface Vlan20
no shutdown Layer 3 VNI
vrf member VRF-BLUE vrf context VRF-RED vrf context VRF-BLUE
ip address 192.168.20.254/24 tag 12345 vni 9999 vni 9998
ipv6 address 2002::1/64 tag 12345 rd auto rd auto
fabric forwarding mode anycast-gateway
address-family ipv4 unicast address-family ipv4 unicast
interface nve1 route-target both auto route-target both auto
Map L2VNI to NVE route-target both auto evpn route-target both auto evpn
source-interface loopback0
host-reachability protocol bgp evpn evpn
member vni 5010 vni 5010 l2 vni 5020 l2
mcast-group 239.1.1.1 rd auto rd auto
member vni 5020 route-target both auto route-target both auto
mcast-group 239.1.1.1
member vni 9999 associate-vrf
member vni 9998 associate-vrf Associate L3VNI to
NVE
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 19
Distributed Anycast Gateway in MP-BGP EVPN
# VLAN to VNI mapping
vlan 20
vn-segment 5020
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 20
EVPN Peer and Endpoint(Host) Discovery
Triggered by Host Communication across the same VLAN/VNI (L2)
End System End System Overlay Forwarding Table
Host1 <MAC-A> , VTEP 1, L2-VNI
S-MAC: MAC-1
BGP EVPN Type-2 MAC update
D-MAC: 3
01:00:5E:01:01:01 ARP Request for IP B
Advertises MAC-A with NH:IP-1 Src MAC: MAC-A
(VTEP-1) L2VNI Outer S-IP: IP-1 Dst MAC: FF:FF:FF:FF:FF:FF
MAC-3
IP-3
VTEP 3
VTEP-3
Outer D-IP: 239.1.1.1
UDP
VXLAN VNID: 5010
ARP Request for IP B 4
Src MAC: MAC-A
Dst MAC: ARP Response from IP
7 FF:FF:FF:FF:FF:FF
2 VTEP 2
B
Src MAC: MAC-B
2 Multicast Group IP-2 Dst MAC: MAC-A End System B
ARP Response from IP MAC-B
MAC-2
B 239.1.1.1 S-MAC: MAC-2 IP-B
Src MAC: MAC-B VTEP-1 2 D-MAC: MAC-1 VTEP-2 3
Dst MAC: MAC-A 5 Outer S-IP: IP-2 ARP Request for IP B
VTEP 1 Outer D-IP: IP-1 Src MAC: MAC-A
End System A 1 IP-1 6 Dst MAC: FF:FF:FF:FF:FF:FF
MAC-A UDP
IP-A MAC-1
ARP Request for IP B VXLAN VNID: 5010
Src MAC: MAC-A Overlay Forwarding Table
Dst MAC: FF:FF:FF:FF:FF:FF ARP Response Host1 <MAC-A> , VTEP 1, L2-VNI
from IP B
Src MAC: MAC-B
Dst MAC: MAC-A
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 21
EVPN Peer and Endpoint(Host) Discovery
Triggered by Host Communication between VLAN/VNI (L3)
End System End System
MAC-3
IP-3
VTEP 3
Overlay Forwarding Table
VTEP-3
Host1 <IP-A> , VTEP 1, L3-VNI
VTEP 2
2 IP-2
MAC-2
2 Multicast Group End System B
ARP Response from MAC-B
VTEP1
VTEP-1
239.1.1.1 IP-B
Src MAC: GW-MAC VTEP-2
Dst MAC: MAC-A
VTEP 1
End System A 1 IP-1
MAC-A
IP-A MAC-1
ARP Request for anycast
GW at VTEP1 Overlay Forwarding Table
Src MAC: MAC-A
Dst MAC: FF:FF:FF:FF:FF:FF
Host1 <IP-A> , VTEP 1, L3-VNI
Src IP : IP-A
DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 22
Packet Walk
Communication between hosts in same VLAN/VNI
Outer S-MAC: MAC-3
Outer S-MAC: MAC-1 Outer D-MAC: MAC-4
Outer D-MAC: MAC-2
3 Outer S-IP: IP-1
Outer S-IP: IP-1 Outer D-IP: IP-4
Outer D-IP: IP-4 Routed Based on
UDP
UDP Outer IP header
VXLAN VNID: 5010
VXLAN VNID: 5010 (L2 VNI)
IP Network S-MAC: MAC-A
S-MAC: MAC-A
UnderlayR Underlay D-MAC: MAC-B
D-MAC: MAC-B outer-1 Router-2 S-IP: IP-A
S-IP: IP-A MAC-2 MAC-3 D-IP: IP-B
D-IP: IP-B 2 IP-2: IP-3: 4
165.123.1.2 140.123.1.2 MAC-4
IP-4:
MAC-1 140.123.1.1
S-MAC: MAC-A IP-1:
D-MAC: MAC-B
VTEP-1 S-MAC: MAC-A VTEP-2
165.123.1.1 D-MAC: MAC-B
S-IP: IP-A
D-IP: IP-B 1 S-IP: IP-A 5
D-IP: IP-B
Host-B
Host-A
MAC-A MAC-B
VXLAN VNID 5010
IP-A: IP-B:
192.168.10.1 192.168.10.20
0 Vlan 10
Vlan 10 VNI 5010
VNI 5010
DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 23
Packet Walk
Communication between hosts in different VLAN/VNI
Outer S-MAC: MAC-3
Outer S-MAC: MAC-1 Outer D-MAC: MAC-4
Outer D-MAC: MAC-2
3 Outer S-IP: IP-1
Outer S-IP: IP-1 Outer D-IP: IP-4
Outer D-IP: IP-4 Routed Based on
UDP
UDP Outer IP header
VXLAN VNID: 9999
VXLAN VNID: 9999 (L3
VNI) IP Network S-MAC: MAC-1
S-MAC: MAC-1
Underlay Underlay D-MAC: MAC-4
D-MAC: MAC-4 Router-1 Router-2 S-IP: IP-A
S-IP: IP-A MAC-2 MAC-3 D-IP: IP-B
D-IP: IP-B 2 IP-2: IP-3: 4
165.123.1.2 140.123.1.2
VTEP-1
S-MAC: MAC-A VTEP-2
D-MAC: GW-MAC (L3 GW) S-MAC: MAC-4
MAC-1 D-MAC: MAC-B (L3 GW)
S-IP: IP-A
MAC-4
1 IP-1: IP-4:
D-IP: IP-B
165.123.1.1
S-IP: IP-A 5 140.123.1.1
D-IP: IP-B
Host-B
Host-A
MAC-A MAC-B
VXLAN L3 VNID 9999 (Tenant VRF A)
IP-A: IP-B:
192.168.10.10 192.168.20.10
Vlan 10, Vlan 20,
VNI 5010 VNI 5020
DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 24
VXLAN BGP Control Plane
EVPN Control Plane --- VM Mobility
NLRI: Spine
• Host H-MAC-1, H-IP-1
• NVE VTEP-1
• VNI 5000
Ext. Community:
• Encapsulation: VXLAN VTEP-1 VTEP-2 VTEP-3 VTEP-4 Leaf
• Cost
• Sequence number :0 Host 1
H-MAC-1
H-IP-1
VLAN 10
VXLAN 5000 MAC IP VNI Next-Hop Encap Seq#
1. Host 1 attaches to VTEP-1
H-MAC-1 H-IP-1 5000 VTEP-1 VXLAN 0
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 25
VXLAN BGP Control Plane
EVPN Control Plane --- VM Mobility
NLRI: Spine
• Host H-MAC-1, H-IP-1
• NVE VTEP-3
• VNI 5000
Ext. Community:
• Encapsulation: VXLAN VTEP-1 VTEP-2 VTEP-3 VTEP-4 Leaf
• Cost
• Sequence number: 1 Host 1
H-MAC-1
H-IP-1 MAC IP VNI Next-Hop Encap Seq#
VLAN 10
VXLAN 5000 H-MAC-1 H-IP-1 5000 VTEP-3 VXLAN 1
2. VTEP-3 detects Host 1, sends MP-BGP update for Host 1 with its own VTEP address and a new seq #1
3. Other VTEPs learn about the new route of Host 1 from VTEP 3 with a higher sequence number and prefer
that update
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 26
EVPN Control Plane --- ARP Suppression
Minimize flood-&-learn behavior for host learning
1
Host-1 sends ARP
Request for H-IP-2
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 27
Functions of VXLAN/EVPN
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 28
Design Options and Use
case
VXLAN Fabric Design with MP-iBGP EVPN
VXLAN Overlay
MP-iBGP EVPN
VTEP
VTEP VTEP
VTEP VTEP
VTEP VTEP
VTEP VTEP VTEP Leaf
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 30
VXLAN Fabric Design with MP-eBGP EVPN
AS 65000
Spine
MP-eBGP Sessions
VXLAN Overlay
Spine EVPN VRF/VRFs Space
VXLAN Overlay
EVPN MP-BGP Border Leaf
VTEP
VTEP VTEP VTEP VTEP VTEP
Leaf
Routing
Protocol
of Choice
Global Default VRF
Or User Space VRFs
IP Routing
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 32
VXLAN Fabric – Service Insertion
Firewall as a default gateway : Centralized Gateway- Firewall bottleneck Transparent Firewall : Inspect and then bridge Traffic between “dirty” VLAN and “clean” VLAN
Tenant Edge Firewall: Traffic between Tenants/VRFs routed via the firewall
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 33
VXLAN Fabric – Service Insertion
• Load Balancer Integration
Border Border • Load Balancer peer with fabric
using EBGP
• Injects VIP via RHI
VXLAN EVPN
Outside
Advertise VIP
x.x.x.x/32 Client
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 34
VXLAN Fabric – Selective Traffic Redirection
• Leverages Policy Based
Border Border Redirect
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 35
VXLAN Fabric – Centralized Route Leaking
• Extranet Support
• Use Cases – Shared Services, External
External Connectivity
Network
•
Border Border
VRF to VRF or VRF to Default
• Centralize Location for leaking routes
VXLAN EVPN
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 36
Peerlink-Less VPC
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 39
VXLAN Fabric – Tenant Routed Multicast
Spine Spine
VXLAN EVPN
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 40
VXLAN EVPN Multi-Site
Site 1 Site 2
Border Border
Gateways Gateways
Convergence
Fault Separate Admin
independent of Single Box
Containment Domains
Network Size
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 41
Summary
Summary
• VXLAN enables scalable Data Center fabrics
• BGP EVPN with VXLAN provides a robust control plane enabling multi-
tenancy, VM mobility , optimizing traffic forwarding
• Seamless integration with service nodes such as Firewalls and Load
balancers and ability to provide shared services
• Fabric can cater to multicast traffic in the overlay
• VXLAN as a DCI with Multi-Site
#CiscoLive DGTL-BRKDCN-1645 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 43
Thank you
#CiscoLive
#CiscoLive