Catalyst 9300 Switching Architecture - BRKARC-3863 - 2018

Download as pdf or txt
Download as pdf or txt
You are on page 1of 116

#CLUS

Catalyst 9300
Switching
Architecture
Subtitle goes here

Minhaj Uddin, Technical Marketing Engineer


BRKARC-3863

#CLUS
A New Era of Networking

Security

Cloud

Video
IOT

Voice
Mobility
Data

Previous Era New Era

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 34
Is Your Network Ready for the New Era?
IP Display/DMS Printer IP Camera LED Lights AP PC/Laptop IP Phone

Does the platform Does the platform Does the platform Does the platform Does the platform let you
support new PoE make it easy to support enough ensure secure adapt to new connectivity
devices efficiently? provision Programmability? network access? requirements?
and scale?

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 4
New Era of Networking – Catalyst 9300

Integrated security Mobility ready IoT ready Cloud ready


Devops Toolkit
Fabric Enabled Wireless CoAP Streaming Telemetry
Network as a Sensor
Unified control and policy POE Enhancements SDA
Encrypted Traffic Analytics
IEEE 1588 Web UI
Macsec Encryption
Patchability
Trustworthy Systems
GIR

© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential #CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 5
“The goal of this session is to
give you an in depth view of the
platform so you can understand
its strength as well as its
limitations …”

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 6
Agenda
• Introduction & Overview
• Platform Architecture, ASIC &
Packet Walks
• Stacking Architecture & High
Availability
• Differentiating Features & IOS-XE
• Wrap up

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 7
Cisco Webex Teams
Questions?
Use Cisco Webex Teams (formerly Cisco Spark)
to chat with the speaker after the session

How
1 Find this session in the Cisco Events App
2 Click “Join the Discussion”
3 Install Webex Teams or go directly to the team space
4 Enter messages/questions in the team space

Webex Teams will be moderated cs.co/ciscolivebot#BRKARC-3863


by the speaker until June 18, 2018.

#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 8
Hardware Innovations
The New Catalyst 9300

1.8-GHz quad-core x86 CPU


Built-in RFID USB 2.0 flash drive/Bluetooth 8 GB of DDR4 DRAM 1x UADP 2.0 ASIC
(passive) dongle** 16 GB flash

Unmatched POE
USB Console
Mini-USB type B Multigigabit Capable Resiliency – Perpetual/Fast Flexible Modular Uplinks
High power - 60W UPOE

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 10
Catalyst 9300– Back View
External Storage
USB 3.0 Removable storage Stack Cables Redundant Fans Redundant Power
(120GB SSD)

Optional Power Supplies


Stackwise-480 Stackpower
(AC+DC)
* Roadmap

#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 11
Catalyst 9300 – 1G Models Overview

UPOE on all Stackwise-480


ports

Stackpower
POE+ on all
ports Zero Footprint Power
Redundancy

Data only on
all ports
Larger Buffers & Scale

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 12
Multigigabit Ethernet
The Problem - Gigabit Bottleneck
Wifi >1G

Cat 5e Cables

Limited to 1G!

Existing Gigabit infrastructure Gigabit Ethernet has been Market needs an innovative
is insufficient to handle .11ac around since 1999 and has technology to support >1Gbps
growth beyond 1Gbps now become the bottleneck over existing cables

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 14
The Solution – Cisco MultiGigabit
WiFi > 1G

Cat 5e Cables

2.5-5G!
MultiGigabit MultiGigabit
Switch Capable AP

Cisco MultiGigabit

Is a game-changing innovation Enables 2.5 and 5 Gbps up to Supports all PoE standards
allowing enterprise networks to 100m on legacy cables up to 60W
evolve beyond 1G

Delivers up to 5X Speeds in Enterprise without replacing Cabling Infrastructure


#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 15
Cisco Blog: Network World:
http://blogs.cisco.com/enterprise/nbase-t-alliance-achieving-a-new - http://w ww.networkworld.com/article/3124948/lan-w an/ieee-sets-
industry-standard new -ethernet-standard-that-brings-5x-the-speed-w ithout-disruptive-
cable-changes.html

802.3bz

Multigigabit Speeds are now IEEE Standard!


Ethernet Alliance
EEE Post: http://w ww.ethernetalliance.org/w p-
http://standards.ieee.org/findstds/standard/802.3bz-2016.html content/uploads/2016/09/EA_IEEE802bz_FINAL_26Sep16.pdf

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 16
What Speeds Are Supported on MultiGigabit
Ports?
10 M
MultiGigabit Phys Are Different than 1Gigabit Phys

MultiGigabit Ports Are Capable of the Following Speeds 100 M


100M / 1Gig / 2.5Gig / 5Gig / 10Gig
No 10M on MultiGigabit Ports
1000 M
2.5Gig and 5Gig Are now standard

The Non-MultiGigabit Ports Are the Same as Previous 2.5 G


Line Cards / Products – Support 10M/100M/1Gig Speeds

MultiGigabit Phys Are Same on Across our 5G


MultiGigabit Switch Family
Half Duplex on Multigigabit ports is not supported
10 G

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 17
MultiGigabit Cabling Investment Protection
Cable 1G 2.5G 5G 10G
Auto-negotiable Speeds – Interoperates with Type
legacy ports at 100 Mbps and higher

Cat5e ✓ ✓ ✓ NA
Brownfield Deployments can leverage existing
Cat5e cables, extending ROI, and supporting
speeds at 2.5G and 5G at a Distance of 100m
Cat6 ✓ ✓ ✓ 55 m

Greenfield Deployments with Cat6a Will


Support 10G – They can also support speeds at Cat6a ✓ ✓ ✓ ✓
2.5G and 5G at a distance of 100m

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 18
Catalyst 9300 Multigigabit Family Shipping

24 x 100/1/2.5/5/10G Ports

36 x 2.5 G Ports 12 x Multigigabit Ports

Highest 2.5G & mGig Density in the Industry


#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 19
Catalyst 9300 Multigigabit Family - New Aug `18

416 Ports of Multigigabit Ports with Stacking

48 x 100/1/2.5/5G Ports with UPOE

Optimized for 802.11 AC


#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 20
Uplink Options on Catalyst 9300
March `18

4x1Gig 8x10Gig 2x40Gig 4x1/2.5/5/10Gig


SFP SFP/SFP+ QSFP Copper

Uplink Modules supported on all 9300 Models


#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 21
Catalyst 9300 – Enabling 25G Ethernet
Shipping

Catalyst 9300 2 x 25G Network Module

• Supported on all C9300 SKU’s


• MACsec-256** capable
• Hardware authenticity support

* Roadmap
** Switch to Switch only

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 22
Catalyst 9300 – Power Supplies & Stacking

Power Supplies Stacking

350WAC 750WDC* 715WAC 1100WAC

Platinum Rated Aug`18

0.5, 1 and 3 meter Options

* Roadmap 23
#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Looking Inside the
Switch
Catalyst 9300: Under the Covers…
CPU
UADP ASICs
Redundant Power Supplies

Downlink Phys Power Stack Conn (x2)


(x24)
Fan FRU (x3)

Back Stack Conn (x2)

USB 3.0

Ethernet And
Console Port

Ampere / Stack
Power Controller
FRU Uplink
Module

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 25
Catalyst 9K Family - x86 CPU

x86 CPU

x86 based 3rd Party Apps

x86 CPU enables hosting containers and 3rd party apps


#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 26
ASICs are a Pillar of Cisco Innovation…
#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 27
Traditional Networking ASICs - Fixed Pipelines
Can lookup these
Fields

Parses &
Understands Fixed Fixed
Parser
number of Bytes
MAC IPv4 ACL QoS
Ether
net
IP Payload Look Look Look Look
up up up up
Ether VXLA Ether
VXLAN net
IP UDP
N net
IP Payload

GRE Ethern
IP GRE
Ethern
IP Payload
et et

MPLS Ethern Fast Memory Lookup Tables


Label IP Payload
et

Not Supported in Hardware


Traditional QoS
Look
ACL
Look

Look

Look
ASIC up up up up

Fixed Pipeline

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 28
New ASICs for New Technology ?

Marketing
Architecture RTL Design Synthesis Floor Planning Fabrication
Requirements

2 – 4 Years

Building a new ASIC takes a lot of time &


money
#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 29
How about CPUs ?

CPUs are highly CPUs are not as


Programmable fast

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 30
Traditional Networking ASICs vs CPUs

Performance Performance

Flexibility Flexibility

Traditional General
Networking Purpose
ASIC CPU

Purpose Built – High Performance General Purpose – Highly Flexible

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 31
Cisco Innovation – UADP ASIC

Performance

In 2013 Cisco Introduced UADP Flexibility


(Unified Access Data Plane)
Programmability

UADP brings Flexibility without compromise on Performance

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 32
UADP Evolution

UADP 1.0 UADP 1.1


1.6 Billion Transistors 3.2 Billion Transistors
36 nm 36 nm

Catalyst Catalyst Catalyst


3850 3650 3650
Catalyst 3850 Catalyst 3650 Catalyst SFP Fiber Multigigabit Mini Multigigabit
Copper Catalyst
3850 SFP+

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 33
UADP 2.0 - Next Generation of ASIC Innovation

Investment Protection
Flexible Pipeline

Universal Deployments
Adaptable Tables

Enhanced Scale/Buffering
Multicore resource share

Shared Up to 2X to 4X
384K Flex Up to 240GE
Counters Lookup Bandwidth Forwarding + TCAM

7.46B Transistors
28nm Technology
Up to 32MB Up to 64K x2
Embedded Packet Buffer Netflow Records
Microcontrollers

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 34
Some of the Key Capabilities of UADP 2.0

Flex Parser
& Recirculation Adaptable Tables
Micro Engines
Programmable Capability
Pipelines

No Compromise on Performance

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 35
UADP 2.0 – Core Architecture
Stack Interface
AQM
Q PBC – Packet Buffers Complex SQS

IQS Ingress Pipeline Egress Pipeline Q Q


Flex Parser EQS
L L L L
o o o o

IGR o
k
o
k
o
k
o
k

Flexible
u u u u
p p p p
T T T T
a a a a
b b b b
l l l l
e e e e

Stage #15 Stage #1


Look up
L L L L
o o o o
o o o o

Ingress
Stage #.. Forwarding Egress Forwarding
k k k k
u u u u
p p p p
T T T T

Stage #2
a a a a

Tables
b b b b
l l l l
e e e e

Controller
Stage #..
X L
o
o
k
u
L
o
o
k
u
L
o
o
k
u
L
o
o
k
u
X Controller
Stage #..
F F
p p p p
T T T T

(IFC) (EFC) Stage #..


a a a a
b b b b
l l l l
e e e e

C C
Stage #2 (Shared
L
o
o
k
u
p
T
L
o
o
k
u
p
T
L
o
o
k
u
p
T
L
o
o
k
u
p
T

Across
a a a a

Stage #8
b b b b
l l l l
e e e e

Stage #1 L L L L

Cores)
o o o o
o o o o
k k k k
u u u u
p p p p
T T T T
a a a a
b b b b

Flex Parser
l
e
l
e
l
e
l
e
EGR
ReWrite
Engine
Encryption Recirculation
Engine Engine
Ingress
Egress
FIFO
FIFO

MACSEC MACSEC

Network Interfaces - Front Panel Ports + CPU + Network Redundant Uplinks (NRU)

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 36
UADP 2.0 – Programmable Pipelines
Stack Interface
AQM
Q PBC – Packet Buffers Complex SQS

IQS Ingress Pipeline Egress Pipeline Q Q


Flex Parser EQS
L L L L
o o o o

IGR o
k
o
k
o
k
o
k

Flexible
u u u u
p p p p
T T T T
a a a a
b b b b
l l l l
e e e e

Stage #15 Stage #1


Look up
L L L L
o o o o
o o o o

Ingress
Stage #.. Forwarding Egress Forwarding
k k k k
u u u u
p p p p
T T T T

Stage #2
a a a a

Tables
b b b b
l l l l
e e e e

Controller
Stage #..
X L
o
o
k
u
L
o
o
k
u
L
o
o
k
u
L
o
o
k
u
X Controller
Stage #..
F F
p p p p
T T T T

(IFC) (EFC) Stage #..


a a a a
b b b b
l l l l
e e e e

C C
Stage #2 (Shared
L
o
o
k
u
p
T
L
o
o
k
u
p
T
L
o
o
k
u
p
T
L
o
o
k
u
p
T

Across
a a a a

Stage #8
b b b b
l l l l
e e e e

Stage #1 L L L L

Cores)
o o o o
o o o o
k k k k
u u u u
p p p p
T T T T
a a a a
b b b b

Flex Parser
l
e
l
e
l
e
l
e
EGR
ReWrite
Engine
Encryption Recirculation
Engine Engine
Ingress
Egress
FIFO
FIFO

MACSEC MACSEC

Network Interfaces - Front Panel Ports + CPU + Network Redundant Uplinks (NRU)

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 37
Programmable Pipelines – Closer Look…
Final Decision on
Packet’s Future
IGR Flex Parser Flex Parser
256 B
Lookup Lookup Lookup Lookup
Table Table Table Table
Stage Stage
#15 #1
Lookup Lookup Lookup Lookup
Stage Table Table Table Table Stage
#.. Flexible #2
15 Ingress Look up Tables 8 Egress
Stage Stage
Programmable #..
Lookup
Table
Lookup
Table
Lookup
Table
Lookup
Table #..
Programmable
Stages (Shared Stages
Stage Stage
#2
Across Cores) #..
Lookup Lookup Lookup Lookup
Table Table Table Table

Stage Stage
#1 #8
Lookup Lookup Lookup Lookup
Table Table Table Table

Flex Parser EGR


TCAM/ SRAM

Flex Parser At each stage, 2 Final Decision on


256 B simultaneous lookups Packet’s Future

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 38
Microcode programs the Pipelines
IGR Flex Parser
Programmed to NG
VXLAN MPLS
understand Protocol

Lookup Lookup
FIB MCast
VXLAN Table Table

Punt Policy Software Features


Lookup Lookup Lookup Lookup
Table Table Table Table
NF MPLS
Flexible
Look up Tables
Egress
Ingress Programmable
Programmable Lookup Lookup Lookup Lookup Sec
SPAN Pipeline
Pipeline Table Table Table Table
(Shared
Stage
VXLAN Across Cores) #..
Lookup Lookup Lookup Lookup
Table Table Table Table Micro Code
L3/L2 SPAN
Lookup Lookup
ACL QOS
Table Table

ASIC
Flex Parser EGR
TCAM/ SRAM
Programmed to
understand
MPLS

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 39
Platform Architecture
&
Layouts
Cisco Catalyst 9300-24/48 Port
Block diagram
Stackwise 480

X86 1.8-GHz
Packet buffer (8 MB) Packet buffer (8 MB)
quad-core CPU

Forwarding controller Forwarding controller FPGA


DRAM – 8 GB
Reassembly Rewrite Reassembly Rewrite Flash
crypto crypto 16 GB

Ingress FIFO Egress FIFO Ingress FIFO Egress FIFO


USB 2.0

Core 1 Core 0
USB 3.0
Network interface
Mgmt Console
10Gx4/40Gx1
1G x8 10Gx4/40Gx1
1G x8

PHY PHY PHY PHY PHY PHY 40G PHY 40G PHY
0 1 2 3 4 5 0 1

TX 0-7 TX 0-7

Cage 1 Cage 2 Cage 1 Cage 2

1-12 13-24 25-35 36-48 1-4 1-4

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 41
Cisco Catalyst 9300 Multigigabit-24
Block diagram
X86 1.8-GHz
Stackwise 480 quad-core CPU

FPGA
ASIC 0 Packet buffer (16 MB) ASIC 1 Packet buffer (16 MB)
DRAM – 8 GB
Flash
Forwarding controller Forwarding controller Forwarding controller Forwarding controller 16 GB

Reassembly Rewrite Reassembly Rewrite Reassembly Rewrite Reassembly Rewrite USB 2.0
crypto crypto crypto crypto
Ingress Egress Ingress Egress Ingress Egress Ingress Egress USB 3.0
FIFO FIFO FIFO FIFO FIFO FIFO FIFO FIFO
Core 1 Core 0 Core 1 Core 0 Mgmt Console

Network interface Network interface

10Gx4/40Gx1
10G x4 10Gx4/40Gx1
10G x4

Mgig Mgig Mgig Mgig Mgig Mgig


40G PHY 40G PHY
PHY PHY PHY PHY PHY PHY
0 1
0 1 2 3 4 5
TX 0-3 TX 0-3

Cage 1 Cage 2 Cage 1 Cage 2

1-6 7-12 13-18 19-24 1-4 1-4


#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 42
Cisco Catalyst 9300 Multigigabit-48UXM
Block diagram
X86 1.8-GHz
Stackwise 480 quad-core CPU

FPGA
ASIC 0 Packet buffer (16 MB) ASIC 1 Packet buffer (16 MB)
DRAM – 8 GB
Flash
Forwarding controller Forwarding controller Forwarding controller Forwarding controller 16 GB

Reassembly Rewrite Reassembly Rewrite Reassembly Rewrite Reassembly Rewrite USB 2.0
crypto crypto crypto crypto
Ingress Egress Ingress Egress Ingress Egress Ingress Egress USB 3.0
FIFO FIFO FIFO FIFO FIFO FIFO FIFO FIFO
Core 1 Core 0 Core 1 Core 0 Mgmt Console

Network interface Network interface

10Gx4/40Gx1
10Gx4 10Gx4/40Gx1
10Gx4 10Gx4
2.5G x4 2.5G x4

MGig 2.5G 2.5G 2.5G 2.5G 2.5G MGig 2.5G 2.5G 2.5G 2.5G MGig
40G PHY 40G PHY
PHY PHY PHY PHY PHY PHY PHY PHY PHY PHY PHY PHY
0 1
1 0 1 2 3 4 2 0 1 2 3 3
TX 0-3 TXI 4-7 TX 8-12 TX 0-3 TXI 4-7 TX 8-12 TX 0-7 TX 0-7

Cage 1 Cage 1 Cage 1 Cage 2 Cage 1 Cage 1 Cage 2

37-40 1-18 41-44 19-36 45-48 1-4 1-4


#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 43
Cisco Catalyst 9300 Multigigabit-48UN
Block diagram
X86 1.8-GHz
Stackwise 480 quad-core CPU

FPGA
ASIC 0 Packet buffer (16 MB) ASIC 1 Packet buffer (16 MB)
DRAM – 8 GB
Flash
Forwarding controller Forwarding controller Forwarding controller Forwarding controller 16 GB

Reassembly Rewrite Reassembly Rewrite Reassembly Rewrite Reassembly Rewrite USB 2.0
crypto crypto crypto crypto
Ingress Egress Ingress Egress Ingress Egress Ingress Egress USB 3.0
FIFO FIFO FIFO FIFO FIFO FIFO FIFO FIFO
Core 1 Core 0 Core 1 Core 0 Mgmt Console

Network interface Network interface


10Gx4/40Gx1
5G x4 10Gx4/40Gx1
5G x4

Mgig Mgig Mgig Mgig Mgig Mgig Mgig Mgig Mgig Mgig Mgig Mgig 40G 40G
PHY PHY PHY PHY PHY PHY PHY PHY PHY PHY PHY PHY PHY PHY
0 1 2 3 4 5 0 1 2 3 4 5 0 1

TX 0-3 TX 0-3

Cage 1 Cage 2 Cage 1 Cage 2

1-24 25-48 1-4 1-4


#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 44
ASIC to port Mapping 9300-48P
9300-Multigigabit48#show platform software fed switch 1 ifm mappings

Interface IF_ID Inst Asic Core Port Type Active


GigabitEthernet1/0/1 0x7 1 0 1 0 NIF Y
!
GigabitEthernet1/0/24 0x1e 1 0 1 23 NIF Y
!
!
GigabitEthernet1/0/25 0x1f 0 0 0 24 NIF Y
!
GigabitEthernet1/0/48 0x36 0 0 0 47 NIF Y Catalyst 9300-48
GigabitEthernet1/1/1 0x37 1 0 1 48 NIF Y Switch Ports Model SW Version SW Image
GigabitEthernet1/1/2 0x38 1 0 1 49 NIF Y Mode
GigabitEthernet1/1/3 0x39 0 0 0 50 NIF Y ------ ----- ----- ---------- ----------
GigabitEthernet1/1/4 0x3a 0 0 0 51 NIF Y ----
! * 1 62 C9300-48P 16.5.1
! CAT9K_IOSXE
TenGigabitEthernet1/1/1 0x3b 1 0 1 52 NIF Y
!
TenGigabitEthernet1/1/4 0x3e 1 0 1 55 NIF Y
!
!
TenGigabitEthernet1/1/5 0x3f 0 0 0 56 NIF Y
!
TenGigabitEthernet1/1/8 0x3f 0 0 0 59 NIF Y
!
!
FortyGigabitEthernet1/1/1 0x43 1 0 1 56 NIF Y
FortyGigabitEthernet1/1/2 0x44 0 0 0 56 NIF Y

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 45
Number of ASICs in different versions of For Your
Switches Reference

Product Version Number of ASICs/Cores Clock Speed Total Bandwidth


Available
24/48 Port 9300 1/2 500 MHz 160 G

24 Port mGig versions 2/4 500 MHz 320G

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 46
Packet Walks
Unicast – within ASIC
Stack Interface
4
AQM
Q PBC – Packet Buffers Complex SQS

IQS Ingress Pipeline Egress Pipeline Q Q


1. Received, processed by 3 L
o
L
o
L
o
L
o

Flex Parser EQS 5


IGR o
k
o
k
o
k
o
k

Flexible
u u u u
p p p p

MACSec and into FIFO


T T T T
a a a a
b b b b
l l l l
e e e e

Stage #15 Stage #1 5. EQS schedule PBC to send


Look up
L L L L
o o o o
o o o o

Ingress
Stage #.. Forwarding Egress Forwarding
k k k k

a copy to EFC and a copy to


u u u u

2. A copy to buffer and a copy


p p p p
T T T T

Stage #2
a a a a

Tables
b b b b
l l l l
e e e e

to IFC Controller X L
o
L
o
L
o
L
o
X Controller ReWrite (includes descriptor)
Stage #..
o o o o

Stage #..
k k k k
u u u u

F F
p p p p
T T T T

(IFC) (EFC) Stage #..


a a a a
b b b b

3. Goes through IFC, result 6. EFC sends results to


l l l l
e e e e

C C
descriptor send to PBC Stage #2 (Shared
L
o
o
k
u
p
T
L
o
o
k
u
p
T
L
o
o
k
u
p
T
L
o
o
k
u
p
T
ReWrite
Across
a a a a

Stage #8
b b b b
l l l l
e e e e

Stage #1
4. Descriptor has local 7. Rewrite the packcet and
L L L L

Cores)
o o o o
o o o o

6
k k k k
u u u u
p p p p
T T T T

send though the egress FIFO


a a a a

destination, PBC sends the


b b b b

Flex Parser
l
e
l
e
l
e
l
e
EGR
info to EQS 2 ReWrite
Engine
Encryption Recirculation
Engine Engine 7
Ingress
Egress
FIFO
FIFO

1 MACSEC

Network Interfaces - Front Panel Ports + CPU + Network Redundant Uplinks (NRU)

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 48
Unicast – Across ASICs on Input
Stack Interface
5 4 AQM
Q PBC – Packet Buffers Complex SQS

IQS Ingress Pipeline Egress Pipeline Q Q


1. Received, processed by 3 IGR
L
o
o
k
L
o
o
k
L
o
o
k
L
o
o
k Flex Parser EQS
Flexible
u u u u
p p p p

MACSec and into FIFO


T T T T
a a a a
b b b b
l l l l
e e e e

Stage #15 Stage #1


Look up
L L L L

2. A copy to buffer and a copy


o o o o
o o o o

Ingress
Stage #.. Forwarding Egress Forwarding
k k k k
u u u u
p p p p
T T T T

Stage #2
a a a a

Tables
b b b b

to IFC
l l l l
e e e e

Controller
Stage #..
X L
o
o
k
L
o
o
k
L
o
o
k
L
o
o
k
X Controller
Stage #..
3. Goes through IFC, result
u u u u

F F
p p p p
T T T T

(IFC) (EFC) Stage #..


a a a a
b b b b
l l l l
e e e e

C C
descriptor send to PBC Stage #2 (Shared
L
o
o
k
u
p
L
o
o
k
u
p
L
o
o
k
u
p
L
o
o
k
u
p
T T T T

Across
a a a a

4. Descriptor has remote Stage #8


b b b b
l l l l
e e e e

Stage #1
destination, PBC sends the
L L L L

Cores)
o o o o
o o o o
k k k k
u u u u
p p p p
T T T T
a a a a

info to IQS
b b b b

Flex Parser
l
e
l
e
l
e
l
e
EGR
2
5. IQS schedule PBC to send ReWrite
the packet with descriptor to Engine
Encryption Recirculation
Stack Interface
Engine Engine
Ingress
Egress
FIFO
FIFO

1 MACSEC

Network Interfaces - Front Panel Ports + CPU + Network Redundant Uplinks (NRU)

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 49
Unicast – Across ASICs on Output
6
Stack Interface
AQM
Q PBC – Packet Buffers Complex SQS

IQS Ingress Pipeline Egress Pipeline Q Q


L
o
L
o
L
o
L
o

Flex Parser EQS 7


IGR o
k
o
k
o
k
o
k

Flexible
u u u u
p p p p
T T T T
a a a a
b b b b
l l l l
e e e e

Stage #15 Stage #1 6. PBC received the frame and


Look up
L L L L
o o o o
o o o o

Ingress
Stage #.. Forwarding Egress Forwarding
k k k k
u u u u

sends the info to EQS


p p p p
T T T T

Stage #2
a a a a

Tables
b b b b
l l l l
e e e e

Controller
Stage #..
X L
o
o
k
L
o
o
k
L
o
o
k
L
o
o
k
X Controller
Stage #..
7. EQS schedule PBC to send
u u u u

F F
p p p p
T T T T

(IFC) (EFC) Stage #..


a a a a
b b b b
l l l l
e e e e

C C
Stage #2 (Shared
L
o
o
k
L
o
o
k
L
o
o
k
L
o
o
k
a copy to EFC and a copy to
ReWrite (includes descriptor)
u u u u
p p p p
T T T T

Across
a a a a

Stage #8
b b b b
l l l l
e e e e

Stage #1 L L L L

Cores)
o o o o

8
o o o o
k k k k
u u u u
p p p p

8. EFC sends results to ReWrite


T T T T
a a a a
b b b b

Flex Parser
l
e
l
e
l
e
l
e
EGR
ReWrite
9. Rewrite the pakcet and send
Engine
Encryption Recirculation out though the egress FIFO
Engine Engine 9
Ingress
Egress
FIFO
FIFO

MACSEC

Network Interfaces - Front Panel Ports + CPU + Network Redundant Uplinks (NRU)

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 50
Multicast – Egress Local Only a single copy of packet in buffer
memory during repliaction
Stack Interface
4
AQM
Q PBC – Packet Buffers Complex SQS

IQS Ingress Pipeline Egress Pipeline Q Q


1. Received, processed by 3 L
o
L
o
L
o
L
o

Flex Parser EQS 5


IGR o
k
o
k
o
k
o
k

Flexible
u u u u
p p p p

MACSec and into FIFO


T T T T
a a a a
b b b b
l l l l
e e e e

Stage #15 Stage #1 5. AQM within EQS generate


Look up
L L L L
o o o o
o o o o

Ingress
Stage #.. Forwarding Egress Forwarding
k k k k
u u u u

2. A copy to buffer and a copy the list of egress port based


p p p p
T T T T

Stage #2
a a a a

Tables
b b b b
l l l l
e e e e

to IFC Controller
Stage #..
X L
o
o
k
u
L
o
o
k
u
L
o
o
k
u
L
o
o
k
u
X Controller
Stage #..
on descriptor, schedule for
F F each egress port
p p p p
T T T T

(IFC) (EFC) Stage #..


a a a a
b b b b

3. Goes through IFC, result


l l l l
e e e e

C C
descriptor send to PBC Stage #2 (Shared
L
o
o
k
u
p
T
L
o
o
k
u
p
T
L
o
o
k
u
p
T
L
o
o
k
u
p
T

Across
a a a a

Stage #8 6. For each egress port, frame


b b b b
l l l l
e e e e

Stage #1
4. Descriptor has local goes though the EFC, ReWriet
L L L L

Cores)
o o o o
o o o o

6
k k k k
u u u u
p p p p
T T T T
a a a a

destination, PBC sends the and Egress FIFO


b b b b

Flex Parser
l
e
l
e
l
e
l
e
EGR
info to EQS 2 ReWrite
Engine
Encryption Recirculation
Engine Engine
Ingress
Egress
FIFO
FIFO

1 MACSEC

Network Interfaces - Front Panel Ports + CPU + Network Redundant Uplinks (NRU)

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 51
Multicast – Egress Remote on Input
Stack Interface
4 AQM
Q PBC – Packet Buffers Complex SQS

IQS Ingress Pipeline Egress Pipeline Q Q


1. Received, processed by 3 IGR
L
o
o
k
L
o
o
k
L
o
o
k
L
o
o
k Flex Parser EQS
Flexible
u u u u
p p p p

MACSec and into FIFO


T T T T
a a a a
b b b b
l l l l
e e e e

Stage #15 Stage #1


Look up
L L L L
o o o o
o o o o

Ingress
Stage #.. Forwarding Egress Forwarding
k k k k
u u u u

2. A copy to buffer and a copy


p p p p
T T T T

Stage #2
a a a a

Tables
b b b b
l l l l
e e e e

to IFC Controller
Stage #..
X L
o
o
k
u
L
o
o
k
u
L
o
o
k
u
L
o
o
k
u
X Controller
Stage #..
F F
p p p p
T T T T

(IFC) (EFC) Stage #..


a a a a
b b b b

3. Goes through IFC, result


l l l l
e e e e

C C
descriptor send to PBC Stage #2 (Shared
L
o
o
k
u
p
T
L
o
o
k
u
p
T
L
o
o
k
u
p
T
L
o
o
k
u
p
T

Across
a a a a

Stage #8
b b b b
l l l l
e e e e

Stage #1
4. Descriptor has remote
L L L L

Cores)
o o o o
o o o o
k k k k
u u u u
p p p p
T T T T
a a a a

destination, PBC sends the


b b b b

Flex Parser
l
e
l
e
l
e
l
e
EGR
info to IQS 2 ReWrite
Engine
5. IQS schedule PBC to send Encryption Recirculation
the packet with descriptor to Engine Engine
Stack Interface Ingress
Egress
FIFO
FIFO

1 MACSEC

Network Interfaces - Front Panel Ports + CPU + Network Redundant Uplinks (NRU)

Descriptor can contains both local and remote


destinations #CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 52
Multicast – Egress Remote Output Replication done on egress
Stack Interface => Efficient use of BW

6
AQM
Q PBC – Packet Buffers Complex SQS

IQS Ingress Pipeline Egress Pipeline Q Q


L
o
L
o
L
o
L
o

Flex Parser EQS 7


IGR o
k
o
k
o
k
o
k

Flexible
u u u u
p p p p
T T T T
a a a a
b b b b
l l l l
e e e e

Stage #15 Stage #1 6. PBC received the frame


Look up
L L L L
o o o o
o o o o

Ingress
Stage #.. Forwarding Egress Forwarding
k k k k
u u u u

and sends the info to EQS


p p p p
T T T T

Stage #2
a a a a

Tables
b b b b
l l l l
e e e e

Controller
Stage #..
X L
o
o
k
u
L
o
o
k
u
L
o
o
k
u
L
o
o
k
u
X Controller
Stage #.. 7. AQM within EQS generate
F F
p p p p
T T T T

(IFC) (EFC) Stage #..


a a a a
b b b b
l l l l
e e e e

C C the list of egress port based


Stage #2 (Shared
L
o
o
L
o
o
L
o
o
L
o
o

on descriptor, schedule for


k k k k
u u u u
p p p p
T T T T

Across
a a a a

Stage #8
b b b b
l l l l
e e e e

Stage #1 L L L L each egress port


Cores)
o o o o
o o o o

8
k k k k
u u u u
p p p p
T T T T
a a a a
b b b b

Flex Parser
l
e
l
e
l
e
l
e
EGR
8. For each egress port, frame
ReWrite
goes though the EFC, ReWriet
Engine
Encryption Recirculation and Egress FIFO
Engine Engine
Ingress
Egress
FIFO
FIFO

MACSEC

Network Interfaces - Front Panel Ports + CPU + Network Redundant Uplinks (NRU)

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 53
Forwarding and TCAM
Resources
UADP 2.0 – Look up Tables
Stack Interface
AQM
Q PBC – Packet Buffers Complex SQS

IQS Ingress Pipeline Egress Pipeline Q Q


Flex Parser EQS
L L L L
o o o o

IGR o
k
o
k
o
k
o
k

Flexible
u u u u
p p p p
T T T T
a a a a
b b b b
l l l l
e e e e

Stage #15 Stage #1


Look up
L L L L
o o o o
o o o o

Ingress
Stage #.. Forwarding Egress Forwarding
k k k k
u u u u
p p p p
T T T T

Stage #2
a a a a

Tables
b b b b
l l l l
e e e e

Controller
Stage #..
X L
o
o
k
u
L
o
o
k
u
L
o
o
k
u
L
o
o
k
u
X Controller
Stage #..
F F
p p p p
T T T T

(IFC) (EFC) Stage #..


a a a a
b b b b
l l l l
e e e e

C C
Stage #2 (Shared
L
o
o
k
u
p
T
L
o
o
k
u
p
T
L
o
o
k
u
p
T
L
o
o
k
u
p
T

Across
a a a a

Stage #8
b b b b
l l l l
e e e e

Stage #1 L L L L

Cores)
o o o o
o o o o
k k k k
u u u u
p p p p
T T T T
a a a a
b b b b

Flex Parser
l
e
l
e
l
e
l
e
EGR
ReWrite
Engine
Encryption Recirculation
Engine Engine
Ingress
Egress
FIFO
FIFO

MACSEC

Network Interfaces - Front Panel Ports + CPU + Network Redundant Uplinks (NRU)

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 55
Flex Tables

SRAM TCAM
Lookup Lookup Lookup Lookup Lookup Lookup Lookup Lookup
Table Table Table Table Table Table Table Table

Lookup Lookup Lookup Lookup Lookup Lookup Lookup Lookup


Table Table Table Table Table Table Table Table

Lookup Lookup Lookup Lookup Lookup Lookup Lookup Lookup


Table Table Table Table Table Table Table Table

Lookup Lookup Lookup Lookup Lookup Lookup Lookup Lookup


Table Table Table Table Table Table Table Table

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 56
ASIC Lookup Tables
Forwarding Resources TCAM Resources

Lookup Lookup Lookup Lookup Lookup Lookup Lookup Lookup


Table Table Table Table Table Table Table Table

Lookup Lookup Lookup Lookup Lookup Lookup Lookup Lookup


Table Table Table Table Table Table Table Table

Lookup Lookup Lookup Lookup Lookup Lookup Lookup Lookup


Table Table Table Table Table Table Table Table

Lookup Lookup Lookup Lookup Lookup Lookup Lookup Lookup


Table Table Table Table Table Table Table Table

Lookup Lookup Lookup Lookup Lookup Lookup Lookup Lookup


Netflow Table Table Table Table Table Table Table Table

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 57
Lookup Tables
Forwarding Resources Feature Resources

• MAC: 32K
Lookup
Table
Lookup
Table
Lookup
Table
Lookup
Table
• Security ACL: 5k
Lookup Lookup Lookup Lookup
Table Table Table Table

• Host Route: 24k • QoS ACL: 5k



Lookup Lookup

IGMP Groups: 8k
Table Table
Lookup
Table
Lookup
Table • Service ACL: 4k
Lookup
Table
Lookup
Table
Lookup
Table
Lookup
Table
• PBR
• LPM Route: 8k
Lookup Lookup Lookup Lookup Lookup
• Netflow ACL
Lookup Lookup Lookup
Table Table Table Table Table • SPAN Table
Multicast Route: 8k
Table Table
• • MACsec
• CoPP
• SGT: 8k
Lookup
Table
Lookup
Table
Lookup
Table
Lookup
Table
Lookup
Table •
Lookup
Tunnel
Table
Lookup
Table
Lookup
Table
• LISP

Netflow Entries: 64k per ASIC


Lookup Lookup Lookup Lookup Lookup Lookup Lookup Lookup
Netflow Table Table Table Table Table Table Table Table

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 58
Scale and TCAM Partition
• Each IPv6 ACL (without port range) requires two TCAM entries.

CAT9K#show platform hardware fed switch active fwd-asic resource tcam utilization 0
CAM Utilization for ASIC Instance [0]
Table Max Values Used Values
--------------------------------------------------------------------------------
Unicast MAC addresses 32768/512 16/22
IGMP and Multicast groups 8192/512 0/0
L2 Multicast groups 8192/512 0/0
Directly or indirectly connected routes 24576/8192 10/21
NAT/PAT SA address and Port 0 0
QoS Access Control Entries 5120 0
Security Access Control Entries 5120 126
Ingress Netflow ACEs 256 9
Policy Based Routing ACEs 1024 0
Egress Netflow ACEs 768 0
Input Microflow policer ACEs 0 0
Output Microflow policer ACEs 0 0
Flow SPAN ACEs 256 0
Control Plane Entries 512 204
Tunnels 512 17
Lisp Instance Mapping Entries 512 3
Input Security Associations 256 0
Output Security Associations and Policies 256 5
SGT_DGT 8192/512 0/1
CLIENT_LE 4096/256 0/0
INPUT_GROUP_LE 1024 0
OUTPUT_GROUP_LE 1024 0
Macsec SPD 256 2

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 59
Dafault SDM Template Not drawn to scale

Network Interface
Forwarding TCAM
MCAST
FIB (8k)
Others
Netflow

(8k) IGMP MAC SEC QoS


(8k) (32k) ACL ACL
SGT (5k) (5k)
(8K)
Host
(16k)
Internal
Internal Resources
Tunnels LISP Resources
(512) (512)

Packet Buffer

Stack Interface
#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 60
High Availability
Stackwise-480
The Stack Ring
480 Gbps capacity
UADP ASIC Stack Interface of UADP ASIC

• 6 rings in total
• 3 rings go East
• 3 rings go West
• Each ring is 40Gbps
• 240Gbps uni-direction
• Spatial Reuse= 480Gbps

Stack
Interface
of UADP
ASIC Assuming 4 x 24-port 9300 Switches
6 Rings in the
Stack
#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 63
Unicast Packet Path on the Stack Ring

Assuming
4
3
2
1
4 x 24-port
9300 Switches

Creating
 Packet segmented into Segments
256 bytes Re-ordering
segments
 Packet travels half the
ring for unicast traffic
 Segments reordered at
destination stack port
 Destination strips the
packet off the stack ring
#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 64
Stack Ring Spatial Reuse
4
3
1
2
Assuming
4 x 24-port
9300 Switches

 Credit based system on


the Stack Ring
 Multiple stack ports
grab the ring that is free
and they have credits
on to transmit
3
1
2
4
 Increases the stack ring
bandwidth to 480Gbps

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 65
Multicast Packet Path on Stack Ring
Assuming
4 x 24-port 3
1
2
4
9300 Switches
 One copy of the source packet
is placed on the rings
 Interested Stack Ports grab the
segments when they see them
 Packet segments travel the
whole ring back to source
 The source strips these
segments off the ring (Source
Stripping)
 Results in efficient replication of
multicast traffic for multiple
Stack Port receivers

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 66
How many Can I stack together?

 Up to 8 Switches can be stacked together


using back stacking cables
 All 9300 models are supported in the stack Up to 8
 All the switches in the stack should run the
same IOS and License

Enforced by Software

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 67
Stack Discovery
 Stack Interfaces brought online
LC Infra
 Infra and LC Domains boot in parallel
 Stack Discovery Protocol discovers Stack LC Infra
topology – broadcast, followed by neighborcast
 In full ring, discovery exits after all members LC Infra
are found.
 In half ring, system waits for 2mins LC Infra
 Active Election begins after
Discovery exits
Stack port 1 cable is connected and the link is up
Stack port 2 cable is connected and the link is up
Waiting for 120 seconds for other switches to boot
%IOSXE-1-PLATFORM: process stack-mgr: %STACKMGR-1-DISC_START: Switch 3 is starting stack discovery.
##All switches in the stack have been discovered
Switch number is 3
%IOSXE-1-PLATFORM: process stack-mgr: %STACKMGR-1-DISC_DONE: Switch 3 has finished stack discovery.
%IOSXE-1-PLATFORM: process stack-mgr: %STACKMGR-1-SWITCH_ADDED: Switch 3 has been added to the stack.

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 68
Stack Active Election

A
1) The stack (or switch) whose member
has the higher user configurable
priority 1–15

2) The switch or stack whose member


has the lowest MAC address

%IOSXE-1-PLATFORM: process stack-mgr: %STACKMGR-1-ACTIVE_ELECTED: Switch 3 has been elected ACTIVE.

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 69
Stack Initialization
 Active starts RP Domain (IOSd, WCM, etc)
locally 2min timer
 Programs hardware on all LC Domains
LC RP Infra A
 Traffic resumes once hardware is programmed
RP LC Infra
S
 Starts 2min Timer to elect Standby
in parallel
 Active elects Standby LC Infra

 Standby starts RP Domain locally


 Starts Bulk Sync with Active RP LC Infra

 Standby reaches “Standby Hot”


GUIDELINE#show switch
Switch/Stack Mac Address : 2037.0652.a580 - Local Mac Address
Mac persistency wait time: Indefinite
H/W Current
%STACKMGR-1-STANDBY_ELECTED: 3 stack-mgr: Switch 2 Switch# Role Mac Address Priority Version State
has been elected STANDBY. ------------------------------------------------------------
1 Member 2037.0653.ca80 5 P6A
P6A Ready
Ready
2 Standby 2037.0653.db00 10 P6A HA
Ready
sync in progress
*3 Active 2037.0652.a580 15 V01 Ready

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 70
HA Best Practices & Recommendations

 Power up the first Switch that you want to make Catalyst9300#switch 1 priority 15
it as Active
A
 Configure Priority of the switch (1-15) – 1 by
default – the higher the better Catalyst9300#switch 2 priority 14

 Power up the second member that you want to S


make as Standby & then power up rest of the Catalyst9300#switch 3 priority 13
members
 To add a member to an existing stack plug in the
stack cable first, then power up the switch Catalyst9300#switch 4 priority 12
 Avoid stack Merge & Stack split if possible

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 71
Stack Member Addition

 Stack discovery initiated and completed


 Plug in the member, completing full ring RP A
 Power up the member
RP S
 Stack Discovery process runs and
completes immediately after
discovery happens
 Active detects the new addition, and
programs the hardware of the member Infra
LC
 Active is not pre-empted by powering on
another member even if it was
High Priority

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 72
Stack Member Addition – Software Upgrade

 All stack members must have common


IOS software version to pair in SSO RP A
redundancy state
 Stack member with version mis-match RP S
with ACTIVE switch will fail to RPR
mode
 Enable “software auto-upgrade enable”
command to automate upgrade process
 System must boot in install mode LC Infra
(default and recommended). Auto
Upgrade not supported in Bundle Mode
Auto Upgrade

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 73
Stack Member Deletion

 Stack discovery initiated and completed


A
 Active detects member removal – and
Clean up process is initiated
S
 Clean-up involves removing TCAM
entries referencing removed member,
MAC addresses, CDP tables – more
like all ports on the member are
shutdown

 Configuration is moved to
Pre-Provisioned state

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 74
Fast Software Upgrade
Fast Software Upgrade
Regular Upgrade Vs Fast Software Upgrade Process

#Install add file image activate commit #Install add file image activate Fastreload
commit

Traffic is impacted throughout the upgrade cycle Traffic impact is reduced by more than half by
seperating the control and data plane updates

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 76
Normal Vs Fast Software Upgrade Impact
Normal Upgrade Time Fast Software Upgrade Impact

50 %
Reduction in
Traffic

Reload Cmd
Traffic Impact
Reload Cmd

472* sec
Traffic
Kernel

IOSd

Kernel
233* sec

IOSd
Control Plane is updated Data Plane is updated
Control Plane + Data Plane impacted during the complete upgrade without any impacting any traffic impacting the traffic

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 77
Fast Software Upgrade
Supported and Unsupported Designs
Layer 2 Access Layer Designs– FSU Supported

STP
L2 Only L2 Only
x x Vlan1-10
L2 Only
MEC MEC
Access Access Access
Layer Layer Layer

Unsupported Designs

L2 Extensions with
Access Layer Device L3 connections with
Routing Protocols

Access Future
Layer

#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 78
Fast Software Upgrade
CLI Commands

• FSU is supported only in install mode


• One step command which activates the fast software upgrade and
commits it

install add file flash:cat9k_iosxe.BLD_V168 activate reloadfast commit

• Fast Reload without Software upgrade

Reload Fast

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 79
StackPower
Power HA - StackPower

HA with 1+N Flexible Power


Zero RPS Redundancy and Resiliency
Footprint Efficient

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 81
Power Redundancy Options
Zero Footprint RPS OR XPS
Day 1

StackPower - Zero Footprint RPS eXpandable Power System (XPS)

Stack of 4 switches Stack of 8 switches


#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 82
How StackPower Works?
StackPower
715 W
• Pools Power from All PS
• All Switches in StackPower
DataStac 715 W 1100W share the available Power in
k
Pool
1100 • Each Switch is given their
715 W
W Minimum Power Budget
715
W

Total Input Power 2530W

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 83
Power Budget Modes
1100 1100
715 W 715 W
W W
715 715
W W

2530W – 30W 1430W – 30W


Power Sharing Mode Redundant Mode

• The Default Mode • User Configurable


• Sum of All PS – 30~60W • Sum of All PS – Largest PS - 30~60W
Global StackPower Reserve = 30W

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 84
Power Priority
Load Shedding
• Standalone Mode • Stack Mode

Low Priority
Low Priority

Load Shedding Based on configured


priority Load Shedding Based on configured
priority
1. Low Priority Ports
2. High Priority Ports 1. Low Priority Ports
2. High Priority Ports
3. Switch Priority – Highest Priority
#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 85
Scale - TCAM,
Queues, Memory,
ACLs…
Higher and Flexible ACL Scale
Feature Resources
ACL Resources
IPv4 Entries 5000 Entries

• Security ACL: 5K IPv6 Entries Half the IPv4


Lookup Lookup Lookup Lookup
Table Table Table Table

• QoS ACL: 5K One type of IPv4 ACL (RACL, 5000 Entries


Lookup Lookup Lookup
PACL, VACL, GACL*)
• Service ACL: 5K
Lookup
Table Table Table Table

• Netflow ACL L4OPs/Label 8 L4OPs


Lookup• SPANLookup Lookup Lookup
Table• MACsec Table Table Table

• CoPP ACL – Ingress/Egress


• Tunnel
Lookup Lookup Lookup Lookup
Table• LISP Table Table Table
1/1/ 1/1/
2 1
• Each ACL policy is
1/1/
reference by a label. 3
This same label is
assigned to multiple
interfaces and VLANs
Label1
#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 87
Netflow on Catalyst 9300
Netflow Enabled Device
Traffic
• Flexible Netflow / Full Netflow
• Source IP Netflow Cache
• Dest IP Flow Packets Bytes/Pac
64k/ASIC • Source Port Information ket
• Dest Port
IP,Ports 32000 1100
• L3 Protocol
32k
IPv4 • Src MAC
Both Ingress and
Ingress egress netflow !
supported
simultaneously
! Etc..
Export
32k Distributed Collector
Forwarding/Export X86 1.8 GHZ
IPv4 CPU
Egress
No impact on CPU
with default timers

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 88
QoS Fundamental Actions
Conditional
Policing Marking

Trust Classification
Unconditional
Marking

Conditional
Marking Policing
Scheduler
8q3t PQ1
Classification PQ or Q
1p7q3t
PQ2
2p6q3t WTD rest
Unconditional Q3
Marking of non
WRED WTD
Q4 queues
Q5
WRED
Q6 on any 4
queues
Q7
Q7

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 89
Catalyst 9300– QoS Scalability

QoS Scale Numbers


Class-maps (Ingress) 1024
Class-maps (egress) 512
Table-maps (ingress) 16
Table-maps (egress) 16
Aggregate Policers 2000
Microflow Policers 24000
Queues/port 8 queues
Buffer/ASIC 16 MB

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 90
QOS – Higher Packet Buffer •

8MB/Core, 16MB/ASIC
Shared across Ingress and Egress
• IQS and SQS intelligently shared the common-shared,
Packet • Buffer organized in cells of 256 Bytes each
Packets to Egress Port Queues Buffer

• WRED Support Added

QOS
5 MB AQM

• Line rate across all ports


• Jumbo Frames Support
Performance

0.5 MB
1.5 MB SQS
1 MB IQS
Packet
Holding Buffer
Packets from the Stack And
Locally Switched Packets
Packets going to Stack
#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 91
Catalyst 9300 Software
IOS XE Evolution
Same Look & Feel, More Powerful Architecture
IOS IOS XE 3.7.x(SE) IOS XE Denali 16.1.1

IOS IOSd Hosted Apps IOSd Hosted Apps

WCM LXC* WCM


Features Features
Features Components Components Components
Wireshark LXC* Wireshark

Common
CommonInfrastructure
Common Infrastructure//HA
HA Infrastructure / HA
Management
ManagementInterface
Management Interface Interface
Crimson
ModuleDrivers
Module Drivers Module Drivers DB

Kernel
Kernel Kernel

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 93
IOS-XE 16

RAFA
One Release Train (Run Any Feature
Operational Efficiency,
Anywhere)
Consistency in Control Plane
Feature Velocity across
Behavior,
Platforms

Patch Updates Comprehensive


Secure Platform
WCM/SANET/etc sub
Programmability
64 Bit ASLR, Mandatory
package upgrade, Peace of Object based model, Access Control for Processes
mind for Customers Netconf/REST Interfaces

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 94
Software-Defined Access
Solution Components
DNA Center:
Simple Workflows

DESIGN PROVISION POLICY ASSURANCE

DNA Center

Network Data Platform APIC-EM Identity Services Engine

Routers Switches Wireless AP WLC

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 95
Catalyst 9300 – Trustworthy Systems
Image PnP
Signing SUDI Support
Authentic OS Two Way Trust

Hardware
Authenticity
Genuine
Hardware

Runtime
Defenses
64 Bit ASLR
Secure Integrity
Trustworthy Systems Boot Verifications
Boot Sequence Malware
Check Protection

Catalyst 9K Family is Built with Security


#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 96
Catalyst 9K Family – Encrypted Traffic Analytics

Cognitive Threat Analytics Engine


StealthWatch
Malware
Records

Cognitive Learning

Benign
Records

Netflow with New Extensions

Encrypted
Threat Analytics
Catalyst 9K Family
#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 97
Catalyst 9300 – Leadership in PoE Features

2-event
Perpetual UPOE Fast UPOE
classification
• Fast power • Uninterrupted PoE • Bypasses Cisco IOS®
negotiation power during control control plane boot
without LLDP plane reboot • Restores power to
• Physical layer PD within 30 sec of
negotiation < 1 sec power resumption

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 98
Graceful Insertion and Removal (GIR)

Hardware
Gracefully remove
replacement or insert a node
without impacting Protocol
traffic exchange
IS-IS,BGP,
Layer 3 OSPF
Software
Layer 2
upgrades
Shutdown

Shutdown

Configuration
changes

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 99
Catalyst 9K Family – Programmability &
Automation

B Open Open
ZTP BootLoader Config

PnP
YANG

Device Bootstrap and Configuration Automation Server Management Tools on


Onboarding through Open Interfaces x86 Infrastructure

Catalyst 9K Offers Complete DevOps Toolkit


#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 100
Catalyst 9K Family – Containers & App Hosting

Linux
Applications
3rd Party Applications

Guest Shell

Open Application Container

API LXC
Network OS

CPU Storage Containers

Catalyst 9K Enables 3rd Party App Integration


#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 101
Wrap up…
Cisco Catalyst 9000 – built to see you through
the next decade
UADP 2.0, 3.0 Standard leadership Platform innovations

Ethernet Alliance and


NBASE-T

IEEE

• Programmable pipeline • 802.3bz density (Multigigabit) • High Availability


• Flexible tables • 802.11ax optimized • X86 with TB storage
• Cisco StackWise® Virtual • 802.3bt scale (60W) • Campus-optimized
• Campus-optimized 25G/100G thermals, mechanicals
• Bluetooth
* Not available at FCS
#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 103
Catalyst 9000 redefines intelligence at the edge
The power of open Cisco IOS XE

Traditional networking Catalyst® 9000 platform


Data models
Standards based, structured
Proprietary, unstructured (MIBs, CLI)
(IETF YANG, OpenConfig)

Data representation and export


Fast, reliable, flexible, streaming
Slow, error-prone, query-based (SNMP)
(NETCONF - XML, RESTCONF – JSON, gRPC)

Data
Raw, Layer 1 to Layer 3 oriented Locally enriched, application oriented

Application Traffic URL stats Policy and Endpoint


response prioritization security profiles
times analytics

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 104
Enterprise Audio-Video Market Is
Ready for Disruption
AV Network over Ethernet Multipurpose Rooms Auditoriums

Audio Player

AVB Talker

Ceiling
Microphones
AVB Switch Cisco®
Catalyst® 3850

AVB Listener Installed Audio Video Distribution


Corporate offices, education, Entertainment, government are
Control PC
entertainment, and healthcare are primary verticals.
some primary verticals. Typical deployments in:
Wall Ceiling  Typical deployments in:  Casinos
Speakers Speakers  Conference Rooms  Courtrooms
AV Talkers and Listeners (endpoints)  Digital Workspaces  Auditoriums
communicating through AVB switch  Auditoriums
AV Endpoints: Speakers, Audio Player, PA systems, DSP devices  Lecture halls
 Multipurpose rooms

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 105
Application Visibility & Control
• Filter Monitoring
Over
Ingress/Egress
interfaces and
direction

• Identify Top Talkers

• Monitor Data over


2, 24 or 48 hours

• Monitor percentage
Bandwidth usage

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 106
MPLS Enables Network Segmentation in Campus

POS Medical Device


Network Other
Network
Doctor Staff

Line of business – BU segmentation Payment Card Industry Hospital Network

INTERNET

Bring-Your-Own-Device (BYOD) Mergers and Acquisitions Multi-Tenancy

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 107
The Catalyst 9K Family
100/40
SD Programmability
/25/10 Security
Access & Automation
/1G

NAT/ App
MPLS AVB
GRE Hosting

DNA Hot
UPoE MultiGigabit
Bonjour Patching

Open and
UADP 2.0/3.0 Extensible
IOS-XE

Enabling the New Era of Networking


#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 108
Some Early Recognitions…

6 Reasons to Migrate to the Catalyst 9000

Cisco Catalyst 9000 Series of Switches –


Software Consumption Done Right!

Cisco Catalyst 9000 Series Switches –


Extending cloud to your network edge

Cisco Catalyst 9000 Series of Switches – Built


with YOU in mind

How the Cat9K Advances Convergence

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 109
#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 110
Catalyst 9K
Subway Learning Map You Are Here

Sunday
Monday Tuesday Wednesday Thursday
(June
(June 11) (June 12) (June 13) (June 14)
10)
8:00–6:00 8:00–
12:00 - 4:00 4:00 – 8:00 8:00–12:00 12:00 - 4:00 4:00 – 8:00 8:00–12:00 12:00 - 4:00 4:00 – 8:00 8:00–12:00 12:00 - 4:00 4:00 – 8:00
12:00

LTRCRS-2017 LTRCRS-2090
Catalyst 9K Innovations Lab Catalyst 9K HA Lab

BRKARC-2035 (1)
Catalyst 9000 Architecture

BRKARC-2007 (1) BRKCRS-2004


TECARC-2901 BRKARC-3873 Application Hosting & Model
Catalyst 9K Seminar Catalyst 9400 Architecture Catalyst 9500 Architecture
BRKCRS-2650 Driven Telemetry
Next Gen High Availability

BRKARC-3863 BRKARC-2007 (2)


BRKARC-2035 (2) Catalyst 9500 Architecture
Catalyst 9300 Architecture Catalyst 9000 Architecture

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Visit World of Solutions…

Catalyst 9300/9500 Catalyst 9400 Catalyst 9K Demos

Reinventing the Enterprise Networks

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 112
Complete your online session evaluation

Give us your feedback to be entered


into a Daily Survey Drawing.
Complete your session surveys through
the Cisco Live mobile app or on
www.CiscoLive.com/us.
Don’t forget: Cisco Live sessions will be available for viewing
on demand after the event at www.CiscoLive.com/Online.

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 113
Continue
your Demos in
the Cisco
Walk-in
self-paced
Meet the
engineer
Related
sessions
education campus labs 1:1
meetings

#CLUS BRKARC-3863 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 114
Thank you

#CLUS
#CLUS

You might also like