Introduction To P Router

Tx Matrix Overview
Terminology
Terminology Description
A single logical routing system

A Routing Matrix consisting of a Fabric Chassis’ and
interconnected T1600 Routing Nodes
Centralized 25T switch card chassis’

TXP Matrix used for interconnecting T1600
Routing Nodes
Either an upgraded T640 or new

T1600 Routing Node which delivers
T1600 Routing Node
subscriber services within a Routing
Matrix system
2 Copyright © 2009 Juniper Networks, Inc. www.juniper.net

T-Series capacity
TX++
40960 10Gports,
8192 40G ports
320T and beyond
.
.
100G/ .
slot T1600 TX Matrix Plus
64 10G ports, 1280 10GEports,
16 40G ports 256 40G ports
Slot
Capacity
40G/
slot
T640 TX Matrix
32 128
10Gports, 10Gports,
8 40Gports 32 40G ports
3 Density
Copyright © 2009 Juniper Networks, Inc. www.juniper.net
What is a TX Matrix Plus routing node?
A Matrix built with T1600 technology!
25 Tbps, 16 x T1600 Routing Node
128 FPC slots with 100G/slot capacity
Interfaces:OC3—OC192 SONET/SDH;
100G, 1G, 10G, 40G Ethernet (IQ);
OC-12 ATM(IQ); DS-3, E-3, Ch-OC12 (IQ); Tunnel services
Routing features: complete feature set for IGP, BGP, MPLS, VPN, Logical
routers, Multicast, IPv6; extensive QoS capabilities; predictable latency and
jitter
High availability features: fully redundant hardware, MPLS fast reroute,
aggregated interfaces, protocol graceful restart, graceful RE switchover, ISSU,
NSR

TX Matrix Plus system description
Chassis
 23” rack enclosure
 Central Routing Entity
 Single Management Interface
 3 Stage CLOS Switch Fabric
 16 horizontal slots for F13
 16 vertical slots for F2
 Redundant power
 Redundant cooling
 Multiple planes per chassis

TX Matrix Plus scaling capabilities
1-16 Line card chassis
128 100GbE
1280 10GbE
256 OC-768/STM-256
1024 OC-192/STM-64

100GE on TX Matrix Plus
T1600 is the only platform that supports 100 Gbps per slot of minimum packet
sized Ethernet traffic
TX Matrix Plus will support up to 128 100GbE connections
Providing 100GE in a timely fashion is key to delivering
cost-effective network virtualization
100GE PIC—Highlights
Only full line rate 100Gbps interface card in the industry

Full IEEE 802.3ba (100GE) compatibility
Full SW support for the 100G traffic management and statistics
Pluggable optical modules support
Dual height PIC type 4
Compatible with existing T1600 FPC-4

A Routing Matrix: Systems Architecture

A Routing Matrix: Systems Architecture
TXP
Matrix Routing Engine Routing Engine
Primary Secondary Matrix Control Paths
(SFC) Matrix Data Paths
T1600 (LCC) T1600 (LCC)
RE: Local RE Redundant RE RE: Local RE Redundant RE
FPC FPC FPC FPC FPC FPC
PIC PIC PIC PIC PIC PIC
TXP Matrix: T1600 Routing Nodes:

 Manages Control Plane & Routing  Fabric Interconnects
 3 Stage CLOS Switch Fabric  RE’s: local chassis management
  Distributed Packet Forwarding
Single Management Interface

TXP SFC description
SFC with 4 LCC

RE/CB 0 RE/CB 1
SFC with 16 LCC Standby Main Chassis
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
RE/CB 0
Standby
RE/CB 1
Main
LCC 00
LCC 02
F13 SIB
F13 SIB
LCC 01
LCC 03
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
Blank
LCC 00 LCC 01
LCC 00 F13 SIB LCC 01
LCC 02
F13 SIB
F13 SIB LCC 03

 Single Management Interface
F2 SIB
 16 horizontal slots for F13
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
Blank
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB

LCC 06
LCC 08
F13 SIB
F13 SIB
LCC 07
LCC 09 LCC 02 F13 SIB LCC 03
 16 vertical slots for F2
 Redundant power
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB

LCC 12
LCC 14
F13 SIB
F13 SIB
LCC 13
LCC 15
LCC 02 F13 SIB
Blank
LCC 03  Redundant cooling
 Multiple planes per chassis
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
LCC 00 F13 SIB LCC 01 LCC 00 F13 SIB LCC 01
LCC 02 F13 SIB LCC 03 LCC 02 F13 SIB LCC 03

Blank
Blank
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
Blank
TXP Routing Matrix: Switching Architecture
SFC with
RE/CB 0 RE/CB 1
4 LCCs Optical Standby Main
RE/CB 0 RE/CB 1 Data Plane
Standby Main
T1600 Plane 0 - Standby
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
T1600 Plane 1
LCC 00 F13 SIB LCC 01 T1600 Plane 2
LCC 02 F13 SIB LCC 03 T1600 Plane 3
T1600 Plane 4
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
Blank

LCC 02 F13 SIB LCC 03 TXP:
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
Blank
LCC 00 LCC 01
F13 SIB
 1 Chassis, 5 Planes
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
LCC 00 LCC 01
LCC 02
F13 SIB
F13 SIB LCC 03
Blank  Single Management Interface
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB
F2 SIB

LCC 02 F13 SIB LCC 03 T1600 Routing Nodes:
Blank  Fiber interconnect
Blank
 RE’s: local chassis management
Blank
 Distributed Packet Forwarding

Building block
crossbar switch
Ingress Egress
16 x 16 crossbar
PFE 0 PFE 0
Any input to any output
Non-blocking design
Cornerstone technology
 T1600
 MX
 TX Matrix Plus
PFE 15 PFE 15

Building block
3 stage CLOS
64 x 64 Crossbar F1_00: Top SIBF13,
Even LCC
SG/Port Phys
00 1/1 14 00 00
F2_00 Slot 0
Phys
8 00
F3_00: Top SIBF13,
Even LCC
Phys SG/Port
00 14 1/1 00
01 1/3 15 01 01 9 01 01 15 1/3 01
02 3/1 12 02 02 0 02 02 12 3/1 02
Switch output is connected to the

03 3/3 13 03 03 1 03 03 13 3/3 03
04 0/1 10 04 04 12 04 04 10 0/1 04
05 0/3 11 05 05 13 05 05 11 0/3 05
06 2/1 8 06 06 4 06 06 8 2/1 06
input of next stage 07 2/3

08 1/2
09 3/2
9
6
7
07
08
09
07
08
09
5
10
11
07
08
09
07 9
08 6
09 7
2/3 07
1/2 08
3/2 09
10 1/0 4 10 10 2 10 10 4 1/0 10
11 3/0 5 11 11 3 11 11 5 3/0 11
12 0/2 2 12 12 14 12 12 2 0/2 12
This is single plane 13 2/2

14 0/0
15 2/0
3
0
1
13
14
15
13
14
15
15
6
7
13
14
15
13 3
14 0
15 1
2/2 13
0/0 14
2/0 15
F3_00: Top SIBF13,
First stage sprays cells evenly

F1_00: Top SIBF13,
Odd LCC Odd LCC
F2_01 Slot 2
SG/Port Phys Phys SG/Port
Phys
00 1/1 14 00 00 00 00 14 1/1 00
8
01 1/3 01 15 1/3 01
15 01 01 9 01
02 3/1 02 12 3/1 02
12 02 02 0 02
Second and third stage transmit

03 3/3 13 03 03 03 03 13 3/3 03
1
04 0/1 10 04 04 04 04 10 0/1 04
12
05 0/3 11 05 05 13 05 05 11 0/3 05
cells according to destination

06 2/1 06 06 06 06 8 2/1 06
8 4
07 2/3 9 07 07 07 07 9 2/3 07
5
08 1/2 6 08 08 10 08 08 6 1/2 08
address(PFE )
09 3/2 7 09 09 09 09 7 3/2 09
11
10 1/0 4 10 10 10 10 4 1/0 10
2
11 3/0 5 11 11 11 11 5 3/0 11
3
12 0/2 2 12 12 14 12 12 2 0/2 12
13 2/2 3 13 13 15 13 13 3 2/2 13
14 0/0 14 14 14 14 0 0/0 14
Fabric cell comments

0 6
15 2/0 1 15 15 7 15 15 1 2/0 15
F1_00: Bottom SIBF13, F3_00: Bottom SIBF13,

Even LCC Even LCC
F2_02 Slot 4
SG/Port Phys Phys SG/Port
Grant
Phys
00 1/1 14 00 00 8 00 00 14 1/1 00

01 1/3 01 9 01 01 15 1/3 01
15 01
02 3/1 02 0 02 02 12 3/1 02
12 02
03 3/3 13 03 03 1 03 03 13 3/3 03
04 0/1 10 04 04 12 04 04 10 0/1 04

Request 05 0/3
06 2/1
07 2/3
11 05
8
9
06
07
05
06
07
13
4
5
05
06
07
05 11
06 8
07 9
0/3 05
2/1 06
2/3 07
08 1/2 6 08 08 10 08 08 6 1/2 08
09 3/2 09 09 11 09 09 7 3/2 09
Payload
7
 10 1/0 4 10 10 2 10 10 4 1/0 10
11 3/0 5 11 11 3 11 11 5 3/0 11
12 0/2 2 12 12 14 12 12 2 0/2 12
13 2/2 3 13 13 15 13 13 3 2/2 13
14 0/0 0 14 14 6 14 14 0 0/0 14
15 2/0 1 15 15 7 15 15 1 2/0 15
F1_00: Bottom SIBF13, F3_00: Bottom SIBF13,

Odd LCC Odd LCC
F2_03 Slot 6 Phys SG/Port
SG/Port Phys
Phys 00 14 1/1 00
00 1/1 14 00 00 00
8
01 1/3 01 15 1/3 01
15 01 01 9 01
02 3/1 02 12 3/1 02
12 02 02 0 02
03 3/3 03 13 3/3 03
13 03 03 1 03
04 0/1 10 04 04 10 0/1 04
04 12 04
05 0/3 11 05 05 11 0/3 05
05 13 05
06 2/1 8 06 06 8 2/1 06
06 4 06
07 2/3 9 07 07 9 2/3 07
07 5 07
08 1/2 6 08 08 6 1/2 08
08 10 08
09 3/2 7 09 09 7 3/2 09
09 11 09
10 1/0 4 10 10 4 1/0 10
10 2 10
11 3/0 5 11 11 5 3/0 11
11 3 11
12 0/2 2 12 12 2 0/2 12
12 14 12
13 2/2 3 13 13 3 2/2 13
13 15 13
14 0/0 14 14 0 0/0 14
13
0
Copyright © 2009 Juniper Networks, Inc. www.juniper.net
14 6 14
15 2/0 1 15 15 1 2/0 15
15 7 15
TXP Switch Fabric
(4 LCCs one plane view on SFC)
Ingress PFE SFC Egress PFE
LCC 0 LCC 0
…
F1 F2 F3
…
FPC 0-7 FPC 0-7
LCC 1 LCC 1
…
…
F1 F2 F3
…
FPC 0-7 FPC 0-7
LCC 2 LCC 2
…
…
F1 F2 F3
…
FPC 0-7 FPC 0-7
LCC 3 LCC3
…
…
F1 F2 F3
…
FPC 0-7 FPC 0-7
F13 SIB_in F2 SIB F13 SIB_out

One F13 SIB
4*F2 SIB and 2*F13 SIB makes “one Plane”
TXP CLOS Switch Fabric (16 LCCs one plane view on
SFC)
Ingress PFE Egress PFE
LCC 0 LCC 0
…
F1 F2 F3
…
FPC 0-7 FPC 0-7
LCC 1 LCC 1
…
…
F1 F2 F3
…
FPC 0-7 FPC 0-7
…
…
LCC 14 LCC 14
…
…
F1 F2 F3
…
FPC 0-7 FPC 0-7
LCC 15 LCC15
…
F1 F2 … F3
…
FPC 0-7 FPC 0-7
F13 SIB F2 SIB F13 SIB

SFC and LCC fabric inter-chassis connection
one plane view
High Speed Link

4 Fiber optical
array cables
T1600 LCC0 TXP SFC
SIBF13
FPC0~7 SIB_L SIBF2S
F1_0 F2
PFE0 ASIC VCSELs
(ODD)
F3_0 SIBF2S
ASIC F2
VCSELs
ASIC
PFE15 F1_1 SIBF2S
VCSELs
(EVEN)
F2
ASIC
F3_1
SIBF2S
F2
To LCC 1

TXP Fabric Architecture
PFE view
HSL operates at
20G(max)

LCC fabric intra-chassis connection
5 plane view
Cable A/B/C/D FPC7
To SIB_F13 SIB_L(slot 0) FPC6
FPC5
Cable A/B/C/D
FPC4
To SIB_F13
SIB_L(slot 1) FPC3
FPC2
Cable A/B/C/D
FPC1
To SIB_F13
SIB_L(slot 2) FPC0
Cable A/B/C/D
To SIB_F13 SIB_L(slot 3) PFE 0
TOP
Cable A/B/C/D
To SIB_F13
SIB_L(slot 4)
PFE 1
Bottom
Electrical High Speed Link
Each line equivalents to 2
HSL(one to top PFE, one to
bottom PFE)
Each HSL operates at 20G(max)
TXP Fabric Architecture
5 plane system overview

TXP Fabric Physical connection ( 2 LCCs, rear view)

SIB-TXP-3D-LCC (on LCC)
Providing connectivity between the
PFE(on FPC) and the SIB F13 on
SFC
 No fabric chip on it
 Electrical to optical conversion of
High speed link
 Conversion between HSL1 and
HSL2 to accommodate Gimlet
FPC(the SG chip)
 Each SIB has 4 optical VCSEL
cables
 5 SIBs on each LCC
 4 +1 redundancy
 Requires the presence of the new
Rear Fan Tray
SIB-TXP-3D-F13 ( on SFC)
Consists of stage 1 and stage 3
fabric chip, providing optical
ingress/egress connectivity to LCCs
and electrical connectivity to stage
2 fabric chip
Port 0 – 7: LCC 0
Port 8 – 15: LCC 2

SIB-TXP-3D-F2S (on SFC)
Consists of stage 2 fabric chip,
providing electrical connectivity
to SIB-F13
 “S” refers to Single SFC
Chassis
 4 SIB-F2s forms a plane
 20 SIB-F2s must be fully
installed regardless of number
of LCCs

Fiber Optic array cable
Total 72 fibers links within each bulkhead connector( each cable)
 36 RX
 36 TX
4 optic cables on each SIB , each cable carries 1 SG’s data to

the CLOS Fabric on SFC.
Each fiber optic cable carries 4 HSL2 link
 On TX Matrix, each cable carries 16 HSL1 links per SIB
 Inter-chassis bandwidth is doubled comparing with TX Matrix
Each SG connects to the 4 PFEs on 2 FPCs.

 4 cables required per SIB_L to connect all 16 PFE within one LCC
to SFC fabric
20 optic cables per plane (4 LCC)
80 optic cables for 5 plane (4LCC)
Fiber Optic array cable
Detail view of AOC Cable

Optic Cable lengths
The fiber-optic array cables connect the switching planes of the

TX Matrix Plus router.
The TXP-F13 SIBs connect to the TXP-T1600/T4000 SIBs.
 Cable lengths from 8 m to 25 m are supported
You must use the same length for every fiber-optic array cable
from a particular T1600 router(LCC) to the TX Matrix Plus router.
However, you do not need to use the same length for all fiber-
optic array cables within a routing matrix.

Optic Cable Management
reading LED status
Two multi-colored LEDs provide optical cable status for each
fiber trunk cable:
 Bottom LED is “RX Optical Power”. It is purely a measure of the
received optical power on all the channels in the fiber bundle.
 Top LED is “Link”. It only activates when “RX Optical Power” goes
green and gives high-level link status.
Link
RX Optical Power

Optic Cable Management
reading LED status
Rx Optical Power Dark = No optical power seen on any channel.
Cable not plugged in on both sides, or loopback not fully inserted
RX Optical Power Yellow = Optical power seen on some channels,
but not all. Cable is dirty or damaged. Bulkhead connector may
be dirty.
RX Optical Power Green = Good optical power seen on all
channels.
Link LED = Flashing Yellow. There is a cable mismatch between
the LCC and SFC sides. Swap cables to correct.
Link LED = Yellow. Transit states indicating HSL2 link is being
established between the fabric ASICs over the cable
Link LED = Green. Successfully established fabric ASICs HSL2
link across the optical connector
SFC: Front view
F2S SIB
Routing Engine (RE)
4 F2S SIB/Plane
Control Board (CB)
Connector Interface
Panel (CIP)

SFC: Rear view
Each F13 SIB connects

to two LCC

F2SIB Slot Numbering (SFC front view)

F2SIB Slot Continued ..
Plane-0 : Slot# 0/0, 0/2, 0/4, 0/6
Plane-1 : Slot# 1/0, 1/2, 1/4, 1/6
Plane-2 : Slot# 2/0, 2/2, 2/4, 2/6
Plane-3 : Slot# 3/0, 3/2, 3/4, 3/6
Plane-4 : Slot# 4/0, 4/2, 4/4, 4/6


F13 SIB slot numbering (SFC rear view)
F13SIB Slot#0
Plane 0
F13SIB Slot# 1
Unused
Each F13 SIB connects
Plane 1
F13SIB Slot#4
Unused to 2 LCCs
Plane 2
Two F13 SIB connect to
F13SIB Slot#7 4 LCCs that makes up
Center Divider
a plane
F13SIB Slot#8
Plane 3
Unused
Plane 4
F13SIB Slot#12
Unused
F13SIB Slot#15

Plane-0 : Slot# 0,1
Plane-1 : Slot# 3,4
Plane-2 : Slot# 6,7
Plane-3 : Slot# 8,9
Plane-4 : Slot# 11,12

TXP SIB states
Offline – the SIB is offline and powered off.

Empty – there is no SIB in the slot
Invalid – the specified slot is not a valid in the TXP4
Activating – intermediate state before SIB goes online/spare
Deactivating – intermediate state before SIB goes offline.
Online – SIB is online. SIB is eligible to carry traffic.
Check – The online/spare SIB has errors. Check state indicates the user needs to
check and fix the cause of the errors. Check state can happen due to link errors or
destination errors.
Fault – the SIB is faulted, it is kept powered on.
Disconnected – LCC SIB is not connected to the SFC SIB
SFC Error – SFC F13 SIB corresponding to the LCC SIB has an error.
Connected – LCC SIB is connected to the SFC SIB but the SFC plane is offline.
Check SFC show chassis fabric plane detail for the reason. This is a new SIB state.
Spare – SIB is spare. Check the plane state to see if the SIB is eligible to take over in
case of a fabric plane failure.

TXP SIB states
States during GRES

 4 active fabric planes are offline|onlined one by one
 Only 3 active planes available to carry traffic
States recovery
 You might need to offline|online to recover SIB states in “check”

TXP SIB plane failover
5 SIB plane
 Plane 0 by default is spare
 All other plane is active plane
An active plane
 4 F2SIB must be active
 At least one F13 SIB must be active
 LCC SIB must also be active
Plane switch over under following condition at system startup

 Fatal errors on SIBs in a plane.
 All inter-chassis HSL2 links to an LCC on a plane in error.
 CLOS errors above the threshold.
 Inter-chassis HSL2 link errors
TXP tends to be more strict on failover policy then TX
 Inter-chassis links are between F1/F3 and F2 on TX
 F1/F3 to F2 has reroute capability from HSL view point
TXP keeps monitoring runtime SFC/LCC error to determine the

switchover
 Mid-plane error
 Inter-chassis hsl2 link error
 Voltage /temperature
FPCs monitor whether they are losing cells sent over the fabric
on a per destination basis.
Each PFE is represented as two destinations – one for high
priority and one for low priority.
In an TXP4 system, there are a total of 128 destinations on each
fabric plane.
If the number of cells lost is beyond a threshold, the FPC stops
using that fabric plane for traffic to be sent to the affected
destination

SIB_0 SIB_0
SIB_1 SFC SIB_1
PFE SIB_2 SIB_2 PFE
F2s F2s
SIB_3 F13 F2s F2s F13 SIB_3
SIB_4 SIB_4
F2s F2s
LCC0 F13 F2s F2s F13 LCC1
F2s F2s
F13 F2s F2s F13
F2s F2s
F13 F2s F2s F13
LCC2 LCC3
SIB_0 F2s F2s SIB_0
F13 F2s F2s F13
SIB_1 SIB_1
PFE SIB_2 SIB_2 PFE
SIB_3 SIB_3
SIB_4 Fiber optic array bundle SIB_4
4 cable/bundle
TXP SIB plane failover—F13 SIB failure
SIB_0 SIB_0
SIB_1 SFC SIB_1
PFE SIB_2 SIB_2 PFE
F2s F2s
SIB_4 SIB_4
F2s F2s
F2s F2s
F13 F2s F2s F13
F2s F2s
F13 F2s F2s F13
LCC2 LCC3
SIB_0 F2s F2s SIB_0
F13 F2s F2s F13
SIB_1 SIB_1
PFE SIB_2 SIB_2 PFE
SIB_3 SIB_3
SIB_4 SIB_4
TXP SIB plane failover—F13 failure
SIB_0 SIB_0
SIB_1 connected SFC fault SIB_1
PFE SIB_2 SIB_2 PFE
F2s F2s
SIB_4 fault SIB_4
F2s F2s
F2s F2s
F13 F2s F2s F13
F2s F2s
F13 F2s F2s F13
LCC2 LCC3
SIB_0 F2s F2s SIB_0
F13 F2s F2s F13
SIB_1 connected fault SIB_1
PFE SIB_2 SIB_2 PFE
SIB_3 SIB_3
SIB_4 SIB_4
TXP SIB plane failover—F2s SIB failure
SIB_0 SIB_0
SIB_1 SFC SIB_1
PFE SIB_2 SIB_2 PFE
F2s F2s
SIB_4 SIB_4
F2s F2s
F2s F2s
F13 F2s F2s F13
F2s F2s
F13 F2s F2s F13
LCC2 LCC3
SIB_0 F2s F2s SIB_0
F13 F2s F2s F13
SIB_1 SIB_1
PFE SIB_2 SIB_2 PFE
SIB_3 SIB_3
SIB_4 SIB_4
TXP SIB plane failover—F2s SIB failure
SIB_0 SIB_0
SIB_1 connected SFC connected SIB_1
PFE SIB_2 SIB_2 PFE
F2s F2s
SIB_4 SIB_4
F2s F2s
F2s F2s
F13 F2s F2s F13
F2s F2s
F13 F2s F2s F13
LCC2 LCC3
SIB_0 F2s F2s SIB_0
F13 F2s F2s F13
SIB_1 connected connected SIB_1
PFE SIB_2 SIB_2 PFE
SIB_3 SIB_3
SIB_4 SIB_4
TXP SIB plane failover— LCC SIB failure
SIB_0 SIB_0
SIB_1 SFC SIB_1
PFE SIB_2 SIB_2 PFE
F2s F2s
SIB_4 SIB_4
F2s F2s
F2s F2s
F13 F2s F2s F13
F2s F2s
F13 F2s F2s F13
LCC2 LCC3
SIB_0 F2s F2s SIB_0
F13 F2s F2s F13
SIB_1 SIB_1
PFE SIB_2 SIB_2 PFE
SIB_3 SIB_3
SIB_4 SIB_4
TXP SIB plane failover—LCC SIB failure
SIB_0 SIB_0
SIB_1 fault SFC connected SIB_1
PFE SIB_2 SIB_2 PFE
F2s F2s
SIB_4 fault SIB_4
F2s F2s
F2s F2s
F13 F2s F2s F13
F2s F2s
F13 F2s F2s F13
LCC2 LCC3
SIB_0 F2s F2s SIB_0
F13 F2s F2s F13
SIB_1 fault connected SIB_1
PFE SIB_2 SIB_2 PFE
SIB_3 SIB_3
SIB_4 SIB_4
TXP SIB plane failover—Two SIB failure
SIB_0 SIB_0
PFE SIB_2 SIB_2 PFE
F2s F2s
SIB_4 fault SIB_4
F2s F2s
F2s F2s
F13 F2s F2s F13
F2s F2s
F13 F2s F2s F13
LCC2 LCC3
SIB_0 F2s F2s SIB_0
F13 F2s F2s F13
PFE SIB_2 SIB_2 PFE
SIB_3 SIB_3
SIB_4 SIB_4
TXP SIB plane failover—Two SIB failure

PFE SIB_2 SIB_2 PFE
F2s F2s
SIB_4 SIB_4
F2s F2s
F2s F2s
F13 F2s F2s F13
F2s F2s
F13 F2s F2s F13
LCC2 LCC3
SIB_0 connected F2s F2s fault SIB_0
F13 F2s F2s F13
PFE SIB_2 SIB_2 PFE
SIB_3 SIB_3
SIB_4 SIB_4
LCC rear view and SFC craft panel
Front panel with LCD UI, status

and alarm indicators
LCC with SIBs installed

Improved fiber optic
interconnects
SFC/LCC RE—shipping with TX Matrix Plus
Front Panel Accessible

 SSDs (two slots)
 4G Compact Flash
 USB 4G
 Offline button
Intel 64 bit Dual Core

Processor
Ships with 16G(SFC) or 8G(LCC)DRAM

 Modular DIMM memory
RE supported in standalone
T1600s

TXP Control plane
Control plane is connected via external ethernet links as well as via internal ethernet
switch on LCC-CB and SFC CIP
RPD only runs on the SFC( build routing table /execute management/configuration etc.)
there is no routing process on the LCC.
All the normal processes on a standalone router are present on the SFC
Routing updates are synced onto other RE via internal process

TXP Control plane
Single chassis Multichassis

T1600 master RE SFC master RE
rpd dcd chassis .. rpd dcd chassisd ..
d
kernel
kernel
LCC master RE
PFE chassisd ksyncd
kernel
chassis
manager pfeman LCC PFE
chassis
pfeman
manager

RE Switchover
RE mastership switchover occurs on loss of chassisd keepalives.
LCC switchover: LCC backup RE becomes master, chassisd and
ksyncd restart on new master, GFPCs reconnect.
SFC switchover: SFC backup RE becomes master, SFC
processes restart, LCC chassisd and ksyncd restart, GFPCs
reconnect, RPD performs protocol graceful restart.
Software supports simultaneous LCC and SFC switchovers.

TXP Control plane failover (SFC RE failed)
SFC RE Master RE
RE Backup RE
RE0 RE1
Ethernet Link
LAN Switch LAN Switch
LCC0 LCC1 LCC2 LCC3
RE0 RE1 RE0 RE1 RE0 RE1 RE0 RE1

Switch Switch Switch Switch Switch Switch Switch Switch
TXP Control plane failover (LCC RE failed)
SFC RE Master RE
RE Backup RE
RE0 RE1
Ethernet Link
LAN Switch LAN Switch
LCC0 LCC1 LCC2 LCC3
RE0 RE1 RE0 RE1 RE0 RE1 RE0 RE1

Switch Switch Switch Switch Switch Switch Switch Switch
Control Board (CB on SFC)
Two redundant in SFC
Paired with an RE (CB0 with RE0, CB1 with RE1)
Providing power to RE
Internal Switch to all FRU

Control Board (CB on LCC)
Two redundant in each LCC
Paired with an RE (CB0 with RE0, CB1 with RE1)
Providing power to RE
Internal Switch to all FRU
Provide control link to SFC CB
Support standalone mode
 upgrade single chassis

Connector Interface Panel (CIP on SFC)
Two redundant in SFC
Hot swappable
Hot removable
Provide control link to other LCC
Provide control link to other SFC
 For multiple SFC(TXP 16)

FPC supported on TXP
T640-FPC1-E2 , T640-FPC1-ES(requires 10.0+)
 4G slot bandwidth
T640-FPC2 , T640-FPC2-E2, T640-FPC2-ES(10.0+)

 16G slot bandwidth
T640-FPC3, T640-FPC3-ES,(T1600-FPC3-ES, 9.6+)

 Support 40G slot bandwidth
T640-FPC4-ES( 9.6+)
 One PIC only support 40G slot bandwidth
T640-FPC4-1P-ES(10.0+)
 One PIC only support 50G slot bandwidth, high density XGE PIC
T1600-FPC4-ES( 9.6+)
 4 PIC, support 100G slot bandwidth
TXP SFC PEM 7 X 60 A
New 7 X 60A PEM for SFC 12.75KW
max capacity, hot swappable 1:1
redundancy
 Individual breakers for each feed
 Removable tray for physical
connections
 Partially populated chassis support
using less than 7 feeds
Regulated and isolated,

with forced air-cooling by internal fans

TXP SFC Power Module
Two 7 input redundant power module

TXP SFC Power Distribution

TXP LCC Power Module
3 input version
Two 3x80A input redundant power module

TXP LCC Power Module
6 input version
Two 6x60A input redundant power module

Cooling on SFC
Two front fan trays
Three front air filters(Air intake)
Four rear fan trays
Two front fan

must work in pair

TXP Packet Flow - Ingress
PIC FPC T-Series FPC SIB

Internet
L2/L3 Processor
WAN SONET/SDH Packet
OC-192 ASIC Fabric
Processing
Switch Switch
Interface Interface
ASIC ASIC
L2/L3
WAN SONET/SDH Packet Queuing
OC-192 ASIC Processing and Memory
Interface ASIC
Queue
Parse packet
L2/L3
Perform header,
route lookup, ExtractFetch packet
lookup key, from
send data
Store packet data
pointers,
perform perform
policing SPQ
accounting,
filtering, and memory
to memory, and
send keytransmit
and data
in memory
scheduling,
segment RED cells
into 64B
queue selection drop cells
pointer to fabric
to route lookup

TXP Packet Flow - Egress
SIB FPC T-Series FPC

Internet WAN
Processor L2/L3 OC-192
Packet SONET/SDH
Fabric ASIC
Processing
Switch Switch
Interface Interface
ASIC ASIC WAN
L2/L3 OC-192
Packet SONET/SDH
Queuing ASIC
Processing
and Memory
Interface ASIC
Reorder cellspacket
Perform
Fetch from
routefabric,
lookup,
from Queue packet
Build L2/L3pointers,
header , perform
extract lookup policing
filtering, key, send data
and Store
WDRR packet datashape
scheduling,
perform in
accounting, output
memory and transmit
to memory,
queue send keyASIC
selection and memory
bandwidth, RED
reassemble drop
packet
cells to L2/L3
data pointer to route lookup

TXP high availability
TXP support GRES starts from 9.6 (first shipping release)
 Graceful RE switchover maintains PFE forwarding
 Routing state will be re-converged
 3 active SIB plane only during switchover
TXP support Graceful Restart

 OSPF,IS-IS,LDP,BGP,PIM,RIP,RSVP
 L2/L3-VPN,TCC,CCC
TXP support NSR starts from 10.0+

 Support OSPF, IS-IS, LDP, PIM and BGP
 Both routing and forwarding state maintained
 ~zero packet loss during switchover
ISSU
 Hitless software upgrade
 Minimal transit traffic loss
TXP migration path
T640 to TXP
 T640 to T1600 (smooth upgrade)
 T1600 to TXP
T1600 to TXP
 Smooth upgrade (10.1+)
 Mixed SIB can be used in upgrade mode
 Requires one reboot when single chassis connects to SFC
 Upgrade T1600-SIB-L,T1600-CB,T1600-RE,Rear Fan Tray
TX to TXP
 Cold upgrade only
 Future release support upgrade similar with T1600 to TXP on data
plane, in addition with a control plane transition from SCC to SFC

TXP migration path
T1600 to TXP upgrade steps:
1. Load same version of JUNOS on T1600 and TXP SFC.
2. Upgrade CB/RE
3. Check and upgrade FPC ROM
4. Set the 'upgrade mode' knob on T1600 and SFC (allow SIB mix)
5. Replace one (redundant) T1600-SIB-I with T1600-SIB-L at a time
6. Manually bring-up optic links on LCC and SFC, repeat step 5
7. Clear the 'upgrade mode' knob after upgrading all 5 planes.
8. Connect Ethernet links of the control plane from LCC to SFCs
9. Set the "Multi-chassis" (M/S) switch on LCC
10. On both T1600 CBs set chassis-ID to 0
11. Create and load a configuration on the Master/active RE of the
SFC, similar but not identical to the LCC configuration.
12. Reboot the LCC
VN2 TXP-3D
TXP-3D – Hardware configuration
Site Name Device Upgrade to Tx Matrix Remark
Hanoi HN-P1-T4000 T4000 T4000-LCC 0 HN-P1-TXP
HN-P4-T1600 T1600 T4000-LCC 2
HN-P2-T4000 T4000 T4000-LCC 0 HN-P2-TXP
Chassis offline T1600 T4000-LCC 2
Hochiminh HCM-P1-T4000 T4000 T4000-LCC 0 HCM-P1-TXP
HCM-P3-T1600 T1600 T4000-LCC 2
HCM-P2-T4000 T4000 T4000-LCC 0 HCM-P2-TXP
Chassis offline T1600 T4000-LCC 2

New TXPBASE-DC chassis bundle capable of connecting upto 2

TXP3D-2LCC-BNDL 1
T4000-LCC, 4 T1600-LCC or a mix.
JUNOS-WW JUNOS Internet Software Worldwide Version 1

(1) Base Bundle AOC to interconnect SIB-TXP-3D-LCC to SIB-
CBL-TXP-AOC-025MBB 80
TXP-3D-F13, 25 meters
Routing engine with dual core 2600MHz processor, SSD and
RE-DUO-C2600-16G-BB 2
16GB memory, Base Bundle
CB-TXP-BB Control Board for TX Matrix Plus, Base Bundle 2
CIP-TXP-BB Connector Interface Panel for TX Matrix Plus, Base Bundle 2
Switch Interface Board for TX Matrix Plus with 3D Technology,

SIB-TXP-3D-F13 5
fabric stages one and three
Switch Interface Board for TX Matrix Plus with 3D Technology,
SIB-TXP-3D-F2S 20
fabric stages two
Cable manger assembly for rear trunk fibers. Attaches to rear side
CBL-MGR-TXP-3D-SFC-S 1
of SFC where F13SIBs are

Switch Interface Board for LCC connection to TX

SIB-TXP-3D-LCC 5
Matrix Plus with 3D Technology
Rear Fan Tray for T-Series chassis configured as LCC of

FAN-R-TXP-3D-LCC-S 1
TX Matrix Plus with 3D Technology, Spare
Cable manger assembly for rear trunk fibers. Attaches to

CBL-MGR-TXP-3D-LCC-S 1
the LCCs that are part of to a TXP with 3D SIBs

Switch Interface Board for LCC connection to TX

SIB-TXP-3D-LCC 20
Matrix Plus with 3D Technology
PWR-T-6-60-DC-S Power entry module 6 X 60A DC, Spare 8
FANTRAY-T4000-S Front Fan Tray, Spare 8
Rear Fan Tray for T-Series chassis configured as

FAN-R-TXP-3D-LCC-S 4
LCC of TX Matrix Plus with 3D Technology, Spare
Routing engine with dual core 1800MHz processor,

RE-DUO-C1800-8G-S 8
SSD and 8GB memory, Spare
Control Board for T-Series LCC, use with TX Matrix

CB-LCC-S 8
Plus, Spare
CRAFT-T-SERIES-S T Series Craft Display Panel 4

Cable manger assembly for rear trunk fibers.
CBL-MGR-TXP-3D-LCC-S Attaches to the LCCs that are part of to a TXP with 4
3D SIBs

TXP-3D – Control Plane
SFC (TXP-3D) LCC
SFC CIP # CIP PORT # LCC Chassis LCC CB # CB Port #
TXP-CIP 0 Port 0 LCC 0 CB 0 Port 0

CB (LCC)
CIP (SFC)
Use UTP CAT5
The cable wiring is straight-
through (not crossover)

TXP-3D – Switching Plane
SFC F13 SIB LCC Chassis SIB SFC F13 SIB LCC Chassis SIB
Slot Port Chassis SIB Port Slot Port Chassis SIB Port
Slot 0 0 LCC 0 Slot 0 0 Slot 6 8 LCC 2 Slot 2 0

F13 SIB F13 SIB Name Data Supported TXP-3D

Slot Plane Configuration TXP-F13-3D to LCC
Slot 0 SIB F13 0 TXP-2 LCC 0 (Port 0-7), LCC 2 (Port 8-15)
0
Slot 1
Slot 2
1
Slot 4
Slot 5
2
Slot 7
3
Slot 9
Slot 10
4
Slot 12
Slot 13
Slot 14
Slot 15

o
Use AOC cable

TXP-3D – Interface numbering
Need to check with ae interface

TXP-3D – Upgrade Procedure
Step Event Traffic Impact Notes
1 Prerequesites No
2 Install the switch-fabric chassis (SFC) (TXP-3D) No
3 Upgrade the SFC (TXP-3D) to 64-bit Junos OS Release 13.3R7 No
4 Upgrade T4000 to 64-bit Junos OS Release 13.3R7 Yes
5 Upgrade all components in the T4000 except the SIBs No
6 Integrating the T4000 into the Tx Matrix Plus Yes
7 Upgrade T1600 to T4000 No
8 Integrating new T4000 into Tx Matrix Plus Yes
9 Rollback Yes

Introduction To P Router

Uploaded by

Copyright:

Available Formats

Introduction To P Router

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Introduction To P Router

Uploaded by

Copyright:

Available Formats

Tx Matrix Overview

A single logical routing system

Centralized 25T switch card chassis’

Either an upgraded T640 or new

2 Copyright © 2009 Juniper Networks, Inc. www.juniper.net

4 Copyright © 2009 Juniper Networks, Inc. www.juniper.net

 Central Routing Entity

 Single Management Interface

 3 Stage CLOS Switch Fabric

 16 horizontal slots for F13

 16 vertical slots for F2

 Multiple planes per chassis

5 Copyright © 2009 Juniper Networks, Inc. www.juniper.net

6 Copyright © 2009 Juniper Networks, Inc. www.juniper.net

Only full line rate 100Gbps interface card in the industry

7 Copyright © 2009 Juniper Networks, Inc. www.juniper.net

8 Copyright © 2009 Juniper Networks, Inc. www.juniper.net

T1600 (LCC) T1600 (LCC)

RE: Local RE Redundant RE RE: Local RE Redundant RE

FPC FPC FPC FPC FPC FPC

PIC PIC PIC PIC PIC PIC

TXP Matrix: T1600 Routing Nodes:

9 Copyright © 2009 Juniper Networks, Inc. www.juniper.net

SFC with 4 LCC

F13 SIB LCC 03

LCC 04 F13 SIB LCC 05

LCC 10 F13 SIB LCC 11

LCC 00 F13 SIB LCC 01 LCC 00 F13 SIB LCC 01

LCC 02 F13 SIB LCC 03 LCC 02 F13 SIB LCC 03

LCC 00 F13 SIB LCC 01

LCC 00 F13 SIB LCC 01

11 Copyright © 2009 Juniper Networks, Inc. www.juniper.net

12 Copyright © 2009 Juniper Networks, Inc. www.juniper.net

Switch output is connected to the

input of next stage 07 2/3

This is single plane 13 2/2

F3_00: Top SIBF13,

First stage sprays cells evenly

Second and third stage transmit

cells according to destination

Fabric cell comments

F1_00: Bottom SIBF13, F3_00: Bottom SIBF13,

F1_00: Bottom SIBF13, F3_00: Bottom SIBF13,

F13 SIB_in F2 SIB F13 SIB_out

F13 SIB F2 SIB F13 SIB

15 Copyright © 2009 Juniper Networks, Inc. www.juniper.net

High Speed Link

16 Copyright © 2009 Juniper Networks, Inc. www.juniper.net

17 Copyright © 2009 Juniper Networks, Inc. www.juniper.net

19 Copyright © 2009 Juniper Networks, Inc. www.juniper.net

20 Copyright © 2009 Juniper Networks, Inc. www.juniper.net

22 Copyright © 2009 Juniper Networks, Inc. www.juniper.net

23 Copyright © 2009 Juniper Networks, Inc. www.juniper.net

4 optic cables on each SIB , each cable carries 1 SG’s data to

Each SG connects to the 4 PFEs on 2 FPCs.

25 Copyright © 2009 Juniper Networks, Inc. www.juniper.net

The fiber-optic array cables connect the switching planes of the

26 Copyright © 2009 Juniper Networks, Inc. www.juniper.net

27 Copyright © 2009 Juniper Networks, Inc. www.juniper.net