WHITE PAPER

The Benefits of Brocade Gen 5

Fibre Channel

The network matters for storage. This paper discusses key

server and storage trends and technology advancements and
explains how Brocade Gen 5 Fibre Channel and Fabric Vision
technology delivers the performance, reliability, and simplicity
needed to meet the new requirements of today’s data center.
Table of Contents

Introduction................................................................................................................................... 4

Overview of Gen 5 Fibre Channel........................................................................................ 4

The Need for Gen 5 Fibre Channel...................................................................................... 5

Application Growth.....................................................................................................................5

High-Density Server Virtualization .................................................................................5

Solid State Drives .......................................................................................................................6

Gen 5 Fibre Channel Storage Arrays..............................................................................6

The Benefits of Higher Speed............................................................................................... 6

Reduced Number of Links.....................................................................................................6

Reduced Power Consumption per Bit............................................................................ 7

Easier Cable Management..................................................................................................... 7

Simplified Infrastructure Upgrades...................................................................................8

Brocade Gen 5 Fibre Channel Unique Innovation......................................................... 8

UltraScale Chassis Connectivity.........................................................................................8

Fabric Vision Technology.....................................................................................................10

Summary.......................................................................................................................................12

3
Introduction
Gen 5 Fibre Channel is the latest evolution in Storage Area Networks (SANs). Based on technology developed by
the T11 technical committee that defines Fibre Channel interfaces, Gen 5 Fibre Channel doubles the data throughput
of 8 Gbps links, from 800 Megabytes per second (MB/sec) to 1600 MB/sec with Gen 5 Fibre Channel.

The benefits of faster networking technology are easy to understand. Data transfers are faster, fewer links are
needed to accomplish the same task, fewer devices need to be managed, and less power is consumed when
Gen 5 Fibre Channel is used. Several server and storage technology advances are pushing up demand for
greater SAN bandwidth, including application and storage capacity growth, high-density server virtualization,
new Gen 5 Fibre Channel storage arrays, and Solid State Drive (SSD) or Flash-based storage. Gen 5 Fibre
Channel enables enterprise data centers to unleash the full potential of these and other advanced technologies.

The advantages of Gen 5 Fibre Channel, however, extend beyond performance and higher throughput. Gen
5 Fibre Channel delivers several important capabilities that take SAN reliability, availability, and management
simplicity to the next level, driving down operational costs. It is these advantages that are driving the
unprecedented adoption rate of Gen 5 Fibre Channel.

Overview of Gen 5 Fibre Channel

Deployed in 90 percent of Fortune 1000 data centers, Fibre Channel (FC) is the de facto
standard for storage networking in the data center. Gen 5 Fibre Channel is the proven,
purpose-built network infrastructure for data center storage, delivering unmatched
reliability, scalability, and 16 Gbps performance.

Brocade offers a comprehensive Gen 5 Fibre Channel SAN portfolio with unique
innovations delivered through the Gen 5 Fibre Channel Application-Specific Integrated
Circuit (ASIC) in combination with Brocade Fabric OS® (FOS) and Brocade Network
Advisor. Brocade Gen 5 Fibre Channel offers:

Unmatched performance for critical and virtualized applications

Hardware and software reliability, consistently delivering six-nines availability

••UltraScale chassis connectivity, providing 64 Gbps parallel Gen 5 Fibre Channel to

drive massive infrastructure consolidation and simplification

••Brocade Fabric Vision technology, an advanced hardware and software solution

that maximizes uptime, optimizes application performance, and simplifies SAN
management through innovative diagnostic, monitoring, and management technology

••A comprehensive Gen 5 Fibre Channel portfolio—from entry level switches to

massively scalable backbones

Brocade switches with Gen 5 Fibre Channel unleash the full potential of high-density
server virtualization, cloud architectures, and Flash storage. This enables organizations to
fully leverage existing and new IT investments as they solve their most difficult business
challenges and address the evolving requirements of today’s data center.

4
The Need for Gen 5 Fibre Channel
Emerging and evolving critical workloads, higher-density virtualization, and cloud-based
architectures are continuing to push the limits of SAN infrastructure. In addition, new
technologies such as Flash-based storage and new Gen 5 Fibre Channel storage arrays
are shifting the focus from storage to interconnect. This trend is driving ever higher Input/
Output (I/O) and bandwidth requirements, driving the need for higher speeds, as well as
more reliable networks.

Multiple server and storage trends and technology advances are driving the need for Gen
5 Fibre Channel, including:

••Increased number and size of applications

••High-density server virtualization
••Solid State Drives (SSDs) and Flash-based storage
••New 16 Gbps disk arrays
Application Growth
The big growth in all computing environments is the increasing size and number of
software applications. The digitization of information and growing use of rich media
and interactive Web 2.0 applications drive greater storage capacity and bandwidth
requirements. From mobile access to email to cloud computing, data needs to be
transported over longer distances at high speeds. In addition, applications such as
databases and other mission-critical applications are growing rapidly, yet continue to
require non-stop availability. With shrinking or non-existent maintenance windows,
performance issues or downtime that impact data access are simply not acceptable. The
storage network must be prepared to handle increased capacity, greater throughput, and
higher levels of resiliency.

High-Density Server Virtualization

When Fibre Channel was first launched, a server was dedicated to a specific application,
resulting in low utilization of server resources. The introduction of server virtualization
changed that paradigm, allowing multiple applications to share a single physical server,
increasing efficiency and driving up server utilization rates. According to ESG’s annual
IT spending intentions survey, increased use of server virtualization has been the top
overall priority of IT organizations for more than two years running. Today, evolving critical
workloads and Tier 1 applications are being hosted on Virtual Machines (VMs). In addition
to increased use of server virtualization, VM densities (the number of VMs hosted on
each physical server) are steadily increasing to 10, 20, or more VMs per physical server—
all booting from the SAN and accessing SAN resources. The increased usage, criticality,
and density of VMs drive demand for higher performance (bandwidth and I/O), as well
as increased reliability and availability from the storage infrastructure. In highly virtualized
environments, any congestion, poor I/O performance, or failures in the storage network
impact a larger number of applications.

5
 Solid State Drives
Flash-based storage and SSDs are driving incredible advances in storage, dramatically
reducing the historical I/O performance gap between the server side (with multi-core
processors and faster memory) and the storage side. SSD storage addresses both I/O
and throughput bottlenecks, enabling faster block- and file-based storage performance
for high-density virtualized workloads and traditional mission-critical applications.
However, this assumes that the storage network can keep pace. Whether they are
deployed as standalone SSD-based arrays or directly connected to the server CPU
and memory bus, SSDs accelerate I/O performance, driving the need for higher I/O
bandwidth performance and greater availability from the storage network.

Gen 5 Fibre Channel Storage Arrays

New 16 Gbps-capable Fibre Channel storage arrays from leading storage providers
are becoming available. To take full advantage of the capabilities offered by this next
generation of storage, users must make sure that the network is not a bottleneck. Data
center managers looking to deploy new Gen 5 Fibre Channel arrays to gain a competitive
advantage need to take a close look at their storage network infrastructure to determine
whether it is capable of delivering the reliability, performance, and operational simplicity
the new technology requires. If not, the bottleneck has simply moved, and you will not
gain the benefits you seek.

These and other data center evolutions are driving the need for Gen 5 Fibre Channel
infrastructures.

The Benefits of Higher Speed

The benefits of faster tools are always the same—more work in less time. By doubling the
speed, Gen 5 Fibre Channel reduces the time to transfer data between two ports. When
more work can be done by a server or storage device, fewer servers, Host Bus Adapters
(HBAs), links, and switches are needed to accomplish the same task. The benefits of
higher speed with Gen 5 Fibre Channel add up, and they include:

••Reduced number of links, HBAs, and switch ports to do the same workload
••Reduced power consumption per bit
••Easier cable management
Reduced Number of Links
As with previous generations of Fibre Channel, the first application of new Fibre Channel
speeds is on Inter-Switch Links (ISLs) between switches. Large fabrics are composed of
many switches, which are connected via multiple ISLs. Reduction of the number of ISLs
between switches is a key benefit of each higher speed. Brocade switches will continue
to support frame-based trunking of 16 Gbps FC links, to yield 128 Gbps FC bandwidth
between any two switches.
The table in Figure 1 shows a simple comparison of the number of links in an 8 Gbps
FC fabric and a 16 Gbps FC fabric. The higher-speed links of 16 Gbps FC eliminate tens
or hundreds of ports as compared to an 8 Gbps FC fabric. The real savings occur when
the number of HBAs, switches, and end devices can be decreased, given the higher
performance of 16 Gbps FC. In the example in Figure 1, a Top of Rack (ToR) switch needs
100 Gbps of bandwidth, so the user needs 8 × 16 Gbps FC ISLs instead of 16 × 8 Gbps
FC ISLs. Similar comparisons between 16 Gbps FC ISLs and 8 Gbps FC ISLs are listed
in the table in Figure 1, to show how fewer ports and links are needed at 16 Gbps FC.

6
 Core-to-core
150 Gbps

Figure 1.
Network design implications

fig06_Gen_5_TB
of Gen 5 Fibre Channel.

ToR switches Blade switches

100 Gbps to core 70 Gbps to core

8 Gbps Gen 5 Fibre Channel

FC Links (16 Gbps) Links

ISLs from ToR switch to core 16 8

ISL from blade switch to core 10 5

Core to core 24 12

Total ISLs 50 25

Total ports 100 50

Reduced Power Consumption per Bit

Besides the reduction in equipment that cuts power consumption dramatically, Gen 5
Fibre Channel also reduces the power required to transfer bits on the link. When the cost
of cabling and Operating Expense (OpEx) such as electricity and cooling are considered,
the Total Cost of Ownership (TCO) is often less when links are run at twice the speed.
The goal of Gen 5 Fibre Channel design is for a 16 Gbps FC port to consume less power
than 2 × 8 Gbps FC links, which deliver the same throughput. Initial estimates for power
consumption indicate that Gen 5 Fibre Channel SFP+ consumes 0.75 watts of power,
while 8 Gbps FC SFP+ consumes 0.5 watts of power. These estimates show that a Gen
5 Fibre Channel link consumes 25 percent less power than 2 × 8 Gbps FC ports.

Easier Cable Management

If fewer links are needed, cable management becomes simpler. Managing cables behind
a desktop or home entertainment center is difficult enough, but managing hundreds of
cables from a single switch or bundles of cable from a server is a daunting task. The
reduction of cables means less troubleshooting and recabling. The cost of cabling is
significant; reducing the number of links by using Gen 5 Fibre Channel links reduces the
work and cost involved in cable management.

7
 Simplified Infrastructure Upgrades
Deploying a Gen 5 Fibre Channel SAN now also makes server and storage upgrades
easier down the road. With a Gen 5 Fibre Channel infrastructure in place, you can easily
and non-disruptively swap out servers and upgrade storage arrays to take advantage of
technology advancements as needed, without impacting the rest of the infrastructure.
This approach is far easier than having to swap out the fabric as well as the server and
storage when an upgrade is needed.

Brocade Gen 5 Fibre Channel Unique Innovation

The benefits associated with higher throughput are compelling. Brocade Gen 5 Fibre
Channel offers additional advantages to meet new and evolving requirements, including:

••Low latency and high I/O Operations per Second (IOPS) performance maximizes the
number of virtual hosts per physical server.

••Data center-proven, purpose-built architectures minimize risk and fault domains of

high-density server virtualization.

••Non-stop networking and automated management minimizes operational cost

and complexity.

••Integrated advanced diagnostics, monitoring, and Reliability, Availability, and

Serviceability (RAS) capabilities simplify management and increase resiliency.

••Integrated ISL data compression and encryption offers bandwidth optimization and
data protection.

••Backward-compatibility with existing infrastructure minimizes the need to rip out and replace.
••Low overhead and low latency eliminates I/O bottlenecks and unleashes the full
performance of Flash, SSD, and 16 Gbps-capable storage.

In addition, Brocade Gen 5 Fibre Channel platforms offer breakthrough technologies

that dramatically simplify SAN deployment and management and drive down operational
costs. These unique capabilities are explored in more detail in the following sections:

••UltraScale chassis connectivity enables higher density and simpler fabrics that reduce
network complexity and cost.

••Fabric Vision technology maximizes uptime, optimizes application performance,

and simplifies SAN management through innovative diagnostic, monitoring, and
management technology.

UltraScale Chassis Connectivity

Brocade UltraScale chassis connectivity provides dedicated 64 Gbps parallel Gen 5 Fibre
Channel inter-chassis links (ICLs) to connect up to 10 Brocade DCX® 8510 Backbones,
enabling flatter, faster, and simpler fabrics that increase consolidation while reducing
network complexity and costs. UltraScale ICLs preserve server and storage ports and
enable highly scalable core-edge and active-active mesh chassis topologies, as shown in
Figures 2 and 3.

8
 Fabric
Edge

fig01_DCX-8510-ICL
Fabric
Core

Figure 2. Core-edge topology with two core chassis and eight edge chassis.

fig02_DCX-8510-ICL

Figure 3. Nine-chassis mesh topology.

Brocade UltraScale—based on Quad Small Form Factor Pluggable (QSFP) technology—

connects the core routing blades of two Brocade DCX 8510 chassis. Each QSFP-based
UltraScale port combines four 16 Gbps links, providing up to 64 Gbps of throughput
within a single cable. Available with Brocade Fabric OS (FOS) v7.0 and later, Brocade
offers up to 32 QSFP UltraScale ports on the Brocade DCX 8510-8 and up to 16 QSFP
UltraScale ports on the Brocade DCX 8510-4.

The optical form factor of the Brocade QSFP-based UltraScale technology offers support
for cable distances up to 100 meters (108 yards), providing greater architectural design
flexibility. Second, the combination of four cables into a single QSFP provides incredible
flexibility for deploying a variety of different topologies, including a massive 9-chassis
full-mesh design with only a single hop between any two points within the fabric.

In addition, the Brocade DCX 8510 UltraScale chassis connectivity provides dramatic
reduction in the number of ISL cables required—a four to one reduction compared to
traditional ISLs with the same amount of interconnect bandwidth. And since the QSFP-
based UltraScale connections reside on the core routing blades instead of consuming
traditional ports on the port blades, up to 33 percent more FC ports are available for
server and storage connectivity.

9
 Fabric Vision Technology
Brocade Fabric Vision technology introduces a breakthrough hardware and software
solution that combines capabilities of the Gen 5 Fibre Channel ASIC, Brocade Fabric
OS, and Brocade Network Advisor to maximize uptime, simplify SAN management,
and provide unprecedented visibility and insight across the storage network. Offering
innovative diagnostic, proactive monitoring, and management capabilities, Brocade
Fabric Vision technology helps administrators avoid problems, maximize application
performance, and reduce operational costs.

Brocade Fabric Vision technology extends Brocade Gen 5 Fibre Channel, offering
technology innovation that is unmatched in the industry. Fabric Vision technology includes:

ClearLink diagnostics: As network speeds increase, media and cable tolerance become
critical for avoiding degraded performance and avoiding Cyclic Redundancy Check (CRC)
errors over time. Brocade ClearLink diagnostics, a patent-pending technology, leverages
a unique operational Diagnostic Port (D_Port) mode to ensure optical and signal integrity
for Gen 5 Fibre Channel optics and cables, simplifying the deployment and support of
high-performance fabrics. By proactively verifying the integrity of critical transceivers,
customers can quickly address any physical layer issues without the need for special
optical testers.

ClearLink is diagnostics technology that allows users to automate a battery of tests to

measure and validate latency and distance across the switch links, as well as to verify the
integrity of the fiber and 16 Gbps transceivers in the fabric—either prior to deployment
or when there are susceptible physical layer issues. In addition, ClearLink diagnostics
requires only the individual ports that are attached to the link being tested to go offline,
leaving the rest of the ports to operate online, in isolation from the link being tested.

In addition to switch-to-switch link validation, Brocade FOS v7.1 provides dynamic

ClearLink support between Gen 5 Fibre Channel switches and Brocade 1860 Fabric
Adapters when running at 16 Gbps speed. Brocade FOS v7.1 also provides ClearLink
support for 16 Gbps switches running in Access Gateway mode, as well as for UltraScale
chassis connectivity links on Brocade DCX 8510 Backbones.

Monitoring and Alerting Policy Suite (MAPS): A new policy-based monitoring and
alerting tool available with Brocade FOS v7.2, MAPS simplifies and automates fabric-wide
threshold configuration and monitoring. MAPS takes the guesswork and complexity out
of defining appropriate threshold-based policies and alerts. It offers predefined monitoring
groups as well as predefined policies, rules, and actions—all based on more than 15 years
of Brocade experience and expertise—to make fabric-wide threshold-based monitoring
and alerting simple. You can configure the entire fabric (or multiple fabrics) at one time,
using common rules and policies, or you can customize policies for specific ports—all
through a single dialog. The integrated dashboard rolls up the overall switch health report,
along with details on out-of-policy conditions, to help pinpoint potential issues. In addition,
customers can use MAPS and Brocade Network Advisor v12.1 to deploy a consistent
configuration policy across the network, further simplifying network operations.

Brocade Fabric Watch customers can easily take advantage of new MAPS capabilities
and usability enhancements. Fabric Watch users can automatically import existing
thresholds into a MAPS policy, enabling a seamless migration to MAPS.

Brocade Flow Vision: A comprehensive tool that allows administrators to identify, monitor,
and analyze specific application and data flows in order to maximize performance, avoid
congestion, and optimize resources. Available with Brocade FOS v7.2, Flow Vision
includes these features:
10
••Flow Monitoring application: Provides comprehensive visibility into flows in the fabric,
including the ability to automatically learn (discover) flows and non-disruptively monitor
flow performance. Users can monitor all flows from a specific host to multiple targets
and Logical Unit Numbers (LUNs) or from multiple hosts to a specific target/LUN.
Users can also monitor all flows across a specific ISL or perform LUN-level monitoring
of specific frame types to identify resource contention or congestion that is impacting
application performance.

••Flow Generator application: Provides a built-in test traffic generator for pretesting and
validating the SAN infrastructure—including internal connections within a switch—for
robustness, before deploying applications.

••Flow Mirroring application: Provides the ability to non-disruptively create copies of

specific application and data flows or frame types that can be captured for deeper
analysis. Flow Mirror is used for in-depth analysis of flows of interest or specific frame
types, such as analysis of Small Computer Systems Interface (SCSI) Reservation
frames, Abort Sequence (ABTS) frames, flows going to a bottlenecked device, or
frames during link bring-up.

Bottleneck Detection: Identifies and alerts administrators of device or ISL congestion

as well as abnormal levels of latency in the fabric. When applied to F_Ports, Bottleneck
Detection can continuously monitor for medium or high levels of latency on a device
port and provides notification on the nature and duration of the latency. Bottleneck
Detection can also serve as a confirmation to host information when storage latencies are
suspected as the cause of poor host performance. The reverse (eliminating the storage
as the source of poor performance) is also true. When applied to E_Ports, Bottleneck
Detection can alert when it detects high levels of latency on an ISL, often the result of
congestion or latency from elsewhere in the fabric, but also a condition that can occur due
to the cumulative effect of device latencies from multiple flows.

Brocade Network Advisor works in conjunction with Bottleneck Detection to automatically

monitor and detect network congestion and latency in the fabric, providing visualization
of bottlenecks in a connectivity map and product tree. Brocade Network Advisor can also
show exactly which devices and hosts are impacted by a bottlenecked port.

Forward Error Correction: Automatically detects and recovers from bit errors, enhancing
transmission reliability and performance.

Virtual Channel-level BB_Credit Recovery: Automatically detects and recovers buffer

credit loss at the Virtual Channel level, providing protection against performance
degradation and enhancing application availability.

Brocade Fabric Vision technology is integrated into Brocade Network Advisor, providing
customizable health and performance dashboard views to simplify SAN configuration
and management, enable proactive management by pinpointing problems faster, and
reduce operational costs. With Fabric Vision technology, Brocade Network Advisor
provides all critical status in one screen, including a summary of switch health, out-of-
policy conditions, top application flows, and historical data, so you can quickly see trends.

11
Summary
Higher throughput is important for keeping up with the many server and storage
technology advancements, but the true value of Brocade Gen 5 Fibre Channel goes well
beyond faster links. Higher reliability and innovative, breakthrough technologies such as
UltraScale chassis connectivity and Fabric Vision technology dramatically simplify SAN
deployment and management and drive down operational costs.

As the recognized leader and innovator in storage networks, Brocade Fibre Channel is
the most trusted, most deployed network infrastructure for enterprise storage. Brocade
continues to drive future Fibre Channel open standards and technology advancements
that deliver unparalleled business value.

Corporate Headquarters European Headquarters Asia Pacific Headquarters

San Jose, CA USA Geneva, Switzerland Singapore
T: +1-408-333-8000 T: +41-22-799-56-40 T: +65-6538-4700
[email protected] [email protected] [email protected]

© 2015 Brocade Communications Systems, Inc. All Rights Reserved. 05/15 GA-WP-313-04
ADX, Brocade, Brocade Assurance, the B-wing symbol, DCX, Fabric OS, HyperEdge, ICX, MLX, MyBrocade, OpenScript,
The Effortless Network, VCS, VDX, Vplane, and Vyatta are registered trademarks, and Fabric Vision and vADX are trademarks
of Brocade Communications Systems, Inc., in the United States and/or in other countries. Other brands, products, or service
names mentioned may be trademarks of others.

Notice: This document is for informational purposes only and does not set forth any warranty, expressed or implied, concerning any
equipment, equipment features, or service offered or to be offered by Brocade. Brocade reserves the right to make changes to this
document at any time, without notice, and assumes no responsibility for its use. This information document describes features that
may not be currently available. Contact a Brocade sales office for information on feature and product availability. Export of technical
data contained in this document may require an export license from the United States government.