BICSI and the ANSI/BICSI 002, Data

Center Design and Implementation

Best Practices
Gautier Humbert, RCDD, CDCDP.
BICSI Mainland Europe District Chair
East Central Europe Datacom BDM

Based on presentation from :

Rick Ciordia, BICSI CALA District Chair
 What is BICSI?

BICSI - Building Industry Consulting Service International, Inc.

BICSI is a non-profit association serving professionals in the
Information & Communications Technology (ICT) community

Core Purpose: Advancing the ICT Community

BICSI‘s Vision: BICSI will be the preeminent global resource for

developing standards, providing credentials,
accreditation and knowledge transfer for the ICT
community
 BICSI Provides ICT Community
Professionals with . . .

• Technical knowledge
• Internationally Adopted Industry Standards
• Technical Publications & Periodicals

• Professional Development
• Globally Recognized Credentials
• Training in ICT Design, Installation and Project Management

• Access to the Local & Global ICT communities

• Educational Conferences & Exhibitions
• Professional Networking
In the Following ICT Community Segments

• Commercial & Residential • Wireless Systems

Infrastructure
• Network Infrastructure
• Datacenters
• Telecommunications Project
• Outside Plant Cabling Management
Distribution
• Installation & Verification of
• Electronic Safety & Security Technology Systems

• Building Automation
Technical Publications
• Information Technology Systems Installation
Methods Manual (ITSIMM)
• Telecommunications Distribution Methods
Manual (TDMM)
• Outside Plant Design Reference Manual
(OSPDRM)
• Electronic Safety and Security Design
Reference Manual (ESSDRM)
• Network Systems and Commissioning (NSC)
Manual
• Telecommunications Project Management
(TPM) Manual

www.bicsi.org/publications
 BICSI International
Standards Program
• Develop standards within all facets of
Information & Communications Technology
(ICT) infrastructure design and installation

• Details:
– Over 450 member worldwide
– Accredited by ANSI
– Develops international open to use/“royalty free”
standards and best practices
www.bicsi.org/standards
 BICSI Standards Within ICT
ISO/IEC
IEEE CENELEC BICSI
& TIA
Defines the message Defines the transmission Defines how to design
and transmission media and system solutions using transmission
characteristics specifications media and systems
Reach of BICSI Standards
ANSI/BICSI 002-2014

Data Center Design and

Implementation Best Practices
 Data Center Standards

ISO/IEC 24764 (2010) ANSI/TIA-942-A (2012)

Information technology — Generic cabling Telecommunications Infrastructure Standard
systems for data centres for Data Centers
44 pages 118 pages

CENELEC EN 50173-5:2007 /
ANSI/BICSI 002-2014
A2:2012 Data Center Design and Implementation
Information technology — Generic cabling Best Practices
systems - Part 5: Data centres
534 pages
48 pages

CENELEC EN 50600 (2012-) ASHRAE TC9.9 (2012)

Information technology — Data centre Thermal Guidelines for Data Processing
facilities and infrastructures Environments, 3rd edition
Multiple Documents 150 pages
 DC Standards
Organization BICSI TIA ISO CENELEC ASHRAE
BICSI 002 TIA 942A ISO 24764 EN 50173-5 ASHRAE
Standard
(2014) (2012) (2010) / EN 50600 TC9.9

US /Canada EU US/Canada
Recognition Worldwide Worldwide
(Int’l) (Int’l) (Int’l)

Electrical Yes Yes

Mechanical Yes Yes

Areas Covered

Telecom Yes Yes Yes Draft

Environment Yes Yes Yes Yes

Site Yes
Security Yes Draft
Standards

BICSI
BICSI Design Classes and
Selection Methodology
 Risk Analysis

Seven considerations:

1. Life safety – If the system failed would lives be at risk

2. Threats – Natural, man-made or technology events
3. Economic loss from loss of data
4. Economic loss from damaged equipment
5. Regulatory or contractual impact
6. Damage to the organization’s reputation
7. Access to redundant off-site processing
 Reliability & Availability

Reliability
• How many times will the equipment work as
expected?

Availability
• How often is the equipment operational?
 Reliability & Availability
Reliability
The probability that equipment or system will perform its intended function without failure over a defined time period

Mean Time Between failure

UPS Manufacturer - 2 years MTBF
1/1/2015 12/31/2016

System Available System Available

Maintenance Maintenance
Week Week
27-Dec-15 27-Dec-16

Year end planned maintenance shutdown (5 days)

Availability
The probability that equipment or system is in condition to perform its intended function
51 weeks of Availability per year
 Availability

Determine the availability class for a data center

• Operational requirements
• Availability requirements
• Impact of down time
• Component and system reliability
• Impact of class on design
 Defining Availability Class
Operational Operational Impact of
Requirements Availability Downtime

Availability Class
 Operational Requirements
Identifying Operational Requirements
Operational Annual Planned
Description
Level Maintenance Hours
Functions are operational less than 24 hours a day and less than 7 days a week.
0 >400 Scheduled maintenance is available during working hours and off hours
Functions are operational less than 24 hours a day and less than 7 days a week.
1 100 - 400 Scheduled maintenance is available during working hours and off hours
Functions are operational 24 hours a day and up to 7 days a week for 50 weeks a year.
2 50 - 99 Scheduled maintenance is available during working hours and off hours
Functions are operational 24 hours a day and up to 7 days a week for 50 weeks or more.
3 0 - 49 No scheduled maintenance is available during working hours
Functions are operational 24 hours a day and up to 7 days a week for 52 weeks a year. No
4 0 scheduled maintenance is available

Key Factor – The amount of time planned for maintenance

 Operational Availability Rating
Allowable Maximum Annual Downtime in Minutes
Operational >5000 500 - 5000 50 - 500 5 - 50 0.5 - 5
Level (> 99%) (> 99% > 99.9%) (> 99.9% > 99.99%) (> 99.99% > 99.999%) (> 99.999% > 99.9999%)

Level 0 0 0 1 2 2

Level 1 0 1 2 2 2

Level 2 1 2 2 2 3

Level 3 2 2 2 3 4

Level 4 3 3 3 4 4

Operational Availability – When the IT services are expected to be available

 Impact of Downtime
Classifying Downtime

Classification Impact of downtime

Local in scope, single site, minor disruption of delay to non-critical

Isolated
objectives

Minor Local in scope, single site, minor disruption of delay to key objectives

Regional in scope, portions of the enterprise, moderate disruption or

Major
delay of key objectives
Multiregional in scope, major portions of the enterprise, significant
Severe
disruption or delay of key objectives
Quality of service delivery across the enterprise, significant disruption or
Catastrophic
delay of key objectives
 Determining Data Center Class

Operational Availability Rank

Impact of downtime
0 1 2 3 4
Isolated Class 0 Class 0 Class 1 Class 3 Class 3
Minor Class 0 Class 1 Class 2 Class 3 Class 3
Major Class 1 Class 2 Class 2 Class 3 Class 3
Sever Class 1 Class 2 Class 3 Class 3 Class 4
Catastrophic Class 1 Class 2 Class 3 Class 4 Class 4
 Availability Concerns

• Component Redundancy
– Redundancy of critical high-risk components
• System Redundancy
– Redundancy at the system level
• Quality
– Commercial or premium grade
• Survivability
– Protection against external events
 Reliability

“Reliability is the probability that a component

or system will perform it’s intended function
within stated conditions, for a specified period
of time without failure”
ANSI/BICSI 002-2014 B.8.1

Reliability is calculated from published MTBF

data for components and systems.
(mean time between failures)
 Reliability
A1
R = .90
RA1A2 = RA1 x RA2
.90 x .90 = .81 A3
R = .95

A2
R = .90

RA = 1 – [(1 - RA1A2) x [(1 – RA3)] RTOTAL = RA X RB

1 – [1 – [(1 - .81) x [(1 - .95)] = .9905 .9905 x .995 - .9855475 (98.5%)

RB = 1 – [(1 – RB1) x [(1 – RB2)] B1 B2

1 – [(1 - ..95) x [(1 - .90)] = .995 R = .95 R = .90
 What is N

• N or Need is the resource required to serve

the IT equipment

• N+1 when components (N) have at least one

independent backup component (+1)
Utilization Efficiency verses Reliability

N = 100 kVA of UPS

N+1 redundancy can be achieved as:

1. 2 x 100 kVA modules = 200 kVA (50% efficient)

2. 3 x 50 kVA modules = 150 kVA (66% efficient)
3. 4 x 33 kVA modules = 132 kVA (75% efficient)
4. 5 x 25 kVA modules = 125 kVA (80% efficient)
 BICSI DC Design Classifications

• Class 0: Single path, and fails to meet one or more

criteria of Class 1
• Class 1: Single path
• Class 2: Single path with redundant components
• Class 3: Concurrently maintainable & operable
• Class 4: Fault tolerant
 Availability Class Prefixes

• Class Fx: Facility (Electrical & Mechanical)

• Class Cx: Cable Plant

• Class Nx: Network Infrastructure
• Class Sx: Data Processing and Storage Systems

• Class Ax: Applications

Class F1 Electrical Example
Alternate
Transfer
Utility Power
Switchgear
Source

Electrical Distribution

UPS Mechanical
Static Maintenance Switchgear
Bypass Bypass

PDU

Critical Non-Critical Mechanical

Loads Loads Loads
Class F2 Electrical Example
Alternate
Transfer Power Source N
Utility
Switchgear Alternate
Power Source +1

Electrical Distribution

Static
UPS UPS
Bypass
Mechanical
N +1 Switchgear

Optional
component
Maintenance Bypass
redundancy
PDU

Critical Non-Critical Mechanical

Loads Loads Loads
 Electrical Class F3
Alternate
Power Source
N
Transfer
Utility
Alternate Switchgear
Power Source
+1

Electrical Distribution Electrical Distribution

Mechanical Mechanical
UPS UPS Switchgear Switchgear

Static N +1 Mechanical
Bypass Loads

Maintenance
Bypass Mechanical
Loads

Output Distribution Switchboard Alternate Switchboard

Critical Mechanical Critical Mechanical

Switchgear Switchgear

PDU Critical PDU

Fans/Pumps

Non-Critical Critical Non-Critical

Loads Loads Loads
 Electrical Class F4
Alternate Alternate
N Power Source
N
Power Source Transfer Transfer
Utility Switchgear
Alternate Switchgear Alternate
+1 Power Source
+1
Power Source

Electrical Distribution Electrical Distribution

UPS UPS Mechanical Mechanical UPS UPS

Switchgear Switchgear
Static N +1 N +1 Static
Bypass Bypass
Maintenance Mechanical Maintenance
Bypass Loads Bypass

Output Distribution Switchboard Output Distribution Switchboard

Critical Mechanical Critical Mechanical

Switchgear Switchgear

Critical
PDU Fans/Pumps PDU

Non-Critical Non-Critical
Loads Loads
Critical
Loads
Telecommunication Classes
Service Service Service Service
Provider Provider Provider Provider

Class C4
C2
C1
C3
Maintenance Maintenance
Hole Hole

Entrance Room Entrance Room

Offices,
Operations Center,
Computer
Support Rooms
Room

Telecomunicaciones Main Dist Area Main Dist Area

Room

Horiz Dist Horiz Dist Horiz Dist Horiz Dist

Area Area Area Area
(Opt.) (Opt.)

Zone Dist Zone Dist Zone Dist

Area Area Area

Equip Equip Equip Equip Equip Equip Equip Equip

Dist Area Dist Area Dist Area Dist Area Dist Area Dist Area Dist Area Dist Area

ITE ITE ITE ITE ITE ITE ITE ITE

Only required for large computer rooms due to port

counts or distance limitations
 Mechanical Class F0 & F1
Pump
DRY-COOLER Indoor Heated Water
Chiller System Indoor Cooled Water

Outdoor Heated Water

Outdoor Cooled Water

CHILLER DX System

CONDENSER CONDENSER

N CRAH

N CRAH N CRAC N CRAC

Single Path
 Mechanical Class F2
AIR-COOLED AIR-COOLED Pump
CONDENSER CONDENSER Indoor Heated Water

Indoor Cooled Water

“N + 1” Chillers, CHILLER CHILLER

Outdoor Heated Water

Pumps and Outdoor Cooled Water

Condensers
• Redundant critical
components
• All power feeds from
common upstream
distribution
N CRAH
• Only redundant
components able to
N CRAH be maintained under
load

+1 CRAH

Single path with redundant components

 Mechanical Class F3
AIR-COOLED AIR-COOLED
CONDENSER CONDENSER Pump

Indoor Heated Water

Indoor Cooled Water

“N + 1” Chillers, CHILLER CHILLER Outdoor Heated Water
Pumps and Outdoor Cooled Water

Condensers
Pipe loops • Redundancy in all areas
recommended • Equipment and controls
with redundant systems
fed from different
electrical distribution
N CRAH
• Maintainable when
actions do not reduce
N CRAH cooling capacity below
"N+1"
+1 CRAH

Concurrently maintainable and operable

 Mechanical Class F4
AIR-COOLED AIR-COOLED AIR-COOLED
CONDENSER CONDENSER CONDENSER Pump

Indoor Heated Water

“N + 1” Chillers, Indoor Cooled Water

CHILLER CHILLER CHILLER
Pumps and Outdoor Heated Water

Condensers Outdoor Cooled Water

Pipe loops
Required
• Redundant equipment and
piping for maintenance
• Power feed so that cooling
capacity does not drop below
N CRAH "N" when maintaining
mechanical or upstream
electrical distribution
N CRAH
• Maintainable when actions do
not decrease cooling capacity
+1 CRAH below "N"

+2 CRAH

Fault tolerant
Class N0/N1 and N4 Network
Class S4 System and A4 Application
 Availability and
Multi-Site Data Centers
• Prior to virtualization, subclasses aligned through
data center

• Today, a single data center may not have

alignment

• Availability class methodology can in discussions

about using multiple data centers to achieve
availability target
Example: Class 3 Availability Using
Three Class 2 Data Centers
 Why BICSI 002?

If you fail to plan,

you are planning to fail!
- Benjamin Franklin
 Thank You!

Gautier Humbert, RCDD, CDCDP

BICSI District Chair Mainland Europe

Email: [email protected]

Legrand Datacom Business Development Manager – Eastern & Central Europe

Email: [email protected]