Data Domain 401

Link aggregation, BOOST, and ifgroups

Ryan Halter
November 14, 2019
 Contents

This presentation provides a brief overview of link aggregation methodologies for

Data Domain, some best practices for their use, and links to further reading material
in the form of KB articles and white papers.

Internal Use - Confidential of2Y © Copyright 2019 Dell Inc.

 Link Aggregation

Internal Use - Confidential of3Y © Copyright 2019 Dell Inc.

 Link aggregation (LAG)

Combining physical network ports to aggregate throughput, enable failover, or both

Two components:
– Logical combination of 2+ physical network links
– A load balancing algorithm

Internal Use - Confidential of4Y © Copyright 2019 Dell Inc.

 Link aggregation

LAG methods on Data Domain:

• failover
– Active/passive failover of links/aggregates

• round robin
– Balances traffic on a group of ports bonded into a virtual interface via round robin

• balanced
– Requires trunking on network switch; load balances via a specified hash algorithm

FO/AL: create aggregates w/ 2+ ports for load balancing, then add those
aggregates to a failover bond

Internal Use - Confidential of5Y © Copyright 2019 Dell Inc.

 Link aggregation

LAG methods on Data Domain, continued:

• LACP
– Standard link aggregation methodology (IEEE 802.1AX / 802.3ad), requires configuration on network
switch. Communicates link status with switch, load balancing via specified hash algorithm

• ifgroups
– DD-native aggregation methodology for use with BOOST workloads and managed file replication

Internal Use - Confidential of6Y © Copyright 2019 Dell Inc.

 Load balancing algorithms
One of these algorithms/hashes must be chosen for Balanced or LACP LAGs:
• XOR-L2
– transmits through a bonded interface with an XOR hash of Layer 2 (inbound and outbound MAC
addresses).

• XOR-L2L3
– transmits through a bonded interface with an XOR hash of Layer 2 (inbound and outbound MAC addresses)
and Layer 3 (inbound and outbound IP addresses).

• XOR-L3L4
– transmits through a bonded interface with an XOR hash of Layer 3(inbound and outbound IP addresses)
and Layer 4 (inbound and outbound ports).

These only affect OUTBOUND traffic – load balancing on

inbound traffic is determined at the source
Internal Use - Confidential of7Y © Copyright 2019 Dell Inc.
 FO/AL
What is it?

FailOver on Aggregated Links

Pros:
– Simple configuration
– Provides load balancing and failover if LACP or
ifgroups can’t be used

Cons:
– Active/passive only – failover links only used if
primary links go down

Internal Use - Confidential of8Y © Copyright 2019 Dell Inc.

 LACP
What is it?

Link Aggregation Control Protocol

– Open standard for trunking network links to improve throughput and provide redundancy

Pros:
– Supported by all major network vendors and OSes
– Allows dynamic link aggregation, load balancing, and failover
– Improved failure handling vs. FO/AL
– All links in aggregate are active

Cons:
– Point-to-point only
– Can’t aggregate links across multiple switches w/o vendor-specific features (e.g. Cisco vPC)
– Doesn’t provide full bandwidth of additional links to the aggregate (protocol overhead)
– All links in aggregate must be of same type and speed
– Requires involving the network team
Internal Use - Confidential of9Y © Copyright 2019 Dell Inc.
 BOOST and ifgroups

Internal Use - Confidential of

10Y © Copyright 2019 Dell Inc.
 BOOST
What is it?

API and libraries owned and developed by Dell EMC

– Originally designed to work with NetBackup OpenStorage API (OST)
– Installed / integrated at client, application, backup server or as file system plugin
(BoostFS)

Allows apps to natively access capabilities of Data Domain

Licensed feature of DDOS

Internal Use - Confidential of

11Y © Copyright 2019 Dell Inc.
 BOOST
What does it enable?

Distributed Segment Processing (DSP)

– Deduplication process offloaded from the DD to the backup server or clients
– Decreases CPU impact on DD
– Up to 50% aggregate throughput improvement
– 20 - 40% lower CPU impact to backup server / app host
– 80 - 99% less LAN bandwidth required
– Process more backups with existing resources

Backup application managed replication via MFR

Virtual synthetics
Main area of focus in this document
as it relates to BOOST
Load balancing and link failover via ifgroups

Internal Use - Confidential of

12Y © Copyright 2019 Dell Inc.
 Reduced CPU utilization

Distributed dedupe = 40% reduction in CPU usage on the client?

• Moving data off of disk and out over the network is CPU intensive
• BOOST may increase CPU usage briefly, but reduces overall vs. non-BOOST*

BOOST: 25% avg. utilization for 1 hour

Non-BOOST: 10% avg. utilization for 8
hours

*This example is for illustration only and not indicative of an actual workload

Internal Use - Confidential of

13Y © Copyright 2019 Dell Inc.
 Reduced LAN bandwidth

Without DD Boost 1
2
ID Segment
Unique?
3 Compress
4 Write

Non-BOOST LAN
Application Server

1 ID Segment With DD Boost 2 Unique?

3 Compress
4 Write

DD Boost LAN
Application Server

Internal Use - Confidential of

14Y © Copyright 2019 Dell Inc.
 ifgroups
What are they?

Dynamic Interface Groups

– Patented Data Domain proprietary method for link aggregation

Pros:
– Enables transparent end-to-end failover and load balancing (round-robin) from client to DD
– No configuration required on network switches
– Limited protocol overhead vs. LACP (up to 60% more efficient)
– Doesn’t require downing the interface for configuration
– Allows routing traffic to specific interfaces by client (FQDN, domain, IP, subnet) or replica (MTree,
remote DD)
– Can contain links of different speeds (be careful with this)

Cons:
– Only useful for BOOST workloads and managed file replication (MFR)
– Customers may not be familiar with ifgroups – most of them will know what LACP is
Internal Use - Confidential of
15Y © Copyright 2019 Dell Inc.
 ifgroup client data flow

Internal Use - Confidential of

16Y © Copyright 2019 Dell Inc.
 Best practices

Internal Use - Confidential of

17Y © Copyright 2019 Dell Inc.
 Best practices and considerations
• IP tied to a DD’s hostname should be on a redundant interface
– Otherwise, configure -failover hostname/IP in DNS or in client hosts file

• If possible, avoid using 1GbE interfaces

– Failing that, set static routes for restores and replication to avoid sending traffic over 1GbE links
– Tie the DD’s hostname and IP to 10GbE interfaces

• LACP can’t aggregate links across switches w/o proprietary features like Cisco vPC
• Don’t mix ifgroups with LACP – use ifgroups alone for best performance
– LACP defeats BOOST’s native load balancing
– ifgroups don’t have the protocol overhead of LACP, so throughput is better

• Don’t mix BOOST and CIFS/NFS traffic on the same physical interface
– Keep filesystem traffic on interfaces that don’t also belong to ifgroups

Internal Use - Confidential of

18Y © Copyright 2019 Dell Inc.
 Best practices and considerations

• All interfaces in an ifgroup must be unique (Ethernet, virtual Ethernet) to ensure

failover in the event of network error
• Any client assigned to an ifgroup must be able to talk to ALL IPs in that ifgroup
• If configuring multiple ifgroups, assign clients to use a specific ifgroup – don’t just
use wildcards
– e.g. client group A uses ifgroup_A, client group B uses ifgroup_B

• When using ifgroups for MFR, configure ifgroups on both source and target DDs
• Configure static routes for restore and replication traffic – if possible, use dedicated
interfaces
– Avoids congestion on interfaces with incoming backups
– Avoids traffic going over 1GbE links

Internal Use - Confidential of

19Y © Copyright 2019 Dell Inc.
 BOOST to ifgroup vs LAG

Internal Use - Confidential of

20Y © Copyright 2019 Dell Inc.
 BOOST to ifgroup vs LAG

Internal Use - Confidential of

21Y © Copyright 2019 Dell Inc.
 Multihoming issues

Internal Use - Confidential of

22Y © Copyright 2019 Dell Inc.
 Example – what to do (mostly)

Internal Use - Confidential of

23Y © Copyright 2019 Dell Inc.
 Example – what to do (mostly)

Internal Use - Confidential of

24Y © Copyright 2019 Dell Inc.
 Example – what to do (mostly)

Internal Use - Confidential of

25Y © Copyright 2019 Dell Inc.
 Notes

• Avoided use of 1GbE links

• Failover bond associated to primary IP with hostname

• Two ifgroups each using multiple physical ports

• MFR traffic using ifgroups and mostly separated from incoming backup streams

• 23% of all outbound traffic is across veth0

• Consider configuring static routes for outbound traffic

Internal Use - Confidential of

26Y © Copyright 2019 Dell Inc.
 Example – what not to do

Internal Use - Confidential of

27Y © Copyright 2019 Dell Inc.
 Example – what not to do

Internal Use - Confidential of

28Y © Copyright 2019 Dell Inc.
 Example – what not to do

Internal Use - Confidential of

29Y © Copyright 2019 Dell Inc.
 Notes

• Failover bond associated to primary IP with hostname

• One ifgroup with only two physical ports

• MFR traffic not configured to use ifgroups

• 1 GbE links in use (actually 10GbE links connected to 1GbE switches)

• 1GbE and 10GbE networks on the same subnet

• 89% of all outbound traffic is across veth0

• Add more physical interfaces to the existing ifgroup

• Create new ifgroup for MFR

• Separate 1GbE and 10GbE networks

• Create static routes to get outbound traffic off of the 1GbE links

Internal Use - Confidential of

30Y © Copyright 2019 Dell Inc.
 Further reading

• Data Domain: Setting up Link Aggregation https://support.emc.com/kb/306105

• Data Domain: Link Failover of Aggregated Links https://support.emc.com/kb/305896

• Data Domain: Replication Network Configuration https://support.emc.com/kb/482579

• Data Domain: Interface-Groups Optimizes Performance For Multi-path Network https://support.emc.com/kb/471582

• Data Domain: How do DDboost Ifgroups work on Data Domain - high level view https://support.emc.com/kb/334871

• How destination IP based routing works in Data Domain https://support.emc.com/kb/530198

• Data Domain: How to add static route to go through a specific interface on DDOS 6.0? https://support.emc.com/kb/500223

• Data Domain: IP Alias Configuration https://support.emc.com/kb/306044

• http://emc.co/collateral/white-papers/h14541-data-domain-dynamic-interface-groups-wp.pdf

• https://www.dellemc.com/en-us/collaterals/unauth/technical-guides-support-information/products/networking-3/docu61798.p
df

Internal Use - Confidential of

32Y © Copyright 2019 Dell Inc.