Module 12: NSX-T Data

Center and Containers

© 2020 VMware, Inc.

Importance
You must understand the network and security requirements to support containers and Kubernetes. NSX-T Data Center
solves these design challenges by creating a DevOps model for your data center environment.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 2

Module Lessons
1. Overview of NSX-T Data Center and vSphere with Tanzu
2. NSX-T Data Center Design: Implications for Scalability
3. Overview of the Tanzu Kubernetes Grid Cluster

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 3

Lesson 1: Overview of NSX-T Data Center and
vSphere with Tanzu

© 2019 VMware Inc. All rights reserved.

Learner Objectives
• Describe the solution overview to support VMware Tanzu in NSX-T Data Center 3.0
• Describe the platform architecture used to support VMware Tanzu
• Describe the Supervisor cluster design and utilization

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 5

About vSphere with Tanzu
vSphere with Tanzu transforms vSphere into a native Kubernetes platform.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 6

vSphere with Tanzu
NSX-T Data Center is the building block for enabling vSphere with Tanzu.

The Networking stack for vSphere with Tanzu is powered

by NSX-T Data Center.
The following options are available to deploy and
consume NSX-T Data Center:
• NSX-T Data Center through VMware Cloud
Foundation
• Standalone NSX-T Data Center

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 7

vSphere with Tanzu and VMware Cloud Foundation 4
vSphere with Tanzu is available through VMware Cloud Foundation 4.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 8

Supervisor Clusters
In vSphere with Tanzu, a Supervisor cluster is a special Kubernetes cluster that uses ESXi as the worker node instead
of Linux.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 9

Supervisor Cluster Control Plane
About the Control Plane VM:
• The control plane VMs are
equivalent to Kubernetes Control
nodes.
• NSX Container Plug-in (NCP)
interacts with NSX.
• The Cloud Native Storage driver is
the API for managing storage.
• Scheduler extension interacts with
the K8 Scheduler and vSphere
DRS.
• Authenticating Proxy makes the
Login API available and redirects
the kubectl vsphere login
command to vCenter Single Sign-
On authentication.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 10

Supervisor Cluster Namespaces
A namespace is a Kubernetes construct that is used to
divide cluster resources:
• Each namespace has its own resource pool.
• Resources are controlled by using resource quotas for
CPU, memory, and storage.
• All workloads in a namespace are bound by a
namespace quota:
– vSphere pods
– Virtual machines

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 11

vSphere Pod Service
The vSphere pod service offers advanced security and performance, without managing clusters.

DevOps:
• Kubernetes API
• Enhanced security and resource isolation
• Performance advantage
• Serverless experience
IT Operator:
• Application-focused management
• Workload visibility in pods and containers

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 12

Network Service: Self-Service Provisioning of Network Resources
DevOps:
• Kubernetes API
• Provision network resources and define ingress paths
IT Operator:
• Define administrator policies for security

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 13

Storage Service: Self-Service Provisioning of Storage Resources
DevOps:
• Kubernetes API
• Provision storage and persistent volume claims
IT Operator:
• Establish resource quotas
• Visibility

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 14

Registry Service
You can manage container images in an embedded container registry.

Embedded image registry:

• Sync project life cycle
• Sync user permissions

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 15

Review of Learner Objectives
• Describe the solution overview to support VMware Tanzu in NSX-T Data Center 3.0
• Describe the platform architecture used to support VMware Tanzu
• Describe the Supervisor cluster design and utilization

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 16

Lesson 2: NSX-T Data Center Design:
Implications for Scalability

© 2019 VMware Inc. All rights reserved.

Learner Objectives
• Describe the networking topology and security features of the Supervisor cluster
• Identify the NSX-T Data Center networking capabilities of the Supervisor cluster
• Describe the Supervisor cluster load-balancing design
• Identify the NSX-T Data Center security capabilities of the Supervisor cluster

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 18

vSphere with Tanzu Network Topology
vSphere with Tanzu uses NSX
network capabilities:
• A Supervisor cluster is isolated
with Tier-1 routers and distributed
firewalls.
• Namespaces are isolated with
distributed port groups and
distributed firewalls.
• All traffic is allowed for all
namespaces by default.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 19

Preinstalled NSX-T Data Center: Networking in the Supervisor Cluster (1)
Design option 1 shows the shared T0 and edge nodes.

NSX-T Data Center Supervisor clusters:

• NSX Manager or NSX Edge nodes and Tier-0 are
preinstalled.
• The ESXi hosts are prepared for NSX-T Data Center.
• Multiple vCenter Server systems can share the single
NSX-T Data Center deployment.
• Multiple Supervisor clusters share edge nodes and Tier-
0.
• Each Supervisor cluster has a dedicated Tier-1.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 20

Preinstalled NSX-T Data Center: Networking in the Supervisor Cluster (2)
Design option 2 shows the shared T0 with dedicated edge nodes.

NSX-T Data Center Supervisor clusters:

• NSX Manager or NSX Edge nodes, and Tier-0 are
preinstalled.
• The ESXi hosts are prepared for NSX-T Data Center.
• Multiple vCenter Server systems can share the single
NSX-T Data Center deployment.
• Multiple Supervisor clusters with dedicated edge nodes
and shared Tier-0.
• Each Supervisor cluster has a dedicated Tier-1.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 21

Supervisor Cluster Networking and Security
The topology and feature map provides the following details:
• Supervisor clusters are isolated with the edge firewall on Tier-1 gateways
and distributed firewall per vNIC.
• Namespaces are isolated with dvPG (NSX Logical Segment) and
distributed firewall.
• IPAM functionality is provided for all Supervisor workloads attached to
NSX.
• The inbound traffic is denied for all namespaces by default.
• The NSX Edge load balancer is used to make the services available to
external networks and ingress traffic.
• Internal communication between vSphere pods and the Supervisor cluster
is provided by the NSX Distributed load balancer that uses the ClusterIP
service type.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 22

Supervisor Cluster IP Addressing
NSX IPAM:
• Private IPv4 address pool for workloads in a
namespace
• Service IPv4 address pool for services available within
the namespace through the K8s ClusterIP
• Public IPv4 address pool for making the services
available outside the Supervisor cluster through K8s
Type Load Balancer, Ingress, and through cloud
provider load balancer to Tanzu Kubernetes Grid
clusters
• Public IPv4 address pool to NAT traffic outside the
Supervisor cluster

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 23

vSphere with VMware Tanzu Deployment Prerequisites
NSX-T Data Center must be installed and configured:
• NSX Manager nodes deployed
• Compute Manager vCenter Server added to NSX
Manager:
– Enable Trust: Yes
• Workload management vSphere cluster prepared with
NSX-T Data Center
• NSX Edge nodes deployed and edge clusters defined

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 24

vSphere with Tanzu Supervisor Cluster
The diagram provides the NSX-T Data Center networking specifics.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 25

vSphere with Tanzu Supervisor Cluster (1)
NSX-T Data Center is deployed.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 26

vSphere with Tanzu Supervisor Cluster (2)
The namespace is created.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 27

vSphere with Tanzu Supervisor Cluster (3)
The vSphere pod is created.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 28

Distributed Load Balancer in the Supervisor Cluster
The Service of type ClusterIP is available for only vSphere with Tanzu.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 29

NSX-T Data Center Server Load Balancing for Supervisor Cluster
Ingress and LoadBalancer type is the K8s service.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 30

NSX-T Data Center Security Features for vSphere with Tanzu (1)
The network policy drives NSX Security.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 31

NSX-T Data Center Security Features for vSphere with Tanzu (2)
North-South communications:
• Ingress to namespace is disabled by default.
• Egress from namespace is enabled by default.
• Namespace to namespace is denied by default.
East-West communications:
• Intranamespace communications are allowed by
default.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 32

NSX-T Data Center Security Features for vSphere with Tanzu
Default DFW policies are available per WCP-enabled
cluster:
• The K8s network policy is not used.
The Supervisor cluster and Guest cluster base rules are
included in DFW on creation:
• The East-West communications between Supervisor
VMs and Guest cluster VMs are allowed by default.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 33

vSphere with Tanzu Security Solution Architecture (1)
North-South communications:
• Ingress to the namespace is disabled by default.
• Egress from the namespace is enabled by default.
• Namespace to namespace is denied by default.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 34

vSphere with Tanzu Security Solution Architecture (2)
East-west communications:
• Intranamespace communications are allowed by
default.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 35

vSphere with Tanzu Supervisor Cluster
The diagram shows the vSphere pod network policy translation.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 36

vSphere with Tanzu Edge and Load-Balancing Sizing

Edge cluster sizing 10 edge nodes

Edge node sizing Large
Supervisor cluster LB sizing Medium K8s API Control VIP
L4 load-balancing for vSphere pods and ingress
for vSphere pods

vSphere with Tanzu fixes and provisions the LB sizing on demand as Supervisor clusters are created:
• The load balancer configuration limits based on the LB instance size are removed.
• Dedicated edge clusters for T0 are allowed.
• Dedicated edge clusters per Supervisor cluster are also permitted.
• Shared T0/T1 edge clusters are also permitted.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 37

Review of Learner Objectives
• Describe the networking topology and security features of the Supervisor cluster
• Identify the NSX-T Data Center networking capabilities of the Supervisor cluster
• Describe the Supervisor cluster load-balancing design
• Identify the NSX-T Data Center security capabilities of the Supervisor cluster

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 38

Lesson 3: Overview of the Tanzu Kubernetes Grid
Cluster

© 2019 VMware Inc. All rights reserved.

Learner Objectives
• Describe Tanzu Kubernetes Grid Service and its use cases
• Understand the Tanzu Kubernetes Grid cluster networking and load-balancing capabilities

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 40

About Tanzu Kubernetes Grid Service for vSphere
Tanzu Kubernetes Grid Service for vSphere provides self-service life cycle
management of Tanzu Kubernetes Grid clusters within the scope of a
Supervisor cluster namespace:
• With this service, Open Container Initiative (OCI) conformant Kubernetes
clusters can be provisioned.
• Control plane and worker nodes are provisioned as virtual machines.
• Authorized users have complete control over an isolated Kubernetes
cluster.
• Tanzu Kubernetes Grid clusters are deployed and configured by using
Cluster API.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 41

Different Profiles of Tanzu Kubernetes Grid

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 42

Use Cases: vSphere with Tanzu and Tanzu Kubernetes Grid Service
The decision to use either vSphere pods or Tanzu Kubernetes Grid Service for vSphere depends on your goals for
deploying and managing Kubernetes workloads on vSphere with Tanzu.

Use vSphere pods: Use Tanzu Kubernetes Grid Service for vSphere:
• For running containers without managing the Tanzu • For running containers on an OCI-conformant
Kubernetes Grid cluster life cycle Kubernetes cluster
• For running containers with strong resource and • When you require root-level control of a Kubernetes
security isolation cluster
• When you do not require management of a Tanzu • When you require an open-source solution for
Kubernetes Grid cluster deployment, monitoring, and operation of Kubernetes
clusters
• When you require customization of a Kubernetes
distribution

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 43

Tanzu Kubernetes Grid Cluster
You design and deploy a Tanzu Kubernetes Grid cluster.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 44

NSX-T Data Center Load Balancing for Tanzu Kubernetes Grid Clusters
You make a service available in the Tanzu Kubernetes Grid cluster.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 45

Networking for the Tanzu Kubernetes Grid Cluster
NSX-T Data Center provides several features for Tanzu Kubernetes Grid clusters:
• Node VM connectivity: NSX-T Data Center LS
• Service Type LB: NSX-T Data Center Cloud Provider LB
The third-party CNI provides pod connectivity and network policy.
Ingress: Use any third-party ingress controller. NSX-T Data Center Ingress is not available to the Tanzu Kubernetes
Grid cluster.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 46

Review of Learner Objectives
• Describe Tanzu Kubernetes Grid Service and its use cases
• Understand the Tanzu Kubernetes Grid cluster networking and load-balancing capabilities

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 47

Key Points (1)
• vSphere with Tanzu transforms vSphere into a native Kubernetes platform.
• vSphere with Tanzu is available through VMware Cloud Foundation 4.
• The Networking stack for vSphere with Tanzu is powered by NSX-T Data Center.
• In vSphere with Tanzu, a Supervisor cluster is a special Kubernetes cluster that uses ESXi as the worker nodes.
• The vSphere pod, network, storage, and registry services support the vSphere with Tanzu Infrastructure.

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 48

Key Points (2)
• The Supervisor cluster supports two design options: Shared Tier-0 and edge nodes, and shared Tier-0 and dedicated
edge nodes.
• Supervisor clusters are isolated with the edge firewall on Tier-1 gateways and distributed firewall per vNIC.
• Supervisor cluster uses the NSX Edge load balancer for external networks and the NSX distributed load balancer
for internal communication.
• Tanzu Kubernetes Grid Service for vSphere provides self-service life cycle management of Tanzu Kubernetes Grid
clusters within the scope of a Supervisor cluster namespace.
Questions?

© 2020 VMware, Inc. VMware NSX-T Data Center: Design | 12 - 49