Configure Multi-Tenancy Ethernet Network#
Note
Multi-tenant cluster features are only available if you have paid for multi-tenant support. Contact Penguin Computing to learn more.
The following sections describe the process for configuring an Ethernet network for ICE ClusterWare ™ multi-tenancy.
Important
This configuration process assumes you have a general understanding of networking, networking switches, and the BGP routing protocol. Contact Penguin Computing if you need assistance configuring your network.
As part of this configuration, you will:
Configure the network fabric by creating custom JSON configuration files for each switch, including:
Setting up interfaces
Configuring VLAN for network isolation between tenancies
Adding a loopback interface
Configuring global BGP properties
Establishing router peers and neighbors
Adding a range of allowed Kubernetes peer neighbors
Complete these steps after installing the multitenancy-clusterware package
and before running the setup-for-mt script.
Example Ethernet Network Topology#
The example topology used for this document is as follows:
Spine0
// \\
// \\
k8s1--Leaf1 Leaf2--n0
k8s2-| |-n1
k8s3-| |-n2
|-n3
Spine0 forms the top of the network and is connected to Leaf1 and Leaf2. The Harvester cluster is associated with Leaf1.
In a production environment, there may be multiple leaf switches with Kubernetes workers and multiple leaf switches with compute nodes. Due to ASIC and offload limitations, ClusterWare multi-tenancy does not support mixing Kubernetes workers and compute nodes on the same switch.
Add Ethernet Switches to ClusterWare#
Add all relevant switches to the ClusterWare control plane.
Create naming pools for the spine and leaf switches:
admin@cwhead1: ~$ cw-clusterctl pools create name=spine pattern="spine{}" admin@cwhead1: ~$ cw-clusterctl pools create name=leaf pattern="leaf{}"Create the Sonic Ethernet switches using the appropriate naming pool:
admin@cwhead1:~$ cw-switchctl create mac=ca:fe:be:ef:00:00 naming_pool=spine type=sonic admin@cwhead1:~$ cw-switchctl create mac=ca:fe:be:ef:01:01 naming_pool=leaf type=sonic admin@cwhead1:~$ cw-switchctl create mac=ca:fe:be:ef:01:02 naming_pool=leaf type=sonic
Ping the management interfaces for each switch to ensure that all your switches are responsive. If a switch is unresponsive, ClusterWare is not able to configure the switch, which may result in unexpected behavior.
Network Fabric Configuration#
The following sections detail steps needed to configure your networking fabric.
Configure Interfaces#
Each switch requires its own configuration information. The configuration varies from switch to switch. For larger fabrics, it is advisable to generate these via a script. In this small switch example fabric, it is reasonable to write these configurations manually.
The following sections detail the configuration files used for the example topology. The configurations are eventually added to the switches you created in the ClusterWare software so they can be managed and updated as you adjust tenancy node membership.
This document uses BGP-unnumbered to advertise fabric neighbor reachability. This is the simplest possible configuration. However, if your environment requires Route Reflectors, has non-SONIC spine switches, or you cannot use IPv6 in your fabric, then BGP Unnumbered is unlikely to work and the spines may need to be manually configured. Reach out to Penguin Computing for alternative fabric configurations.
Interface breakout is not currently supported by the ClusterWare software. If you need any breakouts, conduct those manually via shell before continuing. Consult your Enterprise SONIC documentation for details on how to do this for your specific switch.
Once interfaces are broken out, ClusterWare can manage the interfaces. To tell the ClusterWare software that it should manage an interface, add it to the interfaces sub-object in the config.
To bring up the ISLs, start with the following JSON configuration files for each switch:
spine0.swcfg.json:
{
"interfaces": {
"ports": {
"Ethernet0": {},
"Ethernet4": {},
"Ethernet8": {},
"Ethernet12": {},
"Ethernet16": {},
"Ethernet20": {},
"Ethernet24": {},
"Ethernet28": {}
}
}
}
leaf1.swcfg.json:
{
"interfaces": {
"ports": {
"Ethernet112": {},
"Ethernet116": {},
"Ethernet120": {},
"Ethernet124": {}
}
}
}
leaf2.swcfg.json:
{
"interfaces": {
"ports": {
"Ethernet112": {},
"Ethernet116": {},
"Ethernet120": {},
"Ethernet124": {}
}
}
}
Since leaf1 also provides connectivity to the Harvester nodes, configure a switched VLAN for those nodes:
leaf1.swcfg.json:
{
"interfaces": {
+ "vlans": {
+ "Vlan1": {
+ "ip_addresses": { "ipv4": ["10.0.1.254/24"] }
+ }
+ },
"ports": {
"Ethernet112": {},
"Ethernet116": {},
"Ethernet120": {},
"Ethernet124": {}
}
}
}
This addition configures a routed VLAN with a routed IPv4 interface on the switches. However, the ports to the servers still need to be configured to pass traffic to this VLAN.
leaf1.swcfg.json:
{
"interfaces": {
"vlans": {
"Vlan1": {
"ip_addresses": ["10.0.1.254/24"]
},
},
"ports": {
+ "Ethernet0": {
+ "switchport": {
+ "native": 1,
+ }
+ },
+ "Ethernet4": {
+ "switchport": {
+ "native": 1,
+ }
+ },
+ "Ethernet8": {
+ "switchport": {
+ "native": 1,
+ }
+ },
"Ethernet112": {},
"Ethernet116": {},
"Ethernet120": {},
"Ethernet124": {}
}
}
Note
The ClusterWare software does not support DHCP relays, so if netboot is required for these switches, consult your Enterprise Sonic documentation to manually add the required configuration to the VLAN.
Configure BGP for Reachability Announcements#
To propagate reachability information (of how to route packets from one endpoint to another) throughout the cluster, using BGP Unnumbered is recommended to simplify the configuration needed for each switch. BGP Unnumbered is only guaranteed to work when using switches with the same OS version between the spines and the leaf switches. In this configuration, IPv4 routes have an IPv6 nexthop as described in RFC5549.
If your switches do not run the same OS version between leaf and spine switches, additional configuration and network design are needed. Contact Penguin Computing about your network requirements.
Configure a Loopback#
To do BGP Unnumbered and EVPN-VXLAN, you need a loopback interface on each switch participating in the fabric - ideally with both v4 and v6, even if v6 is not exposed to clients. To configure a loopback interface, modify the above configurations as follows:
spine0.swcfg.json:
{
"interfaces": {
+ "loopbacks": {
+ "Loopback0": {
+ "ip_addresses": { "ipv4": ["172.16.0.0/32"], "ipv6": ["2001:db8::/128"] }
+ }
+ },
"ports": {
"Ethernet0": {},
"Ethernet4": {},
"Ethernet8": {},
"Ethernet12": {},
"Ethernet16": {},
"Ethernet20": {},
"Ethernet24": {},
"Ethernet28": {}
}
}
}
leaf1.swcfg.json:
{
"interfaces": {
+ "loopbacks": {
+ "Loopback0": {
+ "ip_addresses": { "ipv4": ["172.16.0.1/32"], "ipv6": ["2001:db8::1/128"] }
+ }
+ },
"vlans": {
"Vlan1": {
"ip_addresses": ["10.0.1.254/24"]
},
},
"ports": {
"Ethernet0": {
"switchport": {
"native": 1,
}
},
"Ethernet4": {
"switchport": {
"native": 1,
}
},
"Ethernet8": {
"switchport": {
"native": 1,
}
},
"Ethernet112": {},
"Ethernet116": {},
"Ethernet120": {},
"Ethernet124": {}
}
}
leaf2.swcfg.json:
{
"interfaces": {
+ "loopbacks": {
+ "Loopback0": {
+ "ip_addresses": { "ipv4": ["172.16.0.2/32"], "ipv6": ["2001:db8::2/128"] }
+ }
+ },
"ports": {
"Ethernet112": {},
"Ethernet116": {},
"Ethernet120": {},
"Ethernet124": {}
}
}
}
Configure Global BGP Properties#
Each switch requires some global BGP configuration. These settings include router-id, ASN, and some shared properties like peer-groups used to simplify configurations.
For this example configuration, a router-id that is the same as the loopback address and ASN from the 4-byte private ASN space is needed to do EBGP. If your fabric has more complex topology needs, contact Penguin Computing.
spine0.swcfg.json:
{
+ "bgp" : {
+ "asn": 4200000000,
+ "router_id": 172.16.0.0,
+ "ipv4": {
+ "redistribute": "connected"
+ },
+ "ipv6": {
+ "redistribute": "connected"
+ },
+ "evpn": {
+ "enabled": true
+ }
+ },
"interfaces": {
"loopbacks": {
"Loopback0": {
"ip_addresses": { "ipv4": ["172.16.0.0/32"], "ipv6": ["2001:db8::/128"] }
}
},
"ports": {
"Ethernet0": {},
"Ethernet4": {},
"Ethernet8": {},
"Ethernet12": {},
"Ethernet16": {},
"Ethernet20": {},
"Ethernet24": {},
"Ethernet28": {}
}
}
}
The added lines include a BGP configuration that:
Configures the router-id to match the loopback IP
Configures the ASN to 4200000000
Enables the IPv4 address family
Enables the IPv6 address family
Enables EVPN on the BGP router
Similarly, leaf1 and leaf 2 are also configured:
leaf1.swcfg.json:
{
+ "bgp" : {
+ "asn": 4200000001,
+ "router_id": 172.16.0.1,
+ "ipv4": {
+ "redistribute": "connected"
+ },
+ "ipv6": {
+ "redistribute": "connected"
+ },
+ "evpn": {
+ "enabled": true
+ }
+ },
"interfaces": {
"loopbacks": {
"Loopback0": {
"ip_addresses": { "ipv4": ["172.16.0.1/32"], "ipv6": ["2001:db8::1/128"] }
}
},
"vlans": {
"Vlan1": {
"ip_addresses": ["10.0.1.254/24"]
},
},
"ports": {
"Ethernet0": {
"switchport": {
"native": 1,
}
},
"Ethernet4": {
"switchport": {
"native": 1,
}
},
"Ethernet8": {
"switchport": {
"native": 1,
}
},
"Ethernet112": {},
"Ethernet116": {},
"Ethernet120": {},
"Ethernet124": {}
}
}
leaf2.swcfg.json:
{
+ "bgp" : {
+ "asn": 4200000002,
+ "router_id": 172.16.0.2,
+ "ipv4": {
+ "redistribute": "connected"
+ },
+ "ipv6": {
+ "redistribute": "connected"
+ },
+ "evpn": {
+ "enabled": true
+ }
+ },
"interfaces": {
"loopbacks": {
"Loopback0": {
"ip_addresses": { "ipv4": ["172.16.0.2/32"], "ipv6": ["2001:db8::2/128"] }
}
},
"ports": {
"Ethernet112": {},
"Ethernet116": {},
"Ethernet120": {},
"Ethernet124": {}
}
}
}
Note how ASN and router-id changes from switch to switch - this is required for BGP Unnumbered on a spine-leaf topology.
Configure Fabric BGP Peers#
To allow routes to flow between the routers, configure the routers to acknowledge others as peers. In this example, all peers are inside a peer group called “fabric,” which ensures a uniform configuration across the fabric.
spine0.swcfg.json:
{
"bgp" : {
"asn": 4200000000,
"router_id": 172.16.0.0,
+ "peer_groups": {
+ "fabric": {
+ "remote": "EXTERNAL",
+ "bfd": true,
+ "af_ipv4": { "enabled": true },
+ "af_ipv6": { "enabled": true },
+ "af_evpn": true
+ }
+ },
"ipv4": {
"redistribute": "connected"
},
"ipv6": {
"redistribute": "connected"
},
"evpn": {
"enabled": true
}
},
"interfaces": {
"loopbacks": {
"Loopback0": {
"ip_addresses": { "ipv4": ["172.16.0.0/32"], "ipv6": ["2001:db8::/128"] }
}
},
"ports": {
"Ethernet0": {},
"Ethernet4": {},
"Ethernet8": {},
"Ethernet12": {},
"Ethernet16": {},
"Ethernet20": {},
"Ethernet24": {},
"Ethernet28": {}
}
}
}
leaf1.swcfg.json:
{
"bgp" : {
"asn": 4200000001,
"router_id": 172.16.0.1,
+ "peer_groups": {
+ "fabric": {
+ "remote": "EXTERNAL",
+ "bfd": true,
+ "af_ipv4": { "enabled": true },
+ "af_ipv6": { "enabled": true },
+ "af_evpn": true
+ }
+ },
"ipv4": {
"redistribute": "connected"
},
"ipv6": {
"redistribute": "connected"
},
"evpn": {
"enabled": true
}
},
"interfaces": {
"loopbacks": {
"Loopback0": {
"ip_addresses": { "ipv4": ["172.16.0.1/32"], "ipv6": ["2001:db8::1/128"] }
}
},
"vlans": {
"Vlan1": {
"ip_addresses": ["10.0.1.254/24"]
},
},
"ports": {
"Ethernet0": {
"switchport": {
"native": 1,
}
},
"Ethernet4": {
"switchport": {
"native": 1,
}
},
"Ethernet8": {
"switchport": {
"native": 1,
}
},
"Ethernet112": {},
"Ethernet116": {},
"Ethernet120": {},
"Ethernet124": {}
}
}
leaf2.swcfg.json:
{
"bgp" : {
"asn": 4200000002,
"router_id": 172.16.0.2,
+ "peer_groups": {
+ "fabric": {
+ "remote": "EXTERNAL",
+ "bfd": true,
+ "af_ipv4": { "enabled": true },
+ "af_ipv6": { "enabled": true },
+ "af_evpn": true
+ }
+ },
"ipv4": {
"redistribute": "connected"
},
"ipv6": {
"redistribute": "connected"
},
"evpn": {
"enabled": true
}
},
"interfaces": {
"loopbacks": {
"Loopback0": {
"ip_addresses": { "ipv4": ["172.16.0.2/32"], "ipv6": ["2001:db8::2/128"] }
}
},
"ports": {
"Ethernet112": {},
"Ethernet116": {},
"Ethernet120": {},
"Ethernet124": {}
}
}
}
Next, unnumbered peers are added by their interface name.
spine0.swcfg.json:
{
"bgp" : {
"asn": 4200000000,
"router_id": 172.16.0.0,
"peer_groups": {
"fabric": {
"remote": "EXTERNAL",
"bfd": true,
"af_ipv4": { "enabled": true },
"af_ipv6": { "enabled": true },
"af_evpn": true
}
},
+ "neighbors": {
+ "Ethernet0": { "options": {"remote": "UNNUMBERED"}, "peer_group": "fabric" },
+ "Ethernet4": { "options": {"remote": "UNNUMBERED"}, "peer_group": "fabric" },
+ "Ethernet8": { "options": {"remote": "UNNUMBERED"}, "peer_group": "fabric" },
+ "Ethernet12": { "options": {"remote": "UNNUMBERED"}, "peer_group": "fabric" },
+ "Ethernet16": { "options": {"remote": "UNNUMBERED"}, "peer_group": "fabric" },
+ "Ethernet20": { "options": {"remote": "UNNUMBERED"}, "peer_group": "fabric" },
+ "Ethernet24": { "options": {"remote": "UNNUMBERED"}, "peer_group": "fabric" },
+ "Ethernet28": { "options": {"remote": "UNNUMBERED"}, "peer_group": "fabric" }
+ },
"ipv4": {
"redistribute": "connected"
},
"ipv6": {
"redistribute": "connected"
},
"evpn": {
"enabled": true
}
},
"interfaces": {
"loopbacks": {
"Loopback0": {
"ip_addresses": { "ipv4": ["172.16.0.0/32"], "ipv6": ["2001:db8::/128"] }
}
},
"ports": {
"Ethernet0": {"},
"Ethernet4": {},
"Ethernet8": {},
"Ethernet12": {},
"Ethernet16": {},
"Ethernet20": {},
"Ethernet24": {},
"Ethernet28": {}
}
}
}
leaf1.swcfg.json:
{
"bgp" : {
"asn": 4200000001,
"router_id": 172.16.0.1,
"peer_groups": {
"fabric": {
"remote": "EXTERNAL",
"bfd": true,
"af_ipv4": { "enabled": true },
"af_ipv6": { "enabled": true },
"af_evpn": true
}
},
+ "neighbors": {
+ "Ethernet112": { "options": {"remote": "UNNUMBERED"}, "peer_group": "fabric" },
+ "Ethernet116": { "options": {"remote": "UNNUMBERED"}, "peer_group": "fabric" },
+ "Ethernet120": { "options": {"remote": "UNNUMBERED"}, "peer_group": "fabric" },
+ "Ethernet124": { "options": {"remote": "UNNUMBERED"}, "peer_group": "fabric" }
+ },
"ipv4": {
"redistribute": "connected"
},
"ipv6": {
"redistribute": "connected"
},
"evpn": {
"enabled": true
}
},
"interfaces": {
"loopbacks": {
"Loopback0": {
"ip_addresses": { "ipv4": ["172.16.0.1/32"], "ipv6": ["2001:db8::1/128"] }
}
},
"vlans": {
"Vlan1": {
"ip_addresses": ["10.0.1.254/24"]
},
},
"ports": {
"Ethernet0": {
"switchport": {
"native": 1,
}
},
"Ethernet4": {
"switchport": {
"native": 1,
}
},
"Ethernet8": {
"switchport": {
"native": 1,
}
},
"Ethernet112": {},
"Ethernet116": {},
"Ethernet120": {},
"Ethernet124": {}
}
}
leaf2.swcfg.json:
{
"bgp" : {
"asn": 4200000002,
"router_id": 172.16.0.2,
"peer_groups": {
"fabric": {
"remote": "EXTERNAL",
"bfd": true,
"af_ipv4": { "enabled": true },
"af_ipv6": { "enabled": true },
"af_evpn": true
}
},
+ "neighbors": {
+ "Ethernet112": { "options": {"remote": "UNNUMBERED"}, "peer_group": "fabric" },
+ "Ethernet116": { "options": {"remote": "UNNUMBERED"}, "peer_group": "fabric" },
+ "Ethernet120": { "options": {"remote": "UNNUMBERED"}, "peer_group": "fabric" },
+ "Ethernet124": { "options": {"remote": "UNNUMBERED"}, "peer_group": "fabric" }
+ },
"ipv4": {
"redistribute": "connected"
},
"ipv6": {
"redistribute": "connected"
},
"evpn": {
"enabled": true
}
},
"interfaces": {
"loopbacks": {
"Loopback0": {
"ip_addresses": { "ipv4": ["172.16.0.2/32"], "ipv6": ["2001:db8::2/128"] }
}
},
"ports": {
"Ethernet112": {},
"Ethernet116": {},
"Ethernet120": {},
"Ethernet124": {}
}
}
}
Kubernetes Setup#
Harvester HCI 1.5.1 is the only supported distribution of Kubernetes for ClusterWare multi-tenancy. In this example setup, it is assumed that every node is adjacent on the same leaf. In a proper production network, there should be multiple leaf switches in the topology.
In a multi-tenant ClusterWare configuration, Kubernetes nodes peer with other nodes in the same network and with the switches to exchange information about VXLAN tunnels.
Prepare for Kubernetes BGP Peering#
Configure the leaf switches that the Kubernetes nodes are adjacent to so they accept peering sessions from those nodes. A good way to do this is via a range of allowed peers.
leaf1.swcfg.json:
{
"bgp" : {
"asn": 4200000001,
"router_id": 172.16.0.1,
"peer_groups": {
"fabric": {
"remote": "EXTERNAL",
"bfd": true,
"af_ipv4": { "enabled": true },
"af_ipv6": { "enabled": true },
"af_evpn": true
}
},
"neighbors": {
+ "10.0.1.0/24": {
+ "options": {
+ "remote": "EXTERNAL",
+ "bfd": true,
+ "af_ipv4": { "enabled": true },
+ "af_ipv6": { "enabled": true },
+ "af_evpn": true
+ }
+ },
"Ethernet112": { "options": {"remote": "UNNUMBERED"}, "peer_group": "fabric" },
"Ethernet116": { "options": {"remote": "UNNUMBERED"}, "peer_group": "fabric" },
"Ethernet120": { "options": {"remote": "UNNUMBERED"}, "peer_group": "fabric" },
"Ethernet124": { "options": {"remote": "UNNUMBERED"}, "peer_group": "fabric" }
},
"ipv4": {
"redistribute": "connected"
},
"ipv6": {
"redistribute": "connected"
},
"evpn": {
"enabled": true
}
},
"interfaces": {
"loopbacks": {
"Loopback0": {
"ip_addresses": { "ipv4": ["172.16.0.1/32"], "ipv6": ["2001:db8::1/128"] }
}
},
"vlans": {
"Vlan1": {
"ip_addresses": ["10.0.1.254/24"]
},
},
"ports": {
"Ethernet0": {
"switchport": {
"native": 1,
}
},
"Ethernet4": {
"switchport": {
"native": 1,
}
},
"Ethernet8": {
"switchport": {
"native": 1,
}
},
"Ethernet112": {},
"Ethernet116": {},
"Ethernet120": {},
"Ethernet124": {}
}
}
This allows the Kubernetes nodes to begin establishing peering sessions with the leaf.
Labeling the Kubernetes Nodes#
To allow more selective configurations for Kubernetes BGP peering, it's recommended that all Kubernetes nodes are labeled to indicate which network they belong to. This is optional for very small or test deployments, but is necessary when the deployment size is larger than 1 rack.
For the example toplogy:
[user@head01 ~]$ kubectl label node k8s1 topology.kubernetes.io/zone=leaf-l1
[user@head01 ~]$ kubectl label node k8s2 topology.kubernetes.io/zone=leaf-l1
[user@head01 ~]$ kubectl label node k8s3 topology.kubernetes.io/zone=leaf-l1
Although this label can be any valid label, for on-premesis deployments, it's
recommended to use the topology.kubernetes.io/zone label. This also helps
Kubernetes distribute workloads across multiple failure zones, if provided.
Configure Kubernetes BGP Peering#
Because all nodes on the same leaf have similar connectivity, they should be configured in an iBGP full mesh configuration to reduce ASNs required. The configuration can be shared across all nodes with the same ASN.
Important
The namespace: cw-management value is required for the switch
configuration to be managed by the ClusterWare software. Be sure to match
that metadata value in all FRR and EVPN files.
leaf1-nodes-frr.yml:
apiVersion: frrk8s.metallb.io/v1beta1
kind: FRRConfiguration
metadata:
name: leaf1-nodes-frr
namespace: cw-management
spec:
nodeSelector:
matchLabels:
topology.kubernetes.io/zone=leaf-l1
bgp:
routers:
- asn: 4294967294
neighbors:
- address: 10.0.1.254
asn: 4200000001
# Configuring prefixes is optional if your switch is your default gateway.
allowed:
prefixes
# Learning loopbacks over IPv4 is strongly recommended.
- prefix: 172.16.0.0/12
ge: 32
ebgpMultiHop: true
- address: 10.0.1.1
asn: 4294967294
- address: 10.0.1.2
asn: 4294967294
- address: 10.0.1.3
asn: 4294967294
Next, configure the nodes to pass EVPN.
leaf1-nodes-evpn.yml:
apiVersion: frrk8s.metallb.io/v1beta1
kind: FRRConfiguration
metadata:
name: leaf1-nodes-evpn
namespace: cw-management
spec:
nodeSelector:
matchLabels:
topology.kubernetes.io/zone=leaf-l1
raw:
priority: 1
rawConfig: |-
router bgp 4294967294
neighbor 10.0.1.1 capability extended-nexthop
neighbor 10.0.1.1 capability extended-nexthop
neighbor 10.0.1.1 capability extended-nexthop
neighbor 10.0.1.254 capability extended-nexthop
address-family l2vpn evpn
advertise-all-vni
advertise-svi-ip
flooding head-end-replication
neighbor 10.0.1.1 activate
neighbor 10.0.1.2 activate
neighbor 10.0.1.3 activate
neighbor 10.0.1.254 activate
exit-address-family
exit
This creates two FRRConfiguration objects in the cw-management namespace
that match the previously annotated nodes:
leaf1-nodes-frr:
Configure the Kubernetes workers to speak BGP with ASN 4294967294
Set iBGP peering for all 3 Kubernetes nodes in the cluster (based on IPs)
Set eBGP peering to the leaf L1's VLAN (SVI) interface
(Optional) Allow the Kubernetes workers to learn loopbacks
leaf1-nodes-evpn: * Enable the extended-nexthop capability for each peer * Enable the L2VPN EVPN address family and activate it for each peer * Advertise all VNIs discovered on node * Enable multicast head-end-replication
Save all FRR and EVPN files to an accessible location. They are used as part of the multi-tenancy setup script process.
At this point, any new VXLAN overlays created by new tenancies should register themselves and propagate out to the cluster.
Apply Configuration to Multi-Tenant Switches#
Apply the JSON configuration files you created to each Sonic switch. When the JSON file is applied to the switch, the ClusterWare software configures the switch to match the configuration you provided.
You can add the configuration file with a single command:
cw-switchctl -i <switch> apply --replace <path to config>
Where <switch> is replaced by the switch name or UID and
<path to config> is replaced with a path to the configuration file.
Alternatively, you can upload and apply the configuration in separate steps.
Upload the new configuration file to the ClusterWare database, replacing
<switch>with the switch name or UID and<path to config>with the path to the new configuration file:cw-switchctl -i <switch> upload --target <path to config>
Apply the new configuration to the switch:
cw-switchctl -i <switch> apply
For example, running the following command uploads leaf1.swcfg.json to the
database, applies the configuration to Sonic switch leaf1, and updates Sonic
switch leaf1 with the new configuration:
cw-switchctl -i leaf1 apply --replace @$./leaf1.swcfg.json
When compute nodes are added to a tenancy, such as when a tenancy is created, ClusterWare automatically updates the switch configuration and the nodes are isolated from other tenancies. See Tenancy Network Isolation for details.