Network Switches#

Network switches are used to connect compute nodes in a cluster via Ethernet. The primary tool for interacting with switches from the ICE ClusterWare ™ command line is cw-switchctl. Use this tool to add a switch, add or remove attribute groups, view the basic switch hardware, monitor switch status, reboot a switch, or execute commands on a switch. You can also create and modify switches from the Switches Page in the ClusterWare GUI.

Like compute nodes, switches are named by default in the form of nX, where X is a numeric zero-based index. More complicated clusters may benefit from more flexible naming schemes. See Switch Naming Pools for details.

Each switch in ClusterWare is associated with a unique MAC address and must be assigned an associated type that corresponds to the network OS present on the switch. Currently the ClusterWare software supports Sonic, Cumulus, and Arista switches.

Optionally, a Cumulus switch can have an associated zero-touch provisioning (ZTP) configuration file. See Managing Zero-Touch Provisioning (ZTP) for Switches for details.

In a multi-tenant cluster, Ethernet switches are used for network isolation, often in combination with InfiniBand networking. Currently the ClusterWare software only supports Sonic switches in multi-tenant clusters. A JSON configuration file is required for multi-tenant switches. See Tenancy Network Isolation for details.

Create Switches#

To create a switch for a standard cluster:

  1. Use the cw-switchctl command line tool to create a switch:

    cw-switchctl mk mac=<mac address> type=<OS>

    Where <mac address> is replaced with a valid MAC address and <OS> is a switch operating system, such as sonic, cumulus, or arista.

  2. Set up ClusterWare software on the switch OS using one of the following options:

    • [Recommended] Install the clusterware-node package on the switch using the same pre-installer script you would use on a compute node. See Support for Diskful Nodes for details.

    • Enable sshpass on the switch.

      1. Set the _remote_pass and _remote_user reserved attributes on the switch so the switches can communicate with the ClusterWare software.

      2. Install the sshpass package on the head node(s) to enable _remote_pass and _remote_user on the switch.

To create a switch for a multi-tenant cluster, refer to the Ethernet switch configuration steps for multi-tenancy in Configure Multi-Tenancy Ethernet Network and Configure Multi-Tenant Switches.

If you have already configured multi-tenancy and are adding a new switch to your super-cluster, use the following high-level steps:

  1. Use the cw-switchctl command line tool to create a spine or leaf switch:

    cw-switchctl mk mac=<mac address> naming_pool=<spine or leaf> type=sonic

    Where <mac address> is replaced with a valid MAC address and <spine or leaf> is replaced with either "spine" or "leaf" depending on the switch. Only Sonic switches are supported for multi-tenant clusters.

    See Add Ethernet Switches to ClusterWare for details.

  2. Create a configuration file and add it to the switch. See Network Fabric Configuration and Apply Configuration to Multi-Tenant Switches.

  3. Install the clusterware-node package on the switch using the same pre-installer script you would use on a compute node. See Support for Diskful Nodes for details. While technically optional for multi-tenant clusters, this step is highly recommended.

Switch Naming Pools#

The basic create command adds a new switch with the default naming pool and default attribute group, just like compute nodes. Using the default naming pool and attribute group assigns a generic name like "n12", which is superficially indistinguishable from compute nodes "n0" through "n11". Instead, use the ClusterWare naming pool and attribute group functionality to assign a more self-identifying name and permit more efficient management of switches.

For example, you can create a new naming pool called "ztpswitch" and configure the new switch inside that pool with the name "switch0". Subsequent switches can use the same naming pool, which names them "switch1", "switch2", and so on.

cw-clusterctl pools create name=ztpswitch pattern=”switch{}”
cw-switchctl create mac=aa:bb:cc:00:11:22 naming_pool=ztpswitch

After creating the switch naming pool, you can use the following commands to perform an action on a specific switch or a common action on all switches in that naming pool:

cw-switchctl -i switch2 <action>
cw-switchctl -i switch* <action2>

See Node Names and Pools for more about naming pools.