Node Names and Pools#

By default all compute nodes are named nX, where X is a numeric zero-based node index. This pattern can be changed using "nodename" lines found in a cluster configuration file. For example, a line nodename compute{} early in such a file will change the default node naming to computeX. This changes both the default node hostnames as well as the names recognized by the scyld-nodectl command.

For homogeneous clusters where all compute nodes are essentially the same, this is usually adequate, although in more complex environments there is utility in quickly identifying core compute node capabilities reflected by customized hostnames. For example, high memory nodes and general purpose GPU compute nodes could be named "hmX" and "gpgpuX". These names can be assigned via the _hostname attribute as described in Reserved Attributes, although the scyld-nodectl command will still refer to them as "nX".

To support multiple name groupings within the scyld-*ctl tools, the ICE ClusterWare™ system includes the concept of a naming pool. These pools are defined and modified through the scyld-clusterctl pools command line interface. Once the appropriate pools are in place, then compute nodes can be added to those pools. For example:

scyld-clusterctl pools create name=high_mem pattern=hm{} first_index=1
scyld-clusterctl pools create name=general_gpu pattern=gpgpu{} first_index=1
scyld-nodectl -in[37-40] update naming_pool=high_mem
scyld-nodectl -in[41,42] update naming_pool=general_gpu

After these changes the scyld-nodectl status and scyld-nodectl ls output includes the specified nodes as "hm[1-4]" and "gpgpu[1-2]". Any commands that previously used "nX" names now accept "hmX" or "gpgpuX" names to refer to those renamed nodes. The first_index= field of the naming pool forces node numbering to begin with a specific value, defaulting to 0. Any nodes not explicitly attached to a naming pool use the general cluster naming pattern controlled through the scyld-clusterctl --set-naming PATTERN command. This can be considered the default naming pool.

Node Indexing and Grouping in Naming Pools#

Important

When moving multiple compute nodes from one naming pool to another, the node order may not be preserved. Instead, moving them individually, or specifying their MAC addresses in a cluster configuration file, may be more predictable.

When moving a node from one naming pool to another via the scyld-nodectl command, the node index is reset to the next available index in the destination pool. Using an explicit index=X argument allows the cluster administrator to directly control the node renumbering. Note that nodes in different naming pools may have the same index, so in this configuration the index is no longer a unique identifier for individual nodes. Further, the --up, --down, --all node selectors are not restricted to a single naming pool and will affect nodes in all pools that match the selection constraint. Nodes in scyld-nodectl output are ordered by index within their naming pool, although the order of the naming pools themselves is not guaranteed. For example:

[admin@head clusterware]$ scyld-nodectl ls
Nodes
  n1
  n2
  n3
  n4
  n5
  login6
  login7
  login8
  login9

Similarly, the nodes are grouped by naming pool in scyld-cluster-conf save output with "nodename" lines and explicit node indices inserted as needed:

[admin@head clusterware]$ scyld-cluster-conf save -
# Exported Scyld ClusterWare Configuration file
#
# This file contains the cluster configuration.
# Details of the syntax and semantics are covered in the
# Scyld ClusterWare Administrators Guide.
#
nodename n{}

# 10.10.24.0/24 network
domain cluster.local
1 10.10.24.101/24 10.10.24.115
node 1 00:00:00:00:00:01  # n1
node 00:00:00:00:00:02  # n2
node 00:00:00:00:00:03  # n3
node 00:00:00:00:00:04  # n4
node 00:00:00:00:00:05  # n5
nodename login{}
node 6 00:00:00:00:00:06  # login6
node 00:00:00:00:00:07  # login7
node 00:00:00:00:00:08  # login8
node 00:00:00:00:00:09  # login9

The organization of node naming pools is intentionally independent of node networking considerations. The cluster administrator can combine these concepts by creating separate naming pools for each network segment, although this is not necessary.

Secondary DNS names can also be defined using "nodename":

nodename <pattern> <ip> [pool_name]

A "nodename" line containing an IP address (or IP offset such as "0.0.1.0") can define a name change at an offset within the IP space or define a secondary DNS name depending on whether the IP is within a defined network. For example:

iprange 10.10.124.100/24 10.10.124.250
node
node 08:00:27:F0:44:35  # n1 @ 10.10.124.101

nodename hello{}/5 10.10.124.105
node 08:00:27:A2:3F:C9  # hello5 @ 10.10.124.105

nodename world{}/10 10.10.124.155
node 12 08:00:27:E5:19:E5  # world12 @ 10.10.124.157

nodename n%N-ipmi 10.2.255.37 ipmi
# world12 maps to n2-ipmi @ 10.2.255.39

nodename world%N-ipmi/10 10.2.254.37 ipmi
# world12 maps to world12-ipmi @ 10.2.254.39

Note that the "<pattern>/X" syntax defines the lowest node index allowed within the naming pool.

Secondary Naming Pools#

The ClusterWare platform provides a mechanism to include name resolution for non-management interfaces with predictable IP addresses on existing pools of nodes. You can create secondary naming pools via a configuration file or via the ClusterWare command line tools.

Configuration File#

The following configuration file uses the default n{} node naming convention, but also includes a secondary naming pool labelled ib0.

iprange 10.54.50.0/24
node 52:54:00:c6:c3:0a
node 52:54:00:c4:f7:1e
node 52:54:00:46:fe:99
node 52:54:00:dc:f2:e5
node 52:54:00:de:b3:5e
node 52:54:00:17:e7:be
node 52:54:00:48:05:f4
node 52:54:00:88:0f:82

nodename n{}-ib0 0.+1.0.0 ib0

When this configuration is loaded, the system resolves the following through both dnsmasq and scyld-nss:

  • Names n0 through n7

  • Names n0-ib0 through n7-ib0 where those -ib0 addresses are offset into the 10.55.50.0/24 network based on the provided specification (0.+1.0.0)

For example, n0 resolves to 10.54.50.0 and n0-ib0 resolves to 10.55.50.0.

Command Line Tools#

Secondary naming pools can also be created via the following command:

scyld-clusterctl pools create name=ib1 parent= pattern=n{}-ib1 ip_base=10.55.50.0

Where:

  • The parent argument should generally be the name of a primary naming pool. However, the default cluster naming is not recorded as a primary naming pool. Instead, it is stored and retrieved using scyld-clusterctl --set-naming <PATTERN> / --get-naming. Specifying an empty parent value in this command refers to that default cluster naming.

  • The pattern argument is formatted the same as any other ClusterWare naming pattern.

  • The ip_base argument denotes the first IP in the pool and must be explicitly defined rather than using an offset as is supported in configuration files.

After creating the secondary naming pool, the system resolves offset addresses for the secondary interface corresponding to any nodes in the referenced parent pool.