Using Kickstart

The Boot Configurations section discusses the creation and modification of compute node images and how to add locally installed nodes to the system. The ICE ClusterWare™ platform provides support for an additional method of dynamically provisioning compute nodes: Kickstart.

Kickstart Files

A kickstart file configures how a kickstarted node is initialized at boot time. The files are stored on the head node in the /opt/scyld/clusterware/kickstarts/ folder. In a multihead configuration this folder must currently be manually synced between head nodes. These kickstart files typically contain a limited set of variables that will be substituted at download time.

For example, the ClusterWare package includes the basic.ks kickstart file that begins with the following contents:

# Node fields, attributes, status, and hardware are available in
#   dictionaries referenced by name or initial. Some examples:
#
# <node[ip]>
# <a[_boot_config]>
# <hardware[mac]>

# Perform a SOL friendly text-based install.
text

# Pull some basics from the head node.
url --url <root_url()>
lang <head[lang]>
keyboard <head[keymap]>
timezone <head[timezone]>

The angle-bracket notation is used to substitute head-node or compute-node information into the kickstart file, essentially turning it into a kickstart template. When a node begins the kickstart process, it will query the head node for the file and all variables will automatically be substituted as the file is downloaded. This allows admins an opportunity to insert node-specific data into the kickstart. In addition to the <head[name]> syntax, node attributes may be referenced with <a[name]>, and similar template expansions are available for a node's hardware and status information, as well as other head- and system-specific substitutions. By templatizing the kickstart file, admins can better generalize those files for a simpler configuration process with fewer node-specific files or commands. For more information, including a complete list of template variables, see Variable Substitution.

The simple basic.ks performs a boot time install of both the base distribution core packages (e.g., from Rocky8repo) and the associated clusterware-node package which is appropriate for that particular base distribution.

Associate the boot configuration with a kickstart file:

scyld-bootctl -i Rocky8boot update kickstart=basic.ks

and associate a compute node to boot this boot configuration:

scyld-nodectl -i n0 set _boot_config=Rocky8boot

Finally, reboot node n0 to initiate the kickstart, which will take a few minutes to complete.

Once the freshly installed node has booted, there will be a /root/anaconda-ks.cfg file that can be used as a starting point for creating a more generalized kickstart file. If the cluster administrator would like to reinstall the node the exact same way, the simplest thing to do is copy that anaconda-ks.cfg file to the head node's kickstart directory and assign it to be used in the boot configuration:

# Copy the compute node's /root/anaconda-ks.cfg to the head node,
#  and then copy to the head node's kickstart files folder.
cp anaconda-ks.cfg /opt/scyld/clusterware/kickstarts
# And update the boot configuration to use the file.
scyld-bootctl -i Rocky8boot update kickstart=anaconda-ks.cfg

After that file is in place, any compute node booted from that boot configuration without the next or live boot style will boot using the kernel and initramfs from the ISO, and a URL to the kickstart file will be added to the kernel command line. Keep in mind that once a node starts the kickstart process, it is a good idea to change its boot style to next so that it does not reboot at the end of the install process and immediately reinstall. Configuring the kickstart process to end with a shutdown command (see your operating system documentation) is the current best practice.

If a cluster administrator wants to use a different kernel and/or initramfs for kickstarting instead of the ones found on the ISO, those can be replaced just like in any other boot configuration through the update action. Updating them with an empty string will reset them back to the detected paths:

scyld-bootctl -i Rocky8boot update kernel= initramfs=

Kickstart Failing

If a node has been configured to kickstart using a boot configuration provided by a repo created from an ISO file but is failing, then check the console output for the node. If the node is entering the "Dracut Emergency Shell" from the dracut timeout scripts, then you will need to retry and see what messages were on screen prior to the "Warning: dracut-initqueue timeout" messages that flood the screen. One common error is "Warning: anaconda: failed to fetch stage2 from <URL>", where the URL points to a repo on the head node. If this message occurs, please check that you have uploaded the correct ISO into the system.

For CentOS and RHEL, the "boot" ISO files such as CentOS-8.1.1911-x86_64-boot.iso do contain the files necessary to initiate the kickstart process, but do not contain the full package repositories. The cluster administrator must provide appropriate URLs in the kickstart file, or must upload a more complete ISO such as CentOS-8.1.1911-x86_64-dvd1.iso using the scyld-clusterctl command. For example, to replace the ISO originally uploaded into a newly created centos8_repo repo:

scyld-clusterctl repos -i centos8_repo update iso=@CentOS-8.1.1911-x86_64-dvd1.iso