Deploying Images Using Ignition#

To partition nodes during their boot process, the ICE ClusterWare ™ platform leverages tools called Ignition and Butane. Use Ignition and Butane for persistent boot nodes, such as login nodes, storage servers, or other infrastructure nodes.

Both Ignition and Butane originate from the Red Hat CoreOS project. The Ignition tool accepts a machine parsable JSON-formatted configuration file while the Butane tool translates a human-readable YAML format and produces the JSON file that Ignition needs. Details of these projects can be found at:

The functionality provided by Ignition enables the ClusterWare platform to partition and format local disks and deploy a root file system image to the newly prepared drives. This bypasses the previously used kickstart process to deploy locally-installed systems. The behavior of Ignition is controlled through a configuration file and node attributes. Another attribute triggers the installation of the GRUB2 bootloader, resulting in a fully provisioned and ready-to-boot system. By default these systems still use the ClusterWare platform for DHCP and report status back to the head nodes, but post-install changes can reconfigure the networking or disable the ClusterWare and Telegraf agents.

Note

One substantial difference between using Ignition directly and using it via the ClusterWare platform is that Ignition is intended to either produce exactly the desired result or nothing at all. However, when triggered by the _ignition attribute, the tool logs failures and proceeds. Future ClusterWare releases may change this behavior to more closely match the standard Ignition approach.

To use Ignition during node provisioning:

  1. Create an image that contains the desired operating system and software tools. Older ClusterWare images may not contain all the tools necessary for Ignition to properly partition or format the drives, such as sgdisk or mkfs.xfs. Ignition may rely on versions of tools that are not available on older operating systems.

  2. Create a boot configuration using the image you just created:

    cw-add-boot-config --image <IMAGENAME> --boot-config <BOOTNAME>

    Where <IMAGENAME> is the name of the image you just created and <BOOTNAME> is a name for the new boot configuration.

  3. Set node attributes on the target node:

    Attribute

    Example

    Description

    Required

    _boot_config

    DeployBoot

    Name of the boot configuration to deploy

    yes

    _boot_style

    disked

    Must be "disked"

    yes

    _disk_root

    LABEL=root

    Reference to the new root partition / file system

    yes

    _ignition

    config.butane

    Name of the Butane configuration file ending in .butane

    yes

    _bootloader

    grub

    Only the standard GRUB boot loader is currently supported.

    yes

    _wipe_all

    now

    Setting to "now" irreversibly wipes all data from local disks.

    no

  4. Once these attributes are set, reboot the node.

    The node boots using the kernel and initramfs from the specified boot configuration. Within the initramfs, the system optionally wipes all data from the local disks and then executes the various Ignition stages during the normal image download and unpacking process as follows:

    • After networking is established and the node has downloaded its initial attributes.ini file, it detects the _ignition attribute and downloads the Ignition binary from the head node.

    • After downloading the root file system, the node runs the “fetch”, “kargs”, and “disks” stages to download the configuration, check the kernel arguments, and partition local disks.

    • Immediately before unpacking the file system, the node runs the “mount” stage and then proceeds to unpack the image into the mounted partitions.

    • After unpacking the image, the node runs the “files” stage to manipulate the unpacked file system.

    Once Ignition completes, the _bootloader attribute triggers installation of the GRUB2 bootloader as a separate step. A successful installation of grub also updates _boot_style to ensure nodes do not continually loop through deployment. Finally, a reboot and relabel of the file system is triggered.

  5. Copy the Butane configuration to the /opt/scyld/clusterware/kickstarts directory on all head nodes. As long as the file name ends with “.butane” the file will be translated by the Butane command at download time. A file with any other extension must be a valid Ignition configuration file, as it will be passed directly to the Ignition command without translation.

An ignition.log file is produced during the deployment process and can be found in /opt/scyld/clusterware-node/atboot on the booted node. If errors occur, it's likely that the node will not be able to boot. In that case, you can log in to the initramfs shell during the 20 second “press any key” countdown and examine the log file located at /atboot/ignition.log.

If you need to re-image a node after setting up Ignition, set the node to boot from the ClusterWare software and re-image by repeating the same steps. For example:

  1. Reset the _wipe_all and _boot_style reserved attributes.

  2. Reboot the node to reprovision.

Ignition and Butane Configuration File Examples#

The clusterware-tools package includes a very simple Ignition example file at /opt/scyld/clusterware-tools/examples/partitions.butane. That example file partitions the local drive (/dev/vda on a VM) into three partitions, /, /boot, and /boot/efi where /boot is 1024 MB in size, /boot/efi is 200 MB in size, and / encompasses the rest of the drive. A oneshot systemd unit is also created as part of the Butane file to reboot the system to the local installation at the end of deployment process.

A more advanced Butane file (/opt/scyld/clusterware-tools/examples/mdraids.butane) builds on the simple example and demonstrates how to deploy a disk configuration that mirrors partitions, including the /boot/efi partition, on a pair of drives while using other drives as a raid5 scratch partition.

Tip

When starting with either example, review all of the partitions that are created to ensure their mount points exist within the image you will deploy.

Additional examples of Butane configuration files can be found in the Butane project documentation: https://coreos.github.io/butane/examples/