Quickstart#

This getting started guide provides the first steps for onboarding to the ClusterWareAI ™ platform. Use it to create a minimal ClusterWareAI cluster capable of running parallel jobs. The introductory cluster includes:

  • One head node server that manages cluster administration activities.

  • Two provisioned compute node servers, also known as worker nodes, that run end user applications.

The Quickstart assumes you are familiar with administering Red Hat RHEL or CentOS servers.

Tip

See Basic Cluster Architecture if you are new to cluster administration and want to learn more about head nodes, compute nodes, and how ClusterWareAI works.

Before You Begin: Quickstart Prerequisites#

Before you begin, make sure your environment has a minimum of two x86_64 servers, preferably three. One server becomes the ClusterWareAI head node and the remaining servers become ClusterWareAI compute nodes. This Quickstart uses three servers.

  • Head Node: A virtual machine running Red Hat RHEL 8.7 or later or CentOS Stream 8 or later. See Supported Distributions and Features for a full list of Linux distribution options. The head node requires a minimum of:

    • 4 GB RAM

    • 16 GB storage

    • An Ethernet controller connected to a private cluster network to communicate with the compute nodes.

    • An Ethernet controller connected to the external network (public internet) to access the ClusterWareAI repository.

    • Access to a repository for the base operating system (OS) so that the ClusterWareAI software can install additional packages.

  • Compute Nodes: Virtual machines (recommended for this Quickstart) or bare metal servers (recommended for production) with a minimum of:

    • 4 GB RAM

    • An Ethernet controller connected to the same private cluster network as the head node.

    • BIOS configured to PXE boot by default using either Legacy or UEFI mode.

These are the minimum requirements. See Required and Recommended Components for production recommendations.

Create Administrator#

Start by creating a new administrator user on the server you plan to use as your head node. ClusterWareAI cluster administrators require root privileges. Rather than logging in as root, create a dedicated administrator account with sudo privileges. This administrator becomes the first ClusterWareAI user during the installation process.

The example commands create an administrator account named admin1. Replace admin1 with a username appropriate for your environment.

  1. On the head node, create the administrator account:

    useradd admin1

  2. Set a password for the administrator account:

    passwd admin1

  3. Grant the administrator account passwordless full root sudo privileges:

    echo "admin1 ALL=(root) NOPASSWD: ALL" >> /etc/sudoers

  4. Switch to the new administrator account:

    su - admin1

  5. Generate an SSH key pair for the administrator account:

    ssh-keygen

    The SSH key pair is used for passwordless SSH authentication during cluster administration. Accept the default location unless your organization's security policies require a different location.

All remaining Quickstart steps are completed using the administrator user account you just created.

Install the ClusterWareAI Software#

Next, create a cluster configuration file and install the ClusterWareAI software on the head node. During installation, the ClusterWareAI software creates the initial boot image and boot configuration that are used to provision (create and boot) the compute nodes.

Create Cluster Configuration File#

On the head node, create a cluster configuration text file to use when configuring your cluster. The cluster configuration file should identify the private cluster network, the IP address on the network for the first compute node, and a list of MAC addresses to use as compute nodes.

For example:

cat <<-EOF >/tmp/cluster-conf
iprange 10.54.60.0          # starting IP address of node 0
node 52:54:00:a6:f3:3c      # node 0 MAC address
node 40:8d:5c:fa:ea:c3      # node 1 MAC address
EOF

Install the Software Package#

Install the clusterware-installer package on the head node:

sudo dnf install clusterware-installer

The clusterware-installer package contains the cw-install script, which installs the ClusterWareAI software on the head node and completes initial configuration.

Run the Installer#

There are two options when running the ClusterWareAI install script as part of this Quickstart:

  • If you want your compute node OS image to match your head node OS, run the ClusterWareAI installer with only your configuration file as an argument:

    cw-install --config /tmp/cluster-conf

  • To use a different OS or distribution version on your compute nodes, or if your head node is running Red Hat RHEL 8, specify the installation ISO as part of the install command:

    cw-install --config /tmp/cluster-conf --os-iso <ISO-file>

    Where <ISO-file> is the path or URL to a supported compute node OS installation ISO. For example:

    cw-install --config /tmp/cluster-conf --os-iso rhel-9.6-x86_64-dvd.iso

In most environments, the ClusterWareAI software repository is accessed through the head node's second Ethernet controller that is connected to the external network. If a ClusterWareAI software repository is not already configured, the cw-install installer prompts you for your repository credentials and creates /etc/yum.repos.d/clusterware.repo.

After the installation completes:

  • The DefaultImage image is created and contains the initial compute node file system image.

  • The DefaultBoot boot configuration is created. It references DefaultImage and includes boot-time information for compute nodes.

  • Two compute nodes, n0 and n1, are created and assigned to boot using DefaultBoot.

  • admin1 is added as a Full Admin user in the ClusterWareAI software.

Verify the Installation#

Verify the installation succeeded by booting the compute nodes and confirming that the ClusterWareAI software can monitor node status.

  1. Reboot both compute nodes to apply the DefaultBoot boot configuration and DefaultImage image.

  2. Monitor compute node provisioning by checking the node status from the ClusterWareAI software on the head node. The cw-nodectl tool is used when working with compute nodes:

    cw-nodectl status --refresh

    The status argument shows the current node status. Adding --refresh automatically updates the table with any state changes. Initially, the nodes should display a status similar to:

    Node status                                              [ date & time ]
------------------------------------------------------------------------
n[0-1] new

    As provisioning progresses, each node transitions from new to booting and finally to up when the nodes are fully booted and ready. The transition to booting takes a minute or more while the boot process completes hardware initialization, PXE boot provisioning, and early software initialization. The final transition to up can take another minute or more.

    Note

    If the nodes do not boot successfully, see Failing PXE Network Boot.

  3. Exit the node status table with CTRL+C.

  4. After the nodes boot, display detailed node information:

    cw-nodectl ls -L

    Running the command without specifying a compute node displays information about all compute nodes. For example:

    $ cw-nodectl ls -L
Nodes
 n0
  attributes
    _boot_config: DefaultBoot
    description:
    last_modified: 2026-06-05 02:38:29 UTC (13:46:48 ago)
    last_modified_by: admin1
    last_modified_on: de988a0a9082415481a93ef4047790b7
  domain: cluster.local
  hardware
    bios_date: 04/01/2014
    bios_vendor: SeaBIOS
    bios_version: rel-1.16.0-0-gd239552c-rebuilt.opensuse.org
    boot_style: rwram
    cpu_arch: x86_64
    cpu_count: 2
    cpu_model: AMD EPYC-Rome Processor
    last_modified: 2026-06-05 02:18:57 UTC (14:06:20 ago)
    last_modified_on: de988a0a9082415481a93ef4047790b7
    mac: 52:54:00:a6:f3:3c
    mode: BIOS
    product_name: None
    product_uuid: bf286b06-41fd-50f7-aca0-5bf8e0ca0c2a
    product_version: pc-q35-7.1
    ram_total: 7.7 GiB (8025724 KiB)
    storage
      /dev/vda
        size: 1024.0 MiB (1073741824 bytes)
      /dev/vdb
        size: 1024.0 KiB (1048576 bytes)
    vendor: KubeVirt
  hostname: n0
  index: 0
  ip: 10.54.60.0
  last_modified: 2026-06-05 02:18:57 UTC (14:06:20 ago)
  last_modified_on: head1-rocky9
  mac: 52:54:00:a6:f3:3c
  name: n0
  power_uri: none
  type: compute
 n1
  attributes
    _boot_config: DefaultBoot
    description:
    last_modified: 2026-06-05 02:38:29 UTC (13:46:48 ago)
    last_modified_by: admin1
    last_modified_on: de988a0a9082415481a93ef4047790b7
  domain: cluster.local
  hardware
    bios_date: 04/01/2014
    bios_vendor: SeaBIOS
    bios_version: rel-1.16.0-0-gd239552c-rebuilt.opensuse.org
    boot_style: rwram
    cpu_arch: x86_64
    cpu_count: 2
    cpu_model: AMD EPYC-Rome Processor
    last_modified: 2026-06-05 02:18:57 UTC (14:06:20 ago)
    last_modified_on: de988a0a9082415481a93ef4047790b7
    mac: 40:8d:5c:fa:ea:c3
    mode: BIOS
    product_name: None
    product_uuid: bf286b06-41fd-50f7-aca0-5bf8e0ca0c2a
    product_version: pc-q35-7.1
    ram_total: 7.7 GiB (8025724 KiB)
    storage
      /dev/vda
        size: 1024.0 MiB (1073741824 bytes)
      /dev/vdb
        size: 1024.0 KiB (1048576 bytes)
    vendor: KubeVirt
  hostname: n1
  index: 1
  ip: 10.54.60.1
  last_modified: 2026-06-05 02:18:57 UTC (14:06:20 ago)
  last_modified_on: head1-rocky9
  mac: 40:8d:5c:fa:ea:c3
  name: n1
  power_uri: none
  type: compute

    You can also display details for a single node or a set of nodes using the -i <nodes> argument. For example, the following command shows a subset of details for node n0:

    cw-nodectl -i n0 ls -l

Tip

You can also monitor compute node provisioning from the ClusterWareAI graphical user interface (GUI). Compute node status is available from the Overview and Nodes pages. See ClusterWareAI Graphical User Interface for details.

Configure Boot Image and Job Scheduler#

After initial installation, it is common to customize a compute node image by installing a job scheduler and additional software packages required for your end user workloads. The next sections create a customized compute node image with the Slurm job scheduler and an OpenMPI software stack installed.

Create a Custom Image#

The DefaultImage image and DefaultBoot boot configuration should be preserved as a baseline for future customizations. Start by cloning both so you can modify the copies instead.

  1. Clone the default image:

    cw-imgctl -i DefaultImage clone name=NewImage

  2. Clone the default boot configuration:

    cw-bootctl -i DefaultBoot clone name=NewBoot

    As a clone of DefaultBoot, NewBoot is associated with DefaultImage.

  3. Associate NewBoot with NewImage:

    cw-bootctl -i NewBoot update image=NewImage

  4. Configure all compute nodes to use NewBoot instead of DefaultBoot:

    cw-nodectl --all set _boot_config=NewBoot

Install and Configure Slurm#

Slurm is one of the compute node job schedulers supported by the ClusterWareAI platform. Slurm is commonly used for batch workloads, such as AI training, scientific simulations, or data analysis tasks.

Note

In a production cluster, the Slurm controller is installed on a separate server. This Quickstart installs the Slurm controller on the head node for simplicity.

  1. Configure head node hostname resolution by adding the head node to /etc/hosts and restarting the clusterware-dnsmasq service. For example, if the head node uses the private cluster network IP address 10.54.0.60:

    1. Add the head node to /etc/hosts:

      echo "10.54.0.60 $(hostname)" | sudo tee -a /etc/hosts

    2. Restart the clusterware-dnsmasq service:

      sudo systemctl restart clusterware-dnsmasq

    Slurm and other job schedulers require that all nodes in the cluster are able to resolve the hostnames of all other nodes as well as the scheduler server hostname. The ClusterWareAI software provides a DNS server in the clusterware-dnsmasq package.

  2. On the head node, install Slurm and explicitly enable the ClusterWareAI software repository, which was disabled during the install process.

    On Red Hat RHEL or CentOS 8:

    sudo dnf install slurm-cw --enablerepo=cw* --enablerepo=powertools

    On Red Hat RHEL or CentOS 9:

    sudo dnf install slurm-cw --enablerepo=cw* --enablerepo=crb

  3. Initialize Slurm:

    slurm-cw.setup init

  4. Add the Slurm client software to NewImage:

    slurm-cw.setup update-image NewImage

  5. Reboot the compute nodes to apply the image update:

    cw-nodectl --all reboot

  6. Monitor the compute nodes until they return to the up state:

    cw-nodectl status --refresh

  7. Exit the node status table with CTRL+C when all nodes are up.

Verify the Slurm Installation#

Make sure the Slurm services are running as expected and that the compute nodes are managed by Slurm.

  1. Check the job scheduler status:

    slurm-cw.setup status

    Both the Slurm daemon and the munge service should be running.

    If the Slurm daemon or munge service is not running:

    1. Restart the cluster services:

      slurm-cw.setup cluster-restart

    2. Check the job scheduler status again:

      slurm-cw.setup status

  2. Display the Slurm partition and node information:

    sinfo

Install OpenMPI in the Image#

Install OpenMPI and add an example OpenMPI application in NewImage.

  1. Add the administrator account and its SSH public key to NewImage:

    /opt/scyld/clusterware-tools/bin/sync-uids -i NewImage --create-homes \
    --users admin1 --sync-key admin1=/home/admin1/.ssh/id_rsa.pub

    OpenMPI uses SSH to enable multi-threaded communication between compute nodes. SSH requires matching user IDs (UID) and group IDs (GID) and passwordless key-based authentication.

  2. Open NewImage in a chroot environment:

    cw-modimg -i NewImage --freshen --chroot --no-discard --overwrite --upload

  3. Install OpenMPI from within the chroot environment:

    dnf install openmpi4.0

  4. Configure access to Slurm and OpenMPI for the admin1 account:

    echo "module load slurm" >> /home/admin1/.bashrc
echo "module load openmpi" >> /home/admin1/.bashrc

  5. Build an example OpenMPI application.

    1. Change to the OpenMPI examples directory:

      cd /opt/scyld/openmpi/*/gnu/examples

    2. Install make:

      dnf install make

    3. Load the OpenMPI environment module:

      module load openmpi

    4. Build the hello_c example application:

      make hello_c

    5. Copy the application to the admin1 home directory:

      cp hello_c /home/admin1/hello_c

  6. Exit the chroot environment:

    exit

Reboot the Compute Nodes#

Apply the updated image by rebooting the compute nodes.

  1. Reboot the compute nodes to load the updated image:

    cw-nodectl --all reboot

  2. Monitor the compute nodes until they return to the up state:

    cw-nodectl status --refresh

  3. Exit the node status table with CTRL+C when all nodes are up.

Verify the Software Stack#

Verify that the new image was deployed to the compute node by using Slurm to execute programs on the compute node.

  1. Load the Slurm environment module:

    module load slurm

  2. Verify that Slurm can run a command on both compute nodes:

    srun -N 2 hostname

  3. Use Slurm to run one instance of the OpenMPI example application on each compute node:

    srun -N 2 hello_c

Next Steps#

You now have a basic functional cluster with two compute nodes. You also created a new boot configuration and compute node image and deployed the image with additional software to your compute nodes.

The following are additional areas you may want to explore:

  • Interacting with Compute Nodes: Use the cw-nodectl tool to view and manage sets of compute nodes, set node attributes, and more.

  • Boot Configurations: Learn more about boot configuration options, including compute node boot style options.

  • Images: Additional details about how images are stored and deployed to compute nodes.

  • Attribute Groups: Use attribute groups to apply the same attributes to a set of compute nodes, such as the node boot style, boot configuration, or your own custom attributes.

  • Configure Job Schedulers: The ClusterWareAI software supports Slurm and OpenPBS job schedulers.

  • Configure Kubernetes: Use the ClusterWareAI software to manage and monitor worker nodes for supported Kubernetes distributions.

  • ClusterWareAI Health Monitoring System (CHMS): Add health checks to compute nodes to be alerted automatically about common issues, such as system errors or network connectivity problems.