Configure Kubernetes with the ICE ClusterWare Container Registry

After installing Kubernetes, configure Kubernetes with the ICE ClusterWare ™ container registry and DaemonSet. These instructions assume you have installed Kubernetes on control plane and worker nodes with an immutable operating system (OS). Using a container registry is required for Harvester clusters. Other Kubernetes distributions can also use the container registry, but installing the clusterware-node package is supported and recommended.

Prerequisites

• All Kubernetes control plane and worker nodes must use the IP address of the head node(s) as their DNS server.

• Each head node requires local storage at least three times the size of the containers in the registry. If you are only using the default container image provided by the ClusterWare software (<1 GB), the recommended minimum of 100 GB per head node should be sufficient. If you are planning to upload your own container images, consider increasing head node local storage accordingly. See Required and Recommended Components for details.

  Shared storage for the registry is not currently supported. If you use shared storage for your head nodes and want to use Kubernetes with containers, you can install your own registry and add the provided ClusterWare container to your registry.

Configure Kubernetes Registry and Deploy Container

1. Add the list of worker nodes to the ClusterWare software with MAC and IP addresses. Often this is done by creating a JSON file and providing it with the --content argument. For example:

   1. Create a JSON file that contains an array of JSON objects where each object describes a single node. For example:

      [
       { "mac": "00:11:22:33:44:55", "index": 1, "ip": "192.168.0.1" },
       { "mac": "00:11:22:33:44:66", "index": 2, "ip": "192.168.1.1" },
       { "mac": "00:11:22:33:44:77", "index": 3, "ip": "192.168.2.1" }
      ]
      

   2. Include the path to the JSON file when creating the nodes:

      cw-nodectl create --content @file.json
      

   See Node Creation with Known MAC address(es) and The --content argument for additional options and examples.

2. Save a copy of the KubeConfig file that has administrator-level access to your Kubernetes environment to a location accessible by the ClusterWare administrator. Refer to instructions for your distribution for details. For example:

   • RKE2: https://docs.rke2.io/cluster_access

   • Harvester: https://docs.harvesterhci.io/v1.5/faq#how-can-i-access-the-kubeconfig-file-of-the-harvester-cluster

3. [Harvester Clusters Only] Run the following command from a Kubernetes control plane with the clusterware-kubeadm package installed to configure Harvester to run images from the ClusterWare registry and create the DaemonSet for the container:

   /opt/scyld/clusterware-kubeadm/bin/cw-deploy-kube-agents --harvester-registry-configure –label
   

4. [Harvester Clusters Only] Reboot the Kubernetes cluster to apply registry changes.

5. The ClusterWare node container starts on the configured list of worker nodes automatically. The container allows the ClusterWare to manage and monitor the compute nodes. Make sure the nodes are up and available using one of the following methods:

   • Running cw-nodectl status

   • Viewing the nodes in the ClusterWare GUI

   • Viewing the nodes in Grafana

After configuring Kubernetes with the ClusterWare container registry, you can add or remove your own containers from the registry. See Manage the ICE ClusterWare Container Registry for details.