DPUs#

The ClusterWareAI ™ software supports data processing units (DPUs) associated with existing compute nodes. DPUs are managed using the cw-dpuctl tool and have a "host" compute node linked.

Most actions you can take on compute nodes can also be taken on DPUs. For example, you can deploy images and PXE boot DPUs managed by the ClusterWareAI software. Soft power control actions, such as reboot or shutdown, are taken directly on the DPU. Other power control actions, such as power on, power off, or power cycle, are taken on the host compute node associated with the DPU even if the action is run using the cw-dpuctl tool.

ClusterWareAI DPU support is available for NVIDIA BlueField-3 DPUs with DOCA. Additional support may be added in future releases. Contact Penguin Computing to discuss alternative requirements.

Creating DPUs#

Use the cw-dpuctl tool to create a DPU. Specify a MAC address and a compute node host when creating a DPU. For example:

cw-dpuctl create mac=11:11:11:22:22:22 host=n0

DPUs, like compute nodes and switches, are automatically added to the DefaultAttribs attribute group and use the nX naming pool pattern. To differentiate DPUs from compute nodes and switches, consider using a different naming pool for DPUs. See Node Names and Pools for details and Switch Naming Pools for an example.

Booting DPUs#

Secure boot on DPUs is not supported. Disable secure boot before booting a DPU with the ClusterWareAI software.

Disable secure boot via BIOS or Redfish. For example:

curl -k -u name:password -X PATCH -H 'Content-Type: application/json' \
  "https://dpu-bmc0/redfish/v1/Systems/Bluefield/SecureBoot" \
  -d '{"SecureBootEnable": false}'

Reboot the DPU:
```
cw-dpuctl -i <DPU> reboot
```
The system may require multiple reboots before the change takes effect.

Verify the current state of the DPU by comparing the SecureBootCurrentBoot and SecureBootEnabled fields:

[admin@head0 ~]$ curl -k -u name:password "https://dpu-bmc0/redfish/v1/Systems/Bluefield/SecureBoot"
{
  "@odata.id": "/redfish/v1/Systems/Bluefield/SecureBoot",
  "@odata.type": "#SecureBoot.v1_1_0.SecureBoot",
  "Description": "The UEFI Secure Boot associated with this system.",
  "Id": "SecureBoot",
  "Name": "UEFI Secure Boot",
  "SecureBootCurrentBoot": "Enabled",
  "SecureBootDatabases": {
    "@odata.id": "/redfish/v1/Systems/Bluefield/SecureBoot/SecureBootDatabases"
  },
  "SecureBootEnable": false,
  "SecureBootMode": "UserMode"
}

When working with DPUs, you may need to adjust the kernel command line on the boot configuration to allow the DPU OS to start. For example:

cw-bootctl -i DpuBoot update cmdline="selinux=0 ro console=hvc0 console=ttyAMA0 \
   earlycon=pl011,0x13010000 fixrtc net.ifnames=0 biosdevname=0 iommu.passthrough=1"

Updating the cmdline overwrites the existing cmdline, so make sure to include existing commands if needed.

DPU Images#

Like compute nodes, DPUs support deploying images via an associated boot configuration (DefaultBoot and DefaultImage by default). Alternatively, you can use Ignition or Kickstart to deploy DPU images.