Alert Endpoints#

An alert primitive represents a notification from the ClusterWareAI ™ software to a third-party product, such as Slack, Teams, or email. Use alerts to receive timely notification of events that matter to cluster operations, such as node creation, ARS state changes, image updates, or other software events. Alerts define which event sources are monitored, the notification target and configuration to use, and the severity assigned to the notification.

Alert targets must be configured with credentials or permissions that allow ClusterWareAI to post to the target platform. Consult the target platform documentation for any additional requirements.

When issuing requests, the UID field in the URL can be either the actual UID of the primitive or the name of the primitive as given in the name field. For example, an alert named node_created_slack can be referenced through /alert/node_created_slack or /alert/<UID>.

Data Fields#

Alert primitives have several fields:

name
   Required: The name of the alert. Names must start with an alphabet
   character, not a number.

description
   Optional: A text description for the alert.

sources
   Required: A non-empty list of source strings. Each source must be of the
   form ``mqtt:<topic>`` or ``mutator:<ident>``.

output
   Required: The notification provider target type, such as ``slack`` or
   ``email``. The value must identify a known supported target provider.

output_config
   Required: A JSON object containing the configuration required by the
   selected ``output`` provider. The runtime ``message`` field is supplied
   when the alert fires and is not supplied in ``output_config``.

severity
   Optional: The alert severity. Supported values include ``DEBUG``, ``INFO``,
   ``WARNING``, ``ERROR``, and ``CRITICAL``.

enabled
   Optional: A True/False value indicating whether the alert is active. Set
   to ``false`` to disable the alert without deleting it.

mute
   Optional: A True/False value that stops the alert from firing. The
   default is ``false`` for a newly created alert. When cloning, the clone
   retains the template alert's ``mute`` value unless overridden. ``true``
   with no ``mute_until`` mutes the alert indefinitely.

mute_until
   Optional: A Unix timestamp at which a timed mute expires. The alert
   stays muted while ``mute`` is ``true`` and this timestamp is in the
   future.

mute_for
   Optional: A positive duration such as ``30m``, ``1h``, or ``2d``. It
   sets ``mute`` to ``true`` and calculates ``mute_until`` from the current
   time. ``mute_for`` is not stored.

frozen
   Optional: A True/False indicator that allows admins to block changes to
   the alert.

Additional Endpoints#

The alert API provides the following endpoints:

GET /alerts
   Returns a list of all alerts.

POST /alerts
   Creates an alert.

POST /alerts?template=<ALERT>
   Creates an alert by cloning an existing alert (by name or UID). The
   request body must include a new ``name`` and may include field overrides,
   such as severity or frozen.

GET /alert/<ALERT>
   Returns details about a given alert (by name or UID).

PATCH /alert/<ALERT>
   Updates an existing alert (by name or UID).

DELETE /alert/<ALERT>
   Deletes an alert (by name or UID).

Example#

Create an alert that sends node creation notifications to Slack:

curl -X POST https://head1.cluster.local/api/v1/alerts \
     -H "Authorization: Bearer <access_token>" \
     -H "Content-Type: application/json" \
     --data '{
       "name":"node_ars_provision_fail_slack",
       "description":"Notify Slack when a node fails provisioning.",
       "sources":["mqtt:ars/v1/+/events/provisioning_error"],
       "output":"slack",
       "output_config":{"webhook_url":"https://slack.example/services/T000/B000/XXXX"},
       "severity":"INFO",
       "enabled":true
     }'
{"success":true,"data":"341b9947f6e644b78f6d88f5d7f898f4"}

Get alert details by name:

curl -X GET https://head1.cluster.local/api/v1/alert/node_ars_provision_fail_slack \
     -H "Authorization: Bearer <access_token>" \
     -H "Content-Type: application/json" | jq
{
  "success": true,
  "data": {
    "name": "node_ars_provision_fail_slack",
    "description": "Notify Slack when a node fails provisioning.",
    "sources": [
      "ars/v1/+/events/provisioning_error"
    ],
    "output": "slack",
    "output_config": {
      "webhook_url": "https://slack.example/services/T000/B000/XXXX"
    },
    "severity": "INFO",
    "enabled": true,
    "uid": "341b9947f6e644b78f6d88f5d7f898f4"
  }
}

Mute an alert for 30 minutes:

curl -X PATCH https://head1.cluster.local/api/v1/alert/node_ars_provision_fail_slack \
     -H "Authorization: Bearer <access_token>" \
     -H "Content-Type: application/json" \
     --data '{"mute_for":"30m"}'
{"success":true}

Clear a mute:

curl -X PATCH https://head1.cluster.local/api/v1/alert/node_ars_provision_fail_slack \
     -H "Authorization: Bearer <access_token>" \
     -H "Content-Type: application/json" \
     --data '{"mute":false}'
{"success":true}

Clone an existing alert and set a different severity on the new alert:

curl -X POST "https://head1.cluster.local/api/v1/alerts?template=node_ars_provision_fail_slack" \
     -H "Authorization: Bearer <access_token>" \
     -H "Content-Type: application/json" \
     --data '{"name":"node_ars_provision_fail_slack_critical","severity":"CRITICAL"}'
{"success":true,"data":"a914cfbf1ccf4e9a9fe23921a3ebf7da"}