ARS Logging#

Compute node health check failures, ARS remedy suggestions, and remedy execution outcomes are published to MQTT and to the journalctl logs on the head node.

ARS MQTT Logging#

ARS log messages are emitted on MQTT topic ars/v1/logs/state_machine and ars/v1/logs/auto_remediation.

The state machine MQTT topic events and messages include compute nodes transitioning between ARS states (provisioning, available, auto remediation, and so on).

The auto remediation MQTT topic events and messages include:

  • List of health checks failing on a compute node and the suggested remedies for those checks

  • Which remedy was selected and executed on a compute node for failed health checks

You can subscribe to topics using an MQTT client or set up Alerting for the topics to a selected output target, such as Jira or email. See Auditing for more details about MQTT.

ARS journalctl Logging#

The journalctl log files include details such as which health checks failed and which remedy was selected for each compute node. View the log by running the following on the head node:

journalctl --unit ars-auto-remediation

In a cluster with multiple head nodes, run the command on the compute node's parent head node to view node-specific logs. See Booting with Multiple Head Nodes for details about determining parent head node.