Compute Nodes IPMI Access#

ipmitool is a hardware management utility that supports the Intelligent Platform Management Interface (IPMI) specification v1.5 and v2.0.

IPMI is an open standard that defines the structures and interfaces used for remote monitoring and management of a computer motherboard (baseboard). IPMI defines a micro-controller, called the "baseboard management controller" (BMC), which is accessed locally through the managed computer's bus or through an out-of-band network interface connection (NIC).

The root can use ipmitool for a variety of tasks, such as:

  • Inventory a node's baseboards to determine what sensors are present

  • Monitor sensors (fan status, temperature, power supply voltages, etc.)

  • Read and display values from the Sensor Data Repository (SDR)

  • Read and set the BMC's LAN configuration

  • Remotely control chassis power

  • Display the contents of the System Event Log (SEL), which records events detected by the BMC as well as events explicitly logged by the operating system

  • Print Field Replaceable Unit (FRU) information, such as vendor ID, manufacturer, etc.

  • Configure and emulate a serial port to the baseboard using the out-of-band network connection known as serial over LAN (SOL)

Several dozen companies support IPMI, including many leading manufacturers of computer hardware. You can learn more about OpenIPMI from the OpenIPMI project page at http://openipmi.sourceforge.net, which includes links to documentation and downloads.

The node's power_uri field in the database is optional and informs the head node(s) how to control the power to a given node. A plugin interface allows for different forms of power control, currently supporting IPMI for bare metal nodes, and KVM (virsh) or VirtualBox (vbox) for different types of virtual nodes. For example, a power_uri for a VirtualBox virtual node might be:

vbox://192.168.56.1/CW_Compute0

Production system compute nodes are generally bare-metal nodes that can be controlled via the ipmitool command that communicates with the node's Baseboard Management Controller (BMC) interface. Set a power_uri with the appropriate BMC IP address and username/password access credentials for these nodes. For example:

ipmi:///admin:password@172.45.88.1

With power_uri, the head node communicates with that compute node's BMC located at 172.45.88.1 using the username "admin" and password "password" to perform a scyld-nodectl power on, power off, power cycle, shutdown --hard, or reboot --hard.

If for any reason only a specific remote machine can execute ipmitool to control a node, then add that server name, and an optional username and password, to the power_uri. The local head node will ssh to that remote server and execute the ipmitool command from there. For example, the power_uri:

ipmi://remote_server/admin:password@172.45.88.1

sends the ipmitool command details to server "remote_server" for execution.

The scyld-nodectl "soft" shutdown --soft and reboot --soft commands do not use the power_uri. Rather, they ssh to the compute node to execute the local /usr/sbin/shutdown or /usr/sbin/reboot command with appropriate arguments. A simple scyld-nodectl -i <NODE> reboot (or shutdown) first attempts a "soft" action if the node is "up" and the head node can communicate with the node. If the "soft" action is not possible, or does not complete within a reasonable time, then the scyld-nodectl resorts to a "hard" action using the power_uri connection.