Slurm REST API Configuration

Basic setup guide for slurmrestd with the HPC Dashboard

Slurm REST API Configuration

This guide provides a quick overview of setting up the Slurm REST API (slurmrestd) for use with the HPC Dashboard.

Quick Installation

Install Required Packages

Debian/Ubuntu:

sudo apt install slurm-smd slurm-smd-slurmrestd

CentOS/RHEL:

sudo yum install slurm slurm-slurmrestd

Create Service Account

sudo useradd -M -r -s /usr/sbin/nologin -U slurmrestd

Configure Service

Edit the service configuration:

vim /usr/lib/systemd/system/slurmrestd.service

Add:

[Unit]
Description=Slurm REST daemon
After=network-online.target remote-fs.target slurmctld.service
Wants=network-online.target
ConditionPathExists=/etc/slurm/slurm.conf

[Service]
Type=simple
EnvironmentFile=-/etc/sysconfig/slurmrestd
EnvironmentFile=-/etc/default/slurmrestd
# slurmrestd should not run as root or the slurm user.
# Please either use the -u and -g options in /etc/sysconfig/slurmrestd or
# /etc/default/slurmrestd, or explicitly set the User and Group in this file
# an unpriviledged user to run as.
User=slurmrestd
Group=slurmrestd
ExecStart=/usr/sbin/slurmrestd $SLURMRESTD_OPTIONS
# Enable auth/jwt be default, comment out the line to disable it for slurmrestd
Environment=SLURM_JWT=daemon
# Listen on TCP socket by default.
Environment=SLURMRESTD_LISTEN=:6820
ExecReload=/bin/kill -HUP $MAINPID

[Install]
WantedBy=multi-user.target

Start Service

sudo systemctl start slurmrestd
sudo systemctl enable slurmrestd

Generating API Keys

The Slurm REST API uses JWT tokens for authentication:

  1. Get a token (expires in 30 minutes by default, this is an example for one year)
scontrol token username=slurm lifespan=31536000

Security Consideration:

The token above has a 1-year lifespan. Consider setting up a renewal process or using a shorter timeframe for better security.

Testing the Connection

Test that the API is working:

curl -s -k \
-H "X-SLURM-USER-TOKEN:$SLURM_JWT" \
-X GET "http://localhost:6820/slurm/v0.0.40/diag"

Replace v0.0.40 with the appropriate version for your Slurm installation.

Quick Troubleshooting

Connection Issues

If you can't connect to the API:

  1. Check if slurmrestd is running:

    systemctl status slurmrestd

  2. Verify it's listening on the expected port:

    sudo ss -tlnp | grep slurmrestd

Authentication Problems

If you get authentication errors:

  1. Generate a fresh token:

    scontrol token username=slurm lifespan=31536000

  2. For debugging, enable more verbose output:

    export SLURMRESTD_DEBUG=debug

Dashboard Integration

To connect the HPC Dashboard to your Slurm REST API:

  1. Update your .env file:

    SLURM_API_VERSION="v0.0.40"
SLURM_SERVER="192.168.1.5"
SLURM_API_TOKEN=""
SLURM_API_ACCOUNT=""

  2. In a production environment, configure proper SSL/TLS and firewall rules to secure the API endpoint.

Additional Resources

For more detailed information, refer to the official Slurm documentation: