Cluster quick start¶

This section is a quick start for someone who is somewhat familiar with Linux/Unix and is looking at how to start using the University of Tartu UTHPC clusters quickly.

If you're not familiar with UTHPC and Linux in general, many beginner tutorials are available online. UTHPC team is also actively working on providing a better guides.

Request an account¶

In order to open an account with UTHPC, please fill out the form here to ensure that UTHPC team receives all the necessary information to quickly create the account.

Alternatively you can email your request to support@hpc.ut.ee . If you've already got a UT account, please provide your username with the email. If you are a student, you must also CC your supervisor in the email.

Galaxy users¶

To request an access to Galaxy Tartu Ülikool , please fill out the form here to assure a quick response. Alternatively you can send an email to galaxy@hpc.ut.ee but please include also your UT account username.

Cluster login¶

ssh <username>@rocket.hpc.ut.ee

Your home directory¶

The home directory, which makes all files and directories available on all cluster nodes, resides on a shared file system called GPFS.

Quotas manage the Disk space consumption. There are two types of quotas - directory size and file count. By default, a user has 2 TB of $HOME space and a maximum file count of 1 million files.

To see your quota, you can use the myquota command to see your maximum limits and current usage.

Copy data¶

There are multiple ways to transfer files between a local machine and the cluster, mainly depending on your local operating system. For a Unix-like OS, you can use scp, rsync, sftp commands on the command line. If you are on Windows or prefer a Graphical User Interface, FileZilla is one of a tools that you can use.

To copy data to the cluster from your local machine, use the secure copy command scp:

scp /path/to/file <username>@rocket.hpc.ut.ee:/path/to/target_dir/

To retrieve data from the cluster to your local machine:

scp <username>@rocket.hpc.ut.ee:/path/to/file /path/to/target_dir/

Using software¶

You can make use of already pre-installed software, or you can compile and install software on your own. UTHPC uses an environment module system to make software and specific versions available to users:

For example on searching and loading ’python’ software.

1. Check the available ’python’ versions on cluster:

   module av python
   

2. Load the desired version of ’python’:

   module load python/3.14.0
   

3. List loaded modules:

   module list
   

Usually you would have a good idea about what software you need beforehand. For this guide we will use a few Python modules (numpy, scipy, matplotlib) and a Python script to generate a 3D graph of terrain.

[user@login1 ~]$ module av py-scipy py-numpy py-matplotlib

---------------- /gpfs/space/software/cluster_software/modules/spack/linux-rhel9-x86_64/Core ------------------------
   py-matplotlib-inline/0.1.6 (D)    py-matplotlib/3.7.1    py-numpy/1.22.0    py-numpy/1.24.3    py-scipy/1.10.1
   py-matplotlib/3.5.3               py-numpy/1.18.5        py-numpy/1.23.5    py-scipy/1.5.4

[user@login1 ~]$ module load py-numpy py-scipy py-matplotlib
[user@login1 ~]$ module list

Currently Loaded Modules:
  1) glibc/2.34            6) py-kiwisolver/1.4.8  11) py-six/1.16.0                   16) openblas/0.3.30       21) readline/8.3          26) python/3.14.0
  2) py-pybind11/3.0.1     7) py-packaging/25.0    12) py-python-dateutil/2.8.2        17) expat/2.7.3           22) sqlite/3.50.4         27) python-venv/1.0-shofoqw (H)
  3) py-contourpy/1.3.3    8) zlib-ng/2.2.4        13) qhull/2020.2                    18) libffi/3.5.2          23) util-linux-uuid/2.41  28) py-numpy/2.3.4-h6jggxi  (H)
  4) py-cycler/0.12.1      9) py-pillow/12.0.0     14) py-matplotlib/3.10.7            19) ncurses/6.5-20250705  24) xz/5.6.3              29) py-scipy/1.17.0
  5) py-fonttools/4.39.4  10) py-pyparsing/3.2.5   15) gcc-runtime/12.5.0-krjyywc (H)  20) openssl/3.6.0         25) zstd/1.5.7

You now have the modules loaded. Modules make multiple changes to your current environment, most notably the $PATH variable.

Testing your job before submission¶

If you want to check whether your software starts executing before committing time and resources to it, you have an option to interactively test it.

You can open an interactive shell that is running inside a Slurm job. To achieve this, run the following command:

srun -A <accountname> --partition=testing -t 10:00 --cpus-per-task=4 --mem=16G --pty bash

• starts a Slurm process
• with the account (this is required for ETAIS users)
• in the partition testing
• with a Timelimit of 10 minutes
• with 4 cpus
• with 16G of memory
• the command will be executed in a pseudoterminal
• the command to execute will be bash

This command allocates resources on a compute node and once done, you will have an interactive terminal session where you can run the commands you need - like the module load, and run your software. When you continue with the guide to the First job section, everything you do in the terminal can be applied to the #your code goes here section in the job script.

For more information please check out the interactive jobs guide

First job¶

The cluster utilizes a scheduler called Slurm to control job execution and distribute running jobs across available physical resources like memory and CPU cores.

The following is an example of how to run your first job. Before we write the job script for Slurm, we first need to write the Python script that Slurm will run. We will write this into the home directory as terrain.py:

[user@login1 ~]$ vim terrain.py

import numpy as np
import scipy.ndimage
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D

width = 512
height = 512
scale = 0.0025
octaves = 6
persistence = 0.5

terrain = np.random.rand(height, width)
terrain = scipy.ndimage.gaussian_filter(terrain, sigma=30)

x, y = np.meshgrid(np.arange(width), np.arange(height))
fig = plt.figure(figsize=(20, 20))
ax = fig.add_subplot(projection='3d')
terrain_plot = ax.scatter(x, y, terrain, c=terrain, cmap='terrain', marker='.')

plt.savefig("test-terrain-gen.png", dpi=400, bbox_inches="tight", pad_inches=0)

print("Successfully generated random terrain!")
print('File written to home directory: test-terrain-gen.png')

Next, we write the job script. A job script (sbatch file) consists of two main parts - instructions for the scheduler and the actual commands to run for the job, which operate your choice of software.

Start with the scheduler instructions:

#!/bin/bash
#SBATCH -J hello_world
#SBATCH --partition=testing
#SBATCH -t 00:10:00
#SBATCH --cpus-per-task=2
#SBATCH --mem=4GB

# your code goes below

#!/bin/bash
#SBATCH -J hello_world
#SBATCH --partition=testing
#SBATCH --account="ealloc_905b0_something"
#SBATCH -t 00:10:00
#SBATCH --cpus-per-task=2
#SBATCH --mem=4GB

# your code goes below

Then add the part for loading software and running a command:

module purge
module load py-numpy/2.3.4 py-scipy/1.17.0 py-matplotlib/3.10.7

python terrain.py

The module purge here will clear the currently loaded modules and start from a clean environment. The modules are also pinned to specific versions for better reproducability.

The finalized job script looks like this:

#!/bin/bash
#SBATCH -J hello_world
#SBATCH --partition=testing
#SBATCH -t 00:10:00
#SBATCH --cpus-per-task=2
#SBATCH --mem=4G

# your code goes below
module purge
module load py-numpy/2.3.4 py-scipy/1.17.0 py-matplotlib/3.10.7

python terrain.py

You should save it into a file, for example generate_terrain.sh.

This job will run in the same directory that the job script is in and will generate an image as output.png. To download and view the image, refer to the data uploading section above.

As you can see, the script is basically a bash script which means that you can do a lot of tricks in it.

Submit your job¶

Once you have a job definition script, you can submit your job script to the scheduler. Scheduler allocates the requested resources for your job and give you a job id. If the requested resources are available, your job start immediately. Otherwise, the job stays in queue until sufficient resources are available.

To submit your job to Slurm, use the sbatch command:

sbatch generate_terrain.sh

Submitted batch job 15304092

The job will run for about half a minute. During the run time you can continue with the guide to see how to monitor it.

When the job is done you should see a file called output.png in the same folder as the job script. To retrieve it from the cluster you can run scp on your local machine:

scp <username>@rocket.hpc.ut.ee:output.png .

There are various options for different kinds of jobs in cluster. Please review the following sections for more information Submitting Jobs , GPU Computing , Interactive Jobs for more information.

Monitor your job¶

You can inspect the status of your running jobs with the squeue command:

squeue -j 15304092

JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
15304092   testing generat    test_user  R       0:10      1 stage43

Be aware, that if the requested resources aren't available, the job status is ’PENDING’ (PD). The job is in the queue, and starts as soon as the requested resources are available.

You can also see all active submitted jobs with squeue:

squeue -u <test_user>

Cancel your job¶

You can cancel your job via the scancel command by passing the job ID as an argument.

scancel 15304092