Skip to content

Spack for managing software

Spack is a package management tool designed to support multiple versions and configurations of software on a wide variety of platforms and environments. It's designed for large supercomputing centers, where many users and teams share common installations of software on clusters with exotic architectures, using libraries that don't have a standard ABI. Spack is non-destructive: installing a new version doesn't break existing installations, so many configurations can coexist on the same system.

Most importantly, Spack is simple. It offers a simple spec syntax so that users can specify versions and configuration options concisely. Spack is also simple for package authors: package files are in pure Python, and specs allow package authors to maintain a single file for many different builds of the same package.

Spack documentation

UTHPC has been using Spack for almost three years to install software on the cluster.

Quick cheat sheet

  • spack install {package} - installs a package with default specs.
  • spack spec {package} - shows the whole dependency tree for a package.
  • spack list {package} - checks if a package is available for installation from the repositories.
  • spack compilers - displays a full list of available compilers.
  • spack find {package} - checks if a package is already installed, adding a --loaded flag lists all the loaded packages as well.
  • spack load {package} - loads a package and it's dependencies into your environment, adding --first loads the first one even if multiple specs are available.

Getting Spack

Using the preinstalled version

You can use Spack with module load spack. This loads the Spack commands and variables into your environment. The centrally installed Spack location is /gpfs/space/software/tools/global/spack.

This module sets the install path to ~/.spack/soft and modules reside in ~/.spack/modules. You can change these settings with configuration files in ~/.spack.

Using your own Spack instance

You can install your own instance of Spack if you desire more customization options. You can do it with:

git clone -c feature.manyFiles=true https://github.com/spack/spack.git

You would also need to run or put in your .bashrc file the command . spack/share/spack/setup-env.sh.

You can find more information at Spack documentation

Using Spack

Spack has around six thousand packages available to install from its base repository.

Most used commands

  • spack list {keyword} - to search for a package that you need.
  • spack find {keyword} - to see if a package is available.
  • spack spec {package}{spec} - to see what gets installed when you run spack install.
  • spack info {package} - shows information about the package.
  • spack install {package}{spec} - to install the package.
  • spack load {package}{spec} - to load the package in question.

Specs

You can leverage the massive modularity of Spack by specifying concrete specs to packages.

  • @ - specifies a package version, for example @1.2.3.
  • % - specifies the compiler, for example %gcc@9.2.0.
  • ^ - specifies a dependency, for example py-mpi4py ^mpich installs ’mpi4py’ with ’mpich’.
  • / - specifies an hash to use, for example spack load /asfoe4.
  • + and ~ - specifies variants, for example spack install {package} +mpi ~gpu installs the package with ’mpi’ and without ’gpu’ support.
  • key=value - optional variant specifiers.

Why use Spack load instead of module load?

While Lmod with its module works really easily for most users, but it hits a customizability cap quite fast. You can run module load {package} and that's it but what if your workflow improves and you need the same package with a different compiler for example. Putting those changes into module file names is making them unnecessarily long and complicated for everyone involved.

In comes spack load which directly leverages the package manager to get exactly how little or much you need. You can either just run spack load --first {package} or specify the full spec or even a hash. It's also able to load dependencies directly and not assume that they're loaded correctly. This also reduces headache for admins because they don't need to figure out complex systems on how to name modules.

If you need to port your applications/software usage to another user or cluster then Spack gives the best reproducibility for packages.

Creating your own package installations

You can create your own installer files called package.py if you can't find them in the base repository. It needs some knowledge about Spack but the documentation is pretty thorough. You can place created packages in the $spack_home/var/spack/repos/builtin/packages directory.

Issues with Spack

If you encounter any issues or need a specific package installed then don't hesitate to write at support@hpc.ut.ee . You can also request a package installation via our application form .