jupyter.hpc.ut.ee¶

jupyter.hpc.ut.ee is UTHPC centre's JupyterHub.

JupyterHub is a multi-user Jupyter server that provides a web-based login and spawns your own single-user server with JupyterLab.

Getting started¶

JuputerHub is available not only within UT network, but also from the rest of the Internet at jupyter.hpc.ut.ee with your UT credentials. To use JuputerHub, you have to have UTHPC account.

To start your single-user server, you have to choose a profile. Currently, there are 5 available options:

Your single-user server spins up a job in Slurm queue on the UTHPC cluster and it might take sometime to start depending on the resource availability. The started single-user server stays up until either you stop it manually or the time limit kicks in.

As Slurm manages the resources for your single-user server. Be aware if you go over the memory limit, your server is automatically killed without warning. You can view the error messages in the corresponding error file ~/jupyter.xxx.log> (’xxx’ represents a Slurm job id).

Stopping a single-user server¶

Use ’Stop My Server’ button in ’Hub Control Panel’ to shut down your server. Navigate to ’File’ ’Hub Control Panel’. It takes a bit of time to shut down your running server.

Usage¶

Default software and kernels¶

It has preinstalled Python 3.10.1 and R 3.6.3 kernels.

Install packages to R / Python kernel¶

Jupyter users share both R/Python kernels and that's why it's not allowed to install add-on packages directly into kernels. However, you can install packages to your local $HOME.

Regarding python packages, please read section below: Add a custom python/conda environment . Once your custom conda/python kernel created, you proceed installing packages into the virtual environment.

You can install R packages from within JupyterLab following these steps:

1. Open Terminal and create library directory, then close the terminal.

   mkdir -p ~/local/R_libs
   

2. Open Notebook with R kernel, and install packages to the created dir:

   install.packages("<package name>", lib="~/local/R_libs/")
   

3. To use the package, Call the function library and specify the same lib option:

   library("<package name>", lib="~/local/R_libs/")
   

Add a custom python/conda environment¶

Step 1 - activate your environment¶

#python env
source <user_env>/bin/activate

#conda env
conda activate <user_env>

Having activated the env, installs all necessary packages. Then set the env onto your JupyterLab.

Step 2 - install ipykernel to your python/conda environment¶

#python env
python -m pip install ipykernel

#conda env
conda install ipykernel

Step 3 - link the installed ipykernel with your environment¶

python -m ipykernel install --user --name=<user_env>

Step 4 - load your environment or kernel¶

Customize kernels¶

In order for to work in custom kernel, you have to modify a kernel file of your environment.

1. On rocket.hpc.ut.ee machine, your $HOME has your environment kernel file called kernel.json, which is usually located in ~/.local/share/jupyter/kernels/<your env name>/kernel.json or can be found with jupyter-kernelspec list command

2. Secondly, activate your python/conda env then activate the required modules, after that echo $PATH, $LIBRARY_PATH, $LD_LIBRARY_PATH with the following command:

   echo $PATH
   echo $LIBRARY_PATH
   echo $LD_LIBRARY_PATH
   

   Copy the output from the previous commands into your clipboard.

3. Finally, open kernel.json and copy this snippet into kernel.json and substitute <your env bin path> with paths obtained in the step 2 into kernel.json file:

   "env": {
       "PATH": "<your env PATH from step 2>",
       "LD_LIBRARY_PATH": "<your env LD_LIBRARY from step 2>",
       "LIBRARY_PATH": <your env LIBRARY_PATH from step 2>"
   }
   

   If necessary, add a comma at the end of the previous line and ensure } is present, as json requires structure.

4. Save the file and restart Jupyter if you have it running already.

At the end, your kernel.json should look something like:

{
 "argv": [
  "/gpfs/space/home/user/.conda/envs/user_env/bin/python",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "user_env",
 "language": "python",
 "metadata": {
  "debugger": true
 },
 "env": {
  "PATH": "/gpfs/space/home/user/.conda/envs/user_env/bin:/gpfs/space/software/jupyterhub/python/jupyter/bin:/gpfs/software/soft/rocket/linux-centos7-x86_64/gcc-9.2.0/openssl-1.1.1g-pikmcqfnphdwr5n32225b7rf4fpksnr2/bin:/sbin:/bin:/usr/sbin:/usr/bin",
  "LD_LIBRARY_PATH": "/gpfs/space/home/user/.conda/envs/user_env/lib:/gpfs/space/software/jupyterhub/python/jupyter/lib" 
  "LIBRARY_PAHT": "/gpfs/space/home/user/.conda/envs/user_env/lib:/gpfs/space/software/jupyterhub/python/jupyter/lib"
 }
}

Problems and requests¶

This service is currently in beta and under development. If you notice problems or would like any more extensions or features, then please write to support@hpc.ut.ee .

Please provide the following details:

• Error details from the jupyter.xxxxxx.log> file.
• Slurm job id.

FAQ¶

See also¶

• https://jupyter.org .
• Jupyter Documentation .
• Demos and tutorial: https://jupyter.org/try .