jupyter.hpc.ut.ee¶

jupyter.hpc.ut.ee is UTHPC centre's JupyterHub.

JupyterHub is a multi-user Jupyter server that provides a web-based login and spawns your own single-user server with JupyterLab.

Getting started¶

JuputerHub is available not only within UT network, but also from the rest of the Internet at jupyter.hpc.ut.ee with your UT credentials. To use JuputerHub, you have to have UTHPC account.

To start your single-user server, you have to choose a profile. Currently, there are 5 available options:

Your single-user server spins up a job in Slurm queue on the UTHPC cluster and it might take sometime to start depending on the resource availability. The started single-user server stays up until either you stop it manually or the time limit kicks in.

As Slurm manages the resources for your single-user server. Be aware if you go over the memory limit, your server is automatically killed without warning. You can view the error messages in the corresponding error file ~/jupyter.xxx.log> (’xxx’ represents a Slurm job id).

Stopping a single-user server¶

Use ’Stop My Server’ button in ’Hub Control Panel’ to shut down your server. Navigate to ’File’ ’Hub Control Panel’. It takes a bit of time to shut down your running server.

Usage¶

Default software and kernels¶

It has preinstalled Python 3.10.1 and R 3.6.3 kernels.

Installing Packages for R/Python Kernels¶

Jupyter users share both R and Python kernels, which is why it's not allowed to install add-on packages directly into kernels. However, you can install packages in your local $HOME directory.

For Python packages, please refer to the section below: Adding a Custom Python/Conda Environment . Once your custom Conda/Python kernel is created, you can proceed to install packages into the virtual environment.

Installing R Packages¶

Add a custom python/conda environment¶

Step 1 - activate your environment¶

#python env
source <user_env>/bin/activate

#conda env
conda activate <user_env>

Having activated the env, installs all necessary packages. Then set the env onto your JupyterLab.

Step 2 - install ipykernel to your python/conda environment¶

#python env
python -m pip install ipykernel

#conda env
conda install ipykernel

Step 3 - link the installed ipykernel with your environment¶

python -m ipykernel install --user --name=<user_env>

Step 4 - load your environment or kernel¶

Customize kernels¶

In order for to work in custom kernel, you have to modify a kernel file of your environment.

1. On rocket.hpc.ut.ee machine, your $HOME has your environment kernel file called kernel.json, which is usually located in ~/.local/share/jupyter/kernels/<your env name>/kernel.json or can be found with jupyter-kernelspec list command

2. Secondly, activate your python/conda env then activate the required modules, after that echo $PATH, $LIBRARY_PATH, $LD_LIBRARY_PATH with the following command:

   echo $PATH
   echo $LIBRARY_PATH
   echo $LD_LIBRARY_PATH
   

   Copy the output from the previous commands into your clipboard.

3. Finally, open kernel.json and copy this snippet into kernel.json and substitute <your env bin path> with paths obtained in the step 2 into kernel.json file:

   "env": {
       "PATH": "<your env PATH from step 2>",
       "LD_LIBRARY_PATH": "<your env LD_LIBRARY from step 2>",
       "LIBRARY_PATH": <your env LIBRARY_PATH from step 2>"
   }
   

   If necessary, add a comma at the end of the previous line and ensure } is present, as JSON requires structure.

4. Save the file and restart Jupyter if you have it running already.

At the end, your kernel.json should look something like:

{
 "argv": [
  "/gpfs/space/home/user/.conda/envs/user_env/bin/python",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "user_env",
 "language": "python",
 "metadata": {
  "debugger": true
 },
 "env": {
  "PATH": "/gpfs/space/home/user/.conda/envs/user_env/bin:/gpfs/space/software/jupyterhub/python/jupyter/bin:/gpfs/software/soft/rocket/linux-centos7-x86_64/gcc-9.2.0/openssl-1.1.1g-pikmcqfnphdwr5n32225b7rf4fpksnr2/bin:/sbin:/bin:/usr/sbin:/usr/bin",
  "LD_LIBRARY_PATH": "/gpfs/space/home/user/.conda/envs/user_env/lib:/gpfs/space/software/jupyterhub/python/jupyter/lib"
  "LIBRARY_PAHT": "/gpfs/space/home/user/.conda/envs/user_env/lib:/gpfs/space/software/jupyterhub/python/jupyter/lib"
 }
}

Problems and requests¶

This service is currently in beta and under development. If you notice problems or would like any more extensions or features, then please write to support@hpc.ut.ee .

Please provide the following details:

• Error details from the jupyter.xxxxxx.log> file.
• Slurm job id.

FAQ¶

See also¶

• https://jupyter.org .
• Jupyter Documentation .
• Demos and tutorial: https://jupyter.org/try .