Apptainer container setup with tmux sessions on GSI lustre nodes

Apptainer container setup for GSI servers

This instruction explains how to setup a working environment for GSI servers (lustre.hpc.gsi.de nodes). The setup is involved with multiple programs, such as TMux, zsh, apptainer and ssh. A similar setup can should also be achievable using alternatives, like bash and screen.

Important

The server address lustre.hpc.gsi.de contains multiple nodes. Starting a ssh session with this address could be led to a different node, therefore, losing all tmux sessions from the last login. Users should start a ssh session with a specific node address. In the following example, dslustre01.hpc.gsi.de is used as an example.

Step 1: SSH configuration

In the local machine (either your personal computer or a terminal), you should have already created the ssh public and private key. By default, your ssh configuration folder should be ~/.ssh. Open the file config in the folder (create one if config file doesn't exist)

# use any text editor to open the file
nvim ~/.ssh/config

Inside the config file, add following configurations:

Host gsigate
    HostName lxlogin.gsi.de
    User your_user_name

Host lustre01
    HostName dslustre01.hpc.gsi.de
    User your_user_name
    ProxyJump gsigate
    ForwardAgent no
    ForwardX11 yes
    SetEnv LC_TMUX_CONTAINER=1

Host lustre01_root
    HostName dslustre01.hpc.gsi.de
    User your_user_name
    ProxyJump gsigate
    ForwardAgent no
    ForwardX11 yes
    SetEnv LC_TMUX_CONTAINER=1 LC_TMUX_ROOT=1

The variables, LC_TMUX_CONTAINER and LC_TMUX_ROOT, are sent through the ssh to tell whether tmux sessions on the servers will be opened with containers (LC_TMUX_CONTAINER) and whether they have the fake_root option (LC_TMUX_ROOT). The fake_root option allows you to have a "fake sudo" privilege such that you could use dnf install to install necessary packages.

Then ssh into lustre01 with:

ssh lustre01 # Without fake_root options

# or
# ssh lustre01_root # With fake_root options

Step 2: Apptainer image creation

Note

If you already have a container image available (e.g. /lustre/r3b/ywang/containers/fedora40.sif), this step could be skipped.

Apptainer is a container software used in GSI servers and HPC clusters. It enables users to use any operating systems inside another (in our case, Debian 11). The software in the container is no longer limited by the server machine and you could pre-install/install any of your own programs as well. In this instruction, Fedora 40 is used since it provides rolling updates on most commonly used tool-sets, such as GCC 14, Clang 18 and CMake 3.28, etc. A apptainer container can be created by the so-called image, which functions like a blueprint for the container. Images contains all pre-installed software, like operating system, third party tool-sets/libraries or your own programs. To make your life simple, it's strongly recommended to pull an Apptainer image from a Dockerhub repository.

To create an Apptainer image, inside the server machine, run the following command:

cd any_location
apptainer build --fix-perms fedora40.sif docker://yanzhaowang/r3bdev:fedora40

The file path any_location must be shared with different nodes. Therefore, it should be located somewhere in /lustre.
fedora40.sif is the image name that apptainer will use to create the container for each TMUX window.
docker://yanzhaowang/r3bdev:fedora40 is a Dockerhub repository address that apptainer pulls the image from.

Step 3: Overlay folder creation

Note

If you don't need fake_root option, this step can be skipped.

fake_root option requires a folder where apptainer can put installed packages. Simply create an empty folder (somewhere in /lustre or /tmp) with:

mkdir /tmp/my_image.overlay

Step 4: zsh (bash) configuration

In the server machine, open (or create if not existing) the ~/.zshrc file (~/.bashrc if you still use bash) and add following configurations on the top:

if [[ $LC_TMUX_CONTAINER == "1" && -z $APPTAINER_NAME && -n $TMUX ]]; then
    /usr/bin/apptainer exec --bind /lustre,/u --overlay /tmp/my_image.overlay  image_path.sif zsh -d # or bash
    exit
fi

Replace the overlay and image paths with yours.

If some packages need to be installed with root previledge, add the following commands to the top of .bash_profile:

if [[ $LC_TMUX_ROOT == "1" ]]; then
    /usr/bin/apptainer exec -f --bind /lustre,/u --overlay /u/yanwang/apptainer/fedora40.overlay  /lustre/r3b/ywang/containers/fedora40.sif zsh -d
    exit
fi

Again, replace the overlay and image file paths with yours. With this, ssh lustre01_root will directly leads you to the container environment with root previledge.

Step 5: Open the TMUX

To make sure your .zsh script is correctly loaded, first logout and login the session. Then open a new TMUX session with:

tmux new -s test

you could check whether you are in the container with a different operating system by:

cat /etc/os-release