kathoef

File transfers over tailnet for low performance devices

Setting: A Raspberry Pi 1B+ (Rev 1.2) is operated as restic backup server over a tailnet connection. The device is configured as nearly read-only to reduce wear on the SD card, i.e. /tmp, /var/tmp and /var/tmp are mounted as tmpfs and swap is deactivated. File transfers of a few megabytes have always succeeded.

Problem: For file transfers with several gigabytes, the memory usage of the tailscaled process (on the server side) slowly increases; eventually the tailnet connection freezes and the file transfer terminates with IO errors on the restic client side. The server side Raspberry remains inaccessible even after the restic backup task has stopped.

A quick solution leverages Go environment variables on the server side,

kathoef

GDB debugging

srun --partition=vector --mem=10000 --time=01:00:00 --gres=ve:4 --ntasks=1 --nodes=1 --job-name=test --pty bash
module purge
module load vector-env/default
module load ncc/4.0.0 vetop/1.0 nlc/2.3.0
export VE_FORT_UFMTENDIAN=big
export VE_TRACEBACK=FULL
export VE_OMP_NUM_THREADS=4

kathoef

JupyterHub for Docker container course

This JupyterHub setup provides access to a web-based interactive docker terminal environment for use in e.g. Docker course settings. Issues around getting Docker to work on every participant's end device during the class are mitigated, and "bring your own working (!) Docker environment" is not anymore a requirement for taking part in the course. The setup ensures that teachers can focus solely on helping out with the course contents.

This setup is based on the "The Littlest JupyterHub" and leverages "Docker rootless" as provided by Docker Inc. to prevent issues around (1) privilege escalation (for sudo and docker-group Docker usage approaches) and (2) Docker daemon isolation. With this setup, every participant gets their own "safe to use" Docker container working environment, which also (mostly) resembles the Docker environment that would be presented on a local machine.

The setup was tested successfully with Ubuntu 22.04 and Ubuntu 24.04 releases.

kathoef

FROM node:19-bullseye as vscodebundler

RUN npm install -g typescript esbuild vsce
RUN git clone https://github.com/andre-dietrich/liascript-preview-vscode-web
RUN cd liascript-preview-vscode-web && npm install && vsce package && ls

FROM codercom/code-server:4.8.3-bullseye

COPY --from=vscodebundler /liascript-preview-vscode-web/liascript-preview-web-0.2.1.vsix /
RUN code-server --install-extension /liascript-preview-web-0.2.1.vsix

kathoef

Using liascript PDF exporter,

$ docker run -it -v $PWD:/things ubuntu:22.04 bash
$ apt update
$ apt install wget xz-utils python3
$ wget --quiet https://nodejs.org/download/release/v14.20.1/node-v14.20.1-linux-x64.tar.xz
$ mkdir -p /usr/local/lib/
$ tar xf node-v14.20.1-linux-x64.tar.xz -C /usr/local/lib/
$ export PATH=/usr/local/lib/node-v14.20.1-linux-x64/bin:$PATH

kathoef

Setup a rootless Docker build compatible Gitlab runner

Install Docker engine (tested with v20.10.17) and Gitlab runner (tested with v15.2.1) as described here and here.

Identify the UID of the gitlab-runner user.

$ id gitlab-runner
uid=999(gitlab-runner) gid=998(gitlab-runner) groups=998(gitlab-runner)

kathoef

Remote machine,

$ ssh user@host
$ mkdir -p $HOME/vscodeserver
$ cd $HOME/vscodeserver
$ bash install.sh
$ bash start.sh
c773621a-f86f-4f44-99a8-ef149521765a
[2022-04-14T10:41:07.120Z] info code-server 4.2.0 693b1fac04524bb0e0cfbb93afc85702263329bb

kathoef

$ jupyterhub-singleuser --debug
[D 2022-03-16 10:13:11.371 SingleUserNotebookApp application:174] Searching ['/home/jupyter-khoeflich', '/home/jupyter-khoeflich/.jupyter', '/home/jupyter-khoeflich/.local/etc/jupyter', '/opt/tljh/user/etc/jupyter', '/usr/local/etc/jupyter', '/etc/jupyter'] for config files
[D 2022-03-16 10:13:11.371 SingleUserNotebookApp application:731] Looking for jupyter_config in /etc/jupyter
[D 2022-03-16 10:13:11.371 SingleUserNotebookApp application:731] Looking for jupyter_config in /usr/local/etc/jupyter
[D 2022-03-16 10:13:11.371 SingleUserNotebookApp application:731] Looking for jupyter_config in /opt/tljh/user/etc/jupyter
[D 2022-03-16 10:13:11.371 SingleUserNotebookApp application:731] Looking for jupyter_config in /home/jupyter-khoeflich/.local/etc/jupyter
[D 2022-03-16 10:13:11.371 SingleUserNotebookApp application:731] Looking for jupyter_config in /home/jupyter-khoeflich/.jupyter
[D 2022-03-16 10:13:11.371 SingleUserNotebookApp application:731] Looking for jupyter_config i

kathoef

#!/bin/bash

# Uninstall kernel.

# rm -rf ${HOME}/.local/share/jupyter/kernels/my-pykernel/
# rm -rf ${HOME}/my-pykernel/

# Specify a few targets.

if [[ -f environment.yaml ]]; then

kathoef