Singularity Testing

notes.md

Singularity

Singularity basically pulls docker images and dumps them to disk as directories.

Testing

In testing, I discovered some things:

• It seems that when the Dockerfile doesn't have a CMD entry at the end, it just dumps the filesystem contents. If you have a CMD, it dumps a binary.

golang

I created a simple go image called simple-go. The image builds a single binary, simple-go, which prints hello world. I pushed it to my docker registry at docker.io/lcrown/simple-go, then pulled it on the HPC. It correctly outputted hello world.

Reproduce script

# create the project directory
mkdir simple-go
cd simple-go

# initialize the project
go mod init github.com/lcrownover/simple-go
mkdir -p cmd/simple-go/main.go
cat << EOF > cmd/simple-go/main.go
package main

import (
        "fmt"
)

func main() {
        fmt.Println("hello world")
}
EOF

# write the dockerfile per the instructions on the go container:
# https://hub.docker.com/_/golang
cat << EOF > Dockerfile
FROM golang:1.19

WORKDIR /usr/src/app

COPY go.mod ./
RUN go mod download && go mod verify

COPY . .

RUN go build -v -o /usr/local/bin/simple-go ./...

CMD ["simple-go"]
EOF

# build the image
#docker build -t lcrown/simple-go .

# build the image for amd64 since i'm on arm64
docker buildx build --platform linux/amd64 --load -t lcrown/simple-go:v1 .

# then push it to my registry
docker push lcrown/simple-go:v1

# switch to HPC
# build my container from my registry
singularity pull docker://lcrown/simple-go:v1

# now I can just run the .sif file
./simple-go_v1.sif # outputs "hello world"

python

Now, let's create a container with a bunch of python dependencies:

• requests
• pandas
• otter-grader

This is a super common use case, as we want to be able to freeze the versions of each of these dependencies.

Reproduce script

mkdir python-deps-test
cd python-deps-test

# write the main script that will by run by the container
cat << EOF > main.py
#!/usr/bin/env python3

import requests
import pandas as pd

print("success")
EOF

# dependencies
virtualenv venv
source venv/bin/activate
pip install requests pandas otter-grader
pip freeze > requirements.txt

# write the dockerfile
cat << EOF > Dockerfile
FROM python:bullseye
COPY requirements.txt /app/requirements.txt
RUN pip install -r /app/requirements.txt
COPY main.py /app/main.py
CMD ["python3", "/app/main.py"]
EOF

# build the image
#docker build -t lcrown/python-deps-test .

# build the image for amd64 since i'm on arm64
docker buildx build --platform linux/amd64 --load -t lcrown/python-deps-test .

# then push it to my registry
docker push lcrown/python-deps-test

# switch to HPC
# build my container from my registry
singularity build python-deps-test/ docker://lcrown/python-deps-test

# run the .sif
./python-deps-test_v1.sif # outputs "success"

MPI

TBD