FreeBSD podman tech demo

buildah-install.md

This will pull in source code for podman, buildah and related modules, build everything and install to /usr/local. This all happens in a directory named 'build' which can be deleted to clean up or to force a clean build.

mkdir -p build
fetch https://gist.github.com/dfr/ac4dc043ee3780b690c5887a61f53494/raw/1e55da486792ffda61c9d6070d5d834888be9590/buildah-install.sh
chmod +x buildah-install.sh
(cd build && ../buildah-install.sh)

Make a container and run things inside it:

c=$(sudo buildah from docker.io/kwiat/freebsd:13.0-RELEASE)
sudo buildah run $c freebsd-version
sudo buildah run $c ifconfig
sudo buildah rm -a

Download and run images in podman:

sudo podman run --rm docker.io/dougrabson/hello

The containers will use the default 'podman' network which is defined in /usr/local/etc/cni/net.d/87-podman-bridge.conflist. This relies on NAT to allow the container traffic out to the internet and I use pf with the following simple pf.conf:

nat on egress inet from <cni-nat> to any -> (egress)
nat on egress inet6 from <cni-nat> to !ff00::/8 -> (egress)
rdr-anchor "cni-rdr/*"
table <cni-nat>

Note: I'm using the OpenBSD convention to identify the host's main interface by putting it into the 'egress' group using ifconfig, e.g.:

sudo ifconfig vtnet0 group egress

There is a lot of room for improvement in this area - NAT works fairly well for ipv4 but can get confused with ipv6 if the egress interface has non-routable addresses assigned to it. Port mapping is very limited and does not work for connections from localhost. Perhaps someone with better pf skills can help figure out how to get this working (probably needs to NAT from localhost back to the container network).

Stats for running podman containers can be accessed using the 'podman stats' command. This relies on the RACCT accounting framework which is present in GENERIC kernels but must be enabled by adding kern.racct.enable="1" to /boot/loader.conf.

buildah-install.sh

#! /bin/sh

set -x

echo Installing necessary tools and modules
sudo pkg update
sudo pkg install -y bazel go gmake gsed gpgme git bash pkgconf gcc python3
sudo kldload nullfs
sudo kldload fdescfs
sudo kldload pf
sudo sysrc kld_list="nullfs fdescfs pf"

echo Build and install ocijail:
git clone https://github.com/dfr/ocijail.git
(cd ocijail && bazel run //:install -- -s /usr/local/bin)

echo Build and install CNI plugins
git clone -b freebsd https://github.com/dfr/plugins.git
sudo mkdir -p /usr/local/libexec/cni
(cd plugins && ./build_freebsd.sh && sudo cp bin/* /usr/local/libexec/cni)

echo Build and install buildah:
git clone -b v1.28.0 https://github.com/containers/buildah.git
(cd buildah && gmake && sudo install bin/buildah /usr/local/bin)

echo Build and install conmon:
git clone https://github.com/containers/conmon.git
(cd conmon && gmake CC=gcc && sudo install bin/conmon /usr/local/sbin)

echo Build and install podman:
git clone https://github.com/containers/podman.git
(cd podman && gmake && sudo gmake install)

echo Initialise storage:
sudo zfs create -o mountpoint=/var/db/containers zroot/containers

echo Install config files
git clone https://github.com/dfr/containers-etc.git
sudo mkdir -p /usr/local/etc/containers
sudo mkdir -p /usr/local/etc/cni/net.d
sudo cp -r containers-etc/containers/* /usr/local/etc/containers
sudo cp -r containers-etc/cni/net.d/* /usr/local/etc/cni/net.d

I'll update the gist soon - the common changes got merged upstream and I deleted the branch without thinking. You should be able to clone directly from the main branch,

…

On Wed, 3 Aug 2022 at 03:14, outbackdingo ***@***.***> wrote: ***@***.**** commented on this gist. ------------------------------ This part of the install process appears broken, there is no freebsd branch in conmon echo Build and install conmon: git clone -b freebsd https://github.com/dfr/conmon.git (cd conmon && gmake CC=gcc && sudo install bin/conmon /usr/local/sbin) — Reply to this email directly, view it on GitHub <https://gist.github.com/ac4dc043ee3780b690c5887a61f53494#gistcomment-4254062>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABOTJLDDPSCX23NLJRAMTTVXHIZ5ANCNFSM55NFDY5A> . You are receiving this because you authored the thread.Message ID: ***@***.***>

you mean from https://github.com/containers/conmon ?? I tried that and it fails to build, im a marshmallow, directions please :)

root@cbsd:~/build/conmon # gmake
gmake: hack/seccomp-notify.sh: No such file or directory
cc -std=c99 -Os -Wall -Wextra -Werror -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include -I/usr/local/include -DVERSION="2.1.3" -DGIT_COMMIT=""4ca293730749b80893216228dd2b07bd5397e23b"" -o src/conmon.o -c src/conmon.c
src/conmon.c:43:22: error: variable 'dev_null_r_cleanup' set but not used [-Werror,-Wunused-but-set-variable]
cleanup_close int dev_null_r_cleanup = -1;
^
src/conmon.c:44:22: error: variable 'dev_null_w_cleanup' set but not used [-Werror,-Wunused-but-set-variable]
cleanup_close int dev_null_w_cleanup = -1;
^
src/conmon.c:45:22: error: variable 'dummyfd' set but not used [-Werror,-Wunused-but-set-variable]
cleanup_close int dummyfd = -1;
^
3 errors generated.
gmake: *** [Makefile:71: src/conmon.o] Error 1

nevermind it works gmake CC=gcc

I fixed the 'hack/seccomp-notify.sh: No such file or directory' problem and submitted a PR upstream. I also re-added the freebsd branch (with this PR) so the build script should work again.

There still seem to be some issues running containers in buildah and podman though when I test this on a fresh VM. I'll work on debugging it

Running 'kldload fdescfs' helps get buildah going - I updated the build script and freshened up the podman branch so that it contains the support for fdescfs.

Updated to build buildah, conmon and podman from upstream sources rather than my work-in-progress branches.

Hi, just a note, running images via podman fails now with the error: Error: OCI runtime error: ocijail: malformed ociVersion 1.1.0-rc.2 . This seems to be partially because ocijail doesn't parse the version correctly, and partially because it doesn't support versions > 1.0.x.

Hi, just a note, running images via podman fails now with the error: Error: OCI runtime error: ocijail: malformed ociVersion 1.1.0-rc.2 . This seems to be partially because ocijail doesn't parse the version correctly, and partially because it doesn't support versions > 1.0.x.

I'll take some time to update ocijail to work with the latest oci version. In the meantime, it might be best to install from ports or pkg - I'll be updating them too over the next week or so.