Using Docker on TrueNAS SCALE (no Kubernetes)

boot.sh

#!/usr/bin/env bash

#
# Enable docker and docker-compose on TrueNAS SCALE (no Kubernetes)
#
# This script is a hack! Use it at your own risk!!
# Using this script to enable Docker is NOT SUPPORTED by ix-systems!
# You CANNOT use SCALE Apps while using this script!
#
# 1  Create a dedicated Docker zvol on one of your zpools: zfs create -V 100G data/_docker
# 2  Create an ext4 filesystem on your new zvol: mkfs.ext4 /dev/zvol/data/_docker
# 3  Create a directory where you can mount the ext4 filesystem: mkdir /mnt/data/_docker
# 4  Save this script somewhere else on your zpool, not in the Docker dataset
# 5  Edit line 24 and 25 of the script, set paths to the Docker directory and zvol you created
# 7  If this is NOT the first time you run this script (updating to new location/zvol):
#     - Check the contents of /etc/fstab, and remove the old # docker-zvol-ext4 rule if it's there
# 6  You can now start Docker by running the script from the SCALE console
#
# For these changes to persist after SCALE reboots and upgrades, run the script at start-up
#   Schedule this script to run via System Settings -> Advanced -> Init/Shutdown Scripts
#   Click Add -> Type: Script and choose this script -> When: choose to run as Pre Init

## set the path to your docker directory and specify the zvol
docker_directory='/mnt/data/_docker'
docker_zvol='/dev/zvol/data/_docker'

## this part will add a new fstab rule, if the fstab file doesn't contain our special docker-zvol-ext4 comment
## if this is not the first time you run this script, our special comment is already there
## so if you ever change the docker_directory and docker_zvol paths in the future, also remove the old
## fstab rule and comment from /etc/fstab
grep -q 'docker-zvol-ext4' /etc/fstab || {
    printf "# docker-zvol-ext4\n${docker_zvol} ${docker_directory} ext4 rw,relatime,stripe=4 0 0\n" >> /etc/fstab
    mount "${docker_directory}" && echo 'Mounted docker directory'
}

## HEREDOC: docker/daemon.json
read -r -d '' JSON << END_JSON
{
  "storage-driver": "overlay2",
  "data-root": "${docker_directory}",
  "exec-opts": [
    "native.cgroupdriver=cgroupfs"
  ]
}
END_JSON

## path to docker daemon file
docker_daemon='/etc/docker/daemon.json'

if [ ${EUID} -ne 0 ]; then
  echo "Please run this script as root or using sudo"
elif [ "$(systemctl is-enabled k3s)" == "enabled" ]; then
  echo "You can not use this script while k3s is enabled"
elif [ "$(systemctl is-active k3s)" == "active" ]; then
  echo "You can not use this script while k3s is active"
elif ! which docker &> /dev/null; then
  echo "Docker executable not found"
elif ! chmod +x /usr/bin/docker-compose &> /dev/null; then
  echo "Failed to make docker-compose executable"
elif ! install -d -m 755 -- /etc/docker &> /dev/null; then
  echo "Failed to install directory: /etc/docker"
elif [ ! -d "${docker_directory}" ] ; then
 echo "Directory not found: ${docker_directory}"
else
  echo "Checking file: ${docker_daemon}"
  if test "${JSON}" != "$(cat ${docker_daemon} 2> /dev/null)"; then
    echo "Updating file: ${docker_daemon}"
    jq -n "${JSON}" > ${docker_daemon}
    if [ "$(systemctl is-active docker)" == "active" ]; then
      echo "Restarting Docker"
      systemctl restart docker
    elif [ "$(systemctl is-enabled docker)" != "enabled" ]; then
      echo "Enable and starting Docker"
      systemctl enable --now docker
    fi
  fi
fi

New zfs version has overlayfs support natively baked in btw :)

Yeah I just noticed! Super awesome! 😃Looking forward to using it on a stable SCALE release.

Sources indicate it also fixes about 75% of the CPU load issues on SCALE Apps as well :)

Wonderful :) would be worth revisiting the Docker Compose TrueCharts app by that time. It really is comforting to know that one exists for when this Docker hack stops working.

@Jip-Hop - I'm on RC 1 now - I only needed to add the "storage-driver": "overlay2", to the HEREDOC section of the script and everything seems to be working. I did not have to create, format, or even mount an extra zvol.

EDIT: Maybe worth noting, I started with a fresh docker-dataset after I changed the storage-driver.

Great! Thanks for testing and reporting. Didn't expect overlay2 support to be included so quickly. Then I know what I must do when SCALE 22.12.0 is released next month. Glad I can drop the ext4 workaround.😄

From a docker level this should be the only required change.
Though this also means one of the primary reasons not to go with SCALE Apps (performance issues) are also solved for a large portion as well.

I'm back to using the script by tprelog now that I've updated to Bluefin. No need for the ext4 zvol :)

⚠️ In October 2023, TrueNAS SCALE Cobia will be released. At that time, ix-systems is making the switch to containerd and Docker will be removed.

The alternative way of running docker, based on systemd-nspawn is available for testing! 😄

Please vote for this issue: Support for systemd-nspawn Linux Containers (jails for SCALE).

So we have to make our own scripts to "hack" what ix-systems has designed.

If you (or anyone) has useful scripts to enable Docker/Portainer on TrueNAS SCALE then please post them here. 😃

https://forum.level1techs.com/t/truenas-scale-native-docker-vm-access-to-host-guide/190882