Virtual server for hosting multiple sites

__Virtual server for hosting multiple sites.md

Virtual server for hosting multiple sites

This configuration can be used for hosting multiple sites on a single server, using docker and docker compose for each site. Each site needs a unique port, and will be served on a unique domain. For SSL, use Cloudflare as a proxy.

Use EC2 to create a new instance, or a similar setup with your own host.. The image should be Debian. Make sure to maintain a constant IP address. The script can then be copied directly in to the shell when you are ssh’ed in.

Once a site has been started with docker compose, exposing a port internally, use the command add-site domain-name port (e.g. add-site my.website.com 8080) to register this with the Caddy web server. This means that any requests to my.website.com will be forwarded to your container running on the given port (e.g. 8080). Note that the docker-compose file should include restart: always to ensure that it runs when the instance is restarted.

Use the command list-sites to list all registered sites. If you attempt to start two Docker projects with the same IP address, docker-compose will probably complain. However, add-site can be used to bind arbitrarily many domains to the same internal port.

In order to change the port for a given domain, simply call add-site again, and it will be overwritten.

To delete a site, delete the relevant file in /etc/caddy/sites/. E.g. to delete my.website.com, run sudo rm /etc/caddy/sites/my.website.com.caddyfile.

Deploying

Run the attached deploy script, with the following environment variables set. These variables can also be read from a .ec2-deploy file in the current directory (if not already set in the environment).

• DOMAIN: The domain name that will be used to access the site
• DEPLOY_HOST: The address used for connecting to the server push the changes
• DEPLOY_USER: The user of the server. Optional
• DEPLOY_KEYPAIR: The PEM file (ssh -i flag) for connecting to the server. Optional
• DEPLOY_DIR: The path on the remote server to deploy under
• SERVER_PORT: The port exposed by docker compose. This must be unique among VMs on the server.
• PORT_VARIABLE: The name of the environment variable used for passing $SERVER_PORT to the docker compose command. E.g. If PORT_VARIABLE=APP_PORT, docker-compose.yaml could contain the entry services.app.ports of "${APP_PORT:-5000}:5000".
• SOURCE_DIR: The local directory to deploy.

_setup-debian.sh

# Add user to passwordless sudo

sudo groupadd wheel
sudo usermod -aG wheel user
echo '%wheel ALL=(ALL) NOPASSWD: ALL' | sudo tee /etc/sudoers.d/wheel-nopasswd
sudo chmod 0440 /etc/sudoers.d/wheel-nopasswd

# Install Caddy, and start its process
sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
chmod o+r /usr/share/keyrings/caddy-stable-archive-keyring.gpg
chmod o+r /etc/apt/sources.list.d/caddy-stable.list
sudo apt update
sudo apt install caddy

# Install Docker
sudo apt update
sudo apt install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
sudo tee /etc/apt/sources.list.d/docker.sources <<EOF
Types: deb
URIs: https://download.docker.com/linux/debian
Suites: $(. /etc/os-release && echo "$VERSION_CODENAME")
Components: stable
Signed-By: /etc/apt/keyrings/docker.asc
EOF
sudo apt update
sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo usermod -aG docker user

# Add vim and rsync
sudo apt install vim rsync

# Add owner for /var/www
sudo mkdir -p /var/www
sudo chown -R root:www-data /var/www
sudo chmod -R 2755 /var/www

# Set up Caddy config
sudo mkdir -p /etc/caddy/sites/
sudo tee /etc/caddy/Caddyfile > /dev/null << 'EOF'
{
        log {
                level WARN
        }
}

import sites/*.caddyfile
EOF
caddy validate --config /etc/caddy/Caddyfile && sudo systemctl reload caddy

# Define the scripts
sudo tee /bin/add-site > /dev/null << 'EOS'
#!/bin/bash
set -e
if [[ $# -ne 2 ]]; then
    echo "Usage: add-site hostname port" >&2
    echo "e.g.   add-site test.com 8080" >&2
    exit 1
fi
DEPLOY_HOSTNAME="$1"
DEPLOY_PORT="$2"
if [[ "$DEPLOY_HOSTNAME" == */* || "$DEPLOY_HOSTNAME" == "." || "$DEPLOY_HOSTNAME" == ".." ]]; then
    echo "Invalid hostname: $DEPLOY_HOSTNAME" >&2
    exit 1
fi
if ! [[ "$DEPLOY_PORT" =~ ^[0-9]+$ ]] || (( DEPLOY_PORT < 1 || DEPLOY_PORT > 65535 )); then
    echo "Invalid port: $DEPLOY_PORT" >&2
    exit 1
fi
CONFIG_FILENAME="/etc/caddy/sites/$DEPLOY_HOSTNAME.caddyfile"
NEW_CONF=$(cat <<EOF
# add-site $DEPLOY_HOSTNAME $DEPLOY_PORT
$DEPLOY_HOSTNAME {
  reverse_proxy 127.0.0.1:$DEPLOY_PORT
}
EOF
)
if ! sudo -n true 2>/dev/null; then
    echo "Permission denied, cannot update Caddy configuration" >&2
    exit 1
fi
if sudo test -f "$CONFIG_FILENAME"; then
    OLD_CONF=$(sudo cat "$CONFIG_FILENAME")
    HAD_CONF=1
else
    OLD_CONF=""
    HAD_CONF=0
fi
if [[ $HAD_CONF -eq 1 && "$NEW_CONF" == "$OLD_CONF" ]]; then
    echo "No change needed for Caddy configuration" >&2
    exit 0
fi
sudo tee "$CONFIG_FILENAME" > /dev/null <<< "$NEW_CONF"
if caddy validate --config /etc/caddy/Caddyfile; then
    sudo systemctl reload caddy
    echo "Updated Caddy configuration" >&2
elif [[ $HAD_CONF -eq 1 ]]; then
    sudo tee "$CONFIG_FILENAME" > /dev/null <<< "$OLD_CONF"
    echo "Caddy configuration invalid, restored previous $CONFIG_FILENAME" >&2
    exit 1
else
    sudo rm -f "$CONFIG_FILENAME"
    echo "Caddy configuration invalid, removed $CONFIG_FILENAME" >&2
    exit 1
fi
EOS
sudo chmod ugo+rx /bin/add-site

sudo tee /bin/remove-site > /dev/null << 'EOS'
#!/bin/bash
set -e
if [[ $# -ne 1 ]]; then
    echo "Usage: remove-site hostname" >&2
    echo "e.g.   remove-site test.com" >&2
    exit 1
fi
DEPLOY_HOSTNAME="$1"
if [[ "$DEPLOY_HOSTNAME" == */* || "$DEPLOY_HOSTNAME" == "." || "$DEPLOY_HOSTNAME" == ".." ]]; then
    echo "Invalid hostname: $DEPLOY_HOSTNAME" >&2
    exit 1
fi
CONFIG_FILENAME="/etc/caddy/sites/$DEPLOY_HOSTNAME.caddyfile"
if ! sudo -n true 2>/dev/null; then
    echo "Permission denied, cannot update Caddy configuration" >&2
    exit 1
fi
if ! sudo test -f "$CONFIG_FILENAME"; then
    echo "No change needed for Caddy configuration" >&2
    exit 0
fi
OLD_CONF=$(sudo cat "$CONFIG_FILENAME")
sudo rm -f "$CONFIG_FILENAME"
if caddy validate --config /etc/caddy/Caddyfile; then
    sudo systemctl reload caddy
    echo "Updated Caddy configuration" >&2
else
    sudo tee "$CONFIG_FILENAME" > /dev/null <<< "$OLD_CONF"
    echo "Caddy configuration invalid after removal, restored $CONFIG_FILENAME" >&2
    exit 1
fi
EOS
sudo chmod ugo+rx /bin/remove-site

sudo tee /bin/list-sites > /dev/null << 'EOS'
#!/bin/bash
set -e
shopt -s nullglob
CONFIGS=(/etc/caddy/sites/*.caddyfile)
SITES=""
if [[ ${#CONFIGS[@]} -gt 0 ]]; then
    SITES=$(head -n1 -q "${CONFIGS[@]}" | grep '^# add-site' || true)
fi
if [[ -z "$SITES" ]]; then
    echo "No sites configured" >&2
    exit 0
fi
printf '%s\n' "$SITES"
EOS
sudo chmod ugo+rx /bin/list-sites

connect

#!/usr/bin/env bash

set -e

if [ -f .ec2-deploy ]; then
  set -a
  # Use .ec2-deploy file (but only if the values aren’t already set)
  . /dev/stdin <<< "$(sed -n 's/^\([^#][^=]*\)=\(.*\)$/\1=${\1:-\2}/p' .ec2-deploy)"
  set +a
fi

if [ -z "$DEPLOY_HOST" ]; then
  echo "DEPLOY_HOST is not set, please add this to .env or provide it on the command line" >&2
  exit 1
fi

if [ -z "$DEPLOY_USER" ]; then
  DEPLOY_TARGET="$DEPLOY_HOST"
else
  DEPLOY_TARGET="$DEPLOY_USER@$DEPLOY_HOST"
fi

if [ -z "$DEPLOY_KEYPAIR" ]; then
  DEPLOY_FLAG=""
else
  DEPLOY_FLAG="$(printf -- "-i%q" "$DEPLOY_KEYPAIR")"
fi

if [ "$0" = "$BASH_SOURCE" ]; then
  ssh "$DEPLOY_FLAG" "$DEPLOY_TARGET" "$@"
fi

deploy

#!/usr/bin/env bash

set -e

. "$(cd "$(dirname "$(readlink -f "$0")")" && pwd)"/connect

case $SERVER_PORT in
    '') echo "SERVER_PORT is missing" >&2; exit 1 ;;
    *[^0-9]*) echo "PORT_VARIABLE is set, but is not a valid bash identifier" >&2; exit 1 ;;
esac

case $PORT_VARIABLE in
    '') PORT_ASSIGNMENT="" ;;
    *[^A-Za-z0-9_]*) echo "PORT_VARIABLE is set, but is not a valid bash identifier" >&2; exit 1 ;;
    *) PORT_ASSIGNMENT="$PORT_VARIABLE=$SERVER_PORT" ;;
esac

# On the server, start the docker process, clear old docker images and then set up port forwarding
# (assuming the server is set up as in this gist:
# https://gist.github.com/c-harding/59686990f42f4dddae01dffa56618ea4, if not then this stage is
# simply skipped).
(
  RAW_DOMAIN="${DOMAIN/#*\/\//}"
  DEPLOY_COMMAND="$(printf "
    %s docker compose -f%q/docker-compose.yml up --build -d &&
    docker system prune -f &&
    (command -v add-site > /dev/null && add-site %q %d; true)
  " "$PORT_ASSIGNMENT" "${DEPLOY_DIR:?missing, this must be provided to specify where on the server to deploy to}" "$RAW_DOMAIN" "$SERVER_PORT")"

  rsync --exclude=".ec2-deploy" -re"ssh $DEPLOY_FLAG" "${SOURCE_DIR:-.}" "$DEPLOY_TARGET":"$DEPLOY_DIR" &&
  ssh "$DEPLOY_FLAG" "$DEPLOY_TARGET" "$DEPLOY_COMMAND"
)

Todo: add notes about creating new users. Fairly simple:

sudo adduser bivouacker
sudo usermod -aG docker bivouacker
mkdir .ssh
chmod 700 .ssh
touch .ssh/authorized_keys
chmod 600 .ssh/authorized_keys
vim .ssh/authorized_keys

Add new keypair

For CentOS: sudo setsebool -P httpd_can_network_connect 1

Updated to use Debian and Caddy, saving the need for certificate management