A complete checklist on how to set up an Ubuntu server and configure CI/CD using GitHub Actions & Docker.

CI & CD + Ubuntu Server Prep.md

Table of Contents

• Prerequisites
  • Updating Ubuntu & Installing Docker
  • Choosing the Best Reverse Proxy
• Setting Up Caddy
• Setting Up Nginx Proxy Manager
• Configuring Your Own Docker Registry
• Project Configuration
  • GitHub Setup
    • Configuring a Self-Hosted Runner
    • Managing Application Secrets
  • Setting Up GitHub Actions
• CI/CD Cycle Ready
• Stopping a Self-Hosted Runner
• Helpful Resources

Prerequisites

Updating Ubuntu & Installing Docker

1. Update Ubuntu packages - sudo apt-get update && sudo apt-get upgrade && sudo apt-get autoclean && sudo apt-get autoremove && sudo apt update && sudo apt upgrade && sudo apt autoclean && sudo apt autoremove.
2. Install Docker - curl -fsSL https://get.docker.com -o get-docker.sh && sh get-docker.sh.
3. Install Docker for non-root users - dockerd-rootless-setuptool.sh install. This step is optional if you plane use Docker from account with root permission.
4. Run Docker post-install commands if you faced with any issues - post-install docs.

Choosing best reverse proxy

Name	Manage Level	Benefits
Caddy	Require basic Docker knowledge.	Suitable for most everyday routine tasks, it is easy to use and understand with a basic understanding of writing Docker Compose files. All configuration is done individually for each service in docker-compose file. Most possible cases are described in the documentation.
Nginx Proxy Manager	-	Very easy to set up and use. All customization is done in a nice UI that contains all the necessary information.

Caddy configuration

1. Init Docker Swarm - docker swarm init.
2. Create caddy network - docker network create caddy.
3. Create services/caddy folder.
4. Inside caddy folder create docker-compose.yaml file with next content:

services:
  caddy:
    restart: always
    image: lucaslorentz/caddy-docker-proxy:ci-alpine
    ports:
      - 80:80
      - 443:443
    environment:
      CADDY_INGRESS_NETWORKS: caddy
    networks:
      - caddy
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "data:/data"

networks:
  caddy:
    external: true

volumes:
  data: {}

5. Run docker compose up -d to pull & run Caddy.

Nginx Proxy Manager

1. Init Docker Swarm - docker swarm init.
2. Create nginx network - docker network create nginx.
3. Create services/nginx folder.
4. Inside nginx folder create docker-compose.yaml file with next content:

services:
  nginx:
    restart: always
    image: jc21/nginx-proxy-manager:2.11.2
    ports:
      - 80:80
      - 443:443
      - 81:81
    volumes:
      - ./data:/data
      - ./letsencrypt:/etc/letsencrypt

networks:
  default:
    external: true
    name: nginx

5. Run docker compose up -d to pull & run Nginx Proxy Manager.
6. Visit http://<server-ip-address>:81 to open admin interface.

Own Docker Registry

I'm prefer to use self-hosted Docker registry to fully manage my application images. This is best way if you don't wanna use any another services which can store your Docker images.

1. Create services/registry folder.
2. Inside registry create docker-compose.yaml file with next content:

services:
  registry:
    restart: always
    image: registry:2
    environment:
      REGISTRY_AUTH: htpasswd
      REGISTRY_AUTH_HTPASSWD_PATH: /auth/htpasswd
      REGISTRY_AUTH_HTPASSWD_REALM: Registry Realm
      REGISTRY_STORAGE_DELETE_ENABLED: true
    networks:
      - caddy # or nginx
    # labels must be omited if you choose Nginx Proxy Manager as reverse proxy.
    labels:
      caddy: registry.example.com
      caddy.reverse_proxy: "{{upstreams 5000}}" # 5000 is internal app port.
    volumes:
      - ./data:/var/lib/registry
      - ./auth:/auth

volumes:
  data:

networks:
  caddy: # or nginx
    external: true

3. Create auth folder near docker-compose.yaml file.
4. Run docker run --entrypoint htpasswd httpd:2 -Bbn YOUR_USER YOUR_PASS > auth/htpasswd.
5. Run registry - docker compose up -d to pull & run Docker Registry.

To check if Docker Registry is working, you can simply run docker login registry.example.com command. If all is well, you should be prompted to enter your username and password, then display a message indicating successful authorization.

Project configuration

GitHub

Self-hosted runner

1. Open GitHub repository and visit Settings > Actions > Runners.
2. Click "New self-hosted runner".
3. Choose "Runner image" as Linux and "Architecture" as x64.
4. Create new runners/YOUR_APP folder near services folder.
5. Navigate to new folder and run all commands from GitHub guide.
6. Skip first & last command (./run.sh) from docs.
7. Run ./svc.sh install && ./svc.sh start.

Use RUNNER_ALLOW_RUNASROOT="1" environment if you run command from user with root permissions.

I'm prefer use self-hosted runners to have ability for run some workflow commands, like create/edit docker-compose.yaml or .env files.

Application secrets

1. Navigate to Settings > Secrets and variables > Actions.
2. Click "New repository secret" and add all necessary secrets:

Name	Description	Example
REGISTRY_USERNAME	Registry username.	YOUR_USER
REGISTRY_PASSWORD	Registry password.	YOUR_PASS
REGISTRY_URL	Registry URL without http(s)://	registry.example.com
MOUNT_DIR	Full path to the services folder	/root/services
YOUR_APP_ENV	Application env.	HELLO

This is not the best way to manage application secrets, but this is most fast way to run CI/CD. You can take a look into Infisical docs if you wanna use powerful environments manager.

GitHub Actions

1. Create .github/workflows folder inside your project.
2. Add runner.yaml file with next content:

name: CI & CD

on:
  push:
    branches:
      - "production"

jobs:
  setup:
    runs-on: self-hosted
    name: Setup Workflow
    outputs:
      repo_name: ${{ steps.repo-name.outputs.repo_name }}
    steps:
      - name: Set repository name in lowercase
        id: repo-name
        run: echo "repo_name=$(echo ${GITHUB_REPOSITORY##*/} | tr '[:upper:]' '[:lower:]')" >> $GITHUB_OUTPUT

  ci-job:
    runs-on: self-hosted
    name: CI Job
    needs: setup
    env:
      REPO_NAME: ${{ needs.setup.outputs.repo_name }}
    steps:
      - name: Checkout
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Login to Docker Registry
        uses: docker/login-action@v2
        with:
          username: ${{ secrets.REGISTRY_USERNAME }}
          password: ${{ secrets.REGISTRY_PASSWORD }}
          registry: ${{ secrets.REGISTRY_URL }}

      - name: Build & Push app to Docker Registry
        uses: docker/build-push-action@v4
        with:
          push: true
          tags: ${{ secrets.REGISTRY_URL }}/${{ secrets.REGISTRY_USERNAME }}/${{ env.REPO_NAME }}:latest
          # You should use the `file` variable if your Docker file is not in the project root.

  cd-job:
    runs-on: self-hosted
    name: CD Job
    needs: [setup, ci-job]
    env:
      REPO_NAME: ${{ needs.setup.outputs.repo_name }}
    steps:
      - name: Checkout
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Login to Docker Registry
        uses: docker/login-action@v2
        with:
          username: ${{ secrets.REGISTRY_USERNAME }}
          password: ${{ secrets.REGISTRY_PASSWORD }}
          registry: ${{ secrets.REGISTRY_URL }}

      - name: Create/Open project directory
        shell: bash
        env:
          PROJECT_DIR: ${{ secrets.MOUNT_DIR }}/${{ env.REPO_NAME }}
        run: mkdir -p $PROJECT_DIR && cd $PROJECT_DIR

      - name: Creating a new docker-compose.yaml
        shell: bash
        run: |
          cd ${{ secrets.MOUNT_DIR }}/${{ env.REPO_NAME }}

          cp ${{ github.workspace }}/docker-compose.prod.yaml docker-compose.yaml

          sed -i 's/\$REGISTRY_URL/${{ secrets.REGISTRY_URL }}/g' docker-compose.yaml
          sed -i 's/\$REGISTRY_USERNAME/${{ secrets.REGISTRY_USERNAME }}/g' docker-compose.yaml
          sed -i 's/\$APP_NAME/${{ env.REPO_NAME }}/g' docker-compose.yaml

      - name: Creating a new production .env
        shell: bash
        run: |
          cd ${{ secrets.MOUNT_DIR }}/${{ env.REPO_NAME }}

          cat << 'EOF' > .env
          YOUR_APP_ENV=${{ secrets.YOUR_APP_ENV }}
          EOF

      - name: Run services with latest version
        shell: bash
        run: |
          cd ${{ secrets.MOUNT_DIR }}/${{ env.REPO_NAME }}

          docker compose pull
          docker compose up -d --remove-orphans

3. Create docker-compose.prod.yaml file in root of project.

services:
  app-service:
    restart: always
    image: $REGISTRY_URL/$REGISTRY_USERNAME/$APP_NAME:latest
    networks:
      - caddy # or nginx
    # labels must be omited if you choose Nginx Proxy Manager as reverse proxy.
    labels:
      caddy: service-name.example.com
      caddy.reverse_proxy: "{{upstreams PORT}}" # PORT is internal app port.

networks:
  caddy: # or nginx
    external: true

CI/CD cycle ready

Done. Now, when you push new code to your production branch the CI & CD workflow actions will be triggered automatically and after build will pull and run latest version of your application on your server.

Stop self-hosted runner

If you want to remove self-hosted runner, you need to run these commands:

1. Navigate to folder with runner - cd runners/YOUR_APP.
2. Stop & uninstall runner service - ./svc.sh stop && ./svc.sh uninstall.
3. Remove runner from GitHub - ./config.sh remove --token RUNNER_TOKEN.

To retrieve RUNNER_TOKEN token you need to navigate to the GitHub self-hosted runners page, choose your runner and then click on "Delete" button.

Helpful Resources

• Docker & Docker Compose.
• Caddy as a reverse proxy for Docker.
• Nginx Proxy Manager.