CI/CD Workflow Guide: Multi-Architecture Docker Builds

workflow.md

CI/CD Workflow Guide: Multi-Architecture Docker Builds

This guide explains how to implement and maintain CI/CD workflows for building and deploying multi-architecture Docker images with optimized caching strategies.

Overview

Multi-architecture Docker workflow supporting AMD64 + ARM64 builds with:

• GitHub Actions caching: Simple, optimized caching for GHA environment
• Docker Build Cloud for ARM64 only: Cloud builders used exclusively for ARM64 builds for speed
• AMD64 native builds: Standard GitHub runners for AMD64 builds
• Matrix-based job organization: Single job handling both architectures
• Automatic versioning: Semantic versioning extracted from CHANGELOG.md
• GitHub releases: Automated release creation with changelogs

Prerequisites

Required Accounts & Services

• Docker Hub personal account: Create at hub.docker.com
• Docker Build Cloud: Activate 7-day free trial for ARM64 cloud builders (Production only)
• GitHub repository: With proper permissions for releases

Required Repository Secrets

Configure these in GitHub Settings → Secrets and variables → Actions:

Secret	Required For	Description
DOCKER_USERNAME	Both	Docker Hub username
DOCKER_PASSWORD	Both	Docker Hub password or access token
DOCKER_BUILD_CLOUD_ENDPOINT	Production	Docker Build Cloud builder endpoint (e.g., your-username/builder-name)
DISCORD_WEBHOOK	Both (Optional)	Discord webhook URL for notifications

Required Files

• CHANGELOG.md - Following Keep a Changelog format

Architecture Overview

Multi-Architecture Workflow

graph LR
    prepare[Prepare<br/>Extract version &<br/>release notes] --> build[Build<br/>Matrix job:<br/>AMD64 + ARM64]
    build --> create[Create Manifest<br/>Combine arch-specific<br/>images]
    create --> release[Release<br/>GitHub release with<br/>changelogs]
    release --> notify[Notify<br/>Discord status<br/>updates]

Loading

Jobs:

• prepare: Extracts version and release notes from CHANGELOG.md
• build: Matrix job building AMD64 (native) + ARM64 (cloud) simultaneously
• create-manifest: Combines architecture-specific images into multi-arch manifest
• release: Creates GitHub release with changelogs
• notify: Discord notifications for build status

Caching Strategy

GHA Cache vs Registry Cache

There are two main approaches for caching Docker layers in CI/CD:

GitHub Actions Cache (Recommended for Simplicity)

Your implementation uses GitHub Actions cache (type=gha), which provides:

• Simple setup: No additional Docker Hub repository management needed
• Fast cache access: Optimized for GitHub Actions environment
• Automatic cleanup: GHA handles cache eviction automatically
• Free tier friendly: No additional Docker Hub storage costs
• Easy debugging: Visible in GitHub Actions cache section

Cache Configuration (GHA approach):

cache-from: type=gha,scope=gha-cache-{arch}
cache-to: type=gha,scope=gha-cache-{arch},mode=max

Registry-Based Cache (Alternative for Advanced Use Cases)

Registry-based caching stores cache directly in Docker Hub providing:

• Persistent storage: Survives GitHub Actions cache limits and evictions
• Cross-environment compatibility: Accessible from any build environment
• Cloud builder support: Works seamlessly with Docker Build Cloud for ARM64
• No size limitations: Unlike GitHub Actions cache (10GB limit)
• Advanced management: More control over retention policies

Cache Configuration (Registry approach):

cache-from: type=registry,ref=${{ env.DOCKER_USERNAME }}/{repo}:buildcache-{arch}
cache-to: type=registry,ref=${{ env.DOCKER_USERNAME }}/{repo}:buildcache-{arch},mode=max

Multi-Architecture Workflow

Configuration

Your current workflow uses a matrix strategy to build both architectures in a single job. Create .github/workflows/multi-arch.yaml:

name: Docker Multi-Arch Build - Production

on:
  push:
    branches:
      - multi-arch
  workflow_dispatch:

permissions:
  contents: write

env:
  DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
  DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
  DOCKER_BUILD_CLOUD_ENDPOINT: ${{ secrets.DOCKER_BUILD_CLOUD_ENDPOINT }}
  DISCORD_WEBHOOK: ${{ secrets.DISCORD_WEBHOOK }}

jobs:
  prepare:
    name: Prepare Build Info
    runs-on: ubuntu-latest
    outputs:
      version: ${{ steps.version.outputs.version }}
      tag: ${{ steps.version.outputs.tag }}
      release_notes: ${{ steps.release-notes.outputs.notes }}
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Extract version from changelog
        id: version
        run: |
          # Validate CHANGELOG.md exists and is readable
          if [ ! -f "CHANGELOG.md" ]; then
            echo "ERROR: CHANGELOG.md file not found"
            exit 1
          fi

          if [ ! -r "CHANGELOG.md" ]; then
            echo "ERROR: CHANGELOG.md file is not readable"
            exit 1
          fi

          # Extract version from CHANGELOG.md
          CHANGELOG_LINE=$(grep '^## \[' CHANGELOG.md | head -1)
          if [ -z "$CHANGELOG_LINE" ]; then
            echo "ERROR: No version entry found in CHANGELOG.md"
            echo "CHANGELOG.md first few lines:"
            head -5 CHANGELOG.md
            exit 1
          fi

          VERSION=$(echo "$CHANGELOG_LINE" | cut -d'[' -f2 | cut -d']' -f1 | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')

          # Validate version format (basic semantic versioning check)
          if ! echo "$VERSION" | grep -E '^[0-9]+\.[0-9]+\.[0-9]+' > /dev/null; then
            echo "WARNING: Extracted version '$VERSION' doesn't match semantic versioning format (x.y.z)"
            echo "CHANGELOG line: $CHANGELOG_LINE"
            echo "Attempting to continue anyway..."
          fi

          echo "Successfully extracted version from CHANGELOG.md: $VERSION"
          echo "CHANGELOG line: $CHANGELOG_LINE"

          BRANCH=${GITHUB_REF##*/}
          TAG="$BRANCH-$VERSION"
          echo "version=$VERSION" >> $GITHUB_OUTPUT
          echo "tag=$TAG" >> $GITHUB_OUTPUT
          echo "Building tag: $TAG"

      - name: Extract release notes from CHANGELOG
        id: release-notes
        run: |
          NOTES=$(awk "/^## \\[${VERSION//./\\.}\\]/{flag=1; next} /^## \\[/{if(flag) exit} flag" CHANGELOG.md | sed '1d')
          echo "notes<<EOF" >> $GITHUB_OUTPUT
          echo "$NOTES" >> $GITHUB_OUTPUT
          echo "EOF" >> $GITHUB_OUTPUT

  build:
    name: Build ${{ matrix.arch }}
    runs-on: ubuntu-latest
    needs: prepare
    strategy:
      matrix:
        arch: [amd64, arm64]
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Login to Docker Hub
        uses: docker/login-action@v3
        with:
          username: ${{ env.DOCKER_USERNAME }}
          password: ${{ env.DOCKER_PASSWORD }}

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
        with:
          driver: ${{ matrix.arch == 'arm64' && 'cloud' || 'docker' }}
          endpoint: ${{ matrix.arch == 'arm64' && env.DOCKER_BUILD_CLOUD_ENDPOINT || '' }}

      - name: Build and push ${{ matrix.arch }} image
        uses: docker/build-push-action@v6
        with:
          context: .
          push: true
          platforms: linux/${{ matrix.arch }}
          tags: ${{ env.DOCKER_USERNAME }}/project-name:${{ needs.prepare.outputs.tag }}-${{ matrix.arch }}
          # Only use GHA cache for ARM64 (cloud driver) as cache read/write is faster than GitHub's Ubuntu runners for AMD64
          cache-from: ${{ matrix.arch == 'arm64' && 'type=gha,scope=gha-cache-arm64' || '' }}
          cache-to: ${{ matrix.arch == 'arm64' && 'type=gha,scope=gha-cache-arm64,mode=max' || '' }}

  create-manifest:
    name: Create Multi-Arch Manifest
    needs: [prepare, build]
    runs-on: ubuntu-latest
    steps:
      - name: Login to Docker Hub
        uses: docker/login-action@v3
        with:
          username: ${{ env.DOCKER_USERNAME }}
          password: ${{ env.DOCKER_PASSWORD }}

      - name: Create and push multi-arch manifest
        run: |
          docker buildx imagetools create -t ${{ env.DOCKER_USERNAME }}/project-name:${{ needs.prepare.outputs.tag }} \
            ${{ env.DOCKER_USERNAME }}/project-name:${{ needs.prepare.outputs.tag }}-amd64 \
            ${{ env.DOCKER_USERNAME }}/project-name:${{ needs.prepare.outputs.tag }}-arm64

      - name: Install regctl for temporary tag cleanup
        run: |
          wget https://github.com/regclient/regclient/releases/latest/download/regctl-linux-amd64
          chmod +x regctl-linux-amd64
          sudo mv regctl-linux-amd64 /usr/local/bin/regctl

      - name: Delete temporary architecture tags
        run: |
          regctl tag delete ${{ env.DOCKER_USERNAME }}/project-name:${{ needs.prepare.outputs.tag }}-amd64 || true
          regctl tag delete ${{ env.DOCKER_USERNAME }}/project-name:${{ needs.prepare.outputs.tag }}-arm64 || true

  release:
    name: Create GitHub Release
    needs: [prepare, build, create-manifest]
    if: needs.prepare.result == 'success' && needs.build.result == 'success'
    runs-on: ubuntu-latest
    steps:
      - name: Create GitHub Release
        uses: softprops/action-gh-release@v1
        with:
          tag_name: v${{ needs.prepare.outputs.version }}
          name: v${{ needs.prepare.outputs.version }}
          body: |
            ## What's Changed
            ${{ needs.prepare.outputs.release_notes }}
            
            **Docker Images:**
            - `docker pull ${{ env.DOCKER_USERNAME }}/project-name:${{ needs.prepare.outputs.tag }}`
          draft: false
          prerelease: false
          generate_release_notes: false

  notify:
    name: Send Notifications
    needs: [prepare, build, create-manifest]
    if: always()
    runs-on: ubuntu-latest
    steps:
      - name: Notify Success
        if: needs.prepare.result == 'success' && needs.build.result == 'success'
        uses: rjstone/discord-webhook-notify@v1
        with:
          username: Multi Arch Workflow
          severity: info
          details: "✅ Multi-arch build succeeded! Tag: ${{ needs.prepare.outputs.tag }}"
          webhookUrl: ${{ env.DISCORD_WEBHOOK }}

      - name: Notify Failure
        if: needs.prepare.result == 'failure' || needs.build.result == 'failure'
        uses: rjstone/discord-webhook-notify@v1
        with:
          username: Multi Arch Workflow
          severity: error
          details: "❌ Build failed for ${{ needs.prepare.outputs.tag }}"
          webhookUrl: ${{ env.DISCORD_WEBHOOK }}

      - name: Notify Cancelled
        if: needs.prepare.result == 'cancelled' || needs.build.result == 'cancelled'
        uses: rjstone/discord-webhook-notify@v1
        with:
          username: Multi Arch Workflow
          severity: warn
          details: "⚠️ Build cancelled for ${{ needs.prepare.outputs.tag }}"
          webhookUrl: ${{ env.DISCORD_WEBHOOK }}

Key Features

• Matrix-based builds: Single job handling both AMD64 and ARM64 architectures simultaneously
• Selective cloud usage: Docker Build Cloud used exclusively for ARM64 builds for optimal performance
• GitHub Actions caching: Simple caching optimized for GHA environment
• Automatic versioning: Semantic versioning from CHANGELOG.md
• Manifest creation: Combines architecture-specific images into unified multi-arch manifest
• Tag cleanup: Removes temporary architecture-specific tags after manifest creation

CHANGELOG.md Configuration

Format Requirements

Follow Keep a Changelog format:

# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [2.14.07] - 2025-01-20

### Added
- Multi-architecture Docker builds with ARM64 support
- Registry-based caching for improved build performance
- Automated GitHub releases with semantic versioning

### Changed
- Updated CI/CD workflow to use Docker Build Cloud exclusively for ARM64 builds
- Migrated from GitHub Actions cache to registry cache

### Fixed
- Build cache invalidation issues on ARM64 builders
- Concurrent workflow execution conflicts

## [2.14.06] - 2025-01-15

### Added
- Previous version changes...

Version Update Process

1. Add new version header at the top of CHANGELOG.md

   ## [2.14.08] - 2025-01-21

2. Document changes under appropriate categories:

   • Added, Changed, Deprecated, Removed, Fixed, Security

3. Commit and push to trigger workflow

   git add CHANGELOG.md
   git commit -m "chore: bump version to 2.14.08"
   git push origin main  # or staging

4. Workflow automatically:

   • Extracts version from first ## [x.y.z] header
   • Builds images with versioned tags
   • Creates GitHub release (production only)

Semantic Versioning Rules

• MAJOR (x.0.0): Breaking changes, incompatible API changes
• MINOR (x.y.0): New features, backwards-compatible
• PATCH (x.y.z): Bug fixes, backwards-compatible

Docker Build Cloud Setup

1. Create Docker Hub Account

Visit hub.docker.com and sign up.

2. Activate Docker Build Cloud

1. Go to Docker Hub Dashboard
2. Navigate to Build Cloud section
3. Click Start Free Trial (7 days)
4. Create a new builder with a name (e.g., nimbus)

3. Configure Builder Endpoint

There are two ways to configure your builder endpoint:

Option A: Environment Variable (Your Current Approach)

Configure a repository secret DOCKER_BUILD_CLOUD_ENDPOINT with your full builder endpoint.

# Example: your-username/your-builder-name
# If your username is johndoe and builder name is nimbus: johndoe/nimbus

Option B: Inline Configuration

Hardcode the builder name in the workflow:

endpoint: "${{ env.DOCKER_USERNAME }}/nimbus"

Workflow Permissions

Permissions Configuration

permissions:
  contents: write

What it does:

• contents: write: Allows the workflow to create GitHub releases and push tags
• Required for the release job that creates GitHub releases
• Minimal permission scope following principle of least privilege

Why This Permission is Needed:

The production workflow needs to:

1. Create GitHub releases with version tags
2. Push release notes and changelogs
3. Without contents: write, the release job would fail