Docker structure for deploying to multiple environments

dcom.sh

#!/bin/sh

################################################################################
#     This is a super simple, flexible wrapper for docker-compose command.     #
################################################################################

ENV=${1:-prod}
BASE=$(pwd)
shift

COMPOSE=$(docker compose >/dev/null 2>&1 && echo 'docker compose' || echo 'docker-compose')
CONFIG=$([ -f $BASE/docker-compose.${ENV}.yml ] && echo $BASE/docker-compose.${ENV}.yml || echo $BASE/docker-compose.yml)
CMD="$COMPOSE -f $CONFIG $@"

################################################################################

echo "\nUsing services from ${CONFIG/$BASE\//}" >&1 && exec $CMD

docker-compose.debug.yml

version: "3"

services:
  db:
    extends:
      file: services.yml
      service: mysql

  cache:
    extends:
      file: services.yml
      service: redis

  web:
    extends:
      file: services.yml
      service: laravel
    ports:
      - 80:80
    links:
      - cache
      - db
    environment:
      - APP_ENV=debug
      - DATABASE_HOST=db
      - REDIS_HOST=cache
      - REDIS_PORT=6379
      - BLACKFIRE_LOG_LEVEL=2
      - BLACKFIRE_MEMORY_LIMIT=256
      - BLACKFIRE_CLIENT_ID=${BLACKFIRE_CLIENT_ID:-}
      - BLACKFIRE_CLIENT_TOKEN=${BLACKFIRE_CLIENT_TOKEN:-}
    volumes:
      - ./debug:/var/www/html:ro

  agent:
    extends:
      file: services.yml
      service: blackfire
    links:
      - web
    environment:
      - BLACKFIRE_CLIENT_ID=${BLACKFIRE_CLIENT_ID:-}
      - BLACKFIRE_CLIENT_TOKEN=${BLACKFIRE_CLIENT_TOKEN:-}
      - BLACKFIRE_SERVER_ID=${BLACKFIRE_SERVER_ID:-}
      - BLACKFIRE_SERVER_TOKEN=${BLACKFIRE_SERVER_TOKEN:-}

docker-compose.local.yml

version: "3"

services:
  db:
    extends:
      file: services.yml
      service: mysql

  cache:
    extends:
      file: services.yml
      service: redis

  web-1:
    extends:
      file: services.yml
      service: laravel
    ports:
      - 8001:80
    links:
      - cache
      - db
    environment:
      - APP_ENV=local
      - DATABASE_HOST=db
      - REDIS_HOST=cache
      - REDIS_PORT=6379
    volumes:
      - ./dev-1:/var/www/html:ro

  web-2:
    extends:
      file: services.yml
      service: php-apache
    ports:
      - 8002:80
    volumes:
      - ./dev-2:/var/www/html:ro

  admin:
    extends:
      file: services.yml
      service: phpmyadmin
    ports:
      - 8080:80
    links:
      - db

docker-compose.prod.yml

version: "3"

services:
  gateway:
    extends:
      file: services.yml
      service: traefik
    ports:
      - 80:80
      - 443:443

  cache:
    extends:
      file: services.yml
      service: redis

  backend:
    extends:
      file: services.yml
      service: laravel
    links:
      - cache
    environment:
      - REDIS_HOST=cache
      - REDIS_PORT=6379
    volumes:
      - ./webroot:/var/www/html:ro
    labels:
      - "traefik.http.services.laravel.loadbalancer.server.port=80"
      - "traefik.http.routers.laravel-route.rule=Host(`mydomain.com`,`www.mydomain.com`)"
      - "traefik.http.routers.laravel-route.service=laravel"

Experimental Docker structure.md

Beginning

There are many approaches to implementing a reuse of a common preset for Docker services for multiple environments, such as production and local environments.

This makes it possible to ensure the highest consistency for different environments with the same code-base. Implementing reuse of docker compose options also makes it easier to manage them.

I found on github a project called serversideup/spin. They took an approach using a feature called Docker overrides, to change some properties of common services for different environments.

After reading through their documentation, I realized that there are a few real-life cases where this project can not implement (or is difficult to archive).

That's why I decided to talk to the project owner and create this gists, for testing and discussion purposes.

---

Support my activities

If you would like supporting my projects, buy me a coffee 😉.

I really appreciate your love and supports.

---

Use case

Let's assume that we need to build the following environments with the same code-base using the popular framework Laravel. The general requirement is that they should be as consistent as possible, and easy to operate.

General requirements

• Use a redis instance as a cache service
• Limit size of log files for all containers
• Automatically restart the container when it fails
• Simply launch docker for different environments
• Friendly with docker compose CLI

Production

• One web backend can scale to many instances
• Use traefik as gateway, support HTTP and HTTPS
• No need for other services like mysql, phpmyadmin attached

Local

• Two instances for web backend to test 2 different branches of code
• The first web backend runs on port 8001
• Second web backend running on port 8002
• Use additional mysql, phpmyadmin services for local development
• phpmyadmin running on port 8080
• No need to use traefik as gateway as each backend runs on its own port

Debug

• A web backend instance, running on port 80
• A blackfire service for profiling web backend
• Use additional mysql, phpmyadmin services for local development
• phpmyadmin running on port 8080
• No need to use traefik as gateway as each backend runs on its own port

This setup should follow the official Docker guidlines as much as possible.

Implementation

Concept

I was able to easily implement the above requirements in a simple way following the following concept structure.

./
┝━ services.yml
┝━ docker-compose.prod.yml
┝━ docker-compose.local.yml
┝━ docker-compose.debug.yml
┝━ webroot/
┝━ dev-1/
┝━ dev-2/
┝━ debug/
└─ dcom.sh

• services.yml contains definitions for all services
• docker-compose.prod.yml contains services for production
• docker-compose.local.yml contains services for local
• docker-compose.debug.yml contains services for debug
• webroot/ contains source code for production
• dev-1/ and dev-2/ contain contain source code for local
• debug/ contains source code for debug
• dcom.sh is a wrapper script for docker compose

Running services

As a result, you can easily run a single command to start all the necessary services for each predefined environment with docker compose.

For example, to run services for the production environment:

docker-compose -f docker-compose.prod.yml up -d

Very simple, right?

You don't need any other shell script, nor do you need to remember complicated command syntax to run it.

The dcom.sh

You might be wondering: "What is the shell script dcom.sh for?", am I right?

This is a wrapper to shorten your command line. dcom is short for "docker-compose".

We use it like this:

./dcom.sh env_name [arguments]

For example, to run services for the production environment:

./dcom.sh prod up -d

The parameter for dcom.sh is fully compatible with docker-compose interface.

Wait a second!

./dcom.sh prod up -d

Q: I am running this on production environment. As mentioned on the Use case, I want to run the service backend in 2 different instances on production, how can I do that?

A: It's very simple, you just need to run the command below, traefik will act as a load balancer for those instances intelligently.

./dcom.sh prod up -d --scale backend=2

Then check all running services with this:

./dcom.sh prod ps

Ref: Docker Compose CLI reference

---

If you like this project, please support my works 😉.

From Vietnam 🇻🇳 with love.

services.yml

version: "3"

################################################################################
################################################################################

services:

  ##############################################################################
  ##############################################################################
  common:
    platform: ${DOCKER_PLATFORM:-linux/amd64}
    restart:  ${DOCKER_RESTART:-unless-stopped}
    logging:
      driver: ${DOCKER_LOG_DRIVER:-json-file}
      options:
        max-size: ${DOCKER_LOG_SIZE:-32m}
        max-file: ${DOCKER_LOG_COUNT:-10}

  common-debug:
    extends: common
    cap_add:
      - SYS_PTRACE


  ##############################################################################
  ##############################################################################
  traefik:
    image: traefik:2.6
    extends: common
    volumes:
      - ./certs:/etc/certs
      - /var/run/docker.sock:/var/run/docker.sock:ro,cached
    command:
      - "--api.dashboard"
      - "--api.insecure"
      - "--log.level=INFO"
      - "--global.sendAnonymousUsage=false"

      ### HTTP
      - "--entrypoints.http.address=:80"
      - "--entryPoints.http.forwardedHeaders.insecure"

      ### HTTPS
      - "--entrypoints.https.address=:443"
      - "--entrypoints.https.http.tls=true"
      - "--entrypoints.https.http.tls.certResolver=my-acme"

      ### Providers
      - "--providers.docker"
      - "--providers.docker.exposedByDefault=false"
      - "--providers.docker.watch"

      ### Let's encrypt
      - "--certificatesresolvers.my-acme.acme.httpchallenge=true"
      - "--certificatesresolvers.my-acme.acme.httpchallenge.entrypoint=http"
      - "[email protected]"
      - "--certificatesresolvers.my-acme.acme.storage=/etc/certs/acme.json"


  ##############################################################################
  ##############################################################################
  mysql:
    image: mysql:5.7
    extends: common
    environment:
      MYSQL_ROOT_PASSWORD: p@ssw0rd


  ##############################################################################
  ##############################################################################
  memcached:
    image: memcached:1.6-alpine
    extends: common


  ##############################################################################
  ##############################################################################
  redis:
    image: redis:6-alpine
    extends: common
    environment:
      ALLOW_EMPTY_PASSWORD: "yes"


  ##############################################################################
  ##############################################################################
  php-apache:
    image: php:8.1-apache
    extends: common-debug
    healthcheck:
      test: "curl -f 'http://localhost' || exit 1"
      start_period: 30s
      interval: 1m
      timeout: 5s
    labels:
      - "traefik.enable=true"


  ##############################################################################
  ##############################################################################
  laravel:
    extends: php-apache
    environment:
      - APP_ENV=production


  ##############################################################################
  ##############################################################################
  blackfire:
    image: blackfire/blackfire:2
    extends: common-debug
    environment:
      - BLACKFIRE_DISABLE_LEGACY_PORT=8307
      - BLACKFIRE_LOG_LEVEL=4
      - BLACKFIRE_MEMORY_LIMIT=256


  ##############################################################################
  ##############################################################################
  phpmyadmin:
    image: phpmyadmin:apache
    extends: common-debug
    environment:
      - PMA_ARBITRARY=1
      - PMA_HOST=db
      - PMA_USER=root
      - PMA_PASSWORD=p@ssw0rd